Mark RidleyFollowTowards Data Science--ListenShareThis is the final part of a six part series investigating how generative AI productivity tools aimed at developers, like Github Copilot, ChatGPT and Amazon CodeWhisperer, might impact the structure of entire product engineering teams.In this final part we consider many of the complexities we’ve left aside in the previous articles, the irreplaceable value of human engineers in the age of AI, and leave you with a call to action for leaders to reflect, adapt and innovate.We’ve been on quite the journey through this series, and I think it’s worth a quick recap of what we’ve covered in the previous five articles, so that we can introduce all the stuff that we haven’t covered so far:In Part 1 we saw evidence that Generative AI coding tools like Copilot and ChatGPT have the potential to significantly enhance the productivity of people in technical roles, like engineers and data scientists. These tools change the work that developers are required to do by automating tedious, repetitive, time consuming tasks. In theory, it removes aspects of the developer job that they don’t enjoy and makes their valuable work quicker and easier.In Part 2 we considered that increased automation of developer tasks and higher developer productivity could lead to a major shift in the ratio of engineers to product managers on product teams; where teams have historically had around five engineers for every one product manager, with generative AI tools, this ratio could shift closer to 1:1.In Part 3 we looked at how Generative AI coding assistants didn’t just have the capability to assist with basics like refactoring code or writing tests. We explored how a longer term impact could lead to AI tools refactoring legacy code, including creating tests and documentation, and also simplifying some team structures, like mobile app development, that were often created not by value stream differentiation, but simply because the technical skills were different. We also considered the possible impact on junior engineers and their professional development.In Part 4 we considered how this shift could allow companies to drastically reduce engineering headcount for the same output, rapidly increase output with the same budget, or some combination. We started to model how a choice between investment in growth, budget maintenance or cost reduction might impact engineering and product headcount.In Part 5 we investigated how these benefits may impact different types of companies in different ways. New startups should dive right into using the new tools as fast as possible. Larger companies, who potentially have the greatest financial imperative to transform team structures, will struggle to adapt due to cultural inertia, but should be encouraged today to start experimenting with tools and making their own strategic decisions on how to adopt them. We also saw how ‘bodyshopping’ outsourcing firms were the most likely to be negatively impacted, as clients looked to preserve internal employees and save costs elsewhere.Everything we’ve discussed so far has been centred around the discussion of the hypothesis, “The next-generation product engineering team will probably have far fewer engineers”, but I think it’s fair to say that we’ve stopped quite a long way short of proving or disproving that hypothesis.Now, I’d like to address many of the thought provoking challenges I’ve received in conversations on this topic in the last few months, as well as addressing the fact that I’ve made an awful lot of assumptions in the service of my original hypothesis.I’ve left a lot of potential counterarguments unaddressed, not because they haven’t been bothering me, but because there’s just too much uncertainty and too much work still to do in experimenting with these tools and seeing how they really assist teams.Here’s a big old list of many criticisms and unanswered questions:I admit it, I’m impressed with the quality of the tools, but I’m also not trying to make the case that they need to be as good as great developers. I have deliberately avoided modelling a scenario without technical expertise, because in my experience in using the tools they still need a great deal of supervision and guidance. Supervision to make sure that they weren’t doing anything stupid or dangerous (like writing API keys into source code), and guidance to ensure that they were doing the right thing (using the correct language, or framework or design pattern).I have a sense that these AI tools need to be the McDonalds of software engineering; while going to a great, non-chain restaurant with exceptional staff can be a moment of transcendent experience, it’s not always possible. McDonalds is universally clean, cheap, healthy (I mean, not infested with bacteria) and, above all, consistent. As one of my dearest Italian friends once said when confronted with a big chain delivery pizza, “It doesn’t kill you”. To an extent this is the bar we’re shooting for with AI tools.But, I also don’t think this is the end of the story. The tools we see today are nowhere close to the quality that they will be in a year. Even as I edited the original article into this series, there was news ever single day about more improvements; UC Berkeley introduces a model that writes better API calls than GPT4. Microsoft announces ‘dilated attention’, which allows models to scale to billions of tokens with LongNet. StackOverflow announces OverflowAI with promises of even more snarky responses to silly questions (sorry, I mean, better and more useful search).Even if you’re sceptical about the ability of the tools today, I fear it would be short-sighted to ignore the potential of the capabilities they are likely to develop.[Edit: Even in the week or so since I first drafted this article, Stack Overflow has announced OverflowAI, Github has announced additional tools to prevent intellectual property issues and StabilityAI has announced a coding focused LLM. The market moves at astonishing pace]Yep, these are all possible.But, it’s not our first time working around issues like this, and we already have ways of mitigating them. A lot of the companies I talk to are in some kind of paralysis out of a concern that they will be leaking company secrets which will be used to train the LLM further. Whilst this can be true for free and individual subscriptions, I would strongly recommend that you, dear reader, do your own research on the larger providers to understand exactly what the Risk of that is, and what the vendors are doing to address this very reasonable concern. Have a look at some of the FAQs from the big players to see whether there is a sufficiently good answer for your use case and risk profile: OpenAI, Github Copilot, AWS CodeWhisperer (Google Duet is still in closed beta and data security docs weren’t available).It’s a similar case with security and data protection. Most of you reading today are already dependent on Github, Microsoft or AWS security. You probably either store your code in Github, or your apps or data on Azure, GCP or Amazon. Ask yourself why you are content to accept the risk for a hypercloud vendor, but not for a coding Tool. The risk of using ChatGPT is non-negligible, with a data leak reported in May, news of jailbreaking potential reported this week, and persistent data leakage from internal users reported by cloud security vendor, Netskope. As with any other piece of technology, you can choose to simply ban it from your organisation but people, like nature, always find a way. To properly address security issues you need to educate users and provide secure, easy to use alternatives. If OpenAI isn’t up to the task, maybe one of the other vendors is.Another worry is inadvertent exposure to intellectual property risk; for example where the model has been (‘accidentally’?) trained on material which is closed source, and the tool exposes your organisation to the risk that you will be breaching the law (and carry risks to remedy your breach) simply by using the tool. Here’s the bad news — if you think this is a new risk, you should probably take a closer look at your use of Open Source in your organisation. Many companies fall far short of properly managing and understanding the risks of their ‘Software Bill of Materials’ (SBOM), the list of closed and open source dependencies that their software has. You absolutely should be concerned about the risk that one of these tools might incidentally put you in breach of someone else’s Intellectual Property Rights, but you should extend the controls that you already use for Open Source software and your developers’ big, red cut and paste button.These risks are yours, and if you’re serious about investigating the opportunities that these tools may provide, you should also read the docs and speak to the vendors about the steps that they are taking to protect you from this risk. Ensure that you do your homework. The Common Sense privacy report for Copilot scored it 63%, low enough to earn an ‘amber’ warning (it scored particularly low on ‘School’ and ‘Parental Consent’ which dragged it down).This should always be part of your procurement process anyway. You should always consider any tool you are going to put near production code or data from a risk perspective, and it falls to you to decide on your appetite for risk and how to mitigate it.As a more positive note, I think that Salesforce’s recent announcements are a good indication of the direction that these tools will head. A huge part of Salesforce’s marketing push for AI Day focused around what they call ‘Einstein Trust Layer’, which seems to be a genuinely impressive wrapper for different LLM models that goes a long way to securing access and protecting both customer and company information (seriously, check out this video, even though it’s not about code).We’ve just seen seven major tech companies (Amazon, Anthropic, Google, Inflection, Meta, Microsoft, and OpenAI) sign up to voluntary ‘Responsible AI’ commitments that includes watermarking output. It’s reasonable to assume that the largest players in this market, most already companies we entrust with huge amounts of our data and infrastructure, will release similar trust and security wrappers that answer a lot of outstanding questions about intellectual property, security, privacy and the propensity for LLMs to be a little bit toxic and have a questionable relationship with the truth.See also:Yep, you’re right. There’s still an awful lot of jobs that need to be done that aren’t just writing code.This is great!It means we still need people for a bit longer, and starts to show us where we should be providing training for our engineers. But we should also expect that tooling, automation and better tested, higher quality code will start to impact this list of issues positively, too. With more documentation, less ‘clever’ code, cleaner interfaces, better explainability and better test coverage, we may see fewer of these types of challenges anyway.Yes, true. AI isn’t going to solve all the problems, but if you’re an engineer spending more time in meetings than coding, the problem isn’t with your productivity, it’s with how your organisation is run.Maybe AI will solve that one for you one day, but in the shorter term it’s possible that smaller, leaner and more automated organisations won’t need so many meetings.To be frank, for most organisations the best way to improve developer productivity is to do a better job of prioritising work, saying no to low value outcomes and giving more time back to teams to focus on delivering high quality products. But, if we can’t have that, maybe we can have AI coding assistants instead.Ah, this is a good one.One of the surprising bits of feedback I’ve already had for these articles has been, “that’s really interesting, but in my business we don’t have anywhere near enough product support”. It turns out that I was showing an unconscious bias with my 5:1 ratio of engineers, and many tech teams were already struggling hugely because their ratios were not only much higher than that (let’s say 10:1 or more), but also that the product skill wasn’t valued highly enough within the organisation. Some companies still seem to think that engineers write code and product managers are an expensive luxury.I think eliciting requirements from customers is a really big deal. I also think that running an effective and easy to understand commercial prioritisation process is very hard. It’s going to be a while before we can get stakeholders to do their own prompt design for software.Product managers are critical to great product engineering. Product managers focus engineering resource and help the business identify the most valuable outcomes. I think the last thing we need right now is a reduction in product support, even if we can assist some of their daily tasks with automation.My suspicion is that the product manager and tech lead roles are going to be the most critical ones in these new teams.You’re almost certainly right. But if you have, that’s great! Drop me a comment, start a discussion or best of all write a new article explaining your thoughts on Y or Z. (But not X. We don’t talk about X anymore)The point of this entire series of articles has been with one concern in mind: today, I don’t feel that enough product and technology leaders are actively involving themselves in the discussion of what might happen to their teams if the promise of generative AI coding tools is borne out. We are not simply talking about a small boost to developer productivity. It is possible that we’re talking about a change in the way that products are created.If even some of what we’ve discussed is true, it is likely that teams that build software will be resourced quite differently than today, with a significant impact on product and tech staffing budgets. There are risks for disruption across the entire industry, especially in a reduction in developer roles.Although this may be mitigated by companies looking to invest in ever more outcomes, there is a limit to how many value streams a business can support. The companies that take the time to understand and embrace the coming changes carefully and decisively could gain a major competitive advantage.There is also a philosophical aspect to this argument. I believe that there may be an ineffable and irreplaceable value of human engineers, but I don’t think that I could confidently tell you what it is anymore.We know that developers write code, and we can place a value on them based on how quickly they write the code and how often it works. But this statement simply fails to grasp what the real job of an engineer is. For years it was easy to demonstrate that skilled human engineers were able to do things that algorithms couldn’t, but that’s no longer the case. If the line between human and machine is just the level of complexity of solution that can be created, we’re on dangerously shifting sands. As I wrote earlier:Engineers are not just taught how to code. They are taught how to think.We, as an industry, must appreciate that a wide range of engineering tasks can be easily replicated by this new generation of tools. In doing so, we should also understand that there are still unique skills that humans possess that are not so easily replicated. It’s our job, as leaders, to identify what those skills are and invest in people to optimise for that skill.This article is a call to action. If you’re a product or technology leader, you should take this as a challenge to consider what the impact of generative AI tools will be for your teams, and you should include your teams in answering that question.Given time and space to consider how these tools will help them, the most likely source of innovation is going to be your teams themselves. But, as a leader, you also need to acknowledge that the turkeys are going to be somewhat disinclined to vote for Christmas. People may be scared and reluctant to engage because of the very real threat to their employment and you’re going to need to harness their talent to help you understand if you should make budget cuts. Being the boss is hard, but the alternative is that your competitors are more unblinking and decisive than you are.We should be clear that the most interesting, chaotic, undisciplined, impactful and exciting advances will be in those small startups, and in teams outside of tech. Don’t make the mistake of dismissing what non-technical people in your own organisation, or tiny companies in the startup community are doing.Start by educating yourself, and then others in your own business about what the risks really are. Consider the opportunities by looking both internally at curious early adopters inside your business, and outside at what the tech giants and tiny startups are doing. Be aware that these advances are likely to fundamentally reshape your product engineering then get out there, be curious, ask questions and encourage your teams to do the same.I haven’t yet managed to disprove my own hypothesis — but now I’m asking for your help to design the experiment.P.S. If you’re enjoying these articles on teams, check out my Teamcraft podcast, where my co-host Andrew Maclaren and I talk to guests about what makes teams work.----Towards Data ScienceTechnologist, lean evangelist, chaos monkey and Chief Technology Prevention Officer. Loves good coffee, hanging around on ropes and driving about in carsMark RidleyinTowards Data Science--Bex T.inTowards Data Science--9Dominik PolzerinTowards Data Science--38Mark RidleyinTowards Data Science--Kristen WaltersinAdventures In AI--300Zulie RaneinThe Startup--860Malky McEwaninThe Writing Cooperative--113Dominik PolzerinTowards Data Science--38The PyCoachinArtificial Corner--551Walter RheininThe Writing Cooperative--107HelpStatusWritersBlogCareersPrivacyTermsAboutText to speechTeams