As we all start to learn how to live in the world of AI, I can’t help but notice that not all things about it are sunshine and rainbows, even if all the salespeople involved would like you to believe otherwise.
I believe it’s extremely useful as a technology, there’s no debate about that, but I also believe that there’s lots of people that are stretching the usefulness beyond what the technology is capable of.
While useful, the ability to output lots of code, or more simply put, lots of text, comes with its own problems. Somebody has to read all that output and it’s becoming more and more clear to me that most people rarely do, because of the way they are incentivised.
People have been speaking for a while about such problems in the open-source world. You can read Death by a thousand slops for one such example. That problem occurs likely because open-source “contributions” are accessible to many more people, so it’s putting pressure on maintainers to distinguish the useful from the useless.
What I would like to talk about is the dynamic that is occurring in the corporate world. There, it is not the case that there are more contributors. Quite the opposite, people have been laid off with the justification that AI has increased productivity to such an extent that companies can fire some people. There is no proof of that, so people need to take them at their word, but the point is that there are NOT more people contributing code or other useful artifacts to projects.
Writing code is the perfect target
Anthropic, the company behind Claude Code, seems to have found the perfect use case, which so far seems to be the most efficient way for LLM providers to monetize this new technology. And that is coding.
I think there is a multitude of reasons as to why that is and I will go through some of them:
- Software engineering is genuinely hard. It’s hard to write code and it’s even harder to write good code. If you look beyond very simple apps, CRUD type of things that just have an UI, things get complicated really quick. This is why developers might seem “inefficient” to people that have no clue what is happening behind the scene. So business “leaders” (I hate this term so much, but there we go) see developers as some spoiled employees that can’t deliver at the pace they expect when all they have to do is “write code”.
- AI is putting pressure on software companies. Whether justifiably or not (I don’t think anybody can tell at this point), software companies are feeling threatened by the AI companies whose products they are using. The narrative is something along the lines of “If AI can generate or edit images, will people still use Photoshop?” (I think the answer is ‘Yes’, but everyone is entitled to their opinion about the future). You can see this in the way the market has reacted, software companies have dropped 50-60-70% from their all-time highs. So the AI companies look like a threat. But at the same time they present themselves as the solution. The people running the software companies seem to think that if they move significantly faster and deliver more features, they will somehow escape the danger. If you believe AI will do all those things (and again, I’m skeptical), how will delivering more features faster for Photoshop prevent AI from taking over your product. I guess on some level it makes sense, if a bear is coming for you, you still run even if it’s obvious it can outrun you. But anyway, the current dynamic is that software executives want more features delivered faster because AI allows it and they want to outrun the threat.
- Existing code is often messy and undocumented. This is why AI is actually good at this. Writing a new, non-trivial, application with AI is not a walk in the park. It outputs a lot of garbage code and you will have to go in and do lots of cleanup. And there will be lots of back and forward with the agent about the solution. So from this perspective there aren’t lots of productivity gains in my opinion. But most projects aren’t greenfield. They are big, legacy projects, that have been running for years or decades, have messy code, technical debt that has accumulated, undocumented code or even undocumented functionality. As developers, we would like this not to be the case, we would like our past selves or other developers to have written clean code built on a clean architecture, but it happens more often than not. This is where an LLM can shine, it can help a developer (even a new developer to the project) navigate this mess and understand the code.
So these are all good reasons for software companies to push their employees to use LLMs.
It’s not just the code
My latest job experience was with a SaaS company who’s product is basically a pile of tech debt. This makes it a perfect use case for working on it with AI. So adopting Claude Code was pretty useful to developers. We went from having a very hard time understanding the existing messy code, thus spending weeks on a new feature, to Claude being able to explain a lot of it and bringing that time down to days. The rest of the development cycle was pretty much as it used to be before Claude: functional documents written by PMs, technical documents written by developers, the author of a feature or bugfix reading and being familiar with the code, reviews done by other developers.
At this point there was still room for improvement, but you can trust “leadership” to take a good thing too far. There was a decision that Claude would be used for everything in the company. But combined with the push for more and more features with unrealistic deadlines (“because we now have Claude”), the outcome was pretty much this:
- PMs were using Claude to write functional documents that were HUGE and 100% AI generated. Because existing documentation was lacking, there was a decision that the source of truth for existing functionality was the code. That kinda makes sense, but it resulted in functional documents that were a nightmare to read, since they kept referencing code paths and classes to express what they wanted to achieve. They were also functionally bloated. Even if the client really needed some very particular functionality, Claude would make it very generic, trying to cover every other way a client might use the feature. This, of course, would make it harder to implement. So why did the PMs not simplify/reduce the scope? Well, turns out they weren’t reading them. The best proof of that was when talking to the PM about a functional document they had created. At some point in our Slack conversation a developer in the team brought on screen some HTML mock-ups the Claude had created and were attached to the functional document. The PM asked what that was and when they were told they said the dev team to completely ignore those cause he had never opened them.
- Now the dev needs to take the functional document, create a technical document and implement the feature. First, you’d have to read the functional document, and as I said, it was HUGE, it would probably take a full day to read thoroughly and understand. So, naturally, people would take shortcuts. They would skim it, ask Claude to synthesize it and go from there, just having a conversation with Claude. Then, just ask Claude to come up with a technical document. Due to time pressure combined with its size and the amount of detail, there’s no point reading it. It should be based on the conversation and the functional document. And there’s a review process anyway.
- Other devs would need to review the technical document. So being under their own time pressure, they would just ask Claude to do it.
- Implementing the feature is as easy as asking Claude: “Implement this feature based on this functional document and this technical document”. It would output large amounts of code, but no need to read it. Just test the feature and submit the PR for review. That’s what reviews are for, aren’t they?
- If you see a pattern, you can guess what reviewers did when they saw a huge PR waiting for their review. You guessed it, they’d have Claude do it and just paste the output into the PR. Would they read what they submit? Absolutely not. How do I know that? Most of the feedback on code reviews were non-issues. Then the original developer would submit the feedback to his own Claude instance and so on, until the PR was merged.
And when you put all these things together, you see that Uncle’s Bob plea to no ship shit is being heard less and less.
I don’t think this is a singular case, otherwise I wouldn’t have written this article. I actually think this is the prominent way software is being developed right now. Just to be clear, I’m not blaming Claude for this. Claude (or other LLMs) is just a tool, but it should be treated as such. The problem is that everyone is incentivised to trust Claude way more than they should at every step of the process.
You might say it’s a system that works. I beg to differ. As slop accumulates unsupervised, I believe it will reflect on the quality of products and in the cost of maintenance and evolution.
It works great for companies like Anthropic. They get paid for the piles of text Claude both writes and reads at every step. So I doubt there will be any change coming from their side unless pushed.
I’m not at all surprised that executives are trying to push the pedal to the metal. Even with the layoffs, even with the burnouts, they might be able to rollback. They can hire back if needed, there will always be developers willing to pick up the slop for enough money. What has been slightly surprising is developers’ willingness to give up understanding what a piece of software does. There are lots of developers right now that don’t have the slightest clue what they are doing, just prompting away until it works. The delegation of cognition is truly amazing.
Why not vibe accounting?
I was wondering why this way of working has become so dominant in the software industry and is only marginal to non-existent in other cognitive fields.
Why are companies not doing vibe accounting? Just have Claude generate your 10-K. I think it has to do with the need for precision. Being slightly wrong in accounting can have major consequences, so there are people responsible for it. They check and double check.
It should be the same with software. But somewhere in the last 5-6 years the industry has become dominated by a “good enough” attitude. Bugs are being shipped to production and they have zero consequences. People/clients somehow got resigned with the fact that software kinda sucks. It doesn’t have to be this way, yet here we are.
I hope it’s not a trend. Maybe it’s just a temporary hiccup as we get used to this new and very powerful tool we have in our hands.
Leave a Reply