Time to Wall

Why do systems rot?

Here is my observation after 2 years of applying AI to assist in coding: the more convoluted the input codebase is, the less likely an LLM is to produce a satisfying result. It is hardly a revelation. They have been using the expression “Garbage in, garbage out” in the 60s already. The Sixties!

I don’t think people realize the implications of of it though. Programming is a generational process. Output of one iteration is an input to the next one. The progress of those generations - the evolution of the codebase and it’s speed has been naturally limited by the speed at which an engineer or a team of engineers can produce code. With LLMs at our disposal, that limit is gone and it is the best thing that could happen to devs… who know their shit. All the rest might not be so lucky.

Why do systems rot? Common sense would suggest codebases should only get better in time. There is no physical factor to contribute to their decay. No mold, rust, bacteria, winds or rains. The rotting does not take place during runtime but during development. Each line of bad code is a nail in the coffin in which every system eventually gets buried. If the only thing we change is putting LLMs in the hands of both skilled and unskilled developers, we’re basically handing everyone a nailgun (coffin metaphor cont’d).

Each codebase has a lifetime. There is a point at which accumulated complexity is so great, it is not economically feasible to develop further. Similar effect at a single engineer scale is the time it takes for the codebase to go beyond comprehension of a person. This is usually the moment where productivity starts to nosedive. This is where most of the world’s codebases are right now. We compensate by adding workforce, which only exacerbates the problem and kicks the can down the road.

That moment, when we lose ability to understand the codebase - let’s call it the Wall. Most of coding these days is done beyond the Wall. If it was ever quantifiable - Time To Wall would be the most important metric of every single software project. What happens when you throw LLMs into the mix? Iterations are quicker, so naturally Time To Wall goes down. But it is not that simple…

I tried different variants of LLM-assisted coding. Some time I would force myself to go all in, don’t read the code, just check if it works. Some time I would eyeball the LLMs output to see if it makes sense. Some time I would retry a couple of times with different prompts to get to a resulting structure I had in mind. And then the other times, I would just go ahead and fix the structure by hand (copilot still with us), occasionally adding comments for the future LLM iterations to understand the intent of certain structures. Turns out, that last technique, if applied consistently allowed me to keep pushing The Wall away indefinitely. All that at a speed of a small, skilled team.

It sounds easy, but it is not. In order to fix the structure of a generated code, we have to construct a mental model of the target structure first. You also need to be able to recognize defects. The most mind-blowing effect is when your mental model changes and that requires rewriting substantial part of the code. If the initial structure is concise and instructions are clear, LLM can handle the rewrite surprisingly well. Not perfect, but well-enough to clean up by hand. It addresses the missing puzzle to keep the system from rotting - how to safely and repeatably conduct high-impact refactor. Something I though was impossible.

Some time will have to pass for the industry to acknowledge this reality and its implication - that most systems can now be built with 10x less effort and that such productivity can be maintained for years. Existing codebases, especially those fragmented by design (microservices) are beyond repair. A new generation of highly coherent, monolithic codebases will dominate the industry in the next couple of years. The highly skilled developers, those with at least a couple of systems and a lot of programming years under their belt, the clean coders, the domain-driven designers will become the shepherds of that new breed of systems. Others, whether they are the AI deniers, the vibe coders, the no-code evangelists will find it progressively harder to fit into that reality.

I bet 90% of coders think they’re in that high-skill category. Like the 90% of US drivers who think their driving skills are above average. I don’t think more than 1 in 10 devs is capable enough to handle the Wall. If you’re genuinely curious, it is not that hard to find out. Just start a fresh project in Cursor and see how long before your productivity noticeably goes down, or until you stop following what the generated code means. If it is less than a week, then the best, the only thing you can do is to get your coding back in shape by intense and deliberate practice.

Some companies are already acknowledging this reality by raising the hiring bar, deploying targeted junior talent development programs and taking a hard look at their existing staff. The era of free-riding is coming to an end in software and along comes the golden age of craft-centric software engineering…

OK, I might have embellished a bit. But regardless of a probability of this to happen by itself, if there is enough conviction within the community, this can become a self-fulfilling prophecy and I hope you agree it wouldn’t be the worst one. Shall we do something about it?