Why the next major shift in software is not here yet
We are in an era where humans can offload their thinking and hand it over to the machines. I could even call this the biggest change humanity has ever seen. However, from a software development standpoint, it feels like a huge leap, and yet we struggle. To understand what lies ahead for software development, it helps to reflect on major historic shifts in the field.
A brief look into history#
In the 1950s to 1970s, the first shift happened when engineers favoured procedural languages like Fortran over assembly language. Procedural languages significantly reduced the cognitive effort needed to write a program. Engineers could use 47 lines of code to generate a thousand assembly language instructions. This freed engineers from managing registers and memory addresses, letting them focus more on algorithms and reuse.
The second shift happened when engineers favoured object-oriented/functional languages over procedural languages. This opened engineers’ minds to entities and state boundaries. Many excellent enterprise design patterns evolved during this era.
The third shift was the adoption of cloud, microservices and containers. Here, engineers no longer think of logic as confined to one service; instead, they focus on how multiple services work together to solve the business problem. This transition was made possible mainly by standardising transport semantics and deployable units. Several frameworks emerged aiming to accelerate software development. Suddenly, you can replace several lines of code with a single annotation or a CLI command.
As an avid user of LLM agents myself, I truly believe we are in the middle of the next shift: industrialising software development. However, when you look more closely at major shifts in the past, a pattern emerges. With every shift, cognitive load shrank. Engineers had to do less to achieve more. They could express their intent in a software language, framework, or CLI in a few words. Everything else happened under the hood. Trust grew because the systems driving the transition were deterministic. Low-level language compiler, high-level language compiler or a framework/container orchestrator: all of them were deterministic. They all translated the engineer’s intent (as language, commands, annotations) into executable code predictably and reliably.
What is different this time#
Now, engineers are forced to use a probabilistic system to make the next shift. This implies that, based on the input you provide, the agents might give different results that are closer to what is expected. I applaud the recent improvements in models that keep increasing output accuracy. Driving a model’s temperature to zero would not help, since it locks in hallucinations and changes with model updates. Even the best can make mistakes in a probabilistic system. Trust in the system is not high, so engineers are forced to read every line of code the agent generates. Did you notice what happened here? The cognitive load for a human has increased. Engineers do not spend time verifying whether the bytecode generated by a Java compiler makes sense. I keep asking myself whether we are ready for this and whether this is sustainable.
Many argue that the arrival of agents has made software and its engineers irrelevant. I do not agree. It is far cheaper and safer to use a deterministic system like software, yielding mathematically consistent results. Using a multi-billion-parameter model to run trivial deterministic steps is a blatant misuse of computing power. I do not think software engineers are made irrelevant either. Instead, they need to move up a level, as they did during past major shifts. Accountability and trust still lie with humans, not agents. That said, I also believe it is undeniable that agents will play a huge role in software development going forward.
What the agent reads#
Like a human, an agent also operates within the boundaries of available context in a probabilistic manner. I see two inputs to this probabilistic system: the quality of available context and the prompt. A bad input in either parameter will certainly yield poor results from the agent. This is what differentiates an expert engineer from a junior. An expert engineer has better domain context for the problem and waits to gather all necessary information before starting. This is mostly not the case with a junior engineer. For the expert, quality of the available context matters more.
The available context quoted above is not just one input; it spans multiple dimensions where the knowledge must be accurate and free of drift. Some of this dimensional knowledge may relate to technology, some to the organisation, and some to the task at hand based on past prompts. Models trained by frontier AI Labs have high-quality dimensional knowledge for writing code. As I see the agents perform today, they either do archaeology for those missing dimensions or they hallucinate. This compounds to several orders of magnitude in distributed systems. This leads to poor results and wasted tokens.
To truly industrialise software development with agents and let them operate independently, each component of the distributed service needs to provide consistent, cheap, driftless context. Without it, the results will be poor, regardless of prompt quality. Unfortunately, most of today’s solutions, from feeding human-written documentation to vector databases, fail as they tend to drift from reality. To establish trust, such context must be produced deterministically, not probabilistically.
The missing compiler#
The more I think about it, the more I see we are missing a deterministic system like a compiler that works across independently deployed services. Instead of worrying about each line of code, engineers must be accountable for what capabilities components in a distributed system expose and how they compose. A single line in this language would be equivalent to multiple lines of code in a service or across multiple services. Such a language could reduce an engineer’s cognitive load rather than increasing it.
I have discussed a capability-first approach in a distributed environment, with transports acting as costumes for the capabilities. The central store I described in that essay provides agents with accurate, deterministic dimensional knowledge to work with. In a distributed environment, much of the code is just composition of basic capabilities offered by different components. I see that composition as an orchestrated capability owned by any one of those components. It is also important to gather this composition in the central store. Doing so makes the capability a glass box, not a black box, eliminating the need for archaeology across boundaries. An agent equipped with such rich knowledge dimensions (capabilities across services running in an organisation and their composition) can act reliably well, even probabilistically.
Engineers can also focus more on core capabilities’ intents, guardrails, preconditions and their composition across the organisation at a high level. The missing compiler across system boundaries generates composition logic deterministically. The agent implements the core capability within the constraints engineers set. Notice that the agent has a limited surface to hallucinate, yet it is enriched with rich dimensional knowledge produced deterministically. A compiler generating composition logic doesn’t require a cognitive load. Reviews can also be constrained to a smaller surface. Accountability still lies with humans, with agents generating probabilistic code within the boundary humans declare.
Not here yet#
To conclude, software engineering as a field is undergoing a major shift, but the shift is not yet complete. And software engineering is not dying; it is raising its compiler target once more.