The Decisions Nobody Made

Code Is Cheap. Being Wrong Isn’t.

Three sentences I hear more and more often:

“The tests passed.”

“Copilot suggested it.”

“The agent handled the implementation.”

Each sounds reasonable. Together they point to a problem.

Coding agents are good at producing software. They implement features, run migrations, explore unfamiliar APIs, fix compiler errors and grind through the repetitive work that used to eat entire afternoons. We should use them for exactly that.

The trouble starts when delegating implementation turns into delegating judgment.

Responsibility did not move

An agent proposes a concurrency strategy. It works in staging and collapses under production load. Who decided to use it?

“The agent chose it” is not an answer available to a professional engineer.

The agent has no customer relationship to damage, no reputation, no pager, no employment contract and no career. It will not explain the outage to the CEO.

This is not about model intelligence, so better models will not fix it. Responsibility needs someone who can make or refuse a decision and live with the consequences.

An agent can recommend PostgreSQL over DynamoDB, write a solid comparison of both, and generate the Terraform before you finish reading it. But somebody still has to say: for this system, with these consistency requirements, this operating model, these engineers and this expected lifetime, we choose PostgreSQL. That person owns what the choice rules out as much as what it enables.

So the distinction is between bearing responsibility and exercising it.

Engineers still bear responsibility for their systems. But it has become very easy to stop exercising it: accept the generated design, see green tests, merge, move on.

Responsibility you bear but no longer exercise is not delegation. It is exposure.

First, say what kind of software you are talking about

Much of the debate about AI-assisted development goes nowhere because people talk about different kinds of software.

Someone vibe-coding a side project for twelve users and a team maintaining a ten-year-old, business-critical platform share no constraints.

For a prototype, letting the agent build the whole thing can be perfectly rational. If the architecture turns out terrible, delete the repository.

That changes once the software handles revenue, customer data, or processes that have to keep running for another decade. What matters is not whether the code was vibe-coded or written by professionals. What matters is blast radius, reversibility, lifetime, and who pays when it fails.

A customer-facing product deserves diligence. A financial, regulated or safety-critical system deserves enough care that “the agent seemed confident” never shows up in an incident report.

Agents are good at local work

Software gives agents an unusually rich feedback loop, and they have been trained on most of the public code ever written.

The compiler flags wrong syntax. Tests flag behaviour that breaks known expectations. Linters complain, benchmarks produce numbers, CI turns green or red.

That makes a lot of engineering work delegable. Once I have decided the API contract, data model, invariants and acceptance criteria, I am happy to let an agent write the implementation: the boilerplate, the API migration, the library research, the straightforward bug fix, even the review of another agent’s work.

Insisting on doing that work by hand is not craftsmanship. It is wasted time.

The question is where the feedback loop stops being good enough.

What no test will tell

Take a data model. No test tells you that the chosen representation will make every future feature painful.

The compiler cannot tell you that a boundary belongs in another subsystem. A unit test cannot tell you that a queue should not exist. Your integration suite cannot decide whether the system should use threads, async I/O, processes, event-driven messaging or, occasionally, the revolutionary approach of doing one thing after another.

These decisions shape what the product can become, and none of them turns a test red. So the agent optimizes what it can see and finds a good local solution, because that is where the signal is.

Professional engineering often means rejecting a locally attractive solution because of something outside that local view: a global invariant, an operational constraint, an organisational boundary, an obscure customer requirement, or a decision made six years ago for a reason nobody bothered to put in the README.

Humans cannot keep an entire enterprise system in their heads either. The difference is that a responsible engineer knows there are things they do not know, and is accountable for finding out which of them matter before committing to a decision.

Defensive Checks instead of Design

A tempting failure mode is replacing design with verification.

Agents are excellent at producing tests and assertions. That is useful, but more assertions do not make better software. The usual reflex is to check for an impossible state. The better move is to make it unrepresentable.

If a domain object has seventeen possible combinations and only four are valid, you can add thirteen runtime checks. Or you can design a type that only represents the four valid ones.

Generated code tends to be very defensive. Every function validates its input, every exception is caught, every failure has a fallback. And the system as a whole gets harder to reason about. Karpathy described models as “mortally terrified of exceptions”. That is a real problem: defensive code is becoming a substitute for strong invariants.

The responsible engineer asks a different question: which states should this system be able to represent at all? That is an architectural question, not a test-generation one.

The same goes for testing in general. The human job is less and less writing tests and more deciding which tests count as evidence.

The Bun problem

Public agentic success stories are easy to misread.

A mechanical transformation suits agents very well when the target architecture is known and a strong test suite gives fast external feedback. Bun’s Zig-to-Rust port had both: a test suite written in a language independent of the implementation, and an explicit decision to make the Rust a transliteration of the Zig, so nothing had to be redesigned.

Every hard design decision in that project had already been made by humans.

That does not show that agents can design systems. It shows that they are very effective once the hard decisions are made.

And it does not mean a twenty-year-old enterprise application can be treated the same way. Legacy systems are full of implicit knowledge. Some of it lives in tests, some in documentation. Much more lives in production configuration, historical data, operations procedures, and sentences starting with “Yes, that looks strange, but in 2019 …”

The real specification is often just the behaviour customers have learned to depend on.

That is why “rewrite it with agents” is much less convincing for mature systems than it sounds. Generating equivalent code is the easy part. Deciding what “equivalent” means is the hard one.

What to hand over, and what to keep

None of this is an argument for keeping agents on a short leash. Let them loose wherever failure is cheap. Humans spending their days on boilerplate, simple refactoring and mechanical implementation waste brainpower and speed.

The skill is knowing where to let the agent run and where to stay in control. My rules of thumb:

Everything in the Me rows is either choosing a target or noticing that the target was wrong.

Let the agent generate the adapter, but own the domain model. Let it migrate three hundred API calls, but own the transaction boundary. Let it propose five scaling strategies, but decide whether the system needs to scale that way at all. Let it write fifty tests, but decide whether they prove anything.

The line between agent work and human work keeps moving towards the agent, and more will cross it. Choosing and noticing should not, because both need a stake, and a stake is not a capability.

The less reversible a decision, the more explicitly a human should own it, with a real name attached.

The developer job is moving up

This has an uncomfortable consequence for individual contributors.

For a long time, a large part of the job was translating requirements into code. The product owner explained what was needed, the developer built it. Agents make that translation cheap. Once a product owner can describe a well-defined feature, an agent can implement much of it directly.

“I take tickets and turn them into code” is no longer enough.

Typing faster than the agent is not the answer. Owning the feature is. That means understanding four things:

1. Business process and customer value. Why should this exist? Which customer problem does it solve?
2. Architecture. Where should the change live? Which boundaries must stay intact? Does the problem really need concurrency, messaging or another distributed component?
3. Operations. How does the feature behave in production? What must be observable?
4. Verification. What evidence should convince us the requirement is met? Which tests matter?

Agents can help with all four: research customer workflows, compare architectures, draft SLOs, generate threat models, write test suites. But help does not transfer the decision.

You have to be able to explain your choice without saying “that is what the model suggested.” Once that sentence becomes your justification, you are no longer an engineer, just an expendable parrot.

Fewer people, each owning more

There is an optimistic version of this, too.

If implementation gets much cheaper, teams can get smaller while each person owns more. A product owner, a senior engineer and a junior engineer with strong agents may deliver what used to take a much larger team.

But the smaller team cannot be a compressed copy of the old one. Its members need to understand a feature end to end: customer need, architecture, implementation, operation.

That is demanding. It is also closer to what software engineering should have been all along.

What remains ours

Agents made code cheap. They did not make being wrong cheap. They did not change who gets called when production fails, who explains the decision to a customer or the CEO, or who decides a feature should not be built.

We used to think of the diff as our main artifact: the code we wrote and reviewed. More and more, what we own is the constraint set around it: business intent, architecture, invariants, operational requirements, and the evidence by which correctness is judged.

Inside those constraints, let the agent do as much as it can. That is not surrendering engineering to AI. It is putting automation where it belongs.

Delegate hard. Decide deliberately.

And when a decision matters, make sure someone in the room is ready to say:

“I chose this, and I can explain why.”