The fear of losing control over code is not new.
When compilers appeared in the 1950s, many programmers were skeptical. Could a compiler really write machine code as well as a skilled programmer? It turned out it could.
Programming moved to a higher level of abstraction. The complexity of machine instructions was encapsulated, but responsibility for the program’s behavior remained with the developer.
Every new level of abstraction hides some complexity and moves the developer’s attention elsewhere. It does not eliminate responsibility; it changes what we need to understand and control.
AI coding agents are another step in this evolution. But there is a difference: an agent does not just translate an intent into code. It also interprets it, fills in gaps, and makes decisions the developer may never have explicitly considered.
So when the same description produces different programs, who is actually nondeterministic: the model, or the human who left some decisions open?
To understand where uncertainty resides, we built an experimental testbed to quantitatively evaluate how models behave.
The testbed
- One project: a community library. Members, books, loans, renewals, holds, fines.
- Three texts go in:
- The owner's short request, 78 words, quoted below.
- The owner's full decisions: 20 of them, written out as rules, 417 words. The real owner's answers: how the library must work.
- The short request expanded by Claude Opus with no human in the loop: about 2,000 words, every open point settled.
Hi! We run a small community library and need a simple system for loans. Members borrow books for three weeks and can renew them a couple of times, unless someone else is waiting for the book. If a book comes back late we charge 25 cents a day, but we don't want to be harsh with people. Members can put a hold on a book that's out. And we need to keep track of who owes us what.
- The coder: Claude Code, one call, one reply. No tools, no tests, no questions. A 79-word system prompt: you are an engineer, write it, nobody can answer, decide.
- Three models: Haiku, Sonnet, Opus. 30 runs per text and model: 270 programs. Each run in a locked container that reaches only the API.
- Hidden from the models: 19 of the owner's 20 decisions and 31 scenarios with the result the owner expects. The twentieth decision is the order of a list, and we do not score it. Three examples: how long a loan lasts, how many books a member may have out, how long a held book waits for pickup.
- A reference program passes all 31. Break any one rule in it, and the scenarios catch it.
- Two numbers out. Match: the share of scenarios done the owner's way. Spread: how many different programs one text gave. Two programs are the same when they do the same on every scenario.
Where the spread lives
We started with the short request, the 78 words a client would actually send. We ran it 90 times, 30 on each model, then lined the programs up and compared what they do. Run it once and you get a library. Run it again and you get a slightly different library: same books, different rules. 90 runs gave us 64 different programs. Haiku repeated itself only 5 times in 30 runs. Opus 7 times. Sonnet was the calmest, and still gave 17 different programs.
But the differences were far from random.
9 of the 19 decisions: all 90 programs read them the same way. The other 10 split.
Every rule the owner stated reads one way in every program. A loan lasts 21 days: no disagreement in 90 runs. The same goes for plain common sense, like a book on loan cannot be borrowed by someone else.
The splits are all in the decisions the owner never made, or made vaguely. How many books may a member have out at once? What debt blocks a new loan? What does "we don't want to be harsh" mean for a late return? How long does a held book wait for pickup?
The models did not miss these questions. Our prompt asked each run to list every decision it had to make because the text did not say. The lists came back 20 to 24 lines long on average, about the size of the owner's own 20 rules. The pickup window is on 90 lists out of 90. The debt rule, 90 out of 90. The model knows exactly where the gaps are.
The same lists also carry "Loan period is 21 days", which the owner did say, in 90 lists out of 90. "To avoid being harsh, the fine is capped at 1000 cents" stands next to it in the same voice. A rule the owner stated and a rule the model invented look the same on the page.
So the model finds every open question. It answers each one itself. And it writes the answer down as a rule, next to the owner's rules.
The owner's decisions: one program, no misses
Now the owner's 20 rules, 417 words. Same models, same prompt, same tests. 90 runs, one program. Every scenario passes, on Haiku as well as on Opus.
The 90 code files are all different. The 90 programs are one. The model is as random as before; the randomness has nowhere to go.
On everything the owner decided, the programs agree. On everything the owner left open, they split. So the gaps have to be filled before the model sees the text. The question is who fills them.
Can we just let the model decide?
One might argue that this is not really a problem. If a model can turn a short request into a complete text, why involve the owner at all? Let the model fill the gaps once, then build from that.
We tested this too.
We gave the same 78 words to Claude Opus and asked it to write the full text a developer could build from without asking a single question. It came back with about 2,000 words: a table of values, every open point settled, not a human in the loop. Then we compiled it 90 times, exactly as before.
Ninety runs, one program. Haiku, Sonnet and Opus all built the same library from this text. The spread was gone.
The library was wrong in 5 places out of 19, and wrong the same way in all 90 runs. It matched the owner on 80% of the scenarios. The grace rule is the clearest miss. The owner wanted two free days and then every late day charged, so three days late costs 75 cents. The expanded text took the two days off the bill instead: three days late, 25 cents. A renewal counted 21 days from the day you renew, not from the old due date. A held book waited seven days for pickup, not three. And when nobody came for it, the next person in line got seven days too.
Now look at where the five misses are. All five sit among the ten decisions where the short request had split. On the nine decisions where all 90 programs had agreed, the expanded text was right every time. The spread in the previous section was a map of the risk. The expansion erased the map and kept the risk.
Maybe we were unlucky with this one expansion. So we ran the expansion ten times and compiled each one ten times on Haiku. Inside each expansion, no spread at all. Across the ten expansions, 10 different programs, each right on 11 to 15 of the 19 decisions. Three of the owner's rules appeared in none of the 10: the two free days, the three-day pickup window, and what happens when it expires. They could not appear. Nothing in the 78 words points to them. Ten throws or a hundred, it makes no difference.
So the uncertainty had not disappeared. It had moved. Instead of different programs from the same text, we got different texts from the same request. Freezing one of them makes the code consistent. It does not make the decisions right.
The dice are still being thrown. Only at a different stage.
Where responsibility moves
The result is plain, and probably what most readers expected. Here is how we read it. Every run is its own microstate: 270 runs, 270 different code files, with the owner's rules in the text just as much as without them. The model rolls the dice on every run.
Determinism, if the word means anything here, is a property of the macrostate: what the code does, whatever the implementation. By that definition the owner's text gave us determinism, 90 microstates and one macrostate. The short request did not: 90 microstates, 64 macrostates. The dice were thrown in both cases. The difference is where they landed. With the owner's rules in the text, they landed inside the code, where nobody minds. Without them, they landed on the rules of the library.
So the divergence is real, and now we know where it sits: not in the model, in the decisions nobody has made. Determinism cannot come from the model, however big. It cannot come from the model filling the gaps once either; the previous section showed what that buys. It comes from the owner's decisions, and only from them. Determinism, then, needs the business owner in the loop, and much deeper in the loop than a request at the start: in the gaps, answering, before anything is frozen.
But finding the gaps is not the owner's job. It is the model's, and it is good at it: 90 lists out of 90 named the pickup window. Answering them it does badly: on the open questions its guess is a coin toss at best. In our testbed the model was told that nobody could answer, so it decided and moved on. Most agents today do the same by default.
Compilers took the machine instructions away and left the responsibility for the program's behavior with the developer. Agents take the typing away and leave the decisions. It is no longer enough to verify the code. We must also verify the intent the agent inferred, and better, answer its questions before it has to infer anything.
The spread is not the disease. It is a symptom of decisions that have not yet been made.
Who answers them in your process today?
