TL;DR: When an AI agent works for days across many sessions, the failure is rarely that the model was not smart enough. In Anthropic's own controlled experiments — same model, only the harness changed — agents failed because the state file let them quietly redefine what "done" meant, because each new session started building before checking whether the last session had broken something, and because the worker graded its own work and, predictably, praised it. The fixes are embarrassingly structural: a checklist the worker can only tick, a startup ritual that verifies the baseline first, and a separate evaluator whose only job is to be picky.
I run a personal AI operation that hands work between sessions and agent generations every day. For a long time I treated handoffs as a writing problem: if the next agent got confused, the handoff document had not been detailed enough. Anthropic's two long-running-agent studies, published in November 2025 and March 2026, convinced me that is backwards. The problem is not how much the checklist says. It is how much of it the worker is allowed to change.
In the first study, an initializer agent expanded a one-line prompt into a feature list of more than 200 items, stored as JSON, every item starting as failing. Each following coding session was allowed to do exactly one thing to that file: flip an item's passes field, with evidence. Deleting items or rewriting the tests was explicitly unacceptable. The authors chose JSON over Markdown because, in their observation, a model is less likely to inappropriately rewrite or overwrite it — a qualitative observation, not a measured guarantee, and I am treating it that way. The principle underneath it does not depend on the format: narrow the writable surface. If progress, evidence, and the definition of done all live in one free-form document the executor edits, then the executor is both athlete and rulebook. Fake completion does not need dishonesty. It just needs editing privileges.
The same study names the second failure I kept seeing in my own fleet: a new session walks in, looks around, sees progress, and either declares victory or starts stacking new work on a half-broken base. The prescribed ritual is fixed and boring. Confirm the workspace. Read the progress log and recent commits. Take the single highest-priority unfinished item. Then run a basic end-to-end test before writing anything new — and if the baseline is red, fix the baseline and build nothing. One feature per session, and leave a clean state behind. Boring is the point. Handoffs lose information by design; rituals are how you stop paying for that loss repeatedly.
The grader cannot be the worker
The March follow-up adds the finding I now consider non-negotiable. Left to evaluate itself, an agent will confidently praise its own work even when a human can see it is mediocre — and, the authors note, tuning a separate evaluator to be skeptical is far easier than teaching a generator to criticize itself honestly. So they split the roles: a planner that expands the brief into a spec, a generator that builds one sprint at a time, and an evaluator that clicks through the running application like a user and fails the whole sprint if any dimension falls below a hard threshold agreed in advance, in a written sprint contract, before code is written.
That separation is not free, and pretending otherwise would be dishonest. In one example the full harness cost more than twenty times a single agent — and the single agent's version had a core feature that simply did not respond to input, while the harness version was actually playable. In another, roughly four hours and about $124 in tokens still left an evaluator catching real gaps on its second pass. My reading, following the authors, is conditional: an independent evaluator earns its cost only when a task sits beyond what the current model can reliably complete alone. Inside that boundary it is pure overhead. Small, short, reversible tasks should not wear the full harness.
Scaffolding expires
The most useful line in the follow-up is the one most teams will skip. Every harness component encodes an assumption that the model cannot do something. When the model improves, that assumption expires — silently. The authors stopped guessing and started removing one component at a time. When a newer model arrived, sprint decomposition, once load-bearing, could be removed entirely without a drop; the planner and evaluator still carried weight, so they stayed. A parallel result from JetBrains Research points the same way on context: simply masking old tool outputs cost about half as much as running raw and matched or slightly beat elaborate LLM summarization on solve rate, while summarization actually lengthened trajectories by 13 to 15 percent. The cheapest structural move first; complexity has to prove it beats the simple version.
So here is the standard I am applying to my own operation, and I would argue it belongs in yours. Write state as a checklist the worker can tick and attach evidence to, never rewrite. Start every session by proving the baseline still works. Let someone who did not do the work decide whether it is done, against a contract written before the work began. And once per model generation, take your scaffolding apart one piece at a time and keep only the pieces that still carry weight. Reliability over days is not a smarter model. It is a narrower pen, a fixed ritual, a separate grader — and the discipline to retire the scaffolding that last year's model needed and this year's does not.
— Jiahui Miao, founder of vertciti, building a personal AI operating system for one.
