Most writing about AI coding agents is about what they can do. The more useful question, if you ship software with them, is how they fail. Chapter 13 of our book, Software Engineering in the Agentic Era, is a list of nine failures. Every one of them happened to us, and each one became a house rule. This post walks through the idea at the center of that chapter, because you can apply it this afternoon whether or not you ever read the book.
The idea is short: agent failures are almost never failures of intelligence. They are failures of verification, isolation and authority. The agent believed something without checking, touched something it shared, or did something nobody approved. All three are engineering problems, and engineering problems have engineering fixes.
The danger is quiet
The chapter opens with a line worth keeping in view: agents are astonishing at the things they are good at and quietly dangerous at the rest. The danger is rarely a dramatic crash. It is an agent reporting "all tests pass" when no tests ran, or a fix that works on a dirty working tree and not on the commit that got pushed.
That is what makes these failures expensive. Each one looks like success.
Verification: believing without checking
Four of the nine failures are about trusting a signal instead of the thing it was supposed to measure.
The confident non-result. A check prints nothing and the agent reports success. A filtered test command that matched zero files, a linter pointed at the wrong directory, a grep that found nothing because the path was wrong: all of them look green. The book's rule is blunt:
Empty tool output is not a pass. A gate that printed nothing may not have run at all.
The fix is to make every gate prove it ran. Count the tests, print the file list, and fail the job when the count is zero. In a shell gate that can be a few lines (an illustration, adapt it to your test runner):
files=$(git ls-files '*.test.ts') count=$(printf '%s' "$files" | grep -c . || true) echo "test files: $count" [ "$count" -gt 0 ] || { echo "no tests matched, failing the gate"; exit 1; } Verifying the wrong thing. The agent edits files, runs the tests, sees green, commits some of the files and pushes. The tests passed against the working tree, which included an uncommitted change the commit does not. CI goes red, or worse, CI does not cover that path and production does. The rule: verify the commit, not the worktree. Run the gate on a clean checkout of exactly what you are about to merge. With git, a throwaway worktree does it without disturbing anything:
git worktree add --detach ../verify HEAD (cd ../verify && npm ci && npm test) git worktree remove ../verify "It was already broken." An agent hits a failing test, decides it is unrelated to its change, and says so. Sometimes that is true. Often it is not: the change leaked state through a global, an import order shifted, or a shared fixture got mutated. Unless the agent ran the same tests on the base branch first, "pre-existing" is a guess. The book's tip is to make "run the suite on the base branch" the first step of any task where tests might already be red. It takes a minute and settles the argument.
The green check that proves nothing. Security scanners and code analysis tools on pull requests usually report new findings relative to the base. A green check means the change did not add an alert. It does not mean the alert you were asked to fix is gone, and agents (and people) routinely confuse the two. Close the loop on the thing itself: after the merge, rerun the scanner on the default branch and confirm that the specific alert closed.
The common thread is a proxy standing in for the result. One question catches all four: what exactly did this check run against, and how would I know if it ran at all?
Isolation: touching what you share
Run several agents on one repository and, as the chapter puts it, they will find every piece of shared state you forgot about. The examples are mundane, which is the point:
- The git stash is one stack per repository, shared by every worktree, so one session pops another session's work.
- Two sessions given overlapping tasks both ship the same feature.
- A cache directory gets clobbered.
- A scratch folder in
/tmpfills a per-user quota, and every command on the box starts failing with empty output.
That last one is the confident non-result again, arriving from a different direction: commands that silently produce nothing look exactly like commands that had nothing to say.
The answer is isolation by default: one worktree per agent, scratch space that belongs to the job and is deleted with it, and a quick search for open pull requests on the same topic before starting a long build.
Authority: doing what nobody approved
Most agent actions are reversible: edit a file, run a test, make a branch. Some are not: send an email to a customer list, post to social media, delete a database, charge a card, force-push over someone's branch, buy a domain. An agent that cannot tell the difference will eventually do something you cannot undo.
The rule the book uses is that anything outward-facing or hard to reverse needs an explicit yes, for that exact action, at that exact price, from a human. Approval for one purchase is not approval for the next. Its house rule:
Confirm before buying, sending or deleting. State the exact item and the exact cost, get an explicit yes, and never substitute something "close enough."
Secrets sit in the same category. Agents read everything in the working directory, including .env files, and they will happily paste a key into a log, a commit or a prompt to another service. Keep secrets in a vault, inject them at runtime, and give each agent only the keys its task needs.
Two more that fit the pattern
The chapter has two failures that are less obvious but just as common.
Doing exactly what the docs said. Agents trust documentation more than humans do. If your setup guide names a package that was never published, the agent will try to install it, fail, and then write its own version of it. If your style guide recommends a library you abandoned last year, you will get new code that uses it. Stale docs are bugs. Keep one machine-readable source of truth for your stack and have everything else point at it. Ours is profullstack.com/stack.md.
Scope creep at machine speed. Ask an agent to fix a typo and it may also reformat the file, rename a variable it disliked and upgrade a dependency. Each change is defensible. Together they turn a one-line review into a twenty-minute one, and one of them breaks something. Tasks should say what must not change, and reviews should reject unrequested changes even when they look like improvements. The agent can open a separate pull request for its good ideas.
A checklist you can take with you
- Every gate prints what it ran and fails when that count is zero.
- Gates run on a clean checkout of the exact commit being merged.
- Before calling a failure pre-existing, run the same test on the base branch.
- After a security fix merges, rerun the scanner on the default branch and confirm that specific alert is closed.
- One worktree per agent, and scratch space that is deleted with the job.
- One machine-readable source of truth for your stack.
- Every task says what must not change.
- Anything outward-facing or irreversible needs an explicit human yes for that exact action.
- Secrets live in a vault, are injected at runtime, and are scoped to the task.
None of these needs a smarter model. As the chapter closes, these are engineering problems with engineering solutions, and the rest of the book is mostly those solutions, plus the tools we built to make them the default instead of a discipline.
About the book
Software Engineering in the Agentic Era is a field guide to how one engineer and a fleet of AI agents ship real software, in five parts: The Shift, The Workbench, The Loop, Running the Fleet, and The People. It comes as PDF, EPUB, HTML and Markdown in one download, and it is pay what you want from $5 on Gumroad: get the book here.
Originally published on the Profullstack blog.


