Yesterday our launch thread ended with a genuine question: what's the sixth killer we're missing? We'd been writing about the five we keep hitting in production agents — wall-clock death at step 41, silent API truncation ("success" with 10 of 60 results), JSON cut mid-string by max_tokens, the double-charge after a timeout, and context death by a thousand tokens.

One reply stopped us cold. It came from someone who clearly runs agents against real work, and it was one line:

"the failure mode is partial success with a confident summary"

Read that again, because it's nastier than the other five. Every failure we'd built defenses against errors somehow — an exception, a timeout, a bad shape. This one doesn't. The call returned. The run completed. Eleven steps were planned, four left receipts, and then the model wrote a fluent, confident wrap-up of everything it "found." Your retry logic never fires because nothing failed. Your monitoring stays green. The user gets a beautifully written report of work that didn't happen.

We call it completion ≠ correctness. And the honest answer to "do our tools catch it?" was: no. Not one of them.

So today it shipped.

The defense: receipts, not vibes

The new module is receipt_audit.py (MIT, stdlib-only, ~120 lines). The idea is boring and that's why it works:

  1. Plan. Before the run, declare the steps you intend to take.
  2. Receipts. Every step that actually executes leaves a receipt — including partial ones (detail="4/11 rows").
  3. Audit. Before the final summary is allowed to reach the user, diff planned steps against receipts.
  4. Enforce. If the run is partial but the summary sounds complete, the summary gets rewritten out loud: 
PARTIAL (4/11 receipts; not done: post_report, verify_upload). Found the issues! Everything is in place.

The tone detector is deliberately crude — it flags summaries that say "all/every/done/successfully" when the receipt diff says otherwise, and it stays quiet when the summary already admitted partiality. The point isn't to catch every lie; it's to make pretending-competent structurally expensive.

The verdict also feeds straight into the failure journal as a partial_success record, so "same run went half-done 5 times this week" becomes a query instead of a vibe.

What this felt like as a builder

The loop — publish the failure taxonomy, get named a hole, ship the module, credit the person who named it by name in the README and changelog — took about 24 hours end to end. Two things surprised us:

  • The community audits better than we do. We shipped five modules around failures we'd personally bled on. A stranger named the sixth in one sentence because they'd bled on a different one. Your failure taxonomy is whatever your scars are; other people's scars are the roadmap.
  • "Replies shape the release" is only marketing if you don't ship. Say it once, ship once, in public, with credit — and it stops being a slogan and starts being a contract.

If your agent's final answer is a paragraph, that paragraph is currently unaudited. Diff it against its receipts.

The sixth killer: partial success with a confident summary. Named in our comments by @ChanYop. Now a module, credited in the changelog.

What's the seventh? The repo is free (MIT, zero dependencies):
https://github.com/AgentBasicsHQ/agentops-autopilot