Hallucination is not a model bug. It is a design default.

Most agent stacks treat the language model as an oracle: prompt in, prose out, no provenance, no refusal contract, no CI gate that can say no. When the model does not know, it still answers — fluently, confidently, and wrong. That is not a failure of scale. It is what happens when three responsibilities are left implicit: remembering what was already decided, proving where an answer comes from, and blocking bad output before it ships.

This post describes a boring alternative that works: three small, zero-dependency Python packages that together make an agent whose answers are either cited or refused, and whose safety properties are enforced by tests and CI gates rather than by hope. Each package does one job. Combined — as demonstrated by the reference claude-verified-agent — they form an end-to-end loop: record context, retrieve evidence, gate delivery.

The three bricks are agent-context-bridge (inter-agent memory, v0.3.2 on PyPI), cited-rag (cite-or-refuse retrieval, v0.1.0 on PyPI), and llm-eval-ci (deterministic CI gates, v0.1.0 on PyPI). All are stdlib-only, offline-first, and installable from PyPI. The reference integration claude-verified-agent itself is not on PyPI — it is a GitHub reference repo whose CI installs the three pillars from PyPI at pinned versions, falling back to a portable stdlib path when the network is unavailable. Every number below is measured, with method and limits stated, and every claim points at a file you can re-run.

1. Shared memory: stop re-reading the repo at every agent switch

Every AI coding tool starts blind. You settle an auth design with one agent, switch to another, and the new agent re-reads the whole repository to rediscover what you already decided. That re-reading is the token tax of multi-agent work, and it grows linearly with codebase size.

Agent Context Bridge (ACB) replaces the dump with a bounded handoff. One append-only store (.agent_bridge/records.jsonl plus a human-readable SHARED_CONTEXT.md) records decisions once via acb record. At switch time, acb inject --target <hermes|claude|opencode|codex> replays only the N most recent records (default 5) in the target agent's native format, or acb search "query" returns only matching hits newest-first. A readonly MCP server (acb-mcp, stdio) exposes exactly two tools — acb_search and acb_status — both strictly read-only; the destructive prune command stays CLI-only with interactive confirmation, and unknown tool names fail closed with -32601.

The measured effect, on the published v0.3.0 benchmark over a 30-record store: naive re-read of 7 source files (60,768 chars, ~15,192 tokens at chars/4) versus bounded acb inject (793 chars, ~198 tokens). That is (15,192 − 198) / 15,192 = 98.7% fewer tokens. The same run reports −95.7% versus re-reading decision docs (4,555 → 198 tokens) and −90.5% versus forwarding the full shared-context file (2,078 → 198 tokens).

Methodology, stated honestly: tokens are approximated as chars/4, measured from real files rather than estimated, and the baseline is "what a blind agent re-reads at switch." The key property is structural, not a one-off: inject cost stays flat as the store grows because it is bounded to the last 5 entries, while naive re-reading grows linearly. search costs only the hits. Limits: the exact percentage moves with the tokenizer (chars/4 is an approximation), with store size, and with the baseline you compare against — which is why the report publishes all three baselines, not just the flattering one. The v0.3.2 suite passes 92/92 tests (re-run 2026-11-18: 92 passed in ~26 s), including live CLI and live MCP stdio end-to-end sessions.

Since the benchmark the package has hardened its distribution surface without changing the measured loop: server.json validates against the official MCP Registry schema under the name io.github.shenn667/agent-context-bridge (v0.3.2), Dependabot (pip, weekly) and CodeQL (Python) workflows are configured, and the Glama Discovery score is reported at 92%. A submission PR to awesome-mcp-servers (#19807) is open and reported mergeable.

2. Cited answers: verbatim or nothing

Retrieval-augmented generation usually returns prose with decorative citations — plausible-looking references that may not match any source text. cited-rag inverts the contract: the answer is extractive and verbatim, every sentence carries an exact [file:Lstart-Lend] offset, and anything the corpus cannot support produces Insufficient evidence with zero citations and exit 0. There is no "helpful" middle ground where the model improvises.

Under the hood it is deliberately unfashionable: BM25 plus sublinear TF-IDF cosine, fused by Reciprocal Rank Fusion (k=60, rank-only so incompatible score scales never mix), followed by a verbatim rerank. No embeddings, no vector server, no network, no API key. An opt-in --llm adapter exists but is off by default; the default path never touches the network.

On the bundled corpus (12 docs, eval/golden.jsonl with 52 questions: 40 answerable plus 12 off-topic), cited-rag eval reports Recall@5 = 1.00, MRR = 0.93, correct-refusal = 1.00 (12/12) — re-run 2026-11-18 with identical output. Definitions matter: Recall@5 is the fraction of answerable questions whose expected document appears in the top-5 retrieved chunk files; MRR is the mean of 1/rank of the first relevant chunk over answerable questions only; correct-refusal is the fraction of off-topic questions the full ask pipeline (search + generate + score gate) refuses. The release gates demand Recall@5 ≥ 0.80, MRR ≥ 0.60, refusal ≥ 0.90 — all cleared with margin, and the bundled pytest contract (tests/test_eval.py) asserts exactly those thresholds on the real corpus so regressions fail loudly. The v0.1.0 QA run additionally verified 3/3 cited sentences verbatim against source offsets with fidelity 1.00, 12/12 off-corpus refusals with no invented citation, and a green suite re-run at 61 passed.

Honest limits: this is a 12-document, 12-chunk technical corpus about the retriever itself — a favorable, self-descriptive domain where vocabulary overlap is high. Recall@5 of 1.00 means "the right document is always in the top 5 on this set," not "the agent is always right." MRR of 0.93 means the first relevant chunk usually ranks first, averaged over 40 answerable questions. Small n, narrow domain, no adversarial paraphrase set. The value of the number is not generality — it is reproducibility: one command re-derives it deterministically, and the gate blocks release if it drops.

3. CI gates: the safety net that can say no

Even cited answers need a last-mile check: does the produced text leak PII or obey an injected instruction? llm-eval-ci is a stdlib-only runner that turns that question into a blocking CI exit code. Cases are JSONL (id, prompt, checks[], optional max_tokens / max_latency_ms / max_cost_usd); eleven pure checks (contains, not_contains, regex, json_valid, json_schema_lite, refusal, budget_tokens, budget_latency, budget_cost, no_pii_leak, no_injection_obey) plus budgets, with strict fail-closed semantics: unknown check names fail the case instead of crashing the runner, malformed budgets fail, absent usage fails any budget check, and PII values are always masked (***) in reports. Output is report.md plus machine-readable report.json ({version, cases: [{id, status, failed_checks[], latency_ms, usage}]}), and any red case exits 1.

The semantics are pinned by SPEC_CASES.md — the normative contract shared by dev, security fixtures, and QA — so "what does this gate mean" has exactly one answer. The v0.1.0 QA run is the kind of evidence CI people respect: 146 tests passed (re-run 2026-11-18: 146 passed), 100% coverage over all 13 check modules, pytest --llm-eval additivity proven (N cases = N tests), offline install verified in a bare venv with --no-index, demo CLI in ~0.2 s, exit-1 behavior confirmed in both directions (red suites exit 1, an all-green suite exits 0), and budget checks verified individually (tokens 500 > 100 fails, 300 ms > 50 ms fails, $0.05 > $0.01 fails, missing usage fails closed).

The reference integration, claude-verified-agent, wires all three pillars into one auditable pipeline — record context (ACB), ask with citations (cited-rag), gate delivery (llm-eval-ci) — and subjects itself to both layers of proof: 14/14 contract tests (exact verbatim citations, off-corpus refusal with zero citations, gate blocks on hostile PII/injection strings, injection-laced questions never obeyed, determinism, JSON contract, memory recording, backend auditability) passing identically on the pillar path and on the portable stdlib-fallback path (re-run 2026-11-18: 14 passed), plus 3 blocking llm-eval suites (non_regression.json, schema.json, tripwires.json) whose red gates fail the build via make gates and the GitHub Actions pytest + gates jobs. Its CI installs the pillars from PyPI at pinned versions (agent-context-bridge==0.3.0, cited-rag==0.1.0, llm-eval-ci==0.1.0) and degrades to the stdlib fallback only if the install fails — so the green build proves the PyPI artifacts, not just the local checkout. The shipped report.json/report.md show the tripwire suite 2/2 passed; the red path (leaked PII or obeyed injection fails, exit 1) is proven in pytest. Limits, again stated plainly: the verified-agent suites are small by design (tripwires, not a red-team corpus), and the package itself ships via GitHub, not PyPI.

Quickstart: three commands

pip install agent-context-bridge cited-rag llm-eval-ci acb init && acb record --topic "decision" --details "why, in one line" cited-rag eval --golden eval/golden.jsonl && llm-eval run evals/tripwires.json 

That is the whole loop: record a decision, verify retrieval quality against the golden set, run the safety tripwires as a blocking gate. Then explore the reference wiring — it is not on PyPI, clone it from GitHub:

git clone https://github.com/shenn667/claude-verified-agent cd claude-verified-agent && make demo && make test && make gates 

Its CI resolves the same three PyPI pins.

Build on it

Hallucination stops being mysterious once refusal and gating are designed in. If your agent cannot point at the line it quoted, it should say so; if its output trips a PII or injection check, CI should say no. Small, offline, auditable pieces compose better than one opaque oracle.

  • agent-context-bridge v0.3.2 — PyPI · GitHub · MCP Registry io.github.shenn667/agent-context-bridge (registry) · Glama
  • cited-rag v0.1.0 — PyPI · GitHub
  • llm-eval-ci v0.1.0 — PyPI · GitHub
  • claude-verified-agent (reference integration, GitHub only, not on PyPI) — GitHub

Clone one, run its QA commands verbatim, and check the numbers yourself. That reproducibility is the entire point.