A checklist that lives in a wiki is a wish list. A checklist that emits a dated, hashed artifact, and a script that refuses to exit 0 without one, is a gate.
That distinction is the whole article. If your readiness review can pass because someone ticked a box at 5:40pm on a Friday, you have a ritual, not a control. What follows is the receipt model I use for small AI prototypes: one JSON file per gate, a TTL per gate, and a checker that fails closed when anything is missing, stale, failed, or unsigned.
The failure mode this fixes
Gate lists rot in a predictable way. The first sprint, everyone runs the checks. The second sprint, someone reuses last month's result because "nothing changed". The third sprint, the evidence lives in a Slack thread that the search index can't find.
Then a prototype gets promoted on a Tuesday and nobody can answer the only question that matters: what exactly did we verify, against which commit, and how long ago?
Receipts answer that question mechanically. Three rules make it work:
- Every gate writes a file. No file, no pass.
- Every file expires. A seven-day-old latency measurement is a guess.
- One command decides. Exit non-zero unless every required receipt is fresh, passing, and attributable.
Rule 1: the receipt shape
Keep the schema boring. Six fields, one nested evidence block, no free-form prose.
{ "gate": "json_contract", "status": "pass", "checked_at": "2026-11-08T09:12:00Z", "commit": "9f3c1ab", "evidence": { "command": "python -m gates.json_contract --fixtures tests/fixtures/bad_json", "exit_code": 0, "artifact": "artifacts/json_contract_2026-11-08T09-12-00Z.log", "sha256": "3b1f...c07a" }, "signed_by": "taylor" } A few constraints earn their keep.
-
checked_atmust be timezone-aware ISO 8601. A naive timestamp is a bug, not a style choice. -
statusis only everpassorfail. "Partial" is how checklists die. -
evidence.artifactis a path that must exist. A pass with no artifact is a claim. -
signed_byis empty for automated gates and non-empty for anything a human attests to.
Rule 2: gates expire on different clocks
Not everything deserves a daily re-run. Latency drifts hourly; an egress allowlist drifts monthly. Put the TTL in the rules table, not in someone's memory.
| Gate | Question it answers | Evidence artifact | Fail-closed when | TTL |
|---|---|---|---|---|
json_contract | Does a malformed model response get rejected, not repaired? | Rejection log + exit code | Any fixture is accepted | 7 days |
latency_probe | What is p95 right now, under my own payloads? | Timed probe log | p95 over your budget, or <20 samples | 24 h |
degradation_drill | What does the UI show when the model returns 429/500? | Screen capture + response body | Blank screen, spinner, or fake success | 14 days |
cost_ceiling | What stops a runaway loop before it eats the allowance? | Counter test + the guard's source hash | No hard stop, or stop untested | 7 days |
egress_allowlist | Which hosts can the agent actually reach? | Resolved IP set + allowlist diff | Any unlisted host resolves | 30 days |
human_signoff | Who accepts the residual risk? | Signed receipt referencing the other five | Missing name, or signer = author only | 30 days |
Two rows do most of the work. degradation_drill catches the failure users actually see. human_signoff catches the failure teams actually have: nobody owning the decision.
Rule 3: the checker fails closed
The script below is illustrative. Dry-run it against a fake gate-receipts/ directory before you wire it into CI — including the case where the directory doesn't exist, which should be a failure, not a skip.
#!/usr/bin/env python3 """gatecheck.py -- fail closed when a receipt is missing, failed, or stale. Exit codes: 0 every required gate has a fresh, passing, correctly signed receipt 1 at least one gate failed, is stale, or is unsigned 2 the receipt directory is missing or unreadable (also a failure) """ from __future__ import annotations import json import pathlib import sys from datetime import datetime, timedelta, timezone RECEIPT_DIR = pathlib.Path("gate-receipts") # gate name -> (max age in hours, needs a human signature) RULES: dict[str, tuple[int, bool]] = { "json_contract": (24 * 7, False), "latency_probe": (24, False), "degradation_drill": (24 * 14, False), "cost_ceiling": (24 * 7, False), "egress_allowlist": (24 * 30, False), "human_signoff": (24 * 30, True), } def parse_ts(raw: str) -> datetime: ts = datetime.fromisoformat(raw.replace("Z", "+00:00")) if ts.tzinfo is None: raise ValueError("receipt timestamp must be timezone-aware") return ts def check(name: str, max_age_h: int, needs_human: bool, now: datetime) -> str | None: path = RECEIPT_DIR / f"{name}.json" if not path.is_file(): return f"MISSING {name}: no receipt at {path}" try: data = json.loads(path.read_text(encoding="utf-8")) except json.JSONDecodeError as exc: return f"INVALID {name}: unreadable receipt ({exc})" if data.get("gate") != name: return f"INVALID {name}: receipt declares gate={data.get('gate')!r}" if data.get("status") != "pass": return f"FAILED {name}: status={data.get('status')!r}" if not data.get("evidence", {}).get("artifact"): return f"NOEV {name}: pass recorded with no artifact path" age = now - parse_ts(data["checked_at"]) if age > timedelta(hours=max_age_h): return f"STALE {name}: {age.total_seconds() / 3600:.1f}h old, max {max_age_h}h" if needs_human and not data.get("signed_by"): return f"UNSIGNED {name}: human sign-off required" return None def main() -> int: if not RECEIPT_DIR.is_dir(): print(f"FAIL: {RECEIPT_DIR}/ not found -- run the gates first", file=sys.stderr) return 2 now = datetime.now(timezone.utc) problems = [ p for name, (age_h, human) in RULES.items() if (p := check(name, age_h, human, now)) is not None ] for p in problems: print(p, file=sys.stderr) if problems: print(f"FAIL: {len(problems)} gate(s) not satisfied", file=sys.stderr) return 1 print(f"PASS: {len(RULES)} gates fresh and signed") return 0 if __name__ == "__main__": raise SystemExit(main()) Run it locally first, because the exit code is the product:
python tools/gatecheck.py; echo "exit=$?" # FAIL: gate-receipts/ not found -- run the gates first # exit=2 Then in CI, treat anything non-zero as a hard stop:
- name: readiness gate run: python tools/gatecheck.py # exit 1 or 2 must block the merge and the deploy, not warn The exit=2 case matters more than it looks. Most teams write the checker so a missing directory is "nothing to check yet", which silently turns the gate off in exactly the repo state where discipline is lowest.
Rule 4: receipts are written by the gate, never by hand
A hand-written receipt is a comment. Have each gate write its own file, then let the checker read it:
python -m gates.latency_probe \ --samples 30 \ --p95-budget-ms 2500 \ --receipt gate-receipts/latency_probe.json \ --artifact-dir artifacts/ If a gate can't produce an artifact, that's information. It usually means the gate was a vibe, not a test.
Where free-tier tooling fits, and where it doesn't
Two places in this workflow benefit from cheap or free infrastructure, and one place absolutely does not.
For drafting and reviewing the gate criteria, I used MonkeyCode's free model access to generate candidate failure modes for each gate, then cut roughly two thirds of them. Free model access is good at breadth here — it will happily enumerate 40 ways a model endpoint fails, and you pick the six that matter for your traffic. It is not an authority: the pass/fail decision and the signed_by field stay human.
Disclosure: This article was prepared as part of MonkeyCode's product outreach.
For hosting, the free server option is enough to run gatecheck.py behind a small internal endpoint so reviewers can see the current ledger without cloning the repo. What it is not: a durability guarantee. Keep the authoritative receipts in the repo and in CI, treat any hosted view as a cache, and never put a free-tier host in the deploy critical path.
Who should not use this approach: solo throwaway scripts, prototypes with no users and no data, and anything where the gate would cost more to maintain than the incident it prevents. A two-gate ledger is fine. Six gates on a weekend demo is theatre.
A rollout you can finish in one afternoon
- Pick three gates, not six. Start with
json_contract,latency_probe,human_signoff. - Add
gate-receipts/to.gitignoreexcept forhuman_signoff.json, which is a decision record and belongs in review. - Write the receipt from inside each gate, with the command, exit code, and artifact hash.
- Run
gatecheck.pyin CI on the default branch and on any PR touching the model client. - Break it on purpose: delete one receipt, backdate another by 30 days, and clear a
signed_by. All three must exit1. - Log the exit codes for two weeks. If
STALEis your most common failure, your TTLs are too tight, not your team's discipline.
Limitations worth naming
Receipts prove that a check ran. They don't prove the check was good — a badly written latency_probe will happily pass forever. The TTLs above are starting points, not laws; measure your own drift before trusting them. And nothing here replaces a human who can say "I accept this risk" and mean it.
If you want to try the whole loop end to end on a side project, the free model access and free server option are enough to get from an empty repo to a failing-then-passing ledger in an afternoon.