Make the Checklist Exit Nonzero: Six Gates, One TOML File, No Skipped Rows

Last Friday I opened our launch checklist. Fourteen rows. Eleven checkmarks.

Every checkmark was mine. I had ticked them from memory at 6pm.

Then the free-tier canary returned HTTP 200 with error: quota exceeded inside the body. The checklist said green. The product said no.

A checklist you read is a wish you wrote

Why do we keep writing launch checklists as prose? Because prose is cheap and nobody has to run it.

A row that can't fail is decoration. A row that can't name an owner is folklore. A row with a "skip" state is a hole with a checkbox drawn on it.

So I rewrote ours as six commands. The file exits nonzero or it doesn't ship.

Step 1: Write the gates as commands, not sentences

One gate per deploy-blocking risk. Each gate gets a command, an expected exit code, an owner, and one line of why.

# gates.toml — every row must be runnable by a machine [meta] receipt_dir = ".gates" [[gate]] id = "key-present" kind = "blocking" owner = "sam" cmd = 'test -n "$MONKEYCODE_API_KEY"' expect_exit = 0 why = "No key, no call. Stop before we waste an hour." [[gate]] id = "quota-budget" kind = "blocking" owner = "sam" cmd = "python3 scripts/quota_guard.py --budget 200000" expect_exit = 0 why = "Stop at my budget, not at the provider's 429." [[gate]] id = "canary-one-task" kind = "blocking" owner = "sam" cmd = "python3 scripts/canary.py --task summarize --timeout 60" expect_exit = 0 why = "One task, one call. Cheap proof the wire is alive." [[gate]] id = "silent-error-scan" kind = "blocking" owner = "sam" cmd = "! grep -qi 'error' .gates/canary.log" expect_exit = 0 why = "Exit 0 with 'error' in the log is a lie. Catch the lie." [[gate]] id = "rollback-dry-run" kind = "blocking" owner = "sam" cmd = "python3 scripts/rollback.py --dry-run" expect_exit = 0 why = "An untested rollback is a hope, not a plan." [[gate]] id = "latency-note" kind = "advisory" owner = "sam" cmd = "python3 scripts/latency.py --p95-max 8000" expect_exit = 0 why = "Slow is not broken. Record it. Do not block on it." 

Notice gate four. It exists because our worst incident was a process that exited 0 while printing an error. Exit codes lie when nobody reads the log.

Step 2: Run them with a runner that refuses to skip

Forty lines of Python. No framework, no dashboard, no YAML indentation debates.

#!/usr/bin/env python3 """run_gates.py — fail-closed checklist runner (Python 3.11+).""" import hashlib, json, subprocess, sys, time, tomllib from pathlib import Path ROOT = Path(__file__).resolve().parent SPEC = tomllib.loads((ROOT / "gates.toml").read_text()) OUT = ROOT / SPEC["meta"]["receipt_dir"] OUT.mkdir(exist_ok=True) def run(gate): t0 = time.time() p = subprocess.run(gate["cmd"], shell=True, capture_output=True, text=True) log = OUT / f"{gate['id']}.log" log.write_text(p.stdout + p.stderr) return { "id": gate["id"], "kind": gate["kind"], "owner": gate["owner"], "cmd": gate["cmd"], "exit": p.returncode, "expected": gate.get("expect_exit", 0), "seconds": round(time.time() - t0, 2), "log": str(log.relative_to(ROOT)), "sha256": hashlib.sha256(log.read_bytes()).hexdigest()[:16], "why": gate.get("why", ""), } results = [run(g) for g in SPEC["gate"]] for r in results: ok = r["exit"] == r["expected"] r["status"] = "PASS" if ok else "FAIL" blocked = [r for r in results if r["kind"] == "blocking" and r["status"] == "FAIL"] receipt = { "when": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()), "verdict": "BLOCK" if blocked else "GO", "results": results, } (OUT / "receipt.json").write_text(json.dumps(receipt, indent=2)) for r in results: print(f'{r["status"]:5} {r["id"]:18} exit={r["exit"]:<4} owner={r["owner"]}') print(f'\nverdict: {receipt["verdict"]}') sys.exit(1 if blocked else 0) 

There is no SKIP. A missing script returns 127, which never equals 0, so a deleted gate fails loudly. That single decision removed half the arguments we used to have at 6pm.

Run it:

python3 run_gates.py; echo "exit=$?" head -20 .gates/receipt.json 

Step 3: Break a gate on purpose before you trust it

A checklist you have never seen fail is a checklist you have never tested.

# failure fixture: remove the credential, expect BLOCK export MONKEYCODE_API_KEY="" python3 run_gates.py; echo "exit=$?" # exit=1, verdict: BLOCK # second fixture: make the canary exit 0 but log an error printf 'error 429 quota_exceeded\n' > .gates/canary.log python3 run_gates.py; echo "exit=$?" # gate 4 catches it 

Two minutes of deliberate breakage. That is what turned my fourteen checkmarks into six gates I actually believe.

Step 4: Rehearse on free capacity, then pay for the real thing

Here is where the free tier earns its keep. MonkeyCode is an open-source project that, per its operator, offers free model access and a free server option, along with a free token allowance (the operator states 10M tokens; quotas change, so read the current terms before you plan around it).

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The point is not the number. The point is having a disposable target for the runner. My sequence:

1. Point MONKEYCODE_API_KEY and the base URL at the free server.
2. Run all six gates. Save the receipt as receipt-free.json.
3. Fix whatever fails — usually the budget guard.
4. Swap the base URL to the paid endpoint and run the same file again.
5. Diff the two receipts before anyone touches production.

python3 run_gates.py && cp .gates/receipt.json receipts/receipt-free.json 

Same gates, two environments, two hashed receipts. If the paid run is green and the free run was red, you just found a config drift for free instead of at 2am.

I would not call the free tier a production recommendation. I call it a rehearsal room. You cannot break anything that matters there.

The table I keep next to the file

Every row has a name. If a row has no name, delete the row. It was never a gate, it was a feeling.

Where this breaks, and who should skip it

Six gates is a solo-builder shape. It does not cover auth, data residency, or audit trails — do not take this to a compliance review and call it done.

Skip this approach if your deploy already runs a real policy engine. Skip it if you cannot name an owner per row, because the file will quietly become another document nobody runs. Skip it if rollback is genuinely impossible — fix that first, then come back.

Also, pin nothing about free tiers in a contract. Allowances and server terms move. Your gates should read the current number from config, not from a blog post. Mine does:

python3 run_gates.py --budget "${MONTHLY_TOKEN_BUDGET:-200000}" 

One flag today. A different number next month. No rewrite.

The question I actually want answered

Which row on your launch checklist has no owner's name on it? Paste that single row and what evidence it would produce. I want to see the one you would delete first — that tells me which gate to design next.