Retries Count Twice: A Token Budget Ledger Your Team Can Actually Audit

An example scenario (illustrative, not a real team)

Picture a five-person team sharing one free model allowance and one small free server for agent jobs. Two people run nightly summarizers, one runs an interactive assistant, and the on-call rotation reruns failed jobs by hand when something looks stuck. Nobody owns the number, so the first time the allowance gets tight, everyone blames everyone else.

This post is that team's fix: a token budget ledger small enough to live in a wiki page, plus the code to feed it. The scenario is constructed, not observed. The artifacts below are illustrative and were not executed for this article, so treat the numbers as placeholders you replace with your own.

Why free-allowance accounting drifts

Most teams do not lack a usage dashboard; they lack an attribution rule. A dashboard tells you the total moved, while a ledger tells you which task moved it and whether that movement was expected. When those two disagree, the disagreement is usually structural rather than mysterious.

Read the table as a checklist, not a diagnosis. If two of these apply to you, the rest of this post is worth the twenty minutes.

Step 1: Fix the unit of account before you write any code

Decide what one row means. The cleanest definition is one row per attempt, not one row per task, because a task can legitimately cost three attempts and you want to see that. Anything you cannot attach to an attempt should not be in the ledger.

Write down three identifiers: the task, the attempt, and the lane. The lane is the consumer of the budget, which might be a person, a cron job, or a product surface, and it is the field every argument later depends on. Without it you can only report totals, and totals never end an argument.

Step 2: Put the ledger write at the call site

Disclosure: This article was prepared as part of MonkeyCode's product outreach. This account's operator reports that MonkeyCode offers free model access and a free server option, and quotes a free allowance of ten million tokens; I have not verified those terms myself, so confirm the current figures and conditions on the project's official pages before you plan capacity around them.

That caveat matters more than the number. Free allowances change, so build the ledger to read limits from configuration rather than hardcoding a total anywhere in the pipeline.

The wrapper below writes one row per attempt and leans on a unique index to make duplicate writes harmless. It is illustrative: it has not been run against any provider, and the token counts come from whatever response object you pass in.

# illustrative only: no execution, no results claimed import json, sqlite3, time, uuid SCHEMA = """ CREATE TABLE IF NOT EXISTS token_ledger ( row_id TEXT PRIMARY KEY, task_id TEXT NOT NULL, attempt INTEGER NOT NULL, lane TEXT NOT NULL, model_alias TEXT NOT NULL, tokens_estimated INTEGER NOT NULL, tokens_actual INTEGER, -- NULL until the provider reports status TEXT NOT NULL, -- running | settled | aborted started_at REAL NOT NULL, settled_at REAL, UNIQUE (task_id, attempt) ); """ def open_ledger(path="ledger.db"): conn = sqlite3.connect(path) conn.execute(SCHEMA) return conn def start_attempt(conn, task_id, attempt, lane, model_alias, tokens_estimated): row_id = str(uuid.uuid4()) try: conn.execute( "INSERT INTO token_ledger VALUES (?,?,?,?,?,?,?,?,?,?)", (row_id, task_id, attempt, lane, model_alias, tokens_estimated, None, "running", time.time(), None), ) conn.commit() return row_id except sqlite3.IntegrityError: # same (task_id, attempt) already recorded: return the existing row  cur = conn.execute( "SELECT row_id FROM token_ledger WHERE task_id=? AND attempt=?", (task_id, attempt), ) return cur.fetchone()[0] def settle_attempt(conn, row_id, tokens_actual, status="settled"): conn.execute( "UPDATE token_ledger SET tokens_actual=?, status=?, settled_at=? WHERE row_id=?", (tokens_actual, status, time.time(), row_id), ) conn.commit() 

Two design choices are doing the work here. The unique constraint turns a retry storm into one row, and the status column lets you separate "spent tokens" from "produced output" without inventing a fourth table. If your provider reports usage only at the end of a stream, an aborted stream stays running and you must decide explicitly whether to estimate it.

Step 3: Reconcile nightly against the provider total

The ledger is only trustworthy if it can disagree with the provider and survive the comparison. Run one job per night that sums settled rows per lane and compares that total to whatever usage number you can obtain. Store the delta; do not smooth it.

-- illustrative: daily reconciliation skeleton SELECT lane, SUM(COALESCE(tokens_actual, tokens_estimated)) AS ledger_tokens, COUNT(*) FILTER (WHERE status = 'aborted') AS aborted_attempts, COUNT(*) FILTER (WHERE tokens_actual IS NULL) AS unsettled_rows FROM token_ledger WHERE started_at >= :since GROUP BY lane ORDER BY ledger_tokens DESC; 

The unsettled_rows count is the honest one. A non-zero value means your ledger is incomplete, and any total you publish that week should carry that caveat in the same sentence. Teams get into trouble not by having gaps but by presenting a gappy total as if it were whole.

Step 4: Assign four roles and three handoffs

A ledger nobody owns becomes a spreadsheet nobody opens. Keep the role list short enough that names actually get filled in.

There are three handoffs worth naming explicitly. Lane Owners tell the Ledger Scribe about new jobs before those jobs ship, not after. The Scribe posts the weekly delta with the unsettled count attached. The Reviewer either accepts the delta or files one question per unexplained lane.

Step 5: Paste this into the wiki

TOKEN BUDGET LEDGER — one page Limit source: <config path / env var> Last verified: <date> Verified by: <name> Current reported allowance: <number> (operator-supplied; re-check before planning) Weekly run (15 min) 1. Run reconciliation for the last 7 days. 2. Copy the per-lane table into this page. Do not edit numbers by hand. 3. Record delta vs provider total, and the unsettled row count. 4. Flag any lane up >2x week over week without a change note. 5. Reviewer signs off in the run log below. Run log | Date | Scribe | Ledger total | Provider total | Delta | Unsettled | Flagged lanes | |---|---|---|---|---|---|---| Escalation - Delta > 5% for two weeks running -> Budget Owner opens an investigation. - A lane with no owner -> disable the job until an owner is named. 

Keep the page append-only for the run log. Editing history is how a ledger quietly becomes fiction.

How to check that your ledger is honest

Run these four checks before you trust a single weekly number, and treat a failure as a bug in the pipeline rather than a rounding problem.

1. Send twenty known requests through the wrapper and confirm exactly twenty rows exist.
2. Replay one request with the same (task_id, attempt) and confirm the row count does not change.
3. Force one abort mid-stream and confirm the row stays running rather than disappearing.
4. Compare one day of ledger totals against the provider's own usage figure and write the delta down, even if it is zero.

If you cannot obtain a provider-side usage figure at all, say so on the wiki page. An unaudited ledger is still useful for ranking lanes against each other, but it must never be quoted as an absolute spend.

Limitations and who should skip this

This approach breaks down when usage is reported only in aggregate and never per request, because then the ledger can rank lanes but never prove a total. It also adds a database write to every model call, which is the wrong trade for a solo developer running a handful of jobs a day.

Skip it if you have fewer than three people sharing one allowance, if you already have per-task billing from your provider, or if nobody will actually run the weekly fifteen minutes. A ledger that is set up and abandoned is worse than no ledger, because it looks like evidence.

If you want to try the shape of this workflow against a free allowance without committing a paid key, MonkeyCode publishes an open-source client you can run locally, and the project's own documentation is the place to confirm what the free model access and free server option currently include.