Cost per Green Run: Seven Free-Tier Myths and a Ledger You Can Run

Free tokens feel like permission. You stop counting requests and start counting ideas. Then a one-attempt task turns into five attempts. Where did the budget go?

I keep running into the same seven claims about free model access and free servers. Each one sounds reasonable. Each one hides a measurement. So here is a myth-busting FAQ, plus a ledger you can actually run.

Myth 1: "Free tokens mean free iteration"

The claim. If tokens cost nothing, retries cost nothing either.

What to measure. Log attempts, not requests. Count every retry as a line.

Corrected model. Free access removes the invoice, not the cost. Wall clock still ticks. Quotas still burn. Your reviewers still wait. A free tier changes who pays, not whether something is spent.

Ask yourself: how many attempts did your last "cheap" task take?

Myth 2: "Cost per request is the number that matters"

The claim. Average spend per call tells you if you are efficient.

What to measure. Cost per green run. A green run is a task that ends in a state you accept. Everything else is inventory you throw away.

Percentiles lie. Averages lie harder. A workflow that burns 60% of its tokens before the final retry is not cheap. It is expensive with good PR.

Corrected model. Divide all spend by shipped tasks. That number survives contact with reality.

Myth 3: "A cheaper model is a cheaper task"

The claim. Lower price per token equals lower cost per result.

What to measure. Price × attempts × pass rate on your own task set. Not on a leaderboard. On your inputs.

A model that costs half as much and needs three attempts is not half as expensive. It is one and a half times more expensive, plus latency. Do the arithmetic before the migration, not after.

Myth 4: "The provider dashboard is your source of truth"

The claim. The usage page already tells you where tokens went.

What to measure. Attribution. Dashboards aggregate by day or key. They rarely aggregate by task.

Corrected model. Your run log is the source of attribution. The dashboard is a reconciliation check. If the two disagree by more than rounding, one of them is wrong, and it is usually your log.

Myth 5: "A free server is a free build box"

The claim. Move the agent loop onto the free box and forget it.

What to measure. Exit codes and wall clock per run. Interruptions do not announce themselves politely. They show up as a non-zero code or a stalled timestamp.

Corrected model. Unknown is not unlimited. Treat every limit as an open question until a probe answers it. Then write the answer next to the run command.

This is where a project like MonkeyCode is relevant to the workflow. It is an open-source project offering free model access and a free server option. That combination is exactly why a ledger matters: when the compute is free, the only remaining budget is time and retries.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Myth 6: "Retries are noise"

The claim. A few extra attempts do not move the numbers.

What to measure. Failure tax. Divide tokens spent on red runs by tokens spent on green runs.

I call it a tax because it behaves like one. It is invisible per request. It compounds per release. Once you see the ratio, retry policy stops being a vibe.

Corrected model. Retries are the second-largest line item in most agent workflows. They are just never printed.

Myth 7: "The demo works, so production can move to the free tier"

The claim. A working demo is evidence of capacity.

What to measure. Continuity under load, and what happens when a limit changes. Neither is visible in a demo.

Corrected model. Free tiers are excellent for iteration and for teaching. They are a poor substitute for a contract. If you need paging, SLAs, or audit trails, buy them.

The artifact: a cost-per-green-run ledger

One line per attempt. That is the whole design. Here is a stdlib-only script I would run on any JSONL attempt log.

#!/usr/bin/env python3 """ledger.py - cost per green run from an attempt log (JSONL). Record schema (one line per attempt): {"task": str, "attempt": int, "prompt_tokens": int, "completion_tokens": int, "ok": bool, "wall_ms": int} """ import json import sys from collections import defaultdict def load(path): rows = [] with open(path, encoding="utf-8") as fh: for line_no, line in enumerate(fh, 1): line = line.strip() if not line: continue try: rows.append(json.loads(line)) except json.JSONDecodeError as exc: print(f"skip line {line_no}: {exc}", file=sys.stderr) return rows def billable(row): return int(row.get("prompt_tokens", 0)) + int(row.get("completion_tokens", 0)) def main(path): tasks = defaultdict(list) for row in load(path): tasks[row["task"]].append(row) green, red = [], [] for task, attempts in tasks.items(): attempts.sort(key=lambda r: r["attempt"]) tokens = sum(billable(r) for r in attempts) item = { "task": task, "tokens": tokens, "attempts": len(attempts), "wall_ms": sum(int(r.get("wall_ms", 0)) for r in attempts), } (green if attempts[-1].get("ok") else red).append(item) spent_green = sum(i["tokens"] for i in green) spent_red = sum(i["tokens"] for i in red) n_green = len(green) print(f"tasks: {len(tasks)} green: {n_green} red: {len(red)}") print(f"tokens on green runs: {spent_green}") print(f"tokens on red runs: {spent_red}") if n_green: print(f"all-in per green run: {(spent_green + spent_red) / n_green:.0f} tokens") if spent_green: print(f"failure tax: {spent_red / spent_green:.2f}x") extra = sum(i["attempts"] - 1 for i in green + red) print(f"extra attempts: {extra}") if __name__ == "__main__": if len(sys.argv) != 2: raise SystemExit("usage: ledger.py attempts.jsonl") main(sys.argv[1]) 

You still need to write the log. Here is the wrapper shape. It is a template, not a library, and I have not run it against your SDK.

import json import pathlib import time LOG = pathlib.Path("attempts.jsonl") def record(task, attempt, usage, ok, started, path=LOG): row = { "task": task, "attempt": attempt, "prompt_tokens": usage["prompt_tokens"], "completion_tokens": usage["completion_tokens"], "ok": bool(ok), "wall_ms": int((time.perf_counter() - started) * 1000), } with path.open("a", encoding="utf-8") as fh: fh.write(json.dumps(row) + "\n") 

Three rules keep the log honest. One: the caller decides ok, never the model. Two: append-only, never edit history. Three: one task id, one file line per attempt.

Run it

python ledger.py attempts.jsonl 

Illustrative output shape only, your numbers will differ:

tasks: 12 green: 9 red: 3 tokens on green runs: 41200 tokens on red runs: 11800 all-in per green run: 5889 tokens failure tax: 0.29x 

That last line is the interesting one. 0.29x means roughly 22 cents of every token dollar bought nothing you kept.

The server probe

Before I trust any box, free or not, I measure interruptions. Twenty runs, one CSV.

#!/usr/bin/env bash set -u : > probe.csv for i in $(seq 1 20); do start=$(date +%s%N) timeout 300 python agent.py --task "t$i" >/dev/null 2>&1 code=$? end=$(date +%s%N) echo "$i,$code,$(( (end - start) / 1000000 ))" >> probe.csv done awk -F, '$2 != 0 {print "nonzero exit on task", $1, "code", $2}' probe.csv 

One portability note. date +%s%N is GNU. On macOS use gdate or time the loop from Python instead.

Decision table

Limitations, spelled out

• The ledger counts tokens, not currency. Convert with your own rate card.
• It trusts your ok flag. A sloppy gate produces flattering numbers.
• It ignores provider-side rounding and reserved context.
• The probe measures one binary on one box for twenty runs. That is a sample, not a guarantee.
• Log discipline decays. If nobody reads the file, it silently rots.

Who should skip this

If you are on call, skip the free tier for anything that pages you. If your data has contractual handling rules, skip it too. And if you cannot tolerate a quota changing without notice, this workflow is not for you. The ledger will tell you the truth, but it cannot buy you a guarantee.

The corrected mental model

Free access is a testing environment for your process, not a discount on your product. Count attempts. Attribute tokens. Probe the box. Print the failure tax where your team can see it.

If you want to try this without a card, MonkeyCode's free model access and free server option are a low-cost place to start. Run the ledger before you trust it. That is the whole point of the FAQ.