Example scenario, illustrative and not a record of a real project: a solo founder edits an agent loop ten times in one afternoon. Each edit re-fires roughly forty model calls. The free token pool drains before the feature works once.

Looping agents are the current fashion, and loops multiply calls. Everyone writes about orchestration and speed. Fewer people write about the boring part: the same request going out ninety times.

That repetition is the cheapest thing to remove. A replay layer stores each response on disk under a hash of its request. The second identical request never leaves the machine.

The cost model shifts from tokens to kilobytes. Disk is the resource an indie project already owns.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. The operator describes MonkeyCode as an open source project with free model access and a free server option, and mentions a free allowance in the millions of tokens. Treat those terms as operator-supplied and check the project's own repository and pricing page for current limits before you plan around them. Quotas on free offerings move, and this article does not verify them.

The store is a directory of answers

The whole mechanism is a dict of request hash to response body. No database, no service, no schema migration. The code below is illustrative and was not executed by the author.

# cassette.py — illustrative; not executed, not tested import hashlib, json, os, tempfile from pathlib import Path STORE = Path(os.environ.get("CASSETTE_DIR", "./cassettes")) MODE = os.environ.get("CASSETTE_MODE", "replay") # replay | record BUDGET = int(os.environ.get("CASSETTE_LIVE_BUDGET", "0")) VOLATILE = {"request_id", "created", "created_at", "trace_id"} def canonical(payload: dict) -> str: clean = {k: v for k, v in payload.items() if k not in VOLATILE} blob = json.dumps(clean, sort_keys=True, separators=(",", ":")) return hashlib.sha256(blob.encode("utf-8")).hexdigest() def load(key: str): path = STORE / f"{key}.json" return json.loads(path.read_text(encoding="utf-8")) if path.exists() else None def save(key: str, response: dict) -> None: STORE.mkdir(parents=True, exist_ok=True) fd, tmp = tempfile.mkstemp(dir=STORE, suffix=".tmp") with os.fdopen(fd, "w", encoding="utf-8") as fh: json.dump(response, fh) fh.flush() os.fsync(fh.fileno()) os.replace(tmp, STORE / f"{key}.json") class BudgetExhausted(RuntimeError): pass def resolve(payload: dict, call_upstream, spent: list) -> dict: key = canonical(payload) hit = load(key) if hit is not None and MODE == "replay": return hit if spent[0] >= BUDGET: raise BudgetExhausted(f"miss key={key[:12]} budget={BUDGET}") response = call_upstream(payload) spent[0] += 1 save(key, response) return response

Two details carry the safety. The volatile field strip keeps request metadata from poisoning the hash. The temp-file plus os.replace write keeps a killed process from leaving half a JSON file.

Canonical keys decide whether replay is honest

Sort keys and drop timestamps, and the hash becomes stable across runs. Leave sampling in the payload, and the hash tells the truth about it. A call with temperature: 0.9 is not the same call twice.

Use temperature zero, or an explicit seed, for anything you intend to replay. Otherwise the cache is a quiet lie about determinism. The replay returns one valid sample and hides the variance you were measuring.

# loop.py — illustrative wiring, not executed from cassette import resolve spent = [] def call_upstream(payload): return transport.complete(payload) # your own client; not shown  payload = {"model": MODEL, "messages": messages, "temperature": 0} reply = resolve(payload, call_upstream, spent)

The spent list is a placeholder counter. Replace it with a small class or a row in a local SQLite file once two processes share the store. A plain list is not concurrency-safe, and pretending otherwise causes duplicate spend.

Treat one writer as the default. If the loop and a web handler both write, wrap save in a file lock, or give each writer a separate cassette directory and merge later.

Streams and side effects break the pattern

Replay works for whole responses. Streaming responses arrive in fragments, so a crash mid-stream leaves a partial answer. Either buffer the stream before caching, or skip caching for streamed calls.

Tool calls deserve more care. Cache the model's decision, never the tool's effect. Re-executing a cached "issue refund" decision is not idempotent by default.

Replay the plan, then run tools against a test double. Keep the boundary explicit in code, and a stale cassette cannot move money twice.

Where the free server fits

A shared cassette store is more useful than a local one. Put the proxy on a small host, and the laptop, a second machine, and CI all hit the same answers. The operator lists a free server option as part of MonkeyCode, which makes it a natural candidate for that host. Verify three things first: outbound access to the model endpoint, disk persistence across restarts, and whether long-running processes are permitted. Do not assume any of the three.

If the host has no durable disk, replay resets on every deploy. That is still fine for a single afternoon of iteration, and useless for CI.

A check plan you can run yourself

These commands are a plan, not a recorded result. The expected counts are what the plan predicts, not what the author observed.

# illustrative check plan — not executed export CASSETTE_MODE=record CASSETTE_LIVE_BUDGET=200 python loop.py > /dev/null ls cassettes | wc -l # note the count, call it N export CASSETTE_MODE=replay CASSETTE_LIVE_BUDGET=0 python loop.py > /dev/null ls cassettes | wc -l # expect exactly N again

The second run is the real test. A budget of zero means any miss raises BudgetExhausted instead of spending silently. A growing store on the replay pass proves your keys are unstable.

Change one character in a prompt and rerun. The store should gain exactly one file. If it gains forty, normalization is wrong.

Who should skip this

Skip replay if you are measuring model variance, because one cached sample destroys the experiment. Skip it for prompts containing customer data you would not write to disk unencrypted. Skip it if your product's value is fresh sampling on every request.

Solo founders with narrow scope get the most from it. One codebase, one operator, one cassette directory. Delete the directory and you are back to plain calls, with nothing to migrate.

Start with a loop you already run twice a day. Ten minutes on the cache saves the pool you would have spent learning nothing new.