My Idempotency Key Was a Timestamp, So the Tool Ran Twice

Duplicate tool calls are a data-model problem, not a model problem. I learned that after one job created the same row twice. The fix was six lines and one test.

The symptom came back clean

A nightly agent run posted a "row created" webhook. The downstream counter showed two rows for one request. No error. No stack trace. Just double.

I checked the obvious first. Did my retry loop fire twice? No. Did the queue redeliver? Also no. So the duplicate came from a place I had not instrumented.

Sound familiar? Agents that call tools are just clients. And clients duplicate.

Step 1: Freeze the failure into a transcript

Stop debugging against a live endpoint. You cannot diff what you cannot replay.

I added one line to the request loop. Every model response gets appended to a JSONL file. That file became my test fixture.

# trace.py — append-only, one JSON object per line import json, time def record(path, event): event["ts"] = time.time() with open(path, "a") as f: f.write(json.dumps(event) + "\n") 

Log three things only: the tool name, the raw arguments, and the run id. Keep the record boring. Boring records survive refactors.

Step 2: Replay with no network

The replay harness is the real artifact here. It runs offline. No key, no cost, no flaky Wi-Fi.

# replay.py import json def load_events(path="fixtures/duplicate_tool_call.jsonl"): with open(path) as f: return [json.loads(line) for line in f if line.strip()] 

Now I could run the same recorded events a hundred times. Same input, same code path. Any duplicate left is mine.

That is the pivotal question. Is the repeat in the model output, or in my handler? Replay answers it. Reading logs does not.

Step 3: Print the keys, not the logs

I dumped every dedup key the handler computed. Two lines. Same tool. Different keys. That was the whole mystery.

Here is the key I had shipped:

# the bug, in one line def dedup_key(call, ts): return f"{call['name']}:{ts}" 

ts came from time.time(). Two calls meant two timestamps, two keys, two side effects. The guard was structural. The key was temporal. Useless.

Ask yourself what your key actually represents. If the answer is "when it happened," you have no key.

Step 4: Build the key from intent

A key must be a function of what you are about to do. Not when you do it. Not who asked. What.

# idempotency.py import hashlib, json def canonical_call(call): args = call["arguments"] if isinstance(args, str): args = json.loads(args) # tolerate string-encoded args  blob = json.dumps(args, sort_keys=True, separators=(",", ":")) return call["name"], blob def dedup_key(call, run_id): name, blob = canonical_call(call) return hashlib.sha256(f"{run_id}|{name}|{blob}".encode()).hexdigest()[:16] 

Three details carry more weight than they look.

1. sort_keys=True — key order is not part of intent.
2. separators=(",", ":") — whitespace is not part of intent.
3. run_id inside the hash — scope the guard to a run, not eternity.

That third one is a judgement call. Drop it and a legitimate repeat action tomorrow gets swallowed. Keep it and a runaway loop still passes. Choose your failure on purpose.

Step 5: Write the test that fails first

I wrote the assertions before the fix. They failed. That is the point of them.

# tests/test_idempotency.py from replay import load_events from idempotency import dedup_key def test_duplicate_call_applies_once(): events = load_events() ledger = set() applied = 0 for ev in events: if ev["type"] != "tool_call": continue k = dedup_key(ev["call"], ev["run_id"]) if k in ledger: continue ledger.add(k) applied += 1 assert applied == 1 

Run it both ways:

python -m pytest tests/test_idempotency.py -q git stash && python -m pytest tests/test_idempotency.py -q 

If the second command passes, your fixture is wrong, not your code. A regression test that has never failed has never protected you.

Step 6: Add a ceiling above the guard

Dedup handles repeats. It does not handle volume. A loop that mints fresh keys forever still drains your budget.

So I capped the run. Hard stop, logged loudly, no silent continuation.

MAX_TOOL_CALLS = 12 def guard(n, run_id): if n > MAX_TOOL_CALLS: raise RuntimeError(f"tool budget exceeded run={run_id} calls={n}") 

Cheap insurance. Set the number from your worst honest run, then double it. Why double? Because your worst honest run is still not your worst day.

Where the free tier earned its place

I replayed this loop against MonkeyCode's free model access and kept the harness on its free server option. The project currently advertises a free token allowance (10M at the time of writing) plus that free server.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Why does that matter for a debugging story? Because duplicates need repetition to reproduce. Free access let me replay the same prompt repeatedly without watching a meter.

My read: the repeat I saw was sampling variance, not a server fault. I could test that read cheaply. That is all I am claiming.

One caveat, plainly: free access has terms, limits, and availability that can change. Check the project page before you wire a pipeline to it.

Decision table

This is advice, not a measurement. Your numbers will differ from mine.

What I did not prove

I did not prove the model was wrong. I proved my handler had no key.

I did not measure how often duplicates occur. One transcript is not a rate.

And I did not fix the upstream. If the remote tool is not idempotent, a client guard only narrows the window. Send an idempotency header too, when the API accepts one.

Who should skip this

Read-only tools? Skip it. This is pure overhead.

One call per run? A run_id column in a table is enough.

Most importantly: skip it if you cannot express "the same action" in code. A fuzzy key that merges distinct intents is worse than no key. It deletes real work, silently, and no test will catch it for you.

Ask three questions before you ship

1. What is my key a function of?
2. Where is my ceiling?
3. Which test failed before the fix?

No answer to all three means you do not have idempotency. You have hope, wearing retries.

The harness is thirty lines. Write it before the second incident, not after.