Retries Are Not Free: Five Myths, an Attempt Ledger, and a Stop Rule

A job failed at 2am. It passed on the third try. Everyone went back to sleep.

Two hours later, support found three identical orders. The retry worked. The side effects did not.

So what does for attempt in range(3) actually promise you? Nothing. It is a loop, not a contract.

This is the FAQ I keep needing when someone says "just add retries." Free model access and a free server make retries feel cheap. They are not cheap. They are invisible.

Myth 1: "A retry is the same request, sent again"

Test: send one write twice. Count the rows.

A retry is a new request with the same intent. The wire does not know your intent. Only you do.

Give the intent a name. Hash the payload. Store the hash next to the effect.

import hashlib, json def idem_key(payload: dict) -> str: blob = json.dumps(payload, sort_keys=True).encode() return hashlib.sha256(blob).hexdigest()[:32] 

Corrected model: a retry is safe only when the receiver can recognize a replay. Most model APIs will not dedupe for you. Your own store has to.

Myth 2: "Exponential backoff is polite"

Backoff without jitter is a synchronized herd. Twenty workers fail together. Twenty workers retry together.

Test: log attempt timestamps from 20 workers. Look at the gaps. They are identical.

Use full jitter. Pick a random point inside the window, not the window's edge.

sleep = min(cap, base * 2 ** (attempt - 1)) time.sleep(random.uniform(0, sleep)) 

The AWS Architecture Blog write-up on backoff and jitter is still the clearest reference. Read it before you copy a snippet from a gist.

Corrected model: backoff controls your rate. Jitter controls correlation. You need both.

Myth 3: "More retries means more reliability"

Count the multiplier. Three layers with four attempts each. One user click becomes 64 calls.

That is retry amplification. Google's SRE book covers the pattern in Addressing Cascading Failures. A struggling dependency gets hammered hardest exactly when it can least take it.

Set a stop rule with three parts:

1. a deadline, in wall-clock seconds;
2. a hard attempt cap;
3. a retry budget: attempts per successful call, averaged over a window.

The deadline is the part people skip. Attempt caps do not help when each attempt hangs for 30 seconds.

Corrected model: retries are a load amplifier. Budget them like you budget requests.

Myth 4: "If it failed, retry it"

Not every error is retryable. Retrying a 422 just burns your window.

Honor Retry-After when the server sends it. RFC 9110 defines the header. Guessing a backoff while the server hands you a number is just noise.

Corrected model: classify first, retry second. A retry loop without a classifier is a random number generator with extra steps.

Myth 5: "It succeeded, so the pipeline is healthy"

A third-attempt success is a yellow flag. Nobody sees it, because you log outcomes, not attempts.

Test: plot attempts per call over one day. If p95 attempts sits above 1, you are paying a retry tax.

Alert on the ratio, not just on failures. A pipeline at 1.4 attempts per call is failing a third of the time and hiding it.

Corrected model: success rate is a vanity metric here. Attempt rate is the health signal.

The artifact: an attempt ledger

Here is the wrapper I use. It writes one JSON line per attempt, so the retry history outlives the process.

# attempt_ledger.py import hashlib, json, random, time import urllib.error, urllib.request RETRYABLE = {408, 429, 500, 502, 503, 504} def idem_key(payload: dict) -> str: blob = json.dumps(payload, sort_keys=True).encode() return hashlib.sha256(blob).hexdigest()[:32] def call_once(url, payload, key, timeout=10): req = urllib.request.Request( url, data=json.dumps(payload).encode(), headers={"Content-Type": "application/json", "Idempotency-Key": key}, method="POST", ) with urllib.request.urlopen(req, timeout=timeout) as r: return r.status, r.read().decode() def call_with_ledger(url, payload, ledger_path="attempts.jsonl", deadline_s=20, max_attempts=4, base=0.5, cap=8.0): key = idem_key(payload) t0 = time.monotonic() with open(ledger_path, "a") as ledger: for attempt in range(1, max_attempts + 1): start = time.monotonic() status, body, err = None, None, None try: status, body = call_once(url, payload, key) except urllib.error.HTTPError as e: status = e.code except Exception as e: err = type(e).__name__ ledger.write(json.dumps({ "key": key, "attempt": attempt, "status": status, "error": err, "elapsed_s": round(time.monotonic() - start, 3), "total_s": round(time.monotonic() - t0, 3), }) + "\n") ledger.flush() if status and 200 <= status < 300: return status, body retryable = err is not None or status in RETRYABLE remaining = deadline_s - (time.monotonic() - t0) if not retryable or attempt == max_attempts or remaining <= 0: return status, None sleep = min(cap, base * 2 ** (attempt - 1)) time.sleep(random.uniform(0, min(sleep, remaining))) return None, None 

Reproduce it in five minutes

You need a server that fails on purpose. A sixty percent failure rate is enough.

# flaky_server.py import json, random from http.server import BaseHTTPRequestHandler, HTTPServer SEEN, WRITES = {}, [] class H(BaseHTTPRequestHandler): def do_POST(self): n = int(self.headers.get("Content-Length", 0)) payload = json.loads(self.rfile.read(n) or b"{}") key = self.headers.get("Idempotency-Key", "none") if random.random() < 0.6: self.send_response(503) self.end_headers() self.wfile.write(b"busy") return resp = SEEN.setdefault(key, {"order_id": 1000 + len(SEEN)}) WRITES.append(key) # count every write, replays included  body = json.dumps(resp).encode() self.send_response(200) self.send_header("Content-Type", "application/json") self.end_headers() self.wfile.write(body) def log_message(self, *a): pass HTTPServer(("127.0.0.1", 8080), H).serve_forever() 

Run the two pieces and read the ledger.

python flaky_server.py & python -c " from attempt_ledger import call_with_ledger print(call_with_ledger('http://127.0.0.1:8080/orders', {'item': 'book'})) " python -c " import json, collections rows = [json.loads(l) for l in open('attempts.jsonl')] print('attempts:', len(rows)) print('statuses:', collections.Counter(r['status'] for r in rows)) print('max total_s:', max(r['total_s'] for r in rows)) " 

Now delete the setdefault dedupe and run it again. len(WRITES) will exceed the number of unique keys. That gap is your duplicate side effects. The return value never shows it.

Reading the ledger

• One key, four attempts, one 200. Your loop works. Your upstream is sick.
• One key, four attempts, no 200. The stop rule fired. Compare total_s to deadline_s.
• Many keys, one attempt each, all 503. That is an outage, not a retry problem.
• One key, two 200s. Your dedupe is missing. Fix that before you add attempts.

Do you even need this?

Limitations, and who should skip this

• This wrapper is not atomic. A crash between the effect and the ledger write still loses a line.
• A payload hash is not a business key. Two legitimately different orders can share a body.
• If your SDK already retries, wrapping it in this loop multiplies attempts. Pick one layer.
• If the receiver cannot dedupe, the key is decoration. Fix the receiver first.
• Do not use a hand-rolled loop for streaming or long-lived connections. Different failure model.
• Retrying an agent step re-runs its tool calls. That is the expensive part, not the model call.

Why I ran it on free access

I wanted this ledger running for an hour without watching a bill. MonkeyCode's free model access and free server option gave me that, so the model-shaped call in my run used a free endpoint while the flaky stub stayed local.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

One honest caveat: free tier behavior is not paid tier behavior. Latency and limits differ. Measure on the tier you actually ship on.

If you want to watch a retry ledger fill up before you trust your own, the free tier is a fine place to start.