The Timeout Math That Breaks Your Retry Loop

Your client has a 10-second timeout and three retries. You expect a worst case of roughly 30 seconds. Your caller, who put a 30-second deadline on the whole operation, disagrees — and finds out when their request dies anyway.

The problem is that most retry loops treat timeouts as per-attempt values with no global budget. By the end of this article you'll be able to compute a real worst-case latency for a retried call, split a deadline across attempts, and propagate that deadline through your call stack.

Why per-attempt timeouts lie

A timeout on a single HTTP attempt bounds one connection. It says nothing about the total time of the operation. A retry loop that sleeps between attempts adds backoff on top, and any pre-request work (DNS, token refresh, payload serialization) adds more.

Worst case for "10s timeout, 3 retries, 1s backoff":

attempt 1: 10s + 1s backoff attempt 2: 10s + 1s backoff attempt 3: 10s total: 32s

That's before TLS handshakes, connection pool queuing, or a slow middleware layer. A caller with a 30-second deadline doesn't get a slow response — they get an exception from a layer that had no idea a budget existed.

Budget the deadline, not the attempt

Flip the model. Start from the total time you're willing to spend, then divide it across attempts. In Python, time.monotonic() gives you a clock that won't jump when NTP corrects the system time.

import time import httpx def get_with_budget(url: str, budget_s: float, max_attempts: int = 3): start = time.monotonic() per_attempt = budget_s / max_attempts for attempt in range(1, max_attempts + 1): remaining = budget_s - (time.monotonic() - start) if remaining <= 0: raise TimeoutError("deadline exhausted before final attempt") timeout = min(per_attempt, remaining) try: return httpx.get(url, timeout=timeout) except (httpx.TimeoutException, httpx.TransportError): if attempt == max_attempts: raise time.sleep(min(0.5 * 2 ** (attempt - 1), remaining / 2))

Three details matter here. The timeout for each attempt is capped by the remaining budget, so attempt three can never outlive the caller. The backoff sleep is also capped by the remaining budget, so you never wait two seconds when you have 800 milliseconds left. And a transport error (reset connection, DNS failure) retries too, because those are usually transient as well.

Propagate the deadline down the stack

A budget only works if every layer knows it. If your handler has 5 seconds and it calls three services, each service call should get roughly a third of what's left — not a hardcoded 5 seconds each.

class Deadline: def __init__(self, budget_s: float): self._expires = time.monotonic() + budget_s def remaining(self) -> float: return max(0.0, self._expires - time.monotonic()) def handler(deadline: Deadline): # each downstream call shares the same budget  user = fetch_user(deadline) # gets ~1.6s of a 5s budget  orders = fetch_orders(deadline) # gets whatever is left  return render(user, orders)

This is the same idea as gRPC deadline propagation and the Server-Timing-aware patterns in distributed tracing: the deadline travels with the request. If you're on gRPC already, use it natively — pass the deadline into the context rather than setting a fixed timeout per call.

Where this approach still fails

Budget division has real tradeoffs. Know them before you adopt it:

Tradeoff	What happens	Mitigation
Equal division starves the last call	A slow first call leaves the third service almost nothing	Give critical calls a reserved floor
Parallel calls can't share a budget	Three concurrent 1.6s calls each need 1.6s, not a third	Divide by fan-out, not by call count
Server work is invisible	Your budget expires mid-response; you paid for the work and can't use it	Budget only the wait, accept partial loss

The last row deserves emphasis. A timeout you enforce client-side doesn't cancel server-side work. If a call times out and you retry, you can stack duplicate work on the backend. Make retried writes idempotent — an Idempotency-Key header or a deterministic request ID — before you tune budgets.

Set sane defaults at the client level

Keep a default budget for calls where the caller didn't specify one, so an unconfigured code path can't hang forever:

DEFAULT_BUDGET_S = 10.0 def fetch(url: str, deadline: Deadline | None = None) -> httpx.Response: budget = deadline.remaining() if deadline else DEFAULT_BUDGET_S return get_with_budget(url, budget)

Pair this with httpx.Timeout(connect=2.0, read=None, ...) when you do configure per-phase values: connection establishment and response reads usually deserve different limits, and a single flat number hides that distinction.

Key takeaways

A per-attempt timeout bounds one connection, not the operation; the true worst case is attempts plus backoff plus overhead.
Compute each attempt's timeout from the remaining budget on time.monotonic(), and cap backoff sleeps the same way.
Propagate deadlines through your call stack so downstream calls share the budget instead of each carrying a hardcoded maximum.
Client-side timeouts don't cancel server work; make retried writes idempotent before tightening budgets.

Source

Stop Hardcoding Your Timeouts — I added working budget-splitting code, deadline propagation through a call stack, and the tradeoff and idempotency failure modes the original doesn't cover.