LLM API rate limits: not every 429 should be retried

Originally published on Extenify.

LLM API rate limits usually show up the week after launch, not during testing: a burst of users hits your backend, OpenAI, Claude or Gemini answers with HTTP 429, and your UI spins forever. The surprising part is that many 429s are not rate limits at all, and retrying them only burns time. Here is how to tell the difference, retry the ones that can succeed, and stop most of them from happening.

A 429 can mean "slow down" or "pay up"

All three providers reuse the same status code for two very different problems:

• OpenAI sends 429 for traffic ("Rate limit reached", slow_down) and also for billing codes such as credit_balance_exhausted, organization_spend_limit_exceeded and project_spend_limit_exceeded. Check error.code, because error.type can still say insufficient_quota.
• Anthropic (Claude) returns a 429 rate_limit_error when an organization reaches its tier's monthly spend cap, but with no retry-after header and error.details.error_code set to enforced_spend_limit_reached.
• Gemini answers spend-based limits on paid tiers with 429 RESOURCE_EXHAUSTED, the same status it tells you to retry for traffic, so check the message and your billing page.

A rule of thumb that holds up well: a 429 with retry-after is a rate limit; a 429 without one is suspect. Log the body and classify it before you retry anything.

How the limits are actually counted

Each provider measures several things at once and blocks you on whichever runs out first: requests and tokens per minute for all three, plus per-day quotas at OpenAI and Gemini, and separate input/output token limits at Anthropic.

Three details catch people out:

1. Short windows. Anthropic notes that 60 RPM may be enforced as 1 request per second, so 20 parallel calls can fail while your per-minute average looks fine.
2. Failed calls count. OpenAI says unsuccessful requests count toward your per-minute limit, so a tight retry loop keeps you blocked.
3. More keys don't help. Gemini limits are per project, not per API key, and OpenAI and Anthropic limit at the organization or project level.

One retry loop, not two

Both OpenAI's and Anthropic's Node SDKs already retry 429 and 5xx responses twice by default. If you wrap them in your own loop without setting maxRetries: 0, three of your attempts quietly become nine requests.

If you own the loop, keep it small and opinionated. A sketch:

const RETRY = new Set([408, 429, 500, 502, 503, 504, 529]); const BILLING = /credit|spend_limit|usage_limit|insufficient_quota/; async function callModel(send, { tries = 4, budgetMs = 30_000 } = {}) { const t0 = Date.now(); for (let i = 0; ; i++) { const res = await send(); if (res.ok) return res; const err = (await res.clone().json().catch(() => ({}))).error ?? {}; const code = err.code ?? err.details?.error_code ?? ''; if (BILLING.test(code)) throw new Error(`billing: ${code}`); // page someone const hint = Number(res.headers.get('retry-after')) * 1000 || 0; const wait = Math.max(hint, 500 * 2 ** i) + Math.random() * 250; if (!RETRY.has(res.status) || i + 1 >= tries || Date.now() - t0 + wait > budgetMs) { return res; } await new Promise((r) => setTimeout(r, wait)); } } 

The total time budget matters more than the attempt count for anything a user is waiting on. Fail fast with a clear message rather than holding a popup open for two minutes.

Prevent 429s instead of reacting to them

If you see 429s every day, retries are the wrong fix. Pace the traffic:

• Cap concurrency. Most small-backend 429s come from bursts, like a list of 30 items that each trigger a summary. A queue that allows 4 calls in flight removes most of them. With several instances, keep the counter in Redis or route calls through one worker.
• Size max_tokens honestly on OpenAI. It counts a request as the larger of max_tokens and the estimated tokens, so a generous default reserves capacity you never use.
• Cache long prompts on Claude. For most Claude models only uncached input tokens count toward the input-token limit, so prompt caching raises throughput as well as cutting cost.
• Push background jobs to batch APIs. OpenAI's Batch API and Anthropic's Message Batches API have their own limits, which leaves the per-minute budget for users who are waiting.
• Ramp launches gradually. OpenAI's slow_down and Anthropic's acceleration limits trigger on sharp traffic jumps even below your nominal limit.

Per-user quotas for extensions and multi-tenant apps

Provider limits apply to your whole organization, so in a browser extension with thousands of installs one heavy user can cause 429s for everyone. Put your own daily quota in front of the provider, keyed by an identity your server verified (never a client-generated ID, since extension code is readable and editable).

Then return different messages: your own 429 with retry-after for "daily limit reached, resets at midnight UTC", and a 503 for "busy right now, try again in a minute". Users accept a clear limit far more easily than an endless spinner, and you'll see that in your reviews.

For monitoring, log the remaining-requests and remaining-tokens headers on every response, alert when they stay under about 10% for several minutes, and count traffic 429s, billing errors and overloads separately. One billing error deserves an immediate alert; a few traffic 429s an hour don't.

Key takeaways

• Classify every 429 first. Billing and spend-cap 429s never succeed on retry.
• Honor retry-after, add jitter, cap attempts and total wait time.
• Run exactly one retry loop. Set maxRetries: 0 on the SDK if you write your own.
• Prevent bursts with a concurrency cap, realistic max_tokens, prompt caching and batch APIs.
• In multi-user products, add a per-user quota tied to a verified sign-in.

The original post has the header parser, a concurrency limiter and a Redis-style quota in full: read the complete guide to handling LLM API rate limits and 429 errors in Node.js. If you ship an AI-powered extension, Extenify's review and rating monitoring for browser extensions can flag the "doesn't work anymore" reviews that rate limits tend to cause.

What was the first 429 that bit you in production: real traffic, or a billing limit pretending to be one?