One unthrottled endpoint is all it takes to turn a production-ready backend into an outage report. Whether it is a rogue script written by a third-party developer, an aggressive web scraper, or a compromised API key, serving an unlimited number of requests per second per client will eventually break your database connection pool or starve server memory.

Rate limiting is not just a security patch to stop Denial of Service (DoS) attacks. It is essential backend infrastructure for cost control, system stability, and resource fairness across your application's user base.

Here is how to design and build rate limiting that protects your systems without hurting legitimate users.

Choose the Right Algorithm for Your Traffic

Before writing code, match your rate-limiting strategy to your application's workload:

  1. Fixed Window Counter: Tracks requests across fixed time frames (e.g., 100 requests per minute starting at top of the minute). It is fast to implement using Redis atomic counters, but it suffers from edge bursts: a client can exhaust double their limit by placing requests right at the window boundary.
  2. Sliding Window Counter: Smooths out fixed-window boundaries by calculating a weighted average of requests in the current and previous windows. This is the best default choice for standard production REST and GraphQL APIs.
  3. Token Bucket: Adds tokens to a bucket at a constant rate. Requests consume tokens. This allows brief traffic bursts when tokens accumulate, making it suitable for APIs where users upload batch workloads.

Step 1: Implement Tiered Rate Limiting

Never apply a global rate limit across every caller. Anonymous requests, basic users, and enterprise clients require different throughput ceilings.

Here is a practical Express.js middleware backed by Redis that enforces tiered rate limits while returning standard response headers:

import Redis from 'ioredis'; const redis = new Redis(process.env.REDIS_URL); const TIER_LIMITS = { anonymous: { requests: 30, windowInSeconds: 60 }, free: { requests: 100, windowInSeconds: 60 }, enterprise: { requests: 2000, windowInSeconds: 60 }, }; export async function rateLimiter(req, res, next) { const userTier = req.user?.tier || 'anonymous'; const identifier = req.user?.id || req.ip; const config = TIER_LIMITS[userTier]; const key = `ratelimit:${userTier}:${identifier}`; try { const requests = await redis.incr(key); // Set expiration on the first request in the window if (requests === 1) { await redis.expire(key, config.windowInSeconds); } const ttl = await redis.ttl(key); // Standard HTTP Rate Limit Headers res.setHeader('X-RateLimit-Limit', config.requests); res.setHeader('X-RateLimit-Remaining', Math.max(0, config.requests - requests)); res.setHeader('X-RateLimit-Reset', Date.now() + (ttl * 1000)); if (requests > config.requests) { res.setHeader('Retry-After', ttl); return res.status(429).json({ error: 'Too Many Requests', message: `Quota exceeded for ${userTier} tier. Try again in ${ttl} seconds.` }); } next(); } catch (err) { // Fail-open strategy: Keep the API available if cache fails console.error('Rate limiter store error:', err); next(); } } 

Step 2: Handle Gateway-Level Protection

Running all rate-limiting logic inside your application layer can still consume substantial CPU and memory during a severe attack.

For high-throughput systems, split your protection into two layers:

  • Upstream Gateway / Reverse Proxy (Nginx, Kong, Cloudflare): Enforce strict IP-based rate limits to drop junk traffic at the network edge before it hits your server scripts.
  • Application Middleware: Enforce user-based, tenant-based, or route-specific rate limits using business logic inside your app layer.

Here is an example Nginx configuration to drop aggressive IP traffic upstream:

# Limit IP addresses to 10 requests per second with a burst buffer of 20 limit_req_zone $binary_remote_addr zone=ip_limit:10m rate=10r/s; server { location /api/ { limit_req zone=ip_limit burst=20 nodelay; proxy_pass http://backend_upstream; } } 

Step 3: Fail Gracefully and Fail-Open

A rate-limited client should receive clear communication. Always return HTTP status code 429 Too Many Requests alongside a Retry-After header specifying how long to wait before attempting another request.

Additionally, adopt a fail-open design for your rate-limiting storage layer. If Redis goes down or experiences a cluster network partition, your rate limiter should log the error and allow traffic through (next()) rather than throwing a 500 Internal Server Error and taking down your entire API for legitimate traffic.

Step 4: Monitor and Tune Continuously

Rate limits require ongoing observation:

  • Track 429 response spikes: If legitimate users constantly hit rate limits, your thresholds are too low or your frontend client is missing request-deboucing logic.
  • Monitor Redis latency: Ensure your rate-limiting storage lookups add less than 2-3 milliseconds of overhead per request.

About me

Tanmay builds AI agents and automation for businesses and is open to freelance projects. Feel free to connect on LinkedIn.