Picture the log of a public API looking something like this:
401 invalid api key
401 invalid api key
401 invalid api key(tens of thousands of lines like this, from a handful of IPs)
At first glance it looks harmless. Not one of those requests got through. But ask one more question: what did the system have to do before it could answer 401? Usually, look the key up in the database. Every single time. A bot fires ten thousand requests, the database runs ten thousand queries, and the answer is always "not found."
Now flip it around. Say the key you use to sign tokens leaks. The attacker's requests look perfectly valid and pass every check. The problem is no longer requests that fail. It's requests that pass when they shouldn't.
Which raises a few questions:
- Can we bounce garbage keys without asking the database? (API key checksums)
- Can we verify tokens locally and revoke a leaked key all at once? (JWKS and key rotation)
- How fast is "kicking them out," really, and what won't it stop?
In the previous article, we looked at why weak passwords are so easy to crack. This time we move to the builder's side: what the front door of an API should be able to decide on its own before it bothers the database. (Fair warning: this one is technical, with code.)
Why does a garbage key still hit the database?
Most API keys are just random strings, like a8f3k2.... The only way to know if one is valid is to look it up. Not found, reject.
That's fine until a bot sends tens of thousands of random keys a minute. Each one is a lookup. Even with a cache in front, keys that don't exist usually miss the cache and fall straight through to the database.
A very clear example comes from something ordinary: the check digit.
Credit card numbers, and Thai national ID numbers, end with a digit calculated from the others. Mistype one digit and the system knows immediately the number is invalid, without looking anything up.
API keys can do the same. Split the key into three parts:
demo_Oc2SF0n3iNUzTBJX466NZoXBcxzv1f4SMxKL └─┬─┘└──────────────┬─────────────┘└──┬──┘ prefix 30 random chars 6-char checksum The best-known example is GitHub. In 2021 it moved to tokens with a prefix like ghp_, a random body, and a 32-bit CRC32 checksum encoded as six Base62 characters. GitHub's team wrote plainly that this lets them reject fake tokens without hitting the database, and cuts false positives when scanning code for leaked secrets.
What does a checksum look like in code?
A trimmed-down Go example. I used the prefix demo_ so it doesn't collide with anyone's real one.
const ( prefix = "demo_" bodyLen = 30 sumLen = 6 base62 = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz" ) func encodeBase62(n uint32, width int) string { b := make([]byte, width) for i := width - 1; i >= 0; i-- { b[i] = base62[n%62] n /= 62 } return string(b) } // First gate: check format and checksum in memory. No database. func LooksValid(key string) bool { if len(key) != len(prefix)+bodyLen+sumLen || !strings.HasPrefix(key, prefix) { return false } body := key[len(prefix) : len(prefix)+bodyLen] sum := key[len(prefix)+bodyLen:] return encodeBase62(crc32.ChecksumIEEE([]byte(body)), sumLen) == sum } Generating a key uses the same formula: fill the body with crypto/rand, then append the checksum. (Six Base62 characters hold about 56 billion values, more than CRC32's maximum of about 4.3 billion, so it fits.)
But the checksum is only the first gate. The order I like at the gateway:
request arrives with an API key │ ▼ 1. correct format? (length, prefix, alphabet) ──► no: reject │ ▼ 2. checksum matches? ────────────────────────────► no: reject │ (gates 1 and 2 run entirely in memory) ▼ 3. hash the key (e.g. SHA-256), look it up in cache or database │ ▼ 4. repeated failures from the same IP or client ──► rate limit Step 3 hashes first because you shouldn't store raw keys anyway. A randomly guessed key will almost never make it that far.
More on checksums:
A checksum only protects against typos or randomly guessed keys. If the key passes the checksum validation, it will proceed to Layer 3 (key hashing) and Layer 4 (rate limiting) instead.
If you want a tighter first gate, use an HMAC with a server-side secret instead of CRC32. Outsiders can't forge a well-formed key without the secret. The trade-off: external secret scanners can no longer validate the checksum themselves, and you now have one more secret to protect and rotate. Which is better depends on what you're willing to give up.
What are JWT and JWKS, and how are they different?
The names are close enough that people mix them up all the time. Let's take them one at a time.
A JWT (JSON Web Token) is the token a user gets after signing in and then sends with every request to prove who they are. It carries user claims plus a digital signature. Whoever issues it (say, an auth server) signs with a private key; the API on the other end verifies with the public key.
A JWT is a long string in three parts separated by dots. Decoded, it looks roughly like this:
header.payload.signature header {"alg":"ES256","kid":"key-2026-10"} ◄── how it was signed, with which key payload {"sub":"user-1", ◄── who the user is "iss":"https://auth.example.com", ◄── who issued it "aud":"orders-api", ◄── which API it's for "exp":1794836000} ◄── when it expires signature a signature over the first two parts, made with the private key One common misunderstanding: the payload isn't encrypted. It's only encoded (Base64URL), so anyone holding the token can read it. The signature stops tampering; it doesn't hide anything. Never put secrets in a JWT.
A JWKS (JSON Web Key Set) is a JSON file the auth server publishes. It lists the public keys used to verify JWT signatures and contains no user data at all. It usually lives at /.well-known/jwks.json, and each key has a name, its kid (key ID).
{ "keys": [ { "kid": "key-2026-10", "kty": "EC", "crv": "P-256", "alg": "ES256", "use": "sig", "x": "...", "y": "..." }, { "kid": "key-2026-07", "kty": "EC", "crv": "P-256", "alg": "ES256", "use": "sig", "x": "...", "y": "..." } ] } The difference in short:
| JWT | JWKS | |
|---|---|---|
| What it is | A signed token that proves identity | A list of public keys that verify signatures |
| Who issues it | The auth server, one per user | The auth server, one set for every API |
| Who holds it | The user or app, sent with every request | Each API, kept in its cache |
| What's inside | User claims + a signature | Public keys + kid, no user data |
| When it changes | On every sign-in or refresh | When keys are rotated |
They work as a pair. Every JWT says in its header which kid signed it, and the API picks that public key from the cached JWKS to check the signature.
If you want a picture, think of a notice board outside an office with specimen signatures of everyone authorized to sign. A JWT is a signed document; the JWKS is the board of specimens. Every document says who signed it, and the checker compares it against the board. When someone leaves the job, you take their specimen down, and anything they signed stops passing. Removing a kid from the JWKS works the same way.
So why do we need a JWKS if JWTs can already be verified locally? The upside of a JWT is that the API doesn't need to ask the database on every request. The downside is that revoking access one token at a time is hard. Once issued, a token is valid until it expires, unless something else steps in. The JWKS is that something: you can pull a leaked key out all at once, and rotate to a new key without touching every API.
How do you rotate keys without breaking things?
For scheduled rotation, I split it into three steps:
| Step | What you do | Why |
|---|---|---|
| 1. Publish ahead | Add the new public key to the JWKS, keep signing with the old one | Give every API time to pull the new JWKS into its cache |
| 2. Switch signing | New tokens use the new kid | Old tokens still verify because the old key is still published |
| 3. Retire the old key | Remove it from the JWKS | Wait until tokens signed with it have expired |
Step 1 is where people slip. The OpenID Connect spec says verifiers should re-fetch the JWKS when they see an unknown kid. That makes step 1 look optional.
In practice, that sometimes doesn't work. Many JWT libraries rate-limit JWKS re-fetches, so nobody can send random kid values to make your server download the JWKS over and over. Node's jose, for example, uses a 30-second cooldown by default. Switch to the new key right after publishing it, and some API instances will reject perfectly valid tokens for a while. The simple fix: publish the new key, then wait longer than the verifiers' cache before you switch.
Step 3 is the same idea. If access tokens live 15 minutes, wait at least 15 minutes after switching, plus a little for clock skew, before removing the old key.
If the private key leaks, how fast can you kick the attacker out?
Say the private key leaks and an attacker can now sign tokens as anyone. There's no time to wait.
- Generate a new key and start signing with it immediately.
- Remove the leaked
kidfrom the JWKS immediately. - Tell every API to clear its JWKS cache, or wait for the cache to expire.
Once caches update, every token signed with the leaked key fails at the same moment, and each rejection happens inside the API itself, with no database call. That's the part that really is fast.
There's a trade-off on two fronts.
First, real users get kicked out too. Everyone holding a token from the old key needs a new one. If refresh tokens work well, users may not notice. If not, everyone logs in again. And if you suspect refresh tokens leaked as well, you have to revoke those too.
Second, and more important, it isn't immediate if caches are stale. If your APIs cache the JWKS for 10 minutes (jose's default) and you have no way to flush it, forged tokens can keep passing for up to 10 more minutes. So you want a way to refresh the cache on every instance, or a cache short enough for the risk you're willing to accept.
You can kick an attacker out exactly as fast as the cache you configured.
How do you verify JWTs so rotation actually matters?
If verification is sloppy, rotating keys won't save you. RFC 8725, the JWT best current practices, has a long list. The ones I'd put first:
- Pin the accepted algorithms yourself: don't trust the token's
algheader. - Check
issandaudevery time: make sure it came from an issuer you trust and was issued for this API. - Never follow
jkuorx5uin the header: only use the JWKS URL you configured. - Treat
kidas untrusted input: if you use it in a database lookup, guard against injection.
An example with jose in Node.js:
import { createRemoteJWKSet, jwtVerify } from "jose"; const JWKS = createRemoteJWKSet( new URL("https://auth.example.com/.well-known/jwks.json"), { cooldownDuration: 30_000, cacheMaxAge: 600_000 }, ); const { payload } = await jwtVerify(token, JWKS, { issuer: "https://auth.example.com", audience: "orders-api", algorithms: ["ES256"], clockTolerance: 30, }); Which tool for which incident?
In short:
| Incident | Use |
|---|---|
| Bots hammering random API keys | Checksum reject at the gateway + rate limiting |
| One customer's API key leaked | Revoke that key in storage (and cache), issue a new one |
| One user's access token leaked | Short token lifetimes + revoke the refresh token; a jti denylist only if you really need it |
| The JWT signing private key leaked | Emergency rotation: remove the kid from the JWKS, flush caches |
| Nothing has happened | Rotate on a schedule so the team knows the steps before the day it matters |
That last row isn't busywork. Teams that never rotate keys tend to hit surprises during their first emergency rotation, like an old service with a public key hard-coded in its config instead of fetched from the JWKS.
Wrapping up: checksums bounce junk keys first, JWKS lets you pull the keys that got through
Checksums let an API bounce garbage keys before they ever reach the database. JWKS lets an API verify JWTs locally and gives you a switch to revoke a leaked key all at once. Together they catch foreign tokens faster and cut your workload. The next level still needs a hashed lookup and rate limiting behind them.
Further reading
- GitHub Blog: Behind GitHub's new authentication token formats: how GitHub designed tokens with a prefix and checksum, and what it got out of it
- RFC 8725: JSON Web Token Best Current Practices: the full list of JWT verification recommendations
- OpenID Connect Core 1.0, Section 10.1.1: Rotation of Asymmetric Signing Keys: rotating signing keys per the spec
- jose (panva): createRemoteJWKSet: the default cooldown and cache values used in this article
Explore the NEXT4I journey and read the original article at: https://go.next4i.com/next4i-devto-en
Originally published at:
https://www.next4i.com/dev-notes/en/jwks-key-rotation-en