I Swapped a Live nginx onto Post-Quantum TLS 6 Times Under Load. Zero Errors, and the OpenSSL Upgrade Paid for ML-KEM.

TL;DR

• Moving nginx from Ubuntu's OpenSSL 3.0 to OpenSSL 3.5 with X25519MLKEM768, live, under 500 new TLS handshakes per second, dropped zero requests across 6 live swaps, 6 rollbacks (3 regular, 3 fast) and 54 config reloads. Across every run in this phase, over 7 million full handshakes, the error count was zero.
• CPU: ML-KEM adds 68 µs per handshake (+19%), but OpenSSL 3.5 is 77 µs cheaper (−18%) than 3.0 for the same classical handshake on Graviton. Net cost of the whole migration: −2%.
• The one real hazard: a PQ config line is fatal on the old binary. Roll back the binary without the config and the rollback fails silently. Traffic looks fine, and you're still on the new build.

The inspiration

Last part of the series found that 72% of the reachable top-1,000 sites already accept X25519MLKEM768. Now, from SRE standpoint the next followup question would be what does it cost me to switch my own nginx? instead of a benchmark lab — in a real production, with traffic flowing, with a rollback plan.

For most teams "switching" doesn't mean a new nginx. It means putting a newer OpenSSL under the nginx they already run. Ubuntu 24.04 ships OpenSSL 3.0.13, which can't do ML-KEM; we need 3.5. So that's the experiment.

Setup

Same nginx source, same flags, same classical certificate. Only the OpenSSL and the ssl_ecdh_curve line differ. Both binaries share one install prefix, which is what makes a live swap possible.

The target is a c7g.large (2 vCPU Graviton3), the same instance as Parts 1–3. The load generator is a c7g.xlarge in the same AZ and cluster placement group, running a small open-loop Go client. Every request is a brand-new TCP connection and a full TLS 1.3 handshake: no keep-alive, no session resumption, and the session cache is off. That's the worst case for handshake cost. Go's default client offers X25519MLKEM768 first and X25519 second, like current browsers, so the same client negotiates PQ with B and falls back on A.

nginx logs the negotiated group ($ssl_curve) and the worker PID for every request, and the response carries the PID back. That makes every request attributable to a specific binary generation, which is how you can watch a swap happen request by request.

1. The live swap

nginx has a built-in procedure for replacing its own binary without dropping connections:

1. USR2 — the master starts the new binary as a second master, with its own workers. Both generations accept connections.
2. WINCH — the old workers finish what they're doing and exit.
3. QUIT — the old master exits.

I ran A → B this way three times under 500 handshakes/s, each time with the PQ config going in alongside the new binary.

Reading the chart top to bottom:

• Throughput never moves. 270,003 requests across the three swap runs, 0 errors.
• You can see the PQ cost live. The median request steps from about 1.57 ms to 1.67 ms the moment B takes over, and steps back after the rollback.
• The handover is gradual. For the ~6 s between USR2 and WINCH, about half the handshakes are PQ, because old and new workers both accept. During a swap, a fleet serves both stacks for a moment; plan your dashboards accordingly.
• Memory roughly doubles during the overlap (30 → ~69 MB here), because two full sets of workers are running. On this 2-worker box that's trivial. With worker_processes auto on a 64-core host, budget for roughly workers × per-worker memory extra for the length of the overlap. (The figure sums each process's resident memory, so it overstates real usage a little: binary pages are shared.)

In the 3 seconds after every signal (USR2, WINCH, QUIT, across all trials), p99 latency stayed within 0.1 ms of the quiet baseline.

2. The rollback trap

Here's the finding I didn't expect, and it showed up before any load test ran:

nginx: [emerg] SSL_CTX_set1_curves_list("X25519MLKEM768:X25519:prime256v1") failed (SSL: error:0A080106:SSL routines::passed invalid argument:group 'X25519MLKEM768' cannot be set) 

That's the old binary (OpenSSL 3.0) reading the new config. It doesn't skip the unknown group; the whole config fails to load.

Each swap run included a naive rollback: put the old binary back, leave the PQ config in place, send USR2. In all three trials, the new master hit that [emerg], exited, and the existing master carried on serving. No errors, no latency blip, no change in the PQ share. In the chart, it's the lone usr2 line at ~93 s where nothing happens.

That sounds safe, and for traffic it is. But think about what the operator sees: they ran the rollback, nothing broke, and the dashboards look normal. They are still on the new build. The only evidence is an [emerg] line in error.log, and a master PID that didn't change.

Two rollbacks that do work, both tested three times under load with zero errors:

• Fast rollback, while the old master is still alive (after WINCH, before QUIT): send HUP to the old master, which restarts its workers with its in-memory binary and config, then QUIT the new master. The config on disk is irrelevant.
• Late rollback, after QUIT: revert the config first, run nginx -t with the old binary, then swap the binary and USR2.

3. Reloads

Config reloads (nginx -s reload) are the most common production operation, so I ran nine of them, 30 s apart, under 500 handshakes/s: three trials per arm, 54 reloads in all. Zero errors. In the 3 s after each reload, p99 stayed within 0.1 ms of the quiet baseline on both arms. PQ doesn't change reload behaviour.

4. CPU: what a handshake costs

This needed two attempts (more on that below). The second measured CPU used by nginx's own processes only, at four connection rates, three trials each, in shuffled order.

The per-handshake cost is flat from 1,000 to 4,000 handshakes/s, and trial-to-trial spread is under 10 µs. Each figure covers everything nginx does for that request (accept, TLS, HTTP, logging), not only the cryptography. The ceilings are extrapolated: the highest measured rate, 4,000/s, used 1.73 of the 2 cores on arm A with no errors.

Pulled apart:

• Upgrading OpenSSL 3.0 → 3.5 alone: −77 µs per handshake (−18%). OpenSSL 3.5 does the same classical handshake noticeably faster on Graviton.
• Adding X25519MLKEM768 on 3.5: +68 µs (+19%). That's the price of the extra key encapsulation, plus a ClientHello and ServerHello each about 1 kB bigger.
• Net, A → B with PQ: −2%. If you're on a distro OpenSSL today, the upgrade you need for PQ roughly pays for PQ.

One caveat about 4,000/s: for A and B-with-PQ (87–89% host CPU) latency starts to rise, with the median handshake at 1.2–1.3 ms versus 0.7–0.9 ms at lower rates. B-classical's p99 at that rate averages 95 ms across its three trials, so at least one trial had a latency spike. I'm treating 4,000/s as the edge of saturation and not comparing tail latency there.

5. Build, footprint, memory

The binary looks 6× bigger, but A loads its OpenSSL from shared libraries; counted fairly, B is 16% bigger. There's no leak: both arms grew about 2 MB/h, which is allocator warm-up, not a slope that continues. One small oddity: after a live swap or several reloads, the master process settles around 8 MB versus 2 MB after a cold start. It's a fixed ~6 MB per host, not per worker, and it stays flat.

The instrument failed twice first (again)

A recurring theme of this series is that the measuring tool hits its limits before the system under test does.

First, rollback detection. My swap script originally declared success when nginx renamed its pid file to .oldbin. nginx does that before the new binary even parses its config, so my script reported the failed naive rollback as a success. The fix: a swap only counts as successful if, a moment later, two masters are actually running. That's also the check to put in your own runbook.

Second, CPU. The first CPU ladder sampled whole-host CPU, one 60 s run per step, in a fixed order. At 1,000 handshakes/s it showed PQ as cheaper than classical, and one run drifted from 34% to 17% CPU at a constant request rate. Kernel networking, sshd copying the previous run's logs, and ordering effects were all mixed in. The rerun counted only nginx's processes (from /proc/<pid>/stat), used three shuffled trials, and excluded the ramp-up seconds. The same 1,000 handshakes/s step that looked backwards in host CPU (A 19.5%, B-classical 26.8%) is cleanly ordered in nginx CPU (A 0.41 cores, B-classical 0.35).

The runbook

If I were doing this on a production fleet:

1. Build B in CI, with the same configure flags as your current nginx (nginx -V prints them). Budget about 2 minutes for the OpenSSL compile.
2. Check headroom: CPU is roughly a wash if you're coming from a distro OpenSSL 3.0. Memory needs room for a second full set of workers during the swap.
3. Test the new pair before touching the running server: new-nginx -t -c new.conf.
4. Swap binary and config together: USR2 → verify (two masters running, $ssl_curve showing X25519MLKEM768 in the access log) → WINCH.
5. Keep the old master alive until you're satisfied. While it lives, rollback is HUP old + QUIT new, with no config dependency.
6. After QUIT, rollback means config first, then nginx -t with the old binary, then the binary.
7. Alert on [emerg] in error.log and on master PID changes. A failed rollback is otherwise invisible.

Method and limits

• nginx 1.28.0; OpenSSL 3.0.13 (Ubuntu 24.04 package) vs 3.5.2 (upstream, default build options, statically linked); c7g.large target, c7g.xlarge load generator, same AZ, cluster placement group.
• Every request is a full TLS 1.3 handshake with a 3-byte response. That isolates the handshake path and is the worst case; with resumption and keep-alive, real traffic costs much less per request.
• ECDSA P-256 certificate on both arms. This is the KEM-first migration you can ship today; PQ certificates are a separate problem (Part 4).
• The A/B CPU comparison is distro-built OpenSSL 3.0 vs upstream-built 3.5. Part of the 18% could come from build options rather than the version; it is the comparison an operator actually faces.
• One instance type, one region, one load shape. Per-handshake µs will differ on x86 and on bigger instances.
• Everything for this phase (Terraform, build scripts, load generator, scenario orchestration, analysis, and per-run results) is in its own repo, pqc-bench-nginx-swap, commit 9065728. The whole matrix costs under $1 to reproduce.

Next: the SLH-DSA measurements, once the monitor turns green.