I build an open-source gateway on top of OpenResty, and the first version of it was embarrassingly
slow: 18 % of plain nginx's throughput on the same hardware. The biggest single fix turned out
to be one that many reverse proxies, including Nginx Proxy Manager (NPM), still don't apply:
reusing connections to the backend.
This post shows the benchmark, explains the cause, and gives you a fix you can apply to NPM today
without switching anything.
The setup
Everything runs on one GitHub-hosted ubuntu-24.04 runner (4 vCPU AMD EPYC), pinned with
taskset / --cpuset-cpus so the proxies can't steal CPU from the load generator:
| CPU | What runs there |
|---|---|
| 0 | wrk, 64 keep-alive client connections |
| 1 | the backend: nginx:1.28-alpine returning 200 ok from memory |
| 2-3 | the proxy under test (2 vCPU, 2 worker processes) |
Each scenario gets a 5 s warm-up and a 20 s measured run of GET / with a 3-byte response. Tiny
responses make this the worst case for a proxy: almost all the time goes into per-request overhead,
which is exactly what we want to compare.
One fairness note: NPM's worker_processes auto counts the machine's CPUs, not the container's
cpuset, so out of the box it started 8 workers on 2 CPUs. I set it to 2 like the others; otherwise
NPM would look worse than it is.
The results
| Proxy | req/s | p50 | p99 | Memory |
|---|---|---|---|---|
nginx 1.28, upstream with keepalive (baseline) | 107,688 | 0.58 ms | 1.04 ms | 18 MB |
| Corvus 1.5, default proxy host | 79,370 | 0.76 ms | 1.98 ms | 77 MB |
| Corvus 1.5, 30 WAF rules + 1,000-CIDR list + exploit blocking + rate limit | 55,304 | 1.09 ms | 2.93 ms | 81 MB |
| Nginx Proxy Manager 2.17, default proxy host | 14,364 | 2.53 ms | 93.82 ms | 132 MB |
| Nginx Proxy Manager 2.17, "Block Common Exploits" on | 14,384 | 2.41 ms | 83.40 ms | 134 MB |
NPM reaches about 13 % of plain nginx, and its p99 is almost 100 ms. "Block Common Exploits"
makes no measurable difference; the cost is elsewhere.
Why: proxy_pass to a variable
NPM generates one server block per proxy host. The interesting part is in its shared
proxy.conf:
set $forward_scheme http; set $server "10.0.0.21"; set $port 8080; ... proxy_pass $forward_scheme://$server:$port$request_uri; When proxy_pass points at variables instead of an upstream block, nginx has nowhere to keep idle
connections. Every request:
- opens a new TCP connection to the backend (SYN, SYN-ACK, ACK),
- sends one request and reads the response,
- closes the connection, which then sits in
TIME_WAIT.
On a 2-vCPU box that's tens of thousands of connection setups per second, all competing with the
actual proxying. The throughput doesn't just drop, it stalls in bursts: wrk's per-second
rate swung widely (a standard deviation of 12.5k around a mean of 14.4k, peaks near 30k), and the
p90 and p99 latencies (45 ms and 94 ms) come from those stalls.
With an upstream block and keepalive, nginx keeps a pool of idle connections per worker and
reuses them:
upstream app01 { server 10.0.0.21:8080; keepalive 32; } server { location / { proxy_http_version 1.1; proxy_set_header Connection ""; # don't forward "Connection: close" proxy_pass http://app01; } } For my gateway, this one change took the default proxy host from 18 % to 42 % of the baseline.
The rest of the way to ~75 % came from batching the access log (one datagram per ~100 requests
instead of one per request) and skipping rule evaluation for hosts without rules.
If your backends live on other machines (a separate gateway server in front of a LAN, which is
common), the difference grows: each new connection now costs a real network round trip, not a
loopback one.
Fix it in Nginx Proxy Manager (tested on 2.17)
NPM has two hooks that make this possible without forking it.
1. Define the upstream in /data/nginx/custom/http_top.conf (NPM includes this file in the
http block if it exists; /data is the volume you already mount):
upstream app01 { server 10.0.0.21:8080; keepalive 32; } 2. Replace the default location in the proxy host's Advanced tab. When the Advanced config
contains a location /, NPM skips its own:
location / { proxy_http_version 1.1; proxy_set_header Connection ""; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; proxy_pass http://app01; } Then restart the container (or save the host again) so nginx picks up http_top.conf.
To check that it works, put traefik/whoami behind it and look
at the RemoteAddr it prints: with NPM's default location, 20 requests showed 20 different
source ports (20 connections); with the upstream above, 2.
Caveats: this location replaces NPM's generated one, so the host's Access List no longer
applies inside it, and you now maintain the backend address in two places. It's
worth it for busy hosts, not necessarily for every one.
Try it yourself
The benchmark script runs all of the above with one command on any Linux box with Docker and wrk
(it needs sudo once to add the test host names to /etc/hosts):
git clone https://github.com/corvus-gateway/corvus && cd corvus docker build -f deploy/Dockerfile -t corvus:bench . SCENARIOS="nginx-baseline corvus-default npm-default" deploy/bench/bench.sh It sets NPM up through its API exactly like the UI would, and writes the raw wrk output next to the
summary. Method and all numbers: docs/benchmark.md.
If you get very different numbers on your hardware, I'd genuinely like to see them.
About Corvus
Corvus is the gateway from the table: Apache-2.0, one
container, an NPM-style console (it can import your NPM configuration) plus WAF rules, automatic
bans, bot challenges and a waiting room for traffic spikes. Keep-alive pools are the default for
every proxy host, which is why the "30 WAF rules" row is still almost 4x the NPM row.
But you don't need it to benefit from this post: the http_top.conf trick works on the NPM you
already run.
