HTTP/2 Multiplexing Under Load: Stream Prioritization, Head-of-Line Blocking, and the Connection Reuse Trap

HTTP/2 shipped with a promise: a single TCP connection, multiple concurrent streams, no more per-request connection overhead. For microservice clients making hundreds of short-lived RPC calls per second to a small set of upstream hosts, that promise looks compelling. In practice, the multiplexing model introduces a failure surface that HTTP/1.1's per-connection queue model does not have, and Go's net/http client exposes it through a combination of defaults that are reasonable for browser traffic but wrong for dense backend workloads.

The Multiplexing Model and Where It Breaks

HTTP/2 sends multiple request/response exchanges as streams over one TCP connection. Each stream is assigned an integer ID. The HPACK-compressed HEADERS frame opens a stream; DATA frames carry the body; RST_STREAM cancels it. From the client's perspective, all of this is concurrent. From the TCP layer's perspective, it is still a single ordered byte stream.

This is TCP head-of-line blocking in its HTTP/2 form. If a single large response — say, a bulk MongoDB read flushed as a streaming body — monopolizes the TCP send buffer, every other stream on that connection queues behind it at the kernel level. HTTP/2's application-layer flow control (WINDOW_UPDATE frames) cannot solve this because the bottleneck is below it. The receiver advertises window capacity; the sender honors it; neither can reorder bytes already sitting in a kernel socket buffer.

HTTP/3 over QUIC addresses this by multiplexing streams at the transport layer where each stream's loss recovery is independent. That is a real improvement. If your infrastructure is not there yet — and most private service meshes are not — you are operating on HTTP/2 over TCP, and the blocking behavior is real.

What Go's net/http Does With One Connection

Go's HTTP/2 client transport, implemented in golang.org/x/net/http2 and bundled into the standard library via the http2 package inside net/http, maintains a clientConnPool. When the client dials an h2 server, it records the resulting *http2.ClientConn keyed by the host. Subsequent requests to the same host reuse that connection unless:

  • The connection's concurrent stream count hits maxConcurrentStreams (server-advertised via SETTINGS frame, typically 100–250)
  • The connection is marked unhealthy (write failure, GOAWAY received)
  • DisableKeepAlives is set

For a microservice that fans out 50 parallel requests to one upstream, all 50 streams land on the same TCP connection by default. This is the connection reuse trap: the feature that eliminates dial overhead also concentrates all traffic — and all TCP-level interference — onto a single flow.

tr := &http.Transport{ ForceAttemptHTTP2: true, MaxIdleConnsPerHost: 1, // default for h2; pool manages streams, not conns } // This client will funnel all concurrent requests to the same host // through one TCP connection unless the server's SETTINGS limit is hit. client := &http.Client{ Transport: tr, Timeout: 5 * time.Second, }

The subtle part: MaxIdleConnsPerHost and MaxConnsPerHost in http.Transport are designed around HTTP/1.1's model where each connection carries one request. Under HTTP/2, the pool logic in http2.clientConnPool takes over, and those fields do not bound the number of active streams per connection. They bound the number of idle connections kept warm. The distinction matters when you are trying to reason about parallelism limits.

Stream Priority: The Feature That Never Delivered

RFC 7540 defined a dependency tree for stream prioritization. Clients could declare that stream 5 depended on stream 3, with relative weights. Servers could schedule DATA frame transmission accordingly. In theory, latency-sensitive requests could be marked higher priority than background bulk operations.

In practice, stream priority was widely ignored. nginx removed its implementation in 2023. Most h2 server implementations treat all streams as equal-weight round-robin. Go's golang.org/x/net/http2 writes priority information on stream creation but does not implement differential scheduling on the server side. If you are sending a 2MB response and a 100-byte health check response on the same connection, the scheduling depends on the server's write loop, and most write loops are FIFO per connection.

The lesson: do not architect around HTTP/2 priority as a backpressure or QoS mechanism. It is not implemented reliably enough to depend on.

The Write Serialization Problem Inside Go's h2 Transport

Go's HTTP/2 client transport serializes frame writes onto the connection through a single write goroutine (writeLoop). Requests from concurrent goroutines submit frames to a buffered channel; the write goroutine drains and flushes them. This means that if one goroutine is sending a large request body, frames from other goroutines accumulate in the channel. The scheduler sees one blocked goroutine (waiting for the channel to drain) per concurrent request, but the TCP connection sees a single sequential write stream.

For small request bodies — typical of JSON RPC calls — this is fine. For large uploads or streaming requests, you are serializing at two layers: the h2 framing layer and the TCP send buffer. Profiling this pattern with net/http/pprof often reveals goroutines blocked on http2.Framer.WriteData waiting for the write loop, while the p99 latency of unrelated concurrent requests rises.

Remediation: Multiple Connections Without Losing Pool Benefits

The remediation is to bound stream density per connection while allowing multiple connections to the same host. This requires dropping below net/http defaults.

type multiConnTransport struct { pool []*http.Client n int mu sync.Mutex idx uint64 } func newMultiConnTransport(n int) *multiConnTransport { pool := make([]*http.Client, n) for i := range pool { tr := &http2.Transport{ // Each Transport manages its own ClientConnPool. // Force a new dialer so connections are not shared. DialTLS: nil, // uses default } pool[i] = &http.Client{ Transport: tr, Timeout: 5 * time.Second, } } return &multiConnTransport{pool: pool, n: n} } func (m *multiConnTransport) Do(req *http.Request) (*http.Response, error) { idx := atomic.AddUint64(&m.idx, 1) % uint64(m.n) return m.pool[idx].Do(req) }

Round-robin across N independent http2.Transport instances gives N TCP connections to the same host, each carrying maxConcurrentStreams/N streams under full load. The tradeoff is memory and file descriptor cost per additional connection. For a service making 200 concurrent RPC calls to one upstream, 4 connections at 50 streams each distributes TCP write pressure without triggering the server's per-connection stream cap.

Note: golang.org/x/net/http2.Transport (the lower-level type) exposes StrictMaxConcurrentStreams which, when true, queues requests rather than opening new connections when the limit is hit. For strict fan-in control this is useful; for latency-sensitive calls it introduces its own queuing delay.

Observability: What to Instrument

The failure mode is often silent. Latency rises, not error rates, because the requests complete — just slowly. Instrument:

  • Active streams per connection: accessible via Transport.ConnPool if using http2.Transport directly, or via httptrace.ClientTrace with Got1xxResponse and GotConn hooks
  • Write channel depth: requires wrapping the transport; not exposed by default
  • p99 vs p50 divergence: a widening spread between median and tail under increasing concurrency is the signature of per-connection serialization
  • TTFB per request: if time-to-first-byte rises for small requests while throughput stays flat, you are seeing TCP HOL blocking, not server processing time

In a distributed tracing setup (OpenTelemetry with the Go SDK), attach span attributes for the remote address and connection reuse (httptrace.GotConn.Reused). When you correlate connection reuse rate with p99 latency across services in Grafana, the connection concentration pattern becomes visible.

Decision Framework

Use default HTTP/2 with one connection per host when:

  • Concurrent requests per host stay below ~20
  • Request and response bodies are small (under 64KB)
  • Upstream is a managed API gateway that handles h2 multiplexing well
  • Dial latency is the dominant cost (high-latency WAN paths)

Distribute across multiple connections when:

  • Fan-out concurrency per host exceeds 40–50 simultaneous in-flight requests
  • Any request carries a large body that can saturate the TCP send buffer
  • p99/p50 latency ratio exceeds 3x under load and profiling points to write-loop contention
  • You are building an internal service mesh where dial cost is low (same VPC, sub-millisecond RTT)

Consider HTTP/3 / QUIC when:

  • Your infrastructure can support it end-to-end (load balancer, service, client)
  • Stream independence at the transport layer is the actual bottleneck (confirmed by packet loss injection tests)
  • The operational cost of QUIC's UDP path (different firewall rules, observability tooling gaps) is acceptable

Do not rely on HTTP/2 stream priority for:

  • Quality-of-service differentiation between request classes
  • Backpressure signaling between services
  • Latency guarantees on high-priority streams

The multiplexing abstraction is sound at low concurrency. At high concurrency, the single TCP connection becomes a contention point that the application layer cannot fully mitigate. Recognizing which side of that threshold your service operates on — and tuning the transport accordingly — is the difference between a system that holds p99 under load and one that degrades silently.