Nearly every networking book carries the same sentence: Nagle's algorithm will not send a new small packet while unacknowledged data sits on the connection. That is how I learned the rule, and that is how I explained it to others for years. This week I opened the source and found the sentence is wrong for Linux. Then I timed it.

The gap sits in a single byte. When the first write is exactly one MSS long, a request-response round trip takes 0.049 ms. Shorten that same write by one byte, down to 107, and the identical round trip jumps to 41.3 ms. No code changed in between, no setting changed; one segment simply stopped being "full".

This is a different story from the "the knob does not do what it documents" cases on this blog. With tcp_adv_win_scale the problem was a sysctl the kernel no longer reads. Here the documentation is not wrong — today's TCP standard even names the variant, though in an appendix, with a note that the standard itself was never updated to include it. The body of the standard still describes the classic rule. What is missing is something else: nothing tells you which variant runs on your kernel, or that the difference turns on one byte.

The rig: what I timed, and with what

I wrote the smallest request-response rig that does the job. The client issues two separate sendall calls per round (8 bytes, then 24), the server waits for both, then replies with 64 bytes. The client times the whole round with perf_counter, so the number is not an ACK timer reading — it is the latency the application sees.

The host is the Linux VM behind Docker Desktop on my Mac: kernel 6.10.14-linuxkit, CONFIG_HZ=1000 (read out of /proc/config.gz, not guessed). Traffic goes over loopback, but I did not leave lo at its default 65536 MTU — ip link set lo mtu 1500 makes it look like a real Ethernet path, which puts MSS at 1448. Every arm runs 200 transactions, every arm runs three times, and I report the median of the three medians.

I recorded socket state next to the timings, because MSS is not something to assume:

mss:1448 rcvmss:536 ato:40 rto:201 

The table

Arm Change Round-trip latency (median)
A none (defaults) 41.34 ms
B TCP_NODELAY on the client 0.066 ms
C one sendall instead of two 0.050 ms
D TCP_QUICKACK before every recv 0.052 ms
E ip route ... quickack 1 0.052 ms
F ip route ... rto_min 5ms 5.18 ms
G net.ipv4.tcp_pingpong_thresh=255 0.054 ms

The three runs of arm A came out at 41.337 / 41.390 / 41.306 ms. A spread that tight says this is a timer being waited on, not noise. One admission up front: the figure is 41.3, not 40.0. The difference is jiffy resolution, plus the work of the round itself, plus the fact that this runs on a VM. I am timing the wait a user would feel, not the duration of the ACK timer, so billing that 1.3 ms to the timer would be sloppy.

The 0.054 ms in arm G carries an asterisk, which I will come back to — that arm says something different once the run gets longer.

What the standard says, what the code does

RFC 896, 1984. It states the rule like this: when new data arrives from the user, no new segment is sent if any previously transmitted data on the connection remains unacknowledged, and the inhibition is "unconditional; no timers, tests for size of data received, or other conditions are required". The memo says in its own introduction that it specifies no standards; today's TCP standard is RFC 9293 (STD 7, 2022).

The body of RFC 9293 repeats the classic rule: with unacknowledged data outstanding, the sender buffers user data. But Appendix A.3 of the same document is titled "Nagle Modification", says the combination of Nagle and delayed acknowledgements performs poorly for request-response applications, cites Minshall's draft as the fix, and adds that the modification "is implemented in some common operating systems" and that "the TCP standard has not been updated to include this Nagle modification, but implementers may find it beneficial to consider".

So the standard knows the variant exists and offers it as a possibility. That is exactly where I got stuck: on my machine it is not a possibility, it is the thing that runs. Finding out which variant that is means reading code.

The answer lives in net/ipv4/tcp_output.c, across three small functions. This is the one that decides:

static bool tcp_nagle_check(bool partial, const struct tcp_sock *tp, int nonagle) { return partial && ((nonagle & TCP_NAGLE_CORK) || (!nonagle && tp->packets_out && tcp_minshall_check(tp))); } 

partial means the segment is smaller than MSS, packets_out means there is unacknowledged data in flight. The textbook rule would end right there. But one more condition follows:

/* Minshall's variant of the Nagle send check. */ static bool tcp_minshall_check(const struct tcp_sock *tp) { return after(tp->snd_sml, tp->snd_una) && !after(tp->snd_sml, tp->snd_nxt); } 

snd_sml is the end sequence of the last small segment sent. So Linux is not asking "is there unacknowledged data in flight"; it is asking "is there an unacknowledged small segment in flight". The function that maintains the field, tcp_minshall_update, uses a neat condition: skb->len < tcp_skb_pcount(skb) * mss_now. For a single-segment skb that reads "leave it alone if the segment is a full MSS"; for a TSO/GSO skb carrying several segments it reads "leave it alone if the data ends on an MSS boundary". The comment above it explains why the authors avoided the division.

The credits at the top of the file name the source: draft-minshall-nagle-01. I opened the draft — dated 18 June 1999 on the datatracker, status "Expired Internet-Draft (individual)", expired and archived. Its proposal in one sentence: let a small packet that follows one or more maximum-sized packets go out immediately. On Linux, the send decision of every socket with Nagle enabled is shaped by that individual draft, which expired 27 years ago — and today's standard mentions it only in an appendix, as something worth considering.

Measuring that one byte: the rig produces findings of its own

My first attempt used a 1448-byte write and showed no stall, exactly as expected. I could not publish that result, because it had two explanations. The second one lives on the receiving side: the kernel does not delay an ACK once more than rcv_mss bytes have accumulated since the last one. With rcvmss:536, a 1448-byte send gets acknowledged immediately anyway — so Minshall might not be the reason for the missing stall at all.

To disable that second explanation I shrank MSS: TCP_MAXSEG 120 on the client socket, minus the 12 bytes of the timestamp option, leaves an effective MSS of 108, and ss confirmed it (mss:108 rcvmss:108). Now a full-MSS segment does not cross the receiver's threshold — the condition is >, and 108 is not greater than 108. Three arms in that same rig:

First write Meaning Round-trip latency
108 bytes exactly MSS 0.049 ms
107 bytes MSS − 1 41.32 ms
8 bytes small 41.31 ms

107 behaves like 8; 108 stands apart. To be honest, this comparison does not isolate a single variable: 107- and 108-byte segments also leave different flags on the receiver, because tcp_measure_rcv_mss branches at exactly that boundary. What tells me the wait happens on the sender is not this table but arms B and C — two changes made purely on the client side, both of which remove the stall.

To see the wait directly, I looked at the packets. Four rounds of the same rig under tcpdump -ttt:

0.000015 IP 127.0.0.1.55434 > 127.0.0.1.34069: tcp 8 0.045306 IP 127.0.0.1.34069 > 127.0.0.1.55434: tcp 0 0.000009 IP 127.0.0.1.55434 > 127.0.0.1.34069: tcp 24 0.000016 IP 127.0.0.1.34069 > 127.0.0.1.55434: tcp 0 0.000039 IP 127.0.0.1.34069 > 127.0.0.1.55434: tcp 64 

The 8-byte segment leaves, nothing happens on the wire for 45.3 ms, then a bare ACK arrives and the 24-byte segment follows it nine microseconds later. The sender really is waiting; that is a recording now, not an inference. (In the first round of the same capture the 8 and the 24 go out back to back — no stall while the quick-ACK credit still holds.)

Where the 41 ms comes from, and why RTT does not shrink it

The waiting side is the receiver's ACK timer. include/net/tcp.h defines two constants: TCP_DELACK_MAX is HZ/5, i.e. 200 ms, and TCP_DELACK_MIN equals TCP_ATO_MIN at HZ/25, i.e. 40 ms. RFC 1122 set the ceiling: the delay must be less than 0.5 seconds.

The instinct here is that loopback RTT is in microseconds, so the delayed ACK should shrink along with it. It does not — but the reason is not where I first looked. tcp_send_delayed_ack contains a block that consults RTT, and that block sits behind if (ato > TCP_DELACK_MIN). The ato I recorded is 40 and TCP_DELACK_MIN is 40; 40 > 40 is false, so that code never runs. Had I written up the mechanism without tracing it in the source, I would have pointed at the wrong lines.

The real floor is in tcp_event_data_recv:

if (m <= TCP_ATO_MIN / 2) { /* The fastest case is the first. */ icsk->icsk_ack.ato = (icsk->icsk_ack.ato >> 1) + TCP_ATO_MIN / 2; } 

m is the gap between two data arrivals. On a fast path this branch runs every time and drags ato to its own fixed point: half of itself plus 20 ms, which is exactly 40 ms. RTT never enters the arithmetic. That is why the ato:40 in my socket dump is the same on a fast path and a slow one.

There is a ceiling as well, and it hangs off an interesting place:

u32 tcp_delack_max(const struct sock *sk) { u32 delack_from_rto_min = max(tcp_rto_min(sk), 2) - 1; return min(READ_ONCE(inet_csk(sk)->icsk_delack_max), delack_from_rto_min); } 

The delayed-ACK ceiling is tied to the lower bound of the retransmission timeout, and tcp_rto_min reads the route metric only when that metric is locked, falling back to the socket's own value otherwise. Hence arm F: I wrote rto_min 5ms on the route, ss reported ato:4 rto:11, and the round-trip latency fell from 41 ms to 5.18 ms. A retransmission setting governs delayed ACKs without ever being mentioned in that role. The ip-route manual says only that rto_min is "the minimum TCP Retransmission TimeOut to use when communicating with this destination"; it says nothing about delayed ACKs.

The lock detail matters: ip sets the lock bit for you when it writes rto_min (the output reads rto_min lock 5ms). Code that writes the same metric over netlink without that bit would silently do nothing.

My first attempt here produced a wrong measurement too: I put the metric on the 127.0.0.0/8 route and nothing changed. ip route get 127.0.0.1 showed a more specific route matching; writing it there dropped rto in ss. I confirmed the metric had taken effect by reading ss, not by watching the round-trip number fall.

The third gate: does the kernel consider this connection a conversation

Delayed ACKs are not active on every connection. tcp_in_quickack_mode is built like this: the route's quickack metric is enough on its own; failing that, the kernel looks for an unexhausted quick-ACK credit, and that one counts only while the connection is not in ping-pong mode. This first branch is why arm E works — the route metric never consults the ping-pong state. The credit is capped at TCP_MAX_QUICKACKS, which is 16, and decrements per ACK, so in a 200-round arm it runs out after the first 16.

Ping-pong is the kernel's verdict that "this connection is a request-response conversation". Its threshold is a sysctl, tcp_pingpong_thresh, with a documented default of 1. How the counter grows is in the code too: when the kernel sends a data packet and less than ato has passed since the last data arrival, the counter goes up by one. The test is "did you answer quickly" — precisely the signature of a request-response pattern. On a connection streaming bulk data that condition is not met regularly, the counter never climbs to the threshold, and nobody sees 40 ms; an RPC or database protocol trading small requests for small replies gives itself away on the first round.

You can read that verdict from user space. getsockopt(TCP_QUICKACK) returns !inet_csk_in_pingpong_mode(sk) in the kernel, so reading the option tells you whether the socket counts as a conversation. I read it on the receiving side after each of five rounds: 1 right after accept, 0 after the first round, and 0 for the remaining four. That is the shortest possible proof that the threshold is 1.

The asterisk on arm G: raising the threshold does not remove the stall, it rations it

I put 0.054 ms in the table for tcp_pingpong_thresh=255, and leaving it there would have been wrong. Ping-pong is not a verdict, it is a counter; raising the threshold to 255 delays the climb, it does not prevent it. A 200-round run cannot see that. So I ran 600 rounds on a single connection:

rounds over 20 ms: [(256, 41.5), (511, 42.1)] median: 0.052 ms 

The stall did not disappear — it dropped to one round in 255, and at exact round numbers: 256 and 511. I also read the ping-pong state mid-round on the server: during round 256, while the sender was waiting, getsockopt returned 0; by the end of that round, and in round 257, it read 1 again. So the counter reaches the threshold, that round stalls, the state then clears and the counter starts climbing again. I could not pin down in the source which line does the clearing: three call sites invoke inet_csk_exit_pingpong_mode, but all three are connection-setup and duplicate/out-of-window segment paths, which should not fire in my loop. The measurement is solid; this last step of the mechanism stayed open for me.

The practical consequence: tcp_pingpong_thresh is not a fix, it is an instalment plan. On a long-lived RPC connection, users hitting 41 ms once every 255 requests does not count as "solved".

Diagram

"I set NODELAY and it still waits": the cork trap

Go back to the first branch of tcp_nagle_check: nonagle & TCP_NAGLE_CORK. With TCP_CORK on, a partial segment is held unconditionally — so the behaviour that most closely matches the textbook Nagle description is actually cork's, not Nagle's.

The comment in tcp.c is two sentences and both have to be read together: on a corked socket TCP_NODELAY "is remembered, but it is not activated until cork is cleared"; however, setting TCP_NODELAY "make[s] an explicit push, which overrides even TCP_CORK for currently queued segments". So whatever is already queued leaves the moment you set the option; anything you write after that stays under the cork. I timed it:

Client socket Round-trip latency getsockopt
nothing 41.45 ms NODELAY=0, CORK=0
TCP_NODELAY only 0.052 ms NODELAY=1, CORK=0
TCP_CORK, then TCP_NODELAY 207.99 ms NODELAY=1, CORK=1
both, then cork cleared 0.055 ms NODELAY=1, CORK=0

The third row made me laugh: ask the socket with getsockopt and it answers "NODELAY is on", while the latency got five times worse. The 208 ms does not come from the delayed ACK either; with nothing in flight under the cork, the zero-window probe timer ends the wait, and its floor is TCP_RTO_MIN = HZ/5 = 200 ms. Just as 41.3 sits on 40, 207.99 sits on 200.

In the classic pattern of writing headers with write and the body with sendfile, cork exists for exactly that purpose, and the correct use is to clear it once the writes are done — the fourth row. When somebody notices the latency and adds TCP_NODELAY as the fix, nothing changes for the writes that follow.

One more thing had to be ruled out: the kernel has a mechanism that coalesces consecutive small writes on its own (net.ipv4.tcp_autocorking, default 1). I turned it off and measured again: 41.28 ms, against 41.31 ms with it on. The counter said the same thing — across a 200-round run TcpExtTCPAutoCorking stayed at 0. Auto-corking is not the culprit.

How to recognise it in production

I look for three signs. First, the shape of the latency distribution: the problem sits in the middle, not in the tail, and p50 lands straight above 40 ms. Second, the counters — to look across a machine rather than one socket at a time, nstat: in my 200-round run TcpExtDelayedACKs went up by 205 while TcpExtDelayedACKLost stayed at 0. Third, the getsockopt(TCP_QUICKACK) reading above; a 0 means the kernel counts that socket as a conversation.

One thing not to be fooled by: ato:40 in ss -ti output is not a symptom. It is the default written to every socket on the first data packet; in arm G, where the stall never appeared, ato was 40 as well. On the network side you will find nothing at all — ping showing microseconds while the application waits 40 ms is the signature of this stall.

Which arm to pick

First, check whether something already set TCP_CORK. The worst number in this article is there: 208 ms. If your library uses the sendfile pattern, cork is probably on; the right move is to clear it at the right time, not to pile TCP_NODELAY on top of it.

Then fix the application. Arm C — folding two writes into one call — kills the problem at the root without touching a single kernel setting, because no small segment is left to split off. One caveat: the right way is not to copy header and body into one buffer but to use scatter-gather, writev/sendmsg, or MSG_MORE; you get the same single segment with no copy. A client that writes its header and its body with separate write calls is feeding Nagle by hand.

Then TCP_NODELAY. The classic advice, and it works (arm B). The price is more small packets on the wire; on one connection you will not care, across tens of thousands it comes back as bandwidth and per-packet processing cost.

TCP_QUICKACK, if the receiving side is yours. Arm D. I timed a pattern that circulates widely in code: setting the option once after accept. Across three runs nothing changed — 41.28 ms, the same as never setting it at all (41.23 ms). Setting it before every recv gave 0.055 ms. (I ran that sub-experiment at the default loopback MTU, all three arms under identical conditions.) The source explains it: the option takes the socket out of ping-pong mode, and it also marks ICSK_ACK_PUSHED and calls tcp_cleanup_rbuf, which can flush a pending ACK right away. At accept time there is no pending ACK to flush, and the protocol puts the socket back into ping-pong on the next round. Mind the cost: one setsockopt before every read doubles the syscall count on the read path.

Route metrics, when the application is out of reach. quickack 1 (E) removes the stall entirely and its scope is limited to one destination. rto_min (F) shortens the stall but lowers the floor of the retransmission timeout as well; an rto_min of 5 ms on a lossy or high-latency path means needless retransmissions. Both share one cost: more ACKs on the wire — the same bill I wrote for TCP_NODELAY applies here.

tcp_pingpong_thresh goes last, and I no longer count it as a fix. It applies across the whole network namespace, and as measured above it does not remove the stall — it rations it to one round in 255.

And there are levers I could not measure at all. Since kernel 6.15 there are two new socket options: TCP_RTO_MIN_US (45) and TCP_DELACK_MAX_US (46). The second lets you write the delayed-ACK ceiling, in microseconds, onto a single socket; the setsockopt handler returns EINVAL if the value exceeds TCP_DELACK_MAX (200 ms) or falls below 2 jiffies. There is a sysctl in the same family too: net.ipv4.tcp_rto_min_us (not in the v6.10 documentation, present in v6.11), and ip-sysctl.rst spells out the precedence: the route's rto_min ranks highest, then the TCP_BPF_RTO_MIN and TCP_RTO_MIN_US socket options, then this sysctl. I never ran or timed any of the three: the lab kernel is 6.10 and my own server runs 6.8; /proc/sys/net/ipv4/tcp_rto_min_us does not exist in the lab container, and in the run where I tried to write it the round trip stayed at 41.4 ms. Given that arm F demonstrably lowers the ceiling, I infer the effect from the same code path — but an inference is not a measurement.

Version note: which sentence ages

I ran the measurements on 6.10 while the quoted code is today's tree (as of 18 November 2026, stable is 7.3.5 and mainline is 7.4-rc3). I compared every function I quote against the 7.3.5 tag: byte-for-byte identical, which is why the source links below are pinned to that tag. The differences between 6.10 and today:

  • tcp_minshall_check, tcp_minshall_update, tcp_nagle_check: byte-for-byte identical. The article's main finding is version-independent.
  • TCP_DELACK_MIN/MAX, TCP_ATO_MIN, TCP_MAX_QUICKACKS and tcp_event_data_recv: identical.
  • tcp_delack_max: changed. In 6.10 it read the route metric when that metric was locked; in today's tree it goes through tcp_rto_min(), which falls back to the socket's own icsk_rto_min when no locked route metric exists. The outcome at defaults is the same; the difference is that the new socket option can now influence the same ceiling.
  • tcp_in_quickack_mode: now caches the route metric instead of reading it on every call. Same behaviour.

So if you repeat these measurements on 7.x, I would expect arms A, B, C, D, E and G to come out the same. In arm F what changed is the path the ceiling arrives by, not the result.

Closing

What bothers me is not the 41 ms. What bothers me is that for a decade I never opened the source of a sentence I was sure about. The source turned out to be more honest than I expected: today's TCP standard names the variant in an appendix, says it is implemented in some operating systems, and leaves the decision to implementers. What the standard cannot tell me is which one runs on my machine. I learned that by reading the code and measuring.

When a protocol behaviour comes up now, I ask myself a different question: have I seen the code that carries this sentence's name, or do I treat it as true because I know the name? In this example, the distance between those two is exactly one byte.

If you are after the same kind of surprise, the measurement showing TCP_DEFER_ACCEPT does not take seconds and the count of who actually asks for keepalive sit on the same shelf.

Official Sources