OpenAI timeout=60 Never Fired on a Stream With Keep-Alives

The openai Python SDK's timeout=60 limits each socket read, not the request, so a streamed completion that receives an SSE keep-alive comment every 15 seconds never times out, even though your loop never sees an event. Put a wall clock around the whole attempt instead: with AsyncOpenAI, asyncio.timeout(30) ended the same stalled stream at 30.0 to 30.1 seconds in every run.

This story was written with the assistance of an AI writing program. The numbers come from openai 2.43.0 and httpx 0.28.1 on Python 3.14.4, run against a local fake OpenAI-compatible endpoint over plain HTTP and TLS. I did not test OpenAI's own API, and I do not know whether it sends comment lines.

The Server Sends One Token, Then Only Heartbeats

The fake endpoint answers POST /v1/chat/completions with text/event-stream, sends one normal data: chunk containing "Hello", and then stops producing tokens. In the silent variant it sends nothing more and holds the connection open. In the heartbeat variant it writes : keep-alive every 15 seconds and never sends [DONE].

The client copies the settings of a real pipeline I reviewed this week: OpenAI(timeout=60.0, max_retries=0), stream=True, and a for chunk in stream loop. Each scenario ran as its own process under a 150-second timeout command, so exit code 124 means it was still blocked when the cap hit.

Silence Times Out at 60 Seconds; Heartbeats Never Do

Result lines from one plain-HTTP run, all scenarios in parallel:

{"result": "exception", "type": "httpx.ReadTimeout", "msg": "timed out", "is_openai_APITimeoutError": false, "t": 60.6, "scenario": "silent", "tls": false} {"scenario": "comments", "exit": 124} {"result": "exception", "type": "builtins.TimeoutError", "msg": "", "t": 30.1, "scenario": "comments-async", "tls": false}

The silent stream failed at 60.4 to 60.6 seconds across three runs, as expected. The heartbeat stream was still waiting at 150 seconds in all three, including the TLS run. Nothing in the loop body ran after the first chunk.

Look at the type in the silent case: httpx.ReadTimeout, not openai.APITimeoutError. The SDK converts timeouts into its own exception while it sends the request; once you are iterating, the raw httpx exception comes through. An except openai.APITimeoutError around a stream misses it.

Why the Loop Sees Nothing but the Timer Resets

Two pieces combine. First, the SDK's SSE decoder discards comment lines: openai/_streaming.py has if line.startswith(":"): return None, which is correct for the format, so your code never learns that bytes arrived. Second, timeout=60 becomes httpx.Timeout(60), which sets the same 60 seconds for connect, read, write and pool acquisition. None of the four is a total. The read limit restarts on every successful read, and each comment is one.

So the 60 seconds measure silence on the socket, not progress on your completion. A heartbeat every 15 seconds means the socket is never silent for 60.

I hit the same shape with requests last week, when a server dripping one byte every three seconds kept timeout=5 busy for 57 seconds. The difference here is that the bytes are invisible: no partial content, no log line, nothing that tells you the call is still alive.

Who Actually Sends Comment Lines

This is not exotic. The HTML Standard's section on server-sent events suggests sending a comment line every 15 seconds or so, because some legacy proxies drop connections that go quiet. OpenRouter's streaming docs say it occasionally sends comments such as : OPENROUTER PROCESSING to prevent connection timeouts. Any gateway, proxy or self-hosted server that follows that advice turns a per-read timeout into no timeout while the model is stuck.

A Wall-Clock Budget Ended It at 30 Seconds

With AsyncOpenAI, I put asyncio.timeout around the request and the iteration together. This is the exact function I ran this morning, pointed at the fake endpoint through OPENAI_BASE_URL:

import asyncio import openai client = openai.AsyncOpenAI(timeout=60.0, max_retries=0) async def ask(prompt: str, budget_s: float = 30.0) -> str: parts = [] async with asyncio.timeout(budget_s): # wall clock for the whole attempt stream = await client.chat.completions.create( model="gpt-4o-mini", stream=True, messages=[{"role": "user", "content": prompt}]) async for chunk in stream: if chunk.choices and chunk.choices[0].delta.content: parts.append(chunk.choices[0].delta.content) return "".join(parts)

It raised TimeoutError at 30.0 seconds in both runs this morning; an earlier version of the test ended at 30.0 to 30.1 seconds in three runs. I also checked the socket. Right after the timeout, ss showed no established connection from the client, and the server's next keep-alive write, at 45 seconds, failed with BrokenPipeError. Wrapping the loop in async with stream: made no difference in the same two runs; the cancellation had already closed the connection.

In Sync Code, close() Was 15 Seconds Late

Sync code has no cancellation, so I tried a threading.Timer that fires at 30 seconds. Calling stream.close() from the timer did not interrupt the blocked read. The loop only failed when the next keep-alive arrived, with httpx.ReadError: [Errno 9] Bad file descriptor at 45.4 to 45.5 seconds. So a 30-second deadline ended at 45 seconds. I did not test close() against a silent stream.

Shutting down the socket worked. The timer took the raw socket from stream.response.extensions["network_stream"].get_extra_info("socket") and called shutdown(socket.SHUT_RDWR), and the loop ended at 30.4 to 30.5 seconds over plain HTTP and TLS. That reaches into a low-level httpx response extension, so I would rather move the call into async code or a worker process that can be killed.

Takeaways

Treat timeout= on a streamed call as an idle limit. It does not bound how long the attempt runs.
Put one wall-clock budget around the request and the iteration together: asyncio.timeout() with AsyncOpenAI.
In sync code, stream.close() from another thread waits for the next read. Shut the socket down instead.
Around iteration, catch httpx.TimeoutException as well as openai.APITimeoutError.
Test against a server that sends one chunk and then only : keep-alive. Mine is 77 lines of standard-library Python, and one run shows whether your deadline holds.