When you build a streaming AI endpoint in Next.js App Router, handling the happy path is easy. Handling what happens when a user clicks Stop, closes their mobile browser tab, or your upstream LLM provider hangs for 45 seconds is where production bugs and wasted token bills creep in.
If your route handler doesn't propagate abort signals and separate first-token timeouts from total stream duration, your server keeps paying for tokens nobody will ever read—or leaves the UI stuck on an infinite loading spinner.
Here is a clean, dependency-free pattern using standard Web AbortSignal APIs.
1. Combine the Client Disconnect Signal With a Hard Timeout
Next.js Route Handlers pass a standard Request whose req.signal aborts if the client disconnects. On its own, req.signal won't protect you when the upstream provider stalls while the user is still waiting on the page.
Use AbortSignal.any to merge req.signal with an AbortController that enforces your server deadline:
export async function POST(req: Request) { const timeoutController = new AbortController(); const combinedSignal = AbortSignal.any([ req.signal, timeoutController.signal, ]); const hardTimer = setTimeout(() => { timeoutController.abort(new Error("Upstream stream exceeded 30s limit")); }, 30_000); try { const upstream = await callUpstreamModel({ signal: combinedSignal, }); const stream = new ReadableStream({ async start(controller) { const encoder = new TextEncoder(); try { for await (const chunk of upstream) { if (combinedSignal.aborted) break; controller.enqueue(encoder.encode(`data: ${JSON.stringify({ text: chunk })}\n\n`)); } controller.enqueue(encoder.encode("data: [DONE]\n\n")); controller.close(); } catch (err) { if (!req.signal.aborted) { controller.enqueue( encoder.encode( `data: ${JSON.stringify({ error: "stream_interrupted" })}\n\n` ) ); } controller.close(); } finally { clearTimeout(hardTimer); } }, cancel() { timeoutController.abort(new Error("Client cancelled stream")); clearTimeout(hardTimer); }, }); return new Response(stream, { headers: { "Content-Type": "text/event-stream; charset=utf-8", "Cache-Control": "no-cache, no-transform", }, }); } catch (err) { clearTimeout(hardTimer); if (req.signal.aborted) { return new Response(null, { status: 499 }); } return Response.json({ error: "upstream_failed" }, { status: 504 }); } } Two details matter here:
-
cancel()onReadableStream: On several runtimes,req.signalfires only on socket close, whereas canceling the reader on the client triggersstream.cancel(). Hooking both prevents zombie upstream requests. - Always
clearTimeout(hardTimer)in bothfinallyandcancel(): Otherwise long-lived workers hold onto timer callbacks needlessly.
2. Separate Time-to-First-Token (TTFT) From Total Stream Duration
A 30-second total budget is reasonable for a long answer, but waiting 25 seconds for the very first token means the upstream request is almost certainly stuck.
Instead of one flat timer, start with a tight first-token deadline (for example 8_000 ms), and swap it for a sliding inactivity timeout once chunks begin arriving:
let idleTimer = setTimeout(() => { timeoutController.abort(new Error("TTFT timeout")); }, 8_000); const resetIdleTimer = () => { clearTimeout(idleTimer); idleTimer = setTimeout(() => { timeoutController.abort(new Error("Chunk stall timeout")); }, 10_000); }; Calling resetIdleTimer() inside your for await loop fails fast on dead connections in 8 seconds while still allowing genuinely long, actively streaming responses to finish cleanly.
3. Send a Clean Terminal Event Instead of Dropping the Connection
When a stream stalls mid-sentence and the server simply closes or errors the socket, fetch on the client throws a generic network error—and many UIs discard the partial text the user already saw.
Sending a structured JSON frame (such as { error: "stream_interrupted" }) right before controller.close() lets your frontend:
- Keep whatever partial answer already rendered,
- Show an explicit Retry button rather than freezing the cursor,
- Distinguish user-triggered aborts (
AbortErrorfrom clicking Stop) from genuine server timeouts.
Key Takeaways
- Merge
req.signaland your server timeout withAbortSignal.anyso both user cancels and upstream stalls stop token generation immediately. - Enforce a short TTFT / chunk-stall timeout instead of waiting 30+ seconds on a hung socket.
- Wire
ReadableStream.cancel()and emit a structured error frame before closing so the UI handles partial responses gracefully.
How are you wiring aborts and timeouts on your streaming AI routes—native AbortSignal.any, or SDK-level wrappers?
