Stream LLM tokens to browser with fetch, not EventSource

Use fetch + response.body.getReader() to consume model streams and re-emit as SSE events, giving you POST support, custom headers, and AbortController cancellation that EventSource lacks.

June 5, 2026

Summary

Token streaming over 15-40 seconds feels responsive to users instead of a frozen spinner; proper cancellation prevents orphaned GPU jobs and duplicate billing; anti-buffering headers (no-transform, X-Accel-Buffering: no) force proxies to flush tokens immediately instead of batching them at the end.

Why it matters

Implementation verdict

Replaces naive response-waiting patterns and EventSource for LLM endpoints. Requires Next.js 15 Route Handler, AbortController wiring through streamModel generator, TextDecoder buffering to respect TCP boundaries, and maxDuration tuning on Vercel. Ready now—this is production code from spectr-ai's security report tool.

Sources

1.the server forwards hundreds of text fragments coming out of the model in real time
2.EventSource is the obvious tool for SSE, and it handles reconnection for free. But it only does GET requests
3.controller.enqueue(encoder.encode(`data: ${JSON.stringify(event)}\n\n`))
4.no-transform and X-Accel-Buffering: no. These are the anti-buffering headers. no-transform tells proxies not to gzip-buffer the body, and X-Accel-Buffering: no disables nginx's response buffer
5.When the browser aborts, Next.js aborts request.signal, which I pass into streamModel, which passes it to the model fetch

Dev Signal

Get briefs like this in your inbox — free, every weekday.

100+ sources compressed into one 4-minute read. Ranked, cited, implementation-ready.

Read the full issue →All briefs