Streaming without losing the text

Respect the boundaries

The transport can split a UTF-8 character across reads or deliver several events in one chunk. Running JSON.parse on each chunk works only by accident. Decode the bytes first, reconstruct the SSE events and then interpret each event’s data field.

The WHATWG standard defines the protocol. For a POST request carrying JSON, use fetch. Your application backend holds the provider key and forwards the stream. A permanent API key must never be bundled into JavaScript delivered to the browser.

Read events, not chunks

This example uses the stream interface from eventsource-parser, documented in the project repository. Compile it with your application’s bundler. The /api/chat route represents your own backend; it is not an endpoint supplied by this page.

import { EventSourceParserStream } from 'eventsource-parser/stream';

export async function readChat(messages, signal, onDelta) {
  const response = await fetch('/api/chat', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ messages, stream: true }),
    signal
  });
  if (!response.ok) throw new Error(`HTTP ${response.status}`);
  if (!response.body) throw new Error('Missing response stream');
  const type = response.headers.get('content-type') || '';
  if (!type.includes('text/event-stream')) {
    throw new Error('Unexpected response format');
  }
  const reader = response.body
    .pipeThrough(new TextDecoderStream())
    .pipeThrough(new EventSourceParserStream({
      maxBufferSize: 1024 * 1024,
      onError: 'terminate'
    }))
    .getReader();
  let completed = false;
  try {
    while (true) {
      const { value, done } = await reader.read();
      if (done) break;
      if (value.data === '[DONE]') {
        completed = true;
        break;
      }
      const event = JSON.parse(value.data);
      if (event.error) throw new Error('Provider error');
      const text = event.choices?.[0]?.delta?.content;
      if (typeof text === 'string') onDelta(text);
    }
    if (!completed) throw new Error('Response interrupted');
  } finally {
    try { await reader.cancel(); }
    finally { reader.releaseLock(); }
  }
}

The [DONE] marker is a convention of this chat format, not a universal SSE rule. Adapt the completion condition explicitly if your backend uses a different protocol. A final usage event may have an empty choices array; optional access avoids treating that valid event as a failure.

Stop the whole request

Create an AbortController for each generation and pass its signal to the reader. The Stop button calls controller.abort(). Your backend must propagate that cancellation upstream. Closing only the interface leaves the provider generating, potentially consuming more credit.

A new request must also invalidate updates from the previous one. A local generation counter handles the race: before applying a delta, compare the callback’s generation identifier with the current one. Abort reduces unnecessary work; the identifier prevents an already queued callback from writing into the wrong conversation. Use both rather than assuming cancellation instantly removes every pending update.

Keep the partial answer

Useful states include waiting, generating, completed, stopped by the user and failed. Received text still belongs to the message when the connection drops. Preserve it and mark it clearly as “Response interrupted,” instead of deleting it or presenting it as complete.

In React, accumulate deltas in a buffer and refresh the view at frame rate rather than rendering every fragment. Cleanup should cancel a pending animation frame when the component unmounts. Use a Markdown renderer that does not execute arbitrary HTML, and expect code blocks to remain incomplete during generation. Treat partial text as partial text throughout the storage and display layers.

Test the awkward cases

A meaningful test splits the same response in different places: inside an emoji, between a carriage return and line feed, halfway through JSON and just before the blank line that ends an event. The reconstructed output must remain identical. Add comment events, multiline data, usage without choices and premature connection closure.

Also check Stop, conversation switching and two submissions in quick succession. These are client tests, not model speed benchmarks. To understand the actual experience, record time to first content, total duration and the proportion of interrupted generations separately. A polished animation cannot compensate for a transport layer that loses text.