Lab 11 · Capstone 2 — Streaming Resilience

Kavya's chat feature shipped and the metrics came back strange: long-answer generations pinned low-end phones at 100% CPU, users who navigated away still burned server tokens for a minute, and a load balancer restart during peak turned hundreds of half-streamed answers into blank bubbles. None of these showed up on a developer laptop with a healthy server. Streaming resilience is the discipline of treating the stream as state with failure modes — this lab makes each failure measurable, then engineers it away.

Time & tier

~90 minutes hands-on · Ship stage (maps to module M7, playbook PB-48) · $0 — extends your Lab 10 build; everything stays local.

Setup

Start from your completed Lab 10 (real streaming, working stop/retry). Add chaos and speed controls to the mock server — replace the generator and route in server/index.mjs with:

async function* tokens(speed = 30) {
  for (const chunk of ANSWER)
    for (const tok of chunk.match(/.{1,4}/gs) ?? []) {
      await new Promise((r) => setTimeout(r, speed));
      yield tok;
    }
}

app.post("/api/chat", async (req, res) => {
  const { chaos, speed } = req.query; // ?chaos=drop kills mid-stream
  res.setHeader("Content-Type", "text/event-stream");
  res.setHeader("Cache-Control", "no-cache");
  res.flushHeaders();
  let count = 0;
  for await (const tok of tokens(Number(speed) || 5)) {
    if (chaos === "drop" && count === 60) return res.destroy(); // hard kill, no clean event
    res.write(`data: ${JSON.stringify({ delta: tok, seq: count++ })}\n\n`);
  }
  res.write("data: [DONE]\n\n");
  res.end();
});

app.post("/api/chat/resume", express.json(), async (req, res) => {
  // naive resume: replay from a sequence number
  const from = Number(req.body?.from ?? 0);
  res.setHeader("Content-Type", "text/event-stream");
  res.flushHeaders();
  let count = 0;
  for await (const tok of tokens(2)) {
    if (count >= from) res.write(`data: ${JSON.stringify({ delta: tok, seq: count })}\n\n`);
    count++;
  }
  res.write("data: [DONE]\n\n");
  res.end();
});

Note speed now defaults to 5ms — a fast model. That's deliberate: it makes the render-thrash problem measurable. Add the instrumentation counter to ChatPanel.tsx, at the top of the component:

const renders = useRef(0);
renders.current++;
useEffect(() => {
  const id = setInterval(() => console.log("renders so far:", renders.current), 2000);
  return () => clearInterval(id);
}, []);

And the scorecard, resilience-checklist.md:

| Concern        | Before (measured) | After (measured) | Mechanism |
|----------------|-------------------|------------------|-----------|
| Renders per answer          |      |      |           |
| Main-thread blocking (perf) |      |      |           |
| Cancel actually cancels     |      |      |           |
| Unmount mid-stream          |      |      |           |
| Server drop mid-stream      |      |      |           |
| Resume after refresh        |      |      |           |
Every cell cites an observation: a count, a DevTools panel, a repro.

Guided steps

1. 1Measure the thrash baseline. Send a message at ?speed=5 and read the render counter when the answer completes. Expected: renders ≈ token count (many hundreds) — one setMessages per SSE event means one commit per token, and each commit re-runs marked.parse + DOMPurify.sanitize over the entire growing message. Confirm the cost is real: record a Performance profile during a generation and note main-thread occupancy. Both numbers go in the Before column.
2. 2Implement token batching. Buffer incoming deltas in a useRef string and flush to state on a cadence instead of per event: either a requestAnimationFrame loop or a ~50ms interval that runs only while streaming (flushSync never; let React batch). Deltas append to bufferRef.current; the flusher does one setMessages with the accumulated text and clears the buffer; on [DONE]/error, do a final flush and stop the cadence. Expected after: renders per answer drop to roughly duration / 50ms — from hundreds to dozens — with no visible loss of smoothness. Re-measure both numbers.
3. 3Audit cancellation end-to-end. Your Lab 10 Stop calls abort() — but verify what that actually does now: click Stop mid-stream and watch (a) the network tab — does the request cancel? and (b) the server console — does the token loop stop, or does the mock keep generating to completion? Fix the server side: listen for req.on("close") (or check res.writableEnded/req.aborted in the loop) and break out. This is the difference between cancelling the render and cancelling the spend — with a real provider, an unpropagated abort keeps billing you. Prove it: server log shows the loop stopping within one token of Stop.
4. 4Handle unmount mid-stream. Add a route/toggle that unmounts ChatPanel (conditional render is fine). Unmount while streaming. Expected defect: the reader loop keeps running and calls setMessages on an unmounted component (and the flush interval leaks). Fix: abort the in-flight controller and clear the flusher in the effect cleanup; verify with the console warning gone and the network request cancelled on unmount.
5. 5Chaos test: the server drops the connection. Send with ?chaos=drop&speed=30. The server hard-destroys the socket at token 60 — no error event, no [DONE]. Observe what your client does: where does the reader loop end up (the read promise rejects, or resolves done early)? What does the user see? Engineer the degradation: detect abnormal termination (stream ended without [DONE]), keep the partial text, mark the message visibly as interrupted, offer Retry/Resume, re-enable the composer. A stalled variant is worth testing too — add a watchdog: if no event arrives for 10s while streaming, treat as dropped.
6. 6Implement resume. Track the last received seq per in-flight message (persist message text + seq to sessionStorage on each flush). On Resume — after a drop, or after a full page refresh mid-stream — call /api/chat/resume with { from: lastSeq + 1 } and append the remaining deltas to the preserved partial. Prove the refresh case: refresh mid-stream, land back in the app, see the partial restored from storage with a Resume affordance, click it, watch the answer complete without re-streaming from zero (verify via seq values in the network tab).
7. 7Fill the scorecard. Complete every cell of resilience-checklist.md with its measured evidence. Then run the full gauntlet once end-to-end: fast answer (batched, smooth), Stop (server loop halts), unmount (no leak warnings), chaos drop (graceful interrupted state), refresh + resume (completes). One take, no fixes between — this is your regression suite in manual form.
8. 8Write the resilience contract. Ten lines in the checklist file: the client-server streaming contract you now implement — events carry sequence numbers, streams end with an explicit sentinel, aborts propagate to generation, clients batch renders, partials are durable, resume is seq-based. This is the document you'd hand the backend team.

Verify

• •
  Render counts recorded before and after: before ≈ per-token (hundreds), after bounded by flush cadence (dozens) for the same answer.
• •
  Stop click provably halts server-side generation (server log evidence) and the network request shows cancelled.
• •
  Unmount mid-stream produces no state-update warnings and cancels the request.
• •
  Chaos drop leaves partial text visible, an explicit interrupted indicator, and a working Retry/Resume — never a blank bubble or silent stall.
• •
  Refresh mid-stream restores the partial from storage and Resume completes it, verified by seq continuity in the network tab.
• •
  All six scorecard rows cite concrete observations.

Self-grading rubric (pass = 7/10)

• Batching (0–3): ≥ 80% render reduction measured, smooth output, final flush correct (no lost tail tokens) = 3.
• Cancellation depth (0–3): abort propagates to the server loop, unmount cleanup complete = 3; client-only cancel = 1.
• Chaos handling (0–2): drop and stall both degrade to a designed state with recovery = 2.
• Resume (0–2): refresh-resume works with seq-verified no-overlap/no-gap = 2.