Skip to slide
100 / 221

CH 08 · Streaming and Real-Time UX · 05/11

Read as one page

Streaming text while hiding machine protocol

A practical wrinkle: if the model emits a structured trailer (a citation block, Chapter 6) in the same token stream as its prose, you must stream the prose to the user but not the trailer. The technique is a small buffer: as tokens arrive, hold back just enough characters to detect the trailer's opening marker; stream everything before it; once the marker appears, stop streaming visible text and route the rest to your parser. The user sees clean prose; your code still gets the structured data. Build this boundary detection into the streaming path.

Finished this idea?

Your learning progress stays in this browser.