Skip to slide
156 / 221

CH 13 · Reliability: Retries, Idempotency, and Failure Handling · 08/13

Read as one page

Failing within a stream

Because agent responses are streamed (Chapter 8), failures often occur after you've started responding. Handle it:

  • Wrap the orchestration so that on error you can still emit an error event and a terminating sentinel, so the client shows a clean failure rather than hanging.
  • Don't lose persisted progress: the pre-saved user message and any incrementally-saved results survive.
  • Clean up server-side resources (abort the in-flight model call) when the client disconnects, so abandoned turns stop burning tokens.

Finished this idea?

Your learning progress stays in this browser.