Skip to slide
178 / 221

CH 15 · Observability and Evaluation · 03/08

Read as one page

What to capture: the trace

The unit of agent observability is the trace: the complete record of one turn. A good trace includes:

  • Inputs: the assembled context (system prompt version, the messages, the available tools), the user, the model and settings.
  • Each model call: the request, the response, token counts (input/output/reasoning), latency, and finish reason.
  • Each tool call: name, arguments, result (or error), and duration.
  • The loop shape: how many iterations, in what order.
  • Outputs: the final answer, emitted structured data (citations), and any artifacts produced.
  • Outcome: success/failure, and any error details.

The event timeline you already build for streaming and persistence (Chapters 8, 9) is most of this for free: it's a structured, ordered record of the turn. Observability often means also routing that timeline (plus the model-call metadata) to a tracing system where you can search, aggregate, and inspect it.

Finished this idea?

Your learning progress stays in this browser.