Skip to slide
173 / 221

CH 14 · Cost, Latency, and Model Tiering · 12/14

Read as one page

Measure before optimizing

Don't guess where the cost goes; instrument it (Chapter 15). Track tokens per turn, per task type, and per model; track loop depth and tool latency. Usually a small number of patterns dominate the bill (a heavy tool result re-sent every iteration; a flagship model used for a background task). Find those and fix them, rather than micro-optimizing prompts that barely move the needle.

Finished this idea?

Your learning progress stays in this browser.