Skip to slide
170 / 221

CH 14 · Cost, Latency, and Model Tiering · 09/14

Read as one page

Cache aggressively

  • Cache external API calls (Chapter 9): repeated identical lookups become instant and free, and you stay under rate limits.
  • Exploit provider prompt caching where available: keeping a stable prefix (system prompt, tool definitions) constant lets the provider cache it and charge less for repeated input. Structure your context so the invariant parts come first.

Finished this idea?

Your learning progress stays in this browser.