Skip to slide
051 / 170

CH 05 · Prompt Caching and Context Management · 03/15

Read as one page

What breaks the cache

Because hits need exact prefix matches, a handful of innocent-looking actions cause expensive cache misses:

  • Changing the tools list mid-conversation.
  • Switching the model (this changes the model-specific instructions near the front of the prompt).
  • Changing the sandbox configuration, approval mode, or working directory.
  • Rewriting an earlier AGENTS.md message would invalidate the cached prefix. Codex does not rewrite it. render_diff appends a new message, so the old prefix still matches and only the new message is uncached.

The Codex team learned this the hard way: an early bug enumerated MCP tools in an inconsistent order, which silently broke caching on every turn. The fix for necessary mid-session changes is clever and worth stealing: instead of editing an earlier message (which would change the prefix and bust the cache), append a new message at the end describing the change. For example, if the working directory changes, Codex inserts a fresh <environment_context> message at the tail rather than rewriting the original. The prefix stays intact; the cache survives.

A practical habit falls right out of this: lock your AGENTS.md before a long session. If you do change it, Codex appends the new text instead of rewriting the old message, so the cached prefix still matches. The new message is large, and you pay full price for it the first time.

Prompt caching is a prefix match
Prompt caching is a prefix match

Finished this idea?

Your learning progress stays in this browser.