CH 05 · Prompt Caching and Context Management · 03/15
Read as one pageWhat breaks the cache
Because hits need exact prefix matches, a handful of innocent-looking actions cause expensive cache misses:
- Changing the
toolslist mid-conversation. - Switching the model (this changes the model-specific instructions near the front of the prompt).
- Changing the sandbox configuration, approval mode, or working directory.
- Rewriting an earlier
AGENTS.mdmessage would invalidate the cached prefix. Codex does not rewrite it.render_diffappends a new message, so the old prefix still matches and only the new message is uncached.
The Codex team learned this the hard way: an early bug enumerated MCP tools in an inconsistent order, which silently broke caching on every turn. The fix for necessary mid-session changes is clever and worth stealing: instead of editing an earlier message (which would change the prefix and bust the cache), append a new message at the end describing the change. For example, if the working directory changes, Codex inserts a fresh <environment_context> message at the tail rather than rewriting the original. The prefix stays intact; the cache survives.
A practical habit falls right out of this: lock your AGENTS.md before a long session. If you do change it, Codex appends the new text instead of rewriting the old message, so the cached prefix still matches. The new message is large, and you pay full price for it the first time.
Finished this idea?
Your learning progress stays in this browser.