CH 05 · Prompt Caching and Context Management · 01/15
Read as one pagePrompt Caching and Context Management
Prompt caching reuses an unchanged prefix at a fraction of the cost. Compaction and truncation keep long sessions under the context ceiling.
15 slides · 16 min read
In this chapter
- 01Prompt caching: paying once for the stable part
- 02What breaks the cache
- 03Context management: staying under the ceiling
- 04Why it matters
- 05How it works: ordering and compacting
- 06Code MVP: cache-aware ordering and a compactor
- 07Real code: pi places the cache breakpoint
- 08Real code: OpenCode marks the stable edges
- 09Real code: Codex appends and keeps one cache key
- 10Real code: pi will not orphan a tool result
- 11Real code: Codex rebuilds history around a summary
- 12Read the source yourself
- 13Connecting to the bigger picture
- 14Key takeaways
Chapter 4 left us with a problem: the prompt grows every turn, cost grows quadratically, and eventually the context window fills up. This chapter covers the two families of fixes that every serious harness uses. One attacks the cost curve (prompt caching). The other attacks the window ceiling (compaction and friends). They are different problems with different solutions, so keep them separate in your head.
Finished this idea?
Your learning progress stays in this browser.