Skip to slide
061 / 170

CH 05 · Prompt Caching and Context Management · 13/15

Read as one page

Read the source yourself

Read these in order. Each link is the commit this chapter quotes.

  1. pi's marker sites. packages/ai/src/api/anthropic-messages.ts. Search for cache_control. Count how many request shapes get a marker, and which message role is refused one.
  2. OpenCode's budget. packages/ai/src/cache-policy.ts. Read selectPolicy. After the cacheFormat checks, a route missing from RESPECTS_INLINE_HINTS gets NONE, so automatic placement adds nothing. A manual hint already on the request stays. The comment above the set says those providers ignore the tag.
  3. Codex appending a change. environment.rs. Find has_same_diff_value and the <cwd> write inside RenderedEnvironments. Confirm a directory change cannot edit an older <environment_context> message, because the diff only builds a new fragment.
  4. pi's cut and the failed summary. compaction.ts. Read findProjectedCutPoint through isSplitTurn, then getSummarizationFailure. Note which stopReason values must not be saved.
  5. Codex's rebuilt history. compact.rs and prompt.md. Trace one user message from collect_annotated_user_messages to the push of CompactionSummary.

Exercise: Open Gemini CLI's chatCompressionService.ts (Apache-2.0). In findCompressSplitPoint, an entry is a split point only when its role is user and it has no functionResponse. Model entries are never split points. Separately, the function will not return contents.length (compress everything) when the last entry is a model turn that still has a functionCall. Then find the status returned when newTokenCount > originalTokenCount. That comparison is the rebuilt history, which is the summary plus a short acknowledgement plus the kept tail, against the original history. The status is COMPRESSION_FAILED_INFLATED_TOKEN_COUNT. pi's cut and this split share one rule: do not separate a call from its result, and do not keep a summary that failed to shrink the history.

Finished this idea?

Your learning progress stays in this browser.