CH 05 · Prompt Caching and Context Management · 11/15
Read as one pageReal code: pi will not orphan a tool result
Your Compactor.compact keeps item 0 and the last keep_recent items. Suppose keep_recent is 3 and the history is the user request plus three call/result pairs, seven items. The last three start on a tool result whose call sits in the summarized middle. The model then sees a result with no call. pi will not cut there.
prepareCompaction calls findProjectedCutPoint. A ProjectedSessionEntry is one session entry after context edits have been applied. keepRecentTokens is how many recent tokens stay verbatim.
export const DEFAULT_COMPACTION_SETTINGS: CompactionSettings = {
enabled: true,
reserveTokens: 16384, // [1]
keepRecentTokens: 20000, // [2]
};
Source: earendil-works/pi, packages/coding-agent/src/core/compaction/compaction.ts (MIT)
export function shouldCompact(contextTokens: number, contextWindow: number, settings: CompactionSettings): boolean {
if (!settings.enabled) return false;
return contextTokens > contextWindow - settings.reserveTokens; // [3]
}
Source: earendil-works/pi, packages/coding-agent/src/core/compaction/compaction.ts (MIT)
function isCutPointMessage(message: AgentMessage): boolean {
switch (message.role) {
case "user":
case "assistant":
case "bashExecution":
case "custom":
case "branchSummary":
case "compactionSummary":
return true;
case "toolResult":
return false; // [4]
}
return false;
}
Source: earendil-works/pi, packages/coding-agent/src/core/compaction/compaction.ts (MIT)
let accumulatedTokens = 0;
let exceededBudget = false;
let cutIndex = cutPoints[0];
for (let i = endIndex - 1; i >= startIndex; i--) {
const messageTokens = entries[i].messages.reduce((sum, message) => sum + estimateTokens(message), 0);
if (messageTokens === 0) continue;
accumulatedTokens += messageTokens;
if (accumulatedTokens >= keepRecentTokens) {
exceededBudget = true;
cutIndex = cutPoints.find((candidate) => candidate >= i) ?? cutPoints[cutPoints.length - 1]; // [5]
break;
}
}
Source: earendil-works/pi, packages/coding-agent/src/core/compaction/compaction.ts (MIT)
The summary itself is a real model call, not a count of item types. This is the instruction, with the middle sections elided:
const SUMMARIZATION_PROMPT = `The messages above are a conversation to summarize. Create a structured context checkpoint summary that another LLM will use to continue the work.
Use this EXACT format:
## Goal
[What is the user trying to accomplish? Can be multiple items if the session covers different tasks.]
## Constraints & Preferences
- [Any constraints, preferences, or requirements mentioned by user]
- [Or "(none)" if none were mentioned]
// ... (Progress and Key Decisions)
## Next Steps
1. [Ordered list of what should happen next]
## Critical Context
- [Any data, examples, or references needed to continue]
// ...
Keep each section concise. Preserve exact file paths, function names, and error messages.`;
Source: earendil-works/pi, packages/coding-agent/src/core/compaction/compaction.ts (MIT)
What to notice
- [1]
reserveTokensis headroom, 16384 by default. Compaction starts when the context is inside that reserve of the window, so the model still has room to answer. Your toy waits until a fixedmax_tokensand has no separate reserve. - [2]
keepRecentTokensis 20000 by default, kept verbatim. Your toy keeps a count of items, so four huge logs can still blow the budget. - [3] The check is
contextTokens > contextWindow - reserveTokens. Disabled settings skip it. The trigger does not itself delete anything. - [4] A
toolResultis not a legal cut, and neither is asystemmessage, which falls through to the finalreturn false. User, assistant,bashExecution,custom,branchSummary, andcompactionSummaryare legal cuts. The tool result stays with the assistant entry that issued the call, on whichever side of the cut that entry lands. - [5] The walk goes backward from the newest entry. When the estimate crosses
keepRecentTokensat indexi, the cut is the nearest legal point at or afteri. A trailing tool result has no later legal point, so??uses the last one: the assistant message in front of those results. Call and result stay together.
getSummarizationFailure drops a summary whose stopReason is "length". That text is partial, so it must not become the checkpoint. Your summarize never calls a model, so it cannot fail this way.
Read it yourself: In packages/coding-agent/src/core/compaction/compaction.ts, read the rest of SUMMARIZATION_PROMPT, then getSummarizationFailure. Then read prepareCompaction and see what isSplitTurn does. The turn's prefix is summarized on its own, and the messages from the cut onward stay verbatim.
Finished this idea?
Your learning progress stays in this browser.