CH 05 · Prompt Caching and Context Management · 12/15
Read as one pageReal code: Codex rebuilds history around a summary
pi keeps a recent tail verbatim and summarizes what fell off. Codex's local compaction is harsher, and it is closer to a handoff than to your bookends. The compaction turn sends a prompt, takes the assistant's reply as the summary, and builds a new history. Recent tool calls are not copied. They have to survive inside the summary.
SUMMARIZATION_PROMPT is the default user text. config.compact_prompt replaces it when set. The reply is stored as summary_suffix, then summary_text is SUMMARY_PREFIX, a newline, and that reply.
You are performing a CONTEXT CHECKPOINT COMPACTION. Create a handoff summary for another LLM that will resume the task.
Include:
- Current progress and key decisions made
- Important context, constraints, or user preferences
- What remains to be done (clear next steps)
- Any critical data, examples, or references needed to continue
Be concise, structured, and focused on helping the next LLM seamlessly continue the work.
Source: openai/codex, codex-rs/prompts/templates/compact/prompt.md (Apache-2.0)
fn build_compacted_history_with_limit(
mut history: Vec<ResponseItemEnvelope>,
user_messages: &[CompactedUserMessage<'_>],
summary_text: &str,
max_tokens: usize,
) -> Vec<ResponseItemEnvelope> {
let mut selected_messages = Vec::new();
if max_tokens > 0 {
let mut remaining = max_tokens;
for message in user_messages.iter().rev() { // [1]
if remaining == 0 {
break; // [2]
}
let tokens = approx_token_count(&message.message);
Source: openai/codex, codex-rs/core/src/compact.rs (Apache-2.0)
selected_messages.push(ResponseItemEnvelope {
item,
metadata: harness_metadata,
});
if tokens > remaining {
break;
}
remaining = remaining.saturating_sub(tokens);
}
selected_messages.reverse(); // [3]
}
history.extend(selected_messages);
Source: openai/codex, codex-rs/core/src/compact.rs (Apache-2.0)
history.push(ResponseItemEnvelope::new(ContextualUserFragment::into(
CompactionSummary::new(summary_text), // [4]
)));
history
}
Source: openai/codex, codex-rs/core/src/compact.rs (Apache-2.0)
What to notice
- [1]
user_messagescomes fromcollect_annotated_user_messages. That helper keeps real user messages and drops anything that already starts withSUMMARY_PREFIX, so an old summary is not kept as if the user had typed it. The loop walks newest user message first. Assistant turns and tool results are not in this list. - [2]
max_tokenshere isCOMPACT_USER_MESSAGE_MAX_TOKENS, 20000. Thisbreakfires when that budget is already zero, so an older user message is not taken. Between these two excerpts, a message that fits and is text only is copied, and any other message keeps a truncated text prefix. A message that costs more than what is left is still pushed, and the laterbreakstops the walk. - [3]
reverseputs the kept user messages back into time order. They were collected newest first. - [4]
CompactionSummaryis a user-role fragment, pushed last by this function. The caller passed an empty initial context intobuild_compacted_history.build_compaction_replacement_historythen rebuilds the full initial context: every world-state section plus the developer instructions. Prefix items go at the front. The rest follow the four rules ininsert_initial_context_before_last_real_user_or_summary. The first choice is the slot before the last real user message, agent message, or retained tool output, so the summary stays last.
Your summarize invents a one-line count of item types. This prompt asks for progress, constraints, next steps, and the data the next model needs.
Read it yourself: Open codex-rs/prompts/templates/compact/summary_prefix.md and compare it with SUMMARY_PREFIX in codex-rs/core/src/compact.rs, on the line that formats summary_text. Then read insert_initial_context_before_last_real_user_or_summary and check the four placement rules in its comment. In codex-rs/core/src/tasks/compact.rs, see which function runs when the provider's remote compaction support is V2 instead of this local prompt.
Finished this idea?
Your learning progress stays in this browser.