Skip to slide
060 / 170

CH 05 · Prompt Caching and Context Management · 12/15

Read as one page

Real code: Codex rebuilds history around a summary

pi keeps a recent tail verbatim and summarizes what fell off. Codex's local compaction is harsher, and it is closer to a handoff than to your bookends. The compaction turn sends a prompt, takes the assistant's reply as the summary, and builds a new history. Recent tool calls are not copied. They have to survive inside the summary.

SUMMARIZATION_PROMPT is the default user text. config.compact_prompt replaces it when set. The reply is stored as summary_suffix, then summary_text is SUMMARY_PREFIX, a newline, and that reply.

You are performing a CONTEXT CHECKPOINT COMPACTION. Create a handoff summary for another LLM that will resume the task.

Include:
- Current progress and key decisions made
- Important context, constraints, or user preferences
- What remains to be done (clear next steps)
- Any critical data, examples, or references needed to continue

Be concise, structured, and focused on helping the next LLM seamlessly continue the work.

Source: openai/codex, codex-rs/prompts/templates/compact/prompt.md (Apache-2.0)

fn build_compacted_history_with_limit(
    mut history: Vec<ResponseItemEnvelope>,
    user_messages: &[CompactedUserMessage<'_>],
    summary_text: &str,
    max_tokens: usize,
) -> Vec<ResponseItemEnvelope> {
    let mut selected_messages = Vec::new();
    if max_tokens > 0 {
        let mut remaining = max_tokens;
        for message in user_messages.iter().rev() { // [1]
            if remaining == 0 {
                break; // [2]
            }
            let tokens = approx_token_count(&message.message);

Source: openai/codex, codex-rs/core/src/compact.rs (Apache-2.0)

            selected_messages.push(ResponseItemEnvelope {
                item,
                metadata: harness_metadata,
            });
            if tokens > remaining {
                break;
            }
            remaining = remaining.saturating_sub(tokens);
        }
        selected_messages.reverse(); // [3]
    }
    history.extend(selected_messages);

Source: openai/codex, codex-rs/core/src/compact.rs (Apache-2.0)

    history.push(ResponseItemEnvelope::new(ContextualUserFragment::into(
        CompactionSummary::new(summary_text), // [4]
    )));

    history
}

Source: openai/codex, codex-rs/core/src/compact.rs (Apache-2.0)

What to notice

  1. [1] user_messages comes from collect_annotated_user_messages. That helper keeps real user messages and drops anything that already starts with SUMMARY_PREFIX, so an old summary is not kept as if the user had typed it. The loop walks newest user message first. Assistant turns and tool results are not in this list.
  2. [2] max_tokens here is COMPACT_USER_MESSAGE_MAX_TOKENS, 20000. This break fires when that budget is already zero, so an older user message is not taken. Between these two excerpts, a message that fits and is text only is copied, and any other message keeps a truncated text prefix. A message that costs more than what is left is still pushed, and the later break stops the walk.
  3. [3] reverse puts the kept user messages back into time order. They were collected newest first.
  4. [4] CompactionSummary is a user-role fragment, pushed last by this function. The caller passed an empty initial context into build_compacted_history. build_compaction_replacement_history then rebuilds the full initial context: every world-state section plus the developer instructions. Prefix items go at the front. The rest follow the four rules in insert_initial_context_before_last_real_user_or_summary. The first choice is the slot before the last real user message, agent message, or retained tool output, so the summary stays last.

Your summarize invents a one-line count of item types. This prompt asks for progress, constraints, next steps, and the data the next model needs.

Read it yourself: Open codex-rs/prompts/templates/compact/summary_prefix.md and compare it with SUMMARY_PREFIX in codex-rs/core/src/compact.rs, on the line that formats summary_text. Then read insert_initial_context_before_last_real_user_or_summary and check the four placement rules in its comment. In codex-rs/core/src/tasks/compact.rs, see which function runs when the provider's remote compaction support is V2 instead of this local prompt.

Finished this idea?

Your learning progress stays in this browser.