Skip to slide
058 / 170

CH 05 · Prompt Caching and Context Management · 10/15

Read as one page

Real code: Codex appends and keeps one cache key

Your helper appends a <context_update> so the old bytes stay put. Codex picks that string in prompt_cache_key. When AGENTS.md or the working directory changes, it records a new message instead of patching the old one.

    fn prompt_cache_key(&self, responses_metadata: &CodexResponsesMetadata) -> String {
        if let Some(prompt_cache_key) = &self.prompt_cache_key_override {
            return prompt_cache_key.clone(); // [1]
        }

        if let SessionSource::Internal(source) = &self.state.session_source
            && let Some(parent_thread_id) = responses_metadata.parent_thread_id
        {
            return format!("{source}:{parent_thread_id}"); // [2]
        }

        responses_metadata.session_id.clone() // [3]
    }

Source: openai/codex, codex-rs/core/src/client.rs (Apache-2.0)

        if matches!(previous, PreviousSectionState::Known(previous) if previous == &current) {
            return (None, Vec::new()); // [4]
        }

        let previous_may_contain_instructions = match previous {
            PreviousSectionState::Known(previous) => previous.text.is_some(),
            PreviousSectionState::Unknown => true,
            PreviousSectionState::Absent => false,
        };
        let instructions = match (&self.instructions, previous_may_contain_instructions) {
            (Some(instructions), true) => UserInstructions {
                directory: instructions.directory.clone(),
                text: format!("{REPLACEMENT_NOTICE}\n\n{}", instructions.text), // [5]
            },
            (Some(instructions), false) => instructions.clone(),
            (None, true) => UserInstructions {
                directory: None,
                text: REMOVAL_NOTICE.to_string(),
            },
            (None, false) => return (Some(current), Vec::new()),
        };
        (
            Some(current),
            vec![WorldStateUpdate::fragment(instructions)], // [6]
        )

Source: openai/codex, codex-rs/core/src/context/world_state/agents_md.rs (Apache-2.0)

What to notice

  1. [1] This branch returns prompt_cache_key_override when one was set. A guardian review session can force that override to guardian:{parent_thread_id}. An ephemeral fork of a root session can force it to the forked session's id. Every other session uses the later branches in prompt_cache_key.
  2. [2] An internal session that has a parent thread uses {source}:{parent_thread_id}. source prints as guardian or memory_consolidation. Internal sessions of that same kind under that parent share this key. It is not the parent's session_id. A normal parent session falls through to its own id, with no guardian: prefix.
  3. [3] The remaining case uses this session's id. build_responses_request sets the request field from prompt_cache_key() on every turn. A root session sends this key as the Responses session id.
  4. [4] render_diff compares the current AGENTS.md text with the previous snapshot. No change means no new history item, so the prefix is left alone.
  5. [5] A real edit does not rewrite the earlier instructions message. It builds a new UserInstructions value whose text starts with REPLACEMENT_NOTICE: "These AGENTS.md instructions replace all previously provided AGENTS.md instructions." The old message stays byte for byte, so the cached prefix still matches.
  6. [6] WorldStateUpdate::fragment is that new user message. When the diff is non-empty, the history store pushes it onto the end of self.items.

A working-directory change is appended the same way. In environment.rs, has_same_diff_value returns false when cwd, status, error, the primary flag, or the shell differs, and render_diff emits a new <environment_context> fragment. A date, timezone, network, or filesystem change can emit one even when cwd is unchanged.

Read it yourself: In codex-rs/core/src/client.rs, open build_responses_request and search for let prompt_cache_key = Some(self.prompt_cache_key. The next field in that function is the shorthand prompt_cache_key,. A search for prompt_cache_key: lands on a comparison of two requests, not on this assignment. Then open codex-rs/core/src/session/mod.rs and read record_step_world_state_if_changed. record_conversation_items runs only when the rendered diff is non-empty. Follow that into codex-rs/core/src/context_manager/history.rs and find the push onto self.items.

Finished this idea?

Your learning progress stays in this browser.