Skip to content

Chapter 05 · 14 min read · Updated

Context Engineering and Memory

Context is assembled fresh each turn, not stored, and the window is a hard budget. Covers referencing data instead of embedding it.

Prefer one idea per screen? Read it as 18 slides →

If you remember one chapter from this folder, make it this one. Context engineering (deciding exactly what goes into the model on each call) is the discipline that most determines whether an agent feels sharp or scattered. The model can only reason about what's in its context window; everything else may as well not exist. This chapter is about constructing that context well, and about the memory system that feeds it.

05.1

Context is constructed, not stored

The most important reframing: context is not a thing you keep; it's a thing you build fresh for every model call. Each call, your orchestrator assembles:

  • the system prompt (identity, rules, output protocols),
  • the relevant slice of conversation history,
  • the available tools,
  • and injected information (what's available to act on, retrieved snippets, prior-turn summaries).

The model is stateless between calls. "Memory" is your database; "context" is the carefully-chosen subset of that memory you load into a given call. The skill is in the choosing.

05.2

Real code: pi rebuilds the request on every call

pi is a small, readable agent framework, and its packages/agent loop shows "constructed, not stored" in one function. Every model call goes through streamAssistantResponse, once per pass of the loop (agent-loop.ts line 242). Watch for two lists: the transcript the agent keeps, and the request it builds from that transcript.

async function streamAssistantResponse(
  context: AgentContext,
  config: AgentLoopConfig,
  signal: AbortSignal | undefined,
  emit: AgentEventSink,
  streamFunction: StreamFn,
): Promise<AssistantMessage> {
  // Apply context transform if configured (AgentMessage[] → AgentMessage[])
  let messages = context.messages;                                  // [1]
  if (config.transformContext) {
    messages = await config.transformContext(messages, signal);     // [2]
  }

  // Convert to LLM-compatible messages (AgentMessage[] → Message[])
  const llmMessages = await config.convertToLlm(messages);         // [3]

  const llmContext = normalizeContext({ messages: llmMessages });  // [4]

  // Resolve API key (important for expiring tokens)
  const resolvedApiKey =
    (config.getApiKey ? await config.getApiKey(config.model.provider) : undefined) || config.apiKey; // [5]

  const response = await streamFunction(config.model, llmContext, {
    ...config,
    apiKey: resolvedApiKey,
    signal,
  });

Source: earendil-works/pi, packages/agent/src/agent-loop.ts (MIT)

What to notice

  1. [1] context is an AgentContext: the transcript (messages) plus the tools for this run. This line aliases context.messages. If transformContext returns a new array, the rest of the request uses that array, and the trimmed list is never assigned back. The assistant reply is still pushed onto context.messages. A transform that edits the array in place and returns it would change the stored transcript too. The test avoids that by returning slice, which is a new array.
  2. [2] config is the AgentLoopConfig, the settings and hooks for the run. transformContext is optional. Its doc comment names two jobs: pruning old messages, and injecting context from outside. It runs before every call.
  3. [3] An AgentMessage can include kinds a provider does not understand, such as a UI notice. convertToLlm turns each entry into a system, user, assistant, or tool-result message, or drops it. The doc comment says entries the provider cannot understand should be filtered out.
  4. [4] normalizeContext wraps the list in a TranscriptContext, the shape every provider adapter in packages/ai expects. It is a new object, made for this call only.
  5. [5] The key is resolved on every call. The inline comment says the lookup matters because tokens expire. The getApiKey doc says a short-lived token can expire during tool execution.

A naive loop sends one growing array. Fitting the budget then means deleting history, and UI-only entries reach the model. Splitting "what we keep" from "what we send" avoids both.

Read it yourself: open the test "should apply transformContext before convertToLlm" in packages/agent/test/agent-loop.test.ts. It seeds four old messages, sends one new prompt, and keeps the last two. In a local clone, change slice(-2) to slice(-1) and predict the result before you run it. Both toBe(2) checks fail, because the lengths become 1.

05.3

The context window is a budget

Every token you put in context costs money and latency, and crowds out other tokens. So treat the window as a budget to allocate, not a bucket to fill. The instinct to "give the model everything just in case" is wrong: it's expensive, it's slow, and, counterintuitively, it often degrades quality, because important details get diluted among irrelevant ones (the "lost in the middle" effect). Curate ruthlessly.

The context window is a budget
The context window is a budget

05.4

Reference, don't embed

A powerful pattern for document- or data-heavy agents: don't dump content into context; reference it and let the model pull what it needs via tools. Instead of pasting five documents into the prompt, give the model short handles ("doc-0", "doc-1") and a read_document tool. The model reads only what the task requires.

This has three benefits:

  1. Smaller prompts: you pay for content only when it's actually used.
  2. Forced grounding: the model must explicitly fetch content, which makes it engage with the real text rather than half-remembered training data.
  3. Freshness: referencing always pulls the current version, avoiding stale content (see below).

Use short, stable, human-meaningless-but-model-friendly labels for the handles, assigned deterministically, so the model can reference them reliably and can only reference things that actually exist.

05.5

Real code: OpenClaw's memory search returns addresses, not files

OpenClaw is a personal-assistant agent built on pi's agent core. Files under memory/ are not in the default bootstrap list, so they stay on disk until a tool reads them. For a private session, MEMORY.md and USER.md are also copied into the system prompt, up to 20,000 and 4,000 characters by default. memory_search returns a path and a line range, and memory_get reads that slice.

export type MemorySearchResult = {
  path: string;          // [1]
  startLine: number;
  endLine: number;
  score: number;
  vectorScore?: number;
  textScore?: number;
  snippet: string;       // [2]
  source: MemorySource;
  // ... (6 lines: optional rank and project metadata)
  citation?: string;
  provenance?: MemoryEntryProvenance;
};

Source: openclaw/openclaw, packages/memory-host-sdk/src/host/types.ts (MIT)

export function buildMemoryReadResult(params: {
  content: string;
  relPath: string;
  from?: number;
  lines?: number;
  defaultLines?: number;
  maxChars?: number;
  suggestReadFallback?: boolean;
}): MemoryReadResult {
  const fileLines = params.content.split("\n");
  // ... (5 lines: drop the empty entry a trailing newline leaves behind)
  const start = resolveIntegerOption(params.from, 1, { min: 1 });   // [3]
  const requestedCount = resolveIntegerOption(
    params.lines ?? params.defaultLines,
    DEFAULT_MEMORY_READ_LINES,                                      // [4]
    { min: 1 },
  );
  const selectedLines = fileLines.slice(start - 1, start - 1 + requestedCount);
  const moreSourceLinesRemain = start - 1 + selectedLines.length < fileLines.length; // [5]

Source: openclaw/openclaw, packages/memory-host-sdk/src/host/read-file-shared.ts (MIT)

What to notice

  1. [1] Every hit carries path, startLine, and endLine. When citations are included, the snippet gains Source: plus #L<start> for one line, or #L<start>-L<end> for a range. The default mode is auto: ordinary sessions get that line, and a session key with a group or channel segment does not.
  2. [2] snippet is a short preview. vectorScore is the meaning match, textScore is the keyword match, and score is the sort key. If the embedding lookup fails and keyword search is available, the tool returns those keyword hits only. source is memory or sessions. provenance, when set, is the origin: owner, agent, untrusted, or system.
  3. [3] memory_get takes the path back, with an optional from line and lines count. resolveIntegerOption turns a missing from, or one below 1, into line 1.
  4. [4] When lines is omitted, the reader takes up to 120 lines (DEFAULT_MEMORY_READ_LINES), then keeps only what fits in 12,000 characters. The continuation note is added after that cut.
  5. [5] moreSourceLinesRemain means the file goes on past the slice. If more lines remain and the slice itself fit the character budget, the reply ends with [More content available. Use from=N to continue.]

When both tools are on, the prompt says to search, then to memory_get the needed lines, and not to search again when the supplied context already answers. A file under memory/ costs nothing until a search reads it, and then only the lines memory_get returns. MEMORY.md is injected on every private turn, up to the bootstrap cap.

Read it yourself: open extensions/memory-core/src/memory-tool-contract.ts and read buildMemoryPromptSection. When memory_search is off and memory_get is on, the prompt sends the model straight to a known file. Then read buildMemoryReadResultFromSlice in read-file-shared.ts and see what is said when one line is longer than the character budget.

05.6

Distill external and tool data before it lands

When a tool returns a large payload (an API response, a long document, a database result), don't pour the raw firehose into context. Distill it: keep the decision-relevant fields, cap the long ones with a "there's more" flag, and offer a separate tool to fetch detail on demand. A giant unfiltered payload buries the signal and blows the budget. This is context engineering applied at the tool boundary, and it's where many agents quietly go wrong.

05.7

Real code: OpenCode caps every tool result at the boundary

OpenCode is a full coding-agent harness. In its session runner, every local tool result passes through ToolOutput.truncate on the way into the session (step.ts lines 126 to 128: run the tool, then toolOutput.truncate). It caps the payload, says there is more, and leaves a path to the rest. OpenCode is built on the Effect library, so read yield* as "wait for this step", much like await.

const truncate = Effect.fnUntraced(function* (result: Result) {
  if (result.metadata?.truncated !== undefined) return result      // [1]
  const content = result.content
  const text = content.flatMap((item) => (item.type === "text" ? [item.text] : [])).join("\n")
  const limits = state.get()
  const lines = text.split("\n")
  if (text.endsWith("\n")) lines.pop()
  const totalBytes = Buffer.byteLength(text, "utf-8")
  if (lines.length <= limits.maxLines && totalBytes <= limits.maxBytes)   // [2]
    return { ...result, metadata: { ...result.metadata, truncated: false } }

  const kept: string[] = []
  let bytes = 0
  for (const line of lines.slice(0, limits.maxLines)) {
    const size = Buffer.byteLength(line, "utf-8") + (kept.length > 0 ? 1 : 0)
    if (bytes + size > limits.maxBytes) break                      // [3]
    kept.push(line)
    bytes += size
  }
  const file = path.join(directory, fileName())
  yield* fs.ensureDir(directory).pipe(Effect.orDie)
  yield* fs.writeFileString(file, text).pipe(Effect.orDie)         // [4]
  const shown = kept.length > 0 ? `lines 1-${kept.length}` : "0 lines"
  const marker = `[showing ${shown} of ${lines.length}; full output saved to ${file}]` // [5]
  // ... (22 lines: rebuild the content parts so they hold only the kept text, then add the marker)
  return {
    ...result,
    content: bounded,
    metadata: { ...result.metadata, truncated: true, outputPath: file },   // [6]
  }
})

Source: anomalyco/opencode, packages/core/src/tool-output.ts (MIT)

What to notice

  1. [1] If metadata.truncated is already true or false, this function returns the result unchanged. The shell tool sets that flag itself, through ShellResult.metadata. It keeps the tail of the command and points at its own log.
  2. [2] The fast path. Output under both limits (2,000 lines and 50 KiB by default, adjustable through the tool_output setting) goes through unchanged, marked truncated: false.
  3. [3] Whole lines are kept from the top until either limit would be crossed, so the model never sees half a line. Buffer.byteLength counts bytes.
  4. [4] The full text is written first, to a file named tool_<id> under OpenCode's data directory. Those files are deleted after seven days.
  5. [5] The marker names the kept lines and the path of the full output, which still includes the lines already shown. That path is outside the workspace, so the read tool must be allowed to open an external directory.
  6. [6] truncated and outputPath go into the metadata, so later code can tell a cut happened.

A stored tool result is sent again on later calls, so one oversized log taxes every turn after it. A naive cut (the first N characters, then nothing) hides the loss and throws the rest away. Here the model sees the cut, and the full text is one file read away. Choosing which fields matter still belongs inside each tool: this layer does not know their shape.

Read it yourself: open packages/core/test/tool-output.test.ts. "writes oversized text and returns a bounded preview" shows the marker. "skips results that report a truncation state" shows that truncated: false is left alone too. Then answer from the loop: a single 60 KiB line under the default limits leaves the model the marker and nothing else. The shell's own tail is in packages/core/src/shell.ts. ShellResult.metadata in packages/core/src/shell/result.ts copies that boolean onto metadata.truncated.

05.8

The stale-content problem

A specific trap for agents that work with mutable data: if the model is allowed to "remember" content it read several turns ago, it will reason about a stale version after that content has changed. The fix is a discipline: require the model to re-fetch content each turn it needs it, and tell it explicitly that it does not retain content between turns. Yes, this costs a re-read. But for anything where correctness matters, and especially where the data can be edited, fresh-every-turn beats remembered-and-wrong. Reinforce the rule in both the system prompt and the tool descriptions.

05.9

Prior-turn memory: lightweight continuity

The model needs to remember what it did, even if not the full content. A cheap, effective technique: after each turn, summarise the turn's actions into a short note ("generated draft.docx → doc-1; read source.docx → doc-0") and inject that summary into the next call. This gives the model continuity, it knows the handles for things it produced and won't redo work, without re-sending everything. It's a tiny amount of context for a large coherence gain.

05.10

Managing long histories

Conversations grow past the context budget. Strategies, in rough order of preference:

  • Recent-window: include the last N turns verbatim. Simple, and often enough.
  • Summarise-and-prepend: periodically compress older turns into a running summary, keep recent turns verbatim. Preserves the gist while bounding tokens.
  • Selective recall: store all history in the database and retrieve only the turns relevant to the current message (semantic search over history). More complex; useful for very long-lived conversations.
  • Event-log replay with selective inclusion: store rich turn events durably, but include in context only what the current turn needs.

Whatever you choose, the principle holds: history lives in durable storage; you load a curated slice into context.

05.11

Layering the context

Think of context as layers assembled in order:

  1. Standing instructions: the system prompt: identity, universal rules, output protocols. Stable across calls.
  2. Task instructions: anything specific to this task (a selected workflow/template, loaded on demand rather than baked into the system prompt).
  3. Situational injection: what's available right now: the list of accessible documents, the user's current focus, retrieved snippets.
  4. History: the curated conversation slice, plus the prior-turn summary.
  5. The current user message: possibly annotated (e.g. mapping attachments to their handles).

Keeping these as distinct layers makes the assembly logic clear and lets you tune each independently. In particular, loading task instructions on demand (rather than concatenating every possible instruction into the system prompt) keeps every layer focused: the system prompt stays about universal behaviour, and task-specific detail arrives only when relevant.

05.12

Real code: OpenClaw stacks workspace files in order

OpenClaw keeps the standing workspace facts as separate Markdown files, then builds them into the system prompt. These files are the standing-instructions stack: rules, persona, identity, user preferences, and durable memory, in a fixed order. Task instructions, history, and the current message are added by other code. Watch the sort keys, then the short note in front of only three files.

const CONTEXT_FILE_ORDER = new Map<string, number>([          // [1]
  ["agents.md", 10],
  ["soul.md", 20],
  ["identity.md", 30],
  ["user.md", 40],
  ["tools.md", 50],
  ["bootstrap.md", 60],
  ["memory.md", 70],                                           // [2]
]);
// ... (40 lines: lowercase the file name, then look it up in this map)
    .toSorted((a, b) => {                                      // [3]
      if (a.order !== b.order) {
        return a.order - b.order;
      }
      if (a.basename !== b.basename) {
        return a.basename.localeCompare(b.basename);
      }
      // Preserve loader precedence for shared USER defaults and the personal overlay.
      if (a.basename === "user.md") {
        return 0;                                              // [4]
      }

Source: openclaw/openclaw, src/agents/system-prompt-context-files.ts (MIT)

  lines.push("Loaded project context:");
  for (const [basename, guidance] of Object.entries({
    "soul.md": "SOUL.md: persona/tone. Follow it unless higher-priority instructions override.",
    "memory.md":
      "MEMORY.md: durable non-profile facts and decisions; use when relevant unless higher-priority instructions override.",
    "user.md":
      "USER.md: durable user preferences and profile directives; follow unless higher-priority instructions override.",
  })) {
    if (files.some((file) => file.basename === basename)) {    // [5]
      lines.push(guidance);
    }
  }
  // ... (4 lines: a three-sentence note when a personal USER.md is present, then a blank line)
  for (const { file } of files) {
    lines.push(`## ${file.path}`, "", sanitizeContextFileContentForPrompt(file.content), ""); // [6]
  }

Source: openclaw/openclaw, src/agents/system-prompt-context-files.ts (MIT)

What to notice

  1. [1] CONTEXT_FILE_ORDER maps a lowercased file name to a number. prepareContextFilesForPrompt, in the skipped lines, stores that number on each EmbeddedContextFile (a path plus the file's text). A missing name sorts last.
  2. [2] AGENTS.md is first and MEMORY.md is last. Persona, identity, user preferences, and BOOTSTRAP.md sit between them. tools.md is in the map, but it is not in the default file list.
  3. [3] toSorted returns a new array, so the loader's order stays as it was read. Comparison uses the map number, then the file name.
  4. [4] Two USER.md files stay in loader order (return 0 means tied), so the shared file and a personal overlay keep the order the loader produced.
  5. [5] A one-line role is added only for soul, memory, and user, and only when that file is present. Those lines say they give way to higher-priority instructions. AGENTS.md gets no yield sentence. It sorts first.
  6. [6] Each file becomes a heading (its path) and then its text. sanitizeContextFileContentForPrompt strips the default heartbeat prompt out of old templates. The comment says that prompt would otherwise route Claude subscriptions to paid extra usage. The scheduled heartbeat turn still carries the real instructions.

buildSystemPrompt writes this under "Workspace Files (injected)", then appends this run's tools. A naive prompt concatenates files in directory order, with no labels, so a persona file and a memory file look the same.

Read it yourself: open the test "filters blank paths and renders typed project context in canonical order" in src/agents/system-prompt.test.ts. It passes the files out of order. Expect ## AGENTS.md before ## ./SOUL.md, that heading before ## MEMORY.md, a persona line, a memory line, and no "Blank path".

05.13

Map user references to model handles

A small but high-impact detail: when the user attaches or references something, annotate the message so the model is handed the same handle it would use to act on it. If the user attaches a file, prefix the message with "the user attached: doc-2 (contract.pdf)". Now the model knows that "the contract the user just mentioned" is doc-2 and can read it directly. Without this mapping, the model has to guess which handle corresponds to the user's intent, a needless source of error.

05.14

Memory beyond the conversation

"Memory" also includes things that persist across conversations: user preferences, organisation settings, reusable templates, and generated artifacts. These live in your data model (Chapter 9) and are loaded into context when relevant: a user's saved templates appear as available tools/workflows; their generated documents become referenceable handles. Design memory as durable state that you selectively surface, never as an ever-growing prompt.

05.15

Real code: OpenClaw surfaces memory per session

filterBootstrapFilesForSession decides which workspace files this session's prompt includes. A private chat keeps the workspace-root MEMORY.md. A group or channel drops that root file and can keep a nested MEMORY.md. A helper's prompt keeps only AGENTS.md, and a cron run keeps four names. A leaf helper can still call memory_search and memory_get.

const SUBAGENT_BOOTSTRAP_ALLOWLIST = new Set([DEFAULT_AGENTS_FILENAME]); // [1]

const CRON_BOOTSTRAP_ALLOWLIST = new Set([                       // [2]
  DEFAULT_AGENTS_FILENAME,
  DEFAULT_SOUL_FILENAME,
  DEFAULT_IDENTITY_FILENAME,
  DEFAULT_USER_FILENAME,
]);
// ... (29 lines: remove the workspace-root MEMORY.md from shared sessions)
export function filterBootstrapFilesForSession(
  files: WorkspaceBootstrapFile[],                                // [3]
  session?: string | BootstrapSessionContext,
): WorkspaceBootstrapFile[] {
  const { sessionKey, chatType, workspaceDir }: BootstrapSessionContext =
    typeof session === "string" ? { sessionKey: session } : (session ?? {});
  const isSubagent = isSubagentSessionKey(sessionKey);
  const isCron = isCronSessionKey(sessionKey);
  const effectiveChatType = chatType ?? deriveSessionChatTypeFromKey(sessionKey);
  const isNonPrivate =
    isSubagent || isCron || effectiveChatType === "group" || effectiveChatType === "channel"; // [4]
  const privacyFilteredFiles = isNonPrivate
    ? filterRootMemoryBootstrapFiles(files, workspaceDir)
    : files;
  if (isSubagent) {
    return privacyFilteredFiles.filter((file) => SUBAGENT_BOOTSTRAP_ALLOWLIST.has(file.name));
  }
  if (isCron) {
    return privacyFilteredFiles.filter((file) => CRON_BOOTSTRAP_ALLOWLIST.has(file.name)); // [5]
  }
  return privacyFilteredFiles;
}

Source: openclaw/openclaw, src/agents/workspace.ts (MIT)

What to notice

  1. [1] DEFAULT_AGENTS_FILENAME is "AGENTS.md". A helper session's prompt is reduced to that one file.
  2. [2] A cron run keeps AGENTS.md, SOUL.md, IDENTITY.md, and USER.md. MEMORY.md and BOOTSTRAP.md are not on the list.
  3. [3] A WorkspaceBootstrapFile is a name, a path, the text, and a missing flag. The second argument is a session key, or an object that can set the chat type. An explicit chat type overrides the key.
  4. [4] A helper, a cron run, a group, or a channel first drops the workspace-root MEMORY.md. A private chat skips that cut.
  5. [5] The allowlists run after that cut and match on file.name. A group keeps the other files, including USER.md. The root memory file is already gone before the cron list is applied.

Pasting MEMORY.md into every prompt puts those notes in a shared room and crowds a helper. The filter leaves the root file in the private session's prompt and out of the others. Searching memory is a separate path: the leaf-helper tool policy still allows it.

Read it yourself: open the cron test under filterBootstrapFilesForSession in src/agents/workspace-bootstrap.test-utils.ts. It keeps those four names. Then open src/agents/agent-tools.policy.test.ts and confirm a leaf helper may still call memory_search and memory_get.

05.16

The quiet truth about agent quality

Teams obsess over model choice and prompt wording, but in practice the biggest quality wins come from context engineering: giving the model short stable handles, sweeping the right artifacts into reach, injecting a prior-turn summary, distilling tool outputs, and forcing fresh reads of mutable data. None of it is glamorous. All of it is where "it just works" actually comes from. Spend your effort here.

05.17

Read the source yourself

Five files, in the order this chapter built the idea. Each link is pinned to the commit the excerpts came from.

  1. Context is built per call. In pi's packages/agent/src/agent-loop.ts, read streamAssistantResponse. Decide whether transformContext rewrites the stored transcript or only the list for this call. The local messages variable is the request. The assistant reply is appended to context.messages; the trimmed list is not written back. The test "should apply transformContext before convertToLlm" in packages/agent/test/agent-loop.test.ts is the worked example: the context holds four old messages, the loop appends the new prompt, and the transform keeps the last two.
  2. Tool output is capped at the boundary. In OpenCode's packages/core/src/tool-output.ts, read truncate, then the call site toolOutput.truncate in packages/core/src/session/runner/step.ts. If metadata.truncated is already true or false, the result comes back unchanged. The shell keeps the tail in packages/core/src/shell.ts, and packages/core/src/shell/result.ts copies that boolean onto the metadata.
  3. Layers have a fixed order. In OpenClaw's src/agents/system-prompt-context-files.ts, read CONTEXT_FILE_ORDER and buildProjectContextSection. Three names get a guidance line (soul, memory, user). MEMORY.md sorts last among the named files. buildSystemPrompt places that block under "Workspace Files (injected)" (src/agents/system-prompt.ts). The tool list for this run is the buildSystemPromptToolingSection call just below, at line 656.
  4. Memory is surfaced, not always loaded. In OpenClaw's src/agents/workspace.ts, read filterBootstrapFilesForSession. A private chat keeps the workspace-root MEMORY.md. A group or channel drops that root file. A helper's prompt keeps only AGENTS.md, and a cron run keeps four names.
  5. A hit is an address, then a slice. In OpenClaw's extensions/memory-core/src/memory-tool-contract.ts, read MEMORY_SEARCH_TOOL_CONTRACT and buildMemoryPromptSection. When both tools are on, the prompt says to search, then memory_get the needed lines. buildMemoryReadResult takes up to 120 lines when lines is omitted (read-file-shared.ts). buildMemoryReadResultFromSlice may shorten that to the character budget and append the continuation note (lines 15 to 26 and 29 to 51).

Exercise: clone pi at that commit and, in the test from step 1, change messages.slice(-2) to messages.slice(-1). Before you run it, write down which expects fail. Both toBe(2) checks fail, because the lengths become 1: the transform runs before convertToLlm, and the converter in that test only counts what the transform returned.

0 of 18 slides completed

Your learning progress stays in this browser.

Self-check · 10 questions

01 / 10

How does the chapter distinguish "context" from "memory"?

Related chapters