Skip to slide
063 / 221

CH 05 · Context Engineering and Memory · 03/18

Read as one page

Real code: pi rebuilds the request on every call

pi is a small, readable agent framework, and its packages/agent loop shows "constructed, not stored" in one function. Every model call goes through streamAssistantResponse, once per pass of the loop (agent-loop.ts line 242). Watch for two lists: the transcript the agent keeps, and the request it builds from that transcript.

async function streamAssistantResponse(
  context: AgentContext,
  config: AgentLoopConfig,
  signal: AbortSignal | undefined,
  emit: AgentEventSink,
  streamFunction: StreamFn,
): Promise<AssistantMessage> {
  // Apply context transform if configured (AgentMessage[] → AgentMessage[])
  let messages = context.messages;                                  // [1]
  if (config.transformContext) {
    messages = await config.transformContext(messages, signal);     // [2]
  }

  // Convert to LLM-compatible messages (AgentMessage[] → Message[])
  const llmMessages = await config.convertToLlm(messages);         // [3]

  const llmContext = normalizeContext({ messages: llmMessages });  // [4]

  // Resolve API key (important for expiring tokens)
  const resolvedApiKey =
    (config.getApiKey ? await config.getApiKey(config.model.provider) : undefined) || config.apiKey; // [5]

  const response = await streamFunction(config.model, llmContext, {
    ...config,
    apiKey: resolvedApiKey,
    signal,
  });

Source: earendil-works/pi, packages/agent/src/agent-loop.ts (MIT)

What to notice

  1. [1] context is an AgentContext: the transcript (messages) plus the tools for this run. This line aliases context.messages. If transformContext returns a new array, the rest of the request uses that array, and the trimmed list is never assigned back. The assistant reply is still pushed onto context.messages. A transform that edits the array in place and returns it would change the stored transcript too. The test avoids that by returning slice, which is a new array.
  2. [2] config is the AgentLoopConfig, the settings and hooks for the run. transformContext is optional. Its doc comment names two jobs: pruning old messages, and injecting context from outside. It runs before every call.
  3. [3] An AgentMessage can include kinds a provider does not understand, such as a UI notice. convertToLlm turns each entry into a system, user, assistant, or tool-result message, or drops it. The doc comment says entries the provider cannot understand should be filtered out.
  4. [4] normalizeContext wraps the list in a TranscriptContext, the shape every provider adapter in packages/ai expects. It is a new object, made for this call only.
  5. [5] The key is resolved on every call. The inline comment says the lookup matters because tokens expire. The getApiKey doc says a short-lived token can expire during tool execution.

A naive loop sends one growing array. Fitting the budget then means deleting history, and UI-only entries reach the model. Splitting "what we keep" from "what we send" avoids both.

Read it yourself: open the test "should apply transformContext before convertToLlm" in packages/agent/test/agent-loop.test.ts. It seeds four old messages, sends one new prompt, and keeps the last two. In a local clone, change slice(-2) to slice(-1) and predict the result before you run it. Both toBe(2) checks fail, because the lengths become 1.

Finished this idea?

Your learning progress stays in this browser.