CH 05 · Context Engineering and Memory · 03/18
Read as one pageReal code: pi rebuilds the request on every call
pi is a small, readable agent framework, and its packages/agent loop shows "constructed, not stored" in one function. Every model call goes through streamAssistantResponse, once per pass of the loop (agent-loop.ts line 242). Watch for two lists: the transcript the agent keeps, and the request it builds from that transcript.
async function streamAssistantResponse(
context: AgentContext,
config: AgentLoopConfig,
signal: AbortSignal | undefined,
emit: AgentEventSink,
streamFunction: StreamFn,
): Promise<AssistantMessage> {
// Apply context transform if configured (AgentMessage[] → AgentMessage[])
let messages = context.messages; // [1]
if (config.transformContext) {
messages = await config.transformContext(messages, signal); // [2]
}
// Convert to LLM-compatible messages (AgentMessage[] → Message[])
const llmMessages = await config.convertToLlm(messages); // [3]
const llmContext = normalizeContext({ messages: llmMessages }); // [4]
// Resolve API key (important for expiring tokens)
const resolvedApiKey =
(config.getApiKey ? await config.getApiKey(config.model.provider) : undefined) || config.apiKey; // [5]
const response = await streamFunction(config.model, llmContext, {
...config,
apiKey: resolvedApiKey,
signal,
});
Source: earendil-works/pi, packages/agent/src/agent-loop.ts (MIT)
What to notice
- [1]
contextis anAgentContext: the transcript (messages) plus the tools for this run. This line aliasescontext.messages. IftransformContextreturns a new array, the rest of the request uses that array, and the trimmed list is never assigned back. The assistant reply is still pushed ontocontext.messages. A transform that edits the array in place and returns it would change the stored transcript too. The test avoids that by returningslice, which is a new array. - [2]
configis theAgentLoopConfig, the settings and hooks for the run.transformContextis optional. Its doc comment names two jobs: pruning old messages, and injecting context from outside. It runs before every call. - [3] An
AgentMessagecan include kinds a provider does not understand, such as a UI notice.convertToLlmturns each entry into a system, user, assistant, or tool-result message, or drops it. The doc comment says entries the provider cannot understand should be filtered out. - [4]
normalizeContextwraps the list in aTranscriptContext, the shape every provider adapter inpackages/aiexpects. It is a new object, made for this call only. - [5] The key is resolved on every call. The inline comment says the lookup matters because tokens expire. The
getApiKeydoc says a short-lived token can expire during tool execution.
A naive loop sends one growing array. Fitting the budget then means deleting history, and UI-only entries reach the model. Splitting "what we keep" from "what we send" avoids both.
Read it yourself: open the test "should apply transformContext before convertToLlm" in packages/agent/test/agent-loop.test.ts. It seeds four old messages, sends one new prompt, and keeps the last two. In a local clone, change slice(-2) to slice(-1) and predict the result before you run it. Both toBe(2) checks fail, because the lengths become 1.
Finished this idea?
Your learning progress stays in this browser.