Skip to slide
019 / 170

CH 02 · The Agent Loop · 07/11

Read as one page

Real code: pi's two loops, and how a user steers mid-turn

pi (earendil-works/pi) keeps a reusable loop in packages/agent. Your toy has one loop, and you cannot say anything until it returns. runLoop nests two, and the gap between laps is where a mid-stream correction gets in. pi uses "turn" for one lap (one reply plus its tool calls), not for the whole request.

// Check for steering messages at start (user may have typed while waiting)
let pendingMessages: AgentMessage[] = (await config.getSteeringMessages?.()) || [];

// Outer loop: continues when queued follow-up messages arrive after agent would stop
while (true) { // [1]
  let hasMoreToolCalls = true;

  // Inner loop: process tool calls and steering messages
  while (hasMoreToolCalls || pendingMessages.length > 0) { // [2]
    // ... (26 lines: between laps, a hook may swap in a new context or model)
    // Process prepared and queued messages before the next assistant response.
    for (const message of declareToolChanges(currentContext, [...preparedMessages, ...pendingMessages])) {
      // ... (2 lines: announce the message to listeners)
      currentContext.messages.push(message); // [3]
      newMessages.push(message);
    }
    pendingMessages = [];
    // ... (23 lines: a hook may edit the request just before it is sent)
    // Stream assistant response
    const message = await streamAssistantResponse(currentContext, config, signal, emit, streamFunction);
    newMessages.push(message);

    if (message.stopReason === "error" || message.stopReason === "aborted") { // [4]
      // ... (9 lines: close the turn and the run)
      return;
    }
    // ... (38 lines: run the tool calls, shown on the next slide, then close the lap)
    pendingMessages = (await config.getSteeringMessages?.()) || []; // [5]
    // ... (3 lines: if this lap will run again, drop a one-off continuation)
  }

  // Agent would stop here. Check for follow-up messages.
  const followUpMessages = (await config.getFollowUpMessages?.()) || []; // [6]
  if (followUpMessages.length > 0) {
    // ... (2 lines)
    pendingMessages = followUpMessages;
    continue;
  }
  // ... (8 lines: one more lap if a finishTurn hook asked to continue)
  // No more messages, exit
  break;
}

Source: earendil-works/pi, packages/agent/src/agent-loop.ts (MIT)

What to notice

  1. [1] The outer while (true) usually runs once. It repeats when follow-up messages are waiting (see [6]). It also runs one extra lap when a finishTurn hook returns continue and this lap did not already schedule tool results or steering.
  2. [2] The inner loop is your toy's loop. hasMoreToolCalls starts true, so the first lap always happens, and the tool code sets it again after each reply. The second condition keeps going when you have said something new.
  3. [3] Messages waiting for this lap, both steering and anything prepareNextTurn prepared, go into currentContext.messages and into newMessages. declareToolChanges tells the model about a tool-set change, either by inserting a system message or by filling toolsAdded and toolsRemoved on a system message that is already queued.
  4. [4] stopReason records why the model stopped writing. "error" (the reply failed) and "aborted" (the call was cancelled) end the run immediately, and no tool from that reply runs.
  5. [5] After the tool calls, the loop asks for steering messages. Agent.steer() in agent.ts fills that queue. Those tool calls still run, and your correction is appended before the next model call.
  6. [6] Follow-ups wait longer. Agent.followUp() queues messages that are delivered only when the inner loop is done, and continue restarts it.

Read it yourself: in packages/agent/src/agent.ts, read PendingMessageQueue. In the default "one-at-a-time" mode, each drain returns one message, and the loop drains once per lap after tools. Type three corrections during one slow tool call and expect them on three later model calls. All three are in the prompt starting with the third of those calls. Then give your toy a one-at-a-time steering list.

Finished this idea?

Your learning progress stays in this browser.