Skip to slide
020 / 170

CH 02 · The Agent Loop · 08/11

Read as one page

Real code: pi refuses to run a tool call that got cut off

Your toy runs any tool call it gets, as the last bullet above admits. These lines from the same pi loop replace your TOOLS[name](**args), and they hold back on one specific failure your toy cannot even represent: a reply that was cut off mid-call.

// Check for tool calls
const toolCalls = message.content.filter((c) => c.type === "toolCall"); // [1]

const toolResults: ToolResultMessage[] = [];
hasMoreToolCalls = false; // [2]
if (toolCalls.length > 0) {
  // A "length" stop means the output was cut off by the token limit, so
  // every tool call in the message may carry truncated arguments. Fail
  // them all instead of executing potentially borked calls.
  const executedToolBatch =
    message.stopReason === "length" // [3]
      ? await failToolCallsFromTruncatedMessage(toolCalls, emit)
      : await executeToolCalls(currentContext, message, config, signal, emit); // [4]
  toolResults.push(...executedToolBatch.messages);
  hasMoreToolCalls = !executedToolBatch.terminate; // [5]

  for (const result of toolResults) {
    currentContext.messages.push(result); // [6]
    newMessages.push(result);
  }
}

Source: earendil-works/pi, packages/agent/src/agent-loop.ts (MIT)

What to notice

  1. [1] A real reply is a list of content blocks (text, thinking, tool calls), and it can hold several tool calls at once. Your toy's response["type"] was one or the other.
  2. [2] No tool calls leaves hasMoreToolCalls false, which ends the inner loop unless steering is waiting. That is pi's "plain text means done".
  3. [3] "length" means the reply hit the output token limit (the most tokens the model may write in one reply). While a tool call streams as JSON, pi fills its arguments with parseStreamingJson, which uses the partial-json library and will return an object from incomplete JSON. When that block is marked finished, the parser runs parseToolCallArguments, which discards the partial object and stores {} if the JSON is not valid. If the reply is cut off before the block is finished, the salvaged object can remain. Suppose a 300-line file is cut off at line 180: the call can look like a normal argument holding half a file. That is why a "length" stop runs none of the calls. failToolCallsFromTruncatedMessage answers every call with an error that ends "Re-issue the tool call with complete arguments."
  4. [4] Otherwise executeToolCalls runs them: in parallel by default, one at a time if the config or any requested tool asks for sequential mode.
  5. [5] terminate is true only when every result in the batch asks the agent to stop, which a tool or a hook can request. Normally none do, so the loop goes round again to show the model its results.
  6. [6] Each result enters the history carrying the toolCallId of the call it answers: pi's version of the call_id link from "How it works".

Read it yourself: in the same file, read failToolCallsFromTruncatedMessage and shouldTerminateToolBatch. If the model sends three tool calls and only one result sets terminate, does the agent stop? Then give your fake_model a stop_reason field and make the toy answer each call with an error when it is "length".

Finished this idea?

Your learning progress stays in this browser.