Skip to content

Chapter 02 · 14 min read · Updated

The Agent Loop Pattern

The agent loop appends each request and result until the model stops or hits a safety cap. Covers common failure modes to design against.

Prefer one idea per screen? Read it as 15 slides →

The agent loop is the heart of every agent. This chapter examines it as a pattern: its essential shape, its variants, how it terminates, and the safety properties you must build in. Get the loop right and the rest of the system has a solid spine to hang off.

02.1

The essential loop

Stripped to its core, every agent loop is:

messages = [system_prompt, ...history, user_message]
repeat up to N times:
    response = model.call(messages, tools)
    if response has no tool calls:
        return response.text          # the model is done
    results = run(response.tool_calls)  # execute the actions
    messages.append(response)           # record what the model did
    messages.append(results)            # record what the tools returned

That's it. The model is given the conversation and the tool catalog; it either answers or asks for tools; if it asks, you run them, append both the request and the results, and call again. The loop ends when the model answers without requesting tools (or a cap is hit).

This simple structure is deceptively powerful: by chaining tool calls, the model can read a file, use what it learned to search an API, use those results to draft a document, and finally summarise, all from a single user message, with the loop carrying it through.

The agent loop
The agent loop

02.2

Why both the request and the result must be appended

A subtlety that trips up first implementations: when the model requests a tool, you must append both the model's tool-request turn and the tool-result turn before the next call. The providers require this pairing: the model needs to see "I asked for X, and X returned Y" to continue coherently. Most provider APIs also require that every tool call in a turn gets a corresponding result. Skipping a result, or mismatching ids, breaks the next call. A robust loop therefore guarantees a result for every call, synthesising an error result if a tool failed to produce one.

Field check

See it worked. Then try it.

Keep the request and the observation together, so the next model call can reconstruct what happened.

Worked examples

A calendar lookup returns three meetings.

Append the assistant’s lookup request and the matching tool result. The model can now connect those meetings to the requested date.

The lookup times out.

Keep the request and append a matching error result. A failure is still an observation the model needs.

Your turnA file-search tool returns two matching lines. What should the harness add before calling the model again?

02.3

Real code: Codex never sends a call without its output

Codex, OpenAI's open-source coding agent written in Rust, applies the "synthesise a result" rule right before every request to the model: the function below finds tool calls with no output and adds one. Watch where it puts each fix.

pub(crate) fn ensure_call_outputs_present(items: &mut Vec<ResponseItemEnvelope>) {  // [1]
    let mut function_output_ids = HashSet::new();
    // ... (22 lines: record the call_id of every output already in the history)

    // Collect synthetic outputs to insert immediately after their calls.
    // Store the insertion position (index of call) alongside the item so
    // we can insert in reverse order and avoid index shifting.
    let mut missing_outputs_to_insert: Vec<(usize, ResponseItemEnvelope)> = Vec::new();

    for (idx, envelope) in items.iter().enumerate() {
        match &envelope.item {
            ResponseItem::FunctionCall { id, call_id, .. }
                if !function_output_ids.contains(call_id.as_str()) =>  // [2]
            {
                info!("Function call output is missing for call id: {call_id}");
                missing_outputs_to_insert.push((
                    idx,
                    ResponseItemEnvelope::new(ResponseItem::FunctionCallOutput {
                        id: synthetic_output_id("fco", id.as_deref()),
                        call_id: Some(call_id.clone()),
                        name: None,
                        namespace: None,
                        output: FunctionCallOutputPayload::from_text("aborted".to_string()),  // [3]
                        internal_chat_message_metadata_passthrough: None,
                    }),
                ));
            }
            // ... (81 lines: tool-search inserts an empty completed output; custom-tool and local-shell insert "aborted", unless a debug build panics first because the call is not marked inherited)
    // Insert synthetic outputs in reverse index order to avoid re-indexing.
    for (idx, output_item) in missing_outputs_to_insert.into_iter().rev() {
        items.insert(idx + 1, output_item);  // [4]
    }
}

Source: openai/codex, codex-rs/core/src/context_manager/normalize.rs (Apache-2.0)

What to notice

  1. [1] items is the conversation history: each ResponseItemEnvelope wraps one ResponseItem (a message, a tool call, a tool output) plus metadata, and &mut lets the function edit the list in place. The skipped lines collect the call_id of every existing output into sets; a call_id is the id that ties a call to its output.
  2. [2] A second pass visits every call. A FunctionCall whose call_id is not in the set has no output.
  3. [3] The made-up output is the text "aborted", and success is left unset (None), so the next request shows that the call produced no normal result. synthetic_output_id derives its id from the call's own id, so the same history always produces the same request and the provider's prompt cache (its saved work on a prompt it has seen) stays valid.
  4. [4] Each fix goes in at idx + 1, straight after its call, keeping the "I asked for X, X returned Y" order. Every insert shifts later positions by one, so the loop works from the back of the list forward.

This is a net under a net. Cancelling a function or custom tool already records an output whose text contains "aborted by user" (core/src/tools/parallel.rs); a cancelled tool-search call records an empty completed output instead. This function catches what still slips through, such as a conversation forked while a call was running. It runs inside for_prompt on a copy of the history, so only the outgoing request is patched. Without it, one missing output would stay in the history and break every later request.

Read it yourself: in history.rs, read the comment on normalize_history. Its second rule is the mirror problem, an output whose call is gone: find remove_orphan_outputs in normalize.rs to see the fix, then read the test normalize_mixed_inserts_and_removals in history_tests.rs. That test is compiled only without debug assertions; a debug build has normalize_mixed_inserts_and_removals_panics_in_debug instead. Question: for a custom-tool call that is not marked inherited_user_message, the branch calls error_or_panic instead of info!. Local-shell uses the same guard. What does error_or_panic (in core/src/util.rs) do in a debug build?

02.4

Termination and the safety cap

The loop needs two stopping conditions:

  1. Natural termination: the model produces a final answer with no tool calls.
  2. A hard iteration cap: a maximum number of loop turns (commonly 5–15).

The cap is non-negotiable. Without it, a confused model can loop forever, calling a tool, getting a result it doesn't like, calling again, ad infinitum, burning tokens and money. The cap converts "infinite loop" into "bounded, eventually-terminating process." When the cap is hit, return whatever the model has produced so far (or a graceful "I wasn't able to complete this") rather than erroring.

A second guard worth considering: detect non-productive loops: the same tool called with the same arguments repeatedly, and break early. The model is signalling it's stuck.

02.5

Real code: the OpenAI Agents SDK caps turns

The OpenAI Agents SDK, a Python library for building agents, owns the loop. Runner.run(agent, input) hands the work to AgentRunner, and that class's while True: loop calls the model and runs the tools. The cap is a parameter of the call, max_turns, defaulting to 10 (DEFAULT_MAX_TURNS in run_config.py). Watch what happens on the turn that crosses the line.

current_turn += 1  # [1]
if max_turns is not None and current_turn > max_turns:  # [2]
    # ... (7 lines: mark the tracing span "Max turns exceeded")
    max_turns_error = MaxTurnsExceeded(f"Max turns ({max_turns}) exceeded")
    run_error_data = build_run_error_data(  # [3]
        input=original_input,
        new_items=session_items,
        raw_responses=model_responses,
        last_agent=current_agent,
        reasoning_item_id_policy=resolved_reasoning_item_id_policy,
    )
    handler_result = await resolve_run_error_handler_result(  # [4]
        error_handlers=error_handlers,
        error_kind="max_turns",
        error=max_turns_error,
        context_wrapper=context_wrapper,
        run_data=run_error_data,
    )
    if handler_result is None:
        raise max_turns_error  # [5]
    # ... (58 lines: check the handler's output and save it to the session)
    result = RunResult(
        # ... (3 lines: input, new_items, raw_responses)
        final_output=validated_output,  # [6]

Source: openai/openai-agents-python, src/agents/run.py (MIT)

What to notice

  1. [1] This runs once per pass, after any pending input is admitted and before run_single_turn calls the model. A turn here is one model call plus the tool calls it makes (the docstring of Runner.run says so), so the call past the limit is never made or paid for.
  2. [2] Pass max_turns=None and there is no cap. Capped is the default; unlimited is a choice you write down.
  3. [3] build_run_error_data packs up what the run produced so far: the input, the new items (messages, tool calls, tool outputs), the raw model responses, and the last agent that ran. That bundle is the handler's input, a snapshot of the run so far. Runner.run does not return it: the default path raises, and a handler returns its own final_output.
  4. [4] This looks up your function under "max_turns" in the error_handlers dict you passed to Runner.run, and calls it with the error and that snapshot.
  5. [5] With no handler, or one that returns None, the SDK raises MaxTurnsExceeded. That differs from the section's advice: out of the box, hitting the cap is an exception your code must catch.
  6. [6] With a handler, the skipped lines check its final_output against the agent's output type and run the output guardrails (checks on an agent's final answer). If both accept it, the SDK returns a normal RunResult. If the agent expects structured output and the handler's value does not match, validate_handler_final_output raises UserError, the SDK's error for bad caller input. A guardrail trip still raises OutputGuardrailTripwireTriggered.

The shape guards against a runaway bill (the cap is on unless you remove it) and a silent stop (failure is loud unless you write the fallback).

Read it yourself: in docs/running_agents.md, read "Error handlers" for a sample on_max_turns. Then search run_internal/run_loop.py for MaxTurnsExceeded(: the streaming runner repeats this check but does not raise there without a handler. What does it do instead, and where does the error reach the caller? (Hint: _check_errors in result.py.)

02.6

Where the loop lives

A key design decision: does your orchestrator drive the loop, or does the provider's SDK? There are two common arrangements:

  • You own the loop explicitly: your code does the repeat and decides when to stop. Maximum control and visibility; you can inject logic between turns.
  • The provider SDK owns the loop and calls back to you for tool execution: you supply a "run these tools" function and the SDK iterates internally.

A clean hybrid (and a good default) is to own a thin loop that delegates tool execution through a callback, so your orchestrator stays declarative while the provider-specific iteration is encapsulated. Either way, the control (the cap, the termination logic, the event recording) should be yours, not buried in a library.

02.7

Variants of the loop

The basic single loop scales surprisingly far, but you'll encounter variants:

Single-loop (ReAct-style)

One model, one loop, tools available. The model reasons and acts in an interleaved stream. This is the workhorse and is sufficient for most agents, including document assistants. Start here; don't reach for more complexity until you've proven you need it.

Planner–executor

A first model call produces a plan (a list of steps), then a loop executes each step, possibly with its own sub-loops. Useful when tasks are long and benefit from explicit decomposition, or when you want the plan visible/editable by the user. The cost is added complexity and latency; the benefit is structure and steerability.

Multi-agent

Several specialised agents (each its own loop, prompt, and tools) coordinate: a "researcher" hands off to a "writer," or a "supervisor" routes work to "workers." Powerful for genuinely heterogeneous tasks, but it multiplies cost, latency, and failure modes, and the coordination overhead is real. Most products that think they need multi-agent actually need better tools and context in a single loop. Reach for it only when sub-tasks are truly independent and benefit from isolation.

Reflection / critique loops

After producing output, the agent critiques its own work and revises. Improves quality on hard tasks at the cost of extra calls. Often implemented as an explicit "now check your answer" step rather than a separate agent.

02.8

The event timeline

A loop that just returns final text throws away everything interesting that happened along the way. A production loop should emit an ordered event timeline as it runs: "model reasoned," "called tool X," "tool X returned," "model wrote text." This timeline is gold:

  • It drives a rich UI (showing tool activity as it happens, Chapter 8).
  • It's the durable record of the turn (persisted as the conversation's content, Chapter 9).
  • It's the raw material for observability and debugging (Chapter 15).

Design the loop to record an explicit, typed event for every meaningful action, in order. Don't try to reconstruct what happened from the final text afterward; capture it as it occurs.

02.9

Real code: pi's typed event timeline

pi is a small open-source coding agent whose loop lives in packages/agent. Every action the loop takes is reported as one value of a single type, AgentEvent: the timeline this section describes, written as a type. Watch how the events nest: run, then turn, then message and tool.

export type AgentEvent =
  // Agent lifecycle
  | { type: "agent_start" }  // [1]
  | { type: "agent_end"; messages: AgentMessage[] }
  // Turn lifecycle - a turn is one assistant response + any tool calls/results
  | { type: "turn_start" }  // [2]
  | { type: "turn_end"; message: AgentMessage; toolResults: ToolResultMessage[] }
  // Message lifecycle - emitted for system, user, assistant, and toolResult messages
  | { type: "message_start"; message: AgentMessage }  // [3]
  // Only emitted for assistant messages during streaming
  | { type: "message_update"; message: AgentMessage; assistantMessageEvent: AssistantMessageEvent }  // [4]
  | { type: "message_end"; message: AgentMessage }
  // Tool execution lifecycle
  | { type: "tool_execution_start"; toolCallId: string; toolName: string; args: any }  // [5]
  | { type: "tool_execution_update"; toolCallId: string; toolName: string; args: any; partialResult: any }
  | {
      type: "tool_execution_end";
      toolCallId: string;
      toolName: string;
      result: any;
      isError: boolean;  // [6]
      /** Milliseconds `execute()` took, measured with a monotonic clock; absent when the tool did not run. */
      durationMs?: number;
    };

Source: earendil-works/pi, packages/agent/src/types.ts (MIT)

What to notice

  1. [1] agent_start and agent_end bracket one whole run, from the prompt until the loop stops. On the normal path, agent_end carries newMessages: the prompt plus everything the run appended. If the executor throws, handleRunFailure (the catch around that run) emits agent_end with only the synthetic failure message. AgentMessage is pi's message type: the usual roles plus custom kinds an app can add.
  2. [2] A turn is one pass of this chapter's loop: one model reply plus the tool calls it caused. turn_end hands over the reply and the tool results from that pass. An error or aborted stop emits turn_end with an empty result list and does not run tool calls still sitting in the message.
  3. [3] Every message gets message_start and message_end, even one that arrives complete, so a consumer can rely on one rule: a message is final at message_end.
  4. [4] Only the assistant reply streams. Its AssistantMessageEvent carries pieces of text, thinking, or a tool call as they arrive (text_delta, toolcall_delta, and so on).
  5. [5] toolCallId lets a UI match start, progress (partialResult) and end even when several tools run in parallel.
  6. [6] Failure is a field, not a sentence to parse. A monotonic clock, used for durationMs, is a timer that does not jump when someone changes the system time.

Every member has a type string, so switch (event.type) tells TypeScript which fields exist in that branch. Reading event.toolCallId in the turn_end case fails to compile. A log of sentences such as "called tool X" cannot do that, because every reader of the log has to parse text.

Read it yourself: read processEvents in packages/agent/src/agent.ts. The agent's message list grows only on message_end, and the same event writes the saved session (search packages/coding-agent/src/core/agent-session.ts for Handle session persistence). The terminal UI in interactive-mode.ts switches on these same type values. One stream feeds both. Question: which two events would you use to show and hide a spinner next to a running tool?

02.10

Statelessness within, state at the edges

The loop itself is best kept stateless: it takes the assembled context in, runs, and returns a result plus events. Persistence happens at the edges: load history before the loop, save the outcome after. This keeps the loop a pure function of its inputs, which makes it testable, retryable, and easy to reason about. (More in Chapter 9.)

02.11

Common failure modes to design against

  • Runaway iteration → the hard cap.
  • Stuck loops (same call repeated) → repetition detection.
  • Missing tool results → guarantee one result per call.
  • Malformed tool arguments → parse defensively, default to empty, return a useful error the model can recover from.
  • Tool exceptions → catch them and feed the error back as a tool result, so the model can adapt rather than the whole turn crashing.

The theme: the loop is where the model's unreliability meets your code, so the loop is where you must be most defensive.

02.12

Real code: Gemini CLI spots a stuck loop

Gemini CLI, Google's open-source terminal agent, watches every tool call the model requests. The check below is the stuck-loop item from the list above, extended from one repeated call to a repeating cycle.

private checkToolCallLoop(toolCall: { name: string; args: object }): boolean {
  const key = this.getToolCallKey(toolCall);  // [1]
  this.toolCallHistory.push(key);

  // ... (4 lines: keep only the newest 25 keys)

  const n = this.toolCallHistory.length;
  const R = TOOL_CALL_LOOP_THRESHOLD; // 5

  // Check for repeating patterns of cycle length k from 1 to 5
  for (let k = 1; k <= 5; k++) {  // [2]
    const requiredLength = k * R;
    if (n >= requiredLength) {
      const cycle = this.toolCallHistory.slice(-k);
      let isPatternMatch = true;

      for (let i = 0; i < requiredLength; i++) {
        const indexFromEnd = requiredLength - i;
        const actualKey = this.toolCallHistory[n - indexFromEnd];
        const expectedKey = cycle[i % k];  // [3]
        if (actualKey !== expectedKey) {
          isPatternMatch = false;
          break;
        }
      }

      if (isPatternMatch) {
        return true;  // [4]
      }
    }
  }

  return false;
}

Source: google-gemini/gemini-cli, packages/core/src/services/loopDetectionService.ts (Apache-2.0)

What to notice

  1. [1] getToolCallKey fingerprints a call: the tool name plus JSON.stringify(args), hashed with SHA-256. Two calls match only when both are identical. The skipped lines keep the newest 25 keys, five cycle lengths times five repeats.
  2. [2] k is the cycle length. k = 1 catches the same call five times. k = 2 catches A, B, A, B across ten calls. A check for the identical call in a row misses that second shape, and flipping between two actions is a common way agents get stuck.
  3. [3] The window is the last k * 5 keys, and cycle is its last k. cycle[i % k] is what position i must be if that cycle repeats. One mismatch ends the test for this k.
  4. [4] A match returns true. addAndCheck stores Repeated tool call: ... with the arguments on the result. The log records the loop type and strike count; the arguments are sent to the model in the recovery message.

Five different files in a row do not match, because the arguments differ. On a match, core/client.ts gives the model one chance: _recoverFromLoop stops the reply and sends a message asking it to step back. A second detection yields LoopDetected, which the agent layer reports as the error "Loop detected, stopping execution." That error has fatal set to false, so it is not marked as the reason the whole agent halts, and this turn does not continue.

Read it yourself: in loopDetectionService.test.ts, read the tests for a cycle of length 2 and for alternating sequences that vary. Then answer from getToolCallKey: do {"path": "a", "limit": 10} and {"limit": 10, "path": "a"} hash the same? (JSON.stringify keeps key order.) After 30 turns on one prompt the service also asks a model, using LOOP_DETECTION_SYSTEM_PROMPT at the top of the file. What can that model catch that a hash cannot?

02.13

Real code: pi turns every tool failure into a result

pi handles the last two items on the list, malformed arguments and tool exceptions, with one rule: no failure leaves the tool step as an exception. Every failure becomes a tool result marked isError: true that goes back to the model. Two functions in pi's loop file enforce it. The first checks a call before it runs:

const tool = tools.find((t) => t.name === toolCall.name);  // [1]
if (!tool) {
  return {
    kind: "immediate",
    result: createErrorToolResult(`Tool ${toolCall.name} not found`),
    isError: true,
  };
}

try {
  const preparedToolCall = prepareToolCallArguments(tool, toolCall);
  const validatedArgs = validateToolArguments(tool, preparedToolCall);  // [2]
  // ... (36 lines: a permission hook may block the call; checks for a cancelled run)
  return {
    kind: "prepared",
    toolCall,
    tool,
    args: validatedArgs,
  };
} catch (error) {
  return {
    kind: "immediate",
    result: createErrorToolResult(error instanceof Error ? error.message : String(error)),  // [3]
    isError: true,
  };
}

Source: earendil-works/pi, packages/agent/src/agent-loop.ts (MIT)

The second runs it:

try {
  const result = await prepared.tool.execute(  // [4]
    prepared.toolCall.id,
    prepared.args as never,
    signal,
    // ... (4 lines: pass progress updates through)
  );
  // ... (3 lines: stop the timer, stop accepting progress updates)
  return { result, isError: result.isError === true, durationMs };
} catch (error) {
  // ... (3 lines: the same timer and progress cleanup)
  return {
    result: createErrorToolResult(error instanceof Error ? error.message : String(error)),  // [5]
    isError: true,
    durationMs,
  };

Source: earendil-works/pi, packages/agent/src/agent-loop.ts (MIT)

What to notice

  1. [1] The model can name a tool that does not exist. The answer is an "immediate" outcome, a result that is ready without running anything. createErrorToolResult just wraps a message as text.
  2. [2] prepareToolCallArguments runs first. If the tool defines prepareArguments, that hook may rewrite toolCall.arguments before the check. validateToolArguments then checks the arguments against the tool's schema. On a mismatch it throws a message that lists each bad field and repeats the arguments the model sent: exactly what the model needs to fix its next call.
  3. [3] The catch turns that throw into an error result. A "prepared" outcome means the call passed every check and may run.
  4. [4] execute is the tool's own code. signal is the run's abort signal, and it may be undefined. The comment on execute in types.ts tells tool authors to throw on failure or return a result with isError: true, so a throw is a normal way for a tool to fail, not an accident.
  5. [5] The error's message becomes the result, and durationMs is still recorded.

On these two paths the loop emits tool_execution_end and appends a result with the call's id. If the run is aborted mid-batch, the sequential runner stops and the remaining calls in that batch get no result. A loop that lets the exception escape loses the whole turn over one bad file path, and the model never learns what went wrong.

Read it yourself: in the same file, read failToolCallsFromTruncatedMessage. When a reply is cut off by the output token limit, the arguments may parse but be incomplete, so pi runs none of the calls and asks the model to re-issue them. Then read finalizeExecutedToolCall: what happens if your own afterToolCall hook throws?

02.14

Read the source yourself

A reading tour, in order, that walks the loop from the outside in:

  1. The whole loop in one function. pi's runLoop is this chapter's pseudocode at production size. Find where hasMoreToolCalls is set and where turn_end is emitted, and map each to a line of the essential loop.
  2. A loop the library owns. The OpenAI Agents SDK's AgentRunner.run runs while True:. Follow what happens on a handoff (one agent passing control to another): the loop changes current_agent and goes round again.
  3. History repair before every request. Callers clone the history, then Codex's for_prompt takes that copy and normalizes it. List every rule normalize_history enforces.
  4. The other loop detector. Gemini CLI's checkContentLoop catches a model repeating the same text while it streams. Find why it switches itself off inside code blocks.

Exercise: write the essential loop from the start of this chapter as a small Python run_agent(model, tools, max_turns), then add three things from these repos: an "aborted" result for any call left without one, a cycle check for cycle lengths 1 to 3, and an on_max_turns callback that returns a message instead of a fixed string. Then write a fake model that alternates between two tool calls and confirm your detector stops it.

0 of 15 slides completed

Your learning progress stays in this browser.

Self-check · 10 questions

01 / 10

The agent loop terminates under which two conditions?

Related chapters