Chapter 02 · 14 min read · Updated
The Agent Loop Pattern
The agent loop appends each request and result until the model stops or hits a safety cap. Covers common failure modes to design against.
Prefer one idea per screen? Read it as 15 slides →
In this chapter14
- 02.1The essential loop
- 02.2Why both the request and the result must be appended
- 02.3Real code: Codex never sends a call without its output
- 02.4Termination and the safety cap
- 02.5Real code: the OpenAI Agents SDK caps turns
- 02.6Where the loop lives
- 02.7Variants of the loop
- 02.8The event timeline
- 02.9Real code: pi's typed event timeline
- 02.10Statelessness within, state at the edges
- 02.11Common failure modes to design against
- 02.12Real code: Gemini CLI spots a stuck loop
- 02.13Real code: pi turns every tool failure into a result
- 02.14Read the source yourself
The agent loop is the heart of every agent. This chapter examines it as a pattern: its essential shape, its variants, how it terminates, and the safety properties you must build in. Get the loop right and the rest of the system has a solid spine to hang off.
02.1
The essential loop
Stripped to its core, every agent loop is:
messages = [system_prompt, ...history, user_message]
repeat up to N times:
response = model.call(messages, tools)
if response has no tool calls:
return response.text # the model is done
results = run(response.tool_calls) # execute the actions
messages.append(response) # record what the model did
messages.append(results) # record what the tools returned
That's it. The model is given the conversation and the tool catalog; it either answers or asks for tools; if it asks, you run them, append both the request and the results, and call again. The loop ends when the model answers without requesting tools (or a cap is hit).
This simple structure is deceptively powerful: by chaining tool calls, the model can read a file, use what it learned to search an API, use those results to draft a document, and finally summarise, all from a single user message, with the loop carrying it through.
02.2
Why both the request and the result must be appended
A subtlety that trips up first implementations: when the model requests a tool, you must append both the model's tool-request turn and the tool-result turn before the next call. The providers require this pairing: the model needs to see "I asked for X, and X returned Y" to continue coherently. Most provider APIs also require that every tool call in a turn gets a corresponding result. Skipping a result, or mismatching ids, breaks the next call. A robust loop therefore guarantees a result for every call, synthesising an error result if a tool failed to produce one.
Field check
See it worked. Then try it.
Keep the request and the observation together, so the next model call can reconstruct what happened.
Worked examples
A calendar lookup returns three meetings.
Append the assistant’s lookup request and the matching tool result. The model can now connect those meetings to the requested date.
The lookup times out.
Keep the request and append a matching error result. A failure is still an observation the model needs.
02.3
Real code: Codex never sends a call without its output
Codex, OpenAI's open-source coding agent written in Rust, applies the "synthesise a result" rule right before every request to the model: the function below finds tool calls with no output and adds one. Watch where it puts each fix.
pub(crate) fn ensure_call_outputs_present(items: &mut Vec<ResponseItemEnvelope>) { // [1]
let mut function_output_ids = HashSet::new();
// ... (22 lines: record the call_id of every output already in the history)
// Collect synthetic outputs to insert immediately after their calls.
// Store the insertion position (index of call) alongside the item so
// we can insert in reverse order and avoid index shifting.
let mut missing_outputs_to_insert: Vec<(usize, ResponseItemEnvelope)> = Vec::new();
for (idx, envelope) in items.iter().enumerate() {
match &envelope.item {
ResponseItem::FunctionCall { id, call_id, .. }
if !function_output_ids.contains(call_id.as_str()) => // [2]
{
info!("Function call output is missing for call id: {call_id}");
missing_outputs_to_insert.push((
idx,
ResponseItemEnvelope::new(ResponseItem::FunctionCallOutput {
id: synthetic_output_id("fco", id.as_deref()),
call_id: Some(call_id.clone()),
name: None,
namespace: None,
output: FunctionCallOutputPayload::from_text("aborted".to_string()), // [3]
internal_chat_message_metadata_passthrough: None,
}),
));
}
// ... (81 lines: tool-search inserts an empty completed output; custom-tool and local-shell insert "aborted", unless a debug build panics first because the call is not marked inherited)
// Insert synthetic outputs in reverse index order to avoid re-indexing.
for (idx, output_item) in missing_outputs_to_insert.into_iter().rev() {
items.insert(idx + 1, output_item); // [4]
}
}
Source: openai/codex, codex-rs/core/src/context_manager/normalize.rs (Apache-2.0)
What to notice
- [1]
itemsis the conversation history: eachResponseItemEnvelopewraps oneResponseItem(a message, a tool call, a tool output) plus metadata, and&mutlets the function edit the list in place. The skipped lines collect thecall_idof every existing output into sets; acall_idis the id that ties a call to its output. - [2] A second pass visits every call. A
FunctionCallwhosecall_idis not in the set has no output. - [3] The made-up output is the text
"aborted", andsuccessis left unset (None), so the next request shows that the call produced no normal result.synthetic_output_idderives its id from the call's own id, so the same history always produces the same request and the provider's prompt cache (its saved work on a prompt it has seen) stays valid. - [4] Each fix goes in at
idx + 1, straight after its call, keeping the "I asked for X, X returned Y" order. Every insert shifts later positions by one, so the loop works from the back of the list forward.
This is a net under a net. Cancelling a function or custom tool already records an output whose text contains "aborted by user" (core/src/tools/parallel.rs); a cancelled tool-search call records an empty completed output instead. This function catches what still slips through, such as a conversation forked while a call was running. It runs inside for_prompt on a copy of the history, so only the outgoing request is patched. Without it, one missing output would stay in the history and break every later request.
Read it yourself: in history.rs, read the comment on normalize_history. Its second rule is the mirror problem, an output whose call is gone: find remove_orphan_outputs in normalize.rs to see the fix, then read the test normalize_mixed_inserts_and_removals in history_tests.rs. That test is compiled only without debug assertions; a debug build has normalize_mixed_inserts_and_removals_panics_in_debug instead. Question: for a custom-tool call that is not marked inherited_user_message, the branch calls error_or_panic instead of info!. Local-shell uses the same guard. What does error_or_panic (in core/src/util.rs) do in a debug build?
02.4
Termination and the safety cap
The loop needs two stopping conditions:
- Natural termination: the model produces a final answer with no tool calls.
- A hard iteration cap: a maximum number of loop turns (commonly 5–15).
The cap is non-negotiable. Without it, a confused model can loop forever, calling a tool, getting a result it doesn't like, calling again, ad infinitum, burning tokens and money. The cap converts "infinite loop" into "bounded, eventually-terminating process." When the cap is hit, return whatever the model has produced so far (or a graceful "I wasn't able to complete this") rather than erroring.
A second guard worth considering: detect non-productive loops: the same tool called with the same arguments repeatedly, and break early. The model is signalling it's stuck.
02.5
Real code: the OpenAI Agents SDK caps turns
The OpenAI Agents SDK, a Python library for building agents, owns the loop. Runner.run(agent, input) hands the work to AgentRunner, and that class's while True: loop calls the model and runs the tools. The cap is a parameter of the call, max_turns, defaulting to 10 (DEFAULT_MAX_TURNS in run_config.py). Watch what happens on the turn that crosses the line.
current_turn += 1 # [1]
if max_turns is not None and current_turn > max_turns: # [2]
# ... (7 lines: mark the tracing span "Max turns exceeded")
max_turns_error = MaxTurnsExceeded(f"Max turns ({max_turns}) exceeded")
run_error_data = build_run_error_data( # [3]
input=original_input,
new_items=session_items,
raw_responses=model_responses,
last_agent=current_agent,
reasoning_item_id_policy=resolved_reasoning_item_id_policy,
)
handler_result = await resolve_run_error_handler_result( # [4]
error_handlers=error_handlers,
error_kind="max_turns",
error=max_turns_error,
context_wrapper=context_wrapper,
run_data=run_error_data,
)
if handler_result is None:
raise max_turns_error # [5]
# ... (58 lines: check the handler's output and save it to the session)
result = RunResult(
# ... (3 lines: input, new_items, raw_responses)
final_output=validated_output, # [6]
Source: openai/openai-agents-python, src/agents/run.py (MIT)
What to notice
- [1] This runs once per pass, after any pending input is admitted and before
run_single_turncalls the model. A turn here is one model call plus the tool calls it makes (the docstring ofRunner.runsays so), so the call past the limit is never made or paid for. - [2] Pass
max_turns=Noneand there is no cap. Capped is the default; unlimited is a choice you write down. - [3]
build_run_error_datapacks up what the run produced so far: the input, the new items (messages, tool calls, tool outputs), the raw model responses, and the last agent that ran. That bundle is the handler's input, a snapshot of the run so far.Runner.rundoes not return it: the default path raises, and a handler returns its ownfinal_output. - [4] This looks up your function under
"max_turns"in theerror_handlersdict you passed toRunner.run, and calls it with the error and that snapshot. - [5] With no handler, or one that returns
None, the SDK raisesMaxTurnsExceeded. That differs from the section's advice: out of the box, hitting the cap is an exception your code must catch. - [6] With a handler, the skipped lines check its
final_outputagainst the agent's output type and run the output guardrails (checks on an agent's final answer). If both accept it, the SDK returns a normalRunResult. If the agent expects structured output and the handler's value does not match,validate_handler_final_outputraisesUserError, the SDK's error for bad caller input. A guardrail trip still raisesOutputGuardrailTripwireTriggered.
The shape guards against a runaway bill (the cap is on unless you remove it) and a silent stop (failure is loud unless you write the fallback).
Read it yourself: in docs/running_agents.md, read "Error handlers" for a sample on_max_turns. Then search run_internal/run_loop.py for MaxTurnsExceeded(: the streaming runner repeats this check but does not raise there without a handler. What does it do instead, and where does the error reach the caller? (Hint: _check_errors in result.py.)
02.6
Where the loop lives
A key design decision: does your orchestrator drive the loop, or does the provider's SDK? There are two common arrangements:
- You own the loop explicitly: your code does the
repeatand decides when to stop. Maximum control and visibility; you can inject logic between turns. - The provider SDK owns the loop and calls back to you for tool execution: you supply a "run these tools" function and the SDK iterates internally.
A clean hybrid (and a good default) is to own a thin loop that delegates tool execution through a callback, so your orchestrator stays declarative while the provider-specific iteration is encapsulated. Either way, the control (the cap, the termination logic, the event recording) should be yours, not buried in a library.
02.7
Variants of the loop
The basic single loop scales surprisingly far, but you'll encounter variants:
Single-loop (ReAct-style)
One model, one loop, tools available. The model reasons and acts in an interleaved stream. This is the workhorse and is sufficient for most agents, including document assistants. Start here; don't reach for more complexity until you've proven you need it.
Planner–executor
A first model call produces a plan (a list of steps), then a loop executes each step, possibly with its own sub-loops. Useful when tasks are long and benefit from explicit decomposition, or when you want the plan visible/editable by the user. The cost is added complexity and latency; the benefit is structure and steerability.
Multi-agent
Several specialised agents (each its own loop, prompt, and tools) coordinate: a "researcher" hands off to a "writer," or a "supervisor" routes work to "workers." Powerful for genuinely heterogeneous tasks, but it multiplies cost, latency, and failure modes, and the coordination overhead is real. Most products that think they need multi-agent actually need better tools and context in a single loop. Reach for it only when sub-tasks are truly independent and benefit from isolation.
Reflection / critique loops
After producing output, the agent critiques its own work and revises. Improves quality on hard tasks at the cost of extra calls. Often implemented as an explicit "now check your answer" step rather than a separate agent.
02.8
The event timeline
A loop that just returns final text throws away everything interesting that happened along the way. A production loop should emit an ordered event timeline as it runs: "model reasoned," "called tool X," "tool X returned," "model wrote text." This timeline is gold:
- It drives a rich UI (showing tool activity as it happens, Chapter 8).
- It's the durable record of the turn (persisted as the conversation's content, Chapter 9).
- It's the raw material for observability and debugging (Chapter 15).
Design the loop to record an explicit, typed event for every meaningful action, in order. Don't try to reconstruct what happened from the final text afterward; capture it as it occurs.
02.9
Real code: pi's typed event timeline
pi is a small open-source coding agent whose loop lives in packages/agent. Every action the loop takes is reported as one value of a single type, AgentEvent: the timeline this section describes, written as a type. Watch how the events nest: run, then turn, then message and tool.
export type AgentEvent =
// Agent lifecycle
| { type: "agent_start" } // [1]
| { type: "agent_end"; messages: AgentMessage[] }
// Turn lifecycle - a turn is one assistant response + any tool calls/results
| { type: "turn_start" } // [2]
| { type: "turn_end"; message: AgentMessage; toolResults: ToolResultMessage[] }
// Message lifecycle - emitted for system, user, assistant, and toolResult messages
| { type: "message_start"; message: AgentMessage } // [3]
// Only emitted for assistant messages during streaming
| { type: "message_update"; message: AgentMessage; assistantMessageEvent: AssistantMessageEvent } // [4]
| { type: "message_end"; message: AgentMessage }
// Tool execution lifecycle
| { type: "tool_execution_start"; toolCallId: string; toolName: string; args: any } // [5]
| { type: "tool_execution_update"; toolCallId: string; toolName: string; args: any; partialResult: any }
| {
type: "tool_execution_end";
toolCallId: string;
toolName: string;
result: any;
isError: boolean; // [6]
/** Milliseconds `execute()` took, measured with a monotonic clock; absent when the tool did not run. */
durationMs?: number;
};
Source: earendil-works/pi, packages/agent/src/types.ts (MIT)
What to notice
- [1]
agent_startandagent_endbracket one whole run, from the prompt until the loop stops. On the normal path,agent_endcarriesnewMessages: the prompt plus everything the run appended. If the executor throws,handleRunFailure(the catch around that run) emitsagent_endwith only the synthetic failure message.AgentMessageis pi's message type: the usual roles plus custom kinds an app can add. - [2] A turn is one pass of this chapter's loop: one model reply plus the tool calls it caused.
turn_endhands over the reply and the tool results from that pass. Anerrororabortedstop emitsturn_endwith an empty result list and does not run tool calls still sitting in the message. - [3] Every message gets
message_startandmessage_end, even one that arrives complete, so a consumer can rely on one rule: a message is final atmessage_end. - [4] Only the assistant reply streams. Its
AssistantMessageEventcarries pieces of text, thinking, or a tool call as they arrive (text_delta,toolcall_delta, and so on). - [5]
toolCallIdlets a UI match start, progress (partialResult) and end even when several tools run in parallel. - [6] Failure is a field, not a sentence to parse. A monotonic clock, used for
durationMs, is a timer that does not jump when someone changes the system time.
Every member has a type string, so switch (event.type) tells TypeScript which fields exist in that branch. Reading event.toolCallId in the turn_end case fails to compile. A log of sentences such as "called tool X" cannot do that, because every reader of the log has to parse text.
Read it yourself: read processEvents in packages/agent/src/agent.ts. The agent's message list grows only on message_end, and the same event writes the saved session (search packages/coding-agent/src/core/agent-session.ts for Handle session persistence). The terminal UI in interactive-mode.ts switches on these same type values. One stream feeds both. Question: which two events would you use to show and hide a spinner next to a running tool?
02.10
Statelessness within, state at the edges
The loop itself is best kept stateless: it takes the assembled context in, runs, and returns a result plus events. Persistence happens at the edges: load history before the loop, save the outcome after. This keeps the loop a pure function of its inputs, which makes it testable, retryable, and easy to reason about. (More in Chapter 9.)
02.11
Common failure modes to design against
- Runaway iteration → the hard cap.
- Stuck loops (same call repeated) → repetition detection.
- Missing tool results → guarantee one result per call.
- Malformed tool arguments → parse defensively, default to empty, return a useful error the model can recover from.
- Tool exceptions → catch them and feed the error back as a tool result, so the model can adapt rather than the whole turn crashing.
The theme: the loop is where the model's unreliability meets your code, so the loop is where you must be most defensive.
02.12
Real code: Gemini CLI spots a stuck loop
Gemini CLI, Google's open-source terminal agent, watches every tool call the model requests. The check below is the stuck-loop item from the list above, extended from one repeated call to a repeating cycle.
private checkToolCallLoop(toolCall: { name: string; args: object }): boolean {
const key = this.getToolCallKey(toolCall); // [1]
this.toolCallHistory.push(key);
// ... (4 lines: keep only the newest 25 keys)
const n = this.toolCallHistory.length;
const R = TOOL_CALL_LOOP_THRESHOLD; // 5
// Check for repeating patterns of cycle length k from 1 to 5
for (let k = 1; k <= 5; k++) { // [2]
const requiredLength = k * R;
if (n >= requiredLength) {
const cycle = this.toolCallHistory.slice(-k);
let isPatternMatch = true;
for (let i = 0; i < requiredLength; i++) {
const indexFromEnd = requiredLength - i;
const actualKey = this.toolCallHistory[n - indexFromEnd];
const expectedKey = cycle[i % k]; // [3]
if (actualKey !== expectedKey) {
isPatternMatch = false;
break;
}
}
if (isPatternMatch) {
return true; // [4]
}
}
}
return false;
}
Source: google-gemini/gemini-cli, packages/core/src/services/loopDetectionService.ts (Apache-2.0)
What to notice
- [1]
getToolCallKeyfingerprints a call: the tool name plusJSON.stringify(args), hashed with SHA-256. Two calls match only when both are identical. The skipped lines keep the newest 25 keys, five cycle lengths times five repeats. - [2]
kis the cycle length.k = 1catches the same call five times.k = 2catches A, B, A, B across ten calls. A check for the identical call in a row misses that second shape, and flipping between two actions is a common way agents get stuck. - [3] The window is the last
k * 5keys, andcycleis its lastk.cycle[i % k]is what positionimust be if that cycle repeats. One mismatch ends the test for thisk. - [4] A match returns
true.addAndCheckstoresRepeated tool call: ...with the arguments on the result. The log records the loop type and strike count; the arguments are sent to the model in the recovery message.
Five different files in a row do not match, because the arguments differ. On a match, core/client.ts gives the model one chance: _recoverFromLoop stops the reply and sends a message asking it to step back. A second detection yields LoopDetected, which the agent layer reports as the error "Loop detected, stopping execution." That error has fatal set to false, so it is not marked as the reason the whole agent halts, and this turn does not continue.
Read it yourself: in loopDetectionService.test.ts, read the tests for a cycle of length 2 and for alternating sequences that vary. Then answer from getToolCallKey: do {"path": "a", "limit": 10} and {"limit": 10, "path": "a"} hash the same? (JSON.stringify keeps key order.) After 30 turns on one prompt the service also asks a model, using LOOP_DETECTION_SYSTEM_PROMPT at the top of the file. What can that model catch that a hash cannot?
02.13
Real code: pi turns every tool failure into a result
pi handles the last two items on the list, malformed arguments and tool exceptions, with one rule: no failure leaves the tool step as an exception. Every failure becomes a tool result marked isError: true that goes back to the model. Two functions in pi's loop file enforce it. The first checks a call before it runs:
const tool = tools.find((t) => t.name === toolCall.name); // [1]
if (!tool) {
return {
kind: "immediate",
result: createErrorToolResult(`Tool ${toolCall.name} not found`),
isError: true,
};
}
try {
const preparedToolCall = prepareToolCallArguments(tool, toolCall);
const validatedArgs = validateToolArguments(tool, preparedToolCall); // [2]
// ... (36 lines: a permission hook may block the call; checks for a cancelled run)
return {
kind: "prepared",
toolCall,
tool,
args: validatedArgs,
};
} catch (error) {
return {
kind: "immediate",
result: createErrorToolResult(error instanceof Error ? error.message : String(error)), // [3]
isError: true,
};
}
Source: earendil-works/pi, packages/agent/src/agent-loop.ts (MIT)
The second runs it:
try {
const result = await prepared.tool.execute( // [4]
prepared.toolCall.id,
prepared.args as never,
signal,
// ... (4 lines: pass progress updates through)
);
// ... (3 lines: stop the timer, stop accepting progress updates)
return { result, isError: result.isError === true, durationMs };
} catch (error) {
// ... (3 lines: the same timer and progress cleanup)
return {
result: createErrorToolResult(error instanceof Error ? error.message : String(error)), // [5]
isError: true,
durationMs,
};
Source: earendil-works/pi, packages/agent/src/agent-loop.ts (MIT)
What to notice
- [1] The model can name a tool that does not exist. The answer is an
"immediate"outcome, a result that is ready without running anything.createErrorToolResultjust wraps a message as text. - [2]
prepareToolCallArgumentsruns first. If the tool definesprepareArguments, that hook may rewritetoolCall.argumentsbefore the check.validateToolArgumentsthen checks the arguments against the tool's schema. On a mismatch it throws a message that lists each bad field and repeats the arguments the model sent: exactly what the model needs to fix its next call. - [3] The
catchturns that throw into an error result. A"prepared"outcome means the call passed every check and may run. - [4]
executeis the tool's own code.signalis the run's abort signal, and it may be undefined. The comment onexecuteintypes.tstells tool authors to throw on failure or return a result withisError: true, so a throw is a normal way for a tool to fail, not an accident. - [5] The error's message becomes the result, and
durationMsis still recorded.
On these two paths the loop emits tool_execution_end and appends a result with the call's id. If the run is aborted mid-batch, the sequential runner stops and the remaining calls in that batch get no result. A loop that lets the exception escape loses the whole turn over one bad file path, and the model never learns what went wrong.
Read it yourself: in the same file, read failToolCallsFromTruncatedMessage. When a reply is cut off by the output token limit, the arguments may parse but be incomplete, so pi runs none of the calls and asks the model to re-issue them. Then read finalizeExecutedToolCall: what happens if your own afterToolCall hook throws?
02.14
Read the source yourself
A reading tour, in order, that walks the loop from the outside in:
- The whole loop in one function. pi's
runLoopis this chapter's pseudocode at production size. Find wherehasMoreToolCallsis set and whereturn_endis emitted, and map each to a line of the essential loop. - A loop the library owns. The OpenAI Agents SDK's
AgentRunner.runrunswhile True:. Follow what happens on a handoff (one agent passing control to another): the loop changescurrent_agentand goes round again. - History repair before every request. Callers clone the history, then Codex's
for_prompttakes that copy and normalizes it. List every rulenormalize_historyenforces. - The other loop detector. Gemini CLI's
checkContentLoopcatches a model repeating the same text while it streams. Find why it switches itself off inside code blocks.
Exercise: write the essential loop from the start of this chapter as a small Python run_agent(model, tools, max_turns), then add three things from these repos: an "aborted" result for any call left without one, a cycle check for cycle lengths 1 to 3, and an on_max_turns callback that returns a message instead of a fixed string. Then write a fake model that alternates between two tool calls and confirm your detector stops it.
0 of 15 slides completed
Your learning progress stays in this browser.
Self-check · 10 questions
01 / 10
The agent loop terminates under which two conditions?