CH 01 · Anatomy of an AI Agent · 08/12
Read as one pageReal code: one smolagents step, from memory to model and back
smolagents is Hugging Face's compact agent library. _step_stream, on its ToolCallingAgent, rebuilds the context, calls the model, runs the tools, and yields whether this step produced the final answer. The caller, _run_stream, appends the step to memory and streams the final answer to the user. The method yields a stream of events: token deltas (ChatMessageStreamDelta), each requested call (ToolCall), each result (ToolOutput), and one final ActionOutput that says whether the step finished the task. ChatMessage is the model's reply.
def _step_stream(
self, memory_step: ActionStep
) -> Generator[ChatMessageStreamDelta | ToolCall | ToolOutput | ActionOutput]:
# ... (5 lines: docstring)
memory_messages = self.write_memory_to_messages() # [1]
input_messages = memory_messages.copy()
# Add new step in logs
memory_step.model_input_messages = input_messages # [2]
try:
# ... (16 lines: if streaming is on, call generate_stream and show tokens live)
else:
chat_message: ChatMessage = self.model.generate( # [3]
input_messages,
stop_sequences=["Observation:", "Calling tools:"],
tools_to_call_from=self.tools_and_managed_agents,
)
# ... (6 lines: log the reply)
# Record model output
memory_step.model_output_message = chat_message # [4]
memory_step.model_output = chat_message.content
memory_step.token_usage = chat_message.token_usage
except Exception as e:
raise AgentGenerationError(f"Error while generating output:\n{e}", self.logger) from e
# ... (9 lines: parse tool calls out of the reply)
final_answer, got_final_answer = None, False
for output in self.process_tool_calls(chat_message, memory_step): # [5]
yield output
# ... (18 lines: accept at most one final_answer call per step)
yield ActionOutput( # [6]
output=final_answer,
is_final_answer=got_final_answer,
)
Source: huggingface/smolagents, src/smolagents/agents.py (Apache-2.0)
What to notice
- [1] Context, built fresh.
write_memory_to_messagesstarts from the system prompt and asks every saved step to render itself as chat messages. The agent never keeps one growingmessageslist; it rebuilds the list from its records on every call. - [2]
memory_stepis anActionStep, a small record (a Python dataclass) for this one step. The first thing stored in it is exactly what the model is about to see, so any step can be inspected later. - [3] The model call.
tools_to_call_fromsends the tool definitions, plus managed agents: other agents this one can call as if they were tools. The stop sequences ask a model that supports them to halt if it starts writingObservation:, the label reserved for real tool results. A model without that parameter gets the text cut out of the reply afterward. - [4] The reply and its token usage go onto the same record.
- [5] The tools.
process_tool_callsruns each requested tool (in parallel threads when there are several), passes each call and result up as it happens, then writes the calls and their observations (tool results as text) ontomemory_step. - [6] The step reports whether it produced the final answer. A malformed tool call raises
AgentParsingError._run_streamstores that onaction_step.errorand continues, so the next call shows the model the error.AgentGenerationError, the exception in this excerpt, means the model call itself failed. This file treats that as a bug in the agent, re-raises it, and does not seterror. The step is still appended, because thefinallyblock runs, and the run stops.
Why records instead of a message list? The same memory can be rendered more than one way: when the agent revises its plan, it calls write_memory_to_messages(summary_mode=True), which leaves out the system prompt and earlier plans. A plain append-only list cannot do that.
Read it yourself: open src/smolagents/memory.py and read ActionStep.to_messages. Which field becomes which message role, and what does a stored error turn into? Then find self.memory.steps.append(action_step) in _run_stream.
Finished this idea?
Your learning progress stays in this browser.