Skip to slide
008 / 221

CH 01 · Anatomy of an AI Agent · 08/12

Read as one page

Real code: one smolagents step, from memory to model and back

smolagents is Hugging Face's compact agent library. _step_stream, on its ToolCallingAgent, rebuilds the context, calls the model, runs the tools, and yields whether this step produced the final answer. The caller, _run_stream, appends the step to memory and streams the final answer to the user. The method yields a stream of events: token deltas (ChatMessageStreamDelta), each requested call (ToolCall), each result (ToolOutput), and one final ActionOutput that says whether the step finished the task. ChatMessage is the model's reply.

def _step_stream(
    self, memory_step: ActionStep
) -> Generator[ChatMessageStreamDelta | ToolCall | ToolOutput | ActionOutput]:
    # ... (5 lines: docstring)
    memory_messages = self.write_memory_to_messages()  # [1]

    input_messages = memory_messages.copy()

    # Add new step in logs
    memory_step.model_input_messages = input_messages  # [2]

    try:
        # ... (16 lines: if streaming is on, call generate_stream and show tokens live)
        else:
            chat_message: ChatMessage = self.model.generate(  # [3]
                input_messages,
                stop_sequences=["Observation:", "Calling tools:"],
                tools_to_call_from=self.tools_and_managed_agents,
            )
            # ... (6 lines: log the reply)

        # Record model output
        memory_step.model_output_message = chat_message  # [4]
        memory_step.model_output = chat_message.content
        memory_step.token_usage = chat_message.token_usage
    except Exception as e:
        raise AgentGenerationError(f"Error while generating output:\n{e}", self.logger) from e
    # ... (9 lines: parse tool calls out of the reply)
    final_answer, got_final_answer = None, False
    for output in self.process_tool_calls(chat_message, memory_step):  # [5]
        yield output
        # ... (18 lines: accept at most one final_answer call per step)
    yield ActionOutput(  # [6]
        output=final_answer,
        is_final_answer=got_final_answer,
    )

Source: huggingface/smolagents, src/smolagents/agents.py (Apache-2.0)

What to notice

  1. [1] Context, built fresh. write_memory_to_messages starts from the system prompt and asks every saved step to render itself as chat messages. The agent never keeps one growing messages list; it rebuilds the list from its records on every call.
  2. [2] memory_step is an ActionStep, a small record (a Python dataclass) for this one step. The first thing stored in it is exactly what the model is about to see, so any step can be inspected later.
  3. [3] The model call. tools_to_call_from sends the tool definitions, plus managed agents: other agents this one can call as if they were tools. The stop sequences ask a model that supports them to halt if it starts writing Observation:, the label reserved for real tool results. A model without that parameter gets the text cut out of the reply afterward.
  4. [4] The reply and its token usage go onto the same record.
  5. [5] The tools. process_tool_calls runs each requested tool (in parallel threads when there are several), passes each call and result up as it happens, then writes the calls and their observations (tool results as text) onto memory_step.
  6. [6] The step reports whether it produced the final answer. A malformed tool call raises AgentParsingError. _run_stream stores that on action_step.error and continues, so the next call shows the model the error. AgentGenerationError, the exception in this excerpt, means the model call itself failed. This file treats that as a bug in the agent, re-raises it, and does not set error. The step is still appended, because the finally block runs, and the run stops.

Why records instead of a message list? The same memory can be rendered more than one way: when the agent revises its plan, it calls write_memory_to_messages(summary_mode=True), which leaves out the system prompt and earlier plans. A plain append-only list cannot do that.

Read it yourself: open src/smolagents/memory.py and read ActionStep.to_messages. Which field becomes which message role, and what does a stored error turn into? Then find self.memory.steps.append(action_step) in _run_stream.

Finished this idea?

Your learning progress stays in this browser.