Skip to slide
008 / 170

CH 01 · What Is an Agent Harness? · 08/12

Read as one page

Real code: the model answers, the environment acts

The Harness class you just read stores tools, history, memory, and permissions together, and call_model only talks. mini-swe-agent, a small Python coding agent, makes the same cut with protocols. A protocol is a list of methods any matching class must have. The model protocol has query and no execute. The environment protocol has execute and no query.

class Model(Protocol):
    """Protocol for language models."""

    config: Any

    def query(self, messages: list[dict[str, str]], **kwargs) -> dict: ...  # [1]

    def format_message(self, **kwargs) -> dict: ...

    def format_observation_messages(  # [2]
        self, message: dict, outputs: list[dict], template_vars: dict | None = None
    ) -> list[dict]: ...

    def get_template_vars(self, **kwargs) -> dict[str, Any]: ...

    def serialize(self) -> dict: ...


class Environment(Protocol):
    """Protocol for execution environments."""

    config: Any

    def execute(self, action: dict, cwd: str = "") -> dict[str, Any]: ...  # [3]

    def get_template_vars(self, **kwargs) -> dict[str, Any]: ...

    def serialize(self) -> dict: ...

Source: SWE-agent/mini-swe-agent, src/minisweagent/__init__.py (MIT)

What to notice

  1. [1] query takes the transcript and returns one dict. In LitellmModel, that dict is the assistant message, and the parsed actions sit on extra. The full API response is extra["response"]. The method calls the API. It does not run a command. If the reply names a tool the parser does not know, it raises FormatError instead of guessing.
  2. [2] format_observation_messages turns command output into transcript messages. The tool-calling formatter uses role: "tool" and tool_call_id, the id of the call this result answers. The text formatter uses role: "user", because that model class does not pass a tools list to the API.
  3. [3] On the normal path, execute returns a result dict. If the command prints COMPLETE_TASK_AND_SUBMIT_FINAL_OUTPUT and exits 0, LocalEnvironment raises Submitted instead, and that is the environment ending the task. A sandbox is a different class with this same execute.

Agent, like the other two, has config, and the only methods it adds are run and save. The class that implements it is the glue the sketch left as ...: step asks the model, then runs the actions in that reply.

Read it yourself: Read query at line 81 of src/minisweagent/models/litellm_model.py, then _query just above it. You should find litellm.completion there, and no subprocess in the file. Then read execute in src/minisweagent/environments/local.py through _check_finished. Write down the keys of both dicts. The success path has output, returncode, and exception_info. The error path adds extra. Then open step in src/minisweagent/agents/default.py. It is one line: ask, then execute.

Finished this idea?

Your learning progress stays in this browser.