Skip to slide
005 / 221

CH 01 · Anatomy of an AI Agent · 05/12

Read as one page

Real code: Hermes keeps long-term memory in two small files

The memory part persists across sessions and is selectively loaded into context. Hermes's built-in version, MemoryStore, is the object _init_memory creates in the previous slide, and it is deliberately small: two Markdown files, a size budget for each, and a frozen copy for the system prompt. Watch the difference between what is on disk and what the model sees.

class MemoryStore:
    """Bounded curated memory with file persistence; one instance per AIAgent.
    ``_system_prompt_snapshot`` is frozen at load time (prefix-cache stable);
    ``memory_entries`` / ``user_entries`` are live state persisted to disk."""
    # ... (7 lines: a cap on failed memory writes per turn)
    def __init__(self, memory_char_limit: int = 2200, user_char_limit: int = 1375, *,  # [1]
                 memory_enabled: bool = True, user_profile_enabled: bool = True):
        self.memory_entries: list[str] = []  # [2]
        self.user_entries: list[str] = []
        self.memory_char_limit, self.user_char_limit = memory_char_limit, user_char_limit
        self.memory_enabled, self.user_profile_enabled = memory_enabled, user_profile_enabled
        self._system_prompt_snapshot: dict[str, str] = {"memory": "", "user": ""}  # [3]
    # ... (28 lines: the per-turn failure counter)
    def load_from_disk(self):
        # ... (16 lines: docstring; a helper that blocks entries matching a threat pattern)
        for target in ("memory", "user"):
            path = self._path_for(target)  # [4]
            # ... (3 lines: make sure the memories folder exists)
            # Deduplicate (order-preserving, first occurrence wins).
            entries = list(dict.fromkeys(self._read_file(path)))
            self._set_entries(target, entries)
            # ... (7 lines: warn, never truncate, when a file is over its limit)
            self._system_prompt_snapshot[target] = self._render_block(target, [_sanitize(e, path.name) for e in entries])  # [5]

Source: NousResearch/hermes-agent, tools/memory_tool_store.py (MIT)

What to notice

  1. [1] The budgets are in characters, not tokens: 2,200 for the agent's own notes and 1,375 for the user profile by default. A character count does not depend on any model's tokenizer, so the cap means the same thing after you switch models. When a new entry would go over, add refuses it and tells the model to remove or shorten stale entries in the same batch as the new one.
  2. [2] The live state is two plain lists of strings, one remembered fact per entry. On disk they become MEMORY.md and USER.md, entries separated by a § line.
  3. [3] _system_prompt_snapshot is a second copy, already rendered as text. format_for_system_prompt hands the prompt builder this snapshot, never the live lists.
  4. [4] _path_for points into a memories folder inside Hermes's home directory.
  5. [5] The snapshot is rendered at load from those entries, after _sanitize substitutes a [BLOCKED: ...] note for any entry that matches a prompt-injection or data-exfiltration pattern. The live lists and the file keep the original text, so you can still see and delete it.

Why freeze it? Providers cache the start of a prompt: when a request begins with exactly the same bytes as an earlier one, that part is cheaper and faster. A fact saved mid-session would break that cache on every later call. Writes go to disk at once and leave the snapshot alone. A saved fact reaches the prompt the next time the snapshot is rebuilt: at startup, after compression, on a new session or a session switch, and on undo. Pasting the whole live file into every call instead grows without limit, breaks the cache on every save, and carries one poisoned entry into every future session.

Read it yourself: in tools/memory_tool_store.py, read _mutate and add. What does the store do when MEMORY.md exists but cannot be read, and why is "treat it as empty" the dangerous choice? Then search agent/system_prompt.py for load_from_disk and read invalidate_system_prompt: when does a saved fact finally reach the prompt?

Finished this idea?

Your learning progress stays in this browser.