Skip to content
Architecture and System Design chapter route010203040506070809101112131415161718

Foundations

Architecture and System Design

A general, technology-agnostic guide to designing AI agents: the systems, patterns, trade-offs, and best practices that apply when you build an LLM-powered agent that uses tools, processes documents, and acts on a user's behalf.

18 chapters221 slides2 hr 14 minUpdated

Your route through the subject

0/221 slides completed · 0/18 chapters · 0 slides read

Start with: Anatomy of an AI Agent

Begin reading

The chapter map

Follow the route or choose a chapter.
  1. 010/12 completeAnatomy of an AI Agent~14 minReadPlateCode
  2. 020/15 completeThe Agent Loop Pattern~14 minReadField checkPlateCode
  3. 030/18 completeTool Design~16 minReadCode
  4. 040/15 completeModel-Provider Abstraction and Multi-Model Strategy~14 minReadCode
  5. 050/18 completeContext Engineering and Memory~14 minReadPlateCode
  6. 060/10 completePrompt Architecture~4 minRead
  7. 070/7 completeRetrieval: RAG vs Tools vs Long Context~4 minRead
  8. 080/11 completeStreaming and Real-Time UX~5 minRead
  9. 090/12 completeData Modeling for Agents~5 minRead
  10. 100/9 completeDocument and File Processing Pipelines~5 minRead
  11. 110/10 completeSecurity, Auth, and Multi-Tenancy~5 minRead
  12. 120/11 completeSecrets and Bring-Your-Own-Key~4 minRead
  13. 130/13 completeReliability: Retries, Idempotency, and Failure Handling~5 minReadPlate
  14. 140/14 completeCost, Latency, and Model Tiering~5 minRead
  15. 150/8 completeObservability and Evaluation~4 minRead
  16. 160/11 completeScaling and Infrastructure~5 minRead
  17. 170/11 completeDomain-Specific and Regulated-Industry Agents~5 minRead
  18. 180/16 completeReference Architecture and a Design Checklist~5 minReadCode

Chapters

  1. Chapter 1: Anatomy of an AI Agent

    An AI agent has six parts: the model, the loop, tools, context, memory, and the surrounding system. How each one behaves and how they interact in one turn.

    14 min12 slides: Anatomy of an AI Agent0/12 complete

  2. Chapter 2: The Agent Loop Pattern

    The agent loop appends each request and result until the model stops or hits a safety cap. Covers common failure modes to design against.

    14 min15 slides: The Agent Loop Pattern0/15 complete

  3. Chapter 3: Tool Design

    A tool's description doubles as a prompt the model reads before acting. Covers granularity, schema discipline, and structured input design.

    16 min18 slides: Tool Design0/18 complete

  4. Chapter 4: Model-Provider Abstraction and Multi-Model Strategy

    Decoupling an agent from one model provider takes an adapter pattern and a validated registry. Covers tiering models by task and cost.

    14 min15 slides: Model-Provider Abstraction and Multi-Model Strategy0/15 complete

  5. Chapter 5: Context Engineering and Memory

    Context is assembled fresh each turn, not stored, and the window is a hard budget. Covers referencing data instead of embedding it.

    14 min18 slides: Context Engineering and Memory0/18 complete

  6. Chapter 6: Prompt Architecture

    The system prompt is a contract defining rules and the structured protocols your code parses. Covers layered assembly and testing.

    4 min10 slides: Prompt Architecture0/10 complete

  7. Chapter 7: Retrieval: RAG vs Tools vs Long Context

    Long context, RAG, and tool-based retrieval fetch facts with different cost and freshness trade-offs. Covers picking the right default.

    4 min7 slides: Retrieval: RAG vs Tools vs Long Context0/7 complete

  8. Chapter 8: Streaming and Real-Time UX

    Streaming shapes an agent's transport, orchestrator, and data model, not just its interface. Covers SSE versus WebSockets and backpressure.

    5 min11 slides: Streaming and Real-Time UX0/11 complete

  9. Chapter 9: Data Modeling for Agents

    Model conversations as event logs, version every artifact the agent produces, and track long-running work with explicit status fields.

    5 min12 slides: Data Modeling for Agents0/12 complete

  10. Chapter 10: Document and File Processing Pipelines

    Turning an uploaded file into usable text takes ingestion, conversion, and extraction, each prone to failure. Covers sync versus async.

    5 min9 slides: Document and File Processing Pipelines0/9 complete

  11. Chapter 11: Security, Auth, and Multi-Tenancy

    Covers authentication, centralized authorization, and tenant isolation, plus risks like prompt injection and data exfiltration.

    5 min10 slides: Security, Auth, and Multi-Tenancy0/10 complete

  12. Chapter 12: Secrets and Bring-Your-Own-Key

    Bring-your-own-key lets users supply their own model API key, encrypted at rest and resolved by precedence over the operator's.

    4 min11 slides: Secrets and Bring-Your-Own-Key0/11 complete

  13. Chapter 13: Reliability: Retries, Idempotency, and Failure Handling

    Model APIs time out and tools go flaky, so reliable agents expect failure. Covers backoff retries and resuming interrupted work.

    5 min13 slides: Reliability: Retries, Idempotency, and Failure Handling0/13 complete

  14. Chapter 14: Cost, Latency, and Model Tiering

    Cost and latency compound because an agent loop makes several model calls per turn. Covers model tiering as the main lever.

    5 min14 slides: Cost, Latency, and Model Tiering0/14 complete

  15. Chapter 15: Observability and Evaluation

    Observability captures the trace of what an agent did; evaluation measures whether it was any good. Covers building an eval set.

    4 min8 slides: Observability and Evaluation0/8 complete

  16. Chapter 16: Scaling and Infrastructure

    A demo runs on one laptop; a product serves many users with long-lived streams. Covers statelessness and upstream rate limits.

    5 min11 slides: Scaling and Infrastructure0/11 complete

  17. Chapter 17: Domain-Specific and Regulated-Industry Agents

    Agents for law, medicine, and finance carry extra obligations since a wrong answer is costly and experts notice errors. Covers audit trails.

    5 min11 slides: Domain-Specific and Regulated-Industry Agents0/11 complete

  18. Chapter 18: Reference Architecture and a Design Checklist

    A single reference architecture ties every pattern together, from tools and context to security and cost. Use it as a design checklist.

    5 min16 slides: Reference Architecture and a Design Checklist0/16 complete