Skip to content
Agent YAP

Interview prep

AI agent interview questions, answered.

The questions that come up when a team hires someone to build with language models, with the answer a strong candidate gives. Each answer links to the chapter that explains it in full.

60 questions · 10 topics · Free, no sign-up

AI agent fundamentals interview questions

What an agent is, how the loop runs, and how tools are designed.

  1. 01What is an AI agent, and how is it different from a chatbot?

    An agent is a model running in a loop that can take actions through tools, observe the results, and decide its next step, repeating until the task is done. A single prompt returns output once, with no memory and no actions. A chatbot keeps conversation history but still only talks; it cannot change anything in the world. An agent can read a file, call an API, write a document and chain those steps together from one user request. The defining feature is the loop with tools. Everything else in a production agent, such as context assembly, memory and the surrounding system, exists to make that loop useful, safe and reliable.

    Explained in Anatomy of an AI Agent: The spectrum: prompt → chatbot → agent

  2. 02How does the agent loop work, step by step?

    The harness sends the model the conversation plus the list of tools it may use. Every model call ends one of two ways. Either the model asks for a tool call, such as running ls or applying a patch, or it produces a plain assistant message. On a tool request, the harness runs the tool, appends the call and its result to the history, and calls the model again with the new information. On a plain message, the turn ends and control returns to the user. A request to fix failing tests might spin through the loop fifty times. There is no separate planning engine: investigating and course-correcting emerge from this loop. Anthropic describes it as gather context, take action, verify results.

    Explained in The Agent Loop: Concept explanation

  3. 03What is an agent harness, and why does it matter more than the prompt?

    The harness is the ordinary software wrapped around the model. The model runs on the provider's servers, takes text in and produces text out, and cannot touch files, run commands or remember yesterday. The harness gathers instructions, project files, conversation history and the allowed tools, sends them to the model, and when the model asks to run something, the harness actually runs it and hands back the output. The model reasons; the harness acts. Engineers who build these systems find bigger wins in better tool descriptions, permission rules, memory files and context management than in rewording the system prompt. A common rule of thumb puts agent building at roughly 5 percent calling the model in a loop and 95 percent harness work.

    Explained in What Is an Agent Harness?: Concept explanation

  4. 04ReAct-style loop vs planner-executor: which agent pattern should you start with?

    Start with the single-loop, ReAct-style agent: one model, one loop, tools available, with reasoning and acting interleaved in one stream. It is the workhorse and is sufficient for most agents, including document assistants. A planner-executor first makes a model call that produces a plan, then a loop executes each step, possibly with sub-loops. It helps on long tasks that benefit from explicit decomposition, or when users should see and edit the plan, at the cost of extra complexity and latency. Multi-agent setups multiply cost, latency and failure modes; most products that think they need one actually need better tools and context in a single loop. Reflection loops, where the agent critiques and revises its output, raise quality at the price of extra calls.

    Explained in The Agent Loop Pattern: Variants of the loop

  5. 05How do you design good tools for an LLM agent?

    Treat each description as a mini system prompt, because the model decides whether and how to call a tool almost entirely from it: say when and how to use the tool, not only what it does, and name the tool that should come next. Size each tool to one user-level action, such as read a document, rather than open_file plus read_line or one giant do_everything. Offer a cheap, narrow variant beside an expensive, broad one, and let the model batch when it needs many items. Tighten the schema with enums, bounds, required fields and per-field descriptions. Return useful errors and distilled results instead of raw dumps, and offer each tool only on surfaces where it makes sense.

    Explained in Tool Design: A checklist for a well-designed tool

  6. 06What are the common failure modes of an agent loop, and how do you guard against them?

    Five recur. Runaway iteration needs a hard cap on turns, commonly 5 to 15, that returns partial work or a graceful message instead of an error. Stuck loops, where the same call repeats, need repetition detection; Gemini CLI also catches cycles such as A, B, A, B. Missing tool results break the next provider request, so guarantee one result per call and synthesize an error result when a tool produced none. Malformed arguments should be parsed defensively and answered with an error the model can fix. Tool exceptions should be caught and fed back as tool results so the model can adapt instead of the whole turn crashing. The loop is where model unreliability meets your code, so be most defensive there.

    Explained in The Agent Loop Pattern: Common failure modes to design against

Back to topics

Context engineering interview questions

What fills the context window, how it goes wrong, and how to manage it.

  1. 01What is context engineering, and how is it different from prompt engineering?

    Prompt engineering is wording one instruction well. Context engineering is designing and building dynamic systems that provide the right information and tools, in the right format, at the right time, so the model can accomplish the task. Four differences matter. It is a system, not a string: context is the output of code that runs before the model call. It is dynamic, built per request. It aims for the right information at the right time, not everything. And format matters: a clean summary beats a raw data dump. Context covers the system prompt, user request, history, long-term memory, retrieved data, tool definitions and output rules. Most agent failures today are context failures, not model failures.

    Explained in What Context Engineering Is, and Why It Replaced Prompt Engineering: The definition that ties it together

  2. 02What is context rot, and how do you prevent it?

    Context rot is the drop in model quality as the context window fills, even when the token count is well inside the advertised limit. The effective window, where the model actually reasons well, is often far smaller than the technical one, frequently under 256,000 tokens for current models, and no error is thrown when quality slips. Chroma published a study on it, and Anthropic describes a growing context as depleting the model's attention budget: attention spreads thinner across everything in the window, so the part that matters is easier to miss. Prevention is a pre-rot threshold: monitor token counts and trigger compaction or summarization well before the rot zone, instead of waiting for the API to fail at the hard limit.

    Explained in How Context Goes Wrong: Rot, Pollution, and Confusion: Context rot: the most important failure to understand

  3. 03Context rot vs context pollution vs context confusion: what is the difference?

    They are three problems with three different cures. Context rot is a volume problem: the window is too full and quality degrades, so the cure is reduction through compaction and summarization. Context pollution is a relevance problem: stale tool results, near-duplicate snippets or off-topic details distract the model even when the window is not full, so the cure is being selective about what enters and isolating work into separate contexts. Context confusion is a clarity problem: instructions clash, or the model cannot tell instructions from data, often because it has 100 or more tools; the cure is fewer, clearer tools and non-conflicting instructions. The wrong cure does nothing: summarizing history will not fix a bloated toolset.

    Explained in How Context Goes Wrong: Rot, Pollution, and Confusion: How these three relate

  4. 04Compaction vs summarization: which should an agent use to shrink its context?

    Reach for compaction first. Compaction strips information that still exists elsewhere, usually on the filesystem: instead of keeping a 500-line file in the history, keep a reference like Output saved to /src/main.py and re-read the file if needed. It is reversible because the full version is on disk, and Manus applies it to older tool results the agent has already acted on while keeping recent ones in full. Summarization uses the model to condense history and is lossy, so detail is gone for good. Use it only when compaction no longer frees enough space, trigger it at a chosen threshold, keep the most recent tool calls raw to preserve the model's rhythm, and use a fixed summary schema.

    Explained in The Four Moves: Reduce, Offload, Isolate, Retrieve: Move 1: Reduce context

  5. 05What are the four context engineering strategies: reduce, offload, isolate and retrieve?

    Reduce makes what is in the window smaller, compaction first and summarization as a last resort. Offload keeps full data outside the window, for example tool results saved to the filesystem and found later with grep and glob, and pushes most actions into a sandbox through a few general tools instead of hundreds of bound ones. Isolate gives separate work its own context window through sub-agents, sharing full context only when the sub-agent needs the whole trajectory. Retrieve pulls information in on demand for the step that needs it, then lets it leave. None of these needs a smarter model. One warning: do not use RAG to pick tool definitions per step, because a shifting toolset breaks the KV cache and confuses the model.

    Explained in The Four Moves: Reduce, Offload, Isolate, Retrieve: Putting the four moves together

  6. 06How does prompt caching work in an agent, and what breaks the cache?

    The provider saves the computed KV cache state for a prompt prefix and reuses it on later calls that share exactly the same prefix. Reuse stops at the first token that differs, so put stable content such as instructions, tool definitions and examples at the front, and variable content like the latest message and newest tool result at the end. Cached tokens cost roughly a tenth of fresh input, which is why re-sending a growing history costs roughly linear rather than quadratic. Changing the tool list mid-session, switching models, or changing sandbox or approval settings causes misses. For necessary changes, append a new message describing the change instead of editing an earlier one, as Codex does when the working directory changes.

    Explained in Prompt Caching and Context Management: Concept explanation

Back to topics

RAG and agentic RAG interview questions

Retrieval, from the naive pipeline to agents that decide what to fetch.

  1. 01What is RAG, and what are its three core components?

    Retrieval-Augmented Generation retrieves relevant external information at query time and feeds it to the model before it answers. It exists because a model only knows what it learned in training, and that knowledge is frozen at a point in time and can be incomplete or outdated. RAG keeps answers current and grounded in real sources rather than invented from memory. The three components are retrieval, which queries knowledge bases, APIs or vector databases for relevant information; augmentation, which processes and summarizes the retrieved data to fit the query; and generation, which combines the retrieved information with the model's own knowledge to produce the answer. Every later variant builds on these three parts.

    Explained in From Naive RAG to Agentic RAG: The Evolution: Why RAG exists at all

  2. 02Naive RAG vs advanced, modular, graph and agentic RAG: how did RAG evolve?

    Naive RAG uses keyword retrieval such as TF-IDF and BM25 over a static dataset in a plain retrieve-then-read flow; it is simple but matches words, not meaning, so car misses automobile. Advanced RAG adds dense retrieval with embeddings, re-ranking and multi-hop retrieval, at higher compute cost. Modular RAG splits the pipeline into swappable components with hybrid sparse and dense retrieval, tool integration and composable pipelines. Graph RAG adds knowledge graphs to reason over relationships, which helps multi-hop reasoning but needs high-quality graph data. Agentic RAG puts agents in control: they evaluate the query, choose retrieval strategies and retrieve iteratively with feedback. The shift is from retrieval as a fixed step to retrieval as a decision.

    Explained in From Naive RAG to Agentic RAG: The Evolution: The five stages of RAG

  3. 03When should you NOT use agentic RAG?

    When the task is simple fact retrieval or a well-scoped query. Agentic RAG adds adaptability and multi-step reasoning, but also coordination complexity, latency and compute cost, and a modular RAG pipeline gives enough performance at much lower overhead for those cases. Adopt agentic designs selectively, guided by task complexity. Also do not add agents to paper over bad retrieval: weak coverage or badly built indexes sink the answer however clever the orchestration, so fix retrieval and indexing first. If you do go agentic, bound it with planning horizons, tool access policies and explicit stopping criteria, since unbounded agents over-call tools and loop. Gains are largest in structured domains like healthcare, finance and law.

    Explained in Practical Guidance: When and How to Use Agentic RAG: Lesson 1: Agentic RAG is not always the right default

  4. 04What is corrective RAG, and how does it work?

    Corrective RAG judges its own retrieved documents and fixes retrieval when it falls short. It is built on the evaluator-optimizer pattern, with five specialized agents. A context retrieval agent fetches initial documents from a vector database. A relevance evaluation agent flags irrelevant or ambiguous ones. A query refinement agent rewrites the query to improve retrieval. An external knowledge retrieval agent runs web searches or queries other sources when the context is insufficient. A response synthesis agent integrates the validated information into the final answer. The benefits are iterative correction, real-time adaptability and better factuality, because validating content reduces hallucination. The cost is added latency from the correction loop.

    Explained in The Taxonomy of Agentic RAG Systems: Agentic corrective RAG

  5. 05RAG vs long context vs tool-based retrieval: how do you choose?

    Ask in order. If the content is small and always needed whole, put it in long context and do not over-engineer. If it is a bounded, identified set the model can reason about by name or structure, give it read, find and list tools, which fetch exact, current text and allow precise citations. If the corpus is so large the model cannot know what exists, add a search tool, keyword or vector, but keep reads as tools so the model pulls full text for whatever search surfaces. If you need both whole-document work and corpus-wide search, combine them. Long context loses when content is large or changes often; chunked RAG loses when the task needs whole documents, the exact clause or verbatim quotes.

    Explained in Retrieval: RAG vs Tools vs Long Context: A decision guide

  6. 06Do you always need a vector database for RAG?

    No. For an agent working over a bounded, identified set of documents, such as a user's files or a project folder, tool-based retrieval often wins. When the model reads the actual paginated text it can quote verbatim and cite precise locations, which is decisive in law, medicine and finance. Tools fetch the current version every time, while a vector index must be re-embedded when content changes. Explicit fetching grounds the model in real text instead of training priors, and there is no embedding pipeline, vector store or chunking strategy to tune. The cost is extra round trips, reduced with batch tools and cheap targeted tools. Add semantic search as a tool only when corpus scale demands it.

    Explained in Retrieval: RAG vs Tools vs Long Context: Why tools are often the right default for document agents

Back to topics

Agent memory interview questions

How agents write, manage and read memory across runs, and how it fails.

  1. 01Why do AI agents need memory if models have large context windows?

    A plain LLM is stateless: it reads a prompt, responds and forgets, so each request starts blank unless you paste earlier context back in. That is fine for one question but limits an agent working across many steps, tools and days. You cannot dump every past interaction into the prompt either: even very large windows hit limits, raise costs and lose quality when flooded with irrelevant history. So you build a memory layer, a separate system that stores, indexes and retrieves only the relevant context on demand. The survey behind the Practical Guide claims the gap between having memory and not having it is often larger than the gap between different LLM backbones, so memory deserves first-class attention.

    Explained in Why Memory Matters, and the Write-Manage-Read Loop: The problem memory solves

  2. 02What are the four types of agent memory: working, episodic, semantic and procedural?

    Working memory is the context window itself: fast, small and temporary. It fails through attentional dilution, the lost-in-the-middle effect, where information is present but ignored. Episodic memory is a structured logbook of what happened and when, such as requests, tools used, outcomes and errors, stored as summaries rather than raw logs; it supports recall and debugging. Semantic memory holds distilled, durable facts such as a user's tech stack, and its main challenge is curation, or it becomes a junk drawer. Procedural memory holds skills and behavioral rules, such as persona and constraint files like AGENTS.md; it is the most neglected tier and should be treated as code. Build each type only when you need it.

    Explained in The Four Types of Memory: Key takeaways

  3. 03What is the write-manage-read loop for agent memory, and which phase do most systems get wrong?

    It frames memory as three phases rather than store-and-retrieve. Write is when observations, results and reflections enter memory. Manage is ongoing maintenance: pruning, compressing, consolidating and reconciling. Read is retrieving relevant memory and injecting it into the context window when needed. Most implementations get write and read right and neglect manage, so they accumulate information without curation and end up with noise, contradictions and a bloated context. Manage is the hard part: deciding what to keep, what to summarize, what to merge and when to let things age out. Even an explicit heuristic policy for what to store, summarize, escalate to long-term memory or expire forces those decisions into the open.

    Explained in Why Memory Matters, and the Write-Manage-Read Loop: The write-manage-read loop

  4. 04How does memory consolidation work, and how do you handle duplicate or conflicting memories?

    Do not blindly append. In the AWS AgentCore pipeline, an asynchronous LLM step first extracts meaningful information into a schema and skips chatter. Consolidation then retrieves the most similar existing memories from the same namespace and strategy, and asks an LLM to choose ADD for new information, UPDATE when it complements or changes an existing memory, or NO-OP when it is redundant, so loves pizza and likes pizza do not trigger an update. Conflicts are resolved by recency while preserving history: if a budget changes from 500 to 750, the new value becomes active and the old one is marked INVALID rather than deleted, which keeps an audit trail. Failed consolidations retry with exponential backoff and still store the memory.

    Explained in How Memory Is Built and Retrieved: How a production pipeline does write and manage: AgentCore

  5. 05What is semantic drift in agent memory, and how do you prevent it?

    Semantic drift is stored knowledge slowly deviating from the truth through repeated lossy compression. Each re-encoding, from raw text to summary to higher-level reflection, strips nuance. In the SSGM paper's example, I like mild spicy food becomes likes spicy food, then loves very spicy food, until the agent recommends ghost pepper wings. Each step looked reasonable, which is what makes drift dangerous. Related forms include procedural drift, where a convoluted workaround hardens into the default workflow, and goal drift away from the original instructions. The defense is keeping raw memories linked to their summaries so you can reconcile against what actually happened; SSGM formalizes this as anchoring memory against an immutable ledger.

    Explained in How Memory Fails: Drift: knowledge slowly deviating from the truth

  6. 06How do coding agents like Claude Code and Codex remember project context between sessions?

    The model is stateless, so project knowledge lives outside the weights in files the harness loads. CLAUDE.md for Claude Code and AGENTS.md for Codex are markdown files in the repo, injected near the front of the prompt every session, holding conventions, test commands and architecture notes. Claude Code reloads CLAUDE.md after compaction, so persistent rules belong there, not in chat, which gets compacted away. Auto memory holds notes the agent writes itself, loaded in a capped amount at session start; path-scoped rules load only when matching files are touched; a file scratchpad holds intermediate results. A two-tier pattern keeps shared rules in git and personal memory per user, promoting validated discoveries to the shared tier.

    Explained in Memory Outside the Weights: Concept explanation

Back to topics

Multi-agent systems interview questions

When several agents help, when they hurt, and why they fail.

  1. 01Single agent vs multi-agent: how do you decide?

    Ask whether the task is mostly reading or mostly writing. Read tasks, such as research, search and synthesis, split into independent parts, so they parallelize well and suit multi-agent systems. Write tasks, such as code, a long document or a set of file edits, have interdependent parts; splitting them across parallel agents creates coordination problems, so they favor a single agent with unified context. For mixed tasks, separate the phases: parallel reading, then single-threaded writing. That is why Anthropic chose multi-agent for research and Cognition chose a single agent for coding. Weigh cost too: multi-agent systems use roughly 15 times the tokens of a plain chat, against roughly 4 times for a single agent.

    Explained in Single Agent versus Multi-Agent: The Core Tradeoff: The insight that reframes the whole debate: read versus write

  2. 02What is the orchestrator-worker pattern, and how does Anthropic's multi-agent research system use it?

    A lead agent, the orchestrator, analyzes the query, picks a strategy and writes its plan into memory, since the system persists context once conversations pass 200,000 tokens. It spawns subagents, each with a specific assignment, its own context window and its own tools; they search in parallel and condense findings before returning them. The lead synthesizes the results and decides whether to spawn more subagents or refine the strategy. A dedicated Citation Agent then checks every claim against its sources. Parallelism runs at two levels: 3 to 5 subagents at once, each making parallel tool calls, which cut research time by up to 90 percent on complex queries. Anthropic reports this beat a single-agent Claude Opus 4 by 90.2 percent on its research evaluation.

    Explained in The Case For Multi-Agent: Anthropic's Research System: The architecture: orchestrator-worker

  3. 03Why did Cognition argue against building multi-agent systems, and what are its two principles?

    Walden Yan at Cognition argues that the common pattern of splitting work, running subagents in parallel and merging results is fragile. In his Flappy Bird example, one subagent builds a background that looks like Super Mario Bros while another builds a bird that moves nothing like Flappy Bird, and the merge step is stuck. Principle 1: share context, and share full agent traces, not just individual messages, because the trajectory behind an assignment shapes how it should be read. Principle 2: actions carry implicit decisions, and conflicting decisions carry bad results; parallel agents that cannot see each other's choices make clashing assumptions. His default is a single-threaded linear agent, adding a compression step for very long tasks.

    Explained in The Case Against: Cognition's "Don't Build Multi-Agents": Principle 2: Actions carry implicit decisions

  4. 04Why do multi-agent LLM systems fail? Explain the MAST taxonomy.

    MAST, the Multi-Agent System Failure Taxonomy, catalogs 14 failure modes derived from manually annotating over 200 execution traces. They fall into three groups. Specification issues, about 41.8 percent, come from poor prompts, missing role constraints or no stopping criteria: disobeying the task, repeating finished steps, losing history, missing that the task is done. Inter-agent misalignment, about 36.9 percent, covers ignoring other agents, proceeding without clarification, and reasoning that does not match action. Task verification failures, about 21.3 percent, include ending early, skipping validation, or shallow checks like accepting code because it compiles. The same model often does better as a single agent, so coordination, not capability, is what breaks these systems.

    Explained in Why Multi-Agent Systems Fail: The MAST Taxonomy: The three categories of failure

  5. 05What is a subagent, and when should a coding agent use one?

    A subagent is a fresh agent instance with its own context window that the parent hands a task; it works autonomously and returns a result. Its main purpose is context isolation: research or exploration produces noisy intermediate output like file dumps, search results and DOM snapshots, and the subagent absorbs that in its own window and returns only a tidy summary. Give each one a contract: a self-contained input, since it cannot see the parent's history, and a structured output the next step can consume. The costs are real: five parallel subagents use roughly five times the tokens, each pays startup overhead, and for a simple task one can be slower than the main agent. Keep a few, sharply described.

    Explained in Subagents and Multi-Agent Orchestration: Concept explanation

  6. 06Anthropic vs Cognition on multi-agent systems: who is right?

    Both, because single versus multi is a false dichotomy. Both positions center on managing context: the prompts, tool calls, data and communication between agents. Cognition avoids dividing context at all, keeping one agent and scaling its reach with context compression. Anthropic divides context across subagents but invests in making the division clean, with highly specific subagent instructions and precise tool definitions so miscommunication does not creep in. Task shape decides the rest: Anthropic's research agent does a read task that splits into independent subtopics, while Cognition's Devin does a write task that needs the codebase and prior changes in one context. Both agree the last mile from prototype to production is most of the work.

    Explained in Reconciling the Debate: It Was Never Single versus Multi: The hinge: both sides are really talking about context

Back to topics

Tools, MCP and coding agents interview questions

Tool execution, sandboxing, MCP, skills and hooks inside coding agents.

  1. 01What happens between a model emitting a tool call and the agent seeing the result?

    The model only asks; the harness executes. A production harness runs a pipeline: parse the JSON arguments, validate them against the tool's schema, check permissions or approvals, execute (ideally in a sandbox), capture stdout, stderr, exit code and timing, then truncate and format the result. The last step matters more than it looks. A command that prints nothing and exits 0 succeeded, while one that prints nothing and exits 1 failed, so forwarding raw stdout leaves the model guessing. Long output is cut by keeping the head and the tail, because the start shows what was attempted and the end shows the result or error. Skipping these middle steps is what makes toy agents brittle.

    Explained in Tools and Tool Execution: Concept explanation

  2. 02How do sandboxing, approvals and checkpoints keep a coding agent safe?

    Each covers a different gap. The sandbox limits what an executed command can physically touch at the OS level: Codex uses Seatbelt (sandbox-exec) on macOS, Landlock on Linux and a purpose-built sandbox on Windows. Approvals decide what runs without asking, in tiers: safe commands like ls or git status auto-approve, allowlisted patterns pass, and sandbox violations such as network access or writes outside the workspace need explicit approval. For unattended runs, Smart Approvals send risky actions to a guardian subagent instead of a human. Checkpoints snapshot files before edits so you can rewind, but they cover only file changes. A deploy or API call cannot be undone, so actions with external side effects always prompt.

    Explained in Sandboxing, Approvals, and Checkpoints: Concept explanation

  3. 03MCP and skills: how do you extend a coding agent without bloating its context?

    Load both lazily. A skill is a SKILL.md runbook: YAML frontmatter plus an instruction body, optionally with scripts and templates. Through progressive disclosure the agent sees only names and descriptions at startup (about 500 tokens in total in the Seven Pillars example), and a full body, often 5,000-plus tokens, loads only when invoked. MCP is an open protocol that exposes external systems as named tools, often prefixed like mcp__server__action. A server with dozens of tools can flood the window, so defer schemas: load tool names up front and fetch a full schema on demand. Keep tool ordering stable to protect the prompt cache, and remember MCP tools sit outside the harness sandbox and must guard themselves.

    Explained in Skills, MCP, and Deterministic Computation: Concept explanation

  4. 04CodeAct vs JSON tool calls: what is the difference?

    With JSON tool calls the model picks from a fixed menu, emitting something like {"action": "search", "query": "..."} for the harness to run. CodeAct makes the action a short Python program; the sandbox runs it and returns the output or error as the observation. Code is more expressive: one snippet can chain operations, branch, loop and use any library, and when it throws, the agent reads the error and rewrites the code like a developer at a REPL. The CodeAct paper (ICML 2024) reported significantly higher success on complex tool-use tasks than text or JSON actions. The price is safety: arbitrary code execution needs a locked sandbox and strict rules, such as one action per step and no irreversible operations without permission.

    Explained in Autonomous Task Agents: Manus and CodeAct: Concept explanation

  5. 05What are hooks in a coding agent, and how do they differ from skills?

    A hook is a handler that fires at a fixed point in the agent lifecycle: PreToolUse before a tool runs, PostToolUse after it, Stop when a turn ends, SubagentStop when a subagent finishes, and PreCompact before compaction. A skill is invoked when the model decides it is relevant; a hook fires every time its event occurs, whatever the model wants. So skills are suggestions and hooks are rules: a PreToolUse hook that blocks git push blocks it always, not usually. Hooks also run in your own process and cost no model context, unlike an instruction you pay for on every turn. Pair them with the built-in approval policy, which handles the common safety tiers while hooks add project-specific rules.

    Explained in Observability, Hooks, and the Agent SDK: Concept explanation

  6. 06How does a coding agent like Codex assemble its prompt, and why does the order matter?

    A request carries layered instructions, a tools list of JSON schemas, and an input list of typed items. Before the user's message, Codex inserts standing context: a developer message with sandbox and permission rules, optional developer instructions from config, a user message with the aggregated AGENTS.md and skills text, and an <environment_context> block with the working directory and shell. The actual user message goes last. On later turns, earlier items such as tool calls and their results are appended, plus any context section that changed, so the old prompt stays an exact prefix of the new one. That prefix property is deliberate: it is what makes prompt caching work.

    Explained in Building the Prompt: How it works: assembling one request

Back to topics

AI agent system design interview questions

Production architecture: tenancy, keys, reliability, cost and scale.

  1. 01How would you design a production AI agent system end to end?

    Walk the request path. An API tier handles edge hardening (security headers, CORS, rate limiting), authenticates through one middleware, authorizes with centralized access helpers and validates input. The agent core builds context, runs the orchestrator loop and tool executor, and streams a typed event timeline back to the client over SSE. Under it sit a provider layer (adapter, model registry, tiering) and tools that read and write object storage, a relational database and cached external APIs. Durable state lives in the database and object storage, operator and user keys come from a secrets manager, traces flow to observability, and heavy work moves to queue-fed workers when scale demands it. The model is the easy part; the systems engineering around it is the job.

    Explained in Reference Architecture and a Design Checklist: The reference architecture

  2. 02How would you isolate tenants in a multi-tenant AI agent?

    Treat authentication, authorization and isolation as separate questions, since many breaches come from authenticating a user and then assuming they may touch whatever they asked for. For isolation by construction, scope storage keys by owner so the layout reflects ownership and any cross-tenant access needs an explicit, checkable decision. Never let a client name a raw internal id without the access check running on that id. Default new resources to private and model sharing as data, such as a shared_with list or a shares table, that the access helpers consult. Filter list or batch inputs down to accessible ids before acting, and authorize every tool call against the user, not the model's stated intent.

    Explained in Security, Auth, and Multi-Tenancy: Multi-tenancy and isolation

  3. 03How should you store user API keys in a bring-your-own-key (BYOK) agent?

    Keep operator secrets in the environment or a secrets manager, and store user secrets in the database encrypted so a dump alone is useless. Use authenticated symmetric encryption such as AES-256-GCM, with the key derived from an operator secret in the environment, never the database. Generate a fresh random IV per record, because IV reuse with GCM is catastrophic, and store the ciphertext, IV and auth tag. If the tag fails to verify, fail closed and return nothing. Resolve credentials as user key, then operator key, then error, so shared-key and BYOK deployments run on one code path. The UI gets only status and source, never the secret, and secrets never appear in logs.

    Explained in Secrets and Bring-Your-Own-Key: Storing user secrets: encryption at rest

  4. 04How do you make an LLM agent reliable when model APIs and tools fail?

    Assume everything fails and define a response for each case. Parse model output defensively: malformed arguments default to empty, an unknown tool name returns a 'not available' result, and every tool call gets some result back, because providers require that pairing. Retry only transient errors such as timeouts, rate limits and 5xx responses, with capped exponential backoff and jitter, honoring retry-after. Make side-effecting operations idempotent with deterministic keys or upserts so retries cannot duplicate them. Persist progress per unit so a crash resumes instead of restarting. Classify each dependency as fatal or degradable, set timeouts on every external call, emit a clean error event when a stream fails, and cap iterations to stop runaway loops.

    Explained in Reliability: Retries, Idempotency, and Failure Handling: The reliability checklist

  5. 05How do you reduce the cost and latency of an LLM agent?

    Measure first, because a few patterns usually dominate the bill. The biggest lever is model tiering: route each task to the cheapest model that does it well, with the top tier for the main interactive task, the mid tier for bulk extraction and classification, and the low tier for titles and yes/no checks. Turn reasoning off for bulk work and zero the thinking budget where the provider allows. Shrink context, since every token is re-sent on each loop iteration: reference documents through tools, distill tool results, curate history. Batch tools to cut iterations, parallelize independent work, keep a stable prefix for prompt caching, right-size output, and stream so the remaining latency feels shorter.

    Explained in Cost, Latency, and Model Tiering: The cost/latency playbook

  6. 06How do you scale an AI agent backend horizontally?

    Scaling an agent is less about raw compute than about concurrency, statelessness and the slow upstream bottleneck, since your servers mostly wait on the model provider. Keep app servers stateless, with durable state in the database, object storage and cache and identity in per-request tokens, so any instance behind a load balancer can serve any user. Use async, non-blocking I/O so many long-lived SSE streams stay cheap, and stop proxies from buffering them. Move OCR, document conversion and bulk extraction to queue-fed workers that scale separately. Queue and bound fan-out toward providers, because more instances do not raise their rate limits. Start as one well-structured service and let observability decide when to add infrastructure.

    Explained in Scaling and Infrastructure: The scaling mindset

Back to topics

LLM fundamentals interview questions

Attention, scaling laws, alignment, LoRA and mixture of experts.

  1. 01How does self-attention work in a Transformer?

    Self-attention lets every token look at every other token in a single step and decide what to pay attention to, however far apart they are. Each token's embedding produces three vectors: a Query (what I am looking for), a Key (what I contain) and a Value (what I hand over if picked). To decide how much token A attends to token B, the model compares A's Query with B's Key. Softmax turns the scores into weights that sum to 100 percent, and A's new representation is the Values mixed by those weights. Multi-head attention runs several such passes in parallel, each free to learn a different relationship; the original paper used eight heads.

    Explained in The Transformer, the Engine Inside Every Modern Model: The one big idea: attention

  2. 02What is the Chinchilla scaling law, and how did it correct the original scaling laws?

    The 2020 Scaling Laws paper showed that loss falls along a smooth power law as you add parameters, data or compute, so small experiments can forecast a large model's performance before you pay for it. Its advice leaned toward spending extra compute mostly on a bigger model. Chinchilla (DeepMind, 2022) found models were too big and trained on too little data: for a fixed compute budget, grow model size and data in equal measure, roughly 20 training tokens per parameter. Chinchilla, at 70 billion parameters and 1.4 trillion tokens, beat the 280 billion parameter Gopher on almost everything. The smaller, well-trained model is also cheaper to serve, because inference cost depends on size.

    Explained in Scaling Laws and Chinchilla, How Big Should a Model Be?: Paper 2: Chinchilla, the correction

  3. 03How does RLHF work? Walk through the InstructGPT pipeline.

    A pre-trained model is a strong autocomplete, not an assistant, and good behavior is hard to write as a formula, so RLHF learns it from human judgment in three steps. First, supervised fine-tuning: humans write ideal answers and the model learns to imitate them. Second, humans rank several model answers from best to worst, and those rankings train a reward model that scores any answer. Third, reinforcement learning with PPO: the model writes answers, the reward model grades them, and the model shifts toward higher scores. A KL penalty keeps it close to its SFT version so it cannot game the reward model. The 1.3 billion parameter InstructGPT was preferred by humans over the 175 billion parameter GPT-3.

    Explained in Alignment, Turning a Text Predictor Into a Helpful Assistant: Paper 1: InstructGPT and the RLHF recipe

  4. 04DPO vs RLHF: what is the difference?

    Both run on the same fuel, human preference comparisons; they differ in machinery. RLHF trains a separate reward model and then runs a PPO reinforcement learning loop against it, which is complex and can be unstable. DPO, whose subtitle is 'your language model is secretly a reward model', folds that into one direct training step on preference pairs: a prompt, a preferred answer and a rejected answer. The objective makes the preferred answer more likely and the rejected one less likely, while staying close to the original model, the same leash idea built in. DPO is simpler, cheaper and more stable, though big labs still use reinforcement learning for the hardest cases.

    Explained in Alignment, Turning a Text Predictor Into a Helpful Assistant: Paper 2: DPO, the same goal with far less machinery

  5. 05What is LoRA, and why does it make fine-tuning cheap?

    Full fine-tuning updates every parameter, which costs memory, storage (a full-size copy per variant) and time. LoRA bets that the weight change a new task needs has low rank, so two skinny matrices can capture it: a 1000 by 1000 update approximated by a 1000 by 8 matrix times an 8 by 1000 matrix is 16,000 numbers instead of a million. LoRA freezes the base model, adds these low-rank adapters beside the layers being adapted, and trains only them. Trainable parameters can drop by a factor of thousands and adapter files are often a few megabytes. One base model can swap many task adapters, and an adapter can be merged into the weights after training, so inference is no slower.

    Explained in LoRA, Fine-Tuning Giant Models on a Budget: How LoRA works

  6. 06How does a mixture of experts model like Mixtral work, and what is the trade-off?

    A dense model uses every parameter for every token. Mixtral replaces each layer's single feed-forward network with eight experts, and a small router scores them and sends each token to the top two. It holds about 47 billion parameters in total, but only about 13 billion do work per token, so you get large-model capacity at closer to small-model running cost; the paper reports it matching or beating Llama 2 70B. The router learns the division of labor itself, and its choices track grammar and structure more than clean topics. The cost is memory: all eight experts must stay loaded, because the router may pick any of them for the next token.

    Explained in Mixtral and Mixture of Experts, More Brain, Same Speed: How Mixtral is built

Back to topics

Reasoning models interview questions

Chain of thought, ReAct, process supervision, DeepSeek-R1 and beyond.

  1. 01What is chain-of-thought prompting, and why does it improve reasoning?

    Chain of thought asks the model to write out intermediate steps before the final answer. It helps because a model produces one token at a time with a fixed, small amount of computation per token, so jumping straight to a number leaves no room to work. Each written step becomes input for the next, acting as scratch paper the model can build on. You trigger it few-shot, by including examples with worked steps, or zero-shot, by adding a phrase such as 'Let's think step by step'. It needs no retraining, but it is an emergent ability: it barely helps small models and can make them worse, then gives a large improvement once models are big enough.

    Explained in Chain of Thought, Teaching Models to Think Out Loud: The fix: ask it to reason step by step

  2. 02ReAct vs chain of thought: what does ReAct add?

    Chain of thought gives the model an inner monologue, but it can only reason about what it already knows, and when a fact is missing it may confidently invent one. Pure acting, tools without reasoning, flails without a coherent plan. ReAct interleaves the two in a loop: Thought (what do I need next), Action (call a tool such as search), Observation (read the result), repeated until there is enough to answer. Reasoning chooses the next action and interprets results, while observations ground the next thought in real information, which counters hallucination. Because thoughts and actions are written out, you can read the trace and see why the agent acted as it did.

    Explained in ReAct, Reasoning and Acting Together: Two failure modes ReAct fixes

  3. 03Process supervision vs outcome supervision: what is the difference?

    Outcome supervision rewards a solution only on whether the final answer is correct; process supervision rewards each reasoning step. Grading only outcomes can give full marks to a lucky answer reached through flawed steps or mistakes that cancel out. In 'Let's Verify Step by Step', the authors trained an outcome reward model (ORM) and a process reward model (PRM), the PRM learning from human labels on individual steps. They sampled many candidate solutions and let each verifier pick the best. The process verifier chose correct solutions far more reliably, especially on hard math, because it pinpoints where reasoning breaks and is harder to fool with confident but flawed work.

    Explained in Let's Verify Step by Step, Rewarding Good Reasoning: The trap of grading only the final answer

  4. 04How did DeepSeek-R1 learn to reason through reinforcement learning?

    DeepSeek-R1 used reinforcement learning with verifiable rewards: for math and code an answer can be checked automatically (correct or not, tests pass or not), so there is no learned reward model to fool. R1-Zero applied pure RL to a base model with no supervised warm-up, and longer chains of thought, self-checking and reconsidering an approach emerged on their own, including an 'aha moment' where it paused to re-evaluate. Its output was hard to read and sometimes mixed languages, so the full R1 added a small set of clean cold-start examples before RL, followed by more stages. It reached performance comparable to OpenAI's o1, and distillation passed the skill to smaller models.

    Explained in DeepSeek-R1, Learning to Reason Through Reinforcement Learning: DeepSeek-R1-Zero: reasoning appears on its own

  5. 05What are recursive language models, and how do they handle inputs longer than the context window?

    Long inputs fail two ways: they may not fit the context window, and even when they fit, accuracy degrades as length grows. A recursive language model keeps the input outside the model as a variable in a REPL. The root model writes code to inspect it: check its length, search it, slice it into relevant chunks. When a chunk is still too large or complex, the model calls a fresh copy of itself on that chunk, then combines the sub-results. The paper reports handling inputs up to a hundred times larger than the normal window, often beating a plain model even on inputs that fit, at comparable or lower cost per query.

    Explained in Recursive Language Models, Thinking Beyond the Memory Limit: The idea: treat the long prompt as an environment, not a thing to read

  6. 06What is inference-time compute (test-time scaling), and how do reasoning methods use it?

    Inference-time compute means spending more thinking effort at the moment of answering, also called test-time scaling. Chain of thought spends it on writing intermediate steps. Verifier-based selection spends it on generating many candidate solutions and checking them. Recursive language models spend it on programmatically exploring a huge input and recursively breaking it down, with the model deciding on its own how to divide and conquer. The common trade is extra computation for better answers on harder problems. It also explains why the recursive approach can stay cheap: effort goes to the parts of the input that matter instead of processing everything.

    Explained in Recursive Language Models, Thinking Beyond the Memory Limit: Why this works so well

Back to topics

LLM and agent evaluation interview questions

LLM judges, benchmarks, and how to test an agent before a change ships.

  1. 01What is LLM-as-a-judge, and how reliable is it?

    LLM-as-a-judge uses a strong model, such as GPT-4, to compare or score other models' answers in place of a human. It exists because open-ended tasks have no answer key and human voting does not scale to every model tweak. A judge is fast, cheap and available around the clock, so you can run thousands of comparisons in minutes. The 'Judging LLM-as-a-Judge' paper found a strong judge agreed with human preferences about 80 percent of the time, roughly the rate at which two humans agree with each other. That makes it about as reliable as a typical human judge, provided you control for its biases.

    Explained in Judging Models, How Do We Measure Quality?: The big idea: let an LLM be the judge

  2. 02What biases does an LLM judge have, and how do you mitigate them?

    Three are documented. Position bias: the judge favors whichever answer it sees first, so run each comparison with the order swapped and average. Verbosity bias: it prefers longer answers even when a shorter one is better, so watch for padding passing as quality. Self-enhancement bias: it favors answers in its own style and can rate its own model family more highly. None of these makes a judge useless; they make it a tool to use carefully. A practical split is the fast LLM judge for everyday iteration, a fixed set like MT-Bench as a consistent yardstick, and human votes in Chatbot Arena as the final source of truth.

    Explained in Judging Models, How Do We Measure Quality?: Be careful: judges have biases

  3. 03What does SWE-bench actually measure?

    Whether a model can resolve a real GitHub issue in a real codebase. Each task pairs an issue with the pull request that fixed it. The model gets the issue text and a snapshot of the repository from just before the fix, and must produce a patch. SWE-bench applies the patch and runs the project's own tests; if the tests the original fix was meant to satisfy now pass, the issue counts as resolved. That is execution-based evaluation, not text matching. The full set has 2,294 tasks from 12 popular Python projects. Unlike HumanEval-style tasks that ask for one small function, it requires navigating a large repository. The best model the authors tested, Claude 2, resolved about 1.96 percent.

    Explained in SWE-bench, Can a Model Fix a Real Bug in Real Code?: The core idea

  4. 04What is BIG-bench, and what did it show about emergent abilities?

    BIG-bench is a crowdsourced suite of 204 tasks from more than 450 researchers, chosen to be hard for 2022 models and broad rather than deep. It adds a human expert baseline, a cheaper BIG-Bench Lite subset, and a canary string so contamination can be detected. Performance and calibration improved with scale but stayed well below the human raters. Knowledge-heavy tasks improved smoothly, while multi-step tasks such as checkmate-in-one showed breakthrough behavior: near zero for a long time, then a sudden jump past a critical size. Large models were also brittle, with scores swinging on small wording changes, and social bias often grew with scale on ambiguous prompts.

    Explained in BIG-Bench, Testing the Full Breadth of What a Model Can Do: What they found

  5. 05How does Chatbot Arena rank LLMs, and why is it hard to game?

    A user asks a question, two anonymous models answer side by side, and the user votes for the better reply. Names are revealed only after the vote, so brand reputation cannot sway it. Each vote is one head-to-head match, and the paper reports over 240,000 of them. It turns matches into ratings with the Bradley-Terry model, a formal version of the Elo idea where beating a strong opponent counts for more, and reports a confidence interval for each rating. Active sampling favors pairs whose relative strength is still uncertain. Because questions arrive live from real users and there is no published answer key, memorizing a test set does not help.

    Explained in Chatbot Arena, Letting Real People Pick the Winner: The core idea

  6. 06How do you evaluate an AI agent before shipping a prompt, tool or model change?

    Build an eval set of realistic inputs with known-good expectations, covering common cases, tricky ones and past bugs, and run it on every prompt, tool or model change against a baseline. That catches the tweak that fixes one case and silently breaks five others. Measure several dimensions: task success, grounding (does each cited quote appear at the cited location), format compliance, tool-use correctness, refusal behavior, and cost and latency. Grade with deterministic checks wherever possible, an LLM judge with a rubric for subjective quality, calibrated and spot-checked against humans, and human review for the highest-stakes cases. Feed surprising production cases back into the set.

    Explained in Observability and Evaluation: From observability to evaluation

Back to topics