Skip to content
Agent YAP

What is it?

Metric

A metric is the specific rule you use to turn a model's output into a number. Accuracy (the fraction of questions answered correctly) is the most common, but there are many others, and the choice of metric quietly shapes what a benchmark rewards. A poorly chosen metric can make a weak model look strong or vice versa, which is why the BIG-Bench authors warned that even automatic, programmatic metrics can be surprisingly subjective. Whenever you read a benchmark score, it is worth asking exactly what was measured.

From the Benchmarks glossary

Where Metric is explained

  1. Observability and EvaluationArchitecture and System Design · From observability to evaluation
  2. Domain-Specific and Regulated-Industry AgentsArchitecture and System Design · Evaluation with domain rigor
  3. Practical Lessons from Building ManusContext Engineering · Verify with computable, binary checks: the Intern Test
  4. Practical Guidance: When and How to Use Agentic RAGAgentic RAG · Lesson 5: Evaluation must account for process, not just outcomes
  5. What Is an Agent Harness?Building Coding Agents and Harnesses · Real code: Codex splits the harness into crates

Every term, A to Z