Skip to slide
45 / 74

CH 06 · Judging Models, How Do We Measure Quality? · 07/08

Read as one page

How the three pieces fit together

ToolWhat it isStrengthWeakness
MT-BenchFixed hard question setRepeatable, targetedLimited set of questions
Chatbot ArenaLive human voting with EloGold-standard human preferenceSlow, expensive, needs crowds
LLM-as-a-judgeA strong model grades answersFast, cheap, scalableCarries biases, needs care

Together they form a practical toolkit: use the fast LLM judge for everyday iteration, use MT-Bench for a consistent yardstick, and use the human Arena as the ultimate source of truth.

Finished this idea?

Your learning progress stays in this browser.