Skip to slide
31 / 39

CH 04 · Glossary: Benchmarks · 05/13

Read as one page

Human preference

Human preference is a way of scoring answers not against a correct solution but by asking people which answer they like better. It is the natural metric for open-ended tasks where quality is a matter of judgment rather than correctness. Its strength is that it captures what users actually value; its difficulty is that human opinions are noisy and can disagree, so you need many judgments and careful statistics to turn them into a reliable signal. Chatbot Arena is built entirely on human preference.

Finished this idea?

Your learning progress stays in this browser.