CH 04 · Glossary: Benchmarks · 05/13
Read as one pageHuman preference
Human preference is a way of scoring answers not against a correct solution but by asking people which answer they like better. It is the natural metric for open-ended tasks where quality is a matter of judgment rather than correctness. Its strength is that it captures what users actually value; its difficulty is that human opinions are noisy and can disagree, so you need many judgments and careful statistics to turn them into a reliable signal. Chatbot Arena is built entirely on human preference.
Finished this idea?
Your learning progress stays in this browser.