Skip to slide
22 / 53

CH 03 · Let's Verify Step by Step, Rewarding Good Reasoning · 03/07

Read as one page

Two kinds of graders

To put this into practice, the researchers trained two kinds of grading models, both relatives of the reward model you met in folder 01.

The PRM is a step-by-step verifier: a model whose job is to check reasoning, not to produce it. Human labelers went through thousands of solutions marking each step as correct or not, and the PRM learned to imitate those judgments.

Grading the answer versus grading the steps
Grading the answer versus grading the steps

Finished this idea?

Your learning progress stays in this browser.