Skip to slide
20 / 53

CH 03 · Let's Verify Step by Step, Rewarding Good Reasoning · 01/07

Read as one page

Let's Verify Step by Step, Rewarding Good Reasoning

Grading every step of a model's reasoning, not just the final answer, produces a verifier that rejects lucky guesses and finds correct solutions more often.

7 slides · 3 min read

Contour map for Let's Verify Step by Step, Rewarding Good Reasoning

In this chapter

  1. 01The trap of grading only the final answer
  2. 02Two kinds of graders
  3. 03How a verifier makes a model smarter
  4. 04Why process supervision wins
  5. 05Why this paper mattered
  6. 06The one-sentence takeaway

Paper: Let's Verify Step by Step (2023)

Chapters 1 and 2 got the model to reason and act. But a hard question lurks underneath: when a model produces a long chain of reasoning, how do we judge it? The obvious answer is to check whether the final answer is right. This paper shows that the obvious answer is not the best one. It turns out that grading each step of the reasoning, rather than just the final result, makes models dramatically better at hard problems. This is a subtle idea with big consequences.

Finished this idea?

Your learning progress stays in this browser.