Skip to slide
18 / 74

CH 02 · Scaling Laws and Chinchilla, How Big Should a Model Be? · 06/06

Read as one page

The one-sentence takeaway

Model quality improves in a smooth, predictable way as you add parameters, data, and compute, and the smartest way to spend a compute budget is to grow the model and its training data together, not to just make the model enormous.

Next: Chapter 3, Alignment with RLHF and DPO, where a raw text predictor learns to become a helpful assistant.

Finished this idea?

Your learning progress stays in this browser.