CHAPTER 02 · Scaling Laws and Chinchilla, How Big Should a Model Be? · 5 / 5
The one-sentence takeaway
Model quality improves in a smooth, predictable way as you add parameters, data, and compute, and the smartest way to spend a compute budget is to grow the model and its training data together, not to just make the model enormous.
Next: Chapter 3, Alignment with RLHF and DPO, where a raw text predictor learns to become a helpful assistant.