What is it?
Scaling law
A scaling law is a predictable mathematical relationship showing how a model's performance improves as you increase its size, its training data, or its compute. The surprising discovery (Chapter 2) is that this improvement is smooth and forecastable across a huge range, often appearing as a straight line on the right kind of graph.
Scaling laws matter because they remove guesswork. A small experiment can predict how a much larger model will perform before you spend the money to build it.
From the Foundational Modelling glossary