Skip to content
Agent YAP

What is it?

Scaling law

A scaling law is a predictable mathematical relationship showing how a model's performance improves as you increase its size, its training data, or its compute. The surprising discovery (Chapter 2) is that this improvement is smooth and forecastable across a huge range, often appearing as a straight line on the right kind of graph.

Scaling laws matter because they remove guesswork. A small experiment can predict how a much larger model will perform before you spend the money to build it.

From the Foundational Modelling glossary

Where Scaling law is explained

  1. Scaling Laws and Chinchilla, How Big Should a Model Be?Foundational Modelling
  2. Start Here: Foundational Modelling, Explained SimplyFoundational Modelling · The big picture
  3. The Transformer, the Engine Inside Every Modern ModelFoundational Modelling · The one-sentence takeaway

Every term, A to Z