The one-sentence takeaway

The Transformer lets every word in a sentence directly look at every other word and decide what to pay attention to, and stacking that simple idea many times, at scale, is enough to learn language.

Next: Chapter 2, Scaling Laws and Chinchilla, where we ask how big these models should actually be.