Skip to slide
12 / 74

CH 01 · The Transformer, the Engine Inside Every Modern Model · 07/07

Read as one page

The one-sentence takeaway

The Transformer lets every word in a sentence directly look at every other word and decide what to pay attention to, and stacking that simple idea many times, at scale, is enough to learn language.

Next: Chapter 2, Scaling Laws and Chinchilla, where we ask how big these models should actually be.

Finished this idea?

Your learning progress stays in this browser.