Neural network

A neural network is the basic kind of program that all modern AI models are built from. The name comes from a loose inspiration by the brain, but you do not need biology to understand it. Picture a huge set of simple math operations arranged in layers. Numbers flow in at one end (for example, a sentence turned into numbers), pass through layer after layer of multiplications and additions, and a result comes out the other end (for example, a prediction of the next word).

What makes it powerful is that the network has millions or billions of adjustable knobs (see parameters). During training, those knobs are tuned automatically until the network produces good outputs. A "recurrent neural network," mentioned in Chapter 1, is an older style that processed text one word at a time while carrying a memory; the Transformer replaced it.