Positional encoding

Because self-attention looks at all words at the same time, it has no natural sense of order. Without help, "dog bites man" and "man bites dog" would look the same to it. Positional encoding fixes this by adding a small signal to each word's embedding that marks its position in the sequence: first, second, third, and so on.

With positional encoding, the model knows not just which words are present, but in what order, which is essential for meaning.