Skip to slide
10 / 74

CH 01 · The Transformer, the Engine Inside Every Modern Model · 05/07

Read as one page

Encoder and decoder

The original Transformer had two halves, an encoder and a decoder. The encoder reads and understands the input (useful for translation). The decoder generates the output one word at a time. Most chat models today, like GPT, use only the decoder half, because their main job is to generate text. It is worth knowing both exist, because the words "encoder" and "decoder" show up constantly.

Finished this idea?

Your learning progress stays in this browser.