CH 01 · The Transformer, the Engine Inside Every Modern Model · 05/07
Read as one pageEncoder and decoder
The original Transformer had two halves, an encoder and a decoder. The encoder reads and understands the input (useful for translation). The decoder generates the output one word at a time. Most chat models today, like GPT, use only the decoder half, because their main job is to generate text. It is worth knowing both exist, because the words "encoder" and "decoder" show up constantly.
Finished this idea?
Your learning progress stays in this browser.