What is it?
Supervised fine-tuning (SFT)
Supervised fine-tuning is the first step of alignment (Chapter 3). Humans write high-quality example answers to many prompts, and the model is trained to imitate them. "Supervised" means the model learns from labeled examples of the correct behavior.
SFT teaches the model the basic shape of being helpful: when asked a question, give an answer. It is the foundation that later steps (RLHF or DPO) build on.
From the Foundational Modelling glossary
As the Planning and Reasoning glossary puts it
Supervised fine-tuning is training a model to imitate labeled example answers, the straightforward "here is the correct response, copy this style" form of training. It contrasts with reinforcement learning, which uses rewards instead of fixed correct answers. In Chapter 4, DeepSeek-R1-Zero deliberately skipped SFT to see if reasoning could emerge from pure RL, and the full DeepSeek-R1 added just a little clean example data to tidy up presentation. (See the folder 01 glossary for the role of SFT in alignment.)
Where Supervised fine-tuning is explained
- Deep Research Agents and How They Are TrainedBuilding Coding Agents and Harnesses · How they are trained
- Alignment, Turning a Text Predictor Into a Helpful AssistantFoundational Modelling · Paper 1: InstructGPT and the RLHF recipe
- DeepSeek-R1, Learning to Reason Through Reinforcement LearningPlanning and Reasoning · DeepSeek-R1-Zero: reasoning appears on its own