What is it?
Reinforcement learning (RL), PPO, GRPO
Reinforcement learning is a training approach where a system learns by trial and error, guided by rewards for good outcomes. PPO (Proximal Policy Optimization) and GRPO (Group Relative Policy Optimization) are two specific RL algorithms. In agent memory research, RL is used to train an agent to decide when to store, update, retrieve, or delete memories, by rewarding choices that help long-term task success.
From the AI agent glossary
As the Foundational Modelling glossary puts it
Reinforcement learning, or RL, is a style of training where a model learns by trial and error guided by rewards, rather than by copying labeled examples. The model tries an action, receives a reward signal indicating how good the outcome was, and adjusts to earn more reward over time. It is how you train an agent to play a game: not by showing it the perfect moves, but by rewarding it when it wins.
In language models, RL is used to push a model toward responses that score well according to human preferences (see RLHF). It also powers the reasoning models in folder 02, such as DeepSeek-R1.
As the Planning and Reasoning glossary puts it
Reinforcement learning, or RL, is training by trial and error guided by rewards rather than by copying examples. The model tries something, receives a reward signal indicating how good the result was, and adjusts to earn more reward over time.
RL is the engine behind Chapter 4. Rather than being shown how to reason, DeepSeek-R1 was rewarded for reaching correct answers, and it gradually discovered reasoning strategies on its own because those strategies earned more reward. (The basic mechanics also appear in the folder 01 glossary.)
Where Reinforcement learning is explained
- Deep Research Agents and How They Are TrainedBuilding Coding Agents and Harnesses · How they are trained
- Alignment, Turning a Text Predictor Into a Helpful AssistantFoundational Modelling · Paper 1: InstructGPT and the RLHF recipe
- DeepSeek-R1, Learning to Reason Through Reinforcement LearningPlanning and Reasoning · The catch, and the fix: DeepSeek-R1
- Security, Auth, and Multi-TenancyArchitecture and System Design · Application-layer authorization vs database RLS
- Start Here: Planning and Reasoning, Explained SimplyPlanning and Reasoning · The big picture