2017 · ~5 min · 0/7 complete
Attention Is All You Need
Start with attention. Later chapters assume you know how a Transformer moves information between tokens.
AI research papers, explained
Every idea has a starting point. Follow three reading paths and see why each paper belongs where it does.
14 full guides · Three paths · Free, no sign-up
Attention, scale, alignment, adaptation, and the evidence used to judge a model.
0/6 guides completed · Suggested order, every paper is open.
Begin with: Attention Is All You Need2017 · ~5 min · 0/7 complete
Start with attention. Later chapters assume you know how a Transformer moves information between tokens.
2020 / 2022 · ~4 min · 0/6 complete
With the model architecture in place, ask how to divide a training budget between parameters and data.
Useful first: Attention Is All You Need
2022 / 2023 · ~5 min · 0/6 complete
A trained next-token predictor still needs a way to learn which responses people prefer.
Useful first: Scaling laws & Chinchilla
2021 · ~4 min · 0/7 complete
Now compare training an entire model with learning a small update to frozen weights.
Useful first: Attention Is All You Need
2024 · ~3 min · 0/7 complete
LoRA changes what gets trained. Mixture of experts gives a different efficiency lever: which parameters run for each token.
Useful first: Attention Is All You Need, LoRA
2023 · ~3 min · 0/8 complete
Close the model-building path by asking whether the changes actually help, and where automated judges can mislead.
Useful first: InstructGPT & DPO
Work through reasoning traces, tools, verification, reinforcement learning, and context beyond one prompt.
0/5 guides completed · Suggested order, every paper is open.
Begin with: Chain-of-Thought Prompting2022 · ~4 min · 0/8 complete
Start with intermediate reasoning steps. ReAct adds actions to this structure, and verification later asks which steps deserve trust.
Useful first: Attention Is All You Need
2022 · ~3 min · 0/6 complete
Build on a reasoning trace by inserting actions and observations from the environment.
Useful first: Chain-of-Thought Prompting
2023 · ~3 min · 0/7 complete
Once a model produces a sequence of steps, compare judging the whole answer with giving feedback along the way.
Useful first: Chain-of-Thought Prompting, ReAct
2025 · ~3 min · 0/7 complete
Use the distinction between reasoning and reward to understand how reinforcement learning can shape reasoning behavior.
Useful first: Let’s Verify Step by Step, InstructGPT & DPO
Preprint · ~4 min · 0/7 complete
After improving reasoning inside a prompt, consider a different bottleneck: inputs that cannot all fit in one context window.
Useful first: ReAct
Separate broad capabilities, executable software tasks, and human preference.
0/3 guides completed · Suggested order, every paper is open.
Begin with: BIG-Bench2022 · ~4 min · 0/7 complete
Begin with breadth. A diverse collection of tasks makes it easier to see why one score cannot cover every capability.
Useful first: Attention Is All You Need
2023 · ~3 min · 0/6 complete
Move from broad question sets to a concrete engineering outcome: fixing a real issue so its tests pass.
2024 · ~4 min · 0/7 complete
Contrast test-based task success with blind human preference. They answer different questions about model quality.
Useful first: BIG-Bench, Judging LLM-as-a-Judge
Paper selection and the original reading collection curated by Henry Shi. The explanations and dependency notes here are by Agent YAP. Reading order follows concepts, rather than publication dates.