Skip to content

Chapter 00 · 2 min read · Updated

Start Here: Foundational Modelling, Explained Simply

Six papers map the journey from a model's architecture through scale, alignment, efficiency, and evaluation, in the order that builds understanding.

Prefer one idea per screen? Read it as 5 slides →

Welcome. This folder of papers is the bedrock of modern AI. Every chatbot, coding assistant, and AI tool you have used stands on the ideas in these eight papers.

The papers themselves are written for researchers, so they can feel intimidating if you are new. These explanation chapters fix that. They walk you through the same ideas in plain language, with pictures and small examples, so that by the end you understand why each paper mattered, not just what it said.

00.1

How to use these explanations

Read the chapters in order. Each one builds on the last. When you hit a word in bold with a link, like self-attention, you can click it to jump to the glossary, where that idea is explained from scratch. The same words show up again and again across papers, so explaining them once in a shared glossary keeps things clean.

You do not need any math beyond basic arithmetic to follow these chapters. You do need patience and curiosity.

00.2

The big picture

Think of building a modern language model as a four-step journey. Each paper in this folder improves one step.

flowchart TD
    A[Step 1: Architecture<br/>What shape is the model?] --> B[Step 2: Scale<br/>How big, how much data?]
    B --> C[Step 3: Alignment<br/>How do we make it helpful?]
    C --> D[Step 4: Efficiency<br/>How do we train and run it cheaply?]
    D --> E[Step 5: Evaluation<br/>How do we know it is good?]

Here is how the eight papers map onto that journey.

StepQuestion it answersPapers
ArchitectureWhat internal design lets a model understand language?Attention Is All You Need
ScaleHow big should the model and dataset be?Scaling Laws, Chinchilla
AlignmentHow do we turn a raw text predictor into a helpful assistant?InstructGPT (RLHF), DPO
EfficiencyHow do we customize and serve models without huge cost?LoRA, Mixtral (Mixture of Experts)
EvaluationHow do we measure whether a model is actually good?LLM-as-a-Judge / Chatbot Arena

00.4

One idea to hold in your head

Almost everything in this folder is a variation on a single trick: predict the next word. A language model is trained by hiding the next word in billions of sentences and asking the model to guess it. That is it. Everything else, the architecture, the scaling, the alignment, is about making that one trick work better and behave the way we want. Keep that anchor in mind and the rest will click into place.

Ready? Start with Chapter 1: The Transformer.