Skip to slide
31 / 74

CH 04 · LoRA, Fine-Tuning Giant Models on a Budget · 07/07

Read as one page

The one-sentence takeaway

LoRA fine-tunes a giant model cheaply by freezing the original and training only a tiny pair of low-rank matrices that capture the small adjustment a new task requires, which slashes memory and storage, lets one base model wear many swappable adapters, and adds no slowdown when the model runs.

Next: Chapter 5, Mixtral and Mixture of Experts, where models get a bigger brain without getting slower.

Finished this idea?

Your learning progress stays in this browser.