Why normal fine-tuning hurts

When you fine-tune a model the usual way, you adjust all of its parameters. For a model with billions of them, this causes three pains:

1. Memory. Training needs several times more memory than just running the model, because it has to track how to adjust every single parameter. This can require many expensive GPUs.
2. Storage. Every fine-tuned copy is a full-size model. If you want ten specialized versions, you store ten enormous files.
3. Cost and time. Updating billions of dials is slow and burns a lot of money.

For most people and companies, this is simply out of reach. LoRA changes that.