
Parameter-Efficient Fine-Tuning: What to Know Before You Build
TL;DR Parameter efficient fine tuning (PEFT) adapts a pretrained model by training a small set of new or selected weights while the original model stays frozen. The most common method, LoRA, adds two small trainable matrices next to existing weight matrices instead of updating them directly. QLoRA extends this by loading the frozen base model in 4-bit precision, which is



