Beyond LoRA: Can you beat the most popular fine-tuning technique?
Source
huggingface.co
Date
Terms in this piece · Glossary
LoRA — A cheap way to fine-tune a model by training a small add-on layer instead of changing all of the model's weights.
fine-tuning — Taking a trained model and training it a bit more on your own examples so it gets better at one specific job.
Why it matters
LoRAA cheap way to fine-tune a model by training a small add-on layer instead of changing all of the model's weights.Full definition → is the default by habit, not by evidence for your specific task; this lays out the alternatives in PEFT and the criteria for choosing one before you spend the compute.
