QLoRA lowers the memory needed to adapt a pretrained LLMA large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.Full definition → by keeping its main weights compressed and unchanged. Training updates smaller additions called low-rank adapters, while gradients pass through the frozen model stored at four-bit precision. The 2023 paper reports fine-tuningTaking a trained model and training it a bit more on your own examples so it gets better at one specific job.Full definition → a 65-billion-parameter model on one GPU with 48 GB of memory. It also introduces methods to reduce storage overhead and manage memory spikes. The result expands the size of pretrained model that can be adapted on limited hardware; the reported operation is fine-tuning an existing model.
Terms in this piece · Glossary
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
fine-tuning — Taking a trained model and training it a bit more on your own examples so it gets better at one specific job.
quantization — Shrinking a model by storing its numbers less precisely — like rounding — so it runs faster and fits on smaller hardware, at a small quality cost.
LoRA — A cheap way to fine-tune a model by training a small add-on layer instead of changing all of the model's weights.