qlora

Why we included this project

QLoRA is the kind of project you reach for when you want to fine-tune a large open-weights model on your own data but don't have a cluster of GPUs to burn. The method trains a 65B-parameter model on a single 48GB GPU by keeping the backbone frozen and 4-bit quantized, pushing gradients into low-rank adapters instead of the full weights. That cuts memory dramatically while keeping most of the full-precision quality, and the code plugs into Hugging Face's transformers and PEFT with bitsandbytes, so it fits a familiar PyTorch workflow rather than introducing its own stack. The repo ships with runnable scripts for LLaMA 2, several instruction datasets, and the trained Guanaco adapter weights plus hyperparameter recipes you can copy to reproduce results. Be realistic though: this is research code tied to a paper, not a maintained product, so budget time to read the README and adapt the training loop to your data format and hardware.