peft

Why we included this project

Fine-tuning a large language model on a single GPU burns through memory and compute fast. PEFT, the Hugging Face library for parameter-efficient fine-tuning, sidesteps that by training only a small set of extra parameters, like LoRA, IA3, and soft prompts, while the base model stays frozen. It plugs into Transformers for training and inference, manages adapters on image models through Diffusers, and pairs with Accelerate when you need to scale across devices. For teams adapting a foundation model to a specific domain without retraining everything, it is a direct route to results that come close to full fine-tuning quality at a fraction of the cost. It is also the reference implementation for many adapter techniques, which makes it a good place to start if you want to see how these methods actually behave.