OpenDelta

Why we included this project

Training a large pretrained language model from scratch is expensive, and even full fine-tuning can exceed what a single modest GPU can handle. OpenDelta sidesteps that by freezing the backbone and updating only a small set of trainable parameters, which is the idea behind LoRA, prefix tuning, and adapters. You choose where those modules attach across supported transformers, and the default configurations usually work without editing the model's code, even when your custom architecture wraps a supported one as a submodule. That makes it handy for researchers comparing tuning strategies side by side and for teams trying to fit a big model on a tight GPU budget. If you already build on Hugging Face transformers, the short jump from full fine-tuning to delta tuning is the main reason to give it a try.