LLM-Adapters

Why we included this project

This is the code behind an EMNLP 2023 paper on parameter-efficient fine-tuning, and its value is practical: you can compare adapter methods without rebuilding a training loop for each one. LoRA, bottleneck and parallel adapters, prefix tuning, p-tuning, and prompt tuning all sit behind a single interface that extends HuggingFace's PEFT library, so moving a task from LLaMA to OPT, BLOOM, or GPT-J is a flag change rather than a code rewrite. The repo also bundles the math10k and commonsense170k datasets and the benchmark tables from the paper, which gives you ready-made evaluation material and a baseline to sanity-check your own runs. Treat it as reference code for experiments rather than a maintained production SDK; it has not been updated since early 2024.