LLM-Shearing

Why we included this project

Teams that need a smaller language model without paying for a full from-scratch training run will find a lot to work with here. The codebase implements structured pruning, which removes whole attention heads and feed-forward dimensions from an existing model such as Llama-2-7B, then continues pre-training the pruned result so it keeps most of the original's quality. The authors' central claim is that pruning a strong base model costs a fraction of the compute of training a comparable small model from scratch. The repo provides the pipeline as Composer callbacks, handles dynamic batch loading and weight conversion between Hugging Face and Composer formats, and includes the released 1.3B and 2.7B Sheared-LLaMA checkpoints plus evaluation scripts. It is research code rather than a plug-and-play tool, so expect to adapt configs, but the recipe is detailed enough to follow end to end.