Efficient-LLMs-Survey

Why we included this project

This is the companion repository to a peer-reviewed survey on efficient large language models, published in TMLR 2024, and it works as a structured entry point into that research area. Rather than a flat reading list, it groups hundreds of papers into a taxonomy of model-centric, data-centric, and framework-centric techniques, with subsections covering quantization, pruning, distillation, parameter-efficient fine-tuning, speculative decoding, KV-cache optimization, mixture-of-experts architectures, and long-context methods. For a practitioner trying to pick an efficiency approach, that organization helps you see the design space before committing to a technique. The maintainers take pull requests for new papers and for preprints that landed in a venue, so the list keeps up with the field. If you are choosing between quantization and distillation or just getting your bearings on how LLM efficiency work is categorized, this is a good place to start.