Awesome-Knowledge-Distillation-of-LLMs

Why we included this project

Anyone trying to get a smaller model to learn from a larger one will find this reading list hard to beat. It is the companion to the widely cited survey 'A Survey on Knowledge Distillation of Large Language Models,' and the authors sort the literature into knowledge elicitation and distillation algorithms, with a separate section on skill and vertical distillation, so you can jump straight to the technique you care about instead of digging through raw search results. The list is refreshed weekly, a real plus in a field where new distillation methods keep appearing. The survey paper itself supplies the conceptual framing, which makes the collection useful both as a map of the current state of the art and as a starting point for people training or compressing their own LLMs.