Awesome-Knowledge-Distillation

Why we included this project

Researchers and engineers who work on model compression will find this a genuinely useful map of the knowledge distillation literature. The list organizes roughly 650 papers into thematic buckets: knowledge from logits, intermediate layers, graph-based and self-distillation, plus crossovers with GANs, meta-learning, RL, and AutoML, so it is easy to jump straight to the technique you care about. Each entry carries the venue and arXiv identifier, and many link to official code, which saves real time when you want to reproduce a method. It also covers applied areas like NLP, RecSys, and pruning/quantization, giving a practical view of where distillation is used beyond image classification. For anyone starting a distillation project or writing a related survey, this is a solid starting bibliography rather than a random collection of links.