awesome-knowledge-distillation

Why we included this project

Knowledge distillation is one of those topics where the important ideas are spread across decades of papers, and this list does the collecting for you. It gathers the foundational work, from early ensemble methods and Hinton's dark-knowledge paper through attention transfer, data-free distillation, and sequence-level approaches, each entry linking straight to the original. Read in order, it shows how the field actually evolved, which beats digging through search results when you are trying to shrink a trained model down to something that runs on a phone or edge device. It is a reading list rather than a tool you install, so use it to find the specific paper and method that.