torchdistill

Why we included this project

Most knowledge distillation work is spent wiring up teacher-student loops before you ever get to test an idea. torchdistill removes most of that: it is a PyTorch framework that packages more than two dozen distillation methods from major vision and NLP venues, and you assemble an experiment by editing a YAML config instead of writing Python. If you need to pull intermediate representations out of a teacher or student, you point at the module paths rather than reimplementing the forward pass. The same setup works without a teacher, so dropping the teacher entries from a config gives you a plain training harness with no code to write. That makes it a solid base for comparing distillation recipes or reproducing published results.