mdistiller

Why we included this project

Shrinking a heavy image-classification model for deployment usually costs accuracy, and mdistiller goes after that trade-off head-on. The repository bundles the code behind Decoupled Knowledge Distillation (DKD), published at CVPR 2022, and DOT, a distillation-oriented trainer from ICCV 2023, both by the repo's authors. These methods keep a compact student network close to its larger teacher in accuracy, and the package ships reproducible setups on standard benchmarks such as ImageNet and CIFAR-100, plus COCO-style object detection through a Detectron2-based pipeline. ML engineers who need a lighter classifier can run these configs to compare DKD and DOT against plain KD and older techniques like FitNet, AT, and ReviewKD instead of reimplementing the papers by hand, and a clean distiller interface lets you plug in your own method. Plan to configure and script things yourself, since this is a research toolkit rather than a turnkey service.