knowledge-distillation-pytorch

Why we included this project

This repo is a clean, self-contained way to learn knowledge distillation by actually running it. It trains a small student model on the logits of a larger teacher, with two worked examples on CIFAR-10: a five-layer CNN distilled from a ResNet-18, and a ResNet-18 distilled from a ResNeXt-29. Each experiment reads its settings from a single params.json file, so adjusting the temperature or distillation weight is a quick edit instead of a long command line. The repo also ships scripts for hyperparameter search and result synthesis, which turn the usual fiddly tuning into something repeatable, plus pretrained teacher checkpoints you can download. Anyone studying model compression or wanting to reproduce classic KD results can get concrete numbers without rebuilding the whole setup from scratch.