ema-pytorch

Why we included this project

Most PyTorch users have hand-rolled an exponential moving average of their weights at some point, usually as a small helper that never quite feels finished. This library wraps any nn.Module and keeps a shadow copy of the model with a configurable decay, a warmup step, and an update frequency, so you can hold a stable evaluation or teacher model alongside the one being trained. It also implements the post-hoc synthesized EMA from Karras et al., a Switch EMA variant, and a module wrapper that routes EMA outputs into nested submodules for self-supervised setups. That covers diffusion training and any project where a smoothed weight average improves final results, without you maintaining the update loop yourself. If you would rather use a maintained implementation than debug your own, this saves real time.