vector-quantize-pytorch

Why we included this project

Vector quantization turns continuous data into discrete codes, and this library packages the approach as a clean set of PyTorch modules. It is the same technique behind DeepMind's VQ-VAE-2 for images and OpenAI's Jukebox for music, available here as VectorQuantize, ResidualVQ, and GroupedResidualVQ classes you can drop straight into a model. Codebooks update with exponential moving averages by default, or you can switch to the DiVeQ method, which learns them by gradients and needs no auxiliary loss. There are also extras like k-means initialization and stochastic code sampling. Researchers and engineers building VAEs or token-based models for images, audio, or video will save themselves from reimplementing a lot of recent literature, with the caveat that this is a focused building block rather than an end-to-end generation tool.