magvit2-pytorch

Why we included this project

MagViT2's tokenizer is the piece that lets video models work in token space instead of raw pixels, and this repo is a readable PyTorch port of it from the 'Language Model Beats Diffusion' paper. The tokenizer compresses video into discrete codes via finite scalar quantization, so a transformer can learn to generate or understand frames as a sequence of tokens rather than modeling pixels directly. It comes with a training script that wires up the adversarial and multiscale discriminator losses, EMA tracking, and Weights & Biases logging, which removes most of the boilerplate when you're experimenting. Expect research-grade code rather than a polished library: you'll tune hyperparameters yourself, but it's a solid reference if you're replicating or adapting this approach. Tencent has also built a working model on top of this codebase, which is a good sign it's usable.