make-a-video-pytorch

Why we included this project

The key to Meta's Make-A-Video is taking a strong text-to-image diffusion backbone and giving it a sense of time. This PyTorch implementation shows exactly how that happens: pseudo-3D axial convolutions blend information across frames cheaply, and spatio-temporal attention handles long-range motion. The temporal layers are built so they pass 2D image tensors through unchanged and initialize as identity, which means you can train an existing 2D U-Net on images first and then switch to video without redesigning anything. It also includes toggles for training on time and an optional flash-attention path, so porting the pieces into your own diffusion pipeline is straightforward. Just know it is an educational translation of the paper, not a ready-made text-to-video service; you'll wire up the surrounding training and sampling loop yourself.