Text2Video-Zero

Why we included this project

For teams that want to generate video without training a dedicated video model, this project offers a practical shortcut: it takes an off-the-shelf Stable Diffusion model and turns it into a zero-shot video generator by guiding motion in latent space. The official code covers plain text-to-video, pose or edge conditioning, depth control, and an Instruct-Pix2Pix variant for instruction-guided video editing, so one codebase handles several workflows. Anyone already comfortable with Stable Diffusion will find the setup familiar, and since you can load any hosted base or DreamBooth checkpoint, existing models plug in easily. As the reference implementation of the ICCV 2023 paper, it is also a solid entry point for researchers who want to dig into the method itself rather than just consume an API.