phenaki-pytorch

Why we included this project

Phenaki pairs a C-ViViT video tokenizer with a MaskGIT transformer to turn natural language prompts into short video clips, and this repo is a working PyTorch implementation of that architecture. It is aimed at researchers and ML engineers who want to study or reproduce the approach rather than consume it as a service: the whole pipeline is exposed as composable, trainable modules, with trainer classes and a make_video helper that chains scene-by-scene generation with frame priming to reach longer outputs. The code supports training on images first and then fine-tuning on video for sample efficiency, rectangular resolutions, and mixing clips of different lengths in one batch. It also includes the token-critic refinement idea, so you can use a learned critic or the MaskGIT model itself as a self-critic to improve sampling quality. If you are digging into how long-form, text-conditioned video generation works, this is a useful reference for the mechanics.