CoDeF

Why we included this project

Most video editing tools struggle to keep edits consistent from frame to frame. CoDeF sidesteps that by learning a single canonical content field and a temporal deformation field that maps it onto each frame, so an edit applied once carries through the whole clip. That lets you run an image-to-image model like ControlNet on the canonical frame and get a video-to-video translation without flicker or drift, and it also supports keypoint tracking and non-rigid objects like water and smog. The official PyTorch implementation, pretrained checkpoints, and Colab demo make it straightforward to reproduce the CVPR 2024 results and adapt the method to your own footage. Just be ready for a research-grade setup: there's a real training pipeline and data preprocessing, not a plug-and-play tool.