XMem

Why we included this project

XMem approaches video object segmentation as a memory problem, splitting feature storage between a short-term working memory and a long-term memory modeled on the Atkinson-Shiffrin theory. That design lets it track an object through very long clips without the GPU memory blowup you get from methods that keep everything in one bank. The authors report around 20 FPS on long videos, and the repo includes a GUI where you click an object in the first frame and let the model propagate the mask forward. That makes it practical for masked layer insertion, rotoscoping, and other post-production tasks with thousands of frames. It also forms the basis for newer work like Cutie and DEVA, so reading the codebase is a good way to understand modern video segmentation.