co-tracker

Why we included this project

Anyone building video tools that need to follow a specific pixel across frames, whether it's picked by hand or sampled on a grid, will get real use out of CoTracker. The model tracks many points together rather than one at a time, which makes dense motion capture practical and gives it some of the behavior you'd expect from optical flow. It's straightforward to load through PyTorch Hub, and the repo includes both offline and online inference for short clips or longer streams, so it fits video editing, AR, 3D reconstruction, and measuring moving objects in science. The CoTracker3 release also ships the pseudo-labelling pipeline and a rendered dataset if you want to retrain on your own footage. Just remember the code is under a non-commercial Creative Commons license, which may matter depending on how you plan to use it.