Segment-and-Track-Anything

Why we included this project

This project combines the Segment Anything Model with a transformer-based tracker called DeAOT to segment objects in one video frame and carry those masks through the rest of the footage. You can start a mask by clicking, brushing, or typing a text prompt, or let the pipeline detect objects automatically, which makes it practical for video-editing cutouts, annotation work, and tracking objects in camera feeds. A WebUI and a Colab notebook get you testing on your own clips quickly, while the Python API suits batch jobs. It also has an audio-grounding mode that follows the object making a sound in the soundtrack. The tracker has real traction outside the repo, since it underpins published robotics work on open-vocabulary drone tracking from MIT CSAIL and Harvard SEAS.