#247 · Primary category: Video & Animation

CLIP4Clip

activitynet clip didemo lsmdc msrvtt msvd multimodal multimodal-learning multimodality ranking retrieval retrieval-model search video-clip-retrieval video-text-retrieval

An official implementation for "CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrieval"

Project last updated:04/12/24

GitHub Stars

1.0K

Forks

137

Contributors

5

License

MIT

Why we included this project

CLIP4Clip is an official implementation of the paper that adapts a pretrained CLIP encoder for video-text retrieval. The repo compares three ways of turning CLIP into a video retriever: a simple mean-pooled baseline, an LSTM, and a transformer variant, so you can see how much the extra architecture helps on a given dataset. Training commands for MSR-VTT, MSVD, LSMDC, ActivityNet, and DiDeMo are included, which makes reproduction straightforward compared to most research code. The code is modular enough that you can swap the similarity head or the linear patch strategy with command-line flags without rewriting the pipeline. It is a research baseline rather than a production service, so expect to handle data preparation and checkpoints yourself, but for anyone starting on video caption retrieval it is a solid reference.

Articles for this project

No articles for this project yet.

To suggest a topic or contribute an article, contact us.

Related projects in this category