x-clip

Why we included this project

x-clip is a compact PyTorch implementation of CLIP for teams that want to train their own image-text contrastive model rather than fine-tune a pretrained checkpoint. The whole architecture lives in a single class you configure with a handful of dimension arguments, so you can train on your own paired data and end up with a custom embedding space instead of OpenAI's fixed weights. What makes it useful for research is the set of recent tricks exposed as toggles: patch dropout from FLIP, token-level contrastive learning from FILIP, decoupled contrastive loss, and auxiliary self-supervised or masked-language objectives from DeCLIP. You can also swap in an external vision transformer or residual net as the image encoder, which lets you reuse an existing backbone and concentrate on the contrastive head. It is a training library, not a served model, so plan to bring your own data pipeline and evaluation.