#120 · Primary category： Foundation Models
UNITER
Research code for ECCV 2020 paper "UNITER: UNiversal Image-TExt Representation Learning"
Project last updated：06/30/21
GitHub Stars
799
Forks
111
Contributors
3
License
MIT
Why we included this project
UNITER is the official code behind an ECCV 2020 paper on learning a single representation from both images and text, and it remains a useful reference for anyone working on vision-language models. The repo ships pretrained checkpoints for the base and large variants, plus finetuning scripts for benchmarks like VQA, image-text retrieval on COCO and Flickr30k, and referring expression comprehension. Data download scripts, a Docker image, and mixed-precision training support are all included, so reproducing the paper's results on NVIDIA GPUs is a matter of running the provided commands rather than reconstructing the pipeline by hand. Teams that want to see how one multimodal encoder can be adapted to several tasks at once will find the codebase a clear starting point, even if they end up moving to newer architectures.
Articles for this project
No articles for this project yet.
To suggest a topic or contribute an article, contact us.
Related projects in this category
transformers
🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
CLIP
CLIP (Contrastive Language-Image Pretraining), Predict the most relevant text snippet given an image
MiniCPM-V
A Pocket-Sized MLLM for Ultra-Efficient Image and Video Understanding on Your Phone
generative-models
Generative Models by Stability AI
unilm
Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities