#80 · Primary category： Foundation Models
LLaVA-Mini
LLaVA-Mini is a unified large multimodal model (LMM) that can support the understanding of images, high-resolution images, and videos in an efficient manner.
Project last updated：06/29/25
GitHub Stars
577
Forks
34
Contributors
2
License
Apache-2.0
Why we included this project
Most multimodal models pay a heavy price in compute and memory for every image they look at, because each picture becomes hundreds of visual tokens. LLaVA-Mini sidesteps that by compressing an image into a single token, which the authors say cuts FLOPs by about 77% and brings per-image VRAM down from hundreds of megabytes to under a megabyte, while staying close to LLaVA-v1.5 in accuracy. That saving carries over to video, where the model can handle very long clips on a single 24GB GPU, making it a realistic choice for applications that would otherwise be too costly to run. The repo includes a Gradio demo, command-line scripts for images and video, and training and evaluation docs, so you can go from downloading the Hugging Face weights to a working interface fairly quickly. It is a research release built on the LLaVA codebase, so expect to adapt it rather than treat it as a turnkey product, but the efficiency gains are concrete and measurable.
Articles for this project
No articles for this project yet.
To suggest a topic or contribute an article, contact us.
Related projects in this category
transformers
🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
CLIP
CLIP (Contrastive Language-Image Pretraining), Predict the most relevant text snippet given an image
MiniCPM-V
A Pocket-Sized MLLM for Ultra-Efficient Image and Video Understanding on Your Phone
generative-models
Generative Models by Stability AI
unilm
Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities