#256 · Primary category： Computer Vision
Video-LLaMA
[EMNLP 2023 Demo] Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding
Project last updated：06/04/24
GitHub Stars
3.1K
Forks
287
Contributors
7
License
BSD-3-Clause
Why we included this project
Most vision-language models stop at single images. Video-LLaMA extends that to video plus audio, so a language model can answer questions about both what appears on screen and what's said in the soundtrack. The repo is refreshingly concrete: it spells out the architecture, training data, and exact checkpoints, including full-weight 7B and 13B models that skip the delta-weight hassle. You can try the hosted demos before committing to local GPU time, and the two-stage pipeline built on BLIP-2 and MiniGPT-4 is well documented for researchers doing multimodal instruction tuning.
Articles for this project
No articles for this project yet.
To suggest a topic or contribute an article, contact us.
Related projects in this category
opencv
Open Source Computer Vision Library
RuView
π RuView turns commodity WiFi signals into real-time spatial intelligence, vital sign monitoring, and presence detection — all without a single pixel of video.
PaddleOCR
Turn any PDF or image document into structured data for your AI. A powerful, lightweight OCR toolkit that bridges the gap between images/PDFs and LLMs. Supports 100+ languages.
MinerU
Transforms complex documents like PDFs and Office docs into LLM-ready markdown/JSON for your Agentic workflows.
tesseract
Tesseract Open Source OCR Engine (main repository)