Video-LLaVA

Why we included this project

Teams building systems that need to answer questions about video content will find a lot to work with here. Video-LLaVA is a vision-language model that accepts both still images and video frames and produces natural-language responses, so it handles tasks like describing a clip, answering queries about it, or reasoning over visual details. The notable design choice is aligning visual and language features early, before projecting them into the language model, which lets a single encoder serve both images and video rather than requiring separate pipelines. The repo includes pretrained and instruction-tuned checkpoints, plus code for training, evaluation, and inference, so you can run it directly or use it as a baseline for your own experiments. If you're prototyping video understanding or captioning and want a practical open-weight starting point, this is a reasonable candidate to load and test on your own footage.