TinyLLaVA_Factory

Why we included this project

Training your own vision-language model usually means wrestling with codebases built for large GPU clusters, but TinyLLaVA_Factory is designed for the other end of the scale. It is a modular PyTorch and HuggingFace codebase that lets you swap language backbones, vision encoders, and connectors without rewriting the whole pipeline. You can decide how much compute to invest: freeze the base model, fine-tune everything, or use LoRA/QLoRA for a lighter footprint. Because the project emphasizes reproducibility and keeps the modules simple, it also works well as a teaching base for researchers who want to understand how small multimodal models are assembled. If you want to build a compact image-understanding model without committing to enterprise hardware, this is a sensible place to start.