Groma

Why we included this project

Groma is a research-oriented multimodal large language model that grounds its answers in specific parts of an image through a localized visual tokenizer. It accepts user-drawn region boxes as input and can produce long-form responses that point back to the visual context, and it reports state-of-the-art results on referring expression comprehension benchmarks. The repo includes the full training pipeline, pretrained checkpoints, and a grounded conversation dataset built with GPT-4V. Since the code builds on LLaVA and GPT4ROI, it is approachable if you already know those stacks, and the inference script supports fp16, 8-bit, and 4-bit quantization for running on modest hardware. This is a research artifact rather than a turnkey product, so plan to adapt the training and evaluation code to your own data.