GPT4RoI

Why we included this project

GPT4RoI is a research codebase for region-level image understanding: it instruction-tunes a large language model so that instead of describing a whole image, it can take a specific region of interest and answer questions about just that area. That makes it directly relevant to visual grounding and fine-grained visual question answering. The repo includes the full training and inference pipeline plus a Gradio demo, so it's a workable starting point if you want to see how region-conditioned instruction tuning is actually built on top of a multimodal backbone. The released delta weights let you reconstruct the model by combining them with the original LLaMA weights rather than retraining from scratch. One caveat: the code is under a non-commercial license, so treat it as material for academic study, not something to ship in a product.