LISA

Why we included this project

LISA couples an LLM's ability to reason about an image with the pixel-level job of drawing a segmentation mask. Given a question that needs world knowledge or a step-by-step explanation, it answers in plain text and marks the exact object it means, which matters when you want the model to show which part of the image it is talking about rather than just reply. That suits grounded visual question answering, assistive and inspection tools, and similar work where the region of interest carries as much weight as the words. A hosted Gradio demo and local deployment notes make it easy to try the behavior. The repo itself is also a solid reference implementation, with training recipes and the LISA++ variant that adds visual chain-of-thought data.