Visual-RFT

Why we included this project

Visual-RFT is the reference implementation for an approach that takes DeepSeek-R1's reinforcement learning recipe and applies it to vision-language models. Instead of the usual supervised fine-tuning, it runs Qwen2-VL through a GRPO loop where responses are scored by rule-based, task-specific rewards, so the model improves on visual perception tasks through exploration and explicit feedback. That makes it attractive for researchers and engineers who have hit the limits of plain instruction tuning and want better results on open-vocabulary detection, few-shot detection, reasoning grounding, or fine-grained classification. The repo is practical to work with: it ships training code, evaluation scripts, and datasets on Hugging Face, plus tooling to build datasets from your own JSON or HF data, and the authors report that a few hundred samples and around 200 steps are often enough. It also works as a base for reproducing the ICCV 2025 results or adapting the reward design to your own visual tasks.