UI-TARS

Why we included this project

UI-TARS from ByteDance is a family of open vision-language models that work with software the way a person would: the model looks at the screen, works out what to do, and carries out the next click, drag, or key press. That makes it a good fit for automating web pages, desktop apps, Android emulators, and even games without leaning on fragile selectors or DOM hooks. The repository includes deployment instructions, inference scripts, and prompt templates for computer, mobile, and grounding tasks, plus a Python package that turns model output into runnable pyautogui code. A companion desktop app runs the agent on a local device, and the Midscene project covers browser automation separately. Teams testing multimodal agents for real GUI work will find concrete examples and benchmark numbers, with open weights to start from.