AppAgent

Why we included this project

AppAgent turns a multimodal model like GPT-4V into a virtual smartphone user, driving Android apps through their normal on-screen interface instead of requiring back-end access. That makes it a practical option for automating apps that expose no public API: the agent taps, swipes, and types its way through real tasks, learning first during an exploration phase that runs autonomously or from a human demonstration, then reusing that know-how to handle new jobs in deployment. It works through adb against a physical device or emulator and accepts any multimodal model you can plug into its model class, so teams prototyping UI test automation, assistant demos, or app workflow bots can pick it up without much ceremony. The repo also ships a public benchmark and worked examples, which makes it a handy reference for how agent learning and deployment fit together.