clickclickclick

Why we included this project

Instead of just generating text, this framework turns an LLM into an agent that can click through real interfaces. It separates the work into a planner that decides the steps and a finder that locates the UI elements, then executes on Android or macOS. You can run it with remote models like Gemini or GPT-4o, or keep everything local via Ollama. The demos show it drafting a Gmail, finding bus stops in Google Maps, and starting a chess game on Lichess. The code is explicitly experimental, so treat it as a starting point for your own computer-use agent rather than a finished product.