AI-Employe

Why we included this project

Most browser automation tools make you script every selector and wait condition by hand. This project takes a different route: you describe the task in plain language, and GPT-4 Vision works out the clicks and keystrokes through a browser extension backed by a Rust server. The interesting part is how it avoids the usual vision-agent failure mode. Instead of guessing coordinates, it indexes the page's DOM in MeiliSearch and lets the model target elements by their visible text, which cuts down on hallucinated actions. It also watches the DOM changes you make while demonstrating a workflow and feeds those actions back into the prompt to keep the model on task. That makes it a useful reference for teams building their own browser agents or experimenting with RPA-style automation, though you will need to stand up Firebase, Postgres, and MeiliSearch yourself before it runs.