midscene

Why we included this project

Keeping end-to-end tests alive usually means chasing CSS selectors every time the UI shifts. Midscene sidesteps that by letting you write the goal in plain language and having a vision model watch the screen, then click, type, and verify the result itself on web, iOS, and Android. Because the agent reacts to what it sees rather than a selector string, tests hold up much better as interfaces change. Adoption is flexible: a Chrome extension for trying things out without code, SDK bindings that fit into an existing Playwright or Puppeteer workflow, and a CLI for scripted runs. It also works with local models like UI-TARS, which is handy for teams that need to keep automation off third-party clouds. QA engineers and small teams get a way to cut selector maintenance while keeping real browser and app coverage.