SimplerEnv

Why we included this project

Anyone testing generalist robot manipulation policies like RT-1, RT-1-X, or Octo has hit the same problem: real-world evaluation is slow and expensive. SimplerEnv (CoRL 2024) tackles that by moving the evaluation into simulation, wrapping SAPIEN and ManiSkill2/3 environments into ready-made setups for the Google Robot and WidowX+Bridge platforms. It scores policies two ways: visual matching overlays real images onto simulated scenes, and variant aggregation averages results across randomized backgrounds, lighting, and distractors. Together they give the sort of quantitative metrics that make checkpoint selection sensible and flag where a policy is likely to fail once it actually meets a robot. There is also a GPU-parallelized ManiSkill3 path that runs the Bridge environments 10-15x faster, and the repo documents how to add your own environments, robots, and policies, with inference scripts and a Colab notebook for reproducing published results.