skillsbench

Why we included this project

Teams building agents that depend on skills, the modular folders of instructions, scripts, and resources that let a model carry out specialized work, need a way to tell whether those skills actually help and whether the agent is using them well. SkillsBench is a gym-style benchmark that answers both questions on the same workload: each task ships as a task.md with an environment, an oracle solution, and a verifier, so you can measure skill effectiveness and agent behavior side by side. Because tasks are native BenchFlow packages, they run in a Docker or Modal sandbox and can be checked and executed straight from the CLI, which makes the setup useful for teams comparing agent frameworks or models. The tasks-extra examples go further, including a credentialed GPU task for implementing and comparing manifold-constrained hyper-connections in nanoGPT, a good model for benchmarking heavier, resource-hungry workflows. For anyone who wants reproducible, verifiable agent-skill evaluation instead of ad-hoc manual testing, this is a solid reference.