codex-candy-eval

Why we included this project

For anyone who tinkers with the OpenAI Codex CLI locally, this is a small script that answers one narrow question: how often does a given model and reasoning setting nail a deceptively simple candy arithmetic problem, and how many reasoning tokens does that cost. It runs the same prompt a configurable number of times, checks the output against the known answer 21, and reports the pass rate together with token usage. The script has no third-party dependencies, so you can fetch and run it directly over the network as long as the CLI itself is installed and logged in. That makes it a convenient way to compare model versions or reasoning-effort levels quickly, though it is a single-question sanity check rather than a general benchmark suite.