TinyZero

Why we included this project

TinyZero strips a reasoning-model training run down to its essentials. It reproduces the DeepSeek R1-Zero approach on countdown and multiplication tasks, built on top of veRL, and its main finding is that a 3B base model develops self-verification and search behavior through reinforcement learning alone, with no supervised fine-tuning. The training scripts and data-prep steps are short enough to follow by hand, which makes it a good place to see how RL elicits reasoning before you scale up. The authors report you can reproduce the 'aha moment' for under $30 on modest hardware. One caveat: the repo is deprecated and points to the upstream veRL library, so treat it as a learning resource and reference recipe rather than a maintained tool to build on.