minimind

Why we included this project

MiniMind is a rare thing: a full LLM training pipeline small enough that one person can actually run it. The project trains a roughly 64M-parameter model from scratch, with every stage implemented natively in PyTorch rather than through high-level framework wrappers. The code walks through tokenizer training, pretraining, supervised fine-tuning, LoRA, DPO, and reinforcement-learning stages like PPO and GRPO, plus tool use and agentic RL. Because the model is tiny, the SFT stage finishes in about two hours on a single NVIDIA 3090, which makes it practical to reproduce the results and then tinker with the internals. That combination of completeness and small scale is what makes it useful both as a hands-on tutorial for people learning how LLMs work and as a compact reference for engineers who want to see minimal, readable implementations.