build-nanogpt

Why we included this project

For anyone who has read about transformers and still felt the gap between theory and a model that actually trains, this repo is the bridge. It pairs a video lecture with clean, step-by-step git commits that take you from an empty file to a working GPT-2 (124M) reproduction, so you can watch the pieces click together as you go. The training code is practical enough that a fresh pretraining run costs about an hour and under ten dollars on a rented GPU box, which makes it a friendly first real training project. It deliberately sticks to pretraining on internet documents and leaves chat fine-tuning aside, so you come away with a clear picture of the core training loop before layering on conversation. Individuals self-studying deep learning and teams that want an inspectable baseline they can actually run and modify will both find a solid starting point.