nanoGPT

Why we included this project

nanoGPT shows what a GPT training pipeline looks like when you strip away the framework layers. The model definition and training loop each fit in roughly 300 lines of plain PyTorch, and the same code that reproduces GPT-2 (124M) on OpenWebText can be hacked into something custom or used to fine-tune a pretrained checkpoint. The character-level Shakespeare example gets a working model training on a single GPU in about three minutes, which makes it a good place to start if you want to learn the mechanics or run quick experiments. One honest caveat: the author now points newcomers to its successor, nanochat, and treats this repo as deprecated, so treat it as a stable, well-documented reference rather than an actively maintained product.