picoGPT

Why we included this project

Reading about GPT-2 often leaves the internal mechanics feeling like a black box. This project peels that away by compressing the entire forward pass into roughly 40 lines of plain NumPy, with a companion blog post that walks through each piece. It won't help you ship anything: it's slow, processes one sample at a time, and contains no training code. But that minimalism is exactly the point. The repo ships the tokenizer and weight loader, so you can download real GPT-2 checkpoints and watch how embeddings, attention heads, and layer norms produce logits. For anyone who wants to re-derive the architecture before moving to a full framework, it's a compact, readable reference.