tiny-universe

Why we included this project

Tiny-Universe is a hands-on guide to building large language model systems from the ground up, written for developers who already use LLM tooling but want to understand what happens inside it. The project walks through implementing a minimal Transformer, pretraining a small Llama-style model on about 2GB of VRAM, building RAG and GraphRAG pipelines from scratch, assembling a minimal agent, adding an evaluation setup, and even training a diffusion model for image generation. Everything is plain PyTorch with heavily commented code, so you can follow each step without reaching for a high-level framework. The modules are self-contained, which makes it easy to jump straight to the part you care about, whether that is attention mechanics, evaluating model outputs, or tracing a query through a GraphRAG pipeline. If you have basic deep learning knowledge and have only ever called these systems through an API, this is a solid way to see the machinery underneath.