diy-llm

Why we included this project

Most courses teach you how to call an LLM; this one walks you through building the whole thing yourself. It follows the complete pretraining pipeline, from tokenizer construction and Transformer internals to mixture-of-experts, GPU kernels, distributed training, and alignment methods like SFT and RLHF, with six progressive, code-driven assignments modeled on Stanford's CS336. Theory is paired with working code throughout, so a developer comfortable with Python and PyTorch can keep up without prior LLM experience. It was written for Chinese-language learners and adapts the material to domestic models like Qwen and DeepSeek rather than just translating the original course. Don't expect a deployable runtime; this is a structured learning path, and its value is in building a working mental model of the whole LLM stack.