LLMs-from-scratch

Why we included this project

This repository pairs with Sebastian Raschka's book on building a large language model from scratch, and it reads like a guided course rather than a tool you drop into a project. Each chapter works through one concrete stage, from tokenizing text and coding attention mechanisms to implementing a GPT-style model, pretraining on unlabeled data, and finetuning for classification and instruction following, with runnable notebooks and consolidated scripts. The step-by-step progression is genuinely useful for developers who want to see inside a transformer before reaching for a framework, and the included exercise solutions make it practical for self-study or a classroom. It also shows how to load weights from larger pretrained models and apply parameter-efficient finetuning with LoRA, so you can move from toy-scale training to real checkpoints. If you teach, onboard someone into LLM internals, or just want to understand what powers ChatGPT, this is a solid reference to work through.