llm_training_handbook

Why we included this project

This is a handbook for people who actually train large language models, not a package you install. The value is in the copy-paste scripts and commands for the problems that come up in real training runs: model parallelism, getting more throughput, choosing tensor precision and data types, setting hyperparameters, handling instabilities, and debugging software and hardware failures. The SLURM section and the troubleshooting notes are the parts you will reach for when a run stalls, diverges, or crashes. The authors also point you to a companion playbook if you want the conceptual overview instead of the commands. Keep it open next to your training run.