large_language_model_training_playbook

Why we included this project

Long training runs usually break down on operational choices, not on model code. This playbook gathers the kind of practical advice that normally gets scattered across forum threads: picking a parallelization strategy, reading scaling laws to size your model sensibly, and deciding between fp32, fp16, and bf16 for different layers and optimizers. Newer teams get a structured tour through learning-rate schedules, batch sizes, and throughput bottlenecks; veterans can jump straight to detecting instabilities and debugging hardware or software failures. It's a curated document rather than software, so treat it as a starting reading list, with the companion LLM Training Handbook supplying the concrete scripts. Reading it before you commit to a framework or cluster setup helps clarify which knobs actually matter, which makes later tooling choices easier.