rlhf-book

Why we included this project

Anyone fine-tuning open models for a product, or getting their bearings in alignment research, will find this a useful companion. The repository is an open-source textbook that walks through how reinforcement learning from human feedback actually works, from instruction tuning and reward modeling to direct preference optimization, rejection sampling, and distillation. The book text is paired with Python reference implementations, so a reader can run PPO, GRPO, DPO, or another method against a base model instead of only reading about it. The author, Nathan Lambert, also curated the references deliberately at a time when much online material about alignment is low quality, and he documents industry practices like making models more personable that previously had no open research. Teams training or aligning their own models, and researchers who want a structured entry point into the RLHF literature, will both get real value from the code and the explanations that go with it.