alignment-handbook

Why we included this project

Turning a base model into a chatbot that actually follows instructions takes more than a single fine-tuning pass. This handbook walks you through the whole post-training pipeline with runnable recipes: continued pretraining for domain or language adaptation, supervised fine-tuning, reward modeling, rejection sampling, and preference methods like DPO, ORPO, KTO, and IPO. The training scripts scale from DeepSpeed full fine-tuning on multi-GPU nodes down to LoRA or QLoRA on a single consumer GPU, and each recipe is a self-contained YAML config that reproduces a known model, giving you a verified baseline to compare against. If you're responsible for turning an open model into an in-house assistant, the evaluation scripts and dataset-formatting guidance are just as valuable as the training code.