axolotl

Why we included this project

Getting a modern LLM to fine-tune cleanly by hand usually means wiring together tokenizer handling, attention masking, optimizer setup, and accelerator-specific flags across half a dozen libraries. Axolotl puts that whole pipeline behind a declarative YAML config, so you can load a base checkpoint from Llama, Mistral, Qwen, or Gemma and run LoRA, QLoRA, or full fine-tuning without writing a training loop yourself. Beyond single-GPU runs, it covers the awkward parts that appear as models get bigger: multi-node parallelism, sequence parallelism for long contexts, MoE expert quantization, and reinforcement-style recipes like DPO and GRPO. The CLI and Colab notebooks keep quick experiments easy, and the config-first setup means you can record exactly which hyperparameters produced a given checkpoint. That combination suits teams fine-tuning open-weight models in production and researchers who iterate on training recipes.