LLamaTuner

Why we included this project

LLamaTuner is a fine-tuning toolkit aimed at teams that want to adapt an open-weight model to their own data without a big GPU budget. It supports LoRA, QLoRA, and full-parameter training across Llama, Qwen, Baichuan, ChatGLM, and Mistral, and can fine-tune a 7B model on a single 8GB GPU. Beyond instruction tuning it covers continuous pre-training and preference alignment methods like DPO and PPO, and it can train vision-language models such as LLaVA. DeepSpeed integration plus automatic dispatch of FlashAttention and Triton kernels helps when you scale up to multi-node runs of larger models. If you want a single codebase that takes you from data preparation through training to chatting with the result, it is worth a look.