Platypus

Why we included this project

Platypus is the training code behind the fine-tuned LLaMA and LLaMA-2 models of the same name, and it's a good starting point if you want to reproduce a LoRA fine-tuning run instead of building the pipeline yourself. A single finetune.py script plus shell scripts handle launching with torchrun, and the hyperparameters the authors used for the 13B and 30B models are right there to adapt. The repo also includes data refinement code that filters STEM and logic questions from open datasets and deduplicates them with cosine similarity, handy if you want a clean training set. A merge script folds the LoRA weights back into the base model and can combine several adapters, which the authors found helped accuracy when the source datasets barely overlapped. If you're planning to fine-tune an instruction-tuned model on a modest GPU budget, the concrete recipes and configuration notes transfer directly.