mergekit

Why we included this project

mergekit blends two or more pretrained language models into a single checkpoint by working directly in the weight space, so you get the strengths of each model without retraining or running an ensemble. You can mix a chat-finetuned model with a strong base, or fold a task-specific fine-tune into a generalist, and the combined model keeps both sets of skills. The toolkit runs merges out-of-core, meaning even elaborate combinations can be computed on plain CPU or a GPU with as little as 8 GB of VRAM. That makes it practical for individuals and small teams working with modest hardware, and it goes a long way toward explaining why hobbyist model-builders keep reaching for it.