xmtf

Why we included this project

This repository is the companion to the Crosslingual Generalization through Multitask Finetuning paper and the most complete public record of how the BLOOMZ and mT0 models were built. Anyone wanting to reproduce or adapt multilingual instruction tuning will find the full pipeline here: the xP3 and xP3mt task mixtures covering dozens of languages, the scripts that assemble them from PromptSource, the Megatron-DeepSpeed training recipes with SLURM launchers, and the rank and generation evaluation harnesses used to score the released checkpoints. It is not a drop-in library but a step-by-step account of a large-scale training effort, so it is a good reference for planning your own crosslingual finetuning run or for studying how multitask data composition affects zero-shot generalization. The model cards and dataset links also make it a handy starting point for comparing the many released BLOOMZ and mT0 variants.