fairseq2

Why we included this project

fairseq2 is a ground-up reboot of FAIR's earlier fairseq, built for researchers who want to train and fine-tune large sequence models in PyTorch without fighting a monolithic framework. It comes with first-party recipes for instruction fine-tuning and preference optimization, and its trainer scales to 70B-plus parameter models across multi-GPU and multi-node setups using DDP, FSDP, and tensor parallelism. The design is deliberately extensible: you register new models, optimizers, schedulers, and training components through a setuptools-based plugin mechanism instead of forking the library, so your project code stays your own. A C++ streaming data pipeline keeps throughput high, and native vLLM integration covers generation. It is not a turnkey app; rather, it is the substrate several published FAIR projects, from multilingual speech recognition to sentence-level language modeling, actually run on.