Tutel

Why we included this project

Tutel is a PyTorch library that handles the low-level machinery of Mixture-of-Experts models, the kind of plumbing most frameworks expect you to build yourself. Its optimized MoE layers switch parallelism, sparsity, and capacity settings on the fly without the usual penalty, so runtime changes in expert routing don't slow you down. It also ships direct FP8, NVFP4, and MXFP4 inference paths for popular MoE checkpoints like DeepSeek, Kimi, Qwen3, and GLM, and runs on A100/H100 or AMD MI300 hardware. This is a component library rather than a full serving stack, so expect to integrate it into your own training or inference pipeline. Teams already working with MoE architectures who want tighter control over expert allocation and low-precision execution will find it worth a look.