DeepSpeed

Why we included this project

DeepSpeed is the library most large-model teams reach for when a single GPU stops being enough. Its ZeRO optimizer splits model states across devices, so models that would never fit on one card can train anyway, and when memory still runs short it can offload to CPU or NVMe. The same codebase covers serving: checkpoints trained with DeepSpeed, Megatron, or HuggingFace run inference without conversion, with partitioning, kernel injection, and quantization handled automatically. It also has optimizations for mixture-of-experts models and sparse attention, which is where training past the billion-parameter mark tends to get interesting. Because it plugs into PyTorch as an engine rather than a separate framework, existing training code mostly keeps working while the memory and throughput gains arrive.