uccl

Why we included this project

Large-scale LLM training and inference usually bottleneck on GPU-to-GPU data movement, and teams wanting better throughput often have few options beyond tuning the default NCCL stack. UCCL is an open reimplementation of collective communication primitives that keeps the familiar NCCL/RCCL APIs, so existing applications run unmodified while getting faster allreduce and broadcast on many workloads. It also covers point-to-point transfers such as moving KV caches between nodes and shipping weights during reinforcement learning, plus GPU-driven expert-parallel routing for MoE models. The software transport design spreads traffic across multiple network paths and supports heterogeneous hardware from NVIDIA, AMD, and Broadcom, which matches many multi-vendor clusters in the wild. If you run distributed training or inference across several nodes and want to see whether a drop-in library can beat NCCL on latency or bandwidth, this is worth benchmarking.