InferenceX

Why we included this project

The project's core idea is to keep re-running a standard inference benchmark across the major open-source serving stacks, including SGLang, vLLM, and TensorRT-LLM, on current data-center GPUs like the GB200 and GB300 NVL72, B200, and AMD's MI355X. Because the runs are continuous, results capture how kernels and schedulers improve over time, and a single snapshot benchmark goes stale within weeks as upstream software ships updates. InferenceX provides an Apache-2.0 benchmark runner and a public dashboard where you can compare results across stacks and hardware variants without running the workload yourself. That gives procurement, capacity-planning, and framework-selection teams a continuously refreshed source of numbers instead of one-off vendor claims.