vllm-omni

Why we included this project

Text-only inference servers don't fit models that also produce speech, images, video, audio, or robot action sequences. vLLM-Omni extends the vLLM engine to these omni-modality models, handling non-autoregressive architectures like diffusion transformers and heterogeneous outputs through pipelined stage execution and disaggregated resource allocation. That keeps throughput up when output types vary. Teams serving open models such as Qwen3-Omni, TTS systems, or image and video diffusion generators can expose an OpenAI-compatible API with streaming and parallel execution, making it a practical drop-in serving layer. Stable releases track upstream vLLM minor versions, so upgrades follow a predictable cadence rather than a moving target.