index-tts-vllm

Why we included this project

IndexTTS users who hit latency problems in real-time voice cloning or dubbing get a faster path here: the project reimplements the GPT stage on vLLM and wraps the result in a service with an API and a web UI for IndexTTS 1.0, 1.5, and 2. On a single RTX 4090 the authors measured real-time factor dropping from about 0.3 to 0.1 and decode speed rising from roughly 90 to 280 tokens per second. There is even an OpenAI-compatible audio route, though they note the v1/1.5 endpoints were still being stabilized. The speedup is real but lopsided: only the GPT stage runs in parallel, while the mel and vocoder steps stay serial and cap how many concurrent requests you can throw at it. If you already run IndexTTS and want faster batch inference or a cleaner way to serve it, this is a practical drop-in, not a new model.