#193 · Primary category: Speech & Audio

WhisperS2T

asr deep-learning speech-recognition speech-to-text tensorrt tensorrt-llm vad voice-activity-detection whisper

An Optimized Speech-to-Text Pipeline for the Whisper Model Supporting Multiple Inference Engine

Project last updated:08/27/24

GitHub Stars

577

Forks

75

Contributors

3

License

MIT

Why we included this project

WhisperS2T is a practical wrapper around OpenAI's Whisper that lets you swap between several inference engines, including the original implementation, HuggingFace with FlashAttention 2, CTranslate2, and TensorRT-LLM, so you can match the backend to your hardware and latency needs. It does more than just call the model: it batches short speech segments across files, loads large audio asynchronously while transcribing, and applies heuristics to reduce the repeated-text hallucinations Whisper often produces. Word-level alignment and voice activity detection are built in, and transcripts export to SRT, VTT, and other common formats, which makes it handy for captioning and meeting notes. If you already use WhisperX or faster-whisper, the API will feel familiar, and the reported speedups over those tools give a concrete reason to benchmark it on your own audio before switching.

Articles for this project

No articles for this project yet.

To suggest a topic or contribute an article, contact us.

Related projects in this category