#67 · Primary category: Speech & Audio

whisper-timestamped

asr attention-is-all-you-need attention-mechanism attention-model attention-network attention-seq2seq attention-visualization deep-learning machine-learning multilingual-models python python3 pytorch speaker-diarization speech speech-processing speech-recognition speech-to-text transformers whisper

Multilingual Automatic Speech Recognition with word-level timestamps and confidence

Project last updated:08/17/26

GitHub Stars

2.8K

Forks

211

Contributors

11

License

AGPL-3.0

Why we included this project

Anyone shipping subtitles, meeting notes, or searchable audio archives quickly hits the limit of stock Whisper: it predicts timestamps per segment, usually accurate to about a second, which is too coarse when each spoken word needs a precise position. This project computes word-level start and end times by aligning Dynamic Time Warping against Whisper's cross-attention weights, and it attaches a confidence score to every word and segment so downstream filtering has something concrete to work with. The alignment runs as each segment is decoded, with no separate pass, and it drops into the openai-whisper package from either Python or the command line. There is also voice activity detection to suppress the silence-related hallucinations Whisper can produce, plus a language confidence score when no language is given, both of which help when you are running large batches of audio unattended.

Articles for this project

No articles for this project yet.

To suggest a topic or contribute an article, contact us.

Related projects in this category