TheWhisper

Why we included this project

TheWhisper repackages OpenAI's Whisper models into inference engines tuned for streaming and on-device transcription, and the practical differences show up fast. Where stock Whisper works in fixed 30-second chunks, this project lets you adjust chunk size, so live captioning, meeting notes, or voice interfaces can emit text as audio arrives and you can trade latency against accuracy yourself. Dedicated backends target NVIDIA GPUs and Apple Silicon via CoreML, with the README reporting roughly 220 tokens per second for whisper-large-v3 on an L40. A local REST API and an Electron demo app round it out, letting you move from model to working prototype without running a separate server stack. If you are weighing self-hosted or fully offline transcription on hardware you already own, this is a concrete look at the latency and power draw you can expect.