ultravox

Why we included this project

Ultravox skips the usual speech-to-text detour: instead of feeding a transcription into a chatbot, it maps audio straight into the model's own representation space. That direct path is what keeps latency low enough for real back-and-forth conversation, which matters for voice agents, phone bots, and similar real-time interfaces. The repo ships trained weights for a Llama-based model and an 8B variant, plus the training code, so you can run it as-is or retrain the projector against a different open-weight backbone. If you want to host your own speech model instead of renting one, or just want to see how a speech-capable LLM is actually built, this is a good place to start.