#97 · Primary category: Speech & Audio
StyleTTS2
StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Models
Project last updated:08/10/24
GitHub Stars
6.3K
Forks
700
Contributors
12
License
MIT
Why we included this project
StyleTTS2 produces speech that native English listeners rate as natural as human recordings, and it does so without a reference recording for each new voice. The model treats speaking style as a latent variable that a diffusion process shapes from the text alone, which gives the output real prosody instead of a flat, neutral reading. Its zero-shot speaker adaptation is the part most people will care about: with the pre-trained multispeaker models you can clone a voice from a short sample, handy for audiobook or dubbing experiments. The repo includes inference scripts, training code for single- and multi-speaker setups, and a finetuning script for new speakers, so you can go from downloaded checkpoints to audio fairly quickly. It is a research codebase, so expect checkpoints and Python scripts rather than a packaged API.
Articles for this project
No articles for this project yet.
To suggest a topic or contribute an article, contact us.
Related projects in this category
whisper.cpp
Port of OpenAI's Whisper model in C/C++
Real-Time-Voice-Cloning
Clone a voice in 5 seconds to generate arbitrary speech in real-time
VibeVoice
Open-Source Frontier Voice AI
voicebox
The open-source AI voice studio. Clone, dictate, create.
TTS
🐸💬 - a deep learning toolkit for Text-to-Speech, battle-tested in research and production