#97 · Primary category: Speech & Audio

StyleTTS2

adversarial-training deep-learning diffusion-models gan latent-diffusion latent-diffusion-models pytorch speaker-adaptation speech-synthesis text-to-speech tts wavlm

StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Models

Project last updated:08/10/24

GitHub Stars

6.3K

Forks

700

Contributors

12

License

MIT

Why we included this project

StyleTTS2 produces speech that native English listeners rate as natural as human recordings, and it does so without a reference recording for each new voice. The model treats speaking style as a latent variable that a diffusion process shapes from the text alone, which gives the output real prosody instead of a flat, neutral reading. Its zero-shot speaker adaptation is the part most people will care about: with the pre-trained multispeaker models you can clone a voice from a short sample, handy for audiobook or dubbing experiments. The repo includes inference scripts, training code for single- and multi-speaker setups, and a finetuning script for new speakers, so you can go from downloaded checkpoints to audio fairly quickly. It is a research codebase, so expect checkpoints and Python scripts rather than a packaged API.

Articles for this project

No articles for this project yet.

To suggest a topic or contribute an article, contact us.

Related projects in this category