naturalspeech2-pytorch

Why we included this project

Natural Speech 2 is a TTS system that pairs a neural audio codec with continuous latent vectors and a latent diffusion model, and this repository offers a working PyTorch implementation of it. The package installs cleanly and includes the diffusion model, an EnCodec wrapper, and a Trainer class, so you can move from the README's example tensors to training on your own speech or singing audio. Zero-shot synthesis works through a speech prompt encoder that conditions on a short reference clip, and the code adds classifier-free guidance that the original paper didn't include. You also get the phoneme, pitch, and duration encoders, which makes it a convenient base for reproducing the paper's results or tinkering with diffusion-based TTS ideas. The project is clearly marked work-in-progress, so plan on adjusting some internals rather than dropping it straight into production.