soundstorm-pytorch

Why we included this project

SoundStorm generates audio in parallel rather than one token at a time, which is what makes it attractive for speech synthesis. This repo turns the DeepMind approach into a PyTorch library you can actually run: MaskGIT-style iterative demasking over the residual vector-quantized codes from a SoundStream encoder, with a Conformer backbone that fits the audio domain. The components are kept separate, so you can train the model from scratch on raw audio with your own SoundStream encoder, or drop in a TextToSemantic transformer to go from text to speech waveforms. If you are building a text-to-speech pipeline and want the audio decoding stage to be fast rather than the bottleneck, this is a practical codebase to start from, and the accelerate-based trainer is easy to adapt to your own data.