sru

Why we included this project

Training recurrent models in PyTorch often hits a hard ceiling at LSTM throughput, and SRU was built to get past that bottleneck. It is a recurrent cell that rearranges the recurrence so most of the computation can run in parallel, letting it approach convolution-level speed while keeping the sequential modeling you get from an RNN. Its API mirrors nn.LSTM closely enough that swapping it into an existing pipeline takes modest changes, and it installs as a normal PyTorch package with its own CUDA kernels. The original paper reports 10 to 16 times speedup over LSTM, and the newer SRU++ variant layers attention on top of the fast recurrence to cut the compute cost of language-model training. If recurrent layers dominate your latency or training budget, this is a concrete option worth benchmarking against your own workload.