xlstm

Why we included this project

xLSTM reworks the classic LSTM into a modern recurrent architecture, trading the old gating for exponential gating and a matrix-based memory so it can carry long-range context without leaning on Transformer or state-space machinery. The repo is the official PyTorch implementation, with research variants from the paper alongside the optimized Large configuration used to train a 7B model on 2.3T tokens; fast CUDA and Triton kernels handle the heavy lifting, with plain PyTorch fallbacks elsewhere. That combination appeals to two audiences: researchers who want the exact training and kernel code behind the published results, and practitioners who want to drop an xLSTM block into their own model or load the pretrained 7B weights to see how a recurrent backbone compares with transformers and linear attention on real tasks.