RWKV-LM

Why we included this project

RWKV is one of the more interesting bets in the open-weight LLM space because it deliberately abandons the attention mechanism that powers most modern transformers. Instead it runs as a pure recurrent network, so inference cost stays flat as context grows and there is no KV-cache to manage. Those properties matter a lot if you are serving long documents or many concurrent requests on limited hardware. The repository is the reference implementation for the RWKV-7 'Goose' architecture, with training code, CUDA kernels, and demo scripts, so it is the place to go if you want to train or fine-tune your own RWKV model rather than just load a checkpoint. Teams exploring alternatives to transformer-based LLMs for edge or mobile deployment will find the architecture genuinely different from the usual GPT-style stack. It is also a Linux Foundation AI project, which gives some reassurance about long-term stewardship of the codebase.