attention_sinks

Why we included this project

Long-running chat sessions and streaming pipelines eventually push a model past its training window, and that's where this library earns its keep. It implements the attention-sinks technique from the Efficient Streaming Language Models paper, letting a pre-trained model stay fluent well beyond its original context length while keeping memory usage roughly flat. The practical part is that it drops into the Hugging Face transformers API: you swap in AutoModel from attention_sinks, add a couple of parameters, and you're done, no retraining or changes to your generation loop. The README's benchmark charts compare perplexity and VRAM across several models, which helps you judge whether the trade-off.