self-attention-cv

Why we included this project

This PyTorch package gathers the self-attention blocks that keep appearing in vision transformers, saving you from reimplementing them each time. It includes multi-head attention, a transformer encoder, axial attention, a Vision Transformer, a TransUNet encoder, bottleneck attention blocks, and 1D/2D positional embeddings, all written with einsum and einops so the code stays readable. Each module comes with a short runnable example, so you can try it against your own tensors before reading the papers. The author's companion articles walk through the math behind each mechanism, which makes the repo a good learning resource while you prototype. Since the project hasn't been updated in a while, treat it as a reference implementation to build on rather than a production dependency you'd pin.