x-transformers

Why we included this project

x-transformers is aimed at PyTorch users who assemble custom transformer architectures rather than fine-tune a pretrained model. The library keeps its building blocks small: the standard encoder-only, decoder-only, and encoder-decoder setups are all there, along with vision transformer wrappers, and on top of those sits a rotating set of experimental attention variants drawn from recent papers. That combination matters for researchers and engineers who want to try flash attention, other experimental attention mechanisms, or new positional encoding schemes without hand-coding each layer. The code is compact enough to read through and modify, and it installs via pip on top of PyTorch. For teams already comfortable with PyTorch, it is a practical way to get from an idea to a working model without much ceremony.