kat

Why we included this project

KAT is the official PyTorch implementation of the Kolmogorov-Arnold Transformer, a research architecture that replaces the MLP blocks in vision transformers with KAN-style layers built on rational basis functions. The main idea is to make KAN layers scale to large datasets like ImageNet, which plain ViT plus KAN struggles to do. The repo includes the CUDA kernels, a Triton fallback that is much easier to install, and the training pipeline used for the ICLR 2025 paper, so you can reproduce the results or adapt the pieces into your own vision backbone. It is most useful to researchers and engineers who want to benchmark a KAN-based block against a standard ViT baseline, rather than to teams looking for a drop-in production model.