Transformer-Explainability

Why we included this project

Vision Transformer users who have tried to read raw attention maps know how little those reveal: the weights point at scattered patches, not at what actually drove the prediction. This is the reference PyTorch implementation from the CVPR 2021 paper that tackled that, replacing raw attention with relevance scores that propagate back through the network. The Colab notebooks make it easy to load a pretrained ViT or DeiT, feed in an image, and get a heatmap of what the model used to classify it, plus a BERT notebook for sentiment analysis. It is a useful, self-contained baseline for anyone auditing model behavior or debugging why a prediction went wrong, though the authors point to their newer Transformer-MM-Explainability repo for encoder-decoder and multimodal models.