EBT

Why we included this project

Energy-Based Transformers take a different route from the standard feed-forward stack: the model reasons over every token it predicts instead of emitting one forward pass. The repo backs the paper with pretraining and inference scripts for language, image, and video, plus a minimal single-file training loop that pits a Transformer++ baseline against an EBT, which is the fastest way to see the mechanics. It is research code, not a library, so expect to adapt the job scripts, wandb setup, and HPC launchers to your own cluster. If you are working on System 2 reasoning, energy-based objectives, or how alternative attention architectures scale, this is a solid place to start reproducing the paper's experiments.