DALLE-pytorch

Why we included this project

This project reimplements OpenAI's original DALL-E text-to-image transformer in PyTorch, with training and generation code that is readable enough to follow. Researchers and engineers can swap in the DiscreteVAE or a pretrained VQGAN encoder and train a small-scale model on their own image-caption dataset. Because the VAE, autoregressive transformer, and CLIP-style re-ranking step sit as separate components, the code doubles as a working tutorial on how discrete image tokens and transformer generation fit together. For anyone exploring pre-DALL-E-2 pipelines or wanting to reproduce the 2021 approach on modest hardware, this is far more approachable than the paper alone.