custom-diffusion

Why we included this project

Custom Diffusion is a research method from Adobe and CMU, published at CVPR 2023, for teaching a text-to-image model a new concept from a small set of reference photos, typically four to twenty images of a pet, object, or style. The practical gain comes from fine-tuning only the key and value projection matrices in the cross-attention layers, so each added concept costs about 75 MB of extra weights instead of a full model copy. It also handles multiple concepts at once, like combining a new object with an artistic style or two new objects, either by joint training or by merging weight deltas. The repo ships the CustomConcept101 dataset, evaluation prompts, pretrained checkpoints, and integration with Hugging Face diffusers, which makes the pipeline reproducible. Expect to run training scripts on GPU hardware; this is research code, not a turnkey product.