ddpo-pytorch

Why we included this project

Finetuning a diffusion model usually means supervised training on curated image pairs, but this project takes a different route: it applies reinforcement learning through Denoising Diffusion Policy Optimization, letting you push Stable Diffusion toward whatever objective you can score, from compressibility to aesthetics. The LoRA option keeps memory under 10GB, so a single consumer GPU is enough to experiment. The code is built around pluggable prompt and reward functions, so you can define your own scoring criteria without touching the training loop. It's research code rather than a polished library, and the default config is meant for a quick smoke test, so expect to adjust batch sizes and accumulation steps yourself. For researchers or engineers prototyping RL-based finetuning, this is a useful reference.