self-rewarding-lm-pytorch

Why we included this project

Researchers and ML engineers who want to reproduce Meta's self-rewarding language model paper will find this a compact reference implementation that spares them the usual chore of turning the paper into working code. The SelfRewardingTrainer bundles the full iterative loop, where the model generates candidate responses and then scores its own output to build fresh training data for the SFT and DPO stages, and the repo also includes a SPIN trainer for an alternative self-play fine-tuning route. Because it builds on x-transformers, you can plug in a standard decoder and start experimenting. It is best treated as a research tool rather than production training software, but for teams exploring alignment methods that do not rely on human-annotated preference data, the code is far easier to dissect than the original research pipeline.