Relax

Why we included this project

Teams doing reinforcement-learning post-training on large multimodal models will find this one of the few frameworks that handles text, image, video, and audio in a single pipeline. Relax runs as a set of independent Ray Serve services, with Megatron-LM handling training and SGLang handling rollout inference. The rollout, actor, reference, and advantage stages each sit on their own GPU cluster and pass data asynchronously through a streaming transfer queue, which is what lets it scale past the usual synchronous RL loop. It ships with a practical algorithm suite covering PPO, GRPO, GSPO, SAPO, and on-policy distillation, plus an LLM-as-judge reward mode, and it supports agentic multi-turn training with loss masking and elastic rollout scaling for closed-loop agent behaviors. If you already run Ray and Megatron in your stack, this is a serious candidate for production RL post-training rather than a research toy.