cosmos

Why we included this project

Teams working on robots or autonomous vehicles often hit a wall when they don't have enough real-world video to train their perception and control models. Cosmos gives them a way to generate that footage synthetically: a family of world models that take text, images, or short clips and produce video of how a scene will evolve, plus reasoning models that can find objects in 3D, estimate timestamps, and plan an agent's next move. The README is practical about how to actually use it, covering the generator through Diffusers or vLLM-Omni, the reasoner through Transformers or vLLM, and the whole finetuning, checkpoint export, and distillation workflow. A small team can start from the provided checkpoints instead of building a pipeline from scratch. If you need synthetic data and world simulation for physical AI, not another chatbot or video tool, this is a well-documented place to start.