cliport

Why we included this project

CLIPort is a research codebase for language-conditioned robotic manipulation, built around the idea that a robot needs to know both what an object is and where to grasp it. It pairs CLIP's semantic understanding with a TransporterNets-style spatial stream, so a single imitation-learning policy can handle tabletop tasks like stacking blocks, packing objects, and folding cloths from just a handful of demonstrations. The repo is a solid starting point for reproducing the CoRL 2021 results or building on the TransporterNets action formulation, and it includes the pieces you'd need: dataset generation for dozens of simulated tasks, single- and multi-task training scripts, evaluation harnesses, pre-trained checkpoints, and notebooks that visualize pick-and-place predictions. If your work touches pick-and-place, rearrangement, or vision-language grounding, this is one of the more complete reference implementations to dig into.