exo

Why we included this project

exo turns a handful of machines you already own into one distributed inference cluster, so a model too large for any single device, say a 600B-parameter one, can run across several Mac Studios or workstations. The software finds the other exo nodes on your network automatically, then splits the model across them based on a live view of your topology, weighing each machine's resources and the latency between links. MLX handles the inference, tensor parallelism shards the weights, and RDMA over Thunderbolt keeps inter-device latency low enough that adding machines speeds things up instead of slowing them down. It exposes the OpenAI, Claude, and Ollama APIs, so existing clients and tools keep working unchanged, and a built-in dashboard manages the cluster and provides a chat interface. Custom models can be loaded from the Hugging Face hub. That makes it a practical route for teams that want frontier-scale models on their own hardware without renting cloud GPUs, or for anyone experimenting with multi-device inference.