mesh-llm

Why we included this project

Mesh LLM pools GPUs and memory across your machines, or across a shared public mesh, and exposes the result as one OpenAI-compatible API on localhost. You start a single node and add more as the workload grows; the mesh itself decides whether a model runs locally, routes to a peer, or gets split across stages when it is too large for one machine. That serves two situations well: running a private multi-machine inference cluster for agents and chat without building orchestration yourself, and borrowing community-shared capacity when your own hardware runs short. Installation takes a few commands from binaries, a Homebrew formula, or Ubuntu and Arch packages, so the gap between download and a working serving endpoint is small. If you need more aggregate GPU memory than a single workstation provides but do not want to commit to a managed cloud, this is worth a close look.