lollms_hub

Why we included this project

Running several local inference engines usually means juggling different APIs and keeping track of who is using your GPUs. LoLLMs Hub pulls Ollama, vLLM, llama.cpp, and OpenAI-compatible cloud endpoints behind one API that speaks both the Ollama and OpenAI protocols, so applications you already have keep working unchanged. It also handles the housekeeping that gets awkward once more than a couple of people share a machine: API-key authentication, per-user usage tracking, rate limiting, and HTTPS termination. The routing layer is where it gets interesting, since you can define virtual models that pick a backend by priority, least-loaded, or rules like image detection, which lets a text-only model quietly pass vision requests to a VLM. If you are pooling a few machines, the master/slave chaining means only one node needs public exposure while the workers stay on a private network.