#57 · Primary category： Inference & Local Deploy
gpustack
A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.
Project last updated：08/28/26
GitHub Stars
5.6K
Forks
631
Contributors
55
License
Apache-2.0
Why we included this project
GPUStack is a control plane for teams serving open models across a fleet of GPUs, so you don't have to configure each machine by hand. It sets up inference engines like vLLM and SGLang for the hardware you have and exposes an OpenAI-compatible API, which lets existing applications connect without custom integration code. Workloads get scheduled across on-prem servers, Kubernetes clusters, and cloud GPUs, which helps when capacity is spread between environments or demand spikes. It can also launch SSH-accessible GPU instances on demand for fine-tuning and interactive development work. Pre-tuned latency and throughput profiles, plus speculative decoding and extended KV cache support, mean you don't have to tune the whole stack from scratch.
Articles for this project
No articles for this project yet.
To suggest a topic or contribute an article, contact us.
Related projects in this category
ollama
Get up and running with Kimi-K2.6, GLM-5.2, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.
llama.cpp
LLM inference in C/C++
vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
gpt4all
GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.
LocalAI
LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.