#26 · Primary category： Inference & Local Deploy
OpenLLM
Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.
Project last updated：08/24/26
GitHub Stars
12.5K
Forks
836
Contributors
33
License
Apache-2.0
Why we included this project
Self-hosting an LLM usually means picking an inference backend, sizing GPU resources, and wiring everything together by hand. OpenLLM hides that work behind a single openllm serve command, turning an open-weight model like DeepSeek or Llama into an OpenAI-compatible API you can call just like a hosted provider. It includes a chat UI for quick interactive testing, and because it's built on BentoML, the same setup deploys to Docker or Kubernetes without turning into a one-off script. The result is that you can go from a downloaded model to a cloud-deployable endpoint without becoming an expert in every serving framework underneath.
Articles for this project
No articles for this project yet.
To suggest a topic or contribute an article, contact us.
Related projects in this category
ollama
Get up and running with Kimi-K2.6, GLM-5.2, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.
llama.cpp
LLM inference in C/C++
vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
gpt4all
GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.
LocalAI
LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.