#30 · Primary category： Inference & Local Deploy
inference
Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop — all through one unified, production-ready inference API.
Project last updated：08/29/26
GitHub Stars
9.5K
Forks
865
Contributors
161
License
Apache-2.0
Why we included this project
Xinference gives you a single serving layer that makes open-weight models behave like the OpenAI API you may already be using. Point your existing client code at a self-hosted Llama or Qwen instance and most of it just works, since the endpoint speaks the same protocol. It also serves speech recognition and multimodal checkpoints through that same interface, so one deployment can cover chat, transcription, and vision workloads rather than needing a separate stack for each. On the operations side it handles CPU/GPU placement and request batching for you, and offers a CLI and WebUI in addition to the REST API for keeping an eye on what is running. Teams that want control over their data and costs without giving up a familiar API will find it a practical alternative to hosted services.
Articles for this project
No articles for this project yet.
To suggest a topic or contribute an article, contact us.
Related projects in this category
ollama
Get up and running with Kimi-K2.6, GLM-5.2, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.
llama.cpp
LLM inference in C/C++
vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
gpt4all
GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.
LocalAI
LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.