#111 · Primary category： Inference & Local Deploy
vllm-mlx
High-performance OpenAI and Anthropic compatible LLM inference server for Apple Silicon. Native MLX, continuous batching, multimodal models, MCP tool calling, and Claude Code support.
Project last updated：08/26/26
GitHub Stars
1.6K
Forks
219
Contributors
57
License
Apache-2.0
Why we included this project
vllm-mlx turns an Apple Silicon Mac into a real inference server rather than a demo. It runs on the native MLX backend with no conversion step, and it brings continuous batching, a paged KV cache, and prefix caching to concurrent workloads, so requests stay responsive instead of queuing up. The same process handles text, vision-language, and audio models, and it exposes both OpenAI- and Anthropic-compatible endpoints on a single port. That second API is the practical win: point Claude Code or any Anthropic SDK at it and your existing agent tooling works against local weights unchanged. For teams that want private, self-hosted inference with throughput that doesn't embarrass them, that combination is hard to find elsewhere on a Mac.
Articles for this project
No articles for this project yet.
To suggest a topic or contribute an article, contact us.
Related projects in this category
ollama
Get up and running with Kimi-K2.6, GLM-5.2, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.
llama.cpp
LLM inference in C/C++
vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
gpt4all
GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.
LocalAI
LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.