vllm-metal

Why we included this project

The plugin runs vLLM's serving stack on Apple Silicon by pointing the compute backend at MLX instead of CUDA. The familiar vllm CLI and HTTP API carry over, so teams already serving models with vLLM on GPUs can reuse those workflows on arm64 Macs without maintaining a second inference tool. It's community-maintained rather than an official vLLM build, and the v0.2.0 release tightened the attention kernel, reporting an 83x cut in time to first token and 3.6x throughput over v0.1.0. If you want one serving framework that works on both data-center GPUs and local Macs, that consistency is the main reason to try it.