qwen-vllm

Why we included this project

Standing up a Qwen model behind a real HTTP service usually means stitching together vLLM's async server, a client, and a chat UI from scattered examples. This repo does that assembly for you: it shows vLLM's continuous batching running in a background thread behind a FastAPI/uvicorn endpoint, with tokens streamed back to clients, plus a client script and a Gradio chat interface that handle multi-turn conversation. The author also walks through Qwen's prompt format and how the base and chat-tuned checkpoints differ, which helps when you're deciding how to format inputs. It's a demo, not a hardened production service, but as a working reference it saves you the time of reading several official examples and wiring them together yourself.