recipes

Why we included this project

Most of the work of standing up an in-house inference server happens before anything actually runs. You have to figure out which launch flags a given model wants, how to quantize it, and what hardware will hold it in memory. This repo collects working answers to that question, one page per model, with parameter counts, dtype, context length, and serving configs that someone has actually tested against recent open-weight releases from DeepSeek, GLM, Qwen, Llama, and others. The guides are written by people running vLLM in production, so they include details official docs tend to skip, like FP8 block quantization, multi-node setups, and CPU serving on Xeon. If you deploy open-weight models regularly, these per-model configs spare you the work of reconstructing VRAM formulas and speculative-decoding settings from scattered GitHub issues.