1Cat-vLLM

Why we included this project

Tesla V100 cards are still in service in a lot of small shops, but recent vLLM releases have either dropped Volta support or run so slowly on it that serving feels pointless. This fork keeps the vLLM serving stack and adds what's missing for that hardware: SM70-specific kernels, a dedicated V100 FlashAttention backend, and AWQ 4-bit inference paths tuned for Qwen-class models. It ships validated launch profiles for 27B and 35B AWQ models on two- and four-card V100 setups, with long-context defaults and OpenAI-compatible endpoints that standard clients work with. For a team that already owns these cards, that's the difference between standing up a real model server and writing the hardware off. If you're stuck extracting value from older NVIDIA data-center GPUs, the concrete, reproducible serving commands here are worth more than generic advice.