GPTQModel

Why we included this project

Running a large open-weight model on a single consumer GPU usually comes down to quantization, and GPTQModel is one of the more complete toolkits for that job. It implements GPTQ and AWQ and compresses models so they fit in a fraction of the VRAM while staying fast enough for real inference. The hardware coverage is unusually wide: CUDA, ROCm, Intel XPU, Huawei Ascend NPU, and Intel, AMD, and Apple CPUs all have kernels, and compressed checkpoints load through Hugging Face, vLLM, or SGLang, so the same model can move from a laptop test run to a production serving stack. The project also keeps pace with new releases; recent updates added Blackwell GPU kernels and support for a steady stream of new architectures, which matters because quantization code tends to lag behind new models.