auto-round

Why we included this project

AutoRound tackles the part of model deployment most teams dread: shrinking a large language or vision-language model down to 2 to 4 bits without the accuracy collapsing. The sign-gradient descent method, described in the SignRound papers, keeps quality high at those widths, and the project ships a Python API plus a CLI so you can move from quick experiments to scripted batch runs without changing your workflow. The real draw is how the results fit into an existing stack: quantized models export to AutoRound, AutoAWQ, AutoGPTQ, or GGUF formats and load directly into Transformers, vLLM, or SGLang. Calibration-free and model-free modes also cover the case where you lack the reference hardware or data for full tuning. In short, it removes the accuracy loss and format friction that usually make teams put off low-bit quantization.