#174 · Primary category: Inference & Local Deploy

ppq

caffe cuda deep-learning neural-network onnx open-source pytorch quantization

PPL Quantization Tool (PPQ) is a powerful offline neural network quantization tool.

Project last updated:03/28/24

GitHub Stars

1.8K

Forks

285

Contributors

26

License

Apache-2.0

Why we included this project

Trained networks run on floating point, but edge hardware wants fixed-point math, and PPQ exists to manage that conversion. It quantizes ONNX and PyTorch models offline, so instead of a one-size-fits-all default you can set the bit width and granularity per operator or tensor, and bring your own calibration routine. The built-in execution engine runs quantization-aware simulation without Onnxruntime, and its graph passes cover calibration, fusion, bias correction, and layer equalization, so you can shape the pipeline yourself. Deploying to TensorRT, ncnn, MNN, or OpenVINO gets you hardware-specific quantizers and exporters, and a custom CUDA backend is on the table too. For teams that want to tune the compression themselves, this is a practical alternative to black-box conversion tools.

Articles for this project

No articles for this project yet.

To suggest a topic or contribute an article, contact us.

Related projects in this category