#174 · Primary category: Inference & Local Deploy
ppq
PPL Quantization Tool (PPQ) is a powerful offline neural network quantization tool.
Project last updated:03/28/24
GitHub Stars
1.8K
Forks
285
Contributors
26
License
Apache-2.0
Why we included this project
Trained networks run on floating point, but edge hardware wants fixed-point math, and PPQ exists to manage that conversion. It quantizes ONNX and PyTorch models offline, so instead of a one-size-fits-all default you can set the bit width and granularity per operator or tensor, and bring your own calibration routine. The built-in execution engine runs quantization-aware simulation without Onnxruntime, and its graph passes cover calibration, fusion, bias correction, and layer equalization, so you can shape the pipeline yourself. Deploying to TensorRT, ncnn, MNN, or OpenVINO gets you hardware-specific quantizers and exporters, and a custom CUDA backend is on the table too. For teams that want to tune the compression themselves, this is a practical alternative to black-box conversion tools.
Articles for this project
No articles for this project yet.
To suggest a topic or contribute an article, contact us.
Related projects in this category
ollama
Get up and running with Kimi-K2.6, GLM-5.2, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.
llama.cpp
LLM inference in C/C++
vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
gpt4all
GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.
LocalAI
LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.