#181 · Primary category: Inference & Local Deploy
tensorRT_Pro
C++ library based on tensorrt integration
Project last updated:05/24/23
GitHub Stars
2.9K
Forks
573
Contributors
17
License
MIT
Why we included this project
tensorRT_Pro wraps TensorRT in a compact C++ and Python API, so a YOLOv5, YOLOX, or YOLOv7 model can go from PyTorch weights to a running inference engine in a few lines of code. It covers the whole workflow, from ONNX export and engine compilation through serialization and CUDA-accelerated preprocessing and postprocessing, and the same wrapper extends to face, pose, and tracking models like RetinaFace, ArcFace, AlphaPose, CenterNet, and DeepSORT. FP32, FP16, and INT8 build paths are supported, and the bundled benchmark logs give you realistic speed numbers to check against your own hardware before committing to a deployment target. The tutorial videos, notebooks, and a commented community fork make it a workable place to learn TensorRT optimization rather than just copy example code.
Articles for this project
No articles for this project yet.
To suggest a topic or contribute an article, contact us.
Related projects in this category
ollama
Get up and running with Kimi-K2.6, GLM-5.2, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.
llama.cpp
LLM inference in C/C++
vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
gpt4all
GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.
LocalAI
LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.