tensorrt-cpp-api

Why we included this project

Working with TensorRT directly means writing a lot of nvinfer1 boilerplate and keeping track of buffers, streams, and error handling yourself. This library wraps that in a small no-throw C++ API: you hand it an ONNX model, it builds an optimized engine (or loads one from cache), and tensors are keyed by name at the boundary while CUDA streams stay caller-owned. The cache is the part that stands out in production, since it is keyed by model content, build options, TensorRT version, and GPU UUID, so a stale engine gets rebuilt instead of silently running outdated weights. Dynamic shapes, FP16/INT8/FP8 precision, and concurrent inference through an execution-context pool are all built in, and the optional zero-copy Python bindings let you pass CuPy or PyTorch GPU arrays without host round-trips. If you are shipping vision models on NVIDIA GPUs in C++, this saves you from reinventing the plumbing.