nndeploy

Why we included this project

nndeploy is a deployment layer that sits between a trained model and the hardware it runs on. It hides thirteen inference backends, including ONNXRuntime, TensorRT, OpenVINO, MNN, CoreML and AscendCL, behind one C++/Python API, so a workflow defined once can target Windows, macOS, Android, iOS, Jetson and Ascend hardware without reimplementing anything per platform. The standout feature is the visual editor: you assemble preprocessing and model nodes by drag-and-drop, tune parameters with a live preview, and export the result as a JSON graph that Python or C++ loads in production. Parallelism at both pipeline and task level, plus memory tricks like zero-copy and pooling, keep latency down, and the bundled node library already covers LLMs, Stable Diffusion, detection, OCR and segmentation, so a working edge demo takes far less time to put together. For 10B+ generative models, such as large language or AIGC generation models, nndeploy also works well as a visual workflow tool in its own right.