Tengine

Why we included this project

If you need to run neural networks on constrained hardware rather than in a cloud datacenter, Tengine is a serious option. It's a C-based inference engine built for embedded and AIoT devices, where memory and power limits rule out full-size frameworks. The front-end/back-end separation means the same operator graph can be deployed across ARM CPUs, Mali and NVIDIA GPUs, NPUs such as VeriSilicon and NVDLA, and RISC-V cores without rewriting model logic per chip. Conversion tools handle ONNX, Caffe, MXNet, Darknet, TensorFlow, and PaddlePaddle models, and built-in uint8/int8 quantization, along with tools like ACIQ, DFQ, and EasyQuant, help shrink models to fit modest silicon. Teams working on classification, detection, or segmentation on edge devices will find per-board examples for A311D, i.MX 8M Plus, Jetson, and ZCU102 with measured inference latency.