MNN

Why we included this project

MNN is a C++ inference engine for teams that need trained models to run fast on phones, embedded boards, and other hardware too small for a cloud round-trip. It converts models and executes them across CPU, GPU, and vendor-specific accelerators, with recent support for Qualcomm Hexagon DSPs, and it has been exercised inside Alibaba's own production apps. The project has also moved well beyond classic vision models: it now handles on-device LLM workloads such as the Qwen series, and ships sample apps that run offline chat and avatar demos entirely on the device. If you are shipping low-latency, offline inference to edge targets rather than training models, MNN is a runtime worth building on.