mace

Why we included this project

Mobile teams that want to run neural networks directly on phones or tablets will get the most out of this one. MACE is an inference engine for Android, iOS, Linux, and Windows, so it fits any project where a model has to execute on the device itself. It leans on NEON for the CPU, OpenCL for GPUs, and Qualcomm's Hexagon DSPs, and it applies the Winograd algorithm to make convolutions faster. Power and responsiveness get real attention: big.LITTLE scheduling and Adreno GPU hints help keep energy use in check, and OpenCL kernels can be split into small preemptible units so the UI stays fluid while inference runs. It can also protect models by converting them to C++ code with literal obfuscation. And because it reads TensorFlow, Caffe, and ONNX formats, it is a straightforward way to take existing trained models and ship them to resource-constrained devices.