KuiperInfer

Why we included this project

KuiperInfer is a C++ inference engine you build from scratch, so it suits anyone who wants to see how a deep learning framework works under the hood instead of just calling one. The project walks through the pieces step by step, from tensor design and the computational graph to the operators themselves, convolution, pooling, and fully connected layers, then wires them together to run real models like ResNet, YOLOv5, Unet, and the Llama family. Because it ships CPU and CUDA operator backends plus Int8 quantization for LLMs, you get a concrete look at how inference is optimized for edge devices and GPU servers. It doubles as a structured course with video lessons and unit tests, which makes it a solid choice for students preparing for backend or inference-engineering interviews and for developers curious about what happens between model export and prediction.