iree

Why we included this project

IREE tackles the awkward step between training a model and running it on real hardware, whether that means a datacenter GPU or a phone. It does not train models; it compiles models from PyTorch, TensorFlow, JAX, or ONNX ahead of time into a compact intermediate representation that a small runtime then executes on CPUs, CUDA, ROCm, Vulkan, and Metal. Teams that care about fast startup, a small memory footprint, and consistent performance across very different targets get all of that without writing custom kernels for each device. The project also has real-world proof behind it: AMD used an IREE-based build of SDXL for an MLPerf inference benchmark submission.