zml

Why we included this project

ZML is for teams that run neural networks in production and want to squeeze more out of the accelerators they already own. Instead of wrapping a Python runtime, it compiles a model's forward pass directly into accelerator-specific code for NVIDIA, AMD, Intel, TPU, and Trainium, so a single Zig model codebase runs on any of them without rewriting. You describe the model as a Zig struct and build and run it with Bazel; the examples move from a 30-second MNIST smoke test to Llama, Qwen, and LFM checkpoints loaded from Hugging Face, local directories, or S3. Compilation and weight loading happen per accelerator at runtime, which suits engineers who want predictable, low-overhead inference rather than a batteries-included serving layer. The trade-off is that Zig and Bazel familiarity are essentially prerequisites, but the payoff is direct control over the path from model weights to the hardware.