atlas

Why we included this project

Atlas is a from-scratch LLM inference engine written in pure Rust, built for teams that want to run large models on their own hardware rather than paying per-token cloud rates. It ships kernels tuned for specific hardware and model families across NVIDIA, AMD, and Intel. Speculative decoding and KV-cache quantization are built in, and it runs Mamba/SSM architectures as well as plain transformers. The OpenAI-compatible API means existing tooling can point at it with minimal changes, so it works as a drop-in for people already running llama.cpp or vLLM who want something faster and lighter on dependencies. It also documents how to add new hardware targets and new models, which helps if you plan to extend it rather than just consume it. Before betting a production serving layer on it, look closely at the kernel strategy and the AGPLv3 license.