MinivLLM

Why we included this project

Engineers who run vLLM in production without ever tracing what happens inside it will get the most from this project. It is a from-scratch reconstruction of the inference engine, replacing third-party dependencies with self-contained Triton implementations of paged attention and flash attention, so the internals stay open to inspection instead of being buried in a library. The repo bundles benchmarks for both the prefilling and decoding phases, comparing standard PyTorch attention, a naive Triton kernel, and flash attention, which makes the memory and latency trade-offs visible in practice. A step-by-step guide covers model layers, paged attention, CUDA graphs, and scheduling, so it doubles as an onboarding path for teams that want to tune or debug their own serving stack rather than treat it as a black box.