tiny-vllm

Why we included this project

tiny-vllm is two things at once: a working LLM inference server written in C++ and CUDA, and a course that walks you through building it. The author derives the math and the implementation from scratch, covering the full forward pass, KV cache, continuous batching, and PagedAttention in hand-written CUDA kernels, so by the end you have a runnable engine and a real sense of how it works. That makes it useful in two directions. Developers who have used llama.cpp or vLLM without ever looking inside can trace exactly what happens between loading a Safetensors model and generating tokens. Lecturers and self-learners get a ready-made lesson sequence, from bfloat16 and RMSNorm to online softmax and paged KV cache, that works as a university teaching resource or a self-paced project.