tiny-llm

Why we included this project

This course is aimed at systems engineers who have called LLM inference APIs but want to see what actually happens inside them. Over four weeks it walks you through building a miniature serving stack on Apple Silicon, starting with array and matrix operations, assembling a readable Qwen3 model, adding a KV cache, and layering in continuous batching and paged KV caching until you have something close to a tiny vLLM. The whole thing runs on MLX primitives rather than high-level neural-network layers, so each exercise makes you implement the operator yourself in Python, C++, or Metal, with MLX left as the correctness oracle. That is what connects the math to real memory traffic, kernel occupancy, and scheduling decisions. If you want to understand inference serving from the inside instead of treating it as a black box, this is a course to work through chapter by chapter.