efficient-dl-systems

Why we included this project

An open course on the systems side of deep learning, taught at HSE University and the Yandex School of Data Analysis, with the full materials published here as weekly lectures and seminars. It starts with GPU architecture and the CUDA API, then works through mixed-precision training and profiling with PyTorch Profiler and Nsight Systems before getting into distributed training: All-Reduce, tensor and pipeline parallelism, and FSDP2. The later weeks move to serving, where KV caching, continuous batching, FlashAttention, quantization, and speculative decoding get the same hands-on treatment. For engineers who want to understand why a training run or inference endpoint is slow rather than just which library call to make, the notebooks and assignments offer concrete, current practice you can run and adapt. And if you are setting up internal training, the seminar and assignment content doubles as ready-made onboarding material.