how-to-optim-algorithm-in-cuda

Why we included this project

Engineers who write or optimize GPU kernels will find a practical study notebook here: handwritten CUDA implementations of common operations like reduce, softmax, and GEMV sit alongside notes on the CUTLASS, Triton, and PTX layers underneath, so you can see both the kernel and the tooling around it. The material tracks the author's own path from hand-tuned kernels into LLM inference and training optimization, and it reads like someone working through the problems rather than presenting finished code. Tying it together are lecture notes from the CUDA-MODE series and summaries of GPU architecture papers, which give context that raw kernel listings never provide. For anyone learning GPU systems, this is a useful way to build the mental model that textbooks tend to skip.