WebGPT

Why we included this project

WebGPT is a working demonstration that a transformer can run inside a browser tab, with inference happening on the user's machine rather than on a backend. The whole thing is under 1,500 lines of plain JavaScript, using WebGPU to keep inference on the GPU, and the author documents the practical choices that make it fast: kernel fusion, key-value caching, and buffer reuse. For developers who want on-device inference or a deployment where weights never leave the user's machine, it is a useful measure of how far WebGPU has come and what it still costs. Two models are included, a toy GPT-Shakespeare and GPT-2 117M, so you can run something immediately, and the conversion scripts show how to bring your own weights. It reads like an honest engineering log rather than a product pitch, which is the right tone if your goal is to understand what transformer inference actually involves.