web-llm

Why we included this project

WebLLM runs open models like Llama, Phi, Gemma, Mistral, and Qwen entirely inside the browser, accelerated by WebGPU, so there is no server to stand up and no inference bill to manage. Since everything happens on the device, user data never leaves the machine, which is what makes it appealing for privacy-sensitive assistants and offline tools. The engine exposes an OpenAI-compatible chat API, so the same client code you already use against hosted models can point at a local in-browser backend, with streaming, JSON-mode, and function-calling supported. It ships as an npm package with a modular MLCEngine interface, so it drops into existing web apps, Chrome extensions, or worker-based UIs without committing to a particular framework. For teams that want LLM features without hosting them, that is a workable path from prototype to production.