magnitude

Why we included this project

Picking a model that fits your laptop's memory and runs at a usable speed usually means a lot of trial and error. Magnitude handles that sizing for you: it checks your chip, memory, and bandwidth, suggests models with estimated tokens per second, downloads the one you choose, and serves it with speculative decoding and concurrency already configured. The server loads models only when agent requests come in and unloads them when idle, so it suits coding agents and other local tooling without you having to keep a process running. You can use its built-in harness or point OpenCode, Pi, Hermes, Codex, Claude Code, or similar frontends at the loaded model, and because everything stays on your machine it works fully offline. For teams that want a private inference layer without API keys, the one-command setup and automatic model management make it a simpler alternative to hand-tuning Ollama or LM Studio.