picolm

Why we included this project

Most LLM runtimes expect hardware that costs more than the project they are serving, and they simply will not boot on a Raspberry Pi Zero or a RISC-V board with 256MB of RAM. PicoLM takes the opposite route: a from-scratch inference engine in about 2,500 lines of pure C with zero dependencies, compiled to a single ~80KB binary that reads GGUF models directly. It memory-maps the weights and streams them through RAM one layer at a time, which is how a 1.1B model fits in roughly 45MB, while still supporting quantized K-formats, flash attention, ARM NEON and SSE2 SIMD, and multithreaded matmul. PicoLM is also the local brain for the PicoClaw offline agent, with stdin/stdout piping, grammar-constrained JSON for tool calling, and KV cache persistence, so you can drop it into your own automation loop with no Python and no cloud. For teams prototyping edge AI assistants on constrained devices, it is a useful reference for how far a small, hand-tuned C inference stack can go.