llama.cpp

Why we included this project

llama.cpp runs large language models on hardware you actually own, and it does so in plain C/C++ with no external dependencies. It works on a laptop CPU, Apple Silicon, NVIDIA and AMD GPUs, and even RISC-V or WebGPU targets, and its integer quantization keeps memory use low enough that bigger models fit on modest machines. Alongside a command-line tool for interactive sessions, it ships an OpenAI-compatible server with a built-in web UI, so existing apps that speak the standard chat API can point at it as a backend without much work. That makes it a natural fit for on-device assistants, edge deployments, and privacy-sensitive tools where you want the model running on hardware you control. Many other projects in the local-inference ecosystem build on llama.cpp, so getting familiar with it makes the rest of that landscape easier to judge.