go-llama.cpp

Why we included this project

Go developers who want to run a local language model inside their own process rather than calling a separate inference server get a clean path here. The bindings sit high above llama.cpp, keeping the heavy compute in the C/C++ engine so you skip most of the low-level cgo hassle and stay reasonably performant. Only the current GGUF model format is supported, and acceleration is available through OpenBLAS on CPU, CuBLAS on NVIDIA GPUs, ROCm, OpenCL, and Metal on Apple Silicon. That makes the library a good fit for desktop tools and edge services that want to stay self-contained while shipping local models. The upstream engine comes in as a git submodule, and the included example covers building the bindings and running a model well enough to adapt the pattern to your own code.