yzma

Why we included this project

yzma is a Go library that calls llama.cpp directly, so you can load GGUF models, tokenize prompts, run a sampling chain, and stream tokens from inside your own application rather than running a separate server process. It skips CGo in favor of purego and FFI, which keeps builds straightforward on Linux, macOS, and Windows, and it tracks recent llama.cpp releases to stay current with model support and performance fixes. The project also ships a CLI that fetches prebuilt llama.cpp libraries with CUDA/ROCm auto-detection, working examples for vision-language models and interactive chat, and a WebAssembly path that runs models in the browser through TinyGo and WebGPU. If you are writing a Go service or tool that needs embedded local inference, from a desktop GPU down to a Raspberry Pi or Jetson, this is a solid way to do it.