llama.go

Why we included this project

Go developers who need to run LLaMA-style models without cgo or external subprocesses don't have many pure-Go options, and this project is one of the few. It reimplements llama.cpp's tensor math and model architecture in Go, so a single process can load GGML-era LLaMA checkpoints, run local inference, and expose a REST API through the built-in server mode. The multi-threading support and AVX2/NEON optimizations indicate serious attention to performance. That said, the project is essentially in maintenance mode: the author now points to a successor (FastTensors), and support for LLaMA 2, GGUF V3, and newer formats was never completed. So it's a good fit for understanding how an inference engine works in Go or for experimenting with early LLaMA models, not for running current open-weight releases in production.