node-llama-cpp

Why we included this project

This project is a Node.js binding for llama.cpp, letting you run GGUF models locally inside your own process rather than calling a hosted API. It ships pre-built binaries for common platforms and compiles from source when needed, so setup mostly means installing the package and loading a model. The part that stands out is output control: you can force the model to follow a JSON schema or a grammar at generation time, hand it functions to call, and use it for embeddings, which makes raw output usable as program input without fragile parsing. GPU acceleration works through Metal, CUDA, or Vulkan with automatic detection, so there is no backend config to fiddle with. If you are shipping local or on-premise LLM features in a JavaScript app, this is a solid integration point.