llama.rn

Why we included this project

Running a large language model entirely on a phone is often the right call for privacy-sensitive or offline apps, and llama.rn gives React Native developers a working path to that. The JSI bridge loads GGUF models and streams tokens back in real time, while the native side uses Metal on iOS, OpenCL on supported Android GPUs, and experimental Hexagon NPU offload to keep inference responsive. It also supports structured output through GBNF and JSON schema grammar sampling, universal function calling via Jinja templates, embedding and reranking methods, and experimental on-device text-to-speech with reference-audio voice cloning. Prebuilt native binaries ship for both platforms so you skip compiling llama.cpp yourself, though building from source is still possible. The result is a practical foundation for mobile apps that need local model inference without a server round-trip.