#149 · Primary category： Inference & Local Deploy
rkllama
Ollama alternative for Rockchip NPU: An efficient solution for running AI and Deep learning models on Rockchip devices with optimized NPU support ( rkllm )
Project last updated：08/25/26
GitHub Stars
596
Forks
99
Contributors
20
License
GPL-3.0
Why we included this project
RKLLama is a REST server and client that runs LLMs on the NPU of Rockchip boards like the Orange Pi 5 and Radxa Rock, instead of falling back to the CPU. It builds on Rockchip's rkllm runtime and targets the 6 TOPS NPU found on RK3588 and RK3576 hardware. Because it implements the Ollama API and a large part of the OpenAI API, existing clients and agents can connect to it with minimal changes, and it handles dynamic model loading, prompt caching across sessions, and tool calling. The same endpoints also cover vision, image generation, and speech in and out. If you are building an edge or offline AI appliance on Rockchip silicon, this gives you a tested way to keep inference local.
Articles for this project
No articles for this project yet.
To suggest a topic or contribute an article, contact us.
Related projects in this category
ollama
Get up and running with Kimi-K2.6, GLM-5.2, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.
llama.cpp
LLM inference in C/C++
vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
gpt4all
GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.
LocalAI
LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.