#116 · Primary category： Inference & Local Deploy
uzu
A high-performance inference engine for AI models
Project last updated：08/29/26
GitHub Stars
1.7K
Forks
73
Contributors
17
License
MIT
Why we included this project
uzu runs LLMs on the device itself, targeting Mac, iPhone, and iPad apps. Instead of routing prompts to a cloud API, the engine handles the whole local pipeline, including model downloads, memory management, and chat sessions, through a straightforward API that ships in Rust, Python, Swift, and TypeScript. By using Apple's Metal and unified memory, it keeps inference quick and keeps all data on the device, which matters for offline work or privacy-sensitive products. A single runtime can serve several model families, so teams shipping AI features in desktop or iOS apps avoid wiring together separate backends.
Articles for this project
No articles for this project yet.
To suggest a topic or contribute an article, contact us.
Related projects in this category
ollama
Get up and running with Kimi-K2.6, GLM-5.2, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.
llama.cpp
LLM inference in C/C++
vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
gpt4all
GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.
LocalAI
LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.