#87 · Primary category： Inference & Local Deploy
chitu
High-performance inference framework for large language models, focusing on efficiency, flexibility, and availability.
Project last updated：08/28/26
GitHub Stars
3.0K
Forks
256
Contributors
51
License
Apache-2.0
Why we included this project
Most inference engines quietly assume you're on NVIDIA, but Chitu was built for the rest of the hardware world. It serves large models on Ascend, Moore Threads, Muxi, and Hygon accelerators, on pure CPU boxes, and on mixed CPU+GPU setups, so a heterogeneous fleet doesn't force you to re-platform when you want to self-host. Deployment is handled by a single chitu.run binary that bundles multi-node, multi-instance, and prefill-decode separation, so growing from one card to a cluster is a config change rather than a rewrite. The project is especially strong with very large MoE models: it was an early home for DeepSeek-R1 671B, ships FP4 and FP8 kernels that shrink the model to what a smaller team can actually afford, and the CPU+GPU path can run the full model on one machine. If your serving stack has to carry real concurrent traffic across mixed accelerators, it deserves a serious look before you default to the usual NVIDIA-centric options.
Articles for this project
No articles for this project yet.
To suggest a topic or contribute an article, contact us.
Related projects in this category
ollama
Get up and running with Kimi-K2.6, GLM-5.2, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.
llama.cpp
LLM inference in C/C++
vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
gpt4all
GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.
LocalAI
LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.