#168 · Primary category： Inference & Local Deploy
bolt
Bolt is a deep learning library with high performance and heterogeneous flexibility.
Project last updated：04/11/25
GitHub Stars
957
Forks
162
Contributors
16
License
MIT
Why we included this project
Bolt targets developers shipping trained models to phones, embedded boards, and low-power Linux boxes where speed and memory matter. It converts Caffe, ONNX, TFLite, and TensorFlow models to its own format, then runs them on ARM CPUs, Mali and Qualcomm GPUs, and x86 CPUs (AVX2/AVX512). Precision support runs from FP32 and FP16 down to INT8 and 1-bit quantization, so you can dial back accuracy to fit latency and RAM budgets on constrained hardware. Huawei built and used it internally for NLP and common vision workloads, and the repo ships build scripts covering many ARM and x86 targets. If you are deciding whether a model can run on-device fast enough, this gives you a concrete way to measure that before committing to a deployment path.
Articles for this project
No articles for this project yet.
To suggest a topic or contribute an article, contact us.
Related projects in this category
ollama
Get up and running with Kimi-K2.6, GLM-5.2, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.
llama.cpp
LLM inference in C/C++
vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
gpt4all
GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.
LocalAI
LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.