#101 · Primary category： Inference & Local Deploy
Bonsai-demo
Bonsai Demo
Project last updated：08/28/26
GitHub Stars
2.3K
Forks
231
Contributors
26
License
Apache-2.0
Why we included this project
The Bonsai demo is a practical way to test small, heavily compressed language models without renting GPU time. It wraps the 1-bit, ternary, and newer 27B vision-language models behind a llama.cpp server, so a chat interface with vision and tool calling comes up after a couple of setup commands. People who want to run models on laptops, workstations, or edge hardware will appreciate the hardware-specific setup notes and the agent-friendly guide that take you from clone to a working local endpoint. The repo is a demo, not a production service, but it is a solid place to see how these compact models compare with hosted alternatives before you commit to a deployment.
Articles for this project
No articles for this project yet.
To suggest a topic or contribute an article, contact us.
Related projects in this category
ollama
Get up and running with Kimi-K2.6, GLM-5.2, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.
llama.cpp
LLM inference in C/C++
vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
gpt4all
GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.
LocalAI
LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.