#213 · Primary category： Inference & Local Deploy
minigpt4.cpp
Port of MiniGPT4 in C++ (4bit, 5bit, 6bit, 8bit, 16bit CPU inference with GGML)
Project last updated：08/08/23
GitHub Stars
573
Forks
28
Contributors
4
License
MIT
Why we included this project
MiniGPT-4 normally needs a GPU, but this C++ port runs its inference on plain CPUs. Built on the ggml library, it loads quantized 7B or 13B checkpoints and answers questions about images without any dedicated accelerator. Pre-quantized weights are hosted on Hugging Face, and conversion scripts let you quantize a PyTorch checkpoint yourself if you want a specific bit depth. A small Gradio web UI covers interactive use, and the library compiles to a shared object on Windows, Linux, and macOS. It's a research-era port, last touched in 2023, so treat it as a working reference for local multimodal inference rather than a maintained production tool.
Articles for this project
No articles for this project yet.
To suggest a topic or contribute an article, contact us.
Related projects in this category
ollama
Get up and running with Kimi-K2.6, GLM-5.2, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.
llama.cpp
LLM inference in C/C++
vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
gpt4all
GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.
LocalAI
LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.