chatglm.cpp

Why we included this project

ChatGLM.cpp packs several generations of the ChatGLM family, from the 6B models up through GLM-4 and GLM-4V, into a lightweight C++ runtime built on ggml, the same library that powers llama.cpp. It targets laptops and edge hardware: you can convert a Hugging Face checkpoint into a quantized int4 or int8 GGML file, then run it with memory-efficient CPU inference, optionally using NVIDIA or Apple Silicon GPUs for acceleration. Streaming generation, support for P-Tuning v2 and LoRA fine-tuned checkpoints, plus Python bindings and API servers, make it straightforward to embed the models into your own application instead of calling a hosted endpoint. Anyone evaluating self-hosted deployments of the GLM family, or just wanting a llama.cpp-style runtime tuned for the GLM architecture, will find a practical base here.