llama-cpp-agent

Why we included this project

For Python developers who want a local LLM to do more than chat, this framework is worth a look. It wraps llama.cpp to support conversations, structured JSON output, and function calls, even against models that were never fine-tuned for tool use. That's the part that matters in practice: many small models choke on strict JSON or tool-calling, and this library does the parsing and formatting for you. Teams prototyping local, privacy-friendly agents on CPU or modest GPUs should find the setup straightforward. One honest caveat: the project is no longer actively maintained, with the author pointing users to ToolAgents, so treat it as a reference implementation or a starting point rather than a long-term dependency.