#3 · Primary category: LLM Application Frameworks
headroom
Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.
Project last updated:08/29/26
GitHub Stars
68.0K
Forks
5.3K
Contributors
251
License
Apache-2.0
Why we included this project
Headroom cuts the token bill before your prompt ever reaches the model, and it does it without making you change how you call the API. You can run it as a proxy, import it as a Python or TypeScript library, or wire it into a framework, and it figures out on its own whether a payload is JSON, code, logs, diffs, or plain text so it can pick the right compressor. The savings are real: around 20% fewer tokens for coding agents and 60-95% fewer for JSON-heavy payloads. It only compresses the newest content blocks, leaves the prompt-cache hot zone alone, and keeps the originals around so the model can pull back full detail when it needs it. Teams on Claude Code, Cursor, or LangChain/Agno agents with large codebases or long debugging sessions will see the savings add up, and the proxy path needs zero code changes, so you can test it without touching your stack.
Articles for this project
No articles for this project yet.
To suggest a topic or contribute an article, contact us.
Related projects in this category
langchain
The agent engineering platform.
dify
Build Agentic workflows, RAG pipelines, with rich AI model and tool support on one collaborative workspace. Deploy on cloud, VPC, or self-hosted, so teams move from prototype to production without rebuilding the stack.
litellm
The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]
llama_index
LlamaIndex is the leading document agent and OCR platform
pathway
Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.