#4 · Primary category： LLM Application Frameworks
litellm
The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]
Project last updated：08/29/26
GitHub Stars
57.6K
Forks
11.0K
Contributors
1.7K
License
Other
Why we included this project
LiteLLM sits in front of your LLM calls and speaks OpenAI's API format, so a team can point the same code at Anthropic, Bedrock, Azure, Vertex, vLLM, or any of the 100+ supported providers without rewriting the integration. The self-hosted proxy is where the real value shows up: it handles virtual API keys, per-team spend tracking, load balancing, guardrails, and request logging behind a single endpoint. It also speaks MCP and includes an A2A agent gateway, which makes it a workable central control point for agent-heavy setups rather than just a thin API wrapper. If you want one OpenAI-compatible server to standardize on and need to see what each team is spending, this is a solid place to start.
Articles for this project
No articles for this project yet.
To suggest a topic or contribute an article, contact us.
Related projects in this category
langchain
The agent engineering platform.
dify
Build Agentic workflows, RAG pipelines, with rich AI model and tool support on one collaborative workspace. Deploy on cloud, VPC, or self-hosted, so teams move from prototype to production without rebuilding the stack.
headroom
Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.
llama_index
LlamaIndex is the leading document agent and OCR platform
pathway
Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.