ai-gateway

Why we included this project

Teams that call hosted LLM APIs from their applications often end up wiring each provider differently. Envoy AI Gateway gives them a single entry point built on Envoy Gateway, handling authentication, routing, and global rate limits in one place so the platform team does not wire each provider by hand. The two-tier design is the part worth noticing: one gateway fronts all providers centrally, while a second tier sits in front of self-hosted model clusters and picks endpoints to improve inference efficiency. That split appeals to infrastructure engineers running Kubernetes who want one control point for cost and access across many model providers, and since it comes from the Envoy project it inherits gateway semantics people already trust rather than inventing a proprietary proxy.