AI-Gateway

Why we included this project

Standing up a production AI gateway on Azure is mostly a matter of getting the plumbing right, and this repository skips the theory in favor of working examples. It bundles 30-plus Jupyter notebooks, each with Bicep infrastructure templates and APIM policies you can deploy straight into your own subscription. The labs tackle the parts that get sticky as model traffic grows: load balancing across backends, token rate limiting, semantic caching, model routing, and keeping spend under control. On the tooling side you get MCP servers wired up with OAuth and OpenAI-style function calling, plus agent orchestration frameworks and A2A-enabled agents routed through the gateway. If you already work inside Azure API Management and Microsoft Foundry, you get a lot of concrete, copyable setups for securing and observing model traffic, ready to adapt rather than re-derive.