apisix

Why we included this project

APISIX is a good fit for teams that already run an API gateway and now need to handle LLM traffic without adding a separate stack. It fronts providers like OpenAI, Anthropic, AWS Bedrock, DeepSeek, and Ollama, so you can expose many models through one endpoint, with load balancing, automatic failover to a backup provider, and token-based rate limits that keep usage and cost per consumer in check. Beyond the AI features, it's a full cloud-native gateway with dynamic routing, hot-reloadable plugins, authentication, and observability, handling both north-south API traffic and east-west service calls in the same deployment. The Kubernetes ingress controller and support for gRPC, WebSocket, and MQTT make it practical for microservice-heavy stacks. That combination gives you one control plane for regular APIs and LLM workloads alike.