The 10 best AI gateways in 2026

Updated August 4, 2026. We revisit this comparison as the market moves.

The AI gateway went from a developer convenience to a procurement category this year: Gartner and G2 now track it as a market of its own, API-gateway veterans like Kong and F5 ship dedicated AI products, and two of the best-known independents were acquired. This guide compares the ten AI gateways we would actually shortlist in 2026, with pricing, data-residency posture and market status verified against each vendor's own pages and announcements, linked throughout so you can check every claim. If your traffic is text-model routing for a developer team, we also maintain a deeper ranking of the best LLM gateways, the sibling to this guide.

Same disclosure as there: Opper is our product, so read that entry knowing where we stand. Every number here, ours included, comes from public primary sources.

AI gateway, API gateway, LLM gateway: which is which

An API gateway manages traffic to your own services, handling authentication, rate limits and routing for conventional APIs. An AI gateway applies that same control point to model traffic, where the problems change: costs are token-based rather than per-request, requests need routing and fallbacks across model providers, prompts may carry personal data that logging policies have to account for, and outputs need guardrails. An LLM gateway is the text-model core of that layer, and the term AI gateway has come to cover the broader version, including multimodal models and, increasingly, MCP traffic between agents and tools.

The market attacks this from both sides. API-gateway vendors (Kong, F5, Zuplo, the Envoy ecosystem) extend proven traffic infrastructure toward models, while AI-native gateways (Opper, OpenRouter, LiteLLM, Portkey) were built for model traffic from the start. Which side you buy from depends mostly on what you already run, which is what the criteria below are for.

How to choose: six criteria that actually separate them

• Provider breadth. Can it route across the providers you need today and switch when better models ship? Catalogs range from four providers to 500+ models.
• Pricing model. No-markup gateways charge a fee on credit purchases, others price per logged request, per seat, or bundle the gateway into a platform. Free tiers vary from generous to none.
• Data residency and logging. Where does the gateway run, where do prompts and logs live, what is retained by default, and can you pin traffic to EU-hosted models?
• Governance and guardrails. Budgets with hard stops, per-user attribution, audit trails, PII handling, content policies.
• Operations. Hosted gateways run themselves, self-hosted open source gives you full control and the pager duty that comes with it.
• Staying power. After a year of acquisitions and archived projects, who owns the product and whether it is actively developed belongs on the checklist.

Quick picks

• Best for production teams in Europe: Opper, 500+ models, text, image, voice and video, on one OpenAI-compatible API with an EU-hosted control plane
• Best for API-platform enterprises: Kong AI Gateway, governance-grade controls on Kong infrastructure
• Best self-hosted: LiteLLM, the MIT-licensed default, with Envoy AI Gateway as the Kubernetes-native alternative that hit 1.0 this year
• Best free tier: Cloudflare AI Gateway, core features cost nothing
• Best inside a hyperscaler: Azure API Management, token governance and a cross-provider unified model API in your own tenant
• Biggest ecosystem: OpenRouter, 400+ models and the largest developer community

The best AI gateways compared

The market moved again in 2026

The consolidation we tracked in the LLM gateways ranking kept going. Palo Alto Networks completed its acquisition of Portkey in May, making it the core AI gateway of Prisma AIRS. Envoy AI Gateway reached v1.0 in June with Bloomberg, Nutanix and Tencent Cloud cited as adopters, giving open-source infrastructure teams a stability contract. Solo.io's gateway line became agentgateway under the Linux Foundation, backed by Microsoft, T-Mobile and Dell. On the hyperscaler side, OpenAI's frontier models went live on AWS Bedrock in July at OpenAI's direct rates, removing a long-standing objection to the Bedrock route. And OpenRouter remains reportedly in acquisition talks with Stripe at around $10 billion, still unannounced as of this writing.

1. Opper

Opper is the AI gateway on this list built for teams that need European hosting without giving up model choice, and one of the few that treats AI gateway as more than text: 500+ models spanning text, image, voice and video route through one API, with drop-in compatible endpoints for the OpenAI, Anthropic and Gemini SDKs. Fallbacks are built in, and the same model can be wired across multiple upstreams, for example Claude Sonnet on Anthropic direct plus AWS Bedrock, with automatic retry typically completing in about 180ms.

Governance is where it leans in: every call is metered with cost, latency and token accounting included, structured outputs are built into the API, and the base gateway stores no prompts or responses, only metadata. Full tracing, with prompts, responses and evaluation scoring, comes with the Control Plane plan, which raises the credit-purchase fee from 3% to 5.5%. The residency story is unmatched in this table: an EU-hosted control plane, traces and logs in EU storage, and requests pinnable to EU regions on AWS Bedrock, Azure, GCP or Berget AI, with a DPA and published sub-processor list. Pricing is otherwise pay-as-you-go with no markup on inference, BYOK is free, and uptime is published on a public status page.

One honest con: self-hosting is available on enterprise plans only, so smaller teams run on the hosted gateway. For the developer-routing view of the same product, see the LLM gateways ranking.

2. Kong AI Gateway

Kong AI Gateway is the reason this category stopped being a startup niche: it brings API-gateway-grade governance to model traffic, with multi-provider routing, PII sanitization, semantic caching, prompt guards and MCP governance on Kong Gateway and Konnect. Its architecture is a genuine residency advantage, since data planes, the part that sees your prompts, run in your own infrastructure while Konnect control planes are available in six geos including the EU.

Pricing is the friction: the affordable Konnect Plus tier caps AI Gateway at five unique LLM models with one million API requests a month included and $200 per additional million, and removing the cap means a custom-priced enterprise contract. For organizations already running Kong it is the natural choice; for everyone else it is a platform decision, not a gateway decision.

3. LiteLLM

LiteLLM remains the open-source default, MIT-licensed with 140+ provider integrations, virtual keys, budgets, load balancing and guardrails in the free core, and self-hosting means residency is simply your own infrastructure. Its standing is best measured by who leans on it: it appears in essentially every credible roundup, and AWS's own multi-provider gateway guidance literally deploys LiteLLM as the reference architecture.

The costs are operational, you run, scale and patch it yourself, and the enterprise tier with SSO, RBAC and audit logs has no public pricing. Teams that want its flexibility without the pager duty can compare the hosted route in Opper vs LiteLLM.

4. Portkey

Portkey pairs an open-source gateway core with a strong observability suite: a free tier with 10,000 logged requests a month, then a $49/month Production tier with 100,000 logs and 30-day retention, up to enterprise deals with VPC and private-cloud deployment. Since May it is part of Palo Alto Networks, positioned as the core AI gateway of the Prisma AIRS security platform.

The standalone product remains on sale, and what the acquisition means for its independent roadmap is not stated either way. Note that Portkey is observability-first, logging requests is the core function, with retention set by tier, so weigh that against your data-handling requirements.

5. Envoy AI Gateway

Envoy AI Gateway is the vendor-neutral open-source pick for platform teams on Kubernetes: Apache-2.0, built on CNCF's Envoy Gateway, routing to 16 providers including OpenAI, Anthropic, Azure, Bedrock, Gemini and self-hosted models through one OpenAI-compatible API, plus a full MCP gateway. Its v1.0 release in June 2026 declared the control-plane API stable, added multimodal support, and cited Bloomberg, Nutanix and Tencent Cloud as adopters, which is about as production-blessed as open source gets.

The trade-offs are structural: it is Kubernetes-only, there is no managed option (Tetrate sells a hosted router service on top), and cost dashboards and billing are yours to assemble. For infra teams that already run Envoy, it is the obvious candidate.

6. Azure API Management

Azure's AI gateway is a capability set of API Management, not a separate product, and it is more capable than most people assume: token-based rate limits and quotas, token metrics into Azure Monitor, semantic caching, load balancing with circuit breakers, Azure AI Content Safety integration, MCP server export, and a new unified model API in preview that fronts OpenAI, Anthropic and Google Vertex schemas, with backends in Microsoft Foundry or external providers such as Amazon Bedrock. Everything runs in your own tenant and regions, which makes EU residency a configuration choice rather than a negotiation.

Costs follow standard APIM tiers, with a verified free allowance of up to 100,000 requests when created as an AI gateway through Azure AI Foundry. The friction is APIM itself: XML policy configuration, tier-gated features, and several AI capabilities still carrying preview labels.

7. Cloudflare AI Gateway

Cloudflare AI Gateway is the free option that earns its slot: analytics, caching and rate limiting cost nothing, it runs on Cloudflare's edge, and unified billing carries a 5% fee on credits purchased with no markup on provider rates. As a caching and control layer in front of providers you already pay, the price is unbeatable.

Go in with open eyes on data handling: logs including request and response data are enabled by default per gateway unless you opt out, and the docs state no retention period or storage geography, so there is no documented way to keep gateway logs in Europe.

8. Vercel AI Gateway

Vercel AI Gateway routes to hundreds of models with zero markup on tokens including BYOK, and its stated zero-data-retention policy, prompts and outputs deleted immediately after the request completes, is among the strongest retention postures of any hosted option. For teams already deploying on Vercel it is the path of least resistance.

The caveats from our LLM-gateway review stand: privacy features beyond the baseline are metered paid add-ons, with team-wide ZDR-only routing at $0.10 per thousand requests on Pro and Enterprise, and there is no published EU hosting option, a strong retention story without a location story.

9. OpenRouter

OpenRouter is the ecosystem heavyweight, 400+ models from 70+ providers, the largest developer community in the category, a $113M Series B in May, and reported acquisition talks with Stripe as of late July. Inference passes through at provider rates, with a 5.5% fee on credit purchases.

For an AI-gateway evaluation its governance surface is the thinner side of the story: privacy is a per-provider patchwork where retention varies by upstream, and EU processing is enterprise-only. We maintain the full head-to-head at Opper vs OpenRouter.

10. TrueFoundry

TrueFoundry targets regulated enterprises where deployment model is the first question: VPC, on-prem and air-gapped options are first-class, the platform spans serving and MLOps beyond the gateway, and it broadened further by acquiring Seldon AI in June. The free Developer tier covers 50,000 requests a month.

The entry price for production remains the sticking point, Pro starts at $499/month, an order of magnitude above Portkey, which makes sense for enterprise procurement and less sense for small teams.

Also worth knowing

A credible shortlist should at least glance at these. Zuplo bundles an AI gateway into an API-management SaaS from $25/month, the cheapest paid entry anywhere, but routes only four LLM providers. F5 AI Gateway brings a serious runtime-security stack (including the CalypsoAI guardrails F5 acquired) to your own datacenter, with no public pricing and no published provider catalog. Databricks Unity AI Gateway, formerly marketed as Mosaic AI Gateway and listed as Beta in the docs, governs external models from OpenAI, Anthropic, Bedrock and Vertex inside Databricks, compelling if you already live there, unusable if you don't. agentgateway is the Linux Foundation's agent-native gateway, Apache-2.0 and the most MCP-centric of the open options, still young. nexos.ai is a European entrant from Nord Security's founders with a workspace-plus-gateway combo and a €30M Series A, though its pricing and residency terms are not publicly verifiable. And AWS Bedrock is a model platform rather than a cross-cloud gateway, but with OpenAI models now live on it, a single-cloud Bedrock strategy covers more ground than it did last year. Helicone and Bifrost, strong in the developer-tooling lane, are ranked in the LLM gateways article.

If your prompts have to stay in Europe

The enterprise flavor of this SERP makes the residency question sharper, and the field splits cleanly. Self-hosted open source, LiteLLM and Envoy AI Gateway here, inherits whatever infrastructure you run. Azure API Management runs in your own tenant, so EU residency is a region setting. Kong keeps prompt-carrying data planes in your infrastructure with an EU control-plane geo. Among the fully hosted gateways, OpenRouter processes in the EU only on enterprise contracts, Vercel publishes a strong retention policy but no EU region, and Cloudflare documents neither retention period nor storage location for its default-on gateway logs.

Opper is built EU-first, the control plane runs in the EU, traces and logs stay in EU storage, and requests can be pinned to EU-hosted deployments on AWS Bedrock, Azure, GCP or Berget AI, with zero retention available on enterprise plans. The full compliance picture is at /ai-compliance.

Can it route more than text?

Every roundup we analyzed treats AI gateway as text-model traffic, yet the workloads arriving now are not text-only. If your roadmap includes image generation, voice or video, ask the multimodal question early: Opper routes text, image, voice and video models through the same OpenAI-compatible API, and Envoy AI Gateway added multimodal support with its 1.0 release. For the rest of this list, multimodal coverage is worth confirming with the vendor before you commit, because bolting a second gateway onto your stack later defeats the point of having one.

AI gateway FAQ

Try the European pick

If your shortlist ends at "we already run Kong or Azure" you have your answer, and if it ends at "we operate everything ourselves" it is LiteLLM or Envoy AI Gateway. If it ends at production traffic with European hosting, multimodal routing and no markup on inference, create a free Opper account and make your first call in minutes, or browse the 500+ models it routes to first.