LLM Router

Send every request to the right AI model

Routing rules, an LLM classifier, and fallback chains decide which of 700+ models serves each request, with compliance enforced before it reaches a provider.

Get startedBrowse models
simplefallbackcomplexfallback
Start
dynamic / support
Classifier
Classifier
2 outputs
Model
GPT OSS 120BCerebras
Model
Gemini 3 FlashGoogle
Model
Claude Opus 5Model routes
Model
GPT-5.4Model routes
End

Click a model to swap it. Your code keeps calling dynamic/support.

Trusted by 50k+ developers and companies serving 10M+ users

Aixia
evroc
GetTested
Instabridge
LexBox
Ping Payments
Steep
Svenska Bostäder

Challenge

Why is model routing still hardcoded?

Direct integrations pin every feature to one model and one provider, leave failover to hand-rolled retry loops, and give compliance teams no enforcement point in the request path.

Hardcoded model choice

Model names live in application code, so every model change is a pull request and a redeploy instead of a config change.

No failover path

A single rate limit or provider outage becomes your outage, and without an ordered fallback, retry logic ends up scattered through application code.

One model for every request

Simple requests pay flagship prices and hard requests get whatever was hardcoded, because nothing in the stack can weigh cost, latency, or quality per request.

Unenforceable routing policies

Residency and retention requirements live in documents instead of the request path, so nothing technically stops a request from reaching a route your compliance team never approved.

The Opper Way

How the router decides

Explicit mechanisms decide where each request goes, and the results are measured in public

Fallback chains with stable names

An alias maps a stable name to an ordered list of models, a primary plus fallbacks. Your code calls the name, the gateway tries the list in order on timeouts and retriable errors, failing over in roughly 180ms, and you can reorder or repoint the chain at any time without an integration change.

  • Failover in roughly 180ms
  • Repoint without code changes
Read how aliases and fallback chains work
production-mainalias · 3 models
anthropic/claude-sonnet-4-6
Primary
aws/claude-sonnet-4-6-eu
Fallback 1
openai/gpt-5.5
Fallback 2
Tried in order on timeouts, 429 and 529 · failover in roughly 180ms

Routing rules instead of hardcoded names

Route picks the model when a caller does not pass one, with rules at organization and project scope where the most specific rule wins. Set an explicit default, or let Auto choose from preference hints like fast, cheap, quality, or balanced. If a caller requests a model that policy does not permit, the call fails with a clear error, never a silent fallback.

  • Most specific rule wins
  • Every decision on the trace
See how Route works
Organization defaultorg
Auto · prefer: balanced
Project: support-agentmost specific · wins
anthropic/claude-sonnet-4-6
Model blocked by policyclear error
The call fails with a clear error, never a silent fallback
Every decision shows as a Route event on the trace

Compliance policies in the request path

Comply restricts where requests may go: EU-only providers, specific regions or countries, or zero-retention providers only. Anything that breaks a rule is rejected at the gateway before it reaches a provider, so residency requirements are enforced rather than documented.

  • Rejected before reaching a provider
  • Region and retention aware
See AI security and compliance
Routing policy
ProvidersEU-only
Retentionzero retention
aws/claude-sonnet-4-6-eu
Routed
Route outside policy
Rejected at the gateway
Rejected before the request reaches a provider

One name, many providers

Some models are served by more than one provider, and Opper balances across them so a single name keeps working even when one provider is busy. The effect is measurable, our uptime page compares 30-day any-route availability against what a single route to the same model averaged.

  • Balancing built in
  • Measured over 30 days in public
See measured availability
claude-sonnet-4-6one name · 3 routes
Anthropic
529 overloaded
AWS Bedrock
Serving
Azure
Standby
The name keeps answering while a single route is busy

Router overhead, measured

We benchmarked the router tax instead of asserting it away, 200 calls per destination on GPT-4.1 with 95% confidence intervals. Opper matched OpenAI direct on time to first token and throughput within the intervals, and the spread that actually matters was elsewhere, an identical open model varied by roughly 10x in throughput across serving backends.

  • Matches direct within confidence intervals
  • Scripts published for reproduction
Read the latency benchmark
Time to first token, GPT-4.1
OpenAI direct0.712s · 81.8 tok/s
OpenRouter0.640s · 73.2 tok/s
Opper0.744s · 81.5 tok/s
200 calls per destination, 95% confidence intervals. Opper matches direct within the intervals.

Dynamic Routes

Draw a route, deploy it as a model

Design routing on a canvas, then call it like any other model

Routing you can sketch

Dynamic Routes is a canvas for routing, a route is a graph with a start node, the models it passes through, and an end node, and an LLM classifier node can branch a route based on the prompt. Give it a name and a description, save it as a draft, and deploy it when it is ready. A deployed route is callable like any other model under its own id, dynamic/smartroute for a route named smartroute, and a View code panel shows the call to make.

  • Save drafts, deploy when ready
  • Callable like any other model
smartroute
Save draftDeploy
Start
GPT OSS 120BCerebras
End
Deployed routes are callable like any model
model: "dynamic/smartroute"

Model catalog & pricing

Access the latest AI models from leading providers with unified pricing per million tokens.

All prices are per 1M tokens • EU and US regions available • Prices subject to change, see docs for latest

ProviderModelRegionInput (1M tokens)Output (1M tokens)
Loading models...

Custom Models & BYOK

Bring your own API keys or add custom model deployments using the Opper CLI or API.

opper models create example/my-gpt5 azure/gpt5-production YOUR_API_KEY

Looking for a specific model? View complete model list →

FAQ

LLM router FAQ

What is an LLM router?

+
An LLM router is the layer between your application and model providers that decides which model, and which provider behind it, serves each request. Routing decisions can weigh cost, latency, quality, availability, region, or task type, so applications stop hardcoding a single model. Opper routes across 700+ models with explicit primitives, fallback chains, routing rules, and compliance policies.

What is the difference between an LLM router and an LLM gateway?

+
The router is the decision, which model serves this request, and the gateway is the broader layer around it that adds authentication, observability, and an audit trail. On Opper the router lives inside the LLM gateway, so a routing decision, its fallbacks, and its trace all share one API.

What is a semantic router?

+
The term covers two different things. The original semantic-router library routes requests to intents and tools rather than models, it embeds example utterances and picks the closest route by vector similarity without waiting for an LLM. The vLLM semantic-router project applies the idea to model selection instead, a programmable routing layer that picks a model path per request to improve quality, cost, and latency. On Opper you get both styles, rules, preference hints, and fallback chains for explicit routing, and a classifier node that branches on the prompt itself, so every decision stays inspectable.

What is a model router?

+
A model router picks a suitable model for each prompt, usually to save cost on easy requests while keeping quality on hard ones. Microsoft Foundry ships one as a product, a trained language model you deploy that routes each prompt to the most suitable underlying model in real time. The alternative is explicit rules and fallback chains, which trade some automation for decisions you can read, test, and audit, and that is the approach Opper takes, with an Auto option that picks a model from preference hints like fast, cheap, quality, or balanced.

Does an LLM router add latency?

+
Measurably little. In our 2026 benchmark of 200 calls per destination on GPT-4.1, Opper matched OpenAI direct on both time to first token and throughput within confidence intervals, and OpenRouter was actually 70ms faster than OpenAI direct on time to first token. The bigger factor is where a request lands, an identical open model varied by roughly 10x in throughput across serving backends, so routing destination matters far more than router overhead.

How do fallback chains work?

+
A fallback chain maps a stable name to an ordered list of models, a primary plus fallbacks. Your application calls the name, and when the primary times out or returns a retriable error such as a 429 rate limit or 529 overload, the gateway retries the next model in roughly 180ms. You can reorder or repoint the chain at any time without touching application code. Models served by more than one provider are pooled automatically, so a single name keeps working when one provider is busy, and measured any-route availability is published on our uptime page.

Can an LLM router enforce region or compliance rules?

+
Yes, on Opper this is the Comply layer. You can restrict routing to EU-only providers, to specific regions or countries, or to zero-retention providers, and anything that breaks a rule is rejected at the gateway before it reaches a provider. That turns residency requirements into routing policy instead of a code review checklist.

Do I need a trained router model, or are rules enough?

+
Learned routers are real, RouteLLM reports up to 85% cost reduction while maintaining 95% GPT-4 performance by training a router between a strong and a cheap model, and they shine at high volume with stable traffic patterns. Most production teams still start with explicit rules and fallback chains because they are debuggable, auditable, and cheap to change, and that is the model Opper follows, explicit routing plus an Auto option driven by preference hints.

How is an LLM router different from a load balancer?

+
A load balancer spreads identical requests across identical backends. An LLM router makes two extra decisions, which model should answer, and which provider serving that model should get the request. Opper does both, some models are served by more than one provider and the gateway balances across them, so a single model name keeps working even when one provider is busy.

Which LLM router is best?

+
It depends on what you optimize for. OpenRouter is a popular hosted router with a large catalog, LiteLLM is the common self-hosted choice if you can run the infrastructure, and RouteLLM fits teams that want learned cost-quality routing. Opper is built for production teams that want European hosting and explicit routing, fallback chains, routing rules, and compliance policies across 700+ models, with no markup on token rates, and current fees are on our pricing page.
How Instabridge keeps its agent running in 190+ countries

Case Study

How Instabridge keeps its agent running in 190+ countries

Instabridge routes every model through Opper so its in-app agent never depends on a single provider, with automatic fallback when a model degrades and the freedom to swap models without an integration rewrite.

Explore

The gateway around the router

Agents

Route Claude Code and Codex

Benchmark

LLM router latency, measured

Ready to route across 700+ models?

Start with one API call and keep every routing decision a config change

Get startedBrowse models