OpenAI-compatible API endpoint: one base URL for 700+ models (2026 guide)

An OpenAI-compatible API is any endpoint that accepts the same requests and returns the same responses as OpenAI's own, so code written for the OpenAI SDK runs against it after changing the base URL, the API key and the model name. The format has become the default: DeepSeek, Groq, xAI, Mistral and Google all serve one, and so do inference servers like vLLM and Ollama.

What does OpenAI-compatible mean in practice?

The contract is OpenAI's Chat Completions request and response shape: a model, a list of messages, optional tools, response_format and stream, and a response with choices and usage. Every OpenAI-compatible endpoint accepts that shape, but how much of it is honored varies: Anthropic's layer ignores response_format, for example, and Ollama documents a subset of the API. Newer surfaces such as the Responses API are the other gap, since many providers don't serve them at all.

Most providers publish their own OpenAI-compatible base URL for the models they serve, which is useful when you only ever call one of them:

Some come with caveats from their own makers: Anthropic positions its layer for testing and comparing models rather than production, and Google still calls its OpenAI support beta. Each is also a separate account, key, bill and set of quirks.

Opper's LLM gateway, run from Stockholm, puts one OpenAI-compatible endpoint in front of all of them. A model is one string such as deepseek-v4-pro, served by every provider that hosts it, so switching models means changing that string, a failing provider falls back to the next, and usage from every model lands on one bill with no markup on inference.

Quickstart: the OpenAI SDK pointed at Opper

Three things change compared with calling OpenAI directly: the base URL, the API key, and the model id. Create a key in your Opper account, export it as OPPER_API_KEY, and run any of these. The model here is a free one, so the call works before you add a card:

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.opper.ai/v3/compat",
    api_key=os.environ["OPPER_API_KEY"],
)

response = client.chat.completions.create(
    model="gemini/gemma-4-31b",
    messages=[{"role": "user", "content": "What is the capital of France?"}],
)
print(response.choices[0].message.content)

The model string comes in two forms. A bare name such as claude-sonnet-5 is a pool: Opper picks one of the providers serving that model and fails over to the others when one has trouble. A provider/model id such as anthropic/claude-sonnet-5 pins exactly one provider, for when you need the same deployment every time. The quickstart pins gemini/gemma-4-31b because that route is free. The model directory lists every model with its price, context window and hosting region. To move the same code to another model, change the string:

# A pool: Opper picks a provider and fails over to the others
model="claude-sonnet-5"
model="gpt-5.5"
model="gemini-3.6-flash"
model="deepseek-v4-pro"
model="gpt-oss-120b"

# A pin: exactly one provider
model="anthropic/claude-sonnet-5"
model="xai/grok-4.7"
model="mistral/mistral-large-2512"

What works through the endpoint

The core of the OpenAI API carries over, including the parts that tend to break on thinner compatibility layers:

Opper adds a few things the OpenAI SDK can pass as is. A pool already fails over between providers. To fall back to other models too, add a models array next to model, up to ten, tried in order when a provider errors, rate-limits or times out. An X-Opper-Name request header gives the call a function name that its traces carry. On the response, X-Opper-Served-Model says which model answered and, on non-streaming calls, X-Opper-Cost what it cost, which the OpenAI SDK reads through with_raw_response. Here are fallbacks and naming together with structured output:

import os
from openai import OpenAI
from pydantic import BaseModel

client = OpenAI(
    base_url="https://api.opper.ai/v3/compat",
    api_key=os.environ["OPPER_API_KEY"],
    default_headers={"X-Opper-Name": "extract_city"},
)

class City(BaseModel):
    name: str
    country: str
    population: int

response = client.chat.completions.parse(
    model="claude-sonnet-5",
    messages=[{"role": "user", "content": "Tell me about the capital of Sweden."}],
    response_format=City,
    extra_body={"models": ["gpt-5.5", "gemini-3.6-flash"]},
)
print(response.choices[0].message.parsed)

LangChain, the Vercel AI SDK and other frameworks

Any framework that wraps the OpenAI SDK, or lets you set an OpenAI-compatible base URL, connects the same way.

LangChain

ChatOpenAI from the langchain-openai package takes the base URL and key directly:

import os
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    base_url="https://api.opper.ai/v3/compat",
    api_key=os.environ["OPPER_API_KEY"],
    model="claude-sonnet-5",
)

print(llm.invoke("What is the capital of Sweden?").content)

Vercel AI SDK

The AI SDK's @ai-sdk/openai-compatible provider is built for exactly this: createOpenAICompatible with a name, the base URL and the key, then any model id from the catalog.

import { createOpenAICompatible } from "@ai-sdk/openai-compatible";
import { generateText } from "ai";

const opper = createOpenAICompatible({
  name: "opper",
  baseURL: "https://api.opper.ai/v3/compat",
  apiKey: process.env.OPPER_API_KEY,
});

const { text } = await generateText({
  model: opper("claude-sonnet-5"),
  prompt: "What is the capital of Sweden?",
});
console.log(text);

streamText works the same way for chat UIs. Opper also publishes a dedicated provider, @opperai/ai-sdk-provider, built on the same package, and the Vercel AI SDK docs page covers it.

Pydantic AI

Pydantic AI's OpenAIProvider takes the same base URL and key, and the agent then runs on any catalog model:

import os
from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.openai import OpenAIProvider

model = OpenAIChatModel(
    "claude-sonnet-5",
    provider=OpenAIProvider(
        base_url="https://api.opper.ai/v3/compat",
        api_key=os.environ["OPPER_API_KEY"],
    ),
)
agent = Agent(model)
print(agent.run_sync("What is the capital of Sweden?").output)

Other frameworks and tools

Mastra and LiteLLM have their own setup pages in the docs. Anything else that accepts a custom OpenAI base URL, from the OpenAI Agents SDK to chat apps with a "custom OpenAI-compatible provider" field, takes the same three settings: base URL, Opper key, model id. For coding agents there is a shorter path: the Opper CLI launches Claude Code, Codex or OpenCode with their inference already routed, and our guides cover the manual setup for Codex and Claude Code.

Beyond the OpenAI SDK: Anthropic and Gemini on the same base URL

The same base URL works for the Anthropic and Google Gen AI SDKs, since each appends its own paths, /v1/messages for Anthropic and the v1beta paths for Google, and both reach the same 700+ models with the same key. That means a Claude-first codebase can call GPT or Gemini without switching SDKs, and the reverse:

import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic({
  baseURL: "https://api.opper.ai/v3/compat",
  apiKey: process.env.OPPER_API_KEY,
});

const message = await client.messages.create({
  model: "gpt-5.5",
  max_tokens: 1024,
  messages: [{ role: "user", content: "What is the capital of Sweden?" }],
});
console.log(message.content);

The drop-in SDK docs cover each of them, plus the OpenResponses endpoint.

Keep traffic in the EU

Opper runs in Stockholm, but pools mix regions: depending on which provider picks up the call, claude-sonnet-5 can run in the US or in the EU. When the data has to stay in Europe, there are two ways to make sure it does.

On any plan, call an EU route directly. aws/claude-sonnet-5 runs Claude Sonnet 5 on AWS Bedrock in the EU and azure/gpt-5.5 runs GPT-5.5 on Azure in Sweden, and opper.ai/models/eu lists every model that runs in Europe, with location, retention and price per route.

With Control Plane, you set it once instead of in every call. A model access rule with the EU as inference location applies to the whole organization, or to a single project: every pool thins to its EU providers, so claude-sonnet-5 keeps working and only EU routes serve it, while a call to a model with no EU route is refused with a 403 before it reaches a provider. Nobody has to remember the right ids. The model access docs cover the other fields, storage location and provider country among them, and our guide to EU data residency walks through the setup.

OpenAI-compatible API FAQ

Start with one key

Create an account to get a key and try the free models, browse the model directory for ids and prices, or read the Chat Completions reference for every parameter.