By Göran Sandahl -

New OpenAI-compatible endpoint: Use Opper with OpenAI SDKs and frameworks

We're excited to announce that Opper now provides an OpenAI-compatible API endpoint, making it easier than ever to access many models and capabilities through a single API. This compatibility layer allows you to use Opper with any tool or library designed for OpenAI's API or SDKs (such as LangChain, Vercel AI SDK, DSPy, etc).

Introducing the OpenAI-compatible API endpoint

Our new compatibility endpoint (https://api.opper.ai/compat/openai) allows you to use Opper API as a drop-in replacement for OpenAI's API. This means you can leverage Opper's capabilities (50+ models, fallbacks, trace logging, load balancing and centralized billing) while maintaining your existing codebase and tooling. Whether you're using the OpenAI SDK directly or working with higher-level frameworks like LangChain, you can now easily get the benefits of Opper.

Here's a simple example using Deepseek-R1 from provider Groq through our OpenAI-compatible endpoint using the OpenAI SDK:

import { OpenAI } from "openai";

const client = new OpenAI({
    baseURL: "https://api.opper.ai/compat/openai", // Opper API
    apiKey: "-", // must not be blank
    defaultHeaders: { "x-opper-api-key": "OPPER_API_KEY" },
});

const completion = await client.chat.completions.create({
    model: "groq/deepseek-r1-distill-llama-70b", // Opper Model
    messages: [
        {
            role: "user",
            content: "What is the capital of France? Please reverse the name before answering.",
        },
    ],
});

console.log(completion.choices[0].message.content);

This yields:

<think>
Okay, so I need to figure out the capital of France and then reverse the name. Hmm, I know that the capital of France is Paris. Wait, is that right? I'm pretty sure it's Paris, but maybe I should double-check. Let me think... Yes, Paris is definitely the capital. Now, I need to reverse the name. So, Paris spelled backwards would be... let's see, P-A-R-I-S. Reversing the letters would be S-I-R-A-P. So, the reversed name is Sirap. That seems correct. I don't think I made any mistakes there. Paris is the capital, and reversing it gives Sirap. Yeah, that makes sense.
</think>

The capital of France is Paris. When reversed, the name becomes Sirap. 

Answer: Sirap

Since this is fundamentally a REST endpoint, you can use it with any framework that uses OpenAI's Python or TypeScript SDKs. Here are some examples:

LangChain

LangChain is a popular framework for experimenting with LLMs.

from langchain_openai import ChatOpenAI
from langchain.chains import LLMChain
from langchain.prompts import PromptTemplate
import os

# Initialize the LLM
llm = ChatOpenAI(
    base_url="https://api.opper.ai/compat/openai",
    api_key="-", # must not be blank
    default_headers={"x-opper-api-key": os.environ.get("OPPER_API_KEY")},
    model_name="openai/o3-mini",
)

# Create a prompt template
prompt = PromptTemplate(
    input_variables=["question"],
    template="Answer this question: {question}"
)

# Create and run the chain using pipe syntax
chain = prompt | llm

# Run the chain
response = chain.invoke({"question": "What is the capital of Sweden?"})
print(response.content)

# Output: The capital of Sweden is Stockholm.

Vercel AI SDK

The Vercel AI SDK provides a set of utilities for building AI-powered streaming text and chat UIs. Here's how to integrate Opper

mport { createOpenAICompatible } from '@ai-sdk/openai-compatible'
import { streamText, convertToCoreMessages } from 'ai'

export const maxDuration = 30

export async function POST(req: Request) {
  const opper = createOpenAICompatible({
    name: 'opper',
    baseURL: 'https://api.opper.ai/compat/openai', // Opper API
    apiKey: '-', // required but not used
    headers: {
      'x-opper-api-key': process.env.OPPER_API_KEY, // Opper API key
    },
  })

  const { messages } = await req.json()

  const result = streamText({
    model: opper('groq/deepseek-r1-distill-llama-70b'), // Opper Model

    messages: convertToCoreMessages(messages),
  })

  return result.toDataStreamResponse()
}

When to use Opper Structured API vs OpenAI-compatible API?

The Opper Structured API via call() is the recommended for a consistent and predictable experience for developers. It allows focusing on writing structured, maintainable, clean AI code with signficantly less reliance on model specific prompting.

The OpenAI-compatible API is a great way to get started with Opper if you want to manage your prompts fully, but still want to get some of the main benefits of a Unified API, such as:

  1. Go beyond just OpenAI models: Access 50+ models through a single API - switch between models with a single parameter change to find what works best for your use case
  2. Debug and Improve Model Behavior: Built-in logging, tracing, and evaluation tools help you understand model behavior and systematically improve output quality
  3. Manage Rate Limits: Load balancing, fallback across models and providers, and monitoring give you the control and reliability needed for production deployments
  4. Manage API keys across models: Manage API keys on a project level, and allow developers to use this across all models and providers.
  5. Reduce Model Dependencies: As new models emerge, instantly access them through the same API - no need to rewrite code or manage multiple integrations
  6. Cost Control: Unified billing and usage tracking across all models, with a path to optimizing costs with smaller models.

Getting Started

To start using the OpenAI-compatible endpoint:

  1. Sign up for an Opper API key at opper.ai - it's free up to $10/month.
  2. Replace the base URL in your OpenAI client with https://api.opper.ai/compat/openai
  3. Add your Opper API key in the headers as shown in the examples above

What's Next?

For more information on using Tracing, Fallbacks and other features in connection to the OpenAI-compatible API, see our OpenAI-compatible API documentation.

While we recommend using our structured API calls for the best experience, we believe the compatibility API can provide a great way for many to benefit from a model independent inference layer for better resillience, control and quality.

For detailed documentation and more examples, visit our OpenAI-compatible API documentation. If you have any questions or need support, join our Discord community.