Batch API vs Flex vs Standard: Half-Price LLM Tokens, Tested

The same model, with the same weights and the same answers, is sold at up to four prices. What changes is when you get the answer:

• Standard: the normal call. You get the answer in seconds, at list price.
• Flex: the same call, run on spare capacity at about half the price. It can be slow, and can be refused when the provider is busy.
• Batch: send thousands of requests as one job, at about half the price. Results arrive within 24 hours: in our test, anywhere from 3 minutes to 15 hours.
• Priority: faster tokens, at about twice the price.

We ran one job, triaging 100 support tickets, through every tier Opper offers on GPT-6 Luna, Claude Haiku 5.5 and Claude Opus 5, globally and in the EU, and sampled flex against standard every hour for an afternoon. Below: what each one cost, how long it took, whether the answers changed, and which one to pick.

The short answer

Pick standard when a person is waiting. Pick flex for background work that needs each answer in code, inside one request, and can live with a retry. Pick batch for large jobs you can hand off and collect later. Pick priority when latency is the product.

The test: 100 support tickets, every tier

The job is ordinary. 100 customer support messages (billing complaints, bug reports, angry and confused ones) each go to a model with the same system prompt: return JSON with a category, an urgency, a sentiment and a 12-word summary. Every tier got the same 100 messages and the same prompt.

Synchronous tiers ran 10 requests at a time. Batch tiers ran as one 100-request job through Opper's Batch API. Costs are what Opper billed, read from each response and from the batch object.

Global: GPT-6 Luna on OpenAI

Claude Haiku 5.5 on Anthropic

Anthropic has no flex tier. Its batch API runs in the US.

In the EU: Claude Opus 5 and gpt-oss-120b on AWS Bedrock

Every standard, priority and batch request succeeded, and every response was billed at the tier it reported. Flex is the one tier that turned requests away, and only on OpenAI.

What the numbers say

The discount is real and exact. Flex and batch came in at half the standard price, to within the few extra tokens a run happens to produce. Priority cost twice as much. A refused flex request costs nothing.

The answers don't change. Run the same model twice at the standard tier and a few borderline tickets flip category: GPT-6 Luna agreed with its own earlier standard run on 96 of 100 tickets, gpt-oss-120b on 91. Between tiers the difference is the same size: GPT-6 Luna at priority matched its standard answers on 97 of 100, at batch on 96, and at flex on 80 of the 83 it answered. Claude Haiku 5.5 batch matched standard on 95 of 98, and Claude Opus 5 batch on all 89 tickets where both runs returned a category. A tier changes when the tokens are computed, not which model computes them.

Flex depends on the hour, and on the model. We sent the same 100 tickets to flex and standard every hour for an afternoon (UTC, on a weekday):

Times are the median per answered request. GPT-6 Luna on flex answered between 2 and 91 of 100, and the requests it did answer took 10 to 12 seconds against 1.5 on standard: OpenAI turned them away with a 429 (flex_unavailable), Opper retried, and many still failed. gpt-oss-120b on Bedrock flex in the EU answered all 100 every hour, as fast as standard. Earlier the same day, GPT-5.4 mini on OpenAI flex also answered 100 of 100 in two runs, as fast as standard. Spare capacity belongs to one model on one provider at one moment, so measure the model you plan to use, and never put flex in a path without a fallback.

Batch took 3 minutes to 15 hours. Anthropic returned Claude Haiku 5.5 in 3 minutes 25 seconds and Bedrock returned Claude Opus 5 in under 10 minutes. OpenAI took 15 hours for GPT-6 Luna, the same afternoon it ran GPT-5.4 mini in 3 minutes. Every job finished inside the 24-hour promise. Build for the promise, not for the fast case.

Priority bought little here. Short prompts and short answers leave little to speed up: priority cut the median from 1.6 to 1.1 seconds for twice the price. It pays on long outputs and in busy hours, when standard slows down.

Flex vs batch: same price, different contract

On OpenAI, flex is priced at batch rates, so the choice between them isn't price. It's how your code receives the answer.

Flex is a normal API call. Your code sends a request and waits for the answer in the same request, the way it already does. Nothing changes but one field and a longer timeout. That fits agents and pipelines where step 2 needs step 1's answer: evaluations, enrichment, nightly reports, a background agent. The cost is uncertainty: as the hourly test shows, a request can be slow or fail with a 429 when the provider is out of flex capacity, so pair it with a fallback.

Batch is a job. You upload every request at once, get an id back, and collect the results when the job ends. It never refuses for capacity, it just takes longer, sometimes many hours. That fits work you can describe up front: classify 50,000 tickets, embed a document archive, re-score last month's conversations with a new prompt. It doesn't fit anything conversational.

Two practical differences:

• Availability. Flex exists where the provider sells it: OpenAI, Gemini, Vertex AI, AWS Bedrock (including in the EU), DeepInfra. Batch exists for OpenAI, Anthropic, Azure, DeepInfra, and, since November 2026, AWS Bedrock in the EU.
• Size. Bedrock batch jobs need at least 100 requests, and one Opper batch holds up to 50,000. Flex has no minimum.

Bedrock batch inference in the EU (Stockholm)

Claude Opus 5 and Claude Haiku 4.5 batch jobs now run on AWS Bedrock in Stockholm (eu-north-1), at half the EU on-demand price. You send the same POST /v3/batches request, with no AWS account, S3 bucket or IAM role of your own. Opper stages the files in its Stockholm bucket, runs the job and deletes them.

Bedrock doesn't offer batch for Claude Opus 5.5, Sonnet 5 or Haiku 5.5 yet, so those run their batches in the US through Anthropic. As of November 2026, we know of no other hosted AI gateway that runs Bedrock batch in the EU without your own AWS account.

To keep a batch in the EU, name the EU batch endpoint, or set an EU inference rule for your organization so nothing else is allowed:

{
  "endpoint": "/v1/messages",
  "model": "bedrock-batch/claude-opus-5",
  "requests": [ ... ]
}

For synchronous calls at a discount in the EU, flex runs on Bedrock for gpt-oss-120b, gpt-oss-20b, Amazon Nova 2 Lite and Nova Pro, and priority runs on Bedrock, Vertex AI (Gemini) and xAI (Grok). Every EU route is listed with its price.

Zero data retention: batch can't have it

Zero data retention (ZDR) means the provider keeps nothing once your request is answered. A synchronous call can work like that. A batch can't: the provider has to hold your requests until it runs them and the results until you collect them, sometimes for hours. That is why OpenAI's and Anthropic's batch APIs are outside their zero data retention programs.

What that means through Opper:

• Flex and priority carry the same data-handling terms as the route they run on. If your rules allow a route, they allow its tiers.
• Every batch route is marked as retaining data. An organization with a zero data retention rule gets a refusal when it creates a batch, not a surprise later.
• Opper keeps batch data short. Requests are staged until the provider accepts the job, then deleted. Opper deletes the provider's copy as soon as it collects the results. Results stay for your project's retention period, at most 30 days (24 hours if your project's retention is zero), or until you delete the batch.
• On Bedrock in the EU, job files sit in Opper's S3 bucket in Stockholm, are deleted once the results are collected, and any file left behind expires after 7 days. AWS processes the job in EU regions through its EU inference profiles.

If your data can't be at rest anywhere, use standard or flex on a zero data retention route, and set a zero data retention rule so a batch can't slip through.

How to use each tier

All four go through the same API key. Standard, flex and priority are normal calls on the OpenAI-compatible endpoint:

from openai import OpenAI

client = OpenAI(base_url="https://api.opper.ai/v3/compat", api_key="YOUR_OPPER_API_KEY")

resp = client.chat.completions.create(
    model="gpt-6-luna",
    service_tier="flex",          # or "priority", or leave it out for standard
    messages=[{"role": "user", "content": "Triage this ticket: the app logs me out every hour."}],
    timeout=120,                  # flex can be slow
)
print(resp.service_tier, resp.choices[0].message.content)

Flex with a fallback to standard. When flex is refused, Opper retries once and then moves to the next model, so a busy hour costs you the full price instead of a failed request:

{
  "model": "openai:flex/gpt-6-luna",
  "models": ["openai/gpt-6-luna"],
  "messages": [ ... ]
}

Batch is one job. Create it, poll it (or set a callback_url and get a webhook), read the results:

import os, time, requests

BASE = "https://api.opper.ai/v3"
H = {"Authorization": f"Bearer {os.environ['OPPER_API_KEY']}"}
tickets = ["My invoice is wrong", "The app crashes on login", ...]

batch = requests.post(f"{BASE}/batches", headers=H, json={
    "endpoint": "/v1/chat/completions",
    "model": "openai-batch/gpt-6-luna",
    "requests": [
        {"custom_id": f"ticket-{i}", "body": {"messages": [{"role": "user", "content": f"Triage: {t}"}]}}
        for i, t in enumerate(tickets)
    ],
}).json()

while batch["status"] not in ("completed", "expired", "cancelled", "failed"):
    time.sleep(30)
    batch = requests.get(f"{BASE}/batches/{batch['id']}", headers=H).json()

for line in batch.get("results", []):
    print(line["custom_id"], line["error"] or line["response"]["body"]["choices"][0]["message"]["content"])

The request body is OpenRouter's batch format, so code written for OpenRouter's Batch API only needs a new base URL and key. For a Claude batch, set "endpoint": "/v1/messages" and send Anthropic Messages bodies.

Find the models for each tier

Which models sell which tier changes as providers add them. The catalog is the source of truth:

# Batch endpoints, with region and price
curl -s "https://api.opper.ai/v3/models?service_tier=batch&limit=0" \
  | jq -r '.models[] | "\(.id)\t\(.region)\tin \(.pricing.input[0])\tout \(.pricing.output[0])"'

# Flex endpoints that run in the EU
curl -s "https://api.opper.ai/v3/models?service_tier=flex&inference_location=EU&limit=0"

Flex and priority in more detail, per provider: Run models at half the price with flex and priority tiers.

Common questions

What is the difference between batch and flex processing?

Both cost about half the standard price. Flex is a normal synchronous API call with service_tier set to flex: you wait for the answer in the same request, it can be slow and can be refused when the provider has no spare capacity (in Opper's test, GPT-6 Luna flex answered between 2 and 91 of 100 requests depending on the hour). Batch is an asynchronous job: you submit many requests at once and collect the results when the job ends, within 24 hours, and it is never refused for capacity.

Do batch and flex give the same answers as standard?

Yes. The tier changes when the tokens are computed, not the model. In Opper's test of 100 support tickets, GPT-6 Luna at priority and batch matched its own standard answers on 97 and 96 of 100 tickets, the same as the run-to-run variation of the standard tier itself (96), and Claude Haiku 5.5 and Claude Opus 5 batch matched standard on 95 of 98 and 89 of 89.

Can I run batch inference in the EU?

Yes. Through Opper, Claude Opus 5 and Claude Haiku 4.5 batch jobs run on AWS Bedrock in Stockholm (eu-north-1) through Bedrock's EU cross-region inference, at half the EU on-demand price, with no AWS account, S3 bucket or IAM role on your side. Bedrock batch jobs need at least 100 requests. More EU batch models are coming soon.

Is batch processing zero data retention?

No. A batch provider has to store your requests until it runs them and the results until you collect them, so OpenAI's and Anthropic's batch APIs are outside their zero data retention programs. Opper marks every batch route as retaining data, so an organization with a zero data retention rule is refused when it creates a batch. Flex and priority keep the data-handling terms of the route they run on.

Get started

• Batch API docs →: request format, webhooks, limits and billing.
• Service tiers docs →: flex, priority, fallbacks and timeouts.
• Opper platform →: get a key and run your first batch.