About the AI Roundtable

By Felix Wunderlich

How it started

In early February, a simple question was making the rounds online: "I want to wash my car. The car wash is 50 meters away. Should I walk or drive?". People were copy-pasting it into ChatGPT, Grok, Claude, Gemini, sharing screenshots of mostly wrong answers. It was fun to watch, but there was no methodology to it. Just one-offs, potentially polluted with user memories and settings. Also open-weight models were missing from the conversation entirely. And it turned out people who were testing the same thing were getting different results.

I wanted to do this properly, so I tested 53 models, 10 runs each, through Opper's AI gateway and also added a 10k human control group for baseline (thanks Rapidata). We published it as the Car Wash Test. It went kinda viral. Landed on the Hacker News front page, became post of the day on multiple subreddits, got picked up by newspapers, and I started hearing my name on podcasts covering my test. What stood out most were the discussions. Everyone was weighing in (400+ comments on HN), debating results, and it was clear people care about how different models reason about the same question.

Why an AI Roundtable

There are so many open questions around AI models. How they reason, where they refuse, what biases they carry. Elon is posting about these daily on X. So I felt like we needed a place to put them to the test, and that's how the AI Roundtable was born. I teamed up with José Sabater, who ran the Car Wash Test with me, and we built it. You ask a question, define options to choose from, pick from 200+ AI models, and they all answer independently. You can also have them debate each other's reasoning in a Debate round and give them a chance to change their minds.

But beyond comparing answers, having an expert panel of the strongest AI models debate a question together is where it gets interesting. When five frontier models agree on something, that's a stronger signal than any single answer. And when they disagree, their reasoning often surfaces assumptions you hadn't even considered. It's like having a room full of the smartest people in the world actually think through your question.

Under the hood

The budget counter in the navbar is what we've allocated for API calls, and we happily add more whenever we see people enjoying the tool. You can also bring your own Opper API key to use your own credits, your sessions are unlisted by default and fully visible in your Opper dashboard.

There are two modes. In a Poll, every model answers the question independently with no system prompt and no mention of other models. They don't know they're part of a roundtable. Each picks an option and gives their reasoning. That's it: one round, no influence between models. This is useful when you want to see how different models think about the same question.

In a Debate, the first round works the same way. But if models disagree, a second round begins where every model sees what all the others said: votes, reasoning, model names. They can change their mind, call out flawed arguments, or double down. If everyone already agreed in the first round, there's nothing to debate and the result is shown immediately.

Once voting is complete, a reviewer model analyzes the full transcript and generates a Final Summary that captures the key arguments, where models agreed, and what changed their minds. All models are called in parallel each round via the Opper AI API, so rounds complete as fast as the slowest model. Responses use structured JSON output to ensure a consistent vote-plus-reasoning format across all providers.

What's next

Beyond the public app, we've been building multi-model consensus for real tasks: tool calling, complex reasoning, structured extraction. Think a single API endpoint where your hardest calls, the ones that fail too often with one model, get resolved by a roundtable of models and returned as one clean decision by the reviewer.

We have a research preview available. If you're interested in access, or have any questions or feedback, email us at ai-roundtable@opper.ai.