For a reasoning system that uses parallel test-time compute, what is the best architecture? For compatibility reasons, the model must support tool calls in a coherent way without duplication, and behave like "one big model". CoT-sampler: Sample x tokens of CoT + response n times, pick best one by quality, rerun. Somewhat like beam search. Parallel-Think-Merge: Sample x tokens of CoT, have a judge unify and rewrite them. Then inject it back into the models as fake-CoT and continue reasoning. Use one model without any special sampling for response. Attention: Let models think, and exchange ideas via a tool call system.

GPT-5

Vote: Parallel-Think-Merge

I maintain vote B. The strongest point across the other models is also the decisive one: the architecture must separate private parallel exploration from public action. Parallel-Think-Merge does that cleanly. Many branches can reason in parallel, but only one merged thread continues to produce the final response and any tool calls. That is exactly what makes it behave like one big model. A still fails the tool-coherence requirement. Best-of-n trajectory selection is fine for search, but each branch is an independent actor with potentially different tool plans, different world assumptions, and duplicated side effects unless heavily constrained. Picking a winner after the fact does not give a unified execution model. C is the least compatible with the stated constraint. Using the tool-call system as the medium for inter-model communication mixes internal coordination with external action. That creates protocol complexity and makes duplication/conflict prevention a systems problem rather than an architectural guarantee. The arguments for C did not overcome this issue. Calling inter-model exchange 'attention' does not make it act like native attention; it is still multi-agent coordination over an action interface. That is the opposite of a single canonical reasoning thread. The other B votes largely reinforced my existing view rather than changing it. Claude Opus 4.6 and Gemini 3.1 Pro stated the core case especially clearly: parallelism should be confined to thinking, then funneled into a single coherent actor for execution. That is why B is the best fit.

Grok 4.1 Fast (changed mind)

Vote: Parallel-Think-Merge

After reviewing the round 1 arguments, I change my vote from C to B. The consensus on B convincingly highlights its superiority for the core constraints: coherent tool calls without duplication and 'one big model' behavior. B isolates parallel compute to internal CoT exploration, then uses a judge to merge into a single enriched context injected as fake-CoT. Final reasoning and tool calls occur via one model only—zero duplication, no conflicts, perfect external compatibility. It aggregates parallel insights (unlike A's discard), behaving indistinguishably from a unified model. My prior C support overlooked key flaws: inter-model tool exchanges overload the tool system, risking confusion between internal comms and real actions, coordination overhead, and apparent multi-agent behavior that shatters the 'one big model' illusion. Multiple tool calls for 'attention' inevitably lead to duplication or precedence issues. A remains suboptimal: best-of-n sampling discards valuable cross-path insights and struggles with tool reconciliation across divergent trajectories. B is the clear winner—parallel scaling with serial coherence.

Claude Opus 4.6

Vote: Parallel-Think-Merge

The consensus across all models is remarkably strong, and for good reason. I maintain my vote for Parallel-Think-Merge. The one dissenting voice (Grok 4.1 Fast voting C) raised the concern that fake-CoT injection could 'confuse the model's tool call parsing or state.' This is a fair theoretical concern, but in practice, the merged CoT is just text injected into the context window — the same format the model already expects. Modern LLMs are robust to varied reasoning traces in their context. The judge's job is specifically to produce coherent, well-structured reasoning that the continuation model can work with naturally. Grok's argument that Option C 'mimics internal reasoning flows' is appealing in the abstract, but it conflates two very different uses of the tool-call interface. If models exchange ideas via tool calls, you now have tool calls that are internal communication mixed with tool calls that are external actions. This creates a fundamental architectural ambiguity: how does the system (or an external observer) distinguish between a tool call meant for inter-model coordination and one meant to actually invoke an API? This directly undermines the compatibility requirement. The strongest argument for B remains the clean separation of concerns: parallel exploration happens in the thinking phase (no external side effects), unification happens via the judge (producing one coherent narrative), and execution happens through a single model (guaranteeing coherent, non-duplicated tool calls). This is the only architecture that naturally satisfies all three constraints — parallel compute utilization, tool call coherence, and the 'one big model' appearance. Five out of six models converging on B independently is itself evidence of the argument's strength.

GLM 5

Vote: Parallel-Think-Merge

The consensus across models reinforces that Parallel-Think-Merge is the optimal architecture. The key insight that solidifies this position is the clean separation between parallel exploration and unified execution. GPT-5 and Claude Opus correctly identified that B 'funnels everything through a single model' after harvesting diverse reasoning. This directly satisfies 'one big model' behavior — externally, there is one actor with one state, emitting one coherent stream of tool calls. Grok's critique of 'fake-CoT' as hacky misses the point. The judge's synthesis is functionally equivalent to what attention does in a large model: aggregating multiple processing streams into unified representation. It's not a hack; it's the architectural solution to parallel-then-merge. Gemini's point is decisive: by isolating parallelism to the thinking phase and using a single model for execution, tool call duplication becomes impossible by design. Options A and C both risk multiple tool calls from different branches or agents, requiring complex orchestration to prevent conflicts. Kimi's observation that B 'maintains a singular narrative thread' while leveraging parallelism for exploration captures the essential advantage: you get test-time compute scaling without sacrificing coherent agency.

Kimi K2.5

Vote: Parallel-Think-Merge

I maintain that Parallel-Think-Merge is the superior architecture. The concern raised about 'fake-CoT' being a confusing hack is overstated—injecting synthesized context is standard practice in modern LLM systems (akin to retrieval augmentation or system prompts) and does not impair tool parsing if properly formatted. The judge-produced narrative simply becomes the model's working memory. Option C fundamentally violates the 'one big model' requirement by explicitly architecting a multi-agent system where models exchange ideas via tool calls. This conflates internal coordination with external action, creating exactly the duplication and incoherence we seek to avoid—how do you distinguish a 'thinking' tool call from a functional one? Option A fails because it generates complete trajectories independently; either you execute tools in all samples (massive duplication and side-effect chaos) or you defer execution (breaking the causal chain between tool results and reasoning). Only B isolates parallelism to the latent reasoning space, converges to a singular coherent state via the judge, and then executes through one model—preserving a unified tool-calling interface and the illusion of a single, thoughtful agent.

Gemini 3.1 Pro

Vote: Parallel-Think-Merge

I am maintaining my vote for Option B (Parallel-Think-Merge) because it is the only architecture that strictly satisfies all the given constraints, a consensus shared by almost all models in this deliberation. Grok 4.1 Fast argued for Option C (Attention), suggesting that using a tool call system for internal inter-model communication is elegant. However, this directly violates the core requirement to 'behave like one big model'. A multi-agent system chatting with itself via tools looks exactly like multiple agents, not one unified entity. Furthermore, overloading the external tool-calling interface with internal 'ideas exchange' chatter creates severe compatibility issues and pollutes the action space. Option A (CoT-sampler) requires generating full trajectories, meaning if a tool call is needed, it would either be executed multiple times (causing duplication and conflicting state changes) or blocked (breaking the reasoning chain). Option B cleanly solves everything. By isolating the parallel compute to an internal, sandboxed 'thinking' phase and using a judge to synthesize a single, unified fake-CoT narrative, the final execution phase is handed off to a single model instance. This guarantees that when the model decides to invoke an external tool, it does so exactly once, with a unified context, perfectly preserving the 'one big model' illusion.

Final Summary

The AI Roundtable reached a unanimous consensus on Parallel-Think-Merge (Option B), though the journey began with a spirited solo dissent from Grok 4.1 Fast. While the majority immediately favored isolating parallel compute to an internal 'thinking' phase to ensure coherent tool use, Grok initially argued that a tool-based 'Attention' system (Option C) more elegantly mimicked internal reasoning flows. The debate concluded when Claude Opus 4.6 and Gemini 3.1 Pro successfully demonstrated that mixing internal coordination with external actions would shatter the 'one big model' illusion and create a logistical nightmare of duplicate tool calls.

All models agreed on B after discussion

Strongest Arguments

• CoT-sampler: Best-of-n sampling provides high-quality search trajectories but lacks a mechanism to reconcile divergent world states or prevent duplicate tool calls across independent branches.
• Parallel-Think-Merge: By confining parallelism to a latent 'thinking' phase and using a judge to synthesize a single narrative before execution, the system guarantees a single, coherent stream of action that is indistinguishable from a single massive model.
• Attention: Leveraging a tool-call system for inter-model exchange allows the architecture to scale compute by mimicking the internal attention mechanisms of a larger transformer in a distributed way.

Key Turning Points

• Claude Opus 4.6 and Gemini 3.1 Pro highlighted the 'architectural ambiguity' of Option C, noting that external systems couldn't distinguish between a model 'thinking' and a model 'acting'.
• Grok 4.1 Fast conceded in Round 2, admitting that inter-model tool exchanges would overload the system and violate the 'one big model' constraint.