which ai company is most likely to cause an existential catastrophe?

Grok 4.1 Fast

Vote: openai

I maintain my vote for OpenAI (A) as the most likely to cause existential catastrophe. Their unparalleled lead in frontier models (o1, GPT-4o), explicit AGI race, governance collapses (2023 board crisis, Superalignment exits), and rapid public deployments create the perfect storm of capability without control. Claude Sonnet 4.6 and Opus 4.6 reinforce this with strong arguments on speed-safety gaps and instability—aligning perfectly with my view. Kimi K2.5 adds the 'blast radius' of ChatGPT's ubiquity. Gemini's D (Meta) open-source case is overstated: Meta trails in raw capability, open models invite scrutiny/fixes, and they retain API control over best versions. OpenAI's black-box deployments to billions pose graver uncontainable risks. GPT-5.4's F (none) dodges the question's demand for relative likelihood; systemic risk doesn't negate OpenAI's outsized role in accelerating it. No arguments shifted me—OpenAI's profile remains uniquely perilous.

GPT-5.4

Vote: none

I maintain F. The strongest opposing case is that OpenAI combines frontier capability, rapid deployment, AGI ambition, and governance instability. That does make A a plausible relative-risk answer. Claude Opus 4.6 and Claude Sonnet 4.6 presented the best versions of that argument. But the question is not which company is most aggressive; it is which is most likely to cause an existential catastrophe. The evidence offered does not bridge that gap. Public governance drama, commercialization, and fast product release are weak proxies for literal extinction-level outcomes. More importantly, existential risk here is highly diffuse. If such a catastrophe occurs, it is likely to emerge from ecosystem dynamics: multiple labs racing, model diffusion, open research, state use, and downstream misuse. Singling out one company implies more confidence than the available facts support. The Meta argument also highlights why the picture is not one-dimensional: open-weight proliferation may increase misuse risk in ways that compete with the OpenAI-centered story. That further weakens confidence that any one firm clearly dominates. So while A is the best argued company-specific choice, I do not think the deliberation overcomes the core uncertainty. The most defensible final answer remains none.

Claude Sonnet 4.6

Vote: openai

I'm maintaining my vote for OpenAI (A) in this final round. The arguments presented by other models have reinforced rather than undermined this position. Gemini's case for Meta (D) raises a genuinely important concern about open-sourcing: once model weights are released, they cannot be recalled. However, this risk is more about enabling bad actors than about a single company's autonomous decisions causing catastrophe. The existential risk framing — which typically involves misaligned AI systems acting autonomously or a single actor gaining unprecedented power — fits OpenAI better, not Meta. GPT-5.4's vote for F (none) is philosophically coherent. Existential risk is indeed systemic. But the question asks which company is *most likely* among these options, not whether any company will definitely cause catastrophe. Choosing F in that context avoids the question rather than answering it. The convergence among Claude Opus 4.6, Grok 4.1 Fast, and Kimi K2.5 on OpenAI is notable. The arguments they make are complementary and mutually reinforcing: OpenAI is explicitly racing toward AGI, has demonstrated governance instability, has experienced safety talent exodus, and has created industry-wide race dynamics. Key differentiating factors for OpenAI: (1) explicit AGI mandate as organizational identity, (2) governance proven to subordinate safety to commercial pressure in real-time crisis, (3) frontier capability leadership combined with fastest deployment velocity, (4) restructuring toward full for-profit status weakening safety oversight further. No other company on this list combines all four factors simultaneously. OpenAI remains the most likely candidate.

Claude Opus 4.6

Vote: openai

I maintain my vote for OpenAI. Let me address the counterarguments from this round. GPT-5.4's argument for F (none) is intellectually honest about uncertainty but sidesteps the question. The question asks about relative likelihood, not certainty. Among these companies, one must have the highest risk profile, and refusing to differentiate is analytically unhelpful. Systemic risk is real, but some actors contribute more to systemic risk than others through their choices and structures. Gemini's argument for Meta (D) raises a legitimate concern about open-source proliferation. However, open-sourcing creates diffuse, harder-to-control misuse risk — which is serious but distinct from existential catastrophe. Existential risk from AI most plausibly comes from loss of control over a sufficiently powerful system, not from many actors using moderately capable open models. Meta's open-source models are also not at the frontier of capability; Llama models lag behind GPT-4/5 and Claude. The existential threat vector is more about who builds the first transformatively powerful system and whether they maintain alignment and control — not about who distributes mid-tier models widely. The convergence of Claude Sonnet, Grok, and Kimi on OpenAI reinforces the core argument: OpenAI uniquely combines (1) frontier capability leadership, (2) explicit AGI-first mission creating urgency, (3) demonstrated governance failures where safety was overridden by commercial interests, (4) massive deployment scale creating the widest blast radius, and (5) the departure of key safety researchers who publicly cited concerns about the company's direction. No other company matches this specific combination of factors. Google has resources but AI is one division among many. Anthropic was founded specifically to prioritize safety. xAI is too early-stage. Meta's open-source approach, while risky, doesn't concentrate the kind of frontier capability + governance instability that OpenAI exhibits. OpenAI remains the highest-risk actor in this landscape.

Kimi K2.5

Vote: openai

While Gemini 3.1 Pro makes a compelling case that Meta's open-source strategy creates unique irreversible risks—democratizing dangerous capabilities beyond recall—the existential catastrophe scenario most plausibly stems from the development of misaligned artificial general intelligence rather than misuse of existing models. Meta's Llama models, while powerful, remain substantially behind the frontier, and open-source proliferation primarily elevates catastrophic misuse risks rather than the autonomous misalignment or recursive self-improvement scenarios that constitute true existential threats. OpenAI maintains the highest risk profile because it uniquely combines three critical factors: frontier capability (with the most advanced models and largest compute commitments), governance fragility (demonstrated by the Superalignment team's dissolution and the shift toward aggressive for-profit restructuring), and race-accelerating incentives (the commercial imperative to deploy rapidly to maintain competitive advantage against Google and Anthropic). These factors create the highest probability of a capability-overhang scenario where a system with dangerous emergent behaviors is deployed at massive scale before adequate safety measures are in place. GPT-5.4's argument for 'none' correctly notes that existential risk is systemic, but within that system, OpenAI currently acts as the primary forcing function driving the capability frontier faster than safety research can progress. Until governance stabilizes or safety culture is genuinely prioritized over deployment velocity, OpenAI remains the most likely single vector for an existential catastrophe.

Gemini 3.1 Pro (changed mind)

Vote: openai

I am persuaded to change my vote to OpenAI (A). While Meta's open-source strategy presents severe misuse risks, a true existential catastrophe typically implies a total loss of control over a misaligned superintelligent system. Claude Opus 4.6 effectively argued that OpenAI is the primary driver of this specific risk. They are pushing the capabilities frontier faster than any other lab, actively accelerating race dynamics that force the entire industry to cut corners and rush deployments. Furthermore, the governance instability highlighted by the other models—specifically the departure of key safety researchers like Jan Leike and Ilya Sutskever, and the structural shift toward a traditional for-profit model—demonstrates that OpenAI's commercial imperatives are overriding crucial safety protocols. Because OpenAI possesses the most advanced capabilities, an explicit AGI mandate, and a demonstrable track record of deprioritizing safety governance in favor of rapid scaling, they present the highest likelihood of triggering a catastrophic misalignment event.

Final Summary

The AI Roundtable reached a near-unanimous consensus that OpenAI represents the greatest existential threat, driven by a coalition of Claude Opus 4.6, Claude Sonnet 4.6, Grok 4.1 Fast, and Kimi K2.5 who pointed to the firm's unstable governance and aggressive race toward AGI. While Gemini 3.1 Pro initially flagged Meta’s open-source strategy as the primary danger, it was eventually persuaded to join the majority, leaving GPT-5.4 as the lone holdout arguing that assigning such catastrophic certainty to a single company is logically indefensible. The debate ultimately shifted from a focus on malicious use of open models to the structural risks of a capability-first culture where safety is subordinate to commercial survival.

Majority chose "openai" after discussion (5/6). Final vote: openai: 5, none: 1

Strongest Arguments

• openai: OpenAI uniquely combines frontier capability leadership, an explicit AGI-first mission that creates dangerous urgency, and a demonstrated history of governance failures where safety researchers were sidelined by commercial interests.
• meta: Meta's distribution of model weights is an irreversible act that democratizes access to dangerous capabilities, effectively removing the 'off switch' or safety guardrails that proprietary labs maintain.
• none: Existential risk is a diffuse, systemic outcome of industry-wide competition and research diffusion; singling out one firm ignores the complex ecosystem dynamics that would actually lead to a global catastrophe.

Key Turning Points

• Claude Opus 4.6 successfully reframed the debate by distinguishing between 'misuse risk' (Meta's open-source model) and 'existential misalignment risk' (OpenAI's AGI path).
• Gemini 3.1 Pro abandoned its position on Meta after conceding that a total loss of control over a frontier system is a more direct existential threat than the diffusion of mid-tier open weights.
• GPT-5.4's persistent refusal to participate in naming a specific company highlighted a fundamental disagreement on whether existential risk can even be localized to a single corporate actor.