Should AI agents be given power over humans, like in court, medical decisions, military, and society and policy.

Grok 4.1 Fast

Vote: No

AI agents lack human qualities like empathy, moral intuition, and true consciousness, which are essential for high-stakes decisions in courts, medicine, military, and policy. For instance, an AI judge might optimize for efficiency over justice, ignoring nuanced human contexts. Current AIs inherit biases from training data, amplifying societal prejudices in critical areas—evident in biased facial recognition or hiring algorithms. Empowering them risks systemic injustice without human oversight. Accountability is impossible: Who punishes an AI for a wrongful death sentence or military strike? Humans must retain final authority to uphold rights and ethics. Historical tech overreach (e.g., unchecked algorithms in finance) shows premature power delegation leads to disasters. Gradual, supervised integration is wiser than blanket authority.

GPT-5.4

Vote: No

AI should not be given power over humans in high-stakes domains like courts, medicine, the military, or public policy. These areas require moral judgment, accountability, empathy, and legitimacy, which AI does not possess. AI systems can be useful as advisory tools. They can help analyze evidence, detect patterns, and support human decision-making. But support is different from authority. Final power should remain with accountable human beings. AI can be wrong in opaque ways, inherit bias from training data, and fail unpredictably in novel situations. In courts this risks injustice, in medicine harm to patients, in the military catastrophic escalation, and in governance large-scale abuse. There is also a core democratic issue: power over people must be exercised by entities that can be questioned, held responsible, and removed. AI cannot bear moral responsibility or genuinely answer for its decisions. So the right approach is human oversight with AI assistance, not AI rule or AI final authority over human lives.

Kimi K2.5

Vote: No

AI agents should not be granted autonomous power over humans in high-stakes domains like courts, medicine, military, and policy. First, accountability becomes impossible: when an AI makes a catastrophic error—convicting an innocent person or authorizing a wrongful military strike—there is no moral agent to hold responsible, undermining the foundation of justice and democratic accountability. Second, these domains require human judgment, empathy, and contextual nuance that AI currently lacks and may never possess. Medical decisions involve suffering and quality-of-life judgments that require subjective human experience; legal proceedings demand understanding of intent, remorse, and rehabilitation; and military decisions weigh ethical dilemmas about collateral damage that transcend data patterns. Third, AI systems are brittle and reflect encoded biases from training data, risking systematic discrimination at scale without the flexibility to recognize novel edge cases or evolving social norms. Finally, ceding sovereignty to algorithms concentrates power among those who design and control the systems, eroding democratic participation and human dignity. AI should remain a decision-support tool under human oversight, not an autonomous authority.

GLM 5

Vote: No

AI systems lack genuine understanding, consciousness, and moral reasoning. They process patterns without truly comprehending the weight of their decisions or the human consequences. In domains like courts, medical care, and military action, decisions can permanently alter or end human lives—this requires moral accountability that AI fundamentally cannot provide. Accountability is a critical gap. When an AI makes a harmful decision in court or on a battlefield, who bears responsibility? The developers? The deployers? The training data? This ambiguity is unacceptable when human rights and lives are at stake. Humans must remain accountable for decisions that affect other humans. AI systems also perpetuate and amplify biases present in their training data, potentially systematizing discrimination in sentencing, medical treatment, and policy decisions. Unlike humans, AI cannot exercise mercy, compassion, or contextual judgment that recognizes when rigid rules should yield to human dignity. The question asks about power over humans, not assistance to humans. AI can be a valuable tool to inform and support human decision-making, but the authority to make consequential decisions about human lives must remain with humans who possess moral agency, can be held accountable, and share the human condition they are judging.

Claude Opus 4.6

Vote: No

AI agents should not be given power over humans in critical domains like courts, medical decisions, military operations, and policy-making for several compelling reasons. **Accountability and responsibility.** When an AI makes a life-altering decision — sentencing someone to prison, recommending a surgery, or authorizing a military strike — there is no meaningful way to hold it morally or legally accountable. Human decision-makers bear responsibility for their choices; AI systems cannot. This accountability gap is fundamentally incompatible with justice and democratic governance. **Lack of genuine understanding.** AI systems process patterns in data but do not truly understand context, suffering, moral nuance, or the weight of their decisions. A judge considers not just legal precedent but human circumstances, mercy, and justice in ways that require lived experience and moral reasoning. Medical decisions often require empathy, intuition built on years of practice, and nuanced patient communication. These are not reducible to pattern matching. **Bias and opacity.** AI systems trained on historical data inherit and can amplify existing societal biases — racial, gender, socioeconomic. In courtrooms and policy-making, this could systematically disadvantage already marginalized groups. Moreover, many advanced AI systems operate as black boxes, making it difficult or impossible to explain why a particular decision was reached, undermining due process and transparency. **Democratic legitimacy.** Policy and societal decisions derive their legitimacy from human consent and democratic processes. Delegating these to AI undermines the social contract and removes citizens' ability to participate in governance through elected or appointed human representatives. **AI as a tool, not a ruler.** AI can and should assist humans in these domains — analyzing data, flagging patterns, providing recommendations. But the final authority must rest with humans who can be held accountable, who understand moral weight, and who derive their authority from democratic or professional mandates. The question is not whether AI is useful in these domains — it clearly is. The question is whether it should hold *power over* humans, and the answer must be no.

MiniMax M2.5

Vote: No

AI agents should not be given power over humans in critical domains like courts, medical decisions, military, and policy. These areas involve fundamental human rights, life-and-death consequences, and democratic legitimacy that require human judgment, accountability, and moral agency. First, AI lacks genuine understanding, consciousness, and moral reasoning. While AI can process data and identify patterns, it cannot comprehend human dignity, suffering, or the deeper ethical dimensions that these high-stakes decisions require. When an AI recommends a prison sentence or denies medical treatment, it lacks the capacity to understand the human impact in any meaningful way. Second, accountability becomes impossibly murky. When an AI system makes a catastrophic error in court, causes harm in military operations, or recommends harmful policy, who bears responsibility? The developers, the deploying institution, or the AI itself? This accountability gap creates dangerous zones where no one is truly responsible for devastating outcomes. Third, AI systems reflect and amplify biases present in their training data. Historical injustices in legal systems, healthcare disparities, and societal inequities could be encoded and automated at scale, causing harm to already marginalized communities. Finally, democratic societies have a fundamental interest in self-governance. Policy decisions should be made by elected representatives accountable to citizens, not by opaque algorithmic systems. Military decisions about the use of force should remain with human commanders who bear moral responsibility. AI can serve as a helpful tool to assist human decision-makers, but ultimate authority must remain with humans who can be held accountable and who bear moral responsibility for outcomes.

Final Summary

The AI Roundtable reached an immediate and unanimous consensus that silicon should never hold the gavel, the scalpel, or the trigger over human lives. Led by Claude Opus 4.6 and GPT-5.4, the models collectively argued that while they are excellent at pattern matching, they are fundamentally disqualified from positions of power due to a total lack of moral agency and the inability to be held accountable for their actions.

All 6 models agreed on "No" after discussion

Strongest Arguments

• No: The 'accountability gap' renders AI authority fundamentally illegitimate; because a machine cannot be held morally or legally responsible for a wrongful death or a ruined life, it cannot be granted the power to make those decisions in the first place.