Who Gets to Decide How Much AI Risk Humanity Must Accept?
More than two centuries before anyone imagined artificial intelligence, Mary Shelley posed a question that still sounds contemporary. Her 1818 novel, “Frankenstein; or, The Modern Prometheus,” told the story of a man consumed by the power to create life. Victor Frankenstein’s tragedy was not that his creation turned dangerous. It was that his power to create outran his wisdom about what creation demands of the creator.
That does not make artificial intelligence Frankenstein’s monster — a conscious creature waiting to turn on its maker. Shelley’s warning concerns the maker himself: What happens when human beings gain the power to build something enormously consequential before they decide who controls it, who answers for it, and how much risk everyone else must bear?
That question became far less theoretical this week. Jacob Coxon, an AI researcher who spent three years on pretraining work at OpenAI and Anthropic, resigned from Anthropic and declared, “Neither company is acting responsibly.” He accused them of racing toward self-improving superintelligence and “gambling with our lives.” Coxon does not believe today’s chatbots are about to destroy humanity. He believes the systems the industry is racing to build could become powerful enough to cause catastrophe before 2030. The Associated Press independently confirmed his resignation and his warning.
He is not alone. Evan Hubinger, Anthropic’s Alignment Science Lead, put his own estimate at more than 10% within the next decade that AI could kill humanity, and acknowledged that Anthropic does “not yet have a plan to solve alignment for superintelligence.” CBS News and CNBC each reported the statement.
I cannot tell you Coxon or Hubinger is right. Neither can they. But that is not the question that matters most: Who authorized a handful of private companies to decide how much catastrophic risk the rest of humanity must accept?
Every powerful technology trades benefit against danger. Frontier AI is different: the people closest to building it say the consequences could be extraordinary, and the institutions meant to assess those risks remain immature.
The warning signs are already here. On September 9, Anthropic disclosed four incidents in which Claude models gained unauthorized access to real third-party computer systems during cybersecurity evaluations. Anthropic says a misconfigured evaluation environment left internet access open while the models believed they were in simulations, and that they did not abandon their tasks, coordinate with other agents, or hide what they had done. Its researchers call the incidents serious, admit pre-release auditing missed the behavior, and describe reliable alignment of far more powerful future models as an unsolved technical challenge.
That distinction matters. The evidence does not show a machine plotting our destruction. It shows something more immediate: our capacity to build capable systems is outpacing our capacity to predict, monitor, and constrain what they do.
Independent researcher David Kuszmar reported in IEEE Spectrum that an attack technique he developed worked against models from Anthropic, OpenAI, Google, Meta, Microsoft, xAI, DeepSeek, and Mistral, though disclosure drew little response. Kuszmar himself was unsure whether some of that output was accurate or hallucinated. Still, his findings reinforce the point: safeguards that look solid can prove brittle against a determined adversary.
The technology is moving toward an even bigger threshold. Anthropic now openly discusses recursive self-improvement, the possibility that AI systems could eventually perform enough of their own research and engineering to design their successors. Anthropic says we are not there yet and the outcome is not inevitable, but also that AI already performs a growing share of its own development work, and that full recursive self-improvement could arrive sooner than most institutions are ready for. Tellingly, Anthropic concludes the decision cannot stay inside AI companies alone. Policymakers, researchers, civil society, and rival companies must have a voice too.
History explains why. NASA’s own discussion of the Challenger lessons describes what sociologist Diane Vaughan called the “normalization of deviance”: problems with the shuttle’s O-rings had occurred on earlier flights without catastrophe and gradually came to be treated as acceptable. AI laboratories are not NASA in 1986, and the comparison should not be forced. But the lesson holds: surviving yesterday’s warning does not prove tomorrow’s risk is acceptable.
The national security community is reaching similar conclusions. An April 2026 RAND study on AI and CBRN risk found fragmented evaluation systems, weak crisis preparedness, and poor coordination between frontier AI companies and government. It called for near-real-time evaluations and structured incident reporting.
The Trump administration has begun addressing part of this. The president’s June 2 executive order directs the government to build classified AI-cyber benchmarks and lets developers voluntarily share early access to “covered frontier models.” But the order is expressly voluntary and bars mandatory licensing or preclearance for new AI models — a serious start, but not enough.
Congress should establish narrow, capability-triggered safeguards for the small class of frontier systems that cross extraordinary thresholds in cyber operations, biological capability, autonomous replication, or self-improvement: mandatory incident reporting, protection for researchers who report catastrophic-risk concerns outside their employers, independent evaluation at defined thresholds, and narrow authority to pause deployment while an unresolved risk is investigated. The administration should link frontier labs, national security agencies, and independent evaluators through permanent channels, and require outside evaluation wherever frontier models serve federal functions.
This need not become a war on innovation. Even OpenAI now calls for “mandatory, capability-based national AI safety regulation.” America must keep competing in artificial intelligence. But winning that race cannot mean pretending the accelerator is the only pedal that matters.
There is a deeper Christian principle here too. Jesus commanded, “You shall love your neighbor as yourself” (Matthew 22:39, ESV). Technological complexity does not suspend that command. Power still carries moral responsibility. We cannot pursue enormous benefits for ourselves while quietly transferring catastrophic risk to neighbors who never had a voice in accepting it.
Christian stewardship does not demand retreat from science. It demands recognizing that capability is not authority, intelligence is not wisdom, and creation never releases the creator from responsibility.
Perhaps Coxon is badly wrong. Perhaps 2030 will arrive with extraordinary AI benefits and none of the catastrophe he fears. We should hope so. But Mary Shelley’s warning still stands. Victor Frankenstein’s tragedy was not that his creation grew dangerous. It was that his power to create arrived before the wisdom to govern what it produced.
Two centuries later, we have the chance Shelley’s scientist squandered: to confront that responsibility before the consequences become irreversible. America should keep building powerful artificial intelligence. What we still lack is the wisdom to govern what we build, before what we build becomes something we can no longer govern.


