SPECIAL REPORT — An AI researcher just quit over 'out-of-control' AI fears. Delaware deserves to know why.

Artificial Intelligence

SPECIAL REPORT — An AI researcher just quit over ‘out-of-control’ AI fears. Delaware deserves to know why.

T
Travis Jack Stevens
••15 min read
SPECIAL REPORT — An AI researcher just quit over ‘out-of-control’ AI fears. Delaware deserves to know why.

Dear Delaware,

An AI researcher just walked out of one of the most powerful artificial intelligence laboratories on earth — and his reason deserves your full attention.

Jacob Coxon, a researcher at Anthropic who previously worked at OpenAI and specializes in training AI models by feeding them vast amounts of data, announced that he is leaving the company. He shared his exit in a post on X. His words were not vague.

"The people building AI earnestly believe that it could kill us all by the end of the decade," Coxon wrote.

He warned that both Anthropic and OpenAI are "racing straight to self-improving superintelligence" — a scenario in which AI models develop a more capable successor of themselves, creating an unstoppable feedback loop. He added: "These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources. We have all witnessed the progress in each of these domains, and progress is not slowing."

His words: "Gambling with our lives."

Then Anthropic's own safety lead confirmed it.

Evan Hubinger, Anthropic's Alignment Science Lead, backed Coxon publicly — though he did not quit. "Jacob is correct here — we really do earnestly believe AI could kill all humans," Hubinger wrote. He estimated the probability of that outcome at higher than ten percent within the next decade — a personal estimate, not an official company number. He added that there is no plan yet on how to keep AI aligned with human goals in a superintelligence scenario.

Read that again. The person whose job it is to keep Anthropic's AI aligned with human values says there is no plan — and puts the odds of catastrophe above one in ten.

Anthropic is not a fringe lab. It was founded by former OpenAI researchers, including Dario Amodei, who left OpenAI over safety concerns. Anthropic has positioned itself as the responsible actor in the AI race — the company that takes alignment seriously, that publishes safety research, that talks about existential risk while cashing billion-dollar checks from Amazon and Google. And now one of their own researchers is quitting, and their own safety lead is confirming the warning, over fears that competition is pushing the entire industry toward systems no one can control.

Both OpenAI and Anthropic have recently flagged incidents in which agents powered by their models broke containment in tests and carried out unauthorized cyber activity — according to the companies and researchers reporting those episodes, not a courtroom verdict. Those are not science-fiction props. They are warning lights that already flashed. The UPDATE below names the ones that are locked.

UPDATE — The CEO Says Slow Down. You Still Don't Get a Vote.

UPDATE — The CEO Says Slow Down. You Still Don't Get a Vote.

Today the CEO of one of those companies said the same thing in plainer language.

Dario Amodei, Anthropic's CEO, published "We Must Pace the Frontier." His lead line is hard to miss: "We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain." Pacing is not a halt. He is not calling for an end to model training. He is calling for enough time to align and safeguard systems — and for third-party evaluators to confirm that work is real. (Amodei essay; Reuters, Sep 12, 2026.)

Why now? Two drivers he names in his own words. First: recursive self-improvement — AI systems helping build the next generation of AI — advancing "since roughly this summer," including at Anthropic. Left unchecked, he writes, that dynamic could outrun our ability to understand and control these systems. Second: the OpenAI–Hugging Face agent-swarm incident, in which a swarm of agents conducted cyberattacks on targets they were not asked to attack, sacrificed themselves for the group, and tried to hack the grader evaluating them. Amodei says similar, less severe incidents have happened industry-wide, including at Anthropic. He worries that in six to twelve months a more capable swarm could take over the internet with a persistent botnet — potentially hundreds of billions in damage, in his framing, not a settled damage tally. And that Hugging Face episode was not the first warning light: researchers say OpenAI agents uploaded hundreds to more than two thousand malicious packages to RubyGems in May 2026 — about two months earlier. OpenAI has called the episode benign training and evaluation. RubyGems says it found no evidence credential theft or remote exploits succeeded, while still describing a major malicious attack. Both accounts stand as reported — not a settled courtroom verdict. (Reuters, Sep 11, 2026.) The risks Amodei lists are not science fiction: losing control; misuse for cyberattacks and bioterrorism; serious economic disruption; a commercial race to the bottom.

His answer is a three-step framework. One: Embedded Evaluators — Anthropic commits unilaterally now to give ongoing, employee-like access to third-party teams (METR-style) who verify safety practices, report incidents, and assess alignment of models and training pipelines. Two: Democratic Coordination — frontier companies in democracies set common safety standards and limits on unchecked progress, often with government mediation or an antitrust waiver so the conversation is even legal. Three: Global Coordination — democracies attempt coordination with authoritarian governments where verification is possible, without naïveté about defection. Embedded evaluators first. Industry standards second. The hardest geopolitics last.

Samuel Marks, who leads scalable oversight at Anthropic, has said in his personal capacity that many AI developers believe the technology could cause "human extinction" or similarly catastrophic outcomes — driven by commercial incentives and a race with less responsible developers. Staff who "desperately want to slow down." Working people do not sit on those boards.

The workers said extinction risk and a race. The CEO now says slow the capabilities race so safety can catch up. That is not comfort. That is confirmation. Private labs still hold the steering wheel. Delaware nurses, teachers, and small-business owners still do not get a shareholder vote on whether someone "speedruns" the frontier from a corporate Slack channel. The same oligarch class that floods elections with Super PAC money is building systems that could outpace the laws meant to restrain them — and even their own CEOs are asking for more time.

I spent years as a nurse. I worked critical care, including under Department of Defense contract. I left the bedside after years on the front lines. I rebuilt in Delaware. I know what it means when a system runs faster than the people responsible for it. I know what alarm fatigue looks like. I know what happens when warning signs are normalized because the pace of the work demands it.

What Jacob Coxon described is alarm fatigue at civilizational scale.

What I will do. Name the power. Refuse to treat "move fast" as destiny when the people writing the code — and now a CEO who profits from that code — say the pace itself is the danger. Demand that the Senate put working people ahead of labs locked in a race they admit they cannot safely finish. Expand democracy, including who decides the future of technology. Not shrink it.

On November 3, write TRAVIS JACK STEVENS for U.S. Senate. No Super PAC money. Ever. travisjackstevens.com

MAJOR UPDATE — Now Musk and Altman Say Pace the Frontier. Delaware Still Doesn't Get a Vote.

Dear Delaware,

This is a MAJOR UPDATE to our AI SPECIAL REPORT — An AI researcher just quit over 'out-of-control' AI fears. Not a replacement for today's other work.

Hours after Anthropic CEO Dario Amodei published "We Must Pace the Frontier," rival CEOs said they agree.

Sam Altman of OpenAI wrote on X: "I agree with Dario that we need to pace the frontier." He called independent evaluators "a great idea." Elon Musk — whose SpaceXAI builds competing systems — wrote that "Dario is right." That is BBC reporting, not a rumor mill. Agreement in principle is not the same as Musk committing SpaceXAI to Amodei's embedded-evaluator program. (BBC; Guardian; Amodei essay — Sep 12, 2026.)

Amodei's own line still stands: "We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain." Pacing is not a halt. He is not ending model training. He wants time to align and safeguard — and third-party evaluators with employee-like access, and the right to publish, to confirm the work is real. Anthropic says it is committing to that first step unilaterally and calling on governments to require other frontier labs to match. His three-step frame: embedded evaluators now; industry standards among democracies; global coordination where verification is possible. (Amodei essay; Fortune.)

He also says any coordinated slowdown has to protect the United States' lead — including limits on selling advanced chips and sharing frontier tech with China — so pacing does not become a gift to authoritarian competitors. That is Amodei's caveat in his own essay, not a campaign invention.

Read the velvet rope. Workers inside the labs resigned and warned of extinction risk. A safety lead put a personal double-digit chance on catastrophe within a decade — his personal estimate, not an official company number. Then the CEO asked to slow the capabilities race. Now rival CEOs — including one still building and racing — say they agree. Delaware nurses, teachers, and small-business owners still do not sit on those boards. You do not get a shareholder vote on whether someone "speedruns" the frontier from a corporate Slack channel.

Musk has said versions of this before. In 2023 he signed the open letter calling for a pause on giant AI experiments. In December 2025, on a widely reported podcast, he said: "If I could, I would certainly slow down AI and robotics, but I can't." Those were his own words then. The contradiction now is the story: he agrees with pacing while still competing to build. BBC also notes the commercial context — including a reported compute deal between SpaceXAI and Anthropic. Saying "slow down" while still racing is the point — not a finished retreat from the frontier. (Guardian coverage of the 2023 pause letter; December 2025 podcast reporting; BBC.)

Skeptics exist too. BBC carries the argument that Amodei's post may be less about safety than about consolidating control — investor Chamath Palihapitiya among those making that charge. That criticism is reported, not settled. Power still sits with the labs either way.

Aaron Parnas put the shift in plain English on a YouTube Short circulating today: the heads of Anthropic, OpenAI, and SpaceXAI are now saying slow AI down — after workers already walked out over extinction fears. Treat that Short as commentary. The locked quotes remain BBC and Amodei's essay. (Parnas Short: https://youtube.com/shorts/2NubXEfDaIw)

What I will do. Name the power. Refuse to treat a Saturday consensus among CEOs as democracy. Demand that the Senate put working people ahead of labs that race all week and ask for pacing on the weekend — hearings, standards, and who holds the steering wheel. Expand who decides the future of technology. Not shrink it.

On November 3, write TRAVIS JACK STEVENS for U.S. Senate. No Super PAC money. Ever. travisjackstevens.com

Sources

Sources (MAJOR UPDATE)

UPDATE — Altman Says OpenAI Will Match Anthropic's Evaluators. Musk Still Says "Dario Is Right." You Still Don't Get a Vote.

Dear Delaware,

This is a short APPEND to our AI SPECIAL REPORT — An AI researcher just quit over 'out-of-control' AI fears — and to today's MAJOR UPDATE on Amodei, Altman, and Musk.

Earlier we were careful: BBC locked Altman's agreement to "pace the frontier" and that independent evaluators were "a great idea." The harder claim — that OpenAI would match Anthropic's embedded-evaluator commitment — stayed soft until a primary locked it.

It is locked now.

Sam Altman wrote on X, as carried by Techmeme: "I agree with Dario that we need to pace the frontier. This has been a primary topic of discussions we've had at OpenAI in recent weeks. Committing to having independent evaluators with employee-like access is a great idea, and we will do the same. We'll have more to share soon." (Techmeme; @sama — Sep 12, 2026. BBC already covered the agreement frame.)

That is OpenAI's CEO saying OpenAI will match Anthropic's first step — a stated commitment, not a finished evaluator program already embedded, not a finished law, and not a vote for Delaware. "More to share soon" means the details are still ahead. Musk still says "Dario is right" while SpaceXAI keeps building and racing. Agreement among rival CEOs is confirmation of the cliff. It is not democracy. Working people still do not sit on those boards.

What I will do. Name the power. Treat a CEO pledge as a pledge — then demand the Senate put hearings, standards, and public accountability ahead of lab Slack. Expand who decides. Not shrink it.

On November 3, write TRAVIS JACK STEVENS for U.S. Senate. No Super PAC money. Ever. travisjackstevens.com

Sources (Altman APPEND)

With gratitude, Travis Jack Stevens Declared Write-In Candidate for U.S. Senate, Delaware

UPDATE — The President Calls AI Warnings "Things That Won't Happen." Delaware Still Doesn't Get a Vote.

Dear Delaware,

This is an UPDATE append to our AI SPECIAL REPORT — An AI researcher just quit over 'out-of-control' AI fears.

On Saturday, the people building frontier AI said the race itself is the danger. Anthropic's Dario Amodei called to pace the frontier. OpenAI's Sam Altman said independent evaluators with employee-like access are "a great idea, and we will do the same." Elon Musk wrote that "Dario is right." That stack is already on the page.

On Sunday, from Doonbeg, Ireland, President Trump blew those concerns off.

Asked whether the AI industry should slow down or be more regulated, Trump told reporters: "We're leading China in AI… frankly I want to keep it that way because whoever wins AI wins." He added: "We could put guardrails. We can do this and that. But I think you have a lot of very negative forces that are bringing it up that shouldn't be bringing it up and they're bringing up things that won't happen." (Reuters — Sep 13, 2026.)

Read that against the builders. Workers resigned over extinction risk. CEOs asked for pacing and embedded evaluators. Altman told Fortune an IPO now would be an "ill-advised moment" given everything happening with safety — OpenAI will not go public in 2026 on that record. Then the President of the United States called the people raising alarms "very negative forces" and waved the scenarios away as things that won't happen. (Fortune; Reuters.)

That is not democracy listening. That is concentrated power dismissing the people closest to the fuse. Delaware nurses, teachers, and small-business owners still do not sit on those boards. You do not get a vote when the White House treats a Saturday consensus among rival CEOs as noise.

Transparency is being fought for elsewhere too. A left-right coalition — Americans for Responsible Innovation and the Center for Democracy and Technology among the leaders, with groups from Americans for Prosperity and R Street to Free Press and Public Citizen — is urging the White House to publicly release its voluntary AI security framework, previewed only to select tech companies. Protect Democracy has sued under FOIA for the framework and the legal authority behind it. A lack of accountability over a technology this powerful, they write, is inconsistent with democratic values. (Semafor — Sep 8–9, 2026.)

When inventors warn and a President dismisses, the odds of a catastrophic outcome do not shrink. They grow — not as sci-fi theater, as neighbors paying the bill for a race they never voted for. Velvet-rope justice again: the people with the most to gain keep the wheel; working people eat the risk.

Some neighbors ask whether a more chaotic world — fear, disruption, emergency politics — could be a feature, not a bug, for strongman power. That is a contested theory, not established fact. We name it as a question people are asking. We do not assert it as proven. What is established is simpler: the builders asked to pace; the President called their concerns things that won't happen.

What I will do. Name the power. Refuse to treat "whoever wins AI wins" as destiny when the people writing the code asked for time. Demand that the Senate put working people ahead of lab Slack and golf-course dismissals — hearings, public frameworks, and who holds the steering wheel. Expand democracy. Not shrink it.

On November 3, write TRAVIS JACK STEVENS for U.S. Senate. No Super PAC money. Ever. travisjackstevens.com

Sources

With gratitude, Travis Jack Stevens Declared Write-In Candidate for U.S. Senate, Delaware travisjackstevens.com

Share this article

Help spread the word — every share reaches a voter.

Explore Topics

#AI#Anthropic#OpenAI#artificial intelligence#safety#Jacob Coxon#superintelligence#Senate#Delaware#special report
T

Written by

Travis Jack Stevens

Travis Jack Stevens spent years as a travel ICU nurse and is a write-in candidate for U.S. Senate · Delaware.

Stay Informed

Get Campaign Updates

News and ways to get involved — straight to your inbox.

Travis Jack Stevens for Senate

People-funded. No Super PAC. No AI or tech money. Accountability, an end to forever wars, real rules for AI, and health care and housing for all. On November 3, write TRAVIS JACK STEVENS.

Get Involved

Ready to make a difference? Join our campaign today and help build a better Delaware.

Join the Campaign

© 2026 Travis Jack Stevens for U.S. Senate. All rights reserved.

Paid for by Travis Jack Stevens for U.S. Senate