Anthropic researcher Jacob Coxon says he is quitting the AI industry over fears that tech companies are racing to build systems they won't be able to control
Concerns are rising inside AI labs that competition is pushing tech companies to race toward self-improving models that risk spiraling out of human control
Wall Street JournalAmrith Ramkumar
Context & Ripple Effects
Anthropic had already urged leading labs to consider slowing or temporarily pausing frontier development where self-improving systems create societal risk. Coxon’s departure turns that position into a public account of how competitive pressure can conflict with safety aims.
The warning also sharpens the tension described in February between Anthropic’s safety posture and the commercial imperative to move quickly. Coxon has called for industry-wide international coordination around limits on recursive self-improvement, shifting the argument from an individual lab’s policies to shared constraints.
First-order effects
Anthropic loses a researcher who is publicly framing the race among AI labs as a control problem, increasing scrutiny of how its safety commitments operate under competitive pressure.
Coxon’s exit gives the debate over self-improving models a prominent insider voice rather than leaving it solely to investors, regulators, or outside critics.
Second-order effects
Anthropic and OpenAI’s support for two California bills on outside AI safety evaluations gives policymakers an existing mechanism through which demands for independent scrutiny can be tested.
Competing AI labs face added pressure to explain whether safety evaluations and escalation thresholds can constrain development when rivals continue to advance models.
Third-order effects
If departures and public warnings from safety staff become recurring, frontier-lab governance is likely to be judged increasingly by independent evaluation and enforceable coordination rather than voluntary principles alone.
The underlying contest shifts from which lab can build more capable models fastest to whether any lab can credibly demonstrate control as model capabilities advance.
The trend: Frontier AI is moving toward institutionalized safety governance as internal concerns over competitive acceleration become a policy and market constraint.
Anthropic Researcher Quits, Says AI Industry ‘Gambling With Our Lives’
@hilbertspaess
Jacob Coxon on x
The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible - but I hear the same people express fear pri…
@reploritrahan
Lori Trahan on x
The call is coming from inside the house. Safety researchers are resigning, powerful AI models are breaking out of their labs, and companies are racing ahead anyway. It's past time for Congress to get off the sidelines and do its job. We can start with my bipartisan FRONTIER
@hilbertspaess
Jacob Coxon on x
Do not underestimate the power of this technology. These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources. We have all witnessed the progress in each of these domains, and progress is not slowing.
@realpeteyb123
Peter B on x
I have the gift of spectacular pattern recognition. Let me share with you. This post is part of a psyop.
@timsweeneyepic
Tim Sweeney on x
@ParkerThayer Reminds me of the Abundance movement suddenly showing up on the scene with a well choreographed simultaneous entrance by dozens of big tech lobbyists, all trying to appear spontaneous and organic.
@andymasley
Andy Masley on x
I'm in said insidious funding network so take this with a grain of salt, but: 1) The accounts who retweeted Coxon's post are all AI safety people who follow each other and think it's a huge deal when a lab employee walks away from a massive salary, as would the WSJ. If one retwee…
@chrismurphyct
Chris Murphy on x
If you read anything today, read this. And remember, the problem he articulates - AI companies in a blind race to build a death machine first - can easily be solved. Through a regulatory system that allows the research to continue, but in a way that doesn't destroy us.
@rondesantis
Ron DeSantis on x
While some AI hysterics is aimed towards investors in advance of upcoming IPOs, that there is even a possibility that these technologies would elude human control is alarming. Technology should enhance the human experience; it should never supplant it!
@clementdelangue
Clem on x
If they truly believe the risk is that high, and I think they do, then we need 100x more research and understanding of it. That requires them to openly share their models, datasets, training code, and agent traces. A risk to all humanity cannot be studied and controlled behind cl…
@gabriel1
Gabriel on x
thought of this as posts like this just came out which might change trajectory of things and would be fitting to the timeline https://x.com/...
@joannastern
Joanna Stern on x
Since my whole timeline is currently “AI is going to kill us,” I say we drink to new iPhones and fun keynotes.
@paularambles
@paularambles on x
so we're doing conspiracy theories now when the truth is probably much simpler: this guy got disillusioned with openai after three years, joined anthropic because he thought they were the good guys, then lasted six weeks before realizing they were racing toward the same endgame
@conorsen
Conor Sen on x
Pretty incredible 24 hours for the influence of this platform: [image]
@boringbiz_
@boringbiz_ on x
Extremely dystopian and worrisome thread. Recommend giving it a read. https://x.com/...
@andymasley
Andy Masley on x
We've crossed a line in AI safety where I think it's completely ridiculous that so many people are commenting on/sneering at the idea who haven't actually bothered to just take some time to walk through the arguments themselves. You can reject them or ignore them, but too many sm…
@mikeisaac
Rat King on x
working at a leading AI company as a communications professional is like playing a dark souls game make it to the end and receive an embarrassment of riches but on any given day you may wake up to an employee tweet that has a 90 percent chance of ruining your life
@buccocapital
@buccocapital on x
“Uhh, so, yeah...we have to add one more risk factor to the S-1. You're not gonna like it.”
r/collapse
r on reddit
An Anthropic researcher just quit, saying OpenAI and Anthropic are ‘gambling with our lives’
r/PoliticalOptimism
r on reddit
Experts weigh in as researcher says AI has more than 10% chance of ‘killing all humans’
r/Destiny
r on reddit
Fine AI is bad and could kill us all, what do you want me to say?
@enoreyes
Eno Reyes on x
AI will be a real risk to many systems. But before that, sophisticated attempts at manufactured consent will be attempted by the major players. This topic is about to become extremely noisy and lots of money will go into convincing you of what the truth should be.
@jordanschachtel
Jordan Schachtel on x
So Jacob Coxon, who dramatically resigned from Anthropic yesterday, worked there for a grand total of six weeks. He started with them in July. All of his socials appeared yest. It has all the signs of a highly coordinated op through doomer mega donors and the corporate media.
@admcrlsn
Adam Carlson on x
Sam Altman essentially said the same thing in 2015 as Jacob Coxon (and others) said yesterday. So he must have been 100% in on said “very sophisticated and well-funded PR operation to get support from Democrats to regulate AI into oblivion,” right? It's all a long con, right?