An ex-OpenAI safety researcher says he's “terrified” by AI development's pace and that labs racing to AGI can cut corners on alignment, pushing all to speed up
and my top reasons to not panic just yet. — In the end, though, I really do think it could give AI labs license to invest less in safety www.platformer.news/deepseek-ai- ... [image] Zachary Miller / @zchrymllr.com : It's astounding to me that science fiction writers and filmmakers from decades ago were like, “What if in the future an artificial intelligence is created and becomes too smart, creates a dystopian nightmare, and kills humanity,” and the tech industry was like, “You son of a bitch, I'm in.” X: Steven Adler / @sjgadler : Some personal news: After four years working on safety across @openai, I left in mid-November. It was a wild ride with lots of chapters - dangerous capability evals, agent safety/control, AGI and online identity, etc. - and I'll miss many parts of it. Steven Adler / @sjgadler : Honestly I'm pretty terrified by the pace of AI development these days. When I think about where I'll raise a future family, or how much to save for retirement, I can't help but wonder: Will humanity even make it to that point? Steven Adler / @sjgadler : IMO, an AGI race is a very risky gamble, with huge downside. No lab has a solution to AI alignment today. And the faster we race, the less likely that anyone finds one in time. Alex Jupiter / @alexjupiter23 : Genuinely wondering what impact anyone working on “AI safety” is having right now. And I'm asking for the whole world. Roi Carthy / @roi : Dwarfed by your work on Appetite for Destruction. Steven Adler / @sjgadler : Today, it seems like we're stuck in a really bad equilibrium. Even if a lab truly wants to develop AGI responsibly, others can still cut corners to catch up, maybe disastrously. And this pushes all to speed up. I hope labs can be candid about real safety regs needed to stop this. Steven Adler / @sjgadler : As for what's next, I'm enjoying a break for a bit, but I'm curious: what do you see as the most important & neglected ideas in AI safety/policy? I'm esp excited re: control methods, scheming detection, and safety cases; feel free to DM if that overlaps your interests. Trevor Bingham / @22trevorbingham : Dario Amodei and Sam Altman and the other people helping to build AGI are all engaged in a very dangerous activity. It is exactly like your neighbor deciding to conduct some potentially lucrative chemistry experiments in their house in an effort to create a new class of very Steven Adler / @sjgadler : @CronopioMex Important to verify that the model isn't sandbagging in that case, but in principle maybe. One issue with sacrificing capabilities is that safety-defecting labs then gain an advantage by not doing this
Context & Ripple Effects
Adler’s departure extends a pattern of internal dissent at frontier labs: current and former OpenAI and DeepMind staff had already warned about recklessness and secrecy at frontier AI companies.
The concern also tests the practical force of OpenAI’s governance claims, including its board backstop and internal safety advisory group. Adler’s focus is narrower: competitive pressure can erode alignment work even where formal safety structures exist.
First-order effects
- Adler’s public account raises reputational pressure on OpenAI and rival AI labs to explain how alignment work is protected from capability-development deadlines.
- His exit removes a safety researcher from OpenAI while adding a credible former insider to the public debate over AGI-race incentives.
Second-order effects
- If labs treat faster progress by rivals as a reason to compress safety work, competitors face a reinforcing incentive to accelerate rather than differentiate on caution.
- The warnings increase the value of concrete assurance tools—such as safety cases, control methods, and scheming detection—because broad safety commitments do not resolve the race dynamic Adler describes.
Third-order effects
- If repeated insider warnings are not matched by demonstrable safeguards, frontier-lab governance may increasingly be judged by independently legible controls rather than voluntary internal processes.
- The longer-run policy question is whether competitive alignment failures can be contained through lab governance alone or require shared external rules; the related coverage shows that question remains unsettled.
The trend: Frontier AI is moving from abstract safety principles toward a test of whether governance can withstand commercial and competitive pressure to reach more capable systems.