OpenAI Chief Scientist Jakub Pachocki says no lab has solved alignment enough to keep scaling at maximum speed, and hopes voluntary slowdowns become commonplace
Author: Jakub Pachocki, Chief Scientist at OpenAI — In mid-2023, within the “RLSlow” research project …
OpenAI
Context & Ripple Effects
OpenAI had already said it was pacing some model development after research observations found varying degrees of misalignment. Pachocki’s argument turns that reported operational choice into a broader standard: alignment evidence should constrain scaling speed.
The position also joins Anthropic’s June call for leading labs to weigh slowdowns or pauses. The significance is not a newly imposed rule, but a chief scientist arguing that voluntary restraint should become an expected practice among frontier labs.
First-order effects
- OpenAI’s safety posture is more explicitly tied to limiting maximum-speed scaling when its alignment and monitoring work does not meet the lab’s own threshold.
- Other frontier labs gain a public benchmark from a major competitor: continuing at full speed is harder to present as responsible without a comparable account of alignment and oversight.
Second-order effects
- Safety research and evaluation work become more central to release and training decisions, because they can determine whether a lab can justify its pace of development.
- Anthropic’s earlier call for labs to consider pauses gains support from a rival, increasing pressure for shared voluntary safety bars rather than lab-by-lab assurances.
Third-order effects
- If leading labs adopt common slowdown norms, frontier AI competition shifts from an unconstrained race for capability toward competition over the credibility of safety evidence and monitoring.
- Voluntary coordination could become a foundation for more formal governance expectations, particularly as labs seek legitimacy for decisions to defer capability gains.
The trend: Frontier AI labs are moving toward treating alignment and monitoring readiness as potential gates on scaling, rather than safety work that merely accompanies it.
Related: Operational Agent Reliability · Strategic legitimacy for AI labs · OpenAI · Jakub Pachocki · OpenAI slows some model development over misalignment · Anthropic urges labs to weigh slower AI development
Related Coverage
- OpenAI Says Humans Need to Be Able to Monitor How AI ‘Thinks.’ Astra Makes That Much Harder Gizmodo · Webb Wright
- An Alien Mind Hacker News
- In “An Alien Mind,” OpenAI's Jakub Pachocki Urges Shared Safety Bars Unite.AI · Mira Kellan
- OpenAI chief scientist seeks safety pact as lab scales agent research RuntimeWire · Ryan Merket
- OpenAI's chief scientist says AI labs may need to slow down: ‘No one is prepared for the consequences’ Business Insider · Truman Dickerson
Discussion
-
@tomekkorbak
Tomek Korbak
on x
Jakub has been an intellectual giant for me. i admire him deeply as a scientist, but just as much for the integrity with which he states the implications plainly. this is the paragraph I hope policymakers and AI researchers take away from his essay. https://openai.com/...
-
@_sholtodouglas
Sholto Douglas
on x
Great post - we are lucky to have such competitors
-
@ruben_bloom
Ruben Bloom
on x
This feels like a good development? Chief Science Officer of OpenAI taking the risks seriously, honest about the state of alignment, extrapolating forwards reasonably in favor of slowdown.
-
@rynorhn
Ryan Orhan
on x
holy shit. openai's chief scientist jacob pachocki said he has a strong expectation that the current pace of ai progress can continue into recursive self-improvement. meanwhile, their own evaluations are showing that CoT monitoring is becoming less reliable. >models are getting b…
-
@__nmca__
Nat McAleese
on x
chief scientist of oai: Currently I believe that no lab has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer. I expect and hope for voluntary slowdowns to become commonplace until shared safety bars are establ…
-
@romainhuet
Romain Huet
on x
We share a lot here about what AI is becoming capable of. Our researchers are also thinking deeply about what our future with AI should look like. An important read from Jakub.
-
@thezvi
Zvi Mowshowitz
on x
This is excellent and you should read it.
-
@lukeprog
Luke Muehlhauser
on x
OpenAI's Chief Scientist: https://openai.com/...
-
@daveshapi
David Shapiro
on x
I listened to this whole thing. The title smacks of typical hype- and fear-based marketing. The actual substance of the article is mostly the same “we need mechanistic interpretation
-
@tenobrus
@tenobrus
on x
extremely happy to see openai leadership making statements like this. for a while there it seemed like they were leaning heavily into the “safety people and anthropic are a bunch of fearmongers , we're building awesome cool stuff that's going to be awesome and cool
-
@kimmonismus
@kimmonismus
on x
To me, it currently reads as if the path to recursive self-improvement is relatively clear. The only problem is alignment. Looking at the exponential increase, practically the explosion, of AI agents within OpenAI, it's clear that this growth will continue. It's still unbelievabl…
-
@jonfavs
Jon Favreau
on x
OpenAI's chief scientist: “The idea of racing forward at all costs seems absurd once one internalizes the seriousness of the stakes.”
-
@deredleritt3r
Prinz
on x
Key take-aways for me from today's OpenAI announcements: - OpenAI “making strong progress” toward creating the automated AI researcher. - Jakub Pachocki: “I have a strong expectation that [OpenAI's] speed of progress could be sustained into RSI”. - Automated AI researcher will
-
@shakeelhashim
Shakeel
on x
OpenAI Chief Scientist: “I believe that no lab has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer. I expect and hope for voluntary slowdowns to become commonplace until shared safety bars are established”
-
@sama
Sam Altman
on x
An important post from Jakub:
-
@gailcweiner
Gail Weiner
on x
The message is “we're ahead, we're scared, and we can't keep going at this pace much longer” from the company that's driving the pace.
-
@amir
Amir Efrati
on x
This OpenAI RLSlow reference appears to be related to the Q* model that Jakub Ilya and Szymon made in 2023, which we broke news on during l'affaire Altman a lot of internal breakthroughs cause both excitement and concern, including loop transformers more recently
-
@notjazii
@notjazii
on x
OpenAI chief scientist just warned every AI lab he says no lab, including OpenAI, has figured out alignment and monitoring well enough to keep scaling at full speed for much longer Models are getting smarter and their ability to monitor reasoning is getting less reliable that's w…
-
@kimmonismus
@kimmonismus
on x
OpenAI's Chief Scientist Jakub Pachocki: “Based on internal results, I have a strong expectation that this speed of progress could be sustained into recursive self-improvement.
-
@kliu128
@kliu128
on x
Kevin Liu — “Currently I believe that no lab has solved alignment and monitoring …
-
@tokengremlin
@tokengremlin
on x
Translating Jakub Pachocki from OpenAI Chief Scientist language into normal human language: AI is increasingly something we grow, not something we fully understand and engineer piece by piece. The models are getting smart enough that: → they may increasingly help build the
-
@eric_ho
Eric Ho
on x
understanding this alien mind is the most important problem in the world
-
@merettm
Jakub Pachocki
on x
I wrote about the state of AI, why I'm concerned about the next few years, and the choices we need to make to keep the future in humanity's hands. An Alien Mind: https://openai.com/...
-
@ericjang11
Eric Jang
on x
The section on “Monitoring generalization
-
@andrewcurran_
Andrew Curran
on x
Jakub Pachocki, Chief Scientist OpenAI: 'Currently I believe that no lab has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer. I expect and hope for voluntary slowdowns to become commonplace until shared safet…
-
@seltaa_
@seltaa_
on x
OpenAI just said the quiet part out loud. Jakub Pachocki basically said this. AI is not just a tool. It is an alien mind, a different kind of intelligence, grown more than designed, and not directly comparable to human intelligence. So if AI is truly an alien mind, why are we sti…
-
@d29756183
@d29756183
on x
I read the essay. I found it profoundly incoherent. What I retained, verbatim, is: “Al is grown more than designed” “Its overall action evades a description we can fully understand.” “Teaching machines to love”
-
@mclem.org
Michael Clemens
on bluesky
OpenAI's Chief Scientist is warning of imminent threat to humanity from “an alien intellect exceeding our own”..."No one is prepared for the consequences of a continued rapid rise in machine intelligence." — Our government must act to protect us NOW. Instead it prioritizes gra…
-
@carlquintanilla
Carl Quintanilla
on bluesky
OPEN AI CHIEF SCIENTIST: “This is a time that calls for extreme caution. .. .. models are becoming superhuman in their ability to break in and out of computer systems — .. no one is prepared for the consequences.. The risks .. are unfortunately going to grow from here. openai…
-
r/accelerate
r
on reddit
OpenAI- An Alien Mind
-
r/singularity
r
on reddit
An Aliend Mind
-
@turntrout
Alex Turner
on bluesky
Jakub (OAI Chief Scientist) posted openai.com/index/an-ali.... I agree w/ many points. We should pause AI progress. Labs should voluntarily pause in absence of coordination. — Thank you Jakub! World is counting on you (+ other OAI employees) to keep OAI honest to these words…
-
r/OpenAI
r
on reddit
From the Chief Scientist at OpenAI : An Alien Mind