How Rationalism, a movement pioneered by Eliezer Yudkowsky focused on existential superintelligent AI risks, influenced top AI leaders and their alarmist claims
As the tech elite prepares to make billions from the A.I. revolution, the public mood about this technology has taken a dark turn.Forums:r/ArtificialInteligenceForums:r/ArtificialInteligence:How Scared Should We Be of A.I. Right Now?
The article makes the intellectual lineage behind leading AI executives’ catastrophic-risk rhetoric a subject of public scrutiny. Reaction was divided: some commenters challenged the article’s framing, while others questioned why companies warning of extreme risks should be trusted to build the technology.
First-order effects
Eliezer Yudkowsky and Rationalism become an explicit reference point for evaluating top AI leaders’ claims about superintelligent AI risk, rather than background influences on those claims.
AI leaders’ safety messaging faces closer scrutiny from audiences that see existential warnings as either a serious risk assessment or an ideology-shaped argument.
Second-order effects
White House, congressional, and think-tank AI policy circles already associated with effective-altruist influence face greater pressure to distinguish evidence-led safeguards from any one intellectual movement’s priorities.
AI companies contending with the adoption resistance described in earlier executive warnings about the doomer narrative must manage a communications trade-off: emphasizing catastrophic risks can support safety governance while reinforcing public alarm.
Third-order effects
If this framing persists, the White House, Congress, and think tanks will face a more overt contest between governance agendas centered on catastrophic-risk prevention and those centered on documented labor disruption.
AI labs’ long-term legitimacy may depend not only on their safety commitments, but on whether their risk narratives are seen as broadly grounded rather than tied to a single intellectual network.
The trend: Frontier-AI governance is becoming a contest over which risks deserve institutional priority and who has authority to define them.
Move evidence the winds are shifting.... None of the top three reader picks in the normie NYT comment section are buying Cal Newport's op-ed dismissing catastrophic AI risk.
I disagree with the praise for this column. I think it oversimplifies some stuff. For one thing, a lot of rat- and rat-adjacent folks have been pretty deeply involved in the *practicalities* of AI safety. They don't all talk in that weird sweeping way. https://www.nytimes.com/...
Why are the leaders of AI companies saying “Our products are going to extinguish humanity. Invest in us!” They come from an eccentric Berkeley subculture in which this eschatology is a sacred belief. Good analysis in a NYT op-ed by Cal Newport. https://www.nytimes.com/...
well written, well argued presentation of real world harms of AI without going into histrionics it won't get the same amount of pickup bc it's not scifi coded, but it's just as concerned https://www.nytimes.com/...
This reminds me of that tweet which exaggerated Gramsci's influence that made the rounds the other day. Ideas of this broad type aren't that contingent. “If not for these guys” claims like this have a vaguely conspiratorial vibe.
Great piece that helps explain the seemingly contradictory rhetoric emanating from the closed AI model companies. I agree the #1 actionable take is to hold the companies accountable for their product's behavior. https://www.nytimes.com/...
Cal Newport is unable to interface with the arguments themselves. Perhaps this is because of his religious beliefs? It would be heretical to believe humans could ever build god, or that god would let them. Instead of the arguments, Cal writes 80% true things and 20% ad hominem at…
i have some thoughts on this piece, as an outsider to the effective altruist and rationalist movements (even if we're often aligned on ai): 1. i agree with one of the main criticisms articulated here — ai execs shouldn't be able to evade scrutiny / accountability by describing th…
Here is a link to the op-ed. The article aims to dismiss catastrophic risk but, incredibly, it does not offer a single argument. All it can say is that rationalists want to cast themselves as heroic figures. That's not an argument! https://www.nytimes.com/...
I am not a rationalist by any means, but the heuristic of not trusting weird people about the possibilities of new technologies has a pretty lousy track record. https://www.nytimes.com/...
Right but that's because they fundamentally misunderstand the risk. The problem is not that these models are going to act autonomously to hack and attack, it's that the guardrails and protections by the companies providing them are insufficient. These are very different issues!
Why do AI leaders talk as if their products will destroy society as we know it? Because they've drunk the Rationalist kool-aid. We need less apocalypticism and more focus on mundane product safety issues. Cal Newport with some great insights in the NYT: https://www.nytimes.com/..…
2021: “Don't listen to the rationalists; the real AI danger is immediate threats like AI bias” 2026: “Don't listen to the rationalists; the real AI danger is immediate threats like swarm breakouts and bioterror”
Nat, why are you conflating opposition to a NorCal doomsday ideology with opposition to AI safety? You wrote: “it's true that rationalists are *more* focused on catastrophic risks from ai right now — but it's worth asking why, and this piece doesn't really do that.