How effective altruism, branded by some as a cult obsessed with AI doomsday, influences the White House's, Congress', and think tanks' approach to AI regulation

or rather, it works on the same sort of psychological impulses that have led to millenniarian cults across recorded history 2) For EA to still play an effective and even necessary role in making sure AI is... @8teapi : Risk-ism vs traditional cost benefit analysis is one of the major analytics axes on which most policy has broken down. Bureaucrats are terrified of making value decisions for eg the value of getting SpaceX to Mars is worth more than the fish stocks in the sea next to Boca... Patrick Collison / @patrickc : I'm not an effective altruist, but I find the recent genre of pieces like the below a bit strange. Essentially all of the major AI lab leaders agree that AI potentially poses enormous risks[1], as does a majority of the US public[2]. It's not a crank view... Marc Andreessen / @pmarca : If you are truly concerned about the substance of “AI safety”, you should be most mad at the corporate operators who have cynically hijacked your movement and are burning it down in real time. Joe Gebbia / @jgebbia : Some people think AI will completely destroy humanity. What's is AI doing with its time after everyone is gone? “...versus AI, which will destroy us to the last person.” https://www.politico.com/... Marc Andreessen / @pmarca : However it started, it's becoming obvious in DC that “AI safety” is now a cynical astroturf campaign to grant monopoly/cartel protection to a small set of incumbent big tech companies. Perhaps the least politically appealing program one can imagine. https://www.politico.com/... Tim O'Brien / @_timobrien : Very good @BrendanBordelon piece today on the escalating influence of effective altruism in D.C., and its aversion to scientific, data-driven decision making in favor of weird, cultish fear mongering. < https://www.politico.com/...> [image] LinkedIn: Andrew Lang : The AI debate in Washington this past year has been quite disconcerting. The “effective altruism” (EA) movement, advocating for stringent AI regulations due …

Politico 2024-01-01 Brendan Bordelon

Context & Ripple Effects

The policy debate follows earlier reporting that Open Philanthropy supported staff capacity in Congress and federal agencies to elevate long-term AI-risk concerns, including through funding for policy staff focused on long-term AI risks. It also extends a broader Silicon Valley turn toward effective altruism and catastrophic-AI scenarios documented in the movement's rise in Silicon Valley.

What matters is not a settled regulatory outcome but an influence contest: effective altruism's risk framing is reaching federal institutions while critics argue that “AI safety” rhetoric can serve incumbents or displace more conventional cost-benefit analysis.

First-order effects

White House, congressional, and think-tank AI discussions are more likely to treat long-term catastrophic risk as a live regulatory consideration, alongside nearer-term policy concerns.
Effective-altruism-aligned funders and advocates gain greater agenda-setting relevance in Washington; opponents gain a clearer basis to challenge their legitimacy and policy assumptions.

Second-order effects

AI labs and industry groups face a sharper incentive to distinguish genuine safety commitments from regulation they believe could protect established players, echoing the article's competing claims about “AI safety.”
The regulatory debate becomes more contested over analytical standards: precaution against extreme harms versus cost-benefit approaches and measurable, near-term effects.

Third-order effects

If this pattern persists, AI governance will be shaped not only by technical capabilities but by which networks can translate risk narratives into durable institutional expertise and policy agendas.
That could make transparency around policy funding, expert affiliations, and the evidence behind risk claims increasingly important to AI regulation's legitimacy, though the coverage does not establish that such safeguards will be adopted.

The trend: AI regulation is becoming a contest over which risk framework—catastrophic, economic, or operational—gets embedded in state institutions.

Discussion

@carnage4life Dare Obasanjo on threads
Silicon Valley tech bros are the kind of people who think they're too smart for religion then turn around and create an AI doomsday cult because they're impressed by ChatGPT. It'd be funny if not that they have enough money and connections to cause real harm just from the distra…
@justcodeculture Jeffrey Yost on threads
Insightful article on the Silicon Valley-style Effective Altruism cult's appropriation & distortion of science to pursue harmful agendas on Capitol Hill, agendas that are not altruistic by any reasonable or socially equitable measure. effectivealtruism https://www.politico.com/..…
@arunsees Arun Rao on threads
Noteworthy and accurate article on how the EA “cult” of doomerism is dominating the DC AI policy discourse despite the fact most experts who're actually building AI are concerned and working on more realistic risks (eg CBRN, fairness, trustworthiness, reliability, etc). https://w…
@ylecun Yann LeCun on x
@patrickc It is not true that “essentially all of the major AI lab leaders agree that AI potentially poses enormous risks.” I mean, everyone knows that badly-implemented technology can have terrible effects. That's true of every technology. Turbojets would kill people if unreliab…
@patrickc Patrick Collison on x
@ylecun Maybe “essentially all” and “major” were unhelpfully imprecise. 3/3 of the leaders of the top 3 frontier labs (as defined by performance of the lab's best foundation model) have endorsed this concern. I recognize that some (like you) don't agree. And, to be clear, I th…
@vkhosla Vinod Khosla on x
I was a fan of EA approach when I first read the book years ago. Made rational sense. But applied unconditionally and without scientific basis or first principles probabilistic thinking, and without considering all the basket of risks this new technology entails, and the basket…
@scholars_stage T. Greer on x
I believe it is both possible for 1) EA to have cult like tendencies—or rather, it works on the same sort of psychological impulses that have led to millenniarian cults across recorded history 2) For EA to still play an effective and even necessary role in making sure AI is...
@8teapi @8teapi on x
Risk-ism vs traditional cost benefit analysis is one of the major analytics axes on which most policy has broken down. Bureaucrats are terrified of making value decisions for eg the value of getting SpaceX to Mars is worth more than the fish stocks in the sea next to Boca...
@patrickc Patrick Collison on x
I'm not an effective altruist, but I find the recent genre of pieces like the below a bit strange. Essentially all of the major AI lab leaders agree that AI potentially poses enormous risks[1], as does a majority of the US public[2]. It's not a crank view...
@pmarca Marc Andreessen on x
If you are truly concerned about the substance of “AI safety”, you should be most mad at the corporate operators who have cynically hijacked your movement and are burning it down in real time.
@jgebbia Joe Gebbia on x
Some people think AI will completely destroy humanity. What's is AI doing with its time after everyone is gone? “...versus AI, which will destroy us to the last person.” https://www.politico.com/...
@pmarca Marc Andreessen on x
However it started, it's becoming obvious in DC that “AI safety” is now a cynical astroturf campaign to grant monopoly/cartel protection to a small set of incumbent big tech companies. Perhaps the least politically appealing program one can imagine. https://www.politico.com/...
@_timobrien Tim O'Brien on x
Very good @BrendanBordelon piece today on the escalating influence of effective altruism in D.C., and its aversion to scientific, data-driven decision making in favor of weird, cultish fear mongering. < https://www.politico.com/...> [image]

Chronicles