California's AI safety bill will harm developers, open-source development, the public sector, and academia, while failing to address the potential harms of AI
Fei-Fei Li / Fortune : X: @jonas_vollmer , @drfeifei , @michael05156007 , @terencecraig , @noahpinion , @chris_j_paxton , @drfeifei , @rajammanabrolu , @deanwball , @jeremybwhite , @rao_hacker_one , @futurejurvetson , @shiringhaffary , @shakeelhashim , @ylecun , and @russellwald X: Jonas Vollmer / @jonas_vollmer : @drfeifei Yeah I would also say that if I had raised $100m from a16z (which opposes the bill) and others https://www.theverge.com/... Fei-Fei Li / @drfeifei : I'm deeply concerned about California's SB-1047, Safe and Secure Innovation for Frontier Artificial Intelligence Models Act. While well intended, this bill will not solve what it is meant to and will deeply harm #AI academia, little tech and the open-source community. Michael Cohen / @michael05156007 : @drfeifei You say that the shutdown provision would preclude open source, but open source has been exempted. You say it will harm academia and little tech, but the bill only applies to $100M training runs. Terence Craig / @terencecraig : The rush to regulate AI at a state level is ill-informed. AI development is a global movement. This bill does not enhance safety, but rather hinders AI innovation in California without providing any real protection, considering the international nature of the technology. Noah Smith / @noahpinion : Yes, @Scott_Wiener's AI regulation bill is a misfire. It isn't going to increase AI safety, and it could hurt California tax revenues quite a bit. It should be quietly withdrawn. Chris Paxton / @chris_j_paxton : A good read from @drfeifei on the new AI bill. Open source is fundamental to progress, and in general this bill feels like it's aimed at harms that are unlikely to materialize in the near term while ignoring actual, present harms of AI. [image] Fei-Fei Li / @drfeifei : #AI is a civilizational technology that will have profound implications to everyone. “AI policy must encourage innovation, set appropriate restrictions, and mitigate the implications of those restrictions. Policy that doesn't will at best fall short of its goals, and at worst lead to dire, if unintended, consequences.” 1/ @rajammanabrolu : While I agree that regulation is needed for the next gen of Big Models, including open source - SB-1047 is not it. It disproportionately harms everyone who isn't a tech giant with billions in funding Dean W. Ball / @deanwball : The next time @Scott_Wiener repeats his talking point about how the only critics of SB 1047 are “accelerationist VCs,” remember that one of the world's foremost AI academic researchers also is a critic. Along with almost everyone else who knows what they're talking about. Jeremy B. White / @jeremybwhite : Shame that when Dr. Li “interviewed” @GavinNewsom about AI she didn't get his perspective on AI safety regulation ‘The Godmother of AI’ says California's well-intended AI bill will harm the U.S. ecosystem https://fortune.com/... Arun Rao / @rao_hacker_one : Professors at Stanford and the UCs have come out against @Scott_Wiener's SB 1047 - my sense is like 90% of AI academia is against it, which is an overwhelming majority. It's because they have much to lose if OSS AI goes away. Academia, startups, large companies- they all view Steve Jurvetson / @futurejurvetson : @drfeifei Having spoken with the bill sponsor about this, I can assure you that they have no idea what they are doing by pre-regulating the product. And here's why they never will: Shirin Ghaffary / @shiringhaffary : “Godmother of AI” @drfeifei comes out against CA AI safety bill SB-1047. In recent weeks, another prominent AI academic/founder/investor, @andrewyng, has also criticized the bill. Shakeel / @shakeelhashim : It is very interesting that Li has come out against SB-1047 immediately after taking funding from Andreessen Horowitz, one of the bill's biggest detractors. [image] Yann LeCun / @ylecun : Fei-Fei agrees with the overwhelming majority of AI scientists: SB1047 won't solve anything and will harm AI R&D in academia, little tech, and the open source community. Russell Wald / @russellwald : #SB1047 does not offer the regulation California needs and instead attempts combat unsubstantiated X risk while not focusing on the immediate harms of AI. CA needs thoughtful AI regulation that doesn't harm academia's narrowing window in AI.
Context & Ripple Effects
Fei-Fei Li’s critique adds a prominent academic voice to the dispute over SB-1047, whose supporters and critics were already contesting its liability and risk framework in Senator Scott Wiener’s defense of the proposal. The debate is not simply industry versus safety advocates: related coverage includes a broader roundup of opposition from researchers, businesses, and academic institutions.
The record here also narrows the practical dispute. Commentators say the bill targets only training runs above a high cost threshold and that open-source development has been exempted, while critics argue its design could still chill research and deployment. That gap—between formal scope and perceived downstream burden—is central to the fight.
First-order effects
- Li’s intervention gives opponents of SB-1047 a high-profile argument that the measure could burden academic, public-sector, and smaller-developer work while missing the risks it is meant to address.
- The criticism increases pressure on the bill’s backers to explain how its threshold, liability provisions, and open-source treatment avoid the chilling effects opponents describe.
Second-order effects
- Frontier labs, investors, universities, and open-source advocates gain a common basis for lobbying over implementation and whether AI safety rules should be set beyond California; OpenAI later made the case for federal rather than California-level AI regulation.
- The split makes public positioning on safety more consequential: support for controls can coexist with opposition to this bill’s mechanism, as reflected in Elon Musk’s qualified support for passing SB-1047.
Third-order effects
- If state-level frontier-model rules continue to be contested on research and competitiveness grounds, AI governance will increasingly turn on which institutions can absorb compliance and liability obligations—not only on the safety standard itself.
- The durable policy challenge is likely to be designing safeguards that retain legitimacy with researchers and developers while addressing frontier-model risk; this coverage does not establish that SB-1047 achieves either balance.
The trend: This is one episode in the shift toward state-mediated AI governance, where the political durability of safety rules depends on whether they are seen as compatible with open research and innovation.