The US says OpenAI and Anthropic agreed to give the US AI Safety Institute early access to major new AI models to test and evaluate their capabilities and risks
which Leigh Drogen / @ldrogen : There was no way governments (rightly or wrongly) were going to allow the next steps in AI that are going to fundamentally reshape society without having a massive say in when/how they are unleashed Dean W. Ball / @deanwball : OpenAI and Anthropic have signed memoranda of understanding with the US AI Safety Institute to do pre-release testing of frontier AI models. I would be curious to know the terms, given that these are quasi-regulatory agreements. What happens if AISI says, “don't release”? Brian Schatz / @brianschatz : Glad AI companies listened to us and agreed to share their models with the US AI Safety Institute. I take AI developers at their word that AI could become too powerful for any one actor to manage alone. This is an important first step for transparency and responsible innovation. Eric Geller / @ericgeller : NIST has signed research and testing agreements with OpenAI and Anthropic, a key step in enabling the government to access the companies' latest AI models and probe them for safety and security flaws. https://www.nist.gov/... Kass Popper / @foomagemindset : Of note is that Paul Christiano, head of the US AI Safety Institute, is a former OpenAI employee <revolving door emoji> Adam Gleave / @argleave : Great to see US AISI starting to test models prior to deployment! Jack Clark / @jackclarksf : Looking forward to doing a pre-deployment test on our next model with the US AISI! Third-party testing is a really important part of the AI ecosystem and it's been amazing to see governments stand up safety institutes to facilitate this. https://www.nist.gov/... Zvi Mowshowitz / @thezvi : Highly welcome news that they and Anthropic will now do this. Amazing to see the flood of absolutely vile responses. Bill Gurley / @bgurley : Volunteer regulation. Never seen it before. But I do understand it. “Regulation is the friend of the incumbent.” https://www.reuters.com/... LinkedIn: Tayo Adesanya : Today, the U.S. Artificial Intelligence Safety Institute at the U.S. Department of Commerce's National Institute of Standards and Technology … Jonas Schuett : The U.S. AI Safety Institute signed agreements with Anthropic and OpenAI to collaborate on AI safety research, testing and evaluation. …
Context & Ripple Effects
The arrangement turns earlier voluntary White House commitments into a more specific pre-release access channel for two frontier-model developers. It follows the broader voluntary AI safety commitments made to the White House by major AI companies.
OpenAI had already created an internal advisory structure in which its board could withhold a model release despite management’s view that it was safe. External testing adds a government-facing layer alongside that company governance.
First-order effects
- OpenAI and Anthropic must provide the US AI Safety Institute access to major new models before release, giving the institute an earlier window to assess capabilities and risks.
- The institute gains direct visibility into the labs’ pre-release evaluation process; the reported agreement does not establish that it can block a launch.
Second-order effects
- Pre-release evaluation becomes a more concrete operating requirement for the two labs, likely increasing the importance of documentation, testing procedures, and release-readiness coordination.
- Other frontier-model providers face pressure to show comparable safety-review arrangements if government access becomes a practical benchmark for responsible development.
Third-order effects
- If such arrangements spread, frontier-model releases could increasingly be governed through recurring institutional review rather than solely through company-led voluntary commitments.
- The key unresolved issue is whether early-access testing remains advisory or evolves into a de facto gatekeeping mechanism, shaping who can deploy leading models and on what terms.
The trend: This is one step in the institutionalization of frontier AI governance, with government bodies seeking earlier access to models before public deployment.