Discussion

• @andrewcurran_ Andrew Curran on x

  To sum up; Anthropic, OpenAI, Google, Microsoft and xAI all have new pre-release screening agreements with CAISI.  We don't know the details of the new rules yet.  I assume they will be announced with the AI executive order and the AI policy memo, both of which we may get today.

• @_nathancalvin Nathan Calvin on x

  Meta has a partnership with Scale, which itself works with CAISI. Where is @Meta's agreement with CAISI? They are trying to be a real frontier AI developer and should act like it! [image]

• @jtillipman Jessica Tillipman on x

  Piecing all the news together (last week's Pentagon deals + CAISI pre-release screening agreements), these developments show how much leverage the federal government has over frontier AI companies. The government may not need a freestanding statutory mandate to require model

• @jimstewartson @jimstewartson on x

  lol what could go wrong? this industry is just a psychotic as the regime. it's just a giant frantic money grab before the music stops. they're falling over themselves to be in position for a bailout.

• @_nathancalvin Nathan Calvin on x

  The total adjusted AI related annual revenue of companies partnering with CAISI is almost certainly already above $100 billion (depending how you count cloud). Proposals in Congress fund CAISI at $10,000,000 per year, or 1/10,000th as much. (Credit to 5.5 for graphic.) [image]

• @nist @nist on x

  NIST's Center for AI Standards and Innovation (CAISI) signs expanded collaborations with @GoogleDeepMind, @Microsoft, and @xai for pre-deployment evaluations and other research to support frontier AI national security testing. Learn more: https://www.nist.gov/...

• @the_danielking Daniel King on x

  Hmm. If only we could fund an extremely talented team—within government—to carry on the mission of vetting AI models. 🤔

• @deredleritt3r Prinz on x

  @AndrewCurran_ OpenAI and Anthropic renegotiated their existing partnerships with CAISI also. This seems to be the least intrusive way for the U.S. government to review models before their release, so here's hoping that Trump's actual proposal does not go much farther than CAISI …

• @s_oheigeartaigh @s_oheigeartaigh on x

  I don't quite understand it, but it sure seems like good news, and all good news these days should be celebrated without reservation. Some really great people at CAISI too. Well done America.

• @allinallnotbad Samuel Roland on x

  ...Happy to be corrected, licensing is not my area of expertise; but if these are truly voluntary agreements, even if contractual, that seems like a reasonable balance of power.  Government gets oversight, but not veto power, as the companies can withdraw/break the contract at an…

• @googlepubpolicy @googlepubpolicy on x

  We're partnering with CAISI on frontier AI national security testing. More details ↓ https://www.nist.gov/...

• @shakeelhashim Shakeel on x

  My (completely speculative) read of the CAISI announcement, coupled with the NYT/Politico reporting, is that the WH is gearing up to announce *mandatory* predeployment evals, and is encouraging companies to sign these voluntary agreements beforehand so the whole thing seems less …

• @matvelloso Mat Velloso on x

  This was inevitable. 1-Models becoming more powerful 2-Therefore, they also become more dangerous 3-Logically, governments become increasingly concerned and want more control and regulations about it 4-Therefore, the way models have been launched so far won't work anymore. The

• @emollick Ethan Mollick on x

  In addition to the CAISI evaluation, it would be useful if NIST conducted public tests of AI abilities as an independent evaluator - though those obviously should not be pre-release tests & can be done when models are public.  Independent testing is important & getting expensive.

• @shakeelhashim Shakeel on x

  Yes — why is Meta not included here?

• @ashleyrgold Ashley Gold on x

  Important: CAISI told me similar deals with Anthropic and OpenAI, struck in 2024, are still going, under new agreements

• @thomas_woodside Thomas Woodside on x

  Huge news! Now Anthropic, Google, Microsoft, OpenAI and xAI are participating in the program. One frontier AI company is conspicuously absent...

• @blainedilli Blaine Dillingham on x

  Great that CAISI is doing voluntary partnerships, and we should beef up their funding as much as possible. But if we go from voluntary agreements to mandated auditing, we have to ask who has the best incentives and technical capacity to do those audits well. Government could

• @joebotxyz Joe Allen on x

  As we reported a few weeks back, OpenAI recommended the Center for AI Standards and Innovation (CAISI) as the primary interface between tech companies and the US government. Today, something like that deal appears to be in the bag.

• @deanwball Dean W. Ball on x

  No one should overindex on the news about CAISI testing frontier models. They already had those agreements with xAI, OAI, and Anthropic. The fact that the admin is highlighting the news is interesting, but “pre-deployment licensing” is not a foregone conclusion (and that's good)

• @bitcoinconner Conner Brown on x

  Interesting that Meta—the only lab that has consistently championed the importance of Open Source— is conspicuously absent.

• @basedjensen @basedjensen on x

  this is going to slow down releases by a lot

• Chris Lehane Chris Lehane on linkedin

  It's great to see more companies joining us today in working with the US Center for AI Standards and Innovation (CAISI) on national security testing and evaluations …

• Christopher Okpala Christopher Okpala on linkedin

  AI is officially entering the RMF world and most people are not paying attention.  —  The National Institute of Standards and Technology …

• @ericjgeller.com Eric Geller on bluesky

  NIST's AI security center is partnering with Google, Microsoft, and xAI to evaluate their new AI models before deployment: www.nist.gov/news-events/...  “To date, CAISI has completed more than 40 such evaluations, including on state-of-the-art models that remain unreleased.”

• @karlbode.com Karl Bode on bluesky

  all of the stories on this are the same in that way  —  Republicans and Trump are given completely unearned credibility on everything from AI policy making to cybersecurity, as if we haven't witnessed an historically corrupt, authoritarian clown show the last year

• @karlbode.com Karl Bode on bluesky

  again, I'm fascinated with U.S. journalism that crafts a weird alternate reality for the readership where our regulators still function in the public interest, policy is coherent, corruption doesn't exist, and Trump cares about security (or anything outside his own wealth and pow…