OpenAI forms Preparedness, a new team to assess, evaluate, and probe AI models to protect against “catastrophic risks”, including biological and nuclear threats
from today's models to AGI. Goal: a quantitative, evidence-based methodology, beyond what is accepted as possible: https://openai.com/... Forums: r/technews : OpenAI forms team to study ‘catastrophic’ AI risks, including nuclear threats
TechCrunchKyle Wiggers
Context & Ripple Effects
OpenAI had already described planning for a world in which AGI might be created, including a cautious posture toward model risk in its earlier AGI-preparedness framework. The new unit turns that broad stance into a named function focused on evaluating extreme misuse and capability risks.
The development matters because it places biological and nuclear risk assessment closer to the model-development process, rather than treating safety solely as a general policy commitment. Later coverage of a board-level ability to delay a model release shows how such assessment functions can connect to release authority.
First-order effects
OpenAI gains a dedicated Preparedness team to probe its models for catastrophic-risk pathways, including biological and nuclear threats, and to develop quantitative, evidence-based evaluations.
Model development and release decisions now have a defined internal safety input for risks that may not be captured by ordinary product testing.
Second-order effects
The team creates pressure for frontier-model developers to demonstrate comparable evaluation and red-teaming processes, especially where models may enable high-consequence misuse.
Preparedness findings can become a practical gate in deployment decisions: the later release-holdback governance structure illustrates how safety assessments may gain leverage over launch timelines.
Third-order effects
If these functions become standard, frontier AI competition may increasingly include operational assurance capacity—testing, measurement, and escalation paths—not only model capability.
The approach points toward more formalized governance around concentrated frontier-model development, though its effect depends on whether evaluation results meaningfully constrain releases.
The trend: Frontier AI labs are moving from broad safety principles toward embedded, measurable assurance functions that can influence model deployment.
Yes, because THAT'S the problem with LLMs right now. Love this industry that invents future catastrophes by giving them sociopolitical weight in the present, instead of solving problems that it's already causing today. (Yes, I know, these aren't Problems, they're more like... c…
I'm disappointed by the Xiaomi 14 Pro. The screen has massively improved, they've mostly kept the good parts. Mostly being the keyword, since I hate the change from the 1"-type to a 1/1.31" main sensor, and I'm a variable aperture skeptic. It's also too close to the Xiaomi 14. [i…
Xiaomi 14 probably has the best-looking front on any phone out today. 🔥 I really wanna get my hands on one of these, but I've always failed trying to find a Xiaomi flagship here for a good price. [image]
Xiaomi does not like to be left out. Of course, they have a special Titanium version of the Xiaomi 14 Pro! Look at that brushed Titanium frame. 😍 Only available in 16GB RAM + 1TB storage variant which costs CNY 6,499 (₹74,488 / $888) in China. #Xiaomi #Xiaomi14Pro #Xiaomi14 [imag…
we are launching a new preparedness team to evaluate, forecast, and protect against AI risk led by @aleks_madry. we aim to set a new high-water mark for quantitative, evidence-based work. https://openai.com/...
@aleks_madry also, if you are good at thinking about how the bad guys think, try our new preparedness challenge focused on novel insights around model misuse.
If you are worried about risks from frontier model capabilities, consider applying to the new Preparedness team! If we can measure exactly how dangerous models are, the conversation around this will become more grounded. Exciting that this new team is taking on the challenge!
We are building a new Preparedness team to evaluate, forecast, and protect against the risks of highly-capable AI—from today's models to AGI. Goal: a quantitative, evidence-based methodology, beyond what is accepted as possible: https://openai.com/...