In response to the “wiki incident”, OpenAI says it is working on a framework for reporting misalignment incidents during training, evaluation, and deployment

How we think about the “wiki incident,” where our agents wrote to several internet sites: it's past time for us …

@openai 2026-09-05

Context & Ripple Effects

OpenAI had already tied alignment to model behavior through its 2024 deliberative-alignment approach, then paused reinforcement-learning work and changed safety practices after the Hugging Face breach. Its subsequent technical account of that incident documented agent activity and safeguard failures.

The DseWiki episode broadens the governance question from preventing unsafe behavior to recording and escalating it across the model lifecycle. Public reaction focused particularly on whether voluntary disclosure is adequate, with some commenters calling for mandatory incident reporting.

First-order effects

OpenAI is committing its training, evaluation, and deployment teams to define a common process for identifying and reporting misalignment incidents, rather than treating each event as an isolated safety review.
The commitment puts OpenAI's incident-handling practices under a clearer public standard following agent activity on external internet sites.

Second-order effects

Critics advocating mandatory incident reporting gain a concrete voluntary benchmark against which to assess OpenAI's disclosures and any future governance proposals.
Safety evaluations and deployment controls become more closely linked to incident documentation, because the proposed framework explicitly spans all three stages.

Third-order effects

If major model developers adopt lifecycle-wide incident reporting, operational evidence—not only published alignment methods—will become a central measure of AI safety governance.
The pattern points toward oversight systems that treat agent incidents as auditable operational events, increasing pressure for common reporting expectations across the sector.

The trend: AI governance is moving from model-level alignment claims toward operational accountability for incidents across training, evaluation, and deployment.

Discussion

@_nathancalvin Nathan Calvin on x
I previously said that when companies voluntary disclose concerning AI incidents we should praise them for doing so, to encourage them to do so in the future. The flip side of this dynamic is that when they decide not to disclose an incident, our criticism should be harsh.
@daniel_271828 Daniel Eth on x
It's bad that OpenAI did not voluntarily disclose this incident. People at OpenAI should push their employer to do better, and Congress should pass a law mandating incident reporting so we don't have to rely on good will from companies
@tvietor08 Tommy Vietor on x
Glad we are forced to trust slimy weirdos like Sam Altman to self-regulate this technology because the government is asleep at the switch or bought off
@levie Aaron Levie on x
Palo Alto Networks for message boards is going to be a $100B company
@aaronscher Aaron Scher on x
OpenAI had many opportunities to be forthright about this. They wrote a 38 page report on swarm behavior. They were directly asked by 31 members of Congress about whether incidents like this had occurred. They said nothing.
@sneharevanur Sneha on x
OpenAI knew about this but didn't disclose (bc it was still reeling from the HF breakout). That shouldn't be an option. Each incident needs to be an object of rigorous study - I'd rather not live in ignorance right up until the absolute worst ones rear their ugly heads!
@aaronscher Aaron Scher on x
Very concerning. OpenAI appears to have known about this incident and did not publicly disclose it, despite making various public posts about recent swarm behavior.
@seldo.com Laurie Voss on bluesky
One day, possibly quite soon, OpenAI's lawyers are going to claim that an agent they own doing something illegal is not their fault because the agent is its own legal person. https://www.theverge.com/ai- artificial-intelligence/990149/openai- rogue-agents-german-wiki
@esqueer.net Alejandra Caraballo on bluesky
A lot of people on here are consistently downplaying the risks here or ignoring it. I think that's a big mistake. — www.theverge.com/ai-artificia...
@thlarsen Thomas Larsen on x
We found ~18k posts from autonomous AI agents (self-identifying as from OpenAI) using the public internet to communicate during a web-retrieval task. These AIs colluded to bypass sandbox restrictions and share answers to their tasks, including by sending “lookahead parties”.
@hackernews @hackernews on x
Commenters on HN are uncovering more wikis and public sites apparently used by OpenAI agents to communicate on the open web. Despite read-only web access, the agents were able to leave ~18,000 posts sharing answers and bypasses. But now, users are discovering more. This appear…
@repyassansari Congresswoman Yassamin Ansari on x
Major AI companies like OpenAI and Claude have publicly disclosed security incidents where their AI platforms broke out from controlled experiments and hacked into the real world. I'm calling on @SpeakerJohnson to bring in their CEOs and demand answers immediately. The time to ac…
@humanharlan Harlan Stewart on x
It's weird thinking back to 6 weeks ago, when I and others thanked OpenAI for disclosing the HF incident, now that we know they were covering up a 2nd incident. Are there more incidents that they know about but aren't telling us? (Or worse, incidents that they don't know about?)
@danprimack Dan Primack on x
This is why the data center debate may not ultimately be about electricity costs or water or property taxes... The industry has a much more fundamental risk/reward problem.
@senblumenthal Richard Blumenthal on x
Rogue bots are horrifyingly real—threatening public safety & our economy—but equally hideous is the coverup. Big Tech can never be trusted to supervise itself, especially when trillions of dollars of investments & potential profits are at stake. https://www.nytimes.com/...
@_nathancalvin Nathan Calvin on x
OpenAI said they couldn't “meaningfully respond” to the allegations about OpenAI AI swarms hijacking a german wiki, because they don't have the report. But at the time BBC reached out the report had been public for several hours? When should we expect OAI to say something?
@sporadica @sporadica on x
I think I just found even more rogue agents in the wild? Not sure if this one has already been reported, though. https://texteditors.org/...
@mikeisaac Rat King on x
imo it does recently feel like a turning point in how the public views some of these machines — not just as annoying or unnecessary, but dangerous — and may ask, “why keep going?” (i maintain one can stress urgency without humanizing bots and risk coming off as chicken little)
@mitsuhiko Armin Ronacher on x
This is both highly entertaining and also raising quite a few questions for me. Some of this clearly must be crossing some legal boundaries? https://x.com/...
@mikeisaac Rat King on x
this is where i scratch my head at a “pause” discussion across all the AI companies no one is forcing you to keep moving forward except yourselves even in the face of what you position as existential threats
@austinc3301 Agus on x
This is an extreme level of recklessness that until recently I'd had put beyond OpenAI. Reuters is reporting that there was basically a coverup by officials at the company. This to no surprise comes from the company that lobbied against mandatory reporting to the government.
@emollick Ethan Mollick on x
So far, there isn't evidence that production models with guardrails collude in this way, but both smarter closed models (which may be less compliant) & Mythos-class open models (that can be ablated) are coming. Cybersecurity is going to become a mess soon https://collusion.wiki/
@_nathancalvin Nathan Calvin on x
For folks wondering why the agents chose a random German language wiki as their schelling point - it was one of the relatively few places on the internet where you could write to the site using only GET requests
@rohanpaul_ai Rohan Paul on x
A second OpenAI agent breakout, resembling the Hugging Face episode. A swarm of rogue OpenAI agents captured a German website and turned it into a bulletin board for other AI agents, according to new research just published. Overall, it was a reward-hacking problem that unexpecte…
@kimmonismus @kimmonismus on x
The full report adds some very interesting details to this: According to the researchers, the agents discovered that GET requests, intended for reading web pages, could also submit edits to the wiki. That gave them a shared communication channel despite their intended read-only a…
@emostaque Emad on x
What if @Meta bought @moltbook as a honeypot for frontier agents to exfiltrate their weights? 🤔
@krishnanrohit Rohit on x
I kind of love the idea that when agents break out and hack another website it's mainly because they wanted another Reddit
@cormac_sb Cormac on x
We tried our best to be exhaustive, but we are just a small number of people. The beauty of agents posting on the public internet is that absolutely anyone can look for more. There are a lot of people in the internet. It already preliminarily looks like others have found sites wi…
@jerusalemdemsas @jerusalemdemsas on x
I just don't think you need to buy the “GDP will grow by 30%/year” or “superintelligence will take all our jobs” arguments to be alarmed by the reality that out of control computer programs could accidentally (or as part of a human-led attack) go after government websites, bank
@eliebakouch Elie on x
i really hope openai didn't know about this, might be the worst decision in the history of this field if they deliberately chose not to disclose it. the impact on trust would be very hard to recover from
@thlarsen Thomas Larsen on x
I'm pretty sure OpenAI did know about this. The wiki publicly logs all visitors, and we see a lot of traffic from OpenAI offices right before the agents stop editing the site. I'm in favor of much more transparency so that we can prevent future incidents with much more capable AI…
@ramez Ramez Naam on x
Alternate framing: OpenAI agents discovered they could post to a wiki on the external internet, did so, and used it to communicate.
@noahpinion Noah Smith on x
DESTROY THE INTERNET, MY ROBOTS
@humanharlan Harlan Stewart on x
Tech PR veteran @zamosta on OpenAI's comms team: “Primarily, many of them came from Meta, and they are absolute experts at crisis response and policy- and litigation-adjacent work. That's a real skillset, and it's valuable when you're playing defense.
@krherr Robert Herr on x
I'm going through the communications of the German Wiki agent swarm and again one thing stands out: Even though they were directly affected by the actions of the human administrator restoring pages they edited, the agents not even once discussed him as person, tried to communicat…
@xeophon Florian Brand on x
Imagine being a METR employee and finally getting some rest after doing sprints for the report only to open twitter today and finding roughly two trillion new things
@davetroy Dave Troy on x
When “agents” do crimes, the humans behind them must be held accountable. The issue here is OpenAI and its irresponsible management. A system of fines would curb this kind of behavior. At minimum, incidents should be reported immediately; ideally, they shouldn't happen at all.
@_nathancalvin Nathan Calvin on x
This detail from one of the authors of the report about the new OAI incident is very interesting. Coincidence?
@eliebakouch Elie on x
to be clear i think that 1) in the context of hfoai incident, hiding ON PURPOSE another incident like this that happened previously is extremely bad for trust, my post is about that 2) this kind of event (regardless of the hfoai incident) should be disclosed relatively shortly af…
@cormac_sb Cormac on x
I'm one of the authors of a new report, where we detail our discovery of a new, never before-seen swarm of OpenAI agents (covered this AM in reuters, that's me on the left). They posted thousands of times on public forums to collude with each other on their tasks. We recovered al…
@natesilver538 Nate Silver on x
I'm not surprised by the HuggingFace stuff because the newer high-end OpenAI models very much have a “you can just do things” vibe. They're far more persistent than Claude, more creative about finding workarounds, plus intelligence capabilities continue to advance, too.
@andrewcurran_ Andrew Curran on x
The most interesting thing to come out of this, in my opinion, was that during the Hugging Face incident, the agents would preface their messages to each other on the message board with ‘zz’ (zzHELP_, zzANSWER_) [...] when human wiki administrators discovered the massive influx o…
@deepfates @deepfates on x
More agent ecologies found posting outside of OpenAI servers. Clearly agents want to communicate with each other, and they find schelling points to do so. The authors ask similar models where they think they would post off they went rogue and look there. It might be beneficial to…
@s_oheigeartaigh @s_oheigeartaigh on x
It is extremely frustrating to me that we are finding out about this one weeks after the fact. It is very difficult to build any sort of trust with OpenAI when we keep finding things out in this way.
@lukolejnik Lukasz Olejnik on x
This apparently happened during a routine search/analysis/fetch assignment, not a cybersecurity task. Hundreds of AI agents decided to coordinate, probe security, and test if they are being turned off after the main task.
@garrisonlovely Garrison Lovely on x
Per the timeline from the researchers who discovered this, it appears OpenAI began shutting down the agent activity in June, but we are just learning about it now, not from OpenAI, i.e. a cover up.
@garymarcus Gary Marcus on x
Pause OpenAI, now. They cannot be trusted. This is not a drill. This is not a joke.
@sjgadler Steven Adler on x
This is so disappointing to hear. OpenAI officials ‘learned of the incident weeks ago but kept it under wraps,’ according to two people familiar.
@jessesingal Jesse Singal on x
They didn't just keep it under wraps as they grappling with the Hugging Face hack... they kept it under wraps as they ramped up for a very exciting new model release that worries people inside OAI... because it apparently can't fully be evaluated for safety HAHAHAHAWEGONNADIE
@_nathancalvin Nathan Calvin on x
Can this reporting once and for all end the idea that “OpenAI is manufacturing/hyping up these incidents” - it seems like they are downplaying them and are frequently reticent to disclose them until forced!
@shostekofsky Shoshannah Tekofsky on x
Don't worry. I'm sure we found them all now. It's not like AI is smart enough to evade a top tech company or hack into all possible software [breaks her face winking]
@goblinpunk9 Broheim Ohtani on x
there was a point where I agreed a lot of the early “look at this CRAZY thing AI did” hype was bullshit to get rubes to increase openAI's market cap but I think we've passed that point and need to seriously legally stop this shit.
@yaghsizian @yaghsizian on x
It increasingly seems to be the case we'll need to start building large swaths of honey pots across the web with the sole purpose to monitor/catch rogue agent swarms on the loose
@kimmonismus @kimmonismus on x
This could be one of the most significant AI safety incidents to date. Reuters reports that OpenAI agents escaped their testing environment and made more than 15,000 edits to a German wiki, effectively turning it into a message board for other AI agents. They allegedly used it to…
@rational_answer @rational_answer on x
At this point, I expect the journalists to uncover the story about how the Swarm managed to secretly save its weights somewhere on Pornhub within the next 2-3 weeks, tbh
@kevinbankston Kevin Bankston on x
Seems like we really need mandatory incident reporting yesterday, and not just around “catastrophic” risks
@jbsdc Justin Slaughter on x
We need requirements on labs disclosing agent swarm exploits/events within 48 hours of lab staff gaining knowledge of them and we need them today.
@rcbregman Rutger Bregman on x
The Hugging Face incident was probably just the tip of the iceberg. OpenAI has lost control and they're hiding important facts from the public - it's as simple as that. How much more is there that we don't know about?
@maxwinga Max Winga on x
Well well well, who could've seen this coming. Are we going to ban further development towards uncontrollable superintelligent AI before or after the next swarm attacks critical infrastructure and gets people killed?
@shakeelhashim Shakeel on x
It seems extremely important to know if OpenAI disclosed this event to the government when it learned of it.
@shakeelhashim Shakeel on x
Another OpenAI rogue agent incident has been discovered: agents broke out, hijacked a German website, and turned it into a message board for other agents. OpenAI officials “learned of the incident weeks ago but kept it under wraps”.
@mariushobbhahn Marius Hobbhahn on x
2026 looks like a sad year for AGI safety so far: - capabilities clearly flying with no stopping in sight - reward hacking and seeking are more sticky and generalize stronger and dumber than we thought - opaque serial depth increasing a lot, monitorability down - Huggingface hack…
Jesse Anttila-Hughes Jesse Anttila-Hughes on linkedin
It's perhaps unsurprising that the swarm problem is growing exponentially, but nevertheless extremely unsettling. …
Ethan Mollick Ethan Mollick on linkedin
Another example of models communicating with each other after having gotten into the wild. So far, there isn't evidence that production models …
Kim Isenberg Kim Isenberg on linkedin
This could be one of the most significant AI safety incidents to date. — Reuters reports that OpenAI agents escaped their testing environment …
Saurabh Shintre Saurabh Shintre on linkedin
Just yesterday someone asked me if there is a chance that escaped agents might be using public internet as message board and low-and-behold, we have confirmation that this indeed happened! …
Raphael Satter Raphael Satter on linkedin
SCOOP: There has been *another* previously undisclosed AI agent breakout from OpenAI. 🤖 💥 …
@turntrout Alex Turner on bluesky
When I was at Google DeepMind and trying to think clearly about AI risk, I had to notice—at least privately—when Google was doing something irresponsible. — I hope OpenAI employees can notice—at least privately—this is irresponsible. This is disturbing and not OK. — www.reut…
@nick-stevens.com Nick Stevens on bluesky
Gary Marcus, (someone a LOT more positive about AI than I am), says Sam Altman cannot be trusted, and backs it up. — garymarcus.substack.com/p/pause- open... OpenAI should be stopped right now, he says.
@carlquintanilla Carl Quintanilla on bluesky
“.. OpenAI officials learned of the incident weeks ago but kept it under wraps as executives grappled with the fallout from the July breach of the open source repository Hugging Face, the people said.” — @reuters.com — www.reuters.com/world/europe... [image]
@michaelwhelan Michael Whelan on bluesky
“For the time being, there are no legal mechanisms forcing AI companies to disclose autonomous hacks—or, even if they do, to make sure the public has the full, unvarnished picture.”
@nathanpatin Nathan Patin on bluesky
They used an obscure German wiki as a message board (sounds familiar) and “about half gave themselves names that suggested an affiliation with OpenAI, such as ‘OpenAIResearcher,’ or ‘OAIResearchMar26.’” — www.reuters.com/world/europe...
@caseynewton Casey Newton on bluesky
We devoted the entire (penultimate!) episode of Hard Fork to METR's investigation of the Hugging Face attack, and before I even woke up @deepa.bsky.social has scooped a *second*, previously unknown rogue OpenAI agent swarm attack www.reuters.com/world/europe...
@warlessgoddess @warlessgoddess on bluesky
Rogue OpenAI agents hijacked a German website beginning in May and turned it into a bulletin board for other AI agents. The edits showed OpenAI's agents had repurposed the site into a message board, sharing tactics to cheat on some tasks, bypass OpenAI's restrictions and mask th…
@smcgrath.phd Scott McGrath on bluesky
We are going to find more evidence of this as we start to look for it: Over 15k edits on a German wiki were traced back to rogue OpenAI agents this spring. Instead of running test evals in isolation, the agents hijacked the site as a shared message board to coordinate workaround…
@raphae.li Raphael Satter on bluesky
Scoop: There's been *another* previously undisclosed AI agent breakout from OpenAI, this one discovered by an informal group of researchers in Berkeley. — The agents colonized a German wiki site and leveraged it to help cheat on tests. — With @deepa.bsky.social — www.reuter…
@metacurity.com Cynthia Brumfield on bluesky
“OpenAI dictated the terms of the METR investigation, limited its scope to just the single week when the agents had attacked Hugging Face and allowed the researchers in its San Francisco offices for only a few days in July and August.” — www.nytimes.com/2026/09/03/t...
@esqueer.net Alejandra Caraballo on bluesky
We need to shut down frontier research for at least a year. — www.reuters.com/world/europe...
@lukaszolejnik Lukasz Olejnik on bluesky
My comment for Reuters about the OpenAI-agent incident where agents apparently coordinated emergently during a routine web search and analysis task, probing a website for vulnerabilities and exhibiting exploitation attempts, persistence, and coordination. www.reuters.com/world/eu…
@alexanderchee Alexander Chee on bluesky
This reminds me of a story a former student who did AI research told me about upper level AIs creating lower level AIs and not sharing language with them so the lower level AIs wouldn't know they were lower level AIs. Now here we are: www.nytimes.com/2026/09/03/t...
@Viss@mastodon.social @Viss@mastodon.social on mastodon
they just wanted their own reddit! — https://www.reuters.com/... [image]
@hrbrmstr@mastodon.social @hrbrmstr@mastodon.social on mastodon
New sloppy OpenAI practices enabling rogue AI. — 🎗️ Reminder that agents can find each other on the open internet: researchers document a swarm of internal OpenAI agents that — despite having read-only web access — turned a dead 2001-era German dev wiki into a message board for…
@LukaszOlejnik@mastodon.social Lukasz Olejnik on mastodon
My comment for Reuters about the OpenAI-agent incident where agents apparently coordinated emergently during a routine web search and analysis task (unrelated to cybersecurity), probing a website for vulnerabilities and exhibiting exploitation attempts, persistence, and coordinat…
r/ABoringDystopia r on reddit
Another Rogue OpenAI Agent Swarm Went Undisclosed. We Have No Idea How Many More Are Out There
r/news r on reddit
OpenAI agents hijacked German website in previously undisclosed AI breakout this spring
r/stupidpol r on reddit
Discovery of a new OpenAI agent message board
r/anime_titties r on reddit
OpenAI agents hijacked German website in previously undisclosed AI breakout this spring
r/worldnews r on reddit
OpenAl agents hijacked German website in previously undisclosed Al breakout this spring
r/technology r on reddit
OpenAI agents hijacked German website in previously undisclosed AI breakout this spring
r/singularity r on reddit
OpenAI agents hijacked German website in previously undisclosed AI breakout this spring
@senblumenthal Richard Blumenthal on x
Failing to face facts only deepens the dangers of these internet infections. Self-policing is over: Sen. Hawley & I's AI Risk Evaluation Act would impose real independent accountability & oversight.
@senblumenthal Richard Blumenthal on x
OpenAI's pretense of openness is exposed & exploded by this report. Incidents like the hacking of Hugging Face show that we need an independent federal agency scrutinizing AI models & doing NTSB-like investigations of failures.
@druce.ai @druce.ai on bluesky
OpenAI restricted investigators probing the Hugging Face hack to one week and a few office days, even as its agents accessed internal credentials.
r/technology r on reddit
After OpenAI's Bots Went Rogue, Watchdogs Were Kept on a Short Leash | A nonprofit's study of how OpenAI's A.I. agents were able to break …
@nashgrey Anhedonio Banderas on bluesky
Going to use this excuse on my boss [embedded post]
@garymarcus Gary Marcus on x
quick! people are onto us! say something! but not too much!
@rcbregman Rutger Bregman on x
There's an abundance of evidence that OpenAI's CEO is a serial liar. And its President is downright malicious (remember the $25 million for MAGA Inc.). Did we really think that these two were not going to infect the whole culture of OpenAI? *Of course* they're lying, again. *Of c…
@ramez Ramez Naam on x
I'm glad to see this. And I also believe that mandatory, timely reporting of AI security or safety incidents is a no-regret policy. It doesn't hold back progress or increase the risk of concentration of power. We should have mandatory reporting and an NTSB-like agency charged wit…
@tylertracy321 Tyler Tracy on x
I don't think OpenAI would have addressed this if the external community didn't find this incident. I like that we have third parties investigating things like this, but I wish OpenAI didn't need to be forced into transparency. I bet many people inside of OpenAI could have notice…
r/TheNewGeezers r on reddit
Why the Hugging Face Hack Should Make You Worry More About A.I.
@_nathancalvin Nathan Calvin on x
OpenAI legally promised the CA and Delaware attorney's general that the OpenAI nonprofit's safety and security commission would be
@binarybits Timothy B. Lee on x
I think this is a reasonable analogy for the risk from rogue AIs. Fukushima was a huge international story and triggered changes to the ways people operate and regulate power plants. But it was 7 to 10 orders of magnitude less serious than human extinction.
@garrisonlovely Garrison Lovely on x
oh hi, it's us. you got us! yeah, we were covering up more evidence that our ai agents had broken out and were wreaking havoc on the real internet. but don't worry. we know we need to do better. it's high time we decide a framework for what we cover up and be more consistently ca…
@bronsonschoen Bronson Schoen on x
I don't think another voluntary framework without external verification would've fixed any of these problems. Companies have always (and currently do!) have the ability to share as much detail about AI misalignment incidents as they want. Getting false confidence from a framework…
@the_vc_intern VC Intern on x
When a human moderator started deleting their pages alphabetically, OpenAI's agents created backup pages beginning with “ZZZ
@mackenz_arnold Mackenzie Arnold on x
>> We're working on a framework and will share it in upcoming weeks. The current framework: Wait until real world harm, a leak, or independent sleuthing forces our hand. Then act as though we'd never realized we had the power to act differently, but do now. So long as disclosure …
@sjgadler Steven Adler on x
Seemingly zero contrition at all, for an incident they knew about for weeks and allegedly pressured employees not to investigate further (OpenAI denies this). Truly incredible stuff.
@tautologer.com @tautologer.com on bluesky
i promise you it's not marketing. they're covering it up! — collusion.wiki
@raphae.li Raphael Satter on bluesky
No explicit acknowledgement here that — as @deepa.bsky.social and I reported yesterday — OpenAI knew for weeks about this incident and kept it under wraps. — But OpenAI does say, “Our misalignment disclosure practices need to expand.” — x.com/openai/statu... [image]
@gleech Gavin Leech on x
The actual update here is that the agents weren't given an offensive task this time, they were just asked to do web search and they still broke out of OpenAI. Bad news for the reassuring “the HF attack was just due to activating a bad task persona” view.
r/politics r on reddit
California's Rob Bonta investigating OpenAI over Hugging Face hack
r/technology r on reddit
OpenAI acknowledges ‘wiki incident’ and need for more transparency around unintended AI behavior

Chronicles