OpenAI disrupted five covert influence operations in the past three months that used its tools to try to manipulate public opinion or shape political outcomes
firm criticized the UN as well Meta went out of its way to tell the WSJ it didnt connect the firm to the Israeli government though I am not sure how the company tried (or didnt try) to make that connection https://dfrlab.org/... [image] Miles Brundage / @miles_brundage : Very excited about this - by far the most detailed such empirical analysis that I am aware of. Rat King / @mikeisaac : interesting report on how OAI detected influence campaigns using ChatGPT to spread messages across Instagram, Facebook, Twitter, etc one of them was an Israeli political campaign firm spreading islamophobic posts (i wonder if the AI prompt was “try to speak like a zoomer”? ??) [image] LinkedIn: Jennie Baird : It's this throwaway line in this story that most concerns me: “The campaigns often used OpenAI's technology to post political content, Mr. Nimmo said … Katrina Mulligan : Today we announced the disruption of five groups that sought to use our models for covert influence operations seeking to deceptively manipulate public opinion … Forums: r/technology : OpenAI Says Russia and China Used Its A.I. in Covert Campaigns r/anime_titties : OpenAI finds Russian and Chinese groups used its tech for propaganda campaigns r/ActiveMeasures : OpenAI finds Russian and Chinese groups used its tech for propaganda campaigns
Context & Ripple Effects
This disclosure turns a general warning that language models could support disinformation into an operational enforcement case: OpenAI says it identified and interrupted campaigns tied to Russian and Chinese groups, as well as an Israeli political campaign firm. The reported use of social platforms makes model misuse a cross-platform distribution problem, not only a model-provider problem.
Later coverage describes OpenAI’s disruption activity as continuing, including the removal of an Iranian account cluster targeting the US election and a broader tally of more than 20 disrupted deceptive operations in 2024. That arc makes this early set of five cases a marker of sustained threat investigation rather than a one-off moderation event.
First-order effects
- OpenAI cuts off access for five identified operations, disrupting their ability to use its tools to generate or adapt influence content.
- Facebook, Instagram and Twitter are implicated as downstream channels for content associated with the operations, while Meta publicly rejects a claimed connection between the Israeli firm and the Israeli government.
Second-order effects
- The cases put pressure on model providers and social platforms to share indicators and coordinate enforcement, since blocking generation alone does not remove content already distributed elsewhere.
- OpenAI’s public reporting raises the value of dedicated threat-investigation capacity, later exemplified by reporting on its principal threat investigator’s work tracing Russian and Chinese activity.
Third-order effects
- If disclosures and takedowns become routine, access controls, monitoring and incident reporting may become core competitive and governance requirements for general-purpose AI services.
- The pattern points to influence operations treating AI models as interchangeable production tools; durable mitigation will depend on coordination across model access and social distribution, not a single platform’s policies.
The trend: Generative-AI providers are becoming active participants in countering state-linked and political influence operations that span model services and social platforms.