Experts say terrorist groups are using generative AI to bypass the hashing databases used by tech companies to automatically remove violent extremist content
WiredDavid Gilbert
Context & Ripple Effects
The report identifies a weakness in a core automated moderation mechanism: when generative AI can alter extremist material enough to evade known hashes, matching against existing databases no longer reliably catches every reposted variant.
Platforms and hash-database operators face a more immediate detection gap: AI-modified versions of known violent extremist material may not match records that would otherwise trigger automatic removal.
Trust-and-safety teams must put more weight on review and detection methods that can assess altered material, rather than treating hash matches as a sufficient control.
Second-order effects
Content-moderation vendors and platforms are pushed to compete on systems that recognize meaning and harmful context across variants, not solely exact or near-exact file matches.
The reported evasion route strengthens the case for AI providers and platforms to coordinate safeguards, since capabilities available for ordinary content creation can also lower the effort required to repackage prohibited material.
Third-order effects
If generative tools consistently undermine signature-based enforcement, moderation is likely to shift from primarily database-led takedowns toward layered, adaptive detection and human oversight.
The pattern adds pressure for dual-use AI governance that addresses downstream misuse across model access, generation tools, and distribution platforms rather than assigning responsibility to one layer alone.
The trend: Generative AI is turning static, rules-based safety systems into an ongoing adaptation contest between misuse and detection.
This really shows the REAL issues with GenAI right now. It renders traditional schemes to fight hate content (like using hashes) totally ineffective. “Here's How Violent Extremists Are Exploiting Generative AI Tools” https://www.wired.com/...
@GaryMarcus More than 2/3 of those living in a democracy will vote in the next 12 months and our social platforms are not ready for the impact generative AI is going to have on election integrity ... we need strategies to deploys “PETs” now
@GaryMarcus The intellectual fallacy of the techno “optimists” is they only talk about the extreme case of machines exterminating humanity as a whole. It's a classic propaganda ploy But that's not the current risk The current risk is some humans using AI to harm or kill other hum…
@GaryMarcus @AnnaLeptikon Nobody is claiming that AI won't be used to create content. They're claiming that it's not effective. This is not an area on which you can perform science, because the observations consist of the decisions and actions of people. Good luck with the contro…
Almost exactly a year to the day since Meta's chief AI officer told us this wouldn't happen. “Extremist groups have begun to experiment with artificial intelligence, and in particular generative AI, in order to create a flood of new propaganda” https://www.wired.com/...
I would add that it's not *just* violent extremists, but any actors seeking disruption. This will take scams into another dimension. #GenerativeAI https://www.wired.com/...
@GaryMarcus It's worth noting the article was largely about small or unrealized threats and the extensive mitigations underway. e/acc fails to see the threat, doomers the checks and balances.
@pretendsmarts What has begun now is not misinfo but the use of AI precisely to increase volume and reduce cost (as I have said here literally dozens of times)
The ugly side of #AI is the scary part that many don't want to acknowledge. Like anything else—for every good, helpful, miraculous use there's someone brilliant looking at ways to turn it against us. #ArtificialIntelligence #technology #technews #tech https://www.wired.com/...
@GaryMarcus “have begun” ? I mean this was going on for quite some time, now it's just getting easier and easier, so the volume and quality of propaganda is rising, but it's not a completely new phenomena.
This is an important story as it demonstrates how violent extremists are using AI to manipulate and control narratives. In this case, researches found 5k instances weekly of pro-Hamas content. https://www.wired.com/...
Terrorists & extremists are already experimenting with #GenAI tools: • ISIS has an AI tech guide • Neo-Nazis share racist AI promts • Far-right groups have a “guide to memetic warfare” Now @techvsterrorism is working with @Microsoft to combat it... https://www.wired.com/...
Terrorist organizations, neo-Nazis and far-right groups are already experimenting with generative AI tools to help them boost recruitment and avoid censorship on mainstream platforms https://www.wired.com/...
⚡️Announcement | We are embarking on a new pilot project with @Microsoft to develop an AI-powered tool for detecting terrorist or violent extremist content online. The project aims to understand the potential of #AI tech in service of online safety. https://techagainstterrorism.o…
💻 Early Terrorist Adoption of Generative #AI We have archived over 5,000 pieces of AI-generated content. Whilst this represents a fraction of terrorist & violent extremist of material, it does point to the challenge faced by content moderators. Read👇 https://techagainstterrorism.…