Experts say terrorist groups are using generative AI tools to evade the hashing algorithms used by tech companies to automatically remove extremist content
Experts are finding thousands of examples of AI-created content every week that could allow terrorist groups and other violent extremists to bypass automated detection systems.
WiredDavid Gilbert
Context & Ripple Effects
This report identifies a content-moderation weakness created by synthetic material: automated removal systems can be less effective when known extremist content is repeatedly altered with generative tools.
Tech companies that use hash-based removal face more AI-generated variants that may evade their existing automated filters, creating a larger detection and review burden.
Extremist groups gain a way to generate or alter material specifically to avoid known content signatures.
Second-order effects
Platforms will need to complement hash matching with detection methods that assess newly generated or modified material rather than relying only on known-file matches.
The finding increases pressure on AI developers and safety teams to treat misuse safeguards as part of a broader abuse-prevention stack, not solely a content-policy issue.
Third-order effects
If synthetic variation routinely defeats signature-based moderation, safety systems may shift toward more adaptive, behavior- and context-based enforcement—an approach that can be more complex to operate and audit.
The subsequent move from AI-enabled extremist-content evasion to reported chatbot-assisted operational use suggests the relevant governance challenge is the capability’s reuse across different harm domains, not one removal mechanism alone.
The trend:Generative AI is lowering the cost of adversarial adaptation, forcing safety systems built around known harmful artifacts to contend with rapidly produced variants.
Terrorists & extremists are already experimenting with #GenAI tools: • ISIS has an AI tech guide • Neo-Nazis share racist AI promts • Far-right groups have a “guide to memetic warfare” Now @techvsterrorism is working with @Microsoft to combat it... https://www.wired.com/...
Terrorist organizations, neo-Nazis and far-right groups are already experimenting with generative AI tools to help them boost recruitment and avoid censorship on mainstream platforms https://www.wired.com/...
⚡️Announcement | We are embarking on a new pilot project with @Microsoft to develop an AI-powered tool for detecting terrorist or violent extremist content online. The project aims to understand the potential of #AI tech in service of online safety. https://techagainstterrorism.o…
Almost exactly a year to the day since Meta's chief AI officer told us this wouldn't happen. “Extremist groups have begun to experiment with artificial intelligence, and in particular generative AI, in order to create a flood of new propaganda” https://www.wired.com/...
I would add that it's not *just* violent extremists, but any actors seeking disruption. This will take scams into another dimension. #GenerativeAI https://www.wired.com/...
The use of digital platforms to spread violent extremist content is an urgent issue with real-world consequences. By combining Tech Against Terrorism's capabilities with AI, we hope to help create a safer world both online and off. https://techagainstterrorism.org/ ...
💻 Early Terrorist Adoption of Generative #AI We have archived over 5,000 pieces of AI-generated content. Whilst this represents a fraction of terrorist & violent extremist of material, it does point to the challenge faced by content moderators. Read👇 https://techagainstterrorism.…
This is an important story as it demonstrates how violent extremists are using AI to manipulate and control narratives. In this case, researches found 5k instances weekly of pro-Hamas content. https://www.wired.com/...