How AI Dungeon, a text adventure game that uses the GPT-3 text generator, can create sex scenes involving children, and why attempts to stop it aren't working
Tom Simonite / Wired : Tweets: @hkanji , @yudhanjaya , @marklittlenews , @wired , @sesmith , and @etzioni Tweets: Hussein Kanji / @hkanji : Users complained the AI would bring up sexual themes unbidden, for example when they attempted to travel by mounting a dragon and their adventure took an unforeseen turn. https://www.wired.com/... Yudhanjaya Wijeratne / @yudhanjaya : ‘Then the algorithm started to generate disturbing stories, including sex scenes involving children.’ Aaaargh! This is why you always, ALWAYS examine the training data and curate it tightly. Any competent data scientist could have warned you of this. https://www.wired.com/... Mark Little / @marklittlenews : Salutary tale of how an “attempt at combining people and algorithms to police content produced by people and algorithms turned into a mess” https://www.wired.com/... via @wired @wired : “This is not the future for AI that any of us want.” https://www.wired.com/... S. E. Smith / @sesmith : Well, this escalated quickly and perhaps predictably. https://www.wired.com/... Oren Etzioni / @etzioni : Fascinating though not surprising GPT-3 story by @tsimonite https://www.wired.com/...
Context & Ripple Effects
This 2021 Wired investigation was the first big consumer-facing failure story for GPT-3: players of AI Dungeon reported the model raising sexual themes unbidden — even during innocuous actions like mounting a dragon — and researcher @yudhanjaya documented it generating child sex scenes, with the developer's filters proving ineffective. The episode prefigured the entire trust-and-safety buildout that followed.
The arc since then runs through OpenAI's own operations: the company later contracted Sama workers at $2/hour to label exactly this kind of toxic content for ChatGPT, and by 2025 Sam Altman was framing adult-content decisions as beyond OpenAI's mandate entirely.
First-order effects
- Players of AI Dungeon were directly exposed to non-consensual sexual content, including child sexual material, injected into their own adventures — the product's core loop became its liability.
Second-order effects
- OpenAI absorbed the lesson operationally: it hired Kenyan labelers via Sama to classify violent and sexual text before shipping ChatGPT, turning content moderation into a supply chain rather than a filter setting.
Third-order effects
- The industry is settling into a permanent posture where model providers disclaim moral arbitration — Altman's 'not the elected moral police' line — while downstream app makers like AI Dungeon carry the reputational risk of what base models will say.
The trend: Consumer-facing generative text keeps outrunning the moderation layer built around it, pushing safety work down from labs to application developers and outsourced labeling pipelines.