A hacktivist claims DALL-E-like Craiyon, formerly DALL-E mini, generated portraits of brown-skinned women in saris almost every time he ran a blank request
The images represent a glitch in the system that even its creator can't explain. — Like most people who are extremely online … Tweets: @mtrc , @nilchristopher , @sub8u , @flitteronfraud , and @anupkaphle Tweets: @mtrc : Earlier in the week, @NilChristopher emailed me with a fascinating mystery: DALL-E Mini repeatedly produces images of women in saris when prompted with a blank input. He did some digging to find out why, but the mystery persists. A really nice piece: https://restofworld.org/... Nilesh Christopher / @nilchristopher : Why is viral AI image generator DALL.E mini obsessed with brown women in saris? I set out to solve this mystery for @restofworld which even its creator couldn't explain. Read to find out. https://restofworld.org/... @sub8u : Can't make this up. Our future is trying to explain why an AI system came up with something really specific when it wasn't expected to. And pointing fingers at training data! Via @NilChristopher “DALL·E mini has a mysterious obsession with women in saris” https://restofworld.org/... https://twitter.com/... Emily Flitter / @flitteronfraud : The whimsical image generator everyone is suddenly obsessed with has a “glitch,” as this story calls it, that even its creator can't explain. Great illustration by @restofworld of how opaque AI can be. & companies use it for big customer decisions https://restofworld.org/... Anup Kaphle / @anupkaphle : Sari not sorry — DALL·E mini, maybe https://restofworld.org/...
Context & Ripple Effects
Craiyon began life as DALL-E mini, a free, openly accessible reimplementation of OpenAI's DALL-E hosted via Hugging Face, and its viral run made it one of the most-used image generators on the internet. Unlike OpenAI itself, which has been applying a content policy to screen out sensitive or biased images since DALL-E 2 opened up, Craiyon ships with no such filter.
That difference is exactly what this story exposes: Rest of World's reporting, sparked by Nilesh Christopher and traced through tweets from Emily Flitter and Anup Kaphle among others, documents a reproducible glitch — blank prompts yielding portraits of brown-skinned women in saris — that even Craiyon's creator cannot explain.
First-order effects
- Every user who runs an empty prompt on Craiyon gets the same stereotyped output, turning a private training artifact into a publicly reproducible demonstration of bias in a tool with no content policy to mask it.
- Craiyon's creator is now publicly accountable for a failure mode he cannot diagnose, with no documented training-data provenance to fall back on.
Second-order effects
- The episode sharpens the contrast with OpenAI's filtered approach: free, unfiltered generators face pressure either to adopt screening like DALL-E 2's or to publish dataset documentation explaining where outputs like these come from.
- It feeds the broader scrutiny of training corpora — the same dynamic that later took LAION-5B offline after researchers found abuse material inside it — pushing dataset composition from an academic footnote to a reputational liability for model builders.
Third-order effects
- If open models keep surfacing their training data's skews while closed models hide them behind filters, bias auditing shifts from a lab exercise to a public, adversarial process — with the least-resourced model operators holding the most visible failures.
- Sustained episodes like this push dataset provenance and consent toward becoming baseline expectations for anyone shipping a generative model, the same accountability wave that hit Stability AI when artists discovered their work was trained on without notice.
The trend: As image generation goes mass-market, the split between filtered closed models and unfiltered open ones is turning latent training-data bias into a publicly reproducible artifact that creators must explain or defend.