LAION, a research org whose dataset was used to train Stable Diffusion and other models, releases a new dataset it claims has been “thoroughly cleaned” of CSAM
LAION, the German research org that created the data used to train Stable Diffusion, among other generative AI models …
TechCrunchKyle Wiggers
Context & Ripple Effects
LAION’s openly available image corpus became important infrastructure for generative-image research, including use by Stable Diffusion and other models. That role made the dataset’s provenance consequential beyond the organization itself.
The release follows research reporting that LAION-5B contained CSAM and the corpus was later unavailable for download after those findings. LAION’s new release is therefore an attempt to restore a usable corpus after the earlier reported CSAM findings in LAION-5B and the withdrawal of the prior dataset from download.
First-order effects
LAION can offer researchers and model developers a replacement dataset that it says has been cleaned of CSAM, while taking on the burden of substantiating that assurance.
Teams that previously relied on LAION-derived data, including users of Stable Diffusion’s training ecosystem, gain a potential alternative to the unavailable LAION-5B corpus.
Second-order effects
Developers adopting the new corpus will need to assess LAION’s cleaning process rather than treat an open dataset as automatically safe; dataset screening becomes a procurement and governance question.
Other public dataset publishers face stronger pressure to document harmful-content detection and removal, particularly after the earlier Stanford-reported examples exposed risks in large web-scale collections.
Third-order effects
If reusable training corpora increasingly require documented filtering and review, the value of AI data infrastructure will shift from raw scale toward governance, traceability, and confidence in distribution.
The episode may narrow the gap between open research datasets and institutionally managed data pipelines, though the effectiveness of that shift depends on whether cleaning claims can be independently validated.
The trend: Generative-AI training data is moving from minimally curated web-scale collection toward governed corpora whose safety controls are part of the product.
LAION has released a revised version of the LAION-5B dataset to address CSAM concerns we previously highlighted. Here are my impressions. 1/15 https://laion.ai/...
The NCII part is key IMO, because a *lot* of imagery that shows up in random crawls is non-consensual, of dubious provenance or at the very least copyrighted. And also just private imagery, and risky given identity and age ambiguity. 6/15
These are obviously all significant improvements; credit to LAION and the other involved child safety orgs for all their work. It's not quite what I would call a gold standard, but it definitely sets a much better example. 4/15
They also removed all content above a conservative “unsafe” probability; one dataset removing >0.95, which we found covered almost all our matches. The other dataset is far more conservative, removing the majority of NSFW samples. 3/15
They removed content using the instances we found as well as an apparent expanded MD5 hash set, chaffing the removals by removing a large amount of other content, including using keywords that could relate to CSAM and non-CSAM that could contain sensitive info about kids. 2/15
LAION-5B is important reference research dataset for reproducible language-vision foundation models studies. We release Re-LAION-5B as a transparent safety iteration on LAION-5B which fixes issues and allows broad research community to continue using open datasets as reference🧵
Re-LAION-5B fixes the issues reported by Stanford Internet Observatory (SIO) in December 2023 for original LAION-5B. In cooperation with IWF, C3P and David Thiel (SIO), all 1008 links to suspected CSAM in the report are removed from LAION-5B metadata, using safe hash lists.
For the Re-LAION release we remove a much larger pool of ca. 65 million of various neutral links, in addition to 2236 suspected links. This way, none of the suspected links can be identified by a diff between Re-LAION and any older versions of LAION-5B.