Tech companies have been failing to deal with child abuse images for years, and they still don't have a common standard for identifying illegal video content

How PhotoDNA Works — The uploaded image — in this instance a photograph of Dr. Farid — is turned into a square and colors are removed …

New York Times 2019-11-10

Context & Ripple Effects

This piece is the technical companion to the Times' broader finding that tech firms and government have failed for over a decade to curb child sexual abuse imagery, despite the PROTECT our Children Act mandate from 2008. Its core point: PhotoDNA-style perceptual hashing works for still images, but no equivalent common standard exists for video, so every platform improvises its own detection.

The gap this article documents has only widened since. Apple's later exploration of on-device hashing shows how contested even image matching became once it touched user privacy, and researchers found known abuse images still uploading to Twitter as recently as 2023 — evidence that the identification problem persists even where standards do exist.

First-order effects

Platforms handling user video must each build and maintain proprietary detection pipelines rather than adopt a shared hash database, leaving coverage uneven across services and letting identical content slip through on one platform after being caught on another.

Second-order effects

The enforcement vacuum invites escalating outside pressure: advocacy groups like the Heat Initiative moved from reports to a $2M ad campaign specifically targeting Apple's iCloud handling of CSAM, forcing individual companies to defend their choices publicly.

Third-order effects

Generative AI compounds the structural problem: experts warn that synthetic CSAM floods dark web forums with images that have no prior hash to match against, meaning the hash-detection paradigm itself erodes just as video standardization remains unsolved.

The trend: Content moderation is drifting from shared hash-matching of known material toward provenance and creation-time controls, because generative tools produce abuse content that has never been seen before.

Discussion

@whywelook Marvin Heiferman on x
WHY WE LOOK? Because Images of Child Abuse Are Rampant as Tech Companies Look the Other Way. “Any system that allows you to share photos and videos is absolutely infested with child sexual abuse,” said a former security chief at Facebook. https://www.nytimes.com/...
@mheadd Mark Headd on x
Employees at tech companies taking ethical stands on the kind of things they don't want their work to support. Here is an issue that they could make their voices heard on that would help thousands & thousands of exploited children. Let's hear from them. https://www.nytimes.com/..…
@mhkeller Michael Keller on x
As one woman said, now 21 and dealing with lifelong effects of the abuse she and her sister endured: “It's more than just images. When I'm in public with my little sister, and I see some man looking at her, that is one of the first things I think about. You're always worried.” 3/
@mhkeller Michael Keller on x
We found other gaps, too. Many cloud storage companies, including Dropbox, Google Drive and Microsoft OneDrive, don't scan files when they're uploaded, only when shared. We found court cases in which offenders exchanged logins, getting around prevention measures. 12/
@mhkeller Michael Keller on x
Enterprise cloud storage scans even less. Amazon doesn't scan anything at all, and neither does Microsoft Azure. Both cited customer privacy reasons and said their terms of service prohibited illegal imagery. 13/
@mhkeller Michael Keller on x
Tests of Google didn't find similar imagery, but @CdnChildProtect shared documentation with us showing how Google repeatedly refused to remove images the Canadian Center's analysts discovered, including explicit photos of children younger than 12. 10/
@mhkeller Michael Keller on x
We found 75 images that matched on Bing, DuckDuckGo and Yahoo before stopping the program. After our tests, Microsoft said it found a flaw in its scanning practices and was re-examining search results. But subsequent tests found even more sexual abuse imagery. 8/
@mhkeller Michael Keller on x
Here's the full story, with our complete findings and moving portraits by @kholoodeid of abuse survivors and their parents. She has traveled across N. America taking photos for this series, documenting the people this issue affects. Thanks for reading. https://nyti.ms/36PVNXt 15/
@deborahblum Deborah Blum on x
“Amazon, whose cloud storage services handle millions of uploads and downloads every second, does not even look for the imagery.” Child Abusers Run Rampant as Tech Companies Look the Other Way https://nyti.ms/2Q5mLUT
@blmohr @blmohr on x
“Approaches by tech companies are inconsistent, largely unilateral and pursued in secret, often leaving pedophiles and other criminals who traffic in the material with the upper hand.” https://www.nytimes.com/...
@danielle_ivory Danielle Ivory on x
A Canadian child protection center asked Google to take down child sexual abuse imagery in August last year, but Google said it did not meet its threshold for removal, according to documents. https://nyti.ms/... ⁦@mhkeller⁩ ⁦@gabrieldance⁩
@cdnchildprotect @cdnchildprotect on x
Through our work with Project Arachnid we have firsthand experience with the pushback from industry in the removal of child sexual abuse material. Today's @nytimes exposes how victims are harmed when industry doesn't do enough to protect children. https://www.nytimes.com/...
@chiefcmagnus Chris Magnus on x
“The industry's response to video content has been wanting...There's no common standard for identifying illegal video content & many major platforms including AOL, Snapchat & Yahoo don't even scan for it.” https://www.nytimes.com/...
@mohammadfarooq_ Mohammad Farooq on x
Though platforms bar child sexual abuse imagery on the web, criminals are exploiting gaps. Victims are caught in a living nightmare, confronting images again and again. https://www.nytimes.com/...
@antoniogm Antonio García Martínez on x
This story gives due credit to Facebook, which has had a years-long effort to crack down on child abuse on the platform (though where that all stands with usage shifting to encrypted messages remains to be seen). https://www.nytimes.com/...
@claireshrugged Sister Outrider on x
For years tech companies have been fully aware that videos of CSA are circulated via their platforms. And the technology to detect and remove images of CSA exists, but is largely neglected - with devastating, life-long consequences for victims. https://www.nytimes.com/...
@prostasiainc @prostasiainc on x
“It has been 10 years since PhotoDNA was developed at Microsoft, yet the industry's efforts to detect and remove known illegal photos remains uneven and cloaked in secrecy.” https://buff.ly/32wUPvV
@cagoldberglaw Carrie A. Goldberg on x
Any law students looking for a note to write? Read: 1) this article and it's predecessor 2) Paroline v US 3) big tech immunity from liability under cda230 4) tort law theory and history https://www.nytimes.com/... via ⁦@nytimes⁩
@cagoldberglaw Carrie A. Goldberg on x
How we end the epidemic of child sexual abuse imagery online: class actions against big tech using civil prong of 2252(a). NO CDA 230 immunity b/c 2252a is a federal criminal law. https://nyti.ms/2Q5mLUT
@carnage4life Dare Obasanjo on x
Policing child abuse is where big tech incentivized by media to do nothing. On one hand you police it & have stories written about how miserable content moderators jobs are & X million pieces of content found on your service versus you care about privacy https://www.nytimes.com/.…
@joshconstine Josh Constine on x
Read this exposè by @gabrieldance & @mhkeller into how behind tech is on fighting abuse via video, and demand these companies put child safety ahead of business. 5/5 https://www.nytimes.com/...
@jgeltzer Joshua A. Geltzer on x
This 👇 is disturbing on its own. Then add context: child exploitation imagery is a problem tech companies describe as simpler than terrorist content or disinformation & mostly solved. If it's still so bad, think of how much worse the other problems are. https://www.nytimes.com/..…
@profcarroll David Carroll on x
I consider myself a privacy advocate and I have absolutely no problem with platforms scanning for illegal child sexual abuse images. Don't use me as an excuse to look the other way, Big Tech. https://www.nytimes.com/... https://twitter.com/...
@sophiacannon Sophia on x
The two sisters are among the first generation of child sexual abuse victims whose anguish has been preserved on the internet, seemingly forever https://www.nytimes.com/...
@johnc1912 John Carr on x
Another excellent, major story from The New York Times. It details the failings of high tech businesses to address the scourge of child sex abuse still images and videos on the internet. The failures of voluntarism are again manifest. https://www.nytimes.com/...
@ceciliakang Cecilia Kang on x
Child victims of the worst crimes possible continue to be violated over and over as imagines of the crimes circulate online. The Tech Co. in many instances have turned a blind eye. Others claim success with faulty systems. https://www.nytimes.com/... via @mhkeller @gabrieldance
@alexstamos Alex Stamos on x
@benthompson We need some kind of framework that makes it rational for companies to be proactive in looking for harm and then disclosing both what they found and their shortcomings. All the incentives run the other way right now.
@pinboard @pinboard on x
The demand (from the second article) that services like Dropbox auto-scan all images on upload is not consistent with the call to an end to third-party monitoring of private content in the first piece. I don't think the right answer here is obvious, but you can't play both sides
@rasmus_kleis Rasmus Kleis Nielsen on x
Child Abusers Run Rampant as Tech Companies Look the Other Way. “The digital trail of abuse — often stored on Google Drive, Dropbox and Microsoft OneDrive — haunts the sisters relentlessly, they say, as does the fear of a predator recognizing them”. https://www.nytimes.com/...
@rich_harris Rich Harris on x
brilliant, awful story by @mhkeller and @gabrieldance “The companies have the technical tools to stop the recirculation of abuse imagery ... Yet the industry does not take full advantage of the tools.” https://www.nytimes.com/...
@joshconstine Josh Constine on x
Scanning for child abuse imagery isn't an invasion of privacy as tech giants claim. It should be mandatory. I hope lawmakers like @MarkWarner & @HawleyMO make it the law. 4/ https://twitter.com/...
@kevinroose Kevin Roose on x
This story is so powerful because of the subject matter, but also because of the deep research + tech it took to pull off. Our team *built a browser* to look for child sexual abuse content on search engines without displaying it. https://www.nytimes.com/...

Chronicles