Apple made an unforced error by trying to tackle CSAM and child safety issues without soliciting expert advice while adhering to its annual iOS release schedule
Hello friends, and welcome back to Week in Review. — Last week, we dove into the truly bizarre machinations of the NFT market.
TechCrunchLucas Matney
Context & Ripple Effects
When Apple announced on-device CSAM scanning in August 2021, the criticism was immediate and structural: [[a:969445|Stratechery argued scanning should have run on iCloud servers, where users have a lower expectation of privacy]], and WhatsApp publicly called the approach a setback for user privacy, warning it would be more fraught outside the US. The company was caught between two constituencies it had built its brand on serving.
The TechCrunch verdict frames the episode as an unforced error: Apple tried to solve a hard child-safety problem inside the Apple Park vacuum, on its own annual iOS release schedule, without soliciting outside expert advice. It then delayed the rollout days before launch, and two years later scrapped the on-device system entirely as key team members left — while [[a:843783|the Heat Initiative began a $2M ad campaign pressing Apple to do more against CSAM in iCloud]].
First-order effects
Apple pulled the child safety features from its iOS release cycle and committed to collecting stakeholder feedback, ceding the initiative on a policy it had already publicly announced.
Privacy advocates and rivals like WhatsApp gained a concrete talking point: the company marketing privacy-first devices had put a scanning mechanism on them.
Second-order effects
Child advocacy groups shifted from lobbying for new scanning to attacking Apple for scrapping it, as the Heat Initiative's ad campaign shows — Apple now faces pressure from both sides simultaneously.
Competing platforms with server-side scanning, like WhatsApp's parent, could position their own architectures as the privacy-preserving middle ground Apple failed to occupy.
Third-order effects
The episode suggests platform trust features cannot be shipped on a product-release cadence; safety-vs-privacy design now demands outside expert consultation before announcement, or the announcement itself becomes the liability.
If the pattern holds, Apple's child-safety posture stays reactive — squeezed between advocates demanding detection and privacy constituencies forbidding on-device inspection — leaving the CSAM question unresolved on the platform that markets trust hardest.
The trend: Platform trust decisions are moving from internal product timelines to contested public policy processes, where the announcement itself — not just the feature — is the risk.
BREAKING: Apple pauses its plan to do client-side scanning for CSAM. https://www.cnbc.com/... This is a direct response to the outcry from users and civil society. We're not done, but this is a reminder that collective action moves the needle.
I agree with this. @Apple must go further than merely “listening”—it must drop entirely its plans to put a backdoor into systems that provide vital protections to the public. Don't make us fight you for basic privacy rights. https://www.eff.org/...
“It's a major embarrassment for Apple... it's a sign that their internal teams weren't adequately prepared and lacked the ideological diversity to gauge the scope of the issue that they were tackling.” @lucasmtny. https://techcrunch.com/...
I had people messaging me asking if Apple was going to flag photos of their own children as CSAM, due to conflation of technologies. For a firm hailed for marketing, this was a colossal public fuck up that hurt everybody including CSAM victims. Seriously you inexcusable morons.
Immediately after this revelation about heuristically detecting child abuse, they say a machine will scan your pictures for child abuse and report you to the police. Are these technologies connected? No need to be specific here! Also, “children” will be ratted on to their parents…
Apple won't get away with just tweaking it's CSAM plans. Just like WhatsApp with Facebook data sharing and Google with FLoC, only a total backtrack will fix this. https://www.eff.org/...
The most important, and first example, is Apple claiming they have developed machine-learning to detect, without prior knowledge or human input, “sensitive content” in the context of child abuse. There is a machine deciding if a picture is sensitive. What does “sensitive” mean? h…
Here is the link to the original as it appeared on August 5th, 2021. A Thursday. This was intended to be an applauded topic of policy discussion, and not a closeted Friday night news dump disclosure. Apple wanted you to know their action. Now, the content. https://web.archive.org…
Apple's CSAM “solution” was a disjointed executive-driven fantasy project rolled out incompetently to what they thought would be career-benefiting accolades that their own people didn't have basic answers to how or why, or plans for making this work subject to client-side attack.…
(Intermission as I attend to some cooking: Apple should fire everyone involved at this point. This is far enough to be customer-endangering drivel. But we've got so much longer to go in this nightmare of English text.)
I kinda understand why/how Apple got themselves into their current terrible mess over on-phone CSAM scanning, but I wish they'd stop digging and instead openly admit that it's a political quid pro quo to get the SLAs to stop targeting them with vicious spyware like NSO's Pegasus.
Apple's “Child Safety” announcement may be one of the worst public communications failures in recent memory. It is vague where it should be specific, technical where it should be human, and ultimately either an artifact of rank incompetence or a failed gambit. Let's analyze it.
Let's be frank: Apple delaying CSAM is rather useless. No amount of delays or “improvements” will remove the stain the announcement itself & following controversy have created. The damage is already done.
Forget the tech, we can't have an honest discussion about it if we don't acknowledge the realpolitik behind public policy demands for on-device scanning for illicit material, of whatever kind. (CSAM is merely the stuff the Five Eyes think they can rally public support against.)
“Child” has multiple definitions. Biologically, it's before adolescence. Legally and in context of child sexual abuse, it's someone below age of 18. That's the context we're operating under. This literally says Apple will detect teenagers in sexual situations and alert someone. h…
Great move by Apple to reconsider iCloud photo snooping. Scanning content for legality belongs in a publishing context - where one is sharing it with the public - not on a customer's device, and not when storing one's personal effects. https://twitter.com/...
A month ago an Apple exec sent out an internal email talking about “the screeching voices of the minority”, referring to internet people highlighting problems with their child sexual abuse material scanning system. Ivory towers are harsh mistresses. https://twitter.com/...
Just in from Apple on the CSAM tools. The company is taking “additional time over the coming months to collect input and make improvements before releasing these critically important child safety features.” https://twitter.com/...
I've seen two people post PhotoDNA implementations in the last two weeks. One is on the front page of HN. These are algorithms that were secret and under NDA for years before Apple's announcement. (And Apple doesn't even use PhotoDNA.)
To be clear, I think concerns that Apple would take lessons from CSAM detection and then build additional detection apparatuses that are far more controversial (i.e. detect people who write certain things in the Notes app) are misplaced.