Apple delays the rollout of recently announced child safety features, says it will take more time to collect feedback from stakeholders and make improvements

Last month, Apple announced a handful of new child safety features that proved to be controversial, including CSAM detection features for iCloud Photos.

9to5Mac 2021-09-04 Chance Miller

Context & Ripple Effects

Apple’s delay follows its acknowledgment of concerns about the photo-scanning proposal in an internal memo defending the child-protection rationale. Related coverage also characterized the rollout as having been developed without enough outside expert input and under an annual software-release timetable.

The pause matters because iCloud Photos sits at the intersection of child-safety enforcement and user privacy: Apple is moving stakeholder feedback and product changes ahead of deployment.

First-order effects

Apple will not roll out its announced child-safety features, including CSAM detection for iCloud Photos, on the original timetable while it gathers feedback and makes improvements.
Stakeholders who objected to the proposal gain a formal window to press Apple on the design of the delayed features.

Second-order effects

Apple’s product and release teams must separate the child-safety initiative from a schedule-driven launch process that related coverage identified as part of the rollout problem.
The delay raises the bar for Apple to show that any revised iCloud Photos approach addresses the concerns it had previously described as misunderstandings.

Third-order effects

If major platforms treat safety scanning as a stakeholder-governance issue before deployment, public-safety features will face more scrutiny over how enforcement is embedded in consumer services.
The episode points toward more contestable gatekeeping, in which platform decisions affecting private user content are shaped by external review rather than product-release cadence alone.

The trend: Platform safety enforcement is becoming a public-safety governance challenge, with privacy-sensitive product design subject to greater stakeholder scrutiny before launch.

Discussion

@joannastern Joanna Stern on x
Just in from Apple on the CSAM tools. The company is taking “additional time over the coming months to collect input and make improvements before releasing these critically important child safety features.” https://twitter.com/...
@matthew_d_green Matthew Green on x
My suggestions to Apple: (1) talk to the technical and policy communities before you do whatever you're going to do. Talk to the general public as well. This isn't a fancy new Touchbar: it's a privacy compromise that affects 1bn users. 1/
@rickwebb Rick Webb on x
This is good news but if ATT is any guide, the time will not change things much. https://twitter.com/...
@zackwhittaker Zack Whittaker on x
Breaking: Apple has delayed rolling out its CSAM detection technology in iOS 15, citing feedback from customers, advocacy groups, researchers and others — which has been largely negative since the technology was announced in August. https://techcrunch.com/...
@iansherr Ian Sherr on x
Apple's delaying the rollout of CSAM detection after pushback from privacy and security groups https://twitter.com/...
@chancehmiller Chance Miller on x
Almost exactly one month after the initial announcement... https://twitter.com/...
@jonfortt Jon Fortt on x
Apple hits pause on move to scan user devices for images of child sexual abuse. From the company: ⁦@CNBCTechCheck⁩ $AAPL https://twitter.com/...
@tim Tim Bradshaw on x
Just in: Apple delaying CSAM system: “Based on feedback from customers, advocacy groups, researchers & others, we have decided to take additional time over the coming months to collect input and make improvements before releasing these critically important child safety features.”
@reneritchie Rene Ritchie on x
Apple is taking “additional time over the coming months to collect input and make improvements” before releasing their previously announced Child Safety Features. Full statement, full text in ALT: https://twitter.com/...
@viacristiano Cristiano Lima on x
🔥🔥🔥 [Apple spokesman Fred Sainz said he would not provide a statement... ]
@evacide Eva on x
BREAKING: Apple pauses its plan to do client-side scanning for CSAM. https://www.cnbc.com/... This is a direct response to the outcry from users and civil society. We're not done, but this is a reminder that collective action moves the needle.
@_andyburrows Andy Burrows on x
This is incredibly disappointing. @Apple had adopted a proportionate approach that sought to balance user safety and privacy, and should have stood their ground. https://twitter.com/...
@snowden Edward Snowden on x
they really underestimated how hard we can screech
@matthew_d_green Matthew Green on x
(2) Be clear about why you're scanning and what you're scanning. Going from scanning nothing (but email attachments) to scanning everyone's private photo library was an enormous delta. You need to justify escalations like this.
@zamosta Aaron Zamost on x
Sorry but naming the spokesperson here is bullshit. Fine, don't accept a statement and cite the dumb reason why. But don't be a dick, come on. https://twitter.com/...
@granick @granick on x
A success for civil liberties groups' advocacy. It's great that Apple plans to engage with independent privacy and security experts before announcing their genius plans. They should start with end to end encryption for iCloud backups. https://t.co/wMDkphPyEM
@tim Tim Bradshaw on x
UK childrens charity NSPCC says Apple's delay is “incredibly disappointing”. So Apple has now managed to anger both sides of the privacy/protection debate. https://twitter.com/...
@josephfcox Joseph Cox on x
Here is Apple's full statement on its decision to delay the rollout of these child safety features. These include comparing images on device to a list of known child abuse images, and scanning children iMessage attachments for unknown sexual content https://www.vice.com/... https…
@neilcybart Neil Cybart on x
This paragraph just doesn't read as Apple knowing what it will change to calm fears / pushback. Taking the “coming months” to collect input feels like a “we are going to hit pause on this for now and revisit it in the future.” I may be wrong, but it reads weird. https://twitter.c…
@riana_crypto Riana Pfefferkorn on x
@hackermath (I do mean that seriously; legislation will be back on the table soon, not least bc if Apple bins their scanning plan entirely rather than just “pausing” & coming back w/ a new, improved, lemon-scented version, our & other govs will be LIVID. Apple painted itself into…
@sarahjamielewis Sarah Jamie Lewis on x
Took Apple a month from first publishing the proposal to move to “we have decided to take additional time [before rolling this out]”. In a world where so much is forgotten in a 24-hour news cycle, to see a surveillance issue stay relevant for that long is somewhat optimistic.
@drbarnard David Barnard on x
Apple would do well to talk to developers before big developer related announcements as well. Their culture of secrecy is strategy debt from a bygone era that is now doing far more harm than good. https://twitter.com/...
@malwarejake Jake Williams on x
Apple is doing the right thing here. Too bad they didn't get feedback from researchers in the first place. https://twitter.com/...
@matthew_d_green Matthew Green on x
NCMEC deciding to get involved on the advocacy side of this issue sure didn't help them. https://twitter.com/...
@evan_greer Evan Greer on x
“It's encouraging that the backlash has forced Apple to delay this reckless and dangerous surveillance plan, but the reality is that there is no safe way to do what they are proposing,” I told the @washingtonpost for this story https://www.washingtonpost.com/ ...
@neilcybart Neil Cybart on x
In some ways, Apple's problem with CSAM detection went beyond CSAM detection. Here's a snippet from the August 16th daily update. Minor changes aren't going to calm such fears / pushback. https://www.aboveavalon.com/ ... ($) https://twitter.com/...
@riana_crypto Riana Pfefferkorn on x
I find it super interesting that this event to talk up Apple's CSAM plans was canceled on like 14 hours' notice, in terms of what it suggests about the decisionmaking process & timing inside Apple w/r/t deciding to “pause.” https://twitter.com/...
@neilcybart Neil Cybart on x
To be clear, I think concerns that Apple would take lessons from CSAM detection and then build additional detection apparatuses that are far more controversial (i.e. detect people who write certain things in the Notes app) are misplaced.
@carnage4life Dare Obasanjo on x
I struggle to understand the lack of perspective that it takes to start a privacy crusade against companies showing people personalized ads as a privacy invasion only to announce you'll be scanning people's photos on their devices for illegal content. It boggles the mind. https:/…
@evan_greer Evan Greer on x
For decades if not centuries “think of the children” has been the go-to justification for tyrants interested in control, not safety. It feels like the tides are starting to turn as people wise up to the fact that surveillance makes kids less safe, not more https://www.wired.com/.…
@matthew_d_green Matthew Green on x
The degree to which Apple has screwed NCMEC and content scanning in general cannot be overstated.
@matthew_d_green Matthew Green on x
I've seen two people post PhotoDNA implementations in the last two weeks. One is on the front page of HN. These are algorithms that were secret and under NDA for years before Apple's announcement. (And Apple doesn't even use PhotoDNA.)
@futureofprivacy @futureofprivacy on x
Sorry friends, tomorrow's #PrivacyChat with @Apple CPO Jane Horvath is canceled. We'll share a rescheduled date in the coming days. https://twitter.com/...
@tapbot_paul Paul Haddad on x
@reneritchie It's a loss for Apple because all they managed to do is piss off everyone.
@timsweeneyepic Tim Sweeney on x
Great move by Apple to reconsider iCloud photo snooping. Scanning content for legality belongs in a publishing context - where one is sharing it with the public - not on a customer's device, and not when storing one's personal effects. https://twitter.com/...
@aclu @aclu on x
Apple is right to hit the pause button instead of rushing ahead given all the serious privacy & civil liberties concerns. https://twitter.com/...
Nadim Kobeissi🇫🇷🇪 🇺 on x
Thanks Apple execs for kinda sorta listening outside of your zone. Glad I can use my iPhone without feeling like a total dumbass again. Next time some nonsense encroachment on general purpose computing or privacy flares up, the screeching voices of the minority will be waiting!
Nadim Kobeissi🇫🇷🇪 🇺 on x
This shouldn't have been delayed, it should've been canceled altogether
@reckless Nilay Patel on x
Also: Apple sent this statement “on the record” with instruction to attribute it to Apple. Which is a fun redefinition of on the record!
@granick @granick on x
But this is where the difficult and fun part begins. As I wrote about the Apple CSAM scanning proposal, is there any way to engineer client-side scanning where you can be confident it won't devolve to scanning other information stored on a device? https://twitter.com/...
@granick @granick on x
With Apple's announcement that it's going to delay implementation of child safety features until it, you know, actually talks to people and gets feedback, and this news that WhatsApp is going to start encrypting back ups, privacy advocates have a moment to feel good about things.
@backlon Dieter Bohn on x
Think about it: we are days away from a new iPhone event invite and the discussion is about ✅ Anti-steering provisions ✅ Shifty PR tactics ✅ dodgy definitions like “reader app” ✅ CSAM I mean that's one way of clearing the decks I guess but WHEW
@backlon Dieter Bohn on x
This ENTIRE WEEK has been about Apple backing down on its policies in small ways and, now, a big way: https://www.theverge.com/...
@lukolejnik Lukasz Olejnik on x
For some reason, nobody at Apple wanted to link their name with the photo-scanning system. Interesting. https://twitter.com/...
@juliaangwin Julia Angwin on x
Love to see more news outlets demanding on the record statements and names from corporate spokespeople! (This has been our policy @themarkup all along - but for a long time, we felt like we were swimming upstream https://www.getrevue.co/...) https://twitter.com/...
@trotskyplug Sam on x
@viaCristiano The sass of naming him in this sentence, lmao
@evan_greer Evan Greer on x
lol https://twitter.com/...
@daiwaka Daisuke Wakabayashi on x
OK, this might have to be my move from now on. 👏👏👏 https://twitter.com/...
@bobbyallyn Bobby Allyn on x
Why don't all major publications band together and form a pact that we won't quote tech spokespeople anonymously anymore? https://twitter.com/...
@jonbradyphoto Jon Brady on x
reason #429 for adopting the US convention of naming spokespeople: it actually makes them do their jobs https://twitter.com/...
@chris_maag Christopher Maag on x
Unnamed sources play an important role in journalism. But the practice is deeply abused. Hooray to @washingtonpost for starting to reset boundaries. https://twitter.com/...
@dangillmor Dan Gillmor on x
Too early to say Apple is backing down from its plan to open a content-scanning backdoor into iPhones, starting with photos, but postponing the move is progress. So is the company's statement that it'll more seriously discuss this more widely. https://arstechnica.com/...
@adrjeffries Adrianne Jeffries on x
I'm happy to see this catch on. Company spokespeople are still surprised to hear this is @themarkup's policy. cf @JuliaAngwin https://www.getrevue.co/... and @bcmerchant https://www.cjr.org/... https://twitter.com/...
@matthew_d_green Matthew Green on x
In a limited sense I'm grateful to Apple for making such a big splash this summer with their scanning proposal, and for making it so broad and expansive (and unpopular.) It takes a company like Apple to actually bring these ideas out into the public sphere. 5/
@quinnypig Corey Quinn on x
Petition to make this standard journalistic practice, particularly when dealing with Amazon. Own the things you say; if you wouldn't sign it, why are you saying it? https://twitter.com/...
@reneritchie Rene Ritchie on x
This isn't a “win” for privacy or a “loss” for child protection Balancing those isn't some dumb influencer boxing match. It's a complex problem with multiple trade-offs, and deserves way more humility and consideration than the gross clout-chasing, attention-jacking hot quotes ht…
@imeluny Melanie Ensign on x
Takeaway for other orgs: engage with external experts, communities, & activists BEFORE announcing potentially controversial features. You need to be able to cite those discussions upfront. The ability to identify what might be considered controversial is a performance metric. htt…
@evan_greer Evan Greer on x
Organizing works. Apple's dangerous photo & message scanning proposal has been condemned by security experts, civil rights groups, LGBTQ orgs, international human rights advocates and thousands of people from across the political spectrum. Don't let up now https://www.cnbc.com/..…
@jonathanturley Jonathan Turley on x
Apple is “delaying” the release of its NeuralHash technology. https://www.cnbc.com/... However, the company is saying that it will be tinkering with (not cancelling) the program. roohttps://jonathanturley.org/2021/08/ 13/the-shadow-state-neuralhash-and- apples-post-privacy-world/
@snowden Edward Snowden on x
Remember all the tweets about @Apple's insane #spyPhone proposal? Don't ever let anyone tell you that there's nothing you can do when a company announces a plan to screw you. This is a HUGE victory, but remember: this thing isn't dead yet. Be ready to fight if it comes back. http…
@tomwarren Tom Warren on x
good. I hope Apple takes all of the concerns into consideration here https://twitter.com/...
@jbrodkin @jbrodkin on x
There are a ton of headlines saying Apple “delayed” this plan but Apple never announced a release date other than “this year,” and the statement that it will “take additional time over the coming months” doesn't necessarily prevent it from being released before the end of 2021 ht…
@matthew_d_green Matthew Green on x
This is the one I can't entirely blame on poor executive decisions. The technical folks at Apple had to know how broken it would be the second the design became public.
@markgurman Mark Gurman on x
Story on Apple child abuse photo library and Messages feature delay https://www.bloomberg.com/... https://twitter.com/...
@bantg Banteg on x
Apple decided not to roll out client-side scanning, for now. Activism works, guys. Great work everyone who helped to get our concerns heard. https://appleinsider.com/...
@reckless Nilay Patel on x
Apple is delaying its photo scanning features after the backlash. Amazing how secrecy and hubris turned what could have been a major win - encrypted iCloud - into a communications disaster https://www.theverge.com/...
@stevekovach Steve Kovach on x
I'm more surprised Apple didn't wait to dump this news tonight ahead of the holiday weekend. https://www.cnbc.com/...

Chronicles