In response to CSAM detection misuse concerns, Apple says protections will roll out in the US first, then on a country-by-country basis after legal evaluation

Apple this week announced that, starting later this year with iOS 15 and iPadOS 15, the company will be able …

MacRumors 2021-08-07 Joe Rossignol

Context & Ripple Effects

Apple initially paired iOS 15 and macOS Monterey with a plan to scan for known abuse imagery in the U.S. The country-by-country legal review makes that a phased compliance program rather than a uniform global feature rollout. Subsequent coverage shows the proposal drew enough stakeholder scrutiny that Apple delayed the child-safety rollout to gather feedback and make improvements.

First-order effects

Apple must clear legal requirements market by market after its U.S. launch, while users outside the U.S. are excluded from the initial deployment.
The company’s safeguards face immediate scrutiny alongside the planned photo-scanning rollout on iOS 15 and macOS Monterey.

Second-order effects

A jurisdiction-by-jurisdiction schedule increases the importance of Apple’s system controls, including the later-described match threshold before manual account review, because those controls must withstand different legal assessments.
Privacy stakeholders and regulators gain leverage over the release sequence: Apple’s later delay shows that implementation feedback can alter the product timetable.

Third-order effects

Content-safety features on consumer devices are moving toward geographically differentiated operation, with legal acceptability shaping where a single platform can deploy the same enforcement capability.
If that pattern persists, Apple’s global safety-policy design will be governed less by one launch decision than by repeatable audits, thresholds, and local legal review.

The trend: Consumer-platform safety enforcement is becoming a jurisdiction-by-jurisdiction governance problem rather than a one-time global product launch.

Discussion

@sarahjamielewis Sarah Jamie Lewis on x
Clearly a rubicon moment for privacy and end-to-end encryption. I worry if Apple faces anything other than existential annihilation for proposing continual surveillance of private messages then it won't be long before other providers feel the pressure to do the same.
@feross @feross on x
Now that Apple has willingly built spyware into iOS and macOS, within 10 years this tech will: (1) be mandated by government in all end-to-end encrypted apps; and (2) expand to scan for terrorism, disinformation, “misinformation”, then eventually political images and memes. 1/5
@sarahjamielewis Sarah Jamie Lewis on x
You can wrap that surveillance in any number of layers of cryptography to try and make it palatable, the end result is the same. Everyone on that platform is treated as a potential criminal, subject to continual algorithmic surveillance without warrant or cause.
@reckless Nilay Patel on x
Really seems like Apple tried to protect customer data in the cloud by scanning for illegal material locally on the phone, thereby creating a new kind of risk for customer data on the phone.
@mattblaze Matt Blaze on x
The only think preventing this scheme from scanning local photos (which Apple never previously had access to) is the policy and the integrity of the code. Both those things warrant close scrutiny.
@matthew_d_green Matthew Green on x
I spoke to talk radio in LA about Apple's scanning system. I was steeling myself to explain how the tech is problematic even if it might catch some bad people. I didn't need to. They asked me how to disable it.
@reneritchie Rene Ritchie on x
@BriannaWu Yeah, I tend towards privacy extremism. My guess here is Apple could not longer abide CSAM on their servers and this was the best engineering solution they could come up with to avoid scanning libraries online as others do. Feels like a loss though.
@migueldeicaza @migueldeicaza on x
The EU is actively working on legislation on this regard, so I am now convinced that this is Apple influencing the outcome so we don't end up with the equivalent of the cookie pop up for photos and messages: https://ec.europa.eu/... and https://www.consilium.europa.eu/ ...
@intelwire @intelwire on x
Optimal 2 just writing itself out here https://twitter.com/...
@austen @austen on x
No no no. Apple isn't doing something nasty like having a back door on your phone that uploads your images to the cloud scanning and calling the police on you. That would be evil. Big brother! Instead, no backdoor is needed because the phone does it automatically. Privacy! https:…
@ncweaver Nicholas Weaver on x
@migueldeicaza @mattblaze @alexstamos I think the bigger collision problem is the ML porno-detection in iMessage. You can bet someone is going to tweak an image of Elmo so that it shows up as porn, and kids are going to think it is so funny to spread it around...
@briannawu Brianna Wu on x
@reneritchie I'm very strongly against this move. But I also think reasonable people of good will can disagree. I know it's hard to imagine in 2021, but sometimes people can disagree and no one is a villain.
@migueldeicaza @migueldeicaza on x
@icanzilb @DonnyWals @steveriggins Pretty sure this was implemented because of the EU, check the recent legislation weakening privacy for this which was a stop gap for the additional legislation. I think they did this to do this on Apple's terms, rather than a bureaucrat's term (…
@mantia Louie Mantia, Jr on x
Apple making a process to scan hashes won't solve many actual problems, but gives up everything they stood for. I'm uninterested in debate or replies. If you wanna argue with someone, argue with someone else.
@pwnallthethings @pwnallthethings on x
Apples approach bridges the gap to avoid abandoning either. In effect, scanning the content prior to *upload* on *their own service* that until now was functionally breaking principle 2 so that their service can be designed to honor principle 2 in future without the conflict
@friedrichpieter Pieter Friedrich on x
Aside from the shock that @Apple plans to expose every user to snooping powers which can & will inevitably be used to target dissidents, I still don't understand how the US thinks a good way to stop child abuse is to create a centralized database full of pictures of child abuse. …
@rondeibert Profdeibert on x
Apple's solution to a serious, harmful problem (child sexual exploitation) is alarming b/c of the door it opens to other surveillance, and the risks around what countries like China will do with it, once opened. As @josephfcox points out, these are not hypothetical concerns: http…
@pwnallthethings @pwnallthethings on x
The first one brings you not just into conflict with authorities, but into conflict with civil society itself, because the harm is real and widespread, and at a certain point the dams break and the company switches to abandoning the second principle instead.
@briannawu Brianna Wu on x
I spoke with @MissEmmaMcG about the ways Apple's “child protection” technology could be used to hurt LGBT children. Also, @Snowden showed how these technologies are used to spy on other countries in the name of “national security.” https://blog.avast.com/...
@pwnallthethings @pwnallthethings on x
It's tempting to handwave it away as pretextual. It's not. For reference, in 2018 Facebook was doing about 17m referrals *a month* of roughly 700,000 unique images.
@briannawu Brianna Wu on x
2/ It's a hard subject to talk about, but I do think teenagers having some measure privacy and autonomy on figuring out their sexuality is normal and healthy. It's just so easy to imagine ways this could be abused.
@snowden Edward Snowden on x
The media is beginning to write with noticeably more skepticism about @Apple's plan to transform your iPhone into a spyPhone. Keep pushing! https://www.mediaite.com/...
@pwnallthethings @pwnallthethings on x
Fundamentally this technology is attempting to bridge the gap between two very strong social principles. The first is that the child abuse image problem is real and very, very large, involving extraordinary damage to a really depressingly large number of children.
@danny76lopez Danny Lopez on x
@MacRumors @rsgnl So @apple wont unlock a mass shooters Iphone citing privacy and the slippery slope but they are happy to scan everyones images for potential child porn. Kinda like the double standard for privacy they have in China. What am I missing?
@migueldeicaza @migueldeicaza on x
@ncweaver @mattblaze @alexstamos Ah yes, I had not thought of that. That feature and the Siri search felt like padding for softening the blow of the announcement.
@mantia Louie Mantia, Jr on x
I haven't seen any Apple employees publicly criticize this new policy. I know we shouldn't expect that. But it's also kind of... horrible that we cannot expect that. Apple employees surely are themselves angry, but are silenced by their employment.
@matthew_d_green Matthew Green on x
What would you say if Apple announced that Siri will always listen and report private conversations (not just those triggered by “Hey Siri") but only if a really good neural network recognizes them as criminal, and there's PSI to protect you?
@normative Julian Sanchez on x
Apple's iPhone: Now With Built‐ In Surveillance https://www.cato.org/...
@davezatz Dave Zatz on x
@Techmeme @rsgnl How many photos does Apple process through iCloud in a given year? Is an actual user-uploaded photo presented within the report for Apple employees to view? https://twitter.com/...
@tculpan Tim Culpan on x
Apple 2021 is starting to feel like Microsoft circa 2000. Strong with overtones of bully. Use a known atrocity to push acceptance of their own tech agenda. Stand up against the agenda and you're painted as in favor of the atrocity. Back then it was anti-P2P, now pic scanning.
@runasand Runa Sandvik on x
The fact that we are all struggling to understand what Apple is doing on our devices and with our backups is concerning. Even more so when we don't know how this will impact our lives in the future.
@normative Julian Sanchez on x
And a whole bunch of the arguments Apple deployed in the San Bernardino encryption case don't obviously apply if they've already built the scan architecture & a government is just adding items to a preexisting list.
@alexstamos Alex Stamos on x
@mattblaze I'm surprised that they didn't document their new hash. It is, presumably, robust enough to maintain its security properties even if reverse engineered from the binary.
@rondeibert Profdeibert on x
Stepping back, this underscores real risks of what @doctorow calls “digital manorialism” (a reference to medieval political economy I like very much). Apple's walled garden may look attractive some days, but never forget: the warlord calls all the shots https://twitter.com/...
@normative Julian Sanchez on x
I'm curious how far they've thought out the legal end of this. A government (ours or an uglier one) approaches Apple with a court order saying “here's a list of hash values we want you to add to the scan list you're pushing out”. Can they refuse? Or even tell anyone?
@pwnallthethings @pwnallthethings on x
The second principle is that Apple, in general, wants to not hold user data. Folks can debate exactly how much of that is genuine privacy reasons vs just not wanting to deal with floods of subpoenas and risk of holding it, but it's also real. They don't like holding private data.
@reneritchie Rene Ritchie on x
“Why are you defending Apple?!” I'm defending facts and people. I personally may love X, hate Y, not know/care about Z “Then why are you arguing?” Because FUD is harmful and facts matter. If someone wants to love or hate something, they deserve to love or hate it smart! 💛🙏
@pwnallthethings @pwnallthethings on x
Most companies resolve this conflict by dropping one of those principles. Some decide to not do scanning; treating the social harm as an externality or pretending it is not real. Others quietly drop plans to not fully encrypt.
@normative Julian Sanchez on x
If everything operates precisely as Apple intends, scanning client-side is at worst an accounting detail & at best enables greater privacy. If everything does not operate precisely as Apple intends, moving scans client-side is a dangerous Rubicon to cross.
@stevestreza @stevestreza on x
Ah yes, the Apple that infamously and repeatedly has bent over backwards to give China deep access to device and cloud data, we're supposed to trust their spyware because lawyers got involved. Apple is not a privacy company. https://twitter.com/...
@reneritchie Rene Ritchie on x
The technical aspect of this solution appears to be incredibly privacy-centric, and well thought out and implemented. (Time will, obviously, test and tell.) But bad/poor/incorrect technical hot takes are distracting from valid/critical ethical and philosophical discussions. https…
@pwnallthethings @pwnallthethings on x
The alternative to this system is not a world where Apple abandons the first principle; its unsustainable. It's one where they quietly drop the second. And if you care about privacy, them bridging the gap in a way that brings the scanning on device instead of on server is a win.
@briannawu Brianna Wu on x
@reneritchie That part worries me a lot less than the iMessage spying. I feel pretty strongly that it's going to get a lot of queer children disowned and hurt.
@mantia Louie Mantia, Jr on x
I haven't worked at Apple in 10 years and *I* feel bad about this. I feel bad that I contributed even the most trivial visual details, that in the aggregate work to gain customers' trust. Now Apple seeks to erode that trust? I feel bad for contributing to it.
@mantia Louie Mantia, Jr on x
I have no patience for people who won't see how this is going to do more harm than good. Apple building any way to scan images will become a larger problem. There's no reason to believe people who possess these images won't... just simply stop using Apple products.
@khaost Khaos Tian on x
It's really cute that Apple keeps defending the system like this, as if once an authoritarian government passes the law to require device manufacturers to scan for other stuff, the system itself will prevent that from happening. https://twitter.com/...
@normative Julian Sanchez on x
I should add, because a couple folks have noted this: IF this were implemented only as Apple intends and for the purposes it states, then yes, this would be functionally identical to the sort of scanning that's routine on platforms & cloud storage services....
@feross @feross on x
I'm incredibly disappointed that this was approved and built by @Apple. The short-sightedness is staggering. How can they think governments won't demand to expand this? Before today, I believed that Apple genuinely cared about my privacy. But no more. This is a disaster. 5/5 http…
@sarahjamielewis Sarah Jamie Lewis on x
If Apple are successful in introducing this, how long do you think it will be before the same is expected of other providers? Before walled-garden prohibit apps that don't do it? Before it is enshrined in law?
@josephmenn Joseph Menn on x
We have so many important fights taking place in secret courts that it is theoretically possible that Apple's surprising device-scanning system was hatched to settle some hidden legal demand. Probably not, but still. Not a good method for technology or democracies to develop.
@samifathi_ Sami Fathi on x
What isn't getting talked about is the real-world impact CSAM scanning will have. Think about the children who have been exploited and the scars they'll have for the rest of their lives. If finding pedophiles means Apple needs to do on-device processing of photos, then so be it.
@pinboard @pinboard on x
@charlesarthur @mathewi It's the same issue as with FaceID; the fact that the locus of the scanning has moved to the device is highly significant no matter how many safeguards Apple thinks they're putting around it, or how limited the initial scope. It is a big deal and people ar…
@tirsales Sebastian Nerz on x
That's the problem. Technology is neutral - a filter doesn't care whether it's scanning for child abuse or terrorists or legitimate protests against a dictatorship (scanning for powerful images of protest would be easy). https://twitter.com/...
@elcomsoft @elcomsoft on x
“Apple will no longer be able to honestly call iMessage end-to-end encrypted.” - well, _honestly_ they never were. More details (note the date): https://blog.elcomsoft.com/... #icloud https://twitter.com/...
@arnoldkim Arnold Kim on x
Great overview and take on Apple CSAM stuff by @gruber - Apple's messaging/clarity on this could have been much much better, but it's not entirely a messaging issue. https://daringfireball.net/...
@mattrosoff Matt Rosoff on x
Amazing how many sources got this story completely abjectly wrong. The slippery slope arguments are reasonable and worth considering—but if you don't get the technology, you can't even begin to plausibly make those arguments. https://daringfireball.net/...
@parrots Curtis Herbert on x
If you're like me you might have had a hard time cutting through the noise to understand exactly what is going on. This is a very good piece breaking down exactly what Apple is doing, and what they can and cannot see. A good 101 before convos about pros and cons and slopes. https…
@missingkids Ncmec on x
“Apple's expanded protection for children is a game changer,” said John Clark, president and CEO of NCMEC. https://www.apple.com/...
@snowden Edward Snowden on x
No matter how well-intentioned, @Apple is rolling out mass surveillance to the entire world with this. Make no mistake: if they can scan for kiddie porn today, they can scan for anything tomorrow. They turned a trillion dollars of devices into iNarcs—*without asking.* https://twi…
@snowden Edward Snowden on x
🚨🚨 Apple says to “protect children,” they're updating every iPhone to continuously compare your photos and cloud storage against a secret blacklist. If it finds a hit, they call the cops. iOS will also tell your parents if you view a nude in iMessage. https://www.eff.org/...
@swiftonsecurity SoS on x
Just to state: Apple's scanning does not detect photos of child abuse. It detects a list of known banned images added to a database, which are initially child abuse imagery found circulating elsewhere. What images are added over time is arbitrary. It doesn't know what a child is.
@bennypinkas Benny Pinkas on x
I reviewed the Apple PSI system that will be used for detecting photos with CSAM (child sexual abuse) content while keeping the contents of all non-CSAM photos encrypted and private. This system is only used in iCloud Photos — not iMessage.
@marcan42 Hector Martin on x
Good article on how this becomes a problem, even when not intentionally targeted. https://areoform.wordpress.com/ ...
@wagatwe Wagatwe Wanjuki on x
There's already been a lot of smart commentary on why this is a terrible idea. But I also wanted to add another piece of context: Violence against women and children is often used as an excuse to expand state surveillance without actually helping. https://www.apple.com/...
@marcan42 @marcan42 on x
Any automated CSAM matching system introduces an exploitable vulnerability to completely ruin someone's life.
@matthew_d_green Matthew Green on x
So the Apple scanning system just dropped. https://www.apple.com/...
@stroughtonsmith Steve Troughton-Smith on x
Wait, initial hash database comes from the authorities in any given country and isn't verified beforehand? Apple's only way to find out is w/ manual review after a user has been flagged? Using same under-paid & overworked content moderation contractors that could be infiltrated? …
@reckless Nilay Patel on x
Just to back this out farther: Apple already scans iCloud. If they had said “we are now scanning iCloud Photos for CSAM” they would have just matched the rest of the industry. Avoiding that with a complicated local scanning and hashing system created new risks entirely. https://t…
@timmarchman Tim Marchman on x
Not the main point but Facebook's decontextualized CSAM numbers are actually a serious problem according to law enforcement people I've talked to, because they are a distorted lens and mislead the public and policymakers. https://twitter.com/...
@vickerysec Chris Vickery on x
There is legal precedent which explains why Apple should not be allowed to do this- https://arstechnica.com/... The reasoning is very simple: Apple is not law enforcement. No matter how badly a commercial tech company *wants* to wield the power of law, it doesn't. Plain and simpl…
@joewintergreen Joe Wintergreen on x
the idea that a colossal tech company might suddenly decide to pursue an altruistic agenda of abuse prevention is, no joke, the most laughably barefaced complete fucking fairytale bullshit any of us will ever hear in our lives https://twitter.com/...
@samthielman @samthielman on x
This sucks. Any student of the FBI knows what this is: violating civil rights in the name of protecting the victims of revolting edge cases. The next step is always, ALWAYS, citing those violations as precedent. https://www.washingtonpost.com/ ...
@tomgara Tom Gara on x
The child abuse stuff got most of the attention yesterday but the second part seems way more expansive: iPhones will use some kind of computer vision system to search for *any* nudity in images in iMessage, not just illegal stuff https://twitter.com/...
@senblumenthal Richard Blumenthal on x
Thank you @tim_cook for taking action to help catch sexual predators & prevent the horrific abuse of children—now it's time for others in Big Tech to step up & for Congress to pass the EARN IT Act.
@evacide Eva on x
Louder, for the people in the back: it's impossible to build a client-side scanning system that can only be used for sexually explicit images sent or received by children. https://www.eff.org/...
@senblumenthal Richard Blumenthal on x
After a long & forceful push from me & @LindseyGrahamSC, Apple's plans to combat child sexual exploitation are a welcome, innovative, & bold step. This shows that we can both protect children & our fundamental privacy rights. https://www.reuters.com/...
@nytimestech @nytimestech on x
Apple unveils changes to iPhone designed to notify parents of child sexual abuse. But some privacy watchdogs are concerned about the implications. https://www.nytimes.com/...
@jcenters Josh Centers on x
Lot of Apple pundits running out to defend Apple, not many openly saying how bizarre this move is. Hard to believe Apple just up and built an expensive surveillance system that destroys their carefully crafted privacy image on their own accord.
@mala Danny O'Brien🤖 on x
So for the last few years, intelligence agency experts, like @GCHQ, have been pushing for tech companies to put remotely-controlled software to scan for unlawful content in people's devices, rather than surveil on the network. https://www.eff.org/...
@mala Danny O'Brien🤖 on x
This is bad. Really bad. And Apple was embargoing the news, so journalist may not be talking to experts, like @ohemorange and @joncallas at @EFF, about the consequences for innocent users — not just Apple users, but anyone who stores data on their devices. https://twitter.com/...
@matthew_d_green Matthew Green on x
Apple is managing an embargo of reporters right now. It's quite a remarkable thing watching them do this, goal is to make sure everyone reports this when Apple wants and on Apple's terms.
@a_greenberg Andy Greenberg on x
Seems like Apple's idea of doing iCloud abuse detection with this partially-on-device check only makes sense in two scenarios: 1) Apple is going to expand it to non-iCloud data stored on your devices or 2) Apple is going to finally E2E encrypt iCloud? https://www.wired.com/...
@eff @eff on x
Apple's filtering of iMessage and iCloud is not a slippery slope to backdoors that suppress speech and make our communications less secure. We're already there: this is a fully-built system just waiting for external pressure to make the slightest change. https://eff.org/...
@caseynewton Casey Newton on x
I tried to sort through my feelings about Apple's child safety announcements today. The risk of the slippery slope is real — but there are also real harms taking place on iCloud today, and it's good that Apple has chosen to pay attention to them https://www.platformer.news/ ... h…
@benedictevans Benedict Evans on x
There is something quite theological in Apple's idea that doing something in your devices is private and doing exactly the same thing in the cloud is not.
@captn3m0 Nemo on x
Reminder that the Indian government has successfully bullied Apple in the past to make iOS changes (TRAI). Could they pressure Apple into adding new hashes (of political memes) in the next iOS release? I'd rather not find out. https://twitter.com/...
@caseynewton Casey Newton on x
@daiwaka for sure, but I think I'm OK with this tradeoff? FB reported 15.8M cases of CSAM to NCMEC in 2019; Apple reported 205. Seems likely that there is massive under-reporting of very bad stuff on iOS
@matthew_d_green Matthew Green on x
Because surely it will ensure this, right? You'd want to ensure that Apple (or someone who hacks Apple's servers) can't change the database selectively to target it to you — and have a normal CSAM database for everyone else.
@bzamayo Benjamin Mayo on x
If the new Apple photo scanning stuff was looking at any and all photos stored on your device, I think there'd be somewhat of an ethical debate to be had. But it's just applied to iCloud Photos, which makes it no different than any other major web service or social network.
@jsrailton John Scott-Railton on x
NEW: @Apple to start checking photos on iPhone & iCloud against a blacklist of CSAM imagery. To watch out for: Now that the tech is on the table, how soon till China & other authoritarians w/political blacklists lean on Apple to search citizens' phones? https://www.apple.com/... …
@josephfcox Joseph Cox on x
New: much more interesting than Apple checking iCloud for known child abuse images is Apple will scan all images sent and received by children in Messages app to detect nudity. This is for new images, not already known. Expert concerned could be expanded https://www.vice.com/...
@santiagomayer_ Santiago Mayer on x
We can all agree we must do everything we can to stop child abuse. ...But having Apple scan all my pictures and potentially send them to human reviewers is not something I'm comfortable with. https://techcrunch.com/...
@pwnallthethings @pwnallthethings on x
OK so a slightly longer thread on how we got to the Apple CSAM thing and why it's not going away
@wcathcart Will Cathcart on x
Instead of focusing on making it easy for people to report content that's shared with them, Apple has built software that can scan all the private photos on your phone — even photos you haven't shared with anyone. That's not privacy.
@wcathcart Will Cathcart on x
We've worked hard to ban and report people who traffic in it based on appropriate measures, like making it easy for people to report when it's shared. We reported more than 400,000 cases to NCMEC last year from @WhatsApp, all without breaking encryption. https://faq.whatsapp.com/…
@wcathcart Will Cathcart on x
We've had personal computers for decades and there has never been a mandate to scan the private content of all desktops, laptops or phones globally for unlawful content. It's not how technology built in free countries works.
@roi Roi Carthy on x
I literally had to put a story on the front page of the FT to get you guys to remove 120,000 people who traffic in CSAM in **public** WhatsApp groups, Will. https://twitter.com/...
@mikeisaac Rat King on x
(facebook was just waiting for an opportunity like this to hammer apple on privacy. some context from our previous coverage.... https://www.nytimes.com/...
@wcathcart Will Cathcart on x
This is an Apple built and operated surveillance system that could very easily be used to scan private content for anything they or a government decides it wants to control. Countries where iPhones are sold will have different definitions on what is acceptable.
@wcathcart Will Cathcart on x
Can this scanning software running on your phone be error proof? Researchers have not been allowed to find out. Why not? How will we know how often mistakes are violating people's privacy?
@lapcatsoftware Jeff Johnson on x
You know you've done something very wrong when even Facebook is creeped out by it. https://twitter.com/...
@avibarzeev @avibarzeev on x
FB has a “concern” with Apple using crypto tech to identify child exploitation photos on local devices, even though it preserves full privacy for anyone who doesn't exploit kids. Meanwhile FB scans your photos in the cloud. “Privacy” to FB means /their/ privacy, not yours. https:…
@blakereid Blake E. Reid on x
Apple's rake-stepping yesterday is now enabling semi-credible privacy dunks from Facebook 🙃 https://twitter.com/...
@wcathcart Will Cathcart on x
Apple once said “We believe it would be in the best interest of everyone to step back and consider the implications ...” https://www.apple.com/...
@sunchartist @sunchartist on x
Pot calling the kettle black https://twitter.com/...
@susanlitv Susan Li on x
Definitely no love loss between #Apple & #Facebook 👇 $aapl $fb https://twitter.com/...
@can @can on x
@ranjanxroy can you even use WA without giving FB full access to your address book?
@willhamill Will Hamill on x
The danger with using the “Think of the children!” argument to let Apple away with their invasion of your devices and data is it doesn't take a leap to see how once the precedent is established it can trivially be misused. https://twitter.com/...
@pwnallthethings @pwnallthethings on x
WhatsApp: Apple is bad for scanning for CSAM. This is the wrong approach and we would *never* do this. WhatsApp (but quieter) in 2018: we also do this and have since 2011. https://www.judiciary.senate.gov/ ... https://twitter.com/...
@can @can on x
@sp990 @thijsniks @ranjanxroy I'm old enough to remember WhatsApp was about never having any ads when founded. If Signal flip flopped constantly on policy, I'd stop using them too.
@wcathcart Will Cathcart on x
..."it would be wrong for the government to force us to build a backdoor into our products. And ultimately, we fear that this demand would undermine the very freedoms and liberty our government is meant to protect." Those words were wise then, and worth heeding here now.
@pinboard @pinboard on x
@pwnallthethings You're a really smart person and I know you understand the significance of doing this on the device vs. server side. The status quo is a tradeoff (because horrifying amounts of CSEM *do* get uploaded to any image sharing site), and Apple's move is a big shift in …
@st_eppel David Grunwald on x
Hey @Apple - when @Facebook is legitimately able to claim the moral high ground against you, you've probably screwed up bad https://twitter.com/...
@pinboard @pinboard on x
The threat is not just individual governments' misuse of this architecture, but that the whole thing in the hands of supranational entities with state-like power who believe in design by YOLO. It's a planetary social experiment with no checks or controls, run by high-IQ idiots
@floorter Floor on x
It's a bit awkward to see a Facebook representative dunking on Apple about privacy. But in the end it doesn't really matter if WhatsApp implements this if the underlying OS does. https://twitter.com/...
@can @can on x
@sp990 @thijsniks @ranjanxroy And the larger point is that if you have the data and change the policy later, your users are fucked. Signal doesn't have the data to begin with so you can't gotcha people. So not sure if your analogy works here.
@bcrypt Yan on x
there's lots of reasons to object to apple's CSAM proposal but “they shouldn't build software to scan your device” doesn't resonate with me. given a choice between plaintext backups scanned in the cloud and end-to-end-encrypted backups scanned on-device, i'd pick the latter.
@saranbyte Saran on x
It's not a good look for Apple when the head of WhatsApp even thinks it's a bad move 😬 https://twitter.com/...
@aditi_muses Aditi Agrawal on x
Not Twilight Zone: Head of Facebook-owned WhatsApp is berating Apple's latest move for violating people's privacy. Pay attention. Apple didn't create a slippery slope; it's now a free fall off a cliff. https://twitter.com/...
@bcrypt Yan on x
not saying this is the choice we are facing, but i hope this makes it clear that on-device vs in-cloud is not the issue here so much as the scanning itself.
@bcrypt Yan on x
given that CSAM scanning is only enabled for users who opt into icloud photo backups, i'm guessing apple would have built it into icloud if they could. they can't because of end-to-end encryption. https://twitter.com/...
@tihmstar @tihmstar on x
Technically he is not wrong. Yet hearing Whatsapp/Facebook talking about privacy is somewhat ironic afterall. Is really annoying how jailbreaking is still absolutely neccessary to have usable devices. First it was about usability, now it is about security/privacy... https://twitt…
@can @can on x
@thijsniks @ranjanxroy i guess my bar for trusting FB is p low 1) they've used SMS intended for 2FA for advertising 2) terms can change easily 3) if that's the level of comfort we have, apple doesnt have access to your photos either
@pwnallthethings @pwnallthethings on x
The completely perverse thing about the whole discussion is that *on-device* scanning enables you to do equivalent levels of CSAM protection *and then also encrypt everything in the cloud*.
@justinschuh @justinschuh on x
I once heard a friend describe the broader problem here as “there is nothing more legally and ethically fraught than running an open bit bucket on the Internet.” https://twitter.com/...
@pinboard @pinboard on x
The fucked up thing here is that design decisions about a worldwide surveillance architecture of social control get made by a small clique of individual companies, with no accountability to the billions of people their decisions affect, and no consequences for getting it wrong. h…
@bcrypt Yan on x
sorry this should say “because of eventual plans to do e2e encryption for icloud photos, or at least i hope” :)
@wcathcart Will Cathcart on x
Apple has long needed to do more to fight CSAM, but the approach they are taking introduces something very concerning into the world.
@charlesarthur Charles Arthur on x
Interesting WhatsApp declaration. Cathcart links to a blogpost (CEI = child exploitation imagery, same thing as CSAM = child sexual abuse material). Note the number of accounts zapped *per month*. https://twitter.com/... https://twitter.com/...
@chrismessina Chris Messina on x
Children don't have an absolute right to privacy from their parents. Will children really report CSAM that they receive via iMessage? Regardless, they can only block senders, not report them, presuming they can even figure that out. Curious @wcathcart's take on this. https://twit…
@sarahjamielewis Sarah Jamie Lewis on x
Update: These initial expressions of hesitance from Whatsapp are at least a small sign that there is some fight left in corporate e2e providers to reject mandates of on-device mass surveillance. Some reasons for optimism there. https://twitter.com/...
@mikeisaac Rat King on x
re: some of WhatsApp's statements, apple spent most of yesterday rebutting certain claims that WA is pouncing on mostly pointing out that if users turn off upload to iCloud then the system doesn't work and phones/iCloud won't be scanned I expect more back and forth to come https:…
@omanreagan Michael on x
Even the head of Facebook's WhatsApp thinks what Apple is proposing is a setback for privacy. Think about that for a minute. https://twitter.com/...
@luke_metro @luke_metro on x
How long has Facebook waited for a chance like this to dunk on Apple for privacy violations https://twitter.com/...
@earcos Eduardo Arcos on x
100% agree with Will Cathcart. Wrong approach from Apple to a VERY sensitive problem. https://twitter.com/...
@mikeisaac Rat King on x
@joelipper they're loving this
@tomwarren Tom Warren on x
the head of WhatsApp has concerns about Apple's image scanning approach. “Apple has long needed to do more to fight CSAM, but the approach they are taking introduces something very concerning into the world.” https://twitter.com/...
@wcathcart Will Cathcart on x
What will happen when spyware companies find a way to exploit this software? Recent reporting showed the cost of vulnerabilities in iOS software as is. What happens if someone figures out how to exploit this new system?
@wcathcart Will Cathcart on x
Will this system be used in China? What content will they consider illegal there and how will we ever know? How will they manage requests from governments all around the world to add other types of content to the list for scanning?
@kaepora Nadim Kobeissi on x
Apple distributed an internal memo today which referred to pushback against its new content surveillance measures as “the screeching voices of the minority.” I have nothing to add. https://twitter.com/...
@evacide Eva on x
Apple distributed this internal memo this morning, dismissing their critics as “the screeching voices of the minority.” I will never stop screeching about the importance of privacy, security, or civil liberties. And neither should you. https://twitter.com/...
@josephfcox Joseph Cox on x
Re-reading all of Apple's compromises in China to cater to the government there is interesting in light of the company making a system to scan for images stored on phones https://www.nytimes.com/... https://twitter.com/...
@rondeibert Profdeibert on x
Correction: that memo was by MCMEC and not Apple. Still, poorly thought rollout by Apple.
@snowden Edward Snowden on x
If you have a @github account, you can join me in co-signing the first letter uniting security & privacy experts, researchers, professors, policy advocates, and consumers against @Apple's planned moves against all of our privacy. https://appleprivacyletter.com/
@josephfcox Joseph Cox on x
We previously obtained a sample of malware deployed in Xinjiang that scans phones for 70,000 specific files related to politics, Uighurs, etc. Why wouldn't Chinese authorities try to get Apple to leverage its new child abuse system to instead fulfil this https://www.vice.com/...
@snowden Edward Snowden on x
Unbelievable: @Apple now circulating a propaganda letter describing the internet-wide opposition to their decision to start checking the private files on every iPhone against a secret government blacklist as “the screeching voices of the minority.” This has become a scandal. http…
@timsweeneyepic Tim Sweeney on x
It's atrocious how Apple vacuums up everybody's data into iCloud by default, hides the 15+ separate options to turn parts of it off in Settings underneath your name, and forces you to have an unwanted email account. Apple would NEVER allow a third party to ship an app like this.
@caseyjohnston Casey Johnston on x
.@apple remember when you literally couldn't make a functional keyboard. Th txt would loo lk this Answr m btc https://twitter.com/...
@kaepora Nadim Kobeissi on x
@cronokirby Incredibly difficult to resist typing with extreme clarity what I think Apple's security team and especially this Marita Rodrigues should go do with themselves.
@swiftonsecurity SoS on x
@tripswitch It was sent by an Apple employee
@lazerwalker @lazerwalker on x
To be clear, that specific phrasing comes from the NCMEC, not Apple management. This is still... an extremely disappointing way to internally communicate a complicated and nuanced ethical dilemma. https://twitter.com/...
@petersterne Peter Sterne🌹 on x
@SwiftOnSecurity The memo demonstrates that Apple is proud to work with NCMEC, a group that disdains privacy and security advocates. But it's a bit of a leap to say that Apple shares NCMEC's extreme views. Apple itself didn't refer to critics as a “screeching minority”.
@jon_prosser Jon Prosser on x
This is what the NCMEC said in a memo to Apple in response to the backlash to Apple's photo scanning 👇 Absolutely gross to frame this as “good vs. evil” and call fair criticism “screeching voices” Fuuuuuuuck that. Read the full article here: https://www.frontpagetech.com/ ... htt…
@rondeibert Profdeibert on x
Apple's memo calling people who raise concerns “screeching voices of the minority” doesn't instil confidence, and at very least shows terrible community engagement. As @evacide attests, it's only going to mobilize people who care about privacy & security https://twitter.com/...
@swiftonsecurity SoS on x
@weilbyte @tripswitch In a memo sent by Apple executive communications
@swiftonsecurity SoS on x
Despite being moderately passive or even supportive of Apple's goals yesterday, I had an invective-laden breakdown for leadership on how it was rolled out incompetently and why. I deleted it. It's increasingly obvious they weren't unprepared, it's someone's career stepping stone.…
@cronokirby Lúcás Meier on x
@kaepora “while preserving privacy” I genuinely think that Ms. Rodriguez sincerely believes this, which is why it's so irresponsible of us to not be honest about the nature of the systems we're creating.
@bascule @bascule on x
Oh cool, so the counterargument to people's concerns about Apple devices policing photos using a cryptographic Rube Goldberg machine is... an ad hominem https://twitter.com/...
@11rcombs @11rcombs on x
NCMEC executive director calls people worried about tech companies scanning every photo on your phone “the screeching voices of the minority”, in a note an apple VP calls “incredibly motivating” https://9to5mac.com/...
@je5perl Jesper Lund on x
@kaepora “Victimzed children will be rescued” is an amazing claim which cannot really be based on an understanding of the the Apple spyware actually does.
@kaepora Nadim Kobeissi on x
@cronokirby This memo genuinely makes me angry. Not Internet angry, not outrage angry, but genuine real anger. Imagine being so blind and short-sighted, imagine standing up the entire security & privacy community and then writing that off as “the screeching voices of the minority…
@kaepora Nadim Kobeissi on x
@lazerwalker The external note was attached to a team memo from Apple VP Sebastien Marineau-Mes, who introduced it with: “I wanted to share this note that we received today from NCMEC. I found it incredibly motivating, and hope that you will as well.”
@matthew_d_green Matthew Green on x
The fact that internal memos about content scanning are leaking from Apple does not say good things about support for those policies internally. https://www.google.com/...
@kaepora Nadim Kobeissi on x
If you too want to be part of the screeching voices of the minority, sign the open Apple Privacy Letter: https://appleprivacyletter.com/
@matthew_d_green Matthew Green on x
NCMEC also needs to dial it down a lot. This is offensive. Your job is not to toss civil society and Apple's customers under the bus. https://www.google.com/... https://twitter.com/...
@tim Tim Bradshaw on x
Latest reaction to Apple's child safety plans: EU👍 India 😍 UK 🥳 rest of Big Tech 😱 WhatsApp 🤬 https://www.ft.com/...
@kifleswing Kif on x
Critics of Apple's new plan don't see a better-engineered system that improves on what Google and Microsoft have been doing for years. Instead, they see a significant shift in policy from the company that said “what happens on your iPhone stays on your iPhone.” https://twitter.co…
@kurtopsahl Kurt Opsahl on x
WhatsApp gives a strong no to client-side scanning. “It's not how technology built in free countries works.” https://twitter.com/...
@samadlerbell Sam Adler-Bell on x
“Child pornography is so repulsive a crime that those entrusted to root it out may, in their zeal, be tempted to bend or even break the rules. If they do so, however, they endanger the freedom of all of us.” (US v. Coreas) https://twitter.com/...
@mikeisaac Rat King on x
head of WhatsApp on Apple's moves yesterday https://twitter.com/...
@annemariebridy Annemarie Bridy on x
The call to “think of the children” creates unassailable cover for expanding censorship and surveillance. What moral person doesn't want to do everything possible to stop the distribution of CSAM? https://twitter.com/...
@infocyte Steve Raikow on x
@Techmeme @ChanceHMiller Note that the memo doesn't articulate what they claim people are actually ‘misunderstanding’. This response is exactly the kind of passive aggressive rationalization that I'd expect from an abusive bully that is convinced that he's ‘really a good guy’.
@chrisbhoffman Chris Hoffman on x
“We know that the days to come will be filled with the screeching voices of the minority.” https://twitter.com/...

Chronicles