Apple is suspending Siri response grading, the program in which humans review recordings of Siri queries for quality control, in response to privacy concerns

In response to concerns raised by a Guardian story last week over how recordings of Siri queries are used for quality control, Apple is suspending the program world wide.

TechCrunch 2019-08-02 Matthew Panzarino

Context & Ripple Effects

The suspension caps a rapid escalation: a week earlier, The Guardian reported that Apple contractors regularly heard sensitive user information in graded Siri clips, often triggered by accidental activations and without explicit consent. Apple's answer is to halt the program globally rather than patch it mid-flight.

The move set the template for the industry — days later Amazon announced users could opt out of Alexa's human review, citing Apple's and Google's moves. Apple itself followed up within a month with a structural fix, an opt-in review process with work brought in-house and no recording retention.

First-order effects

Contractors doing Siri response grading worldwide lose the work immediately, and Apple loses the quality-control feedback loop on its voice assistant until a replacement exists.

Second-order effects

Rivals are forced into reactive privacy concessions rather than proactive design — Amazon added an Alexa opt-out for human review within days of Apple's suspension, showing how one vendor's exposure becomes every assistant's policy problem.

Third-order effects

Human review of voice recordings shifts from a silent default to a consent-gated feature across the assistant market, pushing vendors toward in-house review and minimal retention as the price of continuing quality control at all.

The trend: Voice assistants are being rebuilt around explicit user consent for human audio review, with each vendor's scandal resetting the industry's privacy floor.

Discussion

@clry2 Chris Yiu on x
The real problem here isn't humans listening to a sample of recordings: all machine learning systems need training to improve. It's the delta between people's expectations vs tech company reality that we need to fix - and that will take both sides https://twitter.com/...
@rustyshelf Russell Ivanovic on x
What happens on your iPhone stays on your iPhone.* .... *Until it turns it doesn't. Then if the story doesn't get much traction and you don't notice we'll still pretend it does. But when it blows up we'll fix it. Clear? Marketing level 💯 https://twitter.com/...
@panzer Matthew Panzarino on x
We are committed to delivering a great Siri experience while protecting user privacy. While we conduct a thorough review, we are suspending Siri grading globally. Additionally, as part of a future software update, users will have the ability to choose to participate in grading."
@ibnkhalid_ Ibrahim on x
“... the company's contractors “regularly” hear confidential and private information while carrying out the grading process, including in-progress drug deals, medical details and people having sex.” Major yikes https://www.theguardian.com/ ...
@reckless Nilay Patel on x
Apple definitely made this change because they wanted to and not because reporters kept asking why Google and Amazon were better about this, that's why it was scrambled out in the middle of the night https://www.theverge.com/...
@paulashby Paul Ashby on x
Hey Siri, please be sure to close that barn door after the last horse leaves, k? #SurveilYourselvesSuckers https://twitter.com/...
@a_bacci Anna Bacciarelli on x
And here's the impact of whistleblowing: Apple suspends the programme worldwide. How predictable that public outcry leads to more responsible corporate behaviour, rather than, say, internal review processes... https://techcrunch.com/...
@martinsfp Martin Sfp Bryant on x
So Apple has suspended humans listening to Siri interactions (and Google has done the same with Assistant in the EU) but let's see how long they both go before quality of service suffers. https://techcrunch.com/...
@backlon Dieter Bohn on x
One very important thing to notice about Apple's statement is that it says it has suspended “grading” Siri recordings. That's not the same thing as promising not to store recordings of your voice on its servers. It's just humans grading Siri's accuracy. https://www.theverge.com/.…
@carnage4life Dare Obasanjo on x
Apple, Amazon & Google have all come under fire for what was a standard industry practice of having humans transcribe speech their voice assistants couldn't understand. Begs the question of what other industry best practices would cause revolt if known? https://techcrunch.com/...
@markgurman Mark Gurman on x
When Apple starts grading Siri recordings again, users will be able to opt out. Apple should have added that ability when this discussion of companies listening to voice assistant records started in May.
@panzer Matthew Panzarino on x
It's suspending the program to do a review of the process. Concerns were raised by a Guardian article citing a contractor that was worried the audio snippets contained information that was too personal or potentially identifiable. https://www.theguardian.com/ ...
@panzer Matthew Panzarino on x
So, Apple is: 1. Suspending the program 2. Doing an audit of the process 3. Adding a way for users to explicitly agree that their snippets can be used in this way to improve Siri
@malteengeler Malte Engeler on x
Letting contractors listen to the recording isn't really the issue though, is it? It's the fact that there ARE recording in the first place which e.g. could be subpoenaed any time. https://twitter.com/...
@panzer Matthew Panzarino on x
Here she something interesting. Apple has globally suspended its Siri quality control grading program — the process it uses to see whether audio snippets are accurate or if there was an accidental trigger https://twitter.com/...

Chronicles