Facebook apologizes after its AI recommendation feature categorized a video of Black men as “about Primates”; Facebook disabled the feature and is investigating

New York Times 2021-09-06 Ryan Mac

Context & Ripple Effects

The incident joins a documented pattern of harmful automated labeling: Google Photos previously mistakenly tagged Black people with a simian term, while Facebook had already apologized over allegations of racism in its workplace culture. Facebook’s decision to disable the feature turns a reputational failure into an operational review of a product-facing AI system.

First-order effects

Facebook has removed the affected recommendation feature from use while it investigates how the category was produced.
The company must identify and correct the recommendation pathway before it can safely restore the feature, with Black users directly affected by the failure.

Second-order effects

Facebook’s earlier response to workplace-racism allegations makes another race-related apology more consequential for the company’s credibility around internal safeguards and product governance.
The recurrence of similar failures across Facebook and Google raises the bar for testing image- and video-related AI recommendations for racist associations before deployment.

Third-order effects

If platforms continue treating these incidents as isolated errors, automated classification tools will retain a recurring trust problem; sustained prevention requires safety review to become part of product release processes.
The pattern points toward greater scrutiny of how large platforms train, test, and disable consumer-facing AI features when they produce discriminatory outputs.

The trend: Consumer AI features are facing rising accountability for discriminatory classifications, shifting attention from model capability to product-safety controls.

Discussion

@tatendacheryl Tatenda Musapatike on x
And this is why I got stay reminding people that tech is racist by virtue of the conscious OR unconscious bias of the not diverse teams that create it. But they're sorry, they'll do better. https://twitter.com/...
@kateconger O...K on x
the fact that this has happened at multiple major tech companies really says something about the way AI is being built 👀 https://www.nytimes.com/...
@bernstein Joe Bernstein on x
As with ads/disinfo, I'd like to see the takeaway from stories like this, in addition to rightful anger, be that Facebook is a shitty technological product that relies on a kind of collective social delusion to take seriously https://twitter.com/...
@bernstein Joe Bernstein on x
No one should take a technology that labels black people as “primates” seriously ever again
@chriskeall Chris Keall on x
Think I'll give Zuckerberg's Metaverse a swerve https://twitter.com/...
@jason_kint Jason Kint on x
If they were an ethical company, they would be proactively trying to break AI like this to make certain the bias isn't there and this doesn't happen. Instead once again, “Facebook shut off the feature and is investigating. It also apologized after we reached out.” - @RMac18 https…
@rmac18 @rmac18 on x
This is what the AI-powered prompt looked like. The 3:30 minute video from the Daily Mail had nothing to do with “primates.” It was a set of two clips featuring Black men getting in altercations with a white male citizen and white police officers. https://www.nytimes.com/... http…
@elamin88 Elamin Abdelmahmoud on x
Oh my god. https://twitter.com/...
@keithkurson Keith Kurson on x
Don't turn on your “artificial intelligence-powered feature” if it can't pass this one fucking test. How are AI features still getting shipped if they can't pass the literal least they could do. https://twitter.com/...
@rmac18 @rmac18 on x
The biggest tech co's have struggled with biases in AI. In 2015, Google Photos categorized Black people as “gorillas.” Six years later, Facebook, which has one of the largest repositories of photos on which to train its AI, is dealing w/ the same issue. https://www.nytimes.com/..…
@nicoleperlroth Nicole Perlroth on x
Everyday we see the limits, biases and pure idiocy of relying solely on AI. And yet we continue to allow ourselves to be guinea pigs. @RMac18 with the latest. https://www.nytimes.com/...
@caro_milanesi Carolina Milanesi on x
“An unforgivable mistake” um yeah that is one way to put it! https://www.npr.org/...
@acidflask @acidflask on x
This industry hasn't learned a thing since Google Photos labelled Black people as gorillas in 2013 https://twitter.com/...
@oncodebrotha83 @oncodebrotha83 on x
Apology?? They think with black people they can get out of it without paying, there needs to be a lawsuit https://twitter.com/...
@ultraviolet @ultraviolet on x
There needs to be proactive anti-racist algorithms or coding AND internal staff trainings to address their biases in how racism percolate through the user experience. https://www.npr.org/...
@tweetsbydarci Darci Groves on x
Welp. This is not how I imagined I'd make the NYT, but at least it was for something important. https://www.nytimes.com/...
@s_m_i Stacy-Marie Ishmael on x
“Dani Lever, a Facebook spokeswoman, said in a statement: “As we have said, while we have made improvements to our A.I., we know it's not perfect, and we have more progress to make. We apologize to anyone who may have seen these offensive recommendations."" “as we have said” http…
@farkasjohan Johan Farkas on x
Facebook's AI labelled black men as ‘primates.’ Underlines the dire need for more research “beyond text-based analyses of overt and blatant racist speech” and into how “social media platforms' specific designs and policies play a key role in shaping racism"https://www.nytimes.com…
@simran Simran Jeet Singh on x
14% of Americans are Black. Only 4.4% of Facebook's US employees are Black. https://twitter.com/...
@blackamazon @blackamazon on x
In 2016 Facebook found most of its mis and disinfo was racially based . They also concertedly made an effort to separate high profile Black orgs from Black technologists who critiques it using “diversity” incubators We actually have to have that reconciliation https://twitter.com…
@161partisan @161partisan on x
At this point I almost expect Facebook to fuck up in the most mind-blowingly offensive ways with no consequences. #toobigtofail https://twitter.com/...
@sifill_ldf Sherrilyn Ifill on x
An “unacceptable error.” Remember when Zuckerberg explained that the failure to take down the Kenosha “call to arms” post that Kyle Rittenhouse responded to was “an operational mistake”? https://twitter.com/...
@rmac18 @rmac18 on x
I couldn't have reported this story if Darci, a former Facebook content design manager of 4 years, had not flagged it to a group of former FB employees and then talked to me. This stuff is so important, but can only come out if brave folks speak up. https://twitter.com/...
@ericproegler @ericproegler on x
Data Science needs the testing discipline. This report is a critical ethics problem. This is (or should be!) a crucial risk to brand and image. And... The number of unseen/unobserved AI errors is some gigantic multiple of the known issues, risking desired outcomes too. https://tw…
@rebel19 Rebecca Carroll on x
I'm so thoroughly exhausted. Facebook Apologizes After A.I. Puts ‘Primates’ Label on Video of Black Men https://www.nytimes.com/...

Chronicles