New Facebook test prompt asks users if someone they know might be “becoming an extremist”; another notifies users who may have been exposed to extremist content

New York (CNN Business)Some Facebook (FB) users in the United States are being served a prompt that asks …

CNN 2021-07-02 Oliver Darcy

Context & Ripple Effects

Facebook has repeatedly used user-facing tests to inform platform decisions, from surveys about trusted news sources to News Feed feedback experiments about what people find valuable. The new prompts extend that testing approach into extremist-content exposure.

The company had also tested crisis-oriented Safety Checks triggered by signals in posts and third-party confirmation. That earlier Safety Check model provides the closest precedent for using the product itself to surface a potential safety concern to users.

First-order effects

Some U.S. Facebook users are immediately asked whether someone they know may be becoming an extremist, while others are notified of possible exposure to extremist content.
Facebook gains direct user responses and engagement signals around a category it is testing, rather than limiting the experiment to ranking feedback.

Second-order effects

The test makes Facebook's safety interventions more visible to the people receiving them, increasing the importance of how accurately and clearly the prompts distinguish a concern from an accusation.
News Feed feedback tests and extremist-exposure notices can place more of Facebook's content-safety workflow at the interface between platform signals and user reporting.

Third-order effects

If Facebook broadens this model, moderation will increasingly combine platform detection with user-facing education, reporting, and feedback mechanisms rather than operating solely through behind-the-scenes enforcement.
The pattern points to safety features becoming a recurring product-testing category alongside ranking and trust surveys, with user response shaping how such interventions are deployed.

The trend: Facebook is bringing more content-safety work into user-facing prompts, applying its established feedback-testing playbook beyond News Feed preferences.

Discussion

@disclosetv @disclosetv on x
NEW - Looks like Facebook's AI is now starting to “think” about your exposure to “harmful extremist content” and your “extremist” friends. https://twitter.com/...
@alexberenson Alex Berenson on x
Yeah, I'm becoming an extremist. An anti-@Facebook extremist. “Confidential help is available?” Who do they think they are? Either they're a publisher and a political platform legally liable for every bit of content they host, or they need to STAY OUT OF THE WAY. Zuck's choice. h…
@oliverdarcy Oliver Darcy on x
Facebook spox confirms to me that this is a small test it is running: “This test is part of our larger work to assess ways to provide resources and support to people on Facebook who may have engaged with or were exposed to extremist content, or may know someone who is at risk...”…
@rachblevins Rachel Blevins on x
Why in the cornbread hell is Facebook asking me if I'm concerned about someone I know becoming an extremist? I'm much more concerned about the world's largest social media site censoring journalists and removing alternative media that challenge the US Gov't narrative... https://t…
@tprstly Theo Priestley on x
It's becoming easier to shop your friends and neighbours to the thought police. https://twitter.com/...
@sub8u Subrahmanyam Kvj on x
The @newyorker has a nice cartoon for this from some time ago 😂 https://twitter.com/... https://twitter.com/...
@rightwingcope Right Wing Cope on x
BREAKING: Facebook is implementing a new system to combat the spread of extremist content on their app https://twitter.com/...
@alexberenson Alex Berenson on x
Sorry. “Others in your situation have received confidential support.” Which is somehow even more infuriating, with its patronizing tone.
@kingfishfan @kingfishfan on x
Every time I open the Facebook app now I get that notification that I “may have been exposed to extremist content” when the funny part is I never even post anything on FB and only ever interact to like ppl's baby and animal pics and sometimes recipes. So that's not creepy at all
@sirhottest @sirhottest on x
Facebook thinks they're yo momma now https://twitter.com/...
@johnwilson John Wilson on x
So creepy. Is the point to snitch on your friends? What happens next? Law enforcement becomes involved, too? https://twitter.com/...
@hamburger Ellis Hamburger on x
do you know someone that is addicted to our products? we have videos about that, but we won't stop hosting, recommending, and monetizing the extremist groups 😁 https://twitter.com/...
@chadloder Chad Loder on x
This is the weakest possible response to what has become Facebook's most profitable demographic. Facebook is YEARS behind. https://twitter.com/...
@drscottjensen Scott Jensen on x
Have gotten numerous reports about this. When everyone is an “extremist”, no one is. It's a huge reason our campaign isn't taking a DIME from big tech. @Facebook https://twitter.com/...
@t_s_p_o_o_k_y Tony Shaffer on x
When did @Facebook become an extension of the @FBI? How is this legal? https://twitter.com/...
@cnnbusiness @cnnbusiness on x
Some US Facebook users are being served a prompt that asks if they are worried someone they know might be becoming an extremist. It's part of a test that stems from a company initiative aiming to combat violent extremism, a Facebook spokesperson says. https://www.cnn.com/...
@strassenkaempfe Tuk on x
it's like when pornhub asks you if you're over 18 lmaooooo https://twitter.com/...
@alt_uscis ALT-immigration on x
Facebook tests prompts that ask users if they're worried a friend is ‘becoming an extremist’ Facebook will have to hire every out of work adult to deal with this @brooklynmarie https://www.cnn.com/...
@fboversight @fboversight on x
“Are you feeling like an insurrectionist? Tell Facebook today!” https://twitter.com/...
@sharylattkisson @sharylattkisson on x
(Facebook knows, because Facebook is the one pushing the propaganda.) (Language warning) https://twitter.com/...
@primalpoly Geoffrey Miller on x
Facebook: 'You're angry and disappointed' People: ‘No, not really’ Facebook: ‘Yes, you really are. We are here to protect you from your anger and disappointment’ People: 'No thanks. I'm cool. ' Facebook: ‘WE WILL PROTECT YOU LIKE IT OR NOT’ https://twitter.com/...
@patrickmoorhead @patrickmoorhead on x
Sounds like what happened in East Germany in the 70s with the Stasi. Sleep well everybody. https://twitter.com/...
@oliverdarcy Oliver Darcy on x
Facebook is directing users to a variety of resources, including Life After Hate, an advocacy group that helps people leave violent far-right movements. Here's our full story: https://www.cnn.com/...
@mattbraynard Matt Braynard on x
If you *didn't* get this message, you're probably doing something wrong. WORK HARDER! https://twitter.com/...
@riptari @riptari on x
Facebook is warning Facebook users about Facebook? https://twitter.com/...
@realerincruz Cruz For on x
Dear Facebook, I am concerned communists/socialists are out of control & are daily infringing on our lives as a free people. Their extremist behavior is costing millions of lives via abortion funding, neglect of regular care for cancer & other patients during COVID panD. @RepMTG …
@minnie_mag Mini on x
Build a system that constantly manipulates with peoples emotions to drive engagement Encourage inflaming outrage bait because it makes money Radicalize boomers so they'll click on your ads ???? Oops we should probably do something about grandpa now that we turned him into a nazi …
@ross Ross Mayfield on x
Relationship Status: so complicated I'm turning them in. NextDoor must copy this Soviet feature to realize the monetization gains of properly channeled outrage. https://twitter.com/...
@ross Ross Mayfield on x
Anyone who lived in autocracy like the USSR or studied it recognizes this for what it is immediately. Will divide friends, neighbors, even families. And deploying it at global scale to avoid accountability for harming democracy by regulators would do even greater dystopian harm. …
@asranomani Asra Q. Nomani on x
Facebook is following the playbook of Mao's Cultural Revolution. From K-12 to social media, we now have snitch squads being created, turning people against each other and creating the platforms to turn each other in as “extremists.” https://twitter.com/...
@chrisblec Chris Blec on x
Today, @Facebook told thousands of people that they may be extremists. These people had no idea they could be extremists before this. Now, they're thinking “Hmm. Maybe I *am* an extremist! Maybe I should start *acting* like one!” Is there not one sane person left at Facebook? htt…

Chronicles