Sources: Facebook ceased research into racial bias in Instagram's moderation tools after finding problematic results; Facebook says the methodology was flawed
In mid-2019, researchers at Facebook began studying a new set of rules proposed for the automated system that Instagram uses …
NBC NewsOlivia Solon
Context & Ripple Effects
Two days before this report, Facebook announced new teams dedicated to studying algorithmic racial bias on its core platform and Instagram — which makes the sourcing here pointed: researchers had already run a bias study on Instagram's proposed moderation rules in mid-2019, found problematic results, and the work was shut down, with the company attributing the outcome to flawed methodology.
Facebook's newly announced bias-research teams start under a credibility cloud: sources say prior internal research into Instagram's moderation tools was halted after problematic results, so researchers and civil-rights observers will judge whether the new teams can publish findings the company dislikes.
Instagram's moderation rule changes proceed without the internal bias evidence base the 2019 study was meant to provide, leaving enforcement decisions on the platform running on the contested methodology dispute.
Second-order effects
Facebook's 'flawed methodology' defense becomes the template every future internal finding will be measured against — the same company that conceded enforcement mistakes to ProPublica in 2018 is now disputing its own researchers' results, sharpening the case for independent audits rather than self-study.
Advertisers and civil-rights groups gain fresh leverage from the sequence: the multicultural affinity retreat shows Facebook changes product behavior under sustained pressure, so the halted study is likely to be cited in the next round of that pressure.
Third-order effects
If the pattern holds — internal research surfaces bias, the research stops, the company disputes the method — the structural endpoint is mandatory external algorithmic audits for moderation systems, with regulators treating self-reported methodology disputes as insufficient.
The broader shift is that platform content moderation becomes a measurable civil-rights surface: enforcement disparities documented from ProPublica's post testing through the race-blind policy findings move bias from a PR question to a compliance question that shapes how automated rules get built and reviewed.
The trend: Platform algorithmic-bias research is colliding with legal and reputational exposure, pushing moderation accountability from voluntary internal study toward independent audit.
NEW: For several weeks I've been talking to FB employees and ex-employees who say that efforts to study and address racial bias on the platform have for years been ignored, slow-walked and, in some cases, suppressed https://www.nbcnews.com/...
@aprilaser
April Glaser on x
“Users on Instagram in the US whose activity .. suggested they were Black were about 50 percent more likely .. to have their accounts automatically disabled by the moderation system than those whose .. were white.” https://www.nbcnews.com/... big story from @oliviasolon
@ekp
Ellen K. Pao on x
Facebook's own research showed its moderation discriminates against Black users; in response, leaders decided to quash the results and future bias bresearch https://www.nbcnews.com/... https://twitter.com/...
@fightfortheftr
@fightfortheftr on x
Thank you for this crucial reporting. We spend a lot of time focused on the harm caused by the content that facebook “allows” and amplifies. We need to be equally concerned about the voices they silence: disproportionately the voices of Black and brown people. https://twitter.com…
@rmac18
@rmac18 on x
.@oliviasolon's piece today on issues of racial bias at Facebook also happens to capture the theme of disillusionment among some Facebook employees. This quote from an internal post is pretty damning. https://www.nbcnews.com/... https://twitter.com/...
@schemaly
Soraya Chemaly on x
Facebook deliberately ignored bias research, employees say. This well describes so much shoddy “reasoning” and the many false equivalences used to erase power. Have personally had conversations like the ones here. It's been years and feels endless. https://www.nbcnews.com/...
@blackamazon
@blackamazon on x
I asked @mosseri this DIRECTLY https://twitter.com/...
@emeralddeleeuw
Emerald de Leeuw on x
And employees pointing out the serious harms caused by their biased technology were told to stop researching it. Is there any end to the unethical disgusting behaviour within Facebook companies? https://twitter.com/...
@jason_kint
Jason Kint on x
🤦🏽♀ ️ This is so telling. “In an effort to be neutral, the company's hate speech policies treat attacks on white people or men in exactly the same way as it treats comments about Black people or women.” https://twitter.com/...
@ncweaver
Nicholas Weaver on x
Nick's Cynical Law of AI: Machine learning is a great way to train a computer to be a racist asshole, and some people seem to like it that way. https://twitter.com/...
@gregotto
Greg Otto on x
Facebook management has repeatedly ignored and suppressed internal research showing racial bias in the way that the platform removes content, according to eight current and former employees https://www.nbcnews.com/... via @nbcnews
@oliviasolon
Olivia Solon on x
Some researchers who found signs of racial bias in the automated content review systems at Facebook and Instagram presented their work to the management team & they were told to immediately stop all research into race and not to share it with co-workers https://www.nbcnews.com/..…
@zackford
Zack Ford on x
Silencing findings of racial bias is never the right strategy. https://twitter.com/...
@chetfaliszek
Chet Faliszek on x
Facebook is a great microcosm of the BS addressing of systemic racism in large institutions and governments. Change will only happen if they are held accountable. https://twitter.com/...
@pegobry
Peg on x
I just want to know the dialogue about how they picked that Sandberg photo. “Can we get one where she frowns harder? ... How about this one, where the shadows make it look like her eyes are fully black? ... Excellent.” https://twitter.com/...
@jeremybmerrill
Jeremy B. Merrill on x
I feel like there's a through line between this and the mixup over CrowdTangle data about inflammatory political news being popular: Facebook resists imperfect research, but doesn't enable better research (or if it's happening internally, doesn't publish the results). https://twi…
@oliviasolon
Olivia Solon on x
In 2016 Zuck had to tell employees why ‘All Lives Matter’ was offensive. In 2020, the company's content moderation policies fail to take this into account. https://www.nbcnews.com/...
@profcarroll
@profcarroll on x
Another brutal blockbuster for Fb rn. Ooooof. https://twitter.com/...