In four new studies with collaboration from Meta, researchers find Facebook's algorithm is “influential” but doesn't necessarily change beliefs, and more

In four new studies, researchers found complicated results from experiments on Facebook's and Instagram's algorithms …

New York Times 2023-07-28

Context & Ripple Effects

The findings add nuance to a long-running dispute over Facebook’s role in polarization. Earlier coverage linked greater time on Facebook with more polarized news consumption, while an earlier critique challenged how the platform’s polarization research classified users’ ideology.

They also arrive as Meta has made its recommendation systems more legible through system cards describing AI recommendations across Facebook and Instagram. The studies distinguish algorithmic influence over what people encounter from a demonstrated change in beliefs.

First-order effects

Meta gains research evidence that its Facebook and Instagram ranking systems affect users’ information environments without establishing that they necessarily alter beliefs.
The results complicate simple claims that feed ranking alone explains political or attitudinal change, while keeping algorithmic influence central to scrutiny of the platforms.

Second-order effects

Researchers and policymakers assessing platform harms will need to separate exposure, engagement, and belief change rather than treating them as interchangeable outcomes.
The work raises the value of access to platform-level data and experiments: prior evidence associated Facebook use with more polarized news consumption, but association and causal belief effects are different questions.

Third-order effects

If this research approach is sustained, debate over recommender systems may shift from broad claims about persuasion toward narrower, measurable pathways by which ranking affects users.
The record also underscores that transparency alone will not settle methodological disputes, given earlier criticism of Facebook’s ideological-classification approach in polarization research.

The trend: Platform-algorithm accountability is moving toward more granular evidence about which user outcomes recommendation systems actually cause.

Discussion

@daveya Davey Alba on threads
Important follow-up: Researchers disagree with Meta's framing of the papers published in Nature and Science today ‼️...
@brendannyhan@mastodon.social Brendan Nyhan on mastodon
New at Nature: Like-minded sources on Facebook are prevalent but not polarizing — https://www.nature.com/... Our key findings: —Median Facebook user gets 50.4% of content from like-minded sources —But reducing exposure by 1/3 for 3 months had no measurable effect on attitude…
@jeffhorwitz Jeff Horwitz on x
Here's the editor of Science. There are some results that should be favorably viewed by Meta. But the company's researcher partners, the work's rapporteur and Science all called Meta's claims about the takeaways overblown, illustrating the need for unmediated access to data.
@mikeisaac @mikeisaac on x
i also think they are not *fully* exculpatory of Meta if you read all the papers, but rather tell us how little we know about how social media affects people and that it will take years of study and to unwind it. Scientists fromt the study hope further research will be possible
@sciencemagazine @sciencemagazine on x
A new special issue of Science investigates social media's impact on elections through a pioneering collaboration between independent academics and Meta researchers. Learn more: https://www.science.org/... #SocialMediaAndElections [image]
@dggoldst Dan Goldstein on x
You will see a bunch of Meta / Facebook papers drop today and soon after Hats off to the academic researchers, but also to the internal Meta researchers The whole idea of doing collaborations with academics was theirs They invested years of tireless effort to get to today
@andreyfradkin @andreyfradkin on x
This is an unprecedented collaboration that shows the quality of social science that can be done if a platform fully cooperates. These papers will go straight to syllabi and will shape our conversation about the role of social media going forward. We need more work like this.
@viacristiano Cristiano Lima on x
Facebook and other social media companies will be citing these studies in every hearing and in response to every congressional letter anytime lawmakers claim platforms are driving political polarization https://www.washingtonpost.com/ ...
@hholdenthorp @hholdenthorp on x
Today, @ScienceMagazine is publishing three papers, a news story, a policy forum, and the editorial below on studies done by academic researchers on data from Facebook addressing the effect of the platform on political attitudes. 1/n https://www.science.org/...
@jeffhorwitz Jeff Horwitz on x
It's worth noting that this was the one observational study. Regarding the experimental papers: internal Meta research has always struggled to show tie short-term tests with “offline” behavior. But few researchers I've spoken to considered those null findings dispositive.
@p_barbera Pablo Barberá on x
Many of these had never been used in academic-industry social science collaborations before, and I hope they can become the standard. More details on all these steps are available in the papers.
@p_barbera Pablo Barberá on x
An important component of this project was all the steps we took to ensure its scientific integrity: pre-registration through OSF, IRB approval, code review of data pipelines and analysis code, the use of a “clean room” for data analysis, etc.
@telliotter Tori Elliott on x
But notably, researchers also found that content that had been flagged by Meta's own fact checkers as misinformation tended to circulate almost exclusively in conservative corners of the company's platforms. [image]
@brianboland Brian Boland on x
@MikeIsaac @sheeraf Looking forward to reading these studies - proves why we need PATA. I worry that this only looked at political polarization and not racial, ethnic, religious etc. Probably can't pull out p99 hate filled feeds etc. It isn't about average impact.
@katieharbath Katie Harbath on x
5. The left and right have very different online experiences - This makes it challenging for platforms when there is an external expectation of being “fair” across the parties regarding how much content is actioned against because how and what they consume is very different.
@andyguess Andy Guess on x
It's interesting to note the disconnect between the relatively powerful effects we document in terms of platform experiences and behavior, and the downstream outcomes focused on knowledge, attitudes, and off-platform behavior. 15/
@shiringhaffary Shirin Ghaffary on x
a lot to dig into in this new set of research re: Facebook/Insta and 2020 elections. definitely challenges the common assumption that social media algorithms inherently drive polarization — but also room for much more research on this
@andyguess Andy Guess on x
Even with the existing algorithm, the baseline proportions are quite low: In the default feed, the average share of users' feeds in our sample containing posts from untrustworthy sources was 2.6% on FB; for uncivil posts, 3.2%; for slur words: .03%. And even less on IG. 11/
@katieharbath Katie Harbath on x
2. Results show the real trade-offs companies face - Making algorithmic changes to reduce certain types of content is not simple. They can reduce some good things and amplify bad ones and vice versa. So choices have to be made.
@katieharbath Katie Harbath on x
7. There is work yet to be done - No company - especially Facebook - should take this research as an excuse not to do this work. I'm breathing a small sigh of relief, but the threats are still there; they are ever-changing, and social media still plays a role.
@nickclegg Nick Clegg on x
Today, academic journals Science and Nature have published four landmark research papers to better understand the impact of Facebook and Instagram on key political attitudes and behaviors during the US 2020 election cycle. https://about.fb.com/...
@nickclegg Nick Clegg on x
The papers are the first of what we expect to be 16 produced as the result of an unprecedented research project, in partnership with external researchers. They include studies of the effects of algorithmic ranking & virality, the prevalence & effects of like-minded information...
@aishacounts Aisha Counts on x
Meta worked with a group of 17 independent researchers to understand Facebook's impact on elections w/ @daveyalba https://www.bloomberg.com/...
@andyguess Andy Guess on x
TL;DR: We pre-registered, outside researchers had final say, and Meta committed not to block the publication of findings they didn't like (they kept their promise). Both Meta and the outside academic research team took user privacy very seriously. 5/
@aishacounts Aisha Counts on x
Researchers found that conservatives engaged more with false news stories on Facebook than liberals during the US 2020 presidential election
@andyguess Andy Guess on x
What did we find? 1. ENGAGEMENT: The chrono feed decreased the time users spent on the platforms (by about 20% on FB and 11% on IG). Engagement on the platforms also went down. 8/ [image]
@aishacounts Aisha Counts on x
And that on the whole, audiences on Facebook are highly politically segregated. That political segregation was even more apparent in Facebook Pages and Groups
@andyguess Andy Guess on x
Today is publication day for the first 4 papers resulting from a unique collaboration between Meta researchers and outside academics to study the political effects of Facebook and Instagram in the 2020 U.S. election! 🧵 1/N
@katieharbath Katie Harbath on x
Here are my seven takeaways: 1. This research is unprecedented - it's important to remember how unique it is for a platform to open up to researchers in this way voluntarily. This took forever to release because both Meta and the researchers wanted to do this the right way.
@daveyalba Davey Alba on x
But none of the studies concluded that there are no negative impacts from social media on the political process. Also, no studies concluded that there are no viable solutions to help potential negative impacts from social media. Looking forward to the rest as they're published!
@daveyalba Davey Alba on x
Lots of outlets are carrying the news of these Facebook studies today. Here's my and @aishacounts' take. We focus on one study that we thought was most revealing: Facebook Misinformation in US Election Reached More Conservatives, Study Says https://www.bloomberg.com/... [image]
@jenjpan Jennifer Pan on x
Contrary to common belief, chronological feed did not significantly alter polarization, politics knowledge, or other survey-based outcomes, even though chronological feed led users to spend much less time on FB and Insta + changed what content they saw. [image]
@jenjpan Jennifer Pan on x
In the 2nd paper, we removed reshared content from Facebook feeds. This decreased political news participants saw in feeds, reduced their clicks on partisan news sources & reduced news knowledge but didn't affect polarization or other attitudes. https://www.science.org/... [image…
@jenjpan Jennifer Pan on x
Like-minded sources on Facebook are prevalent but not polarizing https://www.nature.com/... by @BrendanNyhan Jamie Settle @emilythorson @mwojcieszak
@jenjpan Jennifer Pan on x
Today, 2 papers I coauthored w/ @andyguess @namalhotra @p_barbera are out in Science. These papers, about the role of social media in American democracy, are based on pre-registered experiments with consenting participants conducted w/ @Meta #SocialMediaAndElections
@blackamazon @blackamazon on x
LOL I feel nothing anymore. It's shoddy research, biased , full of questions that needed to be addressed before we got here And I'm not at all paid to fix this shit
@mattlindner Matt Lindner on x
Calling bullshit on this one. https://www.washingtonpost.com/ ... [image]
@mikeisaac @mikeisaac on x
@greglinden @sheeraf the limited nature of the tweaks is what we tried to convey in this piece. headlines could give meta an easy win but the findings are very nuanced
@greglinden Greg Linden on x
@MikeIsaac @sheeraf Yes, small changes to the ranking or features — such as removing the easy reshare button — have small effects that are difficult to detect in experiments. That conclusion is much more limited than saying that algorithms cannot impact people's views.
@mikeisaac @mikeisaac on x
(it is also important to note that news stories marked as untrustworthy by the FB 3rd party fact checkers were in the minority of the overall news stories that circulate on Facebook) These studies are huge and are the most direct access to facebook data yet. And more to come
@mikeisaac @mikeisaac on x
IMPORTANT: consumption of political news on FB and IG was * highly segregated* by ideology —97 percent of the people who read links to “untrustworthy” news stories on the apps during the 2020 election identified as conservative and engaged w/ mostly right-wing content
@mikeisaac @mikeisaac on x
the first four of 16 papers were published after a yearslong collaboration between outside academics and Meta, which gave the researchers access to data of more than 200 million anonymized US FB and IG users the findings are complicated & often contradictory to popular knowledge
@mikeisaac @mikeisaac on x
NEWS: Research from scientists who studied Facebook and Instagram accounts around the 2020 election found that changes to key features in the company's algorithms showed no “detectable changes to peoples polarization or beliefs” w/ @sheeraf https://www.nytimes.com/...

Chronicles