A new study finds that YouTube's late 2019 algorithm actively discourages users from watching radicalizing content, directing them to more mainstream videos

In 2018, Kevin Roose published a piece in the New York Times in which Caleb Cain, a liberal college dropout, described his experience …

Mark Ledwich 2019-12-29

Context & Ripple Effects

The rabbit-hole thesis has defined YouTube coverage since Kevin Roose's 2018 profile of Caleb Cain warned that the platform could be one of the most powerful radicalizing tools of this century, followed by chief product officer Neal Mohan defending the recommender in an interview with the New York Times. The evidence base since then has split: a [[a:947119|report attributing right-wing content's spread to supply and demand rather than recommendations]], and a study of 72M comments across 330,925 videos finding a real radicalization effect.

Mark Ledwich's new study of the late-2019 algorithm lands on the other side of that split, reporting that the recommender actively discouraged radicalizing content and steered viewers toward mainstream videos — aligning with the later analysis of 300,000 viewers from 2016-2019 that found algorithmic radicalization was not widespread and most users stayed in their ideological corners.

First-order effects

YouTube and Neal Mohan gain direct empirical backing for the position Mohan staked out in his New York Times interview, weakening the claim that the recommender itself drives users to extremes.
The finding directly contradicts the 72M-comment study's radicalization conclusion, putting the two camps' methodologies — user-level viewing data versus comment analysis — in open conflict.

Second-order effects

Researchers and outlets covering the beat are forced to adjudicate between incompatible datasets before citing 'the algorithm' as a causal actor, raising the evidentiary bar for future radicalization claims.
Criticism of YouTube shifts toward the supply side — creators like the former alt-right YouTuber who engineered confrontational content for the echo chamber — since the demand-and-recommendation evidence points away from the algorithm as the driver.

Third-order effects

If the pattern across these studies holds, regulatory and journalistic pressure aimed at recommendation algorithms loses its central empirical premise, and platform-accountability debates reorient toward what content gets produced and why audiences seek it out.
The contested record itself becomes structural: any future policy on recommender systems will have to reckon with a literature that reaches opposite conclusions from different methods, making single-study claims harder to legislate on.

The trend: Successive studies are shifting the algorithmic-radicalization debate on YouTube from the recommender as cause toward audience demand and creator supply as the drivers of extreme-content consumption.

Discussion

@random_walker Arvind Narayanan on x
A new paper has been making the rounds with the intriguing claim that YouTube has a *de-radicalizing* influence. https://arxiv.org/... Having read the paper, I wanted to call it wrong, but that would give the paper too much credit, because it is not even wrong. Let me explain.
@random_walker Arvind Narayanan on x
After tussling with these complexities, my students and I ended up with nothing publishable because we realized that there's no good way for external researchers to quantitatively study radicalization. I think YouTube can study it internally, but only in a very limited way.
@kevinroose Kevin Roose on x
This could have been interesting empirical research, but a conclusion like this kind of gives the game away. Oh well! https://twitter.com/...
@austen Austen Allred on x
Wow. Very, very well done analysis of the YouTube algorithm that shows, contrary to NYT claims, YouTube actually *reduces* radicalization. Two questions: 1. If this is true, what does it say about YouTube? 2. If this is true, what does it say about the New York Times? https://twi…
@mark_ledwich Mark Ledwich on x
1. I worked with Anna Zaitsev (Berkely postdoc) to study YouTube recommendation radicalization. We painstakingly collected and grouped channels (768) and recommendations (23M) and found that the algo has a deradicalizing influence. Pre-print: https://arxiv.org/... 🧵
@mark_ledwich Mark Ledwich on x
2. It turns out the late 2019 algorithm *DESTROYS* conspiracy theorists, provocateurs and white identitarians *Helps* partisans *Hurts* almost everyone else. 👇 compares an estimate of the recommendations presented (grey) to received (green) for each of the groups: pic.twitter.com…
@bernstein Joe Bernstein on x
More broadly, I'm concerned there's a way reporting on tech companies cedes the validity of their dismal vision of humanity — input output machines — before the argument has even started
@bernstein Joe Bernstein on x
For example, think about how silly it would be to focus exclusively on YouTube as the vector for radicalizing ISIS members without thinking about recent history, class, temperament, etc
@bernstein Joe Bernstein on x
That's not to say it isn't important to understand how people use YouTube and how YouTube makes it so — it's desperately important for all kinds of reasons. But the narrow focus on radicalization will, to me, always be overly mechanistic and incomplete
@bernstein Joe Bernstein on x
These discussions/disagreements about radicalization and the algorithm are really important, but as always, they seem to me to presume some kind of set understanding or control group state of the human mind and human context, which are irreducibly complex
@cwarzel Charlie Warzel on x
the last story i worked on at @BuzzFeedNews last year tried to suss this out but even in automating user journeys the only thing is clear is that YouTube's recommendation algorithm isn't a partisan monster — it's an engagement monster https://www.buzzfeednews.com/ ... https://twi…
@cwarzel Charlie Warzel on x
this is the key point about studying proprietary algorithms of big platforms w/o inside access. anyone who really cares about this work will tell you that opaque platform design and super personalized algo decisions at massive scale are what holds back our understanding https://t…
@ltf_01 Littlefoot on x
I agree this is a large limitation, albeit standard for the literature on this phenomenon. What's interesting to me is that despite this study and others, many (not necessarily Arvind) are nonetheless willing to accept the NYT's core claim made without ANY data on face value. htt…
@paullewis Paul Lewis on x
This is important for people who care about YouTube radicalisation and may have read the flawed research (which Google will have loved) downplaying the problem. https://twitter.com/...
@mims Christopher Mims on x
Pretty stunning indictment of that “YouTube is fine really” paper and at least one of its authors https://twitter.com/...
@mark_ledwich Mark Ledwich on x
5/ As for b) here is monthly data since Dec 2019. It's not totally clean - there were channels added and some minor changes in processes throughout that period - it slightly noisy but representative of what happened over that period. pic.twitter.com/iJz3IOXXfS
@mark_ledwich Mark Ledwich on x
8/ Recommendation-rabbit-hole proponents have been ignoring evidence and searching for compelling anecdotes since 2018. I wish you all the best in dealing with your dissonance. I hope it's not too painful. END
@mark_ledwich Mark Ledwich on x
2/ Anonymous recs (even when averaged out over all users) could have a different influence compared to personalized ones. This is a legit limitation, but one that applies to all of the studies on recs so far.
@mark_ledwich Mark Ledwich on x
4/ I plan on begging for stats from youtube creators to export data ... from their “YouTube analytics traffic sources -> suggested video” reports to comparing it with our data. In the meantime, you should accept this as the best quality data you have to go on. pic.twitter.com/vsM…
@mark_ledwich Mark Ledwich on x
3/ There are practical reasons that make this extremely difficult - you would need a chrome extension (or equivalent) that captures real recommendations in click-through stats from a representative set of users. I don't plan on doing that.
@random_walker Arvind Narayanan on x
Radicalization via YouTube, as widely understood, is when someone watches a few partisan videos and unwittingly starts a feedback loop in which the algorithm gradually recommends more and more extreme content and the viewer starts to believe more and more of it.
@random_walker Arvind Narayanan on x
Incidentally, I spent about a year studying YouTube radicalization with several students. We dismissed simplistic research designs (like the one in the paper) by about week 2, and realized that the phenomenon results from users/the algorithm/video creators adapting to each other.
@random_walker Arvind Narayanan on x
Others have pointed out many more limitations of the paper, including the fact that it claims to refute years of allegations of radicalization using late-2019 measurements. Sure, but that's a bit like pointing out typos in the article that announced “Dewey Defeats Truman”.
@random_walker Arvind Narayanan on x
Let's not forget: the peddlers of extreme content adversarially navigate YouTube's algorithm, optimizing the clickbaitiness of their video thumbnails and titles, while reputable sources attempt to maintain some semblance of impartiality. (None of this is modeled in the paper.)
@random_walker Arvind Narayanan on x
The key is that the user's beliefs, preferences, and behavior shift over time, and the algorithm both learns and encourages this, nudging the user gradually. But this study didn't analyze real users. So the crucial question becomes: what model of user behavior did they use?
@random_walker Arvind Narayanan on x
The answer: they didn't! They reached their sweeping conclusions by analyzing YouTube *without logging in*, based on sidebar recommendations for a sample of channels (not even the user's home page because, again, there's no user). Whatever they measured, it's not radicalization.
@random_walker Arvind Narayanan on x
If you're wondering how such a widely discussed problem has attracted so little scientific study before this paper, that's exactly why. Many have tried, but chose to say nothing rather than publish meaningless results, leaving the field open for authors with lower standards.
@bostonjoan Joan Donovan, PhD on x
A crucial perspective on the recent non-peer-reviewed paper about “youtube radicalization.” Quantifying radicalization isn't just hard, it's impossible. Sociologists, like @JessieNYC, have been saying this for over a decade. Studying socio-technical systems is messy. https://twit…
@mrbrianhughes Brian Hughes on x
There's still tons of work to be done analyzing the role of algorithmic recommendation in radicalization. No one can legitimately claim an authoritative answer bc, frankly, no one has access (yet) to the granular, longitudinal, user-specific data that we need! https://twitter.com…
@hkanji Hussein Kanji on x
The idea that users are manipulated by an impersonal algorithm into a world of conspiracy theorists, provocateurs and racists is a story that the mainstream media has been eager to promote https://medium.com/... via @nuzzel
@beccalew Becca Lewis on x
Fantastic thread on why quantitative methods are often ill-suited to studying radicalization on YouTube via the algorithm. https://twitter.com/...
@kevinroose Kevin Roose on x
The premise of this article/study is so odd. Studying the YouTube algo of late 2019 (after YouTube made some very well-publicized algo changes to reduce recommendations of extreme content) doesn't say anything about what YouTube recommendations were like before then. https://twit…
@kevinroose Kevin Roose on x
If anything this study (whose methodology is super iffy, given that it analyzes logged-out recommendations and doesn't account for personalization) shows that YouTube's algo changes to reduce extreme content recs are working. Which, great!
@persily Nathaniel Persily on x
Very important study here of absence of radicalization through YouTube recommendation algorithm. Scholars need comprehensive access to YouTube data to confirm these findings. (This study demonstrates why YouTube should not be afraid to share its data with researchers!) https://tw…
@msmelchen Melissa Chen on x
The kids are alright. Not alt-right. https://twitter.com/...
@jeffreyasachs Jeffrey Sachs on x
An interesting critique of the “Actually, YouTube doesn't radicalize people” article currently making the rounds, though ignore the bit about the co-author's intentions. Attn. @Noahpinion https://twitter.com/...
@willfoia Will Feuer on x
Examining millions of YouTube recommendations over the course of a year, two researchers have determined that the platform in fact combats political radicalization. https://www.cnbc.com/...
@antumbral Katelyn Gadd on x
so “the late 2019 version of the algorithm doesn't do the thing” means that past criticisms were invalid and a “crusade” even though youtube has publicly announced changes/fixes to the algorithm multiple times in the last couple years?
@wordsandsuch Vikram Singh on x
Your claims far exceed your evidence. You don't account for the basic premise of the ‘radicalisation’ thesis: that algorithms fine tune reccs based on user data - i.e previously watched videos, search terms, etc. You only use an ‘anonymous’ user's first video recc.
@mark_ledwich Mark Ledwich on x
3. Check out https://www.recfluence.net/ to have a play with this new dataset. We also include categorization from @manoelribeiro et a.l. and other studies so you can see some alternative groupings. All of the code and data is free to review to use https://github.com/... pic.twi…
@mark_ledwich Mark Ledwich on x
4. My new article explains in detail. It takes aim at the NYT (in particular, @kevinroose) who have been on myth-filled crusade vs social media. We should start questioning the authoritative status of outlets that have soiled themselves with agendas. https://medium.com/...
@eladgil Elad Gil on x
YouTube pushes politics mainstream versus radicalizes? https://arxiv.org/... https://twitter.com/...
@can Can Duruk on x
One explanation: YouTube is better thanks to the NYT (and others') investigations of its previous algorithms. https://twitter.com/...
@andre_spicer Andr Spicer on x
Does YouTube's algo direct people down extreme right wing rabbit holes? Study of 800 channels finds it discourages viewers from visiting radicalizing or extremist content. Algo favors mainstream media over independent YouTube channels. Slants left/neutral https://arxiv.org/...
@mattgrossmann Matt Grossmann on x
YouTube's recommendation algorithm actively discourages viewers from visiting radicalizing or extremist content; it favors mainstream media & cable news content over independent channels with a slant towards left-leaning or politically neutral channels https://arxiv.org/...

Chronicles