An ex-ByteDance employee recounts the inner workings of the company's complex content moderation system and the role of China's internet censorship policies
This is the story of Li An, a pseudonymous former employee at ByteDance, as told to Protocol's Shen Lu.
Later coverage widened the issue from moderation mechanics to product governance: former employees alleged pro-China promotion in the TopBuzz news app, while staff interviews described Beijing tightening control over TikTok.
First-order effects
ByteDance faces a more detailed public account tying its moderation operations to China’s censorship framework, raising the stakes for how it explains governance of products used outside China.
Li An’s testimony gives scrutiny of ByteDance’s content decisions a named insider narrative rather than an allegation framed solely around individual posts or outcomes.
Second-order effects
TikTok’s separation from ByteDance’s China-based operations becomes a more consequential point of comparison as the earlier code-access boundary and later reports of tighter Beijing control are read together.
Platforms and contractors serving Chinese internet companies face added pressure to show whether moderation is insulated from domestic censorship requirements, echoing the labor-intensive model described in Beijing’s “censorship factories”.
Third-order effects
The ByteDance coverage points to content moderation becoming a governance and market-access issue, not just a trust-and-safety function, when companies operate across China and overseas markets.
If accounts of centralized control continue to accumulate, the durable competitive question for globally distributed platforms will be whether organizational boundaries can be independently demonstrated rather than merely asserted.
The trend: Global platforms with China-based ownership are being judged increasingly on whether their moderation and operational controls can be shown to be separate from domestic censorship systems.
Read my latest: an ex-ByteDancer walks me through the inner workings of the company's complex content moderation operation and execution in China. They also shared how they felt about developing technologies that censor speech. 1/ https://www.protocol.com/...
Read what it was like to work at ByteDance and build powerful tools for its tens of thousands of censors: breathtaking stuff, as told to the ever-intrepid @shenlulushen for @protocol | China's latest scoop: https://www.protocol.com/...
This is the story of a former content moderation employee at ByteDance, who told @shenlulushen that ByteDance received requests to “develop an algorithm that could automatically detect when a Douyin user spoke Uyghur - and then cut off the livestream.” https://www.protocol.com/..…
This is what violation of freedom of speech is about. It comes from a government and it is on all the country's platforms, incl websites. What Facebook and Twitter do is not, as they are private initiatives and there are loads of alternatives https://www.protocol.com/...
How do people feel about building the system?"The tools I helped create can help fight dangers like fake news.But in China, one primary function of these technologies is to censor speech and erase collective memories of major events, however infrequently this function gets used."
Dr. Li [Wenliang] ... was punished for that. ... we had no idea what's really happening because of authorities' cover-up of the severity of the crisis. Around this time last year, many Chinese tech companies were actively deleting posts, videos, 1/2 https://www.protocol.com/...
“I felt guilt more than anger. I helped develop tools and platforms for content moderation. In other words, I had helped build the system that censored accounts like mine. I was helping to bury myself in China's ever-expanding cyber grave.” https://www.protocol.com/...
The details in this @shenlulushen joint are just nuts: “Not many tech companies in China have so many resources dedicated to moderating content. Other user-generated content platforms in China have nothing on ByteDance.” https://www.protocol.com/...
“When I was at ByteDance, we received multiple requests from the bases to develop an algorithm that could automatically detect when a Douyin user spoke Uyghur, and then cut off the livestream session” https://www.protocol.com/...?
“In other words, I had helped build the system that censored accounts like mine. I was helping to bury myself in China's ever-expanding cyber grave.” https://www.protocol.com/... via @hare_brain
“When I was at ByteDance, we received multiple requests from the bases to develop an algorithm that could automatically detect when a Douyin user spoke Uyghur, and then cut off the livestream session.” https://www.protocol.com/...
Not many Chinese tech companies have so many resources dedicated to moderating content. This is why other user-generated content platforms in China have nothing on ByteDance's popular products. 3/
Interview with a former ByteDance employee: “I couldn't help but feel every day like I was a tiny cog in a vast, evil machine.” @shenlulushen https://www.protocol.com/...
Political speech comprised a tiny fraction of deleted content b/c Chinese people know what not to talk & hello, it's Douyin! Most censored content: porn, nudity, graphic images, curse words, unauthorized live streaming sales and content that violated copyrights. 4/
ByteDance's content moderation policymakers + the team that creates core tech to support its content moderation operation + about 20,000 content moderators, have helped shield ByteDance from major political repercussions and achieve commercial success. 2/
After their own social media account got “bombed” last year, they realized, “I had helped build the system that censored accounts like mine. I was helping to bury myself in China's ever-expanding cyber grave.” /end
Software engineering is political. The choices that you make (or choose not to make) affect people's lives. This is as true in the US and the EU as it is in China. https://www.protocol.com/...
A must-read piece by always-brilliant @shenlulushen that offers important insights into how popular Chinese social media apps build their content moderation mechanisms, the struggles that Chinese employees in those positions have to deal with and more. https://www.protocol.com/..…
“Streamers speaking ethnic languages & dialects that Mandarin-speakers don't understand would receive a warning to switch to Mandarin. If they didn't comply, moderators would respond by manually cutting off the livestreams, regardless of the [...] content.” https://www.protocol.c…
Excellent read from our @protocol |China team's @shenlulushen on the personal struggles of a former ByteDance / TikTok content moderator/ censor. https://www.protocol.com/...
“I knew ByteDance's army of content moderators were using the tools and algorithms that I helped develop to delete content, change the narrative and alter memories of the suffering and trauma inflicted on Chinese people during the COVID-19 outbreak.” https://www.protocol.com/...
But political speech still looms large. ByteDance and other UGC platforms can't afford to fail to delete politically sensitive content. Internet regulators would issue frequent directives to its Content Quality Center, which oversees ByteDance's content moderation operation. 5/
Anything users upload is closely monitored and screened by both machines and humans. 👇This flowchart from a report ByteDance published in 2019 illustrates how live-stream moderation works. & My source also explained the process to me in my story: https://www.protocol.com/... 6/ h…