An ex-ByteDance employee recounts the inner workings of the company's complex content moderation system and the role of China's internet censorship policies
because the human rights of *all* users matter. For more: https://www.hrw.org/... https://twitter.com/... Emily Birnbaum / @birnbaum_e : This is the story of a former content moderation employee at ByteDance, who told @shenlulushen that ByteDance received requests to “develop an algorithm that could automatically detect when a Douyin user spoke Uyghur - and then cut off the livestream.” https://www.protocol.com/... Audrey Conklin / @audpants : “I knew ByteDance's army of content moderators were using the tools and algorithms that I helped develop to delete content, change the narrative and alter memories of the suffering and trauma inflicted on Chinese people during the COVID-19 outbreak.” https://www.protocol.com/... Jules Polonetsky / @julespolonetsky : “When I was at ByteDance, we received multiple requests from the bases to develop an algorithm that could automatically detect when a Douyin user spoke Uyghur, and then cut off the livestream session” https://www.protocol.com/...? Shen Lu / @shenlulushen : Not many Chinese tech companies have so many resources dedicated to moderating content. This is why other user-generated content platforms in China have nothing on ByteDance's popular products. 3/ Shen Lu / @shenlulushen : But political speech still looms large. ByteDance and other UGC platforms can't afford to fail to delete politically sensitive content. Internet regulators would issue frequent directives to its Content Quality Center, which oversees ByteDance's content moderation operation. 5/ Shen Lu / @shenlulushen : How do people feel about building the system?"The tools I helped create can help fight dangers like fake news.But in China, one primary function of these technologies is to censor speech and erase collective memories of major events, however infrequently this function gets used." Shen Lu / @shenlulushen : Political speech comprised a tiny fraction of deleted content b/c Chinese people know what not to talk & hello, it's Douyin! Most censored content: porn, nudity, graphic images, curse words, unauthorized live streaming sales and content that violated copyrights. 4/ Benjamin Pimentel / @benpimentel : Excellent read from our @protocol |China team's @shenlulushen on the personal struggles of a former ByteDance / TikTok content moderator/ censor. https://www.protocol.com/... Shen Lu / @shenlulushen : ByteDance's content moderation policymakers + the team that creates core tech to support its content moderation operation + about 20,000 content moderators, have helped shield ByteDance from major political repercussions and achieve commercial success. 2/ Shen Lu / @shenlulushen : After their own social media account got “bombed” last year, they realized, “I had helped build the system that censored accounts like mine. I was helping to bury myself in China's ever-expanding cyber grave.” /end @protocol : I was a cog in ByteDance's vast censorship machine, as told to Protocol's @shenlulushen. https://www.protocol.com/... Joseph / @joepwilliams31 : The details in this @shenlulushen joint are just nuts: “Not many tech companies in China have so many resources dedicated to moderating content. Other user-generated content platforms in China have nothing on ByteDance.” https://www.protocol.com/... Eva / @evacide : Software engineering is political. The choices that you make (or choose not to make) affect people's lives. This is as true in the US and the EU as it is in China. https://www.protocol.com/... Pernille Tranberg / @pernillet : This is what violation of freedom of speech is about. It comes from a government and it is on all the country's platforms, incl websites. What Facebook and Twitter do is not, as they are private initiatives and there are loads of alternatives https://www.protocol.com/... @protocol : “I felt guilt more than anger. I helped develop tools and platforms for content moderation. In other words, I had helped build the system that censored accounts like mine. I was helping to bury myself in China's ever-expanding cyber grave.” https://www.protocol.com/... Scott Nover / @scottnover : “When I was at ByteDance, we received multiple requests from the bases to develop an algorithm that could automatically detect when a Douyin user spoke Uyghur, and then cut off the livestream session.” https://www.protocol.com/... Allen Tan / @tealtan : “Streamers speaking ethnic languages & dialects that Mandarin-speakers don't understand would receive a warning to switch to Mandarin. If they didn't comply, moderators would respond by manually cutting off the livestreams, regardless of the [...] content.” https://www.protocol.com/... Shen Lu / @shenlulushen : Anything users upload is closely monitored and screened by both machines and humans. 👇This flowchart from a report ByteDance published in 2019 illustrates how live-stream moderation works. & My source also explained the process to me in my story: https://www.protocol.com/... 6/ https://twitter.com/... Byron Wan / @byron_wan : Dr. Li [Wenliang] ... was punished for that. ... we had no idea what's really happening because of authorities' cover-up of the severity of the crisis. Around this time last year, many Chinese tech companies were actively deleting posts, videos, 1/2 https://www.protocol.com/... Hakan / @hatr : “In other words, I had helped build the system that censored accounts like mine. I was helping to bury myself in China's ever-expanding cyber grave.” https://www.protocol.com/... via @hare_brain Jonathan Cheng / @jchengwsj : Interview with a former ByteDance employee: “I couldn't help but feel every day like I was a tiny cog in a vast, evil machine.” @shenlulushen https://www.protocol.com/... William Yang / @williamyang120 : A must-read piece by always-brilliant @shenlulushen that offers important insights into how popular Chinese social media apps build their content moderation mechanisms, the struggles that Chinese employees in those positions have to deal with and more. https://www.protocol.com/...
ProtocolShen Lu
Context & Ripple Effects
This account lands in a documented arc: a year earlier, the New York Times had described Beyondsoft's "censorship factories", showing outsourced human moderation at scale was already an industry in China; by mid-2020, ByteDance had restricted its China-based staff from code bases for overseas products like TikTok (the PingWest report), tacitly acknowledging that domestic compliance machinery and international trust were hard to separate.
What the ex-employee adds is the internal mechanics: internet regulators issuing directives to the Content Quality Center, roughly 20,000 moderators, and a machine-plus-human pipeline ByteDance itself published a flowchart for in 2019. The reported request for an algorithm that detects Uyghur speech and cuts off Douyin livestreams moves the story from human moderation to automated enforcement of linguistic identity — which is why it matters beyond one company.
First-order effects
Douyin streamers speaking ethnic languages face immediate enforcement: moderators reportedly warned them to switch to Mandarin and manually cut off non-compliant livestreams, while the requested detection algorithm would automate that cutoff.
ByteDance's own framing holds that this apparatus shielded it from political repercussions and enabled commercial success — meaning the same system is both a regulatory asset at home and, now public, a reputational liability abroad.
Second-order effects
Overseas scrutiny of TikTok sharpens: the account joins earlier allegations like the TopBuzz pro-China promotion claims from former employees, giving US critics a pattern of testimony rather than isolated incidents to cite.
Rival platforms such as Facebook gain a differentiation argument — moderation governance and data-access controls become competitive selling points when a top short-video competitor's compliance system is publicly tied to state directives.
Third-order effects
If the pattern holds, content-moderation infrastructure built to satisfy national censors becomes a structural barrier to global platform operations — pushing companies toward the kind of organizational separation between domestic and international products that ByteDance's code-access restriction foreshadowed.
Automated detection of protected characteristics (language, ethnicity) inside compliance pipelines points toward a regulatory reckoning over what censorship tooling may be built at all, and whether such systems become subject to export-style scrutiny.
The trend: Platform moderation systems engineered for state compliance are becoming geopolitical liabilities as Chinese consumer platforms scale globally, forcing structural separation between domestic and international operations.
Read my latest: an ex-ByteDancer walks me through the inner workings of the company's complex content moderation operation and execution in China. They also shared how they felt about developing technologies that censor speech. 1/ https://www.protocol.com/...
“I had helped build the system that censored accounts like mine. I was helping to bury myself in China's ever-expanding cyber grave.” https://www.protocol.com/...
To me, this article describes horrifying & pervasive censorship; but for far too many Congressmembers, this article will read like an inspirational roadmap #Section230 https://twitter.com/...
This article about content moderation & censorship at TikTok's parent company in China is very eye opening. It's truly like 1984. It's also a grim reminder that this is the only feasible option if we want to deplatform conspiracy theories on social media. https://www.protocol.com…
I'm mostly reading this like a riddle, trying to figure out what the person's job actually was. Sounds like either ML Tools or SWE. https://www.protocol.com/...
Read what it was like to work at ByteDance and build powerful tools for its tens of thousands of censors: breathtaking stuff, as told to the ever-intrepid @shenlulushen for @protocol | China's latest scoop: https://www.protocol.com/...
The most informational piece I've read about how censorship mechanisms work at ByteDance and the thinking behind it. A first-hand account from an ex-ByteDance engineer, told to my colleague @shenlulushen for @Protocol | China Read this thread 👇 https://twitter.com/...
Former employee explains ByteDance's complex content moderation operation in China: real-time analysis plus 20,000 human moderators censor nudity, profanity, live streams, and political speech https://twitter.com/...
“I helped build ByteDance's vast censorship machine. I wasn't proud of it, and neither were my coworkers. But that's life in today's China.” fascinating interview from @shenlulushen https://www.protocol.com/...
How a tech company chooses to aid and abet human rights violations should influence how democracies respond. This is a crucial point often lost in foreign policy convos. But the reasons for concern abt authoritarian tech in the 1st place are precisely these HR problems. https://t…
Great story by @shenlulushen! ByteDance, owner of TikTok, is a major human rights violator. How ByteDance should be treated in the US should take into account the company's activities inside China —because the human rights of *all* users matter. For more: https://www.hrw.org/... …
“I knew ByteDance's army of content moderators were using the tools and algorithms that I helped develop to delete content, change the narrative and alter memories of the suffering and trauma inflicted on Chinese people during the COVID-19 outbreak.” https://www.protocol.com/...
“When I was at ByteDance, we received multiple requests from the bases to develop an algorithm that could automatically detect when a Douyin user spoke Uyghur, and then cut off the livestream session” https://www.protocol.com/...?
Not many Chinese tech companies have so many resources dedicated to moderating content. This is why other user-generated content platforms in China have nothing on ByteDance's popular products. 3/
But political speech still looms large. ByteDance and other UGC platforms can't afford to fail to delete politically sensitive content. Internet regulators would issue frequent directives to its Content Quality Center, which oversees ByteDance's content moderation operation. 5/
How do people feel about building the system?"The tools I helped create can help fight dangers like fake news.But in China, one primary function of these technologies is to censor speech and erase collective memories of major events, however infrequently this function gets used."
Political speech comprised a tiny fraction of deleted content b/c Chinese people know what not to talk & hello, it's Douyin! Most censored content: porn, nudity, graphic images, curse words, unauthorized live streaming sales and content that violated copyrights. 4/
Excellent read from our @protocol |China team's @shenlulushen on the personal struggles of a former ByteDance / TikTok content moderator/ censor. https://www.protocol.com/...
ByteDance's content moderation policymakers + the team that creates core tech to support its content moderation operation + about 20,000 content moderators, have helped shield ByteDance from major political repercussions and achieve commercial success. 2/
After their own social media account got “bombed” last year, they realized, “I had helped build the system that censored accounts like mine. I was helping to bury myself in China's ever-expanding cyber grave.” /end
The details in this @shenlulushen joint are just nuts: “Not many tech companies in China have so many resources dedicated to moderating content. Other user-generated content platforms in China have nothing on ByteDance.” https://www.protocol.com/...
Software engineering is political. The choices that you make (or choose not to make) affect people's lives. This is as true in the US and the EU as it is in China. https://www.protocol.com/...
This is the story of a former content moderation employee at ByteDance, who told @shenlulushen that ByteDance received requests to “develop an algorithm that could automatically detect when a Douyin user spoke Uyghur - and then cut off the livestream.” https://www.protocol.com/..…
This is what violation of freedom of speech is about. It comes from a government and it is on all the country's platforms, incl websites. What Facebook and Twitter do is not, as they are private initiatives and there are loads of alternatives https://www.protocol.com/...
“I felt guilt more than anger. I helped develop tools and platforms for content moderation. In other words, I had helped build the system that censored accounts like mine. I was helping to bury myself in China's ever-expanding cyber grave.” https://www.protocol.com/...
“When I was at ByteDance, we received multiple requests from the bases to develop an algorithm that could automatically detect when a Douyin user spoke Uyghur, and then cut off the livestream session.” https://www.protocol.com/...
“Streamers speaking ethnic languages & dialects that Mandarin-speakers don't understand would receive a warning to switch to Mandarin. If they didn't comply, moderators would respond by manually cutting off the livestreams, regardless of the [...] content.” https://www.protocol.c…
Anything users upload is closely monitored and screened by both machines and humans. 👇This flowchart from a report ByteDance published in 2019 illustrates how live-stream moderation works. & My source also explained the process to me in my story: https://www.protocol.com/... 6/ h…
Dr. Li [Wenliang] ... was punished for that. ... we had no idea what's really happening because of authorities' cover-up of the severity of the crisis. Around this time last year, many Chinese tech companies were actively deleting posts, videos, 1/2 https://www.protocol.com/...
“In other words, I had helped build the system that censored accounts like mine. I was helping to bury myself in China's ever-expanding cyber grave.” https://www.protocol.com/... via @hare_brain
Interview with a former ByteDance employee: “I couldn't help but feel every day like I was a tiny cog in a vast, evil machine.” @shenlulushen https://www.protocol.com/...
A must-read piece by always-brilliant @shenlulushen that offers important insights into how popular Chinese social media apps build their content moderation mechanisms, the struggles that Chinese employees in those positions have to deal with and more. https://www.protocol.com/..…