Leaked docs and sources describe Facebook's content moderation apparatus and the logistics of trying to moderate billions of posts a week in 100+ languages
tasked with enforcing values for the whole friggin' world—have a McDonald's level turnover rate https://motherboard.vice.com/ ... Mathew Ingram / @mathewi : As this Motherboard piece makes clear, trying to come up with programming-level solutions to problems like hate speech at the scale of Facebook isn't just difficult, it might actually be impossible http://www.techmeme.com/... Sarah Frier / @sarahfrier : “Facebook's content moderation team has been given a Sisyphean task: Fix the mess Facebook's worldview and business model has created, without changing the worldview or business model itself.” A content moderation deep-dive: http://motherboard.vice.com/ ... Jason Koebler / @jason_koebler : For example, “white supremacy” is banned, but “white nationalism” and “white separatism” is not. Facebook says this is to protect separatist movements around the world, even though there are specific carveouts to protect white nationalists in the US pic.twitter.com/kpbdL7nPYT Jason Koebler / @jason_koebler : There's also this, which describes in excruciating detail exactly *when* it's permissible to photoshop anuses onto human beings on Facebook pic.twitter.com/qeMxttGuzw Jason Koebler / @jason_koebler : We also have confirmation that Zuckerberg and Sheryl Sandberg are, from time to time, weighing in on individual content moderation decisions and policies. This obviously doesn't scale well and isn't ideal Mar Hicks / @histoftech : “Facebook's mission has led it down the difficult road of trying to connect the entire world, which it believes necessitates allowing as much speech as possible.” Guess they never read The Hitchhiker's Guide To The Galaxy huh (cf. babelfish). http://motherboard.vice.com/ ... Dory Carr-Harris / @dory_carrharris : this is one of the most important stories we've published this year. if people don't yet understand that we are at a crucial moment in the development of our society and the values that we founded it on, i hope they will after reading this. http://twitter.com/... Charlie Warzel / @cwarzel : what i appreciate about this @motherboard story is how it lays out — in very clear terms — the ungovernable monster that Facebook has created. Unprecedented scale! Maybe it's...not great? http://motherboard.vice.com/ ... http://twitter.com/... Joseph Cox / @josephfcox : When we went to Facebook headquarters to talk to executives, @jason_koebler snapped this photo of motivational phrases. “Make it up as you go,” feels quite apt for Facebook's content moderation https://motherboard.vice.com/ ... pic.twitter.com/eS0DWu6bi5 Joseph Cox / @josephfcox : This is really indicative of why AI is not going to replace humans any time soon. Facebook's AI hasn't yet figured out how to deal with Burmese, because it's encoded by computers differently. “We still don't know if it's really going to work out” https://motherboard.vice.com/ ... pic.twitter.com/oMxNguEthh See also Mediagazer
Context & Ripple Effects
The Motherboard leak is the ground-floor document of a story that kept unfolding for years afterward: Facebook asking a small policy group of engineers, lawyers, and PR staff — covered later by Vanity Fair's look inside the content policy team — to set speech rules for billions of weekly posts in over 100 languages, enforced largely by contractors churning through the job at McDonald's-level turnover.
The leak also prefigures the accountability arc that followed: Facebook's public commitment to contractor working conditions in 2019, the technocratic takedown reports that obscure moderator trauma, and ultimately the Facebook Files framing of executives who believed they could get the system back under control.
First-order effects
- Contractor moderators absorb the direct cost: high-turnover enforcement work on graphic material, with policy carveouts like the white-nationalism allowance and granular image rules landing on their desks as judgment calls.
- Zuckerberg and Sheryl Sandberg personally weighing in on individual decisions means the highest levels of the company remain inside the moderation loop rather than delegating it.
Second-order effects
- Automation hits a hard linguistic wall where AI can't follow — Burmese's different encoding limits machine moderation — pushing more volume back onto human reviewers and raising Facebook's contractor headcount and costs.
- Facebook is forced into public defense mode on labor conditions, issuing statements vouching for its outsourcing partners' standards as scrutiny of the moderator workforce intensifies.
Third-order effects
- If scale makes rule-based moderation effectively impossible — as Mathew Ingram argued in response to the leak — platform governance drifts toward either heavier human infrastructure or narrower product ambitions, with internal research on harms becoming politically dangerous enough that executives later shelved it.
- The pattern points toward external oversight pressure: enormous speech power concentrated in one company with little accountability becomes the standing argument for regulators and for journalists treating leaked internals as the primary record.
The trend: Platform content governance is shifting from an engineering-scalability problem to a human-labor and institutional-accountability problem, with each leak narrowing what Facebook can credibly claim its systems control.