After Big Tech cut thousands of trust and safety staff, Sero AI and other outsourcing startups sell the services, which experts warn bring problems of their own
The burgeoning trust and safety industry promises to help tech companies navigate scrutiny and regulation. But these services bring problems of their own. X: @daphnehk , @gabenicholas , @tnewmstweet , @sayhar , @integrity_inst , and @telliotter LinkedIn: Boris Bukengolts , Jeremy H. Gottschalk , Farzaneh Badiei, PhD , and Jessica Smith X: Daphne Keller / @daphnehk : The Rise of the Vendors is a key development in the history of Trust and Safety. It doesn't get much attention. I suspect it's mostly bad news, but maybe I'm just a curmudgeon. Gabriel Nicholas / @gabenicholas : Fantastic article from @telliotter in @WIRED about the rise (and potential dangers of) content moderation-as-a-service. And happy that I could provide some quotes as well. 100% worth reading. https://www.wired.com/... Tim Newman / @tnewmstweet : “experts warn that outsourcing trust and safety also means outsourcing responsibilities to teams with no power to change the way platforms actually work.” https://www.wired.com/... Sahar Massachi / @sayhar : The most important levers to doing integrity work right — design, ranking changes, setting the right metrics — are explicitly out of the control of vendors. If they handle whack a mole while product teams do the rest — that's great. If they're the only line of defense? Bad news @integrity_inst : Integrity Institute executive director @sayhar was quoted in this @WIRED story about the impact of trust & safety layoffs and the downstream effect on the start-up ecosystem ⤵️ https://www.wired.com/... Tori Elliott / @telliotter : Many tech companies have responded to leaner times by cutting the teams that keep hate, violence, and nudity off their platforms. But now a burgeoning group of startups is taking those skills and selling them back to platforms big and small @WIRED https://www.wired.com/... LinkedIn: Boris Bukengolts : The rise of these startups underscores the ever-evolving landscape of opportunities within the tech industry. — #TechForGood #OnlineSafety #Innovation #TrustAndSafety #DigitalEthics Jeremy H. Gottschalk : It's always a good day when trust & safety makes headlines. — It's critical that T&S stay in the news simply because the amount of people … Farzaneh Badiei, PhD : This is why we need open source digital governance and initiatives like Digital Trust and Safety Partnership that work on coming up with trust and safety pratices! Jessica Smith : Regardless of your industry, the safety of your customers/clients and your team members should always be priority number ONE! …
Context & Ripple Effects
Trust and safety has long faced an authority problem inside tech companies: earlier coverage found that in-house ethicists had limited power to change company decisions. The move from internal teams to specialist vendors makes that limitation more consequential because the people handling harm may sit even farther from product and business decisions.
The story also fits a broader safety-versus-incentives tension. CISA leaders had warned that commercial incentives can eclipse customer safety, while later industry discussion reflected anxiety about a broader retreat from safety commitments.
First-order effects
- Platforms that cut trust and safety staff can shift operational work to Sero AI and similar vendors, creating immediate demand for outsourced review, policy, and integrity services.
- Vendors inherit responsibility for identifying and managing harms, but—per the experts cited—lack the authority to change platform design or success metrics that can produce those harms.
Second-order effects
- Trust and safety becomes more of a procurement relationship: platforms must translate internal risk judgments into vendor scopes, reporting, and escalation paths, while vendors compete to become the external operating layer.
- The separation can weaken the feedback loop from moderation and enforcement work to product teams, increasing the importance of contractual access and escalation mechanisms rather than informal internal influence.
Third-order effects
- If this model persists, trust and safety may evolve from a core platform function into a specialized services market, with accountability split between platforms that control design and vendors that execute safety operations.
- Regulatory and public scrutiny may increasingly test whether outsourcing changes who is operationally responsible without changing who has the power to prevent harm—a form of ethics work serving as moral cover rather than implementation if governance does not follow authority.
The trend: This is one data point in the externalization of platform governance: companies retain control over product incentives while specialist vendors take on more of the safety work and associated scrutiny.