Graphika: fake Chinese account network taken down by Facebook used AI-generated faces, an increasingly common tactic meant to evade detection
TechCrunchTaylor Hatmaker
Context & Ripple Effects
Graphika's finding extends a pattern Facebook has been fighting since late 2019, when it and Twitter shut down networks using AI-generated faces to masquerade as Americans — an operation Facebook tied to Epoch Media Group. The tactic has now surfaced in a Chinese-linked network, meaning synthetic profile photos are no longer confined to one influence operation.
Facebook's countermeasures have scaled alongside the threat: by March 2020 its machine-learning detection tooling had helped remove billions of fake accounts, and in 2021 it partnered with Michigan State University on a method for reverse-engineering deepfakes to identify the model that produced them. Graphika's report is a data point in an escalating contest between generative tooling and platform detection.
First-order effects
Facebook must keep removing networks whose profile imagery is machine-made, raising its enforcement workload even as Graphika's attribution gives its integrity teams named targets.
Second-order effects
Detection vendors like Graphika become load-bearing infrastructure: their network analyses are what let platforms justify coordinated takedowns, pushing Facebook toward deeper technical methods like the model-fingerprinting work with Michigan State.
Third-order effects
If cheap face-generation keeps outrunning detection, platforms will need provenance and verification layers rather than per-network removals — a shift toward treating account authenticity as a systems problem, though how quickly regulators step in remains genuinely open.
The trend: Influence operations are industrializing synthetic identities, forcing platforms into an arms race where each takedown documents the next generation of evasion tactics.
CAUTION: today's Facebook takedown does not mean China-interferes-in-the-2020- election. It means that “individuals in Fujian Province” spectacularly failed to interfere in the 2020 election. Remember: some active measures are only really activated by their post-takedown coverage…
Both Facebook and Twitter have teams of very smart people whose job is stopping foreign trolls from manipulating their platforms. They do it a lot! (See today's China takedown: https://t.co/...) But what's the point if Americans can do the same thing with no consequences?
One way of identifying fake accounts is to reverse search their profile image to see if it was stolen from someone else. I guess it was a matter of time before fake accounts started using AI-generated/altered faces to avoid detection. The future is here. https://techcrunch.com/..…
We found the Chinese operation based on our own internal investigations. It primarily targeted the Philippines and Southeast Asia, but had a very limited amount of US-focused content. @Graphika_NYC's report: https://graphika.com/...
Social media literacy — and maritime security literacy — is national security, too. The @USNavy @USPacificFleet and @US7thFleet will continue to sail wherever international law allows... and continue to communicate timely and accurate info about it. https://twitter.com/...
Because nobody learns anything , Facebook is still leaning heavily on the sinophobia to avoid dealing with the fact it hasn't done its job on any th ing else https://twitter.com/...
More disinformation in the Philippines. 155 accounts, 11 pages, 9 groups and 6 Instagram accounts pushed stories of interest to OFWs, as well as content that was supportive of President Rodrigo Duterte's campaign for re-election in the Philippines. https://www.nytimes.com/...
Nick Clegg acknowledged today that the domestic disinfo threat in 2020 is worse than the 2016 Russian ops: https://www.usatoday.com/... especially tough to deter if you lack the political will to try... https://twitter.com/...
First major spotlight on China's election-time disinformation: Today Facebook says it removed 155 accounts, 11 Pages, 9 Groups, 6 Insta accounts from China — supportive and critical posts on Pete Buttigieg, Vice President Biden and President Trump. https://about.fb.com/...
Lawmakers are increasingly turning to a new ally in their efforts to track foreign election interference: tech companies. Microsoft, Facebook, Twitter and others are frequently in touch with lawmakers about threats from countries such as Russia and China. https://apne.ws/rl7G14o
Facebook's China takedown shows where Beijing's priorities lie: on foreign policy and defense, not election interference, @selectedwisdom tells me. https://www.npr.org/...
Today we announced two CIB takedowns: a Chinese network targeting Southeast Asia, with very limited US-focused activity, and a domestic Philippine network that is linked to the Philippine military and police. https://about.fb.com/...
Here's the @Graphika_NYC report. Overall, 155 accounts on FB, 11 pages, 9 groups, 6 Instagram accounts. 2017: mostly Taiwan 2018-19: + Philippines and South China Sea 2020: + US-centric content. The biggest audience was in the Philippines. https://graphika.com/...