Facebook announces the results of its first Deepfake Detection Challenge, says the winning algorithm spotted deepfakes with an average accuracy of just 65.18%
When [[a:945568|Facebook, Microsoft, the Partnership on AI and seven universities launched the Deepfake Detection Challenge]] in September 2019, the premise was that a shared dataset would crowdsource detectors good enough for platform-scale use. Eighteen months of competition produced a winner that catches fakes only 65.18% of the time on average — a number that lands awkwardly next to Facebook's own deepfakes removal policy, which already covers only certain AI-made videos while exempting satire.
The result also reframes the commercial market around detection: by 2024, [[a:865021|startups were selling deepfake-detection services on accuracy claims the Post found largely untested]], a claim environment this benchmark result cuts against.
First-order effects
Facebook now has hard evidence from its own contest that automated detection is nowhere near reliable enough to anchor its moderation policy, leaving human review and its narrow video-only rules carrying more weight than intended.
Microsoft and the academic partners get a sobering readout on their investment: the best model trained on the challenge data misses roughly one in three deepfakes.
Second-order effects
Vendors claiming near-perfect detection face an unfavorable public baseline — a peer-benchmarked 65.18% makes unverified accuracy marketing harder to defend.
Platforms leaning on detection tools are pushed toward complementary signals like metadata and provenance checks, since detection alone demonstrably underperforms at scale.
Third-order effects
If the gap between generation quality and detection accuracy keeps widening, content authenticity will likely be established upstream — at capture or publication — rather than inferred downstream by classifiers, shifting the burden toward provenance standards and labeling regimes.
Regulators drafting synthetic-media rules will find in this result that mandates to 'detect' deepfakes assume capabilities the field had not demonstrated as of 2020.
The trend: Synthetic-media defense is settling into a layered trust stack where detection is one imperfect signal alongside provenance and labeling, because benchmarked detectors trail generators.
2,114 participants around the globe entered the Deepfake Detection Challenge. We're now sharing the winning models and insights from this first-of-its-kind open initiative to address the challenge of deepfake videos and images. https://ai.facebook.com/... https://twitter.com/...
I was pretty frustrated by the amount of time the ML industry spent making deepfakes better compared to the time spent combating the harm they could create. So we used a competition plus open science to spur more focus. https://venturebeat.com/...
There is never going to be an algorithm that can tell deepfake from real content without context. When we at @pex looked at the problem, the solution is quite obvious. Rather than focusing on the content itself we focus on where it comes from and how it spreads over the internet …
Surprising amount of “this is fine” in this article about how the winning algorithm in a competition could only detect deepfakes with an average accuracy of 65.18 percent. 65.18%!!! I'm bad at maths, but that's not enough percents. https://www.theverge.com/...
Facebook CTO: “Deepfakes are currently not a big issue. But the lesson I learned the hard way over last couple years is not to be caught flat-footed.” https://www.technologyreview.com/ ...
The #DFDC competition has ended! Almost 1 year of hard work to push forward the detection of #DeepFakes. Congratulations to all the participants and, in particular, to the top-5 winners. https://twitter.com/...
It's relatively easy to fool AI models designed to detect fake videos called deepfakes, a competition run by Facebook, Amazon, Microsoft and big-name universities found. That's too bad for US elections. https://www.cnet.com/...