Scientists criticize a lack of transparency in AI research, citing issues of replication in research and unequal access to code, proprietary data, and hardware
Tech giants dominate research but the line between real breakthrough and product showcase can be fuzzy. Some scientists have had enough. Tweets: @techreview , @techreview , @techreview , @techreview , @techreview , @techreview , @techreview , @techreview , @techreview , @techreview , @techreview , @facebookai , @schrep , and @techreview Tweets: @techreview : Google was describing successful trials of an AI that looked for signs of breast cancer in medical images. https://www.nature.com/... @techreview : Haibe-Kains and his colleagues are among a growing number of scientists pushing back against a perceived lack of transparency in AI research. @techreview : “We couldn't take it anymore,” says @bhaibeka, the lead author of the response, who studies computational genomics at the @UofT. “It's not about this study in particular—it's a trend we've been witnessing for multiple years now that has started to really bother us.” @techreview : But in fields like biology and physics—and computer science overall—researchers are typically expected to provide the information needed to rerun experiments, even if those reruns are rare. @techreview : “When we saw that paper from Google, we realized that it was yet another example of a very high-profile journal publishing a very exciting study that has nothing to do with science,” says @bhaibeka. “It's more an [ad] for cool technology. We can't really do anything with it.” @techreview : But according to its critics, the Google team provided so little information about its code and how it was tested that the study amounted to nothing more than a promotion of proprietary tech. @techreview : Tech giants dominate AI research but the line between real breakthrough and product showcase can be fuzzy. Some scientists have had enough. https://www.technologyreview.com/ ... @techreview : In practice, few studies are fully replicated because most researchers are more interested in producing new results than reproducing old ones. @techreview : Replication also allows others to build on those results, helping to advance the field. Science that can't be replicated falls by the wayside. @techreview : Science is built on a bedrock of trust, which typically involves sharing enough details about how research is carried out to enable others to replicate it, verifying results for themselves. This is how science self-corrects and weeds out results that don't stand up. @techreview : Last month @nature published a damning response written by 31 scientists to a study from Google Health that had appeared in the journal earlier this year. https://www.nature.com/... @facebookai : We're committed to open research. As Joëlle Pineau, Managing Director of Facebook AI Research highlights in this @techreview article, open and reproducible research benefits the entire community. https://twitter.com/... Mike Schroepfer / @schrep : Joëlle Pineau, Managing Director of Facebook AI Research: “For the last few years, she has been the driving force behind a change in how AI research is published” https://www.technologyreview.com/ ... Great article - Joelle is pushing the whole industry towards open and reproducible research @techreview : This is a thread about how artificial intelligence is wrestling with a replication crisis. https://www.technologyreview.com/ ...
Context & Ripple Effects
This story is the escalation of a problem documented two years earlier, when a survey of 400 conference papers found just 6% shipped code and 30% shipped test data. What has changed is who is saying it: Haibe-Kains and 30 co-authors took the fight into Nature itself, targeting a Google Health breast-cancer AI study whose trial results were announced before outsiders could verify them.
The critique lands on a structural asymmetry — Big Tech labs control the proprietary data, compute, and code that replication requires — and it arrives just as Facebook AI Research, under Joëlle Pineau, has made open, reproducible research an explicit differentiator. The related coverage shows where this goes next: internal dissent (Timnit Gebru's firing) and reviewers left carrying the ethical-oversight load are the same transparency fight moving from methods to governance.
First-order effects
- The Nature response puts Google Health's breast-cancer study under formal reproducibility scrutiny, pressuring the lab to release code, data, and trial details it has so far kept proprietary.
- Facebook AI Research's Pineau-led openness commitment becomes a competitive contrast: FAIR can now market verifiability against Google's product-showcase style of announcement.
Second-order effects
- Journals and conference reviewers, already the de facto ethics backstop per the related oversight coverage, gain a concrete precedent for demanding code and data as a condition of publication — raising the cost of closed corporate research.
- Other corporate labs face a fork between matching FAIR's disclosure posture or absorbing the reputational discount of the 'product showcase' label that Haibe-Kains' group has attached to unreplicable claims.
Third-order effects
- If disclosure becomes a publication requirement, the compute-and-data moat that lets Big Tech dominate research starts to double as an accountability liability, strengthening the case for external oversight and regulation that the Gebru episode made explicit.
- The later open-source AI boom — models like LLaMA and GPT-3 that the ecosystem depends on — is the downstream test of this fight: whether Big Tech's giants treat openness as durable infrastructure or a revocable courtesy.
The trend: AI research is moving from self-policed corporate announcement toward enforced reproducibility, with journals, open-lab commitments, and internal dissent converging on the same transparency demand.