Scientists criticize a lack of transparency in AI research, citing issues of replication in research and unequal access to code, proprietary data, and hardware
with dangerous consequences. Replication by different researchers would expose problems sooner, making AI stronger for everyone” #NLProc stop publishing what is not replicable https://www.technologyreview.com/ ... Delia Ferreira / @deliaferreira : Algorithmic transparency is essential. Lack of transparency in AI may have dangerous consequences👇 #Algocracy https://twitter.com/... @techreview : But according to its critics, the Google team provided so little information about its code and how it was tested that the study amounted to nothing more than a promotion of proprietary tech. @techreview : Google was describing successful trials of an AI that looked for signs of breast cancer in medical images. https://www.nature.com/... @techreview : “When we saw that paper from Google, we realized that it was yet another example of a very high-profile journal publishing a very exciting study that has nothing to do with science,” says @bhaibeka. “It's more an [ad] for cool technology. We can't really do anything with it.” @techreview : Replication also allows others to build on those results, helping to advance the field. Science that can't be replicated falls by the wayside. @techreview : Science is built on a bedrock of trust, which typically involves sharing enough details about how research is carried out to enable others to replicate it, verifying results for themselves. This is how science self-corrects and weeds out results that don't stand up. @techreview : Haibe-Kains and his colleagues are among a growing number of scientists pushing back against a perceived lack of transparency in AI research. @techreview : “We couldn't take it anymore,” says @bhaibeka, the lead author of the response, who studies computational genomics at the @UofT. “It's not about this study in particular—it's a trend we've been witnessing for multiple years now that has started to really bother us.” @techreview : But in fields like biology and physics—and computer science overall—researchers are typically expected to provide the information needed to rerun experiments, even if those reruns are rare. @techreview : In practice, few studies are fully replicated because most researchers are more interested in producing new results than reproducing old ones. @techreview : Last month @nature published a damning response written by 31 scientists to a study from Google Health that had appeared in the journal earlier this year. https://www.nature.com/... @techreview : Tech giants dominate AI research but the line between real breakthrough and product showcase can be fuzzy. Some scientists have had enough. https://www.technologyreview.com/ ... @facebookai : We're committed to open research. As Joëlle Pineau, Managing Director of Facebook AI Research highlights in this @techreview article, open and reproducible research benefits the entire community. https://twitter.com/... Mike Schroepfer / @schrep : Joëlle Pineau, Managing Director of Facebook AI Research: “For the last few years, she has been the driving force behind a change in how AI research is published” https://www.technologyreview.com/ ... Great article - Joelle is pushing the whole industry towards open and reproducible research @techreview : This is a thread about how artificial intelligence is wrestling with a replication crisis. https://www.technologyreview.com/ ...
Context & Ripple Effects
The transparency critique has been building for years: a survey of 400 algorithms at major conferences found just 6% shipped code and only 30% included test data, so replication was already known to be failing before this story broke.
What changed is that the complaint now has named targets and named defenders. Thirty-one scientists publicly faulted a Google Health study for providing so little detail about its code and testing that it read as promotion of proprietary tech, while Facebook AI Research under Joëlle Pineau has staked out open, reproducible research as an explicit commitment — turning transparency from an academic nicety into a competitive position.
First-order effects
- Google Health takes direct reputational damage: with 31 signatories questioning whether the study is verifiable science at all, its clinical claims carry less weight with the research community.
- Facebook AI Research's reproducibility pledge becomes an immediate recruiting and credibility asset against Google, whose corporate lab model is now the public counterexample.
Second-order effects
- Conferences and journals face pressure to make code and test-data release a review requirement rather than a courtesy, since peer reviewers are already absorbing oversight duties other fields staff separately (ethical-oversight gaps show the same reviewer-burden pattern).
- Labs publishing without replication support risk having their results discounted wholesale, shifting prestige toward groups like FAIR that can afford to release artifacts.
Third-order effects
- If unequal access to compute and proprietary data keeps skewing who can verify whom, AI research risks splitting into a reproducible commons and a promotional tier — a structural divide that later resurfaces when the open-source AI boom built on Big Tech models proves how fragile shared access really is.
- The same dynamic that produced Gebru's firing — corporate labs co-opting or sidelining internal criticism — points toward external checks: research protection norms and eventually regulation as the backstop when self-policing fails.
The trend: AI research is polarizing between corporate labs treating publications as product marketing and institutions making reproducibility a competitive differentiator, with replication failure as the fault line.