Algorithmic audits alone won't provide the accountability needed for AI systems because they lack the necessary market incentives and real government oversight
Alex C. Engler / Fast Company : Tweets: @thyacinth , @greene_dm , @johndavisson , @mona_sloane , @jeffbigham , @ndiakopoulos , @peterkwells , @alexcengler , @mark_riedl , @mathbabedotorg , @alexcengler , @fabiochiusi , and @hypervisible Tweets: @thyacinth : Yes!! The incentive structures in place are very worrisome. Governance does not happen purely via the free market/capitalism - we need supra-corporate forces in place. https://twitter.com/... Dan Greene / @greene_dm : HireVue uses narrow audit by respected researcher to ethicswash their phrenology https://www.fastcompany.com/ ... John Davisson / @johndavisson : Great piece from @AlexCEngler on the challenges and limitations of independent AI audits (and @Hirevue's dubious characterization of the audit it recently submitted to). https://www.fastcompany.com/ ... Dr. Mona Sloane / @mona_sloane : Why we need a definition of “audit” that AI companies are held accountable to: “Controversial AI company HireVue implied that an external audit showed its algorithms had no bias. But a look at the audit itself tells a different story.” @AlexCEngler in https://www.fastcompany.com/ ... Jeffrey P. Bigham, PhD / @jeffbigham : HireVue claimed its computer vision approach to hiring was unbiased, which on the surface seemed straightforwardly false... an example of why it's problematic to define bias in narrowly scoped ways. https://twitter.com/... Nick Diakopoulos / @ndiakopoulos : This is an important critique of the limitations of for-hire algorithm auditing. Accountability only happens when there are institutions with the power to compel or sanction based on information provided by an audit. And auditors need to be independent. https://www.fastcompany.com/ ... Peter Wells / @peterkwells : Ah, Hirevue has an “algorithmic audit” but no backing by an accepted formal process, professional accreditation or legislation &c That's a “review” not an “audit” (see also “data audit”, “data governance audit”, “ethical audit”, &c &c) https://www.fastcompany.com/ ... Alex Engler / @alexcengler : This is the crux - algorithmic auditors need leverage. To insist on data & model access, to insist on a representative set of models, to run their own statistical tests, to make results fully public, etc. https://twitter.com/... Mark O. Riedl / @mark_riedl : Who audits the auditors? https://www.fastcompany.com/ ... Cathy O'Neil / @mathbabedotorg : Alex Engle is right: algorithmic auditing companies like mine are at risk of becoming corrupt. We need more leverage to do things right, with open methodology and results. Where would we get such leverage? Lawsuits, regulatory enforcement, or both. https://www.fastcompany.com/ ... Alex Engler / @alexcengler : I wrote about algorithmic auditing for @FastCompany. The audit by @mathbabedotorg did some good, but HireVue clearly cares more about PR than introspection. If that's the future of the algorithmic auditing, it's not going to work out very well. https://www.fastcompany.com/ ... https://twitter.com/... Fabio Chiusi / @fabiochiusi : “Controversial AI company HireVue implied that an external audit showed its algorithms had no bias. But a look at the audit itself tells a different story.” https://www.fastcompany.com/ ... @hypervisible : Gotta admit, it was a bit shocking when HireVue claimed an independent audit said their algo was free from bias. So about that... https://www.fastcompany.com/ ...
Context & Ripple Effects
Alex C. Engler's argument lands in a stretch of coverage where voluntary scrutiny keeps failing quietly: HireVue implied an external audit cleared its algorithms of bias when the audit itself supported no such claim — what Dan Greene calls ethicswashing by narrow audit. Weeks earlier, Gebru's firing showed how internal critics get co-opted or pushed out, leaving outside reviewers as the last line of defense.
The gap Engler identifies is structural: auditors have no legal or regulatory leverage — no lawsuits, no enforcement — so a company unhappy with findings can simply shop for a friendlier report. The Markup's follow-up makes the same point from the auditor's side, arguing the industry needs standards that hold both auditors and the companies that hire them to account.
First-order effects
- Vendors like HireVue can keep citing voluntary external audits as de facto bias clearances, because nothing obliges them to disclose scope, methods, or unfavorable findings.
- Independent auditors remain dependent on the fees of the companies they assess, so their reports bend toward the client even when researchers flag the incentive problem publicly.
Second-order effects
- Buyers of high-stakes systems — courts using tools like COMPAS, employers using hiring algorithms — gain grounds to demand contractual audit terms with real disclosure obligations rather than accepting marketing claims.
- Audit firms face pressure toward accreditation and standardized methodology, since without it their work is dismissible as advocacy and with it they take on liability for what they sign off on.
Third-order effects
- If Engler's leverage argument holds, AI accountability migrates from voluntary audits to litigation and regulation, with audits becoming evidence in lawsuits rather than press releases.
- That shift converges with the post-Gebru critique that meaningful oversight requires protected critics and supra-corporate enforcement — a governance layer the current market structure cannot supply on its own.
The trend: AI accountability is moving from voluntary third-party audits toward legally enforceable oversight, as audits without regulatory teeth prove easy to co-opt.