Algorithmic audits alone won't provide the accountability needed for AI systems because they lack the necessary market incentives and real government oversight
Alex C. Engler / Fast Company : Tweets: @thyacinth , @greene_dm , @johndavisson , @mona_sloane , @jeffbigham , @ndiakopoulos , @peterkwells , @alexcengler , @mark_riedl , @mathbabedotorg , @alexcengler , @fabiochiusi , and @hypervisible Tweets: @thyacinth : Yes!! The incentive structures in place are very worrisome. Governance does not happen purely via the free market/capitalism - we need supra-corporate forces in place. https://twitter.com/... Dan Greene / @greene_dm : HireVue uses narrow audit by respected researcher to ethicswash their phrenology https://www.fastcompany.com/ ... John Davisson / @johndavisson : Great piece from @AlexCEngler on the challenges and limitations of independent AI audits (and @Hirevue's dubious characterization of the audit it recently submitted to). https://www.fastcompany.com/ ... Dr. Mona Sloane / @mona_sloane : Why we need a definition of “audit” that AI companies are held accountable to: “Controversial AI company HireVue implied that an external audit showed its algorithms had no bias. But a look at the audit itself tells a different story.” @AlexCEngler in https://www.fastcompany.com/ ... Jeffrey P. Bigham, PhD / @jeffbigham : HireVue claimed its computer vision approach to hiring was unbiased, which on the surface seemed straightforwardly false... an example of why it's problematic to define bias in narrowly scoped ways. https://twitter.com/... Nick Diakopoulos / @ndiakopoulos : This is an important critique of the limitations of for-hire algorithm auditing. Accountability only happens when there are institutions with the power to compel or sanction based on information provided by an audit. And auditors need to be independent. https://www.fastcompany.com/ ... Peter Wells / @peterkwells : Ah, Hirevue has an “algorithmic audit” but no backing by an accepted formal process, professional accreditation or legislation &c That's a “review” not an “audit” (see also “data audit”, “data governance audit”, “ethical audit”, &c &c) https://www.fastcompany.com/ ... Alex Engler / @alexcengler : This is the crux - algorithmic auditors need leverage. To insist on data & model access, to insist on a representative set of models, to run their own statistical tests, to make results fully public, etc. https://twitter.com/... Mark O. Riedl / @mark_riedl : Who audits the auditors? https://www.fastcompany.com/ ... Cathy O'Neil / @mathbabedotorg : Alex Engle is right: algorithmic auditing companies like mine are at risk of becoming corrupt. We need more leverage to do things right, with open methodology and results. Where would we get such leverage? Lawsuits, regulatory enforcement, or both. https://www.fastcompany.com/ ... Alex Engler / @alexcengler : I wrote about algorithmic auditing for @FastCompany. The audit by @mathbabedotorg did some good, but HireVue clearly cares more about PR than introspection. If that's the future of the algorithmic auditing, it's not going to work out very well. https://www.fastcompany.com/ ... https://twitter.com/... Fabio Chiusi / @fabiochiusi : “Controversial AI company HireVue implied that an external audit showed its algorithms had no bias. But a look at the audit itself tells a different story.” https://www.fastcompany.com/ ... @hypervisible : Gotta admit, it was a bit shocking when HireVue claimed an independent audit said their algo was free from bias. So about that... https://www.fastcompany.com/ ...