A game made with a real world dataset of defendants shows the shortcomings of COMPAS, an AI-powered risk assessment tool used in the US criminal legal system
Sundays are for making a house into a home … Tweets: Karen Hao / @_karenhao : IT'S HERE!!! The biggest story I've ever worked on. @techreview's very first interactive ever, which walks through a concrete example of AI bias, and why it's so much more complicated than initially meets the eye. https://www.technologyreview.com/ ... 1/ Gerry McGovern / @gerrymcgovern : Software engineers are being asked to articulate what it means to be fair in their code. This is why regulators around the world are now grappling with a question: How can you mathematically quantify fairness? https://www.technologyreview.com/ ... @techreview : In 2016, a @ProPublica investigation argued that an algorithm used in the US criminal legal system was biased against black defendants. Can you make AI fairer with this courtroom algorithm game? (don't worry, it won't involve any coding) https://www.technologyreview.com/ ... https://twitter.com/... @techreview : We need your help to make a real algorithm less biased. We're going to walk through the algorithm, one used to decide who gets sent to jail, and ask you to tweak its various parameters to make its outcomes more fair. Play the game—and read the story. https://www.technologyreview.com/ ... Mary L. Gray / @marylgray : Original, critical, thought-provoking journalism—idata-driven reporting at its best. It doesn't let data sit self-evident. Instead, @_KarenHao and @techreview colleagues ask us to consider how we impose tech on each other, particularly the most disenfranchised 👇🏼👇🏼 👇🏼 https://twitter.com/... Franois Petitjean / @ledataminer : Such a fantastic, didactic tool to explain the issues of fairness and AI to non-specialists. Hard recommend. Via @sarahbmyers. In case you didn'tsee this @westylesty @rgibli @kim_weatherall @Lizzie_OShea @parismarx https://twitter.com/... @ainowinstitute : Great article by @_KarenHao that shows why predictive algorithms don't make the judicial process more fair - and why we need impact assessments 👍 (nice to see shout outs to @ruha9 and our policy director Rashida Richardson) https://twitter.com/... Varoon Mathur / @varoonmathur : This is really good - and highlights well the divide between algorithmic “fairness” and actual justice. https://twitter.com/... Jason Pontin / @jason_pontin : This interactive exploration of the criminal risk assessment algorithm COMPAS by @_KarenHao and @jonathanstray is one of the best things published by @techreview. It unpacks the conflicts inherent in any such tool in a way a linear narrative could not. https://www.technologyreview.com/ ... Hi-Phi Nation / @hiphination : MIT has made a visual to show the exact problems with algorithmic risk assessment to date, and essentially why there is no solution. https://www.technologyreview.com/ ...
Context & Ripple Effects
MIT Technology Review's first interactive — reported by Karen Hao — turns the COMPAS debate into something a reader can play: a game built on a real-world defendants dataset that shows why the tool's bias problem is more complicated than it first appears. The piece lands in a lineage that includes ProPublica's scrutiny of COMPAS and a wider cast — Rashida Richardson, Jonathan Stray — asking how fairness gets quantified in code.
It also arrives just ahead of a documented expansion wave: by early 2020, reporting showed predictive algorithms setting police patrols, prison sentences, and probation rules across the US and Europe, making the question of what 'fair' means in a risk score a live operational issue rather than an academic one.
First-order effects
- Judges, probation officers, and defendants relying on COMPAS scores get a concrete demonstration — playable, not abstract — that the tool's error rates trade off against each other depending on which definition of fairness the designer picks.
- Software engineers building risk tools are now explicitly tasked with articulating fairness in code, the question Gerry McGovern flags as one regulators worldwide are grappling with.
Second-order effects
- Defense counsel gain a template for attacking algorithmic evidence: the same instinct behind civil lawyers' litigation strategies against automated systems that deny poor defendants basic services.
- The scrutiny spreads to newer tools — defense lawyers have since challenged the accuracy of Cybercheck, an AI suspect-identification tool used in thousands of US cases, showing the COMPAS critique generalizing across the criminal-legal AI stack.
Third-order effects
- If the pattern holds, every risk-assessment vendor in the criminal-legal system faces a rising evidentiary burden: accuracy and fairness claims must survive adversarial examination, shifting the market toward tools that can document their trade-offs.
- Regulators' struggle to mathematically quantify fairness points toward formal audit and disclosure requirements for public-safety algorithms — the structural question this interactive is one early data point in.
The trend: Criminal-legal AI is moving from black-box adoption toward contested, litigable tools, as journalists, defense lawyers, and regulators force risk-assessment systems to justify their fairness trade-offs.