Analysis of 7K+ cases: strong bias against black defendants found in crime prediction software that's increasingly relied upon to guide sentencing in US courts
There's software used across the country to predict future criminals. And it's biased against blacks.
Context & Ripple Effects
ProPublica's analysis of more than 7,000 cases lands on a tool already embedded in the system: proprietary recidivism scores that US courts lean on for sentencing decisions, a reliance the New York Times documented a year later as justices weighed the consequences of opaque third-party algorithms. The finding gives that abstract debate a concrete failure mode — the same scores are systematically wrong in one direction for Black defendants.
It also fits a broader arc the corpus keeps returning to: predictive algorithms steering police patrols ([[a:973511|PredPol's bias investigation found it directing officers toward poor, Black, and Latino neighborhoods]]) and pretrial detention (New Jersey's PSA cut jail populations but racial disparities persisted). Bias isn't a bug in one vendor's model; it recurs at every point where scoring meets discretion.
First-order effects
- Black defendants scored by this software carry an inflated risk label into bail and sentencing hearings right now, while courts relying on the tool must defend outputs they can't independently verify.
- The vendor faces immediate pressure to justify its methodology, since the study's case-level evidence undercuts the neutrality claim that made the scores attractive to courts in the first place.
Second-order effects
- Other jurisdictions running similar assessments — including New Jersey's PSA, where disparities persisted despite better outcomes — face calls to audit their own tools rather than assume the problem is isolated to one product.
- Judges' behavior shifts the battleground: the Verge's finding that they use these assessments selectively suggests courts may hedge around untrusted scores instead of abandoning them, keeping vendors in place but shrinking their influence.
Third-order effects
- If the pattern holds — bias documented in policing tools like PredPal-style systems, detention tools like the PSA, and now sentencing support — proprietary criminal-justice algorithms head toward mandatory disclosure and external auditing as a condition of court use, reshaping how vendors sell to government.
- The deeper structural shift is that 'algorithmic' stops functioning as an authority argument in court: once scoring tools are demonstrably biased, their evidentiary weight depends on transparency, not on being math.
The trend: Criminal justice is adopting predictive algorithms faster than any accountability regime can vet them, and each bias finding — in policing, pretrial, and sentencing tools — converts scrutiny from academic critique into procurement and litigation pressure.