How predictive algorithms are increasingly being used to set police patrols, prison sentences, and probation rules in the US and Europe
ideally — remove human bias. A recent United Nations report warned that governments risked “stumbling zombie-like into a digital-welfare dystopia."" https://www.nytimes.com/... Kevin Baron / @defensebaron : Computers? We left this up to computers? https://www.nytimes.com/... via @NYTimes Justin Hendrix / @justinhendrix : An Algorithm That Grants Freedom, or Takes It Away: Across the United States and Europe, software is making probation decisions and predicting whether teens will commit crime. Opponents want more human oversight. https://www.nytimes.com/... @princeton : The topic of #AIHumanRights shaped a conference at @PrincetonCITP in 2019 as social scientists, activists, policy experts and computer scientists offered their insights to @Alston_UNSR for his @UN report on tech in social protection systems. https://www.princeton.edu/... #AIHumanRights https://twitter.com/... @bigbrotherwatch : Almost 200,000 people in Bristol are automatically monitored, ‘risk scored’ + flagged by predictive algorithms for likelihood of antisocial behaviour. Our investigations found multiple local authorities using risk score systems - especially in welfare. https://www.nytimes.com/... Helen Gym / @helengymatlarge : In a justice system already biased against black, brown and low-income people, we don't need algorithms making decisions to jail or release people. @mediamobilizing and @mediajustice compiled a nationwide database of these algorithms. https://www.nytimes.com/...
New York Times
Context & Ripple Effects
This feature lands at the midpoint of a decade-long arc: US courts had already been weighing what it means to lean on proprietary third-party recidivism algorithms for sentencing and bail as far back as 2017, and by late 2019 MIT Technology Review was letting players feel those flaws firsthand through a game built on real defendant data exposing COMPAS's shortcomings. The Times piece widens the lens from one tool to a pattern spanning both sides of the Atlantic.
What makes the moment notable is scale and scope: nearly 200,000 people in Bristol alone are automatically risk-scored and flagged for antisocial behaviour, while probation decisions and teen-crime predictions run through software in the US and Europe. The United Nations has already warned governments against 'stumbling zombie-like into a digital-welfare dystopia,' and activists are explicitly calling for more human oversight.
First-order effects
Defendants, probationers, and Bristol residents are being monitored, risk-scored, and flagged by systems whose logic is largely opaque to them, with judges and probation officers acting on software outputs rather than independent judgment.
Opponents and activists pressing for human oversight gain a concrete target: the specific deployments named in the coverage — COMPAS-style risk scoring, algorithmic probation rules, and Bristol's citywide flagging.
Second-order effects
A legal counter-industry takes shape: civil lawyers are already developing litigation strategies against automated systems that deny poor people basic services, giving affected individuals a procedural path that did not exist when these tools first entered courtrooms.
If the pattern holds, criminal-justice AI procurement shifts from quiet vendor adoption to contested accountability regimes, where transparency requirements, challenge rights, and documented human review become conditions of deployment rather than afterthoughts.
The UN's digital-welfare warning points toward algorithmic governance becoming a cross-border policy issue, with the same risk-scoring infrastructure spanning policing, sentencing, probation, and welfare eligibility treated as one regulatory problem.
The trend: Criminal-justice AI is moving from opaque vendor-driven adoption toward legally contested deployment, as litigation, accuracy challenges, and oversight demands force transparency into systems that once operated unexamined.
For months, @CadeMetz and I have been chipping away at a story about people in the U.S. and Europe whose lives have been affected by government decisions made by algorithms. It included stops in Philadelphia, Rotterdam and Bristol. We're excited to share: https://www.nytimes.com/…
Algorithms are supposed to remove the human bias from the criminal justice system's decisions about liberty and prison. Instead, they are baking in biases under the guise of black-box objectivity. https://www.nytimes.com/...
Across the U.S. and Europe, governments are using predictive algorithms to guide police patrols, set prison sentences, and look for welfare cheats. But some are now pushing back. https://www.nytimes.com/...
algorithms used to predict “criminal risk” and determine probation length, bond amount, fines charged, incarceration sentences, and to determine who is surveilled and who deserves “safety” https://www.nytimes.com/...
“You mean to tell me I'm dealing with all this because of a computer?” Across the U.S. and Europe, software is making probation decisions and predicting whether teens will commit crime. https://www.nytimes.com/...
An algorithm created by a Penn professor deems certain people on probation in Philly as “high risk” without even telling them about it Critics say they're not sure how the decisions are made, and the algorithm takes transparency out of the process https://www.nytimes.com/...
<flips to the business section> Whoa, here is a super-interesting article about algorithms and criminal, unemployment insurance, and at-risk youth systems. https://www.nytimes.com/...
As predictive algorithms spread across the U.S. and Europe, United Nations investigators, civil rights lawyers, labor unions and community organizers are pushing back https://www.nytimes.com/...
Sobering @nytimes report on an increasingly common application of AI Across the United States and Europe, AI is making probation decisions and predicting whether teens will commit crime. Opponents want more human oversight. https://www.nytimes.com/...
Algorithms are based on historical data. When data is produced within a social system that is biased, you will get a biased algorithm. Instead of removing human bias (as is often claimed), you end up further codifying bias in the system. (more) https://www.nytimes.com/...
It's wild that the same algorithmic techniques that predict you'd like alt-right videos screaming about SJWs because you watched a gaming video on YouTube is now being used to determine jail sentences & parole based on predicting one's likelihood to commit https://www.nytimes.com…
Darnell Gates complained that conversations with his probation officers were cold and impersonal. He didn't realize that an algorithm had tagged him high risk until he was told about it during an interview with The New York Times. https://www.nytimes.com/...
“The algorithms are supposed to reduce the burden on understaffed agencies, cut government costs and — ideally — remove human bias. A recent United Nations report warned that governments risked “stumbling zombie-like into a digital-welfare dystopia."" https://www.nytimes.com/...
An Algorithm That Grants Freedom, or Takes It Away: Across the United States and Europe, software is making probation decisions and predicting whether teens will commit crime. Opponents want more human oversight. https://www.nytimes.com/...
The topic of #AIHumanRights shaped a conference at @PrincetonCITP in 2019 as social scientists, activists, policy experts and computer scientists offered their insights to @Alston_UNSR for his @UN report on tech in social protection systems. https://www.princeton.edu/... #AIHuman…
Almost 200,000 people in Bristol are automatically monitored, ‘risk scored’ + flagged by predictive algorithms for likelihood of antisocial behaviour. Our investigations found multiple local authorities using risk score systems - especially in welfare. https://www.nytimes.com/...
In a justice system already biased against black, brown and low-income people, we don't need algorithms making decisions to jail or release people. @mediamobilizing and @mediajustice compiled a nationwide database of these algorithms. https://www.nytimes.com/...