A growing number of civil lawyers in the US are developing litigation strategies to push back against automated systems that deny the poor basic services
A growing group of lawyers are uncovering, navigating, and fighting the automated systems that deny the poor housing, jobs, and basic services. Tweets: @tonyajoriley , @ozloop , @hypervisible , @techreview , @datasociety , @karen_ec_levy , @_karenhao , and @kschwabable Tweets: Tonya Riley / @tonyajoriley : The flip side of this is that low-income Americans have some of the worst privacy and data protection. Yet they are disproportionately subject to the whims of algorithms that rely on that data! https://www.technologyreview.com/ ... Steve Davies / @ozloop : @reinventionfail @blakandblack Richardson thinks he is living in tech heaven. All that data on our phones and all those scary thing in the world. Hey, Richo. Care to tell us how OUR security services are using AI? That's so we can prepare mate @markdreyfusQCMP #Robodebt was a touch up https://www.technologyreview.com/ ... @hypervisible : “... for low-income individuals, the rapid growth and adoption of automated decision-making systems has created a hidden web of interlocking traps.” Excellent read. https://www.technologyreview.com/ ... @techreview : A growing group of lawyers are fighting the algorithms that decide credit scores, which patients receive medical care, which families get access to stable housing, and which children enter foster care. https://www.technologyreview.com/ ... @datasociety : “'We all need more training, more knowledge—not just in the law, but in these systems...Ultimately it's like every case is going to turn into an algorithm case.'” -@datasociety Faculty Fellow @profmgilman quoted in @techreview https://www.technologyreview.com/ ... Karen Levy / @karen_ec_levy : .@profmgilman is a hero and mentor. This is a great piece describing her incredible work upholding the rights of poor people against adverse algorithmic decisions. https://twitter.com/... Karen Hao / @_karenhao : Since September, I've been speaking to civil lawyers around the country about their fight against a web of hidden algorithms that deny their clients basic services. I also spoke with a survivor of algorithmic harm. Her name is Miriam. This is their story. https://www.technologyreview.com/ ... Katharine Schwab / @kschwabable : “They're enmeshed in so many different algorithms that are barring them from basic services. And the clients may not be aware of that, because a lot of these systems are invisible.” Excellent story by @_KarenHao https://www.technologyreview.com/ ...
Context & Ripple Effects
This story closes a loop that opened years earlier: back in 2016, the Guardian reported that employers using algorithms to sift job applications were already filtering out poorer candidates at scale. Since then, scrutiny has spread through the criminal legal system — a game built on real defendant data exposed COMPAS's shortcomings, and the New York Times documented how predictive algorithms now set police patrols, prison sentences, and probation rules.
What changes with this piece is the actor: after years of journalists and researchers doing the uncovering, civil lawyers are formalizing the fight, targeting the automated systems that gate housing, jobs, and basic services for low-income Americans — the population Tonya Riley notes also has the weakest privacy protections against exactly the data those systems run on.
First-order effects
- Vendors and agencies operating benefit-, housing-, and employment-screening algorithms now face organized legal adversaries who know how to demand documentation of training data and decision logic rather than accepting denial letters at face value.
- Low-income applicants denied services by opaque systems gain a practical route to contest outcomes, converting individual rejections into test cases that can establish precedent.
Second-order effects
- Public agencies and landlords procuring screening software will be pushed toward vendors who can explain their models, since an unexplainable system is now a litigation liability rather than just a black box.
- The same playbook built against welfare and housing systems extends naturally to the criminal-legal tools already under fire — COMPAS-style risk scores and predictive policing deployments become next targets for civil suits.
Third-order effects
- If the pattern holds, litigation becomes the de facto audit mechanism for automated decision-making in the US, substituting case-by-case court pressure for the regulation that hasn't materialized — and raising the compliance bar for any vendor selling decision systems into government or essential services.
The trend: Adversarial civil litigation is emerging as America's default accountability layer for algorithms that ration housing, jobs, and public services.