Companies are adjusting premiums and policies for life, car, and home insurance based on new forms of surveillance enabled by tech to collect and analyze data
Some technologies are better left in the laboratory. — Ms. Jeong is a member of the editorial board. Tweets: @mozilla and @flexlibris Tweets: @mozilla : “As machine learning works its way into more and more decisions about who gets coverage and what it costs, discrimination becomes harder to spot,” writes @sarahjeong, in a sobering essay on artificial intelligence & insurance. http://www.nytimes.com/... Alison Macrina / @flexlibris : also very good is @sarahjeong's http://www.nytimes.com/... - here is a real problem that affects many ppl (insurance cos using AI to make decisions that are largely discriminatory) and how the response we hear most often (we'll make the algos less racist!) is actually not possible
Context & Ripple Effects
Sarah Jeong's editorial lands at the center of a widening debate over algorithmic gatekeeping: predictive algorithms already steer police patrols, prison sentences, and probation rules across the US and Europe, and insurance is now the next domain where machine learning decides who gets a basic service and at what price.
The coverage around it shows both sides mobilizing — US auto insurers are already running AI-generated repair claims from photos alone, while civil lawyers are building litigation strategies to push back on automated systems that deny the poor basic services. Jeong's core warning is that when ML sets premiums, discrimination becomes structurally harder to spot.
First-order effects
- Life, car, and home policyholders face premiums and policy terms recalculated from surveillance-derived data they did not explicitly consent to share, shifting cost onto behaviors and profiles rather than traditional actuarial categories.
- Insurers gain a new underwriting edge: continuous data collection lets them reprice risk dynamically, rewarding monitored customers and penalizing those who opt out of surveillance.
Second-order effects
- The lawyers developing anti-automation litigation strategies have a fresh target class — coverage denials and premium spikes traceable to opaque models — extending their docket beyond benefits and credit into insurance.
- As insurers deploy models trained on proxies for protected traits, the research community's alarm over commercial AI that detects race or ethnicity moves from market research into pricing, raising exposure to disparate-impact claims.
Third-order effects
- If the pattern holds, essential financial protections converge with policing and sentencing as domains run by predictive algorithms, forcing regulators to choose between model transparency mandates and accepting unaccountable pricing.
- Opting out of data collection may become a priced luxury, splitting the insured population into surveilled customers who pay less and privacy-preserving customers who subsidize them — a structural inequity regulators have not yet addressed.
The trend: Machine learning is moving from assisting human decisions to silently setting the terms of essential services like insurance, with accountability arriving through litigation faster than through regulation.