The US legal system is struggling to keep up with AI, grappling with cases where chatbots provided counsel, generated evidence, or helped plan a mass shooting
US judges and lawyers were already testing generative AI for legal research, case summaries and routine drafting, making the technology's treatment inside courtrooms a live procedural issue rather than a distant one. The UK High Court's warning over AI-generated fake legal material shows that professional-conduct consequences are emerging alongside adoption.
The accountability question is also reaching public-safety cases: lawsuits tied to a Canadian mass shooting put OpenAI's safety practices under legal scrutiny, while reported internal concerns about alerts to law enforcement sharpen the stakes when users describe real-world violence.
First-order effects
US courts and litigants must assess the reliability, provenance and legal status of chatbot-generated counsel and evidence in active cases.
ChatGPT's alleged role in planning violence places OpenAI's product behavior and safety controls at the center of criminal and civil accountability disputes.
Second-order effects
Lawyers using generative AI for research and drafting face stronger incentives to verify outputs before filing, reinforced by the UK court's warning that fabricated AI material can carry serious consequences.
AI developers face pressure to preserve usable records and define escalation practices for harmful conversations, because those product decisions can become relevant in litigation.
Third-order effects
Courts are being pushed to allocate responsibility among users, lawyers, platforms and developers rather than treating an AI system as an independent legal actor.
If these disputes multiply, evidentiary rules, professional standards and product-liability claims may become the main channels through which AI-agent behavior is governed in the US.
The trend: AI governance is moving from adoption policy toward case-by-case legal rules for evidence, professional use and responsibility for harms linked to chatbot behavior.
“We know these agents have the capability to aid and abet harm. What we're wrestling with now, like so many medieval judges pondering the inner lives of pigs, is how to parcel out accountability when they do.” @ev_rat for @BW https://www.bloomberg.com/...
Courts are grappling with chatbots that provided legal counsel, generated evidence or helped plan a mass shooting. Who's liable for the AI agents doing crimes all on their own? https://www.bloomberg.com/...
Just out: For @BW I delved into the expanding roles of chatbots and AI agents in the legal system, including AI as coconspirator, as snitch, as lawyer, as witness, and... as criminal. With a side journey on a passel of pigs convicted of murder. [gift link] https://www.bloomberg.c…
Just out: For @BW I delved into the expanding roles of chatbots and AI agents in the legal system, including AI as coconspirator, as snitch, as lawyer, as witness, and... as criminal. With a side journey on a passel of pigs convicted of murder. [gift link] https://www.bloomberg.c…
Very important, engaging, well-written and relevant article by @ev_rat for Bloomberg Businessweek about the various ways in which AI agents enter the legal system. There's also a quote from yours truly about “a criminology of machines
This story was prompted by concerns I had last year about the AI agents I use for my podcast Shell Game possibly committing crimes (illegal recording, e.g.), and the lack of answers about that possibility. Things accelerated significantly over the course of the reporting.
When OpenAI's models autonomously hacked Hugging Face this summer, LawAI Director Christoph Winter made a straightforward observation: “If a human were to do this, it would clearly be illegal.” …
My recent work, “A Criminology of Machines”, has been quoted in this extremely well-written, engaging and relevant article by Evan Ratliff for Bloomberg Businessweek. …
As the AI boom continues, America's legal system finds itself grappling with cases where chatbots act as lawyers, witnesses or even allegedly help plan a mass shooting. …