Many hospitals and clinics are introducing AI-powered decision support tools, often novel and unproven, but most patients are unaware of them
Since February of last year, tens of thousands of patients hospitalized at one of Minnesota's largest health systems have had their discharge … Tweets: @rebeccadrobbins , @statnews , @kncukier , @katecrawford , @mkaplanpmp , and @covidblk Tweets: Rebecca Robbins / @rebeccadrobbins : NEW: A consensus is emerging among doctors: Patients need not be told about the AI models advising their care. Docs say AI is one among many decision aids, and that mentioning it may derail a bedside conversation. But is that the right call? With @erbrod: https://www.statnews.com/... Stat / @statnews : At a growing number of hospitals, clinicians use AI-powered tools to predict whether patients will deteriorate or even whether they're likely to die soon. Many are unproven — and often, families aren't told about the tools, a STAT examination found. https://www.statnews.com/... Kenneth Cukier / @kncukier : Excellently reported piece by @rebeccadrobbins and @erbrod, with absolutely crucial points (ie, performance disclosure). But I detest the spooky, anti-AI framing of the article overall. Just imagine it were 1960 and “AI” was replaced with “computers” ... or 1760 and “math”! https://twitter.com/... Kate Crawford / @katecrawford : '...trust in AI and machine learning could plummet if patients “were to find out, after the fact, that there's a rash of this being used without anyone ever telling them.” “That's a scary thing if you think this is the way the future is going to go.” https://www.statnews.com/... Michael A. Kaplan / @mkaplanpmp : “I don't monitor how doctors do their jobs. I just trust that they're doing it well.” ~Katy Granger https://www.statnews.com/... Covidblack / @covidblk : Especially in light of the role that AI and other predictive technologies play within judicial systems, it will be important for healthcare institutions to be transparent in their use of these tools and for patients to be knowledgeable about their influence and efficacy. https://twitter.com/...
Context & Ripple Effects
The disclosure question lands mid-arc in a debate that has been building since early 2020, when reporting on Google's mammography system flagged that clinical AI could amplify overtesting and overdiagnosis rather than fix them. By 2022, hospitals were past piloting: health systems had adopted predictive models to rank ER and ICU patients by risk (Wall Street Journal), even as analysis showed medical data is scarcer and messier than web data, making results misleading more often than marketed.
What changed by the time of this STAT piece is that deployment outpaced candor: a consensus among doctors holds that patients need not be told an AI model advised their care. That position now looks harder to defend given what came after — reporting that some clinicians feel pressure from administrations to defer to sometimes-flawed diagnostic algorithms.
First-order effects
- Patients at large systems like the unnamed Minnesota health network have discharge and triage decisions shaped by novel, clinically unproven models they never learn about, so informed consent effectively excludes part of the decision chain.
- Doctors gain a quieter bedside: treating AI as one tool among many lets them avoid conversations about algorithmic input that they fear would derail care discussions.
Second-order effects
- Vendors selling decision-support tools benefit from a market where no patient-facing disclosure is expected, lowering the scrutiny hospitals apply before adoption at exactly the moment clinicians report being pressured to defer to the algorithm.
- When a model's errors surface — as the mammography and diagnosis-tool coverage suggests they will — hospitals that skipped disclosure inherit a trust deficit on top of the clinical harm, turning an undisclosed tool into a liability story.
Third-order effects
- If undisclosed AI becomes standard inside clinical workflows, oversight shifts entirely to hospital administrators and regulators, because the party with standing to object — the patient — is structurally removed from the loop.
- The pattern points toward formal disclosure standards or rules for clinical AI, the same governance gap already visible where flawed tools reach diagnosis without independent validation.
The trend: Clinical AI is embedding itself into hospital decision-making faster than consent norms or validation requirements can form, leaving disclosure to be settled after deployment rather than before it.