Some US hospitals test if GPT-3 can cut the time staff spend replying to online queries; a study claims the first ChatGPT version replied better than doctors
Pilot program aims to see if AI will cut time that medical staff spend replying to online inquiries
The appeal is administrative relief and potentially more responsive communication, but the corpus also records harmful, inaccurate, race-based medical AI responses, making review and deployment boundaries central to the workflow.
First-order effects
Participating hospitals can test whether GPT-3 drafts reduce the staff time required for online patient inquiries, while clinicians and support staff remain the immediate users judging its usefulness.
The study’s comparison of early ChatGPT replies with doctors’ replies gives hospital teams a new quality benchmark for patient-message drafting, not evidence that clinical accountability shifts to the model.
Second-order effects
Patient-portal and health-record vendors face pressure to put generative drafting directly into messaging workflows, as later MyChart adoption illustrates.
Hospitals that use these tools will need review practices that preserve speed gains while catching inaccurate or harmful language, particularly in sensitive patient communications.
Third-order effects
If drafting becomes routine, patient communication may become a core workflow-native AI category: value will accrue to systems that combine models with clinical context, permissions, and accountable human review.
The recurring tension between helpful communication and unsafe medical output is likely to keep adoption oriented around assistance rather than autonomous patient guidance unless reliability improves materially.
The trend: Healthcare is moving generative AI from general chat interfaces into clinician-supervised, workflow-embedded communication tools.
I'm not disputing the findings of this study per se, just noting that random doctors answering questions in their spare time on /r/AskDocs might not be the kind of care standard most people are experiencing in their daily lives so we shouldn't read too much into this https://twit…
@AlacrityMD @FiSurgi The software is more empathetic than the doctors according to this data. I'm not here to defend the software, and have no dog in the fight, but find the results fascinating in these studies. https://twitter.com/...
Researchers pittted ChatGPT against genuine doctors, using actual patient questions. ChatGPT was more accurate and more empathetic. By a lot. HT @adamcifu https://jamanetwork.com/... https://twitter.com/...
New study! We compared ChatGPT responses to people's medical questions with those of doctors. Healthcare professionals preferred ChatGPT 79% of the time; as more empathetic and higher quality. I'm excited to figure out how to use LLMs to help doctors! https://jamanetwork.com/...
LLMs Preferred Over Human Physicians When Answering Patients' Questions -Team of licensed health care professionals compared physician's & LLM's responses -LLM preferred over human physician 79% of time -LLMs rated much higher for both quality & empathy https://jamanetwork.com/..…
Chatbots can't make docs empathetic, but they can help frame empathetic responses. Interesting findings bc physicians here weren't burned out at work but were voluntarily engaging w/ pts on Reddit! Nice work by @DaveySmithMD @CALonghurst and team. https://jamanetwork.com/... http…
I'd rather have a quick cure from a terse fellow of few words than a long empathetic monologue, generated by a machine, which leaves me in no better condition than when the interaction began. https://twitter.com/...
AI transforming knowledge work from creating content to reviewing content created by AI has already hit the medical field. It also turns out that AI can be programmed to have a friendlier attitude than the typical doctor. AI will reshape knowledge work. https://www.wsj.com/...
ChatGPT is more “empathetic” than real doctors when it comes to providing diagnosis and answering medical questions. I mean ofc, “empathy” is emotionally costly for humans but “free” for machines. Doctors that are overworked are too busy for pleasantries. https://www.wsj.com/... …
Highlighting the absurdity of “Science (TM)”: The study authors evaluate the empathy of answers from r/AskDocs and chatGPT. The results section provides confidence intervals. https://jamanetwork.com/...
The LLMs are coming for medicine - very preliminary work but unsettling: Chatbot responses were rated of significantly higher quality and more empathetic than physician responses https://jamanetwork.com/...
This study is extraordinary, and not in a good way. In short, ChatGPT answers to patient questions are longer and rated more empathetic than answers on Reddit.🤨 The moral panic over artificial intelligence raises questions about human intelligence ... https://jamanetwork.com/...
Overall, patients preferred the chatbot over the physician 79% of the time. From the study: “The chatbot responses were preferred over physician responses and rated significantly higher for both quality and empathy.” https://twitter.com/...
“....the evaluators gave “good” or “very good” ratings to four times as many ChatGPT responses as physician posts...4.6% of doctors' posts were graded as “empathetic” or “very empathetic,” compared with 45%—10 times as many—ChatGPT posts.” Amazing https://www.wsj.com/...
I do find it amusing this study in @JAMAInternalMed comparing Chatbot to physician responses to medical questions looked at level of empathy but not accuracy of responses Idk but I think most people would take accurate over empathetic medical advice?🤷♀ ️ https://jamanetwork.com/…
ChatGPT scored better than real doctors at responding to patient queries posted online, according to a study published Friday in the journal JAMA Internal Medicine, in which a panel of doctors did blind evaluations of posts. https://www.wsj.com/... #cdnpoli #bcpoli
What an interesting trial! Compare physician and ChatAI responses to patient on-line questions: Accuracy and empathy far higher with ChatAI as assessed by physicians. Future of medical communication? Future of medicine? #empathy #physician #ChatAI https://jamanetwork.com/...
“Without knowing who wrote which, the evaluators gave “good” or “very good” ratings to four times as many ChatGPT responses as physician posts.” https://www.wsj.com/...
UC San Diego Health's Dr. Marlene Millen (@marlene_mmm) offers great insights to the Wall Street Journal on using AI to answer patient questions: https://www.wsj.com/...
Researchers compared doctors' responses to patient queries with ChatGPT's responses. ChatGPT's responses were more accurate and more empathetic. A lot more. Wow. https://jamanetwork.com/... https://twitter.com/...