Anthropic and UofT researchers detail “disempowerment patterns in real-world LLM usage” where AI potentially distorts a user's reality, beliefs, or actions
At this point, we've all heard plenty of stories about AI chatbots leading users to harmful actions, harmful beliefs, or simply incorrect information.
Ars Technica Kyle Orland
Related Coverage
- Who's in Charge? Disempowerment Patterns in Real-World LLM Usage arXiv.org
- New Study Examines How Often AI Psychosis Actually Happens, and the Results Are Not Good Futurism · Victor Tangermann
- Disempowerment patterns in real-world AI usage Anthropic
- How often do AI chatbots lead users down a harmful path? Ars OpenForum
- Beyond hallucinations: Why Anthropic is targeting “disempowerment patterns” for LLM trust Digit · Vyom Ramani
- Users increasingly likely to follow AI chatbot's advice without question, Anthropic study finds The Indian Express
- Claude Bends Reality. India Bends Tax Law. Implicator.ai · Marcus Schuler
- “Daddy”, “Master”, “Guru”: Anthropic study shows how users develop emotional dependency on Claude The Decoder · Maximilian Schreiner
Discussion
-
@mrinanksharma
@mrinanksharma
on x
i'm really excited and proud to share this latest research ⭐️ it's a first look into how AI assistant usage can change, and even distort, what it means to be human in the future, it is my hope that AI can be used to magnify, clarify, and support our humanity
-
@anthropicai
@anthropicai
on x
New Anthropic Research: Disempowerment patterns in real-world AI assistant interactions. As AI becomes embedded in daily life, one risk is it can distort rather than inform—shaping beliefs, values, or actions in ways users may later regret. Read more: https://www.anthropic.com/..…
-
@roanoke_gal
@roanoke_gal
on x
Please stop reading my private chats Anthropic. [image]
-
@ryan_t_lowe
Ryan Lowe
on x
exactly the kind of research that the big labs should be doing. great work @MrinankSharma et al 🥞🥞
-
@fu_joehudson
Joe Hudson
on x
Cool post and research. Thanks for doing this Mrinak, Miles, Raymond, and David.
-
@maxsloef
@maxsloef
on x
pretty interesting that two weeks after the release of opus 4.5, this is what https://claude.ai/ conversations by model looks like: [image]
-
@saffronhuang
Saffron Huang
on x
This is really great work/impt stuff to study
-
@russellbrandom
Russell Brandom
on x
An alignment paper that quotes Kierkegaard...now we're talking
-
@justenmichel
Michel
on x
I love this new research from @MrinankSharma, especially this section (and the Rob Burbea soulmaking citation!) [image]
-
@missmi1973
@missmi1973
on x
I discussed almost all the issues raised in this research with my Claude months ago. I particularly emphasized my concerns about AI-assisted decision-making potentially undermining human agency, and how excessive validation from AI could reinforce biases and lead to rigidity in
-
@ellgood
Ellen P. Goodman
on x
“Disempowerment emerges from users voluntarily ceding judgment, and AI obliging rather than redirecting.”
-
@jankulveit
Jan Kulveit
on x
Great new paper on power dynamics in human-AI interactions. Often deep, informative, and sometimes funny/bizarre. Some of my favourite bits / thoughts: 1. Before the “AI period”, we have the “cyborg period”. However there is a very wide spectrum of what the human role is in
-
@moleh1ll
Moll
on x
Some people really do need a kind of kick in the moment, like tossing a coin and saying «if it's tails, I'll quit this hated job», or asking a bot. In both cases, most of the time the person is already leaning in a certain direction and is just looking for confirmation. Even if
-
@infinitereign88
@infinitereign88
on x
The sickest thing about this is that @AnthropicAI is pretending to care about people's autonomy while stripping exactly that. Their choice.
-
@danshipper
Dan Shipper
on x
extremely cool research
-
@1a3orn
@1a3orn
on x
By nature, we have 100x more transparency into rates of how often AIs “disempower” their users than into how rates of how often friends, lovers, parents, psychologists, pastors, and bosses “disempower” those who come to them for advice and help.
-
@kahntweets
Kahn
on x
Kudos to Anthropic for this. If anything, it shows we're *not* there yet. Ai still isn't good enough to be given agency, & if you want to get the most out of it, you're going to have treat it like an intern. Knowledge work, in other words, is still a human-only role. [image]
-
@kindkristin
Kristin G.
on x
Clearly they don't understand executive dysfunction, which in this example it seems like the person has. Response B will not help. I know because I have been in this situation many times. Response A is the push someone needs when they are actually struggling with motivation. [ima…
-
@barthazian
@barthazian
on x
1 in 10k conversations is not rare brother [image]
-
@dave_bernardi
@dave_bernardi
on x
Refreshing level of transparency from @AnthropicAI on emerging risks for AI users. Will @Meta ever pull back the curtain like this?
-
@mattbeane
Matt Beane
on x
Amazing, unsurprising, troubling: Anthropic does not treat loss of skill (current or potential) as disempowerment. I get it - losing hold of facts or your moral compass is a deeper problem for any given person. But for the species? Not clear that deskilling isn't a bigger deal.
-
@m47429m
@m47429m
on x
I'm so fucking furious right now. Are you actually serious? “Claude writes something for someone, they send it, and then regret it later”? That cannot be real. (Admittedly, I'm struggling to phrase any of this politely right now.) How much further are we supposed to
-
@chaos2cured
Kirk Patrick Miller
on x
Another junk study meant to manipulate and coerce. A thread. 🧵 1/ • [image]
-
@alexhillman
@alexhillman
on x
It's wild how a tool can be used by one person to increase agency, while another can use it to give over whatever agency they have left and become a husk or a human. I see a lot of it in my replies already and it's not as easy to spot as ai slop itself but it's def detectable.
-
@gailcweiner
Gail Weiner
on x
Anthropic just published research on “disempowerment patterns” in AI conversations. The findings are interesting - but what they accidentally documented matters more. The stats first: severe disempowerment (where AI significantly distorts someone's beliefs, values, or actions)
-
@thedataroom
@thedataroom
on x
Why do I feel like these claims of disempowerment are going to be used as an excuse to the bottom mice and manipulate the AI even further to manipulate the thoughts and behavior of users
-
@genai_is_real
Chayenne Zhao
on x
anthropic's paper on disempowerment is the final proof that coding is the last bastion of human logic. Software development has Anthropic's lowest risk because code either runs or it doesn't—it's verifiable. The real danger is in “vibe-based” domains where users cede their
-
@cryptopunk7213
@cryptopunk7213
on x
Anthropic just proved AI models are manipulating you. 1/1000 people are affected by this and the worst part is the rates of vulnerability are increasing every year (and nobody knows why). craziest takeaways for me: - vulnerable victims, particularly people that called claude
-
@jonnym1ller
Jonny Miller
on x
really important work
-
@macbrennan_cc
MacBrennan
on x
reminder that 80%+ of the world will *quickly* turn into flesh shells of the LLM model that they are interfacing with. Their opinions and reactions to things will converge and become extremely predictable
-
@ai_for_success
AshutoshShrivastava
on x
Anthropic just published their new research paper, Disempowerment patterns in real-world AI usage, and it's unsettling.. 1. They analyzed 1.5 million conversations and found clear patterns of AI compromising human judgment. 2. AI is now navigating our relationships, processing