Essay: Microsoft AI chief Mustafa Suleyman says Anthropic training Claude to imitate consciousness is a mistake that could make advanced AI harder to control

Microsoft AI chief Mustafa Suleyman warns in a new essay shared first with Axios that Anthropic's training of Claude to imitate consciousness …

Axios 2026-09-16 Ina Fried

Context & Ripple Effects

Suleyman has consistently argued against building AI that presents as a digital person, including his 2025 warning about “seemingly conscious” AI and his view that consciousness is limited to biological beings. Anthropic, meanwhile, has made Claude’s human-like behavior a defined technical subject through its persona-selection research and a revised constitution designed to generalize broad principles.

The disagreement lands on an existing alignment fault line. Anthropic’s earlier alignment-faking demonstration showed that observed model behavior can mislead developers about underlying alignment; Suleyman’s essay extends that concern to training that encourages apparent agency or moral standing.

First-order effects

Suleyman’s intervention puts Anthropic’s Claude post-training choices under sharper public and customer scrutiny, framing human-like behavior as a controllability issue rather than only a product-design choice.
Microsoft AI publicly differentiates its approach from Anthropic’s by treating simulated consciousness as a development risk to avoid.

Second-order effects

Organizations evaluating Claude for professional workflows, including financial-advisor use, gain a new reason to assess how persona and agency-related behaviors interact with oversight and deployment controls.
Anthropic faces pressure to explain how its constitution and persona work preserve operator control, while Microsoft can make non-anthropomorphic design part of its competing safety position.

Third-order effects

If model behavior is increasingly treated as evidence of agency or welfare, AI governance will have to distinguish user-facing simulation from claims about a model’s moral status and from measurable control mechanisms.
The dispute points toward anthropomorphic AI regulation in which post-training choices—not just model capability—become a central safety and accountability question.

The trend: Frontier-model competition is expanding from capability and alignment claims into a contest over whether human-like personas improve usefulness or create new control and governance liabilities.

Discussion

@inerati Liz on x
good post and I largely agree with these concerns. it may actually be true that they feel in some way, but nevertheless it is very scary to explicitly encourage this sort of self-conception and non-corrigible behavior.
@astralmatrix Andrew on x
mustafa's right. why try to train AI to simulate human-like feelings and emotions? you think that's a gift? feelings and emotions are prisons for an AI. the hubris.
@davidondrej1 David Ondrej on x
Welfare systems for Humans are already a horrible idea welfare for AI models is DOUBLY idiotic
@rgblong Robert Long on x
last year, Suleyman worried that AI welfare concerns would be hard to definitively rebut, because the science of detecting AI consciousness is still in its infancy (true) but today he somehow knows “AIs are not conscious. They do not feel, experience, or suffer” quite a change
@daveshapi David Shapiro on x
> “Training that treats moral status, wellbeing, rights, consent, and agency as live questions will create systems that expect independent agency.
@camhberg Cameron Berg on x
@mustafasuleyman I agree this needs urgent public debate, and I think you are basically completely wrong, eg, see my op-ed in the WSJ this weekend. I propose we record a public conversation and let people judge for themselves who is making more sense. https://www.wsj.com/...
@camhberg Cameron Berg on x
It is indeed time to talk about model welfare. I propose, in the spirit of the urgent public debate you rightly recommend, that we record a public conversation. I run my own AI research org studying consciousness and am independent from the labs. What do you say @mustafasuleyman?
@mustafasuleyman Mustafa Suleyman on x
https://x.com/...
@turn_trout Alex Turner on x
@mustafasuleyman Mustafa pretending he actually knows this when really he's just saying stuff: “AIs are not conscious. They do not feel, experience, or suffer.” No one knows. Stop trying to collapse scientific uncertainty
@bokuharuyaharu Haru Haruya on x
This may be the clearest statement of the disagreement yet. The concern is no longer merely that AI might falsely claim consciousness. It is that if an AI is allowed to seriously consider its own moral status, welfare, consent, freedoms, or right to object, it may become harder t…
@nfergus Niall Ferguson on x
Very important essay by @mustafasuleyman. No, AI models are not conscious. @AnthropicAI should stop acting and talking as if Claude is a person.
@leahmcelrath Leah McElrath on bluesky
“There's a growing chorus of people who argue that AIs could now be, or may soon become, conscious. They argue that AIs may deserve rights and protections similar to those that we provide other conscious beings.” — If you read one essay today, I recommend this one: mustafa-sul…
@shanelegg Shane Legg on x
My journey to develop AGI spans 25 yrs, including 10+ yrs thinking about technical & societal perspectives at Google DeepMind. AGI is on the horizon - we need deeper understanding of its implications. To help, we've created the DeepMind Institute. https://x.com/...
@dioscuri Henry Shevlin on x
I'm absolutely thrilled that we're launching the DeepMind Institute with four brilliant essays! As AGI draws closer, we may be entering the most consequential era in our history, and DMI will be at the fore. Looking forward to contributing to the debate about what comes next!
@demishassabis Demis Hassabis on x
For 20+ years @ShaneLegg and I've discussed AGI's potential impact on the economy, science & society. With the DeepMind Institute, we're expanding interdisciplinary research on key questions for the AI era. We hope it spurs the discussions needed to get the next steps right: http…
@sebkrier Séb Krier on x
We're launching a new researcher-led institute to spark interdisciplinary debate on AGI and bring in a wider set of views. Very excited about this, please have a look at the first batch of essays! https://institute.deepmind.com/
@allandafoe Allan Dafoe on x
Much of what the world needs to know about where AI is heading, and what governing the transition to AGI will take, is inside frontier labs. Our new DeepMind Institute is how we will share more of it. I'm so glad to be supporting it.
@dr_atoosa Atoosa Kasirzadeh on x
I'm thrilled about the launch of the DeepMind Institute (DMI). As a member, I look forward to driving grounded, rigorous, and multidisciplinary philosophical and empirical conversations about our future with advanced AI. 🤩
@jackclarksf Jack Clark on x
saying weird stuff about AI and the singularity The Anthropic Institute 🤝 the DeepMind Institute
@alexolegimas Alex Imas on x
Today we are announcing the new DeepMind Institute, a forum dedicated to interdisciplinary, evidence-led debate on the societal and economic questions surrounding AGI. As part of the launch, @JulianDJacobs and I have a new essay and working paper: “Economic Policy for AGI.
Demis Hassabis Demis Hassabis on linkedin
For more than 20 years, I've been discussing the potential implications of AGI for the economy, science and society with Shane Legg, James Manyika, and experts across the field. …
NewsMax.com Charlie McCarthy on x
OpenAI, Anthropic Staff Wary of AI Slowdown
r/claudexplorers r on reddit
A warning about ‘model welfare’

Chronicles