Mustafa Suleyman says “Seemingly Conscious AI” is coming but is something to avoid, and that developers must build AI for people, not to be a digital person
Seemingly Conscious AI is Coming — I write, to think. More than anything this essay is an attempt to think through …
Mustafa Suleyman
Context & Ripple Effects
Suleyman has paired commercialization with a stated goal of “humanist superintelligence,” framing advanced systems around solving social problems rather than becoming ends in themselves. This essay sharpens that product philosophy by drawing a boundary around systems designed to present as persons.
Suleyman gives developers and product teams a clear design preference: optimize AI as an instrument for users, rather than for cues that encourage people to treat it as a digital person.
The intervention raises the bar for AI products marketed through humanlike presence or emotional realism, especially where apparent agency could be mistaken for inner experience.
Second-order effects
Companion-AI developers may face greater pressure to distinguish useful conversational interfaces from products that cultivate perceived personhood; this makes Suleyman’s humanist-superintelligence framing a more concrete product-positioning constraint.
The debate shifts from whether models can convincingly simulate consciousness to how builders should manage user interpretation, bringing safety, UX, and disclosure choices into the same product decision.
Third-order effects
If leading labs adopt this distinction, AI governance could increasingly target anthropomorphic design and relational dependency rather than trying to settle the unresolved question of machine consciousness.
The durable split may be between AI designed to extend human judgment and AI designed to substitute for human relationships—a structural choice with consequences for product norms and oversight.
The trend: AI builders are moving from abstract arguments about machine consciousness toward governance of how humanlike systems are designed, marketed, and interpreted.
FFS. 🤦 LLM are useful tools. But they are very very far from being “conscious”. They have a limited context window, don't update the model based on past experience, can't operate for too long before becoming confused and suffering from context poisoning. [embedded post]
Looks like there was a “pump the brakes on the AI hype” group chat this weekend. Microsoft's AI chief Mustafa Suleyman warns against building AI that seems conscious, saying it fuels “AI psychosis.” — He argues the industry should stick to assistants, not AGI. — Altman, Schm…
Behind the curtain of the “Great and Powerful AI” is an army of programmers frantically pulling levers and turning knobs to trick users into believing AI models are conscious? — mustafa-suleyman.ai/seemingly- co...
The premise is close enough but the main issue is about all LLMs being trained to mimic humans. That's the business incentive. — mustafa-suleyman.ai/seemingly- co...
SCAI deserves our immediate attention. AI development accelerates by the month, week, day. I write this to instill a sense of urgency and open up the conversation as soon as possible. So let's start - weigh in in the comments. — Full essay: mustafa-suleyman.ai/seemingly- co.…
“We must build AI for people; not to be a person”—> — “Moments of disruption break the illusion, experiences that gently remind users of its limitations and boundaries. These need to be explicitly defined and engineered in...” — mustafa-suleyman.ai/seemingly- co... HT @tsee…
Why it matters: to be clear, there's zero evidence of AI consciousness today. But if people just perceive it as conscious, they will believe that perception as reality. Even if the consciousness itself is not real, the social impacts certainly are. 3/
The machine doesn't need to think. It just needs to behave in ways we've learned to trust. That's what's new this time. Not the illusion, we've lived through that before. It's the scale, the persistence, the convenience. Imitation that used to be a spectacle, is now a service
Sorry for the super long crop, but basically in the first part, Mustafa outlines some things that'll create “seemingly conscious AI”, but then he dismisses it by saying that it won't be so, and links to biological naturalist papers to support it. https://mustafa-suleyman.ai/ ...
Reports of delusions, “AI psychosis,” and unhealthy attachment keep rising. And as hard as it may be to hear, this is not something confined to people already at-risk of mental health issues. Dismissing these as fringe cases only help them continue. 5/
What I call Seemingly Conscious AI has been keeping me up at night - so let's talk about it. What it is, why I'm worried, why it matters, and why thinking about this can lead to a better vision for AI. One thing is clear: doing nothing isn't an option. 1/
in a _slightly_ similar vein, eric schmidt coauthored a nyt op-ed that claims that american AI labs are too obsessed with pursuing AGI https://www.nytimes.com/... [image]
this mustafa suleyman blog is SO interesting — i'm not sure i've seen an AI leader write such strong opinions *against* model welfare, machine consciousness etc https://mustafa-suleyman.ai/ ... [image]