Ilya Sutskever, Daniel Gross, and Daniel Levy announce Safe Superintelligence, a US startup “with one goal and one product: a safe superintelligence”

Superintelligence is within reach. Building safe superintelligence (SSI) is the most important technical problem of our time.

Safe Superintelligence Inc. 2024-06-20

Context & Ripple Effects

Safe Superintelligence separates safety from general-purpose AI product development by making it the company’s sole stated objective. Its founders’ framing was clarified the same day as a focus on safety closer to nuclear safety than content moderation.

The announcement became the starting point for a capital-intensive independent lab: SSI later raised $1 billion at a reported $5 billion valuation, showing that investors were willing to fund a narrowly defined safety-and-capability mission before a public product launch.

First-order effects

Sutskever, Gross, and Levy create a new US AI lab whose stated product is safe superintelligence, concentrating the founders’ technical and organizational effort on that single objective.
SSI establishes a differentiated public position: safety is presented as a core technical requirement for advanced AI rather than a downstream trust-and-safety function.

Second-order effects

The startup becomes a new destination for AI researchers and investors who want to back frontier-model work paired explicitly with technical safety, competing with broader AI labs for scarce talent and financing.
Other frontier labs face greater pressure to explain whether their safety work is integral to model development or separate from commercial product deployment.

Third-order effects

If SSI’s model attracts sustained capital and talent, AI-lab competition may increasingly be organized around governance and safety architectures as well as model performance and product distribution.
The move points toward a more segmented frontier-AI market, with specialized labs seeking strategic legitimacy through narrowly articulated missions; whether that produces safer systems depends on execution rather than branding alone.

The trend: Frontier AI development is spawning specialized, heavily funded labs that make technical safety a central institutional mandate rather than an adjacent policy function.

Discussion

@sung.kim.mw Sung Kim on threads
I don't know about this. There is a reason why OpenAI pivoted. They were running out of money. How is SSI any different? Maybe Israel's government is funding them since the Israeli companies have been a no show in AI race, which will be problematic based on their recent track…
@bloombergbusiness @bloombergbusiness on threads
“It will be fully insulated from the outside pressures of having to deal with a large and complicated product and having to be stuck in a competitive rat race,” Sutskever says in an exclusive interview about his latest venture called Safe Superintelligence. …
@brianckeegan.com Brian C. Keegan on bluesky
They are going to raise nine figures from FOMO investors who think that a spicy autocomplete trained on stolen and poisoned data can become omnipotent with just one more parameter, bro. [embedded post]
@ernie.tedium.co Ernie Smith on bluesky
They have to ensure it's safe because it's in the name, just like how OpenAI offers open-source products because they called themselves OpenAI. [embedded post]
@marypcbuk.bsky.social Mary Branscombe on bluesky
Slightly less OpenAI [embedded post]
@niedermeyer.io E.W. Niedermeyer on bluesky
I hope they create a superintelligence that gives them a wedgie and shoves them into a locker [embedded post]
@ilyasut Ilya Sutskever on x
I am starting a new company:
@ssi @ssi on x
Superintelligence is within reach. Building safe superintelligence (SSI) is the most important technical problem of our time. We've started the world's first straight-shot SSI lab, with one goal and one product: a safe superintelligence. It's called Safe Superintelligence
@ilyasut Ilya Sutskever on x
We will pursue safe superintelligence in a straight shot, with one focus, one goal, and one product. We will do it through revolutionary breakthroughs produced by a small cracked team. Join us: https://ssi.inc/
@daniellevy__ Daniel Levy on x
Beyond excited to be starting this company with Ilya and DG! I can't imagine working on anything else at this point in human history. If you feel the same and want to work in a small, cracked, high-trust team that will produce miracles, please reach out.
@zerohedge @zerohedge on x
Sutskever declines to name Safe Superintelligence's financial backers or disclose how much he's raised. https://www.bloomberg.com/...
@pitdesi Sheel Mohnot on x
What do you think the seed round for Safe Super Intelligence looks like? Safe Super Intelligence is their mission, name, and entire product roadmap. Ilya was Chief Scientist at OpenAI. Daniel Gross led Apple's AI efforts Daniel Levy worked alongside Ilya at OpenAI
@emostaque Emad on x
My initial thoughts 1. This will get unlimited funding from DoD 2. This will be the number target for AI PhD spies 3. Given OpenAI & Stability not a good idea to call it Safe. They should have gone for Unsafe. Would also make it edgier.
@emostaque Emad on x
@packyM The business model is they build ASI and either 1. Ask it for the best business model Or 2. It ends all money and is the business model Straight line
@packym Packy McCormick on x
I am dying to know what this means: “Our team, investors, and business model are all aligned to achieve SSI.”
@liron Liron Shapira on x
📎 Hey Ilya, I notice you're desperately trying to save the world from the efforts of @sama, @ylecun, etc, via a plan that depends on racing past their safety and capabilities simultaneously. Given the stakes, why not align words & actions? Why not go public with your criticism?
@mgsiegler M.G. Siegler on x
I'm reminded of Coca-Cola Classic. Safe Superintelligence sounds a lot like ‘OpenAI Original’. https://spyglass.org/...
@autismcapital @autismcapital on x
So the company is about building a cucked AI, it's called SSRI (almost), and the HQ is in Palo Alto and Tel Aviv 😂 The internet is going to love this one 😂
@abidlabs Abubakar Abid on x
Best HN comment. Ever [Screenshot: “based on the naming conventions established by OpenAl and StabilityAI, this may be the most dangerous AI company yet"]
@aravsrinivas Aravind Srinivas on x
The king is back. Exciting!
@garrisonlovely Garrison Lovely on x
Ilya's launching a new company with an ex-Apple AI exec and the optimization team lead from OpenAI. Their website looks like a Craigslist ad lol. A good way to communicate that you care about one thing and one thing only! [image]
@alexlangshur Alex Langshur on x
“safe ai” is quickly become a meaningless platitude. what is “safe”? how do you build “safe” systems? no one has provided a rigid technical definition of “safe” in this context, and whenever people try, they quickly get swept up in the winds of capitalism.
@nathanpmyoung Nathan on x
How AGI companies proliferate: [image]
@rustybrick Barry Schwartz on x
Oooo - Super Safe AI from former OpenAI lead
@pdhsu Patrick Hsu on x
What is Ilya doing next? Check out Safe Superintelligence from @ilyasut, @danielgross, and @daniellevy__ Here's a good morning read for inspiration [image]
@danielgross Daniel Gross on x
It is a great pleasure and honor to cofound this new endeavor with @ilyasut and @daniellevy__:
r/LocalLLaMA r on reddit
Ilya Sutskever starting a new company Safe Superintelligence Inc
r/MachineLearning r on reddit
[N] Ilya Sutskever and friends launch Safe Superintelligence Inc.
@rachelmetz Rachel Metz on x
Ashlee Vance with the scoop on where @ilyasut has been this whole time: working on a new startup called Safe Superintelligence. I have so many questions lol.
@aphysicist Aaron Slodov on x
“Sutskever declines to name Safe Superintelligence's financial backers or disclose how much he's raised.” cool, totally putting my trust in this project
@ashleevance Ashlee Vance on x
.@ilyasut is back, friends. His new company is called Safe Superintelligence Inc. and, well, he's going for the whole enchilada. Straight shot. No near-term products. Superintelligence or bust. Exclusive here https://www.bloomberg.com/...
@dhaval_shroff Dhaval Shroff on x
The first one to exit the AI rat race and build in peace. Hoping to see a completely new take with the “lean, cracked team” :D
@sharongoldman Sharon Goldman on x
AGI or bust, says @ilyasut. Why bother with silly products for consumers or enterprises along the way? 🤔
@then_there_was Andrew Ruiz on x
Interesting. Ilya now working at a research lab that won't launch anything short of safely aligned superintelligence. Daniel Gross and Daniel Levy are cofounders. [image]
r/LocalLLaMA r on reddit
a deep dive into Ilya's SSI
r/singularity r on reddit
The system he [Sutskever] wants to pursue would be more general-purpose and expansive in its abilities. “You're talking about a giant super data center that's autonomously developing technology”.
r/technology r on reddit
OpenAI co-founder Ilya Sutskever starts a new company SSI (safe superintelligence)
r/mlscaling r on reddit
Ilya Sutskever launches ‘Safe Superintelligence’, a new startup to race for AGI by scaling LLMs

Chronicles