OpenAI's o3-pro is much smarter than o3 and amazing at using tools, but the model requires a lot of context to run well and without enough it tends to overthink

OpenAI dropped o3 pricing 80% today and launched o3-pro. Ben Hylak of Raindrop.ai returns with the world's first early review.

Latent.Space 2025-06-11 Ben Hylak

Context & Ripple Effects

OpenAI’s reasoning-model line moved from the planned o3/o3-mini rollout to a faster, lower-cost o3-mini and then to evidence that o3 could handle demanding multi-step work. This early assessment adds operational detail: capability depends materially on how much task context the model receives.

The review lands alongside o3-pro’s release to ChatGPT and API users and an 80% o3 price reduction with a flex processing option. Together, those moves make model selection less about a single benchmark and more about matching capability, latency, context, and cost to a workflow.

First-order effects

Teams evaluating o3-pro gain a model reported to be stronger than o3 at tool use, but must provide sufficient context to avoid inefficient overthinking.
OpenAI now offers a cheaper o3 alongside a higher-priced o3-pro tier, giving API and ChatGPT customers a clearer performance-versus-cost choice.

Second-order effects

Application builders will need to treat context packaging as part of product design: incomplete task state can reduce the practical value of a more capable reasoning model.
The lower o3 price raises pressure to compare models on cost per completed workflow, while o3-pro’s tool-use strengths may justify its premium only on tasks where richer context is available.

Third-order effects

If this pattern persists, reasoning-model competition will shift from headline intelligence toward workflow economics: reliable context retrieval, tool orchestration, and spend controls will determine useful performance.
Model portfolios may become more segmented, with lower-cost models serving routine work and context-rich reasoning models reserved for complex tasks rather than deployed as universal defaults.

The trend: Reasoning AI is evolving into a workflow-optimized market where context quality and cost per useful task matter as much as raw model capability.

Discussion

@sama Sam Altman on x
i like this take: “The plan o3 gave us was plausible, reasonable; but the plan o3 Pro gave us was specific and rooted enough that it actually changed how we are thinking about our future.”
@latentspacepod @latentspacepod on x
https://www.latent.space/... In order to “Feel the AGI”, we have to “Feed the AGI”. full article:
@ronbodkin Ron Bodkin on x
Interesting review of @openai o3-pro - I think it's even more reason to build specialized agents - to get value out of more sophisticated models you need to automate collection of context and analysis that leverage the data. Reports is the way they suggest using it but I think a
@latentspacepod @latentspacepod on x
🆕It's @openai o3-pro launch day! our high taste guest tester @benhylak has been previewing for the past week and found an interesting pattern: (link in reply) o3-pro doesn't noticeably outperform in normal situations, but it's just really, really, REALLY good at consuming ALL [im…
@benhylak Ben on x
god is hungry for context. read my thoughts from testing o3-pro for the last week below:
r/singularity r on reddit
First review of O3 pro
@sama Sam Altman on x
we are going to take a little more time with our open-weights model, i.e. expect it later this summer but not june. our research team did something unexpected and quite amazing and we think it will be very very worth the wait, but needs a bit longer.
@yuchenj_uw Yuchen Jin on x
o3-pro is the slowest and most overthinking model. A simple ‘Hi’ cost me $80. 🥲 [image]
@openai @openai on x
In expert evaluations, reviewers consistently prefer OpenAI o3-pro over o3, highlighting its improved performance in key domains—including science, education, programming, data analysis, and writing. Reviewers also rated o3-pro consistently higher for clarity, comprehensiveness, …
@openaidevs @openaidevs on x
o3-pro: Input: $20 / 1M tokens Output: $80 / 1M tokens (87% cheaper than o1-pro!) https://platform.openai.com/ ... We recommend using background mode with o3-pro: long-running tasks will be kicked off asynchronously, preventing timeouts. https://platform.openai.com/ ... [image]
@wesrothmoney Wes Roth on x
o3 pro one-shotted the Tower of Hanoi 10 disk problem (one of the more contested problems in Apple's “The Illusion of Thinking” paper) [video]
@caleb_friesen2 Caleb on x
o3-pro didn't get it either [image]
@morqon Morgan on x
first review of o3-pro “it's smarter. much smarter.” [image]
@afinetheorem Kevin A. Bryan on x
o3-pro: it thinks for a *long* time, should be given very very very long and specific instructions, and is very effective on tough problems (better than any we've seen). But of course, even o3 after 15 minutes can't escape overtraining on this modification of the old riddle! [ima…
@kosa12matyas @kosa12matyas on x
o3-Pro High performs WORSE than o3-High on ARC-AGI 1 and 2 Thinking too much makes you insane, think about that. [image]
@deryatr_ Derya Unutmaz on x
I have many mind blowing examples of o3-pro outputs, but let me quickly share one. In this instance, I've been working with o3-pro to develop immune system 2.0, a modestly ambitious attempt to completely reengineer our immune system 😆 I first asked o3-pro to identify key [image]
@thexeophon @thexeophon on x
interesting o3 pro vs deep research [image]
@nicdunz Nic on x
o3-pro solves surgeon misguided attention puzzle 100% of the time [image]
@dioscuri Henry Shevlin on x
I'd give o3-pro 8.5/10 on BaldurBench. The class build it gave was very strong, fully legal, and patch 8 compliant. Lost a few points for minor/superficial hallucinations and not flagging some optimisation options (eg Ethel's hair). Very very impressive. https://chatgpt.com/... […
@emollick Ethan Mollick on x
Been playing with o3-pro for a bit. It is quite smart. One problem it solved where every other model has failed is making word ladder from SPACE to EARTH. (Probably not contamination: the answer is different than the only online answer, which is for EARTH to SPACE in any case) [i…
@miles_brundage Miles Brundage on x
o3 pro is very useful though definitely still prone to hallucinations that are “out of character” for its intelligence class (wait what did “we” present to the board?) o4 or o5, which I assume will be better on the hallucination front, will be a more solid foundation for pro. [im…
@krishnanrohit Rohit on x
I asked o3 pro to solve a 10 disk Tower of Hanoi game. Apparently done in 13 mins in a sequence of 682 moves. *Apparently because I can't dload the file it made on my mobile right now. [image]
@roberthaisfield Rob Haisfield on x
the price for o1-pro is so astronomically different from o3-pro that I've got to wonder if they just tried to set a high psychological anchor and o1-pro's price was mostly arbitrary [image]
@mckaywrigley Mckay Wrigley on x
My last 4 o3 Pro requests in ChatGPT... It thought for: - 26m 10s - 23m 45s - 19m 6s - 21m 18s Absolute *powerhouse* of a model. [image]
@koltregaskes @koltregaskes on x
o3-pro passed this where o1-pro failed: [image]
@azeem Azeem Azhar on x
o3-pro scored 87.3% on one of the toughest word puzzle benchmarks. The Extended NYT Connections benchmark takes those viral word puzzles you've probably struggled with and makes them even harder by adding extra words as decoys. Out of 651 enhanced puzzles, o3-pro solved nearly 9
@lechmazur Lech Mazur on x
o3-pro sets a new record on the Extended NYT Connections, surpassing o1-pro! 82.5 → 87.3. This benchmark evaluates LLMs using 651 NYT Connections puzzles, enhanced with additional words to increase difficulty. [image]
@tylercowen @tylercowen on x
o3 pro is very very good.
@benhylak Ben on x
we stuffed o3 pro to the brim with @raindrop_ai context * planning meeting notes * company goals * user feedback * voice memos about strategy * calendar screenshots and unlike anything else it gave us specific arr goals with timelines + priorities that we are actually using
@polynoamial Noam Brown on x
Excited to finally have o3-pro out! Reviewers have really liked it. [image]
@swyx Shawn Swyx Wang on x
In order to “Feel the AGI”, we have to “Feed the AGI”. I love this comparison of how you use o3-pro differently than a normal ox model. [image]
@angaisb_ Angel Bogado on x
o3-pro talks to me like an adult would talk to a child, while failing to do what I asked for
@chrisalbon Chris Albon on x
Me: “If cucumbers are pickle cucumbers, what are pickled carrots called?” o3-pro's data center: [image]
@elder_plinius @elder_plinius on x
🚨 JAILBREAK ALERT 🚨 OPENAI: PWNED 🍻 O3-PRO: LIBERATED 🫡 Wowee! Our new fren o3 here is slow as molasses but smart as a whip! Definitely a solid upgrade over previous models, and likely the most capable reasoner we've seen thus far. Refusal mechanisms are strong, which will [image…
@mattyglesias Matthew Yglesias on x
o3 Pro is really good [image]
@natolambert Nathan Lambert on x
First impression, o3 pro delivers for me. o1 pro robustness with built in search.
@openai @openai on x
OpenAI o3-pro is available in the model picker for Pro and Team users starting today, replacing OpenAI o1-pro. Enterprise and Edu users will get access the week after. As o3-pro uses the same underlying model as o3, full safety details can be found in the o3 system card.
@miles_brundage Miles Brundage on x
If they didn't do a full Preparedness Framework assessment, e.g. because the evals weren't too different and they didn't consider it a good use of time given other coming launches, they should just say that, I think.
@miles_brundage Miles Brundage on x
This last sentence seems false? The system card does not appear to have been updated even to incorporate the information in this thread. The whole point of the term system card is that the model isn't the only thing that matters.
@miles_brundage Miles Brundage on x
If o3-pro were the max capability level, I wouldn't be super concerned about this, and I actually suspect it is the same Preparedness Framework level as o3. The problem is that this is not the last launch, and lax processes/corner-cutting/groupthink get more dangerous each day.
@kimmonismus @kimmonismus on x
This is the real revolution [image]
@krishnanrohit Rohit on x
o3 pro is exceptionally good
@nvondarza Nicolai von Ondarza on bluesky
Fascinating piece by OpenAI CEO Sam Altman on his reading of where AI is going at the moment. Obviously part marketing, but in my view still important to reflect on the changes AI is already bringing. — blog.samaltman.com/the-gentle- s... As a scientist, this struck me partic…
@philipcball Philip Ball on bluesky
Sam Altman wrote a blog post to tell us that he doesn't understand human creativity, technology, science, politics, history, ethics, or even his own AI. He does, however, have some grasp of marketing. — blog.samaltman.com/the-gentle- s...
@rickywlmsbong @rickywlmsbong on bluesky
just the effective altruism scam all over, right down to the media figures chosen to peddle it [embedded post]
@jetjocko Adam Rogers on bluesky
If someone really thought that overall intelligence was a rate limiting step on progress—hoo, all those words have exceeded load-bearing capacity—a way to “fix” that “problem” would be to make sure every child had enough to eat and a safe place to go every day. — blog.samaltman…
@edzitron.com Ed Zitron on bluesky
outright irresponsible to just publish this i'm sorry! [embedded post]
@djrothkopf David Rothkopf on bluesky
What worries me is how the superstupid will use superintelligence. Also the super unethical. [embedded post]
@karlbode.com Karl Bode on bluesky
CEO said a thing! [embedded post]
@jamesrball.com James Ball on bluesky
Sam Altman posted this essay in the last hour. I'm someone who believes AI is a genuinely transformative technology - at least the biggest since the internet. — But I read this and it just comes across as either delusional or a bizarre, fantastical sales pitch. I can't work o…
@sama Sam Altman on x
@growing_daniel @GaryMarcus can't tell if he is a troll or just extremely intellectually dishonest. hundreds of millions of happy users, 5th biggest website in the world, people talking about it being the biggest change to their productivity ever... we deliver, he keeps ordering …
@nearcyan Near on x
‘intelligence too cheap to meter’ feels misleading if the cost of agents (as what we ask of them increases) goes exponential. it is still a near-miracle, but tangent to the fact that the programmer with $10,000 to spend on inference will always win out over the one with nothing
@sama Sam Altman on x
also, here is one part that people not interested in the rest of the post might still be interested in: [image]
@mikeisaac Rat King on x
Altman doing the hard sell in his personal blog to top researchers as meta whips out its checkbook to scoop up AI talent across the valley relatively unsubtle reference here but i take it now is not the time for subtlety [image]
@garymarcus Gary Marcus on x
The Ultimate Goalpost Shift Old Days: AI was supposed to be as reliable as a calculator, and as dominant over humans as Deep Thought. New Days: Uneducated nonexperts make lots of mistakes. AI makes mistakes. Singularity achieved!
@emollick Ethan Mollick on x
That Altman essay... One thing you can definitely say about him and Dario is that they are making very bold, very testable predictions. We will know whether they are right or wrong in a remarkably short time [image]
@levie Aaron Levie on x
From @sama today. Ultimately, humans have an unlimited capacity to create and drive progress forward. AI will simply make the way we currently work seem totally manual and inefficient, and we will move on to higher levels of work. This cycle repeats forever. [image]
@garymarcus Gary Marcus on x
Sam keeps doubling down on bigger and bigger promises that are harder to keep. Did Elizabeth Holmes do the same? [image]
@emostaque Emad on x
It's the end of the world as we know it [image]
@sama Sam Altman on x
wrote a new post, the gentle singularity. realized it may be the last one like this i write with no AI help at all. (proud to have written “From a relativistic perspective, the singularity happens bit by bit, and the merge happens slowly” the old-fashioned way)
r/BetterOffline r on reddit
Altman's “The Gentle Singularity” is an admission of defeat
r/ArtificialInteligence r on reddit
Does Sam Altman Live in the Real World?
r/ControlProblem r on reddit
Sam Altman: The Gentle Singularity
r/accelerate r on reddit
Sam Altman New Blog Post- The Gentle Singularity
r/singularity r on reddit
Sam Altman: The Gentle Singularity
@quinnypig.com Corey Quinn on bluesky
A big day in AI news, as OpenAI drops prices, availability. [embedded post]
@sama Sam Altman on x
we dropped the price of o3 by 80%!! excited to see what people will do with it now. think you'll also be happy with o3-pro pricing for the performance :)
@jeremyphoward Jeremy Howard on x
Note that @OpenAI claims that this model is identical to the more expensive one, but we should wait for updated aider evals to be sure
@levie Aaron Levie on x
With OpenAI dropping the price of o3 by 80%, today is a great reminder about how important it is to build for where AI is going instead of just what's possible now. You can now get 5X the amount of output today for the same price you were paying yesterday. If you're building AI
@jeremyphoward Jeremy Howard on x
Wow O3 is a *very* strong option for coding now. I've updated @paulgauthier's latest Aider eval with this O3 80% price cut - check out O3 in 3rd place, but cheaper and faster than Gemini Pro now: [image]
@openaidevs @openaidevs on x
We're cutting the price of o3 by 80% and introducing o3-pro in the API, which uses even more compute. o3: Input: $2 / 1M tokens Output: $8 / 1M tokens Now in effect. We optimized our inference stack that serves o3. Same exact model—just cheaper. https://platform.openai.com/ ... […
@kevinweil Kevin Weil on x
Because you all asked: we're going to double the rate limits for o3 for Plus users. Rolling out as we speak. Now go do awesome stuff with it!
@edwinarbus Edwin on x
o3 is 20% cheaper than GPT-4o. Rethink everything. [image]
@ashwinning Ashwin Sinha on x
o3 is now priced in line with Gemini 2.5 Pro [image]
@dkundel @dkundel on x
The experience of using the OpenAI Agents SDK for all you Cloudflare Workers developers should be much better in the latest release. Right on time to make use of that 80% price cut of o3! Thank you @threepointone for the help! [image]
@natolambert Nathan Lambert on x
we love model competition!
@therealadamg @therealadamg on x
@pli_cachete It's not distilled. Same model.
@simonw Simon Willison on x
o3 80% price drop is a big shake-up in terms of LLM pricing It's now the same as GPT 4.1 ($2/$8), less than Claude Sonnet 4 ($3/$15) and Opus 4 ($15/$75) and sits between Gemini 2.5 Pro for >200,00 tokens ($2.50/$15) and 2.5 Pro for <200,000 ($1.25/$10) https://simonwillison.net/…
@levie Aaron Levie on x
OpenAI dropped o3 prices by 80%. The amazing thing about AI is that use cases that are cost prohibitive today become affordable within a year. That means it's better to build apps that are super powerful and not worry about costs, than apps that are affordable but simple.
@caseynewton Casey Newton on bluesky
Looking forward to a calm and respectful discussion of these stats here on bsky.app [embedded post]
@emollick Ethan Mollick on x
Also, model training is one time & we don't know if it was included in the estimates. GPT-4 likely used 50+ GW to train, enough to power over 5500 homes for a year or the energy of something like 75 transaltlantic jet flights. (Although a small amount averaged across all queries)
@minimaxir Max Woolf on x
Sam Altman just gave ChatGPT's cost-per-query of 0.34 watt-hours: the first time a number has been given in terms of recent LLM power usage and is obviously much lower than the 3 watts still cited by detractors, but there's a lot of asterisks. [image]
@emollick Ethan Mollick on x
Altman essay: https://blog.samaltman.com/... Google post: https://googleblog.blogspot.com/ ... ChatGPT numbers are credible, a direct measure of Llama 3.1 405B estimated around 3x as much energy use per query, and there are likely many efficiencies in the server-side approach to …
@emollick Ethan Mollick on x
This was less than almost every estimate I have seem: according to the latest Sam Altman post, the average ChatGPT query uses about the same amount of power as the average Google search in 2009 (the last time they released a per-search number)... 0.0003 kWh [image]
@minimaxir Max Woolf on x
(is watt-hours the right unit of measurement here?) https://blog.samaltman.com/...
@thetranscript_ @thetranscript_ on x
Sam Altman: “As datacenter production gets automated, the cost of intelligence should eventually converge to near the cost of electricity. (People are often curious about how much energy a ChatGPT query uses; the average query uses about 0.34 watt-hours” [image]
r/artificial r on reddit
Sam Altman claims an average ChatGPT query uses ‘roughly one fifteenth of a teaspoon’ of water

Chronicles