The price per token for AI models has fallen, but costs for developers are rising as newer reasoning models require more tokens to complete tasks
With models doing more ‘thinking,’ the small companies that buy AI from the giants to create apps and services are feeling the pinch X: @ericjhonsa , @emollick , @mims , @mims , @mims , @mims , and @mims Bluesky: @prietschka , @carnage4life , @hatr , and @jessefelder . Forums: r/BetterOffline X: Eric Jhonsa / @ericjhonsa : Chatbot query growth is bound to slow down over the next couple of years. But that might not mean much for AI compute consumption growth. https://www.wsj.com/... [image] Ethan Mollick / @emollick : AI labs have managed to grab a significant piece of the profit generated by SaaS companies. Interesting piece by @mims: https://www.wsj.com/... [image] Christopher Mims / @mims : First off, is the entire AI boom threatened by the rising cost of smarter-than-ever AI that burns tokens like they're Papiermark in Weimar Germany? It... doesn't seem like it? Lots of levers for companies to pull. And experience of code-writing startups seems atypical. 🧵/2 Christopher Mims / @mims : You've probably seen the posts about how AI is becoming more expensive than ever for a lot of applications — even as cost per token falls. So I said, OK, I'm going to dig into this. What I found was interesting! And it's more complicated than you've heard. 🧵/1 Christopher Mims / @mims : That said, is there a potential knife-fight in the space of code-completion AI startups? I mean, yeah. I was actually not able to get enough into the guts of how this is playing out to make predictions about what will happen. But it's the area of most intense competition. 🧵/3 Christopher Mims / @mims : So where does that leave us? I actually think the rising cost of delivering AI is a feature, not a bug for those competing in this space now. Can't build a moat around your model if everyone can afford to copy what you're doing. Feels like a war of attrition. Christopher Mims / @mims : Also, and I think this will be big in the years ahead: An increasing share of the revenue for AI providers like OpenAI, Anthropic (and eventually Google and Amazon and Microsoft) is just “a share of the revenue of other SaaS companies.” 🧵/4 [image] Bluesky: Paul Rietschka / @prietschka : Remember when inference was supposed to grow cheaper and cheaper? — And the bronies were telling all of us — including people like me, who've been doing ML for ages — that hallucinations would be solved in no time? — It's time they're held to account for all their lies. www.wsj.com/tech/ai/ai-c... Dare Obasanjo / @carnage4life : Earlier today I used ChatGPT Deep Research, it ran for 10 minutes, scanned 35 sites, and generated a multi-page response. Probably burned more compute than a month of casual use. — That's the AI boom in a nutshell, usage exploding faster than costs are dropping. Revenue's up, profits? … Hakan / @hatr : “So while the price of a unit of AI, known as a token, continues to drop, the number of tokens needed to accomplish many tasks is skyrocketing.” — www.wsj.com/tech/ai/ai-c... Jesse Felder / @jessefelder : ‘For all of that investment to pay off, businesses and individuals will eventually have to spend big on these AI-powered services and products. The alternative is that consumers could just use cheaper, less-powerful models that require fewer resources.’ www.wsj.com/tech/ai/ai-c... Forums: r/BetterOffline : Cutting-Edge AI Was Supposed to Get Cheaper. It's More Expensive Than Ever.
Context & Ripple Effects
The story sharpens a longstanding problem for AI-native software: high compute usage can compress gross margins even when a product is delivered like software. Falling unit prices do not settle the economics when the amount of inference needed for a useful result rises.
It also fits coverage of labs emphasizing inference improvements and the “next thing,” alongside continued work on pre-training and inference. The relevant measure for application builders is therefore cost per completed task, not the headline price of an individual token.
First-order effects
- Developers and smaller SaaS companies using reasoning models face higher variable costs per task, pressuring margins or forcing them to limit long-running features, raise prices, or absorb the expense.
- AI labs capture more of the revenue generated by downstream applications as greater token use offsets lower per-token pricing.
Second-order effects
- Application makers will have stronger incentives to route simpler work to cheaper models and reserve reasoning models for tasks where the added cost can be monetized, increasing demand for model-selection and usage-control tooling.
- Cloud providers and AI infrastructure suppliers benefit from inference demand that grows with workload complexity; this reinforces the utilization and return-on-investment stakes in the AI data-center buildout.
Third-order effects
- If task-level costs remain elevated, AI software may develop a more services-like cost structure than conventional SaaS, with durable pressure to prove that automation delivers enough value to cover inference.
- Competition among model providers may increasingly turn on useful work per dollar rather than token-list-price cuts, favoring advances that reduce reasoning-token requirements as much as cheaper compute.
The trend: AI economics is shifting from cheaper tokens to the harder question of whether increasingly capable models can reduce the cost of a completed useful task.