AI-centric businesses face scaling challenges and lower gross margins due to high compute usage and ongoing human support compared to traditional software firms
Andreessen Horowitz : Tweets: @a16z , @mihail_eric , and @raven_protocol Tweets: @a16z : AI may look like software. AI may replace software. But AI companies don't seem to be software companies as we know them. @martin_casado & Matt Bornstein explain how AI business may be part software, part services. https://a16z.com/... Mihail Eric / @mihail_eric : This is an incredibly insightful article about the reality of running AI businesses: https://a16z.com/... @a16z TLDR: Margins are lower than expected, scale is an issue, and building good moats is very hard. Raven Protocol / @raven_protocol : a16z gets it! We have been facing these exactly challenges while working with our pilot customers. Training a single AI model can cost hundreds of thousands of dollars (or more) in compute resources. Retraining is recognized as an ongoing cost, since the changes https://twitter.com/...
Context & Ripple Effects
Writing in early 2020, Andreessen Horowitz partners Martin Casado and Matt Bornstein laid out the case that AI companies are 'part software, part services': training runs cost hundreds of thousands of dollars in compute, retraining never stops, human support stays in the loop, and the resulting gross margins and scaling curves look nothing like traditional software — while durable moats remain hard to build. Raven Protocol's experience with pilot customers confirmed the same cost and scaling pressures from the operator side.
The half-decade of related coverage since reads as a running validation of that thesis: per-token prices keep falling but developer bills rise anyway as reasoning models burn more tokens per task [[a:889649]], the US data center buildout has topped 80 GW amid ROI doubts [[a:892598]], and by 2026 some corporate buyers were rationing AI use outright after budgets evaporated in months [[a:1169887]]. The margin problem Casado and Bornstein flagged turned out to be structural, not transitional.
First-order effects
- AI startups like Raven Protocol must price and staff like service businesses — carrying ongoing compute and human-support costs per customer — so the standard software go-to-market playbook of land-and-expand at near-zero marginal cost doesn't apply.
- Investors evaluating AI companies can't apply traditional software multiples: lower gross margins and hard-to-defend moats mean valuation has to rest on something other than the SaaS comp set.
Second-order effects
- Buyers respond to unpredictable usage-based costs by rationing or tracking AI deployments — the 'tokenmaxxing' pattern in later coverage — which caps revenue growth for AI vendors whose pricing scales with consumption.
- Value migrates toward whoever controls compute supply: the massive data center buildout and its ROI questions show infrastructure owners capturing the economics that application-layer AI firms surrender through high inference costs.
Third-order effects
- If the pattern holds, the AI industry bifurcates into a thin layer of capital-intensive model and infrastructure providers and an application layer that structurally resembles consulting-heavy services firms — a reversal of the software-margin expectations that drew capital in.
- Defensibility shifts away from models themselves (cheap to replicate, expensive to run) toward distribution, proprietary data, and workflow lock-in, echoing Stratechery's argument that AI's first successful wave looks like enterprise computing installations rather than packaged software.
The trend: AI is settling into a cost-of-goods-sold economic structure — inference and retraining scale with every customer — steadily confirming a16z's 2020 warning that these are not software-margin businesses.