In SaaS, Margin Came From Scale. In AI, It's Engineered.

For twenty years, software had one property that made it a beautiful business: once you built it, serving the next customer cost almost nothing. That single fact, near-zero marginal cost, is the engine underneath everything founders take for granted. It's what supported 80% gross margins, what made "just add users" a strategy, and what let a generation of companies raise against a model where scale quietly fixed the economics.

AI breaks that property. Not at the margins, but structurally. And in my experience, most founders building AI products are still running their P&L on SaaS instincts. Their gross margin is lower than they think, harder to predict than their model assumes, and about to become the first question a serious investor asks.

Let me walk through what's actually going on, and what I'd do about it.

Your cost of goods now scales with usage

In traditional SaaS, cost of goods sold is mostly fixed. You pay for hosting, some infrastructure, a support team, and as you add customers, those costs grow slowly while revenue grows fast. COGS shrinks as a share of revenue as you scale. That shrinkage is the whole game.

In an AI product, a large and growing slice of your COGS is variable inference cost: the compute you pay for every time your product calls a model to answer a question, summarise a document, or run an agent. Every prompt, every retrieval, every generated token is a metered charge that scales directly with how much your customers use you. It doesn't sit in fixed R&D the way building the product does. It sits in the cost of delivering the product, and it moves with volume.

The benchmarks make the gap concrete. Traditional SaaS at scale carries cost of goods in the range of 10–25% of revenue, which is what produces those 75–90% gross margins. AI-native products land closer to 40–50% cost of goods, and 50–60% gross margins. That's not one report's outlier. Bessemer's early-2026 pricing playbook puts AI gross margins at 50–60% against 80–90% for traditional SaaS, and a16z's original framing, from Casado and Bornstein's 2020 essay, lands in the same band, describing AI-company margins "often in the 50–60% range" versus a 60–80%+ software benchmark. Worth knowing that a16z themselves called that figure anecdotal at the time; six years of data have simply made it look prescient.

ICONIQ's most recent State of AI report (July 2026) shows the trajectory improving but still structurally below software: gross margins moving from 45% in 2025 to a projected 53% in 2026, and 59% in 2027. Improving, but a long way from the 80% a SaaS board takes as its mental baseline.

The finding I most often have to repeat is that this doesn't fix itself with scale. In ICONIQ's data, model inference rises as a share of total cost as products mature, from around 20% pre-launch to 23% at general availability and scale, while the talent share falls. It goes up, not down, because a better product invites more usage, and more usage is more inference. As Jason Lemkin puts it, as you grow you need ever more inference, and you can't cut it without degrading the product. The SaaS reflex, "margins improve as we grow into our fixed costs," assumes a cost base that AI simply doesn't have.

To translate that 23% into something you can feel: at a scaling-stage AI company, inference alone can consume on the order of €230,000 of every €1M in revenue, before you've paid a single engineer, seller, or marketer. (Two different denominators get used loosely in this debate, so it's worth being precise: the "20%→23%" figure is inference as a share of total cost; the "€230K per €1M" figure is inference as a share of revenue. They're close at scaling-stage companies, but they aren't the same measurement. Keep them straight in your own model.)

A useful way to locate yourself: the more central the model is to what the customer is actually paying for, the lower your structural margin. AI-augmented businesses, where AI is a feature on top of a real software product, target around 80%. AI-enabled businesses land at 60–79%. AI-native businesses, where the model is the product and every interaction is an inference call, sit at 50–60%. A thin wrapper around someone else's model, with little other product value, can sit far worse.

Why your budget breaks

This is the part that catches even financially disciplined founders. It isn't only that AI costs are higher. It's that they behave differently, in ways a SaaS forecasting model isn't built to handle. Deloitte describes it as a structural shift in cost behaviour, one that introduces new volatility into operating expense, margins, forecasts, and capital planning. Traditional SaaS COGS is smooth and roughly linear in customer count. AI COGS is lumpy and driven by usage, which means it's driven by behaviour you don't fully control.

Three forces make it genuinely hard to predict:

Usage is heavy-tailed. Your median user and your power user can differ by an order of magnitude in token consumption. Two accounts on identical plans can generate wildly different inference costs, depending on prompt habits, feature mix, and whether their workflows chain agent calls. A budget built on averages misses this entirely, because a handful of heavy users can consume a disproportionate share of your compute.

Agentic workflows multiply consumption invisibly. This is the trap I'd flag hardest right now. Each agent loop, retry, tool call, and context reload multiplies tokens in ways that don't surface until real users hit the system. The failure pattern shows up in the field: there are documented cases of agent products whose economics inverted somewhere between a few hundred and a thousand active users, not at enterprise scale, but far earlier than anyone modelled, because per-user cost climbed with adoption rather than flattening. The cost wasn't a scaling problem to solve later. It was an architecture problem hiding as one.

Reasoning tokens bill for work the user never sees. Reasoning models generate large volumes of hidden "thinking" tokens, charged at output rates, that never appear in the response. A single complex query can run into dollars in reasoning tokens alone, which is exactly why flat-rate pricing on top of a reasoning model can quietly stop working.

The response isn't a better point estimate. It's a different kind of plan. Model per-user inference at both the median and the 90th percentile, run your P&L across a usage range rather than a single line, and build a worst case where inference expense rises sharply. In my experience the heaviest cohort tends to grow fastest, so plan as though it will.

The levers that actually move margin

The encouraging part: 50–60% is a floor, not a destiny. Founders who underwrite compute at build time and route each task to the cheapest capable model can pull margins back toward software norms. The levers I'd reach for, in rough order of impact:

Instrument before you scale. Measure token cost per customer and per feature from day one. Nothing else on this list works without it, and the failure mode is nearly universal: most teams don't measure token consumption until they're already in production, when the architecture is locked. The goal is to get to where "our AI bill grew 40%" resolves into "this feature costs €0.11 per active user and it retains them, so ship more of it." Same bill, completely different meeting.

The single test I'd run first: take one paying customer, count every model call it takes to serve them for a month, price those calls at your real provider rates, and subtract from what they pay you. If the answer is comfortably positive, you have a business. If you can't run that calculation, that's the problem to fix this week.

Route models by task. This is the biggest single COGS lever. Within one vendor the price spread runs 5x or more, so sending planning and hard reasoning to a strong model while handing retrieval, formatting, and summarisation to a cheaper one cuts cost without touching the outputs that actually matter.

Keep context tight. Passing full conversation history into every sub-call is expensive and usually unnecessary. Passing only the relevant state cuts consumption meaningfully, and in agentic systems the saving compounds.

Price so cost and revenue move together. For most early products the usage-vs-flat debate resolves into a hybrid: a flat base tier with a generous usage cap, then overages or a higher tier for heavy users. Anchor the base fee to the value of your core product and let the usage element absorb variable inference cost, so customers get predictable billing while you protect your margin against spikes. Avoid the two failure modes: pure flat pricing lets your heaviest users erode your margin on their own, and 1:1 token pass-through turns you into a reseller of someone else's compute with no cushion. A target I'd work toward is a 60% gross margin on the median user, with usage capped around 3x that level.

What I'd take away

The mental shift is this: in SaaS, margin was manufactured by scale. In AI, margin is engineered by discipline, in your architecture, your model routing, your instrumentation, and your pricing. Scale alone won't save you here. If anything, scale is what exposes a broken cost structure.

None of this makes AI products worse businesses. A lower gross margin can be entirely rational when the product commands higher revenue per customer, stronger retention, and a wider surface area. It's a re-pricing of where margin gets made, not a crisis. But the questions have changed. Your board is going to ask what your AI gross margin is, separated from the rest of the business, and how inference cost scales as you add users. The founders who can answer, because they instrumented early and understood their cost structure honestly, get to underwrite the same math their investors do. Everyone else finds out when the invoice finally gets a meeting.

Sources: ICONIQ Growth, State of AI (July 2026); Bessemer Venture Partners, AI pricing & monetisation playbook (February 2026); Andreessen Horowitz, "The New Business of AI" (Casado & Bornstein, 2020); Deloitte, CFO guide to AI token economics (2026).