GPU-intensive inference now accounts for roughly 18% of total cloud spend at AI-forward companies, up from about 4% a few years back. That's not a gradual creep — it's a shift in what your biggest cloud cost driver even is. If your FinOps process still treats AI spend the way it treated training spend, you're tracking the wrong thing.
Training cost and inference cost don't behave the same way
Training is a big, discrete, plannable number. You know roughly what a training run costs before you start it, and it ends. Inference doesn't work like that — it scales continuously with usage, which means it grows automatically every time an AI feature succeeds. Ship a feature users like, and your cloud bill goes up in direct proportion, whether or not anyone budgeted for that success.
That's the trap: cost and adoption are now the same curve. A feature that "does well" isn't a pure win on the P&L unless someone modeled the inference cost of it doing well.
Why the number keeps climbing
A few things are compounding at once. AI features have moved from pilot to core product surface — customer support, search, content generation, internal tooling — so the raw number of production inference calls has multiplied. Reasoning models, which trade speed for accuracy by generating longer chains of intermediate computation, cost more per query almost by design. And multimodal inference — images, audio, video — is becoming the default rather than the exception, and all of that is more compute-hungry than plain text.
On the supply side, GPU capacity is tight enough that it's becoming a resellable asset in its own right. Meta is reportedly building out a cloud business specifically to sell surplus AI compute, following a path SpaceX has already used — reportedly moving over a billion dollars a month in excess compute to AI labs. When compute becomes something companies sell to each other on the side, that's a good indicator you shouldn't assume stable per-unit pricing from any single provider going forward.
What actually moves the number
Make cost-per-call visible before launch, not after the bill arrives. If engineering and product can't see the cost of an AI feature at design time, they'll find out the hard way in production. Put it in the review, not the postmortem.
Stop sending every query to your best model. Route simple, high-volume requests to a smaller, cheaper model and save the frontier model for the tasks that actually need it. This is the single biggest lever most teams haven't pulled yet — a lot of production traffic doesn't need what it's currently getting.
Cache and dedupe before the call, not after. Production AI traffic has more repeated and near-duplicate queries than people expect. Catching those before they hit the model is free money.
Know when a reasoning model is actually firing. Extended intermediate computation is expensive, and it's easy for it to get invoked by default rather than by decision. If nobody's watching that switch, it flips more often than it should.
Don't assume single-vendor pricing stays put. With compute now flowing through secondary markets, multi-sourcing GPU capacity is worth evaluating even if you've never needed to before.
Treat inference cost as a design constraint, the same way you'd treat latency or memory. The teams that get burned here aren't the ones with high AI usage — they're the ones who found out how much that usage cost after the feature had already shipped.