Why the cloud playbook can’t answer the AI question

AI presents a fundamentally different problem than cloud cost management, where we answered, “whose spend is this?” largely by tagging the resource. A VM has an owner, a bucket belongs to a team and FinOps optimizes from there. Billing was slow but acceptable, because spend moved inside predictable bands, a human provisioned each resource before it cost anything, and governance capped how fast costs grew. Surprises were unpleasant, rarely existential.

AI took that model, put it to the test and laughed it out of the room. A token call isn’t a resource the way a VM is, so tags have nothing to attach to. To make them affordable, hyperscalers run large models on shared, multi-tenant infrastructure, dropping the per-token price but stripping out the granularity needed to track and control spend.

One API key can carry a dozen workflows across three teams, and the consumer is often an autonomous agent, not a person. The failure modes are also new. An agent can drift off task, loop on a retrieval endpoint hunting for an answer it can’t find, and restart from scratch when it comes up empty. At a few dollars per million tokens, it seems trivial, until that loop runs across thousands of parallel sessions and clears five figures in a day. None of it trips a provisioning gate or maps to a taggable resource. The control points that cloud governance leaned on don’t exist for AI.