Why AI Budgets Live or Die on Cost Per Token

This summer, Anthropic reshaped the enterprise AI invoice without much fanfare. A Forbes Technology Council piece laid it out plainly: the seat still costs $20 per user per month, but it no longer carries a usage allowance, and model consumption across Claude, Claude Code, and Cowork is billed separately at API rates. The same piece pointed to enterprise projects where a single misconfigured workflow ran for months, burning tokens the whole time, before anyone caught the overspend.

The lesson travels well past one vendor. AI spend has stopped behaving like software and started behaving like electricity. The finance teams still budgeting it like SaaS are the ones writing apology memos to the audit committee.

The Invoice Stopped Looking Like Software

Enterprise software billing used to be predictable to the dollar. Multiply seats by price, add a small overage buffer, close the spreadsheet. Renewal was annual, usage was effectively unmetered, and finance never had to ask what any one employee did with the tool.

Token-metered AI breaks every one of those assumptions. The unit of consumption isn't a user. It's a fragment of a word, priced on the way in and priced again, usually several times higher, on the way out. A single power user who hands long documents to a reasoning model can out-spend an entire department of light users on the same seat tier.

A TechBullion feature on why finance teams are auditing where inference runs frames the shift bluntly: CFOs are asking why the GPU bill now dwarfs the data warehouse bill, and the answer almost always traces back to inference running somewhere nobody sized for it.

The Obvious Fix Does Not Fix It

The instinct in most finance orgs is to reach for the familiar lever: cap the seats, negotiate harder at renewal, maybe route everyone to a cheaper model. It feels like control, but the meter has moved somewhere these levers don't reach.

• Seats no longer meter the thing being consumed. A capped seat still permits one developer, one prompt, and one long-running agent to burn a five-figure day. The meter runs on tokens, not logins.
• Model substitution isn't free. Routing routine work to a smaller model saves real money, but output quality and retry rates shift with it. A cheaper model that triggers two extra tool calls per task can cost more than the premium one you were trying to avoid.
• The invoice is only the metered slice. Vector databases, embedding generation, orchestration runtime, cache layers, and egress fees sit outside the token line and routinely add up to as much as the tokens themselves.

There's a second failure mode that tends to go unnoticed: nobody owns the number. When procurement handles the vendor contract, engineering handles the deployment, and the business unit handles the use case, the person who could bend the cost curve is usually three handoffs away from the invoice.

Rebuild the Budget Around the Token

The approach that works treats the token as what the FinOps Foundation now calls the atomic unit of AI value: the smallest thing worth measuring, attributing, and pricing against an outcome. That reframing changes what finance asks for, and from whom.

• Put inference on its own line. Separate it from training, from the data platform, and from general cloud. Inference is the recurring marginal cost of serving the model; training is a one-time capital event.
• Report cost per thousand tokens by workload, not by team or vendor. The support assistant, the code reviewer, and the sales research agent should each have their own blended rate, with input and output split out.
• Pair every token metric with a value metric. Cost per resolved ticket, cost per reviewed pull request, cost per qualified lead. A token number without an outcome number is a trivia question.
• Price the surrounding infrastructure the same way. GPU-hours for self-hosted inference, egress fees for cross-cloud retrieval, storage for embeddings. Give each one a budget owner and a monthly review.

Questions Finance Should Put to Engineering This Quarter

The audit doesn't need to be adversarial. It needs to be specific. A short list cuts through most of the fog:

1. What is the cost per thousand input and output tokens for each production workload, and how has it moved over the last two quarters?
2. Which workloads are running on cloud API, which on reserved GPUs, and which on owned hardware, and what is the per-token cost in each place?
3. What percentage of monthly AI spend is tokens, versus GPU-hours, versus egress, versus the supporting data layer?
4. Who is the named owner for each line, and what authority do they have to change models, cache aggressively, or move a workload?
5. For the three largest workloads, what is the breakeven volume at which reserved capacity or on-prem becomes cheaper than the current API spend?

Any engineering team running real production AI should be able to answer these without a two-week study. If they can't, that is the audit finding, and it's a more useful one than any number on the invoice.