AI Token Spending: A Practical Budgeting Guide
Companies are working out how to manage spending on AI tokens, as the Wall Street Journal has reported. With token-based services, charges can vary with how much the models are used, unlike a subscription billed at a fixed price per seat.
Deloitte describes tokens as units of AI work and notes that consumption-based costs can be difficult to forecast using traditional budgeting habits.
For teams setting a budget, a useful starting point is to identify which workflows use AI, what those workflows are meant to accomplish, and what they cost under current provider terms.
Why token costs resist a traditional budget
Traditional cost models may not capture usage-based AI charges well. Deloitte recommends adapting total-cost-of-ownership approaches to account for AI consumption at the task level.
Three factors can complicate a flat forecast:
- Demand varies by task. A workflow involving multiple reasoning or tool-calling steps may use more tokens than a simple lookup or classification task. Actual usage depends on the model and implementation.
- Usage changes over time. Traffic may shift with launches, seasons, and changes to how a workflow operates. Comparing spend alongside workload volume can help put month-to-month changes in context.
- Purchasing models differ. Organizations may use SaaS AI products, per-token APIs, or self-hosted and reserved compute. Their pricing and cost patterns differ.
A second complication is organizational. A survey of 50 U.S. senior IT leaders conducted with Wakefield Research found that 82% of organizations now track token spend as a distinct IT budget line, much as cloud cost became a boardroom concern before it. Yet 92% said the link between token spend and business outcomes is difficult to measure, and 90% reported unexpected token cost overruns. These results describe the surveyed respondents; they do not establish that the same figures apply to all organizations. Tracking spend and forecasting it are distinct capabilities.
What determines what an LLM costs
There is no single answer to “how much does an LLM cost.” A bill depends on the provider's pricing and how a team uses the service, and per-token rates themselves keep shifting. Relevant factors include:
- Model choice. Compare suitable models for each task, considering both price and whether the output meets the workflow's needs.
- Context volume. Prompts, retrieved documents, conversation history, and attachments may contribute to input-token usage, depending on the provider's billing terms. Review whether the context is needed for the task.
- Output length. Generated-token pricing varies by provider and model. Check the applicable rates before estimating the effect of longer responses.
- Retries and multi-step work. Agents that loop, retry, or re-read material may use more tokens for a request. Measure actual workflow usage rather than inferring it from the visible user interaction.
- Caching and reuse. Where a provider supports caching, check its terms and measure whether repeated prompts or reference material can be handled more efficiently.
- Commitments. Compare committed-use discounts or reserved capacity with pay-as-you-go pricing. Any trade-off between price and flexibility depends on the commitment terms and the stability of the workload.
For planning, consider tracking cost per completed task alongside token prices and total spend. This can help relate costs to workload volume, provided the task and cost are defined consistently.
Set a baseline by workflow
List the AI-enabled workflows that matter and assign an owner to each. Separate materially different uses—such as an internal search tool and a customer-support assistant—so their spending and results can be reviewed on their own terms.
Record provider charges and available usage data, and establish a baseline over a representative period. Note changes to the workflow, model, or provider terms that could make comparisons less useful. Keep records at a level that meets your privacy and security requirements.
Two reporting practices can make a baseline more useful. First, show spend alongside volume, such as requests, tasks, or sessions, to help distinguish increased activity from changes in cost per unit. Second, make relevant consumption information available to the teams responsible for the workflow, subject to privacy and security requirements. Deloitte discusses visibility into AI spend as part of managing that spend.
Build a working forecast
Use observed usage and current provider pricing to estimate a range for each significant workflow. Set out lower, expected, and higher scenarios, and make the assumptions visible. Treat the forecast as a planning tool, not a promise, and revisit it when actual use diverges or the service changes.
Expect the first version to need revision. In the Wakefield survey, half of respondents said their organization had already recalibrated its entire AI budget, while a quarter said they recalibrated only line items as overruns appeared. A pilot can help refine estimates: select work that resembles expected production use, compare charges with the estimate, and assess whether the output meets the team's needs. A small test may not capture every production condition.
The FinOps Foundation's FinOps for AI practice areas group the discipline into inform, optimize, and operate phases. The framework places visibility and forecasting in the inform phase, underscoring the value of identifying workloads when building budgets.
Review cost alongside outcomes, and keep experiments visible
Schedule regular reviews with the people accountable for the workflow, its technical implementation, and its budget. Consider spend alongside adoption, quality, and intended outcomes. When costs rise, identify what changed before deciding whether the change reflects useful activity or an opportunity to adjust the workflow. In the CIO Dive survey, 34% of leaders cited a lack of team awareness of AI spending and token usage as a leading cause of overruns.
Test focused changes before making broad cuts. Options to evaluate include removing context that is not needed, reducing duplicate work, or comparing another suitable configuration. Check results as well as charges so a cheaper change does not undermine the workflow's intended outcome.
Separate pilots from ongoing services in the budget, and decide who reviews a pilot and when. Investigation thresholds can help teams notice changes. If considering automatic hard stops, account for the workflow's importance and risk; interruption may itself have consequences.
A simple review routine
- List significant AI-enabled workflows and name an owner for each.
- Record usage and charges at a level suitable for budgeting and consistent with privacy and security requirements.
- Report spend alongside volume to help interpret changes.
- Build a forecast range from observed usage and current provider terms.
- Review cost alongside quality, adoption, and intended outcomes.
- Investigate significant changes and test targeted adjustments before changing a service broadly.
- Update the forecast when usage or the service changes.
When a new budget line changes quickly, it can be tempting to cut spending broadly. Measuring usage and reviewing costs against actual workflows can support more informed decisions. The goal is not necessarily a smaller bill, but a forecast that reflects usage and helps teams decide what to continue.