What are AI costs and ROI, and why now?
AI costs include every expense associated with using artificial intelligence: direct model costs (tokens, subscriptions and API calls), plus indirect costs such as integration, management, oversight and recovery when something goes wrong. ROI is the relationship between what that deployment produces (saved costs or additional revenue) and what it costs. It sounds simple. In practice, most organisations cannot see either number.
The issue is urgent for three reasons. Adoption has exploded: Stanford's AI Index 2025 found that 78% of organisations used AI in 2024, up from 55% a year earlier, while generative AI use more than doubled from 33% to 71%. Returns lag behind: McKinsey's State of AI from November 2025 found that 88% use AI in at least one function, yet only 39% report any EBIT impact, usually below 5% of EBIT. Finally, consumption pricing does not fit traditional IT budgeting because every additional use moves the bill.
What I notice at the boardroom table: the question is almost never "should we do something with AI?". That decision has long been made. The question nobody can answer precisely is "what does it cost us now, and what does it deliver?". As long as that answer is missing, you are steering on instinct, and every supplier price increase is a surprise rather than a scenario you prepared for.
Why does the token price fall while your bill rises?
This is the central difficulty in managing AI costs. Unit prices fall sharply while total spending rises. Both can be true at the same time.
The price figures are not subtle. At GPT-3.5 performance, Stanford's AI Index 2025 reports a fall from about 20 dollars per million tokens in November 2022 to 0.07 dollars in October 2024: more than a 280-fold reduction in just under two years. Look only at that number and AI appears almost free.
Cost per million tokens at GPT-3.5 performance
Logarithmic scale
>280× cheaper
Yet the bill rises. The reason is an old economic pattern: when something gets cheaper, you use more of it, often so much more that total spending increases despite the lower unit price. A cheap model invites longer prompts, more context, more automated calls, agents that repeat themselves, and applications you would never have considered at the old price. The unit price falls tenfold, use grows a hundredfold, and on balance you pay more.
| What you see | What happens | Budget consequence |
|---|---|---|
| Token prices fall | More applications become viable | More projects start |
| Models get smarter | More context and reasoning steps | More tokens per task |
| Agents become normal | Systems call themselves repeatedly | Consumption scales without a human in the loop |
| Supplier offers a fixed price | Supplier changes price or model | Your cost base is outside your control |
The lesson is not to ignore falling prices. The lesson is that price per token is the wrong control. You manage total consumption, which applications cause it, and whether that consumption creates value. The mechanism can be seen in the analysis of how AI budgets at major technology companies ran out of control and why reversing course was not the answer.
What makes up the real cost of AI?
The invoice from OpenAI, Anthropic or a SaaS supplier is only the visible tip. Total cost of ownership has visible and hidden parts, and the hidden part is often larger.
Visible costs are tokens, subscriptions and API invoices. They appear on an invoice and are relatively easy to find, although their allocation across teams and applications is usually a mystery. Hidden costs include integration, data preparation, management, human oversight, correcting errors and the time staff spend checking AI output. That final category never appears on an AI invoice, but it still weighs on the organisation.
| Cost category | Visible or hidden | Example |
|---|---|---|
| Model use | Visible | Monthly API invoice |
| Licences and subscriptions | Visible | Seats for an AI tool |
| Integration and maintenance | Partly hidden | Building and maintaining connections |
| Data work | Hidden | Preparing, cleaning and unlocking documents |
| Human oversight | Hidden | Checking, correcting and approving output |
| Error recovery | Hidden | A hallucination causing an incorrect customer letter |
| Vendor lock-in | Hidden until it fails | Switching costs when a model disappears |
What I see here: most cost reports I come across only add up invoices. AI then looks cheap. Include oversight, recovery and integration and the picture changes; sometimes a "cheap" application costs more than the manual process it replaces. That is not an argument against AI. It is an argument for counting the whole bill before approving a business case. I explain how to build that full picture step by step in calculating AI TCO; the hidden side, such as energy use that providers deliberately make difficult to measure, has its own in-depth analysis.
Why do 95% fail to achieve ROI, and what does the 5% do differently?
The most frequently cited figure of 2025 comes from MIT NANDA's The GenAI Divide: State of AI in Business 2025. Its finding: roughly 95% of organisations in the dataset achieved no measurable result from generative AI pilots, while only about 5% achieved rapid revenue acceleration. The rest remain stuck with little or no demonstrable effect on the profit and loss account.
More important than the figure is the difference between the winners and everyone else. The same research reveals three patterns that explain why the 5% achieve returns.
First: buying beats building. Organisations that bought a targeted solution from a specialist supplier were roughly twice as likely to succeed as organisations that tried to build a tool themselves (about 67% success for external implementations versus about 33% for internal builds). Almost everywhere, organisations tried to make their own tool, and that proved the least reliable route.
Second: the back office beats the shop window. More than half of generative AI budgets went to sales and marketing tools, while the largest ROI came from back-office automation. The money went to visible, exciting applications; the returns sat in dull, repetitive processes.
Third: McKinsey's data shows that the small group that does scale (about 6% of respondents qualify as high performers) systematically redesigns workflows instead of placing AI as a layer over the existing process. They change the work, not just the tool.
The chart below shows the gap: from broad adoption, through the smaller group reporting some EBIT impact, to the very small group that truly scales.
From broad AI adoption to proven returns
| Stage | Share of organisations |
|---|---|
| Uses AI in at least one function | 88% |
| Reports any EBIT effect | 39% |
| Qualifies as a scaling high performer | 6% |
Source: McKinsey, The State of AI 2025 (November 2025).
AI not always being the answer is not heresy but a business-case question; a separate post explores exactly that.
How do you measure AI ROI without fooling yourself?
Most AI ROI calculations are optimistic because they multiply an invented saving by a large number. "This tool saves every employee 30 minutes a day" sounds impressive, until you notice those 30 minutes never appear in the profit and loss account because nobody works fewer hours and no additional revenue is created.
A useful AI ROI measurement follows three principles. First, measure effects visible in the books, not perceived time savings. Saved time only counts when converted into lower costs (less hiring or overtime) or more output at the same cost. Second, count the full costs, including the hidden categories from the TCO table above, otherwise you compare half the cost side with the whole benefit side. Third, choose a measurement period and baseline in advance, so you do not have to rely afterwards on believing the effect existed.
The basic formula remains simple:
ROI = (benefits in euros − full costs in euros) ÷ full costs in euros
The difficulty is not the formula but filling it in honestly. For an insurer, a concrete example is AI that triages claims: it only saves money if less manual assessment is actually needed, and that saving must be set against the model, integration and oversight costs that cannot simply be removed from claims handling. Counting only tokens makes it look cheap. Counting oversight sharpens the question: does AI move the work, or remove it? I explore why saved time is not a return by itself in measuring AI ROI honestly.
How do you gain control of AI costs? (FinOps for AI)
Over recent years, controlling cloud costs developed into its own discipline: FinOps, the practice of having technical teams, finance and management jointly steer consumption. The FinOps Foundation has now added an explicit AI branch, "FinOps for AI", because AI consumption has the same unpredictability as cloud, only more so.
AI costs only become manageable when three things are in order: visibility, allocation and control. Visibility means knowing how much is consumed, by whom and for what. Allocation means linking those costs to a team, application or customer. Control means using that information to set boundaries: quotas per user, limits per application and a deliberate choice of which model is used where.
- 1
Measure AI use
- 2
Allocate costs
- 3
Set boundaries
- 4
Test returns
- 5
Adjust or stop
A practical self-check to see whether you are in control:
- [ ] Do you know your organisation's total AI spend this month?
- [ ] Can you break it down by team or application?
- [ ] Is one person accountable for the AI budget?
- [ ] Are quotas or limits set per user or application?
- [ ] Do you know which application consumes most and whether it creates most value?
- [ ] Do you have a scenario for a supplier doubling its price?
If you cannot answer more than two of these questions, you are steering by instinct. That is not a disaster, but it is the first thing to repair before rolling out more AI. The practical implementation of these three steps is set out in FinOps for AI in practice. The principle that a fixed supplier price is not the same as a fixed cost base deserves a separate deep dive.
AI costs, vendor lock-in and the bill you do not see coming
The largest hidden cost never appears on an invoice: dependency on your supplier. Many AI tools are a software layer over an OpenAI or Anthropic model (a "wrapper"), and end users often cannot see which model runs underneath, what it truly costs, or what happens when the supplier raises its price or switches the model.
That risk is not theoretical. A supplier price increase flows directly into your cost base without you controlling the levers. The dependency goes deeper than price: the more tightly processes and agents are embedded in one supplier, the harder and more expensive it becomes to switch if that model disappears, becomes more expensive or is unusable for another reason. Moving to a cheaper or inferior alternative does not automatically save your operation.
For boards, two costs meet here. The first is continuity cost: what happens to operations if a critical model is no longer available? The second is energy cost, which matters because the EU AI Act will require transparency around energy use while major providers currently keep those measurements out of independent rankings. Both are part of AI's real cost, even though neither appears on the monthly invoice.
What I see here: in insurance we know how to handle dependency on one party, because reinsurance and diversification solve exactly that problem. With AI, organisations concentrate risk in one supplier in a way they would accept from no other critical supplier. The question "what does it cost if this party disappears?" belongs with AI just as it does with every supplier to a core process.
The biggest misconceptions about AI costs and ROI
Misconception: cheaper tokens mean a lower AI bill. In reality, total spending usually rises because cheaper use invites much more use. Manage total consumption, not unit price.
Misconception: a fixed supplier price is a fixed cost base. In reality, the supplier can raise its price or change the underlying model, changing your cost base without you being able to intervene.
Misconception: time saved is a return. In reality, saved time only counts as ROI when it produces lower costs or higher revenue visible in the books. Perceived time savings are not EBIT.
Misconception: building internally is cheaper than buying. In reality, internal building was the least successful route in MIT's research; targeted buying from a specialist produced results about twice as often.
Misconception: token costs are the AI costs. In reality, integration, oversight, recovery and dependency are often larger than the invoice and appear nowhere on it.
Misconception: rolling out more AI automatically creates returns. In reality, the small group achieving returns does so by redesigning processes, not by placing AI on top of existing work.
A 30-60-90 day plan for controlling AI costs
Getting control of AI costs is not a year-long project. It is a matter of starting in the right order. Consultant language does not help; these steps do.
First 30 days: visibility.
- [ ] Gather three months of AI spending, including AI-enabled SaaS.
- [ ] Appoint one accountable owner for the AI budget, not a committee.
- [ ] Map applications and users.
What you do NOT do here: ban tools immediately. Measure first, because blocking without visibility drives use into the shadows.
Days 30 to 60: allocation and testing.
- [ ] Link costs to teams and applications.
- [ ] Compare the full cost of each major application with its proven effect.
- [ ] Identify applications that consume heavily and create little value.
What you do NOT do here: quantify every application to the last decimal. Focus on the major items, not perfection.
Days 60 to 90: control.
- [ ] Set quotas or limits where consumption is running away.
- [ ] Build a switching scenario for your most critical supplier.
- [ ] Make AI costs a permanent budget line in planning and control.
What you do NOT do here: set controls once and forget them. AI consumption changes every quarter, so review is recurring.
AI costs and regulation
Costs and regulation meet in two places. The first is transparency about energy and model use. The EU AI Act (Regulation 2024/1689) creates documentation and transparency obligations for AI systems, while major providers currently keep energy measurements outside independent rankings. For an organisation, the information needed to calculate the real cost, including sustainability, remains incomplete and will change as enforcement progresses.
The second is oversight as a cost. AI compliance, from assigning accountable owners to arranging human oversight, is not a free extra but a real business-case cost. Anyone calculating benefits without compliance costs is overstating the return. The legal side is covered fully in the EU AI Act and AI governance hubs.
Frequently asked questions
Why are my AI costs rising while AI gets cheaper?
Because the price per token falls while use grows much faster. Cheaper use makes new applications viable, invites longer prompts and more automated calls, and agents that repeat themselves consume without a human in the loop. The unit price falls, total consumption rises faster, and on balance you pay more. Manage total consumption, not unit price.
How do I calculate AI ROI?
Subtract the full costs (tokens, licences, integration, oversight and recovery) from benefits visible in euros (lower costs or higher revenue), then divide by those full costs. The pitfall is counting perceived time savings as a benefit: they only count when they produce lower costs or higher revenue in the books. Use a baseline set in advance and a fixed measurement period.
Why do so many AI projects fail?
According to MIT's The GenAI Divide report (2025), roughly 95% of organisations achieve no measurable result. The main causes are building internally instead of buying targeted solutions, directing budgets to visible sales and marketing applications rather than the back office where ROI sits, and placing AI over existing work instead of redesigning processes.
What is FinOps for AI?
FinOps for AI is the practice of making AI costs manageable by having technical teams, finance and management jointly steer consumption. The FinOps Foundation added it as an explicit branch. Its core is visibility (who consumes what), allocation (linking costs to a team or application) and control (quotas, limits and model choice).
Is an AI wrapper more expensive than using a model directly?
Not necessarily, but the risk is that you cannot see the underlying cost or control price increases and model changes. A wrapper with a fixed price feels predictable until the supplier adjusts its rate. Balance the wrapper's convenience against the loss of control and the dependency you build.
What does AI vendor lock-in cost?
The costs are twofold: direct switching costs when your processes and agents are deeply embedded in one supplier, and continuity costs if a critical model disappears or becomes unusable. Neither appears on the monthly invoice, but both are part of the real cost. Treat an AI supplier as you would any other critical supplier to a core process.
Does AI in the back office produce more value than in sales and marketing?
MIT's research suggests it does: more than half of budgets went to sales and marketing tools, while the largest ROI came from back-office automation. Repetitive, structured processes are better suited to measurable returns than the visible customer-facing applications that usually attract the money.
Go deeper
I explore the deeper questions around AI costs and ROI in separate articles. Together they form the cluster beneath this guide.
Calculating and measuring costs - Calculating the true TCO of AI — why total cost of ownership is higher than the invoice. - FinOps for AI in practice — controlling AI costs through visibility, allocation and control. - Saved time is not ROI — measuring returns the profit and loss account also recognises.
Understanding and controlling costs - Getting a grip on AI costs: why Uber and Microsoft reversed AI — how AI budgets run out of control and why reversal is not the answer. - AI costs: why this is not a bubble but a bill — why rising AI costs are structural, not a bubble. - The end of free AI: why ChatGPT feels less capable — the real price of "free" AI.
Dependency and supplier risk - Do you know whether you are buying an AI wrapper? — recognising whether you control your AI cost base. - AI vendor lock-in: why the kill switch is your problem — how dependency on one supplier reaches into operations. - Why OpenAI is becoming your competitor — what happens when your supplier is also your competitor. - Open source AI as a governance necessity — open models as a counterweight to lock-in.
Returns and the hidden bill - AI is not always the answer — when AI fails the business case. - The four AI users in your organisation — how usage patterns drive consumption and costs. - AI energy use cannot be measured, and that is deliberate — the hidden energy cost behind the AI bill.
For the broader strategic and legal context, see the AI governance and EU AI Act hubs.
About Marc Diks
Marc Diks writes about AI, boards and insurance. He looks at AI from the boardroom table and through the lens of 25 years in insurance, and builds production AI himself without a formal coding background. That combination means he treats AI costs and governance not as an abstract IT issue, but as a board decision with a price tag. More about Marc: /en/about.
Want to discuss controlling AI costs in your organisation? Get in touch or use the AI Act Impact Scanner to see which obligations are coming your way. Subscribe to the newsletter for my weekly view on AI and governance.