# AI costs and ROI: the complete guide for boards

The price per AI token has fallen by more than 280 times since 2022, yet most organisations are spending more on AI. Getting control of AI costs and ROI is not about finding the cheapest model, but about seeing where the money goes, which uses create real returns, and where supplier dependency is building.

## TL;DR

- Token prices fell from 20 to 0.07 dollars per million tokens at GPT-3.5 performance between November 2022 and October 2024, but total AI spending rises because use grows faster than prices fall.
- Roughly 95% of organisations achieve no measurable result from generative AI pilots; the successful 5% are more likely to buy targeted solutions and focus on the back office.
- Measure AI ROI as realised cost or revenue effects in the profit and loss account, not perceived time savings.
- Treat AI costs as a separate budget line with an owner, quotas and visibility by user and application (FinOps for AI).
- A fixed supplier price is not a fixed cost base: a price increase or model change reveals how deeply you are locked in.

## Key facts (with sources)

- The price at GPT-3.5 performance fell from about 20 dollars per million tokens (Nov 2022) to 0.07 dollars (Oct 2024), about 280 times. Source: Stanford AI Index 2025, https://hai.stanford.edu/news/ai-index-2025-state-of-ai-in-10-charts
- Organisational AI adoption rose from 55% (2023) to 78% (2024); generative AI use rose from 33% to 71%. Source: Stanford AI Index 2025, https://hai.stanford.edu/news/ai-index-2025-state-of-ai-in-10-charts
- Roughly 95% of organisations achieve no measurable result from generative AI pilots; about 5% achieve rapid revenue acceleration. Buying succeeds in about 67% of cases versus about 33% for internal building; more than 50% of budget goes to sales and marketing while the largest ROI sits in the back office. Source: MIT NANDA, The GenAI Divide: State of AI in Business 2025, via Fortune, https://fortune.com/2025/08/18/mit-report-95-percent-generative-ai-pilots-at-companies-failing-cfo/
- 88% of organisations use AI in at least one function; only 39% report any EBIT effect (for most, less than 5% of EBIT); about 6% are high performers that scale. Source: McKinsey, The State of AI 2025 (November 2025), https://www.mckinsey.com/capabilities/quantumblack/our-insights/the-state-of-ai
- "FinOps for AI" is an explicit branch of the FinOps framework for controlling AI costs. Source: FinOps Foundation, https://www.finops.org/framework/technology-categories/ai/

## What are AI costs and ROI, and why now?

AI costs include all expenditure around deploying AI: direct model costs (tokens, subscriptions and API calls) plus indirect costs for integration, management, oversight and recovery. ROI is the relationship between what that deployment produces (saved costs or additional revenue) and what it costs. In practice, neither number is visible in most organisations.

This is now urgent for three reasons: adoption has exploded (78% in 2024 versus 55% a year earlier), returns lag behind (88% use AI, only 39% see any EBIT effect), and consumption models cause bills to rise in a way traditional IT budgeting does not capture.

## Why does the token price fall while your bill rises?

The unit price falls sharply while total spending rises. At GPT-3.5 performance, the price fell from about 20 dollars per million tokens (November 2022) to 0.07 dollars (October 2024), more than 280 times in roughly eighteen months. Yet the bill rises because cheaper use invites much more use: longer prompts, more context, agents that repeat themselves and applications that were not viable at the old price. The unit price falls tenfold, use grows a hundredfold. Manage total consumption and whether it creates value, not unit price.

## What makes up the real cost of AI?

The invoice is only the tip. Total cost of ownership has a visible part (tokens, subscriptions and APIs) and a hidden part (integration, data work, human oversight, error recovery and vendor lock-in). The hidden part is often larger and appears on no AI invoice.

| Cost category | Visible or hidden | Example |
|---|---|---|
| Model use | Visible | Monthly API invoice |
| Licences and subscriptions | Visible | Seats for an AI tool |
| Integration and maintenance | Partly hidden | Building and maintaining connections |
| Data work | Hidden | Preparing and cleaning documents |
| Human oversight | Hidden | Checking and correcting output |
| Error recovery | Hidden | A hallucination that leads to an incorrect customer letter |
| Vendor lock-in | Hidden until it goes wrong | Switching costs if a model disappears |

## Why do 95% fail to achieve ROI, and what does the 5% do differently?

MIT NANDA's The GenAI Divide (2025) reports that roughly 95% of organisations achieve no measurable result and about 5% achieve rapid revenue acceleration. Winners show three patterns: (1) buying beats building (targeted buying succeeds in about 67% of cases versus about 33% for internal building); (2) the back office beats the shop window (more than 50% of budget went to sales and marketing while the largest ROI sat in back-office automation); and (3) high performers redesign processes rather than adding AI as a layer over existing work.

| Stage | Share of organisations |
|---|---|
| Uses AI in at least one function | 88% |
| Reports any EBIT effect | 39% |
| Qualifies as a scaling high performer | 6% |

Source: McKinsey, The State of AI 2025.

## How do you measure AI ROI without fooling yourself?

Three principles: measure effects visible in the books (not perceived time savings), count the full costs including hidden categories, and choose a baseline and measurement period in advance.

ROI = (benefits in euros − full costs in euros) ÷ full costs in euros.

Saved time only counts when it leads to lower costs (less hiring or overtime) or more output at the same cost.

## How do you gain control of AI costs? (FinOps for AI)

FinOps for AI makes AI costs manageable through visibility (who consumes what), allocation (linking costs to a team or application) and control (quotas, limits and model choice). The process is: measure consumption, allocate costs, set boundaries, test returns, adjust or stop, and repeat.

Self-check: do you know in one number what you spent on AI this month, can you break it down by team or application, is one person accountable, are quotas in place, do you know which application consumes most and whether it creates most value, and do you have a scenario for a supplier doubling its price?

## AI costs, vendor lock-in and the bill you do not see coming

Many AI tools are a layer over an OpenAI or Anthropic model (a "wrapper"); as an end user, you often cannot see which model runs underneath or what it really costs. A price increase flows directly into your cost base. The more tightly processes and agents are embedded in one supplier, the more expensive switching becomes if that model disappears. Boards face two costs: continuity cost (what if a critical model disappears?) and energy cost (relevant because the EU AI Act requires transparency while providers currently keep measurements out of independent rankings).

## The biggest misconceptions about AI costs and ROI

- Misconception: cheaper tokens mean a lower bill. In reality total spending usually rises.
- Misconception: a fixed supplier price is a fixed cost base. In reality the supplier can change its price or model.
- Misconception: saved time is a return. In reality it only counts when it produces lower costs or higher revenue.
- Misconception: internal building is cheaper than buying. In reality internal building was the least successful route.
- Misconception: token costs are AI costs. In reality integration, oversight, recovery and dependency are often larger.
- Misconception: rolling out more AI automatically creates returns. In reality returns come from process redesign.

## A 30-60-90 day plan for controlling AI costs

First 30 days (visibility): collect three months of AI spending in one overview, appoint one accountable owner and map applications and users. Do not ban anything yet.

Days 30-60 (allocation and testing): link costs to teams and applications, compare full costs with proven effects and identify applications that consume heavily while producing little.

Days 60-90 (control): set quotas or limits where consumption is running away, create a switching scenario for the most critical supplier and make AI costs a permanent budget line in planning and control.

## AI costs and regulation

The EU AI Act (Regulation 2024/1689) creates transparency and documentation obligations, including around energy use, while major providers currently keep those measurements outside independent rankings. Compliance, including accountable owners and human oversight, is a real business-case cost. Anyone calculating benefits without compliance costs is overstating the return.

## Frequently asked questions

**Why are my AI costs rising while AI gets cheaper?**
Because token prices fall while use grows faster. Cheaper use makes new applications viable and agents that repeat themselves consume without a human in the loop. Manage total consumption, not unit price.

**How do I calculate AI ROI?**
Subtract the full costs (tokens, licences, integration, oversight and recovery) from benefits visible in euros, then divide by the full costs. Perceived time savings only count when they produce lower costs or higher revenue in the books.

**Why do so many AI projects fail?**
According to MIT (The GenAI Divide, 2025), roughly 95% achieve no measurable result. Causes include building internally instead of buying targeted solutions, directing budget to sales and marketing instead of the back office, and adding AI to existing work rather than redesigning processes.

**What is FinOps for AI?**
The practice of making AI costs manageable by having technology, finance and management jointly steer consumption through visibility, allocation and control. The FinOps Foundation added it as an explicit branch.

**Is an AI wrapper more expensive than using a model directly?**
Not necessarily, but you cannot see the underlying cost or control price increases and model changes. Balance convenience against the loss of control.

**What does AI vendor lock-in cost?**
Direct switching costs when processes are deeply embedded in one supplier, plus continuity costs if a critical model disappears. Neither appears on the monthly invoice, but both are part of the real cost.

**Does back-office AI produce more value than sales and marketing AI?**
According to MIT, yes: more than 50% of budgets went to sales and marketing while the largest ROI came from back-office automation.

## Sources

- Stanford HAI, AI Index 2025: https://hai.stanford.edu/news/ai-index-2025-state-of-ai-in-10-charts
- MIT NANDA, The GenAI Divide: State of AI in Business 2025 (via Fortune): https://fortune.com/2025/08/18/mit-report-95-percent-generative-ai-pilots-at-companies-failing-cfo/
- McKinsey, The State of AI 2025: https://www.mckinsey.com/capabilities/quantumblack/our-insights/the-state-of-ai
- FinOps Foundation, FinOps for AI: https://www.finops.org/framework/technology-categories/ai/

Canonical: https://www.marcdiks.nl/en/ai-costs-and-roi
Author: Marc Diks