Why Cheap AI Could Cost More

Source

Most software comes with a fairly predictable price tag.

If your company buys 1,000 Microsoft (MSFT) Office licenses or rents computing power from Amazon (AMZN), you can typically estimate the bill.

But AI agents are different.

An agent might spend hours researching a topic, writing software or completing complicated assignments. It might need to search files, call outside tools and make dozens or even hundreds of requests to an AI model.

And every one of those steps consumes computing power.

In other words, the more work we hand over to AI, the more intelligence we’re going to consume.

And as we talked about yesterday, the price of that intelligence can vary enormously depending on which AI model you use.

But a new study uncovered another wrinkle that could make managing those costs even more difficult.

Because even if you know the cost of the model you’re using, you still might not know what the actual job will cost.

And as businesses begin deploying AI agents at scale, that uncertainty will have enormous implications for the economics of AI.

The Agent Tax

The study I mentioned comes from researchers at Stanford and several other universities who wanted to answer a simple question:

How much does it actually cost to put an AI agent to work?

They tested eight leading AI models on real-world coding assignments, tracking the tokens each one consumed. Tokens are the small pieces of information an AI reads and produces.

And because AI companies generally charge based on token use, more tokens mean a bigger bill.

These weren’t simple questions with one-shot answers. The agents had to work through software problems much like human programmers, reading files, deciding what to change, testing their work and trying again when something went wrong.

That’s where the first surprise appeared.

On average, an AI agent working on a coding assignment used roughly 1,200X more tokens than a coding chat. And it used 3,500X more tokens than an AI answering a coding question in a single response.

Most of those extra tokens didn’t come from the answers the AI produced. They came from the information the agent repeatedly fed back into the model as it worked.

That’s because each time an AI agent takes an action, it might need to reread much of what has already happened so it knows where it is in the task.

That’s how a relatively simple job can turn into an enormous amount of AI usage.

But the researchers found something even more surprising.

When they ran the same task using the same AI model multiple times, token consumption could vary by as much as 30X.

That’s because AI agents don’t always take the same path to an answer. One run might find the right solution quickly. Another might spend much longer exploring dead ends before eventually reaching the same place.

But what I find even more fascinating is that spending more tokens didn’t necessarily produce better results.

As these charts from the study show, accuracy often peaked at a moderate level of token use and then stopped improving. In some cases, it even declined.

So the extra computing power was essentially being spent on additional searching and reasoning that didn’t make the final answer any better.

The differences between models were also substantial.

On the same assignments, Kimi-K2 and Claude Sonnet 4.5 consumed, on average, more than 1.5 million additional tokens compared with GPT-5.

What’s more, when researchers asked the models to predict how many tokens they would need before starting a task, they couldn’t do it very well.

Even the best models showed only a weak relationship between their estimates and what they eventually consumed. Overall, they tended to underestimate their own costs.

This study focused specifically on coding agents. But the same basic problem could emerge whenever an agent works through a long, complicated assignment.

And this means businesses might not know what an agent will cost until after the work is finished.

That’s not how businesses prefer to operate. As I wrote about earlier this month, Uber recently found this out the hard way.

But Uber (UBER) isn’t alone.

Microsoft recently began telling its own engineers to pay closer attention to how many AI tokens they consume. The company is setting token budgets for teams and steering workers toward cheaper models when the most powerful ones aren’t necessary.

And Atlassian (TEAM) has introduced monthly AI “wallets” for its developers, ranging from $500 to $2,000 per month.

Its CEO warned that autonomous agents running in the background can consume enormous numbers of tokens without employees necessarily realizing how quickly the bill is growing.

Here’s My Take

AI intelligence keeps getting cheaper.

But price is only half the equation.

A cheap model that takes 20 steps could ultimately cost more than an expensive model that solves the same problem in two. Yet an agent that saves an employee an hour of work could still be a bargain, even if it consumes millions of tokens along the way.

So businesses will have to start thinking about AI in terms of both cost and efficiency.

And that’s where the economics of AI gets interesting.

As intelligence gets cheaper, we’re going to find more things for AI agents to do. And the more work we hand over to them, the more computing power they’ll consume.

Which means the falling cost of AI could ultimately lead us to spend more on it, not less.

And that could keep the AI infrastructure boom running for a lot longer than Wall Street expects.