AI Agent Cost Control: Why the Bill Surprises Everyone

The cost of a prototype agent is easy to estimate and almost always wrong. Someone runs it a few times, notes what each run cost, multiplies by expected volume, and arrives at a number that looks comfortable. Then the agent meets real inputs, and the bill is a multiple of the estimate.

The reason is structural. An agent does not make one model call per task. It makes a variable number of them, and each call carries the accumulated context of every call before it. Cost is a property of how the loop behaves, and a demo rarely exercises the loop the way production does.

Cost is steps times context

Two things multiply. The first is the number of steps: each pass through the loop is a model call, and a task that needs twelve tool calls costs roughly twelve times what a single call does. The second is context: every step re-sends the conversation so far, including every earlier tool result, so each step is more expensive than the last.

That second effect is the one people miss. A loop that runs twice as long does not cost twice as much. It costs noticeably more, because the later steps carry a much larger payload. A confused agent that wanders for forty steps can cost far more than ten well-behaved runs.

This is also why the average is a poor guide. Most runs are cheap and a few are enormous. Budgeting from the average lets the tail destroy the plan.

Put a ceiling on every run

The first control is the simplest and the one most often missing: a hard limit per task. A maximum number of steps, a maximum amount of spend, and a wall-clock timeout. When any of them is hit, the agent stops and reports what it has, instead of continuing on increasingly wrong premises.

This is one of the four constraints covered in AI agent guardrails, and it earns its place because it converts an unbounded risk into a bounded one. The worst case for a single task becomes a known number. Without it, the worst case is whatever the loop decides.

A ceiling also gives you something to alert on. A run that hits its limit is a signal worth looking at, because it usually means the task was mis-scoped, a tool is failing in a way the agent cannot recover from, or the input was something the design never anticipated.

Keep the context small on purpose

Because context drives the cost of later steps, controlling it is the largest lever after the step limit.

Tool results are the usual culprit. A tool that returns an entire record, a full web page, or a long list puts all of it into every subsequent call. Trimming results to the fields the agent actually needs, and summarising or dropping old results once they have been used, keeps the payload from growing without limit.

The design principle is that the agent should be handed what it needs for the next decision, not everything it has ever seen. This is the same instinct as narrow tool design in an MCP server: shaping what comes back is part of the interface, not an afterthought.

Cache what repeats

Many agent workloads send the same large block of text over and over: a long system prompt, a set of instructions, a reference document. Providers generally offer ways to avoid paying full price to reprocess an unchanged prefix on every call, and where the workload has a stable prefix, using that is close to free money.

The practical rule is to put the stable material first and the changing material last, so the unchanged portion can be reused. Beyond that, cache results of deterministic work outside the model entirely. If the agent looks up the same record or runs the same calculation repeatedly, that should hit a cache, not a model.

Match the model to the step

Not every step needs the most capable model. Classifying an input, extracting a field, or deciding between two known routes is routine work that a smaller, cheaper model often handles well. Reserving the expensive model for the steps that actually require judgement can cut cost substantially without touching quality where it matters.

The way to do this safely is to measure. Swap a cheaper model into one step, run the evaluation set, and compare. The discipline from evaluating an AI agent before you trust it applies directly: a cost reduction that quietly lowers accuracy is not a saving, and only a measured comparison tells you which one you have.

Track cost per task, not per call

The number that matters to the business is what one completed task costs, and that is rarely what the provider’s dashboard shows. Instrument the agent so that every run records its total tokens and spend, tagged with the task type and the outcome.

Three things fall out of that. You can see which task types are expensive. You can see whether cost per successful task is stable or creeping up. And you can compare cost against the value of the task, which is the only comparison that decides whether the agent is worth running. An agent that costs more to run than the work it replaces is a failure however well it performs.

When to stop optimising

Cost control has diminishing returns. A low-volume agent with a sensible step limit and tracked spend does not need model routing and prompt caching on day one. The ceiling and the tracking are cheap and should always exist. The rest is worth doing when volume makes the savings real, and not before.

We build agents with budgets enforced in code. See how we approach agent work, or read about why agents fail in production.