Why AI Agents Fail in Production (And What Actually Fixes It)

There is a specific and very common shape to a failed agent project. A prototype gets built in a week and it is genuinely impressive. Leadership sees it. Budget appears. And then, somewhere between month two and month six, it quietly stops being mentioned.

We have been brought in to rescue enough of these to notice that the failure is rarely the model. It is almost always one of five gaps, and all five are known problems with known solutions.

1. There was never a way to tell if it was working

This is the big one. It causes more stalled projects than everything else combined.

A prototype gets evaluated by a person trying it and going “yeah, that’s good.” That works fine for ten examples. It does not work for the thousandth run, and it completely fails the moment you want to change something — because now the only way to know whether your change helped is to try it by hand again, and your memory of how it behaved last week is not a baseline.

So the team stops changing things. The agent gets frozen at whatever quality it happened to reach, because every modification is an unmeasurable risk. That is not a shipped product; that is a hostage situation.

The eval set does not need to be sophisticated. Real inputs, expected outputs, and a grading function — sometimes exact match, sometimes a rubric scored by a model, sometimes a human spot-check on a sample. What matters is that it exists and that it runs automatically.

2. The happy path was the only path

Prototypes are built against clean inputs, because that is what is at hand. Production has the other kind.

The PDF that is a scan of a photocopy. The API that returns a 200 with an empty body. The record with a null where the schema promised a string. The input in a language nobody scoped for. The document that is four hundred pages when everything was tested against ten.

Each of these is individually trivial to handle. Collectively, they are most of the work, and skipping them is why the demo-to-production gap is measured in months.

The fix is unglamorous: every tool call gets explicit failure handling, every parse gets a schema validation with a defined behaviour on failure, and every assumption about input shape gets checked rather than trusted. The agent should have a defined response to “I could not read this,” and that response should not be to hallucinate something plausible.

3. Cost was never modelled

Agent loops multiply token usage in ways that are genuinely hard to intuit. A task that takes eight tool-calling iterations, each carrying the full accumulated context, can cost fifty times what a single completion costs. That is fine at ten runs a day. At ten thousand it is a budget crisis, and it tends to arrive as a surprise.

Three things prevent it:

• Hard spend ceilings per run, enforced in code, not in a spreadsheet projection.
• Context management — summarising or truncating history rather than letting every iteration carry everything that came before.
• Model tiering. Most steps in a real agent loop are classification, routing, or extraction. Those do not need a frontier model. Reserve the expensive model for the steps that genuinely require reasoning, and you often cut cost by an order of magnitude with no measurable quality loss.

4. Failures were silent

A traditional service that breaks throws an error. An agent that breaks returns a confident, well-formatted, entirely wrong answer, and there is nothing in the response that looks like a failure.

This is the most dangerous property of these systems, and it is why observability for agents is a different discipline. You need:

• Full trace persistence. Every tool call, argument, and result for every run, retained long enough to investigate a complaint.
• Confidence signals surfaced, not swallowed. When the agent is uncertain, that uncertainty needs to reach the interface rather than being smoothed away by fluent prose.
• Anomaly detection on distributions. A sudden change in output length, tool-call frequency, or refusal rate is usually the first observable sign that something upstream changed.
• Sampled human review, permanently. Not during rollout — permanently. A small percentage of runs checked by someone who knows what right looks like.

5. Nobody defined what the agent was not allowed to decide

An agent with tools will use them. If one of those tools sends an email, it will send emails, including in situations where a person would have hesitated.

The fix is boring and it works: irreversible actions sit behind gates. Sending, paying, publishing, deleting, and anything with an external side effect requires either an explicit confidence threshold or a human approval step. Everything reversible can proceed freely.

The interesting design question is not whether to have gates but where to put them, and the answer comes from asking what the cost of a wrong action is versus the cost of a delay. High-cost, low-frequency actions gate cheaply. Low-cost, high-frequency actions should not gate at all, or you have simply built a very expensive way to generate work for a human.

The pattern underneath all five

Every one of these failures comes from treating an agent like a feature instead of like a distributed system that happens to have a model in it.

Agents are stateful, non-deterministic, network-dependent, and expensive per operation. Every engineering discipline that applies to that class of system — observability, retries, idempotency, circuit breakers, cost control, graceful degradation — applies here. The novelty of the model layer distracts from the fact that the rest of it is well-understood engineering.

The teams that ship agents successfully are not the ones with the best prompts. They are the ones who built the boring infrastructure first.

Rescuing a stalled project

If you have a prototype that will not cross the line, the sequence that usually works:

1. Build the eval set from the failures you already have. Every case where someone said “it got this wrong” is a test case. You likely have more of these than you think.
2. Instrument before you change anything. You cannot improve what you cannot see, and you want a baseline before you start moving things.
3. Narrow the scope aggressively. Ship the twenty percent of cases the agent handles reliably, route the rest to a human, and widen from there. A narrow thing in production beats a broad thing in staging, permanently.
4. Then start improving quality, with numbers.

Most of the prototype’s domain logic survives this. What gets rebuilt is the execution layer around it, which is usually a few weeks of work rather than a restart.

We do this kind of rescue regularly. See how we approach agent work, or read about evaluating agents properly.