Why AI agents fail in production

Reviewed August 2026 · Naresh Prajapati

Most AI agents fail in production for reasons that have nothing to do with the model. They are demonstrated on clean examples and then meet real data; nobody can measure whether a change made them better or worse; they fail silently instead of loudly; and no one owns them once the person who built them moves on. The model is almost never the weakest part of the system. The engineering around it is.

The eight causes, in order of how often they are the real one

1. 01

   It was demonstrated on the happy path

   The demo used three tidy examples someone picked. Production sends forwarded email chains with six replies quoted underneath, scanned PDFs, a spreadsheet where someone typed "n/a" in a date column, and the same customer spelled four different ways. The agent did not get worse — it was never tested on the work.

   What to do Before building, pull fifty real examples from the last month, including the ones that went wrong. If the agent cannot handle the worst ten, you have found the scope rather than a blocker.

2. 02

   There is no way to tell whether a change helped

   Someone adjusts a prompt because one output looked wrong. Nobody knows what that change did to the other two hundred cases. Without an evaluation set, every improvement is a guess and regressions are invisible until a customer finds one.

   What to do Keep a fixed set of real inputs with known-good outputs, and run it on every change. Twenty cases beats none by an enormous margin, and it takes an afternoon to assemble.

3. 03

   It fails silently

   A broken script throws an error and someone gets paged. A broken agent writes a plausible summary of a document it could not actually read, and nobody notices for a fortnight. Confident wrongness is the characteristic failure mode of these systems, and it is invisible by default.

   What to do Make the agent state what it could not do. Log every tool call with its result, alert on unusual patterns — a sudden drop in tool calls usually means it stopped doing the work and started improvising.

4. 04

   Permissions were widened to make it work

   Something failed during the build, someone granted broader access to unblock it, and that access was never narrowed. The agent now runs with far more authority than the task needs, which is fine until the day it does something unexpected with it.

   What to do Give the agent its own account with exactly the access the task requires, and review it at handover. If a permission was added to fix something, write down why.

5. 05

   Nobody owns it after launch

   Models are deprecated. APIs change their response shape. The team renames a folder the agent depended on. An agent is not a painting you hang on the wall — it is a small piece of software with dependencies, and unowned software rots.

   What to do Name an owner before launch, not after. Decide who is paged when it breaks and what the fallback is while it is down — usually the manual process it replaced, which means keeping that documented.

6. 06

   The process was never written down

   The agent was built from a description of the process rather than the process itself. The person who actually does the work knows about the exception where a particular customer is invoiced differently, and the reason nobody chases invoices in the last week of the quarter. None of that was in the brief.

   What to do Watch the work being done before automating it, and write down the exceptions. This is worth doing even if you never build the agent.

7. 07

   Running cost was never modelled

   The pilot cost a few pounds because it ran twenty times. In production it runs four thousand times a month over longer documents with retries, and someone gets a bill that ends the project. Retries are the usual culprit: an agent that fails and retries three times costs four times as much on exactly the days it is working worst.

   What to do Measure cost per run during the pilot, multiply by real volume, then add the retry rate. Set a hard spend cap so a loop cannot become an invoice.

8. 08

   It acts on consequential things without asking

   The agent that drafts an email is a productivity tool. The agent that sends it is a liability with a keyboard. Teams often move from one to the other without noticing, because sending is the obvious next step and nothing bad happens for the first few weeks.

   What to do Decide which actions need a human gate and enforce it in code rather than by convention. Anything that touches a customer, moves money, or cannot be undone belongs behind one.

Seven questions to ask before you build

If a team cannot answer these, the agent is not ready to be built yet — and answering them is cheaper than discovering them after launch.

• Can you show fifty real examples, including the messy ones?
• Could you tell, tomorrow, whether a change made it better or worse?
• What does it do when it cannot complete the task — and would you find out?
• What account does it run as, and what exactly can that account reach?
• Who is paged when it breaks, and what happens while it is down?
• What does one run cost, and what does that become at real volume?
• Which actions must a human approve, and is that enforced in code?

Common questions