Customer Support Agents: What to Automate and What to Hand to a Person

Customer support is the most common place businesses try to put an AI agent, and the results are uneven. Some deployments quietly remove a large share of repetitive work. Others produce confidently wrong answers that anger customers and create more work than they save.

The difference is rarely the model. It is the shape of the system around it: what the agent is allowed to answer on its own, how it decides when to stop, and what happens when it hands a conversation to a person. Getting those three right matters more than the choice of model.

Start by sorting the work

Support requests are not one kind of thing. Treating them as one is the first mistake.

Some are informational and repetitive: where is my order, how do I reset my password, what are your opening hours. The answer exists, is the same for everyone, and can be retrieved. Some need account data and a simple action, such as resending an invoice. Some need judgement, such as a refund outside policy or a complaint about a serious failure. And some are emotionally charged, where the customer mainly needs to feel heard by a person.

An agent can be excellent at the first group, careful with the second, and should not be left alone with the last two. A design that treats all four the same will fail at the ones that matter most.

Triage comes before answering

The first job of the system is to classify the incoming message: what is it about, how urgent is it, how does the customer seem to feel, and does it match something the agent is allowed to handle. This is a cheap step that a small model does well, and it decides everything that follows.

Good triage also catches the requests that should never be automated: legal threats, safety issues, cancellations a retention team wants to see, and anything from a high-value account. Routing those straight to a person is not a failure of the agent. It is the agent doing its job.

Ground every answer in sources

A support agent that answers from the model’s general knowledge will eventually invent a policy. Answers should come from your documentation, your knowledge base, and the customer’s own account data, retrieved at the moment of the question.

That puts the quality of retrieval at the centre, as discussed in retrieval-augmented search for business: citations, a fresh corpus, and an honest answer when the sources do not cover the question. The right response to “I do not have that information” is a handoff, never a guess.

Separate drafting from sending

Early on, the safest configuration has the agent draft a reply that a person reviews and sends. It captures most of the time saving, since the person edits instead of writing, and it gives you a record of how often the draft needed changes.

As confidence builds, you can let the agent send certain categories on its own. The categories should be explicit and narrow, defined by request type and by how well the drafts have performed. This is the graduated approach in human-in-the-loop AI agents: gate on consequence, and loosen the gate with evidence.

Be careful with actions

Answering a question is low risk. Doing something, such as issuing a refund, changing an address, or cancelling a subscription, is not. Keep actions behind narrowly defined tools with their own limits, so an agent can resend an invoice but cannot refund beyond a set amount without approval.

Those limits belong in code, as laid out in AI agent guardrails. Support is also a place where customers can type anything, so the concerns in prompt injection and AI agents apply directly: assume some messages will try to push the agent into doing something it should not.

Design the handoff as a feature

The moment the agent gives up is the moment the customer’s experience is decided. A bad handoff makes them repeat everything. A good one carries over the whole conversation, a short summary of the issue, what has been tried, the relevant account details, and the agent’s suggested next step.

Tell the customer plainly that a person is taking over and roughly when. Do not leave them in a loop of automated replies. And make it easy to reach a person at any point, because trapping people with a bot is the fastest way to turn a small problem into a lost customer.

Measure what matters

Resolution rate alone is misleading, because an agent can “resolve” a conversation by being unhelpful until the customer leaves. Track whether issues stay resolved, how often customers re-contact about the same thing, how often a person had to correct or take over, customer satisfaction on automated conversations, and cost per resolved issue.

Review a sample of conversations regularly, including the ones the agent handled well. The failures that matter are often quiet. Use the approach in agent observability so you can see the whole conversation and the reasoning behind each reply.

When not to bother

A team handling a few dozen messages a week gains little from an agent and risks the personal touch that small customers value. The case grows with volume and with how repetitive the questions are. Start with the narrow slice that is both common and low-risk, prove it, and expand from there.

We build support agents that know when to step aside. See how we approach agent work, or read about AI agents versus chatbots.