Human-in-the-Loop AI Agents: Where the Approval Step Actually Belongs

“Keep a human in the loop” is the standard answer to every worry about autonomous agents, and it is usually offered as if the phrase were a design. It is not. It is a placement problem, and most teams solve it by putting the person in the wrong place.

The two common versions fail in opposite directions. In the first, a person approves every action the agent takes, and within a week they are clicking approve without reading. In the second, the person is notified after the fact, which is a log with better manners, not a checkpoint.

Approve-everything collapses into approve-nothing

An approval step only works while the person doing the approving is actually deciding. That stops being true quickly when the volume is high and nearly everything is fine.

If ninety-eight of every hundred proposed actions are correct, the reviewer learns that approving is almost always right. The hundredth, the one that matters, arrives looking exactly like the other ninety-nine, and it gets the same half-second of attention. The checkpoint exists on paper and provides no protection in practice.

This is the part that surprises people: adding more approvals can make a system less safe, because it trains the reviewer to stop looking. The goal is not to maximise how often a human is asked. It is to make sure that when a human is asked, the question is worth their attention.

Gate on consequence, not on frequency

The useful question about any action is not “how often does the agent do this?” but “what happens if it is wrong, and can we take it back?”

That produces a short, defensible sort:

• Reversible and low cost. Drafting a reply, tagging a record, filing a document in a folder. Let the agent do these without asking. A wrong one is corrected in seconds.
• Reversible but visible. Posting an internal message, updating a field other people rely on. Let the agent act, but record it where a person can see and undo it.
• Irreversible or external. Sending an email to a customer, moving money, deleting data, changing permissions, anything a third party will see or act on. These are the actions that get a gate.

Most agents perform dozens of actions in the first group for every one in the third. A gate on the third group alone gives a reviewer a small queue of requests that each deserve a real decision.

Approval has to be enforced in code

A gate that the agent is asked to respect is not a gate. If the instruction is “check with the user before sending”, the agent can skip the check, misjudge that it already has permission, or be talked out of it by text it read in a document.

The enforcement point has to be in the tool, not the prompt. The send-email tool should not send. It should create a pending request and return, and a separate path, one the model cannot invoke, releases it once a person approves. This is the same principle as the one in AI agent guardrails: a constraint enforced in code holds regardless of what the model decides to attempt.

It also gives the cleanest place to define the gate. When tools are exposed through something like an MCP server, whether an action needs approval becomes a property of that tool’s definition, visible and testable, rather than a behaviour hoped for in a system prompt.

Make the request answerable in ten seconds

The quality of the approval depends almost entirely on what the reviewer is shown. A request that says “Agent wants to send an email. Approve?” gets approved. A request the reviewer can actually judge looks different.

It shows the exact thing that will happen, not a description of it: the full message, the recipient, the record that will change. It shows why the agent chose it, in a sentence, with the source it relied on. And it shows what is different from the usual case, so that the unusual request stands out from the routine one.

Two further details matter more than they seem. The reviewer needs a way to edit before approving, because the common outcome is “almost right”, and a binary approve-or-reject forces a rejection that costs a whole retry. And rejection needs a reason that goes back to the agent, so the next attempt improves instead of repeating.

The pending state is part of the system

Once an action can wait for a person, the agent has to be able to wait. That is an engineering requirement, not a detail.

The pending request has to be stored durably, so a restart does not lose it. The agent’s work has to be resumable from the point of the gate, which is the same checkpoint discipline described in durable pipelines. And a request nobody answers needs a defined outcome: expire and drop it, escalate it, or hold it, but never silently proceed. An approval that times out into approved is not an approval.

Let the gate move as trust is earned

The right set of gated actions on day one is not the right set after three months. Early on, it is sensible to gate more than seems necessary, because you do not yet know how the agent fails.

The way to loosen it responsibly is to use the record the gate itself produces. If a category of request has been approved unedited several hundred times, that is evidence it can move to the ungated group. If another category is edited or rejected a fifth of the time, that is evidence the agent is not ready for it at all. This is the same measurement discipline as evaluating an agent before you trust it, applied to live traffic: the approvals queue is a continuous test set.

When you do not need any of this

An agent whose actions are all reversible and internal does not need a human gate; it needs good logging and an undo. A human checkpoint is a cost, paid in latency and attention, and it should be spent only where a mistake cannot be quietly reversed.

We build agents with approval gates enforced at the tool layer. See how we approach agent work, or read about why agents fail in production.