Getting a team to use a new AI tool

An AI assistant can produce a useful answer in a demonstration and still add work to someone’s afternoon. The employee may have to find the right record, copy it into a separate window, correct the answer and paste it back into the system that tracks the job.

Before asking why the team is not using the tool, watch one person complete an ordinary task with it. Include the preparation and checking. That observation gives you something specific to fix.

Pick a task people can recognise

Choose a bounded piece of work with a clear finish: draft a reply to a delivery query, extract the renewal date from a contract or assemble a weekly account summary. Agree on what a satisfactory result looks like and which decisions still belong to a person.

For example, in a hypothetical support pilot, the assistant could draft a response using the order record and current delivery policy. The support agent checks the facts and sends it. Refund decisions remain with the existing approval process.

Describe that scope in the tool and in training. Staff should be able to tell whether the next request belongs in the pilot without asking a manager every time.

Compare the whole task

Record how the task works today, including the time spent finding information and correcting mistakes. During the pilot, measure those same activities. A draft produced in ten seconds is not a ten-second resolution if checking it takes several minutes.

Keep quality alongside speed. For a support reply, look for incorrect order details, unsupported promises and requests that needed escalation. Decide who reviews a sample and how they record errors consistently.

Research gives a reason to examine differences within the team. In Generative AI at Work, Erik Brynjolfsson, Danielle Li and Lindsey Raymond studied an assistant introduced to customer support agents. The reported effects varied with worker experience and skill. Those findings concern a particular deployment; they do not establish the outcome for your workflow. Compare your pilot’s results by task and experience level before assuming everyone benefits in the same way.

Train on a case that needs correction

A polished example teaches people what the output should look like. An imperfect example lets them practise deciding whether to use it.

Ask a participant to check a draft against its source record, correct a mistaken detail and route an unresolved request to the right colleague. Show where to report a missing source or an unusable suggestion. Use approved test material, with the same fields and restrictions people encounter during normal work.

Make the approval step explicit. The design questions in human-in-the-loop AI agents also apply to an assistant: the reviewer needs enough evidence to judge the proposed action and a practical way to change or reject it.

Use support requests to choose the next fix

Give the pilot group a named contact and a simple way to attach a task reference. Ask what they were trying to finish, what happened and what they did next. Collect the minimum information needed to reproduce the problem under the company’s data-handling rules.

Look for recurring causes. Repeated permission errors call for an access fix. A missing policy document calls for an ingestion fix. Confusion about when to use the tool may need a clearer scope. Another general training session will not repair a connector that cannot read the required records.

Track each issue to a visible decision. Tell the group which problem was fixed, which needs more evidence and which request is outside the pilot. Include unresolved operational responsibilities in the software handover.

Decide whether to extend the pilot

Logins tell you that people opened the application. To judge whether it helps, examine completed eligible tasks, review effort, error rates and the work people abandoned or completed elsewhere. Ask why someone stopped using it before interpreting their activity count.

Review the evidence with the pilot group at an agreed date. Expand where the complete task is easier and the quality checks hold. Keep a narrower scope where corrections still consume the saving. If a workflow does not benefit, record that result and let people use the established process.

The next rollout should begin with the cases that worked and the changes the team could verify in its own work.