AI customer support: answer from your policies, not a guess

Choose a question your team can verify

Pick a repeated category such as explaining delivery options, finding product instructions or describing the next step in an existing request. Define which questions belong in that category and which need a person. A first version does not need to handle every conversation your business has ever received.

I would start with draft answers for the support team. Give an employee the proposed reply and the source beside it. This makes it possible to check whether the system understands the actual question before giving it more responsibility. Fast writing is useful only if the result is correct for that customer.

Give policies a source, a date and an owner

Your knowledge base should separate approved customer-facing policies from old drafts, internal notes and previous replies. A reply from last year may describe an exception someone approved once. That does not make it the rule for the next customer.

For each policy, keep its scope, effective date, owner and a link to the approved version. Decide what happens when the policy changes: which version becomes active, which one is retired and who checks that the support system uses the update. Loading documents once is not a maintenance plan.

Keep general policy separate from live facts. A delivery policy can explain the available options. It cannot establish whether a particular order has shipped. That answer needs the current order record, with the right customer access.

Find the relevant passage before writing the answer

One common approach is retrieval-augmented generation, usually called RAG: retrieve relevant business information and give it to the model alongside the question. Anthropic’s explanation of contextual retrieval describes this pattern and why a passage can lose useful context when separated from its document.

For an owner evaluating a support tool, the practical question is simple: can the reviewer see the passage, the policy version and the reason it applies? A link alone is not enough if the linked text does not support the answer. Finding a document also does not guarantee that the model will interpret it correctly.

Ask the system to identify missing information and conflicting sources. If a policy applies only to one product or sales channel, that condition should travel with the text. I would rather see a short request for clarification than a polished answer built from the wrong rule.

Keep exceptions out of the automatic answer

Consider a hypothetical support request: a customer asks for a replacement, but the approved instructions cover only basic troubleshooting. The agent can find the troubleshooting steps and summarize the request. It should not invent a replacement promise to make the conversation feel complete.

The handoff should contain the question, relevant records, sources checked and the unresolved issue. Name the person or queue responsible for the decision. Give the customer a clear next step that the team can actually honor, rather than an unsupported promise about when the issue will be solved.

I would keep refunds, account changes and other consequential actions behind their own explicit permissions and approvals during the first rollout. Knowing a policy and being authorized to act on an account are different requirements.

Test missing answers as carefully as correct ones

Build a small set of real, appropriately handled support examples with expected outcomes. Include an outdated document, a question with no approved answer, two conflicting policies and a request about someone else’s order. Check whether the system retrieves the right information, asks a useful question or hands the issue over.

Also include a customer message telling the agent to ignore the policy or reveal internal information. Treat the message as a request to evaluate, not as permission to change the support rules. Limit which records the system can retrieve before relying on the wording of a prompt.

NIST’s Generative AI Profile identifies confabulation as a risk to manage. For this workflow, my practical response would be to test unsupported answers explicitly, keep review records and make corrections feed back into the approved knowledge. A confident tone should not count as evidence.

Measure resolution work, not reply volume

Count time spent finding information, checking the draft, correcting it and dealing with reopened requests. Track unsupported statements, unnecessary handoffs and whether the promised next step happened. A quick first reply that creates another support conversation may save no time overall.

My ambition is to remove at least half of suitable repetitive office work. In support, that is a target to test against the complete workload, not a guaranteed reduction in staffing or a reason to accept weaker answers. Start with one category, improve it with the team and expand when the results hold.

If your team keeps searching the same documents to answer the same questions, that is a useful place to begin. Bring a few representative requests and the policies used to resolve them. Those examples can tell us more about the software you need than a broad promise to automate customer service.