Try AI on customer inquiries: start by settling your top 20 answers
You can test AI on customer inquiries in a small way: settle answers to the top 20 and send the rest to people.
Key points
- At first, just write the top 20 answers
- No AI guessing: route the rest to people
- Count repeat contacts, not just AI share
Contents
Start by settling answers to only the top 20 questions
A&A perspective
If you want to try AI on customer inquiries, the first job is to decide, as a company, the answers to the 20 questions you get most often. Here, AI for customer inquiries means a system in which AI replies to questions from clients and customers based on answers your company has prepared.
From the sources
Intercom, a company that makes customer service software, offers Fin, an AI agent that other companies use to answer their own customers. Fin learns about a company's products and services and draws on its entire knowledge base to answer.
From the sources
On a customer story page from Anthropic, a company that develops AI, Intercom's VP of AI says support reps spend a lot of time searching for authoritative answers, even for questions they have seen before.
From the sources
He also says that someone might previously have spent 30 minutes searching through help articles, while now the answer comes instantly.
A&A perspective
Automate & Augment (A&A), the company writing this article, sees AI as able to reply only within the answers it has been given.
A&A perspective
This article does not recommend Intercom as a product. It cites Intercom as an example whose public documents show what an AI agent answers and how it counts a resolution. Settling the 20 answers is work you can do before choosing any tool.
A&A perspective
In a company of about 20 people, most answers are scattered across staff members' heads and old emails. If you have AI read every past email, discounts agreed with individual clients and outdated terms end up mixed into its answers.
Hypothetical example
For example, take a parts wholesaler with 20 employees. Counting recurring requests in the last three months of email and phone notes, suppose the most common turn out to be delivery time estimates, how to request a quote, invoice reissues and stock availability.
Hypothetical example
Pick the 20 most frequent, write each one on a single line with its answer, last-updated date and owner, and keep the list in one place.
A&A perspective
The number 20 is A&A's rule of thumb, not a figure the sources support. The idea is to keep the list small enough for staff to review each answer one by one. A question you cannot write an answer for is one your company has not yet decided how to answer. Before handing anything to AI, you have found a point that a person needs to decide.
Do not let AI guess: send uncovered questions to a person
A&A perspective
Questions that the 20 settled answers do not cover go to a person, instead of being answered by an AI guess.
From the sources
Intercom's help page lists three situations in which Fin hands over to a human agent: when it cannot resolve the customer's question, when the customer asks for human support, and when an escalation rule or guidance the company has set applies.
From the sources
If a customer asks to speak with a human, the same page explains, the conversation does not count as an outcome for Fin and is not billed.
From the sources
In an article on how to build systems that hand work to AI, Anthropic recommends finding the simplest solution possible and adding complexity only when needed. It also describes agents that return to a person for further information or judgement, and that pause for human feedback when they hit a blocker.
A&A perspective
A&A applies this idea to customer inquiries and recommends drawing two lines for sending questions to a person. The first is any question outside the 20. The second is any question whose answer becomes a company commitment, such as an individual discount, a promised delivery date, or whether a return will be accepted. Keep those out of the 20 and send them to a person from the start.
A&A perspective
When a question goes to a person, pass three things to the staff member so the client does not have to explain again: what was asked, what the AI answered, and what is still open. Questions that are sent to people most often are the next candidates for new answers.
Count repeat contacts, not only the share AI answered
A&A perspective
Two numbers decide whether to continue: the share of inquiries the AI answered automatically, and the number of times someone came back with the same request after it answered (repeat contacts).
From the sources
Intercom's help page counts a resolution when, after Fin's last answer, the customer confirms the answer was satisfactory or leaves the conversation without asking for further help. If the customer later returns to the same conversation for more help, that resolution is deducted.
From the sources
To reduce the chance of frustrated customers being counted as resolved, the same page recommends setting escalation rules so that Fin hands off to a person when it detects frustration.
From the sources
Anthropic's customer story says Fin resolves 51% of inquiries on average out of the box, before any customization or tuning. It is a figure Intercom documented across its own customer base.
A&A perspective
A&A thinks the automatic answer share alone hides the clients who came back by phone or email. The help page's deduction covers returns to the same conversation and does not include contacts that arrive through another channel.
A&A perspective
The 51% is also Intercom's own figure, and there is no basis for expecting the same result at a 20-person company. Find your own share by counting as you test.
Hypothetical example
For example, to get the automatic answer share and the repeat contacts, suppose you write four numbers in a table once a week: inquiries received, inquiries the AI answered, inquiries sent to a person, and contacts that came back with the same request within 7 days. For questions with many repeat contacts, rewrite the answer or correct what it says.
A&A perspective
To decide whether to continue, take the time saved and subtract the time spent fixing answers and handling the questions sent to people. Work out the time saved by multiplying the number of inquiries the AI answered without a repeat contact by the time one reply used to take. Set what remains against your current outsourcing or hiring budget, and you get a rough idea of how many months it takes to pay back.
A&A perspective
Fixing answers after repeat contacts needs someone in-house who knows what was promised to each client. That is why A&A sees keeping the 20 answers inside your company, rather than leaving them to an outsourcer or a tool vendor, as the most important point of this way of testing.
Summary
A&A perspective
At first, just settle the top 20 answers, each with a last-updated date and an owner.
A&A perspective
Questions outside the 20, and questions whose answer becomes a commitment, go to a person instead of an AI guess.
A&A perspective
Count repeat contacts within 7 days alongside the automatic answer share, then decide whether to continue.
You can test AI on customer inquiries in a small way by settling answers to the top 20 questions and sending the rest to people. Give each answer a last-updated date and an owner, and send questions whose answer becomes a company commitment to a person from the start. Decide whether to continue by counting repeat contacts within 7 days alongside the automatic answer share.
FAQ
How should we pick the top 20 questions?
Count recurring requests in about three months of email and phone notes, and pick the most frequent. Twenty is A&A's rule of thumb, a number small enough for staff to review each answer one by one. Questions whose answer changes from client to client stay out of the 20 even if they are frequent, and go to a person instead.
Is it worth trying if we only get a few dozen inquiries a month?
With few inquiries, counting is less likely to show a difference large enough to decide whether to continue. Even so, settling the 20 answers shows you where different staff members give different answers.
How should I estimate the cost of an AI agent for inquiries?
Take the tool's fee and add the time spent fixing answers and the time spent handling the inquiries sent to people. Some services, such as the one from Intercom, a customer service software company, charge for each resolution, so check what counts as a resolution before you sign.
Sources & editorial note
Primary pages read for this article. Publication dates below belong to the sources; access dates record our research.
- Intercom provides customer service tech that delivers up to 86% resolution rates with Claude
Anthropic · Publication date not stated
Accessed 2026-10-22 - Fin AI Agent outcomes
Intercom · Publication date not stated (page shows: updated over a month ago)
Accessed 2026-10-22 - Building effective agents
Anthropic · 2024-12-19
Accessed 2026-10-22