AI practice for new hires in customer service: four steps, a person decides
AI can play the customer in practice; a senior colleague or the owner decides who goes live by asking new questions.
Key points
- You write the card; AI plays a customer
- Review mistakes after; let them clarify
- Test with new questions: you or a senior
- If card fixes grow, narrow or use people
Contents
AI plays only the customer; people write the scenes and the answers
A&A perspective
Having AI play the customer so that a new hire can rehearse a conversation is called AI roleplay. Hand AI only the customer's part, and have people write the practice scenes and the company's answers.
Hypothetical example
Say a new employee or contractor is about to take first-line inquiry calls. You explained the answers out loud, yet on a live call they freeze. And neither you nor a senior colleague has time to play the customer for every rehearsal.
A&A perspective
This is where AI roleplay helps. For phone work, practise by voice; if inquiries come by email or chat, practise in text.
A&A perspective
The customer part itself is easy: the AI you already use can produce it right away. What AI does not know is your company's answers, such as prices, lead times, what must never be promised on the spot, and who takes over. Ask it to "play a customer" without them, and it may invent plausible terms of its own.
From the sources
Duolingo, the language-learning app, says in a product article on its conversation-practice feature (published 2023, updated 2026) that the scenes are written by people and matched to where each learner is in the course. Experts write the opening line of each conversation and also tell the AI where to take it.
From the sources
The article adds that people keep reviewing AI-generated content to make sure it is factually correct.
A&A perspective
Automate & Augment (A&A), the company writing this article, redesigns small companies' work with AI as a given and carries it through to implementation.
A&A perspective
A&A reads Duolingo's setup as a split: AI is the practice partner, and people decide what the right answer is. In a small company, what people write is one practice card that holds the scene and the company's answers.
Hypothetical example
For example, take a web production company with four employees and a few contractors, where a new contractor starts answering inquiry calls. The figures are placeholders for illustration.
Hypothetical example
The card names the scene, "asked about lead time before a quote", and adds the customer's opening line and how the conversation should end. The company's answer is "standard lead time is three weeks from kickoff"; the things not to promise are "rush jobs and discounts"; the handover goes to "the owner".
Hypothetical example
You then give AI the card with an instruction such as: "You are a customer asking about lead time before getting a quote. Do not make up any terms that are not on the card."
A&A perspective
Only someone who knows the company's answers can write the card. Leave it to someone who does not, or to AI, and the practice drifts away from what the company actually says.
Let new hires clarify during practice, and review mistakes afterwards
A&A perspective
Let the rehearsal run, and let the new hire talk to the end. Check mistakes in the answers afterwards, against the card.
From the sources
In its article on the video-call feature (published 2024, updated 2025), Duolingo says the feature is for learners who have no one to practise with every day. When learners do not understand the app's conversation partner, or it speaks too quickly, they can ask it to repeat itself or slow down.
From the sources
Learners are not penalised for mistakes on these calls, and the partner does not correct their grammar out loud. The company describes this as a low-pressure design that helps learners speak freely and build confidence.
A&A perspective
In customer-handling practice, too, asking the caller to repeat or clarify does not count as a failure. On a live call, checking again when you missed a condition is good service, so treat it as a good move in practice as well. Stop the new hire at every slip, and they never practise finishing a conversation.
A&A perspective
When the session ends, keep a written transcript and compare it with the card. Do not take the AI's own summary of the session at face value either; check it against what was actually said.
A&A perspective
Watch for sessions in which AI changed the company's answers. In its summary, for instance, AI may present a lead time that is not on the card as "the correct answer". Do not use that session for scoring. Fix the card and the instructions to AI, then run it again.
From the sources
Duolingo, too, lets users report errors in AI responses and says it uses those reports to improve the model.
A person decides whether to go live, using different questions
A&A perspective
However many AI sessions a new hire completes, that alone is not a pass. Whether they go live is decided by a person who plays the customer and asks questions differently from the practice.
From the sources
Duolingo says that after a practice conversation, AI gives feedback on how accurate the learner's answers were and how complex their sentences were, along with tips for the next conversation.
A&A perspective
A&A treats that feedback as a check on whether the language was right. It does not tell you whether an answer matched the company's, or whether the new hire stayed within what may be promised. Use it to review wording, not to decide a pass.
A&A perspective
The person who knows how far a promise may go is whoever wrote the card. So the pass is decided by that person, or by someone who knows the same answers.
A&A perspective
For the sign-off, a senior colleague or the owner plays the customer. In a company without a senior colleague, the owner plays the customer only at this sign-off, not at every rehearsal.
A&A perspective
Ask the practice questions in the same order, and a new hire who has simply memorised the answers will pass. So change the wording and the order, and look for three things.
A&A perspective
Not promising, on the spot, terms that are not on the card. Asking back about conditions they did not catch. Handing questions they cannot answer to the agreed person.
Hypothetical example
In the web production company example, the owner asks: "We need this for a trade show next month. Can you manage it in two weeks?" The new hire passes by not promising a rush job, asking back for the date they need, and saying they will hand it to the owner.
A&A perspective
However fluent the delivery, a new hire who answers "No problem" on the spot does not yet go live alone.
A&A perspective
The record does not grade skill. It keeps the terms the new hire stated, where they asked back, who they handed over to, and the questions they could not answer. Keep it apart from the AI practice log, and it shows which part of the card to fix next.
If the effort grows, narrow the scenes or go back to practice with a person
A&A perspective
Whether to continue depends not on the number of sessions but on the effort spent on senior checks and card fixes. If that effort grows compared with before, narrow the scenes or go back to practice in which a person plays the customer.
A&A perspective
Before starting, write down today's effort: how often and how long a senior colleague plays the customer, where new hires get stuck, and what wrong information they have given.
A&A perspective
After starting, add up the time spent rewriting cards, fixing AI's errors and running sign-offs, and include what you pay for AI. If the total is more than the effort a senior colleague spent playing the customer before, the case for continuing gets weak.
A&A perspective
There are three ways forward. If it is working, keep going. If AI keeps changing the company's answers, narrow practice to questions whose answers are settled. Refunds, complaints and urgent calls, where the right response changes on the spot, go back to practice with a person.
A&A perspective
Where the company's answer has not been settled yet, settling it comes before any AI practice.
Hypothetical example
In the web production company example, lead-time questions have settled answers, so they are easy to keep practising with AI. Discount requests depend on the owner's judgement, so they stay in practice with a person.
A&A perspective
Freed-up senior time is not profit yet. It only means some room has opened up. Check what that time was used for before deciding whether continuing is worth it.
A&A perspective
Both sources are product descriptions from a language app, and neither tested the effect of customer-service training. Nor is there evidence that more practice sessions make live handling better. Where to draw the line between what AI handles and what people check is A&A's view.
AI is the practice partner; a person decides who goes live
A&A perspective
People write the scenes and the company's answers on a card, and AI only plays the customer
A&A perspective
Let practice run to the end, then check mistakes against the card
A&A perspective
A senior colleague or the owner plays the customer and decides who goes live, using questions that differ from the practice
A&A perspective
If senior checks and card fixes grow, narrow the scenes or go back to practice with a person
How much can be handed to AI depends on what today's AI can do. People hold the card's answers and the pass because today's AI does not know your company's answers. A&A draws the line between people and AI while checking, in its own daily work, what today's AI can actually do.
FAQ
Can we use the feedback AI gives after the conversation to decide a pass?
Use it to review wording, not to decide a pass. The feedback Duolingo describes looks at how accurate the answers were and how complex the sentences were. Whether an answer matches the company's is something a person checks against the card.
Can we use real customers' names or past calls in practice?
Leave them out and practise with made-up names and made-up terms. The card holds only the company's answers, never customer information.
Does this work in a company with no senior colleague to run the sign-off?
Yes. In that case the owner runs the sign-off. Where no one can write the company's answer for a scene, decide that answer before any AI practice.
Sources & editorial note
Primary pages read for this article. Publication dates below belong to the sources; access dates record our research.
- Introducing Duolingo Max, a learning experience powered by GPT-4
Duolingo · 2023-03-14; updated 2026-03-25
Accessed 2026-10-21 - Video Call lets you have real life conversations with Lily
Duolingo · 2024-09-24; updated 2025-01-15
Accessed 2026-10-21