AI customer interview analysis: 3 columns for quote, reading and action
Split AI interview findings into quote, reading and action columns, then check 5 interviews for counterexamples first.
Key points
- AI summaries should trace back to quotes
- Use 3 columns: quote, reading, action
- Hunt for counterexamples in 5 interviews
- Keep 3 of 5 apart from market share
Contents
Take AI summaries that can trace back to quotes
A&A perspective
Take AI summaries in a form that lets you open the exact remark each conclusion came from. AI analysis of customer interviews means having an AI read the interview transcripts and produce summaries or answers to your questions.
A&A perspective
A summary can be full of lines like "looks handy" and "we'd use it." If it drops who said each line and in what situation, it cannot justify what you build. When your manager asks which remark a point rests on, you have no conversation to open.
From the sources
Quillit is software that helps qualitative market researchers analyze interviews, and it runs on AI from the AI company Anthropic. According to Anthropic's customer story, Quillit summarizes interview transcripts and answers questions about the data. It also creates citations that point to the relevant places in transcripts and video recordings.
From the sources
Quillit's product manager says citation accuracy used to be around 60-70%. After the team started using a newer Anthropic model, it was between 89% and 98%. The story does not say how this was measured. The product manager also says this level of precision is crucial for clients' confidence in the insights they receive.
A&A perspective
Automate & Augment (A&A), which writes this article, reads these figures as a reminder that the AI's own citations can be wrong. Because accuracy is not 100%, the person receiving the summary needs a step for opening the original remark and checking it.
A&A perspective
The check is the same whether you use dedicated software or a general-purpose AI. For each sentence in the summary, look for whose remark it came from and at which minute of the conversation.
Use 3 columns: quote, reading and action
A&A perspective
Write the customer's words, your interpretation and the next thing to try (the action) in separate columns of a table. When they share one cell, things the customer never said start to look like the customer's voice.
A&A perspective
A table split into three columns can go straight to your manager. The quote column is the evidence, and the action column is a proposal for where to spend time and money next. Even if a reading turns out to be wrong, you can go back to the quote column and read the remark again.
From the sources
Anthropic has published an article for developers, "Building effective agents." It recommends that, when building systems that use AI, you first look for the simplest solution possible and only increase complexity when needed.
A&A perspective
A&A thinks a spreadsheet is enough for the first table. In that case, you look up where each remark sits in the transcript and type it in by hand. You can decide on a dedicated analysis tool later, once you have put five interviews into the three columns and found that hunting for citations by hand takes too long.
Hypothetical example
For example, suppose a company exploring a booking system as a new business talks to five owners of hair salons and similar shops. One owner says, "While I'm on the phone, my hands stop working on the client." If you summarize this as "they want booking automation," you jump from the remark straight to a conclusion, and your own reading in between disappears.
Hypothetical example
Split into three columns, this remark looks like the table below. The quote column holds the owner's words and where in the conversation they were said. The reading column says, "Taking calls may be a burden because work stops." The action column holds what to ask or try next. The table is a hypothetical example, not a record of real interviews.
| Quote (where in the conversation) | Reading (our interpretation) | Action (what to try next) |
|---|---|---|
| "While I'm on the phone, my hands stop working on the client." (Owner A, minute 12) | Taking calls may be a burden because work stops | Ask in the next interview how many calls come in a week, and what kind |
| "Lots of calls just ask when I have an opening." (Owner D, minute 8) | Most calls may be availability checks | Show open slots on the shop's information page and compare the number of calls |
| "A booking call is also a chat with my regulars." (Owner C, minute 20) | Some customers would miss the calls | Consider limiting the change to calls from new customers |
Hunt for counterexamples first in 5 interviews
A&A perspective
In five interviews, look for remarks that contradict your hypothesis, meaning your current guess, before you gather the similar ones. These counterexamples show you the customers and situations where the hypothesis does not hold.
Hypothetical example
If you ask an AI to "collect every place where they talked about the burden of phone calls," you get a list of remarks that fit the hypothesis. So also ask the same AI, "Did anyone say they don't want fewer calls?"
Hypothetical example
In the table above, Owner C's "A booking call is also a chat with my regulars" is a counterexample. If you dismiss it as a dissenting view, you miss why shops with many regular customers might not use the system.
A&A perspective
When a counterexample appears, use it to narrow down who the system is for. In this example, "shops that only want fewer calls from new customers" becomes a candidate target.
A&A perspective
Five interviews are not enough to say anything statistically. They are a small checkpoint for seeing early whether counterexamples appear. If you have not run your interviews yet, put them into the table once you have five. If five interviews produce no counterexample at all, check whether your questions only draw out answers that fit the hypothesis.
Keep a 3-of-5 count apart from market share
A&A perspective
Write a frequency counted within five interviews as "how many of the five people we asked," and keep it apart from any share of the whole market. The five were not chosen to represent all shops.
Hypothetical example
For example, suppose that in the same hypothetical case, three of the five owners talked about the burden of phone calls. Writing this as "60% of shop owners struggle with phone calls" makes it look like a share that includes shops you never asked. In the table, write "3 of 5" and keep whose remarks they were.
A&A perspective
Use the frequency as a clue to which kinds of shops produce the same remark. If the three who mentioned the burden all run their shops alone, interview more shops like that next. If you need the share for the whole market, you need a separate survey that measures numbers, such as a questionnaire.
A&A perspective
If, after you put five interviews into the three columns, the table shows counterexamples and next questions, A&A thinks the method is worth continuing. If the action column stays empty, revisit your interview questions first.
Summary
A&A perspective
Take AI summaries with citations that trace back to the original remarks. Write the customer's words, your interpretation and the action in three separate columns. In five interviews, look first for counterexamples that contradict your hypothesis. Keep frequencies counted in five interviews apart from shares of the whole market.
Whether an AI summary can justify what you build depends on whether it traces back to the original remarks. Put five of your interviews into the three-column table, see whether counterexamples and next questions appear, and decide whether to keep analyzing with AI or to revisit your questions first.
FAQ
How do we decide whether AI analysis is worth trying in-house?
Decide after putting five interviews into the three-column table, by whether counterexamples and next questions appear in it. If they do, continue. If the action column is empty, revisit your questions first. Five interviews are not enough to say anything statistically; they are a checkpoint for seeing early whether counterexamples appear.
Do we need dedicated analysis software?
You can start with transcripts, a spreadsheet and a general-purpose AI. Once finding citations by hand becomes a burden, choose software on the condition that it can produce citations that trace back to the original remarks.
Can we give interview recordings and transcripts to an AI?
Handle them within the use you described to the people you interviewed. Whether to give them to an outside AI depends on their consent and on your company's rules for handling information.
Sources & editorial note
Primary pages read for this article. Publication dates below belong to the sources; access dates record our research.
- Quillit eliminates 80% of the time-consuming tasks of qualitative research using Claude
Anthropic · Publication date not stated
Accessed 2026-10-23 - Building effective agents
Anthropic · 2024-12-19
Accessed 2026-10-23