Site search misses your articles: add AI search in 1 category first
Keep your site search, add AI search that finds by meaning to one category, then compare to decide whether to expand.
Key points
- Sort failed searches: missing or wording
- Log retyped words and the article opened
- Keep current results; AI adds candidates
- Test 1 category: expand, alias or write
Contents
Sort failed searches to see what to fix
A&A perspective
Sort the searches on your site that return nothing into two groups, missing articles and mismatched wording, and you will know what to fix. Mismatched wording means the answer article exists, but the words the reader typed do not match the words in its title or body.
Hypothetical example
Take a made-up example: a company of about 20 people that sells parts to factories. A customer types "machine stops cause" into the search box on its site. The answer article exists, but its title is "Inspection steps that prevent line stoppages."
Hypothetical example
Nothing comes up, so the customer phones the owner. The owner finds the article and sends its address (URL). Another customer of the same company types "quote for overseas orders." For that one, there is no answer article at all.
A&A perspective
On the search page, both look the same: nothing found. But the first is a problem of reaching an article, and the second is a problem of a missing article.
A&A perspective
In the first case, the owner's job of finding the article and sending its URL again is a candidate for the first task you hand to AI.
A&A perspective
Automate & Augment (A&A), the company writing this article, thinks AI search can fix only the first. By AI search we mean search that finds articles by closeness in meaning rather than by matching words. Missing articles, outdated articles and articles written for a different kind of customer are not solved by changing how search works.
A&A perspective
So collect the searches that returned nothing and label each one "no article," "different words" or "different audience." An article too outdated to use counts as "no article." If "different words" is the larger group, move on to the next step. If "no article" is larger, writing the articles first will help sooner.
Log retyped words to learn how customers ask
A&A perspective
If you keep the words a reader types again right after a failed search, you learn what customers call your articles. Keep each retyped query paired with the article the reader opened last.
From the sources
Spotify, the music and podcast streaming service, described how it used its search logs in a March 2022 engineering blog post. It looked through past logs for searches that succeeded after a first search had failed and the user rephrased it.
From the sources
It then paired the query from before the rephrasing with the podcast episode that was found, and used the pairs as material to train the AI model behind its search. The post explains that the aim was to bring pairs whose words do not match directly into that material.
From the sources
In an August 2021 post, Spotify's research team reported on people searching for podcasts. Compared with people searching for music, they spent more time typing and rewriting their queries. They deleted 53% more characters on average, and their queries were 13% longer in characters.
A&A perspective
A&A reads this as "the less people know the name of what they want, the more they rephrase." Customers of a B2B company also look for articles without knowing their titles, using the words of their own problem. The retyped words become a sample of how your customers put things.
A&A perspective
Keep four things: the first query, the retyped words, the article opened last, and the passage in its body that answers the question.
A&A perspective
Opening an article does not count as finding the answer. Someone on your team reads the body and checks that the answer is there. If the reader landed on an article about an old rule, or one written for a different customer, it does not count as the answer.
A&A perspective
Some sites keep no record of search queries. In that case, take inquiries you actually received, remove names and company names, and rewrite them as questions. Store test questions your staff make up separately from words customers actually typed.
| What to keep | What it tells you | Watch out for |
|---|---|---|
| The first query | What customers call their problem | It may contain names or company names; remove them before moving it |
| The retyped words | What customers call your articles | Pair them with the article opened last |
| The article opened last | Which article the reader reached | Opening it does not count as finding the answer |
| The passage that answers | Whether the answer was really found | Someone on your team reads the body to check |
Keep current results while AI adds candidates
A&A perspective
AI search does not replace your current search. Bring it in to add candidates after the articles your current search already returns.
From the sources
Spotify's search had until then relied mainly on matching words. But users do not always type the exact words for what they want. The engineering post says Spotify had covered this with fuzzy matching and manually added aliases, but those could not capture every natural way of phrasing.
From the sources
The post gives the example query "electric cars climate impact," for which the existing search returned nothing. Spotify first used meaning-based search only for finding podcast episodes, saying it expected that to be where meaning-based matching would help most.
From the sources
Even so, Spotify did not replace its existing search, for two reasons. First, meaning-based search often does not do as well as the existing search when the words match exactly. Second, running it on every query is also more expensive.
From the sources
So meaning-based search became an additional source of candidates. At the end, the top candidates from each source are merged into one list and put in a new order. The post says a test that split users into two groups confirmed that podcast engagement went up, and the feature was rolled out to most users.
A&A perspective
Spotify merged the top candidates and put them in a new order. A&A thinks a different ordering suits a small company's site with no system for reordering results. List the articles your current search returns first, and add the AI search candidates after them. If customers who know the title still reach the article as before, adding AI is unlikely to leave anyone worse off.
A&A perspective
Keep the scope to one category at first (a group of articles on one topic). For the parts company above, that means trying it only on the maintenance and inspection columns, which draw the most inquiries. A narrow scope keeps the candidates few enough for your staff to read and check that each is really an answer.
Test 1 category before and after, then decide
A&A perspective
Decide whether to expand AI search by comparing, within one category, how often searches reach the answer article before and after you add it. Build the questions you compare from the retyping logs and inquiries you collected in the previous step.
From the sources
Spotify used a small set of queries written by people only for evaluation. It says the episodes used for evaluation were kept out of the training material.
A&A perspective
A&A reads this as "set aside the questions you will check with, before you start." For example, if you collect 30 questions, hold back 10 and do not look at them while you adjust the AI search settings. At the end, compare before and after on those 10. Count how many of the 10, typed into the search box, bring up the answer article as a candidate.
A&A perspective
Compare three things. First, the share of questions phrased in problem words that reach the answer article. Second, whether questions that use the title still reach it as before. Third, how much work it takes your staff to check the candidates.
Hypothetical example
Judge whether it pays off with a made-up example as well. Suppose the owner and staff look up an article and send its URL 10 times a week, at 10 minutes each. That is about 7 hours a month. Set those 7 hours against the monthly fee for AI search and the monthly time spent checking candidates. If the 7 hours are larger, see how many months that difference takes to win back the effort of setting it up.
A&A perspective
The result goes one of three ways. If the share reached by problem words rises and title searches do not drop, expand to the next category. If retyping gathers on a few fixed phrasings, adding aliases to the articles is enough. If many questions have no article, write the articles before working on search.
A&A perspective
Spotify's result was higher engagement, not higher sales. The same effect is not guaranteed on a small company's site. And fewer inquiries are not a success if customers simply gave up looking. Read it together with whether the articles they reached really held the answer.
Summary
A&A perspective
Sort the searches that find nothing into missing articles and mismatched wording.
A&A perspective
Keep the retyped words paired with the article opened.
A&A perspective
Keep the articles your current search returns, and let AI search add candidates.
A&A perspective
Compare one category before and after, then decide whether to expand, add aliases or write articles.
When site search cannot find your articles, first tell missing articles apart from mismatched wording before changing the search. If mismatched wording is common, keep your current search, add AI search in one category, compare how often searches reach the answer, and decide whether to expand, add aliases or write articles.
FAQ
How is this different from having AI answer inquiries?
Instead of AI writing an answer, it guides people to articles you already have. Trying AI that writes answers is covered in "Try AI on customer inquiries: start by settling your top 20 answers." If your problem is that the articles exist but people cannot reach them, the steps in this article come first.
Does a site with few articles need AI search?
Sometimes it does not. If you have few articles and retyping gathers on a few fixed phrasings, adding aliases to article titles and descriptions is faster and cheaper. AI search is worth trying when phrasings vary widely and your staff could still name the answer article.
Will logging search queries pick up personal information?
It can. Customers sometimes type names, company names or phone numbers into the search box. Decide how long to keep the logs and who can see them, and remove names and similar details before moving queries into your sorting sheet.
Sources & editorial note
Primary pages read for this article. Publication dates below belong to the sources; access dates record our research.
- Introducing Natural Language Search for Podcast Episodes
Spotify Engineering · 2022-03-17
Accessed 2026-10-23 - Neural Instant Search for Music and Podcasts
Spotify Research · 2021-08-30
Accessed 2026-10-23