AI Quality Monitoring tells you, across all your conversations, which requests your bot resolves and which ones it leaves without a solution. You finally know where to act first, without relying on the few customers who leave a rating.
These are the metrics marked with an AI badge in Analytics. After each conversation, an AI reads the whole exchange and assesses it: what the customer was asking about, whether the request was handled, whether the bot lacked information, and how the customer experienced it.
The number to follow: Resolution
Resolution (marker 1) is the share of conversations where the bot handled the customer's request. Handling means answering, pointing to the right page or contact, or passing the conversation to an agent and confirming it. It is the number that tells you whether your bot is doing its job.
Three neighbouring numbers are often confused. The table below shows what each one measures.
Marker | Metric | The question | Who produces it |
|---|---|---|---|
1 | Resolution | Did the bot handle the request? | The AI, on every conversation where the customer asked for something |
2 | Satisfaction score | Is the customer happy? | Customers themselves, with a thumbs up or down vote. Only a minority vote. |
3 | Automation rate | Did the conversation end without an agent? | A simple count, on all conversations |
In the screenshot (demo data), 98% of conversations end without an agent, but the bot only handled the request in 52% of cases. A bot can keep customers away from agents without actually helping them: that is what Resolution reveals, and what was invisible before AI Quality Monitoring.
Before you present these numbers
Your first number will often be lower than expected. It is the first time all your conversations are measured, by a demanding judge. It is not a drop in your bot's performance. There is no market benchmark: compare your bot with itself, month after month.
"Handled" does not mean "correct answer". The AI does not know your business rules: a wrong but plausible answer can be counted as handled.
The AI can be wrong, in both directions. We do not publish an accuracy rate: we would have to judge thousands of conversations by hand to guarantee one. You can check every verdict (see below).
No comparison with 2025. AI metrics start on 1 January 2026.
Important: do not put these numbers in a contract or a bonus target. Present the Resolution trend and the list of topics that leave the most customers without a solution instead: that is what they measure well.
The four metrics, in plain words
For each conversation, the AI returns four answers. Each one has its own detailed page.
Topics: what the conversation was about, for example "Booking changes". One per conversation, chosen from your list of topics.
AI Resolution: was the request handled? It is the Resolution number in the reports.
Knowledge Gaps: the bot did not have the requested information. Each Knowledge Gap is content to add to your knowledge base.
CX Score: a 0 to 10 rating of the customer's experience. It combines resolution (40%), the customer's tone (30%) and the effort they made to be understood (30%). 7 and above: good; below 5: a problem to address.
Some conversations are not assessed: the customer left before asking for anything (just a "hello"), or without answering a question from the bot. They are classified as Left early and count neither in Resolution nor in the CX Score.
Where to find AI metrics
Open Analytics. By default, it shows the last 30 days, by week. For a monthly review, choose the period, then Monthly on the charts. The table below shows where to find the AI metrics.
Report | What you see there |
|---|---|
The essentials: Resolution, CX Score and Knowledge Gap in Health, AI answers compared with scripted flows in AI Impact, the most frequent topics in Topics | |
Trends over time, and the topics that leave the most customers without a solution (Unresolved Drivers) or without information (Knowledge Gaps by Topic) | |
Customer votes, and the topics that cause the most dissatisfaction (Dissatisfaction Drivers) | |
The resolution rate of each flow you built in the bot (Use Case) |
The Conversations report contains no AI metric: it describes traffic (who, when, from where).
Check an AI verdict
Every number leads to the conversations behind it. You never have to take it on trust.
Click a number to open the list of its conversations.
Each row shows its rating out of 10 (the conversation's CX Score) and its tag: Resolved, Unresolved, Left early, and Knowledge gap where relevant.
Hover over the tag: the AI explains its verdict in one sentence, when this explanation is available.
Read in full the conversations where you disagree. If a verdict looks wrong, report it to your ViaSay Customer Success Manager.
AI metrics start on 1 January 2026
Conversations before 1 January 2026 were not assessed by the AI: the evaluation did not exist yet. Your other metrics (number of conversations, handovers to an agent, customer votes) cover your full history.
If the whole selected period is before 1 January 2026, AI metrics are blurred, with the label "AI analysis unavailable". The figures under the blur are examples, not your data: they only show where the metric will appear.
If the period starts before and ends after, nothing is blurred, but AI metrics only cover the part after 1 January. Example: "Last 12 months" chosen in July 2026 shows a number of conversations over twelve months and a Resolution over seven.
Tip: to read AI metrics safely, choose a start date on or after 1 January 2026.
How the evaluation works
The AI assesses the conversation once it has ended, never during it: it does not slow your bot down and does not change any answer.
A conversation has ended after 15 minutes without a new message, or 24 hours if it was handed over to an agent.
The AI reads the whole exchange, scripted or AI-generated, including the part with an agent. A conversation rightly handed over to an agent counts as handled.
Results arrive shortly after the conversation ends. In practice, a day's conversations are visible the next day. AI metrics are not real-time.
A verdict is final. It is never recomputed: your past numbers do not move.
To learn how ViaSay processes conversation data with AI, see Data Usage and Retention Policies for AI.
Other limits to know
The AI only sees the conversation. It does not know that your website was down, that a flight was cancelled or that the customer had already called.
One topic per conversation, even when the customer raises several. Over large volumes, this evens out.
The AI's criteria improve over time. Verdicts already given do not change: a long trend can therefore mix several versions of the criteria. Major changes are announced in the Changelog.
Next steps
Each guide takes about an hour.
How to improve your resolution: start here. Find the topic that leaves the most customers without a solution, fix one thing, measure again.
How to reduce your knowledge gaps: turn missing information into content to add.
How to improve your CX Score: address the conversations where your customers had to repeat themselves.
Your ViaSay Customer Success Manager can review your first months of data with you: they know what a normal starting point looks like on other bots.