Once your bot is set up, you can monitor its performance, just like the rest of your activities on iAdvize. This focus tells you which indicators to pay attention to for a successful bot.
1. The Automation report
This report allows you to monitor all your bots, in the context of partially or fully automated conversations. Many indicators are included, but to ensure the “good health” and quality of your bot, here are the ones to observe as a priority:
- CSAT (Customer Satisfaction): your bots will never replace humans and their CSAT score rarely equals that of your flesh and blood respondents, but we recommend a CSAT of at least 30% for your fully automated conversations; a lower score is a warning sign and you can look for an explanation in your other indicators.
- transfer failures are also an important factor. These events can always happen on occasions, but a transfer failure rate above 5% is already really high. Here too, you will need to look for an explanation in your other indicators.
- intent detection failure rate: your visitors need to feel understood in order to have a good experience! A high failure rate is therefore alarming; you can check your list of undetected expressions and add to your intents or entities accordingly.
2. The Scenario Analysis report
This report helps you monitor the success of each bot's path with a tree visualization. This allows you to observe your path in detail, especially:
- the individual bot's CSAT: your bots may not all respond equally well to the needs of the visitors they are targeting! Unlike the automation report that displayed their overall performance, the scenario report shows you the individual score of a bot.
- the drop-off rate at each branch: visitors can always drop out of a conversation, including one with a human respondent. However, a high drop-off rate on one of your cards should be investigated: maybe the card is poorly worded, doesn't work as expected, or doesn't exactly meet your visitors' needs.
- the yes/no rate on your “check” cards: in our focus dedicated to the steps in the path, we recommend creating an “understanding” card to analyze the success of your intent detection, and a “have I answered your question?” card to check that your bot has met your visitors' needs. But you can also use them to analyze your bots' relevance in more detail, within the path itself! If your “understanding” card mostly gets “no”, you will probably need to look again at your intent manager.
- the Scenario Analysis report also allows you to compare different versions of your bot, making it a valuable tool. To make your bot's results as easy to read as possible, we recommend updating them all at the same time when you are performing updates. This will avoid creating too many different versions. For best results, updates should be performed every 10 to 15 days if possible.
3. Other reports
There are other reports which are important in managing your bots, even if they are not exclusively dedicated to them. They include:
- the Insights report gives you a kind of “weather report” of topics on your visitors' minds. If you think your intent detection could be improved or your bot does not seem to satisfy your visitors, this report will quickly give you a good overview of the areas to address.
- the Conversations report gives you access to the details of all conversations, but also lets you filter them, including by degree of automation. By filtering your partially or fully automated conversations and selecting the lowest CSATs, you can read those conversations from start to finish and understand how to help them run more smoothly.