The purpose of introducing chatbots and why it is essential to measure their effectiveness

After confirming the purpose of introducing chatbots, we will introduce the reasons why it is essential to measure their effectiveness.

Main Purposes of Chatbot Introduction

Chatbots are mainly used in two major ways: for customers to resolve questions about products and services, and for internal use to resolve questions when conducting business within the company.

For customers, chatbots can be used to reduce the burden of customer support and improve customer satisfaction. For internal use, the objectives are to reduce the time and effort required by help desks and information system departments to respond to inquiries, and to improve productivity by allowing them to focus on more productive tasks in the reduced time.

Why Effectiveness Measurement is Indispensable for Chatbot Operation

Chatbots are introduced for various purposes, such as improving customer satisfaction, business efficiency, and productivity, but of course, simply introducing chatbots will not achieve these purposes. It is necessary to set numerical targets to achieve the objectives and implement the PDCA cycle to realize them. The key to achieving results after implementation is to always set numerical targets, measure the effects regularly, and confirm whether the targets are being achieved.

If they are not being achieved, considering improvement measures is an essential measure to achieve results through the use of chatbots.

For more information on the purpose and effects of introducing chatbots, please see "What do companies gain by introducing chatbots? Purpose and Effectiveness Explained ".

How to measure the effectiveness of chatbots and how to improve them when they are not effective

This section explains how to measure the effectiveness of major chatbots and how to improve them if they are not effective.

▶Number of times the chatbot is displayed

We measure the number of times a chatbot screen is displayed when a user visits a website. If this number is low, it is necessary to improve the display location, color, and design of the chatbot to make users aware of the chatbot's existence.

▶Chatbot usage rate

We measure the number of times the chatbot is displayed by dividing it by the number of users who actually use the chatbot. This will give you an idea of how many users are using the chatbot and its utilization rate.

If the usage rate is low, you may need to make improvements to make it easier for users to ask questions, such as changing the wording of the initial question or making the input fields easier to understand. Also, depending on the range of target users, multilingual support may be considered.

▶Number of times the user reached an answer

Even if the usage rate of chatbots is high, it is meaningless if users' questions are not ultimately resolved. Therefore, measure whether users who have used the chatbot have progressed to the point where they get a final answer.

If this number is low, it means that many users leave in the middle of a question. In order to prevent users from leaving the chatbot, we need to review the chatbot options, or improve the timing of guiding users to a manned response more quickly, so that users can get an answer more smoothly.

▶Satisfaction with the answers

After the response, we will conduct a survey to determine if the objectives were achieved, and tally the results of satisfaction. Reasons for low satisfaction may include "it takes time to get a response" or "we got a response but the information was insufficient. Therefore, methods to improve satisfaction include shortening the flow line leading to the response, improving the accuracy of the response by changing the wording to make it easier to understand, and increasing the amount of information to which the customer is directed.

▶Change in manned response rate

One of the objectives of implementing customer-facing chatbots is to reduce the burden on the customer center. To confirm whether the objective has been achieved, measure how much the response rate of the customer center has decreased before and after the introduction of the chatbot.

If the response rate has not changed that much, then it may be necessary to make improvements such as making the customer aware of the chatbot as mentioned above, changing the chatbot scenario, or switching to a chatbot that is expected to be more effective.

Further points to improve the results of chatbots

Now that we have seen how to measure the effectiveness of chatbots and how to improve it, here are some points to further improve the results.

▶Choose a chatbot that can respond flexibly

One of the reasons for poor results may be that the introduced chatbot cannot respond flexibly to users' questions. In such cases, it is recommended to introduce chatbots that support both one-question-and-one-answer and scenario types, so that they can respond flexibly to users' questions.

In addition, if the chatbot can respond to fluctuations in spelling, such as kanji, hiragana, and katakana, it can reduce the number of users who leave the site in the middle of a question, thereby improving business efficiency and satisfaction.

▶Consider using external services

For better results, among AI-type chatbots, we recommend a type that can be integrated into external services. For example, if the chatbot can be integrated into LINE, it will be easier for users to use and the frequency of chatbot use will increase.

Also, if you are using Microsoft Teams as an internal information sharing tool, we recommend integrating a chatbot into it. In addition to increasing the usage rate of in-house chatbots, it is expected to improve operational efficiency as users can smoothly resolve questions while working.

To improve the results of chatbots, it is important to regularly measure their effectiveness and continue to make appropriate improvements.

Simply introducing a chatbot does not necessarily mean that it will produce positive results. Clarifying the purpose of introduction and creating appropriate scenarios are important, but what is also indispensable in the subsequent operation is the measurement of effectiveness after the introduction.

We need to measure regularly how many users are using the chatbot, whether they are leaving the service, and whether their satisfaction with the answers is not low.

HUE chatbot is recommended when "user satisfaction is not increasing" or "the burden on the customer center is not decreasing" even if you continue to make improvements with the chatbot you are currently using.

It has the advantage of an easy-to-view management screen that displays usage rates, response rates, satisfaction rates, etc. in graphs, making it easy to analyze and improve the usage status. Questions that were not answered well are automatically categorized and the system tells you which ones are of high importance for improvement, so you can correct those that have many issues on the spot, which greatly contributes to improving the effectiveness measurement figures.

In addition, since the system also supports PDCA through effectiveness measurement after operation, it can always contribute to solving users' issues at a high level. In addition, the system is equipped with Japan's largest language dictionary of 2.9 million words (as of May 2021), so there is no need to register new dictionaries, and management can be performed without programming. Please take a look at the following introductory materials if you are considering introducing or switching to chatbots.