Conversational AI 19-WHAT IS THE BEST WAY TO EVALUATE DIALOGUE SYSTEMS?

The primary aim of a dialogue system is that it should enable humans to engage in interactions with software agents. The system should be able to perform to the user’s satisfaction, which is determined primarily by the user’s perception of how well the system worked. Different types of system create different expectations. In task-oriented systems the user has clearly defined goals that can be measured objectively, such as completing a task successfully and efficiently. The user’s perceptions of the usefulness of the system may be affected by features associated with the quality of the interaction, such as word error rate, number of corrections required, whether the system’s persona is perceived as likeable, and so on. These factors could all impact on the user’s level of satisfaction. With non-task-oriented systems user satisfaction is determined primarily by the quality of the interaction. This can be measured through questionnaires, although studies have found that users vary considerably in their assessments and so results can be unreliable.

1.Sucessful task completion 

2.Minimum number of Dialogue

The task should be completed with minimum number of dialogues.

3.Likeable persona

The User Interface of chatbot should be simple so that it is easy to interact.

4.Quality of Interaction

All services should be provided by the chatbot in the interaction.

5.Predicting the user’s expectation

Answering all the user queries is must in the chatbot.