Conversational AI 14-Evaluating Dialogue Systems
Evaluation is important for a number of reasons:
- for developers it is important to determine whether the system performs as expected
- for users it is important to determine whether the system meet s the user’s needs—whether it understands their utterances, whether in the case of task-oriented systems it helps them achieve their goals efficiently, and, in the case of non-task-oriented systems, whether it gives them an enjoyable experience and
- for researchers it is important to establish whether the aims of the research have been met, for example, to validate a new research technique or to investigate whether the system shows improvement on various evaluation metrics against a baseline state-of-the-art system.