engleski

Chatbot evaluation metrics: review paper

The past few years popularity of chatbots is constantly growing, and companies have been focused on developing them more then ever. Therefore, it is not surprising to see the news about various aspects of chatbot, from design and development to commercialization and marketing, are being daily published. Nevertheless, the topic of the chatbot evaluation is very often neglected. The metrics that should be used to evaluate the success of a chatbot are not systematized nor unified. One way to solve this problem is to align the metrics to the different perspectives of the chatbot evaluation: user experience perspective, information retrieval perspective, linguistic perspective, technology perspective and business perspective. In order to build the evaluation framework, the following categories of chatbot should be analyzed: usability, performance, affect, satisfaction, accuracy, accessibility, efficiency, quality, quantity, relation, manner, grammatical accuracy, humanity and business value. This paper provides a review on the evaluation metrics available for measuring success of efforts invested in chatbot, and proposes the chatbot evaluation framework based on five perspectives. The contribution of this paper is to help researchers to identify opportunities for the future research in evaluation of chatbot performance.

chatbot ; chatbot assessment ; chatbot performance ; evaluation metrics

nije evidentirano