Performance Measurement of Natural Dialog System by Analyzing the Conversation

Abstract

The natural dialog system, Chatbot, plays an important role in business domains by properly answering customer queries. In the pattern matching approach of Chatbot development, the user input is matched with a predefined set of responses. The machine learning approach of Chatbot development uses the principles of natural language processing to learn from conversational content. This study focuses on the performance measurement of pattern-based and machine learning-based Chatbot systems. As per the user point of view, performance measurement parameters are the ability to answer quickly, accurately, and comprehensively; to understand questions clearly; user friendliness; personalization options; ethics followed; and ability to process user feedback. The comprehensiveness of the knowledge base, robustness to handle unexpected input, scalability, and interoperability are some of the parameters considered to evaluate the Chatbot system by expert point of view. In this study, the specially designed Chatbot Usability Questionnaire is used to measure the performance of the implemented Chatbot systems.