Performance Analysis and Development of QnA Chatbot Model Using LSTM in Answering Questions

M Ilyas Tri Khaqiqi; Nisa Hanum Harani; Cahyo Prianto

doi:10.33022/ijcs.v12i3.3249

Penulis

M Ilyas Tri Khaqiqi Universitas Logistik dan Bisnis Internasional
Nisa Hanum Harani Universitas Logistik dan Bisnis Internasional https://orcid.org/0000-0003-2218-4165
Cahyo Prianto Universitas Logistik dan Bisnis Internasional https://orcid.org/0000-0003-1857-1189

DOI:

https://doi.org/10.33022/ijcs.v12i3.3249

Kata Kunci:

QnA chatbot, LSTM, model performance, 5-fold cross-validation, performance variation

Abstrak

This research aims to evaluate the performance of a Long Short-Term Memory (LSTM) based chatbot in answering questions (QnA). LSTM is a type of Recurrent Neural Network (RNN) architecture specifically designed to overcome vanishing gradient problems and can store long-term information. The method used is 5-fold cross-validation to train the chatbot model with 15 epochs at each fold using the dataset provided. The results showed variations in model performance at each fold. At the 5th fold, there was a decrease in performance with 84.63% accuracy, 96.36% precision, 64.9% recall, and 69.84% loss value. This finding shows that there is variability in the performance of the QnA chatbot model at each fold. In conclusion, the LSTM chatbot model can provide good answers with high accuracy and precision. Still, performance variations need to be considered in the use of this chatbot.