Image Captioning for Indonesian Traffic Sign Images Using Pretrained CNN and Transformer

Abstract

This research aims to address the lack of understanding of traffic signs in Indonesia through the development of an image captioning model using Inception V3 and Transformer. With this approach, a dataset of traffic sign images consisting of 9,594 images with 31 classes was collected and modified. Model evaluation was conducted using BLEU, ROUGE-L, METEOR, and CIDEr metrics. The research results show good performance with BLEU-1 score of 0.89, BLEU-2 = 0.82, BLEU-3 = 0.75, BLEU-4 = 0.68, CIDEr = 0.57, ROUGE-L = 0.25, and METEOR = 0.26. From these results, it can be indicated that this model can enhance understanding of Indonesian traffic signs. This approach can assist road users in better understanding traffic signs and has the potential to be applied in practical applications to improve traffic safety