Generative image captioning in Urdu using deep learning

Afzal, Muhammad Kashif, Shardlow, Matthew, Tuarob, Suppawong, Zaman, Farooq, Sarwar, Raheem ORCID: https://orcid.org/0000-0002-0640-807X, Ali, Mohsen, Aljohani, Naif Radi, Lytras, Miltiades D, Nawaz, Raheel and Hassan, Saeed-Ul (2023) Generative image captioning in Urdu using deep learning. Journal of Ambient Intelligence and Humanized Computing, 14 (6). pp. 7719-7731. ISSN 1868-5137

Preview

Published Version Available under License Creative Commons Attribution. Download (3MB) | Preview

Abstract

Urdu is morphologically rich language and lacks the resources available in English. While several studies on the image captioning task in English have been published, this is among the pioneer studies on Urdu generative image captioning. The study makes several key contributions: (i) it presents a new dataset for Urdu image captioning, and (ii) it presents different attention-based architectures for image captioning in the Urdu language. These attention mechanisms are new to the Urdu language, as those have never been used for the Urdu image captioning task (iii) Finally, it performs quantitative and qualitative analysis of the results by studying the impact of different model architectures on Urdu’s image caption generation task. The extensive experiments on the Urdu image caption generation task show encouraging results such as a BLEU-1 score of 72.5, BLEU-2 of 56.9, BLEU-3 of 42.8, and BLEU-4 of 31.6. Finally, we present data and code used in the study for future research via GitHub (https://github.com/saeedhas/Urdu_cap_gen).