DEEP LEARNING BASED IMAGE CAPTIONING OVER MULTIMEDIA IMAGES

DEEP LEARNING BASED IMAGE CAPTIONING OVER MULTIMEDIA IMAGES

Dipan M. Arya
AI, Data Science & Machine Learning
Accenture
Ahmedabad, India

ABSTRACT - Image classification is the process of correctly identifying and grouping of a set of images correctly. This requires the capturing of significant elements, their innate qualities, its relationships in a particular image. The captions generated for an identified is called image semantics and is process of generating an apt caption for any interpreted image. The primary focus of research is on allowing computers to interpret and recognize pictures in much the same way humans do. It also presents additional challenges that are both time-consuming and expensive. Semantic correctness and syntactic veracity are the key attributes to gauge image classification models. Researchers are currently coming up with better approaches using computer vision to match the requirement and produce semantically correct descriptions after identifying images correctly. This study presents a comparison of image classification models for image captioning using deep learning techniques maximizing the strength, overcoming limitations and presents an analysis of its performance. The evaluation metrics captures the computational accuracy, analyzing the predictive capacities of each deep learning model. A cross section of the MS Coco data set was analyzed using evaluation metrics widely used for deep learning based automated image semantics. The strengths performances and limitations of each model are compared based on the individual model accuracy scores.

Keywords— BLEU, CBIR, CIDEr, METEOR, SC-NLM, VGGNet, RNN, Inception-V3, Resnet-152