Metaheuristics Optimization with Deep Learning Enabled Automated Image Captioning System

Abstract

Image captioning is a popular topic in the domains of computer vision and natural language processing (NLP). Recent advancements in deep learning (DL) models have enabled the improvement of the overall performance of the image captioning approach. This study develops a metaheuristic optimization with a deep learning-enabled automated image captioning technique (MODLE-AICT). The proposed MODLE-AICT model focuses on the generation of effective captions to the input images by using two processes involving encoding unit and decoding unit. Initially, at the encoding part, the salp swarm algorithm (SSA), with a HybridNet model, is utilized to generate effectual input image representation using fixed-length vectors, showing the novelty of the work. Moreover, the decoding part includes a bidirectional gated recurrent unit (BiGRU) model used to generate descriptive sentences. The inclusion of an SSA-based hyperparameter optimizer helps in attaining effectual performance. For inspecting the enhanced performance of the MODLE-AICT model, a series of simulations were carried out, and the results are examined under several aspects. The experimental values suggested the betterment of the MODLE-AICT model over recent approaches.

1. Introduction

Presently, a significant number of images have been produced from many origins such as advertisements, the internet, document diagrams, and news articles. Such origins have images which viewers should analyze themselves [1]. Many images do not contain descriptions; however, human beings can mostly understand them without having any detailed captions. However, machinery should make an interpretation in certain forms of image captions whenever human beings require automated image captions from it. Image captioning is considered significant on numerous grounds [2]. For instance, it is utilized for automated image indexing. Image indexing plays a vital role in content-based image retrieval (CBIR), and thus is implemented in numerous areas involving digital libraries, biomedicine, the military, education, web searching, and commerce. Mass media platforms such as Twitter and Facebook could straight away produce descriptions from images [3]. The descriptions might involve places (e.g., beach, cafe), things that are worn, and, most significantly, the activities that are taking place.

Image captioning basically involves natural language processing (NLP) and computer vision. Computer vision is helpful in recognizing and understanding the situation in an image [4]; NLP transforms semantic knowledge into a descriptive line. Retrieval of the semantic matter of an image and communicating it in a structure which human beings can understand becomes extremely complex. The complete image captioning method not only gives information, but it also further reveals the connection among the substances [5]. Image captioning consists of numerous applications—for example, as an aid advanced for guiding the person having visual disabilities while traveling alone [6]. This is made possible by changing the scenario into text and transforming the text into voice messages. Image captioning is also utilizes mass communication for the automatic generation of the caption for an image which is posted or to explain a video [7,8]. Moreover, automated image captioning might enhance the Google image search method by changing the image to a caption and, after, by utilizing the keywords for additional related searches [9].

Image realization mostly relies on acquiring image features. The methods utilized for understanding the image of two categories are deep machine learning (ML)-related methods and old machine learning-related methods (2). In conventional ML, handcrafted features, namely the histogram of oriented gradients (HOG), local binary patterns (LBPs), and scale-invariant feature transform (SIFT), and a grouping of these features, were broadly utilized. In such methods, features have been derived from the input unit [10]. They were passed afterward to a classifier such as support vector machines (SVMs) for article classification. Furthermore, real-world data such as video and images become complicated and contain diverse semantic interpretations [11]. Conversely, in ML-related methods, features were studied automatically from training data, and they could manage a big and varied set of videos and images. For instance, convolutional neural networks (CNNs) were broadly employed for feature learning, and a classifier such as Softmax can be utilized for categorization. CNN can be usually tracked by recurrent neural networks (RNNs) for generating captions [12].

This study develops a metaheuristic optimization with a deep learning-enabled automated image captioning technique (MODLE-AICT). The proposed MODLE-AICT model aims for the generation of effective captions to the input images by using two processes involving an encoding unit and a decoding unit. At the encoding part, the salp swarm algorithm (SSA) with a HybridNet model is utilized to generate effectual input image representation using fixed-length vectors. Then, the decoding part includes a bidirectional gated recurrent unit (BiGRU) model to generate descriptive sentences. For examining the enhanced performance of the MODLE-AICT model, a series of simulations were carried out, and the results are examined under several aspects.

2. Prior Image Captioning Techniques

In Zhao et al. [13], a fine-grained, structured attention-related technique was suggested when using the structural features of semantic matters in high-resolution distant sensing images. The segmentation is mutually trained with captioning in a unified outline with no need for pixel-wise annotations. Hoxha et al. [14] provide an RSIR technique which mainly focuses on exploiting and producing written descriptions to precisely define the relations among the matters and their features in RS images including captions (e.g., sentences). The initial level focuses to encrypt the image’s visual characteristics and later convert the encrypted features to a textual description which sums up the image content-containing captions. The next level focuses on converting the produced textual descriptions as to semantically useful feature vectors. Lastly, estimating the likeness among the textual descriptions’ vectors of query images with that of archive images restores images of high likeness to the query image.

Wang et al. [15] suggested an end wise trainable deep bidirectional LSTM (Bi-LSTM) method for addressing the issue. With the combination of two separate LSTM networks and a deep CNN (DCNN), this methodology can learn long-term visual–language interactions with the help of future and historical context data at a high level semantic area. In Chang et al. [16], an advanced image captioning method—with image captioning, object detection, and color analysis—was suggested for the automated generation of the textual descriptions of images. In an encrypted–decrypted method for image captioning, VGG16 can be utilized as an encoder and an LSTM network and can be employed as a decoder.

Xiong et al. [17] recommend a hierarchical transformer-related medical imaging report generation technique. This presented technique has two parts: one is ann image encoder that extracts heuristic visual features through a bottom-up attention algorithm; the other is a non-recurrent captioning decoding technique that enhances computational efficacy through parallel computation. Wang et al. [18] suggested an original methodology to indirectly design the association between areas of interest in an image by a graph NN along with the original context-aware attention system for guiding attention selection by completely memorizing formerly attended visual contents.

In Al-Malla et al. [19], the authors introduced a new method to apply the Generative Adversarial Network into sequence generation. The greedy decoding method is utilized for generating an effective baseline reward for self-critical training. The visual and semantic relationship of diverse objects are combined into local-relation attention. The authors in [20] developed an attention-based encoder–decoder deep model which utilizes convolutional features derived from a CNN model that is pre-trained on ImageNet (Xception) along with the object features derived by the YOLOv4 model, pre-trained on MS COCO. The authors also introduced a novel positional encoding scheme for object features, termed the “importance factor”.

3. The Proposed Model

In this study, a new MODLE-AICT technique has been developed for the generation of effective captions to the input images by using two processes involving an encoding unit and a decoding unit. Primarily, at the encoding part, the SSA with a HybridNet model is utilized to generate effectual input image representation using fixed-length vectors. In addition, the decoding part includes a BiGRU model that is used to generate descriptive sentences. Figure 1 show cases of the block diagram of the MODLE-AICT algorithm.

3.1. Data Pre-Processing

At the preliminary level, data pre-processing is performed in different stages as given below.

• Lower case conversion;
• Removal of punctuation marks to decrease complexity;
• Removal of numeric values;
• Tokenization;
• Vectorization (to turn the original strings into integer sequences where each integer represents the index of a word in a vocabulary).

3.2. Feature Extraction: HybridNet Model

In this work, the HybridNet model is utilized for generating visual features of the input images. Generally, classification requires intra-class, in-variant features, while reconstruction requires the preservation of each dataset. In order to overcome these shortcomings, HybridNet includes the unsupervised path ($E_u$ and $D_u$) and the discriminative path ($E_c$ and $D_c$). These two $E_u$ and $E_c$ encoders take an $x$ input image and generate $h_c$ and $h_u$ representations, whereas decoders $D_c$ and $D_u$ take $h_c$ and $h_u$, respectively, as an input to generate $\widehat{x}$ and $\widehat{x}$ partial reconstructions. At last, the $C$ classifier produces a class prediction by means of a discriminative feature: $\widehat{y}=C\left(h_c\right)$. Although both paths may have analogous architecture, they have to perform complementary and different roles. The discriminative path needs to extract $h_c$ discriminative feature that must be ultimately well crafted to effectively execute a classifier task and produce a ${\widehat{x}}_c$ partial reconstruction that should not be accurate; meanwhile, retaining each dataset is not a behavior that we want to inspire [21]. As a result, the role of unsupervised paths is complementary to the discriminative path through p in $h_u$ the data lost in $h_c$. Consequently, it produces $\widehat{x}$ complementary reconstruction, while, integrating $\widehat{x}$ and $x$, the last reconstruction $\widehat{x}$ is closer to $x$. The architecture of HybridNet is formulated by using the below expression:

$$h_c=E_c\left(x\right) {\widehat{x}}_c=D_c\left(h_c\right) \widehat{y}=C\left(h_c\right)$$

$$h_u=E_u\left(x\right) {\widehat{x}}_u=D_u\left(h_u\right) \widehat{x}={\widehat{x}}_c+{\widehat{x}}_u$$

It should be noted that the ultimate role of reconstruction is to regularize for the discriminative encoding. The major contribution and challenge of the study is to establish a method to guarantee that both paths would actually perform in such way.

The two major problems that we address are the discriminative path to emphasize discriminative features and the fact that we need these two paths to contribute and co-operate to the reconstruction. In fact, with this framework, we might create two paths that work individually: a reconstruction path $\widehat{x}={\widehat{x}}_u=D\left(E\left(x\right)\right)$ and a classification path $\widehat{y}=C\left(E\left(x\right)\right)$ and ${\widehat{x}}_c=0$. We resolve the issue by using the encoder and decoder architecture along with a proper training and loss function. The HybridNet model has two data paths, with one generating a class prediction and both generating partial reconstruction that needs to be integrated. In these subsections, we resolve the problem of training this structural design proficiently. It encompasses terms for stability with ${\Omega }_{stability};$ classification with ${ℒ}_{cls}$; last reconstruction with ${ℒ}_{rec}$; and intermediate reconstruction with ${ℒ}_{rec-interb,l}$ (for layer $l$ and branch $b$).

Moreover, it is followed by a branch complementarity training model. All the terms are weighted through a $\lambda$ variable, respectively:

$$ℒ={\lambda }_c{ℒ}_{cls}+{\lambda }_r{ℒ}_{rec}+{\displaystyle \sum }_{bϵ\left\{c,u\right\},l}{\lambda }_{rb,l}{ℒ}_{rec-interb,l}+{\lambda }_s{\Omega }_{stability}$$

(1)

HybridNet architecture is trained on partially labelled data comprised of unlabeled images ${\mathcal{D}}_{un\sup }={\left\{x^{\left(k\right)}\right\}}_{k=1..N_u}$ and labelled pairs ${\mathcal{D}}_{\sup }={\left\{\left(x^{\left(k\right)}, y^{\left(k\right)}\right)\right\}}_{k=1..N_s}$. All the batches are comprised of $n$ instances, separated into unlabeled images $n_u$ from ${\mathcal{D}}_{un\sup }$ and labelled images $n_s$ from ${\mathcal{D}}_{\sup }$. The classification term is a regular cross-entropy term employed only on the $n_s$-labelled instance, as follows:

$${\ell }_{cls}={\ell }_{CE}\left(\widehat{y}, y\right)=-{\displaystyle \sum }_i{y}_i\log \widehat{y},{ℒ}_{cls}=\frac{1}{n_s}{\displaystyle \sum }_k{\ell }_{cls}\left({\widehat{y}}^{\left(k\right)}, y^{\left(k\right)}\right)$$

(2)

3.3. Hyperparameter Optimization

In order to effectually tune the hyperparameters related to the HybirdNet model, the SSA is exploited. To resolve optimization problems, motion behavior of SSA can be mathematically modelled [22]. Salps are sea creatures that have barrel-shaped, jelly-like bodies and move from place to place by driving water through their bodies from one side to the other sides. They exist as colonies and travel together like chains. Leader and follower are the two most important classes of salps. Leaders lead the chain in a forwarding direction, while followers follow the leader synchronously and in harmony. Similar to a swarm intelligent model, SSA begins with an arbitrary initialization of the swarm of N salps. Variable n is considered to be measured, x symbolizes the position of salp, and y defines the food source-specifying objective of swarm in searching region. Leader salp describes their position by the subsequent formula:

$$x_{i^1}=\{\begin{array}{l}{y}_i+r_1\left(\left(u{b}_i-l{b}_i\right)r_2+l{b}_i\right), r_3\ge 0,\hfill \\ y_i-r_1\left(\left(u{b}_i-l{b}_i\right)r_2+l{b}_i\right), r_3<0,\hfill \end{array}$$

(3)

In Equation (3), in $i$-$\mathrm{th}$ parameter, $x_{i^1}$—position of initial salp; $y_i$—position of food. $u{b}_i$ and $l{b}_i$—upper and lower bounds, and $r_1,$$r_2,$$r_3$—arbitrary number.

Among three arbitrary numbers, $r_1$ inhabits the lead position because it balances exploitation and exploration at the time of searching process. It can be formulated as follows:

$$r_1=2e{(\frac{4I}{L})}^2$$

(4)

In Equation (4), $l$ shows existing iteration; $L$—the formerly determined amount of iterations; $r_2,$$r_3$—arbitrary integer lies within $\left[0, 1\right]$. To update the location according to Newton’s law of motion, the following mathematical expressions are utilized for followers:

$$X_l^j=0.5\lambda t^2+{\delta }_{0}t$$

(5)

where $\ge 2,$$x_l^j$—position of $j$-$\mathrm{th}$ salp in $i$-$\mathrm{th}$ parameter, $t$—time, ${\delta }_0$—initial speed.

$$\lambda =\frac{{\delta }_{f\iota nal}}{{\delta }_0},where \delta =\frac{x-x_0}{r}$$

(6)

Assume that ${\delta }_0=0,$ $t$—iteration in an optimization issue; the abovementioned formula is transformed into succeeding expression:

$$x_l^j=0.5\left(x_l^j+x_l^{j-1}\right)$$

(7)

In Equation (7), $j\ge 2$. This equation demonstrates that follower salps describe the position according to the preceding salps and their own position. When some salps escape from the restricted searching space, they are carried back within the limitation as follows:

$$X_l^j=\{\begin{array}{cc}{l}^j& if{x}_{\iota }^j\le l^j\\ u^j& i{f}_{X_l^j}\ge u^j\\ {\chi }_l^j& otherwise\end{array}$$

(8)

The abovementioned expression is repeatedly executed until the ending condition is met. Note that the food source is sometimes upgraded by exploring and exploiting space around an existing solution, which might determine the best solution. Salp chains, during optimization, have the capacity to move toward global optimum solutions as illustrated in Algorithm 1.

Algorithm 1 Pseudocode of SSA

1: Input: maximum iterations $L$, population size $m,ub,lb,l=I$ 2: Initialization of salp position $\left\{u_1, u_2, u_3, \dots \dots \dots , u_m\right\}$ 3: While (stopping criteria is not fulfilled) 4: Determine fitness of all salps 5: Arrange salp position based on fitness value 6: Define $F$ as optimal place for present population 7: Upgrade Cl 8: For every salp position ($u_i$) 9: If $\left(i\le m/2\right)$ upgrades the position of leading salps 10: Else upgrade the position of follower salp 11: end 12: end 13: Change the salp which crosses higher and lower limits 14: end 15: Display optimum output

3.4. Image Captioning

In this study, the decoding part includes the BiGRU model to generate descriptive sentences. A recurrent neural network (RNN) has been successfully used to handle data sequences in different areas [23]. In RNN, the input sequence $=\left(x_1, \dots , x_T\right)$, hidden vector sequence $h=\left(h_1, \dots , h_T\right)$, and output vector sequence $y=\left(y_1, \dots , y_T\right)$ are derived by the given equations:

$$h_{\tau }=\Phi \left(U{x}_t+WW{h}_{\tau -1}+b\right)$$

(9)

$$y_t=V{h}_t+c$$

(10)

Let $\Phi$ be the activation function, and the popular activation function is generally an element-wise application of the sigmoid function. $U$ refers to the input-hidden weight matrixes, $W$ stands for the hidden-hidden weight matrixes, and, in Equation (10), $b$ denotes the hidden bias vector, $V$ signifies the hidden-output weight matrixes, and $c$ denotes the output bias vector. It is nearly impossible to capture the long-term dependency of RNN, as the gradient tends to explode or vanish. Therefore, some research workers have made every effort to develop a more complex activation function to resolve the shortcomings. For instance, the LSTM unit is initially proposed for capturing the long-term dependency. In recent years, other variants of the recurrent unit, such as GRU, is also devised, which are easier to calculate and have good performance of generalization compared to that of the LSTM unit. Figure 2 depicts the framework of GRU.

LSTM makes use of an output gate for controlling the exposure of the quantity of memory content.

$$h_t=o_{t}tanh\left(c_t\right)$$

(11)

In Equation (11), the output gate is represented as $o_t$ and is calculated as follows:

$$0_t=\sigma \left(W_0\cdot \left[h_{t-1}, x_t, c_l\right]+b_0\right)$$

(12)

In Equation (12), the logistic function is indicated as $\sigma$. The memory cell $c_t$ is preserved by adding some new memories and eliminating (forgetting) current memories:

$$c_t=f_t{c}_{t-1}+i_{\tau }\tilde{c}+b_c$$

(13)

The ${\tilde{c}}_t$ new memories are given by:

$$\tilde{c}=tanh\left(W_c\cdot \left[h_{t-1}, x_t\right]\right)$$

(14)

The extent to add and remove memories can be controlled by the input gate $i_t$ and the forget gate $f_t$. The forget gate can be calculated by the following equation:

$$f_t=\sigma \left(W_f\cdot \left[h_{t-1}, x_t, c_{t-1}\right]+b_f\right)$$

(15)

and $i_t$ is calculated as follows:

$$i_t=\sigma \left(W_i\cdot \left[h_{t-1}, x_t, c_{t-1}\right]+b_i\right)$$

(16)

From the equation, the corresponding bias vector is indicated as $b$. As with the LSTM unit, GRU uses the gate for controlling the data stream inside a unit; however, there is no memory cell. The $h_r$ hidden state is a linear integration of new hidden states ${\tilde{h}}_t$ and the preceding hidden state $h_{t-1}$:

$$h_t=\left(1-z_t\right)h_{t-1}+z_t{\tilde{h}}_t$$

(17)

In Equation (17), the update gate $z_t$ controls how much its new activation is upgraded. It is calculated as follows:

$$z_t=\sigma \left(W_z\cdot \left[h_{t-1}, x_t\right]\right)$$

(18)

The ${\tilde{h}}_t$ new activation is calculated as follows:

$$\tilde{h}=tanh\left(W_h\cdot \left[r_t\odot h_{t-1}, x_t\right]\right)$$

(19)

In Equation (19), the forget gate $r_t$ is the same as the update unit in LSTM:

$$r_l=\sigma \left(W_r\cdot \left[h_{t-1}, x_t\right]\right)$$

(20)

While typical RNN exploits the preceding data, the bi-directional RNN (BRNN) processes information in two directions. The $y$ output of BRNN is attained by measuring the $\overrightarrow{h_r}$ forward hidden sequence and $\overleftarrow{h_t}$ backward sequence as follows:

$${\overrightarrow{h}}_t=\Phi \left(W{W}_{\chi \overrightarrow{h}}{x}_t+W_{\overrightarrow{h}\overrightarrow{h}}{\overrightarrow{h}}_{t-1}+b_{\overrightarrow{h}}\right)$$

(21)

$$\overleftarrow{h_t}=\Phi \left(W_{x\overleftarrow{h}}{x}_{t}xh+W_{\overleftarrow{h}\overleftarrow{h}}{\overleftarrow{h}}_{t-1}+b_{\overleftarrow{h}}\right)$$

(22)

$$y_t=W_{\overrightarrow{h}y}{\overrightarrow{h}}_t+W_{\overleftarrow{h}y}{\overleftarrow{h}}_{t-1}+b_y$$

(23)

Integrating BRNN with GRU provides BIGRU that is utilized for accessing the long-term data sequence in two directions. As a fault, a diagnoses issue can be generally regarded as a classification issue, and cross entropy is adapted as the loss function. The weighted cross entropy is represented as:

$$f\left(\theta \right)=-{\displaystyle \sum }_{n=1}^N{w}_n{\displaystyle \sum }_{i=1}^M{y}_{i}1og\left({\widehat{y}}_i\right)$$

(24)

In Equation (24), $\theta$ indicates the neural network parameter, $N$ represents the sample count, the number of faults is represented as $M$, and the true label can be indicated as $y_i$ and the predicted probability is represented as ${\widehat{y}}_i$.

4. Performance Validation

The experimental validation of the MODLE-AICT model is tested using the Flick8K dataset (https://www.kaggle.com/adityajn105/flickr8k/activity, accessed on 13 March 2022) and MS-COCO 2014 dataset [24]. A comparison study is also made with recent models [25,26,27,28,29,30,31]. A few sample images are depicted in

Table 1. Sample Images and its captions.

Sample Image	Different Captions
	A crowd watching air balloons at night
	A group of hot air balloons lit up at night
	People are watching hot air balloons in the park
	People watching hot air balloons
	Seven large balloons are lined up at night-time near a crowd
	A man climbs a rocky wall
	A climber wearing a blue helmet and headlamp is attached to a rope on the rock face
	A rock climber climbs a large rock
	A woman in purple snakeskin pants climbs a rock
	Person with blue helmet and purple pants is rock climbing
	People on ATVs and dirt bikes are traveling along a worn path in a field surrounded by trees
	Three people are riding around on ATVs and motorcycles
	Three people on motorbikes follow a trail through dry grass
	Three people on two dirt bikes and one four-wheeler are riding through brown grass
	Three people ride off-road bikes through a field surrounded by trees

. It contains 8000 images that are each paired with five different captions which provide clear descriptions of the salient entities and events.

4.1. Performance Measures

To validate the performance of the presented model, a set of four metrices are utilized, such as BLEU, Meter, CIDEr, and Rouge-L. BLEU [25], a commonly utilized metric for estimating the quality of the produced text. For an effectual image captioning outcome, BLEU values are required to be high, and it is defined using Equation (25):

$$BP=\min \left(1,e^{1-\frac{r}{c}}\right)$$

$$BLE{U}_N=BP\times e^{\frac{1}{N}{\displaystyle \sum }_{n=1}^{N}log{p}_n}$$

(25)

where BP denotes penalty factor, $r$ and $c$ represent length of the reference and generated sentences, respectively. METEOR metric relies on word recall rate and single precision weighted harmonic mean. It computes the reconciliation mean of accuracy and recalls amongst the optimum candidate and reference translations. It is defined as follows:

$$METEOR=\left(1-Pen\right)F_{mean}$$

(26)

where α, γ, and θ denotes default parameters.

The CIDEr index assumes each sentence as a ‘‘document’’ and represents in the form of a TF-IDF vector. It computes the cosine similarity amongst the created caption $\left(s_{ij}\right)$ and original caption by the use of a score value.

$$CIDE{r}_n\left(c_i,S_i\right)=\frac{1}{m}{\displaystyle \sum }_j\frac{g^n{\left(c_i\right)}^T{g}^n\left(s_{ij}\right)}{\Vert g^n\left(c_i\right)\Vert \times \Vert g^n\left(s_{ij}\right)\Vert }$$

(27)

ROUGE is another similarity measurement model that is mainly based on the recall rate. It determines the co-occurrence probability of N-gram in the reference translation and the translation to be investigated. It is defined using Equation (28).

$$ROUGE-N=\frac{{{\displaystyle \sum }}_{S\in \left\{ReferencesSummaries\right\}} {{\displaystyle \sum }}_{gra{m}_n\in S}Coun{t}_{match}\left(gra{m}_n\right)}{{{\displaystyle \sum }}_{S\in \left\{ReferencesSummaries\right\}} {{\displaystyle \sum }}_{gra{m}_n\in S}Count\left(gra{m}_n\right)}$$

(28)

4.2. Result Analysis

Table 2. Result analysis of MODLE-AICT algorithm with approaches on Flickr8K dataset.

Methods	BLEU-1	BLEU-2	BLEU-3	BLEU-4
M-RNN Model [24]	59.18	29.09	24.17	14.19
G-NICG Model [26]	64.13	42.64	27.11	16.12
L-Bilinear Model [27]	65.96	43.29	28.63	18.49
DVS Model [28]	58.35	37.98	25.54	17.08
ResNet50 Model [23]	62.65	46.28	37.26	26.16
VGA-16 Model [23]	67.69	44.34	33.99	23.25
HPTDL Model [25]	68.26	46.16	37.81	26.71
MODLE-AICT	69.06	47.26	38.78	27.80

and Figure 3 inspect a detailed result analysis of the MODLE-AICT model on the test Flickr8K dataset [23,24,25,26,27,28]. The results implied that the MODLE-AICT model has gained effectual outcomes over other models. For instance, based on BLEU-1, the MODLE-AICT model obtained a higher BLEU-1 of 69.06, whereas the M-RNN, G-NICG, L-Bilinear, DVS, ResNet50, VGA-16, and HPTDL models attained a lower BLEU-1 of 59.18, 64.13, 65.96, 58.35, 62.65, 67.69, and 68.26, respectively. At the same time, based on BLEU-4, the MODLE-AICT technique has attained a higher BLEU-4 of 27.80, whereas the M-RNN, G-NICG, L-Bilinear, DVS, ResNet50, VGA-16, and HPTDL methods have acquired a lower BLEU-4 of 14.19, 16.12, 18.49, 17.08, 26.16, 23.25, and 26.71, correspondingly.

Table 3. Classification analysis of MODLE-AICT algorithm with approaches on Flickr8K dataset.

Methods	METEOR	CIDEr	Rouge-L
SCST-IN Model [29]	20.00	161.00	49.00
SCST-ALL Model [29]	23.00	154.00	42.00
G-NIC Model [26]	19.00	153.00	43.00
A-NIC Model [26]	21.00	160.00	48.00
DenseNet Model [24]	25.00	173.00	43.00
HPTDL Model [25]	28.00	175.00	46.00
MODLE-AICT	30.00	179.00	53.00

and Figure 4 examine a detailed classification analysis of the MODLE-AICT system on the test Flickr8K dataset. The results implied that the MODLE-AICT technique has attained effectual outcomes over other models. For example, based on METEOR, the MODLE-AICT approach has gained higher METEOR of 30, whereas the SCST-IN, SCST-ALL, G-NIC, A-NIC, DenseNet, and HPTDL methodologies have obtained a lower METEOR of 20, 23, 19, 21, 25, and 28, correspondingly. Meanwhile, based on Rouge-L, the MODLE-AICT approach has received a higher Rouge-L of 53, whereas the SCST-IN, SCST-ALL, G-NIC, A-NIC, DenseNet, and HPTDL algorithms have attained a lower Rouge-L of 49, 42, 43, 48, 43, and 46, correspondingly.

A comparison study of the MODLE-AICT model with recent models on the Flickr8K dataset is shown in Figure 5. The figure implied that the SCST-IN and SCST-ALL models have obtained lower performance than other models. This was followed by the G-NIC, A-NIC, and DenseNet models, which attained moderately closer results. Along with that, the HPTDL model accomplished a reasonable performance. However, the MODLE-AICT model has shown enhanced performance over other models on the test Flickr8K dataset.

The training accuracy (TA) and validation accuracy (VA) attained by the MODLE-AICT approach on the Flickr8K dataset is demonstrated in Figure 6. The experimental outcome implied that the MODLE-AICT technique has gained maximum values of TA and VA. Specifically, the VA seemed to be higher than TA.

The training loss (TL) and validation loss (VL) achieved by the MODLE-AICT methodology on the Flickr8K dataset are established in Figure 7. The experimental outcome inferred that the MODLE-AICT system accomplished the lowest values of TL and VL. Specifically, the VL seemed to be lower than TL.

Table 4. Result analysis of MODLE-AICT algorithm with approaches on MS-COCO 2014 dataset.

Methods	BLEU-1	BLEU-2	BLEU-3	BLEU-4
KNN Model [25]	49.60	28.60	17.00	10.95
G-NICG Model [26]	67.92	46.23	34.03	24.94
L-Bilinear Model [27]	71.75	49.22	34.65	24.35
DVS Model [28]	63.86	44.98	32.58	23.29
ResNet50 Model [23]	73.57	57.21	42.05	32.52
VGA16 Model	70.30	53.72	40.45	30.06
VGA-16 Model [23]	74.28	59.39	43.33	33.96
HPTDL Model [25]	75.12	60.21	44.22	34.75

and Figure 8 depict the detailed results of the analysis of the MODLE-AICT system on the test MS-COCO 2014 dataset. The results implied that the MODLE-AICT approach obtained effectual outcomes over other models. For instance, based on BLEU-1, the MODLE-AICT method gained a higher BLEU-1 of 75.12, whereas the M-RNN, G-NICG, L-Bilinear, DVS, ResNet50, VGA-16, and HPTDL methodologies received a lower BLEU-1 of 49.60, 67.92, 71.75, 63.86, 73.57, 70.30, and 74.28, correspondingly. Nonetheless, based on BLEU-4, the MODLE-AICT technique has gained a higher BLEU-4 of 34.75, whereas the M-RNN, G-NICG, L-Bilinear, DVS, ResNet50, VGA-16, and HPTDL methodologies acquired a lower BLEU-4 of 10.95, 24.94, 24.35, 23.29, 32.52, 30.06, and 33.96, correspondingly.

Table 5. Classification analysis of MODLE-AICT algorithm with approaches to MS-COCO 2014 dataset.

Methods	METEOR	CIDEr	Rouge-L
SCST-IN Model [29]	22.00	109.00	51.00
SCST-ALL Model [29]	25.00	114.00	59.00
G-NIC Model [26]	21.00	111.00	51.00
A-NIC Model [26]	24.00	110.00	58.00
DenseNet Model [24]	24.00	122.00	57.00
HPTDL Model [25]	34.00	125.00	60.00
MODLE-AICT	37.00	129.00	63.00

and Figure 9 review a detailed classification analysis of the MODLE-AICT system on the test MS-COCO 2014 dataset. The results implied that the MODLE-AICT methodology acquired effectual outcomes over other models. For instance, based on METEOR, the MODLE-AICT methods have obtained a higher METEOR score of 37, whereas the SCST-IN, SCST-ALL, G-NIC, A-NIC, DenseNet, and HPTDL approaches have attained a lower METEOR score of 22, 25, 21, 24, 24, and 34, correspondingly. Meanwhile, based on Rouge-L, the MODLE-AICT technique acquired a higher Rouge-L of 63, whereas the SCST-IN, SCST-ALL, G-NIC, A-NIC, DenseNet, and HPTDL algorithms acquired a lower Rouge-L of 51, 59, 51, 58, 57, and 60, correspondingly.

A comparison study of the MODLE-AICT technique with recent models on the MS-COCO 2014 dataset is shown in Figure 10. The figure implied that the SCST-IN and SCST-ALL methodologies acquired a lower performance than the other models. Then, the G-NIC, A-NIC, and DenseNet approaches have gained moderately closer results. Moreover, the HPTDL approach has tried to accomplish reasonable performance. However, the MODLE-AICT system has shown an enhanced performance over other models on the test MS-COCO 2014 dataset.

The TA and VA attained by the MODLE-AICT technique on the MS-COCO 2014 dataset are demonstrated in Figure 11. The experimental outcome implied that the MODLE-AICT method has gained maximum values of TA and VA. Specifically, the VA seemed to be higher than TA.

The TL and VL achieved by the MODLE-AICT approach on the MS-COCO 2014 dataset are established in Figure 12. The experimental outcome inferred that the MODLE-AICT methodology has accomplished least values of TL and VL. Specifically, the VL seemed to be lower than TL. From the detailed results and discussion, it is assured that the proposed model has shown effective outcomes on the image captioning process.

5. Conclusions

In this study, a novel MODLE-AICT technique was developed for the generation of effective captions to inputted images using two processes involving an encoding unit and a decoding unit. Primarily, at the encoding part, the SSA with a HybridNet model is utilized to generate effectual input image representation using fixed-length vectors. In addition, the decoding part includes a BiGRU model used to generate descriptive sentences. The inclusion of an SSA-based hyperparameter optimizer helps in attaining effectual performance. For inspecting the enhanced performance of the MODLE-AICT model, a series of simulations are carried out, and the results are examined under several aspects. The experimental values implied the betterment of the MODLE-AICT model over recent approaches. Thus, the presented MODLE-AICT technique can be exploited as an effectual approach for image captioning. In future, ensemble DL-based fusion models can be designed to enhance the performance.