Image Fusion Techniques Based on Optimization Algorithms: A Review ^†

Abstract

In image processing applications, image fusion techniques gain popularity because they combine the most appropriate features of different source images in order to generate a single image that contains more information and is more beneficial. In this paper, initially, we have analysed the conventional spatial and transform domain image fusion techniques. These techniques face numerous challenges, such as low contrast, noise, and redundancy. To overcome these challenges, adaptive image fusion methods using nature-inspired optimization algorithms (YSGA) are deployed. These algorithms search for the optimal solution for the image fusion technique based on the objective function. Therefore, the main focus of this paper is to study and analyse the optimization algorithms based on various factors.

1. Introduction

Image fusion is a technique that amalgamates multiple data sets from related observations to produce a composite output featuring key aspects of each component. This process involves combining various sensory, temporal, and view images into a single, high-quality image, resulting in enhanced spatial, temporal, and spectral information compared to individual images [1]. The core objective of image fusion is to retain significant aspects of the input images while eliminating unwanted data. This method accommodates images with diverse spectral and spatial properties obtained at different times and using various sensors. Two domains, spatial and frequency, are utilized for image fusion. The spatial domain processes image pixels directly, leading to spatial distortion in the fused images [2]. Hence, frequency domain fusion is preferred, even though it presents shift invariance challenges [3]. Various optimization strategies, including genetic algorithms (GA) [4,5], grasshopper optimization (GO) [6], grey wolf optimization, particle swarm optimization [7], and hybrid combinations [8] are explored to address these issues. The effectiveness of these methods depends on their exploration and exploitation rates, with hybridization increasing computational complexity. Alternative optimization techniques with improved exploration and exploitation rates are sought. One possible solution among these is the Yellow Saddle Goatfish (YSGA) algorithm, which draws inspiration from the hunting behaviour of Yellow Saddle Goatfish.

2. Review of the Literature

In this section, we have reviewed and analysed existing image fusion methods from the literature.

In order to improve fusion efficacy, Shaik Shehanaz, Ebenezer Daniel, and Sivaji Satrasupalli [7] describe an optimal weighted average fusion (OWAF) technique that makes use of particle swarm optimization (PSO). They use the DWT method to split the input from different modalities into subgroups, then PSO is used to reweight the energy bands. MRI-PET, MRI-SPECT, and MRI-CT images are used to assess the approach, which shows robustness against noise and a reduction in calculation time. Using an optimal homomorphic wavelet fusion (OHWF) is recommended by Ebenezer Daniel [8] for the integration of multimodal medical pictures. With the use of the MR-PET, MR-SPECT, MR T1-T2, and MR-CT modalities, this method improves fusion quality by combining homomorphic filtering and wavelet transform. In comparison to other fusion approaches, grey wolf optimization (HG-GWO) produces better outcomes by choosing the optimal scale values based on metrics such as MI, QAB/F STD, and entropy. Tsagaris and Anastassopoulos [9] propose an information-based global measure (IFPM) using mutual information for quantitative evaluation of image fusion methods. This metric ensures that shared information in input photos is considered only once in producing the fused image. Huang and Yang [10] investigate a number of techniques, such as deep learning, transform domain, and spatial domain methods, for merging multimodal medical pictures. Every approach has benefits and drawbacks, with deep learning exhibiting potential despite its limits in terms of data and processing power.

Huang and Yang [10] explore various approaches to fusing multimodal medical images, including deep learning, transform domain, and spatial domain methods. Each method has its advantages and disadvantages, with deep learning showing promise but having limitations in terms of data and computational resources. With an emphasis on high- and low-frequency signal analysis, Kairui Cheng et al. [11] compare wavelet transform-based fusion with Laplace pyramid image fusion. For these signals, they employ various fusion criteria, and they assess performance with respect to quality metrics such as spatial frequency, mean gradient, information entropy, and standard deviation. In this work, wavelet-based fusion seems to have more advantages.

An optimal search approach can be used to describe the hunting strategy for Yellow Saddle Goatfish that Zaldívar, Morales, Rodríguez, Valdivia-G, Cuevas, and Pérez-Cisneros [12] introduce. The fitness value of the two search agent groups—chasers and blockers—used in the method is used to determine how effective the hunting is. This technique has been tried in numerous engineering optimization jobs, demonstrating its effectiveness, precision, and durability. It also demonstrates enhanced optimization results in precision and convergence.

3. Methodology

The medical industry has advanced to the point that a variety of devices that create digital images of the human body are available for purchase, including CT scan machines, MRI machines, and ultrasound equipment. Doctors analyse these pictures to determine different illnesses [13]. Consequently, crisp, high-quality images are needed. In the medical field, image fusion techniques are needed to accomplish this goal. Several fusion techniques are examined and analysed, and difficulties are discovered in this study. Adaptive image fusion is developed utilizing nature-inspired algorithms in response to these issues. Both colour and grayscale image can be processed with the suggested technique. The suggested solution uses a nature-inspired algorithm to tune parameters based on the objective function. The program, which draws inspiration from nature, searches the solution space to find the best solution. As a result, the suggested method varies nature-inspired algorithm parameters like population and iterations and analyses the effect on the final images. The flowchart of the suggested method is shown in Figure 1. input image1 and input image2 are read at the start. The coefficients from both images are then extracted using the first processing model. The optimal weights for the coefficients are then determined by running them through the Nature Inspired (NIA) algorithm. Once the optimal weight values are determined, the coefficients are updated and combined. The coefficients of the two images are fused using the average fusion method. An inverse transform is employed in the last stage to produce an output that is a fused image. Once image fusion is complete, assess the combined images’ performance using several metrics, such as mutual information (MI), peak signal to noise ratio (PSNR), and root mean square error (RMSE).

Yellow Saddle Goatfish Algorithm

The YSG algorithm, a meta-heuristic software that imitates the Yellow Saddle Goatfish’s pack hunting approach, was proposed by Zaldvar et al. in 2018. As shown in Figure 2. There are five steps in the YSG algorithm, which are covered below:

Initial stage: As P ( ), a population of n goatfish is formed at random and uniformly distributed inside the dimensional search space’s perimeter. This is how the initialization is written:

$$p_j^i=rand·\left(b_i^{high}-b_i^{low}\right)+b_i^{low}$$

where $j=1,2,3\dots .n$ (population); $i=1,2,3\dots .d$ (dimension), when the random distribution between is (population); (dimension). The element wise multiplication is represented by a dot. The elementwise multiplication is represented by a dot.

Chaser fish: The YSGA are led in their hunts by a single chaser fish. Which fish will act as the chaser is determined by fitness levels. A fish that is closer to the answer is chosen to be the chaser fish in each group. This chaser fish uses a random stroll to try to locate its target.

Blocker fish: After selecting the chaser fish, every other fish turns into the blocker fish. Blocker fish movement is modelled as an algorithmic spiral. One fish that is closest to its victim is a chaser fish. Blocker fish follow the chaser fish in a spiral pattern, and each blocker fish takes a different path after each iteration.

Exchange of role: Once the blocker and chaser fish are decided, the fish that pursues its prey is situated closer to it. Blocker fish try to stop their victim from escaping. As the prey goes through the hunting area, roles are switched. The blocker fish transforms into the chaser fish if the prey is nearby, and vice versa. An exchange of roles is what is meant by this phenomenon. The element in the algorithm that has the highest fitness is selected as the chaser fish.

Change of zone: The fish group moves to the adjacent sector to look for new prey after fully utilizing the area by hunting every prey. It is shown as:

$$p_g^{t+1}=\frac{{\varnothing }_{best}+p_g^t}{2}$$

where $p_g^{t+1}$ is the fish’s new location. ${\varnothing }_{best}$ is the cluster with the most effective response, $p_g^t$ is the present state of the fish (chaser or blocker). If the value of a chaser fish has increased within a cluster, it gets swapped out for the best worldwide. The local optima are avoided by this method.

4. Results and Discussion

For the diagnosis and evaluation of medical issues, digital image processing is becoming more and more prevalent. A high-quality digital image is needed for the diagnosis. The qualities of the images vary from image to image in reality, though. Consequently, an adaptive-based image fusion model is suggested in this study. Adaptive pre-processing techniques were needed for the image based on its characteristics since they begin to process the image depending on how much noise, enhancement, or fusion is needed. The effectiveness of an image fusion can be assessed using essentially two different sorts of techniques [14]:

1. Qualitative approach: This method compares the original and fused images visually to determine the spectral content. But, since these measurements are subjective, they are typically not preferred.

2. Quantitative approach: YSGO algorithm. A collection of defined image metrics will be used to measure the spectral and spatial similarity between the fused image and the input images. For assessing the visual information, these criteria are typically preferred. The metrics used in this method include the standard deviation, peak signal-to-noise ratio (PSNR), mean squared error (MSE), root mean square error (RMSE), cross correlation (CC), mutual information (MI), and spectral distortion index (SDI). The various performance metrics parameters used to evaluate the efficiency of the Image Fusion technique are displayed in

Table 1. Performance metrics.

S. No	Parameter	Equation
1.	Root Mean Square Error (RMS): The spectral quality can be seen in this parameter. The difference between the reference image and the fused image is used to calculate it.	$RMSE=\sqrt{\frac{\sum _{i=1}^M\sum _{j=1}^N{(I_R\left(i,j\right)-I_F\left(i,j\right))}^2}{MN}}$ [15] $\mathrm{where}MN$$\mathrm{denotes}\mathrm{the}\mathrm{size}\mathrm{of}\mathrm{the}\mathrm{image}.I_R$$\mathrm{and}{I}_F$ is the referenced and fused image.
2.	Peak-Signal-to-Noise-Ratio (PSNR): to determine the overall quality of the fused image, this parameter is computed to its maximum value in image fusion methods.	$PSNR=10{\log }_{10}\frac{L^2}{MSE}$ [16] where $MSE$ denotes the mean square error and $L$ denotes the number of grey levels in the image. In a grey scale image, its value is $255$.
3.	Mutual Information (MI) is a parameter used to compare reference and fused image similarity. The more details and textual information there are, the better the value of MI.	$MI=\sum _{i=1}^M\sum _{j=1}^N{hI}_R{I}_F(i,j)\times {\log }_2\left(\frac{{hI}_R{I}_F(i,j)}{{hI}_R\left(i,j\right)h{I}_F(i,j)}\right)$ [17] where ${hI}_R{I}_F$ denotes the joint grey level histogram of $I_R$ and $I_{F.}$
4.	Structural Similarity Index Measure (SSIM): This metric assesses how comparable the reference and fused images’ structures are. Its value fluctuates between −1 and 1. Value 1 demonstrates that structural information is preserved and that both images are similar.	$SSIM=\frac{(2{\mu }_{I_R}{\mu }_{I_F}+C_1)\left(2{\sigma }_{I_R{I}_F+}{C}_2\right)}{({{\mu }_{I_R}}^2+{{\mu }_{I_F}}^2+C_1)({{\mu }_{I_R}}^2+{{\mu }_{I_F}}^2+C_2)}$ [17] where ${\mu }_{I_R}{\mu }_{I_F}$ means that the referenced and fused image’s mean intensity values are shown. $C_1{C}_2$ are constant.
5.	Entropy: the average quantity of information contained in the fusion image is indicated by the size of information entropy.	$E={\sum }_{i=1}^L{p}_i{\log }_2{p}_i$ $p_i\mathrm{is}\mathrm{the}\mathrm{ratio}\mathrm{of}\mathrm{the}\mathrm{number}\mathrm{of}\mathrm{pixels}\mathrm{whose}\mathrm{Gray}\mathrm{value}\mathrm{is}i\mathrm{to}\mathrm{the}\mathrm{total}\mathrm{number}\mathrm{of}\mathrm{pixels}\mathrm{in}\mathrm{the}\mathrm{image},\mathrm{and}\mathrm{satisfies}{\sum }_{i=1}^L{p}_i$ = 1. The higher value of entropy represents the maximum information contained by image [18].

.

5. Conclusions

An examination of the literature revealed that the high complexity and inconsistent nature of all image fusion techniques are problems. Each image has a different set of qualities [19,20]. As a result, the typical fusion process has drawbacks with fused images such as unclear images, blurry effects, and level of fusion. The results of multimodal medical image fusion research are becoming more and more important, yet issues with colour distortion and feature extraction in the fusion effect are still not entirely resolved. Weights were selected as the coefficients in order to overcome the issues that were previously highlighted. Primary weighted average fusion may not produce a meaningful fused image because of the weight value that was assigned to the fusion process. Therefore, choosing the right weights for multi-level decomposition components through optimal weight selection may improve performance.