# Multispectral and SAR Image Fusion Based on Laplacian Pyramid and Sparse Representation

^{1}

^{2}

^{3}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Proposed Fusion Framework

#### 2.1. LP Generation and Reconstruction

#### 2.2. High-Frequency COMPONENTS Fusion

#### 2.3. Low-Frequency Component Fusion

- (1)
- Patch generation. To make full use of the local information of source images, the sliding window technique is applied to divide the source images A and B into image patches ${\mathit{p}}_{k}^{A},{\mathit{p}}_{k}^{B}\text{}(k=1,2,\cdots ,K\text{}\mathrm{and}\text{}K=(M-\sqrt{n}+1)\times (N-\sqrt{n}+1))$ of size $\sqrt{n}\times \sqrt{n}$, starting from the top-left to the bottom-right with a fixed step length $s$.
- (2)
- Vectorization. Then, image patches ${\mathit{p}}_{k}^{A},{\mathit{p}}_{k}^{B}$ are rearranged to $n\times 1$ vectors ${\mathit{v}}_{k}^{A},{\mathit{v}}_{k}^{B}$ in a column-wise way. Each vector is normalized to zero-mean via subtracting the mean value according to the following Equation (3), and the mean values are stored for subsequent reconstruction process [47],$$\{\begin{array}{l}{\widehat{\mathit{v}}}_{k}^{A}={\mathit{v}}_{k}^{A}-{\overline{v}}_{k}^{A}\cdot 1\\ {\widehat{\mathit{v}}}_{k}^{B}={\mathit{v}}_{k}^{B}-{\overline{v}}_{k}^{B}\cdot 1\end{array}$$
- (3)
- Sparse coding. Calculate the sparse coefficients of vectors ${\widehat{\mathit{v}}}_{k}^{A}$ and ${\widehat{\mathit{v}}}_{k}^{B}$ according to Equation (4) using the simultaneous orthogonal matching pursuit (SOMP) algorithm [48]. The SOMP algorithm is employed here for its high computing efficiency and suitability for image fusion,$$\{\begin{array}{l}{\mathit{\alpha}}_{k}^{A}=\mathrm{arg}\underset{\alpha}{\mathrm{min}}{\Vert \mathit{\alpha}\Vert}_{0}s.t.{\Vert {\widehat{\mathit{v}}}_{k}^{A}-D\mathit{\alpha}\Vert}_{2}^{2}\le \epsilon \\ {\mathit{\alpha}}_{k}^{B}=\mathrm{arg}\underset{\alpha}{\mathrm{min}}{\Vert \mathit{\alpha}\Vert}_{0}s.t.{\Vert {\widehat{\mathit{v}}}_{k}^{B}-D\mathit{\alpha}\Vert}_{2}^{2}\le \epsilon \end{array}$$
- (4)
- Coefficient fusion. The activity level measurement and fusion rule are two important issues in image fusion tasks [47]. In this paper, the absolute value of the sparse coefficient is chosen to describe the activity level, and the popular max-absolute rule is selected as the fusion rule to combine the corresponding sparse coefficients. The detailed fusion process can be described by Equation (5):$${\mathit{\alpha}}_{k}^{F}(t)=\{\begin{array}{l}{\mathit{\alpha}}_{k}^{A}(t),\mathrm{if}\text{}\left|{\mathit{\alpha}}_{k}^{A}(t)\right|\ge \left|{\mathit{\alpha}}_{k}^{B}(t)\right|\\ {\mathit{\alpha}}_{k}^{B}(t),\mathrm{otherwise}\end{array}$$
- (5)
- Vector reconstruction. The fused sparse vector ${\widehat{\mathit{v}}}_{k}^{F}$ is obtained via the fused sparse coefficient ${\mathit{\alpha}}_{k}^{F}$ multiplied by the same dictionary used in Step (3). The local mean subtracted in Step (2) is added back, and the final fused vector ${\mathit{v}}_{k}^{F}$ is obtained.$${\widehat{\mathit{v}}}_{k}^{F}=D{\mathit{\alpha}}_{k}^{F}$$$${\overline{v}}_{k}^{F}=\frac{1}{2}({\overline{v}}_{k}^{A}+{\overline{v}}_{k}^{B})$$$${\mathit{v}}_{k}^{F}={\widehat{\mathit{v}}}_{k}^{F}+{\overline{v}}_{k}^{F}\cdot 1$$
- (6)
- Final reconstruction. Every fused sparse vector ${\mathit{v}}_{k}^{F}$ is reshaped to a $\sqrt{n}\times \sqrt{n}$ patch and placed in the corresponding position in the fused image F. As the patches may be overlapped, the same pixel in the source image may appear in multiple patches. In other words, one position in F may relate to multiple patches. Therefore, each pixel’s value in the fused image F is the average value of the corresponding elements in all related patches. Finally, the fused low-frequency component ${L}_{N}^{F}$ is obtained.

## 3. Experiments

#### 3.1. Experiment Settings

#### 3.1.1. Data Description

#### 3.1.2. Evaluation Metrics

#### 3.1.3. Comparison Methods

#### 3.2. Experimental Results

_{full}), structural similarity (SSIM), and feature mutual information (FMI). We selected them because they measure the spectral preservation, spatial enhancement, and information incorporation abilities. Moreover, the value ranges of them are similar. The weights are equal for them because we considered the abilities of spectral preservation, spatial enhancement and information incorporation equally in this paper. Figure 7 illustrates the CI values of all the methods in three areas, from which we can get a more intuitive impression on the quantitative performance. The proposed LPSR fusion method obtains the highest CI values in all the three areas, which indicates that our method achieves the best performances from a comprehensive angle considering spectral, spatial, and information characteristics. In addition, the order of CI values highly accords with visual inspection results, which validates the effectiveness of the proposed index.

## 4. Discussion

#### 4.1. Adjustment Capability

#### 4.2. Time Complexity

## 5. Conclusions

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Acknowledgments

## Conflicts of Interest

## References

- Pohl, C.; van Genderen, J.L. Review article multisensor image fusion in remote sensing: Concepts, methods and applications. Int. J. Remote Sens.
**1998**, 19, 823–854. [Google Scholar] [CrossRef] [Green Version] - Schmitt, M.; Zhu, X.X. Data fusion and remote sensing: An ever-growing relationship. IEEE Geosci. Remote Sens. Mag.
**2016**, 4, 6–23. [Google Scholar] [CrossRef] - Pastorino, M.; Montaldo, A.; Fronda, L.; Hedhli, I.; Moser, G.; Serpico, S.B.; Zerubia, J. Multisensor and Multiresolution Remote Sensing Image Classification through a Causal Hierarchical Markov Framework and Decision Tree Ensembles. Remote Sens.
**2021**, 13, 849. [Google Scholar] [CrossRef] - Tong, X.; Luo, X.; Liu, S.; Xie, H.; Chao, W.; Liu, S.; Liu, S.; Makhinov, A.; Makhinova, A.; Jiang, Y. An approach for flood monitoring by the combined use of Landsat 8 optical imagery and COSMO-SkyMed radar imagery. J. Photogramm. Remote Sens.
**2018**, 136, 144–153. [Google Scholar] [CrossRef] - Zhang, Y.; Zhang, H.; Lin, H. Improving the impervious surface estimation with combined use of optical and SAR remote sensing images. Remote Sens. Environ.
**2014**, 141, 155–167. [Google Scholar] [CrossRef] - Bao, N.; Li, W.; Gu, X.; Liu, Y. Biomass Estimation for Semiarid Vegetation and Mine Rehabilitation Using Worldview-3 and Sentinel-1 SAR Imagery. Remote Sens.
**2019**, 11, 2855. [Google Scholar] [CrossRef] [Green Version] - Samadhan, C.K.; Priti, P.R. Pixel Level Fusion Techniques for SAR and Optical Images: A Review. Inf. Fusion
**2020**, 59, 13–19. [Google Scholar] - Nomura, R.; Oki, K. Downscaling of MODIS NDVI by Using a Convolutional Neural Network-Based Model with Higher Resolution SAR Data. Remote Sens.
**2021**, 13, 732. [Google Scholar] [CrossRef] - Kong, Y.; Hong, F.; Leung, H.; Peng, X. A Fusion Method of Optical Image and SAR Image Based on Dense-UGAN and Gram–Schmidt Transformation. Remote Sens.
**2021**, 13, 4274. [Google Scholar] [CrossRef] - Jiang, M.; Shen, H.; Li, J.; Zhang, L. An Integrated Framework for the Heterogeneous Spatio-Spectral-Temporal Fusion of Remote Sensing Images. arXiv
**2021**, arXiv:2109.00400. [Google Scholar] - Lin, J.; Zhao, L.; Wang, Q.; Ward, R.; Wang, Z.J. DT-LET: Deep transfer learning by exploring where to transfer. Neurocomputing
**2020**, 390, 99–107. [Google Scholar] [CrossRef] [Green Version] - Lin, J.; Mou, L.; Yu, T.; Zhu, X.; Wang, Z.J. Dual Adversarial Network for Unsupervised Ground/Satellite-To-Aerial Scene Adaptation. In Proceedings of the 28th ACM International Conference on Multimedia, Seattle, WA, USA, 12–16 October 2020. [Google Scholar]
- Lin, J.; Yu, T.; Mou, L.; Zhu, X.; Wang, Z.J. Unifying top–down views by task-specific domain adaptation. IEEE Trans. Geosci. Remote Sens.
**2020**, 59, 4689–4702. [Google Scholar] [CrossRef] - Harris, J.R.; Murray, R.; Hirose, T. IHS transform for the integration of radar imagery with other remotely sensed data. Photogramm. Eng. Remote Sens.
**1990**, 56, 1631–1641. [Google Scholar] - Yonghong, J.; Deren, L. Comparison of IHS transformation for integrating SAR and TM images. J. Remote. Sens. Land Res.
**1997**, 9, 34–39. [Google Scholar] - Shettigara, V.K. A generalized component substitution technique for spatial enhancement of multispectral images using a higher resolution data set. Photogramm. Eng. Remote Sens.
**1992**, 58, 561–567. [Google Scholar] - Yonghong, J. Fusion of landsat TM and SAR images based on principal component analysis. Remote Sens. Technol. Appl.
**2012**, 13, 46–49. [Google Scholar] - Laben, C.; Brower, B. Process for Enhancing the Spatial Resolution of Multispectral Imagery Using Pan-Sharpening. U.S. Patent Application 6,011,875, 4 January 2000. [Google Scholar]
- Gillespie, A.R.; Kahle, A.B.; Walker, R.E. Color enhancement of highly correlated images. I. Decorrelation and HSI contrast stretches. Remote Sens. Environ.
**1986**, 20, 209–235. [Google Scholar] [CrossRef] - Garzelli, A. Wavelet-based fusion of optical and SAR image data over urban area. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci.
**2002**, 34, 59–62. [Google Scholar] - Tateishi, R.; Wikantika, K.; Munadi, K.; Aslam, M. Study on the spectral quality preservation derived from multisensor image fusion techniques between JERS-1 SAR and Landsat TM data. In Proceedings of the 2003 IEEE International Geoscience and Remote Sensing Symposium, IGARSS ’03, Toulouse, France, 21–25 July 2003. [Google Scholar]
- Abdikan, S. Exploring image fusion of ALOS/PALSAR data and LANDSAT data to differentiate forest area. Geocarto Int.
**2018**, 33, 21–37. [Google Scholar] [CrossRef] - Burt, P.J.; Adelson, E.H. Merging images through pattern decomposition. Proc. SPIE Appl. Digit. Image Process.
**1985**, 575, 173–181. [Google Scholar] - Liu, Y.; Liu, S.; Wang, Z. A general framework for image fusion based on multi-scale transform and sparse representation. Inf. Fusion
**2015**, 24, 147–164. [Google Scholar] [CrossRef] - Cheng, J.; Liu, H.; Liu, T.; Wang, F.; Li, H. Remote sensing image fusion via wavelet transform and sparse representation. ISPRS J. Photogramm. Remote Sens.
**2015**, 104, 158–173. [Google Scholar] [CrossRef] - Zhang, W.; Yu, L. SAR and Landsat ETM+ image fusion using variational model. In Proceedings of the 2010 International Conference on Computer and Communication Technologies in Agriculture Engineering, Chengdu, China, 12–13 June 2010. [Google Scholar]
- Huang, B.; Li, Y.; Han, X.; Cui, Y.; Li, W.; Li, R. Cloud removal from optical satellite imagery with SAR imagery using sparse representation. IEEE Geosci. Remote Sens. Lett.
**2015**, 12, 1046–1050. [Google Scholar] [CrossRef] - Alparone, L.; Baronti, S.; Garzelli, A.; Nencini, F. Landsat ETM+ and SAR image fusion based on generalized intensity modulation. IEEE Trans. Geosci. Remote Sens.
**2004**, 42, 2832–2839. [Google Scholar] [CrossRef] - Chibani, Y. Additive integration of SAR features into multispectral SPOT images by means of the à trous wavelet decomposition. J. Photogramm. Remote Sens.
**2006**, 60, 306–314. [Google Scholar] [CrossRef] - Hong, G.; Zhang, Y.; Mercer, B. A wavelet and IHS integration method to fuse high resolution SAR with moderate resolution multispectral images. Photogramm. Eng. Remote Sens.
**2009**, 75, 1213–1223. [Google Scholar] [CrossRef] - Yin, Z. Fusion algorithm of optical images and SAR with SVT and sparse representation. Int. J. Smart Sens. Intell. Syst.
**2015**, 8, 1123–1141. [Google Scholar] - Shao, Z.; Wu, W.; Guo, S. IHS-GTF: A Fusion Method for Optical and Synthetic Aperture Radar Data. Remote Sens.
**2020**, 12, 2796. [Google Scholar] [CrossRef] - Wright, J.; Yang, A.Y.; Ganesh, A.; Sastry, S.S.; Ma, Y. Robust face recognition via sparse representation. IEEE Trans. Pattern Anal. Mach. Intell.
**2008**, 31, 210–227. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Elad, M.; Aharon, M. Image denoising via sparse and redundant representations over learned dictionaries. IEEE Trans. Image Process.
**2006**, 15, 3736–3745. [Google Scholar] [CrossRef] [PubMed] - Yang, J.; Wright, J.; Huang, T.S.; Ma, Y. Image super-resolution via sparse representation. IEEE Trans. Image Process.
**2010**, 19, 2861–2873. [Google Scholar] [CrossRef] [PubMed] - Olshausen, B.A.; Field, D.J. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature
**1996**, 381, 607–609. [Google Scholar] [CrossRef] [PubMed] - Bruckstein, A.M.; Donoho, D.L.; Elad, M. From sparse solutions of systems of equations to sparse modeling of signals and images. SIAM Rev.
**2009**, 51, 34–81. [Google Scholar] [CrossRef] [Green Version] - Yang, B.; Li, S. Multifocus image fusion and restoration with sparse representation. IEEE Trans. Instrum. Meas.
**2009**, 59, 884–892. [Google Scholar] [CrossRef] - Yu, N.; Qiu, T.; Bi, F.; Wang, A. Image features extraction and fusion based on joint sparse representation. IEEE J. Sel. Top. Signal Process.
**2011**, 5, 1074–1082. [Google Scholar] [CrossRef] - Li, S.; Yin, H.; Fang, L. Group-sparse representation with dictionary learning for medical image denoising and fusion. IEEE Trans. Biomed. Eng.
**2012**, 59, 3450–3459. [Google Scholar] [CrossRef] - Zhu, X.X.; Bamler, R. A sparse image fusion algorithm with application to pan-sharpening. IEEE Trans. Geosci. Remote Sens.
**2012**, 51, 2827–2836. [Google Scholar] [CrossRef] - Zhu, Z.; Yin, H.; Chai, Y.; Li, Y.; Qi, G. A novel multi-modality image fusion method based on image decomposition and sparse representation. Inf. Sci.
**2018**, 432, 516–529. [Google Scholar] [CrossRef] - Li, Y.; Sun, Y.; Huang, X.; Qi, G.; Zheng, M.; Zhu, Z. An Image Fusion Method Based on Sparse Representation and Sum Modified-Laplacian in NSCT Domain. Entropy
**2018**, 20, 522. [Google Scholar] [CrossRef] [Green Version] - Aiazzi, B.; Alparone, L.; Baronti, S.; Garzelli, A. Context-driven fusion of high spatial and spectral resolution images based on oversampled multiresolution analysis. IEEE Trans. Geosci. Remote Sens.
**2002**, 40, 2300–2312. [Google Scholar] [CrossRef] - Wang, W.; Chang, F. A Multi-focus Image Fusion Method Based on Laplacian Pyramid. J. Comput.
**2011**, 6, 2559–2566. [Google Scholar] [CrossRef] - Burt, P.; Adelson, E. The Laplacian pyramid as a compact image code. IEEE Trans. Commun.
**1983**, 31, 532–540. [Google Scholar] [CrossRef] - Yang, B.; Li, S. Pixel-level image fusion with simultaneous orthogonal matching pursuit. Inf. Fusion
**2012**, 13, 10–19. [Google Scholar] [CrossRef] - Tropp, J.A.; Gilbert, A.C.; Strauss, M.J. Algorithms for simultaneous sparse approximation. Part I: Greedy pursuit. Signal Process.
**2006**, 86, 572–588. [Google Scholar] [CrossRef] - Hellwich, O.; Reigber, A.; Lehmann, H. Sensor and data fusion contest: Test imagery to compare and combine airborne SAR and optical sensors for mapping. In Proceedings of the IEEE International Geosciences and Remote Sensing Symposium and the 24th Canadian Symposium on Remote Sensing, Toronto, ON, Canada, 24–28 June 2002. [Google Scholar]
- Zhang, J.; Yang, J.; Zhao, Z.; Li, H.; Zhang, Y. Block-regression based fusion of optical and SAR imagery for feature enhancement. Int. J. Remote Sens.
**2010**, 31, 2325–2345. [Google Scholar] [CrossRef] - Ma, X.; Shen, H.; Zhao, X.; Zhang, L. SAR image despeckling by the use of variational methods with adaptive nonlocal functionals. IEEE Trans. Geosci. Remote Sens.
**2016**, 54, 3421–3435. [Google Scholar] [CrossRef] - Yuhas, R.H.; Goetz, A.F.H.; Boardman, J.W. Discrimination Among Semi-Arid Landscape Endmembers Using the Spectral Angle Mapper (SAM) Algorithm. In JPL, Summaries of the Third Annual JPL Airborne Geoscience Workshop; JPL Publication: Washington, DC, USA; NASA: Washington, DC, USA, 1992; pp. 147–149. [Google Scholar]
- Zhou, J.; Civco, D.L.; Silander, J. A wavelet transform method to merge Landsat TM and SPOT panchromatic data. Int. J. Remote Sens.
**1998**, 19, 743–757. [Google Scholar] [CrossRef] - Wang, Z.; Bovik, A.C.; Sheikh, H.R.; Simoncelli, E.P. Image quality assessment: From error visibility to structural similarity. IEEE Trans. Image Process.
**2004**, 13, 600–612. [Google Scholar] [CrossRef] [Green Version] - Haghighat, M.B.A.; Aghagolzadeh, A.; Seyedarabi, H. A non-reference image fusion metric based on mutual information of image features. Comput. Electr. Eng.
**2011**, 37, 744–756. [Google Scholar] [CrossRef] - Lolli, S.; Alparone, L.; Garzelli, A.; Vivone, G. Haze correction for contrast-based multispectral pansharpening. IEEE Geosci. Remote Sens. Lett.
**2017**, 14, 2255–2259. [Google Scholar] [CrossRef] - Alparone, L.; Garzelli, A.; Vivone, G. Intersensor statistical matching for pansharpening: Theoretical issues and practical solutions. IEEE Trans. Geosci. Remote Sens.
**2017**, 55, 4682–4695. [Google Scholar] [CrossRef] - Meng, X.; Li, J.; Shen, H.; Zhang, L.; Zhang, H. Pansharpening with a guided filter based on three-layer decomposition. Sensors
**2016**, 16, 1068. [Google Scholar] [CrossRef] [PubMed] - Wald, L.; Ranchin, T.; Mangolini, M. Fusion of satellite images of different spatial resolutions: Assessing the quality of resulting images. Photogramm. Eng. Remote Sens.
**1997**, 63, 691–699. [Google Scholar]

**Figure 3.**Multispectral and SAR images used for fusion experiments, located in the Trudering region of Munich in Germany. (

**a**) Airborne multispectral images; (

**b**) airborne SAR intensity images.

**Figure 4.**Fusion results of different methods in area 1. (

**a**) HR SAR image; (

**b**) up-sampled LR multispectral image; (

**c**) IHS fusion; (

**d**) BTH fusion; (

**e**) LP-based fusion; (

**f**) AWLPR fusion; (

**g**) IW fusion; (

**h**) GFTD fusion; (

**i**) proposed LPSR fusion; (

**j**) original HR multispectral image.

**Figure 5.**Fusion results of different methods in area 2. (

**a**) HR SAR image; (

**b**) up-sampled LR multispectral image; (

**c**) IHS fusion; (

**d**) BTH fusion; (

**e**) LP-based fusion; (

**f**) AWLPR fusion; (

**g**) IW fusion; (

**h**) GFTD fusion; (

**i**) proposed LPSR fusion; (

**j**) original HR multispectral image.

**Figure 6.**Fusion results of different methods in area 3. (

**a**) HR SAR image; (

**b**) up-sampled LR multispectral image; (

**c**) IHS fusion; (

**d**) BTH fusion; (

**e**) LP-based fusion; (

**f**) AWLPR fusion; (

**g**) IW fusion; (

**h**) GFTD fusion; (

**i**) proposed LPSR fusion; (

**j**) original HR multispectral image.

**Figure 9.**Fusion results of adjustability experiment in area 1. (

**a**) IHS fusion; (

**b**) LPSR-5; (

**c**) LPSR-4; (

**d**) LPSR-3; (

**e**) LPSR-2; (

**f**) LPSR-1; (

**g**) LP-based fusion; (

**h**) original HR multispectral image.

**Figure 10.**Fusion results of adjustability experiment in area 2. (

**a**) IHS fusion; (

**b**) LPSR-5; (

**c**) LPSR-4; (

**d**) LPSR-3; (

**e**) LPSR-2; (

**f**) LPSR-1; (

**g**) LP-based fusion; (

**h**) original HR multispectral image.

**Figure 11.**Fusion results of adjustability experiment in area 3. (

**a**) IHS fusion; (

**b**) LPSR-5; (

**c**) LPSR-4; (

**d**) LPSR-3; (

**e**) LPSR-2; (

**f**) LPSR-1; (

**g**) LP-based fusion; (

**h**) original HR multispectral image.

**Figure 12.**Statistical results of adjustability experiment: (

**a**–

**c**) display the ${\mathrm{CC}}_{full}$, SCC, SSIM values of area 1; (

**d**–

**f**) for area 2; (

**g**–

**i**) for area 3.

Indexes | IHS | BTH | LP | AWLPR | IW | GFTD | LPSR | Ideal |
---|---|---|---|---|---|---|---|---|

CC ^{1} | 0.0418 | 0.4210 | 0.9620 | 0.9753 | 0.9608 | 0.9751 | 0.9235 | 1 |

0.0247 | 0.3566 | 0.8663 | 0.8917 | 0.8598 | 0.8964 | 0.8238 | 1 | |

SAM ^{2} | 4.7668 | 2.9880 | 1.3685 | 1.0067 | 1.2048 | 1.0249 | 1.9790 | 0 |

7.5331 | 5.6051 | 5.1160 | 4.3680 | 4.7799 | 4.3835 | 5.5635 | 0 | |

SCC | 0.9746 | 0.6436 | 0.9504 | 0.9506 | 0.9531 | 0.8714 | 0.9560 | 1 |

SSIM | 0.8998 | 0.6862 | 0.7023 | 0.6632 | 0.6656 | 0.6572 | 0.7340 | 1 |

AG | 20.452 | 19.7403 | 21.0458 | 20.6936 | 21.0910 | 16.1392 | 20.8458 | +∞ |

FMI | 0.5161 | 0.4061 | 0.4794 | 0.4678 | 0.4803 | 0.4652 | 0.5096 | 1 |

^{1,2}The first row of indicator CC/SAM shows the spectral fidelity in the reduced spatial resolution, in order to check the first property of Wald’s protocol. The second row corresponds to the results in the full spatial resolution, which verifies the second and third property of Wald’s protocol.

Indexes | IHS | BTH | LP | AWLPR | IW | GFTD | LPSR | Ideal |
---|---|---|---|---|---|---|---|---|

CC ^{1} | −0.1951 | 0.0304 | 0.8679 | 0.9288 | 0.8582 | 0.9044 | 0.7401 | 1 |

−0.1832 | −0.0142 | 0.6846 | 0.8029 | 0.6598 | 0.7267 | 0.5603 | 1 | |

SAM ^{2} | 6.4627 | 3.3283 | 3.8941 | 2.6314 | 3.7114 | 2.7748 | 4.6309 | 0 |

14.2631 | 9.6311 | 10.4075 | 9.0525 | 10.2610 | 9.4752 | 10.8905 | 0 | |

SCC | 0.9842 | 0.5734 | 0.8498 | 0.5453 | 0.8821 | 0.8571 | 0.9150 | 1 |

SSIM | 0.9681 | 0.7428 | 0.4046 | 0.1429 | 0.3466 | 0.3242 | 0.5777 | 1 |

AG | 24.6415 | 33.3441 | 25.7552 | 19.4118 | 24.7351 | 26.2544 | 23.4259 | +∞ |

FMI | 0.5130 | 0.3785 | 0.4033 | 0.3974 | 0.4071 | 0.4217 | 0.4525 | 1 |

^{1,2}The first row of indicator CC/SAM shows the spectral fidelity in the reduced spatial resolution, in order to check the first property of Wald’s protocol. The second row corresponds to the results in the full spatial resolution, which verifies the second and third property of Wald’s protocol.

Indexes | IHS | BTH | LP | AWLPR | IW | GFTD | LPSR | Ideal |
---|---|---|---|---|---|---|---|---|

CC ^{1} | −0.1942 | 0.2846 | 0.8796 | 0.9335 | 0.8411 | 0.9212 | 0.7690 | 1 |

−0.1923 | 0.2008 | 0.6712 | 0.7926 | 0.6151 | 0.7481 | 0.5668 | 1 | |

SAM ^{2} | 4.1220 | 2.1920 | 2.3637 | 1.5717 | 2.0341 | 1.6525 | 3.0498 | 0 |

7.5990 | 5.6608 | 6.3445 | 4.9756 | 5.8844 | 5.1918 | 6.9901 | 0 | |

SCC | 0.9972 | 0.9199 | 0.9138 | 0.6163 | 0.9458 | 0.9139 | 0.9629 | 1 |

SSIM | 0.9583 | 0.7920 | 0.4885 | 0.2115 | 0.4376 | 0.3468 | 0.6534 | 1 |

AG | 16.9149 | 17.2255 | 17.8394 | 11.6247 | 17.1778 | 13.91111 | 16.9217 | +∞ |

FMI | 0.5164 | 0.4536 | 0.4347 | 0.4223 | 0.4326 | 0.4369 | 0.4873 | 1 |

^{1,2}The first row of indicator CC/SAM shows the spectral fidelity in the reduced spatial resolution, in order to check the first property of Wald’s protocol. The second row corresponds to the results in the full spatial resolution, which verifies the second and third property of Wald’s protocol.

Methods | CC ^{1} | SAM ^{2} | SCC | SSIM | AG | FMI |
---|---|---|---|---|---|---|

IHS | −0.1263 ± 0.2300 | 7.7138 ± 5.9478 | 0.9551 ± 0.0563 | 0.8416 ± 0.1150 | 18.5431 ± 8.5031 | 0.5029 ± 0.0314 |

BTH | 0.1577 ± 0.2531 | 6.8539 ± 4.2926 | 0.6217 ± 0.2489 | 0.5792 ± 0.1902 | 28.9911 ± 12.5180 | 0.3723 ± 0.0569 |

LP | 0.6617 ± 0.1131 | 6.1318 ± 4.1853 | 0.8661 ± 0.0597 | 0.5488 ± 0.1319 | 20.1218 ± 8.7419 | 0.4266 ± 0.0342 |

AWLPR | 0.6933 ± 0.1120 | 5.4492 ± 3.5053 | 0.8731 ± 0.0581 | 0.5031 ± 0.1308 | 21.7485 ± 9.7197 | 0.4326 ± 0.0228 |

IW | 0.6235 ± 0.1338 | 5.9936 ± 4.1849 | 0.8851 ± 0.0528 | 0.5375 ± 0.1205 | 19.3256 ± 8.5587 | 0.4315 ± 0.0355 |

GFTD | 0.6707 ± 0.1356 | 5.6733 ± 3.8064 | 0.7607 ± 0.1582 | 0.4773 ± 0.1363 | 21.2761 ± 12.0001 | 0.4188 ± 0.0317 |

LPSR | 0.5696 ± 0.1320 | 6.6137 ± 4.5335 | 0.9070 ± 0.0493 | 0.6399 ± 0.1134 | 18.9086 ± 8.1401 | 0.4699 ± 0.0315 |

Ideal | 1 | 0 | 1 | 1 | +∞ | 1 |

^{1,2}The values of indicator CC/SAM correspond to the results in the full spatial resolution.

SR | LPSR-1 | LPSR-2 | LPSR-3 | LPSR-4 | LPSR-5 | GFTD | IW | AWLPR | LP | BTH | IHS | |
---|---|---|---|---|---|---|---|---|---|---|---|---|

Time/s | 9308.66 | 2226.38 | 536.63 | 119.59 | 23.36 | 2.95 | 0.2896 | 0.1137 | 0.2047 | 0.0830 | 0.1362 | 0.0234 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Zhang, H.; Shen, H.; Yuan, Q.; Guan, X.
Multispectral and SAR Image Fusion Based on Laplacian Pyramid and Sparse Representation. *Remote Sens.* **2022**, *14*, 870.
https://doi.org/10.3390/rs14040870

**AMA Style**

Zhang H, Shen H, Yuan Q, Guan X.
Multispectral and SAR Image Fusion Based on Laplacian Pyramid and Sparse Representation. *Remote Sensing*. 2022; 14(4):870.
https://doi.org/10.3390/rs14040870

**Chicago/Turabian Style**

Zhang, Hai, Huanfeng Shen, Qiangqiang Yuan, and Xiaobin Guan.
2022. "Multispectral and SAR Image Fusion Based on Laplacian Pyramid and Sparse Representation" *Remote Sensing* 14, no. 4: 870.
https://doi.org/10.3390/rs14040870