# Hyperspectral Image Mixed Noise Removal Using a Subspace Projection Attention and Residual Channel Attention Network

^{1}

^{2}

^{3}

^{4}

^{5}

^{*}

## Abstract

**:**

## 1. Introduction

**Spatial-Domain-Based Methods.**The spatial-domain-based denoising method usually regards the three-dimensional hyperspectral image as an extension of the RGB image and applies the traditional two-dimensional denoising algorithms to denoise the hyperspectral image band by band. Such methods can be categorized into pixel-space-based methods and transform-domain-space-based methods. The former one does not perform any transformation on the pixel value, and it directly performs the denoising operation. The typical algorithms are the non-local mean (NLM) algorithm [13], Bayes algorithm [14], bilateral filtering [15] and denoising methods based on sparse and redundant representations [16], etc. These types of methods are easy to operate and implement, but there are limitations in practical applications, and the effect of removing severe noise is not good. The method based on the transform domain uses the difference in a specific transform domain between the original image and the noise to separate the main signal and the noise. Commonly used specific transform domains are wavelet transform domain and Fourier transform domain. Representative examples of this methodology include adaptive wavelet thresholding [17], block-matching and 3D filtering (BM3D) [18], the extending of BM3D to multiband image denoising and block-matching and 4D filtering (BM4D) [19]. However, the spatial-domain-based denoising methods ignore the correlation between the bands and only have a certain effect for a specific type of noise and thus cannot deal with the mixed noise problem.

**Spectral-Domain-Based Methods.**HSIs contain numerous information in hundreds of bands. Therefore, in addition to the above spatial-domain-based denoising methods, the spectral-domain information of HSIs can also be used for denoising. Green et al. [20] used maximum noise fraction (MNF) transform to remove noise. Donoho et al. [21] proposed a SureShrink algorithm via wavelet transform of the spectral domain signal. The spectral-domain-based denoising methods were proposed from the perspective of signal processing and ignore spatial structure information, which is likely to cause spatial pixel distortion, resulting in limited image restoration effects.

**Prior-Constraint-Based Methods.**Denoising methods based on the spatial domain and the spectral domain could remove certain specific noises, such as Gaussian noise, but these two types of methods only use spatial or spectral information, making the effect of removing mixed noises poor. The denoising method based on prior constraint could make full use of the information in the spatial and spectral domains and has become one of the hot topics and trends in the field of remote sensing images. This method transforms the denoising problem into a prior constraint problem. The prior constraints of remote sensing images directly determine the restoration effect. The low rankness of HSIs in the spectral dimension is a widely used image prior in an HSI denoising task. Representative low-rankness-based methods include, for example, PCA (principal components analysis) [22], LRTA (low-rank tensor approximation) [23], LRTR (low-rank tensor recovery) [24], LRMR (low-rank matrix recovery) [25], NAILRMA (noise-adjusted iterative low-rank matrix approximation) [26], BLRMF (bilinear low-rank matrix factorization) [27], NLR-CPTD (nonlocal low-rank regularized CP tensor decomposition) [28] and so on. Total variation is also a widely used image prior in regard to a denoising problem. Some total-variation-based methods are, for example, SSAHTV (spectral–spatial adaptive total variation) [29], SSTV (spatio-spectral total variation) [30], E3DTV (enhanced-3DTV) [31], etc. Recently, researchers combined low-rank constraints with total variation constraints and have proposed many HSI denoising methods [32,33,34,35]. In summary, the HSI denoising methods based on prior constraints exploit the low-rank and spatial–spectral structure information of HSIs and achieve meaningful results. The subspace representation of spectral vectors in HSIs has been successfully used to remove noise by regularizing the representation coefficients of HSIs, such as FastHyDe (fast hyperspectral denoising) [36], NGmeet (non-local meets global) [37], L1HyMixDe [38], etc.

**Deep-Learning-Based Methods.**Although the above methods can achieve good results, the models of such are fixed and the parameters should be tuned precisely, so the methods may be unstable and sensitive to the data [39]. In recent years, deep learning (DL) has demonstrated better performance than traditional methods in many computer vision tasks [40,41,42,43] and has also been introduced into HSI processing, including fusion [44], unmixing [45,46,47], data enhancement [48] and so on. DL-based denoising methods usually employ supervised models, which take clean and noisy image pairs as the inputs and train the network to learn the prior distribution of clean images, thereby establishing an end-to-end mapping from noisy images to clean images. For the HSI denoising task, the exploration of the combination of spatial and spectral information, the extraction and representation of the features of HSI are the focus of the researchers. For Gaussian noise, Yuan et al. [49] used the spatial band and its adjacent bands as inputs for the model to extract features while introducing multiscale convolution layers to extract multiscale features in spatial and spectral dimensions. Chang et al. [48] proposed the HSI-DeNet, which applied a residual learning strategy, dilated convolution and multichannel filtering and achieved great performance for mixed noise denoising. Zhang et al. [50] proposed a spatial–spectral gradient network (SSGN), which employed a spatial–spectral gradient learning strategy, taking a single band, its horizontal/vertical spatial gradient images and its adjacent spectral gradient as inputs considering that sparse noise is spatially directional and the spectral gradient is used as additional supplementary information. Moreover, recently, some work based on the attention mechanism [39,51,52] has been proposed. Although the existing DL-based HSI denoising methods have achieved great success, recovering high-quality HSIs in difficult scenes, such as severe noise and mixed noise, is still challenging. In addition, these methods still do not fully exploit the local and global spatial–spectral prior information of the HSI because, usually, the convolution operation only exploits the local spatial and spectral prior information of the HSI.

- We put forth a novel HSI mixed noise removal network, termed SPARCA-Net, that is able to fully explore the local and global spatial–spectral information of HSIs other than conventional CNN-based denoising frameworks.
- We propose an OSPA module based on spatial attention to adaptively learn an orthogonal subspace projection that can be used to reconstruct the main feature maps from its subspace, which is able to facilitate spatial structure recovery by utilization of both local and global spatial correlations.
- We design a channel-attention-based RCA module with a cascaded bottleneck structure to progressively exploit the prior spectral information underlying HSIs and use it to adjust the weights between feature maps.

## 2. Related Work

#### 2.1. Problem Formulation

#### 2.2. Subspace Projection for HSIs

#### 2.3. Attention Mechanism

**Channel Attention.**The main idea of channel attention is to extract the feature map of each channel; that is, the response information to different categories of features, and different channels are related to each other and share information. By constructing a channel attention mechanism to express the interaction between channels, the input feature maps are compressed by pooling operation, and then the channel attention map is calculated by sharing multi-layer perceptron. Hu et al. [54] proposed a squeeze-and-excitation network to significantly improve the image classification accuracy. Dai et al. [55] proposed a second-order attention network that explored the feature correlations of intermediate layers for image super-resolution. Qin et al. [56] proposed novel multi-spectral channel attention networks, which preprocess the channel attention mechanism in the frequency domain.

**Spatial Attention.**The spatial attention mechanism can capture important feature information in the spatial domain by paying attention to the parts that matter most in feature maps, which could be used as a complement to channel attention mechanisms. Hu et al. [57] proposed an object relation module for object detection tasks, the idea of which is to process a set of objects by computing the reasoned relation between each other simultaneously instead of individually. Chen et al. [58] proposed a graph-based global reasoning network to capture global relations between relation-aware features. Liu et al. [59] proposed a non-local operation to take the weighted sum of the features at all positions as a response at one position of features. It is worth noting that the cross-attention mechanism was introduced for hyperspectral super-resolution by Yao et al. [60] to exploit the joint spatial and spectral information. For an HSI denoising task, Shi et al. [39] proposed a 3-D convolution-based attention denoising network where the channel attention modules were applied to explore the correlations between spectral channels, and the position attention modules were designed to formulate the interdependencies between pixels on the feature maps. Wang et al. [51] proposed a spatial–spectral cross attention network for HSI denoising in which a spectral–spatial attention block (SSAB) was designed to efficiently utilize the spatial–spectral information. Cao et al. [52] proposed a deep spatial–spectral global reasoning network for HSI mixed noise removal with two novel attention-mechanism-based modules to model and reason global relational information.

## 3. Proposed Method

#### 3.1. Overall Network Architecture

#### 3.2. Orthogonal Subplace Projection Attention Module

**X**usually lies in a low-dimensional signal subspace ${S}_{P}$, with $P\ll B$. We can complete the matrix multiplication through CNNs and implement the generation of signal subspace bases and projection matrices. The specific projection operation is described as follows:

**Bases Generation.**${Y}_{1},{Y}_{2}\in {\mathbb{R}}^{H\times W\times C}$ denote two feature maps from the same HSI outputted by CNNs in different layers; where C indicates the output channels of CNN, let $E=[{e}_{1},\dots ,{e}_{P}]\in {\mathbb{R}}^{n\times P}$ be the orthogonal base matrix of P-dimensional signal subspace of ${Y}_{1}$ and ${Y}_{2}$, where ${e}_{i}\in {\mathbb{R}}^{n\times 1}$ are the basis vectors and $n=H\times W$. The above process can be implemented by CNNs; we can first concatenate ${Y}_{1}$ and ${Y}_{2}$ along the channel dimension to obtain $Y\in {\mathbb{R}}^{H\times W\times 2C}$, then pass $Y$ through a CNN whose output channel number is P and reshape the output to $HW\times P$ to obtain the orthogonal base matrix $E$, as shown in Figure 3.

**Orthogonal Projection.**As we mentioned above, ${e}_{i}\in {\mathbb{R}}^{n\times 1},i=1,\dots ,P$ are the basis vectors of P-dimensional signal subspace ${S}_{P}$. Then, the feature map ${Y}_{1}$ can be projected into ${S}_{P}$ by linear projection. Let $P=E{({E}^{T}E)}^{-1}{E}^{T}$ be the orthogonal projection matrix [61]. To ensure the basis vectors are orthogonal, the normalization term ${({E}^{T}E)}^{-1}$ is required. Therefore, the feature map ${Y}_{1}$ can be reconstructed by orthogonal subspace projection to obtain the “clean” one:

#### 3.3. Residual Channel Attention Module

## 4. Experiments and Discussion

#### 4.1. Synthetic Experiments

**Case 1 (Gaussian non-i.i.d. noise)**: We added zero-mean Gaussian noise with varying intensities randomly selected from 25 to 75 to all bands of the image.

**Case 2 (Gaussian + stripe noise)**: The images were added with the non-i.i.d Gaussian noise mentioned in Case 1. In addition, we randomly selected 30% of the bands to add stripe noise, and the number of stripes in each selected band was randomly set from 5% to 15% of the columns.

**Case 3 (Gaussian + deadline noise):**All bands in the image were corrupted by the non-i.i.d Gaussian noise mentioned in Case 1. On top of this, 30% of the bands were randomly selected to add deadline noise. The number of deadlines in each selected band was randomly set from 5% to 15% of the columns.

**Case 4 (Gaussian + impulse noise)**: All bands were corrupted by non-i.i.d Gaussian noise mentioned in Case 1. Based on this, we randomly selected 30% of the bands to add impulse noise with varying intensities, and the percentage of impulse range was set from 10% to 70% randomly.

**Case 5 (Mixed noise):**All bands were corrupted by Gaussian non-i.i.d noise (Case 1), stripe noise (Case 2), deadline noise (Case 3) and impulse noise (Case 4).

#### 4.2. Real HSI Denoising

#### 4.3. Sensitivity and Ablation Study

## 5. Conclusions

## Author Contributions

## Funding

## Data Availability Statement

## Conflicts of Interest

## References

- Hong, D.; He, W.; Yokoya, N.; Yao, J.; Gao, L.; Zhang, L.; Chanussot, J.; Zhu, X. Interpretable Hyperspectral Artificial Intelligence: When Nonconvex Modeling Meets Hyperspectral Remote Sensing. IEEE Geosci. Remote Sens. Mag.
**2021**, 9, 52–87. [Google Scholar] [CrossRef] - Zhang, B.; Sun, X.; Gao, L.; Yang, L. Endmember Extraction of Hyperspectral Remote Sensing Images Based on the Ant Colony Optimization (ACO) Algorithm. IEEE Trans. Geosci. Remote Sens.
**2011**, 49, 2635–2646. [Google Scholar] [CrossRef] - Hong, D.; Gao, L.; Yokoya, N.; Yao, J.; Chanussot, J.; Du, Q.; Zhang, B. More Diverse Means Better: Multimodal Deep Learning Meets Remote-Sensing Imagery Classification. IEEE Trans. Geosci. Remote Sens.
**2021**, 59, 4340–4354. [Google Scholar] [CrossRef] - Li, J.; Zheng, K.; Yao, J.; Gao, L.; Hong, D. Deep Unsupervised Blind Hyperspectral and Multispectral Data Fusion. IEEE Geosci. Remote Sens. Lett.
**2022**. [Google Scholar] [CrossRef] - Zhang, B.; Yang, W.; Gao, L.; Chen, D. Real-Time Target Detection in Hyperspectral Images Based on Spatial-Spectral Information Extraction. EURASIP J. Adv. Signal Process.
**2012**, 2012, 142. [Google Scholar] [CrossRef][Green Version] - Zhang, B.; Li, S.; Jia, X.; Gao, L.; Peng, M. Adaptive Markov Random Field Approach for Classification of Hyperspectral Imagery. IEEE Geosci. Remote Sens. Lett.
**2011**, 8, 973–977. [Google Scholar] [CrossRef] - Gao, L.; Wang, Z.; Zhuang, L.; Yu, H.; Zhang, B.; Chanussot, J. Using Low-Rank Representation of Abundance Maps and Nonnegative Tensor Factorization for Hyperspectral Nonlinear Unmixing. IEEE Trans. Geosci. Remote Sens.
**2022**, 60, 1–17. [Google Scholar] [CrossRef] - Hong, D.; Gao, L.; Yao, J.; Zhang, B.; Plaza, A.; Chanussot, J. Graph Convolutional Networks for Hyperspectral Image Classification. IEEE Trans. Geosci. Remote Sens.
**2021**, 59, 5966–5978. [Google Scholar] [CrossRef] - Sun, L.; Zhong, F. Mixed Noise Estimation for Hyperspectral Image Based on Multiple Bands Prediction. IEEE Geosci. Remote Sens. Lett.
**2022**, 19, 6007705. [Google Scholar] [CrossRef] - Huang, S.; Zhang, H.; Pižurica, A. Subspace Clustering for Hyperspectral Images via Dictionary Learning with Adaptive Regularization. IEEE Trans. Geosci. Remote Sens.
**2021**, 60, 5524017. [Google Scholar] [CrossRef] - Gao, L.; Du, Q.; Zhang, B.; Yang, W.; Wu, Y. A Comparative Study on Linear Regression-Based Noise Estimation for Hyperspectral Imagery. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens.
**2013**, 6, 488–498. [Google Scholar] [CrossRef][Green Version] - Zhuang, L.; Ng, M.K.; Fu, X. Hyperspectral Image Mixed Noise Removal Using Subspace Representation and Deep CNN Image Prior. Remote Sens.
**2021**, 13, 4098. [Google Scholar] [CrossRef] - Buades, A.; Coll, B.; Morel, J.-M. A Non-Local Algorithm for Image Denoising. In Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), San Diego, CA, USA, 20–26 June 2005; Volume 2, pp. 60–65. [Google Scholar]
- Li, J.; Bioucas-Dias, J.M.; Plaza, A. Hyperspectral Image Segmentation Using a New Bayesian Approach with Active Learning. IEEE Trans. Geosci. Remote Sens.
**2011**, 49, 3947–3960. [Google Scholar] [CrossRef][Green Version] - Paris, S. A Gentle Introduction to Bilateral Filtering and Its Applications. In Proceedings of the ACM SIGGRAPH 2007 Courses, San Diego, CA, USA, 5–9 August 2007. [Google Scholar]
- Elad, M.; Aharon, M. Image Denoising via Sparse and Redundant Representations over Learned Dictionaries. IEEE Trans. Image Process.
**2006**, 15, 3736–3745. [Google Scholar] [CrossRef] [PubMed] - Chang, S.G.; Yu, B.; Vetterli, M. Adaptive Wavelet Thresholding for Image Denoising and Compression. IEEE Trans. Image Process.
**2000**, 9, 1532–1546. [Google Scholar] [CrossRef][Green Version] - Dabov, K.; Foi, A.; Katkovnik, V.; Egiazarian, K. Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering. IEEE Trans. Image Process.
**2007**, 16, 2080–2095. [Google Scholar] [CrossRef] - Maggioni, M.; Katkovnik, V.; Egiazarian, K.; Foi, A. Nonlocal Transform-Domain Filter for Volumetric Data Denoising and Reconstruction. IEEE Trans. Image Process.
**2013**, 22, 119–133. [Google Scholar] [CrossRef] - Green, A.A.; Berman, M.; Switzer, P.; Craig, M.D. A Transformation for Ordering Multispectral Data in Terms of Image Quality with Implications for Noise Removal. IEEE Trans. Geosci. Remote Sens.
**1988**, 26, 65–74. [Google Scholar] [CrossRef][Green Version] - Donoho, D.L.; Johnstone, I.M. Adapting to Unknown Smoothness via Wavelet Shrinkage. J. Am. Stat. Assoc.
**1995**, 90, 1200–1224. [Google Scholar] [CrossRef] - Chang, C.-I.; Du, Q. Interference and Noise-Adjusted Principal Components Analysis. IEEE Trans. Geosci. Remote Sens.
**1999**, 37, 2387–2396. [Google Scholar] [CrossRef][Green Version] - Renard, N.; Bourennane, S.; Blanc-Talon, J. Denoising and Dimensionality Reduction Using Multilinear Tools for Hyperspectral Images. IEEE Geosci. Remote Sens. Lett.
**2008**, 5, 138–142. [Google Scholar] [CrossRef] - Fan, H.; Chen, Y.; Guo, Y.; Zhang, H.; Kuang, G. Hyperspectral Image Restoration Using Low-Rank Tensor Recovery. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens.
**2017**, 10, 4589–4604. [Google Scholar] [CrossRef] - Zhang, H.; He, W.; Zhang, L.; Shen, H.; Yuan, Q. Hyperspectral Image Restoration Using Low-Rank Matrix Recovery. IEEE Trans. Geosci. Remote Sens.
**2014**, 52, 4729–4743. [Google Scholar] [CrossRef] - He, W.; Zhang, H.; Zhang, L.; Shen, H. Hyperspectral Image Denoising via Noise-Adjusted Iterative Low-Rank Matrix Approximation. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens.
**2015**, 8, 3050–3061. [Google Scholar] [CrossRef] - Fan, H.; Li, J.; Yuan, Q.; Liu, X.; Ng, M. Hyperspectral Image Denoising with Bilinear Low Rank Matrix Factorization. Signal Processing
**2019**, 163, 132–152. [Google Scholar] [CrossRef] - Xue, J.; Zhao, Y.; Liao, W.; Chan, J.C.-W. Nonlocal Low-Rank Regularized Tensor Decomposition for Hyperspectral Image Denoising. IEEE Trans. Geosci. Remote Sens.
**2019**, 57, 5174–5189. [Google Scholar] [CrossRef] - Yuan, Q.; Zhang, L.; Shen, H. Hyperspectral Image Denoising Employing a Spectral–Spatial Adaptive Total Variation Model. IEEE Trans. Geosci. Remote Sens.
**2012**, 50, 3660–3677. [Google Scholar] [CrossRef] - Aggarwal, H.K.; Majumdar, A. Hyperspectral Image Denoising Using Spatio-Spectral Total Variation. IEEE Geosci. Remote Sens. Lett.
**2016**, 13, 442–446. [Google Scholar] [CrossRef] - Peng, J.; Xie, Q.; Zhao, Q.; Wang, Y.; Meng, D. Enhanced 3DTV Regularization and Its Applications on Hyper-Spectral Image Denoising and Compressed Sensing. arXiv
**2018**, arXiv:1809.06591. [Google Scholar] - Gao, L.; Yao, D.; Li, Q.; Zhuang, L.; Zhang, B.; Bioucas-Dias, J.M. A New Low-Rank Representation Based Hyperspectral Image Denoising Method for Mineral Mapping. Remote Sens.
**2017**, 9, 1145. [Google Scholar] [CrossRef][Green Version] - He, W.; Zhang, H.; Zhang, L.; Shen, H. Total-Variation-Regularized Low-Rank Matrix Factorization for Hyperspectral Image Restoration. IEEE Trans. Geosci. Remote Sens.
**2016**, 54, 178–188. [Google Scholar] [CrossRef] - He, W.; Zhang, H.; Shen, H.; Zhang, L. Hyperspectral Image Denoising Using Local Low-Rank Matrix Recovery and Global Spatial–Spectral Total Variation. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens.
**2018**, 11, 713–729. [Google Scholar] [CrossRef] - Wang, Y.; Peng, J.; Zhao, Q.; Leung, Y.; Zhao, X.-L.; Meng, D. Hyperspectral Image Restoration Via Total Variation Regularized Low-Rank Tensor Decomposition. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens.
**2018**, 11, 1227–1243. [Google Scholar] [CrossRef][Green Version] - Zhuang, L.; Bioucas-Dias, J.M. Fast Hyperspectral Image Denoising and Inpainting Based on Low-Rank and Sparse Representations. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens.
**2018**, 11, 730–742. [Google Scholar] [CrossRef] - He, W.; Yao, Q.; Li, C.; Yokoya, N.; Zhao, Q. Non-Local Meets Global: An Integrated Paradigm for Hyperspectral Denoising. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, 15–20 June 2019; pp. 6868–6877. [Google Scholar]
- Zhuang, L.; Ng, M.K. Hyperspectral Mixed Noise Removal by L1-Norm-Based Subspace Representation. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens.
**2020**, 13, 1143–1157. [Google Scholar] [CrossRef] - Shi, Q.; Tang, X.; Yang, T.; Liu, R.; Zhang, L. Hyperspectral Image Denoising Using a 3-D Attention Denoising Network. IEEE Trans. Geosci. Remote Sens.
**2021**, 59, 10348–10363. [Google Scholar] [CrossRef] - Zhang, K.; Zuo, W.; Chen, Y.; Meng, D.; Zhang, L. Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising. IEEE Trans. Image Process.
**2017**, 26, 3142–3155. [Google Scholar] [CrossRef][Green Version] - Gao, L.; Han, Z.; Hong, D.; Zhang, B.; Chanussot, J. CyCU-Net: Cycle-Consistency Unmixing Network by Learning Cascaded Autoencoders. IEEE Trans. Geosci. Remote Sens.
**2022**, 60, 5503914. [Google Scholar] [CrossRef] - Hong, D.; Han, Z.; Yao, J.; Gao, L.; Zhang, B.; Plaza, A.; Chanussot, J. SpectralFormer: Rethinking Hyperspectral Image Classification with Transformers. IEEE Trans. Geosci. Remote Sens.
**2021**, 60, 5518615. [Google Scholar] [CrossRef] - Hong, D.; Yao, J.; Meng, D.; Xu, Z.; Chanussot, J. Multimodal GANs: Toward Crossmodal Hyperspectral–Multispectral Image Segmentation. IEEE Trans. Geosci. Remote Sens.
**2021**, 59, 5103–5113. [Google Scholar] [CrossRef] - Zheng, K.; Gao, L.; Hong, D.; Zhang, B.; Chanussot, J. NonRegSRNet: A Non-Rigid Registration Hyperspectral Super-Resolution Network. IEEE Trans. Geosci. Remote Sens.
**2021**, 60, 1–16. [Google Scholar] [CrossRef] - Yao, J.; Meng, D.; Zhao, Q.; Cao, W.; Xu, Z. Nonconvex-Sparsity and Nonlocal-Smoothness Based Blind Hyperspectral Unmixing. IEEE Trans. Image Process.
**2019**, 28, 2991–3006. [Google Scholar] [CrossRef] [PubMed] - Yao, J.; Hong, D.; Xu, L.; Meng, D.; Chanussot, J.; Xu, Z. Sparsity-Enhanced Convolutional Decomposition: A Novel Tensor-Based Paradigm for Blind Hyperspectral Unmixing. IEEE Trans. Geosci. Remote Sens.
**2022**, 60, 5505014. [Google Scholar] [CrossRef] - Hong, D.; Yokoya, N.; Chanussot, J.; Zhu, X.X. An Augmented Linear Mixing Model to Address Spectral Variability for Hyperspectral Unmixing. IEEE Trans. Image Process.
**2019**, 28, 1923–1938. [Google Scholar] [CrossRef] [PubMed][Green Version] - Chang, Y.; Yan, L.; Fang, H.; Zhong, S.; Liao, W. HSI-DeNet: Hyperspectral Image Restoration via Convolutional Neural Network. IEEE Trans. Geosci. Remote Sens.
**2019**, 57, 667–682. [Google Scholar] [CrossRef] - Yuan, Q.; Zhang, Q.; Li, J.; Shen, H.; Zhang, L. Hyperspectral Image Denoising Employing a Spatial–Spectral Deep Residual Convolutional Neural Network. IEEE Trans. Geosci. Remote Sens.
**2019**, 57, 1205–1218. [Google Scholar] [CrossRef][Green Version] - Zhang, Q.; Yuan, Q.; Li, J.; Liu, X.; Shen, H.; Zhang, L. Hybrid Noise Removal in Hyperspectral Imagery With a Spatial–Spectral Gradient Network. IEEE Trans. Geosci. Remote Sens.
**2019**, 57, 7317–7329. [Google Scholar] [CrossRef] - Wang, Z.; Shao, Z.; Huang, X.; Wang, J.; Lu, T. SSCAN: A Spatial–Spectral Cross Attention Network for Hyperspectral Image Denoising. IEEE Geosci. Remote Sens. Lett.
**2022**, 19, 5508805. [Google Scholar] [CrossRef] - Cao, X.; Fu, X.; Xu, C.; Meng, D. Deep Spatial-Spectral Global Reasoning Network for Hyperspectral Image Denoising. IEEE Trans. Geosci. Remote Sens.
**2022**, 60, 5504714. [Google Scholar] [CrossRef] - Cheng, Y.; Zhang, K.; Xu, Z. Theory of Matrix, 3rd ed.; Northwestern Polytechnic University Press: Xi’an, China, 2006. [Google Scholar]
- Hu, J.; Shen, L.; Sun, G. Squeeze-and-Excitation Networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–62 June 2018; pp. 7132–7141. [Google Scholar]
- Dai, T.; Cai, J.; Zhang, Y.; Xia, S.-T.; Zhang, L. Second-Order Attention Network for Single Image Super-Resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, 15–20 June 2019; pp. 11065–11074. [Google Scholar]
- Qin, Z.; Zhang, P.; Wu, F.; Li, X. Fcanet: Frequency Channel Attention Networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada, 11–17 October 2021; pp. 783–792. [Google Scholar]
- Hu, H.; Gu, J.; Zhang, Z.; Dai, J.; Wei, Y. Relation Networks for Object Detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–62 June 2018; pp. 3588–3597. [Google Scholar]
- Chen, Y.; Rohrbach, M.; Yan, Z.; Shuicheng, Y.; Feng, J.; Kalantidis, Y. Graph-Based Global Reasoning Networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, 15–20 June 2019; pp. 433–442. [Google Scholar]
- Liu, M.; Wang, Z.; Ji, S. Non-Local Graph Neural Networks. IEEE Trans. Pattern Anal. Mach. Intell.
**2021**. [Google Scholar] [CrossRef] - Yao, J.; Hong, D.; Chanussot, J.; Meng, D.; Zhu, X.; Xu, Z. Cross-Attention in Coupled Unmixing Nets for Unsupervised Hyperspectral Super-Resolution. In Proceedings of the 16th European Conference on Computer Vision (ECCV), Glasgow, UK, 23–28 August 2020; pp. 208–224. [Google Scholar]
- Meyer, C.D. Matrix Analysis and Applied Linear Algebra; Siam: Philadelphia, PA, USA, 2000; Volume 71. [Google Scholar]
- Zhang, Y.; Li, K.; Li, K.; Wang, L.; Zhong, B.; Fu, Y. Image Super-Resolution Using Very Deep Residual Channel Attention Networks. In Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany, 14–24 May 2018; pp. 286–301. [Google Scholar]
- Paszke, A.; Gross, S.; Massa, F.; Lerer, A.; Bradbury, J.; Chanan, G.; Killeen, T.; Lin, Z.M.; Gimelshein, N.; Antiga, L.; et al. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Proceedings of the 33rd Conference on Neural Information Processing Systems (NeurIPS), Vancouver, BC, Canada, 8–14 December 2019. [Google Scholar]
- Kingma, D.P.; Ba, J. Adam: A Method for Stochastic Optimization. arXiv
**2014**, arXiv:1412.6980. [Google Scholar] - Breiman, L. Random Forests. Mach. Learn.
**2001**, 45, 5–32. [Google Scholar] [CrossRef][Green Version] - Melgani, F.; Bruzzone, L. Classification of hyperspectral remote sensing images with support vector machines. IEEE Trans. Geosci. Remote Sens.
**2004**, 42, 1778–1790. [Google Scholar] [CrossRef][Green Version]

**Figure 1.**Observed HSIs contaminated by mixed noise. (

**a**) The second band image of Pavia University dataset; (

**b**) the 139th band image of URBAN dataset.

**Figure 5.**False color images synthesized with band 50, 70 and 112 of denoised Washington DC data in Gaussian noise case; $\sigma =75$. (

**a**) Clean; (

**b**) noisy; (

**c**) BM4D; (

**d**) LRMR; (

**e**) NAILRMA; (

**f**) FastHyDe; (

**g**) L1HyMixDe; (

**h**) E3DTV; (

**i**) LRTDTV; (

**j**) HSI-DeNet; (

**k**) HSID-CNN; (

**l**) ours.

**Figure 6.**False color images synthesized with band 50, 70 and 110 of the denoised Washington DC data in mixture noise case. (

**a**) Clean; (

**b**) noisy; (

**c**) LRMR; (

**d**) NAILRMA; (

**e**) NGmeet; (

**f**) L1HyMixDe; (

**g**) E3DTV; (

**h**) LRTDTV; (

**i**) HSI-DeNet; (

**j**) HSID-CNN; (

**k**) ours.

**Figure 7.**False color images synthesized with band 14, 24 and 54 of the denoised Pavia Center data in mixture noise case. (

**a**) Clean; (

**b**) noisy; (

**c**) LRMR; (

**d**) NAILRMA; (

**e**) NGmeet; (

**f**) L1HyMixDe; (

**g**) E3DTV; (

**h**) LRTDTV; (

**i**) HSI-DeNet; (

**j**) HSID-CNN; (

**k**) ours.

**Figure 8.**Band-wise PSNR values in the first row and band-wise SSIM values in the second row for denoised Washington DC data. Subfigures in (

**a**,

**f**), (

**b**,

**g**), (

**c**,

**h**), (

**d**,

**i**) and (

**e**,

**j**) correspond to Case 1, Case 2, Case 3, Case 4 and Case 5, respectively.

**Figure 9.**Band-wise PSNR values in the first row and band-wise SSIM values in the second row for denoised Pavia Center data. Subfigures in (

**a**,

**f**), (

**b**,

**g**), (

**c**,

**h**), (

**d**,

**i**) and (

**e**,

**j**) correspond to Case 1, Case 2, Case 3, Case 4 and Case 5, respectively.

**Figure 10.**The denoised results on band 2 of the Indian Pines data. (

**a**) Original; (

**b**) LRMR; (

**c**) NAILRMA; (

**d**) NGmeet; (

**e)**L1HyMixDe; (

**f)**E3DTV; (

**g**) LRTDTV; (

**h**) HSI-DeNet; (

**i**) HSID-CNN; (

**j**) ours.

**Figure 11.**False color images synthesized with band 61, 34 and 1 of denoised Indian Pines data. (

**a**) Original; (

**b**) LRMR; (

**c**) NAILRMA; (

**d**) NGmeet; (

**e**) L1HyMixDe; (

**f**) E3DTV; (

**g**) LRTDTV; (

**h**) HSI-DeNet; (

**i**) HSID-CNN; (

**j**) ours.

**Figure 12.**Classification results for Indian Pines. (

**a**) Original; (

**b**) LRMR; (

**c**) NAILRMA; (

**d**) NGmeet; (

**e**) L1HyMixDe; (

**f**) E3DTV; (

**g**) LRTDTV; (

**h**) HSI-DeNet; (

**i**) HSID-CNN; (

**j**) ours; (

**k**) ground truth and class labels.

**Figure 13.**The denoised results on band 2 of the GF-5 data. (

**a**) Original; (

**b**) LRMR; (

**c**) NAILRMA; (

**d**) NGmeet; (

**e**) L1HyMixDe; (

**f**) E3DTV; (

**g**) LRTDTV; (

**h**) HSI-DeNet; (

**i**) HSID-CNN; (

**j**) ours.

**Figure 14.**False color images synthesized with band 1, 34 and 61 of the denoised GF-5 data. (a) Original; (

**b**) LRMR; (

**c**) NAILRMA; (

**d**) NGmeet; (

**e**) L1HyMixDe; (

**f**) E3DTV; (

**g**) LRTDTV; (

**h**) HSI-DeNet; (

**i**) HSID-CNN; (

**j**) ours.

**Figure 15.**Base visualization. (

**a**) Visualization of basis vectors; (

**b**) ground truth (up), denoising result with OSPA (middle) and result without OSPA (down).

**Table 1.**Quantitative assessment of different algorithms applied to Washington DC Mall data with Gaussian i.i.d. noise.

Index | Noisy HSI | BM4D | LRMR | NAILRMA | FastHyDe | L1HyMixDe | E3DTV | LRTDTV | HSI-DeNet | HSID-CNN | Ours |
---|---|---|---|---|---|---|---|---|---|---|---|

$\mathit{\sigma}\mathbf{=}\mathbf{25}$ | |||||||||||

MPSNR | 20.17 | 33.02 | 34.69 | 37.16 | 38.84 | 36.60 | 34.23 | 35.81 | 37.50 | 37.46 | 38.31 |

MSSIM | 0.3560 | 0.8952 | 0.9175 | 0.9437 | 0.9629 | 0.9343 | 0.9284 | 0.9441 | 0.9608 | 0.9588 | 0.9652 |

MSAM | 0.3383 | 0.0738 | 0.0608 | 0.0445 | 0.0357 | 0.0474 | 0.1058 | 0.0519 | 0.0446 | 0.0454 | 0.0403 |

$\mathit{\sigma}\mathbf{=}\mathbf{50}$ | |||||||||||

MPSNR | 14.15 | 29.45 | 29.90 | 32.46 | 35.10 | 31.57 | 30.89 | 33.36 | 33.60 | 33.42 | 34.46 |

MSSIM | 0.1498 | 0.7888 | 0.8035 | 0.8655 | 0.9254 | 0.8362 | 0.8594 | 0.8997 | 0.9065 | 0.9039 | 0.9206 |

MSAM | 0.6080 | 0.1116 | 0.1045 | 0.0749 | 0.0536 | 0.0821 | 0.1326 | 0.0721 | 0.0702 | 0.0719 | 0.0614 |

$\mathit{\sigma}\mathbf{=}\mathbf{75}$ | |||||||||||

MPSNR | 10.63 | 27.45 | 27.05 | 29.72 | 32.93 | 28.90 | 29.48 | 30.96 | 31.37 | 31.69 | 32.44 |

MSSIM | 0.0780 | 0.6956 | 0.6945 | 0.7959 | 0.8929 | 0.7586 | 0.8241 | 0.8317 | 0.8558 | 0.8612 | 0.8790 |

MSAM | 0.8022 | 0.1409 | 0.1440 | 0.1015 | 0.0679 | 0.1083 | 0.1596 | 0.0979 | 0.0893 | 0.0862 | 0.0775 |

**Table 2.**Quantitative assessment of different algorithms applied to Washington DC Mall data with mixture noise cases.

Index | Noisy HSI | LRMR | NAILRMA | NGmeet | L1HyMixDe | E3DTV | LRTDTV | HSI-DeNet | HSID-CNN | Ours |
---|---|---|---|---|---|---|---|---|---|---|

Case 1 (Gaussian noise) | ||||||||||

MPSNR | 14.54 | 30.30 | 32.29 | 32.59 | 31.31 | 32.04 | 33.11 | 34.08 | 33.87 | 34.74 |

MSSIM | 0.1725 | 0.8103 | 0.8684 | 0.8761 | 0.9029 | 0.8559 | 0.9063 | 0.9143 | 0.9108 | 0.9253 |

MSAM | 0.6261 | 0.1025 | 0.0766 | 0.1027 | 0.0620 | 0.0892 | 0.0746 | 0.0661 | 0.0679 | 0.0601 |

Case 2 (Gaussian noise + Stripes) | ||||||||||

MPSNR | 14.45 | 30.62 | 32.11 | 32.48 | 31.00 | 31.89 | 33.05 | 33.16 | 33.42 | 34.45 |

MSSIM | 0.1697 | 0.8296 | 0.8665 | 0.8744 | 0.8972 | 0. 8524 | 0.9055 | 0.8970 | 0.9036 | 0.9211 |

MSAM | 0.6291 | 0.0970 | 0.0785 | 0.1033 | 0.0649 | 0. 0909 | 0.0784 | 0.0754 | 0.0717 | 0.0628 |

Case 3 (Gaussian noise + Deadlines) | ||||||||||

MPSNR | 14.36 | 29.19 | 31.11 | 31.49 | 29.00 | 31.00 | 32.31 | 34.01 | 33.22 | 34.39 |

MSSIM | 0.1647 | 0.7926 | 0.8353 | 0.8689 | 0.8547 | 0.8434 | 0.8983 | 0.9143 | 0.9003 | 0.9213 |

MSAM | 0.6431 | 0.1248 | 0.0944 | 0.1166 | 0.0927 | 0.1042 | 0.0869 | 0.0666 | 0.0742 | 0.0629 |

Case 4 (Gaussian noise + Impulse noise) | ||||||||||

MPSNR | 12.69 | 28.70 | 26.62 | 28.80 | 29.44 | 31.03 | 31.69 | 33.05 | 32.52 | 33.93 |

MSSIM | 0.1256 | 0.7585 | 0.7423 | 0.8204 | 0.8787 | 0.8376 | 0.8815 | 0.8933 | 0.8853 | 0.9118 |

MSAM | 0.7344 | 0.1571 | 0.2862 | 0.2825 | 0.0880 | 0.1092 | 0.1236 | 0.0755 | 0.0790 | 0.0668 |

Case 5 (Mixture noise) | ||||||||||

MPSNR | 12.38 | 27.11 | 25.07 | 26.71 | 26.41 | 29.44 | 30.22 | 32.47 | 31.83 | 33.56 |

MSSIM | 0.1163 | 0.7253 | 0.7116 | 0.7923 | 0.8060 | 0.8065 | 0.8585 | 0.8804 | 0.8683 | 0.9050 |

MSAM | 0.7634 | 0.1867 | 0.3175 | 0.3108 | 0.1417 | 0.1336 | 0.1337 | 0.0799 | 0.0854 | 0.0697 |

**Table 3.**Quantitative assessment of different algorithms applied to Pavia Centre data with mixture noise cases.

Index | Noisy HSI | LRMR | NAILRMA | NGmeet | L1HyMixDe | E3DTV | LRTDTV | HSI-DeNet | HSID-CNN | Ours |
---|---|---|---|---|---|---|---|---|---|---|

Case 1 (Gaussian noise) | ||||||||||

MPSNR | 14.45 | 27.59 | 30.70 | 29.22 | 31.53 | 28.63 | 31.98 | 31.37 | 31.56 | 31.98 |

MSSIM | 0.1636 | 0.7413 | 0.8629 | 0.7962 | 0.8739 | 0.7678 | 0.8760 | 0.8956 | 0.9017 | 0.9099 |

MSAM | 0.9154 | 0.3223 | 0.1559 | 0.3712 | 0.1483 | 0.2714 | 0.1760 | 0.1719 | 0.1488 | 0.1342 |

Case 2 (Gaussian noise + Stripes) | ||||||||||

MPSNR | 14.39 | 28.73 | 30.58 | 29.06 | 31.24 | 28.42 | 31.11 | 31.21 | 31.42 | 31.70 |

MSSIM | 0.1619 | 0.7937 | 0.8610 | 0.7919 | 0.8660 | 0.7611 | 0.8588 | 0.8964 | 0.8963 | 0.9070 |

MSAM | 0.9186 | 0.2579 | 0.1590 | 0.3794 | 0.1601 | 0.2771 | 0.1545 | 0.1656 | 0.1543 | 0.1333 |

Case 3 (Gaussian noise + Deadlines) | ||||||||||

MPSNR | 14.47 | 28.12 | 30.19 | 29.05 | 29.37 | 28.04 | 30.44 | 31.29 | 31.39 | 31.72 |

MSSIM | 0.1607 | 0.7844 | 0.8591 | 0.8022 | 0.8157 | 0.7517 | 0.8507 | 0.8949 | 0.8952 | 0.9073 |

MSAM | 0.9270 | 0.2832 | 0.1735 | 0.3712 | 0.2463 | 0.2954 | 0.1456 | 0.1645 | 0.1495 | 0.1323 |

Case 4 (Gaussian noise + Impulse noise) | ||||||||||

MPSNR | 12.91 | 26.89 | 24.85 | 25.27 | 29.79 | 27.95 | 30.31 | 30.57 | 30.68 | 30.83 |

MSSIM | 0.1323 | 0.7203 | 0.7007 | 0.7226 | 0.8267 | 0.7276 | 0.8349 | 0.8806 | 0.8766 | 0.8902 |

MSAM | 0.9174 | 0.4354 | 0.3736 | 0.4967 | 0.1843 | 0.3098 | 0.1627 | 0.1660 | 0.1735 | 0.1419 |

Case 5 (Mixture noise) | ||||||||||

MPSNR | 12.72 | 25.71 | 23.80 | 24.31 | 27.75 | 27.01 | 29.05 | 30.19 | 29.79 | 30.35 |

MSSIM | 0.1246 | 0.6933 | 0.6694 | 0.7025 | 0.7655 | 0.7038 | 0.8209 | 0.8742 | 0.8616 | 0.8777 |

MSAM | 0.9313 | 0.4601 | 0.3985 | 0.5047 | 0.2764 | 0.3420 | 0.2912 | 0.1787 | 0.1844 | 0.1654 |

Method | LRMR | NAILRMA | NGmeet | L1HyMixDe | E3DTV | LRTDTV | HSI-DeNet | HSID-CNN | Ours |
---|---|---|---|---|---|---|---|---|---|

Time (s) | 127.8 | 116.5 | 33.9 | 37.4 | 44.4 | 170.1 | 6.1 | 3.3 | 17.0 |

Index | Original | LRMR | NAILRMA | NGmeet | L1HyMixDe | E3DTV | LRTDTV | HSI-DeNet | HSID-CNN | Ours |
---|---|---|---|---|---|---|---|---|---|---|

RF | ||||||||||

OA | 0.8065 | 0.8510 | 0.8520 | 0.8564 | 0.8370 | 0.8639 | 0.8068 | 0.8543 | 0.8414 | 0.8812 |

Kappa | 0.7071 | 0.8150 | 0.8111 | 0.8134 | 0.7613 | 0.7921 | 0.7353 | 0.7701 | 0.7837 | 0.8587 |

SVM | ||||||||||

OA | 0.7606 | 0.8210 | 0.7926 | 0.5948 | 0.8008 | 0.7921 | 0.5912 | 0.8665 | 0.8862 | 0.8997 |

Kappa | 0.6841 | 0.7800 | 0.7592 | 0.5661 | 0.7772 | 0.7457 | 0.5650 | 0.7857 | 0.8158 | 0.8174 |

P | 1 | 10 | 20 | 30 |
---|---|---|---|---|

MPSNR | 33.04 | 33.56 | 33.45 | - |

MSSIM | 0.8952 | 0.9050 | 0.9026 | - |

MSAM | 0.0742 | 0.0697 | 0.0710 | - |

Component Modules | Accuracy Indicators | |||
---|---|---|---|---|

OSPA | CA | MPSNR | MSSIM | MSAM |

✖ | ✖ | 32.04 | 0.8736 | 0.0832 |

✓ | ✖ | 32.55 | 0.8865 | 0.0771 |

✖ | ✓ | 32.89 | 0.8931 | 0.0742 |

✓ | ✓ | 33.56 | 0.9050 | 0.0697 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Sun, H.; Zheng, K.; Liu, M.; Li, C.; Yang, D.; Li, J.
Hyperspectral Image Mixed Noise Removal Using a Subspace Projection Attention and Residual Channel Attention Network. *Remote Sens.* **2022**, *14*, 2071.
https://doi.org/10.3390/rs14092071

**AMA Style**

Sun H, Zheng K, Liu M, Li C, Yang D, Li J.
Hyperspectral Image Mixed Noise Removal Using a Subspace Projection Attention and Residual Channel Attention Network. *Remote Sensing*. 2022; 14(9):2071.
https://doi.org/10.3390/rs14092071

**Chicago/Turabian Style**

Sun, Hezhi, Ke Zheng, Ming Liu, Chao Li, Dong Yang, and Jindong Li.
2022. "Hyperspectral Image Mixed Noise Removal Using a Subspace Projection Attention and Residual Channel Attention Network" *Remote Sensing* 14, no. 9: 2071.
https://doi.org/10.3390/rs14092071