Edge-Preserved Low-Rank Representation via Multi-Level Knowledge Incorporation for Remote Sensing Image Denoising

Abstract

The low-rank models have gained remarkable performance in the field of remote sensing image denoising. Nonetheless, the existing low-rank-based methods view residues as noise and simply discard them. This causes denoised results to lose many important details, especially the edges. In this paper, we propose a new denoising method named EPLRR-RSID, which focuses on edge preservation to improve the image quality of the details. Specifically, we considered the low-rank residues as a combination of useful edges and noisy components. In order to better learn the edge information from the low-rank representation (LRR), we designed multi-level knowledge to further distinguish the edge part and the noise part from the residues. Furthermore, a manifold learning framework was introduced in our proposed model to better obtain the edge information, as it can find the structural similarity of the edge part while suppressing the influence of the non-structural noise part. In this way, not only the low-rank part is better learned, but also the edge part is precisely preserved. Extensive experiments on synthetic and several real remote sensing datasets showed that EPLRR-RSID has superior advantages over the compared state-of-the-art (SOTA) approaches, with the mean edge protect index (MEPI) values reaching at least 0.9 and the best values in the no-reference index BRISQUE, which represents that our method improved the image quality by edge preserving.

1. Introduction

Remote sensing images have been widely used in a variety of areas, such as agricultural resource research, city planning, and others. However, additive noise is a critical problem for remote sensing images [1], which seriously affects their practical use and subsequent processing. Thus, remote sensing image denoising has been a hot issue. As a result, various remote sensing image denoising research works have been performed in the last few decades, especially for multispectral, hyperspectral, and SAR data [2].

Traditional approaches are mainly filter-based denoising methods with strong theoretical support. These filters can operate in either the spatial or frequency domains directly [3]. The Gaussian filter [4], bilateral filter [5], and guided image filter [6] are all classical spatial domain filters, which use image local information to help remove noise. Unlike these previous image filters, a new method called NL-means [7] has been proposed. It utilizes the redundant information of the original image to obtain a better result. Inspired by this, block-matching and 3D filtering (BM3D) [8] makes improvements on the basis of NL-means, which not only uses intra-fragment correlation, but also takes inter-correlation into account. The total variation (TV) algorithm [9] is an anisotropic model whose core is to use the gradient descent method to remove noise. The superiority of this method is that it smooths images while keeping the edges as much as possible. Due to the advantage of preserving the edge information, TV regulation has been added to many state-of-the-art (SOTA) denoising models.

Recently, low-rank representation (LRR) has gained much attention and has been successfully extended to the denoising field [10,11,12]. The low rank is an important feature, which indicates that high-dimensional data can be projected into a low-dimensional subspace. Based on this, several methods have been proposed for denoising. A method called NAILRMA [13] proposes an iterative regularization framework and tries to realize the purpose of denoising from the signal subspace. In NAILRMA, a noise-variance-based adaptive iteration factor selection method is used for each hyperspectral image (HSI) band. Meanwhile, some tensor-based low-rank denoising methods [14,15,16] have achieved satisfactory results.

However, such low-rank-based methods have a common problem, that is the over-smoothing caused by the loss of image edges. This drawback is fairly common due to the characteristic of LRR. The traditional LRR model filters out high-frequency edges as residues, which will be considered as only noise and be disregarded. In order to compensate for this defect, many researchers began to explore methods to enhance image edges. Some research has introduced TV regulation into low-rank models [17,18,19,20], trying to take advantage of the edge-preserving ability of the TV algorithm. A new method that makes use of low-rank regularization and a texture preserving prior [21] has then been proposed. Meanwhile, LRTL0 [22] uses the $l_0$ gradient regularization to realize edge preservation. In addition, more cutting-edge methods combining the latest developments and denoising models have been proposed. Liu et al. proposed a multi-source-information-based method [23], which employs priors to dictionary learning and edge feature predicting. It uses uncontaminated images as references to construct its own denoising object function, so that more details can be preserved through the similarities between the noisy image and clean reference image. A deep learning denoising network named RSIDNet [24] improves the usage of feature extraction through the combination of multiple modules, trying to reduce the loss of important details in the process of image denoising. The nonlocal 3D convolutional neural network (NL-3DCNN) [25] makes the best of the images’ nonlocal similarity and simultaneously utilizes the advantages of machine learning and deep learning. A large amount of novel approaches focus on tensor decomposition [26,27,28], which can avoid converting the original dimensions to keep as much information as possible. Furthermore, some filter-based methods [29,30,31] have been developed in recent years. They improve existing designs from a number of perspectives, including thresholding, image pre-processing, and building 3D dynamic kernels to address the shortcomings of traditional filters. Even though great efforts have been made in the edge-preserving field, the performances of existing methods still need to be improved to extract clearer edges by taking advantage of the edge information in residues.

In this paper, a novel edge-preserved denoising method for remote sensing images named EPLRR-RSID is proposed by concurrently describing the edge characteristics and LRR model. Most existing methods only care about whether the noise in remote sensing images has been effectively removed, which leads to the loss of edge features. Although a few advanced methods have considered the importance of edge preservation, their performances are not good enough due to the effect of inevitable noise. To overcome the aforementioned problems, we explored a more robust method to find edge features. We creatively considered the residues of LRR models as a mixture of high-frequency edge components and noisy components. Thus, we tried to extract clearer edges from the residues to remedy the shortcoming of traditional LRR methods. Unlike conventional methods, we considered the residues of LRR models as a mixture of high-frequency edge components and noisy components. Thus, we tried to extract clearer edges from the residues to remedy the shortcoming of traditional LRR methods. As shown in Figure 1, we first propose a new prior knowledge to better find structural edge information in the enhanced low-rank part of the residues. Then, manifold learning was integrated with our model to gain structural edge information discovered by the prior knowledge much better with affinity graph regularization. Through this framework, valuable edge information was extracted from those residues with smaller distances between them while the noisy part was suppressed due to its disorder. After the multi-level modeling, we can obtain the low-rank component, which contains blurred boundaries and edges, together with the distinct edge component, respectively. Thus, our proposed EPLRR-RSID can better preserve edge information while reducing noise, which provides high-quality denoised images with more edge details.

We list our main contributions as follows:

• Different from conventional LRR-based denoising methods, we took a very different way to deal with low-rank residues, i.e., we did not simply discard them, but viewed those residues as a combination, which includes both edge information and the noisy part. By taking this different view, we tried to extract structural edge information in the residues to enrich the details of remote sensing images. To our best knowledge, this is the first time that the edge part in the residues has been valued and fully utilized.
• In order to better learn the edge information from the residues, we designed a new multi-level prior knowledge regulation. By incorporating it, our proposed method can further separate the edge part from the noise part. Thus, the structural edge information can be further discovered by the manifold learning framework, which can find the structural similarity of the edge part in the enhanced low-rank part of the residues. This enabled our algorithm to preserve more high-frequency edge components.
• Several experiments were conducted to prove the effectiveness of EPLRR-RSID. According to the experimental results, EPLRR-RSID had an outstanding advantage in preserving image edges, which made our method more competitive.

2. Related Works

As we previously mentioned, LRR has laid the foundation for many low-rank-based denoising methods [11,32,33]. In the previous works, we noticed that many researchers tried to design rational image priors that could inspire some significant properties [14]. A low-rank tensor recovery method named LLRT [34] introduces an analysis-based hyper-Laplacian prior to address the problem of ringing artifacts. This prior can help the model better use the structure information of the images. A new kind of mixed prior was proven effective, which combines the external patch prior and the internal self-similarity prior [35]. A three-stage mixed denoising model [36] was creatively proposed, which employs a new global-guided nonlocal prior mechanism. These methods try to find the most-appropriate prior to reconstruct the noise-free image from the low-rank model.

Among these methods, a weighted group LRR model called WGLRR [37] has attracted our attention. According to WGLRR, it is quite hard to recover a noise-free image from a noisy image if there is no side information for a heavily corrupted image. Different from other methods that use prior knowledge, it introduces a local image rank-ordered absolute differences statistic called ROAD [38] to construct its prior knowledge P creatively. After obtaining the P matrix, the WGLRR model is written as

$$\begin{array}{c}\underset{Z,E}{min}{∥Z∥}_{*}+\lambda {∥E∥}_F^2\hfill \\ s.t.P\circ X=P\circ Z+E\hfill \end{array}$$

(1)

where X is the image with the original noise, Z is the low-rank component, and E is the noisy component. P represents a weight matrix to constrain the fidelity of recovered clean patches. “∘” denotes the Hadamard product.

However, such LRR methods have an unavoidable problem: they will filter out high-frequency signals together with corrupt elements during the denoising procedure, which means useful edge information will be abandoned. This drawback is inherited from the characteristic of the traditional LRR model, which only focuses on how to preserve a low-rank structure while ignoring the importance of edge information.

3. Methodology

We define the objective function and provide an optimization strategy that can successfully tackle the problem. What needs to be declared in advance is that the whole model was completed under the NLM framework [39]. We divided the noisy image into several patches to create group matrices, and then, our method was applied to each group matrix.

3.1. Model Formulation

As mentioned earlier, most low-rank-based denoising methods cannot avoid the problem of over-smoothing, because they always filter out high-frequency edge information along with the noise. This problem leads to the reduction of remote sensing image quality and negatively impacts downstream tasks. Hence, we tried to build a denoising model that can preserve edge details to obtain clear and high-quality remote sensing images. We note that the traditional LRR model only cares about keeping the overall image structure, but neglects the edges by filtering them out as part of the residues. Motivated by this phenomenon, we tried to greatly utilize the edge information in the residues to enrich the edge details of the denoised images, which can achieve further improvement of the low-rank model.

Inspired by the previous works on prior knowledge, it can be clearly observed that prior knowledge is useful in the aspect of smoothing noisy parts. In (1), WGLRR uses the ROAD statistic to design its prior knowledge matrix, which can distinguish between clean pixels and noisy pixels. On this basis, we modified the prior knowledge matrix in WGLRR to be more suitable for remote sensing images and integrated it into our model. Therefore, our model is formulated as

$$\begin{array}{c}\underset{Z,R}{min}{∥Z∥}_{*}+\lambda {∥R∥}_{2,1}\hfill \\ s.t.P\circ X=P\circ Z+R\hfill \end{array}$$

(2)

where P represents the prior knowledge matrix we designed for remote sensing images. R represents the low-rank residues. X and Z still have the same meaning as (1).

However, we cannot directly extract edge information from R, because there is much noise in it, which will affect the subsequent learning effect of the edge subspace. Then, we further propose a more relaxed prior knowledge to help better extract edge regions in the residues. By using this prior knowledge, it can ensure that noise and clean pixels are separated and more edge information is preserved in the clean part. Therefore, we introduce a new constraint term on the basis of (2).

$$\begin{array}{c}\underset{Z,R,R_1,R_2}{min}{∥Z∥}_{*}+\lambda {∥R∥}_{2,1}\hfill \\ s.t.P_1\circ X=P_1\circ Z+R\hfill \\ P_2\circ R=P_2\circ R_1+R_2\hfill \end{array}$$

(3)

Through this new term, the original residue R is separated into two parts: edge information $R_1$ and the noise part after extraction $R_2$. Note that we used different P matrices in (3), which is a special design for our multi-level model. Here, we set two different prior matrices for different purposes. $P_1$ was employed to obtain the low-rank part from the noisy image, while $P_2$ was employed to find more edges from the residues. As we anticipated, $P_1$ should be stricter to ensure the noise will be removed as much as possible. The constraint of $P_2$ was relatively loose, because we wanted to use $P_2$ to extract more useful information from the residues, even though some noise will also be introduced. Through setting the two P matrices, we constructed a multi-level model, where the low-rank part and the edge part are obtained, respectively, at the corresponding level.

In order to learn the edge information we need, a manifold learning framework was then incorporated in our model, which can further learn the structural information from $R_1$, so as to extract the edge subspace. Specifically, we tried to find the robust projection $V=(v_1,v_2,...,v_r)$ from $R_1$ and make sure that the geometric structural information of the data will be seized concurrently along the way.

$$\begin{array}{c}\underset{V,Z,W,R,R_1,R_2}{min}\frac{1}{2}{\displaystyle \sum _{i,j}}{∥V^T{r}_{1,i}-V^T{r}_{1,j}∥}_2^2{W}_{ij}+{\lambda }_3{∥W∥}_F^2\hfill \\ s.t.P_1\circ X=P_1\circ Z+R\hfill \\ P_2\circ R=P_2\circ R_1+R_2\hfill \\ V^{T}V=I\hfill \\ W1=1,W_{ii}=0,W\ge 0\hfill \end{array}$$

(4)

where $\frac{1}{2}{\displaystyle \sum _{i,j}}{∥V^T{r}_{1,i}-V^T{r}_{1,j}∥}_2^2$ is a coupling term integrated into our model, because it is impossible to predict which part of the data will be influenced by noise in advance. In short, if two similar samples $r_{1,i}$ and $r_{1,j}$ have very small values, they will mainly contain edge information, which also indicates that the distance between them will be quite small. Therefore, it is possible for us to extract those edge details from residue R. In contrast, the distance will be large if the noise part exceeds the edge information part. As a result, V is adaptively learned with similar residues through decreasing the effect of distant pairs by $\frac{1}{2}{\displaystyle \sum _{i,j}}{∥V^T{r}_{1,i}-V^T{r}_{1,j}∥}_2^2$. Put differently, a dynamic affinity graph W is integrated by utilizing similar points in $R_i$, which is useful for the reconstruction of edge information. To achieve this goal, we set $W_{ij}$ as non-zero when two residue samples were similar. Otherwise, we set $W_{ij}$ as zero, attempting to mitigate the effect of noise. To effectuate our algorithm, we added two nonnegative constraints $W_{ii}=0$ and $W\ge 0$. $W1=1$ was added into the model as well to guarantee that the affinity graph will be normalized.

After all, considering that the purpose of our method is to preserve the edge information as much as possible, we further added a TV operator to our model, which is widely used due to its outstanding performance on edge preservation. We can obtain the complete model:

$$\begin{array}{c}\underset{V,Z,W,R,R_1,R_2}{min}\frac{1}{2}{\displaystyle \sum _{i,j}}{∥V^T{r}_{1,i}-V^T{r}_{1,j}∥}_2^2{W}_{ij}+{\lambda }_1{∥Z∥}_{*}\hfill \\ +{\lambda }_2{∥R∥}_{2,1}+{\lambda }_3{∥W∥}_F^2+{\lambda }_4{∥Z∥}_{TV}\hfill \\ s.t.P_1\circ X=P_1\circ Z+R\hfill \\ P_2\circ R=P_2\circ R_1+R_2\hfill \\ V^{T}V=I\hfill \\ W1=1,W_{ii}=0,W\ge 0\hfill \end{array}$$

(5)

In order to solve the problem of (5), we propose an efficient optimization algorithm. Besides, we also introduced another variable F to avoid Sylvester’s equation appearing in the solution of $\widehat{R}$. First, the following auxiliary terms $S=Z$, $R_1=\widehat{R}$, $Z=T$, and $G=R$ were considered to make (5) easier to solve. Next, the coupling term $\frac{1}{2}{\displaystyle \sum _{i,j}}{∥V^T{r}_{1,i}-V^T{r}_{1,j}∥}_2^2{W}_{ij}$ was reformulated as $tr\left(V^T\left(FL{\widehat{R}}^T\right)V\right)$, where $L=diag{\displaystyle \sum _i}{W}_{ij}-W$ means the graph Laplacian of W. Here, $diag$ means the operation of making a diagonal matrix. In addition, considering the existence of Sylvester’s equation, we introduced another variable F during the solution of $\widehat{R}$. Consequently, the whole model can be described as

$$\begin{array}{c}\underset{V,Z,S,W,R,R_1,R_2,\widehat{R},F,T,G}{min}tr\left(V^T\left(FL{\widehat{R}}^T\right)V\right)+{\lambda }_1{∥S∥}_{*}+{\lambda }_2{∥R∥}_{2,1}+{\lambda }_3{∥W∥}_F^2+{\lambda }_4{∥T∥}_{TV}\hfill \\ s.t.P_1\circ X=P_1\circ Z+R\hfill \\ P_2\circ R=P_2\circ R_1+R_2\hfill \\ R_1=\widehat{R},S=Z,T=Z,G=R\hfill \\ V^{T}V=I\hfill \\ W1=1,W_{ii}=0,W\ge 0\hfill \end{array}$$

(6)

3.2. Design of Multi-Level Knowledge

In this section, we propose the special design of a novel multi-level knowledge regulation. We hope that our prior knowledge can distinguish between clean pixels and noise, so that the manifold learning introduced to our model can better learn the edge details we need. However, this requirement is usually put forward for optical images. Therefore, we attempted to achieve this goal through prior knowledge for remote sensing images. Such prior knowledge should simultaneously combine the characteristics of both optical and remote sensing images. We laid the foundation of our prior knowledge on the basis of existing methods. We have mentioned before that the ROAD statistic was introduced into a trilateral filter to reduce mixed noise. Thanks to that foundation, a new statistic called ROLD [40] has been introduced, which is able to identify contaminated pixels more accurately. ROLD makes an improvement to ROAD, which helps ROLD better distinguish between clean and noisy pixels. Then, Yu et al. [41] created a calculation for the uncorrupted probability of pixel $q_u$.

$$P=exp(-0.01ROAD\left(q_u\right))$$

(7)

We combined (7) with the ROLD statistic to better distinguish the two types of pixels. Furthermore, we attempted to set an adjustable coefficient to meet our needs under multi-level modeling. Therefore, (7) can be written as

$$P=exp(-\alpha ROLD\left(q_u\right))$$

(8)

where $\alpha$ represents the coefficient, helping gain various image features we need under distinct conditions. In our multi-level model, we set different $\alpha$ for different levels. At the first level, the $\alpha$ was large to easily find the difference between clean pixels and noisy pixels. This setting ensured that noise will be filtered out to the greatest extent. At the second level, the $\alpha$ was set to be small relatively so as to preserve the edge information we need. With different values of $\alpha$, we can achieve different P matrices in (3).

3.3. Optimization Procedure

In this section, we try to tackle the problem we propose in (6). An efficient optimization algorithm was applied in our method. Each variable will be optimized one after another. Firstly, we obtain the augmented Lagrange function of (6):

$$\begin{array}{c}\underset{V,Z,T,S,W,R,\widehat{R},R_1,R_2,F,G}{min}tr\left(V^T\left(FL{\widehat{R}}^T\right)V\right)+{\lambda }_1{∥S∥}_{*}\hfill \\ +{\lambda }_2{∥R∥}_{2,1}+{\lambda }_3{∥W∥}_F^2+{\lambda }_4{\left|T\right|}_{TV}+tr\left(Y_1^T(P_1\circ X-P_1\circ Z-R)\right)\hfill \\ +tr\left(Y_2^T(P_2\circ G-P_2\circ R_1-R_2)\right)+tr\left(Y_3^T(S-Z)\right)+tr\left(Y_4^T(\widehat{R}-R_1)\right)\hfill \\ +tr(Y_5^T(F-\widehat{R})+tr(Y_6^T(T-Z)+tr\left(Y_7^T(G-R)\right)\hfill \\ +\frac{\mu }{2}({∥P_1\circ X-P_1\circ Z-R∥}_F^2+{∥P_2\circ G-P_2\circ R_1-R_2∥}_F^2\hfill \\ +{∥S-Z∥}_F^2+{∥\widehat{R}-R_1∥}_F^2+{∥F-\widehat{R}∥}_F^2+{∥T-Z∥}_F^2+{∥G-R∥}_F^2)\hfill \\ s.t.V^{T}V=I\hfill \\ W1=1,W_{ii}=0,W\ge 0\hfill \end{array}$$

(9)

where $Y_1$, $Y_2$, $Y_3$, $Y_4$, $Y_5$, $Y_6$, and $Y_7$ are Lagrange multiples, which are used to deal with constrained optimization problems.

V subproblem:

This subproblem can be solved by removing the unrelated terms from (9):

$$\begin{array}{c}\underset{V}{min}tr\left(V^T\left(FL{\widehat{R}}^T\right)V\right)\hfill \\ s.t.V^{T}V=I\hfill \end{array}$$

(10)

Denoting $FL{\widehat{R}}^T=Q$, we can obtain

$$\begin{array}{c}\underset{V}{min}tr\left(V^{T}QV\right)\hfill \\ s.t.V^{T}V=I\hfill \end{array}$$

(11)

Notice that (11) is an eigenvalue decomposition problem. To solve it, the set of k eigenvectors of the topmost k smallest eigenvalues of Q is needed.

Z subproblem:

By excluding other unrelated variables, we can simplify the optimal problem of Z as follows:

$$\begin{array}{c}\underset{Z}{min}{∥P_1\circ X-P_1\circ Z-R+\frac{Y_1}{\mu }∥}_F^2\hfill \\ +{∥S-Z+\frac{Y_3}{\mu }∥}_F^2+{∥T-Z+\frac{Y_6}{\mu }∥}_F^2\hfill \end{array}$$

(12)

Setting the derivative $\frac{\partial }{\partial Z}=0$, the optimization procedure of Z can be described as:

$$Z=[(P_1\circ X\circ P_1)-(R\circ P_1)+(\frac{Y_1}{\mu }\circ P_1)+S+\frac{Y_3}{\mu }+T+\frac{Y_6}{\mu }]÷(P_1\circ P_1+2I)$$

(13)

where “÷” denotes elementwise division of two matrices.

T subproblem:

$$\underset{T}{min}{\lambda }_4{\left|T\right|}_{TV}+\frac{\mu }{2}{∥T-Z+\frac{Y_6}{\mu }∥}_F^2$$

(14)

To solve this subproblem, an iterative method was developed [42]. Firstly, U is defined as the set of matrix pairs $(u_1,u_2)$, $u_1\in R^{(m-1)\times n}$, $u_2\in R^{m\times (n-1)}$. Then, we can obtain

$$T=U((Z-\frac{Y_6}{\mu })-\frac{2{\lambda }_4}{\mu }L(u_1,u_2))$$

(15)

where L represents the linear operation, which can change the elements to the size of $m\times n$. Finally, the strategy of fast gradient projection (FGP) [43] was used to solve this subproblem.

S subproblem:

$$\underset{S}{min}{\lambda }_1{∥S∥}_{*}+\frac{\mu }{2}{∥S-Z+\frac{Y_3}{\mu }∥}_F^2$$

(16)

By utilizing the singular-value thresholding (SVT), the optimization procedure of S can be described as

$$S={\Phi }_{\frac{{\lambda }_1}{\mu }}(Z-\frac{Y_3}{\mu })$$

(17)

where $\Phi$ represents a soft-thresholding operator.

W subproblem:

$$\begin{array}{c}\underset{W}{min}\frac{1}{2}{\displaystyle \sum _{i,j}}{∥V^T{r}_{1,i}-V^T{r}_{1,j}∥}_2^2{W}_{ij}+{\lambda }_3{∥W∥}_F^2\hfill \\ s.t.W1=1,W_{ii}=0,W\ge 0\hfill \end{array}$$

(18)

It can be found that (18) is independent for each i; therefore, the problem of $W_i$ could be rewritten as follows:

$$\begin{array}{c}\underset{W_i}{min}\frac{1}{2}{\displaystyle \sum _{i,j}}{∥V^T{r}_{1,i}-V^T{r}_{1,j}∥}_2^2{W}_{ij}+{\lambda }_3{∥W_i∥}_F^2\hfill \\ s.t.W1=1,W_{ii}=0,W\ge 0\hfill \end{array}$$

(19)

For simplicity, we used $A_{ij}$ to symbolize $V^T{r}_{1,i}-V^T{r}_{1,j}$; as a result, we have

$$\begin{array}{c}\underset{W_i}{min}{\displaystyle \sum _j}{∥W_i+\frac{A_i}{{\lambda }_3}∥}_2^2\hfill \\ s.t.W1=1,W_{ii}=0,W\ge 0\hfill \end{array}$$

(20)

Furthermore, it can be unconstrained as

$$\underset{W_i}{min}{\displaystyle \sum _j}{∥W_i+\frac{A_i}{{\lambda }_3}∥}_2^2-\eta (1^T{W}_i-1)-{\zeta }^T{W}_i$$

(21)

Next, setting the derivative $\frac{\partial }{\partial W_i}=0$, we have

$$W_i+\frac{1A_i}{{\lambda }_3}-\eta 1-\zeta =0$$

(22)

$W_i$’s j-th entry can be obtained through the following formula.

$$W_{ij}+\frac{A_{ij}}{{\lambda }_3}-\eta -{\zeta }_i=0$$

(23)

Based on the KKT conditions, $W_{ij}$ can be updated as follows:

$$W_{ij}={(\frac{-A_{ij}}{{\lambda }_3}+\eta )}_{+}$$

(24)

R subproblem:

$$\begin{array}{c}\underset{R}{min}{\lambda }_2{∥R∥}_{2,1}+\frac{\mu }{2}{∥P_1\circ X-P_1\circ Z-R+\frac{Y_1}{\mu }∥}_F^2\hfill \\ +\frac{\mu }{2}{∥G-R+\frac{Y_7}{\mu }∥}_F^2\hfill \end{array}$$

(25)

Denote $T=\frac{1}{2}(P_1\circ X-P_1\circ Z+G+\frac{Y_1}{\mu }+\frac{Y_7}{\mu })$, $\tau =\frac{{\lambda }_2}{\mu }$.

The i-$th$ column of R is given as follows:

$$R(:,i)=\left\{\begin{array}{c}\frac{∥t_i∥-\tau }{∥t_i∥}{t}_i,if\tau <∥t_i∥\hfill \\ 0,otherwise\hfill \end{array}\right.$$

(26)

$\widehat{R}$ subproblem:

$$\underset{\widehat{R}}{min}tr\left(V^T\left(FL{\widehat{R}}^T\right)V\right)+\mu {∥\widehat{R}-R_1+\frac{Y_4}{\mu }∥}_F^2+\mu {∥F-\widehat{R}+\frac{Y_5}{\mu }∥}_F^2$$

(27)

Setting the derivative $\frac{\partial }{\partial \widehat{R}}=0$, then the final result of the optimal $\widehat{R}$ can be described as follows:

$$\widehat{R}=\frac{2\mu R_1-V{V}^{T}FL-2Y_4+2\mu F+2Y_5}{4\mu }$$

(28)

$R_1$ subproblem:

$$\underset{R_1}{min}{∥P_2\circ G-P_2\circ R_1-R_2+\frac{Y_2}{\mu }∥}_F^2+{∥\widehat{R}-R_1+\frac{Y_4}{\mu }∥}_F^2$$

(29)

Setting the derivative $\frac{\partial }{\partial R_1}=0$, then the final result of the optimal $R_1$ can be described as follows:

$$R_1=(P_2\circ G-R_2+\widehat{R}+\frac{Y_2}{\mu }+\frac{Y_4}{\mu })÷(P_2+1)$$

(30)

$R_2$ subproblem:

$$\underset{R_2}{min}{∥P_2\circ G-P_2\circ R_1-R_2+\frac{Y_2}{\mu }∥}_F^2$$

(31)

Setting the derivative $\frac{\partial }{\partial R_2}=0$, then we can obtain the optimal $R_2$.

$$R_2=(P_2\circ G)-(P_2\circ R_1)+\frac{Y_2}{\mu }$$

(32)

F subproblem:

$$\underset{F}{min}tr\left(V^T\left(FL{\widehat{R}}^T\right)V\right)+\mu {∥F-\widehat{R}+\frac{Y_5}{\mu }∥}_F^2$$

(33)

Setting the derivative $\frac{\partial }{\partial F}=0$, then F is obtained as follows:

$$F=\frac{2\mu \widehat{R}-P{P}^T\widehat{R}{L}^T-2Y_5}{2\mu }$$

(34)

G subproblem:

$$\underset{G}{min}{∥P_2\circ G-P_2\circ R_1-R_2+\frac{Y_2}{\mu }∥}_F^2+{∥G-R+\frac{Y_7}{\mu }∥}_F^2$$

(35)

Setting the derivative $\frac{\partial }{\partial G}=0$, then G is obtained as follows:

$$G=(P_2\circ P_2\circ R_1+P_2\circ R_2-P_2\circ \frac{Y_2}{\mu }+R-\frac{Y_7}{\mu })÷(1+P_2\circ P_2)$$

(36)

To sum up, Algorithm 1 outlines the initialization of each variable and the multipliers.

Algorithm 1 The proposed EPLRR-RSID algorithm.

1: Input: Data $X_i\in R^{d\times n}$, parameter ${\lambda }_1,{\lambda }_2,{\lambda }_3,{\lambda }_4$ 2: Initialize: $Z=S=W=R=R_1=R_2=\widehat{R}=F=T=G=0$; $Y_1=Y_2=Y_3=Y_4=Y_5=Y_6=Y_7=0$; the projection V with orthogonal column vector; $\rho =1.1$; $\mu ={10}^{-6}$; ${\mu }_{max}={10}^6$; $\epsilon ={10}^{-5}$; 3: while not converged do 4:    Update $V,Z,T,S,W,R,\widehat{R},R_1,R_2,F,G$ by (11), (13), (15), (17), (24), (26), (28), (30), (32), (34), and (36). 5:    Update the multipliers $Y_1,Y_2,Y_3,Y_4,Y_5,Y_6,Y_7$. $Y_1=Y_1+\mu (P_1\circ X-P_1\circ Z-R)$, $Y_2=Y_2+\mu (P_2\circ G-P_2\circ R_1-R_2)$, $Y_3=Y_3+\mu (S-Z)$, $Y_4=Y_4+\mu (\widehat{R}-R_1)$, $Y_5=Y_5+\mu (F-\widehat{R})$, $Y_6=Y_6+\mu (T-Z)$, $Y_7=Y_7+\mu (G-R)$ 6:    Update $\mu$ by $\mu =min(\rho \mu ,{\mu }_{max})$. 7:    Check the following convergence conditions: $\begin{array}{c}{∥P_1\circ X-P_1\circ Z-R∥}_{\infty }<\epsilon ,\hfill \\ {∥P_2\circ G-P_2\circ R_1-R_2∥}_{\infty }<\epsilon ,\hfill \\ {∥S-Z∥}_{\infty }<\epsilon ,{∥\widehat{R}-R_1∥}_{\infty }<\epsilon ,\hfill \\ {∥F-\widehat{R}∥}_{\infty }<\epsilon ,{∥T-Z∥}_{\infty }<\epsilon ,{∥G-R∥}_{\infty }<\epsilon \hfill \end{array}$ 8: end while 9: Output: $V\in R^{d\times r}$ and $Z\in R^{d\times n}$.

4. Results and Analysis

In this section, both simulated and real image data experiments were undertaken to demonstrate the effectiveness of our method. The results of EPLRR-RSID were compared with following denoising methods: LRTA [44], ANLM3D [45], BM4D [46], TDL [47], ITSReg [48], SNLRSF [49], and RCTV [50].

4.1. Experimental Settings

In the experiments of this paper, there were some parameters that needed to be determined. For the parameters in each algorithm, we set the values according to the optimal settings found in their respective literature works. The settings of parameters used in our experiments are shown in

Table 1. The settings of parameters used in our experiments.

Definition	Notation	Value
Patch size	$P_s$	$9\times 9$
The number of similar patches in each group matrix	$L_s$	81
The parameters in prior knowledge matrices	${\alpha }_1$, ${\alpha }_2$	[0.001, 0.01, 0.1, 1]
Hyperparameters	${\lambda }_1,{\lambda }_2,{\lambda }_3,{\lambda }_4$	[0.0001, 0.01, …, 1000]

.

In the case of EPLRR-RSID, the parameters ${\alpha }_1$ and ${\alpha }_2$ in prior knowledge matrices $P_1$ and $P_2$ have a significant impact on the denoising performance and edge preserving performance of the denoised result, as we mentioned earlier. The specific setting of ${\alpha }_1$ and ${\alpha }_2$ depends on the different datasets. We used [0.001, 0.01, 0.1, 1] as the candidate set for selection. Furthermore, the parameters ${\lambda }_1$, ${\lambda }_2$, ${\lambda }_3$, and ${\lambda }_4$ need tuning to find a robust solution under which the best denoising results will be obtained. However, it was hard for us to find a good combination of these parameters due to the complexity of the different combinations. To the best of our current knowledge, there are no accurate and efficient solutions for this problem in the existing literature. Thus, a reasonable strategy was used from [51] to seek the optimal combination of the parameters. The specific determination of these four parameters is discussed in the following section.

4.2. Experiments on Simulated Data

In this section, we chose one HSI dataset, the Pavia city center dataset (Available: http://www.ehu.es/ccwintco/index.php/Hyperspectral4_4Remote4_4Sensing4_4Scenes accessed on 2 November 2022.)), which is shown in Figure 2. The Pavia city center dataset was collected by the Reflective Optics System Imaging Spectrometer (ROSIS-03), whose size is 1096 × 715 × 102. However, we only used the subimage of size 200 × 200 × 80 (only clean bands). In the simulated experiments, we chose four indexes to assess the final denoised result: the peak signal-to-noise ratio (PSNR), the structural similarity (SSIM), the erreur relative globale adimensionnelle de synthese (ERGAS), and the mean spectral angle distance (MSAD). For an observed image whose size is $m\times n\times b$, the definitions are as follows:

$$PSNR=\frac{{255}^2\times m\times n}{{\displaystyle \sum _{i=1}^M}{\displaystyle \sum _{j=1}^N}{\left|I_g(i,j)-{\widehat{I}}_g(i,j)\right|}^2}$$

(37)

where $I_g$ denotes the original image and ${\widehat{I}}_g$ denotes the final result after denoising.

$$SSIM=\frac{\left(2E\left(I_g\right)E\left(I_g\right)\right)\times \left(2{\sigma }_{I\widehat{I}}\right)}{({\left(E\left(I_g\right)\right)}^2+{\left(E\left({\widehat{I}}_g\right)\right)}^2)({{\sigma }_I}^2+{{\sigma }_{\widehat{I}}}^2)}$$

(38)

where $E\left(I_g\right)$ and $E\left({\widehat{I}}_g\right)$ represents the mean value of $I_g$ and ${\widehat{I}}_g$. ${\sigma }_I^2$ and ${\sigma }_{\widehat{I}}^2$ represent the variance of $I_g$ and ${\widehat{I}}_g$. ${\sigma }_{I\widehat{I}}$ is the estimated covariance of image $I_g$ and ${\widehat{I}}_g$, respectively.

$$ERGAS=\sqrt{\frac{1}{b}\sum _{i=1}^b\frac{MSE({I_g}^i,{{\widehat{I}}_g}^i)}{E\left({I_g}^i\right)}}$$

(39)

where ${I_g}^i$ and ${{\widehat{I}}_g}^i$ represent the $i-th$ band of the original image and denoised image.

$$MSAD=\frac{1}{mn}\sum _{i=1}^{mn}\frac{180}{\pi }\times arccos\frac{{\left({\chi }^i\right)}^T·\left({\widehat{\chi }}^i\right)}{∥{\chi }^i∥·∥{\widehat{\chi }}^i∥}$$

(40)

where ${\chi }^i$ and ${\widehat{\chi }}^i$ are the $i-th$ spectral signatures of the original image and denoised image, respectively.

For the simulated process, additive Gaussian noise with different variance was added to the testing images to generate the noisy observations, whose values were 0.02, 0.04, 0.06, 0.08, and 0.1, respectively.

Figure 3 shows the denoised results of the Pavia city center image, which is contaminated by Gaussian noise with a noise variance value of 0.06. Figure 4 presents the magnified results from Figure 3. It is evident that our proposed method outperformed the others and efficiently reduced the Gaussian noise. ANLM3D obviously suffered from the problem of over-smoothing, which led to the loss of significant details in the image. In fact, one can barely recognize the basic outline of the denoised image. The result of the LRTA method still retained the most noise. The other five algorithms basically completed the denoising task and preserved enough edge details. However, we can further compare their denoising performance from the magnified results in Figure 4. From the enlarged local area, there were still unremoved noise points in the homogeneous area in the results of TDL and ITSReg. Although BM4D and SNLRSF showed good denoising performance in these regions, please note that these two methods both removed some obvious and important bright spots, while EPLRR-RSID well preserved these features and had the closest result to the original image.

Table 2. Numerical evaluation for the simulated Pavia city center image.

Noise Variance	Evaluation Index	LRTA	ANLM3D	BM4D	TDL	ITSReg	SNLRSF	RCTV	EPLRR-RSID
0.02	MPSNR	42.0515	40.9386	43.2494	45.5021	43.4947	46.3958	37.3477	48.7693
	MSSIM	0.9875	0.9808	0.9917	0.9708	0.9950	0.9924	0.9930	0.9943
	ERGAS	29.8352	33.8582	25.3636	21.0426	24.8458	18.4313	52.4070	18.0475
	MSAD	0.0659	0.0532	0.0533	0.0388	0.0385	0.0358	0.0719	0.0336
0.04	MPSNR	38.6549	33.9915	38.9835	41.4873	39.9658	42.6664	36.3351	45.8265
	MSSIM	0.9753	0.9189	0.9786	0.9878	0.9833	0.9908	0.9643	0.9927
	ERGAS	43.4777	73.9060	41.3009	31.6011	37.1903	29.0053	57.0773	27.3846
	MSAD	0.0822	0.0729	0.0784	0.0528	0.0495	0.0512	0.0773	0.0428
0.06	MPSNR	36.2204	30.8091	36.5304	38.8493	37.8379	40.2668	34.6877	43.1284
	MSSIM	0.9588	0.8553	0.9630	0.9788	0.9738	0.9847	0.9490	0.9851
	ERGAS	57.5013	105.6283	54.6983	42.5532	47.4484	38.5555	69.2007	35.1563
	MSAD	0.1003	0.0845	0.0956	0.0626	0.0575	0.0638	0.0875	0.0532
0.08	MPSNR	34.3290	29.0707	34.8010	37.0342	36.4610	38.4732	33.9440	40.2722
	MSSIM	0.9390	0.8028	0.9459	0.9692	0.9645	0.9777	0.9392	0.9806
	ERGAS	71.1996	128.5255	66.6934	52.3320	55.4655	47.3746	75.4939	43.2758
	MSAD	0.1178	0.0911	0.1095	0.0719	0.0637	0.0746	0.0929	0.0639
0.1	MPSNR	33.0995	28.0036	33.4699	35.5897	35.5582	36.6549	33.1095	37.9364
	MSSIM	0.9225	0.7624	0.9278	0.9590	0.9572	0.9631	0.9290	0.9718
	ERGAS	81.6941	145.0464	77.6982	61.8014	61.5646	58.2863	81.9037	56.7236
	MSAD	0.1243	0.0958	0.1218	0.0787	0.0687	0.0795	0.0982	0.0675

The best values are highlighted in bold.

presents the four index values of the different compared methods for the Pavia city center dataset with different variances of Gaussian noise. It is easy to observe that our method EPLRR-RSID was more competitive in almost all cases. One can see that SNLRSF, RCTV, and EPLRR-RSID all achieved better results. Among these three methods, RCTV improved the denoising effect via representative coefficient total variation, while SNLRSF attached importance to spatial nonlocal self-similarity. Our method EPLRR-RSID took both the TV term and nonlocal self-similarity into account and further incorporated the extracted edge part, thereby leading to the superiorities of our method on all indexes.

4.3. Experiments on Real HSIs

In our experiments, we adopted two real HSI datasets to further demonstrate the effectiveness of the EPLRR-RSID method. The details of these two datasets are as follows:

1

AVIRIS Indian Pines Dataset (Available: http://www.ehu.es/ccwintco/index.php/Hyperspectral$_$Remote$_$Sensing$_$Scenes (accessed on 15 November 2022): This dataset was acquired by the Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) sensor over the Indian Pines test site in Northwestern Indiana in 1992. The size of the data is 145 × 145 × 220, and the false-color image is presented in Figure 5a. Several bands contained a mixture of water absorption, impulse noise, and Gaussian noise.

2

HYDICE Urban Dataset (Available: http://www.erdc.usace.army.mil/Media/Fact-Sheets/Fact-Sheet-Article-View/Article/610433/hypercube/ (accessed on 15 November 2022): This dataset was acquired by the HYDICE sensor, whose size is 307 × 307 × 210. This dataset includes roads, roofs, grass, and trees. A subimage of size 200 × 200 × 210 was chosen in our experiments, and the false-color image is presented in Figure 5b.

Figure 6 provides the denoising results of different methods on the Indian Pines dataset. It is clear that EPLRR-RSID enhanced the image quality to the utmost extent and preserved more fine details than the other compared algorithms. Among all these methods, we can see that the results of ANLM3D, BM4D, TDL, and ITSReg contained evident blurry areas, which led them to lose many important details. On the other hand, SNLRSF, RCTV, and our proposed EPLRR-RSID method presented denoising images with clear edge details. However, the result of RCTV was not satisfying in homogeneous regions, while SNLRSF and EPLRR-RSID both achieved a balance between noise suppression and edge preservation. This is because SNLRSF and EPLRR-RSID both make the best use of the low-rank characteristic and nonlocal self-similarity of HSIs, further improving the denoising performance of SNLRSF and EPLRR-RSID. In addition, EPLRR-RSID adds the edge part extracted from the low-rank residues, which will greatly enhance edge features in denoised results, giving a distinct advantage to our method.

Figure 7 shows the denoising results of different methods on the HYDICE Urban dataset. As displayed in Figure 7b, LRTA still could not effectively remove noise. Although BM4D can effectively remove some noise and stripes, the problem of over-smoothing still exists, as we feared, in Figure 7d. Remarkably, the denoised image of our method had the best visual effect, showing significant details to the fullest extent. We also present the vertical mean profiles of Band 139 of the HYDICE Urban dataset in Figure 8 to show the different performance of various methods.

As revealed in Figure 8, the curves of all methods have the effect of smoothing compared with Figure 8a. Furthermore, it is worth noting that EPLRR-RSID achieved the smoothest vertical mean profile. Such a phenomenon representing the noise in the original images was removed. Because the curves of the vertical mean profiles are typically believed to be smooth, whereas the complex noise in the original image will inevitably cause the rapid fluctuations in the curves of noisy image, the reduction of rapid fluctuations means that noise in the original image was effectively suppressed after denoising. Therefore, the smoothest curve obtained by EPLRR-RSID revealed that our method had the ability to achieve the best denoising performance in the HYDICE Urban experiment.

We have mentioned that our proposed method has the ability of edge preservation; therefore, the Edge Preservation Index (EPI) was selected to assess this ability of EPLRR-RSID. Using the EPI to assess the image quality, it is necessary to select a homogeneous region whose center is at $(i,j)$ in the observed image firstly. In this region, the calculation of the EPI can be defined as

$$EPI=\frac{{\displaystyle \sum _i^M}{\displaystyle \sum _j^N}\left|{\widehat{I}}_g(i,j)-8\times E\left({\widehat{I}}_g\right)\right|}{{\displaystyle \sum _i^M}{\displaystyle \sum _j^N}\left|I_g(i,j)-8\times E\left(I_g\right)\right|}$$

(41)

where $I_g$, ${\widehat{I}}_g$, $E\left(I_g\right)$, and $E\left({\widehat{I}}_g\right)$ have the same meaning as (37) and (38).

The EPI values of each band for two real HSI datasets are shown in Figure 9. It is obvious that our proposed EPLRR-RSID can achieve the highest EPI values in the vast majority of cases, manifesting the advantage of our method in edge preservation. Meanwhile, we can see that SNLRSF, which employs the nonlocal low-rank factorization, could not obtain satisfactory EPI results. This phenomenon showed that conventional low-rank-based methods have poor performances on edge preservation, while EPLRR-RSID further extracted the edge details to offset this defect.

We calculated the mean EPI values to further evaluate the performance of each algorithm on various HSI datasets. As shown in Figure 10, we can see that EPLRR-RSID had obvious advantages on both datasets. The reason is that our method further extracts edge information from low-rank residues, which made EPLRR-RSID contain more edge details than the other algorithms. Through adding the edge part $R_1$, the ability of edge preservation will be greatly strengthened, although some noise will be introduced inevitably.

For real HSIs, there are no reference images for image quality assessment. In this paper, we adopted a no-reference image quality assessment named BRISQUE [52] to evaluate the two denoised real HSI datasets. BRISQUE employs scene statistics of locally normalized luminance coefficients to evaluate possible losses of naturalness in the image caused by the presence of distortions. It has been proven that this index is comparable with the PSNR and SSIM or even better. The BRISQUE results of different comparison algorithm are shown in

Table 3. BRISQUE value of different denoising methods on the two real HSI datasets.

	LRTA	ANLM3D	BM4D	TDL	ITSReg	SNLRSF	RCTV	EPLRR-RSID
Indian Pines	114.97	115.82	115.94	114.84	116.16	95.47	116.23	91.60
HYDICE Urban	96.11	99.26	101.26	96.16	100.56	97.56	101.14	86.44

The best values are highlighted in bold.

; a lower value represents higher image quality. Thereby, it is obvious that our method had some advantages over all the compared methods on the whole.

4.4. Parameter Sensitivity

In this section, we discuss the selection of four hyperparameters in our methods, along with a test of their impacts on denoising performance. The sensitivity analysis was undertaken on the simulated Pavia city center dataset, of which the noise variance was 0.1. The results are shown in Figure 11 and Figure 12.

As shown in Figure 11a, the MPSNR value of EPLRR-RSID was higher when ${\lambda }_1$ and ${\lambda }_2$ changed in the range of 0.01–10 and 1–10. When fixing ${\lambda }_1$ and ${\lambda }_2$, we can see that the results maintained stability as the ${\lambda }_3$ was within 0.001–1 and ${\lambda }_4$ was within 0.001–0.1. Furthermore, the MSSIM value was comparatively stable, no matter whether the value of ${\lambda }_2$ was high or low, but the results of the MSSIM were clearly better when ${\lambda }_1$ changed in the range of 0.01–1. When fixing ${\lambda }_1$ and ${\lambda }_2$, the changes of ${\lambda }_3$ and ${\lambda }_4$ had little effect on the denoising results. In general, the change of the four parameters had relatively little impact on the MSSIM. This is because the value of the MSSIM itself fluctuates within a quite small range, and the change of parameters will not influence edge extraction greatly. Therefore, there are no significant trend changes in Figure 12.

4.5. Complexity Analysis

Due to the NLM framework we used in EPLRR-RSID, we only discuss the computational complexity on each group of data. We mainly took multiplication operations, nuclear norm minimization, the inverse of a matrix operation, and eigendecomposition into consideration. For one patch of the image, the multiplication operation, nuclear norm minimization, and the inverse of a matrix operation all cost $O\left(n^3\right)$, while the eigendecomposition cost $O\left(d^{2}r\right)$. Therefore, the computational complexity of EPLRR-RSID in each iteration can be written as $O((r+1)n^3+d^{2}r)$.

5. Conclusions

In this paper, a novel denoising algorithm for remote sensing images named EPLRR-RSID was proposed. We attempted to take another perspective of low-rank residuals and preserve those edge details. Therefore, our proposed EPLRR-RSID method extracts the edge information from the residuals by finding robust image projections under a manifold learning framework. As our greatest advantage, our method had excellent performance in edge preservation. We also integrated a multi-level prior knowledge regulation, which helped distinguish pixels of different categories. Our multi-level modeling was proven to be efficient since more structural information can be learned. In this way, our method had more advantages than the existing methods that focus on edge preservation, because the edge information we extracted from the residues contained considerable details. The experimental results showed that EPLRR-RSID is highly effective on synthetic data and real HSI datasets. Furthermore, it was proven that EPLRR-RSID is better than existing denoising methods in edge preservation, indicating that it has the ability to better preserve edge information. More details can be observed and found in the denoised remote sensing images, which help better complete downstream tasks, such as target detection, classification, and clustering. This new method can be used for remote sensing image pre-processing to enhance the performance of subsequent processes. However, there is still a drawback in our method, which is that the running time of EPLRR-RSID depends on the size of the image. When processing large real images, our method was not fast enough. Future works include extending this method into the deep learning field to improve the denoising performance and computational efficiency.