A SAR Image-Despeckling Method Based on HOSVD Using Tensor Patches

Abstract

Coherent imaging systems, such as synthetic aperture radar (SAR), often suffer from granular speckle noise due to inherent defects, which can make interpretation challenging. Although numerous despeckling methods have been proposed in the past three decades, SAR image despeckling remains a challenging task. With the extensive use of non-local self-similarity, despeckling methods under the non-local framework have become increasingly mature. However, effectively utilizing patch similarities remains a key problem in SAR image despeckling. This paper proposes a three-dimensional (3D) SAR image despeckling method based on searching for similar patches and applying the high-order singular value decomposition (HOSVD) theory to better utilize the high-dimensional information of similar patches. Specifically, the proposed method extends two-dimensional (2D) to 3D for SAR image despeckling using tensor patches. A new, non-local similar patch-searching measure criterion is used to classify the patches, and similar patches are stacked into 3D tensors. Lastly, the iterative adaptive weighted tensor cyclic approximation is used for SAR image despeckling based on the HOSVD method. Experimental results demonstrate that the proposed method not only effectively reduces speckle noise but also preserves fine details.

1. Introduction

Synthetic aperture radar (SAR) is an earth observation system that can generate high-resolution remote sensing images, allowing for daylong and all-weather observations. Hence, the SAR system offers unique benefits in various fields, including disaster monitoring, resource exploration, and military applications [1]. However, SAR images are often degraded by speckle noise, which is a natural consequence of coherent scattering phenomena. Unfortunately, speckle noise cannot be eliminated from SAR images and can significantly hinder interpretation, such as target recognition and image segmentation [2]. In contrast to additive white Gaussian noise (AWGN) typically found in optical images, the speckle noise in SAR images is usually multiplicative.

Speckle noise significantly impacts the understanding and interpretation of SAR images. Therefore, despeckling SAR images can considerably improve the subsequent application performance [3,4,5,6]. Owing to the limitations of early SAR image technology, traditional spatial domain despeckling approaches primarily relied on spatial filtering algorithms, such as the Lee filter [7], the Frost filter [8], and the Kuan filter [9], to remove the speckles. These methods tend to over-smooth the image while removing speckle noise, resulting in the loss of textural details. Subsequently, transform domain methods have been proposed for despeckling SAR images. Currently, the most commonly used transform domain despeckling methods include wavelet transform [10,11], contourlet transform [12], and shearlet transform [13]. Despite being more effective than spatial domain despeckling methods, transform domain filtering demands substantial computational resources, can cause image blurring, and introduce artifacts, which adversely affect subsequent image processing. Regularization-based despeckling is another popular approach to SAR images [14,15], which transforms the despeckling problem into a minimization energy function. Regularization models based on the total variance model can effectively suppress speckle noise and maintain texture. In fact, they may create an unpleasant staircase effect. In recent years, sparse representation has emerged as a captivating field of research [16]. Sparse representation aims to represent the original image with as few atoms as possible in an overcomplete dictionary, leveraging natural prior knowledge as a clean image can be represented sparsely while noise cannot. Furthermore, some methods increase adaptability by iteratively and alternately updating the atoms and sparse solutions in the dictionary using a learning strategy [17]. Unfortunately, these methods have limitations in terms of noise separation and computational efficiency. Deep learning has shown significant potential in various research areas and has also made outstanding achievements in the despeckling of SAR images. For instance, the SAR-convolution neural network (CNN) combines homomorphic transformation SAR images with deep CNN [18,19,20]. The development of deep learning has significantly improved the performance and speed of SAR image despeckling by leveraging the powerful feature extraction and adaptive parameter adjustment capabilities of deep learning [21,22]. To achieve optimal despeckling performance, deep learning models heavily rely on training with a large volume of clean real SAR images. Nevertheless, acquiring such datasets can be challenging. As a result, models trained solely on simulated SAR images are often limited to specific scenes or scenarios.

The non-local means (NLM) denoising algorithm uses similar features, including non-adjacent pixels [23]. The fundamental concept is the self-similarity of natural images, which involves estimating clean pixels by searching for similar patches in the entire image or a large window. Therefore, the NLM has achieved promising denoising results and has been widely used in the field of SAR image despeckling. The probabilistic patch-based (PPB) filter [24] replaces the Euclidean distance with a statistical similarity criterion and utilizes an iterative method to update weights. Several algorithms have combined NLM with multiscale geometric transformations and achieved good despeckling results. For instance, SAR-block-matching 3D (SAR-BM3D) filter [25] combined 3D block matching and collaborative filtering. Subsequently, Cozzolino et al. proposed a fast adaptive nonlocal SAR (FANS) filter [26] and enhanced the despeckling efficiency of SAR-BM3D. NLM has also been employed in total variation regularization [27], low-rank approximation [28], and sparse representation [29].

The NLM algorithm is effective in suppressing speckle noise. However, it can also generate artificial texture due to the influence of the blocking criterion and patch-matching accuracy, which can impede image interpretation. Various strategies, such as multiscale [30] and the gray theory [31], have been employed to improve the selection of patches with similar properties. Liang et al. [32] introduced gradient information into the similarity measure by constructing the gradient orientation map and accelerated despeckling using the fast Fourier transform algorithm. Giampaolo et al. [33] proposed an independent model-free NLM despeckling framework that provided a generic solution for despeckling a variety of SAR products. However, most of the above-mentioned despeckling methods rearrange SAR images into vectors or matrices and rely on vector or matrix calculation methods for despeckling. Unfortunately, the vectorization operation destroys the topology between similar patches, resulting in suboptimal despeckling performance. Tensors [34] offer a natural representation for multilinear data, serving as a high-order generalization of vectors and matrices. Liu et al. [35] proposed a method for estimating missing values in tensor visual data by generalizing low-rank matrices to low-rank tensors. Tensor completion techniques have demonstrated significant potential for applications such as image inpainting, video compression, and bidirectional reflectance distribution function (BRDF) data estimation. Zhang et al. [36] introduced the tensor singular value decomposition (t-SVD) for denoising and completion of multilinear data, which can handle a broader range of multilinear data as long as they are compressible in the t-SVD-based representation. Xue et al. [37] developed a method for reducing noise in hyperspectral images (HSI) using CANDECOMP/PARAFAC (CP) decomposition modeling with tensor and rank automatic determination. This algorithm significantly improves the denoising performance of HSI in various quality assessments. This paper proposes a three-dimensional (3D) SAR image-despeckling method based on patch matching and the tensor patch higher-order singular value decomposition (HOSVD) theory to effectively capture the potential information shared among similar patches and better exploit the correlation between different dimensions of SAR images. By using the non-local framework, the classified two-dimensional (2D) similar patches are stacked to construct third-order non-local tensor patches, which can better utilize the similarity of patches. It should be noted that this is the first time higher-order tensor decomposition has been applied to the field of SAR image despeckling. The proposed method effectively achieves high-dimensional SAR image despeckling by utilizing the low-rank tensor approximation technique to exploit the potential correlation and low-rank structure of SAR image data. The effectiveness of the proposed method is evaluated and compared with the existing advanced despeckling algorithms.

The remainder of this paper is structured as follows. Section 2 presents a detailed description of the material and methodology used for despeckling. Section 3 provides an analysis of the experimental results. In Section 4, a comprehensive discussion is presented to provide deeper insights. Finally, Section 5 outlines the conclusions drawn from the method. The framework of the proposed method is illustrated in Figure 1.

2. Materials and Methods

This section presents the proposed method for SAR image despeckling. Prior to the proposed method, the multiplicative noise is converted to an additive model using a logarithmic transformation. Then, the entire image is divided into overlapping patches and similar patches within a local search window are selected for each reference patch. Firstly, the gradient of the image is calculated and used to identify similar patches. Next, these similar patches are combined into third-order tensor patches. Subsequently, the iterative low-rank tensor patch approximation is applied to recover the clean tensor patches. Finally, the ultimate results are obtained by exponential transformation and aggregation of all patch estimates.

2.1. Statistics of Log-Transformed Speckle

Considering an intensity SAR image, the intensity Y is related to the backscatter return X and speckle noise B by the following multiplicative model:

$$Y=BX$$

(1)

Assuming that the speckle noise is fully developed, Goodman’s model indicates that it follows a gamma distribution with a probability density function (PDF), expressed as follows:

$$p\left(B\right)=\frac{L^L{B}^{L-1}}{\mathrm{\Gamma }\left(L\right)}exp\left(-LB\right),B\ge 0$$

(2)

where $\mathrm{\Gamma }(·)$ denotes the gamma function. Logarithmic transformation is often used to convert multiplicative noise into additive ones. Using the logarithmic operator applied on Equation (1):

$$\tilde{Y}=ln\left(Y\right)=ln\left(B\right)+ln\left(X\right)=\tilde{B}+\tilde{X}$$

(3)

The random variable $\tilde{B}$ follows the Fisher–Tippett distribution, defined as follows:

$$p\left(\tilde{B}\right)=\frac{L^L{e}^{\tilde{B}L}}{\mathrm{\Gamma }\left(L\right)}exp(-L{e}^{\tilde{B}})$$

(4)

The mean and variance of $\tilde{B}$ can be computed as follows:

$$E\left[\tilde{B}\right]=\mathrm{\Psi }\left(L\right)-ln\left(L\right),Var\left[\tilde{B}\right]=\mathrm{\Psi }(1,L)$$

(5)

where $\mathrm{\Psi }(·)$ is the digamma function, and $\mathrm{\Psi }(·,L)$ is the polygamma function of order L. Formula (5) shows that the noise has a non-zero mean. Therefore, a de-biasing step is required after inverse logarithmic operations.

2.2. Searching Similar Patches of SAR Images

2.2.1. Measure for Non-Local Similarity

In the presence of noise, a similar patch-searching algorithm may select irrelevant candidates, resulting in undesired artifacts due to averaging the selected patches with irrelevant data values. Conversely, it can be difficult to find enough similar patches in regions with edges or unique structures, leading to the incorrect despeckling of pixels in such areas. To address these weaknesses in the similar patch-searching algorithm, the similarity criterion proposed in [38] is utilized in this study. The similarity criterion uses the gradient information to calculate the similarity between patch x and patch y in the SAR images, as follows:

$$S(x,y)={\parallel x-y\parallel }_{Ga}^2+\rho \left(\overline{E_x}\right){∥E_x-E_y∥}_{Ga}^2$$

(6)

where ${\parallel •\parallel }_{G_a}$ represents the Gaussian norm [39] and $\rho$ is the continuous increasing bounded function. The basic idea behind the similarity criterion is to consider the gradient information, especially in regions near singular points. The gradient vector $|\nabla x(i,j\left)\right|$ of the patch x in the position $(i,j)$ is calculated and expressed as ${\left(E_x\right)}_{ij}=|\nabla x(i,j)|$. The matrix with entries is denoted by $E_x$, while $\overline{E_x}$ represents the average value of the elements of matrix $E_x$.

2.2.2. Computation of Gradient

This paper utilizes the constrained least squares method [40] to estimate gradient information from SAR images. Specifically, given a pixel point $\theta \in {\mathbb{R}}^2$ in a patch, the vector of data values is denoted by f$:={\left[f\left({\theta }_v\right):v=1,\dots ,V\right]}^T$. Let $\left\{Z_u:u=1,\dots ,U\right\},U<V$ be a basis for ${\Pi }_U$, which represents the space of algebraic polynomials of degree $<U$, and put

$$\mathrm{\Phi }:=\left[z_j\left({\theta }_i\right):i=1,\dots ,V,j=1,\dots ,U\right],Z{}^{\left(\phi \right)}\left(\theta \right):={\left[z_j^{\left(\phi \right)}\left(\theta \right):j=1,\dots ,U\right]}^T.$$

   The approximation of the derivative $f^{\left(\phi \right)}$ can be expressed as follows:

$$f^{\left(\phi \right)}\left(\theta \right)\approx {\mathrm{K}}^{\left(\phi \right)}{\left(\theta \right)}^T\mathrm{f}$$

(7)

where K${}^{\left(\phi \right)}\left(\theta \right):={\left[K_v^{\left(\phi \right)}\left(\theta \right):v=1,\dots ,V\right]}^T$ is the appropriate coefficient vector that can be obtained by solving the following convex optimization problem:

$$\underset{{\mathrm{K}}^{\left(\phi \right)}\left(\theta \right)}{min}\sum _{v=1}^V{K}_v^{\left(\phi \right)}{\left(\theta \right)}^2\delta \left(\theta -{\theta }_v\right)s.t.{\mathrm{\Psi }}^T·{\mathrm{K}}^{\left(\phi \right)}\left(\theta \right)={\mathrm{Z}}^{\left(\phi \right)}\left(\theta \right)$$

(8)

The penalty function $\delta$ is typically chosen to be a smooth function that increases rapidly. A typical example is as follows:

$$\delta \left(\theta \right)=exp\left({\parallel \theta \parallel }^2/2\right)$$

(9)

The matrix form ${\mathrm{K}}^{\left(\phi \right)}\left(\theta \right)={\mathrm{M}}^{-1}\mathrm{\Phi }{\left({\mathrm{\Phi }}^T{\mathrm{M}}^{-1}\mathrm{\Phi }\right)}^{-1}{\mathrm{z}}^{\left(\phi \right)}\left(\theta \right)$ can be used to represent the solution to the optimization problem (8), where matrix M is defined as follows:   

                        $\mathrm{M}:=2diag\left(\delta \left(\theta -{\theta }_v\right):v=1,\dots ,V\right)$.

2.3. Tensor and Third-Order Tensor Decomposition

2.3.1. Definition of Tensor

To better exploit the self-similarity of similar patches, 2D patches are converted into third-order tensors for SAR image despeckling. The tensors are equivalent to multidimensional vector arrays, which are vectors and matrices extended to multiple dimensions. Specifically, scalars, vectors, and matrices can be viewed as zero-order, first-order, and second-order tensors, respectively. To further explain the third-order tensor decomposition, a few basic definitions of tensors are provided as follows. The mode-n vector of the third-order tensor is shown in Figure 2.

Definition 1.

Given two tensors $A\in R^{I_1\times I_2\times I_3\dots \times I_N}$ and $B\in R^{I_1\times I_2\times I_3\dots \times I_N}$ of the same dimension, their inner products can be calculated as follows:

$$\langle A,B\rangle =\sum _{i1}\sum _{i2}\sum _{i3}\cdots \sum _{iN}{a}_{i1i2i3\dots iN}·b_{i1i2i3\dots iN}$$

(10)

the Frobenius norm of tensor $A\in R^{I_1\times I_2\times I_3\dots \times I_N}$ can be obtained as follows:

$${\parallel A\parallel }_F=\sqrt{\langle A,A\rangle }=\sqrt{\sum _{i1,i2,\cdots ,iN}{\left|a_{i1i2\dots iN}\right|}^2}$$

(11)

Definition 2.

The elements of tensor A are mapped to the elements of the mode-n matrix $A_{\left(n\right)}\in R^{I_n\times \left(I_1\times \cdots \times I_{n-1}\times I_{n+1}\times \cdots \times I_N\right)}$, and a mode-n expansion of a tensor $A\in R^{I_1\times I_2\times I_3\dots \times I_N}$ of order n can be obtained. All elements of the tensor are expanded using modulo and then rearranged into a two-dimensional matrix.

Definition 3.

The mode-n rank of $A\in R^{I_1\times I_2\times I_3\dots \times I_N}$ for a given tensor A of order N is the rank of the tensor A mode-n expanded matrix $A_{\left(n\right)}$, it can signify ${rank}_n\left(A\right)$.

The corresponding mode-n expansion of the third-order tensor A is shown in Figure 3.

2.3.2. Higher-Order Singular Value Decomposition

The CP decomposition and Tucker decomposition are two commonly used methods for tensor decomposition. The CP decomposition expresses a given observation tensor as a combination of a sequence of rank-one quantities. The Tucker decomposition can be regarded as the extension of the matrix component analysis to higher dimensions. The HOSVD is a special form of the Tucker decomposition and can be obtained by generalizing the mode-n product of the matrix singular value decomposition (SVD) to the nth-order tensor $A\in R^{I_1\times I_2\times I_3\dots \times I_N}$ [41].   

$$A=\mathfrak{R}{\times }_1{U}^{\left(1\right)}{\times }_2{U}^{\left(2\right)}{\times }_3{U}^{\left(3\right)}\times \cdots {\times }_N{U}^{\left(N\right)}$$

(12)

where $U^{\left(n\right)}\in R^{I_n\times I_n},n=1,2,3,\cdots ,N$ is the orthogonal basis matrix and $\mathfrak{R}$ is the kernel tensor obtained by factorization. Unlike SVD for 2D matrices, the kernel tensor $\mathfrak{R}$ coefficient distribution for HOSVD does not have a diagonal structure and is not necessarily non-negative. $\mathfrak{R}$ can be expressed as follows:

$$\mathfrak{R}=A{\times }_1{U}^{{\left(1\right)}^T}{\times }_2{U}^{{\left(2\right)}^T}\times {}_3{U}^{{\left(3\right)}^T}\times \cdots {\times }_N{U}^{{\left(N\right)}^T}$$

(13)

The HOSVD of the third-order tensor $A\in R^{I_1\times I_2\times I_3}$ is expressed as follows:

$$A=\mathfrak{R}{\times }_1{U}^{\left(1\right)}{\times }_2{U}^{\left(2\right)}{\times }_3{U}^{\left(3\right)}$$

(14)

Figure 4 shows the schematic diagram of the HOSVD of the third-order observation tensor.

where $U^{\left(n\right)}\in R^{I_n\times I_n}(n=1,2,3)$ is the orthogonal basis matrix and $\mathfrak{R}\in R^{I_1\times I_2\times I_3}$ is the kernel tensor obtained by factorization. Since the third-order tensor A is obtained by stacking similar patches, the traditional HOSVD denoising algorithm assumes that the core tensor is sparse. The kernel tensor $\mathfrak{R}$ coefficient of the HOSVD decomposition is shrunk by a hard threshold as follows:

$$\widehat{\mathfrak{R}}=T_{\tau }\left(\mathfrak{R}\right)$$

(15)

where $T_{\tau }$ is hard-thresholding and is defined as follows:

$$T_{\tau }\left(\mathfrak{R}\right)=\left\{\begin{array}{cc}\mathfrak{R}(i,j,k),\hfill & \mathfrak{R}(i,j,k)\ge \tau \hfill \\ 0,\hfill & \mathfrak{R}(i,j,k)<\tau \hfill \end{array}\right.$$

(16)

The coefficient of the kernel tensor that is less than the threshold $\tau$ is set to zero by Equation (16), and then the estimated value $\widehat{A}$ of the kernel tensor intercepted by the hard threshold can be obtained by the inverse HOSVD transformation.

$$\widehat{A}=\widehat{\mathfrak{R}}{\times }_1{U}^{\left(1\right)}{\times }_2{U}^{\left(2\right)}{\times }_3{U}^{\left(3\right)}$$

(17)

Therefore, the HOSVD is used for image denoising, mainly by decomposing the high-order structure of the image tensor to extract useful information. Subsequently, the tensor is reconstructed to achieve the denoising effect. For denoising, appropriate thresholds can be selected to remove the noise from the image tensor, and the remaining parts can be recombined to form a denoised image. Let us explore the application of SVD in image denoising. The self-similarity of an image refers to the presence of a high degree of similarity between similar patches. If there is no noise, the similar patch matrix should have a low-rank property when converted to column vectors. Based on this observation, image denoising has been formulated as a low-rank matrix approximation problem. Let $Y\in R^{m\times n}$ be the observation matrix that comprises similar blocks of noisy observation images converted to column vectors, and $X\in R^{m\times n}$ be the corresponding noiseless matrix to be estimated, which has a low-rank property. Typically, the denoising model [28] based on the 2D low rank is approximated as follows:

$$\widehat{X}=arg\underset{X}{min}\frac{1}{2}{\parallel Y-X\parallel }_F^{2}s.t.rank\left(X\right)\le r$$

(18)

The above equation can be solved by performing SVD on the noisy observation matrix $Y\in R^{m\times n}$. In the SVD domain, Y can be decomposed into

$$Y=U\mathrm{\Lambda }{V}^T$$

(19)

the following is

$$Y_r=U{\mathrm{\Lambda }}_r{V}^T$$

(20)

including,

$${\mathrm{\Lambda }}_r=diag\left({\lambda }_1,{\lambda }_2,\cdots ,{\lambda }_r,0,\cdots ,0\right)$$

(21)

That is, the r singular values before the singular value decomposition of Y are retained, and $Y_r$ is the solution of Equation (18). However, the above rank constraint is an NP problem. Determining the shrinkage of rank r for singular values is the key. However, it is difficult to determine a reasonable threshold in despeckling based on SVD. Building on these observations, a new methodology for approximating low-rank tensor patches is proposed in this study. The proposed approach involves penalizing the adaptive weighted singular values of the core tensor, which is obtained via HOSVD, to achieve the low-rank tensor patch approximation.

2.4. SAR Image Despeckling Based on the Iterative Low-Rank Tensor Patch Approximation Algorithm

The basis of HOSVD is different from the fixed basis used in the SAR-BM3D wavelet transform and is obtained by decomposing the 3D observation tensor. The objective function for the low-rank tensor estimation of SAR image despeckling is expressed as follows:

                              $\underset{X}{argmin}\frac{1}{2}{\parallel Y-X\parallel }_F^2+\mathcal{W}{\parallel 𝘍\left(X\right)\parallel }_{⁎}$   

where Y and X denote the noisy and latent clean images, respectively; ${\parallel •\parallel }_{𝘍}$ represents the Frobenius norm, which can be calculated using Equation (11). $\mathcal{W}$ denotes adaptive weights; ${\parallel 𝘍\left(X\right)\parallel }_{⁎}$ denotes the proposed low-rank tensor patch nuclear norm, and $𝘍\left(X\right)$ represents HOSVD, which is used to obtain the low-tensor patches from X.

In the field of SAR image processing, natural scenes are often represented by SAR images that exhibit low-rank features. Generally, ${𝓁}_1$ norm regularization or nuclear norm regularization is frequently employed to approximate the rank function of the image. While these regularization methods can yield sparse solutions, the convexity in both approaches may introduce significant estimation bias [42]. In contrast, non-convex penalties, such as ${𝓁}_q(0\le q<1)$, smoothly clipped absolute deviation (SCAD), or minimax concave (MC) penalties can improve the deviation problem. Non-convex penalty regularization has demonstrated many advantages over convex penalty regularization in numerous applications. In recent years, there has been significant interest in non-convex regularization methods for sparse and low-rank recovery, driven by advancements in non-convex and non-smooth optimization algorithm theories. Building upon these developments, this paper proposes an iterative adaptive weight scheme for the regularization term, the adaptive weights with ${𝓁}_1$-norm can be seen as types of non-convex penalties applied to the core tensor [43]. The proposed scheme involves assigning varying penalty weights to the singular values of the tensor to achieve an optimal approximation of the low-rank tensor patches. Furthermore, soft threshold operators are used to solve the non-convex objective function.

For each tensor patch $A^k,k=1\cdots K$, the count of reference patches is represented by K, and the minimization problem is further expressed as follows:

$$\underset{X}{min}{w}^k{∥A^k∥}_{⁎}=\sum _i{w}_j^k\left|{\lambda }_j^k\right|$$

(22)

the adaptive weight is denoted by $w_j^k$, where ${\lambda }_j^k$ is the elemental value in the core tensor $\mathfrak{R}$. Due to the varying weights $w_j^k$, the adaptive weights with the ${𝓁}_1$-norm penalized optimization problem are non-convex. However, if the weights $w_j^k$ are assigned in a non-increasing manner to the increasing absolute values $\left|{\lambda }_j^k\right|$ of the core tensor ${\mathfrak{R}}^k$, the penalized optimization problem becomes a convex problem [35].

With the above analysis, the third-order tensor A is directly despeckled. The objective function can be expressed as follows:

$$arg\underset{A_x^t}{min}\frac{1}{2}{∥A_y^t-A_x^t∥}_F^2+\sum _j{w}_j^t\left|{\lambda }_j^t\right|,t=1,\dots ,T$$

(23)

the kth reference tensor patch of noisy image Y is denoted by $A_y^t$, where $A_x^t$ is the potential clean tensor patch corresponding to $A_y^t$, and ${\lambda }_j^t$ is the value of the core tensor $\mathfrak{R}$. The adaptive weight is denoted by $w_j^t$, which is assigned to $\left|{\lambda }_j^t\right|$. After obtaining the despeckling estimate $A_x^t$, the despeckled tensor patches are put back into their original positions to form a clean image, which requires an aggregation procedure. Up to this point, the despeckling of the nth image is completed and the despeckled SAR image ${\widehat{X}}_n$ is obtained.

2.5. Soft-Thresholding Proximal Operator

Compared to traditional convex penalty functions, the non-convex penalty functions may be more difficult to solve. Therefore, this paper uses the proximity operator to solve the non-convex penalty function. By solving the proximity problem for the objective function at each iteration, the proximity operator can provide an approximation to the optimal solution of the objective function. This approach effectively overcomes the difficulty of solving non-convex penalty functions and facilitates efficient optimization in high-dimensional spaces. Given a proper and lower semi-continuous penalty function $P_{\lambda }(·)$ and a threshold parameter $\lambda >0$, the scalar proximal projection can be defined as follows:

$${proxp}_{\lambda }\left(h\right)=\underset{x}{argmin}\left\{P_{\lambda }\left(x\right)+\frac{1}{2}{(x-h)}^2\right\}$$

(24)

the ${proxp}_{\lambda }\left(h\right)$ represents the proximity operator of a vector $h=\left[h_1,\dots ,h_n\right]\in R^n$, which can be expressed as follows:

$${proxp}_{\lambda }\left(h\right)={\left[{proxp}_{\lambda }\left(h_1\right),\dots ,{proxp}_{\lambda }\left(h_n\right)\right]}^{\mathrm{T}}$$

(25)

For commonly used proximity operators, the following soft-thresholding proximal operator is used in this paper:

$${proxp}_{\lambda }\left(\mathrm{h}\right)=sign\left(h\right)max\left\{\right|h|-\lambda ,0\}$$

(26)

Given a patch tensor A, $\widetilde{P_{\lambda }}(·)$ denotes a generalized penalty on the elemental values ${\lambda }_i$ of the core tensor $\mathfrak{R}$, which can be expressed as follows:

$$\widetilde{P_{\lambda }}\left(A\right)=\sum _i{P}_{\lambda }\left({\lambda }_i\right)$$

(27)

2.6. Residual Iteration and Adaptive Weight Setting to $w_j^t$

To further enhance the efficacy of SAR image despeckling, the residual image is utilized and iteratively processed. The residual image is effective in capturing the differences between noise and signal. The iterative process facilitates continuous optimization of despeckling outcomes, resulting in improved despeckling effects and image quality. The key element of the iterative algorithm is to add the residual $\left(Y-{\widehat{X}}_n\right)$ from the nth iteration to refine the $(n+1)$th step despeckled image $Y_{n+1}$.

$$Y_{n+1}={\widehat{X}}_n^t+\mu \left(Y-{\widehat{X}}_n^t\right)$$

(28)

where n stands for the number of iterations; and the relaxation parameter is denoted by $\mu$. To incorporate the residual information into the despeckled image, the remaining noise variance should be estimated, which is expressed as follows:

$${\sigma }_n=\eta \sqrt{{\sigma }^2-\frac{1}{q\times r}{∥Y-Y_{n-1}∥}_F^2}$$

(29)

where $\eta$ represents the scaling factor, which is used to control the re-estimation of noise variance. The noise variance of Y is indicated by $\sigma$, and the number of pixels in Y is denoted by $q\times r$.

The iterative adaptive weights $w_j^t$ related to each ${\lambda }_j$ are assigned as follows:

$$w_j^t=\frac{2\sqrt{2}\sqrt{N}{\sigma }_n^2}{\left|{\lambda }_j\right|+\varrho }$$

(30)

where N denotes the number of patches per tensor $A_y^t$; the small positive parameter is represented by $\varrho >0$, which is chosen to prevent division by zero, and ${\sigma }_n$ can be computed using Equation (29). The optimization problem (22) of the proximal soft-threshold operator (26) is solved using an iterative adaptive weight $w_j^t$. The adaptive thresholding retains the large values while filtering out the small values, thereby preserving important structural information in the image. Consequently, solution ${\tau }_j^t$ to the jth element of $A_x^t$ can be expressed as follows:

$${\tau }_j^k=sign\left({\lambda }_j\right)max\left\{\left|{\lambda }_j\right|-w_j^t,0\right\}$$

(31)

2.7. Aggregation of Despeckled Tensor Patches

When the patches are reassembled into the image, a weight is assigned to each tensor patch based on its level of noise. Specifically, the weight is defined as follows:

$$\varpi =\left\{\begin{array}{cc}\hfill \frac{r^2\times N}{r^2\times N+C},& \mathrm{if}C\ge 1\hfill \\ \hfill 0,& \mathrm{otherwise}\hfill \end{array}\right.$$

(32)

where r denotes the patch size; C represents the number of thresholded elements in the core tensor and N is the count of patches in the current tensor. The resulting image can be obtained as follows:

$$\widehat{X}(x,y)=\frac{{\sum }_{i\in N_{x,y}}{\sum }_{j\in j{\left(j\right)}_{x,y}}{\varpi }_{i,j}{\mathrm{\Omega }}_{i,j}}{{\sum }_{i\in N_{x,y}}{\sum }_{j\in j{\left(j\right)}_{x,y}}{\varpi }_{i,j}}$$

(33)

where $N_{x,y}$ represents all tensor patches overlapping in position (x, y); $J{\left(i\right)}_{x,y}$ represents all tensors in the ith tensor that overlap at position (x, y); and ${\Omega }_{i,j}$ represents a pixel located at position (x, y). The flow of the tensor despeckling algorithm is shown in Algorithm 1.

Algorithm 1 Iterative low-rank tensor patch approximation algorithm for SAR image despeckling

Input: SAR image Y, the ENL L, the number of reference patches K, and iteration F

Output: despeckled SAR image X

1: Initialization:

Initialize $X_0=Y$, $Y^0=Y$, SAR image patch tensor A

2: Iteration:

➀ Outer loop: for n = 1:F do

(I) Re-estimate $Y^n$ by (28)

(II) Re-estimate noise variance ${\sigma }_n$ by (28)

➁ Inner loop: for t = 1:T do

(I) Compute $U^{\left(1\right)},U^{\left(2\right)},U^{\left(3\right)}$ and core tensor ${\mathfrak{R}}_{\mathrm{y}}^{\left(\mathrm{t}\right)}$ of $A_{\mathrm{y}}^{\left(\mathrm{t}\right)}$ by HOSVD via Equation (14)

(II) For each ${\lambda }_j$ in core ${\mathfrak{R}}_{\mathrm{y}}^{\left(\mathrm{t}\right)}$ calculate the $w_j^t$ via Equation (22)

(III) Apply threshold $w_j^t$ to ${\lambda }_j$ in ${\mathfrak{R}}_{\mathrm{y}}^{\left(\mathrm{t}\right)}$ via Equation (27)

(IV) Estimate despeckling patches tensor by (23)

End for

➂ Obtain the nth step despeckled SAR image ${\widehat{X}}_n$ via Equation (33)

End for

➃ Obtain the despeckled SAR image X

3. Results

This section presents the despeckling experiments conducted on simulated multiplicative noise images and real SAR images. The experimental results verify the effectiveness of the proposed method. The proposed method is compared with the current state-of-the-art filters, including PPB [24], SAR-BM3D [25], FANS [26], and Mulog [44]. The executable codes for these methods can be downloaded from the authors’ websites, accessed on 29 May 2023. (https://www.charles-deledalle.fr/pages/ppb.php; http://www.grip.unina.it/download/prog/SAR-BM3D/; http://www.grip.unina.it/download/prog/FANS/; https://www.charles-deledalle.fr/pages/mulog.php) To better measure the despeckling performance of the proposed method, three objective evaluation metrics, i.e., peak signal-to-noise ratio (PSNR), feature similarity index (FSIM) [45], and structural similarity index (SSIM) [46], are used for the simulated multiplicative noise images. A higher PSNR indicates better image quality, while a higher FSIM suggests improved preservation of image structure information, which is represented as a number between 0 and 1. For real SAR images, the equivalent number of looks (ENL) and the mean of ratio image (MoR) are used to measure the superiority of the proposed method. Moreover, the ratio image is calculated to objectively evaluate the real SAR image-despeckling performance. To facilitate a well-informed selection of parameters within the proposed algorithm, this paper conducted an analysis of their influence using PSNR and SSIM metrics, as presented in

Table 1. PSNR and SSIM performance comparison with different patch sizes, patch numbers, patch stack numbers, and search windows. The best results are highlighted in bold.

Patch size	5 × 5	6 × 6	7 × 7	8 × 8	9 × 9
PSNR	23.11	23.14	23.26	23.24	23.19
SSIM	0.679	0.681	0.687	0.680	0.679
Patch number	100	150	200	250	300
PSNR	23.23	23.27	23.33	23.31	23.38
SSIM	0.683	0.687	0.691	0.693	0.695
Patch stack number	30	40	50	60	70
PSNR	23.07	23.25	23.40	23.42	23.37
SSIM	0.683	0.687	0.691	0.693	0.695
Search window	10 × 10	15 × 15	20 × 20	25 × 25	30×30
PSNR	23.21	23.34	23.41	23.43	23.45
SSIM	0.686	0.691	0.697	0.696	0.698

. The PSNR and SSIM values were computed by averaging across multiple images. In the proposed algorithm, the patch size was set to 7 × 7 and the search window size was set to 30 × 30. Additionally, the parameter patch number and patch stack number were set to 300 and 60, respectively.

3.1. Experiments on Simulated Multiplicative Noise Images

In this study, initial evaluations were conducted on three simulated multiplicative noise images, i.e., house, monarch, and Napoli. These three images are commonly used as test images in SAR image despeckling. To simulate realistic conditions, multiplicative speckle noise with different appearance levels (L = 1, 2, 4, 8) was added to each noise-free image. Reference images and corresponding noisy images (with L = 2) are presented in Figure 5.

Figure 6 shows the despeckling results of each filter on the house image, along with the relevant quantitative indexes reported in

Table 2. The PSNR, FSIM, and SSIM of different filters on the simulated multiplicative noise images (L = 1, 2, 4, 8). The best results are highlighted in bold.

		L = 1			L = 2			L = 4			L = 8
	Methods	PSNR	FSIM	SSIM	PSNR	FSIM	SSIM	PSNR	FSIM	SSIM	PSNR	FSIM	SSIM
House	Noisy image	12.16	0.427	0.096	14.71	0.504	0.152	17.32	0.584	0.225	20.04	0.663	0.316
	PPB	25.13	0.786	0.642	27.16	0.840	0.724	29.02	0.877	0.786	30.39	0.893	0.823
	FANS	25.34	0.804	0.757	28.66	0.854	0.811	31.17	0.883	0.842	32.95	0.903	0.860
	SAR-BM3D	24.62	0.836	0.772	28.14	0.877	0.816	30.90	0.905	0.845	32.12	0.922	0.863
	Mulog	25.01	0.835	0.783	28.36	0.870	0.822	31.15	0.894	0.847	32.97	0.914	0.862
	Proposed	25.54	0.813	0.733	28.28	0.857	0.797	30.76	0.888	0.836	32.38	0.904	0.854
Monarch	Noisy image	13.47	0.536	0.258	16.04	0.614	0.349	18.75	0.691	0.444	21.52	0.762	0.546
	PPB	23.00	0.826	0.716	24.72	0.866	0.790	25.99	0.892	0.835	27.63	0.916	0.873
	FANS	24.21	0.856	0.805	26.57	0.895	0.864	28.52	0.919	0.900	30.23	0.938	0.924
	SAR-BM3D	23.61	0.853	0.800	26.13	0.890	0.856	28.13	0.915	0.893	29.84	0.933	0.919
	Mulog	23.80	0.866	0.813	26.24	0.901	0.867	28.43	0.924	0.904	30.30	0.942	0.929
	Proposed	23.51	0.843	0.750	26.04	0.890	0.825	28.28	0.921	0.890	30.30	0.943	0.920
Napoli	Noisy image	14.64	0.606	0.229	17.27	0.628	0.337	20.08	0.759	0.463	22.97	0.826	0.593
	PPB	21.74	0.713	0.561	23.23	0.783	0.659	24.94	0.845	0.741	26.40	0.885	0.800
	FANS	22.26	0.710	0.598	24.24	0.804	0.703	26.24	0.869	0.784	28.12	0.910	0.848
	SAR-BM3D	22.64	0.760	0.639	24.42	0.825	0.724	26.36	0.880	0.802	28.12	0.916	0.865
	Mulog	22.43	0.735	0.628	24.08	0.801	0.704	26.02	0.861	0.778	27.96	0.908	0.843
	Proposed	22.33	0.780	0.600	24.15	0.826	0.690	26.14	0.881	0.775	28.04	0.915	0.842

. Visually, compared to the clear images, the PPB filter smooths the image excessively, resulting in the loss of detail and structural information, such as the window details highlighted by the red box. FANS effectively suppresses the noise while preserving the structural information of the image, but it has poor performance in retaining details. Although both SAR-BM3D and Mulog can preserve the details and structural information of the image well and have good despeckling performance in homogeneous regions, some unwanted artifacts are introduced during the despeckling process. Overall, the proposed method effectively despeckles the SAR images while preserving more details and structural information. Regarding the quantitative evaluation, Table 2 shows that Mulog performs well in terms of SSIM and PSNR, while FANS performs well in terms of ENL. The proposed method achieves the highest PSNR when L = 1, and consistently achieves higher FSIM and PSNR than the PPB and SAR-BM3D filters for $L>$ 2.

Figure 7 shows the results of the Monarch image, while the relevant quantitative indexes are reported in Table 2. It can be observed that, although the PPB filter can suppress speckle well, it removes a significant amount of textural details. FANS preserves more details than PPB, but the solution appears slightly over-smoothed. The SAR-BM3D can preserve details and textures well, but it has poor despeckling performance in homogeneous regions, which is opposite to Mulog. Overall, the proposed method can effectively balance despeckling and detail preservation, albeit with some minor artifacts. Regarding the quantitative evaluation, Mulog shows the best results in terms of SSIM and FSIM, while the highest values of PSNR are achieved by FANS. The proposed method consistently outperforms PPB, and when L= 8, it achieves the best performance in both FSIM and PSNR.

Figure 8 shows the results of the Napoli image, with the corresponding quantitative indexes reported in Table 2. It can be seen from Figure 8 that PPB over-smooth the image, resulting in a significant loss of details and texture. FANS and Mulog preserve more details than the PPB, but the resulting solution is slightly over-smoothed. The SAR-BM3D can preserve details and textures well, but its despeckling performance in homogeneous regions is suboptimal. The despeckling ability of the proposed method needs to be strengthened in the homogeneous region, but the texture and details are well preserved, which is better conducive to image interpretation. In terms of quantitative assessment, SAR-BM3D shows the best results in terms of SSIM and PSNR, while the proposed method achieves the highest values of SSIM. It can be observed from Table 2 that the proposed method consistently outperforms the PPB. All of the above objective results show that our proposed method can suppress speckle noise in homogeneous regions and preserve image details well in edge regions.

3.2. Experiments on Real SAR Images

To assess the despeckling performance of the proposed method on real SAR images, three remote-sensing images from different sensors were selected. Figure 9a is a 2-look SAR image, denoted as R1, acquired by the British DRA SAR. Figure 10a is a 1-look SAR image, denoted as R2, acquired by the German TerraSAR-X satellite. The 8-look SAR image, denoted as R3, as shown in Figure 11a, was acquired by Sentinel-1. In the experiment, the ENL values were measured in the blue rectangular areas. The specific results are shown in

Table 3. The ENL, MoR, and times of different filters on real SAR images. The best results are highlighted in bold.

	R1				R2				R3
Methods	ENL1	ENL2	MoR	Time	ENL1	ENL2	MoR	Time	ENL1	ENL2	MoR	Time
Noisy image	2.90	2.62	-	-	2.20	2.62	-	-	22.47	12.78	-	-
PPB	55.58	366.65	1.00	23.08	1345.02	340.70	0.99	22.36	1327.92	1181.27	1.00	23.11
FANS	32.65	74.58	1.02	1.62	100.75	98.03	1.11	1.74	525.66	770.60	1.01	1.58
SAR-BM3D	19.63	22.27	0.99	24.31	127.05	77.99	0.99	24.54	3837.38	111.76	0.99	24.09
Mulog	29.74	69.61	1.07	10.25	216.92	144.60	1.38	10.02	878.76	859.67	1.01	11.36
Proposed	38.04	151.58	0.96	50.63	137.74	151.58	0.99	49.47	401.67	951.58	0.98	49.33

. A larger ENL generally indicates better despeckling ability. Furthermore, ratio images were computed to compare the residual speckle noise. As per the findings of [47], an ideal filter’s ratio image (the ratio between noisy and despeckled images) should solely preserve the speckle. Hence, based on the multiplicative noise model, the MoR of the three images should be 1. To facilitate a comprehensive assessment of the proposed despeckling method’s efficiency in real-world scenarios, a comparative analysis of computational times was performed for different filters. The results are presented in Table 3, which highlights the differences in processing times among the different filters.

Figure 9, Figure 10 and Figure 11 display the despeckling results of real-world SAR images. Compared to the other filters, PPB leads to excessive smoothing, resulting in the loss of a considerable amount of detail and texture information, as shown in Figure 9b and Figure 10b. The SAR-BM3D can preserve details and structural information well but it has poor noise removal performance in homogeneous areas, as shown in Figure 10d. Both FANS and Mulog can reduce noise while preserving details and structural information well; however, they may introduce some unnecessary artifacts, as shown in Figure 9c,e and Figure 10c,e. Since the proposed method is an edge-preserving filter, it focuses on preserving edges and does not over-smooth details. Therefore, it does not over-smooth the homogeneous regions, such as the PPB filter, resulting in lower ENL values. Table 3 also shows that the proposed method can better use the similarity of the image to denoise and retain the detailed information. Figure 12, Figure 13 and Figure 14 show the ratio image results. The despeckled SAR images show that both PPB and FANS tend to remove excessive details during despeckling, especially in SAR images with more structural information, as shown in Figure 13a,b. This excessive despeckling of structural and edge information adversely affects the subsequent SAR image processing. On the other hand, the SAR-BM3D effectively despeckles homogeneous regions while preserving edge regions and structural information, as seen in the ratio image results in Figure 12c and Figure 13c. However, the despeckled SAR images appear distorted, resulting in the blurring of many details. While Mulog preserves details and edges information well, it introduces unnecessary artifacts in homogeneous regions, as shown in Figure 12d and Figure 13d. In contrast, the proposed method achieves perfect homogeneous despeckling while maintaining details and structural information to the greatest extent. To summarize, the experimental comparative analysis of SAR image despeckling shows that the proposed method achieves advanced visual quality, structure preservation, and speckle reduction results. However, a comparison of the filtering time results in Table 3 reveals that the despeckling time for the PPB and SAR-BM3D methods is approximately 23–24 s, while the Mulog method takes 10–11 s, and the FANS method requires only 1–2 s. In contrast, the proposed method exhibits a longer despeckling time of around 49–50 s. This indicates that there is still considerable room for improvement in achieving real-time despeckling with the proposed method. The primary contributing factor is the computationally demanding task of gradient-based similar patch classification, which necessitates significant computational resources. Additionally, the 3D tensor-based despeckling process involves the stacking of numerous 2D similar patches and relies on HOSVD, both of which further contribute to the overall computation time. Nevertheless, in practical applications of SAR despeckling, if real-time processing is not a strict requirement, the preservation of texture and detailed information is of utmost importance, and the proposed method remains a good choice.

4. Discussion

In this paper, a 3D despeckling method is proposed that stacks similar patches into tensor patches and uses the HOSVD method for low-rank tensor approximation despeckling. The proposed approach extends the application of image self-similarity to 3D and better utilizes the spatial geometric structure information of similar patches. The effectiveness of the proposed method is comprehensively evaluated and compared with the state-of-the-art despeckling methods through despeckling experiments on both simulated and real SAR images.

The experimental results demonstrate that the FSIM metric consistently outperforms FANS and PPB, particularly in the case of SAR images with more complex structural information, such as the Napoli image. Despite not having the highest PSNR and SSIM scores, the proposed method consistently outperforms PPB in terms of despeckling value across all view numbers. Visually, the proposed method achieves excellent speckle reduction while preserving image details. Although the ENL value is not the highest, the MoR values demonstrate that the proposed method effectively preserves edge information. Overall, the experimental results demonstrate and validate that the proposed method achieves an optimal balance between preserving an image’s structural and texture information and achieving effective despeckling.

However, it is noted that the proposed method may result in some artifacts while despeckling homogeneous areas. To minimize these artifacts, future work will focus on improving the accuracy of patch classification. Additionally, the process of classifying similar patches based on gradients poses significant computational demands, necessitating substantial computational resources. Moreover, the 3D tensor-based despeckling procedure involves the aggregation of multiple 2D similar patches and relies on HOSVD, both of which contribute to the overall computation time. This complexity is crucial for achieving improved despeckling performance and preserving essential image details. In future research, the authors intend to utilize approximate methods or efficient algorithms to minimize the computational complexity and reduce the despeckling time as much as possible.

5. Conclusions

This paper proposes an SAR image-despeckling method that utilizes high-dimensional information from similar patches by employing a search for similar patches and applying the tensor patch HOSVD theory. Specifically, the method extends 2D to 3D for SAR image despeckling using tensor patches for the first time. The proposed effectively achieves high-dimensional SAR image despeckling using a low-rank tensor approximation technique to exploit the potential correlation and low-rank structure of SAR image data. Previous image low-rank algorithms tended to destroy the topological structures of image patches by pulling similar patches into column vectors, which is detrimental to preserving image details. As a high-order generalization of vectors and matrices, tensors can stack 2D similar patches into 3D tensor patches, enabling HOSVD to directly despeckle the 3D tensor patches and leverage the high-dimensional structural information of similar patches. Extensive experiments demonstrate and validate the superior performance of the proposed method in synthetic and real SAR image despeckling compared with the existing advanced filters in the subjective visual evaluation and objective evaluation indices. However, the computational complexity of classifying similar patches and performing high-order singular value decomposition to approximate 3D tensor patches makes the proposed method time-consuming for despeckling. In the future, the focus will be on optimizing the mathematical algorithm and extending 3D despeckling to higher dimensions. This will leverage the self-similarity information of similar patches for SAR despeckling.