PolSAR Image Feature Extraction via Co-Regularized Graph Embedding

Abstract

Dimensionality reduction (DR) methods based on graph embedding are widely used for feature extraction. For these methods, the weighted graph plays a vital role in the process of DR because it can characterize the data’s structure information. Moreover, the similarity measurement is a crucial factor for constructing a weighted graph. Wishart distance of covariance matrices and Euclidean distance of polarimetric features are two important similarity measurements for polarimetric synthetic aperture radar (PolSAR) image classification. For obtaining a satisfactory PolSAR image classification performance, a co-regularized graph embedding (CRGE) method by combing the two distances is proposed for PolSAR image feature extraction in this paper. Firstly, two weighted graphs are constructed based on the two distances to represent the data’s local structure information. Specifically, the neighbouring samples are sought in a local patch to decrease computation cost and use spatial information. Next the DR model is constructed based on the two weighted graphs and co-regularization. The co-regularization aims to minimize the dissimilarity of low-dimensional features corresponding to two weighted graphs. We employ two types of co-regularization and the corresponding algorithms are proposed. Ultimately, the obtained low-dimensional features are used for PolSAR image classification. Experiments are implemented on three PolSAR datasets and results show that the co-regularized graph embedding can enhance the performance of PolSAR image classification.

1. Introduction

Regardless of the influence of the light and the weather, polarimetric synthetic aperture radar (PolSAR) has the ability to gain high-resolution images. Moreover, PolSAR uses different polarization combinations to obtain more abundant information of land covers than the single-polarization SAR [1,2,3]. Therefore PolSAR is a powerful tool for land cover classification. How to extract appropriate features affects the PolSAR image classification performance greatly. Thus, this paper focuses on feature extraction.

In the early years, Lee et al. [4] put forward the Wishart classifier to classify PolSAR images and the definition of the Wishart distance was given. By using Bayes maximum likelihood classifier specific to the complex Wishart distribution, the Wishart distance was defined. The Wishart classifier becomes one of most classical PolSAR image classification methods and is widely used. Afterwards, Lee et al. [5] cooperated H/A/$\alpha$ decomposition with Wishart classifier for unsupervised PolSAR image classification. Three important polarimetric parameters of H/A/$\alpha$ decomposition (i.e., entropy H, anisotropy A, and alpha angle $\alpha$) and the total backscattering power for H/A/$\alpha$ decomposition SPAN were used to initialize cluster centers for the Wishart classifier in [6]. Wu et al. [7] employed Markov random field (MRF) in the basis of the Wishart distribution and region for PolSAR image classification. The symmetric revised Wishart (SRW) distance is proposed to construct a weighted graph for spectral clustering in [8]. Ersahin et al. [9] used the SRW distance for spectral graph partitioning based on the contour information and spatial proximity. The SRW distance has also been introduced into a superpixel segmentation method, i.e., simple linear iterative clustering (SLIC) for segmenting PolSAR images into superpixels [10]. Recently, the Wishart distance was used in the deep learning architecture, i.e., deep stacking network (DSN), named Wishart DSN for PolSAR image classification [11].

For the majority of PolSAR image classification methods, polarimetric features are usually exploited as features [12,13,14,15,16,17]. Some original PolSAR data, i.e., the covariance matrix, scattering matrix, and coherency matrix can yield polarimetric features. Moreover, polarimetric features are also usually obtained from considerable polarimetric target decomposition methods, namely Yamaguchi decomposition, Pauli decomposition, Yang decomposition, Krogager decomposition, Huynen decomposition, Van Zyl decomposition, Freeman decomposition, and H/A/$\alpha$ decomposition [12,13,14,16]. Zhang et al. [14] used sparse representation based on polarimetric features to classify PolSAR images. The coherency matrix was converted into a 6D feature vector as feature to input deep convolutional neural networks (CNN) [15]. The complex-valued CNN [17] was put forward by using PolSAR data’s amplitude and phase information for PolSAR image classification. In some recent works, visual features, namely color features and texture features, were extracted for PolSAR image classification [18,19,20]. Uhlmann et al. [18] concluded three kinds of features, i.e., polarimetric features, color features and texture features of PolSAR images and evaluated the performance of different combinations of features for image classification. Ren et al. [20] proposed manifold regularized low-rank representation method based on polarimetric features and texture features for PolSAR image classification.

To acquire low-dimensional features for PolSAR image classification, some dimensionality reduction (DR) methods were applied. Tu et al. [12] used Laplacian eigenmaps (LE) to extract low-dimensional features for classification. Shi et al. [13] proposed supervised graph embedding (SGE) based on the discriminative information for supervised PolSAR image classification. Because of the spatial information’s significance, some tensor-based DR methods were introduced for PolSAR image feature extraction, such as tensor local discriminant embedding (TLDE) [21], tensorial independent component analysis (TICA) [22] and tensorial locally linear embedding [23]. Ren et al. [24] proposed the tensor embedding framework for PolSAR image feature extraction.

DR methods conclude linear and nonlinear methods. Traditional DR methods, i.e., linear discriminant analysis (LDA) and principal component analysis (PCA) are supervised and unsupervised linear methods, respectively [25]. The kernel method is used to extend the linear DR methods to nonlinear ones [26], i.e., kernel fisher discriminant analysis and kernel PCA. Some nonlinear DR methods aim to preserve the local or global geometric information of the nonlinear manifold structure, such as Laplacian eigenmap (LE) [27], Isomap [28] and local linear embedding (LLE) [29]. Yan et al. raised a framework to reformulate all the above DR methods by the graph embedding framework [30]. For the graph embedding DR framework, the weighted graph is constructed to represent the structure information of the data which we attempt to preserve in the process of DR. Different weighted graphs generate different methods. The similarity measurement plays a key role in constructing a weighted graph. The weighted graph of LE is constructed based on local pairwise Euclidean distances [27] and the weighted graph of Isomap is constructed based on global pairwise geodesic distances [28].

However, the above DR methods are mainly specific to single-view data. Even for multi-view data, they only directly concatenate vectors from multiple views into a vector and neglect the complementarity and correlation of different views. Kan et al. [31] proposed multiview discriminant analysis (MDA) to extend LDA to multiview cases. Multiview spectral embedding (MSE) was proposed to learn the complementarity of multiple views [32]. Kumar et al. [33] put forward a spectral clustering method based on co-regularization for multiview data. These methods are also based on graph embedding, which construct different graphs for the data from different views.

From above works, we can see that Wishart distance and polarimetric features are two indispensable factors for PolSAR image classification. For graph embedding DR methods, the similarity measurement is a crucial factor for constructing a weighted graph. Different distances can describe the similarity of samples more comprehensively from different perspectives. Therefore both Wishart distance of covariance matrices and Euclidean distance of polarimetric features are used in this paper. To extract more comprehensive information for a better classification performance, this paper attempts to put forward a feature extraction method by combining Wishart distance of covariance matrices and Euclidean distance of polarimetric features. Part of this work was introduced in [34]. Based on the two distances, we construct two weighted graphs to represent the local structure information. To use spatial information and decrease computation cost, we seek neighbouring samples in a local patch instead of globally. Then the co-regularized graph embedding (CRGE) model is formed on the basis of the two weighted graphs and co-regularization. The co-regularization is defined to measure the the dissimilarity of low-dimensional features corresponding to two graphs. The obtained low-dimensional features are used for ultimate image classification.

The remainder of this paper is organized as follows. Some background knowledge is described briefly in Section 2. The proposed co-regularized graph embedding method is presented elaborately in Section 3. Section 4 shows the experiments which are conducted on three PolSAR data sets and the analysis of results. Section 5 gives some discussions specific to the proposed method. At last, Section 6 concludes this paper.

2. Background

2.1. Graph Embedding DR Framework

Given n samples $\mathbf{E}={[{\mathbf{e}}_1,{\mathbf{e}}_2,\cdots ,{\mathbf{e}}_n]}^T\in {\mathbf{R}}^{n\times D}$, where n and D denote the number and the dimensionality of samples, respectively, the graph embedding DR framework which is given in [30] aims to find low-dimensional representations $\mathbf{F}={[{\mathbf{f}}_1,{\mathbf{f}}_2,\cdots ,{\mathbf{f}}_n]}^T\in {\mathbf{R}}^{n\times d}$, where d is the reduced dimensionality and $d<D$.

For graph embedding, we firstly need to construct an intrinsic weighted graph $\mathbf{G}$ to describe the similarity relationship among samples. The optimization problem is given:

$$\begin{array}{cc}\hfill {\mathbf{F}}^{\ast }=& arg\underset{\mathrm{tr}\left({\mathbf{F}}^T\mathbf{P}\mathbf{F}\right)=b}{min}\sum _{i\ne j}{∥{\mathbf{f}}_i-{\mathbf{f}}_j∥}^2{G}_{ij}\hfill \\ \hfill =& arg\underset{\mathrm{tr}\left({\mathbf{F}}^T\mathbf{P}\mathbf{F}\right)=b}{min}\mathrm{tr}\left({\mathbf{F}}^T\mathbf{L}\mathbf{F}\right)\hfill \end{array}$$

(1)

where $\mathbf{L}=\mathbf{D}-\mathbf{G}$, i.e, the Laplacian matrix. $\mathbf{D}$ is a diagonal matrix and its elements $D_{ii}={\sum }_j{G}_{ij}$. $\mathbf{P}$ can be the Laplacian matrix specific to a penalty graph ${\mathbf{G}}_p$ which characterizes the similarity properties which we try to constrain, or a diagonal matrix for scale normalization. b is usually a constant.

2.2. PolSAR Data

For PolSAR, an observed target can be characterized by the scattering matrix $\mathbf{S}$ with size $2\times 2$ [1]. That is,

$$\begin{array}{c}\hfill \mathbf{S}=\left(\begin{array}{cc}{S}_{HH}& S_{HV}\\ S_{VH}& S_{VV}\end{array}\right)\end{array}$$

(2)

where $S_{HH},S_{HV},S_{VH}$ and $S_{VV}$ represent the backscattering coefficients corresponding to four channels: HH, HV, VH and VV. Based on the reciprocity property, $S_{HV}$ is equal to $S_{VH}$.

The scattering matrix $\mathbf{S}$ is able to be vectorized as the scattering vector $\mathbf{k}={[S_{HH},\sqrt{2}{S}_{HV},S_{VV}]}^T$. Regrading mulitilook PolSAR data, the covariance matrix is gained by computing as follows:

$$\mathbf{C}=\frac{1}{n_L}{\mathbf{k}}_i{\mathbf{k}}_i^{\ast T}$$

(3)

where $n_L$ is the number of looks, the superscript * represents the complex conjugation.

2.3. Wishart Distance

The Wishart distance is frequently used as the similarity measurement for PolSAR image classification [1,4,5]. By using Bayes maximum likelihood classifier specific to the complex Wishart distribution, the Wishart distance was obtained [4]. The definition of the original Wishart distance is like:

$$d_W(\mathbf{C},\mathbf{\Sigma })=\mathbf{ln}|\mathbf{\Sigma }|+\mathbf{tr}\left({\mathbf{\Sigma }}^{-\mathbf{1}}\mathbf{C}\right)$$

(4)

where $\mathbf{C}$ is a sample covariance matrix, $\mathbf{\Sigma }$ is the cluster mean of the class. In this paper, we use a symmetric revised Wishart (SRW) distance to construct a graph. Because SRW distance satisfies some conditions for a general metric, such as definiteness, generalised nonnegativity, symmetry [8]. The SRW distance of two $p\times p$ covariance matrices $\mathbf{A}$ and $\mathbf{B}$ is denoted as:

$$d_{SRW}(\mathbf{A},\mathbf{B})=\mathrm{tr}\left({\mathbf{B}}^{-1}\mathbf{A}+\mathbf{B}{\mathbf{A}}^{-1}\right)-p.$$

(5)

3. The Proposed Method

This section will describe the proposed co-regularized graph embedding method elaborately. Figure 1 presents the whole process based on co-regularized graph embedding for PolSAR image classification. The proposed method is in basis of a superpixel rather than a single pixel to decrease computation cost and use spatial information. Therefore, we firstly segment PolSAR images into superpixels which will be described detailedly in Section 4. Based on Wishart distance of covariance matrices and Euclidean distance of polarimetric features, we construct two corresponding weighted graphs. Then the co-regularized graph embedding DR model is built based on the two graphs and co-regularized for feature extraction. The obtained low-dimensional features are used for PolSAR image classification.

3.1. Polarimetric Features

Polarimetric features can describe polarimetric scattering and physical properties of targets and usually can be extracted from PolSAR data and some polarimetric decomposition methods. In the proposed method, we extract thirty polarimetric features from the covariance matrix $\mathbf{C}$ and five decomposition methods (i.e., Pauli decomposition, H/A$/\alpha$ decomposition, Krogager decomposition, Huynen decomposition and Freeman decomposition). In detail,

• six are from the elements of covariance matrix $\mathbf{C}$, i.e., $C_{11},C_{22},C_{33},\mathrm{Re}{C}_{12},\mathrm{Im}{C}_{12},\mathrm{Re}{C}_{13},\mathrm{Im}{C}_{13},$ $\mathrm{Re}{C}_{23}$, $\mathrm{Im}{C}_{23}$;
• three are from Pauli decomposition, i.e., ${\left|a\right|}^2{,\left|b\right|}^2,{\left|c\right|}^2$ which denote powers of a isotropic odd, a even and a $\pi /4$ even scatter;
• three are from Krogager decomposition i.e., $|k_s{|}^2,|k_d{|}^2,{\left|k_h\right|}^2$ which denote powers of a sphere, a diplane and a helix scatter;
• three are from Freeman decomposition i.e., $P_s,P_d,P_v$ which denote powers of a surface, a double-bounce and a volume scatter;
• nine are from Huynen decomposition, i.e., $A_0,B_0+B,B_0-B,C,D,E,F,G,H$ denote information corresponding to symmetry, nonsymmetry and irregularity, linear, curvature, torsion, helicity, glue and orientation;
• six are from H/A$/\alpha$ decomposition, i.e., ${\lambda }_1,{\lambda }_2,{\lambda }_3,H,A,\alpha$ denote three eigenvalues, entropy, anisotropy and alpha angle.

Therefore, sample i is denoted as a 30D feature vector ${\mathbf{e}}_i\in {\mathrm{R}}^{30}$. Here, “sample” means a superpixel rather than a single pixel. The details will be introduced in Section 4.

The similarity of two samples is characterized by the Euclidean distance of polarimetric features. Concretely, for sample i and sample j, the Euclidean distance $d_E$ is

$$d_E({\mathbf{e}}_i,{\mathbf{e}}_j)={\parallel {\mathbf{e}}_i-{\mathbf{e}}_j\parallel }_2$$

(6)

3.2. Constructing Two Graphs

Based on Wishart distance of covariance matrices $d_{SRW}$ and Euclidean distance of polarimetric features $d_E$, we construct two corresponding graphs ${\mathbf{G}}^{\left(1\right)}$ and ${\mathbf{G}}^{\left(2\right)}$.

To use the spatial and reduce the computation burden, we search for k neighbouring samples in a $h\times h$ local region rather than the full image. Specifically, for sample i, we search for k nearest samples in the $h\times h$ region centered on sample i based on $d_{SRW}$ and $d_E$, which are denoted as $O({\mathbf{C}}_i,k)$ and $O({\mathbf{e}}_i,k)$.

Then we construct two graphs ${\mathbf{G}}^{\left(1\right)}$ and ${\mathbf{G}}^{\left(2\right)}$ as follows:

• if ${\mathbf{C}}_j\in O({\mathbf{C}}_i,k)$, $G_{ij}^{\left(1\right)}=e^{-\frac{d_{SRW}({\mathbf{C}}_i,{\mathbf{C}}_j)}{t_1}}$, else $G_{ij}^{\left(1\right)}=0$,
• if ${\mathbf{e}}_j\in O({\mathbf{e}}_i,k)$, $G_{ij}^{\left(2\right)}=e^{-\frac{d_E({\mathbf{e}}_i,{\mathbf{e}}_j)}{t_2}}$, else $G_{ij}^{\left(2\right)}=0$,

where $t_1$ and $t_2$ are two parameters. Here, we select $t_1=max\left(d_{SRW}({\mathbf{C}}_i,{\mathbf{C}}_j)\right)$ and $t_2=max\left(d_E({\mathbf{e}}_i,{\mathbf{e}}_j)\right)$.

3.3. Co-Regularized Graph Embedding Model

Based on two graphs ${\mathbf{G}}^{\left(1\right)}$ and ${\mathbf{G}}^{\left(2\right)}$, we pursue corresponding low-dimensional representations ${\mathbf{F}}^{\left(1\right)}\in R^{d\times n}$ and ${\mathbf{F}}^{\left(2\right)}\in R^{d\times n}$ which can be obtained via the following objective functions:

$$\begin{array}{c}\hfill \underset{{\mathbf{F}}^{\left(1\right)}}{min}\mathrm{tr}\left({{\mathbf{F}}^{\left(1\right)}}^T{\mathbf{L}}^{\left(1\right)}{\mathbf{F}}^{\left(1\right)}\right)\end{array}$$

(7)

$$\begin{array}{c}\hfill \underset{{\mathbf{F}}^{\left(2\right)}}{min}\mathrm{tr}\left({{\mathbf{F}}^{\left(2\right)}}^T{\mathbf{L}}^{\left(2\right)}{\mathbf{F}}^{\left(2\right)}\right)\end{array}$$

(8)

where ${\mathbf{L}}^{\left(1\right)}$ and ${\mathbf{L}}^{\left(2\right)}$ are normalized Laplacian matrices for ${\mathbf{G}}^{\left(1\right)}$ and ${\mathbf{G}}^{\left(2\right)}$. Concretely, ${\mathbf{L}}^{\left(1\right)}=\mathbf{I}-{{\mathbf{D}}^{\left(1\right)}}^{-\frac{1}{2}}{\mathbf{G}}^{\left(1\right)}{{\mathbf{D}}^{\left(1\right)}}^{-\frac{1}{2}}$ and ${\mathbf{L}}^{\left(2\right)}=\mathbf{I}-{{\mathbf{D}}^{\left(2\right)}}^{-\frac{1}{2}}{\mathbf{G}}^{\left(2\right)}{{\mathbf{D}}^{\left(2\right)}}^{-\frac{1}{2}}$. ${\mathbf{D}}^{\left(1\right)}$ and ${\mathbf{D}}^{\left(2\right)}$ are two diagonal matrices specific to ${\mathbf{L}}^{\left(1\right)}$ and ${\mathbf{L}}^{\left(2\right)}$. Furthermore, their diagonal elements are $D_{ii}^{\left(1\right)}={\sum }_j{G}_{ij}^{\left(1\right)}$ and $D_{ii}^{\left(2\right)}={\sum }_j{G}_{ij}^{\left(2\right)}$, respectively.

To not only exploit polarimetric features, but also bring in the Wishart distance, we combine problem (7) and (8) by a co-regularization term. Then the optimization problem of the co-regularized graph embedding DR method is constructed as follows:

$$\begin{array}{cc}\hfill \underset{{\mathbf{F}}^{\left(1\right)},{\mathbf{F}}^{\left(2\right)}}{min}& \mathrm{tr}\left(\alpha {{\mathbf{F}}^{\left(1\right)}}^T{\mathbf{L}}^{\left(1\right)}{\mathbf{F}}^{\left(1\right)}+(1-\alpha ){{\mathbf{F}}^{\left(2\right)}}^T{\mathbf{L}}^{\left(2\right)}{\mathbf{F}}^{\left(2\right)}\right)+\lambda \mathrm{Cor}({\mathbf{F}}^{\left(1\right)},{\mathbf{F}}^{\left(2\right)})\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& {{\mathbf{F}}^{\left(1\right)}}^T{\mathbf{F}}^{\left(1\right)}=\mathbf{I},{{\mathbf{F}}^{\left(2\right)}}^T{\mathbf{F}}^{\left(2\right)}=\mathbf{I}\hfill \end{array}$$

(9)

where $\alpha$ and $1-\alpha$ are parameters to balance two low-dimensional features, $\lambda$ is the parameter for the co-regularization term. Here $\mathrm{Cor}({\mathbf{F}}^{\left(1\right)},{\mathbf{F}}^{\left(2\right)})$ is the co-regularization term, which measures the similarity of two low-dimensional features ${\mathbf{F}}^{\left(1\right)}$ and ${\mathbf{F}}^{\left(2\right)}$. A natural way is as follows:

$$\mathrm{Cor}1({\mathbf{F}}^{\left(1\right)},{\mathbf{F}}^{\left(2\right)})={∥{\mathbf{F}}^{\left(1\right)}-{\mathbf{F}}^{\left(2\right)}∥}_F^2$$

(10)

As with [33], another commonly used similarity measure is as follows:

$$\mathrm{Cor}2({\mathbf{F}}^{\left(1\right)},{\mathbf{F}}^{\left(2\right)})={∥{\mathbf{F}}^{\left(1\right)}{{\mathbf{F}}^{\left(1\right)}}^T-{\mathbf{F}}^{\left(2\right)}{{\mathbf{F}}^{\left(2\right)}}^T∥}_F^2$$

(11)

3.4. Optimization

Specific to two different co-regularization terms (10) and (11), we will describe how to solve the corresponding optimization problems.

(1) When $\mathrm{Cor}1({\mathbf{F}}^{\left(1\right)},{\mathbf{F}}^{\left(2\right)})={∥{\mathbf{F}}^{\left(1\right)}-{\mathbf{F}}^{\left(2\right)}∥}_F^2$, problem (9) becomes

$$\begin{array}{cc}\hfill \underset{{\mathbf{F}}^{\left(1\right)},{\mathbf{F}}^{\left(2\right)}}{min}& \mathrm{tr}\left(\alpha {{\mathbf{F}}^{\left(1\right)}}^T{\mathbf{L}}^{\left(1\right)}{\mathbf{F}}^{\left(1\right)}+(1-\alpha ){{\mathbf{F}}^{\left(2\right)}}^T{\mathbf{L}}^{\left(2\right)}{\mathbf{F}}^{\left(2\right)}\right)+\lambda {∥{\mathbf{F}}^{\left(1\right)}-{\mathbf{F}}^{\left(2\right)}∥}_F^2\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& {{\mathbf{F}}^{\left(1\right)}}^T{\mathbf{F}}^{\left(1\right)}=\mathbf{I},{{\mathbf{F}}^{\left(2\right)}}^T{\mathbf{F}}^{\left(2\right)}=\mathbf{I}\hfill \end{array}$$

(12)

Assuming ${\mathbf{f}}^{\left(1\right)},{\mathbf{f}}^{\left(2\right)}$ are arbitrary column vectors from ${\mathbf{F}}^{\left(1\right)},{\mathbf{F}}^{\left(2\right)}$, the Lagrange function of problem (12) is as follows:

$$L({\mathbf{f}}^{\left(1\right)},{\mathbf{f}}^{\left(2\right)},\mu )=\alpha {{\mathbf{f}}^{\left(1\right)}}^T{\mathbf{L}}^{\left(1\right)}{\mathbf{f}}^{\left(1\right)}+(1-\alpha ){{\mathbf{f}}^{\left(2\right)}}^T{\mathbf{L}}^{\left(2\right)}{\mathbf{f}}^{\left(2\right)}+\lambda {\parallel {\mathbf{f}}^{\left(1\right)}-{\mathbf{f}}^{\left(2\right)}\parallel }_F^2+\mu ({{\mathbf{f}}^{\left(1\right)}}^T{\mathbf{f}}^{\left(1\right)}-1)+\mu ({{\mathbf{f}}^{\left(2\right)}}^T{\mathbf{f}}^{\left(2\right)}-1)$$

(13)

where $\mu$ is the Lagrange multiplier. Sequently, we do the partial derivative of $L({\mathbf{f}}^{\left(1\right)},{\mathbf{f}}^{\left(2\right)},\mu )$. Set $\partial L/\partial {\mathbf{f}}^{\left(1\right)}=0$ and $\partial L/\partial {\mathbf{f}}^{\left(2\right)}=0$, i.e.,

$$\begin{array}{cc}\hfill & \frac{\partial L}{\partial {\mathbf{f}}^{\left(1\right)}}=2\alpha {\mathbf{L}}^{\left(1\right)}{\mathbf{f}}^{\left(1\right)}+2\lambda ({\mathbf{f}}^{\left(1\right)}-{\mathbf{f}}^{\left(2\right)})+2\mu {\mathbf{f}}^{\left(1\right)}=0\hfill \end{array}$$

(14)

$$\begin{array}{cc}\hfill & \frac{\partial L}{\partial {\mathbf{f}}^{\left(2\right)}}=2(1-\alpha ){\mathbf{L}}^{\left(2\right)}{\mathbf{f}}^{\left(2\right)}+2\lambda ({\mathbf{f}}^{\left(2\right)}-{\mathbf{f}}^{\left(1\right)})+2\mu {\mathbf{f}}^{\left(2\right)}=0\hfill \end{array}$$

(15)

The above two equations are reformulated by using a matrix form, i.e.,

$$\left(\begin{array}{cc}\alpha {\mathbf{L}}^{\left(1\right)}+\lambda \mathbf{I}& -\lambda \mathbf{I}\\ -\lambda \mathbf{I}& (1-\alpha ){\mathbf{L}}^{\left(2\right)}+\lambda \mathbf{I}\end{array}\right)\left(\begin{array}{c}{\mathbf{f}}^{\left(1\right)}\\ {\mathbf{f}}^{\left(2\right)}\end{array}\right)=\mu \left(\begin{array}{c}{\mathbf{f}}^{\left(1\right)}\\ {\mathbf{f}}^{\left(2\right)}\end{array}\right)$$

(16)

Through doing the eigenvalue decomposition of matrix $\left(\begin{array}{cc}\alpha {\mathbf{L}}^{\left(1\right)}+\lambda \mathbf{I}& -\lambda \mathbf{I}\\ -\lambda \mathbf{I}& (1-\alpha ){\mathbf{L}}^{\left(2\right)}+\lambda \mathbf{I}\end{array}\right)$, we select the d eigenvectors specific to the smallest d eigenvalues for $\left(\begin{array}{c}{\mathbf{F}}^{\left(1\right)}\\ {\mathbf{F}}^{\left(2\right)}\end{array}\right)$. The detailed algorithm is given in Algorithm 1.

Algorithm 1: Co-regularized graph embedding (CRGE) with Cor1

Input: Two Laplacian matrices ${\mathbf{L}}^{\left(1\right)}$ and ${\mathbf{L}}^{\left(2\right)}$, the reduced dimensionality d. Step1: Do the eigenvalue decomposition of matrix $\left(\begin{array}{cc}\alpha {\mathbf{L}}^{\left(1\right)}+\lambda \mathbf{I}& -\lambda \mathbf{I}\\ -\lambda \mathbf{I}& (1-\alpha ){\mathbf{L}}^{\left(2\right)}+\lambda \mathbf{I}\end{array}\right)$. Step2: Select the d eigenvectors specific to the smallest d eigenvalues as $\left(\begin{array}{c}{\mathbf{F}}^{\left(1\right)}\\ {\mathbf{F}}^{\left(2\right)}\end{array}\right)$. Output: Low-dimensional features ${\mathbf{F}}^{\left(1\right)}$ and ${\mathbf{F}}^{\left(2\right)}$

(2) When $\mathrm{Cor}2({\mathbf{F}}^{\left(1\right)},{\mathbf{F}}^{\left(2\right)})={∥{\mathbf{F}}^{\left(1\right)}{{\mathbf{F}}^{\left(1\right)}}^T-{\mathbf{F}}^{\left(2\right)}{{\mathbf{F}}^{\left(2\right)}}^T∥}_F^2$, based on ${{\mathbf{F}}^{\left(1\right)}}^T{\mathbf{F}}^{\left(1\right)}=\mathbf{I},{{\mathbf{F}}^{\left(2\right)}}^T{\mathbf{F}}^{\left(2\right)}=\mathbf{I}$, ${\parallel \mathbf{A}\parallel }_F^2=\mathrm{tr}\left({\mathbf{AA}}^T\right)$ and regardless of the constant term and the scale term, we approximately have

$$\begin{array}{c}\hfill {∥{\mathbf{F}}^{\left(1\right)}{{\mathbf{F}}^{\left(1\right)}}^T-{\mathbf{F}}^{\left(2\right)}{{\mathbf{F}}^{\left(2\right)}}^T∥}_F^2=-\mathrm{tr}\left({\mathbf{F}}^{\left(1\right)}{{\mathbf{F}}^{\left(1\right)}}^T{\mathbf{F}}^{\left(2\right)}{{\mathbf{F}}^{\left(2\right)}}^T\right)\end{array}$$

(17)

Therefore, problem (9) becomes:

$$\begin{array}{cc}\hfill \underset{{\mathbf{F}}^{\left(1\right)},{\mathbf{F}}^{\left(2\right)}}{min}& \mathrm{tr}\left(\alpha {{\mathbf{F}}^{\left(1\right)}}^T{\mathbf{L}}^{\left(1\right)}{\mathbf{F}}^{\left(1\right)}+(1-\alpha ){{\mathbf{F}}^{\left(2\right)}}^T{\mathbf{L}}^{\left(2\right)}{\mathbf{F}}^{\left(2\right)}\right)-\lambda \mathrm{tr}\left({\mathbf{F}}^{\left(1\right)}{{\mathbf{F}}^{\left(1\right)}}^T{\mathbf{F}}^{\left(2\right)}{{\mathbf{F}}^{\left(2\right)}}^T\right)\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& {{\mathbf{F}}^{\left(1\right)}}^T{\mathbf{F}}^{\left(1\right)}=\mathbf{I},{{\mathbf{F}}^{\left(2\right)}}^T{\mathbf{F}}^{\left(2\right)}=\mathbf{I}\hfill \end{array}$$

(18)

Problem (18) can be solved by using an alternate optimization algorithm for ${\mathbf{F}}^{\left(1\right)}$ and ${\mathbf{F}}^{\left(2\right)}$. Firstly, by doing eigenvalue decomposition of ${\mathbf{L}}^{\left(1\right)}$ and ${\mathbf{L}}^{\left(2\right)}$, d eigenvectors which are specific to d smallest eigenvalues are used as the initialized ${\mathbf{F}}^{\left(1\right)}$ and ${\mathbf{F}}^{\left(2\right)}$. The iterative process is as follows:

• Fixing ${\mathbf{F}}^{\left(1\right)}$, solve for ${\mathbf{F}}^{\left(2\right)}$. Problem (18) becomes:

  $$\begin{array}{cc}\hfill \underset{{\mathbf{F}}^{\left(2\right)}}{min}& \mathrm{tr}\left({{\mathbf{F}}^{\left(2\right)}}^T\left((1-\alpha ){\mathbf{L}}^{\left(2\right)}-\lambda {\mathbf{F}}^{\left(1\right)}{{\mathbf{F}}^{\left(1\right)}}^T\right){\mathbf{F}}^{\left(2\right)}\right)\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& {{\mathbf{F}}^{\left(2\right)}}^T{\mathbf{F}}^{\left(2\right)}=\mathbf{I}\hfill \end{array}$$

  (19)
  Therefore d eigenvectors which are specific to d smallest eigenvalues of $(1-\alpha ){\mathbf{L}}^{\left(2\right)}-\lambda {\mathbf{F}}^{\left(1\right)}{{\mathbf{F}}^{\left(1\right)}}^T$ form ${\mathbf{F}}^{\left(2\right)}$.
• Fixing ${\mathbf{F}}^{\left(2\right)}$, solve for ${\mathbf{F}}^{\left(1\right)}$. Problem (18) becomes:

  $$\begin{array}{cc}\hfill \underset{{\mathbf{F}}^{\left(1\right)}}{min}& \mathrm{tr}\left({{\mathbf{F}}^{\left(1\right)}}^T(\alpha {\mathbf{L}}^{\left(1\right)}-\lambda {\mathbf{F}}^{\left(2\right)}{{\mathbf{F}}^{\left(2\right)}}^T){\mathbf{F}}^{\left(1\right)}\right)\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& {{\mathbf{F}}^{\left(1\right)}}^T{\mathbf{F}}^{\left(1\right)}=\mathbf{I}\hfill \end{array}$$

  (20)
  Therefore d eigenvectors which are specific to d smallest eigenvalues of $\alpha {\mathbf{L}}^{\left(1\right)}-\lambda {\mathbf{F}}^{\left(2\right)}{{\mathbf{F}}^{\left(2\right)}}^T$ form ${\mathbf{F}}^{\left(1\right)}$.

The stop error $\epsilon$ or the maximum number of iterations $T_{max}$ can be used as the stop criterion of the iterative process. The detailed algorithm is given in Algorithm 2.

Based on the obtained low-dimensional features ${\mathbf{F}}^{\left(1\right)}$ and ${\mathbf{F}}^{\left(2\right)}$, $\left(\begin{array}{c}{\mathbf{F}}^{\left(1\right)}\\ {\mathbf{F}}^{\left(2\right)}\end{array}\right)$ is used as features for the final classification.

Algorithm 2: Co-regularized graph embedding (CRGE) with Cor2

Input: Two Laplacian matrices ${\mathbf{L}}^{\left(1\right)}$ and ${\mathbf{L}}^{\left(2\right)}$, the reduced dimensionality d. Initialize: d eigenvectors specific to d smallest eigenvalues of ${\mathbf{L}}^{\left(1\right)}$ form ${\mathbf{F}}^{\left(1\right)}$, d eigenvectors specific to d smallest eigenvalues of ${\mathbf{L}}^{\left(2\right)}$ forms ${\mathbf{F}}^{\left(2\right)}$. Step1: Fix ${\mathbf{F}}^{\left(1\right)}$ to update ${\mathbf{F}}^{\left(2\right)}$ (1) Do the eigenvalue decomposition of the matrix $(1-\alpha ){\mathbf{L}}^{\left(2\right)}-\lambda {\mathbf{F}}^{\left(1\right)}{{\mathbf{F}}^{\left(1\right)}}^T$ (2) Select d eigenvectors specific to d smallest eigenvalues as ${\mathbf{F}}^{\left(2\right)}$ Step2: Fix ${\mathbf{F}}^{\left(2\right)}$ to update ${\mathbf{F}}^{\left(1\right)}$ (1) Do the eigenvalue decomposition of the matrix $\alpha {\mathbf{L}}^{\left(1\right)}-\lambda {\mathbf{F}}^{\left(2\right)}{{\mathbf{F}}^{\left(2\right)}}^T$ (2) Select d eigenvectors specific to d smallest eigenvalues as ${\mathbf{F}}^{\left(1\right)}$ $T=T+1$ until $T=T_{max}$ End iteration Output: Low-dimensional features ${\mathbf{F}}^{\left(1\right)}$ and ${\mathbf{F}}^{\left(2\right)}$

3.5. Computational Complexity Analysis of the Proposed Method

The computational complexity of the proposed method mainly consists of two steps, i.e., constructing two weighted graphs and solving the CRGE model. Firstly, because the neighbouring samples are searched in a local region, where the number of samples is approximately equal to k. Thus, the computation complexity is $O\left(nk\right)$, where n is the number of samples and k is the number of neighbouring samples. Then, for CRGE with Cor1, we need do the eigenvalue decomposition on a $2n\times 2n$ matrix, thus the computational complexity is $O\left({\left(2n\right)}^3\right)$. For CRGE with Cor2, the alternating iteration algorithm is needed. In each iteration, the algorithm relates to two eigenvalue decomposition problems on two $n\times n$ matrices. Thus, the computational complexity is each iteration is $O\left(2n^3\right)$. Therefore, the total computational complexity for the whole iterative process is $O\left(2T{n}^3\right)$, where T is the number of iterations.

4. Experiments

To validate the effectiveness of the co-regularized graph embedding method, experiments are implemented on three PolSAR data sets.

4.1. Description of Data Sets

The first one is the Flevoland data set for a part of cropland from Flevoland in the Netherlands. The size of the Flevoland data set is $750\times 1024$. Fifteen classes of crops are considered: stem beans, potatoes, peas, lucerne, wheat I, forest, beet, bare soil, grass, rapeseed, barely, wheat II, wheat III, water, and buildings. Figure 2a shows the PauliRGB image of this data set, where red is for $|S_{HH}-S_{VV}|$, green is for $|S_{HV}+S_{VH}|$, and blue is for $|S_{HH}+S_{VV}|$. Figure 2b describes the corresponding ground truth for fifteen classes of crops. To conduct more experiments to discuss parameter setting, we firstly select a subarea of the Flevoland data set with $200\times 320$ pixels. The subset consists of nine classes of crops: stem beans, rapeseed, wheat I, grass, lucerne, potatoes, bare soil, wheat II, and sugar beat. The PauliRGB image of the subset and its ground truth map are shown in Figure 2d,e.

The second data set is the Oberpfaffenhofen data set for the Oberpfaffenhofen area from Germany. The data set has $700\times 700$ pixels which include three classes of land covers: wood land, built-up areas and open areas. Figure 3a shows the corresponding PauliRGB image and Figure 3b is the ground truth map.

The third data set is the San Francisco Bay data set for the San Francisco Bay area from the USA. It is four-look L-band data set and has $900\times 1024$ pixels. Four classes of land covers are considered, i.e., mountains, sea, buildings and grass [35]. Figure 4a shows the PauliRGB image and Figure 4b is the ground truth map.

Because of the coherent imaging mechanism, there exist strong speckle noise in PolSAR images. Therefore, the refined Lee filter firstly is exploited for speckle reduction. Furthermore, we segment a PolSAR image into many superpixels to enhance the classification performance and decrease the computation cost. In our experiments, the adaptive superpixel generation method [36] is used to obtain superpixels. Figure 2c, Figure 3c and Figure 4c show the the segmentation results for three data sets, respectively. Then we conduct experiments based on a superpixel rather than a single pixel. The mean of polarimetric feature vectors of pixels in a superpixel is used as the polarimetric feature vector of the superpixel. The mean of covariance matrices of pixels in a superpixel is used as the covariance matrix of the superpixel.

Regarding methods for comparison, the classical PolSAR image classification method, i.e., the Wishart classifier (WC) [4] is used as a baseline. Moreover, two DR methods, i.e., principle component analysis (PCA), Laplacian eigenmaps (LE) [12] are employed for comparison with the proposed method. To further validate the effectiveness of the combination by co-regularization, two extreme cases of the proposed method are also used for comparison. One case when $\lambda =0,\alpha =1$ means that we just use a weighted graph based on Wishart distance, which is equal to Wishart distance-based Laplacian embedding (WDLE). In the same way, the other case when $\lambda =0,\alpha =0$ means that we just use a weighted graph based on Euclidean distance of polarimetric features, which is equal to polarimetric feature-based Laplacian embedding (PFLE). Please note that for WDLE and PFLE, neighbouring samples are searched in a local region, which is different from LE [12] (LE search neighbouring samples globally). In addition, all methods are based on superpixels. To emphasize the role of feature extraction methods in PolSAR image classification, we just employ nearest neighbour (NN) classifier for classification based on obtained low-dimensional features by PCA, LE, PFLE, WDLE and the proposed CRGE. We randomly select $1\%$ samples for training in the NN classifier and compute the average accuracy of 10 experiments as the final accuracy.

4.2. Parameter Setting

The parameter setting is discussed in this part. The experiments which are specific to parameter setting discussion are conducted on the subset of the Flevoland data set. About two types of regularization mentioned in Section 3, i.e., Equations (10) and (11), the corresponding two models are problems (12) and (18). The first one as is denoted as CRGE-Cor1 and the other one is denoted as CRGE-Cor2. Experiments of CRGE-Cor1 and CRGE-Cor2 are carried out with different sizes of the local patch $h\times h$ and the number of neighbouring samples k. The classification accuracies are shown in Figure 5. We list three cases when $h=41$, $h=61$ and $h=81$. Because the proposed method is based on superpixel which contains about 200 pixels, the number of superpixels in the $h\times h$ patch is limited, such as the $41\times 41$ region contains 7 superpixels at most, which is shown in Figure 5a. We can see that CRGE-Cor2 performs much better than CRGE-Cor1. CRGE-Cor1 performs poorly under different sizes of the local region and the number of the neighbouring samples. The highest classification accuracy of CRGE-Cor1 is lower than $90\%$ and is much lower than the classification accuracy of GRGE-Cor2. Therefore, we adopt the second type of regularization, i.e., CRGE-Cor2 for the following experiments. Please note that CRGE means CRGE-Cor2 in the following description. Moreover, regarding the size of the local patch $h\times h$ and the number of neighbouring samples k, taking the classification performance and the computation burden into account, we select $61\times 61$ local region and 10 neighbouring samples, i.e., $h=61,k=10$ for the experiments in the subset of the Flevoland data set.

About the reduced dimensionality d, Figure 6 shows the classification accuracy of six methods with dimensionality which ranges from 1 to 10. We can see that the classification accuracy stays unchange when the dimensionality is larger than 5. Therefore, we select $d=6$ as the reduced dimensionality. Moreover, the proposed co-regularized graph embedding method has a higher accuracy than other methods with ranging reduced dimensionality. When the reduced dimensionality $d=4$, the classification accuracy of the proposed co-regularized graph embedding method reached nearly $100\%$. Moreover, for other three data sets, i.e., the full Flevoland data set, the Oberpfaffenhofen data set and the San Francisco Bay data set, the classification accuracies of the proposed method with dimensionality ranging from 1:10 are shown in Figure 7. We can see that when the dimensionality reaches 6, the classification accuracy almost stop increasing. Therefore, the reduced dimensionality is set as 6 for three data sets. The stop error $\epsilon$ for the iterative process is set as $\epsilon ={10}^{-6}$. The maximum number of iterations $T_{max}$ is set as $T_{max}=10$. The parameter $\alpha$ and $\lambda$ are set as $\alpha =0.1,\lambda =0.2$.

4.3. Experimental Results

Table 1. Classification accuracies of six methods on the subset of the Flevoland data set.

Method	WC		PCA		LE		PFLE		WDLE		CRGE
	UA	PA	UA	PA	UA	PA	UA	PA	UA	PA	UA	PA
Potatoes	99.60	99.30	96.20	96.28	97.84	98.14	92.10	99.28	92.54	96.66	100	99.92
Grass	86.04	96.92	97.81	96.60	89.48	90.09	98.13	94.87	98.70	97.42	98.70	100
Beet	99.75	98.25	90.37	91.87	87.33	95.04	95.44	97.31	95.18	91.16	99.75	99.49
Lucerne	90.29	66.75	81.96	96.29	93.04	89.22	100	84.37	94.17	100	100	100
Wheat I	98.56	100	99.43	100	86.07	95.78	100	100	94.10	95.92	100	99.91
Wheat II	98.44	97.17	100	97.78	82.54	74.59	100	100	100	76.35	100	100
Stem beans	96.33	97.64	96.33	60.97	96.33	56.45	100	91.74	97	92.09	100	87.46
Bare soil	100	100	100	100	88.90	51.89	100	100	100	100	100	100
Rapeseed	100	91.75	100	99.13	100	82.31	100	99.13	77.19	100	100	100
OA	95.50		95.65		91.07		96.76		94.88		99.47

gives classification accuracies of six methods on the subset of the Flevoland data set, which include user accuracy (UA), producer accuracy (PA) for each classes and overall accuracy (OA). Note that the bold means the highest UA, PA or OA. Figure 8 presents the visual classification results maps of six methods on the subset of the Flevoland data set. We can see that the LE performs worst among six methods. WC performs nearly as well as PCA. PFLE performs a little better than PCA and WC. WDLE performs fairly worse than WC, PCA and PFLE. The overall accuracy (OA) of the proposed CRGE is higher than other methods. The user accuracy (UA) for each class of the proposed method is higher than other comparing methods. The overall accuracy of the proposed method is larger than $99\%$ and the classification map is nearly similar to the ground truth map. Moreover, although both PFLE and WDLE cannot achieve a satisfactory classification performance, the combination of two distances based on co-regularization contributes to the improvement of the classification performance.

Table 2. Classification accuracies of six methods on the Flevoland data set.

Method	WC		PCA		LE		PFLE		WDLE		CRGE
	UA	PA	UA	PA	UA	PA	UA	PA	UA	PA	UA	PA
Stem beans	88.71	92.66	79.06	87.11	79.27	95.24	78.16	71.45	76.72	77.43	94.64	94.88
Peas	93.51	98.27	95.03	85.59	94.18	94.83	84.68	89.55	80.38	80.60	98.57	89.88
Forest	94.33	97.87	94.98	98.94	91.39	95.40	75.68	76.74	86.42	87.14	97.31	97.61
Lucerne	83.77	98.34	92.63	87.83	94.03	90.46	76.38	79.20	81.95	78.25	99.81	94.90
Wheat I	86	96.50	90.30	85.78	87.27	83.45	78.06	76.10	81.58	83.29	100	98.35
Beet	95.83	84.75	88.03	89.22	83.77	87.57	71	73.53	73.72	76.68	93.18	92.91
Potatoes	93.85	95.70	91.39	92	91.12	84.79	76.86	77.08	90.49	85.99	97.27	94.87
Bare soil	99.94	35.41	98.44	91.79	99.94	98.91	74.79	76.10	85.12	72.98	99.29	100
Grasses	79.53	91.64	88.93	87.70	82.60	77.47	69.64	78.06	77.80	91.93	95.52	98.49
Rapeseed	77.57	81.13	88.55	88.52	93.79	0.9190	74.79	71.93	79.76	75.23	95.26	98.93
Barely	96.84	72.33	90.92	91.49	85.96	8483	82.64	79.39	78.20	88.38	93.39	98.38
Wheat II	92.38	71.29	92.13	94.74	94.21	95.91	89.59	82.22	78.81	80.08	94.74	98.67
Wheat III	95	97.64	92.41	95.68	90.59	93.26	82.44	83.77	86.50	85.65	96.32	98.08
Water	58.36	100	97.02	99.65	99.09	100	81.01	82.19	98.51	97.25	98.95	99.67
Buildings	81.72	100	88.45	77.53	91.81	100	81.72	97.01	98.53	100	81.72	95.81
OA	87.68		91.66		90.78		78.73		83.77		96.87

gives the classification accuracies of six methods on the Flevoland data set, which include user accuracy (UA), producer accuracy (PA) for each classes and overall accuracy (OA). Figure 9 presents visual classification maps of six methods on the Flevoland data set. It is obvious that PFLE and WDLE performs very badly while LE performs better than them. The reason may be that the Flevoland data set has a large size and contains many classes. Samples from one class distribute quite diversely, which results in that local search just based on one distance is not fit for this data set. However, the proposed CRGE performs best among the comparing methods, which can evaluate the combination of the two distances by co-regularization can improve the classification performance. PCA performs better than LE and WC. The overall accuracy of the proposed CRGE method is about $5\%$ larger than ones of other methods.

Table 3. Classification accuracies of six methods on the Oberpfaffenhofen data set.

Method	WC		PCA		LE		PFLE		WDLE		CRGE
	UA	PA	UA	PA	UA	PA	UA	PA	UA	PA	UA	PA
Built-up areas	36.39	47.52	90.33	90.43	85.80	86.32	95.21	95.16	96.52	94.81	96.83	95.38
Wood land	80.68	89.71	97.80	96.82	95.78	95.85	95.60	95.06	97.12	97.19	99.05	98.67
Open areas	98.89	74.55	95.99	97.60	96.59	96.08	90.91	91.83	93.78	94.80	96.99	98.62
OA	77.41		95.80		94.07		94.13		96.01		98

gives the classification accuracies of six methods on the Oberpfaffenhofen data set, which include user accuracy (UA), producer accuracy (PA) for each classes and overall accuracy (OA). Figure 10 presents visual classification maps of six methods on the Oberpfaffenhofen data set. The size of the local patch is set as $h=81$. The number of neighbouring samples is set as $k=15$. The Wishart classifier performs worst among compared methods mainly because a great amount of “built-up areas” are misclassified as shown in Figure 10b. Table 2 presents that the user accuracy of class “built-up areas” is $36.39\%$ while the user accuracies of other classes are larger than $85\%$. PCA, LE, PFLE and WDLE perform similarly. The classification accuracy of the proposed CRGE method reach $98\%$. The classification map of the proposed method also seem closer to the ground truth map.

Table 4. Classification accuracies of six methods on the San Francisco Bay data set.

Method	WC		PCA		LE		PFLE		WDLE		CRGE
	UA	PA	UA	PA	UA	PA	UA	PA	UA	PA	UA	PA
Sea	97.54	98.26	99.01	97.52	98.93	98.37	90.16	89.77	96.67	95.54	99.06	98.63
Mountains	78.51	56.78	89.10	94.48	90.96	90.92	81.16	80.27	90.12	88.68	96.79	98.38
Grass	71.12	49.40	85.44	82.06	84.99	83.01	77.90	74.55	89.07	86.92	93.11	89.49
Buildings	72.51	92.41	93.08	94.93	93.57	95.03	88.79	90.79	93.54	95.72	95.97	97.52
OA	82.30		93.97		94.06		87.21		93.86		96.80

gives the classification accuracies of six methods on the San Francisco Bay data set, which include user accuracy (UA), producer accuracy (PA) for each classes and overall accuracy (OA). Figure 11 presents the visual classification maps of six methods on the San Francisco Bay data set. The size of the local patch and the number of neighbouring samples are set as $h=101$ and $k=20$. The Wishart classifier still performs worst among compared methods mainly because many samples from three classes of land covers: mountains, grass and buildings are misclassified. PCA, LE and WDLE perform similarly and PFLE performs worse than them. The proposed co-regularized graph embedding method is superior to other methods. The classification accuracy is about $2\%$ larger than ones of other methods. The user accuracy and producer accuracy for each class of the proposed CRGE method are higher than other comparing method. Obviously, the classification map of the proposed method appears more similar to the ground truth map.

4.4. Computational Speed Comparison

Table 5. Computational time of six methods on three data sets.

Method	WC	PCA	LE	PFLE	WDLE	CRGE
Flevoland data set	70 s	29 s	100 s	70 s	90 s	420 s
Oberpfaffenhofen data set	30 s	35 s	57 s	42 s	48 s	180 s
San Francisco Bay data set	17 s	105 s	298 s	256 s	210 s	980 s

shows the computational time of six methods on three data sets. We can see that WC costs little time on the Oberpfaffenhofen data set and the San Francisco Bay data set because the classification of WC is based on the mean of a class. Other methods employ the nearest neighbour (NN) classifier, which is based on each pixel of a class. Therefore, other methods cost more time than WC. However, for the Flevoland data set, WC takes more time than PCA because the number of classes is larger than other datasets’. The computational time of PCA is relatively less than LE, PFLE, WDLE and CRGE because PCA has the explicit projection matrix to extract low-dimensional features, which can reduce computational time. PFLE and WDLE take less time than LE because of local search. The proposed CRGE costs the most time because it has an iterative process to pursue the low-dimensional features.

5. Discussion

Because the Wishart distance of covariance matrices and Euclidean distance of polarimetric features are two important similarity measurements for PolSAR image classification, this paper attempts to propose the co-regularized graph embedding DR method to combine the two distances. Two weighted graphs are constructed corresponding to the two distances. Then the DR model is built based on the two graphs and co-regularization.

Observing the experimental results, we can see that Cor2 is proper for PolSAR image classification and the corresponding DR model can enhance the classification performance as shown in Figure 5. Under different sizes of the local patch and the number of neighbouring samples, CRGE-Cor1 performs much worse than CRGE-Cor2. Therefore, we employ CRGE-Cor2 for experiments. It is obvious that the proposed method performs better than other comparing methods on three data sets, especially, on the Flevoland data set. The proposed method can increase the overall accuracy by $5\%$ and has a better classification map on the Flevoland data set. In fact, classification of the Flevoland data set is a more difficult problem because it contains 15 classes and has a large size, which is a strong evidence of the superiority of the proposed method. Moreover, two extreme cases of the proposed method, i.e., PFLE and WDLE perform worst on the Flevoland data set among six method. On other data sets, both of them cannot achieve a satisfactory classification performance. Therefore, we can conclude that the combination of the two distances by co-regularization indeed can improve the classification performance including the numerical results and visual results.

From Table 5, we can see that the proposed method costs much time for the data sets with a large size. To take a comprehensive consideration of balance the accuracy and speed, for a data set with a small size and many classes, the proposed method is a good choice for PolSAR image classification.

6. Conclusions

This paper put forward a co-regularized graph embedding method in basis of two types of distances (i.e., Wishart distance of covariance matrices and Euclidean distance of polarimetric features) for PolSAR image classification. Two weighted graphs are constructed specific to two distances. Then the DR model is constructed based on two weight graphs and co-regularization. The co-regularization aims to minimize the dissimilarity of low-dimensional features corresponding to two graphs. The co-regularized graph-embedding DR model can achieve a better classification performance than compared methods.

Moreover, the proposed co-regularized embedding method mainly focuses on graphs corresponding to different similarity measurements rather than the multiple data sets from different perspectives. We directly pursue the low-dimensional features specific to different graphs. Therefore, the original data sets become unnecessary and the similarity between two samples is indispensable, which can result in two advantages: (1) for one data set, the proposed method can use different similarity measurements to extract features from different perspectives, which can characterize the data more comprehensively; (2) sometimes great difference in multiple data sets from different perspectives may influence the classification performance or bring challenges for data representation, which can be avoided by the proposed method.

The proposed method has some limitations: (1) the proposed method is specific to two weighted graphs and cannot deal with more graphs; (2) although CRGE with Cor2 can enhance the classification performance, it is time consuming.

In future work, we will mainly focus on extending the proposed method for more graphs, and design the co-regularization which can meet the requirements of accuracy and speed.