A Survey on High-Dimensional Subspace Clustering

Abstract

With the rapid development of science and technology, high-dimensional data have been widely used in various fields. Due to the complex characteristics of high-dimensional data, it is usually distributed in the union of several low-dimensional subspaces. In the past several decades, subspace clustering (SC) methods have been widely studied as they can restore the underlying subspace of high-dimensional data and perform fast clustering with the help of the data self-expressiveness property. The SC methods aim to construct an affinity matrix by the self-representation coefficient of high-dimensional data and then obtain the clustering results using the spectral clustering method. The key is how to design a self-expressiveness model that can reveal the real subspace structure of data. In this survey, we focus on the development of SC methods in the past two decades and present a new classification criterion to divide them into three categories based on the purpose of clustering, i.e., low-rank sparse SC, local structure preserving SC, and kernel SC. We further divide them into subcategories according to the strategy of constructing the representation coefficient. In addition, the applications of SC methods in face recognition, motion segmentation, handwritten digits recognition, and speech emotion recognition are introduced. Finally, we have discussed several interesting and meaningful future research directions.

1. Introduction

Clustering has been one of the most important research topics in both statistical machine learning and data mining, which aims to segment unlabeled samples into several disjoint groups (clusters) by maximizing the difference among different groups and minimizing the difference in the same group. Clustering originated from the anthropological research by Driver and Kroeber in 1932 [1] and was then introduced into psychology [2,3]. Over the last few decades, various clustering methods have been proposed, such as k-means clustering [4,5,6,7], spectral clustering [8,9,10,11], model-based clustering [12,13,14], hierarchical clustering [15,16,17], and online clustering [18,19].

With the rapid development of information technology and the advent of the big data era, data have shown explosive growth and presents complex characteristics such as high dimensionality, nonlinearity, and incompleteness. Although traditional clustering methods have achieved excellent results in low-dimensional data mining tasks, they often encounter serious bottlenecks in high-dimensional data mining tasks, which cannot meet the sparsity of high-dimensional data and avoid the impact of the “curse of dimensionality” [20,21]. Since the inherent characteristics of high-dimensional data, it usually lies in a union of low-dimensional structures instead of being uniformly distributed across the whole space. For instance, Figure 1 shows a set of sample points in ${\mathbb{R}}^3$ from a two-dimensional subspace (the plane ${\mathcal{S}}_1$) and two one-dimensional subspaces (the lines ${\mathcal{S}}_2$ and ${\mathcal{S}}_3$). It is worth pointing out that it is difficult to analyze the data as a whole, while the natural properties of this data can be better reflected in its underlying low-dimensional subspaces (plane or line). In fact, in many practical problems, the data in each class can be well represented by low-dimensional subspaces. For example, the face images of a person under different lighting conditions can be distributed in a nine-dimensional subspace [22,23], and the motion trajectories of rigidly moving objects in a video belong to different subspaces whose dimensions do not exceed three [22]. To explore high-dimensional data in a low-dimensional space, subspace clustering arises at the opportune time [24]. The subspace clustering aims to search for the underlying subspaces in dimension space or data space to obtain corresponding subspaces and clustering results. Over the past several decades, subspace clustering has been widely used in face recognition [25,26,27], motion segmentation [28,29,30], image processing [31,32,33], speech emotion recognition [34,35], social network [36,37], and other fields.

Definition 1 

(Subspace clustering, SC [38]). Given a set of samples $X=[{\mathit{x}}_1,\dots ,{\mathit{x}}_n]\in {\mathbb{R}}^{d\times n}$, where d and n are the number of features and samples, respectively. Assume that X are drawn from a union of k subspaces ${\left\{{\mathcal{S}}_i\right\}}_{i=1}^k$ with unknown dimensionality ${\left\{d_i\right\}}_{i=1}^k$. Subspace clustering aims to segment the samples according to the underlying subspaces which they are drawn from.

Due to the advantages of SC, a number of SC methods have been proposed during the past decades. Figure 2 shows the development trend of SC publications and their citations over the past two decades. It can be seen that SC shows a growth trend in general, and the growth trend has dramatically increased since 2013. According to their mechanisms of representing the subspaces, the existing SC methods can be roughly divided into four main categories: algebraic methods, iterative methods, statistical methods, and spectral clustering-based methods. These four categories of SC methods are briefly discussed below (see [39] for details). Algebraic methods include matrix factorization-based methods [40,41] and generalized principal component analysis (GPCA) [42,43], but algebraic methods are usually highly sensitive to noise and outliers, and the complexity of GPCA increases exponentially with the number and dimensions of subspaces. Iterative methods, such as k-subspaces [44,45,46], alternatively assign the data points to subspaces and fit the subspaces to the corresponding cluster. However, iterative methods usually need to know the number of subspaces and the dimensions of subspaces in advance and are sensitive to the initial point. Statistical methods can be regarded as the probabilistic description based on iterative methods, including probabilistic PCA (PPCA) [47,48], mixture of probabilistic PCA (MPPCA) [47,49], random sample consensus (RANSAC) [50,51], etc. Unfortunately, statistical methods usually require that the dimensions of subspaces are known and equal, and their complexity increases with the number of subspaces. Most recently, spectral clustering-based methods have played a dominant role in SC problems due to their easy implementation and insensitivity to data corruptions and initial points. In the following, we will focus on spectral clustering-based methods.

The spectral clustering-based methods usually perform the following two steps. (1) The representation coefficients are learned by a subspace representation model, which is used to construct the affinity matrix. (2) The final clustering result is obtained by performing spectral clustering on the affinity matrix. The basic framework of the spectral clustering-based SC methods is shown in Figure 3. It is worth pointing out that the representation coefficients learned in the first step play a vital role in the clustering effectiveness. However, owing to the complexity and diversity of the inherent unknown structure of the real data, some assumptions must be introduced on the data distributions, such as sparse or low-rank assumptions in the representation learning, thus leading to different ways to construct or learn representation coefficients.

This survey provides a comprehensive review of the recent development of spectral clustering-based SC methods (for convenience, hereinafter referred to as SC). The main contributions of this survey are summarized as follows:

• We systematically review the recent research progress of SC methods and summarize them into three categories and further subcategories according to the strategy of constructing representation coefficient; see Figure 4.
• We introduce the applications of SC methods in face recognition, motion segmentation, handwritten digits recognition, and speech emotion recognition and present several common benchmark datasets.
• We provide some potential research directions which can promote the development of this field.

The rest of this survey is organized as follows. Section 2, Section 3 and Section 4 introduce low-rank sparse SC methods, local structure preserving SC methods, and kernel SC methods, respectively; see

Table 1. A brief summary of SC methods.

SC Methods	Subcategories	Brief Description
Low-Rank Sparse SC	SR-Based SC [25,52,53,54,55,56,57,58,59,60,61]	Achieve subspace feature selection through sparse representation.
	LRR-Based SC [38,62,63,64,65,66,67,68,69,70,71,72]	Capture the global structure of data through low-rank representation.
	LRSR-Based SC [73,74,75,76,77,78,79,80,81,82,83]	Capture the global structure of data and achieve subspace feature selection by combining low-rank representation with sparse representation.
Local Structure Preserving SC	GR-Based SC [84,85,86,87,88,89,90,91]	Capture the geometric information by combining graph regularization with data self-representation.
	LatR-Based SC [92,93,94,95,96,97,98]	Capture latent representation of data by joint low-rank/sparse recovery and salient feature extraction
	BDR-Based SC [99,100,101,102,103,104]	Pursue the block diagonal structure of coefficient matrix directly by designing special regularization.
Kernel SC	SKL-Based SC [55,105,106,107]	Be able to process the nonlinear structure of the data.
	MKL-Based SC [108,109,110,111,112,113,114,115,116]	Be able to process the nonlinear structure of the data and learn a consensus kernel function adaptively.

Remark: SR indicates sparse representation; LRR indicates low-rank representation; LRSR indicates low-rank sparse representation; GR indicates graph-regularized; LatR indicates latent representation; BDR indicates block diagonal representation; SKL indicates single kernel learning; MKL indicates multiple kernel learning.

. Section 5 introduces the applications of SC methods and presents several common benchmark datasets. Finally, the future research directions and conclusions are presented in Section 6 and Section 7, respectively.

The symbols used throughout this survey are defined at the end of this section. In this survey, all matrices are denoted by capital letters, such as $X,Y$; all vectors are represented by boldface letters, such as $\mathit{x},\mathit{y}$; all scalars are represented by lowercase letters, such as $x,y$. Let ${\mathbb{R}}^n$ and ${\mathbb{R}}^{d\times n}$ be the set of n-dimensional vectors and $d\times n$ matrices, respectively. For any matrix $X\in {\mathbb{R}}^{d\times n}$, let $X_{ij}$ and ${\mathit{x}}_j$ denote its $(i,j)$th element and jth column, respectively. The element-wise $l_1$ norm is defined as ${\parallel X\parallel }_1={\sum }_{i=1}^d{\sum }_{j=1}^n\left|X_{ij}\right|$; ${\parallel X\parallel }_F=\sqrt{{\sum }_{i=1}^d{\sum }_{j=1}^n{\left|X_{ij}\right|}^2}$ denotes the Frobenius norm; ${\parallel X\parallel }_{2,q}={\left({\sum }_{j=1}^n{\left({\sum }_{i=1}^d{\left|X_{ij}\right|}^2\right)}^{q/2}\right)}^{1/q}$ denotes the $l_{2,q}$ norm with $q>0$; the $l_0$ pseudo norm is defined as ${\parallel X\parallel }_0=\#\{X_{ij}\ne 0,\forall i,j\}$, i.e., the total number of non-zero elements in X. Let $r=\mathrm{rank}\left(X\right)$ represent the rank of X. Let ${\sigma }_1\left(X\right)\ge \dots \ge {\sigma }_r\left(X\right)\ge {\sigma }_{r+1}\left(X\right)=\dots ={\sigma }_d\left(X\right)=0$ are the singular value of X. ${\parallel X\parallel }_{K_{k,p}}={\left({\sum }_{i=1}^k{\sigma }_i{\left(X\right)}^p\right)}^{1/p}$ with $p>0$ and $k\in \{1,2,\dots ,d\}$ denotes the Ky Fan p-k norm. The particular case of the Ky Fan norm with $k=r$ is called the Schatten p norm, i.e., ${\parallel X\parallel }_{S_p}={\left({\sum }_{i=1}^r{\sigma }_i{\left(X\right)}^p\right)}^{1/p}$, when $p=1$ yields the nuclear norm ${\parallel X\parallel }_{*}$. In particular, the Schatten 0 pseudo norm is ${\parallel X\parallel }_{S_0}=\#\{{\sigma }_i\left(X\right)\ne 0,\forall i\}$. Let $\mathrm{tr}\left(X\right)$ be the trace of matrix X. We denote $\mathrm{diag}\left(X\right)$ as a vector whose ith element is the ith diagonal element of matrix X, while $\mathrm{Diag}\left(\mathbf{x}\right)$ as a matrix whose ith diagonal element is the ith element of vector $\mathbf{x}$. ⊙ denotes the Hadamard product, i.e., the element-wise product between two matrices. I and $\mathbf{1}$ represent the identity matrix and all one vector, respectively. If the matrix X is positive semi-definite, we denote $X⪰0$. If all the elements of X are nonnegative, we denote $X\ge 0$.

2. Low-Rank Sparse SC

In this section, we will review some classical low-rank sparse SC methods. Before starting the main content of this section, we first introduce some basic definitions of SC.

Definition 2 

(Independent subspace). A collection of subspaces ${\left\{{\mathcal{S}}_i\right\}}_{i=1}^k$ is said to be independent, if $\mathit{dim}\left({\oplus }_{i=1}^k{\mathcal{S}}_i\right)={\sum }_{i=1}^k\mathit{dim}\left({\mathcal{S}}_i\right)$, where $\mathit{dim}\left({\mathcal{S}}_i\right)$ is the dimension of subspace ${\mathcal{S}}_i$ and ⊕ represents the direct sum operation.

Definition 3 

(Disjoint subspace). A collection of subspaces ${\left\{{\mathcal{S}}_i\right\}}_{i=1}^k$ is said to be disjoint, if $\mathit{dim}({\mathcal{S}}_i\oplus {\mathcal{S}}_j)=\mathit{dim}\left({\mathcal{S}}_i\right)+\mathit{dim}\left({\mathcal{S}}_j\right)$ ($\forall i\ne j$), but $\mathit{dim}\left({\oplus }_{i=1}^k{\mathcal{S}}_i\right)\ne {\sum }_{i=1}^k\mathit{dim}\left({\mathcal{S}}_i\right)$.

It is evident that, when the number of subspaces is more than two, the independent subspace condition is much stronger than the disjoint condition, as the independent subspaces must be disjoint subspaces while the converse is not necessarily true.

Definition 4 

(Self-expressiveness property). Each sample in the union of subspaces can be effectively represented as a linear or affine combination of other samples, i.e., $X=XC$ where $X\in {\mathbb{R}}^{d\times n}$ is the given data, and $C\in {\mathbb{R}}^{n\times n}$ is the representation coefficient matrix.

It is worth pointing out that the representation coefficient matrix C can reflect the similarity between samples to a certain extent which will be used to build the affinity matrix. Therefore, the representation coefficient matrix plays a crucial role in spectral clustering-based SC methods. Ideally, the representation coefficient matrix C should satisfy the following diagonal property.

Definition 5 

(Block diagonal property, BDP). Given a sample matrix X drawn from a union of k disjoint subspaces ${\left\{{\mathcal{S}}_i\right\}}_{i=1}^k$, assume that the samples in the same subspace are aligned together, i.e., $X=[X_1,\dots ,X_K]$ where $X_i$ is a collection of $n_i$ samples drawn from the subspace ${\mathcal{S}}_i$. The representation coefficient matrix C is said to satisfy the block diagonal property, if

$$C=\left[\begin{array}{cccc}{C}_1& 0& \cdots & 0\\ 0& C_2& \cdots & 0\\ ⋮& ⋮& \ddots & ⋮\\ 0& 0& \cdots & C_k\end{array}\right]$$

(1)

with $C_i\in {\mathbb{R}}^{n_i\times n_i}$.

Note that, if the representation coefficient matrix C satisfies the block diagonal property (1), the following conclusions can be drawn: (1) the number of blocks represents the number of subspaces; (2) the size of each block represents the dimension of the corresponding subspace; and (3) the samples of the same block belong to the same subspace.

In practical applications, data often contain noise or outliers, so the data can be expressed as $X=YC+E$, where Y is the clean data or dictionary, generally taken as the observation X itself, and E represents the data corruptions, such as error, noise or outliers. In general, different prior constraints are imposed on the representation coefficient matrix C to obtain an ideal structure. Thus, the self-expressiveness based SC methods could be unified as solving the following optimization problems. Therefore, the self-expressiveness based SC methods can be uniformly described as the following optimization problems:

$$\underset{C,E}{min}F\left(C\right)+\lambda R\left(E\right)\mathrm{s}.\mathrm{t}.X=YC+E,C\in \Omega ,$$

(2)

where $F\left(C\right)$ is a penalty term (or regularization term), which is used to constrain the representation coefficient matrix C to keep an ideal structure, and $\Omega$ is the constraint set of C. $\lambda >0$ is a trade-off parameter. $R\left(E\right)$ is an error term describing the difference between the real data and the represented data. Different norms are selected to measure the error according to different noise distributions, which requires some prior knowledge or assumptions. Once the representation coefficient matrix C is determined, the affine matrix A for spectral clustering can be obtained through C. In the existing literature, a commonly used formula is $A=\frac{\left|C\right|+{\left|C\right|}^T}{2}$.

2.1. Sparse Representation Based SC

In recent years, sparse representation has become a research hotspot in machine learning, computer vision, applied mathematics, and so on. Sparsity refers to using as few bases as possible to represent data, that is, using as few non-zero representation coefficients as possible. The position of the non-zero representation coefficients indicates that the data are located in the subspace spanned by its corresponding basis. Some classical sparse SC methods are summarized in

Table 2. The comparison of classical sparse SC methods.

Methods	Year	$\mathit{F}\left(\mathit{C}\right)$	$\mathit{R}\left(\mathit{E}\right)$	$\mathbf{\Omega }$	Theoretical Results
SSC [25,52]	2009	${\parallel C\parallel }_1$	${\parallel E\parallel }_1$	$\left\{C\right|\mathrm{diag}\left(C\right)=\mathbf{0}\}$	BDP for independent subspaces
SSQP [53]	2011	$\parallel C^T{C\parallel }_1$	${\parallel E\parallel }_F^2$	$\left\{C\right|C\ge 0,\mathrm{diag}\left(C\right)=\mathbf{0}$ }	BDP for orthogonal subspaces
W-SSC [54]	2012	${\parallel W\odot C\parallel }_1$	${\parallel E\parallel }_p$	$\left\{C\right|\mathrm{diag}\left(C\right)=\mathbf{0}\}$	-
GS-graph [55]	2015	${\Gamma }_{\mu }^g\left(C\right)$	${\parallel E\parallel }_1$	-	Grouping effect
$l_0$-SSC [56]	2016	${\parallel C\parallel }_0$	${\parallel E\parallel }_F^2$	$\left\{C\right|\mathrm{diag}\left(C\right)=\mathbf{0}\}$	-
SSCNA [57]	2019	${\Phi }_{AL}\left(C\right)$	${\parallel E\parallel }_1$	$\left\{C\right|C^T\mathbf{1}=\mathbf{1},\mathrm{diag}\left(C\right)=\mathbf{0}\}$	-
SSC-SLp [58]	2019	${\parallel C\parallel }_p^p$	${\parallel E\parallel }_F^2$	$\left\{C\right|C^T\mathbf{1}=\mathbf{1},\mathrm{diag}\left(C\right)=\mathbf{0}\}$	-
SRSSC [59]	2019	${\parallel W\odot C\parallel }_1+\mu {\parallel Q\odot C\parallel }_1$	${\parallel E\parallel }_1$	$\left\{C\right|C^T\mathbf{1}=\mathbf{1},\mathrm{diag}\left(C\right)=\mathbf{0}\}$	-
SSC+E [60]	2020	${\sum }_{i=1}^n{\sum }_{j=1}^n{C}_{ij}ln{C}_{ij}$	${\parallel E\parallel }_F^2$	$\left\{C\right|C^T=C,C\ge 0,\mathrm{diag}\left(C\right)=\mathbf{0}\}$	-
AdaptiveSSC [61]	2020	${\parallel C/W\parallel }_1$	${\parallel E\parallel }_F^2$	$\left\{C\right|\mathrm{diag}\left(C\right)=\mathbf{0}\}$	-

Remark: - indicates that this item does not exist.

.

In 2009, Elhamifar and Vidal [25,52] introduced compressed sensing techniques to subspace segmentation and proposed sparse subspace clustering (SSC) to represent each data point as a sparse linear combination of other data points. SSC can be formulated as the following optimization problem:

$$\underset{C,E}{min}{\parallel C\parallel }_1+\lambda {\parallel E\parallel }_1\mathrm{s}.\mathrm{t}.X=XC+E,\mathrm{diag}\left(C\right)=\mathbf{0},$$

(3)

where the constraint $\mathrm{diag}\left(C\right)=\mathbf{0}$ is used to avoid the trivial solution of representing a data point as a linear combination of itself, i.e., $C=I$. In optimization, $l_1$-norm is usually regarded as the best convex relaxation of $l_0$ norm, which can be used to generate sparse solutions due to its nonsmoothness (see Figure 5 for the geometric interpretation). Note that model (3) is a convex optimization, which can be effectively solved by the convex programming algorithms, such as the alternating direction method of multipliers (ADMM) [117] and the semi-smooth Newton augmented Lagrangian method (SSNAL) [118].

For the SCC method (3) without noise, it has been proved that the representation coefficient matrix obtained by (3) has the block diagonal structure (1) under the assumption of independent subspaces. Elhamifar and Vidal [119] extended the condition of independent subspaces to disjoint subspaces and gave the theoretical boundary for the division of disjoint subspaces. Soltanolkotabi and Candès [120] further extended this condition to the case even though underlying subspaces overlap through geometric analysis. However, these analyses are based on the assumption that the data points are exactly lying in the subspace, which is difficult to satisfy in practice. Therefore, Soltanolkotabi et al. [121] generalized the SCC by using geometric functional analysis and proved that this method can accurately recover the underlying subspaces under the minimum requirements on the direction of the subspace and the number of samples in each subspace. Wang et al. [122] theoretically analyzed the SSC with noise and proved that the modified version of SSC can correctly identify the underlying subspace, even for noisy data. However, the SCC method still has some drawbacks. Since SSC seeks a sparse representation of each data individually, it may not accurately capture the global structure, which may lead to over-segmentation [123]. On the other hand, if the correlation between a group of data points is very high, SSC tends to select only one randomly, which will ignore the correlation structure of data from the same subspace [124].

Since SCC essentially solves a constrained lasso regression problem which is a nonsmooth problem, the computational cost of solving the problem is higher than general smooth problems. Consequently, Wang et al. [53] proposed the following subspace segmentation method via quadratic programming (SSQP)

$$\underset{C,E}{min}\parallel C^T{C\parallel }_1+\lambda {\parallel E\parallel }_F^2\mathrm{s}.\mathrm{t}.X=XC+E,C\ge 0,\mathrm{diag}\left(C\right)=\mathbf{0},$$

(4)

where $\parallel C^T{C\parallel }_1$ is equivalent to ${\mathbf{1}}^T{C}^{T}C\mathbf{1}$, which can be used to enforce the block diagonal structure of C. When the subspaces are orthogonal, it has been proved that the representation coefficient matrix obtained by SSQP (4) without noise has a block diagonal structure.

Pham et al. [54] present a weighted sparse subspace clustering (W-SSC) method by embedding weights into the sparse formulation, which can be formulated as

$$\underset{C,E}{min}{\parallel W\odot C\parallel }_1+\lambda {\parallel E\parallel }_p\mathrm{s}.\mathrm{t}.X=XC+E,\mathrm{diag}\left(C\right)=\mathbf{0},$$

(5)

where W is a non-negative weighting matrix, which can take the inverse of a similarity measure between points, such as $W_{ij}=exp{\left\{-\parallel {\mathbf{x}}_i-{\mathbf{x}}_j{\parallel }_2^2/\sigma \right\}}^{-1}$. ${\parallel E\parallel }_p$ is the penalty of noise E, which can be taken as $l_1$ norm, $l_{2,1}$ norm or the squared Frobenius norm, respectively, for the random corruptions, sample-specific corruptions and Gaussian white noise. It is worth pointing out that the SSC model (3) is a special case of (5) with $W=I$ and ${\parallel E\parallel }_p={\parallel E\parallel }_1$. W-SSC uses the spatial relationship among the data to weight the representation coefficient matrix, which improves the performance of SSC significantly. However, W-SSC may fall into a local minimization easily. For this reason, Wang et al. [59] proposed a structural reweight sparse subspace clustering method (SRSSC) by introducing the structural information into W-SSC. If the clustering labels of data ${\mathbf{x}}_i$ and ${\mathbf{x}}_j$ are the same, structural matrix $Q_{ij}=1$; otherwise, $Q_{ij}=0$. Furthermore, it has been proved that the global optimal solution can be found to be better by considering the structure information. However, there are three parameters needed to be adjusted in SRSSC, which is troublesome and time-consuming. In addition, both of the reweight SSC methods are sensitive to data noise.

Inspired by the traditional graph construction methods, Fang et al. [55] proposed a group sparse graph (GS-graph) subspace clustering method by constructing an informative graph using auto-grouped $l_1$ regularization, which can be written as

$$\underset{C,E}{min}{\Gamma }_{\mu }^g\left(C\right)+\lambda {\parallel E\parallel }_1\mathrm{s}.\mathrm{t}.X=XC+E,$$

(6)

where ${\Gamma }_{\mu }^g\left(C\right)={\parallel C\parallel }_1+{\sum }_i{\sum }_{j<k}max\left\{\right|C_{ij}|,|C_{ik}\left|\right\}$ is the auto-grouped octagonal shrinkage and clustering algorithm for regression (OSCAR) which can be regarded as a weighted combination of the $l_1$-norm and a pairwise $l_{\infty }$-norm [125]. Based on this design, GS-graph can maintain both the sparsity and locality of data simultaneously and is insensitive to noise. Moreover, it has proved that GS-graph (6) has the grouping effect.

Most of the existing sparsity-based SC methods are based on the $l_1$ norm of the representation coefficient. Although $l_1$ norm is regarded as the best convex relaxation of $l_0$ pseudo norm and has achieved many achievements, it does not fully reflect the properties of $l_0$ norm. Therefore, Yang et al. [56] directly studied the following $l_0$-induced sparse subspace clustering ($l_0$-SSC)

$$\underset{C,E}{min}{∥C∥}_0+\lambda {\parallel E\parallel }_F^2\mathrm{s}.\mathrm{t}.X=XC+E,\mathrm{diag}\left(C\right)=\mathbf{0}.$$

(7)

It has proved that the $l_0$-SSC can give subspace-sparse representation almost surely for arbitrary distinct underlying subspaces. Although the $l_0$-SSC model (7) is a nonconvex and nonsmooth optimization problem, it can be effectively solved by the proximal gradient descent (PGD) algorithm [56,126] and ADMM [127].

Although the $l_0$ pseudo norm regularization has been widely regarded as the origin model of sparsity and achieved superior clustering performance, its computational cost is expensive, and the convergence of algorithm is hardly guaranteed. A better strategy is to use some nonconvex functions to approximate $l_0$ pseudo norm, such as arctangent–logarithmic function [57] and $l_p$-norm [58]. Dong et al. [57] proposed a sparse subspace clustering via the nonconvex approximation (SSCNA). Specifically, SSCNA used the arctangent–logarithmic function ${\Phi }_{AL}\left(\mathbf{x}\right)=\frac{2}{\pi }{\sum }_{i=1}^{n}arctan\frac{ln\left(\right|x_i|+1)}{v}$ with $0<v<1$ to approximate the $l_0$-norm. It has proved that the proposed nonconvex approximation is closer to $l_0$-norm than the $l_1$-norm and is bounded by $l_0$ pseudo norm (see Figure 6). Dong et al. [58] formulated the sparse subspace clustering as a smoothed $l_p(0<p<1)$ minimization problem (SSC-SLp). It is worth pointing out that the $l_p$-norm can better approximate the $l_0$ pseudo norm than the $l_1$ norm (see Figure 6). In addition, an effective algorithm with convergence was established based on ADMM.

To study the theoretical connection between spectral clustering and SSC, Bai and Liang [60] presented a sparse subspace clustering with entropy-norm (SSC+E), which can be written as the following optimization problem:

$$\underset{C,E}{min}\sum _{i=1}^n\sum _{j=1}^n{C}_{ij}ln{C}_{ij}+\lambda {\parallel E\parallel }_F^2\mathrm{s}.\mathrm{t}.X=XC+E,C^T=C,C\ge 0,\mathrm{diag}\left(C\right)=\mathbf{0},$$

(8)

where ${\sum }_{i=1}^n{\sum }_{j=1}^n{C}_{ij}ln{C}_{ij}$ is the information entropy of C which can describe the stability of variable predicted and degree of information involvement. Based on the above settings, SSC+E can obtain the closed-form solution of the coefficient matrix. It has proved that the optimal solution of (8) is equivalent to a Gaussian kernel as the sparse representation, that is, spectral clustering with Gaussian kernel can be viewed as SSC+E (8). For this reason, SSC+E obtains a sparse similarity matrix by using the Gaussian kernel, which can avoid the complex computation caused by SSC.

To cope with the challenges of data noise and high-dimensional expression profiles in identify cell types, Zheng et al. [61] proposed an adaptive sparse subspace clustering method (AdaptiveSSC), which can be formulated as the following optimization problem:

$$\underset{C,E}{min}{∥\frac{C}{W}∥}_1+\lambda {\parallel E\parallel }_F^2\mathrm{s}.\mathrm{t}.X=XC+E,\mathrm{diag}\left(C\right)=\mathbf{0},$$

(9)

where W is the sample-related weight matrix based on Pearson correlation. If Pearson correlation coefficient between ${\mathbf{x}}_i$ and ${\mathbf{x}}_j$ is greater than 0, $W_{ij}=pearson({\mathbf{x}}_i,{\mathbf{x}}_j)$; otherwise, $W_{ij}=0$. Since AdaptiveSSC uses a data-driven adaptive sparse strategy to maintain the local structure of the data, it is more robust to data noise. However, due to the existence of the $l_1$-norm, the computing efficiency of AdaptiveSSC is still low, especially for high-dimensional datasets.

2.2. Low-Rank Representation Based SC

As mentioned earlier, SSC can accurately restore the underlying subspace under certain conditions. However, sparse representation-based methods aim to find a sparse representation of each data individually, which may not accurately capture the global structure. Moreover, sparse representation-based approaches may not be robust to noise and outliers. In this situation, low-rank representation came into being and was successfully applied to SC problems.

Table 3. The comparison of classical low-rank SC methods.

Methods	Year	$\mathit{F}\left(\mathit{C}\right)$	$\mathit{R}\left(\mathit{E}\right)$	$\mathbf{\Omega }$	Theoretical Results
LRR [38,62]	2010	${\parallel C\parallel }_{*}$	${\parallel E\parallel }_{2,1}$	-	BDP
LRR-PSD [63]	2010	${\parallel C\parallel }_{*}$	${\parallel E\parallel }_{2,1}$	$\left\{C\right|C⪰0\}$	-
LSR [64]	2012	${\parallel C\parallel }_F^2$	${\parallel E\parallel }_F^2$	$\left\{C\right|\mathrm{diag}\left(C\right)=\mathbf{0}\}$	BDP, Grouping effect
CASS [65]	2013	${\sum }_j{\parallel X\mathrm{Diag}\left(C_{:,j}\right)\parallel }_{*}$	${\parallel E\parallel }_F^2$	-	BDP, Grouping effect
NLRR [66]	2016	${\parallel C\parallel }_{K_{k,p}}^p(p>0)$	${\parallel E\parallel }_{2,q}^q(0<q<1)$	-	-
$S_q$-LRR [67]	2016	${\parallel C\parallel }_{S_q}^q(0<q<1)$	${\parallel E\parallel }_1$, ${\parallel E\parallel }_F^2$ or ${\parallel E\parallel }_{2,1}$	-	-
LRRSC [68]	2017	${\parallel C\parallel }_{*}$	${\parallel E\parallel }_{2,1}$	$\left\{C\right|C=C^T\}$	-
CAR [69]	2018	${\sum }_i{\sum }_j{\left|C_{ij}\right|}^{p_{ij}}$	${\parallel E\parallel }_F^2$	-	Grouping effect
$S_q$NM-LRR [70]	2019	${\parallel C\parallel }_{S_q}^q(q=1,2/3\mathrm{or}1/2)$	${\parallel E\parallel }_1$, ${\parallel E\parallel }_F^2$ or ${\parallel E\parallel }_{2,1}$	-	-
SSRSC [71]	2021	${\parallel C\parallel }_F^2$	${\parallel E\parallel }_F^2$	$\left\{C\right|C\ge 0,{\mathbf{1}}^{T}C=s{\mathbf{1}}^T\}$	-
WSLog [72]	2022	${\sum }_{i=1}^{r}ln(w{\sigma }_i^p+1)$	${\parallel E\parallel }_{2,1}$	-	-

Remark: - indicates that this item does not exist.

summarizes the classical low-rank SC methods.

In 2010, Liu et al. [38,62] presented a subspace clustering method based on low-rank representation (LRR) to consider the correlation structure of data. Instead of pursuing a sparse representation in SSC, LRR pursues the lowest-rank representation among data which can better capture the global structure of the data. LRR can be formulated as the following optimization problem:

$$\underset{C,E}{min}{\parallel C\parallel }_{*}+\lambda {\parallel E\parallel }_{2,1}\mathrm{s}.\mathrm{t}.X=XC+E,$$

(10)

where the nuclear norm is used to approximate the rank of C, and the $l_{2,1}$ norm encourages the columns of E to be zero which can better fit the data corruptions. Liu et al. [38] have proved that, if the subspaces are independent, the representation coefficient matrix obtained from the LRR model (10) without noise has the block diagonal structure. Note that the problem (10) is a convex optimization, which can be effectively solved by the convex programming algorithms, such as the augmented Lagrange multiplier (ALM) method [128], accelerated proximal gradient (APG) [129], ADMM [117,130], and SSNAL [118]. It is worth pointing out Favaro et al. [131] study given the closed form of LRR via the singular value decomposition (SVD) of the data matrix.

As discussed earlier, after obtaining the optimal solution $(C^{*},E^{*})$, $C^{*}$ is used to construct the affinity matrix for spectral clustering. Since the affinity matrix is symmetric, it is usually constructed by $A=\frac{\left|C\right|+{\left|C\right|}^T}{2}$, which may result in a poor clustering result. To avoid symmetrization post-processing, Ni et al. [63] introduced a symmetric positive semidefinite constraint into LRR (LRR-PSD), which can be formulated as

$$\underset{C,E}{min}{\parallel C\parallel }_{*}+\lambda {\parallel E\parallel }_{2,1}\mathrm{s}.\mathrm{t}.X=XC+E,C⪰0.$$

(11)

To further reduce computational cost in the LRR scheme, Chen et al. [68] proposed a low-rank representation with symmetric constraint (LRRSC) to avoid iterative SVD operation which can significantly decrease the computational cost for the subspace clustering. LRRSC solves the following optimization problem:

$$\underset{C,E}{min}{\parallel C\parallel }_{*}+\lambda {\parallel E\parallel }_{2,1}\mathrm{s}.\mathrm{t}.X=XC+E,C=C^T.$$

(12)

It is worth pointing out that the affinity matrix, constructed by making full use of the intrinsically geometrical structure of the data points in the symmetric low-rank matrix, can significantly improve the performance of SC.

For LRR, each iteration must perform the singular value decomposition (SVD) due to the nuclear norm, which leads to high computational complexity and dramatically limits its ability to handle high-dimensional data. To handle this problem, Lu et al. [64] proposed a subspace clustering method via least squares regression (LSR), which was presented by exploiting the correlation of data, which solves the following optimization problem:

$$\underset{C,E}{min}{\parallel C\parallel }_F^2+\lambda {\parallel E\parallel }_F^2\mathrm{s}.\mathrm{t}.X=XC+E,\mathrm{diag}\left(C\right)=\mathbf{0}.$$

(13)

It has proved that the optimal solution of LSR satisfies the block diagonal property (1) without noise and with subspace independence. Furthermore, LSR encourages a grouping effect that tends to shrink coefficients of correlated data and group them together. LSR has also been proven to be robust to bounded disturbance. In particular, Zhang et al. [132] proved theoretically that the Frobenius norm can be used as a surrogate of the rank function, and related results can also be found in [133,134]. It is easy to verify that LSR (13) is a strongly convex optimization problem and thus has a closed-form solution. Therefore, compared with the nuclear norm, using the Frobenius norm as a surrogate of the rank function can significantly reduce the computational complexity.

Although LSR has the grouping effect, it cannot produce a sparse solution, which makes the optimal solution obtained by LSR lack interpretability. Lu et al. [65] proposed a correlation adaptive subspace segmentation (CASS) method by using trace Lasso, which can be formulated as

$$\underset{C,E}{min}\sum _j\parallel X\mathrm{Diag}\left(C_{:,j}\right){\parallel }_{*}+\lambda {\parallel E\parallel }_F^2\mathrm{s}.\mathrm{t}.X=XC+E.$$

(14)

CASS is a data correlation-dependent method which simultaneously performs automatic data selection and groups correlated data together. Consequently, CASS can be considered as the adaptive balance between SSC and LSR. Lu et al. generalized the EBD conditions and proved that the solution obtained by (14) has the block diagonal structure (1) if the underlying subspaces are independent. Furthermore, CASS has been proven to have the grouping effect, i.e., the coefficients of a group of correlated data are approximately equal.

Wang et al. [69] presented a correlation adaptive regression (CAR) subspace clustering method, which can be formulated as the following optimization problem:

$$\underset{C,E}{min}\sum _i\sum _j|C_{ij}{|}^{p_{ij}}+\lambda {\parallel E\parallel }_F^2\mathrm{s}.\mathrm{t}.X=XC+E,$$

(15)

where $p_{ij}=1+r_{i,j}$, with $r_{i,j}=\left|{\mathbf{x}}_i^T{\mathbf{x}}_j\right|$ being the correlation between data points ${\mathbf{x}}_i$ and ${\mathbf{x}}_j$. CAR can be seen as a trade-off between $l_1$-norm and Frobenius-norm, that is, $l_1$-norm penalty is imposed on the low correlated data while the Frobenius-norm penalty is imposed on the highly correlated data. Therefore, the CAR model (15) can ensure that the affinity matrix satisfies the inter-subspace sparsity and intracluster uniformity.

To further reveal the inherent correlations among data points, Xu et al. [71] proposed a scaled simplex representation for subspace clustering (SSRSC), which aims to solve the following optimization problem:

$$\underset{C}{min}{\parallel C\parallel }_F^2+\lambda {\parallel E\parallel }_F^2\mathrm{s}.\mathrm{t}.X=XC+E,C\ge 0,{\mathbf{1}}^{T}C=s{\mathbf{1}}^T,$$

(16)

where $0<s<1$ is a scalar. Here, the constraint condition $C\ge 0$ is conducive to encouraging data points from the same subspace to represent each other while suppressing the data from different subspaces to represent each other. Thus, the obtained coefficient matrix C is distinctive. The constraint ${\mathbf{1}}^{T}C=s{\mathbf{1}}^T$ can limit the sum of each coefficient vector $C_i$ to be s, which makes the representation more discriminative since C should be non-negative. Note that the SSRSC model (16) is a convex optimization problem with two equality constraints and one inequality constraint. Thus, it can be solved by ADMM [117], SSNAL [118], and so forth.

Although many achievements have been made on the convex nuclear norm, which is a surrogate of the rank function, the nuclear norm minimization may over-penalize large singular values, resulting in a bias. Hence, a solution obtained by the nuclear norm may be suboptimal since it is not a perfect approximation of the matrix rank. In order to achieve a better approximation for the rank function, many nonconvex surrogate functions of rank have been introduced into the SC problems. Jiang et al. [66] proposed a robust subspace segmentation via nonconvex LRR (NLRR) by replacing the nuclear norm with the Ky Fan p-k norm ($0<p,k\in \{1,\dots ,d\}$) and penalizing the noise via the $l_{2,q}$ norm ($0<q<1$) in LRR (10). A proximal iteratively reweighted optimization algorithm (PIRA) is designed to solve the NLRR. Zhang [67] introduced the nonconvex Schatten-q ($0<q<1$) regularization to approximate the rank function for the SC problem, called $S_q$-LRR, which can be solved by linearized alternating direction method with the adaptive penalty (LADMAP) [130] or the generalized iterated shrinkage (GMST) algorithm [135]. Zhang [70] proposed a Schatten-q norm minimization-based LRR ($S_p$NM-LRR) with $q=1,2/3$ and $1/2$, which can be solved by ADMM [136]. Most recently, Shen et al. [72] presented a non-convex low-rank approximation subspace clustering method based on a weighted Schatten-p norm jointed logarithmic constraint (WSLog). By using the logarithmic function to further tighten Schatten-p norm, WSLog can better recover the low rank of data. In addition, WSLog is more robust to noise due to the data-related weight matrix.

2.3. Low-Rank Sparse Representation Based SC

As analyzed above, LRR can capture global information and successfully recover the corrupted data drawn from independent subspaces. However, LRR often fails in the case of disjoint subspaces or overlapping subspaces. SSC is superior to LRR in this respect. However, it is weaker than LRR in capturing the global structure of subspaces from the corrupted data. Therefore, the natural idea is to combine the low-rank representation with sparse representation.

Table 4. The comparison of classical low-rank sparse SC methods.

Methods	Year	$\mathit{F}\left(\mathit{C}\right)$	$\mathit{R}\left(\mathit{E}\right)$	$\mathbf{\Omega }$	Theoretical Results
MSR [73]	2011	${\parallel C\parallel }_{*}+\mu {\parallel C\parallel }_1$	${\parallel E\parallel }_{2,1}$	$\left\{C\right|{\mathbf{1}}^{T}C={\mathbf{1}}^T\}$	BDP
NNLRS [74]	2012	${\parallel C\parallel }_{*}+\mu {\parallel C\parallel }_1$	${\parallel E\parallel }_{2,1}$	$\left\{C\right|C\ge 0$ }	-
LRSSC [75]	2013	${\parallel C\parallel }_{*}+\mu {\parallel C\parallel }_1$	${\parallel E\parallel }_F^2$	$\left\{C\right|\mathrm{diag}\left(C\right)=\mathbf{0}\}$	BDP
LRRLC [76]	2013	${\parallel C\parallel }_{*}+\mu {\sum }_{i,j}\parallel {\mathbf{x}}_i-{\mathbf{x}}_j{\parallel }_2^2\left|C_{ij}\right|$	${\parallel E\parallel }_{2,1}$	-	-
LRSR [77]	2016	${\parallel Y\parallel }_{*}+{\parallel C\parallel }_{*}+\mu {\parallel J\parallel }_1$	${\parallel E\parallel }_{2,1}$	$\left\{C\right|C=J-\mathrm{Diag}\left(\mathrm{diag}\right(J\left)\right)\}$	-
EnSC [78]	2016	${\mu \parallel C\parallel }_1+\frac{1-\mu }{2}{\parallel C\parallel }_F^2$	${\parallel E\parallel }_F^2$	$X=XC+E,\mathrm{diag}\left(C\right)=\mathbf{0}$	BDP
CAWR [79]	2017	$\parallel (\mathbf{1}{\mathbf{1}}^T-U){\odot C\parallel }_1+\frac{\mu }{2}{\parallel \sqrt{U}\odot C\parallel }_F^2$	${\parallel E\parallel }_F^2$	$X=XC+E$	BDP, Grouping effect
GMC-LRSSC [80]	2020	${\psi }_B\left(\sigma \left(C\right)\right)+\mu {\psi }_B\left(C\right)$	${\parallel E\parallel }_F^2$	$\left\{C\right|\mathrm{diag}\left(C\right)=\mathbf{0}\}$	-
$S_0/l_0$-LRSSC [80]	2020	${\parallel C\parallel }_{S_0}+\mu {\parallel C\parallel }_0$	${\parallel E\parallel }_F^2$	$\left\{C\right|\mathrm{diag}\left(C\right)=\mathbf{0}\}$	-
SCAT [81]	2020	${\sum }_{i,j}\parallel {\mathbf{x}}_i-{\mathbf{x}}_j{\parallel }_2^2{C}_{ij}+\frac{\mu }{2}{\parallel C\parallel }_F^2$	$\Psi \left(E\right)$	$\left\{C\right|C\ge 0,\mathrm{diag}\left(C\right)=\mathbf{0},C^T\mathbf{1}=\mathbf{1}\}$	-
NL-SSLR [82]	2021	${\parallel C\parallel }_{*}{+\mu \parallel W\odot C\parallel }_1+\frac{\alpha }{2}{\parallel C-{\overline{C}}_{NL}\parallel }_F^2$	${\parallel E\parallel }_F^2$	-	-
KSLSR [83]	2022	${\parallel V\odot W\parallel }_F^2$	${\parallel E\parallel }_F^2$	$\left\{V\right|\mathrm{diag}\left(V\right)=\mathbf{0},V^T\mathbf{1}\le k\mathbf{1}\}$	Grouping effect

Remark: - indicates that this item does not exist.

summarizes the classical low-rank sparse SC methods.

In 2011, Luo et al. [73] proposed a multiple subspace representation (MSR) by combining the sparsity and low-rank of the representation coefficient matrix, which can be formulated as the following optimization problem:

$$\underset{C,E}{min}{\parallel C\parallel }_{*}+{\mu \parallel C\parallel }_1+\lambda {\parallel E\parallel }_{2,1}\mathrm{s}.\mathrm{t}.X=XC+E,{\mathbf{1}}^{T}C={\mathbf{1}}^T,$$

(17)

where ${\mathbf{1}}^{T}C={\mathbf{1}}^T$ is used to pursue the linear/affine representation of subspaces. Note that the SSC model (3) and LRR model (10) can be regarded as the special case of (5) with $\mu =\infty$ and $\mu =0$, respectively, and the $l_{2,1}$ norm is more robust against data corruption than the usual Frobenius norm. It has proved that the representation coefficient matrix obtained from the MSR model (17) without noise has the block diagonal structure (1) when the underlying affine subspaces are independent.

A common problem with existing methods is that negative coefficients may be generated, which may cause data to “cancel each other” through complex addition and subtraction [137]. This may result in the potential structure between data not being able to be accurately captured. To deal with this issue, Zhuang et al. [74] proposed a non-negative low-rank and sparse (NNLRS) model by adding a non-negative constraint to the MSR model (17). NNLRS solves the following optimization problem:

$$\underset{C,E}{min}{\parallel C\parallel }_{*}+{\mu \parallel C\parallel }_1+\lambda {\parallel E\parallel }_{2,1}\mathrm{s}.\mathrm{t}.X=XC+E,C\ge 0,$$

(18)

where $C\ge 0$ is used to enforce the representation to be non-negative so that the obtained coefficients can be directly used as the graph weights. The non-negativity could potentially enhance the discriminability of the coefficients such that the data points are more likely reconstructed by the data points from the same subspace.

Wang et al. [75] proposed a low-rank sparse subspace clustering (LRSSC) by directly combining SSC (3) and LRR (10). LRSSC solves the following optimization problem:

$$\underset{C,E}{min}{∥C∥}_{*}+\mu {∥C∥}_1+\lambda {\parallel E\parallel }_F^2\mathrm{s}.\mathrm{t}.X=XC+E,\mathrm{diag}\left(C\right)=\mathbf{0}.$$

(19)

It has proved that the representation coefficient matrices obtained from the LRSSC (19) and the LRSSC with random data have the block diagonal structure (1) when the underlying affine subspaces are independent.

Zheng et al. [76] present a low-rank representation method with local constraint (LRRLC) by combining LRR with local information of data, which can be formulated as

$$\underset{C,E}{min}{\parallel C\parallel }_{*}+\mu \sum _{i,j}\parallel {\mathbf{x}}_i-{\mathbf{x}}_j{\parallel }_2^2|C_{ij}{|+\lambda \parallel E\parallel }_{2,1}\mathrm{s}.\mathrm{t}.X=XC+E,$$

(20)

where ${\sum }_{i,j}\parallel {\mathbf{x}}_i-{\mathbf{x}}_j{\parallel }_2^2\left|C_{ij}\right|$ is used to measure the data correlation of the representation coefficient C, which changes with the distance between samples. Note that LRRLC is able to capture both the global structure by the nuclear norm and the local structure by the locally constrained regularization term simultaneously.

Wang et al. [77] proposed a low-rank subspace sparse representation (LRSR) method, which can be formulated as

$$\underset{Y,C,J,E}{min}{\parallel Y\parallel }_{*}+{\parallel C\parallel }_{*}+{\mu \parallel J\parallel }_1+\lambda {\parallel E\parallel }_{2,1}\mathrm{s}.\mathrm{t}.X=YC+E,C=J-\mathrm{Diag}\left(\mathrm{diag}\left(J\right)\right).$$

(21)

Unlike SSC-based and LRR-based models, LRSR (21) aims to recover both the clean dictionary Y and the representation coefficient C simultaneously. It has proved that LRSR can not only recover the low-rank subspaces, but also obtain a relatively sparse segmentation for the disjoint subspaces or even overlapping subspaces.

As mentioned earlier, the $l_1$-norm regularization can be guaranteed to give a subspace-preserving affinity (i.e., there are no connections between points from different subspaces) under broad conditions. However, the clusters may not be connected (since SSC tends to select only one randomly for a group of highly correlated data points), while Frobenius norm and nuclear norm regularization often improve connectivity, but give a subspace-preserving affinity only for independent subspaces. You et al. [78] proposed a subspace clustering via elastic net regularizer (EnSC), which can be formulated as

$$\underset{C,E}{min}{\mu \parallel C\parallel }_1+\frac{1-\mu }{2}{\parallel C\parallel }_F^2+\lambda {\parallel E\parallel }_F^2\mathrm{s}.\mathrm{t}.X=XC+E,\mathrm{diag}\left(C\right)=\mathbf{0},$$

(22)

where $\mu \in [0,1]$. Note that EnSC (22) can be regarded as a combination of SSC and LSR, and will reduce to each of them when $\mu =1$ and $\mu =0$, respectively. It has drawn the theoretical conditions for the subspace preserving property free of noise and with independent subspaces, that is, if the parameter is in a certain range, the optimal solution has a block diagonal structure. It is easy to verify that EnSC (22) is a convex optimization problem that can be solved by the active-set method [78], ADMM [117], SSNAL [118], and so on.

For the image segmentation problem, Wang and Wu [79] proposed a correlation adaptive weighted regression (CAWR) subspace clustering method via combining the correlation weighted $l_1$-norm and $l_2$-norm

$$\underset{C,E}{min}\parallel (\mathbf{1}{\mathbf{1}}^T-U){\odot C\parallel }_1+\frac{\mu }{2}\parallel \sqrt{U}{\odot C\parallel }_F^2+\lambda {\parallel E\parallel }_F^2\mathrm{s}.\mathrm{t}.X=XC+E,$$

(23)

where $U=X^{T}X$ is the correlation matrix of the data. It can be seen that CAWR introduces the correlation adaptive weight to each coefficient in the $l_1$-norm and $l_2$-norm. It can be seen that CAWR introduces the correlation adaptive weight for both $l_1$ and $l_2$ regularization, which can make better use of data information. Many existing models can be regarded as special cases of CAWR, such as SSC [119], LLR [38], and EnSC [78]. It has proved that CAWR has the ability of subspace selection (by $l_1$ regularization) for uncorrelated data and the grouping effect (by $l_2$ regularization) for highly correlated data. In addition, the optimal solution obtained by CAWR is proved to have the block diagonal structure under noiseless data and independent subspace. However, CAWR lacks robustness to data corruption [91].

Instead of using the convex approximations of rank and $l_0$ pseudo norm, Brbić and Kopriva [80] introduced two $S_0/l_0$ pseudo norm-based nonconvex regularizations for LRSSC. The first method is based on the multivariate generalization of the minimax-concave (GMC) penalty function, which can be formulated as

$$\underset{C,E}{min}{\psi }_B\left(\sigma \left(C\right)\right)+\mu {\psi }_B\left(C\right)+\lambda {\parallel E\parallel }_F^2\mathrm{s}.\mathrm{t}.X=XC+E,\mathrm{diag}\left(C\right)=\mathbf{0}.$$

(24)

where $B\in {\mathbb{R}}^{n\times n}$ is a given matrix, and the GMC penalty is defined by ${\psi }_B\left(\mathbf{x}\right)={\parallel \mathbf{x}\parallel }_1-S_B\left(\mathbf{x}\right)$ with $S_B\left(\mathbf{x}\right)={inf}_{\mathbf{v}\in {\mathbb{R}}^n}\left\{{\parallel \mathbf{v}\parallel }_1+\frac{1}{2}{\parallel B(\mathbf{x}-\mathbf{v})\parallel }_2^2\right\}$. The GMC-LRSSC can maintain the convexity of low-rank and sparsity-constrained subproblems and achieve better approximation for rank and sparsity than nuclear and $l_1$-norms. To further approximate the low-rank and sparsity, the Schatten-0 and $l_0$ pseudo norm regularizations are introduced to LRSSC ($S_0/l_0$-LRSSC), which is

$$\underset{C,E}{min}{\parallel C\parallel }_{S_0}+{\mu \parallel C\parallel }_0+\lambda {\parallel E\parallel }_F^2\mathrm{s}.\mathrm{t}.X=XC+E,\mathrm{diag}\left(C\right)=\mathbf{0}.$$

(25)

It is worth pointing out that simultaneous rank and sparsity regularization can be handled by the proximal average method [138,139], which approximates the proximal map of the joint solution by averaging the solutions obtained separately from low-rank and sparsity subproblems. Although GMC-LRSSC (24) and $S_0/l_0$-LRSSC (25) are nonconvex optimization problems, they can be solved by an improved ADMM [140]. Moreover, the convergence of the two algorithms was established by Brbić and Kopriva [80].

To further reflect the similarities between data points, Zhong and Pun [81] proposed a subspace clustering method with an adaptive transformation matrix (SCAT)

$$\begin{array}{cc}\hfill \underset{C,E}{min}& \sum _{i,j}\parallel {\mathbf{x}}_i-{\mathbf{x}}_j{\parallel }_2^2{C}_{ij}+\frac{\mu }{2}{\parallel C\parallel }_F^2+\lambda \Psi \left(E\right)\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& X=XC+E,C\ge 0,\mathrm{diag}\left(C\right)=\mathbf{0},C^T\mathbf{1}=\mathbf{1},\hfill \end{array}$$

(26)

where $\parallel {\mathbf{x}}_i-{\mathbf{x}}_j{\parallel }_2^2{C}_{ij}$ is used to capture the local structure of data, and the affine constraint $C^T\mathbf{1}=\mathbf{1}$ is used to deal with affine subspaces. $\Psi \left(E\right)$ is a function of E which depends on the distribution of data errors. For instance ${\parallel E\parallel }_F^2$ is used for Gaussian residuals while ${\parallel E\parallel }_1$ for Laplacian. Note that SCAT can be regarded as imposing data similarity information into LSR [64]. Hence, SCAT can simultaneously capture the global (due to the LSR) and the local structure (due to the similarity learning) of data.

Most recently, Zhai et al. [82] proposed a novel scalable nonlocal means regularized sketched reweighted sparse and low-rank (NL-SSLR) SC method for the large hyperspectral images (HSIs) clustering. NL-SSLR aims to explore both local and global structural information simultaneously, which solves the following optimization problem:

$$\underset{C,E}{min}{\parallel C\parallel }_{*}{+\mu \parallel W\odot C\parallel }_1+\frac{\alpha }{2}\parallel C-{\overline{C}}_{NL}{\parallel }_F^2+\lambda {\parallel E\parallel }_F^2\mathrm{s}.\mathrm{t}.X=XC+E,$$

(27)

where ${\overline{C}}_{NL}$ is a nonlocal coefficient matrix obtained by a nonlocal means filter, and W is a weight matrix defined as $W_{ij}=\frac{a}{C_{ij}+b}$ with two small constants a and b. Note that W obtained in this way can better promote sparsity by punishing larger elements, and the two constants a and b are used to avoid overweight, which can be taken as 0.001 in practice. It should be pointed out that the nuclear norm is used to explore the global structure of data, while the reweighted $l_1$-norm regularization and the nonlocal means filter are utilized to explore the spatial correlation information of data fully.

Since LSR [64] uses Frobenius norm regularization, it cannot obtain sparse solution, which may make it sensitive to data corruptions and data dimensions. To handle this issue, Yang et al. [83] proposed the following k-sparse least squares regression (KSLSR) subspace clustering method via 0-1 integer programming

$$\underset{V,W,E}{min}{\parallel V\odot W\parallel }_F^2+\lambda {\parallel E\parallel }_F^2\mathrm{s}.\mathrm{t}.X=X(V\odot W)+E,\mathrm{diag}\left(V\right)=\mathbf{0},V^T\mathbf{1}\le k\mathbf{1},$$

(28)

where $W\in {\mathbb{R}}^{n\times n}$ is the weight value. $V\in {\{0,1\}}^{n\times n}$ is the sparse selection factor, where $V_{ij}=1$ represents that the data ${\mathbf{x}}_j$ are selected for the linear representation of the data ${\mathbf{x}}_i$, and $V_{ij}=0$ indicates that the data ${\mathbf{x}}_j$ are not selected. ${\mathbf{v}}_i^T\mathbf{1}\le k$ indicates that the data ${\mathbf{x}}_i$ can be linearly represented by at most k data points which can be regarded as a transformation of $l_0$ pseudo norm. Once V and w are obtained, the coefficient matrix C can be obtained immediately by $C=V\odot W$. It has proved that the KSLSR model (28) has the k-sparsity and grouping effect, thus KSLSR is robust to data corruptions and dimension of data. However, due to the 0-1 constraint on V, KSLSR is an NP-hard combinatorial optimization problem, which is usually difficult to solve directly [141]. To effectively solve KSLSR, problem (28) was transformed into a continuous optimization problem by equivalently replacing the 0-1 constraint on V with a $l_p$-box intersection [142], i.e., the intersection of a shifted $l_p$-sphere and a solid cube. Figure 7 shows the geometric illustration of the $l_p$-box. Then, the KSLSR with continuous constraints can be solved by ADMM.

3. Local Structure Preserving SC

3.1. Graph-Regularized Based SC

In the previous section, we review some classical low-rank sparse SC methods, such as SSC, LRR, and MSR. Although these methods have reached state-of-the-art performance, they may lose the local structure of data. In other words, the representation coefficients for locally similar features or samples may differ greatly. This may damage the connection of the similarity graph, and ultimately affect the clustering performance. To deal with this issue, a natural idea is to introduce the graph information of the incidence matrix into the subspace representation.

Table 5. The comparison of classical graph-regularized SC methods.

Methods	Year	$\mathit{F}\left(\mathit{C}\right)$	$\mathit{R}\left(\mathit{E}\right)$	Constraint
GLRR [84]	2013	${\parallel C\parallel }_{*}+\mu \mathrm{tr}\left(C^{T}LC\right)$	${\parallel E\parallel }_{2,1}$	$X=XC+E$
LapLRR [85]	2014	${\parallel C\parallel }_{*}+\mu \mathrm{tr}\left(C^{T}LC\right)$	${\parallel E\parallel }_F^2$	$X=XC+E,C\ge 0$
E-SSC [86]	2016	${\parallel C\parallel }_1+\frac{\mu }{2}\mathrm{tr}\left(C^{T}LC\right)$	${\parallel E\parallel }_F^2$	$X=XC+E,\mathrm{diag}\left(C\right)=\mathbf{0},C^T\mathbf{1}=\mathbf{1}$
NSHLRR [87]	2016	${\parallel C\parallel }_{*}+{\mu }_1{\parallel C\parallel }_1+{\mu }_2\mathrm{tr}\left(C{L}_h{C}^T\right)$	${\parallel E\parallel }_1$	$X=XC+E,C\ge 0$
GCLRR [88]	2017	${\parallel C\parallel }_{*}+\mu \mathrm{tr}\left(CL{C}^T\right)$	${\parallel E\parallel }_{2,1}$	$X=XVC+E,V^{T}V=I$
MLLRR [89]	2019	${\parallel C\parallel }_{*}+{\mu }_1{\parallel C\parallel }_1+{\mu }_2\mathrm{tr}\left(CL{C}^T\right)$	${\parallel E\parallel }_{2,1}$	$X=XC+E,C\ge 0$
SGL [90]	2022	${\parallel C\parallel }_F^2+\mu \mathrm{tr}\left(F^{T}LF\right)$	${\parallel E\parallel }_F^2$	$X=XC+E,C\ge 0,C\mathbf{1}=\mathbf{1},F^{T}F=I$
GL$l_{1/2}$RSC [91]	2022	${\parallel C\parallel }_{1/2}^{1/2}+{\mu }_1{\parallel L\odot C\parallel }_{1/2}^{1/2}+{\mu }_2{\parallel P\odot C\parallel }_{1/2}^{1/2}+{\mu }_3{\parallel R\odot C\parallel }_{1/2}^{1/2}$	${\parallel E\parallel }_F^2$	$X=XC+E,C\ge 0$

summarizes the classical graph-regularized SC methods.

To make full use of the correlation of different hyperspectral images and consider their local manifold structure, Lu et al. [84] proposed a graph-regularized low-rank representation (GLRR) subspace clustering method, which can be written as

$$\underset{C,E}{min}{\parallel C\parallel }_{*}+\mu \mathrm{tr}\left(CL{C}^T\right)+\lambda {\parallel E\parallel }_{2,1}\mathrm{s}.\mathrm{t}.X=XC+E,$$

(29)

where $L=D-W$ is the Laplace matrix of X, and W and D are the incidence matrix and symmetric degree matrix of X, respectively; $\mathrm{tr}\left(CL{C}^T\right)$ is called graph regularization, which can be used to maintain the similar local structures of data [143]. GLRR (29) can be regarded as a generalization of the LRR [38] by considering the local geometrical structure of the data. Hence, GLRR can capture both the global structure (due to the LRR framework) and local structure (due to the graph regularization) of the data. It is easy to verify that the optimization problem (29) is convex and thus can be solved by ALM [128], ADMM [117,130], and other convex optimization algorithms. To further improve the GLRR, Liu et al. [85] proposed a Laplacian regularized LRR (LapLRR) method, which can be regarded as a generalization of GLRR (29) by adding a nonnegative constraint. Chen et al. [86] also introduced the Laplace regularization into SSC [52] and LRR [38], and proposed the enhanced sparse subspace clustering method (E-SSC) and enhanced LRR method (E-LRR).

Yin et al. [87] proposed a non-negative sparse hyper-Laplacian regularized low-rank representation method (NSHLRR), which can be formulated as

$$\underset{C,E}{min}{\parallel C\parallel }_{*}+{\mu }_1{\parallel C\parallel }_1+{\mu }_2\mathrm{tr}\left(C{L}_h{C}^T\right)+\lambda {\parallel E\parallel }_1\mathrm{s}.\mathrm{t}.X=XC+E,C\ge 0,$$

(30)

where $L_h$ is the hyper-Laplacian matrix defined in [144]. The Laplacian and the sparsity constraints encourage choosing nearby samples (which most likely belong to the same cluster), rather than the faraway samples (which may belong to other clusters), to represent the sample at the center of the locally linear manifold. Hence, the optimal solution obtained by NSHLRR (30) approximately satisfies the block diagonal structure. By combining graph hyper-Laplacian regularizer with LRR, the NSHLRR can simultaneously capture the global structure and inherent nonlinear geometric information of data. Note that NLSRR can be regarded as an extension of LRRGL [145] by replacing the traditional Laplacian matrix with a hypergraph Laplacian matrix. To improve the robustness of data corruptions, Wang et al. [89] imposed the $l_{1,2}$ regularization to the data corruptions instead of $l_1$ regularization and applied it to tumor clustering.

Du et al. [88] proposed a graph regularized Compact LRR (GCLRR) method by combining dictionary learning with low-rank representation and introducing graph regularization. The GCLRR can be formulated as

$$\underset{C,V,E}{min}{\parallel C\parallel }_{*}+\mu \mathrm{tr}\left(CL{C}^T\right)+\lambda {\parallel E\parallel }_{2,1}\mathrm{s}.\mathrm{t}.X=XVC+E,V^{T}V=I,$$

(31)

where $V^{T}V=I$ is used to eliminate the uncertainty caused by the diagonal structure. Instead of using the original data X as a dictionary, the GCLRR used the linear combination of X as the dictionary, i.e., $A=XV$. Thus, the proposed method can realize dictionary learning and low-rank representation simultaneously. In addition, unlike previous SC methods which treat the affinity matrix construction and clustering algorithm separately, GCLRR integrates these two tasks into a single optimization framework to ensure overall optimization. It is worth pointing out that the GCLRR model (31) can capture both the global subspace structure (by the low-rank representation) and the local geometrical structure (by graph regularization).

Although the above graph-regularized based SC methods have shown good performance, these methods are often inefficient and limited in exploring the potential information of data. To deal with these problems, Kang et al. [90] proposed a scalable graph learning (SGL) subspace clustering method which can be written as

$$\underset{C,F,E}{min}{\parallel C\parallel }_F^2+\mu \mathrm{tr}\left(F^{T}LF\right)+\lambda {\parallel E\parallel }_F^2\mathrm{s}.\mathrm{t}.X=XC+E,C\ge 0,C\mathbf{1}=\mathbf{1},F^{T}F=I.$$

(32)

Since SGL used the Frobenius norm regularization, it avoided the singular value decomposition, which can increase the computational efficiency. With the help of the graph Laplace information, SGL can capture the local information of the data and handle new data. In addition, the connection between SGL and the k-means clustering was established.

To deal with the challenges brought by diverse visual patterns, noise, and complex background in image processing, Francis et al. [91] proposed the following subspace clustering method by integrating the $l_{1/2}$-norm, $l_2$-norm and graph Laplacian regularization (GL$l_{1/2}$RSC). The proposed GL$l_{1/2}$RSC can be formulated as

$$\begin{array}{cc}\hfill \underset{C,E}{min}& {\parallel C\parallel }_{1/2}^{1/2}+{\mu }_1{\parallel L\odot C\parallel }_{1/2}^{1/2}+{\mu }_2{\parallel P\odot C\parallel }_{1/2}^{1/2}+{\mu }_3{\parallel R\odot C\parallel }_{1/2}^{1/2}+\lambda {\parallel E\parallel }_F^2\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& X=XC+E,C\ge 0,\hfill \end{array}$$

(33)

where P and R can describe the correlation of data, which are defined as $P=\mathbf{1}{\mathbf{1}}^T-U$ and $R=\sqrt{U}$ with $U=X^{T}X$. $C\ge 0$ is used to promote better learning of the local geometrical structure and can deliver more physical interpretation to the data points. Based on the above design, GL$l_{1/2}$RSC has the capabilities of $l_{1/2}$-norm, $l_2$-norm, which can achieve the subspace selection and subspace grouping simultaneously. Due to the existence of $l_{1/2}$-norm, the obtained representation coefficient matrix C by GL$l_{1/2}$RSC (33) is sparse and has an accurate block diagonal structure. Moreover, GL$l_{1/2}$RSC can retain the local structure of data and is relatively insensitive to noise and outliers.

3.2. Latent Representation Based SC

The previously mentioned methods generally use the observed data itself as the dictionary. However, when the observed date is insufficient or corrupted by noise, the performance of these methods may be inferior. To solve this issue, latent representation SC methods were presented by joint low-rank recovery and salient feature extraction.

Table 6. The comparison of classical latent representation SC methods.

Methods	Year	$\mathit{F}\left(\mathit{C}\right)$	$\mathit{R}\left(\mathit{E}\right)$	Constraint
LatLRR [92]	2011	${\parallel C\parallel }_{*}+{\parallel H\parallel }_{*}$	${\parallel E\parallel }_1$	$X=XC+HX+E$
rLRR [93]	2014	${\parallel C\parallel }_{*}+{\parallel H\parallel }_{*}+\frac{\mu }{2}\left[\mathrm{tr}\left(C^{T}LC\right)+\mathrm{tr}\left(HXLC{X}^T{H}^T\right)\right]$	${\parallel E\parallel }_{2,1}$	$X=XC+HX+E,C\ge 0$
FLRR [94]	2018	${\parallel C\parallel }_F^2+{\parallel H\parallel }_F^2$	${\parallel E\parallel }_1$	$X=XC+HX+E$
AS-LRC [95]	2019	${\parallel C\parallel }_{*}+{\parallel H\parallel }_{2,1}+{\mu }_1{\parallel (\mathbf{1}{\mathbf{1}}^T-R)\odot C\parallel }_1+{\mu }_2\wp (H,R)$	${\parallel E\parallel }_1$	$X=XC+HX+E$
FLLRSC [96]	2020	${\parallel C\parallel }_F^2+{\parallel H\parallel }_F^2$	${\parallel E\parallel }_{2,1}$	$\tilde{X}=\tilde{X}C+H\tilde{X}+E$
eLatLRR [97]	2021	${\parallel C\parallel }_{*}+{\parallel H\parallel }_{*}+\frac{\mu }{2}{\parallel HX-HXC\parallel }_F^2$	${\parallel E\parallel }_1$	$X=XC+HX+E$
LLRRWD [98]	2022	${\mu }_1\left({\parallel C\parallel }_F^2+{\parallel H\parallel }_F^2\right)+{\mu }_2\mathrm{tr}\left({(P\odot D)}^{T}C\right)$	${\parallel E\parallel }_1$	$X=XC+HX+E,C\ge 0,C_{i,i}=0,C\mathbf{1}=\mathbf{1}$

summarizes the classical latent representation SC methods.

To cope with the impact of insufficient data samples, Liu and Yan [92] proposed a Latent low-rank representation (LatLRR) on the basis of LRR, which imposed low-rank constraints on the column representation coefficient matrix and row representation coefficient matrix at the same time. LatLRR can be formulated as

$$\underset{C,H,E}{min}{\parallel C\parallel }_{*}+{\parallel H\parallel }_{*}+\lambda {\parallel E\parallel }_1\mathrm{s}.\mathrm{t}.X=XC+HX+E,$$

(34)

where $H\in {\mathbb{R}}^{d\times d}$ is used to reflect the information of hidden data. Notice that there is no parameter between C and H in (34) as the strengths of these two parts are automatically balanced. LatLRR aims to use row data information to make up for the corruption of column data information. It has proved that LatLRR (34) can not only deal with insufficient data samples but also robustly extract salient features from corrupted data. However, because the nuclear norm is used as the penalty function, the singular value decomposition (SVD) must be performed twice in each iteration, which leads to high computational complexity and greatly limits its ability to handle high-dimensional data. In addition, Zhang et al. [146] proved that the solution of the noiseless LatLRR (34) is not unique and gave a closed-form solution.

Although the LatLRR method [92] can handle the situation of insufficient data, they may be inaccurate in identifying the underlying subspaces due to the noise, so they may fail to preserve the local structure of data. To further capture the local structure of data, Zhang et al. [93] proposed a regularized low-rank representation (rLRR) method via introducing a data-dependent Laplacian regularization into LatLRR, which can be formulated as

$$\begin{array}{cc}\hfill \underset{C,H,E}{min}& {\parallel C\parallel }_{*}+{\parallel H\parallel }_{*}+\frac{\mu }{2}\left[\mathrm{tr}\left(C^{T}LC\right)+\mathrm{tr}\left(HXLC{X}^T{H}^T\right)\right]+\lambda {\parallel E\parallel }_{2,1}\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& X=XC+HX+E,Z\ge 0,\hfill \end{array}$$

(35)

where $L=D-W$ is the Laplace matrix of X, and W and D are the incidence matrix and symmetric degree matrix of X, respectively. The Laplacian regularization $\mathrm{tr}\left(C^{T}LC\right)$ and $\mathrm{tr}\left(HXLC{X}^T{H}^T\right)$ are used to preserve the locality and similarity of the observed data and salient data, respectively. It should be noted that the idea of Laplacian regularization has been widely used in machine learning (see, e.g., [147,148,149,150]). Note that LatLRR [92] can be regarded as a special case of rLRR with $\mu =0$. By using Laplace regularization to improve LatLRR, rLRR can better preserve the local data information. Hence, rLRR can produce more discriminative low-rank coefficients and robust representations.

Inspired by LSR [64], Yu and Wu [94] proposed to replace the nuclear norm with the Frobenius norm. The optimization problem of the Frobenius norm based low-rank representation (FLRR) method is as follows:

$$\underset{C,H,E}{min}{\parallel C\parallel }_F^2+{\parallel H\parallel }_F^2+\lambda {\parallel E\parallel }_1\mathrm{s}.\mathrm{t}.X=XC+HX+E.$$

(36)

It is easy to obtain a closed-form solution of FLRR (36) by virtue of the properties of the Frobenius norm. Consequently, compared with LatLRR (34), the computational complexity of FLRR can be greatly reduced. In addition, it is proved theoretically that the Frobenius norm can be used as a convex surrogate for the rank function. Following [64], FLRR (36) also has block diagonal property and grouping effect, that is, FLRR tends to assign approximately equal coefficients to a set of highly correlated data and group them together. Although FLLRR has achieved a better clustering performance, it is sensitive to outliers and noise due to the Frobenius norm.

To recover the underlying subspaces more accurately, Zhang et al. [95] proposed an adaptive structure-constrained low-rank coding (AS-LRC) method by combining latent representation of data with automatic weighting learning

$$\begin{array}{cc}\hfill \underset{C,H,R,E}{min}& {\parallel C\parallel }_{*}+{\parallel H\parallel }_{2,1}+{\mu }_1\parallel (\mathbf{1}{\mathbf{1}}^T-R){\odot C\parallel }_1+{\mu }_2\wp (H,R)+\lambda {\parallel E\parallel }_1\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& X=XC+HX+E,\hfill \end{array}$$

(37)

where R is an auto-weighting matrix, $\wp (H,R)={\parallel LX-LXR\parallel }_F^2+\parallel {\mathbf{1}}^T-{\mathbf{1}}^T{R\parallel }_F^2+{\parallel R\parallel }_{2,1}$. Based on the above design, the auto-weighting matrix R is group sparsity, and hence it can better preserve the local information of data. $\parallel (\mathbf{1}{\mathbf{1}}^T-R){\odot C\parallel }_1$ can be used to encourage a block diagonal structure of the representation coefficient C. In addition, AS-LRC (37) selects the $l_{2,1}$-norm to encourage a group of sparse features rather than learning low-rank features by nuclear-norm regularization, which is more robust to noise and outliers.

To improve the performance of LRR for hyperspectral band selection, Sun et al. [96] presented a fast and latent low-rank subspace clustering (FLLRSC) method. The FLLRSC can be expressed as the following optimization problem:

$$\underset{C,H,E}{min}{\parallel C\parallel }_F^2+{\parallel H\parallel }_F^2+\lambda {\parallel E\parallel }_{2,1}\mathrm{s}.\mathrm{t}.\tilde{X}=\tilde{X}C+H\tilde{X}+E,$$

(38)

where $\tilde{X}={\Phi }^{T}X\in {\mathbb{R}}^{m\times n}$ is the projected matrix, and $\Phi =\sqrt{m/n}DFP\in {\mathbb{R}}^{d\times m}$ is the Hadamard random projection matrix. $D\in {\mathbb{R}}^{d\times d}$ is a diagonal matrix whose diagonal elements are taken from −1, 1. $F\in {\mathbb{R}}^{d\times d}$ is the Hadamard matrix. $P\in {\mathbb{R}}^{d\times m}$ is a uniform sampling matrix that randomly samples m columns of $DF$. Thus, FLLRSC first performs Hadamard projection on the original data to reduce the data dimension, and then performs self-representation on the transformed data to obtain the affinity matrix. Based on this design, FLLRSC can greatly reduce computational cost and solve the problem of insufficient sampling. In addition, the coefficient matrix obtained by FLLRSC has the block diagonal structure and is insensitive to noise.

To reduce the effects of noise, Wu et al. [97] extended the LatLRR (eLatLRR) by introducing the correlations between features in the low-rank decomposition process

$$\underset{C,H,E}{min}{\parallel C\parallel }_{*}+{\parallel H\parallel }_{*}+\frac{\mu }{2}{\parallel HX-HXC\parallel }_F^2+\lambda {\parallel E\parallel }_1\mathrm{s}.\mathrm{t}.X=XC+HX+E,$$

(39)

where the term ${\parallel HX-HXC\parallel }_F^2$ is used to establish the relationship between C and H, which can further enhance the data representation. According to the fact that $HX$ is cleaner than X, ${\parallel HX-HXC\parallel }_F^2$ can be used to generate a coefficient matrix C that is more distinctive and satisfies the block diagonal structure. In addition, LatLRR [92] can be considered as a special case of eLatLRR (39) with $\mu =0$. Then, a non-negative constraint and elastic net regularization are integrated into the eLatLRR, which automatically improves the representation ability of the homogeneous samples while suppressing the effect of noise.

To fully consider the local structure of data, Fu et al. [98] proposed a novel latent LRR with weighted distance penalty (LLRRWD), that is,

$$\begin{array}{cc}\hfill \underset{C,H,E}{min}& {\mu }_1\left({\parallel C\parallel }_F^2+{\parallel H\parallel }_F^2\right)+{\mu }_2\mathrm{tr}\left({(P\odot D)}^{T}C\right)+\lambda {\parallel E\parallel }_1\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& X=XC+HX+E,C\ge 0,C_{i,i}=0,C\mathbf{1}=\mathbf{1},\hfill \end{array}$$

(40)

where D and P are the Euclidean distance matrix and weight matrix among samples, respectively. If the labels of $x_i$ and $x_j$ are the same, $P_{ij}=1$; otherwise, $P_{ij}=\theta$ with $\theta >1$. $\mathrm{tr}\left({(P\odot D)}^{T}C\right)$ is called the weighted distance penalty. In particular, if $P_{ij}=1$, the weighted distance will reduce to the traditional Euclidean distance. Different from the Euclidean distance, the weighted distance can not only maintain the local structure, but also enhance the discrimination of affinity matrix. It is worth noting that FLLRR (36) can be regarded as a special case of LLRRWD (40) with ${\mu }_2=0$. Moreover, a weight matrix is imposed on the sparse error norm to reduce the effect of noise and redundancy.

3.3. Block Diagonal Representation Based SC

As mentioned earlier, a good SC model should have the block diagonal property. The existing methods usually pursue the block diagonal structure of the representation coefficient matrix by imposing different structure priors, such as sparsity and low-rank, which is indirect. Under certain subspace assumptions, the optimal solution obtained by these methods may satisfy the block diagonal structure. However, in practical applications, due to data corruption, the block diagonal property cannot necessarily be guaranteed. The natural idea is to directly impose a block diagonal prior to the representation coefficient matrix.

Table 7. The comparison of classical block diagonal representation SC methods.

Methods	Year	$\mathit{F}\left(\mathit{C}\right)$	$\mathit{R}\left(\mathit{E}\right)$	Constraint
BD-SSC [99]	2014	${\parallel C\parallel }_1$	${\parallel E\parallel }_F^2$	$X=XC+E,\mathrm{diag}\left(C\right)=\mathbf{0},\mathrm{rank}\left(L_A\right)=n-k$
BD-LRR [99]	2014	${\parallel C\parallel }_{*}$	${\parallel E\parallel }_F^2$	$X=XC+E,\mathrm{rank}\left(L_A\right)=n-k$
BDR [100]	2019	${\parallel C\parallel }_{\left[k\right]}$	${\parallel E\parallel }_F^2$	$X=XC+E,\mathrm{diag}\left(C\right)=\mathbf{0},C\ge 0,C^T=C$
rBDR [101]	2019	${\parallel C\parallel }_F^2+{\parallel H\parallel }_F^2+\parallel B_{+}-B_{-}{V\parallel }_F^2+\mu {\parallel V\parallel }_{\left[k\right]}$	${\parallel E\parallel }_{2,1}$	$X=XC+HX+E,\mathrm{diag}\left(V\right)=\mathbf{0},V^T=V,V\ge 0$
LBDR [102]	2022	$\frac{1}{2}\parallel X-X_{U_1,U_2}{\parallel }_F^2+\mu {\parallel C\parallel }_{\left[k\right]}$	${\parallel E\parallel }_F^2$	$X_{U_1}=X_{U_1}C+E,\mathrm{diag}\left(C\right)=\mathbf{0},C^T=C,C\ge 0$
ABDR [103]	2022	$\Theta \left(C\right)$	${\parallel E\parallel }_F^2$	$X=XC+E$
PBDR [104]	2023	${\parallel C\parallel }_{\left[k\right]}+\mu {\parallel W_{2}g\left(W_1\tilde{X}\right)-C\parallel }_F^2$	${\parallel E\parallel }_F^2$	$\tilde{X}=\tilde{X}C+E,\mathrm{diag}\left(C\right)=\mathbf{0},C^T=C,C\ge 0$

summarizes the classical block diagonal representation SC.

To directly pursue the block-diagonal structure, Feng et al. [99] proposed block diagonal SSC and LRR through imposing a rank constraint on the graph Laplacian

$$\begin{array}{cc}\hfill \left(\mathrm{BD}\text{-}\mathrm{SSC}\right)& \underset{C,E}{min}{\parallel C\parallel }_1+\lambda {\parallel E\parallel }_F^2\mathrm{s}.\mathrm{t}.X=XC+E,\mathrm{diag}\left(C\right)=\mathbf{0},\mathrm{rank}\left(L_A\right)=n-k,\hfill \end{array}$$

(41)

$$\begin{array}{cc}\hfill \left(\mathrm{BD}\text{-}\mathrm{LRR}\right)& \underset{C,E}{min}{\parallel C\parallel }_{*}+\lambda {\parallel E\parallel }_F^2\mathrm{s}.\mathrm{t}.X=XC+E,\mathrm{rank}\left(L_A\right)=n-k,\hfill \end{array}$$

(42)

where $L_A$ is the Laplacian matrix, which is defined as: $L_A(i,j)=-A(i,j)$ if i ≠ j; $L_A(i,j)={\sum }_{j\ne i}A(i,j)$, otherwise. n and k are the number of samples and categories, respectively. It has been proved that the rank of $L_A$ is equivalent to the number of blocks in the affinity matrix A [151]. Hence, the constraint $\mathrm{rank}\left(L_A\right)=n-k$ enforces the obtained matrix C to form a block diagonal affinity matrix, even for noise data.

Lu et al. [100] constructed a block diagonal matrix induced regularizer and applied it to the SC problem. The proposed block diagonal representation (BDR) SC method aims to solve the following optimization problem:

$$\underset{C,E}{min}{\parallel C\parallel }_{\left[k\right]}+\lambda {\parallel E\parallel }_F^2\mathrm{s}.\mathrm{t}.X=XC+E,\mathrm{diag}\left(C\right)=\mathbf{0},C\ge 0,C^T=C,$$

(43)

where ${\parallel C\parallel }_{\left[k\right]}={\sum }_{i=n-k-1}^n{\sigma }_i\left(L_C\right)$ with Laplacian matrix $L_C=\mathrm{Diag}\left(C\mathbf{1}\right)-C$ being the lock diagonal regularizer. For the BDR model (43) without noise, it has been proved that the obtained representation coefficient matrix has the block diagonal structure under the assumption of independent subspaces. It is worth pointing out that the BDR model is a nonconvex optimization problem due to the k-block diagonal regularizer, which can be solved by the alternating minimization algorithm. Although BDR can obtain a relatively ideal block diagonal coefficient matrix for noise data, it sacrifices the convexity of the model and is more complex than SSC and LRR. In addition, the number of categories (or subspaces) in (43) must be given in advance, which is impossible in practical problems.

To cope with the situation of insufficient data samples, Zhang et al. [101] proposed a novel robust block-diagonal adaptive locality-constrained latent representation (rBDLR) method. The rBDLR can be formulated as the following optimization problem:

$$\begin{array}{cc}\hfill \underset{C,H,E,V,\theta }{min}& {\parallel C\parallel }_F^2+{\parallel H\parallel }_F^2+\parallel B_{+}-B_{-}{V\parallel }_F^2+{\mu \parallel V\parallel }_{\left[k\right]}+\lambda {\parallel E\parallel }_{2,1}\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& X=XC+HX+E,\mathrm{diag}\left(V\right)=\mathbf{0},V^T=V,V\ge 0,\hfill \end{array}$$

(44)

where $B_{+}={\left(\sqrt{\alpha }(C+\theta {\mathbf{1}}^T),\sqrt{\beta }HX\right)}^T$, $B_{-}={\left(\sqrt{\alpha }I,\sqrt{\beta }HX\right)}^T$, $\theta \in {\mathbb{R}}^{n\times 1}$ denotes the bias. $\parallel B_{+}-B_{-}{V\parallel }_F^2$ means that V is approximate by $C+\theta {\mathbf{1}}^T$, which can avoid the overfitting. Specifically, FLRR [94] can be regarded as a special case of rBDLR with $\alpha =0,\beta =0$ and removing the constraints on V. By combining the latent representation with the locality-based block-diagonal regularizer, rBDLR can extract the adaptive locality-preserving salient features jointly, and the optimal solution obtained by the rBDLR model (44) has the block diagonal structure.

To capture the nonlinear structure of data, Xu et al. [102] proposed a novel latent block-diagonal representation (LBDR) subspace clustering method which integrates an autoencoder into the block-diagonal representation

$$\begin{array}{cc}\hfill \underset{C,U_1,U_2,E}{min}& \frac{1}{2}\parallel X-X_{U_1,U_2}{\parallel }_F^2+{\mu \parallel C\parallel }_{\left[k\right]}+\lambda {\parallel E\parallel }_F^2\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& X_{U_1}=X_{U_1}C+E,\mathrm{diag}\left(C\right)=\mathbf{0},C^T=C,C\ge 0,\hfill \end{array}$$

(45)

where $U_1\in {\mathbb{R}}^{l\times d}$ and $U_2\in {\mathbb{R}}^{d\times l}$ are filter matrices, and l is the number of hidden units. $X_{U_1,U_2}\in {\mathbb{R}}^{d\times n}$ represents the autoencoder for data X, which is defined as $X_{U_1,U_2}=U_2\varphi \left(U_{1}X\right)$ with an activation function $\varphi (·)$, and $X_{U_1}\in {\mathbb{R}}^{l\times n}$ represents the latent representation matrix, which is defined as $X_{U_1}=\varphi \left(U_{1}X\right)$. Different from the traditional self-representation, LBDR performs self-representation to the reconstructed data. Note that the dimension of the reconstructed data are smaller than the original data, which can greatly reduce the calculation load. Moreover, it has proved theoretically that the LBDR model can project the original data in the nonlinear space into a new linear space.

Inspired by convex biclustering [152], Lin et al. [103] proposed an adaptive block diagonal representation (ABDR) subspace clustering method via coercively fusing both columns and rows of the coefficient matrix. The ABDR can be formulated as

$$\underset{C,E}{min}\Theta \left(C\right)+\lambda {\parallel E\parallel }_F^2\mathrm{s}.\mathrm{t}.X=XC+E,$$

(46)

where $\Theta \left(C\right)={\sum }_{i,j}{W}_{ij}\parallel C_{.i}-C_{.j}{\parallel }_2+{\sum }_{i,j}{W}_{ij}{\parallel C_{i.}-C_{j.}\parallel }_2$ with $W_{ij}={\iota }_{(i,j)}^{k}exp(\varphi \parallel X_{.i}-X_{.j}{\parallel }_2^2)$. Here, ${\iota }_{(i,j)}^k$ is an indicator function whose value is 1 if $X_{.j}$ is the k-KNN of $X_{.i}$ or 0 otherwise. It has been proved that the representation coefficient matrix obtained by the noiseless ABDR has the block diagonal structure without any structure prior. Unlike the traditional BDR methods, ABDR (46) is a convex optimization problem and does not need to give the number of subspaces in advance.

Although the BDR methods mentioned above have attracted extensive attention due to the diagonal property, the high computing cost greatly limits their performance, especially for high-dimensional data. To deal with this problem, Xu et al. [104] proposed the following projective block diagonal representation (PBDR) subspace clustering method

$$\begin{array}{cc}\hfill \underset{C,W_1,W_2,E}{min}& {\parallel C\parallel }_{\left[k\right]}+\mu \parallel W_{2}g\left(W_1\tilde{X}\right){-C\parallel }_F^2+\lambda {\parallel E\parallel }_F^2\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& \tilde{X}=\tilde{X}C+E,\mathrm{diag}\left(C\right)=\mathbf{0},C^T=C,C\ge 0,\hfill \end{array}$$

(47)

where $\tilde{X}\in {\mathbb{R}}^{d\times m}$ is a small subset selected from X by uniform random sampling, $W_1\in {\mathbb{R}}^{h\times d}$ and $W_2\in {\mathbb{R}}^{m\times h}$ are two weight matrices with h being the number of hidden units, and $g(·)$ is a nonlinear activation function (such as Sigmoid, ReLU, or TanH). It is worth noting that PBDR does not directly process the original data but its subset, which can greatly reduce the dimension of data and thus reduce the computational complexity. It has proved that the optimal solution obtained by the PBDR model (47) has the block diagonal structure. In addition, the proposed PBDR model was further extended to the sparse and low-rank cases, thus improving its ability to capture the global or local structure of data. However, for PBDR, how to select the subset of original data is a difficulty.

4. Kernel SC

4.1. Single Kernel Learning Based SC

Although the previous low-rank sparse SC methods and local structure preserving SC methods have achieved some success, they are limited to the Euclidean distance (due to the linear self-expression among samples) and unable to identify the clusters with nonlinear shaped [153]. To capture the nonlinear structure inherent in many real-world datasets, kernel technology is proposed, which maps the original data to the reproducing kernel Hilbert space (RKHS) [154,155]. In recent years, kernel technology has been successfully applied to the SC problem, which aims to construct the affinity matrix by self-representation in the kernel Hilbert space. The single kernel learning (SKL) based SC methods can be formulated as the following unified framework:

$$\underset{C\in \Omega }{min}\frac{1}{2}\parallel \varphi \left(X\right)-\varphi \left(X\right)C\parallel +F\left(C\right),$$

(48)

where $\varphi (·)$ is a feature map which maps the input data in the original feature space into the reproducing kernel Hilbert space, $F\left(C\right)$ is a penalty term (or regularization term), and $\Omega$ is the constraint set of C. The KSC model (48) can be equivalently written as

$$\underset{C\in \Omega }{min}\frac{1}{2}\mathrm{tr}\left(\varphi {\left(X\right)}^T\varphi \left(X\right)-\varphi {\left(X\right)}^T\varphi \left(X\right)C-C^T\varphi {\left(X\right)}^T\varphi \left(X\right)+C^T\varphi {\left(X\right)}^T\varphi \left(X\right)C\right)+F\left(C\right).$$

(49)

Define the kernel function K with $K_{ij}=〈\varphi \left({\mathbf{x}}_i\right),and\varphi \left({\mathbf{x}}_j\right)〉$, (49) can be transformed to

$$\underset{C\in \Omega }{min}\frac{1}{2}\mathrm{tr}\left(K-2KC+C^{T}KC\right)+F\left(C\right).$$

(50)

The classical SKL-based SC methods are summarized in

Table 8. The comparison of classical SKL-based SC methods.

Methods	Year	Brief Description	Advantage	Disadvantage
KSSC [105]	2014	SKL + SSC	It can handle the nonlinear structure of data and obtain a sparse coefficient matrix.	Its performance greatly depends on the kernel function, and it cannot capture the global structure of data.
KGS-graph [55]	2015	SKL + GS-graph	It can handle the nonlinear structure of data and group highly correlated data together.	Its performance greatly depends on the kernel function, and it cannot capture the global structure of data.
KSCBD [106]	2020	SML + BDR	It can process nonlinear data and obtain a representation coefficient matrix with block diagonal structure.	Its performance greatly depends on the kernel function, and it cannot capture the global structure of data.
ALKBDR [107]	2022	SML + BDR + LRR	It can process nonlinear data and has a stronger generalization ability to process complex data.	Its performance greatly depends on the kernel function, and it has four parameters that need to be adjusted.

.

Patel and Vidal [105] proposed a kernel sparse subspace clustering (KSSC) method by using the kernel trick to extend the SSC, which can be formulated as

$$\underset{C}{min}\mathrm{tr}\left(K-2KC+C^{T}KC\right)+\mu {\parallel C\parallel }_1\mathrm{s}.\mathrm{t}.\mathrm{diag}\left(C\right)=0,C^T\mathbf{1}=\mathbf{1},$$

(51)

where K is the kernel Gram matrix which can be chosen as the polynomial or Gaussian kernels. Since KSSC can be regarded as combining the kernel technique with SSC, it can process the nonlinear structure of data and obtain a sparse coefficient matrix. Note that KSSC (51) is a convex optimization problem, so it can be solved by ADMM [117], SSNAL [118], and so forth.

Fang et al. [55] also extend the GS-graph (6) to the kernelized version (named KGS-graph). The KGS-graph aims to solve the following optimization problem:

$$\underset{C}{min}\mathrm{tr}\left(K-2KC+C^{T}KC\right)+{\Gamma }_{\mu }^g\left(C\right),$$

(52)

where ${\Gamma }_{\mu }^g\left(C\right)={\parallel C\parallel }_1+{\sum }_i{\sum }_{j<k}max\left\{\right|C_{ij}|,|C_{ik}\left|\right\}$. As an extension of the GS-graph (6), the KGS-graph can preserve both the sparsity and locality of data simultaneously and is insensitive to noise. Moreover, it has proved that KGS-graph (52) has the grouping effect.

Yang et al. [106] presented a kernel subspace clustering method with block diagonal prior (KSCBD), which can be formulated as

$$\begin{array}{cc}\hfill \underset{C,B}{min}& \mathrm{tr}\left(K-2KC+C^{T}KC\right)+\frac{\mu }{2}{\parallel C-B\parallel }_F^2\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& B\ge 0,\mathrm{diag}\left(B\right)=0,B^T=B,\mathrm{rank}\left(L_B\right)=n-k,\hfill \end{array}$$

(53)

where K is the kernel Gram matrix, $L_B$ is the Laplacian matrix corresponding to B, and $\mathrm{rank}\left(L_B\right)=n-k$ indicates that the affinity matrix is an ideal graph with k clusters. Since KSCBD embeds the block diagonal prior into the kernel Hilbert space, it can not only process nonlinear data, but the resulting representation coefficient also has a block diagonal structure. However, due to the existence of $n-k$ rank constraints, the problem (53) is NP-hard, which is difficult to solve directly. Therefore, it usually uses the Ky Fans theorem to relax $n-k$ rank constraints and solve the relaxed problem.

Liu et al. [107] presented an adaptive low-rank kernel block diagonal representation (ALKBDR) subspace clustering method, which can be written as

$$\begin{array}{cc}\hfill \underset{\varphi \left(X\right),C,B}{min}& \mathrm{tr}\parallel \varphi \left(X\right)-\varphi \left(X\right)C\parallel +\frac{{\mu }_1}{2}{\parallel C-B\parallel }_F^2+{\mu }_2{\parallel B\parallel }_{\left[K\right]}+{\mu }_3{\parallel \varphi \left(X\right)\parallel }_{*}+{\mu }_4{\parallel E\parallel }_1\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& B\ge 0,\mathrm{diag}\left(B\right)=0,B^T=B,K_G=\varphi {\left(X\right)}^T\varphi \left(X\right)+E,\hfill \end{array}$$

(54)

where $K_G$ is a pre-defined kernel matrix. The ALKBDR model (54) can be seen as combining the kernel learning with low-rank and diagonal representation. Therefore, ALKBDR is not only able to process nonlinear data, but also has a stronger generalization ability to process complex data. However, using only a single pre-defined kernel function greatly limits its clustering performance, and there are four parameters that need to be adjusted, which is tricky and time-consuming.

4.2. Multiple Kernel Learning Based SC

It is worth pointing out that the performance of the kernel clustering will heavily depend on the choice of the kernel function K, which is often affected by prior knowledge about the data [156]. As a result, a good choice of kernel functions is critical for kernel clustering. However, one of the main challenges with kernel methods, in general, is that it is often unclear which kernel is best for a given task. To overcome this challenge, Zhao et al. [157] proposed a multiple kernel learning (MKL) framework by adding an additional procedure to combine a group of pre-specified feature mappings into the kernel clustering. For example, each sample can be mapped to ${\varphi }_{\mathbf{w}}\left(\mathbf{x}\right)={[w_1{\varphi }_1{\left(\mathbf{x}\right)}^T,w_2{\varphi }_2{\left(\mathbf{x}\right)}^T,\cdots w_m{\varphi }_m{\left(\mathbf{x}\right)}^T]}^T$, where ${\left\{{\varphi }_i(·)\right\}}_{i=1}^m$ is a group of feature mappings and $\mathbf{w}\in {\mathbb{R}}^m$ specifies the coefficients of each base kernel. Then, the corresponding kernel function can be expressed as

$${\kappa }_{\mathbf{w}}({\mathbf{x}}_j,{\mathbf{x}}_k)={\varphi }_{\mathbf{w}}{\left({\mathbf{x}}_j\right)}^T{\varphi }_{\mathbf{w}}{\left({\mathbf{x}}_k\right)}^T=\sum _{i=1}^m{w}_i^2{\kappa }_i({\mathbf{x}}_j,{\mathbf{x}}_k).$$

(55)

In particular, if the pre-specified base kernel functions ${\left\{K^i\right\}}_{i=1}^m$ are known in advance, the kernel matrix $K_{\mathbf{w}}$ can be defined as

$$K_{\mathbf{w}}=\sum _{p=1}^m{w}_i^2{K}^i.$$

(56)

Note that, in (56), the $l_2$-norm constraint is used on the kernel weights to avoid sparse solutions, which may cause redundancy of the pre-specified kernel function. In addition, replacing $K_w$ with a convex combination of pre-specified kernel ${\sum }_{i=1}^m{w}_i{K}^i$ can avoid overfitting and obtain a sparse solution. In order to intuitively compare the SKL-based methods and MKL-based methods, Figure 8 illustrates their implementation steps.

Table 9. The comparison of classical MKL-based SC methods.

Methods	Year	Brief Description	Advantage	Disadvantage
SCMK [108]	2017	MKL + F-norm penalty + graph penalty	It can capture the global structure of original data and the local structure of reconstructed data and has the grouping effect.	It is a variable indivisible optimization problem with orthogonal constraint, which is easy to fall into local solutions.
LKG [109]	2019	MKL + LRR	It can capture the global structure of the data and find a consensus kernel from the neighborhood of the candidate kernel.	It fails to fully use the structural information of data, and is sensitive to data corruption.
LLMKL [110]	2019	MKL + LRR + graph penalty	It can preserve both global structure and local structure of data, and learn a consensus kernel from the neighborhood of the predefined kernels.	The decomposition form of the kernel function may not be applicable, and it does not fully use the structure information of data.
LAKRSC [111]	2020	MKL + SR	It can learn a consensus kernel by data decomposition and is robust to noise.	The decomposition form of the kernel function may not be applicable and it does not fully use the structure information of data.
SPMKC [112]	2021	MKL + LRR + graph penalty	It can preserve both and local structure of data, and learn a consensus kernel from the neighborhood of the predefined kernels.	It is an optimization problem with orthogonal constraint which is easy to fall into local solutions, and sensitive to data corruptions.
AWLKSC [113]	2021	MKL + LRR + BDR	It can obtain a coefficient matrix with block diagonal structure, and learn a consensus kernel from the neighborhood of the predefined kernels.	The number of categories (or subspaces) must be given in advance.
SLMKC [114]	2021	MKL + AL + F-norm penalty + graph penalty	It can avoid the extra step of generating affinity matrix by representation coefficient, and preserve the local structure of data.	It is an optimization problem with orthogonal constraint which is easy to fall into local solutions, and sensitive to data corruptions.
SPAKS [115]	2022	MKL + F-norm penalty + graph penalty	It can maintain the local structure of data and divide the highly relevant data into the same groups.	It is limited in capturing the global structure of data, and the feature maps or distance matrices in RKHS need to be given in advance.
PMKSC [116]	2022	MKL + multiview clustering + AL + SEL	It can preserve more discriminative structural information in the kernel space, and learn a consensus kernel from the neighborhood of the predefined kernels.	It does not take full the similarity among multiview data and is an optimization problem with orthogonal constraint which is easy to fall into local solutions.

Remark: AL indicates affinity matrix learning; SEL indicates Self-expressiveness learning.

summarizes some classical MKL-based SC methods.

Inspired by multiple kernel clustering [157,158], Kang et al. [108] proposed a joint similarity learning and clustering with multiple kernel (SCMK) method, which can be written as the following optimization problem:

$$\begin{array}{cc}\hfill \underset{C,P,\mathbf{w}}{min}& \mathrm{tr}\left(K_{\mathbf{w}}-2K_{\mathbf{w}}C+C^T{K}_{\mathbf{w}}C\right)+{\mu }_1{\parallel C\parallel }_F^2+{\mu }_2\mathrm{tr}\left(P^{T}LP\right)\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& C^T\mathbf{1}=\mathbf{1},0\le C\le 1,P^{T}P=I,\sum _{i=1}^m\sqrt{w_i}=1,w_i\ge 0,\hfill \end{array}$$

(57)

where $K_{\mathbf{w}}={\sum }_{i=1}^m{w}_i{K}^i$ with a set of predefined kernels ${\left\{K^i\right\}}_{i=1}^m$, L is the Laplace matrix, and P is an indicator matrix. SCMK can be regarded as integrating the MKL, self-representation in the kernel Hilbert space, and local similarity learning in the original space into a unified framework. Therefore, it can capture both the global structure (due to the Frobenius norm regularization) and local structure (due to the graph regularization) of data and has the grouping effect. In addition, it has proved theoretical relationships to kernel k-means, k-means, and spectral clustering methods. However, SCMK (57) is a variable indivisible optimization problem with an orthogonal constraint, which is easy to fall into local solutions.

The existing MKL-based SC methods usually take the linear combination of the predefined kernels as the used kernel matrix, which may be sensitive to noise and limit the representation capability. To solve this problem, a natural idea is to learn a consensus kernel function and data self-representation at the same time. Consequently, Kang et al. [109] proposed a low-rank kernel learning based graph subspace clustering method (LKG), which can be written as

$$\begin{array}{cc}\hfill \underset{C,K,\mathbf{w}}{min}& \frac{1}{2}\mathrm{tr}\left(K-2KC+C^{T}KC\right)+{\mu }_1{\parallel C\parallel }_{*}+{\mu }_2{\parallel K\parallel }_{*}+{\mu }_3\parallel K-\sum _{i=1}^m{w}_i{K}^i{\parallel }_F^2\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& C\ge 0,K\ge 0,w_i\ge 0,\sum _{i=1}^r{w}_i=0,\hfill \end{array}$$

(58)

where ${\left\{K^i\right\}}_{i=1}^m$ are a set of predefined kernels, and $w_i$ is the weight for the predefined kernel $K^i$. By imposing the nuclear norm regularization, the obtained kernel matrix can capture the global structure of the data. In contrast to existing methods, LKG aims to learn the low-rank kernel matrix through the similarity of the kernel matrix and find a consensus kernel from the neighborhood of the candidate kernel.

To further consider the correlation between samples, Ren et al. [110] presented a local structural graph and low-rank consensus multiple kernel learning (LLMKL) method for subspace clustering, which can be formulated as

$$\begin{array}{cc}\hfill \underset{C,K,B,E,\mathbf{w}}{min}& \frac{1}{2}\mathrm{tr}\left[\left(I-2C+C{C}^T\right)B^{T}B\right]\hfill \\ \hfill & +{\mu }_1{\parallel B\parallel }_{*}+\frac{1}{2}\parallel K-\sum _{i=1}^m{w}_i{K}^i{\parallel }_F^2+{\mu }_3\mathrm{tr}\left(D^{T}C\right)+\lambda {\parallel E\parallel }_1\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& K=B^{T}B+E,\mathrm{diag}\left(C\right)=0,C^T\mathbf{1}=\mathbf{1},C\ge 0,w_i\ge 0,\sum _{i=1}^r{w}_i=1,\hfill \end{array}$$

(59)

where K is the kernel matrix that can be decomposed into $B^{T}B$ and a sparse noise component E, ${\left\{K^i\right\}}_{i=1}^m$ are a set of predefined kernels, $w_i$ is the weight for the predefined kernel $K^i$, and D is the similarity matrix of data with $D_{ij}={\parallel {\mathbf{x}}_i-{\mathbf{x}}_j\parallel }_2^2$. LLMKL jointly integrates the MKL technology, the global structure in the kernel space, the local structure in the original space, and the self-representation in the Hilbert space into a unified optimization model. For this reason, LLMKL can preserve both the global structure and local structure of data, and learn a consensus kernel from the neighborhood of the predefined kernels rather than using the linear combination of the predefined kernels directly, which makes it more robust to data noise.

Xue et al. [111] presented a robust subspace clustering method (LAKRSC) based on nonconvex low-rank approximation and adaptive kernel, which can be written as

$$\begin{array}{cc}\hfill \underset{C,B,Z}{min}& \frac{1}{2}\mathrm{tr}\left[\left(I-2Z+Z{Z}^T\right)B^{T}B\right]+{\mu }_1{\parallel B\parallel }_{\mathbf{w},S_p}^p+{\mu }_2{\parallel C\parallel }_1+{\mu }_3\sum _{i,j}\psi \left(E_{ij}\right)\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& K=B^{T}B+E,{\mathbf{1}}^{T}C=\mathbf{1},Z=C-\mathrm{diag}\left(C\right),\hfill \end{array}$$

(60)

where ${\parallel B\parallel }_{\mathbf{w},S_p}={\left({\sum }_{i=1}^n{w}_i{\sigma }_i^p\right)}^{1/p}$ is the weighted Schatten p-norm with $w_i=\alpha /{\sigma }_i\left(B\right)$ for a constant $\alpha$, and K is the kernel matrix which can be decomposed into $B^{T}B$ and a data corruption E. $\psi \left(E_{ij}\right)=1-exp\left(-E_{ij}^2/\left(2{\varrho }^2\right)\right)$, where $\varrho$ is the size of the Gaussian kernel. It is worth pointing out that LAKRSC uses weighted Schatten p-norm to approximate the rank function, which can better capture the global structure of data. In addition, the kernel function is obtained by data decomposition, which can obtain a better kernel function and better handle data corruption.

Ren and Sun [112] proposed a structure-preserving multiple kernel subspace clustering method (SPMKC), which can be formulated as the following nonconvex optimization problem with an orthogonal constraint, which can be written as

$$\begin{array}{cc}\hfill \underset{C,K,P}{min}& \frac{1}{2}\mathrm{tr}\left(K+C^{T}KC\right)-{\mu }_1\mathrm{tr}\left(KC\right)+{\mu }_2\mathrm{tr}\left(P^{T}LP\right)+{\mu }_3\sum _{i=1}^m{w}_i\parallel K-K^i{\parallel }_F^2+{\mu }_4{\parallel C\parallel }_F^2\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& C\ge 0,C\mathbf{1}=\mathbf{1},\mathrm{diag}\left(C\right)=0,P^{T}P=I,\hfill \end{array}$$

(61)

where ${\left\{K^i\right\}}_{i=1}^m$ are a set of predefined kernels, and the weight value $w_i$ is used to control the contribution of ith predefined kernel $K^i$ to K. Note that SPMKC can be regarded as integrating MKL, low-rank representation, and graph regularization into a unified framework. As a result, SPMKC can preserve both the global structure of the input data in the kernel space and the local structure of the original data, and the obtained representation coefficient has the block diagonal structure.

To construct an ideal k-block diagonal affinity matrix, Guo et al. [113] proposed an automatic weighted multiple kernel learning-based (AWLKSC) robust subspace clustering method, which can be formulated as

$$\begin{array}{cc}\hfill \underset{C,K,w}{min}& \frac{1}{2}\mathrm{tr}\left(K-2KC+C^{T}KC\right)+{\mu }_1{\parallel C\parallel }_{\left[k\right]}+{\mu }_2\sum _{i=1}^m{w}_i\parallel K-K^i{\parallel }_F^2+{\mu }_3{\parallel K\parallel }_{w,S_p}^p\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& C\ge 0,C=C^T,\mathrm{diag}\left(C\right)=0,\hfill \end{array}$$

(62)

where ${\parallel C\parallel }_{\left[k\right]}={\sum }_{i=n-k-1}^n{\sigma }_i\left(L_C\right)$ with Laplacian matrix $L_C=\mathrm{Diag}\left(C\mathbf{1}\right)-C$ is the lock diagonal regularizer, and ${\parallel ·\parallel }_{w,S_p}^p$ is the weighted Schatten-p norm. The AWLKSC can be considered as integrating block diagonal constraints, MKL, and low-rank approximation. Therefore, the coefficient matrix obtained by AWLKSC (62) has the block diagonal structure and is robust to noise.

The existing SC methods first learn a coefficient matrix by data self-expressiveness, and then build the affinity graph based on the coefficient matrix. This makes the quality of the affinity matrix largely depends on the coefficient matrix, and thus the obtained affinity graph may perform poorly. To tackle this issue, Ren et al. [114] presented a simultaneous learning self-expressiveness coefficients and affinity matrix multiple kernel clustering (SLMKC) method, which is formulated as

$$\begin{array}{cc}\hfill \underset{C,K,A,P}{min}& \mathrm{tr}\left(K-2KC+C^{T}KC\right)+{\mu }_1\left({\parallel C\parallel }_F^2+{\parallel A\parallel }_F^2\right)\hfill \\ & +{\mu }_2\mathrm{tr}\left(C{L}_A{C}^T\right)+{\mu }_3\mathrm{tr}\left(P^T{L}_{C}P\right)+{\mu }_4\sum _{i=1}^m{w}_i{\parallel K-K^i\parallel }_F^2\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& A^T\mathbf{1}=\mathbf{1},A⪰0,P^{T}P=I,\hfill \end{array}$$

(63)

where ${\left\{K^i\right\}}_{i=1}^m$ are a set of predefined kernels, A is the affinity matrix, $L_A$ and $L_C$ are the Laplace matrices of A and C, respectively, and $w_i$ is the weight of the ith candidate kernel $K^i$ which can take as $w_i=1/(2\sqrt{\parallel K-k^i{\parallel }_F^2}+\xi )$ with $\xi$ being infinitely close to zero. Unlike the existing MKC methods, SLMKC proposes a unified optimization model to simultaneously learn the consensus kernel, the self-expressiveness coefficient matrix, and the affinity matrix. Under this design, SLMKC can avoid the extra step of generating an affinity matrix by the representation coefficient but still maintain the correlation between the coefficient matrix and affinity matrix. Meanwhile, SLMKC can preserve the local structure of the coefficient matrix and affinity matrix due to the Laplacian regularization.

To strengthen the distinguishability of the affinity matrix, Zhang et al. [115] presented the following self-paced smooth multiple kernel subspace clustering (SPAKS) method by combining the feature smoothing regularizer with multiple kernel learning

$$\begin{array}{cc}\hfill \underset{C,\mathbf{w}}{min}& \sum _{i=1}^m{w}_i\left(\frac{1}{2}\mathrm{tr}(K^i-2K^{i}C+C^T{K}^{i}C)+\frac{{\mu }_1}{2}\mathrm{tr}\left(D^{i}C\right)+\frac{{\mu }_2}{2}{\parallel C\parallel }_F^2\right)+{\mu }_3\sum _{i=1}^m(w_{i}ln\left(w_i\right)-w_i)\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& \mathrm{diag}\left(C\right)=0,C\ge 0,C=C^T,\hfill \end{array}$$

(64)

where ${\left\{K^i\right\}}_{i=1}^m$ are a set of predefined kernels, ${\left\{D^i\right\}}_{i=1}^m$ are the distance matrix on RKHS. $\mathrm{tr}\left(D^{i}C\right)$ is the feature smoothing regularizer which can be regarded as a transformation of the graph Laplace regularity. Base on this design, SPAKS can enable the self-representation coefficients of data points draw from the same subspace to be closer, thus maintaining the local structure of data. In addition, due to the existence of the Frobenius norm, SPAKS can divide the highly relevant data into the same groups. However, SPAKS is limited in capturing the global structure of data, and the feature maps or distance matrices in RKHS need to be given in advance.

Inspired by projective clustering, Sun et al. [116] proposed a projective multiple kernel subspace clustering method (PMKSC), which can be written as

$$\begin{array}{cc}\hfill \underset{H^i,C^i,C}{min}& \sum _{i=1}^m\left(\mathrm{tr}\left(K^i(I-H^i{H^i}^T)\right)+{\mu }_1\parallel H^i-C^i{H}^i{\parallel }_F^2+{\mu }_2{\parallel C^i-C\parallel }_F^2\right)+{\mu }_3{\parallel C|}_F^2\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& {H^i}^T{H}^i=I,S^i\ge 0,C^i\mathbf{1}=\mathbf{1},\mathrm{diag}\left(C^i\right)=0,\forall i,\hfill \end{array}$$

(65)

where ${\left\{K^i\right\}}_{i=1}^m$ are a set of predefined kernels, ${\left\{H^i\right\}}_{i=1}^m$ are the projected kernel partition matrix, ${\left\{C^i\right\}}_{i=1}^m$ are the projective sub-graph corresponding to each basic kernel partition, which is used to preserve the complex relationship between each view representation, and C is the fusion graph which can be directly used as the affinity matrix for spectral clustering. Figure 9 shows the flowchart of PMKSC. By combining the kernel self-expressiveness with multiview data, PMKSC can preserve more discriminative structural information in RKHS, and be robust to data noise and redundancy. In addition, PMKSC incorporated affinity matrix learning into the unified model, avoiding the extra step of generating an affinity matrix by a representation coefficient, which helps improve clustering performance.

5. Application

SC is an effective unsupervised learning problem for high-dimensional data mining, which has been successfully applied to computer vision, pattern recognition, and other fields, such as face recognition [25,26,27], motion segmentation [28,29,30], image processing [31,32,33], and speech emotion recognition [34,35]. In this section, we aim to introduce the application fields of SC in detail and introduce some benchmark datasets.

5.1. Face Recognition

In recent years, face recognition is a biological recognition technology based on human facial feature information, which has been a research hotspot in the field of computer vision and pattern recognition. It has been proved that face images under different lighting or expression transformation can be approximated by a low dimensional subspace. Specifically, a group of face images taken from multiple people can be regarded as a combination of nine-dimensional linear subspaces [22,23]. Therefore, SC has been successfully applied to face recognition. Next, we introduce two benchmark datasets for face recognition, i.e., ORL face dataset and Extended Yale B face dataset. Figure 10 and Figure 11 show some typical images of these two face datasets, respectively.

• ORL face dataset (https://cam-orl.co.uk/facedatabase.html, accessed on 5 October 2022) consists of frontal face images collected by 40 individuals (10 samples per person) under different facial expressions (smiling or not smiling and open or close eyes) and facial details (glasses or no glasses) with $112\times 92$ pixels for each image [159]. To reduce the computational cost, these images have been downsampled into $32\times 32$ pixels [160], so each image can be regarded as a column vector of 1024 dimensions.
• Extended Yale B face dataset (https://paperswithcode.com/dataset/extended-yale-b-1, accessed on 5 October 2022) consists of frontal facial images collected by 38 individuals under 64 different lighting conditions with $192\times 168$ pixels for each image [161]. Following [160], the original images can be downsampled into $32\times 32$ pixels, so each image can be formed as a 1024-dimensional vector.

5.2. Motion Segmentation

Motion segmentation refers to dividing the video sequence into multiple space-time regions according to the motion tracks of different rigid objects. In other words, clustering the feature points on the objects with rigid motion in the video, so that each category corresponds to an independently moving rigid object, and then the motion trajectory is obtained. In particular, the coordinates of points in the trajectory of a moving object can form a low-dimensional subspace, whose dimension is at most 4 [25].

Hopkins155 motion segmentation database (http://www.vision.jhu.edu/data/hopkins155/, accessed on 5 October 2022) is a common benchmark dataset for motion segmentation. This dataset contains 155 video sequences where 120 video sequences contain two motions, and 35 video sequences have three motions [29]. For each sequence, a tracker is used to extract the point trajectories, and the outliers are extracted manually. On average, each sequence with 2 motions has 266 feature trajectories and 30 frames, and each sequence with 3 motions has 398 feature trajectories and 29 frames. It is worth pointing out that each sequence is a sole dataset (i.e., data matrix X), so there are in total 155 SC tasks. Figure 12 show some typical images of the Hopkins155 motion segmentation database.

5.3. Handwritten Digits Clustering

The images of handwritten digits reside in the subspaces of dimension 12 [162], so the problem of clustering images of handwritten digits can be solved by the SC methods. Next, we introduce two benchmark datasets for handwritten digits clustering, i.e., the USPS dataset and MNIST dataset. Figure 13 shows some images of these two datasets.

• USPS database (http://gaussianprocess.org/gpml/data/, accessed on 5 October 2022) contains 9298 grey scale images of handwritten digits 0–9 [163]. Each digit image of USPS database has 16 × 16 pixels, so that each image can be represented by a 256-dimensional vector.
• MNIST database (http://yann.lecun.com/exdb/mnist/, accessed on 5 October 2022) contains 70,000 grey scale images of handwritten digits 0–9 [164]. Each gray image of MNIST database has $28\times 28$ pixels, so each image can be vectorized into a column vector of 784 dimensions.

5.4. Speech Emotion Recognition

Speech emotion recognition aims to analyze speech data and speculate the possible emotions, such as anger, disgust, fear, happiness, neutrality, sadness, and surprise. It can be implemented in the following two steps: (1) Extracting features that can effectively express the emotional content of speech; (2) Clustering the extracted features. Here, we introduce three commonly used benchmark datasets. For more information, see [165].

• The GEneva multimodal emotion portrayals (GEMEP) [166] is a French content corpus containing 1260 emotional speeches delivered by 10 professional actors (5 female) under 18 speech emotion categories. These emotional categories cover the well-known “six major” emotions, as well as the nuances of these emotions (e.g., panic, fear).
• The airplane behavior corpus (ABC) [167] is a German content corpus crafted for the special target application of public transport surveillance. ABC contains 430 corpora delivered by 8 German speakers (4 female) under 6 speech emotion categories (aggressive, cheerful, intoxicated, nervous, neutral, and tired). The numbers of samples of six emotions are 95, 105, 33, 93, 79, and 25, respectively.
• The eNTERFACE corpus [166] is an open English audio-visual emotion database. It consists of 1277 corpora made by 42 English speakers (8 female) from 14 countries under six basic emotions (anger, disgust, fear, happiness, sadness, and surprise). The numbers of samples of six emotions are 215, 215, 215, 207, 210, and 215, respectively.

6. Research Prospect for Subspace Clustering

The emergence of high-dimensional data has promoted the development of traditional machine learning. Although subspace clustering has been developed to varying degrees in many practical applications, there are still several directions to be solved in the future.

6.1. Deep Subspace Clustering

With the development of deep learning technology in recent years, it has been widely used in various fields due to its ability to effectively explore the deep features of data. Deep subspace clustering (DSC) has emerged, which performs subspace clustering based on the low-dimensional features of the original data learned by deep learning techniques.

At present, convolutional neural network [168,169,170,171], generative adversarial network [172,173,174,175], deep auto-encoder [176,177,178,179], and other mainstream deep learning methods have been successfully applied to subspace clustering, and achieved good clustering performance in many fields. Although the DSC methods have improved the clustering accuracy to a certain extent, how to effectively mine the subspace structure inside the data and obtain a more robust data representation still needs further research. In addition, the DSC methods are limited by computationally time-consuming and large memory consumption, so how to quickly solve the DSC needs to be urgently settled.

6.2. Data Outliers Detection

As Hampel et al. [180] pointed out, a routine dataset may contain about 1–10% (or more) of outliers, while for high-dimensional data, this proportion may be larger. In the process of data processing, data outliers have a high impact on the performance of models, and affect the generalization ability of models, even leading to the model being unable to extract effective features. Therefore, it is essential to identify and separate data outliers.

At present, statistical researchers often use robust regression (such as least trimmed squares regression [181], Huber regression [182], and least absolute deviation [183]) or mean-shift model [184] to solve this problem. Up to now, a subspace clustering method with outlier detection has not been found. Consequently, how to combine the subspace clustering with data outliers detection is meaningful research.

6.3. Tuning Parameters Selection

In the subspace clustering methods, there exists exists some tuning parameters, especially in the kernel subspace clustering methods. A good tuning parameter can often determine the clustering performance of the these methods. At present, grid search (such as cross validation), the Bayesian method, or the empirical method are usually used to adjust the tuning parameters; however, this is a time-consuming process. Hence, it is interesting to study how to choose the regulating parameters.

6.4. Kernel Functions Selection

As mentioned earlier, the performance of kernel subspace clustering methods depends heavily on the choice of kernel functions. Therefore, a good choice of kernel functions is crucial for kernel subspace clustering. However, a major challenge of kernel methods is that it is usually unclear which kernel is optimal. Although many kernel function construction methods or multiple kernel learning methods have been proposed, this problem has still not been well resolved. As a consequence, studying the selection of kernel functions is also a promising direction.

6.5. Theory Analysis

According to the previous description, it can be found that most of the subspace clustering methods lack theoretical support, and only a small part of them have investigated the block diagonal property of the self-representation coefficient matrix and grouping effect. In addition, no articles studying the statistical properties of the model (for example, the error bound theory and consistency) and the clustering recovery theory have been found. Studying the theoretical properties of the models can help us understand and analyze them, and thus better apply them to practical problems. Therefore, it is very meaningful to further study the theoretical properties of subspace clustering methods.

7. Conclusions

In this survey, we have reviewed the development of SC methods in the past two decades. According to the strategy of the constructing representation coefficient, we divide the classical SC methods into three categories, i.e., low-rank sparse SC methods, local structure preserving SC methods, and kernel SC methods. Among them, the low-rank sparse SC methods can capture the global structure of data and achieve subspace feature selection by low-rank representation and sparse representation. Local structure preserving SC methods can better capture the geometric information of data. Moreover, kernel SC methods can not only capture the geometric information of data, but also cope with the challenges brought by the nonlinearity of data. Then, the application fields of SC and the commonly used benchmark datasets are introduced. Finally, we have discussed several interesting and meaningful future research directions.