Infrared Small Dim Target Detection Using Group Regularized Principle Component Pursuit

Abstract

The detection of an infrared small target faces the problems of background interference and non-obvious target features, which have yet to be efficiently solved. By employing the non-local self-correlation characteristic of the infrared images, the principle component pursuit (PCP)-based methods are demonstrated to be applicable to infrared small target detection in a complex scene. However, existing PCP-based methods heavily depend on the uniform distribution of the background pixels and are prone to generating a high number of false alarms under strong clutter situations. In this paper, we propose a group low-rank regularized principle component pursuit model (GPCP) to solve this problem. First, the local image patches are clustered into several groups that correspond to different grayscale distributions. These patch groups are regularized with a group low-rank constraint, enabling an independent recovery of different background regions. Then, GPCP model integrates the group low-rank components with a global sparse component to extract small targets from the background. Different singular value thresholds can be exploited for image groups corresponding to different brightness and grayscale variance, boosting the recovery of background clutters and also enhancing the detection of small targets. Finally, a customized optimization approach based on alternating direction method of multipliers is proposed to solve this model. We set three representative detection scenes, including the ground background, sea background and sky background for experiment analysis and model comparison. The evaluation results show the proposed model has superiority in background suppression and achieves better adaptability for different scenes compared with various state-of-the-art methods.

1. Introduction

The infrared search system has the merits of working in all weather, all day and at long ranges, which is applicable to many important fields such as early-warning systems, aerospace technology, remote sensing [1,2,3], etc. In the moving process of the infrared target, it is easy for it to be submerged in high brightness clutter such as clouds, sea-sky-line, etc. In addition, the search system usually needs to detect long-range targets [4], which means the target size is very small and the useful signal is very weak. To adapt to these real-world scenarios, the detection algorithms should be designed to handle the interference of background clutter and achieve the effective extraction of the small (<$9\times 9$ pixels) and weak (<3 SNR) targets.

Over the years, a plethora of small dim target detection algorithms have been proposed. From the perspective of image characteristic utilization, these algorithms can be categorized into three types: target characteristic-based method [5,6,7,8], background characteristic-based method [9,10,11] and target/background characteristic integration-based method [12,13,14,15]. Generally, the infrared small targets appear to have large gray-scale values and are prone to distribution in high-frequency areas. These properties are usually adopted by the target characteristic-based methods for potential target region extraction, such as the local contrast measure [5,16], local entropy measure [3] and frequency-domain saliency region segmentation [17]. These methods are adequate for their relatively uniform background and targets with high brightness. However, due to the lack of background modeling, the strong edges or background clutters can easily be detected as false alarms in the target characteristic-based methods. The background characteristic-based methods can avoid the confusion of the targets and background interference to some extent, in which the background pixels of adjacent image area and successive frames are assumed to be spatially consistent. According to this, the target is detected by removing the predicted background from the original image. However, background characteristic-based methods are not suitable for handling complex backgrounds due to the difficulty of background estimation. It has been extensively shown that using single-target or background information is not effective for detecting small targets in complex situations.

The infrared patch-image (IPI) model [12] is a representative target and background integration-based method. By employing the non-local self-correlation of the background, IPI transfers the small target detection problem to the recovery of a low-rank matrix and a sparse matrix, which correspond to the background component, and target component respectively. The target components are regarded as outliers that increase the rank value of the data matrix and can be efficiently separated by the PCP model. Accurate approximation of the matrix rank is a major difficulty for the IPI model. Overlarge rank is good for background interference suppression but is more likely to cause miss detection. On the contrary, a lower rank will introduce an enormous number of false alarms. Recently, many works concentrate on the approximation of the matrix ranks. Zhang et al. [18] propose the modification of the low-rank constraint of IPI to a tighter-rank surrogate—$l_{2,1}$ norm to remove the unpredictable background residuals. Zhu et al. use a smooth but nonconvex surrogate of the rank based on the Log operator [19], which is closer to the rank minimization optimization than the nuclear norm. In the NIPPS model [20] and PSTNN model [14], by using the partial sum of nuclear norms, the matrix and tensor ranks are approximated by the energy ratio of the principle matrix to adapt to the changeable background. Liu et al. proposed a non-convex tensor low-rank approximation (NTLA) method to adaptively assign different weights to different singular values [21]. The above-mentioned methods focus more on using different surrogates to replace the low-rank constraint. However, due to the intrinsic diversity of different local image regions, using one single low-rank constraint is difficult to describe the whole background. For the sake of a more accurate description model, it is necessary to explore the complexity variations of different local regions and assign different rank thresholds in reconstruction.

Considering the fact that the infrared image is nonuniform and its complexity varies spatially, we establish a novel group regularized PCP model, named as GPCP for the small target detection in complex scenes. The proposed model employs a group low-rank constraint to replace the previous global low-rank constraint in recovering the background component, and enforces using different number of principle components for image data groups corresponding to different complexity and brightness. By minimizing the group low-rank constraints of the GPCP model, more image details can be reconstructed, so that the residual errors can be eliminated from the target components. The contributions of this article can be summarized as follows:

• We analyze the low-rank property of the global data matrix and grouped data matrix, and find there is a significant difference of principle component number in recovering the data matrix with different complexity.
• We propose a group low-rank constraint for background recovery and combine it with a global sparse regularization term for target recovery, which can remove the residual errors in the target component efficiently.
• A customized optimization algorithm is adopted to solve the proposed GPCP model, in which the group low-rank components are decoupled by the ADMM algorithm.

The rest of the sections are organized as follows. In Section 2, some related works on the small target detection are briefly reviewed. Section 3 introduces the algorithm flowchart and implementation details of the proposed detection model. Section 4 gives the experimental results on different background situations to demonstrate the effectiveness of our proposed method. Section 5 concludes the whole paper and discusses future works.

2. Related Work

We briefly review the related work on the small target detection methods using target characteristics, background characteristics and integration idea.

2.1. Target Characteristic-Based Method

The target characteristic-based method mainly focuses on the distinction between the target and its surrounding background. Many representative methods have been proposed in this research branch, such as the local contrast measure (LCM) [5,22], entropy contrast measure (ECM) [23,24], sparse representation-based methods [25], and so on. This type of method utilizes the shape or statistical characteristics of the small target for target detection. However, due to the similar image property of the small target and the strong background edges, the background clutters could easily be mistaken for a target. To address the issue of false detection, relative methods have been proposed to enhance the target intensity while suppressing the background region, such as the weighted local difference measure (WLDM) [6], multiscale local homogeneity measure (MLHM) [7], self-regularized weighted sparse model (SRWS) [26] etc. In the recent studies, saliency features are also utilized to associate the gray intensity with the entropy [19,27] or frequency domain [3,17], which have gained better results in small target detection. It is noticeable that these methods are sensitive to the settings of the target size and window size, which are hard to balance without prior information.

2.2. Background Characteristic-Based Method

The background characteristic-based method is usually based on the assumption that the background pixels are highly correlated, and targets are the parts that break this relationship. So, many background characteristic-based methods study the background estimation algorithms by using the neighboring pixels [9,11,28]. For example, the difference of Gaussian (DoG) filter uses the weighted sum of local neighborhood pixels as the background [29]. To cope with the problem of edge sensitivity, many methods propose to add the orientation information for background estimation, such as the max-mean and max-median filters [30], in which the maximum values of the mean or median arrays of different lines is taken as the background. The above mentioned background characteristic-based methods are all based on the local estimation model, in addition, the estimation strategy usually selects the maximum value of different orientations, which is not accurately designed. Aiming at this problem, some researchers propose to adopt the transform domain information for background suppression [9,31]. In [9], the whole infrared image is transformed into the frequency domain, and the background component is suppressed by removing the low frequency component from the original image. However, this type of method cannot suppress the complex background because the strong edges also belong to high-frequency subbands.

2.3. Target/Background Characteristic Integration-Based Method

Recently, the low-rank sparse model, which integrates the target characteristic and background characteristic by image data decomposition, has achieved considerable advances in the small target detection area. In [12], Gao et al. presented an infrared patch image (IPI) model, in which the target component and background component are assumed to be sparse and low rank, respectively. Considering that equally weighted singular values will restrict the description ability of low rank nature for the background patches, Zhang et al. proposed to modify the nuclear norm regularization to a weighted nuclear norm [13], which makes the model more flexible for complex background. Afterwards, Dai et al. pointed out that when facing extremely complex background, the low rank assumption of IPI model has a mismatching problem, which may lead the strong edges to be considered as outliers [20]. To solve this problem, they adopted the partial sum of singular values to constrain the low-rank background instead of the nuclear norm. Similar idea has also been mentioned in [14], where the partial sum minimization constraint of singular values is extended to the patch-tensor model. In order to transfer the NP-hard problem of PCP model into a non-convex optimization problem, Zhang et al. proposed to apply schatten q-norm and $l_p$-norm to the small target detection area, which is named as NOLC model [32]. To enhance the detection accuracy, in [26], an overlapping edge information is applied to mine the structure information of background. Multiple frames-based models [33] are also reported for small target detection in complex scenes. In  [34], Aliha et al. built a block-matching patch-tensor model based on the spatial–temporal domain to extract inter-frame information. Hu et al. further used a simultaneous sampling in spatial and temporal domains to make full use of the information between multiple frames [26]. Considering the target’s local continuity in the spatial–temporal domain, Li et al.  [35] proposed a spatial regularized spatial–temporal twist tensor model, which can reduce the global noise to some extent.

Recently, convolutional neural network (CNN) began to appear in the infrared small target detection study area. Du et al. [36] proposed a shallow–deep feature-based detection model, which demonstrates that shallow features are important for small target detection. Regarding the feature lacking problem, Bai et al. used a cross-connection bidirectional pyramid network to provide more comprehensive target information [37]. To cope with the miss detection problem, Liu et al. adopted the transformer to learn the correlation of image features in a larger range [38]. Among existing deep learning methods, feature learning still remains challenging due to the small size and non-obvious image features of the infrared small targets.

3. Proposed Small Target Detection Using GPCP

In this section, we first analyze the low-rank property of the patch-image data matrix, including the global data matrix, the bright-uniform part, the dark-uniform part and the cluttered part. Then, a group-regularized principle component pursuit model (GPCP) is constructed according to the diverse characteristics of the local image parts. Finally, the sparse component which includes the small target is separated from the complex background using the GPCP model, as shown in Figure 1. The algorithm steps and results of the traditional PCP model and the proposed GPCP model are also illustrated in Figure 1. Next, we will elaborate the details of the proposed small target detection model.

3.1. Low-Rank Property of Image Groups

The existing PCP-based models mainly focus on the low-rank structure of the global data matrix and ignore the inhomogeneous information among local background regions, which makes these models not suitable to handle complex scenes. As illustrated in the upper part of Figure 1, there are many background clutters (labeled by blue boxes) remaining in the sparse component after the global PCP based decomposition process. It could also be observed that the residual clutters are mainly distributed in the image parts with strong edges or big gray level changes. That is to say, such a decomposition model forms a confusion of the small targets and some background clutters. Since the background component is recovered by a low-rank constraint, the key problem is then transferred into how to determine the rank threshold in the PCP process.

To handle the aforementioned issue, we need to have a deeper understanding of the low-rank characteristics of different background parts. Figure 2 illustrates the eigenvalue curves of the global data matrix and the grouped local data matrix, which are generated by the distribution of data. To avoid the influence of matrix dimension on the result of eigenvalues, the column size of the global data matrix and the grouped data matrix is down-sampled to keep it the same. The X-axis of Figure 2c represents the number of principle components, which is defined as “rank threshold” in the optimization process. Y-axis represents the eigenvalues of data matrix. Here, we set 2 as the boundary of principle components, which means the eigenvectors whose corresponding eigenvalues are greater than 2 are regarded as principle components. From Figure 2c, we can see that the principle component of the global data matrix (red line) is concentrated in the top nine eigenvectors. For the uniform data matrix groups (green and pink lines), the number of principle components is about seven to eight. By comparison, the threshold value of the cluttered part (blue line) is 10, which is much larger than the other two uniform parts and is a bit larger than the global data matrix. This demonstrates that there is a significant difference on the low-rank characteristics among the bright-uniform part, the dark-uniform part and the cluttered part, which motivates us to consider whether we can use a group regularized PCP model to cope with the clutter interference problems in complex situations.

3.2. Construction of the GPCP Model

PCP is a convex model which aims to recover the low-rank matrix when the data matrix is corrupted by gross sparse errors [39] and is playing an important role in the recent patch-image based small target detection methods. Mathematically, it considers the data matrix $D\in R^{n_1\times n_2}$ is composed of a low-rank component L and a sparse component S and solves the following convex optimization problem:

$$\underset{L,S}{\arg \min }{∥L∥}_{*}+\lambda {∥S∥}_0,s.t.∥D-L-S∥\le \epsilon$$

(1)

To recover L and S, the low-rank component L should be limited to the following three conditions:

$$\underset{i}{max}{∥U^{*}{e}_i∥}^2\le \frac{\mu r}{n_1},\underset{i}{max}{∥V^{*}{e}_i∥}^2\le \frac{\mu r}{n_2},{∥U{V}^{*}∥}_{\infty }\le \sqrt{\frac{\mu r}{n_1{n}_2}}$$

(2)

where $L=U\Sigma V^{*}={\displaystyle \sum _{i=1}^r}{\sigma }_i\mu {}_i{v}_i^{*}$. By arranging an appropriate r, the L and S components can be efficiently separated after the PCP operation. Yet a unified r is not suitable to handle the overall data matrix since the image data always corresponds to different complexity. An extreme example is that the small target is located in the smooth background part, meanwhile strong edges exist in the other part of the background. When r is set to a small value, many residual errors will remain in the sparse component; when r is large, the real target will be regarded as the low-rank component. Therefore, the true reason causing missed detection and false detection lies in the data structure diversity of different background parts.

The newly designed GPCP model we consider in this paper assumes the low-rank component L satisfies a group low-rank structure, which is defined as follows:

$$ran{k}_{group}\left(L\right)={\displaystyle \sum _k}{\mu }_k{∥L_k∥}_{*}$$

(3)

where $L_k$ represents the $k^{th}$ group of the low-rank component, ${\mu }_k$ is used to balance the image groups with different data number. In this way, each $L_k$ is considered independent with each other and will correspond to different shrink thresholds $r_k$ for decomposition. The eigen value curves of the data matrixes in Figure 2 also show that compared with the global data matrix, the group data matrix has a better property on the low-rank condition of PCP model. To recover the low-rank components $L_k\left(k=1,2,\dots \right)$ and the spare component S, we need to solve the following GPCP model:

$$\begin{array}{c}\underset{L,S}{\arg \min }{\displaystyle \sum _k}{\mu }_k{∥L_k∥}_{*}+\lambda {∥S∥}_1\hfill \\ s.t.D=L+S+N,L=\left\{L_1,\dots L_k,\dots \right\}\hfill \end{array}$$

(4)

3.3. Small Target Detection Using GPCP

Typically, the small target detection model can be written as follows:

$$D=T+B+N$$

(5)

where D represents the input image, T, B and N represent the target, background and noise, respectively.

In this paper, we follow the basic idea of the infrared patch-image model [12] and denote D as a data matrix, which is composed of column-wise local patches of the input image. To explore the data structure of the background component, the image patch vectors with similar property on gray-scale variation should be clustered together. The complexity and gray level of an image are reflected by the variance value $\sigma$ and the mean value $\mu$, respectively. So, we employ $\left(\mu ,\sigma \right)$ as the data feature descriptor.

Firstly, the image data can be divided into a clutter group and a uniform group according to $\sigma$. Then, for the uniform group, the bright part and the dark part also correspond to different image properties. The dissimilarity degree between two data samples is calculated by:

$$d_1=\left|{\mu }_1-{\mu }_1\right|,d_2=\left|{\sigma }_1-{\sigma }_1\right|$$

(6)

According to Equation (6), large $d_1$ and $d_2$ indicate a big difference between two samples. The k-means cluster algorithm is employed to divide the entire data into three groups: the bright-uniform part, the dark-uniform part and the cluttered part, which is shown in Figure 2. According to the previous discussion in Section 3.1 and Section 3.2, the image data in different groups always corresponds to different low-rank structure and should be regularized separately. So, we propose to use the GPCP model to depict the background patch-image in complex scenes, which is defined as:

$$\begin{array}{c}{∥B∥}_{g*}={\displaystyle \sum _k}{\mu }_k{∥B_k∥}_{*}\hfill \\ B=\left\{B_1,B_2,\dots ,B_k\right\}\hfill \end{array}$$

(7)

where $B_k$ represents the k-th group of the background data, ${\mu }_k$ is used to balance the image groups with different data number, which is defined as:

$${\mu }_k=\frac{datanumberofgroupk}{totalnumber}$$

(8)

Here, we use the group-regularized nuclear norm ${\displaystyle \sum _k}{\mu }_k{∥B_k∥}_{*}$ to approximate the rank property of the background component B, instead of ${∥B∥}_{*}$. So, the whole background is composed of the recovering of these separated image groups. Generally, the image groups containing strong edges and clutters will correspond to a large singular threshold, and the uniform image groups will correspond to a lower one. Compared with the previous detection method which uses one single low-rank constraint for the whole background component, the group low rank regularization can better explore the local structure of the image and lead to a more accurate decomposition result.

In the infrared images, small targets are usually randomly distributed in different groups. So, to keep the sparsity of the entire target component rather than the group component, we use a global sparse constraint for the whole target component T, which is defined as ${∥T∥}_1$. Therefore, the group IPI model is defined as follows:

$$\begin{array}{c}\underset{L,S}{\arg \min }{\displaystyle \sum _k}{\mu }_k{∥B_k∥}_{*}+\lambda {∥T∥}_1\hfill \\ s.t.D=B+T+N,B=\left\{B_1,B_2,\dots ,B_k\right\}\hfill \end{array}$$

(9)

3.4. Optimization Method of the GPCP Model

The objective function defined in Equation (9) is a convex problem which includes two variables B and T to be solved. It should be noticed that the background component B in Equation (9) is composed of several local groups and each group is independent of one another, which has a great difference compared with the traditional PCP model. In accordance with this complex situation, we adopt the ADMM algorithm to decouple the group principle component pursuit model into several sub-problems and alternatively optimize one variable while keeping others fixed. The augmented Lagrangian expression of Equation (9) can be rewritten as the following form:

$$\begin{array}{c}{L}_{\rho }\left(B,T,F\right)={\displaystyle \sum _k}{\mu }_k{∥B_k∥}_{*}+\frac{\rho }{2}{∥D-B-T∥}_F^2\hfill \\ +\lambda {∥T∥}_1+〈F,D-B-T〉\hfill \end{array}$$

(10)

where F represents the dual vector, $\rho >0$ is the penalty parameter. The algorithm flow of ADMM is summarized in Algorithm 1.

Algorithm 1 ADMM (Alternating Direction Method of Multipliers) Algorithm for GPCP model

Input: group number K, regularization parameter $\lambda$, penalty parameter $\rho$, update factor for $\rho$: $\mu \rho$, maximum iteration $max\_iter$, tolerance error $tol$. while not converged do    1. Compute group background component $B_k$ using $U_{k}diag\left(pos\left({\sigma }_k-\frac{1}{\rho }\right)\right)V_k^T$;    2. Combine K group components $B_{1:K}$ into a global form B;    3. Compute target component T using $t{h}_{\frac{2\lambda }{\rho }}\left(D-B+\frac{F}{\rho }\right)$;    4. Update dual factor F using $F^{t+1}=F^t+{\rho }^t\left(T^{t+1}+B^{t+1}-D\right)$;    5. Update penalty factor $\rho$ using ${\rho }^{t+1}=\mu \rho \times {\rho }^t$;    6. Set termination condition:    (1) Compute reconstruction error $err=∥T+B-D∥<tol$;    (2) Target component not change ${\displaystyle \sum _{i,j}}{T}^{t-1}(i,j)={\displaystyle \sum _{i,j}}{T}^t(i,j)$;    (3) Reach the maximum iteration number $max\_iter$. end while Output: Sparse coefficient matrix $X^{\left(k\right)}$.

(1) Solution of background component B

The objective expression with regard to B can be summarized as:

$$L_{\rho }\left(B\right)={\displaystyle \sum _k}{\mu }_k{∥B_k∥}_{*}+\frac{\rho }{2}{∥D-B-T∥}_F^2+〈F,D-B-T〉$$

(11)

The group members in B are independent with each other, so the minimization problem of $\frac{\rho }{2}{∥D-B-T∥}_F^2$ is equal to minimizing $\frac{\rho }{2}{\displaystyle \sum _k}{∥D_k-B_k-T_k∥}_F^2$, and the minimization problem of $〈F,D-B-T〉$ is equal to minimizing $〈F,D_k-B_k-T_k〉$. According to this, Equation (10) can also be described as the following grouped summation form:

$$\begin{array}{c}{L}_{\rho }\left(B_k\right)={\displaystyle \sum _k}{\mu }_k{∥B_k∥}_{*}+〈F,D_k-B_k-T〉\hfill \\ +\frac{\rho }{2}{\displaystyle \sum _k}{∥D_k-B_k-T_k∥}_F^2\hfill \end{array}$$

(12)

For each group, the objective function related to its corresponding background component $B_k$ can be rewritten as the separated group form:

$$\begin{array}{c}{L}_{\rho }\left(B_k\right)={\mu }_k{∥B_k∥}_{*}+〈F,D_k-B_k-T〉\hfill \\ +\frac{\rho }{2}{∥D_k-B_k-T_k∥}_F^2\hfill \\ ={\mu }_k{∥B_k∥}_{*}+\frac{\rho }{2}{∥B_k-\left(D_k-T_k+\frac{F}{\rho }\right)∥}_F^2\hfill \end{array}$$

(13)

The above problem can be solved by the singular value thresholding algorithm [40], which is defined as follows:

$$\begin{array}{c}{B}_k^{t+1}=SV{D}_{\frac{1}{\rho }}\left(D_k-T_k-\frac{F}{\rho }\right)\hfill \\ =U_{k}diag\left(pos\left({\sigma }_k-\frac{1}{\rho }\right)\right)V_k^T\hfill \\ pos\left({\sigma }_k-\frac{1}{\rho }\right)=\left\{\begin{array}{c}{\sigma }_k-\frac{1}{\rho },if{\sigma }_k>\frac{1}{\rho }\hfill \\ 0,otherwise\hfill \end{array}\right.\end{array}$$

(14)

where $U_k$, $V_k$ and ${\sigma }_k$ are the left eigen-vector, right eigen-vector and singular values of matrix $D_k-T_k-\frac{F}{\rho }$, respectively.

(2) Solution of target component

The objective expression with regard to T can be summarized as:

$$\begin{array}{c}{L}_{\rho }\left(T\right)=\lambda {∥T∥}_1+\frac{\rho }{2}{∥D-B-T∥}_F^2+〈F,D_k-B_k-T_k〉\hfill \\ B=\left\{B_1,\dots B_k,\dots \right\}\hfill \end{array}$$

(15)

Similar to Equation (13), the above expression can be rewritten as the following form:

$$L_{\rho }\left(T\right)=\lambda {∥T∥}_1+\frac{\rho }{2}{∥T-\left(D-B+\frac{F}{\rho }\right)∥}_F^2$$

(16)

According to reference [41], the solution of Equation (16) is given by the soft-thresholding function:

$$\begin{array}{c}{T}^{t+1}=t{h}_{\frac{2\lambda }{\rho }}\left(D-B+\frac{F}{\rho }\right)\hfill \\ t{h}_s\left(W\right)=\left\{\begin{array}{c}w-s,w>s\hfill \\ w+s,w<-s\hfill \\ 0,otherwise\hfill \end{array}\right.\hfill \end{array}$$

(17)

in which w represents the element of matrix W, $T^{t+1}$ represents the updated target component in the next iteration.

(3) Update dual factor F and penalty factor $\rho$

The dual factor F and penalty factor $\rho$ are all updated in a standard way as shown in the following:

$$\begin{array}{c}{F}^{t+1}=F^t+{\rho }^t\left(T^{t+1}+B^{t+1}-D\right)\hfill \\ {\rho }^{t+1}=\mu \rho \times {\rho }^t\hfill \end{array}$$

(18)

where $\mu \rho$ is the update factor for $\rho$.

4. Experimental Evaluations

4.1. Experiment Settings

4.1.1. Parameter Settings

In our experiment, the image is divided into $16\times 16$ local patches with 10 pixel step size. The group number is set to 3. The regularization factor $\lambda$ of the target component is $1/\left[\sqrt{min\left(M,N\right)}\right]$, where M and N represent the patch size and patch number, respectively. The penalty factor $\rho$ of the ADMM method is set to 0.001, and the update factor $\mu \rho$ is 1.05. The maximum iteration of ADMM method is set to 500.

4.1.2. Evaluation Metrics

We adopt three metrics to evaluate the performance of the detection algorithms. The first one is receiver operating characteristic (ROC) curve, which describes the sensitivity (or called saliency) of the target after detection operation. The false alarm ratio $F_a$ and probability of detection $P_d$ are employed to form the horizontal and vertical axis of the ROC curve, which are separately defined as below:

$$P_d=\frac{detectedtargetnumber}{realtargetnumber}$$

(19)

$$F_a=\frac{falselydetectedpixelnumber}{totalpixelnumber}$$

(20)

For a randomly selected segmentation threshold, a good detection result should have a low false alarm ratio, while keeping a high target detection rate.

Another two metrics, signal-to-clutter ratio gain (SCRG) and background suppression factor (BSF) are used to measure the information change between the input images and output images. SCRG mainly reflects the enhanced capability to the target and BSF focuses on measuring the suppression effect on the background, which are separately defined as:

$$SCRG=\frac{SCR{G}_{out}}{SCR{G}_{in}},SCR=\frac{\left|{\mu }_t-{\mu }_b\right|}{{\sigma }_b}$$

(21)

$$BSF=\frac{{\left({\sigma }_b\right)}_{in}}{{\left({\sigma }_b\right)}_{out}}$$

(22)

where ${\mu }_t$ and ${\mu }_b$ represent the mean value of the target part and background part, respectively. ${\sigma }_b$ represents the standard deviation of the background part. Larger SCRG and BSF scores indicate a better detection performance.

4.1.3. Baseline Algorithms

To evaluate the performance of our proposed detection algorithm, several state-of-the-art methods are introduced as the comparison group, involving non-convex tensor low-rank approximation method (ASTTV-NTLA) [21], infrared patch image model (IPI) [12], partial sum of tensor nuclear norm-based detection model (PSTNN) [14], total variation regularization-based model (TVPCP) [42], reweighted image patch tensor model (RIPT) [43], non-convex rank approximation minimization joint $l_{1,2}$ norm-based model (NRAM) [18], multiscale patch-based contrast measure-based model (MPCM) [44] and sparse regularization-based spatial–temporal twist tensor (SRSTT) model [35].

Table 1. Detailed parameter settings for compared methods.

Methods	Acronyms	Parameter Settings
Non-convex tensor low-rank approximation method	ASTTV-NTLA	$L=3$, $H=6$, ${\lambda }_{tv}=0.005$, ${\lambda }_s=\frac{H}{\sqrt{max(M,N)\ast L}}$, ${\lambda }_3=100$
Infrared patch image model	IPI	Patchsize: $30\times 30$, step: 10, $\lambda =\frac{1}{\sqrt{min(m,n)}}$, $\epsilon ={10}^{-7}$
Partial sum of tensor nuclear norm-based detection model	PSTNN	Patchsize: $40\times 40$, step: 40, $\lambda =\frac{0.6}{\sqrt{max(n_1,n_2)\ast n_3}}$, $\epsilon ={10}^{-7}$
Total variation regularization-based model	TVPCP	${\lambda }_1=0.005$, ${\lambda }_2=\frac{1}{\sqrt{max(M,N)}}$, $\beta =0.025$, $\gamma =1.5$
Reweighted image patch tensor model	RIPT	Patchsize: $50\times 50$, step: 10, $\lambda =\frac{L}{\sqrt{min(n_1,n_2,n_3)}}$ $L=1$, $H=10$, $\epsilon ={10}^{-7}$
Non-convex rank approximation minimization joint $l_{1,2}$ norm-based model	NRAM	Patchsize: $50\times 50$, step: 10, $\gamma =0.002$, $\lambda =\frac{1}{\sqrt{max(M,N)}}$ $C=\frac{\sqrt{min(M,N)}}{2.5}$, ${\mu }^0=3\sqrt{min(M,N)}$, $\epsilon ={10}^{-7}$
Multiscale patch-based contrast measure-based model	MPCM	Mean filter size: $3\times 3$, $N=3,5,7,9$
Sparse regularization-based spatial–temporal twist tensor	SRSTT	$L=30$, ${\lambda }_1=0.05$, ${\lambda }_2=0.1$, ${\lambda }_3=100$, $\epsilon ={10}^{-7}$, $\mu =0.01$
Group-regularized principle component pursuit	GPCP	Patchsize: $30\times 30$, step: 10, groupnum: 3, $\lambda =\frac{1}{\sqrt{min(m,n)}}$, $\epsilon ={10}^{-7}$

shows the detailed parameter settings of the compared methods in this paper.

4.1.4. Dataset

The full dataset contains 12 sequences. According to the type of detection scene, we have manually divided these sequences into 3 categories, including 3 ground-background sequences, 3 sea-background sequences and 6 sky-background sequences. The frame number, image size and signal-to-clutter information of each sequence are shown in

Table 2. Dataset Information.

	Sequence Name	Frame Number	Image Size	Average SCR
Ground Background	Ground-1	200	$256\times 256$	2.21 dB
	Ground-2	200	$256\times 256$	3.41 dB
	Ground-3	200	$256\times 256$	5.01 dB
Sea Background	Sea-1	100	$128\times 128$	2.29 dB
	Sea-2	87	$284\times 213$	6.32 dB
	Sea-3	185	$252\times 213$	2.28 dB
Sky Background	Sky-1	60	$320\times 240$	6.86 dB
	Sky-2	67	$320\times 240$	0.87 dB
	Sky-3	400	$256\times 172$	4.14 dB
	Sky-4	200	$256\times 208$	−2.56 dB
	Sky-5	40	$128\times 128$	2.73 dB
	Sky-6	40	$256\times 200$	2.44 dB

.

Representative frames of each detection scene are shown in Figure 3, Figure 4 and Figure 5. It is noticeable that the ground-background is the most complex compared with other two situations. The road surface with high gray-scale level leads to a very strong background edge, which causes great interference for detecting the real target. For the scene of sea-background, the warship target usually moves nearby the sea-level line. The clutters caused by the clouds and lighthouses will also increase the difficulty of small target detection. On the other hand, the imaging noise is very high in this situation, as shown in the sequence Sea-1. In the sky-background situation, the target energy is the lowest among these three situations. Specifically, the average signal-to-clutter ratio of sequence Sky-4 is less than zero, which indicates a very challenging task to detect the small target. In other sky-background sequences, the targets are submerged by the clouds from time to time.

4.2. Quantitative Comparison

To evaluate the detection performance of the proposed GPCP model, we first report the ROC curves of 9 infrared small target detection algorithms on the whole dataset, as shown in Figure 6. It can be observed that the curve of GPCP is the closest to the upper left corner, which means for any given false alarm rate, the proposed GPCP model achieves the highest accurate detection rate, and for any given detection rate, the proposed GPCP model achieves the lowest false alarm rate. The first line in

Table 3. The table shows the AUC of 9 small target detection algorithms in the whole dataset and 3 different background categories. For each category, the best results are marked in the red color.

	OURS	ASTTV-NTLA	PSTNN	TVPCP	IPI	NRAM	RIPT	MPCM	SRSTT
All	0.999994	0.7391	0.999992	0.99979	0.9606	0.9485	0.9916	0.9945	0.9147
Ground	0.999999	0.9933	0.999998	0.999993	0.999999	0.9008	0.999999	0.9775	0.90
Sea	1	0.5355	0.999964	1	1	1	0.9785	1	0.8388
Sky	0.999998	0.6072	0.999998	0.9991	0.9132	0.9603	0.9913	0.9995	0.9346

also shows the proposed GPCP model has the highest area under curve (AUC) value in all 9 algorithms, PSTNN is second only to our proposed model. That is to say, the proposed GPCP model has a relatively good detection performance on the whole dataset.

We also report the ROC curves on 3 different background categories: the ground background, the sea background and the sky background, which are illustrated in Figure 6b, Figure 6c and Figure 6d, respectively. Combined with the AUC scores shown in Table 3, we can see that the GPCP curve is the closest to the upper left corner and corresponds to the largest AUC value, which indicates the proposed model has the best detection performance in the ground background. RIPT and IPI are the second- and third-best algorithms in this situation. For the sea background, most algorithms perform well. As the third line of Table 3 shows, the AUC values of IPI, MPCM, NRAM, GPCP and TVPCP are 1. Yet it is worth noting that the ASTTV-NTLA has a relatively small AUC value in this situation. For the sky background, from Figure 6d and the forth line of Table 3, we can see the proposed method and PSTNN have the best detection performance. The AUC values of these two models are the same. The main difference of these two methods lies in the GPCP model performs better in suppressing false alarms and PSTNN performs better in detection rate. Figure 6d can prove this point, in the case of lower false-alarm rate, the proposed GPCP model has a higher accurate detection rate; in the case of a higher detection rate, the PSTNN achieves a lower false-alarm rate.

To analyze the algorithm performance more specifically, the signal-to-clutter gain (SCRG) and background suppression factor (BSF) of 9 algorithms on each individual sequence are also calculated, as shown in

Table 4. The table shows signal-to-clutter ratio gain (SCRG) and background suppression factor (BSF) of 8 small target detection algorithms on 3 ground-background sequences. For each sequence, the best results are marked in the red color.

		OURS	ASTTV-NTLA	PSTNN	TVPCP	IPI	NRAM	RIPT	MPCM	SRSTT
Ground−1	SCRG	23.80	13.79	3.79	0.81	1.19	2.39	0.95	0.69	5.63
	BSF	5402	126.56	9.73	7.13	10.38	18.51	9.45	11.30	10.35
Ground−2	SCRG	0.3038	0.22	0.03	0.004	0.06	0.002	0.13	0.18	0.28
	BSF	10.16	8.54	2.64	2.48	4.22	10.54	7.15	3.38	4.35
Ground−3	SCRG	5.55	5.10	3.03	1.44	2.51	4.43	2.48	0.12	5.50
	BSF	9.04	9.32	5.45	2.44	6.00	7.56	9.29	2.93	2.34

,

Table 5. The table shows signal-to-clutter ratio gain (SCRG) and background suppression factor (BSF) of 8 small target detection algorithms on 3 sea-background sequences. For each sequence, the best results are marked in the red color.

		OURS	ASTTV-NTLA	PSTNN	TVPCP	IPI	NRAM	RIPT	MPCM	SRSTT
Sea−1	SCRG	30,498	0	42.07	943.90	3.82	25.53	4.69	1.38	6099
	BSF	15,784	0	124.55	3146	10.06	58.24	9.69	4.89	14,376
Sea−2	SCRG	15,659	0	12,144	16.60	18.36	11,399	13,648	4.89	0.25
	BSF	13,438	0	13,438	26.71	41.00	1	13,438	15.63	10,458
Sea−3	SCRG	97.74	2323	554.47	17.28	19.46	34.97	2318	1.15	6.48
	BSF	205.78	6461	211.13	14.15	14.46	24.57	1079	2.63	4.18

and

Table 6. The table shows signal-to-clutter ratio gain (SCRG) and background suppression factor (BSF) of 9 small target detection algorithms on 6 sky-background sequences. For each sequence, the best results are marked in the red color.

		OURS	ASTTV-NTLA	PSTNN	TVPCP	IPI	NRAM	RIPT	MPCM	SRSTT
Sky−1	SCRG	17.02	1.52	13.91	1.33	12.63	2.49	3.88	0.34	2.34
	BSF	17.26	3.56	15.39	1.27	15.08	17.58	11.92	8.91	2.31
Sky−2	SCRG	15.89	13.1	13.23	1.42	4.67	6.83	0.02	0.04	7.92
	BSF	20.28	8.67	8.46	1.17	14.18	10.79	8.93	5.91	5.23
Sky−3	SCRG	8.11	4.56	9.39	7.67	7.41	18.57	0.01	0.15	0.98
	BSF	10.99	1.16	16.79	10.74	10.42	1066	172.93	19.22	11.66
Sky−4	SCRG	25.46	0	7073	27.23	30.79	4105	28.13	0.17	59.50
	BSF	28.55	0	1078	16.19	26.63	1023	20.37	4.95	17.33
Sky−5	SCRG	1991	0	125	0.01	8.16	29.68	8.77	0.69	7.13
	BSF	1735	0	95.22	3.80	9.36	38.33	7.68	0.88	32.59
Sky−6	SCRG	15.87	0.006	12.09	13.90	13.66	14.96	6.59	2.19	3.45
	BSF	20.76	8.42	8.53	10.63	10.76	17.57	8.79	9.81	2.71

. A good algorithm should achieve high SCRG and high BSF, which represent the performance on target enhancement and background clutter suppression, respectively. From Table 4, we can see that our proposed method achieves the highest SCRG on all three sequences in the ground background. In Ground−1, the proposed model also achieves the highest BSF value. In Ground−2 and Ground−3, BSF values of the proposed model are a bit lower than the NRAM and ASTTV-NTLA, which means the remained background pixel value of our model is a bit higher than the other two models. Based on the fact that the proposed model has the highest SCRG values in these two sequences, we can conclude that our model still achieves the largest contrast between the target and background. Table 5 shows the algorithm performance on three sequences in the sea background. GPCP model achieves the highest values of SCRG and BSF values in sequence Sea−1 and Sea−2. ASTTV-NTLA model achieves the highest values of SCRG and BSF in sequence Sea−3 due to its multi-frame and TV model, but has a poor performance in Sea−1 and Sea−2. That is because the ASTTV-NTLA model is not suitable for infrared small target detection with low moving speed. The same experiment results also appear in Sky−4 and Sky−5. ASTTV-NTLA model misses all the targets in these two sequences due to the low moving speed. PSTNN also performs well in sequence Sea−3, especially in the SCRG value. This is due in large part to the usage of structure tensor. A prior weight representing the corner feature is added to the target component and makes the extracted target brighter. By comparison, target intensity values of the proposed model are a little bit lower than PSTNN. However, from Table 3, we can see the AUC values of the proposed model is higher than PSTNN. In the sky background, the proposed GPCP model has the largest SCRG and BSF values in sequence Sky−1, Sky−2, Sky−5 and Sky−6. NRAM and PSTNN achieves the highest SCRG and BSF in Sky−3 and Sky−4, respectively. From Figure 5, we can see the targets in Sky−3 are relatively large and has a gray variance. The detection results of NRAM only reserve several pixels in the target center position. By comparison, the proposed model has more pixels of targets in the detection results and is more coincide with the real target. For sequence Sky−4, there are some residual pixels with low values remained in the proposed model compared with PSTNN. The reason lies that in this sequence, the gray-scale difference of the local background regions is not very great. Current group strategy which employs the complexity difference for patch grouping is disabled. Therefore, in sequence Sky−4, GPCP model is almost equal to its baseline IPI. The experiment results in Table 6 also shows the performance of GPCP model is similar to IPI model.

4.3. Qualitative Comparison

To have a more direct and deeper impression on the effect of each method, we select several representative results as well as the corresponding three-dimensional surface results from each type of detection scene for illustration. The small target detection task in the ground background is the most difficult situation. Three representative examples are shown in Figure 7. The detection for sequence Ground−1 is relatively simple because there is a large contrast between the target and its surrounding background. The most challenging factor is the interference caused by the road edge. The proposed GPCP model achieves the best detection performance in this situation. Meanwhile, from the three-dimensional surface results, we can see the proposed method gets the best performance on background suppression among all the 9 algorithms. The detection task for sequence Ground−2 and Ground−3 is more difficult compared with Ground−1. The original images of these two sequences show the targets have been basically submerged into the background, in addition, there are many background clutters with similar appearance to the small dim targets. In these two sequences, only the proposed GPCP model successfully detects the target while suppressing the background clutters. Other methods leave many false alarms in the detection results, as the green box and three-dimensional surfaces show.

Figure 8 illustrates the detection results in the scene of sea background, in which the most challenging factor lies in the background interference caused by the lighthouse and water grass shelter. In sequence Sea−1, it can be seen that the ASTTV-NTLA, IPI, MPCM and PSTNN models have a poor detection performance, where the gray-scale value of the lighthouse outline is even larger than the real target after detection. For the RIPT method, the imaging noise has a certain impact on the detection performance, in which many false alarms are remained in the background part. By comparison, the NRAM, TVPCP and the proposed methods are good at suppressing the strong background edges as well as the imaging noise. In sequence Sea−3, the gray value of the water grass is larger than the target, in addition, both of these two parts have sharp forms in appearance, making the small target hard to be distinguished from the background. In the detection results of SRSTT and TVPCP methods, there are many residual clutters remaining in the background, as the green boxes show. By comparison, the PSTNN, RIPT, NRAM and the proposed methods achieve satisfactory results.

Three representative detection results in the scene of sky background are shown in Figure 9. The target in sequence Sky-2 has a very low contrast compared with the background. In this situation, the ASTTV-NTLA and TVPCP methods fail to suppress the background noise and cannot find the real target. By comparison, the NRAM, PSTNN, RIPT, SRSTT and the proposed GPCP model achieve a better performance on target enhancement. We can see that the decomposition results of these four methods all correspond to a low level background noise. Sequence Sky-5 shows the small target detection results of 9 methods in the case of bright heavy cloud. There are many strong edges in the background part, especially in the top and right side of the image. As shown in Figure 9, the detection results of the IPI, MPCM, PSTNN, RIPT, Tophat and TVPCP methods remain having many background clutters, which are easily to confuse with the real small target. Only the NRAM and the proposed GPCP methods can extract the target while suppressing the clutters simultaneously. The IPI, PSTNN, RIPT and NRAM models are all based on the PCP theory and carry out a global low-rank decomposition to remove the background clutters. By comparison, the proposed group low-rank and sparse decomposition model has a significant effect to cope with the situation with strong background clutters.

4.4. Influence of Grouping Criteria on Our Method

As mentioned above, the proposed model needs to divide the full image into several groups for image decomposition. In this part, an ablation experiment is carried out to discuss the effectiveness of the grouping criteria of the customized group low-rank strategy. The proposed model takes both of the gray-scale level and the clutter level into consideration and divides the data matrix into three groups. By comparison, the first contrast experiment is designed to only use the gray-scale information and divide the data matrix into a bright part and a dark part, which is named as GPCP−Gray. The second contrast experiment employs the variance information and divides the data matrix into a uniform part and a cluttered part, which is named as GPCP−Var. The IPI model, which decomposes the entire image data into a low-rank component and a sparse component plays as the baseline method. The ROC curves of these four experiments are shown in Figure 10.

In this experiment, the GPCP−Var model has the worst performance, especially in the sky background situation. By comparison, the GPCP−Gray model has a slight decline in the ROC curves compared with the proposed GPCP model, while performs better than the global regularized low rank and sparse decomposition model (IPI). This suggests that dividing the data matrix into two parts with different brightness level has a positive influence on background suppression in the PCP process. In addition, from the ROC results of the GPCP−Gray model and the proposed GPCP model, we can see that extracting the cluttered image data and decomposing this part independently can further improve the detection performance.

4.5. Computation Complexity Analysis

The computation complexity of each comparison model is shown in

Table 7. The table shows the computation complexity of 9 small target detection algorithms.

	OURS	ASTTV-NTLA	PSTNN	TVPCP	IPI	NRAM	RIPT	MPCM	SRSTT
Complexity	$\mathrm{O}\left(m(n_1^2+n_2^2+n_3^2)\right)$	$\mathrm{O}(MNLlog(MNL\left)\right)$	$\mathrm{O}(n_1{n}_2{n}_{3}log\left(n_1{n}_2{n}_3\right))$ $+\mathrm{O}\left(n_1{n}_2^2[(n_3+1)/2]\right)$	$O(M{N}^2+N^4)$	$\mathrm{O}\left(m{n}^2\right)$	$\mathrm{O}\left(m{n}^2\right)$	$\mathrm{O}\left(n_1{n}_2{n}_3(n_1{n}_2+n_2{n}_3+n_1{n}_3)\right)$	$\mathrm{O}\left(L^{3}MN\right)$	$\mathrm{O}\left(L{n}_1(n_2^2+n_{3}log((n_2+1)/2))\right)$

. Suppose the image size is $M\times N$ and the patch image size is $m\times n$. The computation cost is $\mathrm{O}\left(L^{3}MN\right)$. The major time-consuming part is saliency map calculation, in each scale, the computation cost is $\mathrm{O}\left(L^{2}MN\right)$. The total cost in all scales is $\mathrm{O}\left(L^{3}MN\right)$. For the patch-based models, including IPI, NRAM and the proposed GPCP model, its computation complexity mainly comes from the SVD decomposition. For an $m\times n$ patch matrix, the computation complexity of SVD is $\mathrm{O}\left(m{n}^2\right)$. For the patch-tensor models, including RIPT, PSTNN, ASTTV-NTLA and SRSTT, the main time-consuming part is the SVD decomposition progress in the frequency domain. For a tensor size with $n_1\times n_2\times n_3$, the computation complexity of SVD is $\mathrm{O}(n_1{n}_2{n}_{3}log\left(n_1{n}_2{n}_3\right))$, the computation complexity of FFT is $\mathrm{O}\left(n_1{n}_2^2[(n_3+1)/2]\right)$. They are faster than the SVD decomposition of patch-based models, which are calculated in the spatial domain. The TVPCP model is a little time consuming due to the matrix inversion calculation. It is worth noting that the proposed GPCP model is faster than its baseline IPI model. From the grouping criteria, we can see that $n_1+n_2+n_3=n$. Based on the fact that $n_1^2+n_2^2+n_3^2<n^2$, the computation cost of GPCP is lower than IPI. In our experiments, for a $256\times 256$ image, GPCP needs 5.9 s to obtain the detection result. By comparison, IPI needs 11.9s. The speed increases doubly.

5. Conclusions

In this paper, a novel group regularized low-rank and sparse decomposition model is proposed for infrared small dim target detection. The traditional decomposition-based models are usually sensitive to strong edges and background clutters due to the ignorance of data structure diversity. The proposed method is able to solve this problem by using a customized group low-rank strategy. Firstly, it exploits different singular value thresholds for the low-rank decomposition of image groups corresponding to different complexity. Then, the newly designed group low-rank regularization is integrated with the sparse constraint for background and target separation, in which more prior information related to data structure can be utilized in the decomposition process. Experimental results on 3 different detection scenes, which includes 12 sequences, have shown the priority of the proposed in terms of probability of detection, false alarm rates, target enhancement and background suppression factors.

There also exist some issues worth considering. For example, we use the brightness and gray-scale variance to divide patches into groups, other strategies such as image feature-based methods can be further considered for patch grouping. This method is also time consuming, especially in the background solving process, other background modeling methods need to be explored in the future work.