Infrared Small Target Detection Based on Non-Overlapping Patch Model via l₀-l₁ Norm

Abstract

Infrared small target detection technology has sufficient applications in many engineering fields, such as infrared early warning, infrared tracking, and infrared reconnaissance. Due to the tiny size of the infrared small target and the lack of shape and texture information, existing methods often leave residuals or miss the target. To address these issues, a novel method based on a non-overlapping patch (NOP) joint l₀-l₁ norm is proposed with the introduction of sparsity regularized principal component pursuit (SRPCP). The NOP model makes the patch lighter in the first place, reducing time consumption. The adoption of the l₀ norm enhances the sparsity of the target, while the adoption of the l₁ norm enhances the robustness of the algorithm under clutter. As a smart optimization method, SRPCP solves the NOP model fittingly and achieves stable separation of low-rank and sparse components, thereby improving detection capacity while suppressing the background efficiently. The proposed method ultimately yielded favorable detection results. Adequate experiment results demonstrate that the proposed method is competitive in terms of background suppression and true target detection with respect to state-of-the-art methods. In addition, our method also reduces the computational time.

1. Introduction

The infrared imaging system is widely applied in the field of target detection. The system has many advantages: simple structure, high resolution, and functionality under all weather conditions. Infrared small target detection is the key technology of the infrared imaging system. In the field of infrared small target detection, information such as shape and texture are difficult to use, which brings great challenges to the detection. Infrared small target detection needs to detect targets of small size, low energy, and strong background clutter. With the widespread application of infrared imaging systems in various areas (e.g., resource exploration, city management, precision guidance), infrared small target detection methods have been developed in a variety of ways. We often divide them into single-frame detection methods and multiple-frame detection methods. The single-frame detection methods not only detect the target, but also serve as an information source for multiple-frame detection [1,2]. This paper is concerned with single-frame detection methods.

In the single-frame detection methods, the robust principal component analysis (RPCA) [3] method significantly improves the detection capability and reduces false alarms. Through low-rank matrix factorization, RPCA converts the target detection problem into an optimization problem. Low rank and sparsity are important concepts in RPCA. A low rank means that the correlation between the vectors in the matrix is high. Therefore, the full information of a low-rank matrix can be recovered from a small number of vectors. Sparsity measures the number of non-zero elements in a vector. When there are rare non-zero elements in a vector, the vector is said to be sparse. Extended to a matrix, if there are only a few non-zero elements in the matrix, we call the matrix a sparse matrix.

In an infrared image with a small target, it can be assumed that the background is low rank and the target is sparse. Common backgrounds of infrared small target images like clouds, waves, and the ground are continuously transformed so they have a spatial correlation. The targets are randomly distributed, and from the perspective of the entire image, they have spatial sparsity. Therefore, the low-rank matrix containing the background and the sparse matrix containing the target can be obtained via the method of low-rank and sparse matrix decomposition, respectively. In addition, the target detection can be completed by the matrix recovery algorithm.

Since its introduction, RPCA theory has been widely applied in the field of infrared small target detection with remarkable success. However, RPCA methods still face problems of long operation time, poor anti-noise performance, and target loss. As an adaptable theory, it is of great significance if we can increase the speed of RPCA operation while improving its detection results. The purpose of this paper is to make effective improvements to the above-mentioned problems of RPCA methods.

In this paper, a novel approach based on the NOP model with SRPCP joint l₀-l₁ norm was proposed. We apply SRPCP on the constructed NOP for the infrared small target detection problem. The contribution of this paper is to improve the performance of background suppression while reducing target loss. In addition, the rapidity of detection has also been improved. The direct adoption of the l₀ norm was introduced to better depict the targets, while the l₁ norm is more robust than the Frobenius norm in the face of a complex background. Through the low-rank and sparse matrix decomposition process via SRPCP, we successfully separated the background component and target component. With the smart structure of NOP and successful integration of SRPCP, the proposed method performs outstandingly in the detection result.

2. Related Works

Many single-frame algorithms have been proposed in recent years in the field of infrared small target detection. In this paper, these methods are divided into two groups: traditional methods based on characteristics of targets and those based on RPCA theory. There are obvious differences between the traditional methods and RPCA methods in terms of solution approach and detection performance. The traditional approaches take a localized view of the detection. These methods argue that the presence of the target destroys the local correlation between background pixels, thus enabling detection by designing the corresponding filter operators. Alternatively, from the perspective of target saliency, local contrast is studied to achieve detection. On the contrary, RPCA solutions take a holistic view. The RPCA method treats the infrared image as a superposition of the whole background and the targets. The background can be considered as low-rank, while the small targets are sparse. By transforming the target detection problem into a problem of decomposing the low-rank and sparse components of the matrix, RPCA solutions achieve target detection, which is considered from the whole image. In view of the detection performance, the traditional methods are fast and can increase the brightness of the target area, but they have poor background suppression ability, which leaves lots of residual; RPCA methods suppress the background well while enhancing targets, but they are not as fast as traditional approaches.

2.1. Traditional Methods

Traditional single-frame infrared small target detection is mainly divided into two categories. One is based on a background-based assumption, which believes that the local background is uniform. The appearance of the target destroys this uniformity; thus, the correlation between background pixels is broken. By exploiting this feature, target detection is accomplished. This type of algorithm includes the top-hat filter (Tophat) [4,5], max-mean filter [6], the two dimensional adaptive least mean square algorithm [7,8]. They have very fast computational speed and can detect targets in real-time. But they perform poorly in background suppression. The other is based on target saliency, via importing the human visual system [9] and calculating the local contrast to obtain a saliency map to detect infrared small targets. These methods include local contrast measure [10], improved local contrast measure (ILCM) [11], etc. Motivated by the local contrast method, the Laplacian Gaussian operator [12] is incorporated into the method due to its sensitivity to noise. To improve the detection performance, the multi-scale contrast method [13] and the weighted singular local contrast method [14] have been proposed. These types of methods enhance the true targets, while they are sensitive to strong clutters and edges.

2.2. RPCA Methods

John Wright et al. [3] proposed the RPCA algorithm in 2009. This algorithm solves the problem of recovering the low-rank matrix and provides theoretical support for the application of low-rank theory in target detection. The original objective function of PCA is:

$$\underset{A,E}{\min } rank(A)+\gamma {\Vert E\Vert }_0,$$

(1)

where A represents the low-rank part, E represents the sparse part, and gamma represents the coefficient of the sparse part, which can be tuned.

John Wright et al. proposed to solve the problem by replacing the rank of A with the matrix nuclear norm of A and the l₀ norm of E with the l₁ norm of E. The optimized objective function is as shown in Equation (2):

$$\underset{A,E}{\min }{\Vert A\Vert }_{*}+\lambda {\Vert E\Vert }_1,$$

(2)

where * indicates the matrix nuclear norm, the sum of the singular values of the matrix; 1 indicates the l₁ norm, the sum of the absolute values of each element of the matrix; lambda represents the coefficient of the sparse part, which can be adjusted.

In order to solve Equation (2), John Wright et al. used the Frobenius norm to constrain the loss term of the objective function. After mathematical derivation, Equation (3) was used by John et al. as the final format to solve this optimization problem.

$$\underset{D=A+E}{\min }{\Vert A\Vert }_{*}+\lambda {\Vert E\Vert }_1+{\Vert D-A-E\Vert }_F^2,$$

(3)

where D is the input image matrix and F represents the Frobenius norm.

RPCA has distinctive characteristics. For the infrared small target detection problem, since RPCA does not need to take into account the target’s morphology and location information, it has a wide range of applications and can achieve robust detection in complex scenarios. In addition, RPCA directly strips the background by low-rank and sparse matrix decomposition, which provides excellent background suppression and yields a clean target image. However, RPCA also has some drawbacks. The algorithm structure of RPCA is complex and requires lots of iteration; thus, the performance is inferior to the traditional method in terms of speed. It is also difficult for RPCA to eliminate the interference of clutter that is brighter than the target. Based on the RPCA theory, a series of infrared small target detection methods have been derived. Lin et al. proposed the inexact augmented Lagrange method (IALM) [15] to solve the low-rank matrix recovery problem, and weighted tensor robust principal component analysis (WTRPCA) [16] adopted IALM to accomplish infrared small target detection. Infrared patch image model (IPI) [17] introduced a patch model by sampling the full picture information step by step through a fixed-sized sliding window, successfully separating the low-rank part and sparse part of the infrared image. The Markov on Gaussian operator [18] combined Markov random field (MRF) on the basis of IPI and reduced the background clutter through the local correlation characteristics of MRF. The non-negative image patch model [19], the non-convex rank approximation method (NRAM) [20], and the non-convex optimization of l_p norm constraint model (NOLC) [21] are successively proposed under the inspiration of IPI. Dai et al. [19] employed the singularity minimization theory, and NRAM combined alternating direction method of multipliers [22] and differences of convex programming [23]. Zhang et al. [21] invoked the l_p norm for optimization. Zhang et al. [24] introduced the tensor nuclear norm and the weighted l₁ norm, and solved the problem on the three-dimensional patch. Inspired by this, Guan et al. [25] improved the low-rank patch background tensor constraint by non-convex tensor rank second generation, enhancing the robustness of the tensor algorithm. In addition, there are areas where morphological theory and RPCA theory can be fused, and Zhu et al. [5] successfully fused the Tophat regularization operator into a low-order tensor complement, exploiting knowledge of the prior target structure.

2.3. Motivation

Influential methods like IPI, NRAM, and NOLC are all based on RPCA theory. However, they have certain problems in the details of infrared small target detection. Among them, IPI introduces the patch model, but it is too redundant and leads to long detection time. On the other hand, IPI adopts the Frobenius norm and has poor suppression of strong edges. Methods like NRAM and NOLC, although they suppress the strong edges by adding constraints, have the problem of target loss.

It can be seen that the redundant patch is very time consuming, and the constraint on the strong edge leads to the target loss problem. We conclude that these methods have a common problem: the effectiveness and timeliness of infrared small target detection cannot be maintained at a high level at the same time.

The time consumption problem results from the above methods having redundant patches. Adjusting the patch size and step size to better detect targets is one research direction [2]. Wang et al. [26] introduced the idea of non-repeating sampling into the process of constructing a patch image; Zhou et al. [27] reduced the size of the patches while maintaining the classical step size, resulting in a reduction of the overlap. Their work has contributed to improving the performance of the patch image. There are also ways to change the sampling method for fast speed; Li et al. [28] used an observation vector to project the original image and obtain an observation matrix for detection; Reference [24] adopted a three-dimensional patch to target very complex scenes.

The detection effect problem results from the above algorithms having accepted the relaxation of the l₀ norm, while the Frobenius norm cannot endure clutter and noise. They do work on variants of the l₁ norm. In order to solve this problem, we investigated the method of expressing sparse terms based on the l₀ norm and its variants.

The l_p norm, log-sum [29], and weighted l₁ norm [30] combined with the minimum optimization are employed to replace Equation (1). Chartrand [31] proposed a proximal p norm to replace the l₁ norm. Sun et al. [32] proposed truncating the l₁ norm and applying it to the singular values of the sparse and low-rank parts. Under the influence of these studies, Zhou et al. [33] proposed Equation (4) based on SPCP [34], which is very close to Equation (1):

$${\Vert A\Vert }_{*}+\beta {\Vert E\Vert }_0+\lambda {\Vert D-A-E\Vert }_F^2$$

(4)

In Equation (4), the l₀ norm is adopted on the sparse term, which is the biggest difference from Equation (3). In addition, Equation (4) adds an adjustable coefficient to the last term. Further, Liu et al. [35] proposed SRPCP, which introduces a regular term to extract the low-rank part of the picture, and proposed a direct l₀ norm for enhancing the sparse term. The idea of SRPCP that directly adopts the l₀ norm without approximation coincides with the mentioned problems: fast detection and pretty detection results.

SRPCP [36] is a new model based on the robust principal component analysis theory. Different from the traditional relaxation method, SRPCP is inspired by the model proposed by Zhou et al. [33], and adopts the “l₀ norm” to constrain the sparse part of the matrix. Further, for the shortcomings of the Frobenius norm in detecting infrared small targets, SRPCP restricts the loss term of the target function by using the l₁ norm instead. The objective function proposed by the SRPCP model is shown in Equation (5) [35]:

$${\Vert A\Vert }_{*}+\beta {\Vert E\Vert }_0+\lambda {\Vert D-A-E\Vert }_1$$

(5)

Compared with Equations (1)–(4), Equation (5) adopts the structure of the “l₀-l₁ norm” to perform sparse decomposition of matrix addresses. The advantage of applying SRPCP reduces the relaxation error caused by the relaxation of the l₀ norm and significantly improves the recovery performance of the low-rank part [36]. It shows robustness against input-intensive noise. In SRPCP, the objective function is Equation (6):

$$G(A,E)={\Vert A\Vert }_{*}+\beta {\Vert E\Vert }_0+\lambda {\Vert D-A-E\Vert }_1$$

(6)

The solution method adopts the heuristic method [36]. The procedure is mainly divided into two steps [35], which will be specifically explained in the solution of the NOP model.

In summary, in order to overcome the shortcomings of models based on PRCA and its series of derivatives, a non-overlapping sliding window model via l₀-l₁ sparse regularization is proposed. The detailed contributions of this paper are as follows.

(1)

In order to reduce the redundancy of patch image, an adaptive non-overlapping patch sampling model is proposed. The proposed model has a lightweight structure, which adjusts the size of the model according to the size of the input image automatically. Compared with the fixed patch size methods, NOP is more flexible when facing different sizes of images. It can greatly reduce the detection time while maintaining the high detection effect. We adopt NOP in many different scenes and detect small targets successfully. NOP provides a simple frame to detect infrared small targets.

(2)

SRPCP is applied to the previously proposed NOP framework. To the best of our knowledge, SRPCP is first applied in the area of infrared small target detection. A direct objective function was adopted. The background reconstruction effect and noise immunity are optimized via the feature of the l₀-l₁ norm. Sparse parts are represented via the l₀ norm, and the objective function is constrained via the l₁ norm. NOP has achieved pretty results in infrared small target detection.

The rest of the paper is organized as follows. In the third part, we briefly introduce the SRPCP theory. In the fourth part, the proposed NOP model and its solution method are presented in detail, showing the complete small target detection process. A series of relevant experiments and experimental results are presented and discussed in detail in Section 4. Finally, the conclusion of this paper is given.

3. Methodology

We briefly specify our contributions at first. Then, we give a detailed description of the proposed method.

For the infrared small target detection issue, the small targets are sparse in the background. SRPCP is extremely advantageous and targeted to this low-rank and sparse matrix decomposition problem. SRPCP enhances the sparsity of the sparse component by directly adopting the l₀-norm, so that small sparse targets are more easily detected. Furthermore, SRPCP improves the robustness of this optimization method under clutter by adopting the l₁-norm instead of the Frobenius norm. It can be seen that SRPCP specifically improves the detection of sparse targets and reduces the target loss phenomenon.

Therefore, we creatively introduced this optimization method into the field of infrared small target detection and achieved favorable detection results. We have made several contributions to make this optimization method SRPCP more fit for our target detection task.

Firstly, in order to construct a low-ranking suitable environment for SRPCP operation, we designed the NOP, as described in Section 3.1. By solving the NOP, SRPCP was successfully used in the field of infrared small target detection.

In addition, we have made important adjustments to the implementation of SRPCP, which was not initially an optimized method for the IR small target domain, and the lambda beta setting in Equation (6) is not suitable for small target detection. We provide an analysis of these important parameters, which is presented in Section 4.3.

After SRPCP is executed, a reconstruction of the image is needed to complete the detection. We have designed the inverse process of NOP construction to make SRPCP perfectly embedded in small target detection.

3.1. The NOP Model

The IPI [17] algorithm proposed dividing the matrix into several small patches and then arranged the elements in the small patches to obtain a patch image with low-rank characteristics, which can be used for low-rank sparse decomposition of the matrix. But the patch image is often too large, resulting in very slow operation. Inspired by previous work on patch parameter settings, we propose an adaptive non-overlapping sliding window sampling method, as shown in Figure 1. The new patch image (non-overlapping patch) in Figure 1 is formed.

In NOP, the window size and the step size are the same; that is, the entire image is sampled without an overlap. The width of the patch is equal to the horizontal step size, and the height of the patch is equal to the vertical step size. The patch size and step size are set as $x\_width=x\_step=\sqrt{m}/p$, $y\_height=y\_step=\sqrt{n}/p$, where m and n denote the width and the height of the original image, respectively, and p represents an empirical parameter. The value of p is determined by the low-rank property of the NOP model. According to Reference [17], the rule is that the greater the percentage of the biggest singular values, the better the low-rank property. We chose p from 0.1 to 5, and we compared the corresponding low-rank property curves of each value. We found out that when p is 0.1, the low-rank property is the lowest, and the target cannot be detected; when p is 2, the low-rank property is the highest, and the detection effect is excellent; when p is 5, the low-rank is reduced and the detection effect becomes worse, and the detection time becomes longer. Therefore, we set p = 2.

The vectorization is achieved by operating on each patch. Each patch is a grayscale matrix consisting of a number of column vectors. We splice all these column vectors into one large column vector from left to right. Starting with the first column vector, each column vector is spliced at the end of the column vector to its left. In this way, each patch is vectorized into a big column vector. By arranging these large column vectors from left to right, the NOP grand matrix is formed.

In order to verify whether the NOP has the condition of low rank and sparse matrix decomposition, we compared the low-rank property of NOP with the IPI model [17] in Figure 2.

In the above figure, we can find out that the low-rank property of NOP and IPI is similar. Both of their singular values drop very fast, which indicates that they are low-rank. Compared with IPI, NOP maintains the low-rank property while reducing the size of the patch.

Therefore, we can still perform RPCA works based on the NOP model. In addition, the adaptability of the proposed method comes from the construction process of NOP. According to the previously mentioned patch sampling formula, unlike those methods with fixed patch size, NOP can adaptively select patch size with the size change of original infrared images. For example, when the original infrared image is 256 × 256, the patch size should be 8 × 8; when the original infrared image is 196 × 196, the patch size should be 7 × 7. Therefore, the selection of patch size in NOP is completely automatic, and users no longer need to manually select the patch size and step size.

3.2. The Surrogate of l₁ Norm

According to RPCA theory, it can be seen that the l₁ norm represents the sum of all elements of the matrix, that is, the sum of non-zero elements, and the l₀ norm is the number of non-zero elements of the matrix. According to RPCA theory, the l₁ norm is a slack representation of the l₀ norm, and the l₀ norm problem is also for NP-hard problems, which is difficult to solve, so most methods under the RPCA framework use l₁ norm to represent the sparseness of the matrix item.

Generally, the l₀ norm is not easy to express. Zhou [34] proposed expressing the l₀ norm for pedestrian detection through the l₁ norm, and successfully applied the sparse representation of the l₀ norm to practice. Therefore, by replacing the l₁ norm with the l₀ norm in G (A, E) in Equation (6) [36], this paper successfully represents the sparse part.

The adoption of l₀ brings advantages. Firstly, the l₀ norm is the original form of RPCA, and it describes the RPCA problem accurately. According to Reference [3], the l₀ norm is the best surrogate for the sparse part of a matrix, but it is difficult to solve. Now that the l₁ norm is the relaxation of the l₀ norm, the l₁ norm is applied to replace the l₀ norm normally, while this surrogate brings a relaxation error. Compared with directly using the l₁ norm, Reference [35] proposed the genuine l₀ norm to enforce the sparsity of the sparse part. According to References [35,36], the usage of the l₀ norm strengthens the sparse component, which can make the target easier to detect, because our method detects small targets by separating low-rank background and sparse components. When the sparse component is enforced, we can obtain the target more effectively. In addition, this surrogate reduces the estimation error of using the l₁ norm.

3.3. The Surrogate of Frobenius Norm

According to the definition of the Frobenius norm, it is a convex function with convex points, which is easy to be derived and calculated. Therefore, the Frobenius norm is widely adopted in RPCA and acts as a constraint of the objective function. However, the Frobenius norm is particularly sensitive to noise and irregular clutter [35,36,37,38], so it cannot guarantee detection robustness under complex backgrounds. In order to solve these defects caused by the Frobenius norm, this paper selects the l₁ norm [35], which is not sensitive to noise and clutter, to constrain the objective function, so as to optimize the detection stability of the algorithm in a complex background.

With the above surrogates, the objective function of NOP has been formed, as shown in Equation (7):

$$\begin{array}{l}\underset{\mathrm{A},\mathrm{E}}{\min } {\Vert A\Vert }_{*}+\beta {\Vert E\Vert }_0+\lambda {\Vert D-A-E\Vert }_1 \\ \mathrm{s}.\mathrm{t}. D=A+E\end{array}$$

(7)

Due to the characteristic of the l₁ norm, it brings great robustness to the detection process. According to Reference [36], the replacement of the Frobenius norm by the l₁ norm significantly improves the performance of the matrix recovery, which is foremost in our method. Usually, the Frobenius norm is used in the third term of Equation (8) due to its easy-to-solve feature. However, compared with the l₁ norm, the Frobenius norm is more sensitive to strong residuals and noises [36]. Infrared small targets are often under complex scenes, which have strong clutter and noises. Hence, the sensitivity of the Frobenius norm to noises and clutters will harm the detection result. For example, Reference [17] chose the Frobenius norm as its noise term, and its detection results have marginal residuals that cannot be suppressed. Therefore, the mentioned surrogate brings robustness to the target detection process, which improves the performance of our method.

The detailed theoretical proof of the above surrogates can be found in References [35,36], respectively.

3.4. NOP Model and Its Solution

As described in Section 4.1, we obtain NOP through the proposed sampling process of the input image. The low-rank and sparse matrix decomposition of NOP is carried out by the SRPCP method. The optimization equation is shown in Equation (8).

The solution is mainly divided into two parts: Firstly, the NOP is preliminarily processed by the augmented Lagrange method (ALM), and the low-rank component named L_cpp is obtained. SRPCP [36] processing is performed on L_cpp to obtain the final low-rank component of NOP. The sparse part of the NOP is obtained by differencing the NOP with its low-rank part (according to Equation (7)). Further, after robust metrics completion (RMC) [39], the sparse and low-rank parts of the image are obtained. Through the rebuilding part, we complete the whole detection. In the algorithm below (Algorithm 1), D represents the NOP matrix and $\lambda$ represents an empirical factor.

Algorithm 1 SRPCP to NOP Model

Input: D, $\lambda$ Initialization: A = zeros(size(D)), E = zeros(Size(D)), $\mu =0.5/{\Vert sign(D)\Vert }_2$ While not converged do $L(A,E,Y,\mu )={\Vert A\Vert }_{*}+\lambda {\Vert E\Vert }_1+\langle Y,D-A-E\rangle +\mu /2\ast {\Vert D-A-E\Vert }_F^2$ $Y=Y+\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$\mu (D-A-E)$}\right.$ $\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$\mu $}\right.=\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$rho$}\right.\ast \raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$\mu $}\right.$ End while Lcpp is obtained. While not converged, do $L_{pre}=L_{cpp};S_{pre}=S$ $L=RMC(\exp {(\raisebox{1ex}{$-1$}\!\left/ \!\raisebox{-1ex}{$4$}\right.\cdot (D-S)}^2),\lambda ,L,S)$ End while Output: L, S

Figure 3 shows the process of our infrared small target detection method, and it is summarized as follows:

(1).

Construction of the NOP model. As the vectorization method mentioned in Section 3.1, by sampling the input image with the mentioned adaptive non-overlapping window from top to bottom and from left to right, we obtain a bunch of small matrices. Then the matrices are vectorized as columns of the proposed NOP model.

(2).

Low-rank and sparse matrix decomposition. Based on the NOP model, we separate its low-rank part and sparse part through the mentioned SRPCP method. According to Equation (7), we apply ALM on NOP at the beginning to get a low-rank part together with NOP as the input of SRPCP. Then the SRPCP is operated. After the SRPCP process, we get the low-rank part named A1 and the sparse part named E1.

(3).

Rebuilding and target detection. We rebuild the matrix by the inverse process of construction. The rebuilding process is accomplished by the inverse process of NOP vectorization. We reduce the column vectors of L and S to small matrices, respectively. By combining these small matrices in order, we obtained the final target matrix and background matrix, which completed the target detection. The background image is rebuilt from A1, the target image is rebuilt from E1. As shown in Figure 3, small targets are detected in the target image.

Complexity: We discuss the computational complexity of our method briefly. As shown in Figure 3, it is easy to see that the computational time of our method mainly consists of two parts: The SVD of NOP and SRPCP. Suppose m and n represent the width and the height of the original image and m > n. Note that NOP is based on sliding window sampling, according to Reference [17], and the computational complexity of NOP is $O(mn\log (mn))$. Respectively, the worst complexity of SRPCP [37] is $O(m{n}^2)$. Based on the aforementioned analysis, the computational complexity of our method is $O(mn\log (mn)+m{n}^2)$.

4. Experiments and Discussion

4.1. Preparation for Experiments

In this section, we establish our data set like Reference [17]. We adopt five sequences based on real scenes and two real imaging sequences. Sequence 1 to sequence 5 are simulated based on five typical real scenarios, which can be found in References [17,20,21], etc. These scenes have been widely used in infrared small target detection for years and have great application value. Sequence 6 and sequence 7 [40] are the latest new data acquired, which have very complicated backgrounds.

We briefly introduce the rules for generating our data set. A synthetic image can be generated [17] by the following Equation (8):

$$f_D(x,y)=\{\begin{array}{c}\begin{array}{l}\max (f_T(x_0,y_0),f_B(x_0,y_0))\\ x\in (1+x_0,n+x_0),y\in (1+y_0,m+y_0)\\ f_B(x,y)\hspace{1em}\hspace{1em}\mathrm{Otherwise},\end{array}\end{array},$$

(8)

where (m, n) represents the size of the image, $f_B(x,y)$ represents the background, $f_T(x,y)$ represents the target, $f_D(x,y)$ represents the synthetic image, and (x₀, y₀) represents the target coordinate.

The position of the target (x₀, y₀) is scattered in the background image using a randomly generated method [17]. In order to ensure that it is not disturbed by the edge of the image, this simulation limits the target not to be located on the image boundary, so that the data structure characteristics of the target can be fully analyzed. To make the characteristics of small targets more obvious, this simulation uses a $3\times 3$ target size for sequence generation. Together with the two brand new sequences from [40], our data set is obtained.

The detailed description of our data set is shown in

Table 1. Description of our sequences.

Sequence	Frame	Image Size	Background Description	Target Description
Seq1	100	150 × 200	Chaotic cloud	Small, medium brightness, scattered in chaotic clouds
Seq2	100	228 × 280	Sea surface and sky	Small, low brightness, scattered in the sea sky
Seq3	100	200 × 250	Massive, broken Cloud	Small and dim
Seq4	100	240 × 281	Manifold, broken Cloud	Small and dim
Seq5	100	140 × 220	Sea with waves	Small, low brightness
Seq6	100	256 × 256	Grass and shrubs	Normal brightness, moves quickly above the background
Seq7	100	256 × 256	Highway and surrounding forest	Very small, medium brightness, moving above roads and forests

:

The descriptions of the targets are drawn both from themselves and from the background in which they are embedded. We consider the target to be dim when the grayscale of the target is close to the area of the background where it is located. The target is thought to be of medium brightness when its grayscale is significantly higher than its background area. We also give descriptions of the real environment in which the targets are located, such as the sky, the ocean, and the clouds. The descriptions of the target velocities in sequence 6 and sequence 7 are derived from observations of their trajectories in multiple consecutive frames.

In this paper, a total of five algorithms of conventional methods and RPCA theoretical methods are compared with the methods of this paper, which are Tophat, ILCM, IPI, WTRPCA, and NRAM.

In addition, in order to evaluate the anti-noise ability of the algorithm, preparation for noise experiments are carried out. Since the target of sequence 3 is very close to the background, the detection is very difficult, so sequence 3 is selected as the basic sequence of the noise experiment, and a noisy picture is generated, as shown in Equation (9):

$$f_N(x,y)=f_D(x,y)+N(\mu ,{\sigma }^2)$$

(9)

The value of $\mu ,{\sigma }^2$ is shown in

Table 2. Description of sequence 8–10.

Sequence Name	$\mathit{\mu }$	${\mathit{\sigma }}^2$
Seq8	0	0.01
Seq9	0	0.02
Seq10	0	0.03

.

The mentioned noise sequences shown in Figure 4a–c represent sequence 8, sequence 9 and sequence 10, respectively.

4.2. Evaluation Metrics

Firstly, we introduce the target area, target neighborhood, and the relationship between them, as shown in Figure 5.

In Figure 5, the small target is inside the blue frame. Between the blue frame and the red frame is the neighborhood area. When evaluating the characteristic information of a small target, it is usually measured through the target area and the target neighborhood [17]. In this paper, the edge length of the neighbor area is set to 40.

4.2.1. SCRG

The signal clutter ratio gain (SCRG) is an important evaluation metric to assess the effect of the infrared small target detection algorithm [17,18,19,20,21,24]. It is defined as Equation (10):

$$SCRG=\frac{SC{R}_{out}}{SC{R}_{in}},$$

(10)

where SCR represents the target signal clutter ratio. It is defined as Equation (11):

$$SCR=\frac{\left|{\mu }_t-{\mu }_b\right|}{{\sigma }_b},$$

(11)

where ${\mu }_t,{\mu }_b$ respectively represent the gray average value of the target area and the target gray average value of the neighborhood; ${\sigma }_b$ indicates the gray standard deviation of the target neighborhood. The larger the SCRG value, the better the detection result [16,17,18,19,20,23].

4.2.2. BSF

The background suppression factor (BSF) is also an important evaluation metric for evaluating the effect of the infrared small target detection algorithm [16,17,18,19,20,23]. It is defined as Equation (12):

$$BSF=\frac{{\sigma }_{in}}{{\sigma }_{out}},$$

(12)

where ${\sigma }_{in},{\sigma }_{out}$ represent the standard deviation of the target neighborhood before and after detection, respectively. The larger the BSF value, the better the detection result [17,18,19,20,21,24].

4.2.3. ROC Curve

The receiver operating characteristic (ROC) curve is an important metric of the accuracy of target detection [17,18,19,20,21,24]. The horizontal axis of the ROC curve is the false-positive ratio (FPR), and the vertical axis is the true-positive ratio (TPR). The area enclosed by the curve and the horizontal and vertical coordinates is called area under ROC curve (AUC). The larger the AUC, the higher the detection accuracy of the algorithm [17]. The definitions of TPR and FPR are described as Equations (13) and (14).

$$TPR=\frac{\mathrm{number} \mathrm{of} \mathrm{true} \mathrm{detections}}{\mathrm{number} \mathrm{of} \mathrm{real} \mathrm{targets}}$$

(13)

$$FPR=\frac{\mathrm{number} \mathrm{of} \mathrm{false} \mathrm{detections}}{\mathrm{number} \mathrm{of} \mathrm{images}}$$

(14)

In this paper, all residual targets outside the target area are considered false alarms. FPR and TPR are calculated based on this assumption.

4.3. Parameter Analysis

According to Equation (7), $\lambda$ and $\beta$ are important parameters that determine the performance of the objective function. In sparse regularization theory, $\lambda$ is a penalty parameter, and its value is usually set as $\lambda =\alpha /\sqrt{\max (m,n)}$. It is used to determine the convergence of the algorithm. As the coefficient of the sparse term, the value of $\beta$ is set as $\beta =M/\sqrt{\min (m,n)}$. It is also related to the brightness of the target component of the final detection result of the algorithm. This paper takes alpha from 0.5 to 2.5 and M from 1 to 5, and examines the relationship between their values and the detection rate. The experimental results indicate that when alpha is 1 and M is 3, we obtain the best detection performance. The ROC curves with different values of alpha and M are shown in Figure 6 and Figure 7, respectively.

4.4. Qualitative Assessment

4.4.1. 2D Image

2D Image is a recognized qualitative metric for evaluating the quality of target detection due to its intuitive and concise expression [17]. Figure 8 screens out the representative images of each group of sequences and gives the 2D Image before and after detection under different algorithms. (a), (b), (c), (d), (e), (f), (g) are the experimental results of the original image, IPI, ILCM, NRAM, WTRPCA, Tophat, and OURS, in order.

Now we analyze the performance of each algorithm. In all sequences, the ILCM method only plays a role in enhancing the contrast; though ILCM can detect targets in some scenes, it cannot suppress the background. In sequences 3 and 4, the NRAM method, which is known for suppressing edges, loses the target and cannot realize the detection; in sequence 2, sequence 3, and sequence 6, the IPI method is not good for background suppression; WTRPCA reduces the detection time, but the performance on background suppression is poor, and strong clutter still exists. Only our method performs well from sequence 1 to sequence 7. Targets are detected clearly, while the background is strongly suppressed.

In addition, we analyze the performance of each algorithm in sequence 8 to sequence 10. It can be seen that the intensity of the noise often overshadows the target, so the Gaussian filtering operation cannot be performed directly. The IPI and NRAM methods cannot finish detection at all; because of the sensitivity of the Frobenius-norm, all pixels are considered as background. The ILCM and Tophat methods even strengthen the image noise. Our method successfully strips the low-rank background under noise interference, and still retains the target’s information against strong noise. Therefore, compared with the NRAM, IPI, and WTRPCA methods, NOP is the only method that can achieve background suppression and target detection under noise.

4.4.2. 3D-Graymap

3D-Graymap is a recognized qualitative metric for evaluating the quality of target detection due to its intuitive and concise expression [17]. In order to compare the detection results of various algorithms intuitively, we selected sequence 1, sequence 3, sequence 4, and sequence 8 for comparison, and Figure 9 shows the relevant result.

In these scenes, the grayscale of the target is close to the background and the background fluctuation is heavy. From Figure 9, we can find out that traditional methods such as Tophat and ILCM can hardly finish the detection. According to the first three rows, the background suppression of the IPI method is not sufficient. Though NRAM can suppress background cleaner, it has a situation where the target is missing according to the first, second, and fourth rows. In addition, the edge false-alarm of NRAM is higher than the target in (d3). Only our method stably suppresses the background and detects the target, showing the strong robustness of NOP. Combined with the analysis in the previous section, NOP has obvious advantages from a qualitative perspective. (Note that Tophat brings filter end effects, so there exist strong bounds in Tophat results. Like Reference [20], we choose to show the original results of each algorithm.)

4.5. Quantitative Assessment

4.5.1. SCRG and BSF

In order to quantitatively evaluate the ability of background suppression and target enhancement, we assess detection performance by SCRG and BSF. (Note that ILCM cannot suppress the background in most scenes, and we do not take it into quantitative comparison.)

The SCRG of the mentioned 7 sequences is shown in

Table 3. The signal clutter ratio gain (SCRG) values of representative sequences.

Method	SEQ1	SEQ2	SEQ3	SEQ4	SEQ5	SEQ6	SEQ7
Tophat	6.0405	1.2973	5.4171	5.1577	3.6813	1.1383	0.9465
WTRPCA	4.5636	2.1331	6.3299	3.2913	4.5254	1.3255	1.9753
NRAM	0.0178	0.0202	0.7820	0.0830	0.0509	1.0876	0.0438
IPI	5.0733	3.7379	7.0554	9.9403	7.2513	4.3866	24.9431
OURS	5.3811	4.2186	13.9293	10.4590	8.5726	3.7526	16.2843

, and the BSF is shown in

Table 4. Background suppression factor (BSF) values of representative sequences.

Method	SEQ1	SEQ2	SEQ3	SEQ4	SEQ5	SEQ6	SEQ7
Tophat	0.6342	0.0909	0.4932	0.3617	0.1462	0.5988	0.4280
WTPRCA	2.3900	1.0711	1.8153	1.4789	1.3726	1.3333	1.4085
NRAM	1.9535	4.8896	19.8741	13.7944	12.2792	9.4922	7.9050
IPI	4.1214	1.3993	8.0542	9.3567	2.4836	5.4149	9.0685
OURS	2.6427	1.8821	25.4126	22.7596	5.1435	3.7921	9.6248

:

The Tophat method enhances the target, but the background is suppressed insufficiently, and the SCRG and BSF of Tophat are relatively low. Compared with Tophat, WTRPCA performs better. IPI achieves high SCRG and BSF in many sequences while it behaves poorly in sequence 3 and sequence 4. NRAM has outstanding BSF, but its SCRG is small due to the target loss problem. The SCRG and BSF of our method are maintained at a high level because the background is fully suppressed and the target is highlighted. Therefore, the background suppression ability of our method is better than Tophat, IPI, and WTRPCA. Furthermore, unlike NRAM, our method avoids the target loss problem, which shows great advantages.

It can be seen that our method achieves high BSF and SCRG, which demonstrates the high efficiency of our method for infrared small target detection.

4.5.2. ROC Curves

AUC represents the area of the shape enclosed by the ROC and the coordinate axis. AUC is used to quantitatively describe the quality of the ROC. The magnitude of the AUC value is positively correlated with the goodness of ROC.

Table 5. The AUC values

AUC	OURS	IPI	WTRPCA	Tophat	NRAM
Seq1	100	100	100	85.1055	100
Seq2	100	100	100	64.4335	100
Seq3	100	99.9481	76.5974	61.4673	97.5132
Seq4	99.9950	99.8055	96.2249	62.4000	93.0045
Seq5	100	100	100	82.6325	100
Seq6	79.4875	60.7	60.3839	66.7627	59.9647
Seq7	99.9624	99.9591	93.9586	66.7035	98.2160

describes the AUC value of each algorithm when processing different sequences. In order to compare the size of AUC more intuitively, we have enlarged the horizontal axis, taking the percentage of FPR as the horizontal axis. It can be seen that the AUC value of our method performs very well. (Note that ILCM cannot suppress the background in most scenes; it is not necessary to calculate its AUC and ROC curves.)

We can find that in sequence 3, sequence 4, sequence 6, and sequence 7, the AUC value of each algorithm is significantly different, so the ROC curves of these four sequences are drawn for display, as shown in Figure 10. The horizontal axis represents the percentage of FPR, and the vertical axis represents the TPR.

Tophat does not perform well in most sequences due to its simple structure. IPI, WTRPCA, and NRAM behave similarly; they all have relatively low FPR. The ROC curves of our method are superior in different sequences. In sequence 6, only our method maintains high detection probability with respect to the same FPR. It can be seen from Table 5 and Figure 10 that the detection result of the proposed method is exceptional, which illustrates that our method is a stable and outstanding algorithm.

4.5.3. Time Consuming

In order to verify the improvement in detection speed, the detection time of NOP is compared with other methods. It can be seen that NOP effectively increases the speed of the algorithm relative to IPI and WTRPCA; the speed is in the same order of magnitude as the NRAM method. However, compared to traditional morphological methods, RPCA derived methods struggle to reach their detection speed.

The running time per frame for each algorithm in different scenes is shown in

Table 6. Calculation times of representative scenes. (Unit: second).

Method	Seq1	Seq2	Seq3	Seq4	Seq5	Seq6	Seq7
ILCM	0.01	0.02	0.02	0.02	0.01	0.02	0.02
Tophat	0.01	0.02	0.02	0.02	0.01	0.02	0.02
WTRPCA	2.86	3.36	3.47	3.31	3.08	3.76	4.72
NRAM	0.40	1.23	0.55	1.26	0.45	1.67	2.06
IPI	7.78	8.46	8.83	8.21	7.98	9.35	10.29
OURS	0.87	0.89	0.92	0.88	0.85	0.97	1.43

. The experimental conditions in this paper are set as follows: Window 10 system, Intel Core i7-8550U CPU, 8GB of RAM, and Matlab R2018a software.

It can be seen that the traditional methods are still very fast in speed. Compared with the classic IPI method, the speed of our method has been greatly improved, and a much better detection result has been achieved.

5. Conclusions

In order to solve the target loss problem of infrared small targets in complex scenes and improve the background suppression performance and detection speed, a novel model based on the non-overlapping patch model via the l₀-l₁ norm is proposed. Firstly, this paper designs the NOP model to successfully reduce the complexity of the calculation, which decreases the time consumed. Further, by introducing SRPCP into the field of infrared small target detection, we successfully solve the NOP model and receive exceptional target detection performance. On the whole, the proposed method reduces the target loss phenomenon while effectively suppressing the background and accelerating the detection speed. Extensive experiments based on diverse scenarios demonstrate the outstanding performance of our method.

Although NOP is a lightweight model and effectively increases the speed of detection, it is still not as fast as traditional methods. In addition, the NOP, like other state-of-the-art methods, cannot adequately suppress clutter that is much stronger than the target. In the follow-up work, this problem may be solved by introducing other feature representations.