SAR Image Reconstruction of Vehicle Targets Based on Tensor Decomposition

Abstract

Due to the imaging mechanism of Synthetic Aperture Radars (SARs), the target shape on an SAR image is sensitive to the radar incidence angle and target azimuth, but there is strong correlation and redundancy between adjacent azimuth images of SAR targets. This paper studies multi-angle SAR image reconstruction based on non-negative Tucker decomposition using adjacent azimuth images reconstructed to form a sparse tensor. Sparse tensors are used to perform non-negative Tucker decomposition, resulting in non-negative core tensors and factor matrices. The reconstruction tensor is obtained by calculating the n-mode product of the core tensor and the factor matrix, and then image reconstruction is realized. The similarity between the original image and the reconstructed image is calculated by using the structural similarity index and the cosine of the angle between the feature vectors. The reconstruction results of three target images of MSTAR show that the reconstructed image has a similarity higher than 95% with the original image in most cases, which can support target recognition under sparse observation to a certain extent.

1. Introduction

Synthetic Aperture Radars (SARs) have the ability to work all day and in all weather conditions, and they have better development potential than optical sensors. Therefore, SARs are widely used in surveying and mapping, geology, agriculture, the military and other fields. The intelligent detection and recognition of SAR image targets [1] have become research hotspots. As deep learning is widely used in tasks in the field of computer vision, recognition tools represented by convolutional neural networks (CNNs) are widely used in the field of SAR image target recognition, and their effect is better than that of traditional methods [2,3,4,5,6]. The distribution of SAR data can be analyzed with the help of new mathematical statistical tools [7,8]. Neural networks use multiple parameters to fit the data distribution, but they require a large number of SAR images of targets for training. There are few public data sets supporting deep learning for SAR image target recognition. The neural network method based on the SAR public data set has the risk of overfitting, so data amplification has become the focus of SAR research. Gao extracted the scattering center feature of a target image, applied the scattering center feature to an SAR target echo signal simulation and reconstructed an SAR image [9]. With the development of a generative countermeasure network, many scholars have introduced it into the field of SAR images. Gao used Conditional Generative Adversarial Networks (CGANs) to generate SAR images, but the inherent speckle noise of SAR images leads to model collapse and image blur during training [10]. With the aim of solving the problems of an unstable model and the poor image quality of CGANs used in SAR target directional and controllable generation, Wang proposed a label coding method, which can improve the quality of generated SAR images to a certain extent [11]. GANs have good performance in the field of natural image generation. However, the training model easily collapses when using a few SAR image samples full of serious speckle noise, so GANs are not suitable for SAR image generation.

Tensors can be used to describe data, such as images and videos. Tensor decomposition divides high-order tensors into products of multiple low-order tensors and can ensure that the spatial structure of high-dimensional data is not destroyed. Therefore, tensor decomposition is widely used in the fields of image processing and computer vision. Ji Liu et al. used SiLRTC, LRTC and FaLRTC algorithms to fill color images and medical images with missing data, and the relative errors with the original images were small [11]. The method of non-negative tensor decomposition emphasizes the non-negativity of the decomposition factor, which is highly interpretable and can better capture the local characteristics of an image [12]. Please replace with the following:

Zhang used the non-negative sparse Tucker decomposition algorithm to fill in the missing image data, which took use of several frame images before and after the missing frame image in the video to reconstruct the missing data. And the reconstruction effect is good [13]. The above tensor analysis method first decomposes and then reconstructs to achieve image restoration and missing data reconstruction according to the correlation between the data in the image and the multi-frame video. With a small amount of data, a high-quality reconstructed image can be obtained.

Multi-angle SAR has the advantage of spatial diversity, and there is redundancy and correlation between images at various azimuth angles [14]. Tensor decomposition can better mine image information from various angles, which is beneficial to SAR image reconstruction. In this paper, non-negative Tucker decomposition is introduced into SAR image processing. After that, the existing target azimuth image and the target azimuth image to be reconstructed are formed into a sparse tensor. Moreover, non-negative Tucker decomposition is performed on the sparse tensor, and it is reconstructed to obtain a dense tensor. Then, the image corresponding to the target azimuth is reconstructed. Finally, the similarity of the reconstructed image and the original image is evaluated using two evaluation parameters, namely, the structural similarity index and the cosine of the included angle of the feature vector, to test the effectiveness of the method in this paper.

The remainder of this article is organized as follows: Section 2 introduces tensor decomposition and image prediction. Section 4 gives the experimental results and analysis based on non-negative tucker decomposition. Finally, Section 5 concludes this article.

2. Tensor Decomposition and Image Prediction

2.1. Basic Concepts of Tensor Decomposition

A tensor is a multidimensional array, a scalar is a rank 0 tensor, a vector is a rank 1 tensor, and a matrix is a rank 2 tensor. For example, a grayscale image is a second-order tensor, a color image is a third-order tensor, and a color video is a fourth-order tensor. The multiplication of tensors and matrices is converted to the multiplication of matrices and matrices. Therefore, matrixize the tensor by expanding it modulo $n$. The $n$ order tensor has $n$ modes, and the expansion in each mode generates a corresponding matrix. The modular $n$ expansion of a tensor $\mathcal{S}\in {ℝ}^{J_1\times J_2\times \cdots \times J_n\times \cdots \times J_N}$ is expressed as ${\mathcal{S}}_{(n)}$. The product of the modular $n$ expansion of a tensor $\mathcal{S}$ and the n-mode product of a matrix $A^{(n)}\in {ℝ}^{I_n\times J_n}$ are written as $\mathcal{S}{\times }_n{A}^{(n)}$ [15]:

$${\left[\mathcal{S}{\times }_n{A}^{(n)}\right]}_{j_1\cdots j_{n-1}{i}_n{j}_{n+1}\cdots j_N}={\displaystyle \sum _{j_n=1}^{J_n}{s}_{j_1\dots j_{n-1}{j}_n{j}_{n+1}\cdots j_N}}{a}_{i_n{j}_n}$$

(1)

where $\mathcal{S}{\times }_n{A}^{(n)}\in {ℝ}^{J_1\times J_2\times \cdots \times I_n\times \cdots \times J_N}$.

Tucker decomposition decomposes an $N$-order tensor $\mathcal{X}\in {ℝ}^{I_1\times I_2\times \cdots \times I_n\times \cdots \times I_N}$ into an n-mode product of a core tensor $\mathcal{S}\in {ℝ}^{J_1\times J_2\times \cdots \times J_n\times \cdots \times J_N}$ and $n$ factor matrices $A^{(n)}\in {ℝ}^{I_n\times J_n}$:

$$\mathcal{X}\approx \mathcal{S}{\times }_1{A}^{(1)}{\times }_2{A}^{(2)}\dots {\times }_N{A}^{(N)}$$

(2)

$$x_{i_1{i}_2\cdots i_N}\approx {\displaystyle \sum _{j_1,j_2,\dots ,j_N}{s}_{j_1{j}_2\cdots j_N}}{a}_{i_1{j}_1}^{(1)}{a}_{i_2{j}_2}^{(2)}\cdots a_{i_N{j}_N}^{(N)}$$

(3)

Non-negative Tucker decomposition is used to add non-negative constraints to the core tensor $\mathcal{S}$ and factor matrix $A^{(n)}$, and the tensor reconstructed by the n-mode product using the core tensor and the factor matrix is denoted as $\widehat{\mathcal{X}}$. However, there are many tensor decomposition parameters, which are difficult to solve. Generally, the alternating method is used to solve the core tensor and factor matrix. When solving the core tensor and factor matrix alternately, the objective function is as follows [16]:

$${\mathcal{J}}_{LS}={\Vert \mathcal{X}-\widehat{\mathcal{X}}\Vert }_F^2$$

(4)

$${\mathcal{J}}_I={\displaystyle \sum _{i_1,i_2,\dots ,i_N}\left\{x_{i_1{i}_2\cdots i_N}\log \frac{x_{i_1{i}_2\cdots i_N}}{{\widehat{x}}_{i_1{i}_2\cdots i_N}}\right.} \left.-x_{i_1{i}_2\cdots i_N}+{\widehat{x}}_{i_1{i}_2\cdots i_N}\right\}$$

(5)

2.2. Tensor Decomposition Missing Value Imputation

A tensor is an extension of a matrix. Tensor decomposition missing value imputation is the same as matrix imputation. The matrix filling process is represented by the example shown in Figure 1. In Figure 1, several elements in the matrix in the upper left corner are missing, and their locations are represented by the string ‘nan’ represented by nan. The values in the corresponding positions can be re-estimated by tensor decomposition and data reconstruction, and filled into the original data element positions, which is represented in the lower left matrix. It can be seen that the missing matrix elements are filled.

Tucker tensor decomposition, also known as higher-order PCA, can be used for feature subspace learning and representation. Multi-angle SAR images are the imaging results of the same target at different azimuth or elevation angles, and they have the same feature subspace [17] and a strong correlation. The third-order tensor composed of multi-angle SAR grayscale images can be represented by the mode-n product of a core tensor and three factor matrices. The factor matrix represents the principal component of the tensor in the corresponding mode, and the core tensor represents and the core tensor represents different component correlation. After the non-negative tensor is decomposed and reconstructed, the predicted value of the corresponding position of the missing value of the original sparse tensor is obtained.

3. SAR Image Reconstruction Based on Non-Negative Tucker Decomposition

3.1. SAR Image Reconstruction

Non-negative Tucker decomposition is used to reconstruct the azimuth image, and the structural similarity index (SSIM) and the cosine of the included eigenvector angle are used to measure the similarity between the original image and the reconstructed image.

The method flow of image reconstruction is as follows:

(I)

The azimuth of the image to be reconstructed is $\beta$, the SAR image azimuth of the constructed tensor is $\gamma$, and the value is shown in formula (6):

$$\gamma =\beta \pm \alpha \times interval$$

(6)

$$\alpha =1,2,3\dots (frames-1)/2,interval>0,frames=2n+1,n\in N$$

where $interval$ represents the azimuth interval, and $frames$ indicates the number of frontal slices constituting a sparse tensor (that is, the total number of azimuth images involved in the reconstruction and to be reconstructed). The selected azimuth images are rotated to azimuth $\beta$ to be reconstructed and cropped to the same size in order to form a third-order tensor $\mathcal{X}$, or the image to be reconstructed, replace it with an all-zero matrix of the same size, as shown in Figure 2.

(II)

Set the rank of the core tensor $\mathcal{S}$ For setting the rank of the core tensor, define, where represents the number of frontal slices of the tensor $frames$ represents the number of frontal slices of the tensor, $width$ represents the width of the frontal slice, and $height$ represents the height of the frontal slice. If $\mathcal{X}\left(i,:,:\right)=0,i=1\dots k,k<frames$ is the reconstructed image detail information, set $\mathcal{S}\in {ℝ}^{(frames-k)\times width\times height},$ $Rank(\mathcal{S})=[frames-k,width,height]$.

(III)

Perform non-negative Tucker decomposition on tensors to obtain core tensors and factor matrices.

(IV)

Use Formula (2) to reconstruct the third-order tensor $\widehat{\mathcal{X}}\approx \mathcal{X}$.

Steps (II)–(IV) are shown in Figure 3.

3.2. Evaluation of Reconstructed SAR Image

In order to evaluate the effect of the reconstructed images, in addition to the intuitive perception of the human eye, the SSIM index and the cosine similarity measure of the feature angle based on a convolutional neural network are also introduced. The similarity between the reconstructed image and the original image is evaluated, and the performance of the method proposed in this paper is judged according to the evaluation effect.

The SSIM index comprehensively considers three factors, namely, brightness, contrast and structure, to evaluate image similarity [18]. Its expression is as follows:

$$\mathrm{S}(x,y)=\frac{\left(2{\mu }_x{\mu }_y+c_1\right)\left(2{\sigma }_{xy}+c_2\right)}{\left({\mu }_x^2+{\mu }_y^2+c_1\right)\left({\sigma }_x^2+{\sigma }_y^2+c_2\right)}$$

(7)

where $\mathrm{S}(x,y)$ is the SSIM index, ${\mu }_x$ is the mean of $x$, ${\mu }_y$ is the mean of $y$, ${\sigma }_x^2$ is the variance of $x$, ${\sigma }_y^2$ is the variance of $y$, and ${\sigma }_{xy}$ is the covariance of $y$. $c_1={\left(k_{1}L\right)}^2$, $c_2={\left(k_{2}L\right)}^2$ is a constant that maintains stability. According to [18], $L$ indicates the pixel dynamic range, $k_1=0.01, k_2=0.03$. The value range of SSIM is [0, 1], and the larger the value, the higher the similarity between the two images.

The cosine of the eigenvector angle measures the feature similarity by calculating the cosine value of the eigenvector angle, as shown in formula (8):

$$\begin{array}{l} \mathrm{similarity} =\cos (\theta )=\frac{features1\cdot features2}{\Vert features1\Vert \Vert features2\Vert }\\ =\frac{{\displaystyle \sum _{i=1}^{n}features{1}_i}\times features{2}_i}{\sqrt{su{m}_{i=1}^n{\left(features{1}_i\right)}^2\times \sqrt{{\displaystyle \sum _{i=1}^n{\left(features{2}_i\right)}^2}}}}\end{array}$$

(8)

The cosine of the angle between the eigenvectors considers the direction between different features, and the value range is [−1, 1]. When the similarity of the two eigenvectors is higher, the cosine value of the included angle of the eigenvectors is closer to 1. When the cosine of the included angle of the feature vector is 0, it means that the two are perpendicular to each other and are not similar. The key to determining whether the cosine of the included angle of the feature vector is accurate and reliable to measure the image similarity lies in the accurate extraction of image features. Convolutional neural networks have powerful feature representation capabilities and can be used as feature extractors. First, the MSTAR data set is used to train a three-class classifier. In order to make the extracted features robust and accurate, the recognition accuracy of the classifier is required to reach 95%. The process of extracting image features is shown in Figure 4. The convolutional neural network is mainly composed of two parts, one is feature extraction (convolution, activation function and pooling), and the other is classification and recognition (full connection layer). Figure 4 presents a structure diagram of a classical convolutional neural network used for SAR vehicle target recognition. In the feature extraction part, the network mainly includes four convolution layers, whose sizes are 3 × 3, 5 × 5, 6 × 6 and 5 × 5, as well as the nonlinear activation function and a pooling layer, which are produced after each convolution layer. The feature extraction section processes the SAR vehicle target image with a size of 100 × 100 × 3 into a feature map with a size of 4 × 4 × 128, which contains target features, and then inputs it into the classification recognition section. The classification and recognition part is composed of two fully connected layers, which convert the flattened feature map into a feature vector with a size of 216, reduce the dimension of the feature vector with a size of 216 into a feature vector with a size of 3 to perform three classification tasks later.

A flowchart of the method proposed in this section is shown in Figure 5.

4. Experiment and Analysis

The experimental data use MSTAR data. The Moving and Stationary Target Acquisition and Recognition (MSTAR) project is sponsored by the U.S. Department of Defense and the Air Force Laboratory and collected by the X-band SAR sensor of Sandia National Laboratory, which is a high-resolution sensor, a spotlight Synthetic Aperture Radar with a resolution of 0.3 m × 0.3 m [19]. The experiments in this paper mainly use three main types of vehicle targets in the MSTAR database: T72, BTR70 and BMP2. Each type of target contains multiple elevation and azimuth SAR images, and some optical images and SAR images are shown in Figure 6.

Firstly, the appropriate azimuth image and the azimuth image to be reconstructed are selected to form the sparse tensor, and the core tensor and factor matrix are decomposed using non-negative Tucker decomposition. The predicted value of the missing part is reconstructed to form the corresponding azimuth image. Finally, the similarity between the reconstructed image and the original image is calculated using the structural similarity index and the cosine of the included angle of the feature vector. To calculate the cosine of the included angle of the feature vector, the convolutional neural network is used to extract the image features. Therefore, a network with a recognition rate higher than 95% is trained first. The main parameters in the non-negative Tucker decomposition are $frames$ and $interval$. Therefore, this section takes different values for the above two parameters, which are discussed further in (II) Vehicle target SAR image reconstruction results.

The experimental process and the corresponding results are as follows:

(I)

Convolutional neural network training results

Using the three-category network contained in Figure 4 to train the MSTAR data of three types of vehicle targets, the final recognition accuracy of the network is 95.91%, as shown in Figure 7. The recognition accuracy rate is greater than 95%, as shown in Figure 8, which meets the requirements as a feature extractor.

(II)

Vehicle target SAR image reconstruction results

Some azimuth SAR images of the three types of vehicle targets are reconstructed. If the azimuth $\gamma$ corresponding image participating in the tensor construction does not exist during reconstruction, the most adjacent azimuth image is selected to replace it. The pitch angles of the images involved in the reconstruction are all 15°. Different values for the parameters $frames$ and $interval$ are set. The structural similarity index and the cosine of the feature vector angle between the reconstructed image and the original image are calculated (for convenience, feature similarity (Feature Similarity, FSIM) is used here). The influence of different parameter values on the reconstruction effect is discussed.

Figure 9, Figure 10 and Figure 11 show the reconstruction results of the partial azimuth angles of three types of military targets. The first row of each image represents the original image. The second line indicates the reconstruction results of $frames=5,interval=1$, followed by the reconstruction results of $frames=5,interval=10$, $frames=3,interval=1$ and $frames=3,interval=10$.

From the perspective of image performance, the visual effect of the reconstructed image is very similar to that of the original image. It can be seen in

Table 1. Similarity calculation of BMP2 reconstruction results.

	Reconstructed Azimuth	113 SSIM FSIM		140 SSIM FSIM		144 SSIM FSIM		150 SSIM FSIM		155 SSIM FSIM
Parameter
frames = 5, interval = 1		0.651	0.977	0.636	0.971	0.668	0.987	0.564	0.983	0.624	0.971
frames = 5, interval = 10		0.573	0.968	0.438	0.952	0.544	0.945	0.499	0.91	0.574	0.984
frames = 3, interval = 1		0.601	0.96	0.577	0.965	0.674	0.972	0.553	0.835	0.612	0.945
frames = 3, interval = 10		0.396	0.613	0.519	0.754	0.49	0.96	0.47	0.814	0.553	0.791

,

Table 2. Similarity calculation of BTR70 reconstruction results.

	Reconstructed Azimuth	69 SSIM FSIM		96 SSIM FSIM		100 SSIM FSIM		106 SSIM FSIM		111 SSIM FSIM
Parameter
frames = 5, interval = 1		0.558	0.925	0.685	0.998	0.627	0.992	0.704	0.989	0.554	0.959
frames = 5, interval = 10		0.558	0.741	0.515	0.989	0.561	0.934	0.516	0.931	0.562	0.972
frames = 3, interval = 1		0.558	0.947	0.612	0.968	0.677	0.984	0.607	0.98	0.591	0.987
frames = 3, interval = 10		0.558	0.974	0.536	0.947	0.546	0.928	0.533	0.933	0.595	0.981

and

Table 3. Similarity calculation of T72 reconstruction results.

	Reconstructed Azimuth	137 SSIM FSIM		164 SSIM FSIM		168 SSIM FSIM		174 SSIM FSIM		179 SSIM FSIM
Parameter
frames = 5, interval = 1		0.658	0.967	0.575	0.996	0.584	0.987	0.618	0.941	0.663	0.989
frames = 5, interval = 10		0.602	0.941	0.558	0.983	0.544	0.99	0.58	0.998	0.574	0.984
frames = 3, interval = 1		0.644	0.994	0.59	0.996	0.675	0.998	0.547	0.963	0.635	0.994
frames = 3, interval = 10		0.548	0.985	0.53	0.982	0.567	0.985	0.562	0.997	0.562	0.987

that the same azimuth angle image is reconstructed. When the azimuth angle interval ($interval$) is smaller, the higher the number of images participating in the reconstruction (the larger the $frames$), and the higher the similarity between the reconstructed image and the original image. The smaller the azimuth interval, the higher the similarity between images; therefore, the more accurate the principal components obtained by the factor matrix during tensor decomposition, the higher the similarity of the reconstructed images. When the azimuth angle interval increases, the similarity between the images is greatly reduced. If the number of reconstructed images increases, the reconstruction accuracy decreases. In the same $frames$ situation, the smaller the $interval$ for the reconstructed image, the more similar it is because the reconstruction method is proposed based on the similarity and redundancy between multi-angle SAR images. The smaller the azimuth interval, the higher the similarity and the more accurate the reconstruction. It can be seen in the table that the main factor affecting the reconstruction of the multi-angle SAR images is the azimuth interval. When the azimuth angle interval is too large, for example, greater than 20°, the quality of the reconstructed image is greatly reduced, and it is no longer applicable.

The reconstructed image has a lower SSIM than the original image, and the missing part is replaced by 0 when building a sparse tensor. After reconstruction, the value of the corresponding position is near 0, and the reconstructed image is obtained after the contrast is stretched. Therefore, there is a certain degree of distortion in image brightness, resulting in an unsatisfactory SSIM index effect. The cosine value of the angle between the reconstructed image and the original image is close to 1. This shows that the reconstructed image is very similar to the original image at the image feature level. The tensor decomposition result can well represent the feature subspace, so the image features reconstructed based on non-negative Tucker decomposition are highly similar to the original images.

5. Conclusions

Starting from the correlation and redundancy of multi-angle SAR images, this paper proposes a vehicle target SAR image reconstruction method based on non-negative Tucker decomposition. The missing azimuth image is reconstructed using the adjacent azimuth image composition tensor. The similarity calculation results confirm that the reconstruction method in this paper is effective. The cosine of the eigenvector angle is introduced in the similarity calculation. Combined with the visual effect, the cosine of the angle between the feature vectors can better evaluate the image similarity. However, there is brightness distortion in the reconstructed image. In future work, we can consider adding a regular term and bias to the non-negative Tucker decomposition in order to solve this problem. It is also possible to explore the effect of the azimuth interval change on the similarity and to add weights in the tensor decomposition in order to obtain a more accurate reconstructed image.

The multi-angle SAR image reconstruction method based on non-negative Tucker decomposition can reconstruct the azimuth image with only a few adjacent azimuth images, and it does not require a large number of samples for training. When the sample data are small, the method in this paper can be used to reconstruct the image as one of the means of data expansion. When using traditional target recognition methods, such as template matching, the stored templates can be reduced. In target recognition, existing templates can be used to generate corresponding azimuth images and then matched. When the azimuth images are separated by 10°, the reconstructed image still has a high feature similarity with the original image. This shows that the method proposed in this paper can support target recognition under the condition of sparse observation to a certain extent.

Future research work mainly considers the following two parts: First, the reconstruction and verification of the SAR image data of other vehicle targets will be added. However, the reconstructed data will be used for vehicle target detection and recognition in SAR images to measure its practicability and robustness.