Identification of Crop Type Based on C-AENN Using Time Series Sentinel-1A SAR Data

Abstract

Crop type identification is the initial stage and an important part of the agricultural monitoring system. It is well known that synthetic aperture radar (SAR) Sentinel-1A imagery provides a reliable data source for crop type identification. However, a single-temporal SAR image does not contain enough features, and the unique physical characteristics of radar images are relatively lacking, which limits its potential in crop mapping. In addition, current methods may not be applicable for time-series SAR data. To address the above issues, a new crop type identification method was proposed. Specifically, a farmland mask was firstly generated by the object Markov random field (OMRF) model to remove the interference of non-farmland factors. Then, the features of the standard backscatter coefficient, Sigma-naught (σ₀), and the normalized backscatter coefficient by the incident angle, Gamma-naught (γ₀), were extracted for each type of crop, and the optimal feature combination was found from time-series SAR images by means of Jeffries-Matusita (J-M) distance analysis. Finally, to make efficient utilization of optimal multi-temporal feature combination, a new network, the convolutional-autoencoder neural network (C-AENN), was developed for the crop type identification task. In order to prove the effectiveness of the method, several classical machine learning methods such as support vector machine (SVM), random forest (RF), etc., and deep learning methods such as one dimensional convolutional neural network (1D-CNN) and stacked auto-encoder (SAE), etc., were used for comparison. In terms of quantitative assessment, the proposed method achieved the highest accuracy, with a macro-F1 score of 0.9825, an overall accuracy (OA) score of 0.9794, and a Kappa coefficient (Kappa) score of 0.9705. In terms of qualitative assessment, four typical regions were chosen for intuitive comparison with the sample maps, and the identification result covering the study area was compared with a contemporaneous optical image, which indicated the high accuracy of the proposed method. In short, this study enables the effective identification of crop types, which demonstrates the importance of multi-temporal radar images in feature combination and the necessity of deep learning networks to extract complex features.

1. Introduction

With the intensive development of agricultural production modes in China, smart agriculture has emerged [1]. There is an urgent need for large-scale and efficient monitoring of crops [2]. Due to remote sensing technology having the benefit of objectivity and economy, its application in the agricultural field is continuously expanding and deepening [3,4]. At present, agricultural remote sensing applications consist of crop type identification [5], yield estimation [6], soil moisture inversion [7], growth and phenological phase monitoring [8,9], etc. Among them, crop type identification is a pre-requisite work for national government departments to grasp the status of crop production, which is of great significance to agricultural management.

Crop type identification by remote sensing is usually carried out by visual interpretation [10] or computer techniques (supervised and unsupervised classification methods [11]). Currently, the employment of optical images for crop type identification has been achieved with good results [12,13]. However, optical remote sensing is susceptible to interference from weather [14,15]. Cloudy and rainy days often occur during the key growing stage of crops. In this case, it is difficult to obtain usable images, which affects the accuracy and timeliness of crop type identification. Fortunately, microwave remote sensing is playing an increasingly important role [16]. synthetic aperture radar (SAR) data can be obtained all -day and in all -weather. Not only can the surface information of the crops be captured, but the leaves, stems, and branches of the crops can be reflected in some way [17]. Moreover, Sentinel-1A images have good spatial and temporal resolution [18], which ensures the reliability of the data and shows potential value in crop type identification. Therefore, Sentinel-1A can be selected as the main data source for this study.

Up to now, scholars have attempted two major directions to improve the accuracy of crop type identification using SAR data. At first, time-series images are considered available to complement the necessary features for crop type identification [19,20]. Second, mainstream deep learning technology is adopted to maximize the utilization of features [21,22].

Due to the complexity and similarity of crop types, the spatial resolution of single-temporal SAR data is limited [23], and it is difficult to identify crop types effectively. On the other hand, multi-temporal SAR images can be used to improve the accuracy of crop type identification [24,25]. It has been shown that crop development over time can lead to changes in the backscatter coefficient of SAR images [26]. The multi-temporal SAR images are richer in features, which cover different stages of the crop. Xiao et al. [27] applied the subspace K-nearest neighbor (KNN) classifier to the Sentinel-1 images of the crop growing season, which indicated the importance of the time-series SAR data for crop type identification applications in rural China. Arias et al. [28] extracted polarimetric features (VH, VV, and VH/VV) from time-series Sentinel-1 images and used supervised classification techniques for crop classification. The results showed that the combination of time-series features provided accurate results, with an overall accuracy of over 70%. However, these studies usually only consider the standard backscatter coefficient, Sigma-naught (σ₀), in the SAR image [29,30], and ignore the influence of the normalized backscatter coefficient by the incident angle, Gamma-naught (γ₀), on crop type identification. It has been proved that incident angle is an important factor affecting SAR backscattering intensity, and the γ₀ can be used to capture the backscatter process of the sensor signal well [31].

The abundance of features makes the processing and analysis of SAR images complicated in the agricultural application field. In order to make full use of the multi-temporal information of SAR images, various methods have been developed. Classifiers such as support vector machine (SVM) [32], random forest (RF) [33] and KNN [34] have been proven to be very helpful. These methods have strong versatility, but the feature extraction ability is limited. In recent years, the deep learning method [35,36] is used as an ensemble framework, which can learn directly from data and conduct classification in an end-to-end manner [37]. Ndikumana et al. [38] proposed a recurrent neural network (RNN) for agricultural land cover classification, which demonstrated the effectiveness of long short-term memory networks (LSTMs) and gated recurrent unit (GRU) for processing sequence features. Zhang et al. [39] employed a new crop discrimination network with multi-scale features (MSCDN) to improve crop classification ability, with an overall accuracy rate of up to 99%. The application of deep learning methods have obtained favorable results. Nevertheless, most of these networks use only one model framework, which potentially restricts their feature mining capabilities. It is still a challenge to develop a network that can efficiently exploit features.

This study aims to develop a deep learning method to improve the accuracy of crop type identification using Sentinel-1A SAR images. First of all, the object Markov random field (OMRF) model was applied to produce a farmland mask, removing non-farmland areas. Then, the backscatter coefficients feature σ₀ and γ₀ were extracted for each type of crop, and Jeffries-Matusita (J-M) distance analysis was carried out to obtain the optimal combination of identification features. Finally, the convolutional-autoencoder neural network (C-AENN) was proposed for learning valid features automatically on a given crop type identification task. The main contributions of this paper are:

• Extracting of backscatter coefficient σ₀ and γ₀ features from the time-series Sentinel-1A images and combining them together, which fully exploits the potential of the time-series Sentinel-1A images in crop type identification.
• Aiming at time-series SAR data, an effective crop type identification classifier C-AENN is proposed. It has outstanding feature learning and representation capabilities, which improves identification accuracy.

The rest of this paper is organized as follows. Section 2 describes the study area and data. Section 3 introduces the method of crop type identification in detail. Section 4 shows the results of identification. Section 5 presents the conclusion.

2. Study Area and Data

2.1. Study Area

The study area is Kaifeng City (113°52′15″E–115°15′42″E, 34°11′45″N–35°01′20″N), Henan Province, China. As shown in Figure 1, it is situated in the center of the Henan Province map, adjacent to the provincial capital, which has an area of about 6266 km². The climate in Kaifeng is a continental monsoon, with an average annual temperature of 14.52 °C and an annual rainfall of 635 mm. The rainfall usually concentrates in July and August each year. In addition, Kaifeng has become an important base for agricultural production. It is located in the plain, and the soil is mostly clay, loess, and sandy soil, which is suitable for crop cultivation. Farmers usually adopt a two-crop-a-year rotation farming mode, one stage is from June to October, defined as stage I. The other stage is from October to June of the following year. This research focuses on stage I. The crops in this period are mainly corn, peanut, and a small amount of rice.

2.2. Data and Preprocessing

2.2.1. Sentinel-1A Data and Preprocessing

Sentinel-1A carries an imaging mission all-day and in all-weather, which is used for observing land and ocean. In particular, Sentinel-1A was launched in April 2014. It is an active microwave remote sensing satellite and the revisit period is 12 days [40]. The specific parameters of the data are shown in

Table 1. Parameters for Sentinel-1A.

Sentinel-1A Parameters	Sentinel-1A
Product type	GRD
Imaging mode	IW
Polarization	VV VH
Resolution	10 × 10 m
Band	C
Pass direction	Ascending
Dates	17 March 2021, 3 July 2021, 15 July 2021, 27 July 2021, 8 August 2021, 20 August 2021, 1 September 2021

. The SAR image on 17 March 2021 was selected for farmland extraction. At that time, the characteristics of farmland and non-farmland (buildings, water) were quite different. Therefore, this study is conducted to create a farmland mask firstly to remove the non-farmland areas. SAR images from 3 July 2021 to 1 September 2021 were used for crop type identification, which almost covered the crop growth cycle (stage I).

The Sentinel-1A data was preprocessed based on the Sentinel Application Platform (SNAP) software provided by the European Space Agency (ESA). The pre-processing mainly consists of: (1) Orbit file application. (2) Thermal noise removal. (3) Calibration. The pixel value in the image was returned in the following form: σ₀ band and γ₀ band. (4) Mosaicking. Multiple images covering the study area were mosaicked together. (5) Speckle filtering. Refined Lee filtering (filter window 7 pixels × 7 pixels) operation was used to reduce the impact of coherent speckle noise. (6) Range-Doppler terrain correction. With the help of the Shuttle Radar Topography Mission (SRTM), precise geographic information was given. (7) Conversion of the backscatter coefficient from linear to dB scale. (8) Study area extraction.

2.2.2. Ground Truth Data and Preprocessing

In order to obtain a comprehensive and accurate view of the crop planting structure in the study area, the local Department of Agriculture and Rural Affairs was visited to obtain a general distribution of crops. A field survey was carried out from July to September 2021 to collect and record sample data of crops in the study area. Due to the family cropping mode of the small-scale peasants in China, the distribution of parcels is relatively fragmented and there are usually multiple crops in one parcel, which increases the difficulty of sample collection. Based on the actual complexity of the situation, parcels with a larger area were selected for sample collection. A Global Positioning System (GPS) device was used to locate the four corners of the parcel to ensure that the ground data was reasonable.

The sample of major crops was converted into shapefile (SHP) files in Geographic Information System (ArcGIS) software, which provided ground data support for crop type identification. The ground truth data includes the types of main crops and their geographic distribution information, as shown in Figure 2. The crop types in the study area were classified into four categories: peanut, maize, rice, and others. The type of others is mostly vegetable mixed planting. Eventually, the 109 typical parcels (158,223 pixels) were selected as samples, including 36 peanuts, 45 corn, 14 rice, and 14 others. In this study, 50% (79,111 pixels) of the samples were randomly selected as training samples and the remaining 50% (79,112 pixels) as test samples. The parameters of the samples are illustrated in

Table 2. Ground truth samples for major crop types in the study area.

Label	Type	Number of Parcels	Total Number of Pixels	Area (km2)	Number of Training Samples	Number of Test Samples
1	Peanut	36	30,394	2.51	15,056	15,338
2	Corn	45	68,767	5.73	34,529	34,238
3	Rice	14	39,113	3.22	19,721	19,392
4	Others	14	19,949	1.67	9805	10,144
Total		109	158,223	13.13	79,111	79,112

.

2.2.3. Optical Reference Data and Preprocessing

Due to the difficulty of obtaining ground truth data covering the entire study area, Sentinel-2A is used as reference data in this study to evaluate the relative accuracy of the crop type identification results qualitatively. Sentinel-2A is tasked with multispectral high-resolution imaging. Its short-wave and near-infrared bands are critical for monitoring crops, and the spectral characteristics of its bands are shown in

Table 3. Spectral description for optical reference data.

Sentinel-2A Bands	Sentinel-2A Spectral Description
Band 1	Coastal aerosol
Band 2	Blue
Band 3	Green
Band 4	Red
Band 5	Red-edge 1
Band 6	Red-edge 2
Band 7	Red-edge 3
Band 8	NIR-1
Band 8A	NIR-2
Band 9	Water vapor
Band 10	SWIR-cirrus
Band 11	SWIR-1
Band 12	SWIR-2
Dates	25 August 2021

. Sentinel-2A images possess good resolvability, however, images can only be obtained under cloudless and rainless conditions. Affected by the weather, only one Sentinel-2A image (25 August 2021) was available during the study period, which covered the study area with relatively little cloudiness.

Sentinel-2A data pre-processing mainly consists: (1) Resampling. The images were resampled to a uniform resolution of 10 m in all bands. (2) Layer stacking. Band combinations were employed to better explain the features in an image. By using band combinations, specific information in the image can be extracted. (3) Mosaicking. (4) Study area extraction.

3. Methodology

3.1. Overview

The flow chart of the proposed method is presented in Figure 3. After the data acquisition and preprocessing, the crop type identification method was mainly divided into 3 steps. In step 1, the farmland areas were first extracted by the OMRF model in order to exclude the interference of non-farmland factors. In step 2, the backward scatter features σ₀ and γ₀ of each crop were extracted. By analyzing the J-M distances of various crop samples under different time phase conditions, the optimal feature combination was selected, which can improve identification accuracy and efficiency. In step 3, a novel classifier, C-AENN, was developed for the generation of the crop type identification model. It can efficiently mine useful features. At the same time, the classic machine learning methods SVM, RF, KNN, and neural network methods artificial neural network (ANN), one dimensional convolutional neural network (1D-CNN), stacked auto-encoder (SAE) were used for comparison. At last, the identification results of the model were evaluated quantitatively and qualitatively.

3.2. Crop Type Identification

3.2.1. Farmland Extraction

Since the backscatter features of crops at some growth stages may be similar to those of non-farm features, it may cause confusion. Therefore, this study first carried out the extraction of farmland areas. For example, in the early stage of crop sowing, some crops and buildings have similar backscatter features, which makes them difficult to be distinguished. Removing the non-farmland regions in advance is beneficial to avoid the interference of other ground objects on the identification result of crop types [41].

This research first selected a SAR image with a large difference between farmland and non-farmland features, and applied the OMRF [42] model to classify the image into buildings, water, and farmland. The OMRF model takes full account of the spatial contextual information of pixels in the image, which has a high resistance to noise. After that, buildings and water were removed and farmland was retained.

In detail, the OMRF model uses basic segmentation methods, such as simple linear iterative clustering (SLIC), mean shift (MS), etc., to segment a given SAR image into several over-segmented regions, firstly, which are denoted by $\mathrm{R}=\{R_i|i=1,2,\cdots ,k\}$. Each $R_i$ is an over-segmented region, and k is the total number of over-segmented regions. These regions are then used to build a regional adjacency graph (RAG).

RAG consists of two random fields, the label field $\mathrm{x}=\{x_i|i=1,2,\cdots ,k\}$ represents the category of each $R_i$, and the feature field $\mathrm{y}=\{y_i|i=1,2,\cdots ,k\}$ represents the average value of each $R_i$. According to the maximum a posteriori (MAP) criterion and Bayesian formulation, the classification problem of the SAR image can be solved by finding the best result $\hat{\mathrm{x}}$ using the posterior probability $P(\mathrm{x}|\mathrm{y})$, which is shown in Equation (1).

$$\hat{\mathrm{x}}=\underset{\mathrm{x}}{\arg \max }P(\mathrm{x}|\mathrm{y})=\underset{\mathrm{x}}{\arg \max }\frac{P(\mathrm{y}|\mathrm{x})P(\mathrm{x})}{P(\mathrm{y})}=\underset{\mathrm{x}}{\arg \max }P(\mathrm{y}|\mathrm{x})P(\mathrm{x})$$

(1)

Then the solution of $\hat{\mathrm{x}}$ can be converted into an energy optimal problem expressed by Equation (2). $E(\mathrm{y}|\mathrm{x})$ is the conditional energy and $E(\mathrm{x})$ is the prior energy. Finally, the best classification result $\hat{\mathrm{x}}$ is found by using the optimization algorithm iterated conditional mode (ICM).

$$\hat{\mathrm{x}}=\underset{\mathrm{x}}{\arg \min }\{-\ln P(\mathrm{y}|\mathrm{x})-\ln P(\mathrm{x})\}=\underset{\mathrm{x}}{\arg \min }\{E(\mathrm{y}|\mathrm{x})+E(\mathrm{x})\}$$

(2)

3.2.2. Feature Extraction and J-M Distance Analysis

The backscatter coefficients σ₀ and γ₀ of each crop sample were extracted from SAR images. In order to find the optimum feature combination for crop identification, J-M distances between various crop samples were calculated under different time-phased conditions.

Based on the conditional probability theory, the J-M distance is employed to calculate the separability of sample characteristics [43], which is illustrated in Equation (3). It is a useful tool to evaluate whether the sample characteristics are qualified. The overall variation range of the J-M distance value is 0–2. When the value exceeds 1.9, it indicates that the selected sample features have good separability. When the value is less than 1.8, it indicates that the selected sample features are moderately separable. However, if the value is less than 1, it is considered that the two types of samples are not separable [44]. Depending on the analysis, the optimal combination of features can be obtained:

$$D_{ij}={\left\{{{\displaystyle {\int }_x\left[\sqrt{P(x/w_i)}-\sqrt{P(x/w_j)}\right]}}^2\mathrm{d}x\right\}}^{0.5}$$

(3)

where $D_{ij}$ represents the separability of class $i$ and class $j$. $P(x/w_i)$ $P(x/w_j)$ are the conditional probability densities of the feature $x$ under class $i$ and class $j$, respectively.

3.2.3. C-AENN Classifier

In order to effectively utilize the optimal feature combination, the classifier C-AENN was built to obtain the crop type identification model. The structure of C-AENN is shown in Figure 4, which consists of three main components: 1D-CNN, SAE, and the Softmax classification layer. CNN [45,46] and SAE [47] are the more widely used methods in deep learning, which are proven to be effective for feature mining and representation [48]. The C-AENN classifier proposed in this paper attempts to combine these two deep learning models to explore their classification capabilities. Specifically, the 1D-CNN model is used to perform feature extraction on the original optimal identification features. Then, the features extracted by 1D-CNN are used as the input of SAE, and further feature mining work is performed to obtain new feature expressions.

In 1D-CNN, three convolutional filters were set up. Multilayer convolution can extract complex features and the relationships which are embedded in the preceding and following sequences. After the convolution layer, the maximum pool and batch normalization layers followed, respectively, which can reduce the output dimensionality and improve network generalization capabilities. At last, a flatten layer was applied to convert multi-dimensional data into one-dimensional data. In SAE, the features extracted by the 1D-CNN were fed into the encoder, and then they were compressed into a latent space representation. Afterwards, the decoder aimed to reconstruct the space representation, which mapped the feature vectors back to the SAE input space size. The number of nodes in each layer of SAE was set to 128, 64, 32, 16, 32, 64 and 128, respectively. Finally, the extracted features were fed into the Softmax layer for classification.

In brief, the C-AENN classifier fused two network frameworks, 1D-CNN and SAE. 1D-CNN was used to extract relevant temporal features from time-series SAR data. SAE projected the feature to a lower-dimensional latent space and then attempted to reproduce its original input. The process used supervised learning to discover internal correlations in the time-series feature dataset and extract as much useful information as possible. Compared with using 1D-CNN or SAE alone, C-AENN can retain and obtain a better representation of SAR data.

3.2.4. Other Classifier

Several existing classifiers were used to compare the proposed methods, for example, the classical machine learning methods SVM, RF, and KNN, and the neural network methods ANN, 1D-CNN, and SAE. The hyperparameters of the SVM, RF, KNN, and ANN are listed in

Table 4. Hyperparameters of the classifier.

Classifier	Parameters	Description	Value
SVM	C	Penalty coefficient	2
	Kernel	Kernel function	Rbf
RF	N_estimators	Number of decision trees	550
KNN	N_neighbors	Number of neighboring points	20
ANN	Hidden_layer_sizes	Number of neurons in the hidden layer	(20, 20, 20, 20)

. The structures of the 1D-CNN and SAE are shown in Figure 4. These two networks were used separately to test the effects.

3.3. Model Accuracy Evaluation

For the purpose of assessing the accuracy of different classifiers, the identification results were evaluated both quantitatively and qualitatively.

For the quantitative evaluation, precision, recall, F1-score, macro-F1, overall accuracy (OA), and Kappa coefficient (Kappa) are used. The calculations of them are illustrated in Equations (4)–(10), respectively. Higher values indicate better results. The definitions of true positive (TP), false positive (FP), false negative (FN), and true negative (TN) in the formulas are given in the confusion matrix, which is demonstrated in

Table 5. Confusion matrix.

		Predicted Label
	Type	A	B
Ground truth	A	True positive (TP)	False negative (FN)
	B	False positive (FP)	True negative (TN)

. A, B are the assumed categories.

• Precision;

$$\mathrm{precision}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}}$$

(4)

• Recall;

$$\mathrm{recall}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}}$$

(5)

• F1-score;

$$\mathrm{F}1\text{-}\mathrm{score}=\frac{2\times \mathrm{precision}\times \mathrm{recall}}{\mathrm{precision}+\mathrm{recall}}$$

(6)

• Macro-F1;

$$\mathrm{macro}\text{-}\mathrm{F}1=\mathrm{Average}\left(\mathrm{F}1\text{-}\mathrm{score}\right)$$

(7)

• OA;

$$\mathrm{OA}=\frac{\mathrm{TP}+\mathrm{TN}}{\mathrm{TP}+\mathrm{TN}+\mathrm{FP}+\mathrm{FN}}$$

(8)

• Kappa.

$$\mathrm{Kappa}=\frac{\mathrm{OA}-{\mathrm{p}}_e}{1-{\mathrm{p}}_e}$$

(9)

$${\mathrm{p}}_e=\frac{(\mathrm{TP}+\mathrm{FN})\times (\mathrm{FP}+\mathrm{TP})+(\mathrm{FP}+\mathrm{TN})\times (\mathrm{FN}+\mathrm{TN})}{{(\mathrm{FP}+\mathrm{TN}+\mathrm{TP}+\mathrm{FN})}^2}$$

(10)

For the qualitative assessment, two approaches were chosen. One approach is to make a visual comparison between the sample maps and the result maps of different classifiers, and the other is to compare the identification result covering the study area with an optical image of the same period. Both quantitative and qualitative evaluation results could support the validity of the proposed methodology.

4. Results

4.1. Temporal Profiles of the Sentinel-1A Backscatter Coefficient

The geometric and physical characteristics of the irradiated object can be reflected to some extent, by the backscatter coefficients in the image, assisting in the identification of the crop types. Figure 5 summarizes the temporal profiles for the four crop types, with each point being the average backscatter coefficient for each crop type sample, which provides temporal dynamic information about crops.

Several vital points of information can be seen in Figure 5, which is concluded in two dimensions.

1.

The necessity of combining VV polarization and VH polarization.

Overall, due to long-term coverage by water layers, rice always exhibited a low backscattering coefficient [49]. In contrast, peanut had a high backscatter coefficient. Moreover, the backscatter coefficient of maize and the “others” category intersected several times, which meant that it was more difficult to distinguish them. According to the fact that the average backscatter coefficient had no intersection with other crops, the peanut was relatively easier to distinguish from other crops under VV polarization, and rice showed a good separability under VH polarization. Furthermore, in July, the stems of corn elongated upward rapidly, so it showed greater variation in the backscatter coefficient under VH polarization. This polarization mode is more sensitive to crop structure changes [50]. While in August, rice was in the filling stage and required a lot of water. VV polarization is very sensitive to water [50]. Thus, rice varied more under the VV polarization mode. In summary, necessary information to differentiate crops was provided by both polarization modes. To better recognize crops, VV and VH polarizations were combined in the study.

2.

The criticality of time-series features.

It can be seen that the trends of the backscatter coefficients σ₀ and γ₀ over time were generally consistent and the average backscatter coefficient σ₀ was 1 dB lower than the γ₀ for all crops. At the beginning of July, peanuts, corn, and rice had just been sown and were in the early stages of growth. The radar response was mainly surface scattering from rough soils, tillage conditions, and soil moisture content. From July to August, crops were in the peak growing season, and the backscatter coefficients gradually increased. They both reached a maximum on 27 July 2021. In August, the backscatter coefficients of crops all decreased, which may be related to variations in crop growth and soil water content. As a whole, changes in peanuts and the “others” category were basically flat. The backscatter coefficients of corn and rice were relatively variable, which indicated the necessity of using time-series images to distinguish them.

4.2. Farmland Mask

The extraction result of the farmland mask is shown in Figure 6. The SAR image on 17 March 2021 was made into a mask by OMRF, as shown in Figure 6a,b. The mask was applied to the time-series SAR images during the study period, as shown in Figure 6c. Subsequent studies were conducted only in the farmland area.

4.3. J-M Distance Analysis of Features

The backscatter coefficients σ₀ and γ₀ of peanuts, corn, rice, and other crops were extracted, which showed different separability under different time-phase conditions. This paper performed J-M distance calculations for samples based on a single-temporal image and multi-temporal images.

Table 6. J-M distances for crops σ₀ and γ₀ features based on a single-temporal image.

J-M Distance	3 July 2021		15 July 2021		27 July 2021		8 August 2021		20 August 2021		1 September 2021
	σ0	γ0	σ0	γ0	σ0	γ0	σ0	γ0	σ0	γ0	σ0	γ0
Peanut–Corn	0.3179	0.3359	0.1189	0.1016	0.1586	0.1549	0.3814	0.3970	0.3652	0.3763	0.2305	0.2499
Peanut–Rice	0.3056	0.3218	0.3519	0.3529	0.1597	0.1600	0.6257	0.6459	0.8617	0.8822	0.7436	1.2088
Peanut–Others	0.1346	0.1430	0.1430	0.1411	0.4775	0.4662	0.2796	0.3043	0.2074	0.2237	0.1694	0.2160
Corn–Rice	0.1021	0.1028	0.2347	0.2356	0.0803	0.0839	0.1675	0.1763	0.3665	0.3834	0.4129	0.8187
Corn–Others	0.1233	0.1251	0.0064	0.0073	0.1337	0.1291	0.0209	0.0204	0.0349	0.0314	0.0175	0.0375
Rice–Others	0.2512	0.2575	0.2372	0.2375	0.3051	0.3079	0.1812	0.1798	0.4727	0.4814	0.4224	0.7027

gives the J-M distances for various crop samples under a single time-temporal image. The maximum J-M distance between each of the two crops was bolded. Based on the previous backscatter coefficient analysis, the VV polarization and VH polarization features were combined, so that σ₀ in Table 6 represents σ₀ VV and σ₀ VH, and γ₀ represents γ₀ VV and γ₀ VH. It can be seen that the J-M distances for σ₀ and γ₀ features were not significantly different and the values were all low, essentially below 1.0, which indicated a large similarity of samples that were not separable. Therefore, σ₀ and γ₀ features were considered in combination to explore crops divisibility. The results are shown in

Table 7. J-M distances for crops σ₀ and γ₀ combination features based on a single-temporal image.

J-M Distance	3 July 2021	15 July 2021	27 July 2021	8 August 2021	20 August 2021	1 September 2021
	σ0-γ0	σ0-γ0	σ0-γ0	σ0-γ0	σ0-γ0	σ0-γ0
Peanut–Corn	0.9114	0.1789	0.5301	0.9521	0.8515	0.3528
Peanut–Rice	0.8095	0.4583	1.0426	1.0700	1.2081	1.3609
Peanut–Others	0.5299	0.2309	0.4803	0.6383	1.6040	0.3380
Corn–Rice	1.1932	0.3219	0.5047	0.4398	0.7158	0.9456
Corn–Others	0.4503	0.0125	0.2444	0.5228	1.7408	0.0814
Rice–Others	0.9770	0.3243	0.5900	0.3238	1.7385	0.8216

. The symbol σ_0-γ₀ represents four features, which are σ₀ VV, σ₀ VH, γ₀ VV and γ₀ VH. It is clearly shown that the combination of σ₀ and γ₀ features was effective in improving the J-M distance values. The J-M distances between some crops already exceeded 1.0, but there was still much space for improvement.

Within a single-temporal SAR image, it is possible for different crops to have similar backscatter responses [51], causing the J-M distance values to perform poorly and, therefore, cannot yet meet the application requirements. However, within multi-temporal SAR images, it is possible to characterize crop growth patterns. Typically, the patterns of growth change for different crops are not identical, and the differences between their temporal trends are key to distinguishing them.

Table 8. J-M distances for crops σ₀ and γ₀ features based on multi-temporal images.

J-M Distance	Single-Temporal Maximum		3 July 2021 15 July 2021		3 July 2021 15 July 2021 27 July 2021		3 July 2021 15 July 2021 27 July 2021 8 August 2021		3 July 2021 15 July 2021 27 July 2021 8 August 2021 20 August 2021		3 July 2021 15 July 2021 27 July 2021 8 August 2021 20 August 2021 1 September 2021
	σ0	γ0	σ0	γ0	σ0	γ0	σ0	γ0	σ0	γ0	σ0	γ0
Peanut–Corn	0.3814	0.3970	0.5193	0.5128	0.6115	0.6187	0.9568	0.9524	0.9841	1.0173	1.0765	1.0781
Peanut–Rice	0.8617	1.2088	0.5292	0.5333	0.6262	0.6383	1.3136	1.3210	1.5586	1.5995	1.6089	1.6497
Peanut–Others	0.4775	0.4662	0.2462	0.2620	0.3585	0.3910	0.6553	0.6996	0.7501	0.7213	0.9794	0.7839
Corn–Rice	0.4129	0.8187	0.4068	0.3785	0.5428	0.5227	0.8225	0.8081	1.1298	1.1625	1.1917	1.2819
Corn–Others	0.1337	0.1291	0.3140	0.2730	0.2695	0.2567	0.3932	0.3719	0.4815	0.4228	0.6176	0.4800
Rice–Others	0.4727	0.7027	0.4556	0.4266	0.6148	0.5865	0.9024	0.8637	1.3391	1.3377	1.4378	1.4031

exhibits the J-M distances for crops σ₀ and γ₀ features based on multi-temporal SAR images. Apparently, the J-M distance tended to increase as the images increased. When the images of 3 July 2021, 15 July 2021, 27 July 2021, 8 August 2021, 20 August 2021 and 1 September 2021 were all combined, defined by T, the J-M distance values exceeded any other combination of SAR images. In particular, σ₀ and γ₀ features had their own special advantages. Between peanut and maize, peanut and rice, and maize and rice, the γ₀ feature was better than the σ₀ feature. The rest were the opposite. Hence, the σ₀ and γ₀ features were combined to exploit whether there was an improvement in the J-M distance values. The results are indicated in

Table 9. J-M distances for crops σ₀ and γ₀ combination features based on multi-temporal images.

J-M Distance	Single-Temporal Maximum	3 July 2021 15 July 2021	3 July 2021 15 July 2021 27 July 2021	3 July 2021 15 July 2021 27 July 2021 8 August 2021	3 July 2021 15 July 2021 27 July 2021 8 August 2021 20 August 2021	3 July 2021 15 July 2021 27 July 2021 8 August 2021 20 August 2021 1 September 2021
	σ0-γ0	σ0-γ0	σ0-γ0	σ0-γ0	σ0-γ0	σ0-γ0
Peanut–Corn	0.9521	1.0565	1.6045	1.8137	1.9698	1.9924
Peanut–Rice	1.3609	1.5980	1.7670	1.9699	1.9949	1.9955
Peanut–Others	1.6040	0.7575	1.6647	1.9427	1.9865	1.9898
Corn–Rice	1.1932	1.7442	1.8415	1.9544	1.9837	1.9961
Corn–Others	0.5228	0.7368	0.9466	1.8338	1.9957	1.9933
Rice–Others	1.7385	1.6092	1.8117	1.9998	1.9999	1.9995

. The combined σ₀ and γ₀ features from SAR image T showed the greatest degree of separation, with the J-M distance values between each crop even exceeding 1.9. It meant that the combination of σ₀ and γ₀ was productive and samples exhibited a good separability. Therefore, this feature was selected as the optimal feature combination for subsequent crop identification.

4.4. Model Generation

The SAR images have two polarizations, four features can be obtained for each SAR image. They are σ₀ VV, σ₀ VH, γ₀ VV, and γ₀ VH. Since the optimal feature combination contains six images, each sample had a feature shape of (1, 24), as shown in Figure 7. In order to facilitate the input of the samples into C-AENN, they were reshaped. The shape of each sample was changed to (12, 2). After that, the reshaped sample features were fed to the classifier.

The classifier C-AENN was built to identify crops. The detailed parameters of C-AENN are shown in

Table 10. Parameters of C-AENN.

Layer	Parameters	Output Shape
Input		(n, 12, 2)
Conv1D	Filters = 32, kernel_size = 7, activation = ‘relu’	(n, 12, 32)
MaxPooling1D	Pool_size = 2	(n, 6, 32)
BatchNormalization		(n, 6, 32)
Conv1D	Filters = 64, kernel_size = 5, activation = ‘relu’	(n, 6, 64)
MaxPooling1D	Pool_size = 2	(n, 3, 64)
BatchNormalization		(n, 3, 64)
Conv1D	Filters = 128, kernel_size = 3, activation = ‘relu’	(n, 3, 128)
MaxPooling1D	Pool_size = 2	(n, 2, 128)
BatchNormalization		(n, 2, 128)
Flatten		(n, 256)
Encoder1	128, activation = ‘relu’	(n, 128)
Encoder2	64, activation = ‘relu’	(n, 64)
Encoder3	32, activation = ‘relu’	(n, 32)
Compressed features	16, activation = ‘relu’	(n, 16)
Decoder1	32, activation = ‘relu’	(n, 32)
Decoder2	64, activation = ‘relu’	(n, 64)
Decoder3	128, activation = ‘relu’	(n, 128)
Classification	Softmax	(n, 4)

. After the training samples were trained, a crop type identification model was generated.

4.5. Accuracy Comparison with Other Classifiers

To demonstrate the effectiveness of the model generated by C-AENN, several classifiers were used for comparison, which included SVM, RF, KNN, ANN, 1D-CNN, and SAE. The models generated by these classifiers were evaluated quantitatively and qualitatively.

4.5.1. Quantitative Evaluation

The test samples were fed into the models, then the outputs of models were compared with the ground truth sample labels to generate confusion matrices, which are shown in Figure 8. According to the confusion matrices, the models generated by the above-mentioned classifiers were quantitatively evaluated. All these models showed good accuracies, which denoted that the optimal identification features played an effective role. The results are listed in

Table 11. Quantitative evaluation of models generated by different classifiers.

		Peanut	Corn	Rice	Other	Macro-F1	OA	Kappa
SVM	Precision	0.91	0.93	0.99	0.83	0.9125	0.9273	0.8956
	Recall	0.92	0.95	0.97	0.79
	F1-score	0.92	0.94	0.98	0.81
RF	Precision	0.85	0.86	0.97	0.78	0.8400	0.8757	0.8191
	Recall	0.88	0.93	0.95	0.54
	F1-score	0.87	0.89	0.96	0.64
KNN	Precision	0.85	0.86	0.99	0.77	0.8450	0.8760	0.8201
	Recall	0.90	0.93	0.91	0.59
	F1-score	0.87	0.89	0.95	0.67
ANN	Precision	0.89	0.94	0.98	0.92	0.9350	0.9358	0.9082
	Recall	0.92	0.93	0.96	0.93
	F1-score	0.91	0.94	0.97	0.92
1D-CNN	Precision	0.97	0.97	0.98	0.94	0.9650	0.9671	0.9529
	Recall	0.93	0.97	0.99	0.96
	F1-score	0.95	0.97	0.99	0.95
SAE	Precision	0.93	0.95	0.99	0.96	0.9550	0.9584	0.9404
	Recall	0.97	0.97	0.97	0.90
	F1-score	0.95	0.96	0.98	0.93
C-AENN	Precision	0.97	0.98	0.99	0.97	0.9825	0.9794	0.9705
	Recall	0.97	0.98	0.99	0.97
	F1-score	0.97	0.98	0.99	0.99

.

Among them, the C-AENN model achieved the best accuracy. Macro-F1 scored 0.9825, OA scored 0.9794, and Kappa scored 0.9705. In addition, 1D-CNN and SAE models also achieved good performance, and the accuracy metrics were slightly lower than the C-AENN model. They achieved Macro-F1 of 0.9650 and 0.9550, OA of 0.9671 and 0.9584, and Kappa of 0.9529 and 0.9404, respectively. Experiments proved that the combination of 1D-CNN classifier and SAE classifier improved the accuracy of using either classifier alone. Macro-F1 and Kappa were improved by approximately 2~3%, and OA increased by approximately 1~2%. Nevertheless, the RF model performed the worst with Macro-F1 at 0.8400, OA at 0.8757, and Kappa at 0.8191. Compared with the results of the proposed C-AENN model, Macro-F1 was approximately 14% lower, OA was about 10%lower, and Kappa was about 15% lower.

Moreover, in order to objectively see the model ability of each classifier, a visual comparison of three more representative accuracy evaluation indicators, Macro-F1, OA, and Kappa, is presented in Figure 9. The accuracy of the models generated by classifiers can be ranked in the following order: C-AENN > 1D-CNN > SAE > ANN > SVM > KNN > RF. It demonstrates that the neural network models outperformed the machine learning models on our dataset. In particular, the C-AENN model exhibited an outstanding capability.

4.5.2. Qualitative Evaluation

The qualitative assessment was carried out in two ways, one of which was to select four typical areas, and compare them with the sample maps for visual interpretation. The second was a comparison with contemporaneous optical images. Both of them gave an intuitive performance assessment of the proposed method.

In the first place, four typical regions A (peanut), B (corn), C (rice), and D (others) were selected, as shown in Figure 10, to present the effects of the models generated by various classifiers. Figure 10a gave the location of the selected regions, and Figure 10b displayed the sample maps for these regions. The comparison results are exhibited in Figure 11. It can be clearly seen that the effects of 1D-CNN, SAE, and C-AENN models were better than others, and there were fewer mistakes. In region A (peanut), numerous peanut pixels were misclassified as maize. In contrast to the sample map, SAE and C-AENN models had better results. In region B (corn), the 1D-CNN, SAE, and C-AENN models showed superiority over the other models. In regions C (rice) and D (others), 1D-CNN and C-AENN models were relatively good. The error rate was significantly reduced. In summary, the proposed C-AENN model had better robustness.

Next, the C-AENN model was used to predict the entire study area. Figure 12 shows a qualitative comparison of the identification result for multiple crops with the optical image of the same period and the identification result covering the study area is given in Figure 12a. Due to the difficulty in obtaining real ground conditions covering the entire study area, the high-resolution the Sentinel-2 optical image on 25 August 2021 was used for qualitative evaluation. The band combinations of Sentinel-2 image can extract specific information to help better understand the ground objects. The image of bands B11 (SWIR-1), B8 (NIR-1), and the B2 (Blue) combination was defined as a pseudo-color image, as shown in Figure 12b. It is mainly used to monitor crops because it applies shortwave and near-infrared bands. Both of these two bands are more sensitive to vegetation.

Ignoring the effects of clouds, the prediction result of the C-AENN model was largely consistent with the pseudo-color image. As can be seen in Figure 12a,b, the study area was dominated by maize cultivation, which is grown in all counties and districts. Additionally, the red rectangular boxes in Figure 12a,b were the main peanut cultivation areas and they were mainly distributed in Weishi, Tongxu, and Qixian counties. Moreover, the red circles in Figure 12a,b were the main rice cultivation areas. The rice was grown in a relatively concentrated area, which was mainly on the border between the Shunhe and Xiangfu districts. Furthermore, due to the cultivation structure of the “others“ type isrelatively complex, there were no distinctive features in Figure 12b. Combined with the fieldwork records, the “others“ category was mainly located in Xiangfu district and Tongxu county, which were basically consistent with the results in Figure 12a. In short, the crop identification results almost matched the distribution in the optical images of the same period, which further validates the effectiveness of the model.

5. Discussion

In this work, the study of crop type identification methods using multi-temporal Sentinel-1A images was carried out. Good results were achieved through feature optimization and building a new classifier. At first, the unique physical characteristics of σ₀ and γ₀ features of SAR images were extracted, and their combination showed a better J-M distance value, which provided an optimal identification feature for the crop classification model. Then, a new classifier C-AENN was developed for the type identification task, which combined the 1D-CNN model and the SAE model. In the end, the quantitative and qualitative evaluation results demonstrated the validity of the method.

First of all, the extraction of the farmland mask was carried out to avoid the interference of non-farmland features on the identification results of crop types. The public mask data Global Food Security-Support Analysis Data at 30 m (GFSAD30) product has been used in many agricultural fields. However, due to the development of urbanization in China, farmland is changing rapidly, therefore, relatively timely and accurate mask data are required. Figure 13 shows the results for the publicly available GFSAD30 mask data and the proposed mask data in this study. Additionally, this paper selected two enlarged regions E and F for comparison. As can be seen from Figure 13, both mask data can detect non-farmland areas well, such as cities and towns, while the proposed mask was better in terms of edge detail for cities and towns, which basically completed the extraction of the farmland.

Next, this paper demonstrated the importance of multi-temporal radar data for crop type identification. Good accuracy can be obtained even using classical machine learning methods such as SVM and RF. The optimal selection of features and the J-M distance analysis played a useful role. In this paper, J-M distance analysis was performed on σ₀ features, γ₀ features, and σ₀ and γ₀ combination features of single-temporal and multi-temporal SAR images. It was found that the combination of σ₀ and γ₀ features can significantly enhance the J-M distance values between crop samples, and the multi-temporal images were apparently better than the single-temporal image. Therefore, multi-temporal σ₀ and γ₀ features combination is the best choice for the optimal identification feature. Most of the previous studies have mostly neglected the role of γ₀, surprisingly, the combination of σ₀ and γ₀ yielded better results, and the application of γ₀ can be paid more attention in subsequent studies.

In the end, this paper demonstrated that the C-AENN classifier outperformed the classical machine learning methods SVM and RF, etc., and the neural network methods 1D-CNN and SAE, etc. Based on the optimal identification feature, the most representative machine learning methods, SVM, RF, and KNN, were chosen for comparison and they were still competitive with other methods in many cases. As can be seen in Figure 8, these machine learning methods achieved better results in the type identification of peanut, corn, and rice crops, but the identification accuracy of the “other” category was lower. However, this misclassification was not so high for neural network methods such as 1D-CNN and SAE. This showed that the neural network approach outperformed the traditional machine learning approach in our dataset. In particular, the proposed C-AENN showed good classification performance in every category, exceeding 97% in all quantitative evaluation metrics. 1D-CNN and SAE also showed better classification accuracy, however, in general, the accuracy was slightly lower than the proposed method. Additionally, this paper explored the time consumption of 1D-CNN, SAE, and C-AENN. Through ten experiments, it was concluded that the time consumption of SAE was relatively less, and the time consumption of 1D-CNN and C-AENN was similar. The result is shown in Figure 14. For research, it is worthwhile to use C-AENN to improve accuracy compared with 1D-CNN alone and SAE alone.

With regard to the selection of network parameters, this paper was based on the data of the study area and multiple debugging. The selection of convolutional kernels in 1D-CNN is illustrated as an example. The size of the convolution kernel was related to the local feature extraction dimension. According to the optimal identification feature shape of (12, 2), this paper attempted smaller convolutional kernels, such as 7, 5, 3. Then, based on several debugging sessions, the optimal parameters can be selected. As for the node settings of SAE, the last layer of 1D-CNN in C-AENN was 256, so some common parameter settings of 128, 64, 32, etc. can be attempted, and the optimal parameters through multiple tests can be determined. It is worth noting that the parameter settings of the deep learning methods have their unexplainable aspects, which need specific analysis for specific problems. This is also the direction that needs to be studied in follow-up research.

6. Conclusions

In this study, a novel method was proposed for crop type identification based on the C-AENN classifier using time-series Sentinel-1A images. The method first removed non-farm areas by the OMRF model. Then features were extracted for each type of crop and the optimal identification feature, the combined time-series backscatter coefficient σ₀ and γ₀, was selected by J-M distance analysis. Finally, the optimal identification feature was used for crop type identification model generation by using the C-AENN classifier, which was fused by 1D-CNN and SAE. The quantitative and qualitative model assessments demonstrated the validity of the proposed method.

In terms of quantitative assessment, the effectiveness of the proposed method was illustrated by comparison with the traditional machine learning methods SVM, RF, and KNN, as well as the deep learning methods ANN, 1D-CNN, and SAE. Six evaluation metrics directly reflected the ability of the various methods to identify different crops. Compared with other methods, the proposed method achieved the highest F1, OA, and Kappa. All of these evaluation indicators were above 97%.

In addition, in terms of qualitative assessment, four typical areas were selected for visual presentation in comparison with the sample maps. The model identification result covering the study area was compared with the contemporaneous optical image, which proved that the proposed method gave satisfactory results. In conclusion, both the quantitative and qualitative assessments proved that the result of crop type identification significantly improved.

Furthermore, a few interesting things are found in this paper, such as the unique advantage of combining multi-temporal σ₀ and γ₀ features; they can obtain higher classification accuracy without using other auxiliary data. In addition, compared with one network framework, a joined neural network plays a better role. This study demonstrates the validity of time-series SAR data and the importance of deep learning methods for feature mining. Both methods have great potential for crop type identification, which have promising application prospects in agricultural management.