A New Coupled Elimination Method of Soil Moisture and Particle Size Interferences on Predicting Soil Total Nitrogen Concentration through Discrete NIR Spectral Band Data

Abstract

Rapid and accurate measurement of high-resolution soil total nitrogen (TN) information can promote variable rate fertilization, protect the environment, and ensure crop yields. Many scholars focus on exploring the rapid TN detection methods and corresponding soil sensors based on spectral technology. However, soil spectra are easily disturbed by many factors, especially soil moisture and particle size. Real-time elimination of the interferences of these factors is necessary to improve the accuracy and efficiency of measuring TN concentration in farmlands. Although, many methods can be used to eliminate soil moisture and particle size effects on the estimation of soil parameters using continuum spectra. However, the discrete NIR spectral band data can be completely different in the band attribution with continuum spectra, that is, it does not have continuity in the sense of spectra. Thus, relevant elimination methods of soil moisture and particle size effects on continuum spectra do not apply to the discrete NIR spectral band data. To solve this problem, in this study, moisture absorption correction index (MACI) and particle size correction index (PSCI) methods were proposed to eliminate the interferences of soil moisture and particle size, respectively. Soil moisture interference was decreased by normalizing the original spectral band data into standard spectral band data, on the basis of the strong soil moisture absorption band at 1450 nm. For the PSCI method, characteristic bands of soil particle size were identified to be 1361 and 1870 nm firstly. Next, normalized index N_p, which calculated wavelengths of 1631 and 1870 nm, was proposed to eliminate soil particle size interference on discrete NIR spectral band data. Finally, a new coupled elimination method of soil moisture and particle size interferences on predicting TN concentration through discrete NIR spectral band data was proposed and evaluated. The six discrete spectral bands (1070, 1130, 1245, 1375, 1550, and 1680 nm) used in the on-the-go detector of TN concentration were selected to verify the new method. Field tests showed that the new coupled method had good effects on eliminating interferences of soil moisture and soil particle size.

1. Introduction

Excessive application of commercial fertilizers has been cited as a source of contamination of surface and groundwater [1,2,3,4]. As one of the important parts of precision agriculture (PA), based on the needs of the plant and the soil nutrients state, variable rate (VR) fertilization can reduce the overuse of manure in the farmlands to protect the soil and environment [5,6,7,8]. Rapid, real-time, and accurate acquisition of high-resolution soil parameter information is the basis of applying VR fertilization operation [9].

As the main nutrient for plant and soil, soil total nitrogen (TN) is one of the key factors that determine plant nutrient levels and soil fertility [10,11]. The estimation of TN concentration status is crucial from agricultural and environmental points of view [12,13,14]. Traditional wet chemical methods of measuring TN concentration are time-consuming, costly, have low efficiency, and also cause environmental pollution due to their consumption of harmful chemicals [15,16,17]. Spectral analysis technology is a fast, nondestructive, on-line, and free pollution method that shows an increasing development potential in TN concentration measurement [18,19,20,21]. Many researchers focused on exploring rapid TN detection methods and corresponding soil sensors based on spectral technology in the past four decades. In the 1980s, Dalal and Henry [22] showed it is possible to measure TN concentration through near-infrared (NIR) reflectance spectroscopy. Since then, many scholars have succeeded to predict TN concentration based on NIR reflectance spectroscopy in the laboratory [23,24,25].

However, collecting soil samples in the field requires a lot of human labor. It will also take a relatively long time to obtain the measurement results [17]. Thus, estimation of TN concentration through NIR reflectance spectroscopy in the laboratory does not satisfy the requirement of VR fertilization operation. Simultaneously, many scholars focus on developing soil parameter sensors to supply rapid and high-resolution acquisition of soil parameters. Sudduth and Hummel [26] designed and tested a portable, NIR spectrophotometer intended for in-field use. Mouazen et al. [27,28] developed a soil sensing device with the measuring range of the spectrometer from 306.5 to 1710.9 nm. Decreasing the capital is significant for the development of soil sensors. Thus, cheaper light-emitting diodes (LEDs), laser diodes, and tungsten halogen light sources are used as the alternative component of the spectrometer to develop soil sensors [29,30,31]. Tang et al. [32] developed a portable SOC analyzer using 850 nm as a sensitive wavelength. An et al. [31] developed a portable TN sensor based on six discrete NIR bands (940, 1050, 1100, 1200, 1300, and 1550 nm) to detect TN content, which used the LEDs as the detection light source. Zhou et al. [30] developed an in-situ TN detector with seven discrete NIR bands (1260, 1330, 1360, 1430, 1530, 1580, and 1660 nm) based on the laser diodes, which realized in-situ detection of TN concentration in the farmlands. Li et al. [33] developed the portable TN concentration detector with six discrete NIR bands (1108, 1248, 1336, 1450, 1537, and 1696 nm), which used the tungsten halogen light sources as the detection light source.

However, several problems remain through spectral technology to predict soil parameters. First, the soil spectra are easily affected by interactions with various soil constituents (e.g., minerals and soil nutrients) [34,35]. Especially, field soil samples have more variations in soil moisture and other physical conditions (i.e., soil particle size, and water). Water and hydroxyls (O-H) have strong influences in the NIR region. Overtones of O-H stretching absorb near 1450 nm, and it conceals the absorption information of soil N-H bonds in the NIR spectroscopy, resulting in the spectral curve only reflecting the changing trend of soil moisture. However, the absorption information of soil N-H bonds reflects the concentration of TN. Reflectance from a soil sample varies with its soil particle size in the NIR regions of the electromagnetic spectrum. As the soil particle size decreased, the reflectance also gradually increased. It also conceals the absorption information of soil N-H bonds in the NIR spectroscopy. Thus, the soil moisture and particle size effects on the NIR spectroscopy must be eliminated before performing the estimation of TN by the NIR spectroscopy [36,37,38].

Many scholars have analyzed the interferences of soil particle size and moisture on soil parameter prediction, and several studies have proposed methods to eliminate these disturbances. For the soil moisture, Tekin et al. [39] used NIR reflectance spectroscopy to study the effects of different soil moisture contents on soil organic matter (SOM) prediction. The root mean square error of validation (RMSEP) and the residual prediction deviation (RPD) for dry soil samples reached 1.26% and 2.83, respectively, and the soil samples with the highest moisture content had values of 1.55% and 2.29, respectively. Zhang et al. [23] studied the real-time NIR spectra of soil samples without artificial drying and sieving, and six sensitive bands were determined for predicting TN concentration through wavelet analysis and continuum removal techniques. Prediction models of TN concentration were established through partial least squares regression (PLSR) and support vector machine (SVM), respectively. For the PLSR model, the coefficient of determination of calibration ($R_c^2$) was 0.602, and the coefficient of determination of prediction ($R_v^2$) was 0.634. For the SVM model, $R_c^2$ reached 0.823, and $R_v^2$ reached 0.810. It revealed that the soil moisture interference was removed from the real-time soil NIR reflectance spectroscopy during TN prediction using the proposed method of wavelet analysis and continuum removal. External parameter orthogonalization, direct standardization, global moisture modeling, slope bias correction, and selective wavelength modeling have also been applied to eliminate the effect of soil moisture on soil NIR reflectance spectroscopy [40]. For the soil particle size, Barthès et al. [41] analyzed the NIR reflectance spectroscopy of sandy soils with different particle sizes in Burkina Faso and Congo and established the prediction models of TN concentration under different soil particle sizes. The results showed that the model with less than 2 mm soil particle size was better than the model established using other soil particle sizes. Bogrekci and Lee [42] investigated the effects of soil particle size on the prediction of soil rapid available phosphorus (P) using ultraviolet, visible, NIR spectroscopy. They found that for the sandy soil, with the decrease in soil particle size, the soil reflectivity increased and the absorbance decreased. The individual prediction method for each soil particle size produced better soil P predictions with lower standard errors of prediction for coarse, medium, and fine sand particles than the method that removed the effect of particle size on absorbance spectra and predicted P concentration of the soil samples.

Summarizing the above research, there was no relevant research on the elimination of soil moisture and particle size effects on predicting soil parameters through the discrete NIR spectral band data. Soil parameter detectors developed based on discrete NIR spectral band data also face serious interferences caused by soil moisture and particle size. The methods used in the laboratory to eliminate the interference of soil particle size and moisture by sieving after grinding and drying cannot be applied to the discrete NIR spectral band data in the field sensing. The above elimination methods of soil particle size and moisture are not suitable for discrete NIR spectral band data acquisition in the field sensing. An et al. [36] studied the effects of soil moisture and particle size on the prediction of TN concentration using a portable TN detector developed based on six discrete NIR bands (940, 1050, 1100, 1200, 1300, and 1550 nm). The moisture absorption index and mixed calibration set were proposed to eliminate the soil moisture and particle size interferences, respectively. However, the mixed calibration set method entailed a complicated process and could not perform real-time elimination of the interference of soil particle size.

The combination effects and real-time elimination methods of soil moisture and particle size on predicting TN concentration through the discrete NIR spectral band data have rarely been explored. The discrete NIR spectral band data can be completely different in the band attribution with continuum spectra, it does not have continuity in the sense of spectra. Therefore, the study of methods for eliminating the effects of soil particle size and moisture on the discrete NIR spectral band data has important theoretical significance and practical application value for improving the performance of detectors based on the discrete NIR spectral band data. The main goal of the paper is to establish an on-line elimination method of soil moisture and particle size interferences, which could be used in the discrete NIR spectral band data to improve the measurement accuracy of soil parameters. Thus, in this study, six discrete NIR spectral bands (1070, 1130, 1245, 1375, 1550, and 1680 nm) used in the on-the-go detector of TN concentration were chosen as the object to perform the research [29]. A new coupled elimination method of soil moisture and particle size interferences on predicting TN concentration through discrete NIR spectral band data was proposed and evaluated.

2. Materials and Methods

2.1. Major Steps of the Study

Figure 1 shows the major procedures in this research, which included three stages. During stage 1, 60 soil samples were collected from Field 1. In order to research the effect of soil moisture on discrete NIR spectral band data, the soil particle size was eliminated by artificial sieving. Additionally, the moisture absorption correction index (MACI) method to eliminate the disturbance of soil moisture on discrete NIR spectral band data was proposed and validated. During stage 2, 24 soil samples were collected from Field 2. Before performing analysis, soil moisture disturbance was eliminated through drying by the oven. Next, four groups of soil samples with different particle sizes were obtained, each group contained six TN concentration levels, and each TN concentration level contained four soil samples. A total of 96 soil samples with different particle sizes and TN concentrations were generated from the 24 soil samples. Next, the effect of soil particle size on discrete NIR spectral band data was explored and the characteristic wavebands of soil particle size were determined. The particle size correction index (PSCI) method was proposed and validated to eliminate the effect of soil particle size on discrete NIR spectral band data.

During stage 3, a field experiment was performed in Field 3 to evaluate the coupled elimination method through the on-the-go detector, which included five processing methods. Finally, five models of TN concentration were calibrated and compared.

2.2. On-The-Go Detector of TN Concentration

An on-the-go detector of TN concentration was previously developed through discrete NIR spectral band data. This detector was reported in our early research [29]. Figure 2 shows the structure of the on-the-go detector, which consisted of mechanical, optical, and control units. The mechanical unit was the basis of the entire detector and supplied a downward force for the work of the subsoiler. Thus, the electrical control cabinet of the mechanical unit was designed as a three-tier structure to integrate the optical and control units. The optical unit is used for sensing soil in the soil detection trench and a control unit embedded into an SVM model for obtaining real-time readings of TN concentration. A tungsten halogen light source was a better choice for the vehicle equipment than LED light and laser sources with respect to light intensity and stability, which provided a broadband source over the 300–2500 nm range. The selected performance parameters of the tungsten halogen light were as follows: <25% of radiation uniformity, <1% of unstable time, 20 × 20 mm radiation area, adjustable radiation intensity, 150 W of maximum power, and 2000 h of average life. The light had to be transmitted to the soil surface with minimal light loss. Therefore, an NIR optical fiber with a transmission range of 800–1900 nm was used. The diameter of the selected filter was 5 mm, and the average full width at half maximum (FWHM) bandwidth was 20 nm. The selected photoelectric sensor was InGaAs with a responding range of 800–1800 nm and a spectral responsiveness of 0–0.9. The sapphire glass attached to the bottom of the sensor mounting plate was used to ensure that the sensors and the filters were free from soil pollution. When the on-the-go detector performed detection of TN concentration in farmland, it was mounted on the tractor through a three-point suspension structure. The depth of the subsoiler that pushed into the soil exceeded 200 mm, which was used to loosen the soil and smash large soil particles. The flattener made the soil more flat and suitable for spectrum detection. The NIR light emitted by the tungsten halogen light source was transmitted through an NIR fiber to the flat detection soil surface. The diffused reflection light of the detected soil was changed into a single-band light by each optical filter, and each photoelectric detector converted the single-band light into an electrical signal. All detected electrical signals were amplified, filtered, and digitized into absorbance data, and then all data were collected and displayed in real-time by the data receiving software embedded with the whiteboard data, by which and detected spectral data the absorbance data were calculated at each sensitive wavelength. For this on-the-go detector, the group of the discrete NIR spectral bands of 1070, 1130, 1245, 1375, 1550, and 1680 nm was selected as the sensitive wavebands to predict TN concentration. Furthermore, the on-the-go detector was used to embed the new coupled elimination method of soil moisture and particle size interferences on predicting TN concentration through discrete NIR spectral band data.

2.3. Experimental Materials and Methods

During stages 1 and 2 (Figure 1), soil samples were collected from the experimental farm of China Agricultural University on the outskirts of Beijing to perform the research of soil moisture and particle size interference elimination method. The soil type was common brown soil in northern China. The experimental area is in the range of north latitude 40.15395721–40.1534362 and east longitude 116.2155061–116.2150130. Since the fertilization usually occurs from the soil surface to the soil depth of 30 cm, soil samples were collected from 5 to 30 cm after removing 5 cm of the topsoil [43]. A total of 60 soil samples used to study the effect of soil moisture were collected at Field 1 (F1, 10 ha). Corn and wheat were cropped in F1 with tillage. In total, 24 soil samples were collected from the Field 2 (F2, 6 ha), which was a standard field without fertilization (the soil nutrient concentration is almost zero). The soil textural fractions (clay, silt, and sand) of F2 were the same with F1, both the two fields were sandy loam. In total, 2–2.5 kg of soil at each sampling point was collected and put it into double-layer cowhide bags for sealing to prevent the loss of soil moisture. A total of 84 soil samples were collected.

The first set of soil samples (N = 60) was used to study the effect of soil moisture. All soil samples were sieved under laboratory conditions using a 20-mesh sifter (0.9 mm) to produce pretreated samples with consistent particle sizes [36].

Another set of soil samples (N = 24) was used to study the effect of soil particle size. In order to obtain soil samples with different TN concentration, the standard soil samples were first dried in a drying box, the temperature was set to 85 ℃, and the time for drying was 24 h [44,45]. The air-dried soil samples were randomly divided into six groups, four in each group. The soil nitrogen solution was configured with urea [46,47,48]. The concentration gradient of the soil nitrogen solution is divided into 1–6 grades, grade 1 nitrogen concentration is 0 g·kg⁻¹, grade 2 nitrogen concentration is 0.04 g·kg⁻¹, grade 3 nitrogen concentration is 0.08 g·kg⁻¹, grade 4 nitrogen concentration is 0.12 g·kg⁻¹, grade 5 nitrogen concentration is 0.16 g·kg⁻¹, and grade 6 nitrogen concentration is 0.2 g·kg⁻¹. When configuring the soil samples, we simulated the average summer soil moisture content (7%) of the cultivated layer of the Shangzhuang Experimental Station to configure the soil samples. After the soil configuration was completed, the drying process was carried out, and all the soil samples were sieved, respectively, 10-mesh sieves (2.0 mm), 20-mesh sieves (0.9 mm), 40-mesh sieves (0.45 mm), and 80-mesh sieves (0.2 mm). Finally, four groups of soil samples with different particle sizes were obtained, each group contained six TN concentration levels, and each TN concentration level contained four soil samples. A total of 96 soil samples with different particle sizes and TN concentrations were generated from the 24 soil samples.

During stage 3 (Figure 1), soil samples for verification were used to evaluate the new coupled elimination method. A verification experiment was conducted on the experimental farm of China Agricultural University. The soil type was common brown soil in northern China Field 3 (F3, 5.5 ha) which was one long-term research site located within 1.0 km of F1. Corn and wheat were cropped in F3 with tillage. The soil textural fractions (clay, silt, and sand) of F3 were the same as that of F1, which were sandy loam. Figure 3 shows the location and soil samples collection point in Field 3.

The experiment consisted of two parts which were data of discrete NIR spectral band data acquisition and soil samples collection. The experimental plot was 240 × 190 m, which was divided into 108 sampling cells. Each cell had a dimension of 5 × 20 m. Data and soil samples were collected every 10 m in each cell, blue cells in Figure 3 show the location of acquiring data and soil samples. All samples were placed in a double-thickness sampling bag to prevent moisture dispersion and then sent to the laboratory for optical and chemical measurements.

2.4. Laboratory Measurement

For each soil sample, it was divided into two parts, one was used for TN concentration measurement, and another was used for NIR reflectance spectroscopy. The TN concentration was tested with the FOSS Kjeltec™2300 Nitrogen Analyzer produced by FOSS, Sweden [49]. After grinding each soil sample, weighing 2.0 g, was put into a long test tube. The long test tube was also filled with 6.2 g catalyst. K₂SO₄:CuSO4.5H₂O was mixed and ground at 30:1. Finally, 20 mL of concentrated sulfuric acid was added to the long test tube for nitrification. The temperature of the nitration furnace was set to 420 °C for 2.5 h [50]. After nitrification, it was cooled down and the total nitrogen concentration was measured with the FOSS Kjeltec™2300 nitrogen analyzer. The NIR reflectance spectroscopy of soil samples was collected by the MATRIX-I Fourier Transform NIR Spectroscopy Analyzer produced by Bruker, Germany. Before soil spectra collection, the analyzer parameters needed to be set. The spectrum collection range, spectrum sampling interval, and scan times were set to 780–2550 nm, 3 nm, and 64 times, respectively. Approximately 20 g of soil samples were put into a quartz cuvette with a diameter of 50 mm, and put into the rotating sample cell of the analyzer for spectrum detection [17,51].

2.5. Model Accuracy and Methodology

In addition to the coefficient of determination (R²), the root mean square error of validation (RMSEP), and the residual prediction deviation (RPD) were used to evaluate the prediction model of TN concentration [52]. Equations (1) and (2) present their calculation Equations.

$$\mathrm{RMSEP} = \sqrt{\frac{{{\displaystyle \sum }}_{j = 1}^{n_p}{\left(y_j-{\widehat{y}}_{pj}\right)}^2}{n_p-1}},$$

(1)

$$\mathrm{RPD} = \sqrt{{{\displaystyle \sum }}_{i = 1}^{n_p}{\left(y_j-{\overline{y}}_j\right)}^2/\left(n_p-1\right)}/\sqrt{{{\displaystyle \sum }}_{i = 1}^{n_p}{\left(y_j-{\widehat{y}}_{pj}\right)}^2/\left(n_p-1\right)},$$

(2)

where j is the number of soil samples, y_j is the TN concentration measured with the FOSS Kjeltec™2300 Nitrogen Analyzer, ${\widehat{y}}_{cj}$ is the TN concentration predicted by the modeling set, ${\widehat{y}}_{pj}$ is the TN concentration predicted by the validation set, ${\overline{y}}_j$ is the average value of the TN concentration measured with the FOSS Kjeltec™2300 Nitrogen Analyzer, n_c is the model set sample number, and n_p is the validation set sample number.

3. Results

3.1. Research on Eliminating the Interference of Soil Moisture

3.1.1. Interference of Soil Moisture on Discrete Spectral Band Data

The first set of 60 calibration samples with consistent particle sizes were measured through the developed on-the-go detector. To analyze the effect of soil moisture on soil discrete NIR spectral band data quantitatively, the spectral data of a soil sample with TN concentration of 0.1135 g·kg⁻¹ was used as the research object. Figure 4 shows the absorbance values at seven sensitive wavelengths under different soil moisture contents. The absorbance curve obtained by the on-the-go detector from 1070 to 1680 nm showed a consistent trend, indicating that the curve was a good reflection of the soil spectral information [53]. Taking the soil absorbance at 1450 nm as an example, when the soil moisture content increased from 3.18% to 14.16%, the soil absorbance increased from 0.79 to 1.11, and the soil spectral absorbance increased by 40.51%. Absorbance data analysis indicated that along with the soil moisture content increase, the absorbance of the soil spectrum increased rapidly. Soil moisture had a serious influence on soil discrete NIR spectral band data [54,55,56]. The effects of soil moisture must be decreased when predicting TN concentration.

Our goal is to eliminate the influence of soil moisture on the prediction of TN concentration through the on-the-go detector. Soil NIR spectroscopy indicates that three distinct moisture absorption peaks are present in the soil NIR spectroscopy at 1450, 1940, and 2210 nm [57,58]. However, in our research, we only focused on the absorption peak of O-H in H₂O at 1450 nm. O-H have strong influences in the NIR spectroscopy. Overtones of O-H stretching absorb near 1450 nm, and it conceals the absorption information of soil N-H bonds, resulting in the spectral curve only reflecting the change trend of soil moisture. However, the absorption information of soil N-H bonds reflects the concentration of TN. If the raw spectrum of the measured soil can be normalized to a standard spectrum with the same soil moisture content, the absorption information of soil N-H bonds on the NIR spectroscopy could be effectively extracted. Thus, the interference of soil moisture on predicting TN concentration could be decreased. However, when using the on-the-go detector to measure soil discrete NIR spectral band data, we were unable to directly obtain the soil moisture content in the fields. Thus, an alternative method must be explored to solve this problem.

3.1.2. Eliminating the Interference of Soil Moisture on Discrete NIR Spectral Band Data

To eliminate the interference of soil moisture, we need to classify different soil samples based on soil moisture content. The previous analysis revealed the difficulty of obtaining soil moisture content in real-time. Therefore, the MACI method was proposed to eliminate the effect of soil moisture on predicting TN concentration through the on-the-go detector. It was used to convert raw soil spectral data into standard spectral data. The MACI value of the on-the-go detector was calculated using Equation (3).

$$\mathrm{MACI} = \frac{1-A_{1450}}{1+A_{1450}}\times 100,$$

(3)

where A₁₄₅₀ is the spectral absorbance value of the soil samples at 1450 nm and MACI is the soil moisture absorption index correlation. Soil samples with different moisture contents can be divided into groups based on MACI values; thus, we divided the detected soil samples into five types based on MACI values.

Table 1. Accuracy of MACI classification and correction factors.

Serial Number	Soil Moisture Content Grade	MACI	Accuracy Rate (%)	Wj
1	Low level (0–3.0)	>9.11	100	1
2	Low-medium level (3.0–6.0)	9.11–6.05	88	0.92
3	Middle level (6.0–10.0)	6.05–2.03	86	0.86
4	High-medium level (10.0–13.0)	2.03–1.12	81	0.76
5	High level (>13.0)	<1.12	100	0.68

Note: MACI = moisture absorption correction index.

shows the five MACI values when the MACI value is used to represent the soil moisture level. Their corresponding ranges of soil moisture content and accuracy are also shown. When the soil moisture was between 0% and 3.0%, the MACI value was less than 8.66. When the soil moisture was 3.0–6.0%, 6.0–10.0%, and 10.0–13.0%, the MACI values were 8.66–12.66, 12.66–14.66, and 14.66–16.66, respectively. When the soil moisture content was higher than 13%, the MACI value was greater than 16.6.

The classification of soil samples was the best according to the classification of MACI values. Although the soil moisture content could not be directly obtained in real-time when the on-the-go detector was used to measure soil in the fields, the soil samples could be classified by MACI.

Each classification group required a correction factor to normalize the spectral data, and Equation (5) was used to calculate the correction factor. In this equation, j (j = 1, 2,..., 5) is the serial number of the taxonomic group, W_j is the correction factor for the taxonomic group j, A_avgj is the soil sample average absorbance of group j, and A_avgs is the soil sample average absorbance of group 1. The soil sample average absorbance value of group 1 was used as the reference absorbance value. Table 1 also shows all five correction factors.

$$W_j = \frac{A_{avgj}}{A_{avgs}}$$

(4)

After determining the correction factor W_j (j = 1, 2,…, 5), the raw absorbance data were corrected according to

$$A_{jc} = A_j\times W_j,$$

(5)

where A_j is the soil sample original absorbance value of group j, A_jc is the soil sample corrected normalized absorbance value, and W_j is the correction factor.

Absorbance data at 1070, 1130, 1245, 1375, 1450, 1550, and 1680 nm were detected using the on-the-go detector. Using the proposed MACI method to eliminate the interference of soil moisture, the wavelength absorbance at 1450 nm was selected to calculate the MACI value, and the raw spectral data were normalized to eliminate the interference caused by soil moisture content. Figure 5 shows the different absorbance curves for the change in soil moisture content after normalization. When the soil moisture content was 3.18% and 14.16%, the absorbance values at 1450 nm were 0.72 and 0.75, respectively; the absorbance value only increased by 4.16%. Compared with the soil spectrum without correction by the MACI method, the effect of soil moisture on soil discrete NIR spectral band data was greatly reduced, and the MACI method proposed in this study was proven to be effective in reducing soil moisture on the discrete NIR spectral band data.

3.2. Research on Eliminating the Interference of Soil Particle Size

3.2.1. Eliminating the Interference of Soil Moisture on Discrete NIR Spectral Band Data

Table 2. Statistics of 96 soil samples prepared. The range of soil total nitrogen (TN) represents the measured TN of soil samples prepared.

Soil Particle Size	TN Level g·kg−1	Range of TN g·kg−1	Particle Size	TN Level g·kg−1	Range of TN g·kg−1
2.0 mm	0	0.005–0.012	0.9 mm	0	0.003–0.012
	0.04	0.036–0.047		0.04	0.029–0.037
	0.08	0.071–0.077		0.08	0.076–0.091
	0.12	0.109–0.123		0.12	0.119–0.133
	0.16	0.162–0.171		0.16	0.153–0.176
	0.2	0.186–0.194		0.2	0.191–0.206
0.45 mm	0	0.006–0.02	0.2 mm	0	0.009–0.018
	0.04	0.031–0.042		0.04	0.026–0.04
	0.08	0.063–0.071		0.08	0.060–0.067
	0.12	0.119–0.131		0.12	0.128–0.138
	0.16	0.142–0.153		0.16	0.153–0.169
	0.2	0.178–0.190		0.2	0.187–0.199

Note: TN = soil total nitrogen.

statistically analyzes the 96 soil samples prepared. The TN concentration in the soil was detected by the FOSS Kjeltec™2300 Nitrogen Analyzer [49]. The TN concentration value shows that the TN concentration of the prepared soil sample is distributed in the range of 0.003–0.206 g·kg⁻¹, with approximately 0.04 g·kg⁻¹ as the TN concentration interval, divided into six TN concentration grades, the gradient distribution of TN concentration grade is reasonable.

According to the analysis in Table 2, although the soil samples are collected from standard fields that are not fertilized all year, there is still TN in the soil samples, which will affect the TN concentration of the prepared soil samples.

Using the drying oven at 80 °C for 24 h, the second set of 96 soil samples was dried to eliminate the influence of soil moisture [17]. Subsequently, all soil samples in the second set were sieved by sifters of different meshes, and discrete NIR spectral band data were obtained by the on-the-go detector. Figure 6 shows the absorbance value of a single soil sample at six discrete NIR spectral bands with the TN concentration of 0.086 g·kg⁻¹, including five curves, which are the original soil spectrum, the soil spectra with soil particle sizes of 2.0, 0.9, 0.45, and 0.2 mm. When the soil particle size decreased from 2 to 0.2 mm, the corresponding soil samples’ average absorbance also decreased from 0.74 to 0.63, and the overall absorbance changed by 14.86%. The results showed that soil particle size had a considerable effect on the soil discrete NIR spectral band data. When the soil particle size was between 0.45 and 0.9 mm, the absorbance of soil discrete NIR spectral band data was relatively stable. Larger and smaller particle sizes will cause a large difference on soil absorbance. Figure 6 also shows that the air-dried method could significantly decrease the interference of soil moisture on the spectral band of 1450 nm. Therefore, no absorption peak was observed at 1450 nm.

3.2.2. Identification of Characteristic Wavebands of Soil Particle Size

Figure 7 shows the spectral standard deviation curves of four different soil particle sizes. It can be concluded from Figure 7 that the standard deviation value in the range of 850–1000 nm was smaller than in the spectral range of 1000–2500 nm. As each spectral standard deviation curve is under the same particle size, the interference of soil particle size is eliminated, and only the TN concentration is different. Thus, the 1000–2500 nm range is the sensitive spectral region of TN concentration. Simultaneously, the black curve in Figure 7 is the spectral standard deviation curve of all 96 soil samples, by the comparative analysis between the five standard deviation curves in the spectral range of 1000–2500 nm. After adding the difference in soil particle size, the black standard deviation curve in the spectral range between 1340–1420 nm and 1795–1955 nm changed up and down. In order to show the changes between different spectral standard deviation curves, spectral ranges 1340–1420 and 1795–1955 nm of the spectral standard deviation curve were extracted in Figure 7. Obviously, this change indicates that the above two spectral regions are closely related to the change of soil particle size and are sensitive spectral regions of soil particle size. Therefore, in this study, the extreme points 1361 and 1870 nm of the two spectral regions were selected as the characteristic wavebands of soil particle size.

3.2.3. Research on the Classification of Soil Particle Size

The 96 soil samples were divided into two groups, with 48 soil samples in each group. Each group of soil samples includes four soil particle size categories, each soil particle size category includes six TN concentration grades, and each soil concentration grade includes two soil samples. The two characteristic wavebands 1361 and 1870 nm were used to classify the soil particle size. The normalized index N_p was used as a single variable to predict the soil particle size, and the N_p is calculated by Equation (6). The soil classification model is constructed by the support vector machine (SVM) algorithm. After many attempts, the penalty parameter c of the SVM classification model is 2 and the kernel function is 1 to obtain the optimal classification result. Figure 8 shows the prediction results of soil particle size based on SVM. The overall classification accuracy rate of the four soil categories is 97.92%. Among the four soil categories, the classification accuracy rate when the soil particle sizes are 2.0, 0.45, and 0.2 mm is 100%, and the classification accuracy rate when the soil particle size is 0.90 mm is 91.67%.

$$N_p = \frac{A_{1870}-A_{1361}}{A_{1870}+A_{1361}},$$

(6)

In Equation (6): A₁₈₇₀ and A₁₃₆₁ are the absorbance of the soil at 1870 and 1361 nm.

3.2.4. Eliminating the Interference of Soil Particle Size

Based on the soil particle size classification model established by SVM, the normalized index N_p was used as a single input to classify the soil particle size. The results show that the method is feasible. Based on the above results, a soil particle size correction coefficient P_c was proposed to correct the origin discrete NIR spectral band data. The soil particle size correction coefficient P_c is obtained by Equation (7),

$$P_c = \frac{\left(\frac{A_{1870}-A_{1361}}{A_{1870}+A_{1361}} \right)}{{\overline{N}}_p},$$

(7)

In Equation (7): A₁₈₇₀ and A₁₃₆₁ are the absorbance of the corrected soil spectrum at 1870 and 1361 nm, and ${\overline{N}}_p$ is the normalized index of the absorbance of the 0.20 mm soil at 1870 and 1361 nm as the reference value. It is generally believed that the smaller soil particle size can eliminate the influence of soil particle size to the greatest extent, so the spectrum data with a particle size of 0.20 mm was determined as the reference spectrum.

The absorbance A_c of discrete NIR spectral band data corrected by the N_p was obtained by Equation (8),

$$A_c = A\times P_c,$$

(8)

In Equation (8): A is the soil absorbance of discrete NIR spectral band data before correction, and A_c is the soil absorbance after correction. In this study, the absorbance at six discrete NIR bands (1070, 1130, 1245, 1375, 1550, 1680 nm) of the on-the-go detector were used to verify the PSCI method. Figure 9 shows the absorbance of discrete NIR spectral band data after soil particle size correction. A comparative analysis of Figure 6 and Figure 9 was computed. For example, at 1245 nm, the difference between the absorbance value with a soil particle size of 0.2 mm and the absorbance value with soil particle sizes 2.0, 0.9, 0.45 mm, and original spectrum were 0.0737, 0.0506, 0.0361, and 0.0607, respectively. After correction with the PSCI method, the difference was 0.0263, 0.0111, −0.0011, and 0.0228, and the difference was reduced by 64.31%, 78.06%, 103.04%, and 62.43% after correction. The results show that the soil particle size correction coefficient can significantly reduce the interference of soil particle size to a greater extent.

3.3. Coupled Elimination Method of Soil Moisture and Particle Size Interferences

In order to achieve on-line coupled elimination of soil moisture and particle size interferences on predicting TN concentration through discrete NIR bands data, we performed detection of TN concentration through the soil sensors developed based on discrete spectral bands in the farmlands. MACI was firstly used to decrease the interference of soil moisture on discrete spectral band data. Then, PSCI was calculated to decrease the effect of soil particle size on the discrete spectral band data after corrected with the MACI method. Finally, TN concentration was obtained through the discrete spectral band data with coupled elimination of soil moisture and particle size interferences. Figure 10 shows the flowchart of the coupled elimination method.

3.4. Evaluation of the New Coupled Elimination Method

Discrete NIR spectral band data was obtained used the on-the-go detector by in-field and laboratory measurement. The spectral data after different moisture and particle size removal processes were used to establish the TN prediction model. The elimination effects of different soil particle size and moisture removal methods on the prediction of TN concentration by discrete NIR spectral band data were evaluated. Figure 11 showed the comparison of elimination effects. The data used for prediction models included (a) raw discrete NIR spectral band data, model 1; (b) discrete NIR spectral band data using the proposed MACI correlation method to eliminate soil moisture disturbance, model 2; (c) discrete NIR spectral band data using the proposed PSCI method to eliminate soil particle size disturbance, model 3; (d) discrete NIR spectral band data using the proposed new coupled elimination method to eliminate soil moisture and particle size disturbances, model 4; (e) discrete NIR spectral band data on dried and ground soil samples, model 5. SVM was used to establish a prediction model for TN concentration based on data treated by different methods. In the established SVM model, the kernel function is the radial basis kernel function, three parameters affecting model regression results: penalty parameter C, radial basis kernel function γ parameter λ, and loss parameter ε, and they are 1.3, 0.25, and 0.01, respectively.

Table 3. Comparison of elimination efforts.

Elimination Method	${\mathit{R}}_{\mathit{v}}^2$	RMSEP(g·kg−1)	RPD
Model 1	0.64	0.278	1.59
Model 2	0.71	0.221	2.06
Model 3	0.73	0.202	2.12
Model 4	0.82	0.166	2.53
Model 5	0.89	0.121	2.72

Note: $R_v^2$ = coefficient of determination of validation; RMSEP = root mean square error of validation; RPD = residual prediction deviation; model 1 = raw discrete NIR spectral band data; model 2 = discrete NIR spectral band data using the proposed MACI correlation method to eliminate soil moisture disturbance; model 3 = discrete NIR spectral band data using the proposed PSCI method to eliminate soil particle size disturbance; model 4 = discrete NIR spectral band data using the proposed new coupled elimination method to eliminate soil moisture and particle size disturbances; model 5 = discrete NIR spectral band data on dried and ground soil samples.

and Figure 10 show the calibration results. For the raw discrete NIR spectral band data, the prediction R² was 0.65. However, when the model was established by discrete NIR spectral band data after eliminating soil moisture and soil particle size interference, the RPD value of prediction model 4 reached 2.59, and the prediction R² reached 0.84, respectively. For models 2 and 3, there is only soil moisture or particle size disturbance eliminated by the MACI or PSCI method, respectively. However, both of them have an improvement of prediction R² for 12.3% and 9.2% compared with model 1. The above analysis shows that the effects of soil moisture and soil particle size on the discrete NIR spectral band were serious. Further, the TN concentration prediction model 5 was established under laboratory conditions through the discrete NIR spectral band data obtained from the soil samples after drying and grinding. The prediction accuracy of model 4 is still below that of model 5. Although, the proposed MACI and PSCI methods were quantitatively proven to have good effects on eliminating soil moisture and soil particle size interferences on predicting TN concentration through discrete NIR spectral band data. However, the 8.5% decrease of prediction was R² between models 4 and 5, which means that the effects of soil moisture and particle size were not eliminated using the new coupled elimination method. Thus, it is necessary to continue to explore other elimination methods of soil moisture and particle size effects on predicting TN concentration through discrete NIR spectral band data.

4. Discussion

4.1. Role of New Coupled Elimination Method in Predicting TN through Discrete NIR Spectral Band Data

From the perspective of cost, the development of real-time rapid soil parameter detectors based on discrete NIR spectral band has become a current research hotspot [29,30,31,32,33]. However, for discrete NIR band data, adjacent spectral data can be completely different in the attribution of the bands, and they do not have continuity in the sense of spectrum. Thus, there is no theoretical basis for direct differentiation of a set of discrete NIR spectral band data that has no connection in the spectral sense [59]. In addition, due to the lack of discrete NIR spectral data, it does not meet the requirements of the Savitzky-Golay convolution derivation method for the number of window points, and its operation cannot be performed [60,61]. Soil particle size and moisture will cause serious interference with discrete NIR band data and affect its use [36]. Therefore, the study of effective correction methods to eliminate the interference of soil particle size and moisture on discrete NIR spectral band data has important theoretical significance and practical application value for improving the performance of instruments developed based on discrete NIR spectral band data.

There is no relevant research on the elimination of soil moisture and particle size effects on predicting soil parameters through the discrete NIR spectral band data. This study analyzes the effects of soil moisture and soil particle size on the discrete NIR spectral band data using qualitative and quantitative methods. The absorbance of soil samples increased with the increase in soil moisture content at the same TN concentration level. As the soil particle size decreased, the absorbance of soil samples decreased gradually at the same TN concentration level [36]. Correspondingly, we proposed a method to eliminate the effects of soil moisture and soil particle size on discrete NIR spectral band data. A field experiment also was conducted to evaluate the elimination effects of the proposed MACI and PSCI methods. When the model was established by discrete NIR spectral band data after eliminating soil moisture and soil particle size interference, the RPD value of prediction model 4 reached 2.59, and the prediction R² reached 0.84, respectively. The results showed that the new coupled elimination method could effectively reduce the interferences of soil particle size and moisture on the prediction of TN concentration.

From the perspective of actual use, the new coupled elimination method can be applied to other on-the-go and portable soil parameter testing equipment developed based on discrete NIR spectral band data. This study facilitates the development of soil sensors based on discrete NIR spectral band data.

4.2. Comparison of the New Coupled Elimination Method to the Similar

Table 4. Validation results of soil total nitrogen (TN) concentration.

N	Spectral Measuring Equipment	Spectral Range	Elimination Factor	Elimination Method	Calibration Algorithm	${\mathit{R}}_{\mathit{v}}^2$	RMSEP	References
48	Portable detector	940, 1050, 1100, 1200, 1300, and 1550 nm	/	/	BPNN	0.45	0.215	[36]
48	Portable detector	940, 1050, 1100, 1200, 1300, 1450, and 1550 nm	Soil moisture and soil particle size	PMAI and mixed calibration set	BPNN	0.76	0.030	[36]
90	FT-NIR analyzer (MATRIX-I, Bruker corp., Germany)	800–2564 nm	Soil moisture	Wavelet decompositions	SVM	0.81	0.053	[23]
140	AgroSpec portable VIS-NIR spectrophotometer (Tec5 Technology for Spectroscopy, Germany)	305–2200 nm	Soil moisture and soil particle size	FD transformation with 31 smoothing points and SNV	Cubist method	0.73	0.071	[62]
708	Veris P4000 (Veris Technologies, Inc., Salina, KS, USA)	343–2202 nm	Soil moisture	EPO	PLS	0.63	0.024	[63]
108	On-the-go detector	1070, 1130, 1245, 1375, 1550, and 1680 nm	/	/	SVM	0.64	0.278	This study
108	On-the-go detector	1070, 1130, 1245, 1361, 1375, 1450, 1550, 1680, and 1870 nm	Soil moisture and soil particle size	New coupled elimination method	SVM	0.82	0.166	This study

Note: N = number of samples; $R_v^2$ = coefficient of determination of validation; RMSEP = root mean scheme.

shows the comparison results of the new coupled elimination method to the similar. According to the evaluation parameters ($R_v^2$ and RMSEP), our study obtains slightly high estimation accuracies compared to the similar in Table 4. Even though the new coupled elimination method was not applied to the on-the-go detector, the estimation accuracies of TN were higher than the portable detector and the Veris P4000. Applying the new coupled elimination method to the on-the-go detector, the estimation accuracy was improved significantly. The $R_v^2$ increased from 0.64 to 0.82, and the RMSEP decreased from 0.278 to 0.166. For the portable detector, the $R_v^2$ was 0.76 with the moisture absorbance correction method and mixed calibration set used in the portable detector. Compared to the similar, the new coupled elimination method achieved better elimination effects of soil particle size and moisture on NIR spectroscopy. These comparison results further verify the validity of the new coupled elimination method in eliminating the effects of soil particle size and moisture on discrete NIR spectral band data.

4.3. Uncertainty in Current Work and Future Work

Although the new coupled elimination method was quantitatively proven to have good effects on eliminating soil moisture and soil particle size interferences on predicting TN concentration using discrete NIR spectral band data, the 8.5% decrease of prediction R² was between models 4 and 5, which means that the effects of soil moisture and particle size were not eliminated using the new coupled elimination method. Thus, it is necessary to continue to explore other elimination methods of soil moisture and particle size effects on predicting TN concentration through discrete NIR spectral band data.

Additionally, the experimental conditions in fields are complex, and different regions and soil types also will affect the measurement of discrete NIR spectral band data of soil.

Moreover, field vehicle-mounted detectors are still limited in terms of coving large fields in a short time. The unmanned aerial vehicles (UAVs) have provided the potential alternative to the quick estimation of soil properties coving large fields in a short time. Thus, the hyperspectral cameras or the soil detectors based on the discrete NIR spectral band data mounted on the UAVs will be explored in future research. Therefore, further experiments are needed to verify the effectiveness and practicability of the proposed methods.

5. Conclusions

Commercialized soil nutrients detectors developed based on discrete NIR spectral band data cannot perform on-line elimination of soil moisture and particle size disturbances on predicting TN concentration. Bringing the detected data back to the laboratory for particle size and moisture interferences’ elimination is necessary. This process severely restricts the efficiency of field testing and cannot achieve a quick estimation of TN concentration in the farmlands. This paper reports on the online coupled elimination method of soil moisture and particle size interferences on predicting TN concentration through the discrete NIR spectral band data. The field test also was performed to evaluate the proposed coupled elimination method through the on-the-go detector.