From Laboratory to Proximal Sensing Spectroscopy for Soil Organic Carbon Estimation—A Review

Abstract

Rapid and cost-effective soil properties estimations are considered imperative for the monitoring and recording of agricultural soil condition for the implementation of site-specific management practices. Conventional laboratory measurements are costly and time-consuming, and, therefore, cannot be considered appropriate for large datasets. This article reviews laboratory and proximal sensing spectroscopy in the visible and near infrared (VNIR)–short wave infrared (SWIR) wavelength region for soil organic carbon and soil organic matter estimation as an alternative to analytical chemistry measurements. The aim of this work is to report the progress made in the last decade on data preprocessing, calibration approaches, and system configurations used for VNIR-SWIR spectroscopy of soil organic carbon and soil organic matter estimation. We present and compare the results of over fifty selective studies and discuss the factors that affect the accuracy of spectroscopic measurements for both laboratory and in situ applications.

1. Introduction

Food production requires fertile soils that can be deteriorated by intensive agricultural practices [1]. In order to have sustainable agricultural systems, economic viability for both farmers and society should be considered, but environmental costs also need to be taken into account, avoiding the implementation of practices that could lead to irreversible soil degradation [2]. Sustainable land use management can be achieved by maintaining and enhancing ecosystem services that include environmental, physical, and socioeconomic aspects [3,4]. The consequences of soil degradation have been recognized by the scientific community, but it was not until 1987 that the United Nations Environment Programme (UNEP) made an agreement with the International Soil Reference and Information Centre (ISRIC) for the execution of the Global Assessment of Soil Degradation (GLASOD) project that produced a world map of human-induced soil degradation [5]. Although a potentially nonrenewable natural resource, the EU Thematic Strategy for Soil Protection was issued in 2006, identifying eight main threats to soils i.e., organic matter decline, erosion, compaction, decline of biodiversity, salinization, contamination, sealing, and landslides [6]. In addition to that, the Sustainable Development Goals, adopted by the United Nations on September 2015, identified the importance of preserving soil resources from degradation in order to achieve such goals [7,8]. A key indicator of soil fertility, and thus soil quality, is soil organic matter (SOM) content [9]. Soil organic carbon (SOC) as a component of SOM affects soil fertility, but also has an impact on climate change, as soil represents one of the largest terrestrial carbon pools [10]. SOC losses could be induced from global warming that stimulates the rate of SOM decomposition, land use changes, and plant cover reduction [11,12]. To that effect, estimating SOC stocks is considered crucial for greenhouse gas (GHG) emission reports, which is not a very feasible task on large scales. Precision agriculture technologies could positively contribute to GHG emissions mitigation [13]. The various methods for SOC content determination can be categorized as analytical methods (i.e., dry and wet combustion), remote sensing based methods (i.e., space-borne and airborne), spectroscopy methods (i.e., visible near-infrared and short wave infrared, VNIR-SWIR, and mid-infrared, MIR spectroscopy), and laser-induced breakdown spectroscopy and inelastic neutron scattering methods [14]. Yet, there are still challenges that need to be addressed concerning data harmonization between different sources due to diversions in spatial and temporal data resolution, as well as a lack of the necessary information about soil sampling design and, in numerous cases, an absence of important auxiliary variable measurements for SOC stock estimation (i.e., bulk density) [15].

All analytical SOC measuring protocols have been thoroughly described in the literature [16], while remote sensing techniques for SOC estimation have been recently reviewed in detail [17]. However, there is a need to review the research work in laboratory and proximal sensing techniques in the VNIR-SWIR region, due to their significance on the development of nondestructive SOC estimation.

Therefore, this work aims to introduce the basic principles and approaches used for the analysis of spectroscopic measurements for SOC and/or SOM estimation, and not the humus fraction. Specifically, we present advances in soil reflectance spectroscopy from laboratory to proximal sensing applications, including a broad range of different approaches to inform the reader about progress made in the past decade. To that end, we cite selected papers that, to our knowledge, cover most of the methodologies used for modelling the acquired spectroscopic data, regarding multivariate statistical methods and preprocessing techniques, as well as the efforts made to address the effects of external factors on the accuracy of SOC/SOM predictions.

2. Laboratory VNIR-SWIR Spectroscopy

Over recent decades, the use of soil reflectance spectroscopy in laboratory conditions has gained much attention. Due to the principles of energy–matter interactions, a material can reflect, absorb, scatter, and emit electromagnetic radiation in a characteristic manner that depends on its molecular composition and shape, resulting in a unique spectral signature [18]. A sensor could measure the reflectance of an object at a wide area of wavelengths, providing information about its constituents.

Soil spectroscopy in the VNIR-SWIR region (400–2500 nm) has been evaluated as a possible alternative method for monitoring soil parameters, to address the need for continuous information about soil’s condition, while reducing the cost of soil analysis [19,20,21]. The nondestructive nature of this technique enables simultaneous and repeatable measurements to be made, giving it a significant advantage over conventional laboratory measurements [22]. Exploiting the direct spectral responses of SOM—mainly due to overtones, as well as the bending and stretching of NH, OH, and CH groups [23]—has become one of the most studied and accurately measured soil properties regarding soil reflectance spectroscopy applications [23,24]. In addition to this, the presence of SOM affects the color of the soil, and hence, could be directly related to the visible region of the spectrum [25]. Research was initially focused on evaluating this technique in controlled laboratory conditions, which would make it less time consuming and expensive compared to conventional soil analysis [26]. In addition to that, the use of chemical reagents is not required, which is safe for the environment and provides safer working conditions [27]. Nevertheless, soil samples that are measured in laboratory conditions need the conventional preparation of drying and sieving prior to spectroscopic measurements [28].

One of the first studies that identified the potential to measure SOM content was made by Krishnan et al. [29], who observed that the slope of the spectral curve at around 800 nm increased with increasing SOM content. Dalal and Henry, on the other hand, observed that variations in soil moisture content and soil texture were factors that affected the entire spectral signature of soil samples with the same OC content [30].

2.1. Preprocessing Techniques

Soil matrix consists of numerous parameters that are characterized by low concentrations and have overlapping absorptions that interfere with the spectroscopic measurements. For that reason, soil spectra in the VNIR-SWIR region are considered nonspecific, broad, and with low signal. To that end, spectral analysis requires the use of multivariate approaches to extract hidden information [31]. Commonly, prior to calibration, raw soil spectral data are subjected to preprocessing techniques to remove or minimize noise, enhance their signal, and select characteristic spectral features to improve the subsequent calibration models [32]. Preprocessing techniques are mathematical procedures that transform reflectance measurements and are able to remove variability from light scattering effects and enhance spectral features. These techniques can be categorized in scatter correction methods and spectral derivatives [33]; however, selecting the proper preprocessing technique is not a common procedure. Dotto et al. applied seven of the most common preprocessing techniques (i.e., Savitzky-Golay first derivative, normalization by range, standard normal variates (SNV), multiplicative scatter-correction, continuum removed reflectance (CRR), and the transformation to absorbance and application of a Savitzky-Golay first derivative with a first-order polynomial and a window size of 5 nm) and compared their performance within a wide range of multivariate calibration algorithms [34]. CRR was indicated as the most proper preprocessing technique for SOC prediction, irrespective of the multivariate method applied.

Another aspect of consideration is outlier detection and removal [35]. Though most studies remove extreme values from the modelling procedure, there is a notion that they should not be excluded from the dataset for a robust model to be able to detect such values [36]. An example is the work of De Santana et al. that aimed at providing adequate information about the methodology used for the evaluation of outliers [37], and proposed a new method for their detection using the random forest (RF) technique. It was found that RF generated a lower number of outliers from partial least square regression (PLSR).

2.2. Multivariate Calibrations

Spectral data analysis is performed with the use of multivariate statistical methods, the success of which is highly dependent on the selected calibration method [38]. PLSR is the most commonly-used linear method to describe the relationship between spectral data and soil properties due to its interpretability and low computation time [39,40]. Thus, it has been observed that the aforementioned relationship is not always linear, and that therefore PLSR could be considered insufficient for modelling soil properties [41]. Several studies have attempted to address this problem using machine learning techniques, in the hope that this would produce more accurate results than linear regression [27,28,42,43]. As Davis et al. [16] stated, the diverse and various methodologies for SOC estimation limit the researchers’ ability of the to compare the results of their studies in order to generate conclusive results. Therefore, laboratory-based studies focus on finding the most accurate and effective approach for such a calibration. Hence, differences in these studies are mainly in the protocol used and the selection of the preprocessing technique and multivariate statistical method. The accuracy of a model can be assessed using a variety of different methods and metrics. The most common are (i) the coefficient of determination (R²), which measures the percentage of variance of the dependent variable, as explained by the independent variable, (ii) the root mean square error (RMSE), (iii) the residual prediction deviation (RPD), which is the ratio of the standard deviation of the measured reference values to RMSE, and more recently, (iv) the ratio of performance to interquartile range (RPIQ).

For robust VNIR-SWIR soil spectral measurements, it is important to construct a database that is representative of soil variability for each studied area [44]. For the generation of the required data variability, Sithole et al. [45] acquired samples from various depths and different tillage systems that produced a robust PLSR predictive model for SOC with R² = 0.993, RMSEP = 0.157, and RPD = 2.55, i.e., with very high accuracy in terms of its ability to predict SOC. The modified PLSR was utilized by Heinze et al. [46], as it was considered more accurate than PLSR. Soil samples were acquired from three long-term experimental sites that resulted in high diversity in SOC content (3.4–18 mg g ⁻¹) and soil texture range. The results for SOC predictions were found to be very accurate, with R² above 0.96 for the two of the three sites, while poorer predictions were found for the site with a high sand content and smaller SOC range. Similarly, results were found by Stenberg et al. claiming that a high sand content reduces the accuracy of SOC estimation [39]. In contrast to previous studies, Bikindou et al. reported the ability of soil reflectance spectroscopy to accurately determine SOC in sandy soils with low SOM concentrations [47].

The accuracy and variability of results, depending on the multivariate statistical technique used for modelling SOC predictions, is evident in large number of studies that have evaluated and compared different algorithms. The available spectroscopic data have increased, and along with the use of machine learning techniques, created new opportunities for data analysis in the agricultural sector [48]. Spectroscopic models based on the M5 algorithm [49] were evaluated by Rossel et al. [50] using the Australian soil library. The library comprises over 10,000 samples with SOC measurements, providing an adequate sample size, which is an important parameter for the application of machine learning techniques. SOC models performed well, with an RPD of 2.17, and provided information about the important wavelengths used to partition the data and to predict the SOC content. Comparing PLSR, support vector machines regression (SVMR) and multivariate adaptive regression splines (MARS), in salt affected soils, Nawar et al. [32] reported different prediction accuracies that were dependent both on the multivariate technique and the preprocessing method. Overall, MARS, in combination with continuum removal for spectral preprocessing, showed better results (R² = 0.81), while PLSR performed better than SVMR. The study also showed a high level of correlation of SOM and raw spectra between the 550–680 nm regions, probably related to soil color. Due to the complex relationship between spectral data and soil parameters that is not always considered linear, de Santana et al. [37] compared RF to PLSR; the results were marginally better for RF, as they were able to identify outliers using a proximity matrix. In the same line, SVMR was found to perform better from linear multivariate methods (principal component analysis-PCA, PLSR) and back-propagation neural networks [51]. The selection of the most appropriate data pretreatment and calibration can be a laborious procedure due to the various combinations that can be applied. Gholizadeh et al. [52] compared some of the most common calibration techniques i.e., PLSR, RF, BRT, SVMR, and MBL with a new data-mining engine PARACUDA-II^® [53]. PARACUDA-II^® is based on the all possibilities approach (APA) and a conditional Latin hypercube sampling (cLHs) algorithm. It also has the capability of parallel programming, to assess all the possible combinations of eight different spectral preprocessing techniques against the original reflectance and the chemical data prior to model development. The performance was better compared to all other five multivariate techniques, with a R² = 0.80. The development of large soil spectral libraries has also given rise to the opportunity to explore the potential of deep learning in the analysis of such data. Padarian et al. [54] presented one of the first studies to evaluate convolutional neural networks (CNN), while introducing the concept of spectrograms to visualize soil spectra; it was found that CNN outperformed both the PLSR and Cubist models.

2.2.1. Feature Selection

Variable selection algorithms are applied to reduce the complexity of spectral data, and preselecting the useful features of the spectrum for calibration could increase the predictive ability of a model. Among the various techniques that could be used when selecting an optimum set of features, genetic algorithms have been shown to perform well for SOC prediction models [41,55,56]. Considering the complexity of genetic algorithms and the risk of overfitting when there are more than 200 variables, Peng et al. [42] evaluated a successive projection algorithm (SPA) combined with SVMR and PLSR to select the important wavelengths. This resulted in selecting 28 wavelength regions mainly found in the near infrared region, which is highly correlated with SOC content. The best predictive model was the combination of SPA-SVMR (R² = 0.73) compared to SPA-PLSR (R² = 0.62). To make a step further, Chen et al. [57] proposed a combination of RF and a back propagation network to refine the selective informative wavelengths. It has been also suggested that selecting the inputs of a least squares SVMR model based on the regression coefficients obtained by PLS analysis could provide better prediction models [58]. Recently, Raj et al. [59] utilized a series of variable indicators (Pearson’s correlation coefficient (r), biweight midcorrelation (bicor), mutual information based adjacency (AMI), variable importance in the projection (VIP)) combined with an ordered predictor selection (OPS) method to choose the optimum number of spectral variables. The best results were obtained with the use of the AMI indicator combined with SVMR (R² = 0.68) compared to the use of the full spectrum PLSR model. Bayer et al. [60] compared a physically-based model, where spectral feature analysis is coupled with multiple linear regression techniques (MLR), with a PLSR model. The results showed that model calibrations using significant spectral wavelengths could provide similar results with PLSR for SOC predictions.

2.2.2. Calibration Dataset

Apart from the proper preprocessing and calibration techniques, the stratification of local spectral libraries by homogeneity criteria of the spectral data regarding soil classes and land uses could lead to more accurate SOC predictions, as Moura-Bueno et al. [61] noted in their study. In the same context, Gupta et al. [36] applied a locally-weighted PLSR algorithm and selected an appropriate distance metric for calculating the similarity between a test sample and the calibration samples, and compared it to global calibrations with Lasso and ridge regression. Clingensmith et al. [62] evaluated four different calibration subsetting techniques coupled with PLSR, sparse partial least squares regression (SPLSR), and the heteroscedastic effects model (HEM). It was found that although the SPLSR and HEM algorithms can reduce the number of predictors, they yielded minor improvements in SOC predictions, probably due to the detritus nature of SOM.

Calibration sample size is also a factor that has been proven to affect model calibration; considering the use of soil reflectance spectroscopy as a cost-efficient method, it is important to determine the optimum number of samples for soil modelling. In general, a larger calibration sample size contributes to more robust models. Debaene et al. [63], after evaluating different sampling schemes on a field scale level, concluded that the use of K-means clustering provided better results over random sampling selection, analyte concentration, and PCA scores. However, it was observed that increasing the number of samples after a certain number did not provide proportional results in the decrease of RMSE, and thus, was not significant. In the same scope, Lucà et al. [64] found different levels of model performance variation with a selected calibration size that ranged from 14 to 144 samples and different modelling approaches, i.e., PCR, PLSR, and SVMR. Overall the SVMR model provided the best performance with 72 samples and an average RMSE of prediction of 1.10. It was also reported that creating a robust model depends on the representativeness of the data that can be selected based on the spectral characteristics and/or the physicochemical properties. Hence, the quality of the calibration dataset is affected by more factors than its size, but also by the spatial variability of the studied area.

2.3. Soil Moisture Effects

Since laboratory measurements are mainly conducted on air-dried and sieved soil samples, transferring such models to field conditions entails issues to be addressed, notably soil moisture that is known to affect the soils spectral signature [20,23,65,66]. Therefore, several studies have attempted to create prediction models for different levels of soil moisture [67,68,69]. The main absorption peaks of water content can be found near 1400, 1900, and 2200 nm due to the overtones and vibrations of the O–H group [70]. To quantify the effect of soil water content on the prediction models, Marakkala Manage et al. [71] used six different matrix potentials that could also provide information about the relationship between soil moisture, SOC, and soil texture. A typical decrease in reflectance was observed with the increase of water content [72], though the most significant change in the spectral signature was found in the transition from dry to wet soils, where water retention forces change from adsorptive to capillary forces. The results indicated that for in situ measurements of SOC content, it is best for soils to be close to dry. Nocita et al. [73] proposed the normalized soil moisture index (NMSI) [74] to classify artificially-moistened soil samples to estimate SOC content from samples with unknown soil moisture contents. It was observed that there was an overlap in the mean spectra in the regions from 400 to 1000 nm at soil moisture levels above 0.15 gW gS⁻¹, while in the region of 1100–2500 nm, there were minor but detectable differences among samples. It was also found that calibration models developed from moist samples had better accuracy for SOC predictions than models created from dry samples. Hence, in the development and exploitation of national and global spectral libraries of dry soil samples, there is a need to remove the effect of soil moisture when moist samples are to be measured.

Several methods have been proposed to remove the effect of soil moisture from soil spectra [75]. The most common are direct standardization (DS) [76] and external parameter orthogonalization (EPO) [77]. Minasny et al. [78] reported that with increasing soil moisture, the accuracy of the PLSR model for SOC was decreasing, but with the use of the EPO algorithm, the accuracy of the wettest soil sample was comparable to that of a dry soil sample. Specifically, PLSR results ranged from R² = 0.52–0.78, while EPO-PLSR ranged from R² = 0.75–0.80. They concluded that it is preferable for the calibration sample size to be greater than 100 samples, and suggested further investigation to validate this method in open-field conditions. Similar results were reported in the study of de Santana [79]. Comparing DS and EPO, Roudier et al. [80] concluded that both approaches could mitigate the effect of soil moisture when using soil spectral libraries to develop calibration models, and that both performed similarly. It was also observed that the number of calibration samples affected the performance of DS and EPO, with the latter performing better with a limited number of samples. In addition to that, different soil types affected the prediction models (i.e., soils with uniform textures generated better results compared to soils formed from complex parent materials). Soil roughness is another factor that affects spectroscopic measurements, as it causes light scattering, and thus decreases reflectance. Rodionov et al. [81] aimed to classify the effects of soil moisture and roughness and use these finding for in situ applications. To do so, they created different datasets with disturbed and undisturbed soil samples. The disturbed samples were artificially wetted at six different soil moisture contents and sieved at seven different classes (the size of their aggregations did not exceed 30 mm). The undisturbed samples were air dried for two weeks; during that time, VNIR-SWIR spectra were recorded. They concluded that for sieved samples, it was possible to predict SOC at different soil moisture levels. Regarding soil roughness, an overestimation of SOC content was observed with an increase of aggregate size.

2.4. Soil Spectral Libraries for Local Calibrations

One of the first studies that tried to unlock the potential of soil spectral libraries (SSLs) and develop accurate calibration models was conducted by Shepherd and Walsh [82]. Consequently, several attempts have been made towards the creation of regional, continental, and global SSLs [20,83,84,85,86,87]. Considering the local character of predictions in most studies, Stevens et al. [83] evaluated the potential of using the LUCAS soil database [88] to possibly cover soil heterogeneity. It was observed that the accuracy of the predictions depended on soil classes (i.e., cropland, grassland, woodland mineral, and organic soils) and the use of auxiliary predictors (i.e., sand and clay). Better prediction models were developed for croplands, grasslands, and mineral soils, while sand content most affected the error of predictions when the SOC content was low. In a similar manner, Liu et al. [89] evaluated whether the classification of the Chinese SSL, according to the spectrally- and conventionally-derived soil type, could improve SOC estimations. Overall SOC prediction was improved by soil type stratification. The model’s accuracy was slightly lower when soil types were spectrally derived, probably due to the misclassification of some soil types. From another perspective, the use of large-scale SSLs for accurate soil properties predictions on local scales could be challenging due to their diversity and heterogeneity. Lobsey et al. [90] developed a new method, named ReSampling-Local (RS-LOCAL), that selects a subset from global SSL spiked with a small representative sample from a local site. They concluded that the proposed method could not only improve results, but also reduce the number of calibration samples, thus reducing the cost. In the same line, Gogé et al. [91] aimed to identify the best strategy to utilize a national SSL. They compared the models generated from a national soil database and the same database spiked with subsets from a local library using a fast Fourier transform local weighted method, and reported moderate improvements in accuracy.

2.5. Current Trends

Though VNIR-SWIR soil measurements are considered cost effective compared to analytical chemical analyses, purchasing a spectrometer is still expensive. Currently, there have been efforts to create smaller and cheaper spectrometers. The reduced cost is due to the smaller spectral range these sensors can cover. Therefore, it is imperative to specify the important wavelengths that are correlated with the estimation of a specific soil’s properties. Evaluations of the accuracy of such devices is still at a very early stage, with Barthès et al. [92] comparing a JDSU MicroNIR 2200 spectrophotometer (Milpitas, CA, USA) equipped with an InGaAs array detector and a spectral range of 1151–2186 nm with a Foss NIRSystems 5000 (Laurel, MD, USA). The results were promising, though more research needs to be done.

2.6. Summary of VNIR-SWIR Spectroscopy Research Results on a Laboratory Scale

The work analyzed above on VNIR-SWIR spectroscopy under laboratory conditions for SOC/SOM estimations has been summarized in

Table 1. Literature review comparing different multivariate methods for SOC/SOM estimations under laboratory conditions.

	Reference	Spectral Range (nm)	Multivariate Method	R2	RMSE	RPD
1	Vohland et al. (2011) [41]	400–2500	PLSR	0.10–0.72	0.37–0.65%	1.01–1.75
			GA-PLSR	0.13–0.68	0.41–0.64%	-
			SVMR	0.73–0.81	0.38–0.86%	-
2	Minasny et al. (2011) [78]	350–2500	PLSR	0.56–0.83	0.50–1.72 (log[g 100 g−1])	-
			EPO-PLSR	0.82–0.89	0.26–0.46 (log[g 100 g−1])	-
3	Rossel et al. (2012) [50]	350–2500	M5 algorithm	-	0.26 (log10C)	2.16
4	Bikindou et al. (2012) [47]	1100–2500	PLSR	0.91	0.045%	-
5	Bayer et al. (2012) [60]	350–2500	PLSR	0.69	0.45%	1.53
			MLR	0.74	0.36%	1.93
6	Nocita et al. (2013) [73]	350–2500	PLSR	0.25–0.63	12.17–30.21 (g kg−1)	1.63–1.89
7	Stevens et al. (2013) [83]	400–2500	SVM	0.67–0.86	4.0–15 (g kg−1)	1.74–2.62
			Cubist	0.76–0.89	6.4–50.6 (g kg−1)	1.99–2.88
8	Xuemei (2013) [58]	325–1075	PLSR	0.77–0.80	3.32–3.91 (g kg−1)	-
			SVM	0.83–0.86	3.21–3.70 (g kg−1)	-
9	Ηeinze et al. 2013 [46]	400–2500	MPLS	0.41–0.99	-	1.25–8.02
10	Debaene et al. 2014 [63]	350–2220	PLSR	0.42–0.72	0.12–0.27%	1.0–2.0
11	Rodionov et al. (2014) [81]	350–2500	PLSR	0.84–0.88	0.64–0.75 (g kg−1)	2.49–2.92
12	Rienzi et al. (2014) [67]	340–2220	PLSR	0.63–0.88	4.02–7.13 (g kg−1)	-
13	Gogé et al. (2014) [91]	400–2500	PLSR, FFT-LW	0.10–0.58	3.39–7.63 (g kg−1)	0.57–1.37
14	Peng et al. (2014) [42]	350–2500	SVMR	0.72	2.83 (g kg−1)	1.86
			SPA-PLSR	0.62	3.23 (g kg−1)	1.63
			SPA-SVMR	0.73	2.78 (g kg−1)	1.89
15	Wijewardane et al. (2016) [68]	350–2500	PLSR	0.40–0.71	0.73–1.01%	-
16	Morellos et al. (2016) [43]	305–2200	PCR	0.72	0.08%	1.89
			PLSR	0.71	0.08%	1.86
			LS-SVM	0.84	0.06%	2.25
			Cubist	0.79	0.07%	2.15
17	Nawar et al. (2016) [32]	350–2500	PLSR	0.50–0.79	0.28–0.42%	1.41–2.16
			SVR	0.51–0.75	0.26–0.37%	1.43–2.0
			MARS	0.66–0.81	0.22–0.33%	1.74–2.27
18	Roudier et al. (2017) [80]	350–2500	PLSR	-	0.93–1.60%	-
19	Lobsey et al. (2017) [90]	350–2500	PLSR,cubist	0.78–0.84	0.48–1.16%	-
20	Luca et al. (2017) [64]	350–2500	PCR	0.69	0.88%	-
			PLSR	0.79	0.71%	-
			SVMR	0.82	0.68%	-
21	Jiang et al. (2017) [93]	350–2500	PLSR	0.79–0.90	0.54–0.88%	-
22	Hong et al. (2018) [69]	350–2500	PLS-SVM	0.70–0.76	-	1.87–2.06
23	Xu et al. (2018) [51]	350–2500	PCR	0.81	6.01 (g g−1)	2.31
			PLSR	0.85	5.48 (g g−1)	2.54
			BPNN	0.86	5.16 (g g−1)	2.69
			SVMR	0.88	4.85 (g g−1)	2.84
24	de Santana et al. (2018) [37]	350–2500	RF	0.8	5.46 g/dm3	-
			PLSR	0.75	6.19 g/dm3	-
25	Raj et al. (2018) [59]	350–2500	SVM	0.68	0.62%	1.79
			PLSR	0.65	0.64%	1.72
26	Gholizadeh et al. (2018) [52]	350–2500	PLSR	0.63	0.29%	-
			RF	0.65	0.23%	-
			BRT	0.68	0.25%	-
			SVMR	0.71	0.20%	-
			MBL	0.78	0.20%	-
			PARACUDA	0.80	0.12%	-
27	Gupta et al. (2018) [36]	350–2500	PLSR	0.60–0.70	0.18–0.20%	-
28	Liu et al. (2018) [89]	350–2500	PLSR	0.51–0.82	1.63–3.18 (g kg−1)	1.44–2.37
29	Sithole et al. (2018) [45]	450–2500	PLSR	0.99	0.16%	2.55
30	Ludwig et al. (2018) [55]	400–2500	GA-PLSR	-	0.04%	2.58
			SVMR	-	0.04%	2.67
			improved GA-PLSR	-	0.04%	2.89
31	Marakkala Manage et al. (2018) [71]	350–2500	PLSR	0.48–0.82	0.001–0.003 (kg kg−1)	-
32	Vibhute et al. (2018) [40]	350–2500	PLSR	0.72–0.89	3.51–5.64 (g kg−1)	-
33	de Santana et al. (2019) [79]	1150–2500	PLSR	0.86	1.85 (g/dm3)	2.59
			EPO-PLSR	0.81–0.85	2.15–2.16 (g/dm3)	2.02
34	Padarian, et al. (2019) [54]	350–2500	PlS	0.35	130.5 (g kg−1)	-
			Cubist	0.79	43.75 (g kg−1)	-
			CNN	0.88	32.14 (g kg−1)	-
			CNN multi	0.69	16.81 (g kg−1)	-
35	Moura-Bueno et al. (2019) [61]	350–2500	PLSR	0.74	0.52%	-
			MLR	0.72	0.57%	-
			SVM	0.72	0.55%	-
			RF	0.72	0.56%	-
36	Clingensmith et al. (2019) [62]	350–2500	PLSR	0.42–0.53	0.32–0.48%	1.30–1.47
			SPLSR	0.45–0.65	0.31–0.42%	1.34–1.69
			HEM	0.44–0.63	0.31–0.43%	1.34–1.64
37	Barthes et al. (2019) [92]	1151–2186	PLSR	0.64–0.82	-	1.4–2.3

, in order to provide easy access to the methodology used and the results achieved in each case.

3. Proximal Soil Sensing

Regarding the need for continuous monitoring in the agricultural domain and for site specific applications, in situ soil sensors have been developed for rapid, cost-effective, and quasi real-time data acquisition that can provide high-resolution maps [94]. Proximal soil sensing is defined as the use of field-based sensors that are in close proximity with the ground, i.e., within a maximum distance of two meters [94]. To that end, proximal sensing directly in the field became a challenge, and gained interest due to its potential advantages [95]. These technologies concern either on-the-go sensors, mounted on agricultural vehicles, or hand-held instruments that can be used for site-specific management (e.g., variable rate applications) [96]. Due to the high sampling density these sensors provide, they are considered more effective in capturing field variability, hence addressing the problem of selecting the correct soil sampling strategy that will ensure representative soil samples. The accuracy of a sensor can be measured by means of measurement repeatability at the same time and place, and the correlation between reference measurements of soil properties [97]. Although promising, in situ soil reflectance spectroscopy applications require proper environmental conditions and various pretreatment methods to mitigate the effect of moisture content, soil roughness, and vegetation cover [98].

The use of proximal soil sensing techniques could increase the number of measured soil samples that are needed for an adequate characterization of soil heterogeneity on a field scale. To that end, researchers have focused on developing or updating already-developed sensors [96,99]. Achieving successful measurements would be of great benefit for agriculture, as they can assist on delineating management zones and control the inputs applied in the environment using precision agriculture techniques [100]. In this section, we present several prototypes that have been developed in research studies, as well as evaluations of commercially-used VNIR-SWIR sensors, and discuss the limitations and hindrances of their use.

3.1. Commercial Available In situ Soil Sensors

Bricklemyer and Brown [101] tested a commercially-available, on-the-go soil sensor (Veris Technologies Inc., Salina, KS, USA) in eight fields. Two approaches for model calibration were evaluated: (i) using measurements of the seven out of eight fields to independently predict SOC from the remaining field and continuing this procedure until each of the eight fields was predicted, (ii) adding nine randomly-selected samples from the remaining field to the calibration dataset. The accuracy was poor for both approaches with an RPD = 0.1, probably due to low SOC variability. Due to spectral correlation between soil properties, combining different sensors could potentially improve the accuracy of in situ measurements. To that end, Kweon et al. [102] utilized a dual-wavelength (660 and 940 nm), on-the-go soil optical sensor coupled with electrical conductivity measurements. The relationship between these two wavelengths and SOM at levels below 5% was considered linear, and therefore, MLR was used for model calibration while slope, curvature, and elevation were also used as independent variables. The various combinations of the predictor values provided different results, with R² ranging from 0.55 to 0.94. These variations were attributed to the soil moisture content and temperature. In the same line, Knadel et al. [103] compared the predictive ability between a single sensor and a fusion of sensors (VNIR-SWIR, EC and temperature), and found that SOC estimation could be improved by the use of sensor fusion. Wetterind et al. [104] tested the P4000 drill rig-mounted spectrometer (Veris^® Technologies, Salina, KS, USA) and also reported that predictions for SOM were marginally improved with a sensor combination. Similar results were reported by Pei et al. [105]. Utilizing the same sensor, Veum et al. [106] aimed to address the effect of soil moisture by applying the EPO transformation in conjunction with the Bayesian Lasso, along with additional covariate information. The accuracy for models trained on dry spectra and tested on field moist spectra was improved. Kuang et al. [107] compared the accuracy between PLSR and artificial neural networks (ANN) for online spectral measurements, and concluded that ANN provided better results in cross validation (R² = 0.83) compared to PLSR (R² = 0.71). Though using an independent dataset for calibration, the model’s performance dropped significantly for both ANN and PLSR calibrations, with R² = 0.49 and R² = 0.46, respectively. Sorenson et al. [108] performed in situ spectral measurements with the same sensor as Wetterind et al. [104]. For model calibration, they evaluated different multivariate techniques, i.e., MARS, ANN, SVM, PLSR, RF, and Cubist. The lowest RMSE values were found with the use of Cubist and the highest with ANN. They also reported that a model’s accuracy increases when it is developed from a homogeneous dataset, though this comes at the expense of model transferability.

3.2. Experimental Prototypes of In situ Soil Sensors

Apart from commercially-available equipment, researchers aimed to develop prototypes for in situ applications. To this end, Kodaira and Shibusawa [109] upgraded a prototype real-time soil sensor (RTSS) (SAS 1000, SHIBUYA MACHINERY Co., Ltd., Japan). The system was mounted on a tractor and comprised a sensor unit, a touch panel, a soil penetrator, and a set of probes. The sensor unit had two spectrophotometers, with spectral range from 350–1100 nm and 950–1700 nm respectively, and a differential global positioning system. The sensor was designed to acquire spectra at adjusted depths every 4 s which, based on the speed of the tractor (0.56 m s⁻¹), resulted in 144 measurements from 31.48 ha; this sampling density was considered adequate for high resolution mapping. For model calibration, the PLSR method with full cross calibration resulted in an accuracy of R² = 0.90 for SOM. Field conditions are highly variable and should be taken into consideration during measurements therefore, Kuang and Mouazen [110] utilized an online sensor developed by Mouazen and Ramon [111] that penetrated soil to a depth of 15 cm. Comparing the PLSR models generated for processed soils, fresh soils, and online measurements, it was observed that MC and increased clay content decreased the model’s accuracy, while models created from dry samples with high clay content increased the accuracy. To address disturbing factors of in-field spectral measurements (i.e., illumination conditions), Rodionov et al. [112] developed a closed chamber with a commercial VNIR-SWIR spectrometer mounted on a tractor. The system was tested in a small-scale, bare soil field and at different spectral acquisition modes, i.e., stop-and-go and continuous. The modelling approach also accounted for soil moisture and roughness according to Rodionov et al. [81]. Regarding the continuous mode, spectral discontinuities were observed, and thus, the stop-and-go mode was considered more appropriate, providing an R² = 0.65. Although the results were significantly lower from laboratory measurements (R² = 0.94), they provided the potential for SOC pattern recognition. On-the-go spectral measurements are also affected by sensor movement and heterogeneous soil; hence, Franceschini et al. [113] aimed to mitigate these effects by comparing the results of utilizing the EPO, DS, and orthogonal signal correction (OSC) [114]. The in-situ measurements were performed in relatively dry soil conditions to minimize the effect of soil moisture. It was observed that the albedo of field measurements was generally lower than that of laboratory measurements, probably due to differences in the sensor configuration and illumination conditions during acquisition. Although an improvement in the model’s accuracy was observed with the use of OSC, the results were substantially inferior compared to those of laboratory-generated models, indicating that factors like soil-to-sensor distance and angle, gravels or straws, and changes in the illumination conditions should also be considered.

While proximal sensors have been largely developed and used to overcome the constraints of conventional laboratory chemical analyses, in some cases, the use of a single sensor is inadequate for the simultaneous estimation of several soil properties [115]. To address this, Rossel et al. [116] developed an integrated Soil Condition Analysis System (SCANS). SCANS is a fusion of proximal sensing technologies, analytic statistics, and smart engineering that can analyze a 1.20 m soil core after extracting it from the field. A VNIR-SWIR spectrometer, an active γ-ray attenuation sensor for bulk density estimation, a visible camera, and a Lepton long wave infrared camera mounted on the platform made it possible to estimate approximately thirteen physical, biochemical, and mineralogical soil properties. A total of 150 soil cores were derived and measured from a 600-ha cattle grazing farm. Model calibration was performed with the Cubist algorithm combined with the RS-LOCAL algorithm, providing a more efficient exploitation of the local spectral libraries. Τhe EPO algorithm was also used to eliminate the effect of water in the soil spectra in order to achieve the transferability of the calibrated models with models from spectral libraries of dry soil. The results were promising for SOC estimation with a good level of accuracy (R² = 0.83) for a repeated 10-fold cross validation and R² = 0.81 for independent validation. The advantage of SCANS was that it provided accurate, fast, and cost-efficient predictions at high spatial resolution and from various depths.

3.3. Handheld Proximal Sensors

Apart from spectroradiometers mounted on vehicles, portable instruments equipped with a handheld contact probe are also used for direct in situ measurements. Cozzolino et al. [117] used the ASD FieldSpec III spectroradiometer with spectral analysis from 350–2500 nm (Analytical Spectral Devices, ASD, Boulder, CO, USA) and utilized the range 350–1850 nm. The spectral measurements were performed in 68 soil samples whose surface was flattened. The PLSR model calibration using only the NIR (950–1800 nm) wavelength region reported an RPD = 1.8 for SOC that suggests moderate accuracy. Gras et al. [118] evaluated seven different practices for acquiring spectra in the field using an ASDLabSpec 5000 spectrometer (Analytical Spectral Devices, Boulder, CO, USA). These procedures were applied either directly to the soil surface (in cores extracted with an auger) or to clods crumbled from the cores. No external validation was performed, as the aim was to select the most efficient approach for spectral acquisition. Different procedures gave different RPD values, and specifically for SOM estimations, RPD ranged from 2.1 for clods crumbled from cores after half a day, to 2.8 in raw core measurements. The same spectrophotometer was also used by Cambou et al. [119]; the spectral measurements were made on the outer core of samples collected with a manual auger. PLSR and seven different preprocessing techniques were used for model calibration and resulted in R² = 0.75 for SOC estimations and SOC. The poor results obtained could possibly be related to the fact that spectral measurements were not performed on the same samples that were conventionally analyzed.

3.4. Photosynthetic and Nonphotosynthetic Vegetation Affecting In situ Measurements

Although bare soil conditions are the most favorable for in situ measurements, in real field conditions, the presence of either green vegetation or straw cover is very common, and may lead to overestimations of SOC [120]. Rodionov et al. [121] aimed, in their study using the same spectra, to distinguish SOC from photosynthetic and nonphotosynthetic vegetation. They conducted an experiment under laboratory conditions to estimate the effect of vegetation fractional cover. Soil samples were placed in petri dishes with an increasing degree of plants and straws (24 soil samples with 13 coverage degrees). To characterize straw cover, they used the Cellulose Absorbance Index (CAI), and for green vegetation various, known indices were estimated, such as the Normalized Difference Vegetation Index (NDVI). The aforementioned indices were subtracted from field measurements to prevent SOC overestimation. Increasing vegetation coverage resulted in higher reflectance in the whole spectral range and distinctive absorption peeks at 600–700 nm. Because the spectral response of straw was not very distinctive, both the use of CAI and NDVI was recommended. Specifically, CAI values ≥ 0.70 and NDVI values ≥ 0.80 were indicative of a SOC overestimation. Although this study confirmed the overestimation of SOC content with the presence of green vegetation and straw coverage, the model gave average predictions with R² = 0.58–0.66. The authors suggested the importance of estimating the vegetation coverage degree prior to SOC estimations.

3.5. Spiking Techniques

To address the lack of adequate local data for SOC and to better exploit existing SSls, the spiking technique is frequently evaluated. It is mostly used to augment an existing SSL with local spectra to improve the models accuracy [122,123,124]. Nawar et al. [125] showed that by using spiking for model calibration, the sample selection method and the number of selected samples also affects the model’s accuracy. Specifically, a similarity analysis gave better predictive models with a minimum number of spiked samples compared to Kennard stone and random sampling. Guerrero et al. [126] suggested that an extra-weighted spiking subset could increase the prediction accuracy over spiked calibrations. Collecting field data and creating new SSLs for unprocessed field samples is a laborious procedure. To that end, Kühnel and Bogner [127] proposed the use of the synthetic minority oversampling technique (SMOTE) [128] to address the problem of limited field data. The technique resides in the principal of giving extra weight to existing soil samples to generate new synthetic spectra, hence augmenting existing datasets. This method aims to compensate for the loss of significant spectral data when using an EPO or DS algorithms to remove the effect of MC or for the time-consuming measurement of samples at different MC levels. It was observed that synthetic spectra gave more similar spectral responses with in situ spectra rather than spectra from air-dried, sieved samples. The augmentation of the spectral libraries with the synthetic spectra provided lower RMSE and higher RPD compared to spiking. However, the importance of selecting a representative site samples is significant.

3.6. Summary of VNIR-SWIR Soil Proximal Sensing Research Results in Field Scale

The work analyzed above on VNIR-SWIR soil proximal sensing under laboratory conditions for SOC/SOM estimations has been summarized in

Table 2. Literature review comparing different multivariate methods for SOC/SOM estimations under field conditions.

	Reference	Spectral Range (nm)	Multivariate Method	R2	RMSE	RPD
1	Bricklemyer and Brown (2010) [101]	350–2500	PLSR	0.00–0.42	-	1.0–1.3
2	Cozzolino et al. (2013) [117]	350–1850	PLSR	0.81	-	1.8
3	Kodaira and Shibusawa (2013) [109]	400–1700	PLSR	0.9	0.35%	2.9
4	Kweon et al. (2013) [102]	660 and 940	MLR	0.55–0.94	0.11–0.77%	1.50–4.27
5	Kuang and Mouazen, (2013) [110]	305–2200	PLSR	-	1.29–1.90 (g kg−1)	2.01–2.24
6	Gras et al. (2014) [118]	350–2500	MPLS	0.77–0.86	-	2.1–2.8
7	Knadel et al. (2015) [103]	350–2200	PLSR	0.57–0.94	-	1.4–3.9
8	Rodionov et al. (2015) [112]	350–2500	PLSR	0.65	-	-
9	Wetterlind et al. (2015) [104]
10	Ji et al. (2015) [75]	350–2500	PLSR	0.63–0.70	0.21–0.27 (g kg−1)	1.44–1.79
11	Kuang et al. (2015) [107]	305–2200	PLSR	0.37–0.81	1.46–3.88%	1.15–2.29
			ANN	0.39–0.90	1.22–3.66%	1.22–3.01
12	Rodionov et al. (2016) [121]	350–2500	PLSR	0.84	0.73%	2.53
13	Cambou et al. (2016) [119]	350–2500	PLSR	0.75	-	2
14	Viscarra Rossel et al. (2017) [116]	350–2500	CUBIST	0.81	0.0041%	-
15	Kühnel and Bogner (2017) [127]	350–2500	smote/PLSR	0.40–0.86	1.90–16.63 (mg g−1)	-
16	Sorenson et al. (2017) [108]	350–2200	MARS	0.76	0.66%	2
			ANN	0.01	1.56%	0.9
			SVMR	0.75	0.67%	2
			PLSR	0.54	0.90%	1.5
			RF	0.78	0.62%	2.1
			Cubist	0.8	0.60%	2.2
17	Veum et al. (2018) [106]	350–2200	PLSR	0.23–0.82	0.19–0.96 g 100 g−1	-
18	Nawar et al. (2018) [125]	305–2200	PLSR	0.74–0.78	0.16–0.18%	1.97–2.14
19	Nawar et al. (2019) [124]	305–2200	RF	0.12–0.75	0.17–0.33%	1.08–2.04
20	Pei et al. (2019) [105]	343–2222	PLSR	0.8	-	-
			NN	0.86	-	-
			RT	0.69	-	-
			RF	0.58	-	-

, in order to provide easy access to the methodology used and the results achieved in each case.

4. Discussion

Soil properties’ monitoring and estimation are essential processes for site-specific management practices, though there is a lack of up-to-date and consistent information on their spatial distribution [129]. Considering the laborious and costly procedure of collecting a sufficient number of samples to develop high-resolution maps, more efficient methods have been evaluated as alternatives to support or, to some extent, replace time-consuming chemical soil analysis. The interaction of soil properties with electromagnetic radiation in the VNIR-SWIR spectral region (400–2500 nm) has provided the grounds by which to assess this method for soil properties estimations. The first attempts began in controlled laboratory conditions, and subsequently, in-field applications appeared. SOC, as one of the most important soil quality parameters, has gained much attention due to its characteristic interactions with electromagnetic radiation.

The procedure for spectroscopic measurement, modelling, and prediction of SOC requires well-defined steps, though the procedures in each step vary among studies. Firstly, soil samples from the studied area are analyzed with conventional laboratory measurements. Even these standardized procedures could differ in each study [32,119], as it was observed that laboratory measurements could also entail errors [130] that affect the predictive ability of the models, since they are dependent upon these specific laboratory reference measurements [131]. Laboratory soil spectroscopy has been evaluated for more than twenty years, and despite the development of soil spectral libraries on a global scale, there is still not an agreed upon protocol for performing such measurements. That raises issues when it comes to sharing and comparing soil spectral data [132]. Although laboratory measurements are held in controlled conditions, each laboratory has a different measurement protocol (e.g., sample preparation, instrument configuration, and different instrument). Soil spectra are affected by systematic (i.e., different instruments) and nonsystematic effects (i.e., random noise and instabilities). To overcome such hindrances, Ben Dor et al. [132] proposed the internal soil standard and a simple protocol to measure soil properties under laboratory conditions; thanks to these efforts, soil spectral libraries from different sources could be merged. Prior to the multivariate analysis, preprocessing techniques are utilized to remove physical phenomena from spectra, enhance spectral features, and improve the model’s performance [33]. The most efficient and commonly-used preprocessing techniques seem to be the first derivative [133], standard normal variate [134], and continuum removal [34].

Several studies have demonstrated that important wavelength selection could improve the accuracy of a model by removing unnecessary information and improving a model’s interpretability [41,42,55,56,135]. Another factor that affects the accuracy of a model is the calibration size. At present, it is gaining much attention, because it could also prevent soil sampling from being a waste of resources [83]. However, finding the optimum calibration size is not an easy task [64]. It has been reported that with an increasing calibration size, the model’s accuracy is increased up to a certain point were no significant improvements can be further observed. Moreover, deciding the calibration sampling strategy could define the representativeness of the calibration dataset. To that end, Kennard–Stone sampling, Fuzzy c-means sampling, and cLHS could provide better results than random sampling [63].

The use of local models for estimating soil properties of a different region could be proven insufficient due to underrepresented ranges of soil property values and differences in soil types in a given area [136]. To cover global soil diversity and variation, many countries have started initiatives to create national, regional, and global SSLs (Australia, European Union, Brazil, and China) [20,83,84,85,86,87], while there are also projects providing publicly-available data, containing metadata regarding soils’ key properties and their spectral signature [88,137]. Building these large databases is more cost effective than using conventional analytical methods for the determination of soil properties. The increased size of the data generated by the creation of SSLs, along with improvements in computational resources, enabled the use of machine learning techniques for model calibration [138]. Machine learning and data mining techniques could generate information by exploring the relationships between existing data, and has the ability to learn without being strictly programmed. Using global or national models for local calibrations could also prove to be problematic because it usually captures general trends with a greater level of generalization. To address this shortcoming, several methods are now being successfully evaluated to increase the accuracy of such predictions, like spiking [93,126] and RS-LOCAL [90]. These techniques can augment an SSL with site-specific samples minimizing the required reference measurements while capturing the respective variability.

Although acquiring the spectroscopic data is a feasible and rapid procedure, information is still hidden in the data. The potential of VNIR-SWIR spectroscopy to predict SOC is highly dependent on the selection of the multivariate calibration technique. Apart from the most commonly-used PLSR method, large SSLs provide the opportunity for machine learning techniques (ANN, RF, MARS, SVM, Cubist) to be increasingly evaluated, and in most cases they outperformed PLSR. Specifically, PLSR and other linear approaches may fail to successfully represent the complex relationship between the measured reflectance spectra and soil attributes, while they are also prone to model noise. Compared to PLSR, SVM has demonstrated better performance, probably thanks to its tolerance in the presence of outliers and spectral noise [22]. SVM use kernel functions to project the data onto a new hyperspace where complex nonlinear patterns can be simply represented; therefore, it is capable of learning in a high-dimensional feature space with fewer training data [58]. MARS is a nonparametric method that is also used to estimate complex nonlinear relationships, and in most cases, has outperformed both PLSR and SVM. MARS is a generalization of recursive partitioning regression approaches which generates piece-wise functions that are aggregated in terms of an additive model; therefore, the accuracy of the predictions is higher when the underlying function is continuous [27]. Random forests is also a nonparametric algorithm that is composed of several regression or classification trees that are a combination of bootstrap aggregation and random feature selection. By that means, it is able to rank controlling factors of SOM and provide the relative importance of spectral variables [139]. The Cubist is a rule-based model that is able to generate more comprehensive predictive accuracy than conventional statistical methods with minimized risk of over fitting. It can include various predictor variables (categorical or continuous), and the importance of a variable can be automatically obtained to interpret the variable contribution mechanism in the final predictor model [140]. Deep learning algorithms such as ANN are based on the ability to learn during the training procedure in which they are presented with inputs and a set of expected outputs. Although they have shown good accuracy, their use is limited due to factors such as the selection of the hidden layer size, the requirement of a large number of training data, and the tendency to overfit the data [141]. In a similar manner, CNN is a neural network that includes one or more convolutional layers in its architecture, and has the ability to exploit local correlations to make them more attractive than the fully-connected neural networks [54]. Despite the superiority shown in these techniques, there is a lack of consistency in the results; hence, PLSR is still utilized, as it provides sufficient accuracy [45]. Variations in the study areas and in the selection of calibration data and their size, numerous combinations of preprocessing, and multivariate methods are some of the reasons that no calibration method has achieved universal acceptance.

When it comes to the validation procedure, cross validation [117,123] is the most commonly-used method due to the small sample size of the dataset, though one should keep in mind that there is the risk of overfitting the model; therefore, testing its performance on independent datasets usually provides less accuracy [46].

Considering in situ soil measurements, they also share the same issues with laboratory soil spectroscopy when it comes to model calibration, but with the additional hindrance of external factors affecting the measurements. The most frequently-addressed parameter is soil moisture. Soil moisture has a distinct impact on soil reflectance by decreasing the albedo [113]. To that end, methods to eliminate this effect such as EPO and DS have been proposed with relatively good results [51,142]. Attempts to create samples with variable soil moisture content have also been made, though they are somewhat time consuming [73,80,143]. Additional factors that could interfere with in situ measurements are ambient conditions such as temperature, wind, and precipitation [144,145], soil roughness that could be altered by field operations, stones, plant residues and movement during spectral acquisition, i.e., vibrations [112,113,121]. Moreover, it should be noted that in-field measurements are usually performed in the top layer of the soil, with an exception being studies where the sensor penetrates the soil, which may impact the model’s quality because laboratory reference measurements concern samples taken from a layer. Different systems with various configurations regarding the mode of spectral acquisition, the sensor distance from the ground, and illumination conditions vary among studies, increasing the obstacle of comparing the results from one study to another. Regarding the experimental stage of most studies, it is difficult to create a common protocol for in situ, on-the-go measurements.

Differences in evaluations of results are also observed among studies. The most frequent statistical indicators are RMSE, RPD, and R². It should be noted that the error should be referred to using the same units as those of the analyzed soil property [146]; most studies do not use the same evaluation methods, as shown in Table 1 and Table 2. In addition to that, the normalization of the data due to skewness makes it even harder to compare studies.

5. Conclusions

The objective of this review was to summarize the progress made during the last decade using laboratory and proximal soil sensing in the VNIR-SWIR region for SOC estimations. Laboratory measurements are considered to be a well-established method for soil properties estimations, but there is not yet a commonly accepted universal model with broad application. The attention that soil spectroscopy has gained is evident on the efforts of creating SSLs worldwide, albeit, with a nonagreed protocol. Research has also been made towards the use of proximal soil sensors for in situ applications that could estimate SOC in real time and on larger spatial scales. Overall, the results are promising for SOC estimations, and more research needs to be done in terms of selecting the proper spectral range of the sensor, the preprocessing methods, and the calibration techniques. Detrimental effects such as soil moisture, soil roughness, vegetation cover, and others that affect SOC spectral response need to be addressed for model transferability from laboratory to in field applications. Due to the various inconsistencies among studies, it is suggested that articles should include more information about the experimental design, the criteria used for the selection of the chemometric approach, and the pre- and post- processing procedures to facilitate comparisons of results among studies. The advent of much smaller and more affordable spectrometers could potentially provide even more rapid soil properties estimations to assist farmers in selecting the most appropriate management practices with respect to soil preservation.