Validation of NASA SMAP Satellite Soil Moisture Products over the Desert of Kuwait

Abstract

The goal of this study is to validate and analyze NASA’s Soil Moisture Active Passive (SMAP) products over the desert of Kuwait. The study period was between April 2015 and April 2020. The study domain includes a mission candidate calibration/validation (Cal/Val) site that comprises six permanent soil moisture stations used to verify SMAP estimates. In addition, intensive field campaigns were conducted within and around the candidate Cal/Val site during the study period to collect additional thermogravimetric samples. The mean difference (MD), root mean squared difference (RMSD), unbiased root mean square difference (ubRMSD), and correlation coefficient (R) were computed to assess the agreement between SMAP SM products and in situ observations. The comparison of the six ground station sensors’ observations with the thermogravimetric samples led to an absolute mean bias (AMB) of 0.034 m³ m⁻³, which was then used to calibrate the sensors and bias-correct their measurements. The temporal consistency of the readings from the test site and calibrated sensors was assessed using the mean relative difference (MRD) and its standard deviation of relative difference (SDRD). Using a sampling density analysis, it was determined that a minimum of four ground stations would be required to validate the test site. Furthermore, the consistency between SMAP satellite soil moisture data and those derived from the Soil Moisture and Ocean Salinity (SMOS) satellite operated by the European Space Agency, and their agreement with in situ samples, was analyzed. The comparison of SMAP and SMOS soil moisture data with in situ observations showed that both satellites successfully captured the spatial and temporal distribution of soil moisture. For SMAP and SMOS, the lowest ubRMSE statistics were 0.043 m³ m⁻³ and 0.045 m³ m⁻³, respectively, which are slightly higher than the mission target of 0.04 m³ m⁻³.

1. Introduction

Soil moisture (SM) influences plant growth and vegetation restoration in arid and semi-arid regions [1]. It is also an important indicator of soil characteristics [2,3]. Several factors contribute to soil moisture variation, including precipitation, plant transpiration, soil evaporation, surface runoff, and underground percolation. Soil moisture in the unsaturated zone changes as a result of precipitation recharge and water exchange with the atmosphere and groundwater. Water exchange between the unsaturated zone and the atmosphere has been studied primarily in relation to soil moisture variations and the ways in which they affect the atmospheric boundary layer processes affecting weather and climate [4,5]. Soil moisture is a heterogeneous variable that exhibits small-scale differences depending on soil drainage patterns and characteristics [6,7,8,9]. Information regarding soil moisture conditions can be obtained via in situ measurements, remote sensing observations, and physical-based models. The most reliable source to understand SM variability and distribution is in situ measurements. The ground-based measurements, however, are limited in terms of number and spatial coverage, particularly in arid regions. They can be collected from permanent stations or intensive field campaigns that involve gravimetric sampling. Integrating SM data from various sources can be used to create a robust dataset that can be used for calibrating and validating satellite SM estimates [10,11,12,13]. Satellite measurements, despite the complexity of their retrievals, can cover large areas, steep terrain, and dense vegetation [14].

Routine observation of soil moisture is made possible by microwave remote sensing. The use of remote measurements from space allows for frequent, global observations of soil moisture over vast areas. Since microwave measurements are not affected by cloud cover and variable surface solar radiation, various microwave remote sensors have been widely used in the spatial and temporal distribution of global-scale mapping of SM [13,15,16]. These sensors include the Scanning Multi-channel Microwave Radiometer (SMMR), the Special Sensor Microwave Imager (SSM/I), the Advanced Microwave Scanning Radiometer (AMSRE and AMSR2), Soil Moisture and Ocean Salinity (SMOS), and Soil Moisture Active Passive (SMAP). The SM data products bestowed by each satellite depend on its specific operating frequency and overpass time. Hence, surface characteristics such as surface roughness, vegetation cover, dielectric constant, and topography influence microwave measurements in distinct ways [17,18,19,20]. Microwave remote sensing uses the L, C, and X bands to obtain SM, which all have limitations as to how well they penetrate the soil depth and reflect surface roughness in order to obtain accurate SM values. Hence, L-band sensors (SMAP and SMOS) can penetrate deeper compared to C- and X-band sensors (AMSRE and AMSR2) [21,22,23]. Microwave brightness temperature (BT) measurements are primarily used to estimate SM since physical conditions such as surface temperature and emissivity have a significant impact on BT measurement. The frequent overpasses of satellites can provide different SM measurements [24,25]. The correct interpretation of remotely sensed SM data requires the determination and understanding of the sources of their bias with respect to in situ observations, especially in information-poor regions such as the desert of Kuwait.

In this study, SM products from satellite observations in Kuwait are examined. In Kuwait, remote sensing of soil moisture and field measurements has been carried out since the year 2000, as part of various research projects funded by Kuwait University and the Kuwait Foundation for the Advancement of Sciences (KFAS). During the SRTM mission in February 2000, approximately 100 soil samples were collected [26]. To determine soil moisture from microwave satellite brightness temperatures, a statistical inversion model was developed using two frequencies, namely 6.6 GHz and 37 GHz [26,27]. Such models are suitable for desert areas such as Kuwait, where vegetation is scarce, and the surface is moderately rough. Field experiments were also performed to support the Advanced Microwave Scanning Radiometer—Earth Observing System (AMSR-E) [28]. Field-estimated soil moisture values up to 5 cm deep were compared with AMSR-E soil moisture values. The Kuwait Institute of Scientific Research (KISR) desert, a protected site in Sulaibiya, was monitored for soil moisture measurements between April 2011 and September 2013 [28].

The Kuwait site was selected as an international candidate site for Cal/Val by NASA’s SMAP mission [29]. It is worth noting that the Kuwait site is the only Cal/Val site in a hyper-arid environment in the northern hemisphere. The purpose of this study is to validate SMAP SM observations against in situ soil moisture measurements collected in the desert of Kuwait. The focus in this paper is on the verification of SMAP SM products; SMOS was the best choice for this as it operates in the same L-band microwave frequency, and it was introduced to analyze the consistency with SMAP and compare the retrieval performance. Hence, SMAP SM data products (SMAP_L3_SMP, SMAP_L3_SM_P_E, and SMAP_L2_SM_SP) and SMOS SM data products (SMOS MIR_SMUDP2 and SMOS L4 CATDS SM) were introduced to analyze the consistency with SMAP and compare the retrieval performance. There have been several studies investigating the accuracy of SMAP volumetric soil moisture (VSM) estimations [30,31,32,33,34,35,36,37], as well as work on SMOS validation [38,39,40,41,42,43]. The validation process of SMAP SM products is conducted over the 36 km, 9 km, and 3 km SM products. To our knowledge, this is the first time that SMAP has been validated over the land conditions of the Middle East. A combination of ground-based data and thermogravimetric measurements, as well as spaceborne retrievals, was used to analyze the temporal stability of and spatial variability within the experimental site [44].

2. Methodology

2.1. Study Area

Kuwait is located on the northeastern corner of the Arabian Peninsula, between approximately 28°30′N to 30°05′N latitude and 46°39′E to 48°35′E longitude. The land surface slopes gradually northeastward, with an average gradient of 2 m km⁻¹. Kuwait’s surface area is approximately 17,800 km², covered by recent sediment deposits [45]. Eolian deposits account for 50% of the surface deposits. Other types found are playa, residuals, desert plain, and coastal deposits. The climate is arid, with hot to very hot dry summers and cool to mild rainy wets. The maximum temperature can be as high as 50 °C, while a minimum of −4 °C was recorded at Kuwait International Airport on 20 January 1964. Mean annual temperatures are 37 °C for July and 14 °C for January. The mean annual rainfall is 115 mm, with variability from year to year (28–260 mm) and from place to place. The evaporation rate ranges from 4.6 mm per day in January to 22.9 mm per day in June. The Kuwait SMAP candidate site for Cal/Val (36 km × 36 km) is located on the western side of Kuwait City. It is a homogenous desert site with very sparse vegetation that possesses six permanent stations (Figure 1) (details in Section 2.2.2), which are located in the largest representative area of soil classes. In our previous study of the sources of soil moisture variations in the Kuwait desert [44], it was found that there is no clear correlation between soil moisture and elevation for most of the locations in the test site. This could be attributed to the specific drainage pattern in the desert environment. Moreover, it was found that there is a spatial and temporal stability of soil moisture for all different soil classes in the test site [44,45,46]. In general, the main source of soil moisture variations in the desert of Kuwait is the seasonal precipitation. The Kuwait desert test site is considered unique in the whole Middle East region and considered very important for the validation and calibration of SMAP soil moisture data [29] by NASA as a desert target site due to its exceptional homogeneity as well as its surface and climate conditions, as has been discussed in [30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46].

2.2. Datasets

The datasets used in this study were collected from (i) thermogravimetric soil sampling, (ii) permanent station sensors, and (iii) remotely sensed data from SMAP and SMOS satellites across the test site.

2.2.1. Thermogravimetric Dataset

During the pre-launch phase of the NASA SMAP satellite launch, Kuwait’s SMAP team began collecting thermogravimetric data in order to establish a calibration and validation test site for SMAP over the Kuwait desert. From January 2013 onwards, surveys were conducted over the five SMAP grids (Figure 1a), such as A, B, C, D, and G. Grids A, E, and F were discarded due to their proximity to the Kuwait border and military restrictions. Grid D was excluded from the validation analysis due to its proximity to a water body. Grid G contains some agricultural land, and Grid C contains oil fields. These fields were also excluded from the analysis. Consequently, Grid B (36 km × 36 km) was selected as a validation test site as it is a unique site that is exempt from all constraints mentioned above and is referred to hereafter as the test site (Figure 1c). Over the course of three years (December 2014–April 2018), eighty-three field visits were conducted, and around 2500 thermogravimetric soil samples were collected from the top five centimeters of soil. The soil samples were collected with a metallic cylinder having a radius of 4.3 cm and a height of 5 cm (Figure 1b). The sensors used in the SMAP and SMOS satellites for SM data product retrievals are operated at L-band frequency, which has approximately 5 cm penetration depth for medium soil moisture levels [47]. Moreover, the SMAP products provide an estimate of surface soil moisture in the top 5 cm on average. The variable sensing depth uncertainties are embedded within the product uncertainty [48]. At the site, samples were weighed immediately and dried in an oven at 105 °C for 48 h, and then the dry weights were determined. The volumetric water content of each soil sample was determined by combining the gravimetric water content with the bulk density of each sample. The thermogravimetric datasets were used to calibrate the station sensor readings and establish the temporal stability of the test site. In addition to regular field visits around the test site, intensive field campaigns were conducted over the 36 km² test site [44] every 3 km. These soil moisture data were used to define the sampling density and upscale the test site.

2.2.2. Permanent Ground Station Dataset

The Kuwait SMAP test site has six permanent ground stations installed within the 36 km × 36 km test site, equipped with (a) Campbell sensors (accuracy ±2.5%), (b) Stevens Model Hydra Probe sensors (accuracy ±1%), and (c) ML3 soil moisture sensors manufactured by Delta-T (accuracy ±2.5%) (Figure 1b,c) based on the general soil profiles. The six stations collect soil moisture as well as other meteorological data, including rainfall, soil temperature, wind speed, wind direction, relative humidity, air temperature, etc. All six permanent stations have ground soil sensors buried at a depth of 5 cm, 10 cm, and 25 cm. The Campbell station has, in addition to the previous depths, soil sensors buried at 50 cm and 1 m depths (Figure 1b). However, in the study domain, the accuracy of these ground sensors may vary due to the specific composition [42] of desert soil types, which is different from those used in the default calibration. The ground stations were installed within three different soil classes, such as Gp03, Gp16, and Gp19 (Figure 1b,c) [40], of various accuracies. Nevertheless, the sensors were calibrated using a standard calibration method (details given in Section 3.2). Satellite soil moisture estimates using microwave observation in the L band are sensitive to the top 5 cm, the near-surface soil column, which is consistent with the sensing depth of the deployed sensors.

Kuwait University established remote access to these stations for the transfer of data in near-real time, varying from every four minutes to every hour. A significant portion of the test Grid B was composed of soils of different types, namely Gp03, Gp16, Gp19, Cp07, and Gp11 [40,41,42]. WS1, WS2, and WS4 are located in Gp03; WS3 and WS6 are located in Gp16, and station WS5 is located in Gp19 [42]. Even though Cp07 and Gp11 also encompass a large range, installing the weather station was challenging due to its unlatched nature and potential danger. In the validation grid of both 3 km and 9 km resolution grids, the stations (WS3 and WS6) are equipped with Campbell and Stevens Model Hydra Probes, while the stations (WS1, WS2, WS4, and WS5) installed in the other zone are equipped with Stevens Model Hydra Probes and ML3 soil sensors. In order to calibrate these sensors, thermogravimetric samples were collected at the same time. However, for a study of the temporal stability of these sensors, average monthly values were used from September 2015 to April 2020.

Five field visits were conducted between December 2014 and August 2015. Then, 78 field visits were conducted between September 2015 and April 2018, which gave a total of 83 field visits for the whole period (December 2014–April 2018). Soil samples at 5 cm depth were collected around the six permanent stations at a rate of 5 samples per station. These samples were collected continuously during the wet season and less frequently during the dry, hot season. In order to prepare for the inter-comparison study, soil moisture data from each station were averaged every 30 min at 6 am and 6 pm in accordance with the satellite passing time corresponding to each test grid resolution. In addition, daily average rainfall was collected from the station for data analysis. Accordingly, Campbell and Stevens Model Hydra Probes are evaluated on 3 km and 9 km resolution grids, whereas all sensors (Campbell, Stevens Model Hydra Probe, and ML3 soil sensors) are evaluated on 36 km grids.

2.2.3. SMAP Dataset

NASA launched the Soil Moisture Active Passive (SMAP) satellite on 31 January 2015, to estimate soil moisture and freeze/thaw as a global average using an L-band radiometer and an L-band radar. SMAP began acquiring routine science data on 31 March 2015 [49]. SMAP provides the radar (3 km) and radiometer (36 km and 9 km) resolutions, as well as radar–radiometer (9 km) resolutions, for soil moisture over the SMAP grid at intervals of three days. The SMAP radar stopped transmitting on 7 July 2015 [49] and, therefore, radar (3 km) and radar–radiometer 9 km data products are available till this date only. Now, SMAP provides data on global soil moisture and freeze/thaw states every 2–3 days using data from an L-band (1.40 GHz) radiometer only. SMAP provides SM data products at 3 km, 9 km, and 36 km spatial grid resolutions at three-day intervals with an approximate nominal spatial resolution of 9 km, 33 km, and 40 km, respectively [31,50,51,52]. For the validation study, the following soil moisture data products during ascending (6 pm) and descending (6 am) passes were obtained from NASA’s Distributed Active Archive Center (DAAC) at NSIDC [53] for the corresponding latitude/longitude of the Kuwait SMAP test site between April 2015 and April 2020. The SMAP Soil Moisture dataset used for this study comprises Level 3_36 km (SMAP_L3_SMP) passive microwave radiometer data, Level 3_9 km (SMAP_L3_SM_P_E) passive microwave radiometer data, and Level 2_3 km data (SMAP_L2_SM_SP), a combination of SMAP radiometer and Sentinel-1A and 1B radar data [54,55,56]. The SMAP Level-3 (L3) 9 km soil moisture data derived from SMAP Level-1C (L1C) interpolated brightness temperatures using Backus–Gilbert optimal interpolation techniques [53]. SMAP validation sites of 36 km (entire grid, yellow)/9 km (cyan)/3 km (green) were identified in the test sites and are presented in Figure 1c. The 9 km and 3 km grids were selected around WS3 since it was the first ground station to be installed and had a continuous record of VSM data. According to the specified resolution grid coordinates, SMAP provides specific soil moisture data over each resolution grid.

2.2.4. SMOS Dataset

In November 2009, the European Space Agency (ESA) launched the Soil Moisture and Ocean Salinity (SMOS) satellite mission. The satellite is fitted with a microwave synthetic aperture radiometer operating at 1.4 GHz, and many changes have been made to the algorithms for estimating soil moisture. SMOS daily NRT data were used for this study from April 2015 to April 2020. Operational Science Data product MIR_SMUDP2 is assimilated from the ESA SMOS Online Dissemination Service for a spatial resolution of 15 km in an equal-area grid system ISEA 4H9 of spatial resolution (15 km hexagonal cyan) (shown in Figure 1c) and, to fit in with the SMAP L3 SM_P (36 km) validation grid spatial resolution, SMOS data at 15 km were resampled to 36 km spatial resolution. This was done through the interpolation of available SMOS 15 km SM grids within the whole 36 km test grid. Similarly, to evaluate the 9 km and 3 km soil moisture data, the level 4 CATDS high-resolution soil moisture data product (1 km) also derived from ESA SMOS L2 soil moisture was used. The retrieved soil moisture was downscaled by DISaggregation based on the Physical And Theoretical scale Change (DISPATCH) algorithm and the use of information for land surface temperature and vegetation index obtained from satellite observation (NASA Terra/Aqua MODIS) [57,58,59,60,61,62,63]. The SMOS L4 CATDS SM data are publicly available at [64]. The 1 km L4 data were resampled through interpolation of available 1 km L4 SM data to obtain the 9 km and 3 km, respectively, and used for evaluation. Both the ascending (6 am) and descending (6 pm) pass soil moisture data products (L2 and L4) were used in this analysis.

2.2.5. Radio Frequency Contamination and Filtration

Even though the L-band 1.4 GHz frequency falls under the protected portion exclusively allocated to microwave radiometers deployed in spaceborne remote sensing missions [65], some adjacent out-of-band emission leakage and illegal transmitters cause Radio Frequency Interference (RFI) contamination during the satellite missions [66,67]. This RFI leads to degradation in the quality of the radiometric data to be used to retrieve critical geophysical parameters such as soil moisture, sea surface salinity, sea surface wind, and vegetation index. Hence, an RFI detection and mitigation algorithm has been developed and implemented in L-band microwave radiometer satellite missions. During the validation study, an RFI survey was conducted to identify the anthropogenic emissions over the neighborhood of the test grid and found that the primary sources were due to military and airport radars, such as those installed in the Ali Al Salem Air force base, Ahmed Al Jaber Air Base and Camp Buerhing, and Kuwait International Airport, and hence we ensured the distance of the test grid from these locations.

The satellite data products ensure quality through the support of quality flags. For SMAP soil moisture data (L3_36 km, L3_9 km, and L2_3 km), the RFI filtration is taken from the TH_aft and TV_aft bit values from the cell of TB quality flags with value 0 of the corresponding grid observations from L1C_TB data [68,69]. In the case of SMOS data (L2 SMUDP and L4 CATDS SM), the RFI_prob > 0.1 is considered to eliminate RFI [70]. Moreover, the validation side, grid cell B, was far from open water (Figure 1a), which should have contributed to improving the quality of soil moisture data [44,69].

2.3. Soil Moisture Data Analysis Techniques

The main purpose of this study was to evaluate the top 5 cm soil moisture derived from satellite measurements of SMAP and SMOS in comparison to the soil moisture obtained from ground station sensors at different grid spatial resolutions (36 km, 9 km, and 3 km) across the test site. The following methods have been used for soil moisture data analysis.

2.3.1. Thermogravimetric and Ground Station SM Data Analysis

The thermogravimetric soil moisture data collected from December 2014 to April 2018 were used for two purposes. The first purpose was to calibrate the ground stations’ sensors and the second one was to assess the test sites’ temporal stability. Temporal stability is often called time stability, and other terms, such as rank stability or order stability, have been proposed and discussed in [71]. The temporal stability of the test site was computed through the mean relative difference (MRD) value (

Table 1. Statistical metrics used for soil moisture evaluation.

Metric	Symbol	Definition	Range	Perfect Score
Mean Relative Difference	MRD	${\delta }_{\left(i\right)}=\frac{1}{t}{\displaystyle {\displaystyle \sum }_{j=1}^t}\frac{S_{ij }-S_j}{S_j}$	[−∞, +∞]	0
Standard Deviation of Relative Difference	SDRD	$$\begin{gathered} {\sigma }_{\delta () \left(i\right)}{}^2 \\ =\frac{1}{t-1}{\displaystyle {\displaystyle \sum }_{j=1}^t}[\frac{(S_{ij }-S_j)}{S_j} \\ -\overline{\delta } \left(i\right)] \end{gathered}$$	[−∞, +∞]	0
Absolute Mean Bias	AMB	$AMB=\left|VS{M}_{ST}-VS{M}_{TH}\right|$	[−∞, +∞]	0
Standard Deviation	SD	$SD=\sqrt{\frac{{{\displaystyle \sum }}_{i=1}^n{\left(X_i-\overline{X}\right)}^2}{n-1}}$	[−∞, +∞]	0
Mean Difference	MD	$\frac{{{\displaystyle \sum }}_{i=1}^n{\theta }_{s \left(i\right)}-{\theta }_{m\left(i\right)}}{n}$	[−∞, +∞]	0
Root Mean Square Error	RMSE	$\sqrt{\frac{1}{n}{\displaystyle {\displaystyle \sum }_{i=1}^n}{\left({\theta }_{s\left(i\right)}-{\theta }_{m\left(i\right)}\right)}^2}$	[0, +∞]	0
Unbiased Root Mean Square Difference	ubRMSD	$\sqrt{{\mathrm{RMSE}}^2-{\mathrm{MD}}^2}$	[0, +∞]	0
Correlation Coefficient	R	$\frac{{{\displaystyle \sum }}_{i=1}^n({\theta }_{s\left(i\right)}-{\mu }_s)({\theta }_{m\left(i\right)}-{\mu }_m)}{\left(n-1\right){\sigma }_s {\sigma }_m}$	[−1, 1]	1

) between the $S_{ij}$ (jth sample at the ith site among the n sites within the study region) and $S_j$ (computed average among all sites for a given date and time j (j = 1 to t)). The mean relative difference compares the value at a particular site to the average over the area of study, and the site is considered a representative of the large-scale average if its MRD is nearly zero. The mean relative difference compares the value at a particular site to the average over the area of study, and the standard deviation of the relative differences (SDRD) (Table 1) between the $S_{ij}$, $S_j$, and $\overline{\delta }$ (average of MRD values). The most reliable temporal stability test site exhibits MRD values and SDRD values close to zero [72].

The SM data from ground sensors were taken for this study simultaneously with the thermogravimetric measurements, and hence the samples were coincident. The calibration of station sensors was estimated through the absolute mean bias (AMB) between the station sensor data ($VS{M}_{ST})$ and thermogravimetric data ($VS{M}_{TH}$) (from September 2015 till April 2018).

The temporal stability of ground sensors was determined by the calculation of the MRD and SDRD of VSM values recorded by the ground sensors simultaneously with the thermogravimetric measurements. A consistently low range of MRD and SDRD values denotes the temporal stability of ground sensors even though it exhibits a potentially biased relationship between the individual site and the overall average, and the site is considered representative of the large-scale average if its MRD and SDRD are near zero [72].

2.3.2. Soil Moisture Sampling Density Analysis

The test site encompasses a 36 km × 36 km area and includes areas that are restricted due to military operations and oil refineries. Thus, the distribution of ground station installations was not even and uniform. Two intensive field campaigns were conducted on 20 February and 19 March 2016, to collect additional soil moisture observations and support those collected from the permanent network [44]. The total number of observations from the intensive field campaign and the permanent stations was upscaled to prepare for the verification of satellite SM products. The collected soil moisture observations during the field campaigns should certainly improve the quality of the upscaled soil moisture estimates. Statistical analysis of the volumetric soil moisture values derived from the total 322 samples showed a very low variability of MRD = ±0.005 m³ m⁻³. Results indicate that the volumetric water content was spatially and temporally stable across the entire test site during the field campaigns. To study the sampling density within the test site (36 km × 36 km), a total of 144 soil samples were taken at every 3 km² grid at 5 cm depth in 20 February 2016 to determine the thermogravimetric soil moisture measurements. The standard deviation was estimated from 144 samples corresponding to different spatial resolutions between the individual value ($X_i$) and the mean value of $X_i$ ($\overline{X}$). The uniform SD value for different spatial resolutions shows the strong harmony of the sampling density. A bilinear interpolation of the VSM data led to an abridgment of 16 samples between 3 km and 6 km, nine samples between 6 km² and 21 km², and four samples between 21 km² and 36 km².

2.3.3. Satellite Soil Moisture Data Product Analysis

SMAP daily soil moisture data for the ascending and descending passes were extracted from the Level_3_36 km (SMAP_L3_SMP), Level_3_9 km (SMAP_L3_SM_P_E), and Level-2_3 km products (SMAP_L2_SM_SP). On the other hand, the ESA SMOS L2_MIR_SMUDP2 daily data product was obtained at ascending/descending passes for 15 km equal-area grid system ISEA 4H9 and the level 4 CATDS high-resolution soil moisture data product at 1 km grid resolution. The correlation coefficient (R) value for SMAP and SMOS monthly average VSM data at different spatial resolutions during the am/pm passes was estimated for April 2015 till April 2020. Moreover, SMAP and SMOS soil moisture data were discretely validated with respect to station sensors at the different spatial resolutions of 36 km, 9 km, and 3 km (from April 2015 till April 2020), and a comparison of accuracy for SMAP and SMOS VSM with respect to station VSM for the same am/pm passes was performed in terms of the four statistics (MD, RMSE, ubRMSD, and R). Statistical metrics’ equations are tabulated below (Table 1). These statistical studies led to the spatial–temporal characteristic analysis of VSM data.

For spatial analysis, for VSM, ${\theta }_{s\left(i\right)}$ represents the SMAP VSM and ${\theta }_{m\left(i\right)}$ represents the SMOS VSM, while ${\mu }_s$ and ${\mu }_m$ represent the average soil moisture value of SMAP and SMOS, and ${\sigma }_s$ and ${\sigma }_s$ denote its respective standard deviation values. However, for the spatial–temporal study, the values of ${\theta }_{s\left(i\right)}$ represent the satellite VSM and ${\theta }_{m\left(i\right)}$ is the station sensor VSM, while ${\mu }_s$ and ${\mu }_m$ represent the average soil moisture values of satellites and SMOS, and ${\sigma }_s$ and ${\sigma }_s$ denote its respective standard deviation values.

The four statistical indicators, namely MD, RMSE, ubRMSE, and R, are useful for estimating the quality of the regional soil moisture distribution of SMAP and SMOS at different spatial resolutions over the test site. The SMAP satellite data products at different resolutions were directly retrieved from SMAP level 3 and level 2 soil moisture products, while SMOS level 2 soil moisture data (15 km in an equal-area grid) were resampled to meet the 36 km spatial resolution requirement. For 9 km and 3 km resolutions, the SMOS level 4 CATDS high-resolution downscaled data product (1 km) was used.

3. Results

3.1. Temporal Stability of Test Site

The average thermogravimetric VSM data at 5 cm depth over the test site (Figure 1c) were collected during 83 field visits in four years, from December 2014 to April 2018. During the wet season, the maximum thermogravimetric monthly VSM was 0.09 m³ m⁻³ (Figure 2). On the other hand, during the dry season, the minimum thermogravimetric monthly VSM was 0.011 m³ m⁻³ due to prolonged drought in summer (Figure 2). The simultaneous rainfall data during gravimetric sampling were not available on the days of the field trips. However, the gravimetric soil moisture variations in Figure 2 can be correlated to the rainfall data illustrated in Figures 9a–c and 10a–c between April 2015 and April 2018. The high SM value (0.09 m³ m⁻³) in Figure 2 could be due to the heavy rainfall events that occurred for many days during the months of November and December 2015 (as shown in Figures 9a–c and 10a–c).

To assess the temporal stability of soil moisture conditions at the test site, the collected thermogravimetric VSM data were analyzed using the mean relative difference (MRD) and standard deviation of relative difference (SDRD). Figure 3 illustrates the even distribution of MRD values, which are close to zero, except for December 2015, which was due to the exceptionally heavy rain events during the months of November and December 2015, as shown in Figures 9a–c and 10a–c. The MRD determines whether a particular test site consistently reports lower or higher values than the average condition in the entire test site. Figure 3 illustrates the consistency of the average thermogravimetric VSM, while the small SDRD (narrow error bars) demonstrates the low variance of the VSM and, consequently, the uniform distribution of soil moisture conditions at the site. The consistency and low variance of thermogravimetric VSM, and the MRD graph and SDRD value presented in Figure 3, show that the validation test site complies with the temporal stability criteria for a prolonged period, suggesting therefore the temporal consistency and homogeneity of the data sample.

3.2. Calibration of Ground Station Sensors

Calibration was performed by comparing the station sensor readings with reliable thermogravimetric soil moisture data obtained from the 78 field samples (September 2015–April 2018). The soil moisture sensor almost showed positive bias with thermogravimetric measurements, and the continuous comparison of collected VSM data with ground station sensor data resulted in an absolute mean bias (AMB) of 0.034 m³ m⁻³, a root mean square error (RMSE) of 0.042 m³ m⁻³, and an R value of 0.63 (Figure 4). It is important to note that the bias of the sensor reading is more pronounced during wet seasons, while it is minimal during dry seasons. The AMB (0.034 m³ m⁻³) was used to calibrate and bias-correct the entire dataset of ground sensors. Consequently, SM data obtained from satellite observations were validated based on post-calibration and bias-corrected ground sensor measurements.

3.3. Analysis of the Temporal Variability of Soil Moisture from Ground Stations

Bias-corrected monthly-averaged sensor soil moisture measurements from September 2015 through April 2020 are plotted in Figure 5. It was observed that the maximum value of 0.157 m³ m⁻³ VSM occurred during the wet season (January 2016). As the dry season progressed (October 2018), the maximum VSM declined to 0.011 m³ m⁻³, with an average VSM of 0.064 m³ m⁻³ over the course of the observations (Figure 5).

The consistency among the readings from ground sensors installed in the test site was also assessed. It was expected, given the homogenous surface condition within the site and the prevailing dray climate condition, that the sensors would report consistent readings of soil moisture throughout the sampling period, suggesting therefore the temporal stability of the monitoring network. An analysis of the MRD and SDRD values of ground sensors’ VSM after calibration was accomplished (Figure 4). Unlike the MRD and SDRD patterns of the thermogravimetric VSM, the sensors’ MRD and SDRD error-bound (SDRD = 0.023 m³ m⁻³) values were not close to zero, as shown in Figure 6. The low variance of VSM displayed in the MRD graph judiciously demonstrates its temporal stability, even if there may be a potentially biased correlation between the individual site and the overall average.

3.4. Sampling Density Inference

Based on the bilinearly interpolated VSM data within the test site (36 km × 36 km), taken from the 144 thermogravimetric soil samples from every 3 km² grid at 5 cm depth during the intensive field trip of 20 February 2016, the standard deviation of sampling density at different spatial resolutions was estimated. This included 16 samples between 3 km and 6 km, nine samples between 6 km² and 21 km², and four samples between 21 km² and 36 km² (Figure 7), within 11 separate extent scales.

Figure 7 shows a linear plot of the VSM standard deviation versus the number of samples at different spatial resolutions, and it reveals an increasing trend between 3 km² and 36 km² resolution, from 0.0176 m³ m⁻³ to 0.053 m³ m⁻³. A similar study was conducted by [41]. The average value of this standard deviation for the reading is projected to an approximate result of 0.02 m³ m⁻³, satisfying the minimum requirement of four sampling units every 21 km² for upscaling the entire 36 km² test site.

3.5. Assessment of Satellite Soil Moisture Retrievals

3.5.1. Intercomparison of SMAP and SMOS VSM Products

For the intercomparison study, the 15 km available grids of SMOS SM data were interpolated over a 36 km spatial grid, while, for 3 km and 9 km spatial resolution, the 1 km available grids of SMOS SM data were interpolated. In order to reduce the high variability of VSM data products, the monthly average of satellite SM data was computed through a MATLAB code on the basis of a minimum of five available daily data over corresponding grids at different spatial resolutions for the am/pm passes from April 2015 to April 2020. An overestimation of SMAP VSM data was noticed in both pm and am passes (Figure 8a,b). To identify any trend line setting between the VSM data products of SMAP and SMOS, a linear fit was plotted for am/pm passes (Figure 8a,b).

From the linear fit plot and statistical metrics (Figure 8a,b), we found that SMAP_SMOS_36 km_am and SMAP_SMOS_pm_36 km had good correlations of R = 0.89 and 0.78, respectively, whereas SMAP_SMOS_am_9 km had a poor correlation of R = 0.47. SMAP_SMOS_ 9 km pm had a good correlation of R = 0.78, while SMAP_SMOS_9 km am and SMAP_SMOS_3 km pm had lower but acceptable correlations of R = 0.55 and 0.60, respectively. The ubRMSE values of the am/pm passes of the SMAP and SMOS SM products also justified the good, medium, and poor correlations of the SM data products at different spatial grid resolutions. In general, the SM products from both sensors were in agreement for the 36 km am–pm passes and 9 km pm pass, while, for the 3 km am–pm pass and 9 km pm pass, the SMAP SM products significantly overestimate compared to SMOS.

3.5.2. Validation of Satellite SM Products from the 6 pm Pass

The validation of soil moisture products involved data at 5 cm depth from the SMAP and SMOS satellites, and the calibration of ground station sensors’ daily data with daily rainfall data, which were plotted at different spatial resolutions, such as 36 km, 9 km, and 3 km, respectively, for 6 pm passes from April 2015 till April 2020 (Figure 9a–c). The calibrated station sensor reading averaged between 5.45 pm and 6.15 pm was taken as the reference to validate the SMAP (ascending—6 pm) and SMOS (descending—6 pm) data.

The analysis of VSM data (Figure 9a–c) shows that the data available over the Kuwait test site for SMAP 36 km and 9 km are overall higher than SMOS 36 km and 9 km, while SMAP values for the 3 km product are lower than the SMOS data. The peaks in soil moisture data in response to rain events are evident for both satellites, despite some noticeable discrepancies. The four statistical evaluations of the satellite data show the slightly better performance of SMOS SM products when using ground measurements for the 6 pm pass, as per the statistics indicated in Figure 9a–c.

3.5.3. Validation of Satellite SM Products from the 6 am Pass

Similar to the previous section, the SMAP and SMOS SM products were validated against ground observations using the 36 km, 9 km, and 3 km, products for the 6 am pass. The values collected from April 2015 to April 2020 are displayed in Figure 10a–c. Ground observations were averaged between 5.45 am and 6.15 am, taken as the reference to validate the SMAP (descending—6 am) and SMOS (ascending—6 am) data.

The overall variability of soil moisture during the study period from the SMAP, SMOS, and ground stations for the 36 km, 9 km, and 3 km products is illustrated in Figure 10a–c, respectively. The performance of the SM products with the 6 am overpass data is sensitive for both satellites compared to the 6 pm (statistics from Figure 10a–c). While the satellite SM products from the 6 am overpass seem to respond to rainfall events, as in the case of the 6 pm overpass, both satellite products tend to persistently overestimate soil moisture during dry periods. The study captures the covariance with a correlation (R) between satellite SM products’ data and sensor data, in addition to the range of soil moisture observations through ubRMSE estimations. The plateaus of SMAP and SMOS that correspond to dry periods are in the order of 0.05 m³ m⁻³, whereas the reported observations are much lower and fluctuate around 0.01 m³ m⁻³.

The performance metrics reported in Figure 9a–c and Figure 10a–c include an average ubRMSE value of 0.043 m³ m⁻³, which is close to yet slightly higher than the ubRMSE mission target for SMAP, which is 0.04 m³ m⁻³ [73]. The average ubRMSE value of 0.045 m³ m⁻³ was estimated for SMOS, which is also close to yet higher than the ubRMSE SMOS mission target, which is 0.04 m³ m⁻³ [38,74]. Overall, the SMAP data have slightly better accuracy during the 6 am overpasses than 6 pm ones, and vice versa for SMOS. On the other hand, SMOS shows a slightly drier pattern than SMAP.

4. Discussions

The collected soil moisture observations from the permanent stations and the gravimetric sampling during the field visits confirmed the prevailing low soil moisture values, which are expected in a hyper-climate such as the one in Kuwait. However, one should note that the satellite SM products’ results were systematically higher than those of in situ observations during the dry periods. Overestimation was reported at the three different resolutions, which is expected as they all use the L-band brightness temperature, with the exception of the 3 km product, which introduces radar data. The L-band microwave frequency is characterized by higher penetration in the soil, especially in the dry seasons, and this might explain the higher soil moisture products from satellites, as it detects lower-depth moisture. Moreover, the systematic overestimation of the satellite soil moisture products could be attributed due to the inaccurate characterization of the desert soil texture and dielectric properties. Another reason that could lead to the overestimation in the SM products could be related to the discrepancies in the determination of the effective temperatures used in the radiative transfer models. The effective temperature is modeled globally and then input into the radiative transfer model to determine surface emissivity, which is a proxy for the soil dielectric constant and soil moisture. Errors in the determined effective temperature because of a lack of accurate characterization of soil thermal and physical properties such as thermal conductivity and porosity could generate a bias, which may propagate to soil moisture product retrievals.

The current study also evaluated the performance of spatial and temporal variations in SMAP and SMOS surface SM products generated for the state of the Kuwait desert test site concerning the ground station sensor measurements for different spatial distributions on a daily and monthly basis as a reference. Throughout the study period, the steadiness of SMAP and SMOS soil moisture data products in accordance with the seasonal variations was clear; in particular, the VSM data rise at every rain event, inconsistent with the ground station, as detected in another relevant study [75]. The spatial–temporal VSM plot at different spatial resolutions shows a gradual decrease in the VSM data of ground station sensors after a rain event due to the amassed water in the rain gauge, whereas the VSM data from SMAP and SMOS show isolated VSM values at rain events [76].

The discrepancies between satellite SM products and in situ observations in response to the rainfall events recorded during the study period could be attributed to the combination of the above-mentioned factors, related to the lack of an accurate characterization of the desert soil, in addition to the potential mismatch between the timing of the rainfall and the overpass time of SMAP and SMOS, which is either early in the morning, at 6 am, or late in the day, at 6 pm. In arid regions such as those in the Arabian Peninsula, the prevailing atmospheric pressure suppresses rainfall events by preventing the updrafts and condensation. As a result of the local sea land breeze and convergence during the day, local convective cells may form—typically in the afternoon—and trigger local rainfall events [77]. The likelihood of rainfall occurrence in the afternoon, not early in the morning or late in the day, maximizes the lag with respect to the satellite overpass times. In deserts, the total rainfall will rather rapidly infiltrate and recharge the water table or evaporate, generating, therefore, a bias between the daily record of precipitation and the satellite retrievals.

The noticeable temporal stability of the soil moisture observations that are collected at the test site corroborated its suitability as a calibration and validation site for the mission. The homogeneity and temporal stability of the Kuwait test site were observed in [44]. The consistency of soil moisture values throughout the seasons, with an occasional response to a few rainfall events, should be reflected in the satellite measurements of brightness temperatures and hence the retrieved soil moisture. The quasi-static surface conditions in the absence of predominant vegetation cover, as is the case of temperate weather sites, makes the desert site an ideal calibration target for the satellite readings. The Kuwait site was selected as a candidate site for the NASA SMAP Cal/Val mission by JPL, and it was compared with other core validation sites in [30,48] and found to be exceptionally homogeneous; it could credibly estimate average soil moisture with a reduced number of stations.

The analysis of the standard deviation of the 144 thermogravimetric VSM data collected at each 3 km grid cell on 20 February 2016 led to an average value of standard deviation of 0.02 m³ m⁻³, which is suitable for the extent of the grid cell and could be considered suitable for upscaling the entire 36 km test site, using a relationship that could be established with the six permanent ground stations, which could be addressed in future work. Previous studies stressed the importance of the use of spatially distributed sparse and permanent ground-based observations in reducing the detrimental impact of spatial sampling errors and improving the upscaling techniques during the validation of satellite data products [11,41,78]. The use of a single-point observation over a less heterogeneous landscape with a more consistent soil type, such as Kuwait’s homogeneous desert test site, was proven to be feasible [79].

The spatial–temporal stability analysis of the satellite VSM data products (April 2015–April 2020) at different spatial resolutions (36 km, 9 km, and 3 km) at both overpass times (am–pm) across the desert test site indicates a slight overestimation pattern for SMAP during some days in summer with respect to SMOS soil moisture retrievals. This overestimation pattern may be due to the static water effect, which was discussed in [76]. Moreover, in the desert, sand often accumulates over the buried soil moisture sensors during heavy sandstorms and increases their effective depth, which can lead to some biases between soil sensors and satellite estimates [80].

The baseline science requirement for SMAP is to provide estimates of soil moisture in the top 5 cm of soil with a ubRMSE < 0.04 m³ m⁻³ volumetric (after removal of long-term mean bias) in the case of active/passive and passive products and RMSE < 0.06 m³ m⁻³ (after removal of long-term mean bias) in the case of the active product for the range of 10–40 km spatial resolution over the global land area, excluding regions of snow and ice, frozen ground, mountainous topography, open water, urban areas, and vegetation with water content greater than 5 kg m^–2 (averaged over the spatial resolution scale) [73]. However, for SMOS, and in the case of bare soil measurements, for which the influence of near-surface soil moisture on surface water fluxes is strong, it has been shown that a random error of 0.04 m³ m⁻³ allows a reasonable estimation of the evaporation and soil transfer parameters [38,74].

This long-term (five-year span) study for the validation of SMAP using ground observations reported ubRMSE values that are close to yet somewhat higher than the mission target of 0.04 m³ m⁻³, which implies that there is a need to advance the retrieval of soil moisture in arid regions. This could be achieved through more studies that involve, in addition to in situ VSM, surface observations of the brightness temperature and backscatter from ground-based radiometers and scatterometers such as those deployed in Kuwait, and which could be used in future work to enhance radiative transfer modeling in desert sites. The analysis of the determined correlation coefficient (R) between SMAP and in situ VSM observation showed persistently high correlation values, indicating a strong covariance between both datasets, which are dominated by plateaus during dry periods that could explain the high covariance and occasional responses to sporadic rainfall events.

5. Conclusions

The candidate calibration and validation test site considered in this study is located in Kuwait, a homogeneous, sparse desert area. In addition, the site has stable surface conditions throughout the seasons, which makes it an ideal sensor calibration target that is also suitable for the validation of satellite retrievals. This study leveraged in situ soil moisture observations from a permanent soil moisture monitoring network that consists of six stations, which reported measurements between April 2015 and April 2020. These observations were supported by additional high-density in situ data collected around the permanent monitoring network stations during a series of intensive field campaigns in the desert of Kuwait. The consistency between SMAP and SMOS SM products was also covered in this study.

The study demonstrated the agreement between the SMAP and SMOS satellites in capturing the seasonal variations in accordance with the in situ ground station sensors’ readings. Both satellites accurately captured rainfall events, although some sporadic contradictions in the temporal variations of soil moisture inference could be seen at different spatial resolutions at different overpass timings. The evaluation of the spatial–temporal variance of satellites was performed through four statistical metrics, with the estimation of MD, RMSE, ubRMSE, and R. A slightly better correlation coefficient was exhibited by SMOS for 36 km pm, 9 km pm, 3 km pm, and 9 km am, while, for 36 km am and 3 km am, SMAP showed a better correlation coefficient. These minor discrepancies may be attributed to the higher volume of SMAP data availability over the test site. The analysis of the ubRMSE values of SMAP at different spatial resolutions and overpass times showed that, on average, the metric was slightly higher than the SMAP mission target.