# On the Evaluation of Both Spatial and Temporal Performance of Distributed Hydrological Models Using Remote Sensing Products

^{1}

^{2}

^{3}

^{4}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Methodology

#### 2.1. Study Area and Data

^{2}(Figure 1). The digital elevation model (DEM) from the Shuttle Radar Topography Mission shows that the elevation of the catchment varies in a wide range, from 330 to 1120 m above mean sea level (a.m.s.l). Despite the study area covering a relatively large area with high variation in topography, there is only one weather station located in the area. Observed rainfall shows an average annual rainfall of 1165 mm with high temporal variation and out of which more than 80% of the rainfall occurs between June and October. The average monthly minimum and maximum temperature in the area are 11 °C and 34 °C, respectively. The dominant land uses/land covers in the area are agricultural, forest, and range grass, accounting for about 36%, 32%, and 25% of the study area, respectively. The dominant soil type in the area is sandy clay loam with low water holding capacity [32]. The observed weather data were procured from the India Meteorological Centre, Bhubaneswar, India. Furthermore, sunshine hours were used for calculating the solar radiation data for the considered catchment.

#### 2.2. Spatiotemporal and Temporospatial Model Performance Evaluation

#### 2.3. Reference Spatiotemporal ETa

#### 2.4. The SWAT Model

#### 2.5. Model Setup, Parameter Variation, and Model Evaluation for ETa

#### 2.5.1. Model Setup

^{−1}), ${R}_{n}$ is the net radiation (MJ m

^{−2}, d

^{−1}), G is the heat flux to the soil (MJ m

^{−2}d

^{−1}), ${\rho}_{a}$ is the air density (kg m

^{−3}), ${c}_{p}$ is the specific heat at constant pressure (MJ kg

^{−1}°C

^{−1}), ${e}_{z}^{o}$ and ${e}_{z}$ are the saturated and the actual vapor pressure at height z (kPa), respectively, ${r}_{a}$ is the bulk surface aerodynamic resistance for water vapor (s m

^{−1}), $\lambda $ is the latent heat of vaporization (MJ kg

^{−1}), $\gamma $ is the psychrometric constant (kPa °C

^{−1}), and ${r}_{c}$ is the canopy surface resistance (s m

^{−1}).

#### 2.5.2. Parameter Variation and Model Performance Evaluation Scenarios

_{S}and S

_{T}are the spatiotemporal and temporospatial model performance indices, respectively, ${T}_{ETa}^{j}$ and ${S}_{ETa}^{k}$ are the statistical indices for temporal and spatial model performance, respectively, for actual evapotranspiration (ETa) at the pixel j and time step k, npixels is the number of pixels, and k is the number of evaluation time steps. For this analysis, we selected a MOD16 A2 ETa product as reference data with ${S}_{ETa}$ is SPAEF (Equation (6)) and ${T}_{ETa}$ is NSE (Equation (7)) as an example in this case (Section 3.1).

_{T}) and temporospatial (T

_{S}) performance indices (Table 2). For this evaluation, different ETa products (MOD16 A2, SEEBop, and TerraClimate) were used to draw reliable conclusions. The relation between spatiotemporal and temporospatial model performance was analyzed based on 2000 model runs (Section 3.2).

_{T}) and temporospatial (T

_{S}) performance statistics (Table 2) are calculated using Equations (3) and (4), respectively, while the temporal T (e.g., SPAEF, NSE, RMSE, RSE, absolute bias aBIAS) and spatial S (e.g., KGE, NSE, RMSE, RSR, aBIAS) performance are calculated as follows:

## 3. Results and Discussion

#### 3.1. Spatiotemporal and Temporospatial Model Performance

#### 3.2. The Relation between Spatiotemporal and Temporospatial Model Performance

_{T}) and temporospatial (T

_{S}) model performance varies depending on the reference ETa data and the statistical indices used for evaluation. For example, when KGE and SPAEF were used to derive the spatiotemporal (S

_{-KGE}) and temporospatial (T

_{-SPAEF}) mode of performance statistics, S

_{-KGE}and T

_{-SPAEF}could be highly positively correlated (Figure 5a), highly negatively correlated (Figure 5b), or uncorrelated (Figure 5c). A high positive correlation between S

_{-KGE}and T

_{-SPAEF}indicates that an increase in the spatiotemporal model performance will likely lead to an increase in temporospatial model performance. However, a high negative correlation between S

_{-KGE}and T

_{-SPAEF}means that an increase in spatiotemporal model performance will likely result in a decrease in temporospatial model performance and vice versa. An uncorrelated S

_{-KGE}and T

_{-SPAEF}indicates that the spatiotemporal model performance cannot be inferred from the temporospatial model performance and vice versa. The results show that even if the same statistical index was used for spatiotemporal and temporospatial model evaluation (e.g., S

_{-NSE}and T

_{-NSE}, Figure 5a) it does not always guarantee that an increase in the spatiotemporal model performance will lead to an increase in the temporospatial model performance (Figure 5a–c). Overall, the results show that both spatiotemporal and temporospatial model performance evaluation is needed.

## 4. Conclusions

## Author Contributions

## Funding

## Data Availability Statement

## Acknowledgments

## Conflicts of Interest

## References

- Klingler, C.; Schulz, K.; Herrnegger, M. Large-Sample Data for Hydrology and Environmental Sciences for Central Europe. Earth Sci. Data
**2021**, 13, 4529–4565. [Google Scholar] [CrossRef] - Brocca, L.; Tullo, T.; Melone, F.; Moramarco, T.; Morbidelli, R. Catchment Scale Soil Moisture Spatial-Temporal Variability. J. Hydrol.
**2012**, 422–423, 63–75. [Google Scholar] [CrossRef] - Wilson, D.J.; Western, A.W.; Grayson, R.B. Identifying and Quantifying Sources of Variability in Temporal and Spatial Soil Moisture Observations. Water Resour. Res.
**2004**, 40, W02507. [Google Scholar] [CrossRef] - Thomas, A. Spatial and Temporal Characteristics of Potential Evapotranspiration Trends over China. Int. J. Climatol.
**2000**, 20, 381–396. [Google Scholar] [CrossRef] - Edmunds, W.M.; Fellman, E.; Goni, I.B.; Prudhomme, C. Spatial and Temporal Distribution of Groundwater Recharge in Northern Nigeria. Hydrogeol. J.
**2002**, 10, 205–215. [Google Scholar] [CrossRef][Green Version] - Neitsch, S.; Arnold, J.; Kiniry, J.; Williams, J. Soil & Water Assessment Tool Theoretical Documentation Version 2009; Texas Water Resources Institute: College Station, TX, USA, 2011; Available online: https://swat.tamu.edu/media/99192/swat2009-theory.pdf (accessed on 17 April 2022).
- Arnold, J.G.; Srinivasan, R.; Muttiah, R.S.; Williams, J.R. Large Area Hydrologic Modeling and Assessment Part I: Model Development. J. Am. Water Resour. Assoc.
**1998**, 34, 73–89. [Google Scholar] [CrossRef] - Beven, K.J.; Kirkby, M.J. A Physically Based, Variable Contributing Area Model of Basin Hydrology. Hydrol. Sci. Bull.
**1979**, 24, 43–69. [Google Scholar] [CrossRef][Green Version] - Liang, X.; Lettenmaier, D.P.; Wood, E.F.; Burges, S.J. A Simple Hydrologically Based Model of Land Surface Water and Energy Fluxes for General Circulation Models. J. Geophys. Res.
**1994**, 99, 14415–14428. [Google Scholar] [CrossRef] - Kumar, R.; Samaniego, L.; Attinger, S. Implications of Distributed Hydrologic Model Parameterization on Water Fluxes at Multiple Scales and Locations. Water Resour. Res.
**2013**, 49, 360–379. [Google Scholar] [CrossRef] - Samaniego, L.; Kumar, R.; Attinger, S. Multiscale Parameter Regionalization of a Grid-Based Hydrologic Model at the Mesoscale. Water Resour. Res.
**2010**, 46, W05523. [Google Scholar] [CrossRef][Green Version] - Campo, L.; Caparrini, F.; Castelli, F. Use of Multi-Platform, Multi-Temporal Remote-Sensing Data for Calibration of a Distributed Hydrological Model: An Application in the Arno Basin, Italy. Hydrol. Process.
**2006**, 20, 2693–2712. [Google Scholar] [CrossRef] - Dembélé, M.; Ceperley, N.; Zwart, S.J.; Salvadore, E.; Mariethoz, G.; Schaefli, B. Potential of Satellite and Reanalysis Evaporation Datasets for Hydrological Modelling under Various Model Calibration Strategies. Adv. Water Resour.
**2020**, 143, 103667. [Google Scholar] [CrossRef] - Herman, M.R.; Nejadhashemi, A.P.; Abouali, M.; Hernandez-Suarez, J.S.; Daneshvar, F.; Zhang, Z.; Anderson, M.C.; Sadeghi, A.M.; Hain, C.R.; Sharifi, A. Evaluating the Role of Evapotranspiration Remote Sensing Data in Improving Hydrological Modeling Predictability. J. Hydrol.
**2018**, 556, 39–49. [Google Scholar] [CrossRef] - Immerzeel, W.W.; Droogers, P. Calibration of a Distributed Hydrological Model Based on Satellite Evapotranspiration. J. Hydrol.
**2008**, 349, 411–424. [Google Scholar] [CrossRef] - Jiang, L.; Wu, H.; Tao, J.; Kimball, J.S.; Alfieri, L.; Chen, X. Satellite-Based Evapotranspiration in Hydrological Model Calibration. Remote Sens.
**2020**, 12, 428. [Google Scholar] [CrossRef][Green Version] - Koch, J.; Demirel, M.C.; Stisen, S. The SPAtial EFficiency Metric (SPAEF): Multiple-Component Evaluation of Spatial Patterns for Optimization of Hydrological Models. Geosci. Model Dev.
**2018**, 11, 1873–1886. [Google Scholar] [CrossRef][Green Version] - Mendiguren, G.; Koch, J.; Stisen, S. Spatial Pattern Evaluation of a Calibrated National Hydrological Model—A Remote-Sensing-Based Diagnostic Approach. Hydrol. Earth Syst. Sci.
**2017**, 21, 5987–6005. [Google Scholar] [CrossRef][Green Version] - Odusanya, A.E.; Mehdi, B.; Schürz, C.; Oke, A.O.; Awokola, O.S.; Awomeso, J.A.; Adejuwon, J.O.; Schulz, K. Multi-Site Calibration and Validation of SWAT with Satellite-Based Evapotranspiration in a Data-Sparse Catchment in Southwestern Nigeria. Hydrol. Earth Syst. Sci.
**2019**, 23, 1113–1144. [Google Scholar] [CrossRef][Green Version] - Rajib, A.; Evenson, G.R.; Golden, H.E.; Lane, C.R. Hydrologic Model Predictability Improves with Spatially Explicit Calibration Using Remotely Sensed Evapotranspiration and Biophysical Parameters. J. Hydrol.
**2018**, 567, 668–683. [Google Scholar] [CrossRef] - Rientjes, T.H.M.; Muthuwatta, L.P.; Bos, M.G.; Booij, M.J.; Bhatti, H.A. Multi-Variable Calibration of a Semi-Distributed Hydrological Model Using Streamflow Data and Satellite-Based Evapotranspiration. J. Hydrol.
**2013**, 505, 276–290. [Google Scholar] [CrossRef] - Stisen, S.; Koch, J.; Sonnenborg, T.O.; Refsgaard, J.C.; Bircher, S.; Ringgaard, R.; Jensen, K.H. Moving beyond Run-off Calibration—Multivariable Optimization of a Surface–Subsurface–Atmosphere Model. Hydrol. Process.
**2018**, 32, 2654–2668. [Google Scholar] [CrossRef] - Nguyen, V.T.; Dietrich, J.; Uniyal, B. Modeling Interbasin Groundwater Flow in Karst Areas: Model Development, Application, and Calibration Strategy. Environ. Model. Softw.
**2020**, 124, 104606. [Google Scholar] [CrossRef] - Nguyen, T.V.; Kumar, R.; Lutz, S.R.; Musolff, A.; Yang, J.; Fleckenstein, J.H. Modeling Nitrate Export From a Mesoscale Catchment Using StorAge Selection Functions. Water Resour. Res.
**2021**, 57, e2020WR028490. [Google Scholar] [CrossRef] - Nash, J.E.; Sutcliffe, J.V. River Flow Forecasting through Conceptual Models Part I—A Discussion of Principles. J. Hydrol.
**1970**, 10, 282–290. [Google Scholar] [CrossRef] - Gupta, H.V.; Kling, H.; Yilmaz, K.K.; Martinez, G.F. Decomposition of the Mean Squared Error and NSE Performance Criteria: Implications for Improving Hydrological Modelling. J. Hydrol.
**2009**, 377, 80–91. [Google Scholar] [CrossRef][Green Version] - Roberts, N.M.; Lean, H.W. Scale-Selective Verification of Rainfall Accumulations from High-Resolution Forecasts of Convective Events. Mon. Weather Rev.
**2008**, 136, 78–97. [Google Scholar] [CrossRef] - Koch, J.; Jensen, K.H.; Stisen, S. Toward a True Spatial Model Evaluation in Distributed Hydrological Modeling: Kappa Statistics, Fuzzy Theory, and EOF-Analysis Benchmarked by the Human Perception and Evaluated against a Modeling Case Study. Water Resour. Res.
**2015**, 51, 1225–1246. [Google Scholar] [CrossRef] - Mu, Q.; Zhao, M.; Running, S.W. MODIS Global Terrestrial Evapotranspiration (ET) Product (MOD16A2/A3); Algorithm Theoretical Basis Document Collection 5. 2013; 66p. Available online: https://modis-land.gsfc.nasa.gov/pdf/MOD16ATBD.pdf (accessed on 17 April 2022).
- Senay, G.B.; Bohms, S.; Singh, R.K.; Gowda, P.H.; Velpuri, N.M.; Alemu, H.; Verdin, J.P. Operational Evapotranspiration Mapping Using Remote Sensing and Weather Datasets: A New Parameterization for the SSEB Approach. J. Am. Water Resour. Assoc.
**2013**, 49, 577–591. [Google Scholar] [CrossRef][Green Version] - Abatzoglou, J.T.; Dobrowski, S.Z.; Parks, S.A.; Hegewisch, K.C. TerraClimate, a High-Resolution Global Dataset of Monthly Climate and Climatic Water Balance from 1958–2015. Sci. Data
**2018**, 5, 170191. [Google Scholar] [CrossRef][Green Version] - Uniyal, B.; Dietrich, J.; Vu, N.Q.; Jha, M.K.; Arumí, J.L. Simulation of Regional Irrigation Requirement with SWAT in Different Agro-Climatic Zones Driven by Observed Climate and Two Reanalysis Datasets. Sci. Total Environ.
**2019**, 649, 846–865. [Google Scholar] [CrossRef] - Fick, S.E.; Hijmans, R.J. WorldClim 2: New 1-Km Spatial Resolution Climate Surfaces for Global Land Areas. Int. J. Climatol.
**2017**, 37, 4302–4315. [Google Scholar] [CrossRef] - Hijmans, R.J.; Cameron, S.E.; Parra, J.L.; Jones, P.G.; Jarvis, A. Very High Resolution Interpolated Climate Surfaces for Global Land Areas. Int. J. Climatol.
**2005**, 25, 1965–1978. [Google Scholar] [CrossRef] - Harris, I.; Jones, P.D.; Osborn, T.J.; Lister, D.H. Updated High-Resolution Grids of Monthly Climatic Observations—The CRU TS3.10 Dataset. Int. J. Climatol.
**2014**, 34, 623–642. [Google Scholar] [CrossRef][Green Version] - Kobayashi, S.; Ota, Y.; Harada, Y.; Ebita, A.; Moriya, M.; Onoda, H.; Onogi, K.; Kamahori, H.; Kobayashi, C.; Endo, H.; et al. The JRA-55 Reanalysis: General Specifications and Basic Characteristics. J. Meteorol. Soc. Jpn.
**2015**, 93, 5–48. [Google Scholar] [CrossRef][Green Version] - Wang-Erlandsson, L.; Bastiaanssen, W.G.M.; Gao, H.; Jägermeyr, J.; Senay, G.B.; Van Dijk, A.I.J.M.; Guerschman, J.P.; Keys, P.W.; Gordon, L.J.; Savenije, H.H.G. Global Root Zone Storage Capacity from Satellite-Based Evaporation. Hydrol. Earth Syst. Sci.
**2016**, 20, 1459–1481. [Google Scholar] [CrossRef] - Arnold, J.G.; Fohrer, N. SWAT2000: Current Capabilities and Research Opportunities in Applied Watershed Modelling. Hydrol. Process.
**2005**, 19, 563–572. [Google Scholar] [CrossRef] - Kim, N.W.; Chung, I.M.; Won, Y.S.; Arnold, J.G. Development and Application of the Integrated SWAT-MODFLOW Model. J. Hydrol.
**2008**, 356, 1–16. [Google Scholar] [CrossRef] - Nguyen, V.T.; Dietrich, J. Modification of the SWAT Model to Simulate Regional Groundwater Flow Using a Multicell Aquifer. Hydrol. Process.
**2018**, 32, 939–953. [Google Scholar] [CrossRef] - Rafiei, V.; Ghahramani, A.; An-Vo, D.A.; Mushtaq, S. Modelling Hydrological Processes and Identifying Soil Erosion Sources in a Tropical Catchment of the Great Barrier Reef Using SWAT. Water
**2020**, 12, 2179. [Google Scholar] [CrossRef] - Monteith, J.L. Evaporation and Environment. Symp. Soc. Exp. Biol.
**1965**, 19, 205–234. [Google Scholar] - Allen, R.G. A Penman for All Seasons. J. Irrig. Drain. Eng.
**1986**, 112, 348–368. [Google Scholar] [CrossRef] - Allen, R.G.; Jensen, M.E.; Wright, J.L.; Burman, R.D. Operational Estimates of Reference Evapotranspiration. Agron. J.
**1989**, 81, 650–662. [Google Scholar] [CrossRef] - Priestley, C.H.B.; Taylor, R.J. On the Assessment of Surface Heat Flux and Evaporation Using Large-Scale Parameters. Mon. Weather Rev.
**1972**, 100, 81–92. [Google Scholar] [CrossRef] - Hargreaves, G.H.; Samani, Z.A. Reference Crop Evapotranspiration from Temperature. Appl. Eng. Agric.
**1985**, 1, 96–99. [Google Scholar] [CrossRef] - Howell, T.; Evett, S.R. The Penman-Monteith Method; USDA Agricultural Research Service: Bushland, TX, USA, 2001.
- Abiodun, O.O.; Guan, H.; Post, V.E.A.; Batelaan, O. Comparison of MODIS and SWAT Evapotranspiration over a Complex Terrain at Different Spatial Scales. Hydrol. Earth Syst. Sci.
**2018**, 22, 2775–2794. [Google Scholar] [CrossRef][Green Version] - Kannan, N.; White, S.M.; Worrall, F.; Whelan, M.J. Sensitivity Analysis and Identification of the Best Evapotranspiration and Runoff Options for Hydrological Modelling in SWAT-2000. J. Hydrol.
**2007**, 332, 456–466. [Google Scholar] [CrossRef]

**Figure 3.**(

**a**–

**c**) Spatial and (

**d**) temporal variations of ETa from MOD16 A2, SSEBop, and TerraClimate in the study area during the period from 2003 to 2010.

**Figure 4.**Spatiotemporal and temporospatial and model performance represented by (

**a**) the spatial distribution of the NSE and (

**b**,

**c**) the histogram and cumulative plots of NSE, (

**d**) the temporal variation of SPAEF, (

**e**,

**f**) the histogram and cumulative plots of SPAEF. The ideal value of NSE and SPAEF is 1.

**Figure 5.**Spatiotemporal and temporospatial model performance evaluation against different ETa products (

**a**) MOD16 A2 ETa, (

**b**) SSEBop, and (

**c**) TerraClimate. Results are from 2000 model runs, lines are the regression lines and numbers inside the plot are the correlation values. In all correlation plots, the points located closer to the lower-left corner indicate a better model performance.

Nr. | Parameters | Description | Parameter Range | |
---|---|---|---|---|

Min | Max | |||

1 | r_CN2^{FRSD} | SCS runoff curve number of forest (FRSD), agriculture (AGRL), and range grass (RNGE) lands | −0.2 | 0.2 |

2 | r_CN2^{AGRL} | −0.2 | 0.2 | |

3 | r_CN2^{RNGE} | −0.2 | 0.2 | |

4 | v_ESCO^{FRSD} | Soil evaporation compensation factor of forest agriculture, and range grass lands | 0 | 1 |

5 | v_ESCO^{AGRL} | 0 | 1 | |

6 | v_ESCO^{RNGE} | 0 | 1 | |

7 | v_EPCO^{FRSD} | Plant uptake compensation factor of forest agriculture, and range grass lands | 0 | 1 |

8 | v_EPCO^{AGRL} | 0 | 1 | |

9 | v_EPCO^{RNGE} | 0 | 1 | |

10 | v_GWQMN | Groundwater baseflow thereshold (mm) | 0 | 2000 |

11 | v_GW_REVAP | Groundwater “revap” coefficient | 0.02 | 0.2 |

12 | v_REVAPMN | Groundwater “revap” threshold (mm) | 0 | 500 |

13 | r_SOL_AWC^{SOIL1} | Soil available water content of soil classes 1 and 2 | −0.2 | 0.2 |

14 | r_SOL_AWC^{SOIL2} | −0.2 | 0.2 | |

15 | r_SOL_K^{SOIL1} | Soil hydraulic conductivity of soil classes 1 and 2 (mm/h) | −0.2 | 0.2 |

16 | r_SOL_K^{SOIL2} | −0.2 | 0.2 | |

17 | v_CANMX | Maximum canopy storage (mm) | 0 | 5 |

Spatiotemporal Statistic S_{T} (Equation (3)) | Temporospatial Statistic T_{S} (Equation (4)) | ||
---|---|---|---|

Notation | Range and Ideal Value | Notation | Range and Ideal Value |

S_{-NSE} | [−1, ∞) | T_{-SPAEF} | [−1, ∞) |

S_{-NSE} | [−1, ∞) | T_{-NSE} | [−1, ∞) |

S_{RMSE} | [0, ∞) | T_{RMSE} | [0, ∞) |

S_{RSR} | [0, ∞) | T_{RSR} | [0, ∞) |

S_{aBIAS} | [0, ∞) | T_{aBIAS} | [0, ∞) |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Nguyen, T.V.; Uniyal, B.; Tran, D.A.; Pham, T.B.T.
On the Evaluation of Both Spatial and Temporal Performance of Distributed Hydrological Models Using Remote Sensing Products. *Remote Sens.* **2022**, *14*, 1959.
https://doi.org/10.3390/rs14091959

**AMA Style**

Nguyen TV, Uniyal B, Tran DA, Pham TBT.
On the Evaluation of Both Spatial and Temporal Performance of Distributed Hydrological Models Using Remote Sensing Products. *Remote Sensing*. 2022; 14(9):1959.
https://doi.org/10.3390/rs14091959

**Chicago/Turabian Style**

Nguyen, Tam V., Bhumika Uniyal, Dang An Tran, and Thi Bich Thuc Pham.
2022. "On the Evaluation of Both Spatial and Temporal Performance of Distributed Hydrological Models Using Remote Sensing Products" *Remote Sensing* 14, no. 9: 1959.
https://doi.org/10.3390/rs14091959