Estimation of Potential Evapotranspiration in the Yellow River Basin Using Machine Learning Models

Abstract

Potential evapotranspiration (PET) is an important input variable of many ecohydrological models, but commonly used empirical models usually input numerous meteorological factors. In consideration of machine learning for complex nonlinear learning, we evaluated the applicability of three machine learning algorithms in PET estimation in the Yellow River basin (YRB), in addition to determining significant factors affecting the accuracy of machine learning. Furthermore, the importance of meteorological factors at varying altitudes and drought index grades for PET simulation were evaluated. The results show that the accuracy of PET simulation in the YRB depends on the input of various meteorological factors; however, machine learning models including average temperature (T_mean) and sunshine hours (n) as input achieved satisfactory accuracy in the absence of complete meteorological data. Random forest generally performed best among all investigated models, followed by extreme learning machine, whereas empirical models overestimated or underestimated PET. The importance index shows that T_mean is the most influential factor with respect to PET, followed by n, and the influence of T_mean on PET gradually decreased with increased altitude and drier climate, whereas the influence of n shows the opposite trend.

1. Introduction

Evapotranspiration refers to the process by which water on the land surface is converted into water vapor and enters the atmosphere. It is linked to the water cycle, carbon cycle, and energy cycle [1] and is an important part of the energy budget of the earth–air system [2,3]. However, actual evapotranspiration cannot be directly observed in many parts of the world [4,5], and atmospheric evaporation must be represented by other variables. The concept of potential evapotranspiration (PET) provides a convenient metric to estimate the maximum moisture loss from the atmosphere [6]. PET is an indispensable and important input variable used in many rainfall–runoff and ecosystem models [7,8,9,10] for estimation of the drought severity index [1,11,12] and assessment of climate change impacts [13], in addition to many other uses. Therefore, it is particularly important to accurately evaluate PET in different watersheds.

There have been more than 200 years of research on evapotranspiration theory and models. To date, 50 empirical models have been developed that can be used to calculate PET [14] according to a physical mechanism. However, these models often provide inconsistent results due to differing assumptions and input data. In general, these models can be divided into four categories, namely mass-transport-based, temperature-based, radiation-based, and comprehensive models [10]. Comparative studies show that the calculation results obtained by diverse empirical models in the same watershed vary considerably [6,13,15]. Among the four types of models, the results obtained with mass-transport-based methods are highly biased, as the theoretical basis of most methods differs considerably from the current understanding of PET [16]. The most widely used temperature-based method is the Hargreaves–Samani model (H-S), the most widely used radiation-based method is the Priestley–Taylor model (P-T), and the Penman synthesis method (PM) is the typical comprehensive mode. Among them, the most effective method for estimating PET is internationally recognized as the Penman–Monteith (FAO-PM) model recommended by the Food and Agriculture Organization of the United Nations (FAO), which has been proven to accurately estimate the value of PET under varying conditions [17,18]. However, because the FAO-PM model requires extensive and detailed meteorological data, the application this model is limited [19,20].

In recent decades, with the widespread application of machine learning and its advantages with respect to learning complex, nonlinear relationships, algorithms such as artificial neural networks, support vector machine, and random forest have also been applied to various processes in the water cycle, such as runoff simulation [21], rainfall simulation [22], drought index prediction [23], etc. Machine learning is also frequently used in PET modeling [24,25], and researchers have conducted many studies to compare the regional applicability of various models [26,27,28,29,30]. Research results show that using machine learning to estimate watershed PET provides satisfactory accuracy without complete meteorological data input. However, the accuracy of the same machine learning algorithm is not consistent across climate regions, and previous studies have not reached an agreement on the optimal algorithm for estimating PET. In addition, due to the lack of complete meteorological data, studies have employed various meteorological factors to establish models. Although scholars have also studied the impact of meteorological factors on PET estimation, these conclusions are not fully applicable in the Yellow River basin due to geographical location and climatic conditions. Therefore, in this study, we will compare and analyze the applicability of support vector regression, random forest, and extreme learning machine in the Yellow River basin.

In the context of global warming, the global climate system is undergoing rapid and widespread changes [31]. Climate change is expected to increase the frequency and intensity of extreme events, such as extreme rainfall erosivity [32], drought [33], and heatwave events [34], whereas the Yellow River basin in arid and semi-arid areas is more likely to show a significant drought trend in terms of area, frequency, and severity [35]. The Yellow River is the mother river of the Chinese nation. Not only does the total population of the province in which the river basin is located account for 30.3% of the country’s total population but it is also the main producing area of China’s agricultural products and plays a very important role in China’s economic and social development and ecological security [36]. With a vast territory and complex and diverse topography, the precipitation time in the basin is concentrated, with a difference between the northern and the southern regions [37]. Studies have shown that 90% of the surface precipitation in the arid region returns to the atmosphere in the form of water vapor [38]. Therefore, there is an acute contradiction between water supply and demand in the YRB [39]. Therefore, when using machine learning for PET modeling in the YRB, it is necessary to consider not only the importance of meteorological factors at different geographical locations but also the importance of meteorological factors for PET modeling under various drought conditions.

Based on the above content, the purpose of this study is to Section 1 determine the optimal combination of input meteorological factors when using machine learning for PET modeling in the YRB Section 3.1, Section 2 analyze and compare the results of machine learning and empirical models from both qualitative and quantitative perspectives Section 3.2 and Section 3.3 and discuss the consistency with previous studies Section 4.1, Section 3 determine the factors that significantly affect the accuracy of PET estimates Section 4.2, and Section 4 assess the importance of meteorological factors for PET simulation at different geographical locations under meteorological droughts Section 4.3.

2. Materials and Methods

2.1. Study Area

The Yellow River basin (YRB) originates from the Qinghai-Tibet Plateau in Qinghai Province and flows through 9 provinces, with a total length of 5464 km before finally flowing into the Bohai Sea via Shandong. The Yellow River (32°–42° N, 95°–119° E) is the fifth longest river in the world and the second longest river in China, with a total drainage area of 7.95 × 10⁵ km² [40]. The northwest of the YRB is an arid and semi-arid continental monsoon climate, and the southeast is a semi-humid environment [41]. The average annual temperature is about 4–14 °C, which varies with latitude and altitude [42]. The annual precipitation in the YRB generally occurs more in the south and less in the north, gradually decreasing from southeast to northwest, with an average annual precipitation of 461 mm.

In order to explore the “dynamic changes” in PET in different regions, the entire YRB was divided into 8 sub-basins (Figure 1) according to the Code of Practice for Water resources bulletin (National Standard of the People’s Republic of China) as follows: I. watershed above Longyangxia, II. Longyangxia-Lanzhou watershed, III. Lanzhou-Hekou Town watershed, IV. Hekou Town-Longmen watershed, V. Longmen-Sanmenxia watershed, VI. Sanmenxia-Huayuankou watershed, VII. watershed below Huayuankou, and VIII. Neiliu District. For the convenience of description, the YRB was divided as follows: upper reaches, above Hekou Town; middle reaches, Hekou Town-Huayuankou; and lower reaches, below Huayuankou [43]. The data of each sub-basin were obtained by averaging the data of stations in and around the basin, and the data of the entire YRB was calculated as the average of the data of the 115 stations.

2.2. Empirical Models for Estimation of PET

The International Food and Agriculture Organization of the United Nations (FAO) recommends using the Penman–Monteith model (FAO-PM) as the most effective method to estimate PET. This method was also selected as the standard in this study. The formula is as follows [17]:

$$E{T}_o=\frac{0.408\Delta \left(R_n-G\right)+\gamma \frac{900}{T_{mean}+273}{u}_2\left(e_s-e_a\right)}{\Delta +\gamma \left(1+0.34u_2\right)}$$

(1)

where $E{T}_o$ is the reference evapotranspiration (mm day⁻¹), $R_n$ is the net radiation (MJ m⁻² day⁻¹), $G$ is the soil heat flux density (MJ m⁻² day⁻¹), T_mean is the average daily air temperature (°C), $u_2$ is the wind speed at 2 m height (m s⁻¹), $e_s$ is the saturation vapor pressure (kPa), $e_a$ is the actual vapor pressure (kPa), ($e_s-e_a$) is the saturated vapor pressure difference (kPa), $\Delta$ is the vapor pressure curve slope (kPa °C⁻¹), and $\gamma$ is the psychrometric constant (kPa °C⁻¹).

The three empirical models selected for this study are the temperature-based Hargreaves–Samani model [44], the radiation-based Priestley–Taylor [45], and the Penman synthesis model [46]. Temperature-based methods are the most widely used, owing to their simple formulation and minimal requirements for data input [7]. The radiation-based method is mainly used to estimate area-scale PET, providing superior calculation results than the temperature-based method in some areas, as the temperature-based method only considers the influence of a single factor in PET calculation [11,47]. The Penman model of the comprehensive method is the predecessor of the FAO-PM model. In the FAO-PM method, Monteith added a new coefficient (rs) based on the Penman equation to represent the crop surface roughness. The formulae and required data for the three empirical models are shown in

Table 1. Empirical models and formulae for calculating PET.

Method	Category	Abbreviation	Formulation
Hargreaves–Samani	Temperature-based method	H-S	$PET=\frac{{\left[0.0023R_a(T_{mean}+17.8)(T_{\max }-T_{\min })\right]}^{0.5}}{\lambda }$	(2)
Priestley–Taylor	Radiation-based method	P-T	$PET=\frac{1.26\left(R_n-G\right)}{\lambda }\frac{\Delta }{\Delta +\gamma }$	(3)
Penman	Combination method	PM	$PET=\frac{\Delta \left(R_n-G\right)}{\lambda \left(\Delta +\gamma \right)}+\frac{\gamma }{\Delta +\gamma }\frac{6.43\left(1+0.53u_2\right)\left(e_s-e_a\right)}{\lambda }$	(4)

Note: $PET$ is potential evapotranspiration (mm day⁻¹), R_a is extraterrestrial radiation (MJ m⁻² day⁻¹), T_max is daily maximum temperature (°C), T_min is daily minimum temperature (°C), and $\lambda$ is latent heat of vaporization (kPa °C⁻¹).

.

2.3. Machine Learning Algorithms

Because the FAO-PM model required detailed daily meteorological data, its application is limited. A machine learning algorithm can obtain satisfactory PET estimates without complete meteorological data. The following three machine learning algorithms were selected because they are usually computationally efficient and are good at learning complex nonlinear relationships.

Support vector regression (SVR) is a kernel-based supervised learning algorithm proposed by Vapnik [48]. Not only can it handle classification problems, but it is also often used to solve regression problems in fields such as meteorology, hydrology, and environmental science [49]. There are many options for the kernel function form of SVR, including linear kernel function, polynomial kernel function, radial basis kernel function, etc. The radial-based kernel function was selected in this study. Detailed principles of SVR are described by Vapnik [48]. Furthermore, the grid search method was used to determine the optimal parameters in support vector regression and random forest. The grid search method involves determining optimal parameter values by traversing different parameter combinations within a given range.

Random forest (RF) was improved by Breiman based on the bagging model, which is a tree-based supervised ensemble learning model. Similar to SVR model application scenarios, RF models can be used for prediction and regression problems and can provide high-accuracy predictions without overfitting the dataset [50]. Important parameters in the model include the number of trees and the eigenvalues used by each tree. The RF model can also determine the relative importance of meteorological factors with respect to potential evapotranspiration. The indicator selected in this study is %IncMSE, i.e., increase in mean squared error. The working principle of the index is to assign a value to each input factor. If a variable is important, the model’s error will increase after its value is replaced [28,51]. Therefore, variables with higher values are more important.

Extreme learning machine (ELM) is a relatively new machine learning algorithm proposed by Huang et al. [52]. This method was developed from a neural network with a single hidden layer. Compared with traditional neural networks, it has the advantages of simple structure, fast learning speed, random generation of the number of nodes in the hidden layer and its corresponding parameters, and no need to adjust [53], which are widely used in regression, classification, and prediction. The type of activation function used in this study is the same as that of SVR, both of which are radial basis functions, and the number of hidden neurons is 20.

The daily data used in this study include the mean temperature (T_mean), maximum and minimum temperature (T_max and T_min), relative humidity (RH), wind speed (uz), and sunshine hours (n) from 1969 to 2018. Becuase there are few missing values and abnormal values in the data, we used the average correction method to process these values, that is, the average of the two observed values before and after were used correct the abnormal value. The dataset from 1969 to 2018 was used for calculation in the empirical models. When using machine learning algorithms to estimate potential evapotranspiration, all data (meteorological and PET (FAO-PM) data) were first divided into training (80%, 1969–2008) and testing periods (20%, 2009–2018). Then, all training period data were used for machine learning modeling, the meteorological data in the testing period were used for prediction, and the PET data in the testing period were used to verify the accuracy of the simulation results.

2.4. Evaluation Indicators

In order to evaluate the simulation effect of the machine learning model, four evaluation indicators were selected: coefficient of determination ($R^2$) to judge the fitting degree of the regression equation, Nash–Sutcliffe efficiency ($NSE$), and Kling-Gupta efficiency ($KGE$) [54,55,56], which fully consider the three components of the error of the Nash–Sutcliffe efficiency model and the percentage deviation ($PBIAS$, %). The accuracy of machine learning was evaluated according to the following formulae:

$$R^2=\frac{{\displaystyle \sum _{\mathrm{i}=1}^n{\left[\left({\widehat{y}}_i-{\mu }_{\widehat{y}}\right)(y_i-{\mu }_y)\right]}^2}}{{\displaystyle \sum _{i=1}^n{(y_i-{\mu }_y)}^2{\displaystyle \sum _{i=1}^n{\left({\widehat{y}}_i-{\mu }_{\widehat{y}}\right)}^2}}}$$

(5)

$$NSE=1-{\displaystyle \sum _{i=1}^n{\left({\widehat{y}}_i-y_i\right)}^2}/{\displaystyle \sum _{i=1}^n{\left(y_i-{\mu }_y\right)}^2}$$

(6)

$$KGE=1-\sqrt{{\left(\alpha (y,\widehat{y})-1\right)}^2+{\left(\beta (y,\widehat{y})-1\right)}^2+{\left(\gamma (y,\widehat{y})-1\right)}^2}$$

(7a)

$$\beta (y,\widehat{y})={\sigma }_{\widehat{y}}-{\sigma }_y$$

(7b)

$$\gamma (y,\widehat{y})=\frac{C{V}_{\widehat{y}}}{C{V}_y}=\frac{{\sigma }_{\widehat{y}}/{\mu }_{\widehat{y}}}{{\sigma }_y/{\mu }_y}$$

(7c)

$$PBIAS=\left({\displaystyle \sum _{i=1}^n\left({\widehat{y}}_i-y_i\right)}/{\displaystyle \sum _{i-1}^n{y}_i}\right)\times 100\%$$

(8)

where $y_i$ is the observed value of the sample, ${\widehat{y}}_i$ is the simulated value of the model, ${\mu }_y$ is the mean observed value, ${\mu }_{\widehat{y}}$ is the mean simulated value, $n$ is the sample size, $\alpha (y,\widehat{y})$ is the linear correlation coefficient, $\beta (y,\widehat{y})$ is the deviation ratio, ${\sigma }_y$ is the standard deviation of the observed value, $\gamma (y,\widehat{y})$ is the coefficient of variation, and ${\sigma }_{\widehat{y}}$ is the standard deviation of the simulated values.

The root mean square error ($RMSE$) was used to measure the deviation between the predicted value of the model and the true value. The smaller the value, the smaller the deviation. In this study, RMSE was used to evaluate the stability of the machine learning model. The calculation formula is as follows:

$$RMSE=\sqrt{{\displaystyle \sum _{i=1}^n{(y_i-{\widehat{y}}_i)}^2}/n}$$

(9)

The model results were evaluated using the corrected AIC ($AICc$), which considers both the computational accuracy of the model and the number of factors used for model computation. The calculation formula of the above indicators is as follows:

$$AICc=\ln \left(SSE/n\right)+\left(n+k\right)/\left(n-k-2\right)$$

(10)

where $SSE$ is the residual sum of squares, and $k$ is the number of model variables.

In order to eliminate the influence of different levels of data, the meteorological data need to be normalized before establishing a machine learning model as the input of the model. The normalized data formula is as follows:

$$x^{\prime }=\frac{x-\min \left(x\right)}{\max \left(x\right)-\min \left(x\right)}$$

(11)

The drought index (D) is the ratio of potential evapotranspiration to precipitation, which is an index used to measure the degree of regional meteorological drought. The formula has obvious advantages in studying climate change and drought change [57]:

$$\mathrm{D}=\frac{\overline{PET}}{\overline{P}}$$

(12)

where $\overline{PET}$ is the multiyear average PET from 1969 to 2018 (calculated by FAO-PM), and $\overline{P}$ is the precipitation during the same period.

3. Results

3.1. Performance Evaluation of Machine Learning Models

In this section, the PET estimated from recommended FAO-PM model are taken as reference values, the performance and stability of the machine learning model in the training and testing periods under different scenarios are compared, and the optimal scenario for the machine learning model is selected.

3.1.1. Machine Learning Model Input Scenario

The input scenarios of the machine learning model were determined according to the magnitude of the nonparametric Spearman correlation coefficient between the daily PET and the input factors, i.e., the average daily air temperature (T_mean, °C), daily average relative humidity (RH, %), wind speed at 10 m height (uz, m s⁻¹), and sunshine hours (n, h). First, the Spearman correlation coefficient between the daily PET in the YRB and each of the input factors was determined (Figure 2), and the factor with the largest correlation coefficient relative to daily PET was selected as Scenario 1. Scenarios 2–4 were determined by adding the next highest correlation factor on the basis of Scenario 1. The final results are shown in

Table 2. Input scenarios for machine learning models.

Input Scenario	Model Input Factor(s)
Scenario 1	Tmean
Scenario 2	Tmean + n
Scenario 3	Tmean + n + uz
Scenario 4	Tmean + n + uz + RH

.

3.1.2. Performance of the Machine Learning Model during Training and Testing Periods

To evaluate the accuracy of the training and testing period models, according to the selected four scenarios, the evaluation indices (R², NSE, and KGE) of three machine learning models in eight sub-basins were calculated under each scenario. Each box plot was generated based on the evaluation results of each model in 115 meteorological stations, as shown in Figure 3 and Figure 4.

According to the results of the training period (Figure 3), the performance of every model continued to improve with increased input data. No significant differences were observed in terms of accuracy between the machine learning models under each input scenario. According to the results of the testing period (Figure 4), the evaluation indicators of the three models fluctuated considerably with increased input data, with outliers in multiple sub-basin stations, indicating that the model became less robust with increased input variables when simulating different meteorological stations, possibly because the model was overfit with a large number of input factors during the training period [58].

A comprehensive comparison of the performances of the SVR, RF, and ELM models in the training and testing periods under the four scenarios shows that Scenario 2 is optimal, taking the average temperature and sunshine hours as the input variables of PET. The accuracy of Scenario 2 during the testing period was better than that of Scenario 1, with fewer abnormal data point in the simulation results than in Scenarios 3–4. The models performed well for Scenario 2, with a median KGE of more than 0.82 at 115 stations.

3.1.3. Stability Evaluation of Machine Learning Models

To evaluate the stability of the model, the absolute change in RMSE during the testing period was compared to the training period under different climate scenarios for the three machine learning models, as previously reported by Huang et al. [27]. The smaller the absolute value of RMSE, the better the stability of the model.

According to analysis from the perspective of four climate scenarios (Figure 5), the sum of RMSE absolute values in the four scenarios of SVR, RF, and ELM models were 6.66, 18.37, 61.05, 90.61 (SVR); 6.33, 12.51, 50.88, and 64.98 (RF) and 6.13, 20.23, 75.57, and 90.89 (ELM), respectively. The results show that the stability of the three models decreased with increased input factors, and the stability of Scenario 3 differed considerably from that of Scenario 2. In the scenarios in which the input factor is greater than or equal to 2, the RF model performs best, with the strongest stability among the three investigated models.

According to the results of the eight sub-basins in the hydrological division among the three scenarios except Scenario 1, the median variation in RMSE in sub-basin VII was the smallest among other sub-basins. Although the stability of the model in Scenario 2 was worse than that in Scenario 1, the value of RMSE between the testing and training periods of each sub-basin varied from −0.21 to 0.77, and the stability results were acceptable. The RMSE value in Scenario 3 varied from −0.09 to 2.22, with poor stability results. Therefore, the simulated data in Scenario 2, with T_mean and n as input variables of PET, were selected for the three machine learning models in the subsequent comparative analysis. In this scenario, the accuracy of the model simulation was balanced by the number of input factors.

3.2. Spatiotemporal Variation of PET

In this section, we qualitatively analyze the trend changes of empirical and machine learning models under Scenario 2 on month and annual scales (Figure 6). The average statistical results for data from the same month data in all years are shown in Figure 6a. The results of different models show that PET in the YRB exhibits a significant intra-annual change trend, and its value reaches the maximum in summer (June, July, and August), accounting for 40–45% of the total PET during the year, followed by spring, autumn, and winter. In the empirical models, the monthly average results of the temperature-based H-S model fluctuated in the FAO-PM model, that is, in the summer with higher temperature, the simulation results were higher than the FAO-PM results, with increased deviation with lower temperatures in the remaining seasons. The radiation-based P-T model overestimated PET, but the calculation results of the model in winter (December, January, and February) were consistent with those of the FAO-PM model. The PM model based on the comprehensive method considerably overestimated all monthly statistics. The results of the three machine learning models were most consistent with the recommended method. Compared with FAO-PM, the simulation results presented lower values for February to June, with smaller overestimation in other months.

The multiyear average PET results (Figure 6b) show a fluctuating trend in the YRB (FAO-PM). In the empirical models, the H-S model underestimated throughout the study period, the P-T and PM models considerably overestimate, and the annual value results differed considerably from those obtained using the recommended method. In the machine learning models, the simulation results of RF and ELM performed well during the training period (1969–2008), and the results for multiple periods were close those obtained with the FAP-PM model. Although the SVR simulation results presented with low values, they also captured the trend in the data. In the testing period (2009–2018), the simulation results of the machine learning models were lower than those in the training period.

According to the above analysis, with respect to the overall change trend, the empirical and machine learning models showed a similar trend compared with the standard FAO-PM model. However, the P-T and PM are numerically overestimated, and the H-S model underestimated. Although the results of the three machine learning models were consistent with those obtained with the FAO-PM model, the training period results of the machine learning model were better than those in the testing period.

The trends (Mann–Kendall) of annual PET values from 1969 to 2018 in 115 stations were analyzed, and we applied Kriging spatial interpolation of multiyear average PET in the YRB. The results are shown in Figure 7. PET in the YRB is high in the east and low in the west and gradually decreases from the east to the west. Among the 115 stations, the number of stations showing a significant increase (p < 0.05) accounted for 32.2% of the total number of stations, and most of these stations are located in the upper YRB; most of the stations that showed a significant decrease are located in the lower YRB.

3.3. Comparative Analysis of Machine Learning Models and Empirical Models

In order to further verify the applicability of machine learning, in this section, we compare the results of machine learning models under Scenario 2 with those obtained using the three empirical models. In the comparative analysis, PBIAS and AICc were used for evaluation (

Table 3. Evaluation indicators of daily PET in machine learning model and empirical models from 1969 to 2018.

Sub-Basin	Evaluation Indicator	SVR	RF	ELM	H-S	P-T	PM
I	PBIAS	−1.95	−1.05	−1.30	5.90	29.90	35.90
	AICc	−2.17	−2.13	−2.16	0.51	1.77	0.59
II	PBIAS	−4.80	−2.90	−3.80	−0.25	32.80	31.70
	AICc	−1.47	−1.53	−1.35	3.32	2.15	0.72
III	PBIAS	−4.90	−1.00	−1.80	−13.70	3.80	26.10
	AICc	−1.13	−1.02	−1.21	0.63	1.59	0.81
IV	PBIAS	−6.15	−1.50	−2.10	−8.30	14.55	26.30
	AICc	−1.07	−1.17	−1.18	0.38	1.98	0.76
V	PBIAS	−5.00	−1.80	−3.10	0.40	27.00	29.00
	AICc	−1.33	−1.40	−1.36	0.17	1.74	0.74
VI	PBIAS	−7.7	−3.6	−6.1	−0.60	16.20	26.70
	AICc	−0.54	−0.97	−0.25	0.28	1.60	0.78
VII	PBIAS	−2.95	0.60	0.40	0.15	7.55	28.30
	AICc	−1.41	−1.42	−1.43	0.25	1.13	0.88
VIII	PBIAS	−5.80	−1.10	−2.2	−15.55	8.20	24.60
	AICc	−1.06	−1.08	−1.16	0.80	1.92	0.76

Note: Light blue indicates the model with the smallest deviation in the sub-basin, and light orange represents the AICc value of the model with best performance in the sub-basin.

). PBIAS (%) was used to describe the deviation of the model simulation results from the true value. Because the types and quantities of meteorological factors input by various PET empirical models and machine learning models differed considerably, we calculated the AICc value, and the number of input factors was considered when comparing the accuracy of the models.

According to the PBIAS values, the three machine learning models all minorly underestimated in the entire study area, and RF performed best among the three models. The simulation results of the radiation-based P-T and synthesis method PM considerably overestimated in most sub-basins. Whether the simulation results of the temperature-based H-S model were overestimated is related to the simulation area. Based on the results of the whole research period, considering the deviation of the model results (PBIAS) and the number of model simulation accuracy and variables (AICc), the machine learning models were generally better than the empirical models, and the RF and ELM models achieved the best performance, indicating that the machine learning models have strong applicability for PET simulation in the YRB.

The analysis results of the above evaluation indices are consistent with the results of qualitative analysis (Figure 6), with the results of the evaluation indicators demonstrating the performance of the model in different sub-basins of the YRB in more detail. Considering the simulation accuracy of the model and the number of variables, the three best-performing models were RF > ELM > H-S.

4. Discussion

4.1. Comparative Analysis of PET Calculation Models

The empirical and machine learning models were qualitatively and quantitatively compared. According to the research results in the YRB, due to the strong regional applicability of the empirical models, both the P-T and PM models considerably overestimated, whereas the H-S model fluctuated with respect to the FAO-PM results. These results are consistent with the development conditions of the original model. Studies have shown that the radiation-based P-T model was developed for a warm and humid climate [6], which explains the poor performance of the method in the arid and semi-arid YRB. The PM method computes the PET for an open water surface, so it usually obtains the highest value [1]. The temperature-based H-S model was originally developed for dry climates [59], which is consistent with the FAO’s suggestion that the H-S model can be used to estimate PET in areas with only the lowest and highest temperatures [17]. Studies have shown that H-S models often overestimate or underestimate PET [60], so they are more suitable for the estimation of PET in the YRB than the P-T and PM models.

According to the results obtained in the study area, the machine learning models are more regionally applicable than the empirical models and require less meteorological input to achieve satisfactory model accuracy. As the most commonly used machine learning algorithm, the SVR model performed poorly in this study, although previous studies using this method in different study areas resulted in to the opposite conclusion [27,61]. The RF in the six models showed overall performance [62], indicating that the RF model in the YRB is more applicable with fewer overfitting problems [46]. In a commercial farm study in Florida, Granata [61] found that RF provided the best predictions with a limited number of input variables (net solar radiation, wind speed, mean relative humidity, and mean temperature). The ELM model is slightly less accurate than the RF model but performed better in some sub-basins in this study and was proven efficient in previous studies [63]. Researchers strongly recommend using this algorithm in arid and semi-arid areas.

According to the above analysis, the machine learning models are generally better than the empirical models [58]. Furthermore, none of the six methods selected in this study achieve optimal results in all cases [53], and there is no scientific consensus with respect to the optimal algorithm to estimate PET. Therefore, selecting a calculation model without comparison of PET in the Yellow River Basin will result in a considerable error in the research results.

4.2. Factors Affecting the Accuracy of Machine Learning Estimation

Analysis of variance (ANOVA) can be used to test which factors have significant effects on the results. In this study, ANOVA was used to test which factors, input scenarios, machine learning models, or their interaction exerted significant effects on the accuracy of PET simulation.

Table 4. ANOVA results for scenarios and models.

	SS	df	MS	F Value	p-Value	F Crit
Scenarios	0.1654	3	0.0551	31.0516	6.15 × 10−6	3.4903
Models	0.0131	2	0.0065	3.6784	0.0568	3.8853
Scenarios and models	0.0219	6	0.0036	2.0540	0.1360	2.9961
Inside	0.0213	12	0.0018
Total	0.2217	23

Note: SS, sum of squares; df, degrees of freedom; MS, mean square.

shows the ANOVA results. The F crit (3.4903) of scenario factors at a 95% confidence level was lower than the F value (31.0516) calculated for different scenarios, whereas the F crit (3.8853) of model factors at a 95% confidence level was higher than the F value (3.6784) calculated by different models. The F crit (2.9961) of the interaction between scenarios and models at the 95% confidence level was higher than F value (2.0540). According to the above analysis results, the accuracy of PET simulation in the YRB depends on the input scenarios of different meteorological factors.

Our results are consistent with those reported in previous studies [61,64], indicating that the performance of machine learning models varies depending on the number of inputs and the predicted length of time. Generally, the accuracy of models with complete meteorological factors is better than that of models with incomplete input data [28,65]. However, due to economic conditions, geographical location, and other factors, meteorological stations in some areas do not collect data on all meteorological factors. In such cases, in order to use machine learning algorithms to simulate local PET, the accuracy of the input factors can be controlled the factors that are important for PET simulation can be selected as inputs to control the simulation results of the model [27,66,67].

Using machine learning to simulate PET in the YRB, the model’s accuracy depends on the input of meteorological factor data. A lack of the most important factor data in PET simulation results in a significant decline in model performance. Therefore, the appropriate input factors should be selected when using machine learning for PET modeling.

4.3. Importance of Meteorological Factors of Altitude and Drought Index for PET Predictions

With respect to the importance of meteorological factors in PET simulation at different altitudes and drought indices in the YRB, the altitudes and drought indices in the YRB were divided into deciles. Figure 8 shows the proportion of importance of four meteorological factors in PET simulation under 10 altitude (left) and drought index (right) grade intervals. The results show that T_mean accounted for the largest proportion, followed by n, and the sum of the two accounted for 82–88% of the total importance index [68]. Previous studies revealed that temperature has a considerable impact on PET in the constructed PET model [69]. Zhao et al. [70] also found that the contribution rate of n to PET was second only to T_mean in multiple climatic zones in China.

The importance of T_mean shows a decreasing trend with increased altitude, whereas that of n shows the opposite trend. The importance of uz and RH shows a small downward and upward trend with increased altitude, indicating that the importance of the T_mean factor in the simulation of PET in regions with high altitudes (source region of the Yellow River) is lower than that in regions with relatively low altitudes (lower Yellow River). The n factor is more important in higher altitude areas [71].

Previous studies mostly discussed the relationship between PET and drought [72,73], as well as the impact of climate change on drought [70,74]. In this study, we focused on the degree of influence of meteorological factors on PET simulation at different drought index stations (Figure 8, right). The importance of T_mean exhibits an increasing tendency with an increase in the drought index, whereas the importance of n decreases, indicating that the importance of T_mean is higher in dry areas than in humid areas, and the importance of n is opposite. For uz and RH, with a total importance proportion of less than 20%, uz shows a small upward trend with increased drought index, whereas the importance of RH changed little in different drought index levels.

According to the above analysis, the changing trend of the same meteorological factor differs depending on the altitude and drought index grade interval, although the overall importance ranking is unchanged, namely T_mean > n > RH > uz. The results of the importance index differ from those of the Spearman correlation coefficient, as the simulation of PET is a complex, nonlinear calculation process [65,75].

5. Conclusions

(1)

Average temperature (T_mean) and sunshine hours (n) can be used as input combinations of the model to obtain satisfactory daily potential evapotranspiration (PET) predictions when lacking complete meteorological variables and using machine learning to estimate PET in the Yellow River basin (YRB).

(2)

A comparison of the selected machine learning and empirical models shows that machine learning models limited to two input variables generally perform better than empirical models, and the empirical models usually overestimate or underestimate. Among the six models, random forest (RF) performs best in general, followed by extreme learning machine; none of the six methods selected in this study are optimal for prediction in all watersheds.

(3)

Analysis of variance results show that the accuracy of PET simulation in the YRB depends on the input scenario of different meteorological factors at the significance level of 0.05 relative to machine learning models.

(4)

According to the importance index provided by RF simulation, T_mean is the most important factor, followed by n. However, the influence of T_mean on PET gradually decreases with increased altitude and gradually increases with a drier climate, whereas the influence of n shows the opposite trend. The importance of relative humidity is higher than that of wind speed, in contrast to the ranking calculated by the Spearman correlation coefficient with PET, indicating that the simulation of PET is a complex, nonlinear calculation process.

The results of this study will help researchers who select appropriate meteorological factors to calculate PET in the YRB and provide a reference for researchers to simulate or evaluate the performance of different models. Furthermore, under global warming climate conditions, decision makers can use these results to manage agricultural water resources in the YRB. In this research, only three commonly used machine learning algorithms were selected for comparison, and a comparative study of multiple algorithms should be conducted in the future.