Short-Term Probabilistic Forecasting Method for Wind Speed Combining Long Short-Term Memory and Gaussian Mixture Model

Abstract

Wind speed forecasting is advantageous in reducing wind-induced accidents or disasters and increasing the capture of wind power. Accordingly, this forecasting process has been a focus of research in the field of engineering. However, because wind speed is chaotic and random in nature, its forecasting inevitably includes errors. Consequently, specifying the appropriate method to obtain accurate forecasting results is difficult. The probabilistic forecasting method has considerable relevance to short-term wind speed forecasting because it provides both the predicted value and the error distribution. This study proposes a probabilistic forecasting method for short-term wind speeds based on the Gaussian mixture model and long short-term memory. The precision of the proposed method is evaluated by prediction intervals (i.e., prediction interval coverage probability, prediction interval normalized average width, and coverage width-based criterion) using 29 monitored wind speed datasets. The effects of wind speed characteristics on the forecasting precision of the proposed method were further studied. Results show that the proposed method is effective in obtaining the probability distribution of predicted wind speeds, and the forecast results are highly accurate. The forecasting precision of the proposed method is mainly influenced by the wind speed difference and standard deviation.

1. Introduction

Wind speed forecasting is extremely relevant to disaster prevention [1]. Accordingly, many wind speed prediction and warning systems have been installed to alert running trains to the danger of extreme winds, such as along high-speed railway lines [2,3], long-span bridges, wind power stations, and offshore platforms. However, owing to the chaotic and random wind speed fluctuations, the precise prediction of wind speeds is difficult to achieve [1].

Several prediction methods have been proposed to provide accurate wind speed forecasts. These methods can be classified into physical, statistical, intelligent, and combined categories [4]. Physical methods predict wind speeds using numerical weather prediction (NWP) and atmospheric data [5]. Yang et al. [6] proposed the implementation of a new simulation ensemble method using NWP to improve the wind speed prediction of storms. Wang et al. [7] proposed an NWP wind speed sequence transfer correction algorithm that improves the input and output of the correction model by introducing sequence transfer relations. Although physical methods can achieve excellent performance in wind speed prediction, their implementation in short-term wind speed forecasting is difficult because they are time-consuming and expensive. In contrast, statistical methods compared with physical methods are simpler in terms of methodological complexity [1]. For the reason that statistical methods use a considerable amount of historical data for prediction, they offer more advantages in terms of time consumption [8]. Typical statistical methods include the autoregressive sliding average model (ARMA) [9], the autoregressive integrated moving average (ARIMA) model [10], the ARIMA with exogenous variables model [11], and the Kolmogorov–Zurbanko filter model [12]. Huang et al. [13] developed the ARMA–generalized autoregressive conditional heterogeneity model to analyze and calculate the time-varying standard deviation of non-stationary wind speed accurately. Ouarda et al. [14] introduced a wind speed predictor (an atmospheric circulation index) to a non-smooth statistical model to explain interannual variability. Jeong et al. [15] developed a statistical post-processing method to improve the accuracy of the NWP. Galanis et al. [16] proposed and tested a hybrid optimization technique using Bayesian modeling combined with a nonlinear Kalman filter. The method not only reduces systematic bias but also supports significant limitations on error variability and associated prediction uncertainty. However, statistical methods have certain inadequacies, such as the low prediction accuracy of low-order models and the difficulty in estimating the parameters of high-order models for the autoregressive model [17]. Statistical models are prone to falling into local optima in the nonlinear time-series forecasting of wind speed series with stochasticity [18].

With the development of machine learning technology, artificial neural networks [19], extreme learning machines (ELM) [20], and support vector machines (SVM) [21] have been widely used in wind speed forecasting. Kumar et al. [22] proposed a method for wind speed and power prediction using a nonlinear autoregressive network with exogenous inputs and SVM, enabling the selection of the most appropriate data segment from available data. In recent years, deep neural networks, such as long short-term memory (LSTM) [23], convolutional neural networks [23], and gated recurrent units [24], have also been used for wind speed forecasting [25,26]. The performance of single models is limited, and they remain incompatible with the chaotic and stochastic nature of wind speeds [18]. Combined models [27] are applied to wind speed prediction and are presumed to be combinations of modal decomposition, prediction methods, and parameter optimization that typically provide highly accurate predictions. Zhang et al. [28] proposed a hybrid model based on variational mode decomposition (VMD)–wavelet transform (WT) and a principal component analysis–back propagation–radial basis function neural network. The model can decompose a nonstationary wind speed series into several relatively stationary intrinsic mode functions (IMFs) by VMD in the data preprocessing stage. Moreover, it can predict and reconstruct each IMF separately, greatly improving the accuracy of short-term wind speed prediction. Liu et al. [29] proposed a hybrid forecasting system that predicted wind speeds 24 h before occurrence. Furthermore, they optimized the parameters of a recurrent neural network (RNN) containing long-term and short-term memory frameworks using ant colony optimization and genetic algorithms to improve the stability of the prediction. Bahrami et al. [30] proposed a hybrid model based on the WT and gray models. In this model, the WT technique removes high-frequency components from the original data and improves the prediction accuracy. Li et al. [31] proposed a hybrid model using data decomposition, LSTM network prediction, regularized ELM network prediction residual modeling, and inverse empirical WT (IEWT) reconstruction. Furthermore, they demonstrated that IEWT can improve the wind speed prediction accuracy and stability of LSTM. Wang et al. [32] proposed a hybrid model consisting of feature selection, prediction, system optimization, and system evaluation modules to complement existing research in this area. Li et al. [33] preprocessed data using a hybrid decomposition method that coupled the ensemble patch transform and complete ensemble empirical mode decomposition with adaptive noise. Furthermore, they used temporal convolutional networks (TCNs) for prediction to achieve improved accuracy and stability of results.

However, because wind speeds are chaotic and naturally random, forecasting inevitably generates errors [1]. The results of the aforementioned models complicate the formulation of appropriate instructions to achieve accurate prediction results [34,35]. A probabilistic forecasting method is required for short-term wind speed forecasting [36] because it provides both a forecasting value and an error distribution. Probabilistic prediction methods based on conditional probability [37] and multivariate probability theory [38] have been proposed mainly for wind power prediction error analysis. The probabilistic prediction of wind power is achieved by calculating the prediction interval or distribution function of the wind speed prediction error.

In this study, a probabilistic prediction method for short-term wind speed based on LSTM and a Gaussian mixture model (GMM) is proposed. The precision of the proposed method is evaluated by prediction intervals, as indicated by the prediction interval coverage probability (PICP), prediction interval normalized average width (PINAW), and coverage width-based criterion (CWC) using the monitored wind speed. In addition, a multiple regression analysis method was utilized to evaluate the effects of wind speed characteristics (including the mean, standard deviation, maximum difference, skewness, kurtosis, and difference) on the forecasting precision of the proposed method.

2. Methodology

Wind speed consists of the mean wind speed and turbulence. Mean wind speed consistently follows fundamental physics laws and is relatively easy to forecast using physical, statistical, and intelligent methods. Wind speed turbulence is a stochastic sequence that consistently affects the precision of wind speed forecasting. For the reason that turbulence intensity is related to the mean wind speed, the forecasting error is also related to wind speed. In the present study, a short-term probabilistic wind speed forecasting method combining LSTM and GMM is proposed as follows. The wind speed is first predicted by the LSTM method. The forecasting error is assumed to directly vary with wind speed and to be normally distributed. The joint probability density function (PDF) of forecast wind speeds and errors is then calculated using the GMM method. Finally, the probability distribution of the forecast wind speed is obtained by combining the predicted wind speed and the conditional probability distribution of the forecasting error.

The flowcharts of the proposed method, the denoising method of wind speed data, and the fundamentals of the LSTM and GMM are presented in this section.

2.1. Flowchart of Methodology

A flowchart of the proposed method is shown in Figure 1 and briefly explained as follows.

(1)

The wind speed in the current step, $u_a$, is assumed to be the real-time monitored data obtained by an anemometer. The data are filtered in real time using the Mallat algorithm [39] to reduce the influence of the high-frequency component on the precision of wind speed prediction.

(2)

The wind speed in the next step, $u_{a+1}$, is then predicted using the LSTM method from the denoised sequence of the previous wind speed. The forecast wind speed, ${\widehat{u}}_a$, is treated as the mean value of the probabilistically predicted wind speed in the next step.

(3)

The errors of the previously predicted wind speed, $E$, are calculated by comparing the measured wind speed, $u$, and forecast speed, $\widehat{u}$.

(4)

The key parameters of the joint PDF of the predicted wind speeds and errors are identified using the GMM method.

(5)

The conditional probability distributions and corresponding covariance of the prediction errors of the forecast wind speed, ${\widehat{u}}_a$, are calculated.

(6)

The forecast wind speed, ${\widehat{u}}_a$, is the mean, and the covariance of error, ${\sigma }_e$, is the covariance of ${\widehat{u}}_a$. The probability distributions of the predicted wind speed follow the normal distribution, $N({\widehat{u}}_a,{\sigma }_e)$.

The confidence interval of the forecast wind speed and transcendence probability of a certain wind speed are easily calculated from the normal distribution of $N({\widehat{u}}_a,{\sigma }_e)$.

2.2. LSTM Method

The LSTM method was first proposed by Hochreater and Schmidhube [40] under the fundamental concept of the RNN method. The LSTM method overcomes the problems of gradient disappearance and explosion during the long sequence training [41] of the RNN. Accordingly, the LSTM model has been widely used in wind speed forecasting because it is suitable for long sequences.

The fundamental LSTM cell, shown in Figure 2, mainly consists of three gate units: input, output, and forget gates. The implementation of the cell state updates and computation of the LSTM outputs are as follows:

$$g_i=\sigma (W_i[h_{t-1},x_t]+b_i)$$

(1)

$$g_f=\sigma (W_f[h_{t-1},x_t]+b_f)$$

(2)

$$c_t=g_i\times (W_c[h_{t-1},x_t]+b_c)+g_f\times c_{t-1}$$

(3)

$$g_o=\sigma (W_o[h_{t-1},x_t]+b_o)$$

(4)

$$h_t=g_o\tanh (c_t)$$

(5)

$$\sigma (z)=\frac{1}{1+e^{-z}}$$

(6)

where σ is the logistic sigmoid function; $W$ is the weight; $b$ is the bias; $g_i$, $g_f$, and $g_o$ are the input, oblivion, and output gates, respectively; $c_t$ is the state vector; t is the timestamp; and $h_t$ is the output at timestamp t.

The LSTM parameters are set as follows. “*”means matrix multiplication.

• To avoid overfitting, the number of LSTM network layers must not be overly high [25]. Accordingly, in the present study, the network only consists of two LSTM layers: a dropout and two fully connected layers.
• The maximum number of epochs is set to 50.
• In the training process, the batch size for updating the weights of the LSTM network was set to 128.
• An Adam optimizer [43] with a learning rate of 10⁻⁴ is used.

2.3. GMM Method

The PDF can be estimated using GMM, which is an unsupervised machine learning method. In using the GMM, the presumption is that the PDF of samples can be decomposed into multiple Gaussian PDFs with different weights [44]. Each component of a Gaussian distribution is treated as a component that can be described by two parameters: the mean and the covariance. The GMM is expressed as follows:

$$p(\mathbf{s})={\displaystyle {\sum }_{k=1}^K{\pi }_k}M(s|{\mu }_k,{\displaystyle {\sum }_k}),0<{\pi }_k<1$$

(7)

where $K$ is the number of components; $k$ is the serial number of components; ${\pi }_k$ is the weight of components; ${{\displaystyle \sum }}_{k=1}^K{\pi }_k=1$; $M(s|{\mu }_k,{\displaystyle {\sum }_k})$ is a normal distribution; $s$ is a vector containing multiple variables; and ${\mu }_k$ and ${\sum }_k$ are the mean and covariance, respectively.

Expectation maximization (EM) [45] was applied to identify the parameters from the Gaussian distribution in each component. The EM algorithm consists of two steps: Steps E and M. Step E calculates the posterior distribution after initializing the parameters of the GMM. Step M updates the parameters of the GMM for a new round based on the posterior distribution of Step E. Finally, the parameters of the GMM are obtained after repeating the two steps multiple times until convergence is achieved. In the present study, the number of components in the GMM was set to 1 to simplify the calculation.

2.4. Conditional PDF of Forecasting Errors

As mentioned, wind speeds were first predicted using the LSTM method. However, owing to the random nature of wind speeds, a certain error occurs between the predicted and real wind speeds. The forecasting errors of the previous steps can be obtained by comparing the forecast values with the monitored wind speeds. In addition, because the turbulence intensity or random intensity of wind speeds is related to the actual wind speed, the forecasting errors are assumed to depend only on wind speed values [46]. With this hypothesis, the joint PDF of predicted wind speeds and errors can be identified using the GMM method. The conditional PDF of the forecasting errors is then calculated under a certain forecast wind speed, ${\widehat{u}}_i$.

The two variables, represented by E₁, are predictive errors. The random variable, E₂, represents the predictive values because new data are composed of a predictive error and value. The joint PDF of two-dimensional random variables ($E_1,E_2)$ is written as

$$f\left(e_1,e_2\right)=p\left(s\right)$$

(8)

where $s$ is ${\left[\begin{array}{cc}{e}_1& e_2\end{array}\right]}^T$, and $e_1$ and $e_2$ are the values of variables $E_1$ and $E_2$, respectively.

The marginal PDF of the predicted value, $E_2$, is given by the following:

$$f_{E_2}\left(e_2\right)={\displaystyle {\int }_{-\infty }^{+\infty }f\left(e_1,e_2\right)d{e}_1}$$

(9)

Finally, according to the conditional probability formula, the conditional probability density of variable $E_1$ at variable $E_2=e_2$ is as follows:

$$f_{E_1\left|E_2\right.}\left(e_1\left|e_2\right.\right)=\frac{f\left(e_1,e_2\right)}{f_{E_2}\left(e_2\right)}$$

(10)

2.5. Indicators of Prediction Precision

The precision of probabilistic forecasting methods is evaluated using the prediction intervals indicated by PICP, PINAW, and CWC.

The most important characteristic of prediction intervals is their coverage probability. The number of target values covered by the constructed prediction intervals is represented by the PICP, which is given as follows:

$$PICP=\frac{1}{N}{\displaystyle {\sum }_{t=1}^N{\epsilon }_t}\begin{array}{cc};& {\epsilon }_t\end{array}=\left\{\begin{array}{ccc}1,& if& s_t\in \left[L_t,U_t\right]\\ 0,& & otherwise\end{array}\right.$$

(11)

where $N$ is the number of prediction samples, $s_t$ is the actual wind speed at time step t; $\left[L_t,U_t\right]$ is the prediction interval at a certain confidence level (${\epsilon }_t=1$ if $s_t\in \left[L_t,U_t\right]$, otherwise ${\epsilon }_t=0$).

A higher PICP indicates better probabilistic prediction. In addition, under the same PICP values, narrow predictions indicate that the forecasting results have less discreteness. The prediction interval average width (PIAW) and PINAW (a normalized PIAW) are frequently utilized to evaluate the width of prediction intervals. They are expressed as follows:

$$PIAW=\frac{1}{N}{\displaystyle {\sum }_{t=1}^N\left|U_t-L_t\right|}$$

(12)

$$PINAW=\frac{1}{N\left(y_{\max }-y_{\min }\right)}{\displaystyle {\sum }_{t=1}^N\left|U_t-L_t\right|}$$

(13)

where $y_{max}$ and $y_{min}$ are the maximum and minimum values of the predicted wind speed, respectively; the PINAW is the average width of the prediction intervals at a certain confidence level.

In addition, a CWC was used for the probabilistic prediction methods, as follows:

$$\begin{array}{c}CWC=PINAW\left[1+\varphi \left(PICP\right)\cdot \exp \left(-\eta \left(PICP-PINC\right)\right)\right]\\ \varphi \left(PICP\right)=\left\{\begin{array}{cc}1,\hfill & \begin{array}{cc}if& PICP<PINC\end{array}\\ 0,\hfill & \begin{array}{cc}otherwise& \end{array}\end{array}\right.\end{array}$$

(14)

where $\eta$ is the hyperparameter (generally equal to 50) [47], and PINC is the prediction interval nominal confidence level.

Small CWC values indicate better overall prediction performance.

3. Validation

The proposed method was validated using real wind speeds measured on a bridge site. The wind speed data were measured using Windmasterpro anemometers installed on top of the Pingtan Strait Public Rail Bridge in Fujian Province, China; 29 segments had relatively high wind speeds. Each segment was measured for 20 min, and a total of 1200 wind speed points were obtained.

3.1. Data Sources and Processing

The wind speed segments have been utilized to test the effectiveness of the proposed method, as shown in Figure 3. These are referred to as Datasets 1–3. In each segment, the training/validation set consists of the first 1000 points (indicated by the black line in the figure), and the remaining 200 points (indicated by the red line) comprise the testing set.

3.2. Forecasting Wind Speed by LSTM

The LSTM model was first trained and validated using 1000 wind speed data points. To avoid overfitting, an early stopping method was applied to the training process. After the LSTM model was well trained, the next 200 wind speed points were first predicted using their previous 10 wind speed points as input. In each step, the measured data are denoised in real time using the Mallat algorithm.

The predicted and measured wind speeds, i.e., Datasets 1–3, are compared in Figure 4. The comparison shows that the forecast and measurement results generally agree; however, some differences are observed. This confirms that wind speed uncertainty exists in the use of the LSTM method.

3.3. GMM Fitting and Probabilistic Forecasting Results

The forecasting errors of the LSTM method are calculated by comparing the forecast and measurement results, as shown in Figure 4. The key parameters of the joint PDF of the predicted values and errors are identified using the GMM given in Equation (7); the corresponding PDF can be easily calculated. The calculated joint PDF and the PDF of Datasets 1–3 are shown in Figure 5.

When wind speed is predicted, its corresponding conditional probability distributions can be calculated from the joint PDF using Equation (10). With the forecast wind speed as the mean value and the covariance of the errors as the covariance of the predicted wind speed, the probability distributions of the forecast wind speed is finally obtained. The probability distribution of the forecast and measured wind speeds in Datasets 1–3 are compared in Figure 6. The results show that the probability density kernel distinctly follows the measured wind speed. Moreover, the probability of the forecast wind speed approaching the measured wind speed is evidently higher than the probability of its deviation from the measured values.

The probability distributions of the forecast and measured wind speeds at two instances are compared in Figure 7. The results show that although the probability distribution kernel approaches the measurement results, a distinct gap exists between the forecast and measured results. The uncertainty of the wind speed and LSTM method can be better understood with the aid of Figure 7.

4. Precision Analysis

In this section, the precision of the proposed method is evaluated based on the indicators (PICP, PINAW, and CWC) of the forecasting results of Datasets 1–29. Their characteristics vary according to the mean, standard deviation, maximum difference, skewness (third-order central moment), kurtosis (fourth-order central moment), and difference (difference among adjacent samples), as summarized in

Table 1. Characteristic values of 29 wind speed sequences.

Dataset No.	Mean (m/s)	Standard Deviation (m/s)	Maximum Difference (m/s)	Skewness (m3/s3)	Kurtosis (m4/s4)	Difference (m/s)
1	15.72	7.59	1.11	−0.33	3.47	0.48
2	16.67	5.95	1.22	−0.19	2.30	0.28
3	17.28	5.60	1.03	0.05	2.48	0.30
4	17.20	7.94	1.31	−1.41	5.31	0.36
5	16.96	4.34	0.75	0.07	2.72	0.22
6	15.89	11.84	1.77	1.16	4.93	0.35
7	16.74	3.17	0.58	−0.05	2.70	0.21
8	15.03	5.20	0.89	0.32	2.99	0.23
9	15.81	3.36	0.53	−0.1	2.90	0.27
10	15.69	4.79	0.95	0.19	2.08	0.19
11	15.08	7.74	1.60	0.05	2.27	0.24
12	16.00	5.15	0.81	0.23	3.70	0.24
13	15.41	7.40	1.25	0.36	3.05	0.28
14	15.24	7.43	1.18	−1.48	6.24	0.19
15	11.64	9.59	1.53	−0.76	4.47	0.29
16	12.23	11.32	2.01	−0.74	3.57	0.64
17	10.21	11.96	2.24	0.80	3.58	0.36
18	8.77	5.12	0.94	0.12	2.77	0.18
19	8.33	8.70	1.53	0.52	3.10	0.23
20	13.00	3.57	0.88	0.08	1.75	0.10
21	11.46	3.01	0.46	−0.47	3.34	0.09
22	11.00	4.82	0.93	0.90	3.52	0.13
23	7.82	6.10	1.16	0.67	2.98	0.23
24	7.51	4.40	0.75	0.67	2.92	0.22
25	9.21	5.02	1.09	−0.78	3.04	0.06
26	11.91	5.29	0.99	0.33	2.38	0.13
27	8.08	6.68	1.75	0.67	1.96	0.23
28	9.93	7.32	1.32	−0.17	3.28	0.18
29	10.36	13.09	1.65	0.42	4.67	0.25

.

4.1. Precision Indicators

The confidence level was set to 95% to calculate the PICP, PINAW, and CWC indicators for each dataset. The prediction intervals at the 95% confidence level of Dataset 1 were used as examples. The calculated PICP, PINAW, and CWC values of Dataset 1 were 97.37%, 2.500, and 0.582, respectively. These results verify the effectiveness of the proposed method. In addition, the actual value is generally located at the center of the confidence interval. This indicates the reliability of the proposed method.

The PICP, PINAW, and CWC values of Datasets 2–29 are similarly calculated with prediction intervals at the 95% confidence level, as summarized in

Table 2. PICP, PINAW, and CWC of 29 wind speed sequences.

Dataset No.	PICP (%)	PINAW	CWC
1	97.37	2.500	0.582
2	94.21	3.923	1.087
3	97.37	1.061	0.393
4	97.89	1.211	0.445
5	96.32	0.910	0.262
6	96.32	4.449	0.528
7	96.32	0.731	0.393
8	97.89	1.153	0.301
9	98.42	0.976	0.429
10	95.79	0.840	0.44
11	97.37	1.340	0.477
12	98.95	1.011	0.219
13	97.37	1.656	0.287
14	94.21	1.039	1.006
15	96.84	2.352	0.236
16	94.21	6.022	1.491
17	97.37	2.214	0.249
18	95.79	0.872	0.379
19	97.89	1.237	0.201
20	92.11	0.444	0.691
21	97.89	0.413	0.286
22	97.37	1.046	0.241
23	91.05	0.661	3.217
24	91.05	0.835	2.936
25	95.26	0.357	0.321
26	98.42	0.732	0.303
27	88.42	1.005	10.644
28	97.37	0.436	0.218
29	96.84	2.703	0.235
mean	95.99	1.522	0.983

. The mean values of PICP, PINAW, and CWC for Datasets 1–29 were 95.99%, 1.522, and 0.983, respectively. The results show that the coverage probability of the prediction interval reaches 96% and the normalized average width of the prediction interval is 1.522. The CWC value is an indicator of PICP and PINAW. These indicators confirm the satisfactory performance of the proposed method.

4.2. Effects of Wind Speed Characteristic on Precision

The multiple regression analysis method [48] was implemented to evaluate the effects of wind speed characteristics on forecasting precision. The indicators, PICP, PINAW, and CWC, were considered dependent variables, and five wind speed characteristics were regarded as independent variables. The regression coefficients of each independent variable on each dependent variable were calculated to determine the impact of the characteristics on forecasting precision. A higher regression coefficient indicates a greater influence on the indicator.

The regression coefficients of each indicator against the five wind speed characteristics are shown in Figure 8. The impact percentages of different wind speed characteristics on the indicators are shown in Figure 9. The results show that the standard deviation and difference in wind speed considerably affect the indicators PICP and PINAW, respectively. Both the standard deviation and the difference affect the CWC. The other wind speed characteristics have a negligible effect on forecasting precision. For the reason that the standard deviation and difference are stochastic characteristics of wind speed, the proposed method is also sensitive to these characteristics.

5. Conclusions

In this study, a short-term probabilistic forecasting method for wind speeds combining LSTM and GMM was proposed. The effectiveness of the proposed method was validated using actual data measured at a bridge site. The forecasting precision was evaluated using the PICP, PINAW, and CWC as indicators. The main conclusions are as follows. (1) The proposed method is better suited to the measured values and is effective in obtaining the probability distribution of forecast wind speeds. (2) The calculated average PICP, PINAW, and CWC values for the proposed method for all wind speed samples were 96%, 1.51, and 0.98, respectively. The proposed method exhibits high forecasting precision. (3) The forecasting accuracy of the proposed method is mainly sensitive to the wind speed difference and standard deviation. The foregoing also indicates that the stochastic characteristics of wind speed considerably affect the forecasting precision of the proposed method.