Next Article in Journal
Special Issue on New Trends in Telecommunications Engineering
Next Article in Special Issue
Research on Low-Frequency Oscillation Damping Control of Wind Storage System Based on Pareto and Improved Particle Swarm Algorithm
Previous Article in Journal
Facial Emotion Recognition for Photo and Video Surveillance Based on Machine Learning and Visual Analytics
Previous Article in Special Issue
Improved Combined Inertial Control of Wind Turbine Based on CAE and DNN for Temporary Frequency Support
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Short-Term Wind Power Forecasting Based on VMD and a Hybrid SSA-TCN-BiGRU Network

School of Artificial Intelligence and Data Science, Hebei University of Technology, Tianjin 300132, China
*
Author to whom correspondence should be addressed.
Appl. Sci. 2023, 13(17), 9888; https://doi.org/10.3390/app13179888
Submission received: 14 August 2023 / Revised: 29 August 2023 / Accepted: 30 August 2023 / Published: 31 August 2023
(This article belongs to the Special Issue State-of-the-Art of Power Systems)

Abstract

:
Wind power generation is a renewable energy source, and its power output is influenced by multiple factors such as wind speed, direction, meteorological conditions, and the characteristics of wind turbines. Therefore, accurately predicting wind power is crucial for the grid operation and maintenance management of wind power plants. This paper proposes a hybrid model to improve the accuracy of wind power prediction. Accurate wind power forecasting is critical for the safe operation of power systems. To improve the accuracy of wind power prediction, this paper proposes a hybrid model incorporating variational modal decomposition (VMD), a Sparrow Search Algorithm (SSA), and a temporal-convolutional-network-based bi-directional gated recurrent unit (TCN-BiGRU). The model first uses VMD to break down the raw power data into several modal components, and then it builds an SSA-TCN-BIGRU model for each component for prediction, and finally, it accumulates all the predicted components to obtain the wind power prediction results. The proposed short-term wind power prediction model was validated using measured data from a wind farm in China. The proposed VMD-SSA-TCN-BiGRU forecasting framework is compared with benchmark models to verify its practicability and reliability. Compared with the TCN-BiGRU, the symmetric mean absolute percentage error, the mean absolute error, and the root mean square error of the VMD-SSA-TCN-BiGRU model reduced by 34.36%, 49.14%, and 55.94%.

1. Introduction

1.1. Research Background

With the rapid development of the social economy, the energy demand continues to increase, especially the excessive consumption of fossil fuels such as coal, oil, and natural gas. This has caused problems such as environmental pollution and resource depletion [1]. Due to limited natural resources [2], vigorously developing new energy is beneficial for sustainable social development [3]. Therefore, it is imperative to explore novel energy sources urgently, promote the development and application of clean energy, and increase the proportion of clean energy in the national energy structure. Consequently, wind power generation, as a clean energy technology, is increasingly gaining attention in the field of power systems research. Wind power generation involves converting mechanical energy into electrical energy, which does not cause environmental pollution and can effectively replace traditional thermal power generation, thereby reducing the consumption of fossil fuels. Wind power generation offers numerous advantages, including the following: (1) Wind energy is a renewable and sustainable clean energy source. (2) Wind power is an environmentally friendly source of electricity. (3) Wind turbines occupy less land and are easy to expand.
Wind energy is a pollution-free renewable energy source that has developed rapidly due to its mature technology [4]. Nevertheless, due to its random fluctuations and uncertainties, ensuring the real-time balance of power generation and consumption in the power system is challenging. Short-term wind power prediction is an effective tool to overcome the problems caused by intermittent and fluctuating wind power. Therefore, improving the accuracy of short-term wind power prediction has crucial academic value and economic significance [5].

1.2. Related Works

Currently, domestic and foreign wind power prediction models are mainly categorized as physical methods, statistical models, machine learning, and deep learning methods. Among them, deep learning models are the main focus of current research. The physical model does not require historical data and needs to be combined with wind farm topography and meteorological conditions. However, the complexity of the method needs to be lowered and is computationally challenging. Statistical models are modeled after historical data on wind farms, such as probabilistic autoregression [6] and probability mass bias [7], which builds a relationship between wind power generation and explanatory variables to forecast power generation. These models need to be more capable of dealing with substantial data uncertainty. In nonlinear data, the performance of these models needs to be improved.
Machine learning models are effective in handling nonlinear data. An improved support vector machine (ISVM) was used to forecast short-term wind power [8]. Li et al. used an improved extreme learning machine (ELM) for wind power prediction [9]. However, these models need to be improved in handling large datasets. Deep learning has become a research hotspot in wind power prediction due to its rapid development in recent years. Common deep learning methods include deep neural networks (DNNs), convolutional neural networks (CNNs), recurrent neural networks (RNNs), etc. The long short-term memory network (LSTM) is proposed based on RNNs, which can solve the problem of RNN gradient disappearance. Ewee et al. used a heap-based optimizer (HBO) to optimize the LSTM prediction of wind [10]. Although a gated recurrent unit (GRU) can solve the gradient disappearance problem like LSTM, it has a simpler model structure and a faster training speed and is widely used in wind power prediction. Yang et al. performed a multi-step wind prediction using a combined model of GRU [11]. A CNN is utilized in time series prediction for accurate wind speed prediction using CNN-extracted features [12]. Yildiz et al. converted features obtained after the variational mode decomposition (VMD) process into images and employed an improved residual-based deep CNN to predict wind power [13]. However, CNNs can only extract local features and ignore the relationship between the time series data. For this purpose, Lara et al. used a TCN for time series prediction [14]. A TCN takes into account the timing characteristics and allows for flexibility in adding sensory fields. Zhu et al. used a TCN for wind prediction and compared it with other recurrent neural networks. The experiments showed that the TCN had better prediction results for time series data [15].
Due to the complexity of time series forecasting, it is difficult for a single model to predict the results accurately. Some scholars proposed a hybrid model to improve accuracy. Hossain et al. proposed a hybrid CNN-GRU model, which effectively extracts raw data features using a CNN. A GRU predicts wind power using the input features and verifies the model’s effectiveness by comparing the prediction results with those of a single model [16]. Kosana et al. first preprocessed the raw data using GAN and proposed a hybrid TCN-GRU model for wind speed prediction, which was evaluated using four metrics [17]. Chen et al. applied CNN-LSTM to predict wind speed and concluded that the combined model can deeply extract the temporal and spatial correlation features simultaneously [18]. Due to the non-stationarity and nonlinearity of time series data, Li et al. chose the VMD algorithm to decompose the wind speed. They used the particle swarm optimization (PSO) method to optimize the bi-directional long and short-term memory (Bi-LSTM) prediction model [19]. Huang et al. proposed ensemble empirical mode decomposition (EEMD) to decompose the original wind speed, the decomposed components were predicted using gaussian process regression (GPR) and LSTM, and finally, all of the predicted components were superimposed to obtain the final prediction result, and the experimental results show the validity of the proposed model [20].

1.3. Main Contributions

This paper proposes a TCN-BiGRU method for wind power prediction based on VMD decomposition and SSA optimization. The original wind power data are first decomposed via VMD to obtain a number of intrinsic mode function (IMF) components, and each component is combined with the remaining features of the original data; then, TCN-BiGRU is used to predict the combined data separately, and SSA is used to optimize the number of iterations, learning rate, and number of neurons of TCN-BiGRU to improve the prediction accuracy of the model, and finally, the results of each component of the prediction are summed to obtain the final prediction results. In this paper, the model is used to predict the power of wind turbines in domestic wind farms, and the wind power prediction results of different models obtained every 5 min are compared to verify the effectiveness of the proposed model. The proposed model can accurately predict the wind power output and prepare for scheduling in advance to ensure the smooth operation and safe stability of the power grid. The following are the major contributions:
(1)
By employing the VMD method to decompose the original power data, noise can be reduced, and the lag in the prediction model can be addressed.
(2)
The TCN-BiGRU combination model is utilized to forecast the decomposed power components, and the accumulated predicted components yield the final prediction result.
(3)
To further enhance the prediction accuracy of the model, the SSA algorithm is employed to optimize the hyperparameters in the TCN-BiGRU combination model. This optimization process helps to obtain the optimal parameters, leading to an improvement in the prediction accuracy of the model.

2. Methodology

In this paper, VMD decomposes the raw data, TCN-BiGRU is the basic model for wind power prediction, and sampling SSA optimizes the model’s hyperparameters. Here are the theories and methods related to the model.

2.1. Variational Modal Decomposition

Variational modal decomposition (VMD) is an adaptive signal decomposition method [21] that can effectively suppress the modal aliasing phenomenon that occurs in empirical modal decomposition (EMD) [22]. Unlike EMD, VMD determines each component’s center frequency and bandwidth by iteratively searching for the optimal solution. VMD is based on a non-recursive method to process the original signal. The resulting IMFs are mutually distinct, preventing modal aliasing and resulting in better decomposition and a higher robustness. The decomposition process of VMD is the construction and solution process of a variational problem.
Assuming that the original power series is f(t), decomposing the power into k components, the corresponding constrained variational model is
{ min { u k } , { ω k } k = 1 k t [ ( δ ( t ) + j π t ) u k ( k ) ] e j ω k t 2 2 s . t . k = 1 K u k = f ( t )
where { u k } , { ω k } is the set of central frequencies corresponding to the subsignals, k is the number of modes in the decomposition, t is the sampling moment, t is the Dirac function, is the convolution operation, and f(t) is the original power sequence.
To reconstruct the constrained variational problem as an unconstrained variational problem, the expression of the augmented Lagrangian function is
L ( { u k } , { ω k } , λ ) = α k = 1 k t [ ( δ ( t ) + j π t ) u k ( t ) ] e j ω k t 2 2 + f ( t ) k = 1 k u k ( t ) 2 2 + λ ( t ) , f ( t ) k = 1 k u k ( t )
where λ ( t ) is the Lagrange multiplier operator and α is the quadratic penalty factor.
By searching for the saddle points of the increasing Lagrangian function, it is alternatively possible to find the iterated u k n + 1 , ω k n + 1 , and λ k n + 1 , whose expressions are as follows:
u ^ k n + 1 ( ω ) = f ^ ( ω ) i k u ^ i ( ω ) + λ ^ ( ω ) / 2 1 + 2 α ( ω ω k ) 2
ω k n + 1 = 0 ω | u ^ k ( ω ) | 2 d ω 0 | u ^ k ( ω ) | 2 d ω
λ ^ n + 1 ( ω ) = λ ^ n ( ω ) + T ( f ^ ( ω ) k u ^ k n + 1 ( ω ) )
where u ^ k n + 1 is the Wiener filter for each modal component and ω k n + 1 is the frequency center of the corresponding modal component.
k modal components are obtained when the iterative stop of Equation (6) is satisfied.
k = 1 k ( u k n + 1 u k n 2 2 / u k n 2 2 ) < ε
where ε is the convergence progress.

2.2. Time Convolutional Network

TCN is a convolutional neural network specifically designed for analyzing time series data [23,24,25]. It consists of three key components: causal convolution, dilated convolution, and residual connectivity.

2.2.1. Causal Convolution

Causal convolution [26] serves two essential functions in TCN [27]: ensuring that the network generates an output of the same length as the input and preventing leakage from future to past. Causal convolution differs from a standard convolutional neural network in that it utilizes a unidirectional structure, ensuring that the model input and output are the same size. However, it is important to note that causal convolution is often limited by its receptive field, which restricts its ability to incorporate long-term historical information for accurate predictions.

2.2.2. Dilated Convolution

In contrast to conventional ordinary convolution, the dilation convolution, as shown in Figure 1, can better solve the problem of the restricted perceptual field by performing interval sampling at the input. The dilation convolution can be expressed as Equation (7) with the input X = ( x 0 , x 1 , , x t ) and the filter f : { 0 , 1 , 2 , , n 1 } R .
F ( T ) = ( X d f ) ( T ) = i = 0 n 1 f ( i ) x T d i
where d is the expansion factor and n is the size of the filter.

2.2.3. Residual Connection

Increasing the hidden layers can enhance the TCN model’s perceptual field. However, when there are too many hidden layers, problems such as gradient disappearance may occur and affect the model’s performance. To address these challenges, TCN incorporates residual blocks [28]; the residual connection can effectively reduce the training difficulty, and 1 × 1 conv can adjust the width of the residual tensor. Figure 2 shows the basic structure of the residual connection.
The residual block operation is shown in Equation (8) as follows:
o = A c t i v a t i o n ( x + F ( x ) )
where x R n , F ( ) is the extended convolution and A c t i v a t i o n ( ) is the activation function.

2.3. Bi-directional Gated Recurrent Unit

Conventional RNNs have a limited feature extraction ability for time series and a weak ability to mine deep-level features. Hence, increasing the number of network layers is often necessary to make up for this deficiency. However, increasing the number of network layers may lead to problems such as gradient disappearance and gradient explosion. To address these challenges, researchers proposed a model called GRU [29], which is a simplified LSTM model [30,31,32]. The GRU model contains only two gating mechanisms: reset gate and update gate. This gating mechanism can effectively extract temporal information, has fewer parameters, and requires fewer computational resources than the LSTM model. The basic structure of GRU is shown in Figure 3, and the formulas are (9)–(12).
z t = σ ( W z x t + U z h t 1 )
r t = σ ( W r x t + U r h t 1 )
h ˜ t = tanh ( r U h t 1 + W x t )
h t = ( 1 z t ) h ˜ t + z t h t 1
where x t and h t 1 are the input information at the current moment and the hidden state at the previous moment, respectively, σ is the sigmoid function, W z and U z are the weights of the update gate, W r and U r are the weights of the reset gate, U and W are the weights of h ˜ t , r t is the state of the reset gate, and denotes the Hadmard product.
In order to effectively mine the laws of forward and backward information in time series data, the BiGRU [33,34,35] model employs a structure consisting of a bi-directional recurrent neural network with forward and backward propagation. The structure is shown in Figure 4. In contrast to the unidirectional GRU model, BiGRU can simultaneously consider the changing pattern of the data before and after to mine the time series features more comprehensively. The formula of BiGRU is as follows:
h t = G R U ( x t , h t 1 )
h t = G R U ( x t , h t 1 )
h t = W h t h t + W h t h t + b t
where W h t and h t are the state and weight of the forward hidden layer at time t , W h t and h t are the state and weight of the backward hidden layer at time t , and b t is the bias at time t .

2.4. Sparrow Search Algorithm

The Sparrow Search Algorithm (SSA) is an intelligent heuristic optimization algorithm that categorizes the sparrow population into discoverers, followers, and vigilantes.
A discoverer is a well-adapted individual in a population whose task is to search for food sources and provide its location to other followers. Typically, the number of discoverers in the population ranges from 10% to 20% of the total population. The equation for this location update is as follows:
X i j t + 1 = { X i j t exp ( i α M ) , R 2 < S T X i j t + Q L , R 2 S T
where t is the number of iterations, X i j is the position of the i th sparrow in the j th dimension, α ( 0 , 1 ] is the random number, M is the maximum number of iterations, Q is a random number obeying the positive, too, R 2 [ 0 , 1 ] and S T [ 0.5 , 1 ] are the warning and safety values, respectively, and L is the unit matrix of 1 × j .
In addition to the discoverers, there are followers whose locations are updated as follows:
X i j t + 1 = { Q exp ( X w j t X i j t i 2 ) , i > n 2 X p j t + 1 + | X i j t X p j t + 1 | A + L , i n 2
where X p j is the iterative optimal position, X w j is the iterative global worst position, and A is a 1 × j matrix with a value of 1 or −1, A + = A T ( A A T ) 1 .
When a sparrow finds danger, it issues an alert and calls a vigilante, and the location is updated as follows:
X i j t + 1 = { X b j t + β ( X i j t X b j t ) ,   f i > f g X i j t + k ( X i j t X w j t ( f i f ω ) + ξ ) , f i = f g
where X b j is the global best position, β is the step control parameter, k [ 1 , 1 ] represents the uniform random numbers, ξ is the minimal actual number, and f g and f w are the best and worst adaptations of the current sparrow, respectively.

3. Proposed Model

3.1. Algorithm Overview

The nonlinear and non-smooth nature of wind power makes accurate prediction challenging. VMD decomposition can decompose the raw wind power data into several modal components, reduce the raw data’s noise, and reduce the series’ complexity. During the model training process, the hyperparameters of the model impact its performance. To optimize the model’s hyperparameters and obtain the optimal model, SSA is utilized.

3.2. Model Structure and Algorithm Flow

Numerous factors affect wind power, and it is difficult for the traditional single model to effectively extract the temporal features of wind power. To solve the above problems, this paper proposes a combined TCN-BiGRU model, which enables the model to effectively mine the features of time series data. As Figure 5 shows, the structure of the TCN-BiGRU model consists of two layers of TCN blocks, two layers of BiGRU, and fully connected layers. Initially, the TCN block extracts the time series feature information, and the feature matrix extracted by the TCN is input into the BiGRU. The bi-directional information flow of time series information is captured through the BiGRU to learn the dynamic change pattern of wind power features. Finally, the dimensional change is performed through the fully connected layer, so it is possible to effectively predict the wind power at the next moment.
As the TCN-BiGRU model features numerous hyperparameters, and manually adjusting the hyperparameters of the model has a significant impact on the prediction results, it is proposed to use SSA [36,37,38] to find the number of iterations, the learning rate, and the number of neurons and find the convolution kernel size in TCN-BiGRU to obtain the optimal model and improve the prediction accuracy.
The model optimization process can be divided into the following steps:
(1)
Determine the TCN-BiGRU parameters to be optimized and initialize the population size of the sparrows, the maximum number of iterations, the percentage of discoverers, the percentage of followers, the warning value, the initial positions of the sparrows, and the optimization interval of the hyperparameters.
(2)
Calculate the initial fitness of the individual sparrows and save the optimal fitness and the corresponding sparrows’ positions.
(3)
Update the positions of the discoverers, followers, and alerters.
(4)
Calculate the fitness of the sparrow population and update the optimal fitness and sparrow positions.
(5)
Judge whether the optimal fitness is satisfied or whether the maximum number of iterations is reached. If the condition is satisfied, the optimal parameters are obtained. Otherwise, proceed to step 3.
Since wind power generation is nonlinear and non-stationary, the original wind power generation is first decomposed into multiple subsequences and residuals through signal decomposition. Then, these subsequences and residuals are used as the inputs of the model for wind power generation prediction to effectively reduce the adverse effects of the nonlinear and non-stationary characteristics of wind power generation on the prediction model and improve the prediction accuracy. To this end, the paper introduces VMD to the above model and constructs the VMD-SSA-TCN-BiGRU model, whose structure is shown in Figure 6 below, and its algorithmic flow is as follows:
(1)
Data preprocessing. Since the data extracted from the SCADA system will have some missing values, the original missing values are filled to form the complete time series data.
(2)
The power in the original data is decomposed into several components and residuals via VMD, which was introduced in Section 2.1. Each component and residual are combined with the remaining features of the wind power data to form new data, and then the data are normalized to divide the training and testing sets.
(3)
The parameters of SSA and TCN-BiGRU are initialized, the composed new data are inputted into the TCN-BiGRU model as shown in Figure 5, and the number of iterations, learning rate, number of neurons, and a convolution kernel size of the TCN-BiGRU are optimized via SSA to obtain the optimal model parameters and improve the prediction accuracy.
(4)
The predicted values of all the subsequences are accumulated to output the final results of the model’s one-step-ahead prediction, and the constructed model is evaluated.

4. Experiment Design and Comparative Analysis

4.1. Experimental Setup

The experiment code is implemented using the PyTorch framework with PyCharm 2021.3 as the platform. The operating system is Windows 11 and the hardware device is an NVIDIA GeForce RTX 3060, NVIDIA Corporation. NVIDIA is located in Santa Clara, CA, USA.

4.2. Dataset Description

The dataset used in this paper is the measured data of wind turbine 44 at a wind farm in China. This turbine has a cut-in wind speed of 3 m/s, a rated wind speed of 9 m/s, and a rated power of 2.5 MW. Selected for analysis from 1 December 2020 to 31 January 2021, the historical SCADA data are sampled at the second level, including the wind speed, power, ambient temperature, nacelle temperature, wind direction, angle to the wind, generator speed, and other characteristics.

4.3. Data preprocessing Feature Selection

To address the issue of uneven sampling intervals in the original data, the data are resampled at a frequency of 5 min. After sampling, the total number of data samples is 17,857, with some missing values due to fan failure on 16 December 2020. In this paper, the missing data are filled with zero values. After filling in the complete time series data, the wind direction is normalized using trigonometric functions. The sin and cos values of the wind direction are taken to jointly characterize the wind direction information. The remaining features are normalized to the [0, 1] interval using the max–min normalization process of Equation (19).
x = x x min x max x min
where x is the original sequence, x max is the maximum value of the sequence, and x min is the minimum value of the sequence.

4.4. Feature Selection

To reduce the complexity of the wind power prediction model, this paper employs the Pearson correlation coefficient to analyze the factors influencing wind farms. The Pearson correlation coefficient measures the degree of linear correlation between two characteristic variables. It is calculated by the ratio between the covariance and the standard deviation of the two variables. A correlation coefficient of 1 indicates a perfect positive correlation, −1 indicates a perfect negative correlation, and 0 indicates no correlation between the two variables. As shown in Figure 7, a darker color indicates a stronger correlation. It can be found that the wind power is strongly correlated with the wind speed, generator speed, impeller speed, and theoretical power, but there is a weak correlation with the wind angle, yaw angle, and absolute wind direction, and a negative correlation with the ambient temperature and cabin temperature. This article selects wind power, wind speed, wind direction, generator speed, impeller speed, cabin temperature, and environmental temperature as the input features of the model. For example, an increase in the wind speed corresponds to an increase in power. When the wind speed reaches the rated wind speed, the wind power will also correspondingly reach the rated power because the wind speed significantly impacts the wind power. For the other features, they also have a certain impact on the wind power, and using multiple features as inputs for the prediction model can make the model’s prediction more effective.

4.5. VMD Decomposition Results

In order to enhance the accuracy and stability of the wind power prediction model, it is necessary to solve the issues of significant fluctuations of the original wind power sequence and the large mutual interference of signals at different frequencies. This paper employs VMD decomposition to break down the original sequence into multiple sub-series with different frequencies so that the signals in each sub-series have higher purity and less interference. This approach effectively reduces the noise and interference in the original sequence, thereby simplifying the power sequence. However, selecting the number of decompositions, K, is crucial to the VMD decomposition. A value of K that is too large will generate redundant modes and cause modal mixing. At the same time, a value of K that is too small will lead to modal under-decomposition and reduce the model’s prediction accuracy. The central frequency method is based on spectral analysis in signal processing, and the central idea is to determine the decomposition structure of a signal based on its frequency domain characteristics. In the spectrum of a signal, each peak represents a frequency component in the signal, so detecting the peak in the spectrum can determine the number of frequency components with significant energy in the signal. Based on this idea, the central frequency method uses the spectral information of the signal to determine the K value of the VMD decomposition. This article adopts the central frequency method to determine the size of K. The value of α in VMD can also have an impact on the decomposition. The larger the value of α , the smoother the decomposition result, but it may cause aliasing and information loss between patterns. The smaller the value of α , the closer the decomposition result is to the original signal, but it may cause the decomposition mode to be unsmooth with noise and oscillation. This article uses a grid search to determine the value of α as 2200.
Table 1 shows the relationship between the K value taken and the center frequency. When the K value is fixed, an increase exceeding 50% in the center frequency between adjacent modes indicates that the adjacent modes are dissimilar. From Table 1, it can be seen that when the K value is from 1 to 8, the center frequencies between the adjacent modes are above 50%, which means that it does not cause modal confusion. However, when K = 9, the increase in mode 7 and mode 8 is only 35.02%, and it is considered that the two modes are similar. There is modal confusion, so the K value of 8 is chosen as an appropriate number of decompositions. Figure 8 illustrates the modal components and residuals obtained from the original power decomposition.

4.6. Performance Metrics

In this paper, the symmetric evaluation absolute percentage error E S M A P E , the mean absolute error E M A E , the root mean square error E R M S E , and the fit coefficient R 2 are selected as the evaluation indexes of the model, where E S M A P E is the metric used to describe the relative error of a set of forecasts and their labels as a percentage, E M A E represents the mean prediction error of the predicted value of wind power from the actual value, E R M S E represents the deviation of the predicted wind power at the extreme value point, and R 2 represents the degree of fit of the predicted curve to the true curve, and the following are the above equations:
E S M A P E = 100 % N t = 1 N | x ^ t x t | ( | x ^ t | + | x t | ) / 2
E M A E = 1 N t = 1 N | x ^ t x t |
E RMSE = 1 N t = 1 N ( x ^ t x t ) 2
R 2 = 1 t = 1 N ( x ^ t x t ) 2 t = 1 N ( x ¯ t x t ) 2
where N is the number of prediction series, x ^ t and x t are the predicted and true values of wind power, respectively, and x ¯ t is the average true value of power.

4.7. Results and Discussion

In this study, the dataset is split into a 70% training set and a 30% testing set. The step size is set to 20. This means that the historical data from t -20 to t moments are used to predict the wind power at t +1 moments. The proposed VMD-SSA-TCN-BiGRU model is initialized with the following parameters: The number of decompositions of VMD is 8. The population size of the SSA algorithm is 30. The producers account for 20%, the number of iterations is 20, and the search dimension is 4, which are the number of iterations, the number of neurons, the learning rate, and the convolutional kernel size. The search range of the number of iterations is [50, 300], the search range of the neurons is [50, 150], the search range of the learning rate is [0.001, 0.01], and the search range of the convolutional kernel size is [2, 30]. The expansion factor in TCN is [1, 2, 4, 8, 16], the size of the convolutional filter is 64, and the parameter optimization algorithm obtains the convolutional kernel size. The BiGRU has two layers of the network. The number of neurons is obtained using the optimization algorithm, Adam is used as the optimizer, the batch_size is 512, and mean square error is used as the loss function. To verify the validity of the model proposed in this paper, comparison experiments are performed with the ELM [39,40], BiLSTM, BiGRU, Transformer [41,42], TCN, N-BEATSX [43], TCN-BiGRU, and SSA-TCN-BiGRU models. The evaluation metrics E S M A P E , E M A E , E R M S E , and R 2 are used in Table 2 to evaluate the advantages and disadvantages of each model. Figure 9 shows the fitted curves of the actual and predicted values as well as the local enlargement.
Figure 9 shows the fitting curves of the different models. The wind power prediction curves for the last 5 days and different time periods are drawn. It can be found that the overall trend of the predicted and actual values of the nine models is roughly the same. However, it can be seen that the fitting performance of ELM, BiLSTM, BiGRU, TCN, N-BEATSX, and Transformer for a single model is not as good as that of the combined TCN-BiGRU model, and the combined SSA-TCN-BiGRU model is closer to the actual value than the TCN-BiGRU model. After adding VMD, it is evident that it will reduce the overall lag of the prediction results and improve the prediction curve fitting results. It can be found that the overall prediction results of the VMD-SSA-TCN-BiGRU model proposed in this paper are better than those of the rest of the models.
This paper performs several experiments using identical model parameters and calculates the average value based on 10 trials. Table 2 shows the comparison of the evaluation metrics of different models. The E S M A P E of the VMD-SSA-TCN-BiGRU model is 2.601, while the E S M A P E of the remaining models ELM, BiLSTM, BiGRU, Transformer, TCN, N-BEATSX, TCN-BiGRU, and SSA-TCN-BiGRU are 6.765, 6.312, 5.942, 7.403, 5.805, 4.138, 3.963, and 2.998. The E M A E of the VMD-SSA-TCN-BiGRU model is 38.845, while the remaining models ELM, BiLSTM, BiGRU, Transformer, TCN, N-BEATSX, TCN-BiGRU, and SSA-TCN-BiGRU have E M A E values of 116.351, 109.581, 108.621, 136.527, 103.863, 80.315, 76.375, and 51.137. The proposed model has the most petite E M A E . The E R M S E is highly variable. The RMSE of the VMD-SSA-TCN-BiGRU model is 55.107, while the E R M S E of the remaining models ELM, BiLSTM, BiGRU, Transformer, TCN, N-BEATSX, TCN-BiGRU, and SSA-TCN-BiGRU are 162.763, 156.92, 152.863, 190.164, 148.232, 127.764, 125.071, and 85.972, respectively. The VMD-SSA-TCN-BiGRU model’s R 2 reached 0.997. Based on the evaluation metrics in Table 2, it can be found that the proposed model in this paper outperforms the rest of the models in wind power prediction and can predict wind power more accurately.
Figure 10 shows the scatter plots of the actual and predicted power of the nine models. The horizontal axis of each subgraph represents the actual power corresponding to the model, while the vertical axis represents the predicted power corresponding to the model, including ELM, BiLSTM, BiGRU, Transformer, TCN, N-BEATSX, TCN-BiGRU, SSA-TCN-BiGRU, and VMD-SSA-TCN-BiGRU, which visually shows that the deviation between the true and predicted values of the single model is larger. The deviation of the combined model between the actual and predicted values of VMD-SSA-TCN-BiGRU is significantly smaller and more concentrated than that of TCN-BiGRU and SSA-TCN-BiGRU, which can indicate that the proposed model in this paper has significant advantages in wind power prediction.
To illustrate the effectiveness of the proposed model more intuitively, Figure 11 compares the eight models in terms of E S M A P E , E M A E , and R 2 . When comparing the combined TCN-BiGRU model to the best single model N-BEATSX described above, E S M A P E decreases by 4.23%, E M A E decreases by 4.91%, E R M S E decreases by 2.11%, and R 2 improves by 1.76%, which shows that TCN can effectively extract complex wind power timing features, and BiGRU can learn dynamically changing wind power. After adding the SSA algorithm, E S M A P E decreases by 24.35%, E M A E decreases by 33.04%, E R M S E decreases by 31.26%, and R 2 improves by 1.22%, indicating that SSA can find the optimal model parameters and improve the model’s performance. After adding VMD, E S M A P E decreases by 13.24%, E M A E decreases by 24.03%, E R M S E decreases by 35.90%, and R 2 improves by 0.60%, which shows that the VMD decomposition can effectively reduce the adverse effects of nonlinear and non-stationary characteristics of wind power on prediction.
From the above chart, it can be observed that the hybrid VMD-SSA-TCN-BiGRU model proposed in this article has a significant decrease in the SMAPE, MAE, and RMSE, while the R2 improves. Compared to the current time series’ new algorithm N-BEATSX [44], the TCN can extract deep-level features from the time series, while BiGRU can effectively predict wind power. Although TCN-BiGRU [31] can improve the accuracy of wind power prediction, due to the numerous hyperparameters in the model, it is difficult to find suitable parameters manually. Therefore, hyperparameter optimization algorithms are needed. This paper uses SSA to optimize the hyperparameters of the TCN-BiGRU model, which has a good global search ability and convergence. There are many factors that affect wind power, and the original wind power series fluctuates greatly. Therefore, it is necessary to decompose the original wind power data. This article uses VMD [45,46] to decompose the original wind power data, which can effectively reduce the interference of the wind power data and reduce the overall lag of the model prediction.

5. Conclusions

This article proposes a short-term wind power prediction method for VMD-SSA-TCN-BiGRU. The proposed VMD-SSA-TCN-BiGRU prediction model reduces the SMAPE, MAE, and RMSE by 13.24%, 24.03%, and 35.90%, respectively, compared to the SSA-TCN BiGRU model. The hybrid model was proven to be effective in predicting wind power through four evaluation indicators and draws the following conclusions:
(1)
VMD can reduce noise in the original power and alleviate the overall lag of the prediction model.
(2)
The TCN-BiGRU model can effectively extract temporal features and accurately predict wind power. And using SSA to optimize the hyperparameters of the TCN-BiGRU model further improves the performance of the model.
(3)
This paper verifies the model’s validity through the measured data of wind turbine 44 in a domestic wind farm. It conducts a comparative analysis with the remaining eight models, and the results show that the model proposed in this paper has better stability and a higher accuracy.
Wind power prediction is mainly used for power dispatch, but the information provided by wind power point prediction to power system personnel is limited. In the future, interval prediction [47] based on the point prediction model in this paper can provide more useful credit and better dispatch of the power system.

Author Contributions

Conceptualization, Methodology, Validation, Writing, and Software, Y.Z.; Conceptualization, D.S., K.J. and Y.G.; Supervision and Funding Acquisition, Y.Z. and L.Z. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

The authors do not have permission to share data.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Ehigiamusoe, K.U.; Lean, H.H. Effects of Energy Consumption, Economic Growth, and Financial Development on Carbon Emissions: Evidence from Heterogeneous Income Groups. Environ. Sci. Pollut. Res. 2019, 26, 22611–22624. [Google Scholar] [CrossRef]
  2. Raju, S.; Dinesh Kumar, M.; Balasubramanian, M.; Arul Jeya Kumar, A. Experimentalstudies on Replacement of Steel Stirrups by Sisal Fiber Reinforced Polymers. IOP Conf. Ser. Mater. Sci. Eng. 2020, 912, 062058. [Google Scholar] [CrossRef]
  3. Krishnaraj, L.; Kumar, V.P.; Balasubramanian, M.; Kumar, N.; Shyamala, T. Futuristic Evaluation of Building Energy Simulation Model with Comparison of Conventional Villas. Int. J. Constr. Manag. 2022, 22, 31–40. [Google Scholar] [CrossRef]
  4. Smolarz, A.; Lezhniuk, P.; Kudrya, S.; Komar, V.; Lysiak, V.; Hunko, I.; Amirgaliyeva, S.; Smailova, S.; Orazbekov, Z. Increasing Technical Efficiency of Renewable Energy Sources in Power Systems. Energies 2023, 16, 2828. [Google Scholar] [CrossRef]
  5. Peng, X.; Cheng, K.; Lang, J.; Zhang, Z.; Cai, T.; Duan, S. Short-Term Wind Power Prediction for Wind Farm Clusters Based on SFFS Feature Selection and BLSTM Deep Learning. Energies 2021, 14, 1894. [Google Scholar] [CrossRef]
  6. Dowell, J.; Pinson, P. Very-Short-Term Probabilistic Wind Power Forecasts by Sparse Vector Autoregression. IEEE Trans. Smart Grid 2016, 7, 763–770. [Google Scholar] [CrossRef]
  7. Wan, C.; Wang, J.; Lin, J.; Song, Y.; Dong, Z. Nonparametric Prediction Intervals of Wind Power via Linear Programming. IEEE Trans. Power Syst. 2018, 33, 1074–1076. [Google Scholar] [CrossRef]
  8. Li, C.; Lin, S.; Xu, F.; Liu, D.; Liu, J. Wind Power Prediction Based on Data Mining Technology and Improved Support Vector Machine Method: A Case Study in Northwest China. J. Clean. Prod. 2018, 205, 909–922. [Google Scholar] [CrossRef]
  9. Li, H.; Zou, H. Short-Term Wind Power Prediction Based on Data Reconstruction and Improved Extreme Learning Machine. Arab. J. Sci. Eng. 2022, 47, 3669–3682. [Google Scholar] [CrossRef]
  10. Ewees, A.A.; Al-qaness, M.A.; Abualigah, L.; Abd Elaziz, M. HBO-LSTM: Optimized Long Short Term Memory with Heap-Based Optimizer for Wind Power Forecasting. Energy Convers. Manag. 2022, 268, 116022. [Google Scholar] [CrossRef]
  11. Yang, K.; Wang, B.; Qiu, X.; Li, J.; Wang, Y.; Liu, Y. Multi-Step Short-Term Wind Speed Prediction Models Based on Adaptive Robust Decomposition Coupled with Deep Gated Recurrent Unit. Energies 2022, 15, 4221. [Google Scholar] [CrossRef]
  12. Harbola, S.; Coors, V. One Dimensional Convolutional Neural Network Architectures for Wind Prediction. Energy Convers. Manag. 2019, 195, 70–75. [Google Scholar] [CrossRef]
  13. Yildiz, C.; Acikgoz, H.; Korkmaz, D.; Budak, U. An Improved Residual-Based Convolutional Neural Network for Very Short-Term Wind Power Forecasting. Energy Convers. Manag. 2021, 228, 113731. [Google Scholar] [CrossRef]
  14. Lara-Benítez, P.; Carranza-García, M.; Luna-Romera, J.M.; Riquelme, J.C. Temporal Convolutional Networks Applied to Energy-Related Time Series Forecasting. Appl. Sci. 2020, 10, 2322. [Google Scholar] [CrossRef]
  15. Zhu, R.; Liao, W.; Wang, Y. Short-Term Prediction for Wind Power Based on Temporal Convolutional Network. Energy Rep. 2020, 6, 424–429. [Google Scholar] [CrossRef]
  16. Hossain, M.A.; Chakrabortty, R.K.; Elsawah, S.; Ryan, M.J. Very Short-Term Forecasting of Wind Power Generation Using Hybrid Deep Learning Model. J. Clean. Prod. 2021, 296, 126564. [Google Scholar] [CrossRef]
  17. Kosana, V.; Teeparthi, K.; Madasthu, S. A Novel and Hybrid Framework Based on Generative Adversarial Network and Temporal Convolutional Approach for Wind Speed Prediction. Sustain. Energy Technol. Assess 2022, 53, 102467. [Google Scholar]
  18. Chen, Y.; Zhang, S.; Zhang, W.; Peng, J.; Cai, Y. Multifactor Spatio-Temporal Correlation Model Based on a Combination of Convolutional Neural Network and Long Short-Term Memory Neural Network for Wind Speed Forecasting. Energy Convers. Manag. 2019, 185, 783–799. [Google Scholar] [CrossRef]
  19. Li, J.; Song, Z.; Wang, X.; Wang, Y.; Jia, Y. A Novel Offshore Wind Farm Typhoon Wind Speed Prediction Model Based on PSO–Bi-LSTM Improved by VMD. Energy 2022, 251, 123848. [Google Scholar] [CrossRef]
  20. Huang, Y.; Liu, S.; Yang, L. Wind Speed Forecasting Method Using EEMD and the Combination Forecasting Method Based on GPR and LSTM. Sustainability 2018, 10, 3693. [Google Scholar] [CrossRef]
  21. Li, L.; Meng, W.; Liu, X.; Fei, J. Research on Rolling Bearing Fault Diagnosis Based on Variational Modal Decomposition Parameter Optimization and an Improved Support Vector Machine. Electronics 2023, 12, 1290. [Google Scholar] [CrossRef]
  22. Gao, X.; Li, X.; Zhao, B.; Ji, W.; Jing, X.; He, Y. Short-term Electricity Load Forecasting Model based on EMD-GRU with Feature Selection. Energies 2019, 12, 1140. [Google Scholar] [CrossRef]
  23. Tang, J.; Chien, Y.-R. Research on Wind Power Short-Term Forecasting Method Based on Temporal Convolutional Neural Network and Variational Modal Decomposition. Sensors 2022, 22, 7414. [Google Scholar] [CrossRef] [PubMed]
  24. Zhu, J.; Su, L.; Li, Y. Wind Power Forecasting based on New Hybrid Model with TCN Residual Modification. Energy AI 2022, 10, 100199. [Google Scholar] [CrossRef]
  25. Limouni, T.; Yaagoubi, R.; Bouziane, K.; Guissi, K.; Baali, E.H. Accurate One Step and Multistep Forecasting of Very Short-term PV Power using LSTM-TCN Model. Renew. Energy 2023, 205, 1010–1024. [Google Scholar] [CrossRef]
  26. Zhang, X.; You, J. A Gated Dilated Causal Convolution Based Encoder-Decoder for Network Traffic Forecasting. IEEE Access 2020, 8, 6087–6097. [Google Scholar] [CrossRef]
  27. Pei, Y.; Huang, C.-J.; Shen, Y.; Wang, M. A Novel Model for Spot Price Forecast of Natural Gas Based on Temporal Convolutional Network. Energies 2023, 16, 2321. [Google Scholar] [CrossRef]
  28. Chen, K.; Chen, K.; Wang, Q.; He, Z.; Hu, J.; He, J. Short-Term Load Forecasting with Deep Residual Networks. IEEE Trans. Smart Grid 2019, 10, 3943–3952. [Google Scholar] [CrossRef]
  29. Wang, Y.; Gui, R. A Hybrid Model for GRU Ultra-Short-Term Wind Speed Prediction Based on Tsfresh and Sparse PCA. Energies 2022, 15, 7567. [Google Scholar] [CrossRef]
  30. Lim, S.-C.; Huh, J.-H.; Hong, S.-H.; Park, C.-Y.; Kim, J.-C. Solar Power Forecasting Using CNN-LSTM Hybrid Model. Energies 2022, 15, 8233. [Google Scholar] [CrossRef]
  31. Liu, W.; Liu, Y.; Fu, L.; Yang, M.; Hu, R.; Kang, Y. Wind Power Forecasting Method based on Bidirectional Long Short-Term Memory Neural Network and Error Correction. Electr. Power Compon. Syst. 2022, 49, 1169–1180. [Google Scholar] [CrossRef]
  32. Ying, H.; Deng, C.; Xu, Z.; Huang, H.; Deng, W.; Yang, Q. Short-term Prediction of Wind Power based on Phase Space Reconstruction and BiLSTM. Energy Rep. 2023, 9, 474–482. [Google Scholar] [CrossRef]
  33. Bao, K.; Bi, J.; Ma, R.; Sun, Y.; Zhang, W.; Wang, Y. A Spatial-Reduction Attention-Based BiGRU Network for Water Level Prediction. Water 2023, 15, 1306. [Google Scholar] [CrossRef]
  34. Wang, Y.; Zou, R.; Liu, F.; Zhang, L.; Liu, Q. A Review of Wind Speed and Wind Power Forecasting with Deep Neural Networks. Appl. Energy 2021, 304, 117766. [Google Scholar] [CrossRef]
  35. Lv, S.; Wang, L.; Wang, S. A Hybrid Neural Network Model for Short-Term Wind Speed Forecasting. Energies 2023, 16, 1841. [Google Scholar] [CrossRef]
  36. An, G.; Jiang, Z.; Chen, L.; Cao, X.; Li, Z.; Zhao, Y.; Sun, H. Ultra Short-Term Wind Power Forecasting Based on Sparrow Search Algorithm Optimization Deep Extreme Learning Machine. Sustainability 2021, 13, 10453. [Google Scholar] [CrossRef]
  37. Gao, X.; Guo, W.; Mei, C.; Sha, J.; Guo, Y.; Sun, H. Short-term Wind Power Forecasting based on SSA-VMD-LSTM. Energy Rep. 2023, 9, 335–344. [Google Scholar] [CrossRef]
  38. Huang, Z.; Huang, J.; Min, J. SSA-LSTM: Short-Term Photovoltaic Power Prediction based on Feature Matching. Energies 2022, 15, 7806. [Google Scholar] [CrossRef]
  39. Acikgoz, H.; Yildiz, C.; Sekkeli, M. An Extreme Learning Machine Based Very Short-term Wind Power Forecasting Method for Complex Terrain. Energy Sources Part A Recovery Util. Environ. Eff. 2020, 42, 2715–2730. [Google Scholar] [CrossRef]
  40. Niu, H.; Yang, Y.; Zeng, L.; Li, Y. ELM-QR-Based Nonparametric Probabilistic Prediction Method for Wind Power. Energies 2021, 14, 701. [Google Scholar] [CrossRef]
  41. Qu, K.; Si, G.; Shan, Z.; Kong, X.; Yang, X. Short-term Forecasting for Multiple Wind Farms based on Transformer Model. Energy Rep. 2022, 8, 483–490. [Google Scholar] [CrossRef]
  42. Bommidi, B.S.; Teeparthi, K.; Kosana, V. Hybrid Wind Speed Forecasting using ICEEMDAN and Transformer Model with Novel Loss Function. Energy 2023, 265, 126383. [Google Scholar] [CrossRef]
  43. Olivares, K.G.; Challu, C.; Marcjasz, G.; Weron, R.; Dubrawski, A. Neural Basis Expansion Analysis with Exogenous Variables: Forecasting Electricity Prices with NBEATSx. Int. J. Forecast. 2023, 39, 884–900. [Google Scholar] [CrossRef]
  44. Shuxin, T.; Feng, Z.; Yang, F.; Zhenkun, L.; Yang, M. Short-term Prediction of Offshore Wind Power based on an Improved N-BEATSX Model. In Proceedings of the 2023 IEEE 6th International Electrical and Energy Conference (CIEEC), Hefei, China, 12–14 May 2023; pp. 3812–3817. [Google Scholar]
  45. Lu, P.; Ye, L.; Pei, M.; Zhao, Y.; Dai, B.; Li, Z. Short-term Wind Power Forecasting based on Meteorological Feature Extraction and Optimization Strategy. Renew. Energy 2022, 184, 642–661. [Google Scholar] [CrossRef]
  46. Ye, L.; Li, Y.; Pei, M.; Zhao, Y.; Li, Z.; Lu, P. A Novel Integrated Method for Short-term Wind Power Forecasting based on Fluctuation Clustering and History Matching. Appl. Energy 2022, 327, 120131. [Google Scholar] [CrossRef]
  47. Hanifi, S.; Liu, X.; Lin, Z.; Lotfian, S. A Critical Review of Wind Power Forecasting Methods—Past, Present and Future. Energies 2020, 13, 3764. [Google Scholar] [CrossRef]
Figure 1. Dilated convolution structure.
Figure 1. Dilated convolution structure.
Applsci 13 09888 g001
Figure 2. Residual connection.
Figure 2. Residual connection.
Applsci 13 09888 g002
Figure 3. Gated recurrent unit.
Figure 3. Gated recurrent unit.
Applsci 13 09888 g003
Figure 4. Bi-directional gated recurrent unit.
Figure 4. Bi-directional gated recurrent unit.
Applsci 13 09888 g004
Figure 5. TCN-BiGRU model structure.
Figure 5. TCN-BiGRU model structure.
Applsci 13 09888 g005
Figure 6. VMD-SSA-TCN-BiGRU model structure.
Figure 6. VMD-SSA-TCN-BiGRU model structure.
Applsci 13 09888 g006
Figure 7. The heatmap of the correlation coefficients of input variables.
Figure 7. The heatmap of the correlation coefficients of input variables.
Applsci 13 09888 g007
Figure 8. Raw power and VMD decomposition results.
Figure 8. Raw power and VMD decomposition results.
Applsci 13 09888 g008
Figure 9. Forecasting curves of different models.
Figure 9. Forecasting curves of different models.
Applsci 13 09888 g009
Figure 10. Scatter plot of actual and predicted values.
Figure 10. Scatter plot of actual and predicted values.
Applsci 13 09888 g010
Figure 11. Comparison of evaluation indicators of different models.
Figure 11. Comparison of evaluation indicators of different models.
Applsci 13 09888 g011
Table 1. Central frequencies of each mode under different K values.
Table 1. Central frequencies of each mode under different K values.
Modal KK = 3K = 4K = 5K = 6K = 7K = 8K = 9
Modal 10.0490.0350.0250.0230.0210.0190.018
Modal 22.5801.9241.4991.3621.2531.1471.118
Modal 311.7748.6026.3795.7434.5673.8103.667
Modal 4 21.98614.46913.1689.7558.6288.433
Modal 5 27.81824.39517.50315.55715.128
Modal 6 39.58229.63225.47424.016
Modal 7 52.65238.40933.713
Modal 8 60.30545.521
Modal 9 65.454
Table 2. Performance index prediction results.
Table 2. Performance index prediction results.
Model E S M A P E E M A E E R M S E R 2
ELM6.765116.351162.7630.938
BiLSTM6.312109.581156.9240.957
BiGRU5.942108.621152.8630.958
Transformer7.403136.527190.1640.930
TCN5.805103.863148.2320.962
N-BEATSX4.13880.315127.7640.975
TCN-BiGRU3.96376.375125.0710.979
SSA-TCN-BiGRU2.99851.13785.9720.991
VMD-SSA-TCN-BiGRU2.60138.84555.1070.997
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Zhang, Y.; Zhang, L.; Sun, D.; Jin, K.; Gu, Y. Short-Term Wind Power Forecasting Based on VMD and a Hybrid SSA-TCN-BiGRU Network. Appl. Sci. 2023, 13, 9888. https://doi.org/10.3390/app13179888

AMA Style

Zhang Y, Zhang L, Sun D, Jin K, Gu Y. Short-Term Wind Power Forecasting Based on VMD and a Hybrid SSA-TCN-BiGRU Network. Applied Sciences. 2023; 13(17):9888. https://doi.org/10.3390/app13179888

Chicago/Turabian Style

Zhang, Yujie, Lei Zhang, Duo Sun, Kai Jin, and Yu Gu. 2023. "Short-Term Wind Power Forecasting Based on VMD and a Hybrid SSA-TCN-BiGRU Network" Applied Sciences 13, no. 17: 9888. https://doi.org/10.3390/app13179888

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop