Multi-Step Ahead Short-Term Electricity Load Forecasting Using VMD-TCN and Error Correction Strategy

Abstract

The electricity load forecasting plays a pivotal role in the operation of power utility companies precise forecasting and is crucial to mitigate the challenges of supply and demand in the smart grid. More recently, the hybrid models combining signal decomposition and artificial neural networks have received popularity due to their applicability to reduce the difficulty of prediction. However, the commonly used decomposition algorithms and recurrent neural network-based models still confront some dilemmas such as boundary effects, time consumption, etc. Therefore, a hybrid prediction model combining variational mode decomposition (VMD), a temporal convolutional network (TCN), and an error correction strategy is proposed. To address the difficulty in determining the decomposition number and penalty factor for VMD decomposition, the idea of weighted permutation entropy is introduced. The decomposition hyperparameters are optimized by using a comprehensive indicator that takes account of the complexity and amplitude of the subsequences. Besides, a temporal convolutional network is adopted to carry out feature extraction and load prediction for each subsequence, with the primary forecasting results obtained by combining the prediction of each TCN model. In order to further improve the accuracy of prediction for the model, an error correction strategy is applied according to the prediction error of the train set. The Global Energy Competition 2014 dataset is employed to demonstrate the effectiveness and practicality of the proposed hybrid model. The experimental results show that the prediction performance of the proposed hybrid model outperforms the contrast models. The accuracy achieves 0.274%, 0.326%, and 0.405 for 6-steps, 12-steps, and 24 steps ahead forecasting, respectively, in terms of the mean absolute percentage error.

1. Introduction

Electric load forecasting (ELF) is essential for the formulation of economical, reliable, and secure operating strategies for the whole power system [1]. Given the difficulty in storing a large amount of electric energy, precise ELF is considered crucial for the management of generation capacity, scheduling, peak reduction, market evaluation, and so on [2]. Depending on the prediction horizon, electric load forecasting methods are classified into short-term (a few minutes to 24 h), long-term (1 to 10 years), and intermediate-term (a few days to several months) load forecasting [3]. Despite a common practice of using weather-based models for load prediction, they are of less importance to short horizons, because weather-related variables change constantly over a short period of time. Therefore, it is necessary to develop the modeling methods where only historical load data are required, such as termed univariate methods. In the event that weather data and forecasts are unavailable or unaffordable, it is necessary to adopt a univariate method [4].

In general, short-term load forecasting (STLF) methods can be classified into three categories: statistic-based methods, machine learning (ML)-based methods, and hybrid approaches, i.e., the statistic-ML methods.

• According to the statistic-based STLF methods, it is assumed that the current observation is a linear combination of its historical values. Typically, those statistic-based methods include the Holts–Winters (HW) method and the autoregressive moving average (ARMA) methods and its variants. The HW method relies on exponential smoothing to encode historical values that are then used to forecast load for the present and future. For example, in 2011, Taylor adopted five exponentially weighted methods to forecast intraday load, with these methods used in combination to produce the best possible result [5]. In 2018, Mi et al., proposed a hybrid model, and applied the HW method to smooth the original load series [6]. The ARMA is aimed to describe the autocorrelation in time series data. For example, in literature [7,8,9], ARMA, ARIMA, and seasonal ARIMA were used, respectively, to forecast hourly load. However, the dynamic nature of load series hindered statistic-based models from representing its dynamics and capturing the inner non-linearity [10].
• The ML-based STLF methods are operated by learning and mining the historical load features and establishing a prediction model for load forecasting. The commonly adopted approaches include support vector regression (SVR) and artificial neural networks (ANNs). For example, Chen et al., put forward an SVR model to predict the demand response baseline for office buildings [11]. ANNs perform well in feature extraction and nonlinear mapping [12], which is effective in addressing the weaknesses of the time series model. The primary neural networks applied for short-term forecasting include the multi-layer perceptron (MLP) [13] and the feedforward deep neural network (DNN) [14]. With the advancement of machine learning, the recurrent neural network (RNN) has been demonstrated as one of the most effective models for processing time series data. For example, in literature [15,16], the electric load forecasting models based on LSTM were proposed, with the results showing that LSTM can produce a better performance in prediction and achieve a higher accuracy compared to primary ANNs. As a variant of LSTM, the gated recurrent unit (GRU) features a simpler network structure. In [17,18], GRU was introduced for short-term load forecasting. However, it remains necessary to enhance the performance in feature extraction due to the lack of convolution in the model. To address these problems, convolutional neural networks (CNN) have been adopted for their excellent capability of feature extraction. In [19,20], a CNN-LSTM model and a CNN-GRU model were proposed, with CNN layers used for feature extraction from the input data and RNN layers used for sequence learning. As suggested by the research results, CNN hybrid methods outperform the RNN-based model in terms of accuracy. More recently, a specialised CNN architecture known as temporal convolutional networks (TCN) has received popularity due to their applicability to process time series data [21], with the results showing that TCN outperforms LSTM in the accuracy of forecasting. Distinct from one-dimensional CNN [22], TCN relies on stacked CNN and dilated casual convolution, which enables the extraction of time series dependencies between the long intervals of historical data [23].
• According to the hybrid (statistics-ML-based) approach, statistical theory is applied to decompose time series data, ML methods are used to analyse and predict the decomposition, and finally the prediction results of each decomposition part are aggregated [24]. For example, Hasmat Malik et al., proposed a hybrid model where empirical mode decomposition (EMD) and neural networks were combined for multi-step ahead load forecasting [25]. Jian Li et al., integrated empirical mode decomposition (EEMD) with LSTM [26]. Zhang et al., put forward a load prediction method based on complete ensemble empirical mode decomposition (CEEMD) and LSTM [27], with the results suggesting that hybrid approaches are conducive to improving the accuracy of forecasting. However, there are two technical problems with the EMD-like decomposition algorithms. On the one hand, the number of their decompositions is inconsistent when a complex time series is decomposed, and it is difficult to determine the number of predictive models for the subsequent compilation of the program, which will lead to interruptions. On the other hand, EMD-like algorithms are usually disadvantaged by such problems as boundary effects, mode overlap, and the sensitivity to noise, which may affect the final outcome of forecasting. As a novel multiresolution technique intended for signal processing, variational mode decomposition (VMD) [28] is entirely non-recursive and has a solid theoretical foundation. According to the experimental results shown in [29], VMD outperforms CEEMD in terms of decomposition. By decomposing the original series into linear and nonlinear parts, VMD reduces the difficulty of forecasting and ensures the high level of prediction accuracy [30]. However, the application of VMD remains problematic in reality, especially when it comes to determining the hyperparameters [31]. To solve this problem, there have been different evaluation methods proposed that take into account the number of decomposition modes. For example, Li Y et al., used the ratio of residual energy to determine the optimal decomposed number [32]. However, this method produces too many decomposed subsequences, which may lead to it being extremely time consuming. In [33], permutation entropy (PE) was applied to minimize the complexity of subsequences. However, PE is not a comprehensive indicator as it ignores the amplitude of subsequences. To a large extent, validating the VMD model for decomposing data hinges on the purpose of the experiment. Therefore, it is essential to adopt an appropriate evaluation method for the VMD model.

As discussed above, considering the unique advantage of VMD in non-linear and irregular signal decomposition, along with the excellent performance of TCN in prediction, a hybrid model based on VMD and TCN is proposed in this paper for electric load forecasting. The original load series is decomposed into several subsequences based on VMD, and TCN models are applied to train each subsequence. Then, the prediction results obtained by each TCN model are combined to obtain the primary prediction results. According to the analysis of the primary results, there is a certain characteristic in the error series that conforms to the Gaussian distribution. Moreover, an error correction strategy is adopted to further improve forecasting accuracy.

Herein, the publicly available GEFCom dataset [34] is used to validate the proposed method. The main contributions of this paper are detailed as follows:

• In the process of data pre-processing, a novel approach was proposed to determine the most suitable VMD decomposition number and penalty factor based on weighted permutation entropy (WPE). As a more comprehensive indicator, WPE was used to evaluate the time complexity and amplitude of the decomposed subsequences.
• A VMD-TCN model framework was constructed. VMD, as a competitive method of signal decomposition, was applied to reveal the irregular characteristics of the original series. The TCN network, as a method of deep learning, was adopted to explore the hidden high-level interdependence of the electrical load data, which leads to the highly effective prediction.
• An error correction strategy was applied. For the error series between the primary forecasting results and the actual load, an extra TCN model was established again for the purpose of learning and forecasting. The final forecasting results were obtained by summing the error correction values and primary prediction results.

The rest of this paper is organized as follows. In Section 2, the basic methodology of VMD, WPE, and TCN is described. Section 3 introduces the whole process of our proposed forecasting framework. In Section 4, a case study is conducted to validate the method proposed in this paper. Finally, the conclusion is drawn in Section 5.

2. Methodology

2.1. Variational Mode Decomposition

As a novel, non-stationary, signal-adaptive decomposition estimation method proposed by Konstantin Dragomiretskiy in 2014, variational mode decomposition (VMD) is capable of decomposing the original complex non-linear signal into multiple simpler intrinsic mode functions (IMFs), thus reducing the difficulty in making predictions and improving its accuracy. Different from EMD-like algorithms, where recursive method is applied to decompose modes, VMD relies on non-recursive decomposition to process the original signal, which is effective in suppressing mode overlap and improving reliability [35].

VMD assumes that each “mode” is a finite bandwidth with unique center frequencies. The whole process to decompose the original signal X into finite bandwidth IMFs $u_k(t)(i=1,2,\cdots ,K)$ could be described as follows:

• For each subsequence, Hilbert transform is applied to calculate its analytical signal. Then, obtain its unilateral frequency spectrum

  $$\left[\delta (t)+\frac{j}{\pi t}\right]\ast u_k(t)$$

  (1)

  where $\delta (t)$ is the Dirac distribution, $\ast$ is convolution operator, j represents the imaginary part, which satisfies $j^2=-1$.
• Estimate the center frequency of each mode by hybrid exponential tuned $e^{-j{\omega }_{k}t}$ to shift it to the baseband

  $$\left[(\delta (t)+\frac{j}{\pi t})\ast u_k(t)\right]e^{-j{\omega }_{k}t}$$

  (2)
• Calculate the bandwidth of each modal signal based on Gaussian smoothing, and the constrained variational problem could be described as

  $$\left\{\begin{array}{l}\underset{\left\{u\right\},\left\{\omega \right\}}{\min }\left\{{\displaystyle \sum _k{‖{\partial }_t\left[(\delta (t)+\frac{j}{\pi t})\ast u_k(t)\right]e^{-j{\omega }_{k}t}‖}^2}\right\}\\ s.t.{\displaystyle \sum _k{u}_k=f}\end{array}\right.$$

  (3)

  where K is the decomposition number, $\left\{u\right\}=\left\{u_1,\cdots ,u_K\right\}$ represent the set of modal functions, and $\left\{\omega \right\}=\left\{{\omega }_1,\cdots ,{\omega }_K\right\}$ is the set of center frequencies that correspond to the modal functions.
• In order to make the above constrained variational problem unconstrained, quadratic penalty factor $\alpha$ and Lagrangian multiplication operator $\lambda (t)$ are introduced, where $\alpha$ maintains the reconstruction precision of the signal and $\lambda (t)$ maintains the strictness of the constraint. The extended Lagrangian expression is shown in (4):

  $$\begin{array}{l}L(\left\{u_k\right\},\left\{{\omega }_k\right\},\lambda )=\alpha {{\displaystyle \sum _k‖{\partial }_t[(\delta (t)+\frac{j}{\pi t})\ast u_k(t)]e^{-j{\omega }_{k}t}‖}}^2+\\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}{‖f(t)-{\displaystyle \sum _k{u}_k(t)}‖}_2^2+〈\lambda (t),f(t)-{\displaystyle \sum _k{u}_k(t)}〉\end{array}$$

  (4)
• Alternating direction method of multipliers (ADMM) is adopted to solve the original minimization problem. In the expanded Lagrange expression, $u_k^{n+1}$, ${\omega }_k^{n+1}$, and ${\lambda }^{n+1}$ are alternately updated to find the “saddle point”, where $u_k^{n+1}$ could be transformed into the frequency domain using the Fourier isometric transformation

  $${\hat{u}}_k^{n+1}(\omega )=\frac{\hat{f}(\omega )-{\displaystyle \sum _{i\ne k}{\hat{u}}_i(\omega )+\frac{\hat{\lambda }(\omega )}{2}}}{1+2\alpha {(\omega -{\omega }_k)}^2}$$

  (5)

  $${\omega }_k^{n+1}=\frac{{\displaystyle {\int }_0^{\infty }\omega \left|{\hat{u}}_k^{\hat{n}}{(\omega )}^2\right|d\omega }}{{\displaystyle {\int }_0^{\infty }\left|{\hat{u}}_k^{\hat{n}}{(\omega )}^2\right|d\omega }}$$

  (6)

  where $\hat{n}$ is the number of iterations and $\hat{f}(\omega )$, ${\hat{u}}_i(\omega )$, $\hat{\lambda }(\omega )$, and ${\hat{u}}_k^{\hat{n}+1}(\omega )$ denote the Fourier transformation of $f(t)$, $u(t)$, $\lambda (t)$, and $u_k^{\hat{n}+1}(t)$, respectively.

To select the appropriate hyperparameters is crucial for the VMD model, especially the number of the decomposed modes K and the quadratic penalty factor $\alpha$. When different K values are set, there would be a significant difference. Too many modes can lead to mode aliasing and additional noise, while too few components will cause the underdecomposition of the original signal. In addition, the penalty factor can also make difference to the outcome of decomposition. The smaller the penalty factor, the larger the bandwidth of each IMF component, and vice versa.

The effect caused by different values of K and $\alpha$ on VMD results is analyzed using GEFCom dataset. To begin with, $\alpha$ is set to 100, and K is set to 2, 3, 5, 7, and 9, respectively. The center frequency of each IMF varies with K, as shown in Figure 1. It can be seen from the figure there is a significant impact of K value on the outcome of decomposition. When K is set to 2, the bandwidth of IMF2 is relatively large, indicating the under-decomposition of the original signal. As the value of K increases, the central frequency bandwidth of each IMF is reduced gradually. When K = 9, multiple IMFs show frequencies aliasing at the beginning of iterations, which suggests the over-decomposition of the original signal and the emergence of superfluous components.

Secondly, K is set to 7, and $\alpha$ is set to 50, 200, 500, 1000, and 2000, respectively. The center frequency of each IMF varies with $\alpha$, as shown in Figure 2. It can be seen from the figure that the value of $\alpha$ has a considerable impact on the decomposition result as well. When $\alpha$ is set to 5, the IMFs lead to severe frequency aliasing at the beginning of the iterations, with the frequencies of IMF3 and IMF7 being highly comparable. When $\alpha$ increases to 200, the frequency aliasing between the IMFs disappears. However, with a further increase to 500, the frequency of IMF7 fluctuates, which indicates the failed convergence of decomposition. By further increasing to 2000, the fluctuation of the high frequency component discontinues, and the decomposition results converge again. It can be seen from above that two parameters, K and $\alpha$, can make a significant difference to the outcome of decomposition and it is difficult to determine them. To solve this problem, a novel approach to optimize K and $\alpha$ is proposed in this paper by introducing weighted permutation entropy.

2.2. Weighted Permutation Entropy

Permutation entropy (PE) is a measure of the complexity of a time series signal [36]. For each subsequence obtained from VMD, the complexity can be measured by the magnitude of the PE. The higher the PE, the more complex the subsequence is, and vice versa, the more regular the subsequence is. For a time series containing N observations $\left\{X(i),i=1,2,\dots ,N\right\}$, with given embedding dimension m and delay time $\tau$, the embedding vectors are represented as

$$\left(\begin{array}{cccc}x(1)& x(1+\tau )& \cdots & x(1+(m-1)\tau )\\ ⋮& ⋮& \cdots & ⋮\\ x(j)& x(j+\tau )& \cdots & x(j+(m-1)\tau )\\ ⋮& ⋮& & ⋮\\ x(I)& x(I+\tau )& \cdots & x(I+(m-1)\tau )\end{array}\right)$$

(7)

Each row in matrix is an arranged in subvectors, where I is the number of arranged subvectors. In order to find the degree of correlation between the data, each of the subvectors is rearranged in ascending order. The process is shown

$$x(i+(j_1-1)\tau )\le x(i+(j_2-1)\tau )\le \cdots \le x(i+(j_m-1)\tau )$$

(8)

where $j_i$ indicates the position of the data before being rearranged. Then, the original subvector is mapped onto $\left[j_i,j_2,\cdots ,j_m\right]$, which is one of the m! permutations of m distinct symbols [1, 2, 3, …, m].

Introduce a probability distribution $P_1,P_2,\dots ,P_L$ to represent the frequency of occurrence of each subvectors in the order of rearrangement, where $L\le m!$. The PE of the time series $\left\{X(i),i=1,2,\dots ,N\right\}$ can be obtained by calculating the Shannon entropy of L distinct symbols as:

$$H_{\mathrm{PE}}(m)=-{\displaystyle \sum _{i=1}^L{P}_i\ln P_i}$$

(9)

By normalizing it, we obtain

$$0\le {\overline{H}}_{\mathrm{PE}}=H_{\mathrm{PE}}/\ln (m!)\le 1$$

(10)

The PE of each IMF could be calculated according to Equation (10), denoted as $\left\{P_{\mathrm{E}}(1),P_{\mathrm{E}}(2),\cdots ,P_{\mathrm{E}}(K)\right\}$. Since various forecasting methods based on signal decomposition are capable of improving prediction accuracy, modal decomposition is effective in reducing volatility and eliminating the unrelated noise, so as to make it easier to predict each component. However, there are significant variations in the amplitude of the decomposed subsequences, with the modes of larger amplitude exerting a more significant effect on the summed prediction results. To predict the time series, it is necessary to have the decomposed subsequences with minimum time complexity, but the effect of subsequence amplitude is also worth taking into account. Based on the weight permutation entropy (WPE) algorithm [37], the average amplitude ratio of each IMF to the original load series is used to determine the weight.

It is assumed that the average amplitude of the original load series is A, and the average amplitude of each subsequence can be denoted as $\left\{A_1,A_2,\cdots ,A_k\right\}$. Then, the average amplitude can be calculated using the following equation.

$$A=\frac{1}{n}{\displaystyle \sum _{i=1}^n\left|x_i\right|}$$

(11)

Let $w_i=A_i/A$, then the WPE value can be expressed as

$$P_{\mathrm{EW}}=w_1{P}_{\mathrm{E}1}+w_2{P}_{\mathrm{E}2}+\cdots +w_K{P}_{\mathrm{EK}}$$

(12)

$P_{\mathrm{EW}}$ takes into account both time complexity and amplitude for the decomposition results. For the forecasting problem, it is necessary to minimize $P_{\mathrm{EW}}$.

Table 1. WPE values for different K and α for VMD decomposition of GEFCom dataset.

	$\mathit{\alpha }$ = 100	$\mathit{\alpha }$ = 500	$\mathit{\alpha }$ = 1000	$\mathit{\alpha }$ = 2000	$\mathit{\alpha }$ = 3000	$\mathit{\alpha }$ = 4000	$\mathit{\alpha }$ = 5000
$K$ = 2	0.9675	0.9680	0.9686	0.9631	0.9522	0.9355	0.9978
$K$ = 3	0.9960	0.9970	0.9969	0.9962	0.9939	0.9894	0.9851
$K$ = 4	0.7349	0.7354	0.7258	0.6921	0.9026	0.6808	0.8088
$K$ = 5	0.6375	0.6007	0.5921	0.6327	0.5773	0.6236	0.5776
$K$ = 6	0.5708	0.5738	0.5772	0.5390	0.5104	0.5642	0.5634
$K$ = 7	0.5696	0.5699	0.5699	0.5659	0.5671	0.5222	0.5360
$K$ = 8	0.5410	0.5687	0.5667	0.5665	0.5670	0.5644	0.5411

lists the calculation result of $P_{\mathrm{EW}}$ under different K and $\alpha$.

As shown in Table 1, when $\alpha$ is fixed, $P_{\mathrm{EW}}$ decreases significantly with the increase in K, which confirms the capability of decomposition to reduce signal complexity. However, when $\alpha$ takes larger values, further raising K will increase the complexity of the decomposition results. Table 1 also shows that when K is set to 6 and $\alpha$ is set to 3000, the combined PWE reaches its minimum, which means that the weighted time complexity of subsequences is the lowest. Therefore, the hyperparameters of VMD are set as K = 6, and $\alpha$ = 3000, with the remaining parameters set by making reference to literature [32]. Besides, the tolerance is set to 0.3, and $\epsilon$ is set to 10⁻⁶.

2.3. Temporal Convolutional Network

The temporal convolutional network (TCN) is a variant of the convolutional neural network. Instead of using pooling layers as traditional convolutional networks, it gradually increases the receptive field through multiple dilated casual convolution layers, with the output containing more information, which makes the TCN more suitable for time series prediction [38]. The structure of dilated casual convolution layers is shown in Figure 3. Causal convolution refers to the practice of obtaining the output of time t through the convolution of elements at time t and earlier in the previous layer. It prevents the risk of information leakage, thus meeting the requirements of load forecasting. For a convolutional network, the receptive field can be expanded in three ways. The first one is to increase the size of the kernel, the second one is to increase the number of layers, and the last one is to perform a pooling layer before the convolution layer. However, the first two approaches will increase the number of parameters of network, while the third approach will cause the loss of some information. Dilated convolution provides a solution that does not increase the number of network parameters while expanding the receptive field of the output. The receptive field of TCN with dilation base b, kernel size k, and number of layers n is expressed as:

$$r=1+{\displaystyle \sum _{i=0}^{n-1}(k-1)\cdot b^i=1+(k-1)\cdot \frac{b^n-1}{b-1}}$$

(13)

In terms of dilated casual convolution, the overall characteristic of a long series can be effectively captured by adjusting hyperparameters such as kernel size, the number of convolution layers, and the dilation factor d. In addition, to maintain the whole sequence information, the output of each layer is zero padded to match the number of input sequences [39]. For a one-dimensional time series, the mathematical model of dilated casual convolution is expressed as follows:

$$y(s)=(x\ast f)(s)={\displaystyle \sum _{i=0}^{k-1}f(i)\ast x_{s-d\times i}}$$

(14)

where x represents the input, y denotes the output, d denotes the dilation factor, and $\ast$ refers to the convolutional operator.

In general, the feature extraction ability of the neural network is gradually improved with the depth of the network. However, the increase in depth can also lead to a range of problems such as gradient disappearance and explosion, etc. As for the degradation of deep neural networks, it could be addressed through residual connection (skip connect). For those neural networks with the same number of layers, the network with residual block can also converge faster, which allows neural networks to adopt a deeper design. Figure 4 shows the structure of residual connection. In order to prevent overfitting, regularization is introduced via dropout after every convolutional layer. Rectified linear unit (Relu) is used as the activation function. To normalize the input of hidden layers, which counteracts the exploding gradient problem among others, weight normalization is performed for every convolutional layer. The original input is processed by 1 × 1 convolution and adds to its output, which can be expressed as Equation (15). Furthermore, TCN is more memory efficient than recurrent neural networks due to the shared convolution architecture that allows them to process long sequences in parallel [40].

$$o=activation(x+F(x))$$

(15)

3. The Proposed Forecasting Model

The framework of the short-term load forecasting method proposed in this paper consists of VMD, TCN, and error correction strategy, as shown in Figure 5. It can be sum-marized as the following steps:

• Determine the optimal decomposition number K and penalty factor $\alpha$. Based on permutation entropy, the hyperparameters are determined by minimizing the time complexity of the decomposed subsequences. Considering the variation in amplitudes of the subsequences, the high amplitudes also have a relatively more significant impact on the final prediction results. Therefore, a comprehensive indicator $P_{\mathrm{EW}}$ is introduced to evaluate the effect of different hyperparameter values, and the hyperparameter with the minimum $P_{\mathrm{EW}}$ is treated as the optimal.
• VMD decomposition. The original load sequence is denoted as $X=\left\{x_1,x_2,\cdots ,x_n\right\}$, where n represents the number of observations. There are K intrinsic mode functions obtained based on VMD, and they are denoted as $u(1)\cdots u(K)$. Then, each subsequence is normalized to (0, 1), respectively.
• Split train set and test set. A moving window scheme is used to create the input–output pairs that will be fed into neural network. Figure 6 illustrates the process of applying the moving window over the complete time series. The input data of each sample can be represented as $I=\left\{x_1,x_2,\cdots ,x_L\right\}$, where L refers to the sequence length. The label of each sample can be represented as $P=\left\{x_{L+1},x_{L+2},\cdots ,x_{L+S}\right\}$, where S represents the forecasting steps. When different values of S are chosen, the S step future loads can be predicted.
• Each subsequence is trained and predicted using a TCN model. For neural networks, weights updating is a non-convex optimization problem, as the 40 values of weights are related to the convergence efficiency and generalization ability of the network [41]. For the convolutional layers in TCN, weights are randomly initialized through a Gaussian distribution with a mean of 0 and a variance of 0.01. The primary forecasting results $X_{\mathrm{pred}}$ are obtained by aggregating the prediction of each TCN model. $X_{\mathrm{pred}}$ can be expressed as follows.

  $$X_{\mathrm{pred}}={\displaystyle \sum _{i=1}^K\hat{u}(i)}$$

  (16)

  where $\hat{u}(i)$ denotes the prediction of the i-th subsequence.
• Apply the error correction strategy. According to the central limit theorem, a certain pattern of errors between the actual historical data and the primary results can be found out. The distribution of error series conforms to the Gaussian distribution. Thus, an error correction strategy is applied. The trained forecasting model is applied to predict the train set, and forecasting values for the train set are obtained, which denotes as $X_{\mathrm{train}}$. The error series $e_{\mathrm{train}}$ for the train set can be expressed as

  $$e_{\mathrm{train}}={\hat{X}}_{\mathrm{train}}-X_{\mathrm{train}}$$

  (17)

  An extra TCN model is adopted to reveal the intrinsic characteristics of error series. Then, the primary results and the correction values $X_e$ are summed to obtain the final prediction results.

  $$\hat{X}=X_{\mathrm{pred}}+X_e$$

  (18)
• By comparing the final prediction result $\hat{X}$ with the actual load on the test set, performance indicators are calculated to evaluate the model.

4. Experiment and Discussion

The simulation and validation of the model proposed in this paper were performed on a computer (CPU: Intel I7 3.0 GHz, installed RAM: 16 GB, GPU: Nivida GeForce GTX 1050 Ti, operating system: Windows 10 64-bit). The training and analyses for this study were conducted in the Python 3.8 environment, with the Pytorch 11.3 CUDA version as the framework applied to construct the deep learning model.

4.1. Experimental Dataset

GEFCom 2014 dataset consists of three years of hourly load from the New England ISO. Given a strong seasonality of the power load time series, the data recorded in 2014 were used as the experimental dataset that involves 8760 observations, with the first ninety percent, i.e., the first 7884 observations as the training set and the last ten percent, i.e., the last 876 observations, as the test set. The original load series is shown in Figure 7.

As shown in Figure 6, the original load is a complicated series, showing distinct non-linear and non-stationary characteristics. According to the decomposition results shown in Figure 8, the average amplitude of IMF1 is relatively larger and varies slowly, with the trend component of the original sequence captured. As for subsequences IMF2 and IMF3, they show a certain regularity and a relatively evident periodicity, as a result of which they can be treated as periodic components. The average amplitude of IMF4, IMF5, and IMF6 declines in order, showing poor regularity but significant fluctuation. In order to eliminate the effect of amplitude while improving the efficiency of processing for the subsequences, each subsequence is normalized separately to (0, 1). The normalization equation is shown as follows.

$$x_i^{\prime }=\frac{x_i-x_{\min }}{x_{\max }-x_{\min }}$$

(19)

4.2. Performance Indicators

Herein, the root mean square error (RMSE), the mean absolute percentage error (MAPE), and R-square (R²) are adopted to evaluate the forecasting result produced by the proposed model. RMSE is a commonly used measure of the differences between the actual and predicted values. MAPE is applicable to intuitively interpret the relative error. R-square is used to indicate the predictive fit of the model. These metrics are detailed as follows:

• RMSE is defined as follows:

  $$\mathrm{RMSE}=\sqrt{\frac{1}{n}{\displaystyle \sum _{i=1}^n{({\hat{x}}_i-x_i)}^2}}$$

  (20)
• MAPE is defined as follows:

  $$\mathrm{MAPE}=\frac{100\%}{n}{\displaystyle \sum _{i=1}^n\left|\frac{{\hat{x}}_i-x_i}{x_i}\right|}$$

  (21)
• R² is defined as follows.

  $$R^2=1-\frac{{\displaystyle {\sum }_i{({\hat{x}}_i-x_i)}^2}}{{(x_i-{\overline{x}}_i)}^2}$$

  (22)

  where $x_i$ and ${\hat{x}}_i$ are the actual and forecasting load value at time i.

4.3. Training Model

When each mode subsequence is trained and predicted, the forecast steps are defined by the problem, and the input data size must be decided. Since the contribution of far back historical data to the prediction results is relatively low, data redundancy occurs. However, it is possible for the selection of too short historical data as an input feature to cause information loss, which is likely to affect the prediction accuracy. Since the sampling period is one hour and the load series has a 24-h periodicity, the size of the input data is set to 24, i.e., the past 24 h data are used to predict the future load.

As different subsequences of the VMD decomposition tend to have different characteristics, each subsequence is predicted using a TCN model separately. The TCN structure contains three convolutional layers, a fully connected layer with 64 neurons, and an output layer with S neurons, where S represents the forecasting steps. For convolutional layers, the kernel size is set to 4, the number of filters is set to 10, and the stride is set to 1. The Adam optimizer is employed to update these networks, with the learning rate set to 0.001. The number of training epochs is set to 50, and the batch size is set to 32. The primary prediction results are obtained by summing the predictions of each TCN model.

Figure 9a,b show the error series between the actual load and the primary predicted results on the training set and its distribution, respectively. As can be seen from the figures, the prediction error follows a certain pattern, and the probability density of the error approximately conforms to a Gaussian distribution with a mean value of −20. The prediction results on the test set can be compensated for by using the error correction strategy. Furthermore, an extra TCN model is adopted to capture the hidden interdependence of the error series and improve the primary prediction results for the test data. The prediction results on the test set error series are shown in Figure 10, with the MAPE being 3.49% and the RMSE being 16.29 kW. The TCN model maintains a high level of forecasting accuracy for the error series, which ensures that the error correction strategy is effective in improving the final prediction accuracy.

4.4. Experimental Result

The prediction models with 6, 12, and 24 steps ahead are constructed, respectively. To analyze the prediction performance of the proposed model, the TCN and the VMD-TCN are treated as baseline models. Figure 11 shows the comparison of prediction results with 6-steps on the test set. It can be seen from the figure that all three models have captured the periodicity and fluctuations in the load series. However, the fitting performance of the TCN is relatively poor at peak hours and the inflection observations. Figure 12 shows the mean absolute error for the proposed and contrast methods. It can be found out that the purple line, which indicates the mean absolute error of the proposed method, approaches zero the most.

Table 2. Performance evaluations of different models for the GEFCom datasets.

Model	Forecast Steps
	6 Steps			12 Steps			24 Steps
	MAPE/ %	RMSE/ kW	R2	MAPE/ %	RMSE/ kW	R2	MAPE/ %	RMSE/ kW	R2
TCN	1.545	66.617	0.9826	2.553	105.911	0.9561	3.452	138.850	0.9246
VMD + TCN	0.711	29.645	0.9965	0.861	31.873	0.9963	1.117	43.439	0.9924
VMD + TCN + EC	0.274	12.452	0.9994	0.326	16.296	0.9986	0.405	17.81	0.9977

lists the values of the three error indicators for the proposed and contrast methods, including the RMSE, the MAPE, and R². Apparently, the proposed method outperforms the TCN and the VMD-TCN, with the RMSE and MAPE indicator being the smallest relative to other methods. The prediction accuracy shows a significant improvement due to signal decomposition, which verifies that the interdependence of the time series can be fully captured by decomposing the load series with non-stationary characteristics into the relatively stable trend and periodic components. By comparison, the error correction strategy fully exploits the information in the error series on the training set and further improves the prediction accuracy. Under the context of 6-steps ahead forecasting, the MAPE and RMSE of the VMD-TCN are reduced by 53.98% and 55.50% compared to the TCN, respectively. Relative to the VMD-TCN, the MAPE and RMSE of the proposed method are reduced by 61.46% and 57.99%, respectively. In the 12-steps and 24-steps ahead forecasting, the proposed model also performs better. As the number of forecasting steps increases, the prediction accuracy of the models declines gradually due to cumulative errors, despite the overall accuracy of the proposed model remaining at a high level.

4.5. Comparison

To verify the superiority of TCN in the time series prediction, we also evaluated the performance of LSTM and GRU, which have so far been considered as state-of-the-art in forecasting methods. LSTM introduces cell state on the basis of RNN to achieve long-term memory, which is conducive to solving the gradient explosion or gradient disappearance for RNN. Compared to LSTM, GRU involves fewer parameters since it lacks an output gate. The network structure of LSTM and GRU contain two hidden layers with 64 neurons and output layers with S neurons based on the forecasting steps. The training hyperparameters are same to make it comparable to the TCN model.

Table 3. Performance evaluations of contrast models for the GEFCom datasets.

Model	6 Steps			12 Steps			24 Steps
	MAPE/ %	RMSE/ kW	R2	MAPE/ %	RMSE/ kW	R2	MAPE/ %	RMSE/ kW	R2
LSTM	1.875	82.325	0.9737	3.316	135.77	0.9279	3.828	170.556	0.9146
GRU	1.881	84.123	0.9723	3.244	129.238	0.9347	3.924	156.930	0.9037
TCN	1.545	66.617	0.9826	2.553	105.911	0.9561	3.452	138.850	0.9246
VMD-LSTM	1.351	63.965	0.9839	1.856	79.211	0.9755	2.801	111.853	0.9511
VMD-GRU	1.373	64.035	9.9836	1.735	75.846	0.9775	2.949	120.529	0.9432
VMD-TCN	0.711	29.645	0.9965	0.861	31.873	0.9963	1.117	43.439	0.9924

lists the forecasting results as obtained by using the same dataset for comparison.

As shown in Table 3, the three hybrid models outperform the single ML-based models. This is because VMD decomposes the original series into multiple simpler subsequences, thus reducing the prediction error caused by series volatility. By comparing different prediction models, it can be discovered that the prediction accuracy decreases with the number of the forecasting steps increasing. The prediction accuracy of LSTM and GRU is found to be comparable. Although LSTM outperforms the GRU with 6-steps and 24-steps forecasting slightly, it remains inferior to the TCN model. Boxplots are used to visualize the absolute values of forecasting errors in Figure 13, where the box size indicates the spread of prediction errors, + indicates the prediction error outliers, and the orange line indicates the median absolute error. With the different forecasting steps, it can be seen from Figure 13 that the prediction error range of the TCN is narrowest while LSTM and GRU are wider. In addition, the forecasting outliers and the median absolute error of the TCN are smaller than that of contrast models. The comparative experiment was conducted to further demonstrate the significant advantage of the TCN model in power load prediction.

Table 4. Execution time of contrast models for the GEFCom datasets.

Model	6 Steps	12 Steps	24 Steps
LSTM	448.74s	424.59s	436.17s
GRU	374.29s	361.78s	368.92s
TCN	63.17s	67.28s	65.56s
VMD-LSTM	2629.76s	2686.79s	2647.61s
VMD-GRU	2113.85s	2241.80s	2286.35s
VMD-TCN	275.36s	280.62s	280.09s

shows the training time required for different models, revealing that LSTM takes the longest training time, while GRU takes slightly less training time than LSTM. With a 6-step forecasting horizon, TCN takes significantly less training time, which is only 14.07% of LSTM and 16.87% of GRU, respectively. Since the decomposition method increases the forecasting tasks, the execution for VMD-LSTM and VMD-GRU becomes time consuming. By comparing 6-steps forecasting, VMD-TCN is only 10.47% of VMD-LSTM and 13.03% of VMD-GRU in time consumtion.

5. Conclusions

Based on deep learning techniques, the TCN neural networks are advantageous in modeling series data with excellent learning capacities. Herein, a novel hybrid model was proposed by combining a TCN with VMD and an error correction strategy for muti-step electricity load forecasting. To determine the hyperparameters for the VMD algorithm, weighted permutation entropy was introduced to evaluate the decomposition results. As a more comprehensive indicator, WPE takes account of the complexity of decomposed subsequences and their impact on amplitude. By decomposing the original load series with non-linearity and irregularity into several simpler subsequences, VMD reduces the difficulty of prediction. TCN models are established, respectively, to extract the different characteristics of subsequences, with the primary results obtained by summing the predictions of each TCN model. Given a certain pattern in the error series between primary results and actual values, an extra TCN model was applied to capture the interdependence in the error series. Furthermore, the error correction strategy was adopted to improve forecasting accuracy. In terms of forecasting accuracy and time-efficiency, the hybrid model was validated through comparison against other contrast models. Future work will focus on considering the impact of more complex environmental factors, such as temperature and humidity, improving the modeling accuracy.