# A Hybrid Feature Pyramid CNN-LSTM Model with Seasonal Inflection Month Correction for Medium- and Long-Term Power Load Forecasting

^{*}

## Abstract

**:**

## 1. Introduction

- A time series feature pyramid structure based on causal dilated convolution is proposed. Multiscale feature extraction and fusion are used to improve the prediction accuracy of the nonlinear model, which avoids the problem of insufficient feature extraction;
- A seasonal inflection month correction strategy is proposed. A unified model is constructed to predict and correct the seasonal inflection monthly load. It can overcome the problem of large model volume and improve the model’s ability to process and predict load seasonality;
- The method proposed in this paper is evaluated on the actual dataset, and the effectiveness and superiority of the method are verified with experiments.

## 2. Feature Pyramid CNN-LSTM Network

#### 2.1. Causal Dilated Convolution

#### 2.2. Feature Pyramid Structure of the Time Series

#### 2.3. Feature Pyramid CNN-LSTM Network

## 3. A Hybrid Feature Pyramid CNN-LSTM Neural Network Model Incorporating Seasonal Inflection Point Month Load Correction

## 4. Experiment Analysis

- Constructing a forecast study of the annual monthly load without inflection point monthly correction to verify the effectiveness of the feature pyramid EEMD-ARIMA-CNN-LSTM model (FPEACL);
- Constructing a separate model and forecasting the seasonal inflection point monthly load to verify the necessity of screening the inflection point monthly load forecasts;
- Using the results of the inflection point monthly load forecast to correct the initial annual monthly load forecast to output the global final forecast.

^{2}) were mainly used. For MAE and RMSE, the closer to zero means the smaller the prediction error, and the closer to 1 for R

^{2}means the better the fit, while MAPE needs to be compared between different models to be meaningful, and the smaller the value means the higher the prediction accuracy.

#### 4.1. Annual Monthly Load Initial Forecast Analysis

- CNN-LSTM: The input sequence was extracted using three one-dimensional convolution layers, and the feature map of the last layer was output to the LSTM to generate a prediction;
- MultiCNN-LSTM: The input sequence was extracted using three convolution layers with different convolution kernel sizes, and the feature maps of the three layers were added and fused to output to the LSTM to generate a prediction;
- TCN-LSTM: The input sequence was extracted using a TCN residual block, and then it was output to the LSTM to generate a prediction;
- Proposed method: The input sequence was extracted using three convolutional layers with different dilation rates, and the features of three different scales were added and fused to the LSTM to generate a prediction.

^{2}was equivalent, RMSE decreased by 34%, and MAPE decreased by 27%, indicating that the modeling prediction method based on different IMF component characteristics was effective. EACL, EAMCL, and EATCL used different CNN-LSTM combined networks to predict high-frequency components. Compared with EAL, the fitting degree was improved, and each error was reduced to varying degrees, indicating that the method of using CNN to extract features and LSTM prediction was effective. FPEACL used the feature pyramid CNN-LSTM hybrid neural network to predict the high-frequency component IMF1. Compared with the model based on other CNN-LSTM combined networks, MAPE, MAE and RMSE were the smallest in terms of error comparison. From the point of view of fitting degree, R

^{2}was closer to 1, and the predicted load was basically consistent with the actual data.

#### 4.2. Inflection Point Monthly Load Correction Analysis

#### 4.2.1. Seasonal Inflection Point Monthly Load Independent Forecast Analysis

^{2}was much worse than the composite metrics for the 12-month period and even had negative values, implying a worse fit than the average forecast method. This shows that the seasonal inflection month forecasts obtained while ignoring the effect of seasonal changes on monthly loads were unreliable. In contrast, screening out the seasonal inflection months and training the model to predict them in a targeted manner yielded smaller errors and better fits.

#### 4.2.2. Integrated Forecast Analysis Incorporating Monthly Load Correction at Seasonal Inflection Points

## 5. Conclusions and Future Work

## Author Contributions

## Funding

## Data Availability Statement

## Conflicts of Interest

## References

- Zhang, M.; Liu, Y.; Li, D.; Cui, X.; Wang, L.; Li, L.; Wang, K. Electrochemical Impedance Spectroscopy: A New Chapter in the Fast and Accurate Estimation of the State of Health for Lithium-Ion Batteries. Energies
**2023**, 16, 1599. [Google Scholar] [CrossRef] - Wang, N.R.; Li, Z.M. Short term power load forecasting based on BES-VMD and CNN-Bi-LSTM method with error correction. Front. Energy Res.
**2023**, 10, 1076529. [Google Scholar] [CrossRef] - Fan, G.F.; Liu, Y.R.; Wei, H.Z.; Yu, M.; Li, Y.H. The new hybrid approaches to forecasting short-term electricity load. Electr. Power Syst. Res.
**2022**, 213, 108759. [Google Scholar] [CrossRef] - Zhang, G.P. Time series forecasting using a hybrid ARIMA and neural network model. Neurocomputing
**2003**, 50, 159–175. [Google Scholar] [CrossRef] - Ghofrani, M.; Ghayekhloo, M.; Arabali, A.; Ghayekhloo, A. A hybrid short-term load forecasting with a new input selection framework. Energy
**2015**, 81, 777–786. [Google Scholar] [CrossRef] - Almaghrabi, S.; Rana, M.; Hamilton, M.; Rahaman, M.S. Solar power time series forecasting utilising wavelet coefficients. Neurocomputing
**2022**, 508, 182–207. [Google Scholar] [CrossRef] - Liu, X.Z.; Xiao, Z.Y.; Zhu, R.B.; Wang, J.; Liu, L.; Ma, M.D. Edge sensing data-imaging conversion scheme of load forecasting in smart grid. Sustain. Cities Soc.
**2020**, 62, 102363. [Google Scholar] [CrossRef] - Sun, H.; Yang, D.; Du, J.; Li, P.; Wang, K. Prediction of Li-ion battery state of health based on data-driven algorithm. Energy Rep.
**2022**, 8, 442–449. [Google Scholar] [CrossRef] - Shao, Z.; Zheng, Q.R.; Liu, C.; Gao, S.Y.; Wang, G.; Chu, Y. A feature extraction- and ranking-based framework for electricity spot price forecasting using a hybrid deep neural network. Electr. Power Syst. Res.
**2021**, 200, 107453. [Google Scholar] [CrossRef] - Nguyen, H.P.; Baraldi, P.; Zio, E. Ensemble empirical mode decomposition and long short-term memory neural network for multi-step predictions of time series signals in nuclear power plants. Appl. Energy
**2021**, 283, 116346. [Google Scholar] [CrossRef] - Liu, D.N.; Zeng, L.; Li, C.B.; Ma, K.L.; Chen, Y.J.; Cao, Y.J. A Distributed Short-Term Load Forecasting Method Based on Local Weather Information. IEEE Syst. J.
**2018**, 12, 208–215. [Google Scholar] [CrossRef] - Yang, X.B. A Novel Extrapolation-Based Grey Prediction Model for Forecasting China’s Total Electricity Consumption. Math. Probl. Eng.
**2021**, 2021, 5576830. [Google Scholar] [CrossRef] - Paparoditis, E.; Sapatinas, T. Short-Term Load Forecasting: The Similar Shape Functional Time-Series Predictor. IEEE Trans. Power Syst.
**2013**, 28, 3818–3825. [Google Scholar] [CrossRef] [Green Version] - Kazemzadeh, M.R.; Amjadian, A.; Amraee, T. A hybrid data mining driven algorithm for long term electric peak load and energy demand forecasting. Energy
**2020**, 204, 117948. [Google Scholar] [CrossRef] - Bashir, T.; Haoyong, C.; Tahir, M.F.; Liqiang, Z. Short term electricity load forecasting using hybrid prophet-LSTM model optimized by BPNN. Energy Rep.
**2022**, 8, 1678–1686. [Google Scholar] [CrossRef] - Buyuksahin, U.C.; Ertekina, S. Improving forecasting accuracy of time series data using a new ARIMA-ANN hybrid method and empirical mode decomposition. Neurocomputing
**2019**, 361, 151–163. [Google Scholar] [CrossRef] [Green Version] - Zhang, Z.C.; Hong, W.C. Application of variational mode decomposition and chaotic grey wolf optimizer with support vector regression for forecasting electric loads. Knowl.-Based Syst.
**2021**, 228, 107297. [Google Scholar] [CrossRef] - Liao, Z.F.; Pan, H.H.; Fan, X.P.; Zhang, Y.; Kuang, L. Multiple Wavelet Convolutional Neural Network for Short-Term Load Forecasting. IEEE Internet Things J.
**2021**, 8, 9730–9739. [Google Scholar] [CrossRef] - Neeraj, N.; Mathew, J.; Agarwal, M.; Behera, R.K. Long short-term memory-singular spectrum analysis-based model for electric load forecasting. Electr. Eng.
**2021**, 103, 1067–1082. [Google Scholar] [CrossRef] - Pelka, P. Analysis and Forecasting of Monthly Electricity Demand Time Series Using Pattern-Based Statistical Methods. Energies
**2023**, 16, 827. [Google Scholar] [CrossRef] - Cui, Z.; Kang, L.; Li, L.; Wang, L.; Wang, K. A hybrid neural network model with improved input for state of charge estimation of lithium-ion battery at low temperatures. Renew. Energy
**2022**, 198, 1328–1340. [Google Scholar] [CrossRef] - Wu, Q.Y.; Guan, F.; Lv, C.; Huang, Y.Z. Ultra-short-term multi-step wind power forecasting based on Cnn-LSTM. IET Renew. Power Gener.
**2021**, 15, 1019–1029. [Google Scholar] [CrossRef] - Kim, T.Y.; Cho, S.B. Predicting residential energy consumption using CNN-LSTM neural networks. Energy
**2019**, 182, 72–81. [Google Scholar] [CrossRef] - Guo, X.F.; Gao, Y.; Li, Y.P.; Zheng, D.; Shan, D. Short-term household load forecasting based on Long- and Short-term Time-series network. Energy Rep.
**2021**, 7, 58–64. [Google Scholar] [CrossRef] - Zhu, R.J.; Guo, W.L.; Gong, X.J. Short-Term Load Forecasting for CCHP Systems Considering the Correlation between Heating, Gas and Electrical Loads Based on Deep Learning. Energies
**2019**, 12, 3308. [Google Scholar] [CrossRef] [Green Version] - Eskandari, H.; Imani, M.; Moghaddam, M.P. Convolutional and recurrent neural network based model for short-term load forecasting. Electr. Power Syst. Res.
**2021**, 195, 107173. [Google Scholar] [CrossRef] - Yin, L.F.; Xie, J.X. Multi-temporal-spatial-scale temporal convolution network for short-term load forecasting of power systems. Applied Energy
**2021**, 283, 116328. [Google Scholar] [CrossRef] - Shao, X.R.; Kim, C.S.; Sontakke, P. Accurate Deep Model for Electricity Consumption Forecasting Using Multi-Channel and Multi-Scale Feature Fusion CNN-LSTM. Energies
**2020**, 13, 1881. [Google Scholar] [CrossRef] [Green Version] - Guo, X.F.; Zhao, Q.N.; Zheng, D.; Ning, Y.; Gao, Y. A short-term load forecasting model of multi-scale CNN-LSTM hybrid neural network considering the real-time electricity price. Energy Rep.
**2020**, 6, 1046–1053. [Google Scholar] [CrossRef] - Zhang, J.A.; Liu, C.Y.; Ge, L.J. Short-Term Load Forecasting Model of Electric Vehicle Charging Load Based on MCCNN-TCN. Energies
**2022**, 15, 2633. [Google Scholar] [CrossRef] - Wang, X.; Cai, Z.; Luo, Y.; Wen, Z.; Ying, S. Long Time Series Deep Forecasting with Multiscale Feature Extraction and Seq2seq Attention Mechanism. Neural Process. Lett.
**2022**, 54, 3443–3466. [Google Scholar] [CrossRef] - Lin, T.Y.; Dollar, P.; Girshick, R.; He, K.M.; Hariharan, B.; Belongie, S. Feature Pyramid Networks for Object Detection. In Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2017), Honolulu, HI, USA, 21–26 July 2017; pp. 936–944. [Google Scholar]
- Cui, Z.Y.; Li, Q.; Cao, Z.J.; Liu, N.Y. Dense Attention Pyramid Networks for Multi-Scale Ship Detection in SAR Images. IEEE Trans. Geosci. Remote Sens.
**2019**, 57, 8983–8997. [Google Scholar] [CrossRef] - Liu, C.; Sun, B.; Zhang, C.H.; Li, F. A hybrid prediction model for residential electricity consumption using holt-winters and extreme learning machine. Appl. Energy
**2020**, 275, 115383. [Google Scholar] [CrossRef] - Chu, Y.Y.; Xu, P.; Li, M.X.; Chen, Z.; Chen, Z.B.; Chen, Y.B.; Li, W.L. Short-term metropolitan-scale electric load forecasting based on load decomposition and ensemble algorithms. Energy Build.
**2020**, 225, 110343. [Google Scholar] [CrossRef] - Bedi, J.; Toshniwal, D. Deep learning framework to forecast electricity demand. Appl. Energy
**2019**, 238, 1312–1326. [Google Scholar] [CrossRef] - Saini, L.M. Peak load forecasting using Bayesian regularization, Resilient and adaptive backpropagation learning based artificial neural networks. Electr. Power Syst. Res.
**2008**, 78, 1302–1310. [Google Scholar] [CrossRef] - Sharma, A.; Jain, S.K. A novel seasonal segmentation approach for day-ahead load forecasting. Energy
**2022**, 257, 124752. [Google Scholar] [CrossRef]

**Figure 8.**Comparison of annual initial monthly load forecasting results between FPEACL and other models.

**Figure 9.**Comparison of independent load forecasting results for the monthly seasonal inflection points.

**Figure 10.**Comparison of final annual monthly load forecasting results between CFPEACL and other models.

IMF1 | IMF2 | IMF3 | IMF4 | IMF5 | IMF6 | Res |
---|---|---|---|---|---|---|

0.535714 | 0.198413 | 0.162698 | 0.027778 | 0 | 0 | 0 |

**Table 2.**The best parameter combination of ARIMA, with p ranging from 0 to 12, and d and q ranging from 0 to 3.

(p, d, q) | |
---|---|

IMF1 | (12, 0, 0) |

IMF2 | (6, 0, 1) |

IMF3 | (6, 0, 1) |

IMF4 | (3, 2, 1) |

IMF5 | (1, 2, 0) |

IMF6 | (0, 2, 1) |

Res | (12, 1, 2) |

Items | Parameters | |
---|---|---|

Software | Operating System | Windows 10 |

Experimental Platform | Anaconda 3 | |

Python | 3.7 | |

TensorFlow | 2.2.0 | |

Hardware | CPU | AMD Ryzen 5 3550H |

CPU main frequency | 2.10 GHz |

**Table 4.**Comparison of monthly load initial forecast results indicators for the whole year between FPEACL and other models.

Model | MAPE | MAE | RMSE | R^{2} |
---|---|---|---|---|

LSTM | 4.910% | 1.6848 | 2.4821 | 0.8230 |

EA | 5.682% | 1.8803 | 2.5110 | 0.8188 |

EAL | 4.161% | 1.3948 | 1.6847 | 0.8188 |

EACL | 3.906% | 1.2279 | 1.6400 | 0.9227 |

EAMCL | 4.163% | 1.2856 | 1.6367 | 0.9230 |

EATCL | 3.875% | 1.1830 | 1.6250 | 0.9241 |

FPEACL | 3.967% | 1.1713 | 1.5355 | 0.9322 |

**Table 5.**The IMF zero-crossing rates for monthly load training and validation sets at the seasonal inflection point.

IMF1 | IMF2 | IMF3 | IMF4 | IMF5 | Res |
---|---|---|---|---|---|

0.6032 | 0.2619 | 0.0873 | 0 | 0 | 0 |

**Table 6.**Comparison of independent load forecasting result indicators for the monthly seasonal inflection points.

Model | MAPE | MAE | RMSE | R^{2} |
---|---|---|---|---|

LSTM | 2.267% | 0.8185 | 1.0561 | 0.5989 |

EA | 5.448% | 1.9206 | 2.5183 | −1.2808 |

EAL | 4.808% | 1.7170 | 1.9654 | −1.2808 |

EACL | 3.517% | 1.2451 | 1.5817 | 0.1002 |

EAMCL | 3.507% | 1.2587 | 1.6208 | 0.0552 |

EATCL | 3.875% | 1.1830 | 1.8089 | −0.1768 |

FPEACL | 3.715% | 1.3300 | 1.8305 | −0.2051 |

Independent FPEACL | 1.642% | 0.5869 | 0.7549 | 0.7950 |

**Table 7.**The comparison between CFPEACL and other models of final annual monthly load forecasting results.

Model | MAPE | MAE | RMSE | R^{2} |
---|---|---|---|---|

LSTM | 4.910% | 1.6848 | 2.4821 | 0.8230 |

EA | 5.682% | 1.8803 | 2.5110 | 0.8188 |

EAL | 4.161% | 1.3948 | 1.6847 | 0.8188 |

EACL | 3.906% | 1.2279 | 1.6400 | 0.9227 |

EAMCL | 4.163% | 1.2856 | 1.6367 | 0.9230 |

EATCL | 3.875% | 1.1830 | 1.6250 | 0.9241 |

FPEACL | 3.679% | 1.1713 | 1.5355 | 0.9322 |

CFPEACL | 2.446% | 0.7302 | 0.9835 | 0.9722 |

Model | Time (s) |
---|---|

LSTM | 185 |

EA | 2483 |

EAL | 1659 |

EACL | 1805 |

EAMCL | 1849 |

EATCNL | 1820 |

FPEACL | 1824 |

CFPEACL | 2922 |

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |

© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Cheng, Z.; Wang, L.; Yang, Y.
A Hybrid Feature Pyramid CNN-LSTM Model with Seasonal Inflection Month Correction for Medium- and Long-Term Power Load Forecasting. *Energies* **2023**, *16*, 3081.
https://doi.org/10.3390/en16073081

**AMA Style**

Cheng Z, Wang L, Yang Y.
A Hybrid Feature Pyramid CNN-LSTM Model with Seasonal Inflection Month Correction for Medium- and Long-Term Power Load Forecasting. *Energies*. 2023; 16(7):3081.
https://doi.org/10.3390/en16073081

**Chicago/Turabian Style**

Cheng, Zizhen, Li Wang, and Yumeng Yang.
2023. "A Hybrid Feature Pyramid CNN-LSTM Model with Seasonal Inflection Month Correction for Medium- and Long-Term Power Load Forecasting" *Energies* 16, no. 7: 3081.
https://doi.org/10.3390/en16073081