A Combined Model Incorporating Improved SSA and LSTM Algorithms for Short-Term Load Forecasting

Abstract

To address the current difficulties and problems of short-term load forecasting (STLF), this paper proposes a combined forecasting method based on the improved sparrow search algorithm (ISSA), with fused Cauchy mutation and opposition-based learning (OBL), to optimize the hyperparameters of the long- and short-term-memory (LSTM) network. For the sparrow-search algorithm (SSA), a Sin-chaotic-initialization population, with an infinite number of mapping folds, is first used to lay the foundation for global search. Secondly, the previous-generation global-optimal solution is introduced in the discoverer-location update way, to improve the adequacy of the global search, while adaptive weights are added to reconcile the ability of the local exploitation and global search of the algorithm as well as to hasten the speed of convergence. Then, fusing the Cauchy mutation arithmetic and the OBL strategy, a perturbation mutation is performed at the optimal solution position to generate a new solution, which, in turn, strengthens the ability of the algorithm to get rid of the local space. After that, the ISSA-LSTM forecasting model is constructed, and the example is verified based on the power load data of a region, while the experimental comparison with various algorithms is conducted, and the results confirm the superiority of the ISSA-LSTM model.

1. Introduction

High stability and economy of the power supply as well as providing customers with high quality power are the main tasks of power systems. However, because electrical energy cannot be stored too much and the five links of electrical energy generation, transmission, distribution, transformation, and consumption must be finished at the same time, it is necessary to make the system follow the load changes to produce electrical energy, i.e., to realize the dynamic balance of electrical-energy production and consumption, otherwise the quality and economy of electrical energy cannot be guaranteed, which may even have a serious impact on the safe and stable running of the system [1]. Load forecasting (LF) can be, generally, divided into four types: long-term, medium-term, short-term, and ultra-short-term, and the related literature surveys account for 16%, 20%, 58%, and 6% of past research efforts, respectively. This shows that STLF is a key focus and hot topic in this field [2].

STLF has been developed through decades of research and can be divided into three main types of forecasting methods; the first category is traditional statistical methods, mainly containing linear-regression (LR) [3], autoregression (AR) [4], and auto-regressive-moving-average (ARMA) methods [5]. Statistical methods are simple in structure and easy to model, but the distribution characteristics of the input data have a large impact on their model output. The second category is machine-learning approaches, including gray systems, an artificial neural network (ANN) [6], and a support-vector machine (SVM) [7]. The SVM algorithm can be used for linear/nonlinear problems with a low generalization-error rate and can solve high-dimensional problems in traditional algorithms, but it is slow to converge and has low accuracy when dealing with large data-volume-time series. A back-propagation (BP) neural network in the ANN method has a strong nonlinear mapping capability and can automatically extract data input and output features as well as adjust the network weights to adapt during the training process, but its speed of convergence is slow and prone to fall into local minimization, plus the features need to be manually specified for time-series data, which destroys the integrity of the time series. The third category is the combined-model-prediction method, which generally optimizes multiple hyperparameters present in the model by combining optimization algorithms, such as PSO-BP, PSO-LSTM, etc.

Short-term power-load data are usually compound time series containing its own load fluctuations and related factors, which are time series and non-linear, and it is difficult for statistical methods to model for non-linear time series; although traditional machine-learning methods can overcome this obstacle, the time-series integrity of the input information is difficult to be preserved [8]. In recent years, with the improvement of device-computing ability, deep learning, such as a deep neural network (DNN) [9] and a deep belief network (DBN) [10], has been able to develop at a high speed and has become a hot spot for load-prediction research [11], which DNN and DBN having been applied to improve the prediction accuracy compared with traditional algorithms. A recurrent neural network (RNN) is capable of processing time series of arbitrary length, in principle by using neurons with self-feedback to make the network have short-term memory, and it usually uses a gradient-descent algorithm, but the problem of gradient explosion and disappearance occurs when the input sequence is long. LSTM networks have been widely used in time-series-processing problems, by introducing gating mechanisms to improve the aforementioned problems [12,13,14]. However, the values of these hyperparameters, such as the number of iterations, the learning rate, and the number of hidden layers with their internal neurons set in the LSTM model, are often not optimal, which leads to the model not achieving the best prediction results. The number of iterations and the learning rate are related to the training process and the effectiveness of the LSTM model as well as the number of hidden layers and their internal neurons that affect the fitting capacity of the LSTM [15]. Hyperparameters are usually set by manual experience, which has poor generality and high uncertainty. Therefore, in this paper, a combination of algorithms is considered to build a prediction model.

A comprehensive analysis of the algorithm’s solution speed, stability, and convergence accuracy in the literature [16] showed that the sparrow-search algorithm is highly competitive with the gray-wolf-optimization (GWO) algorithm, the particle-swarm-optimization (PSO) algorithm, and the gravitational-search algorithm (GSA), but the SSA suffers from the problem that the population diversity reduces late in the iteration, so is prone to fall into local extremes [17]. In view of this, this paper makes a series of improvements to the SSA and, also, combines it with LSTM to propose an ISSA-LSTM algorithm, applying it to STLF. Finally, this paper conducts an example validation, based on the load data of a region, and also conducts an experimental comparison with various algorithms, so as to confirm the superiority of the ISSA-LSTM model.

2. Long- and Short-Term Memory Network

LSTM is a recurrent neural-network model that is improved on the basis of the recurrent neural-network (RNN) model, and the composition of RNN is shown in Figure 1 below. The traditional neural-network model is only able to establish weight connections between layers, while the neurons of each RNN cell in RNN can also establish weight connections between them, which is the biggest difference between RNN and traditional neural networks. The neurons in the traditional RNN structure are also connected to each other with weight values. That is, as the sequence advances, the hidden layer located at the front has an effect on the later ones. Therefore, RNNs are better than other kinds of neural-network models for the temporal-sequence problem. However, RNNs cannot handle the problem of long-range dependencies and are highly susceptible to gradient disappearance and explosion, due to their directional loop for information transfer [18]. However, LSTM solves the undesirable problems of RNN mentioned above. First, LSTM can learn on long-term dependent information. Second, since the core idea of LSTM is to include three gating units within each RNN unit, the information of key nodes is selected to be remembered or forgotten, thus well improving the problem of gradient disappearance and explosion that occurs in RNN models on long time-series problems [19]. The left half of Figure 2 below shows the Simple Recurrent Network (SRN) unit, and the right half shows the LSTM module used in the hidden layer of the RNN.

Taking (x₁, x₂,…, x_t) as the series of inputs to the model, and setting the model hidden states as (h₁, h₂, …, h_t), we have Equations (1)–(5) at moment t.

$$i_t=\delta (W_{hi}{h}_{t-1}+W_{xi}{x}_t+b_i)$$

(1)

$$f_t=\delta (W_{hf}{h}_{t-1}+W_{hf}{x}_t+b_f)$$

(2)

$$c_t=f_t\cdot c_{t-1}+i_t\cdot g(W_{hc}{h}_{t-1}+W_{xc}{x}_t+b_c)$$

(3)

$$o_t=\delta (W_{ho}{h}_{t-1}+W_{ox}{x}_t+W_{co}{c}_t+b_o)$$

(4)

$$h_t=o_t\cdot g(c_t)$$

(5)

The f_t, i_t, o_t, and c_t in Equations (1)–(5) denote the oblivion gate, input gate, output gate, and cell state, respectively. W_f, W_i, W_c denote the matrix weights of the corresponding gates, respectively. b_f, b_i, b_c, b_o denote the bias terms of each gate, respectively. · denotes the vector inner product, g(·) and δ(·) denote the sigmoid-function variation and tanh-function variation, respectively.

A linear-regression layer is also added to satisfy the need of implementing prediction using LSTM. The linear-regression-layer expression is set as shown in Equation (6).

$$y_t=W_{yo}{h}_t+b_y$$

(6)

The b_y in the above Equation (6) denotes that the threshold value of the linear-regression layer and y_t denotes the result predicted by the model.

3. Sparrow Search Algorithm

The SSA is a novel intelligent-optimization algorithm recently introduced by Jiankai Xue, inspired by the predatory and anti-predatory behaviors of sparrows in biology. The size of the sparrow population is denoted by N. The sparrow-set matrix is shown in Equations (7) and (8) below. The i = (1,2,…,N) in Equation (8), and d is the dimension of the variable.

$$X={\left[x_1,x_2,\cdots ,x_N\right]}^T$$

(7)

$$x_i=\left[x_{i,}{}_1,x_{i,2},\cdots ,x_{i,d}\right]$$

(8)

The matrix of fitness values of these sparrows is shown in Equations (9) and (10), where N denotes the number of sparrows and the value in each F_x denotes the adaptation value of an individual.

$$F_x={\left[f(x_1),f(x_2),\cdots ,f(x_N)\right]}^T$$

(9)

$$f(x_i)=\left[f(x_i{}_{,1}),f(x_i{}_{,2}),\cdots ,f(x_i{}_{,d})\right]$$

(10)

The sparrow with the better fitness value was the first to get food and acted as a discoverer, leading the entire population closer to the food location. The position of the discoverer was calculated according to Equation (11).

$$X_{i,j}^{t+1}=\left\{\begin{array}{cc}{X}_{i,j}^t\cdot \exp \left(\frac{-i}{\alpha \cdot ite{r}_{\max }}\right),& R_2<ST\\ X_{i,j}^t+Q\cdot L,& R_2⩾ST\end{array}\right.$$

(11)

The t in the above Equation (11) denotes the number of current iterations, j = (1,2, ⋯, d) and $X_{i,j}^t$ means the information of the position of the sparrow with number i in the j-th dimension. iter_max means the maximum number of iterations, α is a random number within (0, 1), R₂(R₂ ∈ [0, 1]), and ST(ST ∈ [0.5, 1]) in sequence means the alert value and the safety value. Q denotes a random number that submits to a normal distribution of [0, 1]. L denotes the all-1 matrix of 1 × d. When R₂ < ST, it means the vicinity is safe and the discoverer practices an extensive search pattern. If R₂ ≥ ST, which indicates that the discoverer has detected the danger, it tells the entire population that it needs to move to another safe location.

The positions of the followers are calculated according to Equation (12).

$$X_{i,j}^{t+1}=\left\{\begin{array}{cc}Q\cdot \exp \left(\frac{X_{worst}^t-X_{i,j}^t}{i^2}\right),& i>\frac{N}{2}\\ X_p^{t+1}+\left|X_{i,j}^t-X_p^{t+1}\right|\cdot A^{+}\cdot L,& other\end{array}\right.$$

(12)

In the above Equation (12), X_worst represents the global worst position. A denotes a 1 × d matrix with all elements arbitrarily given the values one or minus one, and, in addition, A⁺ = A^T(AA^T)⁻¹. When i > 0.5N, it means that the i-th follower with poor adaptation has not eaten food, and, therefore, its own energy value is low. In order to replenish its own energy, it needs to go to other places to forage for food.

When foraging begins, some sparrows are selected by the population to be on guard. When danger appears nearby, the discoverers and followers will abandon the food they have found and flee to other locations. We selected 10–20% of the number of sparrows, noted as SD, and SD sparrows were arbitrarily selected from the population for early alert in each generation. The formula for calculating its position is shown in Equation (13).

$$X_{i,j}^{t+1}=\left\{\begin{array}{ll}{X}_{best}^t+\beta \cdot \left|X_{i,j}^t-X_{best}^t\right|,\hfill & f_i>f_g\hfill \\ X_{i,j}^t+k\cdot \frac{\left|X_{i,j}^t-X_{worst}^t\right|}{f_i-f_w+\epsilon },\hfill & f_i=f_g\hfill \end{array}\right.$$

(13)

X_best in the above Equation (13) represents the best global position. β is the adjustment factor for the step size, which is a normally distributed random number with a variance value of one and a mean value of zero. k denotes a uniform random number within [−1, 1]. f_i is the current sparrow fitness value. f_g and f_w are, in turn, the optimal- and the worst-adaptation degrees of the current global. ε denotes the minimum constant, which is mainly used to prevent the denominator from being zero. When f_i > f_g, it represents that the sparrow is at the outermost part of the population and is prone to encounter predators, which are more dangerous at this time. When f_i = f_g, it represents that the sparrow in the center of the population senses danger and will approach the other sparrows at this time. k indicates the orientation of the sparrow’s movement, and is a step adjustment factor.

4. Improved Sparrow-Search Algorithm

4.1. Sin-Chaos-Initialization Population

Chaos is frequently applied to address optimization-search problems. The Tent model and the Logistic model are the most commonly used chaotic models, but both of them are limited in the number of mapping folds. Unlike the first two models, the number of mapping folds of the sine-chaotification model is unrestricted. Haidong Yang et al. [20] demonstrated that the Sin model has better chaotic properties than the Logistic model, so this paper uses Sin chaos for the SSA algorithm. The Sin chaos 1-dimensional self-mapping expression is shown in Equation (14).

$$\left\{\begin{array}{ll}{x}_{n+1}=\sin \frac{2}{x_n},\hfill & n=0,1,\cdots ,N\hfill \\ -1⩽x_n⩽1,\hfill & x_n\ne 0\hfill \end{array}\right.$$

(14)

The initial value in Equation (14) cannot be set to zero because if the initial value is zero, immobility and zeros will be generated at [−1, 1]. The relationship between initial value sensitivity, traversal, randomness, and the number of iterations of the Sin-chaotic 1-dimensional self-map is shown in Figure 3 [20]. From the sub-graphs (a) and (b) in Figure 3, it is found that when the initial values are set differently, the chaotic sequences produced are different, and from Figure 3c it is seen that when a certain number of generations are updated, the system will traverse the whole solution region.

4.2. Dynamic Self-Adaptation Inertia Weights

For the role of the discoverer, if it starts to approach the optimal solution of the whole space at the beginning of the algorithm iteration, search accuracy will be low. Since the search is too small, it is easy to cause the problem of not being able to get rid of the local extreme-value space. In this paper, we introduce the previous generation of global optimal solutions in the discoverer-position-calculation formula, so that the discoverer position is influenced by both the previous generation of discoverer positions and the previous generation of global optimal solutions, which can effectively prevent the situation that the best expectation value found by the algorithm is always a local extreme value. Furthermore, this paper refers to the concept of inertia weights and adds the dynamic inertia weight parameter w to the way of discoverer position calculation [21]. In the early stage of the iterative process, the value of w is larger, which makes the global exploration better. In addition, at the later stage of the iteration, w starts to decrease adaptively, which makes the local search better and also allows the algorithm to converge faster. w is calculated as shown in Equation (15), and the improved discoverer-location-update method is shown in Equation (16).

$$\omega =\frac{e^{2\left(1-t/{\mathit{iter}}_{\max }\right)}-e^{-2\left(1-t/{\mathit{iter}}_{\max }\right)}}{e^{2\left(1-t/{\mathit{iter}}_{\max }\right)}+e^{-2\left(1-t/{\mathit{iter}}_{\max }\right)}}$$

(15)

$$X_{i,j}^{t+1}=\left\{\begin{array}{cc}{X}_{i,j}^t+\omega \left(f_{j,g}^t-X_{i,j}^t\right)·rand,& R_2<ST\hfill \\ X_{i,j}^t+Q,& R_2\ge ST\hfill \end{array}\right.$$

(16)

In Equation (16), $f_{j,g}^t$ denotes the best solution for the entire space in the j-th dimension in the last generation, and rand represents a random number between 0 and 1.

4.3. Improved Scouting-Warning-Sparrow-Update Formula

The formula for calculating the position of the detection-warning sparrow is improved according to Equation (17).

$$X_{i,j}^{t+1}=\left\{\begin{array}{cc}{X}_{best}^t+\beta \left(X_{i,j}^t-X_{best}^t\right),& f_i\ne f_g\hfill \\ X_{best}^t+\beta \left(X_{worst}^t-X_{best}^t\right),& f_i=f_g\hfill \end{array}\right.$$

(17)

Equation (17) represents that if the sparrow is not in the best location, it flies to a location randomly between itself and the best location. Otherwise, its location will be randomly chosen between the worst and the best locations.

4.4. Incorporating Cauchy Variation and Opposition-Based Learning Strategies

OBL is a method proposed by Tizhoosh, which aims to find the corresponding backward solution based on the current solution through a backward-learning mechanism, and then it saves the relatively better solution after evaluation and comparison. In order to enhance the ability of individuals to solve the best solution, this paper incorporates the OBL strategy into the SSA, which is characterized mathematically as shown in Equations (18) and (19).

$$X_{best}^{’}\left(t\right)=ub+r\oplus \left(lb-X_{best}\left(t\right)\right)$$

(18)

$$X_{i,j}^{t+1}=X_{best}^{’}\left(t\right)+b_1\oplus \left(X_{best}\left(t\right)-X_{best}^{’}\left(t\right)\right)$$

(19)

$X_{best}^{’}$ in Equation (18) denotes the inverse solution according to the tth generation optimal solution, and r is a 1 × d (d is the spatial dimension) matrix of random numbers, which obeys a (0, 1) standard uniform distribution. ub and lb denote the upper and lower bounds, respectively. In addition, ⊕ represents the exclusive OR, and b₁ is the control parameter of information exchange [22], which is calculated as shown in Equation (20).

$$b_1={\left(ite{r}_{\max }-t/ite{r}_{\max }\right)}^t$$

(20)

The Cauchy variation is derived from the Cauchy distribution, and Equation (21) is the one-dimensional Cauchy distribution probability-density expression.

$$f(x)=\frac{1}{\pi }\cdot \frac{a}{a+x^2},x\in (-\infty ,+\infty )$$

(21)

When a = 1, it is known as the standard Cauchy distribution. Figure 4 shows the image of the probability density of the Gaussian distribution as a function of the Cauchy distribution.

From Figure 4, it can be clearly observed that the left and right parts of the Cauchy distribution are flat and long, approaching 0 more gently, with a slower speed compared to the Gaussian distribution, and with a smaller peak near the (0, 0) point compared to the latter, so the Cauchy variation has a more outstanding perturbation ability compared to the Gaussian variation. Hence, applying the Cauchy variant to the target location calculation, the global-search performance of the SSA is upgraded by using the perturbation ability of the Cauchy operator.

The location update method is shown in Equation (22), and the cauchy(0, 1) in Equation (22) denotes the standard Cauchy distribution.

$$X_{i,j}^{t+1}=X_{best}(t)+cauchy(0,1)\oplus X_{best}(t)$$

(22)

In addition, note that the Cauchy distributed random-variables-generating function be η, whose expression is shown in Equation (23) below.

$$\eta =\tan [(\xi -0.5)\pi ]$$

(23)

In Equation (23), tan is the tangent function and ξ denotes a random number between 0 and 1.

In order to further upgrade the algorithm’s optimization-seeking ability, this paper adopts a method that alternates the OBL strategy and the Cauchy variational-operator-perturbation strategy with a certain probability, thereby realizing the dynamic calculation of updated target locations. The former is used to obtain the reverse solution through the reverse-learning mechanism, thus enlarging the search space of the algorithm. The latter uses the Cauchy variational operator to derive a new solution, by performing a perturbation-variation operation at the best solution location, which ameliorates the drawback that the algorithm cannot be detached from the local space. The target position is updated with the selection probability P_s [22] in Equation (24), as the basis for strategy selection.

$$P_{\mathrm{s}}=-\exp {(1-\frac{t}{ite{r}_{\max }})}^{20}+\theta$$

(24)

θ in Equation (24) denotes the adjustment parameter, and its numerical size is taken as 1/20 in this paper.

If rand < P_s, the target position is updated using the OBL strategy of Equations (18)–(20), and vice versa, so the Cauchy variation-perturbation strategy of Equation (22) is selected.

After choosing the perturbation strategy mentioned above, although it reduces the occurrence of the phenomenon of the algorithm falling into the local space, it brings a new problem, that the fitness value of the new location is derived after the perturbation variation is performed, so it is impossible to determine whether it is superior to the fitness value of the original location. Accordingly, this paper introduces a greedy rule as shown in Equation (25). The position fitness value of x is denoted by f(x), and the need to recalculate the position information is determined by comparing the adaptation values of the two positions before and after the update.

$$\left\{\begin{array}{ll}{X}_{best}=X_{i,j}^{t+1},& f(X_{i,j}^{t+1})<f(X_{best})\hfill \\ X_{best}=X_{best},& f(X_{i,j}^{t+1})\ge f(X_{best})\hfill \end{array}\right.$$

(25)

5. Load-Forecasting Models

5.1. Algorithmic Flow of the Model

The flow of the improved SSA, incorporating the Cauchy variation and OBL combined with the LSTM network for prediction, is shown in Figure 5. The left part of Figure 5 shows the ISSA algorithm flow, and the right part shows the algorithm flow of the LSTM model.

The following are the specific steps to optimize the SSA by incorporating Cauchy variation and OBL strategy.

(1)

Initialize parameters, such as population size N, spotter proportion PD, alert threshold R₂, scout proportion SD, and the largest number of iterations, and initialize the population with Sin-chaos mapping using Equation (14).

(2)

The adaptation of each sparrow is calculated according to Equation (9), and the present worst and best adaptation are marked, along with the corresponding location information.

(3)

The sparrows with the better adaptation were chosen as discoverers in the population, and the discoverers updates their position according to Equation (16).

(4)

The other sparrows in the population play the role of followers, and the followers recalculate their locations according to Equation (12).

(5)

A randomly chosen fraction of sparrows in the population play the role of vigilantes, and the vigilantes update their position according to Equation (17).

(6)

Calculate the value of Ps according to Equation (24).

(7)

A choice is made between two strategies, Cauchy mutation perturbation and OBL, based on the value of Ps, thus perturbing the present best solution and, in turn, generating a new solution.

(8)

Based on the greedy rule, use Equation (25) to see if the location should be recalculated.

(9)

Determine whether the algorithm reaches the end condition of the maximum number of iterations. If the end condition is not met, then skip to part (2) above, if it is, then proceed to the next step.

(10)

The program process ends and outputs the best hyperparameter result calculated.

The optimal hyperparameters output from the ISSA are used to construct the LSTM prediction model, and the input data are imported into the forecasting model to derive the power-load-forecasting results.

5.2. Structure of the Model

An ISSA-LSTM-based STLF model is established, and the basic structural framework is illustrated in Figure 6. The left part of Figure 6 illustrates the input of the model, including the load-history values and related influencing factors. The middle part is the constructed load-forecasting model, and the input data are imported into the ISSA-LSTM model to, finally, obtain the forecasting results.

6. Model Experiment

6.1. Performance Comparison of Sparrow Search Algorithm before and after Improvement

(1)

Benchmark function

The performance of the SSA and ISSA algorithms are compared based on eight benchmark-test functions. Their specific formulas and their respective corresponding function images are shown in Appendix A. The test functions include single-peak and multi-peak functions, with dimension d of 10, 30, and 100, respectively. To fairly verify the effectiveness of the ISSA algorithm, the test was conducted in the same operating environment, and the computer processor used was an IntelCore i3-10100F (Santa Clara, CA, USA), the graphics card was a GTX 1060 3G, the SSD capacity size was 500 GB, and the memory was 16 GB. The computer operating system was Microsoft Windows 10, and the simulation software was MATLAB2020a. Ensure that the general conditions are consistent, and set the number of populations to 10, the number of iterations to 1000, PD = 0.7, SD = 0.2 and ST = 0.6, while each algorithm is run 50 times independently.

The information on the range of values and optimal solutions of the eight benchmark-test functions is given in

Table 1. Benchmark function.

Function No.	Function Type	Range of Values	Optimal Solution
F1	Sphere	[−100, 100]	0
F2	Schwefel’s 2.22	[−10, 10]	0
F3	Schwefel’s 1.2	[−100, 100]	0
F4	Rosenbrock’s	[−100, 100]	0
F5	Quartic	[−1.28, 1.28]	0
F6	Rastrigin	[−5.12, 5.12]	0
F7	Ackely	[−32, 32]	0
F8	Griewank	[−600, 600]	0

.

(2)

Performance-comparison analysis of algorithm before and after improvement

The specific results of the two algorithms, for optimizing the eight benchmark-test functions, are given in

Table 2. Results of algorithm-optimization benchmark-test function.

Function	Algorithm	Average Value	Standard Deviation	Average Value	Standard Deviation	Average Value	Standard Deviation
		d = 10		d = 30		d = 100
F1	SSA	2.933 × 10−75	1.581 × 10−74	1.306 × 10−62	6.536 × 10−62	3.011 × 10−53	1.702 × 10−52
	ISSA	0	0	0	0	0	0
F2	SSA	7.612 × 10−37	4.302 × 10−36	2.008 × 10−30	8.912 × 10−30	2.617 × 10−30	9.886 × 10−30
	ISSA	1.055 × 10−258	0	1.017 × 10−261	0	1.005 × 10−273	0
F3	SSA	1.517 × 10−33	7.921 × 10−33	1.688 × 10−29	7.772 × 10−29	6.947 × 10−26	3.229 × 10−25
	ISSA	0	0	0	0	0	0
F4	SSA	5.223 × 10−27	2.582 × 10−26	0	0	0	0
	ISSA	0	0	0	0	0	0
F5	SSA	2.256 × 10−150	1.105 × 10−149	1.401 × 10−109	7.992 × 10−110	8.529 × 10−100	4.507 × 10−99
	ISSA	0	0	0	0	0	0
F6	SSA	0	0	0	0	0	0
	ISSA	0	0	0	0	0	0
F7	SSA	9.238 × 10−16	0	9.238 × 10−16	0	9.238 × 10−16	0
	ISSA	0	0	0	0	0	0
F8	SSA	0	0	0	0	0	0
	ISSA	0	0	0	0	0	0

.

According to Table 2 below, for single-peaked functions F₁ to F₅, the solving accuracy and speed of convergence of the ISSA in different dimensions are greatly improved, compared with the SSA algorithm, and the average value can reach the theoretical optimal solution with the smallest standard deviation. In the optimized multi-peak function F₆ to F₈, the improved sparrow algorithm has the highest solution efficiency in 10 dimensions, 30 dimensions, or 100 dimensions, and it can effectively move away from the local optimum and achieve the desired effect, which indicates that the improved strategy is feasible and effective. In addition, with the gradual increase in dimensionality, it appears that the SSA-solution accuracy decreases, while the ISSA-solution accuracy basically does not change, and, even, further improves in F₂ accuracy, which shows excellent stability.

The optimization effects and convergence plots, combining the single-peak and multi-peak functions, are shown in Figure 7. From Figure 7a–h, it can be found that the ISSA shows more excellent performance in both solution accuracy and speed of convergence, and it can be observed from the two indexes of the mean and standard deviation of the multiple optimization search that the value of the ISSA is smaller, indicating that the stability and robustness of the ISSA are significantly superior to that of the SSA.

6.2. Load-Forecasting-Model Experiment

6.2.1. Input-Data Prerocessing

(1)

Abnormal-data handling

Linear interpolation is used to replace anomalous data and fill in missing data. The load values at time t and t + iΔt are known to be L_t and L_t + iΔt, and if the load data at intermediate time t + jΔt is missing, the load value at t + jΔt at this time is shown in Equation (26).

$$L_{t+j\Delta t}=L_t+j\times \frac{L_{t+i\Delta t}-L_t}{i},j<i$$

(26)

(2)

Data normalization

Since various meteorological factors and load values have different units, which may have different degrees of influence in the operation process, to avoid affecting the forecasting results of the model due to the difference in data magnitude, it is necessary to speciate the data so that each influencing factor is within the same range of values, and then use this as the input variable of the model. In addition, this is also in consideration of the data requirements of the neural network. The optimal solution-finding process becomes significantly smoother after normalizing the data, which makes it easier for the algorithm to converge to the best solution and facilitate subsequent trend analysis of the load data. The data-normalization formula is shown in Equation (27).

$$x^R=\frac{x-x_{\min }}{x_{\max }-x_{\min }}$$

(27)

In Equation (27), x^R is the normalized value, x is the original value of the load data, and x_max and x_min represent the maximum and minimum load values of the same day, respectively.

6.2.2. Evaluation Indicators

The experimental process in this paper involves two evaluation indexes, Mean Absolute Percentage Error (MAPE) and Root Mean Square Error (RMSE), respectively. Using y_i to denote the actual value of the load data, ${\hat{y}}_i$ to denote the forecasting result, and $\overline{y}$ to denote the average of the actual values, the specific formulas for MAPE and RMSE are shown in Equations (28) and (29) below.

(1)

MAPE

The MAPE range is [0, +∞). The smaller its value is, the smaller the difference between ${\hat{y}}_i$ and y_i, and the better the calculation. When its value is 0, it indicates a perfect model, and when its value is greater than 1, it indicates an inferior model.

$$\mathrm{MAPE}={\displaystyle \sum _{t=1}^n\left|\frac{y_i-{\widehat{y}}_i}{y_i}\right|}\times \frac{100}{n}$$

(28)

(2)

RMSE

The RMSE ranges from [0, +∞), ${\hat{y}}_i$ when coinciding exactly with y_i is equal to 0, i.e., the perfect model. The larger the error is, the larger the RMSE. In addition, the RMSE is more sensitive to the outlier response, and if the error is large, the effect of the error on the RMSE is not proportional.

$$\mathrm{RMSE}=\sqrt{\mathrm{MSE}}=\sqrt{\frac{1}{n}{\displaystyle \sum _{i=1}^n{\left(y_i-{\widehat{y}}_i\right)}^2}}$$

(29)

6.2.3. Predictive-Modeling Experiments

The experimental data are historical load data and load-related influencing factors of a region, with a sampling interval of 1 h and 24 sampling points in a day. The load-related influencing factors include meteorological factors and week types. Meteorological factors include maximum temperature, minimum temperature, average temperature, and ambient relative humidity. Week types were classified into four categories: Monday (0.7), Tuesday to Friday (0.8), Saturday (0.4), and Sunday (0.3).

For the hyperparameter-selection problem with LSTM, this paper adopts the improved sparrow algorithm ISSA, which incorporates the Cauchy variation and OBL, to search for the hyperparameters of LSTM. The hyperparameters include the number of iterations (iter), the learning rate (lr) and the number of neurons in the two hidden layers (L₁ and L₂). The mean square difference between the forecasted and true values calculated by the network is used as the fitness function, and the specific formulas are given in Equations (30)–(32). With the ISSA’s optimization search, a set of hyperparameters is able to be found, which makes the trained network have the lowest error.

$$fi{t}_i=\frac{1}{2}\left({\mathrm{MSE}}_{train}+{\mathrm{MSE}}_{\mathit{test}}\right)$$

(30)

$${\mathrm{MSE}}_{train}=\frac{1}{n}{\left(y_{train}-{\hat{y}}_{train}\right)}^2$$

(31)

$${\mathrm{MSE}}_{\mathit{test}}=\frac{1}{n}{\left(y_{test}-{\hat{y}}_{test}\right)}^2$$

(32)

(1)

LSTM-based prediction model

The selected load dataset in this paper is recorded 97 samples after data preprocessing, and the first 96 samples are used as the training set for the training of the prediction model. The remaining one sample of the data set is used as the test set, to forecast the load data of the last day. The LSTM model is set to 200 for L₁, 200 for L₂, 1000 for iter, and 0.0050 for lr. The model-prediction result data and the real value data are given in

Table 3. Comparison of output values of LSTM model with real values.

Serial No.	True Value (MW)	Predicted Values (MW)	Serial No.	True Value (MW)	Predicted Values (MW)
1	59.61	67.78	13	76.44	76.55
2	57.95	60.61	14	82.43	85.52
3	55.66	58.18	15	84.79	84.27
4	53.91	55.48	16	86.20	84.82
5	52.89	55.53	17	85.94	87.40
6	55.02	55.55	18	87.40	84.55
7	60.14	58.61	19	85.19	82.21
8	67.52	64.19	20	87.81	89.72
9	80.02	80.41	21	84.95	85.01
10	86.16	84.50	22	83.27	81.54
11	90.40	87.04	23	75.93	74.92
12	89.53	86.77	24	68.55	66.86

below.

The comparison of the prediction curves generated by the LSTM-based prediction model, with the true value curves of the load data, is illustrated in Figure 8.

(2)

SSA-LSTM-based prediction model

A forecasting model based on SSA-LSTM was established. The results of the hyperparameters of the LSTM model, after parameter search by the SSA, are given in

Table 4. The result of SSA search for hyperparameters.

Hyperparameters	Value
L1	270
L2	383
iter	175
lr	0.0060

below.

The LSTM network was rebuilt using the above hyperparameters to obtain the prediction results. The model-prediction result data and the real value data are given in

Table 5. Comparison of the output values of the SSA-LSTM model with the real values.

Serial No.	True Value (MW)	Predicted Values (MW)	Serial No.	True Value (MW)	Predicted Values (MW)
1	59.61	64.62	13	76.44	76.79
2	57.95	56.33	14	82.43	83.13
3	55.66	57.40	15	84.79	83.82
4	53.91	55.59	16	86.20	84.92
5	52.89	53.27	17	85.94	87.28
6	55.02	52.68	18	87.40	85.42
7	60.14	57.90	19	85.19	83.69
8	67.52	65.31	20	87.81	87.31
9	80.02	80.39	21	84.95	86.74
10	86.16	85.96	22	83.27	81.91
11	90.40	88.76	23	75.93	76.27
12	89.53	88.87	24	68.55	68.63

below.

The comparison of the prediction curves generated, based on the SSA-LSTM prediction model, with the true value curves of the load data is shown in Figure 9.

(3)

Predictive model based on ISSA-LSTM

An ISSA-LSTM-based STLF model is developed. After the algorithm is run, the adaptation curve can be obtained as shown in Figure 10. The curve indicates that with the ISSA seeking, the algorithm can search for a set of hyperparameters. When the network is trained using this set of hyperparameters, it can make its error lower, so it is a decreasing curve.

In addition, the change curves of the hyperparameters of the model at the end of the run are illustrated in Figure 11, from which it can be observed that the values of the four hyperparameters of the model eventually converge to a certain value, as the number of iterations increases. The specific values of each hyperparameter are given in

Table 6. The result of ISSA search for hyperparameters.

Hyperparameters	Value
L1	87
L2	260
iter	232
lr	0.0083

.

The LSTM network was rebuilt using the above hyperparameters to obtain the prediction results. The model-prediction result data and the real value data are given in

Table 7. Comparison of ISSA-LSTM model output values with real values.

Serial No.	True Value (MW)	Predicted Values (MW)	Serial No.	True Value (MW)	Predicted Values (MW)
1	59.61	62.69	13	76.44	77.05
2	57.95	56.24	14	82.43	84.90
3	55.66	55.79	15	84.79	85.03
4	53.91	53.50	16	86.20	86.05
5	52.89	53.69	17	85.94	88.07
6	55.02	53.64	18	87.40	85.19
7	60.14	59.10	19	85.19	84.65
8	67.52	66.60	20	87.81	88.97
9	80.02	79.66	21	84.95	87.13
10	86.16	85.04	22	83.27	81.70
11	90.40	88.61	23	75.93	75.38
12	89.53	88.24	24	68.55	68.38

.

The comparison of the prediction curves generated, based on the ISSA-LSTM prediction model, with the true value curves of the load data is shown in Figure 12.

6.3. Load-Forecasting-Model Experiment

The prediction results of six models, LSTM, SSA-LSTM, ISSA-LSTM, PSO-LSTM [23], PSO-BP [24], and PSO-LSSVM [25], were put together for comparison, and then the prediction index values of each of the six models were calculated. The results of the model comparison built in this paper are illustrated in Figure 13 below. As can be observed from Figure 13 below, the ISSA-LSTM model has the most similar fluctuation magnitude to the actual value and achieves good accuracy in peak prediction. Although the prediction effect of some points is slightly inferior to other models, the overall prediction accuracy and stability are higher, and the fit is better.

The calculation results of the prediction index and prediction accuracy of each model are given in

Table 8. Model-prediction index and accuracy-calculation results.

Predictive Models	MAPE	RMSE	Prediction Accuracy
LSTM	0.0296	2.6337	97.04%
SSA-LSTM	0.0196	1.6956	98.04%
ISSA-LSTM	0.0158	1.4204	98.42%
PSO-LSTM	0.0273	2.8022	97.27%
PSO-BP	0.0322	3.0263	96.78%
PSO-LSSVM	0.0331	3.1177	96.69%

. As can be seen from the data in Table 8, that the MAPE and RMSE of the model built by LSTM, in combination with the PSO, SSA, and ISSA, are further reduced, and the prediction accuracy is improved. In addition, among the six models, the forecasting accuracy of the ISSA-LSTM model proposed in this paper is the highest.

7. Conclusions

A series of improvements are made to the SSA, which has been improved to improve its search performance significantly and can effectively get rid of the local optimum. Firstly, sparrow-position initialization is very important for global search, and this paper uses Sin-chaos initialization to initialize the population, thus enriching the diversity of solutions. Secondly, this paper introduces the dynamic adaptive-weight factor, which effectively balances the local and global-excavation ability of the algorithm. Finally, this paper integrates the Cauchy variation and OBL strategies, thus reducing the probability that the algorithm cannot get out of the local extremes, thus enhancing the global exploration ability of the algorithm.

An ISSA based on fused Cauchy variance and OBL is proposed to optimize the combined-forecasting method of LSTM hyperparameters with it, and an ISSA-LSTM-based STLF model is developed. ISSA-LSTM reduces the influence of human factors on LSTM and improves the ability of the model to capture the characteristics of power-load data. The experimental results indicate that the proposed model has higher forecasting accuracy compared with the LSTM, SSA-LSTM, PSO-LSTM, PSO-BP and PSO-LSSVM models, which provides a new idea for STLF.

The next step can be considered to continue to improve the optimization mechanism and algorithm structure of the sparrow algorithm, or try to integrate the advantages of other intelligent algorithms to propose intelligent algorithms with better performance for load forecasting. In addition, we have read a lot of the literature from other fields [26,27,28,29,30,31], and in the future we can consider how to incorporate them into our research topics.