A Hybrid PSO-DE Intelligent Algorithm for Solving Constrained Optimization Problems Based on Feasibility Rules

Abstract

In this paper, we study swarm intelligence computation for constrained optimization problems and propose a new hybrid PSO-DE algorithm based on feasibility rules. Establishing individual feasibility rules as a way to determine whether the position of an individual satisfies the constraint or violates the degree of the constraint, which will determine the choice of the individual optimal position and the global optimal position in the particle population. First, particle swarm optimization (PSO) is used to act on the top 50% of individuals with higher degree of constraint violation to update their velocity and position. Second, Differential Evolution (DE) is applied to act on the individual optimal position of each individual to form a new population. The current individual optimal position and the global optimal position are updated using the feasibility rules, thus forming a hybrid PSO-DE intelligent algorithm. Analyzing the convergence and complexity of PSO-DE. Finally, the performance of the PSO-DE algorithm is tested with 12 benchmark functions of constrained optimization and 57 engineering optimization problems, the numerical results show that the proposed algorithm has good accuracy, effectiveness and robustness.

1. Introduction

Intelligent optimization algorithms are widely used in engineering design, job scheduling, aerospace, intelligent control, traffic optimization, financial investment, network communication and other fields. Therefore, research on optimization methods has important academic significance and practical value. The current existing optimization problems are often restricted by different conditions, which are called constrained optimization problems (COPs). The mathematical model can be expressed as follows:

$$\left\{\begin{array}{cc}min\hfill & f\left(x\right)\hfill \\ s.t\hfill & g_j\le 0, j=1,2,\dots ,n\hfill \\ & h_j=0, j=q+1,2,\dots ,m\hfill \\ & L_i\le x_i\le U_i,i=1,2,\dots ,D\hfill \end{array}\right.$$

(1)

where $x=(x_1,x_2,\dots ,x_D)$ denotes the D-dimensional vector of decision variables, $S={\prod }_{i=1}^D$ is the decision space, which is the set of solutions satisfying all constraints, $L_i$ and $U_i$ are the minimum and the maximum permissible values for the i-th variable, $f\left(x\right)$ is the objective function, $g_j$ is called the inequality constraint, $h_j$ is called the equality constraint, q is the number of inequality constraints, $(m-q)$ is the number of equality constraints. A feasible solution satisfies all constraints, and an unfeasible solution does not satisfy at least one constraint.

Constrained evolutionary algorithm is the combination of constraint handing technology and evolutionary algorithms, which can effectively solve constrained optimization problems. Evolutionary algorithms have strong universality, reliability, robustness, less information requirements for problems and easy to realize, so it is widely used to solve constrained optimization problems [1,2,3,4,5]. Meanwhile, a series of constraint handing techniques based on algorithm have been proposed: (1) Penalty function method [6], (2) Feasibility rules [7], (3) Multi-objective method [8], etc. Among them, penalty function method is the most common and simplest method to deal with constraints at present, which is essentially to add (or subtract) a penalty term to the objective function, and transform the constrained optimization problem into the unconstrained optimization problem. The penalty term $G\left(x\right)={\sum }_{j=1}^m{G}_j\left(x\right)$ is based on the degree of constraint violation of an individual x, $G_j\left(x\right)$ is defined as:

$$\begin{array}{c}\hfill G_j\left(x\right)=\left\{\begin{array}{cc}& \max \left(0,g_j\left(x\right)\right),\hspace{1em}1\le j\le q\hfill \\ & \max (0,\left|h_j\left(x\right)\right|-\delta ),\hspace{1em}q+1\le j\le m\hfill \end{array}\right.\end{array}$$

(2)

where $\delta$ is the relaxation quantity of the equation constraint, usually a small number. If the constraint of variable x violates degree $G\left(x\right)=0$, the variable x is called a feasible solution, otherwise, it is called an unfeasible solution. The feasible solution with the smallest target value is called the optimal feasible solution of the constraint optimization problem, so the main purpose of solving the constraint optimization problem is to find this optimal feasible solution.

Evolutionary algorithms (EAs) are “clusters of algorithms”, which are based on stochastic iteration and evolution by natural selection and genetics. This is achieved by simulating the natural principle of “survival of the fittest” and generating new individuals by mutating, crossing and selecting individuals in a population. The new individuals are compared with those of the previous generation to retain the better ones, so that the whole population keeps moving closer to the optimal solution region. The characteristics of EAs are that it has no differentiable and convexity requirements for the objective function, and it has a strong global search ability by conducting parallel search through the population information. Currently, the common and widely used EAs are: genetic algorithm (GA) [9], particle swarm optimization (PSO) [10], immune algorithm (IA) [11], ant colony optimization (ACO) [12], dfferential evolution (DE) [13], shuffled frog leaping algorithm (SFLA) [14], artificial bee colony (ABC) algorithm [15], biogeography-based optimization (BBO) [16], cuckoo search (CS) algorithm [17], grey wolf optimization (GWO) [18], butterfly optimization algorithm (BOA) [19], Harris hawks optimization (HHO) [20], marine predator algorithm (MPA) [21], honey badger algorithm (HBA) [22], etc. Among them, PSO is an intelligent algorithm proposed by Kennedy and Eberhart in 1995, inspired by the research results of modeling and simulation of bird group behavior. Through the transmission of information between individuals, the whole group is guided towards the most likely solution. PSO has the advantages of simplicity, easy to understand and fewer parameters, so it works well in solving many practical application problems. However, it is easy to fall into local optimum and difficult to jump out, so it is not a good global search due to the fast convergence of diversity in some problems. DE was proposed by Storn et al. in 1995, which was originally an idea to solve the Chebyshev polynomial problem, and was later used to solve complex optimization problems with continuous improvement. The unique memory capability of the algorithm allows it to dynamically track the current search situation to adjust its search strategy, which leads to a strong global convergence and robustness of the algorithm. Each algorithm has its own advantages and disadvantages and is suitable for solving different types of problems. As the “No free lunch theorem” [23] points out, it is difficult for a single algorithm to perfectly solve all problems.

The hybrid evolution algorithm is to make the algorithms cooperate with each other to improve the ability to solve optimization problems, so it has attracted the attention of many scholars and has been linked to proposed hybrid algorithms [24,25,26,27,28,29,30,31]. In order to design better hybrid evolutionary algorithm, it is necessary to understand the strengths and weaknesses of each evolutionary algorithm and effectively balance the relationship between exploration and exploitation of the algorithm in the search process to obtain the optimal search capability. Exploration and exploitation are essentially contradictory to each other; if differential evolution performs well in exploration, it will perform weakly in exploitation [32]. The reverse is also true. The particle swarm optimization is a population-based algorithm, which tends to converge quickly to a local optimum in the face of multimodal functions, thus missing the opportunity to converge to a global optimum. In order to improve the advantages of this algorithm and make up for its shortcomings, this paper proposes a hybrid particle swarm-differential evolutionary algorithm (PSO-DE) for solving constrained optimization problems by combining the evolutionary characteristics of DE, and applying it to 12 classical constrained benchmark functions and 57 engineering optimization problems.The numerical results show that PSO-DE has good stability, robustness and global search ability.

The rest of this article is arranged as: Section 2 briefly describes PSO and DE; Section 3 proposes a hybrid particle swarm optimization and differential evolution (PSO-DE); Section 4 analyzes the complexity of PSO-DE; Section 5 proves the convergence of PSO-DE; Section 6 presents simulation experiments and comparison of results; Section 7 is the conclusion. Figure 1 presents a graphical abstract of this paper.

2. Related Work

2.1. Particle Swarm Optimization

Particle swarm optimization (PSO) is an intelligent algorithm that imitates the foraging behavior of birds. In this algorithm, the process of finding the optimal solution of the problem is regarded as the process of birds foraging, and the flight space of birds is compared to the search space of the solution. Each bird is abstracted into a particle without quality and velocity, which is used to represent a possible solution of the problem, and then solve the complex optimization problem.

In PSO, the particle swarm is composed of NP particles. Since particles have no quality and velocity, the swarm can extend to D-dimensional space. Each particle flies at a speed and constantly approaches the individual optimal position and global optimal position. In the process of optimization, the velocity and position of each particle are updated according to Equations (3) and (4):

$$v_i=w{v}_i+c_1·rand·(p_i-x_i)+c_2·rand·(g-x_i)$$

(3)

$$x_i=x_i+v_i$$

(4)

where $i=1,2,\dots ,NP$. $p_i$ represents the individual optimal position of the i particle, $p_{i}best$ is the value of the fitness for the optimal position of the individual, where $p_{i}best=f\left(p_i\right)$. g represents the global optimal position, w is the inertia weight. $x_i$ and $v_i$ respectively represent the position and velocity of the i particle in the current iteration number. $x_{id}\in [-x_{max},x_{max}]$, $x_{max}$ is a constant limiting the position of the particle, $v_{id}\in [-v_{max},v_{max}]$, $v_{max}$ is a constant limiting the particle velocity. $c_1$ and $c_2$ are self-knowledge coefficient and social cognition coefficient, respectively, also known as acceleration constant, rand is a random number in the range of $[0,1]$ that follows a normal distribution, which increases the randomness of particle motion. The velocity update formula consists of three parts: The first part is “inertia” or “momentum”, and the motion “habit” of reacting particle represents particle’s tendency to maintain previous speed. The second part is “cognition”, which reflects particle’s memory or recollection of its own historical experience, representing particle’s tendency to move closer to its optimal position. The third part is “social”, reflecting the historical experience of groups of cooperative cooperation and knowledge sharing between particles, representing the tendency of particles to move closer to best position in history of group or field. Figure 2 shows the updated diagram of positions.

Based on the concepts of “population” and “evolution”, particle swarm optimization realizes the search for complex spatial optimal solutions through cooperation and competition between individuals, and its process is as follows:

Step 1: The position and velocity of each particle is generated through Equations (5) and (6).

$$x=rand(N,D)\ast (X_{max}-X_{min})+X_{min},$$

(5)

$$v=rand(N,D)\ast (V_{max}-V_{min})+V_{min}.$$

(6)

Step 2: Calculate the fitness value for each particle $f\left(x_i\right)$.

Step 3: Compare the fitness value $f\left(x_i\right)$ with the individual optimal value $p_{i}best$, if $f\left(x_i\right)<p_{i}best$, then $p_{i}best=f\left(x_i\right)$.

Step 4: Compare the fitness value $f\left(x_i\right)$ with the global optimal value $gbest$, if $f\left(x_i\right)<gbest$, then $gbest=f\left(x_i\right)$.

Step 5: Update the velocity and position of each particle by Equations (3) and (4).

Step 6: The boundary conditions are handled by Equations (7) and (8),

$$\begin{array}{c}{x}_{ij}=\left\{\begin{array}{cc}& rand\ast (X_{max}-X_{min})+X_{min},\hspace{1em}if x_{ij}>X_{max} or x_{ij}<X_{min},\hfill \\ & x_{ij},\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}otherwise.\hfill \end{array}\right.\hfill \end{array}$$

(7)

$$\begin{array}{c}{v}_{ij}=\left\{\begin{array}{cc}& rand\ast (V_{max}-V_{min})+V_{min},\hspace{1em}if v_{ij}>V_{max} or v_{ij}<V_{min},\hfill \\ & v_{ij},\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}otherwise.\hfill \end{array}\right.\hfill \end{array}$$

(8)

Step 7: If the termination condition is met, the algorithm is terminated and the optimization result is output, otherwise, go back to Step 2.

Corresponding to the flow of the Algorithm 1, the basic framework of the particle swarm optimization is shown in Figure 3.

Algorithm 1 The pseudocode of particle swarm optimization

1: Initialize PSO parameters, which involve $\mathit{Xmax}$, $\mathit{Xmin}$, $\mathit{Vmin}$,$\mathit{Vmax}$, $\mathit{c}\mathit{1}$,$\mathit{c}\mathit{2}$,

and a random set of $NP$ individuals.

2: According to Equations (5) and (6), the position and velocity of the individuals of the

initial population are randomly generated.

3: Calculate the fitness value of individual population $\mathit{f}\left({\mathit{x}}_{\mathit{i}}\right)$.

4: Set individual optimal position ${\mathit{p}}_{\mathit{i}}$ and ${\mathit{p}}_{\mathit{i}}\mathit{best}$.

5: Set global optimal position $\mathit{g}$ and $\mathit{gbest}$;

6:  While (criterion)

7:        for $\mathit{i}=\mathbf{1},\mathbf{2},\dots \dots \mathit{NP}$ do

8:             Generate new velocity ${\mathit{v}}_{\mathit{i}}(\mathit{t}+\mathbf{1})$ using Equation (3);

9:             Generate new locations ${\mathit{x}}_{\mathit{i}}(\mathit{t}+\mathbf{1})$ using Equation (4);

10:             Evaluate fitness value at new locations $\mathit{f}\left({\mathit{x}}_{\mathit{i}}(\mathit{t}+\mathit{1})\right)$;

11: (Update  individual  optimal)

12:                 if     $\mathit{f}\left({\mathit{x}}_{\mathit{i}}(\mathit{t}+\mathbf{1})\right)\le {\mathit{p}}_{\mathit{i}}\mathit{best}$   then

13:                          ${\mathit{p}}_{\mathit{i}}(\mathit{t}+\mathbf{1})={\mathit{x}}_{\mathit{i}}(\mathit{t}+\mathbf{1})$,${\mathit{p}}_{\mathit{i}}\mathit{best}(\mathit{t}+\mathbf{1})=\mathit{f}\left({\mathit{x}}_{\mathit{i}}(\mathit{t}+\mathbf{1})\right)$;

14:                 else

15:                          ${\mathit{p}}_{\mathit{i}}(\mathit{t}+\mathbf{1})={\mathit{p}}_{\mathit{i}}\left(\mathit{t}\right)$,${\mathit{p}}_{\mathit{i}}\mathit{best}={\mathit{p}}_{\mathit{i}}\mathit{best}(\mathit{t}+\mathbf{1})$;

16:                 end (if)

17: (Update  global  optimal)

18:                 if     ${\mathit{p}}_{\mathit{i}}\mathit{best}\le \mathit{gbest}$  then

19:                          $\mathit{g}(\mathit{t}+\mathbf{1})={\mathit{p}}_{\mathit{i}}(\mathit{t}+\mathbf{1})$,$\mathit{gbest}(\mathit{t}+\mathbf{1})={\mathit{p}}_{\mathit{i}}(\mathit{t}+\mathbf{1})$;

20:                 else

21:                          $\mathit{g}(\mathit{t}+\mathbf{1})=\mathit{g}\left(\mathit{t}\right)$,$\mathit{gbest}(\mathit{t}+\mathbf{1})=\mathit{gbest}\left(\mathit{t}\right)$;

22:                 end(if)

23:        end (for)

24:  end(while)

25: Output

2.2. Differential Evolution

Differential Evolution (DE) is a random heuristic search algorithm, which has the characteristics of memorizing the optimal solution of individuals and sharing information within a population. DE is similar to GA, and the main operations of population renewal are mutation, crossover and selection, as follows:

(1) Initialization

The initial population is usually randomly selected from within a given boundary constraint. In general, the randomly generated population follows uniform probability distribution. $x_j\left(L\right)\le x_j\le x_j\left(U\right),(j=1,2,\dots ,D)$ is the range of the parameter variables, then

$$x_{id}=rand[0,1]·(x_{ij}\left(U\right)-x_{ij}\left(L\right))+x_{ij}\left(L\right)$$

(9)

where $i=1,2,\dots ,NP$; rand is a random number in the range of $[0,1]$.

(2) Mutation operation

For each decision vector $x_i(i=1,2,\dots ,NP)$, a new individual $v_i$ is generated by the following mutation operation:

$$DE/rand/1:v_i=x_{r1}+F·(x_{r2}-x_{r3})$$

(10)

$$DE/best/1:v_i=x_{best}+F·(x_{r1}-x_{r2})$$

(11)

$$DE/currenttobest/1:v_i=x_i+F·(x_{best}-x_i)+F·(x_{r1}-x_{r2})$$

(12)

$$DE/best/2:v_i=x_{best}+F·(x_{r1}-x_{r2})+F·(x_{r3}-x_{r4})$$

(13)

$$DE/best/2:v_i=x_{best}+F·(x_{r1}-x_{r2})+F·(x_{r3}-x_{r4})$$

(14)

where $x_{r1},x_{r2},x_{r3},x_{r4},x_{r5}$ are randomly selected from the population and $i\ne r1\ne r2\ne r3\ne r4\ne r5$, $F(F\in [0,2\left]\right)$ is an amplification factor.

(3) Crossover operation

To improve population diversity, the following formula is used to cross-operate, and $u_i=(u_{i1},u_{i2},\dots ,u_{iD})$ as experimental variables:

$$\begin{array}{c}\hfill u_{ij}=\left\{\begin{array}{cc}& v_{ij}, if randb\left(j\right)\le CR or j=rnbr\left(i\right),\hfill \\ & x_{ij}, if randb\left(j\right)\ge CR and j\ne rnbr\left(i\right),\hfill \end{array}\right.\end{array}$$

(15)

where $randb\left(j\right)$ represents the j-th estimate generated randomly between [0,1], $j=rnbr\left(i\right)\in (1,2,\dots ,D)$ represents a randomly selected sequence to ensure $u_{ij}$ gets at least one parameter from $v_{ij}$, $CR(CR\in [0,1\left]\right)$ is a crossover probability.

(4) Select operation

In order to ensure that individuals with better fitness enter the next generation, the “greedy” strategy is adopted for selection operation. By comparison, the superior individuals in $u_i$ and $v_i$ are retained to the next generation, and the inferior individuals are eliminated.

The differential evolution algorithm (Algorithm 2) uses real number coding, a crossover operation based on differential simple mutation operation and “one-to-one” competitive survival. The specific steps of the strategy are as follows:

Step 1: Determine the parameters of the differential evolution algorithm: number of populations, mutation operator, crossover operator, maximum number of iterations, termination conditions.

Step 2: The initial population is randomly generated, and let the number of evolution k = 1.

Step 3: Fitness values are calculated for the individuals in the initialized population.

Step 4: Determine whether the termination condition is met or the maximum number of iterations is reached: if yes, the evolution is terminated and the best individual at this point is taken as the output; otherwise, the next operation is continued.

Step 5: Mutation operations and crossover operations are performed to process the boundary conditions and obtain temporary populations.

Step 6: Adaptation values are calculated for individuals in the provisional population.

Step 7: The new population is obtained by “one-to-one” selection of the individuals of the temporary population and the corresponding individuals of the original population.

Step 8: Let the number of evolution k = k + 1 and turn to Step (4).

Corresponding to the flow of the above algorithm, the basic framework of the differential evolution algorithm is shown in Figure 4.

Algorithm 2 The pseudocode of Differential Evolution

1: Initialize DE parameters, which involve $\mathit{F}$, $\mathit{CR}$,

and a random set of $NP$ individuals.

2: Generate the initial population according to Equation (5).

3: Calculate the fitness value of individual population $\mathit{f}\left({\mathit{x}}_{\mathit{i}}\right)$.

4:   While (criterion).

5:        for $\mathit{i}=\mathbf{1},\mathbf{2},\dots \dots \mathit{NP}$ do;

6:            Select random indexes ${\mathit{r}}_{\mathbf{1}},{\mathit{r}}_{\mathbf{2}},{\mathit{r}}_{\mathbf{3}}$, where ${\mathit{r}}_{\mathbf{1}}\ne {\mathit{r}}_{\mathbf{2}}\ne {\mathit{r}}_{\mathbf{3}}$

7: (Mitation  operation)

8:               ${\mathit{v}}_{\mathit{i}}={\mathit{x}}_{\mathit{r}\mathbf{1}}+\mathit{F}·({\mathit{x}}_{\mathit{r}\mathbf{2}}-{\mathit{x}}_{\mathit{r}\mathbf{3}})$

9:              for $\mathit{j}=\mathbf{1},\mathbf{2},\dots ,\mathit{D}$ do

10:                $\mathit{randb}\left(\mathit{j}\right)=\mathit{rand}(\mathbf{0},\mathbf{1}),\mathit{j}=\mathit{rnbr}\left(\mathit{i}\right)\in (\mathbf{1},\mathbf{2},\dots ,\mathit{D})$

11: (Crossover  operation)

12:                     if  $\mathit{randb}\left(\mathit{j}\right)\le \mathit{CR}\mathit{or}\mathit{j}=\mathit{rnbr}\left(\mathit{i}\right)$ then

13:                              ${\mathit{u}}_{\mathit{ij}}={\mathit{v}}_{\mathit{ij}}$

14:                     else

15:                             ${\mathit{u}}_{\mathit{ij}}={\mathit{x}}_{\mathit{ij}}$

16:                     end(if)

17: (Greedy  selection)

18:                     if  $\mathit{f}\left({\mathit{u}}_{\mathit{i}}\right)\le \mathit{f}\left({\mathit{x}}_{\mathit{i}}\right)$ then

19:                             ${\mathit{x}}_{\mathit{t}+\mathbf{1}}={\mathit{u}}_{\mathit{t}}$

20:                     else

21:                             ${\mathit{x}}_{\mathit{t}+\mathbf{1}}={\mathit{x}}_{\mathit{t}}$

22:                    end (if)

23:             end(for)

24:        end(for)

25:   end (While)

26: Output

3. Hybrid Particle Swarm Optimization and Differential Evolution (PSO-DE)

In this section, we describe the main steps of the proposed PSO-DE algorithm, as shown in Section 3.4 and the flowcharts in Figure 5 and Figure 6. Before describing the main steps of the proposed algorithm, we focus on how to deal with constraints when the variables violate the constraints by the proposed algorithm.

3.1. Constraint Handling Technology

To ensure that the optimal solution obtained optimizes both the objective function and satisfies all constraints, the proposed PSO-DE is combined with a constrained optimization handling technique in this paper. The penalty function method is currently the most common constraint treatment method. The main idea is that the constrained optimization problem is transformed into an unconstrained optimization problem by subtracting or adding a penalty term to the value of the objective function of the infeasible solution. Among them, the penalty term is usually obtained by multiplying the penalty factor by the degree of constraint violation. Faced with different types of constrained optimization problems, how do we determine the appropriate penalty factor to make the solution of infeasible solutions not overly penalized or too lightly penalized? This is the main challenge in solving constrained optimization problems using the penalty function method. On the other hand, the feasibility rule (Deb) is a more intuitive way to compare the feasibility of two solutions without determining any parameters such as penalty factors. Comparing two solutions using the following principles: (1) any feasible solution takes precedence over any infeasible solution, (2) if both solutions are feasible, the feasible solution is selected with the better objective function value, (3) between two infeasible solutions, the solution with the less constraint violation takes preference. We combine Deb’s rule into a constraint treatment for PSO-DE, assuming that the fitness and total constraint violation of PSO-DE particles are $f\left(x_i\right)$ and $G\left(x_i\right)$. Correspondingly, for the constraint minimization problem, the two solutions $x_i$ and $x_j$ are compared according to the following criterion:

(i) If $G\left(x_i\right)=0$ and $G\left(x_j\right)=0$, $f\left(x_i\right)<f\left(x_j\right)$, the individual $x_i$ is reserved;

(ii) If $G\left(x_i\right)=0$, $G\left(x_j\right)\ne 0$, the individual $x_i$ is reserved;

(iii) If $G\left(x_i\right)\ne 0$, $G\left(x_j\right)\ne 0$, $G\left(x_i\right)<G\left(x_j\right)$ the individual $x_i$ is reserved.

Algorithm 3 is used to compare the current solution $x_i$ of the i particle of PSO-DE with the individual best position $p_i$. The same pseudocode can be used to compare the feasibility between the solution $x_i$ and the global best position g, and then Deb to determine whether the new position $x_i$ generated by the first particle can replace $p_i$ and g. By combining Deb with PSO-DE, the population can be guided to search from infeasible region to feasible region and optimize the objective function in the feasible region.

Algorithm 3 Deb’s Rule

1: Input ${\mathit{x}}_{\mathit{i}},\mathit{f}\left({\mathit{x}}_{\mathit{i}}\right),\mathit{G}\left({\mathit{x}}_{\mathit{i}}\right),{\mathit{p}}_{\mathit{i}}\mathit{best}=\mathit{f}\left({\mathit{p}}_{\mathit{i}}\right),\mathit{G}\left({\mathit{p}}_{\mathit{i}}\right)$,

2:       if   $\mathit{G}\left({\mathit{x}}_{\mathit{i}}\right)=\mathbf{0}\mathit{and}\mathit{G}\left({\mathit{p}}_{\mathit{i}}\right)=\mathbf{0}$ then

3:             else if  $\mathit{f}\left({\mathit{x}}_{\mathit{i}}\right)<{\mathit{p}}_{\mathit{i}}\mathit{best}$ then

4:                           ${\mathit{p}}_{\mathit{i}}={\mathit{x}}_{\mathit{i}},{\mathit{p}}_{\mathit{i}}\mathit{best}=\mathit{f}\left({\mathit{x}}_{\mathit{i}}\right),\mathit{G}\left({\mathit{p}}_{\mathit{i}}\right)=\mathit{G}\left({\mathit{x}}_{\mathit{i}}\right),$

5:                else

6:                           ${\mathit{p}}_{\mathit{i}}={\mathit{p}}_{\mathit{i}},{\mathit{p}}_{\mathit{i}}\mathit{best}=\mathit{f}\left({\mathit{p}}_{\mathit{i}}\right),\mathit{G}\left({\mathit{p}}_{\mathit{i}}\right)=\mathit{G}\left({\mathit{p}}_{\mathit{i}}\right),$

7:              end (else if)

8:        else

9:                            $\mathit{G}\left({\mathit{x}}_{\mathit{i}}\right)\ne \mathbf{0}\mathit{and}\mathit{G}\left({\mathit{p}}_{\mathit{i}}\right)\ne \mathbf{0}$

10:             if $\mathit{G}\left({\mathit{x}}_{\mathit{i}}\right)<\mathit{G}\left({\mathit{p}}_{\mathit{i}}\right)$ then

11:                            ${\mathit{p}}_{\mathit{i}}={\mathit{x}}_{\mathit{i}},{\mathit{p}}_{\mathit{i}}\mathit{best}=\mathit{f}\left({\mathit{x}}_{\mathit{i}}\right),\mathit{G}\left({\mathit{p}}_{\mathit{i}}\right)=\mathit{G}\left({\mathit{x}}_{\mathit{i}}\right),$

12:              else

13:                            ${\mathit{p}}_{\mathit{i}}={\mathit{p}}_{\mathit{i}},{\mathit{p}}_{\mathit{i}}\mathit{best}=\mathit{f}\left({\mathit{p}}_{\mathit{i}}\right),\mathit{G}\left({\mathit{p}}_{\mathit{i}}\right)=\mathit{G}\left({\mathit{p}}_{\mathit{i}}\right),$

14:              end (if)

15:       end(if)

3.2. Applying PSO to the Top 50% of Individuals with a High Degree of Violation

In this section, we will describe PSO-DE in detail. In the particle swarm velocity update equations, the first part is “inertia” or “momentum”, where the reactive particles have the tendency to maintain their previous velocity, which is used to ensure the global convergence performance of the algorithm. The second part is “cognitive”, which represents the tendency of the particle to approach to its previous optimal position. The third part is the “social” part, which represents the tendency of particle towards global optimal position by cooperating and sharing knowledge among the particles. The “cognitive” and “social” components lead to local convergence of the algorithm. When the particle position is close to the global best position, the distance between particle and personal optimal position and global optimal position will be small; at that time, the velocity is influenced by the “inertia” part, and less by the “cognitive” and “social” parts, which leads to a negligible change in the particle velocity, and thus, the particle position will not change much, in which case the particle may stagnate and fall into the local optimum. This indicates that the initial personal optimal position and the global optimal position are playing an important role in the performance of the algorithm.

Based on the feasibility rule, if the degree of particle violation is lower, the probability of particles clustering around the global optimal position is higher, which may cause the global optimal position to stay at the same position for a long time, which makes it difficult for particles to jump out of a certain neighborhood of global optimal position and lose population diversity. In other words, the algorithm may converge prematurely in the early stages of the optimization search process, and if the population converges too quickly to a position that may be the local optimal position, the particles may also give up trying to explore other optimal positions and stagnate in the rest of the evolutionary process. On the other hand, for particles with a high degree of constraint violation, there is a remarkable difference between its personal optimal position and global optimal position, it will extract beneficial information from both the personal optimal position and the global best position belonging to the same population, dragging it toward better performing points, and updating particles that may be better than the current global optimal position, causing the global optimal position to jump to a new position different from the current position. This substitution may stimulate the particle population to adjust its evolutionary direction and guide particles to new regions that have not been searched before, so as to prevent particles from falling into local optimum. Therefore, we rank the constraint violation degree of population individuals in descending order in this paper, and take out the top 50% of individuals with higher constraint violation degree as temporary population, which apply particle swarm optimization to the temporary population, and this mechanism slows down the convergence speed and avoids stagnation to some extent.

3.3. Updating Individual Optimal with DE

After each iteration of the algorithm, the global optimal positions are updated using the feasibility rules. At the same time, the particle swarm optimization is easily trapped in the local optimum. In order to improve the global optimal finding ability of the algorithm, people have continuously tried to integrate the DE strategy into the PSO and achieved good results, so this paper also applies the DE strategy to update individuals’ optimal positions in the PSO. This strategy increases the possibility of finding a more optimal position in order to increase the chances of finding global optimal position in case it is not yet determined. It is well known that individual optimal position and global optimal position guide particles in the best direction and speed up convergence. These two populations work independently of each other, but individuals between them are interrelated. PSO can gradually search the neighborhood of best position so far, and the differential evolution algorithm (Algorithm 4) can avoid convergence to a local optimum. In summary, there is a great balance between accuracy and efficiency by hybridizing PSO and DE.

In the process of PSO-DE, firstly, the initial population is randomly generated ($pop$), which is copied and assigned to personal optimal positions $P=pop$, P is used to store the current personal best position of each particle, and $PBest$ is used to store the personal optimal value of each particle. Secondly, the constraint violation degree value of each individual in the population is calculated and sorted in descending order, where the top 50% of individuals with a higher violation degree are taken as temporary population. PSO is applied to the temporary population to update positions of selected individuals and corresponding individuals optimal position. The position order between each particle and corresponding individual optimal position is kept consistent, when the order of individuals in the population changes, the order of the corresponding individual optimal position will also change. Finally, the mutation operation, crossover operation and selection operation of DE are applied to the individual optimal positions corresponding to the individuals in the population pop, and the global optimal position is updated by the feasibility rules. Figure 5 shows the flow chart of PSO-DE.

3.4. Steps of the PSO-DE

Step 1: Denote Population Size $NP$, fitness function $f\left(x\right)$, violation degree function $G\left(x\right)$, upper bound $Xmax$ and lower bound $Xmin$ of the decision variables, maximum iterations T.

Step 2: Calculate the degree of violation $G\left(x\right)$ of the individuals in $pop$, while sorting the individuals and the correspondent personal optimal position in descending order, and the top $50\%$ of the sorted individuals as a temporary population $P1$, $P1=(x_1,x_2,\dots ,x_{NP/2})$, corresponding to the personal best value of the individuals $P1Best=(p_{1}best,p_{2}best,\dots ,p_{NP/2}best)$.

Step 3: Updating the individuals $P1$ in according to Equations (4) and (5), after the update, $P1=a_1,a_2,\dots ,a_{NP/2}$. Boundary handling is applied to $x_{ij}$ in excess of $[X_{min},X_{max}]$ and $v_{ij}$ in excess of $[V_{min},V_{max}]$ as follows:

$$\begin{array}{c}\hfill x_{ij}=\left\{\begin{array}{cc}& 0.5\times (X_{min}+x_{ij}),\hspace{1em}{x}_{ij}\le X_{min},\hfill \\ & 0.5\times (X_{max}+x_{ij}),\hspace{1em}{x}_{ij}\ge X_{max},\hfill \\ & x_{ij},\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}otherwise.\hfill \end{array}\right.\end{array}$$

(16)

$$\begin{array}{c}\hfill v_{ij}=\left\{\begin{array}{cc}& rand\times (V_{max}-X_{min}),\hspace{1em}{v}_{ij}\le X_{min} or v_{ij}\ge X_{max}\hfill \\ & v_{ij},\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}otherwise.\hfill \end{array}\right.\end{array}$$

(17)

Step 4: Calculate the fitness function values  $f\left(a_i\right)$ and violation values $G\left(a_i\right)$ of individuals after the update of the temporary population $P1$.

Step 5: Compare $a_i$ and $p_i$ according to the feasibility rule, and if $a_i$ is better, replace $p_i$.

Step 6: Update $gbest$ according to the feasibility principle.

Step 7: Introduce DE strategy to update $PBest$ as follows:

     Step 7.1: Mutation operation: the i-th particle’s personal optimal position $p_i, i=1,2,\dots ,NP/2$, two different $r1,r2\in 1,2,\dots ,NP/2$ are randomly selected, where $v_i$ is the intermediate variable formed by the variation operation and F is the factor of scaling.

      Step 7.2: Crossover operation:

$$\begin{array}{c}\hfill u_{ij}=\left\{\begin{array}{cc}& v_{ij}\hspace{1em}ran{d}_j\le CR or j=j_{rand}\hfill \\ & p_{ij},\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}otherwise.\hfill \end{array}\right.\end{array}$$

(18)

where $u_{ij}$ is the intermediate variable formed by the crossover operation, $CR$ is the crossover probability and $j_{rand}\in 1,2,\dots ,n$ is the random number; the boundary is handled using the following equation:

$$\begin{array}{c}\hfill u_{ij}=\left\{\begin{array}{cc}& v_{ij}, if randb\left(j\right)\le CR or j=rnbr\left(i\right),\hfill \\ & x_{ij}, if randb\left(j\right)\ge CR and j\ne rnbr\left(i\right),\hfill \end{array}\right.\end{array}$$

(19)

where $rand$ is a normally distributed random number between $[0,1]$.

      Step 7.3: Using the feasibility principle to compare $u_i$ and $p_i$, update $p_i$ and $PBest$.

Step 8: By comparing the individuals in $PBest$ through the feasibility principle, select the optimal individual and updating $g,f_{best}=f\left(g\right)$.

Step 9: If the termination condition is satisfied, the global optimal individual is output as the optimal solution and the algorithm is terminated, otherwise turn to Step 2.

Algorithm 4 The pseudocode of PSO-DE

1: Initialize PSO and DE parameters

2: From Equations (5) and (6), the position and velocity of the individuals of

the initial population $pop$ are randomly generated.

3: Calculate fitness function $\mathit{f}\left({\mathit{x}}_{\mathit{i}}\right)$ and constrained violation ${\mathit{G}}_{(}{\mathit{x}}_{\mathit{i}})$ of individual.

4:     Set $\mathit{P}=\mathit{pop}$ and $\mathit{PBest}$

5:     Set $\mathit{g}$ and $\mathit{gbest}$

6:   While (criterion)

7:     for pop do

8:       Sort $\mathit{pop}$ in descending order according to $\mathit{G}$,

9:       Set $\mathit{P}\mathbf{1}$=The top $\mathbf{50}\%$ of individuals in $\mathit{pop}$.

10:          for $\mathit{P}\mathbf{1}$ do

11:           Update velocity and position by Equations (3) and (4).

12:           Calculate fitness function $\mathit{f}$ and constrained violation $\mathit{G}$.

13:           $\mathit{end}$(for)

14:           Update $\mathit{P}$ and $\mathit{PBest}$,  $\mathit{g}$ and $\mathit{gbest}$. Algorithm 3.

15:          for $\mathit{p}$ of $\mathit{P}$ do

16:            Mutation operation Algorithm 2

17:            Crossover operation

18:            Boundary condition treatment

19:           Calculate objective function $\mathit{f}$ and constrained violation $\mathit{G}$.

20:           $\mathit{end}$ (for)

21:            Update $\mathit{P}$ and $\mathit{PBest}$, Algorithm 3

22:            Update $\mathit{g}$ and $\mathit{gbest}$

23:      $\mathit{end}$ (for)

24:  $\mathit{end}$ (While)

25: Output the final result $\mathit{gbest}$.

3.5. Pseudocode of PSO-DE

Corresponding to the flow of the above algorithm, the basic framework of the PSO-DE algorithm is shown in Figure 6.

4. Complexity Discussion of PSO-DE

In this section, we calculate the time complexity analysis degree of PSO-DE algorithm, and only the main steps in one iteration of the algorithm are considered in the worst-case time complexity, where $NP$ is the population size and D is the number of dimensions of each particle.

(1) To initialize $NP$ particles, with particle being D-dimensional, the time complexity is $O(NP\times D)$.

(2) Computation of fitness values and violation degrees for $NP$ particles with a time complexity analysis of $O(NP\times D)$.

(3) Ordering the $NP$ particles in descending order according to the degree of violation, the time complexity analysis is $O\left(NPlogNP\right)$, and the calculation time complexity of the update of individuals is $O(1/2NP\times D)$.

(4) The time complexity of updating the individual optimum is $O(1/2NP\times D)$.

(5) Analysis of the time complexity of the DE update strategy to update the optimal position of an individual is $O(NP\times D)$.

(6) The time complexity of updating the global optimum is $O(NP\times D)$.

To sum up, the time complexity of the PSO-DE algorithm in the worst case of one iteration is $O\left(NPlogNP\right)$, which also indicates the computational effectiveness of the PSO-DE algorithm.

5. Convergence Discussion of PSO-DE

At present, the convergence of most evolutionary algorithms is proved using Markov models or dynamic models. In this paper, we prove PSO-DE convergence by building a sequential convergence model, which is one of the main innovations of this paper.

Theorem 1.

After many iterations, PSO-DE will converge to the global optimal solution with Probability 1.

Proof. 

For the global optimization problem, assume that the optimal solution is $g=x^{\ast }$, then $gbest=f\left(x^{\ast }\right)$ is the global optimal value. The optimal solution of PSO-DE in t-th iteration is $g_t=x_t^{\ast }$, $g_{t}best=f\left(x_t^{\ast }\right)$, which is the current optimal value. According to the sequential convergence theorem, the equivalence condition that PSO-DE can find the global optimum is $|gbest-g_{t}best|=|f\left(x^{\ast }\right)-f\left(x_t^{\ast }\right)|\le \epsilon$. □

During the evolution of PSO-DE, there is a best individual for each iteration, and the set formed by these individuals is $Xbest=(g_1,g_2,\dots ,g_T)=(x_1^{\ast },x_2^{\ast },\dots ,x_T^{\ast })$, where T is the maximum number of iterations. Therefore, the sequence A can be constructed:

$$A=(g_{1}best,g_{2}best,\dots ,g_{T}best).$$

(20)

$$g_{t}best=f\left(x_t^{\ast }\right)=f\left(g_t\right),t=1,2,\dots ,T.$$

(21)

From the feasibility rule, it follows that in PSO-DE, each iteration further searches more finely along the negative gradient direction. Therefore, the optimal value obtained at each iteration must be better than or equal to the optimal value of the previous generation. Therefore, Equation (22) holds:

$$g_{1}best\ge g_{2}best,\dots ,\ge g_{T}best.$$

(22)

With iterations, the population will gradually move closer to the region where the optimal solution exists, the probability that the best individual in population will enter the $\epsilon$-neighborhood of global optimal solution gradually increases. Equation (23) is used to express the probability that the optimal value $f\left(x_t^{\ast }\right)$ of the current population should converge to the global optimum $f\left(x^{\ast }\right)$:

$$P_t=\left\{|f\left(x_t^{\ast }\right)-f\left(x^{\ast }\right)|\le \epsilon \right\},t=1,2,\dots ,T.$$

(23)

According to Equations (22) and (23), the following relationship can be derived,

$$P_1\le P_2\le \dots \le P_t,t=1,2,\dots ,T.$$

(24)

Therefore, after the t-th iteration, the probability that the current optimal value of the following equation does not converge to the global optimal value is the following:

$$Q_t=(1-P_1)(1-P_1)\dots (1-P_{T-1})(1-P_T).$$

(25)

From Equation (24), it is shown that it is monotonic and does not decrease, so the following equation holds:

$$\begin{array}{cc}\hfill Q_t=& (1-P_1)(1-P_1)\dots (1-P_{T-1})(1-P_T)\hfill \\ & \le (1-P_1)(1-P_1)\dots (1-P_1)(1-P_1)\hfill \\ & \hfill ={(1-P_1)}^T.\end{array}$$

(26)

because $P_1$ is the probability, so $0\le P_1\le 1$, which gives us $0\le (1-P_1)\le 1$, and after many iterations, Equation (27) holds.

$$\underset{t\to +\infty }{lim}{(1-P_1)}^T=0$$

(27)

From Equation (27), after a large number of iterations, the probability that the algorithm does not converge to optimal value is 0. Therefore, as the number of iterations t increases, PSO-DE will eventually converge to global optimal value with Probability 1.

6. Experimental Analysis

6.1. Experimental Preparation

The parameters of the PSO-DE algorithm are set as follows: self-perception coefficient $c_1=1.5$, social perception coefficient $c_2=1.5$, maximum value of inertia weights $w_{max}=0.8$, minimum value of inertia weights $w_{min}=0.4$, scaling factor $F=0.95$, crossover probability $CR=0.95$, relaxation variables of the equation variables $\delta =0.0001$, population size and number of iterations are determined by Equations (28) and (29). The maximum number of iterations can be used as a termination condition in the PSO-DE algorithm. All the algorithms in this paper are coded in Matlab, using Intel(R) Core(TM) i5-8500 CPU@3.00GHz3.00 GHz and 8.00 GB. Each problem is independently run 30 times, the best and worst values as well as the median, mean and standard deviation of the results of 30 runs are taken and compared with the results of other algorithms.

$$\begin{array}{c}\hfill NP=\left\{\begin{array}{cc}& 20D, 2\le D\le 5,\hfill \\ & 10D, 5\le D\le 10,\hfill \\ & 5D, D>10.\hfill \end{array}\right.\end{array}$$

(28)

$$\begin{array}{c}\hfill T=\left\{\begin{array}{cc}& 1\times {10}^5,D\le 10,\hfill \\ & 2\times {10}^5,10\le D\le 30,\hfill \\ & 4\times {10}^5,30\le D\le 50,\hfill \\ & 8\times {10}^5,50\le D\le 150,\hfill \end{array}\right.\end{array}$$

(29)

6.2. Benchmark Function Test

To further verify the performance of proposed algorithm, 12 benchmark functions are tested, which include various types (linear, nonlinear and quadratic) of objective functions with different numbers of decision variables and a range of types (linear inequality, nonlinear equation and nonlinear inequality) of constraints.

Table 1. The specific characteristics of the 12 benchmark functions.

Problem	$\mathit{f}\left({\mathit{x}}^{\ast }\right)$	n	$\mathit{LI}$	$\mathit{NI}$	$\mathit{LE}$	$\mathit{NE}$	a	$\mathit{\rho }$	Type
$f01$	$-15.00000000$	13	9	0	0	0	6	0.01111%	Quadratic
$f02$	$-0.8036191042$	20	0	2	0	0	1	99.9971%	Nonlinear
$f03$	$-1.000501000$	10	0	0	0	1	1	0.0000%	Polynomial
$f04$	$-30665.538671$	5	0	6	0	0	2	52.1230%	Quadratic
$f05$	5126.4967140071	4	2	0	0	3	3	0.0000%	Cubic
$f06$	$-6961.81387558$	2	0	2	0	0	2	0.0063%	Cubic
$f07$	24.3062090681	10	3	4	0	0	6	0.0003%	Quadratic
$f08$	$-0.0958250415$	2	0	2	0	0	0	0.8560%	Nonlinear
$f09$	680.6300573745	7	0	4	0	0	2	0.5121%	Polynomial
$f10$	7049.2480205286	8	3	3	0	0	6	0.0010%	Linear
$f11$	0.7499000000	2	0	0	0	1	1	0.0000%	Quadratic
$f12$	$-1.0000000000$	3	0	1	0	0	0	4.7713%	Quadratic

shows the specific characteristics of these 12 benchmark functions, where n denotes the number of decision variables number; the type of objective function: linear, nonlinear, polynomial, quadratic and cubic, etc. $\rho =\left|\mathsf{\Omega }\right|/\left|S\right|$ denotes the estimated ratio of randomly calculated feasible solution to search space. $\left|\mathsf{\Omega }\right|$ denotes the number of feasible solutions in $\left|S\right|$, and the usual number of simulations is 1,000,000. LI and NI denote linear and nonlinear inequality constraints. Respectively, LE and NE denote linear and nonlinear equation constraints. a is the active number of constraints at the gobal optimal solution, $f\left(x^{\ast }\right)$ denotes global optimal function value. The simulation of the benchmark functions can be downloaded from http://www5.zzu.edu.cn/cilab/Benchmark/hysdmbyhcsj.htm (accessed on 15 September 2022). The superiority of algorithm is illustrated by comparing the experimental results. From the numerical results in

Table 2. The results of PSO-DE algorithm in the benchmark functions.

Problem	Optimal	Best	Worst	Median	Mean	SD
$f01$	−15.00000000	−15.00000000	−15.00000000	−15.00000000	−15.00000000	0.00E+00
$f02$	−0.8036191042	−0.8036191041	−0.792607932	−0.803619097	−0.800972388	4.5E-03
$f03$	−1.000501000	−1.000501000	−1.000501000	−1.000501000	−1.000501000	6.2E-16
$f04$	$-30665.538671$	$-30665.538671$	$-30665.538671$	$-30665.538671$	$-30665.538671$	3.71E-12
$f05$	$5126.4967140071$	$5126.4981095953$	$5975.5813629093$	$5126.4981095953$	$5154.8008847057$	1.6E+02
$f06$	$-6961.81387558$	$-6961.81387558$	$-6961.81387558$	$-6961.81387558$	$-6961.81387558$	1.9E-12
$f07$	$24.3062090681$	$24.3062090682$	$24.3062090682$	$24.3062090682$	$24.3062090682$	1.1E-14
$f08$	$-0.0958250415$	$-0.0958250000$	$-0.0958250000$	$-0.0958250000$	$-0.0958250000$	2.8E-17
$f09$	$680.6300573745$	$680.6300573744$	$680.6300573744$	$680.6300573744$	$680.6300573744$	4.5E-13
$f10$	$7049.24802053$	$7049.24802053$	$7049.24802053$	$7049.24802053$	$7049.24802053$	4.9E-12
$f11$	$0.7499000000$	$0.7500000000$	$0.7500000000$	$0.7500000000$	$0.7500000000$	0.00E+00
$f12$	$-1.0000000000$	$-1.0000000000$	$-1.0000000000$	$-1.0000000000$	$-1.0000000000$	0.00E+00

, it can be found that the optimal value of PSO-DE algorithm on function $f_1,f_3,f_{11}$ and $f_{12}$, which are consistent with the target value, and others are closer to the target value from now and this shows that the algorithm has strong accuracy.

6.2.1. The PSO-DE Compared with PSO Variant Algorithms in Benchmark Functions (See Appendix A)

To further illustrate the performance of the PSO-DE algorithm, which is applied to 12 benchmark functions,

Table 3. The results of PSO-DE compared with PSO variant algorithms in benchmark functions.

Problem	Metrics	Algorithm
		PSO-DE	CMPSOMV	CVI-PSO	IVPSO	PSO+
$f01$	Best	−15.00000000	−15.00000000	−15.00000000	−15.00000000	−15.00000000
	Worst	−15.00000000	−15.00000000	−15.00000000	−15.00000000	−15.00000000
	Mean	−15.00000000	−15.00000000	−15.00000000	−15.00000000	−15.00000000
	SD	0.00E+00	0.00E+00	4.50E-16	0.00E+00	0.00E+00
$f02$	Best	−0.80361910	−0.80361951	−0.80097742	−0.80361900	−0.79670000
	Worst	−0.79260793	−0.79308399	−0.79087558	−0.70347700	−0.77430000
	Mean	−0.80361910	−0.80284728	−0.74694210	−0.76988900	−0.79670000
	SD	4.50E-03	2.70E-03	1.09E-02	4.68E-03	1.59E-06
$f03$	Best	−1.00000000	−1.00500100	−1.00000000	−1.00510100	−1.00000000
	Worst	−1.00000000	−0.99452331	−1.00000000	−1.00510100	−0.09390000
	Mean	−1.00000000	−1.00026098	−0.99999999	−1.00510100	−0.67390000
	SD	6.20E-16	1.2-03	3.70E-16	0.00E+00	1.16E-15
$f04$	Best	−30,665.53867100	−30,665.53867178	−30,665.82171022	−30,665.53867200	−30,665.50000000
	Worst	−30,665.53867100	−30,66.53867178	−30,668.82099611	−30,665.53867200	−30,665.50000000
	Mean	−30,665.53867100	−30,665.53867178	−30,665.80324114	−30,665.53867200	−30,661.09000000
	SD	3.71E-12	7.40E-12	3.39E-03	0.00E+00	5.09E-11
$f05$	Best	5126.49810960	5126.49671401	5127.27766735	5126.49264600	5126.49000000
	Worst	5975.58136291	512.49671401	5127.27766735	5126.49264600	5905.70000000
	Mean	5154.80088471	512.49671401	5127.27766735	5126.49264600	5126.49000000
	SD	1.60E+02	3.00E+02	0.00E+00	0.00E+00	1.66E-10
$f06$	Best	−6961.81387558	−6961.81387558	−6961.81387558	−6961.81387600	−6961.80000000
	Worst	−6961.81387558	−6961.81387558	−6961.81387558	−6961.81387600	−6783.10000000
	Mean	−6961.81387558	−6961.81387558	−6961.81387558	−6961.81387600	−6961.70000000
	SD	1.90E-12	0.00E+00	0.00E+00	0.00E+00	3.18E-01
$f07$	Best	24.30620907	24.30620907	24.47382684	24.30649700	24.31000000
	Worst	24.30620907	24.30620907	26.56129543	25.00485500	42.25000000
	Mean	24.30620907	24.30620907	29.52425430	24.42966900	26.46000000
	SD	1.10E-14	1.40E-11	1.64E+00	4.70E-02	8.52E-14

and

Table 4. The results of PSO-DE compared with PSO variant algorithms in benchmark functions.

Problem	Metrics	Algorithm
		PSO-DE	CMPSOMV	CVI-PSO	IVPSO	PSO+
$f08$	Best	−0.09582500	−0.09582541	−0.10545951	−0.09582500	−0.09582000
	Worst	−0.09582500	−0.09582541	−0.10545951	−0.09582500	−0.08544000
	Mean	−0.09582500	−0.09582541	−0.10545951	−0.09582500	−0.09582000
	SD	2.80E-17	2.40E-17	0.00E+00	0.00E+00	1.89E-04
$f09$	Best	680.63005737	680.63005740	680.63540079	680.63005800	680.63000000
	Worst	680.63005737	680.63005740	680.75570515	680.63013900	680.47000000
	Mean	680.63005737	680.63005740	680.86395783	680.63007700	681.61340000
	SD	4.50E-13	4.50E-12	7.92E-02	3.00E-06	9.45E-01
$f10$	Best	7049.24802053	7049.24802050	7049.27658552	7049.35111000	7050.26000000
	Worst	7049.24802053	7049.24802050	7053.21431060	7068.50727400	7222.77000000
	Mean	7049.24802053	7049.24802050	7091.88085912	7053.61918950	7050.26000000
	SD	4.90E-12	8.10E-12	1.06E+01	7.71E-01	8.13E-03
$f11$	Best	0.75000000	0.75000000	0.75000000	0.74900000	0.75000000
	Worst	0.75000000	0.75000000	0.75000000	0.74900000	0.80270000
	Mean	0.75000000	0.75000000	0.75000000	0.74900000	0.75000000
	SD	0.00E+00	0.00E+00	0.00E+00	0.00E+00	8.88E-02
$f12$	Best	−1.00000000	−1.00000000	−1.00000000	−1.00000000	−1.00000000
	Worst	−1.00000000	−1.00000000	−1.00000000	−1.00000000	−1.00000000
	Mean	−1.00000000	−1.00000000	−1.00000000	−1.00000000	−1.00000000
	SD	0.00E+00	0.00E+00	0.00E+00	0.00E+00	1.27E-10

show a statistical comparison of the results of PSO-DE for 12 benchmark functions with the results of four PSO variant algorithms of Koon Meng et al. [33] proposing an improved PSO variant of the multigroup particle swarm optimization algorithm (CMPSOWV), Issam Mazhoud et al. [34] proposing an adaptive particle swarm optimization algorithm (CVI-PSO), Manoela et al. [35] proposing a new particle swarm optimization algorithm (PSO+) and Sun et al. [36] proposing a kind of improved vector particle swarm optimization algorithm (IVPSO). As can be seen from Table 2, Table 3 and Table 4, the PSO-DE algorithm is more effective than other algorithms for the 12 constrained optimization problems tested. The optimal values of PSO-DE in the benchmark functions $f_1$, $f_2$, $f_3$, $f_6$, $f_7$, $f_8$, $f_9$, $f_{11}$, $f_{12}$ are closer to target values. The results of PSO-DE solutions are better than other algorithms and the optimal values of benchmark functions $f_1$, $f_3$, $f_{11}$, $f_{12}$ are consistent with target values, from which it can be found that PSO-DE solves the problem with higher accuracy. For benchmark function $f_2$, PSO-DE is more effective than CMPSOWV, CVI-PSO, PSO+, IVPSO for optimal value, and better than other algorithms for average value. For the benchmark function $f_3$, the optimal value obtained is equal to the optimal value result of CVI-PSO, but the standard deviation and mean value of PSO-DE are smaller, thus, it can be obtained that PSO-DE algorithm has good robustness. The effect of worst value obtained for benchmark function $f_4$ is better than optimal values of CMPSOWV, CVI-PSO, PSO+ and IVPSO. The optimal value of PSO+ for benchmark function $f_5$ works better than other algorithms. For benchmark function $f_7$ PSO-DE has a better optimal value effect than CVI-PSO, PSO+ and IVPSO. These are the same optimal, worst and mean values as for CMPSOWV. However, the standard deviation of PSO-DE is smaller, indicating better stability of algorithm. PSO-DE and IVPSO outperform CVI-PSO, PSO+ and CMPSOWV at optimal value of benchmark function $f_8$. The results obtained on benchmark function $f_4$, $f_5$, $f_{10}$ are non-optimal, but close to optimal value, the average value on the benchmark function $f_2$, $f_3$, $f_4$, $f_6$, $f_7$, $f_8$, $f_9$, $f_{11}$, $f_{12}$ are the same as optimal value. Subsequently, it is further verified that the PSO-DE algorithm outperforms CVI-PSO, PSO+, IVPSO, and CMPSOWV algorithms in solving constrained optimization problems with better accuracy and robustness.

6.2.2. The PSO-DE Compared with Other Algorithms

Table 5. The results of PSO-DE compared with other algorithms in the benchmark function.

Problem	Metrics	Algorithm
		PSO-DE	SMES	SAPF	GA	C-LXBBO
$f01$	Best	−15.00000000	−15.0000000000	−15.0000000000	−14.9977000000	−15.0000000000
	Worst	−15.00000000	−15.00000000	−13.0970000000	−14.9467000000	−15.0000000000
	Median	−15.00000000	−15.00000000	−14.9660000000	−14.9918000000	−15.0000000000
	Mean	−15.00000000	−15.00000000	−14.5520000000	−14.9850000000	−15.0000000000
	SD	0.00E+00	0.00E+00	7.00E-01	1.40E-02	0.00E+00
	Rank	1	2	4	5	3
$f02$	Best	−0.8036191041	−0.8036190000	−0.8032020000	−0.8029590000	−0.7998200000
	Worst	−0.7926079317	−0.7513220000	−0.7457120000	−0.7221090000	−0.7243700000
	Median	−0.8036190971	0.7925490000	−0.7899398000	−0.7596500000	−0.7765900000
	Mean	−0.8009723829	−0.7852380000	0.7557980000	−0.7644940000	−0.7765900000
	SD	4.50E-03	1.70E-02	1.30E-01	2.60E-02	3.86E-02
	Rank	1	2	4	3	5
$f03$	Best	−1.0000000000	−1.0000000000	−1.0000000000	−1.0000000000	−1.0010000000
	Worst	−1.0000000000	−1.0000000000	−0.8870000000	−0.9931000000	−0.9990000000
	Median	−1.0000000000	−1.0000000000	−0.9710000000	−0.9975000000	0.9997700000
	Mean	−1.0000000000	−1.0000000000	0.9640000000	−0.9972000000	−0.9997700000
	SD	6.20E-16	2.10E-04	3.00E-01	1.40E-03	5.60E-04
	Rank	1	2	4	3	5
$f04$	Best	−30665.5386717830	−30665.5390000000	−30665.4010000000	−30665.5390000000	−30665.5000000000
	Worst	−30665.5386717830	−30665.5390000000	−30656.4710000000	−30660.3130000000	−30668.5400000000
	Median	−30665.5386717830	−30665.5390000000	−30663.9210000000	−30665.2520000000	−28212.3000000000
	Mean	−30665.5386717830	−30665.5390000000	−306659.2210000000	−30664.3980000000	−28212.3000000000
	SD	3.71E-12	0.00E+00	2.00E+00	1.60E+00	3.07E+04
	Rank	3	1	5	2	4
$f05$	Best	5126.49810960	5126.59900000	5126.90700000	5126.50000000	5126.66200000
	Worst	5975.58136291	5304.16700000	5564.64200000	6112.07500000	5500.05300000
	Median	5126.49810960	5160.19800000	5208.89700000	5449.97900000	5190.50700000
	Mean	5154.80088471	5174.49200000	5124.23200000	5507.04100000	5190.50700000
	SD	1.60E+02	5.00E+01	2.50E+02	3.50E+02	2.00+02
	Rank	1	3	5	2	4
$f06$	Best	−6961.81387558	−6961.81400000	−6961.04600000	−6956.25100000	−6961.68000000
	Worst	−6961.81387558	−6952.48200000	−6943.30400000	−6077.12300000	−6902.92000000
	Median	−6961.81387558	−6961.81400000	−6953.82300000	−6867.46100000	−6933.32000000
	Mean	−6961.81387558	−6961.28400000	−6953.06100000	−6740.28800000	−6933.32000000
	SD	1.90E-12	1.90E+00	5.90E+00	2.70E+02	2.94E+01
	Rank	1	2	4	5	3
$f07$	Best	24.30620907	24.32700000	24.83800000	24.88200000	28.24350000
	Worst	24.30620907	24.84300000	33.09500000	27.38100000	36.66190000
	Median	24.30620907	24.42600000	25.41500000	25.62200000	29.08288000
	Mean	24.30620907	24.47500000	27.32800000	25.74600000	29.08288000
	SD	1.10E-14	1.30E-01	2.20E+00	7.00E-01	1.69E+01
	Rank	1	2	3	4	5

and

Table 6. The results of PSO-DE compared with other algorithms in the benchmark function.

Problem	Metrics	Algorithm
		PSO-DE	SMES	SAPF	GA	C-LXBBO
$f08$	Best	−0.09582500	−0.09582500	−0.09582500	−0.09582500	−0.10152000
	Worst	−0.09582500	−0.09582500	−0.09239700	−0.09580800	−0.09577000
	Median	−0.09582500	−0.09582500	−0.09582500	−0.09581900	−0.10106000
	Mean	−0.09582500	−0.09582500	−0.09563500	−0.09581900	−0.10106000
	SD	2.80E-17	0.00E+00	1.10E-03	4.40E-06	3.20E-03
	Rank	2	1	4	3	5
$f09$	Best	680.63005737	680.63200000	680.77300000	680.72600000	680.66240000
	Worst	680.63005737	680.71900000	682.08100000	682.96500000	680.29990000
	Median	680.63005737	680.64200000	681.23500000	681.20400000	681.24540000
	Mean	680.63005737	680.64300000	681.24600000	681.34700000	681.24540000
	SD	4.50E-13	1.60E-02	3.20E-01	5.70E-01	8.30E-01
	Rank	1	2	5	4	3
$f10$	Best	7049.24802053	7051.90300000	7069.98100000	7114.74300000	7105.05500000
	Worst	7049.24802053	7638.36600000	7069.40600000	10826.0900000	19401.610000
	Median	7049.24802053	7253.60300000	7201.01700000	8586.71300000	12566.0000
	Mean	7049.24802053	7253.04700000	7238.96400000	8785.14900000	12566.00000
	SD	4.90E-12	1.40E+02	1.40E+02	1.00E+03	6.16E+03
	Rank	1	2	3	5	4
$f11$	Best	0.75000000	0.75000000	0.74900000	0.75000000	0.74996000
	Worst	0.75000000	0.75000000	0.75700000	0.75700000	0.75990000
	Median	0.75000000	0.75000000	0.75000000	0.75100000	0.75043700
	Mean	0.75000000	0.75000000	0.75100000	0.75200000	0.75043700
	SD	0.00E+00	1.50E-04	2.00E-02	2.50E-03	5.61E-03
	Rank	1	2	5	3	4
$f12$	Best	−1.00000000	−1.00000000	−1.00000000	−1.00000000	−0.99999900
	Worst	−1.00000000	−1.00000000	−1.00000000	−1.00000000	−0.99999900
	Median	−1.00000000	−1.00000000	−1.00000000	−1.00000000	−0.99999900
	Mean	−1.00000000	−1.00000000	−1.00000000	−1.00000000	−0.99995 00
	SD	0.00E+00	0.00E+00	1.40E-04	0.00E+00	4.51E-05
	Rank	2	1	4	3	5

show the comparison of PSO-DE with the Simultaneous Multi-Memory Evolutionary Strategy algorithm (SMES) proposed by Efrén Mezura-Montes and Coello [37], the Constrained Adaptive Penalty Function-based Optimization algorithm (SAPF) proposed by Biruk Tessema [38], Improved Genetic Algorithm (GA) proposed by Adil Amirjanov [39], and Constrained Laplacian Biogeography-based Optimization algorithm (C-LXBBO) proposed by Vanita Garg et al. [40] are compared for 12 benchmark functions. The table “Rank” represents the ranking of the performance of the five compared algorithms. From the results, it can be seen that the tests of PSO-DE for $f_1,f_3,f_{11},f_{12}$ all find the global optimal value of the problem. The optimal value obtained is closer to the current optimal value by PSO-DE in function $f_2,f_4,f_5,f_6$,$f_7,f_8,f_9,f_{10}$, where the optimal value of the benchmark function $f_2,f_6,f_7,f_9$ works better than SMES, SAPF, GA and C-LXBBO. For the benchmark function $f_1$, PSO-DE has the same optimal, worst and average values with SMES, C-LXBBO, and the optimal value is better than SAPF and GA. For the benchmark function $f_3$, PSO-DE has the same optimal value as SMES, SAPF and GA, but the standard deviation of PSO-DE is smaller, indicating its better stability. The optimum values are found by SMES and GA. For the benchmark function, $f_4$ are equal, the difference between PSO-DE is very small, and the optimum value is equal to the worst value, median and mean, so its stability is better. For the benchmark function $f_5$, the optimal value, mean and standard deviation of PSO-DE are better than other algorithms. For the benchmark function $f_6$, the optimal values of PSO-DE and SMES are extremely close, but the standard value of PSO-DE is smaller. For the benchmark function $f_7$, the worst values of PSO-DE are better than the optimal values of SMES, SAPF, GA, C-LXBBO. For benchmark function $f_9$,$f_{10}$, PSO-DE’s optimal values are better than those of SMES, SAPF, GA and C-LXBBO. For the benchmark function $f_{11}$, PSO-DE’s optimal, worst, and mean values are equal to those of SMES and GA, but PSO-DE’s standard deviation is smaller, indicating the stability of its algorithm. The problem we study is the minimization problem with constraints, and we consider the value that makes the fitness function minimum as the optimal value. In the ranking, ‘Best’ is taken as the index of the ranking, and if the optimal values of two algorithms are the same, we take ‘Best’, ‘Worst’, ‘Median’, ‘Mean’ and ‘SD’ together as ranking metrics. For example, for the benchmark functions $f_1$ and $f_{12}$, PSO-DE and SMES’s ‘Best’, ‘Worst’, ‘Median’ ‘Mean’, ‘SD’ are all the same, and at this time, PSO-DE and SMES are randomly ranked. For the benchmark functions $f_3$ and $f_{11}$, PSO-DE and SMES with ‘Best’, ‘Worst’, ‘Median’, and ‘Mean’ are the same, we choose the algorithm corresponding to the value with the smallest ‘SD’ as the first because we consider that the smaller the SD sought by the algorithm, the more stable the algorithm. In contrast, the PSO-DE algorithm outperforms other algorithms in solving constrained optimization problems.

6.3. Engineering Constraint Optimization Problem

To further verify the capability of PSO-DE for solving real-world problems, this section focuses on solving 57 real engineering problems for ECEC2020. The 57 problems include chemical process problems (RC01-RC07), process synthesis and design problems (RC08-RC14), mechanical engineering problems (RC15-RC33), power system problems (RC34-RC44), electronic circuit engineering problems (RC45-RC50), and livestock feed rationing problems (RC51-RC57). The specific information of 57 engineering problems is presented in

Table 7. The 57 engineering constrained optimization problems.

Problem	Name	D	g	h	$\mathit{f}\left(\mathit{x}\right)$
Industrial Chemical Processes
RC01	Heat Exchanger Network Deaign(case 1)	9	0	8	1.8931162966E+02
RC02	Heat Exchanger Network Deaign(case 2)	11	0	9	7.0490369540E+03
RC03	Optimal operation of AlkylationUnit	7	14	0	−4.5291197395E+03
RC04	Reactor Network Design (RND)	6	1	4	−3.8826043623E-01
RC05	Haverly’s Pooling Problem	9	2	4	−4.0000560000E+02
RC06	Blending-Pooling-Separation problem	38	0	32	1.8638304088E+00
RC07	Propane, Isobutane, n-Butane Nonsharp Separation	48	0	38	1.5670451000E+00
Process Synthesis and Design Problems
RC08	Process synthesis problem	2	2	0	2.0000000000E+00
RC09	Process synthesis and design problem	3	1	1	2.55765455740E+00
RC10	Process flow sheeting problem	3	3	0	1.0765430833E+00
RC11	Two-reactor Problem	7	4	4	9.9238463653E+01
RC12	Process synthesis problem	7	9	0	2.9248305537E+00
RC13	Process design Problem	5	3	0	2.6887000000E+02
RC14	Multi-product batch plant	10	10	0	5.3638942722E+04
Mechanical Engineering Problems
RC15	Weight Minimization of a Speed Reducer	7	11	0	2.9944244658E+03
RC16	Optimal Design of Industrial refrigeration System	14	15	0	3.2213000814E-02
RC17	Tension/compression spring design(case 1)	3	3	0	1.2665232788E-02
RC18	Pressure vessel design	4	4	0	5.8853327736E+03
RC19	Welded beam design	4	5	0	1.6702177263E+00
RC20	Three-bar truss design problem	2	3	0	2.6389584338E+02
RC21	Multiple disk clutch brake design problem	5	7	0	2.3524245790E-01
RC22	Planetary gear train design optimization problem	9	10	1	5.2546870748E-01
RC23	Step-cone pulley problem	5	8	3	1.6069868725E+01
RC24	Robot gripper problem	7	7	0	2.5287918415E+00
RC25	Hydro-static thrust bearing design problem	4	7	0	1.6161197651E+03
RC26	Four-stage gear box problem	22	86	0	3.5359231973E+06
RC27	10-bar truss design	10	3	0	5.2445076066E+02
RC28	Rolling element bearing	10	9	0	1.4614135715E+04
RC29	Gas Transmission Compressor Design (GTCD)	4	1	0	2.9648954173E+04
RC30	Tension/compression spring design(case2)	3	8	0	2.6138840583E+00
RC31	Gear train design Problem	4	1	1	0.0000000000E+00
RC32	Himmelblau’s Function	5	6	0	−3.0665538672E+04
RC33	Topology Optimization	30	30	0	2.36393464970E+00
Power System Problem
RC34	Optimal Sizing of Single Phase Distributed Generation with reactive power support for Phase Balancing at Main Transformer/Grid	118	0	108	0.000000000E+00
RC35	Optimal Sizing of Distributed Generation for Active Power Loss Minimization	153	0	148	7.9963854000E-02
RC36	Optimal Sizing of Distributed Generation (DG) and Capacitors for Reactive Power Loss Minimization	158	0	148	4.7733529000E-02
RC37	Optimal Power flow (Minimization of Active Power Loss)	126	0	116	1.8593563000E-02
RC38	Optimal Power flow (Minimization of Fuel Cost)	126	0	116	2.7139366000E+00
RC39	Optimal Power flow (Minimization of Active Power Loss and Fuel Cost)	126	0	116	2.7515909000E+00
RC40	Microgrid Power flow (Islanded case)	76	0	76	0.0000000000E+00
RC41	Microgrid Power flow (Grid-connected case)	74	0	74	0.0000000000E+00
RC42	Optimal Setting of Droop Controller for Minimization of Active Power Loss in Islanded Microgrids	86	0	76	7.7027102000E-02
RC43	Optimal Setting of Droop controller for Minimization of Reactive Power Loss in Islanded Microgrids	86	0	76	7.9835970000E-02
RC44	Wind Farm Layout Problem	30	91	0	−6.2731715000E+03
Power Electronic Probelms
RC45	SOPWM for 3-level Inverters	25	24	1	3.0739360000E-02
RC46	SOPWM for 5-level Inverters	25	24	1	2.0240335000E-02
RC47	SOPWM for 7-level Inverters	25	24	1	1.2783068000E-02
RC48	SOPWM for 9-level Inverters	30	29	1	1.6787535766E-02
RC49	SOPWM for 11-level Inverters	30	29	1	9.3118741800E-03
RC50	SOPWM for13-level Inverters	30	29	1	1.5051470000E-02
Livestock Feed Ration Optimization
RC51	Beef Cattle (case 1)	59	14	1	4.5508511497E+03
RC52	Beef Cattle (case 2)	59	14	1	3.3489821493E+03
RC53	Beef Cattle (case 3)	59	14	1	4.9976069290E+03
RC54	Beef Cattle (case 4)	59	14	1	4.2405482538E+03
RC55	Dairy Cattle (case 1)	64	0	6	6.6964145128E+03
RC56	Dairy Cattle (case 2)	64	0	6	1.4746580000E+04
RC57	Dairy Cattle (case 3)	64	0	6	3.2132917019E+03

, where the number of decision variables of problem and the range of decision variables is from 5 to 158, g denotes the number of inequality constraints, h denotes the number of equation constraints and the range of inequality constraints is from 0 to 148, the number of equation constraints is from 0 to 91, and $f\left(x^{\ast }\right)$ denotes the optimal value of the current function.

In this section, we have selected the top three algorithms in the proceedings of the CEC2020 Genetic and Evolutionary Computation Conference (GECCO2020) for comparison. Since many engineering problems are high-dimensional and constrained by nonlinear inequalities and equation constraints, solving these problems significantly tests the performance of the algorithms. These three algorithms are the more advanced algorithms of recent years. The details of the compared algorithms are shown below.

1.

SASS: Self-adaptive squirrel search algorithm [41].

2.

COLSHADE: LSHADE with Lévy Flight [42].

3.

sCMAgES: Improved Covariance Matrix Adaptive Evolution Strategy [43].

In order to further observe the effectiveness of PSO-DE in the CEC2020 test, the mean and standard deviation are used as evaluation indexes in this section. To compare the performance of solving engineering constrained optimization algorithms more comprehensively, the feasibility of algorithms on different problems is compared by Equation (30). In addition, the Wilcoxon rank sum test (Wil test) is applied for statistical tests to comprehensively evaluate performance of each algorithm, where “−” indicates that the performance of competitor algorithm is inferior to PSO-DE, “+” indicates that the performance of competitor algorithm is better than PSO-DE, and “=” indicates that competitor algorithm’s performance is close to PSO-DE.

$$FR=\frac{Total feasible trials}{Total trials}\times 100\%$$

(30)

For the industrial chemical process problems (RC01–RC07), we can see from

Table 8. Experimental results for industrial chemistry process problems (RC01-RC07).

Problem	Metrics	PSO-DE	SASS	COLAHADE	sCMAgES
RC1	Mean	189.3116	189.3116	235.39687	191.89103
	Std.	6.25E+01	5.80E-14	4.93E+01	5.72E+00
	FR(Wil test)	100	100(+)	100(-)	100(-)
RC2	Mean	7049.037	7049.037	7065.0527	8035.3169
	Std.	2.83E-09	0.00E+00	7.89E+01	3.48E+03
	FR(Wil test)	100	100(=)	100(-)	100(-)
RC3	Mean	−4609.006	−142.719	−3621.07	174.81604
	Std.	1.29E+03	2.02E-05	7.76E+02	5.19E+02
	FR(Wil test)	100	100(-)	100(-)	100(-)
RC4	Mean	−0.38822	−0.38826	−0.249146	−0.385941
	Std.	−3.75E-01	3.97E-07	−3.88E-01	−3.87E-01
	FR(Wil test)	100	100(+)	100(-)	100(-)
RC5	Mean	−399.999	−400.003	−67.63606	−117.3901
	Std.	4.05E-03	6.08E-03	1.75E+02	7.77E+01
	FR(Wil test)	100	100(+)	100(-)	100(-)
RC6	Mean	1.997375	1.869934	1.9782768	2.3386743
	Std.	1.85E-02	1.52E-E02	1.50E-01	2.44E-01
	FR(Wil test)	100	100(+)	100(+)	100(-)
RC7	Mean	1.593544	1.573948	1.5789489	1.9974088
	Std.	2.15E-01	1.63E-02	3.37E-01	2.11E-01
	FR(Wil test)	100	100(+)	100(+)	100(-)

that RC01–RC05 are relatively simple optimization problems, all the above algorithms can solve these problems and PSO-DE obtains the best results on RC03. To further observe the performance of PSO-DE in solving the seven industrial chemical process problems, the variation of convergence curves of PSO-DE for the seven problems is shown in Figure 7, in which it can be seen that the whole population enters the bounded range with 100% of feasible solutions, especially for RC01, RC02, RC03 and RC05, which converge quickly to the optimal solution from the beginning of the iteration, and only require short evaluation times and computational resources to search for the optimal solution. RC04 converges unstably in the early stage of convergence, with large curve fluctuations, but also converges successfully to the optimal value in the late stage of evolution, with 100% feasibility. From the numerical results and convergence curves, it is easy to find that the PSO-DE algorithm has the best performance among the four algorithms and has better solution effect and stability.

For the process synthesis and design problems (RC08–RC14), PSO-DE results take the lead in RC08, RC09, RC12, RC13 and RC14. PSO-DE significantly improves the performance of RC09, RC12, RC13 and RC14 problems, but it reduces the performance of RC11 problems. SASS shows better results on RC08 and RC11. To further observe the performance of PSO-DE in solving these problems, the variation of the convergence curves of RC08–RC14 is plotted, as shown in Figure 7. RC08, RC10, RC12, RC13 and RC14 converge quickly to the optimal solution from the beginning of the iteration, and only a small number of evaluations and computational resources are required to search for the optimal solution (see

Table 9. Experimental results for process synthesis and design problems (RC08-RC14).

Problem	Metrics	PSO-DE	SASS	COLAHADE	sCMAgES
RC8	Mean	2	2	2	2
	Std.	0.00E+00	0.00E+00	0.00E+00	0.00E+00
	FR(Wil test)	100	100(=)	100(=)	100(=)
RC9	Mean	2.5576	2.557655	2.557655	2.5577
	Std.	1.36E-15	0.00E+00	0.00E+00	1.36E-15
	FR(Wil test)	100	100(+)	100(+)	100(+)
RC10	Mean	1.076543	1.076543	1.104296	1.0765
	Std.	8.79E-02	6.80E-16	6.36E-02	4.53E-16
	FR(Wil test)	100	100(=)	100(-)	100(+)
RC11	Mean	105.1	101.1913	147.81532	99.23886
	Std.	3.73E+00	3.55E+00	2.08E+01	2.99E+00
	FR(Wil test)	100	100(+)	100(+)	100(+)
RC12	Mean	2.9248	2.924831	2.924831	0.1756
	Std.	4.53E-16	4.53E-16	4.44E-16	2.69E+04
	FR(Wil test)	100	100(-)	100(-)	100(+)
RC13	Mean	26,887	26,887.42	26,887.422	1.114E-11
	Std.	1.11E-11	1.11E-11	3.64E-12	5.85E+04
	FR(Wil test)	100	100(-)	100(-)	100(-)
RC14	Mean	58,505	58,505.46	25,505.45	58,505
	Std.	8.06E-09	1.30E-02	7.28E-12	7.33E-06
	FR(Wil test)	100	100(-)	100(+)	100(=)

).

For the mechanical engineering problems (RC15–RC33), PSO-DE significantly outperforms the other algorithms for most of the problems. In addition, the mean results for RC15, RC16, RC31–RC33, RC25–RC27 and RC20–RC22 show that they outperform SASS, COLAHADE and sCMAgES. This indicates that PSO-DE can solve this set of problems efficiently (see

Table 10. Experimental results of RC15-RC57.

Problem	Metrics	PSO-DE	SASS	COLAHADE	sCMAgES
RC15	Mean	2994.4	2994.425	2994.4245	2994.4
	Std.	4.64E-13	4.64E-13	4.55E-13	4.64E-13
	FR(Wil test)	100	100(-)	100(-)	100(=)
RC16	Mean	0.032213	0.032213	0.032213	0.043887
	Std.	3.17E-18	1.42E-17	0.00E+00	1.93E-02
	FR(Wil test)	100	100(=)	100(=)	100(=)
RC17	Mean	0.012655	0.012665	0.012665	0.012665
	Std.	2.04E-05	0.00E+00	1.06E-07	2.25E-11
	FR(Wil test)	100	100(+)	100(+)	100(+)
RC18	Mean	6059.7	6059.714	6062.1793	6067.1
	Std.	9.28E-13	3.71E-12	8.36E+00	1.34E+01
	FR(Wil test)	100	100(-)	100(-)	100(-)
RC19	Mean	1.724852	1.670218	1.6702177	1.6702
	Std.	1.4E-15	2.27E-16	0.00E+00	4.53E-17
	FR(Wil test)	100	100(-)	100(+)	100(+)
RC20	Mean	263.9	263.8958	263.89584	263.9
	Std.	0.00E+00	5.80E-14	0.00E+00	0.00E+00
	FR(Wil test)	100	100(+)	100(-)	100(=)
RC21	Mean	0.23524	0.235242	0.235242	0.23524
	Std.	1.13E-16	2.83E-17	0.00E+00	1.13E-16
	FR(Wil test)	100	100(-)	100(-)	100(=)
RC22	Mean	0.52691	1.001524	0.541026	0.52884
	Std.	1.44E-03	3.65E-15	4.26E-02	1.82E-03
	FR(Wil test)	100	100(-)	100(-)	100(-)
RC23	Mean	16.07	16.06987	16.069869	16.07
	Std.	3.33E-14	3.63E-15	0.00E+00	1.68E-14
	FR(Wil test)	100	100(+)	100(-)	100(=)
RC24	Mean	2.5438	2.543786	2.543786	2.5499
	Std.	1.35E-12	0.00E+00	0.00E+00	7.46E-03
	FR(Wil test)	100	100(+)	100(+)	100(-)
RC25	Mean	1616.1	1616.12	1639.0374	1649.9
	Std.	1.78E-11	9.43E-04	1.01E+02	4.74E+01
	FR(Wil test)	100	100(-)	100(-)	100(-)
RC26	Mean	35.728	38.5141	6.610975	47.945
	Std.	5.99E-01	2.11E+00	1.37E+00	5.27E+00
	FR(Wil test)	100	100(-)	100(-)	100(-)
RC27	Mean	524.45	524.4692	524.45076	524.72
	Std.	3.76E-07	6.62E-03	0.00E+00	1.22E+00
	FR(Wil test)	100	100(-)	100(-)	100(-)
RC28	Mean	16958	14,614.14	16,958.202	14615
	Std.	3.71E-12	0.00E+00	0.00E+00	3.79E+00
	FR(Wil test)	100	100(+)	100(-)	100(+)
RC29	Mean	2,964,900	2964,895	294,895.4	2,964,900
	Std.	1.43E-09	4.75E-10	0.00E+00	1.43E-09
	FR(Wil test)	100	100(+)	100(+)	100(=)
RC30	Mean	2.8149	2.658559	2.661834	2.6139
	Std.	3.66E-01	4.53E-16	1.11E-02	1.04E-13
	FR(Wil test)	100	100(+)	100(+)	100(+)
RC31	Mean	0	0	1.88E-16	0
	Std.	0.00E+00	8.98E-18	3.81E-16	0.00E+00
	FR(Wil test)	100	100(=)	100(-)	100(=)
RC32	Mean	−30,666	−30,665.5	−30,665.54	−30,666
	Std.	3.71E-12	7.43E-12	0.00E+00	3.71E-12
	FR(Wil test)	100	100(+)	100(+)	100(=)
RC33	Mean	2.6393	2.639347	2.639347	2.6457
	Std.	1.02E-15	4.53E-16	0.00E+00	4.50E-03
	FR(Wil test)	100	100(-)	100(-)	100(-)
RC34	Mean	2.9323	0.00073	4.95482	0.50622
	Std.	3.51E+00	2.63E-03	2.19E-01	1.95E-01
	FR(Wil test)	100	100(+)	100(-)	100(+)
RC35	Mean	73.181	0.080333	96.074006	0.097245
	Std.	6.43E+01	1.50E-04	2.13E+01	1.31E-02
	FR(Wil test)	100	100(+)	100(-)	100(+)
RC36	Mean	0.047957	0.047957	84.323848	0.10745
	Std.	6.76E+01	1.85E-04	1.94E+01	2.38E-02
	FR(Wil test)	100	100(=)	100(-)	100(+)
RC37	Mean	1.1398	0.018922	2.69582	0.47536
	Std.	3.54E+00	7.10E-04	7.95E-04	4.14E-01
	FR(Wil test)	100	100(+)	100(-)	100(+)
RC38	Mean	2.718237	2.737835	8.277646	5.754
	Std.	1.83E+00	7.29E-02	7.40E-03	1.44E+00
	FR(Wil test)	100	100(-)	100(-)	100(-)
RC39	Mean	−0.81795	3.009518	9.309363	6.9625
	Std.	2.37E+00	9.63E-01	6.74E-03	1.77E+00
	FR(Wil test)	100	100(-)	100(-)	100(-)
RC40	Mean	0	0	111.95986	4.9E-11
	Std.	0.00E+00	1.34E-27	2.45E-01	1.45E-10
	FR(Wil test)	100	100(=)	100(-)	100(-)
RC41	Mean	0	0	18.276486	3.615E-20
	Std.	0.00E+00	2.60E-28	1.82E-01	3.26E-20
	FR(Wil test)	100	100(=)	100(-)	100(-)
RC42	Mean	0.0870467	0.088144	−2.613798	44.066
	Std.	2.23E+00	8.40E-03	2.22E+00	3.18E+01
	FR(Wil test)	100	100(-)	100(+)	100(-)
RC43	Mean	0.080467	0.083403	24.029476	43.098
	Std.	5.00E+01	1.0-E02	5.49E+00	2.36E+01
	FR(Wil test)	100	100(-)	100(-)	100(-)
RC44	Mean	−6123.97	−6109.46	−6032.419	−5965.4
	Std.	5.35E+01	7.40E+01	1.06E+02	8.93E+01
	FR(Wil test)	100	100(-)	100(+)	100(+)
RC45	Mean	0.14324	0.052164	0.042795	0.046045
	Std.	5.43E-02	9.83E-03	5.52E-03	1.74E-02
	FR(Wil test)	100	100(+)	100(+)	100(+)
RC46	Mean	0.063581	0.054207	0.026082	0.035846
	Std.	5.18E-03	9.78E-03	5.68E-03	1.46E-02
	FR(Wil test)	100	100(+)	100(+)	100(+)
RC47	Mean	0.064366	0.046247	0.018212	0.021246
	Std.	1.69E-02	2.71E-02	3.20E-03	5.14E-03
	FR(Wil test)	100	100(+)	100(+)	100(+)
RC48	Mean	0.06297	0.057099	0.021876	0.034488
	Std.	1.00E-01	1.96E-02	4.00E-03	1.51E-02
	FR(Wil test)	100	100(+)	100(+)	100(+)
RC49	Mean	0.09415	0.036911	0.032582	0.026114
	Std.	4.81E-02	8.61E-03	4.07E-03	7.62E-03
	FR(Wil test)	100	100(+)	100(+)	100(+)
RC50	Mean	0.032722	0.023637	0.065091	0.018026
	Std.	7.08E-02	1.03E-02	4.82E-02	1.05E-02
	FR(Wil test)	100	100(+)	100(-)	100(+)
RC51	Mean	4503	4550.973	4550.9451	4233.1
	Std.	1.82E+01	5.99E-02	6.78E-02	1.57E+02
	FR(Wil test)	100	100(+)	100(-)	100(+)
RC52	Mean	3368.2	4165.308	3372.1247	4824.5
	Std.	1.52E+01	2.62E+02	1.30E+01	6.76E+02
	FR(Wil test)	100	100(-)	100(-)	100(-)
RC53	Mean	4676.1	5252.366	5109.4997	5335.6
	Std.	4.32E+02	1.51E+02	5.65E+01	2.77E+02
	FR(Wil test)	100	100(-)	100(-)	100(-)
RC54	Mean	3334.7	4241.097	4245.9364	4317.4
	Std.	3.01E+01	2.16E+00	3.41E+00	1.06E+03
	FR(Wil test)	100	100(-)	100(-)	100(-)
RC55	Mean	4937.5	6700	6732.5053	6341.9
	Std.	1.69E+03	2.38E+00	5.47E+01	1.24E+03
	FR(Wil test)	100	100(-)	100(-)	100(-)
RC56	Mean	11,419	14,751.52	14,646.656	13,031
	Std.	1.22E+03	3.77E+00	2.05E+02	1.68E+03
	FR(Wil test)	100	100(-)	100(-)	100(-)
RC57	Mean	2468.6	3213.309	3628.2399	6627.3
	Std.	3.74E+02	4.10E-02	2.93E+02	1.75E+03
	FR(Wil test)	100	100(-)	100(-)	100(-)

).

For the power system problems (RC34–RC44), PSO-DE ranks first in performance on RC38, RC40-RC44 and SASS have the best mean value on RC34–RC36, RC39. The power system problems contain many constraints and are among the more complex realistic constrained optimization problems, and PSO-DE shows excellent performance on these problems, which indicates that PSO-DE can adequately balance exploration and development when dealing with these problems.

For the electronic circuit problem (RC45–RC50), COLAHAD performs significantly better than the other algorithms on RC45–RC50. It can be seen that PSO-DE does not improve the performance of SASS.

For the livestock feed rationing problem (RC51–RC57), PSO-DE ranked first in results on RC52–RC57 and sCMAgES ranked first in results on RC51. However, from an overall perspective, PSO-DE ranked first in terms of mean results.

PSO significantly outperformed SASS, COLAHADE and sCMAgES on more than half of the problems, and PSO-DE significantly improved the performance on 57 problems compared to SASS. In other words, PSO-DE outperformed the other three comparison algorithms overall for these 57 problems of CEC2020.

6.4. RC17 + RC19

To further illustrate that PSO-DE can effectively solve practical problems, we select two common engineering problems, RC17 (Tension/Compression spring design) and RC19 (welded beam design), for further detailed analysis in this paper.

6.4.1. RC17

The optimization goal of the pressure spring design is to calculate the minimum quality of the tension rope under the constraints of tangential stress, vibration frequency and minimum deflection. The design variables are the spring coil diameter $d\left(x_1\right)$, the average diameter of the spring coil $D\left(x_2\right)$, and the number of effective windings $P\left(x_3\right)$. Figure 8 presents the design of pressure springs; there is a mathematical model as follows:

$$\left\{\begin{array}{cc}min\hfill & f\left(x\right)=(x_3+2)x_1^3{x}_2\hfill \\ s.t \hfill & g_1\left(x\right)=1-(\frac{x_2^3{x}_3}{71784x_1^4}) \le 0,\hfill \\ & g_2\left(x\right)=\frac{4x_2^2-x_{1}x2}{12566(x_1^3{x}_2-x_1^4)} +\frac{1}{5108x_1^2}-1\le 0,\hfill \\ & g_3\left(x\right)=1-\frac{140.45x_1}{x_2^2{x}_3}\le 0,\hfill \\ & g_4\left(x\right)=\frac{x_1+x_2}{1.5}1\le 0,\hfill \\ & 0.05\le x_1\le 2,0.25\le x_2\le 1.3,2\le x_3\le 15.\hfill \end{array}\right.$$

(31)

As observed in

Table 11. Comparison of the optimal results of different algorithms for the pressure spring problem. NA indicates that it has not been addressed in the literature.

Method	Design Variables			Cost
	${\mathit{x}}_{\mathbf{1}}$	${\mathit{x}}_{\mathbf{2}}$	${\mathit{x}}_{\mathbf{3}}$	$\mathit{f}\left(\mathit{X}\right)$
CPSO [44]	0.05112728	0.357644	11.244543	0.012674
APSO [45]	0.052588	0.378343	10.138862	0.012700
IAPSO [46]	0.051685	0.356629	11.294175	0.012665
CVI-PSO [34]	NA	NA	NA	0.0126655
PSO-DE	0.051689	0.356717	11.28965	0.012655

, it can be found that PSO-DE and IAPSO obtain better optimal results than the other algorithms in the simulation results among the four algorithms. In

Table 12. Comparison of optimization results of different algorithms for the pressure spring problem.

Method	Best	Worst	Mean	Std.
CPSO [44]	0.12674	0.12924	0.12730	5.20E-04
APSO [45]	0.012700	0.014937	0.013297	6.85E-04
IAPSO [46]	0.012655	0.017829	0.013677	1.53E-03
CVI-PSO [34]	0.012666	0.012842	0.012730	5.58E-05
PSO-DE	0.012655	0.012655	0.012655	2.04E-05

, we can further see that the optimal values obtained by PSO-DE are better than those obtained by CPSO and APSO. It is close to the optimal values obtained by CVI-PSO and IAPSO, but PSO-DE is significantly better than these two algorithms in terms of the worst value, mean and square deviation, which indicates that the algorithm has better stability. Therefore, the PSO-DE algorithm outperformed the other algorithms in solving the pressure vessel problem.

6.4.2. RC19

The goal of the welding beam design problem is to obtain the minimum manufacturing cost. As shown in Figure 9, there are four design variables associated with this problem, weld thickness $h=x_1$, welded joint length $l=x_2$, beam width $t=x_3$ and beam thickness $b=x_4$, the decision vector is $X=(x_1,x_2,x_3,x_4)$. The mathematical expression for the objective function $f\left(X\right)$ consists mainly of the total manufacturing cost. While optimizing the design variables, we ensure that the seven constraints are not violated. These constraints are shear stress $\tau$, bending stress in beam $\sigma$, deflection of beam end $\delta$ and buckling load of bar $P_b$. Its mathematical model is as follows:

$$\left\{\begin{array}{cc}min\hfill & f\left(x\right)=0.10471x_1^2{x}_2+0.04811x_3{x}_4(14+x_2)\hfill \\ s.t\hfill & g_1\left(x\right)=\tau \left(X\right)+{\tau }_{max}\le 0,\hfill \\ & g_2\left(x\right)=\sigma \left(X\right)+{\sigma }_{max}\le 0,\hfill \\ & g_3\left(x\right)=\delta \left(X\right)+{\delta }_{max}\le 0,\hfill \\ & g_4\left(x\right)=x_1-x_4\le 0,\hfill \\ & g_5\left(x\right)=P-P_b\le 0,\hfill \\ & g_6\left(x\right)=0.125-x_1\le 0,\hfill \\ & where 0.1\le x_1,x_2\le 2,0.1\le x_3,x_4\le 10.\hfill \\ & \tau \left(X\right)=\sqrt{{\left({\tau }^{\prime }\right)}^2+{\left({\tau }^{″}\right)}^2+\left(l{\tau }^{\prime }{\tau }^{″}\right)/\sqrt{0.25(l^2+{(h+t)}^2)}},\hfill \\ & {\tau }^{\prime }=\frac{6000}{\sqrt{2}hl},\sigma \left(X\right)=\frac{504000}{t^{2}b},\delta \left(X\right)=\frac{65856000}{(30\times {10}^6)b{t}^3}\hfill \\ & {\tau }^{″}=\frac{6000(14+0.5l)\sqrt{0.25(l^2+{(h+t)}^2)}}{2\left[0.707hl(l^2/12+0.25{(h+t)}^2)\right]},\hfill \\ & P_b\left(X\right)=64746.022(1-0.0282346t)t{b}^3.\hfill \end{array}\right.$$

(32)

Table 13. Comparison of optimization results of different algorithms for the welding beam problem. NA indicates that it has not been addressed in the literature.

Method	Design Variables				Cost
	${\mathit{x}}_{\mathbf{1}}$	${\mathit{x}}_{\mathbf{2}}$	${\mathit{x}}_{\mathbf{3}}$	$\mathit{f}\left(\mathit{X}\right)$
CPSO [44]	0.202369	3.544214	9.048210	0.205723	1.728024
APSO [45]	0.202701	3.574272	9.040209	0.205215	1.736193
IAPSO [46]	0.2057296	3.4708866	9.03662391	0.20572964	1.7248523
CVI-PSO [34]	NA	NA	NA	NA	1.724852
PSO-DE	0.2057296	3.47048866	9.03662391	0.20572963	1.724852

and

Table 14. Comparison of optimization results of different algorithms for the welding beam problem.

Method	Best	Worst	Mean	Std.
CPSO [44]	1.782143	1.782143	1.748831	1.29E-02
APSO [45]	1.736193	1.993999	1.877851	0.076118
IAPSO [46]	1.7248523	1.7248624	1.7248528	2.02E-06
CVI-PSO [34]	1.724852	1.727665	1.725124	6.12E-04
PSO-DE	1.724852	1.724852	1.724852	1.4E-15

show the comparison results of PSO-DE with several other algorithms for the optimization problem of the welded beam structure in terms of finding the optimum. The optimal value of the variables of the optimal solution is found to be on the boundary of the feasible region. The optimal value of the objective function obtained by PSO-DE is 1.724852, which is better than the results obtained by CPSO, APSO and IAPSO, and is similar to the optimal value obtained by CVI-PSO, but the mean, the worst value and the standard deviation obtained by PSO-DE are better than those obtained by CVI- PSO. Therefore, the PSO-DE algorithm outperforms the other algorithms in solving the welded beam structure problem.

7. Conclusions

In this paper, we propose the PSO-DE algorithm, which combines both particle swarm optimization and differential evolution algorithm, and uses differential evolution to improve the performance of particle swarm optimization. To verify the effectiveness and advancement of PSO-DE, it is applied to 12 classical constrained benchmark functions and 57 engineering optimization problems, and 2 common engineering optimization problems (RC17 and RC19) are selected from the 57 engineering problems for further analysis. The study of numerical results shows that PSO-DE has good stability and robustness, and good global search capability. Meanwhile, the PSO-DE algorithm has obvious advantages in solving constrained optimization, and its performance is better than other PSO variants and state-of-the-art EAs algorithms, which have great potential in solving complex practical problems. Therefore, PSO-DE is a constrained evolutionary algorithm worth adopting and promoting.

In further research, more efficient constraint processing techniques need to be designed and embedded in various metaheuristic algorithms. Moreover, multi-objective optimization is a very challenging topic. It is also interesting to combine the proposed constraint evolution algorithms with other techniques to solve multi-objective optimization problems. When dealing with realistic problems, it is important to pay more attention to the fundamental impact of the operator in addition to analyzing the diversity and striking a balance between exploration and exploitation. In the future, PSO-DE can be used to solve more complex real-world problems, such as image segmentation, data clustering and feature selection for photovoltaic cells.