Guided Hybrid Modified Simulated Annealing Algorithm for Solving Constrained Global Optimization Problems

Abstract

In this paper, a hybrid gradient simulated annealing algorithm is guided to solve the constrained optimization problem. In trying to solve constrained optimization problems using deterministic, stochastic optimization methods or hybridization between them, penalty function methods are the most popular approach due to their simplicity and ease of implementation. There are many approaches to handling the existence of the constraints in the constrained problem. The simulated-annealing algorithm (SA) is one of the most successful meta-heuristic strategies. On the other hand, the gradient method is the most inexpensive method among the deterministic methods. In previous literature, the hybrid gradient simulated annealing algorithm (GLMSA) has demonstrated efficiency and effectiveness to solve unconstrained optimization problems. In this paper, therefore, the GLMSA algorithm is generalized to solve the constrained optimization problems. Hence, a new approach penalty function is proposed to handle the existence of the constraints. The proposed approach penalty function is used to guide the hybrid gradient simulated annealing algorithm (GLMSA) to obtain a new algorithm (GHMSA) that finds the constrained optimization problem. The performance of the proposed algorithm is tested on several benchmark optimization test problems and some well-known engineering design problems with varying dimensions. Comprehensive comparisons against other methods in the literature are also presented. The results indicate that the proposed method is promising and competitive. The comparison results between the GHMSA and the other four state-Meta-heuristic algorithms indicate that the proposed GHMSA algorithm is competitive with, and in some cases superior to, other existing algorithms in terms of the quality, efficiency, convergence rate, and robustness of the final result.

1. Introduction

Optimization problems arise in different applications fields, such as technical sciences, industrial engineering, economics, networks, chemical engineering, etc. See for example [1,2,3,4,5]

In general, the constrained optimization problem can be formulated as follows:

$$\begin{array}{cc}\underset{x\in {\mathbb{R}}^n}{min}& f\left(x\right),\hfill \\ s.t& g_{{}_l}\left(x\right)\le 0,l=1,2,\dots ,q,\hfill \\ & h_{{}_d}\left(x\right)=0,d=1,2,\dots ,m,m<n\hfill \\ & a_{{}_i}\le x_{{}_i}\le b_{{}_i},i=1,2,\dots ,n,\hfill \end{array}$$

(1)

where $a_{{}_i}\in \{\mathbb{R}\cup \{-\infty \}\}$, and $b_{{}_i}\in \{\mathbb{R}\cup \{\infty \}\}$.

The functions $f\left(x\right)$, $g_l\left(x\right)$, $h_j\left(x\right)$: ${\mathbb{R}}^n\to \mathbb{R}$ are real valued functions, n denotes the number of variables in x, q is the number of inequality constraints, m is the number of equality constraints, a is a lower bounded on x and b is an upper bounded on x. The objective function f, the inequality constraints $g_{{}_l}$, $l=1,2,\dots ,q$, and the equality constraint $h_{{}_d}$, $d=1,2,\dots ,m$, are assumed to be continuously differentiable nonlinear functions.

Recently, there has been great development of optimization algorithms that are proposed to find global solutions to optimization problems. See for example [2,6,7,8].

The global optimization methods are used to prevent convergence to local optima and increase the probability of finding the global optimum [9].

The numerical global optimization algorithms can be classified into two classes: deterministic and stochastic methods. In stochastic methods, the minimization process depends partly on probability. In deterministic methods, in contrast, no probabilistic information is used [9].

So, for finding the global minimum of the unconstrained problem by using deterministic methods, it needs an exhaustive search over the feasible region of the function f and additional assumptions for the function f. On the contrary, to find the global minimum of the unconstrained problems, by using stochastic methods, one can prove the asymptotic convergence in probability, i.e., these methods are asymptotically successful with probability 1, see for example [10,11,12]. In general, the computational results of the stochastic methods are better than those of the deterministic methods [13].

Due to those reasons, a meta-heuristics strategy (stochastic method) is used to guide the search process [13]. Hence a meta-heuristic is a technique designed for solving a problem more quickly when classic methods are too slow, or for finding an approximate solution when classic methods fail to find any exact or near-exact solution. This is achieved by trading optimality, completeness, accuracy, or precision for speed [14,15,16].

The simulated-annealing algorithm (SA) is one of the most successful meta-heuristic strategies. In fact, the numerical results display that the simulated annealing technique is very efficient and effective for finding the global minimizer. See, for example, [2,5,17,18,19].

On the other hand, the gradient method is the most inexpensive method for finding a local minimizer of a continuously differentiable function. It has been proved that the gradient algorithm converges locally to a local minimizer [20]. Therefore, if a line-search (L) is added to the gradient method (G) as a globalization strategy, the resulting algorithm is globally convergent to a local minimizer (GL) [9,21,22].

Hence, when the simulated-annealing algorithm (SA) as a global optimization algorithm is combined with the line-search gradient method (GL) as a globally convergent method, the result is the hybrid gradient simulated annealing algorithm (GLMSA) [23]. The idea behind this hybridization is to gain the benefits and advantages of both the GL algorithm and the MSA algorithm.

As a matter of fact, the numerical results demonstrated that the (GLMSA) algorithm is a very efficient, effective and strong competitor for finding the global minimizer. For example, Table 4 of [23] shows that the GL algorithm is able to reach the optimum point of all test problems whose objective functions have only one minimum point (no local minima except the global one. i.e., convex function) and it is stuck at a local minimum for test problems whose objective functions have several local minima (with one global minimum, i.e., non-convex function). Table 6 of [23] demonstrates that the SMA modified simulated annealing algorithm finds the global minimum of all test problems from any starting point of the feasible search space S. However, the GLMSA hybrid gradient simulated annealing algorithm is faster than MSA; also, GLMSA is efficient and effective compared to other meta-heuristic algorithms.

All the above have motivated and encouraged us to generalize the GLMSA algorithm to solve Problem (1).

The literature review analysis shows that the handling constraint which is based on a penalty function is considered the most popular implemented mechanism; this is due to its simplicity and ease of implementation [24,25,26,27]. A penalty technique transforms Problem (1) into an unconstrained problem by adding the penalty term of each constraint violation to the objective function value. The remainder of this paper is organized as follows. The next section provides a brief description of the GLMSA algorithm. Constraint handling, the penalty function method, proposed penalty method and interior-point algorithm are presented in Section 3. A guided hybrid simulated annealing algorithm to solve constrained problems is presented in Section 4. Numerical results are given in Section 5. Section 6 contains some concluding remarks.

Note: Section Abbreviations provides a list of the abbreviations and symbols which are used in this paper.

2. Summarized Description of GLMSA Algorithm

The GLMSA algorithm has been designed for solving unconstrained optimization problems; in this paper the GLMSA algorithm is generalized to solve Problem (1). The GLMSA algorithm contains two approaches to find a new step at each iteration, the first one is the gradient method. In this approach, a candidate point is generated and it might be accepted or rejected. If the objective function f is decreased at this point, then it will be accepted, otherwise, the second approach will be used to generate another point.

2.1. The First Approach (Gradient Method)

The gradient method solves an unconstrained optimization problem iteratively, such that at each iteration, a step in the direction of the negative gradient is computed and added to the current point as follows. Given an initial guess $x_{{}_0}\in {\mathbb{R}}^n$, the gradient method generates a sequence $\left\{x_{{}_k}\right\}$, $k\ge 0$ of the objective function of the unconstrained optimization problem such that:

$$x_{k+1}=x_{{}_k}+d_{{}_k},$$

(2)

where $d_{{}_k}$ is the first step, and it is defined by:

$$d_k=-\left|{\alpha }_k\right|g\left(x_{{}_k}\right),$$

(3)

where $g\left(x_{{}_k}\right)$ the gradient vector of the function f at point $x_{{}_k}$ and ${\alpha }_k$ is a step length along the negative gradient direction ($-g\left(x_{{}_k}\right)$). The step length ${\alpha }_k$ along the $-g\left(x_{{}_k}\right)$ is defined by:

$${\alpha }_k=\frac{f\left(x_k\right)}{\Vert g\left(x_{{}_k}\right){\Vert }_2^2}.$$

(4)

The G gradient algorithm is listed in Algorithm 1 of [23]. The step length ${\lambda }_{{}_k}$ that is computed by the backtracking line-search approach is very important for global convergence of the gradient method. The following section presents a brief description of the backtracking line-search approach for globalizing the gradient method.

Globalizing the First Approach (Gradient Method)

To make the gradient method capable of finding a local minimizer $x^{*}$ of the objective function of the unconstrained optimization problem from any starting point $x_{{}_0}$, the G algorithm (gradient algorithm) is combined with the L algorithm (line-search algorithm) in order to obtain globally convergent algorithm $GL$. This algorithm is listed in Algorithm 1 below and it contains the first approach (gradient algorithm G) and the backtracking line-search algorithm L.

Algorithm 1 Line-Search Gradient Algorithm “GL”

Input: $f:{\mathbb{R}}^n\to \mathbb{R}$, $f\in C^1$, $\gamma \in (0,1)$, $k=0$, a starting point $x_{{}_k}\in {\mathbb{R}}^n$ and $\epsilon >0$.

Output: $x^{*}=x_{{}_{ac}}$ the local minimizer of f, $f\left(x^{*}\right)$, the value of f at $x^{*}$

1: Set $x_{ac}=x_0$.             ▹$x_{ac}$ is accepted solution.

2: Compute $f_{ac}=f\left(x_{ac}\right)$, $g_{ac}=g\left(x_{ac}\right)$ and $d_{{}_k}$.

3: while $\parallel g_{ac}{\parallel }_2>\epsilon$ do ▹$g_{{}_{ac}}$ is the value of the gradient vector at the accepted point $x_{ac}$.

4:    Set $k=k+1$.

5:    $x_{{}_k}=x_{ac}+d_{{}_k}$      ▹$x_{ac}$ is the accepted point form the previous iteration.

6:    Compute $f_{{}_k}=f\left(x_{{}_k}\right)$

7:    Set $\lambda =1$.

8:    while $f_{{}_k}>f_{ac}+\gamma \lambda g_{{}_{ac}}^T{d}_{{}_k}$ do

9:       Set $\lambda =\frac{\lambda }{2}$

10:     $x_{{}_k}=x_{{}_{ac}}-\lambda g_{{}_{ac}}$         ▹ in this paper the value of $\gamma$ is ${10}^{-4}$.

11:     Compute $f_{{}_k}=f\left(x_{{}_k}\right)$

12:    end while

13:    Set $x_{{}_{ac}}\leftarrow x_{{}_k}$ and $f_{ac}\leftarrow f\left(x_{{}_k}\right)$.

14:    Compute $g_{{}_{ac}}=g\left(x_{ac}\right)$ and $d_{{}_k}$.

15: end while

16: return $x_{ac}$ the local minimizer and its function value $f_{ac}$

For more details about the gradient method and the backtracking line-search approach see [23]. The second approach of the GLMSA algorithm is presented in the following subsection.

2.2. The Second Approach (Simulated Annealing SA)

It must be noted that the modified simulated annealing algorithm in [23] contains three alternatives to generate a new point, but in this paper, the first alternative is considered to generate a new point. This procedure is very important for reducing the function evaluations from three times at each iteration to one function evaluation for every iteration, because we need to allow for more inner iterations when solving constrained optimization problems. This procedure guarantees that the parameters of the penalty function are increasing enough because it is a necessary condition for non-stationary penalty functions [28], i.e., when $k\to \infty$, parameters must also go to infinity.

The second point is generated by

$$x_{k^{\prime }+1}=x_{{}_{ac}}+{\psi }_{{}_k^{\prime }},$$

(5)

where $x_{{}_{ac}}$ is the best point which is accepted so far and ${\psi }_{{}_k^{\prime }}$ is the step of the second approach and computed by Algorithm 2 below.

Algorithm 2 The second approach to generate the step ${\psi }_{{}_k^{\prime }}$.

Step 1: Set $k^{\prime }=0$.

Step 2: Compute ${\omega }_{k^{\prime }}={10}^{(0.1*k^{\prime })}$.

Step 3: Generate a random vector $X_k^{\prime }\in {[-1,1]}^n$.

Step 4: Compute $D_{{}_{{}_k^{\prime }}}^i=\frac{-1+{(1+{\omega }_{{}_k^{\prime }})}^{|X_{{}_k^{\prime }}^i|}}{{\omega }_{{}_k^{\prime }}}$, $i=1,2,\dots ,n$.    ▹n is the number of variables.

Step 5: Set $D{X}_{{}_k^{\prime }}^i=sign(X_{{}_k^{\prime }}^i)$.

Step 6: Compute $D{E}_{{}_k^{\prime }}^j=D_{{}_k^{\prime }}^i\ast D{X}_i^j$.

Step 7: Compute ${\psi }_{{}_k^{\prime }}^i=b_{{}_i}\ast D{E}_{{}_k^{\prime }}^i$.    ▹$b_{{}_i}$ is the upper bound of the feasible search space.

Step 8: $k^{\prime }\leftarrow k^{\prime }+1$.

Step 9: Repeat steps 2–8 until $k^{\prime }=N$.   ▹N is the number of iterations and it is given in advance.

The gradient line-search algorithm (GL) has been listed in Algorithm 1 and a modified simulated annealing algorithm (MSA) is illustrated by Algorithm 3.

Algorithm 3 Modified Simulated-Annealing “MSA”.

Input: $x_{ac}$, $f_{ac}$, N and T.       ▹T control parameter (Temperature)

Output:$x_{best}$ is the best point of N points and it value $f_{best}$

1: for $k^{\prime }=0\to N$ do

2:    $x_{{}_{{}_k^{\prime }}}=x_{ac}+{\psi }_{{}_k^{\prime }}$, using Equation (5).

3:    Compute $\Delta f=f(x_{{}_{{}_k^{\prime }}})-f_{ac}$.

4:    if $\Delta f<0$ then

5:       Set $x_{a{c}_{{}_k^{\prime }}}\leftarrow x_{{}_{{}_k^{\prime }}}$, $f_{a{c}_{{}_k^{\prime }}}\leftarrow f(x_{{}_{{}_k^{\prime }}})$.

6:    else

7:       Generate a random number $\beta \in (0,1)$

8:       if $\beta <e^{-\frac{\Delta f}{T}}$ then

9:         Set $x_{a{c}_{{}_k^{\prime }}}\leftarrow x_{{}_{{}_k^{\prime }}}$, $f_{a{c}_{{}_k}}\leftarrow f(x_{{}_{{}_k^{\prime }}})$.

10:       end if

11:    end if

12: end for

13: return $x_{ac}$ and its function value $f_{ac}$.         ▹$f_{ac}=f\left(x_{ac}\right)$.

where N is the maximum number of possible trials (Length Markov Chains of MSA) and T is the control parameter (temperature). For more details about the MSA algorithm, please, see [23].

For a detailed description of the simulated annealing algorithm SA see for example [18,29,30,31].

As we have mentioned above, Algorithm 1 (gradient line-search algorithm (GL)) is hybridized with Algorithm 3 (a modified simulated annealing algorithm (MSA)) to get the LGMSA algorithm that solves the unconstrained optimization problem.

In the next section, the LGMSA algorithm is guided to solve Problem (1) by using the penalty function method. There are many methods for handling the existence of the constraints in the constrained problem.

3. Constraints Handling

The algorithms which have been proposed to solve unconstrained optimization problems are unable to deal directly with constrained optimization problems. There are several approaches proposed to handle the existence of the constraints, see for example [27,32,33]. The most popular of them is the penalty function method.

The penalty function method is a successful technique for handling constraints [27,34,35].

3.1. Penalty Function Methods

The penalty methods have been most widely studied and used due to their simplicity in implementation. The major definition of the penalty function methods is the degree to which each constraint is penalized [28]. There are several types of penalty methods that are used to penalize the constraints in constrained optimization problems.

Three groups of penalty function methods are most popular; the first one is a group of methods of static penalties. In these methods, the penalty parameter does not depend on the current iteration, i.e., parameters remain constant through the evolutionary process [24,36].

The second one is a set of methods of dynamic penalties. In these methods the penalty parameters are usually dependent on the current iteration, in other words, the penalty parameters are functions in the iteration k, i.e., they are non-stationary. See [24,37,38].

The third is a set of methods of adaptive penalties; in this group penalty parameters are updated for every iteration [24].

The next section presents a suggested penalty function method with dynamic and adaptive parameters.

Proposed Penalty Function Method

This section shows how Problem (1) is transformed to an unconstrained optimization problem which is simple bounded as follows:

$$\begin{array}{cc}\underset{x\in {\mathbb{R}}^n}{min}& \theta (x,r)=f\left(x\right)+rp\left(x\right),\hfill \\ s.t& a_{{}_i}\le x_{{}_i}\le b_{{}_i},i=1,2,\dots ,n,\hfill \end{array}$$

(6)

where $f\left(x\right)$ is the original objective function in Problem (1), r is a penalty parameter. The penalty term $p\left(x\right)$ is defined by:

$$p\left(x\right)=\sum _{l=1}^q{\left(max\{0,g_{{}_l}\left(x\right)\}\right)}^2+\sum _{j=1}^m{\left|h_{{}_j}\left(x\right)\right|}^2.$$

(7)

The difference between the penalty function methods is in the way of defining the penalty term and its parameter r [24].

The penalty function methods force infeasible points toward the feasible region by step-wise increasing the penalty; r is used in the penalizing function $p\left(x\right)$.

Therefore, the solution $x^{*}$ minimizes the objective function of Problem (6) and also minimizes the objective function of Problem (1), i.e., as long as $k\to \infty$ and $r_{{}_k}\to \infty$, $x^{*}$ approaches the feasible region and $r_{{}_k}p\left(x\right)\to 0$ [28].

In this paper, the penalty function method has two parameters—the first one is r which penalizes the inequality constraint that is violated, i.e., when $g_{{}_l}\left(x\right)>0$. The second parameter is t which penalizes the equality constraint $h_{{}_j}\left(x\right)$ whose value is not equal to zero.

Accordingly, the $\theta (x,r)$ function is defined by:

$$\theta (x,r)=f\left(x\right)+\frac{r}{2}{p}_{{}_1}\left(x\right)+\frac{t}{2}{p}_{{}_2}\left(x\right),$$

(8)

where $p_{{}_1}\left(x\right)={\displaystyle \sum _{l=1}^q}{\left(max\{0,g_{{}_l}\left(x\right)\}\right)}^2$, $p_{{}_2}\left(x\right)={\displaystyle \sum _{j=1}^m}{\left|h_{{}_j}\left(x\right)\right|}^2$ and r and t are the parameters for inequality and equality constraints respectively.

The parameters r and t are updated at each iteration k as follows.

$$\begin{array}{cc}& r_{{}_{k+1}}=r_{{}_k}+{\phi }_{{}_k}\ast {\Phi }_{{}_k},\hfill \\ & t_{{}_{k+1}}=t_{{}_k}+1,\hfill \end{array}$$

(9)

where the parameter ${\phi }_{{}_k}$ is updated by:   

$${\phi }_{{}_k}=\left\{\begin{array}{cc}0\hfill & \mathrm{if}{g}_{{}_l}\left(x\right)\le 0,\hfill \\ 2\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

(10)

The parameter ${\phi }_{{}_k}$ is an adaptive parameter, i.e., when the candidate solutions are out of the feasible region then ${\phi }_{{}_k}$ penalizes a violated constraint by multiplying the term ${\Phi }_{{}_k}$ by 2, where $r_0=1$ is the initial value of r. The parameter $\Phi$ is updated as follows: ${\Phi }_{{}_{k+1}}={\Phi }_{{}_k}+1$, $t_{{}_0}=1$.

Note: The equality constraint is more difficult than the inequality constraint because the size of the feasible region of the equality constraint is smaller than the size of the feasible region of the inequality constraint. For example, f$(x,y)=xy$ s.t $h(x,y)=x^2+y^2-1=0$ and $f(x,y)=xy$ s.t $g(x,y)=x^2+y^2-1<=0$. The first problem is much harder than the second because in the first problem the size of the feasible region is the circumference of the circle while in the second problem, the feasible region is the whole disk. So, the parameter $t\left(k\right)$ must be taken carefully.

3.2. Mechanism of Working of the Penalty Function Method

The penalty method solves the general Problem (1), during a succession of unconstrained optimization problems.

Let us discuss two examples in order to illustrate how the parameters of the penalty function are run.

The first example is very easy (one dimension); minimize $f\left(x\right)=x^2-3$ subject to $g\left(x\right)=0.5-0.5x\le 0$, where $S=[-6,6]$ is the search domain.

If we want to find the optimal solution of the objective function $f\left(x\right)=x^2-3$ as an unconstrained problem, it is clear that the global solution to this problem is the point $x^{*}=0$, such that $f\left(x^{*}\right)=-3$, for $x\in \mathbb{R}$, but when we want to find the optimal solution of the objective function $f\left(x\right)=x^2-3$ subject to $g\left(x\right)\le 0$, in this case, the problem is very difficult because we have to find the point $x^{*}$ that minimizes $f\left(x\right)$ and at the same time it must satisfy the condition of the constraint $g\left(x\right)\le 0$, which is why we need to apply the penalty function.

Hence, the problem $f\left(x\right)=x^2-3$ subject to $g\left(x\right)=0.5-0.5x\le 0$ is transformed into $\theta (x,r)=x+\frac{r}{2}(max{\{0,(\frac{1}{2}-0.5x)\}}^2)$, if $g\left(x\right)>0$; ($g\left(x\right)$ is violated), the first derivative is computed by the function $\theta (x,r)$; $\frac{d\theta (x,r)}{dx}=1-\frac{r}{2}(\frac{1}{2}-0.5x)$, then $1-\frac{r}{2}(\frac{1}{2}-0.5x)=0$; $x^{*}=1-\frac{4}{r}$, when $r=\{1,2,3,\dots ,\infty \}$, then $x^{*}=\{-3,-1,\frac{-1}{3},\dots ,1\}$, $f\left(x^{*}\right)=\{6,-2,\frac{-26}{9}\dots ,-2\}$ and $g\left(x^{*}\right)=\{2,1,\frac{2}{3},\dots ,0\}$, i.e., when $r\to \infty$, $x^{*}\to 1$, $g\left(x^{*}\right)\to 0$, $rp\left(x^{*}\right)\to 0$, $f\left(x^{*}\right)\to -2$, and $\theta (x^{*},r)\to -2$.

Hence, the optimal point is $x^{*}=1$, such that $f\left(x^{*}\right)=-2$ and the constraint $g\left(x^{*}\right)=0$ is satisfied.

Figure 1 illustrates the behavior of the penalty functions; $r{p}_{{}_1}\left(x\right)$, $r{p}_{{}_2}\left(x\right)$ and $r{p}_{{}_3}\left(x\right)$ and the objective function $f\left(x\right)$ of the original problem (constrained problem) and the objective function $\theta (x,r)$ of the transformed problem (unconstrained problem) for all $x\in S=[-6,6]$.

Example 2: minimize $-xy$ s.t $g(x,y)=x+2y-4\le 0$; $\theta (x,y,r)=-xy+\frac{r}{2}(max{\{0,(x+2y-4)\}}^2)$, if $g(x,y)>0$; ($g(x,y)$ is violated), the gradient vector is computed by the function $\theta (x,y)$; $g(x,y)=[-y+r(x+2y-4),-x+2r(x+2y-4\left)\right]$, hence, $(x^{*},y^{*})=(\frac{2}{1-\frac{1}{4r}},\frac{1}{1-\frac{1}{4r}})$, then $(x^{*},y^{*})\to (2,1)$ as $r\to \infty$; this is why it must allow for the parameters $r_{{}_k}$ and $t_{{}_k}$ to increase as long as there exists a violated constraint, i.e., when a process of searching for a solution is an infeasible region.

To ensure that the process of searching for the optimal solution remains within the search domain, the interior-point algorithm is used. Therefore, the next section presents a brief description of this technique.

3.3. Interior-Point Method

The interior-point method is used in this paper, when a simple bounded exists in the test problem. Therefore, the interior point technique is used to ensure that the candidate solution lies inside a feasible region. This technique is used as follows at each iteration k, a damping parameter ${\tau }_k$ is applied to insure that $x_{k+1}$ is feasible with respect to the limits $a_i\le x_i\le b_i$, $i=1,2,\dots n$ and $k=1,2,\dots M$ as the inner loop of Algorithm 4, ref. [39].

Algorithm 4 Guided Hybrid Modified Simulated-Annealing Algorithm (GHMSA).

Input:$f\left(x\right),g_{{}_l}\left(x\right)\mathrm{and}{h}_{{}_d}\left(x\right):{\mathbb{R}}^n⟶\mathbb{R}$, $x_{{}_0}\in {\mathbb{R}}^n$, M, T, $T_{{}_f}$, $T_{out}$, $\epsilon$, $r_{{}_0}$, ${\Phi }_{{}_0}$ and $t_{{}_0}$.

1: set $x_{{}_{ac}}=x_{{}_0}$   ▹ at the beginning we accept the initial point $x_{{}_0}$ as an optimal solution.

2: compute $\theta \left(x_{ac}\right)=f\left(x_{ac}\right)+\frac{r_{{}_k}}{2}{p}_{{}_1}\left(x_{ac}\right)+\frac{t_{{}_k}}{2}{p}_{{}_2}\left(x_{ac}\right)$           ▹ Using Formula (8).

3: set ${\theta }_{{}_b}=\theta \left(x_{ac}\right)$ and ${\theta }_{{}_{\delta }}=1$. ▹ The values of ${\theta }_{{}_b}$ and ${\theta }_{{}_{\delta }}=1$ are updated after M iterations.

4: while $(T>T_{{}_f}$ and ${\theta }_{\delta }>\epsilon )$ or $(T>T_{out})$ do         ▹$T_{out}<T_{{}_f}\le {10}^{-4}$ are as stopping criteria.

5:     for $k=0$ to M do

6:       compute $\theta \left(x_{ac}\right)=f\left(x_{ac}\right)+\frac{r_{{}_k}}{2}{p}_{{}_1}\left(x_{ac}\right)+\frac{t_{{}_k}}{2}{p}_{{}_2}\left(x_{ac}\right)$.

7:       set ${\theta }_{ac}=\theta \left(x_{ac}\right)$.

8:       compute $x_{{}_1}=x_{ac}+d_{{}_k}$.            ▹$d_{{}_k}$ is competed by (16).

9:       go to Formula (8) to ensure that the point $x_{{}_1}$ lies inside ${[a,b]}^n$▹ by Formula (14).

10:       compute $\Delta \theta =\theta \left(x_{{}_1}\right)-{\theta }_{{}_{ac}}$

11:       if $\Delta \theta <0$ then

12:         go to Algorithm 1.

13:       else

14:         go to Formula (5) to generate other point.

15:       end if

16:     end for

17:     compute ${\Phi }_{{}_{k+1}}={\Phi }_{{}_k}+1$        ▹ here update penalty parameters.

18:     $T=r_{{}_T}\ast T$         ▹ decrease temperature, where $r_{{}_T}=0.8$.

19:     compute ${\theta }_{\delta }=|{\theta }_{{}_b}-{\theta }_{{}_{ac}}|$ and ${\theta }_{{}_b}\leftarrow {\theta }_{ac}$.▹${\theta }_{\delta }$ is a stopping criterion when the solutions converge in the accumulation point for all iterations.

20: end while

21: Set $x_{{}_g}\leftarrow x_{{}_{ac}}$, ${\theta }_{{}_g}\leftarrow {\theta }_{ac}$

22: return $x_{{}_g}$ the global minimizer and the value of the objective function $\theta \left(x_{{}_g}\right)$ at $x_{{}_g}$.

The damping parameter ${\tau }_k$ is defined to be:

$${\tau }_k=min\{1,\underset{i}{min}\{u_k^i,v_k^i\}\},$$

(11)

where

$$u_k^i=\left\{\begin{array}{cc}\frac{\left[a^i-x_k^i\right]}{\Delta x_k^i}\hfill & \mathrm{if}{a}^i>-\infty and\Delta x_k^i<0,\hfill \\ 1,\hfill & \mathrm{otherwise},\hfill \end{array}\right.$$

(12)

$$v_k^i=\left\{\begin{array}{cc}\frac{\left[b^i-x_k^i\right]}{\Delta x_k^i}\hfill & \mathrm{if}{b}^i<\infty and\Delta x_k^i>0,\hfill \\ 1\hfill & \mathrm{otherwise},\hfill \end{array}\right.$$

(13)

where $a^i$ and $b^i$ are the lower and upper bounds of the domain of the problem respectively, $i=1,2,\cdots n$, n is the number of variables of function in problem, $x_{{}_k}^i$ is the component ith of variable x at iteration k and $\Delta x_{{}_k}$ denotes the steps which are obtained by either Formula (2) or by Formula (5).

Since the $\left\{x_{{}_k}\right\}$ is always required to satisfy, for all k, $a<x_k<b$, and then the point $x_{{}_{k+1}}$ is computed by:

$$x_{{}_{k+1}}=x_{{}_k}+0.99{\tau }_{{}_k}\Delta x_{{}_k},$$

(14)

where the constant $0.99$ is a damping parameter to ensure that $x_{{}_k}$ is feasible with respect to the domain of function in the problem.

4. The Proposed Algorithm for Solving Constrained Optimization Problems (GHMSA)

According to the above procedures the GLMSA Algorithm is capable of solving Problem (1) as a constrained optimization problem during the solving of Problem (6) as an unconstrained optimization problem, hence there are some changes to the objective function $\theta (x,r)$ in Problem (6) to fit with the first step of the GLMSA Algorithm as follows.

• the function $f\left(x\right)$ is replaced by the function $\theta (x,r)$ defined in Equation (8), and then calculate

  $${\alpha }_{{}_k}=\frac{\theta \left(x_{{}_{ac}}\right)}{\Vert g(x_{{}_{ac}},r_{{}_k}){\Vert }_2^2},$$

  (15)

  where $x_{ac}$ is the accepted solution at iteration k,

  $$d_{{}_k}=-\left|{\alpha }_{{}_k}\right|g(x_{{}_{ac}},r_{{}_k}),$$

  (16)

  where the parameter $r_{{}_k}$ might denote r only or t only or both together according to a type of constrained optimization problem, for example, if the constraints contain mixed constraints inequality and equality, then $r_{{}_k}=(r_{{}_k},t_{{}_k})$.
• if the constrained problem contains simple bounded, we use Formula (14) to limit the new point inside this simple bounded.

In light of the above procedures, we rename the GLMSA Algorithm the “Guided Hybrid Modified Simulated-Annealing Algorithm” with the abbreviation “GHMSA”.

Setting Parameters of GHMSA Algorithm

The choice of a cooling schedule has an important impact on the performance of the simulated-annealing algorithm. The cooling schedule includes two terms: the initial value of the temperature T and the cooling coefficient $r_T$ which is used to reduce T. Many suggestions have been proposed in the literature for determining the initial value of the temperature T and the cooling coefficient $r_T$, see for example [4,18,40,41,42].

In general, it is a unanimous fact that the initial temperature T must be sufficiently high (to ensure escape from local points) and $r_T\in (0.1,1)$ [7,43,44]. In this section, we suggest that the initial value of T be related to the number of variables and the value of $f\left(x\right)$ at the starting point $x_0$. The cooling coefficient is taken to be $r_T\in [0.8,1)$ to decrease the temperature T slowly.

Therefore, the parameters used in Algorithm 4 are presented as follows. M is the inner loop maximum number of iterations, T is the control parameter (Temperature), $Tout$ is a final value of T, $r_{{}_T}$ is the cooling coefficient and $T_{{}_f}$ is a final value of T if it is sufficiently small.

The setting of parameters is as follows: $T={10}^4$, $\epsilon ={10}^{-6}$, $T_f={10}^{-14}$, $T_{out}={10}^{-20}$, $r_T=0.8$, and $M=10n$.

5. Numerical Result

To test the effectiveness and efficiency of the proposed algorithm, the algorithm is run on some test problems. The test problems are divided into two sets. The first set of test problems are taken from [45]. They are 24 well-known constrained real-parameter optimization problems. The objective functions in these problems take different shapes and the number of variables is between 2 and 24. These test problems also contain four types of constraints as follows: (LI) denotes a linear inequality, (LE) is a linear equality, (NI) refers to a nonlinear inequality, and (NE) denotes a nonlinear equality. They are listed in

Table 1. List of first and second types of test problems and their exact solutions.

pr	n	$\mathit{f}\left({\mathit{x}}^{*}\right)$	Kind of Function	$\mathit{LI}$	$\mathit{NI}$	$\mathit{LE}$	$\mathit{NE}$	a
G1	13	−15	quadratic	9	0	0	0	6
G3	10	−1.0005001000	polynomial	0	0	0	1	1
G4	5	−30,665.5386717834	quadratic	0	6	0	0	2
G5	4	5126.4967140071	cubic	2	0	0	3	3
G6	2	−6961.8138755802	cubic	0	2	0	0	2
G7	10	24.3062090681	quadratic	3	5	0	0	6
G8	2	−0.0958250415	nonlinear	0	2	0	0	0
G9	7	680.6300573745	polynomial	0	4	0	0	2
G10	8	7049.2480205286	linear	3	3	0	0	6
G11	2	0.7499000000	quadratic	0	0	0	1	1
G12	3	−1.0000000000	quadratic	0	1	0	0	0
G13	5	0.0539415140	nonlinear	0	0	0	3	3
G14	10	−47.7648884595	nonlinear	0	0	3	0	3
G15	3	961.7150222899	quadratic	0	0	1	1	2
G16	5	−1.9051552586	nonlinear	4	34	0	0	4
G18	9	−0.8660254038	quadratic	0	13	0	0	6
G19	15	32.6555929502	nonlinear	0	5	0	0	0
G24	10	−5.5080132716	polynomial	0	0	0	1	1

, where $f\left(x^{*}\right)$ is the best known optimal function value and a denotes the active constraint number at the known optimal solution. “The information mentioned in Table 1 is taken from [46]”.

The GHMSA Algorithm solved 18 test problems out of the 24 because the other problems are either not continuous or not differentiable. The second set of test problems contains four known non-linear engineering design optimization problems. These test problems do not have known exact solutions.

5.1. Results of “GHMSA” Algorithm

The GHMSA algorithm is programmed using MATLAB version 8.5.0.197613 (R2015a) and it is run on a personal laptop and the machine epsilon about $1\times {10}^{-16}$.

The results of our algorithm are compared against the results of the CB-ABC Algorithm in [47], the CCiALF Algorithm in [48], the NDE Algorithm in [49] and the CAMDE Algorithm in [50].

Liang et al. [45] suggested that the achieved function error values of the obtained optimal solution x after $5\times {10}^3$, $5\times {10}^4$ and $5\times {10}^5$ function evaluations (FES) are summarized in terms of {Best, Median, Worst, c, $\overline{v}$ ($\overline{v}=\frac{p\left(x\right)}{q+m}$, $p\left(x\right)$ is a penalty term in Equation (7)), Mean, s.d}.

The results are listed in

Table 2. Error values achieved if FES $=5\times {10}^3$, FES $=5\times {10}^4$, FES $=5\times {10}^5$ for G1, G3, G4, G5, G6 and G7.

FES		G1	G3	G4	G5	G6	G7
	Best	$1.93\times {10}^{-05}$ (0)	$2.70\times {10}^{-04}$ (0)	$2.68\times {10}^{-09}$ (0)	0.02 (3)	$8.00\times {10}^{-08}$ (0)	−1.26 (8)
	Median	$8.44\times {10}^{-05}$ (0)	$9.34\times {10}^{-04}$ (0)	$1.5\times {10}^{-05}$ (0)	0.41 (3)	$3.7\times {10}^{-06}$ (0)	0.206 (8)
	Worst	$9.99\times {10}^{-05}$ (0)	1 (0)	$4.25\times {10}^{-05}$ (0)	13.18 (3)	$2.89\times {10}^{-04}$ (0)	28.16 (8)
$5\times {10}^3$	c	0, 0, 0	0	0, 0, 0	0, 3, 3	0, 0, 0	0, 8, 8
	$\overline{v}$	0	$4.22\times {10}^{-04}$	0	0.02	0	0.016
	Mean	$8.15\times {10}^{-05}$	$1.77\times {10}^{-01}$	$1.9\times {10}^{-05}$	1.97	$1.7\times {10}^{-05}$	6.644
	s.d	$1.70\times {10}^{-05}$	0.380881	$1.48\times {10}^{-05}$	3.44	$5.5\times {10}^{-05}$	9.222
	Best	0 (0)	$3.62\times {10}^{-06}$ (0)	$1.09\times {10}^{-11}$ (0)	0.0016 (3)	$8\times {10}^{-08}$ (0)	−0.07 (4)
	Median	0 (0)	$3.64\times {10}^{-06}$ (0)	$8.00\times {10}^{-11}$ (0)	0.02 (3)	$1\times {10}^{-06}$ (0)	−$4.55\times {10}^{-03}$ (4)
	Worst	0 (0)	$3.99\times {10}^{-06}$ (0)	$9.82\times {10}^{-11}$ (0)	1.32 (3)	$4\times {10}^{-05}$ (0)	0.819 (4)
$5\times {10}^4$	c	0, 0, 0	0, 0, 0	0, 0, 0	0, 0, 3	0, 0, 0	0, 0, 0, 4
	$\overline{v}$	0	$1.51\times {10}^{-06}$	0	$3.45\times {10}^{-04}$	0	$2\times {10}^{-04}$
	Mean	0	$3.71\times {10}^{-06}$	$6.90\times {10}^{-11}$	0.166	$5\times {10}^{-06}$	−0.04
	s.d	0	$1.24\times {10}^{-07}$	$2.86\times {10}^{-11}$	0.3237	$8\times {10}^{-06}$	0.004
	Best	0 (0)	$9.99\times {10}^{-07}$ (0)	$1.09\times {10}^{-11}$ (0)	0.0016 (0)	$8\times {10}^{-08}$ (0)	$-1\times {10}^{-04}$ (0)
	Median	0 (0)	$2.58\times {10}^{-06}$ (0)	$8.00\times {10}^{-11}$ (0)	0.0233 (0)	$1\times {10}^{-06}$ (0)	−$3\times {10}^{-05}$ (0)
	Worst	0 (0)	$8.50\times {10}^{-06}$ (0)	$9.82\times {10}^{-11}$ (0)	1.3182 (0)	$4\times {10}^{-05}$ (0)	${410}^{-05}$ (0)
$5\times {10}^5$	c	0, 0, 0	0, 0, 0	0, 0, 0	0, 0, 0	0, 0, 0	0, 0, 0
	$\overline{v}$	0	$4.05\times {10}^{-07}$	0	$3.45\times {10}^{-05}$	0	$2.3\times {10}^{-05}$
	Mean	0	$2.37\times {10}^{-06}$	$6.90\times {10}^{-11}$	0.166	$5\times {10}^{-06}$	$-4\times {10}^{-05}$
	s.d	0	$1.92\times {10}^{-06}$	$2.86\times {10}^{-11}$	0.03237	$8\times {10}^{-06}$	$4\times {10}^{-05}$

,

Table 3. Error values achieved when FES $=5\times {10}^3$, FES $=5\times {10}^4$, FES $=5\times {10}^5$ for Problems G8, G9, G10, G11, G12 and G13.

FES		G8	G9	G10	G11	G12	G13
	Best	$1.05\times {10}^{-10}$ (0)	1.0467 (0)	4.77 (0)	$1.9\times {10}^{-04}$ (1)	0 (0)	$1.25\times {10}^{-04}$ (0)
	Median	$6.52\times {10}^{-09}$ (0)	1.494 (0)	17.67 (0)	$1.04\times {10}^{-03}$ (1)	0 (0)	$3.83\times {10}^{-03}$ (0)
	Worst	$4.13\times {10}^{-08}$ (0)	3.42 (0)	300.41 (0)	$5.66\times {10}^{-03}$ (1)	0 (0)	$9.83\times {10}^{-02}$ (0)
$5\times {10}^3$	c	0, 0, 0	0, 0, 0	0, 0, 0	0, 0, 1	0, 0, 0	0, 0, 0
	$\overline{v}$	0	0	0	0.00173	0	$7.27\times {10}^{-05}$
	Mean	$9.24\times {10}^{-09}$	1.863	49.86	0.00161	0	0.01171
	Std	$1.20\times {10}^{-08}$	0.9631	75.17	0.00151	0	0.02013
	Best	$1.05\times {10}^{-10}$ (0)	0.3489 (0)	0.10 (0)	$6.67\times {10}^{-05}$ (0)	0 (0)	$8.96\times {10}^{-06}$ (0)
	Median	$6.52\times {10}^{-09}$ (0)	$4.98\times {10}^{-01}$ (0)	0.35 (0)	$9.60\times {10}^{-05}$ (0)	0 (0)	$6.93\times {10}^{-05}$ (0)
	Worst	$4.13\times {10}^{-08}$ (0)	1.14 (0)	6.01 (0)	$9.96\times {10}^{-05}$ (0)	0 (0)	$2.94\times {10}^{-04}$ (0)
$5\times {10}^4$	c	0, 0, 0	0, 0, 0	0, 0, 0	0, 0, 0	0, 0, 0	0, 0, 0
	$\overline{v}$	0	0	0	$4.72\times {10}^{-06}$	0	$2.02\times {10}^{-06}$
	Mean	$9.24\times {10}^{-09}$	$6.21\times {10}^{-01}$	1.00	$9.60\times {10}^{-05}$	0	$4.39\times {10}^{-05}$
	Std	$1.20\times {10}^{-08}$	0.321034	1.50	$1.70\times {10}^{-06}$	0	$6.53\times {10}^{-05}$
	Best	$1.05\times {10}^{-10}$ (0)	$6.58\times {10}^{-05}$ (0)	0.02 (0)	$6.67\times {10}^{-05}$ (0)	0 (0)	$8.50\times {10}^{-06}$ (0)
	Median	$6.52\times {10}^{-09}$ (0)	$8.53\times {10}^{-05}$ (0)	0.09 (0)	$9.60\times {10}^{-05}$ (0)	0 (0)	$5.70\times {10}^{-05}$ (0)
	Worst	$4.13\times {10}^{-08}$ (0)	$9.88\times {10}^{-05}$ (0)	1.50 (0)	$9.96\times {10}^{-05}$ (0)	0 (0)	$9.90\times {10}^{-05}$ (0)
$5\times {10}^5$	c	0, 0, 0	0, 0, 0	0, 0, 0	0, 0, 0	0, 0, 0	0, 0, 0
	$\overline{v}$	0	0	0	$4.72\times {10}^{-06}$	0	$4.90\times {10}^{-07}$
	Mean	$9.24\times {10}^{-09}$	$8.38\times {10}^{-05}$	0.25	$9.60\times {10}^{-05}$	0	$5.50\times {10}^{-05}$
	Std	$1.20\times {10}^{-08}$	$1.52\times {10}^{-05}$	0.38	$1.70\times {10}^{-06}$	0	$2.70\times {10}^{-05}$

and

Table 4. Error values achieved when FES $=5\times {10}^3$, FES $=5\times {10}^4$, FES $=5\times {10}^5$ for Problems G14, G15, G16, G18, G19 and G24.

FES		G14	G15	G16	G18	G19	G24
	Best	$4.25\times {10}^{-01}$ (3)	−$5.83\times {10}^{-02}$ (2)	$4.31\times {10}^{-04}$ (0)	0.01 (0)	−$5.58\times {10}^{-02}$ (3)	$5.10\times {10}^{-12}$ (0)
	Median	1.4 (3)	0.18 (2)	0.0064 (0)	0.21 (0)	$4.28\times {10}^{-01}$ (3)	$9.05\times {10}^{-12}$ (0)
	Worst	1.53 (3)	55.40 (2)	0.0181 (0)	0.79 (0)	27.6 (3)	$9.99\times {10}^{-12}$ (0)
$5\times {10}^3$	c	0, 3, 3	0, 1, 2	0, 0, 0	0, 0, 0	0, 0, 3	0, 0, 0
	$\overline{v}$	$2.62\times {10}^{-02}$	$1.16\times {10}^{-03}$	0	0	$9.80\times {10}^{-03}$	0
	Mean	1.23	2.60	0.0076	0.27	4.07	$8.57\times {10}^{-12}$
	s.d	0.40	10.79	0.0046	0.21	7.69	$1.32\times {10}^{-12}$
	Best	$2.85\times {10}^{-07}$ (0)	$1.12\times {10}^{-07}$ (0)	$7.70\times {10}^{-11}$ (0)	$4.51\times {10}^{-06}$ (0)	−$5.58\times {10}^{-03}$ (3)	$5.10\times {10}^{-12}$ (0)
	Median	$4.68\times {10}^{-05}$ (0)	$6.96\times {10}^{-06}$ (0)	$8.40\times {10}^{-11}$ (0)	$7.97\times {10}^{-05}$ (0)	$4.28\times {10}^{-02}$ (3)	$9.05\times {10}^{-12}$ (0)
	Worst	$9.11\times {10}^{-05}$ (0)	$4.75\times {10}^{-04}$ (0)	$8.90\times {10}^{-11}$ (0)	$9.88\times {10}^{-05}$ (0)	2.76 (3)	$9.99\times {10}^{-12}$ (0)
$5\times {10}^4$	c	0, 0, 0	0, 0, 0	0, 0, 0	0, 0, 0	0, 0, 3	0, 0, 0
	$\overline{v}$	$3.62\times {10}^{-05}$	$8.10\times {10}^{-06}$	0	0	$9.80\times {10}^{-04}$	0
	Mean	$5.17\times {10}^{-05}$	$5.08\times {10}^{-05}$	$8.40\times {10}^{-11}$	$6.83\times {10}^{-05}$	$4.07\times {10}^{-01}$	$8.57\times {10}^{-12}$
	s.d	$2.42\times {10}^{-05}$	$1.20\times {10}^{-04}$	$3.40\times {10}^{-12}$	$2.65\times {10}^{-05}$	0.76855	$1.32\times {10}^{-12}$
	Best	$2.85\times {10}^{-07}$ (0)	$1.12\times {10}^{-07}$ (0)	$7.70\times {10}^{-11}$ (0)	$4.51\times {10}^{-06}$ (0)	−$9.94\times {10}^{-05}$ (0)	$5.10\times {10}^{-12}$ (0)
	Median	$4.68\times {10}^{-05}$ (0)	$6.96\times {10}^{-06}$ (0)	$8.40\times {10}^{-11}$ (0)	$7.97\times {10}^{-05}$ (0)	−$3.21\times {10}^{-05}$ (0)	$9.05\times {10}^{-12}$ (0)
	Worst	$9.11\times {10}^{-05}$ (0)	$4.75\times {10}^{-04}$ (0)	$8.90\times {10}^{-11}$ (0)	$9.88\times {10}^{-05}$ (0)	$8.70\times {10}^{-05}$ (0)	$9.99\times {10}^{-12}$ (0)
$5\times {10}^5$	c	0, 0, 0	0, 0, 0	0, 0, 0	0, 0, 0	0, 0, 0	0, 0, 0
	$\overline{v}$	$3.62\times {10}^{-05}$	$8.10\times {10}^{-06}$	0	0	$8.09\times {10}^{-05}$	0
	Mean	$5.17\times {10}^{-05}$	$5.08\times {10}^{-05}$	$8.40\times {10}^{-11}$	$6.83\times {10}^{-05}$	−$1.86\times {10}^{-05}$	$8.57\times {10}^{-12}$
	s.d	$2.42\times {10}^{-05}$	$1.20\times {10}^{-04}$	$3.40\times {10}^{-12}$	$2.65\times {10}^{-05}$	$6.76\times {10}^{-05}$	$1.32\times {10}^{-12}$

; where c is a concatenation of three numbers indicating the violated constraint number at the median solution by more than 1.0, between 0.01 and 1.0, and between 0.0001 and 0.1, respectively. $\overline{v}$ is the mean value of the violations of all constraints at the median solution. The numbers in the parenthesis after the error value of the Best, Median, Worst solution are the constraint numbers not satisfying the feasible condition of the Best, Median, and Worst solutions, respectively. Table 2, Table 3 and Table 4 denote that the GHMSA can determine feasible solutions at each run utilizing $5\times {10}^3$ FES for 12 test problems {G01, G03, G04, G06, G08, G09, G10, G12, G13, G16, G18, G24}. As for problems G11, G14 and G15, the GHMSA Algorithm finds feasible solutions by using $5\times {10}^4$ FES. For the other three test problems, {G05, G07, G19}, the GHMSA Algorithm is able to reach feasible solutions by using $5\times {10}^5$ FES.

Assume that if the result x is a feasible one satisfying $(f\left(x\right)-f\left(x^{*}\right)\le$ 0.0001, then x is in a neighborhood (near-optimal) of the optimal point $x^{*}=x_{{}_g}$. Table 2, Table 3 and Table 4 indicate that the GHMSA Algorithm can get near-optimal points for six problems, { G01, G04, G06, G08, G12, G24,} by using only $5\times {10}^3$ FES, { G03, G11, G13, G14, G15, G16,G18} by using only $5\times {10}^4$ FES and { G07, G09, G13, G19} by using only $5\times {10}^5$ FES. However, the GHMSA Algorithm failed to satisfy $(f\left(x\right)-f\left(x^{*}\right)\le$ 0.0001, for two problems {G05, G10}. As suggested by [45],

Table 5. Number of FES to achieve the fixed accuracy level $((f\left(x\right)-f\left(x^{*}\right))\le 0.0001)$, success rate, feasible rate and success performance.

pr	Best	Median	Worst	Mean	s.d	f.r (%)	s.r (%)	s.p
G1	1964	2360	2748	2386.68	172.3774278	100	100	2386.68
G3	7681	11,558	13,545	11,566.82353	1167.105632	100	100	11,566.82353
G4	1906	4417	4924	4295.6	563.9451037	100	100	4295.6
G5	-	-	-	-	-	0	0	-
G6	3628	4455	5409	4388.851852	390.8762903	100	100	4388.851852
G7	34,773	210,502	500,559	259,738.33	224,164.15	100	100	259,738.33
G8	794	1108	1350	1109.9615	133.37526	100	100	1109.9615
G9	417,565	$4.33\times {10}^{05}$	495,232	444,552.9	31,764.59	100	100	444,552.9
G10	-	-	-	-	-	0	0	-
G11	6248	8146	9877	8233.92	917.8058	100	100	8233.92
G12	117	230	339	226.6	57.190209	100	100	226.6
G13	12,800	37,261	80,814	42,242.04	17,190.94051	100	100	42242.04
G14	28,366	54,687	71,293	52,486.30769	16,047.27821	100	100	52,486.30769
G15	9258	25,435	90,720	30,647.44	19,355.55703	100	100	30,647.44
G16	5758	9199	11,398	8970.76	1060.014463	100	100	8970.76
G18	10,300	36,198	85,882	42,434.56	20,906.48809	100	100	42,434.56
G19	73,800	193,000	499,000	247,000	187,852.1	100	100	247,000
G24	537	755.5	999	744.846154	103.587456	100	100	744.846154

presents the Best, Median, Worst, Mean, and s.d values of successful run, feasible rate, success rate, and success performance over 40 runs. Let us define the following:

Feasible run: A run through which at least one feasible solution is found in Max FES.

Successful run: A run during which the algorithm finds a feasible solution x satisfying $(f\left(x\right)-f\left(x^{*}\right)\le$ 0.0001.

Feasible rate (f.r) = (# of feasible runs)/ total runs.

$\mathbf{Successrate}(\mathbf{s}.\mathbf{r})=$ (# of successful runs) / total runs.

$\mathbf{Successperformance}(\mathbf{s}.\mathbf{p})=$ mean (FES for successful runs) × (# of total runs)/(# of successful runs).

Table 5 shows that the GHMSA Algorithm obtains a 100% feasible rate and success rate for all 18 problems with the exception of problems G05 and G10.

For achieving the success condition during the view of success performance in Table 5, the GHMSA Algorithm needs:

(1)

$117\le$ FES $\le 4.924\times {10}^3$ for 5 problems i.e., {G01, G04, G08, G12, G24}.

(2)

$3628\le$ FES $\le 1.4\times {10}^4$ for 4 problems i.e., {G03, G06, G11, G16}.

(3)

$9258\le$ FES ≤ 90,720 for 4 problems i.e., {G13, G14, G15, G18}.

(4)

34,773 ≤ FES ≤ 500,559 for 3 problems i.e., {G07, G09, G19}.

The GHMSA Algorithm failed to achieve the success condition for two problems, i.e., {G05, G10}. More information about the performance of the GHMSA Algorithm for solving these problems is given in Figure 2, Figure 3 and Figure 4. We have plotted the relationship between $lo{g}_{10}(f\left(x\right)-f\left(x^{*}\right))$ and FES for showing the convergence of the GHMSA at the median run over 40 independent runs. So the convergence graphs of these problems in Figure 2, Figure 3 and Figure 4 show that the error values decrease dramatically with increasing FES for all test problems.

5.2. Performance of GHMSA Algorithm Using Statistical Hypothesis Testing

In this section, we use statistical hypothesis testing to evaluate the efficiency of the GHMSA Algorithm versus the efficiency of the CB-ABC, the CCiALF, the NDE and the CAMDE Algorithms.

A statistical hypothesis is a surmise about a population parameter. This expectation might be true or false. The null hypothesis is denoted by $H_{{}_0}$, and it is a statistical hypothesis that announces that there is no difference between a parameter and a specific value or that there is no difference between two parameters. The alternative hypothesis is indicated by $H_{{}_a}$, and it is a statistical hypothesis that declares a specific difference between a parameter and a specific value or states that there is a difference between two parameters. Hypothesis testing is a form of inferential statistic which authorizes us to draw conclusions on a whole population based on a representative sample [51]. Parametric tests can provide trustworthy results with distributions that are skewed and non normal. Parametric analysis can produce reliable results even if the continuous data are non normally distributed. We just have to be sure that the sample size is greater than 30. A one sample t-test is one of the parametric tests that is used to compare the mean (Average) of a sample with a mean of the population. The important conditions for using the one-sample t-test are independence and normality (or sample size $>30$). In our study the sample size is 50, i.e., the number of runs is 50 randomly (from any starting point) run; this criterion is suggested by [45]. The significance level in this study is 95%, i.e., $\alpha =0.05$. Our hypotheses are formulated in the following:

$H_{{}_0}$: the mean (average) of the results of the GHMSA Algorithm and the mean (average) of the results of other algorithms are equal.

$H_{{}_a}$: the mean (average) of the results of the GHMSA Algorithm and the mean of the results of other algorithms are different.

The above hypotheses can be formulated in Equation (17).

$$\begin{array}{c}{H}_{{}_0}:M{e}_{GHMSA}=M{e}_{Algorith{m}_{{}_l}},\\ H_{{}_a}:M{e}_{GHMSA}\ne M{e}_{Algorith{m}_{{}_l}},\end{array}$$

(17)

where l denotes one of the algorithms, CB-ABC, CCiALF, NDE and CAMDE, and $Me$ denotes the average results of the algorithms.

In order to compare the performance of the GHMSA Algorithm with the CB-ABC, the CCiALF, the NDE and the CAMDE Algorithms, the t-test with a significance level of $\alpha =0.05$ is performed. To perform the t-test, the hypotheses in Equation (17) are considered.

Statistical processes are performed by using the SPSS Program. Rejecting or accepting $H_{{}_0}$ is based on the value of the p-value (Sig. (2-tailed)) according to Column 1 of

Table 6. How the null hypothesis is rejected (or accepted) and the decision is made.

p-Value	${\mathit{H}}_{{}_0}$	t	Decision
<$\alpha$	reject	<0	1
<$\alpha$	reject	>0	−1
>$\alpha$	accept	-	0

. While the performance of the algorithm based on the value of the t-test is in Column 3 of Table 6. So, Column 4 of Table 6 takes three values according to the probabilities in (18).

$$\mathrm{Decision}=\left\{\begin{array}{cc}1\hfill & \mathrm{then}M{e}_{GHMSA}<M{e}_{Algorith{m}_{{}_{a}l}},\hfill \\ -1\hfill & \mathrm{then}M{e}_{GHMSA}>M{e}_{Algorith{m}_{{}_l}},\hfill \\ 0\hfill & \mathrm{then}M{e}_{GHMSA}=M{e}_{Algorith{m}_{{}_l}}.\hfill \end{array}\right.$$

(18)

The results of the GHMSA are compared to the results of the CB-ABC, the CCiALF, the NDE and the CAMDE Algorithms. The statistical hypotheses in Equation (17) are tested by using the t-test.

Table 7. Comparison of results for test problems G01 to G08.

pr	s.t	CB-ABC	CCiALF	NDE	CAMDE	GHMSA
G1	b.s	−15	−15	−15	−15	−15
	mean	−15	−15	−15	−15	−15
	s.d	$5.03\times {10}^{-15}$	$2.39\times {10}^{-08}$	0	0	0
	decision	0	0	0	0	0
	FES	135,180	30,819	240,000	240,000	5773.84
G3	b.s	−1.0005	−1.000501	−1.0005001	−1.000500	−1.000009
	mean	−1.0005	−1.000501	−1.0005001	−1.000500	−1.000002
	s.d	$3.64\times {10}^{-07}$	$1.69\times {10}^{-08}$	0	$6.80\times {10}^{-16}$	$1.92\times {10}^{-06}$
	decision	‡	‡	‡	‡
	FES	90,090	87,860	240,000	240,000	62,546.38462
G4	b.s	−30,665.54	−30,665.539	−30,665.539	−30,665.53867	−30,665.53867
	mean	−30,665.54	−30,665.539	−30,665.539	−30,665.53867	−30,665.53867
	s.d	$8.72\times {10}^{-11}$	$9.80\times {10}^{-06}$	0	$3.71\times {10}^{-12}$	$3.49\times {10}^{-07}$
	decision	0	0	0	0
	FES	45,045	26,268	240,000	240,000	9671.32
G5	b.s	5126.50	5126.4967	5126.49671	5126.496710	5126.49833
	mean	5126.50	5126.497	5126.49671	5126.496710	5126.662712
	s.d	$1.07\times {10}^{-10}$	$9.17\times {10}^{-08}$	0	$2.78\times {10}^{-12}$	0.03442
	decision	−1	−1	−1	−1
	FES	135,180	156,248	240,000	240,000	33,917.7702
G6	b.s	−6961.81	−6961.814	−6961.813875	−6961.81388	−6961.813826
	mean	−6961.81	−6961.814	−6961.813875	−6961.81388	−6961.813811
	s.d	$1.82\times {10}^{-12}$	$5.19\times {10}^{-11}$	0	0	$9.20\times {10}^{-06}$
	decision	1	0	1	1
	FES	45,045	17,573	240,000	240,000	8921.518519
G7	b.s	24.3062	24.3062	24.306209	24.30621	24.30610911
	mean	24.3062	24.3062	24.306209	24.30621	24.30617
	s.d	$4.16\times {10}^{-07}$	$6.82\times {10}^{-07}$	$1.35\times {10}^{-14}$	$8.55\times {10}^{-15}$	$4.34\times {10}^{-05}$
	decision	1	1	1	1
	FES	135,180	8745	240,000	240,000	259,738.33
G8	b.s	−0.095825	−0.095825	−0.095825	−0.09583	−0.0958141
	mean	−0.095825	−0.095825	−0.095825	−0.09583	−0.0957819
	s.d	$2.87\times {10}^{-17}$	$1.07\times {10}^{-15}$	0	$1.42\times {10}^{-17}$	$2.58\times {10}^{-05}$
	decision	1	1	1	−1
	FES	8000	4812	240,000	240, 000	2394.577

The mark ‡ means that we do not use G03 to compare the result of the GHMSA with results of the four algorithms because the h(x*) = 0.0001, i.e., ῡ = 0.0001 in [45], but ῡ for the GHMSA is 4.05 × 10⁻⁰⁷, see Table 1, Table 2 and Table 8.

,

Table 8. Statistical results of “GHMSA” Algorithm for first set of test problems and four mechanical engineering problems.

pr	Best	Median	Worst	Mean	s.d	FES
G1	−15	−15	−15	−15	0	5773.84
G3	−1.000009	−1.000003	−1.000001	−1.000002	$1.91679\times {10}^{-06}$	62,546.38462
G4	−30,665.538672	−30,665.538672	−30,665.53867	−30,665.538672	$3.49\times {10}^{-07}$	9671.32
G5	5126.49833	5126.520053	5127.81491	5126.662712	0.03442	33,917.7702
G6	−6961.813826	−6961.813811	−6961.81377	−6961.813811	$9.19642\times {10}^{-06}$	8921.518519
G7	24.30610911	24.30618042	24.30625377	24.30617	$4.34\times {10}^{-05}$	259,738.33
G8	−0.0958141	−0.095824999	−0.09582499	−0.0957819	$2.58\times {10}^{-05}$	2394.577
G9	680.6301232	680.6301426	680.6301562	680.6301412	$1.52\times {10}^{-05}$	444,552.9
G10	7049.271862	7049.689888	7049.460323	7049.336552	$2.69\times {10}^{-02}$	290,146
G11	0.74999176	0.749996	0.75	0.7499961	0.0000001	8233.92
G12	−1	−1	−1	−1	0	1515
G13	0.053950002	0.053998358	0.054040318	0.053996327	$2.73\times {10}^{-05}$	53,754
G14	−47.76497953	−47.76493525	−47.76488874	−47.76494056	$2.51\times {10}^{-05}$	52,486.30769
G15	961.71502	961.71502	961.715107	961.7149837	$1.31\times {10}^{-04}$	38,609.24
G16	−1.905155259	−1.905155259	−1.905155259	−1.905155259	$2.06\times {10}^{-10}$	36,346.76
G18	−0.866025404	−0.865945746	−0.865926597	−0.865958115	$2.85\times {10}^{-05}$	42,434.56
G19	32.65549	32.65556	32.65568	32.6555744	$6.76\times {10}^{-05}$	247,295.25
G24	−5.508013272	−5.508013272	−5.508013272	−5.508013272	$1.09\times {10}^{-10}$	2460.038
Enp1	5885.332773	5885.332773	5885.332773	5885.332773	$2.2\times {10}^{-12}$	32,129
Enp2	0.012665233	0.012665268	0.012665243	0.012665334	$1.54\times {10}^{-09}$	9970
Enp3	1.724852306	1.724852306	1.724852306	1.724852306	$1.33\times {10}^{-16}$	24,270
Enp4	2994.471066	2994.471066	2994.471066	2994.471066	$4.27\times {10}^{-15}$	16,764

,

Table 9. Comparison of results for test problems G09 to G15.

pr	s.t	CB-ABC	CCiALF	NDE	CAMDE	GHMSA
G9	b.s	680.63	680.63	680.630057	680.63006	680.6301232
	mean	680.63	680.63	680.630057	680.63006	680.6301412
	s.d	$2.77\times {10}^{-09}$	$5.43\times {10}^{-08}$	0	$2.32\times {10}^{-13}$	$1.52\times {10}^{-05}$
	decision	0	0	−1	−1
	FES	45,045	12,801	240,000	240,000	444,552.9
G10	b.s	7049.25	7049.248	7049.24802	7049.24802	7049.271862
	mean	7049.25	7049.248	7049.24802	7049.24802	7049.336552
	s.d	$3.98\times {10}^{-05}$	$6.04\times {10}^{-07}$	$3.41\times {10}^{-09}$	$4.39\times {10}^{-12}$	$2.69\times {10}^{-02}$
	decision	−1	−1	−1	−1
	FES	135,180	2858	240,000	240,000	240,146
G11	b.s	0.7499	0.749896	0.749999	0.749900	0.74999176
	mean	0.7499	0.749898	0.749999	0.749900	0.7499961
	s.d	$1.29\times {10}^{-10}$	$2.05\times {10}^{-16}$	0	$1.13\times {10}^{-16}$	0.0000001
	decision	−1	−1	1	−1
	FES	90,090	168,448	240,000	240,000	8233.92
G12	b.s	−1	−1	−1	−1	−1
	mean	−1	−1	−1	−1	−1
	s.d	0	$7.76\times {10}^{-11}$	0	0	0
	decision	0	0	0	0
	FES	13,500	17,892	240,000	240,000	1515
G13	b.s	0.053942	0.053942	0.0539415	0.05394	0.053950002
	mean	0.06677	0.053943	0.0539415	0.05394	0.053996327
	s.d	$6.91\times {10}^{-02}$	$4.03\times {10}^{-06}$	0	$2.32\times {10}^{-17}$	$2.73\times {10}^{-05}$
	decision	1	−1	−1	−1
	FES	198,270	19,883	240,000	240,000	53,754
G14	b.s	−47.7649	−47.764900	−47.7648885	−47.764890	−47.76497953
	mean	−47.7649	−47.764900	−47.7648885	−47.764890	−47.76494056
	s.d	$1.02\times {10}^{-05}$	$4.04\times {10}^{-08}$	$5.14\times {10}^{-15}$	$2.21\times {10}^{-14}$	$2.51\times {10}^{-05}$
	decision	1	1	1	1
	FES	239,715	152,697	240,000	240,000	52,486.30769
G15	b.s	961.715	961.715	961.7150223	961.715020	961.71502
	mean	961.715	961.715	961.7150223	961.715020	961.7149837
	s.d	$2.81\times {10}^{-11}$	$1.86\times {10}^{-08}$	0	$5.80\times {10}^{-13}$	$1.31\times {10}^{-04}$
	decision	0	0	1	1
	FES	135, 180	77, 910	240, 000	240, 000	38, 609.24

and

Table 10. Comparison of results for test problems G16, G18, G19, G24 and Enp1-Enp4.

pr	s.t	CB-ABC	CCiALF	NDE	CAMDE	GHMSA
G16	b.s	−1.905	−1.905155	−1.90515525	−1.905160	−1.905155259
	mean	−1.905	−1.905155	−1.90515525	−1.905160	−1.905155259
	s.d	$7.90\times {10}^{-11}$	$9.77\times {10}^{-09}$	0	$4.53\times {10}^{-16}$	$2.06\times {10}^{-10}$
	decision	1	1	1	0
	FES	45,045	196,196	240,000	240,000	36,346.76
G18	b.s	−0.866025	−0.866026	−0.8660254	−0.86603	−0.866025404
	mean	−0.866025	−0.866026	−0.8660254	−0.86603	−0.865958115
	s.d	$1.72\times {10}^{-08}$	$3.58\times {10}^{-07}$	0	$4.53\times {10}^{-17}$	$2.85\times {10}^{-05}$
	decision	−1	−1	−1	−1
	FES	135,180	8742	240,000	240,000	42,434.56
G19	b.s	32.6556	32.655610	32.65559377	32.655590	32.65549
	mean	32.6556	32.660770	32.65562603	32.655590	32.6555744
	s.d	$1.88\times {10}^{-05}$	$2.35\times {10}^{-04}$	$3.73\times {10}^{-05}$	$7.11\times {10}^{-15}$	$6.76\times {10}^{-05}$
	decision	0	1	1	0
	FES	198,270	240,000	240,000	240,000	247,295.25
G24	b.s	−5.508	−5.508013	−5.50801327	−5.508010	−5.508013272
	mean	−5.508	−5.508013	−5.50801327	−5.508010	−5.508013272
	s.d	$7.15\times {10}^{-15}$	$1.30\times {10}^{-08}$	0	$9.06\times {10}^{-16}$	$1.09\times {10}^{-10}$
	decision	1	1	1	1
	FES	27,000	6450	240,000	240,000	2460.038
Enp1	b.s	6059.71	6059.714335	6059.714335	6059.714335	5885.332773
	mean	6126.62	6059.714335	6059.714335	6059.714335	5885.332773
	s.d	$1.14\times {10}^{02}$	$1.01\times {10}^{-11}$	$4.56\times {10}^{-07}$	$1.22\times {10}^{-06}$	$2.2\times {10}^{-12}$
	Decision	1	1	1	1
	FES	15,000	12,000	20,000	10,000	32,1290
Enp2	b.s	0.012665	0.012665233	0.012665232	0.012665233	0.01266523
	mean	0.012671	0.012665251	0.012668899	0.012666981	0.01266533
	s.d	$1.42\times {10}^{-05}$	$9.87\times {10}^{-08}$	$5.38\times {10}^{-06}$	$3.65\times {10}^{-06}$	$1.54\times {10}^{-09}$
	Decision	1	0	1	1
	FES	15,000	5000	24,000	10,000	9970
Enp3	b.s	1.724852	1.724852	1.724852309	1.724852	1.724852
	mean	1.724852	1.724852	1.724852309	1.724852	1.724852
	s.d	0	$5.11\times {10}^{-07}$	$3.73\times {10}^{-12}$	$2.32\times {10}^{-13}$	$1.33\times {10}^{-16}$
	Decision	0	0	1	0
	FES	15,000	10,000	8000	10,000	24,270
Enp4	b.s	2994.471066	2994.471066	2994.471066	2994.471066	2994.471065
	mean	2994.471066	2994.4710660	2994.47106610	2994.471066	2994.471065
	s.d	$2.48\times {10}^{-07}$	$2.31\times {10}^{-12}$	$4.17\times {10}^{-12}$	$2.20\times {10}^{-12}$	$4.27\times {10}^{-15}$
	Decision	0	0	1	0
	FES	15,000	10,000	18,000	10,000	16,764

present these results.

The results of the GHMSA are compared versus the four meta-heuristic algorithms in the literature. The results of statistical tests are presented in Table 7, Table 9 and Table 10. In Table 7, Column 1 presents the abbreviation of the test problems denoted by pr. Column 2 presents the results of the s.t which include {b.s, mean, s.d, Decision }, where Decision denotes wins, losses and draws of the GHMSA compared with the other algorithms. Columns 3–7 give the results of the five algorithms. Table 9 and Table 10 are similar to Table 7.

After executing the pairwise t-test for all algorithms, if the GHMSA Algorithm is superior, inferior or equal to the compared algorithm denoted by $algorith{m}_l$, then the decision is set to 1, –1 and 0 respectively, as we have shown in Table 6. The left of Figure 5 summarizes the results that are presented in Table 7, Table 8, Table 9 and Table 10 regarding the decision. The left of Figure 5 shows that the GHMSA Algorithm was superior at {7, 6, 9, 5} problems, equal at {6, 6, 3, 5} problems and inferior at {4, 5, 5, 7} problems compared to the CB-ABC, the CALF, the NDE and the CAMDE Algorithms, respectively. However, the GHMSA is inferior at seven problems compared to the CAMDE, but the GHMSA needs 1,590,905 as a total FES versus the CAMDE needing 4,320,000, as shown in Figure 6. To gain the success condition from the point of view of successful execution, the GHMSA needs less than $5\times {10}^3$ FES for five problems, i.e., {G01, G04, G08, G12, G24} versus the CAMDE needing at least $5\times {10}^3$ FES for two problems, i.e., {G08, G12}. We can say that the percentage of superior, equal and inferior of the GHMSA are 40%, 30%, 30% respectively.

For the four engineering problems, we give a brief description. The pressure vessel problem is a practical problem that is often used as a benchmark problem for testing optimization algorithms [52]. The left of Figure 7 shows the structure of this issue, where a cylindrical pressure vessel is capped at both ends by hemispherical heads. The aim of the problem is to find the minimum total cost of fabrication, including costs from a combination of welding, material and forming. The thickness of the cylindrical skin,$x_{{}_1}\left(Ts\right)$, thickness of the spherical head, $x_{{}_2}\left(T_h\right)$, the inner radius, $x_{{}_3}\left(R\right)$, and the length of the cylindrical segment of the vessel, $x_{{}_4}\left(L\right)$, were included as the optimization design variables of the problem. The GHMSA Algorithm obtains these results: $x_{GHMSA}=$ {0.778168641375105, 0.384649162627902, 40.3196187240987, 200}, i.e., $f\left(x_{GHMSA}\right)=$ 5885.332774, c = {0, –$3.8858\times {10}^{-16}$, $1.1642\times {10}^{-97}$, −40}, i.e., $\overline{v}=$ 0; the left of Figure 8 shows a convergence graph of the GHMSA to the best solution for this problem.

Another well-known engineering optimization task is the design of a tension (compression spring) for a minimum weight. This problem has been studied by several authors. For example, [52]. The right of Figure 7 shows a tension (compression spring) with three design variables. It needs to minimize the weight of a tension (compression string) subject to constraints on minimum deflection, shear stress, surge frequency, limits on outside diameter and on design variables. The design variables are the wire diameter, $d\left(x_{{}_1}\right)$, the mean coil diameter, $D\left(x_{{}_2}\right)$, and the number of active coils, $P\left(x_{{}_3}\right)$. The GHMSA obtains these results: $x_{GHMSA}=$ {0.0516890825110813, 0.356718255308635, 11.2889355307237}, i.e., $f\left(x_{GHMSA}\right)=$ 0.01266523279, $c=$ {$-1.55\times {10}^{-10}$, $4.44\times {10}^{-16}$, –4.05379, –0.72773}, i.e., $\overline{v}=$$1.11\times {10}^{-16}$. The convergence graph for Engp2 is presented on the right of Figure 8. The welded beam design optimization problem has been solved by many researchers [52]. The left of Figure 9 shows the welded beam structure which consists of a beam A and the weld required to hold it to member B. The goal of this problem is to minimize the overall cost of fabrication, subject to some constraints. This problem has four design variables—$x_1$, $x_2$, $x_3$ and $x_4$—with constraints of shear stress $\tau$, bending stress in the beam $\sigma$, buckling load on the bar $P_c$, and end deflection on the beam $\delta$. The GHMSA obtains these results: $x_{GHMSA}=\{0.205729642092758,3.4704886133955,9.03662391715327,0.205729639752274\}$, i.e., $f\left(x_{GHMSA}\right)=1.7248523060$, $c=$ {–$9.03\times {10}^{-08}$, –$4.02\times {10}^{-05}$, $2.34\times {10}^{-09}$, –3.43298, –0.08073, –0.23554, –$8.73\times {10}^{-09}$ }, i.e., $\overline{v}$ = $3.3429\times {10}^{-10}$. The convergence graph for Engp3 is presented by the left of Figure 10.

The speed reducer design problem is one of the benchmark structural engineering problems [52]. It has seven design variables as described in the right of Figure 9, with the face width $x_{{}_1}$, module of teeth $x_{{}_2}$, number of teeth on pinion $x_{{}_3}$, length of the first shaft between bearings $x_{{}_4}$, length of the second shaft between bearings $x_{{}_5}$, diameter of the first shaft $x_{{}_6}$, and diameter of the first shaft $x_{{}_7}$. The aim of this problem is to minimize the total weight of the decelerator. The GHMSA obtains these results: $x_{GHMSA}=\{3.499999999,0.7,17,7.3,7.715319913,3.350214666,5.286654465\}$, i.e, $f\left(x_{GHMSA}\right)=$2994.471066, $c=\{-0.073915,-0.198,-0.49917,-0.90464,8.6365\times {10}^{-11}$, $-1.0931\times {10}^{-11},-0.7025,2.86\times {10}^{-10},-0.58333,-0.051326,-1.{944210}^{-10}\}$, i.e, $\overline{v}=2.6\times {10}^{-11}$. The convergence graph for Engp4 is presented by the right of Figure 10. The four engineering problems are used to compare the performance of the GHMSA against the CB-ABC, the CCiALF, the NDE and the CAMDE Algorithms. Statistical hypotheses in Equation (17) are used to compare the mean of the GHMSA with means of the CB-ABC, the CCiALF, the NDE and the CAMDE Algorithms. Rows 22–41 of Table 10 present the statistical comparisons of the GHMSA versus the four Algorithms for engineering problems Enp1 to Enp4. The right of Figure 5 gives the number of “wins-draws-losses” of the GHMSA compared with the CB-ABC, the CCiALF, the NDE and the CAMDE for Enp1 to Enp4. Figure 11 shows the convergence graph of standard deviation for problems Enp1, Enp2, Enp3 and Enp4 for the five algorithms. The relation between the four engineering problems {Enp1, Enp2, Enp3 and Enp4} and their values for $lo{g}_{10}(s.d)$ are plotted. From the right of Figure 5 and Figure 11, it can be said that the performance of the GHMSA algorithm is better than the other algorithms for problems Enp1 to Enp4, for the following reasons:

(1) The GHMSA obtains a minimum value of objective function (5885.332774) for engineering problem Enp1 (pressure vessel), the point minimum $x^{*}$ is feasible; many of the algorithms obtained a value of objective function equal to or greater than 6059.71. see for example [48,49,50,52,53,54,55,56,57,58].

In addition to that, if $10\le x_{{}_4}\left(L\right)<\infty$, then $f\left(x^{*}\right)=5804.37621675626$, otherwise if $10\le x_{{}_4}\left(L\right)<208$, then $f\left(x^{*}\right)=5866.99226593889$, where L is shown in the left of Figure 8.

(2) The right of Figure 5 shows that the GHMSA Algorithm does not fall at any problem versus the other algorithms.

(3) The GHMSA is superior at {2, 1, 4, 2} problems versus the CB-ABC Algorithm, the CCiALF Algorithm, the NDE and the CAMDE Algorithm, respectively.

(4) The GHMSA is equal at {2, 3, 0, 2} problems versus the CB-ABC Algorithm, the CCiALF Algorithm, the NDE and the CAMDE Algorithm, respectively.

(5) Figure 11 shows that the GHMSA Algorithm converges to zero for the standard deviation (s.d). See the green color.

6. Conclusions and Future Work

The unconstrained nonlinear optimization algorithms have been guided to find the global minimizer of the constrained optimization problem. A result, Algorithm “GHMSA”, has been proposed for finding the global minimizer of the non-linear constrained optimization problem. Algorithm “GHMSA” contains a new technique that is applied to convert the constrained optimization problem into the unconstrained optimization problem. The results of the algorithm demonstrate that the proposed penalty function is a good technique to make the unconstrained algorithm able to deal with the constrained optimization problem. The interior-point algorithm keeps the candidate solutions inside the domain search. The results of some nonlinear constrained optimization problems and four non-linear engineering optimization problems show that the GHMSA algorithm has superiority over the other four algorithms in some test problems. For the future work, the proposed algorithm can be enhanced and modified to solve the multi-objective function, and the convergence analysis of the modified simulated annealing algorithm will be performed.

Moreover, it will be considered in future work to propose a new free derivative to approximate the gradient vector that will be combined (hybridized) with a new simulated annealing algorithm to solve unconstrained optimization, constrained, or multi-objective optimization problems. Convergence analysis of the GMLSA and GHMAS algorithms will be considered in future work.