Trust-Region Based Penalty Barrier Algorithm for Constrained Nonlinear Programming Problems: An Application of Design of Minimum Cost Canal Sections

Abstract

In this paper, a penalty method is used together with a barrier method to transform a constrained nonlinear programming problem into an unconstrained nonlinear programming problem. In the proposed approach, Newton’s method is applied to the barrier Karush–Kuhn–Tucker conditions. To ensure global convergence from any starting point, a trust-region globalization strategy is used. A global convergence theory of the penalty–barrier trust-region (PBTR) algorithm is studied under four standard assumptions. The PBTR has new features; it is simpler, has rapid convergerce, and is easy to implement. Numerical simulation was performed on some benchmark problems. The proposed algorithm was implemented to find the optimal design of a canal section for minimum water loss for a triangle cross-section application. The results are promising when compared with well-known algorithms.

1. Introduction

Solving linear programming problems via the simplex method has never had any true competition until Karmarkar presented a new polynomial time method [1,2]. The polynomial complexity of Karmarkar’s algorithm has an advantage in comparison with the exponential complexity of the simplex procedure [3,4]. Schrijver [5,6] presented an improvement of Karmarkar’s method. Malek et al. [7] investigated an improved interior-point-based technique dealing with linear programming problems more efficiently. Tse et al. [8] presented an extension of Karmarkar’s algorithm for dealing with quadratic programming. The success of the interior-point algorithm in handling linear programming problems has encouraged researchers to implement the linearization methods to handle nonlinear programming problems [9,10]. Kebbiche et al. [11] investigated a projective interior-point algorithm for solving general convex nonlinear problems. Herskovits et al. [12] presented an interior-point-based approach for solving bilevel programming problems with convex lower-level problems. Zhang et al. [13] presented an interior-point method-based program combined with the homogenization theory on micromechanics to analyze shakedown behavior at the macro-scale. Schmidt [14] presented an interior-point method for nonlinear optimization problems with locatable and separable no-smoothness. Sakineh [15] presented novel interior-point algorithms for solving nonlinear convex optimization problems. Scheunemann et al. [16] presented an algorithm for rat- independent small-strain crystal plasticity based on the infeasible primal–dual interior-point method

In this research, a penalty method is introduced with a barrier method to the transform constrained nonlinear programming (CNP) problem into an unconstrained nonlinear problem. Newton’s method is applied to the barrier Karush–Kuhn–Tucker conditions. Newton’s method has the advantage that it converges quadratically to a stationary point under reasonable assumptions, but it has the disadvantage that the starting point must be sufficiently closed to the stationary point to guarantee convergence. To overcome this disadvantage and to guarantee convergence from any starting point, we use the trust-region strategy. The trust-region strategy can induce strongly global convergence, which is a very important method for solving a nonlinear programming problem and is more robust when it deals with rounding errors. It does not require the objective function of the model to be convex. Additionally, it does not require the Hessian of the objective function to be positive definite. For more details, see [17,18,19,20,21,22,23,24,25,26,27].

A global convergence theory of the PBTR algorithm is investigated. The PBTR algorithm always outputs a new point at each iteration, in order to improve the performance of the lgorithm, one need to solve the subproblem many times This shows that the PBTR algorithm is globally convergent under required assumptions.

A numerical simulation of the PBTR algorithm was performed on benchmark problems. Additionally, the PBTR algorithm was implemented to find the optimal design of a canal section for minimum water loss for a triangle cross-section application. The numerical results to benchmark problems are promising when compared with well-known algorithms.

Our balance idea mainly differs from these algorithms in the method of computing trial steps, the method of ensuring that ${\tilde{x}}_k$ and $y_k$ are strictly positive, the method to update the penalty parameter, the method to update the Hessian matrix, and the analysis of the global convergence. The mechanism of our method seems simpler than these methods. Moreover, numerical experiments display that the suggested PBTR algorithm performs effectively and efficiently in pursuance.

Notations: We use $\parallel .\parallel$ to denote the Euclidean norm. The $i-th$ component of any vector such as $\tilde{x}$ is written as ${\tilde{x}}^{\left(i\right)}$. Subscript $k$ refers to iteration indices. For example, $f_k\equiv f\left({\tilde{x}}_k\right)$, $G_k\equiv G\left({\tilde{x}}_k\right)$, $\nabla f_k\equiv \nabla f\left({\tilde{x}}_k\right)$, and $\nabla G_k\equiv \nabla G\left({\tilde{x}}_k\right)$, and so on to symbolize the value of the function at a particular point. We introduce the main framework of the PBTR algorithm to solve COP in the following section.

This paper is structured as follows. In Section 2, we clarify the outline of the penalty–barrier Newton’s method to transform the CNP problem into an unconstrained problem. The outline of the PBTR algorithm is investigated in Section 3. Section 4 presents a global convergence analysis for the PBTR algorithm. Section 5 addresses the simulation experiments and canal application. Finally, our observations and future work are discussed in Section 6.

2. Penalty-Barrier Newton’s Method

In this paper, we consider the following CNP problem

$$\begin{array}{ll}minimize\hfill & f\left(x\right)\hfill \\ subject to\hfill & g_l\left(x\right)=0,\hfill \\ \hfill & g_i\left(x\right)\le 0,\hfill \\ \hfill & x\ge 0,\hfill \end{array}$$

(1)

where $f:{\Re }^n\to \Re$, $g_l:{\Re }^n\to {\Re }^m$, and $g_i:{\Re }^n\to {\Re }^p$ are at least twice continuously differentiable functions.

By adding slack variables to the inequality constraints $g_i\left(x\right)$, the CNP problem can be reformulated into the following problem

$$\begin{array}{ll}minimize\hfill & f\left(x\right)\hfill \\ subject to\hfill & g_l\left(x\right)=0,\hfill \\ \hfill & g_i\left(x\right)+s=0,\hfill \\ \hfill & x\ge 0,s\ge 0,\hfill \end{array}$$

where $s\in {\Re }^p$ is a slack variable. To simplify, the above problem can be written in the form:

$$\begin{array}{ll}minimize\hfill & f\left(\tilde{x}\right)\hfill \\ subject to\hfill & G\left(\tilde{x}\right)=0,\hfill \\ \hfill & \tilde{x}\ge 0,\hfill \end{array}$$

where $\tilde{x}={(x,s)}^T\in {\Re }^{n+p}$ and $G\left(\tilde{x}\right)={(g_l\left(x\right),g_i\left(x\right)+s)}^T\in {\Re }^{m+p}$.

By using the penalty method, we can write the above problem as follows:

$$\begin{array}{ll}minimize\hfill & f\left(\tilde{x}\right)+\frac{\nu }{2}\parallel G\left(\tilde{x}\right){\parallel }^2\hfill \\ subject to\hfill & \tilde{x}\ge 0,\hfill \end{array}$$

(2)

where $\nu >0$ is a penalty parameter. For more details, see [28].

Associated with the above problem, the Lagrangian function is given by

$$L\left(\tilde{x},y;\nu \right)=f\left(\tilde{x}\right)-y^T\tilde{x}+\frac{\nu }{2}\parallel G\left(\tilde{x}\right){\parallel }^2,$$

(3)

where the vector $0\le y\in {\Re }^{n+p}$ is the Lagrange multiplier vector associated with the inequality constraint $\tilde{x}$.

Motivated by the barrier method, which is discussed in [29,30,31,32,33], Problem (2) can be written as follows

$$\begin{array}{ll}& \mathit{minimize} f(\tilde{x})-\omega {\sum }_{i=1}^{n+p}\ln ({\tilde{x}}^{(i)})+\frac{v}{2}\parallel G(\tilde{x}){\parallel }^2,\\ & \tilde{x}>0\end{array}$$

(4)

for decreasing sequence of barrier parameters $\omega$ converging to zero, see [34]. Let

$${\psi }^{\omega }(\tilde{x})=f(\tilde{x})-\omega {\sum }_{i=1}^{n+p}\ln ({\tilde{x}}^{(i)})$$

(5)

The first-order necessary conditions for a point ${\tilde{x}}_{*}$ to be a local minimizer of Problem (4) are given by $\nabla f\left({\tilde{x}}_{*}\right)-\omega {\tilde{X}}_{*}{}^{-1}e+\nu \nabla G\left({\tilde{x}}_{*}\right)G\left({\tilde{x}}_{*}\right)=0,$ and ${\tilde{x}}_{*}>0,$ where $\tilde{X}$ is the diagonal matrix whose diagonal entries are $\left({\tilde{x}}_1,\dots ,{\tilde{x}}_{n+p}\right)$. Let $y\in {\Re }^{n+p}$ be an auxiliary variable that is equal to $\omega {\tilde{X}}^{-1}e$. Then, the above conditions can be written as follows

$$\nabla f\left({\tilde{x}}_{*}\right)-y_{*}+\nu \nabla G\left({\tilde{x}}_{*}\right)G\left({\tilde{x}}_{*}\right)=0,$$

(6)

$${\tilde{X}}_{*}{y}_{*}-\omega e=0,$$

(7)

and

$${\tilde{x}}_{*}>0,\hspace{1em}{y}_{*}>0.$$

The conditions (6) and (7) are called the barrier Karush–Kuhn–Tucker (KKT) conditions. For more details, see [34]. Applying Newton’s method to the above nonlinear system, then we have

$$\left(\begin{array}{cc}H+\nu \hspace{0.17em}\nabla G(\tilde{x})\nabla G{(\tilde{x})}^T& -I\\ Y& \tilde{X}\end{array}\right)\left(\begin{array}{c}{d}_{\tilde{x}}\\ d_y\end{array}\right)=-\left(\begin{array}{c}\nabla f(\tilde{x})-y+\nu \hspace{0.17em}\nabla G(\tilde{x})G(\tilde{x})\\ \tilde{X}-\omega e\end{array}\right)$$

(8)

where $H$ is the Hessian matrix of the barrier function (5) or an approximation to it and $Y$ is a diagonal matrix whose diagonal entries are $\left(y_1,\dots ,y_{n+p}\right)$. Eliminating $d_y$ by using

$$dy=-y+\omega {\tilde{X}}^{-1}e-{\tilde{X}}^{-1}Y{d}_{\tilde{x}},$$

(9)

We have the following system

$$(B+\nu \nabla G\left(\tilde{x}\right)\nabla G\left(\tilde{x}{)}^T\right)d_{\tilde{x}}=-\left(\nabla {\psi }^{\omega }\left(\tilde{x}\right)+\nu \nabla G\left(\tilde{x}\right)G\left(\tilde{x}\right)\right),$$

(10)

where $B=H+{\tilde{X}}^{-1}Y$ and $\nabla {\psi }^{\omega }\left(\tilde{x}\right)=\nabla f\left(\tilde{x}\right)-y$.

It is easy to see that the step generated by (10) coincides with the solution of the following quadratic programming subproblem

$$minimize\hspace{1em}q\left(d\right)={(\nabla {\psi }^{\omega }\left(\tilde{x}\right)+\nu \nabla G\left(\tilde{x}\right)G\left(\tilde{x}\right))}^{T}d+\frac{1}{2}{d}^{T}Ad,$$

(11)

where $A=B+\nu \nabla G\left(\tilde{x}\right)\nabla G{(\tilde{x})}^T$ and to simplify, we use $d$ instead of $d_{\tilde{x}}$.

To guarantee convergence from any starting point, we use the trust-region strategy, which is clarified in the following section.

3. Outline of PBTR Algorithm

In this section, we offer the outline for solving the following trust-region sub-problem, which is associated with Problem (11)

$$\begin{array}{l}minimize\hspace{1em}{q}_k\left(d\right)={(\nabla {\psi }^{\omega }\left(\tilde{x}\right)+{\nu }_k\nabla G\left({\tilde{x}}_k\right)G\left({\tilde{x}}_k\right))}^{T}d+\frac{1}{2}{d}^T{A}_{k}d\hfill \\ subject to\hspace{1em}\parallel d\parallel \le {\delta }_k,\hfill \end{array}$$

(12)

where ${\delta }_k>0$ is the radius of the trust-region. To evaluate the trial step $d_k$, any approximation to the solution of the above Subproblem (12) can be used as long as a fraction of the Cauchy decrease condition holds. The fraction of the Cauchy decrease condition is the condition that a fraction of the predicted decrease obtained by the Cauchy step is less than or equal to the predicted decrease obtained by $d_k$. Therefore, a conjugate gradient method [35] is used to solve the Subproblem (12). In the following Algorithm 1, the main steps to solve Subproblem (12) are offered.

Algorithm 1 The algorithm of the conjugate gradient method

Step 1. Set $0=d_0\in {\Re }^{n+p}$, $r_0=-\left(\nabla f_k-y_k+{\nu }_k\nabla G_k{G}_k\right)$, and $v_0=r_0$. Step 2. For $j=0,\dots ,\left(n+p\right)$ do     Evaluate $A_k=B_k+{\nu }_k\nabla G_k\nabla G_k^T$,     $c_j=\frac{r_j^T{r}_j}{v_j^T{A}_k{v}_j}$, and     ${\gamma }_j$ such that $\parallel d_j+{\gamma }_j{v}_j\parallel ={\delta }_k$.     If $v_j^T{A}_k{v}_j\le 0$, then set $d_k=d_j+{\gamma }_j{v}_j$ and Stop.     Else, set $d_{j+1}=d_j+c_j{v}_j$ and     $r_{j+1}=r_j-c_j{A}_k{v}_j$.     If $\frac{r_{j+1}}{r_0}\le {\epsilon }_0$, set $d_k=d_{j+1}$ and Stop.     Evaluate ${\overline{q}}_j=\frac{r_{j+1}^T{r}_{j+1}}{r_j^T{r}_j}$ and the new direction is $v_{j+1}=r_{j+1}+{\overline{q}}_j{v}_j$.

Once the step $d_k$ is computed, we set ${\tilde{x}}_{k+1}={\tilde{x}}_k+d_k$. To ensure that ${\tilde{x}}_{k+1}>0$, we need to compute the damping parameter ${\phi }_k$ at every iteration $k$. The damping parameter ${\phi }_k$ is computed as follows:

$$\phi =\min \left\{\underset{i}{\min }\left\{h_k^{\left(i\right)}\right\},1\right\},$$

(13)

where

$$h_k^{\left(i\right)}=\left\{\begin{array}{ll}\frac{-{\tilde{x}}_k^{\left(i\right)}}{d_k^{\left(i\right)}},\hfill & if d_k^{\left(i\right)} < 0 \hfill \\ 1\hfill & otherwise.\hfill \end{array}\right.$$

Another damping parameter ${\sigma }_k$ in the step may be needed to satisfy ${\tilde{x}}_{k+1}>0$, where ${\sigma }_k$ is chosen as follows. If $\left({\tilde{x}}_k+{\phi }_k{d}_k\right)>0$, we set ${\sigma }_k=1$. Otherwise, we set ${\tilde{x}}_{k+1}={\tilde{x}}_k+{\sigma }_k{\phi }_k{d}_k$, such that ${\sigma }_k\in \left[1-\theta \parallel d_k\parallel ,1\right]$ and $0<\theta$ is a pre-specified fixed constant. It is easy to see that $1-{\sigma }_k=O\left(\parallel d_k\parallel \right)$. To simplify, we assume ${\mu }_k={\sigma }_k{\phi }_k$.

To test the scaled step ${\mu }_k{d}_k$ to decide if it will be accepted or not, we need the following merit function

$${\mathsf{\Phi }}^{\omega }\left({\tilde{x}}_k;{\nu }_k\right)={\psi }^{\omega }\left({\tilde{x}}_k\right)+\frac{{\nu }_k}{2}\parallel G\left({\tilde{x}}_k\right){\parallel }^2.$$

(14)

Additionally, we need to define an actual reduction $Are{d}_k$ and a predicted reduction $Pre{d}_k$.

The actual reduction is defined as follows

$$Are{d}_k={\mathsf{\Phi }}^{\omega }\left({\tilde{x}}_k;{\nu }_k\right)-{\mathsf{\Phi }}^{\omega }\left({\tilde{x}}_k+{\mu }_k{d}_k;{\nu }_k\right),$$

and it can be written as follows,

$$Are{d}_k={\psi }^{\omega }\left({\tilde{x}}_k\right)-{\psi }^{\omega }\left({\tilde{x}}_{k+1}\right)+\frac{{\nu }_k}{2}\left[\parallel G_k{\parallel }^2-\parallel G_{k+1}{\parallel }^2\right].$$

(15)

The predicted reduction is given by

$$Pre{d}_k=-\nabla {\psi }^{\omega }{({\tilde{x}}_k)}^T{\mu }_k{d}_k-\frac{1}{2}{\mu }_k^2{d}_k^T{B}_k{d}_k+\frac{{\nu }_k}{2}\left[\parallel G_k{\parallel }^2-\parallel G_k+\nabla G_k^T{\mu }_k{d}_k{\parallel }^2\right],$$

(16)

and it can be written as follows,

$$Pre{d}_k=q_k\left(0\right)-q_k\left({\mu }_k{d}_k\right),$$

(17)

where $q\left(d\right)$ is defined in (11).

Our way for testing the step $d_k$ and updating the radius ${\delta }_k$ is obviously in the following Algorithm 2.

Algorithm 2 Testing the step $d_k$ and updating ${\delta }_k$

Choose $0<{\beta }_1<{\beta }_2<1$, $0<{\alpha }_1<1<{\alpha }_2$, and ${\delta }_{min}\le {\delta }_0\le {\delta }_{max}$.     While $\frac{Are{d}_k}{Pre{d}_k}\in \left(0,{\beta }_1\right)$ or $Pre{d}_k\le 0$.     Put ${\delta }_k={\alpha }_1\parallel d_k\parallel$ and return to evaluate a new $d_k$ and end while.     If $\frac{Are{d}_k}{Pre{d}_k}\in \left[{\beta }_1,{\beta }_2\right)$. Put ${\tilde{x}}_{k+1}={\tilde{x}}_k+{\mu }_k{d}_k$ and ${\delta }_{k+1}=\max \left({\delta }_k,{\delta }_{min}\right)$ and end if.     If $\frac{Are{d}_k}{Pre{d}_k}\in \left[{\beta }_2,1\right]$. Put ${\tilde{x}}_{k+1}={\tilde{x}}_k+{\mu }_k{d}_k$ and ${\delta }_{k+1}=\min \left\{{\delta }_{max},\max \left\{{\delta }_{min},{\alpha }_2{\delta }_k\right\}\right\}$     End if.

To update the positive penalty parameter ${\nu }_k$, we use the following Algorithm 3.

Algorithm 3 How to update ${\nu }_k$

Set ${\nu }_0=1$. Evaluate $Pre{d}_k$, which is given by (5). If

$$\hspace{1em}Pre{d}_k\ge \parallel \nabla G_k{G}_k\parallel \min \left\{\parallel \nabla G_k{G}_k\parallel ,{\delta }_k\right\}.$$ (18)

Set ${\nu }_{k+1}={\nu }_k$. Else, set ${\nu }_{k+1}=2{\nu }_k$. End if For more details, see [36].

Finally, if the conditions $\parallel \nabla {\psi }_k^{\omega }\parallel +\parallel X_k{Y}_{k}e-{\omega }_{k}e\parallel +\parallel \nabla G_k{G}_k\parallel \le ϵ$ or $\parallel d_k\parallel \le {\epsilon }_1$ are held for some $0<ϵ$ and $0<{\epsilon }_1$, then the algorithm is stopped.

The major steps of the PBTR algorithm are offered in the following Algorithm 4.

Algorithm 4 PBTR algorithm

Step 0. Given $0<x_0$. Evaluate $y_0$. Set ${\omega }_0=0.1$ and ${\nu }_0=1$.          Pick out $ϵ$, ${\epsilon }_1$, ${\alpha }_1$, ${\alpha }_2$, ${\beta }_1$, and ${\beta }_2$ such that $0<ϵ$, $0<{\epsilon }_1$ $0<{\alpha }_1<1<{\alpha }_2$, and          $0<{\beta }_1<{\beta }_2<1$.          Pick out ${\delta }_0$, ${\delta }_{min}$, and ${\delta }_{max}$ such that ${\delta }_{min}\le {\delta }_0\le {\delta }_{max}$. Set $k=0$. Step 1. If $\parallel \nabla {\psi }_k^{\omega }\parallel +\parallel X_k{Y}_{k}e-{\omega }_{k}e\parallel +\parallel \nabla G_k{G}_k\parallel \le ϵ$, then stop. Step 2. To compute the step $d_k$, use Algorithm 1. Step 3. If $\parallel d_k\parallel \le {\epsilon }_1$, then stop. Step 4. (a) Evaluate the damping parameter ${\phi }_k$ using (13).           (b) Put ${\tilde{x}}_{k+1}={\tilde{x}}_k+{\phi }_k{d}_k$.           (c) If ${\tilde{x}}_{k+1}>0$, then go to Step 5.           Else, put ${\tilde{x}}_{k+1}={\tilde{x}}_k+{\sigma }_k{\phi }_k{d}_k$, where ${\sigma }_k\in \left[1-\theta \parallel d_k\parallel ,1\right]$.           End if. Step 5. Update the Lagrange multiplier $y_{k+1}$, by using the following equation

$$y_{k+1}={\omega }_k{X}_k{}^{-1}e-X_k{}^{-1}{Y}_k{\mu }_k{d}_k.$$ (19) The above equation is obtained from (9). Step 6. To test the scaled step ${\mu }_k{d}_k$ and update ${\delta }_k$, use Algorithm 2. Step 7. Taking ${\omega }_{k+1}\in \left(0,{\omega }_k\right)$. Step 8. Use Algorithm 3 to update the penalty parameter ${\nu }_k$. Step 9. Put $k=k+1$ and return to Step 1.

The analysis of the global convergence for PBTR algorithm is dedicated in the following section.

4. Global Convergence Analysis for PBTR Algorithm

To prove the theory of the global convergence for the PBTR algorithm, we need the following assumptions.

The required assumptions:

Let ${\left\{{\tilde{x}}_k\right\}}_{k\ge 0}$ be the sequence of points produced by the PBTR algorithm and let $\mathsf{\Omega }$ be a convex subset of ${\Re }^{n+p}$ that contains all iterates ${\tilde{x}}_k>0$ and ${\tilde{x}}_k+{\mu }_k{d}_k>0$ for all trial steps $d_k$.

We offered the following assumptions of $\mathsf{\Omega }$ such that it is necessary to prove the theory of the global convergence.

[$N{A}_1$.] The functions $f\left(\tilde{x}\right)$, $G\left(\tilde{x}\right)\in {\mathcal{C}}^2$ for all $\tilde{x}\in \mathsf{\Omega }$.

[$N{A}_2$.] All of $f\left(\tilde{x}\right)$, $\nabla f\left(\tilde{x}\right)$, ${\nabla }^{2}f\left(\tilde{x}\right)$, $G\left(\tilde{x}\right)$, $\nabla G\left(\tilde{x}\right)$, ${\nabla }^2{G}_i\left(\tilde{x}\right)$ for $i=1,\dots ,m+p$,

and $\left(\nabla G_k\right){[{(\nabla G_k)}^T\left(\nabla G_k\right)]}^{-1}$ are uniformly bounded in $\mathsf{\Omega }$.

[$N{A}_3$.] The sequence $\left\{y_k\right\}$ is bounded.

[$N{A}_4$.] The sequence of approximated Hessian matrices $\left\{B_k\right\}$ is bounded.

The linearly independent assumption on $\nabla G\left(\tilde{x}\right)$ does not assume the above required assumptions. Then, there are other kinds of stationary points, which are introduced in the following definitions.

Definition 1

(A Fritz John point). A point ${\tilde{x}}_{*}\in \Omega$ is called a Fritz John (FRJO) point if there are ${\eta }_{*}\in \Re$ and a Lagrange multiplier vector ${\lambda }_{*}\in {\Re }^{m+p}$ that are not all zero such that:

$${\eta }_{*}\left(\nabla f\left({\tilde{x}}_{*}\right)-y\right)+\nabla G\left({\tilde{x}}_{*}\right){\lambda }_{*}=0,$$

(20)

$$G\left({\tilde{x}}_{*}\right)=0,$$

(21)

$${\tilde{X}}_{*}{y}_{*}-{\omega }_{*}e=0,$$

(22)

$${\eta }_{*}, y_{*}\ge 0.$$

(23)

The above conditions are FRJO conditions. For more details, see [28].

If${\eta }_{*}\ne 0$, then the FRJO conditions (20–23) are the KKT conditions and the point$\left({\tilde{x}}_{*},1,\frac{{\lambda }_{*}}{{\eta }_{*}}\right)$is the KKT point.

Definition 2

(Infeasible FRJO point). A point ${\tilde{x}}_{*}\in \Omega$ is called an infeasible FRJO point if there are ${\eta }_{*}\in \Re$ and a Lagrange multiplier vector ${\lambda }_{*}\in {\Re }^{m+p}$ that are not all zero such that:

$${\eta }_{*}\left(\nabla f\left({\tilde{x}}_{*}\right)-y\right)+\nabla G\left({\tilde{x}}_{*}\right){\lambda }_{*}=0,$$

(24)

$$\nabla G\left({\tilde{x}}_{*}\right)G\left({\tilde{x}}_{*}\right)=0, \parallel G\left({\tilde{x}}_{*}\right)\parallel >0$$

(25)

$${\tilde{X}}_{*}{y}_{*}-{\omega }_{*}e=0,$$

(26)

$${\eta }_{*}, y_{*}\ge 0.$$

(27)

The above conditions are infeasible FRJO conditions. For more details see [28]

If${\eta }_{*}\ne 0$then, the conditions (24–27) are called infeasible KKT conditions and the point$\left({\tilde{x}}_{*},1,\frac{{\lambda }_{*}}{{\eta }_{*}}\right)$is an infeasible KKT point.

The following two lemmas give conditions that are equivalent to the conditions that are given in Definitions 1–2.

Lemma 1.

Assume$N{A}_1$-$N{A}_3$hold. A subsequence$\left\{{\tilde{x}}_{k_i}\right\}$of${\left\{{\tilde{x}}_k\right\}}_{k\ge 0}$satisfies infeasible FRJO conditions if it satisfies:

(1)

$\mathrm{l}\mathrm{i}{\mathrm{m}}_{k_i\to \infty }\left\{{\tilde{X}}_{k_i}{y}_{k_i}-{\omega }_{k_i}e\right\}=0$.

(2)

$\mathrm{l}\mathrm{i}{\mathrm{m}}_{k_i\to \infty }\parallel G_{k_i}\parallel >0.$

(3)

$\mathrm{l}\mathrm{i}{\mathrm{m}}_{k_i\to \infty }\left\{\mathrm{m}\mathrm{i}{\mathrm{n}}_d\parallel G_{k_i}+\nabla G_{k_i}^T{\mu }_{k_i}d{\parallel }^2\right\}=\mathrm{l}\mathrm{i}{\mathrm{m}}_{k_i\to \infty }\parallel G_{k_i}{\parallel }^2$.

Proof. 

Let ${\overline{d}}_k$ be the solution of the problem ${\min }_d\parallel G_k+\nabla G_k^T{\mu }_{k}d{\parallel }^2$. Then, ${\overline{d}}_k$ satisfies the following equation

$${\mu }_k^2\nabla G_k\nabla G_k^T{\overline{d}}_k+{\mu }_k\nabla G_k{G}_k=0.$$

(28)

We will consider two cases: Firstly, if ${\lim }_{k\to \infty }{\overline{d}}_k=0$ in (28), then we have

$$\underset{k\to \infty }{\lim }\left\{{\mu }_k\nabla G_k{G}_k\right\}=0.$$

Secondly, if ${\lim }_{k\to \infty }{\overline{d}}_k\ne 0$. From the limit of condition 3, we have

$$\underset{k\to \infty }{\lim }\left\{{\mu }_k^2{\overline{d}}_k^T\nabla G_k\nabla G_k^T{\overline{d}}_k+2{\mu }_k{\overline{d}}_k^T\nabla G_k{G}_k\right\}=0.$$

(29)

Multiplying (28) from the left $2{\overline{d}}_k^T$ and subtracting it from (29), we have ${\lim }_{k\to \infty }\left\{{\mu }_k^2{\overline{d}}_k^T\nabla G_k\nabla G_k^T{\overline{d}}_k\right\}=0$. Hence, ${\lim }_{k\to \infty }\left\{{\mu }_k\nabla G_k{G}_k\right\}=0$. This implies that in either case, we have

$$\underset{k\to \infty }{\lim }\left\{\nabla G_k{G}_k\right\}=0,$$

where, ${\mu }_k\in \left(0,1\right]$. Taking ${({\lambda }_k)}_i={(G_k)}_i$, $i=1,\dots ,m+p$. Using Condition (25), we have ${\lim }_{k\to \infty }{({\lambda }_k)}_i\ge 0$, and ${\lim }_{k\to \infty }{({\lambda }_k)}_i>0$, for some $i$. Then, ${\lim }_{k\to \infty }\nabla G_k{\lambda }_k=0,$ and hence the conditions of Definition 2 hold in the limit with ${\eta }_{*}=0$. □

Lemma 2.

Assume$N{A}_1$-$N{A}_3$hold. A subsequence$\left\{{\tilde{x}}_{k_i}\right\}$of${\left\{{\tilde{x}}_k\right\}}_{k\ge 0}$satisfies FRJO conditions if it satisfies:

(1)

$\mathrm{l}\mathrm{i}{\mathrm{m}}_{k_i\to \infty }\left\{{\tilde{X}}_{k_i}{y}_{k_i}-{\omega }_{k_i}e\right\}=0$.

(2)

For all$k_i$,$\parallel G_{k_i}\parallel >0$and$\mathrm{l}\mathrm{i}{\mathrm{m}}_{k_i\to \infty }{G}_{k_i}=0.$

(3)

$\mathrm{l}\mathrm{i}{\mathrm{m}}_{k_i\to \infty }\left\{mi{n}_d\left\{\frac{\parallel G_{k_i}+\nabla G_{k_i}^T{\mu }_{k_i}d){\parallel }^2}{\parallel G_{k_i}{\parallel }^2}\right\}\right\}=1$.

Proof. 

Let ${\overline{z}}_k$ be the solution of the following problem

$$\underset{z}{\min }\parallel {\widehat{U}}_k+\nabla G_k^T{\mu }_{k}z{\parallel }^2,$$

(30)

where ${\widehat{U}}_k$ is a unit vector in the direction of $G_k$ and $z=\frac{d}{\parallel G_k\parallel }$. Then, ${\overline{z}}_k$ satisfies

$${\mu }_k^2\nabla G_k\nabla G_k^T{\overline{z}}_k+\nabla G_k{\widehat{U}}_k{\mu }_k=0.$$

(31)

We will consider two cases: Firstly, if ${\lim }_{k\to \infty }{\overline{z}}_k=0$ in the above equation, then we have

$$\underset{k\to \infty }{\lim }\nabla G_k{\widehat{U}}_k{\mu }_k=0.$$

Secondly, if ${\lim }_{k\to \infty }{\overline{z}}_k\ne 0$. Consider the following limit, which is equivalent to the limit in Condition (21)

$$\underset{k\to \infty }{\lim }\left\{\underset{z\in {\Re }^{n+p}}{\min }\parallel {\widehat{U}}_k+\nabla G_k^T{\mu }_{k}z{\parallel }^2\right\}=1,$$

and considering the fact that ${\overline{z}}_k$ is the minimization of Problem (31), we have

$$\underset{k\to \infty }{\lim }\left\{{\mu }_k^2{\overline{z}}_k{}^T\nabla G_k\nabla G_k^T{\overline{z}}_k+2{\mu }_k{\widehat{U}}_k^T\nabla G_k^T{\overline{z}}_k\right\}=0.$$

Multiplying (31) from the left by $2{\overline{z}}_k^T$ and subtracting it from the above limit, we have

$$\underset{k\to \infty }{\lim }{\mu }_k^2{\overline{z}}_k\nabla G_k\nabla G_k^T{\overline{z}}_k=0.$$

This implies ${\lim }_{k\to \infty }\left\{{\mu }_k\nabla G_k{\widehat{U}}_k\right\}=0$. Hence, in both cases, we have

$$\underset{k\to \infty }{\lim }\left\{\nabla G_k{\widehat{U}}_k\right\}=0.$$

The rest of the proof utilizes arguments similar to those in the above lemma. □

4.1. Fundamental Lemmas

We offer in this section some fundamental lemmas that are required in the subsequent proofs.

In the following lemma, we prove that $Pre{d}_k$ at any iteration $k$ is at least equal to the decrease in the quadratic model $q_k\left(d_k^{cp}\right)$ where $d_k^{cp}$ is the Cauchy step.

Lemma 3.

Suppose$N{A}_1$-$N{A}_4$. Then, there exists a constant$0<K_1$for all$k>\overline{k}$ such that,

$$Pre{d}_k\ge K_1{\mu }_k\parallel \nabla {\mathsf{\Phi }}^{\omega }\left({\tilde{x}}_k;{\nu }_k\right)\parallel min\left\{{\delta }_k,\frac{\parallel \nabla {\mathsf{\Phi }}^{\omega }\left({\tilde{x}}_k;{\nu }_k\right)\parallel }{\parallel A_k\parallel }\right\},$$

(32)

where $\nabla {\mathsf{\Phi }}^{\omega }\left({\tilde{x}}_k;{\nu }_k\right)=\nabla {\psi }^{\omega }\left(\tilde{x}\right)+{\nu }_k\nabla G\left({\tilde{x}}_k\right)G\left({\tilde{x}}_k\right)$.

Proof. 

Since the conjugate gradient method is used to evaluate an approximate solution to the Subproblem (12), then a fraction of the predicted decrease obtained by the Cauchy step $d_k^{cp}$ is less than or equal to the predicted decrease obtained by the step $d_k$. That is

$$q_k\left(0\right)-q_k\left(d_k\right)\ge \vartheta \left[q_k\left(0\right)-q_k\left(d_k^{cp}\right)\right],$$

(33)

for some $\vartheta \in \left(0,1\right]$, where $d_k^{cp}$ is defined as

$$d_k^{cp}=-t_k^{cp}\nabla {\mathsf{\Phi }}^{\omega }\left({\tilde{x}}_k;{\nu }_k\right),$$

and the Cauchy parameter $t_k^{cp}$ is defined by

$$t_k^{cp}=\{\begin{array}{ll}\frac{{\parallel \nabla {\mathsf{\Phi }}^{\omega }({\tilde{x}}_k;{\nu }_k)\parallel }^2}{{(\nabla {\mathsf{\Phi }}^{\omega }({\tilde{x}}_k;{\nu }_k))}^T{A}_k\nabla {\mathsf{\Phi }}^{\omega }({\tilde{x}}_k;{\nu }_k)}& \mathit{if}\frac{{\parallel \nabla {\mathsf{\Phi }}^{\omega }({\tilde{x}}_k;{\nu }_k)\parallel }^3}{{(\nabla {\mathsf{\Phi }}^{\omega }({\tilde{x}}_k;{\nu }_k))}^T{A}_k\nabla {\mathsf{\Phi }}^{\omega }({\tilde{x}}_k;{\nu }_k)}\le {\delta }_k\\ & \mathit{and}{(\nabla {\mathsf{\Phi }}^{\omega }({\tilde{x}}_k;{\nu }_k))}^T{A}_k\nabla {\mathsf{\Phi }}^{\omega }({\tilde{x}}_k;{\nu }_k)>0,\\ \frac{{\delta }_k}{\parallel \nabla {\mathsf{\Phi }}^{\omega }({\tilde{x}}_k;{\nu }_k)\parallel }& \mathit{otherwise}.\end{array}$$

(34)

We will consider two cases:

(i)

If $d_k^{cp}=-\frac{{\delta }_k}{\parallel \nabla {\mathsf{\Phi }}^{\omega }\left({\tilde{x}}_k;{\nu }_k\right)\parallel }\left(\nabla {\mathsf{\Phi }}^{\omega }\left({\tilde{x}}_k;{\nu }_k\right)\right)$ and $\parallel \nabla {\mathsf{\Phi }}^{\omega }\left({\tilde{x}}_k;{\nu }_k\right){\parallel }^3\ge {\delta }_k[\left(\nabla {\mathsf{\Phi }}^{\omega }\left({\tilde{x}}_k;{\nu }_k\right){)}^T{A}_k\left(\nabla {\mathsf{\Phi }}^{\omega }\left({\tilde{x}}_k;{\nu }_k\right)\right)\right]$, then

$$\begin{array}{l}{q}_k\left(0\right)-q_k\left(d_k^{cp}\right)=-{(\nabla {\mathsf{\Phi }}^{\omega }\left({\tilde{x}}_k;{\nu }_k\right))}^T{d}_k^{cp}-\frac{1}{2}{d}_k^{c{p}^T}{A}_k{d}_k^{cp}\\ =\frac{{\delta }_k}{\parallel \nabla {\mathsf{\Phi }}^{\omega }\left({\tilde{x}}_k;{\nu }_k\right)\parallel }\parallel \nabla {\mathsf{\Phi }}^{\omega }\left({\tilde{x}}_k;{\nu }_k\right){\parallel }^2\\ -\frac{1}{2}\frac{{\delta }_k^2}{\parallel \nabla {\mathsf{\Phi }}^{\omega }\left({\tilde{x}}_k;{\nu }_k\right){\parallel }^2}(\left(\nabla {\mathsf{\Phi }}^{\omega }\left({\tilde{x}}_k;{\nu }_k\right){)}^T{A}_k\left(\nabla {\mathsf{\Phi }}^{\omega }\left({\tilde{x}}_k;{\nu }_k\right)\right)\right)\\ \ge \frac{1}{2}{\delta }_k\parallel \nabla {\mathsf{\Phi }}^{\omega }\left({\tilde{x}}_k;{\nu }_k\right)\parallel .\end{array}$$

(35)

(ii)

If $d_k^{cp}=-\frac{\parallel \nabla {\mathsf{\Phi }}^{\omega }\left({\tilde{x}}_k;{\nu }_k\right){\parallel }^2}{{(\nabla {\mathsf{\Phi }}^{\omega }\left({\tilde{x}}_k;{\nu }_k\right))}^T{A}_k\left(\nabla {\mathsf{\Phi }}^{\omega }\left({\tilde{x}}_k;{\nu }_k\right)\right)}\left(\nabla {\mathsf{\Phi }}^{\omega }\left({\tilde{x}}_k;{\nu }_k\right)\right)$, and $\parallel \nabla {\mathsf{\Phi }}^{\omega }\left({\tilde{x}}_k;{\nu }_k\right){\parallel }^3\le {\delta }_k(\left(\nabla {\mathsf{\Phi }}^{\omega }\left({\tilde{x}}_k;{\nu }_k\right){)}^T{A}_k\left(\nabla {\mathsf{\Phi }}^{\omega }\left({\tilde{x}}_k;{\nu }_k\right)\right)\right)$, then

$$\begin{array}{ll}& q_k(0)-q_k\left(d_k^{cp}\right)=-{(\nabla {\mathsf{\Phi }}^{\omega }({\tilde{x}}_k;{\nu }_k))}^T{d}_k^{cp}-\frac{1}{2}{d}_k^{c{p}^T}{A}_k{d}_k^{cp}\\ & =\frac{1}{2}\frac{{\parallel \nabla {\mathsf{\Phi }}^{\omega }({\tilde{x}}_k;{\nu }_k)\parallel }^4}{{(\nabla {\mathsf{\Phi }}^{\omega }({\tilde{x}}_k;{\nu }_k))}^T{A}_k(\nabla {\mathsf{\Phi }}^{\omega }({\tilde{x}}_k;{\nu }_k))}\\ & \ge \frac{{\parallel \nabla {\mathsf{\Phi }}^{\omega }({\tilde{x}}_k;{\nu }_k)\parallel }^2}{2\parallel A_k\parallel }.\end{array}$$

(36)

From inequalities (33), (35), and (36), we have,

$$q_k\left(0\right)-q_k\left(d_k\right)\ge K_1\parallel \nabla {\mathsf{\Phi }}^{\omega }\left({\tilde{x}}_k;{\nu }_k\right)\parallel \min \left\{{\delta }_k,\frac{\parallel \nabla {\mathsf{\Phi }}^{\omega }\left({\tilde{x}}_k;{\nu }_k\right)\parallel }{\parallel A_k\parallel }\right\}.$$

Since ${\mu }_k\in \left(0,1\right]$, then from the above inequality and using the fact

$$q_k\left(0\right)-q_k\left({\mu }_k{d}_k\right)\ge {\mu }_k\left[q_k\left(0\right)-q_k\left(d_k\right)\right]$$

we have

$$q_k\left(0\right)-q_k\left({\mu }_k{d}_k\right)\ge K_1{\mu }_k\parallel \nabla {\mathsf{\Phi }}^{\omega }\left({\tilde{x}}_k;{\nu }_k\right)\parallel \min \left\{{\delta }_k,\frac{\parallel \nabla {\mathsf{\Phi }}^{\omega }\left({\tilde{x}}_k;{\nu }_k\right)\parallel }{\parallel A_k\parallel }\right\}.$$

From (17) and using the above inequality, we have

$$Pre{d}_k\ge K_1{\mu }_k\parallel \nabla {\mathsf{\Phi }}^{\omega }\left({\tilde{x}}_k;{\nu }_k\right)\parallel \min \left\{{\delta }_k,\frac{\parallel \nabla {\mathsf{\Phi }}^{\omega }\left({\tilde{x}}_k;{\nu }_k\right)\parallel }{\parallel A_k\parallel }\right\}.$$

□

Lemma 4.

Suppose$N{A}_1$-$N{A}_4$, then there exists a constant 0 < K² such that

$$\left|Are{d}_k-Pre{d}_k\right|\le K_2{\mu }_k{\nu }_k\parallel d_k{\parallel }^2.$$

(37)

Proof. 

From Equation (15), we have

$$Are{d}_k={\psi }^{\omega }\left({\tilde{x}}_k\right)-{\psi }^{\omega }\left({\tilde{x}}_k+{\mu }_k{d}_k\right)+\frac{{\nu }_k}{2}\left[\parallel G\left({\tilde{x}}_k\right){\parallel }^2-\parallel G\left({\tilde{x}}_k+{\mu }_k{d}_k\right){\parallel }^2\right].$$

From the above definition of $Are{d}_k$, using (16), and Cauchy–Schwarz inequality, we have

$$\begin{array}{l}\left|Are{d}_k-Pre{d}_k\right|\le \frac{{\mu }_k^2}{2}\left|d_k^T\left(H_k-{\nabla }^2\psi \left({\tilde{x}}_k+{\xi }_1{\mu }_k{d}_k\right)\right)d_k\right|\\ +\frac{{\mu }_k^2}{2}\left|d_k^T{\tilde{X}}_k^{-1}{Y}_k{d}_k\left|+{\nu }_k{\mu }_k\right|\left(\nabla G_k-\nabla G\left({\tilde{x}}_k+{\xi }_2{\mu }_k{d}_k\right)\right)G_k{d}_k\right|\\ +\frac{{\nu }_k}{2}{\mu }_k^2|d_k^T[\nabla G_k\nabla G_k^T-\nabla G\left({\tilde{x}}_k+{\xi }_2{\mu }_k{d}_k\right)\nabla G\left({\tilde{x}}_k+{\xi }_2{\mu }_k{d}_k{)}^T\right]d_k|\end{array}$$

for some ${\xi }_1$ and ${\xi }_2\in \left(0,1\right)$. Using the required assumptions, there are then positive constants ${\kappa }_1$, ${\kappa }_2$, and ${\kappa }_3$ such that

$$\left|Are{d}_k-Pre{d}_k\right|\le {\mu }_k\left[{\kappa }_1\parallel d_k{\parallel }^2+{\kappa }_2{\nu }_k\parallel d_k{\parallel }^3+{\kappa }_3{\nu }_k\parallel d_k{\parallel }^2\right].$$

(38)

Since ${\nu }_k>0,$ the above inequality can then be written as

$$\left|Are{d}_k-Pre{d}_k\right|\le K_2{\mu }_k{\nu }_k\parallel d_k{\parallel }^2.$$

This completes the proof. □

In the following section, we study the convergence of ${\left\{{\tilde{x}}_k\right\}}_{k\ge 0}$ when ${\nu }_k$ goes to infinity as $k$ is very large.

4.2. Studying the Convergence if ${\nu }_k\to \infty$

Let $\left\{k_j\right\}$ be an infinite subsequence of indices indexing iterates of acceptable steps. From Algorithm 3, we know,

$$Pre{d}_k<\parallel \nabla G_k{G}_k\parallel \min \left\{\parallel \nabla G_k{G}_k\parallel ,{\delta }_k\right\},$$

(39)

for all $k\in \left\{k_j\right\}$. That is, the sequence $\left\{{\nu }_k\right\}$ goes to infinity.

In the following two lemmas, we prove that if ${\nu }_k\to \infty$ as $k\to \infty$, then the subsequence of ${\left\{{\tilde{x}}_k\right\}}_{k\ge 0}$ satisfies infeasible FRJO conditions or FRJO conditions.

Lemma 5.

Assume$N{A}_1$-$N{A}_4$hold and${\nu }_k\to \infty$as$k\to \infty$. If there exists a subsequence$\left\{k_i\right\}$of indices indexing iterates that satisfy$\parallel G_k\parallel \ge {\epsilon }_1>0$for all$k\in \left\{k_i\right\}$, then in the limit, the infeasible FRJO conditions are held on it.

Proof. 

By contradiction, we prove this lemma. To simplify, let $\left\{k_i\right\}$ be the whole iteration sequence index $\left\{k\right\}$. Assume that there is not any subsequence of iterates in the limit, satisfying the infeasible FRJO conditions. Then, from Definition 2, we have $\parallel \nabla G_k{G}_k\parallel \ge {\epsilon }_1,$ and from Lemma 1, we have $|\parallel G_k{\parallel }^2-\parallel G_k+{\mu }_k\nabla G_k^T{d}_k{\parallel }^2|\ge {\epsilon }_1$ for some ${\epsilon }_1>0.$ We will consider two cases: Firstly, if $\parallel \nabla G_k{G}_k\parallel \ge {\epsilon }_1$, then

$$\parallel \nabla {\psi }^{\omega }\left({\tilde{x}}_k\right)+{\nu }_k\nabla G_k{G}_k\parallel \ge {\nu }_k\parallel \nabla G_k{G}_k\parallel -\parallel \nabla {\psi }^{\omega }\left({\tilde{x}}_k\right)\parallel \ge {\nu }_k{\epsilon }_1-\parallel \nabla {\psi }^{\omega }\left({\tilde{x}}_k\right)\parallel .$$

(40)

From inequalities (32), (40) and the fact

$$\parallel A_k\parallel =\parallel B_k+{\nu }_k\nabla G_k\nabla G_k^T\parallel \le {\nu }_k\parallel \nabla G_k\nabla G_k^T\parallel +\parallel B_k\parallel ,$$

we have

$$Pre{d}_k\ge K_1{\mu }_k\left[{\nu }_k{\epsilon }_1-\parallel \nabla {\psi }^{\omega }\left({\tilde{x}}_k\right)\parallel \right]\min \left\{{\delta }_k,\frac{{\nu }_k{\epsilon }_1-\parallel \nabla {\psi }^{\omega }\left({\tilde{x}}_k\right)\parallel }{{\nu }_k\parallel \nabla G_k\nabla G_k^T\parallel +\parallel B_k\parallel }\right\}.$$

From assumptions $N{A}_1$-$N{A}_4$ and for very large $k$, we have

$$Pre{d}_k\ge K_1{\mu }_k{\nu }_k{\epsilon }_1\min \left\{{\delta }_k,\frac{{\epsilon }_1}{\parallel \nabla G_k\nabla G_k^T\parallel }\right\}.$$

(41)

However, ${\nu }_k\to \infty$; hence, there are an infinite number of acceptable iterates at which inequality (39) holds. Using inequalities (39) and (41), then

$$\parallel \nabla G_k{G}_k\parallel \min \left\{\parallel \nabla G_k{G}_k\parallel ,{\delta }_k\right\}\ge K_1{\mu }_k{\nu }_k{\epsilon }_1\min \left\{{\delta }_k,\frac{{\epsilon }_1}{\parallel \nabla G_k\nabla G_k^T\parallel }\right\}$$

From the above inequality, we notice that the right-hand side $\to \infty$ as $k\to \infty$, but the left-hand side is bounded. This gives a contradiction unless δ_k → 0.

Secondly, if $|\parallel G_k{\parallel }^2-\parallel G_k+{\mu }_k\nabla G_k^T{d}_k{\parallel }^2|\ge {\epsilon }_1$ for some ${\epsilon }_1>0$, we consider two cases as $k\to \infty$ and ${\nu }_k\to \infty$:

(i)

If $\parallel G_k{\parallel }^2-\parallel G_k+\nabla G_k^T{\mu }_k{d}_k{\parallel }^2\ge {\epsilon }_1$, then

$${\nu }_k\left\{\parallel G_k{\parallel }^2-\parallel G_k+\nabla G_k^T{\mu }_k{d}_k{\parallel }^2\right\}\ge {\nu }_k{\epsilon }_1\to \infty .$$

That is $Pre{d}_k\to \infty$. Hence, the right-hand side goes to zero while the left-hand side of inequality (39) goes to infinity. However, this is a contradiction with the assumption in this case.

(ii)

If ${\parallel G_k\parallel }^2-\parallel G_k+\nabla G_k^T{\mu }_k{d}_k{\parallel }^2\le -{\delta }_1,$ then

$${\nu }_k\{\parallel G_k{\parallel }^2-\parallel G_k+\nabla G_k^T{\mu }_k{d}_k{\parallel }^2\}\le -{\nu }_k{\epsilon }_1\to -\infty .$$

Similar to the case (i), $Pre{d}_k\to -\infty$. However, $Pre{d}_k>0,$ and this gives a contradiction. Hence, the lemma is proven.

The behavior of Algorithm 4 is studied in the following lemma when ${\mathrm{liminf}}_{k\to \infty }\parallel G_k\parallel =0$ and ${\nu }_k\to \infty$ as $k\to \infty$. □

Lemma 6.

Assume$N{A}_1$-$N{A}_4$hold and${\nu }_k\to \infty$as$k\to \infty$. If there exists a subsequence$\left\{k_i\right\}$of iterates that satisfies$\parallel G_k\parallel >0$for all$k\in \left\{k_i\right\}$and$li{m}_{k_i\to \infty }\parallel G_{k_i}\parallel =0$, then there exists a subsequence of${\left\{{\tilde{x}}_k\right\}}_{k\ge 0}$indexed$\left\{k_i\right\}$that satisfies in the limit the FRJO conditions.

Proof. 

This lemma is proven by contradiction, and to simplify the notation, we let $\left\{k_i\right\}$ be the whole iteration sequence index $\left\{k\right\}$. Assume that there is not any subsequence of iteration sequence ${\left\{{\tilde{x}}_k\right\}}_{k\ge 0}$ that satisfies FRJO conditions in the limit. Then, from Lemma 2 and for all $0<{\epsilon }_1$, we have

$$\frac{|\parallel G_k{\parallel }^2-\parallel G_k+\nabla G_k^T{\mu }_k{d}_k{\parallel }^2|}{\parallel G_k{\parallel }^2}\ge {\epsilon }_1,$$

(42)

for all $k$ sufficiently large. Now, we study three cases as $k\to \infty$:

(i)

If ${\mathrm{liminf}}_{k\to \infty }\frac{d_k}{\parallel G_k\parallel }=0$, we have a contradiction with inequality (42).

(ii)

If ${\mathrm{limsup}}_{k\to \infty }\frac{d_k}{\parallel G_k\parallel }=\infty$. From Subproblem (12), we have

$$\nabla {\psi }^{\omega }\left({\tilde{x}}_k\right)+{\nu }_k\nabla G\left({\tilde{x}}_k\right)G\left({\tilde{x}}_k\right)=-\left(A_k+{\overline{\rho }}_{k}I\right)d,$$

(43)

where $0<{\overline{\rho }}_k$ is the Lagrange multiplier vector corresponding with the constraint $\parallel d\parallel \le {\delta }_k$. Hence, inequality (32) can be written as follows

$$Pre{d}_k\ge K_1{\mu }_k\parallel \nabla {\psi }^{\omega }\left({\tilde{x}}_k\right)+{\nu }_k\nabla G\left({\tilde{x}}_k\right)G\left({\tilde{x}}_k\right)\parallel \min \left\{{\delta }_k,\frac{\parallel \left(A_k+{\overline{\rho }}_{k}I\right)d_k\parallel }{\parallel A_k\parallel }\right\}.$$

However, $A_k=B_k+{\nu }_k\nabla G_k\nabla G_k^T$, hence

$$Pre{d}_k\ge K_1{\mu }_k\parallel \nabla {\psi }^{\omega }\left({\tilde{x}}_k\right)+{\nu }_k\nabla G\left({\tilde{x}}_k\right)G\left({\tilde{x}}_k\right)\parallel \min \left\{{\delta }_k,\frac{\parallel \left[\frac{1}{{\nu }_k}{B}_k+\left(\nabla G_k\nabla G_k^T+\frac{{\overline{\rho }}_k}{{\nu }_k}I\right)\right]d_k\parallel }{\parallel \frac{1}{{\nu }_k}{B}_k+\nabla G_k\nabla G_k^T\parallel }\right\}.$$

(44)

If ${\nu }_k\to \infty$, then inequality (39) holds at an infinite number of acceptable steps, and it can be written as follows

$$Pre{d}_k<\parallel \nabla G_k{\parallel }^2\parallel G_k{\parallel }^2.$$

(45)

From inequalities (44) and (45), we have

$$K_1{\mu }_k\parallel \nabla {\psi }^{\omega }\left({\tilde{x}}_k\right)+{\nu }_k\nabla G\left({\tilde{x}}_k\right)G\left({\tilde{x}}_k\right)\parallel \min \left\{{\delta }_k,\frac{\parallel \left[\frac{1}{{\nu }_k}{B}_k+\nabla G_k\nabla G_k^T+\frac{{\overline{\rho }}_k}{{\nu }_k}I\right]d_k\parallel }{\parallel \frac{1}{{\nu }_k}{B}_k+\nabla G_k\nabla G_k^T\parallel }\right\}\le {\kappa }_4^2\parallel G_k{\parallel }^2,$$

where ${\kappa }_4=su{p}_{\tilde{x}\in \mathsf{\Omega }}\parallel \nabla G_k\parallel$. Dividing the above inequality by $\parallel G_k\parallel$, then

$$K_1{\mu }_k\parallel \nabla {\psi }^{\omega }\left({\tilde{x}}_k\right)+{\nu }_k\nabla G\left({\tilde{x}}_k\right)G\left({\tilde{x}}_k\right)\parallel \min \left\{\frac{{\delta }_k}{\parallel G_k\parallel },\frac{\parallel \left[\frac{1}{{\nu }_k}{B}_k+\nabla G_k\nabla G_k^T+\frac{{\overline{\rho }}_k}{{\nu }_k}I\right]d_k\parallel }{\parallel \frac{1}{{\nu }_k}{B}_k+\nabla G_k\nabla G_k^T\parallel \parallel G_k\parallel }\right\}<{\kappa }_4^2\parallel G_k\parallel .$$

(46)

From the above inequality, we notice that the right-hand side $\to 0$ as $k\to \infty .$ This implies that

$$\parallel \nabla {\psi }^{\omega }\left({\tilde{x}}_k\right)+{\nu }_k\nabla G\left({\tilde{x}}_k\right)G\left({\tilde{x}}_k\right)\parallel \frac{\parallel \left[\nabla G_k\nabla G_k^T\right]d_k\parallel }{\parallel \nabla G_k\nabla G_k^T\parallel \parallel G_k\parallel },$$

is bounded over the subsequence $\left\{k\right\}$ where ${\lim }_{k\to \infty }\frac{d_k}{\parallel G_k\parallel }=\infty$. That is, either $\frac{d_k}{\parallel G_k\parallel }$ lies in the null space of $\nabla G_k\nabla G_k^T$ or $\parallel \nabla {\psi }^{\omega }\left({\tilde{x}}_k\right)+{\nu }_k\nabla G\left({\tilde{x}}_k\right)G\left({\tilde{x}}_k\right)\parallel \to 0$. The first possibility occurs when $\frac{d_k}{\parallel G_k\parallel }$ lies in the null space of the matrix $\nabla G_k\nabla G_k^T$, which contradicts with assumption (42). That is a subsequence of ${\left\{{\tilde{x}}_k\right\}}_{k\ge 0}$ in the limit satisfies the FRJO conditions. The second possibility implies $\parallel \nabla {\psi }^{\omega }\left({\tilde{x}}_k\right)+{\nu }_k\nabla G\left({\tilde{x}}_k\right)G\left({\tilde{x}}_k\right)\parallel \to 0$, as $k\to \infty$. Hence, ${\nu }_k\parallel \nabla G_k{G}_k\parallel$ must be bounded, and we have $\nabla G_k{G}_k\to 0$. That is, a subsequence of ${\left\{{\tilde{x}}_k\right\}}_{k\ge 0}$ in the limit satisfies the FRJO conditions.

(iii)

If ${\mathrm{limsup}}_{k\to \infty }\frac{d_k}{\parallel G_k\parallel }<\infty$ and ${\mathrm{liminf}}_{k\to \infty }\frac{d_k}{\parallel G_k\parallel }>0$, then $\parallel d_k\parallel \to 0$. Hence, as $k\to \infty$, we notice that the right-hand side of inequality (46) tends to zero as in the second case. This implies that

$$\parallel \nabla {\psi }^{\omega }\left({\tilde{x}}_k\right)+{\nu }_k\nabla G\left({\tilde{x}}_k\right)G\left({\tilde{x}}_k\right)\parallel \frac{\parallel \nabla G_k\nabla G_k^T{d}_k\parallel }{\parallel \nabla G_k\nabla G_k^T\parallel \parallel G_k\parallel }\to 0.$$

That is, either $\parallel \nabla {\psi }^{\omega }\left({\tilde{x}}_k\right)+{\nu }_k\nabla G\left({\tilde{x}}_k\right)G\left({\tilde{x}}_k\right)\parallel \to 0$ or $\frac{\parallel \nabla G_k\nabla G_k^T{d}_k\parallel }{\parallel \nabla G_k\nabla G_k^T\parallel \parallel G_k\parallel }\to 0$. As in Case (ii), we can prove in the limit that the subsequence of ${\left\{{\tilde{x}}_k\right\}}_{k\ge 0}$ satisfies the FRJO conditions. This completes the proof.

The convergence of the sequence ${\left\{{\tilde{x}}_k\right\}}_{k\ge 0}$ is studied when ${\nu }_k$ is bounded in the following section. □

4.3. Studying the Convergence if ${\nu }_k$ Is Bounded

We study in this section the convergence when ${\nu }_k$ is bounded. Therefore, we set ${\nu }_k=\tilde{\nu }<\infty$ for all $\tilde{k}\le k$.

Lemma 7.

Assume$N{A}_1$-$N{A}_4$hold. At any given iteration of indexed$k$at which$\parallel \nabla {\Phi }^{\omega }\left({\tilde{x}}_k;\tilde{\nu }\right)\parallel +\parallel {\tilde{X}}_k{Y}_{k}e-{\omega }_{k}e\parallel +\parallel \nabla G\left({\tilde{x}}_k\right)G\left({\tilde{x}}_k\right)\parallel >ϵ$, there exists a constant$K_3>0$ such that

$$Pre{d}_k\ge K_3{\delta }_k.$$

(47)

Proof. 

Since $A_k=B_k+\tilde{\nu }\nabla G_k\nabla G_k^T$ and using the required assumptions, we can say that $\parallel A_k\parallel \le {\kappa }_5$, where $0<{\kappa }_5$ is a constant. From the method of updating the barrier parameter ${\omega }_k$, we have $\parallel {\tilde{X}}_k{Y}_{k}e-{\omega }_{k}e\parallel \le \frac{ϵ}{3}$ for very large $k$. Then, $\parallel \nabla {\mathsf{\Phi }}^{\omega }\left({\tilde{x}}_k;\tilde{\nu }\right)\parallel +\parallel \nabla G\left({\tilde{x}}_k\right)G\left({\tilde{x}}_k\right)\parallel >\frac{2ϵ}{3}$. Assume that $\parallel \nabla {\mathsf{\Phi }}^{\omega }\left({\tilde{x}}_k;\tilde{\nu }\right)\parallel >\frac{ϵ}{3}$; then, $\parallel \nabla G_k{G}_k\parallel >\frac{ϵ}{3}$. Using Lemma 3, we have

$$Pre{d}_k\ge K_1{\mu }_k\parallel \nabla {\mathsf{\Phi }}^{\omega }\left({\tilde{x}}_k;\tilde{\nu }\right)\parallel \min \left\{{\delta }_k,\frac{\parallel \nabla {\mathsf{\Phi }}^{\omega }\left({\tilde{x}}_k;\tilde{\nu }\right)\parallel }{\parallel A_k\parallel }\right\}\ge \frac{K_1ϵ}{3}\min \left\{1,\frac{ϵ}{3{\kappa }_5{\delta }_{\max }}\right\}{\delta }_k.$$

Now, consider the case when $\parallel \nabla G_k{G}_k\parallel >\frac{ϵ}{3}$, and using (18), we have

$$Pre{d}_k\ge \frac{ϵ}{3}\min \left\{\frac{ϵ}{3{\delta }_{\max }},1\right\}{\delta }_k.$$

Taking $K_3=\min \left\{\frac{K_1ϵ}{3}\min \left\{1,\frac{ϵ}{3{\kappa }_5{\delta }_{\max }}\right\},\frac{ϵ}{3}\min \left\{\frac{ϵ}{3{\delta }_{\max }},1\right\}\right\}$, the result follows. □

Lemma 8.

Assume$N{A}_1$-$N{A}_4$. hold. If$\parallel \nabla {\Phi }^{\omega }\left({\tilde{x}}_k;\tilde{\nu }\right)\parallel +\parallel {\tilde{X}}_k{Y}_{k}e-{\omega }_{k}e\parallel +\parallel \nabla G\left({\tilde{x}}_k\right)G\left({\tilde{x}}_k\right)\parallel >ϵ$, then for some finite$j$, the condition$Are{d}_{k_j}\ge {\beta }_{1}Pre{d}_{k_j}$will be satisfied. That is, after finitely many trial steps computations, we can find an acceptable step.

Proof. 

From inequalities (18) and (37), we have

$$\left|\frac{Are{d}_k}{Pre{d}_k}-1\right|=\frac{\left|Are{d}_k-Pre{d}_k\right|}{Pre{d}_k}\le \frac{K_2\tilde{\nu }{\delta }_k^2}{K_3{\delta }_k}\le \frac{K_2\tilde{\nu }{\delta }_k}{K_3}.$$

That is, ${\delta }_{k_j}$ turns into a small value and ultimately after a finite number of trials if $d_{k_j}$ is rejected. This means that the acceptance rule (for j finite) will be met and hence the proof is completed. □

Lemma 9.

Assume$N{A}_1$-$N{A}_4$hold. If$\parallel \nabla {\Phi }^{\omega }\left({\tilde{x}}_k;\tilde{\nu }\right)\parallel +\parallel {\tilde{X}}_k{Y}_{k}e-{\omega }_{k}e\parallel +\parallel \nabla G\left({\tilde{x}}_k\right)G\left({\tilde{x}}_k\right)\parallel >ϵ$, at$j^{th}$trial step of any iteration$k$satisfies

$$\parallel d_{k^j}\parallel \le \frac{\left(1-{\beta }_1\right)K_3}{2\tilde{\nu }{K}_2},$$

(48)

then the trial step must be accepted.

Proof. 

By contradiction, we can prove this lemma. Suppose that the inequality (48) holds and $d_{k^j}$ is rejected. Using inequalities (37) and (47), we have

$$\left(1-{\beta }_1\right)<\frac{\left|Are{d}_{k^j}-Pre{d}_{k^j}\right|}{Pre{d}_{k^j}}<\frac{K_2\tilde{\nu }\parallel d_{k^j}{\parallel }^2}{K_3\parallel d_{k^j}\parallel }\le \frac{\left(1-{\beta }_1\right)}{2}.$$

This gives a contradiction with the assumption. This completes the proof.

The main global convergence result for Algorithm 4 is proven in the following theorem.

Theorem 1.

Assume$N{A}_1$-$N{A}_4$hold. Then,${\left\{{\tilde{x}}_k\right\}}_{k\ge 0}$ satisfies

$$\underset{k\to \infty }{\mathrm{liminf}} \left[ \parallel \nabla {\psi }^{\omega }\left({\tilde{x}}_k\right)\parallel +\parallel {\tilde{X}}_k{Y}_{k}e-{\omega }_{k}e\parallel +\parallel \nabla G_k{G}_k\parallel \right]=0.$$

(49)

Proof. 

Firstly, we show that

$$\underset{k\to \infty }{\mathrm{liminf}}\left[\parallel \nabla {\mathsf{\Phi }}^{\omega }\left({\tilde{x}}_k;\tilde{\nu }\right)\parallel +\parallel {\tilde{X}}_k{Y}_{k}e-{\omega }_{k}e\parallel +\parallel \nabla G_k{G}_k\parallel \right]=0.$$

(50)

By contradiction, we will prove (50). For all $k$, we suppose that

$\parallel \nabla {\mathsf{\Phi }}^{\omega }\left({\tilde{x}}_k;\tilde{\nu }\right)\parallel +\parallel {\tilde{X}}_k{Y}_{k}e-{\omega }_{k}e\parallel +\parallel \nabla G_k{G}_k\parallel >ϵ$. Let $k\ge \tilde{k}$ and $k^j\ge \overline{k}$ where $j$ is the trial step of the iteration $k$. From Lemma 7, we have

$${\mathsf{\Phi }}_{k^j}^{\omega }-{\mathsf{\Phi }}_{k^j+1}^{\omega }=Are{d}_{k^j}\ge {\beta }_{1}Pre{d}_{k^j}\ge {\beta }_1{K}_3{\delta }_{k^j}.$$

(51)

for any acceptable step indexed $k^j$. If $k\to \infty$, then

$$\underset{k\to \infty }{\lim }{\delta }_{k^j}=0.$$

(52)

This means that the radius ${\delta }_{k^j}$ is not bounded below. We will consider two cases:

(i)

If $j=1$, then ${\delta }_{k^1}\ge {\delta }_{\min }$. Hence, in this case, ${\delta }_{k^j}$ is bounded below.

(ii)

If $j>1$, then there exists at least one rejected trial step. Hence, at the rejected trial step, $d_{k^i}$ for all $i=1,2,\dots j-1$, and using Lemma 9, we have

$$\parallel d_{k^i}\parallel >\frac{\left(1-{\beta }_1\right)K_3}{2\tilde{\nu }{K}_2}.$$

However, if $d_{k^j}$ is rejected, then we have

$${\delta }_{k^j}={\eta }_1\parallel d_{k^{j-1}}\parallel >{\eta }_1\frac{\left(1-{\beta }_1\right)K_3}{2\tilde{\nu }{K}_2},$$

see Step 5 in Algorithm 4. Hence, ${\delta }_{k^j}$ is bounded below. This contradicts with (52). That is, the supposition is wrong and (50) hold. This also leads to (49) hold and the proof of this theorem is completed. Hence, the PBTR algorithm terminates because

$$\parallel \nabla {\psi }^{\omega }\left({\tilde{x}}_k\right)\parallel +\parallel {\tilde{X}}_k{Y}_{k}e-{\omega }_{k}e\parallel +\parallel \nabla G_k{G}_k\parallel <ϵ$$

5. Numerical Results

In order to validate the effectiveness of the proposed algorithm, it was applied to some benchmark problems [37] and an engineering application.

5.1. Benchmark Test Problems

Benchmark problems are listed in [37] to show the effectiveness of the PBTR algorithm. For comparison, we have included the corresponding results of the PBTR algorithm against the corresponding numerical results in [38,39]. This is summarized in

Table 1. Comparison of methods in [38,39] with PBTR algorithm.

Problem	Name	No. of Variables	Feasibility of Initial Solution	Niter
				Method [38]	Method [39]	Algorithm (PBTR)
P1	hs006	2	Infeasible	7	5	4
P2	hs007	2	Infeasible	9	8	7
P3	hs008	2	Infeasible	14	6	8
P4	hs009	2	feasible	10	6	7
P5	hs012	2	feasible	5	7	4
P6	hs024	2	feasible	14	4	9
P7	hs026	3	Feasible	19	19	14
P8	hs027	3	Infeasible	14	18	12
P9	hs028	3	Feasible	6	2	3
P10	hs029	3	Feasible	8	6	9
P11	hs030	3	Feasible	7	6	8
P12	hs032	3	Feasible	24	5	6
P13	hs033	3	Feasible	29	6	8
P14	hs034	3	Feasible	30	5	7
P15	hs036	3	Feasible	10	7	9
P16	hs037	3	Feasible	7	6	6
P17	hs039	4	Infeasible	19	23	5
P18	hs040	4	Infeasible	4	3	6
P19	hs042	4	Infeasible	6	3	7
P20	hs043	4	feasible	9	7	6
P21	hs046	4	feasible	25	10	10
P22	hs047	5	feasible	25	17	12
P23	hs048	5	feasible	6	2	3
P24	hs049	5	feasible	69	16	10
P25	hs050	5	feasible	11	8	6
P26	hs051	5	feasible	8	2	3
P27	hs052	5	Infeasible	4	2	3
P28	hs053	5	Infeasible	5	4	4
P29	hs056	7	feasible	12	5	4
P30	hs060	3	Infeasible	7	7	5
P31	hs061	3	Infeasible	44	7	8
P32	hs063	3	Infeasible	5	5	4
P33	hs073	4	Infeasible	16	7	8
P34	hs078	5	Infeasible	4	4	5
P35	hs079	5	Infeasible	7	4	4
P36	hs080	5	Infeasible	6	5	6
P37	hs081	5	Infeasible	9	6	7
P38	hs093	6	feasible	12	6	5

where Niter refers to the number of iterations.

The numerical experiments showing that the PBTR algorithm and algorithms in [38,39] have a similar cost per iteration, and it is sufficient to compare them by only counting iterations.

The algorithm has the ability to locate the optimal solution for either feasible or infeasible initial reference points. For all problems, these algorithms achieved the same optimal solution in [37].

Figure 1 declares the number of iterations required for each problem with different methods. From Figure 2, it is clear that the PBTR algorithm has an average iteration number less than that reported in [38,39]. We used the logarithmic performance profiles proposed by [40], and our profiles are based on Niter. We observe that the PBTR algorithm performs quite well compared with the algorithms of [38,39]. The performance profile in terms of Niter is given in Figure 3 and shows a distinction for the PBTR algorithm over the algorithms of [38,39].

5.2. Engineering Application

The PBTR algorithm is also applied to the engineering application problem called the optimal design of a canal section for minimum water loss for a triangle cross-section. Much of the water utilized by mankind is utilized in irrigation (see Figure 4). Numerous diverse irrigation methods are connected over the years, but water has been continuously passed on and distributed using canals. Today, the expanded world population and the subsequent demand for advancement have obliged researchers to consider better water canal networks that classify much more water while keeping its quality. The uncertainty of the canal’s nature may cause the washout of the canals to communicate water during the duration of high flow, which may lead to the overall failure of many surface water exchequer systems. Thus, the loss of water from water system canals must be minimized. More than half of the water stock at the top of the canal is misplaced in leakage and evaporation by the time water comes to the field [41]. Leakage is the imperative portion of the total water loss. In fact, the noteworthy portion of loss comes from the evaporation; however, seepage takes place in a canal in small amounts. The correct lining may stop this seepage misfortune, but the change over the time in lining makes finding the correct lining a troublesome issue to solve.

Indeed, in spite of the fact that the evaporation loss changes with time whether it is winter or summer, conjointly, concrete lining conditions (cracks, etc.) influence the seepage misfortune; they can be estimated under certain conditions. In this manner, the design of a canal cross-section ought to be optimized considering minimization of the seepage misfortune and evaporation loss over time. This study aimed to minimize the cross-section area of the triangle cross-section of the current canal section problem concerning water losses. This application has been studied by many authors, see [42,43,44,45,46].

Problem formulation The optimal canal section problem for the triangle cross-section is formulated as follows

$$\begin{array}{ll}minimize\hfill & F={10}^{-6}{x}_2{[{(4\pi -{\pi }^2)}^{0.77}+{(2x_1)}^{1.5}]}^{0.77}+5\times {10}^{-6}{x}_1{x}_2\hfill \\ \hfill & \hfill \\ subject to\hfill & 1.737\frac{x_1^{1.5}{x}_2^{2.5}}{{(1+x_1^2)}^{0.25}}\ln \left(\frac{x_4{(1+x_1^2)}^{0.5}}{6x_1{x}_2}+0.625x_3\frac{{(1+x_1^2)}^{0.75}}{{(x_1{x}_2)}^{1.5}}\right)=-1,\hfill \\ \hfill & \hfill \\ \hfill & {10}^{-7}\le x_3\le {10}^{-5},\hfill \\ \hfill & {10}^{-6}\le x_4\le {10}^{-3},\hfill \end{array}$$

where $F$ is the cross-section area of the triangle cross-section, $x_1$ is the side slope of the canal (m), and $x_2$ is the normal depth of the canal (m). The equality constraint is called the resistance equation, and it is the main constraint of this model. The variables $x_3$ and $x_4$ represent the average velocity (m/s) and the average roughness height of canal lining (m). The ranges of some variables $x_3$ and $x_4$ are reported in [41].

A minimum seepage loss of the rectangular sections is calculated by using the PBTR algorithm, and the results are given in the last column of

Table 2. Comparison of methods in [44,46] with the PBTR algorithm.

Name	Method (GA) [44]	Method (SQP) [44]	Method [46]	Algorithm (PBTR)
Best F	1.699 × 10−5	1.699 × 10−5	1.885 × 10−5	1.643 × 10−5
$x_1$	0.622	0.617	0.520	0.606
$x_2$	2.885	2.899	3.203	2.976
$x_3$	1.932	1.928	1.874	1.918
max/min constrain	8.49 × 10−8	8.53 × 10−8	8.86 × 10−8	8.23 × 10−8
No. of iteration	175	27	64	12

. The results of our approach are compared with the optimum sizes of the channels found in [44,46]. Two approaches are used in [44] to solve this problem, the genetic algorithm (GA) and the sequential quadratic programming technique (SQP). From the results, it is clear that our algorithm and algorithms [44,45,46] have a similar cost per iteration, and it is sufficient to compare them by only counting iterations. That is, the PBTR algorithm is much faster than previous algorithms. It generally has given better results than previous studies.

We offered the numerical results of the PBTR algorithm, which have been performed on a laptop with 8 GB RAM, usb 3 (10×), nvidia GEFORCE GT, and Intel inside Core (TM)i7-2670QM CPU 2.2 GHz. The PBTR algorithm was run using MATLAB (R2013a)(8.2.0.701)64-bit(win64).

Given a starting point $x_0>0$, the following parameter setting is used: ${\delta }_{min}={10}^{-3}$, ${\delta }_0=max\left(\parallel d_0^{cp}\parallel ,{\delta }_{min}\right)$, ${\delta }_{max}={10}^3{\delta }_0$, ${\beta }_1={10}^{-4}$, ${\beta }_2=0.75$, ${\alpha }_1=0.5$, ${\alpha }_2=2$, $ϵ={10}^{-7}$, ${\epsilon }_0={10}^{-8}$, and ${\epsilon }_1={10}^{-10}$.

6. Concluding Remarks

We described the penalty–barrier trust-region PBTR algorithm to solve the nonlinear programming problem. We used the penalty method with the barrier method to transform the CNP problem into an unconstrained optimization problem. Newton’s method was applied to the barrier Karush–Kuhn–Tucker conditions. From the bade side of Newton’s method, it may not converge at all if the starting point is far away from the solution. The trust-region approach is a very successful approach to ensure global convergence from any starting point. The trust-region technique can induce strongly global convergence, which is a very important technique for solving optimization problems and is more robust when they deal with rounding errors. A global convergence theory of the PBTR algorithm is proven under four required assumptions.

The algorithm PBTR is performed on some of the test problems listed in [37]. A preliminary numerical experiment on the PBTR algorithm is introduced. The performance of the PBTR algorithm is reported. The numerical results show that our approach is of value and merits further investigation.

The PBTR algorithm is additionally applied to the optimal design of a canal section for minimum water loss for a triangle cross-section problem. This study examined the effectiveness of our algorithm in the design of the minimum water loss canal section. Design problems are solved for the triangle canal section using our algorithm, which gives reliable results in a quicker time. This algorithm should be applied to the other version of open channel problems (having different geometries such as parabolic sections or considering cost as an objective).

There are many questions that should be answered for future work:

• Updating the PBTR algorithm to be able to treat nondifferentiation nonlinear programming problems.
• Updating the PBTR algorithm by using the nonmonotone mechanism.