Investigation of Optimization Algorithms for Neural Network Solutions of Optimal Control Problems with Mixed Constraints
Abstract
:1. Introduction
2. Necessary Optimality Conditions for OCP with Mixed Constraints
 The dynamics of the state variable;$${\dot{x}}_{i}^{*}={g}_{i},\hspace{1em}i=1,\u2025,n;$$
 Conditions of stationarity with respect to $x$ and $u$, respectively:$${\dot{\lambda}}_{i}^{*}=\frac{\partial F}{\partial {x}_{i}}{\displaystyle \sum _{i=1}^{n}\frac{\partial {g}_{i}}{\partial {x}_{i}}}{\lambda}_{i}{\displaystyle \sum _{l=1}^{p}\frac{\partial {c}_{l}}{\partial {x}_{i}}{\mu}_{l}},\hspace{1em}i=1,\u2025,n;$$$$\frac{\partial F}{\partial {u}_{s}}+{\displaystyle \sum _{i=1}^{n}\frac{\partial {g}_{i}}{\partial {u}_{s}}}{\lambda}_{i}+{\displaystyle \sum _{l=1}^{p}\frac{\partial {c}_{p}}{\partial {u}_{s}}{\mu}_{l}}=0,\hspace{1em}\mathrm{s}=1,\u2025,k;$$
 Additional nonrigidity conditions;$${\mu}_{l}\cdot {c}_{l}=0,\hspace{1em}l=1,\u2025,p;$$
 Nonnegativity condition;$${\mu}_{l}\ge 0,\hspace{1em}l=1,\u2025,p;$$
 Mixed constraints;$${c}_{l}\le 0,\hspace{1em}l=1,\u2025,p;$$
 Initial conditions;$${x}_{i}^{*}({t}_{0})={x}_{0i},\hspace{1em}i=1,\u2025,n;$$
 Transversality conditions;$${\lambda}_{i}^{*}({t}_{1})={\lambda}_{0}^{*}\cdot {\mathsf{\Phi}}_{{x}_{i}}({t}_{1},{x}^{*}({t}_{1})),\hspace{1em}i=1,\u2025,n.$$
3. Building a Neural Network Structure for Solving OCP
3.1. General Optimization Scheme for A Neural Network Solution
Algorithm 1 Optimization of a NNS  
Input: NNS,${\epsilon}_{1}$, ${\epsilon}_{2}$  
Output: ${y}^{*}$  
1  $\mathrm{put}i=0$; 
2  $\mathrm{initialize}\mathrm{weight}\mathrm{coefficients}{y}_{i}={({w}_{x},{w}_{\lambda},{w}_{\mu},{w}_{u},{b}_{x},{b}_{\lambda},{b}_{\mu},{b}_{u},{v}_{x},{v}_{\lambda},{v}_{\mu},{v}_{u})}^{T}$; 
3  $\mathrm{calculate}\mathrm{the}\mathrm{values}\mathrm{of}\mathrm{the}\mathrm{optimized}\mathrm{function}E({y}_{i})$; 
4  update weight coefficients ${y}_{i+1}={({\tilde{w}}_{x},{\tilde{w}}_{\lambda},{\tilde{w}}_{\mu},{\tilde{w}}_{u},{\tilde{b}}_{x},{\tilde{b}}_{\lambda},{\tilde{b}}_{\mu},{\tilde{b}}_{u},{\tilde{v}}_{x},{\tilde{v}}_{\lambda},{\tilde{v}}_{\mu},{\tilde{v}}_{u})}^{T}$; 
5  $\mathrm{calculate}\mathrm{the}\mathrm{values}\mathrm{of}\mathrm{the}\mathrm{optimized}\mathrm{function}E({y}_{i+1})$; 
6  if $\left\rightE({y}_{i+1})\left\right\le {\epsilon}_{1}$or$\left\rightE({y}_{i+1})E({y}_{i})\left\right\le {\epsilon}_{2}$then go to step 7; else ${y}_{i}={y}_{i+1},i=i+1$ and go to step 4; 
7  Return${y}^{*}={y}_{i+1}$. 
3.2. Genetic Optimization Algorithm
3.3. Basic Particle Swarm Algorithm
Algorithm 2 Basic Particle Swarm Algorithm  
Input: $\mathrm{NNS},\mathrm{the}\mathrm{size}\mathrm{of}\mathrm{the}\mathrm{neighborhood}\sigma N$$,\mathrm{the}\mathrm{values}\mathrm{of}\mathrm{the}\mathrm{maximum}\mathrm{influence}{\phi}_{1,\mathrm{max}}$$\mathrm{and}{\phi}_{2,\mathrm{max}}$$,\mathrm{as}\mathrm{well}\mathrm{as}\mathrm{the}\mathrm{maximum}\mathrm{speed}{v}_{\mathrm{max}}$  
Output: $b$  
1  $\mathrm{initialize}\mathrm{a}\mathrm{random}\mathrm{population}\mathrm{of}\mathrm{individuals}\left\{{x}_{i}\right\},i\in [1,N]$$,\mathrm{as}\mathrm{well}\mathrm{as}\mathrm{the}\mathrm{n}\mathrm{element}\mathrm{velocity}\mathrm{vector}\mathrm{of}\mathrm{each}\mathrm{individual}\left\{{v}_{i}\right\},i\in [1,N]$; 
2  $\mathrm{calculate}\mathrm{the}\mathrm{best}\mathrm{position}\mathrm{for}\mathrm{each}\mathrm{individual}\mathrm{so}\mathrm{far}:{b}_{i}\leftarrow {x}_{i},i\in [1,N]$; 
3  if $\left\rightE({b}_{i})\left\right\le \epsilon $then to step 13; else $\mathrm{put}i=0$ and to step 3; 
4  $\mathrm{put}{H}_{i}\leftarrow \{\sigma $$\mathrm{the}\mathrm{nearest}\mathrm{neighbors}\mathrm{for}{x}_{i}\}$; 
5  $\mathrm{put}{h}_{i}\leftarrow \mathrm{arg}{\mathrm{min}}_{x}\{f(x):x\in {H}_{i}\}$; 
6  $\mathrm{generate}\mathrm{a}\mathrm{random}\mathrm{vector}{\phi}_{1}$$,\mathrm{where}{\phi}_{1}\in U[0,{\phi}_{1,\mathrm{max}}]$; 
7  $\mathrm{generate}\mathrm{a}\mathrm{random}\mathrm{vector}{\phi}_{2}$$,\mathrm{where}{\phi}_{2}\in U[0,{\phi}_{2,\mathrm{max}}]$; 
8  $\mathrm{calculate}{v}_{i}\leftarrow {v}_{i}+{\phi}_{1}\circ ({b}_{i}{x}_{i})+{\phi}_{2}\circ ({h}_{i}{x}_{i})$; 
9  if $\left{v}_{i}\right<{v}_{\mathrm{max}}$then adjust $\mathrm{the}\mathrm{speed}{v}_{i}\leftarrow {v}_{i}{v}_{\mathrm{max}}/\left{v}_{i}\right$; 
10  $\mathrm{calculate}{x}_{i}\leftarrow {x}_{i}+{v}_{i}$; 
11  $\mathrm{calculate}{b}_{i}\leftarrow \mathrm{arg}{\mathrm{min}}_{x}\{f({x}_{i}):f({b}_{i})\}$; 
12  if $i<N$then $\mathrm{put}i=i+1$ and go to step 4; else go to step 3; 
13  Return$b=\mathrm{arg}\mathrm{min}\{f({b}_{i}),i=1,N\}$. 
3.4. Gravitational Search Algorithm
 The law of gravitation: each particle attracts others and the force of attraction between two particles is directly proportional to the product of their masses and inversely proportional to the distance between them. Note that, unlike the Universal Law of Gravitation, the square of the distance is not used, which provides the numerical algorithm with more efficient results.
 Law of motion: the current speed of any particle is equal to the sum of the part of the speed at the previous moment of time and the change in speed, which is equal to the force with which the system acts on the particle, divided by the inertial mass of the particle.
Algorithm3 Gravitational Search Algorithm  
Input: $\mathrm{NNS}$ is small constant  
Output: ${p}^{*}$  
1  $\mathrm{initialize}\mathrm{a}\mathrm{random}\mathrm{of}\mathrm{the}\mathrm{system}(\mathrm{a}\mathrm{population}\mathrm{is}\mathrm{a}\mathrm{set}\mathrm{of}\mathrm{different}\mathrm{pairs}\mathrm{of}\mathrm{weighting}\mathrm{factors})S={\{{p}_{i}^{1},{p}_{i}^{2},\dots ,{p}_{i}^{3}\}}_{i=1}^{N}$$,\mathrm{initialize}{\xi}_{i}$ are random variables uniformly distributed from zero to one; 
2  for i = 1:N do $\hspace{1em}\mathrm{calculate}\mathrm{the}\mathrm{value}\mathrm{of}\mathrm{the}\mathrm{fitness}\mathrm{function}f({p}_{i})$; end 
3  update the gravitational constant, best and worst particles, and masses for$i=1:N\mathbf{do}\phantom{\rule{0ex}{0ex}}{M}_{ai}(t)={M}_{pi}(t)={M}_{ii}(t)={M}_{i}(t)=\frac{{m}_{i}(t)}{{\displaystyle {\sum}_{j=1}^{N}{m}_{j}(t)}};$ $\phantom{\rule{0ex}{0ex}}{m}_{i}(t)=\frac{f({p}_{i})\underset{j=1,N}{\mathrm{max}}f({p}_{j})}{\underset{j=1,N}{\mathrm{min}}f({p}_{j})\underset{j=1,N}{\mathrm{max}}f({p}_{j})};$ end 
4  $\mathrm{calculate}\mathrm{the}\mathrm{value}\mathrm{of}\mathrm{the}\mathrm{gravitational}\mathrm{constant}:\phantom{\rule{0ex}{0ex}}G(t)=\frac{{G}_{0}}{{e}^{\beta t}},\beta 0;$ 
5  $\mathrm{calculation}\mathrm{of}\mathrm{the}\mathrm{resulting}\mathrm{force}\mathrm{in}\mathrm{different}\mathrm{directions}:\phantom{\rule{0ex}{0ex}}\mathbf{for}i=1:N\mathbf{do}\mathbf{}\phantom{\rule{0ex}{0ex}}\hspace{1em}\mathbf{for}j=1:N\mathbf{do}\mathbf{}$ $\hspace{1em}\hspace{1em}\mathbf{if}\mathbf{}j!=i\mathbf{then}\phantom{\rule{0ex}{0ex}}\hspace{1em}\hspace{1em}\hspace{1em}{F}_{ij}(t)=G(t)\frac{{M}_{pi}(t)\times {M}_{aj}(t)}{\left\right{p}_{i},{p}_{j}\left\right+\epsilon}({p}_{j}(t){p}_{i}(t));$ $\hspace{1em}\hspace{1em}\hspace{1em}{F}_{i}(t)+={\xi}_{i}{F}_{ij}(t)$ end end 
6  $\mathrm{calculation}\mathrm{of}\mathrm{accelerations}\mathrm{and}\mathrm{speeds}:\phantom{\rule{0ex}{0ex}}\mathbf{for}i=1:N\mathbf{do}\phantom{\rule{0ex}{0ex}}\begin{array}{l}\hspace{1em}{a}_{i}(t)={M}_{ii}(t);\hfill \\ \hspace{1em}{v}_{i}(t+1)={({\varsigma}_{1},\u2025,{\varsigma}_{n})}^{T}\times {v}_{i}(t)+\frac{{F}_{i}(t)}{{a}_{i}(t)};\hfill \end{array}$ end 
7  $\mathrm{updating}\mathrm{the}\mathrm{positions}\mathrm{of}\mathrm{particles}(\mathrm{pairs}\mathrm{of}\mathrm{weighting}\mathrm{factors}):\phantom{\rule{0ex}{0ex}}\mathbf{for}i=1:N\mathbf{do}\phantom{\rule{0ex}{0ex}}\hspace{1em}{p}_{i}(t+1)={p}_{i}(t)+{v}_{i}(t+1).$ end 
8  ${p}^{*}=\mathrm{arg}\mathrm{min}\{f({p}_{i}),i=1,N\}$ 
9  if $\left\rightE({p}^{*})\left\right\le \epsilon $ then go to step 10; else go to step 2; 
10  Return${p}^{*}$. 
4. Computational Experiments
5. Conclusions and Future Work
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
 Rizaner, F.B.; Rizaner, A. Approximate Solutions of Initial Value Problems for Ordinary Differential Equations Using Radial Basis Function Networks. Neural Process. Lett. 2017, 48, 1063–1071. [Google Scholar] [CrossRef]
 Vasiliev, A.N.; Tarkhov, D.A. Neural networks as a new universal approach to numerical solution of mathematical physics problems. Neurocomput. Dev. Appl. Radiotech. 2004, 7, 111–118. [Google Scholar]
 Nazemi, A.; Karami, R. A Neural Network Approach for Solving Optimal Control Problems with Inequality Constraints and Some Applications. Neural Process. Lett. 2017, 45, 995–1023. [Google Scholar] [CrossRef]
 Effati, S.; Pakdaman, M. Optimal control problem via neural networks. Neural Comput. Appl. 2013, 23, 2093–2100. [Google Scholar] [CrossRef]
 Ruthotto, L.; Osher, S.J.; Li, W.; Nurbekyan, L.; Fung, S.W. A machine learning framework for solving highdimensional mean field game and mean field control problems. Proc. Natl. Acad. Sci. USA 2020, 117, 9183–9193. [Google Scholar] [CrossRef] [PubMed] [Green Version]
 Ghasemi, S.; Nazemi, A.R. A Neural Network Method Based on MittagLeffler Function for Solving a Class of Fractional Optimal Control Problems. AUT J. Model. Simul. 2018, 50, 211–218. [Google Scholar] [CrossRef]
 Yadav, N.; Yadav, A.; Kumar, M. An Introduction to Neural Network Methods for Differential Equations; Springer: Berlin, Germany, 2015; Volume 1, p. 114. [Google Scholar]
 Kmet, T.; Kmetova, M. Neural Networks Solving Free Final Time Optimal Control Problem; Springer: Berlin, Germany, 2012; Volume 7505, pp. 174–186. [Google Scholar]
 Kheyrinataj, F.; Nazemi, A. Fractional power series neural network for solving delay fractional optimal control problems. Connect. Sci. 2019, 53, 53–80. [Google Scholar] [CrossRef]
 Irigoyen, E.; Galvan, J.B.; PerezIlzarbe, M.J. Neural networks for constrained optimal control of nonlinear systems. In Proceedings of the IEEEINNSENNS International Joint Conference on Neural Networks (IJCNN 2000), Como, Italy, 27 July 2000; Volume 4, pp. 299–304. [Google Scholar] [CrossRef]
 Liangyu, M.; Yinping, G. Superheated steam temperature predictive optimal control based on external timedelay BP neural network and a simpler PSO algorithm. In Proceedings of the 31st Chinese Control Conference, Hefei, China, 25–27 July 2012; pp. 4050–4055. [Google Scholar]
 Li, H.; Zhang, D.J.; Lin, J.; Li, W. Application based on neural network of the boiler optimal control in power station. In Proceedings of the International Conference on Machine Learning and Cybernetics, Guangzhou, China, 18–21 August 2005; Volume 2, pp. 1192–1195. [Google Scholar] [CrossRef]
 Li, D.; Tian, X.; Ji, Z. A new numerical approach based on piecewise interpolation for solving a class of fractional optimal control problems. In Proceedings of the 35th Chinese Control Conference (CCC), Chengdu, China, 27–29 July 2016; pp. 10442–10446. [Google Scholar] [CrossRef]
 Gaiduk, A.R.; Vershinin, Y.A.; West, M.J. Neural networks and optimization problems. In Proceedings of the International Conference on Control Applications, Glasgow, UK, 18–20 September 2002; 2002; Volume 1, pp. 37–41. [Google Scholar] [CrossRef]
 Supeng, Z.; Wenxing, F.; Jun, Y.; Jianjun, L. Applying RBF neural network to missile control system parameter optimization. In Proceedings of the 2nd International Asia Conference on Informatics in Control, Automation and Robotics (CAR 2010), Wuhan, China, 6–7 March 2010; pp. 337–340. [Google Scholar] [CrossRef]
 Yildiz, A.R. Comparison of evolutionarybased optimization algorithms for structural design optimization. Eng. Appl. Artif. Intell. 2013, 26, 327–333. [Google Scholar] [CrossRef]
 Tautou, L.; Pierreval, H. Using evolutionary algorithms and simulation for the optimization of manufacturing systems. In Proceedings of the INRIA/IEEE Symposium on Emerging Technologies and Factory Automation. ETFA’95, Paris, France, 10–13 October 1995; Volume 3, pp. 509–516. [Google Scholar] [CrossRef]
 Narukawa, K.; Rodemann, T. Examining the Performance of Evolutionary ManyObjective Optimization Algorithms on a RealWorld Application. In Proceedings of the Sixth International Conference on Genetic and Evolutionary Computing, Kitakyushu, Japan, 25–28 August 2012; pp. 316–319. [Google Scholar] [CrossRef]
 Vikhar, P.A. Evolutionary algorithms: A critical review and its future prospects. In Proceedings of the International Conference on Global Trends in Signal. Processing, Information Computing and Communication, Jalgaon, India, 22–24 December 2016; pp. 261–265. [Google Scholar] [CrossRef]
 Mahammed, N.; Benslimane, S.M.; Ouldkradda, A.; Fahsi, M. Evolutionary Business Process Optimization using a MultipleCriteria Decision Analysis method. In Proceedings of the International Conference on Computer, Information and Telecommunication Systems (CITS), Colmar, France, 11–13 July 2018; pp. 1–5. [Google Scholar] [CrossRef]
 Gu, F.; Li, K.; Liu, Y. A Hybrid Evolutionary Algorithm for Solving Function Optimization Problems. In Proceedings of the 12th International Conference on Computational Intelligence and Security (CIS), Wuxi, China, 16–19 December 2016; pp. 526–529. [Google Scholar] [CrossRef]
 Wen, L.; Ximing, L. An effective hybrid evolutionary algorithm for constrained engineering optimization. In Proceedings of the Advanced Information Technology, Electronic and Automation Control. Conference, Chongqing, China, 19–20 December 2015; pp. 930–933. [Google Scholar] [CrossRef]
 Xu, Q.; Xu, Z.; Ma, T. A Short Survey and Challenges for Multiobjective Evolutionary Algorithms Based on Decomposition. In Proceedings of the International Conference on Computer, Information and Telecommunication Systems (CITS), Beijing, China, 28–31 August 2019; pp. 1–5. [Google Scholar] [CrossRef]
 Yadav, N.; Yadav, V.; Verma, P. Role of Evolutionary algorithms in Software Reliability Optimization. In Proceedings of the International Conference System Modeling & Advancement in Research Trends (SMART), Moradabad, India, 25–27 November 2016; pp. 45–48. [Google Scholar] [CrossRef]
 He, Z.; Yen, G.G.; Lv, J. Evolutionary Multiobjective Optimization with Robustness Enhancement. Proc. IEEE Trans. Evol. Comput. 2020, 24, 494–507. [Google Scholar] [CrossRef]
 Sharma, J.; Singhal, R.S. Comparative research on genetic algorithm, particle swarm optimization and hybrid GAPSO. In Proceedings of the 2nd International Conference on Computing for Sustainable Global Development (INDIACom), New Delhi, India, 11–13 March 2015; pp. 110–114. [Google Scholar]
 Li, T.; Lai, X.; Wu, M. An Improved TwoSwarm Based Particle Swarm Optimization Algorithm. In Proceedings of the 6th World Congress on Intelligent Control. and Automation, Dalian, China, 21–23 June 2006; pp. 3129–3133. [Google Scholar] [CrossRef]
 Yitong, L.; Mengyin, F.; Hongbin, G. A Modified Particle Swarm Optimization Algorithm. In Proceedings of the Chinese Control Conference, Zhangjiajie, China, 26–31 July 2007; pp. 479–483. [Google Scholar] [CrossRef]
 Sheikhpour, S.; Sabouri, M.; Zahiri, S. A hybrid Gravitational search algorithm—Genetic algorithm for neural network training. In Proceedings of the 21st Iranian Conference on Electrical Engineering (ICEE), Mashhad, Iran, 14–16 May 2013; pp. 1–5. [Google Scholar] [CrossRef]
 Li, W. An Improved Gravitational Search Algorithm for Optimization Problems. In Proceedings of the Chinese Control and Decision Conference (CCDC), Nanchang, China, 3–5 June 2019; pp. 2605–2608. [Google Scholar] [CrossRef]
Optimization Methods  Advantages  Disadvantages 

Gradient descent 


Genetic algorithm 


Gravity search 


Particle swarm optimization 


Algorithm  Parameters  Value 

Genetic algorithm  N_t—number of split points  (10, 50, 100, 200, 400) 
size—number of individuals in the population  (400, 500, 600, 800, 1000)  
pcross—percentage of individuals not subject to crossbreeding  (0.5, 0.55, 0.6, 0.65, 0.7)  
pmut—percentage of individuals subject to mutation  (0.1, 0.15, 0.2, 0.25, 0.3)  
Particle swarm optimization  N_t—number of split points  (10, 50, 100, 200, 400) 
Size—maximum number of particles in the system  (50, 100, 150)  
c0, c1—accelerations  (0.5, 1.5)  
w—inertia weight  0.75  
Gravity search  N_t—number of split points  (10, 50, 100, 200, 400) 
Size—maximum number of particles in the system  (50, 100, 150)  
G0—gravitational constant  100.0  
Alpha—coefficient of regulation of the monotony of the gravitational constant  20.0 
Algorithm  $\mathit{C}\mathit{o}\mathit{u}\mathit{n}\mathit{t}$  $\mathit{E}(\mathit{y})$  $\mathbf{\Delta}\mathit{J}(\mathit{u})$  $\mathbf{\Delta}\overline{\mathit{x}}$  $\mathbf{\Delta}\overline{\mathit{u}}$ 

Gradient descent  45,981  0.0101829  0.0004729  0.0109726  0.2697622 
Genetic algorithm  10,316  0.0347283  0.0039726  0.0286430  0.1952428 
Gravity search  12,532  0.0811294  0.0089354  0.1294026  0.1962571 
Particle swarm optimization  13,941  0.1952745  0.0298056  0.2197037  0.1681095 
Algorithm  $\mathit{T}\mathit{i}\mathit{m}\mathit{e}\mathit{I}\mathit{t}$  $\mathit{T}\mathit{i}\mathit{m}\mathit{e}$ 

Gradient descent  1.582092474  72,746.1939 
Genetic algorithm  3.195061662  39,350.3794 
Gravity search  5.603621509  53,413.7201 
Particle swarm optimization  5.162834727  46,160.9052 
Algorithm  $\mathbf{\Delta}{\overline{\mathit{x}}}_{1}$  $\mathbf{\Delta}{\overline{\mathit{x}}}_{2}$  $\mathbf{\Delta}\overline{\mathit{u}}$ 

Gradient descent  202.07342431  304.51482502  302.32155329 
Genetic algorithm  203.26250408  317.43769457  302.27943816 
Gravity search  215.19872669  319.05872786  303.42128037 
Particle swarm optimization  232.72998470  325.55120591  286.98811501 
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. 
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Bolodurina, I.; Zabrodina, L. Investigation of Optimization Algorithms for Neural Network Solutions of Optimal Control Problems with Mixed Constraints. Machines 2021, 9, 102. https://doi.org/10.3390/machines9050102
Bolodurina I, Zabrodina L. Investigation of Optimization Algorithms for Neural Network Solutions of Optimal Control Problems with Mixed Constraints. Machines. 2021; 9(5):102. https://doi.org/10.3390/machines9050102
Chicago/Turabian StyleBolodurina, Irina, and Lyubov Zabrodina. 2021. "Investigation of Optimization Algorithms for Neural Network Solutions of Optimal Control Problems with Mixed Constraints" Machines 9, no. 5: 102. https://doi.org/10.3390/machines9050102