The Optimal Erection of the Inverted Pendulum

Abstract

The erection of the inverted pendulum is a classic control problem, which has appeared in several variants. One of the most challenging is the minimum-time erection of a pendulum that is mounted on a moving cart. The aim is to erect the pendulum from the ‘straight-down’ (stable equilibrium) to a ‘straight-up’ (unstable equilibrium) position in minimum time. The swing-up maneuver is usually addressed using a pre-defined control strategy, e.g., energy-based control or selecting the switching times in a bang-bang structure. The aim of this paper is to show that the minimum-time solution may have a singular arc, with the optimal control taking a bang-singular-bang form. The singular arc segment of the control law is a feedback law that is derived herein with the solution discussed. A sensitivity analysis of the solution structure is also performed by varying the model parameters. Finally, the time-optimal solution is compared with that obtained using an energy-based control strategy.

1. Introduction

The inverted pendulum is a classical prototype problem that captures the essential features of several practical problems, such as bicycle and motorcycle balancing [1], unicycle riding and balancing [2], skier and skater dynamics [3], human and humanoid balancing [4], and spacecrafts and rockets at launch [5]. The related control problems have been tackled using a number of different approaches. In [6], the tilt motion of a two-wheeled inverted pendulum is stabilized using a self-tuning proportional-integral-derivative (PID) controller. Both linear quadratic regulator (LQR) and PID control are used in [7] to control a nonlinear inverted pendulum mounted on a motor-driven cart, with the aim of reaching a given cart position and stabilizing the pendulum in the (unstable) upright position. The two control methods are designed using the linearized model. In [8], partial feedback linearization is employed on a two-wheel cart based inverted pendulum in order to design two nonlinear controllers that control its position and orientation, while keeping the tilt angle within a given range. Neural network control has also been applied with the aim of balancing the pendulum with no a priori knowledge of the dynamics [9]. In [10], three types of fuzzy control schemes are applied to a two-wheeled inverted pendulum, which is investigated both numerically and experimentally. Sliding mode control is used in [11], and compared with PID control in [12]. The PID controller shows a faster convergence time than a second-order sliding-mode regulator. Finally, ref. [13] shows the effectiveness of active disturbance rejection control in stabilizing an inverted pendulum mounted on a unmanned aerial vehicle in the presence of external disturbances and model mismatch.

The aforementioned research focuses on the stabilization of an inverted pendulum in the upright position. One of the most challenging variants of the inverted pendulum problem is the erection of a pendulum that is mounted on a horizontally movable carriage. In this problem, the pendulum is initially in a ‘straight-down’ (stable) equilibrium. The controlled horizontal motion of the carriage is used to erect the pendulum and place it in ‘straight-up’ (unstable) equilibrium. In this case, a standard controller is typically designed to ensure stability around the upright position—a swing-up controller is employed to erect the pendulum from the straight-down position. Energy-based control strategies have been widely employed in the swing-up controller design [14,15,16]. The basic idea to use the energy rate as the controlled variable in order to increase the total pendulum energy. Using this idea, rule-based fuzzy control is used; see, for example, [17,18]. Proportional-velocity controllers have been also used [19,20], with the idea of moving the cart left and right repeatedly with a PID controller, until the pendulum reaches the straight-up position. This research assumed a pre-defined control strategy for the swing-up controller, but without time requirements to perform the maneuver.

This work deals with the minimum-time erection of an inverted pendulum within a nonlinear optimal-control framework. An early treatment of this problem is tackled in Section 9.3.13 of the classic optimal control book [21] and in [22], where the solution structure is ‘assumed’ to have a bang-bang structure, with a fixed number of switching times determined by optimization. In [23], the time-optimal swing-up is considered from a theoretical perspective by using geometric control theory. The controlled variable is the cart accelerations and it is shown that, with the formulated problem, no singular arcs are possible.

In this paper, a deeper investigation of the time-optimal control problem is carried out, but without a pre-defined control structure assumed and by controlling the cart force (instead of its acceleration). In contrast to prior investigations, it is shown that a singular arc may occur under certain circumstances—under these circumstances the control structure is bang-singular-bang in form. The control law in the singular sub-arc is derived analytically and compared with the (sub-optimal) bang-bang solution. A sensitivity analysis with respect to the model parameter and initial condition is performed to highlight the circumstances under which a singular arc occurs. Finally, the optimal solution is compared with an energy-based control strategy.

A dynamic model of the system is derived in Section 2. The optimal control problem is studied in Section 3. The simulation results are presented in Section 4, with the conclusions given in Section 5.

2. Dynamic Model

The system is shown in Figure 1, which comprises a pendulum with length l and mass m that is mounted on a cart with mass M. The pendulum has a rotational freedom $\theta$ (with respect to the vertical axis), while the cart has a translational freedom y. The acceleration due to gravity is g, while there is also an external horizontal force F acting on the cart. In sum, the model have two freedoms ($\theta$ and y), and one control input F.

The equation of motion can be derived from the kinetic energy T, the potential energy V and the virtual work $\delta W$

$$\begin{array}{ccc}\hfill T& =& \frac{1}{2}M{\dot{y}}^2+\frac{1}{2}m\left({(\dot{y}+l\dot{\theta }cos\theta )}^2+{(l\dot{\theta }sin\theta )}^2\right),\hfill \end{array}$$

(1)

$$\begin{array}{ccc}\hfill V& =& -mglcos\theta ,\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hfill \end{array}$$

(2)

$$\begin{array}{ccc}\hfill \delta W& =& F\delta y.\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hfill \end{array}$$

(3)

Using Hamilton’s principle, one obtains

$$\begin{array}{c}\hfill \left[\begin{array}{cc}M+m& mlcos\theta \\ mlcos\theta & m{l}^2\end{array}\right]\left[\begin{array}{c}\ddot{y}\\ \ddot{\theta }\end{array}\right]=\left[\begin{array}{c}ml{\dot{\theta }}^{2}sin\theta +F\\ -mglsin\theta \end{array}\right],\end{array}$$

(4)

which can be reduced to

$$\begin{array}{c}\hfill \left[\begin{array}{cc}1& ϵlcos\theta \\ cos\theta & l\end{array}\right]\left[\begin{array}{c}\ddot{y}\\ \ddot{\theta }\end{array}\right]=\left[\begin{array}{c}ϵl{\dot{\theta }}^{2}sin\theta +gu\\ -gsin\theta \end{array}\right],\end{array}$$

(5)

where

$$\begin{array}{c}\hfill ϵ=\frac{m}{M+m}\mathrm{and}u=\frac{F}{g(M+m)}.\end{array}$$

(6)

Equation (5) are identical to those reported in [21], when y is in the units of l, and the accelerations are in units of g—as a consequence time is in units of $\sqrt{l/g}$.

Since most solvers cannot deal with second order differential equations, it is convenient to reduce (5) to the first-order state-space form. By introducing state variables $v_y=\dot{y}$ and $v_{\theta }=\dot{\theta }$, one obtains the following:

$$\begin{array}{c}\hfill \left[\begin{array}{c}\dot{y}\\ \dot{\theta }\\ {\dot{v}}_y\\ {\dot{v}}_{\theta }\end{array}\right]=\left[\begin{array}{c}{v}_y\\ v_{\theta }\\ \frac{(v_{\theta }^2+cos\theta )ϵsin\theta +u}{1-ϵ{cos}^2\theta }\\ -\frac{ϵv_{\theta }^{2}sin\theta cos\theta +sin\theta +ucos\theta }{1-ϵ{cos}^2\theta }\end{array}\right],\end{array}$$

(7)

which is in standard state-space form. From now on $x={[y,\theta ,v_y,v_{\theta }]}^T$ represents a vector of the scaled state variables.

3. Optimal Control Problem

The optimal control problem (OCP) is one of finding a control u, with $\left|u\right|\le u_{max}=1$, which moves the system from the stable equilibrium $y=\dot{y}=\theta =\dot{\theta }=0$ (straight-down) to the unstable equilibrium $y=\dot{y}=\dot{\theta }=0$ and $\theta =\pi$ (straight-up) in the minimum time. Note that both the initial and terminal states are specified.

The optimal control problem will be tackled in the following steps. First, the control Hamiltonian is derived, which is shown to be linear in the control u. Next, the switching function is derived (from the control Hamiltonian). Finally, under the assumption that a singular arc might exist, a feedback control law for the singular sub-arc is derived. This calculation is based on the assumption that the switching function and its various time derivatives must be zero. The control law for the singular arc is expressed both explicitly and as the a solution of a Riccati differential equations—these control laws are equivalent.

The OCP cost is simply the elapsed time

$$J=t_f.$$

(8)

The control Hamiltonian is given by

$$\mathcal{H}={\lambda }_1{v}_y+{\lambda }_2{v}_{\theta }+{\lambda }_3\frac{(v_{\theta }^2+cos\theta )ϵsin\theta +u}{1-ϵ{cos}^2\theta }-{\lambda }_4\frac{ϵv_{\theta }^{2}sin\theta cos\theta +sin\theta +ucos\theta }{1-ϵ{cos}^2\theta }$$

(9)

where $\lambda ={[{\lambda }_1,{\lambda }_2,{\lambda }_3,{\lambda }_4]}^T$ are the Lagrange multipliers. Evidently

$$\begin{array}{c}\hfill {\mathcal{H}}_u=\frac{{\lambda }_3-{\lambda }_{4}cos\theta }{1-ϵ{cos}^2\theta }.\end{array}$$

(10)

The co-state equations for the OCP are given by

$$\begin{array}{c}\hfill \dot{\lambda }=-\frac{\partial \mathcal{H}}{\partial x},\end{array}$$

(11)

which in expanded form are

$$\begin{array}{ccc}\hfill {\dot{\lambda }}_1& =& 0,\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hfill \end{array}$$

(12)

$$\begin{array}{ccc}\hfill {\dot{\lambda }}_2& =& \frac{ϵ(1-2{cos}^2\theta )({\lambda }_3-{\lambda }_4{v}_{\theta }^2)+({\lambda }_4-{\lambda }_3ϵv_{\theta }^2)cos\theta -{\lambda }_{4}usin\theta }{1-ϵ{cos}^2\theta }+\hfill \\ & & \frac{2\{[({\lambda }_3-{\lambda }_4{v}_{\theta }^2)ϵsin\theta -{\lambda }_{4}u]cos\theta -({\lambda }_4-{\lambda }_3ϵv_{\theta }^2)sin\theta +{\lambda }_{3}u\}ϵsin\theta cos\theta }{{(1-ϵ{cos}^2\theta )}^2},\hfill \end{array}$$

(13)

$$\begin{array}{ccc}\hfill {\dot{\lambda }}_3& =& -{\lambda }_1,\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hfill \end{array}$$

(14)

$$\begin{array}{ccc}\hfill {\dot{\lambda }}_4& =& -{\lambda }_2-2ϵv_{\theta }sin\theta \frac{{\lambda }_3-{\lambda }_{4}cos\theta }{1-ϵ{cos}^2\theta }.\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hfill \end{array}$$

(15)

The optimal control $u^{*}$ is given by

$$\begin{array}{c}\hfill u^{*}=\underset{u}{min}\mathcal{H}(x^{*},{\lambda }^{*},u).\end{array}$$

(16)

Since $\mathcal{H}$ is linear in the control u, the optimal control is given by

$$\begin{array}{c}\hfill u^{*}=\left\{\begin{array}{cc}-1& \mathrm{if}\Phi >0\\ \mathrm{unknown}& \mathrm{if}\Phi =0\\ 1& \mathrm{if}\Phi <0,\end{array}\right.\end{array}$$

(17)

where the switching function is

$$\begin{array}{c}\hfill \Phi ={\lambda }_3-{\lambda }_{4}cos\theta ;\end{array}$$

(18)

note that $(1-ϵ{cos}^2\theta )>0$. The control is either bang-bang, or possibly a bang-singular-bang (if $\Phi =0$ over a finite interval).

The transversality condition associated to the free final time is

$$\begin{array}{c}\hfill \mathcal{H}\left(t_f\right)=-1;\end{array}$$

(19)

see, for example, Equation (8.73) in [1]. Since $\mathcal{H}$ is not explicitly time-dependent, (19) implies that $\mathcal{H}\left(t\right)=-1$. Since the state variables are specified at both the initial and final times, there are no pre-specified boundary conditions on the Lagrange multipliers. See Equation (8.72) in [1].

Singular Arc

The optimal control on the singular arc will be computed using the high-order minimum principle [24,25]. In the case of a single control, the generalized necessary condition for optimality is

$${(-1)}^k\frac{\partial }{\partial u}\left[{\left(\frac{d}{dt}\right)}^{2k}\frac{\partial \mathcal{H}}{\partial u}\right]\ge 0,$$

(20)

which will be checked numerically a posteriori. The degree of singularity of the arc is $2k$, which corresponds to the lowest-order time derivative of ${\mathcal{H}}_u$ (or the switching function $\Phi$) in which u appears explicitly. Equation (20) reduces to the classic Legendre–Clebsch condition for $k=0$, and to the Kelley condition if $k=1$ (singular arc of degree two), which is the case of interest here.

Under the assumption that a singular arc exists, the switching function given in (18) vanishes. In order to gain access to the control, (10) has to be differentiated with respect to the time at least twice; it is well known that the first re-appearance of u in the switching function differentiation process must occur in an even-order derivative [26]. One can now calculate $\dot{\Phi }(x,\lambda )$ and $\ddot{\Phi }(x,\lambda ,u)$.

In order to enforce $\Phi (x,\lambda )=0$, $\dot{\Phi }(x,\lambda )=0$ and $\ddot{\Phi }(x,\lambda ,u)=0$, we exploit the fact that ${\lambda }_1$ is constant (c); see (12), and assemble the following system of equations:

$$\left[\begin{array}{cccc}1& 0& 0& 0\\ 0& 0& \ast & \ast \\ \ast & \ast & \ast & \ast \\ \ast & \ast & \ast & \ast \end{array}\right]\left[\begin{array}{c}{\lambda }_1\\ {\lambda }_2\\ {\lambda }_3\\ {\lambda }_4\end{array}\right]=\left[\begin{array}{c}c\\ 0\\ 0\\ 0\end{array}\right];$$

(21)

where the second, third and fourth rows in (21) correspond to $\Phi (x,\lambda )=0$, $\dot{\Phi }(x,\lambda )=0$ and $\ddot{\Phi }(x,\lambda ,u)=0$, respectively—(21) can be solved for the co-state variables. The expressions in (21) are too cumbersome to report explicitly—these calculations were carried out using a symbolic mathematics tool.

The next step is to substitute the solution of (21) into (9) using $\mathcal{H}=-1$. This gives

$$u^{*}\left(x\right)=\frac{n_0\left(x\right)+n_1\left(x\right)c}{d_0\left(x\right)+d_1\left(x\right)c},$$

(22)

where

$$\begin{array}{ccc}\hfill n_0& =& 2{cos}^3\theta -(1+ϵ{cos}^2\theta )cos\theta \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hfill \end{array}$$

$$\begin{array}{ccc}& & +2(1-2ϵ{cos}^2\theta )v_{\theta }^2-(1-3ϵ{cos}^2\theta )v_{\theta }^2{cos}^2\theta ,\hfill \\ \hfill n_1& =& [2(1-2ϵ{cos}^2\theta )-(1-3ϵ{cos}^2\theta ){cos}^2\theta ]v_{\theta }^2{v}_y\hfill \\ & & -[1-(2-ϵ){cos}^2\theta ]v_{y}cos\theta +[1-ϵ(2-{cos}^2\theta )]v_{\theta }^{2}cos\theta \hfill \end{array}$$

(23)

$$\begin{array}{ccc}& & +[2(2-ϵ{cos}^2\theta ){cos}^2\theta -(3-ϵ{cos}^2\theta )]v_{\theta }\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hfill \end{array}$$

(24)

$$\begin{array}{ccc}\hfill d_0& =& 2sin\theta {cos}^2\theta ,\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hfill \end{array}$$

(25)

$$\begin{array}{ccc}\hfill d_1& =& 2(v_{\theta }+v_{y}cos\theta )sin\theta cos\theta .\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hfill \end{array}$$

(26)

Since the denominator of (22) is given by

$$d_0\left(x\right)+d_1\left(x\right)c=sin\left(2\theta \right)[cos\theta +c\left((v_{\theta }+v_{y}cos\theta )\right],$$

the solution of (22) is singular when $\theta =k\frac{\pi }{2}$ ($k\in \mathbb{Z}$). The constant c in (22) is a free parameter to be optimised. For this problem, the condition (20) along the singular arc (i.e., when enforcing $\Phi =0$) is given by

$$\frac{{\lambda }_{4}sin\left(2\theta \right)}{{(1-ϵcos\theta )}^2}\ge 0,$$

(27)

which can be checked numerically a posteriori.

For numerical reasons, it may be prudent to describe the optimal control in differential form. To do this, we replace the first row of (21) with $\stackrel{⃛}{\Phi }(x,\lambda ,u,\dot{u})=0$ to obtain

$$\left[\begin{array}{cccc}\ast & \ast & \ast & \ast \\ 0& 0& \ast & \ast \\ \ast & \ast & \ast & \ast \\ \ast & \ast & \ast & \ast \end{array}\right]\left[\begin{array}{c}{\lambda }_1\\ {\lambda }_2\\ {\lambda }_3\\ {\lambda }_4\end{array}\right]=\left[\begin{array}{c}0\\ 0\\ 0\\ 0\end{array}\right].$$

(28)

If a non-trivial solution exists, the matrix on the left-hand side of (28) is singular. Setting the determinant of this matrix to zero gives

$$\begin{array}{c}\hfill \dot{u}=a_0\left(x\right)+a_1\left(x\right)u+a_2\left(x\right)u^2,\end{array}$$

(29)

which is a Riccati equation—the state-dependent coefficients are

$$\begin{array}{ccc}\hfill a_0& =& \frac{1}{2{sin}^2\theta }\{v_{\theta }(3cos{\theta }^2+5ϵ)\hfill \\ & & +\frac{v_{\theta }^3[3(4-ϵ{cos}^2\theta )ϵ{cos}^4\theta -(1+14ϵ){cos}^2\theta +8ϵ-2]}{2(1-ϵ{cos}^2\theta )cos\theta }\hfill \\ & & +\frac{v_{\theta }[(2+3ϵ{cos}^2\theta )ϵ{cos}^2\theta -5]}{1-ϵ{cos}^2\theta }\hfill \end{array}$$

(30)

$$\begin{array}{ccc}& & -\frac{(4-ϵ)(2-ϵ){cos}^4\theta -2(5-2ϵ){cos}^2\theta +3}{2v_{\theta }(1-ϵ{cos}^2\theta )cos\theta }\}\hspace{1em}\hfill \end{array}$$

(31)

$$\begin{array}{ccc}\hfill a_1& =& \frac{1}{2sin\theta }\left[\frac{v_{\theta }(4-3{cos}^2\theta )}{cos\theta }+\frac{(8-3ϵ){cos}^2\theta -5}{v_{\theta }(1-ϵ{cos}^2\theta )}\right]\hspace{1em}\hspace{1em}\hspace{1em}\hfill \end{array}$$

(32)

$$\begin{array}{ccc}\hfill a_2& =& -\frac{2cos\theta }{v_{\theta }(1-ϵ{cos}^2\theta )}.\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hfill \end{array}$$

(33)

It is worth noting that $u^{*}$ in (22) is a solution of (29), and thus one can use either (22) or (29). The initial condition $u_0$ on (29) is given by

$$u_0={\left.\frac{n_0\left(x\right)+n_1\left(x\right)c}{d_0\left(x\right)+d_1\left(x\right)c}\right|}_{x=x\left(t_i\right)},$$

(34)

where $t_i$ is the start time of the singular arc.

In sum, the optimal control along the singular arc is given by either (22) or (29) with its initial condition (33)—in either case, c is a parameter to be optimized. The form of the $a_0$ and $a_1$ coefficients suggest that (29) will have an escape time when $\theta =k\frac{\pi }{2}$ ($k\in \mathbb{Z}$).

4. Numerical Solution

Four solution protocols are considered with $ϵ=0.5$, with the results compared against those found in [21]. The time-optimal solution is also compared with an energy-based control strategy.

4.1. Baseline

The first solution to the OCP is found numerically using a direct collocation method. The transcription of the OCP into a nonlinear program (NLP) is achieved using GPOPS-II [27], which employs a Legendre–Gauss–Radau (LGR) discretization with mesh refinement. The derivatives are computed through the automatic-differentiation (AD) tool ADIGATOR [28]. The NLP solver is IPOPT [29]. The setup includes a mesh tolerance for GPOPS-II of ${10}^{-6}$, with a error for the IPOPT solver of ${10}^{-7}$. All simulations are started with the default mesh of GPOPS-II, which consists of 10 mesh fractions and 4 collocation points per fraction. The mesh refinement method employed is the hp-PattersonRao [27].

The solution is shown in Figure 2a: the optimal control is bang ($u=+1$ for $t<0.69$), bang ($u=-1$ for $t<0.70$), singular ($t<1.17$), bang ($u=-1$ for $t<2.72$), bang ($u=+1$ for $t<4.32$) and bang ($u=-1$ for $t<4.915$). The maneuver time is $t_f^{\left(i\right)}=4.9151$ in this case. Not surprisingly, numerical issues are clearly visible along the singular arc. As a side note, the computed solution was used to verify that $\mathcal{H}\equiv -1$ for the entire simulation, and that ${\mathcal{H}}_u=0$ between 0.70 and 1.17, i.e., along the singular arc. This is possible since the co-state of the OCP, which is used to evaluate $\mathcal{H}$, can be estimated from the co-state of the associated NLP [30,31].

In the second solution, the OCP is divided in three phases: before, during and after the singular arc (as was suggested by the solution in Figure 2a). In the second phase, the optimal control is computed using (29), while the first and third bang phases are computed as before. The switching time between the phases is subject to optimization. In order to compute the singular phase, the state vector was inflated to include the control, i.e., $x^{\left(ii\right)}={[y,\theta ,v_y,v_{\theta },u]}^T$. To this end, the dynamic system (7) included (29)—the related initial condition was left free to be optimized ($u_0\approx -0.96$ was obtained in this case). The solution is shown in Figure 3a, while in the (b) part of the figure, a side view of the pendulum and cart during the optimal erection is shown; the singular-arc phase is shown in gray. The switching times obtained by the optimization process are 0.6880 s, 0.7079 s, 1.1737 s, 2.7218 s, and 4.3217 s, with the singular arc occurring between 0.7079 s and 1.1737 s. The maneuver time is again $t_f^{\left(ii\right)}=4.9151$, which is identical to $t_f^{\left(i\right)}$ (indeed, the singular control is essentially a low-pass filtered version of the oscillating control shown in the first simulation). The optimal control over the singular arc satisfies (27).

In the third solution, a pure bang-bang solution is pre-imposed, as shown in [21]. In other words, the control is only allowed to be $u=\pm 1$. Again, the GPOPS-II–ADIGATOR–IPOPT combination is employed, with the switching times as optimization parameters. In this case, Figure 2b shows the solution, which is identical to the one reported in [21], with $t_f^{\left(iii\right)}=4.9161$, which is greater than $t_f^{\left(i\right)}$ and is thus sub-optimal.

The fourth solution protocol employs a regularized control that was designed to avoid the singular arc. This is a well-known approach which is employed practically in a number of applications including the minimum-time maneuvering of road vehicles [32]. In this case, a Lagrange term is added to the performance index so that

$$J=t_f+{\int }_0^{t_f}{w}_u{u}^{2}dt,$$

(35)

where $w_u$ is a control weighting that must be chosen ‘small enough’. As a consequence, the control Hamiltonian becomes

$$\begin{array}{c}\hfill {\mathcal{H}}^{\left(2\right)}=\mathcal{H}+w_u{u}^2,\end{array}$$

(36)

which is no longer linear in u. It is now possible to solve for the control explicitly using ${\mathcal{H}}_u=0$ to obtain

$$\begin{array}{c}\hfill u^{*}=\frac{1}{2w_u}\frac{{\lambda }_3-{\lambda }_{4}cos\theta }{1-ϵ{cos}^2\theta };\end{array}$$

(37)

there is no singularity. As a general rule, $w_u$ is chosen so as not to affect significantly the solution, while avoiding the numerical issues associated to the singular arc. The new problem can be solved as in the first scenario, i.e., without dividing the problem in multiple phases. The solution is shown in Figure 4a for $w_u={10}^{-3}$ (which gives $t_f^{\left(iv\right)}\approx t_f^{\left(i\right)}$), while in (b) the solution with $w_u$ up to 0.1 are shown ($t_f$ are 4.9156 and 4.9347 for $w_u={10}^{-2}$ and $w_u={10}^{-1}$, respectively).

4.2. Sensitivity to $ϵ$

The aim of this section is to examine the control structure for different values of the mass ratio for $0\le ϵ<1$.

The $ϵ=0$ case is of little physical relevance because either the pendulum bob is massless, or the mass of the cart is infinite. In this case, the problem can be computed, with the result being little more than the limit case for small $ϵ$. The $ϵ=1$ case corresponds to a massless cart or a bob of infinite mass. In this case, numerical issues arise from singularities in (7) when $\theta =0or\pi$. Neither singularity is avoidable because they will occur, respectively, at the beginning and end of the pendulum-erection maneuver. For these reasons, these extreme cases are not considered here.

Figure 5 shows the structure of the optimal solution as a function of the model parameter $ϵ$. A hundred simulations were performed with $ϵ$ ranging from 0 to $0.98$ in steps of $0.01$, with the corresponding switching times retained. The solution has a pure bang-bang structure with four bangs in the range $0<ϵ<0.23$—no singular arc occurs when most of the system mass in concentrated in the cart. For $ϵ>0.23$, a singular arc arises at $t\approx 1.076$ s, splitting the second bang in a singular-bang sub-arc. As $ϵ$ is increased, the solution structure becomes bang-singular-bang with a lengthening singular sub-arc and with the initial bang-bang sub-arc decreasing in duration. When $ϵ>0.56$, the second bang sub-arc vanishes, and the structure becomes bang-singular-bang up to $ϵ\approx 0.98$ with an ever increasing singular arc duration. When $ϵ>0.98$, there are numerical issues relating to the denominator in (7), which terminates the investigation. The results suggest that the greater the mass of the pendulum, the larger the duration of the singular arc.

4.3. Sensitivity to Initial Conditions

Here we investigate briefly the influence of initial conditions on the optimal control solution structure. Since the cart position is a cyclic coordinate [33], the optimal solution is invariant under changes in the initial cart position. As can be seen in Figure 6, the optimal control structure does react to changes in the initial velocity condition. As the cart’s initial velocity increases, the duration of the singular arc decreases—when $\dot{y}\left(0\right)=0.2$ there is no singular arc, and the solution has a pure bang-bang structure. Changes in the pendulum’s initial angular velocity have a similar effect. When $\dot{\theta }\left(0\right)=0.1$, the solution contains a singular arc, which then vanishes for $\dot{\theta }\left(0\right)=0.2$, and a pure bang-bang structure results.

4.4. Comparison to Energy-Based Control Strategy

The aim of this section is to compare the obtained time-optimal solution with that found using an energy-based controller—see, for example, [16]— which drives the mechanical energy of the pendulum from $E_i=-ϵ$ at $t=0$ to $E_f=+ϵ$ at $t=t_f$. The employed swing-up controller is

$$u=k_1(E_f-E)sign\left(\dot{y}\right)+k_{2}sign\left(y\right)log\left(1-\frac{\left|y\right|}{y_{max}}\right),$$

(38)

while a LQR controller ensures stability around the straight-up position and is enabled when $\theta \ge 0.9\pi$, i.e., in the linear region of the pendulum. In (37), E is the mechanical energy (kinetic plus gravitational) of the pendulum, $k_1,k_2$ are the controller parameters, and $y_{max}$ is the bound for the cart position. Here $k_1=1,k_2=1.5$ are selected by trial and error to ensure quick swing-up, while $y_{max}=1$. The first term in (37) injects energy into the system (i.e., $\frac{d}{dt}(T+V)=u\dot{y}>0$), while the second term bounds the cart position within $[-y_{max},+y_{max}]$. The computed control u is saturated to $\pm u_{max}$.

Figure 7 shows the numerical solution with $ϵ=0.5$ as compared to the corresponding optimal-time solution. At the beginning, the pendulum trajectory is close to the optimal one. However, as the time increases, the solution becomes sub-optimal. At $\theta =0.9\pi$ (i.e., $t\approx 4.684$s, see the vertical solid lines), the swing-up maneuver ends and the LQR takes control of the system. At this time, the system is still moving with a non-zero velocity and the cart position is $y\ne 0$. Therefore, the LQR controller takes some time to drive the system state to the desired final values (i.e., $\theta =\pi ,y=\dot{y}=\dot{\theta }=0$). In contrast, the time-optimal control does not need a stabilizing controller at the end of the maneuver because the control is able to reach the target final state.

5. Conclusions

We have shown that the solution structure of the minimum-time inverted-pendulum erection problem is more complex than was classically believed. In particular, we have shown that the solution may contain singular sub-arcs. The duration of these sub-arcs varies with both $ϵ$ and changes in the initial velocity conditions. A pre-imposed bang-bang structure is shown to be sub-optimal. It is demonstrated that the singular control is a state feedback control that is responsive to the first problem Lagrange multiplier, which is an optimizable constant. The traditional, pragmatic and, as some think, ‘dirty’ use of a regularization Lagrange term in the cost function is shown to be effective for this problem, while also reducing considerably the complexity of the problem solution. Finally, the optimal-time solution has been compared against that obtained by using energy-based control for the swing-up maneuver, showing that the latter is sub-optimal.