A New Method for Determining the Degree of Controllability of State Variables for the LQR Problem Using the Duality Theorem

Abstract

The control performance of a dynamic system can be checked by the degree of controllability. In this work, we present a new method for determining the degree of observability of state variables for the linear quadratic optimal estimation (LQE) problem. We carried out the calculation of the degree of controllability for the linear quadratic optimal control (LQR) problem using a duality theorem. Compared with the traditional measures of controllability such as determinant, trace, and maximal eigenvalue of the inverse controllability Gramian, the proposed degree of controllability was developed for each state variable and takes into account both the controllability Gramian and the cost function. The new method is convenient to apply to LQR problem. In the numerical simulation, we determined the influence of the model parameters on the degree of controllability. Besides that, we analyzed the degree of controllability, which gives an insight into the relationship between the system model design and the control performance.

1. Introduction

“Degree of controllability” is a scalar measure of controllability to check how controllable a given system is [1,2,3]. It reflects the efficiency of the control effort.

The check of controllability only provides an answer to the binary question of whether or not a dynamical system is completely controllable. However, in practice we always need to know how controllable a given system is. Therefore, several authors proposed the concept of degree of controllability (DOC) as a measure of the controllability of a given system [1,2,3]. The degree of controllability analysis can help us choose the optimal parameters, number, and locations of actuators for a control system, which is of substantial importance in the aerospace industries to solve optimal control problems (the actuator placement in flexible spacecraft, sensor and actuator positioning for active vibration control, etc.) [2,4,5,6]. In addition, the study of the importance or lack of importance of each control variable to the various outputs of the system is useful for decoupling or model reduction in systems [2,7].

There are many definitions of the degree of controllability. The most widely-known definition is related to the minimum control input energy required to transfer any initial state $x_0$ to the origin [1,2,3,4,7]. If a system requires smaller input energy than others to regulate the system, it can be considered more controllable [3].

Kalman et al. first discussed the method for determining the degree of controllability in the work [1], where the authors considered the weighted trace of the controllability Gramian as a measure of controllability. After that, some authors proposed a series of new criteria to determine the degree of controllability and observability [2,7,8,9,10,11,12]. In [2], Muller calculated the degree of observability under the condition of the minimum energy problem. The author considered the maximum eigenvalue, trace, and determinant of the inverse Gramian matrix as a control measure. Instead of analyzing the minimum energy problem, Viswanathan used the recovery region to determine the degree of controllability. The authors chose a suitable terminal time T to solve this problem [7,9,13,14]. For modal systems, Hadman demonstrated the relationship between observation vectors, control vectors, and system eigenvectors. This approach has an obvious physical meaning and is easy to realize [10].

However, all the above-mentioned methods are based on an open-loop control system without considering the optimal feedback, and the control law does not change over time $u_k=const$. In this work, we have discussed the degree of controllability in the framework of the linear quadratic optimal control (LQR) system. We provide a degree of controllability in a closed-form for feedback control, where the control law is calculated by minimizing the cost function, $u_k=-K_k{x}_k$. We considered the efficiency of the control effort in this case as the control performance of the response process (amplitude and convergence time of the initial state $x_0$) and the cost of input energy, which can be directly reflected by the value of quadratic performance index $\mathit{P}$.

The cost function (or performance index $\mathit{P}$) plays a very important role in the LQR control scenario. First of all, it affects the control strategy (control law): we find the optimal solutions $u_k$ by minimizing the cost function $J_k$. Secondly, the cost function is also an indicator to evaluate the control performance of a given system. Ham and Brown studied the degree of observability directly from the filtering results of the linear quadratic optimal estimation (LQE) problem [11,12]. Then, we could consider the degree of controllability in LQR (linear quadratic optimal regulation) problem as a dual task. In the work of [11,12], the filtering result (ratio of final and initial covariance matrix) can indeed reflect the measure of observability in different directions, but this method does not clarify the reason for the filtering result. Meanwhile, the methods of [2,7,9,10] have not taken into account the influence of the cost function or weighting matrix $\mathit{P}$, $\mathit{Q}$, $\mathit{R}$. Baram, Goshen-Meskin, and Silva demonstrated the difference between the standard controllability and controllability with $\mathit{P}$, $\mathit{Q}$, and $\mathit{R}$ in [15,16,17]. However, they briefly discussed these differences in measures of controllability.

Therefore, in this work, we propose a new method for determining the controllability and observability, which takes into account both the influences of the initial values of ${\mathit{P}}_{\mathbf{0}}$, $\mathit{Q}$, and $\mathit{R}$ and the controllability (observability) Gramian on the $\mathit{P}$ convergence process for each state variable. This method is based on local controllability (observability) [18], explaining the relationship between the standard controllability (observability) matrix and the convergence results of $\mathit{P}$.

Furthermore, the existing criteria always use determinant, trace, and maximal eigenvalue of the inverse characteristic controllability or observability matrix as controllability measures. The proposed measures of controllability (observability) was developed for specific state variables, which allows us to get a priori information about the control performance of each state variable in advance. It is very convenient for calculation and application in practice.

Since the determination of the degree of controllability is complex and requires laborious preliminary calculations, for the purpose of effectively studying the degree of controllability, we first compose a dual system and determine its degree of observability (a simple criterion for degree of observability is proposed in this paper). After that, according to the duality theorem [19,20,21], this value is equivalent to the degree of controllability of the original system.

The prospects for future research include using the wavelet transform and fractal principle [22,23,24,25,26,27,28,29,30] as the qualitative analysis tools for the system models. The degree of observability of state variables depends on the features of the models’ description. Investigating the influence of these features on the properties of system models is a promising area of research. The study of the dynamic object was carried out by modeling the object, represented by time series, using wavelet and fractal analysis. The determination of the chaos indices and the analysis of special states were carried out using the wavelet transform. The analysis of the wavelet coefficients allows us to reveal the nonstationarity of the models and obtain the characteristics of the nonstationarity. The study of the degree of observability and controllability using these methods makes it possible to obtain a more complete connection between the qualitative characteristics of the systems under study.

The paper has the following structure: In Section 2, we present our study of the duality of recursive process $\mathit{P}$ in linear quadratic regulation (LQR) and estimation (LQE). In Section 3, we present a method for determining the degree of observability of state variables. In Section 4, using the duality theorem, the degree of controllability is calculated in a similar way by using the duality theorem. In Section 5, we determined the dependences of the degree of controllability on the parameters of model matrix and control matrix. The numerical example of inverted pendulum on a carriage shows that the degree of controllability analysis can help us predict the control performance of different state variables. As the degree of the controllability increases, the settling time of the response process becomes shorter and the amplitude of the response process decreases.

2. The Duality of Linear Quadratic Regulation (LQR) and Estimation (LQE)

Linear quadratic regulation (LQR) and linear quadratic estimation (LQE) are widely used as control and estimation methods. These methods are based on Pontryagin’s maximum principle and Bellman’s dynamic programming [31,32,33]. In 1960, Kalman found the duality theorem by solving the optimal control (LQR) and optimal estimation (LQE) problems [19,20]. Kalman proved that the filtering problem is a duality of the noise-free regulator problem. The solutions of the optimal estimation problem and the optimal regulator problem are equivalent under the duality relations [19,20].

2.1. Linear Quadratic Estimation (LQE)

Let us consider a linear dynamic system

$$\left\{\begin{array}{c}{\mathit{x}}_k={\mathit{F}}_{k,k-1}{\mathit{x}}_{k-1}+{\mathit{w}}_k\hfill \\ {\mathit{z}}_k={\mathit{H}}_k{\mathit{x}}_k+{\mathit{v}}_k\hfill \end{array}\right.$$

(1)

where ${\mathit{x}}_k$ is an n-state vector of the system (the components $x_k^i$ of ${\mathit{x}}_k$ are called state variables); ${\mathit{w}}_k$ is an r-Gaussian random process noise vector with zero mean and covariance matrix $\mathit{Q}=E\left[{\mathit{w}}_j{\mathit{w}}_k^T\right]$; ${\mathit{z}}_k$ is an m-measurement vector; ${\mathit{v}}_k$ is an m-Gaussian random measurement noise vector with zero mean and covariance matrix $\mathit{R}=E\left[{\mathit{v}}_j{\mathit{v}}_k^T\right]$; ${\mathit{F}}_{k,k-1}$ is an $n\times n$ transition matrix; ${\mathit{H}}_k$ is an $m\times n$ observation matrix. Moreover ${\mathit{v}}_j$ and ${\mathit{w}}_k$ are uncorrelated with each other, i.e., $E\left[{\mathit{v}}_j{\mathit{w}}_k^T\right]=0$ at any j and k.

In the LQE problem, the optimal solutions ${\mathit{P}}_k$ and ${\mathit{K}}_k$ are generated by the recursion relations [19,20]:

$$\left\{\begin{array}{c}{\widehat{\mathit{x}}}_k={\mathit{F}}_{k,k-1}{\widehat{\mathit{x}}}_{k-1}+{\mathit{K}}_k\left({\mathit{z}}_k-{\mathit{H}}_k{\mathit{F}}_{k,k-1}{\widehat{\mathit{x}}}_{k-1}\right)\hfill \\ {\mathit{P}}_k=\left({\mathit{F}}_{k,k-1}-{\mathit{K}}_k{\mathit{H}}_k\right){\mathit{P}}_{k-1}{\left({\mathit{F}}_{k,k-1}-{\mathit{K}}_k{\mathit{H}}_k\right)}^T+\mathit{Q}+{\mathit{K}}_k\mathit{R}{\mathit{K}}_k^T\hfill \\ {\mathit{K}}_k={\mathit{F}}_{k,k-1}{\mathit{P}}_k{{\mathit{H}}_k}^T{\left({\mathit{H}}_k{\mathit{P}}_k{{\mathit{H}}_k}^T+{\mathit{R}}_k\right)}^{-1}\hfill \end{array}\right.$$

(2)

where ${\widehat{\mathit{x}}}_k$ is an n-state vector estimate, ${\mathit{P}}_k$ is covariance matrix of the estimation errors ${\tilde{\mathit{x}}}_k$, ${\tilde{\mathit{x}}}_k$ is the estimation error and ${\tilde{\mathit{x}}}_k={\mathit{x}}_k-{\widehat{\mathit{x}}}_k$, ${\mathit{K}}_k$ is optimal gain matrix. ${\mathit{P}}_k$, ${\mathit{K}}_k$ are obtained iteratively by minimizing the cost function $J_k$, where $J_k={\mathit{P}}_k=E\left[{\tilde{\mathit{x}}}_k{\tilde{\mathit{x}}}_k^T\right]$.

After that, we can calculate the optimal values of ${\mathit{K}}_k$ and ${\mathit{P}}_k$ iteratively after setting the initial conditions ${\mathit{P}}_0$, $\mathit{Q}$, $\mathit{R}$.

2.2. Linear Quadratic Regulator (LQR)

Consider a dynamic system with the following form

$$\left\{\begin{array}{c}{\mathit{x}}_{k+1}={\mathit{F}}_{k+1,k}{\mathit{x}}_k+{\mathit{B}}_k{\mathit{u}}_k\hfill \\ {\mathit{u}}_k=-{\mathit{K}}_k{\mathit{x}}_k\hfill \end{array}\right.$$

(3)

where ${\mathit{x}}_k$ is an n‒state vector of the system, the control law ${\mathit{u}}_k$ is an r-vector, and ${\mathit{u}}_k=-{\mathit{K}}_k{\mathit{x}}_k$, ${\mathit{F}}_{k+1,k}$ is $n\times n$ state transition matrix, ${\mathit{B}}_k$ is $n\times r$ control matrix, ${\mathit{K}}_k$ is $r\times n$ optimal control gain matrix.

In the LQR problem, the optimal solutions ${\mathit{P}}_k$, ${\mathit{K}}_k$ are determined by the recursion relations [19,20]:

$$\left\{\begin{array}{c}{\mathit{x}}_{k+1}={\mathit{F}}_{k+1,k}{\mathit{x}}_k+{\mathit{B}}_k{\mathit{u}}_k\hfill \\ {\mathit{P}}_k={\left({\mathit{F}}_{k+1,k}-{\mathit{B}}_k{\mathit{K}}_k\right)}^T{\mathit{P}}_{k+1}\left({\mathit{F}}_{k+1,k}-{\mathit{B}}_k{\mathit{K}}_k\right)+\mathit{Q}+{\mathit{K}}_k^T\mathit{R}{\mathit{K}}_k\hfill \\ {\mathit{K}}_k={\left({\mathit{B}}_k^T{\mathit{P}}_{k+1}{\mathit{B}}_k+{\mathit{R}}_k\right)}^{-1}{\mathit{B}}_k^T{\mathit{P}}_{k+1}{\mathit{F}}_{k+1,k}\hfill \end{array}\right.$$

(4)

where ${\mathit{P}}_k$ is a matrix of quadratic form of the performance index under optimal regulation, $\mathit{Q}$ is state-cost weighting matrix, $\mathit{R}$ is control-cost weighting matrix, and ${\mathit{P}}_k$ is the $r\times n$ optimal control gain matrix and is determined by minimizing the cost function $J_k={\mathit{x}}_k^T{\mathit{P}}_k{\mathit{x}}_k=J_{k+1}+{\mathit{x}}_k^T\mathit{Q}{\mathit{x}}_k+{\mathit{u}}_k^T\mathit{R}{\mathit{u}}_k$. ${\mathit{P}}_k$, $\mathit{Q}$, and $\mathit{R}$ are positive-definite symmetric matrices. It should be noted that the physical meanings of ${\mathit{P}}_k$, $\mathit{Q}$, and $\mathit{R}$ in the LQR problem and LQE problem are different. ${\mathit{x}}_k^T\mathit{Q}{\mathit{x}}_k$ denotes the distance between the actual state and the desired state in the hyperplane. ${\mathit{u}}_k^T\mathit{R}{\mathit{u}}_k$ denotes the cost of the energy. The selection of $\mathit{Q}$ and $\mathit{R}$ will influence the performance of transient response process.

The optimal values of ${\mathit{K}}_k$ and ${\mathit{P}}_k$ can be calculated iteratively after setting the initial conditions ${\mathit{P}}_N$, $\mathit{Q}$, $\mathit{R}$, where N is the terminal time.

2.3. Duality Theorem

Consider a dynamic model in the LQR regulation problem

$${\mathit{x}}_{k+1}={\mathit{F}}_{k+1,k}{\mathit{x}}_k+{\mathit{B}}_k{\mathit{u}}_k$$

(5)

and its dual system model in the LQE estimation problem

$$\left\{\begin{array}{c}{\mathit{x}}_k={\mathit{F}}_{k,k-1}^T{\mathit{x}}_{k-1}+{\mathit{w}}_k\hfill \\ {\mathit{z}}_k={\mathit{B}}_k^T{\mathit{x}}_k+{\mathit{v}}_k\hfill \end{array}\right.$$

(6)

According to the duality theorem [19,20], if the given initial conditions ${\mathit{P}}_N$, $\mathit{Q}$, and $\mathit{R}$ in these two systems are equal, respectively, the obtained optimal solutions ${\mathit{P}}_k$, ${\mathit{K}}_k$ in system (5) and (6) will also be the same, not only for the ranks of these matrices, but also for each element in the matrices ${\mathit{P}}_k$, ${\mathit{K}}_k$.

Since the value of the optimal solution of ${\mathit{P}}_k$ and ${\mathit{K}}_k$ in the LQR and LQE of the dual system are the same, then the criteria for qualitative characteristics based on $\mathit{P}$, $\mathit{Q}$, and $\mathit{R}$ of the original system and the dual system will also be the same. That is, the degrees of observability and controllability are dual. We can demonstrate the duality of the degrees of observability and controllability in various forms, for example [4,7,10].

Next, we considered a numerical criterion for the degree of observability, which is more useful in practice. Using the criterion of the degree of observability for the analysis of state variables of the dual system, it is possible to calculate the degree of controllability of the corresponding state variables of the original system.

3. A Method for Determining the Degree of Observability of State Variables

A qualitative characteristic of the observed components of the state vector is the degree of observability. The degree of observability is an abstract characteristic, and the approach presented in [34,35,36] allows for determining which of the components of the state vector are observed better. The approach not only gives an assessment of the quality of observability of the system but also allows a comparison of the observability of the components of the state vectors of various systems.

We assume that the directly measured components of the state vector have the maximum degree of observability. The degree of observability of the directly measured component of the state vector is consider to be 1.

Assume that one component of the state vector is measured, the observation matrix ${\mathit{H}}_k=\left[\begin{array}{cccc}1& 0& \cdots & 0\end{array}\right]$.

Divide each measurement interval into n subintervals and denote the measurements $z_1,z_2,\dots ,z_n$ by the state at the first subinterval ${\mathit{x}}_1$.

$$\begin{array}{c}{z}_1=\mathit{H}{\mathit{x}}_1+v_1,\\ z_2=\mathit{H}\mathit{F}{\mathit{x}}_1+\mathit{H}{\mathit{w}}_1+v_2,\\ \dots \dots \dots \\ z_n=\mathit{H}{\mathit{F}}^{n-1}{\mathit{x}}_1+\mathit{H}{\mathit{F}}^{n-2}{\mathit{w}}_1+\cdots +\mathit{H}{\mathit{w}}_{n-1}+v_n.\end{array}$$

(7)

Or in the matrix form:

$${\mathit{z}}^{\ast }=\mathit{O}{\mathit{x}}_1+{\mathit{v}}^{\ast }$$

(8)

where

$${\mathit{z}}^{\ast }=\left[\begin{array}{c}{\mathit{z}}_1\hfill \\ {\mathit{z}}_2\hfill \\ \cdots \hfill \\ {\mathit{z}}_n\hfill \end{array}\right],\mathit{O}=\left[\begin{array}{c}\mathit{H}\\ \mathit{H}\mathit{F}\\ \cdots \\ \mathit{H}{\mathit{F}}^{n-1}\end{array}\right],{\mathit{v}}^{\ast }=\left[\begin{array}{c}{\mathit{v}}_1^{+}\\ {\mathit{v}}_2^{+}\\ \cdots \\ {\mathit{v}}_n^{+}\end{array}\right]=\left[\begin{array}{c}{v}_1\\ \mathit{H}{\mathit{w}}_1+v_2\\ \begin{array}{ccc}\cdots & \cdots & \cdots \end{array}\\ \mathit{H}{\mathit{F}}^{n-2}{\mathit{w}}_1+\cdots +\mathit{H}{\mathit{w}}_{n-1}+v_n\end{array}\right].$$

From Equation (8), we can rewrite the state vector ${\mathit{x}}_1$ as follows:

$${\mathit{x}}_1={\mathit{O}}^{-1}{\mathit{z}}^{\ast }-{\mathit{O}}^{-1}{\mathit{v}}^{\ast }$$

(9)

We introduce the concept of equivalent measurement noise. Then, according to the Equation (7), the equivalent measurement noise for an arbitrary component of the state vector has the following form:

$${\varsigma }^{\ast i}={\alpha }_{i1}{v}_1+{\alpha }_{i2}{v}_2+\cdots +{\alpha }_{in}{v}_n.$$

(10)

where ${\alpha }_{i1},{\alpha }_{i2},\dots ,{\alpha }_{in}$ are elements in ith row of matrix ${\mathit{O}}^{-1}$.

Determine the variance of the equivalent measurement noise for the ith component $R^{\ast i}$ by the coefficients ${\alpha }_{i1},{\alpha }_{i2},\dots ,{\alpha }_{in}$.

$$R^{\ast i}=E\left[{\left({\varsigma }^{\ast i}\right)}^2\right]=\left[{\alpha }_{i1}^2+{\alpha }_{i2}^2+\dots +{\alpha }_{in}^2\right]R$$

(11)

where $R=E\left[v_j{v}_k\right]$ is the variance of the measurement noise v of the directly measured component of the state vector.

We can judge a measure of observability by two characteristics: accuracy of estimation and time of convergence. Thus, the criterion for the degree of observability has the following form [34,35,36]:

$$Do{O}^i=\frac{P_0^i{R}^1}{P_0^1{R}^{\ast i}}$$

(12)

where $P_0^1$ is the initial variance of the directly measured component of the state vector. The criterion (12) may be used for any $t_0$.

The degree of observability for an arbitrary state component is a scalar in the criterion (12). Thus, the proposed criterion allows comparing the degrees of observability for a specific state component in various systems.

This criterion directly uses the ratio of P and R. If variance P of the directly measured state component converges in one step, then for the other state components P may converge in several steps. According to Shannon’s information theory, if the projection of the measurement noise ${\mathit{v}}^{\ast }$ on the ith coordinate is relatively small, then the ith state component could get more information from the measurements vector ${\mathit{z}}^{\ast }$. As a result, the estimation errors of the ith component will be faster reduced to 0. The larger the value of $\frac{P_0^i}{R^{\ast i}}$, the faster the convergence speed of performance index P. This analysis is applicable to any initial time $t_0$.

4. A Method for Determining the Degree of Controllability of State Variables

Assume, without loss of generality, that the control u is a scalar, and the control matrix $\mathit{B}$ is an $n\times 1$ matrix.

$${\mathit{x}}_{k+1}=\mathit{F}{\mathit{x}}_k+\mathit{B}{u}_k$$

(13)

The cost function has the following form

$$J={\mathit{x}}_0^T{\mathit{P}}_0{\mathit{x}}_0={\mathit{x}}_N^T{\mathit{P}}_N{\mathit{x}}_N+\sum _{k=1}^{N-1}{\mathit{x}}_k^T\mathit{Q}{\mathit{x}}_k+\sum _{k=0}^{N-1}{u}_k^{T}R{u}_k$$

(14)

Since the value of ${\mathit{P}}_k$, ${\mathit{K}}_k$ in the estimation process of the dual system and ${\mathit{P}}_k$, ${\mathit{K}}_k$ in the regulation process of the original system are the same, we can use the degree of observability (12) of the dual system to determine the degree of controllability of the original system.

The dual system for estimation problem has the following form

$$\left\{\begin{array}{c}{\mathit{x}}_{k-1}={\mathit{F}}^T{\mathit{x}}_k+{\mathit{w}}_k\hfill \\ {\mathit{z}}_{k-1}={\mathit{B}}^T{\mathit{x}}_{k-1}+{\mathit{v}}_{k-1}\hfill \end{array}\right.$$

(15)

The controllability matrix of the dual system has the form $\mathit{O}=\left[\begin{array}{c}{\mathit{B}}^T\\ {\mathit{B}}^T{\mathit{F}}^T\\ \cdots \\ {\mathit{B}}^T{\left({\mathit{F}}^{n-1}\right)}^T\end{array}\right]$, and the equivalent measurement noise is

$$R^{\ast i}=E\left[{\left({\varsigma }^{\ast i}\right)}^2\right]=\left[{\alpha }_{i1}^2+{\alpha }_{i2}^2+\cdots +{\alpha }_{in}^2\right]R^1$$

(16)

where ${\alpha }_{i1},{\alpha }_{i2},\dots ,{\alpha }_{in}$ are elements in the ith row of matrix ${\mathit{O}}^{-1}$.

Thus the degree of observability (12) of the dual system will be

$$Do{O}^i\left(dual\right)=\frac{P_N^i{R}^1}{P_N^1{R}^{\ast i}}$$

(17)

where $P_N^1$, $R^1$ are initial conditions of the estimation process in dual system.

Since the initial conditions of the dual system ${\mathit{P}}_N$, $\mathit{Q}$, R are the same, then the degree of controllability of the original system will be

$$Do{C}^i\left(original\right)=Do{O}^i\left(dual\right)=\frac{P_N^i{R}^1}{P_N^1{R}^{\ast i}}$$

(18)

where $R^1$ is initial control-cost weighting matrix for control u; $R^{\ast i}$ is the equivalent control‒cost weighting matrix of the ith component of the state vector and the value of it is the same as (17). As in Equation (17), the criterion (18) for the degree of controllability can also be used for any terminal time $t_N$.

The physical meaning of expression (18) may be explained as follows. If the projection of the control costs $R^{\ast i}$ on the ith coordinate is less, the energy consumption will be less to control the ith state component. In other words, the ith state variable is easier to control. This idea was first mentioned in [1] to evaluate the controllability of the whole system.

In summary, we proposed a new method for determining the degree of controllability of specific state variables using the duality theorem. This criterion allows for comparing the degree of controllability of state components in different systems.

5. Simulation Results and Its Discussion

In order to study the proposed degree of controllability for specific state variables, we used an example of the stabilization models of a pendulum on a carriage. We studied the value of the degree of controllability and the control performance of the system under different system model parameters.

The inverted pendulum model is shown in Figure 1.

Figure 1 introduces the following symbols: m—pendulum mass; M—carriage mass; f—force applied by the motor to the carriage through the belt; $2l$—pendulum length; l—distance from the pivot point to the center of mass of the pendulum; I—moment of inertia of the pendulum; $\theta \left(t\right)$—the deviation angle of the pendulum from the vertical, in the position of unstable equilibrium is zero, increases clockwise; $x\left(t\right)$—carriage position.

Near the equilibrium point, the nonlinear model can be linearized as

$$\begin{array}{c}(M+m)\ddot{x}+ml\ddot{\theta }\approx f;\hfill \\ lm\ddot{x}+\left(I+m{l}^2\right)\ddot{\theta }\approx lmg\theta \hfill \end{array}$$

(19)

The discrete state-space model of the system has the form

$$\begin{array}{c}{\mathit{x}}_{k+1}=\mathit{F}{\mathit{x}}_k+\mathit{B}{f}_k;\hfill \\ {\mathbf{y}}_{k+1}=\mathbf{C}{\mathit{x}}_k\hfill \end{array}$$

(20)

where $\mathit{F}$ is state transition matrix, $\mathit{B}$ is control matrix, $\mathit{C}$ is output matrix, $f_k$ is input,

$${\mathit{x}}_k=\left[\begin{array}{c}{x}_k\hfill \\ {\dot{x}}_k\hfill \\ {\theta }_k\hfill \\ {\dot{\theta }}_k\hfill \end{array}\right],\mathit{F}=\left[\begin{array}{cccc}1& \Delta T& 0& 0\\ 0& 1& \frac{m^2{l}^{2}g\Delta T}{D}& 0\\ 0& 0& 1& \Delta T\\ 0& 0& \frac{mgl\left(M+m\right)\Delta T}{D}& 1\end{array}\right],\mathit{B}=\left[\begin{array}{c}0\\ \frac{I+m{l}^2}{D}\Delta T\\ 0\\ \frac{ml}{D}\Delta T\end{array}\right],\mathbf{C}=\left[\begin{array}{cccc}1& 0& 0& 0\\ 0& 0& 1& 0\end{array}\right].$$

Here $\Delta T$ is sample time, $\Delta T=1s$ [37,38]; $D=\left|\begin{array}{cc}M+m& ml\\ ml& I+m{l}^2\end{array}\right|=\left(I+m{l}^2\right)\left(M+m\right)-m^2{l}^2$; $I=\frac{4}{3}m{l}^2$.

The degree of controllability for each component:

$$Do{C}^i=\frac{P_N^{i}R}{P_N{R}^{\ast i}}$$

(21)

where R is control‒cost weighting matrix; $R^{\ast i}$ is equivalent control‒cost weighting matrix of the ith component state; $P_N^i$ is cost function of the ith component in the state vector.

Since the controllability matrix and its inverse matrix $\mathit{O}=\left[\begin{array}{c}{\mathit{B}}^T\\ {\mathit{B}}^T{\mathit{F}}^T\\ {\mathit{B}}^T{\mathit{F}}^T{\mathit{F}}^T\\ {\mathit{B}}^T{\mathit{F}}^T{\mathit{F}}^T{\mathit{F}}^T\end{array}\right]$ and ${\mathit{O}}^{◡1}$ are difficult to express analytically, we study the degree of controllability by simulation.

From the Equation (17) it is obvious that if the value of $\frac{R}{R^{\ast i}}$ is larger, then the degree of controllability $Do{C}^i$ for the ith component will be larger. The ith state component will be easier to control. The $\frac{R}{R^{\ast i}}$ depends on the matrix parameters of $\mathit{B}$ and $\mathit{F}$. We studied the relationship between the degree of controllability $Do{C}^i$ and the parameters M, m, l, and g.

The parameters of the system model are given in

Table 1. Parameters of the system model.

Parameters	M	m	l	g
Value	0.7 kg	0.4 kg	980 $\frac{\mathrm{cm}}{{\mathrm{s}}^2}$	30 cm

.

Set the initial values of weighting matrices ${\mathit{P}}_N$, $\mathit{Q}$, R as follows

$${\mathit{P}}_N=\left[\begin{array}{cccc}1& 0& 0& 0\\ 0& 1& 0& 0\\ 0& 0& 1& 0\\ 0& 0& 0& 1\end{array}\right];\mathit{Q}={\mathbf{C}}^T\mathbf{C}=\left[\begin{array}{cccc}1& 0& 0& 0\\ 0& 0& 0& 0\\ 0& 0& 1& 0\\ 0& 0& 0& 0\end{array}\right];R=1.$$

We investigated the control performance at different parameters m and l.

First, we study the system as m changes, while the other parameters are given the values shown in Table 1. Then, we obtain the relationship between $Do{C}^i$ and m, shown in Figure 2 and Figure 3.

Figure 2 and Figure 3 show that as the the pendulum mass m increases, the value of $\frac{R}{R^{\ast i}}$ decreases, and the degree of controllability $Do{C}^i$ decreases, i.e., the state variables of the system become harder to control.

Then, we can check the control performance under different conditions (different parameters of the system model m, l):

(1) Carriage position x response process (Figure 4)

(2) Pendulum angle $\theta$ response process (Figure 5)

The simulation results confirm the relationship between the degree of controllability $Do{C}^i$ and the change in the parameter m. When m increases, the degree of controllability $Do{C}^i$ becomes smaller, the initial states of the system ${\mathit{x}}_0$ converges more slowly and the amplitude becomes larger.

If other parameters are fixed, and only l changes, then

Figure 6 and Figure 7 show that with an increase in the length of the pendulums l, the value $\frac{R}{R^{\ast i}}$ decreases, degree of controllability $Do{C}^i$ decreases, i.e., it is harder to control the components of the system state vector.

The control performance is shown in Figure 8 and Figure 9.

(1) Carriage position x response process

(2) Pendulum angle $\theta$ response process

Obviously, with a relatively small length of the rod l, the initial state ${\mathit{x}}_0$ of the system would quicker converge to 0 and the amplitude of the response process would be smaller, i.e., the system becomes easier to control. The settling time and the rise time of the system become shorter. The simulation results could be explained by real physical principles. The shorter the rod is, the easier it is to control the deviation angle of the pendulum. Consequently, the response process will converge more quickly.

6. Conclusions

In this work, we proposed a new method for determining the degree of controllability of specific state variables. The proposed method not only considers the affect of controllability/observability Gramian, but also the affect of the cost function and weighting matrix. Using this method, it will be more convenient for investigating LQR (linear quadratic optimal control) problems. We carried out a numerical simulation of the inverted pendulum on a carriage. Furthermore, we demonstrated the relationship between degree of controllability and control performance by the transient responses process at different system model parameters. We determined the influence of the model parameters on the degree of controllability, which allows us to design system models with the desired properties. The proposed criterion can compare the degree of controllability of the same component of the state vector in two different systems.