#
H_{∞} Control for Markov Jump Systems with Nonlinear Noise Intensity Function and Uncertain Transition Rates

^{*}

## Abstract

**:**

_{∞}control is investigated for Markov jump systems with nonlinear noise intensity function and uncertain transition rates. A robust $\mathscr{H}$

_{∞}performance criterion is developed for the given systems for the first time. Based on the developed performance criterion, the desired $\mathscr{H}$

_{∞}state-feedback controller is also designed, which guarantees the robust $\mathscr{H}$

_{∞}performance of the closed-loop system. All the conditions are in terms of linear matrix inequalities (LMIs), and hence they can be readily solved by any LMI solver. Finally, a numerical example is provided to demonstrate the effectiveness of the proposed methods.

## 1. Introduction

_{∞}control for MJSs with nonlinear noise intensity function and uncertain transition rates has not been fully investigated. It still remains important and challenging.

_{∞}control for MJSs with nonlinear noise intensity function and uncertain transition rates. First, the robust $\mathscr{H}$

_{∞}performance criterion is derived. Then, $\mathscr{H}$

_{∞}controller design method is presented based on the proposed performance criterion. Instead of using the traditional Young inequality, we adopt an improved bounding technique for the uncertain terms. As a benefit, the obtained controller design method only needs to solve a set of pure linear matrix inequalities (LMIs) rather than NLMIs, which can be readily solved by any LMI solver. Finally, a numerical example is given to show the effectiveness of the proposed methods.

^{n}and ℝ

^{n×m}denote respectively the n-dimensional Euclidean space and the set of all n × m real matrices. The superscript T denotes transpose. The notation X ≥ Y (respectively, X > Y) where X and Y symmetric matrices, means that X − Y is positive semidefinite (respectively, positive definite). I denotes the identity matrix of appropriate dimension. L

_{2}[0, ∞) is the space of square integrable functions. $\left(\mathrm{\Omega},\mathcal{F},{\left\{{\mathcal{F}}_{t}\right\}}_{t\ge 0},\mathcal{P}\right)$ is a complete probability space with a filtration {$\mathcal{F}$

_{t}}

_{t}

_{≥0}satisfying the conditions that it is right continuous and $\mathcal{F}$

_{0}contains all $\mathcal{P}$−null sets. $\mathcal{E}${⋅} stands for the mathematical expectation. trace{A} denotes the trace of a matrix A. We use ∗ as an ellipsis for the terms that are introduced by symmetry. diag{·,⋯,·} stands for a block-diagonal matrix. Matrices, if their dimensions are not explicitly stated, are assumed to be compatible for algebraic operations.

## 2. Problem Description

_{2}[0, ∞], ${z}_{t}\in {\mathbb{R}}^{{n}_{z}}$ is the controlled output, $f:{\mathbb{R}}^{{n}_{x}}\times \mathbb{S}\to {\mathbb{R}}^{{n}_{x}\times {n}_{w}}$ is the nonlinear noise intensity function, and ${w}_{t}\in {\mathbb{R}}^{{n}_{w}}$ is an n

_{w}-dimention Brownian motion satisfying ℇ{dw

_{t}} = 0, ℇ{dw

_{t}(dw

_{t})

^{T}} = Idt. The mode jump process {r

_{t}} is a continuous-time, discrete-state Markov process taking values in a finite set $\mathbb{S}=\{1,2,\dots s\}$ and has mode transition probabilities:

_{Δ→0}{o(Δ)/Δ}=0, ${\widehat{\pi}}_{ij}\ge 0$ for i ≠ j, is the transition rate from mode i at time t to mode j at time t + Δ, and ${\widehat{\pi}}_{ii}=-{\displaystyle {\sum}_{j=1,j\ne i}^{s}{\widehat{\pi}}_{ij}}$ for each mode i. In addition, we assume that the Markov process {r

_{t}} is independent of the Brownian motion {w

_{t}}.

_{i}, $\forall i\in \mathbb{S}$ is a known constant matrix.

_{ij}) denotes the uncertainty of the mode transition rate matrix. For all $i,j\in \mathbb{S},j\ne i,{\pi}_{ij}(\ge 0)$ denote the estimate of ${\widehat{\pi}}_{ij}$ and $\mathrm{\Delta}{\pi}_{ij}={\widehat{\pi}}_{ij}-{\pi}_{ij}$ is the estimate error, which can take any value in [−ε

_{ij}, ε

_{ij}]; for all $i\in \mathbb{S}$, one has ${\pi}_{ij}=-{\displaystyle {\sum}_{j=1,j\ne i}^{s}{\pi}_{ij}}$ and $\mathrm{\Delta}{\pi}_{ij}=-{\displaystyle {\sum}_{j=1,j\ne i}^{s}\mathrm{\Delta}{\pi}_{ij}}$.

**Definition 1.**[19] The system (7) is said to be robustly stochastically stable if with v

_{t}= 0

**Definition 2.**[19] Given a scalar γ > 0, the system (7) is said to be robustly stochastically stable with disturbance attenuation level γ if it is robustly stochastically stable, and under zero initial conditions, ${\Vert {z}_{t}\Vert}_{{\epsilon}_{2}}^{2}\le {\gamma}^{2}{\Vert {v}_{t}\Vert}_{2}^{2}$ is satisfied for all nonzero v

_{t}∈ L

_{2}[0, ∞) and over all the admissible noise intensity functions and transition rates satisfying (4)–(5), where

**Lemma 1.**(Schur Complement [20]) Given constant matrices Ω

_{1}, Ω

_{2}and Ω

_{3}, with${\mathrm{\Omega}}_{1}={\mathrm{\Omega}}_{1}^{T}$ and$0<{\mathrm{\Omega}}_{2}={\mathrm{\Omega}}_{2}^{T}$ then${\mathrm{\Omega}}_{3}^{T}{\mathrm{\Omega}}_{2}^{-1}{\mathrm{\Omega}}_{3}-{\mathrm{\Omega}}_{1}<0$if and only if

_{t}= i) will be denoted by A

_{i}.

## 3. Robust Stochastic $\mathscr{H}$_{∞} Performance Analysis

_{∞}performance analysis problem of system (7) is considered. The following theorem gives a robust stochastic $\mathscr{H}$

_{∞}performance criterion for system (7).

**Theorem 1.**Consider the MJS (1) with nonlinear noise intensity function and uncertain transition rates. Given the controller gains K

_{i}, $i\in \mathbb{S}$, the closed-loop system (7) is robustly stochastically stable with disturbance attenuation level γ if there exist matrices P

_{i}> 0, M

_{ij}≥ 0, i, $j\in \mathbb{S}$, j ≠ i, and scalars α

_{i}, $i\in \mathbb{S}$ such that for$\forall i\in \mathbb{S}$ the following LMIs are feasible.

**Proof**. Construct a stochastic Lyapunov function candidates as

_{i}$i\in \mathbb{S}$ are symmetric positive definite matrices with appropriate dimensions to be determined.

_{t}, r

_{t}}. Then, for each r

_{t}= i, $i\in \mathbb{S}$, it can be shown that

_{ij}≥ 0, ∆π

_{ij}∈ [−ε

_{ij}, ε

_{ij}], ∀i, $j\in \mathbb{S}$, j ≠ i, and (10), we have

_{i}< 0. By the Schur complement, one can see that (8) guarantees Ψ

_{i}< 0. Thus, $\mathcal{L}V({x}_{t},i,t)+{z}_{t}^{T}{z}_{t}-{\gamma}^{2}{v}_{t}^{T}{v}_{t}\le \phantom{\rule{0.2em}{0ex}}0$ holds. Integrating both sides with respect to t over the time period [0, ∞), we have

_{0}, r

_{0}, 0) = 0 and ℇV (x

_{∞}, r

_{∞}, ∞) ≥ 0, thus (20) guarantees ${\Vert {z}_{t}\Vert}_{{\epsilon}_{2}}^{2}\le \gamma 2{\Vert {v}_{t}\Vert}_{2}^{2}$.

_{t}= 0. By following similar procedures as previously, we obtain

_{i}< 0. Thus $\mathcal{L}$V (x(t), t, i) < 0 holds for every x

_{t}≠ 0. Therefore, by Definition 2 and [21], the closed-loop system (7) is robustly stochastically stable with v

_{t}= 0. This completes the proof. □

**Remark 1.**It is well known that the uncertainty domain (5) can be formulated into a fix polytope ([8,22]), and then one can test the robust stochastic $\mathscr{H}$

_{∞}performance by solving large arrays of simultaneous LMIs corresponding to the vertices of the uncertainty polytope. However, as the number of modes increases, the number of LMIs increases exponentially. Thus it leads to a combinatoric complexity explosion ([23,24]). Therefore, the element-wise description (i.e.,(5)) for the uncertain transition rates has attracted researcher’s attention since the number of resulted LMIs increases only in function of power along with the increase of the number of modes. When using the element-wise description, a key problem is how to bound the uncertain term$\sum}_{j=1}^{s}{\widehat{\pi}}_{ij}{P}_{j$. The literature [25] considered the robust stabilization problem with the element-wise description for the uncertain transition rates. The proposed controller design method in [25] only involves LMIs, but they are usually conservative. The main reason for this is that when bounding the uncertain term$\sum}_{j=1}^{s}{\widehat{\pi}}_{ij}{P}_{j$ all transition rates are enlarged as their upper bounds so that the relationship among them is not taken into account. By utilizing the relationship among the transition rates, literature [16,17] obtained less conservative results than that of [25]. However, in the stability analysis of [16,17], the traditional Young inequality is used to bound the uncertain term$\sum}_{j=1}^{s}{\widehat{\pi}}_{ij}{P}_{j$ Such bounding technique produces the quadratic form of P

_{j}− P

_{i}(for example, in [17] the quadratic terms$({P}_{j}-{P}_{i}){T}_{ij}^{-1}({P}_{j}-{P}_{i})$ arise), which results that the obtained controller design method involves a set of NLMIs. Unfortunately, such NLMIs cannot be completely solved yet up to now [18]. Based on the bounding technique of [17], a LMI method for controller design is proposed in [26], but it is at the expense of the increase of conservatism (i.e., the method in [26] has a higher conservatism than that of [17]). From (14)–(16), one can see that our adopting bounding technique avoids the production of quadratic form of P

_{j}− P

_{i}. It is easy to prove that the bounding technique does not increase conservatism comparing with those of [16] and [17]. Moreover, as shown in next section, the obtained controller design method only involves a set of pure LMIs rather than NLMIs, which can be readily solved by any LMI solver. Therefore, the merit of the adopted bounding technique is that the obtained controller design method is not only LMI method but also not at the expense of the increase of conservatism comparing with the results in [16,17] and [26].

## 4. Robust Stochastic $\mathscr{H}$_{∞} Controller Design

_{∞}controller design problem of system (1) is considered. The following theorem is proposed to design the robust stochastic $\mathscr{H}$

_{∞}controller with the form (6) for system (1).

**Theorem 2.**Consider the MJS (1) with nonlinear noise intensity function and uncertain transition rates. The closed -loop system (7) is robustly stochastically stable with disturbance attenuation level γ if there exist matrices Q

_{i}> 0, N

_{ij}≥ 0, Y

_{i}, i, $j\in \mathbb{S}$, j ≠ i, and scalars β

_{i}, $i\in \mathbb{S}$ such that for$\forall i\in \mathbb{S}$, the following LMIs are feasible.

**Proof.**First of all, by Theorem 1, we know that system (7) with nonlinear noise intensity function and uncertain transition rates is robustly stochastically stable with disturbance attenuation level γ if inequalities (8)–(10) holds. By using the Schur complement and noticing (8)–(10) are equivalent to the following (26)–(28), respectively.

_{i}, I,⋯, I} and applying the change of variable N

_{ij}= Q

_{i}M

_{ij}Q

_{i}, Y

_{i}= K

_{i}Q

_{i}, and ${\beta}_{i}={\alpha}_{i}^{-1}$, one can obtain the inequality in (22). Performing a congruence transformation to the inequality in (27) by diag{β

_{i}I, I}, one can obtain the inequality in (23). Performing a congruence transformation to the inequality in (28) by diag{Q

_{i}, I}, one can obtain the inequality in (24). In addition, due to Y

_{i}= K

_{i}Q

_{i}, the desired controller gain is given by ${K}_{i}={Y}_{i}{Q}_{i}^{-1}$. This completes the proof. □

**Remark 2.**In view of Theorem 2, the $\mathscr{H}$

_{∞}control problem for MJSs with nonlinear noise intensity function and uncertain transition rates can be solved in terms of the feasibility of LMIs in (22)–(24). Note that the inequalities in (22)–(24) are not only linear with respect to variables Q

_{i}, N

_{ij}, Y

_{i}and β

_{i}but also linear with respect to the scalar γ

^{2}. Then, the robust $\mathscr{H}$

_{∞}controller with minimum guaranteed cost can be readily found by solving the following convex optimization problem:

^{1/2}.

## 5. Numerical Example

_{∞}controller is designed such that the resulting closed-loop system is robustly stochastically stable with disturbance attenuation level γ over all the admissible nonlinear noise intensity functions and uncertain transition rates. By solving

**COP**, the obtain minimum disturbance attenuation level is γ* = 0.5801 with the corresponding controller gain matrices

_{t}, 1) = 0.1 sin ‖x

_{t}‖ x

_{t}and f(x

_{t}, 2) = −0.2 cos ‖x

_{t}‖ x

_{t}, which obviously satisfy (29). In each simulation run, the transition rates are randomly varying but satisfy (30). To make the simulation more persuasive, simulation results with 1000 random samplings are shown in this example.

_{0}= [−1 1]

^{T}when u

_{t}= 0 and v

_{t}= 0. It is shown that the open-loop system is not stable since its state trajectories are not convergent along with the increase of time.

_{t}= cos(t)e

^{−0.1t}. Figure 3 shows the trajectories of functional cost $J({T}_{f})={\displaystyle {\int}_{0}^{{T}_{f}}\left\{{z}_{t}^{T}{z}_{t}-{\gamma}_{0}^{2}{v}_{t}^{T}{v}_{t}\right\}}dt$ under zero-initial-state condition, where γ

_{0}= 0.5801. One can see from Figure 3 that J(T

_{f}) always becomes negative along with the increase of T

_{f}in all the 1000 simulations. Therefore, the closed-loop systems achieves the desired disturbance attenuation level.

## 6. Conclusion

_{∞}control problem is investigated for MJSs with nonlinear noise intensity function and uncertain transition rates. Attention is focused on the design of a controller such that the closed-loop system is robustly stochastically stable and guarantees a desired robust $\mathscr{H}$

_{∞}performance over all the admissible noise intensity functions and transition rates. The developed method for controller design is in terms of linear matrix inequalities, which can be readily solved by any LMI solver. The effectiveness of the method is illustrated by a numerical example.

## Acknowledgments

## Author Contributions

## Conflicts of Interest

## References

- Stoica, A.; Yaesh, I. Jump-Markovian based control of wing deployment for an uncrewed air vehicle. J. Guid. Control Dyn.
**2002**, 25, 407–411. [Google Scholar] - Sworder, D.D.; Rogers, R.O. An LQG solution to a control problem with solar thermal receiver. IEEE Trans. Autom. Control
**1983**, 28, 971–978. [Google Scholar] - Ploplys, N.J.; Kawka, P.A.; Alleyne, A.G. Closed-loop control over wireless networks. IEEE Control Syst. Mag.
**2004**, 24, 58–71. [Google Scholar] - Ugrinovskii, V.A.; Pota, H.R. Decentralized control of power systems via robust control of uncertain Markov jump parameter systems. Int. J. Control
**2005**, 78, 662–677. [Google Scholar] - Svensson, L.E.O.; Williams, N. Optimal monetary policy under uncertainty: A Markov jump-linear-quadratic approach. Fed. Reserve St. Louis Rev.
**2008**, 90, 275–293. [Google Scholar] - Ji, Y.; Chizeck, H.J. Controllability, stabilizability, and continuous-time Markovian jump linear quadratic control. IEEE Trans. Autom. Control
**1990**, 35, 777–788. [Google Scholar] - Feng, X.; Loparo, K.A.; Ji, Y.; Chizeck, H.J. Stochastic stability properties of jump linear systems. IEEE Trans. Autom. Control
**1992**, 37, 38–53. [Google Scholar] - Costa, O.L.V. do Val, J.B.R.; Geromel, J.C. Continuous-time state-feedback H
_{2}-control of Markovian jump linear system via convex analysis. Automatica**1999**, 35, 259–268. [Google Scholar] - Do Val, J.B.R. Geromel, J.C.; Goncalves, A.P.C. The $\mathscr{H}$
_{2}-control for jump linear systems: Cluster observations of the Markov state. Automatica**2002**, 38, 343–349. [Google Scholar] - De Farias, D.P.; Geromel, J.C. do Val, J.B.R.; Costa, O.L.V. Output feedback control of Markov jump linear systems in continuous-time. IEEE Trans. Autom. Control
**2000**, 45, 944–949. [Google Scholar] - Costa, O.L.V.; Guerra, S. Robust linear filtering for discrete-time hybrid Markov linear systems. Int. J. Control
**2002**, 75, 712–727. [Google Scholar] - Costa, O.L.V.; Fragoso, M.D.; Todorov, M.G. Continuous-Time Markov Jump Linear Systems; Asmussen, S., Gani, J., Jagers, P., Kurtz, T.G., Eds.; Springer: Berlin/Heidelberg, Germany, 2013. [Google Scholar]
- Yue, D.; Han, Q.-L. Delay-dependent exponential stability of stochastic systems with time-varying delay, nonlinearity, and Markovian switching. IEEE Trans. Autom. Control
**2005**, 50, 217–222. [Google Scholar] - Senthilkumar, T.; Balasubramaniam, P. Delay-dependent robust stabilization and $\mathscr{H}$
_{∞}control for nonlinear stochastic systems with Markovian jump parameters and interval time-varying delays. J. Optim. Theory Appl.**2011**, 151, 100–120. [Google Scholar] - Zhang, Y. Stability of discrete-time delay Markovian jump systems with stochastic non-linearity and impulses. IET Control Theory A
**2013**, 7, 2178–2187. [Google Scholar] - Xiong, J.; Lam, J.; Gao, H.; Ho, D.W.C. On robust stabilization of Markovian jump systems with uncertain switching probabilities. Automatica
**2005**, 41, 897–903. [Google Scholar] - Xiong, J.; Lam, J. Robust $\mathscr{H}$
_{2}control of Markovian jump systems with uncertain switching probabilities. Int. J. Syst. Sci.**2009**, 40, 255–265. [Google Scholar] - Shu, Z.; Lam, J. Static output-feedback stabilization of discrete-time Markovian jump linear systems: A system augmentation approach. Automatica
**2010**, 46, 687–694. [Google Scholar] - Xu, S.; Chen, T. Robust $\mathscr{H}$
_{∞}control for uncertain stochastic systems with state delay. IEEE Trans. Autom. Control**2002**, 47, 2089–2094. [Google Scholar] - Boyd, S.; El Ghaoui, L.; Feron, E.; Balakrishnan, V. Linear Matrix Inequalities in System and Control Theory; SIAM: Pennsylvania, PA, USA, 1994. [Google Scholar]
- Mao, X. Robustness of exponential stability of stochastic differential delay equation. IEEE Trans. Autom. Control
**1996**, 41, 442–447. [Google Scholar] - El Ghaoui, L.; Rami, M.A. Robust state-feedback stabilization of jump linear systems via LMIs. Int. J. Robust Nonlinear Control
**1996**, 6, 1015–1022. [Google Scholar] - Ugrinovskii, V.A. Randomized algorithms for robust stability and guaranteed cost control of stochastic jump parameters systems with uncertain switching policies. J. Optim. Theory Appl.
**2005**, 124, 227–245. [Google Scholar] - Barmish, B.R.; Shcherbakov, P.S. On avoiding vertexization of robustness problems: the approximate feasibility concept. IEEE Trans. Autom. Control
**2002**, 47, 819–824. [Google Scholar] - Boukas, E. K.; Shi., P.; Benjelloun, K. On stabilization of uncertain linear systems with jump parameters. Int. J. Control
**1999**, 72, 842–850. [Google Scholar] - Wang, G.; Zhang, Q. Robust $\mathscr{H}$
_{∞}control of Markovian jump systems with uncertain switching probabilities. Asian J. Control**2012**, 14, 1407–1410. [Google Scholar]

© 2015 by the authors; licensee MDPI, Basel, Switzerland This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Wang, X.; Guo, Y.
*H*_{∞} Control for Markov Jump Systems with Nonlinear Noise Intensity Function and Uncertain Transition Rates. *Entropy* **2015**, *17*, 4762-4774.
https://doi.org/10.3390/e17074762

**AMA Style**

Wang X, Guo Y.
*H*_{∞} Control for Markov Jump Systems with Nonlinear Noise Intensity Function and Uncertain Transition Rates. *Entropy*. 2015; 17(7):4762-4774.
https://doi.org/10.3390/e17074762

**Chicago/Turabian Style**

Wang, Xiaonian, and Yafeng Guo.
2015. "*H*_{∞} Control for Markov Jump Systems with Nonlinear Noise Intensity Function and Uncertain Transition Rates" *Entropy* 17, no. 7: 4762-4774.
https://doi.org/10.3390/e17074762