Full Information H₂ Control of Borel-Measurable Markov Jump Systems with Multiplicative Noises

Abstract

This paper addresses an $H_2$ optimal control problem for a class of discrete-time stochastic systems with Markov jump parameter and multiplicative noises. The involved Markov jump parameter is a uniform ergodic Markov chain taking values in a Borel-measurable set. In the presence of exogenous white noise disturbance, Gramian characterization is derived for the $H_2$ norm, which quantifies the stationary variance of output response for the considered systems. Moreover, under the condition that full information of the system state is accessible to measurement, an $H_2$ dynamic optimal control problem is shown to be solved by a zero-order stabilizing feedback controller, which can be represented in terms of the stabilizing solution to a set of coupled stochastic algebraic Riccati equations. Finally, an iterative algorithm is provided to get the approximate solution of the obtained Riccati equations, and a numerical example illustrates the effectiveness of the proposed algorithm.

1. Introduction

Markov jump systems belong to a class of multimodal stochastic dynamical models with regime switching governed by a Markov chain, and have found wide-spread applications ranging from manipulator robot [1], biology of viruses [2], portfolio selection [3] to transmission control of wire/wireless network [4]. According to the number of states included in the state space of Markov chain, finite and infinite Markov jump systems are classified and investigated, respectively. After a half century’s research, the control theory for finite Markov jump systems has reached a remarkable degree of maturity, such as stability analysis [5], observability and detectability [6], linear quadratic (LQ) optimal control [7], filter design [8] and finite-time control [9]. When the state space of Markov chain is expanded to an infinite set, there will arise some properties essentially different from the finite case. For example, it was pointed out in [10] that asymptotic mean square stability, stochastic $L_2$-stability and exponential mean square stability are no more equivalent for countably infinite Markov jump systems, while these notions were proven in [11] to be equivalent for finite Markov jump systems. The technical difficulty for handling infinite Markov jumps was also exhibited in [12] which is concerned with the stability of a class of stochastic time-delay differential dynamic systems with infinite Markovian switchings. Besides the intrinsic theoretical merits, it is more often appropriate to characterize the real scenarios by infinite state space. An illustrative example is presented in [13], where the change of atmospheric conditions is modeled as a Markov chain taking values in a Borel-measurable set. By now, an increasing interest was attracted to the control issues of Markov jump systems with Markov chain taking values in a Borel set [14,15,16]. However, compared to the existing literature of finite Markov jump systems, there remain many gaps in the study of countably-infinite/Borel-measurable Markov jump systems, which deserves more attention.

$H_2$ control, based on the seminal work of state-space description [17], was at the core of robust control theory. In contrast to $H_{\infty }$ control (another popular robust control approach), $H_2$ control aims to minimize the perturbation influence on the output response caused by the exogenous additive white noise with known distribution; while $H_{\infty }$ control method deals with the disturbance attenuation problem in the presence of random disturbance whose statistical law is unclear but total energy is finite. For finite Markov jump systems, $H_2$ control problem was elaborately addressed in [18,19,20] and the references therein. In a recent paper [21], an $H_2$ control problem was studied for discrete-time Markov jump systems where the Markov chain has a Borel state space. Based on the accessible information of output variable, the optimal $H_2$ controller is obtained via an optimal filter, which produces a separation principle for the concerned dynamics. Different from the problem formulation of [21], this study focuses on a class of discrete-time stochastic systems subject to both Borel-measurable Markov jump parameter and multiplicative noises. The importance for studying these type of systems lies in the following two aspects. On one hand, it was recognized that the multiplicative noise perfectly depicts the stochastic fluctuation of physical parameter caused by uncertain environment; on the other hand, Markov chain with a Borel-measurable state space can provide substantial benefit for real applications, e.g., the networked control systems analyzed in [22]. Besides, the information of system state is assumed to be fully accessible to measurement, and the optimal $H_2$ controller will be selected among the admissible control set with a prescribed dynamic structure but unfixed dimensions to minimize the $H_2$ norm of resulting closed-loop system.

The main contribution of this paper is as follows. Firstly, we present the characterization of $H_2$ norm in terms of controllability and observability Gramians, which quantify the stationary variance of the perturbed output response. This result can be regarded as an extension of Proposition 3.1 [21] to the considered systems. It is remarkable that this formula is not only necessary to handle the considered $H_2$ optimal control problem, but also paves the way for further studying $H_2$ filter about the concerned systems. Different from the method used in [21], our technique takes full advantage of the internal stability and avoids the analysis of the asymptotic tendency about the covariance of the augmented state variable, which is generally very challenging, especially taking into account that the considered model is more complex than that of [21]. Secondly, among all $n_c$-dimensional dynamic stabilizing controllers, the optimal $H_2$ control strategy is achieved by a zero-order controller with feedback gain determined by the stabilizing solution to a set of coupled stochastic algebraic Riccati equations. This control design allows an off-line computation and hence can be readily realized in practice. Particularly, the obtained Riccati equations involve integrals over a continuous interval, which are completely different from the Riccati equations associated to finite Markov jump systems. Hence, an iterative algorithm is proposed to seek the numerical solution for these coupled Riccati equations, which can approximate the exact value of real solution according to any prescribed accuracy.

The rest of this paper is organized as follows. In Section 2, we give some preliminaries and then derive the Gramian representation of $H_2$ norm for the considered Borel-measurable Markov jump systems. Section 3 proceeds with the discussion of $H_2$ optimal control problem. In Section 4, a numerical algorithm is proposed for solving the coupled stochastic algebraic Riccati equations. Section 5 ends this paper with a concluding remark.

The notations adopted in this paper are standard. $R^n$: n-dimensional real Euclidean space; $R^{m\times n}$: the normed linear space of all m by n real matrices; $\parallel ·\parallel$: the Euclidean norm of $R^n$ or the operator norm of $R^{m\times n}$; $A^{\prime }$: the transpose of a matrix (or vector) A; $Tr\left(A\right)$: the trace of a square matrix A; $S_n$: the set of $n\times n$ real symmetric matrices; $A>0$$(\ge 0)$: A is a positive (semi-)definite symmetric matrix; $I_n$: the $n\times n$ identity matrix; ${\delta }_{(·)}$: the Kronecker functional; $\mathbb{Z}+:=\{0,1,2,\cdots \}$.

2. ${\mathbf{H}}_{\mathbf{2}}$ Norm and Gramian

On a complete probability space $(\Omega ,\mathcal{F},\mathcal{P})$, we consider the following discrete-time Markov jump system with multiplicative noises:

$${\mathbf{G}}_0\left\{\begin{array}{c}x(t+1)=A_0\left(r_t\right)x\left(t\right)+{\displaystyle \sum _{k=1}^d}{A}_k\left(r_t\right)x\left(t\right)w_k\left(t\right)+B_v\left(r_t\right)v\left(t\right),\hfill \\ z\left(t\right)=C\left(r_t\right)x\left(t\right),x\left(0\right)\in R^n,t\in {\mathbb{Z}}_{+},\hfill \end{array}\right.$$

(1)

where $x\left(t\right)\in R^n$, $z\left(t\right)\in R^{n_z}$ and $v\left(t\right)\in R^{n_v}$ represent the system state, measurement output and exogenous random disturbance, respectively. The multiplicative noise $\left\{w\left(t\right)\right|w\left(t\right)={(w_1\left(t\right),\cdots ,w_d\left(t\right))}^{\prime },t\in {\mathbb{Z}}_{+}\}$ is a stationary process satisfying $Ew\left(t\right)=0$ and $E\left[w\left(t\right)w{\left(s\right)}^{\prime }\right]=I_d{\delta }_{(t-s)}$. In (1), ${\left\{r_t\right\}}_{t\in \mathbb{Z}+}$ is a Markov chain taking values in a Borel set $\mathfrak{S}$ and having a transition probability kernel $\mathcal{G}(·,·)$ with respect to a measure $\mu$ defined on $\mathfrak{S}$. For any given $\mathcal{B}\in \sigma \left(\mathfrak{S}\right)$ (Borel $\sigma$-algebra of $\mathfrak{S}$), there exists a uniform bounded probability density $g(·,·)$ such that

$$\begin{array}{c}\hfill \mathcal{G}(r_t,\mathcal{B})=\mathcal{P}(r_{t+1}\in \mathcal{B}|r_t)=\underset{\mathcal{B}}{\int }g(r_t,s)\mu \left(ds\right).\end{array}$$

(2)

Moreover, the initial distribution of Markov chain is described by $P(r_0\in \mathcal{B})=\underset{\mathcal{B}}{\int }\nu \left(s\right)\mu \left(ds\right)$ where $\nu >0$ is absolutely integrable with respect to $\mu$. Thus, the probability density function of ${\left\{r_t\right\}}_{t\in {\mathbb{Z}}_{+}}$ is formulated as follows:

$$\begin{array}{c}\hfill \pi (0,\ell )=\nu (\ell ),\pi (t+1,\ell )=\underset{\mathfrak{S}}{\int }g(s,\ell )\pi (t,s)\mu \left(ds\right),t\in {\mathbb{Z}}_{+}.\end{array}$$

In the subsequent discussion, there is some constant $k>1$ such that $g(s,\ell )<k$ for arbitrary $s,\ell \in \mathfrak{S}$, and ${\left\{r_t\right\}}_{t\in {\mathbb{Z}}_{+}}$ is a uniform ergodic Markov chain [23]. The disturbance signal ${\left\{v\left(t\right)\right\}}_{t\in {\mathbb{Z}}_{+}}\in R^{n_v}$ is a sequence of zero mean white noises satisfying $Ev\left(t\right)=0$ and $E\left[v\left(t\right)v{\left(s\right)}^{\prime }\right]=I_{n_v}{\delta }_{(t-s)}$.

Throughout this paper, we assume that three stochastic processes ${\left\{r_t\right\}}_{t\in {\mathbb{Z}}_{+}}$, ${\left\{w\left(t\right)\right\}}_{t\in {\mathbb{Z}}_{+}}$ and ${\left\{v\left(t\right)\right\}}_{t\in {\mathbb{Z}}_{+}}$ are mutually independent. All the coefficients of system (1) belong to ${\mathbb{H}}_{\infty }^{m\times n}$ with suitable dimensions m and n, where ${\mathbb{H}}_{\infty }^{m\times n}$ is a Banach space $\{H(·):\mathfrak{S}\to R^{m\times n}|\mathrm{ess}{sup}_{\ell \in \mathfrak{S}}\parallel H(\ell )\parallel <\infty \}$ equipped with the norm ${\parallel H\parallel }_{\infty }=\mathrm{ess}{sup}_{\ell \in \mathfrak{S}}\parallel H(\ell )\parallel$. Besides, ${\mathbb{H}}_1^{m\times n}$ is also a Banach space defined by $\{H(·):\mathfrak{S}\to R^{m\times n}|{\int }_{\mathfrak{S}}\parallel H(\ell )\parallel \mu (d\ell )<\infty \}$ equipped with the norm ${\parallel H\parallel }_1:={\int }_{\mathfrak{S}}\parallel H(\ell )\parallel \mu (d\ell )$. Let ${\mathcal{F}}_k$ be the $\sigma$-algebra $\sigma \{r_t,w\left(s\right),v\left(s\right)|0$$\le t\le k,0\le s\le k-1\}$. When $k=0$, we set ${\mathcal{F}}_0=\sigma \left\{r_0\right\}$.

To make the presentation more concise, the following linear operators will be used:

$$\left\{\begin{array}{c}\mathcal{E}\left(X\right)(\ell )=\underset{\mathfrak{S}}{\int }g(\ell ,s)X\left(s\right)\mu \left(ds\right),\hfill \\ \mathcal{L}\left(Y\right)(\ell )={\displaystyle \sum _{k=0}^d}\underset{\mathfrak{S}}{\int }g(s,\ell )A_k\left(s\right)Y\left(s\right)A_k{\left(s\right)}^{\prime }\mu \left(ds\right),\hfill \\ \mathcal{T}\left(X\right)(\ell )={\displaystyle \sum _{k=0}^d}{A}_k{(\ell )}^{\prime }\mathcal{E}\left(X\right)(\ell )A_k(\ell ),\hfill \end{array}\right.$$

(3)

where $X\in {\mathbb{H}}_{\infty }^n$ and $Y\in {\mathbb{H}}_1^n$. It can be verified that $\mathcal{L}$ maps ${\mathbb{H}}_1^n$ to ${\mathbb{H}}_1^n$, while $\mathcal{E}$ and $\mathcal{T}$ map ${\mathbb{H}}_{\infty }^n$ to ${\mathbb{H}}_{\infty }^n$. Moreover, $\mathcal{L}$ and $\mathcal{T}$ are adjoint with respect to the following bilinear operator:

$$\langle X,Y\rangle =\underset{\mathfrak{S}}{\int }Tr\left[X{(\ell )}^{\prime }Y(\ell )\right]\mu (d\ell ),X\in {\mathbb{H}}_{\infty }^n,Y\in {\mathbb{H}}_1^n.$$

(4)

Definition 1

([16]). The following discrete-time stochastic system with Borel-measurable Markov jump parameter (denoted by $(\mathbb{A};\mathbb{P})$):

$$x(t+1)=A_0\left(r_t\right)x\left(t\right)+\sum _{k=1}^d{A}_k\left(r_t\right)x\left(t\right)w_k\left(t\right),t\in {\mathbb{Z}}_{+}$$

(5)

is said to be strongly exponentially mean square stable (SEMSS) if $r\left(\mathcal{L}\right)<1$, where $r\left(\mathcal{L}\right)$ indicates the spectral radius of the operator $\mathcal{L}$.

Dince all the coefficients of $\mathcal{L}$ are real matrices, by the Krein–Rutman theorem [24], there must exist a real eigenvalue $\lambda$ of $\mathcal{L}$ and a corresponding real eigenvector $0\ne X\in S_n\cap {\mathbb{H}}_1^n$ such that $r\left(\mathcal{L}\right)=\lambda$ and $\mathcal{L}\left(X\right)=\lambda X$.

The following Lyapunov-type stability theorem follows directly from Theorems 2.4, 2.5 and 2.6 of [25].

Lemma 1.

If$(\mathbb{A};\mathbb{P})$is SEMSS, then

 (i) 

for any$0\le M\in {\mathbb{H}}_{\infty }^n$, the following linear equation admits a unique$0\le U\in {\mathbb{H}}_{\infty }^n$such that

$$U(\ell )=\mathcal{T}\left(U\right)(\ell )+M(\ell ),\ell \in \mathfrak{S};$$

(6)

 (ii) 

for any$0\le N\in {\mathbb{H}}_1^n\cap {\mathbb{H}}_{\infty }^n$, the following linear equation admits a unique$0\le V\in {\mathbb{H}}_1^n\cap {\mathbb{H}}_{\infty }^n$such that

$$V(\ell )=\mathcal{L}\left(U\right)(\ell )+N(\ell ),\ell \in \mathfrak{S}.$$

(7)

Lemma 2.

Let$x^0\left(t\right)$be the state trajectory of system (1) corresponding to the initial state$x\left(0\right)=0$, then for any$\ell \in \mathfrak{S}$and$t\in {\mathbb{Z}}_{+}$, there holds

$$\begin{array}{cc}E\left[x^0(t+1)x^0{(t+1)}^{\prime }g(r_t,\ell )\right]& =\underset{\mathfrak{S}}{\int }g(s,\ell )\left\{\sum _{k=0}^d{A}_k\left(s\right)E\left[x^0\left(t\right)x^0{\left(t\right)}^{\prime }g(r_{t-1},\ell )\right]A_k{\left(s\right)}^{\prime }\right\}\mu \left(ds\right)\hfill \\ & +\underset{\mathfrak{S}}{\int }\pi (t,s)g(s,\ell )B_v\left(s\right)B_v{\left(s\right)}^{\prime }\mu \left(ds\right).\hfill \end{array}$$

(8)

Proof. 

Set ${\tilde{\mathcal{F}}}_t={\mathcal{F}}_t\vee \sigma \left(w\left(t\right)\right)\vee \sigma \left(v\left(t\right)\right)$, then it is obvious that ${\mathcal{F}}_t\subset {\tilde{\mathcal{F}}}_t$. In terms of the state equation of (1), it can be computed that

$$\begin{array}{c}\hfill E\left[x^0(t+1)x^0{(t+1)}^{\prime }g(r_t,\ell )\right]=E\left\{E\left[x^0(t+1)x^0{(t+1)}^{\prime }g(r_t,\ell )\right|{\tilde{\mathcal{F}}}_{t-1}]\right\}\\ \hfill =E\left\{E\left[A_0\left(r_t\right)x^0\left(t\right)+\sum _{k=1}^d{A}_k{\left(r_t\right)}^{\prime }{x}^0\left(t\right)w_k\left(t\right)+B_v\left(r_t\right)v\left(t\right)\right]\right[A_0\left(r_t\right)x^0\left(t\right)+\sum _{k=1}^d{A}_k{\left(r_t\right)}^{\prime }{x}^0\left(t\right)w_k\left(t\right)\\ \hfill +B_v\left(r_t\right)v\left(t\right){]}^{\prime }g(r_t,\ell )\left|{\tilde{\mathcal{F}}}_{t-1}\right\}.\end{array}$$

(9)

Because $x^0\left(t\right)$, $w_k\left(t\right)$ and $v\left(t\right)$ are all ${\tilde{\mathcal{F}}}_t$-measurable and mutually independent, it follows from (9) that

$$\begin{array}{c}E\left[x^0(t+1)x^0{(t+1)}^{\prime }g(r_t,\ell )\right]\hfill \\ =E\left\{E\left[\sum _{k=0}^d{A}_k\left(r_t\right)x^0\left(t\right)x^0{\left(t\right)}^{\prime }g(r_t,\ell )A_k{\left(r_t\right)}^{\prime }+B_v\left(r_t\right)B_v{\left(r_t\right)}^{\prime }g(r_t,\ell )|{\tilde{\mathcal{F}}}_{t-1}\right]\right\}\hfill \\ =E\underset{\mathfrak{S}}{\int }\left[\sum _{k=0}^d{A}_k\left(s\right)x^0\left(t\right)x^0{\left(t\right)}^{\prime }g(s,\ell )A_k{\left(s\right)}^{\prime }+B_v\left(s\right)B_v{\left(s\right)}^{\prime }g(s,\ell )\right]g(r_{t-1},s)\mu \left(ds\right).\hfill \end{array}$$

(10)

By Fubini’s theorem, Equation (10) can be rewritten as

$$\begin{array}{c}E\left[x^0(t+1)x^0{(t+1)}^{\prime }g(r_t,\ell )\right]\\ =\underset{\mathfrak{S}}{\int }g(s,\ell )\left\{\sum _{k=0}^d{A}_k\left(s\right)E\left[x^0\left(t\right)x^0{\left(t\right)}^{\prime }g(r_{t-1},s)\right]A_k{\left(s\right)}^{\prime }+B_v\left(s\right)B_v{\left(s\right)}^{\prime }g(s,\ell )E\left[g(r_{t-1},s)\right]\right\}\mu \left(ds\right).\end{array}$$

(11)

According to the formula (7) of [26], there stands $E\left[g(r_{t-1},s)\right]=\pi (t,s)$, which together with (11) justifies the validity of (8). The proof is ended.    □

Remark 1.

If we denote$\tilde{Y}(t,\ell )=E\left[x^0\left(t\right)x^0{\left(t\right)}^{\prime }g(r_{t-1},\ell )\right]$and$\tilde{X}(t,\ell )=\underset{\mathfrak{S}}{\int }\pi (t,s)g(s,\ell )B_v\left(s\right)$$B_v{\left(s\right)}^{\prime }\mu \left(ds\right)$, then Lemma 2 implies that$\tilde{Y}(t,\ell )$satisfies the following Lyapunov equation:

$$\begin{array}{c}\hfill \left\{\begin{array}{c}\tilde{Y}(t+1,\ell )=\mathcal{L}\left(\tilde{Y}\left(t\right)\right)(\ell )+\tilde{X}(t,\ell ),\hfill \\ \tilde{Y}(0,\ell )=0\in S_n,t\in {\mathbb{Z}}_{+},\ell \in \mathfrak{S}.\hfill \end{array}\right.\end{array}$$

(12)

By induction, it can be derived that the solution to (12) can be represented as$\tilde{Y}\left(t\right)={\displaystyle \sum _{k=0}^{t-1}}{\mathcal{L}}^{t-k-1}\left(\tilde{X}\left(k\right)\right)$for$t\ge 1$.

Next, we present the main result of this section, which led to the Gramian characterization for $H_2$ norm of system (1).

Theorem 1.

If $(\mathbb{A};\mathbb{P})$ is SEMSS, then the following equations admit a unique pair of solutions $0\le U\in {\mathbb{H}}_{\infty }^n$ and $0\le Y\in {\mathbb{H}}_{\infty }^n\cap {\mathbb{H}}_{\infty }^1$:

$$\begin{array}{c}\hfill \left\{\begin{array}{c}U(\ell )=\mathcal{T}\left(U\right)(\ell )+C{(\ell )}^{\prime }C(\ell ),\left(\mathrm{observability}\mathrm{Gramian}\right)\hfill \\ Y(\ell )=\mathcal{L}\left(Y\right)(\ell )+X(\ell ),\left(\mathrm{controllability}\mathrm{Gramian}\right),t\in {\mathbb{Z}}_{+},\ell \in \mathfrak{S},\hfill \end{array}\right.\end{array}$$

(13)

where

$$\begin{array}{c}\hfill X(\ell )=\underset{\mathfrak{S}}{\int }\pi \left(s\right)g(s,\ell )B_v\left(s\right)B_v{\left(s\right)}^{\prime }\mu \left(ds\right),\pi \left(s\right):=\underset{k\to \infty }{lim}\pi (k,s).\end{array}$$

(14)

Furthermore, for the output response $z\left(t\right)$ of system (1) corresponding to arbitrary initial state $x\left(0\right)=x_0\in R^n$, we have

$$\begin{array}{c}\hfill \underset{t\to \infty }{lim}E{\parallel z\left(t\right)\parallel }^2=\underset{\mathfrak{S}}{\int }Tr\left[C\left(s\right)Y\left(s\right)C{\left(s\right)}^{\prime }\right]\mu \left(ds\right)\end{array}$$

(15)

$$\begin{array}{c}\hfill =\underset{\mathfrak{S}}{\int }\underset{\mathfrak{S}}{\int }\pi \left(s\right)g(s,\ell )Tr\left[B_v{\left(s\right)}^{\prime }U(\ell )B_v\left(s\right)\right]\mu \left(ds\right)\mu (d\ell ).\end{array}$$

(16)

Proof. 

Above all, since ${\left\{r_t\right\}}_{t\in \mathbb{Z}+}$ is a uniform ergodic Markov chain, by Theorem 16.2.1 of [23], there exists an invariant probability density $\pi$ with the property $\pi \left(s\right)=\underset{k\to \infty }{lim}\pi (k,s)$ and $\pi (\ell )=\underset{\mathfrak{S}}{\int }\pi \left(s\right)g(s,\ell )\mu \left(ds\right)$. Hence, X is well defined in (14). Further, the existence and uniqueness of the solutions to (13) are guaranteed by Lemma 1. To proceed, it can be derived from the linearity of system (1) that

$$\begin{array}{c}\hfill z\left(t\right)=C\left(r_t\right)x\left(t\right)=C\left(r_t\right)x(t;0,v)+C\left(r_t\right)x(t;x_0,0):=z(t;0,v)+z(t;x_0,0),\end{array}$$

(17)

where $z(t;0,v)$ is the zero-initial-state output response influenced by the exogenous disturbance v and $z(t;x_0,0)$ is the unforced output response arising from the initial state $x_0$. Noticing that $x(t;0,v)$ is just $x^0\left(t\right)$ in Lemma 2, we have

$$\begin{array}{cc}{E\parallel z(t;0,v)\parallel }^2& =E\left\{Tr\left[z(t;0,v)z{(t;0,v)}^{\prime }\right]\right\}=E\left\{Tr\left[C\left(r_t\right)x^0\left(t\right)x^0{\left(t\right)}^{\prime }C{\left(r_t\right)}^{\prime }\right]\right\}\hfill \\ & =E\left\{E\left\{Tr\left[C\left(r_t\right)x^0\left(t\right)x^0{\left(t\right)}^{\prime }C{\left(r_t\right)}^{\prime }\right]\right|{\tilde{\mathcal{F}}}_{t-1}\}\right\}\hfill \\ & =E\underset{\mathfrak{S}}{\int }Tr\left[C\left(s\right)x^0\left(t\right)x^0{\left(t\right)}^{\prime }C{\left(s\right)}^{\prime }g(r_{t-1},s)\right]\mu \left(ds\right)\hfill \\ & =\underset{\mathfrak{S}}{\int }Tr\left\{C\left(s\right)E\left[x^0\left(t\right)x^0{\left(t\right)}^{\prime }g(r_{t-1},s)\right]C{\left(s\right)}^{\prime }\right\}\mu \left(ds\right).\hfill \end{array}$$

(18)

By Remark 1, it follows from (18) that

$$\begin{array}{cc}\hfill {E\parallel z(t;0,v)\parallel }^2& =\underset{\mathfrak{S}}{\int }Tr\left[C\left(s\right)\tilde{Y}(t,s)C{\left(s\right)}^{\prime }\right]\mu \left(ds\right)\hfill \\ & =\underset{\mathfrak{S}}{\int }Tr\left\{C\left(s\right)\left[\sum _{k=0}^{t-1}{\mathcal{L}}^{t-k-1}\left(\tilde{X}\left(k\right)\right)\left(s\right)\right]C{\left(s\right)}^{\prime }\right\}\mu \left(ds\right)\hfill \\ & =\underset{\mathfrak{S}}{\int }Tr\left\{C\left(s\right)\left[\sum _{k=0}^{t-1}{\mathcal{L}}^{t-k-1}\left(X\right)\left(s\right)\right]C{\left(s\right)}^{\prime }\right\}\mu \left(ds\right)\hfill \\ & +\underset{\mathfrak{S}}{\int }Tr\left\{C\left(s\right)\left[\sum _{k=0}^{t-1}{\mathcal{L}}^{t-k-1}(\tilde{X}\left(k\right)-X)\left(s\right)\right]C{\left(s\right)}^{\prime }\right\}\mu \left(ds\right),\hfill \end{array}$$

(19)

where X is given by (14). Let $Y(\ell )={\displaystyle \sum _{t=0}^{\infty }}{\mathcal{L}}^t\left(X\right)(\ell )$, then it can be verified that Y satisfies the second equation of (13). Now, taking the limit $t\to \infty$ in the first term of the last equality of (19), we get

$$\begin{array}{c}\underset{t\to \infty }{lim}\underset{\mathfrak{S}}{\int }Tr\left\{C\left(s\right)\left[\sum _{k=0}^{t-1}{\mathcal{L}}^{t-k-1}\left(X\right)\left(s\right)\right]C{\left(s\right)}^{\prime }\right\}\mu \left(ds\right)\hfill \\ =\underset{\mathfrak{S}}{\int }Tr\left\{C\left(s\right)\left[\underset{t\to \infty }{lim}\sum _{k=0}^{t-1}{\mathcal{L}}^{t-k-1}\left(X\right)\left(s\right)\right]C{\left(s\right)}^{\prime }\right\}\mu \left(ds\right)\hfill \\ =\underset{\mathfrak{S}}{\int }Tr\left\{C\left(s\right)\left[\underset{t\to \infty }{lim}\sum _{\tau =0}^{t-1}{\mathcal{L}}^{\tau }\left(X\right)\left(s\right)\right]C{\left(s\right)}^{\prime }\right\}\mu \left(ds\right)=\underset{\mathfrak{S}}{\int }Tr\left[C\left(s\right)Y\left(s\right)C{\left(s\right)}^{\prime }\right]\mu \left(ds\right).\hfill \end{array}$$

(20)

By the assumption that $(\mathbb{A};\mathbb{P})$ is SEMSS (i.e., $r\left(\mathcal{L}\right)<1$), we can infer from [27] that

$$\begin{array}{c}\hfill \parallel C\left(s\right)\left[\sum _{k=0}^{t-1}{\mathcal{L}}^{t-k-1}(\tilde{X}\left(k\right)-X)\left(s\right)\right]C{\left(s\right)}^{\prime }{\parallel }_{\infty }\le \zeta \sum _{k=0}^{t-1}{\parallel {\mathcal{L}}^{t-k-1}(\tilde{X}\left(k\right)-X)\left(s\right)\parallel }_{\infty }\\ \hfill \le \zeta \sum _{k=0}^{t-1}\lambda {\alpha }^{t-k-1}{\parallel \tilde{X}\left(k\right)-X\parallel }_{\infty },\end{array}$$

(21)

where $\zeta =\parallel C\left(r_t\right)C{\left(r_t\right)}^{\prime }{\parallel }_{\infty }>0$, $\lambda >0$ and $0<\alpha <1$. In view of $\pi \left(s\right)=\underset{k\to \infty }{lim}\pi (k,s)$, we have $\underset{k\to \infty }{lim}{\parallel \tilde{X}\left(k\right)-X\parallel }_{\infty }=0$. Thus, taking the limit $t\to \infty$ in the second term of the last equality of (19) leads to that

$$\begin{array}{c}\hfill \underset{t\to \infty }{lim}\underset{\mathfrak{S}}{\int }Tr\left\{C\left(s\right)\left[\sum _{k=0}^{t-1}{\mathcal{L}}^{t-k-1}(\tilde{X}\left(k\right)-X)\left(s\right)\right]C{\left(s\right)}^{\prime }\right\}\mu \left(ds\right)=0.\end{array}$$

(22)

Combining (19), (20) and (22), we can conclude that

$$\begin{array}{c}\hfill \underset{t\to \infty }{lim}E{\parallel z(t;0,v)\parallel }^2=\underset{\mathfrak{S}}{\int }Tr\left[C\left(s\right)Y\left(s\right)C{\left(s\right)}^{\prime }\right]\mu \left(ds\right).\end{array}$$

(23)

Moreover, because $(\mathbb{A};\mathbb{P})$ is SEMSS, it is deduced from (17) that there exist $\gamma >0$ and $0<q<1$ such that

$$\begin{array}{c}\hfill {E\parallel z\left(t\right)-z(t;0,v)\parallel }^2=E\parallel z(t;x_0,0){\parallel }^2\le \gamma q^t{\parallel x_0\parallel }^2\to 0(t\to \infty ),\end{array}$$

(24)

which implies that

$$\begin{array}{c}\hfill \underset{t\to \infty }{lim}{E\parallel z\left(t\right)\parallel }^2=\underset{t\to \infty }{lim}E{\parallel z(t;0,v)\parallel }^2=\underset{\mathfrak{S}}{\int }Tr\left[C\left(s\right)Y\left(s\right)C{\left(s\right)}^{\prime }\right]\mu \left(ds\right).\end{array}$$

(25)

Hence, Equation (15) is validated.

Next, let us show (16). To this end, we make use of the bilinear operator (4) to reformulate (15) as follows:

$$\begin{array}{c}\hfill \underset{t\to \infty }{lim}E{\parallel z\left(t\right)\parallel }^2=\underset{\mathfrak{S}}{\int }Tr\left[C\left(s\right)Y\left(s\right)C{\left(s\right)}^{\prime }\right]\mu \left(ds\right)=\langle C^{\prime }C,Y\rangle =\langle U-\mathcal{T}\left(U\right),Y\rangle \\ \hfill =\langle U,Y\rangle -\langle \mathcal{T}\left(U\right),Y\rangle =\langle U,Y\rangle -\langle U,\mathcal{L}\left(Y\right)\rangle =\langle U,Y-\mathcal{L}\left(Y\right)\rangle \\ \hfill =\langle U,X\rangle =\underset{\mathfrak{S}}{\int }\underset{\mathfrak{S}}{\int }\pi \left(s\right)g(s,\ell )Tr\left[B_v{\left(s\right)}^{\prime }U(\ell )B_v\left(s\right)\right]\mu \left(ds\right)\mu (d\ell ),\end{array}$$

(26)

where the third and seventh equalities follow from the first and second equations of (13), respectively. The proof is completed.    □

Remark 2.

In the literature of $H_2$ control theory, the stationary variance of output response caused by the exogenous white noise can serve as an $H_2$ norm of the perturbed stochastic dynamics. That is,

$$\begin{array}{c}\hfill \parallel {\mathbf{G}}_0{\parallel }_2^2=\underset{t\to \infty }{lim}E{\parallel z\left(t\right)\parallel }^2.\end{array}$$

(27)

Hence, Theorem 1 supplies a Gramian characterization for the $H_2$ norm of system (1) in the presence of additive white noise disturbance v, and makes it possible to explore $H_2$ optimal control problem for the considered systems.

3. ${\mathit{H}}_{\mathbf{2}}$ Optimal Control

In this section, we are concentrated on an $H_2$ optimal control problem for the following controlled discrete-time Borel-measurable Markov jump systems:

$$\left\{\begin{array}{c}x(t+1)=A_0\left(r_t\right)x\left(t\right)+G_0\left(r_t\right)u\left(t\right)+{\displaystyle \sum _{k=1}^d}[A_k\left(r_t\right)x\left(t\right)+G_k\left(r_t\right)u\left(t\right)]w_k\left(t\right)+B_v\left(r_t\right)v\left(t\right),\hfill \\ z\left(t\right)=\left[\begin{array}{c}C\left(r_t\right)x\left(t\right)\\ D\left(r_t\right)u\left(t\right)\end{array}\right],x\left(0\right)\in R^n,t\in {\mathbb{Z}}_{+},\hfill \end{array}\right.$$

(28)

where $u\left(t\right)\in R^{n_u}$ is the control input. Without loss of generality, we assume that $D{(\ell )}^{\prime }D(\ell )>\epsilon I_{n_u}$ ($\ell \in \mathfrak{S}$) for some $\epsilon >0$. In what follows, let us consider the following type of $n_c$-dimensional dynamic controllers:

$$\mathbf{G}\left\{\begin{array}{c}\widehat{x}(t+1)={\widehat{A}}_0\left(r_t\right)\widehat{x}\left(t\right)+{\widehat{G}}_0\left(r_t\right)\widehat{u}\left(t\right)+{\displaystyle \sum _{k=1}^d}[{\widehat{A}}_k\left(r_t\right)\widehat{x}\left(t\right)+{\widehat{G}}_k\left(r_t\right)\widehat{u}\left(t\right)]w_k\left(t\right),\hfill \\ y\left(t\right)=\widehat{C}\left(r_t\right)\widehat{x}\left(t\right)+F\left(r_t\right)\widehat{u}\left(t\right),\widehat{x}\left(0\right)\in R^{n_c},t\in {\mathbb{Z}}_{+}.\hfill \end{array}\right.$$

(29)

Through taking $u\left(t\right)=y\left(t\right)$ and $\widehat{u}\left(t\right)=x\left(t\right)$, the dynamic controller (29) is incorporated in the perturbed system (28) and the following augmented system is obtained:

$${\mathbf{G}}_c\left\{\begin{array}{c}{x}_c(t+1)=A_0^c\left(r_t\right)x_c\left(t\right)+{\displaystyle \sum _{k=1}^d}{A}_k^c\left(r_t\right)x_c\left(t\right)w_k\left(t\right)+B^c\left(r_t\right)v\left(t\right),\hfill \\ z_c\left(t\right)=C^c\left(r_t\right)x_c\left(t\right),t\in {\mathbb{Z}}_{+}.\hfill \end{array}\right.$$

(30)

In (30), the closed-loop state and the coefficients are given by

$$\begin{array}{ccc}& & x_c\left(t\right)=\left[\begin{array}{c}x\left(t\right)\\ \widehat{x}\left(t\right)\end{array}\right],A_k^c\left(r_t\right)=\left[\begin{array}{cc}{A}_k\left(r_t\right)+G_k\left(r_t\right)F\left(r_t\right)& G_k\left(r_t\right)\widehat{C}\left(r_t\right)\\ {\widehat{B}}_k\left(r_t\right)& {\widehat{A}}_k\left(r_t\right)\end{array}\right],\hfill \\ & & B^c\left(r_t\right)=\left[\begin{array}{c}{B}_v\left(r_t\right)\\ 0\end{array}\right],C^c\left(r_t\right)=\left[\begin{array}{cc}C\left(r_t\right)& 0\\ D\left(r_t\right)F\left(r_t\right)& D\left(r_t\right)\widehat{C}\left(r_t\right)\end{array}\right].\hfill \end{array}$$

(31)

If system (30) is SEMSS when $v\left(t\right)\equiv 0$, we will call (29) a stabilizing dynamic controller. In the sequel, the set of all stabilizing controller with the form (29) will be denoted as $\mathbb{K}$. In the special case of $n_c=0$, $\mathbf{G}$ turns out to be a zero-order dynamic controller, i.e., a state-feedback controller $u\left(t\right)=F\left(r_t\right)x\left(t\right)$.

Our objective is to find a stabilizing controller in the admissible control set $\mathbb{K}$ such that the $H_2$ norm of (30) is minimized. Below, we introduce a standard notion which will be used in the subsequent discussion.

Definition 2.

For the following system:

$$x(t+1)=A_0\left(r_t\right)x\left(t\right)+G_0\left(r_t\right)u\left(t\right)+\sum _{k=1}^d[A_k\left(r_t\right)x\left(t\right)+G_k\left(r_t\right)u\left(t\right)]w_k\left(t\right),t\in {\mathbb{Z}}_{+},$$

(32)

if there is $u\left(t\right)=F\left(r_t\right)x\left(t\right)$ such that the resulting closed-loop system of (32) is SEMSS, then we say $(\mathbb{A},\mathbb{G})$ is stochastically stabilizable and $F\left(r_t\right)\in R^{n_u\times n}$ is called a stabilizing feedback gain. Moreover, if there is $H\left(r_t\right)\in R^{n\times n_z}$ such that the following system is SEMSS:

$$\left\{\begin{array}{c}x(t+1)=[A_0\left(r_t\right)+H\left(r_t\right)C\left(r_t\right)]x\left(t\right)+{\displaystyle \sum _{k=1}^d}{A}_k\left(r_t\right)x\left(t\right)w_k\left(t\right),\hfill \\ z\left(t\right)=C\left(r_t\right)x\left(t\right),t\in {\mathbb{Z}}_{+},\hfill \end{array}\right.$$

(33)

then we say $(\mathbb{C},\mathbb{A})$ is stochastically detectable.

The following result is an extension of Proposition 3.3 ([21]), which tackles the existence and uniqueness of the stabilizing solution to a set of coupled stochastic algebraic Riccati equations. The detailed proof can be presented by following the same line as [21].

Lemma 3.

If $(\mathbb{A},\mathbb{G})$ is stochastically stabilizable and $(\mathbb{C},\mathbb{A})$ is stochastically detectable, then the following set of coupled stochastic algebraic Riccati equations admit a unique solution $0\le P\in {\mathbb{H}}_{\infty }^n$:

$$\left\{\begin{array}{c}P(\ell )=\mathcal{T}\left(P\right)(\ell )+C{(\ell )}^{\prime }C(\ell )-{[\Phi \left(P\right)(\ell )]}^{\prime }{[\Pi \left(P\right)(\ell )]}^{-1}[\Phi \left(P\right)(\ell )],\hfill \\ \Pi \left(P\right)(\ell )={\displaystyle \sum _{k=0}^d}{G}_k{(\ell )}^{\prime }\mathcal{E}\left(P\right)(\ell )G_k(\ell )+D{(\ell )}^{\prime }D(\ell )>0,\hfill \\ \Phi \left(P\right)(\ell )={\displaystyle \sum _{k=0}^d}{G}_k{(\ell )}^{\prime }\mathcal{E}\left(P\right)(\ell )A_k(\ell ),\ell \in \mathfrak{S}.\hfill \end{array}\right.$$

(34)

Moreover, $F\left(r_t\right)=-{[\Pi \left(P\right)\left(r_t\right)]}^{-1}\Phi \left(P\right)\left(r_t\right)$ is a stabilizing feedback gain (i.e., P is a stabilizing solution to (34)).

Now, we are prepared to give the main result of this section, which provides an $H_2$ optimal control design for the considered systems.

Theorem 2.

If $(\mathbb{A},\mathbb{G})$ is stochastically stabilizable and $(\mathbb{C},\mathbb{A})$ is stochastically detectable, then the $H_2$ optimal controller in the set $\mathbb{K}$ is given by

$$u^{*}\left(t\right)=F^{*}\left(r_t\right)x\left(t\right)=-{[\Pi \left(P\right)\left(r_t\right)]}^{-1}[\Phi \left(P\right)\left(r_t\right)]x\left(t\right),$$

(35)

where P is the stabilizing solution of (34). Moreover, the minimal $H_2$ norm of ${\mathbf{G}}_c$ is represented as

$$\begin{array}{cc}\hfill \underset{\mathbf{G}\in \mathbb{K}}{min}\parallel {\mathbf{G}}_c{\parallel }_2^2& =\parallel {\mathbf{G}}_{u^{*}}{\parallel }_2^2\hfill \\ & =\underset{\mathfrak{S}}{\int }\underset{\mathfrak{S}}{\int }\pi \left(s\right)g(s,\ell )Tr\left[B_v{\left(s\right)}^{\prime }P(\ell )B_v\left(s\right)\right]\mu \left(ds\right)\mu (d\ell ),\hfill \end{array}$$

(36)

where ${\mathbf{G}}_{u^{*}}$ denotes the system (28) driven by $u^{*}\left(t\right)=F^{*}\left(r_t\right)x\left(t\right)$.

Proof. 

By Lemma 3, Equation (34) admits a unique stabilizing solution $0\le P\in {\mathbb{H}}_{\infty }^n$ and $u^{*}\left(t\right)$ is a stabilizing state-feedback controller. Furthermore, we can rewrite (34) as follows:

$$\begin{array}{c}\hfill P(\ell )=\sum _{k=0}^d{[A_k(\ell )+G_k(\ell )F^{*}(\ell )]}^{\prime }\mathcal{E}\left(P\right)(\ell )[A_k(\ell )+G_k(\ell )F^{*}(\ell )]+\tilde{C}{(\ell )}^{\prime }\tilde{C}(\ell ),\ell \in \mathfrak{S},\end{array}$$

(37)

where $F^{*}(\ell )$ is determined by (35) and $\tilde{C}(\ell )={\left[C{(\ell )}^{\prime }{F}^{*}{(\ell )}^{\prime }D{(\ell )}^{\prime }\right]}^{\prime }$. Via simple calculations, it can be verified that the solution P of (37) is indeed the observability Gramian of ${\mathbf{G}}_{u^{*}}$. From Theorem 1 and Remark 2, we derive that

$$\begin{array}{c}\hfill \parallel {\mathbf{G}}_{u^{*}}{\parallel }_2^2=\underset{\mathfrak{S}}{\int }\underset{\mathfrak{S}}{\int }\pi \left(s\right)g(s,\ell )Tr\left[B_v{\left(s\right)}^{\prime }P(\ell )B_v\left(s\right)\right]\mu \left(ds\right)\mu (d\ell ).\end{array}$$

(38)

Next, it remains to show that $u^{*}\left(t\right)$ achieves the minimal $H_2$ norm among the set $\mathbb{K}$. To this end, for any dynamic controller $\mathbf{G}\in \mathbb{K}$, the corresponding closed-loop system is given by (30). It is easy to get the observability Gramian equation for (30), which is given as follows:

$$\begin{array}{c}\hfill U^c(\ell )=\sum _{k=0}^d{\left[A_k^c(\ell )\right]}^{\prime }\mathcal{E}\left(U^c\right)(\ell )\left[A_k^c(\ell )\right]+C^c{(\ell )}^{\prime }{C}^c(\ell ),\ell \in \mathfrak{S}.\end{array}$$

(39)

Making use of Theorem 1 and Remark 2 again, we have

$$\begin{array}{c}\hfill \parallel {\mathbf{G}}_c{\parallel }_2^2=\underset{\mathfrak{S}}{\int }\underset{\mathfrak{S}}{\int }\pi \left(s\right)g(s,\ell )Tr\left[B^c{\left(s\right)}^{\prime }{U}^c(\ell )B^c\left(s\right)\right]\mu \left(ds\right)\mu (d\ell ).\end{array}$$

(40)

Now, according to the structure of $A_k^c(\ell )$, we can separate $U^c(\ell )$ into four blocks as follows:

$$\begin{array}{c}\hfill U^c(\ell )=\left[\begin{array}{cc}{U}_{11}^c(\ell )& U_{12}^c(\ell )\\ {U_{12}^c(\ell )}^{\prime }& U_{22}^c(\ell )\end{array}\right].\end{array}$$

(41)

Then, by combing (37) and (39), it can be verified that the following block matrix

$$\begin{array}{c}\hfill \overline{U}(\ell ):=\left[\begin{array}{cc}{U}_{11}^c(\ell )& U_{12}^c(\ell )\\ {U_{12}^c(\ell )}^{\prime }& U_{22}^c(\ell )\end{array}\right]-\left[\begin{array}{cc}P(\ell )& 0\\ 0& 0\end{array}\right](\ell \in \mathfrak{S})\end{array}$$

(42)

satisfies the following Lyapunov equation:

$$\begin{array}{c}\hfill \overline{U}(\ell )=\sum _{k=0}^d{\left[A_k^c(\ell )\right]}^{\prime }\mathcal{E}\left(\overline{U}\right)(\ell )\left[A_k^c(\ell )\right]+{[F^{*}(\ell )-F(\ell )-\widehat{C}(\ell )]}^{\prime }\Pi \left(P\right)(\ell )[F^{*}(\ell )-F(\ell )-\widehat{C}(\ell )],\end{array}$$

(43)

where $F(\ell )$ and $\widehat{C}(\ell )$ are the coefficients of (29). Because $\mathbf{G}\in \mathbb{K}$ is a stabilizing controller, we have that system (30) is SEMSS when $v\left(t\right)\equiv 0$. Hence, by Lemma 1 (i), we can derive from $\Pi \left(P\right)(\ell )>0$ (see (34)) that the above Lyapunov Equation (43) admits a unique solution $0\le \overline{U}(\ell )\in {\mathbb{H}}_{\infty }^{2n}$. Therefore, from (40) and (42), it follows that

$$\begin{array}{c}\hfill \parallel {\mathbf{G}}_c{\parallel }_2^2=\underset{\mathfrak{S}}{\int }\underset{\mathfrak{S}}{\int }\pi \left(s\right)g(s,\ell )Tr\left[B^c{\left(s\right)}^{\prime }\overline{U}(\ell )B^c\left(s\right)\right]\mu \left(ds\right)\mu (d\ell )\\ \hfill +\underset{\mathfrak{S}}{\int }\underset{\mathfrak{S}}{\int }\pi \left(s\right)g(s,\ell )Tr\left[B_v{\left(s\right)}^{\prime }P(\ell )B_v\left(s\right)\right]\mu \left(ds\right)\mu (d\ell )\\ \hfill \ge \underset{\mathfrak{S}}{\int }\underset{\mathfrak{S}}{\int }\pi \left(s\right)g(s,\ell )Tr\left[B_v{\left(s\right)}^{\prime }P(\ell )B_v\left(s\right)\right]\mu \left(ds\right)\mu (d\ell )=\parallel {\mathbf{G}}_{u^{*}}{\parallel }_2^2,\end{array}$$

(44)

which completes the proof.    □

Remark 3.

The $H_2$ optimal control design proposed in Theorem 2 can be regarded as a generalization of Theorem 7.6 of [25]. More specifically, if the state space of ${\left\{r_t\right\}}_{t\in {\mathbb{Z}}_{+}}$ reduces to a finite set, then the above result produces an $H_2$ optimal controller for discrete-time stochastic systems with finite Markov jump parameters and multiplicative noises, as considered in [25].

4. Numerical Algorithm

From the preceding analysis, the applicability of $H_2$ optimal controller depends on how to get the solution of coupled stochastic algebraic Riccati Equation (34). Based on the grid of state space $\mathfrak{S}$ and the iterative algorithm proposed in [28], we can present the following (Algorithm 1) procedure to compute the approximate solution of (34).

Algorithm 1: Numerical algorithm for solving (34)

Input:  The number of nodes on the grid of $\mathfrak{S}$ and the selected accuracy $ϵ>0$; 1: Submitting the coefficients on node i to the iterative algorithm of [28]; 2: Computing the numerical solution of (34) on node i; 3: Counting the number of iterations on node i; Output:  Numerical solution of (34) on each node and the number of iterations on each node.

Example 1.

Consider a thermal receiver with dynamics given by (28) with $d=1$. Let $r_t$ be a Markov chain with the state space $\mathfrak{S}=\left\{\right\{1\}\times [0,1],\{2\}\times [0,1\left]\right\}$, which stands for two atmospheric conditions (sunny and cloudy). Given $r_t=\{i,\ell \}$ ($i=1,2$ and $\ell \in [0,1]$), ℓ is supposed to have a Beta distribution on $[0,1]$ with parameters $\alpha =\beta =2$, i.e., $\ell \sim Be(2,2)$. By (2), the transition probability of Markov chain $r_t$ is represented as follows:

$$\begin{array}{c}\hfill p_{ij}=\mathcal{P}\left(r_{t+1}=(j,[0,1])|r_t=(i,\ell )\right)=\mathcal{G}((i,\ell ),(j,[0,1]))={\int }_0^1{p}_{ij}f\left(s\right)ds,i,j=1,2,\end{array}$$

(45)

where $f\left(s\right)=6s(1-s)$ ($s\in [0,1]$) is the probability density function of $Be(2,2)$. Obviously, Equation (45) implies that

$$\begin{array}{c}\hfill g((i,\ell ),(j,s))=p_{ij}(6s-6s^2),s\in [0,1],i,j=1,2.\end{array}$$

(46)

Thus, the coupled stochastic algebraic Riccati Equation (34) can be written as follows:

$$\left\{\begin{array}{c}P(i,\ell )={\displaystyle \sum _{k=0}^d}{A}_k{(i,\ell )}^{\prime }\left[{\displaystyle \sum _{j=1}^2}{\int }_0^1{p}_{ij}(6s-6s^2)P(j,s)ds\right]A_k(i,\ell )+C{(i,\ell )}^{\prime }C(i,\ell )\hfill \\ -{[\Phi \left(P\right)(i,\ell )]}^{\prime }{[\Pi \left(P\right)(i,\ell )]}^{-1}[\Phi \left(P\right)(i,\ell )],\hfill \\ \Pi \left(P\right)(i,\ell )={\displaystyle \sum _{k=0}^d}{G}_k{(i,\ell )}^{\prime }\left[{\displaystyle \sum _{j=1}^2}{\int }_0^1{p}_{ij}(6s-6s^2)P(j,s)ds\right]G_k(i,\ell )+D{(i,\ell )}^{\prime }D(i,\ell )>0,\hfill \\ \Phi \left(P\right)(i,\ell )={\displaystyle \sum _{k=0}^d}{G}_k{(i,\ell )}^{\prime }\left[{\displaystyle \sum _{j=1}^2}{\int }_0^1{p}_{ij}(6s-6s^2)P(j,s)ds\right]A_k(i,\ell ),(i,\ell )\in \mathfrak{S}.\hfill \end{array}\right.$$

(47)

Now, we divide the interval $[0,1]$ into 100 equal segments and the state space becomes:

$$\begin{array}{c}\hfill {\mathfrak{S}}_i=\{\left\{i\right\}\times [0,\frac{1}{n}],\cdots ,\left\{i\right\}\times [\frac{m-1}{n},\frac{m}{n}],\cdots ,\left\{i\right\}\times [\frac{n-1}{n},1]\},i=1,2,n=100.\end{array}$$

(48)

Further, let us utilize the Trapezoidal Rule to compute the approximate value of the integral in (47):

$$\begin{array}{c}\hfill {\int }_0^1{p}_{ij}(6s-6s^2)P(j,s)ds\approx p_{ij}6[\frac{0}{100}-{\left(\frac{0}{100}\right)}^2]P(j,\frac{n}{100})·\frac{1}{200}\\ \hfill +2\sum _{n=1}^{99}{p}_{ij}6[\frac{n}{100}-{\left(\frac{n}{100}\right)}^2]P(j,\frac{n}{100})·\frac{1}{200}\\ \hfill +p_{ij}6[\frac{100}{100}-{\left(\frac{100}{100}\right)}^2]P(j,\frac{n}{100})·\frac{1}{200}\\ \hfill =\sum _{n=1}^{99}{p}_{ij}6[\frac{n}{100}-{\left(\frac{n}{100}\right)}^2]P(j,\frac{n}{100})·\frac{1}{100}:={\mathcal{E}}_{ij}.\end{array}$$

(49)

Hence, for the node $(i,{\ell }_m)$$(i=1,2;{\ell }_m=\frac{m}{100},m=0,1,\cdots ,100)$, the numerical solution of (34) can be approximately expressed by:

$$\begin{array}{c}\hfill \left\{\begin{array}{c}P(i,{\ell }_m)={\displaystyle \sum _{k=0}^d}{A}_k{(i,{\ell }_m)}^{\prime }\left({\displaystyle \sum _{j=1}^2}{\mathcal{E}}_{ij}\right)A_k(i,{\ell }_m)+C{(i,{\ell }_m)}^{\prime }C(i,{\ell }_m)\hfill \\ -{[\Phi \left(P\right)(i,{\ell }_m)]}^{\prime }{[\Pi \left(P\right)(i,{\ell }_m)]}^{-1}[\Phi \left(P\right)(i,{\ell }_m)],\hfill \\ \Pi \left(P\right)(i,{\ell }_m)={\displaystyle \sum _{k=0}^d}{G}_k{(i,{\ell }_m)}^{\prime }\left({\displaystyle \sum _{j=1}^2}{\mathcal{E}}_{ij}\right)G_k(i,{\ell }_m)+D{(i,{\ell }_m)}^{\prime }D(i,{\ell }_m)>0,\hfill \\ \Phi \left(P\right)(i,{\ell }_m)={\displaystyle \sum _{k=0}^d}{G}_k{(i,{\ell }_m)}^{\prime }\left({\displaystyle \sum _{j=1}^2}{\mathcal{E}}_{ij}\right)A_k(i,{\ell }_m).\hfill \end{array}\right.\end{array}$$

(50)

At this stage, we have transformed (47) into a set of discrete-time algebraic Riccati equations as considered in [28]. By applying the iterative algorithm of [28], we can calculate the approximate solution of $P(i,{\ell }_m)$ for every node.

Here, we set $p_{11}=0.9$, $p_{12}=0.1$, $p_{21}=0.7$ and $p_{22}=0.3$. The coefficients of (28) are determined by

$$\begin{array}{c}\hfill \left\{\begin{array}{c}{A}_j(i,\ell )=A_j(i,1)+\ell [A_j(i,2)-A_j(i,1)],\hfill \\ G_j(i,\ell )=G_j(i,1)+\ell [G_j(i,2)-G_j(i,1)],\hfill \\ B^v(i,\ell )=B^v(i,1)+\ell [B^v(i,2)-B^v(i,1)],\hfill \\ C(i,\ell )\equiv 0.1884,D(i,\ell )\equiv 1,(i,j=1,2)\hfill \end{array}\right.\end{array}$$

(51)

with the following parameters:

$$\begin{array}{ccc}& & A_0(1,1)=1.9,A_0(1,2)=1.7,A_0(2,1)=1.2,A_0(2,2)=1.4,\hfill \\ & & B^v(1,1)=1.2,B^v(1,2)=0.9,B^v(2,1)=1.1,B^v(2,2)=0.7,\hfill \\ & & A_1(1,1)=1.4,A_1(1,2)=1.3,A_1(2,1)=1.6,A_1(2,2)=1.8,\hfill \\ & & G_1(1,1)=1.3,G_1(1,2)=1.4,G_1(2,1)=1.3,G_1(2,2)=1.4,\hfill \\ & & G_0(1,1)=1.1,G_0(1,2)=1.4,G_0(2,1)=1.0,G_0(2,2)=0.\hfill \end{array}$$

(52)

For a given accuracy $ϵ=1\times {10}^{-5}$, by applying Algorithm 1, we have calculated (via Matlab) the numerical solution of (34) on each node, which is displayed in Figure 1.

5. Conclusions

In this paper, we studied the $H_2$ optimal control problem for Borel-measurable Markov jump systems with multiplicative noises. $H_2$ norm is introduced by the stationary variance of perturbed output response, which can be quantified by controllability and observability Gramians of the considered systems. Moreover, by means of the stabilizing solution to a set of coupled stochastic algebraic Riccati equations, $H_2$ optimal controller is obtained. The current study will yield some interesting open topics. For example, when the information of system state is only partially accessible to measurement, as considered in [21], how to settle the $H_2$ optimal control problem for the considered systems. This issue no doubt deserves a further research.