Adaptive Consensus of the Stochastic Leader-Following Multi-Agent System with Time Delay

Abstract

For the multi-agent system with time delay and noise, the adaptive consensus of tracking control problems is discussed by the Lyapunov function. The main purpose of this study is to design an adaptive control protocol for the system, such that even if there exists time delay among agents, the protocol can still ensure the consensus of the stochastic system. The main contribution is to revise the protocols that were previously only applicable to system without time delay. Because the system is inevitably disrupted by time delay and noise during the interactive process, achieving coordination and consensus is difficult. To enable the followers to track the leader, a novel adaptive law depending on the Riccati equation is firstly proposed, and the adaptive law is different from previous mandatory control law completely depending on a known function. The ability to be altered online based on the state of system is a major feature of the adaptive law. When there are interactive noise and time delay between the followers and leader of the system, a special Lyapunov function is constructed to prove the adaptive consensus. And the upper bound of time delay is obtained by using the Itô integral theory. Finally, if the time delay of the system approaches zero, it is shown that the adaptive law still ensures that each follower tracks the leader under simpler conditions.

1. Introduction

The multi-agent system can complete a complex task through mutual coordination among agents, which has become a research hotspot in current academic research. Centralized control and distributed control are two main aspects of current research on multi-agent applications. The focus of current research is distributed control, since it is more fault-tolerant to the environment and has lower cost requirements than centralized control. The application scope of distributed control in multi-agent systems includes unmanned aerial vehicles, smart grid, target tracking, traffic control and other fields [1,2,3]. The core of many distributed control systems is to seek a suitable control protocol that makes it possible for all agents to reach the same state, which is called the consensus of system. Currently, the research topics of the consensus focus on random disturbance control, finite time control, event-triggered control, distributed optimal control and so on.

Since Visek et al. [4] proposed a special mathematical model and discovered that all agents ultimately reach the same state under specific conditions, the multi-agent system has quickly attracted the attention of a large number of scholars. Recently, Qin et al. [5] and Amirkhani et al. [6] reviewed the theoretical progress of the consensus and introduced some difficulties in the system. In order to achieve the consensus, it is often necessary to constrain the topology of system and construct an appropriate control protocol. For an undirected graph, connectivity is usually required, while it is balanced for a directed graph. This paper mainly studies the adaptive consensus on a directed graph. Our goal is to build an adaptive protocol that enables the followers to track a certain objective. Moreover, the problem is disturbed by noise and has a hysteresis effect. Up to now, numerous academics have investigated the leader-following consensus from various angles. Jiang et al. [7] discussed the tracking issue when the equations of state contain time-varying matrices. A similar consensus was analyzed in the event-triggered mechanism [8,9,10]. Zhang et al. [11] extended the tracking problem to stochastic system and utilized mathematical expectation to analyze the problem. The multi-agent system mentioned in these references all have definite models. However, the internal structure of the system is often uncertain in complex environments, so the adaptive control methods are proposed to continuously update the structure of the system.

Adaptive control technology is a method that automatically adjusts its own control parameters with the change of the environment to achieve the best performance. Adaptive consensus can be defined as that the state of all agents is finally consistent due to the adaptive control technology. Adaptive law can be seen as the changing law of the control parameters, and it is usually represented by a differential equation. Adaptive algorithms are usually characterized by information and intelligence; the information of this paper mainly comes from the state of system, and the intelligence is determined by the adaptive laws. The algorithm is often combined with machine learning theory and applied to some game scenarios. Adaptive control was initially applied in the aerospace field, and Whitaker is crucial to the advancement of the method. Currently, this special technology has found extensive use in fields such as aerospace, power, transportation, robotics, etc. The creation of a suitable adaptive law is the crux of the challenge for this technology. For a multi-agent system, when the mandatory gain is independent of the states of output and input, Li et al. [12] and Cheng et al. [13] respectively analyzed the average consensus. Zong et al. [14] investigated the random weak consensus under mandatory gain. For the adaptive gain that can be dynamically modified according to the current state, Knotek et al. [15] established an adaptive control law with decay gain, and the edge-based adaptive techniques for a nonlinear multi-agent system were taken into consideration by Yu et al. [16]. Luo et al. [17] analyzed a gradient-descent-based adaptive law and gave a scheme for the optimal control problem of uncertain multi-agent system. Li et al. [18] proposed a value iteration strategy and used the gradient descent method to update the weights. For a self-organized system, some important self-organized models were discussed in [19], and a self-organized interlimb coordination control was analyzed in [20]. For the optimal control problem of discrete systems, Peng et al. [2] designed a strategy for the adaptive adjustment of weight vectors based on neural network approximation. Nevertheless, these studies did not consider the effects of noise and time delay. Since the system is inevitably disturbed by time delay and noise at the same time, it is necessary to study the adaptive consensus under noise and time delay.

Currently, there have been many research conclusions about the consensus under noisy environments, but less research has been conducted on the topic of adaptive consensus. In fact, the interactive network among agents is subject to noise, so the stochastic multi-agent system should be considered. Itô integral theory provides an important tool for the adaptive problems of stochastic system. When agents have noise perturbations during communication, Duan et al. [21] designed an adaptive control protocol and proved that the tracking error of the problem is bounded. Huang [22] discussed the adaptive consensus of uncertain system, and proved agents can obtain average consensus in the almost sure sense. Xiao et al. [23] proposed the adaptive finite-time control protocols for a leaderless system, and proved similar properties hold for systems with a leader. The bipartite adaptive consensus of the stochastic system were taken into account in [24,25]. However, these references did not consider the interference of time delay. Time delay often degrades the performance of the control system and disrupts the stability of the system. Furthermore, the presence of time delay causes the great difficulties in the analysis and synthesis of the control system.

When the system is jointly disturbed by time delay and noise during the interactive process, the dynamical model of the system has a more complex form. There are currently only a few papers that consider the adaptive consensus in this situation. When the adaptive gain is mandatory, Zong et al. [14] analyzed the tracking problem in the case of the joint disturbance of noise and time delay. Also, a neural network approach was employed to analyze the topic for mandatory gain [26]. In practical applications, the mandatory gain has to be accurately selected based on the actual situation, which is often quite difficult. This paper will consider an adaptive control law that can dynamically adjust on the basis of state. For the tracing problem of multi-agent system, we first propose an adaptive control protocol and design a novel adaptive law, then the Lyapunov function is used to prove the adaptive consensus of the system. Finally, when the time delay trends to zero, we simplify the conditions for the system to attain the adaptive consensus. The significance of this paper is to revise the control protocol that were previously only applicable to a system without time delay. Our proposed adaptive control protocol can ensure the consensus of a system under the interference of noise and time delay. The contributions are as follows:

(1)

For a stochastic multi-agent system, a novel adaptive control law is firstly proposed when there is a lag phenomenon in the interactive process. The control laws in [12,13,14] were all mandatory and often required precise selection to determine the specific form. The adaptive control proposed in this paper can be dynamically adjusted based on the current state of the system, thus avoiding the difficulty of precise selection.

(2)

No matter whether the stochastic multi-agent system has time delay or not, the adaptive control law can ensure the consensus. However, the adaptive laws in [15,16] were only applied to multi-agent systems without delay and noise. Additionally, the sufficient conditions of consensus in this paper are simpler for the case without delay.

(3)

Compared with some early references in [21,24], the final tracking error in this paper has a smaller value under the adaptive law. Furthermore, when the intensity of noise approaches zero, the final dynamic error will trend to zero. However, many previous conclusions can only converge to a non-zero constant.

2. Theoretical Basis

The system in this work includes one leader and N followers, denoted as $v_0$, $v_1$, ⋯, and $v_N$, respectively. $\mathcal{G}=(\mathcal{V},\mathcal{N},\mathcal{A})$ represents a digraph among the followers. $\mathcal{V}=\{v_1,v_2,\cdots ,v_N\}$ and $\mathcal{N}\subseteq \mathcal{V}\times \mathcal{V}$ is the set of the followers and edges, respectively. $\mathcal{A}=\left[e_{ij}\right]\in {\mathcal{R}}^{N\times N}$ is called adjacency matrix, its elements satisfy $e_{ij}=1$ if and only if $(v_i,v_j)\in \mathcal{N}$, or else, $e_{ij}=0$. ${\mathcal{N}}_i=\{v_j\in V:(v_j,v_i)\in \mathcal{N}\}$ is the neighbor set, and $L_{\mathcal{G}}=\left[l_{ij}\right]$ is the Laplace matrix, where $l_{ii}={\displaystyle \sum _{j\ne i}}{e}_{ij}$ and $l_{ij}=-e_{ij}$, $i\ne j$. In addition, assuming $\tilde{\mathcal{G}}$ is a digraph composed of all agents, and the matrix $L_{\tilde{\mathcal{G}}}$ is defined by $\left[\begin{array}{cc}0& 0_{1\times N}\\ -E_0·1_N& L_{\mathcal{G}}+E_0\end{array}\right]$, where $E_0=\mathrm{diag}\{e_{10},e_{20},\cdots ,e_{N0}\}$ and $1_N={[1,1,\cdots ,1]}^{\mathrm{T}}$. The difference between the two digraphs is that $\tilde{\mathcal{G}}$ contains the node of leader, while $\mathcal{G}$ does not.

Supposing the leader $v_0$ is globally reachable in this paper, which means a directed path from each follower $v_i$ to the leader $v_0$ can be found. When all elements of the adjacency matrix $\mathcal{A}$ satisfy ${\displaystyle \sum _{j=1}^N}{e}_{ij}={\displaystyle \sum _{j=1}^N}{e}_{ji}$, the digraph is a balanced graph. The following lemmas are introduced.

Lemma 1 

([27]). Assuming $\tilde{\mathcal{G}}$ is a digraph, the three properties are equivalent:

(1) 

The node $v_0$ is globally reachable.

(2) 

For the matrix $H=L_{\mathcal{G}}+E_0$, the real parts of all eigenvalues are positive.

(3) 

Further suppose the digraph $\mathcal{G}$ is balanced, then $H+H^T$ is positive definite.

Lemma 2 

([28]). For the matrices $M_1$, $M_2$, $M_3$ and $M_4$, the Kronecker product of two matrices is represented by the symbol $\otimes$. Assuming the four matrices have appropriate dimensions, then the following properties hold:

(1) 

$M_1\otimes (M_2+M_3)=M_1\otimes M_2+M_1\otimes M_3$.

(2) 

$(M_1\otimes M_2)\otimes M_3=M_1\otimes (M_2\otimes M_3)$.

(3) 

$(M_1\otimes M_2)(M_3\otimes M_4)=M_1{M}_3\otimes M_2{M}_4$.

(4) 

${(M_1\otimes M_2)}^T=M_1^T\otimes M_2^T$.

(5) 

$\mathit{tr}(M_1\otimes M_2)=\mathit{tr}\left(M_1\right)\mathit{tr}\left(M_2\right)$.

3. The Adaptive Consensus

Considering a multi-agent system, its dynamic behavior can be expressed as

$$\begin{array}{c}\hfill {\dot{x}}_i\left(t\right)=A{x}_i\left(t\right)+B{u}_i\left(t\right),i=1,2,\cdots ,N.\end{array}$$

(1)

In the equation, $u_i\left(t\right)\in {\mathbf{R}}^p$ denotes the input and needs to be devised, $x_i\left(t\right)\in {\mathbf{R}}^n$ represents the state of the position. A is a $n\times n$ order constant matrix, B is a $n\times p$ order constant matrix, and the two matrices are known. The model of leader is represented as

$$\begin{array}{c}\hfill {\dot{x}}_0\left(t\right)=A{x}_0\left(t\right).\end{array}$$

(2)

In order to obtain the adaptive consensus of system (1), the key issue is to construct a control protocol $u_i\left(t\right)$ containing adaptive gain based on the communication graph among agents, and then use the state $x_i\left(t\right)$ to design the adjustment method of the adaptive gain. The adaptive method can rely on relatively little prior knowledge about the model. If the system (1) is not disturbed by time delay and noise, a general control protocol can be represented as $u_i\left(t\right)=cK{\displaystyle \sum _{j\in N_i}}{e}_{ij}(x_j\left(t\right)-x_i\left(t\right))$, where c is a coupling weight and K is a feedback gain matrix. The protocol was investigated in [29,30], who pointed out that the constant c is related to the global information of the system. When there exists noise interference and time delay in the process of communication, this paper proposes an adaptive control protocol, designs an novel adaptive control law, and analyzes the impact of time delay on the system.

For n dimensional probability space $(\Omega ,\mathcal{F},P)$, the standard Brownian motions in the space are denoted by $W_i\left(t\right)\in {\mathbf{R}}^n$, the standard white noise is written as ${\eta }_i\left(t\right)\in {\mathbf{R}}^n$ and satisfies ${\int }_0^t{\eta }_i\left(s\right)\mathrm{d}s=W_i\left(t\right)$. For the system (1), the control protocol perturbed by noise and time delay is designed as

$$\begin{array}{c}\hfill u_i\left(t\right)=s_i\left(t\right)K\left[\sum _{j\in N_i}{e}_{ij}(x_j(t-\tau )-x_i(t-\tau ))+e_{i0}(x_0(t-\tau )-x_i(t-\tau ))+e_{i0}{\sigma }_{0i}{\eta }_i\left(t\right)\right].\end{array}$$

(3)

In the protocol, $\tau >0$ is time delay, ${\sigma }_{0i}$ is noise intensity, the constants $e_{ij}$ and $e_{i0}$ indicate the weights of digraphs in the multi-agent system, the matrix $K\in {\mathbf{R}}^{p\times n}$ is called a feedback gain matrix. The adaptive gain $s_i\left(t\right)$ satisfies $\underline{\theta }\le s_i\left(t\right)\le \overline{\theta }$, where $\underline{\theta }$ and $\overline{\theta }$ are two positive constants. The difficulty of solving adaptive control problems lies in designing an appropriate adaptive control law. For this control protocol (3), in order to obtain the adaptive consensus of the system, the main difficulty is to construct a differential equation that the gain $s_i\left(t\right)$ satisfies.

When the control protocol (3) does not contain time delay and noise, many scholars have already studied the adaptive consensus. Li et al. [31] considered the adaptive tracking problem of system with a leader. The adaptive event-triggering theory was discussed for a linear time-varying system in [32]. Deng et al. [33] analyzed the adaptive tracking problem of high-order system. However, time delay and noise are inevitable in the process of agent interaction. For leaderless multi-agent system, Wu et al. [34] designed an adaptive control protocol in noisy environments. The adaptive consensus with multiplicative noise was analyzed in [35]. Duan et al. [21] discussed one order leader-following system with noise in the absence of time delay. In this section, the adaptive problem of system (1) and (2) will be studied under the control protocol (3), which not only considers the impact of noise, but also considers the effect of time delay, so it is more in line with real scenarios.

If the adaptive gain is mandatory, such as $s_i\left(t\right)=s\left(t\right)=\frac{1}{1+t}$ or $\frac{log(1+t)}{1+t}$, there have been many results. The mean square consensus was achieved in [12,13]. Zong et al. [14] investigated the adaptive protocol of the system under time delay and noise. Nevertheless, the mandatory gain has to be accurately selected in order to satisfy the limiting conditions, which is often quite difficult. Therefore, the adaptive gain that can be dynamically adjusted according to the state has obvious advantages in practical applications. In order to solve the consensus of the system (1)–(3), we construct a novel adaptive law as

$$\begin{array}{c}\hfill {\dot{s}}_i\left(t\right)={\epsilon }_i{\left(t\right)}^{\mathrm{T}}\sum _{j=1}^N{h}_{ij}\Gamma {\epsilon }_j\left(t\right)-(s_i\left(t\right)-\delta ).\end{array}$$

(4)

where the constant $\delta >\frac{1}{{\lambda }_{min}(H^{\mathrm{T}}+H)}$, the dynamic error ${\epsilon }_i\left(t\right)=x_i\left(t\right)-x_0\left(t\right)$, and the symbol $h_{ij}$ is the element of H. The adaptive law (4) can continuously improve the structure of the model by extracting model’s information, thereby enabling the model to more and more accurate. It is worth noting that the adaptive laws proposed in most of the literature are different, such as the mandatory adaptive law [13,14], the decaying adaptive law [15], the edge-based adaptive law [16], etc. The advantages of the adaptive law (4) is that it can be applied to multi-agent systems with noise and time delay. In order to prove the consensus of system, the solution of the algebraic Riccati equation is used to build the matrix $\Gamma$. Let $K=B^{\mathrm{T}}P$, the matrix $\Gamma =PBK$ is called adaptive gain matrix in (4), and P is a positive matrix and satisfies the algebraic Riccati equation

$$\begin{array}{c}\hfill A^{\mathrm{T}}P+PA-PB{B}^{\mathrm{T}}P+kI=0,(k>0).\end{array}$$

(5)

The above equation has been widely applied to prove the stability of the system since it was proposed. Generally, the matrix P can be used to construct Lyapunov functions, combined with the special form of the Riccati equation, it is easy to verify the conditions of the stability theorem.

Remark 1. 

The adaptive law (4) has a simpler structure and can be rewritten as

$$\left(\begin{array}{c}{\dot{s}}_1\left(t\right)\\ {\dot{s}}_2\left(t\right)\\ ⋮\\ {\dot{s}}_N\left(t\right)\end{array}\right)=\left(\begin{array}{cccc}{\epsilon }_1{\left(t\right)}^T& 0& \cdots & 0\\ 0& {\epsilon }_2{\left(t\right)}^T& \cdots & 0\\ ⋮& ⋮& & ⋮\\ 0& 0& \cdots & {\epsilon }_N{\left(t\right)}^T\end{array}\right)(H\otimes PBK)\left(\begin{array}{c}{\epsilon }_1\left(t\right)\\ {\epsilon }_2\left(t\right)\\ ⋮\\ {\epsilon }_N\left(t\right)\end{array}\right)-\left(\begin{array}{c}{s}_1\left(t\right)-\delta \\ s_2\left(t\right)-\delta \\ ⋮\\ s_N\left(t\right)-\delta \end{array}\right).$$

Although many different forms of adaptive laws have been proposed, most cannot be represented by the Kronecker products, which will make previous adaptive laws appear more complex. In addition, for the mandatory gain $s\left(t\right)$ proposed in many literature, the two constraints ${\int }_0^{\infty }s\left(t\right)\mathrm{d}t=\infty$ and ${\int }_0^{\infty }{s}^2\left(t\right)\mathrm{d}t<\infty$ need to be used, such as the continuous mandatory gain in references [13,14] and the discrete mandatory gain in reference [34]. The adaptive gain proposed in this article will automatically adjust according to the current state.

Let $\epsilon \left(t\right)={[{(x_1\left(t\right)-x_0\left(t\right))}^{\mathrm{T}},{(x_2\left(t\right)-x_0\left(t\right))}^{\mathrm{T}},\cdots ,{(x_N\left(t\right)-x_0\left(t\right))}^{\mathrm{T}}]}^{\mathrm{T}}$, the dynamic error equation can be abbreviated as

$$\begin{array}{c}\hfill \mathrm{d}\epsilon \left(t\right)=[(I_N\otimes A)\epsilon \left(t\right)-(S\left(t\right)H\otimes BK)\epsilon (t-\tau )]\mathrm{d}t-(S\left(t\right)E_0{C}_0\otimes BK)\mathrm{d}W.\end{array}$$

(6)

where $S\left(t\right)=\mathrm{diag}\{s_1\left(t\right),s_2\left(t\right),\cdots ,s_N\left(t\right)\}$ is a diagonal matrix, $I_N$ is an identity matrix, $C_0=\mathrm{diag}\{{\sigma }_{01},{\sigma }_{02},\cdots ,{\sigma }_{0N}\}$ is the matrix corresponding to noise intensity, $\mathrm{d}W$ is $nN$ dimensional standard Brownian motion, and $E_0=\mathrm{diag}\{e_{10},e_{20},\cdots ,e_{N0}\}$ reflects the interaction of the system. Equation (5) is known as a stochastic differential equation, which includes a differential part and random part. The random part can reflect the changes of disturbance. The following theorem demonstrates the adaptive consensus of the system (1)–(3) when the adaptive law adopts the Equation (4).

Theorem 1. 

Assuming that the digraph $\tilde{\mathcal{G}}=(\tilde{\mathcal{V}},\tilde{\mathcal{N}},\tilde{\mathcal{A}})$ for a system of $N+1$ agents is made up of N followers and one leader, and that its subgraph $\mathcal{G}$ for all followers is a balanced graph. For the multi-agent system determined by the Equations (1)–(3), if there exists a positive constant ξ satisfying

$$\begin{array}{ccc}\hfill k& >& \xi {\overline{\theta }}^2{\lambda }_{max}\left(H{H}^T\right){\lambda }_{max}^4\left(P\right){\lambda }_{max}^2\left(B{B}^T\right)+4{\xi }^{-1}{\tau }^2{\overline{\theta }}^2{\lambda }_{max}\left(H{H}^T\right){\lambda }_{max}^2\left(P\right){\lambda }_{max}^2\left(B{B}^T\right)\hfill \\ & & +4{\xi }^{-1}{\tau }^2{\lambda }_{max}\left(A{A}^T\right)\hfill \end{array}$$

(7)

then the mean square bounded consensus can be gained under the adaptive law (4), i.e.,

$$\begin{array}{c}\hfill \underset{t\to +\infty }{lim}E{|x_i\left(t\right)-x_0\left(t\right)|}^2={ϵ}_1\end{array}$$

(8)

where E represents the expectation, and ${ϵ}_1$ is a small constant independent of time t.

Proof. 

The Lyapunov function is chosen as follows,

$$\begin{array}{ccc}\hfill V_1\left(t\right)& =& V_{11}\left(t\right)+V_{12}\left(t\right)\hfill \\ & =& \epsilon {\left(t\right)}^{\mathrm{T}}(I_N\otimes P)\epsilon \left(t\right)+w_1{\int }_{t-\tau }^t|\epsilon \left(s\right){|}^2\mathrm{d}s+w_2{\int }_{-\tau }^0{\int }_{t+\theta }^t{\left|\epsilon \left(s\right)\right|}^2\mathrm{d}s\mathrm{d}\theta \hfill \\ & & +w_3{\int }_{-\tau }^0{\int }_{t+\theta }^t{\left|\epsilon (s-\tau )\right|}^2\mathrm{d}s\mathrm{d}\theta +V_{12}\left(t\right),\hfill \end{array}$$

where the function $V_{12}\left(t\right)={\displaystyle \sum _{i=1}^N}{(s_i\left(t\right)-\delta )}^2$. The Lyapunov function is mainly divided into three parts, the first part $\epsilon {\left(t\right)}^{\mathrm{T}}(I_N\otimes P)\epsilon \left(t\right)$ is similar to the construction of Lyapunov functions in most references, the special integral part was referred to as a degenerate functional and was used by Kolmanovskii et al. [36]. The last part $V_{12}\left(t\right)$ is a commonly used form in most of the literature when discussing adaptive consensus, and after taking the derivative of this function, the adaptive law can be used to eliminate some unnecessary terms in the following calculations. If the time t is less than $-\tau$ in the double integral ${\int }_{-\tau }^0{\int }_{t+\theta }^t{\left|\epsilon (s-\tau )\right|}^2\mathrm{d}s\mathrm{d}\theta$, we assume ${\epsilon }_i\left(t\right)$ equals to the initial value ${\epsilon }_i\left(0\right)$.

Applying the Itô formula and the error closed-loop systems (6), the random differentiation is expressed as

$$\begin{array}{c}\hfill \mathrm{d}{V}_1\left(t\right)={\mathcal{L}}_1{V}_1\left(t\right)\mathrm{d}t+2\epsilon {\left(t\right)}^{\mathrm{T}}[S\left(t\right)E_0{C}_0\otimes PBK]\mathrm{d}W,\end{array}$$

(9)

where the first term is defined as

$$\begin{array}{ccc}\hfill {\mathcal{L}}_1{V}_1\left(t\right)& =& \epsilon {\left(t\right)}^{\mathrm{T}}[I_N\otimes (A^{\mathrm{T}}P+PA)]\epsilon \left(t\right)-2\epsilon {\left(t\right)}^{\mathrm{T}}[S\left(t\right)H\otimes PBK]\epsilon (t-\tau )\hfill \\ & & +\mathrm{tr}\left\{S^2\left(t\right)E_0^2{C}_0^2\otimes K^{\mathrm{T}}{B}^{\mathrm{T}}PBK\right\}+w_1{\left|\epsilon \left(t\right)\right|}^2-w_1{\left|\epsilon (t-\tau )\right|}^2\hfill \\ & & +w_2{\tau \left|\epsilon \left(t\right)\right|}^2-w_2{\int }_{t-\tau }^t{\left|\epsilon \left(s\right)\right|}^{2}ds+w_3\tau {\left|\epsilon (t-\tau )\right|}^2\hfill \\ & & -w_3{\int }_{t-\tau }^t{\left|\epsilon (s-\tau )\right|}^2\mathrm{d}s+{\dot{V}}_{12}\left(t\right).\hfill \end{array}$$

Using the adaptive laws, we can obtain the following equation by combining the derivative rule of the composite function,

$$\begin{array}{ccc}& & {\dot{V}}_{12}\left(t\right)\hfill \\ & =& 2\sum _{i=1}^N(s_i\left(t\right)-\delta ){\dot{s}}_i\left(t\right)\hfill \\ & =& 2\sum _{i=1}^N\left[(s_i\left(t\right)-\delta ){\epsilon }_i{\left(t\right)}^{\mathrm{T}}\sum _{j=1}^N{h}_{ij}PBK{\epsilon }_j\left(t\right)\right]-2\sum _{i=1}^N{(s_i\left(t\right)-\delta )}^2\hfill \\ & =& 2\sum _{i=1}^N\left[s_i\left(t\right){\epsilon }_i{\left(t\right)}^{\mathrm{T}}\sum _{j=1}^N{h}_{ij}PBK{\epsilon }_j\left(t\right)\right]-2\delta \sum _{i=1}^N\left[{\epsilon }_i{\left(t\right)}^{\mathrm{T}}\sum _{j=1}^N{h}_{ij}PBK{\epsilon }_j\left(t\right)\right]-2\sum _{i=1}^N{(s_i\left(t\right)-\delta )}^2\hfill \\ & =& 2\epsilon {\left(t\right)}^{\mathrm{T}}[S\left(t\right)H\otimes PBK]\epsilon \left(t\right)-\delta \epsilon {\left(t\right)}^{\mathrm{T}}[(H^{\mathrm{T}}+H)\otimes PBK]\epsilon \left(t\right)-2\sum _{i=1}^N{(s_i\left(t\right)-\delta )}^2.\hfill \end{array}$$

By Lemma 1, we can obtain the matrix $H+H^{\mathrm{T}}$ is positive definite, which means all eigenvalues are greater than zero. Thus, we can obtain $\delta {\lambda }_{min}(H^{\mathrm{T}}+H)>1$ by the known condition $\delta >\frac{1}{{\lambda }_{min}(H^{\mathrm{T}}+H)}$. From the Ricatti equation, we have,

$$\begin{array}{ccc}& & {\mathcal{L}}_1{V}_1\left(t\right)\hfill \\ & =& \epsilon {\left(t\right)}^{\mathrm{T}}[I_N\otimes (A^{\mathrm{T}}P+PA)]\epsilon \left(t\right)-\delta \epsilon {\left(t\right)}^{\mathrm{T}}[(H^{\mathrm{T}}+H)\otimes PBK]\epsilon \left(t\right)\hfill \\ & & +2\epsilon {\left(t\right)}^{\mathrm{T}}[S\left(t\right)H\otimes PBK]\left[\epsilon \left(t\right)-\epsilon (t-\tau )\right]+\mathrm{tr}\left\{S^2\left(t\right)E_0^2{C}_0^2\otimes K^{\mathrm{T}}{B}^{\mathrm{T}}PBK\right\}\hfill \\ & & +w_1{\left|\epsilon \left(t\right)\right|}^2-w_1{\left|\epsilon (t-\tau )\right|}^2+w_2{\tau \left|\epsilon \left(t\right)\right|}^2-w_2{\int }_{t-\tau }^t{\left|\epsilon \left(s\right)\right|}^{2}ds\hfill \\ & & +w_3{\tau \left|\epsilon (t-\tau )\right|}^2-w_3{\int }_{t-\tau }^t{\left|\epsilon (s-\tau )\right|}^{2}ds-2\sum _{i=1}^N{(s_i\left(t\right)-\delta )}^2\hfill \\ & \le & \epsilon {\left(t\right)}^{\mathrm{T}}[I_N\otimes (A^{\mathrm{T}}P+PA-PBK)]\epsilon \left(t\right)\hfill \\ & & +2\epsilon {\left(t\right)}^{\mathrm{T}}[S\left(t\right)H\otimes PBK]\left[\epsilon \left(t\right)-\epsilon (t-\tau )\right]+\mathrm{tr}\left\{S^2\left(t\right)E_0^2{C}_0^2\otimes K^{\mathrm{T}}{B}^{\mathrm{T}}PBK\right\}\hfill \\ & & +w_1{\left|\epsilon \left(t\right)\right|}^2-w_1{\left|\epsilon (t-\tau )\right|}^2+w_2{\tau \left|\epsilon \left(t\right)\right|}^2-w_2{\int }_{t-\tau }^t{\left|\epsilon \left(s\right)\right|}^{2}ds\hfill \\ & & +w_3{\tau \left|\epsilon (t-\tau )\right|}^2-w_3{\int }_{t-\tau }^t{\left|\epsilon (s-\tau )\right|}^{2}ds-2\sum _{i=1}^N{(s_i\left(t\right)-\delta )}^2\hfill \\ & =& -{k\left|\epsilon \left(t\right)\right|}^2+2\epsilon {\left(t\right)}^{\mathrm{T}}[S\left(t\right)H\otimes PBK]\left[\epsilon \left(t\right)-\epsilon (t-\tau )\right]+\mathrm{tr}\left\{S^2\left(t\right)E_0^2{C}_0^2\otimes K^{\mathrm{T}}{B}^{\mathrm{T}}PBK\right\}\hfill \\ & & +w_1{\left|\epsilon \left(t\right)\right|}^2-w_1{\left|\epsilon (t-\tau )\right|}^2+w_2{\tau \left|\epsilon \left(t\right)\right|}^2-w_2{\int }_{t-\tau }^t{\left|\epsilon \left(s\right)\right|}^{2}ds+w_3\tau {\left|\epsilon (t-\tau )\right|}^2\hfill \\ & & -w_3{\int }_{t-\tau }^t{\left|\epsilon (s-\tau )\right|}^{2}ds-2\sum _{i=1}^N{(s_i\left(t\right)-\delta )}^2.\hfill \end{array}$$

(10)

Note the inequality $2ab\le \xi a^2+\frac{1}{\xi }{b}^2$ for any positive constant $\xi$, we have

$$\begin{array}{ccc}& & 2\epsilon {\left(t\right)}^{\mathrm{T}}[S\left(t\right)H\otimes PBK]\left[\epsilon \left(t\right)-\epsilon (t-\tau )\right]\hfill \\ & \le & \xi \epsilon {\left(t\right)}^{\mathrm{T}}[S\left(t\right)H\otimes PBK]{[S\left(t\right)H\otimes PBK]}^{\mathrm{T}}\epsilon \left(t\right)+{\xi }^{-1}{|\epsilon \left(t\right)-\epsilon (t-\tau )|}^2\hfill \\ & \le & \xi {\overline{\theta }}^2{\lambda }_{max}\left(H{H}^{\mathrm{T}}\right){\lambda }_{max}^4\left(P\right){\lambda }_{max}^2\left(B{B}^{\mathrm{T}}\right)|\epsilon \left(t\right){|}^2+{\xi }^{-1}{|\epsilon \left(t\right)-\epsilon (t-\tau )|}^2.\hfill \end{array}$$

(11)

Now, we can obtain from the above inequality,

$$\begin{array}{ccc}& & {\mathcal{L}}_1{V}_1\left(t\right)\hfill \\ & \le & -{k\left|\epsilon \left(t\right)\right|}^2+\xi {\overline{\theta }}^2{\lambda }_{max}\left(H{H}^{\mathrm{T}}\right){\lambda }_{max}^4\left(P\right){\lambda }_{max}^2\left(B{B}^{\mathrm{T}}\right)|\epsilon \left(t\right){|}^2+{\xi }^{-1}{|\epsilon \left(t\right)-\epsilon (t-\tau )|}^2\hfill \\ & & +{\overline{\theta }}^2{\lambda }_{max}^3\left(P\right){\lambda }_{max}^2\left(B{B}^{\mathrm{T}}\right)max\left\{{\left(e_{i0}{\sigma }_{0i}\right)}^2\right\}+w_1|\epsilon \left(t\right){|}^2-w_1{\left|\epsilon (t-\tau )\right|}^2\hfill \\ & & +w_2{\tau \left|\epsilon \left(t\right)\right|}^2-w_2{\int }_{t-\tau }^t{\left|\epsilon \left(s\right)\right|}^{2}ds+w_3\tau {\left|\epsilon (t-\tau )\right|}^2\hfill \\ & & -w_3{\int }_{t-\tau }^t{\left|\epsilon (s-\tau )\right|}^{2}ds-2\sum _{i=1}^N{(s_i\left(t\right)-\delta )}^2\hfill \\ & =& -{\Lambda }_1{\left|\epsilon \left(t\right)\right|}^2-{\Lambda }_2{\left|\epsilon (t-\tau )\right|}^2+{\xi }^{-1}{|\epsilon \left(t\right)-\epsilon (t-\tau )|}^2\hfill \\ & & +{\overline{\theta }}^2{\lambda }_{max}^3\left(P\right){\lambda }_{max}^2\left(B{B}^{\mathrm{T}}\right)max\left\{{\left(e_{i0}{\sigma }_{0i}\right)}^2\right\}\hfill \\ & & -w_2{\int }_{t-\tau }^t{\left|\epsilon \left(s\right)\right|}^{2}ds-w_3{\int }_{t-\tau }^t{\left|\epsilon (s-\tau )\right|}^{2}ds-2\sum _{i=1}^N{(s_i\left(t\right)-\delta )}^2,\hfill \end{array}$$

(12)

where the two constants in the above inequality are denoted as

$${\Lambda }_1=k-\xi {\overline{\theta }}^2{\lambda }_{max}\left(H{H}^{\mathrm{T}}\right){\lambda }_{max}^4\left(P\right){\lambda }_{max}^2\left(B{B}^{\mathrm{T}}\right)-w_1-w_2\tau$$

and ${\Lambda }_2=w_1-w_3\tau$.

According to the error closed-loop equation, it obtains from the Hölder inequality,

$$\begin{array}{ccc}& & {|\epsilon \left(t\right)-\epsilon (t-\tau )|}^2\hfill \\ & =& {\left|{\int }_{t-\tau }^t\mathrm{d}\epsilon \left(s\right)\right|}^2\hfill \\ & =& {\left|{\int }_{t-\tau }^t\left[(I_N\otimes A)\epsilon \left(s\right)-\left(S\left(s\right)H\right)\otimes \left(BK\right)\epsilon (s-\tau )\right]\mathrm{d}s-{\int }_{t-\tau }^t\left(S\left(s\right)E_0{C}_0\right)\otimes \left(BK\right)\mathrm{d}W\right|}^2\hfill \\ & \le & 4{\left|{\int }_{t-\tau }^t(I_N\otimes A)\epsilon \left(s\right)\mathrm{d}s\right|}^2+4{\left|{\int }_{t-\tau }^t\left(S\left(s\right)H\right)\otimes \left(BK\right)\epsilon (s-\tau )\mathrm{d}s\right|}^2\hfill \\ & & +4{\left|{\int }_{t-\tau }^t\left(S\left(s\right)E_0{C}_0\right)\otimes \left(BK\right)\mathrm{d}W\right|}^2\hfill \\ & \le & 4\tau {\lambda }_{max}\left(A{A}^{\mathrm{T}}\right){\int }_{t-\tau }^t|\epsilon \left(s\right){|}^2\mathrm{d}s+4\tau {\overline{\theta }}^2{\lambda }_{max}\left(H{H}^{\mathrm{T}}\right){\lambda }_{max}^2\left(P\right){\lambda }_{max}^2\left(B{B}^{\mathrm{T}}\right){\int }_{t-\tau }^t{\left|\epsilon (s-\tau )\right|}^2\mathrm{d}s\hfill \\ & & +4{\overline{\theta }}^2{\lambda }_{max}^2\left(P\right){\lambda }_{max}^2\left(B{B}^{\mathrm{T}}\right)max\left\{{\left(e_{i0}{\sigma }_{0i}\right)}^2\right\}{\left|{\int }_{t-\tau }^t\mathrm{d}W\right|}^2.\hfill \end{array}$$

So, we obtain

$$\begin{array}{ccc}& & {\mathcal{L}}_1{V}_1\left(t\right)\hfill \\ & \le & -\frac{{\Lambda }_1}{{\lambda }_{max}\left(P\right)}\epsilon {\left(t\right)}^{\mathrm{T}}(I_N\otimes P)\epsilon \left(t\right)-{\Lambda }_2|\epsilon (t-\tau ){|}^2-\left[w_2-4{\xi }^{-1}\tau {\lambda }_{max}\left(A{A}^{\mathrm{T}}\right)\right]{\int }_{t-\tau }^t{\left|\epsilon \left(s\right)\right|}^2\mathrm{d}s\hfill \\ & & -\left[w_3-4{\xi }^{-1}\tau {\overline{\theta }}^2{\lambda }_{max}^2\left(B{B}^{\mathrm{T}}\right){\lambda }_{max}^2\left(P\right){\lambda }_{max}\left(H{H}^{\mathrm{T}}\right)\right]{\int }_{t-\tau }^t{\left|\epsilon (s-\tau )\right|}^2\mathrm{d}s\hfill \\ & & -{\alpha }_1\left[w_1{\int }_{t-\tau }^t{\left|\epsilon \left(s\right)\right|}^2\mathrm{d}s+w_2{\int }_{-\tau }^0{\int }_{t+\theta }^t{\left|\epsilon \left(s\right)\right|}^2\mathrm{d}s\mathrm{d}\theta +w_3{\int }_{-\tau }^0{\int }_{t+\theta }^t{\left|\epsilon (s-\tau )\right|}^2\mathrm{d}s\mathrm{d}\theta +V_{12}\left(t\right)\right]\hfill \\ & & +{\alpha }_1\left[w_1{\int }_{t-\tau }^t{\left|\epsilon \left(s\right)\right|}^2\mathrm{d}s+w_2{\int }_{-\tau }^0{\int }_{t+\theta }^t{\left|\epsilon \left(s\right)\right|}^2\mathrm{d}s\mathrm{d}\theta +w_3{\int }_{-\tau }^0{\int }_{t+\theta }^t{\left|\epsilon (s-\tau )\right|}^2\mathrm{d}s\mathrm{d}\theta +V_{12}\left(t\right)\right]\hfill \\ & & +{\overline{\theta }}^2{\lambda }_{max}^3\left(P\right){\lambda }_{max}^2\left(B{B}^{\mathrm{T}}\right)max\left\{{\left(e_{i0}{\sigma }_{0i}\right)}^2\right\}-2\sum _{i=1}^N{(s_i\left(t\right)-\delta )}^2\hfill \\ & & +4{\xi }^{-1}{\overline{\theta }}^2{\lambda }_{max}^2\left(P\right){\lambda }_{max}^2\left(B{B}^{\mathrm{T}}\right)max\left\{{\left(e_{i0}{\sigma }_{0i}\right)}^2\right\}{\left|{\int }_{t-\tau }^t\mathrm{d}W\right|}^2\hfill \end{array}$$

(13)

where ${\alpha }_1$ is a positive constant that will be determined later.

From the known condition (7), we have

$$\begin{array}{ccc}& & \frac{k-\xi {\overline{\theta }}^2{\lambda }_{max}\left(H{H}^{\mathrm{T}}\right){\lambda }_{max}^4\left(P\right){\lambda }_{max}^2\left(B{B}^{\mathrm{T}}\right)}{\tau }\hfill \\ & >& 4{\xi }^{-1}\tau {\overline{\theta }}^2{\lambda }_{max}\left(H{H}^{\mathrm{T}}\right){\lambda }_{max}^2\left(P\right){\lambda }_{max}^2\left(B{B}^{\mathrm{T}}\right)+4{\xi }^{-1}\tau {\lambda }_{max}\left(A{A}^{\mathrm{T}}\right)\hfill \end{array}$$

We can select $w_2$ and $w_3$ to satisfy

$$w_2>4{\xi }^{-1}\tau {\lambda }_{max}\left(A{A}^{\mathrm{T}}\right),w_3>4{\xi }^{-1}\tau {\overline{\theta }}^2{\lambda }_{max}\left(H{H}^{\mathrm{T}}\right){\lambda }_{max}^2\left(P\right){\lambda }_{max}^2\left(B{B}^{\mathrm{T}}\right)$$

and

$$w_2+w_3<\frac{k-\xi {\overline{\theta }}^2{\lambda }_{max}\left(H{H}^{\mathrm{T}}\right){\lambda }_{max}^4\left(P\right){\lambda }_{max}^2\left(B{B}^{\mathrm{T}}\right)}{\tau }$$

From the above equation, we have

$$k-\xi {\overline{\theta }}^2{\lambda }_{max}\left(H{H}^{\mathrm{T}}\right){\lambda }_{max}^4\left(P\right){\lambda }_{max}^2\left(B{B}^{\mathrm{T}}\right)-w_2\tau >w_3\tau$$

Now, we can select $w_1$ to satisfy

$$k-\xi {\overline{\theta }}^2{\lambda }_{max}\left(H{H}^{\mathrm{T}}\right){\lambda }_{max}^4\left(P\right){\lambda }_{max}^2\left(B{B}^{\mathrm{T}}\right)-w_2\tau >w_1>w_3\tau$$

which implies ${\Lambda }_1=k-\xi {\overline{\theta }}^2{\lambda }_{max}\left(H{H}^{\mathrm{T}}\right){\lambda }_{max}^4\left(P\right){\lambda }_{max}^2\left(B{B}^{\mathrm{T}}\right)-w_1-w_2\tau >0$ and ${\Lambda }_2=w_1-w_3\tau >0$.

On the other hand, the positive constant ${\alpha }_3$ is selected to satisfy

$$\begin{array}{ccc}& & {\alpha }_1\le 2,{\alpha }_1\le \frac{{\Lambda }_1}{{\lambda }_{max}\left(P\right)},\hfill \\ & & {\alpha }_1\le \frac{w_2-4{\xi }^{-1}\tau {\lambda }_{max}\left(A{A}^{\mathrm{T}}\right)}{w_1+w_2\tau },\hfill \\ & & {\alpha }_1\le \frac{w_3-4{\xi }^{-1}\tau {\overline{\theta }}^2{\lambda }_{max}^2\left(B{B}^{\mathrm{T}}\right){\lambda }_{max}^2\left(P\right){\lambda }_{max}\left(H{H}^{\mathrm{T}}\right)}{w_3\tau }.\hfill \end{array}$$

(14)

Note ${\int }_{-\tau }^0{\int }_{t+\theta }^t{\left|\epsilon \left(s\right)\right|}^2\mathrm{d}s\mathrm{d}\theta \le \tau {\int }_{t-\tau }^t{\left|\epsilon \left(s\right)\right|}^2\mathrm{d}s$ and ${\int }_{-\tau }^0{\int }_{t+\theta }^t{\left|\epsilon (s-\tau )\right|}^2\mathrm{d}s\mathrm{d}\theta \le \tau {\int }_{t-\tau }^t{\left|\epsilon (s-\tau )\right|}^2\mathrm{d}s$, the following inequality can be given from the Equation (13),

$$\begin{array}{ccc}& & {\mathcal{L}}_1{V}_1\left(t\right)\hfill \\ & \le & -{\alpha }_1{V}_1\left(t\right)-{\Lambda }_2{\left|\epsilon (t-\tau )\right|}^2-\left[w_2-4{\xi }^{-1}\tau {\lambda }_{max}\left(A{A}^{\mathrm{T}}\right)-{\alpha }_1(w_1+w_2\tau )\right]{\int }_{t-\tau }^t{\left|\epsilon \left(s\right)\right|}^2\mathrm{d}s\hfill \\ & & -\left[w_3-4{\xi }^{-1}\tau {\overline{\theta }}^2{\lambda }_{max}^2\left(B{B}^{\mathrm{T}}\right){\lambda }_{max}^2\left(P\right){\lambda }_{max}\left(H{H}^{\mathrm{T}}\right)-{\alpha }_1{w}_3\tau \right]{\int }_{t-\tau }^t{\left|\epsilon (s-\tau )\right|}^2\mathrm{d}s\hfill \\ & & -(2-{\alpha }_1)\sum _{i=1}^N{(s_i\left(t\right)-\delta )}^2+{\overline{\theta }}^2{\lambda }_{max}^3\left(P\right){\lambda }_{max}^2\left(B{B}^{\mathrm{T}}\right)max\left\{{\left(e_{i0}{\sigma }_{0i}\right)}^2\right\}\hfill \\ & & +4{\xi }^{-1}{\overline{\theta }}^2{\lambda }_{max}^2\left(P\right){\lambda }_{max}^2\left(B{B}^{\mathrm{T}}\right)max\left\{{\left(e_{i0}{\sigma }_{0i}\right)}^2\right\}{\left|{\int }_{t-\tau }^t\mathrm{d}W\right|}^2\hfill \\ & \le & -{\alpha }_1{V}_1\left(t\right)+{\overline{\theta }}^2{\lambda }_{max}^3\left(P\right){\lambda }_{max}^2\left(B{B}^{\mathrm{T}}\right)max\left\{{\left(e_{i0}{\sigma }_{0i}\right)}^2\right\}\hfill \\ & & +4{\xi }^{-1}{\overline{\theta }}^2{\lambda }_{max}^2\left(P\right){\lambda }_{max}^2\left(B{B}^{\mathrm{T}}\right)max\left\{{\left(e_{i0}{\sigma }_{0i}\right)}^2\right\}{\left|{\int }_{t-\tau }^t\mathrm{d}W\right|}^2\hfill \end{array}$$

By using $\mathrm{d}\left(e^{\gamma t}{V}_1\left(t\right)\right)=\gamma e^{\gamma t}{V}_1\left(t\right)\mathrm{d}t+e^{\gamma t}\mathrm{d}{V}_1\left(t\right)$ and integrating on both sides of the formula, it follows from the Equation (9),

$$\begin{array}{ccc}\hfill e^{\gamma t}\mathrm{E}{V}_1\left(t\right)& =& \mathrm{E}{V}_1\left(0\right)+\gamma \mathrm{E}{\int }_0^t{e}^{\gamma s}{V}_1\left(s\right)\mathrm{d}s+\mathrm{E}{\int }_0^t{e}^{\gamma s}\mathrm{d}{V}_1\left(s\right)\hfill \\ & \le & \mathrm{E}{V}_1\left(0\right)-({\alpha }_1-\gamma )\mathrm{E}{\int }_0^t{e}^{\gamma s}{V}_1\left(s\right)\mathrm{d}s+[{\overline{\theta }}^2{\lambda }_{max}^3\left(P\right){\lambda }_{max}^2\left(B{B}^{\mathrm{T}}\right)max\left\{{\left(e_{i0}{\sigma }_{0i}\right)}^2\right\}\hfill \\ & & +4{\xi }^{-1}\tau nN{\overline{\theta }}^2{\lambda }_{max}^2\left(P\right){\lambda }_{max}^2\left(B{B}^{\mathrm{T}}\right)max\left\{{\left(e_{i0}{\sigma }_{0i}\right)}^2\right\}]\frac{e^{\gamma t}-1}{\gamma }\hfill \\ & \le & \mathrm{E}{V}_1\left(0\right)+\frac{{\mu }_1}{\gamma }{e}^{\gamma t}\hfill \end{array}$$

where $\gamma$ is chosen to satisfy $\gamma <{\alpha }_1$, the symbol E represents the expectation of the random variable, and the positive constant ${\mu }_1$ is defined as follows

$${\mu }_1={\overline{\theta }}^2{\lambda }_{max}^3\left(P\right){\lambda }_{max}^2\left(B{B}^{\mathrm{T}}\right)max\left\{{\left(e_{i0}{\sigma }_{0i}\right)}^2\right\}+4{\xi }^{-1}\tau nN{\overline{\theta }}^2{\lambda }_{max}^2\left(P\right){\lambda }_{max}^2\left(B{B}^{\mathrm{T}}\right)max\left\{{\left(e_{i0}{\sigma }_{0i}\right)}^2\right\}.$$

So, we have

$$\begin{array}{c}\hfill \mathrm{E}{V}_1\left(t\right)\le \mathrm{E}{V}_1\left(0\right)e^{-\gamma t}+\frac{{\mu }_1}{\gamma }\end{array}$$

(15)

Note ${\left|\epsilon \left(t\right)\right|}^2\le \frac{\epsilon {\left(t\right)}^{\mathrm{T}}(I_N\otimes P)\epsilon \left(t\right)}{{\lambda }_{min}\left(P\right)}\le \frac{V_1\left(t\right)}{{\lambda }_{min}\left(P\right)}$, we can obtain

$$\begin{array}{c}\hfill \underset{t\to +\infty }{lim}\mathrm{E}{|x_i\left(t\right)-x_0\left(t\right)|}^2={ϵ}_1\end{array}$$

and ${ϵ}_1$ is a small constant independent of time t. □

Remark 2. 

Under the random noise disturbance, little papers discuss the adaptive consensus of multi-agent systems in the presence of time delays. Theorem 1 indicates that the adaptive control law (4) can ensure that the dynamic error between the followers and the leader can converge to a small number ${ϵ}_1$ in the mean square sense. Looking back at the above proof, it can be found that ${ϵ}_1=\frac{{\mu }_1}{\gamma {\lambda }_{min}\left(P\right)}=\mathsf{\Xi }max\left\{{\left(e_{i0}{\sigma }_{0i}\right)}^2\right\}$, where $\mathsf{\Xi }$ is a constant. So this boundary ${ϵ}_1$ tends to zero when the noise intensity of the system approaches zero.

Remark 3. 

Formula (7) can be transformed into

$$\begin{array}{c}\hfill \tau <\sqrt{\frac{k\xi -{\xi }^2{\overline{\theta }}^2{\lambda }_{max}\left(H{H}^T\right){\lambda }_{max}^4\left(P\right){\lambda }_{max}^2\left(B{B}^T\right)}{4{\overline{\theta }}^2{\lambda }_{max}\left(H{H}^T\right){\lambda }_{max}^2\left(P\right){\lambda }_{max}^2\left(B{B}^T\right)+4{\lambda }_{max}\left(A{A}^T\right)}}.\end{array}$$

So the upper limit of time delay can be obtained as

$$\frac{k}{4\overline{\theta }{\lambda }_{max}^2\left(P\right){\lambda }_{max}\left(B{B}^T\right)\sqrt{{\lambda }_{max}\left(H{H}^T\right)\left[{\overline{\theta }}^2{\lambda }_{max}\left(H{H}^T\right){\lambda }_{max}^2\left(P\right){\lambda }_{max}^2\left(B{B}^T\right)+{\lambda }_{max}\left(A{A}^T\right)\right]}}.$$

Although the constant time delay in this paper cannot be directly extended to time-varying delay, the above formula gives the range of time delay, which can provide some reference for future work.

Now, we further analyze the adaptive control law (4), and investigate whether multi-agent system can still achieve the consensus under $\tau =0$. For this case, the adaptive law is kept unchanged, and the control protocol is constructed as

$$\begin{array}{c}\hfill u_i\left(t\right)=s_i\left(t\right)K\left[\sum _{j\in N_i}{e}_{ij}(x_j\left(t\right)-x_i\left(t\right))+e_{i0}(x_0\left(t\right)-x_i\left(t\right))+e_{i0}{\sigma }_{0i}{\eta }_i\left(t\right)\right],\end{array}$$

(16)

where ${\eta }_i\left(t\right)\in {\mathbf{R}}^n$ is n dimensional standard white noise. The abbreviated form of the error dynamic equation is represented by

$$\begin{array}{c}\hfill \mathrm{d}\epsilon \left(t\right)=[I_N\otimes A-S\left(t\right)H\otimes BK]\epsilon \left(t\right)\mathrm{d}t-(S\left(t\right)E_0{C}_0\otimes BK)\mathrm{d}W.\end{array}$$

(17)

Theorem 2. 

Assuming the digraph $\tilde{\mathcal{G}}=(\tilde{\mathcal{V}},\tilde{\mathcal{N}},\tilde{\mathcal{A}})$ has the same properties as Theorem 1. If the control protocol of the multi-agent system (1) and (2) satisfies (16) and the adaptive law is shown in (4), then the system can achieve the mean square bounded consensus, i.e.,

$$\begin{array}{c}\hfill \underset{t\to +\infty }{lim}E{|x_i\left(t\right)-x_0\left(t\right)|}^2={ϵ}_2\end{array}$$

(18)

where ${ϵ}_2$ is a small constant independent of time t.

Proof. 

The Laypunov function is denoted as

$$\begin{array}{c}\hfill V_2\left(t\right)=V_{21}\left(t\right)+V_{22}\left(t\right)=\epsilon {\left(t\right)}^{\mathrm{T}}(I_N\otimes P)\epsilon \left(t\right)+\sum _{i=1}^N{(s_i\left(t\right)-\delta )}^2,\end{array}$$

where P satisfies the Equation (5). We can obtain from the Itô formula

$$\begin{array}{c}\hfill \mathrm{d}{V}_2\left(t\right)={\mathcal{L}}_2{V}_2\left(t\right)\mathrm{d}t-2\epsilon {\left(t\right)}^{\mathrm{T}}[S\left(t\right)E_0{C}_0\otimes PBK]\mathrm{d}W,\end{array}$$

(19)

and the operator ${\mathcal{L}}_2$ satisfies

$$\begin{array}{ccc}\hfill {\mathcal{L}}_2{V}_2\left(t\right)& =& \epsilon {\left(t\right)}^{\mathrm{T}}[I_N\otimes (PA+A^{\mathrm{T}}P)]\epsilon \left(t\right)-2\epsilon {\left(t\right)}^{\mathrm{T}}[S\left(t\right)H\otimes PBK]\epsilon \left(t\right)+{\dot{V}}_{12}\left(t\right)\hfill \\ & & +\mathrm{tr}\left\{S{\left(t\right)}^2{E}_0^2{C}_0^2\otimes K^{\mathrm{T}}{B}^{\mathrm{T}}PBK\right\},\hfill \end{array}$$

Using the similar method, we can obtain the following equality from the adaptive law (4)

$$\begin{array}{c}\hfill {\dot{V}}_{22}\left(t\right)=2\epsilon {\left(t\right)}^{\mathrm{T}}[S\left(t\right)H\otimes PBK]\epsilon \left(t\right)-\delta \epsilon {\left(t\right)}^{\mathrm{T}}[(H^{\mathrm{T}}+H)\otimes PBK]\epsilon \left(t\right)-2\sum _{i=1}^N{(s_i\left(t\right)-\delta )}^2.\end{array}$$

Lemma 1 indicates that the minimum eigenvalue of the matrix $H^{\mathrm{T}}+H$ satisfies ${\lambda }_{min}(H^{\mathrm{T}}+H)>0$. Using the known condition $\delta >\frac{1}{{\lambda }_{min}(H^{\mathrm{T}}+H)}$, we have

$$\begin{array}{ccc}\hfill {\mathcal{L}}_2{V}_2\left(t\right)& =& \epsilon {\left(t\right)}^{\mathrm{T}}[I_N\otimes (PA+A^{\mathrm{T}}P)]\epsilon \left(t\right)-\delta \epsilon {\left(t\right)}^{\mathrm{T}}[(H+H^{\mathrm{T}})\otimes \left(PBK\right)]\epsilon \left(t\right)\hfill \\ & & +\mathrm{tr}\left\{S{\left(t\right)}^2{E}_0^2{C}_0^2\otimes K^{\mathrm{T}}{B}^{\mathrm{T}}PBK\right\}-2\sum _{i=1}^N{(s_i\left(t\right)-\delta )}^2\hfill \\ & \le & \epsilon {\left(t\right)}^{\mathrm{T}}[I_N\otimes (PA+A^{\mathrm{T}}P)]\epsilon \left(t\right)-\delta {\lambda }_{\min }(H+H^{\mathrm{T}})\epsilon {\left(t\right)}^{\mathrm{T}}[I_N\otimes \left(PBK\right)]\epsilon \left(t\right)\hfill \\ & & +\mathrm{tr}\left\{S{\left(t\right)}^2{E}_0^2{C}_0^2\otimes K^{\mathrm{T}}{B}^{\mathrm{T}}PBK\right\}-2\sum _{i=1}^N{(s_i\left(t\right)-\delta )}^2\hfill \\ & \le & -{k\left|\epsilon \left(t\right)\right|}^2-2\sum _{i=1}^N{(s_i\left(t\right)-\delta )}^2+{\overline{\theta }}^{2}max\left\{{\left(e_{i0}{\sigma }_{0i}\right)}^2\right\}{\lambda }_{max}^3\left(P\right){\lambda }_{max}^2\left(B{B}^{\mathrm{T}}\right)\hfill \\ & \le & -\frac{k}{{\lambda }_{max}\left(P\right)}\epsilon {\left(t\right)}^{\mathrm{T}}(I_N\otimes P)\epsilon \left(t\right)-2\sum _{i=1}^N{(s_i\left(t\right)-\delta )}^2\hfill \\ & & +{\overline{\theta }}^{2}max\left\{{\left(e_{i0}{\sigma }_{0i}\right)}^2\right\}{\lambda }_{max}^3\left(P\right){\lambda }_{max}^2\left(B{B}^{\mathrm{T}}\right)\hfill \\ & \le & -min\left\{\frac{k}{{\lambda }_{max}\left(P\right)},2\right\}{V}_2\left(t\right)+{\overline{\theta }}^{2}max\left\{{\left(e_{i0}{\sigma }_{0i}\right)}^2\right\}{\lambda }_{max}^3\left(P\right){\lambda }_{max}^2\left(B{B}^{\mathrm{T}}\right)\hfill \end{array}$$

From the formula $\mathrm{d}\left(e^{\gamma t}{V}_2\left(t\right)\right)=\gamma e^{\gamma t}{V}_2\left(t\right)\mathrm{d}t+e^{\gamma t}\mathrm{d}{V}_2\left(t\right)$, we can obtain the following inequality from the Equation (19),

$$\begin{array}{ccc}\hfill e^{\gamma t}\mathrm{E}{V}_2\left(t\right)& =& \mathrm{E}{V}_2\left(0\right)+\gamma \mathrm{E}{\int }_0^t{e}^{\gamma s}{V}_2\left(s\right)\mathrm{d}s+\mathrm{E}{\int }_0^t{e}^{\gamma s}\mathrm{d}{V}_2\left(s\right)\hfill \\ & \le & \mathrm{E}{V}_2\left(0\right)-\left(min\left\{\frac{k}{{\lambda }_{max}\left(P\right)},2\right\}-\gamma \right)\mathrm{E}{\int }_0^t{e}^{\gamma s}{V}_2\left(s\right)\mathrm{d}s\hfill \\ & & +{\overline{\theta }}^{2}max\left\{{\left(e_{i0}{\sigma }_{0i}\right)}^2\right\}{\lambda }_{max}^3\left(P\right){\lambda }_{max}^2\left(B{B}^{\mathrm{T}}\right)\frac{e^{\gamma t}-1}{\gamma }\hfill \\ & \le & \mathrm{E}{V}_2\left(0\right)+\frac{{\mu }_2}{\gamma }{e}^{\gamma t}\hfill \end{array}$$

where $\gamma <min\left\{\frac{k}{{\lambda }_{max}\left(P\right)},2\right\}$ and ${\mu }_2={\overline{\theta }}^{2}max\left\{{\left(e_{i0}{\sigma }_{0i}\right)}^2\right\}{\lambda }_{max}^3\left(P\right){\lambda }_{max}^2\left(B{B}^{\mathrm{T}}\right)$. Hence, divide the inequality by $e^{\gamma t}$, it obtains

$$\begin{array}{c}\hfill \mathrm{E}{V}_2\left(t\right)\le \mathrm{E}{V}_2\left(0\right)e^{-\gamma t}+\frac{{\mu }_2}{\gamma }.\end{array}$$

(20)

Finally, the mean square bounded consensus is obtained as follows

$$\begin{array}{c}\hfill \underset{t\to +\infty }{lim}\mathrm{E}{|x_i\left(t\right)-x_0\left(t\right)|}^2={ϵ}_2\end{array}$$

where ${ϵ}_2$ is a small constant independent of time t. □

Remark 4. 

Under the same adaptive law (4), the conditions of Theorem 2 are much simpler than those of Theorem 1, which can greatly expand the application range of the adaptive law in the problem. Moreover, Theorems 1 and 2 show that the adaptive law (4) can ensure that followers can track leader in the mean square sense, regardless of whether the stochastic multi-agent system has a time delay or not.

Remark 5. 

Hu et al. [37] designed a dynamic output-feedback controller by using the relative state information, and achieved the consensus by adjusting the internal state of the controller. Compared with the literature, the consensus in this paper can be achieved by adjusting the adaptive gain of system. Although both control strategies can achieve the consensus, [37] did not consider the impact of time delay, and the adaptive gain is mandatory.

4. Simulation

To analyze the validity of main conclusions, assuming that the system covers one leader and three followers, we conduct numerical simulations in one- and two-dimensional space respectively, and verify that the adaptive control law of this paper can make all followers track the target regardless of whether the system has time delay.

Example 1. 

For the system in one-dimensional space, let the leader be globally reachable, and the digraph ${\mathcal{G}}_1$ formed by the followers be balanced, its adjacency matrix is represented by $\mathcal{A}=\left(\begin{array}{ccc}0& 0& 1\\ 1& 0& 0\\ 0& 1& 0\end{array}\right)$. Using the definition of Laplacian matrix $L_{{\mathcal{G}}_1}$, we can obtain the matrix $H=L_{{\mathcal{G}}_1}+E_0=\left(\begin{array}{ccc}2& 0& -1\\ -1& 1& 0\\ 0& -1& 2\end{array}\right)$, where $E_0=\left(\begin{array}{ccc}1& 0& 0\\ 0& 0& 0\\ 0& 0& 1\end{array}\right)$ is the communication matrix between the leader and the followers. The leader-following multi-agent system is represented by

$$\begin{array}{c}\hfill {\dot{x}}_i\left(t\right)=-0.3x_i\left(t\right)+0.4u\left(t\right),{\dot{x}}_0\left(t\right)=-0.3x_0\left(t\right).\end{array}$$

For the above one-dimensional multi-agent system, taking $k=0.8$, the matrix $P=1.0432$ can be obtained from the Riccati equation $A^{\mathrm{T}}P+PA-PB{B}^{\mathrm{T}}P+kI=0$. After simple calculation, we obtain the adaptive gain matrix $\Gamma =PB{B}^{\mathrm{T}}P=0.1741$. Since the minimum eigenvalue of $H^{\mathrm{T}}+H$ is 1, we take the constant $\delta =1.02$ to ensure $\delta >\frac{1}{{\lambda }_{min}(H^{\mathrm{T}}+H)}$, so the adaptive law can be represented by ${\dot{s}}_i\left(t\right)=0.1741{(x_i\left(t\right)-x_0\left(t\right))}^{\mathrm{T}}{\displaystyle \sum _{j=1}^N}{h}_{ij}(x_j\left(t\right)-x_0\left(t\right))-(s_i\left(t\right)-1.02)$. For the system, if $\tau$ is $0.13$, the noise intensity is $0.23$, the constant $\overline{\theta }$ is 1.4, and the constant $\xi$ is 0.052, then the condition of Theorem 1 holds as $k=0.8>\xi {\overline{\theta }}^2{\lambda }_{max}\left(H{H}^{\mathrm{T}}\right){\lambda }_{max}^4\left(P\right){\lambda }_{max}^2\left(B{B}^{\mathrm{T}}\right)+4{\xi }^{-1}{\tau }^2{\overline{\theta }}^2{\lambda }_{max}\left(H{H}^{\mathrm{T}}\right){\lambda }_{max}^2\left(P\right){\lambda }_{max}^2\left(B{B}^{\mathrm{T}}\right)+4{\xi }^{-1}{\tau }^2{\lambda }_{max}\left(A{A}^{\mathrm{T}}\right)=0.6447$. At this point, Figure 1 shows the trend of tracking error over time, and Figure 2 shows the trajectory of the adaptive gain. Under the combined effects of noise and time delay, it can be seen that the state errors eventually converge to a small range. For the system with $\tau =0$, we maintain the topological structure and dynamic equations of the problem invariable, which means the conditions of Theorem 2 hold. Under the same adaptive control law, the noise intensity is increased to 0.9, the system can still attain the mean square bounded consensus. Figure 3 and Figure 4 show the trajectories of dynamic error and adaptive gain of each agents.

Example 2. 

In the two dimensional space, assuming the leader is globally reachable, the digraph ${\mathcal{G}}_2$ composed of followers is balanced, and the matrix $H=L_{{\mathcal{G}}_2}+E_0=\left(\begin{array}{ccc}2& -1& 0\\ -1& 1& 0\\ 0& 0& 2\end{array}\right)$. The system is represented by

$${\dot{x}}_i\left(t\right)=\left(\begin{array}{cc}0& -0.8\\ 0.8& 0\end{array}\right)x_i\left(t\right)+\left(\begin{array}{cc}0.4& 0.1\\ 0.1& 0.4\end{array}\right)u\left(t\right),{\dot{x}}_0\left(t\right)=\left(\begin{array}{cc}0& -0.8\\ 0.8& 0\end{array}\right)x_0\left(t\right).$$

For two dimensional system with time delay, the constant k in the Riccati equation is taken as 0.4, we can obtain $P=\left(\begin{array}{cc}1.4388& -0.0350\\ -0.0350& 1.6538\end{array}\right)$ and the adaptive gain matrix $\Gamma =PB{B}^{\mathrm{T}}P=\left(\begin{array}{cc}0.3441& 0.1721\\ 0.1721& 0.4559\end{array}\right)$. Due to ${\lambda }_{min}(H^{\mathrm{T}}+H)=0.7693$, the constant $\delta$ in the adaptive control law is taken as 1.35 in order to satisfy $\delta >\frac{1}{{\lambda }_{min}(H^{\mathrm{T}}+H)}$. Let the time delay $\tau =0.0041$, the noise intensity ${\sigma }_{0i}=0.33$, the constant $\overline{\theta }=3.4$, and $\xi =0.0041$, we can obtain that the condition of Theorem 1 holds as $k=0.4>\xi {\overline{\theta }}^2{\lambda }_{max}\left(H{H}^{\mathrm{T}}\right){\lambda }_{max}^4\left(P\right){\lambda }_{max}^2\left(B{B}^{\mathrm{T}}\right)+4{\xi }^{-1}{\tau }^2{\overline{\theta }}^2{\lambda }_{max}\left(H{H}^{\mathrm{T}}\right){\lambda }_{max}^2\left(P\right){\lambda }_{max}^2\left(B{B}^{\mathrm{T}}\right)+4{\xi }^{-1}{\tau }^2{\lambda }_{max}\left(A{A}^{\mathrm{T}}\right)=0.3881$. Figure 5 and Figure 6 describe the trajectory of dynamic error and adaptive gain of system with time delay in a noisy environment. It can be seen that all components of the three followers can track the target. When the time delay disappears, we maintain the above adaptive law unchanged, and then the conditions of Theorem 2 hold. Let the noise intensity ${\sigma }_{0i}=0.24$, and the trends of the dynamic error and the adaptive gain of the system are shown in Figure 7 and Figure 8.

In order to compare the differences between the adaptive control protocol proposed in this paper and some previous papers, we once again simulate the one-dimensional multi-agent system in Example 1, and take the noise intensity as 0.2. Under the mandatory gain $a_i\left(t\right)=\frac{log(1+t)}{1+t}$ and the adaptive law (4), we simulate the dynamic error and the gain of two situations, respectively, as shown in Figure 9 and Figure 10. The black curve represents the situation of mandatory gain, the other colors represent the changes of three agents under the control law (4). From the two figures, it can be seen that the adaptive control protocol proposed in this paper has a faster rate of convergence, so three followers can track the leader in a shorter time. In addition, the mandatory gain will eventually converge to zero, while the adaptive gain (4) will converge to a non-zero constant.

5. Conclusions

For the tracking issues, adaptive control is analyzed in cases both with and without time delay. Firstly, the adaptive control protocol of the stochastic system is given in the presence of time delay, and the adaptive law is designed. The adaptive control law depends on the solution of the Riccati equation and can be abbreviated into matrix form by the Kronecker products. Then, it was proved that the followers can track the target in the mean square sense, and the dynamic error can obtain to a very little constant. Compared with the previous references, the final dynamic error has a smaller value, and when the noise intensity converges to zero, this dynamic error value also trends to zero. It should be noted that the method of proof can not be directly extended to the case of variable delay. In the future, it is meaningful to further explore the adaptive consensus of multi-agent system with variable delay, and the output feedback control with time delay also needs additional investigation.