A Dropout Compensation ILC Method for Formation Tracking of Heterogeneous Multi-Agent Systems with Loss of Multiple Communication Packets

Abstract

In this paper, the formation tracking problem for heterogeneous multi-agent systems with loss of multiple communication packets is considered using the iterative learning control (ILC) method. A dropout compensation ILC method is presented to construct effective distributed iterative learning protocols. The convergence conditions are given based on the frequency-domain analysis by using the general Nyquist stability criterion and Greshgorin’s disk theorem. The results show that the multi-agent system with different packet loss rate can achieve formation tracking without reducing the convergence speed. Numerical simulation results show the effectiveness of the proposed dropout compensation ILC method.

1. Introduction

In recent years, with the rapid development of communication and computing technology, people pay more and more attention to the control of multi-agent systems [1,2,3]. “Agent” can be broadly defined as a computing entity that is adaptive, autonomous, and has a certain life cycle in a particular environment. Drones, driverless cars, robots, etc., can all be called “Agent”. Apart from these, single-agent has played an important role in aerospace, military, and real life. Furthermore, the development of intelligent control, modern communication technologies, artificial intelligence concepts, etc., as well as people inspired by the phenomenon of animal cluster activity in nature, have pushed agent control research into a new stage of development.

By working together, multi-agent can accomplish some parallel, complex tasks, more efficiently than single-agent. For instance, multiple satellites orbiting the Earth to complete designated tasks and multiple drones can work together to transport goods in disaster relief.

In recent years, many researchers have considered applying iterative learning control (ILC) to multi-agent systems with repetitive work. ILC is a tracking control method for systems that require high precision and repeat the same actions. In a study by Ahn and Chen [4], ILC was applied to the formation control of a multi-agent system for the first time. Since then, more and more researchers have applied ILC to the practical application of multi-agent system. Ahn, Moore, and Chen [5] proposed an ILC law to maintain the flight path of satellite formation. Since the satellite rotates the earth periodically, the position related interference can be considered as the time periodic interference. According to Chen and Jia [6], multiple robots can guarantee formation using iterative learning. Furthermore, in order to ensure the safe running interval of trains, Sun, Hou, and Li [7] studied the corresponding ILC scheme of multi train cooperation. In studies by Hock and Schoellig [8] and Schoellig and colleagues [9], a group of four rotor aircrafts kept a given formation and accurately tracked the desired trajectory with ILC.

However, considering the limited capacity of the communication channel, the actual control design based on wireless communication is difficult. In addition, various industrial production processes and equipment are very complex. For these reasons, communication delay, packet loss, quantization and so on will cause the ILC of multi-agent to not converge [10,11]. There are specific researches (see [12,13,14,15,16,17]) on communication delay and quantization, but few on communication packet loss. Zhang and Li [18] designed an asynchronous event triggering protocol to solve the consistency problem of multi-agent system in the case of external interference, parameter uncertainty, time-delay, and packet loss by ΣΔ quantizer. By using the extended Lyapunov–Krasovskii functional, a consistency criterion in the form of matrix inequality was derived to ensure the consistency of multi-agent system with delay and packet loss under H1 controller. Firstly, Zhang and Li [19] considered the packet loss problem in the quantitative ILC of a multi-agent system. Considering the nonlinearity of quantization and the randomness of packet loss, a method of reducing learning gain to obtain accurate tracking was proposed. However, the results showed that the convergence rate slows down with the increase of the packet loss rate. All the results from their study [19] are for homogeneous multi-agent systems and assume that multiple agents have the same packet loss rate. The packet loss compensation for heterogeneous multi-agent systems with different packet loss rates is a more challenging problem.

This paper explores the ILC method with packet loss compensation for a heterogeneous multi-agent system. The question of under what conditions, with the increase of packet loss rate, the convergence speed of iterative learning algorithm will not slow down is solved. Firstly, the ILC model of a heterogeneous multi-agent system is established, and then the frequency domain model is obtained from the time domain model. When different communication packets are lost between agents, the previous iterative data is used to form the current iterative data. ILC in frequency domain is a two-dimensional system in which information propagates on two independent axes: frequency axis and iterative axis. The convergence of ILC is reduced to the convergence of any frequency in the iterative dimension, which is realized by the transformation of frequency and parameter iteration. After the exchange of variables and parameters, Z-transform is performed in the iterative domain. The convergence of the system error is analyzed by using the eigenvalue of the characteristic equation. Since the characteristic equation is in matrix form, it is difficult to get the eigenvalue of the characteristic equation. We use the generalized Nyquist criterion and Gershgorin’s disk theorem to analyze the eigenvalue range of the system and obtain the condition of the system convergence. The simulation results show that the compensation method proposed in this paper can make the system error convergence, and the convergence speed will not slow down.

This paper consists of the following parts. In Section 2, an ILC model in frequency domain is established for heterogeneous multi-agent systems and the control objective for formation tracking of a multi-agent system is given. A dropout compensation ILC method is proposed in Section 3. In Section 4, the convergence conditions are analyzed. The results are verified by numerical simulation in Section 5.

2. Problem Formulation and Preliminaries

2.1. Preliminaries

The interaction topologies in a multi-agent system are modeled by an undirected graph $\mathcal{G}$ = ($\mathcal{V},\mathcal{E},\mathcal{A}$), where $\mathcal{V}=\left\{v_1, \dots , v_n\right\}$ is the set of vertices, $\mathcal{E}\subseteq \mathcal{V}\times \mathcal{V}$ is the set of the edge, and $\mathcal{A}=[a_{ij}], a_{ij}\ge 0$ is the weighted adjacency matrix of the graph $\mathcal{G}$. If $(v_i, v_j)\in \mathcal{E}$, $a_{ij}=a_{ji}>0$, otherwise, $a_{ij}=a_{ji}=0$. Each edge $(v_i, v_j)$ represents information that can be transmitted between the $i$th agent and the $j$th agent. In addition, we assume that $a_{ii}=0$. The neighboring set of agent, $v_i$, is ${\mathcal{N}}_i=\left\{j: (v_i, v_j)\in \mathcal{E}\right\}$. The Laplacian matrix $\mathcal{L}=[l_{ij}]\in {ℝ}^{n\times n}$ of graph $\mathcal{G}$ is denoted as $\mathcal{L}=\mathcal{D}-\mathcal{A}$, where $\mathcal{D}\triangleq \mathrm{diag}\left\{d_1, \cdots , d_n\right\}$ with $d_i={\displaystyle \sum }_{j=1}^n{a}_{ij}$. A path in the undirected graph $\mathcal{G}$ is denoted as a finite sequence $\left\{v_{i_1},v_{i_2},\cdots ,v_{i_j}\right\}$ and $(v_{i_l},v_{i_{l+1}})\in \mathcal{E},l=1,2,\cdots ,j-1$.

2.2. Problem Formulation

2.2.1. Control Objective

Consider the heterogeneous linear SISO (Single Input and Single Output) discrete-time multi-agent system of n agents with topology graph G. The dynamic of the $i\mathrm{th}$ agent is modeled by the following dynamic system.

$$\{\begin{array}{c}{x}_{i,k}(t+1)=A_i{x}_{i,k}(t)+B_i{u}_{i,k}(t)\\ y_{i,k}(t)=C_i{x}_{i,k}(t) \end{array}$$

(1)

where $i=1, 2, \cdots , n$ is the index of the $i\mathrm{th}$ agent, $t=0, 1, 2, \cdots , T$ is the discrete-time index, and $k=1, 2,\cdots$ is the index of iteration. $x_{i,k}(t)\in ℝ$ is the state of the $i$th agent in time $t$ and $k$th iteration. Similarly,$u_{i,k}(t)\in ℝ$ is the control input signal and $y_{i,k}(t)\in ℝ$ is the output signal. Multiple agents have different dynamics which are denoted as $A_i$, $B_i,$ and $C_i$. Taking the Z-transformation of Equation (1), we get

$$Y_{i,k}(z)=\frac{C_i[B_i{U}_{i,k}(z)+x_{i,k}(0)]}{z-A_i}$$

(2)

where $\mathrm{z}=e^{j\omega }, \omega \in [-\pi ,\pi )$ with $j=\sqrt{-1}$ and $\omega$ is frequency, and $x_{i,k}(0)$ is the initial state of $i$th agent in the $k$th iteration; let $G_i(z)=\frac{C_i{B}_i}{z-A_i}$, ${\gamma }_{i,k}(z)=\frac{C_i}{z-A_i}{x}_{i,k}(0)$, then (2) can be rewritten as

$$Y_{i,k}(z)=G_i(z)U_{i,k}(z)+{\gamma }_{i,k}(z)$$

(3)

The desired formation is defined as the relative state values between agents. The relative state values could be position, velocity, and so on. The control objective for the multi-agent systems in this paper is to find an appropriate control input sequence $u_{i,k}(t), i=1, 2, \cdots , n, t=0, 1, 2, \cdots , T$ such that every agent can move in the desired formation tracking trajectory accurately in the sense of expectation as the iterative number $k$ tends to infinity. That is, $\underset{k\to \infty }{\lim }{y}_{i,k}(t)=y_i^d(t)$. $y_i^d(t)$ is the desired output of the $i$th agent and denoted as

$$y_i^d(t)=r(t)+d_i(t)$$

(4)

where $r(t)$, $t=0, 1, 2,\cdots ,T$ is the desired reference trajectory of multi-agent systems. $r(t)$ can also be called the desired trajectory of a virtual leader. In practice, not all agents can obtain $r(t)$, only some of them can. Let $\mathsf{\Phi }=\mathrm{diag}\left\{{\phi }_1, {\phi }_2, \cdots ,{\phi }_n\right\}$ denote the reference-accessibility matrix, which is a diagonal, non-negative, real matrix. If the $i$th agent can obtain direct information about the $r(t)$, then ${\phi }_i>0, i=1, 2, \cdots , n$; otherwise, ${\phi }_i=0, i=1, 2, \cdots , n$. In (4), $d_i(t)$ is the desired output deviation from desired reference trajectory of the $i$th agent. $d_{ij}(t)=d_i(t)-d_j(t)$ represents the desired relative formation between the $i$th agent and the $j$th agent. The formation tracking error ${\delta }_{i,k}(t)$ is denoted as ${\delta }_{i,k}(t)=d_i(t)-y_{i,k}(t)$ and the output error of the $i$th agent in $k$th iteration can be denoted as

$$e_{i,k}(t)=r(t)+{\delta }_{i,k}(t)$$

(5)

By applying the Z-transform to Equation (5), we get

$$E_{i,k}(z)=R(z)+{\Delta }_{i,k}(z)$$

(6)

If the following condition (7) is met

$$\underset{k\to \infty }{\lim }{E}_{i,k}(z)=0$$

(7)

that is, $\underset{k\to \infty }{lim}{Y}_{i,k}(z)=Y_i^d(z)$, the multi-agent systems realize formation tracking control.

2.2.2. Structure of ILC for Networked Multi-Agent Systems

The following ILC law is used to achieve control objectives (7)

$$u_{i,k+1}(t)=u_{i,k}(t)+{\Gamma }_i\left\{{\displaystyle \sum }_{jϵN_i}{a}_{i,j}[{\delta }_{i,k}(t+1)-{\delta }_{j,k}(t+1)]+{\phi }_i{e}_{i,k}(t+1)\right\}$$

(8)

where ${\Gamma }_i\in ℝ$ is the learning gain for ith agent. The formulation of control input of the ith agent requires the control input of the last iteration, error, and information of adjacent agents. Due to the limited bandwidth of communication channel, noise interference, and signal fading, when a multi-agent transmits information through the wireless network, random packet loss is introduced into the multi-agent. For the communication graph of multi-agent systems shown in Figure 1, a block diagram of ILC for multi-agent systems with loss of multiple communication packets is illustrated in Figure 2. When the $i$th agent and the $j$th agent exchange information with each other through the wireless network, the $i$th agent receives a signal from the $j$th agent, which is denoted as

$${\dot{\delta }}_{j,k}^i(t)={\mathsf{\xi }}_{k,ij}{\delta }_{j,k}(t)$$

(9)

where ${\mathsf{\xi }}_{k,ij}$ obeys Bernoulli distribution and ${\delta }_{j,k}(t)$ is the formation tracking error of the $i$th agent. When the $j$th agent sends a signal to the $i$th agent and there is no packet loss in the $k$th iteration, let ${\mathsf{\xi }}_{k,ij}=1$. Otherwise, ${\mathsf{\xi }}_{k,ij}=0$. Since the $i$th agent can obtain its own output $y_{i,k}(t)$ and the desired output deviation $d_i(t)$ without transmission network, there is no packet loss for the $i$th agent itself. That is, ${\delta }_{i,k}(t+1)$ does not have packet loss for the $i$th agent. It is worth noting that, unlike a previous study [18], the scenario discussed in this paper is closer to the engineering practice, assuming that the packet loss rates of communication between agents are different.

3. Dropout Compensation ILC Method for Multi-Agent Systems with Loss of Multiple Communication Packets

In this section, firstly, because the iterative data are changing in the iterative domain, the previous iterative data are used to compensate for the missing data in the iterative process. Secondly, considering that ILC is a two-dimensional system, the convergence of any frequency in the iterative dimension is analyzed by variable iteration and parameter iteration. We make the following assumptions:

Assumption 1.

The dynamics $G_i$ for each agent is stable, that is, $\left|A_i\right|<1$, $i=1, 2,\cdots , n$. Without loss of generality, the system relative degree is one, that is, $G_i\ne 0$.

Remark 1.

If an agent’s dynamics $G_i$ is not stable, we can use the local feedback of its output $y_i$ to make it stable.

Assumption 2.

The initial reset condition satisfies all agents and the desired input for each iteration, that is, $x_{i,k}(0)$ can be abbreviated to $x_i(0)$ and $C_i{x}_{i,k}(0)=y_i^d(0), k\in 1, 2, 3\cdots , i=1, 2,\cdots , n$.

Remark 2.

Assumption 2 is common in ILC of multi-agent systems (see [13,16,19]). If the initial reset condition is not met, we can regard it as the problem of robustness to the initial shifts. In our future work, the ILC with initial-state learning for formation tracking control of multi-agent systems will be explored.

Assumption 3.

The graph $\mathcal{G}$ of multi-agent system (1) is an undirected graph. In addition, its graph is connected, that is, there is a path between any two agents so that they can exchange information.

Lemma 1.

[20]: If an irreducible matrix $M=(a_{i,j})\in {ℂ}^{n\times n},n\ge 1$ is weakly generalized diagonally dominant and at least one of the rows is strictly diagonally dominant, $M$ is nonsingular.

Remark 3.

Assumption 3 implies that the $(\mathcal{L}+\mathsf{\Phi })\in {ℂ}^{n\times n}$. In $\mathcal{L}+\mathsf{\Phi }$, the magnitude of the diagonal entry is ${\phi }_i+{\displaystyle \sum }_{jϵN_i}{a}_{i,j}, i=1,2,\cdots ,n$ and the sum of the magnitudes of all non-diagonal entries is ${\displaystyle \sum }_{jϵN_i}{a}_{i,j}, i=1,2,\cdots ,n$. Since ${\phi }_i+{\displaystyle \sum }_{jϵN_i}{a}_{i,j}\ge {\displaystyle \sum }_{jϵN_i}{a}_{i,j}, i=1,2,\cdots ,n$ and at least one of the rows is satisfied ${\phi }_i+{\displaystyle \sum }_{jϵN_i}{a}_{i,j}>{\displaystyle \sum }_{jϵN_i}{a}_{i,j}$, matrix $\mathcal{L}+\mathsf{\Phi }$ is nonsingular by Lemma.

3.1. Dropout Compensate ILC Method

To achieve the control objective (7), we propose the ILC law to compensate the loss of packets.

$$u_{i,k+1}(t)=u_{i,k}(t)+{\Gamma }_i\left\{{\displaystyle \sum }_{jϵN_i}{a}_{i,j}[{\delta }_{i,k}(t+1)-{\tilde{\delta }}_{j,k}^i(t+1)]+{\phi }_i{e}_{i,k}(t+1)\right\}$$

(10)

When there are packet losses during multiple agents’ communication, we utilize the previous iterative data to compensate this iterative data. In (10), ${\tilde{\delta }}_{j,k}(t+1)$ is data which the ith agent received from the jth agent

$${\tilde{\delta }}_{j,k}^i(t+1)={\mathsf{\xi }}_{k,ij}{\delta }_{j,k}(t+1)+[1-{\mathsf{\xi }}_{k,ij}]{\delta }_{j,k-1}(t+1)$$

(11)

where ${\mathsf{\xi }}_{k,ij}$ is mentioned above. In practice, we can utilize a register to store each iterative data. If there is no packet loss, the next iterative data replace the old data. Otherwise, the old data are retained.

Remark 4.

In order to make the multi-agent system closer to the practical engineering applications, ${\mathsf{\xi }}_{k,ij}$ could be different for different agents. That means the multi-agent system is heterogeneous in this paper.

Since ${\Delta }_{i,k}(z)=D_i(z)-Y_{i,k}(z)$, (6) can be rewritten as

$$E_{i,k}(z)=R(z)+{\Delta }_{i,k}(z)$$

(12)

By combining the Z-transform of (10) with (12), we get

$$U_{i,k+1}(z)=U_{i,k}(z)+z{\Gamma }_i\left\{{\displaystyle \sum }_{jϵN_i}{a}_{i,j}[E_{i,k}(z)-{\tilde{E}}_{j,k}(z)]+{\phi }_i{E}_{i,k}(z)\right\}$$

(13)

where ${\tilde{E}}_{j,k}(z)={\mathsf{\xi }}_{k,ij}{E}_{j,k}(z)-(1-{\mathsf{\xi }}_{k,ij})E_{j,k-1}(z)$. According to Equations (3), (11)–(13) and Assumption 2, we have

$$\begin{array}{c}{E}_{i,k+1}(z)-E_{i,k}(z)\\ =-Y_{i,k+1}(z)+Y_{i,k}(z)\\ =G_i(z)[U_{i,k}(z)-U_{i,k+1}(z)]\\ =-z{G}_i(z){\Gamma }_i\left\{{\displaystyle {\displaystyle \sum }_{jϵN_i}}{a}_{i,j}[E_{i,k}(z)-{\tilde{E}}_{j,k}(z)]+{\phi }_i{E}_{i,k}(z)\right\}\\ =-z{G}_i(z){\Gamma }_i\left\{{\displaystyle {\displaystyle \sum }_{jϵN_i}}{a}_{i,j}[E_{i,k}(z)-{\mathsf{\xi }}_{k,ij}{E}_{j,k}(z)-(1-{\mathsf{\xi }}_{k,ij})E_{j,k-1}(z)]+{\phi }_i{E}_{i,k}(z)\right\}\end{array}$$

(14)

Let $H_i(z)=G_i(z){\Gamma }_i$, $H(z)=diag\left\{H_1(z),H_2(z)\cdots H_n(z)\right\}$, $H_i(z)\in ℂ$, $H(z)\in {ℂ}^{n\times n}$. Let $E_k(z)={[E_{1,k}(z),E_{2,k}(z),\cdots ,E_{n,k}(z)]}^T$, $E_k(z)\in {ℂ}^{n\times 1}$, then (14) can be rewritten as

$$E_{k+1}(z)-E_k(z)=-zH(z)[({\mathcal{L}}_{\mathsf{\xi }}+\mathsf{\Phi })E_k(z)-{\mathcal{A}}_{\mathsf{\xi }}{E}_{k-1}(z)]$$

(15)

where ${\mathcal{L}}_{\mathsf{\xi }}=[l_{ij,\mathsf{\xi }}]\in {ℝ}^{n\times n}$ and ${\mathcal{A}}_{\mathsf{\xi }}=[a_{ij,\mathsf{\xi }}]\in {ℝ}^{n\times n}$,

$$\begin{array}{c}{l}_{ij,\mathsf{\xi }}=\{\begin{array}{c}-a_{i,j}{\mathsf{\xi }}_{k,ij}, jϵN_i\\ {\displaystyle {\displaystyle \sum }_{jϵN_i}}{a}_{i,j}, i=j\\ \hspace{1em}\hspace{1em} 0, others \end{array},\\ a_{ij,\mathsf{\xi }}=\{\begin{array}{c}{a}_{i,j}[1-{\mathsf{\xi }}_{k,ij}], jϵN_i\\ 0, others\end{array}.\end{array}$$

(16)

The relationship among the (k − 1)th, the kth, and the (k + 1)th system error is shown in (15). Our goal is to converge the system error when the number of iterations tends to infinity. Thus, (15) will be further analyzed in the iterative domain.

3.2. Analysis in Iterative Domain

ILC is a method to control systems operating in a repetitive mode over a finite interval with the requirement that the system accurately follows the desired output trajectory. The same actions are executed again and again. By using the previously executed information, the tracking accuracy is improved from repetition to repetition. It has long been recognized that ILC is a two-dimensional (2D) system in which information travels on two independent axes: the time axis and the iteration axis. Chow and Fang [21,22] proposed ILC 2D analysis methods for discrete-time and continue-time systems. After Z-transformation of system dynamics, ILC is considered as a 2D system in frequency domain and iteration domain in this paper. The following analysis is based on the decomposition of 2D dynamics into two separate 1D problems. $E_k(z)$ is considered as a 2D function $E(k,z)$. Exchanging variable frequency and parameter iteration, the convergence of the system is considered as a 1D problem in the iteration domain. Thus, we exchange variable $z$ and parameter $k$. $E_k(z)$ can be rewritten as $E_z(k)$, that is, $E_z(k)={[E_{1,z}(k),E_{2,z}(k),\cdots ,E_{n,z}(k)]}^T$, ${\mathsf{\xi }}_{k,ij}$ can be rewritten as ${\mathsf{\xi }}_{ij}(k)$. Then, (15) can be rewritten as

$$E_z(k+1)-E_z(k)=-zH(z)[(\mathcal{L}(k)+\mathsf{\Phi })E_z(k)-\mathcal{A}(k)E_z(k-1)]$$

(17)

where $\mathcal{L}(k)=[l_{ij}(k)]\in {ℝ}^{n\times n}$ and $\mathcal{A}(k)=[a_{ij}(k)]\in {ℝ}^{n\times n}$,

$$\begin{array}{c}{l}_{ij}(k)=\{\begin{array}{c}-a_{i,j}{\mathsf{\xi }}_{ij}(k), jϵN_i\\ \hspace{1em}{\displaystyle {\displaystyle \sum }_{jϵN_i}}{a}_{i,j}, i=j\\ \hspace{1em}\hspace{1em}\hspace{1em} 0, others \end{array},\\ a_{ij}(k)=\{\begin{array}{c}{a}_{i,j}[1-{\mathsf{\xi }}_{ij}(k)], jϵN_i\\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em} 0, others\end{array}.\end{array}$$

(18)

Take Z-transformation for (17)

$$\mathrm{𝕫}{\mathbb{E}}_z(\mathrm{𝕫})-{\mathbb{E}}_z(\mathrm{𝕫})=-zH(z)[(\mathcal{L}(\mathrm{𝕫})+\mathsf{\Phi }){\mathbb{E}}_z(\mathrm{𝕫})-{\mathrm{𝕫}}^{-1}\mathcal{A}(\mathrm{𝕫}){\mathbb{E}}_z(\mathrm{𝕫})]$$

(19)

where $\mathcal{L}(\mathrm{𝕫})=[l_{ij}(\mathrm{𝕫})]\in {ℝ}^{n\times n}$ and $\mathcal{A}(\mathrm{𝕫})=[a_{ij}(\mathrm{𝕫})]\in {ℝ}^{n\times n}$,

$$\begin{array}{c}{l}_{ij}(\mathrm{𝕫})=\{\begin{array}{c}-a_{i,j}{\mathsf{\xi }}_{ij}(\mathrm{𝕫}), jϵN_i\\ \hspace{1em}{\displaystyle {\displaystyle \sum }_{jϵN_i}}{a}_{i,j}, i=j\\ \hspace{1em}\hspace{1em}\hspace{1em} 0, others \end{array},\\ a_{ij}(\mathrm{𝕫})=\{\begin{array}{c}{a}_{i,j}[1-{\mathsf{\xi }}_{ij}(\mathrm{𝕫})], jϵN_i\\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em} 0, others\end{array}.\end{array}$$

(20)

(19) leads to

$$\left\{(\mathrm{𝕫}-1)I+zH(z)[{\mathcal{L}}^{\prime }(\mathrm{𝕫})+\mathsf{\Phi }]\right\}{\mathbb{E}}_z(\mathrm{𝕫})=0$$

(21)

where ${\mathcal{L}}^{\prime }(\mathrm{𝕫})=[l_{ij}^{\prime }(\mathrm{𝕫})]\in {ℝ}^{n\times n}$,

$$l_{ij}^{\prime }(\mathrm{𝕫})=\{\begin{array}{c}-a_{i,j}{\mathrm{𝕫}}^{-1}[\mathrm{𝕫}{\mathsf{\xi }}_{ij}(\mathrm{𝕫})+1-{\mathsf{\xi }}_{ij}(\mathrm{𝕫})], jϵN_i\\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em} {\displaystyle {\displaystyle \sum }_{jϵN_i}}{a}_{i,j}, i=j\\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em} 0, others\end{array}$$

(22)

From Equation (21), we can obtain characteristic equation of the system in iterative domain. In the next section, the convergence of the learning system is discussed based on the characteristic equation.

4. Convergence Analysis

As we know, when we say that an iterative learning algorithm converges, it means that the system error trends to zero as the number of iterations approaches infinity. For the proposed dropout compensation ILC method, we will analyze the convergence of formation tracking for the multi-agent systems in this section.

Theorem 1.

Consider the multi-agent system (1), let Assumptions 1–3 hold and the learning law (10) be applied. Given that any one of the following conditions is satisfied for all $i=1,2,\cdots ,n$,

$$\begin{array}{c}1. (1>A_i>0)\wedge (P_{i,1}>0)\wedge (P_{i,2}>0)\wedge (P_{i,3}>0)\wedge (P_{i,4}>0)\wedge (P_{i,5}>0),\\ 2. (-1<A_i<0)\wedge (P_{i,1}>0)\wedge (P_{i,2}<0)\wedge (P_{i,3}>0)\wedge (P_{i,4}<0)\wedge (P_{i,5}>0),\end{array}$$

(23)

where

$$\begin{array}{c}{P}_{i,1}=2-Q_i,\\ P_{i,2}=-16{(1-A_i)}^2+4(1-A_i)(3-A_i)Q_i+(A_i-2)Q_i{}^2,\\ P_{i,3}=2{(1-A_i)}^2-(1-A_i)Q_i,\\ P_{i,4}=-4{(1-A_i)}^2+2(1-A_i)(3-A_i)Q_i+(A_i-2)Q_i{}^2,\\ P_{i,5}=4{(1-A_i)}^2-4(1-A_i)Q_i+S_i,\\ Q_i={\Gamma }_i{B}_i{C}_i(K_i+{\phi }_i),\\ S_i={({\Gamma }_i{B}_i{C}_i)}^2(2K_i{\phi }_i+{\phi }_i{}^2),\\ K_i={\displaystyle {\displaystyle \sum }_{j\in N_i}}{a}_{i,j}.\end{array}$$

(24)

then the formation tracking objective (7) is achieved.

Proof. 

Let $S(\mathrm{𝕫},z)=\det \left\{(\mathrm{𝕫}-1)I+zH(z)[{\mathcal{L}}^{\prime }(\mathrm{𝕫})+\mathsf{\Phi }]\right\}$. The characteristic equation is $S(\mathrm{𝕫},z)=0$. When $\mathrm{𝕫}=1$, ${\mathcal{L}}^{\prime }(1)=\mathcal{L}$. Thus, $S(1,z)=\det [zH(z)(\mathcal{L}+\mathsf{\Phi })]$. In Remark 3, $\mathcal{L}+\mathsf{\Phi }$ proves to be nonsingular. Therefore, matrix $zH(z)(\mathcal{L}+\mathsf{\Phi })$ is nonsingular also and $S(1,z)\ne 0$. □

When $\mathrm{𝕫}\ne 1$, we need to prove that the modules of all roots of (25) are smaller than units.

$$\det \left\{I+\frac{zH(z)[\mathcal{L}(\mathrm{𝕫})+\mathsf{\Phi }]}{\mathrm{𝕫}-1}\right\}=0$$

(25)

Let $F(\mathrm{𝕫},z)=\frac{zH(z)[\mathcal{L}(\mathrm{𝕫})+\mathsf{\Phi }]}{\mathrm{𝕫}-1}$, based on the general Nyquist stability criterion [23], the modules of all roots of (25) are smaller than units, if the eigenloci $\lambda [F(\omega ,\beta )]$ of

$$F(\omega ,\beta )=\frac{e^{j\beta }H(e^{j\beta })[\mathcal{L}(e^{j\omega })+\mathsf{\Phi }]}{e^{j\omega }-1}$$

(26)

does not enclose the point $(-1,j0)$ for $\beta \in [-\pi ,\pi )$, $\omega \in [-\pi ,\pi ),$ and $\omega \ne 0$. Based on Greshgorin’s disk theorem, we have $\lambda [F(\omega ,\beta )]\in {{\displaystyle \mathrm{U}}}_{i=1}^n{F}_i$ for all $\beta \in [-\pi ,\pi )$, $\omega \in [-\pi ,\pi ),$ and $\omega \ne 0$, where

$$\begin{array}{cc}{F}_i=\{\varsigma :\varsigma \in C,|& \varsigma -\frac{e^{j\beta }{H}_i(e^{j\beta })({{\displaystyle \sum }}_{j\in N_i}{a}_{i,j}+{\phi }_i)}{e^{j\omega }-1}|\\ & \le {\displaystyle {\displaystyle \sum }_{j\in N_i}}\left|\frac{a_{i,j}{e}^{j\beta }{H}_i(e^{j\beta })[e^{j\omega }{\mathsf{\xi }}_{ij}(e^{j\omega })+1-{\mathsf{\xi }}_{ij}(e^{j\omega })]}{e^{j\omega }(e^{j\omega }-1)}\right|\}\end{array}$$

(27)

Since ${\xi }_{ij}(e^{j\omega })$ can only be $0$ or $1$, $e^{j\omega }{\xi }_{ij}(e^{j\omega })+1-{\xi }_{ij}(e^{j\omega })$ is $1$ or $e^{j\omega },$ respectively. Taking $H_i(e^{j\beta })$ as $\frac{{\Gamma }_i{B}_i{C}_i}{(e^{j\beta }-A_i)}$, we can rewrite (27) as

$$F_i=\left\{\varsigma :\varsigma \in C,\left|\varsigma -\frac{e^{j\beta }{\Gamma }_i{B}_i{C}_i(K_i+{\phi }_i)}{(e^{j\omega }-1)(e^{j\beta }-A_i)}\right|\le \left|\frac{K_i{\Gamma }_i{B}_i{C}_i}{(e^{j\omega }-1)(e^{j\beta }-A_i)}\right|\right\}$$

(28)

So the eigenloci $\lambda [F(\omega ,\beta )]$ does not enclose the point $(-1,j0)$ for $\beta \in [-\pi ,\pi )$, $\omega \in [-\pi ,\pi ),$ and $\omega \ne 0$ when the point $(-a,j0)$ with $a\ge 1$ is not in the disc $F_i$ for all $i=1,2,\cdots ,n$, $\beta \in [-\pi ,\pi )$, $\omega \in [-\pi ,\pi ),$ and $\omega \ne 0$. That is, $\left|a+e^{j\beta }{\Gamma }_i{B}_i{C}_i(K_i+{\phi }_i){[(e^{j\omega }-1)(e^{j\beta }-A_i)]}^{-1}\right|>\left|K_i{\Gamma }_i{B}_i{C}_i{[(e^{j\omega }-1)(e^{j\beta }-A_i)]}^{-1}\right|$ for all $i=1,2,\cdots ,n$, $\beta \in [-\pi ,\pi )$, $\omega \in [-\pi ,\pi ),$ and $\omega \ne 0$, when $a\ge 1$. We denote $f_i(a)$ as

$$f_i(a)={\left|a(e^{j\omega }-1)(e^{j\beta }-A_i)+e^{j\beta }{\Gamma }_i{B}_i{C}_i(K_i+{\phi }_i)\right|}^2$$

(29)

Note that ${\left|a+\frac{e^{j\beta }{\Gamma }_i{B}_i{C}_i(K_i+{\phi }_i)}{(e^{j\omega }-1)(e^{j\beta }-A_i)}\right|}^2-{\left|\frac{K_i{\Gamma }_i{B}_i{C}_i}{(e^{j\omega }-1)(e^{j\beta }-A_i)}\right|}^2={\left|\frac{1}{(e^{j\omega }-1)(e^{j\beta }-A_i)}\right|}^2{f}_i(a)$ for all $a\ge 1$, $\beta \in [-\pi ,\pi )$, $\omega \in [-\pi ,\pi ),$ and $\omega \ne 0$. So, ${\left|a+\frac{e^{j\beta }{\Gamma }_i{B}_i{C}_i(K_i+{\phi }_i)}{(e^{j\omega }-1)(e^{j\beta }-A_i)}\right|}^2-{\left|\frac{K_i{\Gamma }_i{B}_i{C}_i}{(e^{j\omega }-1)(e^{j\beta }-A_i)}\right|}^2>0$ as long as $f_i(a)>0$ for all $a\ge 1$, $\beta \in [-\pi ,\pi )$, $\omega \in [-\pi ,\pi ),$ and $\omega \ne 0$. Further, we can show that

$$\begin{array}{c}{f}_i(a)=2(1-\cos \omega )(1+A_i^2-2A_i\cos \beta )a^2\\ +2a{\Gamma }_i{B}_i{C}_i(K_i+{\phi }_i)[\cos \omega -1-A_i\cos (\omega -\beta )+A_i\cos (\beta )]+{({\Gamma }_i{B}_i{C}_i)}^2(2K_i{\phi }_i+{\phi }_i^2).\end{array}$$

(30)

Using the conditions of Theorem 1, we can prove $f_i(a)>0$ for all $a\ge 1$, $\beta \in [-\pi ,\pi )$, $\omega \in [-\pi ,\pi ),$ and $\omega \ne 0$ (see Appendix of [17]). Then the eigenloci of $\lambda [F(\omega ,\beta )]$ for all $\beta \in [-\pi ,\pi )$, $\omega \in [-\pi ,\pi ),$ and $\omega \ne 0$ does not enclose the point $(-1,j0)$. The modules of all roots of (25) are smaller than units. That is, the system achieves a consensus asymptotically.

Multiply $\mathrm{𝕫}-1$ on both sides of Equation (21) and take the limit

$$\underset{\mathrm{𝕫}\to 1}{\lim }\left\{\mathrm{𝕫}(\mathrm{𝕫}-1)I+zH(z)[{\mathcal{L}}^{\prime }(\mathrm{𝕫})+\mathsf{\Phi }]\right\}(\mathrm{𝕫}-1){\mathbb{E}}_z(\mathrm{𝕫})=0$$

(31)

According to the final value theorem, we get

$$\left\{zH(z)[\mathcal{L}+\mathsf{\Phi }]\right\}{E}_z(\infty )=0$$

(32)

As the previous certificate, $det\left\{zH(z)[\mathcal{L}+\mathsf{\Phi }]\right\}\ne 0$. Thus, $rank\left\{zH(z)[\mathcal{L}+\mathsf{\Phi }]\right\}=n$. Based on Sylvester inequality, we get $rank\left\{zH(z)[\mathcal{L}+\mathsf{\Phi }]\right\}+rank(E_z(\infty ))\le n$. Thus $rank[E_z(\infty )]=0$ and the final value of $E_z(k)$ is ${\mathbf{0}}_n$. Theorem 1 is thus proved.

When considering random communication packet dropout, the ILC algorithm can converge using the compensation method we proposed. When considering the random packet loss, the ILC algorithm can use this proposed compensation method to converge. Theorem 1 gives the convergence condition of learning gain design and proves that the convergence condition is independent of communication packet loss. According to the final value theorem, the error of formation tracking converges to zero as the number of iterations goes to infinity.

5. Simulation

Firstly, consider the multi-agent system in Section 2.2.1. and let Assumptions 1–3 hold. Next, the learning law (10) is applied and the learning gains satisfy the condition (23). The results of numerical simulations are discussed in this section.

In order to illustrate the effectiveness of the proposed dropout compensation ILC method, we considered a heterogeneous multi-agent system consisting of four agents in the undirected graph as shown in Figure 3. It can be seen that the graph is a connected graph and satisfies Assumption 3. Only the second agent and the third agent can obtain the desired reference trajectory.

The Laplacian matrix $\mathcal{L}$ of graph $\mathcal{G}$ and the reference-accessibility matrix $\mathsf{\Phi }$ were

$$\begin{array}{c}\mathcal{L}=\left[\begin{array}{ccc}4& -1.7& \begin{array}{cc}-2.3& 0\end{array}\\ -1.7& 1.7& \begin{array}{cc}0& 0\end{array}\\ \begin{array}{c}-2.3\\ 0\end{array}& \begin{array}{c}0\\ 0\end{array}& \begin{array}{cc}\begin{array}{c} 3.8\\ -1.5\end{array}& \begin{array}{c}-1.5\\ 1.5\end{array}\end{array}\end{array}\right],\\ \mathsf{\Phi }=\left[\begin{array}{ccc}0& 0& \begin{array}{cc}0& 0\end{array}\\ 0& 1& \begin{array}{cc}0& 0\end{array}\\ \begin{array}{c}0\\ 0\end{array}& \begin{array}{c}0\\ 0\end{array}& \begin{array}{cc}\begin{array}{c}1\\ 0\end{array}& \begin{array}{c}0\\ 0\end{array}\end{array}\end{array}\right].\end{array}$$

Then it was obtained that the information interaction weight between agents can be obtained, as shown in

Table 1. Interaction weight of agent.

$K_1=4$	$K_2=1.7$	$K_3=3.8$	$K_4=1.5$
${\phi }_1=0$	${\phi }_2=1$	${\phi }_3=1$	${\phi }_4=0$

.

For every agent in Figure 3, the system parameters were defined in

Table 2. System parameters.

$A_1=-0.16$	$A_2=-0.18$	$A_3=-0.12$	$A_4=-0.32$
$B_1=0.416$	$B_2=-0.15$	$B_3=0.6$	$B_4=0.05$
$C_1=-0.38$	$C_2=-0.7$	$C_3=0.53$	$C_4=0.59$

.

Let ${\Gamma }_1=-0.813$, ${\Gamma }_2=1.31$, ${\Gamma }_3=0.57,$ and ${\Gamma }_4=0.92$. Then, we can obtain the parameters in

Table 3. Convergence condition parameter.

$Q_1=0.514$	$Q_2=0.371$	$Q_3=0.870$	$Q_4=0.041$
$S_1=0$	$S_2=0.083$	$S_3=0.283$	$S_4=0$
$P_{1,1}=1.486$	$P_{2,1}=1.629$	$P_{3,1}=1.130$	$P_{4,1}=1.959$
$P_{1,2}=-14.563$	$P_{2,2}=-17.005$	$P_{3,2}=-9.514$	$P_{4,2}=-27.168$
$P_{1,3}=2.095$	$P_{2,3}=2.347$	$P_{3,3}=1.534$	$P_{4,3}=3.431$
$P_{1,4}=-2.184$	$P_{2,4}=-3.083$	$P_{3,4}=-0.542$	$P_{4,4}=-6.617$
$P_{1,5}=2.997$	$P_{2,5}=3.900$	$P_{3,5}=1.402$	$P_{4,5}=6.755$

for Theorem 1.

Therefore, the learning gains satisfied the second condition in Theorem 1. To perform our simulation tests, we considered the desired reference trajectory as

$$r(t)=1.5+\sin \frac{\pi t}{25}$$

(33)

and the desired output deviation of each agent as

$$\begin{array}{c}{d}_1(t)=0,d_2(t)=-1.5-\sin \frac{\pi t}{25},\\ d_3(t)=1.5+\sin \frac{\pi t}{25},d_4(t)=0.03t.\end{array}$$

(34)

Set the initial state of four agents as: $x_1(0)=-4.277$, $x_2(0)=0$, $x_3(0)=6.1333$, and $x_4(0)=2.806$, which satisfies Assumption 2. When the iteration step is 500 and $t=1,2,\cdots ,150$, the trajectory of the multi-agent system without packet loss is shown as Figure 4. It can be seen that the desired formation is well achieved to track the desired reference trajectory by the desired output deviation.

In order to measure the formation accuracy quantitatively, the disagreement among all agents on their output errors was defined as: $error(k)=\frac{1}{nT}{\displaystyle \sum }_{i=1}^n{\displaystyle \sum }_{t=1}^T\left|e_{i,k}(t)\right|$. When $\underset{k\to \infty }{\lim }error(k)=0$, multi-agent system achieved formation tacking. Figure 5 shows that formation performance of the system at the first 500 iterations. Obviously, when the iteration is around 350 the system error can converge to zero and achieve formation tracking.

It is assumed that there is no compensation when considering communication packet loss, that is

$$u_{i,k+1}(t)=u_{i,k}(t)+{\Gamma }_i\left\{{{\displaystyle \sum }}_{jϵN_i}{a}_{i,j}[{\delta }_{i,k}(t+1)-{\mathsf{\xi }}_{k,ij}{\delta }_{j,k}(t+1)]+{\phi }_i{e}_{i,k}(t+1)\right\}$$

(35)

$w{\mathrm{here} \mathsf{\xi }}_{k,ij}=1$ when there is no packet loss, otherwise, ${\mathsf{\xi }}_{k,ij}=0$. In Figure 6, it can be seen that the system error cannot converge to zero in different packet loss rates as iteration increases. When iteration is 500 and packet dropout rate is 10%, the trajectory of the multi-agent system without packet loss compensation is shown as Figure 7.

When there are different packet loss rates in a multi-agent system (Figure 8), it can be seen that system errors can converge to zero using (10) to compensate packet loss. The convergence speed hardly gets slower as the packet loss rates increase, as shown in Figure 8b. In addition, the multi-agent system error still can converge to zero around the 350th iteration. When iteration is 350 and packet loss rate is 10%, the trajectory of the multi-agent system with packet loss compensation is shown as in Figure 9.

In order to demonstrate the effectiveness of proposed method, the comparison results with the method of Zhang and Li [19] are shown in Figure 10. If there is data loss, the information for the current iteration will be discarded without any compensation. Therefore, although the error curve can converge to 0, the convergence speed gets slower as the packet loss rate increases [19], while the convergence speed hardly gets slower, as shown in Figure 8b, by the proposed method.

6. Conclusions

ILC-based formation tracking for heterogeneous multi-agent systems is a challenging problem since different communication packets are lost between agents. When packet loss occurs at different rates, the tracking error of ILC does not converge. This paper explores the ILC method of packet loss compensation for a heterogeneous multi-agent system. Since the iterative data are constantly changing in the iterative domain, it is effective to use previous data to compensate for the lost data in this iterative process. Based on the ILC model for multi-agent systems in frequency domain, the convergence of the system is considered as a 1D problem in the iterative domain, and the convergence of the system is analyzed. By using the generalized Nyquist criterion and Gershgorin’s disk theorem, the eigenvalue range of the system characteristic equation is analyzed, and the convergence condition of the system error is obtained. The dropout compensation ILC method proposed in this paper can not only make the error converge to zero with the increase of iteration times, but also the maintain the convergence speed under the packet loss scenario. The numerical simulation results demonstrate the effectiveness of the proposed method.