Formation Control of Multiple Autonomous Underwater Vehicles under Communication Delay, Packet Discreteness and Dropout

Abstract

Effective communication between multiple autonomous underwater vehicles (AUVs) is necessary for formation control. As the most reliable underwater communication method, acoustic communication still has many constraints compared with radio communication, which affects the effectiveness of formation control. Therefore, this paper proposes a formation control scheme for multiple AUVs under communication delay, packet discreteness and dropout. Firstly, the communication delay is estimated based on the kernel density estimation method. To solve the problem of packet discreteness and dropout, the curve fitting method is used to predict the states of the AUV. Secondly, a follower controller is designed based on the leader–follower approach using input–output feedback linearization, which is proven to be stable with Lyapunov stability theory. Then, some simulation results are presented to demonstrate the stability and accuracy of the formation control in different communication environments. Finally, the field tests on the lake show that the scheme introduced in this paper is valid and practical.

1. Introduction

Autonomous underwater vehicles (AUVs) are widely used in military and commercial fields due to their small size, low cost, high degree of autonomy and flexible deployment. They are irreplaceable tools in applications such as accident rescue, seabed topographic mapping, object detection, observation of ocean phenomena and marine resource development [1,2,3,4,5,6,7,8]. With the increasing complexity of underwater tasks, it is necessary to improve work efficiency through the cooperation of multiple AUVs, which can even complete tasks that cannot be performed by a single AUV [9]. At the same time, formation control has become a fundamental issue in the cooperative control of multiple AUVs. In recent years, many scholars have carried out research on the formation control of multiple agents and the common methods are as follows: the virtual structure method [10,11]; the behavior-based strategy [12]; the leader–follower approach [13,14,15,16,17,18,19,20]; the potential field function approach [21]; the path-following method [22]; and the consensus theory method [23,24]. Due to the complicated underwater environment, most methods cannot be directly applied to the formation control of multiple AUVs. However, the leader–follower approach is the most widely used in engineering because of its simple structure and easy expansion [25]. In this method, the leader moves according to a given trajectory, and the followers track the leader to maintain the desired formation structure. Therefore, formation control can be achieved by changing the predefined trajectory of the leader and the expected structure of the formation.

In order to realize formation control, AUVs need to exchange some key information with each other through wireless communication. At present, there are three main wireless communication methods: radio communication, optical communication and acoustic communication. Radio signals decay quickly underwater and optical communication range is short and requires high-quality water. Therefore, these two communication methods are rarely used in underwater environments. Acoustic communication is the most effective method because of its slow attenuation speed and long transmission distance [26,27]. However, acoustic communication also has non-negligible limitations, including delay, path loss, limited bandwidth, multipath and so on [28,29]. Acoustic communication is one of the most significant impacts on formation control schemes due to communication delay, packet discreteness and dropout. At present, many scholars have studied underwater wireless sensor networks (UWSN) to solve communication constraint problems. In [30], the author surveys recent routing protocols for UWSN to inspire researchers to design efficient routing protocols. In [31], a probabilistic weight-based energy-efficient cluster routing is presented for large-scale wireless sensor networks protocol. Simulation results show that this protocol has better performance in many aspects, such as delay, throughput and packet delivery rate. In [32], Wang proposes a dual prediction data reduction approach for wireless sensor networks based on the Kalman filter. With respect to data reduction, data accuracy and energy consumption, the suggested method has produced the best results. The above work contributes significantly to the improvement of communication effectiveness. In this paper, we use the designed network protocol and solve the problems of communication delay, packet discreteness and dropout in formation control.

Communication delay consists of the propagation delay and parsing delay of the packet. Propagation delay refers to the travel time of the packets underwater, which is related to the underwater acoustic transmission speed and transmission distance. Parsing delay is the conversion time between acoustic and electrical signals, which is associated with packet size, communication protocol and other factors. The communication delay has the greatest influence on formation control. Although it can be calculated by the timestamping of each sent packet, this will increase the packet size and communication delay [33]. Many researchers have focused on dealing with communication delay and they usually assume that communication delay is constant, bounded, or small. Some of the representative results are presented below. In [34], Yang assumes that communication delay is constant and the simulation results demonstrate that the decoupled controller proposed in the paper can tolerate a constant delay. In [35], Yan designs a coordinated control protocol under a bounded communication delay, which is less than the communication period. It turns out that the system with delay is still stable but takes more time than the system without delay. In [36], Chen proposes a formation control method for a homogeneous and a heterogeneous AUV group combining the consensus theory and leader–follower method under communication delay. However, time-varying communication delay is assumed to have a fixed upper limit in the paper. In [37], a distributed leader–follower formation controller is proposed based on state feedback and consensus algorithms under a time-varying differentiable delay. Ref. [38] regards communication delay as a function. Both the function and its derivative have an upper bound and a leader–follower controller is designed to achieve formation control. All the above methods have solved the problem of formation control under communication delay to some extent. However, in actual applications, the assumption that the communication delay is constant or bounded is unreasonable because the propagation delay and parsing delay are difficult determine when AUVs run in an unknown environment [39]. The assumption that communication delay is small is also very dangerous, as the proposed control method may be unstable if the actual delay is too large [40]. Therefore, it is not certain whether the above approaches are extremely practical in formation control under communication delay.

Packet discreteness and dropout in acoustic communication are mainly caused by path loss, noise and multipath propagation. It has rarely been considered in previous studies. In [41], Chen proposes a distributed event-triggered communication mechanism to overcome the problems of packet discreteness and dropout. This method is mainly ap-plied to linear models and is not easily implemented in engineering. Most researchers assume that AUVs can communicate underwater continuously with acoustic communication [42]. However, it is obviously unreasonable in practical applications. On the one hand, the high-powered acoustic modem cannot work continuously for a long time. On the other hand, packet dropout is inevitable and can also be considered a special case of packet discreteness. Discrete packets will reduce the frequency of information exchange between multiple AUVs, which decreases the accuracy and stability of formation control. As a result, packet discreteness and dropout are also important issues that must be considered for formation control.

Based on the above discussion, this paper is concerned with the formation control of multiple AUVs under communication delay, packet discreteness and dropout. The main contributions are as follows:

(1)

The communication delay is estimated based on the kernel density estimation method. Kernel density estimation is a non-parametric estimation method that does not need prior knowledge and an accurate mathematical model of communication delay. Instead, we can obtain an accurate distribution of delay with this method according to the characteristics and properties of delay values based on underwater experiments. Compared with other methods, this method has more extensive applications.

(2)

The packet discreteness and dropout problems are solved by information prediction based on the curve fitting method. This method can be used to predict the key states of the leader AUV to generate a continuous and precise trajectory for the follower AUVs.

(3)

We derive a kinematic model for the error of the formation control system. The follower controller is designed using the input–output feedback linearization method and the stability of this method is proved by Lyapunov stability theory.

(4)

Both the simulation in MATLAB and the field tests on the lake are carried out to verify the feasibility of the scheme presented in this paper.

This paper is organized as follows. Section 2 includes the kinematic model of the AUV and the mathematical description of formation control. Section 3 proposes methods to solve the communication delay, packet discreteness and dropout problems. Section 4 presents the formation control scheme of multiple AUVs. Section 5 and Section 6 show the simulation and test results. Finally, Section 7 gives the conclusions and future perspectives.

2. Problem Formulation and Preliminaries

2.1. AUV Model

In actual engineering applications, AUVs run in a three-dimensional underwater environment. However, most underwater missions, such as ocean floor surveys, only require multiple AUVs to move at a constant depth. Therefore, without loss of generality, only the movement in the horizontal plane is considered. Roll, pitch and heave are ignored in this paper. To focus on the communication constraints, the ocean currents and obstacles in the underwater environment are ignored. Thus, the kinematics equation with 3-DOF of the AUV is presented as follows:

$${\dot{\eta }}_i=J\left({\psi }_i\right){\nu }_i$$

(1)

where $N$ represents the number of AUVs; ${\eta }_i={[x_i,y_i,{\psi }_i]}^{\mathrm{T}},i=1,2,\dots ,N$ represents the pose of AUV_i in the global coordinate frame, consisting of the position $(x_i,y_i)$ and yaw ${\psi }_i\in \left[0,2\pi \right)$ of AUV_i; $v_i={[u_i,v_i,r_i]}^{\mathrm{T}}$ is the standard velocity vector of AUV_i in the body coordinate frame; $u_i$ and $v_i$ are respectively the velocity in surge and sway; $r_i$ is the yaw velocity. The transformation matrix $J\left({\psi }_i\right)$ can be described as

$$J\left({\psi }_i\right)=\left[\begin{array}{ccc}\cos ({\psi }_i)& -\sin ({\psi }_i)& 0\\ \sin ({\psi }_i)& \cos ({\psi }_i)& 0\\ 0& 0& 1\end{array}\right]$$

(2)

2.2. Description of Formation Control

In the leader–follower formation control method, the leader moves according to a given trajectory and the followers maintain a desired distance $L^d\in R^{(N-1)\times 1}$ and angle ${\Phi }^d\in R^{(N-1)\times 1}$ from the leader. The relationship between the leader AUV₁ and the follower AUV_i is shown in Figure 1, where $O_1=(x_1,y_1)$, $O_i=(x_i,y_i)$ and ${O^{\prime }}_i=({x^{\prime }}_i,{y^{\prime }}_i)$ are the leader’s position, the follower’s position, and the follower’s desired position, respectively; $d$ denotes the distance between the center of mass and the front of the AUV; $\left(L_i^d,{\Phi }_i^d\right)$ represents the desired distance and angle relative to the leader. $l={[l_x,l_y]}^{\mathrm{T}}$ and $l^d={[l_x^d,l_y^d]}^{\mathrm{T}}$ are the actual distance and the desired distance between the leader and the follower in the body coordinate frame of the leader, respectively.

$$l^d={[l_x^d,l_y^d]}^{\mathrm{T}}=L_i^d{\left[\cos ({\Phi }_i^d),\sin ({\Phi }_i^d)\right]}^{\mathrm{T}}$$

(3)

According to Equation (1), the actual distance can be calculated as follows:

$$l={[l_x,l_y]}^{\mathrm{T}}=G({\Psi }_1){\left[x_1-x_i-d\sin \left({\Psi }_i\right),y_1-y_i-d\cos \left({\Psi }_i\right)\right]}^{\mathrm{T}}$$

(4)

where $G({\Psi }_1)=\left[\begin{array}{cc}\cos ({\psi }_1)& -\sin ({\psi }_1)\\ \sin ({\psi }_1)& \cos ({\psi }_1)\end{array}\right]$ and ${\Psi }_1\in \left[0,2\pi \right)$ is the yaw of the leader AUV₁. The tracking errors of the follower AUV_i are defined as:

$$E={\left[e_x,e_y\right]}^{\mathrm{T}}=l-l^d={[l_x-l_x^d,l_y-l_y^d]}^{\mathrm{T}}$$

(5)

where $e_x$ and $e_y$ are the errors in the x and y directions, respectively. Based on the above analysis, the formation control problem can be transformed into a problem of the follower tracking a series of desired target points. Formation control can be realized by designing a follower controller to eliminate tracking errors.

3. Solutions of Communication Delay, Packet Discreteness and Dropout

To achieve formation control, each follower AUV needs constant and accurate information from the leader AUV. However, when the AUV moves underwater, the conditions of acoustic communication become complex, which causes communication delay, packet discreteness and dropout. The information flow of the communication from the leader to the follower is shown in Figure 2. The horizontal axis above represents the time when a series of packets are sent by the leader AUV, where the number in the square boxes represents the serial number of packets sent by the leader AUV (i.e., 1, 2, 3, 4 in the square boxes), $T_i$ is the time when the packet $i$ is sent by the leader, $T$ represents a fixed interval between the sent packets. The horizontal axis below represents the time when a series of packets are received by the follower AUV, where the numbers in the squares represent the serial number of packets received by the follower AUV (i.e., 1, 3, 4 in the squares), $t_i$ denotes the time when the packet $i$ is received by the followers, $\left\{{\tau }_i\right\}$ is a set of communication delay (for example, packet 1 is sent and received at $T_1$ and $t_1$, respectively, then ${\tau }_1$ can be calculated as ${\tau }_1=t_1-T_1$) and $\left\{t_i^{\prime }\right\}$ is a set of the discrete time between the received packets. It can be seen from Figure 2 that the sent packets may be received after a period of time or not. For example, packet 1 sent at time $T_1$ is received by the follower at time $t_1$, but packet 2 sent at time $T_2$ is not received. In this case, packet 1 is received by the follower with a communication delay, and packet 2 sent by the leader is lost.

In short, the follower cannot obtain constant and accurate states of the leader because of communication delay, packet discreteness and dropout. In Section 3.1, the kernel density method is used to build the communication delay distribution model and generate the delay subject to the distribution model. Furthermore, in Section 3.2, we use the curve fitting method to predict the states of the leader to solve the problems of packet discreteness and dropout.

3.1. Estimation of Communication Delay

Communication delay is composed of the propagation delay and parsing delay of the packets. In practice, the propagation delay is related to the speed and distance of propagation. The propagation speed of the sound is affected by environmental factors such as temperature, depth, salinity, etc. Therefore, we adopt a propagation model approved by the United Nations Educational, Scientific and Cultural Organization (UNESCO) to calculate the speed of sound underwater [43]. Then, the propagation delay can be approximately expressed as:

$${\tau }_1^{\prime }=\frac{\Vert O_1{O}_i\Vert }{v_{\mathrm{sound}}}$$

(6)

where $\Vert O_1{O}_i\Vert$ is the Euclidean distance between the leader and the follower and $v_{\mathrm{sound}}$ is the speed of the sound.

Based on the experimental data, the parsing delay is obtained using the kernel density method. Let $t_0{}^1,t_0{}^2,\dots ,t_0{}^n$ be independent and identically distributed samples of the parsing delay. Let $f\left(x\right)$ be the probability density function of the samples. Then, the estimated probability density function [44] can be written as:

$$\widehat{f}(x)=\frac{1}{2nh}{\displaystyle \sum _{i=1}^n{K}_h\left(x-x_i\right)}=\frac{1}{nh}{\displaystyle \sum _{i=1}^{n}K\left(\frac{x-x_i}{h}\right)}$$

(7)

where $h>0$ is the bandwidth, a neighborhood of the variable $x$; $n$ is the total number of samples; $K\left(x\right)$ is a kernel function and the following three equations are satisfied.

$$\{\begin{array}{l}K\left(x\right)\ge 0\\ K\left(x\right)=K\left(-x\right)\\ {\displaystyle {\int }_{-\infty }^{+\infty }K\left(x\right)}dx=1\end{array}$$

(8)

According to Equation (7), given the total number of samples $n$, $\widehat{f}(x)$ is associated with the bandwidth $h$ and kernel function $K\left(x\right)$. If $h$ is too small, the number of samples used to estimate $f(x)$ is also small, resulting in a decrease in the bias and an increase in the variance of the estimation. Conversely, if $h$ is too large, we can obtain an inaccurate estimation with a higher bias and lower variance. Therefore, how to obtain the best $h$ is extremely critical.

Define the integral mean square error function as:

$$MISE(h)=E\int {(\widehat{f}(x)-f(x))}^{2}dx$$

(9)

To minimize $MISE(h)$, the optimal bandwidth is calculated as:

$$h_{\mathrm{opt}}=\arg \underset{h}{\min }MISE(h)={\left\{\frac{\int K^2(t)dt}{n{\left[\int t^{2}K(t)dt\right]}^2\int {\left[f^{″}(x)\right]}^{2}dx}\right\}}^{\frac{1}{5}}$$

(10)

where $f″\left(x\right)$ can be approximately replaced by $\widehat{f}″(x)$.

The Gaussian function has convenient mathematical properties and satisfies the requirements of the kernel function, so it is chosen as the kernel function, that is, $K\left(x\right)=\Phi \left(x\right)$. Then, the best bandwidth can be calculated according to Equation (10).

$$h_{\mathrm{opt}}=1.05\times n^{-\frac{1}{5}}$$

(11)

By combining Equations (7) and (11), we can obtain the estimated probability density function:

$$\widehat{f}(x)=0.907n^{-\frac{3}{5}}{\displaystyle \sum _{i=1}^n\Phi \left(x-x_i\right)}$$

(12)

After that, we introduce the method of generating the parsing delay that conforms to the distribution. Let the random variable $Y$ obey a uniform distribution between 0 and 1, i.e., $Y\sim U\left(0,1\right)$. Define the random variable of the parsing delay as:

$$X=\Psi \left(Y\right)$$

(13)

where $\Psi (\cdot )$ is a monotonically increasing mapping function from $Y$ to $X$. Then,

$$F\left(x\right)=P\left(X\le x\right)=P\left(Y\le {\Psi }^{-1}\left(x\right)\right)=G\left({\Psi }^{-1}\left(x\right)\right)$$

(14)

where $F(\cdot )$ and $G(\cdot )$ are the distribution functions of $X$ and $Y$, respectively. Hence, we can obtain $G\left(y\right)=y$ and

$${\Psi }^{-1}\left(x\right)=F\left(x\right)={\displaystyle \underset{-\infty }{\overset{x}{\int }}\widehat{f}(t)}dt$$

(15)

The mapping function $\Psi (\cdot )$ can be determined according to Equation (15). By combining Equations (13) and (15), the parsing delay can be calculated as:

$${\tau }_2^{\prime }=\Psi \left(Random\left(0,1\right)\right)$$

(16)

where ${\tau }_2^{\prime }$ represents the parsing delay and $Random\left(0,1\right)$ denotes the uniformly distributed random number generated by the computer in the interval $\left(0,1\right)$.

Finally, we can obtain the propagation delay and parsing delay of acoustic communication from Equations (6) and (16), respectively:

$$\tau ={\tau }_1^{\prime }+{\tau }_2^{\prime }$$

(17)

where $\tau$ is the estimation of the communication delay.

3.2. Prediction of Leader States

When multiple AUVs move in formation underwater, the followers can only receive discrete states of the leader due to packet discreteness and dropout. To improve the accuracy and stability of formation control, the followers need continuous states of the leader. Therefore, the curve fitting method is applied to predict the states of the leader [45]. The reliable and discrete packet of leader states received by the followers is defined as $\left(x_1\left(t_1\right),y_1\left(t_1\right)\right),\left(x_1\left(t_2\right),y_1\left(t_2\right)\right),\dots ,\left(x_1\left(t_K\right),y_1\left(t_K\right)\right)$, where $K$ represents the number of received packets. Let $k$ be the minimum number of packets required f the curve fitting method. If $K$ is not less than $k$, a third-degree polynomial (simulation results show that a higher-degree polynomial does not mean a more precise prediction) is used to predict the x-coordinate of the leader.

$$\widehat{x_1} \left(m\right)=A_x{m}^3+B_x{m}^2+C_{x}m+D_x$$

(18)

To obtain polynomial parameters, linear equations are expressed in matrix form:

$${\rm T}M=X$$

(19)

where ${\rm T}=\left[\begin{array}{cccc}{t}_1^3& t_1^2& t_1& 1\\ t_2^3& t_2^2& t_2& 1\\ \dots & \dots & \dots & \dots \\ t_K^3& t_K^2& t_K& 1\end{array}\right]$, $M={\left[\begin{array}{cccc}{A}_x& B_x& C_x& D_x\end{array}\right]}^{\mathrm{T}}$ and $X={\left[\begin{array}{cccc}{x}_1\left(t_1\right)& x_1\left(t_2\right)& \dots & x_1\left(t_k\right)\end{array}\right]}^{\mathrm{T}}$.

The pseudoinverse method is used to solve Equation (19). We can then obtain:

$$M=T^{LM}X={\left(T^{\mathrm{T}}T\right)}^{-1}{T}^{\mathrm{T}}X$$

(20)

where $T^{LM}={\left(T^{\mathrm{T}}T\right)}^{-1}{T}^{\mathrm{T}}$ is the left inverse of $T$. Then, when $t\in \left(t_K,t_{K+1}\right)$, the x-coordinate of the leader is calculated as:

$$\widehat{x_1} \left(t\right)=P(t)M$$

(21)

where $P(t)=\left[\begin{array}{cccc}{t}^3& t^2& t& 1\end{array}\right]$.

The y-coordinate of the leader can also be predicted in the same way, i.e., $\widehat{y_1} \left(m\right)=A_y{m}^3+B_y{m}^2+C_{y}m+D_y$. Referring to Equations (19)–(21), the y-coordinate of the leader can be obtained as:

$$\widehat{y_1} \left(t\right)=P(t)N$$

(22)

where $N={\left[\begin{array}{cccc}{A}_y& B_y& C_y& D_y\end{array}\right]}^{\mathrm{T}}$. Equations (21) and (22) are the predictions of the leader states.

4. Formation Control Scheme

In this paper, it is assumed that the leader can move along the predefined trajectory. Then, the formation control scheme can be understood as the path tracking algorithm for the followers, composed of the generation of continuous trajectories of the leader and the design of the controller for the followers. The former provides an accurate reference trajectory for the followers under communication delay, packet discreteness and dropout. The latter designs a motion controller for the followers to achieve tracking control of the desired path. Figure 3 shows the system control block diagram and the dashed box is a low-level closed-loop controller of velocity, which was designed in previous work and is not the focus of this paper.

4.1. Generate Continuous Trajectories of the Leader

In this section, combined with Section 3.1 and Section 3.2, the algorithm pseudocode of a continuous trajectory of the leader is shown in Algorithm 1. Define the information of the leader received by the followers as $\left(x_1\left(t_i\right),y_1\left(t_i\right),u_1\left(t_i\right),{\Psi }_1\left(t_i\right)\right)$, where $i=1,2,\dots$ represents the sequence number of the packets.

Algorithm 1. Continuous Trajectories of the Leader

1:  The followers receive packets from the leader $\left(x_1\left(t_i\right),y_1\left(t_i\right),u_1\left(t_i\right),{\Psi }_1\left(t_i\right)\right)$

2: if $K<k$ 3:  if$t=t_i$, where $t$ denotes the time 4:   then we obtain the communication delay using Equation (17) and the states of the leader are corrected 5: $$\widehat{x_1} \left(t_i\right)=x_1\left(t_i\right)+\tau u_1\left(t_i\right)\sin \left(\Psi \left(t_i\right)\right)$$ (23) $$\widehat{y_1} \left(t_i\right)=y_1\left(t_i\right)+\tau u_1\left(t_i\right)\cos \left(\Psi \left(t_i\right)\right)$$ (24) 6:  else $t\ne t_i$ 7:   then continuous trajectories of the leader are predicted 8: $$\widehat{x_1} \left(t\right)=x_1\left(\arg \underset{t_i}{\min }(t-t_i)\right)+\left(t-t_i\right)u_1\left(\arg \underset{t_i}{\min }(t-t_i)\right)\sin \left(\arg \underset{t_i}{\min }(t-t_i)\right)$$ (25) $$\widehat{y_1} \left(t\right)=y_1\left(\arg \underset{t_i}{\min }(t-t_i)\right)+\left(t-t_i\right)u_1\left(\arg \underset{t_i}{\min }(t-t_i)\right)\cos \left(\arg \underset{t_i}{\min }(t-t_i)\right)$$ (26)

9:  else$K\ge k$ 10:  if$t=t_i$ 11:   then we obtain the communication delay using Equation (17) and the states of the leader are corrected 12: $$\widehat{x_1} \left(t_i\right)=x\left(t_i+\tau \right)=P(t_i+\tau )M$$ (27) $$\widehat{y_1} \left(t_i\right)=y\left(t_i+\tau \right)=P(t_i+\tau )N$$ (28) 13:  else $t\ne t_i$ 14:   then $M$ and $N$ are updated using Equation (20) and the continuous trajectories of the leader are predicted according to Equations (21) and (22) 15: end

4.2. Design the Follower Controller

In the formation control of multiple AUVs, the desired distance $L^d$ and angle ${\Phi }^d$ between the leader and the followers can be constant to maintain a fixed formation or time-variant to form formations of different structures. In this section, the follower controller is designed to track the leader with an expected distance and angle to achieve formation control. Firstly, the system error kinematic model is derived based on the formation model in Section 2.2. Then, a stable follower controller is designed with the input–output feedback linearization method. Finally, the stability of the controller is proved using Lyapunov stability theory.

Taking the derivative of Equations (3)–(5), we can obtain:

$$\stackrel{·}{e_x}=\stackrel{·}{l_x}-\stackrel{·}{l_x^d}$$

(29)

$$\stackrel{·}{l_x^d}=\stackrel{·}{L_i^d}\cos \left({\Phi }_i^d\right)-L_i^d\stackrel{·}{{\Phi }_i^d}\sin \left({\Phi }_i^d\right)$$

(30)

$$\begin{array}{c}\stackrel{·}{l_x}=-\stackrel{·}{{\Psi }_1}\sin \left({\Psi }_1\right)\left(x_1-x_i-d\sin \left({\Psi }_i\right)\right)+\cos \left({\Psi }_1\right)\left(\stackrel{·}{x_1}-\stackrel{·}{x_i}-d\stackrel{·}{{\Psi }_i}\cos \left({\Psi }_i\right)\right)\\ -\stackrel{·}{{\Psi }_1}\cos \left({\Psi }_1\right)\left(y_1-y_i-d\cos \left({\Psi }_i\right)\right)-\sin \left({\Psi }_1\right)\left(\stackrel{·}{y_1}-\stackrel{·}{y_i}+d\stackrel{·}{{\Psi }_i}\sin \left({\Psi }_i\right)\right)\\ =-r_1{l}_y+u_1-\stackrel{·}{x_i}\cos \left({\Psi }_1\right)+\stackrel{·}{y_i}\sin \left({\Psi }_1\right)-d{r}_i\cos \left({\Psi }_1-{\Psi }_i\right)\end{array}$$

(31)

where $r_1=\stackrel{·}{{\Psi }_1}$ and $r_i=\stackrel{·}{{\Psi }_i}$ are the angular velocities of the leader and follower, respectively and $u_1=\stackrel{·}{x_1}\cos \left({\Psi }_1\right)-\stackrel{·}{y_1}\sin \left({\Psi }_1\right)$ represents the linear velocity of a surge by the leader. Then, Equation (31) can be simplified as:

$$\begin{array}{c}\stackrel{·}{l_x}=-r_1{l}_y+u_1-\left(\stackrel{·}{y_i}\cos \left({\Psi }_i\right)+\stackrel{·}{x_i}\sin \left({\Psi }_i\right)\right)\sin \left({\Psi }_e\right)\\ +\left(\stackrel{·}{y_i}\sin \left({\Psi }_i\right)-\stackrel{·}{x_i}\cos \left({\Psi }_i\right)\right)\cos \left({\Psi }_e\right)-d{r}_i\cos \left({\Psi }_e\right)\end{array}$$

(32)

where ${\Psi }_e={\Psi }_1-{\Psi }_i$ denotes the error of yaw. In addition, we consider an underactuated AUV and ignore the linear velocity in sway. Therefore, we can obtain:

$$\{\begin{array}{c}\stackrel{·}{y_i}\cos \left({\Psi }_i\right)+\stackrel{·}{x_i}\sin \left({\Psi }_i\right)=0\\ \stackrel{·}{x_i}\cos \left({\Psi }_i\right)-\stackrel{·}{y_i}\sin \left({\Psi }_i\right)=u_i\end{array}$$

(33)

where $u_i$ represents the linear velocity of a surge by the follower. Then, Equation (32) can be written as:

$$\stackrel{·}{l_x}=-r_1{l}_y+u_1-u_i\cos \left({\Psi }_e\right)-d{r}_i\cos \left({\Psi }_e\right)$$

(34)

Combining Equations (29), (30) and (34) yields:

$$\stackrel{·}{e_x}=-r_1\left(L_i^d\sin \left({\Phi }_i^d\right)+e_y\right)+u_1-u_i\cos \left({\Psi }_e\right)-d{r}_i\cos \left({\Psi }_e\right)-\stackrel{·}{L_i^d}\cos \left({\Phi }_i^d\right)+L_i^d\stackrel{·}{{\Phi }_i^d}\sin \left({\Phi }_i^d\right)$$

(35)

Similarly, we can obtain:

$$\stackrel{·}{e_y}=r_1\left(L_i^d\cos \left({\Phi }_i^d\right)+e_x\right)-u_i\sin \left({\Psi }_e\right)+d{r}_i\sin \left({\Psi }_e\right)-\stackrel{·}{L_i^d}\sin \left({\Phi }_i^d\right)-L_i^d\stackrel{·}{{\Phi }_i^d}\cos \left({\Phi }_i^d\right)$$

(36)

Therefore, the system error kinematic model can be given by:

$$\stackrel{·}{E}=\left[\begin{array}{c}\stackrel{·}{e_x}\\ \stackrel{·}{e_y}\end{array}\right]=AE+B{U}_i+C$$

(37)

where $U_i={\left[u_i,r_i\right]}^{\mathrm{T}}$ is the control input of the system, $A=\left[\begin{array}{cc}0& -r_1\\ r_1& 0\end{array}\right]$, $B=\left[\begin{array}{cc}-\cos \left({\Psi }_e\right)& -d\cos \left({\Psi }_e\right)\\ -\sin \left({\Psi }_e\right)& d\sin \left({\Psi }_e\right)\end{array}\right]$ and $C=\left[\begin{array}{c}-r_1{L}_i^d\sin \left({\Phi }_i^d\right)-\stackrel{·}{L_i^d}\cos \left({\Phi }_i^d\right)+L_i^d\stackrel{·}{{\Phi }_i^d}\sin \left({\Phi }_i^d\right)+u_1\\ r_1{L}_i^d\cos \left({\Phi }_i^d\right)-\stackrel{·}{L_i^d}\sin \left({\Phi }_i^d\right)-L_i^d\stackrel{·}{{\Phi }_i^d}\cos \left({\Phi }_i^d\right)\end{array}\right]$.

For Equation (37), we use the input–output feedback linearization method and obtain:

$$U_i=B^{-1}(-KE-AE-C)$$

(38)

where $K=\left[\begin{array}{cc}{k}_1& 0\\ 0& k_2\end{array}\right]$ and $k_1,k_2>0$. The formation control law of the system is obtained using Equation (38).

Lyapunov stability theory is used to prove that the system is stable. Define the Lyapunov function as:

$$V=\frac{1}{2}\left(e_x^2+e_y^2\right)$$

(39)

Obviously, the function $V$ is positive definite. The derivative of $V$ is:

$$\stackrel{·}{V}=e_x\stackrel{·}{e_x}+e_y\stackrel{·}{e_y}$$

(40)

Considering Equation (38), Equation (40) can be written as:

$$\stackrel{·}{V}=-k_1{e}_x^2-k_1{e}_y^2$$

(41)

where $\stackrel{·}{V}$ is negative definite. Therefore, the control system is asymptotically stable according to Lyapunov stability theory.

5. Simulation Results

To verify the effectiveness and robustness of the proposed scheme, four different methods of formation control were simulated in MATLAB under the same experimental scenario and different communication environments.

5.1. Simulation Environment

• Experimental methods

Method-1. Ignoring the problems of communication delay, packet discreteness and drop, we only adopt the controller designed in Section 4.2. In this method, it is assumed that the states of the leader at time $t_i$ remain constant for the followers until the next time the packet is successfully received by the followers.

Method-2. Considering the actual communication conditions, the problems of the communication delay, packet discreteness and dropout are solved in Section 3, the continuous trajectories of the leader are generated in Section 4.1 and the follower controller is designed in Section 4.2. In this method, $k=5$ is used to predict the state of the leader.

Method-3. We use the formation control method combining the consensus theory and leader–follower method under communication delay in [36]. The communication delay is assumed to have a fixed upper limit.

Method-4. The formation control method under event-triggered mechanism is adopted as in [41], which can overcome the problems of packet discreteness and dropout.

• Experimental scenario

The multiple AUV system consists of one leader AUV and three followers AUV_i, I = 1,2,3. The leader moves along a predefined trajectory and the followers keep a relative distance and angle from the leader to form a square formation. The motion parameters and formation structures of the four AUVs are shown in

Table 1. Motion parameters and formation structures of multiple AUVs.

Parameter	Leader AUV	Follower AUV1	Follower AUV2	Follower AUV3
Starting point (x, y)	(0,0)	(0,0)	(0,0)	(0,0)
Yaw (rad)	$\pi /2$	$\pi /2$	$\pi /2$	$\pi /2$
Linear velocity (m/s)	1.0289	none *	none	none
Yaw velocity (rad/s)	−0.0175	none	none	none
$\mathrm{Formation} \mathrm{structures} (L^d,{\Phi }^d$)	none	$\left(10,\pi \right)$	$\left(5\sqrt{2},\frac{5\pi }{4}\right)$	$\left(5\sqrt{2},\frac{3\pi }{4}\right)$
Maximum linear velocity (m/s)	1.5433	1.5433	1.5433	1.5433
Maximum yaw velocity (rad/s)	0.1	0.1	0.1	0.1

* None means that the parameter is not fixed or nonexistent.

.

• Communication environments

Underwater acoustic communication environments are simulated in the simulation and the evaluation indexes of the communication quality include communication delay, the interval of sent packets and packet dropout rate.

Table 2. Different communication environments.

Parameter	Environment-1	Environment-2	Environment-3	Environment-4
Communication delay (s)	$N\left(3,0.1\right)$ *	$N\left(3,0.1\right)$	$N\left(5,0.1\right)$	$N\left(3,0.1\right)$
Interval of sent packets (s)	1	2	1	1
Packet dropout rate (%)	20	20	20	40

* The communication delay is subject to distribution $N\left(3,0.1\right)$.

shows the different communication environments in the simulation.

5.2. Simulation Results and Discussion

Simulation results in Environment-1 are shown in Figure 4, Figure 5 and Figure 6 and Figure 7a. Figure 4a,b provide the motion trajectories of AUVs with Method-1 and Method-2, respectively, which are the direct performance of the formation control scheme. It can be seen from Figure 4a that with Method-1, the follower can follow the leader to some extent but cannot accurately maintain the desired square formation. Method-2 is used in Figure 4b, in which multiple AUVs move in a square formation and the effectiveness and robustness are significantly better than those of Method-1.

Figure 5 shows the effect of solving the problems of communication delay, packet discreteness and dropout. Figure 5a illustrates the results of the communication delay estimation, where $Dela{y}_r$, $Dela{y}_p$ and $Dela{y}_e$ denote the real communication delay between the leader AUV and the follower AUV₁, the estimation of the delay and the error of the estimated delay, respectively. As we can see from the figure, the error of the estimated delay is small and the effectiveness of the kernel density method is significant. The prediction error of the states of the leader is shown in Figure 5b. The figure shows that the follower can accurately predict the states of the leader.

Figure 6 shows the formation errors of Method-1 and Method-2. In Figure 6a, with respect to the desired distance of the formation $L^d$, there is a significant error in Method-1, while the error gradually converges in Method-2. The same result is shown in Figure 6b regarding the desired angle of the formation ${\Phi }^d$. For further analysis, Figure 7a shows the tracking errors between the follower AUV₁ and the desired target point in Method-1 and Method-2, i.e., $e_x$ and $e_y$. In Method-1, due to the communication delay, packet discreteness and dropout, the follower cannot obtain a continuous and accurate motion trajectory of the leader, so the tracking errors are not converged. In Method-2, the communication delay is estimated, and the states of the leader are predicted so that the follower obtains a continuous and accurate motion trajectory of the leader. Although Method-2 adopts the same follower controller as Method-1, the tracking errors converge asymptotically.

Figure 7b–d propose the tracking errors between the follower AUV₁ and the desired target point in the other communication environments. It is clear that compared with Method-2, the errors of Method-1 do not converge and fluctuate greatly in different communication environments. Specifically, the interval of sent packets is larger in Figure 7b than in Figure 7a, resulting in a greater fluctuation of the tracking errors. There is a longer delay time in Figure 7c than in Figure 7a, causing an increase in the average tracking error. Compared with Figure 7a, the packet dropout rate is higher in Figure 7d, resulting in a larger maximum tracking error, average tracking error and a greater fluctuation of the tracking errors. However, the tracking errors always converge to zero in Method-2, indicating that this method can be applied for formation control under different communication environments.

Table 3. Simulation results in Method-1.

Parameter	Environment-1	Environment-2	Environment-3	Environment-4	Average
Errors of distance (m)	3.902	4.988	5.754	4.658	4.826
Errors of angle (rad)	0.226	0.261	0.284	0.253	0.256

and

Table 4. Simulation results in Method-2.

Parameter	Environment-1	Environment-2	Environment-3	Environment-4	Average
Errors of estimated communication delay (s)	0.092	0.095	0.087	0.098	0.093
Errors of the prediction of the leader states (m)	0.239	0.236	0.277	0.232	0.246
Errors of distance (m)	0.242	0.234	0.283	0.251	0.216
Errors of angle (rad)	0.021	0.022	0.026	0.021	0.023

show the main experimental results using Method-1 and Method-2. The errors of distance and angle between the leader and the followers are included in the tables. Furthermore, Figure 4 also shows the errors of the estimated communication delay and the prediction of the leader trajectory in Method-2, which are small enough. Comparing the two tables, we can see that the formation errors are smaller in Method-2, which proves the effectiveness and robustness of the formation control scheme for multiple AUVs under communication delay, packet discreteness and dropout.

Finally, we compare the results of the method in this paper with previous methods as shown in

Table 5. Average formation errors in all four methods.

Parameter	Method-1	Method-2	Method-3	Method -4
Errors of distance (m)	4.826	0.216	2.137	2.695
Errors of angle (rad)	0.256	0.023	0.152	0.169

. It shows the results of average formation errors in all four methods under different communication environments. We can see from the table that the errors in Method-2 are much lower than those in other methods. Compared with Method-2, the communication delay in Method-3 has an upper bound. Therefore, the accuracy of this method cannot be guaranteed with time-varying unbounded communication delay. In addition, packet discreteness and dropout also affect the control effect of Method-3 to some extent. As for Method-4, it solves the problems of packet discreteness and dropout with the event-triggered mechanism. However, the communication delay has a great influence on this method. The effect of formation control in this method is inferior to that in Method-3. In conclusion, compared with previous methods, the proposed method in this paper can significantly improve the accuracy of formation control under communication delay, packet discreteness and dropout.

6. Field Tests

Three self-developed “TS-100” AUVs were used on a lake to verify the proposed scheme. In addition, the formation control effect of Method-1 and Method-2 proposed in this paper was compared during the field tests.

6.1. Vehicle Characteristics

The “TS-100” AUV, as shown in Figure 8, has a torpedo-like design, underactuated propulsion and a maximum speed of 5 kn. It is equipped with an acoustic communication system, Doppler velocity log (DVL), inertial navigation system (INS) and conductivity temperature and depth (CTD) sensor. The acoustic communication system is composed of an acoustic modem and transducer, which can send and receive packets underwater. During the test, the leader AUV moved according to a preset trajectory and communicated periodically with the followers in the form of a broadcast. The communication interval was set as a constant 5 s. The followers only received the packets from the leader and attempted to form an expected formation using Method-1 and Method-2. The formation control scheme was written in C++ and ran on the AUV’s control unit in real-time.

6.2. Preparation and Scenario

Before the test, the communication delay was measured to obtain the data on the parsing delay. The specific method is to place two AUVs adjacent to each other underwater, as shown in Figure 9. One AUV periodically sends packets and records the time when the packets are sent in an offline log file, and the other AUV receives packets and records the time when the packets are received in another offline log file. Then, we can calculate the communication delay according to the two log files. Since the two AUVs are very close to each other, the propagation delay can be ignored and the communication delay obtained can be considered the parsing delay. According to Section 3.1, the estimated probability density function of the parsing delay is calculated as shown in Figure 10.

During the test, three AUVs were deployed, including one leader and two followers. Multiple AUVs moved approximately 1.2 km in a southwest direction at a fixed depth and the formation structures were equilateral triangles and straight lines in the first half and second half of the course, respectively. The test scenario is shown in Figure 11. The AUV’s voyage data, including the time when the packets were sent and received, was recorded in an offline log file. The log files of all the AUVs were used later to calculate the communication delay and the errors in formation control.

The test was divided into two parts. In the first part, the communication environment was regarded as ideal, and we adopted Method-1 for formation control. In the second part, we considered the actual communication constraints and adopted Method-2 for formation control. Several tests were conducted using the two methods and the acoustic communication environment was basically the same in each test. The results of multiple tests using the same method do not differ significantly. Therefore, we arbitrarily took two test results for comparison and analysis, which are shown in Figure 12, Figure 13, Figure 14, Figure 15 and Figure 16 and

Table 6. Test results with Method-1.

Parameter	Test-1	Test-2	Test-3	Test-4	Average
Packet dropout rate (%)	31.3	32.4	31.8	31.1	31.5
Errors of distance (m)	12.204	11.989	12.221	12.187	12.150
Errors of angle (rad)	0.408	0.419	0.412	0.399	0.410

and

Table 7. Test results with Method-2.

Parameter	Test-1	Test-2	Test-3	Test-4	Average
Packet dropout rate (%)	33.1	31.2	30.5	32.3	31.8
Errors of estimated communication delay (s)	0.103	0.092	0.096	0.108	0.998
Errors of the prediction of the leader states (m)	0.833	1.142	0.986	1.048	1.002
Errors of distance (m)	5.453	5.398	5.429	5.364	5.411
Errors of angle (rad)	0.218	0.205	0.211	0.196	0.208

.

6.3. Test Results and Discussion

Figure 12 shows the motion trajectories of the multiple AUVs. We can see that the multiple AUVs maintain an equilateral triangle formation in the first half of the course and change into a straight-line formation in the second half. The overall and local enlarged figures describe that the followers can follow the leader in both methods. However, the accuracy of formation control using Method-2 is significantly better than that using Method-1.

Figure 13 shows the results of the communication delay estimation and the prediction of the leader states using Method-2. Figure 13a illustrates the results of the communication delay estimation, where $Dela{y}_r$, $Dela{y}_p$ and $Dela{y}_e$ denote the real communication delay between the leader AUV and the follower AUV₁, the estimation of the delay and the error of the estimated delay, respectively. From the figure, we can find that the method of delay estimation works well and the error of the estimation is small. It can also be seen that there is packet dropout in the acoustic communication, such as five consecutive packet dropouts between t = 200 s and t = 230 s (that is, the follower receives no packet during that time). Figure 13b shows the error of the prediction of leader states. Compared with the simulation, the interval of sent packets and packet dropout rate in the field tests are larger, so the error of prediction is also slightly larger. When the follower cannot receive the packets from the leader for a long time, a cumulative error of prediction may occur. Nevertheless, the error varies within an acceptable range and will be eliminated with the update of the leader states.

Figure 14 shows the tracking errors of follower AUV₁ and follower AUV₂. Even though the initial error is larger in Method-2, the steady-state error is smaller. However, the errors suddenly increase when the formation structures change (t = 579 s and t = 560 s). This is due to the rapid change in the desired target points of the follower. The rise in time is required to form a new formation. Therefore, after approximately 50 s, multiple AUVs reform the formation and the errors tend to converge again. Since the scheme in this paper does not consider the influence of external environmental disturbances such as water currents on formation control, the tracking errors fluctuate slightly in actual applications.

Figure 15 shows the actual distance between the leader and the followers. Combined with the parameter of formation structures in Figure 11, we can see that the actual distance is closer to the desired in Method-2. Figure 16 shows the actual angles between the leader and the followers. It can be seen that the followers cannot maintain the desired angles well with the leader in both methods. In Method-1, it is mainly caused by the problem of communication constraints. In Method-2, it may be affected by underwater environmental disturbances.

The detailed results of the field tests are shown in Table 6 and Table 7. From the tables, we can see that the packet dropout rate of acoustic communication in each test is almost the same; that is, the communication environment can be considered the same. The estimation of communication delay and prediction of leader states in Method-2 are so precise that the average errors of distance and angle between the leader and the followers (5.411 m and 0.208 rad) are smaller than those in Method-1 (12.15 m and 0.41 rad). A comparison of the two tables shows the effectiveness and robustness of the kernel density estimation and curve fitting methods, as well as the designed follower controller.

It is worth noting that the acoustic communication system and navigation equipment can affect the effectiveness of formation control. The method in this paper works well on the “TS-100” AUVs. However, it is not only applicable to a specific communication system. We believe that our method can improve the accuracy and robustness of formation control even for different communication systems. As for navigation equipment, we use an advanced integrated navigation method to improve the accuracy of the individual location. So, the navigation errors in formation control are small enough to be ignored, even though they can actually decrease the accuracy of formation control a little.

7. Conclusions

In this paper, the formation control of multiple AUVs under communication delay, packet discreteness and dropout has been investigated. The kernel density estimation and curve fitting methods have been used to solve the problem of communication constraints. Then a continuous and precise trajectory of the leader was generated for the followers. On this basis, a follower controller designed based on input–output feedback linearization can achieve accurate tracking of the leader. In this way, the formation control of multiple AUVs is realized. Simulation and field test results show that acoustic communication constraints have a great impact on formation control. However, the scheme presented can improve the accuracy and stability of formation control under an unsatisfactory communication environment. Therefore, the scheme proposed can be widely used in actual applications. For example, with this scheme, multiple AUVs can perform efficient and full coverage detection of underwater ecological environments or dangerous underwater targets. In addition, multiple AUVs equipped with different sensors can expel and even capture dangerous underwater targets through precise formation control. Furthermore, multiple AUVs can be used as mobile nodes in underwater communication networks. The movement of AUVs in a precise formation construction may improve the efficiency and coverage of communication networks. In conclusion, the formation control scheme presented in this paper can be applied to various underwater tasks, which will make some contribution to the field of ocean engineering.

Although the method in this paper has some practicability, it also ignores some potential problems. For example, model uncertainty and external disturbances such as ocean currents and obstacles are not considered, which to some extent affects the accuracy and stability of formation control and the safety of the AUVs when they move in a complex underwater environment. These problems have been studied by many researchers in recent years [46,47,48,49,50]. Moreover, 3-D coordinated formation control, which can be used for more underwater missions, is not studied in this paper. In the future, the problem of formation control subject to them will be solved based on the research in this paper.