Autonomous Multi-Robot Search for a Hazardous Source in a Turbulent Environment

Abstract

Finding the source of an accidental or deliberate release of a toxic substance into the atmosphere is of great importance for national security. The paper presents a search algorithm for turbulent environments which falls into the class of cognitive (infotaxi) algorithms. Bayesian estimation of the source parameter vector is carried out using the Rao–Blackwell dimension-reduction method, while the robots are controlled autonomously to move in a scalable formation. Estimation and control are carried out in a centralised replicated fusion architecture assuming all-to-all communication. The paper presents a comprehensive numerical analysis of the proposed algorithm, including the search-time and displacement statistics.

1. Introduction

Search strategies for finding an emitting source of hazardous substance based on sporadic sensory cues are a topic of great importance. In the context of national security, for example, the search could be for the source of an accidental or deliberate biochemical agent release into the atmosphere [1], which would require urgent attention. Similar techniques are also used in search and rescue missions and to explain the foraging behaviour of animals. In many studies of these search techniques, the searching agent is assumed to be mobile and capable of sensing the emitted substance from the source. The sensing cues are often sporadic, fluctuating and discontinuous, due to turbulent transport through the medium over a large search domain [2]. The objective of the search is to find the emitting source in the shortest possible time.

Recent advances in biochemical sensing technologies [3,4] have made the deployment of robots to search for such emitting sources very attractive [5]. Developing autonomous robots equipped with appropriate sensing capability to explore and search in harsh and dangerous environments, such as toxic or flammable ones, is the ultimate goal of this research. Due to its importance, the topic has attracted a great deal of interest from the scientific community. A survey including a taxonomy of pre-2008 search algorithms can be found in [6]. The dominant class of methods are bio-inspired strategies [7,8,9], which mimic the use of olfactory senses by bacteria, insects and other animals to localise food sources, detect predators, or find mates. Typically, the search actions depend on sensory cues. For example, upon sensing an odor signal, male moths surge upwind in the direction of the flow. When odor information vanishes, they exhibit random cross-wind casting or zigzagging to perform a local search until the plume is reacquired. Chemotaxis algorithms are a bio-inspired class of algorithms, in which the search is carried out by climbing the concentration gradient [10]. These strategies are effective close to the source where the odor plume can be considered as a continuous cloud. In the absence of positive detections, the robot may stay in one position, move linearly, or carry out a random walk. Once the chemical is detected, motion is directed towards higher gradients of concentration. These bio-inspired techniques are simple, as they require only limited spatial perception [11], however they are mostly ad hoc. Another popular class of algorithms, not mentioned in [6], are stochastic source-seeking algorithms [12,13,14]. While these algorithms are theoretically sound, they rely on smooth gradients of concentration. In the presence of turbulence, however, where the plume of the tracer randomly breaks up into time-varying disconnected patches resulting in intermittent measurements, these stochastic source-seeking algorithms are not practical.

Recently, a class of algorithms referred to as infotaxis [15] was developed specifically for searching in turbulent flows. In the absence of a smooth distribution of concentration, caused by turbulence, this strategy directs the robot towards the highest information gain. As a theoretically principled approach, with the source-term estimation being carried out in the Bayesian framework and the robot motion control being based on information-theoretic principles, the infotaxic strategy has been adopted by a number of research groups [16,17,18,19,20,21,22,23,24].

The goal of Bayesian estimation is to construct the posterior probability density function (PDF) of the source parameter vector, which typically includes its location, release rate and size. In [15] the estimation was carried out using an approximate grid-based nonlinear filtering technique [25] on a two-dimensional parameter space consisting of the x and y Cartesian coordinates of the source location. The source-release rate was assumed known, as well as the environmental parameters. The search was carried out using a single robotic platform. Grid-based nonlinear filtering was subsequently replaced with the sequential Monte Carlo method (also known as the particle filter) in [21,22,23,26]. The particle filter is more efficient than grid-based methods and is therefore capable of estimating the source-release rate jointly with the source location. Multi-robot infotaxis were investigated in [16,22,24].

Sequential estimation of the source posterior PDF using the particle filter is problematic when the state (the source parameter vector) is random, but stationary [27]. The absence of dynamics in the state implies that the exploration of the parameter space is limited to the first time index only. Even by artificially introducing dynamics to the state (e.g. a random walk-like dynamic with a small variance), the depletion of particles is inevitable over time and causes the collapse of the PF for long-duration searches [23]. In this paper we apply a Rao–Blackwell dimension reduction method to estimate the posterior PDF of the source parameter vector. This method computes the source release-rate, conditioned on the source location, analytically and uses the Monte Carlo method only in the space of x and y coordinates of the source. As a batch method, it does not suffer from depletion of particles over time. This paper studies the multi-robot search assuming a replicated centralised fusion with all-to-all communication. The robots move in a scalable formation, where the maximum size of the formation is determined by the specified communication range. Control actions are made autonomously by all robots using entropy reduction as the information theoretic measure. In computing the reward, however, we also introduce a cost of motion. The paper analyses the statistical patterns of the search, in particular the formation displacements and its scale.

The organisation of the paper is as follows. Mathematical models are presented in Section 2. The method of source parameter estimation is described in Section 3. Robotic platform motion control is explained in Section 4. Section 5 presents the numerical results, through simulations and using an experimental dataset characterised by a turbulent flow. Finally, the conclusions are drawn in Section 6.

2. Mathematical Models

2.1. Robot Motion Model

In this section we describe the motion model of each individual robotic platform. Suppose there are $N\ge 1$ robotic platforms, with the pose of the ith platform at time $t_k$ denoted by a vector ${\mathbf{\theta }}_k^i={[{({\mathbf{r}}_k^i)}^{⊺},{\varphi }_k^i]}^{⊺}$, $i=1,\cdots ,N$. Here ${\mathbf{r}}_k^i={(x_k^i,y_k^i)}^{⊺}$ is the robot location and ${\varphi }_k^i$ is its heading. The motion of the group of robots is coordinated, i.e they move in a formation. In particular, we use a circular formation with the N robots spread approximately equally on the perimeter of a circle (the formation is controlled to be equally spread on a circle, but due to process noise in the motion model, this can be only approximately achieved), whose center at time $t_k$ is $(x_k^c,y_k^c)$ with the radius ${\rho }_k\in [r_{\min },r_{\max }]$. The minimum radius $r_{\min }$ is introduced to prevent the robots from colliding; the maximum radius $r_{\max }$ is to ensure all-to-all communication amongst the platforms. The bearing of the ith platform within the formation is fixed, while its radius is allowed to vary. The position of the ith platform is defined by the offset $(\Delta x_i,\Delta y_i)$ from the centre $(x_k^c,y_k^c)$. All platforms in the formation have approximately equal orientation, while the offset is given by:

$$\Delta x_i=2\rho \cos (2\pi i/N),\Delta y_i=2\rho \sin (2\pi i/N)$$

(1)

for $i=1,\cdots ,N$.

The biochemical sensors are activated at time instants $t_k$, $k=1,2,\cdots$ to report concentration measurements. During the interval of time $[t_{k-1},t_k]$ between two consecutive sensing instants, the formation is moving. The duration of this interval, $T_k=t_k-t_{k-1}\ge 0$, is referred to as the travel time. The assumption is that sensing is suppressed during the travel time. The motion of the formation during the interval $[t_{k-1},t_k]$ is controlled by the following input parameters: the linear velocity $V_k$, the angular velocity ${\mathrm{\Omega }}_k$, and the formation scale increment $\Delta {\xi }_k={\xi }_k-{\xi }_{k-1}$ (which can be positive or negative). Given the control vector ${\mathbf{u}}_k={[V_k,{\mathrm{\Omega }}_k,{\xi }_k,T_k]}^{⊺}$, the dynamics of ith platform during a short integration time $[t-\delta ,t]$, where $t\in [t_{k-1}+\delta ,t_k]$ and $\delta \ll T_k$, can be modelled by a Markov process with the transitional density $\pi ({\mathbf{\theta }}_t^i|{\mathbf{\theta }}_{t-\delta }^i,{\mathbf{u}}_k)=\mathcal{N}({\mathbf{\theta }}_t^i;\beta ({\mathbf{\theta }}_{t-\delta }^i,{\mathbf{u}}_k),{\mathbf{Q}}_k)$. The mean of this normal density, $\beta ({\mathbf{\theta }}_{t-\delta }^i,{\mathbf{u}}_k)$, is a nonlinear function specified as follows:

$$\beta ({\mathbf{\theta }}_{t-\delta }^i,{\mathbf{u}}_k)={\mathbf{\theta }}_{t-\delta }^i+\delta \left[\begin{array}{c}{V}_k\cos ({\varphi }_{k-1}^i)\\ V_k\sin ({\varphi }_{k-1}^i)\\ {\mathrm{\Omega }}_k\end{array}\right]+B+C$$

(2)

where:

$$B=\frac{(t-t_{k-1})\Delta {\xi }_k}{T_k}\left[\begin{array}{c}\Delta x_i\\ \Delta y_i\\ 0\end{array}\right],t\in [t_{k-1}+\delta ,t_k],$$

describes the motion due to the formation-scale change. Vector C in (2) introduces the corrections to the pose in order to remove the difference between the desired (deterministic) pose:

$${\overline{\mathbf{\theta }}}_{k-1}^i=\frac{1}{N}\sum _{j=1}^N{\mathbf{\theta }}_{k-1}^j+{\xi }_{k-1}\left[\begin{array}{c}\Delta x_i\\ \Delta y_i\\ 0\end{array}\right]$$

and the actual pose (perturbed by process noise) ${\mathbf{\theta }}_{k-1}^i$ at the previous sensing time $t_{k-1}$, i.e., $C=-({\overline{\mathbf{\theta }}}_{k-1}^i-{\mathbf{\theta }}_{k-1}^i)(t_k-t)/T_k$. Without the correction term C, the platforms could drift and break the formation during the search time. Finally, ${\mathbf{Q}}_k$ is the covariance matrix which accounts for stochastic disturbances in motion. Figure 1 illustrates a simulated path of a robot formation based on the described motion model. In this illustration, the formation of $N=5$ robots is expanding (the scales are ${\xi }_1=1$, ${\xi }_2=4$ and ${\xi }_3=8$). Note that, despite random individual paths, the formation at measurement time instants $t_1,t_2,t_3$ is almost a perfect pentagon. Apart from some randomness due to process noise, the motion model keeps the formation with the fixed bearings relative to its centre and a fixed north (i.e., the formation does not rotate).

2.2. Measurement Model

Dispersion of the emitted hazardous substance in a turbulent environment is modelled using the Lagrange particle encounters model developed in [15]. Suppose that the emitting source is located at coordinates ${\mathbf{r}}_0={(X_0,Y_0)}^{⊺}$ and its release rate, or strength, is $Q_0$. The source parameter vector is denoted $\mathbf{\eta }={[{\mathbf{r}}_0^{⊺}{Q}_0]}^{⊺}$. The particles released from the source propagate with combined molecular and turbulent isotropic diffusivity D, but can also be advected by wind. The released particles have an average lifetime of $\tau$ before being absorbed. Let the average wind characteristics be the speed U and direction, which by convention, coincides with the direction of the x axis.

Suppose a spherical sensor of small radius a is mounted on the ith robot platform, whose pose at time k is ${\mathbf{\theta }}_k^i={[x_k^i,y_k^i,{\varphi }_k^i]}^{⊺}$ (robot location is assumed non-coincidental with the source location ${\mathbf{r}}_0$). This sensor will experience a series of encounters with the particles released from the emitting source. The average rate of encounters can be modelled as follows [15]:

$$R_{\mathbf{\eta }}({\mathbf{r}}_k^i)=\frac{Q_0}{\ln \left(\frac{\lambda }{a}\right)}\exp \left[\frac{(X_0-x_k^i)U}{2D}\right]·K_0\left(\frac{d_k^i({\mathbf{r}}_0)}{\lambda }\right)$$

(3)

where D, $\tau$ and U are known environmental parameters,

$$d_k^i({\mathbf{r}}_0)=\sqrt{{(x_k^i-X_0)}^2+{(y_k^i-Y_0)}^2}$$

(4)

is the distance between the source and ith sensor platform, $K_0$ is the modified Bessel function of the second kind of order zero, and

$$\lambda =\sqrt{\frac{D\tau }{1+\frac{U^2\tau }{4D}}}.$$

(5)

depends on environmental parameters only.

The stochastic process of sensor encounters with the dispersed particles is modelled by a Poisson distribution. The probability that a sensor at location ${\mathbf{r}}_k^i$ encounters $z\in {\mathbb{Z}}^{+}\cup \{0\}$ particles (z is a non-negative integer) during a time interval $t_0$ is then:

$$\mathcal{P}(z;{\mu }_k^i)=\frac{{({\mu }_k^i)}^z}{z!}{e}^{-{\mu }_k^i}$$

(6)

where ${\mu }_k^i=t_0·R_{\mathbf{\eta }}({\mathbf{r}}_k^i)$ is the mean number of particles expected to reach the sensor at location ${\mathbf{r}}_k^i$ during $t_0$.

3. Source Parameter Estimation

3.1. Problem Specification

Let $z_k^i$ denote the sensor measurement recorded by the ith robot platform at time $t_k$. The sequence of such measurements from platform i, starting from the beginning of the search until $t_k$ is a vector ${\mathbf{z}}_{1:k}^i={[z_1^i,\cdots ,z_k^i]}^{⊺}$, $i=1,\cdots ,N$. Similarly, the measurements from all platforms up to time $t_k$ is a vector ${\mathbf{z}}_{1:k}={\left[{({\mathbf{z}}_{1:k}^1)}^{⊺},\cdots ,{({\mathbf{z}}_{1:k}^N)}^{⊺}\right]}^{⊺}$. Accordingly we can define the vector of sensor measurement locations corresponding to ${\mathbf{z}}_{1:k}$ as ${\mathbf{r}}_{1:k}={\left[{\mathbf{r}}_1^1,\cdots ,{\mathbf{r}}_k^1,\cdots ,{\mathbf{r}}_1^N,\cdots ,{\mathbf{r}}_k^N\right]}^{⊺}$. This vector is assumed to be known. Whenever in the text we refer to the measurement vector ${\mathbf{z}}_{1:k}$, we implicitly assume that it is in pair with ${\mathbf{r}}_{1:k}$.

Assuming all-to-all communication between the platforms for exchanging mutually their latest measurements and positions (ignoring the time-delays and bandwidth limitations), ${\mathbf{z}}_{1:k}$ and ${\mathbf{r}}_{1:k}$ are available at every platform at time $t_k$ for processing. Thus, every platform can independently carry out centralised fusion in order to estimate the source parameter vector $\mathbf{\eta }$. This type of fusion architecture is known as replicated centralised.

Assuming the sensor measurements, conditioned on $\mathbf{\eta }$, are independent, the likelihood function of the measurement vector ${\mathbf{z}}_{1:k}$ can be written as a double product $\ell ({\mathbf{z}}_{1:k}|\mathbf{\eta })={\prod }_{i=1}^N{\prod }_{j=1}^k\mathcal{P}\left(z_j^i;t_0{R}_{\mathbf{\eta }}({\mathbf{r}}_j^i)\right)$.

We adopt the Bayesian estimation framework with the goal of computing the posterior PDF $p(\mathbf{\eta }|{\mathbf{z}}_{1:k})$. In this framework, in addition to $\ell ({\mathbf{z}}_{1:k}|\mathbf{\eta })$, one needs to specify the prior distribution of the parameter vector $\pi (\mathbf{\eta })$. Then, using the Bayes’ rule, the posterior PDF is:

$$p(\mathbf{\eta }|{\mathbf{z}}_{1:k})=\frac{\ell ({\mathbf{z}}_{1:k}|\mathbf{\eta })\pi (\mathbf{\eta })}{\int \ell ({\mathbf{z}}_{1:k}|\mathbf{\eta })\pi (\mathbf{\eta })d\mathbf{\theta }}.$$

(7)

For the described problem, (7) cannot be solved analytically and we need to apply a numeric approximation. However, it will be shown that, assuming the prior of the source strength $\pi (Q_0)$ is a gamma distribution, the posterior $p(Q_0|{\mathbf{z}}_{1:k},{\mathbf{r}}_0)$ can be solved analytically. Because the posterior $p(\mathbf{\eta }|{\mathbf{z}}_{1:k})$, using the chain rule, can be expressed as:

$$p(\mathbf{\eta }|{\mathbf{z}}_{1:k})=p(Q_0|{\mathbf{z}}_{1:k},{\mathbf{r}}_0)p({\mathbf{r}}_0|{\mathbf{z}}_{1:k}),$$

(8)

x we will only need to apply a numeric approximation to estimate the posterior $p({\mathbf{r}}_0|{\mathbf{z}}_{1:k})$.

3.2. Solution

It is reasonable to assume that the source strength is independent of its location, and hence $\pi (\mathbf{\eta })=\pi (Q_0)\pi ({\mathbf{r}}_0)$. Let us adopt for $\pi (Q_0)$ a gamma distribution, with the shape parameter ${\kappa }_0$ and the scale parameter ${\vartheta }_0$:

$$\pi (Q_0)=\mathcal{G}(Q_0;{\kappa }_0,{\vartheta }_0)=\frac{Q_0^{({\kappa }_0-1)}{e}^{-Q_0/{\vartheta }_0}}{{\vartheta }_0^{{\kappa }_0}\mathrm{\Gamma }({\kappa }_0)}.$$

(9)

Note that for suitably chosen hyperparameters ${\kappa }_0$ and ${\vartheta }_0$, the support of this prior can cover a large span of possible values of $Q_0$.

The conjugate prior of the Poisson distribution is the gamma distribution [28]. Therefore, the posterior $p(Q_0|{\mathbf{r}}_0,{\mathbf{z}}_{1:k})$ is also a gamma distribution, $p(Q_0|{\mathbf{r}}_0,{\mathbf{z}}_{1:k})=\mathcal{G}(Q_0;{\kappa }_k,{\vartheta }_k)$, with parameters ${\kappa }_k$ and ${\vartheta }_k$ expressed analytically as a function of ${\mathbf{r}}_0$ and ${\mathbf{z}}_{1:k}$ as follows (see Appendix A):

$$\begin{array}{ccc}\hfill {\kappa }_k& =& {\kappa }_0+\sum _{i=1}^N\sum _{j=1}^k{z}_j^i,\hfill \end{array}$$

(10)

$$\begin{array}{ccc}\hfill {\vartheta }_k& =& \frac{{\vartheta }_0}{1+{\vartheta }_0{\displaystyle \sum _{i=1}^N}{\displaystyle \sum _{j=1}^k}{\rho }_{{\mathbf{r}}_0}({\mathbf{r}}_j^i)}.\hfill \end{array}$$

(11)

where ${\rho }_{{\mathbf{r}}_0}({\mathbf{r}}_k^i)=t_0{R}_{\mathbf{\eta }}({\mathbf{r}}_k^i)/Q_0$, that is:

$${\rho }_{{\mathbf{r}}_0}({\mathbf{r}}_k^i)=\frac{t_0}{\ln \left(\frac{\lambda }{a}\right)}\exp \left[\frac{(X_0-x_k^i)U}{2D}\right]·K_0\left(\frac{d_k^i({\mathbf{r}}_0)}{\lambda }\right).$$

(12)

It remains to compute the posterior PDF $p({\mathbf{r}}_0|{\mathbf{z}}_{1:k})$:

$$p({\mathbf{r}}_0|{\mathbf{z}}_{1:k})=\frac{g({\mathbf{z}}_{1:k}|{\mathbf{r}}_0)\pi ({\mathbf{r}}_0)}{\int g({\mathbf{z}}_{1:k}|{\mathbf{r}}_0)\pi ({\mathbf{r}}_0)d{\mathbf{r}}_0}.$$

(13)

Compared to (7), which is a three-dimensional PDF, the posterior in (13) is two-dimensional. This dimension reduction improves the accuracy of the numerical solution. After a few lines of mathematical derivations one can show that the analytic expression for the likelihood function $g({\mathbf{z}}_{1:k}|{\mathbf{r}}_0)$, which features in (13), is given by (see Appendix A):

$$\begin{array}{cc}\hfill g({\mathbf{z}}_{1:k}|{\mathbf{r}}_0)=& {\vartheta }_0^{{\sum }_{i=1}^N{\sum }_{j=1}^k{z}_j^i}\frac{\mathrm{\Gamma }\left({\kappa }_0+{\displaystyle \sum _{i=1}^N}{\displaystyle \sum _{j=1}^k}{z}_j^i\right)}{\mathrm{\Gamma }({\eta }_0)}·\prod _{i=1}^N\prod _{j=1}^k\frac{{[{\rho }_{{\mathbf{r}}_0}({\mathbf{r}}_j^i)]}^{z_j^i}}{z_j^i!}.\hfill \end{array}$$

(14)

A plethora of techniques is available for the numerical computation of (13), from grid-based numerical integration methods to Monte Carlo methods (e.g. the Markov Chain Monte Carlo, importance sampling and population Monte Carlo). We choose a Monte Carlo method which, for a given $\pi ({\mathbf{r}}_0)$ and ${\mathbf{z}}_{1:k}$, approximates the posterior PDF $p({\mathbf{r}}_0|{\mathbf{z}}_{1:k})$ by a random sample ${\{{\mathbf{r}}_{0,k}^{(m)}\}}_{1\le m\le M}$ as follows:

$$p({\mathbf{r}}_0|{\mathbf{z}}_{1:k})\approx p_M({\mathbf{r}}_0|{\mathbf{z}}_{1:k})=\frac{1}{M}\sum _{m=1}^M\delta ({\mathbf{r}}_0-{\mathbf{r}}_{0,k}^{(m)}).$$

Here $\delta ({\mathbf{r}}_0-{\mathbf{r}}_{0,k}^{(m)})$ denotes the delta-Dirac mass located at ${\mathbf{r}}_{0,k}^{(m)}$. As the size of the sample $M\to \infty$, the moments of $p_M({\mathbf{r}}_0|{\mathbf{z}}_{1:k})$ converge almost surely to the moments of $p({\mathbf{r}}_0|{\mathbf{z}}_{1:k})$. An advantage of the Monte Carlo method over the grid-based (deterministic) integration is that the former positions its integration points (i.e., samples) in the regions of high probability [29].

The basic steps in estimation of $p(\mathbf{\eta }|{\mathbf{z}}_{1:k})$ expressed by (8) are summarised in Algorithm 1. The expected a posteriori point estimates of the source location and its strength can be computed from the output ${\left\{\left({\mathbf{r}}_{0,k}^{(m)},{\kappa }_k,{\vartheta }_k^{(m)}\right)\right\}}_{1\le m\le M}$ as follows:

$$\begin{array}{ccc}\hfill {\widehat{\mathbf{r}}}_0& =& \frac{1}{M}\sum _{m=1}^M{\mathbf{r}}_{0,k}^{(m)}\hfill \end{array}$$

(15)

$$\begin{array}{ccc}\hfill {\widehat{Q}}_0& =& \frac{1}{M}\sum _{m=1}^M{\kappa }_k·{\vartheta }_k^{(m)}\hfill \end{array}$$

(16)

Algorithm 1 Estimation of $p(\mathbf{\eta }|{\mathbf{z}}_{1:k})$

1: Input: $\pi ({\mathbf{r}}_0)$, ${\kappa }_0$, ${\vartheta }_0$, ${\mathbf{z}}_{1:k}$, M 2: Estimate $p({\mathbf{r}}_0|{\mathbf{z}}_{1:k})$ by a random sample ${\{{\mathbf{r}}_{0,k}^{(m)}\}}_{1\le m\le M}$ 3: Compute ${\kappa }_k$, Equation (10) 4: Compute ${\vartheta }_k^{(m)}$ using ${\mathbf{r}}_{0,k}^{(m)}$, Equation (11); $m=1,\cdots ,M$ 5: Output: ${\left\{\left({\mathbf{r}}_{0,k}^{(m)},{\kappa }_k,{\vartheta }_k^{(m)}\right)\right\}}_{1\le m\le M}$

Finally, the Monte Carlo method which estimates $p({\mathbf{r}}_0|{\mathbf{z}}_{1:k})$ in line 2 of Algorithm 1, was implemented using iterated importance sampling with progressive correction (IIS-PC) [30]. Full details of the implementation are given in [31]. A desirable property of importance sampling is to draw samples from an importance density that result in sample weights with a minimal variance. The question is how to design this importance density. The key idea behind IIS-PC is to achieve the goal of drawing samples with a minimal variance in a sequential manner, by constructing a sequence of target distributions from which to draw samples. The first target distribution is the prior, while every subsequent target distribution should increasingly resemble the posterior. A target distribution which can be used in this context at iteration $s=1,\cdots ,H$ of the IIS-PC is:

$$p_s({\mathbf{r}}_0|{\mathbf{z}}_{1:k})\propto {\left[g({\mathbf{z}}_{1:k}|{\mathbf{r}}_0)\right]}^{{\mathrm{\Gamma }}_s}\pi ({\mathbf{r}}_0),$$

(17)

where $g({\mathbf{z}}_{1:k}|{\mathbf{r}}_0)$ is given by (14), ${\mathrm{\Gamma }}_s={\sum }_{v=1}^s{\gamma }_v$ with ${\gamma }_v\in (0,1]$ and ${\mathrm{\Gamma }}_H=1$. Note that ${\mathrm{\Gamma }}_s$ is an increasing function of s, upper bounded by one. As a consequence, the intermediate likelihood is broader than the true likelihood, particularly in the early stages (for small s). Thus, the sequence of target distributions in (17) gradually introduces the correction imposed by the measurement ${\mathbf{z}}_k$ on the prior $\pi ({\mathbf{r}}_0)$. To derive any benefits from IIS-PC, it is required after each stage to remove the lowly weighted members of the sample and diversify the remaining ones. Lowly weighted members are removed by resampling [27], while diversification of the remaining samples is performed by Markov transitions whose stationary distribution is the target distribution $p_s({\mathbf{r}}_0|{\mathbf{z}}_{1:k})$. The outcome is a diverse sample located in the region of the parameter space where the intermediate likelihood has non-zero values. The choice of correction factors ${\gamma }_1,\cdots ,{\gamma }_H$, as well as the number of iterations H are design parameters. Further details can be found in [31].

4. Robot Formation Control

Suppose at time $t_{k-1}$ the robotic platforms have processed all available measurements included in the vector ${\mathbf{z}}_{1:k-1}$ and estimated the posterior PDF $p(\mathbf{\eta }|{\mathbf{z}}_{1:k-1})$ using the method described in Section 3. Let $\mathcal{A}\subset {\mathbb{R}}^2$ denote a designated search area, which includes the source location ${\mathbf{r}}_0$. The key aspect of search is, for each robotic platform of the formation, to decide autonomously where to move next within $\mathcal{A}$ in order to acquire a new concentration measurement at $t_k$, denoted ${\mathbf{z}}_k={[z_k^1,z_k^2,\cdots ,z_k^N]}^{⊺}$.

An autonomous multi-robot search can be formulated as a partially-observed Markov decision process (POMDP) [32]. The elements of a POMDP include an information state, the space of admissible actions (controls) and a reward function. The information state, adopting the Bayesian framework for estimation of source parameters, is the posterior PDF $p(\mathbf{\eta }|{\mathbf{z}}_{1:k-1})$. Current knowledge about the source position and strength is fully specified by this density. A decision in the context of search is the selection of a control vector ${\mathbf{u}}_k\in {\mathcal{U}}_k$, where ${\mathcal{U}}_k$ is the space of admissible actions. Finally, the reward function maps each admissible action into a non-negative real number, which represents a measure of the action’s expected information gain. An optimal strategy selects, at each time, the action with the highest reward. Admissible actions can be formed with one or multiple steps ahead. According to the motion model introduced in Section 2.1, the space of admissible actions ${\mathcal{U}}_k$ is continuous with four dimensions: $V_k$, ${\mathrm{\Omega }}_k$, ${\xi }_k$ and $T_k$. In order to reduce the computational complexity of the numerical optimisation, we discretise ${\mathcal{U}}_k$ and consider only myopic (one step ahead) control.

If $\mathbb{V}$, $\mathbb{O}$, $\mathbb{S}$ and $\mathbb{T}$ denote the sets of possible discrete-values of $V_k$, ${\mathrm{\Omega }}_k$, ${\xi }_k$ and $T_k$, respectively, then ${\mathcal{U}}_k$ is the Cartesian product $\mathbb{V}\times \mathbb{O}\times \mathbb{S}\times \mathbb{T}$. The myopic selection of the control vector at time $t_k$ is expressed as:

$${\mathbf{u}}_k=\arg \underset{\mathbf{v}\in {\mathcal{U}}_k}{\max }\mathbb{E}\left\{\mathcal{D}\left(p(\mathbf{\eta }|{\mathbf{z}}_{1:k-1}),{\mathbf{z}}_k(\mathbf{v})\right)\right\}$$

(18)

where $\mathcal{D}\left(p(\mathbf{\eta }|{\mathbf{z}}_{1:k-1}),{\mathbf{z}}_k(\mathbf{v})\right)$ is the reward function. Note that the reward function depends on the future measurement ${\mathbf{z}}_k(\mathbf{v})$, assuming the control vector $\mathbf{v}\in {\mathcal{U}}_k$ has been applied. In reality, this future measurement is not available (the decision has to be made at time $t_{k-1}$), and therefore the expectation operator $\mathbb{E}$ with respect to the prior measurement PDF features in (18). Previous studies of search strategies [16,23] found that the reward function measuring the information gain as the entropy reduction, results in the most efficient search. However, the earlier work neglected that, while traveling, robots incur certain cost. Assuming $\alpha$ is the cost of travel per unit distance, we adopt the following specification for the expected reward function:

$$\begin{array}{c}\mathbb{E}\left\{\mathcal{D}\left(p(\mathbf{\eta }|{\mathbf{z}}_{1:k-1}),{\mathbf{z}}_k(\mathbf{v})\right)\right\}=\hfill \\ \left[H_{k-1}-\mathbb{E}\{H_k\left({\mathbf{z}}_k(\mathbf{v})\right)\}\right]e^{-\alpha s_k(\mathbf{v})}\hfill \end{array}$$

(19)

where $s_k(\mathbf{v})=V_k{T}_k\ge 0$ is the travelled distance under action $\mathbf{v}$, $H_{k-1}$ is the current entropy (based on ${\mathbf{z}}_{1:k-1}$), i.e.,

$$H_{k-1}=-\int p(\mathbf{\eta }|{\mathbf{z}}_{1:k-1})\ln p(\mathbf{\eta }|{\mathbf{z}}_{1:k-1})d\mathbf{\eta },$$

(20)

$H_k({\mathbf{z}}_k(\mathbf{v}))$ is the future entropy (after applying the control $\mathbf{v}$):

$$H_k({\mathbf{z}}_k(\mathbf{v}))=-\int p(\mathbf{\eta }|{\mathbf{z}}_{1:k-1},{\mathbf{z}}_k(\mathbf{v}))\ln p(\mathbf{\eta }|{\mathbf{z}}_{1:k-1},{\mathbf{z}}_k(\mathbf{v}))d\mathbf{\eta }.$$

(21)

and $\mathbb{E}\{H_k\left({\mathbf{z}}_k(\mathbf{v})\right)\}$ is its expected value, with respect to the probability mass function $P\{{\mathbf{z}}_k|{\mathbf{z}}_{1:k-1}\}=\int \ell ({\mathbf{z}}_k|\mathbf{\eta })p(\mathbf{\eta }|{\mathbf{z}}_{1:k-1})d\mathbf{\eta }$:

$$\mathbb{E}\{H_k\left({\mathbf{z}}_k(\mathbf{v})\right)\}=\sum _{z_k}P\{{\mathbf{z}}_k|{\mathbf{z}}_{1:k-1}\}·H_k\left({\mathbf{z}}_k(\mathbf{v})\right).$$

(22)

The exponential term, $e^{-\alpha s_k(\mathbf{v})}$, in (19) incorporates the cost of travel, penalising longer travel distances.

The computation of entropy $H_{k-1}$ in (20) is carried out as follows. Recall that $p(\mathbf{\eta }|{\mathbf{z}}_{1:k-1})$ is approximated by a random sample ${\{({\mathbf{r}}_{0,k-1}^{(m)},{\kappa }_{k-1},{\vartheta }_{k-1}^{(m)})\}}_{1\le m\le M}$. From this representation, one can compute a random sample ${\{{\mathbf{\eta }}_{k-1}^{(m)}\}}_{1\le m\le M}$, with uniform weights $1/M$, where ${\mathbf{\eta }}_{k-1}^{(m)}={[{({\mathbf{r}}_{0,k-1}^{(m)})}^{⊺},Q_{0,k-1}^{(m)}]}^{⊺}$ and $Q_{0,k-1}^{(m)}\sim \mathcal{G}(Q_0;{\kappa }_{k-1},{\vartheta }_{k-1}^{(m)})$, for $m=1,\cdots ,M$. Then $p(\mathbf{\eta }|{\mathbf{z}}_{1:k-1})\approx \frac{1}{M}{\sum }_{m=1}^M\delta (\mathbf{\eta }-{\mathbf{\eta }}_{k-1}^{(m)})$, which leads to the approximation $H_{k-1}\approx -\frac{1}{M}\ln \frac{1}{M}$.

The computational expense of exact computation of $\mathbb{E}\{H_k\left({\mathbf{z}}_k(\mathbf{v})\right)\}$ grows exponentially with N because the sum in (22) is N-dimensional: it requires consideration of all possible combinations of measurement outcomes from N platforms. Hence we resort to a Monte Carlo approximation. For a command vector $\mathbf{v}\in {\mathcal{U}}_k$, first we draw a random sample ${\{{\mathbf{z}}_k^{(j)}\}}_{1\le j\le J}$ from $P\{{\mathbf{z}}_k|{\mathbf{z}}_{1:k-1}\}$ using the following procedure. We randomly select a sample ${\mathbf{\eta }}_{k-1}^{*}$ from ${\{{\mathbf{\eta }}_{k-1}^{(m)}\}}_{1\le m\le M}$, and then draw N times from the likelihood $\ell ({\mathbf{z}}_k|{\mathbf{\eta }}_{k-1}^{*})$. By repeating this procedure S times, $J=SN$ samples of the measurement outcomes from N platforms ${\{{\mathbf{z}}_k^{(j)}\}}_{1\le j\le J}$ are created. Then (22) is simply approximated as: $\mathbb{E}\{H_k\left({\mathbf{z}}_k(\mathbf{v})\right)\}=\frac{1}{J}{\sum }_{j=1}^J{H}_k\left({\mathbf{z}}_k^{(j)}(\mathbf{v})\right)$.

The search algorithm needs to decide when to stop the search and report its final estimates of source parameters, denoted ${\widehat{\mathbf{r}}}_0$ and ${\widehat{Q}}_0$. The termination criterion is based on the spatial spread of the random sample ${\{{\mathbf{r}}_{0,k}^{(m)}\}}_{1\le k\le M}$, computed as the trace of its sample covariance matrix. When this spread is below a certain threshold, denoted $\varpi$, the search is terminated.

The key assumption of the replicated centralised fusion architecture is that the same input data (measurements ${\mathbf{z}}_{1:k}$ and the corresponding locations ${\mathbf{r}}_{1:k}$) are available to all robotic platforms for data fusion (parameter estimation and robot motion control). However, this assumption is not sufficient. As the Monte Carlo method plays a role in the data fusion, we must also ensure that the same pseudo-random generators, using the same seed, are running on each individual platform. In this way all platforms reach the same decision on the motion control vector ${\mathbf{u}}_k$ and subsequently apply the motion model described in Section 2.1, knowing their own identification number i in the formation.

5. Numerical Results

5.1. Illustrative Run

First we illustrate a typical run of the multi-robot search algorithm using the following parameters (all physical quantities are in arbitrary units (a. u.)):

• True source parameters: $X_0=-150$, $Y_0=150$, $Q_0=4$;
• Search area $\mathcal{A}=500\times 500$ and number of platforms $N=5$;
• Motion model parameters: $r_{\min }=1$, $r_{\max }=100$, $\delta =0.25$, $\mathbb{V}=\{1\}$, $\mathbb{O}=\{-5,-2.5,0,2.5,5\}$, $\mathbb{S}=\{1,2,4,8\}$, $\mathbb{T}=\{0.25$, $0.5,1,2,4,8,16,32,64,128,256\}$, and based on (1), the initial scale is ${\xi }_0=2$;
• Measurement model parameters: $a=1$, $D=1$, $\tau =250$, $U=0.25$, $t_0=1$;
• Algorithm parameters: ${\kappa }_0=3$, ${\vartheta }_0=5.2$, $M=1000$, $\pi ({\mathbf{r}}_0)$ is the uniform distribution over the search area $\mathcal{A}$, $J=M$, the cost of travel $\alpha =0.01$, termination threshold $\varpi =6.25$.

The initial position of the centroid of the formation was at $(200,-250)$. The results of a typical run are shown in Figure 2. Figure 2a shows the search area, the paths of the multi-robot formation at $k=2$ and the source location at $(X_0,Y_0)$ with the contour plot of the corresponding mean rate $R_{\mathbf{\eta }}$ of (3). The random samples ${\{{\mathbf{r}}_{0,k}^{(m)}\}}_{1\le k\le M}$, approximating the posterior $p({\mathbf{r}}_0|{\mathbf{z}}_{1:k})$, are shown as brown dots. Note that the search area is more than ten times bigger than the area where the source can be sensed. Consequently, the measurements ${\mathbf{z}}_{1:k}$ are initially zero vectors for a long time; on this occasion for $k=1,\cdots ,32$. At $k=33$ the first non-zero measurement is recorded by one of the sensors, resulting in the posterior PDF $p({\mathbf{r}}_0|{\mathbf{z}}_{1:k=33})$, approximated by the sample ${\{{\mathbf{r}}_{0,k=33}^{(m)}\}}_{1\le k\le M}$ shown in Figure 2b. This figure also shows the search paths of the formation until the first detection. The search continues until $k=48$, when the termination criterion is reached. The source parameter estimates at this stage are ${\widehat{\mathbf{r}}}_0=(-151.45,149.98)$ and ${\widehat{Q}}_0=4.3$. The posterior PDF of $Q_0$ at $k=48$ is shown in Figure 2c. Note the narrow distribution of this posterior compared to the prior $\pi (Q_0)$. The total search time of this run was 2092 a.u.

5.2. Monte Carlo runs

In order to understand the performance characteristics of the search algorithm, 200 Monte Carlo runs were performed of various scenarios. Unless otherwise specified, the parameters used were the same as specified in Section 5.1, but with the bigger search area, $\mathcal{A}=750\times 750$.

Figure 3 shows the mean search time when varying the number of platforms, N, from 1 to 10. Both the source location and initial centroid position were drawn at random from the uniform distribution over the search area. There is initially a significant decrease in the mean search time as platforms are added, but this eventually levels off. Even with a large number of platforms the formation still needs to get quite close to the source before there is a significant probability of a non-zero detection on any of the sensors, so the search time becomes dominated by the time spent exploring the area before that detection.

Figure 4 shows the mean search time when varying the side length of the search area in the range $[200,1000]$. The search was performed using $N=5$ platforms and both the source location and initial centroid position were again drawn at random from the uniform distribution over the search area. Over this range of side lengths, the mean search time is approximately linear in the area searched.

Figure 5 shows the effect of travel cost on the formation displacements chosen by the search algorithm. The travel cost $\alpha$, was increased from 0.01 to 0.02 and the results are shown for 1 and 5 search platforms. In accordance with intuition, increasing the travel cost results in a shift of the histogram towards smaller displacements. Another interesting observation can be made from Figure 5: the histograms clearly has two peaks. One corresponds to the short movements, while the other peak corresponds to the long “jumps” of 32 or more units in length. This appears to be very similar to the displacement patterns of foraging animals [2,33,34]. They too typically combine the phases of long non-sensing displacement, with short sensing (and reactive) search phases. This strategy is often referred to as intermittent search.

Figure 6 shows a Q-Q plot with $95\%$ confidence intervals [35], comparing the empirical PDF of the search times (for $N=1$ and $N=5$ robotic platforms) with a fitted inverse Gaussian distribution. The match between the empirical and the proposed theoretical PDF can be accepted with $95\%$ confidence, if the confidence limits, shown as green lines in Figure 6, do not cross the red dashed line. The empirical search time samples were obtained with the source location fixed at $(X_0,Y_0)=(187.5,187.5)$ and the initial robot formation centroid at $[187.5,-187.5]$. As found in our previous work [23], the search time for a single search platform is well modelled by an inverse Gaussian. The model, however, does not hold as strongly for $N=5$ searching platforms, especially for shorter search times.

In all Monte Carlo runs of the proposed autonomous multi-robot search, the hazardous source was found and localised with accuracy determined by the termination criterion. In particular, the RMS localisation error was found to roughly correspond to $\sqrt{\varpi }=2.5$ a.u.

5.3. Experimental Results

The search algorithm was also evaluated on an experimental data set, collected by COANDA Research & Development Corporation using their large recirculating water channel. The source was releasing fluorescein dye at a constant rate from a narrow tube. The data is a sequence of 340 frames of instantaneous concentration field measurements in the vertical plane, sampled every $10/23$ s. The size of each frame is $49\times 98$ pixels, where each pixel corresponds to a square area of $2.935\times 2.935$ mm${}^2$. The sequence of frames, in the form of a video, is included in the supplementary material; the actual dataset can be obtained from the authors on request.

The frames from this experimental dataset were scaled by a factor of 3 using bicubic scaling and placed in the top left corner of a $500\times 500$ search area. The simulated measurements from the previous section were replaced with the rounded integer taken from the closest spatial and temporal sample from the experimental dataset.

Figure 7 shows an illustrative run of the search algorithm on the experimental dataset at times $k=0$, 10, 20 and 30. The algorithm terminated, reporting the location of the source, just after the last frame shown, at $k=32$.

Figure 8 shows a Q-Q plot of 200 samples of the search times for $N=1$ and $N=5$ search platforms versus a fitted inverse Gaussian distribution, with the initial formation centroid position fixed at $[125,-125]$. As found for the simulated plume in Figure 6, and in our previous work [23], the search times for the experimental plume can be accurately modelled by an inverse Gaussian distribution.

6. Summary

The paper proposed an algorithm for autonomous search for an emitting source of hazardous material transported through the environment by a turbulent flow. The search was designed for a group of robots connected in a network with all-to-all communication.

The source parameter estimation was carried out in the Bayesian framework: the posterior density of source strength, conditioned on the source location, was carried out analytically. The posterior density of source location, on the other hand, was computed numerically, using a Monte Carlo method. The source parameter estimation, carried out in this manner, overcomes the problems encountered in previous implementations, such as the assumption that the source strength is known in using the grid-based estimation, or the depletion of particles, when the particle filter is applied.

The robots are moving in a controllable formation, with control parameters including the linear and angular velocity, the travel time and the scale of formation. The reward function for choosing the robot formation control vector was selected as the entropy reduction, with the built-in travel cost.

Numerical results, using both simulated and real data, demonstrate reliable performance: the success rate in finding the source is $100\%$, with localisation accuracy determined by the termination criterion. Furthermore, the analysis of the algorithm reveals: (1) a diminishing return on increasing the number of platforms in formation; (2) a linear growth of the mean search time with the search area; (3) an increase in the cost of travel resulting in shorter formation displacements; (4) the search displacements are in accordance with the intermittent search strategy; and (5) the PDF of search time for a single searcher is well-modelled by the inverse Gaussian distribution.

There are many avenues for future work. For example, one could explore distributed, rather than centralised, source parameter estimation and robot control. This could take into account a more realistic communication network with limited bandwidth and time delays. The implicit assumption in the presented work was that the search area is an open field. The search in an area with obstacles and known map would be another complementary future research direction: it would require a different dispersion model, and modifications of the parameter estimation and robot control algorithms. Finally, the search in an area with obstacles and an unknown map would require robots to be equipped with appropriate ranging sensors for localisation and mapping. Carrying out autonomously and simultaneously three functions: search, localisation and mapping, is the ultimate goal of this research.