Null-Space-Based Multi-Player Pursuit-Evasion Games Using Minimum and Maximum Approximation Functions

Abstract

In this article, pursuit and evasion policies are developed for multi-player pursuit–evasion games, while obstacle avoidance and velocity constraints are considered simultaneously. As minimum and maximum approximation functions are both differentiable, pursuit and evasion objectives can be transformed into solving the corresponding differential expressions. For obstacle avoidance, a modified null-space-based approach is designed, which can ensure that all pursuers and evaders of pursuit–evasions are safe to minimize pursuit objective and maximize evasion objective, respectively. Rigorous theoretical analyses are provided to design constrained pursuit and evasion policies with obstacle avoidance. Finally, the performance of proposed policies is demonstrated by simulation results in 3-dimensional space.

1. Introduction

In practice, there are a great number of applications that can be formulated as multi-player pursuit–evasion (PE) games, such as aircraft control [1], missile guidance [2], underwater defense [3], etc. [4,5,6]. Basically, there are two parties in multi-player PE games, which are generally called pursuers and evaders, respectively. The main objective of pursuers is to capture as many evaders as possible, whereas evaders intend to escape. The PE game is one basic formulation of differential games, which can be seen as a subproblem for complex game applications, especially N vs. 1 PE games. Starting with the outstanding work of Isaacs in [7], a great number of researchers have been dedicated to studying multi-player PE games from various perspectives of different fields, such as control, multi-objective optimization, differential equations, machine learning, robotics, etc.

As the basis of PE games, the N vs. 1 PE game, which has N pursuers and 1 evader, has been studied extensively and deeply. By introducing a concept of hyper-reachable domain, the PE differential game for satellites has been investigated in [8] with continuous thrust, whereas a sufficient condition has been proposed for capture using the hyper-reachable domain. In convex polygonal environments, two pursuit strategies have been proposed for a discrete-time PE game in [9]. These strategies can give upper bounds on the time-to-capture and then ensure the capture will occur exactly in finite time. For one superior evader with large velocity, authors in [10] analyzed the minimum number of pursuers to guarantee capture. Given appropriate initial configurations, the necessary and sufficient conditions have been presented to regularize the encirclement of PE games. For a PE game with N pursuers, the min–max Q-learning algorithm was developed in [11]. The high-dimensional state space has been transformed into a low-dimensional manifold, and the continuous action space has been transformed into a feature-based space simultaneously. Considering a single and faster evader, authors in [12] proposed a deep reinforcement learning approach for PE games with multiple homogeneous pursuers. A reward structure was built, including individual and group rewards, while the shared experience has been used to train a policy for a given number of pursuers. Although the learning-based solution to PE games has drawn much attention in recent years, the traditional solution still plays a significant role in practical applications.

For PE games with multiple pursuers and multiple evaders, there are a great number of studies, some of which may be more complicated than the N vs. 1 PE game. A general method is to decompose a N vs. M PE game into several N vs. 1 subproblems via task assignment, which mainly use geometrical information of game configurations. In [13], a dynamic divide and conquer approach is employed to simplify the PE game according to the instantaneous configuration. The proposed task allocation algorithm is based on the Apollonius circle, which can guarantee the fastest capture. In the convex domain, analytical barriers were solved in [14] to separate pursuit and evasion winning regions. Then, task assignment can be transformed into a simplified 0–1 integer programming that can utilize geometrical information due to proposed analytical barriers. In addition, a practical and effective method is to solve partial differential equations, which can obtain pursuit and evasion policies directly. For multi-player PE games with $N\ge M$, a continuously differentiable value function was designed in [15], for which saddle-point strategies can be obtained by applying the Hamilton–Jacobi–Isaacs approach. In bounded convex environments, a distributed algorithm was developed for multi-player PE games in [16]. Although evasion policies are unavailable for pursuers, a global area-minimization strategy was proposed using the Voronoi tessellation, which leads to the successful capture in finite time. Compared with solutions based on geometrical information, solutions based on differential equations can be easily extended into high-dimensional spaces.

In practical applications, obstacles are ubiquitous and must be avoided to guarantee safety for robots, e.g., aircrafs flying in city environments, underwater robots cruising in reef waters, etc. Generally, traditional methods of obstacle avoidance include artificial potential field [17], genetic algorithm [18], neural network based method [19], learning-based method [20], etc. There are many studies of PE games with avoidance, such as [21,22,23]. In addition, a practical and effective approach is to utilize the null space of an obstacle avoidance task, which can address the conflict between different objectives [24]. The kernel idea of the null-space-based approach is to project the low-level task vector into the null space of the higher task vector. For an autonomous underwater vehicle, the obstacle avoidance strategy based on the null-space-based approach was proposed in [25], which can ensure the robot moving to target safely. For a 3-dimensional multi-robot formation problem, null-space-based obstacle avoidance was proposed in [26] combining the nonlinear model predictive control. It can lead to robots that can keep stable formation while simultaneously avoiding obstacles. Because there are two tasks for every player in PE games, i.e., pursuit/evasion objective and obstacle avoidance, it is reasonable and practical to select a null-space-based approach to study multi-player PE games in obstacle environments.

Since a great number of applications can be formulated as PE games, there have been fruitful achievements to solve PE games with differential kinematics, input constraints, obstacles, etc. Compared with the existing literature, this article uses minimum and maximum approximation functions to design differentiable performance index functions for multiple pursuers and multiple evaders, respectively. In addition, the null-space-based obstacle avoidance is proposed to guarantee safety for all players. The main contributions of this work can be summarized as follows:

1.

For pursuers and evaders, corresponding differentiable objectives are designed using minimum and maximum approximation functions, respectively.

2.

A modified null-space-based approach is applied for obstacle avoidance, which can guarantee safety for all players in the game process.

3.

Pursuit and evasion policies are explicitly provided for multi-player PE games considering obstacle avoidance and velocity constraints simultaneously.

This article is organized as follows. Section 2 formulates multi-player PE games and introduces differentiable minimum and maximum approximation functions. Section 3 designs modified null-space-based obstacle avoidance, constrained pursuit, and evasion policies. Section 4 provides simulation results to demonstrate the performance of proposed policies in 3-dimensional space. Finally, this article is concluded in Section 5.

2. Problem Formulation and Preliminaries

As the most representative differential game, the PE game has drawn a great amount of attention in theoretical analyses and practical applications. Generally, complicated differential games can be simplified as several PE games and then they can be well solved if the PE games are studied completely. For example, there are three parties in a reach–avoid game, i.e., attackers, targets, and defenders, whereas attackers vs. targets and attackers vs. defenders can be seen as two PE games. Therefore, to solve differential games, the PE game is a vital problem that needs to be studied carefully. In addition, there are many antagonistic tasks formulated as PE games, such as air-to-air combat, underwater defense, missile guidance, etc. To obtain pursuit and evasion policies, this article addresses PE games in the obstacle environment, which needs to consider game objective and safety simultaneously. In multi-player PE games, the classical kinematics of players in 3-dimensional space can be described by some first-order differential equations, i.e.,

$$\left\{\begin{array}{c}{\dot{p}}_i=u_i\hfill \\ {\dot{e}}_j=v_j\hfill \end{array}\right.,$$

(1)

where $p_i={[p_{x,i},p_{y,i},p_{z,i}]}^{\mathrm{T}},i=1,2,\cdots ,N$ and $e_j={[e_{x,j},e_{y,j},e_{z,j}]}^{\mathrm{T}},j=1,2,\cdots ,M$ are locations of the ith pursuer and the jth evader, respectively; $u_i={[u_{x,i},u_{y,i},u_{z,i}]}^{\mathrm{T}}$ and $v_j={[v_{x,j},v_{y,j},v_{z,j}]}^{\mathrm{T}}$ are constrained control inputs, which are saturated as $\parallel u_i\parallel \le u_{max,i}$ and $\parallel v_j\parallel \le v_{max,j}$, respectively. Initially, the relative distances between two parties of the PE game, i.e., pursuers and evaders, are neither too small nor too large, which is important to ensure that PE games make sense. Given initial location configuration of PE games, the existence and uniqueness of motion trajectories generated by Equation (1) can be guaranteed if pursuit and evasion policies are fixed, i.e., $u_i$ and $v_j$. As the major objective of this article is to design pursuit and evasion polices, Equation (1) is used to represent kinematics of players. Practically, the following proposed policies can be considered as control references for actual robots with dynamics. Generally, the control functional classes can be velocity, heading, etc., in practical applications. For most of robots described by differential kinematics, these control functional classes will be viewed as control references to operate drive motors, rudder propellers, steering gears for unmanned aerial vehicles, autonomous underwater vehicles, unmanned ground vehicles, etc. From the view of pursuers, two assumptions are provided in this article as follows:

Assumption 1.

If an evader has been captured by any pursuer in PE games, we assume that it will be destroyed. Then, it can be considered to have vanished, which implies that we will not take it into account during the following game.

Assumption 2.

At least one pursuer is faster than all evaders, i.e., $\exists i\le N,u_{max,i}>v_{max,j},\forall j=1,2,\cdots ,M$.

Remark 1. 

For the sake of pursuer, Assumption 2 is provided in this article, which can guarantee that the fastest pursuer can take charge of PE games even if $N<M$. However, this velocity assumption can be relaxed in some cases, such as allowing some evaders to escape successfully, with initial appropriate configuration, in space expulsion tasks, etc.

For each evader, the minimum distance between it and all pursuers can be denoted by

$${\alpha }_j=min\left\{\parallel e_j-p_1\parallel ,\cdots ,\parallel e_j-p_N\parallel \right\},j=1,2,\cdots ,M,$$

where $\parallel e_j-p_i\parallel$ denotes the classical Euclidean distance between the jth evader and the ith pursuer in the PE game. Of these minimum distances, the maximum one can be denoted by

$$\beta =max\left\{{\kappa }_1{\alpha }_1,{\kappa }_2{\alpha }_2,\cdots ,{\kappa }_M{\alpha }_M\right\},$$

whereas the minimum one can be denoted by

$$\gamma =min\left\{{\kappa }_1{\alpha }_1,{\kappa }_2{\alpha }_2,\cdots ,{\kappa }_M{\alpha }_M\right\},$$

where ${\kappa }_j$ is the survival flag of the jth evader, which can be represented by

$${\kappa }_j=\left\{\begin{array}{cc}1,\hfill & j\mathrm{th}\mathrm{evader}\mathrm{is}\mathrm{survival}\hfill \\ 0,\hfill & j\mathrm{th}\mathrm{evader}\mathrm{is}\mathrm{captured}\mathrm{and}\mathrm{then}\mathrm{vanishes}\hfill \end{array}\right..$$

Once any one of all minimum distances, i.e., ${\alpha }_1,{\alpha }_2,\cdots ,{\alpha }_M$, reduces to zero, it is clear that the corresponding evader will be captured by pursuers. Then, from the view of pursuers, it is natural to make all minimum distances as small as possible. One practical approach is to minimize the maximum one, i.e., $\beta$, and then ${\alpha }_1,{\alpha }_2,\cdots ,{\alpha }_M$ all will be smaller. Therefore, it follows that pursuers may probably capture all evaders, which implies that the performance index function for all pursuers can be designed as follows:

$$J_p=\underset{\parallel u_i\parallel \le u_{max,i}}{min}\beta .$$

(2)

To avoid capture, the evasion objective is to get all evaders far from pursuers, which implies that all minimum distances, i.e., ${\alpha }_1,{\alpha }_2,\cdots ,{\alpha }_M$, become as large as possible. From the view of evaders, one practical approach is to maximize the minimum one, i.e., $\gamma$, and then ${\alpha }_1,{\alpha }_2,\cdots ,{\alpha }_M$ all will be larger. Therefore, it leads to all evaders probably escaping without being captured by pursuers, which implies that the performance index function for all evaders can be designed as follows:

$$J_e=\underset{\parallel v_j\parallel \le v_{max,j}}{max}\gamma .$$

(3)

Because $\beta$ and $\gamma$ are both non-differentiable, it is hard or impossible to obtain polices of pursuit evasion games via $J_p$ and $J_e$. Therefore, an intuitive idea is to find some appropriate differentiable functions to approximate $\beta$ and $\gamma$. Referring to [27,28], two ingenious lemmas of minimum and maximum approximation functions can be given as follows.

Lemma 1. 

Given a set of L positive numbers $\left\{a_1,\cdots ,a_L\right\}$, the minimum value $a_m={\min }_{l\in L}\left\{a_l\right\}$ can be approximated by two minimum differentiable functions, i.e.,

$${\underline{\sigma }}_{\delta }(a_1,\cdots ,a_L)=\sqrt[\delta ]{{\displaystyle \frac{1}{{\sum }_{l=1}^L{a}_l^{-\delta }}}},{\overline{\sigma }}_{\delta }(a_1,\cdots ,a_L)=\sqrt[\delta ]{{\displaystyle \frac{L}{{\sum }_{l=1}^L{a}_l^{-\delta }}}},$$

which satisfy the following properties:

$${\underline{\sigma }}_{\delta }\le a_m\le {\overline{\sigma }}_{\delta },\underset{\delta \to \infty }{lim}{\underline{\sigma }}_{\delta }=\underset{\delta \to \infty }{lim}{\overline{\sigma }}_{\delta }=a_m.$$

Lemma 2. 

Given a set of L positive numbers $\left\{a_1,\cdots ,a_L\right\}$, the maximum value $a_M={\max }_{l\in L}\left\{a_l\right\}$ can be approximated by two maximum differentiable functions, i.e.,

$${\underline{\rho }}_{\delta }(a_1,a_2,\cdots ,a_L)=\sqrt[\delta ]{{\displaystyle \frac{{\sum }_{l=1}^L{a}_l^{\delta }}{L}}},{\overline{\rho }}_{\delta }(a_1,a_2,\cdots ,a_L)=\sqrt[\delta ]{\sum _{l=1}^L{a}_l^{\delta }},$$

which satisfy the following properties:

$${\underline{\rho }}_{\delta }\le a_M\le {\overline{\rho }}_{\delta },\underset{\delta \to \infty }{lim}{\underline{\rho }}_{\delta }=\underset{\delta \to \infty }{lim}{\overline{\rho }}_{\delta }=a_M.$$

The rigorous theoretical proofs can be found in [27,28]. Based on the two above lemmas, $\alpha$, $\beta$, and $\gamma$ can be approximated by these differentiable minimum and maximum approximation functions, which can be represented by

$${\alpha }_j^p(e_j,p)={\overline{\sigma }}_{\delta }\left(\parallel e_j-p_1\parallel ,\cdots ,\parallel e_j-p_N\parallel \right)={\left({\displaystyle \frac{N}{{\sum }_{i=1}^N{\parallel e_j-p_i\parallel }^{-\delta }}}\right)}^{{\displaystyle \frac{1}{\delta }}},$$

$$\widehat{\beta }(e,p)={\overline{\rho }}_{\delta }\left({\kappa }_1{\alpha }_1^p(e_1,p),\cdots ,{\kappa }_M{\alpha }_M^p(e_M,p)\right)={\left(\sum _{j=1}^M{\displaystyle \frac{{\kappa }_j^{\delta }N}{{\sum }_{i=1}^N{\parallel e_j-p_i\parallel }^{-\delta }}}\right)}^{{\displaystyle \frac{1}{\delta }}},$$

$${\alpha }_j^e(e_j,p)={\underline{\sigma }}_{\delta }\left(\parallel e_j-p_1\parallel ,\cdots ,\parallel e_j-p_N\parallel \right)={\left({\displaystyle \frac{1}{{\sum }_{i=1}^N{\parallel e_j-p_i\parallel }^{-\delta }}}\right)}^{{\displaystyle \frac{1}{\delta }}},$$

$$\widehat{\gamma }={\underline{\sigma }}_{\delta }\left({\kappa }_1{\alpha }_1^e(e_1,p),\cdots ,{\kappa }_M{\alpha }_M^e(e_M,p)\right)={\left({\displaystyle \frac{1}{{\sum }_{j=1}^M{\kappa }_j^{-\delta }{\sum }_{i=1}^N{\parallel e_j-p_i\parallel }^{-\delta }}}\right)}^{{\displaystyle \frac{1}{\delta }}}.$$

Then, the objectives for pursuers and evaders, i.e., $J_p$ and $J_e$, can be modified by

$${\widehat{J}}_p=\underset{\parallel u_i\parallel \le u_{max,i}}{min}\widehat{\beta },$$

(4)

$${\widehat{J}}_e=\underset{\parallel v_j\parallel \le v_{max,j}}{max}\widehat{\gamma }.$$

(5)

3. Multi-Player Pursuit–Evasion Games with Obstacles

For PE games with obstacles, there will be an safe objective for every player that they have to avoid obstacles. Generally, the behavior based approach, the artificial potential field approach, the dynamic window approach, etc., can be used to keep away from obstacles. Compared with minimizing $J_p$ and maximum $J_e$, the safety task, i.e., obstacle avoidance, has high priority for all players. Therefore, we employ the null-space-based approach to obstacle avoidance for multi-player PE games when considering $J_p$, $J_e$, and safety simultaneously. The null-space-based obstacle avoidance is one of the behavior based approaches, which has been applied in mobile robots, autonomous underwater vehicles, and other applications. [24,25,29,30].

3.1. Null-Space Based Obstacle Avoidance

For multi-player PE games, if there is no danger for one player, the main objective is to capture or escape. If the distance between one obstacle and one player is smaller than some safety threshold value, the main objective is to avoid the obstacle, and the pursuit and evasion objectives should be considered simultaneously.

Based on the different priorities, the null-space-based approach is to make the most of the null space of safety objective. Then, the conflict between safety and pursuit or evasion can be eliminated in multi-player PE games, whereas the pursuit and evasion objectives can also be taken into account. Based on the configuration of player and obstacle, a task variable can be defined as a distance function [24], i.e.,

$$\tau (g_s,o_k)=\parallel g_s-o_k\parallel ,$$

(6)

where $g_s$ can be any ith pursuer or jth evader, and $o_k$ is the location of the kth obstacle and $k=1,2,\cdots ,K$. In this article, we assume that all obstacles are static, but the time-varying $o_k$ can also be addressed via the proposed method [30]. Therefore, the differential equivalent can be represented by

$$\dot{\tau }(g_s,o_k)={\displaystyle \frac{\partial \parallel g_s-o_k\parallel }{\partial g_s}}{\dot{g}}_s={\displaystyle \frac{{(g_s-o_k)}^{\mathrm{T}}}{\parallel g_s-o_k\parallel }}{\dot{g}}_s\triangleq {\mathbf{J}}_{s,k}{\mu }_s,$$

(7)

where ${\mu }_s$ can be any $u_i$ or $v_j$, while ${\mathbf{J}}_{s,k}\in {\mathbb{R}}^{1\times 3}$ is the configuration-dependent Jacobian matrix.

Referring to [24,25], the desired task variable can be selected as ${\tau }_d(g_s,o_k)=d_{s,k}$, where $d_{s,k}$ is the safety threshold value between player $g_k$ and obstacle $o_k$. Due to $\mathbf{rank}$$\left({\mathbf{J}}_{s,k}\right)=1$, we can make use of Moore–Penrose Jacobian to design the minimum-norm velocity that can fulfill the obstacle avoidance task. Therefore, the desired control input for obstacle avoidance can be designed as

$$\begin{array}{cc}\hfill {\mu }_s^o=& {\mathbf{J}}_{s,k}^{†}\left({\dot{\tau }}_d(g_s,o_k)+{\lambda }_1\stackrel{\sim }{\tau }(g_s,o_k)\right)={\lambda }_1{\mathbf{J}}_{s,k}^{\mathrm{T}}{\left({\mathbf{J}}_{s,k}{\mathbf{J}}_{s,k}^{\mathrm{T}}\right)}^{-1}\left(d_{s,k}-\parallel g_s-o_k\parallel \right),\hfill \end{array}$$

(8)

where ${\lambda }_1>0$ is a designed parameter and $\stackrel{\sim }{\tau }(g_s,o_k)=d_{s,k}-\parallel g_s-o_k\parallel$ is the task error. The null space of ${\mathbf{J}}_{s,k}$ can be obtained by

$$\mathcal{N}\left({\mathbf{J}}_{s,k}\right)=I_3-{\mathbf{J}}_{s,k}^{†}{\mathbf{J}}_{s,k}=I_3-{\displaystyle \frac{(g_s-o_k){(g_s-o_k)}^{\mathrm{T}}}{\parallel g_s-o_k{\parallel }^2}},$$

where $I_3$ is the identity matrix. Therefore, if there will be some danger from the kth obstacle, i.e., $\parallel g_s-o_k\parallel \le d_{s,k}$, the overall control input can be designed as

$$\begin{array}{c}\hfill {\overline{\mu }}_s={\mu }_s^o+{\lambda }_2\mathcal{N}\left({\mathbf{J}}_{s,k}\right){\mu }_s^{p\left(e\right)},\end{array}$$

(9)

where ${\lambda }_2>0$ is a designed parameter and ${\mu }_s^{p\left(e\right)}$ is the control input for pursuit or evasion objectives, which will be designed by applying $\widehat{\beta }$ and $\widehat{\gamma }$ in detail. The second term, i.e., $\mathcal{N}\left({\mathbf{J}}_{s,k}\right){\mu }_s^{p\left(e\right)}$, implies that the low-priority task is projected onto the null space of the immediately high-priority task. Then, it can lead to removing the velocity components of the low-priority task that would conflict with the high-priority task.

It is worth noting that there is a special case that ${\mu }_s^o=\mathbf{0}$ and the projection of ${\mu }_s^{p\left(e\right)}$ onto the null space $\mathcal{N}\left({\mathbf{J}}_{s,k}\right)$ is a zero vector simultaneously. It leads to the overall control input ${\overline{\mu }}_s=\mathbf{0}$, and then the player of PE games will be frozen and no longer move. To address this special case, a small modification can be employed, which is to rotate ${\mu }_s^{p\left(e\right)}$ with a small angle, shown in Figure 1. It can make the projection onto the null space be nonzero. Therefore, the overall control input can be modified as follows:

$$\begin{array}{c}\hfill {\overline{\mu }}_s=\left\{\begin{array}{cc}\mathcal{N}\left({\mathbf{J}}_{s,k}\right)R\left(\epsilon \right){\mu }_s^{p\left(e\right)},\hfill & \parallel {\mu }_s^o+{\lambda }_2\mathcal{N}\left({\mathbf{J}}_{s,k}\right){\mu }_s^{p\left(e\right)}\parallel =0\hfill \\ {\mu }_s^o+{\lambda }_2\mathcal{N}\left({\mathbf{J}}_{s,k}\right){\mu }_s^{p\left(e\right)},\hfill & \mathrm{otherwise}\hfill \end{array}\right.,\end{array}$$

(10)

where $R\left(\epsilon \right)\in {\mathbb{R}}^{3\times 3}$ is the 3-dimensional rotation matrix with small $\epsilon$, which can be represented by

$$\begin{array}{cc}\hfill R\left(\epsilon \right)=& \left[\begin{array}{ccc}{cos}^2\epsilon & -sin\epsilon cos\epsilon +cos\epsilon {sin}^2\epsilon & {sin}^2\epsilon +{cos}^2\epsilon sin\epsilon \\ sin\epsilon cos\epsilon & {cos}^2\epsilon +{sin}^3\epsilon & -cos\epsilon sin\epsilon +{sin}^2\epsilon cos\epsilon \\ -sin\epsilon & cos\epsilon sin\epsilon & {cos}^2\epsilon \end{array}\right].\hfill \end{array}$$

Generally, $\epsilon$ is a minor angle that can be either positive or negative. We can select $\epsilon =\frac{\pi }{180}$ as a trivial example.

Remark 2. 

In practice, the design of a safety threshold value mainly depends on the electromechanical characteristics of players in PE games. In addition, the surface gradient of obstacles also needs to be considered, whereas if players are constrained by rotation capacity, the rotation radius should also be taken into account. For some player and obstacle, the corresponding safety threshold value should be different. To simplify, we can select the largest one for each obstacle.

3.2. Pursuit Policies

For N pursuers, the main objective is to capture all evaders and avoid obstacles simultaneously. Based on the above null-space-based approach (8), the corresponding control input for obstacle avoidance can be represented by

$$\begin{array}{c}\hfill u_i^o={\lambda }_1{\displaystyle \frac{p_i-o_k}{\parallel p_i-o_k\parallel }}(d_{i,k}-\parallel p_i-o_k\parallel ).\end{array}$$

(11)

If there is no danger for the ith pursuer, the main objective for PE games is to minimize $\widehat{\beta }$, i.e., Equation (4). Therefore, the control input for the pursuit objective can be obtained by

$$\begin{array}{cc}\hfill u_i^p=& arg\underset{\parallel u_i\parallel \le u_{max,i}}{min}{\left({\displaystyle \frac{\partial \widehat{\beta }(e,p)}{\partial p_i}}\right)}^{\mathrm{T}}{u}_i.\hfill \end{array}$$

(12)

To explicitly express pursuit policies, the following lemma is provided.

Lemma 3. 

For $x,y,z\in \mathbb{R}$ with at most one zero variable and $c>0$, an optimization problem can be described by

$$\begin{array}{c}\hfill O{P}_1:\underset{c_1^2+c_2^2+c_3^2=c^2}{min}\left\{c_{1}x+c_{2}y+c_{3}z\right\},\end{array}$$

and then the corresponding solution can be represented by

$$\begin{array}{c}\hfill c_1=-{\displaystyle \frac{cx}{\sqrt{x^2+y^2+z^2}}},c_2=-{\displaystyle \frac{cy}{\sqrt{x^2+y^2+z^2}}},c_3=-{\displaystyle \frac{cz}{\sqrt{x^2+y^2+z^2}}}.\end{array}$$

(13)

In addition, another optimization problem can be described by

$$\begin{array}{c}\hfill O{P}_2:\underset{c_1^2+c_2^2+c_3^2=c^2}{max}\left\{c_{1}x+c_{2}y+c_{3}z\right\},\end{array}$$

and then the corresponding solution can be represented by

$$\begin{array}{c}\hfill c_1={\displaystyle \frac{cx}{\sqrt{x^2+y^2+z^2}}},c_2={\displaystyle \frac{cy}{\sqrt{x^2+y^2+z^2}}},c_3={\displaystyle \frac{cz}{\sqrt{x^2+y^2+z^2}}}.\end{array}$$

(14)

Proof. 

Due to $c_1^2+c_2^2+c_3^2=c^2$, an appropriate variable substitution is to define two new variables $\theta ,\psi \in [0,\frac{\pi }{2}]$. For sake of expression, we assume that $x,y\ne 0$. For the first optimization problem $O{P}_1$, we can assume that

$$\begin{array}{c}\hfill c_1=-csin\theta \mathrm{sign}\left(x\right),c_2=-ccos\theta sin\psi \mathrm{sign}\left(y\right),c_3=-ccos\theta cos\psi \mathrm{sign}\left(z\right).\end{array}$$

(15)

Then, the solutions to $c_1$, $c_2$, and $c_3$ are changed to seek optimized $\theta$ and $\psi$. By this variable substitution, $O{P}_1$ can be transformed as follows:

$$\begin{array}{cc}\hfill O{P}_1\iff & \underset{0\le \theta ,\psi \le \frac{\pi }{2}}{min}\left\{-csin\theta \left|x\right|-ccos\theta sin\psi \left|y\right|-ccos\theta cos\psi \left|z\right|\right\}\hfill \\ \hfill \iff & \underset{0\le \theta ,\psi \le \frac{\pi }{2}}{max}\left\{sin\theta \left|x\right|+cos\theta sin\psi \left|y\right|+cos\theta cos\psi \left|z\right|\right\}.\hfill \end{array}$$

According to the uncoupling property, a subproblem of $O{P}_1$ can be solved with the following process, i.e.,

$$\begin{array}{cc}\hfill {\overline{OP}}_1:& \underset{0\le \psi \le \frac{\pi }{2}}{max}\left\{sin\psi \left|y\right|+cos\psi \left|z\right|\right\}\hfill \\ \hfill \iff & \underset{0\le \psi \le \frac{\pi }{2}}{max}\left\{\sqrt{y^2+z^2}sin\left(\psi +arctan\left({\displaystyle \frac{\left|z\right|}{\left|y\right|}}\right)\right)\right\}\hfill \\ \hfill \iff & \psi ={\displaystyle \frac{\pi }{2}}-arctan\left({\displaystyle \frac{\left|z\right|}{\left|y\right|}}\right).\hfill \end{array}$$

Based on this solution to $\psi$, $O{P}_1$ can be solved as follows:

$$\begin{array}{cc}\hfill O{P}_1\iff & \underset{0\le \theta \le \frac{\pi }{2}}{max}\left\{sin\theta \left|x\right|+cos\theta \sqrt{y^2+z^2}\right\}\hfill \\ \hfill \iff & \underset{0\le \theta \le \frac{\pi }{2}}{max}\left\{\sqrt{x^2+y^2+z^2}sin\left(\theta +arctan\left({\displaystyle \frac{\sqrt{y^2+z^2}}{\left|x\right|}}\right)\right)\right\}\hfill \\ \hfill \iff & \theta ={\displaystyle \frac{\pi }{2}}-arctan\left({\displaystyle \frac{\sqrt{y^2+z^2}}{\left|x\right|}}\right).\hfill \end{array}$$

Therefore, the corresponding solution to $O{P}_1$, i.e., Equation (13), can be obtained via the inverse variable substitution.

To solve the second optimization problem $O{P}_2$, a relevant variable substitution can be provided as follows:

$$\begin{array}{c}\hfill c_1=csin\theta \mathrm{sign}\left(x\right),c_2=ccos\theta sin\psi \mathrm{sign}\left(y\right),c_3=ccos\theta cos\psi \mathrm{sign}\left(z\right).\end{array}$$

(16)

where $\theta ,\psi \in [0,\frac{\pi }{2}]$ are two new variables to be optimized. Similar to the above analyses, the corresponding solution to $O{P}_2$, i.e., Equation (14), can be obtained via the inverse variable substitution.

Therefore, the proof is complete. □

According to Lemma 3, the control input for pursuit objective can be calculated as

$$\begin{array}{c}\hfill u_i^p=-{\displaystyle \frac{u_{max,i}}{∥{\displaystyle \frac{\partial \widehat{\beta }(e,p)}{\partial p_i}}∥}}{\displaystyle \frac{\partial \widehat{\beta }(e,p)}{\partial p_i}},\end{array}$$

(17)

where the gradient value can be calculated as

$$\begin{array}{cc}\hfill {\displaystyle \frac{\partial \widehat{\beta }(e,p)}{\partial p_i}}=& N{\left(\sum _{j=1}^M{\displaystyle \frac{{\kappa }_j^{\delta }N}{{\sum }_{i=1}^N{\parallel e_j-p_i\parallel }^{-\delta }}}\right)}^{{\displaystyle \frac{1}{\delta }}-1}\left(\sum _{j=1}^M{\displaystyle \frac{{\kappa }_j^{\delta }{\parallel e_j-p_i\parallel }^{-\delta -2}(p_i-e_j)}{{\left({\sum }_{i=1}^N{\parallel e_j-p_i\parallel }^{-\delta }\right)}^2}}\right).\hfill \end{array}$$

Considering the pursuit objective and obstacle avoidance simultaneously, the unconstrained control input can be designed as follows:

$$\begin{array}{c}\hfill u_i=\left\{\begin{array}{cc}{u}_i^p,\hfill & \parallel p_i-o_k\parallel >d_{i,k},\forall k\hfill \\ {\overline{u}}_i,\hfill & \parallel p_i-o_k\parallel \le d_{i,k}\hfill \end{array}\right.,\end{array}$$

(18)

where ${\overline{u}}_i$ can be calculated according to Equations (10), (11), and (17), i.e.,

$$\begin{array}{c}\hfill {\overline{u}}_i=\left\{\begin{array}{cc}\mathcal{N}\left({\mathbf{J}}_{i,k}\right)R\left(\epsilon \right)u_i^p,\hfill & \parallel u_i^o+{\lambda }_2\mathcal{N}\left({\mathbf{J}}_{i,k}\right)u_i^p\parallel =0\hfill \\ u_i^o+{\lambda }_2\mathcal{N}\left({\mathbf{J}}_{i,k}\right)u_i^p,\hfill & \mathrm{otherwise}\hfill \end{array}\right..\end{array}$$

Considering the saturation limitation $u_{max,i}$, the pursuit policy of the ith pursuer based on Equation (18) can be designed by

$$\begin{array}{c}\hfill {\widehat{u}}_i=\left\{\begin{array}{cc}{u}_i,\hfill & \parallel u_i\parallel \le u_{max,i}\hfill \\ u_{max,i}{\displaystyle \frac{u_i}{\parallel u_i\parallel }},\hfill & \parallel u_i\parallel >u_{max,i}\hfill \end{array}\right..\end{array}$$

(19)

3.3. Evasion Policies

For M evaders, the main objective is to escape and avoid obstacles. By applying the null-space-based approach, the corresponding control input for obstacle avoidance can be obtained from Equation (8) as follows:

$$\begin{array}{c}\hfill v_j^o={\lambda }_1{\displaystyle \frac{e_i-o_k}{\parallel e_j-o_k\parallel }}(d_{j,k}-\parallel e_j-o_k\parallel ),\end{array}$$

(20)

If there is no danger for the jth evader, the main objective for PE games is to maximize $\widehat{\gamma }$, i.e., Equation (5). Therefore, according to Lemma 3, the control input for evasion objective can be obtained by

$$\begin{array}{cc}\hfill v_j^e=& arg\underset{\parallel v_j\parallel \le v_{max,j}}{max}{\left({\displaystyle \frac{\partial \widehat{\gamma }(e,p)}{\partial e_j}}\right)}^{\mathrm{T}}{v}_j={\displaystyle \frac{v_{max,j}}{∥{\displaystyle \frac{\partial \widehat{\gamma }(e,p)}{\partial e_j}}∥}}{\displaystyle \frac{\partial \widehat{\gamma }(e,p)}{\partial e_j}},\hfill \end{array}$$

(21)

where the gradient value can be calculated as

$$\begin{array}{cc}\hfill {\displaystyle \frac{\partial \widehat{\gamma }(e,p)}{\partial e_j}}=& {\left({\displaystyle \frac{1}{{\sum }_{j=1}^M{\kappa }_j^{-\delta }{\sum }_{i=1}^N{\parallel e_j-p_i\parallel }^{-\delta }}}\right)}^{{\displaystyle \frac{1}{\delta }}+1}\left({\kappa }_j^{-\delta }\sum _{i=1}^N{\parallel e_j-p_i\parallel }^{-\delta -2}(e_j-p_i)\right).\hfill \end{array}$$

Considering the evasion objective and obstacle avoidance simultaneously, the unconstrained control input can be designed as follows:

$$\begin{array}{c}\hfill v_j=\left\{\begin{array}{cc}{v}_j^e,\hfill & \parallel e_j-o_k\parallel >d_{j,k},\forall k\hfill \\ {\overline{v}}_j,\hfill & \parallel e_j-o_k\parallel \le d_{j,k}\hfill \end{array}\right.,\end{array}$$

(22)

where ${\overline{v}}_j$ can be calculated according to Equations (10), (20), and (21), i.e.,

$$\begin{array}{c}\hfill {\overline{v}}_j=\left\{\begin{array}{cc}\mathcal{N}\left({\mathbf{J}}_{j,k}\right)R\left(\epsilon \right)v_j^e,\hfill & \parallel v_j^o+{\lambda }_2\mathcal{N}\left({\mathbf{J}}_{j,k}\right)v_j^e\parallel =0\hfill \\ v_j^o+{\lambda }_2\mathcal{N}\left({\mathbf{J}}_{j,k}\right)v_j^e,\hfill & \mathrm{otherwise}\hfill \end{array}\right..\end{array}$$

Considering the saturation limitation $v_{max,j}$, the evasion policy of the jth evader based on Equation (22) can be designed by

$$\begin{array}{c}\hfill {\widehat{v}}_j=\left\{\begin{array}{cc}{v}_j,\hfill & \parallel v_j\parallel \le v_{max,j}\hfill \\ v_{max,j}{\displaystyle \frac{v_j}{\parallel v_j\parallel }},\hfill & \parallel v_j\parallel >v_{max,j}\hfill \end{array}\right..\end{array}$$

(23)

Remark 3. 

For N pursuers and M evaders, pursuit and evasion policies can be realized by Equations (19) and (23), respectively. Based on locations of players and obstacles, ${\widehat{u}}_i$ and ${\widehat{v}}_j$ can be obtained directly. Therefore, the computational complexity of proposed policies only depends on the numbers of players in PE games, i.e., N and M.

4. Simulation Results

In practice, there are a great number of applications that can be modeled as multi-player PE games, such as air-to-air combat, underwater defense, and others. The player can be the unmanned aerial vehicle, the autonomous underwater vehicle, a mobile robot, and so on. Although their dynamics may vary for different actuators, we can use Equation (1) to describe their kinematics. Then, the proposed pursuit policies (19) and evasion policies (23) can be regarded as desired velocity for actuators. The MATLAB R2015b simulator is utilized to demonstrate the effectiveness of the proposed pursuit and evasion polices, and the system environment includes Windows 10, Intel Core(TM) i7-11370H CPU@3.30 GHz.

In multi-player PE games, we select three pursuers, two evaders, and two obstacles, i.e., $N=3$, $M=2$, and $K=2$. The initial locations of three pursuers are set as $p_1={[-5,0,5]}^{\mathrm{T}}$m, $p_2={[0,0,5]}^{\mathrm{T}}$m, and $p_3={[5,0,5]}^{\mathrm{T}}$m, and the corresponding velocity constraints are $u_{max,1}=2$ m/s, $u_{max,2}=3$ m/s, $u_{max,3}=3$ m/s, respectively. The initial locations of two evaders are set as $e_1={[0,5,5]}^{\mathrm{T}}$m and $e_2={[10,8,5]}^{\mathrm{T}}$m, and the corresponding velocity constraints are $v_{max,1}=2.5$ m/s and $v_{max,2}=2$ m/s. To ensure that two obstacles can affect the multi-player PE games, their locations and safety threshold values are set as $o_1={[10,15,5]}^{\mathrm{T}}$m, $d_{·,1}=5$ m, $o_2={[-2,10,5]}^{\mathrm{T}}$m, and $d_{·,2}=3$ m, respectively. In addition, we select ${\lambda }_1=10$ and ${\lambda }_2=1$.

The space trajectories of all pursuers and evaders are illustrated in Figure 2. Because we set same altitude for all players, all trajectories in the horizontal plane can be shown in Figure 3. It is clear that the first evader $e_1$ is captured by the second pursuer $p_2$ at ${[-9.8460,11.7871,5.0000]}^{\mathrm{T}}$m in $8.2260$ s, while the second evader $e_2$ is captured by the third pursuer $p_3$ at ${[22.3342,21.5044,5.0000]}^{\mathrm{T}}$m in $9.2260$ s. Although $\parallel e_1-p_1\parallel <\parallel e_2-p_1\parallel$, the first pursuer $p_1$ is assigned to the second evader $e_2$ due to its small velocity constraint. In addition, all trajectories can keep away from the two obstacles by corresponding safety threshold values. Applying minimum and maximum approximation functions, $\widehat{\beta }$ and $\widehat{\gamma }$ are illustrated in Figure 4. As $\widehat{\beta }$ is used to approximate the maximum of all minimum distances, it can be seen from Figure 4a that $\widehat{\beta }$ will approach to 0 until both evaders are captured. In addition, as $\widehat{\gamma }$ is used to approximate the minimum of all minimum distances, Figure 4b demonstrates that $\widehat{\gamma }$ approaches 0 if each evader is captured by any pursuers, i.e., at $8.2260$ s and $9.2260$ s, respectively.

To illustrate the null-space-based obstacle avoidance, we can use a weight based approach, which can be represented by

$$\begin{array}{c}\hfill {\overline{\mu }}_s={\mu }_s^o+{\lambda }_2{\mu }_s^{p\left(e\right)},\end{array}$$

which does not take ${\mathbf{J}}_{s,k}$ into account. Two comparison results are given in Figure 5 and Figure 6, which are in 3-dimensional space and the longitudinal plane. For the pursuer, the initial location is $p={[5,-10,0]}^{\mathrm{T}}$m and the velocity constraint is $u_{max}=3$ m/s; for the evader, $e={[5,10,5]}^{\mathrm{T}}$m and $v_{max}=1$ m/s. Similarly, to guarantee that obstacles can affect PE games, we select $o_1={[5,0,3]}^{\mathrm{T}}$m, $d_{·,1}=5$ m, $o_2={[5,15,8]}^{\mathrm{T}}$m, and $d_{·,2}=4$ m, respectively. From Figure 6a, it is clear that the trajectories of pursuer and evader both penetrate the safety threshold spaces, which may lead to some crashes between players and obstacles. It is implies that the player robot may be destroyed before completing game tasks and then advantages can be increased to the other party of PE games. From Figure 6b, both trajectories are outside safety threshold spaces. Therefore, the proposed null-space-based approach can provide safety assurances for all players.

5. Conclusions

For multi-player PE games, we have proposed pursuit and evasion polices while obstacle avoidance and velocity constraints are taken into account simultaneously. Minimum and maximum approximation functions are used to construct differentiable performance index for pursuers and evaders. To avoid obstacles, the null-space-based approach is applied in this article, while the modification is designed for the special case. Then, constrained pursuit and evasion policies are provided via rigorous theoretical analyses. In future work, we can extend the proposed policies to reach–avoid games. Complex system dynamics can be considered, which can lead to more practicability for the proposed policies. In future work, an interesting direction is to study multi-player PE games in convex or nonconvex domains with moving obstacles. In addition, there are still various open problems in multi-player PE games, such as the time-varying number of players, the limited communication bandwidth, the distributed pursuit and evasion policies, etc.