Survey on Mission Planning of Multiple Unmanned Aerial Vehicles

Abstract

The task assignment issue and the path planning problem of Multiple Unmanned Aerial Vehicles (Multi-UAV) are collectively referred to as the Mission Planning Problem (MPP). This review article provides an update on the progress of the MPP on Multi-UAV. Focusing on the burning issue of task assignment, this paper focuses on the comparison of the characteristics of the mathematical programming method, heuristic algorithm, negotiation algorithm and neural networks. According to different constraints, trajectory planning can be divided into 2 dimension coverage, 3 dimension cooperation, and 4 dimension space-time cooperation. Combined with typical research, common collaborative guidance methods are introduced, and the key development direction of this field is prospected. The article shows that, although the MPP has been extensively studied, ongoing research is required. In particular, it is necessary to pay attention to the timeliness of the task assignment, the information coupling exists in MPP, and the problems caused by multiple constraints of Multi-UAV and environmental uncertainty.

1. Introduction

Over the last decades, Unmanned Aerial Vehicles (UAVs) have increasingly been widely used in a range of applications, such as surveillance [1,2], agriculture [3,4], photogrammetry [5], civil security [6] and even the military [7]. With the development of hardware and software that enables higher levels of autonomy, the types of missions that UAVs handle have become more intelligent. From the perspective of military applications, smart UAVs can stay on the battlefield for a long time to conduct long-term reconnaissance and surveillance. And they can even conduct strike missions, and can rely on the information-based combat system to resend the striking effect for damage assessment. At the same time, it is composed of a distributed decentralized network, which is highly replaceable and has little impact on the breakdown of the system. It can easily replace various workloads and is highly adaptable to diverse combat tasks. Compared with manned aircraft, UAVs have a significant cost-effective advantage, which can consume a large amount of the enemy’s high-cost air defence force to achieve efficient and cost-effective combat. In terms of platform design, it has subverted the equipment design idea of the main battle platform of the previous combat system as the support, focusing on high performance and large integration. Therefore, intelligent UAVs will definitely become a research hotspot and further improve their level of automation and intelligence to enhance their role in various applications.

At present, there are bottlenecks in the detection range, penetration capability and the striking effect of a single UAV. The trade-off between low-cost and complex functions limits the development of UAVs. However, through the teamwork of UAVs with simple functions, the system functions are distributed to multiple individuals and form a scale effect. The effectiveness can match or even exceed an advanced single UAV. In the foreseeable future, complex and changeable missions will require the teamwork of heterogeneous unmanned system to achieve common objectives. At the same time, Multiple Unmanned Aerial Vehicles (Multi-UAV) improve the robustness of the entire system and are more adaptable to complex situations. Therefore, the unmanned formation will become the most important part of the future battlefield, and it will act in the manner of implementing saturated attack operations, active attrition operations, and distributed attack-defense operations, which will bring new challenges to the design and integration of UAVs, especially in terms of autonomy and cooperation [8].

Relying on the network, implementing coordinated attack and defense in the form of formation through autonomous coordination is the basic combat style of low-cost UAVs. The decision-making system provides an efficient task assignment strategy that can be implemented by the unmanned system, and the management system ensures the reliable operation of the formation. The decision-making and management system determines the degree of autonomy of the formation and the level of forming self-organization management.

In short, task assignment is to divide the tasks of formation members according to the battlefield situation and effectiveness function, and it is the command input of path planning [9,10]. Path planning is the executor of the assignment results [11,12]. Therefore, task assignment and path planning are the basis for determining whether the formation’s comprehensive combat effectiveness can be maximized, and are the key basis for formation coordination. In practice, however, information coupling naturally exists in Mission Planning Problem (MPP), since both the implementation of task assignment and path planning reference the output of each other [13]. Path planning algorithms must have the ability to replan the trajectories online in complex constraints and dynamic environments to deal with emergencies. However, the change in the motion path and the dynamic of the scene also requires the task allocation algorithm to have the ability of online reassignment. Although it is not necessarily real-time, it must be online. Many scholars have carried out researches on task assignment, path planning, and their coupling respectively [14].

The remainder of the paper is organized as follows. In Section 2, the mathematical model and common algorithms of task assignment are reviewed. In Section 3, several usual cooperative path planning scenarios and algorithms are introduced. Then the current challenges and future research directions are discussed in Section 4. Finally, Section 5 draws the conclusion.

2. Task Assignment of Multi-UAV

The primary problem that unmanned formation needs to solve is the problem of target assignment. Task assignment refers to assigning one or more targets to one or more aircraft, according to certain tactical indicators under the given situation and performance, so that the overall efficiency of the aircraft formation can be optimized [15]. The task assignment can be divided into pre-assignment and re-assignment. Pre-assignment means offline task allocation based on known scene information before the execution process, and usually is applicable to static problems. Re-assignment refers to reassigning tasks according to new scene information during task execution. The common triggers for reassignment are summarized below.

(1)

Changes in target quantity or characteristics: The number of targets may vary as the mission is performed. Some targets may be destroyed, or some new ones may be detected. And the intelligence gathered earlier about the targets may also be inaccurate. Depending on the new scenario, tasks need to be reassigned.

(2)

Variation in the number or status of UAV formation: During the execution of the mission, UAVs may be attacked and no longer suitable for the assigned mission. Sometimes changes in the flight path caused by dynamic scenes make the UAV more suitable for other tasks. It is necessary to reassign tasks according to the variation.

Figure 1 shows a typical Multi-UAV combat scenario.

In this typical scenario, the intelligence obtained by satellites is used for target pre-allocation, and the command and control center issues tasks to the manned and unmanned swarms, and the unmanned swarms carry out task segmentation and plan flight trajectories through internal negotiation. During the execution of the mission, the no-fly zone is avoided, and collision avoidance within the formation is carried out at the same time.

And its key technologies are shown in Figure 2.

(1)

Formation principle: It can be a single objective function or multiple objective functions. Commonly used functions are time consumption, economic consumption, the proceeds from performing tasks, etc.

(2)

Task assignment algorithm: What calls for special attention is the coupling among tasks and the robustness in uncertain environments.

(3)

Path planning algorithm: The ability of path conflict avoidance is required in cluttered environment. And the reliability of the cooperative guidance law also deserves attention.

(4)

Leave-join formation management: The formation needs to have the ability to resist disturbance and reconfigure.

Among them, the task assignment is essentially a constrained integer combinatorial optimization problem. When the scale of the problem expands, the time cost of solving the problem will become huge and even unacceptable, so the combinatorial optimization problem is also called Nondeterminism Polynomial (NP) hard problem.

In the existing studies, most of the Multi-UAV are assumed to be heterogeneous, that is, there are differences in the nature, function, load quantity and type, and combat capabilities of the UAVs. At the same time, it is necessary to consider the actual various constraints. The task assignment mathematical model is usually established as below [16].

$$F=max\sum _{i=1}^{N_u}\left(\sum _{j=1}^{N_t}{r}_{ij}{x}_{ij}\right),x_{ij}\in \left\{0,1\right\},$$

(1)

$$H=min\sum _{i=1}^{N_u}\left(\sum _{j=1}^{N_t}{c}_{ij}{x}_{ij}\right),x_{ij}\in \left\{0,1\right\},$$

(2)

$$\sum _{j=1}^{N_t}{x}_{ij}\le L_t^i,\forall i\in I,$$

(3)

$$L_j^{min}\le \sum _{i=1}^{N_u}{x}_{ij}\le L_j^{max},\forall j\in J,$$

(4)

$$\sum _{i=1}^{N_u}\sum _{j=1}^{N_t}{x}_{ij}=min\left\{\sum _{i=1}^{N_u}{L}_t^i,N_t\right\},$$

(5)

In formula (1), $x_{ij}$ indicates that target j is assigned to UAV i, and $r_{ij}$ represents the reward obtained. There are $N_u$ UAVs and $N_t$ tasks in total. So formula (1) represents the need to maximize the proceeds F. In formula (2), $c_{ij}$ represents the cost of assigning target j to UAV i, and this formula aims to minimize the cost of UAV formation H. Formula (3) manifests that UAV i can only perform $L_t^i$ tasks at most, and formula (4) means the number n of UAVs assigned to task j, $n\in \left[L_j^{min},L_j^{max}\right]$. Finally, formula (5) ensures that all the tasks are assigned.

From the perspective of the relationship between UAVs and the ground control center, there are three typical formation architectures: centralized architecture, distributed architecture, and mixed architecture, as shown in Figure 3.

In the Multi-UAV formation target assignment problem, for small-scale formations, centralized algorithms have been widely studied and applied, which can ensure the global optimal solution. However, the decision is made at the ground control center, which puts higher demands on the computing power and communication stability of the ground station. The robustness of the centralized system is poor, once the ground station is attacked, the entire system will collapse.

For large-scale clusters, limited by the computing and communication capabilities of airborne equipment, distributed negotiation algorithms have attracted a widespread attention. In the distributed architecture, each UAV is equal, and the consensus is reached through information exchange. This architecture puts more emphasis on the autonomy of UAV and the scale of the formation is more flexible. But frequent information exchange is required.

Some researchers researched on hybrid consensus algorithms, adopted the idea of hierarchical control, combined centralized and distributed, and obtained a favorable effect. This mixed architecture combines the advantages of the above two structures, and is suitable for cooperation among multiple clusters, which is the main development direction in the future.

As shown in Figure 4, task assignments can be broadly categorized into coordinated, distributed, stochastic, deterministic, evolutionary, and multi-fusion-based algorithms. In this section, different task assignment algorithms are extensively investigated, comparing their key features and characteristics.

2.1. Target Assignment Based on Mathematical Programming Algorithm

The mathematical programming method is also called Exact Algorithm (EA). Commonly used methods include the exhaustive method, Branch-and-Bound (B&B) algorithm, satisficing decision, Hungarian algorithm [17], and so on.

The basic idea of the exhaustive method is to verify all possible solutions one by one. It is a violent algorithm, also known as the enumeration method. But when the scale of the problem is enlarged, the time spent on solving the problem is unacceptable [18]. Therefore, EA is only suitable for solving small-scale problems and problems that do not require high real-time performance.

The B&B method is a common method for solving integer linear programming problems. A B&B algorithm consists of a systematic enumeration of candidate solutions by means of state space search: the set of candidate solutions is considered as a rooted tree. The algorithm explores the branches of this tree, which represent subsets of the solution set. Before enumerating a branch, it is checked against the upper and lower bounds and is discarded if it cannot produce a better solution.

Martin et al. proposed the formulation of a single-task robot, single-robot task, time-extended assignment, multi-robot task allocation problem with multiple, nonlinear criteria, and obtain an allocation by the B&B algorithm in low-scale problems [19]. The B&B algorithm has been proven to be able to obtain the optimal allocation, and it has a faster calculation speed in the size of the problem does not exceed a certain limit.

Ye et al. proposed a Multi-UAV target allocation method based on satisficing decision, and apply satisficing decision theory to solve the multi-target allocation problem of Multi-UAV cooperation [20]. Mills-Tettey et al. presented the dynamic Hungarian algorithm for the assignment problem with changing costs. The goal is to efficiently recalculate an optimal assignment when changes in the edge costs occur. And the result is a provably optimal solution [17].

EA is a classic method for centralized target assignment, which has the characteristics of simple structure and ensuring global optimality. However, due to the slow computing speed and the explosive growth of computation and storage requirements, it is not suitable for large-scale problems.

2.2. Target Assignment Based on Heuristic Algorithm (HA)

For small-scale formations, the EA above can obtain the global optimal solution, but as the complexity of the problem increases, the amount of calculation is also increasing explosively, and the traditional EA is no longer applicable. Instead of solving it exactly, the HA no longer blindly searches the global space but combines random operations with heuristics to seek a compromise between computing time and optimal performance, while taking into account both computing efficiency and performance [21].

Many biological groups in nature, especially social animals, eventually exhibit group intelligent behavior through simple interactions. Thus, Swarm Intelligence (SI) is defined as the collective behavior of a decentralized or self-organized system [22]. Typical SI algorithms include ant colony algorithm, Particle Swarm Optimization (PSO) [23], gray wolf algorithm [24], genetic algorithm [25] and so on. After years of development, SI algorithms have been widely used in various optimization fields such as path planning and target assignment.

Zhen et al. presented an intelligent cooperative mission planning scheme for UAV swarm, to search and attack the time-sensitive moving targets in uncertain dynamic environments, by a hybrid artificial potential field and ant colony optimization method [26]. Geng et al. focused on the problem of robot rescue task allocation, and a modified PSO algorithm is proposed, which is characterized by a flexible assignment decoding scheme to avoid the generation of unfeasible assignments [27]. Li et al. proposed a distributed cooperative task allocation strategy based on the algorithm of the improved gray wolf algorithm to quickly and effectively plan the cooperative task path with a large number of working task points [28]. Ma et al. proposed a coordinated algorithm combing the genetic algorithm and cluster algorithm to solve the problem of task assignment and path planning. The coordinated algorithm can be used to determine the required amount of UAVs and find the optimal path for every UAV [29].

In the multi-objective optimization solution, it is necessary to take into account each sub-objective at the same time, but the sub-objectives may conflict with each other, so the optimal solution is usually not unique but forms a non-inferior solution set, also known as Pareto optimality. For complex multi-objective optimization problems, multi-objective heuristic optimization algorithms are usually adopted [30,31].

Zhang et al. proposed a clone selection algorithm-based approach to simultaneously optimize four objectives in Multi-UAV task allocation, i.e., maximizing the number of successfully allocated tasks, maximizing the benefits, minimizing resource costs, and minimizing time costs [32]. To overcome the two fundamental drawbacks: limited flight duration and limited loading capacity, an approximate two-phase approach was developed to find approximate Pareto solutions, which is more computationally inexpensive [33].

However, the application of multi-objective optimization based on the Pareto theory in high-dimensional space also has limitations, mainly because the Pareto dominant solution does not work and the diversity maintenance mechanism is far away from the Pareto front in the sparse area [34]. To achieve multi-objective optimization on task assignments, the deep Q-learning is introduced to simplify the prioritization of the original task set, and the modified shift-based density estimation method is proposed to improve the conventional Pareto algorithm [35].

To sum up, the target assignment based on HA is to search for the optimal solution by means of group rolling optimization, but the algorithm is prone to precociousness. At the same time, large computation increases the search time, so it is not suitable for environments with strong real-time performance and large environmental uncertainties. That is why HAs are mostly used in static target assignment problems.

2.3. Target Assignment Based on Negotiation Algorithm

Both the EA and HA belong to centralized algorithms. The central node is responsible for information collection, processing, and task delivery. Higher requirements are placed on the computing power, reliability, and stability of data interaction. Due to the over-reliance on the central node, the robustness of the formation is poor. With the development of intelligence and the expansion of formation scale, distributed control capability is an inevitable requirement. Compared with the centralized one, the distributed algorithm distributes the amount of calculation to each UAV, and the UAVs are equal, able to perceive and make decisions by themselves, and realize situation communication, task coordination, and assignment through the inter-machine data link. Common negotiation algorithms include auction algorithm and Contract Net Protocol (CNP).

The auction algorithm simulates human auction activities to auction, bid, and allocate targets. De et al. addressed both the decentralized automated guided vehicles control and the resource management by proposing a decentralized task allocation algorithm based on sequential single-item auctions, taking into account resource constraints [36].

Choi et al. proposed the Consensus-Based Auction Algorithm (CBAA) and its generalization to the multi-assignment problem, i.e., the Consensus-Based Bundle Algorithm (CBBA) [37]. Hunt extended CBBA to the Consensus-Based Grouping Algorithm (CBGA), by introducing the collective decision-making behavior of social animals [38,39]. Bertuccelli et al. extended the CBBA to solve the problem of heterogeneous vehicle target assignment and took into account the obstacle region in order to generate a collision-free path while reducing the sensitivity of the mission planner to the sensor measurement noise [40].

The CNP is a tasking-sharing protocol in multi-agents, proposed by Smith [41], which consists of multiple nodes that can exchange information with each other. The core idea of the CNP is to realize the assignment, adjustment, and migration of tasks by imitating the “bid mechanism” in the real market transaction process, which belongs to a negotiation-oriented task decision-making and negotiation method.

Zhen et al. proposed an improved CNP scheme to deal with heterogeneous overloading and time sequences problem. Situation-based advantage value is adopted as a principle, and two improved CNP algorithms are designed for the one-to-many and many-to-one modes [42].

Compared with traditional methods, CNP has more advantages in dealing with large-scale task allocation problems due to its strong robustness and dynamism. The fly in the ointment is that to achieve the best performance, frequent data communication between UAVs is required, which will undoubtedly increase the risk of exposure for clusters performing tasks in enemy areas. Therefore, how to reduce the frequency and the data of communications have become a frontier issue in distributed task planning. Xie et al. proposed a distributed Multi-UAV task allocation method based on improved CNP to solve the local cooperative task allocation problem of heterogeneous Multi-UAV in the communication-constrained environment. The adaptive maximum number setting method of information transfer times and the information consistency method are proposed to solve conflicts in the local communication network. And Gini coefficient based resource consumption allocation algorithm is designed to keep the resource difference within a reasonable range, in the process of coalition formation. However, there are still many efforts to be made to further improve negotiation efficiency [43].

2.4. Task Assignment Based on Neural Networks (NN)

Facing the complex and changeable battlefield situation, highly dynamic attack and defense put forward extremely high requirements on the optimality and real-time performance of decision-making, but the above algorithms are insufficient in terms of computational efficiency and global optimality. With the development of artificial intelligence technology, NN has been widely studied in the fields of image recognition, intelligent control, etc. By virtue of its powerful exploration and optimization capabilities, Reinforcement Learning (RL) is considered to be the most likely means to achieve “general intelligence”. RL is based on dynamic programming theory and inspired by behaviorist psychology. By balancing “exploration” and “exploitation”, it learns optimal strategies by receiving environmental rewards [44]. In Go and video games, complex RL has reached or exceeded human performance. RL has also been applied to target assignment and intelligent control [45].

Gao et al. used deep RL for task scheduling and trajectory planning in collaborative perception detection [46]. Luo proposed a data-driven policy optimization with deep RL for the adversarial missile-target assignment by designing a comprehensive reward function. The learned policy can implicitly model the penetration of missiles under an adversarial environment in a data-driven way [47]. Zhao et al. proposed a Q-learning based fast task allocation algorithm by developing a Q network that encodes the allocation rules. The proposed algorithm is computationally efficient by offloading the online computation to an offline learning procedure and is able to handle the effect of environment uncertainty [48]. Aiming at the coordinated dynamic task allocation in the presence of environment uncertainty, Liu et al. proposed the multi-agent RL-based coordinated network to send requests and a Q network to determine whether to participate in the dynamic task [49].

With the development of multi-agent problems, the combination of RL and Pareto to solve multi-objective optimization problems has also been studied. Traditional deep RL methods train neural networks to maximize the single cumulative reward. As an improvement, Multi-Objective Deep Reinforcement Learning (MODRL) algorithms simultaneously train multiple policies using different reward functions, by sharing environmental information across all policies. Sullivan et al. designed a MODRL algorithm based on Proximal Policy Optimization (PPO) in a shared environment with a multi-objective solution space. Policies using different reward functions may also learn from each other, resulting in a more stable and efficient training phase [50].

However, the difference between the simulation environment and the real scene puts forward higher requirements for generalization ability and robustness. And how to reconcile the performance-oriented philosophy of RL with the stability focus of application is still an open question [51].

3. Cooperative Path Planning (CPP) for Multi-UAV

Multi-UAV is coordinated by coordination algorithms with each UAV under the control of the local guidance law. The coordination variable is the bond that links all UAVs together. Generally speaking, aiming to introduce coordination strategies into the path planning of each UAV, CPP is an important way to realize formation [12].

CPP is by no means a simple superposition of single path planning, but a process of mutual restraint and coordination. For example, it needs to consider: time constraints in simultaneous arrival or sequential arrival; space constraints in formation flight or all-around attack, and space-time constraints in time-space coupling problems [52]. CPP algorithms also require the ability of online re-planning. During the mission execution, when the scene or mission changes, the UAVs need to re-plan the path according to their current states. The key to CPP is to deal with the relationship between time, space and tasks among multiple trajectories.

For the cooperative path planning of Multi-UAV, researchers have conducted sufficient research from both mathematical methods, graph-based path planning, and artificial intelligence algorithms. This section introduces CPP algorithms from the perspective of task requirements. Due to the diversity of tasks, CPP problems can be generally divided into three categories: 2 Dimension (2D) coverage, 3 Dimension (3D) cooperation, and 4 Dimension (4D) space-time cooperation [53,54].

(1)

2D coverage: It means a UAV group conducts a blanket search or surveillance of an area. In such missions, UAV’s power consumption and the coverage area are limited. Multi-UAV can complete tasks through the division of labor.

(2)

3D cooperation: Paths are planned in three-dimensional space, usually with time or line-of-sight angle as the coordinating variable, to achieve simultaneous or sequential strikes. Commonly used methods to coordinate time are: adjusting flight speed and adjusting the length of the flight path.

(3)

4D space-time cooperation: Different from the above-mentioned 3D cooperation, 4D space-time cooperation plans the position of UAVs at each moment, which can more accurately adjust the arrival time and achieve higher space utilization. However, due to the increased time degree of freedom, the scale and the difficulty of problem-solving are higher.

3.1. Path Planning in 2D Coverage Tasks

In area monitoring and surveillance, Multi-UAV cooperate with each other, subdivide the detection area considering their state and capabilities, and determine the optimal coverage path [55,56], as shown in Figure 5.

In this kind of task, the initial location and sensors of each UAV may be different, but they all need to plan a path with minimum energy consumption, number of turns and completion time under the premise of increasing coverage [57]. Ghaddar et al. proposed an energy-aware path planning algorithm to cover an area by grid-mapping, grid-subdivision, and area partitioning, which can be applied in scenarios where a single and multiple UAVs are used [58]. Xin et al. proposed a general multi-robot task model, named multi-point dynamic aggregation, to formulate the motion planning problem in cooperative multi-area coverage. And a rule-based heuristic for the distributed motion planning of each single robot is adopted [59].

Although scholars have done in-depth research, there are still some problems to be solved. In CPP problems, There are some constraints that need to be considered [53].

(1)

Maneuverability constraints: When performing path planning, the UAV’s maneuverability, such as the turning radius, is usually not considered. This will affect the final coverage. Furthermore, the heterogeneity of UAVs, such as different mobility and sensing capability should also be taken seriously.

(2)

Environmental constraints: When designing the algorithm, not only static no-fly zones, but also dynamic obstacles in the environment should be considered to avoid potential danger.

(3)

Task constraints: Given the complexity of tasks, some task constraints also need to be considered, such as certain areas need to be scouted from a specified angle, etc.

3.2. Path Planning in 3D Cooperation

Space coordination can significantly disperse air defense firepower, avoid collisions within formations, and improve the strike effect, which has high application value. Figure 6 shows a typical scene of Multi-UAV 3D space collaboration.

In this typical scenario, the UAVs bypass the no-fly zones and interception system according to the assigned mission, and strike or monitor the target simultaneously or sequentially from different directions. The flight path needs to meet the scene constraints, the UAV’s maneuverability constraints, and performance indicators.

In 3D coordinated path planning, there are usually two coordination variables: arrival time and line-of-sight angle.

3.2.1. Time-Coordinated Path Planning

When arrival time is adopted as the coordination variable, it is called time-coordinated path planning. Time coordination means that Multi-UAV coordinates their distance and speed relative to the target through communication, so that the time of arrival tends to be consistent. In 2006, Jeon et al. proposed the time-coordinated guidance law for the first time when they studied the coordinated attack of anti-ship missiles [60]. The simultaneous arrival of the missiles can bring greater pressure to the air defense system, thereby improving the ability of coordinated strike penetration.

When adjusting the speed as a method, it is required that the speed of the UAV can be changed within a certain range. The acceleration on the line-of-sight direction is adjusted to make Multi-UAV achieve the arrival time consensus. Li et al. designed an adaptive fixed-time cooperative guidance law to intercept the maneuvering target for simultaneous arrival and the convergence analysis indicates that the consensus errors converge to the origin in the fixed-time interval [61]. For multiple missiles to simultaneously attack a maneuvering target, Dong et al. designed a fixed-time cooperative guidance law. The tangential acceleration is calculated from a distributed cooperative protocol to synchronize the time-to-go of multi-missiles within a fixed time [62].

For the problem of salvo attack with acceleration limitation, the consistency of arrival time can be achieved by adjusting the length of the flight path. Kang et al. proposed a Model Predictive Control (MPC)-based cooperative guidance law to perform a salvo attack against a stationary target, which guarantees that multi-missiles hit the target simultaneously without explicitly using the time-to-go or its estimate [63]. A modified cooperative guidance law is presented to avoid singularity existing in the guidance law proposed by Jeon. A sign function is adopted to mitigate the effect of the small leading angle, and the convergence of times-to-go is guaranteed by the Lyapunov stability analysis [64]. The problem of cooperative path planning where UAVs should arrive at the target simultaneously or sequentially is also addressed by adjusting the Dubins curve, while minimizing the total mission time. Sequential arrival is realized by sorting the flight paths and prolonging them if necessary to accomplish the desired time delays [65]. The application of artificial intelligence algorithms directly maps the situation information into cooperative guidance instructions, giving the UAV the ability to perceive and replan in real time. Luo et al. studied the problem of Multi-UAV cooperative penetration and designed a penetration strategy based on the deep deterministic policy gradient, which enables Mutli-UAV to break through the interception and reach the target area simultaneously [66]. Yu et al. investigated the cooperative guidance strategy for multiple hypersonic gliding vehicles. Under the premise of ensuring the coordination of the attack angles, the angle of attack curve is optimized to achieve different jump maneuvers, and then the arrival time is coordinated [67]. Song et al. proposed a time cooperative guidance law based on a Long Short-Term Memory (LSTM) network and a modified artificial potential field for multi-hypersonic vehicles. The novel time potential field is developed to adjust the flight time, according to the time-to-go predicted by LSTM [68].

However, most researches focus on the static or low-speed target, the impact time is set before the mission rather than coordinated during the mission execution [69]. In addition, some complex constraints need also to be considered, such as heat flow, overload, and dynamic pressure for hypersonic vehicles. And how to estimate the time-to-go quickly and accurately is also the key to the time cooperative guidance law [68]. Furthermore, in the actual battlefield, the complex weather, and electromagnetic environment require UAVs to have higher reliability, and the fault information will also be propagated in the UAV formation [70,71]. Therefore, how to isolate the fault information, manage the departure and entry of the faulty aircraft, and develop a robust cooperative guidance law has also attracted much attention [72,73].

3.2.2. Space-Coordinated Path Planning

When the line-of-sight angle is adopted as coordination variable, it is called space-coordinated path planning. Space coordination refers to the coordination of Multi-UAV through communication to obtain the desired line-of-sight angle, and then fly to the target from different directions. Based on a large number of randomly generated direction lines, Babel et al. proposed a path planning algorithm that allows arbitrary flight directions arbitrary turn angles. The method can also be applied for quick in-flight replanning of flight paths during the mission [74].

For cooperative attack, time coordination refers to the coordination of multiple aircraft in the line of sight direction, while space coordination refers to the coordination in the direction of the normal line of sight. In the line-of-sight direction, the time convergence of the coordinated attack is ensured by introducing the remaining time. On the normal line-of-sight, the acceleration of the aircraft is controlled to ensure that the attack angle converges to the desired angle to achieve spatial coordination.

Usually, space-coordinated path planning is not used alone, but combined with time-coordinated path planning to carry out the 3D space-time coordinated guidance. Song et al. proposed a finite-time cooperative guidance law, which is continuous and requires no information on target maneuvers. The acceleration command on the line-of-sight direction was designed to guarantee that all missiles reach the target simultaneously, and the acceleration command in the normal direction was developed to ensure the finite-time convergence of the line-of-sight angular rate and angle [69]. To improve the robustness of the cooperative guidance system, an appointed-time extended state observer is developed by Zhang et al. for compensating the guidance commands [75]. Based on finite-time consensus theory and the super-twisting control algorithm, the cooperative guidance law in the direction of the line of sight is derived. Then angle cooperation is guaranteed by adopting finite-time sliding mode control and the super-twisting control algorithm [76].

Cooperative guidance law against maneuvering target is practical but difficult to design [77]. To estimate and compensate for the unknown target acceleration, Dong et al added a fixed-time observer in the guidance law [62]. Zhang et al. designed a fixed-time cooperative guidance law. In line-of-sight direction, a consensus protocol is designed based on the fixed-time differentiator and the bi-limit homogeneity theory. In normal direction, two continuous adaptive fixed-time guidance laws are designed to guarantee line-of-sight angular rates achieve convergence [78]. Teng et al. designed a distributed finite-time cooperative guidance law in the line-of-sight direction to guarantee the consensus of impact times. In the vertical direction, a nonlinear disturbance observer is adopted to estimate the target’s acceleration in finite time [79]. Chen et al. studied the problem of cooperative interception of strong maneuvering targets by multiple missiles with weaker maneuverability in three-dimensional space, proposed a guidance law based on the cooperative coverage strategy, and proved that the cooperative interception of targets can be achieved under the acceleration limit. The relations among the number of missiles, the initial array position of terminal guidance, and the coverage area of the target’s large maneuver were also analyzed [80].

However, there is a certain similarity in the flight paths planned by the cooperative guidance law, which will increase the probability of collisions between UAV formations. Therefore, the path conflict avoidance problem needs to be considered in the design of the space-time cooperative guidance law. Liu et al. designed a generalized conflict graph that encodes the traveling time and possible path conflicts. At the same time, for the real-time requirements of large-scale system computing, a greedy algorithm is proposed. The solution has approximately optimal performance, but can greatly reduce the computational complexity [81]. But this problem becomes more prominent in the crowded environment, and how to use airspace more efficiently will also become a future research direction.

3.3. Path Planning in 4D Space-Time Cooperation

With the development of unmanned technology, the tasks performed by UAV clusters are becoming more and more complex, and the number of drones in the formation is also increasing. In complex missions, the area where the formation is located maybe full of obstacles. UAVs should fly through the desired waypoints without any collision as well as complete the assigned tasks [82]. However, for high-speed moving UAVs, avoiding obstacles and other UAVs requires strong environmental awareness. In recent years, scholars consider the problem of collision avoidance when planning trajectories, and four-dimensional coordinated path planning algorithms are proposed, where time variable is taken into account. However, due to the increased time degree of freedom, the difficulty of solving the problem is higher than that of 3D path planning.

Liu et al. designed a spatial refined voting mechanism for PSO in the four-dimensional CPP problem and the objective function is designed by considering not only the obstacle and threat area, but also the arrival time and other constraints [83]. Although the current air traffic management architectures are widely used in commercial aviation, they have strictly constrained configuration options that limit their ability to manage the traffic demand. Vitale et al. developed a novel framework to optimize 4D trajectories for UAVs, while maximizing the available capacity. A network manager is designed to consider UAV requests and a linear-Gaussian system is adopted to address mobility uncertainties [84]. Dai et al. proposed a conflict-free A* algorithm based on the first-come-first-served scheme. A novel design of heuristic function a conflict detection and resolution strategy is contained in the algorithm [85].

Integrating collision avoidance requirements into the CPP problem forms a 4D trajectory planning problem, which can achieve more precise space-time control and improve airspace utilization. However, how to solve this complex optimization problem to meet real-time requirements remains to be further studied.

4. The Key Issues

At present, the target allocation and collaborative trajectory planning in the Multi-UAV formation have achieved phased outcomes and have obtained a certain degree of application. UAV swarms such as “Gremlins” and “Coyote” in the United States, Russia’s “Thunder” and “Lightning”, and ”Mosquito” in the United Kingdom have initially possessed the ability to coordinate reconnaissance and strike.

Although the target allocation and cooperative guidance of Multi-UAVs have achieved fruitful results, there are still problems in simple and ideal research scenarios and simplified constraints and there are many challenges in practical applications, which are summarized in

Table 1. Issues to be considered in practical applications.

	Task Assignment	Path Planning
Constraints	number and capabilities of UAVs characteristics of tasks time window and precedence	scene constraints (no-fly zones etc.) maneuverability of UAVs
Evaluation metrics	the proceeds received or the value of destroyed targets fuel or time consumption loss of UAVs	degree of task completion safety of UAVs time or energy efficiency
Challenges	high time complexity task re-assignment coupling among tasks uncertainty of environment	high time complexity collision avoidance environmental uncertainty internal or external distractions

.

4.1. Online Target Assignment in Dynamic Environment

Most of the existing research on target assignment focuses on off-line task planning based on known situation information before task execution. However, in the face of complex and changeable battlefield environments, it is more practical to assign online targets based on real-time reconnaissance information. Some scholars have carried out work on the problem of target assignment in dynamic environments. Yang et al. studied the problem of distributed UAVs task reassignment in dynamic environments, and proposed a distributed method to reduce the burden of computing and communication [86]. How to improve the solution efficiency and stability of complex optimization problems still needs further research.

At the same time, the high dynamics and uncertainty of the environment will also affect the target allocation, which requires higher robustness and the capability of reassignment of the mission planner. At this point, target allocation and trajectory planning are no longer decoupled, and the coupling between them must be considered.

In addition, in the human-machine collaborative environment, not only the external environment will affect the overall situation, but also human decision-making will affect the situational awareness of the unmanned formation. Mahadevan et al. researched on the cooperative stacking problem between manipulators and humans. The manipulator can perform online target assignments according to human actions, to achieve the purpose of human-machine collaborative work [87]. However, the human-in-the-loop mission planning problem has not been thoroughly studied. At the same time, the difference in perception and decision-making ability between humans and machines also makes it imperative to improve the technology of UAV sensing and decision-making [88].

4.2. Coupling among Missions

Most of the researches on target assignment at this stage assumes that each task is independent of the other, and whether a certain task is completed or not has no effect on the execution of other tasks. However, different tasks are often given different priorities due to their urgency and importance [89]. In addition, certain tasks need to be executed in strict order, such as reconnaissance-strike-assessment, etc. In this case, the UAVs are also required to meet the sequential constraints. Ye et al. developed an extended CBBA to solve the multi-task assignment problem with task coupling constraints in the heterogeneous Multi-UAV system [90]. The task coupling constraints usually include the following aspects:

(1)

Order constraints: In some scenarios, tasks have different priorities and need to be executed simultaneously or sequentially.

(2)

Time window constraints: In practical application, some missions need to be completed within a certain time window.

4.3. Collision Avoidance between UAVs

Generating collision-free trajectories is a necessary condition for the Multi-UAV to perform missions safely. However, this will become challenging when a large number of high-speed maneuvering aircraft are moving in a crowded space [91]. In previous studies, most of the collision avoidance algorithms were placed on the logic of the formation flight control algorithm, but this will inevitably affect the accuracy of path tracking. Therefore, the direct generation of collision-free paths has also attracted the attention of scholars.

Qing et al. proposed offline and online solutions to deal with a collision-free trajectory generation problem for UAV swarm formation rendezvous. First, a geometric path based on improved ant colony optimization is found. Then incorporate the zeroing control barrier function constraints and the previous trajectories into the optimal problem to meet safety-critical requirement [92].

However, the optimization algorithm cannot guarantee global explicit feasibility, and there may still be singularities that cause the system to fall into a deadlock. Therefore, some researchers have developed a rule-based adaptive deadlock solution to ensure global feasibility [93]. 4D path planning algorithms are also getting more and more attention. However, facing the increased time dimension, how to efficiently and quickly solve the problem of path conflicts among multiple UAVs remains to be further studied.

4.4. Strong Robust Cooperative Guidance Law

Most of the existing studies on the cooperative guidance of Multi-UAV focus on the ideal situation, without considering the reliability of the cooperative guidance law. Communication delays, measurement noise, and aircraft failures all place high demands on the robustness of the guidance law. If some aircraft in the formation is intercepted or damaged, the entire cooperative mission will be affected. Hu et al. aimed to develop a fault-tolerant cooperation framework for networked UAVs. A cooperative navigation strategy based on network graph theory was proposed to coordinate all the connected UAVs in the swarm, and in the case of damage to the actuators of some UAVs during the mission, a decentralized task reassignment algorithm is then applied [94].

However, only actuator faults are considered, many other external disturbances are not concerned. Therefore, it is necessary to carry out research on a robust cooperative guidance law, and cooperate with the management mechanism of departure and entry to achieve reliable mission coordination [95].

5. Conclusions

UAVs play an important role in various industries and cooperation between Multi-UAV has become a research hotspot. This paper reviews the target assignment and path planning of Multi-UAV. Investigated and analyzed the research status, and summarized the existing research results and the challenges faced. The computation time grows exponentially with the problem size. Awareness of the dynamic environment affects the optimality of decision-making. The coupling between target allocation and trajectory planning compounds the problem. And in crowded environments, unmanned swarms are prone to collisions. All these challenges limit the practical application of Multi-UAV. Finally, this paper makes a prospect for the future. In the face of Multi-UAV mission planning, researchers should focus on the problem of target allocation in dynamic uncertain environments, coupling among missions, collision avoidance between UAVs, and the design of robust cooperative path planning.