Interference Avoidance through Periodic UAV Scheduling in RIS-Aided UAV Cluster Communications

Abstract

This article investigates the transmission of downlink control signals for multiple unmanned aerial vehicle (UAV) clusters in collaborative search and rescue operations in mountainous environments. In this scenario, a reconfigurable intelligent surface (RIS) mounted on the UAV is utilized to overcome obstacles between the ground base station (BS) and UAVs. By leveraging the fixed channel of the RIS to the BS, the line-of-sight (LoS) path characteristics of the air-to-air channel, and the position information of the UAV, the RIS forms a directional beam by adjusting the RIS coefficient, which points towards UAVs in the cluster. To ensure low delay in control signaling and UAV state transmission, we adopt semi-persistent scheduling (SPS), which allocates pre-specified periodic intervals to each UAV for the formation of corresponding RIS coefficients. The allocation of time slots is constrained by the transmission intervals required by different UAVs and the number of RISs available. We propose a time slot scheduling scheme for UAVs to reduce inter-cluster interference caused by RIS beams. The time slot allocation problem is formulated as a combinatorial optimization problem. To solve this problem, we first propose an intuitive greedy scheme called local interference minimization (LIM). Building upon the LIM scheme, we propose a rollout-based algorithm called rollout interference minimization (RIM). Through simulation, we compare the LIM and RIM schemes with the benchmark scheduling scheme. The results demonstrate that our proposed scheme significantly reduces interference between UAV clusters while satisfying the conditions of periodic transmission and RIS quantity constraints.

1. Introduction

Unmanned aerial vehicles (UAVs) have been widely utilized in various industries due to their flexibility and rapid deployment capabilities. Consequently, the wireless communication of UAVs has garnered significant attention from researchers [1]. In this article, we address the issue of control signaling transmission for UAV clusters employed in mountain search and rescue operations. In this system, a ground base station (BS) is responsible for transmitting downlink signaling to control multiple UAVs, thereby facilitating the formation of a cluster to carry out missions. However, in mountainous environments, the line of sight between the BS and the UAV can often be obstructed. To overcome this challenge, we propose the use of an RIS-UAV, which involves a UAV hovering over the BS and equipped with multiple reconfigurable intelligent surfaces (RIS) serving as relays. The RIS-UAV stores position-based RIS coefficients and generates signal beams directed towards different spatial positions. By utilizing position information acquired from each UAV, the BS can select the appropriate RIS coefficient and form a highly directional beam that covers each individual UAV.

The 3rd Generation Partnership Project (3GPP) Release 17 research project presents a class of communication demands specifically designed for machine communication within the industrial Internet of Things. These demands cater to applications requiring deterministic and periodic communication, with stringent requirements for low-transmission latency [2]. Deterministic periodic communication finds relevance in the transmission of control signals and feedback of UAV status information in UAV clusters. Within an RIS-aided UAV cluster system, the allocation of scheduling time slots for each UAV becomes a critical consideration. On one hand, an RIS can only generate a single group of coefficients within a time slot, and different UAVs necessitate distinct communication periodicity. As a result, the scheduling of time slots must concurrently cater to the communication resource requirements of all users. On the other hand, the number of UAVs that can be scheduled simultaneously in each time slot is limited by the number of RIS. Additionally, the beam formed by the RIS coefficient may potentially lead to interference with UAVs in adjacent clusters. Consequently, the scheduling of UAVs should strive to mitigate interference between adjacent UAV clusters.

1.1. Prior Works

UAVs are becoming increasingly prominent in various fields, including environmental safety and social governance. For instance, Zhang et al. [3] explored the utilization of swarm UAVs for target search in unknown environments and proposed a collaborative method. Consequently, wireless communication in UAV systems has gained significant attention as a fundamental aspect of UAV applications [4]. Unlike conventional cellular systems, wireless communication in cellular-connected UAV swarms presents unique challenges due to the characteristics of machine communication and the specific network topology [5,6]. A fundamental concern in cellular-connected UAVs is ensuring effective information transmission between UAVs and ground base stations. In [7], the authors designed UAV paths based on a radio map to maintain connectivity between the base stations and UAVs. In [8], the authors proposed a three-dimensional system model that incorporates UAVs and employs an analytical framework to assess system coverage. In [9], the joint optimization of UAV operation time, communication scheduling, trajectory, and transmission power was investigated to maximize data upload throughput while adhering to quality of service (QoS) requirements and energy constraints. Furthermore, the authors in [10] examined the application of large-scale multiple-input multiple-output (MIMO) and UAV communication, obtaining a lower bound for UAV capacity, and deriving optimal antenna spacing and UAV distribution strategies that maximize this capacity. In [11], the authors analyzed the coverage probability of UAV-assisted small base stations for clustered users while considering millimeter wave and directional beamforming based on MIMO. Furthermore, the authors in [12] analyzed the coverage probability of UAV-assisted cellular networks with incomplete beam alignment using the random geometry theory.

In environments with blocking, the mounting of RIS on UAVs presents a promising solution [13,14]. Compared to ground-based RIS, UAV-mounted RIS offers line-of-sight (LoS) paths to UAVs, leading to improved performance of UAV swarms. For example, in [15], a hovering UAV equipped with RIS is designed, with optimization conducted for RIS association, hovering altitude, and RIS phase shifting. In [16], the authors focus on the weight design of the RIS to facilitate communication between the base station (BS) and suburban users, aiming to reduce the number of RIS elements. Additionally, the authors in [17] investigate the optimal UAV altitude and element quantity for the UAV-RIS system. Mu et al. [18] analyze RIS deployment under different multiple access schemes and jointly optimize the coefficient of the RIS and power allocation at access points.

Eliminating interference is also a crucial challenge in cellular UAVs. Efforts have been made to alleviate interference between UAV communications and ground users communications [19], for example, the interference between UAVs and ground users are reduced through resource allocation and interference cancellation designs [20]. Liu et al. propose using multiple antennas on the UAV to improve the transmission rate and avoid interference to ground users [21]. The widely employed beamforming design, relying on channel state information (CSI), effectively reduces inter-cell interference and maximizes the sum rate [22]. When accurate channel state information (CSI) is not shared between cells, optimizing beam scheduling proves to be an efficient method for reducing inter-cell interference. Sha et al. [23] minimize the number of inter-cell beam collisions for two cells by optimizing the scheduling order of beams. Additionally, they utilize graph theory to optimize beam scheduling and avoid interference in multiple cells [24]. Although the communication interference among UAV clusters is similar to that among ground cells, the UAV’s high mobility characteristics pose difficulties in utilizing complete CSI for interference cancellation. As a result, resource scheduling optimization emerges as an effective approach to eliminate inter-cluster interference.

In a conventional UAV swarm system, the central access point periodically transmits information to UAVs to achieve various objectives, such as collision avoidance and mission planning [25]. To accomplish this in UAV swarms, it is envisioned to adopt a semi-persistent scheduling approach with periodic allocation of time slots, similar to the 3GPP standard of cellular vehicle-to-everything (C-V2X) sidelink Mode 4 [26]. Since UAVs may vary in type and mission, the required scheduling time intervals for UAVs could differ accordingly. In such cases, when aiming to reduce interference between UAV clusters through scheduling, the UAV scheduling scheme must adhere to specific periodic constraints.

We have compiled a summary of the relevant literature in

Table 1. Summary of related literature.

Existing Works	Cellular-Connected UAV	UAV-Mounted RIS	Interference Avoidance	Scheduling Optimization	Periodic Scheduling
[9]	✓			✓
[13,14,15]		✓
[16,17,18]		✓		✓
[19]	✓		✓	✓
[20,21]	✓		✓
[22,23,24]			✓	✓
This Paper	✓	✓	✓	✓	✓

, presenting the various research themes and scenarios investigated in each study, while emphasizing the differences from our research. Compared to the existing literature, this article explores the utilization of RIS on UAVs to enhance cellular-connected UAV communication in scenarios involving multiple UAV clusters, while taking into account periodic transmission constraints.

1.2. Motivation and Contributions

This paper introduces a novel system involving multiple UAV clusters assisted by RIS, where the UAVs are scheduled periodically to receive signals from the RIS mounted on a UAV. Our focus is on mitigating interference among the UAV clusters by optimizing the allocation of UAV scheduling time slots while taking into account different UAV scheduling cycles and the constraint of limited RIS arrays. The contributions of this article are outlined below.

(1)

We present a UAV cluster system that leverages RIS mounted on UAVs to establish LoS channels between BS and UAVs, and enhance wireless connectivity in obstructed environments. Additionally, we propose designing the RIS coefficients based on the position information of UAVs to minimize the channel estimation overhead.

(2)

This study demonstrates that by designing RIS with appropriate coefficients, it is possible to estimate the interference between UAV clusters based on the positions of the UAVs. We introduce a UAV interference matrix to describe the interference relationship between UAV clusters. In the context of periodical scheduling, we derive the degree of interference between UAV clusters by analyzing the scheduling period and initial scheduling time slot of each UAV. Subsequently, we formulate the UAV scheduling problem as an integer optimization problem.

(3)

The time slot scheduling problem for UAVs is approached as a multi-stage decision problem, which can be equivalently modeled as an optimal path search problem through the establishment of a UAV scheduling graph. To tackle this issue, we introduce a novel local interference minimization (LIM) scheme that aims to minimize interference arising from newly scheduled UAVs. Additionally, we propose a rollout interference minimization (RIM) algorithm that builds upon the LIM scheme by utilizing its calculation results to determine optimal decisions for subsequent nodes.

(4)

The performance of the proposed LIM and RIM schemes is evaluated through simulations and compared to that of the basic sequential scheduling scheme. The results show that the proposed schemes effectively reduce the interference level between UAV clusters while satisfying the constraints of given scheduling periods and a limited number of RISs.

1.3. Organization

The rest of the paper is organized as follows. Section 2 presents the RIS-aided UAV communication system and formulates the problem. In Section 3, the problem is reformulated as a path-finding problem by analyzing the inter-cluster interference. The LIM and RIM schemes are then proposed. Section 4 shows the simulation results, and the final section concludes the paper.

2. System Models and Problem Formulations

The UAV clusters are depicted in Figure 1, where the red arrows represent the inter-cluster interference. We focus on the downlink signal transmission, where the RIS equipped with RIS-UAV reflects the signal from the BS to the UAVs within the cluster. The RIS-UAV remains fixed in position relative to the BS and is equipped with L RIS arrays. Estimating the channel of the RIS is often a complex task due to the large number of RIS elements [27], leading to significant delays in UAV cluster transmission. To mitigate the overhead associated with channel estimation, we can leverage the positional information of the UAVs to simplify the computation of RIS coefficients.

Given the spatial positions of the RIS and a UAV, as ${\mathit{p}}_{\mathbf{1}}$ and ${\mathit{p}}_{\mathbf{2}}$, respectively, the direction vector can be written as ${\mathit{p}}_{\mathbf{2}}-{\mathit{p}}_{\mathbf{1}}=\left(x,y,z\right)$. The relative distance, zenith angle, and azimuth angle are provided as follows:

$$\begin{array}{cc}\hfill & z=∥{\mathit{p}}_{\mathbf{2}}-{\mathit{p}}_{\mathbf{1}}∥\hfill \\ \hfill & \varphi =arctan\left(\frac{y}{x}\right)\hfill \\ \hfill & \theta =arctan\left(\frac{z}{\sqrt{x^2+y^2}}\right).\hfill \end{array}$$

(1)

The horizontal and vertical angles of RIS are represented by ${\varphi }_R$ and ${\theta }_R$, which are determined by the orientation and pitch angles of the RIS-UAV. It is assumed that the channel between the RIS and the other UAV is the LoS channel, and the RIS array is a rectangle array. By denoting ${\varphi }^{\prime }={\varphi }_R+\varphi$, and ${\theta }^{\prime }={\theta }_R+\theta$, the channel can be modeled as [28]

$${\mathit{h}}_{RU}={\alpha }_{RU}\mathit{d}\left({\varphi }^{\prime }\right)\otimes \mathit{d}\left({\theta }^{\prime }\right),$$

(2)

where ${\alpha }_{RU}$ is the channel coefficient. $\mathit{d}\left({\varphi }^{\prime }\right)$ and $\mathit{d}\left({\theta }^{\prime }\right)$ are the steering vectors with the horizontal angle ${\varphi }^{\prime }$ and vertical angle ${\theta }^{\prime }$. These steering vectors can be approximated using discrete Fourier transform (DFT) weights. Assuming that both BS and UAV have a single antenna, and RIS consists of $N_R$ elements, the channel between the BS and RIS is denoted as ${\mathit{h}}_{BR}$. The received signal at the UAV can be expressed as

$$y_U=\mathrm{vec}\left\{{\alpha }_{RU}\mathit{d}\left({\varphi }^{\prime }\right)\otimes \mathit{d}\left({\theta }^{\prime }\right)\right\}{\mathit{c}}_R\odot {\mathit{h}}_{BR}\tilde{s}+\tilde{n},$$

(3)

where ⊗ is the Kronecker product, ⊙ is the element-wise product, $\tilde{s}$ is the transmitted symbol and $\tilde{n}$ is the Gaussian noise. vec{} denotes the vectorization operation, and ${\mathit{c}}_R$ are the coefficients of RIS. As the motion of the UAV causes rapid changes in ${\alpha }_{RU}$, while ${\mathit{h}}_{BR}$ experiences slower changes, it is possible to set the RIS coefficients to match the cascaded channel without considering the coefficient ${\alpha }_{RU}$. Therefore, the RIS coefficients should be chosen to satisfy the condition

$$\beta {\tilde{\mathit{c}}}_R\odot {\mathit{h}}_{BR}={\left(\mathrm{vec}\left\{\mathit{d}\left({\varphi }^{\prime }\right)\otimes \mathit{d}\left({\theta }^{\prime }\right)\right\}\right)}^H,$$

(4)

where $\beta$ is a normalization parameter to ensure ${∥{\tilde{\mathit{c}}}_R∥}^2=1$. The coefficients ${\tilde{\mathit{c}}}_R$ for all spacial positions are calculated based on ${\mathit{h}}_{BR}$ at the BS, and transmitted to the RIS-UAV. According to (4), the RIS coefficients enable the RIS to generate a directional beam that points toward a specific UAV. Thus, each UAV is associated with its corresponding beam. Consequently, in this system, the inter-cluster interference is predominantly influenced by the positions of the UAVs and the beam directions, owing to the LoS characteristics of the UAV channels [29]. Therefore, the interference received by a UAV can be estimated based on its location [30].

Interference among multiple UAV clusters can be intricate due to the dynamic positions of the UAVs. Building upon the concept of collision beams introduced in previous work [23], we propose the UAV interference matrix (UIM) to quantify the interference between UAVs belonging to different clusters. We assume there are $N_c$ UAV clusters, each comprising M UAVs to be scheduled within a single scheduling frame. We denote the UAV j in cluster i as UAV $\left(i,j\right)$, which is served by the beam $\left(i,j\right)$. Notably, beam $\left(i^{\prime },j^{\prime }\right)$ is directed to UAV $\left(i^{\prime },j^{\prime }\right)$, but it can cause interference with UAV $\left(i,j\right)$. We define the UAV interference matrix ${\mathit{C}}_{i,i^{\prime }}$ with dimensions of $M\times M$, to represent the interference received by cluster i originating from cluster $i^{\prime }$. The element $c_{i,i^{\prime }}\left(j,j^{\prime }\right)$ represents the interference level that UAV $\left(i,j\right)$ experiences from beam $\left(i^{\prime },j^{\prime }\right)$.

Interference alone is insufficient to determine the system’s performance. Therefore, we employ the estimated signal-to-interference ratio (SIR) to derive the UAV interference matrix (UIM). Given the unknown nature of the small-scale channel ${\alpha }_{RU}$, we utilize the transmission power and beamforming gain to estimate the SIR, and calculate the UIM elements. The beamforming gain function is determined based on factors such as the relative position of the RIS and UAVs, distance, beam width, and beam direction. The gain function is given by [31]

$$g\left(w_h,w_v,\Delta \theta ,\Delta \varphi \right)=\frac{\pi }{w_h}{e}^{\eta {\left(\frac{\Delta \theta }{w_h}\right)}^2}\frac{\pi }{w_v}{e}^{\eta {\left(\frac{\Delta \varphi }{w_v}\right)}^2},$$

(5)

where $\eta$ is $-4log2$, $w_h$ and $w_v$ are the horizontal and vertical beam widths, respectively. $\Delta \theta$ is the angular difference between the beam direction and the receiver. The variables $z_{i,j}$, ${\varphi }_{i,j}$, and ${\theta }_{i,j}$ are used to represent the distance, relative horizontal angle, and relative vertical angle between UAV j and RIS-UAV i. It is assumed that the beam is perfectly aligned with the UAV without deviations caused by coefficient quantification. The angular deviation between UAV j and the interference beam is calculated as $\Delta \theta =\left|{\theta }_{i^{\prime },j^{\prime }}-{\theta }_{i^{\prime },j}\right|$, and $\Delta \varphi =\left|{\varphi }_{i^{\prime },j^{\prime }}-{\varphi }_{i^{\prime },j}\right|$. The SIR of UAV j under the interference of beam $\left(i^{\prime },j^{\prime }\right)$ can be estimated by

$$\begin{array}{cc}\hfill \eta \left(i,j,i^{\prime },j^{\prime }\right)& =\frac{S}{I}=\frac{g_U\left(w_h,w_v,0,0\right)L\left(z_{i,j}\right)}{g_I\left(w_h,w_v,\Delta \theta ,\Delta \varphi \right)L\left(z_{i^{\prime },j}\right)}\hfill \\ \hfill & =\frac{f\left(z_{i,j}\right)}{f\left(z_{i^{\prime },j}\right)e^{\eta {\left(\frac{\Delta \theta }{w_h}\right)}^2+\eta {\left(\frac{\Delta \varphi }{w_v}\right)}^2}},\hfill \end{array}$$

(6)

where the function $f\left(z\right)$ is the path loss function. Without loss of generality, we assume the elements of UIM can only take on the values of 0 or 1, which, respectively, indicate the absence or presence of strong interference. The SIR in (6) is compared to a predefined threshold in order to determine the value of the UIM. Given the transmission power, the beam width, and locations of UAVs, the signal-to-interference ratio (SIR) is calculated for $\left(i,j\right)$. Specifically, the element of UIM $c_{i,i^{\prime }}\left(j,j^{\prime }\right)$ is set to 1 if the estimated SIR $\eta \left(i,j,i^{\prime },j^{\prime }\right)$ is greater than a preset threshold $\eta th$. It indicates that the beam $\left(i^{\prime },j^{\prime }\right)$ causes strong interference to UAV $\left(i,j\right)$, as shown in (7).

$$c_{i,i^{\prime }}\left(j,j^{\prime }\right)=\left\{\begin{array}{ccc}\hfill 1& & \eta \left(i,j,i^{\prime },j^{\prime }\right)<\eta th\hfill \\ \hfill 0& & \eta \left(i,j,i^{\prime },j^{\prime }\right)⩾\eta th.\hfill \end{array}\right.$$

(7)

In a scheduling frame, the UAVs within each cluster are scheduled periodically with their associated beams. Due to different task requirements, the scheduling cycles of each UAV can vary. However, all UAVs will be scheduled repeatedly within a period determined by the least common multiple (LCM) of all UAV scheduling periods. Let the scheduling cycle of $\left(i,j\right)$ be denoted as $T_j^i$, $i=1,\dots ,N_c$, $j=1,\dots ,M$. The system performance can then be evaluated over a period given by $N=LCM\left(T_j^i\left|i=1,\dots ,N_c,j=1,\dots ,M\right.\right)$. Since each UAV is scheduled repeatedly, the time slots in which a UAV is scheduled are determined by its initial time slot. Let $s_j^i$ represent the initial scheduling time slot of UAV $\left(i,j\right)$. The initial time slot of each UAV is restricted by its required cycle, i.e.,

$$1⩽s_j^i⩽T_j^i,s_j^i\in {\mathbb{Z}}^{+},i=1,\dots ,N_c,j=1,\dots ,M.$$

(8)

The UAV $\left(i,j\right)$ is scheduled in the time slots given by

$$\begin{array}{cc}\hfill & {\mathit{s}}_j^i={\mathbf{1}}_N\left(s_j^i+z{T}_j^i\right),\hfill \\ \hfill & i=1,\dots ,N_c,j=1,\dots ,M,z=1,\dots ,\frac{N}{T_j^i},\hfill \end{array}$$

(9)

where ${\mathbf{1}}_N\left(\mathit{x}\right)$ is a vector of length N, where elements specified by indices $\mathit{x}$ are equal to 1, while the remaining elements are equal to 0. The UAV $\left(i,j\right)$ is scheduled in the time slots that correspond to the 1 elements of its scheduling vector.

Assuming each RIS-UAV possesses L RIS arrays, it is capable of scheduling a maximum of L UAVs within a single time slot. The scheduling time slots of UAVs must satisfy the constraint

$$\sum _{j=1}^M{\mathit{s}}_j^i\prec \left(L+1\right){\mathbf{1}}_N,i=1,\dots ,N_c,j=1,\dots ,M,$$

(10)

where ${\mathbf{1}}_N$ is an all 1 vector of length N, and ≺ indicates that all elements of ${\mathit{s}}_j^i$ are less than $L+1$. In the time period N, we can express the total interference received by UAV $\left(i,j\right)$ from beam $\left(i^{\prime },j^{\prime }\right)$ as

$$I\left(s_j^i,s_{j^{\prime }}^{i^{\prime }}\right)=c_{i,i^{\prime }}\left(j,j^{\prime }\right){\left({\mathit{s}}_j^i\right)}^{\mathrm{T}}{\mathit{s}}_{j^{\prime }}^{i^{\prime }}.$$

(11)

We aim to minimize the overall interference level, and this optimization problem is mathematically formulated as

$$\begin{array}{cc}\hfill & \underset{s_j^i}{argmin}\gamma =\sum _{i=1}^{N_c}\sum _{j=1}^M\sum _{i^{\prime }=1}^{N_c}\sum _{j^{\prime }=1}^{M}I\left(s_j^i,s_{j^{\prime }}^{i^{\prime }}\right)\hfill \\ \hfill & s.t.\left(8\right),\left(9\right),\left(10\right).\hfill \end{array}$$

(12)

The constraint condition (8) ensures that the initial slot is smaller than the scheduling periods of UAVs. Moreover, (9) stems from the periodic scheduling restriction that requires each UAV to be allocated on periodic time slots according to its period. Furthermore, constraint condition (10) is imposed to limit the quantity of RIS of each cluster.

3. UAV Scheduling to Avoid Interference

3.1. UAV Scheduling Graph

The optimization problem (12) is an integer programming problem. Traditional methods, such as continuous relaxation, are not suitable for this problem due to the constraint $\left(9\right)$ which requires strict integer variables. However, we can approach it as a multi-stage decision problem, where each stage involves assigning a beam with its initial time slot. There are a total of $M{N}_c$ UAVs in the system, and their initial time slots are allocated successively by polling each cluster. The collection of all initial time slots for the UAVs forms the UAV scheduling vector, which is defined as

$$\begin{array}{cc}\hfill {\tilde{\mathit{s}}}_{M{N}_c}& =\left({\tilde{s}}_1,\dots ,{\tilde{s}}_{M{N}_c}\right)\hfill \\ \hfill & =\left(s_1^1,\dots ,s_1^{N_c},s_2^1,\dots ,s_{M-1}^{N_c-1},s_M^1,\dots ,s_M^{N_c}\right),\hfill \end{array}$$

(13)

where the initial time slot of UAV k is denoted as ${\tilde{s}}_k$. The relationship between ${\tilde{s}}_k$ and $s_j^i$ is determined by the mapping rule ${\tilde{s}}_k=s_{\lceil k/M\rceil }^{mod\left(k,M\right)}$. The allocation of initial time slots for $M{N}_c$ UAVs can be described by a UAV scheduling graph, where the nodes correspond to different selections of the initial slots of each UAV.

An example of the UAV scheduling graph is presented in Figure 2. The graph consists of $M{N}_c+1$ layers, where each layer represents a stage in the allocation of initial slots to the beams. The state ${\tilde{\mathit{s}}}_k$ denotes the UAV scheduling vector in stage k. In the initial stage, denoted as stage 0, no time slot is allocated and, thus, $\widetilde{{\mathit{s}}_0}$ is empty. In stage $k>0$, the decision of ${\tilde{s}}_k$ updates the UAV scheduling vector as ${\tilde{\mathit{s}}}_k=\left({\tilde{\mathit{s}}}_{k-1},{\tilde{s}}_k\right)$. Each node in the graph corresponds to a possible value of the UAV scheduling vector ${\tilde{\mathit{s}}}_k$ in stage k. It is important to note that the available time slots for a current UAV are determined by the previous selections of other UAVs, resulting in a connection between each node in stage k and a node in stage $k-1$. Finally, in the last layer, the nodes are referred to as destinations, and each destination node represents an allocation result of the initial time slots for all $M{N}_c$ UAVs, denoted as ${\tilde{\mathit{s}}}_{M{N}_c}$.

For convenience, we denote the scheduling cycles of UAVs as ${\tilde{T}}_k=T_{\lceil k/M\rceil }^{mod\left(k,M\right)}$. In stage k, the UAV k has a scheduling period of ${\tilde{T}}_k$. Therefore, the selection of time initial slot falls within the range of $\left(1,\dots ,{\tilde{T}}_k\right)$. However, the selection of initial time slots is also constrained by the number of RIS arrays, as shown in Equation (10). In the graph, there are some nodes that are not achievable, indicated by dash arrows. In the example illustrated in Figure 2, we have $N_c=2$, $M=2$, $T_1^1=T_2^1=T_2^2=2$, and $T_1^2=4$. Both clusters have only one RIS array. The process of reaching the destination node ${\tilde{\mathit{s}}}_4=\left(1,1,2,2\right)$ is as follows: At stage 1, the initial time slot of UAV $\left(1,1\right)$ is set to 1, i.e., ${\tilde{s}}_1=1$. Then, ${\tilde{s}}_1,{\tilde{s}}_2$, and ${\tilde{s}}_3$ are set to be 1, 2, and 2 respectively. In stage 3, it is observed that if ${\tilde{s}}_1=1$, then ${\tilde{s}}_3$ cannot be 1 or 3 due to the limited number of UAVs that can be scheduled, which is constrained by the number of available RIS arrays.

3.2. Local Interference Minimization Scheme

In the UAV scheduling graph, each destination node ${\tilde{\mathit{s}}}_{M{N}_c}$ is associated with a specific total interference level. The main objective of the UAV scheduling problem is to identify the destination node that exhibits the lowest interference level. To achieve this, an exhaustive search scheme can be employed to consider every possible destination node. However, even with the constraints imposed by the RIS number, the total number of destination nodes remains substantial, on the order of ${\prod }_{k=1,\dots ,M{N}_c}{T}_k$. Consequently, the exhaustive search or directing dynamical programming approach becomes excessively complex. To address this complexity, we propose a greedy scheme known as local interference minimization. This scheme selects the UAV scheduling time slot based on minimizing the current interference level at each decision stage.

Consider a node in stage $k-1$, where $k=1,\dots ,M{N}_c$, and $k-1$ UAVs are already allocated with their initial time slots in the vector ${\tilde{\mathit{s}}}_{k-1}$. The interference level in stage $k-1$ is denoted as ${\gamma }_{k-1}\left({\tilde{\mathit{s}}}_{k-1}\right)$. Each node in stage k is assigned a cost, represented as ${\gamma }_k\left({\tilde{\mathit{s}}}_k\right)$. The LIM scheme selects ${\tilde{s}}_k$ to minimize ${\gamma }_k\left({\tilde{\mathit{s}}}_k\right)$. The interference level of the LIM scheme in stage k can be derived using the formula for interference level in Equation (11).

$$\begin{array}{cc}\hfill & {\gamma }_{k,LIM}=\underset{{\tilde{\mathit{s}}}_k}{min}{\gamma }_k\left({\tilde{\mathit{s}}}_k\right)\hfill \\ \hfill & ={\gamma }_{k-1,LIM}+\underset{{\tilde{s}}_k\in {\mathcal{N}}_{k-1}}{min}{\tilde{\gamma }}_k\left({\tilde{s}}_k\right)\hfill \\ \hfill & ={\gamma }_{k-1,LIM}+\underset{{\tilde{s}}_k\in {\mathcal{N}}_{k-1}}{min}\sum _{k^{\prime }=1}^k\left(I\left({\tilde{s}}_k,{\tilde{s}}_{k^{\prime }}\right)+I\left({\tilde{s}}_{k^{\prime }},{\tilde{s}}_k\right)\right)\hfill \\ \hfill & ={\gamma }_{k-1,LIM}+\underset{{\tilde{s}}_k\in {\mathcal{N}}_{k-1}}{min}\sum _{i^{\prime }=1}^{N_c^{\prime }}\sum _{j^{\prime }=1}^{M^{\prime }}\left(I\left(s_j^i,s_{j^{\prime }}^{i^{\prime }}\right)+I\left(s_{j^{\prime }}^{i^{\prime }},s_j^i\right)\right),\hfill \end{array}$$

(14)

where $N_c^{\prime }=mod\left(k-1,M\right)$, $M^{\prime }=\lceil k-1/M\rceil$ and $i=mod\left(k,M\right)$, $j=\lceil k/M\rceil$ as shown in Equation (13). The set ${\mathcal{N}}_{k-1}$ contains all feasible initial slots for UAV k that satisfy the RIS constraints. ${\tilde{\gamma }}_k\left(s_k\right)$ represents the local interference, which includes the interference between UAV k and the previously scheduled UAVs.

The interference level caused by the scheduling collision of two UAVs, denoted as $I\left(s_i^j,s_{i^{\prime }}^{j^{\prime }}\right)$, can be calculated using a simple formula. Let the initial time slots of two UAVs be $s_j^i$ and $s_{j^{\prime }}^{i^{\prime }}$, and their periods be $T_j^i$ and $T_{j^{\prime }}^{i^{\prime }}$. According to the condition for the existence of integer solutions of Diophantine equations [32], the two UAVs will collide if there exists an integer q such that $s_j^i-s_{j^{\prime }}^{i^{\prime }}=qϵ\left(T_j^i,T_{j^{\prime }}^{i^{\prime }}\right)$, where $ϵ\left(a,b\right)$ represents the greatest common divisor (GCD) of integer numbers a and b. Given the scheduling periods and allocation result, the collision ratio can be determined when two UAVs collide with each other. The collision ratio is given by the following conclusion: If two UAVs with transmit periods $T_1$ and $T_2$ cause collisions, the ratio of collision time slots is $ϵ\left(T_1{T}_2\right)$ as N approaches infinity. To prove this result, we assume that two UAVs collide in one slot and then collide again after $n_1$ periods of UAV 1 and $n_2$ periods of UAV 2, then we have $n_1{T}_1=n_2{T}_2$. By letting $T_1={\xi }_1ϵ\left(T_1{T}_2\right)$ and $T_2={\xi }_2ϵ\left(T_1{T}_2\right)$, we obtain

$$\frac{n_1}{n_2}=\frac{T_2}{T_1}=\frac{{\xi }_1}{{\xi }_2}.$$

(15)

Given that ${\xi }_1,{\xi }_2$ are prime numbers, the minimum integer numbers that satisfy Equation (15) are $n_1={\xi }_2$ and $n_2={\xi }_1$. As a result, the next collision occurs with an interval of ${\xi }_2{T}_1=\frac{T_1{T}_2}{ϵ\left(T_1{T}_2\right)}$, which is the least common multiple (LCM) of their periods. Since collisions occur at every $\frac{T_1{T}_2}{ϵ\left(T_1{T}_2\right)}$ time slots, the proportion of collision time slots is the reciprocal of this value. When N is large, the number of collision time slots between two UAVs in the N time slots can be approximated by

$$I\left(s_j^i,s_{j^{\prime }}^{i^{\prime }}\right)=\left\{\begin{array}{ccc}0& & \frac{s_j^i-s_{j^{\prime }}^{i^{\prime }}}{ϵ\left(T_j^i,T_{j^{\prime }}^{i^{\prime }}\right)}\notin \mathbb{Z}\hfill \\ \frac{c_{i,i^{\prime }}\left(j,j^{\prime }\right)Nϵ\left(T_j^i{T}_{j^{\prime }}^{i^{\prime }}\right)}{T_j^i{T}_{j^{\prime }}^{i^{\prime }}}& & \frac{s_j^i-s_{j^{\prime }}^{i^{\prime }}}{ϵ\left(T_j^i,T_{j^{\prime }}^{i^{\prime }}\right)}\in \mathbb{Z}.\hfill \end{array}\right.$$

(16)

The details of the LIM algorithm are presented in Algorithm 1. In the algorithm, steps 4–9 identify all candidate time slots that meet the constraint of the limited number of RIS. Steps 10–13 are dedicated to selecting the initial time slot for the UAV in order to minimize local interference with the already scheduled UAVs.

Algorithm 1: Local interference minimization (LIM).

3.3. LIM-Based Rollout Scheme

The rollout algorithm is a general approach for enhancing the solutions of multi-stage decision problems. The main concept involves taking a one-step lookahead to evaluate neighboring nodes using a basic heuristic strategy. It has been demonstrated in [33] that if the base strategy exhibits the property of sequential consistency, a rollout algorithm based on this strategy can yield benefits. A strategy satisfies the sequential consistent condition if it leads to the same decision sequence regardless of the stage it starts. A strategy fulfills the condition of sequential consistency if it consistently leads to the same decision sequence, regardless of the starting stage. In the context of UAV slot allocation, a sequential consistent strategy adheres to the following principle: for $k=1,\dots ,M{N}_c$, if the strategy selects the initial time slots sequence $\left(s_k,s_{k+1},\dots ,s_{M{N}_c}\right)$ when commencing at stage $k-1$, it will select the time slot sequence $\left(s_{k+1},\dots ,s_{M{N}_c}\right)$ when starting at stage k. In the LIM algorithm, the choice of $s_k$ depends on the local interference ${\tilde{\gamma }}_k\left(s\right)$, and the selected time slot sequence remains unchanged regardless of the starting stage. Thus, the LIM algorithm satisfies the condition of sequential consistency and can serve as the fundamental strategy for constructing a rollout algorithm.

Algorithm 2 presents the procedure for the rollout algorithm based on the LIM. In stage k, the RIM algorithm applies the LIM based on all feasible ${\tilde{\mathit{s}}}_k$ to reach a destination, and subsequently selects the ${\tilde{\mathit{s}}}_k$ with the minimum interference level. Steps 4–8 identify all candidate time slots that satisfy the RIS constraint for each cluster, followed by the application of the LIM algorithm for different selections of time slot $s_k\in {\mathcal{N}}_{k-1}$. Finally, steps 9 and 10 determine the optimal time slot and update the UAV scheduling vector.

Algorithm 2: Rollout interference minimization (RIM).

To illustrate the improvement of the RIM algorithm compared to the LIM algorithm, we provide an example in Figure 3. In this example, each UAV has two choices for the initial time slot. We focus on applying one step of the RIM algorithm at stage k and compare it to the LIM algorithm. The node ${\widehat{s}}_{k-1}$ represents the UAV scheduling vector in stage $k-1$, the two time slot selections for UAV k result in two nodes, ${\widehat{\mathit{s}}}_{k,1}$ and ${\widehat{\mathit{s}}}_{k,2}$. Assuming that ${\gamma }_k\left({\widehat{\mathit{s}}}_{k,1}\right)<{\gamma }_k\left({\widehat{\mathit{s}}}_{k,2}\right)$, the LIM scheme will select ${\widehat{\mathit{s}}}_{k,1}$ based on the node ${\widehat{s}}_{k-1}$, as indicated by the blue arrow in the figure. On the other hand, the RIM scheme will apply the LIM algorithm based on both nodes ${\widehat{\mathit{s}}}_{k,1}$ and ${\widehat{\mathit{s}}}_{k,2}$, and find their corresponding destination nodes ${\tilde{\mathit{s}}}_{M{N}_c,LIM}$ and ${\tilde{\mathit{s}}}_{M{N}_c,RIM}$, which are represented by the red arrow in the figure. The interference levels of these nodes are denoted as ${\gamma }_{M{N}_c,LIM}$ and ${\gamma }_{M{N}_c,RIM}$.

Because the LIM scheme satisfies the sequential consistency condition, applying LIM from ${\widehat{\mathit{s}}}_{k-1}$ will result in the same destination as applying LIM on ${\widehat{\mathit{s}}}_{k,1}$, and we have

$$\begin{array}{cc}\hfill {\gamma }_{M{N}_c,LIM}=& \underset{\begin{array}{l}{\tilde{s}}_k,\dots ,{\tilde{s}}_{M{N}_c}\\ {\tilde{\mathit{s}}}_{k-1}={\widehat{\mathit{s}}}_{k-1}\end{array}}{min}{\gamma }_{M{N}_c}\left({\tilde{\mathit{s}}}_{M{N}_c}\right)\left(w/\mathrm{LIM}\right)\hfill \\ \hfill =& \underset{\begin{array}{l}{\tilde{s}}_{k+1},\dots ,{\tilde{s}}_{M{N}_c}\\ {\tilde{\mathit{s}}}_k={\widehat{\mathit{s}}}_{k,1}\end{array}}{min}{\gamma }_{M{N}_c}\left({\tilde{\mathit{s}}}_{M{N}_c}\right)\left(w/\mathrm{LIM}\right).\hfill \end{array}$$

(17)

Assuming the RIM algorithm selects node ${\widehat{\mathit{s}}}_{k,2}$ as indicated by the blue arrows in the figure, we have

$$\begin{array}{cc}\hfill {\gamma }_{M{N}_c,RIM}=& \underset{\begin{array}{l}{\tilde{s}}_{k+1},\dots ,{\tilde{s}}_{M{N}_c}\\ {\tilde{\mathit{s}}}_k={\widehat{\mathit{s}}}_{k,2}\end{array}}{min}{\gamma }_{M{N}_c}\left({\tilde{\mathit{s}}}_{M{N}_c}\right)\left(w/\mathrm{LIM}\right).\hfill \end{array}$$

(18)

The RIM algorithm selects ${\tilde{\mathit{s}}}_k$, which leads to the minimum interference, indicating that ${\gamma }_{M{N}_c,RIM}⩽{\gamma }_{M{N}_c,LIM}$. This demonstrates that applying RIM at stage k from ${\tilde{\mathit{s}}}_{k-1}$ can reduce the interference level.

3.4. Complexity Analysis

Based on the UAV scheduling graph and the aforementioned parameters, the number of destination nodes is in the order of $\prod T_k,k=1,\dots ,M_{max}{N}_c$, which is the complexity order of an exhaustive search. The LIM algorithm determines the initial time slot for one UAV in every stage. Then its complexity is $O\left(N_c{M}_{max}{T}_{max}\right)$, where $M_{max}$ and $T_{max}$ denote the maximum number of UAVs in one cluster and the maximum periods of all UAVs, respectively. The RIM algorithm, on the other hand, employs the LIM algorithm to select a node in a maximum of $T_{max}$ iterations, resulting in a complexity of $O\left(N_c^2{M}_{max}^2{T}_{max}^2\right)$.

4. Simulation Results and Discussion

In this section, we evaluate the proposed scheme using Monte Carlo simulations. Matlab is chosen as the simulation tool due to its efficient matrix operation functions. We simulate the interference between UAV clusters by generating a random UAV cluster scenario. Based on this, we employ the aforementioned scheduling scheme to determine the transmission time slots of the UAVs and assess the interference of the entire system.

The sequential scheduling approach is used as the benchmark, in which time slots are allocated to UAVs in a sequential manner, with any time slots that violate the RIS constraint being skipped. Figure 4 illustrates an example of time slot selection and the resulting UAV collisions. The figure depicts two UAV clusters, each consisting of four UAVs. The x-axis represents the indices of the time slots, while the y-axis represents the indices of the UAVs within each cluster. The scheduling cycles of the UAVs are given by the vectors $\left(T_1^{N_c},\dots ,T_M^{N_c}\right)=\left(4,8,12,16\right)$ and $\left(T_2^{N_c},\dots ,T_M^2\right)=\left(8,8,14,16\right)$. The LCM of the UAV periods is denoted by $N=40$. The dashed vertical line in the figure represents the time slot at which a UAV collision occurs. It can be observed that with the sequential scheme, UAV scheduling collisions occur in a total of eight time slots, whereas with the LIM scheme, only one time slot experiences a collision. Notably, no UAV scheduling collisions occur with the RIM scheme.

We evaluate the average interference level per time slot across various schemes in our simulation. To generate the elements of the interference matrix, we employ random variables following a Bernoulli distribution. The interference probability, which indicates the likelihood of significant interference between two UAVs, is defined as the expectation of the Bernoulli distribution. For every interference probability, we conduct 100 random interference scenarios. By employing the aforementioned scheduling algorithm, we determine the scheduling time slots for the UAVs and statistically analyze the average interference level for each time slot. The simulation results are given in

Table 2. The average interference level per time slot with two UAV clusters.

(a) $\mathit{M}=4$, $\mathit{L}=2$, Schedule Periods $\left(4,8,12,16\right)$, $\left(8,8,14,16\right)$.
interference probability	0.2	0.4	0.6	0.8	1
sequential schedule	0.132	0.237	0.351	0.491	0.604
LIM	0.022	0.047	0.075	0.149	0.225
RIM	0	0	0	0	0
(b) $\mathit{M}=\mathbf{6}$, $\mathit{L}=\mathbf{4}$, Schedule Periods $\left(\mathbf{4},\mathbf{8},\mathbf{8},\mathbf{12},\mathbf{16},\mathbf{20}\right)$, $\left(\mathbf{8},\mathbf{8},\mathbf{14},\mathbf{16},\mathbf{18},\mathbf{20}\right)$.
interference probability	0.2	0.4	0.6	0.8	1
sequential schedule	0.276	0.458	0.754	0.881	1.200
LIM	0.033	0.089	0.126	0.205	0.298
RIM	0	0.001	0.003	0.009	0.016
(c) $\mathit{M}=\mathbf{8}$, $\mathit{L}=\mathbf{4}$, Schedule Periods $\left(\mathbf{4},\mathbf{6},\mathbf{8},\mathbf{8},\mathbf{12},\mathbf{14},\mathbf{16},\mathbf{20}\right)$, $\left(\mathbf{4},\mathbf{6},\mathbf{8},\mathbf{8},\mathbf{14},\mathbf{16},\mathbf{18},\mathbf{20}\right)$.
interference probability	0.2	0.4	0.6	0.8	1
sequential schedule	0.399	0.767	1.312	1.766	2.145
LIM	0.056	0.162	0.262	0.405	0.548
RIM	0.006	0.027	0.035	0.064	0.095

and

Table 3. The average interference level per time slot with three UAV clusters.

(a) $\mathit{M}=4$, $\mathit{L}=4$, Schedule Periods $\left(4,8,12,16\right)$, $\left(8,8,14,16\right)$, $\left(8,12,16,20\right)$.
interference probability	0.2	0.4	0.6	0.8	1
sequential schedule	0.424	0.867	1.274	1.736	2.135
LIM	0.089	0.216	0.353	0.496	0.595
RIM	0.004	0.013	0.026	0.037	0.047
(b) $\mathit{M}=\mathbf{6}$, $\mathit{L}=\mathbf{4}$, Schedule Periods $\left(\mathbf{4},\mathbf{8},\mathbf{8},\mathbf{12},\mathbf{16},\mathbf{20}\right)$, $\left(\mathbf{8},\mathbf{8},\mathbf{14},\mathbf{16},\mathbf{18},\mathbf{20}\right)$, $\left(\mathbf{6},\mathbf{8},\mathbf{14},\mathbf{16},\mathbf{18},\mathbf{20}\right)$.
interference probability	0.2	0.4	0.6	0.8	1
sequential schedule	0.639	1.240	1.756	2.534	3.121
LIM	0.122	0.223	0.456	0.821	1.343
RIM	0.007	0.029	0.063	0.106	0.129
(c) $\mathit{M}=\mathbf{8}$, $\mathit{L}=\mathbf{4}$, Schedule Periods $\left(\mathbf{4},\mathbf{6},\mathbf{8},\mathbf{8},\mathbf{12},\mathbf{14},\mathbf{16},\mathbf{20}\right)$, $\left(\mathbf{4},\mathbf{6},\mathbf{8},\mathbf{8},\mathbf{14},\mathbf{16},\mathbf{18},\mathbf{20}\right)$, $\left(\mathbf{4},\mathbf{6},\mathbf{6},\mathbf{8},\mathbf{14},\mathbf{14},\mathbf{18},\mathbf{20}\right)$.
interference probability	0.2	0.4	0.6	0.8	1
sequential schedule	1.034	2.227	3.405	4.783	6.114
LIM	0.208	0.626	1.212	0.405	0.548
RIM	0.065	0.191	0.442	0.665	0.881

for two and three UAV clusters, respectively.

A comparison of Table 2a–c reveals that an increase in the number of UAVs per cluster leads to a corresponding increase in average interference level. In the scenario of two clusters with four UAVs per cluster, the RIM scheme can achieve complete interference-free communication. However, in the scenario of six UAVs per cluster, the RIM scheme is unable to completely avoid interference in high-probability scenarios. When comparing Table 2b and Table 3a, it is evident that despite having the same total number of UAVs, the average interference level is higher in the three-cluster scenario. This is due to the presence of more sources of interference in the three-cluster scenario.

We evaluate the degree of interference reduction after implementing LIM and RIM algorithms in diverse scenarios with varying numbers of clusters and UAVs. For the case of two UAV clusters, it is shown that the LIM scheme can reduce the interference level by 63–75% when compared to the sequential scheduling scheme. Furthermore, the RIM scheme can reduce the interference level by 96–100%. When there are two UAV clusters and each cluster has four UAVs, the RIM scheme completely eliminates interference between UAV clusters. In the scenario with three UAV clusters, the LIM scheme achieves 57–88% reduction in interference compared to the baseline scheme. The RIM scheme, building upon the LIM scheme, provides an additional 64–93% reduction in interference.

5. Future Directions

In this section, we identify two problems associated with the application of the current scheduling scheme in practical scenarios. Furthermore, we propose potential solutions to address these issues.

The rollout scheme exhibits polynomial complexity, as demonstrated by the aforementioned complexity analysis. However, the computational complexity of the rollout algorithm remains excessively high in certain cases. On one hand, when UAVs experience rapid movements, it becomes crucial to expedite the calculation of time slot scheduling. On the other hand, while the numbers of clusters and UAVs are typically not extensive, the RIM algorithm can effectively manage the resulting increase in complexity as the number of UAVs increases. Nevertheless, the analysis of computational complexity reveals that the RIM algorithm faces challenges in efficiently handling scenarios with large scheduling cycles. In such cases, a UAV scheduling scheme with lower complexity becomes essential.

One potential approach to mitigate the complexity associated with the rollout algorithm is to replace the entire basic heuristic strategy with imprecise interference estimates. Specifically, instead of exclusively executing the greedy algorithm from the current stage to a destination node during the computation of node values in a stage, only a restricted number of steps are evaluated. In the context of this article, the greedy algorithm can be utilized to schedule a limited quantity of UAVs after the current stage, evaluating the partial interference caused by the scheduled UAVs. This strategy can effectively reduce complexity within the rollout algorithm.

In real-world scenarios, the position information of UAVs is often imperfect, resulting in inaccurate interference information among them. Consequently, there is a pressing need to evaluate the uncertainty of the interference information and develop a resilient UAV scheduling strategy. A pertinent research question to address is how to design RIS weights to mitigate interference when the UAV location information is inaccurate. While a wider beam formed by the RIS can enhance the reliability of UAV coverage, it also increases the likelihood of interference between UAVs. Additionally, when the position of the UAV is uncertain, it is imperative to adjust the value of the UAV interference matrix to account for the interference error caused by the positional uncertainty. This adjustment will enhance the robustness of the UAV scheduling scheme.

6. Conclusions

In this work, we studied the UAV scheduling problem within the context of an RIS-aided multi-cluster UAV system. In this system, the BS establishes periodic communication with the UAVs through the support of RIS arrays carried by an RIS-UAV. We develop a UAV interference matrix to effectively capture and describe the interference between UAV clusters. Taking into account the constraints imposed by periodic transmission and the limited number of RIS arrays, we devise UAV scheduling time slots that aim to mitigate inter-cluster interference. To solve the scheduling problem, we reformulate it as a multi-stage path-finding problem and introduce the LIM algorithm. This algorithm effectively minimizes the local interference at each stage. Furthermore, we propose the RIM algorithm, a rollout-based UAV scheduling algorithm that builds upon the LIM algorithm. The simulation results highlight the effectiveness of the RIM scheme in significantly reducing inter-cluster interference when compared to both the baseline approach and the LIM algorithm.