Resource Allocation in UAV-Enabled NOMA Networks for Enhanced Six-G Communications Systems

Abstract

Enhancing energy efficiency, content distribution, latency, and transmission speeds are vital components of communication systems. Multiple access methods hold great promise for boosting these performance indicators. This manuscript evaluates the effectiveness of Non-Orthogonal Multiple Access (NOMA) and Orthogonal Multiple Access (OMA) systems within a single cell, where users are scattered randomly and rely on relays for dependability. This paper presents a model for improving energy efficiency, content distribution, latency, and transmission speeds in communication systems using NOMA and OMA systems within a single cell. Additionally, this paper also proposes a caching strategy using unmanned aerial vehicles (UAVs) as aerial base stations for ground users. These UAVs distribute cached content to minimize the overall latency of content demands from ground users while modifying their positions. We carried out simulations using various cache capacities and user counts linked to their respective UAVs. Furthermore, we evaluated OMA and NOMA in terms of the achievable rate and energy efficiency. The proposed model has achieved noteworthy enhancement across various scenarios including different sum rates, numbers of mobility users, diverse cache sizes, and amounts of power allocation.

1. Introduction

The growth of wireless networks has been rapid over the past decade. Fifth-generation (5G) technology supports emerging fields such as machine-to-machine (M2M) communication, enhanced mobile broadband, high-speed entertainment, and virtual/augmented reality. However, 5G may struggle to satisfy the increasing demands for data traffic, spectral efficiency, massive connectivity, and capacity while maintaining fairness [1,2,3,4,5,6]. Additionally, the widespread adoption of the Internet of Things (IoT) presents significant challenges to wireless networks [7]. The exponential rise in data transmission driven by emerging technologies has imposed immense strains on ground-based communication infrastructure, resulting in increased delays and reduced network efficiency. Current solutions like D2D communication and multiple access techniques exhibit limitations in deployment, coverage, power, and capacity. Orthogonal multiple access lacks the spectral efficiency required for 6G networks, and increasing terrestrial base stations is impractical. In particular, the International Telecommunication Union (ITU) projects a 10,000-fold increase in wireless data traffic by 2030 compared to 2010. This surge in data traffic will place considerable strain on communication infrastructure, particularly ground base stations, and will lead to increased transmission delays for content downloads.

The main challenges when designing infrastructure for future wireless networks include deployment strategies, coverage area, power consumption, link capacity, and transmission delay. One technology that addresses these challenges is device-to-device (D2D) communication in sixth-generation (6G) networks. As a result, researchers are working to develop 6G wireless systems that explore the terahertz (THz) band, ranging from 0.1 to 10 THz [4,8,9,10]. Multiple access techniques can assist in satisfying 6G requirements. Non-orthogonal multiple access (NOMA) has attracted interest for its ability to increase spectral efficiency. Orthogonal multiple access (OMA), which assigns orthogonal resources to individual users, lacks the spectral efficiency needed for 6G. Time-division multiple access (TDMA), frequency0division multiple access (FDMA), and code-division multiple access (CDMA) are examples of OMA.

NOMA is available in power and code domain varieties and allows multiple users to share the same resource, resulting in efficient spectrum utilization and superior connectivity compared to OMA. Increasing the number of terrestrial base stations is not a practical or efficient strategy. Caching has been identified as a beneficial method to decrease transmission latency in cellular networks, particularly in compact base stations. Traditional caching approaches primarily address users’ needs to consistently connect to the same base station, offering limited benefits to mobile users. Several solutions exist to overcome this issue, such as incorporating unmanned aerial vehicles (UAVs), also known as drones, into wireless networks. These UAVs are comparatively easier to manage and more economical to deploy [10,11,12,13,14]. To address these challenges, we propose a UAV-enabled non-orthogonal multiple access model for 6G networks. This model features aerial UAV stations working with a ground base station to service users, storing popular content locally to reduce delays. We make assumptions about UAV capabilities and define system parameters like time slots. We then detail the achievable rate and power allocation based on user distance from the UAV based on NOMA principles. The mobility and transmission models account for factors impacting UAV-to-ground and UAV-to-UAV connections like line-of-sight probability, signal attenuation, and free space path loss. This novel approach promises to alleviate the strains on ground infrastructure to enable efficient, high-capacity 6G networks. Algorithm 1 is used to form NOMA channel capacity versus bandwidth at different signal-to-noise ratios as shown in Figure 1.

Algorithm 1: Channel Capacity vs. Bandwidth

Input: BW = [1 8 × 10³ 8 × 104 8 × 105 8 × 106]; SNR_dB = [0 5 10 20 30] Start Procedure   SNR = 10.^(SNR_dB/10); % SNR in linear scale   For k = 1:length(SNR)          C(k,:) = BW.*log2(1 + SNR(k)); %capacity   End For  Figure  mesh(BW,SNR_dB,C);  xlabel(‘Bandwidth (Hz)’);  ylabel(‘SNR [dB]’)  zlabel(‘Data Rate (bist/sec)’)  set(gca, ‘xscale’, ‘log’)  set(gca, ‘zscale’, ‘log’) End Procedure

The emergence of 6G networks promises major advances over existing 5G capabilities. Key features of 6G that enable transformative applications include utilization of higher frequency bands above 100 GHz, integration of advanced artificial intelligence, enhanced mobile broadband speeds, a focus on energy efficiency and sustainability, and ultra-dense device connectivity. This work proposes a novel model that aligns well with these 6G trends and requirements. Specifically, our use of unmanned aerial vehicles (UAVs) as aerial base stations, along with deep reinforcement learning and deep neural networks for resource and power allocation, is designed to leverage 6G’s higher bandwidth, AI integration, fast data rate support, green communications, and ability to serve massive device densities from the sky.

Unmanned aerial vehicles (UAVs), also known as drones, are aircraft systems that function without an onboard human pilot. They can be controlled remotely by an operator on the ground or programmed to autonomously follow pre-defined flight paths using sophisticated onboard computers and navigation systems. UAVs come in various forms, ranging from small quadcopters to large fixed-wing drones and serve numerous purposes, such as aerial photography, surveillance, agriculture, scientific research, and telecommunications. UAVs offer multiple advantages for 6G wireless networks, including [15,16,17,18,19,20,21,22,23]:

• Quick and versatile deployment: UAVs can be deployed swiftly and in diverse locations, making them well-suited for providing temporary or supplementary network coverage during special events, disaster recovery scenarios, or in hard-to-reach areas.
• Real-time position adjustments: UAVs can modify their positions in real-time based on user demand and network conditions, optimizing coverage areas and providing better connectivity and targeted capacity improvements where needed.
• Decreased transmission latency: As aerial base stations, UAVs can minimize the distance between users and base stations, reducing transmission latency and enhancing service quality.
• Improved spectral efficiency: UAVs can use advanced 6G technologies, such as NOMA, to boost spectral efficiency and accommodate more users within the same frequency band.
• Optimized energy consumption: UAVs can be outfitted with energy-saving technologies and intelligent power management systems to maximize energy efficiency while delivering reliable wireless services.

UAVs have the potential to address various challenges faced by 6G wireless networks through their flexible deployment, real-time position adjustment, decreased transmission latency, and enhanced spectral and energy efficiency. Integrating UAVs into the 6G ecosystem can improve network performance and reduce transmission delays. In this paper, we proposed a model for wireless networks that incorporates UAV-mounted base stations with NOMA capabilities, where UAVs can cooperate and exchange cached data using deep reinforcement learning [24,25]. The main contributions of the proposed framework and algorithms are as follows:

• We present a unique model for 6G wireless networks integrating UAV-mounted base stations with NOMA capabilities. This model uses UAVs to enhance network performance and reduce transmission delays.
• We propose a novel reinforcement learning approach. In this approach, a UAV, acting as a reinforcement learning agent, interacts with its environment to learn the optimal policy for maximizing its cumulative reward. The UAV makes decisions including user association, caching, and power at every time slot based on specific conditions.
• We effectively manage a large state space in our model by combining DRL with a deep neural network (DNN). This approach enables our model to handle the vast number of states arising due to the dynamic and continuous nature of the UAV’s environment.
• Our model is designed to enable continuous adaptation. The parameters of the actor and critic networks in our model are updated at every time slot, which allows the agent UAV to continuously adapt and respond to changes in the environment.
• Our model is capable of serving multiple users simultaneously from a single source, thereby significantly improving network efficiency.
• We propose a novel power distribution mechanism based on user fairness, their requirements, and quality of service (QoS) expectations.
• We eliminate the need for scheduling in NOMA, thereby reducing transmission latency—a significant improvement over existing models.
• We expand the system’s coverage area with the assistance of cooperative NOMA, offering higher spectral efficiency.

This manuscript is divided into five distinct sections. Section 2 reviews prior research relevant to our area of study. Section 3 introduces the suggested framework and details the various stages within this framework. Section 4 presents the experimental data from several test cases and includes a discussion section. Lastly, Section 5 contains the concluding remarks and a list of references.

2. Literature Review

This section provides an overview of the diverse technologies and strategies utilized for resource allocation. It mainly focuses on the unique approaches recently featured in several respected publications. Sun et al. offered an optimal power and subcarrier allocation strategy in FD-MC-NOMA systems, which improved the overall rate performance. Their proposition is backed by theoretical analysis and simulation outcomes. The significant obstacles outlined in this paper include exploring multiple access strategies, dealing with more realistic channel models and practical restrictions, and contemplating the use of machine learning or artificial intelligence in resource allocation.

Wang et al. [26] examine the efficacy of downlink and uplink NOMA within a densely populated wireless network, considering variables such as user density, network magnitude, and power distribution. They present the issue as a stochastic geometry-based study to extract analytical expressions for coverage likelihood and area spectral efficiency (ASE). They conduct an in-depth analysis of downlink and uplink NOMA performance in the dense wireless network, including the derivation of coverage probability and ASE under varying user densities, network sizes, and power allocation strategies. The authors validate their theoretical work with simulation results that showcase the superiority of NOMA over OMA in densely populated wireless networks. This paper’s key challenges revolve around the assumption of flawless successive interference cancellation (SIC) at the receiver end, a condition that may only be occasionally achievable in real-world situations. Furthermore, there is a need to research the performance of the proposed integrated resource allocation algorithm within more intricate network scenarios.

Zeng and Zhang [27] delve into optimizing a UAV’s path to enhance the energy efficiency of wireless communication systems, considering variables like transmission power, flight velocity, and altitude. They propose an algorithm for trajectory optimization that utilizes successive convex optimization (SCO) techniques. This algorithm iteratively hones the UAV’s path and transmission power to maximize the energy efficiency of the communication system. The authors validate their proposal with simulation results that illustrate the suggested trajectory optimization algorithm’s effectiveness in augmenting UAV communication systems’ energy efficiencies. The results indicate that the proposed algorithm significantly surpasses traditional fixed-trajectory methods. The key challenges outlined in this paper involve integrating other practical factors, such as user mobility, latency requirements, or dynamic network conditions, into the trajectory optimization problem for UAV communication systems. Additionally, they need to explore the application of machine learning or artificial intelligence techniques to optimize the trajectory and resource allocation in UAV communication systems, potentially leading to more efficient and adaptable algorithms.

Zhang et al. [28] delve into the issue of simultaneously optimizing a UAV’s path and power allocation to amplify the sum rate of a UAV relay-assisted NOMA network. This considers limitations such as the UAV’s maximum speed, altitude restrictions, and transmission power. The authors proposed a joint trajectory and power optimization algorithm that relies on successive convex programming (SCP) and alternating optimization techniques. This algorithm iteratively refines the UAV’s path and power allocation to maximize the sum rate of the NOMA network. The authors validate their proposal with simulation results that illustrate the efficiency of the suggested joint trajectory and power optimization algorithm for UAV relay-assisted NOMA networks. The results reveal that the proposed algorithm surpasses other conventional methods in terms of sum rate performance. The primary challenges discussed in this paper revolve around integrating other practical factors, such as user mobility, latency requirements, or dynamic network conditions, into the optimization issue for UAV relay-assisted NOMA networks. Additionally, there is a need to explore the application of machine learning or artificial intelligence techniques for optimizing the trajectory and resource allocation in UAV relay-assisted NOMA networks, potentially leading to more efficient and adaptive algorithms.

Zhang et al. [29] investigate the resource allocation issue in F-RANs utilizing NOMA to maximize the network’s total weighted utility while maintaining user fairness. They present the problem as a non-convex optimization issue involving joint power allocation and user clustering. To resolve the resource allocation problem, the authors suggest a two-stage algorithm. They provide a theoretical analysis of the proposed algorithm, including its convergence characteristics and the optimality of its solutions. The key challenges highlighted in this paper involve applying the proposed algorithm to other multiple access schemes like orthogonal multiple access (OMA) or different variations of NOMA and comparing the performance and fairness implications. Additionally, there is a need to integrate other practical factors, such as user mobility, latency requirements, or energy efficiency constraints, into the resource allocation issue in F-RANs with NOMA.

X Liu et al. [30] delve into the issue of jointly optimizing a UAV’s path and power allocation to maximize the sum rate of a UAV-assisted NOMA network. This considers restrictions like the maximum UAV speed, altitude limits, and transmission power. The authors propose a joint trajectory and power control algorithm based on block coordinate descent (BCD) and successive convex programming (SCP) techniques. This algorithm iteratively refines the UAV’s path and power allocation to maximize the sum rate of the NOMA network. The authors validate their proposal with simulation results that illustrate the effectiveness of the proposed joint trajectory and power control algorithm for UAV-assisted NOMA networks. The results reveal that the proposed algorithm surpasses other conventional methods in terms of sum rate performance. The main challenges discussed in this paper involve integrating other practical factors, such as user mobility, latency requirements, or dynamic network conditions, into the optimization problem for UAV-assisted NOMA networks. Additionally, there is a need to explore the application of machine learning or artificial intelligence techniques for optimizing the trajectory and resource allocation in UAV-assisted NOMA networks, potentially leading to more efficient and adaptable algorithms.

Ghafoor et al. [31] proposed a novel approach that optimizes energy efficiency (EE) by employing user equipment (UE) clustering (UE-C) with downlink hybrid NOMA (H-NOMA) assisted beyond 5G (B5G) HetNets. The proposed method involves creating an optimization problem that includes UE admission into a cluster, UE association with a base station (BS), and power allocation supported by H-NOMA, which incorporates OMA and NOMA schemes in the macro base station (MBS) only and heterogeneous network (HetNet) environments. The problem is formulated as a non-linear concave fractional programming (CFP) issue, which is then transformed into a concave optimization, i.e., mixed-integer non-linear programming (MINLP) problem, using the Charnes–Cooper transformation (CCT). The paper employs a two-phase ε-optimal outer approximation algorithm (OAA) to solve the MINLP problem. Simulation results demonstrate that the proposed approach surpasses H-NOMA with MBS only regarding UE admission, UE association, throughput, and EE. Consequently, the paper contributes to creating a novel resource allocation method that maximizes EE using UE-C with downlink H-NOMA-assisted B5G HetNets. The methodology involves formulating and solving an optimization problem using a two-phase ε-optimal outer approximation algorithm. The key challenges outlined in this paper involve applying the proposed algorithm to other multiple access schemes, such as orthogonal multiple access (OMA) or other variations of NOMA, and comparing its performance in terms of power efficiency, QoS satisfaction, and harvested energy. Additionally, there is a need to explore the performance of the proposed integrated resource allocation algorithm in more complex network scenarios.

Zhu et al. [10] examine the application of intelligent reflecting surfaces (IRSs) in both ground and aerial vehicular networks. IRSs have the ability to reflect signals and alter propagation directions in a highly smart and energy-efficient manner, thereby establishing intelligent radio environments. The ease and low cost of IRS deployment make it a promising technology for enhancing signal strength, physical layer security, and positioning accuracy. The paper provides an extensive review of current research on various IRS applications in terrestrial and aerial vehicular communications, following a discussion on its fundamental knowledge and key characteristics. The paper also identifies the challenges and discusses the future perspectives of IRS-assisted 6G vehicular communications to enhance current works and inspire more innovative ideas. Consequently, the paper’s contribution lies in providing a thorough survey of the application of IRS in terrestrial and aerial vehicular communications in the 6G era. The methodology involves discussing the basic knowledge and main characteristics of IRS and summarizing current research works while identifying challenges and future perspectives. The result is an improved understanding of the potential of IRS-aided 6G vehicular communications. The main challenges presented in this article involve investigating the performance of the proposed joint optimization algorithm in more complex network scenarios, such as multi-UAV or heterogeneous networks, and evaluating its scalability and adaptability. Also, they need to examine the caching scheme for reducing transmission delay in the network.

Ji et al. [32] proposed the beamforming control and trajectory design algorithm based on a multi-pass deep Q-network (BT-MP-DQN). The UAV plays the role of the agent in this model, periodically monitoring the state of the multicast network and adapting to dynamic environmental changes. Two types of actions are incorporated in the system: discrete actions related to the UAV’s movement and continuous actions associated with the beamforming design. The sum rate maximization problem was tackled through the simultaneous design of the UAV’s movement, the RIS reflection matrix, and the UAV-to-user beamforming. Simulation results demonstrate the effectiveness of the proposed algorithm. Furthermore, the UAV-enabled multicast network outperformed traditional multicast channels with static transmitters, highlighting the efficacy of the proposed mobile UAV transmitter model. The main challenges presented in this article involve incorporating other practical considerations, such as user mobility, latency requirements, or dynamic network conditions, into the resource allocation problem for UAV-enabled NOMA networks. Also, they need to examine the caching scheme for reducing transmission delay in the network.

Gkonis et al. [33] proposed a transmission scheme utilizing non-orthogonal multiple access (NOMA) and code reuse. This approach constructs a correlation matrix of the received data at the transmitter, with feedback as only the primary eigenvector of the equivalent channel matrix, which is derived after principal component analysis (PCA) at the receiver. Users with improved channel quality and reduced multiple access interference are chosen as potential candidates for their assigned code to be reused. According to the results, their proposed approach can achieve a nearly 33% code assignment gain (CAG) when successive interference cancellation (SIC) is employed in mobile receivers. Even without SIC, a tolerable average bit error rate (BER) degradation can still maintain the CAG. Future work intends to extend this algorithm to 5G multicellular configurations, incorporating additional selection criteria such as requested service, intercell interference, and handover rate. A comprehensive summary of recent related works is presented in

Table 1. Summary of recent related works.

Refs.	Objectives	Limitations
Wang et al. [26]	Evaluate NOMA downlink and uplink performance in dense networks Assess impact of density, network size, power distribution on performance Validate theoretical findings through simulations Comparative analysis of NOMA and OMA in dense deployments Determine effectiveness of NOMA for dense wireless networks	Overly idealized SIC assumption that may not match real-world conditions Lack of evaluation in complex network scenarios restricts conclusions on algorithm robustness and scalability
Zeng and Zhang [27]	Optimize UAV path to improve wireless communication energy efficiency Propose SCO-based algorithm to iteratively optimize trajectory and power Validate the proposed approach through simulations Develop efficient UAV path planning technique for energy-efficient wireless coverage	Lack of real-world considerations in trajectory optimization reduces applicability No exploration of ML/AI for enhancing system efficiency and adaptability Long time computation Neglects UAV battery consumption Neglects UAV caching scheme
Zhang et al. [28]	Jointly optimize UAV trajectory and power allocation Maximize sum rate of UAV-relayed NOMA network Account for constraints like UAV speed, altitude, power Propose SCP and alternating optimization-based algorithm Develop efficient joint trajectory and power optimization for UAV NOMA networks	Long time computation Neglects UAV battery consumption Neglects UAV caching scheme No exploration of ML/AI for enhancing system efficiency and adaptability
Liu et al. [30]	Propose BCD and SCP-based algorithm Jointly optimize UAV trajectory and power allocation Develop efficient joint optimization for trajectory and power in UAV NOMA networks Maximize sum rate of UAV-assisted NOMA network	High complexity Ignores processing time Neglects UAV battery consumption Neglects UAV caching scheme
Ghafoor et al. [31]	Optimize energy efficiency in B5G HetNets Propose UE clustering with downlink H-NOMA Formulate joint optimization problem for UE admission, BS association, power allocation Transform non-convex issue into concave optimization using CCT Develop efficient joint EE optimization approach for next-generation HetNets	Lack of comparison to other access schemes limits understanding of performance tradeoffs No evaluation in complex network settings restricts conclusions on real-world viability Ignores sum of rate
Ji et al. [32]	Propose BT-MP-DQN algorithm for UAV multicast networks UAV as agent to monitor state and adapt to dynamics Incorporate discrete actions for movement and continuous actions for beamforming Develop adaptive deep reinforcement learning framework for joint UAV trajectory and beamforming optimization	No caching scheme explored for reducing delay and enhancing performance High complexity Ignores processing time Neglects UAV battery consumption Ignores sum of rate
Nasir et al. [34]	Optimize total power and BW Maximize bandwidth	Ignores processing time Neglects UAV battery consumption No caching scheme explored for reducing delay and enhancing performance
Sun et al. [35]	UAV optimization Optimize trajectory planning and maximize sum of rate	No caching scheme explored for reducing delay and enhancing performance Neglects UAV battery consumption
Chang et al. [36]	Boost the energy efficiency BS-assisted communication	Small cell network Long time computation No caching scheme explored for reducing delay and enhancing performance Ignores sum of rate

.

3. Methdology and Proposed Model

Figure 2 shows a proposed 6G network model with a ground base station and multiple aerial stations servicing land-based users. We have assumed for this model that the drones dynamically navigate above the dispersed users, storing popular content for easy access. This strategy is a potential way to reduce delays and lessen the load on the primary data link. It is also important to note that in our network diagram, a drone moves at a restricted speed and has set limits on its communication and storage abilities. The ground control center is referred to as the ground base station (GBS), while the UAVs are collectively referred to as U. Each UAV serves a group of users, denoted as S, with Su representing the total user base in our network. It is also important to note that the UAVs have a defined total working duration, T, which is subdivided into multiple time slots, t. A comprehensive list of these system model parameters can be found in

Table 2. Parameters of the system model.

Notation	Description
$S$	The set of all users
$S_1 and S_2$	User 1 and user 2
$U$	The set of all UAVs
$u$	The UAV
$GBS$	Ground base station
$t$	$\mathrm{Time} \mathrm{slot} t\in \left[0,T\right]$
$T$	Total operational time
${\alpha }_1,{\alpha }_2,\dots {\alpha }_n$	Fractional coefficient of the transmitted power
$W$	Total transmitted power
$f$	Rayleigh fading channel
${TP}_{u,GBS}$	Transmit power of UAV u to ground terminal g
${SNR}_{u,\mathrm{g}}$	The signal to noise ratio between UAV u and user S or to ground terminal g
$P_{u,g\left(los\right)}$	The probability of LoS connection
$P_{u,g\left(nlos\right)}$	The probability of NLoS connection
µ	The LoS and NLoS attenuation factors
${ch}_{u,s}^c$	Cache hit indicator variable for the home UAV
${ch}_{u^{\prime },s}^c$	Cache hit indicator variable at the foreign UAV (u′).
$d_{u,s}^c$	The transmission delay of content (c) for user (s), originating from home UAV u
$d_{u^{\prime },s}^c$	The transmission delay of content (c) for user (s), originating from foreign UAV u
$\omega \left(n\right|{\rho }^{\omega })$	Actor network parameters (256,relu, sigmoid)
$\gamma (n,x|{\rho }^{\gamma })$	Critic network parameters (256,relu, linear)
Batch size (BZ)	256
Memory size (MZ)	106

. Following this, we will outline our achievable rate, power allocation vs. number of users, mobility, transmission, and caching models.

3.1. The Achievable Rate

In the context of the downlink NOMA accommodating N users, we can conceptualize user 1 (S1) as being positioned at the maximum distance from the UAV, while user n (Sn) is located closest to the UAV. The principle of NOMA stipulates that the user who is most distant from the UAV is allocated a greater power level, whereas the user in closest proximity to the UAV receives a correspondingly lower power allotment. Let ${\mathit{S}}_{\mathbf{1}},{\mathit{S}}_{\mathbf{2}},\dots {\mathit{S}}_{\mathit{N}}$ denote the messages to be transmitted to the users. Hence, the NOMA signal can be expressed as:

$$S_{NOMA}=\sqrt{W}\left(\sqrt{{\alpha }_1}{S}_1+\sqrt{{\alpha }_2}{S}_2+\cdots +\sqrt{{\alpha }_N}{S}_N\right)$$

(1)

where W is the total transmitted power and α₁, α₂, up to αn are the fractional coefficients of the transmitted power. Also, α1 must be greater than α₂ up to α_N.

$${\alpha }_1>{\alpha }_2>\cdots >{\alpha }_N,{{\alpha }_1>{\alpha }_2+{\alpha }_3+\cdots +{\alpha }_N,\alpha }_1+{\alpha }_2+\cdots +{\alpha }_n=1$$

Equation (1) can be re-expressed as:

$$S_{NOMA}=\sqrt{W}\sum _{i=1}^N\sqrt{{\alpha }_i}{S}_i$$

(2)

The received signal at the i-th user can be expressed as:

$$S_{i,NOMA}=f_i{S}_{NOMA}+N_i$$

(3)

where ${\mathit{N}}_{\mathit{i}}$ is the AWGN vector of zero mean and variance, f_i is the Rayleighfading channel, and ${\mathit{S}}_{\mathit{N}\mathit{O}\mathit{M}\mathit{A}}$ is the NOMA signal using Equations (1) and (2). Therefore, Equation (3) can be re-expressed as:

$$S_{i,NOMA}=f_i\sqrt{W}\left(\sqrt{{\alpha }_1}{s}_1+\sqrt{{\alpha }_2}{s}_2+\cdots +\sqrt{{\alpha }_N}{s}_N\right)+N_i$$

(4)

At the first (far) user, the received vector is given as:

$$s_{1,NOMA}=f_1\sqrt{w}\left(\sqrt{{\alpha }_1}{S}_1+\sqrt{{\alpha }_2}{S}_2+\cdots +\sqrt{{\alpha }_N}{S}_N\right)+N_1$$

(5)

where $\sqrt{{\mathit{\alpha }}_{\mathbf{1}}}{\mathit{s}}_{\mathbf{1}}$ is the desired term and dominating term for the first or the fastest user. These terms $\sqrt{{\mathit{\alpha }}_{\mathbf{2}}}{\mathit{s}}_{\mathbf{2}}+\cdots +\sqrt{{\mathit{\alpha }}_{\mathit{N}}}{\mathit{s}}_{\mathit{N}}$ are interference with lower amount of power. ${\mathit{N}}_{\mathbf{1}}$ is the noise. Now, direct decoding is performed to recover or estimate the ${\mathit{s}}_{\mathbf{1}}$ signal, which corresponds to the farthest user or the first user, which is assigned to a higher amount of power. The signal-to-interference-to-noise ratio (SINR) for decoding the 1st user signal is given by:

$${\gamma }_{1,NOMA}=\frac{{\alpha }_{1}W{\left|f_1\right|}^2}{{\alpha }_{2}W{\left|f_1\right|}^2+{\alpha }_{3}W{\left|f_1\right|}^2+\cdots +{\alpha }_{N}W{\left|f_1\right|}^2+{\sigma }^2}$$

(6)

At the second user, the received vector is given as:

$$s_{2,NOMA}=f_2\sqrt{W}\left(\sqrt{{\alpha }_1}{S}_1+\sqrt{{\alpha }_2}{S}_2+\cdots +\sqrt{{\alpha }_N}{S}_N\right)+N_2$$

(7)

In order to retrieve the data stream vector of the second user, denoted as ${\mathit{s}}_{\mathbf{2}}$, we must initially decode the data stream vector of the first user, identified as ${\mathit{s}}_{\mathbf{1}}$. This process involves taking ${\mathit{s}}_{\mathbf{1}}$, multiplying it by the square root of α₁, and then again by the square root of P as well as f2. The resulting value is then subtracted from ${\mathit{s}}_{\mathbf{2}}$, as per the NOMA principle. This methodology is referred to as the successive interference cancellation process (SIC). Following this, we proceed with the direct decoding of the data stream vector pertaining to the second user. The concept of the SIC is applied to obtain:

$$s_{2,NOMA}^{\prime }=f_2\sqrt{W}\left(\sqrt{{\alpha }_2}{S}_2+\cdots +\sqrt{{\alpha }_N}{S}_N\right)+N_2$$

(8)

Now, direct decoding is performed to estimate ${\mathit{s}}_{\mathbf{2}}$. The SINR for decoding the 2nd user signal is given by:

$${\gamma }_{2,NOMA}=\frac{{\alpha }_{2}W{\left|f_2\right|}^2}{{\alpha }_{3}W{\left|f_2\right|}^2+{\alpha }_{4}W{\left|f_2\right|}^2+\cdots +{\alpha }_{N}W{\left|f_2\right|}^2+{\sigma }^2}$$

(9)

The same manner is employed up to the Nth user. In general, the i-th SINR is given by:

$${\gamma }_{i,NOMA}=\frac{{\alpha }_{i}W{\left|f_i\right|}^2}{{\alpha }_{i+1}W{\left|f_i\right|}^2+\cdots +{\alpha }_{N}W{\left|f_i\right|}^2+{\sigma }^2}$$

(10)

$${\gamma }_{i,NOMA}=\frac{{\alpha }_{i}W{\left|f_i\right|}^2}{W{\left|f_i\right|}^2{\sum }_{j=i+1}^N{\alpha }_j+{\sigma }^2}$$

(11)

Using the Shannon capacity formula, the achievable rate (bps/Hz) of the i-th user can be given as:

$$R_{i,NOMA}={\mathrm{l}\mathrm{o}\mathrm{g}}_2\left(1+{\gamma }_{i,NOMA}\right)$$

(12)

The sum rate of all the NOMA users is given by:

$$R_{NOMA}=\sum _{i=1}^N{R}_{i,NOMA}$$

(13)

Therefore, Equation (13) can be re-expressed as:

$$R_{NOMA}=\sum _{i=1}^{N-1}{\mathrm{l}\mathrm{o}\mathrm{g}}_2\left(1+\frac{{\alpha }_{i}W{\left|f_i\right|}^2}{W{\left|f_i\right|}^2{\sum }_{j=i+1}^N{\alpha }_j+{\sigma }^2}\right)+{\mathrm{l}\mathrm{o}\mathrm{g}}_2\left(1+\frac{{\alpha }_{N}W{\left|f_N\right|}^2}{{\sigma }^2}\right)$$

(14)

At a high SINR and $\left|{\mathit{f}}_{\mathbf{1}}\right|=\left|{\mathit{f}}_{\mathbf{2}}\right|=\cdots =\left|{\mathit{f}}_{\mathit{n}}\right|,$ Equation (14) becomes:

$$R_{NOMA}\approx {\mathrm{l}\mathrm{o}\mathrm{g}}_2\left(\frac{W{\left|f_n\right|}^2}{{\sigma }^2}\right)$$

(15)

At the receiver side, the UAV implements the SIC. The near user signal is decoded first. The SINR for decoding the i-th user signal is given by:

$${\gamma }_{i,NOMA}=\frac{{\alpha }_{i}W{\left|f_i\right|}^2}{W{\sum }_{\begin{array}{c}j=1\\ j\ne i\end{array}}^N{\alpha }_j{\left|f_j\right|}^2+{\sigma }^2}$$

(16)

The SINR for decoding the 1st user signal is given by:

$${\gamma }_{1,NOMA}=\frac{{\alpha }_{1}P{\left|h_1\right|}^2}{{\sigma }^2}$$

(17)

The achievable rate (bps/Hz) of the i-th user can be given as:

$$R_{i,NOMA}={\mathrm{l}\mathrm{o}\mathrm{g}}_2\left(1+{\gamma }_{i,NOMA}\right)$$

(18)

Therefore, Equation (19) can be re-expressed as:

$$R_{NOMA}=\sum _{i=2}^N{\mathrm{l}\mathrm{o}\mathrm{g}}_2\left(1+\frac{{\alpha }_{i}W{\left|f_i\right|}^2}{W{\sum }_{j=1}^{i-1}{\alpha }_j{\left|f_j\right|}^2+{\sigma }^2}\right)+{\mathrm{l}\mathrm{o}\mathrm{g}}_2\left(1+\frac{{\alpha }_{1}W{\left|f_1\right|}^2}{{\sigma }^2}\right)$$

(19)

Comparing the NOMA equation with the OMA equation in the case of two users’ uplink, we determine that: ${\mathit{R}}_{\mathit{O}\mathit{M}\mathit{A}}{\le \mathit{R}}_{\mathit{N}\mathit{O}\mathit{M}\mathit{A}}$.

3.2. Power Allocation vs. Number of Users

With the continuous influx of users into the network, the power allotment to the user nearest to the source experiences a gradual decline. This results in a corresponding decrease in the achievable data rate for that user. One possible remedy for this situation is to employ dynamic modulation of the power amplification (PA) coefficients, dictated by the channel state information (CSI) values. The advantages of this strategy encompass an increase in the total data rate, improved energy efficiency, and the assurance of equitable treatment among users. Let us define the capacity of the far (Equation (20)) and near (Equation (21)) users as:

$$R_r={\mathrm{l}\mathrm{o}\mathrm{g}}_2\left(1+\frac{{\mathsf{\alpha }}_{r}W{\left|f_r\right|}^2}{{\mathsf{\alpha }}_{n}W{\left|f_r\right|}^2+{\sigma }^2}\right)$$

(20)

$$R_n={\mathrm{l}\mathrm{o}\mathrm{g}}_2\left(1+\frac{{\mathsf{\alpha }}_{n}W{\left|f_n\right|}^2}{{\sigma }^2}\right)$$

(21)

where:

• ${\mathit{f}}_{\mathit{r}}$: is the Rayleigh fading coefficient for the far user;
• ${\mathit{f}}_{\mathit{n}}$: is the Rayleigh fading coefficient for the near user;
• ${\mathsf{\alpha }}_{\mathit{r}}$: is the PA coefficient for the far user;
• ${\mathsf{\alpha }}_{\mathit{n}}$: is the PA coefficient for the near user;
• $\mathit{W}$: is the total transmitted power;
• ${\mathit{\sigma }}^{\mathbf{2}}$: is the noise power.

$${\mathsf{\alpha }}_r+{\mathsf{\alpha }}_n=1,{\mathsf{\alpha }}_r>{\mathsf{\alpha }}_n$$

Let ${\mathit{R}}^{\mathbf{*}}$ by the target rate of the far user. Our goal is to choose ${\mathsf{\alpha }}_{\mathit{r}} \& {\mathsf{\alpha }}_{\mathit{n}}$ such that ${\mathit{R}}_{\mathit{r}}\ge {\mathit{R}}^{\mathbf{*}}$, using Equation (22) as:

$$R^{\mathrm{*}}={\mathrm{l}\mathrm{o}\mathrm{g}}_2\left(1+\frac{{\mathsf{\alpha }}_{r}W{\left|f_r\right|}^2}{{\mathsf{\alpha }}_{n}W{\left|f_r\right|}^2+{\sigma }^2}\right)$$

(22)

Take ${\mathbf{2}}^{\mathit{x}}$ for both sides as:

$$2^{R^{\mathrm{*}}}=1+\frac{{\mathsf{\alpha }}_{r}W{\left|f_r\right|}^2}{{\mathsf{\alpha }}_{n}W{\left|f_r\right|}^2+{\sigma }^2}$$

(23)

$$\xi =\frac{{\mathsf{\alpha }}_{f}W{\left|f_r\right|}^2}{{\mathsf{\alpha }}_{n}W{\left|f_r\right|}^2+{\sigma }^2}$$

(24)

By substituting about ${\mathsf{\alpha }}_{\mathit{n}}=\mathbf{1}-{\mathsf{\alpha }}_{\mathit{f}}$,

$${\mathsf{\alpha }}_r=\frac{\xi \left(W{\left|f_r\right|}^2+{\sigma }^2\right)}{P{\left|f_r\right|}^2\left(\xi +1\right)}$$

(25)

Also, ${\mathsf{\alpha }}_{\mathit{f}}$ does not exceed 1. Thus,

$${\mathsf{\alpha }}_r=min\left(1,\frac{\xi \left(W{\left|f_r\right|}^2+{\sigma }^2\right)}{W{\left|f_r\right|}^2\left(\xi +1\right)}\right)$$

(26)

3.3. Mobility

One of the defining characteristics of UAVs when functioning as airborne base stations is their mobility. A ground user, labeled as ($\mathit{s}\in \mathit{S}$), is assumed to be randomly positioned at fixed locations during the time slot $\mathit{t}\in \left[0,\mathit{T}\right]$. This can be represented in a 3D Cartesian coordinate system as ${\mathit{E}}_{\mathit{s}}\left(\mathit{t}\right)={\left[{\mathit{x}}_{\mathit{s}}\left(\mathit{t}\right),{\mathit{y}}_{\mathit{s}}\left(\mathit{t}\right)\right]}^{\mathit{T}}$. Furthermore, the location of the UAVs during the time slot t is captured in the same 3D Cartesian coordinate system, denoted as ${\mathit{R}}_{\mathit{u}}\left(\mathit{t}\right)={\left[{\mathit{x}}_{\mathit{u}}\left(\mathit{t}\right),{\mathit{y}}_{\mathit{u}}\left(\mathit{t}\right)\right]}^{\mathit{T}}$. The UAV’s horizontal speed constraint can be defined as $‖{\underset{\mathit{R}}{\to }}_{\mathit{u}}\left(\mathit{t}\right)‖\le \mathbf{max}\mathit{horizontal}\mathit{speed}$. Consequently, the spatial separation between a ground user h and its associated UAV at any given time t can be delineated as ${\mathit{d}}_{\mathit{s},\mathit{u}}\left(\mathit{t}\right)=\sqrt{{‖{\underset{\mathit{R}}{\to }}_{\mathit{u}}\left(\mathit{t}\right)-{\underset{\mathit{E}}{\to }}_{\mathit{s}}\left(\mathit{t}\right)‖}^2}$.

3.4. Transmission

One of the defining characteristics of UAVs when functioning as airborne base stations is their mobility. In our transmission model, we accommodate both air-to-ground and air-to-air channels, the details of which will be elaborated in subsequent sections.

• Air-to-Ground Model

Our proposed model enables two possible routes for airborne-to-terrestrial communication: the first being from the unmanned aerial vehicle (UAV) to the end user, and the second from the UAV to the ground base station (GBS). For the purposes of this model, we represent a terrestrial terminal by the symbol g, which could be either a ground user (s) or a GBS. The quality of the connection between a UAV (denoted u) and a ground terminal (g) is subject to several factors. These include the carrier frequency, the probability of achieving direct (line-of-sight or LoS) and indirect (non-line-of-sight or NLoS) communication, as well as the attenuation, or weakening, of both LoS and NLoS signals. In urban environments, physical obstructions can hinder the connection between a UAV and a ground terminal. Considering the average path loss derived from Equation (28), we can articulate the average signal-to-noise ratio (SNR) linked with a ground terminal (g) in relation to a specific UAV (u) as follows:

$${{PL}^{\prime }}_{u,g}\left(t\right)=P_{u,g\left(los\right)}\times {PL}_{u,g\left(los\right)}+P_{u,g\left(nlos\right)}\times {PL}_{u,g\left(nlos\right)}$$

(27)

where:

• u is a UAV;
• g is a user s ∈ S or the ground base station (GBS);
• ${\mathit{P}}_{\mathit{u},\mathit{g}\left(\mathit{l}\mathit{o}\mathit{s}\right)}$ is the probability of LOS connection;
• ${\mathit{P}}_{\mathit{u},\mathit{g}\left(\mathit{n}\mathit{l}\mathit{o}\mathit{s}\right)}$ is the probability of NLOS connection;
• ${\mathit{P}\mathit{L}}_{\mathit{u},\mathit{g}\left(\mathit{l}\mathit{o}\mathit{s}\right)}$ is the path loss of the LoS channel between the UAV (u) and user (S) or ground base station (GBS);
• ${\mathit{P}\mathit{L}}_{\mathit{u},\mathit{g}\left(\mathit{n}\mathit{l}\mathit{o}\mathit{s}\right)}$ is the path loss of the LoS channel between the UAV (u) and user (S) or ground base station (GBS).

$${SNR}_{u,g}\left(t\right)=\frac{{TP}_{u,g}}{\frac{{10}^{{PL}_{u,g}^{\prime }\left(t\right)}}{{\sigma }^2}}$$

(28)

Here,${\mathit{T}\mathit{P}}_{\mathit{u},\mathit{g}}$ is the transmission power of UAV u to ground terminal g, σ² = Bu,gN0 is the variance of addictive white Gaussian noise (AWGN), Bu,g is the bandwidth used, and N0 is denoted as the power spectral density.

B.

Air-to-Air Model

For communication between UAVs, we utilize the concept of free space path loss. Given the elevated altitude of the UAVs and the unobstructed space between them, this model is particularly apt. We envisage a scenario where a UAV, denoted as u ∈ U, transmits a signal to a neighboring UAV, labeled as u′ ∈ U. As such, the power level at which UAV u ∈ U dispatches the signal to another UAV u ∈ U is modeled as follows:

$${TS}_{u,u^{\prime }}\left(t\right)={RS}_{u^{\prime }}\left(t\right)-{Gn}_u\left(t\right)-{Gn}_{u^{\prime }}\left(t\right)$$

(29)

where${\mathit{T}\mathit{P}}_{\mathit{u},\mathit{u}}$ is the transmitted signal by UAV u, the received signal is ${\mathit{R}\mathit{S}}_{{\mathit{u}}^{\prime }}\left(\mathit{t}\right)$ by UAV u′, and${\mathit{G}\mathit{n}}_{\mathit{u}}\left(\mathit{t}\right)$ and ${\mathit{G}\mathit{n}}_{{\mathit{u}}^{\prime }}\left(\mathit{t}\right)$ are the gains of UAV u ∈ U and UAV u′ ∈ U, respectively.

3.5. Caching

• Cache Hit at home UAV

In an initial case, the content (c) desired by a user (s) is present in their home UAV (u). Consequently, we establish ${\mathit{c}\mathit{h}}_{\mathit{u},\mathit{s}}^{\mathit{c}}$ as a cache hit indicator variable for the home UAV (u). This variable indicates whether the requested content (c) by the user (s) is indeed cached in their home UAV (u).

$${\mathit{c}\mathit{h}}_{\mathit{u},\mathit{s}}^{\mathit{c}}=\left\{\begin{array}{c}\mathbf{1},\mathit{if}\mathit{content}\left(\mathit{c}\right)\mathit{requested}\mathit{by}\mathit{user}\left(\mathit{s}\right)\mathit{is}\mathit{cached}\mathit{at}\mathit{home}\mathit{UAV}\hfill \\ \mathbf{0},\mathit{otherwise}\hfill \end{array}\right.$$

(30)

Furthermore, we can compute the transmission delay for the initial scenario as follows:

$$d_{u,s}^c=\frac{{CS}^c}{{LC}_{u,s}}$$

(31)

where ${\mathit{d}}_{\mathit{u},\mathit{s}}^{\mathit{c}}$ represents the transmission delay of content (c) for user (s), originating from home UAV u. The size of the requested content is represented by ${\mathit{C}\mathit{S}}^{\mathit{c}}$, while ${\mathit{L}\mathit{C}}_{\mathit{u},\mathit{s}}$ denotes the link capacity between the home UAV u (belonging to the set U) and user s (part of the set S).

B

Cache Hit at foreign UAV

In the second situation, the content (c) requested by a user (s) is not stored in their own UAV (u), but it is accessible in a foreign UAV (u′). Therefore, we introduce ${\mathit{c}\mathit{h}}_{{\mathit{u}}^{\prime },\mathit{s}}^{\mathit{c}}$ as an indicator variable of cache hit at the foreign UAV (u′). This variable signifies whether the content (c) sought by the user (s) is cached in the foreign UAV (u′).

$${\mathit{c}\mathit{h}}_{{\mathit{u}}^{\prime },\mathit{s}}^{\mathit{c}}=\left\{\begin{array}{c}\mathbf{1},\mathit{if}\mathit{content}\left(\mathit{c}\right)\mathit{requested}\mathit{by}\mathit{user}\left(\mathit{s}\right)\mathit{is}\mathit{cached}\mathit{at}{\mathit{u}}^{\prime }\hfill \\ \mathbf{0},\mathit{otherwise}\hfill \end{array}\right.$$

(32)

Furthermore, we can compute the transmission delay for the initial scenario as follows:

$$d_{u^{\prime },s}^c=\frac{{CS}^c}{{LC}_{u,s}+{LC}_{u,u^{\prime }} }$$

(33)

where ${\mathit{d}}_{{\mathit{u}}^{\prime },\mathit{s}}^{\mathit{c}}$ represents the transmission delay of content (c) for user (s), originating from foreign UAV u. The size of the requested content is represented by ${\mathit{C}\mathit{S}}^{\mathit{c}}$, while ${\mathit{L}\mathit{C}}_{\mathit{u},\mathit{s}}$ denotes the link capacity between the home UAV u (belonging to the set U) and user s (part of the set S). ${\mathit{L}\mathit{C}}_{\mathit{u},{\mathit{u}}^{\prime }}$ is the link capacity between home u and foreign UAV u′.

C

Cache Miss at home and foreign UAV

In a third situation, we posit that the content (c) a user (s) is seeking is not stored in either their home UAV (u) or the foreign UAV (u′). As a result, the requested content is retrieved directly from the ground base station (GBS). The reward function (RW) is designed such that it is inversely related to the total transmission delay necessary for downloading the content requested by any user h, who is part of the set H.

D

Cache sharing to reduce transmission delay and better management of power consumption

In our manuscript, we propose a deep reinforcement learning (DRL) algorithm to minimize the transmission delay of user-requested content. This problem involves several factors, including user association ($\tau$), communication rates between different entities (UAVs, users, and ground base stations), and cache availability in both home and foreign UAVs. In our model, the agent is a UAV that interacts with its environment to learn the optimal policy for maximizing its cumulative reward. The environment encapsulates the UAV’s mobility state, its distance to associated users (affecting channel rates and delay calculation), and cache availability in home and foreign UAVs. At each time slot, the agent selects an optimal action, represented by a tuple ($\tau$*, ${\mathit{c}\mathit{h}}_{\mathit{u},\mathit{s}}^{\mathit{c}},$ and ${\mathit{T}\mathit{P}}_{\mathit{u},\mathit{g}}$), based on its environment observation. Considering the large number of states in our model, we combine DRL with a deep neural network (DNN) to manage this vast state space. The environment provides the agent with four types of information at each time slot: delays between the user and the home UAV (${\mathit{d}}_{\mathit{u},\mathit{s}}^{\mathit{c}}$), the foreign UAV (${\mathit{d}}_{{\mathit{u}}^{\prime },\mathit{s}}^{\mathit{c}}$), and the ground base station (${\mathit{d}}_{\mathbf{G}\mathbf{B}\mathbf{S},\mathit{s}}^{\mathit{c}}$), as well as the agent UAV’s horizontal speed. Our reinforcement learning approach functions on a stationary control policy, where the agent UAV makes decisions such as user association, caching, and power at every time slot based on specific conditions. Given the continuous nature of our problem’s state and action spaces, owing to the highly dynamic environment and mobile nature of UAVs and users, we adopt the deep deterministic policy gradient (DDPG) approach. This actor–critic-based algorithm is particularly suited for continuous domains. The DDPG employs two deep neural networks: the actor network and the critic network. The actor network maps the state to an action, establishing the policy, while the critic network evaluates the quality of this policy by critiquing the actions produced by the actor network. The parameters of both networks are updated at every time slot, enabling the agent to continuously adapt to changes in the environment.

Algorithm 2 proposes a deep reinforcement learning algorithm to minimize delays in UAV-based cache sharing. It initializes experience replay memory, mini-batches, and critic and actor networks. In each episode, it resets the environment, takes actions based on the actor network, stores transitions, samples experiences, and updates network parameters. Exploration noise is added to actions and reduced over time. The episode terminates when the requested content is fully downloaded. Subsequent simulations will evaluate the algorithm’s effectiveness in minimizing delay.

Algorithm 2 outlines our proposed deep reinforcement learning (DRL)-based cache-sharing algorithm aimed at minimizing delay. The algorithm initiates by setting up the experience replay memory buffer of size ‘MZ’ and a mini batch of size ‘BZ’. It also establishes the critic and target critic network with weights as well as the actor and target actor network with weights ${\rho }^{\omega },{\rho }^{{\omega }^{\prime }},{\rho }^{\gamma },\mathrm{a}\mathrm{n}\mathrm{d} {\rho }^{{\gamma }^{\prime }}.$ The ε-greedy probability is initialized with εMa and εMin values. At the start of each episode, the environment is reset, providing the initial state. The actor network then takes this state as input, which leads to an action ‘j(t)’. This action is performed by the agent, leading to the next state ‘n(t + 1)’ and yielding a reward ‘RW(t)’. The transition tuple (consisting of n(t), j(t), RW(t), and n(t + 1)) is then stored in the memory buffer ‘MZ’. Once the memory buffer has gathered sufficient experience, the algorithm randomly samples from it. The parameters of the actor, target actor, critic, and target critic networks are subsequently updated to new weights, and the algorithm proceeds to the next time slot ‘t’. An exploration noise ‘Γ’ is added to the actions, which reduces over time. An episode reaches its terminal state when the requested content of a user ‘s’ is fully downloaded.

Algorithm 2: Cache Sharing Algorithm

Input: initialize BZ = 256; initialize MZ = 106; initialize $\omega \left(n\right|{\rho }^{\omega })$ as actor network parameters; $\mathrm{i}\mathrm{n}\mathrm{i}\mathrm{t}\mathrm{i}\mathrm{a}\mathrm{l}\mathrm{i}\mathrm{z}\mathrm{e} \gamma (n,\mathrm{j}|{\rho }^{\gamma })$ as critic network parameters; randomly initialize weights ${\rho }^{\omega },{\rho }^{{\omega }^{\prime }},{\rho }^{\gamma },{\rho }^{{\gamma }^{\prime }}$; greedy probability ${\epsilon }^{max},{\epsilon }^{min}$; initialize the experience cache space Γ. Start Learning   for episode from 1 to limit do           Initialize a noise function and receive the initial state           for t = 1 to T do             Observe n(t) & take action j(t) = $\omega \left(n\right|{\rho }^{\omega })$ + Γ             Apply j(t), obtain n(t + 1), and calculate RW(t)             Save (n(t), j(t), RW(t), n(t + 1)) in MZ             Randomly sample mini batch             Update ${\rho }^{\omega },{\rho }^{{\omega }^{\prime }},{\rho }^{\gamma },{\rho }^{{\gamma }^{\prime }}$             Γ = ${\epsilon }^{min}$ + (${\epsilon }^{max}$ − ${\epsilon }^{min}$)/exp(−ϵ t);             until maximum iterations;     End for End Learning

4. Experimental Results

We performed a simulation analysis to evaluate our proposed model within a service region spanning 500 m. In this setup, a GBS is linked to a content center, and UAVs equipped with caching capabilities cater to terrestrial users. The locations of both the UAVs and users were randomly determined using a homogeneous Poisson point process (HPPP) within the designated service area. The specific parameters related to the system used in this simulation are detailed in

Table 3. Simulation parameters.

Notation	Description	Value
$u\in U$	Number of UAVs	4
${ALT}_u$	Altitude of UAV	180 m
${PL}_{los}$	Path loss for LoS	3
${PL}_{nlos}$	Path loss for NLoS	80
$TP$	Transmit power of UAV	Range 20–40 dBm
$BW$	Bandwidth	30 MHz
$Max Speed$	Maximum speed of UAVs	30 m/s
${\sigma }^2$	Variance of the Gaussian noise	−174 dBm/Hz
${Cache Size}_c$	Cache size	30 MB
$s\in S$	Number of users	20
$f_c$	Carrier frequency	2 GHz

.

4.1. Achievable Rate vs. SNR Results

Figure 3 shows the achievable rate bps/Hz versus a downlink system’s SNR for NOMA and OMA. Also, Figure 4 shows the achievable rate bps/Hz versus an uplink system’s SNR for NOMA and OMA.

4.2. Achievable Rate vs. Transmit Power Results

Figure 5 shows the achievable rate versus the transmit power for downlink NOMA in scenarios of imperfect SIC.

4.3. Sum Rate vs. Number of Users Results

Figure 6 shows the sum rate versus the number of NOMA and OMA systems users at different transmitted power values, which are the dash lines for OMA and the solid lines for NOMA.

4.4. Sum Rate vs. SNR Results

Figure 7 shows the achievable sum rate bps versus the SNR for different pairing methods with respect to TDMA and single carrier NOMA. The hybrid NOMA near–far pairing out forms all schemes with SNRs greater than the typical 32 dB.

4.5. Achievable Capacity vs. Transmitted Power Results

Figure 8 shows the achievable capacity (bits/Hz) versus the transmitted power in dBm for the first and second users. In the case of two users, the signal transmitted by the UAV can be expressed as:

$$x=\sqrt{P}\left(\sqrt{{\alpha }_1}{x}_1+\sqrt{{\alpha }_2}{x}_2\right)$$

(34)

where $\mathit{P}$ is the transmit power, ${\mathit{\alpha }}_{\mathbf{1}}>{\mathit{\alpha }}_{\mathbf{2}}$, ${\mathit{\alpha }}_{\mathbf{1}}+{\mathit{\alpha }}_{\mathbf{2}}=\mathbf{1}$.

At the first (far) user, the received vector is given as:

$$y_1=h_1\sqrt{P}\left(\sqrt{{\alpha }_1}{x}_1+\sqrt{{\alpha }_2}{x}_2\right)+w_1$$

(35)

$$y_1=h_1\sqrt{P}\sqrt{{\alpha }_1}{x}_1+h_1\sqrt{P}\sqrt{{\alpha }_2}{x}_2+w_1$$

(36)

Now, direct decoding is performed to estimate ${\mathit{x}}_{\mathbf{1}}$.

The SINR for decoding the 1st (far) user signal is given by:

$${\gamma }_1=\frac{{\alpha }_{1}P{\left|h_1\right|}^2}{{\alpha }_{2}P{\left|h_1\right|}^2+{\sigma }^2}$$

(37)

The achievable rate (bps/Hz) of the far user can be given as:

$$R_1={\mathrm{l}\mathrm{o}\mathrm{g}}_2\left(1+\frac{{\alpha }_{1}P{\left|h_1\right|}^2}{{\alpha }_{2}P{\left|h_1\right|}^2+{\sigma }^2}\right)$$

(38)

At the second (near) user, the received vector is given as:

$$y_2=h_2\sqrt{P}\sqrt{{\alpha }_1}{x}_1+h_2\sqrt{P}\sqrt{{\alpha }_2}{x}_2+w_2$$

(39)

Firstly, direct decoding for the ${\mathit{x}}_{\mathbf{1}}$ signal is performed, then the concept of the SIC is applied as:

$$y_2^{\prime }=h_2\sqrt{P}\sqrt{{\alpha }_1}{x}_1+h_2\sqrt{P}\sqrt{{\alpha }_2}{x}_2+w_2-h_2\sqrt{P}\sqrt{{\alpha }_1}{\widehat{x}}_1$$

(40)

Now, direct decoding for the near user signal ${\mathit{x}}_{\mathbf{2}}$ is performed.

The SINR for decoding user 1 signal at user 2 is given by:

$${\gamma }_{\mathrm{1,2}}=\frac{{\alpha }_{1}P{\left|h_2\right|}^2}{{\alpha }_{2}P{\left|h_2\right|}^2+{\sigma }^2}$$

(41)

The corresponding achievable rate (bps/Hz) is given as:

$$R_{\mathrm{1,2}}={\mathrm{l}\mathrm{o}\mathrm{g}}_2\left(1+\frac{{\alpha }_{1}P{\left|h_2\right|}^2}{{\alpha }_{2}P{\left|h_2\right|}^2+{\sigma }^2}\right)$$

(42)

After the cancellation of the user 1’s signal, the SINR at user 2 for decoding its own signal is:

$${\gamma }_2=\frac{{\alpha }_{2}P{\left|h_2\right|}^2}{{\sigma }^2}$$

(43)

The corresponding achievable rate (bps/Hz) is given as:

$$R_2={\mathrm{l}\mathrm{o}\mathrm{g}}_2\left(1+\frac{{\alpha }_{2}P{\left|h_2\right|}^2}{{\sigma }^2}\right)$$

(44)

4.6. BER vs. Transmitted Power Results

Figure 9 shows the Bit Error Rate (BER) relative to the transmitted power for far and near users.

4.7. Caching Results

Figure 10, Figure 11 and Figure 12 illustrate the performance of the system as it relates to the number of episodes. Figure 10, Figure 11 and Figure 12 depict the proportions of cache hits corresponding to the content requested by users from their home UAVs, neighboring UAVs, and base station requests, respectively. Figure 13 shows that the average delay at the start of the simulations was 2.6 s, which was directly proportional to the cache miss rate in the home and neighbor UAVs.

4.8. Discussion

In the case of the downlink system model, the achievable rate capacity of the orthogonal multiple access system equals mainly the rate of the non-orthogonal multiple access system at a high SNR and the same channel coefficients for all the users. In the case of the uplink system model, we determine that the achievable rate capacity of the orthogonal multiple access system is less than or equal to the achievable rate capacity of the non-orthogonal multiple access system at a high SNR and the same channel coefficients for all the users. Figure 3 shows the achievable rate bps/Hz versus a downlink system’s SNR for NOMA and OMA. Also, Figure 4 shows the achievable rate bps/Hz versus an uplink system’s SNR for NOMA and OMA. In Figure 5, the achievable rate capacity bps/Hz vs. the transmitted power in dBm at different values of ε, for ε = 0, is the case of perfect successive interference cancellation; as the value of ε is increased, the achievable rate is decreased. Figure 6 shows the sum rate versus the number of NOMA and OMA systems users at different transmitted power values, which are the dashed lines for OMA and the solid lines for NOMA. Figure 7 shows the achievable sum rate bps versus the SNR for different pairing methods with respect to TDMA and single-carrier NOMA. The hybrid NOMA near–far pairing out forms all schemes with SNRs greater than the typical 32 dB. Figure 8 shows the achievable capacity (bits/Hz) versus the transmitted power in dBm for the first and second users. Furthermore, it shows the rate for decoding the first user signal at the second user. Figure 10, Figure 11 and Figure 12 showed the provided insights into the system’s performance over multiple episodes. They show the percentages of cache hits for users’ requested content in their home UAVs, foreign UAVs, and GBS requests (Figure 10, Figure 11 and Figure 12, respectively). The results demonstrate that at the beginning of the simulations, the cache hit rate in the home UAVs was 16.4%, with 9.9% in the foreign UAVs, and the remaining 73.5% of requests were fulfilled via the base station. However, over time, the home and foreign UAVs learned to cache new content, increasing the cache hit rate to 20.5% and 14.9%, respectively. This improvement in the caching strategy reduced the average request from the base station, which decreased to 64.5. There was a direct correlation between the initial average delay of 2.6 s in the simulations and the cache miss rate in both the home and neighbor UAVs, as depicted in Figure 13. As the cache miss rate decreased, the average delay proportionately fell to 2.3 s. The total average rate declined from an initial 7.3 Mbps to 6.8 Mbps in tandem with the escalating cache hit rate in the UAVs. In essence, the reductions in cache miss rate, average delay, and total average rate were intricately intertwined, while the cache hit rate and total average rate shared an inverse relationship. These findings suggest that the proposed caching strategy using UAVs effectively improves cache hit rate over time, leading to a reduced workload on the base station and more efficient use of network resources.

5. Conclusions

The paper presents a promising model to improve communication systems’ performances using NOMA for network-enabled UAVs as aerial base stations for ground users. The proposed model offers several advantages, including serving numerous users at once from a single source, distributing power based on users’ fairness and QoS expectations, and eliminating the need for scheduling in NOMA. The caching strategy using UAVs is also shown to be effective in improving the cache hit rate over time, which in turn leads to a reduction in the average request from the base station. This approach can reduce transmission delays and improve overall network performance. The results of the study demonstrate that the achievable rate capacity of the OMA system is less than or equal to that of the NOMA system at high SNR and the same channel coefficients for all the users. This suggests that NOMA may be a more effective multiple-access method in certain scenarios.