Joint Clustering and Resource Allocation Optimization in Ultra-Dense Networks with Multiple Drones as Small Cells Using Game Theory ^†

Abstract

In this study, we consider the combination of clustering and resource allocation based on game theory in ultra-dense networks that consist of multiple macrocells using massive multiple-input multiple-output and a vast number of randomly distributed drones serving as small-cell base stations. In particular, to mitigate the intercell interference, we propose a coalition game for clustering small cells, with the utility function being the ratio of signal to interference. Then, the optimization problem of resource allocation is divided into two subproblems: subchannel allocation and power allocation. We use the Hungarian method, which is efficient for solving binary optimization problems, to assign the subchannels to users in each cluster of small cells. Additionally, a centralized algorithm with low computational complexity and a distributed algorithm based on the Stackelberg game are provided to maximize the network energy efficiency (EE). The numerical results demonstrate that the game-based method outperforms the centralized method in terms of execution time in small cells and is better than traditional clustering in terms of EE.

1. Introduction

The global mobile data traffic is witnessing an exponential increase, and the number of global mobile connected devices is forecast to increase rapidly from 8.8 billion in 2018 to 13.1 billion in 2023 [1]. The deployment of ultra-dense networks (UDNs) as a type of cellular network is an extremely promising technology for supporting the huge number of connections, especially in the applications of massive Internet of Things (IoT). In comparison with present network technologies, UDNs help to considerably improve the network capacity, spectral efficiency, as well as coverage with a high quality-of-service (QoS) by reducing the distance between the base stations (BSs) and user equipment (UE) [2,3]. However, the highly dense deployment of small cells in UDNs causes high interference, high power consumption, and high computational complexity, which must be addressed to satisfy the stringent constraints of the beyond fifth-generation networks (B5G) [4].

In UDNs, small cells are smaller than ever, and therefore the distance of neighboring small cells decreases. As a result, the intercell interference is extremely high. In addition, the interference is especially hard to control becaused small cells are randomly distributed. There are several studies on interference mitigation in UDNs. In [5], to cope with co-channel interference, the authors used a conflict graph based on machine learning to form a clustering problem and subchannel allocation (SCA) problem for throughput maximization. Two other user-centric clustering methods, using k-means algorithms and jamming strategies, were proposed in [6,7]. These papers show that the clustering-based approach is an efficient tool for mitigating interference in UDNs. However, the clustering methods in [5,6,7] did not consider the signal to interference plus noise ratio (SINR), which directly influences the data rate. This can lead to a decline in the throughput, which is an important advantage of UDNs compared with conventional networks. In [8], an adaptive clustering method was proposed to maximize the spectral efficiency and the network throughput using two stages: offline and online. Additionally, sum-rate maximization was taken into account as the main objective in a user-centric clustering method, a modified Louvain method, proposed in [9].

Owing to maneuverability, using unmanned aerial vehicles (UAVs) in communications is becoming increasingly popular because the networks can be quickly built and have flexibility. Compared with small terrestrial base stations, drones can easily move to form many different structures of networks to serve users more efficiently. Additionally, the channels between drones and users are usually line-of-sight channels because drones fly at high altitudes where obstacles to the users are less likely. It is also convenient for network operators because small cells can be easily added or removed to increase performance or save cost depending on the number of connections as well as the requirements of users. Thus, in communications, using UAVs as small flying base stations is a promising solution to serve hotspots where there is a massive number of users such as at sports events, concerts, exhibitions, and fairs, or provide mission-critical services, e.g., disaster recovery, traffic congestion, etc. [10,11]. UAVs, as flying base stations, can be widely used in many scenarios such as cognitive radio networks [11], ultra-reliable low-latency communications [12], communications using reconfigurable intelligent surface [13], and caching [14]. Despite the advantages of UAVs compared with terrestrial base stations, they still face many challenges in the attempts at employing UAVs in wireless networks. For example, in terms of deployment, the authors of [15] proposed a deployment method for multiple UAVs with the aim of maximizing both the coverage area and the coverage lifetime, whereas a Q-learning method with the reward being the sum of user data rates was used in [16]. Regarding area throughput and energy management, mixed-integer programming problems were formulated and efficiently solved for establishing the optimal location and action scheduling under the constraints of battery and powered energy in [17,18]. Additionally, the authors of [19] considered improving the spectrum efficiency by optimizing bandwidth allocation, power allocation, and trajectory for one UAV. On the other hand, due to the fast movements and jitters of UAVs, there are quick changes in the channels between the UAVs and ground users [20]. Therefore, real-time computing is crucial in UAV-assisted networks, especially in UDNs with many small cells [11,21].

In UDNs, the complexity of optimal resource allocation problems is very high because massive numbers of small cells and small UEs (SUEs) are randomly and densely deployed in large-scale areas. Game theory (GT), which is a distributed mathematical framework, is efficiently used for decoupling an extremely complex problem into multiple subproblems with lower complexity. Thus, there are several studies that investigated the applications of GT in UDNs. In [22], two noncooperative games (NCGs) were designed for solving the power allocation and user association with constraints on the quality of service (QoS). The proposed combination of backhaul game and access game significantly improved the total rate compared with the hierarchical game. Additionally, in [23], the authors considered small cell networks with nonorthogonal multiple access (NOMA) with the aim of network throughput maximization. The results showed the effectiveness of NOMA and the locally cooperative game in interference mitigation in UDNs. In spite of some improvements in [22,23], the main objective was to maximize the throughput; in UDNs, the energy issue is more critical due to the high power consumption of a massive number of network elements. Energy efficiency (EE), which is defined as the ratio of total data rate to total power consumption, is a useful metric for modeling the trade-off between throughput and energy. However, EE maximization problems are nonconvex optimization problems and much harder than throughput maximization problems or power minimization problems. In [24,25],  two optimization problems (i.e., SCA and power allocation (PA)) were considered with PA problems modeled by the Stackelberg game (SG) in order to maximize the EE. Although there were some considerable improvements in terms of EE, there was a limitation in the applicability of the used model of UDNs, which had only one macro base station (MBS) and one BS in a network. In the previous works, either cooperation or noncooperation between network elements was considered, but these elements have the characteristics of both collaboration and competition,. In detail, small-cell base stations (SBSs) prefer to be in joint clusters for cooperatively transmitting signals to UEs. This leads to a decrease in intracluster interference. On the other hand, the SBSs in different clusters tend to conflict regarding power usage. This means that increasing the power of an SBS in a cluster also increases the interference for SUEs in adjacent clusters. Therefore, there is a lack of a game-based paradigm that has the characteristics of both cooperation and competition of network elements in order to concurrently mitigate interference, maximize the network EE in drone-aided UDNs with multiple MBSs, and handle a massive number of randomly distributed drones.

Motivated by the aforementioned discussion, this paper proposes a mathematical model to jointly design the clustering and resource allocation for maximizing the total network EE in UDNs consisting of multiple MBSs and many small-cell drones (SCDs). Cells served by MBSs are macrocells, and cells served by drones are small cells. We propose a coalition game for clustering SCDs to restrict the intercell interference. The SCDs in each cluster cooperatively transmit the signals to UEs. The SCA methods for two tiers are provided to choose the subchannel with the highest channel gain for each UE. To reduce the computational complexity, centralized and distributed PA methods are proposed in terms of maximizing the EE. The main contributions of our paper are summarized as follows:

• Adjacent SCDs tend to join together and cooperatively serve UEs because they highly interfere with each other. Thus, we form this cooperation with a coalition game, with the utility function being the signal-to-interference ratio (SIR).
• To fully eliminate the intracluster interference, we model the SCA for small cells as a binary optimization problem. Then, the Hungarian method is proposed to obtain the solution. Meanwhile, the SCA for macrocells chooses the subchannel with the highest channel gain without any constraint.
• To reduce the complexity of nonconvex PA optimization problems, we propose an iterative centralized PA algorithm to obtain at least a locally optimal solution in terms of maximizing the EE. Inequalities are used for relaxing the complex objective functions of EE maximization into convex functions that are easy to solve with programming tools.
• The distributed PA algorithm based on SG consists of two subcooperative games with MBSs as leaders and SCDs as followers. This algorithm decouples the very complex PA optimization problems into multiple low-complextiy and convex ones that are extremely useful for solving optimization problems with a huge number of variables.

Compared with the conference version [26], a macrocell layer consisting of multiple MBSs is added to the networks. As such, the interference environment is more complex due to intercell interference. Additionally, the objective function is to minimize the network EE, which is more practical to use energy more efficiently compared with the objective of maximizing the sum-rate in UDNs but is more complex to solve because of the fractional-type function. Moreover, we propose a centralized approach to solve PA as a benchmark and the distributed one using the Stackelberg game, and  multiple simulations with different aspects such as EE, sum rate, and execution time were performed to prove the efficiency of the proposed methods. The rest of this paper is organized as follows: Section 2 presents the system model and optimization problems used in this paper. A small-cell clustering method based on the coalition game is described in Section 3. Then, the SCA for both macrocells and small cells as well as the beamforming method for MBSs are discussed in Section 4. In addition, Section 5 describes both centralized and distributed optimization methods for solving PA problems. Numerical simulations proving the efficiency of our proposed methods are outlined in Section 6. Section 7 provides the conclusions of this study.

2. System Model and Problem Formulation

2.1. System Model

In this paper, we consider the downlink transmission in a two-tier UDN that consists of multiple macrocells and a massive number of UAVs serving as small-cell base stations with highly dense deployments. MBSs and SCDs share the same N orthogonal subchannels with bandwidth $BW$ per subchannel to serve their UEs. The advantage of using multiple orthogonal subchannels is that multiple UEs can be simultaneously served in these subchannels without interference. One typical macrocell with two subchannel is illustrated in Figure 1. In the macrocell tier, each F MBS is equipped with T array antennas to transmit the signal to M MUEs by using massive multiple-input multiple-output (mMIMO) techniques. In the small-cell tier, to serve U UEs, many B SCDs equipped with one omni-directional antenna each are randomly deployed according to the independent homogeneous Poisson point processes (PPP) $\Phi$ with density $\lambda$ in the coverage of MBSs ($B>U$) [24]. When UEs are located in the coverage of one or some SCDs, they are served by SCDs, i.e., SUEs. On the other hand, if the UEs are not in the coverage of any SCD, then the power of SCDs is not high enough to serve these UEs or causes extremely high interference with surrounding SCDs, especially in UDNs. Therefore, they are served by MBSs, i.e., MUEs. We used the main mathematical notation, which is shown in

Table 1. The description of the main mathematical notation.

Notation	Definition
$\mathcal{F}$	$\left\{1,2,\dots ,F\right\}$: The set of MBSs
$\mathcal{B}$	$\left\{1,2,\dots ,B\right\}$: The set of SCDs
$\mathcal{C}$	$\left\{1,2,\dots ,C\right\}$: The set of clusters of SCDs
${\mathcal{C}}_i$	$\left\{1,2,\dots ,C_i\right\}$: The set of SCDs in cluster i
$\mathcal{U}$	$\left\{1,2,\dots ,U\right\}$: The set of SUEs
$\mathcal{M}$	$\left\{1,2,\dots ,M\right\}$: The set of MUEs
${\mathcal{U}}_{{\mathcal{C}}_i}$	$\left\{1,2,\dots ,U_{{\mathcal{C}}_i}\right\}$: The set of SUEs served by cluster ${\mathcal{C}}_i$
${\mathcal{U}}_b$	$\left\{1,2,\dots ,U_b\right\}$: The set of SUEs served by SCD b
${\mathcal{M}}_f$	$\left\{1,2,\dots ,M_f\right\}$: The set of MUEs served by MBS f
$s_{m,f}^{\left(n\right)}$	$\in \mathbb{C}$: The instant desired symbol of MUE m served by MBS f in subchannel n
$p_{m,f}^{\left(n\right)}$	$\mathbb{E}\left\{s_{m,f}^{\left(n\right)}{s}_{m,f}^{\left(n\right)H}\right\}$: The transmit power of MBS f to MUE m in subchannel n
$s_{u,b}^{\left(n\right)}$	$\in \mathbb{C}$: The instant desired symbol of SUE m served by SCD b in subchannel n
$p_{u,b}^{\left(n\right)}$	$\mathbb{E}\left\{s_{u,b}^{\left(n\right)}{s}_{u,b}^{\left(n\right)H}\right\}$: The transmit power of SCD b to SUE u in subchannel n
${\mathit{h}}_{m,f}^{\left(n\right)}$	$\in {\mathbb{C}}^{1\times T}$: The channel response from MBS f to MUE m in subchannel n
$h_{m,b}^{\left(n\right)}$	$\in \mathbb{C}$: The channel response from interference source SCD b to MUE m in subchannel n
$h_{u,b}^{\left(n\right)}$	$\in \mathbb{C}$: The channel response from SCD b to SUE u in subchannel n
${\mathit{h}}_{u,f}^{\left(n\right)}$	$\in {\mathbb{C}}^{1\times T}$: The channel response from interference source MBS f to SUE u in subchannel n
${\mathit{w}}_{m,f}$	$\in {\mathbb{C}}^{T\times 1}$: Beamforming vector at MBS f to serve MUE m
$a_{m,f}^{\left(n\right)}$, $a_{u,{\mathcal{C}}_i}^{\left(n\right)}$	subchannel allocation indicator
$y_{m,f}^{\left(n\right)}$	Received signal at MUE m served by MBS f in subchannel n
$y_{u,{\mathcal{C}}_i}^{\left(n\right)}$	Received signal of SUE u served by cluster ${\mathcal{C}}_i$ in subchannel n
$I_{u,\mathcal{F}}^{\left(n\right)}$	${\displaystyle {\sum }_{f\in \mathcal{F}}{\sum }_{m\in {\mathcal{M}}_f}{a}_{m,f}^{\left(n\right)}{\left|{\mathit{h}}_{u,f}^{\left(n\right)}{\mathit{w}}_{m,f}\right|}^2{p}_{m,f}^{\left(n\right)}}$: The interference power from MBSs
$I_{u,\mathcal{C}\setminus {\mathcal{C}}_i}^{\left(n\right)}$	${\displaystyle {\sum }_{{\mathcal{C}}_j\in \mathcal{C}\setminus {\mathcal{C}}_i}{\sum }_{b^{\prime }\in {\mathcal{C}}_j}{\sum }_{u^{\prime }\in {\mathcal{U}}_{{\mathcal{C}}_j}}{a}_{u^{\prime },{\mathcal{C}}_j}^{\left(n\right)}{\left|h_{u,b^{\prime }}^{\left(n\right)}\right|}^2{p}_{u^{\prime },b^{\prime }}^{\left(n\right)}}$: The interference power from other clusters except cluster ${\mathcal{C}}_i$
$I_{\mathrm{cross}-\mathrm{talk}}^{\left(n\right)}$	${\displaystyle {\sum }_{m^{\prime }\in {\mathcal{M}}_f\setminus m}{a}_{m^{\prime },f}^{\left(n\right)}{\left|{\mathit{h}}_{m,f}^{\left(n\right)}{\mathit{w}}_{m^{\prime },f}\right|}^2{p}_{m^{\prime },f}^{\left(n\right)}}$: The cross-talk interference power from MBS f
$I_{m,\mathcal{F}\setminus f}^{\left(n\right)}$	${\displaystyle {\sum }_{f^{\prime }\in \mathcal{F}\setminus f}{\sum }_{m^{\prime }\in {\mathcal{M}}_{f^{\prime }}}{a}_{m^{\prime },f^{\prime }}^{\left(n\right)}{\left|{\mathit{h}}_{m,f^{\prime }}^{\left(n\right)}{\mathit{w}}_{m^{\prime },f^{\prime }}\right|}^2{p}_{m^{\prime },f^{\prime }}^{\left(n\right)}}$: The interference power from MBSs except MBS f
$I_{m,\mathcal{C}}^{\left(n\right)}$	${\displaystyle {\sum }_{{\mathcal{C}}_i\in \mathcal{C}}{\sum }_{b\in {\mathcal{C}}_i}{\sum }_{u\in {\mathcal{U}}_{{\mathcal{C}}_i}}{a}_{u,{\mathcal{C}}_i}^{\left(n\right)}{\left|h_{m,b}^{\left(n\right)}\right|}^2{p}_{u,b}^{\left(n\right)}}$: The interference power from the clusters of SCDs
$\mathcal{CN}(0,{\sigma }^2)$	A complex Gaussian random variable with zero mean and variance ${\sigma }^2$

, to design tbe optimization problems. At each MBS, because of the mMIMO, the number of antennas of MBS f is much greater than the number of its authorized MUEs ($T\gg M_f$). Each signal $s_{m,f}^{\left(n\right)}$ is multiplied by a beamforming vector ${\mathit{w}}_{m,f}\in {\mathbb{C}}^{T\times 1}$, and then the aggregation of processed signals of MUEs that are served by MBS f is transmitted.

The aggregation ${\mathit{x}}_f^{\left(n\right)}$ at MBS f in subchannel n is the sum of $M_f$ processed signals:

$${\mathit{x}}_f^{\left(n\right)}={\sum }_{m\in {\mathcal{M}}_f}{a}_{m,f}^{\left(n\right)}{\mathit{w}}_{m,f}{s}_{m,f}^{\left(n\right)},$$

(1)

where the binary indicator of SCA is defined as

$$a_{m,f}^{\left(n\right)}=\left\{\begin{array}{cc}1,& \mathrm{if}\mathrm{MUE}\mathrm{m}\mathrm{is}\mathrm{assigned}\mathrm{MBS}\mathrm{f}\mathrm{in}\mathrm{subchannel}n,\\ 0,& \mathrm{otherwise}.\end{array}\right.$$

The received signal $y_{m,f}^{\left(n\right)}$ at MUE m served by MBS f in subchannel n is expressed by

$$\begin{array}{c}{y}_{m,f}^{\left(n\right)}=\underset{\mathrm{desired}\mathrm{signal}}{\underbrace{{\mathit{h}}_{m,f}^{\left(n\right)}{\mathit{w}}_{m,f}{s}_{m,f}^{\left(n\right)}}}+\underset{\mathrm{cross}-\mathrm{talk}\mathrm{interference}}{\underbrace{{\sum }_{m^{\prime }\in {\mathcal{M}}_f\setminus m}{a}_{m^{\prime },f}^{\left(n\right)}{\mathit{h}}_{m,f}^{\left(n\right)}{\mathit{w}}_{m^{\prime },f}{s}_{m^{\prime },f}^{\left(n\right)}}}+\underset{\mathrm{interference}\mathrm{from}\mathrm{other}\mathrm{MBSs}}{\underbrace{{\sum }_{f^{\prime }\in \mathcal{F}\setminus f}{\mathit{h}}_{m,f^{\prime }}^{\left(n\right)}{\mathit{x}}_{f^{\prime }}^{\left(n\right)}}}\hfill \\ \hfill +\underset{\mathrm{interference}\mathrm{from}\mathrm{clusters}}{\underbrace{{\sum }_{{\mathcal{C}}_i\in \mathcal{C}}{\sum }_{b\in {\mathcal{C}}_i}{\sum }_{u\in {\mathcal{U}}_{{\mathcal{C}}_i}}{a}_{u,{\mathcal{C}}_i}^{\left(n\right)}{h}_{m,b}^{\left(n\right)}{s}_{u,b}^{\left(n\right)}}}+n_m,\end{array}$$

(2)

where the aggregation of signals at MBS $f^{\prime }$ is ${\mathit{x}}_{f^{\prime }}^{\left(n\right)}={\sum }_{m^{\prime }\in {\mathcal{M}}_{f^{\prime }}}{a}_{m^{\prime },f^{\prime }}^{\left(n\right)}{\mathit{w}}_{m^{\prime },f^{\prime }}{s}_{m^{\prime },f^{\prime }}^{\left(n\right)}$, and $n_m\sim \mathcal{CN}(0,{\sigma }^2)$ is additive white Gaussian noise (AWGN) at MUE m.

The received signal $y_{u,{\mathcal{C}}_i}^{\left(n\right)}$ of SUE u served by cluster ${\mathcal{C}}_i$ in subchannel n is given by

$$\begin{array}{c}\hfill y_{u,{\mathcal{C}}_i}^{\left(n\right)}=\underset{\mathrm{desired}\mathrm{signal}}{\underbrace{\sum _{b\in {\mathcal{C}}_i}{h}_{u,b}^{\left(n\right)}{s}_{u,b}^{\left(n\right)}}}+\underset{\mathrm{interference}\mathrm{from}\mathrm{MBSs}}{\underbrace{\sum _{f\in \mathcal{F}}{\mathit{h}}_{u,f}^{\left(n\right)}{\mathit{x}}_f^{\left(n\right)}}}+\underset{\mathrm{interference}\mathrm{from}\mathrm{other}\mathrm{clusters}}{\underbrace{\sum _{{\mathcal{C}}_j\in \mathcal{C}\setminus {\mathcal{C}}_i}\sum _{b^{\prime }\in {\mathcal{C}}_j}\sum _{u^{\prime }\in {\mathcal{U}}_{{\mathcal{C}}_j}}{a}_{u^{\prime },{\mathcal{C}}_j}^{\left(n\right)}{h}_{u,b^{\prime }}^{\left(n\right)}{s}_{u^{\prime },b^{\prime }}^{\left(n\right)}}}+n_u,\end{array}$$

(3)

where $n_u\sim \mathcal{CN}(0,{\sigma }^2)$ is the AWGN of SUE u, $h_{u,b}^{\left(n\right)}=g_{u,b}^{\left(n\right)}\sqrt{{\beta }_{u,b}^{\left(n\right)}}$ is the channel response, $g_{u,b}^{\left(n\right)}$ is a complex Gaussian random variable with zero mean and unit variance that represents the small-scale fading of the channel from SCD b to SUE u, and ${\beta }_{u,b}^{\left(n\right)}$ is the large-scale coefficient of this channel. Therefore, the SINRs at SUE u and MUE m are, respectively, written as

$${\gamma }_u=\frac{{\sum }_{b\in {\mathcal{C}}_i}|h_{u,b}^{\left(n\right)}{|}^2{p}_{u,b}^{\left(n\right)}}{I_{u,\mathcal{F}}^{\left(n\right)}+I_{u,\mathcal{C}\setminus {\mathcal{C}}_i}^{\left(n\right)}+{\sigma }_u^2},$$

(4)

$${\gamma }_m=\frac{|{\mathit{h}}_{m,f}^{\left(n\right)}{\mathit{w}}_{m,f}{|}^2{p}_{m,f}^{\left(n\right)}}{I_{\mathrm{cross}-\mathrm{talk}}^{\left(n\right)}+I_{m,\mathcal{F}\setminus f}^{\left(n\right)}+I_{m,\mathcal{C}}^{\left(n\right)}+{\sigma }_m^2}.$$

(5)

2.2. Problem Formulation

As the deployment of small cells in UDNs becomes extremely dense, the problem of effectively using resources becomes more important than ever. In this section, we formulate the optimization problems for the resource allocation of both spectrum and power according to the processing operations in Figure 2. The overall architectures of the proposed optimization solutions are shown in Figure 2 with the names of the 6 problems in blocks. The detailed description and algorithms are presented in Section 3, Section 4 and Section 5. The objective functions at different stages are different to obtain an optimal solution for multiple proposed objectives, such as interference mitigation in clustering, a guarantee of high-gain channels in subchannel allocation, and efficient energy usage in power allocation.

2.2.1. Subchannel Allocation (SCA)

Due to the high efficiency of spectral usage, co-channel assignment is usually used in practical systems [27]. However, this causes high interference between the two tiers in UDNs and is hard to control, especially with the random deployment of small cells. Thus, the optimization problems with the objective to choose the highest channel gain guarantee that the BSs serve their UEs in good subchannels.

SCA for macrocells: We consider MBS f serving $M_f$ MUEs. By using mMIMO technology at each MBS, the number of MUEs in each subchannel causes a minor interference effect. The target is to find a subchannel with the highest gain for each MUE. The optimization problem can be expressed as

$$\begin{array}{cc}\hfill \underset{a_{m,f}^{\left(n\right)}}{\max }& {\sum }_{m=1}^{M_f}{\sum }_{n=1}^N{\mathrm{VM}}_{m,n}{a}_{m,f}^{\left(n\right)}\hfill \end{array}$$

(6a)

$$\begin{array}{cc}\hfill \mathsf{s}.\mathsf{t}.& {\sum }_{n=1}^N{a}_{m,f}^{\left(n\right)}=1,m=1,\dots ,M_f,\hfill \end{array}$$

(6b)

$$\begin{array}{cc}\hfill & a_{m,f}^{\left(n\right)}\in \{0,1\},n=1,\dots ,N,\forall m\in {\mathcal{M}}_f,\hfill \end{array}$$

(6c)

where ${\mathrm{VM}}_{m,n}=\mathrm{trace}\left\{∥{\mathit{h}}_{m,f}^{\left(n\right)}∥\right\}$. (6b) indicates that each MUE $m\in {\mathcal{M}}_f$ authorizes only one subchannel. (6c) represents the constraint of the binary values of the SCA. If $a_{m,f}^{\left(n\right)}=1$, then MUE m is served by MBS f through subchannel n, and vice versa if $a_{m,f}^{\left(n\right)}=0$. Optimization problem (6) is denoted by (P3) in Figure 2. Problem (6) is integer linear programming because (6a) and (6b) are affine functions, and (6c) is the constraint of two discrete values of variables.

SCA for small cells: In Figure 2, after clustering the small cells, the SCDs in each cluster cooperatively transmit the same signal to their SUEs. Therefore, the power of the desired signal at each SUE is improved compared with that using only one serving SCD. Considering cluster ${\mathcal{C}}_i$, to find the appropriate subchannels for the SUEs in set ${\mathcal{U}}_{{\mathcal{C}}_i}$, an optimization problem is formulated as

$$\begin{array}{cc}\hfill \underset{a_{u,{\mathcal{C}}_i}^{\left(n\right)}}{\max }& {\sum }_{u=1}^{U_{{\mathcal{C}}_i}}{\sum }_{n=1}^N{\mathrm{VC}}_{u,n}{a}_{u,{\mathcal{C}}_i}^{\left(n\right)}\hfill \end{array}$$

(7a)

$$\begin{array}{cc}\hfill \mathsf{s}.\mathsf{t}.& {\sum }_{u=1}^{U_{{\mathcal{C}}_i}}{a}_{u,{\mathcal{C}}_i}^{\left(n\right)}=1,n=1,\dots ,N,\hfill \end{array}$$

(7b)

$$\begin{array}{cc}\hfill & {\sum }_{n=1}^N{a}_{u,{\mathcal{C}}_i}^{\left(n\right)}=1,\forall u\in {\mathcal{U}}_{{\mathcal{C}}_i},\hfill \end{array}$$

(7c)

$$\begin{array}{cc}\hfill & a_{u,{\mathcal{C}}_i}^{\left(n\right)}\in \{0,1\},n=1,\dots ,N,\forall u\in {\mathcal{U}}_{{\mathcal{C}}_i},\hfill \end{array}$$

(7d)

where ${\mathrm{VC}}_{u,n}={\sum }_{b\in {\mathcal{C}}_i}|h_{u,b}^{\left(n\right)}{|}^2$ is the sum of channel gains from SCDs in cluster ${\mathcal{C}}_i$ to SUE u. (7b), (7c) indicates that each subchannel n is authorized by at most one SUE $u\in {\mathcal{U}}_{{\mathcal{C}}_i}$ and vice versa. (7d) is the constraint of the binary values of SCA. If $a_{u,{\mathcal{C}}_i}^{\left(n\right)}=1$, then SUE u is served by the SCDs in cluster ${\mathcal{C}}_i$ through subchannel n, and vice versa if $a_{u,{\mathcal{C}}_i}^{\left(n\right)}=0$. Optimization problem (7) is denoted by (P2) in Figure 2. Problem (7) is integer linear programming because (7a)–(7c) are affine functions, and (7d) is the constraint of two discrete values of variables. The solving methods for (P2) and (P3) are proposed in Section 4.1.

2.2.2. Power Allocation (PA)

In UDNs, the distance between the SCDs and SUEs as well as the distance between small cells are very small: from a few dozen to a few hundred meters. Therefore, increasing the power at an SCD to improve the signal to its UEs causes extremely high interference to other small cells compared with that of conventional networks. In addition, different tiers have different objectives that need to be optimized. For that reason, after SCA, we formulate two optimization problems for PA in macrocells and small cells (Figure 2).

PA for MBSs (EE maximization): Each MBS, equipped with massive antennas, is connected directly to a core network. MBSs, using high power with wide coverage areas, guarantee the serving of all UEs that are not connected to any small cell. Therefore, an important problem for MBSs is maximizing the data rate of the MUEs to satisfy the standards of the new generations of future networks (B5G) while efficiently using energy. EE, which is the ratio of data rate to power consumption, is widely used for evaluating the efficiency of energy usage. The optimization problem of PA for MBSs is as follows

$$\begin{array}{cc}\hfill \underset{p_{m,f}^{\left(n\right)}}{\max }& \frac{{\sum }_{m=1}^{M}BW{log}_2(1+{\gamma }_m)}{{\sum }_{f\in \mathcal{F}}{P}_f^{\mathrm{total}}}\hfill \end{array}$$

(8a)

$$\begin{array}{cc}\hfill \mathsf{s}.\mathsf{t}.& {\sum }_{n=1}^N{\sum }_{m\in {\mathcal{M}}_f}{a}_{m,f}^{\left(n\right)}{p}_{m,f}^{\left(n\right)}\le P_{\mathrm{MBS}}^{max},\forall f\in \mathcal{F},\hfill \end{array}$$

(8b)

$$\begin{array}{cc}\hfill & p_{m,f}^{\left(n\right)}\ge 0,\forall f\in \mathcal{F},\forall m\in \mathcal{M},n=1,\dots ,N,\hfill \end{array}$$

(8c)

$$\begin{array}{cc}\hfill & BW{log}_2(1+{\gamma }_m)\ge Tp{m}_{min},\forall m\in \mathcal{M},\hfill \end{array}$$

(8d)

where the variable $p_{m,f}^{\left(n\right)}$ represents the transmit power of MBS f to MUE m in subchannel n. $P_f^{\mathrm{total}}=P_f^{\mathrm{circuit}}+{\sum }_{n=1}^N{\sum }_{m\in {\mathcal{M}}_f}{a}_{m,f}^{\left(n\right)}{p}_{m,f}^{\left(n\right)}$ is the sum of power fo ther hardware circuits and power for transmitting signal in all subchannels of MBS f, $P_{MBS}^{max}$ is the maximum power of each MBS, and $Tp{m}_{min}$ is the minimum throughput of each MUE. (8b) and (8c) are the constraints of the transmit power of MBSs, which range from 0 to $P_{MBS}^{max}$. (8d) guarantees the QoS of all UEs.

PA for SCDs (EE maximization): Due to the short distance between the SCDs and SUEs, the channel links experience the line of sight (LoS). However, with a large number of small cells, the power consumption rises considerably. Thus, the target of SCDs is effectively using limited energy. The optimization problem for maximizing the EE of small cells is formulated as

$$\begin{array}{cc}\hfill \underset{p_{u,b}^{\left(n\right)}}{\max }& \frac{{\sum }_{u=1}^{U}BW{log}_2(1+{\gamma }_u)}{{\sum }_{{\mathcal{C}}_i\in \mathcal{C}}{\sum }_{b\in {\mathcal{C}}_i}{P}_b^{\mathrm{total}}}\hfill \end{array}$$

(9a)

$$\begin{array}{cc}\hfill \mathsf{s}.\mathsf{t}.& {\sum }_{n=1}^N{\sum }_{u\in {\mathcal{U}}_{{\mathcal{C}}_i}}{a}_{u,{\mathcal{C}}_i}^{\left(n\right)}{p}_{u,b}^{\left(n\right)}\le P_{\mathrm{SCD}}^{max},\forall b\in \mathcal{B},\hfill \end{array}$$

(9b)

$$\begin{array}{cc}\hfill & p_{u,b}^{\left(n\right)}\ge 0,\forall b\in \mathcal{B},\forall u\in \mathcal{U},n=1,\dots ,N,\hfill \end{array}$$

(9c)

$$\begin{array}{cc}\hfill & BW{log}_2(1+{\gamma }_u)\ge Tp{s}_{min},\forall u\in \mathcal{U},\hfill \end{array}$$

(9d)

where $p_{u,b}^{\left(n\right)}$ denotes the transmit power of SCD b to SUE u in subchannel n. $P_b^{\mathrm{total}}=P_b^{\mathrm{circuit}}+{\sum }_{n=1}^N{\sum }_{u\in {\mathcal{U}}_{{\mathcal{C}}_i}}{a}_{u,{\mathcal{C}}_i}^{\left(n\right)}{p}_{u,b}^{\left(n\right)}$ is the sum of power for hardware circuits and power for transmitting signal in all subchannels of SCD $b\in {\mathcal{C}}_i$, and $P_{SCD}^{max}$ is the maximum transmit power of each SCD, $Tp{s}_{min}$ is the minimum throughput of each MUE. (9b) and (9c) represent the range of transmit power. To guarantee the QoS, (9d) is used for all SUEs.

3. SCDs Clustering by Coalition Game (P1)

In this section, we introduce a coalition game for clustering small cells. In UDNs, an effective clustering method is essential because of three main reasons, as follows:

• Mitigating interference: Intercell interference is a big challenge in UDNs due to the ultra-dense density of small cells, especially for SUEs in the overlapping areas of small cells. Formulating clusters that combine small cells seriously interfering with each other is an appropriate solution. The SCDs in the same cluster cooperatively transmit the signal to UEs so as there is no intracluster interference.
• Enhancing the quality of the signal: In UDNs, the SUEs in the overlapping areas of two or more small cells are usually far from their primal SCDs, while they receive high interference from the others. In many cases, using some of these SCDs along with primary SCDs to serve UEs helps improve the power of the desired signal. Furthermore, multiple SCDs transmitting the same signal to a UE also improves the diversity gain.
• Decreasing handover processes: The smaller the size of small cells, the larger the number of handover processes. Moreover, when UEs move at high speed, the central controller has to compute the complex handover processes with the massive number of UEs in a short time. Therefore, if small cells join together to form temporarily bigger cells, the number of handover processes significantly declines.

Therefore, adjacent small cells have an incentive to join clusters to obtain a higher value for the objective instead of independent transmission. The coalition game. as a type of cooperative game where players tend to cooperate to achieve higher total utility, is appropriate to model this situation. The standard form of a coalition game is $CG=〈\mathit{PL},\mathit{v}〉$, where $\mathit{PL}={\mathcal{B}}_{\mathrm{active}}$ is the set of players (i.e., SCDs, which serve at least one SUEs), and $\mathit{v}\left(\mathcal{C}\right)\subseteq {\mathbb{R}}^C$ is the set of nontransferable utility functions of clusters [28]. The utility function of cluster ${\mathcal{C}}_i$ is defined as

$$v\left({\mathcal{C}}_i\right)={\sum }_{u\in {\mathcal{U}}_{{\mathcal{C}}_i}}SI{R}_u,$$

(10)

where $SI{R}_u$ is the SIR of the links from cluster ${\mathcal{C}}_i$ to SUE u and is expressed as

$$SI{R}_u=\frac{{\sum }_{b^{\prime }\in {\mathcal{C}}_i}{\left|{\overline{h}}_{u,b^{\prime }}\right|}^2}{{\sum }_{{\mathcal{C}}_j\in \mathcal{C}\setminus {\mathcal{C}}_j}{\sum }_{b\in {\mathcal{C}}_j}|{\overline{h}}_{u,b}{|}^2},$$

(11)

where ${\overline{h}}_{u,b}=\frac{1}{N}{\sum }_{n=1}^N{h}_{u,b}^{\left(n\right)}$ is the average of channel gain from SCD b to SUE u. The solution of this coalition game is the set of coalitions of players or clusters of SCDs in UDNs [28]. In these clusters, there is no SCD that has an incentive to change clusters for achieving higher utility. This is the state where all clusters have reached equilibrium. The two operations that change the SCDs between clusters are split-merge and swap [29]. If SCD $b\in {\mathcal{C}}_j$ leaves ${\mathcal{C}}_j$ to join ${\mathcal{C}}_i$, then the operation is denoted as $\left\{{\mathcal{C}}_i,{\mathcal{C}}_j\right\}\to \left\{{\mathcal{C}}_i\cup \left\{b\right\},{\mathcal{C}}_j\setminus \left\{b\right\}\right\}.$ In addition, if SCD $b\in {\mathcal{C}}_j$ swaps with $b^{\prime }\in {\mathcal{C}}_i$, then the operation is described as $\left\{{\mathcal{C}}_i,{\mathcal{C}}_j\right\}\to \left\{{\mathcal{C}}_i\cup \left\{b\right\}\setminus \left\{b^{\prime }\right\},{\mathcal{C}}_j\cup \left\{b^{\prime }\right\}\setminus \left\{b\right\}\right\}.$ SCD b tends to leave ${\mathcal{C}}_j$ to join ${\mathcal{C}}_i$ or prefer ${\mathcal{C}}_i$ to ${\mathcal{C}}_j$ if $v({\mathcal{C}}_i\cup \left\{b\right\})+v({\mathcal{C}}_j\setminus \left\{b\right\})>v\left({\mathcal{C}}_i\right)+v\left({\mathcal{C}}_j\right).$

To obtain the solution to this coalition game, we use Algorithm 1. In this algorithm, we use the variable $d_{max}$ for determining the SUEs that are less than $d_{max}$ m away from SCD b. Then, the set of neighbor clusters ${\mathcal{C}}_{\mathrm{near}}$ that consists of these SUEs is determined. Therefore, different from [30], we only consider some close clusters with SCD b because the interference from b has a minor impact on far SUEs at very high frequency. In addition, variables g and $g_n$ are used for choosing the best option for SCD b to move to a new cluster. If the number of SCDs in each cluster increases, then the number of pairs of SCD and SUE with a distance larger than $d_{max}$ increases. Thus, we use $N_{max}$ for limiting the number of SCDs in each cluster.

Algorithm 1 Clustering SCDs using the coalition game.

1: Input: Channel matrix $\mathit{H}$, the set of players $\mathit{PL}={\mathcal{B}}_{\mathrm{active}}$, locations of SCDs and SUEs, the maximum number of SCDs in a cluster $N_{max}$, the number of subchannels N, $d_{max}$ 2: Randomly initialize a partition ${\mathcal{C}}^{\left(0\right)}$, $l=0$ 3: while ${\mathcal{C}}^{\left(l\right)}\ne {\mathcal{C}}^{\left(old\right)}$ 4:    ${\mathcal{C}}^{\left(old\right)}\leftarrow {\mathcal{C}}^{\left(l\right)}$ 5:    for $b\in {\mathcal{B}}_{\mathrm{active}}$ 6:       Assume $b\in {\mathcal{C}}_j$, ${\mathcal{C}}_{\mathrm{near}}$ is the set of clusters that have some SUEs covered with radius $d_{max}$ and center b,  gap $g=0$ 7:       for ${\mathcal{C}}_i\in {\mathcal{C}}_{\mathrm{near}}$ 8:          if $|{\mathcal{C}}_i|=N_{max}$ and $U_{{\mathcal{C}}_i}+U_b-U_{b^{\prime }}\le N$ and $U_{{\mathcal{C}}_j}+U_{b^{\prime }}-U_b\le N$ 9:             ${\mathcal{C}}^{\left(tmp\right)}\leftarrow$ swap SCD b and SCD $b^{\prime }\in {\mathcal{C}}_i$ 10:             $g_n=v({\mathcal{C}}_i\cup \left\{b\right\}\setminus \left\{b^{\prime }\right\})+v({\mathcal{C}}_j\cup \left\{b^{\prime }\right\}\setminus \left\{b\right\})-v\left({\mathcal{C}}_i\right)-v\left({\mathcal{C}}_j\right)$ 11:          else if $U_{{\mathcal{C}}_i}+U_b\le N$ 12:             ${\mathcal{C}}^{\left(tmp\right)}\leftarrow$ SCD b joins ${\mathcal{C}}_i$ 13:             $g_n=v({\mathcal{C}}_i\cup \left\{b\right\})+v({\mathcal{C}}_j\setminus \left\{b\right\})-v\left({\mathcal{C}}_i\right)-v\left({\mathcal{C}}_j\right)$ 14:          if $g_n>g$ 15:             ${\mathcal{C}}^{\left(ok\right)}\leftarrow {\mathcal{C}}^{\left(tmp\right)}$, $g=g_n$ 16:       ${\mathcal{C}}^{\left(l\right)}\leftarrow {\mathcal{C}}^{\left(ok\right)}$ 17:    $l\leftarrow l+1$ 18: Output: The set of clusters $\mathcal{C}$

4. Subchannel Allocation (P2, P3) and Beamforming Design for MBSs (P4)

In this section, we propose methods to solve the SCA problems (6) and (7). Satisfactory subchannels are allocated to UEs with high channel gain while guaranteeing the constraints are met. In addition, the beamforming design for MBSs is also described.

4.1. Subchannel Allocation

(P2) For clusters, constraints (7b)–(7d) are the same as the constraints of an assignment problem where each task is assigned to only one worker to minimize the cost. One of the effective methods to solve assignment problems is the Hungarian method [31]. Compared with other methods, the Hungarian algorithm can solve large-scale assignment problems in polynomial time. Addtionally, it is a deterministic and global algorithm that guarantees the optimal solution is the best assignment. Regarding flexibility, the Hungarian method can be applied to solve assignment problems with equal or unequal numbers of resources and tasks, especially in the cases of variable numbers of network elements in clusters in our proposed SCA problem. Therefore, we use the Hungarian algorithm for making decisions in choosing the optimal subchannels for SUEs. Matrix ${\mathbf{VC}}_{maxprob}$, which consists of the value of ${\mathrm{VC}}_{u,n}$ with all pairs of SUE $u\in {\mathcal{C}}_i$ and subchannel n, is given in

Table 2. The matrix of values of ${\mathrm{VC}}_{u,n}$.

SUE∖Channel	1	2	…	N
1	${\mathrm{VC}}_{1,1}$	${\mathrm{VC}}_{1,2}$	…	${\mathrm{VC}}_{1,N}$
2	${\mathrm{VC}}_{2,1}$	${\mathrm{VC}}_{2,2}$	…	${\mathrm{VC}}_{2,N}$
…	…	…	…	…
$U_{{\mathcal{C}}_i}$	${\mathrm{VC}}_{U_{{\mathcal{C}}_i},1}$	${\mathrm{VC}}_{U_{{\mathcal{C}}_i},2}$	…	${\mathrm{VC}}_{U_{{\mathcal{C}}_i},N}$

.

To obtain the solution to the optimization problem (7), we use the Hungarian method in Algorithm 2 [30]. Because of the limitation of the number of UEs in each cluster by N, the complexity of Algorithm 2 is $O\left(N^4\right)$ [31].

Algorithm 2 The Hungarian method for SCA in each cluster.

1: Input: VC matrix ${\mathbf{VC}}_{maxprob}$. 2: Convert to minimization problem by subtracting the maximum of the matrix from all elements ${\mathbf{VC}}_{minprob}=\{{\mathbf{VC}}_{i,k}=\left|{\mathbf{VC}}_{i,k}-\max \left({\mathbf{VC}}_{maxprob}\right)\right||i=1,\dots ,U_{{\mathcal{C}}_i}\mathsf{and}k=1,\dots ,N\}$. 3: Extend ${\mathbf{VC}}_{minprob}$ to a square matrix with the size of $N\times N$ 4: Obtain modified matrix: 5:    Subtract the smallest entry from all entries in each row of ${\mathbf{VC}}_{minprob}$. 6:    Subtract the smallest entry from all entries in each column of ${\mathbf{VC}}_{minprob}$. 7: Repeat 8:    Cover all zeros in the matrix with minimum number of vertical/horizontal lines $n{l}_{min}$. 9:    if $n{l}_{min}<N$ then 10:       Subtract the lowest number from all elements that are not covered by any line. 11:       Add this lowest number to elements that are crossed by any two lines. 12: Until $n{l}_{min}=N$. 13: Obtain the optimal association from zeros in the processed matrix. 14: Output: The association matrix $\mathit{X}=\left\{x_{i,k}\right|i=1,\dots ,,U_{{\mathcal{C}}_i}\mathsf{and}k=1,\dots ,N\}$.

(P3) For macrocells, constraint (6b) requires one MUE to be allocated only one subchannel. Moreover, the number of MUEs in a subchannel is not important due to the mMIMO MBSs. Therefore, the solution to the optimization problem (6) can be easily obtained by choosing the maximum value of ${\mathrm{VM}}_{m,n},n=1,\dots ,N$ for each MUE m.

4.2. Beamforming Design

According to the model in Figure 2, after SCA, we need to design the beamforming vectors for MBSs. Assuming that the CSI is known at the MBSs, we introduce two types of appropriate beamforming vectors for two cases.

• If there are multiple MUEs that are served by MBS f in subchannel n, then the beamforming vector is one vector in the null space of the channel matrix from MBS f to all of its MUEs except MUE m, and is expressed as

  $$\begin{array}{c}\hfill {\mathit{w}}_{m,f}=\mathrm{Null}\left({\mathit{H}}_{{\mathcal{M}}_f\setminus m,f}^{\left(n\right)}\right).\end{array}$$
  Therefore, we can neglect the cross-talk interference in (2).
• If only one MUE m is served by MBS f in subchannel n, then according to the zero-forcing beamforming, the beamforming vector is as follows

  $$\begin{array}{c}\hfill {\mathit{w}}_{m,f}={\left({\mathit{h}}_{m,f}^{\left(n\right)}\right)}^{†}/∥{\left({\mathit{h}}_{m,f}^{\left(n\right)}\right)}^{†}∥.\end{array}$$
  In both cases, the magnitude of the beamforming vector is normalized to 1 ($\left|\right|{\mathit{w}}_{m,f}\left|\right|=1$). Thus, the PA for MBSs and SCDs only depends on $p_{m,f}^{\left(n\right)}$ and $p_{u,b}^{\left(n\right)}$.

5. Power Allocation for Macrocells and Small Cells (P5, P6)

Power allocation (PA) is a promising approach for interference mitigation in UDNs. In this section, we propose both a distributed method and a centralized method to obtain the solutions to the optimization problems (8) and (9). It is clear that the powers of MBSs and SCDs are variables in both optimization problems (8) and (9). This causes these optimization problems to be extremely complex and hard to solve. In addition, the channels from MBSs to MUEs are usually more strongly affected by environmental factors than the channel between SCDs and SUEs, while guaranteeing the QoS. Therefore, the optimization problem of MBSs has higher priority than that of SCDs. We assume that the interference from small-cell tiers to each MUE m is a constant at first as follows:

$$I_{m,\mathcal{C}}^{\left(n\right)}={\sum }_{{\mathcal{C}}_i\in \mathcal{C}}{\sum }_{b\in {\mathcal{C}}_i}{\sum }_{u\in {\mathcal{U}}_{{\mathcal{C}}_i}}{a}_{u,{\mathcal{C}}_i}^{\left(n\right)}{\left|h_{m,b}^{\left(n\right)}\right|}^2{P}_{SCD}^{max}/N.$$

(12)

Each active SCD uses the maximum power of $P_{SCD}^{max}$, which is divided equally to th subchannels, to serve theSUEs. Thus, the interference from the small-cell tier to MUEs is extremely high when first solving the PA optimization problem of MBSs. This helps the QoS constraints to remain satisfied after obtaining the solution to the PA optimization problem of SCDs with very low interference.

5.1. Centralized Power Allocation

In this subsection, we propose practical methods to solve centralized PA problems for macrocells and small cells. In these methods, we combine three techniques: changing variables, approximating objective functions, and the Dinkelbach algorithm. To this end, the very complex maximizing EE optimization problems especially in UDNs are converted into much easier convex problems in iterative algorithms. This is a promising approach to achieving real-time computing in multitier massive-cell networks such as UDNs.

The EE maximization PA problem for macrocells is expressed as follows

$$\begin{array}{cc}\hfill \underset{p_{m,f}^{\left(n\right)}}{\max }& \frac{{\displaystyle {\sum }_{m=1}^M{log}_2\left({\displaystyle 1+\frac{|{\mathit{h}}_{m,f}^{\left(n\right)}{\mathit{w}}_{m,f}{|}^2{p}_{m,f}^{\left(n\right)}}{I_{\mathrm{cross}-\mathrm{talk}}^{\left(n\right)}+I_{m,\mathcal{F}\setminus f}^{\left(n\right)}+I_{m,\mathcal{C}}^{\left(n\right)}+{\sigma }_m^2}}\right)}}{{\sum }_{f\in F}{P}_f^{total}}=\frac{{\phi }_{\mathcal{M}}\left({\mathit{P}}_{\mathcal{F}}\right)}{{\pi }_{\mathcal{M}}\left({\mathit{P}}_{\mathcal{F}}\right)}\hfill \end{array}$$

(13a)

$$\begin{array}{cc}\hfill \mathsf{s}.\mathsf{t}.& {\sum }_{n=1}^N{\sum }_{m\in {\mathcal{M}}_f}{a}_{m,f}^{\left(n\right)}{p}_{m,f}^{\left(n\right)}\le P_{\mathrm{MBS}}^{max},\forall f\in \mathcal{F},\hfill \end{array}$$

(13b)

$$\begin{array}{cc}\hfill & p_{m,f}^{\left(n\right)}\ge 0,\forall f\in \mathcal{F},\forall m\in \mathcal{M},n=1,\dots ,N,\hfill \end{array}$$

(13c)

$$\begin{array}{cc}\hfill & {\gamma }_m\ge {\gamma }_{m min},\forall m\in \mathcal{M},\hfill \end{array}$$

(13d)

where ${\gamma }_{m min}\triangleq 2^{Tp{m}_{min}/BW}-1$ is the minimum SINR to guarantee the QoS of MUEs. By using the beamforming vectors in Section 4.2, the value of cross-talk interference $I_{\mathrm{cross}-\mathrm{talk}}^{\left(n\right)}$ in the macrocells equals 0. $I_{m,\mathcal{C}}^{\left(n\right)}$ is fixed by using estimation (12). We can easily convert constraint (13d) into a linear inequality becuase each ${\gamma }_m$ in (13d) is an affine-affine fraction. Therefore, all constraints in the optimization problem (13) are linear. Let $I_m\left({\mathit{P}}_{\mathcal{F}}\right)\triangleq I_{\mathrm{cross}-\mathrm{talk}}^{\left(n\right)}+I_{m,\mathcal{F}\setminus f}^{\left(n\right)}+I_{m,\mathcal{C}}^{\left(n\right)}.$ Definitely, in (13a), a given UE m is served by MBS f in subchannel n, so $p_{m,f}^{\left(n\right)}>0$ with constraint (13d). For $x_m={\left|{\mathit{h}}_{m,f}^{\left(n\right)}{\mathit{w}}_{m,f}\right|}^2{p}_{m,f}^{\left(n\right)}$, $y_m=I_m\left({\mathit{P}}_{\mathcal{F}}\right)+{\sigma }_m^2$, and at the $\kappa$-th iteration ${\overline{x}}_m={\left|{\mathit{h}}_{m,f}^{\left(n\right)}{\mathit{w}}_{m,f}\right|}^2{p}_{m,f}^{\left(n\right)\left(\kappa \right)}$, ${\overline{y}}_m=I_m\left({\mathit{P}}_{\mathcal{F}}^{\left(\kappa \right)}\right)+{\sigma }_m^2$, where ${\mathit{P}}_{\mathcal{F}}^{\left(\kappa \right)}$, is a feasible point of optimization problem (13). Following the method in [32] and using inequality (A1) in Appendix A.1, we have ${\phi }_{\mathcal{M}}\left({\mathit{P}}_{\mathcal{F}}\right)\ge {\tilde{\phi }}_{\mathcal{M}}({\mathit{P}}_{\mathcal{F}},{\mathit{P}}_{\mathcal{F}}^{\left(\kappa \right)}),$ where ${\displaystyle {\tilde{\phi }}_{\mathcal{M}}({\mathit{P}}_{\mathcal{F}},{\mathit{P}}_{\mathcal{F}}^{\left(\kappa \right)})\triangleq {\sum }_{m=1}^M\left({\tilde{a}}_m^{\left(\kappa \right)}-{\tilde{b}}_m^{\left(\kappa \right)}/x_m-{\tilde{c}}_m^{\left(\kappa \right)}{y}_m\right)}$ and ${\displaystyle {\tilde{a}}_m^{\left(\kappa \right)}={log}_2\left({\displaystyle 1+\frac{{\overline{x}}_m}{{\overline{y}}_m}}\right)+\frac{2{\overline{x}}_m}{ln\left(2\right)({\overline{x}}_m+{\overline{y}}_m)},}$ ${\tilde{b}}_m^{\left(\kappa \right)}=$ ${\displaystyle \frac{{\overline{x}}_m^2}{ln\left(2\right)({\overline{x}}_m+{\overline{y}}_m)},{\tilde{c}}_m^{\left(\kappa \right)}=\frac{{\overline{x}}_m}{ln\left(2\right){\overline{y}}_m({\overline{x}}_m+{\overline{y}}_m)}.}$

Let ${\lambda }_{\mathcal{M}}^{\left(\kappa \right)}\triangleq {\phi }_{\mathcal{M}}\left({\mathit{P}}_{\mathcal{F}}^{\left(\kappa \right)}\right)/{\pi }_{\mathcal{M}}\left({\mathit{P}}_{\mathcal{F}}^{\left(\kappa \right)}\right)$. According to the Dinkelbach algorithm, the suboptimization problem to find ${\mathit{P}}_{\mathcal{F}}^{(\kappa +1)}$ for iteration $(\kappa +1)$ is as follows

$$\begin{array}{cc}\hfill \underset{p_{m,f}^{\left(n\right)}}{\max }& {\tilde{\phi }}_{\mathcal{M}}({\mathit{P}}_{\mathcal{F}},{\mathit{P}}_{\mathcal{F}}^{\left(\kappa \right)})-{\lambda }_{\mathcal{M}}^{\left(\kappa \right)}{\pi }_{\mathcal{M}}\left({\mathit{P}}_{\mathcal{F}}\right)\mathsf{s}.\mathsf{t}.\left(13\mathrm{b}\right),\left(13\mathrm{c}\right),\left(13\mathrm{d}\right).\hfill \end{array}$$

(14)

Because ${\tilde{a}}_m^{\left(\kappa \right)},{\tilde{b}}_m^{\left(\kappa \right)},{\tilde{c}}_m^{\left(\kappa \right)}>0$, the objective function in (14), which is the sum of pairs of convex functions, i.e., $constant\times variable$ and $-constant/variable$ ($constant>0$), is convex. Therefore, problem (14) is convex due to the convexity of the objective function and the affine functions at the constraints. The proof of the convergence is provided in Appendix B.1. Based on [33], the computational complexity of each iteration for solving (14) is $\mathcal{O}\left({\left(MFN\right)}^2{(F+MFN+M)}^{2.5}+{(F+MFN+M)}^{3.5}\right)$ with the number of decision variables $MFN$ and the number of constraints $F+MFN+M$.

It is given that the UE u is served by cluster ${\mathcal{C}}_i$, which combines SCDs b. The EE maximization PA problem for small cells is given by

$$\begin{array}{cc}\hfill \underset{p_{u,b}^{\left(n\right)}}{\max }& \frac{{\displaystyle {\sum }_{u=1}^U{log}_2\left({\displaystyle 1+\frac{{\sum }_{b\in {\mathcal{C}}_i}|h_{u,b}^{\left(n\right)}{|}^2{p}_{u,b}^{\left(n\right)}}{I_{u,\mathcal{F}}^{\left(n\right)}+I_{u,\mathcal{C}\setminus {\mathcal{C}}_i}^{\left(n\right)}+{\sigma }_u^2}}\right)}}{{\sum }_{{\mathcal{C}}_i\in \mathcal{C}}{\sum }_{b\in {\mathcal{C}}_i}{P}_b^{\mathrm{total}}}=\frac{{\phi }_{\mathcal{U}}\left({\mathit{P}}_{\mathcal{C}}\right)}{{\pi }_{\mathcal{U}}\left({\mathit{P}}_{\mathcal{C}}\right)}\hfill \end{array}$$

(15a)

$$\begin{array}{cc}\hfill \mathsf{s}.\mathsf{t}.& {\sum }_{n=1}^N{\sum }_{u\in {\mathcal{U}}_{{\mathcal{C}}_i}}{a}_{u,{\mathcal{C}}_i}^{\left(n\right)}{p}_{u,b}^{\left(n\right)}\le P_{\mathrm{SCD}}^{max},\forall b\in \mathcal{B},\hfill \end{array}$$

(15b)

$$\begin{array}{cc}\hfill & p_{u,b}^{\left(n\right)}\ge 0,\forall b\in \mathcal{B},\forall u\in \mathcal{U},n=1,\dots ,N,\hfill \end{array}$$

(15c)

$$\begin{array}{cc}\hfill & {\gamma }_u\ge {\gamma }_{s min},\forall u\in \mathcal{U},\hfill \end{array}$$

(15d)

where ${\gamma }_{s min}\triangleq 2^{Tp{s}_{min}/BW}-1$ is the minimum SINR that guarantees the QoS of the SUEs. Using the same technique as for the PA problem for macrocells, (15d) can be easily converted into a linear constraint. Therefore, the optimization problem (15) is a linear-constrained optimization problem. However, the objective function of the optimization problem (15) is more challenging than the optimization problem (13) of macrocells because there is more than one variable in the numerator of ${\gamma }_u$. Let $I_u\left({\mathit{P}}_{\mathcal{C}}\right)=I_{u,\mathcal{F}}^{\left(n\right)}+I_{u,\mathcal{C}\setminus {\mathcal{C}}_i}^{\left(n\right)}.$ To find the lower bound of ${\phi }_{\mathcal{U}}\left({\mathit{P}}_{\mathcal{C}}\right)$, we use inequality (A3) in Appendix A.2. In detail, for $x_{u,b}={\left|h_{u,b}^{\left(n\right)}\right|}^2{p}_{u,b}^{\left(n\right)}$, $y_u=I_u\left({\mathit{P}}_{\mathcal{C}}\right)+{\sigma }_m^2$, and at the $\kappa$th iteration ${\overline{x}}_{u,b}={\left|h_{u,b}^{\left(n\right)}\right|}^2{p}_{u,b}^{\left(n\right)\left(\kappa \right)}$, ${\overline{y}}_u=I_u\left({\mathit{P}}_{\mathcal{C}}^{\left(\kappa \right)}\right)+{\sigma }_u^2$, where ${\mathit{P}}_{\mathcal{C}}^{\left(\kappa \right)}$ is a feasible point of the optimization problem (15). Following inequality (A3), we have ${\phi }_{\mathcal{U}}\left({\mathit{P}}_{\mathcal{C}}\right)\ge {\tilde{\phi }}_{\mathcal{U}}({\mathit{P}}_{\mathcal{C}},{\mathit{P}}_{\mathcal{C}}^{\left(\kappa \right)}),$ where ${\displaystyle {\tilde{\phi }}_{\mathcal{U}}({\mathit{P}}_{\mathcal{C}},{\mathit{P}}_{\mathcal{C}}^{\left(\kappa \right)})\triangleq {\sum }_{u=1}^U\left({\tilde{a}}_u^{\left(\kappa \right)}-{\sum }_{b\in {\mathcal{C}}_i}{\tilde{b}}_{u,b}^{\left(\kappa \right)}/x_{u,b}-{\tilde{c}}_u^{\left(\kappa \right)}{y}_u\right)}$ and ${\overline{s}}_u={\sum }_{b\in {\mathcal{C}}_i}{\overline{x}}_{u,b},$ ${\displaystyle {\tilde{a}}_u^{\left(\kappa \right)}={log}_2\left({\displaystyle 1+\frac{{\overline{s}}_u}{{\overline{y}}_u}}\right)+\frac{2{\overline{s}}_u}{ln\left(2\right)({\overline{s}}_u+{\overline{y}}_u)},}$ ${\displaystyle {\tilde{b}}_{u,b}^{\left(\kappa \right)}=\frac{{\overline{x}}_{u,b}^2}{ln\left(2\right)({\overline{s}}_u+{\overline{y}}_u)}, {\tilde{c}}_u^{\left(\kappa \right)}=\frac{{\overline{s}}_u}{ln\left(2\right){\overline{y}}_u({\overline{s}}_u+{\overline{y}}_u)}.}$

Let ${\lambda }_{\mathcal{U}}^{\left(\kappa \right)}\triangleq {\phi }_{\mathcal{U}}\left({\mathit{P}}_{\mathcal{C}}^{\left(\kappa \right)}\right)/{\pi }_{\mathcal{U}}\left({\mathit{P}}_{\mathcal{C}}^{\left(\kappa \right)}\right)$. According to the Dinkelbach algorithm, the suboptimization problem to find ${\mathit{P}}_{\mathcal{C}}^{(\kappa +1)}$ for iteration $(\kappa +1)$ is as follows

$$\begin{array}{cc}\hfill \underset{p_{u,b}^{\left(n\right)}}{\max }& {\tilde{\phi }}_{\mathcal{U}}({\mathit{P}}_{\mathcal{C}},{\mathit{P}}_{\mathcal{C}}^{\left(\kappa \right)})-{\lambda }_{\mathcal{U}}^{\left(\kappa \right)}{\pi }_{\mathcal{U}}\left({\mathit{P}}_{\mathcal{C}}\right)\mathsf{s}.\mathsf{t}.\left(15\mathrm{b}\right),\left(15\mathrm{c}\right),\left(15\mathrm{d}\right).\hfill \end{array}$$

(16)

The same as for optimization problem (14),  optimization problem (16) is also a convex problem. The proof of convergence is presented in Appendix B.2. Figure 3 shows a diagram for solving the centralized PA optimization problems, with $ϵ$ denoting the tolerance value and $n_{max}$ being the maximum number of iterations. If the values of the objective functions in both (14) and (16) are lower than the tolerance value $ϵ$ or $n_{lp}\ge n_{max}$, with $n_{lp}$ denoting the number of iterations, then the convergence holds true. With the number of decision variables $BUN$ and the number of constraints $B+BUN+U$, the computational complexity of each iteration for solving (14) is $\mathcal{O}\left({\left(BUN\right)}^2{(B+BUN+U)}^{2.5}+{(B+BUN+U)}^{3.5}\right)$.

5.2. Distributed Power Allocation by the Stackelberg Game

The solution of centralized PA methods is the best option for PA to MBSs and SCDs. However, when the number of small cells increases, the execution time of these methods also significantly rises. Therefore, centralized PA methods cannot satisfy the strict time constraints of UDNs in B5G. In this subsection, we propose a distributed PA method based on a SG to considerably reduce the complexity of the original optimization problems while retaining an acceptable efficiency of the solution. A SG is a type of NCG where players have no incentive to collaborate. There are two levels of priority in a SG. In detail, players with a high level of priority are called leaders and the others are followers. When starting a SG, leaders choose optimal actions first. Then, followers see the actions of the leaders to choose their optimal actions. Thus, a SG is also called a two-stage NCG.

The distance from MBSs to MUEs is usually much larger than the distance between SCDs to SUEs. Therefore, the problems of reflection, diffraction, and scattering influence the received signal at MUEs more than at SUEs. Meanwhile, MBSs must guarantee serving all UEs that are out of the coverage of the small cells, with the constraint of data rate. Therefore, MBSs, as leaders, have a high level of priority, and SCDs are followers in this SG.

5.2.1. NCG for Leaders (MBSs)

We formulate the optimization problem (8) as an NCG, where $NC{G}_{leader}=〈{\mathit{PL}}_{\mathcal{F}},{\mathit{P}}_{\mathcal{F}},u{l}_{\mathcal{F}}〉$, where ${\mathit{PL}}_{\mathcal{F}}=\mathcal{F}$ is the set of players (i.e., MBSs), ${\mathit{P}}_{\mathcal{F}}$ is a matrix of PA for all MBSs, and $u{l}_{\mathcal{F}}\left(f\right)$ is the utility function of the EE of MUEs served by MBS f. The solution to $NC{G}_{leader}$ is a Nash equilibrium where no MBS has an incentive to change its power, and the power of each MBS is the best response to the power of other MBSs [30]. By using an NCG, the optimization problem (8) can be divided into multiple suboptimization problems for MBSs to obtain the best response for each player. In each subOP, we find an optimal solution for an MBS while the power of the other MBSs is fixed. A subOP for MBS f is formulated as

$$\begin{array}{cc}\hfill \underset{{\mathit{P}}_f}{\max }& {\displaystyle u{l}_{\mathcal{F}}\left(f\right)=\frac{{\sum }_{m\in {\mathcal{M}}_f}{log}_2(1+{\gamma }_m)}{P_f^{total}}=\frac{A_f\left({\mathit{P}}_f\right)}{B_f\left({\mathit{P}}_f\right)}}\hfill \end{array}$$

(17a)

$$\begin{array}{cc}\hfill \mathsf{s}.\mathsf{t}.& {\sum }_{n=1}^N{\sum }_{m\in {\mathcal{M}}_f}{a}_{m,f}^{\left(n\right)}{p}_{m,f}^{\left(n\right)}\le P_{\mathrm{MBS}}^{max},\hfill \end{array}$$

(17b)

$$\begin{array}{cc}\hfill & p_{m,f}^{\left(n\right)}\ge 0,\forall m\in {\mathcal{M}}_f,n=1,\dots ,N,\hfill \end{array}$$

(17c)

$$\begin{array}{cc}\hfill & {\gamma }_m\ge {\gamma }_{m min},\forall m\in {\mathcal{M}}_f.\hfill \end{array}$$

(17d)

The objective function (17a) is a concave–convex fraction. We can use the Dinkelbach method to solve the optimization problem (17). In detail, we introduce a new variable ${\lambda }_f$, and the optimization problem (17) can be rewritten as

$$\begin{array}{cc}\hfill \underset{{\mathit{P}}_f}{\max }& U{T}_f({\mathit{P}}_f,{\lambda }_f)=A_f\left({\mathit{P}}_f\right)-{\lambda }_f{B}_f\left({\mathit{P}}_f\right)\mathsf{s}.\mathsf{t}.\left(17\mathrm{b}\right),\left(17\mathrm{c}\right),\left(17\mathrm{d}\right).\hfill \end{array}$$

(18)

The objective function in (18) is a concave function in the maximization problem, so the optimization problem (18) is a convex problem. Therefore, the optimization problem (18) can be effectively solved by using CVX or CVXPY [34,35]. The optimal solution ${\mathit{P}}_f^{*}$ is also the optimal solution to optimization problem (18) if $U{T}_f({\mathit{P}}_f^{*},{\lambda }_f^{*})=0$. We use Algorithm 3 to find the solution for the NCG $NC{G}_{leader}$. With the number of decision variables $M_{f}N$ and the number of constraints $1+M_{f}N+M_f$, the computational complexity of each iteration for solving (18) is $\mathcal{O}\left({\left(M_{f}N\right)}^2{(1+M_{f}N+M_f)}^{2.5}+{(1+M_{f}N+M_f)}^{3.5}\right)$. It is clear that the complexity of (18) is much lower than that of (14) because $M_f<M$ and only one MBS is considered.

Algorithm 3 SG-based PA for macrocells and small cells.

1: Initialize power matrix ${\mathit{P}}_{MBS}^{\left(0\right)}$ that meets constraints (17b), (17c), (17d), $k=0$ 2: while the convergence is not reached 3:    ${\mathit{P}}_{MBS}^{\left(k\right)}={\mathit{P}}_{MBS}^{(k-1)}$ 4:    for $f\in \mathcal{F}$ 5:       ${\mathit{P}}_f^{\left(tmp\right)}={\mathit{P}}_f^{\left(k\right)}$ 6:       while $U{T}_f({\mathit{P}}_f^{\left(tmp\right)},{\lambda }_f)>ϵ$ 7:          ${\lambda }_f=A_f\left({\mathit{P}}_f^{\left(tmp\right)}\right)/B_f\left({\mathit{P}}_f^{\left(tmp\right)}\right)$, ${\mathit{P}}_f^{\left(tmp\right)}={\mathit{P}}_f^{*}$ where ${\mathit{P}}_f^{*}$ is the solution of (18) 8:       ${\mathit{P}}_f^{\left(out\right)}={\mathit{P}}_f^{\left(tmp\right)}$ 9:    ${\mathit{P}}_{MBS}^{\left(k\right)}={\mathit{P}}_{MBS}^{\left(out\right)}$, $k=k+1$ 10: Initialize power matrix of SCDs ${\mathit{P}}_{cluster}^{\left(0\right)}$ that meets constraints (19b), (19c), (19d), $k=0$ 11: while the convergence is not reached 12:    ${\mathit{P}}_{cluster}^{\left(k\right)}={\mathit{P}}_{cluster}^{(k-1)}$ 13:    for ${\mathcal{C}}_i\in \mathcal{C}$ 14:       ${\mathit{P}}_{{\mathcal{C}}_i}^{\left(tmp\right)}={\mathit{P}}_{{\mathcal{C}}_i}^{\left(k\right)}$ 15:       while $U{T}_{{\mathcal{C}}_i}({\mathit{P}}_{{\mathcal{C}}_i}^{\left(tmp\right)},{\lambda }_{{\mathcal{C}}_i})>ϵ$ 16:          ${\lambda }_{{\mathcal{C}}_i}=A\left({\mathit{P}}_{{\mathcal{C}}_i}^{\left(tmp\right)}\right)/B\left({\mathit{P}}_{{\mathcal{C}}_i}^{\left(tmp\right)}\right)$, ${\mathit{P}}_{{\mathcal{C}}_i}^{\left(tmp\right)}={\mathit{P}}_{{\mathcal{C}}_i}^{*}$ where ${\mathit{P}}_{{\mathcal{C}}_i}^{*}$ is the solution of (20) 17:       ${\mathit{P}}_{{\mathcal{C}}_i}^{\left(out\right)}={\mathit{P}}_{{\mathcal{C}}_i}^{\left(tmp\right)}$ 18:    ${\mathit{P}}_{cluster}^{\left(k\right)}={\mathit{P}}_{cluster}^{\left(out\right)}$, $k=k+1$

5.2.2. NCG for Followers (Clusters)

We formulate optimization problem (9) as an NCG $NC{G}_{follower}=〈{\mathit{PL}}_{\mathcal{C}},{\mathit{P}}_{\mathcal{C}},u{l}_{\mathcal{C}}〉$, where ${\mathit{PL}}_{\mathcal{C}}=\mathcal{C}$ is the set of players (i.e., clusters), ${\mathit{P}}_{\mathcal{C}}$ is a matrix of PA for all clusters of SCDs, and $u{l}_{\mathcal{C}}\left({\mathcal{C}}_i\right)$ is the utility function of the EE of cluster ${\mathcal{C}}_i$. The PA problem for SCDs in (9) can be divided into multiple suboptimization problems by using NCG with players as clusters. A suboptimization problem of cluster ${\mathcal{C}}_i$ is as follows

$$\begin{array}{cc}\hfill \underset{{\mathit{P}}_{{\mathcal{C}}_i}}{\max }& {\displaystyle u{l}_{\mathcal{C}}\left({\mathcal{C}}_i\right)=\frac{{\sum }_{u\in {\mathcal{U}}_{{\mathcal{C}}_i}}{log}_2(1+{\gamma }_u)}{{\sum }_{b\in {\mathcal{C}}_i}{P}_b^{\mathrm{total}}}=\frac{A_{Ci}\left({\mathit{P}}_{{\mathcal{C}}_i}\right)}{B_{Ci}\left({\mathit{P}}_{{\mathcal{C}}_i}\right)}}\hfill \end{array}$$

(19a)

$$\begin{array}{cc}\hfill \mathsf{s}.\mathsf{t}.& {\sum }_{n=1}^N{\sum }_{u\in {\mathcal{U}}_{{\mathcal{C}}_i}}{p}_{u,b}^{\left(n\right)}\le P_{\mathrm{SCD}}^{max},\forall b\in {\mathcal{C}}_i,\hfill \end{array}$$

(19b)

$$\begin{array}{cc}\hfill & p_{u,b}^{\left(n\right)}\ge 0,\forall b\in {\mathcal{C}}_i,n=1,\dots ,N,\hfill \end{array}$$

(19c)

$$\begin{array}{cc}\hfill & {\gamma }_u\ge {\gamma }_{s min},\forall u\in {\mathcal{U}}_{{\mathcal{C}}_i}.\hfill \end{array}$$

(19d)

To obtain the best response of cluster ${\mathcal{C}}_i$, the power of the other clusters is fixed. This best response is the solution to optimization problem (19). We use the Dinkelbach method to solve the optimization problem (19) because its objective function is a concave–convex fraction. In detail, we introduce a new variable ${\lambda }_{{\mathcal{C}}_i}$, and the optimization problem (19) can be rewritten as

$$\begin{array}{cc}\hfill \underset{{\mathit{P}}_{{\mathcal{C}}_i}}{\max }& U{T}_{{\mathcal{C}}_i}({\mathit{P}}_{{\mathcal{C}}_i},{\lambda }_{{\mathcal{C}}_i})=A_{Ci}\left({\mathit{P}}_{{\mathcal{C}}_i}\right)-{\lambda }_{{\mathcal{C}}_i}{B}_{Ci}\left({\mathit{P}}_{{\mathcal{C}}_i}\right)\mathsf{s}.\mathsf{t}.\left(19\mathrm{b}\right),\left(19\mathrm{c}\right),\left(19\mathrm{d}\right).\hfill \end{array}$$

(20)

The objective function in (20) is a concave function, so the optimization problem (20) is a convex optimization problem. Therefore, the optimization problem (20) can be effectively solved by using CVX or CVXPY [34,35]. The optimal solution ${\mathit{P}}_{{\mathcal{C}}_i}^{*}$ is also the optimal solution to the optimization problem (20) if $U{T}_{{\mathcal{C}}_i}({\mathit{P}}_{{\mathcal{C}}_i}^{*},{\lambda }_{{\mathcal{C}}_i}^{*})=0$. We use Algorithm 3 to find the solution to the NCG $NC{G}_{follower}$. With the number of decision variables $U_{{\mathcal{C}}_i}{C}_{i}N$ and the number of constraints $C_i+U_{{\mathcal{C}}_i}{C}_{i}N+U_{{\mathcal{C}}_i}$, the computational complexity of each iteration for solving (20) is $\mathcal{O}\left({\left(U_{{\mathcal{C}}_i}{C}_{i}N\right)}^2{(C_i+U_{{\mathcal{C}}_i}{C}_{i}N+U_{{\mathcal{C}}_i})}^{2.5}+{(C_i+U_{{\mathcal{C}}_i}{C}_{i}N+U_{{\mathcal{C}}_i})}^{3.5}\right)$. The complexity of (20) is much lower than that of (16) since $U_{{\mathcal{C}}_i}<U$ and $C_i<B$.

6. Simulation Results

Next, we investigated the performance of the proposed methods in a two-tier UDN. Three hexagon macrocells with a coverage radius of 100 m were deployed next to each other. In each macrocell, one MBS equipped with 128 antennas was located in the center of the cell. In addition, many small cells that had a radius of 20 m were randomly distributed by PPP. A small cell had an SCD equipped with one omnidirectional antenna. The UEs in the coverage of any small cell were served by the cluster that contained that small cell, and the others were served by theMBSs. The parameters in the simulations are given in

Table 3. Simulation parameters.

Parameter	Numerical Value
Carrier frequency/Total bandwidth	2 GHz/20 MHz
Bandwidth per subchannel	4 MHz
# subchannels	5
Path loss from MBSs to UEs	$128.1+37.6{log}_{10}d$ [dB], d in km
Path loss from SCDs to UEs	$140.7+36.7{log}_{10}d$ [dB], d in km
Shadowing standard deviation	8 dB
Noise power density	−174 dBm/Hz
Maximum transmit power of MBS	46 dBm
Maximum transmit power of SCD	30 dBm
Circuit power of MBS	$20\%P_{MBS}^{max}$
Circuit power of SCD	$20\%P_{SCD}^{max}$
The radius of a macrocell	100 m
The radius of a small cell	20 m
# transmit antennas per MBS	128
The minimum throughput per MUE	0.1 Mbps
The minimum throughput per SUE	1 Mbps

[36]. Additionally, we introduced two more clustering methods for comparison. In the traditional clustering method, each SUE is served by the nearest SCD. Meanwhile, with random clustering, each SUE randomly connected to an SCD, which had a distance to this SUE of less than $d=3\times$ the radius of small cells. On the other hand, equal power allocation, where the power of $P_{MBS}^{max}/2$ of each MBS is equally divided among its MUEs and the power $P_{SCD}^{max}$ of each active SCD is equally divided among the subchannels, was also used for comparison with the EE maximization PA methods. To prove the performance improvement, the proposed methods were compared with conventional methods based on multiple aspects such as convergence speed of PA methods, energy efficiency, sum rate, and transmit power usage. The acronyms for the combinations of clustering and PA methods are given in

Table 4. ID of strategies for comparison.

ID	Clustering	PA for Macrocells	PA for Small Cells
CGCO	Coalition Game	Centralized optimization	Centralized optimization
CGDO	Coalition Game	Game-based optimization	Game-based optimization
TCDO	Traditional clustering	Game-based optimization	Game-based optimization
RCDO	Randomly clustering	Game-based optimization	Game-based optimization
TCEP	Traditional clustering	Equal power allocation	Equal power allocation

. To solve the optimization problems and build simulating programs, CVXPY and PYTHON were used [35]. The computing platform was a PC with CPU@3.7 GHz and 32 GB RAM memory.

Figure 4 illustrates an example of a triple-macrocell UDN that combines 100 SUEs and 24 MUEs with a density of SCDs of 2000 SCDs/km${}^2$. In the first close-up in Figure 4a, two adjacent small cells use two same subchannels to serve their SUEs. This causes high interference for these SUEs. Meanwhile, after using the coalition game, these two small cells form a cluster to cooperatively transmit (the first close-up in Figure 4b). Therefore, there is no interference in a cluster. Another advantage of clustering that is shown in the second close-up is to enhance the quality of the desired signal. The SUE (red) subchannel is served by two small cells tgat have almost the same distance to it instead of being served by a single cell. When the density of SCDs is very high, these situations occur many times. Thus, clustering SCDs based on coalition game, and cooperative transmission help to efficiently mitigate intercell interference and improve the power of the received signal.

6.1. Convergence Speed of PA Methods

In this subsection, we consider the convergence of CGCO and CGDO in the network that has an SCD density of 2000 SCDs/km${}^2$, 100 SUEs, 3 MBSs, and 24 MUEs. The changes and differences in the quantity of the EE of these methods, along with the iterations in the macrocell tier and small-cell tier, are shown in Figure 5a,b, respectively. It is given that one iteration of GT methods is complete after updating the best responses of all players. At the convergence, the EE of the CGDO is always lower than that of the CGCO. In detail, the EE of the CGDO is lower than that of CGCO in the macrocell tier and small-cell tier by 0.65% and 15.39%, respectively. However, the CGDO almost converges to the equilibrium after only one loop for updating all the best responses of the players, while it takes three and four iterations in the macrocell tier and small-cell tier, respectively, for the CGCO to achieve an EE of more than 95% at the convergence. On the other hand, in Figure 5b, the EE of the CGDO after one iteration is greater than the value of the convergence. This is reasonable because, in game theory, the value of the utility function at the Nash equilibrium may not be the best solution.

6.2. Performance Analysis of CGCO, CGDO, and TCDO

We evaluated the performance of the proposed methods regarding the EE, the sum rate, the total transmit power, and the execution time in different scenarios with the number of network elements that are given in

Table 5. The different network size scenarios.

Density (SCDs/km${}^{\mathbf{2}}$)	500	600	700	800	900	1000
# MBSs	3	3	3	3	3	3
# MUEs	9	12	15	18	21	24

, and the number of SUEs equaled 60% of the number of SCDs.

Figure 6 displays data on the changes in the quantity of the EE of CGCO, CGDO, TCDO, RCDO, and TCEP in the small-cell tier when the density of SCDs increases from 500 to 1000 SCDs/km${}^2$. To overview, the EE of all methods witnesses a decrease with different levels since the denser the deployment of small cells is, the higher interference is. The EE of CGCO is greatest in both the five methods since it uses the centralized PA method. In detail, the EE of CGDO equals 74.09% of the EE of CGCO on average. Meanwhile, the EE of CGDO is greater from 11.15% to 17.23% than the one of TCDO in all testing scenarios. In addition, CGCO, CGDO, and TCDO outperform RCDO and TCEP in terms of the EE. Therefore, using the coalition game for clustering small cells before SCA and PA based on distributed method helps improve the efficiency of energy usage in UDNs compared with the traditional clustering and the random clustering.

Along with the EE, the computing execution time is an important factor to evaluate the performance of different methods.

Table 6. The average execution time for solving one optimization problem of PA methods for different network sizes.

Density (SCDs/km${}^{\mathbf{2}}$)	500	600	700	800	900	1000
CGCO MC (ms)	113	129	148	163	183	195
CGCO SC (ms)	243	284	323	364	379	411
CGDO MC (ms)	157	165	168	176	183	180
CGDO SC (ms)	125	124	124	122	115	118
TCDO MC (ms)	160	176	171	176	180	180
TCDO SC (ms)	118	112	114	109	103	104

describes the data in detail regarding the time required for the PA of CGCO, CGDO, and TCDO. Thanks to (12), the intensity of interference from small cells is pre-estimated in the PA optimization problems of the macrocell tier. Therefore, the execution time of the PA methods in the macrocell tier does not depend on the density of SCDs. On the other hand, the execution time for the PA of the MBSs of CGDO and TCDO is higher than the one of CGCO when the number of MUEs is small. The reason is that optimization problem (18) with the objective, a logarithmic function, takes a longer time to solve than optimization problem (16) with the objective as the sum of affine functions and rational functions when the number of variables is slightly different. However, the execution time in macrocells is less important than that in small cells with much more variables in the OPs. Considering the small-cell tier, when the density of the SCDs changes from 500 to 1000 SCDs/km${}^2$, the execution time of the CGCO significantly increases by 1.7 times, while those of the distributed methods CGDO and TCDO do not depend on the number of SCDs. Additionally, the execution time of CGCO is always longer, from 1.95 to 3.49 times that of CGDO. In large-scale UDNs, quick adaptation is key. If a very complex algorithm is used and the execution time is long, when the solution is obtained, it is not optimal anymore because the channel and user locations have changed too much. Therefore, the trade-off between EE and a quick execution time using our proposed PA method is essential and effective in UDNs.

Figure 7 shows the average of the sum rate and the total power consumption of CGCO, CGDO, and TCDO in the macrocell tier of UDNs. The level of the sum-rate growth of the three methods gradually decreases when the number of small cells increases in a certain area. The sum rate of these methods increases and reaches the maximum at a density of 900 SCDs/km${}^2$. Then, it gradually decreases when densities are greater than 900 SCDs/km${}^2$ despite the increased power usage because MUEs experience extreme interference from the small-cell tier. To clearly identify this trend, we added two more scenarios with densities of 1100 and 1200 SCDs/km${}^2$, and we set the number of MUEs to 24. Additionally, in comparison with traditional clustering, SCDs were used in more subchannels for transmitting the signals to both their primal SUEs and the other SUEs in their clusters. This causes more interference from small cells to macrocells. Therefore, MBSs with CGDO use more power to achieve a higher sum rate than those of TCDO.

Figure 8 describes the sum rate and the total transmit power of the three methods, namely CGCO, CGDO, and TCDO, in the small-cell tiers of UDNs with different network sizes. When the density of the SCDs increases, the energy consumption considerably increases, while the rate of increase in the sum rate declines. Due to using centralized optimization, the data rate of CGCO is always larger than those of the others, while the total transmit power of CGCO is the lowest for the three methods. The power consumption of CGDO is lower by 9.51% to 15.19% than that of TCDO. On the other hand, when the deployment of SCDs becomes denser, to achieve the same data rate, the SCDs use much more power. For example, in Figure 8a, the sum rates, which CGDO and TCDO achieve with small cells with densities of 800 and 900 SCDs/km${}^2$, are nearly equal. Meanwhile, the power consumption of CGDO and TCDO significantly increases in this range of density, as shown in Figure 8b. Moreover, the data rate of CGDO is slightly higher than that of TCDO. To summarize, the clustering method based on the coalition game helps to achieve a higher data rate than the traditional clustering method with lower power consumption.

6.3. Execution Time of Coalition Game (Algorithm 1) and the Hungarian Method (Algorithm 2)

Using the same simulation parameters as in Section 6.2, we evaluate the execution time of Algorithms 1 and 2 in

Table 7. The execution time of the clustering-method-based coalition game and SCA-based Hungarian method for different network sizes.

Density (SCDs/km${}^{\mathbf{2}}$)	500	600	700	800	900	1000
Coalition game ($\mathsf{\mu }$s)	282.2	309.1	426.7	543.4	586.9	1073.8
The Hungarian method ($\mathsf{\mu }$s)	542.5	734.5	816.4	925.8	976.6	1688.6

. It is clear that the execution times of the coalition game and the Hungarian method increase with the increasing density of SCDs for the different levels. In detail, from 500 to 1000 SCDs/km${}^2$, the execution time of the coalition game increases 3.8 times and that of the Hungarian method increases 3.1 times. The computational complexity of the Hungarian method is $O\left(N^4\right)$ in the worst case for the SCA in each cluster. Thus, its computational time has a linear relationship with the number of clusters in the small-cell tier.

7. Conclusions

We investigated the problem of resource allocation for UDNs with two tiers consisting of multiple MBSs and massive randomly distributed drones in this study. We proposed a paradigm that combines three processes: clustering using the coalition game, SCA using the Hungarian method, and PA using the SG as a distributed optimization method. Our simulations proved that the execution time of the centralized method for PA is much higher, by 1.95 to 3.49 times, than that of our distributed method, while the network EE of the proposed method using clustering based on the coalition game is improved by 11.15% to 17.23% compared with that of the distributed one with traditional clustering. Therefore, the centralized method is more suitable for UDNs with a small number of network elements, while the distributed method can be efficiently employed in large-scale UDNs where many MBSs and drones communicate simultaneously. Moreover, real-time computing plays an important role in drone-aided UDNs due to the mobility of both base stations (i.e., drones) and users. The execution time of the coalition game and the Hungarian method showed that they can be applied to real-time systems in UDNs.

For future work, with a massive number of users, data traffic is extremely high in drone-aided networks. To avoid network data congestion, drones equipped with caching storage, which can prestore common data packets, are a promising solution. However, the optimization problem for caching is integer programming with a large number of decision variables. Therefore, low-complexity algorithms for solving caching problems with the objective of minimizing the risk of network congestion are essential to be considered as an extension of this method. Additionally, to guarantee the quality of the signal received by users, drone-to-user communications need to be line of sight. Thus, the trajectory design for multiple drones in order to adapt to the movements of users is also an open topic to be addressed in the future.