Machine Learning Approaches for Sharing Unlicensed Millimeter-Wave Bands in Heterogeneously Integrated Sensing and Communication Networks

Abstract

Due to the increasing demand of high data rate, spectrum scarcity is a key problem for providing unprecedented capacity in diversified applications for future wireless networks. Therefore, the efficiently shared use of unlicensed bands is one of the promising solutions for addressing the spectrum scarcity issue. We study decentralized machine learning approaches using the paradigm of integrated sensing and communication (ISAC) for the shared use of unlicensed millimeter-wave bands. We first present a 5G–WiFi fusion protocol stack for sharing unlicensed millimeter-wave bands, and then design an ISAC-based access protocol and an ISAC based coexistence protocol integrated with decentralized learning function to achieve the efficiently shared use of unlicensed bands. Using the coexistence protocol, we propose promising decentralized machine learning approaches to share unlicensed millimeter-wave bands. Finally, simulations are provided to verify the performance of the proposed scheme, where the results have shown that the proposed scheme greatly reduces the search space of the solution and effectively protects the communication performance of the WiFi system compared to traditional schemes, which indicates that simultaneous transmissions of 5G-U and WiFi at the 60 GHz band are feasible under the proposed scheme.

1. Introduction

With the proliferation of various smart devices, the traffic of wireless data services will increase by 10,000 times by the year 2030 [1,2,3]. In this context, spectrum scarcity is a key challenge to provide such an unprecedented capacity for massive wireless applications such as industrial internet of things [4,5,6,7]. In order to meet the surging traffic demand, it is generally believed that the capacities of future wireless networks can be enhanced based on three dimensions, i.e., improving spectrum efficiency, increasing cell density and expanding the spectrum. However, due to the limitations of physical layer technologies, the capacity improvement from the spectral efficiency enhancement is limited. Therefore, expanding the spectrum with the network densification is a more direct way to enhance the system capacity. Although licensed bands have good transmission characteristics, they are very crowded and thus cannot meet the demand of high rate and large bandwidth in future wireless communication networks. Consequently, unlicensed bands have been recently proposed to be used by cellular systems for improving the capacities of future wireless networks. Nevertheless, the cellular system that accesses the same unlicensed band may degrade the performance of existing systems, such as the WiFi system. Therefore, to make full use of unlicensed bands, the efficiently shared use of unlicensed bands by multiple systems under the premise of protecting the performance of existing systems is of critical importance to enhance the overall capacities of future wireless networks.

Recently, many research efforts have been made to investigate the shared use of unlicensed bands from both the academic and industry communities. To be specific, LTE-Advanced has been suggested for deployment in the unlicensed band of 5 GHz as supplemental downlink by Qualcomm and Ericsson, which is known as LTE-Unlicensed (LTE-U) [8]. Furthermore, since 2014, Huawei, NTT and DoCoMo have also been studying licensed-assisted access through LTE (LAA-LTE). By deploying pre-commercial multi-cell networks, all the above companies have demonstrated that LTE-U/LAA-LTE has better capacity and coverage performance at the unlicensed band of 5 GHz than WiFi alone. The analytical and experiment results in reference [9] have also shown that LTE-U users are able to manage interference effectively through coordinating with existing users, and thus can achieve harmonious coexistence with existing unlicensed users. In reference [10], the channel allocation problem between LTE-U and device-to-device users, for sharing both unlicensed and licensed bands, has been studied considering the co-channel interference, where the results have shown that the overall capacity can be largely improved when unlicensed bands are shared by multiple systems. In reference [11], a coexistence mechanism was proposed to manage the backoff of LTE-U and WiFi for sharing the 5 GHz band, which has a performance superior to the listen-before-talk and carrier sense adaptive transmission. In reference [12], a survey of spectrum sharing technologies in LTE was presented, where the television (TV) white-space band, 3.5 GHz and 5 GHz unlicensed bands were considered. In reference [13], the joint interference coordination mechanism and spectrum sharing protocol in unlicensed bands were designed, where the results indicate that they can enhance the throughput of LTE-U and satisfy the traffic needs of 802.11 users. In reference [14], a duty cycle adjustment approach was designed to dynamically configure transmission gaps to ensure better coexistence between LTE and WiFi, where the results have indicated that the proposed approach can improve energy efficiency and throughput. In reference [15], an investigation was conducted on Internet of Things technologies deployed within unlicensed spectrum and licensed cellular, such as shared spectrum management and interference management. In reference [16], a matching-based unlicensed and licensed bands allocation scheme was proposed to optimize the cellular users’ utilities as well as guarantee the rate constraints for both WiFi and cellular users. In reference [17], joint resource management and a user association problem were formulated to maximize the number of QoS-preferred users, where a listen-before-talk-based flexible coexistence framework was proposed. In reference [18], a blockchain-based approach was designed for achieving secure spectrum sharing for machine-to-machine and human-to-human coexistence in heterogeneous networks, where a comprehensive architecture overview of the proposed framework and its implementation procedures were provided. In reference [19], the LTE/WiFi spectrum sharing problem, to maximize the LTE throughput while protecting the WiFi system, was investigated, where the simulation results show that, even without signalling exchanges, the proposed scheme has a similar performance to that of the genie-aided exhaustive search algorithm. In reference [20], the integrated use of 28GHz and 60GHz bands over 5G Unlicensed (5G-U) was proposed, and strategies of spectrum management in 5G-U were also discussed, where the results show that the harmonized utilization of unlicensed and licensed bands can largely enhance the performance in both network-centric and device-centric metrics. In reference [21], a data-driven spectrum-sharing scheme among multiple operators was designed to enhance the spectrum utilization in millimeter-wave (mmWave) cellular networks. In reference [22], clustering-based spectrum-sharing schemes with lower overhead and complexity were proposed to improve the sum rate for a multi-operator mmWave cellular network.

However, most of previous works concentrate mainly on the shared use of unlicensed bands with lower frequency. As the unlicensed bands with lower frequency are becoming more crowded, there is an increasing need for utilizing unlicensed mmWave bands that have a potentially large bandwidth. Although existing works [21,22] investigated the mmWave spectrum sharing problem, they mainly focused on sharing the licensed mmWave bands between homogeneous systems. Therefore, we considered the spectrum sharing in V-Band (57–64 GHz) mainly for the following reasons. First, although the 28 GHz and E-Band (71–76 GHz and 81–86 GHz), that can be used for the 5G mmWave cellular networks, offer huge spectrum availability in the order of some GHz, they may be insufficient to support the ever-growing bandwidth-hungry scenarios, which require data rates on the order of up to tens of Gbit/s, such as massive drone surveillance, autonomous driving, Virtual Reality (VR), Augmented Reality (AR), Synchronized Reality (SR), and Mixed Reality (MR)[23]. Second, existing wireless systems like IEEE 802.11ad and IEEE 802.11ay have worked on V-Band, however, other communication systems may join in the future due to the unlicensed feature. Therefore, if future cellular systems want to utilize V-Band, efficient spectrum sharing is the best way to enhance the spectral efficiency and protect the performance of existing systems.

Despite the many advantages of the shared use of unlicensed mmWave bands, it is still facing some technical challenges. Firstly, due to the different medium access control (MAC) mechanisms, coexistent systems that share the same unlicensed band cannot implement information interaction, which makes it difficult to share the spectrum efficiently. Secondly, since coexistent systems are generally not able to acquire full knowledge of a wireless environment and complete information on the opposite actions, most existing schemes, relying on the assumptions that coexistent systems have global information of the wireless environment, will be infeasible in practice. Finally, with the increasing of network density, centralized solutions will bring significant signaling overheads and, thus, are less efficient despite their optimality. For these reasons, one key obstacle to achieving the efficiently shared use of unlicensed bands is how to design an efficient coexistence protocol and efficiently decentralized learning algorithms for spectrum sharing. Fortunately, the emergence of the paradigm of integrated sensing and communication (ISAC), that is regarded as an emerging technology towards 6G and next generation WLAN [24], provides potential solutions for coexistence and information exchange between heterogeneous systems. On the other hand, the decentralized machine learning theory provides a mathematical tool for studying scenarios with incomplete information in wireless environments, thereby avoiding the heavy signaling overhead brought on by the frequent exchange of global information [25,26,27]. Therefore, ISAC and decentralized learning approaches can be applied to the shared use of unlicensed bands for obtaining efficient solutions. The scope of this article is thus to study integrated ISAC and decentralized learning approaches for the shared use of unlicensed mmWave bands.

Motivated by these challenges, in this paper, different from existing works [21,22] that ignored mmWave spectrum sharing between heterogeneous systems, we study the unlicensed mmWave bands sharing problem in heterogeneous networks, to maximize the sum rate of all cellular user equipments (CUEs) as well as protecting the performance of each piece of WiFi user equipment (WUE). The contributions are summarized as follows.

• We present a fusion MAC framework for 5G-U systems, based on which we design an ISAC-based access protocol and an ISAC-based coexistence protocol for CUEs and WUEs to efficiently share unlicensed mmWave bands.
• We investigate the unlicensed mmWave bands sharing problem for maximizing the sum rate of CUEs, with consideration of the interference constraints of each WUE and the limitation of radio frequency (RF) links for each mmWave gNB in heterogeneous networks.
• We formulate the considered problem as a potential game and prove the existence of the Nash equilibrium (NE) for the proposed game, and then propose five decentralized machine learning algorithms to obtain the NE of the proposed game.
• We provide extensive simulations to evaluate the performances of the proposed scheme from various aspects, where the results have demonstrated the effectiveness of the proposed scheme, and have shown that the proposed scheme can enhance the spectrum utilization and reduce the search space for the considered problem, as well as effectively guarantee the performance of the existing system.

The rest of this article is arranged as follows. In Section 2, the access protocol, coexistence protocol, system model and problem formulation are presented. The proposed potential game is analyzed in Section 3. In Section 4, decentralized machine learning solutions to the shared use of unlicensed bands are designed. In Section 5, a performance evaluation is provided. The conclusions are drawn in Section 6.

2. Coexistence Protocol, System Model and Problem Formulation

2.1. Coexistence Protocol for Sharing Unlicensed Bands

There are a lot of unlicensed bands utilized for various applications, such as the very crowded 900 MHz and 2.4 GHz bands, 3.5 GHz, 5 GHz and 60 GHz bands. More specifically, the band at 900 MHz is currently utilized by cordless phones and ZigBee. The bands at 2.4 GHz and 5 GHz are proposed to be used by the LTE technology as a secondary carrier [9], which are currently also used for Bluetooth and WiFi. The 3.5 GHz band, which is mainly used for satellite and military communications, has been also proposed to be used for small cell deployment scenarios. The 60 GHz bands, which belong to mmWave bands, are well suited for short distance communications such as peer-to-peer communications, 5G cellular and Gbps WiFi (e.g., IEEE 802.11ad/802.11 ay). Since 900 MHz and 2.4 GHz bands are very crowded, the unlicensed bands can be more likely shared by the cellular system, and other communication systems are mainly 3.5 GHz, 5 GHz and 60 GHz bands. The characteristics of these unlicensed bands vary greatly, and are listed in

Table 1. Comparison of different unlicensed bands.

Unlicensed	Spectrum	Bandwidth	Degree of	Incumbent	Coexistence
Bands	Range		Congestion	Users	Mechanisms
900 MHz	890 ∼ 960 MHz	50 MHz	High	Cordless phones	Transmit power control
				IEEE 802.15.4	Channel allocation
				Zigbee	Transmission duration control
2.4 GHz	2.4∼ 2.5 GHz	83.5 MHz	Very high	IEEE 802.11 WLAN	Listen-before-talk
				Bluetooth	Emission mask
				Zigbee	Maximum transmit power
3.5 GHz	3.55∼3.7 GHz	150 MHz	Low	Naval Radar	Dynamic frequency selection
				Fixed Satellite systems	Transmit power control
5 GHz	5.15∼5.925 GHz	495 MHz	Low	Radiolocation	Listen-before-talk
				Earth Exploration	Dynamic frequency selection
				Space Research	Transmit power control
60 GHz	57∼64 GHz	7 GHz	Low	IEEE 802.11ad/ay	Listen-before-talk
					Spatial reuse
					Transmit power control

. The major technologies for the shared use of unlicensed bands to improve utilization efficiency fall into four aspects, i.e., user association, channel allocation/selection, power control and transmission duration control. Without a loss of generality, we concentrate on the shared use of unlicensed mmWave bands by user association and power control, however, the proposed algorithms can be easily expanded to channel allocation/selection and transmission duration control.

Inspired by reference [13], we design a 5G–WiFi fusion protocol stack, as shown in Figure 1, which consists of a cellular part and a WiFi part where each part includes an independent RF unit. The 5G–WiFi fusion protocol stack has the following functions: (i) the WiFi part of the 5G–WiFi fusion protocol stack detects and decodes the physical layer (PHY) frame from WiFi systems in order to obtain the interference level from each CUE link to each WUE link; (ii) the WiFi part of the 5G-WiFi fusion protocol stack reports this interference information to the cellular part, and then the cellular part informs all CUEs of this inference information. Using the 5G–WiFi fusion protocol stack, we design an access protocol for CUEs and WUEs to share the unlicensed mmWave bands, as shown in Figure 2, where CUEs first use beams embedded in orthogonal sequences with fixed transmission power to conduct beamforming training, and, at the same time, the WiFi AP can sense and detect the interference level from each cellular link to each WiFi link. After that, the WiFi AP broadcasts the PHY frame, which includes the detected interference information. The WiFi part of the 5G–WiFi fusion protocol detects and decodes the PHY frame from the WiFi AP, then reports the result to the cellular part of the 5G–WiFi fusion protocol. The cellular part of the 5G gNB informs all CUEs of this inference information, based on which of the CUEs first estimates the channel gains from their available beam association directions to WiFi transmission links and then adjust their respective transmission strategy space to avoid strong inference to WiFi transmission links. After that, all CUEs perform decentralized machine learning to learn the joint user association and transmission power selection scheme, based on which all cellular links transmit data. Next, we propose a coexistence protocol with decentralized learning function for the shared use of unlicensed millimeter-wave bands, on which many learning algorithms can be run to obtain the solutions. In particular, motivated by reference [10], we design an integrated sensing- and communication-based coexistence protocol integrated with decentralized learning function for 5G-U and WiFi, coexisting as shown in Figure 3, where the full duty cycle includes sensing subframes (SSs), reserved WiFi subframes and sharing transmission subframes. The sensing subframe is used to find the cleanest channel to avoid collisions with on-going transmissions as well as to learn the optimal action for each transmission link, and the reserved WiFi subframe is used for the exclusive transmission of WiFi to protect the WiFi performance, while the sharing transmission subframe is designed for simultaneous transmissions of 5G-U and WiFi to improve the spectrum efficiency because of the short transmission range of mmWave communications. Compared to the traditional centralized scheduling scheme, the proposed scheme can not only be used for spectrum sharing in sub 6GHz bands, but also can be utilized for adaptively selecting the transmission strategy according to the changes of the network under highly dynamic interference environments.

2.2. System Model and Problem Formulation

Consider an uplink heterogeneous network including N cellular gNBs (i.e., 5G NodeB), L WiFi access points (APs), U CUEs and W WUEs, where the sets of gNBs, APs, CUEs and WUEs can be expressed as $\mathcal{N}=\{1,2,\dots ,N\}$, $\mathcal{L}=\{1,2,\dots ,L\}$, $\mathcal{U}=\{1,2,\dots ,U\}$ and $\mathcal{W}=\{1,2,\dots ,W\}$, respectively. The channel gain from CUE u to gNB n, including the large-scale and small-scale fading, is expressed as

$$\begin{array}{c}\hfill g_{u,n}^c=h_{u,n}^2{10}^{-\frac{{\overline{L}}_{u,n}}{10}},\end{array}$$

(1)

in which ${\overline{L}}_{u,n}$ represents the average path loss between gNB n and CUE u in dB, and $h_{u,n}$ is the channel gain of small-scale fading. In the same way, we can obtain the interference channel gains between CUEs and APs, and between WUEs and gNBs. For the blockage model of the link from CUE u to gNB n, the model in reference [28,29] is considered, and then the line-of-sight (LOS) communication probability for the transmission link with length $d_{u,n}$ can be defined as

$$\begin{array}{c}\hfill P{r}_{u,n}^{los}=\exp (-\alpha d_{u,n}),\end{array}$$

(2)

where $d_{u,n}$ represents the distance from CUE u to gNB n, and $\alpha$ is used to capture the density and size of obstacles. The path loss between CUE u and gNB n in dB at 60 GHz frequency for LOS and for non-line-of-sight (NLOS) can be given by [30]

$$L_{u,n}=\left\{\begin{array}{c}68+21.7\log \left(d\left[m\right]\right)+{\eta }_{LOS},\mathrm{for}\mathrm{LOS}\mathrm{link}\hfill \\ 68+30.1\log \left(d\left[m\right]\right)+{\eta }_{NLOS},\mathrm{for}\mathrm{NLOS}\mathrm{link},\hfill \end{array}\right.$$

(3)

where ${\eta }_{LOS}\in \mathcal{CN}(0,0.88)$ and ${\eta }_{NLOS}\in \mathcal{CN}(0,1.55)$.

Then, the average path loss between gNB n and CUE u can be given by

$$\begin{array}{c}\hfill {\overline{L}}_{u,n}=P{r}_{u,n}^{los}{L}_{u,n}^{los}+(1-P{r}_{u,n}^{los})L_{u,n}^{nlos}.\end{array}$$

(4)

In this work, we use the sector antenna model in reference [31,32,33,34,35] to approximate the antenna gain. Let ${\varphi }_{u,n}^t$ and ${\varphi }_{u,n}^r$ be the beamwidths of the transmitter u and receiver n, respectively. The transmission gain between the transmitter u and receiver n can be expressed as

$$\begin{array}{c}\hfill g_{u,n}^t\left({\varphi }_{u,n}^t\right)=\left\{\begin{array}{cc}\frac{2\pi -(2\pi -{\varphi }_{u,n}^t)z}{{\varphi }_{u,n}^t},\hfill & \mathrm{in}\mathrm{the}\mathrm{main}\mathrm{lobe},\hfill \\ z,\hfill & \mathrm{in}\mathrm{side}\mathrm{lobes},\hfill \end{array}\right.\end{array}$$

(5)

in which $0\le z<1$ denotes the gain of the side lobe. By replacing ${\varphi }_{u,n}^t$ with ${\varphi }_{u,n}^r$, the reception gain $g_{u,n}^r\left({\varphi }_{u,n}^r\right)$ between receiver n and transmitter u can be similarly obtained.

Then, the signal to interference and noise ratio (SINR) of the signal from CUE u to gNB n at the receiving end is given by

$$\begin{array}{c}\hfill {\mathrm{SINR}}_{u,n}=\frac{p_u{g}_{u,n}^t{g}_{u,n}^c{g}_{u,n}^r}{{\displaystyle \sum _{k\in \mathcal{U}\setminus u}{p}_k{g}_{k,n}^t{g}_{k,n}^c{g}_{k,n}^r+I_{u,n}^{wifi->cellular}+B{N}_0}},\end{array}$$

(6)

where ${\displaystyle I_{u,n}^{wifi->cellular}=\sum _{w\in \mathcal{W}}{p}_w{g}_{w,n}^t{g}_{w,n}^c{g}_{w,n}^r}$ represents the experienced interference of the link between CUE u and gNB n from the WiFi system. Note that here we assume that, during the uplink transmissions of CUEs, the WUEs also perform uplink transmissions for simplicity. However, our approach can be easily extended to the hybrid transmission scenario with both uplink and downlink transmissions for the WiFi network.

Then, the achievable rate of CUE u in the uplink transmission can be given by

$$\begin{array}{c}\hfill {\displaystyle R_u=B\sum _{n\in \mathcal{N}}{x}_{u,n}{\log }_2(1+{\mathrm{SINR}}_{u,n}),}\end{array}$$

(7)

where $x_{u,n}$ is the binary selection variable, and $x_{u,n}=1$ if CUE $\mathit{u}$ chooses gNB $\mathit{n}$ as the saving base station, and otherwise $x_{u,n}=0$.

In order to maximize the sum rate of all CUEs considering the interference constraints of all WUEs, the unlicensed mmWave bands sharing problem in heterogeneous networks can be modeled as

$$\begin{array}{ccc}\hfill & & {\displaystyle \mathbf{P}1:\underset{\begin{array}{c}\mathit{x},\mathit{p}\end{array}}{max}\sum _{u\in \mathcal{U}}{R}_u}\hfill \\ \hfill \mathrm{s.t.}& & \mathrm{C}1:{\sum }_{u\in \mathcal{U}}{x}_{u,n}\le N_n^{RF},\forall n,\hfill \\ \hfill & & \mathrm{C}2:{\sum }_{n\in \mathcal{N}}{x}_{u,n}=1,\forall u\in {\mathcal{U}}_1,\hfill \\ \hfill & & \mathrm{C}3:x_{u,n}=\{0,1\},\forall u\in \mathcal{U},n\in \mathcal{N},\hfill \\ \hfill & & \mathrm{C}4:I_w^{cellular->wifi}\le I_w^{thr},\forall w\in \mathcal{W},\hfill \\ \hfill & & \mathrm{C}5:p_u\in {\mathcal{P}}_u,\forall u\in \mathcal{U},\hfill \end{array}$$

(8)

where $\mathit{x}=\{x_{u,n},u\in \mathcal{U},n\in \mathcal{N}\}$, $\mathit{p}=\{p_u,u\in \mathcal{U}\}$ represent the gNB selection policy profile and transmission power policy profile, respectively. Constraints C1, C2 and C3 indicate that each gNB can, at most, serve $N_n^{RF}$ CUEs simultaneously because of the limited number of RF chains, and each CUE can select one gNB as its serving base station at each time, Constraint C4 ensures that the interference from all CUEs to each WUE must be less than a predetermined interference threshold for avoiding serious interference to each WUE. Constraint C5 provides transmission power limitation constraints, where the power of each CUE $u,\forall u\in \mathcal{U}$ could be selected from the predetermined set ${\mathcal{P}}_u$.

3. Potential Game for Sharing Unlicensed Millimeter-Wave Bands

Since CUEs generally maximize their respective utilities in the decentralized learning environment, the learning process can be modeled as a noncooperative game $\mathcal{G}=[\mathcal{U},{\left\{{\mathcal{A}}_u\right\}}_{u\in \mathcal{U}},{\left\{U_u\right\}}_{u\in \mathcal{U}}]$, where $\mathcal{U}$, ${\mathcal{A}}_u$ and $U_u$ denote the set of CUEs (i.e., players), the action set of CUE u and the utility of player u, respectively. As we mainly focus on jointly optimizing the gNB selection and transmission power allocation of CUEs for the coexisting of cellular and WiFi systems, the action of CUE u is the combination of available transmission power levels and available candidate gNBs, and can be denoted by $a_u=(x_{u,n},p_u),\forall u\in \mathcal{U}$, where n is one of the available gNBs for CUE u. However, the proposed learning framework is also applicable to optimize other radio resources, such as channel selection, transmission duration or their combinations. To protect the WiFi performance, the strategy space of each CUE should be constrained to a reasonable range. We assume that each CUE can obtain the interference experienced by each WUE from the cellular system according to the proposed 5G–WiFi fusion protocol stack and the designed access protocol for sharing the unlicensed mmWave bands. For simplicity, we distribute the interference threshold of each WUE to each CUE according to the interference proportion generated by each CUE. Therefore, if the CUEs in $\mathcal{M}$ share the unlicensed mmWave band with WiFi link w, to ensure that the tolerable interference of all 5G-U links to WUE w should be no more than ${\overline{I}}_w^{thr}$, which denotes the tolerant interference of WiFi link w, the interference borne by each CUE $u\in \mathcal{M}$ can be expressed as

$$\begin{array}{cc}\hfill & {\widehat{I}}_u^{cellular->wifi}\hfill \\ \hfill =& \frac{{\overline{I}}_{u->w}^{cellular->wifi}}{{\displaystyle \sum _{u\in \mathcal{M}}{\overline{I}}_{u->w}^{cellular->wifi}}}({\overline{I}}_w^{thr}-I_w^{wifi->wifi})\hfill \\ \hfill =& \frac{{\overline{I}}_{u->w}^{cellular->wifi}}{I_w^{cellular->wifi}}({\overline{I}}_w^{thr}-I_w^{wifi->wifi}),\hfill \end{array}$$

(9)

where ${\overline{I}}_{u->w}^{cellular->wifi}$ represents the average interference of all available beam association directions for CUE u to WUE w. Based on the estimated channel gains from its available beam association directions to WUE w, each CUE $u\in \mathcal{M}$ should exclude the strategies that will result in an interference larger than ${\widehat{I}}_u^{cellular->wifi}$. Of course, if there is more than one WiFi link sharing the unlicensed mmWave band with the CUEs in $\mathcal{M}$, the above process can be repeated to further exclude the improper strategies for each CUE.

Following references [32,33], the utility of UE u is given by

$$\begin{array}{c}\hfill {\displaystyle {\tilde{U}}_u=U_u+\sum _{k\in {\mathcal{U}}_u}{U}_k,}\end{array}$$

(10)

with

$$\begin{array}{c}\hfill {\displaystyle U_u=R_u+\eta \sum _{n\in \mathcal{N}}\{x_{u,n}{\Delta }_n\Phi (N_n^{RF},\sum _{u\in \mathcal{U}}{x}_{u,n})\}}\end{array}$$

(11)

and

$$\begin{array}{c}\hfill {\displaystyle {\Delta }_n=\sum _{u\in \mathcal{U}}{x}_{u,n}-N_n^{RF},}\end{array}$$

(12)

where ${\mathcal{U}}_u$ denotes the neighboring CUEs set of CUE u, which can be defined based on interference distance similar to [33], $\eta$ is the non-negative penalty factor and penalty function $\Phi (x,y)=-1$ if $x<y$ and 0 otherwise. The second term in (11) indicates that the CUE who selects a strategy that violates constraint C1 will be punished.

Theorem 1.

Game $\mathcal{G}$ is an exact potential game.

Proof. 

Let ${\overline{\mathcal{U}}}_u$ be the set of players excluding player u and its neighbors, then we have $\mathcal{U}=\left\{u\right\}\cup {\mathcal{U}}_u\cup {\overline{\mathcal{U}}}_u$. The potential function of game $\mathcal{G}$ can be constructed as

$$\begin{array}{c}\hfill {\displaystyle \Psi =\sum _{u\in \mathcal{U}}{U}_u.}\end{array}$$

(13)

Assuming any player u unilaterally changes the strategy from $a_u$ to $a_u^{\prime }$, suppose that an arbitrary player u unilaterally changes its strategy from $a_u$ to $a_u^{\prime }$, then the change in the potential function resulted in this unilateral change can be expressed in (14).

$$\begin{array}{cc}\hfill & \Psi (a_u^{\prime },a_{-u})-\Psi (a_u,a_{-u})\hfill \\ \hfill =& {\displaystyle \sum _{u\in \mathcal{U}}{U}_u(a_u^{\prime },a_{-u})-\sum _{u\in \mathcal{U}}{U}_u(a_u,a_{-u})}\hfill \\ \hfill =& {\displaystyle \{U_u(a_u^{\prime },a_{-u})+\sum _{u\in {\mathcal{U}}_u}{U}_u(a_u^{\prime },a_{-u})\}-\{U_u(a_u,a_{-u})+\sum _{u\in {\mathcal{U}}_u}{U}_u(a_u,a_{-u})\}}\hfill \\ \hfill & {\displaystyle +\sum _{u\in {\overline{\mathcal{U}}}_u}(U_u(a_u^{\prime },a_{-u})-U_u(a_u,a_{-u}))}\hfill \\ \hfill \stackrel{(\ast )}{=}& {\displaystyle \{U_u(a_u^{\prime },a_{-u})+\sum _{u\in {\mathcal{U}}_u}{U}_u(a_u^{\prime },a_{-u})\}-\{U_u(a_u,a_{-u})+\sum _{u\in {\mathcal{U}}_u}{U}_u(a_u,a_{-u})\}}\hfill \\ \hfill =& {\tilde{U}}_u(a_u^{\prime },a_{-u})-{\tilde{U}}_u(a_u,a_{-u}).\hfill \end{array}$$

(14)

in which $(\ast )$ is due to the fact that the strategy of player u affects only the utilities of its neighboring CUEs, which results in $U_u(a_u^{\prime },a_{-u})-U_u(a_u,a_{-u})=0$ for every $u\in {\overline{\mathcal{U}}}_u$. Based on the definition in reference [36], game $\mathcal{G}$ is an exact potential game whose potential function is $\Psi$. Moreover, it has at least one pure strategy. The proof is completed. □

Theorem 2.

If all gNBs have more beams than the number of CUEs in the system and ${\displaystyle \eta >\overline{\eta }=\sum _{u\in \mathcal{U}}{R}_u}$, the NE of game $\mathcal{G}$ must be feasible.

Proof. 

Assumingthat the strategy profile $(a_1^{*},\dots ,a_U^{*})$ is a pure policy NE of the game $\mathcal{G}$ that is not satisfied with constraint C1, there must be some CUEs associated with gNBs, which have a greater number of CUs than the supported beams. Let player $k(1\le k\le U)$ be one of these CUEs. According to the assumption of this theorem, player k must be able to select a new strategy $a_k^{\prime }$ for associating with another gNB, which has a smaller number of associated CUEs than its supported beams.

$$\begin{array}{cc}\hfill & {\tilde{U}}_k(a_k^{*},a_{-k}^{*})-{\tilde{U}}_k(a_k^{\prime },a_{-k}^{*})\hfill \\ \hfill =& {\displaystyle \sum _{u\in {\mathcal{U}}_k\cup k}{R}_u(a_k^{*},a_{-k}^{*})-\sum _{u\in {\mathcal{U}}_k\cup k}{R}_u(a_k^{\prime },a_{-k}^{*})+\left[F(a_k^{*},a_{-k}^{*})-F(a_k^{\prime },a_{-k}^{*})\right]}\hfill \\ \hfill =& {\displaystyle \sum _{u\in {\mathcal{U}}_k\cup k}{R}_u(a_k^{*},a_{-k}^{*})-\sum _{u\in {\mathcal{U}}_k\cup k}{R}_u(a_k^{\prime },a_{-k}^{*})-\eta <0.}\hfill \end{array}$$

(15)

We define ${\displaystyle F(a_1,\dots ,a_U)=\eta \sum _{u\in {\mathcal{U}}_k\cup k}\sum _{n\in \mathcal{N}}\{x_{u,n}{\Delta }_n\Phi (N_n^{RF},\sum _{u\in \mathcal{U}}{x}_{u,n})\}}$, then we have (15). According to Definition 1 in reference [32], (15) is contrary to the assumption that $(a_1^{*},\dots ,a_U^{*})$ is a pure strategy NE of game $\mathcal{G}$. Then, it is easily concluded that a strategy profile that is not satisfied with constraint C1 will definitely not be a pure strategy NE of game $\mathcal{G}$. Accordingly, if the conditions of this theorem hold, the pure strategy NE of game $\mathcal{G}$ must be feasible. □

Theorem 3.

If the number of beams of all gNB is greater than the number of CUEs in the network and ${\displaystyle \tilde{\eta }>\overline{\eta }=\sum _{u\in \mathcal{U}}{R}_u}$, game $\mathcal{G}$ has at least one feasible pure strategy NE.

Proof. 

The proof process is the same as that of Theorem 2 in reference [32], and thus is omitted here. □

4. Decentralized Machine Learning Approaches for Sharing Unlicensed Millimeter-Wave Bands

In this section, we provide the details of decentralized machine learning approaches for sharing unlicensed millimeter-wave bands. In the learning process, each CUE acts as a player, and its action set is composed of the combinations of its available transmission power levels and candidate gNBs. For the ease of the following descriptions, we denote by $p_u=\{p_{u1},\dots ,p_{u{K}_u}\}$ the mixed strategy of CUE u with ${\sum }_{k=1}^{K_u}{p}_{uk}=1$, where $p_{uk}$ represents the probability that CUE u chooses its k-th pure strategy (i.e., action) $a_{uk}\in {\mathcal{A}}_u$ and $K_u$ is the number of actions of player u. Moreover, we denote by $a_u^t$ and $p_u^t\left(a_{uk}\right)$ the action selected by player u at time t and the probability of choosing action $a_{uk}$, respectively.

4.1. Stochastic Learning for Sharing Unlicensed Millimeter-Wave Bands

A stochastic learning process involves a set of learning automata, each of which is considered to be a decision-maker who can learn the best strategy from a set of actions through repeated interactions with a random environment [37]. More specifically, for each action chosen by the learning automaton, the environment will evaluate the action, and send feedback to the automaton. According to the feedback, the automaton chooses the next action. With the learning process progressing, the automaton can learn the optimal action from the random environment asymptotically. As such, stochastic learning was widely used in discrete power control [37], channel selection [38] and relay selection [39]. During the stochastic learning process, each player u chooses its strategy according to a time-varying action choice probability distribution, where its action probability is updated according to the following rule:

$$\begin{array}{c}\hfill \left\{\begin{array}{c}{p}_u^{t+1}(a_{uk}\ne a_u^t)=p_u^t\left(a_{uk}\right)-b{r}_u^t{p}_u^t\left(a_{uk}\right),\hfill \\ p_u^{t+1}(a_{uk}=a_u^t)=p_u^t\left(a_{uk}\right)+b{r}_u^t(1-p_u^t\left(a_{uk}\right)),\hfill \end{array}\right.\end{array}$$

(16)

where $b\in (0,1)$ represents step size, and $r_u^t$ denotes the random payoff of player u choosing the k-th action at time t. To calculate the random payoff $r_u^t\in [0,1)$ of player u at time t, a normalization function $\Xi \left(x\right):x\in \mathbb{R}\to [0,1)$ is defined as $r_u^t=\Xi \left(U_u^t\right)$. Based on the analysis above, the stochastic learning process for the shared use of the unlicensed mmWave bands can be described using Figure 4a, the details of which are summarized as follows: (1) Initialize the action probability $p_u=\{p_{u1},\dots ,p_{u{K}_u}\}$ with $p_{u1}=1/K_u$; (2) Each player chooses an action according to its action probability vector $p_u$; (3) All players adhere to their respective actions in a decision period, and then estimate their respective received utilities; (4) All players update their mixed strategies according to (16), and if the algorithm converges then output the strategy of all players, otherwise go to step (2).

4.2. No-Regret Learning for Sharing Unlicensed Millimeter-Wave Bands

No-regret learning is also known as regret-matching, the stationary solution of which exhibits no regret [40]. Due to its ability to converge to a correlation equilibrium (CE), no-regret learning is widely used in resource allocation problems [41]. For each player in no-regret learning, the probability of choosing a strategy is proportional to the “regret” of not choosing another strategy. The results in reference [40] have shown that, if the probability distribution is selected according to their proposed approach, the no-regret algorithm will reach the CE. Next, we introduce how to obtain the the probability distribution for the joint gNB selection and power control design of the shared use of an unlicensed mmWave band. During the no-regret learning process, each player u will choose its strategy $a_{uk}$ at each time t according to the probability distribution $p_u^t\left(a_{uk}\right),a_{uk}\in {\mathcal{A}}_u$. For the time $t=1$, $p_u^t\left(a_{uk}\right)=\frac{1}{{\mathcal{A}}_u}$. At the time $t>1$, $p_u^{t+1}\left(a_{uk}\right)$ is updated according the following rule:

$$\begin{array}{c}\hfill \left\{\begin{array}{c}{p}_u^{t+1}(a_{uk}\ne a_u^t)=\frac{1}{\mu }{R}_u^t(a_u^t,a_{uk}),\hfill \\ {\displaystyle p_u^{t+1}(a_{uk}=a_u^t)=1-\sum _{a_{uk}\ne a_u^t}{p}_u^{t+1}\left(a_{uk}\right),}\hfill \end{array}\right.\end{array}$$

(17)

where $\mu$ is a normalization factor, $a_u^t$ denotes the previously chosen strategy at time t and $R_u^t(a_u^t,a_{uk})=\max \{{\Gamma }_u^t(a_u^t,a_{uk}),0\}$ is the regret of choosing strategy $a_u^t$ instead of $a_{uk}(a_{uk}\ne a_u^t)$ with ${\Gamma }_u^t(a_u^t,a_{uk})=\frac{1}{t+1}{\sum }_{\tau =1}^{t+1}(U_u^{\tau }(a_{uk},a_{-u}^{\tau })-U_u^{\tau }(a_u^t,a_{-u}^{\tau }))$. Then, the no-regret learning process for the shared use of unlicensed bands is shown in Figure 4b, and the detailed steps are described as follows: (1) initialize the action probability of player u choosing action $a_{uk}$ with $p_u^t\left(a_{uk}\right)=\frac{1}{{\mathcal{A}}_u}$; (2) at iteration t, each player chooses an action based on its action probability vector $p_u^t$; (3) all players adhere to their respective actions in a decision period, and then estimate their respective received utilities and calculate the regrets of the chosen strategies; (4) all players update their mixed strategies via (17) and, if the algorithm converges, then output the strategy of all players, otherwise go to step (2).

4.3. Binary Log-Linear Learning for Sharing Unlicensed Millimeter-Wave Bands

Binary log-linear learning was first proposed in reference [42], which has been widely used for power control, channel selection and base station sleeping [43,44]. At each iteration t of binary log-linear learning, a random player u is selected to replace its current action $a_u^t$ with an exploratory action $a_{uk}\in {\mathcal{A}}_u$, while the actions of all other players are kept unchanged. Denote by $U_u^t(a_u^t,a_{-u}^t)$ and ${\overline{U}}_u^t(a_{uk},a_{-u}^t)$ the utilities when player u chooses the current action $a_u^t$ and the exploratory action $a_{uk}$, respectively. Then, the action probabilities of each player u are updated based on the following rule:

$$\begin{array}{c}\hfill \left\{\begin{array}{c}{p}_u^{t+1}(a_u^{t+1}=a_{uk})=\frac{\exp \{\beta {\overline{U}}_u^t(a_{uk},a_{-u}^t)/C\}}{\Phi },\hfill \\ p_u^{t+1}(a_u^{t+1}=a_u^t)=\frac{\exp \{\beta U_u^t(a_u^t,a_{-u}^t)/C\}}{\Phi },\hfill \end{array}\right.\end{array}$$

(18)

with

$$\begin{array}{c}\hfill \Phi =\exp \left\{\beta U_u^t(a_u^t,a_{-u}^t)\right\}+\exp \left\{\beta {\overline{U}}_u^t(a_{uk},a_{-u}^t)\right\},\end{array}$$

(19)

in which $\beta$ represents a learning parameter and C denotes a scaling parameter. The binary log-linear learning process for the shared use of the unlicensed band can be summarized using Figure 4c, and the details are described as follows: (1) initialize the action probability $p_u=\{p_{u1},\dots ,p_{u{K}_u}\}$ with $p_{u1}=1/K_u$; (2) each player chooses an action based on the action probability vector $p_u$; (3) all players adhere to their respective actions in a decision period, and then estimate their respective received utilities; (4) all player update their mixed strategies via (18), and, if the algorithm converges, then output the strategy of all players, otherwise go to step (2).

4.4. Q-Learning for Sharing Unlicensed Millimeter-Wave Bands

Q-learning is a reinforcement learning technique and it can determine an optimal action policy with an unknown system transition model, which is widely used for spectrum allocation and channel selection [45]. In Q-learning, a decision-maker learns its policy according to the observed environment, and performs the action to optimize its utility function in an environment of uncertainty. Generally, Q-learning consists of three components, namely, state, action and reward. In this work, we focus on single-state Q-learning, each player’s action set is composed of the combinations of the available transmission power levels and the candidate gNBs, and the reward of an action is formulated as the predicted reward function of the data rate. The Q value when player u chooses its k-th action is updated based on the following rule:

$$\begin{array}{c}\hfill {\mathcal{Q}}_{uk}^{t+1}=(1-{\zeta }^t){\mathcal{Q}}_{uk}^t+{\zeta }^t{r}_{uk}^t,\end{array}$$

(20)

where $r_{uk}=U_u(a_{uk},a_{-u})$, and ${\zeta }^t\in [0,1)$ represents the learning rate, satisfying that ${\sum }_{t=0}^{\infty }{\zeta }^t=\infty$, ${\sum }_{t=0}^{\infty }{\left({\zeta }^t\right)}^2<\infty$. Then, the policy of player u is updated according to

$$\begin{array}{c}\hfill p_u^{t+1}(a_u^{t+1}=a_{uk})=\frac{\exp [{\mathcal{Q}}_{uk}^{t+1}/\varrho ]}{{\sum }_{j=1}^{K_u}\exp [{\mathcal{Q}}_{uj}^{t+1}/\varrho ]},\end{array}$$

(21)

where $\varrho$ is used to control the tradeoff between exploitation and exploration. The Q-learning process for the shared use of the unlicensed band can be described using Figure 4d, and the details are described as follows: (1) initialize the action probability $p_u=\{p_{u1},\dots ,p_{u{K}_u}\}$ with $p_{u1}=1/K_u$; (2) each player selects an action based on the action probability vector $p_u$; (3) all players adhere to their respective actions in a decision period, and then estimate their respective received utilities and calculate the Q values according to (20); (4) all players update the mixed strategies through (21), and, if the algorithm converges, then output the strategy of all players, otherwise go to step (2).

4.5. Multi-Armed Bandit Learning for Sharing Unlicensed Millimeter-Wave Bands

Multi-armed bandit learning is one of the classic reinforcement learning techniques which aims at maximizing the total rewards on the condition that some rewards will be obtained after each agent pulls an arm (action) [46,47]. For the multi-armed bandits with multiple agents, the agents affect each other, and thus the obtained reward of each agent is dependent on both its own actions and the joint strategy profile of other agents. Therefore, it is important for multi-agent scenarios to have some kind of equilibrium or stable state. So far, many algorithms, such as $ϵ$-greedy, upper confidence bound (UCB), Softmax, and Pursuit, have been proposed to solve various kinds of multi-armed bandit problems, where the UCB scheme is very suitable for the bandit problems with stochastic stationary characteristics. Therefore, the UCB scheme can be used for the multi-armed bandit learning problem of joint gNB selection and power control for the shared use of unlicensed bands. At each iteration in the UCB scheme, an upper-bound of the average reward for every arm k of each player u is estimated as follows:

$$\begin{array}{c}\hfill I_{uk}^t={\overline{U}}_{uk}^t+\sqrt{\frac{2\ln \left(t\right)}{T_{uk}^{t-1}}},\end{array}$$

(22)

where ${\overline{U}}_{uk}^t$ represents the average reward of arm k for player u at round t, and $T_{uk}^{t-1}$ denotes the number of rounds where arm k is chosen up to round $t-1$. Then, the arm with the highest estimated bound is chosen according to the following rule:

$$\begin{array}{c}\hfill a_u^t=\underset{a_{uk}\in {\mathcal{A}}_u}{argmax}{I}_{uk}^t.\end{array}$$

(23)

The multi-armed bandits learning process for the shared use of the unlicensed band can be described using Figure 4e, and the details are described as follows: (1) initialize the parameters and play each arm once; (2) each player chooses the action with the highest estimated bound according to (23); (3) all players adhere to their respective actions in a decision period, and then estimate their respective average reward of each arm according to (22); (4) if the algorithm converges, then output the strategy of all players, otherwise go to step (2).

5. Computational Complexity Analysis

In this section, we analyze the complexities of the proposed algorithms. For the stochastic learning algorithm, its complexity can be expressed as $T_{iter1}(\mathcal{O}\left(C1\right)+\mathcal{O}\left(C2\right)+\mathcal{O}\left(C3\right))$, where $C1$ denotes the complexity of estimating the number of interfering users, $C2$ represents the complexity during the process of updating the action selection vectors, which includes the operations of two vector sums and one scalar-vector product, and $C3$ represents the complexity during the procedure of user selection. For the binary log-linear learning algorithm, its complexity can be given as $T_{iter1}(\mathcal{O}\left(C1\right)+\mathcal{O}\left(C2\right)+\mathcal{O}\left(C3\right))$, where $\mathcal{O}\left(C1\right)$ is caused by the estimating the number of interfering users, $\mathcal{O}\left(C2\right)$ is due to the updating action selection involving the operations of two exponents, one sum and two divisions, and $C3$ is a small constant which represents the complexity during the the procedures of user selection. For the Q learning algorithm, its complexity can be given by $\mathcal{O}\left(\frac{n}{{ϵ}^2}log\left(n\right)\right)$, where n represents the number of state–action pairs for achieving the $ϵ$-optimal solution with high probability. For the multi-armed bandit learning algorithm, in order to show the complexity of each algorithm more clearly, we provide the comparison of running time for each algorithm in Section 6.

6. Simulation Results and Analysis

We consider an uplink network with 5G-U and WiFi coexisting, which is composed of 4 gNBs, L CUEs, 2 APs and 2 WUEs, where CUEs and WUEs are distributed randomly in a circle with a radius of 100 m. Each gNB has six RF chains. We assume that all communication links perform uplink transmissions in sharing transmission subframes and share one mmWave band with bandwidth 1.5 GHz. The main simulation parameters are listed as follows. The available transmission power levels for each CUE is set to $\{0.01,0.05,0.1,0.15,0.2\}$ watts and the power level of each WUE is fixed at 0.1 watts. The thermal noise density is set to −174 dBm/Hz, and the beamwidths of the WUEs, 5G-U base stations and APs are all set to 20°. Unless otherwise specified, the tolerant interference threshold of each WiFi link is set to −62 dBm. The parameters related to the learning algorithms are listed as follows: $b=0.05$, $\mu ={10}^{10}$ bps, $C={10}^{13}$ bps, $\beta =10$, $\varrho ={10}^{10}$ bps, the normalization function for stochastic learning algorithm $\Xi \left(x\right)=(x+\gamma )/\gamma$ with $\gamma ={10}^{14}$ bps.

Figure 5 presents the sum rates varying with different numbers of CUEs under different learning algorithms. As can be seen from Figure 5a, binary log-linear learning has a better sum rate of the 5G-U system than other learning algorithms. The Q-learning algorithm has a sum rate of the 5G-U system that is slightly worse than that of binary log-linear learning, which, however, is slightly better than that of multi-armed bandit learning. The sum rate of no-regret learning is the worst of all, and stochastic learning is better than no-regret learning. This means that binary log-linear learning, Q-learning and multi-armed bandit learning can reach better NE of the spectrum sharing problem. From Figure 5b, it can be easily found that the better the achieved NE, the worse the sum rate of the WiFi system will be. Moreover, we can see that, with the help of constrained actions of each CUE, the transmissions of the 5G-U system have no serious effects on the WiFi system when the number of CUEs is small. Therefore, simultaneous transmissions of WiFi and 5G-U at the 60 GHz unlicensed band are feasible if the number of accessed CUEs is controlled reasonably. From Figure 5c, it can observed that the sum rate of the 5G-U system and WiFi system has the same trend as that of the sum rate of the 5G-U system, the reason for which is due to the fact that the sum rate of the 5G-U system is much larger than that of the WiFi system.

Figure 6 provides the sum rates varying with different beamwidths of CUEs under different learning techniques, where the number of CUEs is set to 16. We can find from Figure 6a that the sum rates of all learning techniques for the 5G-U system decrease with the beamwidth of CUEs. This is because, as the beamwith of CUEs increases, the directional transmission gain decreases. In Figure 6b, it can seen that the sum rates of all learning techniques for the 5G-U system decrease with the beamwidth of CUEs, the reason for which is that the interference with the WiFi system will become serious as the beamwidth of CUEs increases. Moreover, we also see that, when the beamwidth exceeds 40°, the sum rates of binary log-linear learning and Q-learning will decrease sharply. Therefore, the reasonable control of beamwidth of CUEs is also important for the shared use of 60 GHz unlicensed bands. From Figure 6c, it can be observed that the sum rates of the 5G-U system and WiFi system also have the same trend sd that of the sum rate in the 5G-U system.

Figure 7 depicts a comparison between the proposed scheme and the traditional scheme in terms of system sum rate, in which the number of CUEs is set to 16. It can observed that the sum rate performance of the WiFi system can be effectively protected under the proposed scheme, even at a lower tolerant interference threshold, while that of the traditional scheme is seriously deteriorated under the lower tolerant interference threshold, so that WUEs cannot access the channel.

Figure 8 plots a comparison of different learning techniques in terms of convergence speed, where the number of CUEs is set to 20. We can see that the multi-armed bandit learning converges faster that other learning techniques, binary log-linear learning has the second fastest convergence speed followed by stochastic learning, while Q-learning and no-regret learning have slower convergence speeds. It can be concluded that the binary log-linear learning, that has the best sum rate of 5G and WiFi systems and acceptable convergence speed, is well suited for the joint gNB selection and power control problem regarding the shared use of 60 GHz unlicensed bands.

7. Conclusions

The efficient shared use of unlicensed mmWave bands is one of the promising solutions to deal with the challenge of spectrum scarcity in future networks. In this work, we first provide a 5G–WiFi fusion protocol stack for sharing unlicensed mmWave bands, and then design an ISAC-based access protocol and an ISAC-based coexistence protocol, integrated with a decentralized learning framework, on which machine learning algorithms can be run in order to obtain the joint user association and power control strategy. We then provide the details of multiple machine learning algorithms for the shared use of unlicensed mmWave bands, including stochastic learning, no-regret learning, log-linear learning, Q-Learning and multi-armed bandit learning. Later, we provide simulations for the aforementioned learning algorithms from various aspects. The results have shown that the proposed scheme greatly reduces the search space of the solution and effectively protects the performance of the WiFi system compared to the traditional scheme; binary log-linear learning is suitable for the spectrum sharing problem at 60 GHz unlicensed bands, and the simultaneous transmissions of 5G-U and WiFi at the 60 GHz unlicensed band are feasible under the proposed scheme. Meanwhile, from the simulation results, some insights can be observed regarding how to choose the learning algorithms to achieve a balance between computational complexity and a good sum rate performance.