Age of Synchronization Minimization Algorithms in Wireless Networks with Random Updates under Throughput Constraints ^†

Abstract

This study considers a wireless network where multiple nodes transmit status updates to a base station (BS) through a shared bandwidth-limited channel. Considering the random arrival of status updates, we measure the data freshness with the age of synchronization (AoS) metric; specifically, we use the time elapsed since the latest synchronization as a metric. The objective of this study is to minimize the weighted sum of the average AoS of the entire network while meeting the minimum throughput requirement of each node. We consider both the central scheduling scenario and the distributed scheduling scenario. In the central scheduling scenario, we propose the optimal stationary randomized policy when the transmission feedback is unavailable and the max-weight policy when it is available. In the distributed scenario, we propose a distributed policy. The complexity of the three scheduling policies is significantly low. Numerical simulations show that the policies can satisfy the throughput constraint in the central controlling scenario and the AoS performance of the max-weight policy is close to the lower bound.

1. Introduction

With the development of various time-sensitive applications, such as the industrial Internet of Things and automatic driving, the freshness of information has attracted more and more attention. It is important to transmit time-sensitive information to the central controller in a timely manner in order to ensure safety and efficient control in these networks. Figure 1 shows a scenario for the industrial IoT where the central processor monitors the real-time status of various network components, including materials, equipment, power, transportation, staff, etc.

To evaluate the freshness of status updates, researchers have introduced the concept of the age of information (AoI) [1], which is defined as the time elapsed since the the generation of the most recent status update. Previous studies [2,3,4,5,6,7,8,9,10] have focused on developing scheduling algorithms that minimize the average AoI across the network in the generate-at-will scenario, which means the generation of status update packets can be controlled. Various low-complexity algorithms were presented in [2] to minimize the average AoI in the network, demonstrating their proximity to the theoretical lower bound. Power constraints and cross-layer scheduling were considered in [6], where the authors formulated the AoI optimization problem as a constrained Markov decision process and proposed an effective asymptotic optimal scheduling algorithm.

In addition to the generate-at-will scenario, researchers also focus on the random-update scenario, where status updates arrive at terminal devices randomly. Several scheduling algorithms have been proposed in [11,12,13,14,15,16] to minimize the average AoI. However, in the random-update scenario, the AoI may not show the desynchronization of the information. For example, if no new update arrives after the latest update is received, the information at the receiver remains fresh while the AoI still keeps increasing.

Therefore, the concept of the age of synchronization (AoS) was proposed in [17], which is defined as the time elapsed since the desynchronization of the freshest information at the receiver. An age-based max-weight policy was proposed in [18] to minimize the time-average AoS in error-free transmission scenarios. In [19], scheduling algorithms based on the Markov decision process and the restless multi-arm bandit were designed to optimize AoS in unreliable wireless transmission channels. Additionally, when transmitting over unreliable time-varying wireless channels, the authors in [20] modeled the AoS optimization problem as a constrained Markov decision process and revealed a threshold structure for the optimal policy. The primary objective of these works was to minimize the time-average AoS of the whole system. However, in broadcast services, network users may possess varying levels of importance and diverse requirements, such as different throughput needs. Previous studies have investigated the design of scheduling strategies that optimize data freshness, taking into account heterogeneous requirements such as timely throughput and transmission delay [5,21,22,23]. These works indicated that the AoS optimization strategies may cause unfairness among different users.

To address this issue, we consider a model in which each user in the network has a time-average throughput need. Our objective is to propose scheduling policies that not only minimize the time-average AoS for network users but also ensure that the throughput requirement of each user is met. Under different conditions, we propose three corresponding scheduling policies. To be specific, we propose the optimal stationary randomized policy, the max-weight policy, and the distributed policy to deal with ACK-available/ACK-unavailable and centralized/distributed scenarios, respectively. Part of this work has been published in Proceedings of the 2022 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB) [24].

The rest of the paper is structured in the following manner. Section 2 introduces the system model, the constraints, and the definition of AoS, and the optimization problem is modeled. In Section 3, a lower bound of the AoS optimization problem is presented. In Section 4, we proposed three scheduling policies—the optimal stationary randomized policy, the max-weight policy, and the distributed policy—and give the corresponding algorithms. In Section 5, the performance of the policies is shown and compared with numerical simulations. Finally, Section 6 concludes the paper.

2. System Model

We consider a model where M nodes transmit random status updates to a base station (BS), as illustrated in Figure 2. The time is slotted and denoted by $k\in \{1,2,\dots ,K\}$. An indicator function ${\mathrm{\Lambda }}_i\left(k\right)$ records whether a new update arrives at node $i\in \{1,2,\dots ,M\}$ in the time slot before time slot k. ${\mathrm{\Lambda }}_i\left(k\right)$ follows an independent and identical Bernoulli distribution with parameter ${\lambda }_i$.

The nodes transmit status updates the BS through a shared channel. $u_i\left(k\right)$ denotes the transmission indicator function, meaning that $u_i\left(k\right)=1$ suggests node i transmits an update to the BS at the beginning of time slot k. The transmission from node i succeeds with probability $p_i$. An indicator of successful transmission is denoted by $d_i\left(k\right)$. If $d_i\left(k\right)=1$, the transmission succeeds and the BS receives the update from node i by the end of time slot k.

If $u_i\left(k\right)=1$ at time slot k, node i transmits an update to the BS. $d_i\left(k\right)=1$ with probability $p_i$ and $d_i\left(k\right)=0$ with probability $1-p_i$. If $u_i\left(k\right)=0$, indicating that node i does not transmit to the BS in time slot k, $d_i\left(k\right)$ must be 0.

2.1. Age of Synchronization

The age of synchronization (AoS) describes the time elapsed since the last time that the newest information at the receiver was synchronized with the transmitter. we define $s_i\left(k\right)$ as the AoS of node i at the end of time slot k. The evolution of the AoS with random arrival and successful transmission is illustrated in Figure 3. While the latest update is successfully transmitted in time slot 5, the AoS of node i drops to 0 and remains 0 until a new update arrives during time slot 7. Then, the AoS of node i keeps increasing by 1 in every subsequent time slot until the new update is received by the BS in time slot 12.

As the BS is only interested in the newest updates, each node always transmits the newest updates in the buffer to the BS. The evolution of $s_i\left(k\right)$ follows:

$$s_i\left(k\right)=\left\{\begin{array}{cc}0,\hfill & \mathrm{if}{d}_i\left(k\right)=1;\hfill \\ 0,\hfill & \mathrm{if}{s}_i(k-1)=0,\mathrm{and}{\mathrm{\Lambda }}_i\left(k\right)=0;\hfill \\ s_i(k-1)+1,\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

(1)

We consider that the AoSs of different nodes have different importance, which is represented by ${\alpha }_i>0$, and we are interested in the expected weighted sum AoS of the network. We denote the expected weighted-sum AoS during the first K time slots under scheduling policy $\pi$ as

$$J_K^{\pi }=\frac{1}{KM}\mathbb{E}\left[\sum _{k=1}^K\sum _{i=1}^M{\alpha }_i{s}_i\left(k\right)|s_i\left(0\right)\right],$$

(2)

where the expectation is denoted with respect to the randomness of the channel and the scheduling policy.

Without loss of generality, we assume that the sources are synchronized at the beginning, so the initial values of the AoS follow $s_i\left(0\right)=0,\forall i\in \{1,\dots ,M\}$.

2.2. Constraints

We consider that the nodes transmit to the BS through a bandwidth-limited channel and the bandwidth requires that one node at most can transmit in each time slot. This bandwidth constraint can be formulated as

$$\sum _{i=1}^M{u}_i\left(k\right)\le 1,\forall k\in \{1,\dots ,K\}.$$

(3)

We consider that each node in the network has a minimum throughput requirement and let $q_i$ denote the minimum time-average throughput requirement of node i. The long-term time-average throughput under a specific policy $\pi$ denoted by ${\widehat{q}}_i^{\pi }$ can be computed by

$${\widehat{q}}_i^{\pi }:=\underset{K\to \infty }{lim}\frac{1}{K}\sum _{k=1}^K{\mathbb{E}}_{\pi }\left[d_i\left(k\right)\right].$$

(4)

Then, the throughput constraint can be expressed as

$${\widehat{q}}_i^{\pi }\ge q_i,\forall i\in \{1,\dots ,M\}.$$

(5)

The minimum throughput requirement of the entire network is feasible under the following assumption.

Assumption 1.

$$\begin{array}{cc}\hfill \sum _{i=1}^M\frac{q_i}{p_i}& \le 1,\hfill \end{array}$$

(6)

$$\begin{array}{cc}\hfill q_i& \le {\lambda }_i,\forall i\in \{1,\dots ,M\}.\hfill \end{array}$$

(7)

2.3. Optimization Problem

We now formulate the optimization problem as follows.

Problem 1

(AoS optimization).

$${\pi }^{*}=\underset{\pi \in \Pi }{argmin}\left\{\underset{K\to \infty }{lim}{J}_K^{\pi }\right\},$$

(8a)

$$\begin{array}{cc}\hfill s.t.& {\widehat{q}}_i^{\pi }\ge q_i,\forall i\in \{1,\dots ,M\};\hfill \end{array}$$

(8b)

$$\begin{array}{cc}\hfill & \sum _{i=1}^M{u}_i\left(k\right)\le 1,\forall k\in \{1,\dots ,K\}.\hfill \end{array}$$

(8c)

Equation (8b) describes the throughput constraints and (8c) describes the bandwidth constraints. We denote the optimal expected weighted-sum AoS under the AoS-optimal policy as

$$Ao{S}^{*}=\underset{K\to \infty }{lim}{J}_K^{{\pi }^{*}}$$

(9)

Then, we give a lower bound of the AoS optimization problem (Equations (8a)–(8c)) first. In three different cases, three scheduling policies are proposed.

3. Lower Bound

A lower bound of Equations (8a)–(8c) is obtained as follows.

Theorem 1.

The AoS optimization problem (Equations (8a)–(8c)) can be lower bounded by Equations (10a)–(10c) for every network setup $(M,{\lambda }_i,p_i,q_i,{\alpha }_i)$.

$$Ao{S}_{LB}=\underset{{\widehat{q}}_i}{min}\left\{\frac{1}{2M}\sum _{i=1}^M{\alpha }_i\left(\frac{1}{{\widehat{q}}_i}+\frac{2-2{\lambda }_i}{{\lambda }_i^2}{\widehat{q}}_i-\frac{2-{\lambda }_i}{{\lambda }_i}\right)\right\},$$

(10a)

$$\begin{array}{cc}\hfill s.t.& {\widehat{q}}_i\ge q_i,\forall i\in \{1,\dots ,M\};\hfill \end{array}$$

(10b)

$$\begin{array}{cc}\hfill & \sum _{i=1}^M{u}_i\left(k\right)\le 1,\forall k\in \{1,\dots ,K\}.\hfill \end{array}$$

(10c)

Proof. 

Let $\Omega$ be the sample space associated with the network and $\omega \in \Omega$ be a certain sample path. Let $D_i\left(K\right)={\sum }_{k=1}^K{d}_i\left(k\right)$ denote the number of updates successfully transmitted from node i during the first K time slots under a certain sample path $\omega$. Define $U_i\left[m\right]$ as the time index of the m-th arrival of a new update and $T_i\left[m\right]$ as the time index of the m-th successful transmission from node i, as shown in Figure 4. Then, the inter-delivery time can be computed by

$$I_i\left[m\right]=T_i\left[m\right]-T_i[m-1],\forall m\in \{2,\dots ,D_i\left(K\right)\}$$

(11)

and $I_i\left[1\right]=T_i\left[1\right]$. Denote the time from the m-th delivery to the next arrival as

$$\begin{array}{c}\hfill X_i\left[m\right]=\underset{j}{min}\left\{U_i\left[j\right]\right|U_i\left[j\right]>T_i[m-1]\}-T_i[m-1],\forall m\in \{2,\dots ,D_i\left(K\right)\}\end{array}$$

(12)

and $X_i\left[1\right]=U_i\left[1\right]$ for simplicity.

Then, the K time slots can be decoupled as

$$K=\sum _{m=1}^{D_i\left(K\right)}{I}_i\left[m\right]+R_i,\forall i\in \{1,\dots ,M\},$$

(13)

where $R_i$ denotes the rest slots after the $D_i\left(K\right)$-th delivery. $X_i^{\prime }$ denotes the time from the last delivery to the next arrival or K. Since the arrival of updates follows a Bernoulli distribution with the parameter ${\lambda }_i$, $X_i\left[m\right]$ and $X_i^{\prime }$ follow a geometric distribution with the parameter ${\lambda }_i$, which is independent from $I_i\left[m\right]$. During each $I_i\left[m\right]$, the AoS increases after $X_i\left[m\right]$ slots from 0 to $I_i\left[m\right]-X_i\left[m\right]$, so the time-average AoS of node i is

$$\begin{array}{cc}\hfill \frac{1}{K}\mathbb{E}\left[\sum _{k=1}^K{s}_i\left(k\right)\right]& =\frac{1}{K}\mathbb{E}\left[\sum _{m=1}^{D_i\left(K\right)}\frac{(I_i\left[m\right]-X_i\left[m\right])(I_i\left[m\right]-X_i\left[m\right]+1)}{2}+\frac{(R_i-X_i^{\prime })(R_i-X_i^{\prime }+1)}{2}\right]\hfill \end{array}$$

(14)

$$\begin{array}{cc}\hfill & \stackrel{\left(a\right)}{=}\frac{1}{2K}\mathbb{E}\left[\sum _{m=1}^{D_i\left(K\right)}\left[I_i^2\left[m\right]+X_i^2\left[m\right]-X_i\left[m\right]\right]+R_i^2+X_i^{\prime 2}-X_i^{\prime }-2K{X}_i\left[m\right]+K\right]\hfill \end{array}$$

(15)

$$\begin{array}{cc}\hfill & \stackrel{\left(b\right)}{=}\frac{1}{2K}\mathbb{E}\left[\sum _{m=1}^{D_i\left(K\right)}{I}_i^2\left[m\right]+R_i^2+(D_i\left(K\right)+1)(X_i^{\prime 2}-X_i^{\prime })-2K{X}_i^{\prime }+K\right],\hfill \end{array}$$

(16)

where (a) is derived by merging linear terms to K using Equation (13) and (b) is because $X_i\left[m\right]$ and $X_i^{\prime }$ are independent identically.

The first part of Equation (16) can be lower bounded by Jensen’s inequality

$$\begin{array}{c}\hfill \sum _{m=1}^{D_i\left(K\right)}{I}_i^2\left[m\right]+R_i^2\ge \frac{1}{D_i\left(K\right)+1}{\left[\sum _{m=1}^{D_i\left(K\right)}{I}_i\left[m\right]+R_i\right]}^2=\frac{K^2}{D_i\left(K\right)+1}.\end{array}$$

(17)

To calculate the second part of Equation (16), we can use the property of $X_i^{\prime }$ and ${\widehat{q}}_i$:

$$\begin{array}{cc}\hfill \mathbb{E}\left[X_i^{\prime }\right]& =\frac{1}{{\lambda }_i},\hfill \end{array}$$

(18)

$$\begin{array}{cc}\hfill \mathbb{E}\left[X_i^{\prime 2}\right]& =\frac{2-{\lambda }_i}{{\lambda }_i^2},\hfill \end{array}$$

(19)

$$\begin{array}{cc}\hfill \underset{K\to \infty }{lim}\frac{D_i\left(K\right)}{K}& ={\widehat{q}}_i.\hfill \end{array}$$

(20)

Combining Equations (16)–(19) and (2), we have

$$\begin{array}{c}\hfill \frac{1}{K}\mathbb{E}\left[\sum _{k=1}^K{s}_i\left(k\right)\right]\ge \frac{K}{2(D_i\left(K\right)+1)}+\frac{D_i\left(K\right)+1}{K}·\frac{1-{\lambda }_i}{{\lambda }_i^2}-\frac{1}{{\lambda }_i}+\frac{1}{2}.\end{array}$$

(21)

Take $K\to \infty$ in Equation (21) and apply Equation (20):

$$\underset{K\to \infty }{lim}\frac{1}{K}\mathbb{E}\left[\sum _{k=1}^K{s}_i\left(k\right)\right]\ge \frac{1}{2{\widehat{q}}_i}+\frac{1-{\lambda }_i}{{\lambda }_i^2}{\widehat{q}}_i-\frac{2-{\lambda }_i}{2{\lambda }_i}.$$

(22)

Summing over $i\in \{1,\dots ,M\}$ gives the limitation of $J_K$:

$$\underset{K\to \infty }{lim}{J}_K\ge \frac{1}{2M}\sum _{i=1}^M{\alpha }_i\left(\frac{1}{{\widehat{q}}_i}+\frac{2-2{\lambda }_i}{{\lambda }_i^2}{\widehat{q}}_i-\frac{2-{\lambda }_i}{{\lambda }_i}\right).$$

(23)

The lower bound for Equations (10a)–(10c) is proved.    □

4. Scheduling Policies

In this section, we propose three scheduling policies. When a central controller schedules the transmissions, we propose the optimal stationary randomized policy in the case where transmission acknowledgement is unused and we propose the max-weight policy in the case where the central controller can get transmission acknowledgement. We consider the BS as the central controller for simplicity. When there is no central scheduling and each node is unaware of the status of the other nodes, we propose the distributed policy.

4.1. Optimal Stationary Randomized Policy

In the case without transmission acknowledgement, the BS cannot compute the AoS of the nodes simultaneously. Therefore, the scheduling policy needs to be independent of the AoS. If a policy is entirely based on probability and will not change with the state of the network, it is called a stationary randomized policy and denoted by $R\in {\Pi }_R$. A stationary randomized policy R chooses node i to transmit with probability ${\mu }_i\in (0,1]$ and keeps idle with probability ${\mu }_{idle}$. ${\mu }_i$ has the property that ${\sum }_{i=1}^M{\mu }_i+{\mu }_{idle}=1$ and ${\mu }_i=\mathbb{E}\left[u_i\left(k\right)\right]$. Then, we can determine the throughput and AoS of the stationary randomized policy R:

Proposition 1.

The long-term throughput and the expected time-average AoS of node i under a policy $R\in {\Pi }_R$ is

$$\begin{array}{cc}\hfill {\widehat{q}}_i^R& =p_i{\mu }_i;\hfill \end{array}$$

(24)

$$\begin{array}{cc}\hfill \underset{K\to \infty }{lim}\frac{1}{K}\mathbb{E}\left[\sum _{k=1}^K{s}_i\left(k\right)\right]& =\frac{1}{p_i{\mu }_i}-\frac{1}{{\lambda }_i+p_i{\mu }_i-{\lambda }_i{p}_i{\mu }_i}.\hfill \end{array}$$

(25)

Proof. 

Since the probabilities of both transmitting to node i and transmitting successfully—which refer to ${\mu }_i$ and $p_i$, respectively—are determined and independent, the BS receives an update from node i in a time slot with probability $p_i{\mu }_i$. This implies the long-term throughput ${\widehat{q}}_i^R=p_i{\mu }_i$. The inter-delivery time $I_i\left[m\right]$ now follows a geometric distribution with parameter $p_i{\mu }_i$.

Let the sum AoS during an inter-delivery time $I_i\left[m\right]$ be $A_i\left[m\right]={\sum }_{k=k_i\left[m\right]+1}^{k_i\left[m\right]+I_i\left[m\right]}{S}_i\left(k\right)$, where $k_i\left[m\right]={\sum }_{j=1}^{m-1}{I}_i\left[j\right]$ if $m>1$ and $k_i\left[m\right]=0$ if $m=1$. $A_i\left[m\right]$ can be derived by summing over the possible values of waiting time $X_i\left[m\right]$.

$$\begin{array}{cc}\hfill A_i\left[m\right]=& \sum _{x=1}^{I_i\left[m\right]}\frac{1}{2}(I_i\left[m\right]-x+1)(I_i\left[m\right]-x){\lambda }_i{(1-{\lambda }_i)}^{x-1}\hfill \end{array}$$

(26)

$$\begin{array}{cc}\hfill =& \frac{1}{2}{I}_i\left[m\right](I_i\left[m\right]+1)-\frac{I_i\left[m\right]}{{\lambda }_i}+\frac{1-{\lambda }_i}{{\lambda }_i^2}\left[1-{(1-{\lambda }_i)}^{I_i\left[m\right]+1}\right].\hfill \end{array}$$

(27)

The distribution of $I_i\left[m\right]$ gives

$$\begin{array}{cc}\hfill \mathbb{E}\left[I_i\left[m\right]\right]& =\frac{1}{p_i{\mu }_i};\hfill \end{array}$$

(28)

$$\begin{array}{cc}\hfill \mathbb{E}\left[I_i^2\left[m\right]\right]& =\frac{2-p_i{\mu }_i}{{\left(p_i{\mu }_i\right)}^2};\hfill \end{array}$$

(29)

$$\begin{array}{cc}\hfill \mathbb{E}\left[{(1-{\lambda }_i)}^{I_i\left[m\right]}\right]& =\frac{p_i{\mu }_i(1-{\lambda }_i)}{1-(1-{\lambda }_i)(1-p_i{\mu }_i)}.\hfill \end{array}$$

(30)

Combining Equations (27)–(30), the expectation of $A_i\left[m\right]$ is given as

$$\mathbb{E}\left[A_i\left[m\right]\right]=\frac{1}{{\left(p_i{\mu }_i\right)}^2}-\frac{1}{p_i{\mu }_i({\lambda }_i+p_i{\mu }_i-{\lambda }_i{p}_i{\mu }_i)}.$$

(31)

The inter-delivery times $I_i\left[m\right]$, $\forall m\in \{1,\dots ,D_i\left(K\right)\}$ for each node i are independently identically distributed, so the expected time-average AoS of node i is equal to the ratio of expectation of $A_i\left[m\right]$ to $I_i\left[m\right]$, which leads to Equation (25).    □

Substituting Equations (24) and (25) into Equations (8a)–(8c), the computation of the optimal stationary randomized policy can be formulated as follows.

Problem 2

(AoS optimization over stationary randomized policies).

$$Ao{S}_{R^{*}}=\underset{R\in {\Pi }_R}{min}\left\{\frac{1}{M}\sum _{i=1}^M{\alpha }_i\left(\frac{1}{p_i{\mu }_i}-\frac{1}{{\lambda }_i+p_i{\mu }_i-{\lambda }_i{p}_i{\mu }_i}\right)\right\},$$

(32a)

$$\begin{array}{cc}\hfill s.t.& p_i{\mu }_i\ge q_i,\forall i\in \{1,\dots ,M\};\hfill \end{array}$$

(32b)

$$\begin{array}{cc}\hfill & \sum _{i=1}^M{\mu }_i\le 1.\hfill \end{array}$$

(32c)

Compare Equations (32a)–(32c) with the lower bound of Equations (10a)–(10c). The throughput ${\widehat{q}}_i^{\pi }$ solving the lower bound can also satisfy Equations (32b) and (32c). Replacing $p_i{\mu }_i$ with ${\widehat{q}}_i^{\pi }$ and subtracting the expression in the minimization function of $Ao{S}_{R^{*}}$ from that of $2Ao{S}_{LB}$ gives

$$\begin{array}{c}\frac{1}{M}\sum _{i=1}^M{\alpha }_i\left(\frac{1}{{\widehat{q}}_i^{\pi }}+\frac{2-2{\lambda }_i}{{\lambda }_i^2}{\widehat{q}}_i^{\pi }-\frac{2-{\lambda }_i}{{\lambda }_i}\right)-\frac{1}{M}\sum _{i=1}^M{\alpha }_i\left(\frac{1}{{\widehat{q}}_i^{\pi }}-\frac{1}{{\lambda }_i+{\widehat{q}}_i^{\pi }-{\lambda }_i{\widehat{q}}_i^{\pi }}\right)\hfill \end{array}$$

$$\begin{array}{c}=\frac{1}{M}\sum _{i=1}^M{\alpha }_i\left(\frac{2-2{\lambda }_i}{{\lambda }_i^2}{\widehat{q}}_i^{\pi }-\frac{2-{\lambda }_i}{{\lambda }_i}+\frac{1}{1-(1-{\lambda }_i)(1-{\widehat{q}}_i^{\pi })}\right)\hfill \end{array}$$

(33)

$$\begin{array}{c}>\frac{1}{M}\sum _{i=1}^M{\alpha }_i\left(\frac{2-2{\lambda }_i}{{\lambda }_i^2}{\widehat{q}}_i^{\pi }-\frac{2-{\lambda }_i}{{\lambda }_i}+1\right)\hfill \end{array}$$

(34)

$$\begin{array}{c}=\frac{1}{M}\sum _{i=1}^M{\alpha }_i\left(\frac{2(1-{\lambda }_i)({\widehat{q}}_i^{\pi }-{\lambda }_i)}{{\lambda }_i^2}\right)\hfill \end{array}$$

(35)

Though Equation (35) is smaller than 0 for ${\widehat{q}}_i^{\pi }<{\lambda }_i$, it is much smaller than the common part $1/M{\sum }_{i=1}^M{\alpha }_i/{\widehat{q}}_i^{\pi }$. Therefore, $Ao{S}_{R^{*}}$ can be bounded by $2Ao{S}_{LB}$ adding a small incremental.

Solving Problem 2 means finding the optimal set of scheduling probabilities ${\left\{{\mu }_i^{*}\right\}}_{i=1}^M$. For simplicity, we use $B_i\left({\mu }_i\right)$ to represent $\frac{{\alpha }_i}{M}(\frac{1}{p_i{\mu }_i}-\frac{1}{{\lambda }_i+p_i{\mu }_i-{\lambda }_i{p}_i{\mu }_i})$. Let ${\omega }_i\ge 0$, $\forall i\in \{1,\dots ,M\}$ and $\gamma \ge 0$ be the Lagrange multiplier. The Lagrange equation is defined as

$$\mathcal{L}({\mu }_i,{\omega }_i,\gamma )=\sum _{i=1}^M{B}_i\left({\mu }_i\right)+\sum _{i=1}^M{\omega }_i(q_i-p_i{\mu }_i)+\gamma (\sum _{i=1}^M{\mu }_i-1).$$

(36)

Then, the KKT conditions are

$$\begin{array}{cc}\hfill {\nabla }_{{\mu }_i}\mathcal{L}({\mu }_i,{\omega }_i,\gamma )=\frac{\partial B_i\left({\mu }_i\right)}{\partial {\mu }_i}-{\omega }_i{p}_i+\gamma & =0,\forall i;\hfill \end{array}$$

(37)

$$\begin{array}{cc}\hfill {\omega }_i(q_i-p_i{\mu }_i)& =0,\forall i;\hfill \end{array}$$

(38)

$$\begin{array}{cc}\hfill \gamma \left(\sum _{i=1}^M{\mu }_i-1\right)& =0.\hfill \end{array}$$

(39)

We change the form of $B_i\left({\mu }_i\right)$ as follows

$$\begin{array}{cc}\hfill B_i\left({\mu }_i\right)& =\frac{{\alpha }_i}{M}\left(\frac{1}{p_i{\mu }_i}-\frac{1}{{\lambda }_i+p_i{\mu }_i-{\lambda }_i{p}_i{\mu }_i}\right)\hfill \end{array}$$

(40)

$$\begin{array}{cc}\hfill & =\frac{{\alpha }_i}{M{p}_i{\mu }_i}\left(1-\frac{1}{\frac{{\lambda }_i}{p_i{\mu }_i}+1-{\lambda }_i}\right).\hfill \end{array}$$

(41)

$B_i\left({\mu }_i\right)$ can be expressed as the product of two parts that are both positive and monotonically decreasing with ${\mu }_i$, so $B_i\left({\mu }_i\right)$ monotonically decreases with ${\mu }_i$ and $\frac{\partial B_i\left({\mu }_i\right)}{\partial {\mu }_i}<0$. This gives

$$\gamma =-\frac{\partial B_i\left({\mu }_i\right)}{\partial {\mu }_i}+{\omega }_i{p}_i>{\omega }_i{p}_i\ge 0.$$

(42)

Combining Equations (39) and (42) we get

$$\sum _{i=1}^M{\mu }_i=1.$$

(43)

According to Equation (38), a node i indicates whether ${\omega }_i=0$ or $p_i{\mu }_i=q_i$. If $p_i{\mu }_i=q_i$, ${\omega }_i\ge 0$ and $\gamma \ge -\frac{\partial B_i\left({\mu }_i\right)}{\partial {\mu }_i}$. If ${\omega }_i=0$, Equation (42) turns to $\gamma =-\frac{\partial B_i\left({\mu }_i\right)}{\partial {\mu }_i}$. We can derive the monotonicity of $\frac{\partial B_i\left({\mu }_i\right)}{\partial {\mu }_i}$ by calculating the second partial derivative of $B_i\left({\mu }_i\right)$.

$$\begin{array}{c}\hfill \frac{{\partial }^2{B}_i\left({\mu }_i\right)}{\partial {\mu }_i^2}=\frac{{\alpha }_i{p}_i^2}{M}\left(\frac{1}{{\left(p_i{\mu }_i\right)}^3}-\frac{{(1-{\lambda }_i)}^2}{{({\lambda }_i+p_i{\mu }_i-{\lambda }_i{p}_i{\mu }_i)}^3}\right).\end{array}$$

(44)

It is clear that $1>{(1-{\lambda }_i)}^2$ and ${\left(p_i{\mu }_i\right)}^3<{({\lambda }_i+p_i{\mu }_i-{\lambda }_i{p}_i{\mu }_i)}^3$, so $\frac{{\partial }^2{B}_i\left({\mu }_i\right)}{\partial {\mu }_i^2}>0$ and $\frac{\partial B_i\left({\mu }_i\right)}{\partial {\mu }_i}$ increases monotonically with ${\mu }_i$.

Note that $\gamma$ is shared by all ${\mu }_i$, so ${\mu }_i$ can be derive by comparing $-\frac{\partial B_i(q_i/p_i)}{\partial {\mu }_i}$ and $\gamma$. When $-\frac{\partial B_i(q_i/p_i)}{\partial {\mu }_i}>\gamma$, ${\mu }_i$ is the solution of $\gamma =-\frac{\partial B_i\left({\mu }_i\right)}{\partial {\mu }_i}$, which is bigger than $\frac{q_i}{p_i}$; otherwise, it is ${\mu }_i=\frac{q_i}{p_i}$.

We can get the optimal ${\left\{{\mu }_i^{*}\right\}}_{i=1}^M$ with the following steps. Start from a large $\gamma$ so that ${\mu }_i=\frac{q_i}{p_i}$, $\forall i$. Now, we have ${\sum }_{i=1}^M{\mu }_i={\sum }_{i=1}^M\frac{q_i}{p_i}\le 1$. When $\gamma$ decreases to the point that $\gamma <-\frac{\partial B_i(q_i/p_i)}{\partial {\mu }_i}$, ${\mu }_i$ will be replaced by the solution of $\gamma =-\frac{\partial B_i\left({\mu }_i\right)}{\partial {\mu }_i}$, which increases gradually with the decrease in $\gamma$. Let $\gamma$ decrease until ${\sum }_{i=1}^M{\mu }_i$ reaches 1. That is when the solution is reached.

Theorem 2

(Optimal stationary randomized policy). The optimal stationary randomized policy $R^{*}$ solving Equations (32a)–(32c) is the set of scheduling probabilities ${\left\{{\mu }_i^{*}\right\}}_{i=1}^M$ derived from Algorithm 1.

Algorithm 1 Solution to the optimal stationary randomized policy

1: Initialization: M, ${\alpha }_i$, $p_i$, $q_i$, ${\lambda }_i$ and $B_i\left({\mu }_i\right)=\frac{{\alpha }_i}{M}(\frac{1}{p_i{\mu }_i}-\frac{1}{{\lambda }_i+p_i{\mu }_i-{\lambda }_i{p}_i{\mu }_i})$ 2: ${\mu }_i=\frac{q_i}{p_i}$, $\forall i\in \{1,2,\dots ,M\}$ 3: ${\gamma }_i=-\frac{\partial B_i\left({\mu }_i\right)}{\partial {\mu }_i}$ 4: $\gamma ={max}_i{\gamma }_i$ 5: while ${\sum }_{i=1}^M{\mu }_i<1$do 6:    decrease $\gamma$ slightly 7:    for each $i\in \{1,2,\dots ,M\}$ do 8:      if  $-\frac{\partial B_i\left({\mu }_i\right)}{\partial {\mu }_i}>\gamma$ then 9:         replace ${\mu }_i$ by the solution of $\gamma =-\frac{\partial B_i\left({\mu }_i\right)}{\partial {\mu }_i}$ 10:      end if 11:    end for 12: end while 13: Output: ${\gamma }^{*}=\gamma$ and ${\mu }_i^{*}={\mu }_i$, $\forall i\in \{1,2,\dots ,M\}$

4.2. Max-Weight Policy

When the transmission acknowledgement is used, the BS can make decisions according to the AoS of the network. The idea of the max-weight policy is to optimize the network by controlling the expected increase in the Lyapunov function. The Lyapunov function becomes large with a large AoS or a lack in throughput.

We define the throughput debt $x_i\left(k\right)$ as the difference between the target throughput and the number of actually delivered updates. A large $x_i\left(k\right)$ indicates that node i needs more transmissions to satisfy the throughput constraint.

$$x_i(k+1)=k{q}_i-\sum _{t=1}^k{d}_i\left(t\right).$$

(45)

If $x_i\left(k\right)<0$, node i leads the target throughput so that the BS has no motivation to choose node i from the perspective of throughput. Therefore, we just focus on positive $x_i\left(k\right)$. We use operator ${(·)}^{+}=max\{(·),0\}$ to denote the positive part of the throughput debt $x_i^{+}\left(k\right)=max\{x_i\left(k\right),0\}$. Then, the throughput constraint can be satisfied when

$$\underset{K\to \infty }{lim}\frac{1}{K}\mathbb{E}\left[\sum _{k=1}^K{x}_i^{+}\left(k\right)\right]<\infty .$$

(46)

Define the system state at the beginning of time slot k as $S_k={(s_i\left(k\right),x_i^{+}\left(k\right))}_{i=1}^M$ and the Lyapunov function as

$$L\left(S_k\right):=\frac{1}{2}\sum _{i=1}^M\left({\alpha }_i{s}_i^2\left(k\right)+V{\left(x_i^{+}\left(k\right)\right)}^2\right),$$

(47)

where V is a positive real number indicating the importance of the throughput constraint. Thus, $L\left(S_k\right)$ grows with the AoS or the throughput debt. We define the Lyapunov drift as

$$\mathrm{\Delta }\left(S_k\right):=\mathbb{E}\{L\left(S_{k+1}\right)-L\left(S_k\right)|S_k\}.$$

(48)

The Lyapunov drift describes the expected change in the Lyapunov function in each time slot, so $L\left(S_k\right)$ can be restricted by reducing $\mathrm{\Delta }\left(S_k\right)$. To help execute the max-weight policy, an upper bound of $\mathrm{\Delta }\left(S_k\right)$ is given below.

Proposition 2.

The Lyapunov drift can be upper-bounded by

$$\mathrm{\Delta }\left(S_k\right)\le -\sum _{i=1}^M{W}_i\left(k\right)\mathbb{E}\left\{u_i\left(k\right)\right|S_k\}+C\left(k\right),$$

(49)

where

$$\begin{array}{cc}\hfill W_i\left(k\right)& =\frac{{\alpha }_i{p}_i}{2}{\left(s_i\left(k\right)+1\right)}^2+V{p}_i{x}_i^{+}\left(k\right);\hfill \end{array}$$

(50)

$$\begin{array}{cc}\hfill C\left(k\right)& =\sum _{i=1}^M\left({\alpha }_i\left(s_i\left(k\right)+\frac{1}{2}\right)+V\left(q_i{x}_i^{+}\left(k\right)+\frac{1}{2}\right)\right).\hfill \end{array}$$

(51)

Proof. 

$$\begin{array}{cc}\hfill \mathrm{\Delta }\left(S_k\right)& =\frac{1}{2}\sum _{i=1}^M{\alpha }_i\mathbb{E}\{s_i^2(k+1)-s_i^2\left(k\right)|S_k\}\hfill \\ \hfill & +\frac{V}{2}\sum _{i=1}^M\mathbb{E}\{{\left[x_i^{+}(k+1)\right]}^2-{\left[x_i^{+}\left(k\right)\right]}^2|S_k\}.\hfill \end{array}$$

(52)

Consider the latter part first. By the definition of $x_i\left(k\right)$,

$$\begin{array}{cc}\hfill {\left[x_i^{+}(k+1)\right]}^2& ={[max\{x_i\left(k\right)-d_i\left(k\right)+q_i,0\}]}^2\hfill \\ \hfill & \le {[max\{x_i^{+}\left(k\right)-d_i\left(k\right)+q_i,0\}]}^2\hfill \\ \hfill & \le {[x_i^{+}\left(k\right)-d_i\left(k\right)+q_i]}^2\hfill \\ \hfill & \le {\left[x_i^{+}\left(k\right)\right]}^2+2x_i^2\left(k\right)(-d_i\left(k\right)+q_i)+1.\hfill \end{array}$$

(53)

Take the expectation and use the property of $d_i\left(k\right)$:

$$\begin{array}{c}\hfill \frac{V}{2}\sum _{i=1}^M\mathbb{E}\{{\left[x_i^{+}(k+1)\right]}^2-{\left[x_i^{+}\left(k\right)\right]}^2|S_k\}\le V\sum _{i=1}^M\left(x_i^{+}\left(k\right)\left(-p_i\mathbb{E}\left\{u_i\left(k\right)\right|S_k\}+q_i\right)+\frac{1}{2}\right).\end{array}$$

(54)

The next step is dealing with the former part of $\mathrm{\Delta }\left(S_k\right)$.

$$\begin{array}{cc}\hfill \mathbb{E}\left\{s_i^2(k+1)\right\}& \le 0·\mathbb{E}\left\{d_i\left(k\right)\right\}+{(s_i\left(k\right)+1)}^2(1-\mathbb{E}\left\{d_i\left(k\right)\right\})\hfill \end{array}$$

(55)

$$\begin{array}{cc}\hfill & ={(s_i\left(k\right)+1)}^2(1-p_i\mathbb{E}\left\{u_i\left(k\right)\right|S_k\}).\hfill \end{array}$$

(56)

The inequality is because the AoS remains 0 rather than $s_i\left(k\right)+1$ if $s_i\left(k\right)=0$. So, we have

$$\begin{array}{c}\hfill \mathbb{E}\{s_i^2(k+1)-s_i^2\left(k\right)|S_k\}\le -{(s_i\left(k\right)+1)}^2{p}_i\mathbb{E}\left\{u_i\left(k\right)\right|S_k\}+2s_i\left(k\right)+1.\end{array}$$

(57)

Combining Equations (52), (54), and (57) completes the proof.    □

We can find from the form of Equations (49)–(51) that $W_i\left(k\right)$ and $C\left(k\right)$ fully depend on the system state $S\left(k\right)$. What our policy controls is the set of $u_i\left(k\right)$. We want to choose a node i that can reduce $S_k$ most to transmit in each time slot. So, the node with the largest $W_i\left(k\right)$ is the optimal choice. That is how max-weight works.

Lemma 1.

The max-weight policy satisfies any feasible set of minimum throughput requirements ${\left\{q_i\right\}}_{i=1}^M$.

Lemma 2

(Upper bound of the max-weight policy). The optimal expected time-average AoS of the max-weight policy can be upper-bounded by

$$Ao{S}_{MW}\le \frac{2}{M}\sum _{i=1}^M\frac{{\alpha }_i}{p_i{\mu }_i}+V.$$

(58)

Lemmas 1 and 2 are from Theorems 6 and 7 in [25]. The upper bound of max-weight is near four times the lower bound in Equations (10a)–(10c). Though $Ao{S}_{MW}$ has a large upper bound compared to $Ao{S}_{R^{*}}$, it was found in the numerical simulation that max-weight performs much better than the optimal stationary randomized policy.

4.3. Distributed Policy

When the number of nodes increases, it is difficult and costly to apply the centralized policies proposed above. As an alternative, we study the distributed policy, where a node knows neither the system states of the other nodes nor when the other nodes are transmitting. It is possible that more than one node transmits to the BS in the same time slot. So, we first relax the bandwidth constraints (Equation (8c)) to a time-average constraint

$$\frac{1}{K}\sum _{k=1}^K\sum _{i=1}^M\mathbb{E}\left[u_i\left(k\right)\right]\le 1.$$

(59)

Then, we solve the optimization problem with the Lagrange multiplier method.

Problem 3

(Relaxed AoS optimization).

$$\begin{array}{c}\hfill Ao{S}_{re}^{*}=\underset{\pi \in \Pi }{min}\left\{\underset{K\to \infty }{lim}\frac{1}{KM}\sum _{k=1}^K\sum _{i=1}^M\left[{\alpha }_i\mathbb{E}\left[s_i\left(k\right)\right]+C\left(\mathbb{E}\left[u_i\left(k\right)\right]-\frac{1}{M}\right)+{\theta }_i\left(\frac{q_i}{p_i}-\mathbb{E}\left[u_i\left(k\right)\right]\right)\right]\right\},\end{array}$$

(60a)

$$\begin{array}{cc}\hfill s.t.& {\theta }_i\ge 0,\forall i\in \{1,\dots ,M\};\hfill \end{array}$$

(60b)

$$\begin{array}{cc}\hfill & C\ge 0,\hfill \end{array}$$

(60c)

where ${\theta }_i$ and C are Lagrange multipliers respectively associated with throughput constraints $\mathbb{E}\left[u_i\left(k\right)\right]\ge \frac{q_i}{p_i}$ and the relaxed bandwidth constraint (Equation (59)).

The form of Equation (60a) is independent for each node. Thus, the relaxed optimization problems associated with each individual node can be solved separately. From the perspective of the nodes, each node only considers its own AoS, and the action space is simplified to either transmitting or not in each time slot. As the following proposition shows, the transmission policy has a threshold structure.

Proposition 3.

(Threshold policy). The relaxed optimization problem (Equations (60a)–(60c)) for each single node i has an optimal threshold policy. Under the policy, node i chooses to transmit when $s_i\left(k\right)\ge S_i$ and to remain idle when $0\le s_i\left(k\right)<S_i$ in time slot k. For given C and ${\theta }_i$, $\forall i\in \{1,\dots ,M\}$, and if $C>{\theta }_i$, the threshold is

$$\begin{array}{c}\hfill S_i=⌊\frac{5}{2}-\frac{1}{p_i}-\frac{1}{{\lambda }_i}+\sqrt{{\left(\frac{5}{2}-\frac{1}{p_i}-\frac{1}{{\lambda }_i}\right)}^2+2\left(\frac{C-{\theta }_i}{{\alpha }_i{p}_i}-\frac{1-2{\lambda }_i}{{\lambda }_i}\frac{1-p_i}{p_i}\right)}⌋.\end{array}$$

(61)

If $C\le {\theta }_i$, $S_i=0$.

When $C\le {\theta }_i$, the coefficient of $\mathbb{E}\left[u_i\left(k\right)\right]$ in the objective function (Equation (60a)) is $C-{\theta }_i\le 0$. In that case, node i will always choose to transmit, as there is no constraint on transmission, which is equivalent to $S_i=0$. When $C>{\theta }_i$, the proof of Proposition 3 follows from Appendix E in [19].

The final issue is how to choose the values of C and ${\theta }_i$. In order to examine the influence of the multipliers on the objective function, we can derive a lower bound of $Ao{S}_{re}^{*}$. Let $C>{\theta }_i$, as the function will fail to meet the bandwidth constraint otherwise. We use $\mathbb{E}\left[s_i\left(k\right)\right]\ge S_i-1$ and $\mathbb{E}\left[u_i\left(k\right)\right]\ge 0$

$$\begin{array}{c}\hfill \underset{C,{\theta }_i}{max}\left(L(C,{\theta }_i)\right)\le Ao{S}_{re}^{*},\end{array}$$

(62)

where

$$\begin{array}{c}\hfill L(C,{\theta }_i)=\frac{1}{M}\sum _{i=1}^M\left[{\alpha }_i(S_i-1)-\frac{C}{M}+\frac{q_i{\theta }_i}{p_i}\right].\end{array}$$

(63)

Taking the partial derivative of C and ${\theta }_i$

$$\begin{array}{cc}\hfill \frac{\partial L(C,{\theta }_i)}{\partial {\theta }_i}& =-\frac{1}{M{\varphi }_i}+\frac{q_i}{M{p}_i};\hfill \end{array}$$

(64)

$$\begin{array}{cc}\hfill \frac{\partial L(C,{\theta }_i)}{\partial C}& =\frac{1}{M}\left(-1+\sum _{i=1}^M\frac{1}{{\varphi }_i}\right),\hfill \end{array}$$

(65)

where

$$\begin{array}{c}\hfill {\varphi }_i=p_i\sqrt{{\left(\frac{5}{2}-\frac{1}{p_i}-\frac{1}{{\lambda }_i}\right)}^2+2\left(\frac{C-{\theta }_i}{{\alpha }_i{p}_i}-\frac{1-2{\lambda }_i}{{\lambda }_i}\frac{1-p_i}{p_i}\right)}.\end{array}$$

(66)

Let Equation (64) equal 0

$$\begin{array}{cc}\hfill \frac{1}{{\varphi }_i}& =\frac{q_i}{p_i},\forall i\in \{1,\dots ,M\};\hfill \end{array}$$

(67)

Substituting Equation (67) into (65), we derive

$$\frac{\partial L(C,{\theta }_i)}{\partial C}=\frac{1}{M}\left(-1+\sum _{i=1}^M\frac{q_i}{p_i}\right)\le 0.$$

(68)

To maximize $L(C,{\theta }_i)$, we can continuously decrease C until Equation (65) equals 0. The initial value of C is given by Equation (67) and $C-{\theta }_i\ge 0$, $\forall i$. The values of C and ${\theta }_i$ can be given by Algorithm 2.

Then, the distributed policy works in the following way. Each node i monitors its own AoS $s_i\left(k\right)$ and transmits to the BS when $s_i\left(k\right)$ exceeds the threshold $S_i$. If multiple nodes attempt to transmit to the BS in a time slot, only one randomly selected node will be able to transmit, and the transmissions of the other nodes will be treated as failures.

Algorithm 2 Solution of the distributed policy

1: Initialization: M, ${\alpha }_i$, $p_i$, $q_i$ and ${\lambda }_i$ 2: ${\chi }_i=\frac{{\alpha }_i{p}_i}{2}\left({\left(\frac{1}{q_i}\right)}^2-{\left(\frac{5}{2}-\frac{1}{p_i}-\frac{1}{{\lambda }_i}\right)}^2+\frac{2(1-2{\lambda }_i)(1-p_i)}{{\lambda }_i{p}_i}\right)$, $\forall i\in \{1,2,\dots ,M\}$ 3: $C={max}_i\left\{{\chi }_i\right\}$ 4: ${\chi }_i=min\{C,{\chi }_i\}$, $\forall i\in \{1,2,\dots ,M\}$ 5: ${\varphi }_i=p_i\sqrt{{\left(\frac{5}{2}-\frac{1}{p_i}-\frac{1}{{\lambda }_i}\right)}^2+2\left(\frac{{\chi }_i}{{\alpha }_i{p}_i}-\frac{(1-2{\lambda }_i)(1-p_i)}{{\lambda }_i{p}_i}\right)}$, $\forall i\in \{1,2,\dots ,M\}$ 6: while ${\sum }_{i=1}^M{\varphi }_i^{-1}<1$do 7:    decrease C slightly 8:    repeat steps 4 and 5 9: end while 10: Output: $C^{*}=C$ and ${\theta }_i^{*}=C^{*}-{\chi }_i$, $\forall i\in \{1,2,\dots ,M\}$

5. Simulation Results

In this section, we provide simulation results for the optimal stationary randomized policy, the max-weight policy, and the distributed policy. For comparison, we simulated the round robin policy, the largest-weighted-debt-first policy, and the drift-plus-penalty policy. The round robin policy, which has no randomness and functions regardless of the system state, chooses each node that will transmit in turn. The largest-weighted-debt-first policy is proposed in [26] and chooses the nodes with the highest $x_i\left(k\right)/p_i$ in time slot k. The drift-plus-penalty policy proposed from [25] chooses the node with the highest $W_i\left(k\right)=\frac{{\beta }_i{p}_i}{2}{h}_i\left(k\right)+V{p}_i{x}_i^{+}\left(k\right)$, where $h_i\left(k\right)$ is the age of information of node i and ${\beta }_i$ is a positive real value associated with node i. When transmission feedback is unavailable, only the optimal stationary randomized policy and the round robin policy can be used.

We simulated a network with M nodes. Node i has the weight ${\alpha }_i=i/{\sum }_{i=1}^{M}i$. Arriving rates of the M new updates are evenly distributed between $0.5$ and $0.1$. The set of channel reliability values $p_i$ is a vector of M evenly spaced points between $0.9$ and $0.1$. A parameter $\epsilon \in [0,1)$ shows the hardness of the throughput constraints to be satisfied. The set of $q_i/p_i$ sums to $\epsilon$ and is proportional to a set of M evenly spaced points between $0.06$ and $0.1$. The importance of the throughput constraint in the max-weight was set to $V=100$.

Figure 5 shows the performance of the throughput of the policies over time. The max normalized throughput debt was defined as ${max}_i\{x_i^{+}(K+1)/\left(K{q}_i\right)\}$. The system was set with $M=10$, $K={10}^5$, and $\epsilon =0.8$, and the results were taken as the average from 100 simulations. After the initial rise, the max normalized throughput debt of the optimal stationary randomized policy, the max-weight policy, the round robin policy, the largest-weighted-debt-first policy, and the drift-plus-penalty policy gradually approached zero with different speeds, which proves that these policies can satisfy throughput constraints in steady state. It is reasonable that the largest-weighted-debt-first policy descended fastest, as its optimization objective is the same as this measurement. Notice that the max normalized throughput debt of the distributed policy could not reach zero, meaning that the distributed policy could not give a throughput guarantee. However, compared with the results for the distributed policy without the multipliers ${\theta }_i$, our distributed policy has made great progress in meeting the throughput constraints.

Figure 6 compares the expected weighted-sum AoS of five policies over time and the lower bound defined by Equations (10a)–(10c). The system was set with $M=10$, $K={10}^5$, and $\epsilon =0.8$. The results were taken as the average from 100 simulations. The optimal stationary randomized policy and the round robin policy had significant disadvantages compared to the other policies because they are unable to use the system states. However, the optimal stationary randomized policy performed much better than the round robin policy. The steady AoS of the optimal stationary randomized policy was close to twice the lower bound, which matches the previous analysis. When transmission feedback was available, it was clear that max-weight performed best regarding the expected weighted-sum AoS and was close to the lower bound. Though the AoS of the distributed policy grew faster at the beginning due to the threshold structure, it gradually converged to a good level close to the largest-weighted-debt-first policy.

Figure 7 compares the expected weighted-sum AoS of the policies and the lower bound with different numbers of nodes, which were set by $M\in \{5,10,\dots ,45,50\}$. The total simulation time was $K={10}^5$. Each result was the average of the data for the time duration $0.9\times {10}^5$ to ${10}^5$. The hardness of throughput constraints was $\epsilon =0.8$. The performance of the six policies generally kept stable gaps and moved closer to the lower bound while the number of nodes grew. When the number of nodes was very large, the distributed policy showed greater potential for progress and the AoS of the max-weight policy was very close to the lower bound.

Figure 8 compares the expected weighted-sum AoS under different throughput constraints. We set the hardness of the throughput constraints $\epsilon$ to change from $0.5$ to $0.95$. A larger $\epsilon$ means the throughput constraints are closer to the upper bound limited by Assumption 1. The system was set with $M=10$, $K={10}^5$. Each result was the average of the data for the time duration $0.9\times {10}^5$ to ${10}^5$. When the hardness of satisfying the throughput constraints increased, each policy had to prioritize the throughput and had fewer resources available to reduce the AoS. Thus, it can be seen that the gap between the max-weight, the drift-plus-penalty, and the largest-weighted-debt-first policies and the gap between the optimal stationary randomized and round robin policies decreased with the growth in $\epsilon$. The distributed policy showed a similar trend as the max-weight policy.

6. Conclusions

In this study, we investigated a wireless network that transmits time-sensitive data from multiple nodes to a single base station (BS) over a shared imperfect channel while ensuring the minimum throughput requirement of each node. We measured the freshness of information stored at the BS with a metric called the age of synchronization (AoS). The objective of this study was to minimize the time-average AoS of the entire network while meeting the throughput requirement of each node. We proposed three low-complexity scheduling policies under different scenarios: the optimal stationary randomized policy, the max-weight policy, and the distributed policy. Numerical simulations showed that, in the central control scenario, the optimal stationary randomized policy and the max-weight policy could meet the minimum throughput requirement well. In the case where transmission feedback was unavailable to the BS, the optimal stationary randomized policy showed good AoS performance, and when transmission feedback was available, the max-weight policy closely approximated the AoS lower bound. In the distributed scenario, though the throughput requirement could not be guaranteed, the distributed policy made a great contribution toward meeting the throughput requirement and performed well in terms of AoS.