Analysis of a Queue with General Service Demands and Multiple Servers with Variable Service Capacities

Abstract

We present the study of a non-classical discrete-time queuing system in which the customers each request a variable amount of service, called their “service demand”, from a system with multiple servers, each of which can provide a variable amount of service, called their “service capacity”, in each time slot. The service demands are independent from customer to customer and follow a general distribution, whereas the service capacities follow a phase-type distribution and are independent from server to server and from slot to slot. Since an exact analytical analysis for this general queuing system is infeasible, we propose several approximations for the key performance characteristics in this system such as the mean system content and the mean customer delay in steady state. The accuracy of each of these approximations is compared to simulations using several numerical examples.

1. Introduction

Consider a queuing phenomenon in which the time it takes to serve each customer is variable. Two of the most common causes for this variability in the service times are that the amount of work that each customer requires from the system can vary from customer to customer, and that the rate at which the server(s) in the system can serve the customers can vary over time and/or from server to server in the case of multi-server systems.

In the classical queuing-theory literature, several kinds of queuing models are used to study such queuing phenomena. One class of queuing models uses the concept of “service times”, i.e., the time it takes to serve each customer is modeled explicitly. These service times are usually assumed to be independent and identically distributed (i.i.d.) from customer to customer. This kind of queuing model is commonly used to model queuing phenomena where the reason for the variable service times is the first reason above, i.e., the demands of each customer are variable. Another class of queuing models that is widely used in the literature uses the concept of “service rate”, especially in continuous-time queuing models and for queuing phenomena where the reason for the variable service times is the second reason above, i.e., that the capacity of the system to serve customers varies over time.

However, in some queuing phenomena, both the service requirements of each customer and the rate of service over time can be variable. Examples include grocery stores with variable-sized shopping carts and servers with variable levels of experience, web services with variable types of requests and variable server capacity due to spot pricing, Wi-Fi connections with variable packet sizes and variable connection quality, etc. For these kinds of queuing phenomena, neither the concept of service time nor the concept of service rate is sufficient to fully describe the queuing system. For these kinds of queuing phenomena, it is more precise to model the “service demands”, i.e., the amount of work that each customer requires from the system, and the “service capacities” of the server(s), i.e., the amount of work that each server can provide in a given time slot, explicitly.

Several studies in the scientific literature have proposed and analyzed queuing models with variable service demands and variable service capacities or rates of service in both continuous time (see, e.g., [1,2,3]) and in discrete time (see, e.g., [4,5,6,7,8,9,10,11,12]). However, these discrete-time studies are primarily focused on single-server queuing phenomena. To the best of our knowledge, no studies in the literature have studied discrete-time queuing models with variable service demands, variable service capacities, and multiple servers.

Even more classical multi-server queuing models are notoriously difficult to study analytically. For instance, in the queuing-theory literature, no closed-form expressions have been reported for the steady-state system-content or customer-delay distributions of the continuous-time $G/G/c$ queuing model with c servers and generally distributed inter-arrival times and service times. The same is true for the discrete-time $Ge{o}^X/GI/c$ queuing model with so-called general independent arrivals from slot to slot (i.e., with geometrically distributed inter-arrival times between batch arrivals), generally distributed service times and c servers. Herein, the standard Kendall’s notation $A^X/B/c$ is used to describe a queuing model, as can be seen in [13], where A refers to the distribution of the inter-arrival times; X refers to a random variable for the arrival batch size that is only present in case of batch arrivals; B refers to the service-time distribution; and c refers to the number of servers. Exact results for the steady-state system-content or customer-delay distributions have only been reported for special cases, where specific model restrictions are introduced on, e.g., the number of servers, the distribution of the inter-arrival times, and/or the service-time distribution. Examples of such classical multi-server queuing models are [14,15,16,17,18] in a continuous-time setting and [19,20,21,22,23,24,25,26] in a discrete-time setting.

It may seem surprising that multi-server systems are more difficult to analyze than single-server systems with variable service capacity. After all, a system with two servers which each have a service capacity of x seems reasonably similar to a system with one server with a service capacity of $2x$. However, there is a crucial difference between those two systems, which is that the single-server system is work-conserving whereas the multi-server system is not. Indeed, if there is exactly one customer in the system, then in the multi-server system, this customer will be served by one of the two servers while the other server is idle and its service capacity remains unused, whereas in the single-server system, the entire service capacity is used to serve the customer. This fundamental difference is what makes it impossible to precisely analyze the multi-server system using the techniques available in the current literature.

Therefore, in this paper, we develop several techniques which may be used to obtain approximations for the key performance characteristics such as the mean number of customers in the system at the beginning of an arbitrary slot or the mean delay of an arbitrary customer. Some of these are focused on being accurate only in specific situations, such as low or high traffic load. We also propose and analyze one approximation, called the Two-Phase approximation, which is aimed at being accurate regardless of the situation. These various approximations are then verified using several numerical simulations.

The rest of this paper is structured as follows. In Section 2, we formalize the multiserver queuing model studied in this paper and define the mathematical notation used in this paper. The various approximations for the probability generating functions (pgfs) and means of the system-content and customer-delay distributions are discussed in Section 3. In Section 4, we compare the accuracy of these approximations against simulations for several example scenarios. Section 5 provides a brief conclusion to the paper. Finally, in Appendix A and Appendix B, we state and prove two theorems which are used in the analysis of the considered queuing model.

The rest of this paper is structured as follows. In Section 2, we formalize the multiserver queuing model studied in this paper and define the mathematical notation used in this paper. The various approximations for the probability generating functions (pgfs) and means of the system-content and customer-delay distributions are discussed in Section 3. In Section 4, we compare the accuracy of these approximations against simulations for several example scenarios. Section 5 provides a brief conclusion to the paper. Finally, in Appendix A and Appendix B, we state and prove two theorems which are used in the analysis of the considered queuing model.

2. Queuing Model

In this section, we give a precise description of the queuing model studied in this paper, as well as the mathematical notation used in the analysis.

The studied queuing model is a discrete-time queuing model, i.e., time is divided into (time) slots. Customers that arrive during a time slot are only allowed into the system at the end of that slot. The number of customers that arrive during time slot k is denoted as $A_k$. These numbers of arrivals $\left\{A_k\right|k=1,2,...\}$ are assumed to be i.i.d. random variables with a general common distribution with pgf $A\left(z\right)$, i.e.,

$$A\left(z\right)\triangleq \sum _{n=0}^{\infty }P\left(\mathrm{arbitrary}\mathrm{slot}\mathrm{contains}n\mathrm{arrivals}\right)z^n.$$

(1)

The mean number of arrivals per slot is denoted as $\lambda \triangleq A^{\prime }\left(1\right)$.

The “service demands”, i.e., the amount of work that each customer requires from the system, are assumed to be positive integer numbers of “work units”. The service demand of customer k is denoted as $S_k$. The service demands $\left\{S_k\right|k=1,2,...\}$ are assumed to be i.i.d. random variables with a general common distribution with pgf $S\left(z\right)$. The mean service demand is denoted as $\tau \triangleq S^{\prime }\left(1\right)$.

There are c servers in our queuing model for some given integer $c\ge 1$. In each time slot, each of these servers has a “service capacity”, which is the maximum amount of work that the server can provide to customers during that time slot. The service capacity of server i during slot k is an integer number of work units, denoted as $R_{i,k}$. The service capacities $\left\{R_{i,k}\right|i=1,2,...,c,k=1,2,...\}$ are assumed to be i.i.d. random variables with a common distribution with pgf $R\left(z\right)$. The mean service capacity of a server (per slot) is denoted as $\mu \triangleq R^{\prime }\left(1\right)$. The pgf $R\left(z\right)$ is assumed to be a rational function. We therefore introduce the mutually prime polynomials $P_R\left(z\right)$ and $Q_R\left(z\right)$ such that

$$R(1/z)=\frac{P_R\left(z\right)}{Q_R\left(z\right)},$$

(2)

and we let m denote the degree of $Q_R\left(z\right)$.

The $R_{i,k}$ work units of service capacity are always executed in a work-conserving manner, i.e., if the available service capacity of a server is larger than the (remaining) service demand of the customer in service by that server, then a next customer can immediately begin service (during the same time slot). This repeats until either the available service capacity of the server runs out or there are no more customers left in the queue. As such, it is possible that many customers are served during one slot by one server. Conversely, if the remaining service demand of the customer in service by server i is larger than $R_{i,k}$, then this remaining service demand simply decreases by $R_{i,k}$ and the service of the customer continues in the next slot $k+1$. As such, one customer’s service may last many slots.

The scheduling mechanism by which customers are allocated to servers is as follows. There is a single queue which is shared between all servers. Each customer can only receive service from a single server. At the beginning of each slot, the customer at the front of the queue goes to a random server that has at least 1 work unit of service capacity available. This is repeated until either there are no more customers in the queue or all servers are serving customers with a total (remaining) service demand that is greater than or equal to their service capacity during that slot.

The number of customers in the system at the beginning of the kth time slot is denoted as $B_k$. This is also called the system content at the beginning of the kth slot. The delay of the kth customer, which is the whole number of slots between the end of this customer’s arrival slot and the end of the last slot during which the customer receives service, is denoted as $D_k$. The two main performance characteristics that we are interested in are the limiting distributions of $B_k$ and $D_k$ as k goes to infinity, i.e., in so-called “steady state”. When these two random variables have a limiting distribution, the system is said to be “stable”. This is the case when the mean total service demand of all customers arriving per slot is less than the mean service capacity per slot, i.e., $\lambda \tau <c\mu$, or alternatively, when the load $\rho =\lambda \tau /\mu c<1$. When the system is stable, we denote the pgf of the limiting distribution of $B_k$ and $D_k$ as, respectively, $B\left(z\right)$ and $D\left(z\right)$. Alternatively, $B\left(z\right)$ is the pgf of the system content at the beginning of an arbitrary slot in steady state, and $D\left(z\right)$ is the pgf of the delay of an arbitrary customer in steady state.

A concept that will also be used in the analysis of this system is the concept of “unfinished work”, which is simply the sum of all remaining work units of the service demand of all customers in the system. The unfinished work observed by a customer C is defined as the unfinished work in the system right after customer C arrives, i.e., it is the sum of three independent terms:

• The (remaining) service demand of all customers already in the system before the arrival slot of customer C and still in the system at the end of the arrival slot of customer C;
• The work units of other customers arriving in the same slot as customer C but arriving before customer C;
• The service demand $S_C$ of customer C itself.

3. Approximations

As mentioned in the introduction, obtaining the exact expressions for the system-content and the customer-delay distributions for this general model is extremely difficult. To the best of our knowledge, none of the techniques commonly used in the queuing-theory literature can solve this queuing model exactly. Therefore, in this section, we will explore several approaches to obtain approximations for the pgfs of the system-content and customer-delay distribution.

3.1. Geometric Service Times: The GI-Geom-c Approximation

If the mean service demand $\tau$ of a customer is larger than the mean service capacity $\mu$ of each server, then we can obtain our first approximation by assuming that, in each server, only one customer may be in service at the same time, and in each slot, the service of any customer in service ends with the probability of $\mu /\tau$, independently from slot to slot and from customer to customer. Under these assumptions, the system behaves identically to the classical multi-server model with (shifted) geometric service times with mean $\tau /\mu$. An expression for the pgf of the steady-state system content and customer delay was obtained in [19]. We refer to this approximation as the GI-Geom-c approximation.

3.2. Deterministic Service Times: The GI-D-c Approximation

Similarly to the previous approximation, when $\tau$ is an integer multiple of $\mu$, we can obtain another approximation by considering the system as a classical multi-server queuing system with deterministic service times of $\tau /\mu$ slots. An expression for the pgf of the system content for this type of system was obtained in [20], and an expression for the pgf of the customer delay was obtained in [21]. We refer to this approximation as the GI-D-c approximation.

3.3. The Single-Server Approximation

The next approximation that we will investigate, which we will refer to as the Single-Server approximation, is to consider the c servers as a single server whose service capacity in each slot is equal to the combined service capacity of the c servers in the original queuing system during that same slot. In other words, this is the same as assuming that the c servers all work together on serving the customer at the front of the queue, in a strictly FCFS manner, rather than each server serving their own customer(s) or potentially being idle if the number of customers in the system is less than c.

Since the service capacity of any given server in an arbitrary slot in steady state is assumed to be independent from the service capacities of the other servers and to have rational pgf $R\left(z\right)$, the combined service capacity of all c servers has pgf $R{\left(z\right)}^c$. This pgf $R{\left(z\right)}^c$ is also a rational function, where the denominator of $R{(1/z)}^c$ is a polynomial in z of degree $cm$, with m being the degree of the denominator of $R(1/z)$. An expression for the pgf, mean, and other moments of the system-content and customer-delay distributions in this single-server model was obtained in [10]. Note that the roots for $\xi$ of the characteristic equation $1-zR{(1/\xi )}^c=0$ can be found by calculating the roots of the characteristic equation $1-z_{i}R(1/\xi )=0$ of the single-server system, where $z_i,i=0,...,m-1$ are the complex cth order roots of z. This implies that the former set of roots are distinct for all z if and only if the latter roots are distinct for all z (as can also be seen in [10], Appendix D).

3.4. The Infinite-Server Approximation

Another approximation, which is only useful for situations where the load is very low, is to assume that each arriving customer always finds an idle server and can begin its service in the first slot after its arrival slot. This is effectively the same as assuming that there are an infinite number of servers, so we refer to this approximation as the Infinite-Server approximation.

With this assumption of an infinite number of servers, the distribution of the customer delay is very straightforward to calculate. Since each customer always finds an idle server upon arrival, it can begin service in the next slot after its arrival, and it finishes service as soon as the cumulative service capacity of the server it is assigned to matches or exceeds the service demand of the customer. We can therefore use Theorem A1 in Appendix A to obtain our approximation of the pgf $D\left(z\right)$ of the customer delay $D_C$ as

$$D\left(z\right)=\frac{z-1}{z}\sum _{k=0}^{m-1}\frac{S\left({\alpha }_k\left(z\right)\right)}{R^{\prime }(1/{\alpha }_k\left(z\right))}·\frac{{\alpha }_k\left(z\right)}{{\alpha }_k\left(z\right)-1},$$

(3)

where the zeros ${\alpha }_k\left(z\right)$ are, as defined in Theorem A1, the m zeros for $\xi$ of (A2). Note that this result does not depend on the pgf $A\left(z\right)$ of the number of arrivals, as is expected for an infinite-server system.

3.5. The Two-Phase Approximation

Obviously, the above Infinite-Server approximation is very poor if the load is high, but it turns out to be a very good approximation for low loads. In fact, unlike all our other approximations so far, this approximation does become exact as the load goes to 0.

Conversely, the behavior of the Single-Server approximation is in many regards the opposite. For low loads, where most customers are served while they are the only customer in the system, the Single-Server approximation assumes that these lone customers are served with the combined service capacity of all servers in the system, while in reality, they only receive service from a single server with a mean service capacity $\mu$. Thus, if the load is low, the Single-Server approximation is expected to be a poor approximation. However, for a high load, where most of the time there are many customers in the system and the delays of most customers are very long, the Single-Server approximation becomes a lot better. Indeed, while a customer C is waiting in the queue because all servers are fully occupied, the system with multiple servers behaves very similarly to the Single-Server approximation with regard to customer C. In this case, the number of unfinished work units that are “in front” of customer C, i.e., belonging to customers that are either in service or in front of customer C in the queue, is easily seen to decrease every slot by the combined service capacity of all servers. The difference between both systems only manifests itself for customer C when customer C enters service. At that point, it only takes approximately $\tau /\mu$ slots on average before customer C leaves the multi-server system. As this part, $\tau /\mu$ does not depend on the arrival rate $\lambda$ while the total mean customer delay grows unboundedly as $\lambda$ increases, the relative difference between the multi-server system and the Single-Server approximation goes to 0 as the load goes to 1.

We therefore have one approximation, namely the Infinite-Server approximation, that is very good if the load is low, but terrible if the load is high, and another approximation, namely the Single-Server one, that is good for high load, but poor for low load. Luckily, it is possible to combine both approaches and create an approximation that is good for both high and low load. This is exactly what we will do in this subsection.

Specifically, we proceed by approximating the number of slots that the customer spends in the shared queue, i.e., the “waiting time”, and the number of slots that the customer spends in service, i.e., the “service time”, separately, and we make the simplifying assumption that these two quantities are independent.

Note that the service-time distribution is not simply a function of the service-demand and the service-capacity distributions. It also depends on other quantities including the distribution of the number of arrivals and the load. For instance, when the load is low, most customers arrive in an empty system, and the available service capacity for the first slot of service of a customer will be close to the service capacity in an arbitrary slot. On the contrary, if the load is high, most customers’ first slot of service is also the last slot of service of a different customer, and the available service capacity of the server during that slot is divided between both customers. Generally, determining the exact service-time distribution is expected to be as difficult as determining the distribution of the steady-state system content or customer delay.

As already indicated, the goal in this subsection is to obtain an approximation which is simultaneously accurate for both high and low loads. Because the waiting time is the dominant term of the customer delay when the load is high, while the service time is the dominant term when the load is low, we will attempt to find an approximation for the waiting-time distribution which is as accurate as possible when the load is high while we will attempt to find an approximation for the service-time distribution which is as accurate as possible when the load is low. The latter is achieved by assuming that, during the first slot of service of an arbitrary customer C, the distribution of the service capacity that the server can spend on this customer is the same as the distribution of the service capacity in an arbitrary slot, i.e., that the server that the customer went to does not have other customers making use of the server’s available service capacity. The probability that this is true approaches 1 as the load approaches 0, because then the expected number of other customers in the system at the beginning of the arrival slot of an arbitrary customer also approaches 0. With this assumption, our approximation for the service-time distribution is in fact the same as the distribution of the customer delay under the Infinite-Server approximation. We denote the pgf of this distribution as $S_{\infty }\left(z\right)$.

Approximating the waiting-time distribution of the arbitrarily selected customer C is more difficult. As the waiting time is the dominant term of the customer delay $D_C$ for a high load, we want our approximation to be as accurate as possible when the load is high. Under high load, in most slots, it is the case that all servers are fully occupied serving customers, i.e., using all their available service capacity. During these slots, the service process is work-conserving, and the unfinished work in the system decreases by the combined service capacity of all servers (and increases by the combined service demand of all arrivals), just like in a single-server system. It is therefore reasonable to assume that if the load is high, the distribution of the unfinished work (both at the beginning of an arbitrary slot and as observed by an arbitrary customer) is similar to that in a single-server system with the same total service capacity (as considered under the Single-Server approximation), with the relative difference between the two decreasing as the load increases.

We will now obtain an approximation for the pgf $V\left(z\right)$ of the unfinished work $V_C$ observed by an arbitrary customer C. As noted in Section 2, $V_C$ consists of three independent terms:

• The (remaining) service demand of all customers already in the system before the arrival slot of customer C and still in the system at the end of the arrival slot of customer C;
• The work units of other customers arriving in the same slot as customer C but arriving before customer C;
• The service demand $S_C$ of customer C itself.

We now consider each of these three terms separately. The third term, of course, simply has pgf $S\left(z\right)$. For the second term, it is a well-known property from renewal theory (see, e.g., [27,28]) that for any queue with independent, ordered arrivals, the pgf $F\left(z\right)$ of the number of customers arriving in the same slot as an arbitrary customer C but arriving before customer C is given by

$$F\left(z\right)=\frac{A\left(z\right)-1}{\lambda (z-1)}.$$

(4)

The pgf of the total number of work units of these customers, i.e., the second term above, is then given by $F\left(S\right(z\left)\right)$. For the first term, due to the reasoning in the previous paragraphs, the pgf of the first term is approximately equal to $U\left(z\right)/A\left(S\right(z\left)\right)$ (as also seen in Appendix B), where $U\left(z\right)$ is now the pgf of the unfinished work at the beginning of an arbitrary slot in the multi-server system as obtained under our Single-Server approximation, for which an expression can be obtained from Theorem A2 (for the case where the single server has a service-capacity pgf $R{\left(z\right)}^c$ and mean service capacity $c\mu$).

We now wish to approximate the pgf $W\left(z\right)$ of the waiting time $W_C$ of the arbitrary customer C. To do this, we begin by defining ${\widehat{V}}_C$ as the unfinished work observed by customer C belonging to customers other than C. In other words, ${\widehat{V}}_C$ is equal to the sum of the first and second term of the unfinished work $V_C$ observed by customer C above. We also define $R^{(k,c)}$ as the combined service capacity of all c servers during the k slots following the arrival slot of customer C. Note that the pgf of $R^{(k,c)}$ is $R{\left(z\right)}^{kc}$, while the pgf $\widehat{V}\left(z\right)$ of ${\widehat{V}}_C$ is

$$\begin{array}{cc}\hfill \widehat{V}\left(z\right)& =\frac{U\left(z\right)}{A\left(S\right(z\left)\right)}\frac{A\left(S\right(z\left)\right)-1}{\lambda \left(S\right(z)-1)}.\hfill \end{array}$$

(5)

If $R\left(z\right)$ is a rational function, where $P_R\left(z\right)$ and $Q_R\left(z\right)$ denote the mutually prime polynomials in, respectively, the numerator and denominator of $R(1/z)$, we can use Theorem A2 to obtain the expression

$$\begin{array}{cc}\widehat{V}\left(z\right)& =\frac{c\mu -\lambda \tau }{\lambda }·\frac{z-1}{1-R{(1/z)}^{c}A\left(S\left(z\right)\right)}·\frac{A\left(S\right(z\left)\right)-1}{S\left(z\right)-1}\\ & ·{\displaystyle \prod _{\zeta \in {\mathcal{S}}_R^{-1}}}{\left(\frac{1-\zeta }{z-\zeta }\right)}^{{\mu }_{\zeta }c}{\displaystyle \prod _{\xi \in {\mathcal{N}}_{T_c}^{-}}}{\left(\frac{z-\xi }{1-\xi }\right)}^{n_{\xi }},\end{array}$$

(6)

where ${\mathcal{S}}_R^{-1}$ denotes the set of poles of $R(1/z)$, ${\mu }_{\zeta }$ denotes the multiplicity of the pole $\zeta \in {\mathcal{S}}_R^{-1}$, ${\mathcal{N}}_{T_c}^{-}$ denotes the set of zeros of $T_c\left(z\right)\triangleq Q_R{\left(z\right)}^c-A\left(S\left(z\right)\right)P_R{\left(z\right)}^c$ inside or on the unit circle, excluding the zero at $z=1$, and $n_{\xi }$ denotes the multiplicity of a zero $\xi$ in this set.

Now note that if this combined service capacity $R^{(k,c)}$ of all servers during the k slots after the arrival slot of customer C is larger than ${\widehat{V}}_C$, then customer C cannot be waiting in the queue anymore after k slots. Indeed, while a customer is in the queue, all servers must be fully occupied, in which case the unfinished work in front of that customer decreases by the combined service capacity of all servers. Thus, if the customer C would still be waiting in the queue after k slots, the unfinished work in front of the customer would be equal to ${\widehat{V}}_C-R^{(k,c)}$, which in this case is negative, and therefore impossible. In conclusion, we have the following implication:

$$R^{(k,c)}>{\widehat{V}}_C\Rightarrow W_C<k.$$

(7)

Unfortunately, the reverse of this implication is not necessarily true. In other words, if the combined service capacity $R^{(k,c)}$ during the k slots after customer C’s arrival slot is less than or equal to the unfinished work ${\widehat{V}}_C$, then that does not necessarily mean that the customer C is still waiting in the queue after k slots. Indeed, it may happen that some of these ${\widehat{V}}_C$ work units in front of customer C belong to a customer $C_2$ that entered the service with a server whose (remaining) available service capacity is less than the service demand of the customer $C_2$. In that case, the service of customer $C_2$ will not finish during that slot, even if there are other servers with available service capacity, because each customer can only receive the service from a single server. These other servers will instead serve different customers, potentially including the arbitrary customer C. Thus, while these excess work units of the service demand of customer $C_2$ count towards the unfinished work ${\widehat{V}}_C$ in front of customer C, they do not inhibit customer C from going to a different server. Note that intuitively, such a situation where the reverse of (7) does not hold, it seems more likely in case the variance of the service demands is higher.

Calculating the exact value that the combined service capacities of the servers needs to reach in order for customer C to be able to begin its service, or calculating the exact waiting time $W_C$ of customer C, is very difficult. Therefore, we make a final simplifying assumption that the implication (7) is nonetheless a two-way relationship. In other words, we assume that

$$R^{(k,c)}>{\widehat{V}}_C\iff W_C<k.$$

(8)

Note that if the service demands of all customers other than C are equal to one work unit, then this relationship (8) is always true, because if ${\widehat{V}}_C\ge R^{(k,c)}$ then there are ${\widehat{V}}_C-R^{(k,c)}\ge 0$ customers in front of customer C in the queue at the beginning of the kth slot after the arrival slot of customer C, so that customer C is still waiting in the queue after k slots. However, if the service demands of the other customers are not always equal to 1 work unit, the assumption that (8) is true is an approximation. Moreover, as already indicated above, it is intuitively expected that (8) is more likely to break down in case of service demands with a higher variance.

Under the assumption that the waiting time $W_C$ ends when the accumulated service capacity $R^{(k,c)}$ matches or exceeds ${\widehat{V}}_C$, we can use Theorem A1 to obtain our approximation for the pgf $W\left(z\right)$ of the waiting time as

$$\begin{array}{cc}W\left(z\right)\hfill & =\frac{z-1}{z}{\displaystyle \sum _{k=0}^{cm-1}}\frac{\widehat{V}\left({\alpha }_k^{\left(c\right)}\left(z\right)\right)}{c{R}^{\prime }(1/{\alpha }_k^{\left(c\right)}\left(z\right))R{(1/{\alpha }_k^{\left(c\right)}\left(z\right))}^{c-1}}·\frac{{\alpha }_k^{\left(c\right)}\left(z\right)}{{\alpha }_k^{\left(c\right)}\left(z\right)-1}\hfill \\ \hfill & =\frac{c\mu -\lambda \tau }{\lambda }{\displaystyle \sum _{k=0}^{cm-1}}\frac{1-z}{c{R}^{\prime }(1/{\alpha }_k^{\left(c\right)}\left(z\right))R{(1/{\alpha }_k^{\left(c\right)}\left(z\right))}^{c-1}}·\frac{{\alpha }_k^{\left(c\right)}\left(z\right)}{S\left({\alpha }_k^{\left(c\right)}\left(z\right)\right)-1}\hfill \\ \hfill & ·\frac{1-A\left(S\left({\alpha }_k^{\left(c\right)}\left(z\right)\right)\right)}{z-A\left(S\left({\alpha }_k^{\left(c\right)}\left(z\right)\right)\right)}{\displaystyle \prod _{\zeta \in {\mathcal{S}}_R^{-1}}}{\left(\frac{1-\zeta }{{\alpha }_k^{\left(c\right)}\left(z\right)-\zeta }\right)}^{{\mu }_{\zeta }c}{\displaystyle \prod _{\xi \in {\mathcal{N}}_{T_c}^{-}}}{\left(\frac{{\alpha }_k^{\left(c\right)}\left(z\right)-\xi }{1-\xi }\right)}^{n_{\xi }},\hfill \end{array}$$

(9)

where ${\alpha }_k^{\left(c\right)}\left(z\right)$ denotes the kth zero for $\xi$ of $Q_R{\left(\xi \right)}^c-z{P}_R{\left(\xi \right)}^c$.

Our final approximation for the pgf $D\left(z\right)$ of customer delay is then obtained using our aforementioned assumption that the waiting time and service time of a customer are independent. This leads to $D\left(z\right)=W\left(z\right)S_{\infty }\left(z\right)$, where $S_{\infty }\left(z\right)$ is the pgf of the customer delay under the Infinite-Server approximation.

We refer to the approximation derived in this subsection as the Two-Phase approximation, highlighting the fact that this approximation estimates the durations of the queuing phase and the service phase separately. Informally, this model essentially assumes that, while a customer C is waiting in the queue, the customer C must wait until the unfinished work belonging to other customers with service priority over C is executed, which is assumed to happen at a rate corresponding to the combined service capacity of all servers. In the second phase, while the customer C is in service, this customer must wait until the $S_C$ work units of its own service demand are executed, which is assumed to happen at a rate corresponding to the service capacity of a single server.

While this model only provides an approximation for the pgf of the customer delay, and while there is no clear way for obtaining a similar approximation for the pgf of the system content, it is well known that an approximation for the mean system content can always be obtained from the obtained approximation of the mean customer delay using Little’s law (as can be seen in [29,30]), as this (simple) relationship applies to any queuing system in steady state.

3.6. Monte-Carlo Simulations

Finally, in order to be able to compare the accuracy of the various approximations derived in this paper, we also use Monte-Carlo simulations of the model with multiple servers over ${10}^8$ time slots. While these simulations are in most cases more accurate than the other approximations derived in this paper and are therefore taken as the “ground truth” in the next section, these simulations can often take an extremely long time to complete, whereas the analytical expressions usually only take a few seconds to evaluate. For brevity, these simulations are referred to as the Monte-Carlo estimate in the next section.

4. Numerical Examples

In this section, we consider several numerical examples to test the accuracy of the various approximations described in the previous section.

In our first numerical example, shown in Figure 1, we study the impact of the load $\rho$ on the accuracy of our approximations of the mean customer delay. We consider a queuing system with $c=5$ servers, where the service capacities of the servers are independent and have a negative-binomial distribution with 5 phases and with a mean value of $\mu =10$ work units. The service demands have a (shifted) negative-binomial distribution, with five phases but with a mean value of $\tau =50$ work units. The number of arrivals per slot follows a Poisson distribution with mean $\lambda$ that is scaled such that the system obtains the desired load $\rho =\lambda \tau /\mu c$. In this example, $\lambda$ thus equals $\rho$.

Unsurprisingly, as the load $\rho$ increases to 1, the mean customer delay approaches ∞. As can be seen in Figure 1a, all of our approximations model this behavior correctly except for the Infinite-Server approximation, which does not take the arrival rate into account at all. For high loads, however, it is difficult to see in Figure 1a which approximations are most accurate. Therefore, in Figure 1b, we show each of our estimates relative to the estimate from the Monte-Carlo simulations, i.e., Figure 1b shows the ratio between each of our mean delay approximations and the Monte-Carlo estimate. In this figure, it can be seen that the Infinite-Server approximation is very accurate if the load is less than 0.4, but falls off dramatically afterwards, as expected. Conversely, the Single-Server approximation is very inaccurate for loads less than 0.9, but becomes more and more accurate as the load approaches 1. This agrees with the discussion in Section 3.5. It can also be seen that the GI-D-c approximation consistently underestimates the mean delay, which is in accordance with observations in [8,10] that a lower variance of the service-demand or service-capacity distributions typically results in lower mean system content as well as mean delay in single-server systems. The same thus also appears to be true for this multi-server model. The GI-Geom-c approximation interestingly underestimates the mean customer delay for low loads while it overestimates the mean delay for high loads. This can be explained by the fact that this approximation estimates the mean service time for all loads as simply $\tau /\mu$, in this case five slots, whereas the real mean service time for low load is slightly higher, namely approximately 5.6 slots, as can be seen, e.g., from the simulation results in Figure 1a, as the load $\rho$ goes to 0. For high loads, this model overestimates the mean customer delay because it assumes that the service times are geometrically distributed, whereas the relatively low variance (in comparison to the geometric distribution) of both the service-demand and the service-capacity distributions causes the real service times to have relatively low variance as well. This is true for all values of the load $\rho$. Finally, the Two-Phase approach accurately approximates the mean customer delay for both high and low loads $\rho$, as intended. Even for intermediate values of the load $\rho$, the Two-Phase estimate of the mean customer delay never deviates from the value obtained by the Monte-Carlo simulations by more than 5% (i.e., the ratio between the Two-Phase estimate and the Monte-Carlo estimate in Figure 1b is always between 0.95 and 1.05).

In our second numerical example, shown in Figure 2, we study the effect of the number of servers on the accuracy of our various approximations of the mean customer delay. The distributions of the service demands and the service capacities are the same as in the first numerical example. The distribution of the number of arrivals per slot is again a Poisson distribution, although for this example, the arrival rate $\lambda$ is always chosen such that the load $\rho =0.95$. This high load makes the Infinite-Server estimate a poor approximation, similar to the previous example for high $\rho$, especially if the number of servers c is low. It is no surprise that as the number of servers increases, the quality of the Infinite-Server approximation increases while the quality of the Single-Server approximation decreases. The Two-Phase approximation is relatively accurate regardless of the number of servers, deviating by no more than 4% from the Monte-Carlo estimate. As in the previous example, the GI-D-c approximation consistently underestimates the mean customer delay. The GI-Geom-c approximation overestimates the mean delay because the load is high, although it appears to become very accurate as the number of servers increases.

However, this last observation is mostly a coincidence, as we will demonstrate in our third numerical example, shown in Figure 3. In this example, all the parameters are the same as in our second example, with the only difference being that the service demands follow a “bursty” distribution, where each customer has a 95% chance of having a geometrically distributed service demand with mean 30 work units and a 5% chance of having a (shifted) geometrically distributed service demand with mean 430 work units. The mean service demand is still $\tau =50$ work units. Due to the fact that there is much more variance in the service demands, the mean customer delay is much higher in this case. As neither the GI-Geom-c nor the GI-D-c approximation take the variance of the service-demand distribution into account, these approximations significantly underestimate the mean customer delay. The Two-Phase approximation is also less accurate than in the previous example, deviating up to 25% from the Monte-Carlo estimate. This is because the assumption that (8) is true in the Two-Phase approach overestimates the waiting time $W_C$ by an amount that depends on the variance of the service-demand distribution. Indeed, as explained in Section 3.5, this assumption that (8) is true is exact if the service demands of the customers other than C are all deterministically equal to one work unit. Using a similar reasoning, it can be seen that the higher the variance of the service demands becomes, the less realistic this assumption becomes. For instance, if both the variance of the service demands as well as the amount of unfinished work ${\widehat{V}}_C$ in front of customer C are large, then it is not unlikely that most of these work units are nevertheless only part of the service demand of a single customer, and are therefore only occupying a single server, leaving the other servers free to serve other customers, potentially including the arbitrary customer C. Therefore, we conclude that the Two-Phase approximation becomes less accurate if the variance of the service demands is high. In Figure 3, it can even be seen that the Single-Server approximation is more accurate than the Two-Phase approximation in this case, although this is aided by the fact that the load $\rho =0.95$ is relatively high.

In our next numerical example, shown in Figure 4, we study the impact of the ratio $\tau /\mu$, which is a rough approximation for the mean “service time”. In this example, there are $c=5$ servers, and the service capacities follow a uniform distribution between 0 and 10 work units (inclusive), so that the mean service capacity $\mu$ of a server is equal to 5 work units, while the service demands are deterministically equal to $\tau$ work units. As the GI-D-c approximation is only defined when $\tau /\mu$ is a positive integer, in this example, we will only consider values of $\tau$ which are multiples of the mean service capacity $\mu =5$. The arrival process is again a Poisson process, and the arrival rate $\lambda$ is scaled such that the load $\rho$ is always equal to 0.9. In Figure 4, it can be seen that as the ratio $\tau /\mu$ increases, the mean customer delay also increases. This is a consequence of the fact that the mean “service time” of an arbitrary customer increases as $\tau /\mu$ increases. Note that in Figure 4, the Two-Phase approximation is relatively close to the Monte-Carlo estimate, with a relative error of no more than 5%. This is in agreement with the discussion in the previous numerical example with respect to the impact of the variance of the service-demand distribution on the accuracy in the Two-Phase approximation, because the variance of the service demands in this example is low (zero).

The mean customer delay is not the only performance characteristic that one might be interested in. Furthermore, the variance, other moments, tail probabilities, or the entire probability mass function of the customer delay may be of interest. In Figure 5, we show the probability mass function of the customer delay for the various approximations discussed in Section 3. The arrival, service-demand, and service-capacity distributions and the number of servers $c=5$ are the same as in Figure 1, with the arrival rate $\lambda$ chosen such that the load $\rho =0.7$. In Figure 5, it can be seen that the Two-Phase approach provides a good approximation for the probability distribution of the customer delay. In fact, it appears that the rate of exponential decay of the probability mass function is the same for the Single-Server approximation and the Two-Phase approximation as for the real multi-server queuing system. This is not only the case for this particular example, but it is true for all other examples that we have explored. This can be explained as follows. As mentioned in Section 3.5, when all servers are busy in a certain slot k, i.e., the total available service capacity is utilized during slot k, and the total number of unfinished work units in front of a customer in the queue decreases by the total combined service capacity of all servers. This is the same for both the Single-Server approximation and the Two-Phase approach (in the term corresponding to the queuing phase) as for the real system. Furthermore, as the delay of a customer C increases, the probability that all servers are occupied during all the slots where this customer C is in the system approaches 1. If there are many customers in front of customer C when this customer C arrived, then it will take a similar amount of time for customer C to reach the front of the queue regardless of whether there is only one server or multiple (provided, of course, that the combined service capacity is the same). Therefore, the ratio of $P(D_C=n+1)/P(D_C=n)$ should be similar for large n for the Single-Server estimate, the Two-Phase approach, and the real multi-server queuing system.

In our last numerical example, shown in Figure 6, we study the impact of the variance of the service-capacity distribution on the mean customer delay. In this example, there are $c=5$ servers, and the service capacities follow a negative-binomial distribution with a mean value of $\mu =5$ work units and r phases, for various r. As r increases, the variance of the service-capacity distribution decreases. The service demands follow a (shifted) negative-binomial distribution with five phases and a mean of $\tau =10$ work units, and the arrivals follow a Poisson distribution with a mean arrival rate $\lambda$ chosen such that the load $\rho =0.95$. Note that the GI-Geom-c and GI-D-c estimates do not depend on r, as expected. In Figure 6, it can be seen that the variance of the service-capacity distribution plays a relatively small role in determining the mean customer delay. Indeed, the more servers there are, the smaller the impact of the variance of the service capacity of a single server becomes.

5. Conclusions

In this paper, we studied a multi-server queuing model, where each customer can only receive service from a single server. This means that if there are fewer customers in the system than the number of servers, some of the available service capacity is not used, making the system not work-conserving and much harder to analyze. While obtaining exact expressions for the pgfs of the system content and the customer delay does not appear feasible using current techniques, we studied several approximations and compared the obtained estimates using numerical examples. One approximation in particular, the Two-Phase approximation, provides good approximation for both the mean customer delay and the tail probabilities of the delay, for both low and high traffic loads. This approximation works especially well when the variance of the service demands is low.