Next Article in Journal
An Enhanced Lightning Attachment Procedure Optimization Algorithm
Next Article in Special Issue
Bi-Level Multi-Objective Production Planning Problem with Multi-Choice Parameters: A Fuzzy Goal Programming Algorithm
Previous Article in Journal
Drum Water Level Control Based on Improved ADRC
Previous Article in Special Issue
The Role of Façade Materials in Blast-Resistant Buildings: An Evaluation Based on Fuzzy Delphi and Fuzzy EDAS
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:

A New Method for Markovian Adaptation of the Non-Markovian Queueing System Using the Hidden Markov Model

Faculty of Technical Sciences, University of Novi Sad, TrgDositejaObradovi’ca 6, 21 000 Novi Sad, Serbia
Department of Mobile Machinery and Railway Transport, Faculty of Transport Engineering, Vilnius Gediminas Technical University, Plytinėsg.27, LT-10105 Vilnius, Lithuania
Author to whom correspondence should be addressed.
These authors contributed equally to this paper.
Algorithms 2019, 12(7), 133;
Submission received: 27 May 2019 / Revised: 21 June 2019 / Accepted: 25 June 2019 / Published: 28 June 2019
(This article belongs to the Special Issue Algorithms for Multi-Criteria Decision-Making)


This manuscript starts with a detailed analysis of the current solution for the queueing system M/Er/1/∞. In the existing solution, Erlang’s service is caused by Poisson’s arrival process of groups, but not individual clients. The service of individual clients is still exponentially distributed, contrary to the declaration in Kendall’s notation. From the related theory of the Hidden Markov Model (HMM), for the advancement of queueing theory, the idea of “hidden Markov states” (HMS) was taken. In this paper, the basic principles of application of HMS have first been established. The abstract HMS states have a catalytic role in the standard procedure of solving the non-Markovian queueing systems. The proposed solution based on HMS exceeds the problem of accessing identical client groups in the current solution of the M/Er/r queueing system. A detailed procedure for the new solution of the queueing system M/Er/1/∞ is implemented. Additionally, a new solution to the queueing system M/N/1/∞ with a normal service time N(μ, σ) based on HMS is also implemented.

1. Introduction

Queueing theory is a subfield of probability theory within mathematics. It was founded just over 100 years ago, by the publication of works and successful practical application of the Danish mathematician, statistician, and engineer Agner Krarup Erlang (1878–1929). In his honor, the system solved by M/M/1 and the unit of communication activity bears Erlang’s name. The effective practical application of the new queueing theory has given a strong impetus for it is further development. At the same time, the tasks posed by the practice require the solutions of non-Markovian and non-stationary queueing systems with a permanent imperative according to the standard set by Erlang—a transparent analytical solution.
The M/G/1 model was solved in 1930 by the Austrian-French engineer and mathematician, Félix Pollaczek (1892–1981) [1]. Our contemporary, Sir John Frank Charles Kingman, in 1961 [2], published an approximate formula for the mean waiting time in a G/G/1 queue. Although approximate, it is considered to be a very precise formula, especially for a system operating close to saturation. Of course, the most important contributions to the development of queueing theory must include contributions by David George Kendall (1918–2007) and another contemporary Jonh Dutton Conant Little (born 1928). The solutions of non-stationary queueing systems begin with the works by Andrey Nikolaevich Kolmogorov (1903–1987) [3], which emerged almost simultaneously with the first general solutions of the non-Markov queueing systems in 1930 [1].
Nevertheless, in spite of the great effort, intensive development of the world economy and the requirements of practice have far exceeded the achievements of queueing system researchers. The emergence and design of the development of computers in the 1980s and 1990s resulted in the prognosis that each queueing system could be solved by the use of simulations. This attitude has somewhat discouraged further efforts in the analytical approach of queueing theory and was consistently described by Koenigsberg in the set and reasoned antithesis [4]. His absolutely correct assessment of the necessity of an analytical approach and positive development prognosis was confirmed by Schwartz, Selinka, and Stoletz, especially for non-stationary time-dependent queueing systems [5]. The analytical approach to solving the queueing system remains an imperative. This imperative does not exist in itself, but is encouraged by the practical application of the queueing system.
The theory of the Hidden Markov Model (HMM) was founded by Baum and Petrie in the mid-1960s [6]. It is based on the simple idea that in the observed system, the Markov processes are a priori, and that the existence of hidden Markov states can be confirmed statistically, representing the criterion for optimizing the maximum probability. HMM is classified into methods of artificial intelligence. It is characterized by an intense development [7] and capacity for effectively solving a wide range of real tasks: character recognition [8], medicine [9], helping blind people [10], infrastructure management [11], driver fatigue detection [12], etc.
Basically, HMM is related to queueing theory. It also shares the same analytical problems in the principles of the functioning of a state in a non-Markovian domain [13]. The application of the idea of hidden states from HMM in queueing systems has an initially modest, but extremely important, realized methodological idea in the birth–death process [14].
The synthesis of HMM and queueing theory leads to the idea of abstract hidden states of Markov (HMS) in the first-order transition probability distribution among the hidden states with a possible catalytic function. This idea is not new, and HMS have already been developed for some higher-order Markov models [15]. For non-Markov queueing systems, the HMS function is contrary to convolution. It is based on the process of the decomposition of exponential random variables to the sum of exponential distributions. The exact case is Erlang’s distribution, which is a well-known relation presented in the following (1):
E r ( k , λ ) = i = 1 k E ( λ )
The idea of the decomposition of Erlang’s distribution and reduction to the Markov queueing system was applied in the simplest case—the intensity of Erlang’s second-order distribution Er(2, μ), in the artificial intelligence hybrid known as Ant Colony Optimization [16].
In addition to Erlang’s distribution, we have at our disposal the relationship between an exponential distribution and normal distribution using the Central Limit Theorem [17].

2. Analysis of the Current Solution Queueing Systems M/Er/1/∞ and Ek/Er/1/∞

The recent solution of the M/Ek/r queueing system from 1968/69 [18,19] was based on the assumption that customers arrive at Poisson’s arrival process and are serviced by the Erlang distribution Er(k, µ). A well-known convolution was used to reduce the Markov queueing system (1). The analytical solution assumes the following flow diagram (Figure 1).
In this queueing system, the services of the Erlang distribution Er(k, μ) are realized through the “k” exponential E(μ) phases. The solution of the stationary state follows the set of equilibrium equations, which, along with the change pn = xn, requires the solution of the polynomial (2).
( λ + μ ) x r = λ + μ x r + 1
The roots of Equation (2) can be numerically determined very efficiently [20]. The solution of the queueing system Ek/Er/1 had a similar problem which was also successfully mathematically solved in a similar way [21,22,23,24].
However, a comparative analysis of the mean number of customers in the simplest M/M/1/∞ and M/Ek/r/∞ systems leads us to a kind of paradox. Their relationship is as follows (3):
ρ 1 ρ M / M / 1 / ρ k 1 ρ k k + 1 2   M / E k / 1 / ,   ( k 2 ) N ;   1 μ   = k μ k ;   ρ = λ μ = ρ k = k λ μ k < 1
Apparently, the current solution of M/Ek/r is “mechanically” configured to serve a group of “k” clients. If each client is served by the exponential distribution, a group of “k” clients will be served by the Erlang distribution. In practice, we have found this system in stochastic inventory management. At the same time intervals 1/λ, stocks are supplemented with the same quantity of “k” integer “customers” (Figure 2). The current solution of the M/Ek/r is not entirely adequate for the following reasons:
  • Customers do not arrive at Poisson’s arrival process—a group of “k” customers arrive at Poisson’s arrival process. For this reason, the ratio of the number of clients in the queueing systems is (3);
  • Customers are not served by Erlang’s distribution—customers are served by the exponential distribution. A group of “k” customers are served by Erlang’s distribution.
In recent solutions of the queueing system M/Ek/r, the state of the queue is redefined as the number of service stages remaining in the system [23]. Solutions of the first half of the century are still dominant [25], although these do not allow for successive transitions through the states of the system, as in the system M/M/1/∞. Simply, the queueing system M/Ek/r from Figure 1 does not have mutual transitions from neighboring states.
Paragraph (3) is not in agreement with the well-known Pollaczek–Khintchine formula. For the same mean service rate, the Erlang distribution has a lower value of variance than the exponential one. Therefore, the M/Ek/1/∞ system must have a smaller mean number of customers in the queue. The reason for this relationship is found in the projected flow diagram (Figure 1). Exponential phases of customer services through which the Erlang distribution is convoluted are partially realized, depending on the quantity λ and μ. For the interval 0 < ρ < 1, for small values, ρ → 0 solutions are satisfactory, but when ρ → 1, there are large deviations. When dynamically observed, the higher arrival rate λ, the earlier the convolution of the exponential customer services is interrupted, and we obtain a service of the Erlang order, which is an average of less than k. Therefore, a valid solution is only approximate for favorable relationships λ and μ.

3. Analytical Solution of the Queueing System M(λ)/Ek(μ)/1/∞

3.1. Elementary Case of Poison’s Birth and Erlang’s Death

The elementary case of the queueing system is based on the birth–death process. Therefore, consider the elementary system with two states X0 and X1. Let the time of birth be exponentially distributed E(λ) in this system, and the dying time be distributed according to Erlang’s distribution Er(k, μk).
Suppose that the birth–death process is ergodic and after a sufficiently long period, it achieves a stationary regime of operation with the following conditions: λ(t) = λ = const > 0, μ(t) = μ = const > 0, k ∈ N, k > 1.
This queueing system can be reduced to a Markov system by forming abstract hidden Markov states (HMS) based on the basic concept of HMM and the basic analytical description of the convolution of exponential distributions (1). The queueing system states hk−1, hk−2, …, h2, h1(k ∈ N) are abstract states and have a catalitic role. In order to achieve the convolution of “k” exponential distributions, the system needs to introduce “k − 1” abstract hidden Markov states in the process of customer services. The indexes of abstract states hi are descendent. The symbolic equivalence of the birth–death process can be reduced to convincing by Markov with abstract, hidden states (Figure 3).
The solution of this system is based on the calculation of the probability values of all states: P(X0) = p0 and P(X1) = p1, as well as the probability of the state of abstract HMS hk−1, hk−2, …, h2, h1, respectively P(hi) = ph(i). We can calculate them with ease based on the existing analytical rules of the stationary regime of the birth–death process.
The key question is, how are the probabilities of abstract HMS ph(i) classified? The existing system (Figure 2) has a “k + 1” state, and we only need two basic states: P0 and P1.
From the system of the birth–death process from two states, we know that in the first birth with intensity λ, the system goes into state X1. Analogously, the intensity of dying μk from the beginning of the convolution translates the system into X0 state, and the probability of abstract HMS is thus associated with the preceding state according to the following rule (4):
P 0 = P ( X 0 ) + P ( h k 1 ) + + P ( h 1 ) = p 0 + i = 1 k 1 p h ( 1 ) , P 1 = P ( X 1 ) = p 1
In the opposite case, when the birth time is distributed according to Erlang’s distribution Er (k, λk) and dying along the exponential E(μ), abstract HMS are formed for the purpose of convolution, but their probability will be added to the state P1.
This describes the key catalytic principle of applying abstract HMS. The principle of grouping probabilities of basic and hidden states is given in Figure 4.

3.2. The Probability of State of the Queueing System M(λ)/Ek(μ)/∞

The establishment of the elementary case of a queueing system with a finite state number X0, X1, …, Xn, with the exponentially distributed inter-arrival-time E(λ) and the service time distributed according to Erlang’s distribution Er(k, μk), k ∈ N, is shown in Figure 5.
Assume that the queueing system is ergodic and that it is in a stationary operating mode. The probabilities of the state X0, X1, …, Xn, in the stationary mode of operation, are denoted by the large letter Pi(t) = Pi = const, i ∈ [1, n]. The probabilities of the abstract states hi,j are denoted by the small letter pi,j(t) = pi,j = const, i ∈ [1, k−1], j ∈ [1, n]. For the X0 state, the balance (differential) equation of the stady-state (stationary operating mode) is (5)
X 0 : 0 = λ P 0 + μ k p k 1 , 1 p k 1 , 1 = λ μ k P 0
For the abstract HMS hk−1,1, hk−2,1, …, h2,1, h1,1 which are between the states X0 and X1, the system of balance equations of the stady-state is (6)
{ h k 1 , 1 : 0 = μ k p k 1 , 1 + μ k p k 2 , 1 p k 1 , 1 = p k 2 , 1 p k 2 , 1 = λ μ k P 0 h 2 , 1 : 0 = μ k p 2 , 1 + μ k p 1 , 1 p 2 , 1 = p 1 , 1 p 1 , 1 = λ μ k P 0 h 1 , 1 : 0 = μ k p 1 , 1 + μ k P 1 p 1 , 1 = P 1 P 1 = λ μ k P 0
For the state of X1, the balance equation of the stady-state is (7)
X 1 : 0 = λ P 0 λ P 1 μ k P 1 + μ k p k 1 , 2 p k 1 , 2 = λ P 0 + λ P 1 + μ k P 1 μ k p k 1 , 2 = ( λ μ k ) 2 P 0
For the abstract HMS hk−1,2, hk−2,2, …, h2,2, h1,2 that are located between the states X1 and X2, the system of balance equations of the stady-state is (8)
{ h k 1 , 2 : 0 = μ k p k 1 , 2 + μ k p k 2 , 2 p k 1 , 2 = p k 2 , 2 p k 2 , 2 = ( λ μ k ) 2 P 0 h 2 , 2 : 0 = μ k p 2 , 2 + μ k p 1 , 2 p 2 , 2 = p 1 , 2 p 1 , 2 = ( λ μ k ) 2 P 0 h 1 , 2 : 0 = μ k p 1 , 2 + μ k P 2 p 1 , 2 = P 2 P 2 = ( λ μ k ) 2 P 0
For the state of X2, the balance equation of the stady-state is (9)
X 2 : 0 = λ P 1 λ P 2 μ k P 2 + μ k p k 2 , 2 p k 1 , 2 = λ P 1 + λ P 2 + μ k P 2 μ k p k 1 , 3 = ( λ μ k ) 3 P 0
Prescriptively, in accordance with the standard solution procedure, we can obtain solutions for all remaining system states (10):
X n 1 : 0 = λ P n 2 λ P n 1 μ k P n 1 + μ k p n , 2 p k 1 , n = ( λ μ k ) n P 0 { h k 1 , n : 0 = μ k p k 1 , n + μ k p k 2 , n p k 1 , n = p k 2 , n p k 2 , n = ( λ μ k ) n P 0 h 2 , n : 0 = μ k p 2 , n + μ k p 1 , n p 2 , n = p 1 , n p 1 , n = ( λ μ k ) n P 0 h 1 , n : 0 = μ k p 1 , n + μ k P n p 1 , n = P n P n = ( λ μ k ) n P 0
The probability value P0 can be obtained from the normative Equation (11):
P 0 + P 0 j = 1 k 1 λ μ k s 0 = P 0 j = 1 k 1 ( λ μ k ) 1 + P 0 λ μ k P 1 + P 0 j = 1 k 1 ( λ μ k ) 2 s 1 = P 0 j = 1 k ( λ μ k ) 2 + + P 0 ( λ μ k ) 2 P n + P 0 j = 1 k 1 ( λ μ k ) n s n = P 0 j = 1 k ( λ μ k ) n + = 1
By solving the sum in (11), we can obtain (12)
P 0 + ( k 1 ) P 0 λ μ k s 0 + P 0 λ μ k + ( k 1 ) P 0 ( λ μ k ) 2 s 1 + + P 0 ( λ μ k ) n + ( k 1 ) P 0 ( λ μ k ) n + 1 s n + = 1
The following geometric order is obtained (13):
P 0 + k P 0 λ μ k + k P 0 ( λ μ k ) 2 + k P 0 ( λ μ k ) 3 = 1
where the condition λ < μk gives the equation for the P0 (14) calculation.
We introduce the tag λ μ k = ρ k :
P 0 + k P 0 j = 1 ( λ μ k ) j = P 0 + k P 0 ( 1 1 ρ k 1 ) = P 0 ( 1 ρ k + k ρ k 1 ρ k ) = 1
The probability value of P0 is equal (15) to
P 0 k P 0 + P 0 k 1 ρ k = 1 P 0 = 1 ρ k 1 ρ k + k ρ k
The probabilities hk−1,j, hk−2,j, …, h2,j, h1,j (j ∈ N) are abstract probabilities. The analytical process takes them to the catalytic at each transition from the state Xj to the state Xj−1, for j ∈ [1, ∞). Therefore, the probabilities of pi,j states hi,j are abstractly transmitted only for the realization of the service by Erlang’s distribution. Now, the new code of the sum of probabilities of real and abstract states Si (Figure 6) can be marked:
This is the demanded system with the exponentially distributed inter-arrival-time E(λ) and the service time distributed according to Erlang’s distribution Er(k, μk). We temporarily observe the label “k, μ” with the note that the service rate is distributed by Erlang’s “k” order with the intensity “μk” (Figure 7):
The probabilities of the state P(Si) = si can be given by (16)
s i = 1 ρ k 1 ρ k + k ρ k ( ρ k i + ( k 1 ) ρ k i + 1 ) = ρ k i 1 ρ k 1 ρ k + k ρ k ( 1 + k ρ k ρ k ) = ρ k i ( 1 ρ k )

3.3. Comparison of the System M(λ)/M(μe)/1/∞ and M(λ)/Ekk)/1/∞

Now assume that the two queueing systems M(λ)/M(μ)/1/∞ and M(λ)/Ekk)/1 have an identical arrival rate and equal mean service time. Then, this relationship can be expanded with “λ” (17), as follows:
1 μ = k μ k λ μ = λ k μ k
By adopting the adopted columns for ρ and ρk, it follows that (18)
ρ = k ρ k ρ k = ρ k
From the relationship (18), a new relationship of the variance for the same mean time of service in the queueing systems M(λ)/M(μ)/1/∞ and M(λ)/Ek(μk)/1/∞ (19) can be obtained:
1 μ = k μ k 1 μ 2 > k μ k 2
In accordance with the Pollaczek–Khinchine formula, the queueing system with Erlang’s distribution of service times must have a lower mean number of customers in the system and queue. Contrary to paragraph (3) for the existing solution, we prove this statement for the new solution of the queueing system M(λ)/Ek(μk)/1/∞.
First, we recall the formula for calculating the probability state of the system M(λ)/M(μ)/1/∞, ρ = λ/μ (20):
p i = ρ i ( 1 ρ )
We note that the probability calculus formulas (20) are analogous to (16). On the basis of this analogy, from the familiar formulas of the system M(λ)/M(μe)/1/∞ with smoothness, we proceed to the formula of the system M(λ)/Ek(μk)/1/∞. Now, the mean number of customers in the system M(λ)/Ek(μk)/1/∞ and in the queue equals (21)
L s = i = 0 i ρ k i ( 1 ρ k ) = ρ k 1 ρ k , L q = ρ k 2 1 ρ k ,
By substituting (19) with (21), we obtain the basis for comparing the mean number of customers in the system and queue, i.e., for k > 1 and ρ < 1, the mean number of customers in the system M(λ)/Ek(μk)/1/∞ is always lower than the mean number of customers in the system M(λ)/M(μe)/1/∞. The same applies to the mean number of customers in the queue (22):
L s = ρ k   1 ρ k = ρ k ρ M ( λ ) / E k ( μ k ) / 1 / < ρ 1 ρ M ( λ ) / M ( μ e ) / 1 / ; L q = ( ρ k ) 2   1 ρ k = ρ 2 k ( k ρ ) < ρ 2 1 ρ
Unlike the relation in the previous queueing system solution M(λ)/Ek(μk)/1/∞ (3), the solution with the abstract HMS (21) completely meets the logic of the Pollaczek–Khinchine formula.
Using the Pollaczek–Khinchine formula for the queueing system M(λ)/Ek(μk)/1/∞, we can verify the validity of the patterns (21) and the relation (22). Mean service time and variance are distributed according to Erlang’s distribution with known parameters of mathematical expectation and standard deviation k/μk and k/μk2, respectively. The ratio ρ is “k” times greater than ρk (18). If in the Pollaczek–Khinchine formula we reduce the intensity and the variance, we obtain (23), which is identical to the form (22) obtained:
L s = ρ k + ( ρ k ) 2 + λ 2 V a r ( s ) k 2 ( 1 ρ k ) = ρ k + ( ρ k ) 2 + λ 2 k ( k μ 2 ) 2 ( 1 ρ k ) = ρ k + ( ρ k ) 2 + ρ 2 k 2 2 ( 1 ρ k ) = ρ k ρ
By applying the Little formula, it follows that, for the same mean service time (18) and the relationships obtained (22), on average, customers spend more time on the M(λ)/M(μe)/1/∞ system. The amount of time φ is the same (24), as follows:
φ = λ ρ k ρ   λ ρ 1 ρ = 1 ρ k ρ < 1
The same logic is applied for the mean wait in the queue. The explanation of this relationship is based on the relationship between the entropy of the exponential and Erlang’s distribution. Among all continuous probability distributions with support [0, ∞), the exponential distribution has the largest entropy. Therefore, the Markov queueing systems M/M/n/∞ have the greatest disadvantage in terms of access and service, and the greatest burden is thus on the service channel. In this case, in general, entropy is inversely proportional to the degree of system organization.

4. Queueing System M(λ)/N(ω, σ)/1/∞

4.1. An Approximate Solution

For Erlang’s distribution Ek(k, μk) and a normal distribution N(ω, σ) with a probability density function (25),
f k ( t ) = μ k k t k 1 e μ k t ( k 1 ) ! , f ( t ) = 1 σ 2 π e ( t ω ) 2 2 σ 2
With a sufficiently large value of the Erlang distribution line, based on the Central Limit Theorem (CLT) of Lindeberg–Levy, we can establish the following relations (26):
ω = k μ k μ k = k ω , σ = k μ k μ k = k σ
For negative values of k, μk, ω, and σ, from the expression (26), we can obtain a quadratic equation. The solution of this square equation by “k ≥ 1” is equal to (27) the following:
μ k 2 = k 2 ω 2 = k σ 2 k 2 ω 2 k σ 2 = 0 , k 1 , 2 = 1 σ 2 ± 1 σ 4 2 ω 2   ,   k 1 = 0 k 2 = ω 2 σ 2
We can only find the solution k2. In the case of distributing the service time by the normal distribution N(ω, σ), which in the application of the queueing system does not have negative values f(t) ≥ 0, t ∈ (0, ∞), due to the known rule (ω − 3σ) > 0 (with probability p = 0.9996), or the minimum value of the parameter “k”, we can expect 9. Thanks to the relation (26), the non-Markov queueing system M(λ)/N(ω, σ)/1/∞ can be reduced to the Markov queueing system with Erlang’s distribution of service times M(λ)/E(ω2/σ2,ω/σ2)/1/∞.
However, this form of expressing the normal distribution over Erlang’s is approximate, because the quantity (28) is a rational positive number (ω2/σ2) ∈ R+, i.e., a parameter of the Erlang distribution “k” will often not be the positive integer, k N. For practical use, the substitution (28) is approximative:
ω 2 σ 2 N ,   E r ( ω 2 σ 2 k , ω σ 2 μ n ) N ( ω , σ ) ,  

4.2. Calibration of the Solution for a Normal Service Time

For each value of the non-negative rational number k ∈ R+ of the Erlang distribution curve obtained on the basis of (28), we can determine the value of Δ using the integer finite. We recall that in the case of the normal distribution of the service time N(ω, σ), the value of the parameter k ≥ 9 (29):
k = ω 2 σ 2 9 , k i n t ( k ) = Δ , 0 < Δ < 1
If we remember the definitions of the rational number, i.e., a number that can always be represented by a split of two integers, the rational value of the Erlang distribution parameters k ∈ R+ can be represented by a fraction formed by two selected natural numbers (a, b) ∈ N, which are greater than zero. Let k′ = int(k) and k″ = int(k) + 1. It follows that (30)
k = a k + b k a + b
After determining the natural numbers (a, b), it is necessary to solve two systems:
  • The first Erlang system of order k′, i.e., queueing system M(λ)/E(k′, ω2/σ2)/1/∞. Let us denote its probability as si′;
  • The second Erlang system of order k″, i.e., queueing system M(λ)/E(k″, ω2/σ2)/1/∞. Let us identify its probabilities with si″.
If ni denotes the probability of the state of the system M(λ)/N(ω, σ)/1/∞, after calculating the numbers (a, b) ∈ N, the probabilities of the state ni (31) can be calculated as follows:
n i = a s i + b s i a + b
It is clear that in the calibration, the elementary-normative condition (32) is fulfilled:
i = 0 n i = i = 0 a s i + b s i a + b = i = 0 ( a s i + b s i ) a + b = a i = 0 s i + b i = 0 s i a + b = a + b a + b = 1
This means that with a high precision, we can calculate the probabilities of the system M(λ)/N(ω,σ)/1/∞.
Calculations of the normal distribution can be substituted for sufficiently large k values. The formula for calculating the mean number of customers in the system and the queue (20) can also be applied k ∈ R+ for the purpose of the compulsory adjustment of μn = ω/σ according to formulas (26) and (27).

5. Conclusions

The application of abstract HMS with a catalytic function opens up many new possibilities in the analytical approach to the solution of the non-Markov queueing system. The process of reducing the non-Markov queueing system to Markov queueing systems is general. It is based on known convolutions and relations between exponential distributions and related distributions: Hyperexponential, Weibull, Rayleigh, Laplace, Gamma, etc. The concept of application is elastic, and it is applicable to heterogeneous queueing systems and various disciplines of services, etc.
The solutions of the queueing system M(λ)/Ek(μk)/1/∞ and M(λ)/N(ω, σ)/1/∞ were intended to demonstrate the effectiveness of the new proposed HMS-based method. The correctness of the new method is confirmed by the Pollaczek–Khinchine formula.
Based on the established principles of applying the HMS and the procedure, queueing system solutions with n channels of services and a finite number of positions in the queue are easy to obtain: M(λ)/Ek(μk)/n/m and M(λ)/N(ω, σ)/n/m. The extension of the HMS application principle provides solutions with queueing systems Ei/Ej/m/n (ij) i, j ∈ N, and by applying calibration, all queueing systems with a normally distributed time are analytically available.
The reasoned idea and principles of applying HMS can easily be applied to time-dependent queueing systems.

Author Contributions

Conceptualization, O.P.; Formal analysis, P.E.; Methodology, I.T.; Project administration, I.T.; Supervision, O.P.; Validation, Ž.J. and G.S.; Writing—original draft, I.T., Ž.J., G.S. and P.E.


This research received no external funding.

Conflicts of Interest

The authors declare no conflict of interest.


  1. Pollaczek, F. Uebereine Aufgabe der Wahrscheinlichkeits theory. Math. Z. 1930, 32, 64–100. [Google Scholar] [CrossRef]
  2. Kingman, J.F.C. The single server queue in heavy traffic. Math. Proc. Camb. Philos. Soc. 1961, 57, 902–904. [Google Scholar] [CrossRef]
  3. Kolmogorov, A. Sur le problèmed’attente. MatematicheskiiSbornik 1931, 38, 101–106. (In French) [Google Scholar]
  4. Koenigsberg, E. Is queueing theory dead? Omega 1991, 19, 69–78. [Google Scholar] [CrossRef]
  5. Schwarz, J.A.; Selinka, G.; Stolletz, R. Performance analysis of time-dependent queueing systems: Survey and classification. Omega 2016, 63, 170–189. [Google Scholar] [CrossRef] [Green Version]
  6. Baum, L.; Petrie, T. Statistical Inference for Probabilistic Functions of Finite State Markov Chains. Ann. Math. Stat. 1966, 37, 1554–1563. [Google Scholar] [CrossRef]
  7. Li, Y. Hidden Markov models with states depending on observations. Pattern Recognit. Lett. 2005, 26, 977–984. [Google Scholar] [CrossRef]
  8. Chen, M.Y.; Kundu, A.; Zhou, J. Off-line handwritten word recognition using a hidden Markov model type stochastic network. IEEE Trans. Pattern Recognit. Mach. Intell. 1994, 16, 481–496. [Google Scholar] [CrossRef]
  9. Uğuz, H.; Kodaz, H. Classification of internal carotid artery Doppler signals using fuzzy discrete hidden Markov model. Expert Syst. Appl. 2011, 38, 7407–7414. [Google Scholar] [CrossRef]
  10. Gu, F.; Zhang, H.; Zhu, D. Blind separation of non-stationary sources using continuous density hidden Markov models. Digit. Signal Process. 2013, 23, 1549–1564. [Google Scholar] [CrossRef]
  11. Kobayashi, K.; Kaito, K.; Lethanh, N. A statistical deterioration forecasting method using hidden Markov model for infrastructure management. Transp. Res. Part B Methodol. 2012, 46, 544–561. [Google Scholar] [CrossRef] [Green Version]
  12. Rongrong, F.; Hong, W.; Wenbo, Z. Dynamic driver fatigue detection using hidden Markov model in real driving condition. Expert Syst. Appl. 2016, 63, 397–411. [Google Scholar]
  13. Holzmann, H.; Schwaiger, F. Testing for the number of states in hidden Markov models. Comput. Stat. Data Anal. 2016, 100, 318–330. [Google Scholar] [CrossRef]
  14. Fadiloglu, M.M.; Yeralanm, S. Models of production lines as quasi-birth-death processes. Math. Comput. Model. 2002, 35, 913–930. [Google Scholar] [CrossRef] [Green Version]
  15. Ching, W.K.; Huang, X.; Ng, M.K.; Siu, T.K. Hidden Markov Chains. In Markov Chains. International Series in Operations Research & Management Science; Mamon, R.S., Elliott, R.J., Eds.; Springer: Boston, MA, USA, 2013; Volume 189, pp. 201–230. [Google Scholar]
  16. Tanackov, I.; Stojić, G.; Tepić, J.; Kostelac, M.; Sinani, F.; Sremac, S. Golden Ratio (Sectiona Aurea) in Markovian Ants AI Hybrid. Adaptive and Intelligent Systems, ICAIS 2011, Lecture Notes in Computer Science; Springer: Berlin, Germany, 2011; Volume 6943, pp. 356–367. [Google Scholar]
  17. Whitt, W. A broad view of queueing theory through one issue. Queueing Syst. 2018, 89, 3–14. [Google Scholar] [CrossRef]
  18. Heffer, J.C. Steady-state solution of the M/Ek/c (0, FIFO) queueing system. INFOR J. Can. Oper. Res. Soc. 1969, 17, 16–30. [Google Scholar]
  19. Mayhugh, J.O.; Mc Cormick, R.E. Steady state solution of the queue M/Ek/r. Manag. Sci. 1968, 14, 692–712. [Google Scholar] [CrossRef]
  20. Poyntz, C.D.; Jackson, R.R.P. The steady-state solution for the queueing process Ek/Em/r. Oper. Res. Q. 1973, 24, 615–625. [Google Scholar] [CrossRef]
  21. Adan, I.; Zhao, Y. Analyzing GI/Er/1 queues. Oper. Res. Lett. 1996, 19, 183–190. [Google Scholar] [CrossRef]
  22. Adan, I.J.B.F.; Van de Waarsenburg, W.A.; Wessels, J. Analyzing Ek/E/c queues. Eur. J. Oper. Res. 1996, 92, 112–124. [Google Scholar] [CrossRef]
  23. Martínez, J.M.V.; Vallejos, R.A.C.; Barría, M.M. On the limiting probabilities of the M/Er/1 queueing system. Stat. Probab. Lett. 2014, 88, 56–61. [Google Scholar] [CrossRef]
  24. Wang, K.H.; Kuo, M.Y. Profit analysis of the M/Er/1 machine repair problem with a non-reliable service station. Comput. Ind. Eng. 1997, 32, 587–594. [Google Scholar] [CrossRef]
  25. Adan, I.; Resing, J. Queueing Systems; Einhoven University of Technology: Eindhoven, The Netherlands, 2015. [Google Scholar]
Figure 1. Flow diagram of the M/Er/r current solution.
Figure 1. Flow diagram of the M/Er/r current solution.
Algorithms 12 00133 g001
Figure 2. Inventory management queueing system for recent solutions M/Ek/r.
Figure 2. Inventory management queueing system for recent solutions M/Ek/r.
Algorithms 12 00133 g002
Figure 3. Process of Markovization of the elementary birth–death process based on the abstract hidden Markov states (HMS) and the convolution of the exponential distribution.
Figure 3. Process of Markovization of the elementary birth–death process based on the abstract hidden Markov states (HMS) and the convolution of the exponential distribution.
Algorithms 12 00133 g003
Figure 4. The principle of applying abstract hidden Markov states (HMS) in the queueing system.
Figure 4. The principle of applying abstract hidden Markov states (HMS) in the queueing system.
Algorithms 12 00133 g004
Figure 5. Queueing system M(λ)/Ek(μ)/∞.
Figure 5. Queueing system M(λ)/Ek(μ)/∞.
Algorithms 12 00133 g005
Figure 6. Sums of probability of real and abstract states.
Figure 6. Sums of probability of real and abstract states.
Algorithms 12 00133 g006
Figure 7. Markov queueing system with an Erlang distribution of services.
Figure 7. Markov queueing system with an Erlang distribution of services.
Algorithms 12 00133 g007

Share and Cite

MDPI and ACS Style

Tanackov, I.; Prentkovskis, O.; Jevtić, Ž.; Stojić, G.; Ercegovac, P. A New Method for Markovian Adaptation of the Non-Markovian Queueing System Using the Hidden Markov Model. Algorithms 2019, 12, 133.

AMA Style

Tanackov I, Prentkovskis O, Jevtić Ž, Stojić G, Ercegovac P. A New Method for Markovian Adaptation of the Non-Markovian Queueing System Using the Hidden Markov Model. Algorithms. 2019; 12(7):133.

Chicago/Turabian Style

Tanackov, Ilija, Olegas Prentkovskis, Žarko Jevtić, Gordan Stojić, and Pamela Ercegovac. 2019. "A New Method for Markovian Adaptation of the Non-Markovian Queueing System Using the Hidden Markov Model" Algorithms 12, no. 7: 133.

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop