A Bayesian Approach Based on Bayes Minimum Risk Decision for Reliability Assessment of Web Service Composition

Abstract

Web service composition is the process of combining and reusing existing web services to create new business processes to satisfy specific user requirements. Reliability plays an important role in ensuring the quality of web service composition. However, owing to the flexibility and complexity of such architecture, sufficient estimation of reliability is difficult. In this paper, the authors propose a method to estimate the reliability of web service compositions based on Bayes reliability assessment by considering it to be a decision-making problem. This improves the testing efficiency and accuracy of such methods. To this end, the authors focus on fully utilizing prior information of web services to increase the accuracy of prior distributions, and construct a Markov model in terms of the reliabilities of the web composition and each web service to integrate the limited test data. The authors further propose a method of minimum risk (MMR) to calculate the initial values of hyperparameters satisfying the constraint of minimal risk of the wrong decision. Experiments demonstrate that the proposed method is capable of efficiently utilizing prior module-level failure information, comparing with the Bayesian Monte Carlo method (BMCM) and expert scoring method (ESM), when the number of failures increased from 0 to 5, reducing the required number of test cases from 19.8% to 28.9% and 6.1% to 14.1% separately, improving the reliability assessment of web service compositions, and reducing the expenses incurred by system-level reliability testing and demonstration.

1. Introduction

Web services are service-oriented architecture technologies that are executed using standard web protocols to ensure their operation varying platforms. With the continuous maturity of service-oriented technology and the promotion of service-oriented architecture (SOA) [1], multiple atomic web services have been implemented to achieve more powerful service compositions while satisfying relevant rules and addressing the demands of businesses. To maintain the quality of web service composition and utilize testing resources reasonably, assessment methods for web service compositions are required to exhibit high error detection capabilities, low cost consumption, and broad applicability [2].

In software reliability testing, the purpose of reliability growth testing is to identify and incorporate corrective actions that improve the reliability of the system. Following the completion of growth testing, a reliability demonstration is performed to verify a specific reliability requirement. Both broad and deep studies have been conducted on software reliability assessment to expose software defects and improve software reliability at early stages. When the number of test cases is very high or the test data are limited, software input is selected using statistical methods to promote rapid reliability growth of the software to ensure high reliability software demonstration testing [3]. In this context, Bayes reliability inference has been extensively applied to reliability estimation, and a wealth of experts and prior knowledge about system/subsystem performances are associated with it [1]. The Bayesian approach has been adopted because it combines subjective judgment or prior experience with data obtained from test samples. In other words, it uses a combination of existing experience and new test data to assess reliability metrics. In this approach, a posterior distribution is derived based on a prior distribution and a likelihood function, and reliability testing is conducted based on this derived distribution. Following the addition of new sample data, this posterior distribution is employed as a prior distribution during the process of constructing a new posterior distribution. Under the Bayesian framework, historical data are usually used for the analysis of prior distributions with certain parameters [4]. However, the origin of the required prior distributions remains a major concern in Bayesian reliability analysis [5]. The appropriate choice of priors is critical to capture useful information present in historical data. The fundamental advantage of adopting the Bayesian approach is that it augments the quality of data, thereby reducing the uncertainty involved in decision making.

Thus, in this case, the prior information is derived from historic data or experts’ suggestions. The two types of information regarding unknown parameters can be expressed in the form of statistical distributions, from which further statistical inferences can be made. Then, the posterior distribution is evaluated by integrating the prior distribution and the likelihood function constructed based on the sampling distribution of the data. Bayesian estimation with prior distribution has been extensively studied in the literature, for the reliability assessment, some studies are shown in

Table 1. Different methods on the reliability assessment.

Authors	Year	Approaches
Anderson-Cook [6]	2009	By considering the totality of data simultaneously, instead of performing analyses on each data type separately, for decision-making; however, it does not apply the service composition.
Csenki, A [7]	2009	The authors studied an averaging method that combines the opinions of different experts. However, it does not adequately explain the deviations of different opinions and require a more complex model such as the Bayesian model
Moala F A, Rodrigues J, Tomazella V L D [8]	2009	The authors compare the posterior densities of the reliability function by using several examples.
M. Burgman, M. McBride, R. Ashton [9]	2011	The study proposed a method of multiple prior information integration is to average all multiple priors. However, the deviations of the different opinions are not properly quantified in the averaging approach.
A. OHgan, C. Buck, A. Daneshkhah [10]	2006	Linear and geometric pooling methods allow unequal weights for prior information and focus on multiple priors; however, it over depends the experts’ judgement.
K. McConway [11]	1978	The study considered system configuration and structure; the system prior can be derived from subsystem priors.
Li Z S, Guo J, Xiao N C, et al. [12]	2017	The study proposed an integration-based method to estimate and pool the weighted priors; it utilizes Bayes’ theorem to integrate heterogeneous priors.
Pham, N. H. Tran, S. Ren, W. Saad and C. S. Hong [13]	2020	A sampling-based Markov approximation (MA) approach is proposed to solve the combinatorial NP-hard problem, to overcome this issue of requires a long convergence time.
Di Mauro, M. Longo, F. Postiglione, and M. Tambasco [14]	2017	For a chain of network nodes in Service Function Chains, a double-layer model is adopted, where Reliability Block Diagram describes the high-level dependencies among the architecture components, availability analysis is carried out to characterize the minimal configuration of the overall system.
Bian, X. Huang, Z. Shao, X. Gao and Y. Yang [15]	2015	The authors proposed the DISCCA algorithm that guides the service nodes towards the Nash Equilibrium with short latency and low congestion, through decision making by individual users with local information to improve the reliability of system.
M. Di Mauro, M. Longo, and F. Postiglione [16]	2018	For Service Function Chaining, propose a Universal Generating Function (UGF) approach, which minimizes deployment cost while respecting a given availability requirement.

. The integration of multiple prior conditions, especially when these prior conditions are derived from various unique information sources, is challenging in Bayesian inference. The challenge is to integrate these distributions in a reasonable and effective manner and obtain a single probability distribution by combining all prior information and knowledge. Anderson-Cook [6] highlights the problem that “If the authors have multiple small data sets that are each individually insufficient to answer the question of interest, then combining them and incorporating engineering or scientific understanding of the process should allow us to extract more from that collection of data.” However, the models need to be carefully considered and evaluated to ensure that they accurately reflect the data and the underlying physical processes. The models have to be simple enough that they can be distinguished by the data, but at the same time complex enough to capture the physical processes. Jacobs et al. [7] studied an averaging method that combines the opinions of different experts. However, it does not adequately explain the deviations of different opinions and require a more complex model such as Bayesian model. In [8], the Bayesian approach is introduced to reliability using several examples, but not suitable for the web service composition. Considering both system and subsystem level data, other methods were proposed, such as the average multiple priors [9], pooling method [10], and integrating the derived prior through the system structure [11]. In [12] an integration-based method is proposed to estimate and pool the weighted priors; it utilizes Bayes’ theorem to integrate heterogeneous priors. In addition, in the service composition concept, where the nodes are traversed in a given order to provide a service, some related techniques have been applied to solve the raised reliability issues, (e.g., Markov chain [13], Stochastic Reward Networks [14], Game theory [15], and Universal Generating Functions [16]). In the case of a complex system comprising a large number of subsystems and components in the field of reliability engineering, prior knowledge of system reliability can provide information about the subsystem and system levels. In the case of web service composition, system-level module reliability is dependent on subsystem module reliability performance. Therefore, utilizing the full range of existing reliability prior information and testing data at subsystem levels and integrating such multilevel prior information during testing is a challenging task.

The calculation of hyperparameters in priors is another aspect that needs to be addressed. If there are no reasonable grounds to specify the hyperparameters of the priors, the advantage of Bayesian analysis can be compromised due to the introduction of additional uncertainty. Unfortunately, the assumption of arbitrary values for the hyperparameters of Bayesian priors is not uncommon [17]. Other existing studies on hyperparameters, such as the invariance principle [18], maximum entropy [19], and Jeffrey principle [20], concern pure heuristic methods, which require large amounts of data and are difficult to confirm, thereby making them unsuitable for practical application [21]. The evaluation should be based on the complete range of prior information, and ignoring this issue may result in costly and inaccurate reliability testing.

In this paper, the authors propose a reliability assessment method for web service compositions based on Bayes theory by fully utilizing prior information of web services. The authors propose the method considering two stages of web service compositions—reliability growth stage and reliability demonstration stage—building the reliability test frame for web service composition. Based on the reliabilities of various web services, the authors constructed a Markov model to calculate the reliability of the target web service composition.

The model both reflects the different in reliability growth and shows the feature of web service composition. To improve the efficiency of the estimation, depending on the failure information of web services to estimate the prior distribution of web service composition, the authors focused on solving the prior distribution and propose a method of minimum risk (MMR) to calculate hyperparameters and reduce the sample size adequately to meet a product’s reliability specifications. Experiments demonstrate that the proposed method is capable of completely utilizing prior failure information during web service testing, decreasing the number of test cases, and improving testing efficiency.

The remainder of this paper is organized as follows. The reliability assessment model for web service compositions based on the Bayes method is presented in Section 2.1. The assessment method for web service compositions and the hyperparameter solving method based on MMR are discussed in Section 2.2. The experiments and their results are discussed in Section 3. Finally, the paper is concluded in Section 4.

2. Materials and Methods

2.1. Web Service Composition Reliability Model

Web service composition is the process of combining different services into a single service to perform more complex functions. Owing to the lack of prior system-level information and experimental data for web service compositions, the system reliability is required to be assessed based solely on subsystem-level information. In this section, an assessment method based on a system structure with subsystem-level priors and data is introduced. The authors utilized the existing reliability prior information and testing data at sub-web services level, and integrated them to evaluate the web service composition. The authors defined an abstract web service to be a set of web services equipped with the same function, and specific web services were selected from the abstract web service to integrate the composition. Figure 1 illustrates that the web service composition includes abstract web service candidates, $W_i$, where $W_i=\left\{w_{11},w_{12},\dots ,w_{1n}\right\}$ and each $w$ denotes a specific web service, and that optimal web services are selected from a set of functionally equivalent web service candidates.

The web service composition integrates specific web services following a certain logic. Assuming that all alternative web service candidates are available, the communication links for user-invoked web services depend on quality of service (QoS) properties including response-time, location, and latency, and using this, the invoked web service sets: $\left\{w_{11},w_{22},w_{32},w_{43}\right\}$, illustrated in Figure 2, can be obtained.

To record the invocation of web services, the authors constructed a real-time data processing system, as depicted in Figure 3. The system includes Apache Kafka, Spark Streaming and database (DB), and web services deployed on Nginx. The invocation information of the web services was monitored and collected to calculate the associated transition probabilities. Among the constituent systems, Apache Spark exhibited high performance for both batch and streaming data, and Spark Streaming is an extension of the core Spark API that enables scalable, high-throughput, fault-tolerant stream processing of live data streams. The purpose of Kafka is to accumulate and, subsequently, re-compute the data. The authors utilized Spark Streaming to obtain data from the log of web services, which were then transmitted to the computation engine. Spark streaming was used as the computation engine, which directly programs based on SQL, to count the number of invocations and the number of errors corresponding to each web service. Considering the high dimensionality of queries, the results were stored on the OLAP database.

The authors used the aforementioned real-time data processing system to record the telemetry data regarding the number of invocations and the number of errors corresponding to each web service. In this paper, the authors considered the overtime of response as web service failure and used the proportion of invocation failures to calculate the reliability of web services. Let us assume that corresponding to a duration $t=t_n-t_k$, the numbers of invocations and failures of the web service $w$ is $C_n$ and $n$, respectively, at time $t_n,\mathrm{and}$ $C_k,$ and $k$, respectively, at time $t_k.$ Therefore, the probability of failure of the web service $w$ corresponding to the duration $t$ is $F\left(w\right)=\frac{n-k}{C_n-C_k}$, and its reliability is $r\left(w\right)=1-F\left(w\right)$.

In service-oriented architecture (SOA), to estimate the characteristics of dynamic and self-adaptability of web service composition, the authors used a Markov model to construct the evaluation model based on the state of software and state transition to describe the activity of software. Web service composition can be considered to be a task flow from an initial state to a final state. Corresponding to each task node, the specific web service is selected to be its state. If the currently selected web service corresponding to the task node is successfully invoked, the process continues to the next one, or it selects another task node, until it is successfully invoked. In this paper, the selection of a web service denotes the active state, and the currently selected web service depends on the current state and not on previous behavior. This successfully reflects the Markov behavior—the user invokes each subsequent web service based on the corresponding transition probability.

Figure 4 depicts the Markov model of the web service composition illustrated in Figure 2. Assuming the failure state to be $F$ in which case each web service probably fails, the web service with failure will enter in state $F$. In addition, $S$ denotes the final state and the specific web service $w_{ij}$ denotes a basic logical unit. Therefore, a series of web service flows can be established according to the composition.

Based on the information recorded by the data processing system, the transition probability $p_{ij}$ of the transition from the web service $i$ to the web service $j$ can be calculated, which is expressed by the values over the arcs in Figure 4. The collection of $p_{ij}\in \left[0,1\right]$ describes the state transition relations between web services. The transition probability is given by

$$p_{ij}={\displaystyle \sum }_{k=1}^s{p}_{sk}\times [\frac{\left|interact\left(c_i,c_j\right)\right|}{{\left|interact\left(c_i,c_j\right)\right|}_{i=1,2,\dots ,N}}{]}_{ci,cj\in Sk}$$

where $S$ denotes the number of web service composition flows, $p_{S_k}$ denotes the transition probability of the web service flow $S_k$ in the web service composition, $N$ denotes the number of web services, and $interact\left(c_i,c_j\right)$ denotes the number of interactions of web service $i$ with web service $j$ in the web service composition flow, $S_k$.

Therefore, in the case of a web service competition with $n$ web services, the transition matrix of web service can be expressed as follows.

$$p=\left[\begin{array}{ccc}{p}_{11}& \cdots & p_{1n}\\ ⋮& \ddots & ⋮\\ p_{m1}& \cdots & p_{mn}\end{array}\right]$$

(1)

Let us assume that the web service composition includes $n$ web services and that web service $i$ denotes the initial state. During successful operation of the web service composition, the web service $i$ is transformed into its final state $S$ with probability $p_{iS}$. The transition probability is given by the following equation:

$$p_{is}+{\displaystyle \sum }_{j=1}^n{p}_{ij}=1$$

(2)

Equation (2) is derived from the web service composition in the absence of any failures. However, in practice, failure of web service compositions is common. Let $r_i$ denote the reliability of the web service. Thus, when the web service $i$ is invoked, its probability of failure is $1-r_i$. In the Markov model of web service composition with $n+2$ states, the transition matrix in the existence of failures is given by

$$\{\begin{array}{ccc}{q}_{ij}& =& r_i\ast p_{ij},i=1,2,\dots ,n;j=1,2,\dots ,n\hfill \\ q_{iF}& =& 1-r_i,i=1,2,\dots ,n\hfill \\ q_{SS}& =& q_{FF}=1\hfill \\ q_{ij}& =& 0,others\hfill \end{array}$$

(3)

The transition matrix of a web service composition with $n$ web services can be expressed as follows.

$$Q={\left[\begin{array}{cc}\widehat{P}& C\\ 0& I\end{array}\right]}_{\left(n+2\right)\times \left(n+2\right)}$$

(4)

where $\widehat{P}$ is an $n\times n$ matrix, ${\left[q_{ij}\right]}_{n\times n}$, denoting the single-step control transition probability between n web services, and $C$ is an $n\times 2$ matrix denoting the control transition probability between the absorption states, $S$ and $F$. In the web service composition depicted in Figure 4, the control transition matrix is given by

$$\begin{gathered} Q=\left[\begin{array}{cccccc}0& p_{12}{r}_1& p_{13}{r}_1& 0& p_{1s}{r}_1& 1-r_1\\ 0& 0& 0& r_2& 0& 1-r_2\\ 0& 0& 0& r_3& 0& 1-r_3\\ p_{41}{r}_4& p_{42}{r}_4& p_{43}{r}_4& p_{44}{r}_4& 0& 1-r_4\\ 0& 0& 0& 0& 1& 0\\ 0& 0& 0& 0& 0& 1\end{array}\right]\begin{array}{c}1\\ 2\\ 3\\ 4\\ S\\ F\end{array} \\ 1 2 3 4 S F \end{gathered}$$

(5)

This yields the following equation expressing the relationship between the reliability, $R_s$, of the web service composition and the reliability, ${\mathrm{p}}_{\mathrm{is}}$, of each web service.

$$R_s={\displaystyle \sum }_{i=1}^n{\left(I_n-\widehat{Q}\right)}_{1i}^{-1}{r}_i{p}_{is}$$

(6)

where $I_n$ denotes the identity matrix of order n, and $\widehat{Q}$ denotes the no-absorption transition probability matrix of order n. Using this equation, the reliability information of component levels can be fully utilized to calculate the reliability of the entire web service composition.

2.2. Reliability Assessment Method Based on Bayes Minimum Risk Decision

2.2.1. Reliability Assessment Scenario

In this paper, the authors performed reliability assessment based on Bayesian reliability assessment [22], by focusing on the problem of accurately and objectively verifying software reliability requirements. Bayesian reliability assessment is a statistical theory based on the Bayesian interpretation of probability. The probability of an event represents the degree of belief that it will occur, and prior information is used to predict the posterior distribution of reliability. Consider the parameter, θ, to be a random variable. The existing information was used to obtain prior distribution, the sample $X=\left(x_1,x_2,\dots x_n\right)$, the prior information and current information was integrated to calculate the posterior distribution and statistically infer the value of $\theta$. The mathematical model can be expressed by $f(x|\theta )\propto L(\theta |x)\pi \left(\theta \right)$, where $\pi \left(\theta \right)$ denotes the prior probability density function of $\theta$, $f(x|\theta )$ denotes the posterior probability density function, and $L(\theta |x)$ denotes the likelihood function.

The authors proposed a two-stage reliability assessment of web service compositions—reliability growth stage and reliability demonstration stage. Assuming the reliability requirement to be $\left(p_0,c\right)$, the probability of $x$ failures in n test cases follows the binomial distribution—i.e., the probability of failure of the web service composition is $p_s$, which follows $Beta\left(a,b\right)$. Next, based on the aforementioned information obtained via reliability growth assessment, the distribution of the software failure probability, p, during reliability demonstration assessment was given by binary experiment—acceptable or rejected—the binomial sampling model with its conjugate beta distribution was widely used. The choice of the beta distribution as a prior is useful for several reasons: it is flexible enough to describe a variety of prior beliefs, it ensures that reliability is between (0, 1), and it is the conjugate prior for the binomial distribution [23,24]. The prior probability density of $p_s$ can be expressed as follows:

$$\pi \left(p_s\right)=\frac{p^{a-1}{\left(1-p\right)}^{b-1}}{B\left(a,b\right)}$$

(7)

where, $B\left(a,b\right)={\int }_0^1{x}^{a-1}{\left(1-x\right)}^{b-1}dx$.

During the reliability growth stage, the prior probability density $p_s$ is confirmed and ${\widehat{p}}_s$ (the estimate of $p_s$) is compared with the reliability requirement $p_0$—if ${\widehat{p}}_s\le p_0$, reliability demonstration testing is conducted in the next step, if ${\widehat{p}}_s>p_0$, need to repair, reliability growth testing is conducted again. Assuming that there are $x$ failures in $n$ test cases, the posterior probability density of $p_s$ of the web service composition is given by:

$$\begin{array}{ccc}f\left(p|x,n,a,b\right)& =& Be\left(a+x,b+N-x\right)\hfill \\ & =& \frac{p^{a+x-1}{\left(1-p\right)}^{b+n-x-1}}{B\left(a+x,b+n-x\right)}\hfill \end{array}$$

(8)

The expectation of the posterior probability density, $p_s$, is:

$$\begin{array}{ccc}{\widehat{p}}_s^{\prime }& =& E\left(p_s|x,n,a_0,b_0\right){\int }_0^1{p}_{s}B\left(a_0+x,b_0+n-x\right)d{p}_s\hfill \\ & =& \frac{a_0+x}{b_0+n-x}\hfill \end{array}$$

(9)

The posterior expected value of $p_s$ is known to be the minimum secondary loss estimate of the software failure reliability. It is used as the estimated value of the current probability of the web service composition. If ${\widehat{p}}_s^{\prime }<p_0$, the current reliability of the web service composition satisfies the reliability requirement, and the reliability growth test is deemed to be complete.

During the reliability demonstration stage, the posterior distribution $p_s$ of the probability of failure during the growth test can be utilized, which integrates the information with the reliabilities of the web services and the web service composition. Thus, the distribution of failure probabilities can be obtained based on the prior information in the demonstration stage. The number of test cases that satisfy the reliability requirement, $\left(p_0,c\right)$, is taken to be the minimum value of $n$ in Equation (10):

$$\begin{array}{ccc}P\left(p_s\le p_0\right)& =& {\int }_0^{p_0}f(p|x,n,a,b)dp\hfill \\ & =& {\int }_0^{p_0}\frac{p^{a+x-1}{\left(1-p\right)}^{b+n-x-1}}{B\left(a+x,b+n-x\right)}dp\hfill \\ & \ge & \mathrm{c}\hfill \end{array}$$

(10)

Furthermore, by (10), after $n$ test cases during which no failure is observed in reliability demonstration assessment, the reliability of web service composition satisfies the reliability requirement and can be accepted.

2.2.2. Bayes Decision

Reliability assessment was used to verify whether the software reliability satisfies the pre-defined reliability requirement, and the test result was assumed to be binary—acceptable or rejected. This transforms it into a decision-making problem (hypothesis testing) [23]. Let ${\mathrm{H}}_0$ and $H_1$ denote the two decision states. ${\mathrm{H}}_0$ denotes the test result meets reliability requirement, $H_1$ denotes the test result does not meet the reliability requirement. The result of the software reliability assessment can be described by

$$H_0:\theta \le {\theta }_0, H_1:\theta >{\theta }_0,$$

where $\theta$ denotes the actual failure rate, ${\mathsf{\theta }}_0$ denotes the failure rate of the reliability requirement, and the test significance level is taken to be $c (0<c<1)$. In this decision-making problem, the authors utilized the Bayesian decision function to describe the satisfaction of the reliability requirement by the assessed software in the presence of a certain number of failures. The decision function is given by

$$\delta \left(x\right)=P(H_0|X=x)$$

where $x$ denotes the number of failures during reliability assessment based on Bayes’ concept of assessment. The actual failure rate, $\mathsf{\theta }$, of the web service composition is taken to be a random variable, in this paper, assuming the prior distribution of parameter $\theta$ to be $G\left(\theta \right)$; therefore, the Bayesian decision function is given by

$${\delta }_G\left(x\right)=\{\begin{array}{c}1, \theta \le {\theta }_0\\ 0, \theta >{\theta }_0\end{array}$$

according to (8), assuming the hyperparameter $a$ and $b$ are known, after computing. When the number of failures allowed by reliability requirement are 0, 1, and 2, $N_0,N_1,\mathrm{and}{N}_2$ denote the upper limit number of test cases that need to be running, respectively. ${\delta }_G\left(x\right)$ can be transformed into the following (11):

$${\delta }_G\left(x\right)=\{\begin{array}{c}1, x=0 and n<N_0,x=1 and n<N_1,x=2 and n<N_2\dots \\ 0, \theta >{\theta }_0\end{array}$$

(11)

The authors considered the result of reliability testing as Bayes decision problem; once the test result is wrong, which is bias from truth, it will cause the loss. In this paper, the authors utilized the Bayes decision idea in reliability testing of web service composition. In order to reduce the loss and improve accuracy of reliability testing, and control the bias of prior distribution $G\left(\theta \right)$ of parameter $\theta$, in the next chapter, the authors propose the method of minimum risk (MMR) to produce the result with the lowest risk of decision.

2.2.3. Bayes Prior Hyperparameter Solving Method Based on MMR

In Bayesian reliability assessment, prior information is used to capture existing knowledge about unknown parameters. The efficiency and accuracy of the assessment is determined in terms of the prior probability distribution. An accurate prior distribution dramatically improves the required number of test cases [24]. The parameters of the prior distribution are referred to as hyperparameters, such as by $a$ and $b$ in (7). The values of the hyperparameters determine the accuracy of the prior probability distribution, which is an important factor in Bayesian decision-making [25]. A fundamental shortcoming of reliability assessment methods of web service compositions is that due to the accumulation of a large number of failures at each web service testing phase and a small number of failures in the web service composition testing phase, the lack of prior information makes the hyperparameters inaccurate and increases the risk of wrong decisions. In this section, the authors present a method for calculating a priori parameters based on an MMR, to minimize the risk of wrong Bayes decisions during reliability testing. The proposed method utilizes module-level prior information alongside the failure data of the constituent web services.

In Section 2.2, the authors consider the reliability assessment of web services as a decision-making problem; if wrong testing happened, incorrect assessments induce the loss of accuracy. The authors used the loss function to describe the risk (12), which represents the loss induced by wrong assessment. To measure the loss accurately, the authors used the risk function to express the expected value of the loss. Consequently, the decision-making problem can be converted into that of identifying a method that minimizes the risk during testing. The loss function can be indicated in following:

$$\begin{gathered} L_0=\left(\theta ,d_0\right)=a\left(\theta -{\theta }_0\right)I\left(\theta >{\theta }_0\right) \\ {L}_1\left(\theta ,d_1\right)=a\left(\theta -{\theta }_0\right)I\left(\theta \le {\theta }_0\right) \end{gathered}$$

(12)

where a denotes a positive constant and $d=\left\{d_0,d_1\right\}$ indicates the test results, so according to result of reliability testing, the decision rule is: $d_0$ denotes that accepted $H_0$, $d_1$ denotes that rejected $H_0$. $I\left(\theta >{\theta }_0\right)$ represents the actual failure probability of the web service composition being larger than the reliability requirement. On the other hand, $I\left(\theta <{\theta }_0\right)$ represents the actual failure probability of web service composition being smaller than the reliability requirement. $L_0=\left(\theta ,d_0\right)$ denotes the custom risk corresponding to cases in which the software is accepted despite the failure probability being larger than the reliability requirement and $L_1=\left(\theta ,d_1\right)$ denotes the producer risk corresponding to cases in which the software is rejected despite the failure probability being smaller than the reliability requirement.

If the sample set is denoted by $\Omega =\left\{x|x\rangle 0\right\}$, the parameter set is denoted by $\Theta =\{\theta >0|{\int }_{\mathsf{\Omega }}f\left(x|\theta \right)dx=1\}$, then, corresponding to the prior distribution $G\left(\theta \right)$, the risk function of the decision function, $\delta \left(x\right)$, is given by

$$\begin{array}{ccc}R\left(\delta \left(x\right),G\left(\theta \right)\right)& =& E_{\left(x,\theta \right)}\left[L\left(\theta ,\delta \left(x\right)\right)\right]\hfill \\ & =& {\int }_{\Theta }\left[{\int }_{\Omega }L\left(\theta ,\delta \right)f\left(x|\theta \right)dx\right]dG\left(\theta \right)\hfill \end{array}$$

(13)

substituting (12) into (13) yields

$$\begin{gathered} R\left(\delta \left(x\right),G\left(\theta \right)\right)={\int }_{\Theta }{\int }_{\Omega }\left[L_0\left(\theta ,d_0\right)f\left(x|\theta \right)\delta \left(x\right)+L_1\left(\theta ,d_1\right)f\left(x|\theta \right)\left(1-\delta \left(x\right)\right)\right]dxdG\left(\theta \right) \\ ={\int }_{\Theta }{\int }_{\Omega }\left[a\left(\theta -{\theta }_0\right)\left(1-I\left(\theta <{\theta }_0\right)\right)\delta \left(x\right)+L_1\left(\theta ,d_1\right)\left(1-\delta \left(x\right)\right)\right]f\left(x|\theta \right)dxdG\left(\theta \right) \\ ={\int }_{\Theta }{\int }_{\Omega }\left[a\left(\theta -{\theta }_0\right)\delta \left(x\right)+L_1\left(\theta ,d_1\right)\right]f\left(x|\theta \right)dxdG\left(\theta \right) \end{gathered}$$

(14)

and further substituting (11) and ${\int }_{\mathrm{\Omega }}f(x|\theta )dx=1$ into (14) yields

$$R\left(\delta \left(x\right),G\left(\theta \right)\right)=a{\int }_{x\in \left\{x\right|\theta \le {\theta }_0\}}\beta \left(x\right)dx+C_G$$

(15)

where

$$\begin{gathered} \beta \left(x\right)={\int }_{\Omega }^{{\theta }_0}\left(\theta -{\theta }_0\right)f\left(x|\theta \right)\pi \left(\theta \right)d\theta \\ \begin{array}{ccc}{C}_G& =& {\int }_{\Theta }{L}_1\left(\theta ,d_1\right)\pi \left(\theta \right)d\theta \\ & =& a{\int }_0^{{\theta }_0}\left({\theta }_0-\theta \right)\pi \left(\theta \right)d\theta \end{array} \end{gathered}$$

Therefore, during reliability assessment of web services, substituting the distribution function, $f(x|\theta )$, of the number of failures and the prior probability density function, $\mathsf{\pi }\left(\mathsf{\theta }\right)$, of failure probabilities into (15), the risk function of the decision function, $\delta \left(x\right)$, can be obtained as the following.

$$R=a{\int }_0^e{\int }_0^{p_0}\left(p-p_0\right)C_N^x{p}^x{\left(1-p\right)}^{N-x}\frac{p^{a-1}{\left(1-p\right)}^{b-1}}{B\left(a,b\right)}dpdx+a{\int }_0^{p_0}\left(p-p_0\right)\frac{p^{a-1}{\left(1-p\right)}^{b-1}}{B\left(a,b\right)}dp$$

(16)

where $e$ denotes the maximum number of permissible failures during the assessment of the web service composition. After obtaining the estimated values, ${\widehat{p}}_1,{\widehat{p}}_2,\dots ,{\widehat{p}}_n$, of the failure probabilities corresponding to all web services, the authors can utilize (6) to calculate the estimated value, $p_s$, of the failure probability of the web service composition as follows.

$${\widehat{p}}_s=F\left({\widehat{p}}_1,{\widehat{p}}_2,\dots ,{\widehat{p}}_n\right)$$

(17)

The prior probability distribution of the failure probability, $p_s$, follows $Beta\left(a,b\right)$. Therefore, the expected value of the failure probability, $p_s$, of the web service composition is given by

$$\overline{p_s}=E\left(\pi \right)={\int }_0^1{p}_{s}Be\left(a,b\right)d{p}_s=\frac{a}{a+b}$$

(18)

The expected value of ${\mathrm{p}}_{\mathrm{s}}$ is taken to be the estimated value of the failure probability of the web service composition, yielding

$$\frac{a}{a+b}=F\left({\widehat{p}}_1,{\widehat{p}}_2,\dots ,{\widehat{p}}_n\right)$$

(19)

Therefore, the problem of calculating prior hyperparameters, a and b, can be transformed into the problem of identifying the optimal solutions of (20) while satisfying the MMR.

$$\begin{gathered} Min\left\{R\right\}=Min\left\{a{\int }_0^e{\int }_0^{P_0}\left(p-p_0\right)C_N^x{p}^x{\left(1-p\right)}^{N-x}\frac{p^{a-1}{\left(1-p\right)}^{b-1}}{B\left(a,b\right)}dpdx+a{\int }_0^{P_0}\left(p-p_0\right)\frac{p^{a-1}{\left(1-p\right)}^{b-1}}{B\left(a,b\right)}dp\right\} \\ \frac{a}{a+b}=F\left({\widehat{p}}_1,{\widehat{p}}_2,\dots ,{\widehat{p}}_n\right)a>0,b>0 \end{gathered}$$

(20)

where $x$ denotes the permissible number of failures and $N$ denotes the total number of test cases. Finally, based on the values of the hyperparameters, $a \mathrm{and} b$, the failure probability distribution can be obtained, and the number of test cases, $\mathrm{n}$, in (10) satisfies the reliability requirement of the web service composition assessment.

3. Results and Discussion

The authors constructed a controlled environment comprising a set of web services and deployed four web services in the Amazon web service (AWS). Each service was executed on an Amazon machine image equipped with CentOS 8 (x86_64), ASP.NET Core [26]. The architecture of the web service composition in this paper is illustrated in Figure 5. It includes four web services, and the arrows in the figure denote their invocations. The authors utilized the tool, SoapUI, to generate the soap requests according to web service description language (WSDL) [27] to simulate the process of user-invoked web services.

3.1. Reliability Assessment of Web Service Composition

In this paper, the authors primarily considered the errors induced by web service responses. To this end, the authors utilized a very useful Linux utility called traffic control (tc) to assign a threshold of 10 s for the response time to adequately simulate and control the network latency. If a response is so slow that its delay is noticeable by a customer (i.e., 20% or more), then the service is deemed to have effectively failed.

The reliability requirement of the web service composition set by the developer is $p=0.004$. Reliability assessment of the web services revealed the corresponding failure probabilities, as presented in

Table 2. Failure probabilities of web services.

Web Service	Failure Probability
$w_1$	0.081
$w_2$	0.024
$w_3$	0.096
$w_4$	0.038

—${\widehat{p}}_1=0.081$, ${\widehat{p}}_2=0.024$, ${\widehat{p}}_3=0.096$ and ${\widehat{p}}_4=0.038$. By (6), the failure probability, $p_s$, of the web service composition can, therefore, be calculated as follows.

$${\widehat{p}}_s=F\left({\widehat{p}}_1,{\widehat{p}}_2,\dots ,{\widehat{p}}_4\right)=0.0081$$

(21)

Substituting (21) into (19) yields

$$\frac{a}{a+b}=0.0081$$

(22)

Using (20), the values of the hyperparameters were calculated to be a = 1 and b = 122.5 following MMR. Then, reliability assessment was initiated with reliability growth assessment. By (21), the reliability, ${\widehat{p}}_s$, of the web service composition was calculated to be $0.0081$ and, as $0.0081>0.004$, the reliability of web service composition was deemed to not satisfy the pre-defined reliability requirement, after repairing. Thus, reliability growth assessment was continued. When performing the 250th test case, the failure occurred, then correcting it. By (8), the failure probability can be calculated as follows.

$${\widehat{p}}_s^{\prime }=\frac{a_0+x}{a_0+b_0+n}=\frac{1+1}{1+122.5+250}=0.0054>0.004$$

Evidently, the current reliability of web service composition still did not satisfy the reliability requirement. Thus, testing was continued until the 1529th test case, when the 6th failure was observed. The corresponding failure probability was calculated as follows.

$${\widehat{p}}_s^{\prime }=\frac{a_0+x}{a_0+b_0+n}=\frac{1+6}{1+122.5+1829}=0.0036<0.004$$

At this point, the reliability of the web service composition finally satisfied the reliability requirement, and therefore, the software reliability growth assessment was deemed to be complete. Meanwhile, the values of the hyperparameters of the posterior distribution of the failure probability $p$ were calculated to be $a_0=7, \mathrm{and} b_0=1760.1$.

$$f\left(p_s\right)=Beta\left(7, 1760.1\right)=\frac{p_s^6{\left(1-p_s\right)}^{1759.1}}{B\left(7, 1760.1\right)}$$

To verify whether the reliability of the web service composition satisfied the reliability requirement, the following equation was used, following (20), to calculate the maximum number of test cases required for the reliability demonstration test.

$${\int }_0^{p_0}\frac{p^{a_0+j-1}{\left(1-p\right)}^{b_0+N_{j+1}-j-1}}{B\left(a_0+j,b+N_{j+1}-j\right)}dp\ge c$$

Finally, the authors compared the required numbers of test cases using MMR, Bayesian Monte Carlo method (BMCM) [28], and expert scoring method (ESM) [29] to obtain a reliability demonstration requirement of $\left(p_0,c\right)=\left(0.005,0.9\right)$. With an increasing number of failures, the maximum number of test cases corresponding to different methods was required to satisfy the requirements presented in

Table 3. Number of reliability demonstration test cases with different methods.

Number of Failures	Number of Test Cases
	BMCM	ESM	MMR
0	4602	3932	3692
1	7635	5478	4573
2	9402	7123	6432
3	10,041	8232	8923
4	11,900	10,345	10,401
5	16,104	13,335	11,452

.

In the case of BMCM, the unavailability of prior information during the first execution of the web service composition induced a uniform distribution of the hyperparameters over the domain. In this case, Beta(2, 2) was adopted to represent the lack of prior knowledge, which assigned the prior reliability centers around 0.5, with equal probabilities of failure and success. Then, the Bayes formula based on the prior distribution was used to calculate the $n$ test cases. In a beta-binomial conjugation framework, it is useful to think that the hyperparameters of the prior distribution (beta distribution) of system failure probability correspond to a certain number of pseudo observations with the properties specified by these hyperparameters [30,31].

In the case of ESM, let us assume that there are $n$ experts who assign judgment intervals of $\left[{\theta }_L,{\theta }_H\right]$ corresponding to the failure probability, ${\theta }_k$, associated to the $k$th stage of growth assessment, and that ${\overline{\theta }}_L$ and ${\overline{\theta }}_H$ denote the means of ${\theta }_L$ and ${\theta }_H$, respectively, over all experts. Then, the expectation, $E$, and the variance, $V$, can be calculated using the following equations.

$$\begin{gathered} E=\frac{{\overline{\theta }}_L+{\overline{\theta }}_H}{2} \\ V=\frac{{\left({\overline{\theta }}_H-{\overline{\theta }}_L\right)}^2}{12} \end{gathered}$$

The hyperparameters, $a \mathrm{and} b$, can be evaluated using the following equations.

$$\begin{gathered} a=\frac{E·\left(V-nE\right)}{D} \\ b=\frac{\left(n-E\right)\left(V-nE\right)}{D} \end{gathered}$$

where $D=\left(n-1\right)E^2+n\left(E-V\right)$.

The judgment interval, $\left[{\theta }_L,{\theta }_H\right]$, of each expert reflects their judgement as elicited via interviews. Following the completion of the three steps of reliability growth assessment presented in

Table 4. Information of experts.

Stage in Growth Testing	Expert 1	Expert 2	Expert 3	Expert 4	Beta Distribution		Hyperparameter
					Expected	Variance	$\mathit{a}$	b
1	(0.003, 0.0072)	(0.0035, 0.0075)	(0.0040, 0.0074)	(0.0038, 0.0072)	0.0054	1.17 × 10−6	17.58	140.41
2	(0.0080, 0.0088)	(0.0060, 0.0086)	(0.0065, 0.0090)	(0.0063, 0.0085)	0.0077	3.42 × 10−6	21.22	270.78
3	(0.0075, 0.0095)	(0.0077, 0.0093)	(0.0076, 0.0092)	(0.0078, 0.95)	0.0085	2.48 × 10−6	33.27	311.92

, the posterior distribution was obtained to be Beta (61.27,324.92).

The numbers of required test cases corresponding to different methods are presented in Table 3. The required number of test cases in the case of BMCM is $n=4602$, which implies that if no failure is observed in 4602 successive test cases, the reliability of web service composition is deemed to satisfy the reliability requirement, $\left(p_0,c\right)=\left(0.005,0.9\right)$, and the software can be accepted. However, if a failure is noticed in, say, the 3021st test case, the assessment must be continued for 4614 (7635 − 3021 = 4614) further test cases. If no failure is observed during these iterations, the web service composition is deemed to satisfy the reliability demonstration requirement. However, if an additional failure was noticed during this stage as well at, say, the 5500th test case, the assessment is to be continued for another additional 3902 (9402 − 5500 = 3902) test cases. The satisfaction of the reliability demonstration requirement is then adjudged once again. The process continues in an analogous fashion.

The experimental data indicated that the number of test cases required by MMR was significantly lower than that required by BMCM and ESM. Corresponding to 0 failures, the numbers of test cases required by BMCM and ESM were 4602 and 3932, respectively, whereas that required by MMR was merely 3692. This represents reductions of 910 and 670, respectively, corresponding to reduction rates of 19.8% and 14.6%, respectively. Furthermore, in the case of MMR, the required number of test cases exhibited the smallest increment when the number of failures was increased from 0 to 1—for BMCM, ESM, and MMR, the required numbers of test cases increased by 65.9%, 39.3%, and 23.9%, respectively, in this case. Similarly, when the number of failures was increased from 4 to 5, the increase in the numbers of test cases required by BMCM, ESM, and MMR were observed to be 35.3%, 28.9%, and 10.1%, respectively.

Thus, the experiments demonstrated that in the case of identical prior information, MMR was the most accurate method to calculate the prior distribution, and it significantly reduced the number of test cases required during reliability demonstration assessment. In the case of ESM, the opinions of experts made the value of hyperparameters more subjective, which wielded a greater influence on the postdistribution. In the case of BMCM, the lack of prior information induced a uniform initial distribution of the hyperparameter over the domain. Subsequently, the distributions were repeatedly sampled to obtain the expected values of the hyperparameters. This procedure usually involves complex numerical calculations and is difficult to implement.

3.2. Goodness-of-Fit of the Distribution

In this subsection, the authors analyze the goodness-of-fit of the distribution to the truth data; maximum likelihood estimation (MLE) was used to evaluate the accuracies of the corresponding distributions obtained using hyperparameters estimated using BMCM, ESM, and MMR. MLE is a method to evaluate models using specific data—given different prior to distributions; it determines the corresponding likelihoods of the actual sample to determine the confidences of the prior distributions. The likelihood is termed the confidence factor; and it is proportional to the proximity of the prior distribution to the ground truth. Obviously, the distribution that is closest to the ground truth is deemed to be the best.

MLE was conducted using the following procedure. Let us assume that the failure probability distribution of the web service composition, $X=\left\{p_1,p_2\dots p_n\right\}$, is given by $f(X|\theta )$, where $\theta$ denotes the hyperparameters, $a$ and $b$, and the density function of the prior distribution of $\theta$ is $\pi \left(\theta \right)$. After obtaining $X$, the density function $\pi (\theta |X)$ of the posterior distribution can be expressed as

$$\pi \left(\theta |X\right)=\frac{f(X|\theta )\pi \left(\theta \right)}{{\int }_{\theta }^{ }f(X|\theta )\pi \left(\theta \right)d_{\theta }}$$

(23)

where $f(X|\theta )$ reflects the knowledge of $\theta$ based on the observed $X$. If there are two parameters, ${\theta }_1$ and ${\theta }_2$, the authors obtain

$$f(X|{\theta }_1)>f(X|{\theta }_2)$$

(24)

From the perspective of statistics, the probability of observed $X$ with the parameter ${\theta }_1$ is higher than $X$ with ${\theta }_2$. Thus, (24) can be interpreted as the likelihood of $\theta ={\theta }_1$ being larger than that of $\theta ={\theta }_2$. Thus, when $X$ is constant, $f(X|\theta )$ is independent of the prior distribution of $\theta$ as the function with $\theta$; instead, it merely reflects the characteristics of the sample.

The denominator of (23) is given by

$$m\left(X\right)={\int }_{\theta }^{ }f(X|\theta )\pi \left(\theta \right)d_{\theta }$$

(25)

This is the marginal distribution of $X$ that depends on $f(X|\theta )$. $f(X|\theta )$ is the conditional distribution density and, for a given $\theta$, the marginal distribution describes the action of the sample $X$ under the prior distribution, $\pi \left(\theta \right)$, and the sample follows this distribution. If $\pi \left(\theta \right)$ follows a certain distribution, the parameter of $\pi \left(\theta \right)$ is obtained from the maximum value of $m\left(X\right)$—i.e., the MLE.

If there are $m$ prior distributions that follow $f(X|\theta )$, the failure probability samples are $p_1,p_2\dots p_n.$ Let ${\pi }_i\left(\theta \right)$ be the $i$^th prior distribution based on the sample. If the ground truth prior distribution is denoted by $\pi \left(\theta \right)$, the $m$ prior distributions can be integrated into the ground truth prior distribution using the following equation.

$$\pi \left(\theta \right)={\displaystyle \sum }_{i=1}^m{\epsilon }_i{\pi }_i\left(\theta \right)$$

(26)

where ${\epsilon }_i$ denotes the weight of $i$th prior distribution, ${\pi }_i\left(\theta \right)$, and $\sum _{i=1}^m{\epsilon }_i=1$.

The marginal distribution of the prior distribution, ${\pi }_i\left(\theta \right)$, is given by

$$m\left(x|{\pi }_i\right)={\int }_{\theta }^{ }f\left(X|\theta \right){\pi }_i\left(\theta \right)d_{\theta }$$

(27)

The marginal distribution of the ground truth prior distribution, $\pi \left(\theta \right)$, is given by

$$m\left(x|\pi \right)={\int }_{\theta }^{ }f(X|\theta )\pi \left(\theta \right)d_{\theta }$$

(28)

The failure data, $p_1,p_2\dots p_n$, generated by the marginal distribution, $m\left(x|{\pi }_i\right)$, can be used to obtain the following likelihood function.

$$L\left(X|{\pi }_i\right)={\displaystyle \prod }_{i=1}^{n}m(x_i|{\pi }_i)$$

(29)

By MLE, $L\left(X|{\pi }_i\right)$ is bigger to represent the closer proximity to the ground truth, the weight of ${\pi }_i\left(\theta \right)$ is bigger, high probability to choose this prior distribution. The confidence factor can be obtained as follows.

$${\epsilon }_k=\frac{L\left(X|{\pi }_i\right)}{{\sum }_{i=1}^{i}L\left(X|{\pi }_i\right)}$$

(30)

The confidence factor represents the probability of a prior distribution being close to the truth distribution. Thus, if the confidence factor of a distribution is high, it is highly feasible to choose it as a prior distribution.

Next, the authors compared the accuracies of the distribution constructed using the hyperparameters estimated using BMCM, ESM, and MMR. The failure probability follows $Beta\left(a,b\right)$, where $a$ and $b$ are hyperparameters.

First, certain defects were introduced into the web services, following the discussion presented in [32]. The web service composition considered in this section is depicted in Figure 4. The actual failure probability following the introduction of the defects was regarded as the ground truth, the failure probabilities obtained via various method were regarded as evaluation values, $X$ denotes the ground truth data, and MLE was used to determine the distribution that is closest to the truth. The failure probabilities of each web service obtained via module-level testing are presented in

Table 5. Failure probabilities of web services.

Web Services	Failure Probability
$w_1$	0.011
$w_2$	0.034
$w_3$	0.096
$w_4$	0.018

—${\widehat{p}}_1=0.011$, ${\widehat{p}}_2=0.034$, ${\widehat{p}}_3=0.096$ and ${\widehat{p}}_4=0.018$. Based on injected defects, the ground truth of the failure probability of the web service composition was calculated to be $X=\left(0.184, 0.299, 0.156\right)$, which follows $Beta\left(3.0, 6.0\right)$.

Then, the authors calculated the failure probabilities using the predictions obtained from BMCM, ESM, and MMR as evaluation values. In the case of BMCM, the initial non-informative prior distribution was assumed to be $Beta\left(2, 2\right)$. In the case of ESM, after assessment over 10 h, the prior distribution of the web service composition was estimated to by expert interview and the initial prior distribution was $Beta\left(3, 6.3\right)$. In the case of MMR, by (20) and (22), the initial prior distribution was $Beta\left(2.5, 3.9\right)$.

After three steps of growth assessment, the probability distributions corresponding to the three methods were observed to be ${\pi }_1=Beta\left(3.1, 5.9\right)$, ${\pi }_2=Beta\left(2.7, 6.6\right)$, and ${\pi }_3=Beta\left(3.5, 7\right)$ (${\pi }_1$ denotes MMR, ${\pi }_2$ denotes BMCM, and ${\pi }_3$ denotes ESM).

By (27), the marginal distribution functions are given by

$$\begin{gathered} m\left(x|{\pi }_1\right)=\frac{\mathsf{\Gamma }\left(a+b\right)}{\mathsf{\Gamma }\left(a\right)\mathsf{\Gamma }\left(b\right)}{x}^{a-1}{\left(1-x\right)}^{b-1}=\frac{\mathsf{\Gamma }\left(9\right)}{\mathsf{\Gamma }\left(3.1\right)\mathsf{\Gamma }\left(5.9\right)}{x}^{2.1}{\left(1-x\right)}^{4.9} \\ m\left(x|{\pi }_2\right)=\frac{\mathsf{\Gamma }\left(a+b\right)}{\mathsf{\Gamma }\left(a\right)\mathsf{\Gamma }\left(b\right)}{x}^{a-1}{\left(1-x\right)}^{b-1}=\frac{\mathsf{\Gamma }\left(9.3\right)}{\mathsf{\Gamma }\left(2.7\right)\mathsf{\Gamma }\left(6.6\right)}{x}^{1.7}{\left(1-x\right)}^{5.6} \\ m\left(x|{\pi }_3\right)=\frac{\mathsf{\Gamma }\left(a+b\right)}{\mathsf{\Gamma }\left(a\right)\mathsf{\Gamma }\left(b\right)}{x}^{a-1}{\left(1-x\right)}^{b-1}=\frac{\mathsf{\Gamma }\left(10.5\right)}{\mathsf{\Gamma }\left(3.5\right)\mathsf{\Gamma }\left(7\right)}{x}^{2.5}{\left(1-x\right)}^6 \end{gathered}$$

Then, by (29), the likelihood functions are given by

$$\begin{gathered} L\left(X|{\pi }_1\right)=8.3003e-005 \\ L\left(X|{\pi }_2\right)=4.0761e-005 \\ L\left(X|{\pi }_3\right)=1.2588e-005 \end{gathered}$$

By (30), the respective confidences are given by

$$\begin{gathered} {\epsilon }_1=0.6707 \\ {\epsilon }_2=0.3294 \\ {\epsilon }_3=0.1321 \end{gathered}$$

As to ${\epsilon }_1>{\epsilon }_2>{\epsilon }_3$, the prior distribution corresponding to MMR is closest to the ground truth and is, therefore, the best candidate for the prior distribution.

The curves of Beta corresponding to the three different methods are illustrated in Figure 6. Line 0 corresponds to the ground truth, Line 1 corresponds to the prior distribution, ${\pi }_1=Beta\left(3.1, 5.9\right)$, Line 2 corresponds to the prior distribution, ${\pi }_2=Beta\left(2.7, 6.6\right)$, and Line 3 corresponds to the prior distribution, ${\pi }_3=Beta\left(3.5, 7\right)$. Line 1, corresponding to MMR, is evidently a significantly better fit compared to the other curves. Therefore, calculating the hyperparameters using MMR is the most reasonable approach in this case.

The accuracy of reliability estimation depends on the prior distribution. Based on the aforementioned discussion, the prior distribution constructed using hyperparameters calculated using MMR is the closest to the ground truth distribution. Thus, MMR can be used efficiently utilize the prior information of web services, and its results exhibit the lowest decision risk and are more objective and consistent with the actual situation.

4. Conclusions

In this paper, a reliability assessment method was proposed for web service compositions. To this end, the authors used transfer probabilities to represent the inter-relationships between the constituent web services, and used a Markov model to transfer the probability matrix and calculate the reliability of web service composition. The proposed method makes full use of failures data of the web services, thereby adequately resolving the problem of a lack of prior information in web service compositions.

The authors divided the reliability assessment procedure into two stages—the reliability growth stage and the reliability demonstration stage. The primary problems faced during the reliability growth testing are inaccurate prior distributions and the requirement of large numbers of test cases, which increases the test duration. The authors used Bayes reliability assessment for reliability testing, thereby fully utilizing prior information of the web services, and proposed reliability demonstration assessment based on prior information. The authors assumed the number of failures, $x$, to follow a binomial distribution, $Beta\left(a,b\right)$. The authors compared the estimated failure probabilities, $p_s$, to $p_0$ and formulated a completion condition for the reliability growth assessment. During the reliability demonstration stage, the authors considered the reliability assessment result as essentially a Bayesian decision-making (hypothesis test) problem. During reliability assessment, prior hyperparameters affect the accuracy of the prior probability distribution and play an important role in the Bayesian decision. The authors discussed the importance of hyperparameters in the Bayesian decision, and proposed the MMR method to calculate prior hyperparameters. Experiments were conducted to demonstrate that the proposed method exhibited a smaller number of required test cases compared to two other methods. This increases the efficiency of reliability demonstration assessment and avoids the adoption of rash inferences about the reliability of web service compositions.

In future research, the authors intend to consider web service architectures of more complicated designs that require significantly more calculations, considering different reliability growth models; the efficiency of the proposed method should also be improved. In addition, reliability assessment based on MMR should be repeated to further improve the proposed method. The final aim is to efficiently utilize available resources during reliability assessment of web service compositions.