Bayesian Meta-Analysis for Binary Data and Prior Distribution on Models

Abstract

In meta-analysis, the structure of the between-sample heterogeneity plays a crucial role in estimating the meta-parameter. A Bayesian meta-analysis for binary data has recently been proposed that measures this heterogeneity by clustering the samples and then determining the posterior probability of the cluster models through model selection. The meta-parameter is then estimated using Bayesian model averaging techniques. Although an objective Bayesian meta-analysis is proposed for each type of heterogeneity, we concentrate the attention of this paper on priors over the models. We consider four alternative priors which are motivated by reasonable but different assumptions. A frequentist validation with simulated data has been carried out to analyze the properties of each prior distribution for a set of different number of studies and sample sizes. The results show the importance of choosing an adequate model prior as the posterior probabilities for the models are very sensitive to it. The hierarchical Poisson prior and the hierarchical uniform prior show a good performance when the real model is the homogeneity, or when the sample sizes are high enough. However, the uniform prior can detect the true model when it is an intermediate model (neither homogeneity nor heterogeneity) even for small sample sizes and few studies. An illustrative example with real data is also given, showing the sensitivity of the estimation of the meta-parameter to the model prior.

1. Introduction

Meta-analysis has been widely applied in many research areas and is of particular importance in healthcare studies. When there exist different randomized controlled clinical trials (or studies) of a particular medical treatment, a meta-analysis may be conducted to determine what final conclusion can be drawn on the important question from each study, the effectiveness of the treatment.

One of the cases that has received more attention in the literature is the meta-analysis for binary data [1]. On the one hand, because it is very common for effectiveness to be measured through a binary variable according to whether or not a certain objective has been achieved (to survive, do not relapse or to reach a low viral load). Charles et al. [2] found that half of trials calculated their sample size based on a binary outcomes. On the other hand, binary outcomes have different statistical considerations to using continuous outcomes. The Bayesian random-effects model for meta-analysis given by Sutton and Abrams [3] would not be suitable for modeling binary data:

$$\begin{array}{c}\hfill x_i\sim \mathcal{N}({\theta }_i,{\tau }_i),i=1,\dots ,k\\ \hfill {\theta }_i\sim \mathcal{N}(\theta ,\tau ),\\ \hfill \theta \sim [-,-]\tau \sim [-,-],\end{array}$$

(1)

where $x_i$ denotes an observed effect for each of k studies, $\theta$ the estimated pooled effect, and ${\tau }^2$ is an estimate of the between-study variance. For binary data, the preceding normal hierarchical model has been applied to the logit transformation $y_i=log[x_i/(n_i-x_i)]$ with the reparametrization $log[{\theta }_i/(1-{\theta }_i)]$, where $x_i$ denotes the number of successes at the ith study [4,5,6,7]. However, this normal approximation does not work properly when the samples sizes ($n_i$) are small or when the number of successes is zero, even if a continuity correction is applied to the original data, as it was shown by Sweeting et al. [8].

Moreno et al. [9] proposed an objective Bayesian meta-analysis model for binary data in which no continuity correction is required. The Bayesian model proposed for the study i is based on the binomial distribution $\left\{M_i:\mathrm{Bin}(x_i,{\theta }_i,n_i),\pi \left({\theta }_i\right)\right\}$, and the linking distribution between the parameters of each study ${\theta }_i$ and the meta-parameter $\theta$, $\pi ({\theta }_i,\theta )$, belongs to the Fréchet class of bidimensional distributions with fixed marginals $\pi \left({\theta }_i\right)$ and $\pi \left(\theta \right)$. The objective Bayesian analysis assumes that these marginals are uniform priors, Unif$\left({\theta }_i\right|0,1)$ and Unif$\left(\theta \right|0,1)$.

The model parameters will therefore be the parameters for the k studies, ${\theta }_1,\dots ,{\theta }_k$, and the meta-parameter $\theta$. However, if some of the ${\theta }_i^{\prime }s$ are equal the dimension of the model would be reduced. In [10], the authors proposed to study the between-sample heterogeneity as a model selection problem, clustering the parameters ${\theta }_1,\dots ,{\theta }_k$ based on the samples $(x_1,n_1),\dots ,(x_k,n_k)$. They adopted a Bayesian approach based on product partition models proposed in [11,12]. Bayesian model selection process requires the definition of a specific model prior.

In the absence of information about the models, the uniform prior is the most common prior assumed in a Bayesian model selection problem. However, this prior does not consider the structure of the cluster problem and other alternative model priors are possible such as considering the uniform distribution in each of the hierarchy levels of the clusters, or even considering the Poisson-Intrinsic prior proposed by Casella et al. [13] which penalizes the number of clusters. Although all these priors can be considered as not informative as they do not add new information to that provided by the data, the prior probabilities assigned to each partition vary. Due to the sensitivity of the estimation of the meta-parameter to the chosen cluster, we analyze in this paper the characteristics of these model priors and in which cases each one my be preferable.

A frequentist evaluation is carried out with simulated data, where different number of studies, sample sizes, and real clusters are considered. The rest of the paper is organized as follows. The binomial Bayesian model is presented in Section 2, where the Bayesian procedure for clustering the samples and the likelihood of the meta-parameter are also given. In this section, the four model priors to be compared will be presented. The simulated data and the results of the frequentist validation are described in Section 3. Section 4 provides one illustrative example with a real dataset. Finally, Section 5 summarizes the main conclusions drawn and presents some concluding remarks.

2. The Bayesian Binomial Model

Assume a meta-analysis involving k studies that provide k independent discrete samples which follow a binomial distribution $\{\mathrm{Bin}\left(x_i\right|n_i,{\theta }_i),i=1,\dots ,k\}$, where ${\theta }_i$ represents the treatment effectiveness, $n_i$ the number of patients, and $x_i$ the number of successful treatments, conditional on the study i. We assume weak prior information on the conditional treatment effectiveness ${\theta }_i$. Accordingly, the uniform prior $\mathrm{Unif}\left({\theta }_i\right|0,1)$ is used [14,15]. The Bayesian sampling model ($M_i$) for $i=1,\dots ,k$ studies is then given by

$$M_i:\left\{\mathrm{Bin}\left(x_i\right|n_i,{\theta }_i),\pi \left({\theta }_i\right)\propto {\mathbf{1}}_{(0,1)}\left({\theta }_i\right)\right\},$$

(2)

where

$$\mathrm{Bin}\left(x_i\right|n_i,{\theta }_i)=\left({\displaystyle \genfrac{}{}{0pt}{}{n_i}{x_i}}\right){\theta }_i^{x_i}{(1-{\theta }_i)}^{n_i-x_i},x_i=0,1,\dots ,n_i,$$

(3)

and ${\mathbf{1}}_A$ is the indicator function that takes a value of 1 to all elements of A, and 0 elsewhere.

The meta-model is defined by a patient in a virtual study, which is not affected by between-study variability. The variable x is a binary latent variable and the meta-parameter $\theta$ defines the probability of success for this virtual patient. The distribution of this meta-variable x is the Bernoulli meta-model $\mathrm{Ber}\left(x\right|\theta )$, where the meta–parameter $\theta$ represents the true (unconditional) treatment effect. The objective Bayesian meta-model M is then given by

$$M:\left\{Pr\left(x\right|\theta )={\theta }^x{(1-\theta )}^{1-x},\pi \left(\theta \right)={\mathbf{1}}_{(0,1)}\left(\theta \right)\right\}.$$

(4)

2.1. The Linking Distribution

A distribution $\pi \left({\theta }_i\right|\theta )$ is needed to link the experimental parameters ${\theta }_i$ and the meta-parameter $\theta .$ This linking distribution should ensure there is coherence between the conditional and marginal distributions of the experimental parameters and the meta-parameter. This requires that the corresponding bivariate distribution belongs to the class of bivariate distributions with given marginals. The class of bivariate distributions solving this problem is called the Frèchet class:

$${\int }_0^1\pi ({\theta }_i,\theta )d{\theta }_i=\pi \left(\theta \right)\mathrm{and}{\int }_0^1\pi ({\theta }_i,\theta )d\theta =\pi \left({\theta }_i\right).$$

(5)

Following Moreno et al. [9], a candidate $\pi ({\theta }_i,\theta )$ is constructed using the intrinsic priors for model selection [16]. The conditional intrinsic linking distributions $\{{\pi }^I\left({\theta }_i\right|\theta ,t),t=1,2,\dots \}$ arises from the model comparison between the meta-model M and the experimental model $M_i$. For any positive integer t, the intrinsic method gives the conditional intrinsic prior as a Beta-Binomial mixture,

$${\pi }^I\left({\theta }_i\right|\theta )=\sum _{z=0}^t\mathrm{Bin}\left(z\right|t,\theta )\times \mathrm{Beta}\left({\theta }_i\right|z+1,t-z+1).$$

(6)

In general, the bivariate intrinsic prior ${\pi }^I({\theta }_i,\theta |t)$ enjoys two interesting properties. One is that it belongs to the Fréchet class with marginals $\pi \left({\theta }_i\right)$ and $\pi \left(\theta \right)$ following a uniform distribution. A second one is that the concentration degree of ${\pi }^I\left({\theta }_i\right|\theta ,t)$ around $\theta$ is controlled by the training sample size t, the larger the t the larger the concentration degree. Note that the correlation coefficient between ${\theta }_i$ and $\theta$ is $\rho =t/(t+1)$. In practice, the hyperparameter t is fixed, assuming a large enough correlation between ${\theta }_i$ and $\theta$. We assume in our examples a correlation of 0.98, which implies that $t=48$. Hence, for the sake of simplicity in notation, we refer to the linking distribution ${\pi }^I\left({\theta }_i\right|\theta )$ rather than ${\pi }^I\left({\theta }_i\right|\theta ,t)$.

As it is assumed that ${\theta }_i,i=1,\dots ,k$ are conditional independent given $\theta$, the linking distribution of ${\theta }_1,\dots ,{\theta }_k$ conditional on $\theta$ is given by

$${\pi }^I({\theta }_1,\dots ,{\theta }_k|\theta )=\prod _{i=1}^k{\pi }^I\left({\theta }_i\right|\theta ).$$

(7)

2.2. Clusters

The previous section assumes that there are k experimental parameters ${\theta }_i,i=1,\dots ,k$ to be estimated. However the dimension of the experimental model can be reduced if some of the ${\theta }_i$’s are equal. Following Moreno et al. [10], model estimation in this parametric setting is a problem of clustering the parameters ${\theta }_1,\dots ,{\theta }_k$, based on the samples $x_1,\dots ,x_k$ from the experiments. We first define what is meant by cluster. The samples $x_i$ and $x_j,$$i\ne j$, from $f\left(x\right|{\theta }_i,n)$ and $f\left(x\right|{\theta }_j,n),$ respectively, are said to be in the same cluster if ${\theta }_i={\theta }_j.$ The between-sample heterogeneity is then determined by the number of clusters and by the location of the samples $(x_1,n_1),\dots ,(x_k,n_k)$ within these clusters.

To cluster the samples we adopt the product partition model approach proposed by Barry and Hartigan [12], together with a Bayesian model selection procedure based on Bayes factors for the intrinsic priors for the model parameters.

We employ the following notations and expressions in the meta-analysis conducted [13]. For a given p, we define a partition of the samples into p clusters by the vector ${\mathbf{r}}_p=(r_1,\dots ,r_k)$, where $r_i,i=1,\dots ,k$, is an integer between 1 and p denoting the cluster to which $x_i$ is assigned. Figure 1 shows the possible clustering structures for $k=3$, and their corresponding ${\mathbf{r}}_p$.

2.3. The Likelihood of $\theta$ for a Particular Partition

The likelihood of $\theta$ will depend on the partition of the samples. Given a partition ${\mathbf{r}}_p=(r_1,\dots ,r_k)$, the sampling distribution of $x=(x_1,\dots ,x_k)$ given in (2) is

$$f\left(x\right|p,{\mathbf{r}}_p,{\theta }_p)=\prod _{j=1}^p\left({\displaystyle \genfrac{}{}{0pt}{}{m_j}{s_j}}\right){\theta }_j^{s_j}{(1-{\theta }_j)}^{m_j-s_j},$$

(8)

where ${\theta }_p=({\theta }_1,\dots ,{\theta }_p)$ is an unknown parameter of dimension p, the component ${\theta }_j$ in (8) corresponds to $r_i=j$, and $m_j={\sum }_{i:r_i=j}{n}_i$ and $s_j={\sum }_{i:r_i=j}{x}_i$ are the sample size and number of success of the cluster j. The likelihood of a particular partition, for example, ${\mathbf{r}}_2=(1,2,2)$, is

$$f\left(x\right|2,{\mathbf{r}}_2=(1,2,2),{\theta }_2)=\left({\displaystyle \genfrac{}{}{0pt}{}{n_1}{x_1}}\right){\theta }_1^{x_1}{(1-{\theta }_1)}^{n_1-x_1}\left({\displaystyle \genfrac{}{}{0pt}{}{n_2+n_3}{x_2+x_3}}\right){\theta }_2^{x_2+x_3}{(1-{\theta }_2)}^{(n_2+n_3)-(x_2+x_3)}.$$

The heterogeneity partition ${\mathbf{r}}_k=(1,2,3,\dots ,k)$ has the corresponding likelihood function given by

$$f\left(x\right|k,{\mathbf{r}}_k,{\theta }_k)=\prod _{i=1}^k\left({\displaystyle \genfrac{}{}{0pt}{}{n_i}{x_i}}\right){\theta }_i^{x_i}{(1-{\theta }_i)}^{n_i-x_i},$$

and the homogeneity partition ${\mathbf{r}}_1=(1,1,\dots ,1)$ has the corresponding likelihood function given by

$$f\left(x\right|1,{\mathbf{r}}_1,{\theta }_1)=\left({\displaystyle \genfrac{}{}{0pt}{}{{\sum }_{i=1}^k{n}_i}{{\sum }_{i=1}^k{x}_i}}\right){\theta }_1^{{\sum }_{i=1}^k{x}_i}{(1-{\theta }_1)}^{{\sum }_{i=1}^k(n_i-x_i)}.$$

Now, integrating out ${\theta }_p$ with the intrinsic prior ${\displaystyle \pi \left({\theta }_p\right|p,{\mathbf{r}}_p)=\int {\pi }^I({\theta }_1,\dots ,{\theta }_p|\theta ){\mathbf{1}}_{(0,1)}}$$\left(\theta \right)d\theta$, we obtain the likelihood of $\theta$, conditional on the cluster model $(p,{\mathbf{r}}_{\mathbf{p}})$ given by

$$\begin{array}{c}\hfill f\left(x\right|p,{\mathbf{r}}_p,\theta )=\prod _{j=1}^p{\int }_0^{1}f\left(x\right|p,{\mathbf{r}}_p,{\theta }_p)\pi \left({\theta }_j\right|\theta )d{\theta }_j=\\ \hfill ={(1+t)}^p{(1-\theta )}^{tp}\prod _{j=1}^p\frac{\mathsf{\Gamma }(s_j+1)\mathsf{\Gamma }(m_j+t-s_j+1)}{\mathsf{\Gamma }(m_j+t+2)}{}_3{F}_2({\mathbf{a}}_j,{\mathbf{b}}_j,\frac{\theta }{\theta -1}),\end{array}$$

(9)

where ${}_3{F}_2(\mathbf{v},\mathbf{w},z)$ denotes the generalized hypergeometric function with argument z and vector parameters $\mathbf{v}$ and $\mathbf{w}$ of dimensions 3 and 2. In this case, the parameters ${\mathbf{a}}_j=(-t,-t,s_j+1)$ and ${\mathbf{b}}_j=(1,-m_j-t+s_j)$ are related with the number of 1’s and 0’s in cluster $j,$ respectively.

2.4. The Likelihood of $\theta$ the Prior Distribution over the Partitions

To derive the likelihood function of $\theta$ we need to integrate out (9) with respect to a discrete prior on $(p,{\mathbf{r}}_p)$. The (unconditional) likelihood of $\theta$ for the data $x$ is given by

$$f\left(x\right|\theta )=\sum _{p=1}^k\left(\sum _{{\mathbf{r}}_p}f\left(x\right|p,{\mathbf{r}}_p,\theta )\pi (p,{\mathbf{r}}_p|k)\right).$$

(10)

The prior distribution on the partitions $\pi (p,{\mathbf{r}}_p|k)$ plays an important role in the estimation of the parameter $\theta$ [13]. We consider here four priors on $(p,{\mathbf{r}}_p)$ which are motivated by reasonable but different assumptions. The four selected prior distribution assume the absence of prior information about the models, but ranges from the assignment of high prior probability at the boundary $p=1$ and $p=4$ (homogeneity and heterogeneity structures, respectively) to other intermediate situations that moderate the a priori assignment to these two clusters or considers them all equally probable.

• The Uniform prior.
  The first prior proposed is the uniform prior (U), which gives the same probability to every model, that is,

  $${\displaystyle {\pi }^U(p,{\mathbf{r}}_p|k)=\frac{1}{{\mathcal{B}}_k},}$$

  (11)

  where ${\mathcal{B}}_k$, the Bell number, is the number of subsets a set of size k can be partitioned into. Figure 2 shows the prior probabilities for each partition when four studies are considered. In this example, the Bell number is 15. This choice does not take into account the level of complexity of each partition.
• The Hierarchical Uniform Prior with 2 levels (HU2).
  As recommended by Casella et al. [13], a hierarchical uniform prior can be appropriate to take into account the different levels of complexity of the partitions. This prior distribution distinguishes two levels of complexity in the partitions. The first level is given by the number of clusters p in which the k samples are grouped. The second level will be given by the number of possible partitions of the k samples into p clusters. Let ${\mathcal{R}}_p$ represent this set of partitions into p clusters, which we call the cluster class. The number of partitions in ${\mathcal{R}}_p$ is given by the Stirling number of the second kind $\mathcal{S}(k,p)$ and can be written as

  $${\displaystyle \mathcal{S}(k,p)=\sum _{1\le k_1\le \dots \le k_p}\left({\displaystyle \genfrac{}{}{0pt}{}{k_1+\dots +k_p}{k_1\cdots k_p}}\right)\frac{1}{R(k_1,\dots ,k_p)},}$$

  (12)

  where $\left({\displaystyle \genfrac{}{}{0pt}{}{k_1+\dots +k_p}{k_1\cdots k_p}}\right)$ is the multinomial coefficient and $R(k_1,\dots ,k_p)={\prod }_{i=1}^k\left[{\sum }_{j=1}^{p}I(k_j=i)\right]!$ corrects the count by considering the redundant strings corresponding to the vector $(k_1,\dots ,k_p)$. For instance, to calculate the Stirling number $\mathcal{S}(4,2)$, there are two possible vectors $(k_1,k_2)$, the vector $(1,3)$, and the vector $(2,2)$, and the Stirling number would be

  $${\displaystyle \mathcal{S}(4,2)=\left({\displaystyle \genfrac{}{}{0pt}{}{4}{1,3}}\right)\frac{1}{R(1,3)}+\left({\displaystyle \genfrac{}{}{0pt}{}{4}{2,2}}\right)\frac{1}{R(2,2)}=\frac{4!}{1!3!}\frac{1}{1!1!}+\frac{4!}{2!2!}\frac{1}{2!}=4+3=7,}$$

  (13)

  which is the number of possible partitions for $p=2$ and $k=4$.
  The hierarchical uniform distribution for 2 levels will be given by the decomposition

  $${\displaystyle {\pi }^{HU2}(p,{\mathbf{r}}_p|k)=\pi \left({\mathbf{r}}_p\right|p,k)\pi \left(p\right|k)=\frac{1}{\mathcal{S}(k,p)}\frac{1}{k}.}$$

  (14)
  Figure 3 shows the prior probabilities for each partition using the hierarchical uniform prior with 2 levels with 4 studies. Note that this hierarchical distribution assigns a higher prior probability to cases of homogeneity and heterogeneity.
• The Hierarchical Uniform Prior with 3 levels (HU3).
  Following Casella et al. [13] and Moreno et al. [10], the prior specification for $(p,{\mathbf{r}}_p)$ can be decomposed in three levels:

  $${\displaystyle {\pi }^{HU3}(p,{\mathbf{r}}_p|k)=\pi (p,{\mathbf{r}}_p|{\mathcal{R}}_{p;k_1,\dots ,k_p},k)\pi \left({\mathcal{R}}_{p;k_1,\dots ,k_p}\right|p,k)\pi \left(p\right|k).}$$

  (15)
  Unlike the previous prior distribution, the hierarchical uniform prior with 3 levels considers the number of ways the integer k can be partitioned into p clusters. We will call it the number of configuration classes within each ${\mathcal{R}}_p$ and it will be denoted by $b(k,p)$. In our illustrative example with $k=4$, this value is equal to 1 for $p=1,3,4$ ($b(4,1)=b(4,3)=b(4,4)=1)$, and only for the cluster class $p=2$ there are two configuration classes, corresponding to the configurations $x|xxx$ and $xx|xx$, so $b(4,2)=2$.
  The hierarchical uniform prior with 3 levels is given by the expression

  $${\pi }^{HU3}(p,{\mathbf{r}}_p|k)=\frac{k_1!·\dots ·k_p!}{k!}\frac{R(k_1,\dots ,k_p)}{b(k,p)}\frac{1}{k}.$$

  (16)
  Figure 4 shows the prior probabilities for each partition using the hierarchical uniform prior with 3 levels and 4 studies.
• The Hierarchical Poisson Prior with 3 levels (HP3).
  Casella et al. [13] argue that when analyzing a cluster problem of a sample size k, the extreme case of having k clusters should be given a priori a smaller probability than that given to any other case. Extending this argument for any k, it might be reasonable that the prior distribution on the number of clusters $\pi \left(p\right|k)$ might be a truncated Poisson distribution $\mathcal{P}\left(p\right|\lambda )$, where $\lambda$ is an unknown parameter. We can assume an intrinsic prior ${\pi }^I\left(\lambda \right|{\lambda }_0=1)$ for $\lambda$, constructed by testing the Poisson null hypothesis $H_0:\lambda ={\lambda }_0$ versus $H_1:\lambda \in R^{+}$ [17],

  $${\pi }^I\left(\lambda \right|{\lambda }_0=1)=\frac{{\lambda }^{-1/2}}{\mathsf{\Gamma }(1/2)}{e}^{-(\lambda +1)}{}_0{F}_1(1/2,\lambda ),$$

  (17)

  where ${}_0{F}_1(1/2,\lambda )$ denotes the confluent hypergeometric function. The reason for taking ${\lambda }_0=1$ is that the one cluster model is the reference model throughout the analysis. The resulting marginal intrinsic distribution for p is

  $${\pi }^I\left(p\right|k)=\frac{m^I\left(p\right)}{{\sum }_{p=1}^k{m}^I\left(p\right)},p=1,\dots ,k,m^I\left(p\right)={\int }_0^{+\infty }\frac{{\lambda }^p{e}^{-\lambda }}{p!}{\pi }^I\left(\lambda \right|{\lambda }_0=1)d\lambda .$$

  (18)
  We cannot assume a Poisson distribution for the other two levels of the hierarchical structure because there is no a clear order in relation to complexity. For this reason, a uniform distribution is assumed for the other two levels. The hierarchical Poisson prior will be given by

  $${\pi }^{HP3}(p,{\mathbf{r}}_p|k)=\frac{k_1!·\dots ·k_p!}{k!}\frac{R(k_1,\dots ,k_p)}{b(k,p)}{\pi }^I\left(p\right|k).$$

  (19)
  Figure 5 shows the prior probabilities for each partition using the hierarchical Poisson prior with 4 studies. The prior probability for the homogeneity cluster is more than four times higher than the prior probability of the heterogeneity case.

Finally, from (10) and the priors defined in (11), (14), (16) and (19), the (unconditional) likelihood of $\theta$ for the data $x$ is given by

$$f\left(x\right|\theta )=\sum _{p=1}^k\left(\sum _{{\mathbf{r}}_p}f\left(x\right|p,{\mathbf{r}}_p,\theta )\pi (p,{\mathbf{r}}_p)\right).$$

(20)

2.5. Bayesian Model Averaging in the Meta-Analysis

The BMA approach to meta-analysis involves averaging over all the possible models (heterogeneity structures or partitions) when making inferences about the parameter of interest $\theta$.

In this case, the posterior probabilities correspond to those of any heterogeneity structure given by a pair $(p,{\mathbf{r}}_p)$, which is represented by

$$Pr(p,{\mathbf{r}}_p|x,k)=\frac{m_{{\mathbf{r}}_p}\left(x\right|p,{\mathbf{r}}_p)\pi (p,{\mathbf{r}}_p)|k}{{\sum }_{p=1}^k\left({\sum }_{{\mathbf{r}}_p}{m}_{{\mathbf{r}}_p}\left(x\right|p,{\mathbf{r}}_p)\pi (p,{\mathbf{r}}_p|k)\right)},$$

(21)

where ${\displaystyle m_{{\mathbf{r}}_p}\left(x\right|p,{\mathbf{r}}_p)=\int f\left(x\right|p,{\mathbf{r}}_p,{\theta }_p)\pi \left({\theta }_p\right|p,{\mathbf{r}}_p)d{\theta }_p}$ is the marginal of the data $x$ conditional on model $(p,{\mathbf{r}}_p),$ with $f\left(x\right|p,{\mathbf{r}}_p,{\theta }_p)$ and $\pi \left({\theta }_p\right|p,{\mathbf{r}}_p)$. These posterior model probabilities $Pr(p,{\mathbf{r}}_p|x)$ are the weights for the meta inference.

The posterior distribution for the parameter of interest $\theta$ becomes

$$\pi \left(\theta \right|x)=\sum _{{\mathbf{r}}_p}\pi \left(\theta \right|x,p,{\mathbf{r}}_p)Pr(p,{\mathbf{r}}_p|x,k),$$

(22)

where

$$\pi \left(\theta \right|x,p,{\mathbf{r}}_p)=\frac{f\left(x\right|p,{\mathbf{r}}_p,\theta )}{{\int }_0^{1}f\left(x\right|p,{\mathbf{r}}_p,\theta )d\theta },0<\theta <1.$$

(23)

The posterior distribution in (22) is computed numerically using Wolfram Mathematica (see code in the supplementary material Section).

3. Simulated Data and Frequentist Validation

3.1. Simulated Data

This section presents the simulated data used in the frequentist validation. The data have been simulated from binomial distributions, where the number of studies included in the meta-analysis (k), the partition for the data (${\mathbf{r}}_p$), and the sample size within each study (n) vary between simulations.

The values for the number of studies (k) in the meta-analysis are 3, 5. Other greater values of k are obviously possible. For instance, we developed the case $k=8$ (see supplementary material Section) where the conclusions obtained are similar to that in the cases $k=3$ and 5. Therefore, in order to facilitate the reading of the

Table 1. Parameters for the simulation data.

True Model (${\mathbf{r}}_{\mathit{p}}$)	Parameters (${\mathit{\theta }}_{\mathit{i}}$’s)	Sample Sizes (${\mathit{n}}_{\mathit{i}}$)
$k=3$
$1,1,1$	$(0.5,0.5,0.5)$	$(10,30,100,300)$
$1,1,2$	$(0.5,0.5,0.2)$	$(10,30,100,300)$
$1,2,3$	$(0.7,0.5,0.2)$	$(10,30,100,300)$
$k=5$
$1,1,1,1,1$	$(0.5,0.5,0.5,0.5,0.5)$	$(10,30,100,300)$
$1,1,1,2,2$	$(0.5,0.5,0.5,0.2,0.2)$	$(10,30,100,300)$
$1,1,2,2,3$	$(0.7,0.7,0.5,0.5,0.2)$	$(10,30,100,300)$
$1,2,3,4,5$	$(0.9,0.7,0.5,0.3,0.1)$	$(10,30,100,300)$

we only present the cases $k=3$ and $k=5$. These numbers of studies are manageable to do this simulation exercise. In this respect, Davey et al. [18] conducted an extensive review of the Cochrane Database of Systematic Reviews (CDSR) and pointed out that just under $75\%$ of the meta-analyses contained five or fewer studies.

The sample size within each study is also a crucial parameter of the simulation. For simplicity we assume a common sample size for the k studies and this sample size takes values of 10, 30, 100, and 300. Finally different “true” partitions are considered for each k, where the heterogeneity and homogeneity cases are always included and one or two intermediate cases are also analyzed. Table 1 shows the parameters of the simulated data.

The ${\theta }_i$ parameters used in the simulation are sufficiently disparate between clusters to expect that with moderate sample sizes, the Bayesian selection process will be able to detect the true model. For all simulation scenarios, 500 simulations were performed. To analyze the properties of the prior distributions over posterior probabilities of the partitions we show the proportion of times the true model is found as the model with the highest posterior probability and the mean posterior probability in those cases in which the true model is found as the most probable. We also show the number of cases the homogeneity cluster (${\mathbf{r}}_1=(1,1,\dots ,1)$) and the heterogeneity case (${\mathbf{r}}_k=(1,2,3,\dots ,k)$) are found as the most probable model.

3.2. Frequentist Evaluation

Figure 6 and Figure 7 show the results of the frequentist validation for the case $k=3$ and true partitions ${\mathbf{r}}_1=(1,1,1),{\mathbf{r}}_2=(1,1,2)$ and ${\mathbf{r}}_3=(1,2,3)$ corresponding to a situation of homogeneity, intermediate heterogeneity, and heterogeneity, respectively. As expected, for the true case of homogeneity ${\mathbf{r}}_1$, the uniform prior shows worse performance as it is the only one that does not assume an a priori preference for the homogeneity. However, the results show how the uniform prior reaches a proportion of correct choices close to $70\%$ with sample sizes greater than 100. The mean posterior probabilities reach values greater than $30\%$ and $40\%$ for sample sizes of 100 and 300, respectively. Observe that $b(3,p)=1,1\le p\le 3,$ thus the results from HU2 and HU3 priors are identical. The hierarchical Poisson prior, which assigns a higher prior probability to the homogeneity case, reaches a proportion of correct choices higher than $95\%$ even with sample sizes of 10, and posterior probabilities higher than $50\%$. All the prior distributions show a good performance for high sample sizes.

When the true partition is an intermediate one, the results vary. Figure 6 shows the results for the true case ${\mathbf{r}}_2=(1,1,2)$. The uniform prior shows the best performance although the proportion of right choices is smaller than $50\%$ for $n=10$. The mean posterior probability reaches a value of $80\%$ with a sample size of 300. With the hierarchical Uniform priors the true model is never chosen as the most probable for a sample size of $n=10$. In this case, the mean of the posterior probability for the true partition when it is found as the most probable does not exist and it is shown as 0 in the figure. For small sample sizes these prior distributions found the heterogeneity case as the most probable (Figure 7). The hierarchical Uniform priors only achieve the $50\%$ of right choices for a sample size of 300. The hierarchical Poisson prior improves the behavior of the hierarchical Uniform priors although for a small sample size of $n=10$, it chooses the homogeneity case more than $85\%$ of the simulations (Figure 7).

When the true model is the heterogeneity case, i.e., ${\mathbf{r}}_3=(1,2,3)$, the hierarchical Uniform priors (HU2 and HU3) show the best performance even with small sample sizes. Surprisingly, the uniform prior shows worse results than those of the hierarchical Poisson prior. This can be explained by the higher prior probabilities assigned to intermediate partitions by the uniform prior, which hinder the identification of the heterogeneity case as the true model. The proportion of right choices and the mean posterior probabilities for the true model are near to $100\%$ for all the prior models when the sample size is 300.

Figure 8 and Figure 9 show the results of the frequentist validation for the case $k=5$ and true partitions ${\mathbf{r}}_1=(1,1,1,1,1)$, homogeneity case, ${\mathbf{r}}_2=(1,1,1,2,2)$ and ${\mathbf{r}}_3=(1,1,2,2,3),$ intermediate situations and ${\mathbf{r}}_5=(1,2,3,4,5)$, heterogeneity case.

The analysis of the homogeneity case with $k=5$ shows a bad performance of the uniform prior, even worse than observed for $k=3$. The proportion of right choices only exceed $50\%$ for a sample size of 300. As it is shown in the Figure 9, the uniform prior found as the most probable model some intermediate models as the proportion of cases in which the heterogeneity case is chosen is 0. As it was found with $k=3$, the hierarchical Poisson prior shows a better performance than the HU2 and HU3 which becomes similar as the sample size increases. Some results obtained for the HU2 and HU3 priors are quite similar, showing an overlapping behavior in some cases.

Once again, the analysis of the intermediate cases with $k=5$ shows a similar behavior to that observed with $k=3$. The uniform prior reaches a higher proportion of correct choices, although for the case of two clusters $(p=2)$, the hierarchical Poisson improves it for sample sizes greater than 100. With a moderate number of clusters ($p=3$), all the prior models show difficulties to choose the true model with small sample sizes. In the case of the hierarchical Poisson prior, it chooses the homogeneity case for small sample sizes, while the uniform prior chooses other intermediate models (Figure 9). The hierarchical uniform priors never choose the true model with sample sizes smaller than 300, showing preference for the heterogeneity case.

For the heterogeneity case ${\mathbf{r}}_5=(1,2,3,4,5)$, the proportion of right choices show a U-shape for all model priors except the Uniform prior. For small sample sizes, the greater prior probability assigned to the extreme cases leads to a preference for the heterogeneity case (the homogeneity case is never chosen as it is shown in Figure 9). As the sample size increases, the importance of the prior information is reduced and other intermediate partitions are chosen, probably due to the small difference in the true probability of success between the 5 studies (see Table 1). Finally, with a sample size of 300, all model prior distributions choose the true model.

An additional analysis for the case $k=8$ is shown in the supplementary material Section. The results are very similar to those obtained for the case $k=5$.

4. An Illustrative Example with Real Data

In this section, we show an illustrative example with real data to analyze the impact of the prior models over the estimation of the meta-parameter $\theta$. With the objective to determine the effectiveness of granulocyte transfusions compared to no granulocyte transfusions for treating infections in patients with neutropenia or disorders of neutrophil function in reducing mortality, Stanworth et al. [19] conducted a meta-analysis. The dataset in

Table 2. Data in Stanworth et al. [19].

Study	Treatment
	Events	Total
Herzig 1977 ($x_1$)	1	16
Higby 1975 ($x_2$)	2	17
Scali 1978 ($x_3$)	0	13
Vogler 1977 ($x_4$)	7	17

is extracted from Stanworth et al. [19] and corresponds to the mortality analysis in four studies ($k=4$, subgroup analysis for studies transfusing greater than $1\times {10}^{10}$ granulocytes at days 20–22) for granulocyte transfusions for treating infections in patients with neutropenia or neutrophil dysfunction treated with transfusion (Treatment).

This is a good example to apply the model proposed by Moreno et al. [9] as the number of cases is small and there are even no cases in one study. For $k=4$, there are 15 possible partitions and the estimation of the meta-parameter $\theta$ will depend on the partition considered. Figure 10 shows the posterior mean conditioned on each partition. The posterior mean varies from the $0.1929$ obtained for the partition ${\mathbf{r}}_2=(1,1,1,2)$ to $0.1344$ for the partition ${\mathbf{r}}_3=(1,2,3,2)$.

The four prior models are applied to this dataset. The prior probabilities assigned to each partition can be shown in Figure 2, Figure 3, Figure 4 and Figure 5. To include into the analysis the model uncertainty, the posterior distributions for $\theta$ are all averaged in the BMA posterior distribution, and this BMA posterior distribution depends on the model priors assumed.

Table 3. Top cluster models in Stanworth et al. for treatment data.

Prior # 1: Uniform		Prior # 2: HU with 2 levels		Prior # 3: HU with 3 Levels		Prior # 4: HP with 3 Levels
Cluster Model	Post. Prob.	Cluster Model	Post. Prob.	Cluster Model	Post. Prob.	Cluster Model	Post. Prob.
$x_1{x}_2{x}_3|x_4$	0.19	$x_1|x_2\left|x_3\right|x_4$	0.37	$x_1|x_2\left|x_3\right|x_4$	0.37	$x_1|x_2\left|x_3\right|x_4$	0.21
$x_1{x}_3\left|x_2\right|x_4$	0.14	$x_1{x}_2{x}_3|x_4$	0.11	$x_1{x}_2{x}_3|x_4$	0.10	$x_1{x}_2{x}_3{x}_4$	0.18
$x_1{x}_2\left|x_3\right|x_4$	0.11	$x_1{x}_3\left|x_2\right|x_4$	0.09	$x_1{x}_3\left|x_2\right|x_4$	0.09	$x_1{x}_2{x}_3|x_4$	0.14
$x_1\left|x_2{x}_3\right|x_4$	0.11	$x_1{x}_2{x}_3{x}_4$	0.08	$x_1{x}_2{x}_3{x}_4$	0.08	$x_1{x}_3\left|x_2\right|x_4$	0.8
$x_1|x_2\left|x_3\right|x_4$	0.09	$x_1{x}_2\left|x_3\right|x_4$	0.08	$x_1{x}_2\left|x_3\right|x_4$	0.08	$x_1{x}_3|x_2{x}_4$	0.08
the rest	<0.09	the rest	<0.07	the rest	<0.07	the rest	<0.07
BMA estimates of the meta-parameter $\theta$
Posterior mean: 0.164		Posterior mean: 0.159		Posterior mean: 0.159		Posterior mean: 0.163
$95\%$ HDI: 0.050–0.317		$95\%$ HDI: 0.049–0.312		$95\%$ HDI: 0.049–0.311		$95\%$ HDI: 0.048–0.327

shows the top cluster models for the four model priors.

Top cluster models and their posterior probabilities are sensitive to the model prior. For the uniform prior model, the most probable model is $\left\{x_1{x}_2{x}_3\right|x_4\}$, with a posterior probability of $0.19$. However, this model is found as the second most probable model for the hierarchical uniform priors, and reaches the third position for the Poisson prior. These last models found the heterogeneity case as the most probable model.

Table 2 shows a different mortality rate between studies, where study 4 stands out with a mortality rate of $41.18\%$, while the mortality rate for the other studies do not exceed $12\%$. The uniform prior is the only model capable of detecting this structure as the most probable model. The other prior models discard the homogeneity case but cannot detect the intermediate model, showing preference for the heterogeneity case. In this example, the uniform prior would be the best choice.

This analysis also points out the importance of the BMA estimation as it allows the model uncertainty to be included in the estimation of the meta-parameter. As can be seen, the estimation of the meta-parameter by BMA is less sensitive to the choice of the prior distribution for the models, ranging from 0.159 for the hierarchical Uniform priors to 0.164 for the Uniform prior.

5. Conclusions

Bayesian methods for the design, analysis, and synthesis of clinical trials have been developed in several areas including meta-analysis where the structure of the between-sample heterogeneity is essential in estimating the meta-parameter. As part of the design of the Bayesian framework, we address the question from a different standpoint, arguing that between-sample heterogeneity is a clustering problem and that model uncertainty can be incorporated into the inference using a Bayesian procedure. Under this procedure, the posterior probabilities of the cluster models are computed and the definition of the prior distribution over the models takes on special importance.

Meta-analysis for binary data is an increasingly used tool for estimating the effectiveness of a certain treatment. Meta-analysis for binary data presents interesting statistical challenges that have been addressed in the literature, such as the presence of zeros, that make it difficult to apply logit transformations to the data [20,21]. The definition of an objective Bayesian meta-analysis for binary data that does not require transformations to the data represented an advance in literature [9].

The objectivity of this analysis is given by the prior distribution assumed for the experimental parameters (${\theta }_i$) and the meta-parameter ($\theta$) [22]. However, when the between-sample heterogeneity is considered in the analysis as a problem of clustering the experimental parameters (${\theta }_i$), the objectivity remains in doubt since the Bayesian model selection requires the definition of the prior distributions over the models. The hierarchical structure of the clusters does not allow to conclude that the Uniform prior distribution is the best or unique option. Moreno et al. [10] proposed to use the hierarchical Uniform prior with three levels, but other options are possible.

In this paper we analyze the properties of four model priors assuming the absence of prior information about the cluster model: the Uniform prior, the hierarchical Uniform prior with two and three levels and the hierarchical Poisson prior. There are other priors proposed by the literature for the problem of clustering, such as the Ewens–Pitman prior [23,24,25] or the Jensen–Liu prior [26]. However, these prior distributions require the assessing of a hyperparameter that reflects the a priori information about the models.

A first conclusion achieved from the frequentist validation is that none of the prior distributions is completely non-informative. The posterior probabilities for the models are very sensitive to the model priors, even with moderately large sample sizes. A useful guideline for daily practice could be as follows. If you consider that the homogeneity case is probable, the hierarchical Poisson prior for small samples sizes, or the hierarchical Uniform prior for moderately large sample sizes are the best options. If you consider that the heterogeneity case is probable, the hierarchical Uniform priors are preferable. Finally, if you consider that the real cluster is not the homogeneity or heterogeneity cases, the Uniform prior can be used for small number of studies and sample sizes.

A second conclusion is that carrying out a meta-analysis based on a single partition (even if it is the partition with the maximum posterior probability) can obtain biased results as it ignores a very important part of the uncertainty around the estimation of the meta-parameter. The BMA procedure offers a natural way to incorporate this model uncertainty into the estimation of $\theta$ [27].

As the BMA procedure implies estimating the meta-parameter for all possible partitions, computational difficulties arise when the number of studies k is moderately large due to time required for estimation. In that case, a set of good cluster models can certainly by found using a stochastic algorithm [13].