Computation of the Distribution of the Sum of Independent Negative Binomial Random Variables

Abstract

The distribution of the sum of negative binomial random variables has a special role in insurance mathematics, actuarial sciences, and ecology. Two methods to estimate this distribution have been published: a finite-sum exact expression and a series expression by convolution. We compare both methods, as well as a new normalized saddlepoint approximation, and normal and single distribution negative binomial approximations. We show that the exact series expression used lots of memory when the number of random variables was high (>7). The normalized saddlepoint approximation gives an output with a high relative error (around 3–5%), which can be a problem in some situations. The convolution method is a good compromise for applied practitioners, considering the amount of memory used, the computing time, and the precision of the estimates. However, a simplistic implementation of the algorithm could produce incorrect results due to the non-monotony of the convergence rate. The tolerance limit must be chosen depending on the expected magnitude order of the estimate, for which we used the answer generated by the saddlepoint approximation. Finally, the normal and negative binomial approximations should not be used, as they produced outputs with a very low accuracy.

1. Introduction

The negative binomial (NB) distribution is a discrete probability distribution that models counts [1]. It widely used in statistics, from statistics of accidents [2] to animal counts [3]. The NB distribution can be used to describe the distribution of the number of successes or failures. Suppose that there is a sequence of independent Bernoulli trials, each trial having two potential outcomes called “success” and “failure”. The probability of success is $p$ and of failure $q=1-p$. We observe this sequence until a predefined number, $r$, of successes has occurred. Then, the random number of failures has the NB distribution $X ~ NB\left(r; p\right)$ with density $P\left(X=x\right)$, x being a particular realization of X:

$$P\left(X=x\right)=\frac{\left(\mathrm{x}+r-1\right)!}{x!\left(r-1\right)!} p^r{\left(1-p\right)}^x$$

(1)

with 0 < p < 1, x and r being integer > 0. The mean is $\mu =r \left(1-p\right)/p$.

The moment-generating function of the NB distribution is:

$$M\left(t\right)={\left(\frac{q}{1-p e^t}\right)}^r$$

(2)

An alternative parametrization $X ~ NB\left(\mu , \theta \right)$ can also be derived from assuming that the mean parameter of a Poisson distribution has a gamma distribution:

$$P\left(X=x\right)=\frac{\mathsf{\Gamma }\left(\mathrm{x}+\theta \right)}{x!\mathsf{\Gamma }\left(\theta \right)} {\left(\frac{\theta }{\mu +\theta }\right)}^{\theta }{\left(\frac{\mu }{\mu +\theta }\right)}^x$$

(3)

with $\mu$ > 0 and $\theta$ > 0. Note that $\theta$ is not necessarily an integer, contrary to $r$ in (1); hence, the gamma function is used in (3) instead of a factorial, with $\mathsf{\Gamma }\left(x\right)=\left(x-1\right)!$. The variance of the NB distribution is $\mu \left(1+\mu /\theta \right)$. As $\theta$ approaches infinity, the NB distribution tends to follow the Poisson distribution, with the mean $\mu$.

1.1. Sum of Negative Binomials

The sum of independent NB variables is of special interest in different contexts, such as the study of animal distribution [4,5], fecal egg counts in infected goats [6], the number of emergency medical calls [7], empirical distribution of the duration of wet periods in days [8] or insurance risk [9]. When the sum of several independent NB counts is available, determining the distribution of $\sum X_i$ with $X_i ~ NB\left(r_i;p_i\right)$ is a problem. When the $p_i$s are all the same and equal to p, a classical result is $\sum X_i ~ NB\left(\sum r_i;p\right)$ [10], but more general forms without this constraint are often needed. For example, if counts are available for various spatial or temporal units of the form $X ~ NB\left({\mu }_i;r_i\right)$, $p_i$ being $r_i/\left({\mu }_i+r_i\right)$, it implies that the $p_i$s are not all the same, because ${\mu }_i$ varies among the units [4].

With the mean and variance of the $NB\left(r;p\right)$ distribution being $r\left(1-p\right)/p$ and $r\left(1-p\right)/p^2$, respectively, it follows that the mean and variance of the sum of n-independent NB variables are respective:

$$mean\left(S_n\right)={\sum }_{i=1}^n\left(r_i\left(1-p_i\right)/p_i\right) \mathrm{and} var\left(S_n\right)={\sum }_{i=1}^n\left(r_i\left(1-p_i\right)/p_i{}^2\right)$$

(4)

Our paper has developed some novel methods in relation to the practical computation and use of the convolution approach [9]. However, the paper also collects five different methods and presents them in one place, a useful resource for the working data scientist or statistician. We describe and reference these methods and outline the computational difficulties in getting them to work. We also point the reader to the freely available R software that implements each of the methods (plus a sixth method based purely on simulation).

Two methods have been published to estimate the distribution of the sum of NB independent variables using a finite-sum exact expression [11] or the convolution method [9]. However, the computer implementation of both methods was not available, and we have detected potential problems when a practitioner implements them. The finite-sum exact expression computer implementation is relatively straightforward, but memory overflow can occur, and the time of computing will increase as a function of the factorial of the number of observations, x. This precision was not given in the original publication [11]. The convolution method is very complex to implement and has been described as being “cumbersome” [12]; indeed, we found that its implementation was not straightforward and was even counterintuitive. The method uses a sum to infinity, and the condition to stop the recursion was not defined in the original publication.

Our solution for computation of the convolution method, presented here, is novel and has proven to be robust for extensive testing. A naïve tolerance condition has been used by one of the authors of this note (MG) (recursion stops when the change is lower than the tolerance limit) as in [4,5], but the other author (JB) found that outputs can be strongly biased in some conditions. It has been the beginning of a collaboration between the two authors to understand and solve the origin of this bias. We detected two problems: (1) the tolerance check must be applied only when the first-order change of the estimate is negative (convergence criteria being adaptive), and (2) the value of the tolerance must be proportional to the expected estimate. Then, it was necessary to have an estimate of the density to set the tolerance, to better estimate the correct density. To solve this, we used the saddlepoint approximation of the density. We show that the absolute error of this approximation can be on the order of 5%, being too high to be used in many applications, but it is sufficiently low to define a correct tolerance to be used with the convolution method.

1.2. Normal and Negative Binomial Approximations

When working on the sum of variables, the first thought is to use the central limit theorem [13] that establishes that, in many situations, the distribution of the sum-independent random variables tends to go toward a normal distribution. An alternative is to model the distribution of the sum of NB variables, as an NB distribution is based on the observation that the distribution of the sum of NB variables is a mixture NB distribution [9], according to Theorem 2, proposed by Makun, Abdulganiyu, Shaibu, Otaru, Okubanjo, Kudi, and Notter [6].

1.3. Finite-Sum Exact Expression

An exact form for the distribution of the sum of NB is:

$$P\left(S_n=x\right)={\sum }_{{\mu }_1+\cdots +{\mu }_n=x}{\prod }_{j=1}^n\frac{\mathsf{\Gamma }\left({\mu }_j+\theta \right)}{\mu !\mathsf{\Gamma }\left(r{\theta }_j\right)}{p}_j{}^{{\theta }_j}{q}_j{}^{{\mu }_j}$$

(5)

The expression (5) is compact and the exact value can be computed [11].

1.4. Approximation by Convolution

When $X_i ~ NB\left(r_i;p_i\right)$, with i from 1 to n, the distribution of $S_n=\sum X_i$ is a mixture NB [9], with the probability mass function being approximated by:

$$P\left(S_n=x\right)=R{\sum }_{k=0}^{\infty }{\delta }_k\frac{\mathsf{\Gamma }\left(r+x+k\right)}{\mathsf{\Gamma }\left(r+k\right)x!}{M}_1{}^{r+k}{\left(1-M_1\right)}^x, x=0, 1, 2, \cdots$$

(6)

$$\begin{gathered} \mathrm{where} r={\sum }_{i=1}^n{r}_i , \mathrm{and} M_1=ma{x}_j\left(p_j\right). \\ R={\prod }_{j=1}^n{\left(\frac{q_j{M}_1}{\left(1-M_1\right)p_j}\right)}^{-r_j} \\ \mathrm{and} {\delta }_{k+1}=\frac{1}{k+1}{\sum }_{i=1}^{k+1}i {\xi }_i {\delta }_{k+1-i}, k=0, 1, \dots \mathrm{with} {\delta }_0=1 \mathrm{and} \\ {\xi }_i=\sum _{j=1}^n\frac{r_j{\left(1-\left(1-M_1\right)p_j/q_j{M}_1\right)}^i}{i} \end{gathered}$$

Expression (6) is used iteratively, with k being the counter of the rank of iterations, but a condition to stop the iterations when a certain level of approximation is reached was not defined in the original publication [9].

1.5. Saddlepoint Approximation

The saddlepoint approximation method provides a highly accurate approximation formula for any probability density function (continuous distribution) or probability mass function (discrete distribution) of a distribution, based on the moment-generating function [14].

Taking the log of the moment-generating function of the NB distribution (2) and summing over n-independent NB variables, the cumulant of sum of NBs is:

$$K\left(t\right)={\sum }_{i=1}^n{r}_i\left(log\left(q_i\right)-log\left(1-p_i e^t\right)\right)$$

$$\mathrm{Or} K\left(t\right)={\sum }_{i=1}^n{\theta }_i\left(log\left({\theta }_i\right)-log\left({\theta }_i+{\mu }_i\left(1-e^t\right)\right)\right)$$

(7)

The first and second order of the derivatives of $K\left(t\right)$ are:

$$K^{\prime }\left(t\right)={\sum }_{i=1}^n\frac{{\theta }_i {\mu }_i e^t}{{\theta }_i+{\mu }_i\left(1-e^t\right)}$$

(8)

$$K^{″}\left(t\right)={\sum }_{i=1}^n\frac{{\theta }_i {\mu }_i \left({\theta }_i+{\mu }_i\right) e^t}{{\left({\theta }_i+{\mu }_i\left(1-e^t\right)\right)}^2}$$

(9)

The saddlepoint, $s_x$, is found by solving $K^{\prime }\left(s_x\right)=x$. Once $s_x$ is found, $P\left(S_n=x\right)$ can be approximated by:

$$P\left(S_n=x\right)\approx \frac{1}{\sqrt{2\pi K^{″}\left(s_x\right)}} e^{\left(K\left(s_x\right)-x s_x\right)}$$

(10)

The value $P\left(S_n=x\right)$ is normalized to ensure that $\sum P\left(S_n\right)=1$.

In the remainder of this note, we describe the computational problems that applied statisticians or practitioners face in implementing the distribution of the sum of NB-independent variables using finite-sum exact expression [11], the convolution method [9], saddlepoint approximation, or the approximation by normal and NB distributions. We describe how these have been overcome in the publicly available R package (HelpersMG package version 5.9 and higher (https://CRAN.R-project.org/package=HelpersMG, accessed on 6 February 2023). The code can be checked after loading this package with the command ?dSnbinom.

2. Computations

Figure 1 gives two examples of the sum S of independent NB random variables, and how these distributions are approximated using the four methods (convolution, saddlepoint, single normal, single NB) outlined in this note. In (A), we use n = 10, j = 1…n, $p_j=0.4+\frac{j}{10}$ and $r_j=j\times 10$, and in (B) n = 2, $p_j=\frac{j}{10}$, and $r_j=j$.

2.1. Normal and Negative Binomial Approximations

When n is large and standard deviation is small as compared to the mean, the normal approximation with $P\left(S_n=x\right)={\int }_{x-0.5}^{x+0.5}\mathcal{N}\left(\mu , \sigma \right)$, where $\mathcal{N}\left(\mu , \sigma \right)$ is the normal probability density function with $\mu =mean\left(S_n\right)$ and $\sigma =$ $\sqrt{var\left(S_n\right)}$ can be used as an approximation for the distribution of the sum of independent NB random variables (Figure 1A). However, for a small n or large standard deviation, as compared to the mean (corresponding to a highly skewed distribution), the approximation can be very poor (Figure 1B). The NB distribution modeled with the probability density function, $NB\left(\mu , \theta \right)$, such that $\mu =mean\left(S_n\right)$ and $\theta =mean{\left(S_n\right)}^2/\left(var\left(S_n\right)-mean\left(S_n\right)\right)$, better fits the exact distribution of the sum of NB variables, but still with a bias (Figure 1B). This confirms that the distribution of the sum of independent NB variables is not an NB, as wrongly stated in [6]. It is, rather, a mixture NB (see below) [9]. In summary, the normal and NB approximations generate the highest errors (>30% in some cases) and they should not be used, especially as there are better alternatives.

2.2. Finite-Sum Exact Expression

This method permits the calculation of the exact value for $P\left(S_n=x\right)$. It will therefore be used as a reference here.

For the finite form exact expression method [11], a table of n columns with all the combinations of integers, from 0 to x, that produce a sum of x ($m_1+\cdots +m_n=x)$, must be first established. The number of different ways to distribute x-indistinguishable objects into n-distinguishable categories is C(x + n − 1, n − 1). This is the memory-consuming part of the Vellaisamy and Upadhye [11] method. The density $P\left(X=x\right)$ in Equation (1) is calculated n times for each of these combinations in the final table (the ${\prod }_{j=1}^n$ part of Equation (5)). This is the computationally time-consuming part of the method.

When n and/or x are large, this method requires lot of iterations. For example, there are 1,081,575 different combinations of 17 objects in nine categories. Then, Equation (1) must be applied 9,734,175 times to estimate $P\left(S_n=x\right)$ when using Equation (5).

2.3. Approximation by Convolution

The coefficients of Equation (6) are iteratively defined, and we rewrite the published formula to make the computation more efficient using the recursion:

$$W{\left(S_n=x\right)}_0=\frac{\mathsf{\Gamma }\left(r+x\right)}{\mathsf{\Gamma }\left(r\right)x!}{M}_1{}^r{\left(1-M_1\right)}^x$$

$$W{\left(S_n=x\right)}_{k+1}=W{\left(S_n=x\right)}_k+{\delta }_{k+1}\frac{\mathsf{\Gamma }\left(r+x+k+1\right)}{\mathsf{\Gamma }\left(r+k+1\right)x!}{M}_1{}^{r+k+1}{\left(1-M_1\right)}^x$$

$$P{\left(S_n=x\right)}_k=R W{\left(S_n=x\right)}_k$$

(11)

Intermediate estimates in (11) used log of expressions to prevent a computing overflow. The conditions to stop the iterations were not defined in Furman’s original publication. A typical method in such situations is to stop the recursion when the change in the final output is below a defined tolerance. However, it cannot be used in the context of Equations (6) or (11), because, at the beginning of iterations, the change in $P\left(S_n=x\right)$ is sometimes so small that recursions will be stopped immediately and the resulting probability $P\left(S_n=x\right)$ will be biased. An example of this is shown in Figure 2A, which shows the value of $P{\left(S_7=6\right)}_k$ (n = 7, j = 1…n, p_j = j/10, and r_j = j) as a function of the recursive iterations $k$ from Equation (6). For the first eight iterations, the change in $P{\left(S_7=6\right)}_k$ is very small. To alleviate this problem, many iterations can be used, but without being sure that they are sufficient, and it is done at the expense of running time. This solution was chosen with at least 1000 iterations in [5], but this number of iterations is not always large enough to ensure a correct estimate when n or x are large.

A better approach came from the study on the trend of the rate of change of $P\left(S_n=x\right)$ according to the rank of iteration k: $P_k-P_{k-1}$ vs. $P_{k+1}-P_k$, where $P_k$ denotes the value of $P\left(S_n=x\right)$ at iteration k. In its initial phase, the rate of change of $P$ is positive, with $P_k-P_{k-1}>P_{k+1}-P_k$, then it shows a peak and becomes negative, with $P_k-P_{k-1}<P_{k+1}-P_k$ (Figure 2B). The tolerance threshold must be used only after the occurrence of the peak to ensure that the phase of rapid change of $P$ is reached. The number of iterations before the peak is dependent on the values of n, x, r_i and p_i, and cannot be easily anticipated at the beginning of the iterations. We have therefore developed an adaptative strategy to stop the recursion when two conditions are met: the rate of change of $P\left(S_n=x\right)$ is negative, and the change of $P\left(S_n=x\right)$ is less than the user-defined tolerance. The tolerance value must be lower than $P\left(S_n=x\right)$ or the output will be biased. As an example, if $\mu$ = (0.01, 0.02, 0.03) and θ = (2, 2, 2), then $P\left(S_3=20\right)=7.73139\times {10}^{-35}$ using the exact method. If the Furman method is used with the tolerance set to 10⁻¹², $P\left(S_3=20\right)=3.879379\times {10}^{-35}$, which is two times lower than the exact answer. The solution is to define a tolerance much lower than the anticipated results, for example, here, with the tolerance being 10⁻⁴⁵, $P\left(S_3=20\right)=7.73139\times {10}^{-35}$, which is the correct probability. This can be done using the saddlepoint estimate (see below).

The comparison of the results obtained by Equation (6), with an adaptative stopping of the recursion and tolerance setup, using saddlepoint approximation (see below) and Equation (5), are shown in

Table 1. Comparison of accuracy and computing time of the sum of n numbers x = 3, 5, 8, 10, and 15 obtained from NB distributions with j = 1…n, p_j = j/10, r_j = j, and n from 2 to 7 based on Equations (5), (11), and (10). For each (n, x) combination in (A), the top number is the probability $P\left(S_n=x\right)$ and the bottom number is the number of iterations. In (B), the number of recursions required to stabilize $P\left(S_n=x\right)$ is shown. The $P\left(S_n=x\right)$ values are exactly the same as those in (A) and are not shown. In (C), the $P\left(S_n=x\right)$ values for saddlepoint approximation are shown. Computing times for the different methods are shown at the right of each table. The code for Equation (5) was parallelized on an 8-core computer in R 4.2.3 and HelpersMG package version 5.9 (https://CRAN.R-project.org/package=HelpersMG, accessed on 6 February 2023).

A: Vellaisamy and Upadhye [11]: Exact Probabilities						No Parallel	Parallel 8-Core
	x = 3	x = 5	x = 8	x = 10	x = 15	Time (s)	Time (s)
n = 2	0.02320400	0.03403236	0.04283461	0.04425234	0.03856123	0.001	0.011
	16	36	81	121	256
n = 3	0.00273650	0.00730772	0.01724312	0.02421915	0.03607386	0.003	0.011
	40	126	405	726	2176
n = 4	0.00020980	0.00094784	0.00408465	0.00785680	0.02099302	0.014	0.012
	80	336	1485	3146	13,056
n = 5	0.00001503	0.00010490	0.00076597	0.00196540	0.00920145	0.062	0.015
	140	756	4455	11,011	62,016
n = 6	0.00000131	0.00001291	0.00014555	0.00047692	0.00365038	0.249	0.023
	224	1512	11,583	33,033	248,064
n = 7	0.00000017	0.00000218	0.00003427	0.00013604	0.00154413	0.906	0.049
	336	2772	27,027	88,088	868,224
B: Furman [9]: Convolution				$\mathrm{Tol}=P_{saddlepoint}\left(S_n=x\right)\times {10}^{-10}$
	x = 3	x = 5	x = 8	x = 10	x = 15	Time (s)
n = 2	13	14	15	16	18	0.007
n = 3	19	20	23	24	27	0.008
n = 4	27	29	32	34	38	0.009
n = 5	39	42	45	48	54	0.009
n = 6	58	62	67	70	79	0.009
n = 7	92	97	104	109	122	0.011
C: Normalized Saddlepoint Approximation
	x = 3	x = 5	x = 8	x = 10	x = 15	Time (s)
n = 2	0.02372254	0.03448835	0.04314218	0.04442429	0.03841261	0.007
n = 3	0.00283042	0.00748306	0.01754862	0.02458058	0.03637448	0.007
n = 4	0.00021836	0.00097613	0.00418037	0.00802118	0.02132508	0.008
n = 5	0.00001571	0.00010840	0.00078653	0.00201341	0.00938611	0.008
n = 6	0.00000137	0.00001337	0.00014977	0.00048960	0.00373283	0.008
n = 7	0.00000018	0.00000226	0.00003531	0.00013984	0.00158133	0.018

with the corresponding computing time. This table is similar to those used in Tables 1 and 2 in Vellaisamy and Upadhye [11].

2.4. Saddlepoint Approximation

The saddlepoint approximation (we used the Brent’ algorithm [15] for the minimization needed to find the saddlepoint) is computationally fast. However, the estimate must be normalized so that the density function sums to 1 [16]. The normalization used the sum of $P\left(S_n=x\right),$ with $x$ from 0 to $mean\left(S_n\right)+Max\sqrt{var\left(S_n\right)}$, with $mean\left(S_n\right)$ and $var\left(S_n\right)$ from Equation (4) and $Max=20$. A test was performed to ensure that $P\left(S_n=mean\left(S_n\right)+Max\sqrt{var\left(S_n\right)}\right)$ was 0 or that the $Max$ was increased until this condition was reached. The relative difference between the exact value of $P\left(S_n=x\right)$ and the saddlepoint approximation can be sometimes of the order of 5% (Figure 1). On the other hand, this approximation is good enough to set the tolerance of the approximation by convolution, using a tolerance equal to $P_{saddlepoint}\left(S_n=x\right)\times {10}^{-10}$.

The tolerance value to cut the iterations for an approximate Furman [9] method must be of the same order as the value of $P\left(S_n=x\right),$ multiplied by the tolerance and set at the value of 10⁻¹⁰, to have an estimate precise up to the 10th digit. The difficulty is that $P\left(S_n=x\right)$ needs to be estimated here. The chosen solution was to use a rough estimate of $P\left(S_n=x\right)$ from the very fast saddlepoint approximation method first. This approach proved to be very efficient, because the estimates of the approximate Furman [9] method are exactly the same as for the exact method (Table 1A).

Equation (5) has the advantage that it is parallelizable, but for a large n and x (see Table 1A), doing so requires a large number of both iterations and memory. Equations (6) and (11), however, are not disadvantaged by these problems. Vellaisamy and Upadhye [11] indicated that Equation (5) required less computing time than Equation (6), even for n = 7 and n = 15. This would be true only if the authors used a very large number of iterations to stop the iterations in Equation (6), or if their conditions to stop the iterations were sub-optimal.

As a general conclusion, we consider that the approximate form of distribution for the sum of independent NB proposed by Furman [9] should be used in all the contexts, whatever the parameters n, x, p_i or r_i. The tolerance can be approximated by using the value of $P\left(S_n=x\right)$, estimated using the saddlepoint approximation method. This solution is used by default in the R package HelpersMG (version > 5.9), available in CRAN: The Comprehensive R Archive Network [17].