An Algorithm for Nonparametric Estimation of a Multivariate Mixing Distribution with Applications to Population Pharmacokinetics

Abstract

Population pharmacokinetic (PK) modeling has become a cornerstone of drug development and optimal patient dosing. This approach offers great benefits for datasets with sparse sampling, such as in pediatric patients, and can describe between-patient variability. While most current algorithms assume normal or log-normal distributions for PK parameters, we present a mathematically consistent nonparametric maximum likelihood (NPML) method for estimating multivariate mixing distributions without any assumption about the shape of the distribution. This approach can handle distributions with any shape for all PK parameters. It is shown in convexity theory that the NPML estimator is discrete, meaning that it has finite number of points with nonzero probability. In fact, there are at most N points where N is the number of observed subjects. The original infinite NPML problem then becomes the finite dimensional problem of finding the location and probability of the support points. In the simplest case, each point essentially represents the set of PK parameters for one patient. The probability of the points is found by a primal-dual interior-point method; the location of the support points is found by an adaptive grid method. Our method is able to handle high-dimensional and complex multivariate mixture models. An important application is discussed for the problem of population pharmacokinetics and a nontrivial example is treated. Our algorithm has been successfully applied in hundreds of published pharmacometric studies. In addition to population pharmacokinetics, this research also applies to empirical Bayes estimation and many other areas of applied mathematics. Thereby, this approach presents an important addition to the pharmacometric toolbox for drug development and optimal patient dosing.

1. Introduction

Pharmacokinetic studies in healthy volunteers commonly collect multiple observations in each subject. These datasets often arise from Phase I clinical trials and have been traditionally analyzed by noncompartmental methods or by modeling via the standard two-stage approach. Patient datasets often contain complex dosage regimens and only a few (or one) observation(s) per patient. To analyze such sparse datasets, e.g., in pediatric or critically ill patients, population modeling is required, since noncompartmental analysis and the standard two-stage approach are not applicable [1,2]. Population modeling has been shown to estimate PK parameters without bias and with good precision for such sparse datasets [3]. Parametric population modeling algorithms are commonly used and assume typically either normal or log-normal distributions for the between subject variability of PK parameters. While this assumption is made for virtually every parametric population PK model, it is difficult to prove, especially for datasets with a small number of subjects. Nonparametric population modeling can describe a multivariate distribution of PK parameters without assuming any shape of the PK parameter distribution. This is a key advantage of the nonparametric approach that is based on the exact log-likelihood. The present work comprehensively describes the foundation of this nonparametric estimation algorithm for the first time. This algorithm has been used in hundreds of peer-reviewed papers.

Pharmacometric observations can be described statistically by a mixture model. In this case, the probability of random variable arguments (the PK population model) of the pharmacokinetic compartmental model are described by a mixing distribution. The problem of estimating the mixing distribution from a set of pharmacometric observations can be stated as follows. Let ${\mathit{Y}}_1,\dots ,{\mathit{Y}}_N$ be a sequence of independent but not necessarily identically distributed random vectors constructed from one or more observations from each of N subjects in the population. Let ${\mathbf{\theta }}_1,\dots ,{\mathbf{\theta }}_N$ be a sequence of independent and identically distributed random vectors that represent unknown parameter values of N subjects. The $\mathbf{\theta }$ belong to a compact subset $\Theta$ of Euclidean space with common but unknown distribution F. In other words, F is a distribution of the parameters in the population model. $\Theta$ represents the parameter space, and the dimension of this space is the number of parameters in the population model. The $\left\{{\mathbf{\theta }}_i\right\}$ are not observed. It is assumed that the conditional densities $p\left({\mathit{Y}}_i\right|{\mathbf{\theta }}_i)$ are known, for $i=1,\dots ,N$. $p\left({\mathit{Y}}_i\right|{\mathbf{\theta }}_i)$ represents the model of observations $y_i$ given parameter values ${\theta }_i$ including uncertainties of the measurement protocol. The mixing distribution of ${\mathit{Y}}_i$ with respect to F is given by $p\left({\mathit{Y}}_i\right|F)=\int p\left({\mathit{Y}}_i\right|{\mathbf{\theta }}_i)dF\left({\mathbf{\theta }}_i\right)$. Because of independence of the $\left\{{\mathit{Y}}_i\right\}$, the mixing distribution of the $\left\{{\mathit{Y}}_i\right\}$ with respect to F is given by

$$L\left(F\right)=p({\mathit{Y}}_1,...,{\mathit{Y}}_N|F)=\prod _{i=1}^N\int p\left({\mathit{Y}}_i|{\mathbf{\theta }}_i\right)dF\left({\mathbf{\theta }}_i\right)$$

(1)

The mixing distribution problem is to maximize the likelihood function $L\left(F\right)$ with respect to all probability distributions F on $\Theta$.

Remark 1.

The distribution $F^{ML}$ that maximizes $L\left(F\right)$ is a consistent estimator of the true mixing distribution; which means that $F^{ML}$ will converge to the true distribution if the number of subjects is large. This was proved originally by Kiefer and Wolfowitz in 1956 [4]. The consistency of $F^{ML}$ is especially important for our application to population pharmacokinetics where $F^{ML}$ is used as a prior distribution for Bayesian dosage regimen design [5].

The algorithm described in this paper differs from most other published methods in a number of ways. Our algorithm allows for high dimensional parameter space $\Theta$. Most published methods require the dimension of $\Theta$ to be small and many require the dimension of $\Theta$ to be 1, i.e., these methods required the number of parameters in the population model to be small. We have treated examples where the dimension of $\Theta$ is as high as 29, see Section 3.

Most published algorithms require the $\left\{{\mathit{Y}}_i\right\}$ to be identically distributed and assume that the population model $\left\{p\left({\mathit{Y}}_i\right|{\mathbf{\theta }}_i)\right\}$ is rather simple, such as $p\left({\mathit{Y}}_i\right|{\mathbf{\theta }}_i)$ is a multivariate normal density with mean vector ${\mathbf{\theta }}_i$ and covariance matrix $\Sigma$. Even if $\Sigma$ is unknown and has to be estimated, the structure of this model is straightforward. However, the estimation of $\Sigma$ has to be done carefully to avoid singularities, see Wang and Wang [6]. As will be described in Section 3, we allow $p\left({\mathit{Y}}_i\right|{\mathbf{\theta }}_i)$ to be calculated from a system of nonlinear ordinary differential–algebraic equations.

We now describe the details of our algorithm. It was proved by Lindsay [7] and Mallet [8], under simple hypotheses on the population model $\left\{p\left({\mathit{Y}}_i\right|{\mathbf{\theta }}_i)\right\}$, that the global maximizer $F^{ML}$ of $L\left(F\right)$ could be represented by a discrete distribution with support on at most N points, i.e., a distribution with nonzero probability located on at most N points.

Remark 2.

One way to motivate this result by Lindsay and Mallet is as follows: Suppose by some lucky chance, we knew the exact parameters for each subject. How can we package this into a distribution? Answer: The “empirical distribution” of the exact parameters. That is the discrete distribution supported at the N exact parameters with equal weights. It turns out in this case that this empirical distribution is also the nonparametric maximum likelihood (NPML) distribution of the parameters. What is remarkable is that if we only have noisy measurements $(Y_1,...,Y_N)$ of the N subjects only indirectly related to the subject parameters, the structure of the NPML is the same. A discrete distribution supported at N points. Of course, in this real case, the position and weights of the N support points are unknown. Finding these positions and weights is the subject of this paper.

This result leads immediately to a finite dimensional optimization problem for $F^{ML}$, namely to maximize the likelihood function

$$L(\mathbf{\lambda },\mathbf{\varphi })=\prod _{i=1}^N\sum _{k=1}^K{\lambda }_{k}p\left({\mathit{Y}}_i|{\mathbf{\varphi }}_k\right)$$

(2)

with respect to the support points $\mathbf{\varphi }=({\mathbf{\varphi }}_1,\dots ,{\mathbf{\varphi }}_K)$ and weights $\mathbf{\lambda }=({\lambda }_1,\dots ,{\lambda }_K)$ such that ${\mathbf{\varphi }}_k\in \Theta ,{\lambda }_k\ge 0$ for $k=1,\dots ,K$, $K\le N$ and ${\sum }_{k=1}^K{\lambda }_k=1$.

In our algorithm $l(\mathbf{\lambda },\mathbf{\varphi })=logL(\mathbf{\lambda },\mathbf{\varphi })$ is maximized, so that

$$l(\mathbf{\lambda },\mathbf{\varphi })=\sum _{i=1}^{N}log\sum _{k=1}^K{\lambda }_{k}p\left({\mathit{Y}}_i|{\mathbf{\varphi }}_k\right)$$

(3)

and the maximization problem becomes

$$\mathrm{maximize}l(\mathbf{\lambda },\mathbf{\varphi })$$

(4)

such that $\mathbf{\varphi }\in {\Theta }^K$, $\mathbf{\lambda }=({\lambda }_1,...,{\lambda }_K)\in {\mathbb{R}}_{+}^K$, $K\le N$ and ${\sum }_{k=1}^K{\lambda }_k=1$.

Although the maximization problem in Equation (4) is finite dimensional, it is still high-dimensional. The dimension of the maximization problem in Equation (4) is $N(dim\Theta )+(N-1)$.

The optimization problem in Equation (4) is naturally divided into two problems:

Problem 1. Given a set of support points $\left\{{\mathbf{\varphi }}_k\right\}$, find the optimal weights $\left\{{\lambda }_k\right\}$.

Problem 2. Given the solution to Problem 1, find a better set of support points.

Problems 1 and 2 are solved cyclically until convergence, i.e., no significant improvement in $l(\mathbf{\lambda },\mathbf{\varphi })$.

Problem 1 is a convex programming problem. In our algorithm, we solve this problem by the primal-dual interior-point (PDIP) method. This type of method is standard in convex optimization theory (see Boyd and Vandenberghe [9]). However, the exact implementation for a specific problem varies from problem to problem. The exact details of our implementation are described in the Appendix. See also Bell [10], Baek [11] and Yamada et al. [12]. Our PDIP implementation is fast and can easily handle thousands of variables.

Finding a better set of support points in Problem 2 is a more difficult problem. This location problem is a nonconvex global optimization problem with many local extrema and whose dimension is potentially $N\times dim\Theta$. The details of our algorithm, called the adaptive grid (AG) method, will be described in Section 2.3 and in Algorithm 1.

Algorithm 1 Nonparametric adaptive grid (NPAG) algorithm. Input: $(\mathit{Y},{\mathbf{\varphi }}^0,a,b,{\Delta }_D,{\Delta }_L,{\Delta }_F,{\Delta }_e,{\Delta }_{\lambda })$, $a$ and $b$ are the lists of lower and upper bounds, respectively, of $\Theta$; ${\Delta }_D$ is the minimum distance allowable between points in the estimated $F^{ML}$. ${\Delta }_x$ see Section 2.7. Output: $(\mathit{\varphi },\mathit{\lambda },l(\mathit{\lambda },\mathit{\varphi }\left)\right)$
1:	procedureNPAG($\mathit{Y}$, ${\mathbf{\varphi }}^0,\mathit{a},\mathit{b},{\Delta }_D$)	▹ Estimate $F^{ML}$ given $\mathit{Y}$
2:	Initialization: $\mathbf{\varphi }={\mathbf{\varphi }}^0$, $LogLike=-{10}^{30}$, $F_0={10}^{30}$, $F_1=2\ast F_0$, $eps=0.2$, ${\Delta }_e={10}^{-4}$, ${\Delta }_F={10}^{-2}$, ${\Delta }_L={10}^{-4}$, ${\Delta }_{\lambda }={10}^{-3}$, $n=0$
3:	while $eps\ge {\Delta }_e$ or $|F_1-F_0|\ge {\Delta }_F$ do
4:	Calculate $\Psi \left(\mathbf{\varphi }\right)$	▹$N\times K$ matrix $\left\{p\left(Y_i\right|{\varphi }_k)\right\}$
5:	$[\widehat{\mathbf{\lambda }}\left(\mathbf{\varphi }\right),l(\widehat{\mathbf{\lambda }}\left(\mathbf{\varphi }\right),\mathbf{\varphi })]⟵\mathrm{PDIP}(\Psi \left(\mathbf{\varphi }\right))$	▹ Appendix A
6:	if $\left(\mathrm{MAXCYCLES}==0\right)$ then
7:	$F_{est}^{ML}⟵l(\widehat{\mathbf{\lambda }}\left(\mathbf{\varphi }\right),\mathbf{\varphi })$
8:	$\mathbf{\lambda }⟵\widehat{\mathbf{\lambda }}\left(\mathbf{\varphi }\right)$
9:	return $[\mathbf{\varphi },\mathbf{\lambda },F_{est}^{ML}]$
10:	end if
11:	$n⟵n+1$
12:	${\mathbf{\varphi }}^c⟵\mathrm{CONDENSE}(\mathbf{\varphi },\widehat{\mathbf{\lambda }}\left(\mathbf{\varphi }\right),{\Delta }_{\lambda })$	▹ Algorithm 3
13:	$[\widehat{\mathbf{\lambda }}\left({\mathbf{\varphi }}^c\right),l(\widehat{\mathbf{\lambda }}\left({\mathbf{\varphi }}^c\right),{\mathbf{\varphi }}^c)]⟵\mathrm{PDIP}(\Psi \left({\mathbf{\varphi }}^c\right))$	▹$\mathrm{PDIP}\hspace{0.17em}\mathrm{returns}{G}^n$
14:	$NewLogLike=l(\widehat{\mathbf{\lambda }}\left({\mathbf{\varphi }}^c\right),{\mathbf{\varphi }}^c)$
15:	if $\left(n>\mathrm{MAXCYCLES}\right)$ then
16:	$F_{est}^{ML}⟵l(\widehat{\mathbf{\lambda }}\left({\mathbf{\varphi }}^c\right),{\mathbf{\varphi }}^c)$
17:	$\mathbf{\lambda }⟵\widehat{\mathbf{\lambda }}\left({\mathbf{\varphi }}^c\right)$
18:	return $[\mathbf{\varphi },\mathbf{\lambda },F_{est}^{ML}]$
19:	end if
20:	if $|NewLogLike-LogLike|\le {\Delta }_L$ and $eps>{\Delta }_e$ then
21:	$eps=eps/2$	▹ Adjust precision
22:	end if
23:	if $eps\le {\Delta }_e$then	▹ check EXIT conditions
24:	$F_1=NewLogLike$
25:	if $|F_1-F_0|\le {\Delta }_F$ then
26:	$F_{est}^{ML}⟵F_1$
27:	$\mathbf{\varphi }⟵{\mathbf{\varphi }}^c;\mathbf{\lambda }⟵\widehat{\mathbf{\lambda }}\left({\mathbf{\varphi }}^c\right)$
28:	return $[\mathbf{\varphi },\mathbf{\lambda },F_{est}^{ML}]$
29:	else
30:	$F_0=F_1$; $eps=0.2$	▹ Reset Algorithm
31:	end if
32:	end if
33:	$\mathbf{\varphi }⟵{\mathbf{\varphi }}^e⟵\mathrm{EXPAND}({\mathbf{\varphi }}^c,eps,\mathbf{a},\mathbf{b},{\Delta }_D)$	▹ Algorithm 2
34:	$LogLike\leftarrow NewLogLike$
35:	end while
36:	end procedure

Roughly speaking, Problems 1 and 2 are solved as follows. An initial large grid of possible support points is defined in the hypercube $\Theta$. Problem 1 is solved on this large grid. After PDIP, most of the original grid points are removed due to near-zero weights, leaving a smaller high-probability grid. Problem 1 is then solved on this smaller grid. Then, the adaptive grid method for Problem 2 takes place. For each remaining grid point, up to $2\times dim\Theta$ new (daughter) support points are added. A daughter point outside the search space $\Theta$ or too close to a parent point is discarded. The new grid contains the current high-probability points plus the added daughter points. The algorithm is then ready for Problem 1 again. By construction, each iteration increases the value of $l(\mathbf{\lambda },\mathbf{\varphi })$. This process continues until the function $l(\mathbf{\lambda },\mathbf{\varphi })$ does not significantly change.

1.1. Comparable Methods

Because of space limitations, in this section, we only discuss NPML methods that optimize Equation (4), methods that treat multivariate distributions, and methods which allow general conditional probabilities $\left\{P({\mathit{Y}}_i,{\mathbf{\theta }}_i)\right\}$. As explained in this paper, any such NPML algorithm has to address two problems: locations of support points and weights of support points. NPAG does locations by an adaptive grid method and weights by the primal-dual interior-point (PDIP) method. The algorithms discussed in this seection are summarized in Table 1.

The original methods of Lindsay [7] and Mallett [8] were based on algorithms of optimal design in the style of Fedorov [13]. In Schumitzky [14], an algorithm was proposed which did both locations and weights by the expectation maximization algorithm (EM). It was stable but also slow.

In Lesperance and Kalbfleisch [15], a new method was introduced which did weights by the dual method described in Section 5 of Lindsay [7] and locations by what they called the intra-simplex direction method (ISDM). Even though the Lesperance and Kalbfleisch paper was restricted to univariate distributions, the ISDM method has been generalized to the multivariate case. To briefly describe ISDM, let $D(\mathbf{\theta },F)$ be the directional derivative of $logL\left(F\right)$ in the direction of the Dirac distribution ${\delta }_{\mathbf{\theta }}$ supported at $\mathbf{\theta }\in \Theta$. (This function is defined in Section 4 below). ISDM is an iterative algorithm. At stage k, let $F^k$ be the current estimate $F^{ML}$. Then, find all the local maxima of $D(\mathbf{\theta },F^k)$. These local maxima are added to the current set of support points and a new $F^{k+1}$ is calculated. If there are no new local maxima, then the algorithm is done.

In Pilla, Bartolucci, and Lindsay [16], another new method was developed where the locations were found by an initial fine grid. However, the weights were found by a dual version of the PDIP method.

In Savic, Kjellsson, and Karlsson [17], a nonparametric method (NONMEM-NP) was added to the popular NONMEM^® program. NONMEM-NP is a hybrid parametric–nonparametric approach The locations of support points were found by a parametric maximum likelihood algorithm. Then, the weights were found by maximizing Equation (4) relative to the newly found support points. NONMEM-NP can handle high-dimensional and complex multivariate distributions. An extension to NONMEM-NP was developed in Savic and Karlsson [18] where additional support points are added to the original set. A comparison between NONMEM-NP and NPAG is discussed in Leary [19].

In Wang and Wang [6], a new algorithm was developed for multivariate distributions. The locations were found by a combination of EM and a variant of ISDM. The weights were found by a family of quadratic programs. In [6], examples are performed for 8- and 13-dimensional mutivariate mixing distributions.

Table 1. Table of NPML Methods.

Author(s)/Reference/Date	Problem 1 Method	Problem 2 Method
Lindsay [7] (1983)	Convex Geometry	VDM
Mallet [8] (1986)	Optimal Design	VDM
Lesparance and Kalbfleisch [15] (1992)	Semi-Infinite Programming	ISDM
Savic, Kjellsson and Karlsson [17] (2009)	Parametric NONMEM	None
Pilla, Bartolucci and Lindsay [16] (2006)	Dual of PDIP	Adaptive Grid
Schumitzky [14] (1991)	EM	EM
X. Wang and Y. Wang [6] (2015)	Quadratic Programming	ISDM
NPAG [20] (2001)	PDIP	Adaptive Grid

Legend: VDM vertex direction method; EM expectation–maximization; ISDM intra-simplex direction method; NPML nonparametric maximum likelihood; PDIP primal-dual interior-point method.

Table 1. Table of NPML Methods.

Author(s)/Reference/Date	Problem 1 Method	Problem 2 Method
Lindsay [7] (1983)	Convex Geometry	VDM
Mallet [8] (1986)	Optimal Design	VDM
Lesparance and Kalbfleisch [15] (1992)	Semi-Infinite Programming	ISDM
Savic, Kjellsson and Karlsson [17] (2009)	Parametric NONMEM	None
Pilla, Bartolucci and Lindsay [16] (2006)	Dual of PDIP	Adaptive Grid
Schumitzky [14] (1991)	EM	EM
X. Wang and Y. Wang [6] (2015)	Quadratic Programming	ISDM
NPAG [20] (2001)	PDIP	Adaptive Grid

Legend: VDM vertex direction method; EM expectation–maximization; ISDM intra-simplex direction method; NPML nonparametric maximum likelihood; PDIP primal-dual interior-point method.

Note: The quadratic programming algorithm (QP) of Wang and Wang [6] has an attractive feature. For a prescribed set of support points, QP finds the zero probabilities exactly. Thus, QP avoids the grid condensation step where support points from PDIP with sufficiently low probabilities are deleted. However, QP and PDIP are based on different numerical methods and a comparison of the efficiency of both algorithms has not been determined.

We finally mention that the NPML problem is a special case of a finite mixture model problem with unknown supports and weights. For a discussion of this approach, see Tatarinova and Schumitzky [21].

The algorithms which have shown by published examples to handle the highest dimensional multivariate problems are NONMEM-NP, Wang and Wang [6], and NPAG.

1.2. Benders Decomposition

For any set of grid points $\mathbf{\varphi }=({\mathbf{\varphi }}_1,...,{\mathbf{\varphi }}_m)$ in ${\Theta }^m$, let $\mathbf{\lambda }=\widehat{\mathbf{\lambda }}\left(\mathbf{\varphi }\right)$ be the corresponding set of optimal weights given by the PDIP method. Then, the function $F\left(\mathbf{\varphi }\right)=l(\widehat{\mathbf{\lambda }}\left(\mathbf{\varphi }\right),\mathbf{\varphi })$ depends only on $\mathbf{\varphi }$ and can be maximized directly. For optimization methods, this technique is called Benders decomposition. The NPAG algorithm maximizes $F\left(\mathbf{\varphi }\right)$ by an adaptive search method. In a method proposed by James Burke, $F\left(\mathbf{\varphi }\right)$ is maximized by a Newton-type method. Since the function $F(\mathbf{\varphi }$) is not necessarily differentiable, a relaxed Newton method must be used similar to what is described in the Appendix for the primal-dual algorithm. For details of Benders decomposition as applied to our problem, see Bell [10], Baek [11] and Jordan-Squire [22].

Founded on this prior work, the present study aimed to comprehensively describe, for the first time, the nonparametric adaptive grid algorithm (NPAG). This approach uses the exact log-likelihood to solve population modeling problems and does not make any assumptions about the shape of the PK parameter distributions. We illustrated the features and capabilities of this algorithm using a population PK modeling example. This algorithm presents the computational foundation of several hundred peer-reviewed papers, to date, and is ideally suited to optimize individual patient dosage regimens. The output of the NPAG algorithm becomes the input of the BestDose™ patient dosing software which is used at the bedside in real-time [23].

2. Materials and Methods

2.1. Pmetrics

The simulations and NPAG optimizations in this paper can be duplicated in R, using programs in the Pmetrics package [24]. R and Pmetrics are free software. R is available from many download sites. Pmetrics is available from lapk.org. NPAG is run using the NPrun() command in Pmetrics. Sample datasets and compartmental models are also available at lapk.org.

2.2. NPAG Subprograms

NPAG is a Fortran program consisting of a number of subroutines as described below. The main program performs the adaptive grid (AG) method (consisting of expansion and compression algorithms) and calls the primal-dual interior-point (PDIP) subprogram. The PDIP algorithm solves the maximization problem of Equation (4) for a fixed grid and is described precisely in the Appendix.

2.3. NPAG Implementation (NPAG—Algorithm 1)

For the purpose of this discussion, we can think of PDIP as a function $\widehat{\mathbf{\lambda }}$ from ${\Theta }^m$ into the set $S^m$ = $\{\mathbf{\lambda }\in {\mathbb{R}}_{+}^m:{\sum }_{k=1}^m{\lambda }_k=1\}$ defined as follows: If $\mathbf{\varphi }=({\mathbf{\varphi }}_1,...,{\mathbf{\varphi }}_m)$ then $\widehat{\mathbf{\lambda }}\left(\mathbf{\varphi }\right)=({\widehat{\lambda }}_1,...,{\widehat{\lambda }}_m)$ maximizes Equation (4) relative to the fixed set of grid points $({\mathbf{\varphi }}_1,...,{\mathbf{\varphi }}_m)$. In this case, we write G = $(\mathbf{\varphi },\widehat{\mathbf{\lambda }}\left(\mathbf{\varphi }\right))$ and $l\left(G\right)$ = $l(\mathbf{\varphi },\widehat{\mathbf{\lambda }}\left(\mathbf{\varphi }\right))$.

In NPAG, there are two types of grids: expanded and condensed. The expanded grids are the initial grid and the grids after grid expansion (Algorithm 2). The condensed grids are generated by grid condensation (Algorithm 3). Each cycle of NPAG begins with an expanded grid. The likelihood calculation is done on the condensed grids.

Now for the adaptive grid method. Assume that $\Theta$ is a bounded Q-dimensional hyper-rectangle. Initially, we let ${\mathbf{\varphi }}_{expanded}^0=({\mathbf{\varphi }}_1^0,...,{\mathbf{\varphi }}_M^0)$ be the set of M Faure grid points in $\Theta$ (see [25,26,27]). Alternatively, we could initially let ${\mathbf{\varphi }}_{expanded}^0$ be generated by a uniform distribution on $\Theta$ or by a prior run of the program.

Remark. The Faure grid points for a hyper-rectangle $\Theta$ are a low-discrepancy set which in some sense optimally and uniformly covers $\Theta$. In our implementation of NPAG, the Faure point sets come in discrete sizes which nest with each other. (Allowable number of points equals 2129, 5003, 10,007, 20,011, 40,009, 80,021, and multiples of 80,021.) This nesting property is useful for checking the optimality of $F^{ML}$ (see Section 4). We have found that replacing the initial Faure set with a set generated by a uniform distribution on $\Theta$ increases the time to convergence but results in the same optimal distribution.

Now set $G_{expanded}^0$ = $({\mathbf{\varphi }}^0,\widehat{\mathbf{\lambda }}\left({\mathbf{\varphi }}^0\right))$. Our approach is to generate a sequence of solutions $G^n$ to Equation (4) of increasingly greater likelihood, where unless otherwise specified, $G^n$ refers to the condensed grid at the $n^{th}$ cycle of the algorithm. If $G^n$ has log-likelihood negligibly different than $G^{n-1}$, then $G^n$ is considered the optimal solution to Equation (4) and is relabeled $F^{ML}$. If not, then the process continues using the ${\mathbf{\varphi }}^n$ as the new seed. This loop is repeated until $F^{ML}$ is found.

The stopping conditions for NPAG are defined precisely in Algorithm 1. If the stopping conditions are not met prior to a set maximum number of iterations, the program will exit after writing the last calculated $G^n$ into a file.

Algorithm 2 EXPAND. Input: $\mathbf{\varphi }$ = $({\varphi }_1,\cdots ,{\varphi }_K)$, ${\Delta }_G$, $\Theta =[a_1,b_1]\times [a_2,b_2]\times ...\times [a_Q,b_Q]$, $\mathit{a}=[a_1,\cdots ,a_Q]$, $\mathit{b}=[b_1,\cdots ,b_Q]$, ${\Delta }_D$. Output: ${\mathbf{\varphi }}^{\prime }$=$({\varphi }_1^{\prime },\cdots ,{\varphi }_M^{\prime })$, where $M\le K(1+2Q)$. Note: In this algorithm, $\mathbf{\varphi }$ = $({\varphi }_1,\cdots ,{\varphi }_K)$ is a $Q\times K$ matrix, with $Q=dim\Theta$
	functionExpand($\mathbf{\varphi }$, ${\Delta }_G$, $\mathit{a}$, $\mathit{b}$, ${\Delta }_D$)
2:	Initialize: $[Q,K]=size\left(\mathbf{\varphi }\right)$, $\mathbf{I}=\mathbf{Q}\times \mathbf{Q}$ Identity matrix, $\mathrm{new}\mathbf{\varphi }⟵\mathbf{\varphi }$
	for $k=1,\dots ,K$ do	▹$K=\mathrm{number}\hspace{0.17em}\mathrm{of}\hspace{0.17em}\mathrm{of}\hspace{0.17em}\mathrm{input}\hspace{0.17em}\mathrm{support}\hspace{0.17em}\mathrm{points}$
4:	for $d=1,\dots ,Q$ do	▹$Q=dim\Theta$
	$T\left(d\right)={\Delta }_G(b\left(d\right)-a\left(d\right))$
6:	if $\varphi (d,k)+T\left(d\right)\le b\left(d\right)$ then	▹ Check upper boundary
	${\mathbf{\varphi }}^{+}=\mathbf{\varphi }(:,k)+T\left(d\right)\mathbf{I}(:,d)$
8:	$dist={10}^{30}$
	end if
10:	for $k_{in}=1:\mathrm{length}\left(\mathrm{new}\mathbf{\varphi }\right)$ do
	$\mathrm{newdist}=\sum \mathrm{abs}({\mathbf{\varphi }}^{+}-\mathrm{new}\mathbf{\varphi }(:,k_{in}))./(\mathit{b}-\mathit{a})$	▹ x ./y done
	component-wise
12:	$dist=min(dist,\mathrm{newdist})$
	end for
14:	if $dist\ge {\Delta }_D$ then	▹ Check distance to new support point
	$\mathrm{new}\mathbf{\varphi }⟵\left[\mathrm{new}\mathbf{\varphi },{\mathbf{\varphi }}^{+}\right]$
16:	end if
	if $\varphi (d,k)-T\left(d\right)\ge a\left(d\right)$ then	▹ Check lower boundary
18:	${\mathbf{\varphi }}^{-}=\mathbf{\varphi }(:,k)-T\left(d\right)\mathbf{I}(:,d)$
	$dist={10}^{30}$
20:	end if
	for $k_{in}=1:\mathrm{length}\left(\mathrm{new}\mathbf{\varphi }(1,:)\right)$ do
22:	$\mathrm{newdist}=\sum (abs({\mathbf{\varphi }}^{-}-\mathrm{new}\mathbf{\varphi }(:,k_{in}))./(b-a))$	▹ x./y done
	component-wise
	$dist=min(dist,\mathrm{newdist})$
24:	end for
	if $dist\ge {\Delta }_D$ then	▹ Check distance to new support point
26:	$\mathrm{new}\mathbf{\varphi }⟵[\mathrm{new}\mathbf{\varphi },{\mathbf{\varphi }}^{-}]$
	end if
28:	end for
	end for
30:	$\mathbf{\varphi }⟵new\mathbf{\varphi }$
	end function

Algorithm 3 Condense algorithm. Input: $(\mathbf{\varphi },\mathbf{\lambda },{\Delta }_{\lambda })$, Output: ${\mathbf{\varphi }}^c$ Note: ${\mathbf{\varphi }}^c$ is considered a subset of $\mathbf{\varphi }$
	functionCondense($\mathbf{\varphi },\mathbf{\lambda },{\Delta }_{\lambda }$)
	ind = find ( $\mathbf{\lambda }>(max\mathbf{\lambda }){\Delta }_{\lambda }$ )	▹ Inequality and max are performed component-wise
	${\mathbf{\varphi }}^c=\mathbf{\varphi }(:,\mathbf{ind})$
	return ${\mathbf{\varphi }}^c$
	end function

2.4. Grid Expansion (EXPAND—Algorithm 2)

The crux of the adaptive grid method is how to go from $G^0$ to $G^1$ or, more generally, from $G^n$ to $G^{n+1}$. The details of doing this are now explained roughly below and precisely in Algorithm 1.

Let Q be the dimension of $\Theta$. Suppose at stage n we have a grid of high-probability support points ${\mathbf{\varphi }}^n$. We then add $2Q$ daughter points for each support point ${\mathbf{\varphi }}_k\in {\mathbf{\varphi }}^n$. The daughter points are the vertices of a small hyper-rectangle centered at each ${\mathbf{\varphi }}_k$ with size proportional to the original size of the hyper-rectangle defining $\Theta$. The size of this small hyper rectangle decreases as the accuracy of the estimates increases. (See Algorithm 2).

Let ${\mathbf{\varphi }}_{expanded}^{n+1}$ = ${\mathbf{\varphi }}^n\cup \mathrm{Daughter}-\mathrm{Points}$. Then the PDIP subprogram is applied to ${\mathbf{\varphi }}_{expanded}^{n+1}$ resulting in the new solution set $G_{expanded}^{n+1}$ = $({\mathbf{\varphi }}_{expanded}^{n+1},\widehat{\mathbf{\lambda }}\left({\mathbf{\varphi }}_{expanded}^{n+1}\right))$ (see Algorithm 1). The solution set $G_{expanded}^{n+1}$ is now ready for grid condensation.

2.5. Grid Condensation (CONDENSE—Algorithm 3)

The above solution set $G_{expanded}^{n+1}$ may have many support points with low probability. We remove all support points which have corresponding probability less than $(max\mathbf{\lambda }){\Delta }_{\mathbf{\lambda }}$, where $\mathbf{\lambda }$ is the vector of current probabilities and the default for ${\Delta }_{\mathbf{\lambda }}$ is ${10}^{-3}$. (Note that the remaining probabilities are not normalized at this point.) The probabilities of the remaining support points are normalized by a second call to the PDIP subprogram. This second call to PDIP is fast. The likelihood associated with these remaining support points and normalized probabilities is then used to update the program control parameters and check for convergence (Algorithm 1 and Section 2.7). If convergence is attained, then the output of this second call to PDIP provides the support points and probabilities of the final solution. If convergence is not attained, then the remaining support points are sent to the grid expansion subprogram (Algorithm 2), initializing the next cycle.

At the end of the program, the output of this second call to PDIP provides the location and weights of the final solution.

2.6. PDIP Subprogram—See Appendix A

The PDIP subprogram finds the optimal solution to Equation (4) with respect to $\mathbf{\lambda }$ for fixed $\mathbf{\varphi }$. PDIP employs a primal-dual interior-point method that uses a relaxed Newton method to solve the corresponding Karush–Kuhn–Tucker equations. (See Equations (14)–(17) of Appendix A).

For any $\mathit{Y}$=(${\mathit{Y}}_1,...,{\mathit{Y}}_N$) and any $\mathbf{\varphi }$=(${\mathbf{\varphi }}_1,...,{\mathbf{\varphi }}_K)\in {\Theta }^K$, the input to the PDIP subprogram is the $N\times K$ matrix $\left\{p\left({\mathit{Y}}_i\right|{\mathbf{\varphi }}_k)\right\}$. The output consists of the optimal weights $\widehat{\mathbf{\lambda }}\left(\mathbf{\varphi }\right)$ and the corresponding log-likelihood $l(\widehat{\mathbf{\lambda }}\left(\mathbf{\varphi }\right),\mathbf{\varphi })$. An in-depth description of the PDIP algorithm and its implementation is presented in Appendix A. See also [10,11,12].

2.7. NPAG Stopping Conditions

As explained above, a potential solution to $F^{ML}$ is not accepted as a global optimum until successive sequences of $G^n$ produce final distributions evaluating to sufficiently close log-likelihood. The various upper and lower bounds $\Delta$ for NPAG control and stopping conditions are defined below and are used in Algorithms 1–3.

Δ_L

Primary upper bound on the allowable difference between two successive estimated log-likelihoods; the default initialization is ${10}^{-4}$.

Δ_F

Secondary upper bound on the allowable difference between two successive estimated log-likelihoods of potential $F^{ML}$; the default initialization is ${10}^{-2}$.

Δ_e

Sets an upper bound on the accuracy variable $eps$ of Algorithm 1. The default initialization for ${\Delta }_e$ is ${10}^{-4}$. The default initialization for $eps$ is $0.2$ and is stepped down until $eps\le {\Delta }_e$ ${\Delta }_F$ and ${\Delta }_e$ define the two stopping conditions for Algorithm 1.

Δ_D

Sets a lower bound on how close two support points can get; the default initialization is ${10}^{-4}$.

Δ_λ

Sets a lower bound factor on the probabilities of the weights $\lambda$; the default initialization is ${10}^{-3}$.

2.8. Calculation of $p({\mathit{Y}}_i\mid {\varphi }_k)$

Given observations ${\mathit{Y}}_i$, $i=1,...,N$ and grid points ${\mathbf{\varphi }}_k$, $k=1,...,K$, the PDIP subprogram only depends on the $N\times K$ matrix $\left\{p\left({\mathit{Y}}_i\right|{\mathbf{\varphi }}_k)\right\}$. NPAG can be used for any problem once this matrix is defined. However, the default setting of NPAG is for the problem of population pharmacokinetics. For a good background of population pharmacokinetics, see Davidian and Giltinan [28,29].

In population pharmacokinetics, generally ${\mathit{Y}}_i=({\mathit{y}}_{i,1},...,{\mathit{y}}_{i,M})$ is a matrix of vector observations for the ith subject. Since NPAG allows multiple outputs, each ${\mathit{y}}_{i,m}$ is itself a q-dimensional vector ${\mathit{y}}_{i,m}$ =$(y_{i,m,1},\cdots ,y_{i,m,q})$. The observations $y_{i,m,j}$, are then typically given by a regression equation of the form:

$$\begin{array}{cc}\hfill y_{i,m,j}& =f_{i,m,j}\left({\mathbf{\theta }}_i\right)+{\nu }_{i,m,j},j=1,\cdots ,q\hfill \\ \hfill {\nu }_{i,m,j}& \sim N(0,{\left({\sigma }_{i,m,j}\left({\mathbf{\theta }}_i\right)\right)}^2)\hfill \\ \hfill {\mathbf{\theta }}_i& \mathrm{are}\hspace{0.17em}\mathrm{unobserved}\hspace{0.17em}\mathrm{parameters}\hspace{0.17em}\mathrm{specific}\hspace{0.17em}\mathrm{for}\hspace{0.17em}{\mathit{Y}}_i\hfill \end{array}$$

(5)

In the above Equation (5), $f_{i,m,j}$ is a known nonlinear function depending on the model structure, the dosage regimen, the sampling schedule, all covariates and of course the subject-specific parameter vector ${\mathbf{\theta }}_i$. Except for simple models, $f_{i,m,j}$ requires the solution of (possibly nonlinear) ordinary differential equations.

In the current implementation of NPAG, it is assumed that the $({\mathit{y}}_{i,1},...,{\mathit{y}}_{i,M})$ are independent. Then

$$p\left({\mathit{Y}}_i\right|{\mathbf{\varphi }}_k)=\frac{exp\left({\displaystyle -\frac{1}{2}\sum _{m=1}^M({\mathit{y}}_{i,m}-{\mathit{f}}_{i,m}\left({\mathbf{\varphi }}_k\right)){\Sigma }_{i,m}^{-1}\left({\mathbf{\varphi }}_k\right){({\mathit{Y}}_{i,m}-{\mathbf{f}}_{i,m}\left({\mathbf{\varphi }}_k\right))}^T}\right)}{{\prod }_{m=1}^M\sqrt{{\left(2\pi \right)}^{q}det{\Sigma }_{i,m}\left({\mathbf{\varphi }}_k\right)}}$$

(6)

where ${\mathit{f}}_{i,m}=(f_{i,m,1},...,f_{i,m,q})$ and ${\Sigma }_{i,m}$ = $diag({\sigma }_{i,m,1}^2,...,{\sigma }_{i,m,q}^2)$. For the purposes of matrix multiplication in Equation (6),we think of ${\mathit{y}}_{i,m}$ and ${\mathit{f}}_{i,m}$ as q-dimensional row vectors.

To complete the description of Equation (6) we need to model the standard deviation terms ${\sigma }_{i,m,j}$ of the assay noise. In our implementation of NPAG, four different models are allowed. Let

$${\alpha }_{i,m,j}\left({\mathbf{\varphi }}_k\right)=c_0+c_1{f}_{i,m,j}\left({\mathbf{\varphi }}_k\right)+c_2{f}_{i,m,j}^2\left({\mathbf{\varphi }}_k\right)+c_3{f}_{i,m,j}^3\left({\mathbf{\varphi }}_k\right)$$

(7)

and set

$$\begin{array}{cc}\hfill {\sigma }_{i,m,j}& =\left\{\begin{array}{cc}{\alpha }_{i,m,j}\hfill & \mathrm{assay}\hspace{0.17em}\mathrm{error}\hspace{0.17em}\mathrm{polynomial}\hspace{0.17em}\mathrm{only}\hfill \\ \gamma {\alpha }_{i,m,j}\hfill & \mathrm{multiplicative}\hspace{0.17em}\mathrm{error}\hfill \\ \sqrt{{\alpha }_{i,m,j}^2+{\gamma }^2}\hfill & \mathrm{additive}\hspace{0.17em}\mathrm{error}\hfill \\ \gamma \hfill & \mathrm{constant}\hspace{0.17em}\mathrm{level}\hspace{0.17em}\mathrm{of}\hspace{0.17em}\mathrm{error}\hfill \end{array}\right.\hfill \end{array}$$

(8)

The parameter $\gamma$ in Equation (8) is a variance factor. Artificially increasing the variance during the first several cycles of NPAG increases the likelihood for each $\mathbf{\varphi }$, allowing the algorithm to use these cycles to find a better initial state from which to begin optimization. NPAG also has an option to “optimize” $\gamma$. This changes NPAG from a nonparametric method to a “semiparametric” method and will not be discussed here. The interested reader can consult [12].

Next, if $c_0=0$ in Equation (7), then ${\alpha }_{i,m,j}$ can become small for certain values of $\mathbf{\varphi }$ that in early iterations can be far from optimal. This, in turn, causes numerical problems as the likelihood is infinite if ${\sigma }_{i,m,j}=0$. One way to avoid this problem is to take ${\sigma }_{i,m,j}=constant$. Another way would be to assume that ${\alpha }_{i,m,j}$ is known and is given by

$${\alpha }_{i,m,j}=c_0+c_1{y}_{i,m,j}+c_2{y}_{i,m,j}^2+c_3{y}_{i,m,j}^3$$

(9)

That is, to approximate $\sigma$ by using a polynomial of the observed values rather than model predicted values. In our experience with NPAG, the approximation of Equation (9) is useful for ensuring computational stability (especially during the early cycles of the algorithm). However, from a theoretical perspective, this change violates the conditions of maximum likelihood and will not be discussed here. Again, the interested reader can consult [12].

2.9. Convergence

For a given initial grid ${\mathbf{\varphi }}^0$, the NPAG algorithm is only guaranteed to find a local maximum of $L\left(F\right)$. More precisely, if ${\mathbf{\varphi }}^{*}$ is the final grid of NPAG starting from ${\mathbf{\varphi }}^0$, then $\widehat{\mathbf{\lambda }}\left({\mathbf{\varphi }}^{*}\right)$ is a global maximum on ${\mathbf{\varphi }}^{*}$ but the support points ${\mathbf{\varphi }}^{*}$ may be only a local maximum.

Global convergence of a nonparameteric maximum likelihood method for estimation of a multivariate mixing distribution is difficult. For one-dimensional distributions the problem is straightforward. The idea of proof goes back to at least Fedorov [13] in 1972, which involves the use of directional derivatives.

Let F be any distribution on $\Theta$. Then, the directional derivative of $logL\left(F\right)$ in the direction of the Dirac distribution ${\delta }_{\mathbf{\theta }}$ supported at $\mathbf{\theta }$ is defined by

$D(\mathbf{\theta },F)$ = [${\sum }_{i=1}^{N}P\left({\mathit{Y}}_i\right|\mathbf{\theta })/P\left({\mathit{Y}}_i\right|F)]-N$, $\mathbf{\theta }\in \Theta$, where $p\left({\mathit{Y}}_i\right|F)=\int p\left({\mathit{Y}}_i\right|\mathbf{\theta })dF\left(\mathbf{\theta }\right)$. Let $F_k$ be the current NPML estimate at iteration k. The Fedorov method involves maximizing $D(\mathbf{\theta },F_k)$ for $\mathbf{\theta }\in \Theta$, at every iteration. Then, the point at which the maximum occurs is added in an optimal way to $F_k$ to give $F_{k+1}$. Under the assumptions of regularity, Fedorov shows that $L\left(F_k\right)$ converges to $L\left(F^{ML}\right)$, see Fedorov [13], (Theorem 2.5.3). Many improvements to this method have been made. In Lesperance and Kalbfleisch [15] and Wang and Wang [6], instead of just adding the point at which $D(\mathbf{\theta },F_k)$ occurs, all the points where local maxima occur are added in an optimal way. Again, under the assumptions of regularity, convergence as above is proved. In one dimension, these methods are efficient. In higher dimensions, these methods are not computationally practical.

We now suggest a method to check whether the final distribution of NPAG is globally optimal and, if not optimal, how close it is to the optimal. It also involves the use of the directional derivative $D(\mathbf{\theta },F)$, but only at the last iteration of NPAG. Now define

$$D\left(F\right)=\underset{\mathbf{\theta }\in \Theta }{max}D(\mathbf{\theta },F)$$

Note that the $max$ in the above expression is only over $\Theta$ and not over ${\Theta }^N$. It is proved in Lindsay [7] that $F^{*}$ is a global maximum of $L\left(F\right)$, i.e., $F^{*}$=$F^{ML}$, if and only if $D\left(F^{*}\right)=0$. Even if $D\left(F^{*}\right)\ne 0$, it is useful to make this computation as it is also proved in Lindsay [7] that $L\left(F^{ML}\right)-L\left(F^{*}\right)\le D\left(F^{*}\right)$, so this last expression gives an estimate of the accuracy of the final NPAG result.

Now, even though we said above it is not practical to calculate $D\left(F\right)$ at every iteration of an algorithm, we are just suggesting to make this calculation at the end of the algorithm. This calculation can be performed by a deterministic or stochastic optimization algorithm.

3. Examples

First of all, the NPAG program has been used successfully in high-dimensional and very complex pharmacokinetic–pharmacodynamic models. In Ramos-Martin et al. [30], the NPAG program was used for a population model of the pharmacodynamics of vancomycin for coagulase-negative staphylococci (CoNS) infection in neonates. Vancomycin is an antibiotic used to treat a number of serious bacterial infections. CoNS are the most commonly isolated pathogens in the neonatal intensive care unit. This model had 7 nonlinear differential equations and 11 random parameters. The population was a combination of 300 experimental and animal subjects. In Drusano et al. [31], the NPAG program was used for a population model of two drugs for the treatment of tuberculosis. This model had 5 nonlinear differential equations, 3 nonlinear algebraic equations, 1671 observations from 6 outputs and 29 random parameters. In the algebraic equations, the state variables were only defined implicitly and had to be solved for by an iterative method.

The above two examples are too complex to use for simulation purposes. Consequently, we present here a simpler model which has an analytic solution and which can be checked by other algorithms. Nevertheless, the estimation of parameters in this model is not trivial. We consider a three-compartment PK model with a continuous IV infusion into the central compartment and a bolus input into the absorption compartment. The individual subject model is described by the following differential equations:

$$\begin{array}{cc}\hfill \frac{\mathrm{d}{x}_1}{\mathrm{d}t}& =-K_a{x}_1,x_1\left(t\right)=\left\{\begin{array}{cc}0\hfill & \mathrm{for} 0\le t<5\hfill \\ b\hfill & \mathrm{if} t=5\hfill \end{array}\right.\hfill \\ \hfill \frac{\mathrm{d}{x}_2}{\mathrm{d}t}& =K_a{x}_1-\left(K_e+K_{cp}\right)x_2+K_{pc}{x}_3+r\left(t\right),x_2\left(0\right)=0\hfill \\ \hfill \frac{\mathrm{d}{x}_3}{\mathrm{d}t}& =K_{cp}{x}_2-K_{pc}{x}_3,x_3\left(0\right)=0\hfill \end{array}$$

and output equation

$$y_1\left(t\right)=x_2\left(t\right)/V_c+w\left(t\right),w\left(t\right)\sim N(0,{\sigma }^2),\sigma =5.5$$

The inputs are a bolus $b=2000$ mg at $t=5$ Hr and a continuous infusion $r\left(t\right)=500$ Hr${}^{-1}$, for $0<t<16$ Hr. This model has 5 random parameters (V, $K_a$, $K_e$, $K_{cp}$, $K_{pc}$). A diagram of this model is given in Figure A1. It is known that this model is structurally identifiable, see Godfrey [32]. However, we have found that for a continuous IV infusion, the parameters $K_{cp}$ and $K_{pc}$ can be difficult to estimate in a noisy environment.

The details of the simulation are as follows. There were 300 simulated subjects. The random variables $(V,K_a,K_{cp},K_{pc})$ were independently simulated from normal distributions with means respectively equal to $(1.2,0.8,2.0,0.2)$ and standard deviations equal to 25% coefficient of variation.

The random variable $K_e$ was independently simulated from a bimodal mixture of two normal distributions with means respectively equal to $0.5$ and $1.5$, with standard deviations equal to $10\%$ coefficient of variation, and with weights equal to $0.2$ and $0.8$. This distribution would apply to an elimination rate constant with a bimodal distribution where $80\%$ of the subjects have a mean of $1.5$, and only $20\%$ have a mean of $0.5$. The power of the nonparametric method allows the detection of the $20\%$ group.

Eleven observations were taken at times $t=0.25,1.0,4.98,5.25,5.5,6.0,7.0,8.5,10.0,$ $13.0,16.0$.

These sampling times were chosen in an ad hoc fashion and are not to be considered optimal. In Figure A2, we show the profiles of the 300 noisy model outputs $y_1$. These profiles are plotted as piecewise linear functions with nodes at the observation times.

The initial Faure set had $80,321$ support points dispersed in the volume

$$\left(K_a,V,K_e,K_{cp},K_{pc}\right)\in \left[(0.01,2.0),(0.01,2.5),(0.0001,2.0),(0.0,4.0),(0.0001,2.0)\right]$$

The assay error (Equation (8)) is not always known. An approximate assay error polynomial can be inferred from literature and Pmetrics includes a routine to estimate the assay error polynomial from the data. Another approach to analyzing data with unknown measurement error is to run successive NPAG optimizations using decreasing error magnitude for each new run. An advantage of this approach is that model development is faster. The first cycle of NPAG, which begins with a relatively large measurement error, can be initialized using a relatively small number of support points. Each NPAG solution is used as a prior to skip the first (and most computationally burdensome) step in the next NPAG run. We demonstrate this approach can converge to the correct solution on this simulated data.

Convergence for this problem was accomplished after applying NPAG four times. For the first application, $\sigma =0.025Y_{simulated}$. The output distribution of this first application was used as a prior to start NPAG again, this time with $\sigma =7.96+0.0125Y_{simulated}$. The output of this second application was used as a prior to start a third NPAG run, this time with $\sigma =7.96+0.0065Y_{simulated}$. Finally, the output density of this third run was used as a prior to run NPAG a fourth time, this time with $\sigma =5.5$, the same as for the simulation. The step down in assumed observation error happened at convergence for each previous error levels: at cycles 4513, 5972, and 6791. Final convergence happened at cycle 8012. There are 284 support points in the final density.

The simulated and estimated marginal distributions are shown in Figure A3 and Figure A4. It is seen that the estimated marginal distributions are similar to the simulated histograms. In particular, the bimodal shape of $K_e$ was uncovered. Similarity is tested using the R routine mtsknn.eq(..., k = 3), which returns a $pval=0.5809$. mtsknn.eq applies a K—nearest neighbors approach to estimate the probability that two non-parametric distributions arise from the same distribution.

NPAG is designed to estimate the whole joint distribution of the parameters. As mentioned earlier, the estimate $F^{ML}$ is especially important for our application to population pharmacokinetics where $F^{ML}$ is used as a prior distribution for Bayesian dosage regimen design. However, $F^{ML}$ is a consistent estimator of the true mixing distribution and consequently, the moments of $F^{ML}$ should be consitent estimators of the true moments. Means and variances of parameter estimates for $F^{ML}$ can be easily obtained by integrating the corresponding marginal distributions. So as a check of this fact, in

Table A1. Simulation versus optimization. Row 1: True simulated means for each parameter. Row 2: NPAG estimates of corresponding means.

	Ka	Vc	Ke	Kcp	Kpc
${\mu }_{SIM}$	0.7948080	1.1982732	1.3162403	1.9700861	0.2028168
${\mu }_{NPAG}$	0.8003521	1.2054767	1.3108326	1.9780514	0.2059419

, the comparisons of estimated versus simulated means and variances are shown. Again, results are quite accurate (see Table A1 and

Table A2. Parameter Covariance Matrices.

Simulation	Ka	V	Ke	Kcp	Kpc
Ka	0.0215919795
V	0.0011916465	0.0468421011
Ke	−0.0015398993	−0.0005839048	0.1563336061
Kcp	0.0009717487	−0.0032613149	−0.0019679155	0.1391046100
Kpc	−0.0000572936	0.0004080801	0.0007402169	0.0004695783	0.0013055173
NPAG	Ka	V	Ke	Kcp	Kpc
Ka	0.0241232043
V	0.0038827461	0.0537290654
Ke	−0.0009243608	−0.0075706783	0.1686441632
Kcp	−0.0017343788	−0.0083046624	−0.0022665464	0.1617627081
Kpc	0.0006769795	0.0005601072	0.0027358931	0.0007782649	0.0021142178

).

Finally, in Figure A5 we include a graph of Predicted versus Observed values which shows the all around good fit of the data. The predicted (right panel: Predicted Bayesian) values are gotten as follows: For each subject, the Bayesian mean estimate of the parameters are found using the final NPAG distribution as a prior and that subject’s observations. Then, based on these parameter means, the subject’s concentration profile is calculated. The predicted (left panel: Predicted Population) is the weighted average of the evaluation of all support points for subject model.

4. Conclusions

We have provided the first comprehensive description of the NPAG algorithm for estimating multivariate mixing distributions. This algorithm can describe between subject variability without assuming any shape for the distribution of PK parameters and is excellently suited for optimizing patient dosing [33,34,35]; NPAG is based on an iterative algorithm employing the Primal-Dual Interior-Point method and an Adaptive Grid method. This approach can handle pharmacokinetic, pharmacodynamic and other models with a high number of estimated model parameters and is based on the exact log-likelihood. A detailed description of NPAG is provided along with an application for a common two-compartment PK models. Finally, the NPAG algorithm is arguably the most efficiently parallelizable population modeling algorithm, since it can parallelize over both subjects and support points. This allows one to readily implement this algorithm on supercomputers as our laboratory has done on research projects. In addition to population pharmacokinetics, this research also applies to empirical Bayes estimation, see Koenker and Mizera [36] and to many other areas of applied mathematics, see Banks et al. [37]. Overall, the NPAG algorithm provides an important addition to the pharmacometric toolbox for drug development and optimal patient dosing at the interface of applied mathematics, biomedical scientists, and clinicians.

5. Supplementary Material

5.1. Recent References Using Npag

As a means for readers to more quickly assess the applicability of NPAG in a wide variety of pharmacometric studies, we refer here to the first 10 references on a recent PubMed search for papers that use NPAG. Note that all population PK studies using Pmetrics employ NPAG.

• Pharmacodynamics of Posaconazole in Experimental Invasive Pulmonary Aspergillosis: Utility of Serum Galactomannan as a Dynamic Endpoint of Antifungal Efficacy. Gastine S, Hope W, Hempel G, Petraitiene R, Petraitis V, Mickiene D, Bacher J, Walsh TJ, Groll AH. Antimicrob Agents Chemother. 2020 Nov 9:AAC.01574-20. doi: 10.1128/AAC.01574-20. Online ahead of print. PMID: 33168606
• Cerebrospinal fluid penetration of ceftolozane/tazobactam in critically ill patients with an indwelling external ventricular drain. Sime FB, Lassig-Smith M, Starr T, Stuart J, Pandey S, Parker SL, Wallis SC, Lipman J, Roberts JA. Antimicrob Agents Chemother. 2020 Oct 19:AAC.01698-20. doi: 10.1128/AAC.01698-20. Online ahead of print. PMID: 33077655
• Population Pharmacokinetics of Continuous-Infusion Meropenem in Febrile Neutropenic Patients with Hematologic Malignancies: Dosing Strategies for Optimizing Empirical Treatment against Enterobacterales and P. aeruginosa. Cojutti PG, Candoni A, Lazzarotto D, Filì C, Zannier M, Fanin R, Pea F. Pharmaceutics. 2020 Aug 19;12(9):785. doi: 10.3390/pharmaceutics12090785. PMID: 32825109 Free PMC article.
• Caspofungin Weight-Based Dosing Supported by a Population Pharmacokinetic Model in Critically Ill Patients. Märtson AG, van der Elst KCM, Veringa A, Zijlstra JG, Beishuizen A, van der Werf TS, Kosterink JGW, Neely M, Alffenaar JW. Antimicrob Agents Chemother. 2020 Aug 20;64(9):e00905-20. doi: 10.1128/AAC.00905-20. Print 2020 Aug 20. PMID: 32660990 Free PMC article.
• Ethionamide Population Pharmacokinetic Model and Target Attainment in Multidrug-Resistant Tuberculosis. Al-Shaer MH, Märtson AG, Alghamdi WA, Alsultan A, An G, Ahmed S, Alkabab Y, Banu S, Houpt ER, Ashkin D, Griffith DE, Cegielski JP, Heysell SK, Peloquin CA. Antimicrob Agents Chemother. 2020 Aug 20;64(9):e00713-20. doi: 10.1128/AAC.00713-20. Print 2020 Aug 20. PMID: 32631828
• Development and validation of a dosing nomogram for amoxicillin in infective endocarditis. Rambaud A, Gaborit BJ, Deschanvres C, Le Turnier P, Lecomte R, Asseray-Madani N, Leroy AG, Deslandes G, Dailly É, Jolliet P, Boutoille D, Bellouard R, Gregoire M; Nantes Anti-Microbial Agents PK/PD (NAMAP) study group. J Antimicrob Chemother. 2020 Oct 1;75(10):2941-2950. doi: 10.1093/jac/dkaa232. PMID: 32601687
• Population Pharmacokinetics and Target Attainment of Cefepime in Critically Ill Patients and Guidance for Initial Dosing. Al-Shaer MH, Neely MN, Liu J, Cherabuddi K, Venugopalan V, Rhodes NJ, Klinker K, Scheetz MH, Peloquin CA. Antimicrob Agents Chemother. 2020 Aug 20;64(9):e00745-20. doi: 10.1128/AAC.00745-20. Print 2020 Aug 20. PMID: 32601155
• Baclofen self-poisoning: Is renal replacement therapy efficient in patient with normal kidney function? Brunet M, Léger M, Billat PA, Lelièvre B, Lerolle N, Boels D, Le Roux G. Anaesth Crit Care Pain Med. 2020 Oct 14:S2352-5568(20)30230-7. doi: 10.1016/j.accpm.2020.07.021. Online ahead of print. PMID: 33068797
• Ceftriaxone dosing in patients admitted from the emergency department with sepsis. Heffernan AJ, Curran RA, Denny KJ, Sime FB, Stanford CL, McWhinney B, Ungerer J, Roberts JA, Lipman J. Eur J Clin Pharmacol. 2020 Sep 24. doi: 10.1007/s00228-020-03001-z. Online ahead of print. PMID: 32974748
• Population Pharmacokinetic Models of Anti-Tuberculosis Drugs in Patients: a Systematic Critical Review. Otalvaro JD, Hernandez B E AM, Rodriguez CA, Zuluaga AF. Ther Drug Monit. 2020 Sep 18. doi: 10.1097/FTD.0000000000000803. Online ahead of print. PMID: 32956238

5.2. Recent Studies to Which NPAG Can Be Applied

The references below use excellent parametric methods. All of these methods have the same mathematical structure as our nonparametric NPAG. The main difference is that in the parametric methods, it is assumed that the population distribution is multivariate normal with unknown mean vector and unknown covariance matrix. In NPAG, we do not make any parametric assumptions about the population distribution, as discussed in our paper. Otherwise, the population analysis problem is the same. Bulitta et al. [38] and Shah et al. [39] use the method S-ADAPT. S-ADAPT is based on the ADAPT package developed by D’Argenio, Wang and Schumitzky. Ishihara et al. [40] and Soraluce et al. [41] use the industry standard parametric version of NONMEM. Allard et al. [42] uses the stochastic approximation expectation maximization method (SAEM). NPAG can run any problem that is run by any of these programs.

5.3. Comparison of NPAG to Classical Population Analysis Programs

Several studies have compared NPAG to a parametric algorithm. Bustad et al. [43] compared the statistical consistency and efficiency of ITS and two older NONMEM^® routines, FO and FOCE to that of NPEM and NPAG on simulated datasets; the nonparametric methods were more consistent and efficient. Prémaud et al. [44] also compared NPAG to NONMEM^® FOCE, but the two algorithms converged to distinctly different results. NPAG converged to an estimator with better predictive performance allowing for its use in therapeutic drug monitoring. More recently, de Velde et al. [45] compared NONMEM^® FOCE-I to NPAG, with both methods converging to similar parameter estimates. In each of the above studies, NPAG converged to a distribution with greater variance.