A Parametric Bayesian Approach in Density Ratio Estimation

Abstract

This paper is concerned with estimating the ratio of two distributions with different parameters and common supports. We consider a Bayesian approach based on the log–Huber loss function, which is resistant to outliers and useful for finding robust M-estimators. We propose two different types of Bayesian density ratio estimators and compare their performance in terms of frequentist risk function. Some applications, such as classification and divergence function estimation, are addressed.

1. Introduction

The problem of estimating the ratio of two densities appears in many areas of statistical and computer science. Density ratio estimation (DRE) is widely considered to be the most important factor in machine learning and information theory (Sugiyama et al. [1], and Kanamori et al. [2]). In a series of papers, Sugiyama et al. [3,4] developed DRE in different statistical data analysis problems. Some useful applications of DRE are as follows: non-stationary adaptation (Sugiyama and Müller [5] and Quiñonero-Candela et al. [6]), variable selection (Suzuki et al. [7]), dimension reduction (Suzuki and Sugiyama, [8]), conditional density estimation (Sugiyama et al. [9]), outlier detection (Hido et al. [10]), and posterior density estimation (Thomas et al. [11]), among others. This paper addresses cases where the density belongs to the parametric distributions (e.g., exponential family of distributions). Parametric methods are usually favorable thanks to the existence of closed forms (mostly simple and explicit formulas) and hence help enhance computational efficiency.

Several estimators of the ratio of two densities have recently been proposed. One of the simplest approaches to estimate the density ratio $p/q$, where p and q are two probability density (or mass) functions (PDFs or PMFs), is called “plug-in”, and the ratio of the estimated densities is computed. Alternatively, one can estimate the ratio of two densities directly. Several approaches have recently been explored to estimate the ratio, including the moment matching approach (Gretton et al. [12]) and the density matching approach (Sugiyama et al. [13]). There are other works, such as Nguyen [14] and Deledalle [15], which studied the application of DRE in estimating Kullback–Leibler (KL) divergence (or more generally the $\alpha$-divergence function, also known as f-divergence) and vice versa. The main objective of this paper is to address Bayesian parametric estimation with some commonly used loss functions for the ratio of $p/q$ and to compare the proposed estimators with other estimators in the literature.

The remainder of the paper is organized as follows. In Section 2, we discuss the DRE methodology and introduce some useful definitions and related examples. Section 3 discusses how to find a Bayesian DRE under several loss functions for any arbitrary prior density on the parameter, and provides some interesting examples from the exponential families. In Section 4 we study some of the DRE applications. Some numerical illustrations for considering the efficiency of the proposed DREs are given in Section 5. Finally, we make some concluding remarks in Section 6.

2. Density Ratio Estimation for the Exponential Distribution Family

Let $X|\eta$ and $Y|\gamma$ be conditionally independent multivariate random variables

$$\begin{array}{cc}\hfill p(x\mid \eta )& =h\left(x\right)exp\left\{{\eta }^{\top }s\left(x\right)-\kappa \left(\eta \right)\right\},\hfill \end{array}$$

(1)

$$\begin{array}{cc}\hfill q(y\mid \gamma )& =h\left(y\right)exp\left\{{\gamma }^{\top }s\left(y\right)-\kappa \left(\gamma \right)\right\},\hfill \end{array}$$

(2)

where $\eta$, $\gamma \in {\mathbb{R}}^d$ are natural parameters, $s(·)$ is a sufficient statistic, and $\kappa (·)=logc(·)$ is the log–normalizer, which ensures that the distribution integrates to one, that is, $c\left(a\right)=\int h\left(x\right)exp\left\{a^{\top }s\left(x\right)\right\}dx$, for $a=\eta$ and $\gamma$.

Consider the problem of estimating the density ratio

$$\begin{array}{c}\hfill r(t;\eta ,\gamma )=\frac{p(t\mid \eta )}{q(t\mid \gamma )},\end{array}$$

(3)

which is obviously proportional to $exp\left\{{(\eta -\gamma )}^{\top }s\left(t\right)\right\}$. So, one can merge two natural parameters $\eta$ and $\gamma$ into one single parameter $\beta =\eta -\gamma$ (it can be a vector), and write the density ratio in (3) as follows

$$r(t;\theta )=exp\left\{\alpha +{\beta }^{\top }s\left(t\right)\right\},$$

(4)

where $\alpha =\kappa \left(\gamma \right)-\kappa \left(\eta \right)$, and $\theta =(\alpha ,{\beta }^{\top })$ are parameters of interest. Note that since $r(t;\theta )$ itself belongs to the exponential family, the normalization term $\alpha$ can be considered as $-logN\left(\beta \right)$, where $N\left(\beta \right)=\int q\left(t\right)exp\left\{{\beta }^{\top }s\left(t\right)\right\}dt$, which guarantees $\int q(t\mid \gamma )r(t;\theta )dt=1$ and hence $q\left(t\right)r(t;\theta )$ becomes a valid PDF (PMF).

For instance, suppose two normal densities $p(t\mid {\mu }_1,{\sigma }_1^2)=\mathrm{N}(t\mid {\mu }_1,{\sigma }_1^2)$ and $q(t\mid {\mu }_2,{\sigma }_2^2)=\mathrm{N}(t\mid {\mu }_2,{\sigma }_2^2)$. They correspond to (1) and (2) respectively. One can easily verify that $\eta ={(\frac{{\mu }_1}{{\sigma }_1^2},\frac{-1}{2{\sigma }_1^2})}^{\top }$, $\gamma ={(\frac{{\mu }_2}{{\sigma }_2^2},\frac{-1}{2{\sigma }_2^2})}^{\top }$, $s\left(t\right)={(t,t^2)}^{\top }$, and $\kappa \left(\eta \right)=\frac{{\mu }_1^2}{2{\sigma }_1^2}+log{\sigma }_1^2$, $\gamma \left(\eta \right)=\frac{{\mu }_2^2}{2{\sigma }_2^2}+log{\sigma }_2^2$, and so according to (4), we have $\alpha =log\frac{{\sigma }_1}{{\sigma }_2}+\frac{{\mu }_1^2}{2{\sigma }_1^2}-\frac{{\mu }_2^2}{2{\sigma }_2^2}$ and $\beta ={\left(\frac{{\mu }_2}{{\sigma }_2^2}-\frac{{\mu }_1}{{\sigma }_1^2},\frac{1}{2{\sigma }_1^2}-\frac{1}{2{\sigma }_2^2}\right)}^{\top }$.

Another interesting point is the relationship between DRE and the probabilistic classification problem (Sugiyama et al. [16]). Suppose $Z=\left\{0,1\right\}$ is a binary response variable which shows whether an observation t is drawn from either $p\left(t\right)$ (say $Z=0$) or $q\left(t\right)$ (say $Z=1$). In other words $p\left(t\right)=\mathbb{P}(t\mid Z=0)$ and $q\left(t\right)=\mathbb{P}(t\mid Z=1)$. The standard logistic regression model is:

$$\mathbb{P}(Z=1\mid t)=\frac{exp({\beta }_0+{\beta }^{\top }t)}{1+exp({\beta }_0+{\beta }^{\top }t)},$$

and consequently, $\mathbb{P}(Z=0\mid t)=\frac{1}{1+exp({\beta }_0+{\beta }^{\top }t)}$ and

$$\frac{\mathbb{P}(Z=1\mid t)}{\mathbb{P}(Z=0\mid t)}=exp({\beta }_0+{\beta }^{\top }t).$$

(5)

Letting $\pi =\mathbb{P}(Z=1)=\int \mathbb{P}(Z=1\mid t)p\left(t\right)dt$, and applying Bayes’ rule, we have

$$\begin{array}{cc}\hfill p\left(t\right)& =\frac{exp({\beta }_0+{\beta }^{\top }t)}{(1+exp({\beta }_0+{\beta }^{\top }t))\pi },\hfill \\ \hfill q\left(t\right)& =\frac{p\left(t\right)}{(1+exp({\beta }_0+{\beta }^{\top }t))(1-\pi )},\hfill \end{array}$$

and hence

$$\begin{array}{c}\hfill \frac{p\left(t\right)}{q\left(t\right)}=\frac{1-\pi }{\pi }exp({\beta }_0+{\beta }^{\top })=exp\left(\alpha +log\frac{1-\pi }{\pi }+{\beta }^{\top }t\right),\end{array}$$

which indeed has the form of $exp(\alpha +{\beta }^{\top }t)$ with $\alpha ={\beta }_0+log\frac{1-\pi }{\pi }$ and hence has a density ratio model in (4).

However if our response $Z=\left\{0,1,\dots ,m-1\right\}$, we can extend Equation (5) to the multinomial logit model, and we have

$$\frac{\mathbb{P}(Z=k\mid t)}{\mathbb{P}(Z=0\mid t)}=exp({\beta }_{0k}+{\beta }_k^{\top }t),k=1,\dots ,m-1.$$

(6)

Analogously, letting ${\mathbb{P}}_k\left(t\right)=\mathbb{P}(t\mid Z=k)$ and ${\pi }_k=\mathbb{P}(Z=k)$ for $k=1,\dots ,m-1$, we have $\frac{{\mathbb{P}}_k\left(t\right)}{{\mathbb{P}}_0\left(t\right)}=exp({\alpha }_k+{\beta }_k^{\top })$, with ${\alpha }_k={\beta }_{0k}+log\frac{1-{\sum }_{k=1}^{k-1}{\pi }_k}{{\pi }_k}$.

In fact, a conceptually simple solution to the DRE is to separately estimate both densities and calculate the ratio. This can be done by replacing the estimators of the parameters into each density. This approach is known as a plug-in density ratio estimation, defined below:

$${\widehat{r}}_{plug}\left(t\right)=\frac{p(t\mid \widehat{\eta }\left(t\right))}{q(t\mid \widehat{\gamma }\left(t\right))},$$

(7)

where $\widehat{\eta }(·)$ and $\widehat{\gamma }(·)$ are estimates of parameters of $p_{\eta }(·)$ and $q_{\gamma }(·)$ based on a sample of size $n_p$ and $n_q$, respectively.

3. Bayesian DRE

Consider the loss function $\ell (\theta ,\delta (t\left)\right)$ and its average under long-term (repeated) use of $\delta \left(t\right)$. In estimating $\theta$ it is called frequentist risk and given by

$$\begin{array}{cc}\hfill R(\theta ,\delta )& =E^{T|\theta }[\ell (\theta ,\delta )]\hfill \end{array}$$

(8)

$$\begin{array}{c}={\int }_t\ell (\theta ,\delta \left(t\right))P_{\theta }\left(dt\right),\hfill \end{array}$$

(9)

where ${\mathbb{E}}^{T\mid \theta }(·)$ is the expectation with respect to the arbitrary measurable cumulative density function (CDF) $T\sim P(t\mid \theta )$. Given any prior distribution $\pi (·)$, with the CDF $\mathsf{\Pi }(·)$, one can define the integrated risk (Bayes risk), which is the frequentist risk averaged over the values of $\theta$ according to prior distribution $\pi \left(\theta \right)$ and posterior distribution $\pi \left(\theta \right|t)$.

$${\mathbb{E}}^{\theta \mid t}\left[R\left(\delta \right(t),\theta )\right]={\int }_{\mathsf{\Theta }}{\int }_{\mathcal{T}}\ell (\theta ,\delta \left(t\right))P_{\theta }\left(dt\right)\mathsf{\Pi }\left(d\theta \right)dt.$$

(10)

Finally, the Bayes estimator is the minimizer of the Bayes risk (10). It can be shown that the minimizer of the above expression also minimizes the posterior risk function ${\mathbb{E}}^{\theta \mid t}\left[\ell (\theta ,\delta (t\left)\right)\right]$, and hence is the Bayes estimator (see Lehman and Casella [17]).

Next, we will address the log–Huber loss and reason beyond choosing such a loss function, in order to study the efficiency of the DRE.

3.1. log–Huber’s Robust Loss

One of the weakness of a squared error loss function $z^2$, with $z=\delta -\theta$, is that it can over-emphasize outliers. Other loss functions can be used to avoid this issue. For instance, one can use the absolute loss function $\left|z\right|$, but since it is not a differentiable function at the origin, Huber [18] proposed the following function instead:

$$h\left(z\right)=\left\{\begin{array}{cc}{z}^2\hfill & \left|z\right|\le 1,\hfill \\ 2\left|z\right|-1\hfill & \left|z\right|\ge 1,\hfill \end{array}\right.$$

(11)

which is less sensitive to outliers and more robust and equivalent to the squared error loss ($L_2$ loss) for errors that are smaller than 1, and absolute error loss ($L_1$ loss) for larger errors (see Figure 1). This loss borrows the advantages of $L_1$ and $L_2$ losses, and does not have their disadvantage. That is, it is not sensitive to outliers (as opposed to $L_2$) and it is differentiable everywhere (as opposed to $L_1$). In practice, optimizing the Huber loss (and consequently, log–Huber) is much faster because it enables us to use standard smooth optimization methods (e.g., quasi-Newton) instead of linear programming. Thanks to its robustness, the Huber loss is widely used in robust regression problems (e.g., Huber and Ronchetti [19], Chapter 7).

In the context of DRE, we propose taking $z=log\frac{\widehat{r}\left(t\right)}{r(t,\theta )}$ in (11), which yields the log–Huber loss function $\ell (\widehat{r},r)={(log\frac{\widehat{r}}{r})}^2$, for $e^{-1}\le \frac{\widehat{r}}{r}\le e$ (and $\ell (\widehat{r},r)=|log\frac{\widehat{r}}{r}|$, otherwise).

The corresponding frequentist risk function to the log–Huber loss function is given by

$$R(\widehat{r},\theta )=\left\{\begin{array}{cc}\int \int {(log\frac{\widehat{r}}{r})}^{2}dP\left(x\right)dP\left(y\right)\hfill & e^{-1}\le \frac{\widehat{r}}{r}\le e,\hfill \\ 2\int \int \left(|log\frac{\widehat{r}}{r}|-1\right)dP\left(x\right)dP\left(y\right)\hfill & 0<\frac{\widehat{r}}{r}\le e^{-1},\frac{\widehat{r}}{r}\ge e,\hfill \end{array}\right.$$

(12)

where $\theta =(\eta ,\gamma )$.

3.1.1. Log–$L_2$ Loss Function

The log–$L_2$ loss ${(logz)}^2$, with $z=\frac{\widehat{r}}{r}$, puts a small weight whenever the density ratio estimator $\widehat{r}$ and the true density ratio r are close, and puts proportionally more weight when they are significantly different. Lemma 2 provides the Bayesian DRE of the density ratio r in (3). An interesting representation of Bayesian DRE is the Bayesian DRE that can be expressed in terms of the plug-in DRE in (7).

Theorem 1.

For any PDF p and q belonging to (1) and (2), respectively, the Bayesian DRE of under log–$L_2$ loss $L(\widehat{r},r)={(log\frac{\widehat{r}}{r})}^2$, and prior distribution $\pi \left(\theta \right)$, for $\theta =(\eta ,\gamma )$, is given by

$${\widehat{r}}_{\pi }\left(t\right)={\widehat{r}}_{plug}\left(t\right)H\left(t\right),$$

(13)

where ${\widehat{r}}_{plug}$ is obtained by replacing the posterior expectations of unknown parameters for given t:

$${\widehat{r}}_{plug}\left(t\right)=\frac{exp\{\langle s\left(t\right),{\mathbb{E}}^{\eta \mid X=t,Y=t}\eta \rangle -log{\mathbb{E}}^{\eta \mid X=t,Y=t}logc\left(\eta \right)\}}{exp\{\langle s\left(t\right),{\mathbb{E}}^{\gamma \mid X=t,Y=t}\gamma \rangle -log{\mathbb{E}}^{\gamma \mid X=t,Y=t}logc\left(\gamma \right)\}\}},$$

(14)

the correction factor is given by

$$H\left(t\right)=\frac{exp\{log{\mathbb{E}}^{\eta \mid X=t,Y=t}c\left(\eta \right)-{\mathbb{E}}^{\eta \mid X=t,Y=t}logc\left(\eta \right)\}}{exp\{log{\mathbb{E}}^{\gamma \mid X=t,Y=t}c\left(\gamma \right)-{\mathbb{E}}^{\gamma \mid X=t,Y=t}logc\left(\gamma \right)\}}.$$

(15)

Proof. 

The Bayesian DRE ${\widehat{r}}_{\pi }\left(t\right)$ is the minimizer of the posterior risk and

$$\frac{\partial }{\partial log{\widehat{r}}_{\pi }}{\mathbb{E}}^{\eta \mid X=t,Y=t}{\left(log{\widehat{r}}_{\pi }-logr\right)}^2=0$$

implies $log{\widehat{r}}_{\pi }\left(t\right)={\mathbb{E}}^{\eta \mid X=t,Y=t}log{r}_{\pi }$, or equivalently,

$${\widehat{r}}_{\pi }\left(t\right)=exp\left\{{\mathbb{E}}^{\eta \mid X=t,Y=t}logr\left(t\right)\right\},$$

with $\frac{{\partial }^2}{\partial log{\widehat{r}}_{\pi }^2}{\mathbb{E}}^{\eta \mid X=t,Y=t}{\left(log\widehat{r}-logr\right)}^2\ge 0$. Therefore

$$\begin{array}{cc}\hfill {\widehat{r}}_{\pi }\left(t\right)& =exp\left\{{\mathbb{E}}^{\eta |X=t}log{p}_{\eta }\left(t\right)-{\mathbb{E}}^{\gamma |Y=t}log{q}_{\gamma }\left(t\right)\right\}\hfill \\ & =\frac{exp\left\{{\mathbb{E}}^{\eta |X=t}log{p}_{\eta }\left(t\right)\right\}}{exp\left\{{\mathbb{E}}^{\gamma |Y=t}log{q}_{\gamma }\left(t\right)\right\}}\hfill \\ & =\frac{h_1\left(t\right)exp\left\{\widehat{\eta }\top \left(x\right)s_1\left(t\right)-log{c}_1\left(\widehat{\eta }\left(t\right)\right)\right\}}{h_2\left(t\right)exp\left\{\widehat{\gamma }\top \left(t\right)s_2\left(t\right)-log{c}_2\left(\widehat{\gamma }\left(t\right)\right)\right\}}\hfill \\ & \times \frac{exp\left\{c_1\left({\widehat{\eta }}_1\left(t\right)\right)-\widehat{log{c}_1\left({\eta }_1\right)}\left(t\right)\right\}}{exp\left\{c_2\left({\widehat{\eta }}_2\left(t\right)\right)-\widehat{log{c}_2\left({\eta }_2\right)}\left(t\right)\right\}}\hfill \\ & ={\widehat{r}}_{plug}\left(t\right)H\left(t\right).\hfill \end{array}$$

This completes the proof. ☐

In

Table 1. Correction factor $H\left(t\right)$ and conditions when $H\left(t\right)=1$ associated with log–$L_2$ loss. $Bet$ beta distribution; $Bin$: binary distribution; $Gam$: gamma distribution; $Ge$: geometric distribution; $IG$: inverse gamma conjugate prior distribution; P: Poisson distribution; $Pa$: Pareto distribution; W: Weibull distribution.

Density	Conjugate Prior	Correction Factor $\mathit{H}\left(\mathit{t}\right)$	Condition for $\mathit{H}=1$
$Gam({\alpha }_i,{\beta }_i)$	${\beta }_i\sim Gam(a_i,b_i)$	${({\alpha }_1+a_1)}^{{\alpha }_1}{({\alpha }_2+a_2)}^{{\alpha }_2}exp\{-{\alpha }_1\psi ({\alpha }_1+a_1)-{\alpha }_2\psi ({\alpha }_2+a_2)\}$	${\alpha }_1={\alpha }_2$, $a_1=a_2$
$\mathbb{N}({\theta }_i,{\sigma }_i^2)$	${\theta }_i\sim \mathbb{N}({\mu }_i,{\tau }_i^2)$	$exp\{1/2(\frac{1}{b_1(1+b_1)}-\frac{1}{b_2(1+b_2)})\}$	$\frac{b_1}{b_2}=\frac{1+b_1}{1+b_2}$, where $b_i={\sigma }_i^2/{\tau }_i^2$
$\mathbb{N}({\theta }_i,{\sigma }_i^2)$	${\sigma }_i^2\sim IG({\alpha }_i,{\beta }_i)$	$\sqrt{\frac{{\alpha }_1-1/2}{{\alpha }_2-1/2}}exp\{1/2(\psi ({\alpha }_2+1/2)-\psi ({\alpha }_1+1/2))\}$	${\alpha }_1={\alpha }_2$
$Pa(a_i,b_i)$	$b_i\sim Gam({\alpha }_i,{\beta }_i)$	$\frac{{\alpha }_1+1}{{\alpha }_2-1}exp\{\frac{{\alpha }_1-{\alpha }_2}{{\alpha }_1{\alpha }_2}+\psi \left({\alpha }_2\right)-\psi \left({\alpha }_1\right)\}$	${\alpha }_1={\alpha }_2$
$W({\lambda }_i,k_i)$	${\lambda }_i\sim IG({\alpha }_i,{\beta }_i)$	${\alpha }_1^{k_1}/{\alpha }_2^{k_2}exp\{k_2\psi ({\alpha }_2+1)-k_1\psi ({\alpha }_1+1)\}$	$k_1=k_2$, ${\alpha }_1={\alpha }_2$
$Bin(n_i,p_i)$	$p_i\sim Bet({\alpha }_i,{\beta }_i)$	$\frac{{\alpha }_2+{\beta }_2+n_2}{{\alpha }_1+{\beta }_1+n_1}\frac{{\beta }_1+n_1-t}{{\alpha }_2+n_2-t}\times exp\{\psi ({\beta }_2+n-t)-\psi ({\beta }_1+n_1-t)+\psi ({\alpha }_1+{\beta }_1+n_1)-\psi ({\alpha }_2+{\beta }_2+n_2)\}$	${\alpha }_1={\alpha }_2,{\beta }_1={\beta }_2,n_1=n_2$
$P\left({\lambda }_i\right)$	${\lambda }_i\sim Gam({\alpha }_i,{\beta }_i)$	1	$\forall {\alpha }_i,{\beta }_i$, and ${\lambda }_i$
$Ge\left(p_i\right)$	$p_i\sim Bet({\alpha }_i,{\beta }_i)$	$exp\{\psi ({\beta }_1+t)-\psi ({\alpha }_1+1)+\psi ({\beta }_2+t)-\psi ({\alpha }_2+1)\}\frac{{\beta }_2+t}{{\beta }_1+t}\frac{{\alpha }_1+1}{{\alpha }_2+1}$	${\alpha }_1={\alpha }_2,{\beta }_1={\beta }_2$

we explore the correction factor $H(·)$ in (15) for certain densities which belong to the exponential family. Let the likelihood functions below, with sub indices $i=1,2$ representing the underlying distributions, be drawn from p and q according to (1) and (2), respectively. Note that posing constraints on the hyper-parameters leads to having $H\left(t\right)$ as a constant function in t. In fact, $H\left(t\right)=1$ induces the Bayesian DRE ${\widehat{r}}_{\pi }$ to coincide with the plug-in DRE ${\widehat{r}}_{plug}$.

Note that the notation $\psi \left(\alpha \right)$ below is for a “digamma” function, and is given by $\frac{\partial \mathsf{\Gamma }\left(\delta \right)/\partial \delta }{\mathsf{\Gamma }\left(\delta \right)}$. Also, in Table 1 and

Table 2. KL divergence between p and q. ${\chi }^2$: chi-squared distribution; U: uniform distribution; $Ray$: Rayleigh distribution.

Density	KL
${\mathbb{N}}_p({\theta }_i,{\Sigma }_i)$	$\frac{1}{2}\left[{({\theta }_1-{\theta }_2)}^{\prime }{\Sigma }_2^{-1}({\theta }_1-{\theta }_2)+tr\left({\Sigma }_2^{-1}{\Sigma }_1\right)-p-log\frac{det\left({\Sigma }_1\right)}{det\left({\Sigma }_2\right)}\right]$
$\mathbb{N}({\theta }_i,{\sigma }_i)$	$\frac{1}{2{\sigma }_2^2}\left[{({\theta }_1-{\theta }_2)}^2+{\sigma }_1^2-{\sigma }_2^2\right]-\frac{1}{2}log\frac{{\sigma }_1^2}{{\sigma }_2^2}$
Bet$(a_i,b_i)$	$log\frac{Bet(a_2,b_2)}{Bet(a_1,b_1)}+\psi \left(a_1\right)(a_1-a_2)+\psi \left(b_1\right)(b_1-b_2)+\psi (a_1+b_1)(a_2-a_1+b_2-b_1)$
Gam$(k_i,{\lambda }_i)$	$(\frac{{\lambda }_1}{{\lambda }_2}-1)k_1+(k_1-k_2)(log{\lambda }_1+\psi \left(k_1\right))-log\frac{\mathsf{\Gamma }\left(k_1\right){\lambda }_1^{k_1}}{\mathsf{\Gamma }\left(k_2\right){\lambda }_2^{k_2}}$
${\chi }^2\left(k_i\right)$	$log\frac{\mathsf{\Gamma }\left(\frac{k_2}{2}\right)}{\mathsf{\Gamma }\left(\frac{k_1}{2}\right)}+\frac{1}{2}\psi \left(\frac{k_2}{2}\right)(k_1-k_2)$
log–$\mathbb{N}({\theta }_i,{\sigma }_i^2)$	$\frac{1}{2{\sigma }_2^2}\left[{({\theta }_1-{\theta }_2)}^2+{\sigma }_1^2-{\sigma }_2^2\right]-log\frac{{\sigma }_1^2}{{\sigma }_2^2}$
Ray$\left({\sigma }_i\right)$	$2log\frac{{\sigma }_2}{{\sigma }_1}+\frac{{\sigma }_1^2-{\sigma }_2^2}{{\sigma }_2^2}$
Pa $(m_i,{\alpha }_i)$	$log{\left(\frac{m_1}{m_2}\right)}^2-log\frac{a_2}{a_1}+\frac{a_2}{a_1}-1$
U$(0,{\theta }_i)$	$log\left(\frac{{\theta }_2}{{\theta }_1}\right)$ Provided ${\theta }_2>{\theta }_1$

, $Gam$, P, $Pa$, W, $Bin$, $Bet$, ${\chi }^2$, $Ray$, U, and $Ge$ stand for gamma, Poisson, Pareto, Weibull, binomial, beta, chi–squared, Rayleigh, uniform, and geometric distributions, respectively.

Definition 1.

In the exponential distribution family in (1), Bregman divergence associated with a real-valued strictly convex and differentiable function $c(·)$ is defined by

$$B(\eta ,\gamma )=\kappa \left(\eta \right)-\kappa \left(\gamma \right)-\langle \eta -\gamma ,\nabla c\left(\gamma \right)\rangle ,$$

(16)

where $\langle ·,·\rangle$ is the inner product and it can be shown that for all regular exponential families (see Brown [20]), $\kappa (·)$ is strictly convex, and furthermore $B(\eta ,\gamma )=log\frac{c\left(\eta \right)}{c\left(\gamma \right)}-\langle \eta -\gamma ,{\mathbb{E}}^q\left(s\left(T\right)\right)\rangle$.

Lemma 1.

The Kullback–Leibler (KL) divergence between $p_{\eta }$ and $q_{\gamma }$ in model (1) is equal to the Bregman divergence between natural parameters. That is:

$$KL\left(p\right(·\left|\eta \right),q(·|\gamma \left)\right)=B(\eta ,\gamma ),$$

where the KL divergence (Kullback–Leibler, [21]), between densities $f_1$ and $f_2$, $KL(f_1,f_2)={\mathbb{E}}^{f_1}log{f}_1/f_2$.

Proof. 

See Nielsen and Nock [22]. ☐

Lemma 2.

Consider ${\mathbb{E}}^p{\left(log\widehat{r}\left(t\right)-logr\left(t\right)\right)}^2$ as a loss function where the random variable T follows from p as in (1) and the prior function $\pi \left(\theta \right)$ on $\theta =(\eta ,\gamma )$, then the Bayesian DRE of $r(t;\theta )$ is given by

$${\widehat{r}}_{\pi }\left(t\right)=exp\left\{\mathbb{E}\left(KL(p,q)\mid X=t,Y=t\right)\right\},$$

(17)

and in addition for the natural exponential family model (1) it can also be expressed as

$${\widehat{r}}_{\pi }\left(t\right)=exp\left\{\mathbb{E}\left(B(\eta ,\gamma )|X=t,Y=t\right)\right\}.$$

(18)

Proof. 

For simplicity, assume notations ${\widehat{r}}_{\pi }\left(t\right)$ and $r(t;\eta ,\gamma )$ as $\widehat{r}$ and r respectively.

The Bayesian estimator $\widehat{r}$ in estimating r is the minimizer of $\mathbb{E}(\ell (\widehat{r},r)\mid X=t,Y=t)$, with $\ell (\widehat{r},r)={\mathbb{E}}^p{(log\frac{\widehat{r}}{r})}^2$, and therefore $\frac{\partial }{\partial log\widehat{r}}\mathbb{E}({\mathbb{E}}^p{\left(log\widehat{r}-logr\right)}^2\mid X=t,Y=t)=0$ implies $log{\widehat{r}}_{\pi }\left(t\right)=\mathbb{E}({\mathbb{E}}^{p}logr\mid X=t,Y=t)=\mathbb{E}(KL(p,q)\mid X=t,Y=t)$. Applying Lemma 1 and the fact that $\frac{{\partial }^2}{\partial log{\widehat{r}}^2}\mathbb{E}({\mathbb{E}}^p{\left(log\widehat{r}-logr\right)}^2\mid X=t,Y=t)\ge 0$, completes the proof. ☐

3.1.2. Log-$L_1$ Loss Function

For some larger errors in loss function (12), one needs to consider the log–$L_1$ loss function $\ell (\tilde{r},r)=2|log\frac{r}{\tilde{r}}|-2$. Let $\tilde{r}$ be the corresponding Bayesian DRE. As in Theorem 1 we can also express ${\tilde{r}}_{\pi }\left(t\right)$ in terms of the product of the correction factor and the plug-in DRE. That is, under log–$L_1$ loss $L(\tilde{r},r)=|log\frac{\widehat{r}}{r}|$, and prior distribution $\pi \left(\theta \right)$, for $\theta =(\eta ,\gamma )$, we have

$${\tilde{r}}_{\pi }\left(t\right)={\tilde{r}}_{plug}\left(t\right)H^{\prime }\left(t\right),$$

(19)

where ${\tilde{r}}_{plug}\left(t\right)$ and $H^{\prime }\left(t\right)$ are obtained in the same fashion in (14) and (15) except applying median $\mathbb{M}(·\mid X=t,Y=t)$ instead of $\mathbb{E}(·\mid X=t,Y=t)$. Notice that for instance the results in Table 1 hold wherever the posterior densities turn out to be symmetric about their means (or medians).

Similar calculation to Lemma 2, with loss function ${\mathbb{E}}^p|log\widehat{r}\left(t\right)-logr\left(t\right)|$, suggests

$${\tilde{r}}_{\pi }\left(t\right)=exp\left\{\mathbb{M}\left(B\right(\eta ,\gamma )\mid X=t,Y=t)\right\}.$$

Similar to (17) and (18), we can write ${\tilde{r}}_{\pi }\left(t\right)=exp\left\{\mathbb{M}\left(KL\right(p,q)\mid X=t,Y=t)\right\}$, whose expectations are replaced by medians.

3.2. Examples of the Bayesian DRE and Some Applications

For instance, we consider the problem of finding Bayesian DRE under log–$L_1$ and log–$L_2$ losses for two different normal densities.

Example 1.

Let $X|{\theta }_1\sim \mathbb{N}({\theta }_1,{\sigma }_1^2)$ be independent of $Y|{\theta }_2\sim \mathbb{N}({\theta }_2,{\sigma }_2^2),$ with the known variances and independent prior distributions ${\theta }_i\sim \mathbb{N}({\xi }_i,{\tau }_i^2)$ for $i=1,2$. Hence, the posterior densities are given by

$$\begin{array}{cc}\hfill {\theta }_1|X=t& \sim \mathbb{N}\left(\frac{{\xi }_1{\tau }_1^2+{\tau }_1^{2}t}{{\tau }_1^2+{\sigma }_1^2},\frac{{\tau }_1^2{\sigma }_1^2}{{\tau }_1^2+{\sigma }_1^2}\right),\hfill \\ \hfill {\theta }_2|Y=t& \sim \mathbb{N}\left(\frac{{\xi }_2{\tau }_2^2+{\tau }_2^{2}t}{{\tau }_2^2+{\sigma }_2^2},\frac{{\tau }_2^2{\sigma }_2^2}{{\tau }_2^2+{\sigma }_2^2}\right),\hfill \end{array}$$

yielding the Bayesian DRE under either loss functions $L(\widehat{r},r)={(log\frac{\widehat{r}}{r})}^2$ or $L(\widehat{r},r)={|log\frac{\widehat{r}}{r}|}^2$, as below:

$${\tilde{r}}_{\pi }\left(t\right)={\widehat{r}}_{\pi }\left(t\right)=\frac{{\sigma }_2}{{\sigma }_1}exp\left(\frac{{\sigma }_1^2-{\sigma }_1^2+{\left(\frac{{\xi }_1{\sigma }_1^2+t{\tau }_1^2}{{\sigma }_1^2+{\tau }_1^2}-\frac{{\xi }_2{\sigma }_2^2+t{\tau }_2^2}{{\sigma }_2^2+{\tau }_2^2}\right)}^2}{2{\sigma }_2^2}\right).$$

(20)

If we assume the multivariate case, that is, $X|{\theta }_1\sim {\mathbb{N}}_p({\theta }_1,{\Sigma }_1)$ and $Y|{\theta }_2\sim {\mathbb{N}}_p({\theta }_2,{\Sigma }_2)$, where both covariance matrices are known, since the conjugate prior for the mean is p-variate normal, ${\theta }_i\sim {\mathbb{N}}_p({\xi }_i,V_i)$ for $i=1,2$, hence the posterior is p-variate normal. The results are analogous to the univariate case and ${\theta }_1|t\sim {\mathbb{N}}_p(W_1({\Sigma }_1^{-1}t+V_1^{-1}{\xi }_1),W_1)$ and ${\theta }_2|t\sim {\mathbb{N}}_p(W_2({\Sigma }_2^{-1}t+V_2^{-1}{\xi }_2),W_2)$ with $W_i={({\Sigma }_i^{-1}+V_i^{-1})}^{-i}$. Therefore the Bayesian DRE for the ratio of two multivariate normal densities equals to

$$\begin{array}{c}\hfill \frac{1}{2}\left[{(W_1({\Sigma }_1^{-1}t+V_1^{-1}{\xi }_1)-W_2({\Sigma }_2^{-1}t+V_2^{-1}{\xi }_2))}^{\prime }{\Sigma }_2^{-1}(W_1({\Sigma }_1^{-1}t+V_1^{-1}{\xi }_1)\right.\\ \hfill \left.-W_2{({\Sigma }_2^{-1}t+V_2^{-1}{\xi }_2)}^{\prime }+tr\left({\Sigma }_2^{-1}{\Sigma }_1\right)-\frac{1}{2}log\frac{det\left({\Sigma }_1\right)}{det\left({\Sigma }_2\right)}\right].\end{array}$$

As we saw in the previous section, the key point is to find the KL divergence between two densities p and q (or equivalently, Bregman divergence in the cases of distributions belonging to exponential family), and we have a closed form for the divergence. Table 2 represents the KL divergence between p and q.

Example 2.

Let $X|{\sigma }_1^2\sim Ray\left({\sigma }_1^2\right)$ and $Y|{\sigma }_2^2\sim Ray\left({\sigma }_2^2\right)$ be two independent random variables from a Rayleigh distribution with the PDF $p\left(x\right|{\sigma }^2)=\frac{x}{{\sigma }^2}exp(-\frac{x^2}{2{\sigma }^2})$ for $x>0$ and ${\sigma }^2>0$. The KL divergence between $p\left(x\right|{\sigma }_1^2)$ and $p\left(y\right|{\sigma }_2^2)$ is given in Table 2. Assuming ${\lambda }_i=2{\sigma }_i^2$, and choosing the inverse gamma conjugate prior distribution (IG) with parameters α and β, with the PDF $\pi \left(\lambda \right|\alpha ,\beta )=\frac{{\beta }^{\alpha }}{\gamma \left(\alpha \right)}{\lambda }^{-(\alpha +1)}exp(-\frac{\beta }{\lambda })$ for $\lambda >0$, yields the posterior distribution as follows:

$${\lambda }_i\mid t\sim IG(1+{\alpha }_i,{\beta }_i+t^2).$$

(21)

Therefore, the Bayesian DRE for the ratio of two Rayleigh densities is the exponential function of the conditional expectation of their KL loss divergence function. That is,

$$exp\left({\mathbb{E}}^{{\lambda }_2|t}{\lambda }_2-{\mathbb{E}}^{{\lambda }_1|t}log{\lambda }_1+{\mathbb{E}}^{{\lambda }_1|t}{\lambda }_1{\mathbb{E}}^{{\lambda }_2|t}{\lambda }_2^{-1}-1\right).$$

By making use of properties of $\lambda \sim IG(\alpha ,\beta )$, such as $\mathbb{E}(log\lambda )=log\beta -\psi \left(\alpha \right)$, where $\psi \left(\alpha \right)$ is a “digamma” function, the Bayesian DRE under loss function ${\mathbb{E}}^p{\left(log\widehat{r}\left(t\right)-logr\left(t\right)\right)}^2$ is given by

$${\widehat{r}}_{\pi }\left(t\right)=\frac{{\beta }_2+t^2}{{\beta }_1+t^2}exp\left(\frac{{\alpha }_2-{\alpha }_1}{{\alpha }_1{\alpha }_2}+\frac{{\alpha }_1}{1+{\alpha }_2}\frac{{\beta }_1+t^2}{{\beta }_2+t^2}-\psi \left({\alpha }_2\right)+\psi \left({\alpha }_1\right)-1\right).$$

Since the median of an inverse gamma distribution does not have a closed form, we cannot express an explicit formula for the Bayesian DRE under loss function ${\mathbb{E}}^p|log\widehat{r}\left(t\right)-logr\left(t\right)|$, and consequently for the log–Huber loss function, and they must be calculated iteratively.

The following remarks are some clarifications related to Table 2 and the obtained Bayes estimators in the previous examples in Section 3.2.

Remark 1.

The Bayes DRE ${\widehat{r}}_{\pi }\left(t\right)$ is connected to samples X and Y via the posterior density $\eta \mid X=t$ and $\gamma \mid Y=t$.

Remark 2.

From Table 2 it can be seen that the KL divergence between two log–normal PDFs is the same as in normal distributions, since it is known that KL is invariant under parameter transformations.

Example 3.

Suppose $X\sim Uniform(0,{\theta }_1)$ is independent of $Y\sim Uniform(0,{\theta }_2)$, with ${\theta }_1<{\theta }_2$. It is easy to check that the Pareto distribution is conjugate to a uniform. Hence, assume that $\pi \left({\theta }_i\right)={\alpha }_i{\beta }_i^{{\alpha }_i}{\theta }_i^{-({\alpha }_1+1)}$ for $\theta \ge {\beta }_i$ and $i=1,2$. Therefore,

$${\theta }_i\mid t\sim Pareto(1+{\alpha }_i,max({\beta }_i,t)).$$

Moreover, assuming p and q are the PDFs of X and Y, respectively, $KL(p,q)=log{\theta }_2-log{\theta }_1$ (see Table 2). Employing the fact that the transformation $log(z/b)$ has an exponential distribution with mean of a, given $Z\sim Pareto(a,b)$, and hence ${\mathbb{E}}^{Z}logZ=a+logb$, after some calculation we have the Bayes DRE associated with loss function ${\mathbb{E}}^p{\left(log\widehat{r}\left(t\right)-logr\left(t\right)\right)}^2$:

$${\widehat{r}}_{\pi }\left(t\right)=\frac{max({\beta }_2,t)}{max({\beta }_1,t)}+{\alpha }_2-{\alpha }_1.$$

Similar to Example 2, the Bayes DRE under ${\mathbb{E}}^p|log\widehat{r}\left(t\right)-logr\left(t\right)|$ must be calculated numerically.

4. Other Applications

Here, we discuss some of other applications of the DRE method.

1

Estimating the $\mathit{\alpha }$-divergence function between two probability densities:

A discrepancy measure between densities $p_{\eta }$ and $q_{\gamma }$ applicable to the class of Ali-Silvey [23] distance, also known as $\alpha$-divergence (Csiszàr, [24]), is given by

$${\ell }_{\alpha }(p(·\mid \eta ),q(·|\gamma ))={\int }_{{\mathbb{R}}^d}{h}_{\alpha }\left(\frac{p\left(t\right|\eta )}{q\left(t\right|\gamma )}\right)dP\left(t\right|\gamma ),$$

(22)

where

$$h_{\alpha }\left(z\right)=\left\{\begin{array}{c}\frac{4}{1-{\alpha }^2}(1-z^{-(1+\alpha )/2})\mathrm{for}\left|\alpha \right|\le 1,\hfill \\ -log\left(z\right)/z\mathrm{for}\alpha =-1,\hfill \\ log\left(z\right)\mathrm{for}\alpha =1.\hfill \end{array}\right.$$

Note that some of the notable divergence functions, such as Kullback–Leibler, reverse Kullback–Leibler (RKL), and Hellinger divergence functions correspond to $\alpha =1,-1$, and 0, respectively, and belong to this class. So, if $r\left(t\right)$ is estimated by ${\widehat{r}}_{\pi }\left(t\right)$ under log–$L_2$ (or ${\tilde{r}}_{\pi }$(t) under log–$L_1$ losses), then applying the Monte Carlo approximations method, the $\alpha$-divergence is also estimated by

$$d_{\alpha }\left(t\right)=1/n\sum _{i=1}^n{h}_{\alpha }\left({\widehat{r}}_{\pi }\left(t_i\right)\right),$$

(23)

where $t_i$ are drawn from $p(·\mid \eta )$. Note that there are other works in order to estimate the $\alpha$-divergence loss function (e.g., Póczos and Schneider [25]), but our estimator in (23) is based on the Bayesian parametric method based on the DRE. In the next section we show the performance of the proposal estimator $d_{\alpha }$.

2

Plug-in-type estimation of the density ratio under KL loss function:

Consider the plug-in density estimator, say $p_{\widehat{\eta }}\left(t\right)$, for estimating $p_{\eta }$ in the exponential family (1), based on the KL loss. We have

$$\begin{array}{cc}\hfill KL(p\left(t\right|\eta ),p\left(t\right|\widehat{\eta }))& =\int log\frac{p\left(\eta \right|t)}{p\left(t\right|\widehat{\eta })}P\left(t\right|\eta )dt\hfill \\ & =\int p\left(t\right|\widehat{\eta })log\frac{exp\left\{\langle \eta ,s\left(t\right)\rangle -c\left(\eta \right)\right\}}{exp\left\{\langle \widehat{\eta }\left(t\right),s\left(t\right)\rangle -c\left(\widehat{\eta }\left(t\right)\right)\right\}}dt\hfill \\ & =\langle \eta -\widehat{\eta }\left(t\right),{\mathbb{E}}^{p}s\left(T\right)\rangle -\left(c\left(\eta \right)-c(\widehat{\eta }\left(x\right)\right)\hfill \\ & =\langle \eta -\widehat{\eta }\left(t\right),\frac{\partial c\left(\eta \right)}{\partial \eta }\rangle -\left(c\left(\eta \right)-c(\widehat{\eta }\left(t\right)\right).\hfill \end{array}$$

(24)

Next, finding the plug-in density estimator $p(t,\widehat{\eta }\left(t\right))$ that minimizes KL loss is equivalent to finding the point estimator $\widehat{\eta }\left(t\right)$ which minimizes the posterior expectation associated with the loss function in (24). Therefore, $\frac{\partial }{\partial \widehat{\eta }}{\mathbb{E}}^{\eta |t}KL(p\left(t\right|\eta ),p\left(t\right|\widehat{\eta }))=0$ implies $c_1\left(\widehat{\eta }\right)={\mathbb{E}}^{\eta |t}\frac{c\left(\eta \right)}{\eta }$, and hence the Bayes estimator of $\eta$ is given by

$${\widehat{\eta }}_1\left(t\right)=c_1^{-1}\left({\mathbb{E}}^{\eta |t}\frac{c_1\left(\eta \right)}{\eta }\right).$$

(25)

Similar arguments can be applied to $q_{\widehat{\gamma }}\left(t\right)$ for estimating $q_{\gamma }$ in the exponential family (2), and we have:

$${\widehat{\gamma }}_2\left(y\right)=c_2^{-1}\left({\mathbb{E}}^{\gamma |t}\frac{c_2\left(\gamma \right)}{\gamma }\right).$$

(26)

By setting the case when both densities follow the identical distribution from (1) (e.g., the ratio of two normal or two Poisson, etc.), substituting the Bayes estimators obtained in (25) and (26) into the plug-in estimator of $r\left(t\right)$ gives

$$\widehat{r}\left(t\right)=exp\{{({\widehat{\eta }}_1\left(t\right)-{\widehat{\eta }}_2\left(t\right))}^{\top }s\left(t\right)-\left(c_1({\widehat{\eta }}_1\left(t\right)-c_2({\widehat{\eta }}_2\left(t\right)\right)\}.$$

(27)

Note that $\tilde{r}\left(t\right)$ can be obtained similarly by replacing posterior medians ${\tilde{\eta }}_i$ instead of posterior expectations ${\widehat{\eta }}_i$ for $i=1,2$ above.

5. Numerical Illustrations

We conclude this section with some numerical illustrations of log–Huber risk performance of the Bayesian and plug-in when both p and q are two normal models (belong to model (1) and (2)) with a common location parameter $\theta$. We show that the performance of plug-in and Bayes are quite similar and by selecting the hyper–parameters these two density ratio estimators coincide and hence have the same frequentist risk. We start with comparing risk performance under the log–$L_2$ first and then extend them to log–Huber loss.

Figure 2 exhibits the frequentist risk performance of the plug-in DRE ${\widehat{r}}_{plug}$ and the Bayes DRE ${\widehat{r}}_{\pi }$ under log–$L_2$ for all possible values of $\theta$ using Theorem 1.

Figure 3 shows the changing frequentist risk performance of the plug-in DRE ${\widehat{r}}_{plug}$ and the Bayes DRE ${\widehat{r}}_{\pi }$ under log–$L_2$ when the ratio of variances ${\sigma }_1^2/{\sigma }_2^2$ varies (means are known) using Theorem 1. It can be seen the performance of plug-in estimation is better for larger ratios of variances.

Finally, Figure 4 and Figure 5 depict the risk functions of the Bayes and the plug-in DRE under the log–Huber loss over possible ranges of the mean and variance, respectively, in Example 1. It is clear from Figure 5, in contrast to Figure 3 (risk functions of Bayes DRE under log–$L2$ and plug-in DRE estimator), that the Bayes estimator DRE has a lower risk with respect to the plug-in DRE, and hence is more robust under the variation of variances.

6. Concluding Remarks

Estimation pf the ratio of two or more densities has received widespread attention in recent years. Until now, most of the methods have been concentrated on solving this problem via nonparametric approaches. In this paper, we focused on a parametric Bayesian approach, when distributions come from the canonical form of the exponential family. We applied the log–Huber loss function to investigate the utility of the Bayesian and plug-in DREs. Our results confirm that Bayesian DRE along with the plug-in DRE (based on posterior expectations) perform similarly under log–Huber loss functions with the possibility of being exactly equal when the correction factor $H=1$. This is a somewhat different result from a non-parametric point of view, which often happens when the plug-in estimators perform poorly as opposed to empirical non-parametric Bayesian methods which typically include stochastic processes such as the Gaussian process and the Dirichlet process. There are instances (see Krnjajic et al. [26] and the references therein) where for certain types of count data, the nonparametric Bayesian methodology provides enhanced flexibility to fit the data, provides rich posterior inferences, and provides finer predictive inference under a set of carefully selected criteria. However, there is an apparent major drawback. These processes have an infinite number of dimensions, and thus naive algorithmic approaches to compute posteriors are generally infeasible. Finally, the application to estimate the $\alpha$-divergence between two PDFs was discussed.