Proximal Nested Sampling with Data-Driven Priors for Physical Scientists ^†

Abstract

Proximal nested sampling was introduced recently to open up Bayesian model selection for high-dimensional problems such as computational imaging. The framework is suitable for models with a log-convex likelihood, which are ubiquitous in the imaging sciences. The purpose of this article is two-fold. First, we review proximal nested sampling in a pedagogical manner in an attempt to elucidate the framework for physical scientists. Second, we show how proximal nested sampling can be extended in an empirical Bayes setting to support data-driven priors, such as deep neural networks learned from training data.

1. Introduction

In many of the sciences, not only is one interested in estimating the parameters of an underlying model, but deciding which model is best among a number of alternatives is of critical scientific interest. Bayesian model comparison provides a principled approach to model selection [1] that has found widespread application in the sciences [2].

Bayesian model comparison requires computation of the model evidence:

$$\mathcal{Z}=p\left(y|M\right)=\int \mathrm{d}xp(y|x,M)p\left(x|M\right)=\int \mathrm{d}x\mathcal{L}\left(x\right)\pi \left(x\right),$$

(1)

also called the marginal likelihood, where $y\in {\mathbb{R}}^m$ denotes data, $x\in {\mathbb{R}}^n$ parameters of interest, and M the model under consideration. We adopt the shorthand notation for the likelihood of $\mathcal{L}\left(x\right)=p(y|x,M)$ and prior of $\pi \left(x\right)=p\left(x|M\right)$. Evaluating the multi-dimensional integral of the model evidence is computationally challenging, particularly in high dimensions. While a number of highly successful approaches to computing the model evidence have been developed, such as nested sampling (e.g., [2,3,4,5,6,7,8]) and the learned harmonic mean estimator [9,10,11] Need to update arxiv number of normalizing flow harmonic paper when have it., previous approaches do not scale to the very high-dimensional settings of computational imaging, which is our driving motivation.

The proximal nested sampling framework was introduced recently by a number of authors of the current article in order to open up Bayesian model selection for high-dimensional imaging problems [12]. Proximal nested sampling is suitable for models for which the likelihood is log-convex, which are ubiquitous in the imaging sciences. By restricting the class of models considered, it is possible to exploit structure of the problem to enable computation in very high-dimensional settings of $\mathcal{O}\left({10}^6\right)$ and beyond.

Proximal nested sampling draws heavily on convex analysis and proximal calculus. In this article we present a pedagogical review of proximal nested sampling, sacrificing some mathematical rigor in an attempt to provide greater accessibility. We also provide a concise review of convexity and proximal calculus to introduce the background underpinning the framework. We assume the reader is familiar with nested sampling, and hence we avoid repeating an introduction to nested sampling and instead refer the reader to other sources that provide excellent descriptions [2,3,8]. Finally, for the first time, we show in an empirical Bayes setting how proximal nested sampling can be extended to support data-driven priors, such as deep neural networks learned from training data.

2. Convexity and Proximal Calculus

We present a concise review of convexity and proximal calculus to introduce the background underpinning proximal nested sampling to make it more accessible.

2.1. Convexity

Proximal nested sampling draws on convexity, key concepts of which are illustrated in Figure 1. A set $\mathcal{C}$ is convex if for any $x_1,x_2\in \mathcal{C}$ and $\alpha \in (0,1)$ we have $\alpha x_1+(1-\alpha )x_2\in \mathcal{C}$. The epigraph of a function $f:{\mathbb{R}}^n\to \mathbb{R}$ is defined by $\mathrm{epi}\left(f\right)=\{(x,\gamma )\in {\mathbb{R}}^n\times \mathbb{R}|f\left(x\right)\le \gamma \}$. The function f is convex if and only if its epigraph is convex. A convex function is lower semicontinuous if its epigraph is closed (i.e., includes its boundary).

2.2. Proximity Operator

Proximal nested sampling leverages proximal calculus [13,14], a key component of which is the proximity operator, or prox. The proximity operator of the function f with parameter $\lambda$ is defined by

$${\mathrm{prox}}_f^{\lambda }\left(x\right)=\underset{u}{argmin}\left[{f\left(u\right)+\parallel u-x\parallel }^2/2\lambda \right].$$

(2)

The proximity operator maps a point x towards the minimum of f while remaining in the proximity of the original point. The parameter $\lambda$ controls how close the mapped point remains to x. An illustration is given in Figure 2.

The proximity operator can be considered as a generalisation of the projection onto a convex set. Indeed, the projection operator can be expressed as a prox by

$${\Pi }_{\mathcal{C}}\left(x\right)=\underset{u}{argmin}\left[{\chi }_{\mathcal{C}}\left(u\right)+{\parallel u-x\parallel }^2/2\right],$$

(3)

with function f given by the characteristic function ${\chi }_{\mathcal{C}}\left(x\right)=\infty$ if $x\notin \mathcal{C}$ and zero otherwise.

2.3. Moreau–Yosida Regularisation

The final component required in the development of proximal nested sampling is Moreau–Yosida regularisation (e.g., [14]). The Moreau–Yosida envelop of a convex function $f:{\mathbb{R}}^n\to \mathbb{R}$ is given by the infimal convolution:

$$f^{\lambda }\left(x\right)=\underset{u\in {\mathbb{R}}^N}{inf}f\left(u\right)+\frac{{\parallel u-x\parallel }^2}{2\lambda }.$$

(4)

The Moreau–Yoshida envelope of a function can be interpreted as taking its convex conjugate, adding regularisation, before taking the conjugate again [14]. Consequently, it provides a smooth regularised approximation of f, which is very useful to enable the use of gradient-based computational algorithms (e.g., [15]).

The Moreau–Yosida envelope exhibits the following properties. First, $\lambda$ controls the degree of regularisation with $f^{\lambda }\left(x\right)\to f\left(x\right)$ as $\lambda \to 0$. Second, the gradient of the Moreau–Yosida envelope of f can be computed through its prox by $\nabla f^{\lambda }\left(x\right)=(x-{\mathrm{prox}}_f^{\lambda }\left(x\right))/\lambda$.

3. Proximal Nested Sampling

The challenge of nested sampling in high-dimensional settings is to sample from the prior distribution subject to a hard likelihood constraint [2,3,8]. Proximal nested sampling addresses this challenge for the case of log-convex likelihoods, which are widespread in computational imaging problems. In this section we review the proximal nested sampling framework [12] in a pedagogical manner, sacrificing some mathematical rigor in an attempt to improve readability and accessibility.

3.1. Constrained Sampling Formulation

Consider a prior and likelihood $\pi \left(x\right)\propto \exp (-f(x\left)\right)$ and $\mathcal{L}\left(x\right)\propto \exp (-g(x\left)\right)$, where the log-likehood $g=-log\mathcal{L}$ is a convex lower semicontinuous function. The log-prior $f=-log\pi$ need only be differentiable or convex (it need not be convex if it is differentiable).

We consider sampling from the prior $\pi \left(x\right)$, such that $\mathcal{L}\left(x\right)>L^{*}$ for some likelihood value $L^{*}\ge 0$. Let ${\iota }_{L^{*}}\left(x\right)$ and ${\chi }_{L^{*}}\left(x\right)$ be the indicator function and characteristic function corresponding to this constraint, respectively, defined as

$${\iota }_{L^{*}}\left(x\right)=\left\{\begin{array}{cc}1,\hfill & \mathcal{L}\left(x\right)>L^{*},\hfill \\ 0,\hfill & \mathrm{otherwise},\hfill \end{array}\right.\mathrm{and}{\chi }_{L^{*}}\left(x\right)=\left\{\begin{array}{cc}0,\hfill & \mathcal{L}\left(x\right)>L^{*},\hfill \\ +\infty ,\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

(5)

Since log is monotonic, $\mathcal{L}\left(x\right)>L^{*}$ is equivalent to $g\left(x\right)<\tau$ for $\tau =-log{L}^{*}$. Explicitly define the convex set of the likelihood constraint by ${\mathcal{B}}_{\tau }=\left\{x\right|g\left(x\right)<\tau \}$. Then, ${\chi }_{L^{*}}\left(x\right)$ is equivalent to ${\chi }_{{\mathcal{B}}_{\tau }}\left(x\right)$, where ${\chi }_{{\mathcal{B}}_{\tau }}\left(x\right)=\infty$ if $x\notin {\mathcal{B}}_{\tau }$ and zero otherwise.

Let ${\pi }_{L^{*}}\left(x\right)=\pi \left(x\right){\iota }_{L^{*}}\left(x\right)$ represent the prior distribution with the hard likelihood constraint $\mathcal{L}\left(x\right)>L^{*}$. Since ${\iota }_{L^{*}}\left(x\right)=\exp (-{\chi }_{L^{*}}\left(x\right))$, then we have

$$-log{\pi }_{L^{*}}\left(x\right)=-log\pi \left(x\right)+{\chi }_{{\mathcal{B}}_{\tau }}\left(x\right).$$

(6)

To sample from the constrained prior we require sampling techniques that firstly can scale to high-dimensional settings and that secondly can support the convex constraint ${\chi }_{{\mathcal{B}}_{\tau }}\left(x\right)$.

3.2. Langevin MCMC Sampling

Langevin Markov chain Monte Carlo (MCMC) sampling has been demonstrated to be highly effective at sampling in high-dimensional settings by exploiting gradient information [15,16]. The Langevin stochastic differential equation associated with distribution $p\left(x\right)$ is a stochastic process defined by

$$\mathrm{d}x\left(t\right)=\frac{1}{2}\nabla logp\left(x\left(t\right)\right)\mathrm{d}t+\mathrm{d}w\left(t\right),$$

(7)

where $w\left(t\right)$ is Brownian motion. This process converges to $p\left(x\right)$ as time t increases and is therefore useful for generating samples from $p\left(x\right)$. In practice, we compute a discrete-time approximation of $x\left(t\right)$ by the conventional Euler–Maruyama discretisation:

$$x^{(k+1)}=x^{\left(k\right)}+\frac{\delta }{2}\nabla logp\left(x^{\left(k\right)}\right)+\sqrt{\delta }{w}^{(k+1)},$$

(8)

where $w^{\left(k\right)}$ is a sequence of standard Gaussian random variables and $\delta$ is a step size.

Equation (8) provides a strategy for sampling in high-dimensions. However, notice that the updates rely on the score of the target distribution $\nabla logp(·)$. Nominally the target distribution must therefore be differentiable, which is not the case for our target of interest given by Equation (6). The prior may or may not be differentiable but the likelihood constraint certainly is not. Proximal versions of Langevin sampling have been developed to address the setting where the distribution is log-convex but not necessarily differentiable [15,16]. We follow a similar approach.

3.3. Proximal Nested Sampling Framework

The proximal nested sampling framework follows by taking the constrained sampling formulation of Equation (6), adopting Langevin MCMC sampling of Equation (8), and applying Moreau–Yosida regularisation of Equation (4) to the convex constraint ${\chi }_{{\mathcal{B}}_{\tau }}\left(x\right)$ to yield a differentiable target. This strategy yields (see [12]) the update equation:

$$x^{(k+1)}=x^{\left(k\right)}+\frac{\delta }{2}\nabla log\pi \left(x^{\left(k\right)}\right)-\frac{\delta }{2\lambda }\left[x^{\left(k\right)}-{\mathrm{prox}}_{{\chi }_{{\mathcal{B}}_{\tau }}}\left(x^{\left(k\right)}\right)\right]+\sqrt{\delta }{w}^{(k+1)},$$

(9)

where $\delta$ is the step size and $\lambda$ is the Moreau–Yosida regularisation parameter.

Further intuition regarding proximal nested sampling can be gained by examining the term $v^{\left(k\right)}=-[x^{\left(k\right)}-{\mathrm{prox}}_{{\chi }_{{\mathcal{B}}_{\tau }}}\left(x^{\left(k\right)}\right)]$, together with Figure 3. The vector $v^{\left(k\right)}$ points from the sample $x^{\left(k\right)}$ to its projection onto the likelihood constraint. If the sample $x^{\left(k\right)}$ is already in the likelihood-restricted prior support ${\mathcal{B}}_{\tau }$, i.e., $x\in {\mathcal{B}}_{\tau }$, the term $v^{\left(k\right)}$ disappears and the Markov chain iteration simply involves the standard Langevin MCMC update. In contrast, if $x^{\left(k\right)}$ is not in ${\mathcal{B}}_{\tau }$, i.e., $x\notin {\mathcal{B}}_{\tau }$, then a step is taken in the direction $v^{\left(k\right)}$, which acts to move the next iteration of the Markov chain in the direction of the projection of $x^{\left(k\right)}$ onto the convex set ${\mathcal{B}}_{\tau }$. This term therefore acts to push the Markov chain back into the constraint set ${\mathcal{B}}_{\tau }$ if it wanders outside of it. (Note that proximal nested sampling has some similarity with Galilean [17] and constrained Hamiltonian [18] nested sampling. In these approaches, Makov chains are also considered and if the Markov chain steps outside of the likelihood-constraint then it is reflected by an approximation of the shape of the boundary.)

We have so far assumed that the (log) prior is differentiable (see Equation (9)). This may not be the case, as is typical for sparsity-promoting priors (e.g., $-log\pi \left(x\right)=\parallel {\Psi }^{†}{x\parallel }_1+\mathrm{const}.$ for some wavelet dictionary $\Psi$). Then, we make a Moreau–Yosida approximation of the log-prior, yielding the update equation:

$$x^{(k+1)}=x^{\left(k\right)}-\frac{\delta }{2\lambda }\left[x^{\left(k\right)}-{\mathrm{prox}}_{-log\pi }^{\lambda }\left(x^{\left(k\right)}\right)\right]-\frac{\delta }{2\lambda }\left[x^{\left(k\right)}-{\mathrm{prox}}_{{\chi }_{{\mathcal{B}}_{\tau }}}\left(x^{\left(k\right)}\right)\right]+\sqrt{\delta }{w}^{(k+1)}.$$

(10)

For notational simplicity here we have adopted the same regularisation parameter $\lambda$ for each Moreau–Yosida approximation.

With the current formulation we are not guaranteed to recover samples from the prior subject to the hard likelihood constraint due to the approximation introduced in the Moreu–Yosida regularisation and due to the approximation in discretising the underlying Langevin stochastic differential equation. We therefore introduce a Metropolis–Hastings correction step to eliminate the bias introduced by these approximations and ensure convergence to the required target density (see [12] for further details).

Finally, we adopt this strategy for sampling from the constrained prior in the standard nested sampling strategy to recover the proximal nested sampling framework. The algorithm can be initalised with samples from the prior as described by the update equations above but with the likelihood term removed, i.e., with $[x^{\left(k\right)}-{\mathrm{prox}}_{{\chi }_{{\mathcal{B}}_{\tau }}}\left(x^{\left(k\right)}\right)]\to 0$.

3.4. Explicit Forms of Proximal Nested Sampling

While we have discussed the general framework for proximal nested sampling, we have yet to address the issue of computing the proximity operators involved. As Equation (2) demonstrates, computing proximity operators involves solving an optimisation problem. Only in certain cases are closed form solutions available [13]. Explicit forms of proximal nested sampling must therefore be considered for the problem at hand.

We focus on a common high-dimensional inverse imaging problem where we acquire noisy observations, $y=\Phi x+n$, of an underlying image x via some measurement model $\Phi$, in the presence of Gaussian noise n (without loss of generality we consider independent and identically distributed noise here). We consider a Gaussian negative likelihood, $-log\mathcal{L}\left(x\right)={∥y-\Phi x∥}_2^2/2{\sigma }^2+\mathrm{const}.$, and a sparsity-promoting prior, $-log\pi \left(x\right)=\mu {∥{\Psi }^{†}x∥}_1+\mathrm{const}.$, for some wavelet dictionary $\Psi$. The prox of the prior can be computed in closed-form by [13]

$${\mathrm{prox}}_{-log\pi }^{\lambda }\left(x\right)=x+\Psi \left({\mathrm{soft}}_{\lambda \mu }\left({\Psi }^{†}{x}^{\prime }\right)-{\Psi }^{†}x\right),$$

(11)

where ${\mathrm{soft}}_{\lambda }(·)$ is the soft thresholding function with threshold $\lambda$ (recall $\mu$ is the scale of the sparsity-promoting prior, i.e., the regularisation parameter, defined above). However, the prox of the likelihood is not so straightforward. The prox for the likelihood can be recast as a saddle-point problem that can be solved iteratively by a primal dual method initialised by the current sample position (see [12] for further details):

• $z^{(i+1)}=z^{\left(i\right)}+{\delta }_1\Phi {\overline{x}}^{\left(i\right)}-{\mathrm{prox}}_{{\chi }_{{\mathcal{B}}_{{\tau }^{\prime }}^{\prime }}}(z^{\left(i\right)}+{\delta }_1\Phi {\overline{x}}^{\left(i\right)}),$

  $$\mathrm{where}{\mathrm{prox}}_{{\chi }_{{\mathcal{B}}_{{\tau }^{\prime }}^{\prime }}}\left(z\right)={\mathrm{proj}}_{{\mathcal{B}}_{{\tau }^{\prime }}^{\prime }}\left(z\right)=\left\{\begin{array}{cc}z,\hfill & \mathrm{if}z\in {\mathcal{B}}_{{\tau }^{\prime }}^{\prime },\hfill \\ \frac{z-y}{{\parallel z-y\parallel }_2}\sqrt{2\tau {\sigma }^2}+y,\hfill & \mathrm{otherwise};\hfill \end{array}\right.$$
• $x^{(i+1)}=(x^{\prime }+x^{\left(i\right)}-{\delta }_2{\Phi }^{†}{z}^{(i+1)})/2$;
• ${\overline{x}}^{(i+1)}=x^{(i+1)}+{\delta }_3(x^{(i+1)}-x^{\left(i\right)})$.

Combining these algorithms to efficiently compute prox operators with the proximal nested sampling framework, we can compute the model evidence to perform Bayesian model comparison in high-dimensional settings. We can also obtain posterior distributions with the usual weighted samples from the dead points of nested sampling. This allows one to recover, for example, point estimates such as the posterior mean image.

4. Deep Data-Driven Priors

While hand-crafted priors, such as wavelet-based sparsity promoting priors, are common in computational imaging, they provide only limited expressivity. If example images are available an empirical Bayes approach with data-driven priors can be taken, where the prior is learned from training data. Since proximal nested sampling requires only the log-likelihood to be convex, complex data-driven priors, such as represented by deep neural networks, can be integrated into the framework. Through Tweedie’s formula we describe how proximal nested sampling can be adapted to support data-driven priors, opening up Bayesian model selection for data-driven approaches. We take a similar approach to [19], where data-driven priors are integrated into Langevin MCMC sampling strategies, although in that work model selection is not considered.

4.1. Tweedie’s Formula and Data-Driven Priors

Tweedie’s formula is a remarkable result in Bayesian estimation credited to personal correspondence with Maurice Kenneth Tweedie [20]. Tweedie’s formula has gained renewed interest in recent years [19,21,22,23] due to its connection to score matching [24,25,26] and denoising diffusion models [27,28], which, as of this writing, provide state-of-the-art performance in deep generative modelling.

Tweedie’s result follows by considering the following scenario. Consider x sampled from a prior distribution $q(·)$ and noisy observations $z\sim \mathcal{N}(x,{\sigma }^{2}I)$. Tweedie’s formula gives the posterior expectation of x given z as

$$E\left(x\right|z)=z+{\sigma }^2\nabla logp\left(z\right),$$

(12)

where $p\left(z\right)$ is the marginal distribution of z (for further details see, e.g., [21]). The critical advantage of Tweedie’s formula is that it does not require knowledge of the underlying distribution $q(·)$ but rather only the marginalised distribution of the observation. Equation (12) can be interpreted as a denoising strategy to estimate x from noisy observations z. Moreover, Tweedie’s formula can also be used to relate a denoiser (potentially a trained deep neural network) to the score $\nabla logp\left(z\right)$.

In a data-driven setting, where the underlying prior is implicitly specified by training data (which are considered to be samples from the prior), there is no guarantee that the underlying prior, and therefore the posterior, is well-suited for gradient-based Bayesian computation such as Langevin sampling; e.g., it may not be differentiable. Therefore, we consider a regularised version of the prior defined by Gaussian smoothing:

$$p_{ϵ}\left(x\right)={\left(2\pi ϵ\right)}^{-n/2}\int \mathrm{d}{x}^{\prime }\exp \left(\right|x-x^{\prime }{\parallel }_2^2/\left(2ϵ\right))q\left(x^{\prime }\right).$$

(13)

This regularisation can also be viewed as adding a small amount of regularising Gaussian noise. We can therefore leverage Tweedie’s formula to relate the regularised prior distribution $p_{ϵ}\left(x\right)$ to a denoiser $D_{ϵ}$ trained to recover x from noisy observations $x_{ϵ}\sim \mathcal{N}(x,ϵI)$; i.e., the score of the regualised prior can be related to the denoiser by

$$\nabla log{p}_{ϵ}\left(x\right)={ϵ}^{-1}(D_{ϵ}\left(x\right)-x).$$

(14)

Denoisers are commonly integrated in proximal optimisation algorithms in replacement of proximity operators, giving rise to so-called plug-and-play (PnP) methods [29,30] and, more recently, also into Bayesian computational algorithms [19]. Typically, denoisers are represented by deep neural networks, which can be trained by injecting a small amount of noise in training data and learning to denoise the corrupted data. While a noise level $ϵ$ needs to be chosen, as discussed above, this is considered a regularisation of the prior and so the denoiser need not be trained on the noise level of a problem at hand. In this manner, the same denoiser can be used for multiple subsequent problems (hence the PnP name). The learned score of the regularised prior inherits the same properties as the denoiser, such as smoothness, and hence the denoiser should be considered carefully. Well-behaved denoisers have been considered already in PnP methods (in order to provide convergence guarantees) and a popular approach for imaging problems is the DnCNN model [30], which is based on a deep convolutional neural network and that is (Lipschitz) continuous.

4.2. Proximal Nested Sampling with Data-Driven Priors

By Tweedie’s formula, the standard proximal nested sampling update of Equation (9) can be revised to integrate a learned denoiser, yielding

$$x^{(k+1)}=x^{\left(k\right)}-\frac{\alpha \delta }{2ϵ}\left[x-D_{ϵ}\left(x^{\left(k\right)}\right)\right]-\frac{\delta }{2\lambda }\left[x^{\left(k\right)}-{\mathrm{prox}}_{{\chi }_{{\mathcal{B}}_{\tau }}}\left(x^{\left(k\right)}\right)\right]+\sqrt{\delta }{w}^{(k+1)},$$

(15)

where we have included a regularisation parameter $\alpha$ that allows us to balance the influence of the prior and the data fidelity terms [19]. We typically consider a deep convolutional neural network based on the DnCNN model [30] since it is (Lipschitz) continuous and has been demonstrated to perform very well in PnP settings [19,30]. Again, this sampling strategy can then be integrated into the standard nested sampling framework.

We can therefore support data-driven priors in the proximal nested sampling framework by integrating a deep denoiser that learns to denoise training data, using Tweedie’s formula to relate this to the score of a regularised data-driven prior.

5. Numerical Experiments

The new methodology presented allows us to perform Bayesian model comparison between a data-driven and hand-crafted prior (validation of proximal nested sampling in a setting where the ground truth can be computed directly has been performed already [12]). We consider a simple radio interferometric imaging reconstruction problem as an illustration. We assume the same observational model as Section 3.4, with white Gaussian noise giving a signal-to-noise ratio (SNR) of 15 dB. The measurement operator $\Phi$ is a masked Fourier transform as a simple model of a radio interferometric telescope. The mask is built by randomly selecting $50\%$ of the Fourier coefficients. A Gaussian likelihood is used in both models. For the hand-crafted prior, we consider a sparsity-promoting prior using a Daubechis 6 wavelet dictionary. We base the data-driven prior on a DnCNN [30] model trained on galaxy images extracted from the IllustrisTNG simulations [31]. We also consider an IllustrisTNG galaxy simulation, not used in training, as the ground-truth test image. We generate samples following the proximal nested sampling strategies of Equations (10) and  (15) for the hand-crafted and data-driven priors, respectively. Posterior inferences (e.g., posterior mean image) and the model evidence can then be computed from nested sampling samples in the usual manner. The step size $\delta$ is set to ${10}^{-7}$, the Moreau–Yosida regularisation parameter $\lambda$ to $5\times {10}^{-7}$, and the regularisation strength of wavelet-based model $\mu$ to $5\times {10}^4$. We consider noise level $ϵ\simeq 8.34$ and set the regularisation parameter $\alpha$ of the data-driven prior to $3.5\times {10}^{-7}$. For the nested sampling methods, the number of live and dead samples is set to ${10}^2$ and $2.5\times {10}^3$, respectively. For the Langevin sampling, we use a thinning factor of 20 and set the number of burn-in iterations to ${10}^2$.

Results are presented in Figure 4. The data-driven prior results in a superior reconstruction with an improvement in SNR of $1.2$ dB, although it may be difficult to tell simply from visual inspection of the recovered images. Computing the SNR of the reconstructed images requires knowledge of the ground truth, which clearly is not accessible in realistic settings involving real observational data. The Bayesian model evidence, computed by proximal nested sampling, proves a way to compare the hand-crafted and data-driven models without requiring knowledge of the ground truth and is therefore applicable in realistic scenarios. We compute log evidences of $-2.96\times {10}^3$ for the hand-crafted prior and $-1.35\times {10}^3$ for the data-driven prior. Consequently, the data-driven model is preferred by the model evidence, which agrees with the SNR levels computed from the ground truth. These results are all as one might expect since learned data-driven priors are more expressive than hand-crafted priors and can better adapt to model high-dimensional images.

6. Conclusions

Proximal nested sampling leverages proximal calculus to extend nested sampling to high-dimensional settings for problems involving log-convex likelihoods, which are ubiquitous in computational imaging. The purpose of this article is two-fold. First, we review proximal nested sampling in a pedagogical manner in an attempt to elucidate the framework for physical scientists. Second, we show how proximal nested sampling can be extended in an empirical Bayes setting to support data-driven priors, such as deep neural networks learned from training data. We show only preliminary results for learned proximal nested sampling and will present a more thorough study in a follow-up article.