A Variational Surface-Evolution Approach to Optimal Transport over Transitioning Compact Supports with Domain Constraints

Abstract

We examine the optimal mass transport problem in ${\mathbb{R}}^n$ between densities with transitioning compact support by considering the geometry of a continuous interpolating support boundary $\mathsf{\Gamma }$ in space-time within which the mass density evolves according to the fluid dynamical framework of Benamou and Brenier. We treat the geometry of this space-time embedding in terms of points, vectors, and sets in ${\mathbb{R}}^{n+1}=\mathbb{R}\times {\mathbb{R}}^n$ and blend the mass density and velocity as well into a space-time solenoidal vector field $\mathbf{W}|\mathsf{\Omega }\to {\mathbb{R}}^{n+1}$ over a compact set $\mathsf{\Omega }\subset {\mathbb{R}}^{n+1}$. We then formulate a joint optimization for $\mathbf{W}$ and its support using the shaped gradient of the space-time surface $\mathsf{\Gamma }$ outlining the support boundary $\partial \mathsf{\Omega }$. This easily accommodates spatiotemporal constraints, including obstacles or mandatory regions to visit.

1. Introduction

Optimal mass transport (OMT) has a long history, beginning with Gaspard Monge [1] in 1781, and put into a more modern form solvable via linear programming by Leonid Kantorovich [2,3,4,5,6,7]. In recent years, OMT has undergone a huge surge, with many diverse applications including signal/image processing, computer vision, machine learning, data analysis, meteorology, statistical physics, quantum mechanics, and network theory [8,9,10,11,12,13].

Our interest in the present work is extending the Benamou and Brenier computational fluid dynamical (CFD) approach to OMT, where the domains have transitioning compact support and may also be subject to potential spatiotemporal constraints (obstacles to avoid or mandatory regions to visit). We recall that in the seminal work [14], Benamou and Brenier compute the Wasserstein 2-metric $W_2$ via the minimization of kinetic energy subject to a continuity constraint. They moreover compute the optimal path (i.e., geodesic) in the space of probability densities [15]. In image processing, this gives a natural interpolation path between two images, where the intensity is treated as a “generalized mass”. This is important for problems in deformable registration and image warping; see [16,17] and the references therein.

Recently, this same fluid dyamical framework for OMT was related to the Schrödinger bridge problem in the continuum limit of Brownian particles exhibiting different empirical distributions at two different times. In [18], Chen et al. precisely elaborate this relationship, which they leverage in order to develop a stochastic control formulation of the fluid dynamic OMT problem which allows for an “a priori” evolution as in the Schrödinger bridge.

Novelties and Contributions Presented in This Work

The fluid dynamical formulation of Benamou and Brenier, however, does not deliberately take into account that the regions of interest in image-based applications are necessarily compact, and may even have non-convex regions of support when considering segmented subregions of interest in medical imaging applications (for example, the deformation of the cardiac ventricle as illustrated in an experiment in Section 5.1). In such cases, boundary conditions must crucially be imposed, not only to maintain mass conservation but also to prevent mass exchanges (even if balanced) between identified subregions and their backgrounds. However, such boundary conditions impose constraints on the optimization problem, which may not be compatible with the classical unconstrained solution.

It is this precisely this issue that motivated the present work. To treat the problem of OMT on densities with compact support, we propose a synergy of methods from OMT as well as level set evolutions [19,20]. Level set methods are a powerful way of implementing interface (boundary) evolution problems and thus have become a standard for a number of approaches in computer vision for segmentation, so-called active contour and active surface methods; see [21] and the references therein. We may not only impose mass-preserving boundary conditions along such modeled interfaces but may also consider how the variation of such interfaces changes the constrained minimizing solution, thereby yielding a joint optimization of the mass transport along the interior of the support together with a geometric optimization of the support boundary itself. This synergy of level set shape optimization and OMT, we believe to be novel, and to potentially have a number of applications, particularly to digital pathology.

Furthermore, with direct modeling and control of the support boundaries not only for the initial and final mass configurations but also along its intervening transport, we may readily handle intermediate spatiotemporal constraints with no additional modeling or computational machinery. While not the original motivation that led to the present work, it nonetheless yields a powerful and highly useful extension of the optimal mass transport problem that is naturally handled with the joint framework of fluid dynamics and geometric interface evolution. We may, for example, employ it to solve a variation of the original problem posed by Monge, where a pile of dirt must be moved from one location to another but with a river passing in between with a set of bridges which constrain the transport paths in between. We may even, as illustrated in an experiment in Section 5.2.2, readily apply it to solve yet another extension of the OMT problem where the initial density is known but where only the final support is known, not the final density itself.

Finally, while the focus of this work is to develop and computationally demonstrate a novel joint variational formulation for OMT combining the fluid dynamical framework of Benamou and Brennier with active geometric methods for the evolution of interfaces, we present two mathematical conjectures in Section 3.4.6, supported by our developed variational formulas, regarding the existence of solutions to the OMT problem in the presence of spatiotemporal constraints.

2. Spatiotemporal Hypersurface

We use bold notation exclusively to denote space-time points, vectors, and sets and non-bold notation for similar entities in time or space only, as well as for scalar variables. Accordingly, $\mathbf{X}$ will represent an arbitrary spatiotemporal point in ${\mathbb{R}}^{n+1}$

$$\mathbf{X}=(t,x)=(\underset{t}{\underbrace{X_0}},\underset{x}{\underbrace{X_1,X_2,\dots ,X_n}})$$

which pairs a temporal coordinate $t\in \mathbb{R}$ with a spatial coordinate $x\in {\mathbb{R}}^n$. We will denote the spatiotemporal basis vectors by ${\mathbf{e}}_0,{\mathbf{e}}_1,{\mathbf{e}}_2,\dots ,{\mathbf{e}}_n$ so that

$$\mathbf{X}=\underset{t}{\underbrace{X_0{\mathbf{e}}_0}}+\underset{x}{\underbrace{X_1{\mathbf{e}}_1+X_2{\mathbf{e}}_2+\cdots +X_n{\mathbf{e}}_n}}$$

2.1. Assumptions

2.1.1. Compact Support

We consider a spatiotemporal density $\rho$ in the form of a positive scalar function $\mathbf{X}\to \rho \left(\mathbf{X}\right)$ in ${\mathbb{R}}^{n+1}$ whose restriction to the $t=0$ hyperplane matches a given initial spatial density ${\rho }_0$ and whose restriction to the $t=1$ hyperplane matches a given spatial target density ${\rho }_1$. We assume that the initial spatial density ${\rho }_0$ has compact support ${\mathsf{\Omega }}_0\subset {\mathbb{R}}^n$ and that the target spatial density ${\rho }_1$ has compact support ${\mathsf{\Omega }}_1\subset {\mathbb{R}}^n$. We further assume that the full spatiotemporal density $\rho$ also has compact support $\mathsf{\Omega }\subset {\mathbb{R}}^{n+1}$ in space-time, sandwiched between the hyperplanes $t=0$ and $t=1$, which may be constructed by a continuous family of intermediate compact spatial supports ${\mathsf{\Omega }}_{\left[t\right]}\subset {\mathbb{R}}^n$ with ${\mathsf{\Omega }}_{\left[0\right]}={\mathsf{\Omega }}_0$ and ${\mathsf{\Omega }}_{\left[1\right]}={\mathsf{\Omega }}_1$ as follows:

$$\mathsf{\Omega }=\left\{\mathbf{X}=(t,x)|0\le t\le 1,x\in {\mathsf{\Omega }}_{\left[t\right]}\right\}$$

(1)

2.1.2. Balanced Density

We assume that the initial and target spatial densities ${\rho }_0$ and ${\rho }_1$ both have unit mass. We impose a similar constraint on the spatiotemporal density $\rho$, summarizing these assumptions as follows.

$${\int }_{{\mathsf{\Omega }}_0}{\rho }_0\left(x\right)dx={\int }_{{\mathsf{\Omega }}_1}{\rho }_1\left(x\right)dx={\int }_{\mathsf{\Omega }}\rho \left(\mathbf{X}\right)d\mathbf{X}=1$$

(2)

(The reason we do not start with the stronger constraint ${\int }_{{\mathsf{\Omega }}_{\left[t\right]}}\rho (x,t)dx=1$ for all $0\le t\le 1$ is that both this as well as the weaker total spatiotemporal mass constraint are global mass preservation constraints that will automatically be satisfied later when we impose the much stronger local mass preservation constraint. The main point in even presenting the total spatiotemporal constraint here is to reinforce the embedded space-time interpretation of the problem, thereby allowing us to interpret $\rho$ as a unit spatiotemporal mass density directly in ${\mathbb{R}}^{n+1}$.)

2.1.3. Smoothness

We assume that the initial and target spatial densities ${\rho }_0$ and ${\rho }_1$ are differentiable within their support (we do not require ${\rho }_0$ and ${\rho }_1$ to be zero along the boundaries of their supports; as such, they may discontinuously drop to zero across the spatial boundaries ${\mathsf{\Gamma }}_0$ and ${\mathsf{\Gamma }}_1$ respectively). For ${\mathsf{\Omega }}_0$ and ${\mathsf{\Omega }}_1$, the spatiotemporal density $\rho$ is differentiable within its support (we do not require $\rho$ to be zero along the boundary of its support, a necessary freedom along the flat temporal faces ${\mathsf{\Gamma }}_0$ and ${\mathsf{\Gamma }}_1$; otherwise, we could not impose $\rho ={\rho }_o$ and $\rho ={\rho }_1$ along these components). For $\mathsf{\Omega }$, we also assume that the portion of the spatiotemporal boundary $\partial \mathsf{\Omega }$ that lies strictly within $0<t<1$, which we denote by $\mathsf{\Gamma }$, is differentiable. The remaining portions of $\partial \mathsf{\Omega }$ are provided by the embeddings of ${\mathsf{\Omega }}_0$ and ${\mathsf{\Omega }}_1$ within the hyperplanes $t=0$ and $t=1$ to form two flat temporal faces of $\mathsf{\Omega }$, which we denote by ${\mathsf{\Gamma }}_0$ and ${\mathsf{\Gamma }}_1$.

2.1.4. Piece-Wise Smooth Boundary

As such $\partial \mathsf{\Omega }=\mathsf{\Gamma }\cup {\mathsf{\Gamma }}_0\cup {\mathsf{\Gamma }}_1$ will have the form of a compact hypersurface in ${\mathbb{R}}^{n+1}$ with a well-defined outward normal everywhere except along the borders of the two temporal faces where ${\mathsf{\Gamma }}_0$ and ${\mathsf{\Gamma }}_1$ connect to the intervening surface $\mathsf{\Gamma }$. We may also describe the intervening spatiotemporal boundary component $\mathsf{\Gamma }$ as the swept-out surface generated by embedding the boundaries ${\mathsf{\Gamma }}_{\left[t\right]}=\partial {\mathsf{\Omega }}_{\left[t\right]}$ of the deforming spatial supports ${\mathsf{\Omega }}_{\left[t\right]}$ into the corresponding temporal hyperplanes. The spatiotemporal boundary notation and decomposition is summarized as follows:

$$\partial \mathsf{\Omega }=\underset{\underset{\left(\text{temporal face}\right)}{{\mathsf{\Gamma }}_0}}{\underbrace{\left\{\mathbf{X}=(0,x)|x\in {\mathsf{\Omega }}_0\right\}}}\cup \underset{\underset{\left(\text{temporal face}\right)}{{\mathsf{\Gamma }}_1}}{\underbrace{\left\{\mathbf{X}=(1,x)|x\in {\mathsf{\Omega }}_1\right\}}}\cup \underset{\underset{\left(\text{swept-out surface}\right)}{\mathsf{\Gamma }}}{\underbrace{\left\{\mathbf{X}=(t,x)|0\le t\le 1,x\in \stackrel{{\mathsf{\Gamma }}_{\left[t\right]}}{\overbrace{\partial {\mathsf{\Omega }}_{\left[t\right]}}}\right\}}}$$

(3)

2.1.5. Smooth Extension

Finally, in order to treat the density $\rho$ and its support $\mathsf{\Omega }$ as independent entities to be optimized later on, we will also consider $\rho$ within the support $\mathsf{\Omega }$ to be the restriction of a positive density that smoothly extends $\rho$ beyond its modeled support. As such, perturbations to the support boundary $\mathsf{\Gamma }$ may be considered independently of perturbations to the density values $\rho$. Similar treatment will be applied to upcoming vector fields to be defined over the same compact support $\mathsf{\Omega }$.

2.2. Local Geometry

In this section, we explore the relationship between the local geometry of the spatial support boundary ${\mathsf{\Gamma }}_{\left[t\right]}$ and the swept-out spatiotemporal boundary $\mathsf{\Gamma }$.

2.2.1. Parameterization

Let $s=(s_1,\dots ,s_{n-1})$ represent isothermal coordinates with unit speed at a point $x\in$${\mathsf{\Gamma }}_{\left[t\right]}$ along the boundary of the spatial support ${\mathsf{\Omega }}_{\left[t\right]}\subset {\mathbb{R}}^n$ at time t strictly between 0 and 1. The Riemannian metric tensor of ${\mathsf{\Gamma }}_{\left[t\right]}$ in these coordinates is therefore the $(n-1)\times (n-1)$ identity matrix at the point x with $n-1$ orthonormal tangent vectors $\frac{\partial {\mathsf{\Gamma }}_{\left[t\right]}}{\partial s_k}\in {\mathbb{R}}^n$ for $k=1,\dots ,n-1$. From (3), we know that the corresponding spatiotemporal point $\mathbf{X}=(t,x)$ belongs to the $\mathsf{\Gamma }$ portion of the spatiotemporal support boundary $\partial \mathsf{\Omega }$, which may be locally parameterized as follows:

$$\mathsf{\Gamma }(t,s)=\left(t,{\mathsf{\Gamma }}_{\left[t\right]}\left(s\right)\right)=(\underset{t}{\underbrace{{\mathsf{\Gamma }}_0}},\underset{{\mathsf{\Gamma }}_{\left[t\right]}\left(s\right)}{\underbrace{{\mathsf{\Gamma }}_1,{\mathsf{\Gamma }}_2,\dots ,{\mathsf{\Gamma }}_n}})$$

(4)

2.2.2. Unit Normal

In these coordinates, we compute the following n tangent vectors to $\mathsf{\Gamma }$ in ${\mathbb{R}}^{n+1}$:

$$\frac{\partial \mathsf{\Gamma }}{\partial t}=\left(1,\frac{\partial {\mathsf{\Gamma }}_{\left[t\right]}}{\partial t}\right)\mathrm{and}\frac{\partial \mathsf{\Gamma }}{\partial s_k}=\left(0,\frac{\partial {\mathsf{\Gamma }}_{\left[t\right]}}{\partial s_k}\right),k=1,\dots ,n-1$$

(5)

Since the unit outward normal $N\in {\mathbb{R}}^n$ to the spatial boundary ${\mathsf{\Gamma }}_{\left[t\right]}$ is orthogonal to the $n-1$ spatial tangent vectors $\frac{\partial {\mathsf{\Gamma }}_{\left[t\right]}}{\partial s_k}\in {\mathbb{R}}^n$, it follows that $(\alpha ,N)\in {\mathbb{R}}^{n+1}$ will be orthogonal to the $n-1$ spatiotemporal tangent vectors $\frac{\partial \mathsf{\Gamma }}{\partial s_k}$ expressed in (5) for any choice of scalar $\alpha$. Orthogonality to the additional spatiotemporal tangent vector $\frac{\partial \mathsf{\Gamma }}{\partial t}$ expressed in (5) also requires $\alpha =-\frac{\partial {\mathsf{\Gamma }}_{\left[t\right]}}{\partial t}·N$, yielding the following construction of the spatiotemporal unit outward normal $\mathbf{N}\in {\mathbb{R}}^{n+1}$:

$$\mathbf{N}=\left\{\begin{array}{cc}\frac{\left(-\frac{\partial {\mathsf{\Gamma }}_{\left[t\right]}}{\partial t}·N,N\right)}{\sqrt{1+{\left(\frac{\partial {\mathsf{\Gamma }}_{\left[t\right]}}{\partial t}·N\right)}^2}},\hfill & \mathbf{X}\in \mathsf{\Gamma }\hfill \\ +{\mathbf{e}}_0=+(1,0,\dots ,0),\hfill & \mathbf{X}\in {\mathsf{\Gamma }}_1\hfill \\ -{\mathbf{e}}_0=-(1,0,\dots ,0),\hfill & \mathbf{X}\in {\mathsf{\Gamma }}_0\hfill \end{array}\right.$$

(6)

2.2.3. Metric Tensor

Using the tangent vectors (5), we may express the Riemannian metric tensor at the point $\mathbf{X}$ in the form of the following $n\times n$ matrix:

$$\underset{\text{I-fundamental form}}{\underbrace{\mathbf{I}(s_1,\dots ,s_{n-1},t)}}=\left[\begin{array}{cc}\mathcal{I}& {\left(\frac{\partial {\mathsf{\Gamma }}_{\left[t\right]}}{\partial s}\right)}^T\frac{\partial {\mathsf{\Gamma }}_{\left[t\right]}}{\partial t}\\ {\frac{\partial {\mathsf{\Gamma }}_{\left[t\right]}}{\partial t}}^T\left(\frac{\partial {\mathsf{\Gamma }}_{\left[t\right]}}{\partial s}\right)& 1+\frac{\partial {\mathsf{\Gamma }}_{\left[t\right]}}{\partial t}·\frac{\partial {\mathsf{\Gamma }}_{\left[t\right]}}{\partial t}\end{array}\right]$$

(7)

where $\frac{\partial {\mathsf{\Gamma }}_{\left[t\right]}}{\partial s}$ denotes the $n\times (n-1)$ matrix, whose columns consist of the orthonormal tangent vectors to the spatial boundary ${\mathsf{\Gamma }}_{\left[t\right]}$ at the point x. Using the determinant formula, $det\left[\begin{array}{cc}A& u\\ v^T& \alpha \end{array}\right]=\alpha detA-v^T\mathrm{adj}Au$ (for any matrix A, vector u and v, and scalar $\alpha$), we may compute:

$$det\mathbf{I}=\left(1+\frac{\partial {\mathsf{\Gamma }}_{\left[t\right]}}{\partial t}·\frac{\partial {\mathsf{\Gamma }}_{\left[t\right]}}{\partial t}\right)-\underset{{∥\frac{\partial {\mathsf{\Gamma }}_{\left[t\right]}}{\partial t}∥}^2-{\left(\frac{\partial {\mathsf{\Gamma }}_{\left[t\right]}}{\partial t}·N\right)}^2}{\underbrace{{\displaystyle \sum _{k=1}^{n-1}}{\left(\frac{\partial {\mathsf{\Gamma }}_{\left[t\right]}}{\partial s_k}·\frac{\partial {\mathsf{\Gamma }}_{\left[t\right]}}{\partial t}\right)}^2}}=1+{\left(\frac{\partial {\mathsf{\Gamma }}_{\left[t\right]}}{\partial t}·N\right)}^2$$

2.2.4. Area Element

The relationship between the area element $d{S}_{\left[t\right]}$ of the spatial boundary surface ${\mathsf{\Gamma }}_{\left[t\right]}$, the temporal element $dt$, and the area element $d\mathbf{S}$ of the swept-out spatiotemporal hypersurface $\mathsf{\Gamma }$ can be expressed via the square root of the determinant of the first fundamental form shown above:

$$d\mathbf{S}=\sqrt{1+{\left(\frac{\partial {\mathsf{\Gamma }}_{\left[t\right]}}{\partial t}·N\right)}^2}d{S}_{\left[t\right]}dt$$

(8)

2.2.5. Normal Variations

Finally, using the parameterization (4) for a variation $\delta \mathsf{\Gamma }$ allows us to relate a variation of the swept-out spatiotemporal hypersurface to time parameterized variations $\delta {\mathsf{\Gamma }}_{\left[t\right]}$ of the spatial support boundaries as follows: $\delta \mathsf{\Gamma }=\left(0,\delta {\mathsf{\Gamma }}_{\left[t\right]}\right)$. Combining this with (6) yields the following relationship between variations of ${\mathsf{\Gamma }}_{\left[t\right]}$ and variations of $\mathsf{\Gamma }$ along their respective normal directions:

$$\delta \mathsf{\Gamma }·\mathbf{N}=\frac{\delta {\mathsf{\Gamma }}_{\left[t\right]}·N}{\sqrt{1+{\left(\frac{\partial {\mathsf{\Gamma }}_{\left[t\right]}}{\partial t}·N\right)}^2}}$$

If we now further combine this with (8), we see that the normal variation of the spatiotemporal hypersurface measured against its spatiotemporal area element matches the normal variation of the corresponding spatial boundary measured by its respective area element and time element:

$$\left(\delta \mathsf{\Gamma }·\mathbf{N}\right)d\mathbf{S}=\left(\delta {\mathsf{\Gamma }}_{\left[t\right]}·N\right)d{S}_{\left[t\right]}dt$$

(9)

3. Spatiotemporal Formulation of Optimal Mass Transport

The fluid dynamical framework of Benamou and Brenier considers two time-evolving entities, a scalar mass density $\rho (t,x)$ and a velocity field $v(t,x)\in {\mathbb{R}}^n$, which are coupled by the local mass-preserving continuity constraint $\frac{\partial \rho }{\partial t}+{\nabla }_x·\left(\rho v\right)=0$.

Notice that in the standard manner regarding $\rho v$ as an ordinary 3-vector, the continuity constraint means that density and spatial momentum form a 4-vector with respect to the standard Minkowski metric in the standard physics setting [22]. We will exploit this observation and develop an equivalent formulation using a single space-time solenoidal vector field $\mathbf{W}$ with simple normal boundary conditions along the spatiotemporal hypersurface $\partial \mathsf{\Omega }$ that enable convenient numerical solutions of PDEs directly within an $n+1$ dimensional space-time grid, with no need to treat the temporal and spatial dimensions separately or differently.

3.1. Spatiotemporal Advection Field $\mathbf{U}$

We begin by noting that in the combined spatiotemporal variable $\mathbf{X}=(t,x)$, the continuity constraint $\frac{\partial \rho }{\partial t}+{\nabla }_x·\left(\rho v\right)=0$ can be written as

$$\nabla \rho ·\mathbf{U}+\rho \nabla ·\mathbf{U}=0$$

(10)

where ∇ and $\nabla ·$ represent the full spatiotemporal gradient and divergence operators in ${\mathbb{R}}^{n+1}$ and where $\mathbf{U}\in {\mathbb{R}}^{n+1}$ denotes the following vector field:

$$\mathbf{U}\left(\mathbf{X}\right)\dot{=}\left(1,v\right)=(\underset{1}{\underbrace{U_0}},\underset{v}{\underbrace{U_1,U_2,\dots ,U_n}})$$

Note that $\mathbf{U}$ is tangent to the characteristics of this linear first-order PDE (10) in $\rho$ and therefore defines the trajectories along which mass is transported across space-time. Since the spatiotemporal hypersurface $\partial \mathsf{\Omega }$, more specifically, its swept-out portion $\mathsf{\Gamma }$, defines the boundary of the evolving support for $\rho$, we can conclude that these advection trajectories must flow along the hypersurface $\mathsf{\Gamma }$ itself, never across it (neither inward nor outward). In other words, mass cannot be transported outside of its support neither forward in time (which excludes outward-flowing characteristics) nor backward in time (which excludes inward-flowing characteristics). This leads to the boundary condition $\mathbf{U}·\mathbf{N}=0$ along $\mathsf{\Gamma }$, which may be written in terms of v, N, and $\frac{\partial {\mathsf{\Gamma }}_{\left[t\right]}}{\partial t}$ as follows:

$$\underset{(\mathbf{U}·\mathbf{N}=0)}{\underbrace{v·N=\frac{\partial {\mathsf{\Gamma }}_{\left[t\right]}}{\partial t}·N}}$$

(11)

If we plug this constraint between the support evolution $\frac{\partial {\mathsf{\Gamma }}_{\left[t\right]}}{\partial t}$ and the velocity field v into (6), we obtain the following alternative expression for the outward unit normal $\mathbf{N}\in {\mathbb{R}}^{n+1}$ of the swept-out hypersurface $\mathsf{\Gamma }$ in terms of the outward normal $N\in {\mathbb{R}}^n$ of the support boundary ${\mathsf{\Gamma }}_{\left[t\right]}$ in space at time t:

$$\begin{array}{cc}\hfill \mathbf{N}\left(\mathbf{X}\right)& =\frac{\left(-v·N,N\right)}{\sqrt{1+{\left(v·N\right)}^2}},\mathbf{X}\in \mathsf{\Gamma }\hfill \end{array}$$

(12)

3.2. Solenoidal Vector Field $\mathbf{W}$

While the use of the advection field $\mathbf{U}$ merges the spatial and temporal derivatives into a single derivative ${\mathbb{R}}^{n+1}$ operator in (11), it still keeps the density variable $\rho$ separate. We now merge these two entities by defining another spatiotemporal vector field whose first (temporal) component represents the mass density $\rho \left(\mathbf{X}\right)$, and whose remaining components represent the momentum vector $\mathrm{p}\left(\mathbf{X}\right)=\rho v$ in ${\mathbb{R}}^n$:

$$\mathbf{W}\left(\mathbf{X}\right)=\rho \mathbf{U}=\left(\rho ,\mathrm{p}\right)=(\underset{\rho }{\underbrace{W_0}},\underset{\mathrm{momentum} \mathrm{p}=\rho v}{\underbrace{W_1,W_2,\dots ,W_n}})$$

(13)

However, rather than considering (13) to be the definition of $\mathbf{W}$ in terms of the density $\rho$ and momentum $\mathrm{p}$, we instead consider it in reverse to be the definition of $\rho$ and $\mathrm{p}$ in terms of the space-time vector field $\mathbf{W}$ subject to the continuity constraint (10), which now simplifies to a coordinate-free solenoidal condition on $\mathbf{W}$:

$$\nabla ·\mathbf{W}=0$$

(14)

Multiplying the boundary condition $\mathbf{U}·\mathbf{N}=0$ along $\mathsf{\Gamma }$ presented in (11) by $\rho$ yields a similar vanishing flux condition for $\mathbf{W}$ across the swept-out portion of the spatiotemporal hypersurface $\partial \mathsf{\Omega }$. For the remainder of $\partial \mathsf{\Omega }$, we combine $\mathbf{N}=\pm {\mathbf{e}}_0$ from (6) with (13) to obtain flux conditions for $\mathbf{W}$ along the temporal faces ${\mathsf{\Gamma }}_0$ and ${\mathsf{\Gamma }}_1$ as well, in terms of the known starting and target densities ${\rho }_0$ and ${\rho }_1$.

The combination of these constraints is easily summarized now in terms of $\mathbf{W}$ and its spatiotemporal domain $\mathsf{\Omega }$. Namely, we seek a solenoidal vector field $\mathbf{W}$ within $\mathsf{\Omega }$ with the following prescribed flux conditions along the full spatiotemporal domain boundary $\partial \mathsf{\Omega }$:

$$\mathbf{W}·\mathbf{N}=\left\{\begin{array}{cc}0,\hfill & \mathbf{X}\in \mathsf{\Gamma }\hfill \\ -{\rho }_0,\hfill & \mathbf{X}\in {\mathsf{\Gamma }}_0\hfill \\ +{\rho }_1,\hfill & \mathbf{X}\in {\mathsf{\Gamma }}_1\hfill \end{array}\right.$$

(15)

3.3. Extended Velocity $\mathbf{V}$

3.3.1. Local Kinetic Energy

Before setting up the variational problem, we seek an expression for the local measure of kinetic energy $\mathrm{T}(t,x)$

$$\mathrm{T}\left(\mathbf{X}\right)=\frac{1}{2}\rho v·v$$

in terms of the solenoidal field $\mathbf{W}$. If we express this purely in terms of $\mathbf{W}$, we obtain the following expression which, unfortunately, is not coordinate-free:

$$\mathrm{T}\left(\mathbf{X}\right)=\frac{1}{2}\left(\frac{\stackrel{{\rho }^2(1+v·v)}{\overbrace{\mathbf{W}·\mathbf{W}}}}{\underset{\rho }{\underbrace{\mathbf{W}·{\mathbf{e}}_0}}}-\underset{\rho }{\underbrace{\mathbf{W}·{\mathbf{e}}_0}}\right)$$

3.3.2. Extended Velocity

We may resolve this by introducing the following extended velocity field $\mathbf{V}\in {\mathbb{R}}^{n+1}$ which extends the transport velocity v from ${\mathbb{R}}^n$ into ${\mathbb{R}}^{n+1}$ by adding a temporal component equal to $-\frac{1}{2}{\parallel v\parallel }^2$ as follows:

$$\mathbf{V}\doteq \left(-\frac{1}{2}{\parallel v\parallel }^2,v\right)=(\underset{\frac{-{\parallel v\parallel }^2}{2}}{\underbrace{V_0}},\underset{v}{\underbrace{V_1,V_2,\dots ,V_n}})$$

(16)

Notice that, just like the advection field $\mathbf{U}$, the extended velocity $\mathbf{V}$ depends only upon the spatial velocity v itself, and therefore contains no additional information. We may now express the local kinetic energy compactly and as being coordinate-free in terms of the solenoidal field $\mathbf{W}$ and the extended velocity field $\mathbf{V}$ by their inner product:

$$\mathrm{T}=\mathbf{W}·\mathbf{V}$$

3.3.3. Generalized Momentum

Notice that if we multiply the extended velocity $\mathbf{V}$ by $\rho$ we obtain the generalized momentum $\mathbf{P}$, which we construct in terms of the kinetic energy $\mathrm{T}$ and the momentum $\mathrm{p}$ as follows:

$$\mathbf{P}=\left(-\frac{1}{2}\rho {\parallel v\parallel }^2,\rho v\right)=\left(-\mathrm{T},\mathrm{p}\right)=(\underset{-\mathrm{T}}{\underbrace{P_0}},\underset{\mathrm{p}}{\underbrace{P_1,P_2,\dots ,P_n}})$$

Both the solenoidal field $\mathbf{W}$ and the generalized momentum field $\mathbf{P}$ go to zero as the density $\rho$ goes to zero, even in cases where the velocity v becomes undefined. While we will not be utilizing $\mathbf{P}$ in our upcoming variational formulation, it is nevertheless useful to interpret the extended velocity $\mathbf{V}$ as $\mathbf{P}/\rho$ for $\rho >0$ just as we may interpret the advection field $\mathbf{U}$ as $\mathbf{W}/\rho$ for $\rho >0$ without explicit reference to the velocity v. We now summarize and consolidate our notation for the four spatial–temporal vector fields in ${\mathbb{R}}^{n+1}$ as follows:

$$\begin{array}{cc}\text{solenoidal field}\hfill & \text{generalized momentum}\hfill \\ \mathbf{W}=(\rho ,\mathrm{p})\hfill & \mathbf{P}=(-\mathrm{T},\mathrm{p})\hfill \\ \\ \text{advection field}\hfill & \text{extended velocity}\hfill \\ \mathbf{U}=\frac{\mathbf{W}}{\rho }=(1,v)\hfill & \mathbf{V}=\frac{\mathbf{P}}{\rho }=\left(-\frac{1}{2}{\parallel v\parallel }^2,v\right)\hfill \end{array}$$

3.4. Variational Formulation

3.4.1. Action Integral

We begin by constructing the action integral to be minimized over $\mathsf{\Omega }\subset {\mathbb{R}}^{n+1}$ in terms of the solenoidal field $\mathbf{W}$ as follows:

$$E(\mathbf{W},\mathsf{\Gamma })={\int }_0^1{\int }_{{\mathsf{\Omega }}_{\left[t\right]}}\stackrel{T}{\overbrace{\frac{1}{2}\rho {\parallel v\parallel }^2}}dxdt={\int }_{\mathsf{\Omega }}\mathbf{W}·\mathbf{V}d\mathbf{X}$$

(17)

Note that the two unknowns are only $\mathbf{W}$ and its support $\mathsf{\Omega }$ (or equivalently, the swept-out boundary $\mathsf{\Gamma }$), even though we have expressed the action compactly also in terms of $\mathbf{V}$. We may compute $\mathbf{V}$ directly from $\mathbf{W}$

$$\mathbf{V}\doteq \left(-\frac{1}{2}{\parallel v\parallel }^2,v\right)=\mathbf{U}-\frac{1}{2}\left(\mathbf{U}·\mathbf{U}+1\right){\mathbf{e}}_0=\frac{\mathbf{W}}{\mathbf{W}·{\mathbf{e}}_0}-\frac{1}{2}\left(\frac{\mathbf{W}·\mathbf{W}}{{\left(\mathbf{W}·{\mathbf{e}}_0\right)}^2}+1\right){\mathbf{e}}_0$$

(18)

with the following compatible flux conditions obtained by plugging in (15):

$$\begin{array}{cc}\hfill \mathbf{V}·\mathbf{N}& =\left\{\begin{array}{cc}-\left(1+\frac{1}{2}{\parallel v\parallel }^2\right)\left(\mathbf{N}·{\mathbf{e}}_0\right),\hfill & \mathbf{X}\in \mathsf{\Gamma }\hfill \\ +\frac{1}{2}{\parallel v\parallel }^2,\hfill & \mathbf{X}\in {\mathsf{\Gamma }}_0\hfill \\ -\frac{1}{2}{\parallel v\parallel }^2,\hfill & \mathbf{X}\in {\mathsf{\Gamma }}_1\hfill \end{array}\right.\hfill \end{array}$$

We incorporate the solenoidal (mass preservation) constraint (14) through a Lagrange multiplier $\lambda$ over $\mathsf{\Omega }$ and the flux constraints through additional Lagrange multipliers ${\alpha }_0$, ${\alpha }_1$, and $\alpha$ along the boundaries ${\mathsf{\Gamma }}_0$, ${\mathsf{\Gamma }}_1$, and $\mathsf{\Gamma }$, respectively:

$$E={\int }_{\mathsf{\Omega }}\mathbf{W}·\mathbf{V}+\underset{\stackrel{\text{solenoidal}}{\text{constraint}}}{\underbrace{\lambda \nabla ·\mathbf{W}}}d\mathbf{X}+\underset{\text{flux constraints}}{\underbrace{{\int }_{\mathsf{\Gamma }}\alpha \mathbf{W}·\mathbf{N}d\mathbf{S}+{\int }_{{\mathsf{\Gamma }}_0}{\alpha }_0\left(\stackrel{-\mathbf{W}·{\mathbf{e}}_{\mathbf{0}}}{\overbrace{\mathbf{W}·\mathbf{N}}}+{\rho }_0\right)dx+{\int }_{{\mathsf{\Gamma }}_1}{\alpha }_1\left(\stackrel{\mathbf{W}·{\mathbf{e}}_0}{\overbrace{\mathbf{W}·\mathbf{N}}}-{\rho }_1\right)dx}}$$

(19)

3.4.2. First Variation

In Appendix A, we compute the variation of (19) which reduces to

$$\begin{array}{cc}\hfill \delta E=& {\int }_{\mathsf{\Omega }}\delta \mathbf{W}·\left(\mathbf{V}-\nabla \lambda \right)d\mathbf{X}+{\int }_{\mathsf{\Gamma }}\mathbf{W}·\left(\mathbf{V}-\nabla \lambda \right)\delta \mathsf{\Gamma }·\mathbf{N}d\mathbf{S}\hfill \end{array}$$

(20)

as long as the solenoidal and flux constraints for $\mathbf{W}$ are satisfied and as long as we further impose $\delta \mathbf{W}·{\mathbf{e}}_0=0$ at $t=0$ and $t=1$ to preserve the initial and final densities ${\rho }_0$ and ${\rho }_1$ as well as $\delta \mathsf{\Gamma }=0$ at $t=0$ and $t=1$ to preserve the initial and final spatial supports ${\mathsf{\Omega }}_0$ and ${\mathsf{\Omega }}_1$.

3.4.3. Optimality Condition for $\mathbf{W}$

By inspection of (20), optimality for $\mathbf{W}$ can only be achieved if we can solve

$$\nabla \lambda =\mathbf{V}$$

(21)

for $\lambda$ within the interior of $\mathsf{\Omega }$. If we separately equate the spatial and temporal components

$$\stackrel{\nabla \lambda }{\overbrace{(\underset{\frac{\partial \lambda }{\partial t}}{\underbrace{\frac{\partial \lambda }{\partial X_0}}},\underset{\frac{\partial \lambda }{\partial x}}{\underbrace{\frac{\partial \lambda }{\partial X_1},\frac{\partial \lambda }{\partial X_2},\dots ,\frac{\partial \lambda }{\partial X_n}}})}}=\stackrel{\mathbf{V}}{\overbrace{(\underset{\frac{-{\parallel v\parallel }^2}{2}}{\underbrace{V_0}},\underset{v}{\underbrace{V_1,V_2,\dots ,V_n}})}}$$

then we see that (21) amounts to a more compact, coordinate-free expression of the well-known Hamilton Jacobi equation:

$$\frac{\partial \lambda }{\partial t}+\frac{1}{2}{∥\frac{\partial \lambda }{\partial x}∥}^2=0$$

(22)

However, (21) will not admit a solution unless $\mathbf{V}$ is a conservative (irrotational) vector field. As such, the gradient must be related to the non-conservative (solenoidal) portion of the extended velocity field $\mathbf{V}$. Accordingly, we consider the Helmholtz decomposition, expressing $\mathbf{V}$ as the sum of two vector fields:

$$\mathbf{V}=\underset{\stackrel{\text{irrotational}}{(\text{curl-free})}}{\underbrace{{\mathbf{V}}^{\Vert }}}+\underset{\stackrel{\text{solenoidal}}{\left(\text{divergence-free}\right)}}{\underbrace{{\mathbf{V}}^{\perp }}}$$

(23)

where ${\mathbf{V}}^{\perp }$ denotes the divergence-free component ($\nabla ·{\mathbf{V}}^{\perp }=0$) and where ${\mathbf{V}}^{\Vert }$ denotes the curl-free component ($\nabla \times {\mathbf{V}}^{\Vert }=0$) which can be written as the gradient of a scalar potential function.

We can now always choose the Lagrange multiplier $\lambda$ to satisfy

$${\mathbf{V}}^{\parallel }=\nabla \lambda \mathrm{and}{\mathbf{V}}^{\perp }=\mathbf{V}-\nabla \lambda$$

(24)

which annihilates the curl-free component ${\mathbf{V}}^{\Vert }$ in (20) and yields a cleaner expression of the optimality condition purely in terms of $\mathbf{V}$ as follows:

$${\mathbf{V}}^{\perp }=0$$

(25)

3.4.4. Partial Gradient with Respect to $\mathbf{W}$ (Fixed Support)

In fact, the solenoidal component ${\mathbf{V}}^{\perp }$ of the extended velocity represents the gradient of the action integral E with respect to the solenoidal field $\mathbf{W}$ alone, keeping the spatiotemporal support $\mathsf{\Omega }$ fixed:

$${\nabla }_{\mathbf{W}}E={\mathbf{V}}^{\perp }$$

(26)

This can be seen immediately by plugging $\nabla \lambda ={\mathbf{V}}^{\Vert }$ and $\delta \mathsf{\Gamma }=0$ into the first variation (20) to obtain

$$\begin{array}{cc}\hfill \delta E{|}_{(\delta \mathsf{\Gamma }=0)}=& {\int }_{\mathsf{\Omega }}\delta \mathbf{W}·{\mathbf{V}}^{\perp }d\mathbf{X}\hfill \end{array}$$

(27)

Note, however, that the Helmholtz decomposition is not unique over compact domains but depends upon the boundary conditions chosen for ${\mathbf{V}}^{\Vert }$ or ${\mathbf{V}}^{\perp }$. Since we are treating the support of $\mathbf{W}$ as fixed, then the gradient field ${\nabla }_{\mathbf{W}}E$ must not contribute flux anywhere along the spatiotemporal boundary $\partial \mathsf{\Omega }$ (in order to respect local mass conservation). This means, by (26), that the flux of ${\mathbf{V}}^{\perp }$ must vanish, which makes the decomposition (23) unique. Plugging (24) into the solenoidal and vanishing flux conditions for ${\mathbf{V}}^{\perp }$ yields a Poisson equation with Neumann boundary conditions for $\lambda$:

$$\begin{array}{cc}\hfill \mathsf{\Delta }\lambda & =\nabla ·\mathbf{V},\mathbf{X}\in \mathsf{\Omega }\hfill \\ \hfill \nabla \lambda ·\mathbf{N}& =\mathbf{V}·\mathbf{N},\mathbf{X}\in \partial \mathsf{\Omega }\hfill \end{array}$$

(28)

Since $\mathbf{N}=\pm {\mathbf{e}}_0$ along the temporal faces of $\mathsf{\Omega }$, we may separate the Neumann boundary expressions along the three different portions of $\partial \mathsf{\Omega }$ as follows:

$$\begin{array}{cc}\hfill \nabla \lambda ·\mathbf{N}& =\left\{\begin{array}{cc}+\frac{1}{2}{\parallel v\parallel }^2,\hfill & \mathbf{X}\in {\mathsf{\Gamma }}_0\hfill \\ -\frac{1}{2}{\parallel v\parallel }^2,\hfill & \mathbf{X}\in {\mathsf{\Gamma }}_1\hfill \\ \mathbf{V}·\mathbf{N},\hfill & \mathbf{X}\in \mathsf{\Gamma }\hfill \end{array}\right.\hfill \end{array}$$

(29)

3.4.5. Shape Optimality Condition for $\mathsf{\Gamma }$

By inspection of (20), shape optimality for $\mathsf{\Gamma }$ requires

$$\mathbf{W}·\left(\mathbf{V}-\nabla \lambda \right)=0,\mathbf{X}\in \mathsf{\Gamma }$$

(30)

along the support boundary $\mathsf{\Gamma }$. Given our choice (24) for the Lagrange multiplier $\lambda$, we may rewrite the condition purely in terms of the extended velocity $\mathbf{V}$ as

$$\mathbf{W}·{\mathbf{V}}^{\perp }=0,\mathbf{X}\in \mathsf{\Gamma }$$

(31)

Notice, however, that this shape optimality condition (31) is just a weaker version of the optimality condition ${\mathbf{V}}^{\perp }=0$ for the solenoidal field $\mathbf{W}$, despite our independent treatment of $\mathbf{W}$ and its support boundary $\mathsf{\Gamma }$ (recall our smooth-extension assumption in Section 2.1 in which vector fields defined over the compact support $\mathsf{\Omega }$ are treated as restrictions of vector fields that smoothly extend beyond the support).

This leaves us with the conundrum that if we are able to optimize $\mathbf{W}$ for any candidate spatiotemporal support $\mathsf{\Omega }$, then not only does the partial gradient vanish with respect to $\mathbf{W}$ (and its smooth extension) but then so does the partial gradient with respect to $\mathsf{\Gamma }$ as well! Not only does this counter-intuitively imply that all smoothly changing support evolutions between ${\mathsf{\Omega }}_0$ and ${\mathsf{\Omega }}_1$ represent critical points of the action integral (17) but it completely sabotages our ultimate shape optimization strategy, in which we hoped to propose an initial compact support evolution between ${\mathsf{\Omega }}_0$ and ${\mathsf{\Omega }}_1$, optimize $\mathbf{W}$ over the proposed spatiotemporal support $\mathsf{\Omega }$, and then evolve the support boundaries $\mathsf{\Gamma }$ according to the shape gradient.

3.4.6. Conjectures on the Existence of Optimal Transport with Intermediate Support Constraints

A potential explanation for this conundrum is that the constrained minimization of the action integral (17) with respect to $\mathbf{W}$ over an imposed spatiotemporal support $\mathsf{\Omega }$ may actually be an ill-posed problem unless the imposed support happens to match the optimal one already. If so, then a minimizer $\mathbf{W}$ would not exist in the absence of further constraints, despite the existence of a well-defined gradient (26). This would imply that the gradient flow does not converge, not even asymptotically.

Assuming that this conjecture is correct, it is natural to wonder what would happen in the gradient descent for $\mathbf{W}$ if we impose a non-optimal support $\mathsf{\Omega }$ over space-time. Mass would likely accumulate forever toward an infinite density along portions of the constrained support boundary that lie strictly within the interior of the optimal support and would likely decay forever toward zero density within portions of the constrained support that lie outside the optimal one. Imposing lower and upper bounds on the density could potentially render the problem well posed in such scenarios.

Since the conjectures apply to any non-optimal choice of imposed spatiotemporal support, we can extrapolate them to more general conjectures in terms of the existence of solutions to the optimal transport problem in the presence of intermediate support constraints as follows.

Conjecture 1.

A minimal action, mass-preserving flow which transports a compactly supported, uniformly continuous density ${\rho }_0$ to another compactly supported, uniformly continuous density ${\rho }_1$ does not exist (without breaking uniform continuity) when constraints are placed on the intermediate support, unless the constraints are also satisfied by the unconstrained solution.

In the case of positive support constraints (requiring mass within regions that would otherwise fall outside the support of the unconstrained solution), then any proposed solution can likely be replaced by a lower cost solution that transports less mass into superfluous regions and therefore brings the density in such regions closer to zero. In the case of negative support constraints (prohibiting mass within regions that would otherwise fall inside the support of the unconstrained solution, perhaps due to an obstacle in the way), then any proposed solution can likely be replaced by a lower cost solution that accumulates more mass along the boundaries of the excluded regions.

Conjecture 2.

Imposing lower and upper bounds on the density may lead to the existence of solutions to the same constrained optimal transport problem, even when the intermediate support constraints are violated by the unconstrained solution.

In the case of positive support constraints, a lower bound (if greater then 0) on the density would prevent replacing a proposed solution which already hits this lower density bound with a lower cost solution that brings the density closer to zero in regions where the lower bound is already achieved. In the case of negative support constraints, a finite upper bound would prevent replacing a proposed solution which already hits this upper density bound with a lower cost solution that further raises the density near or along support boundaries where the upper bound is already achieved.

4. Shape Optimization Strategy with Added Density Prior

Regardless of whether the first conjecture is correct, the fact that the shape gradient always vanishes if $\mathbf{W}$ is optimized (should an optimizer exist) for any arbitrary estimate of the support $\mathsf{\Omega }$, means that a gradient descent strategy to optimize the shape of the support boundary $\mathsf{\Gamma }$ cannot be employed using the variational formulation developed thus far. Motivated by the second conjecture, we will introduce a density prior (though not in the form of hard bounds as postulated in the conjecture but rather as soft bounds) that can be conveniently added to the variational formulation already developed. This will take the form of an additional potential to the Benamou–Brenier action introduced generally in Section 4.1 with an explicit, intuitive example later in Section 4.4.

(While for now we only present a potential term which depends upon the density values, we could introduce a spatiotemporal dependence as well an added potential term if we wished to soften the spatiotemporal support constraints and directly penalize mass in the proscribed regions as well as lack of mass in the prescribed regions. The density prior proposed here, instead, is intended to be paired with hard constraints on the spatiotemporal support by forcing the support boundary to keep mass out of the proscribed regions and within the prescribed regions but without infinitely piling up nor thinning out along such boundaries.)

The added potential term will allow us to approach the optimal mass transport problem between compact supports as a shape optimization problem, where an initial space-time support estimate $\mathsf{\Omega }$ is constructed via a smooth spatiotemporal hypersurface $\mathsf{\Gamma }$ which interpolates between the support boundaries $\partial {\mathsf{\Omega }}_0$ and $\partial {\mathsf{\Omega }}_1$ of the initial and final densities, where the imposed-support optimal transport problem is solved along this estimate by optimizing the solenoidal field $\mathbf{W}$ along this support estimate, and where the optimized $\mathbf{W}$ is used to determine the total shape gradient to optimally perturb the support estimate using a gradient descent step for the hypersurface $\mathsf{\Gamma }$. The process is then repeated until this shape evolution process converges. The ingredients for this process are outlined as follows.

4.1. Density Prior

Consider a density prior on $\mathbf{W}$ in the form of a convex penalty function $h\left(\rho \right)\ge 0$ applied to the density values $\rho =\mathbf{W}·{\mathbf{e}}_0$. We could, for example, set $h=0$ (i.e., no penalty) for values of $\rho$ within a desired range with a progressively increasing penalty ($h>0)$ for values of $\rho$ that move further away from the desired range. Adding such a soft prior to the action integral, together with the Lagrange multipliers in (19), yields a new energy functional to be minimized as follows:

$$\tilde{E}=E+\underset{\text{density prior}}{\underbrace{{\int }_{\mathsf{\Omega }}h(\stackrel{\rho }{\overbrace{\mathbf{W}·{\mathbf{e}}_0}})d\mathbf{X}}}$$

(32)

The first variation of the new combined energy functional $\tilde{E}$ becomes

$$\begin{array}{cc}\hfill \delta \tilde{E}& =\delta E+{\int }_{\mathsf{\Omega }}\dot{h}(\mathbf{W}·{\mathbf{e}}_0){\mathbf{e}}_0·\delta \mathbf{W}d\mathbf{X}+{\int }_{\mathsf{\Gamma }}h(\mathbf{W}·{\mathbf{e}}_0)\delta \mathsf{\Gamma }·\mathbf{N}d\mathbf{S}\hfill \\ \hfill & ={\int }_{\mathsf{\Omega }}\delta \mathbf{W}·\left(\underset{\tilde{\mathbf{V}}}{\underbrace{\mathbf{V}+\dot{h}{\mathbf{e}}_0}}-\nabla \lambda \right)d\mathbf{X}+{\int }_{\mathsf{\Gamma }}\left(\mathbf{W}·\left(\mathbf{V}-\nabla \lambda \right)+h\right)\delta \mathsf{\Gamma }·\mathbf{N}d\mathbf{S}\hfill \end{array}$$

(33)

assuming again, as in (20), that the solenoidal and flux constraints are satisfied for $\mathbf{W}$, and that both $\delta \mathbf{W}·{\mathbf{e}}_0=0$ and $\delta \mathsf{\Gamma }=0$ are imposed along the temporal boundaries $t=0$ and $t=1$ in order to preserve the initial and final densities and spatial supports.

4.2. Optimality Conditions and Partial Gradients

The optimality condition for $\mathbf{W}$ now becomes

$$\underset{\tilde{\mathbf{V}}}{\underbrace{\mathbf{V}+\dot{h}{\mathbf{e}}_0}}-\nabla \lambda =0$$

(34)

while the optimality condition for $\mathsf{\Gamma }$ now becomes

$$\mathbf{W}·\left(\mathbf{V}-\nabla \lambda \right)+h=0$$

(35)

and is not automatically satisfied by the condition for $\mathbf{W}$ anymore. As such, the conundrum we faced before the introduction of the density penalty function h is now resolved.

Choosing the Lagrange multiplier such that

$${\tilde{\mathbf{V}}}^{\parallel }=\nabla \lambda \mathrm{and}{\tilde{\mathbf{V}}}^{\perp }=\tilde{\mathbf{V}}-\nabla \lambda$$

where ${\tilde{\mathbf{V}}}^{\parallel }$ and ${\tilde{\mathbf{V}}}^{\perp }$ denote the curl-free and divergence-free components in the Helmholtz decomposition of

$$\tilde{\mathbf{V}}=\mathbf{V}+\dot{h}{\mathbf{e}}_0$$

(36)

(assuming vanishing flux conditions for ${\tilde{\mathbf{V}}}^{\perp }$) yields the following partial gradient with respect to $\mathbf{W}$

$${\nabla }_{\mathbf{W}}\tilde{E}={\tilde{\mathbf{V}}}^{\perp }=\tilde{\mathbf{V}}-\nabla \lambda$$

(37)

and the following partial gradient with respect to $\mathsf{\Gamma }$

$${\nabla }_{\mathsf{\Gamma }}\tilde{E}=\left(\mathbf{W}·\left(\mathbf{V}-\nabla \lambda \right)+h\right)\mathbf{N}=\left(\mathbf{W}·{\tilde{\mathbf{V}}}^{\perp }+h-\rho \dot{h}\right)\mathbf{N}$$

(38)

Both of these partial gradient expressions require that we first solve the following Poisson equation for the Lagrange multiplier $\lambda$ (in order to compute ${\tilde{\mathbf{V}}}^{\perp }$) with Neumann boundary conditions:

$$\begin{array}{cc}\hfill \mathsf{\Delta }\lambda & =\nabla ·\tilde{\mathbf{V}},\mathbf{X}\in \mathsf{\Omega }\hfill \\ \hfill \nabla \lambda ·\mathbf{N}& =\tilde{V}·\mathbf{N},\mathbf{X}\in \partial \mathsf{\Omega }\hfill \end{array}$$

(39)

4.3. Optimization in $\mathbf{W}$ (Fixed Support)

4.3.1. Computing an Initial Solenoidal Field ${\mathbf{W}}_{\mathrm{init}}$

If we combine the solenoidal constraint and boundary flux conditions for $\mathbf{W}$ with the additional constraint that the initial field ${\mathbf{W}}_{\mathrm{init}}$ be conservative as well, then we may plug $\mathbf{W}=\nabla \mathsf{\Phi }$ into (14) and (15), for some scalar spatiotemporal function $\mathsf{\Phi }$ to obtain the Laplace equation with Neumann boundary conditions (non-homogeneous along the two temporal faces):

$$\begin{array}{cc}\hfill \mathsf{\Delta }\mathsf{\Phi }& =0,\mathbf{X}\in \mathsf{\Omega }\hfill \\ \hfill \nabla \mathsf{\Phi }·\mathbf{N}& =\left\{\begin{array}{cc}0,\hfill & \mathbf{X}\in \mathsf{\Gamma }\hfill \\ -{\rho }_0,\hfill & \mathbf{X}\in {\mathsf{\Gamma }}_0\hfill \\ +{\rho }_1,\hfill & \mathbf{X}\in {\mathsf{\Gamma }}_1\hfill \end{array}\right.\hfill \end{array}$$

(40)

A solution will exist as long as $0={\int }_{d\mathsf{\Omega }}\nabla \mathsf{\Phi }·\mathbf{N}d\mathbf{S}={\int }_{d\mathsf{\Omega }}\mathbf{W}·\mathbf{N}d\mathbf{S}$, which in this case is equivalent to our balanced assumption ${\int }_{{\mathsf{\Omega }}_0}{\rho }_{0}dx={\int }_1{\rho }_{1}dx$ for the initial and target densities. The solution will be unique up to an additive constant, which will then disappear after applying the gradient to obtain the following initial vector field:

$${\mathbf{W}}_{\mathrm{init}}=\nabla \mathsf{\Phi }$$

(41)

4.3.2. Iterated Gradient Descent for the Solenoidal Field $\mathbf{W}$

Referring to our k’th iterated estimate of the optimal $\mathbf{W}$ over our current support estimate as ${\mathbf{W}}_k$ (beginning with ${\mathbf{W}}_0={\mathbf{W}}_{\mathrm{init}}$), a descent step can be taken by constructing ${\tilde{\mathbf{V}}}_k^{\perp }={\tilde{\mathbf{V}}}_k-\nabla \lambda$ after solving the Poisson Equation (39) for $\lambda$ over the existing space-time support $\mathsf{\Omega }$ and then applying a gradient update step

$$\begin{array}{cc}\hfill {\mathbf{W}}_{k+1}& \to {\mathbf{W}}_k-{\gamma }_k{\tilde{\mathbf{V}}}_k^{\perp }\hfill \end{array}$$

(42)

We may use Newton’s method to determine the optimal step factor ${\gamma }_k$ by solving

$$\begin{array}{cc}\hfill 0& =\frac{d{E}_{k+1}}{d{\gamma }_k}={\int }_{\mathsf{\Omega }}{\tilde{\mathbf{V}}}_{k+1}\left({\gamma }_k\right)·{\tilde{\mathbf{V}}}_k^{\perp }d\mathbf{X}\hfill \end{array}$$

for each gradient step. We repeat this process until ${\tilde{\mathbf{V}}}_{k+1}^{\perp }\approx 0$

4.4. Total Shape Gradient

We obtain the total shape gradient $\nabla \tilde{E}$ with respect to the current support boundary $\mathsf{\Gamma }$ by plugging the optimized value of $\mathbf{W}$ over that support (now causing the first variation (33) to depend only upon the shape perturbation $\delta \mathsf{\Gamma }$) into the partial gradient ${\nabla }_{\mathsf{\Gamma }}\tilde{E}$. Since ${\tilde{\mathbf{V}}}^{\perp }=0$ when $\mathbf{W}$ has been optimized for $\mathsf{\Gamma }$, the partial gradient ${\nabla }_{\mathsf{\Gamma }}\tilde{E}$ expression (38) simplifies into the following expression for the total shape gradient:

$$\begin{array}{cc}\hfill \nabla \tilde{E}& =\left(h-\rho \dot{h}\right)\mathbf{N}\hfill \end{array}$$

(43)

An interesting choice for the convex density penalty function h might be, for example,

$$h\left(p\right)=\left\{\begin{array}{cc}{\beta }_{lo}\frac{{\left(\rho -{\rho }_{lo}\right)}^2}{\rho },\hfill & \rho <{\rho }_{lo}\hfill \\ 0,\hfill & {\rho }_{lo}\le \rho \le {\rho }_{hi}\hfill \\ {\beta }_{hi}{\left(\rho -{\rho }_{hi}\right)}^2,\hfill & \rho >{\rho }_{hi}\hfill \end{array}\right.$$

where ${\rho }_{lo}=min\left({min}_{x\in {\mathsf{\Omega }}_0}{\rho }_0\left(x\right),{min}_{x\in {\mathsf{\Omega }}_1}{\rho }_1\left(x\right)\right)$ and ${\rho }_{hi}=max\left({max}_{x\in {\mathsf{\Omega }}_0}{\rho }_0\left(x\right),{max}_{x\in {\mathsf{\Omega }}_1}{\rho }_1\left(x\right)\right)$ denote the lower and upper bounds for a preferred (non-penalized) density range matching the combined ranges of the initial and final densities (to introduce a soft form of maximum principle and minimum principle), and where ${\beta }_{lo},{\beta }_{hi}>0$ represent weighting factors to tune the penalty strength outside of the preferred range. Here, h is chosen to be continuous in its first derivative (though not in its second derivative, which still remains non-negative for convexity)

$$\dot{h}\left(p\right)=\left\{\begin{array}{cc}{\beta }_{lo}\left(\frac{{\rho }^2-{\rho }_{lo}^2}{{\rho }^2}\right),\hfill & \rho <{\rho }_{lo}\hfill \\ 0,\hfill & {\rho }_{lo}\le \rho \le {\rho }_{hi}\hfill \\ 2{\beta }_{hi}\left(\rho -{\rho }_{hi}\right),\hfill & \rho >{\rho }_{hi}\hfill \end{array}\right.\ddot{h}\left(p\right)=\left\{\begin{array}{cc}\frac{2{\beta }_{lo}{\rho }_{lo}^2}{{\rho }^3},\hfill & \rho <{\rho }_{lo}\hfill \\ 0,\hfill & {\rho }_{lo}\le \rho \le {\rho }_{hi}\hfill \\ 2{\beta }_{hi},\hfill & \rho >{\rho }_{hi}\hfill \end{array}\right.$$

as well as tend to infinity when $\rho \to \infty$ and when $\rho \to 0$. Plugging this into the total shape gradient expression (43) would yield

$$\nabla \tilde{E}=\left(h-\rho \dot{h}\right)\mathbf{N}=\left\{\begin{array}{cc}2{\beta }_{lo}{\rho }_{lo}\left(\frac{{\rho }_{lo}-\rho }{\rho }\right),\hfill & \rho <{\rho }_{lo}\hfill \\ 0,\hfill & {\rho }_{lo}\le \rho \le {\rho }_{hi}\hfill \\ {\beta }_{hi}\left({\rho }_{hi}^2-{\rho }^2\right),\hfill & \rho >{\rho }_{hi}\hfill \end{array}\right.$$

which, if applied in the opposite (descent) direction, yields positive (outward) pressure along portions of the current support boundary $\mathsf{\Gamma }$ where the optimized density $\rho$ exceeds the desired upper bound ${\rho }_{hi}$ while yielding negative (inward) pressure along portions of $\mathsf{\Gamma }$ where the optimized density falls below the desired lower bound ${\rho }_{lo}$.

5. Results

We conclude with two experimental results which illustrate the benefits of this variational approach, stemming in particular from its separate yet coupled optimization of the compact spatiotemporal support and the density within. While the mathematical formulation of the approach is fully outlined here, the numerical implementation strategies are still under investigation and, as such, the following results are intended to be preliminary indications of what we may expect after more sophisticated numerical strategies have been further explored and developed.

5.1. Interpolation between Two Different Non-Convex Supports

In this first example, we tackle the problem of interpolating between cardiac images captured at two different moments within the heartbeat cycle shown on the left and the right in Figure 1 Notice that both cell boundaries represent non-convex shapes with several small concavities. We also see structures of interest inside the ventricle (papillary muscle cross sections), which not only move and deform along with the rest of the image but which also change in their topological appearance. At a coarse scale, the boundary shapes appear to be similar, but detailed inspection reveals that finer scale protrusions and intrusions around the boundary differ, especially the concavity along the upper right edge of the left image which disappears in the right image. Nevertheless, both boundaries exhibit a matching simple topology, which we would like to preserve when morphing one into the other. Numerically, this can be challenging without an explicit model of the support boundary, making it difficult to guarantee that small-scale protrusions do not break off during transport to yield transitional topological changes.

Using these two cardiac images as the starting and ending densities at time $t=0$ and $t=1$, respectively, we solve the compact optimal transport problem with the variational approach outlined in this paper, using a 3D space-time grid for the solenoidal vector field $\mathbf{W}$ with 64 temporal slices, each of size 128 × 128 (same resolution as the two input images). A matching level set grid $\mathsf{\Psi }$ is used to represent the spatiotemporal support as the set $\mathsf{\Psi }<0$. We see the density (temporal component of $\mathbf{W}$) in Figure 1 at equally spaced intervals along the computed transport. We can see this more explicitly by visualizing the entire swept-out hypersurface $\mathsf{\Gamma }$ (the portion of the spatiotemporal support boundary strictly between 0 and 1) in Figure 2. This is very easily rendered as the zero level set of $\mathsf{\Psi }$ and clearly reveals a smooth homotopy connecting the two end curves, one exposed within the top rendering and the other within the bottom).

5.2. Optimal Transport with Support Constraints

This next experiment illustrates through an intuitive toy example an important and useful extension to the class of optimal transport problems which, to the best of our knowledge, is not accommodated by prior methods but is easily handled within the methodology outlined here. Namely, how might one compute the optimal transport between densities subject to constraints imposed on where (and possibly even when) mass is allowed or not allowed to move along the way?

5.2.1. Known Densities with Intermediate Support Constraints

We can easily motivate the utility of such constraints if we go back to the classic problem posed by Gaspard Monge. In formulating the problem to optimally move a pile of dirt from one place to another, no constraints were imposed on the path taken by each portion of moved dirt. While the unconstrained optimal solution may yield a realistic and realizable transport strategy in several practical circumstances, this cannot always be guaranteed. For example, suppose the task is to move a pile of dirt across a limited number of bridges to the other side of a river. The unconstrained solution could easily yield an impractical transport strategy which involves crossing open portions of the river. A related larger-scale problem might involve planning the motion of massive numbers of land troops distributed over a set of territories to another set of territories, taking into account geographical barriers such as mountains and bodies of water as well as political barriers, which would render certain intermediate territories out of bounds.

Even when the topology of the desired transport is known, optimization subject to geometric constraints can still be non-trivial. For example, transporting a single pile of dirt across a single bridge which is narrower than the base of the pile is already an interesting problem. The unconstrained optimal transport will likely want to move some portion of dirt outside the confines of the bridge. Should all of that excess dirt simply be re-routed and accumulated along the closest edge of the bridge, or should some of it be moved more centrally inside the bridge, which would make the trajectory deviate even further from the unconstrained optimum but attenuate an otherwise massive spike in density along the edge? Clearly there is a trade-off that is not easily intuited directly from the unconstrained solution.

These types of constraints are easily imposed using the variational framework presented in this paper due to the explicit representation and separate-yet-coupled evolution of the support boundary. As long as we choose an initial spatiotemporal support that satisfies the provided set of spatiotemporal constraints, the calculated gradient descent evolution of the resulting spatiotemporal hypersurface can simply be set to zero locally wherever its application would otherwise violate the constraints.

5.2.2. Unknown Final Density but with Known Support

Another extension of problems that are easily accommodated by this coupled variational approach include scenarios where the support of the target density is given but the target density itself is unknown (and therefore part of the optimization). Such a problem can easily be transformed into the problem of a known final density with intermediate constraints on the support by treating the desired final density as the halfway point ($t=\frac{1}{2}$) in transporting the initial density at $t=0$ back to itself again at $t=1$. In this way, the desired final support becomes a constraint on the intermediate support instead. Optimization of this reconfigured problem will yield both a forward copy (from $t=0$ to $t=\frac{1}{2}$) and a backward copy (from $t=\frac{1}{2}$ to $t=1$) of the optimal transport for the original problem as well as the optimal target density itself at $t=\frac{1}{2}$. As such, from an implementation standpoint, this class of problems can be handled the same way as the class of problems just described above.

A practical application for this form of constrained optimal transport would be to compute the most efficient evacuation strategy to clearing mass out of a given subregion while keeping it within a larger encompassing region that already contains pre-existing mass. In this case, we know the initial density and support, and we know the final support, simply the initial support minus the subregion to be evacuated, but we do not know or otherwise want to constrain the resulting new density within the now reduced support. We therefore seek the least costly way (according to the Wasserstein metric) to redistribute the mass originally contained within the subregion into its surrounding, already occupied neighborhood. Such a problem may arise, for example, when seeking to clear extensive zones of all materials and/or personnel while keeping them within the confines of larger zones whose occupants are free to be internally relocated if needed.

We illustrate precisely this scenario with an intuitive toy problem in Figure 3. We start with uniformly distributed mass at $t=0$ with density 3.0 across a disk representing the global confines. In turn, we define the target distribution at $t=1$ to be the same but impose a hole in the intermediate support at $t=\frac{1}{2}$ within the center of the disk. Solving the constrained optimal transport problem between these matching uniform distributions causes the initially filled hole to be evacuated from $t=0$ to $t=\frac{1}{2}$ as illustrated from left-to-right and top-to-bottom in Figure 3, and then to be refilled from $t=\frac{1}{2}$ to $t=1$ as also illustrated in Figure 3 when read in reverse. The optimal redistribution of mass is attained at $t=\frac{1}{2}$ and is displayed in grayscale at the end of the figure.

Even in this simplest illustrative example with constant density and maximal symmetry, it is by no means intuitively obvious how far away mass should be displaced from the hole boundary compared to how much it should be allowed to accumulate along the boundary. In fact, the density would become infinite if the evacuated mass were to remain strictly along the boundary. To obtain a better sense of where displaced mass accumulates upon pre-existing mass (displaced mass includes mass evacuated from the hole as well as mass moved away to make room), we show the net density change in Figure 4 (left side), which attains a maximum rise of 2.3 all around the boundary of the evacuated hole and gradually rolls off further outward.

We can make this toy problem even more interesting by evacuating mass near the boundary of the disk rather than its center. Looking at the resulting net density increase in Figure 4 (right side), we can make several observations. First, as expected, the redistribution is no longer symmetric. The symmetry is broken two different ways. First, the rise in density is much higher (3.0, a full 100% jump) along the bottom border of the hole compared to the top border. This is unsurprising since there is not as much room to move away from the hole, and so the same amount of evacuated mass distributed over a thinner local neighborhood must necessarily result in a larger accumulated density.

However, we also notice that the density jump near the top of the hole (2.4), while lower than the bottom, still exceeds the symmetric jump (2.3) when the hole was centered inside the disk. This differential grows as we travel along the border of the hole toward the bottom. This means that some of the mass within the lower half of the hole, which was all evacuated downward from the centered hole, actually was evacuated upward from the hole near the boundary of the confining domain. As such, at least two interesting interplays are relevant in this optimization. First, as in the symmetric case, how far away should mass be displaced away from borders versus accumulated along borders? Second, when displacement distances are limited, what is the right trade-off between higher accumulation at nearby borders versus more costly redirection toward farther borders of the region to be evacuated? These considerations are both naturally handled by this coupled variational framework.