# Image Segmentation with a Priori Conditions: Applications to Medical and Geophysical Imaging

^{1}

^{2}

^{3}

^{*}

^{†}

## Abstract

**:**

## 1. Introduction

## 2. Mathematical Modelling

#### 2.1. A Priori Conditions

#### 2.2. Minimization Problem and Evolution Equation

#### 2.3. Existence, Uniqueness of the Solution

- $\tilde{F}\in C([0,T]\times \overline{\mathsf{\Omega}}\times \mathrm{I}\phantom{\rule{-1.79993pt}{0ex}}\mathrm{R}\times (\mathrm{I}\phantom{\rule{-1.79993pt}{0ex}}{\mathrm{R}}^{\mathrm{n}}-\left\{0\right\})\times {\mathrm{S}}^{\mathrm{n}}),$ where ${S}^{n}$ denotes the space of $n\times n$ symmetric matrices equipped with the usual ordering.
- There exists a constant $\gamma \in \mathrm{I}\phantom{\rule{-1.79993pt}{0ex}}\mathrm{R}$ such that for each:$$(t,x,p,X)\in [0,T]\times \overline{\mathsf{\Omega}}\times (\mathrm{I}\phantom{\rule{-1.79993pt}{0ex}}{\mathrm{R}}^{\mathrm{n}}-\left\{0\right\})\times {\mathrm{S}}^{\mathrm{n}},$$$$u\mapsto \tilde{F}(t,x,u,p,X)-\gamma u$$
- For each $R>0$, there exists a continuous function ${w}_{R}:[0,\infty [\u27f6[0,\infty [$ satisfying ${w}_{R}\left(0\right)=0$ such that if $X,Y\in {S}^{n}$ and ${\mu}_{1},{\mu}_{2}\in [0,\infty [$ satisfy:$$\left(\begin{array}{cc}X& 0\\ 0& Y\end{array}\right)\le {\mu}_{1}\left(\begin{array}{cc}I& -I\\ -I& I\end{array}\right)+{\mu}_{2}\left(\begin{array}{cc}I& 0\\ 0& I\end{array}\right)$$$$\begin{array}{c}\tilde{F}(t,x,u,p,X)-\tilde{F}(t,y,u,q,-Y)\hfill \\ \ge -{w}_{R}({\mu}_{1}{\left(\right|x-y|}^{2}+\rho {(p,q)}^{2})\hfill \\ +{\mu}_{2}+|p-q|+|x-y|(1+max(\left|p\right|,\left|q\right|\left)\right)),\hfill \end{array}$$
- $B\in C(\mathrm{I}\phantom{\rule{-1.79993pt}{0ex}}{\mathrm{R}}^{\mathrm{n}}\times \mathrm{I}\phantom{\rule{-1.79993pt}{0ex}}{\mathrm{R}}^{\mathrm{n}})\bigcap {\mathrm{C}}^{1,1}(\mathrm{I}\phantom{\rule{-1.79993pt}{0ex}}{\mathrm{R}}^{\mathrm{n}}\times (\mathrm{I}\phantom{\rule{-1.79993pt}{0ex}}{\mathrm{R}}^{\mathrm{n}}\setminus \left\{0\right\})).$
- For each $x\in \mathrm{I}\phantom{\rule{-1.79993pt}{0ex}}{\mathrm{R}}^{\mathrm{n}}$, the function $p\mapsto B(x,p)$ is positively homogeneous of degree one in p, i.e., $B(x,\lambda p)=\lambda B(x,p)$, $\forall \lambda \ge 0,p\in \mathrm{I}\phantom{\rule{-1.79993pt}{0ex}}{\mathrm{R}}^{\mathrm{n}}\setminus \left\{0\right\}$.
- There exists a positive constant $\theta $ such that $<\nu \left(z\right),{D}_{p}B(z,p)>\ge \theta $ for all $z\in \partial \mathsf{\Omega}$ and $p\in \mathrm{I}\phantom{\rule{-1.79993pt}{0ex}}{\mathrm{R}}^{\mathrm{n}}-\left\{0\right\}$. Here, $\nu \left(z\right)$ denotes the unit outer normal vector of $\mathsf{\Omega}$ at $z\in \partial \mathsf{\Omega}.$

- Thanks to [20], we already know that $F\in C([0,T]\times \overline{\mathsf{\Omega}}\times (\mathrm{I}\phantom{\rule{-1.79993pt}{0ex}}{\mathrm{R}}^{\mathrm{n}}\setminus \left\{0\right\})\times {\mathcal{S}}^{\mathrm{n}})$. Since, $\eta ,{\mathsf{\Phi}}_{0}\in C\left(\overline{\mathsf{\Omega}}\right)$, it is clear that $g\in C(\overline{\mathsf{\Omega}}\times \mathrm{I}\phantom{\rule{-1.79993pt}{0ex}}\mathrm{R})$ and therefore $\tilde{F}\in C([0,T]\times \overline{\mathsf{\Omega}}\times \mathrm{I}\phantom{\rule{-1.79993pt}{0ex}}\mathrm{R}\times (\mathrm{I}\phantom{\rule{-1.79993pt}{0ex}}{\mathrm{R}}^{\mathrm{n}}\setminus \left\{0\right\})\times {\mathcal{S}}^{\mathrm{n}}$.
- Note that since F does not depend on u. It suffices to show that $u\to g(x,u)-\gamma u$ is non-decreasing on $\mathrm{I}\phantom{\rule{-1.79993pt}{0ex}}\mathrm{R}$. On can easily see that the condition is satisfied for all $x\in \overline{\mathsf{\Omega}}$ as soon as $\gamma <0$.
- Since g does not depend on X, it suffices to check the condition on F. We refer the reader to Gout and Le Guyader [20] for the proof.

## 3. Experimental Results

- –
- Following the considered application, the initial condition can be a set of points, or a curve (then can be constructed from a set points using a basic spline function (see Gout et al. [22])).
- –
- The stop criterion can be either a preset number of iterations or a check that the solution is stationary.
- –
- The distance is normalized in order to have the same weight between a priori information of the image and geometrical constraints.
- –
- The discretization is made using finite differences as done in Chan and Vese [11].
- –
- In the numerical examples, we take $\delta t=0.1$, the regularization term is equal to 0.8.

#### 3.1. Impact of the Initial Guess on the Segmentation Process

#### 3.2. Quantitative Performance

- –
- The Jaccard index [30], or Intersection over Union (IoU), is a commonly used metric in segmentation. It is defined as the area of intersection between the predicted segmentation map and the ground truth, divided by the area of union between the predicted segmentation map and the ground truth:$$\mathrm{IoU}=J(G,P)={\displaystyle \frac{\left|G\cap P\right|}{\left|G\cup P\right|}},$$
- –
- The Dice coefficient (Dice) is a popular metric for image segmentation, especially in medical imaging. This coefficient can be defined as twice the overlap area of predicted and ground-truth maps, divided by the total number of pixels in both images:$$\mathrm{Dice}={\displaystyle \frac{2\left|G\cap P\right|}{\left|G\right|+\left|P\right|}}$$When applied to binary segmentation maps, and referring to the foreground as a positive class, the Dice coefficient is essentially identical to the F1 score:$$\mathrm{Dice}={\displaystyle \frac{2TP}{2TP+FP+FN}}=\mathrm{F}1,$$(The F1 score, which is defined as the harmonic mean of precision (Prec) and recall (Rec): $\mathrm{F}1={\displaystyle \frac{2\phantom{\rule{4.pt}{0ex}}\mathrm{Prec}\phantom{\rule{4.pt}{0ex}}\mathrm{Rec}}{\mathrm{Prec}\phantom{\rule{4.pt}{0ex}}+\phantom{\rule{4.pt}{0ex}}\mathrm{Rec}}},$ where Prec = $\frac{TP}{TP+FP}$ and Rec = $\frac{TP}{TP+FN}}.$), where TP refers to the true positive fraction, FP refers to the false positive fraction, and FN refers to the false negative fraction.
- –
- The Hausdorff distance (Hd) evaluates the quality of the segmentation boundaries by computing the maximum distance between the prediction and its ground truth:$$\mathrm{Hd}=max\left\{\underset{p\in \partial P}{sup}\phantom{\rule{4pt}{0ex}}\underset{z\in \partial G}{inf}{\u2225p-z\u2225}_{2},\underset{z\in \partial G}{sup}\phantom{\rule{4pt}{0ex}}\underset{p\in \partial P}{inf}{\u2225p-z\u2225}_{2}\right\}$$

#### 3.3. Applications to Medical Imaging

#### 3.4. Applications to Geophysical Imaging

## 4. Conclusions

## Author Contributions

## Funding

## Acknowledgments

## Conflicts of Interest

## References

- Fukushima, K. Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biol. Cybern.
**1980**, 36, 193–202. [Google Scholar] [CrossRef] [PubMed] - Waibel, A.; Hanazawa, T.; Hinton, G.; Shikano, K.; Lang, K.J. Phoneme recognition using time-delay neural networks. IEEE Trans. Acoust. Speech Signal Process.
**1989**, 37, 328–339. [Google Scholar] [CrossRef] - LeCun, Y.; Bottou, L.; Bengio, Y.; Haffner, P. Gradient-based learning applied to document recognition. Proc. IEEE
**1998**, 86, 2278–2324. [Google Scholar] [CrossRef][Green Version] - Hochreiter, S.; Schmidhuber, J. Long short-term memory. Neural Comput.
**1997**, 9, 1735–1780. [Google Scholar] [CrossRef] - Badrinarayanan, V.; Kendall, A.; Cipolla, R. Segnet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell.
**2017**, 39, 2481–2495. [Google Scholar] [CrossRef] - Goodfellow, I.; Pouget-Abadie, J.; Mirza, M.; Xu, B.; Warde-Farley, D.; Ozair, S.; Courville, A.; Bengio, Y. Generative adversarial nets. In Proceedings of the Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, Montreal, QC, Canada, 8–13 December 2014; pp. 2672–2680. [Google Scholar]
- Minaee, S.; Boykov, Y.Y.; Porikli, F.; Plaza, A.J.; Kehtarnavaz, N.; Terzopoulos, D. Image Segmentation Using Deep Learning: A Survey. IEEE Trans. Pattern Anal. Mach. Intell.
**2021**, 1. [Google Scholar] [CrossRef] - Fantazzini, A.; Esposito, M.; Finotello, A.; Auricchio, F.; Conti, M.; Pane, B.; Spinella, G. 3D Automatic Segmentation of Aortic Computed Tomography Angiography Combining Multi-View 2D Convolutional Neural Networks. Cardiovasc. Eng. Tech.
**2020**, 11, 576–586. [Google Scholar] [CrossRef] - Luo, Q.; Wang, L.; Lv, J.; Xiang, S.; Pan, C. Few-shot learning via feature hallucination with variational inference. In Proceedings of the IEEE Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA, 3–8 January 2021; pp. 3963–3972. [Google Scholar]
- Kass, M.; Witkin, A.; Terzopoulos, D. Snakes: Active contour models. Int. J. Comput. Vis.
**1988**, 1, 321–331. [Google Scholar] [CrossRef] - Chan, T.F.; Vese, L.A. Active contours without edges. IEEE Trans. Image Process.
**2001**, 10, 266–277. [Google Scholar] [CrossRef][Green Version] - Mumford, D.; Shah, J. Optimal approximation by piecewise smooth functions and associated variational problems. Comm. Pure Appl. Math.
**1989**, 42, 577–685. [Google Scholar] [CrossRef][Green Version] - Vese, L.; Le Guyader, C. Variational Methods in Image Processing; CRC Press: Boca Raton, FL, USA; Taylor and Francis: Oxfordshire, UK, 2015; 410p, ISBN 9781439849736. [Google Scholar]
- Sethian, J.A. A fast marching level set method for monotonically advancing fronts. Proc. Natl. Acad. Sci. USA
**1996**, 93, 1591–1595. [Google Scholar] [CrossRef] [PubMed][Green Version] - Forcadel, N.; Le Guyader, C.; Gout, C. Generalized fast marching method: Applications to image segmentation. Numer. Algorithms
**2008**, 48, 189–211. [Google Scholar] [CrossRef] - Caselles, V.; Kimmel, R.; Sapiro, G. Geodesic Active Contours. Int. J. Comput. Vis.
**1997**, 22, 61–79. [Google Scholar] [CrossRef] - Le Guyader, C.; Lambert, Z.; Petitjean, C. Analysis of the weighted Van der Waals-Cahn-Hilliard model for image segmentation. In Proceedings of the 10th International Conference on Image Processing Theory, Tools and Applications IPTA 2020, Paris, France, 9–12 November 2020. [Google Scholar]
- Cremers, D.; Sochen, N.; Schnörr, C. Towards Recognition-Based Variational Segmentation Using Shape Priors and Dynamic Labeling. Lect. Notes Comput. Sci.
**2003**, 2695, 388–400. [Google Scholar] - Gout, C.; Le Guyader, C.; Vese, L. Segmentation under geometrical conditions using geodesic active contours and interpolation using level set methods. Numer. Algorithms
**2005**, 39, 155–173. [Google Scholar] [CrossRef][Green Version] - Gout, C.; Le Guyader, C. Geodesic active contour under geometrical conditions: Theory and 3D applications. Numer. Algorithms
**2008**, 48, 105–133. [Google Scholar] - Gout, C.; Vieira-Teste, S. Using deformable models to segment complex structures under geometric constraints. In Proceedings of the 4th IEEE Southwest Symposium on Image Analysis and Interpretation, Austin, TX, USA, 2–4 April 2000; pp. 101–105. [Google Scholar]
- Gout, C.; Lambert, Z.; Apprato, D. Data Approximation: Mathematical Modelling and Numerical Simulations; INSA Rouen Normandie; EDP Sciences: Les Ulis, France, 2019; 168p, ISBN 9978-2-7598-2367-3. [Google Scholar]
- Osher, S.; Sethian, J.A. Fronts propagating with curvature dependent speed: Algorithms based on Hamilton-Jacobi formulations. J. Comput. Phys.
**1988**, 79, 12–49. [Google Scholar] [CrossRef][Green Version] - Osher, S.; Fedkiw, R. Level Set Methods and Dynamic Implicit Surfaces; Springer: Cham, Switzerland, 2003. [Google Scholar]
- Weickert, J.; Kühne, G. Fast methods for implicit active contours models. In Geometric Level Set Methods in Imaging, Vision, and Graphics; Springer: New York, NY, USA, 2003. [Google Scholar]
- Cohen, L.D. On active contours models and balloons. Comput. Vision Graph. Image Process. Image Underst.
**1991**, 53, 211–218. [Google Scholar] [CrossRef] - Ronneberger, O.; Fischer, P.; Brox, T. U-net: Convolutional networks for biomedical image segmentation. In Proceedings of the MICCAI International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany, 5–9 October 2015; Springer: Cham, Switzerland, 2015; pp. 234–241. [Google Scholar]
- Menze, B.H.; Jakab, A.; Bauer, S.; Kalpathy-Cramer, J.; Farahani, K.; Kirby, J.; Burren, Y.; Porz, N.; Slotboom, J.; Wiest, R.; et al. The multimodal brain tumor image segmentation benchmark (BRATS). IEEE Trans. Med. Imaging
**2014**, 34, 1993–2024. [Google Scholar] [CrossRef] - Kingma, D.P.; Ba, J. Adam: A method for stochastic optimization, International Conference on Learning Representations. arXiv
**2015**, arXiv:1412.6980. [Google Scholar] - Jaccard, P. Distribution de la flore alpine dans le bassin des Dranses et dans quelques régions voisines. Bull. SociéTé Vaudoise Sci. Nat.
**1901**, 37, 241–272. [Google Scholar] - Kim, B.; Ye, J.C. Cycle-Consistent Adversarial Network with Polyphase U-Nets for Liver Lesion Segmentation. 2018. Available online: https://openreview.net/pdf?id=SyQtAooiz (accessed on 2 March 2021).
- Laffon, E.; Laurent, F.; Bernard, V.; De Boucaud, L.; Ducassou, D.; Marthan, R. Noninvasive assessment of pulmonary arterial hypertension by MR phase-mapping method. J. Appl. Physiol.
**2001**, 90, 2197–2202. [Google Scholar] [CrossRef] [PubMed][Green Version] - Laffon, E.; Vallet, C.; Bernard, V.; Montaudon, M.; Ducassou, D.; Laurent, F.; Marthan, R. A computed method for non invasive MRI assessment of pulmonary arterial hypertension. J. Appl. Physiol.
**2004**, 96, 463–468. [Google Scholar] [CrossRef] - Debroux, N.; Le Guyader, C. A joint segmentation/registration model based on a nonlocal characterization of weighted total variation and nonlocal shape descriptors. SIAM J. Imaging Sci.
**2018**, 11, 957–990. [Google Scholar] [CrossRef] - Ozeré, S.; Le Guyader, C.; Gout, C. Joint segmentation/registration model by shape alignment via weighted total variation minimization and nonlinear elasticity. SIAM J. Imaging Sci.
**2015**, 8, 1981–2020. [Google Scholar] [CrossRef] - Gout, C.; Le Guyader, C.; Romani, L.; Saint-Guirons, A.-G. Approximation of surfaces with fault(s) and/or rapidly varying data, using a segmentation process, Dm-splines and the finite element method. Numer. Algorithms
**2008**, 48, 67–92. [Google Scholar] [CrossRef] - Le Guyader, C.; Apprato, D.; Gout, C. On the Construction of Topology-Preserving Deformation Fields. IEEE Trans. Image Process.
**2012**, 21, 1587–1599. [Google Scholar] [CrossRef] [PubMed]

**Figure 1.**Three-band image (

**top left**), user-defined constraint (

**top right**), initial contour automatically defined by our approach (

**bottom left**) and final contour (

**bottom right**).

**Figure 2.**We use Gout et al. [19] algorithm. The initial guess is given inside the region of interest, it is defined by $\Psi ={(X-39)}^{2}+{(Y-47)}^{2}-{18}^{2}$, the time step is equal to 0.6, space step is equal to 0.1, distance is computed using the fast marching method (Sethian [14]) and the regularization term is equal to 0.8.

**Top left**: Interpolation condition and initial closed contour.

**Top right**: iteration 120.

**Bottom left**: Iteration 240.

**Bottom right**: Iteration 420.

**Figure 3.**Final result (iteration 420), with the interpolation conditions (3 points) using Gout et al. algorithm [19].

**Figure 4.**Using our proposed approach, the process does not require an initial guess, the interpolation conditions automatically initiate the process.

**Left**: interpolation conditions.

**Right**: final result after 80 iterations.

**Figure 5.**We use Gout et al. [19] algorithm. The initial guess is given outside the region of interest, it is defined by $\Psi ={\displaystyle \frac{{(X-30)}^{2}}{{15}^{2}}}+{\displaystyle \frac{{(Y-30.5)}^{2}}{{23}^{2}}}-1$, the time step is equal to 0.3, space step is equal to 0.1, distance is computed using the fast marching method (Sethian [14]) and the regularization term is equal to 0.8.

**Top left**: Interpolation condition and initial guess.

**Top right**: iteration 80.

**Bottom left**: Iteration 100.

**Bottom right**: Iteration 160.

**Figure 6.**Final result (iteration 260), with the interpolation conditions (3 points) using Gout et al. algorithm [19].

**Figure 7.**

**Left**: interpolation conditions.

**Right**: final result after 8 iterations using our proposed approach.

**Figure 8.**Brain Tumor Segmentation (using BRATS dataset [28]). Several results from our method using labeled data on the BRATS dataset [28]. The tumor is underlined in yellow. First row: center slices of input. Second row: initial conditions for our algorithm. Third row: ground-truth labels. Fourth row: results from the supervised U-Nets learning method introduced by Ronnenberger et al. [27]. Fifth row: results from our proposed method. The scores in the bottom of each results denote Dice score.

**Figure 9.**Example of an image sequence (courtesy of CHU Bordeaux, France). The arrow shows the pulmonary artery to segment.

**Figure 10.**Blood vessel image, user-defined constraint (

**top right**), initial contour (

**bottom left**) and final (

**bottom right**) contours. Let us note that we have successfully segmented the complete image sequence.

**Figure 11.**

**Left**: 3D seismic dataset (courtesy of TotalEnergies, Pau, France). Two strong and continuous reflectors (Horizon A and Horizon B) appear.

**Right**: 2D image (extracted from the 3D seismic dataset) with vertical faults and layers. Considering the complexity of such dataset, it is impossible to segment it without a priori given conditions. The segmentation process consists in finding layers and/or faults.

**Figure 12.**

**Left**: The red zone underlines the vertical fault (with dotted segments).

**Right**: As a geometric conditions, we consider a set of points given by the user.

**Figure 13.**Final result: we can see that the searched layer is correctly segmented. On this specific dataset, the final contour has to reach the edge of the image: the contour goes beyond the last point to interpolate (right side) to join the edge of the image.

**Figure 14.**

**Top**: Another 3D view of the 3D geological dataset of Figure 5.

**Bottom**: From the 3D dataset, we use a set of points (well data given by geologists), and each surface/layer is then constructed with a ${D}^{m}$-spline operator (see [22] for more details on ${D}^{m}$-spline surface approximation). We here automatically construct 7 surfaces that will be considered as an initial condition. Next step consists in the 3D segmentation process from these initial conditions—this last part is a work in progress.

Method | Example 1 | Example 2 |
---|---|---|

Gout et al. [19] | 420 iterations | 260 iterations |

Our method | 80 iterations | 8 iterations |

**Table 2.**Accuracies of segmentations models on a third of labeled data from BraTS Dataset [28]. We use a Nvidia GeForce RTX 2080 Ti 11G Turbo Edition (Boost frequence: 1545 MHz, GPU memory: 11 GB).

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Khayretdinova, G.; Gout, C.; Chaumont-Frelet, T.; Kuksenko, S.
Image Segmentation with a Priori Conditions: Applications to Medical and Geophysical Imaging. *Math. Comput. Appl.* **2022**, *27*, 26.
https://doi.org/10.3390/mca27020026

**AMA Style**

Khayretdinova G, Gout C, Chaumont-Frelet T, Kuksenko S.
Image Segmentation with a Priori Conditions: Applications to Medical and Geophysical Imaging. *Mathematical and Computational Applications*. 2022; 27(2):26.
https://doi.org/10.3390/mca27020026

**Chicago/Turabian Style**

Khayretdinova, Guzel, Christian Gout, Théophile Chaumont-Frelet, and Sergei Kuksenko.
2022. "Image Segmentation with a Priori Conditions: Applications to Medical and Geophysical Imaging" *Mathematical and Computational Applications* 27, no. 2: 26.
https://doi.org/10.3390/mca27020026