# InvMap and Witness Simplicial Variational Auto-Encoders

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Notions of Computational Topology

#### 2.1. Simplicial Complexes

**Definition**

**1.**

**Definition**

**2.**

- Each face of any simplex of K is also a simplex of K.
- The intersection of any two simplices of K is either empty or a face of both simplices.

**Definition**

**3.**

**Definition**

**4.**

#### 2.2. Betti Numbers

## 3. Related Work

#### 3.1. Non-Linear Dimensionality Reduction

#### 3.2. Variational Auto-Encoder

#### 3.3. Topology and Auto-Encoders

## 4. Implementation Details

## 5. Problem Formulation

#### 5.1. Datasets

#### 5.2. Illustration of the Problem

## 6. InvMap VAE

#### 6.1. Method

#### 6.2. Results

## 7. Witness Simplicial VAE

#### 7.1. Method

#### 7.1.1. Witness Complex Construction

#### 7.1.2. Witness Complex Simplicial Regularization

- It does not depend on any embedding whereas in [28] the author was relying on a UMAP embedding for his simplicial regularization of the decoder.
- We use only one witness simplical complex built from the input data whereas the author of [28] was using one fuzzy simplicial complex built from the input data and a second one built from the UMAP embedding and both were built via the fuzzy simplicial set function provided with UMAP (keeping only simplices with highest probabilities).

- ${\mathcal{L}}_{SE}$ the simplicial regularization term for the encoder.
- ${\mathcal{L}}_{SD}$ the simplicial regularization term for the decoder.
- e and d, respectively, the (probabilistic) encoder and decoder.
- K a (witness) simplicial complex built from the input space.
- $\sigma $ a simplex belonging to the simplicial complex K.
- ${\sigma}_{j}$ the vertex number j of the $dim\left(\sigma \right)$-simplex $\sigma $ which has exactly $dim\left(\sigma \right)+1$ vertices. ${\sigma}_{j}$ is, thus, a data point in the input space X.
- $\mathrm{MSE}\phantom{\rule{4.pt}{0ex}}(a,b)$ the Mean Square Error between a and b.
- ${\mathbb{E}}_{{\lambda}_{j}\sim Dir(dim\left(\sigma \right)+1,\alpha )}$ the expectation for the ${\left({\lambda}_{j}\right)}_{j=0,\dots ,dim\left(\sigma \right)}$ following a symmetric Dirichlet distribution with parameters $dim\left(\sigma \right)+1$ and $\alpha $. When $\alpha =1$, which is what we used in practice, the symmetric Dirichlet distribution is equivalent to a uniform distribution over the $dim\left(\sigma \right)$-simplex $\sigma $, and as $\alpha $ tends towards 0, the distribution becomes more concentrated on the vertices.

#### 7.1.3. Witness Simplicial VAE

- Perform a witness complex filtration of the input data to obtain a persistence diagram (or a barcode).
- Build a witness complex given the persistence diagram (or the barcode) of this filtration, and potentially any additional information on the Betti numbers which should be preserved according to the problem (number of connected components, 1-dimensional holes…).
- Train the model using this witness complex to compute the loss ${\mathcal{L}}_{WSVAE}$ of Equation (5).

#### 7.1.4. Isolandmarks Witness Simplicial VAE

- ${\mathcal{L}}_{WSVAE}$ the loss of the Witness Simplicial VAE.
- l the number of landmarks.
- ${\parallel .\parallel}_{F}$ the Frobenius norm.
- ${D}^{*}$ the approximate geodesic distance matrix of the landmarks points in the input space computed once before learning.
- D the Euclidean distance matrix of the encodings of the landmarks points computed at each batch.
- K the Isomap kernel defined as $K\left(D\right)=-0.5(I-\frac{1}{l}A){D}^{2}(I-\frac{1}{l}A)$ with I the identity matrix and A the matrix composed only by ones.

#### 7.2. Results

## 8. Discussion

## 9. Conclusions

## Author Contributions

## Funding

## Data Availability Statement

## Acknowledgments

## Conflicts of Interest

## Abbreviations

AE | Auto-Encoder |

i.i.d. | independent and identically distributed |

ELBO | Evidence lower bound |

Isomap | Isometric Mapping |

k-nn | k-nearest neighbours |

MDPI | Multidisciplinary Digital Publishing Institute |

MSE | Mean Square Error |

s.t. | such that |

TDA | Topological Data Analysis |

UMAP | Uniform Manifold Approximation and Projection |

VAE | Variational Auto-Encoder |

WC | Witness complex |

## Appendix A. Variational Auto-Encoder Derivations

#### Appendix A.1. Derivation of the Marginal Log-Likelihood

#### Appendix A.2. Derivation of the ELBO

## Appendix B. UMAP-Based InvMap-VAE Results

**Figure A1.**UMAP-based InvMap-VAE applied to the swissroll data set, $weigh{t}_{IM}=10$ and trained for 1000 epochs. Betti numbers ${\beta}_{0}=1$ and ${\beta}_{1}=0$ are preserved between the original dataset (

**a**), the latent representation ((

**c**) for training and (

**e**) for test), and the reconstruction ((

**d**) for training and (

**f**) for test). Latent representations are similar to the UMAP embedding.

## Appendix C. Illustration of the Importance of the Choice of the Filtration Radius Hyperparameter for the Witness Complex Construction

**Figure A2.**Examples of bad witness complexes (WC) constructions for the Swissroll. On the top (

**a**,

**b**), the built witness complex is bad because it has two connected components instead of one. On the bottom (

**c**,

**d**), the witness complex is bad again, but because the radius filtration chosen is too high.

## Appendix D. Bad Neural Network Weights Initialization with Witness Simplicial VAE

**Figure A3.**Worse results of the Witness Simplicial VAE applied to the open cylinder after 200 epochs, each row is a different neural network initialization: latent representations of the training set on the left and corresponding reconstruction on the right. Hyper parameters: batch size = 128, 10 landmarks, ${r}_{filtration}=6$, weights ${w}_{SE}={w}_{SD}=10$. (

**a**) Latent representation. (

**b**) Reconstruction. (

**c**) Latent representation. (

**d**) Reconstruction.

## References

- Goodfellow, I.; Pouget-Abadie, J.; Mirza, M.; Xu, B.; Warde-Farley, D.; Ozair, S.; Courville, A.; Bengio, Y. Generative Adversarial Nets. In Advances in Neural Information Processing Systems 27 (NIPS 2014); Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N., Weinberger, K.Q., Eds.; Curran Associates, Inc.: Red Hook, NY, USA, 2014; Volume 27. [Google Scholar]
- Kingma, D.P.; Welling, M. Auto-Encoding Variational Bayes. In Proceedings of the ICLR, Banff, AB, Canada, 14–16 April 2014. [Google Scholar]
- Rezende, D.J.; Mohamed, S.; Wierstra, D. Stochastic Backpropagation and Approximate Inference in Deep Generative Models. In Proceedings of the 31st International Conference on Machine Learning, Beijing, China, 21–26 June 2014; Xing, E.P., Jebara, T., Eds.; PMLR: Bejing, China, 2014; Volume 32, pp. 1278–1286. [Google Scholar]
- Medbouhi, A.A. Towards Topology-Aware Variational Auto-Encoders: From InvMap-VAE to Witness Simplicial VAE. Master’s Thesis, KTH Royal Institute of Technology, Stockholm, Sweden, 2022. [Google Scholar]
- Hensel, F.; Moor, M.; Rieck, B. A Survey of Topological Machine Learning Methods. Front. Artif. Intell.
**2021**, 4, 52. [Google Scholar] [CrossRef] [PubMed] - Ferri, M. Why Topology for Machine Learning and Knowledge Extraction? Mach. Learn. Knowl. Extr.
**2019**, 1, 115–120. [Google Scholar] [CrossRef] - Edelsbrunner, H.; Harer, J. Computational Topology-an Introduction; American Mathematical Society: Providence, RI, USA, 2010; pp. I–XII, 1–241. [Google Scholar]
- Wikipedia, the Free Encyclopedia. Simplicial Complex Example. 2009. Available online: https://en.wikipedia.org/wiki/File:Simplicial_complex_example.svg (accessed on 18 March 2021).
- Wikipedia, the Free Encyclopedia. Simplicial Complex Nonexample. 2007. Available online: https://commons.wikimedia.org/wiki/File:Simplicial_complex_nonexample.png (accessed on 18 March 2021).
- Wilkins, D.R. Algebraic Topology, Course 421; Trinity College: Dublin, Ireland, 2008. [Google Scholar]
- de Silva, V.; Carlsson, G. Topological estimation using witness complexes. IEEE Symp. Point-Based Graph.
**2004**, 4, 157–166. [Google Scholar] - Rieck, B. Topological Data Analysis for Machine Learning, Lectures; European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases. 2020. Available online: https://bastian.rieck.me/outreach/ecml_pkdd_2020/ (accessed on 12 November 2020).
- Pearson, K. LIII. On lines and planes of closest fit to systems of points in space. Lond. Edinb. Dublin Philos. Mag. J. Sci.
**1901**, 2, 559–572. [Google Scholar] [CrossRef] - Hotelling, H. Relations Between Two Sets of Variates. Biometrika
**1936**, 28, 321–377. [Google Scholar] [CrossRef] - Lee, J.A.; Verleysen, M. Nonlinear Dimensionality Reduction, 1st ed.; Springer Publishing Company, Incorporated: New York, NY, USA, 2007. [Google Scholar]
- Tenenbaum, J.B.; de Silva, V.; Langford, J.C. A Global Geometric Framework for Nonlinear Dimensionality Reduction. Science
**2000**, 290, 2319. [Google Scholar] [CrossRef] [PubMed] - Kruskal, J. Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis. Psychometrika
**1964**, 29, 1–27. [Google Scholar] [CrossRef] - Kruskal, J. Nonmetric multidimensional scaling: A numerical method. Psychometrika
**1964**, 29, 115–129. [Google Scholar] [CrossRef] - Borg, I.; Groenen, P. Modern Multidimensional Scaling: Theory and Applications; Springer Series in Statistics; Springer: Berlin/Heidelberg, Germany, 2005. [Google Scholar] [CrossRef]
- Van der Maaten, L.; Hinton, G. Visualizing data using t-SNE. J. Mach. Learn. Res.
**2008**, 9, 2579–2605. [Google Scholar] - Hinton, G.E.; Roweis, S. Stochastic Neighbor Embedding. In Advances in Neural Information Processing Systems 15 (NIPS 2002); Becker, S., Thrun, S., Obermayer, K., Eds.; MIT Press: Cambridge, MA, USA, 2002; Volume 15. [Google Scholar]
- McInnes, L.; Healy, J.; Melville, J. UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction. arXiv
**2018**, arXiv:1802.03426. [Google Scholar] - Kingma, D.P.; Welling, M. An Introduction to Variational Autoencoders. Found. Trends Mach. Learn.
**2019**, 12, 307–392. [Google Scholar] [CrossRef] - Gabrielsson, R.B.; Nelson, B.J.; Dwaraknath, A.; Skraba, P.; Guibas, L.J.; Carlsson, G.E. A Topology Layer for Machine Learning. arXiv
**2019**, arXiv:1905.12200. [Google Scholar] - Polianskii, V. An Investigation of Neural Network Structure with Topological Data Analysis. Master’s Thesis, KTH Royal Institute of Technology, Stockholm, Sweden, 2018. [Google Scholar]
- Moor, M.; Horn, M.; Rieck, B.; Borgwardt, K.M. Topological Autoencoders. arXiv
**2019**, arXiv:1906.00722. [Google Scholar] - Hofer, C.D.; Kwitt, R.; Dixit, M.; Niethammer, M. Connectivity-Optimized Representation Learning via Persistent Homology. arXiv
**2019**, arXiv:1906.09003. [Google Scholar] - Gallego-Posada, J. Simplicial AutoEncoders: A connection between Algebraic Topology and Probabilistic Modelling. Master’s Thesis, University of Amsterdam, Amsterdam, The Netherlands, 2018. [Google Scholar]
- Gallego-Posada, J.; Forré, P. Simplicial Regularization. In Proceedings of the ICLR 2021 Workshop on Geometrical and Topological Representation Learning, Virtual, 3–7 May 2021. [Google Scholar]
- Zhang, H.; Cisse, M.; Dauphin, Y.N.; Lopez-Paz, D. mixup: Beyond Empirical Risk Minimization. In Proceedings of the International Conference on Learning Representations, Vancouver, BC, Canada, 30 April–3 May 2018. [Google Scholar]
- Verma, V.; Lamb, A.; Beckham, C.; Courville, A.C.; Mitliagkas, I.; Bengio, Y. Manifold Mixup: Encouraging Meaningful On-Manifold Interpolation as a Regularizer. arXiv
**2018**, arXiv:1806.05236. [Google Scholar] - Khrulkov, V.; Oseledets, I.V. Geometry Score: A Method For Comparing Generative Adversarial Networks. arXiv
**2018**, arXiv:1802.02664. [Google Scholar] - Perez Rey, L.A.; Menkovski, V.; Portegies, J. Diffusion Variational Autoencoders. In Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI-20, Yokohama, Japan, 11–17 July 2020; pp. 2704–2710. [Google Scholar] [CrossRef]
- Paszke, A.; Gross, S.; Massa, F.; Lerer, A.; Bradbury, J.; Chanan, G.; Killeen, T.; Lin, Z.; Gimelshein, N.; Antiga, L.; et al. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems 32; Wallach, H., Larochelle, H., Beygelzimer, A., d’Alché-Buc, F., Fox, E., Garnett, R., Eds.; Curran Associates, Inc.: Red Hook, NY, USA, 2019; pp. 8024–8035. [Google Scholar]
- Kingma, D.P.; Ba, J. Adam: A Method for Stochastic Optimization. arXiv
**2014**, arXiv:1412.6980. [Google Scholar] - Simon, S. Witness Complex. 2020. Available online: https://github.com/MrBellamonte/WitnessComplex (accessed on 20 December 2022).
- Maria, C.; Boissonnat, J.D.; Glisse, M.; Yvinec, M. The Gudhi Library: Simplicial Complexes and Persistent Homology; Technical Report; Springer: Berlin/Heidelberg, Germany, 2014. [Google Scholar]
- Maria, C. Filtered Complexes. In GUDHI User and Reference Manual, 3.4.1th ed.; GUDHI Editorial Board: San Francisco, CA, USA, 2021. [Google Scholar]
- Pedregosa, F.; Varoquaux, G.; Gramfort, A.; Michel, V.; Thirion, B.; Grisel, O.; Blondel, M.; Prettenhofer, P.; Weiss, R.; Dubourg, V.; et al. Scikit-learn: Machine Learning in Python. J. Mach. Learn. Res.
**2011**, 12, 2825–2830. [Google Scholar] - Marsland, S. Machine Learning-An Algorithmic Perspective; Chapman and Hall/CRC Machine Learning and Pattern Recognition Series; CRC Press: Boca Raton, FL, USA, 2009; pp. I–XVI, 1–390. [Google Scholar]
- Arvanitidis, G.; Hansen, L.K.; Hauberg, S. Latent Space Oddity: On the Curvature of Deep Generative Models. In Proceedings of the ICLR (Poster), Vancouver, BC, Canada, 30 April–3 May 2018. [Google Scholar]
- Schönenberger, S.T.; Varava, A.; Polianskii, V.; Chung, J.J.; Kragic, D.; Siegwart, R. Witness Autoencoder: Shaping the Latent Space with Witness Complexes. In Proceedings of the NeurIPS 2020 Workshop on Topological Data Analysis and Beyond, Virtual, 6–12 December 2020. [Google Scholar]
- Jang, U.; Jha, S.; Jha, S. On the Need for Topology-Aware Generative Models for Manifold-Based Defenses. In Proceedings of the International Conference on Learning Representations, Addis Ababa, Ethiopia, 26–30 April 2020. [Google Scholar]

**Figure 1.**Examples of simplices: a point (dimension equal to 0, or 0-simplex), a line segment (dimension equal to 1, a 1-simplex), a triangle (dimension equal to 2, a 2-simplex) and a tetrahedron (dimension equal to 3, a 3-simplex). Image from [7].

**Figure 2.**(

**a**) A valid (geometric) simplicial complex. (

**b**) A non-valid (geometric) simplicial complex. On the left we can see a valid geometric simplicial complex of dimension three (image from [8]), on the right we can see a non-valid geometric simplicial complex (image from [9]) because the second condition of the Definition 2 is not fulfilled.

**Figure 3.**Example of the construction of a witness complex ${\mathrm{W}}_{r}(S,L)$ for a dataset $S\subset {\mathbb{R}}^{2}$ of 11 points using a subset $L\subset S$ of 5 landmarks points and for a given radius $r\in \mathbb{R}$.

**Figure 4.**Different objects and their first Betti numbers (the cube, sphere, and torus are empty). Figures from [12].

**Figure 5.**(Vietoris–Rips) filtration of points sampled from a circle. Images from [12].

**Figure 7.**Open cylinder (left) and swissroll (right) training datasets. (

**a**) Dataset 1: Open cylinder ${\beta}_{0}=1,{\beta}_{1}=1$. (

**b**) Dataset 2: Swissroll ${\beta}_{0}=1,{\beta}_{1}=0$.

**Figure 8.**Standard VAE applied to the open cylinder dataset - pytorchseed = 1, trained for 500 epochs. (

**a**) is the latent representation of the open cylinder in a 2-dimensional space of a standard VAE. We can see two 1-dimensional holes so ${\beta}_{1}=2$ (Betti number 1 is equal to 2) instead of 1. (

**b**–

**e**) are different views of the reconstruction in the original 3-dimensional space.

**Figure 9.**Latent representation (left) and reconstruction (right) of the Standard VAE applied to the open cylinder-pytorchseed = 6, trained for 500 epochs. In this case, we can see that for the latent representation (

**a**) we have ${\beta}_{1}=0$ instead of 1. This means for example that from the latent representation we would not know that it is actually possible to go from the yellow part to the blue part without passing through the red part, because in this bad latent representation there is a discontinuity in the green region. In addition to that, the discontinuity in the green region of the latent space (

**a**) implies a discontinuity in the green region of the reconstructed cylinder (

**b**) so the reconstruction is also bad.

**Figure 10.**Latent representation (left) and reconstruction (right) of the Standard VAE applied to the open cylinder-pytorchseed = 2, trained for 500 epochs. We can see that for the latent representation (

**a**) we have ${\beta}_{1}=1$ like to the original open cylinder dataset. However we can see three distinct parts for the blue region which is problematic if we want to interpolate in this region in the latent space, and as we can see in the reconstruction (

**b**) it implies discontinuities in the blue region of the reconstructed cylinder.

**Figure 11.**Latent representation (top left) and reconstruction (down) of the Standard VAE applied to the open cylinder, trained for 500 epochs. We can see again that for the latent representation (

**a**) we have ${\beta}_{1}=1$ like to the original open cylinder dataset. However, the latent representation is bad because the “color order” is not preserved so this latent representation would not be useful for interpolations, it is like dividing the cylinder in top and down regions. Indeed, we can see in (

**c**) that this implies a discontinuity between top part and down part for example with the orange region. In addition to that, we have also a longitudinal discontinuity in the green region as shown in (

**d**).

**Figure 12.**Latent representation (top left) and reconstruction (down) of the Standard VAE applied to the swissroll - pytorchseed = 1, trained for 500 epochs. We can see that after 500 epochs, for the latent representation (

**a**) we have ${\beta}_{1}=2$ whereas it is equal to 0 for the original swissroll dataset. Moreover, we can visualize a discontinuity in the latent representation (

**a**) for all the colors except for the yellow region. This discontinuity is retrieved again for the reconstruction as seen in (

**c**,

**d**).

**Figure 13.**Latent representation (left column) and corresponding reconstruction (right column) of the Standard VAE applied to the swissroll - pytorchseed = 2, trained for 100; 500; and 10,000 epochs. After 100 epochs we can see an overlapping between the beginning and the end of the swissroll in the latent representation (

**a**) which has ${\beta}_{1}=1$ instead of 0, and the reconstruction is bad (

**b**). Then, the more the model is trained, the better is the reconstruction as we can see after 10,000 epochs for example (

**f**), but the latent representation (

**e**) is separated in many connected components whereas the original swissroll has ${\beta}_{0}=1$.

**Figure 14.**Latent representation (left) and corresponding reconstruction (right) of the Standard VAE applied to the swissroll for different initializations - pytorchseed = 3 (top) and pytorchseed = 4 (down), trained for 500 epochs. Again, discontinuities in the latent representation are transferred to the reconstructed swissroll. These representations are not useful for interpolating in the latent space. (

**a**) Latent space (after 500 epochs) for a random initialization with pytorchseed = 3. (

**b**) Reconstruction (after 500 epochs) for a random initialization with pytorchseed = 3. (

**c**) Latent space (after 500 epochs) for another random initialization with pytorchseed = 4. (

**d**) Reconstruction (after 500 epochs) for the other random initialization with pytorchseed = 4.

**Figure 15.**Isomap-based InvMap-VAE applied to the open cylinder dataset, $weigh{t}_{IM}=1$ and trained for 500 epochs. Betti numbers ${\beta}_{0}=1$ and ${\beta}_{1}=1$ are preserved between the original dataset (

**a**), the latent representation ((

**c**) for training and (

**e**) for test), and the reconstruction ((

**d**) for training and (

**f**) for test). Moreover, on the contrary to the Isomap embedding (

**b**), the latent representations are not too thin or “compressed”, which is better for interpolations.

**Figure 16.**Isomap-based InvMap-VAE applied to the swissroll dataset, $weigh{t}_{IM}=1$ and trained for 500 epochs. Betti numbers ${\beta}_{0}=1$ and ${\beta}_{1}=0$ are preserved between the original dataset (

**a**), the latent representation ((

**c**) for training and (

**e**) for test), and the reconstruction ((

**d**) for training and (

**f**) for test). Moreover, on the contrary to the Isomap embedding (

**b**), the latent representations do not have empty regions, which is better for interpolations, although lines appear and are retrieved in the reconstructions. We can notice that the spacing between these lines is actually related to the curvature of the manifold.

**Figure 17.**Isomap-based InvMap-VAE applied to the swissroll dataset, $weigh{t}_{IM}=10$ and trained for 1000 epochs. Betti numbers ${\beta}_{0}=1$ and ${\beta}_{1}=0$ are preserved between the original dataset (

**a**), the latent representation ((

**c**) for training and (

**e**) for test), and the reconstruction ((

**d**) for training and (

**f**) for test). Moreover, on the contrary to the Isomap embedding (

**b**), the latent representations are not too thin or “compressed”, which is better for interpolations.

**Figure 18.**Open cylinder witness complex (WC) construction for different filtration parameters. For the persistence diagrams (

**a**,

**b**,

**d**), on the x and y axis are the radius filtration, red points represent Betti 0 (connected components) and blue points represent Betti 1 (1-dimensional holes). In images (

**c**,

**d**,

**f**) are shown only the 1-dimensional simplices (grey edges) of the witness complexes.

**Figure 19.**Witness Simplicial VAE applied to the open cylinder after 100 epochs, each row is a different neural network initialization: latent representations of the training set on the left and corresponding reconstruction on the right. Hyper parameters: batch size = 128, 10 landmarks, ${r}_{filtration}=6$, weights ${w}_{SE}={w}_{SD}=10$. (

**a**) Latent representation. (

**b**) Reconstruction. (

**c**) Latent representation. (

**d**) Reconstruction. (

**e**) Latent representation. (

**f**) Reconstruction.

**Figure 20.**Witness Simplicial VAE applied to the open cylinder after 1000 epochs: training set (left) and test set (right), latent representation (1st row) and reconstruction (2nd and 3rd rows). Hyper parameters: batch size = 128, pytorchseed = 6, 10 landmarks, ${r}_{filtration}=6$, weights ${w}_{SE}={w}_{SD}=10$. (

**a**) Latent representation (training set). (

**b**) Latent representation (test set). (

**c**) Reconstruction view 1 (training set). (

**d**) Reconstruction view 1 (test set). (

**e**) Reconstruction view 2 (training set). (

**f**) Reconstruction view 2 (test set).

**Figure 21.**Witness Simplicial VAE applied to the swissroll after 100 epoch: loss (

**a**), latent representation of the training set (

**b**), and reconstruction of the training set (

**c**,

**d**) from different views. Hyper parameters: batch size = 128, pytorchseed = 1, 32 landmarks, ${r}_{filtration}=6$, weights ${w}_{SE}={w}_{SD}=10$.

**Figure 22.**Isolandmarks Witness Simplicial VAE applied to the swissroll after 500 epoch: loss (

**a**), latent representation of the training set (

**b**), and reconstruction of the training set (

**c**,

**d**) from different views. Hyper parameters: batch size = 128, pytorchseed = 1, 32 landmarks, ${r}_{filtration}=6.12$, weights ${w}_{SE}={w}_{SD}=10$ and ${w}_{iso}=0.001$.

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |

© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Medbouhi, A.A.; Polianskii, V.; Varava, A.; Kragic, D.
InvMap and Witness Simplicial Variational Auto-Encoders. *Mach. Learn. Knowl. Extr.* **2023**, *5*, 199-236.
https://doi.org/10.3390/make5010014

**AMA Style**

Medbouhi AA, Polianskii V, Varava A, Kragic D.
InvMap and Witness Simplicial Variational Auto-Encoders. *Machine Learning and Knowledge Extraction*. 2023; 5(1):199-236.
https://doi.org/10.3390/make5010014

**Chicago/Turabian Style**

Medbouhi, Aniss Aiman, Vladislav Polianskii, Anastasia Varava, and Danica Kragic.
2023. "InvMap and Witness Simplicial Variational Auto-Encoders" *Machine Learning and Knowledge Extraction* 5, no. 1: 199-236.
https://doi.org/10.3390/make5010014