# Symmetries among Multivariate Information Measures Explored Using Möbius Operators

^{*}

## Abstract

**:**

_{3}. Relations among the set of functions on the lattice are transparently expressed in terms of the operator algebra, and, when applied to the information measures, can be used to derive a wide range of relationships among diverse information measures. The Möbius operator algebra is then naturally generalized which yields an even wider range of new relationships.

## 1. Introduction

## 2. Preliminaries

#### 2.1. Information Theory

#### 2.2. Lattice Theory

## 3. Möbius Dualities

#### 3.1. Möbius Inversion

#### 3.2. Möbius Operators

**Definition**

**1.**

**Definition**

**2.**

## 4. Connections to the Deltas

**Theorem**

**1.**

**Corollary**

**1.**

## 5. Symmetries Reveal a Wide Range of New Relations

**Theorem**

**2.**

## 6. Relation to Probability Densities

#### 6.1. Conditional log Likelihoods and Deltas

**Theorem**

**3.**

#### 6.2. Towards Prediction

## 7. Generalizing the Möbius Operators

**Definition**

**3.**

**Definition**

**4.**

**Theorem**

**4.**

## 8. Discussion

## Author Contributions

## Funding

## Conflicts of Interest

## References

- Singer, A. Maximum entropy formulation of the Kirkwood superposition approximation. J. Chem. Phys.
**2004**, 121, 3657–3666. [Google Scholar] [CrossRef] [PubMed] - Cochran, R.V.; Lund, L.H. On the Kirkwood Superposition Approximation. J. Chem. Phys.
**1964**, 41, 3499–3504. [Google Scholar] [CrossRef] - McGill, W.J. Multivariate information transmission. Psychometrika
**1954**, 19, 97–116. [Google Scholar] [CrossRef] - Bell, A.J. The co-information lattice. In Proceedings of the 4th International Symposium on Independent Component Analysis and Blind Signal Separation (ICA2003), Nara, Japan, 1–4 April 2003. [Google Scholar]
- Jakulin, A.; Bratko, I. Quantifying and Visualizing Attribute Interactions: An Approach Based on Entropy. arXiv, 2004; arXiv:cs.AI/0308002. [Google Scholar]
- Tin, H.K. On the Amount of Information. Theory Prob. Appl.
**1962**, 7, 439–447. [Google Scholar] [CrossRef] - Han, T.S. Multiple Mutual Information and multiple interactions in frequency data. Inf. Control.
**1980**, 46, 26–45. [Google Scholar] [CrossRef] - Galas, D.J.; Sakhanenko, N.A.; Ignac, T.; Skupin, A. Describing the complexity of systems: Multi-variable “set complexity” and the information basis of systems biology. J. Comput. Biol.
**2014**, 21, 118–140. [Google Scholar] [CrossRef] [PubMed] - Sakhanenko, N.A.; Galas, D.J. Biological data analysis as an information theory problem: Multivariable dependence measures and the Shadows algorithm. J. Comput. Biol.
**2015**, 22, 1005–1024. [Google Scholar] [CrossRef] [PubMed] - Watanabe, S. Information theoretic analysis of multivariate correlation. IBM J. Res. Dev.
**1960**, 4, 66–82. [Google Scholar] [CrossRef] - Davey, B.A.; Priestley, H.A. Introduction to Lattices and Order, 3rd ed.; Cambridge University Press: Cambridge, UK, 2002; ISBN 978-0-521-78451-1. [Google Scholar]
- Chow, C.K.; Liu, C.N. Approximating discrete probability distributions with dependence trees. IEEE Trans. Inf. Theory
**1968**, 14, 462–467. [Google Scholar] [CrossRef] [Green Version] - Galas, D.J.; Nykter, M.; Carter, G.W.; Price, N.D.; Shmulevich, I. Biological information as set based complexity. IEEE Trans. Inf. Theory
**2010**, 56, 667–677. [Google Scholar] [CrossRef] [PubMed] - Carter, G.W.; Galitski, T.; Galas, D.J. Maximal Extraction of Biological Information from Genetic Interaction Data. PLoS Comput. Biol.
**2009**, 5, e1000347. [Google Scholar] [CrossRef] [PubMed] - Priestley, H.A. Representation of distributive lattices by means of ordered Stone spaces. Bull. Lond. Math. Soc.
**1970**, 2, 186–190. [Google Scholar] [CrossRef] - Bettencourt, L.M.A.; Gintautas, V.; Ham, M. Identification of Functional Information Subgraphs in Complex Networks. Phys. Rev. Lett.
**2008**, 100, 238701–238704. [Google Scholar] [CrossRef] [PubMed] - Knuth, K.E. Valuations on Lattices and their Application to Information Theory. In Proceedings of the 2006 IEEE World Congress on Computational Intelligence, Vancouver, BC, Canada, 16–21 July 2006. [Google Scholar]
- Knuth, K.H. Lattices and their consistent quantification. Annalen der Physik
**2017**. [Google Scholar] [CrossRef] - Baudot, P.; Bennequin, D. The Homological Nature of Entropy. Entropy
**2015**, 17, 3253–3318. [Google Scholar] [CrossRef] [Green Version]

**Figure 1.**The Hasse diagram of the subset lattice for three variables. The numbers in black are the variable subsets, while the Möbius function $\mu \left(\nu ,\tau \right)$ on this lattice (1 or −1) is indicated in red.

**Figure 2.**The Möbius operators define the duality relationships between the functions on the subset lattice.

**Figure 3.**Diagram of the mappings of the functions on the subset lattice into one another by the operators. The operators $\widehat{P}$ and $\widehat{R}$ are: $\widehat{P}=\widehat{X}\widehat{M},\widehat{R}=\widehat{X}\widehat{m}$.

**Figure 4.**The four-variable lattice showing the 4 join-irreducible elements that generate the symmetric deltas as in Equation (16c). Möbius function values are shown on the right, and the red lines connect the elements of the delta function, $\mathsf{\Delta}\left(234;1\right)$, which form a 3D-cube.

**Figure 5.**A simple modifcation of one of the functions by lattice complementation modifies the postion of functions in the mapping diagram. The original diagram is on the left, the result of $\mathsf{\Delta}$ modified by complementation is on the right.

**Figure 6.**Generalized Möbius operator relations. A diagram of the relations among the functions as determined by the operators. The upper two arrows represent the generalized Möbius inversion relations. The function ${g}_{\eta}\left(\tau \right)$ is the designation of the function created by the operator ${F}_{\eta}$. The ${S}_{3}$ structure is reflected in the similarity with the diagram of Figure 3. Note that when $\eta =\varnothing $ the figure becomes identical to Figure 3.

**Figure 7.**Decomposing the 3D-cube Hasse diagram into two squares (2D hypercubes), by passing a plane through the center of the cube three different ways.

**Table 1.**The product table for the six operators above. The operators $\widehat{P}$ and $\widehat{R}$ are defined as $\widehat{P}=\widehat{X}\widehat{M},\widehat{R}=\widehat{X}\widehat{m}$. The convention is that the top row is on the right and the left column on the left in the products indicated; e.g., $\widehat{M}\widehat{X}=\widehat{R},\widehat{X}\widehat{M}=\widehat{P}$. The orange indicates the identity operator.

Right | |||||||
---|---|---|---|---|---|---|---|

$\widehat{\mathit{I}}$ | $\widehat{\mathit{m}}$ | $\widehat{\mathit{X}}$ | $\widehat{\mathit{M}}$ | $\widehat{\mathit{P}}$ | $\widehat{\mathit{R}}$ | ||

Left | $\widehat{\mathit{I}}$ | $\widehat{I}$ | $\widehat{m}$ | $\widehat{X}$ | $\widehat{M}$ | $\widehat{P}$ | $\widehat{R}$ |

$\widehat{\mathit{m}}$ | $\widehat{m}$ | $\widehat{I}$ | $\widehat{P}$ | $\widehat{R}$ | $\widehat{X}$ | $\widehat{M}$ | |

$\widehat{\mathit{X}}$ | $\widehat{X}$ | $\widehat{R}$ | $\widehat{I}$ | $\widehat{P}$ | $\widehat{M}$ | $\widehat{m}$ | |

$\widehat{\mathit{M}}$ | $\widehat{M}$ | $\widehat{P}$ | $\widehat{R}$ | $\widehat{I}$ | $\widehat{m}$ | $\widehat{X}$ | |

$\widehat{\mathit{P}}$ | $\widehat{P}$ | $\widehat{M}$ | $\widehat{m}$ | $\widehat{X}$ | $\widehat{R}$ | $\widehat{I}$ | |

$\widehat{\mathit{R}}$ | $\widehat{R}$ | $\widehat{X}$ | $\widehat{M}$ | $\widehat{m}$ | $\widehat{I}$ | $\widehat{P}$ |

**Table 2.**The 3 × 3 matrix representation of symmetric group ${S}_{3}$ and the corresponding Möbius operators. The one-line notation on the left shows the permutations.

One-line Notation:(Permutation) | Matrix Representation (Left Action Convention) | Möbius Operator |
---|---|---|

123 | $\left(\begin{array}{ccc}1& 0& 0\\ 0& 1& 0\\ 0& 0& 1\end{array}\right)$ | $\widehat{I}$ |

213 | $\left(\begin{array}{ccc}0& 1& 0\\ 1& 0& 0\\ 0& 0& 1\end{array}\right)$ | $\widehat{m}$ |

132 | $\left(\begin{array}{ccc}1& 0& 0\\ 0& 0& 1\\ 0& 1& 0\end{array}\right)$ | $\widehat{M}$ |

321 | $\left(\begin{array}{ccc}0& 0& 1\\ 0& 1& 0\\ 1& 0& 0\end{array}\right)$ | $\widehat{X}$ |

231 | $\left(\begin{array}{ccc}0& 1& 0\\ 0& 0& 1\\ 1& 0& 0\end{array}\right)$ | $\widehat{P}$ |

312 | $\left(\begin{array}{ccc}0& 0& 1\\ 1& 0& 0\\ 0& 1& 0\end{array}\right)$ | $\widehat{R}$ |

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Galas, D.J.; Sakhanenko, N.A.
Symmetries among Multivariate Information Measures Explored Using Möbius Operators. *Entropy* **2019**, *21*, 88.
https://doi.org/10.3390/e21010088

**AMA Style**

Galas DJ, Sakhanenko NA.
Symmetries among Multivariate Information Measures Explored Using Möbius Operators. *Entropy*. 2019; 21(1):88.
https://doi.org/10.3390/e21010088

**Chicago/Turabian Style**

Galas, David J., and Nikita A. Sakhanenko.
2019. "Symmetries among Multivariate Information Measures Explored Using Möbius Operators" *Entropy* 21, no. 1: 88.
https://doi.org/10.3390/e21010088