Synergy and Redundancy Measures: Theory and Applications to Characterize Complex Systems and Shape Neural Network Representations
A special issue of Entropy (ISSN 1099-4300). This special issue belongs to the section "Information Theory, Probability and Statistics".
Deadline for manuscript submissions: closed (20 April 2024) | Viewed by 4341
Special Issue Editor
Interests: data analysis; causal inference; dimensionality reduction; neuroscience; sensitivity analysis; structure learning; information decomposition; information bottleneck
Special Issue Information
Dear Colleagues,
An important aspect of how sources of information are distributed across a set of variables concerns whether different variables provide redundant, unique, or synergistic information when combined with other variables. Intuitively, variables share redundant information if each variable carries individually the same information carried by other variables. Information carried by a certain variable is unique if it is not carried by any other variables or their combination, and a group of variables carries synergistic information if some information arises only when they are combined.
Recent advances have contributed toward building an information-theoretic framework to determine the distribution and nature of information extractable from multivariate data sets. Measures of redundant, unique, or synergistic information characterize dependencies between the parts of a multivariate system and can help to understand its function and mechanisms. Furthermore, these measures are also useful to analyze how information is distributed across layers in neural networks or can be used as cost functions to shape the structure of data representations learned by the networks.
This Special Issue welcomes contributions on advances in both the theoretical formulation and applications of information-theoretic measures of synergy and redundancy. Encompassed topics include:
- Advances in a multivariate formulation of redundancy measures or in the comparison of alternative proposals, addressing their distinctive power to capture relevant structures in both synthetic and experimental data sets;
- Applications to understand interactions in real complex systems;
- Advances in the estimation of information-theoretic quantities from high-dimensional data sets;
- Applications for feature selection and sensitivity analysis;
- Analysis of the distribution and nature of information across layers in neural networks;
- Design of deep learning models to obtain robust or disentangled data representations.
Dr. Daniel Chicharro
Guest Editor
Manuscript Submission Information
Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.
Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Entropy is an international peer-reviewed open access monthly journal published by MDPI.
Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.
Keywords
- mutual information
- synergy
- redundancy
- unique information
- neural networks
- disentanglement
- feature extraction
- representation learning
- partial information decomposition
