Submit to Special Issue Submit Abstract to Special Issue Review for Entropy Propose a Special Issue

Journal Menu

Journal Browser

► Journal Browser

Causality and Complex Systems

Special Issue Editors
Special Issue Information
Keywords
Published Papers

A special issue of Entropy (ISSN 1099-4300). This special issue belongs to the section "Complexity".

Deadline for manuscript submissions: 24 September 2024 | Viewed by 23924

Share This Special Issue

Special Issue Editors

Prof. Dr. Jiang Zhang

E-Mail Website
Guest Editor

School of Systems Sciences, Beijing Normal University, Beijing 100875, China
Interests: complex systems; machine learning; causal emergence; scaling theory

Dr. Peng Cui

E-Mail Website
Guest Editor

Department of Computer Science, Tsinghua University, Beijing 100084, China
Interests: artificial intelligence; stable learning; complex systems; graph neural network; causality; information theory; recommendation system; machine learning; data mining; multimedia; network embedding.

Dr. Hector Zenil

E-Mail Website
Guest Editor

School of Biomedical Engineering and Imaging Sciences, King’s College London, London WC2R 2LS, UK
Interests: algorithmic information theory; complex systems; philosophy of algorithmic randomness; Kolmogorov complexity; computational biology; digital philosophy; programmability; natural computation; cellular automata; algorithmic probability; logical depth; measures of sophistication
Special Issues, Collections and Topics in MDPI journals

Special Issue Information

Dear Colleagues,

Complex Systems, e.g., living systems, environmental systems, health and medical systems, socioeconomic systems, and even online communities are all unified wholes that are formed by a large number of interacting units. One of the reasons for the system complexity is the wide existence of entangled causal structures. The causality may be emergent, which means stronger causal laws may exist at the macro-scale rather than the micro-scale (e.g., statistical mechanics). The causal force also may cross different levels. For example, downward causality, meaning the collective behaviors or the aggregated variables of the whole system (e.g., price) may have an effect on individual behaviors that widely exist in living systems or social systems. Therefore, understanding the emergence and evolution of these causal structures on various scales within a large complex system is very important.

However, how to discover intricate causal relationships and identify emerging causal laws on a macro level from the behavioral data of complex systems, and how to use these causal relationships and laws to infer new information are all tough problems. New emergent machine learning technologies (e.g., causal representation learning, causal reinforcement learning); information theory (e.g., information decomposition); causal discovery; causal inference, and so forth will offer us new solutions. This Special Issue focuses on, but is not limited to, the following topics:

Causal discovery;

Causal inference;

Causal emergence;

Downward causality;

Measures of complexity and causality;

Complex system modeling;

Causal machine learning;

Causal representation learning;

Causal reinforcement learning;

Information decomposition;

Related applications.

Prof. Dr. Jiang Zhang
Dr. Peng Cui
Dr. Hector Zenil
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Entropy is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

causal discovery
causal inference
causal emergence
downward causality
causal representation learning
causal reinforcement learning
renormalization
multi-scale analysis
information decomposition
information geometry

Published Papers (13 papers)

Download All Papers

Order results

Result details

Show export options Show export options

Select all

Export citation of selected articles as:

25 pages, 1320 KiB

Open AccessFeature PaperArticle

Kernel-Based Independence Tests for Causal Structure Learning on Functional Data

by Felix Laumann, Julius von Kügelgen, Junhyung Park, Bernhard Schölkopf and Mauricio Barahona

Entropy 2023, 25(12), 1597; https://doi.org/10.3390/e25121597 - 28 Nov 2023

Cited by 1 | Viewed by 996

Abstract

Measurements of systems taken along a continuous functional dimension, such as time or space, are ubiquitous in many fields, from the physical and biological sciences to economics and engineering. Such measurements can be viewed as realisations of an underlying smooth process sampled over the continuum. However, traditional methods for independence testing and causal learning are not directly applicable to such data, as they do not take into account the dependence along the functional dimension. By using specifically designed kernels, we introduce statistical tests for bivariate, joint, and conditional independence for functional variables. Our method not only extends the applicability to functional data of the Hilbert–Schmidt independence criterion (hsic) and its d-variate version (d-hsic), but also allows us to introduce a test for conditional independence by defining a novel statistic for the conditional permutation test (cpt) based on the Hilbert–Schmidt conditional independence criterion (hscic), with optimised regularisation strength estimated through an evaluation rejection rate. Our empirical results of the size and power of these tests on synthetic functional data show good performance, and we then exemplify their application to several constraint- and regression-based causal structure learning problems, including both synthetic examples and real socioeconomic data. Full article

(This article belongs to the Special Issue Causality and Complex Systems)

► Show Figures

Figure 1

19 pages, 1776 KiB

Open AccessArticle

Neural Causal Information Extractor for Unobserved Causes

by Keng-Hou Leong, Yuxuan Xiu, Bokui Chen and Wai Kin (Victor) Chan

Entropy 2024, 26(1), 46; https://doi.org/10.3390/e26010046 - 31 Dec 2023

Viewed by 1132

Abstract

Causal inference aims to faithfully depict the causal relationships between given variables. However, in many practical systems, variables are often partially observed, and some unobserved variables could carry significant information and induce causal effects on a target. Identifying these unobserved causes remains a challenge, and existing works have not considered extracting the unobserved causes while retaining the causes that have already been observed and included. In this work, we aim to construct the implicit variables with a generator–discriminator framework named the Neural Causal Information Extractor (NCIE), which can complement the information of unobserved causes and thus provide a complete set of causes with both observed causes and the representations of unobserved causes. By maximizing the mutual information between the targets and the union of observed causes and implicit variables, the implicit variables we generate could complement the information that the unobserved causes should have provided. The synthetic experiments show that the implicit variables preserve the information and dynamics of the unobserved causes. In addition, extensive real-world time series prediction tasks show improved precision after introducing implicit variables, thus indicating their causality to the targets. Full article

(This article belongs to the Special Issue Causality and Complex Systems)

► Show Figures

Figure 1

56 pages, 2990 KiB

Open AccessReview

Emergence and Causality in Complex Systems: A Survey of Causal Emergence and Related Quantitative Studies

by Bing Yuan, Jiang Zhang, Aobo Lyu, Jiayun Wu, Zhipeng Wang, Mingzhe Yang, Kaiwei Liu, Muyun Mou and Peng Cui

Entropy 2024, 26(2), 108; https://doi.org/10.3390/e26020108 - 24 Jan 2024

Viewed by 1999

Abstract

Emergence and causality are two fundamental concepts for understanding complex systems. They are interconnected. On one hand, emergence refers to the phenomenon where macroscopic properties cannot be solely attributed to the cause of individual properties. On the other hand, causality can exhibit emergence, meaning that new causal laws may arise as we increase the level of abstraction. Causal emergence (CE) theory aims to bridge these two concepts and even employs measures of causality to quantify emergence. This paper provides a comprehensive review of recent advancements in quantitative theories and applications of CE. It focuses on two primary challenges: quantifying CE and identifying it from data. The latter task requires the integration of machine learning and neural network techniques, establishing a significant link between causal emergence and machine learning. We highlight two problem categories: CE with machine learning and CE for machine learning, both of which emphasize the crucial role of effective information (EI) as a measure of causal emergence. The final section of this review explores potential applications and provides insights into future perspectives. Full article

(This article belongs to the Special Issue Causality and Complex Systems)

► Show Figures

Figure 1

17 pages, 930 KiB

Open AccessArticle

Detection of Anticipatory Dynamics between a Pair of Zebrafish

by Wei-Jie Chen, I-Shih Ko, Chi-An Lin, Chun-Jen Chen, Jiun-Shian Wu and C. K. Chan

Entropy 2024, 26(1), 13; https://doi.org/10.3390/e26010013 - 21 Dec 2023

Viewed by 819

Abstract

Anticipatory dynamics (AD) is unusual in that responses from an information receiver can appear ahead of triggers from the source, and direction of information flow (DIF) is needed to establish causality. Although it is believed that anticipatory dynamics is important for animals’ survival, natural examples are rare. Time series (trajectories) from a pair of interacting zebrafish are used to look for the existence of AD in natural systems. In order to obtain the DIF between the two trajectories, we have made use of a special experimental design to designate information source. However, we have also used common statistical tools such as Granger causality and transfer entropy to detect DIF. In our experiments, we found that a majority of the fish pairs do not show any anticipatory behaviors and only a few pairs displayed possible AD. Interestingly, for fish in this latter group, they do not display AD all the time. Our findings suggest that the formation of schooling of fish might not need the help of AD, and new tools are needed in the detection of causality in AD system. Full article

(This article belongs to the Special Issue Causality and Complex Systems)

► Show Figures

Figure 1

18 pages, 952 KiB

Open AccessArticle

Causal Factor Disentanglement for Few-Shot Domain Adaptation in Video Prediction

by Nathan Cornille, Katrien Laenen, Jingyuan Sun and Marie-Francine Moens

Entropy 2023, 25(11), 1554; https://doi.org/10.3390/e25111554 - 17 Nov 2023

Viewed by 953

Abstract

An important challenge in machine learning is performing with accuracy when few training samples are available from the target distribution. If a large number of training samples from a related distribution are available, transfer learning can be used to improve the performance. This paper investigates how to do transfer learning more effectively if the source and target distributions are related through a Sparse Mechanism Shift for the application of next-frame prediction. We create Sparse Mechanism Shift-TempoRal Intervened Sequences (SMS-TRIS), a benchmark to evaluate transfer learning for next-frame prediction derived from the TRIS datasets. We then propose to exploit the Sparse Mechanism Shift property of the distribution shift by disentangling the model parameters with regard to the true causal mechanisms underlying the data. We use the Causal Identifiability from TempoRal Intervened Sequences (CITRIS) model to achieve this disentanglement via causal representation learning. We show that encouraging disentanglement with the CITRIS extensions can improve performance, but their effectiveness varies depending on the dataset and backbone used. We find that it is effective only when encouraging disentanglement actually succeeds in increasing disentanglement. We also show that an alternative method designed for domain adaptation does not help, indicating the challenging nature of the SMS-TRIS benchmark. Full article

(This article belongs to the Special Issue Causality and Complex Systems)

► Show Figures

Figure 1

28 pages, 1013 KiB

Open AccessPerspective

Efficient, Formal, Material, and Final Causes in Biology and Technology

by George F. R. Ellis

Entropy 2023, 25(9), 1301; https://doi.org/10.3390/e25091301 - 05 Sep 2023

Cited by 3 | Viewed by 2448

Abstract

This paper considers how a classification of causal effects as comprising efficient, formal, material, and final causation can provide a useful understanding of how emergence takes place in biology and technology, with formal, material, and final causation all including cases of downward causation; they each occur in both synchronic and diachronic forms. Taken together, they underlie why all emergent levels in the hierarchy of emergence have causal powers (which is Noble’s principle of biological relativity) and so why causal closure only occurs when the upwards and downwards interactions between all emergent levels are taken into account, contra to claims that some underlying physics level is by itself causality complete. A key feature is that stochasticity at the molecular level plays an important role in enabling agency to emerge, underlying the possibility of final causation occurring in these contexts. Full article

(This article belongs to the Special Issue Causality and Complex Systems)

► Show Figures

Figure 1

13 pages, 775 KiB

Open AccessReview

Comparison of Bootstrap Methods for Estimating Causality in Linear Dynamic Systems: A Review

by Fumikazu Miwakeichi and Andreas Galka

Entropy 2023, 25(7), 1070; https://doi.org/10.3390/e25071070 - 17 Jul 2023

Viewed by 1143

Abstract

In this study, we present a thorough comparison of the performance of four different bootstrap methods for assessing the significance of causal analysis in time series data. For this purpose, multivariate simulated data are generated by a linear feedback system. The methods investigated are uncorrelated Phase Randomization Bootstrap (uPRB), which generates surrogate data with no cross-correlation between variables by randomizing the phase in the frequency domain; Time Shift Bootstrap (TSB), which generates surrogate data by randomizing the phase in the time domain; Stationary Bootstrap (SB), which calculates standard errors and constructs confidence regions for weakly dependent stationary observations; and AR-Sieve Bootstrap (ARSB), a resampling method based on AutoRegressive (AR) models that approximates the underlying data-generating process. The uPRB method accurately identifies variable interactions but fails to detect self-feedback in some variables. The TSB method, despite performing worse than uPRB, is unable to detect feedback between certain variables. The SB method gives consistent causality results, although its ability to detect self-feedback decreases, as the mean block width increases. The ARSB method shows superior performance, accurately detecting both self-feedback and causality across all variables. Regarding the analysis of the Impulse Response Function (IRF), only the ARSB method succeeds in detecting both self-feedback and causality in all variables, aligning well with the connectivity diagram. Other methods, however, show considerable variations in detection performance, with some detecting false positives and others only detecting self-feedback. Full article

(This article belongs to the Special Issue Causality and Complex Systems)

► Show Figures

Figure 1

28 pages, 4671 KiB

Open AccessArticle

Neural Information Squeezer for Causal Emergence

by Jiang Zhang and Kaiwei Liu

Entropy 2023, 25(1), 26; https://doi.org/10.3390/e25010026 - 23 Dec 2022

Cited by 8 | Viewed by 2365 | Correction

Abstract

Conventional studies of causal emergence have revealed that stronger causality can be obtained on the macro-level than the micro-level of the same Markovian dynamical systems if an appropriate coarse-graining strategy has been conducted on the micro-states. However, identifying this emergent causality from data is still a difficult problem that has not been solved because the appropriate coarse-graining strategy can not be found easily. This paper proposes a general machine learning framework called Neural Information Squeezer to automatically extract the effective coarse-graining strategy and the macro-level dynamics, as well as identify causal emergence directly from time series data. By using invertible neural network, we can decompose any coarse-graining strategy into two separate procedures: information conversion and information discarding. In this way, we can not only exactly control the width of the information channel, but also can derive some important properties analytically. We also show how our framework can extract the coarse-graining functions and the dynamics on different levels, as well as identify causal emergence from the data on several exampled systems. Full article

(This article belongs to the Special Issue Causality and Complex Systems)

► Show Figures

Figure 1

15 pages, 775 KiB

Open AccessArticle

Flickering Emergences: The Question of Locality in Information-Theoretic Approaches to Emergence

by Thomas F. Varley

Entropy 2023, 25(1), 54; https://doi.org/10.3390/e25010054 - 28 Dec 2022

Cited by 8 | Viewed by 2741

Abstract

“Emergence”, the phenomenon where a complex system displays properties, behaviours, or dynamics not trivially reducible to its constituent elements, is one of the defining properties of complex systems. Recently, there has been a concerted effort to formally define emergence using the mathematical framework of information theory, which proposes that emergence can be understood in terms of how the states of wholes and parts collectively disclose information about the system’s collective future. In this paper, we show how a common, foundational component of information-theoretic approaches to emergence implies an inherent instability to emergent properties, which we call flickering emergence. A system may, on average, display a meaningful emergent property (be it an informative coarse-graining, or higher-order synergy), but for particular configurations, that emergent property falls apart and becomes misinformative. We show existence proofs that flickering emergence occurs in two different frameworks (one based on coarse-graining and another based on multivariate information decomposition) and argue that any approach based on temporal mutual information will display it. Finally, we argue that flickering emergence should not be a disqualifying property of any model of emergence, but that it should be accounted for when attempting to theorize about how emergence relates to practical models of the natural world. Full article

(This article belongs to the Special Issue Causality and Complex Systems)

► Show Figures

Figure 1

59 pages, 15006 KiB

Open AccessArticle

Causality Analysis with Information Geometry: A Comparison

by Heng Jie Choong, Eun-jin Kim and Fei He

Entropy 2023, 25(5), 806; https://doi.org/10.3390/e25050806 - 16 May 2023

Cited by 2 | Viewed by 1596

Abstract

The quantification of causality is vital for understanding various important phenomena in nature and laboratories, such as brain networks, environmental dynamics, and pathologies. The two most widely used methods for measuring causality are Granger Causality (GC) and Transfer Entropy (TE), which rely on measuring the improvement in the prediction of one process based on the knowledge of another process at an earlier time. However, they have their own limitations, e.g., in applications to nonlinear, non-stationary data, or non-parametric models. In this study, we propose an alternative approach to quantify causality through information geometry that overcomes such limitations. Specifically, based on the information rate that measures the rate of change of the time-dependent distribution, we develop a model-free approach called information rate causality that captures the occurrence of the causality based on the change in the distribution of one process caused by another. This measurement is suitable for analyzing numerically generated non-stationary, nonlinear data. The latter are generated by simulating different types of discrete autoregressive models which contain linear and nonlinear interactions in unidirectional and bidirectional time-series signals. Our results show that information rate causalitycan capture the coupling of both linear and nonlinear data better than GC and TE in the several examples explored in the paper. Full article

(This article belongs to the Special Issue Causality and Complex Systems)

► Show Figures

Figure 1

15 pages, 2121 KiB

Open AccessArticle

Inferring a Causal Relationship between Environmental Factors and Respiratory Infections Using Convergent Cross-Mapping

by Daipeng Chen, Xiaodan Sun and Robert A. Cheke

Entropy 2023, 25(5), 807; https://doi.org/10.3390/e25050807 - 17 May 2023

Cited by 1 | Viewed by 1823

Abstract

The incidence of respiratory infections in the population is related to many factors, among which environmental factors such as air quality, temperature, and humidity have attracted much attention. In particular, air pollution has caused widespread discomfort and concern in developing countries. Although the correlation between respiratory infections and air pollution is well known, establishing causality between them remains elusive. In this study, by conducting theoretical analysis, we updated the procedure of performing the extended convergent cross-mapping (CCM, a method of causal inference) to infer the causality between periodic variables. Consistently, we validated this new procedure on the synthetic data generated by a mathematical model. For real data in Shaanxi province of China in the period of 1 January 2010 to 15 November 2016, we first confirmed that the refined method is applicable by investigating the periodicity of influenza-like illness cases, an air quality index, temperature, and humidity through wavelet analysis. We next illustrated that air quality (quantified by AQI), temperature, and humidity affect the daily influenza-like illness cases, and, in particular, the respiratory infection cases increased progressively with increased AQI with a time delay of 11 days. Full article

(This article belongs to the Special Issue Causality and Complex Systems)

► Show Figures

Figure 1

16 pages, 4618 KiB

Open AccessArticle

Schizophrenia MEG Network Analysis Based on Kernel Granger Causality

by Qiong Wang, Wenpo Yao, Dengxuan Bai, Wanyi Yi, Wei Yan and Jun Wang

Entropy 2023, 25(7), 1006; https://doi.org/10.3390/e25071006 - 30 Jun 2023

Cited by 1 | Viewed by 986

Abstract

Network analysis is an important approach to explore complex brain structures under different pathological and physiological conditions. In this paper, we employ the multivariate inhomogeneous polynomial kernel Granger causality (MKGC) to construct directed weighted networks to characterize schizophrenia magnetoencephalography (MEG). We first generate data based on coupled autoregressive processes to test the effectiveness of MKGC in comparison with the bivariate linear Granger causality and bivariate inhomogeneous polynomial kernel Granger causality. The test results suggest that MKGC outperforms the other two methods. Based on these results, we apply MKGC to construct effective connectivity networks of MEG for patients with schizophrenia (SCZs). We measure three network features, i.e., strength, nonequilibrium, and complexity, to characterize schizophrenia MEG. Our results suggest that MEG of the healthy controls (HCs) has a denser effective connectivity network than that of SCZs. The most significant difference in the in-connectivity strength is observed in the right frontal network (

p = 0.001

). The strongest out-connectivity strength for all subjects occurs in the temporal area, with the most significant between-group difference in the left occipital area (

p = 0.0018

). The total connectivity strength of the frontal, temporal, and occipital areas of HCs exhibits higher values compared with SCZs. The nonequilibrium feature over the whole brain of SCZs is significantly higher than that of the HCs (

p = 0.012

); however, the results of Shannon entropy suggest that healthy MEG networks have higher complexity than schizophrenia networks. Overall, MKGC provides a reliable approach to construct MEG brain networks and characterize the network characteristics. Full article

(This article belongs to the Special Issue Causality and Complex Systems)

► Show Figures