Research

Jump to: Other

12 pages, 285 KiB

Open AccessArticle

Variable Selection of Spatial Logistic Autoregressive Model with Linear Constraints

by Yunquan Song, Yuqi Su and Zhijian Wang

Entropy 2022, 24(11), 1660; https://doi.org/10.3390/e24111660 - 15 Nov 2022

Viewed by 1540

In recent years, spatial data widely exist in various fields such as finance, geology, environment, and natural science. These data collected by many scholars often have geographical characteristics. The spatial autoregressive model is a general method to describe the spatial correlations among observation [...] Read more.

In recent years, spatial data widely exist in various fields such as finance, geology, environment, and natural science. These data collected by many scholars often have geographical characteristics. The spatial autoregressive model is a general method to describe the spatial correlations among observation units in spatial econometrics. The spatial logistic autoregressive model augments the conventional logistic regression model with an extra network structure when the spatial response variables are discrete, which enhances classification precision. In many application fields, prior knowledge can be formulated as constraints on the parameters to improve the effectiveness of variable selection and estimation. This paper proposes a variable selection method with linear constraints for the high-dimensional spatial logistic autoregressive model in order to integrate the prior information into the model selection. Monte Carlo experiments are provided to analyze the performance of our proposed method under finite samples. The results show that the method can effectively screen out insignificant variables and give the corresponding coefficient estimates of significant variables simultaneously. As an empirical illustration, we apply our method to land area data. Full article

(This article belongs to the Special Issue Statistical Methods for Complex Systems)

17 pages, 1399 KiB

Open AccessArticle

Evolution of Cohesion between USA Financial Sector Companies before, during, and Post-Economic Crisis: Complex Networks Approach

by Vojin Stević, Marija Rašajski and Marija Mitrović Dankulov

Entropy 2022, 24(7), 1005; https://doi.org/10.3390/e24071005 - 20 Jul 2022

Viewed by 1629

Abstract

Various mathematical frameworks play an essential role in understanding the economic systems and the emergence of crises in them. Understanding the relation between the structure of connections between the system’s constituents and the emergence of a crisis is of great importance. In this [...] Read more.

Various mathematical frameworks play an essential role in understanding the economic systems and the emergence of crises in them. Understanding the relation between the structure of connections between the system’s constituents and the emergence of a crisis is of great importance. In this paper, we propose a novel method for the inference of economic systems’ structures based on complex networks theory utilizing the time series of prices. Our network is obtained from the correlation matrix between the time series of companies’ prices by imposing a threshold on the values of the correlation coefficients. The optimal value of the threshold is determined by comparing the spectral properties of the threshold network and the correlation matrix. We analyze the community structure of the obtained networks and the relation between communities’ inter and intra-connectivity as indicators of systemic risk. Our results show how an economic system’s behavior is related to its structure and how the crisis is reflected in changes in the structure. We show how regulation and deregulation affect the structure of the system. We demonstrate that our method can identify high systemic risks and measure the impact of the actions taken to increase the system’s stability. Full article

(This article belongs to the Special Issue Statistical Methods for Complex Systems)

► Show Figures

Figure 1

25 pages, 1359 KiB

Open AccessArticle

Estimating Sentence-like Structure in Synthetic Languages Using Information Topology

by Andrew D. Back and Janet Wiles

Entropy 2022, 24(7), 859; https://doi.org/10.3390/e24070859 - 22 Jun 2022

Cited by 1 | Viewed by 1393

Abstract

Estimating sentence-like units and sentence boundaries in human language is an important task in the context of natural language understanding. While this topic has been considered using a range of techniques, including rule-based approaches and supervised and unsupervised algorithms, a common aspect of [...] Read more.

Estimating sentence-like units and sentence boundaries in human language is an important task in the context of natural language understanding. While this topic has been considered using a range of techniques, including rule-based approaches and supervised and unsupervised algorithms, a common aspect of these methods is that they inherently rely on a priori knowledge of human language in one form or another. Recently we have been exploring synthetic languages based on the concept of modeling behaviors using emergent languages. These synthetic languages are characterized by a small alphabet and limited vocabulary and grammatical structure. A particular challenge for synthetic languages is that there is generally no a priori language model available, which limits the use of many natural language processing methods. In this paper, we are interested in exploring how it may be possible to discover natural ‘chunks’ in synthetic language sequences in terms of sentence-like units. The problem is how to do this with no linguistic or semantic language model. Our approach is to consider the problem from the perspective of information theory. We extend the basis of information geometry and propose a new concept, which we term information topology, to model the incremental flow of information in natural sequences. We introduce an information topology view of the incremental information and incremental tangent angle of the Wasserstein-1 distance of the probabilistic symbolic language input. It is not suggested as a fully viable alternative for sentence boundary detection per se but provides a new conceptual method for estimating the structure and natural limits of information flow in language sequences but without any semantic knowledge. We consider relevant existing performance metrics such as the F-measure and indicate limitations, leading to the introduction of a new information-theoretic global performance based on modeled distributions. Although the methodology is not proposed for human language sentence detection, we provide some examples using human language corpora where potentially useful results are shown. The proposed model shows potential advantages for overcoming difficulties due to the disambiguation of complex language and potential improvements for human language methods. Full article

(This article belongs to the Special Issue Statistical Methods for Complex Systems)

► Show Figures

Figure 1

17 pages, 1502 KiB

Open AccessArticle

Feature Importance in Gradient Boosting Trees with Cross-Validation Feature Selection

by Afek Ilay Adler and Amichai Painsky

Entropy 2022, 24(5), 687; https://doi.org/10.3390/e24050687 - 13 May 2022

Cited by 18 | Viewed by 4935

Abstract

Gradient Boosting Machines (GBM) are among the go-to algorithms on tabular data, which produce state-of-the-art results in many prediction tasks. Despite its popularity, the GBM framework suffers from a fundamental flaw in its base learners. Specifically, most implementations utilize decision trees that are [...] Read more.

Gradient Boosting Machines (GBM) are among the go-to algorithms on tabular data, which produce state-of-the-art results in many prediction tasks. Despite its popularity, the GBM framework suffers from a fundamental flaw in its base learners. Specifically, most implementations utilize decision trees that are typically biased towards categorical variables with large cardinalities. The effect of this bias was extensively studied over the years, mostly in terms of predictive performance. In this work, we extend the scope and study the effect of biased base learners on GBM feature importance (FI) measures. We demonstrate that although these implementation demonstrate highly competitive predictive performance, they still, surprisingly, suffer from bias in FI. By utilizing cross-validated (CV) unbiased base learners, we fix this flaw at a relatively low computational cost. We demonstrate the suggested framework in a variety of synthetic and real-world setups, showing a significant improvement in all GBM FI measures while maintaining relatively the same level of prediction accuracy. Full article

(This article belongs to the Special Issue Statistical Methods for Complex Systems)

► Show Figures

Figure 1

31 pages, 536 KiB

Open AccessArticle

A TOPSIS-Inspired Ranking Method Using Constrained Crowd Opinions for Urban Planning

by Sujoy Chatterjee and Sunghoon Lim

Entropy 2022, 24(3), 371; https://doi.org/10.3390/e24030371 - 05 Mar 2022

Cited by 2 | Viewed by 2259

Abstract

Crowdsourcing has become an important tool for gathering knowledge for urban planning problems. The questions posted to the crowd for urban planning problems are quite different from the traditional crowdsourcing models. Unlike the traditional crowdsourcing models, due to the constraints among the multiple [...] Read more.

Crowdsourcing has become an important tool for gathering knowledge for urban planning problems. The questions posted to the crowd for urban planning problems are quite different from the traditional crowdsourcing models. Unlike the traditional crowdsourcing models, due to the constraints among the multiple components (e.g., multiple locations of facilities) in a single question and non-availability of the defined option sets, aggregating of multiple diverse opinions that satisfy the constraints as well as finding the ranking of the crowd workers becomes challenging. Moreover, owing to the presence of the conflicting nature of features, the traditional ranking methods such as the Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) cannot always be feasible as the optimal solutions in terms of multiple objectives cannot occur simultaneously for the conflicting cases (e.g., benefit and cost criteria) for urban planning problems. Therefore, in this work, a multi-objective approach is proposed to produce better compromised solutions in terms of conflicting features from the general crowd. In addition, the solutions are employed to obtain a proper ideal solution for ranking the crowd. The experimental results are validated using two constrained crowd opinion datasets for real-world urban planning problems and compared with the state-of-the-art TOPSIS models. Full article

(This article belongs to the Special Issue Statistical Methods for Complex Systems)

► Show Figures

Figure 1

25 pages, 814 KiB

Open AccessArticle

An Information Theoretic Approach to Symbolic Learning in Synthetic Languages

by Andrew D. Back and Janet Wiles

Entropy 2022, 24(2), 259; https://doi.org/10.3390/e24020259 - 10 Feb 2022

Cited by 2 | Viewed by 1724

Abstract

An important aspect of using entropy-based models and proposed “synthetic languages”, is the seemingly simple task of knowing how to identify the probabilistic symbols. If the system has discrete features, then this task may be trivial; however, for observed analog behaviors described by [...] Read more.

An important aspect of using entropy-based models and proposed “synthetic languages”, is the seemingly simple task of knowing how to identify the probabilistic symbols. If the system has discrete features, then this task may be trivial; however, for observed analog behaviors described by continuous values, this raises the question of how we should determine such symbols. This task of symbolization extends the concept of scalar and vector quantization to consider explicit linguistic properties. Unlike previous quantization algorithms where the aim is primarily data compression and fidelity, the goal in this case is to produce a symbolic output sequence which incorporates some linguistic properties and hence is useful in forming language-based models. Hence, in this paper, we present methods for symbolization which take into account such properties in the form of probabilistic constraints. In particular, we propose new symbolization algorithms which constrain the symbols to have a Zipf–Mandelbrot–Li distribution which approximates the behavior of language elements. We introduce a novel constrained EM algorithm which is shown to effectively learn to produce symbols which approximate a Zipfian distribution. We demonstrate the efficacy of the proposed approaches on some examples using real world data in different tasks, including the translation of animal behavior into a possible human language understandable equivalent. Full article

(This article belongs to the Special Issue Statistical Methods for Complex Systems)

► Show Figures

Figure 1

19 pages, 1004 KiB

Open AccessArticle

Variational Embedding Multiscale Sample Entropy: A Tool for Complexity Analysis of Multichannel Systems

by Hongjian Xiao and Danilo P. Mandic

Entropy 2022, 24(1), 26; https://doi.org/10.3390/e24010026 - 24 Dec 2021

Cited by 7 | Viewed by 2865

Abstract

Entropy-based methods have received considerable attention in the quantification of structural complexity of real-world systems. Among numerous empirical entropy algorithms, conditional entropy-based methods such as sample entropy, which are associated with amplitude distance calculation, are quite intuitive to interpret but require excessive data [...] Read more.

Entropy-based methods have received considerable attention in the quantification of structural complexity of real-world systems. Among numerous empirical entropy algorithms, conditional entropy-based methods such as sample entropy, which are associated with amplitude distance calculation, are quite intuitive to interpret but require excessive data lengths for meaningful evaluation at large scales. To address this issue, we propose the variational embedding multiscale sample entropy (veMSE) method and conclusively demonstrate its ability to operate robustly, even with several times shorter data than the existing conditional entropy-based methods. The analysis reveals that veMSE also exhibits other desirable properties, such as the robustness to the variation in embedding dimension and noise resilience. For rigor, unlike the existing multivariate methods, the proposed veMSE assigns a different embedding dimension to every data channel, which makes its operation independent of channel permutation. The veMSE is tested on both stimulated and real world signals, and its performance is evaluated against the existing multivariate multiscale sample entropy methods. The proposed veMSE is also shown to exhibit computational advantages over the existing amplitude distance-based entropy methods. Full article

(This article belongs to the Special Issue Statistical Methods for Complex Systems)

► Show Figures

Figure 1

30 pages, 1019 KiB

Open AccessArticle

Intuitionistic Fuzzy Synthetic Measure on the Basis of Survey Responses and Aggregated Ordinal Data

by Bartłomiej Jefmański, Ewa Roszkowska and Marta Kusterka-Jefmańska

Entropy 2021, 23(12), 1636; https://doi.org/10.3390/e23121636 - 06 Dec 2021

Cited by 7 | Viewed by 1899

Abstract

The paper addresses the problem of complex socio-economic phenomena assessment using questionnaire surveys. The data are represented on an ordinal scale; the object assessments may contain positive, negative, no answers, a “difficult to say” or “no opinion” answers. The general framework for Intuitionistic [...] Read more.

The paper addresses the problem of complex socio-economic phenomena assessment using questionnaire surveys. The data are represented on an ordinal scale; the object assessments may contain positive, negative, no answers, a “difficult to say” or “no opinion” answers. The general framework for Intuitionistic Fuzzy Synthetic Measure (IFSM) based on distances to the pattern object (ideal solution) is used to analyze the survey data. First, Euclidean and Hamming distances are applied in the procedure. Second, two pattern object constructions are proposed in the procedure: one based on maximum values from the survey data, and the second on maximum intuitionistic values. Third, the method for criteria comparison with the Intuitionistic Fuzzy Synthetic Measure is presented. Finally, a case study solving the problem of rank-ordering of the cities in terms of satisfaction from local public administration obtained using different variants of the proposed method is discussed. Additionally, the comparative analysis results using the Intuitionistic Fuzzy Synthetic Measure and the Intuitionistic Fuzzy TOPSIS (IFT) framework are presented. Full article

(This article belongs to the Special Issue Statistical Methods for Complex Systems)

► Show Figures

Figure 1

14 pages, 5888 KiB

Open AccessArticle

A Method for Estimating the Entropy of Time Series Using Artificial Neural Networks

by Andrei Velichko and Hanif Heidari

Entropy 2021, 23(11), 1432; https://doi.org/10.3390/e23111432 - 29 Oct 2021

Cited by 26 | Viewed by 4712

Abstract

Measuring the predictability and complexity of time series using entropy is essential tool designing and controlling a nonlinear system. However, the existing methods have some drawbacks related to the strong dependence of entropy on the parameters of the methods. To overcome these difficulties, [...] Read more.

Measuring the predictability and complexity of time series using entropy is essential tool designing and controlling a nonlinear system. However, the existing methods have some drawbacks related to the strong dependence of entropy on the parameters of the methods. To overcome these difficulties, this study proposes a new method for estimating the entropy of a time series using the LogNNet neural network model. The LogNNet reservoir matrix is filled with time series elements according to our algorithm. The accuracy of the classification of images from the MNIST-10 database is considered as the entropy measure and denoted by NNetEn. The novelty of entropy calculation is that the time series is involved in mixing the input information in the reservoir. Greater complexity in the time series leads to a higher classification accuracy and higher NNetEn values. We introduce a new time series characteristic called time series learning inertia that determines the learning rate of the neural network. The robustness and efficiency of the method is verified on chaotic, periodic, random, binary, and constant time series. The comparison of NNetEn with other methods of entropy estimation demonstrates that our method is more robust and accurate and can be widely used in practice. Full article

(This article belongs to the Special Issue Statistical Methods for Complex Systems)

► Show Figures

Figure 1

26 pages, 2190 KiB

Open AccessArticle

Using the Relative Entropy of Linguistic Complexity to Assess L2 Language Proficiency Development

by Kun Sun and Rong Wang

Entropy 2021, 23(8), 1080; https://doi.org/10.3390/e23081080 - 20 Aug 2021

Cited by 2 | Viewed by 2757

Abstract

This study applies relative entropy in naturalistic large-scale corpus to calculate the difference among L2 (second language) learners at different levels. We chose lemma, token, POS-trigram, conjunction to represent lexicon and grammar to detect the patterns of language proficiency development among different L2 [...] Read more.

This study applies relative entropy in naturalistic large-scale corpus to calculate the difference among L2 (second language) learners at different levels. We chose lemma, token, POS-trigram, conjunction to represent lexicon and grammar to detect the patterns of language proficiency development among different L2 groups using relative entropy. The results show that information distribution discrimination regarding lexical and grammatical differences continues to increase from L2 learners at a lower level to those at a higher level. This result is consistent with the assumption that in the course of second language acquisition, L2 learners develop towards a more complex and diverse use of language. Meanwhile, this study uses the statistics method of time series to process the data on L2 differences yielded by traditional frequency-based methods processing the same L2 corpus to compare with the results of relative entropy. However, the results from the traditional methods rarely show regularity. As compared to the algorithms in traditional approaches, relative entropy performs much better in detecting L2 proficiency development. In this sense, we have developed an effective and practical algorithm for stably detecting and predicting the developments in L2 learners’ language proficiency. Full article

(This article belongs to the Special Issue Statistical Methods for Complex Systems)

► Show Figures

Figure 1

20 pages, 1730 KiB

Open AccessArticle

Asymptotic Information-Theoretic Detection of Dynamical Organization in Complex Systems

by Gianluca D’Addese, Laura Sani, Luca La Rocca, Roberto Serra and Marco Villani

Entropy 2021, 23(4), 398; https://doi.org/10.3390/e23040398 - 27 Mar 2021

Cited by 2 | Viewed by 2629

Abstract

The identification of emergent structures in complex dynamical systems is a formidable challenge. We propose a computationally efficient methodology to address such a challenge, based on modeling the state of the system as a set of random variables. Specifically, we present a sieving [...] Read more.

The identification of emergent structures in complex dynamical systems is a formidable challenge. We propose a computationally efficient methodology to address such a challenge, based on modeling the state of the system as a set of random variables. Specifically, we present a sieving algorithm to navigate the huge space of all subsets of variables and compare them in terms of a simple index that can be computed without resorting to simulations. We obtain such a simple index by studying the asymptotic distribution of an information-theoretic measure of coordination among variables, when there is no coordination at all, which allows us to fairly compare subsets of variables having different cardinalities. We show that increasing the number of observations allows the identification of larger and larger subsets. As an example of relevant application, we make use of a paradigmatic case regarding the identification of groups in autocatalytic sets of reactions, a chemical situation related to the origin of life problem. Full article

(This article belongs to the Special Issue Statistical Methods for Complex Systems)

► Show Figures

Figure 1

20 pages, 5268 KiB

Open AccessArticle

Exploring the Neighborhood of q-Exponentials

by Henrique Santos Lima and Constantino Tsallis

Entropy 2020, 22(12), 1402; https://doi.org/10.3390/e22121402 - 11 Dec 2020

Cited by 2 | Viewed by 2068

Abstract

The q-exponential form

e_{q}^{x} \equiv {[1 + (1 - q) x]}^{1 / (1 - q)} (e_{1}^{x} = e^{x})

is obtained by optimizing the nonadditive entropy [...] Read more.

The q-exponential form

e_{q}^{x} \equiv {[1 + (1 - q) x]}^{1 / (1 - q)} (e_{1}^{x} = e^{x})

is obtained by optimizing the nonadditive entropy

S_{q} \equiv k \frac{1 - \sum_{i} p_{i}^{q}}{q - 1}

(with

S_{1} = S_{B G} \equiv - k \sum_{i} p_{i} ln p_{i}

, where BG stands for Boltzmann–Gibbs) under simple constraints, and emerges in wide classes of natural, artificial and social complex systems. However, in experiments, observations and numerical calculations, it rarely appears in its pure mathematical form. It appears instead exhibiting crossovers to, or mixed with, other similar forms. We first discuss departures from q-exponentials within crossover statistics, or by linearly combining them, or by linearly combining the corresponding q-entropies. Then, we discuss departures originated by double-index nonadditive entropies containing

S_{q}

as particular case. Full article

(This article belongs to the Special Issue Statistical Methods for Complex Systems)

► Show Figures

Figure 1

Other

Jump to: Research

23 pages, 3656 KiB

Open AccessSystematic Review

A Review of Technological Forecasting from the Perspective of Complex Systems

by Lijie Feng, Qinghua Wang, Jinfeng Wang and Kuo-Yi Lin

Entropy 2022, 24(6), 787; https://doi.org/10.3390/e24060787 - 04 Jun 2022

Cited by 7 | Viewed by 2937

Abstract

Technology forecasting (TF) is an important way to address technological innovation in fast-changing market environments and enhance the competitiveness of organizations in dynamic and complex environments. However, few studies have investigated the complex process problem of how to select the most appropriate forecasts [...] Read more.

Technology forecasting (TF) is an important way to address technological innovation in fast-changing market environments and enhance the competitiveness of organizations in dynamic and complex environments. However, few studies have investigated the complex process problem of how to select the most appropriate forecasts for organizational characteristics. This paper attempts to fill this research gap by reviewing the TF literature based on a complex systems perspective. We first identify four contexts (technology opportunity identification, technology assessment, technology trend and evolutionary analysis, and others) involved in the systems of TF to indicate the research boundary of the system. Secondly, the four types of agents (field of analysis, object of analysis, data source, and approach) are explored to reveal the basic elements of the systems. Finally, the visualization of the interaction between multiple agents in full context and specific contexts is realized in the form of a network. The interaction relationship network illustrates how the subjects coordinate and cooperate to realize the TF context. Accordingly, we illustrate suggest five trends for future research: (1) refinement of the context; (2) optimization and expansion of the analysis field; (3) extension of the analysis object; (4) convergence and diversification of the data source; and (5) combination and optimization of the approach. Full article

(This article belongs to the Special Issue Statistical Methods for Complex Systems)

► Show Figures

Figure 1

Journal Menu

Journal Browser

Statistical Methods for Complex Systems

Share This Special Issue

Special Issue Editors

Special Issue Information

Keywords

Published Papers (13 papers)

Research

Other

Further Information

Guidelines

MDPI Initiatives

Follow MDPI