Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
4.7
2.7
Journals
Entropy
Special Issues
Machine Learning Meets Stochastic Processes: New Trends for Understanding Complex...
entropy-logo
Submit to Entropy Review for Entropy Propose a Special Issue

Journal Menu

► Journal Menu

Journal Browser

► Journal Browser
share announcement

Machine Learning Meets Stochastic Processes: New Trends for Understanding Complex Systems

A special issue of Entropy (ISSN 1099-4300).

Deadline for manuscript submissions: closed (30 September 2020) | Viewed by 25174

Special Issue Editors

Dr. Gianni Pagnini

E-Mail Website
Guest Editor
1. BCAM - Basque Center for Applied Mathematics, Alameda de Mazarredo 14, E-48009 Bilbao, Basque Country, Spain
2. IKERBASQUE - Basque Foundation for Science, Calle de María Díaz de Haro 3, E-48013 Bilbao, Basque Country, Spain
Interests: stochastic processes; anomalous diffusion; fractional calculus
Dr. Santiago Mazuelas

E-Mail Website
Guest Editor
1. Basque Center for Applied Mathematics, Alameda de Mazarredo 14, E-48009 Bilbao, Basque Country, Spain
2. Ikerbasque - Basque Foundation for Science, Calle de María Díaz de Haro 3, E-48013 Bilbao, Basque Country, Spain
Interests: machine learning; sequential inference; supervised classification
Dr. Alessandro De Gregorio

E-Mail Website
Guest Editor
Department of Statistical Sciences, "Sapienza" University of Rome, P.le Aldo Moro, 5, 00185 Rome, Italy
Interests: random evolutions: transport processes, random flights; limit theorems; hyperbolic Brownian motions and random walks in hyperbolic manifolds; statistical inference for stochastic differential equations and transport processes: parametric estimation problems, parametric hypotheses tests, penalized estimators; connections between probability theory and nonlinear diffusion equations; stochastic analysis

Special Issue Information

Dear Colleagues,

Machine learning techniques have been used with great success in multiple scientific and engineering fields, that can leverage large amounts of data. Recently, the interplay between machine learning and stochastic processes has been explored through techniques such as supervised learning methods for determining equations' solutions or unsupervised learning methods for selecting system models. Such techniques can be used in numerous applications, including biological systems, finance, genomics, political analysis, migration flow analysis, and social network analysis.

This Special Issue is open to theoretical and applied contributions joining the fields of stochastic process and machine learning. Foreseen contributions include the following:

  • Methods that exploit data samples and time series to improve the efficiency or accuracy of algorithms in order to solve stochastic differential equations, hidden Markov models, and partial differential equations.
  • Statistical learning methodologies for diffusion processes and/or point processes.
  • Techniques exploring the usage of stochastic process to improve the performance of machine learning techniques, for instance, by providing additional training data.

Dr. Gianni Pagnini
Dr. Santiago Mazuelas
Dr. Alessandro De Gregorio
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Entropy is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • time series
  • statistical learning
  • statistics for stochastic processes
  • diffusion processes
  • point processes
  • stochastic differential equations
  • hidden Markov models

Published Papers (7 papers)

Download All Papers
Order results
Result details
Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:

Research

Jump to: Review

11 pages, 877 KiB  
Article
Improving the Accuracy of Nearest-Neighbor Classification Using Principled Construction and Stochastic Sampling of Training-Set Centroids
by Stephen Whitelam
Entropy 2021, 23(2), 149; https://doi.org/10.3390/e23020149 - 26 Jan 2021
Viewed by 1262
Abstract
A conceptually simple way to classify images is to directly compare test-set data and training-set data. The accuracy of this approach is limited by the method of comparison used, and by the extent to which the training-set data cover configuration space. Here we [...] Read more.
A conceptually simple way to classify images is to directly compare test-set data and training-set data. The accuracy of this approach is limited by the method of comparison used, and by the extent to which the training-set data cover configuration space. Here we show that this coverage can be substantially increased using coarse-graining (replacing groups of images by their centroids) and stochastic sampling (using distinct sets of centroids in combination). We use the MNIST and Fashion-MNIST data sets to show that a principled coarse-graining algorithm can convert training images into fewer image centroids without loss of accuracy of classification of test-set images by nearest-neighbor classification. Distinct batches of centroids can be used in combination as a means of stochastically sampling configuration space, and can classify test-set data more accurately than can the unaltered training set. On the MNIST and Fashion-MNIST data sets this approach converts nearest-neighbor classification from a mid-ranking- to an upper-ranking member of the set of classical machine-learning techniques. Full article
(This article belongs to the Special Issue Machine Learning Meets Stochastic Processes: New Trends for Understanding Complex Systems)
►▼ Show Figures

Figure 1

13 pages, 1453 KiB  
Article
Optimal Randomness for Stochastic Configuration Network (SCN) with Heavy-Tailed Distributions
by Haoyu Niu, Jiamin Wei and YangQuan Chen
Entropy 2021, 23(1), 56; https://doi.org/10.3390/e23010056 - 31 Dec 2020
Cited by 11 | Viewed by 2637
Abstract
Stochastic Configuration Network (SCN) has a powerful capability for regression and classification analysis. Traditionally, it is quite challenging to correctly determine an appropriate architecture for a neural network so that the trained model can achieve excellent performance for both learning and generalization. Compared [...] Read more.
Stochastic Configuration Network (SCN) has a powerful capability for regression and classification analysis. Traditionally, it is quite challenging to correctly determine an appropriate architecture for a neural network so that the trained model can achieve excellent performance for both learning and generalization. Compared with the known randomized learning algorithms for single hidden layer feed-forward neural networks, such as Randomized Radial Basis Function (RBF) Networks and Random Vector Functional-link (RVFL), the SCN randomly assigns the input weights and biases of the hidden nodes in a supervisory mechanism. Since the parameters in the hidden layers are randomly generated in uniform distribution, hypothetically, there is optimal randomness. Heavy-tailed distribution has shown optimal randomness in an unknown environment for finding some targets. Therefore, in this research, the authors used heavy-tailed distributions to randomly initialize weights and biases to see if the new SCN models can achieve better performance than the original SCN. Heavy-tailed distributions, such as Lévy distribution, Cauchy distribution, and Weibull distribution, have been used. Since some mixed distributions show heavy-tailed properties, the mixed Gaussian and Laplace distributions were also studied in this research work. Experimental results showed improved performance for SCN with heavy-tailed distributions. For the regression model, SCN-Lévy, SCN-Mixture, SCN-Cauchy, and SCN-Weibull used less hidden nodes to achieve similar performance with SCN. For the classification model, SCN-Mixture, SCN-Lévy, and SCN-Cauchy have higher test accuracy of 91.5%, 91.7% and 92.4%, respectively. Both are higher than the test accuracy of the original SCN. Full article
(This article belongs to the Special Issue Machine Learning Meets Stochastic Processes: New Trends for Understanding Complex Systems)
►▼ Show Figures

Figure 1

20 pages, 364 KiB  
Article
Sequentially Estimating the Approximate Conditional Mean Using Extreme Learning Machines
by Lijuan Huo and Jin Seo Cho
Entropy 2020, 22(11), 1294; https://doi.org/10.3390/e22111294 - 13 Nov 2020
Viewed by 1222
Abstract
This study examined the extreme learning machine (ELM) applied to the Wald test statistic for the model specification of the conditional mean, which we call the WELM testing procedure. The omnibus test statistics available in the literature weakly converge to a Gaussian stochastic [...] Read more.
This study examined the extreme learning machine (ELM) applied to the Wald test statistic for the model specification of the conditional mean, which we call the WELM testing procedure. The omnibus test statistics available in the literature weakly converge to a Gaussian stochastic process under the null that the model is correct, and this makes their application inconvenient. By contrast, the WELM testing procedure is straightforwardly applicable when detecting model misspecification. We applied the WELM testing procedure to the sequential testing procedure formed by a set of polynomial models and estimate an approximate conditional expectation. We then conducted extensive Monte Carlo experiments to evaluate the performance of the sequential WELM testing procedure and verify that it consistently estimates the most parsimonious conditional mean when the set of polynomial models contains a correctly specified model. Otherwise, it consistently rejects all the models in the set. Full article
(This article belongs to the Special Issue Machine Learning Meets Stochastic Processes: New Trends for Understanding Complex Systems)
25 pages, 462 KiB  
Article
Inference for Convolutionally Observed Diffusion Processes
by Shogo H Nakakita and Masayuki Uchida
Entropy 2020, 22(9), 1031; https://doi.org/10.3390/e22091031 - 15 Sep 2020
Viewed by 1980
Abstract
We propose a new statistical observation scheme of diffusion processes named convolutional observation, where it is possible to deal with smoother observation than ordinary diffusion processes by considering convolution of diffusion processes and some kernel functions with respect to time parameter. We discuss [...] Read more.
We propose a new statistical observation scheme of diffusion processes named convolutional observation, where it is possible to deal with smoother observation than ordinary diffusion processes by considering convolution of diffusion processes and some kernel functions with respect to time parameter. We discuss the estimation and test theories for the parameter determining the smoothness of the observation, as well as the least-square-type estimation for the parameters in the diffusion coefficient and the drift one of the latent diffusion process. In addition to the theoretical discussion, we also examine the performance of the estimation and the test with computational simulation, and show an example of real data analysis for one EEG data whose observation can be regarded as smoother one than ordinary diffusion processes with statistical significance. Full article
(This article belongs to the Special Issue Machine Learning Meets Stochastic Processes: New Trends for Understanding Complex Systems)
►▼ Show Figures

Figure 1

38 pages, 1196 KiB  
Article
De-Biased Graphical Lasso for High-Frequency Data
by Yuta Koike
Entropy 2020, 22(4), 456; https://doi.org/10.3390/e22040456 - 17 Apr 2020
Cited by 3 | Viewed by 2834
Abstract
This paper develops a new statistical inference theory for the precision matrix of high-frequency data in a high-dimensional setting. The focus is not only on point estimation but also on interval estimation and hypothesis testing for entries of the precision matrix. To accomplish [...] Read more.
This paper develops a new statistical inference theory for the precision matrix of high-frequency data in a high-dimensional setting. The focus is not only on point estimation but also on interval estimation and hypothesis testing for entries of the precision matrix. To accomplish this purpose, we establish an abstract asymptotic theory for the weighted graphical Lasso and its de-biased version without specifying the form of the initial covariance estimator. We also extend the scope of the theory to the case that a known factor structure is present in the data. The developed theory is applied to the concrete situation where we can use the realized covariance matrix as the initial covariance estimator, and we obtain a feasible asymptotic distribution theory to construct (simultaneous) confidence intervals and (multiple) testing procedures for entries of the precision matrix. Full article
(This article belongs to the Special Issue Machine Learning Meets Stochastic Processes: New Trends for Understanding Complex Systems)
►▼ Show Figures

Figure 1

Review

Jump to: Research

16 pages, 2276 KiB  
Review
Boltzmann Machines as Generalized Hopfield Networks: A Review of Recent Results and Outlooks
by Chiara Marullo and Elena Agliari
Entropy 2021, 23(1), 34; https://doi.org/10.3390/e23010034 - 29 Dec 2020
Cited by 17 | Viewed by 6646
Abstract
The Hopfield model and the Boltzmann machine are among the most popular examples of neural networks. The latter, widely used for classification and feature detection, is able to efficiently learn a generative model from observed data and constitutes the benchmark for statistical learning. [...] Read more.
The Hopfield model and the Boltzmann machine are among the most popular examples of neural networks. The latter, widely used for classification and feature detection, is able to efficiently learn a generative model from observed data and constitutes the benchmark for statistical learning. The former, designed to mimic the retrieval phase of an artificial associative memory lays in between two paradigmatic statistical mechanics models, namely the Curie-Weiss and the Sherrington-Kirkpatrick, which are recovered as the limiting cases of, respectively, one and many stored memories. Interestingly, the Boltzmann machine and the Hopfield network, if considered to be two cognitive processes (learning and information retrieval), are nothing more than two sides of the same coin. In fact, it is possible to exactly map the one into the other. We will inspect such an equivalence retracing the most representative steps of the research in this field. Full article
(This article belongs to the Special Issue Machine Learning Meets Stochastic Processes: New Trends for Understanding Complex Systems)
►▼ Show Figures

Figure 1

27 pages, 855 KiB  
Review
An Appraisal of Incremental Learning Methods
by Yong Luo, Liancheng Yin, Wenchao Bai and Keming Mao
Entropy 2020, 22(11), 1190; https://doi.org/10.3390/e22111190 - 22 Oct 2020
Cited by 50 | Viewed by 7488
Abstract
As a special case of machine learning, incremental learning can acquire useful knowledge from incoming data continuously while it does not need to access the original data. It is expected to have the ability of memorization and it is regarded as one of [...] Read more.
As a special case of machine learning, incremental learning can acquire useful knowledge from incoming data continuously while it does not need to access the original data. It is expected to have the ability of memorization and it is regarded as one of the ultimate goals of artificial intelligence technology. However, incremental learning remains a long term challenge. Modern deep neural network models achieve outstanding performance on stationary data distributions with batch training. This restriction leads to catastrophic forgetting for incremental learning scenarios since the distribution of incoming data is unknown and has a highly different probability from the old data. Therefore, a model must be both plastic to acquire new knowledge and stable to consolidate existing knowledge. This review aims to draw a systematic review of the state of the art of incremental learning methods. Published reports are selected from Web of Science, IEEEXplore, and DBLP databases up to May 2020. Each paper is reviewed according to the types: architectural strategy, regularization strategy and rehearsal and pseudo-rehearsal strategy. We compare and discuss different methods. Moreover, the development trend and research focus are given. It is concluded that incremental learning is still a hot research area and will be for a long period. More attention should be paid to the exploration of both biological systems and computational models. Full article
(This article belongs to the Special Issue Machine Learning Meets Stochastic Processes: New Trends for Understanding Complex Systems)
►▼ Show Figures

Figure 1

Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:
 
Displaying articles 1-7
Back to TopTop