Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
4.7
2.7
Journals
Entropy
Special Issues
Information Theory for Data Science
entropy-logo
Submit to Special Issue Submit Abstract to Special Issue Review for Entropy Propose a Special Issue

Journal Menu

► Journal Menu

Journal Browser

► Journal Browser
share announcement

Information Theory for Data Science

A special issue of Entropy (ISSN 1099-4300). This special issue belongs to the section "Information Theory, Probability and Statistics".

Deadline for manuscript submissions: 15 December 2024 | Viewed by 7841

Special Issue Editors

Prof. Dr. Irad E. Ben-Gal

E-Mail Website
Guest Editor
1. Department of Industrial Engineering, The Iby and Aladar Fleischman Faculty of Engineering, Tel Aviv University, Ramat-Aviv 69978, Israel
2. Laboratory of AI Business and Data Analytics (LAMBDA), Tel Aviv University, Ramat-Aviv 69978, Israel
Interests: analytics; machine learning; probability; artificial intelligence
Special Issues, Collections and Topics in MDPI journals
Dr. Amichai Painsky

E-Mail Website
Guest Editor
Department of Industrial Engineering, The Iby and Aladar Fleischman Faculty of Engineering, Tel Aviv University, Tel Aviv-Yafo 69978, Israel
Interests: statistical learning; predictive modeling; inference problems; information theory and learning; data analysis and applications
Special Issues, Collections and Topics in MDPI journals

Special Issue Information

Dear Colleagues,

Data science is a field of study that focuses on the extraction of valuable information from noisy data, and incorporates various disciplines, such as data engineering, data visualization, predictive analytics, data mining, machine learning and statistics. In recent years, there has been a rapidly growing interest in the mathematical and theoretical aspects of data science. This manifests in probabilistic and statistical models striving to provide performance guarantee, robustness, fairness, explainability and to generate reusable and interpretable results. 

For this Special Issue, we invite contributions that focus on information theoretic methods for data science domains. We welcome unpublished original work on both the theory and the practice of the abovementioned areas.

Prof. Dr. Irad E. Ben-Gal
Dr. Amichai Painsky
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Entropy is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • data science
  • data mining
  • machine learning
  • explainable AI
  • statistics and inference
  • predictive modeling
  • big data
  • data analytics

Published Papers (4 papers)

Download All Papers
Order results
Result details
Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:

Research

18 pages, 2746 KiB  
Article
Hybrid DAER Based Cross-Modal Retrieval Exploiting Deep Representation Learning
by Zhao Huang, Haowu Hu and Miao Su
Entropy 2023, 25(8), 1216; https://doi.org/10.3390/e25081216 - 16 Aug 2023
Viewed by 969
Abstract
Information retrieval across multiple modes has attracted much attention from academics and practitioners. One key challenge of cross-modal retrieval is to eliminate the heterogeneous gap between different patterns. Most of the existing methods tend to jointly construct a common subspace. However, very little [...] Read more.
Information retrieval across multiple modes has attracted much attention from academics and practitioners. One key challenge of cross-modal retrieval is to eliminate the heterogeneous gap between different patterns. Most of the existing methods tend to jointly construct a common subspace. However, very little attention has been given to the study of the importance of different fine-grained regions of various modalities. This lack of consideration significantly influences the utilization of the extracted information of multiple modalities. Therefore, this study proposes a novel text-image cross-modal retrieval approach that constructs a dual attention network and an enhanced relation network (DAER). More specifically, the dual attention network tends to precisely extract fine-grained weight information from text and images, while the enhanced relation network is used to expand the differences between different categories of data in order to improve the computational accuracy of similarity. The comprehensive experimental results on three widely-used major datasets (i.e., Wikipedia, Pascal Sentence, and XMediaNet) show that our proposed approach is effective and superior to existing cross-modal retrieval methods. Full article
(This article belongs to the Special Issue Information Theory for Data Science)
►▼ Show Figures

Figure 1

23 pages, 2054 KiB  
Article
On Sequential Bayesian Inference for Continual Learning
by Samuel Kessler, Adam Cobb, Tim G. J. Rudner, Stefan Zohren and Stephen J. Roberts
Entropy 2023, 25(6), 884; https://doi.org/10.3390/e25060884 - 31 May 2023
Cited by 2 | Viewed by 2633
Abstract
Sequential Bayesian inference can be used for continual learning to prevent catastrophic forgetting of past tasks and provide an informative prior when learning new tasks. We revisit sequential Bayesian inference and assess whether using the previous task’s posterior as a prior for a [...] Read more.
Sequential Bayesian inference can be used for continual learning to prevent catastrophic forgetting of past tasks and provide an informative prior when learning new tasks. We revisit sequential Bayesian inference and assess whether using the previous task’s posterior as a prior for a new task can prevent catastrophic forgetting in Bayesian neural networks. Our first contribution is to perform sequential Bayesian inference using Hamiltonian Monte Carlo. We propagate the posterior as a prior for new tasks by approximating the posterior via fitting a density estimator on Hamiltonian Monte Carlo samples. We find that this approach fails to prevent catastrophic forgetting, demonstrating the difficulty in performing sequential Bayesian inference in neural networks. From there, we study simple analytical examples of sequential Bayesian inference and CL and highlight the issue of model misspecification, which can lead to sub-optimal continual learning performance despite exact inference. Furthermore, we discuss how task data imbalances can cause forgetting. From these limitations, we argue that we need probabilistic models of the continual learning generative process rather than relying on sequential Bayesian inference over Bayesian neural network weights. Our final contribution is to propose a simple baseline called Prototypical Bayesian Continual Learning, which is competitive with the best performing Bayesian continual learning methods on class incremental continual learning computer vision benchmarks. Full article
(This article belongs to the Special Issue Information Theory for Data Science)
►▼ Show Figures

Figure 1

15 pages, 969 KiB  
Article
Semi-Supervised k-Star (SSS): A Machine Learning Method with a Novel Holo-Training Approach
by Kokten Ulas Birant
Entropy 2023, 25(1), 149; https://doi.org/10.3390/e25010149 - 11 Jan 2023
Cited by 3 | Viewed by 1579
Abstract
As one of the entropy-based methods, the k-Star algorithm benefits from information theory in computing the distances between data instances during the classification task. k-Star is a machine learning method with a high classification performance and strong generalization ability. Nevertheless, as a standard [...] Read more.
As one of the entropy-based methods, the k-Star algorithm benefits from information theory in computing the distances between data instances during the classification task. k-Star is a machine learning method with a high classification performance and strong generalization ability. Nevertheless, as a standard supervised learning method, it performs learning only from labeled data. This paper proposes an improved method, called Semi-Supervised k-Star (SSS), which makes efficient predictions by considering unlabeled data in addition to labeled data. Moreover, it introduces a novel semi-supervised learning approach, called holo-training, against self-training. It has the advantage of enabling a powerful and robust model of data by combining multiple classifiers and using an entropy measure. The results of extensive experimental studies showed that the proposed holo-training approach outperformed the self-training approach on 13 out of the 18 datasets. Furthermore, the proposed SSS method achieved higher accuracy (95.25%) than the state-of-the-art semi-supervised methods (90.01%) on average. The significance of the experimental results was validated by using both the Binomial Sign test and the Friedman test. Full article
(This article belongs to the Special Issue Information Theory for Data Science)
►▼ Show Figures

Figure 1

14 pages, 570 KiB  
Article
A Bayesian Mixture Cure Rate Model for Estimating Short-Term and Long-Term Recidivism
by Rolando de la Cruz, Claudio Fuentes and Oslando Padilla
Entropy 2023, 25(1), 56; https://doi.org/10.3390/e25010056 - 28 Dec 2022
Cited by 2 | Viewed by 1634
Abstract
Mixture cure rate models have been developed to analyze failure time data where a proportion never fails. For such data, standard survival models are usually not appropriate because they do not account for the possibility of non-failure. In this context, mixture cure rate [...] Read more.
Mixture cure rate models have been developed to analyze failure time data where a proportion never fails. For such data, standard survival models are usually not appropriate because they do not account for the possibility of non-failure. In this context, mixture cure rate models assume that the studied population is a mixture of susceptible subjects who may experience the event of interest and non-susceptible subjects that will never experience it. More specifically, mixture cure rate models are a class of survival time models in which the probability of an eventual failure is less than one and both the probability of eventual failure and the timing of failure depend (separately) on certain individual characteristics. In this paper, we propose a Bayesian approach to estimate parametric mixture cure rate models with covariates. The probability of eventual failure is estimated using a binary regression model, and the timing of failure is determined using a Weibull distribution. Inference for these models is attained using Markov Chain Monte Carlo methods under the proposed Bayesian framework. Finally, we illustrate the method using data on the return-to-prison time for a sample of prison releases of men convicted of sexual crimes against women in England and Wales and we use mixture cure rate models to investigate the risk factors for long-term and short-term survival of recidivism. Full article
(This article belongs to the Special Issue Information Theory for Data Science)
►▼ Show Figures

Figure 1

Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:
 
Displaying articles 1-4
Back to TopTop