Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
4.7
2.7
Journals
Entropy
Special Issues
Information-Theoretic Criteria for Statistical Model Selection
entropy-logo
Submit to Entropy Review for Entropy Propose a Special Issue

Journal Menu

► Journal Menu

Journal Browser

► Journal Browser
share announcement

Information-Theoretic Criteria for Statistical Model Selection

A special issue of Entropy (ISSN 1099-4300). This special issue belongs to the section "Information Theory, Probability and Statistics".

Deadline for manuscript submissions: closed (30 September 2023) | Viewed by 2208

Special Issue Editors

Dr. Abhijit Mandal

E-Mail Website
Guest Editor
Department of Mathematical Sciences, University of Texas at El Paso, El Paso, TX 79968, USA
Interests: robust inference; nonparametric methods; high-dimensional data; biostatistics; signal processing
Dr. Suneel Babu Chatla

E-Mail Website
Guest Editor
Department of Mathematical Sciences, University of Texas at El Paso, El Paso, TX 79968, USA
Interests: nonparametric methods; high-dimensional data; dependent data; statistical computing

Special Issue Information

Dear Colleagues,

Model selection has always been a popular topic in the statistics literature. Information-theoretic criterion is one popular approach for selecting the best statistical model for a given dataset. Some well-known information-theoretic criteria are Akaike Information Criterion (AIC), Bayesian Information Criterion (BIC), Mallow's Cp statistic, etc. These criteria are based on the principle of minimizing information loss when describing the data through a model. They are particularly useful when comparing models with different structures, as they provide a quantitative measure of the trade-off between model accuracy and complexity. Information-theoretic criteria are widely used for model selection in various fields, including physics, engineering, finance, statistics, and data science.

This Special Issue aims to highlight the versatility and importance of information-theoretic criteria for model selection. We welcome newly developed statistical methods that demonstrate the practicality and effectiveness of these criteria in different fields.

Dr. Abhijit Mandal
Dr. Suneel Babu Chatla
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Entropy is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • high-dimensional data
  • model selection
  • goodness-of-fit
  • data science
  • machine learning
  • big data

Published Papers (2 papers)

Download All Papers
Order results
Result details
Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:

Research

22 pages, 5319 KiB  
Article
On the Relationship between Feature Selection Metrics and Accuracy
by Elise Epstein, Naren Nallapareddy and Soumya Ray
Entropy 2023, 25(12), 1646; https://doi.org/10.3390/e25121646 - 11 Dec 2023
Viewed by 1011
Abstract
Feature selection metrics are commonly used in the machine learning pipeline to rank and select features before creating a predictive model. While many different metrics have been proposed for feature selection, final models are often evaluated by accuracy. In this paper, we consider [...] Read more.
Feature selection metrics are commonly used in the machine learning pipeline to rank and select features before creating a predictive model. While many different metrics have been proposed for feature selection, final models are often evaluated by accuracy. In this paper, we consider the relationship between common feature selection metrics and accuracy. In particular, we focus on misorderings: cases where a feature selection metric may rank features differently than accuracy would. We analytically investigate the frequency of misordering for a variety of feature selection metrics as a function of parameters that represent how a feature partitions the data. Our analysis reveals that different metrics have systematic differences in how likely they are to misorder features which can happen over a wide range of partition parameters. We then perform an empirical evaluation with different feature selection metrics on several real-world datasets to measure misordering. Our empirical results generally match our analytical results, illustrating that misordering features happens in practice and can provide some insight into the performance of feature selection metrics. Full article
(This article belongs to the Special Issue Information-Theoretic Criteria for Statistical Model Selection)
►▼ Show Figures

Figure 1

22 pages, 2074 KiB  
Article
Modified Local Linear Estimators in Partially Linear Additive Models with Right-Censored Data Based on Different Censorship Solution Techniques
by Ersin Yılmaz, Dursun Aydın and S. Ejaz Ahmed
Entropy 2023, 25(9), 1307; https://doi.org/10.3390/e25091307 - 07 Sep 2023
Viewed by 599
Abstract
This paper introduces a modified local linear estimator (LLR) for partially linear additive models (PLAM) when the response variable is subject to random right-censoring. In the case of modeling right-censored data, PLAM offers a more flexible and realistic approach to the estimation procedure [...] Read more.
This paper introduces a modified local linear estimator (LLR) for partially linear additive models (PLAM) when the response variable is subject to random right-censoring. In the case of modeling right-censored data, PLAM offers a more flexible and realistic approach to the estimation procedure by involving multiple parametric and nonparametric components. This differs from the widely used partially linear models that feature a univariate nonparametric function. The LLR method is employed to estimate unknown smooth functions using a modified backfitting algorithm, delivering a non-iterative solution for the right-censored PLAM. To address the censorship issue, three approaches are employed: synthetic data transformation (ST), Kaplan–Meier weights (KMW), and the kNN imputation technique (kNNI). Asymptotic properties of the modified backfitting estimators are detailed for both ST and KMW solutions. The advantages and disadvantages of these methods are discussed both theoretically and practically. Comprehensive simulation studies and real-world data examples are conducted to assess the performance of the introduced estimators. The results indicate that LLR performs well with both KMW and kNNI in the majority of scenarios, along with a real data example. Full article
(This article belongs to the Special Issue Information-Theoretic Criteria for Statistical Model Selection)
►▼ Show Figures

Figure 1

Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:
 
Displaying articles 1-2
Back to TopTop