An Ensemble Feature Selection Approach for Analysis and Modeling of Transcriptome Data in Alzheimer’s Disease

Abstract

Data-driven analysis and characterization of molecular phenotypes comprises an efficient way to decipher complex disease mechanisms. Using emerging next generation sequencing technologies, important disease-relevant outcomes are extracted, offering the potential for precision diagnosis and therapeutics in progressive disorders. Single-cell RNA sequencing (scRNA-seq) allows the inherent heterogeneity between individual cellular environments to be exploited and provides one of the most promising platforms for quantifying cell-to-cell gene expression variability. However, the high-dimensional nature of scRNA-seq data poses a significant challenge for downstream analysis, particularly in identifying genes that are dominant across cell populations. Feature selection is a crucial step in scRNA-seq data analysis, reducing the dimensionality of data and facilitating the identification of genes most relevant to the biological question. Herein, we present a need for an ensemble feature selection methodology for scRNA-seq data, specifically in the context of Alzheimer’s disease (AD). We combined various feature selection strategies to obtain the most dominant differentially expressed genes (DEGs) in an AD scRNA-seq dataset, providing a promising approach to identify potential transcriptome biomarkers through scRNA-seq data analysis, which can be applied to other diseases. We anticipate that feature selection techniques, such as our ensemble methodology, will dominate analysis options for transcriptome data, especially as datasets increase in volume and complexity, leading to more accurate classification and the generation of differentially significant features.

1. Introduction

High throughput molecular biology technologies, such as whole-genome sequencing, have been used extensively in a research capacity to investigate disease mechanisms, allowing a deeper analysis at the cellular level and more reliable results [1]. In the era of personalized medicine there is a constant need to develop robust computational approaches for big data analysis in order to explore molecular entities and hidden patterns in a more realistic manner [2]. Towards overcoming these challenges, remarkable progress has been observed through different generations of sequencing technology providing explosive growth in omics data [3]. Single-cell sequencing, as an emerging technique, has revolutionized the way diseases are studied at the cellular level. This technology can accurately study individual cells and explore pathological mechanisms at the single-cell level to indicate diagnostic biomarkers or potent therapeutic targets [4]. The single-cell RNA sequencing (scRNA-seq) technique allows detection and quantitative analysis of mRNA molecules at a single cell resolution instead of bulk RNAseq studies which investigate global gene expression. It is a complex process, which involves single cell isolation and capture, lysis of cells, reverse transcription, amplification, and library preparation [5,6]. However, although experimental approaches are rapidly increasing, in silico pipelines for handling raw data files remain limited. A typical scRNA-seq dataset includes thousands of cells and their corresponding transcripts, therefore, from a computational perspective, we have to deal with a large amount of complex data (features) for each cell with ultra-high dimensionality and huge volume [7].

To effectively analyze these complex data, feature selection is a critical aspect of machine learning data preprocessing, which has gained significant importance in recent years due to the exponential growth in the size of real-world datasets [8]. The feature selection process aims to identify and select relevant features to improve model performance and efficiency. By removing irrelevant features, it can also improve the efficiency of the algorithm. In the context of scRNA-seq data, feature selection is particularly important as it can help identify the most important genes and cellular pathways, leading to a better understanding of the cellular functionality [9].

Machine learning (ML) processes are the first choice for mining information from such datasets as they can tackle part of this data complexity. In terms of high dimensionality due to the large number of feature spaces (genes), ML approaches to feature extraction can address this challenge by removing as much noisy and redundant information from the extracted features as possible [10]. In some cases, feature importance scores generated by ML methods may be unreliable, which can lead to instability and make it challenging to distinguish between important and unimportant features [11]. While filter and wrapper feature selection methods are commonly used, they also have some limitations. One common issue with filter approaches is that they tend to overlook how the classifier interacts with each feature and evaluate each feature independently, neglecting feature correlations. Additionally, it is difficult to determine the appropriate threshold for selecting only relevant features and excluding noise [12]. Wrapper feature selection also has several drawbacks, including a greater risk of overfitting, longer training times, reliance on a specific classifier, and higher computational cost and discriminatory power [13].

Ensemble feature selection frameworks provide a dependable method to decrease the dimensionality of high-dimensional data and pinpoint the most meaningful features in a case study [14,15]. An ensemble method that combines the results of multiple feature selection techniques can overcome the limitations of using single feature selection methods. This approach can lead to improved performance and robustness compared to any individual method. Therefore, it is important to utilize specific feature selection methods to minimize irrelevant and redundant features, which will decrease complexity in the analysis and models and improve the overall modeling process efficiency [16]. In a recent study, a gene selection pipeline combining filter, wrapper, and unsupervised methods was developed to select the relevant features in causing Alzheimer’s disease (AD) [17]. An ensemble method based on consensus-guided unsupervised feature selection was proposed to identify Huntington’s disease-associated genes [18].

In this study, we present the scEFS (ensemble feature selection for single-cell RNA-seq data) method, an ensemble framework that takes advantages of three different feature selection methods for single-cell RNA-seq data. These methods are combined using a ranking strategy to determine the best genes. We evaluate the effectiveness of various feature selection strategies in a prediction task using single-cell RNA-sequencing data from a mouse model with Alzheimer’s-like pathology and phenotypes (CK-p25 model). Our goal is to demonstrate the complexity of scRNA-seq datasets and investigate the impact of ensemble feature selection on the prediction accuracy using traditional classifiers.

2. Materials and Methods

Our scEFS approach is summarized in two main steps: (a) the application of three main feature selection methods; (b) the genes prioritization through an ensemble voting technique, called Borda.

2.1. Feature Selection Techniques

DUBStepR (determining the underlying basis using stepwise regression) is a feature selection algorithm that is used to identify important genes for a given phenotype [19]. It uses a novel measure called the density index (DI) to identify gene–gene correlations and to evaluate the inhomogeneity in feature space. It leverages these correlations to perform stepwise regression analysis to select the most informative genes. The algorithm is designed to work with high-dimensional datasets and can be applied to a wide range of biological and medical applications. This tool can be useful in identifying key genes and pathways associated with a specific phenotype or disease and can aid the development of new diagnostic or therapeutic strategies. One limitation of the DUBStepR algorithm is that it is based on gene–gene correlations, which may not always accurately reflect the underlying biology of a phenotype or disease. Additionally, the algorithm is based on stepwise regression, which can be sensitive to the choice of initial genes and may not always converge to the optimal solution.

Random forest is a popular machine learning algorithm that is commonly used for classification and regression tasks. One of the key strengths of random forest is its ability to provide insight into the relative importance of each feature (or variable) in the dataset. Variable importance, as calculated by random forest, can be used as a feature selection method. The idea is to rank the features based on their importance and select the top-ranking features for further analysis or use in a predictive model. This method has several advantages over traditional feature selection methods since random forest can handle non-linear relationships and interactions between features, and it does not require the assumption of independence between features. However, there are also some limitations to using random forest for feature selection. One main limitation is that the feature importance scores can be affected by the correlation between features. It should be used in combination with other methods and domain knowledge to come to a more informed decision.

Furthermore, we applied the statistical strategy using the BPSC package [20]. BPSC (Beta-Poisson model for single-cell RNA-seq data analyses) is an R tool for the analysis of single-cell RNA-seq data. It uses a statistical model called the Beta-Poisson model to analyze the expression of genes in individual cells. The Beta-Poisson model is a flexible model that can handle over-dispersed count data, which are a common characteristic of single-cell RNA-seq data. One of the major advantages of BPSC is that it is able to handle high-dimensional single-cell data and it can be used to identify biologically meaningful gene expression patterns. However, as with any computational method, it also has some limitations. The performance of BPSC depends on the quality of the input data and the specific characteristics of the sample.

2.2. Important Genes Prioritization

The three main FS methods are combined, creating a consensus score regarding the importance of genes under the perspective of predicting process accuracy. Our aim here is to export the optimal list of genes which offer a capable separation among the cell classes in our single-cell RNA-seq experimental dataset. We employed the Borda count, a single-winner election technique in which voters rank candidates or choices according to preference [21]. By awarding each candidate a certain number of points for each ballot depending on the number of lowest-rated contestants, the Borda count determines the outcome of a debate or the winner of an election. The choice or candidate with the most points after all votes have been tallied is declared the winner. A ranking list was constructed for each FS method, and using Borda count, we received a list of ranked genes that include all the values from the variable importance scores from the three main FS methods. As a result, the genes with the highest score are the key players for the better separability of our cell samples regarding their classes.

3. Results and Discussion

We evaluated the classification performance of our scEFS with the three main feature selection methods using expression profiles from a single-cell RNA-seq analysis. All methods were applied to the GSE103334 dataset from Gene Expression Omnibus [22]. This study tracks the microglia activation in neurodegeneration, examining their phenotypic heterogeneity and transcriptional dynamic. The experimental approach utilized a neurodegeneration mouse model with Alzheimer’s-like pathology and phenotypes (CK-p25 model), discovering novel microglia cell states and uncovering the underlying transcriptional programs. In our analysis, this case is considered a classification task for the accurate prediction of selected datasets derived from two different timepoints (0 weeks and 2 weeks) to assess microglia activation during the progression of neurodegeneration between CK-p25 animals and controls.

The implementation process involved calculating the p-value of each feature using the BSPC package and retaining only the top 200 features with the lowest p-values. Additionally, the varImp function from the random forest classifier was used to determine the significance of each variable in predicting the class attribute of the dataset. Only the top 200 most important variables were kept. Similarly, the DUBStepR algorithm was executed by isolating the same number of the most dominant genes.

A Venn diagram was used as a visual approach to organize and compare differentially expressed genes obtained from the various applied feature selection methods. It comprises a useful tool for illustrating the relationship and overlap between different datasets as well as highlighting the percentage of genes selected by each method. As depicted in Figure 1, the results of each technique yielded a set of characteristics that exhibited a maximum of 31% unique representation. There was minimal overlap between the feature selection methods, with a total of 2% overlap observed among all the different methods, and even the overlap between pairs of methods did not exceed 8%. Given this limited overlap, an ensemble method was proposed to identify potential gene markers by utilizing a ranking of all of these distinct gene groups. This limited overlap can be attributed to the dissimilar perspectives of each technique, since each feature selection method utilizes a distinct strategy to find the most dominant genes.

The kNN classifier was used to provide a comparison between the three different techniques and the initial data. The k-nearest neighbor (kNN) method is a well-known classification approach in the fields of data mining and statistics due to the ease with which it can be implemented and the high classification performance it offers [23]. In each case study, we initially trained each classification algorithm by providing it with the set of 200 informative genes selected using each gene selection method. The goal was to assess how well the aforementioned groups can predict the class feature and compare them based on metrics such as accuracy, sensitivity, specificity, and F1 score, as

Table 1. Prediction accuracy with kNN classifier prediction accuracy based on 200 informative genes by each gene selection method and initial data.

Filter Data	Accuracy	Kappa	Precision	Recall	F1	Elapsed Time (m)
Initial Data	0.77	0.55	0.73	0.86	0.79	4.25
Variable Data	0.69	0.39	0.72	0.63	0.67	1.03
Importance Data	0.81	0.63	0.83	0.78	0.81	1.3
Statistical Data	0.60	0.21	0.72	0.34	0.46	1.01
EFSM	0.82	0.65	0.83	0.81	0.82	1

shows. In Figure 2, the receiver operating characteristics (ROC) curve is provided and the area under the curve (AUC) is used as a measure of performance between all groups.

In order to understand the high-level functions and utilities of the biological system according to the isolated genes, we conducted an enrichment analysis using Enrichr https://maayanlab.cloud/Enrichr/ (accessed on 6 November 2022) [24,25]. GO function and Reactome https://reactome.org/ (accessed on 6 November 2022) [26] pathway enrichment analyses were performed for the twelve DEGs, as Figure 1 shows. Following enrichment analysis, we concluded that AD is proven. The enriched GO terms were divided into biological process (BP), molecular function (MF), and cellular component (CC) ontologies (Tables S1–S4). The results of the GO analysis indicated that DEGs were mainly enriched in BP, including mRNA metabolic process, oxidative RNA demethylation, regulation of core promoter binding, and glandular epithelial cell development (Figure S1A). MF analysis revealed that the DEGs were significantly enriched endoribonuclease activity, producing 3′-phosphomoesters, guanylate kinase activity, cAMP binding, and microtubule plus-end binding (Figure S1B). For CC, the DEGs were enriched in a collage-containing extracellular matrix, a basement membrane and endosome lumen (Figure S1C). The results of the Reactome pathway analysis showed that DEGs were mainly enriched in pathways such as in assembly of collagen fibrils extracellular matrix organization, collagen formation, and cell–cell communication (Figure S1D). More precisely, a Dst (dystonin) gene encodes a member of the plakin protein family of adhesion junction plaque proteins acting as an integrator of intermediate filaments and microtubule cytoskeleton networks. A Dst regulates the organization and stability of the microtubule network of sensory neurons to allow axonal transport and mediates docking of the dynein/dynactin motor complex to vesicle cargos for retrograde axonal transport through its interaction with TMEM108 and DCTN1, while loss of function can cause hereditary sensory, autonomic neuropathy type 6, and epidermolysis bullosa simplex [27]. Biological process analysis indicated that a Dst was significantly enriched in hemidesmosome assembly and retrograde axonal transport (Figure S2A, Table S6). Deficits in the latter are associated with the pathogenesis of multiple neurodegenerative diseases, including amyotrophic lateral sclerosis [28]. Pathway enrichment analysis showed that a Dst was mainly enriched in the RHO family of GTPases (Figure S2D, Table S8) which are involved in the regulation of cell migration and cell adhesion and play important roles in neuronal development [29].

A Rapgef4 (Rap guanine nucleotide exchange factor 4) gene is involved in the regulation of neuronal action potential and the development of the nervous system and plays a role in postsynaptic density and glutamatergic synapse. For BF, Rapgef4 was enriched in the regulation of peptide hormone secretion, the regulation of insulin and protein secretion, respectively (Figure S3A, Table S9). Peptide hormones are also the hypothalamic peptide hormones such as CRH, GHIGH, and TRH and their function is implicated in the regulation of peptide-containing secretory neurons [30]. The results of MF analysis revealed that Rapgef4 was mainly enriched cyclic adenosine monophosphate (cAMP) binding, cyclic nucleotide binding, and guanyl-nucleotide exchange factor activity (Figure S3B, Table S10). It should be noticed that cAMP is involved in neuronal functionality and is known to activate and integrate a variety of downstream pathways. cAMP-dependent signaling takes place in neuronal metabolism, growth cone motility, and neuroprotection in the central nervous system [31]. Reactome pathway analysis indicated that Rapgef4 was significantly enriched in Rap1 and the integrin signaling R-HAS pathway, platelet aggregation, and glycagon-like peptide (Figure S3C, Table S11). Neuronal Rap1 regulates energy balance, glucose homeostasis, and leptin actions [32]. Moreover, the results of GO analysis indicated that Pax6, a gene which the encoded highly conserved transcription factor is essential in the formation of tissues and organs during embryonic development, was significantly enriched in BP, including regulation of core promoter binding, glandular epithelial cell development, and neuron fate commitment (Figure 3A). Pax6 plays a critical role for neural stem cell proliferation and neurogenesis in many regions of the central nervous system, including the cerebral cortex [33]. It also controls neuronal development and targets a large number of promoters in neural progenitor cells [34]. Furthermore, the results of the Reactome pathway analysis showed that Cask was mainly enriched in dopamine neurotransmitter release cycle R-HAS and assembly and cell surface presentation of NMDA receptors R-HAS (Figure 3B). The encoded multidomain scaffolding protein is highly expressed in the mammalian nervous system and participates in brain development [35]. It was originally identified as a binding partner of neurexins, transmembrane proteins expressed in neurons, and neuroendocrine cells, while loss of its action affects synaptic function in cortical excitatory neurons [36]. Recent studies indicated that CASK is localized to the nucleus in both mouse neuronal cultures and brain tissues [37].

Cathepsin B (CTSB), another high-scored gene in our analysis, plays a neuroprotective role in AD and elevated levels have been associated with targeting intervention against the disease. It is considered as a candidate protease for the generation of N-terminally truncated Aβ in astroglial cell cultures [38]. Enhanced hippocampal and cortical amyloid depositions were also observed in the cathepsin B-deficient mouse model of AD overexpressing human amyloid protein precursors (hAPP) [39]. According to our GO analysis, MF showed that Ctsb was enriched in cysteine-type peptidase activity (Figure S6B, Table S20). New insights into the role of cysteine protease inhibitor, such as cysteine cathepsins and calpain 1, in neuroinflammation have been recently reported [40,41]. For CC, Ctsb was mainly enriched in endolysosome lumen, endolysosome, and endosome lumen (Figure 3C). Alterations in endolysosomal trafficking can induce neurodevelopmental progression [42]. In particular, the accumulation of protease-deficient LAMP1-positive organelles has been observed in axonal distensions near extracellular Aβ plaques in AD modes, indicating defects in lysosome maturation with AD pathogenesis [43]. Lastly, the GO analysis showed that Zfhx3 in BF was significantly enriched in the regulation of neuronal differentiation, adhesion, and brain development (Figure 3D). ZFHX3 (zinc finger homeobox 3) encodes a transcription factor with multiple homeodomains and zinc finger motifs which regulates myogenic and neuronal differentiation. The survival of neurons by inducing platelet-derived growth factor receptor β expression is promoted through the signaling pathway involving cAMP-responsive element-binding protein (CREB) and ZFHX3 [44]. The KEGG pathway map (Figure S8) shows significant expressed genes identified in the discrimination of the cell classes [45].

Given a dataset with ultra-high dimensionality we have two main pillars to deal with its complexity: dimensionality reduction and feature selection methods. The first pillar transforms features into a lower dimension trying to keep the pairwise sample distances as efficiently as possible based on the original feature space. The latter pillar includes a simpler method that isolates a list of features that are the most dominant features in the entire dataset. In this study, we focus on the second way since we are also interested in identifying dominant genes in scRNA-seq data that can improve the performance of various supervised learning tasks such as the classification process. An indicative feature selection categorization includes three main strategies: filter-based, wrapper-based, and ensemble-based strategies [46]. More specifically, filter methods select features based on a performance measure independent of the data modeling algorithm used, while they can classify individual features or evaluate entire subsets of features. Indicative measures for selecting the best features through such methods are information, distance, consistency, similarity, and statistical measures. The reason is that it is computationally more feasible and is performed by selecting the variables that have a higher value in a predefined numerical function that estimates the weight of the variable for the classification task. Functions incorporate various criteria such as information gain, mutual information, chi-square, odds ratio, relevance score, and correlation coefficient [47]. Wrappers are feature selection algorithms that assess a subset of features based on the accuracy of a predictive model developed with them. The evaluation is carried out with the assistance of a classifier that provides an estimate of the importance of a certain group of attributes [48]. This class of approaches has proven effective; however, their high computational cost limits their use. In terms of ensemble methods, we refer to a compilation of subsets of features derived from a variety of different base classifiers.

The list of ranked genes obtained by the present approach includes all values from the variable importance scores derived from the three feature selection methods. Using Borda count, the aim of this score is to identify genes that have high scores across all feature selection methods, so as to obtain robust and remarkable gene markers that drive the structure of the given transcriptomics dataset [49]. A gene will rank low, for example, if it presents the best value in one feature selection method and is somewhere in the middle of the ranking list for the other methods. On the other hand, a gene will earn a high Borda score if it ranks third in all scores (even it is not in the top positions) because its function was significant in all feature selection methods. According to this process, we ensure that we obtain a fairer genes hierarchy and we do not miss any important gene due to the limitations and weaknesses of each individual method. Population heterogeneity, spatial heterogeneity, and temporal heterogeneity are three distinct groups of biologically relevant heterogeneity. Cellular heterogeneity provides insights into the network connectivity and plays an important role in regulating intrinsic cell fate decisions. Data distribution derives the selection of the optimal visualization tool such as methods for classification and pathway modeling [50]. Dimensionality is the typical problem of maintaining several features in a heterogeneous dataset with a number of instances from an analytics perspective, such as in scRNA-seq [51]. The main purpose of this research is to develop an ensemble feature selection approach for transcriptome data using established feature selection techniques. The evaluation of the results is based on both how effectively the classification models respond and how adequately the enrichment analysis works.

4. Conclusions

Rapid advances in next-generation sequencing technologies are providing important insights into complex biological systems. scRNA-seq is a particularly promising transcriptomics technology; however, the high-dimensional nature of its data poses a significant challenge for downstream analysis. In this study, we proposed an ensemble feature selection methodology for scRNA-seq data, specifically in the context of Alzheimer’s disease. Our ensemble approach combines various feature selection strategies to obtain the most dominant genes in an AD scRNA-seq dataset, which can be considered as potential regulators in cellular mechanisms. An enrichment analysis was performed on the highest-scoring genes, which showed profound alterations in biological process, molecular function, and cellular component (CC) ontologies related to the disorder. We anticipate that this approach is promising for identifying potential transcriptome biomarkers through AD scRNA-seq data analysis and can be applied to other disease contexts. The encouraging results provided by this study justify the significance and suggest further implementation on datasets with high dimensionality.