Next Article in Journal
Polo-like Kinase 4: A Multifaceted Marker Linking Tumor Aggressiveness and Unfavorable Prognosis, and Insights into Therapeutic Strategies
Previous Article in Journal
MRI Assessment of Changes in Tumor Vascularization during Neoadjuvant Anti-Angiogenic Treatment in Locally Advanced Breast Cancer Patients
Previous Article in Special Issue
Inflammatory Microenvironment in Early Non-Small Cell Lung Cancer: Exploring the Predictive Value of Radiomics
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Radiomics Features on Magnetic Resonance Images Can Predict C5aR1 Expression Levels and Prognosis in High-Grade Glioma

Department of Radiology, Renmin Hospital of Wuhan University, Wuhan 430000, China
*
Author to whom correspondence should be addressed.
These authors contributed equally to this work.
Cancers 2023, 15(18), 4661; https://doi.org/10.3390/cancers15184661
Submission received: 15 August 2023 / Revised: 16 September 2023 / Accepted: 18 September 2023 / Published: 21 September 2023
(This article belongs to the Collection Radiomics and Cancers)

Abstract

:

Simple Summary

High-grade glioma is a complex disease characterized by genome instability caused by the accumulation of genetic alterations. Identifying and evaluating the oncogenes involved is crucial for determining treatment strategies and evaluating prognosis. In this study, we suggest a potential role for C5aR1 as a biomarker of glioma prognosis. Using machine learning approaches based on paired MRI and RNA sequencing data, our results show that radiomics MRI features can be used to build models that can noninvasively predict C5aR1 expression and the prognosis of patients with high-grade glioma. The radiomics models yield satisfactory performances in predicting C5aR1 expression. In addition, our findings revealed associations between MRI radiomics and immune-related features. As an effective and reproducible tool, our radiomics model may support clinical decision making and individualized treatment.

Abstract

Background: The complement component C5a receptor 1 (C5aR1) regulates cancer immunity. This retrospective study aimed to assess its prognostic value in high-grade glioma (HGG) and predict C5aR1 expression using a radiomics approach. Methods: Among 298 patients with HGG, 182 with MRI data were randomly divided into training and test groups for radiomics analysis. We examined the association between C5aR1 expression and prognosis through Kaplan–Meier and Cox regression analyses. We used maximum relevance–minimum redundancy and recursive feature elimination algorithms for radiomics feature selection. We then built a support vector machine (SVM) and a logistic regression model, investigating their performances using receiver operating characteristic, calibration curves, and decision curves. Results: C5aR1 expression was elevated in HGG and was an independent prognostic factor (hazard ratio = 3.984, 95% CI: 2.834–5.607). Both models presented with >0.8 area under the curve values in the training and test datasets, indicating efficient discriminatory ability, with SVM performing marginally better. The radiomics score calculated using the SVM model correlated significantly with overall survival (p < 0.01). Conclusions: Our results highlight C5aR1’s role in HGG development and prognosis, supporting its potential as a prognostic biomarker. Our radiomics model can noninvasively and effectively predict C5aR1 expression and patient prognosis in HGG.

1. Introduction

High-grade gliomas (HGGs) are the most common malignant brain tumors. The standard treatment guidelines recommend surgery, radiotherapy, concomitant temozolomide chemotherapy and adjuvant chemotherapy [1,2]. Despite the increased rates of maximal safe resection and considerable efforts in combination with targeted therapy, HGG progresses or inevitably recurs due to its invasive nature and treatment resistance [3]. HGG is characterized by prominent genetic heterogeneity; thus, its treatment relies on precise genetic information for its diagnostic and prognostic value [4]. Several markers, including isocitrate dehydrogenase (IDH) mutations and 1p/19q codeletion status, have become clinically used prognostic factors [5]. The integration of molecular profiling and precision therapy is important for expanding the possibilities of an effective personalized therapy for HGG.
As a recognized G-protein-coupled receptor, complement C5a receptor 1 (C5aR1) is a crucial component of the complement cascade in innate immune responses. Therapeutic C5aR1 antagonists play well-documented roles in multiple inflammatory disorders [6], and accumulating evidence indicates that C5a regulates cancer immunity [7]. The C5a/C5aR axis recruits myeloid-derived suppressor cells into tumor sites and mediates their capacity to suppress cytotoxic CD8 + T-cell function, thereby promoting tumor growth [8,9]. Additionally, elevated C5aR1 expression has been linked to poor survival in patients with head and neck squamous cell carcinomas [10]. Here, we highlight the function of the C5aR1 gene in HGG, which has not yet been studied.
The evaluation of intra-tumoral genetic heterogeneity is of great clinical significance. Accurate detection of C5aR1 status in clinical practice depends on tissue sampling through invasive surgery, which is inefficient and insufficient for acquiring the full genetic information of carcinoma tissue. Therefore, a non-invasive method that can reflect the entire genetic picture is urgently needed. Mounting evidence has shown that radiomics has various applications in HGG, including early diagnosis and typing [11,12], as well as the prediction of biological behavior and prognosis [13,14]. Certain important molecular markers, including IDH mutation and 1p/19q codeletion status, have been predicted accurately using MRI-based radiomics features [11,15]. However, so far, no study has predicted C5aR1 expression using a radiomics approach.
In the current work, we aimed to construct a preoperative contrast-enhanced MRI-based radiomics model for predicting C5aR1 expression in HGG. The aim in doing so was to investigate the association of radiomics prediction with the immunity and prognosis of patients with HGG. We then integrated bioinformatics analysis of a TCGA cohort in order to analyze C5aR1 expression in HGG and evaluated its association with clinicopathological data and prognosis. The molecular mechanisms underlying C5aR1 expression and its relationship with the immune microenvironment were also investigated.

2. Materials and Methods

2.1. Patients and Datasets

The transcriptional, clinical, and follow-up data of 298 patients with HGG were retrieved from the TCGA database (https://portal.gdc.cancer.gov/, accessed on 1 January 2023), while normal para-cancer samples were acquired from the GTEx database [16,17]. The corresponding medical imaging data were downloaded from TCIA (https://www.cancerimagingarchive.net/, accessed on 1 January 2023). For the normalization of the RNA sequencing (RNA-seq) data, raw data were transformed into transcripts per kilobase (TPM) values using the Toil software (https://github.com/BD2KGenomics/toil accessed on 1 January 2023) [18]. Clinical data included age, gender, grade, treatment regimen (radiotherapy or chemoradiotherapy), and the status of the O(6)-methylguanine-DNA methyltransferase (MGMT) promoter, IDH, and 1p/19q codeletion. The HGG cohort was first categorized into high- and low-C5aR1 groups according to the median expression value of C5aR1 using the R package Survminer.

2.2. Prognostic Analyses Based on the TCGA Cohort

The prognostic significance of the C5aR1 gene and clinicopathological factors for HGG were estimated using the survival package of R. Kaplan–Meier curves were first utilized for the survival analysis of each variable, and variations in survival probability among different groups were assessed using the log-rank test. Univariate Cox proportional hazards regression was used to determine variables that were significantly associated with overall survival (OS). Multivariate Cox regression analysis was performed to detect whether the significant variables were independent of other variables.
Subsequently, a subgroup analysis was conducted using univariate Cox regression to estimate the survival benefit of C5aR1 in different clinical subgroups. Hazard ratios (HR), 95% confidence intervals (CI), and p values were measured to express the Cox model results, which were visualized using the R package Forestplot. We performed time-dependent ROC curve analyses to validate the prognostic value of C5aR1 expression using the time-ROC package in R.

2.3. Comprehensive Analysis of Functional and Immune Characteristics Based on TCGA

The analysis of differentially expressed genes (DEGs) was performed with the R package DESeq2, while the GO and KEGG pathway enrichment analyses were conducted using the R package clusterProfiler [19]. An adjusted p value < 0.05, obtained using the Benjamini–Hochberg approach, and a false discovery rate > 0.1 were considered significant. Moreover, for the analysis of immune infiltration, the gene expression profiles (in TPM format) were imported into CIBERSORTx (https://cibersortx.stanford.edu/ accessed on 1 January 2023) to examine the relative fraction of 22 types of immune cells with 1000 permutations [20], and the variations in the immune cell fraction between the high- and low-C5aR1 groups were compared using the R package Limma.

2.4. MRI Images Preprocessing

Spatial resampling, image intensity normalization, and bias field correction were applied to reduce MR signal intensity inhomogeneity. We used the N4ITK algorithm for bias field correction to minimize the effects of magnetic field inhomogeneity.

2.5. Volume-of-Interest Segmentation

The 3D Slicer software (version 4.10.2) was used for manual volume-of-interest (VOI) segmentation. Each image was segmented by a senior radiologist with more than 10 years of working experience, who was blinded to the patient’s clinical data or diagnosis.
Twenty patients were randomly selected to evaluate the reliability of radiomics features. The obtained images were analyzed and segmented by an experienced radiologist. Based on the 20 paired segmentation results, radiomics features were automatically extracted, and the intraclass correlation coefficient (ICC) was calculated and utilized to measure inter- and intra-observer agreement.

2.6. Radiomics Features Extraction and Normalization

Radiomics features were obtained using PyRadiomics [21]. A total of 107 features were extracted in order to analyze VOI’s shape, size, intensity, morphology, and texture. The radiomic features were normalized separately for each scanner type and for each training and testing set (standardization by z-scores algorithm in the caret package of R, i.e., mean: 0 and standard deviation: 1) to minimize the effects of variability between scanner types (1.5T vs. 3T).

2.7. Radiomics Features Selection

Features with ICC values above 0.75 were chosen for further analysis. To prevent overfitting, we analyzed these features and removed redundant subsets using a feature selection procedure. The maximum relevance–minimum redundancy (mRMR) approach was used to rank the importance of features (“mRMR” package in R CRAN). The input features were ranked by optimizing the mutual information (MI) between class labels and minimizing the MI between other features. Features selected using the mRMR method were further screened using recursive feature elimination (RFE). Using RFE, the features were selected by gradually decreasing the number of features studied. Features with absolute minimum weight were removed from the feature set until the remaining features reached the desired number.

2.8. Logistic Regression and Support Vector Machine Model Establishment

Two machine learning approaches were used to develop the radiomics prediction models for the experiment: logistic regression (LR) and support vector machine (SVM). The remaining features with nonzero coefficients were retained and used in a multivariate backward stepwise LR with the minimum Akaike information criterion to establish the model. The use of SVM as a classifier is primarily based on its ability to generate classification hyperplanes, such that the margins between them and the closest instances are low. After retaining the best-performing model, the radiomics score (Rad-score) was calculated.

2.9. Model Validation and Evaluation

ROC analysis was used to determine the diagnostic significance of the current models. The assessment metrics included accuracy, specificity, sensitivity, positive predictive value and negative predictive value. A calibration curve was plotted to identify the calibration of the radiomics model using the Hosmer–Lemeshow goodness-of-fit test. The Brier score assessed the overall model performance (measure package in R). The pROC package in R was used to calculate the area under the ROC curve. The R tool rmda package was used to analyze the decision curves.

2.10. TCIA and TCGA Data Combination

By combining the TCIA and TCGA datasets, we obtained 182 overlapping data points of patients with HGG. The Rad-score calculated using the SVM model was merged with the clinical data. Next, we used the cutoff value of the Rad-score to classify the Rad-score as a binary categorical variable, with low vs. high groups. Based on the low- and high-Rad-score group, we constructed a clinical characteristics table using the CBCgrps R package. The cutoff value was calculated using the survminer R package. The correlation between the Rad-score and immune-related genes [22] was analyzed using Spearman’s correlation (stats packages in R).

2.11. Statistical Analysis

The Wilcoxon rank-sum test was used to analyze differences in C5aR1 expression between HGG and para-cancer tissues. Wilcoxon rank-sum and Pearson’s χ2 test were performed to compare continuous and categorical clinical variables between the two groups, respectively. Spearman’s correlation was used to estimate the correlation between C5aR1 and clinicopathological variables, and the results are presented in a heatmap. Statistical tests were performed using R; in all cases, the two-sided significance level was set at 0.05.
The overall study workflow is shown in Figure 1.

3. Results

3.1. C5aR1 Expression and Correlation with Clinical Baseline Characteristics

All 298 patients with HGG obtained from the TCGA database were categorized into two groups based on a cutoff value of 2.6834 for subsequent analyses, with one group with high (174 samples) and one with low (124 samples) C5aR1 expression. Table 1 shows significant variations observed between the high- and low-C5aR1 groups for age, grade, status of MGMT promoter, IDH, and 1q/19q codeletion, with all associated p values being less than 0.001. No statistically significant differences were found for gender, radiotherapy, or chemotherapy between the two groups. Wilcoxon rank-sum testing revealed that C5aR1 expression was markedly elevated in the HGG samples compared to that seen in the normal para-cancer samples (p < 0.001; Figure 2a).
Moreover, as shown in Figure 2b, C5aR1 expression positively correlated with age (Spearman’s R = 0.27) and grade (Spearman’s R = 0.59). A moderate negative association was observed between C5aR1 expression and MGMT promoter status (Spearman’s R = −0.32), IDH status (Spearman’s R = −0.5), and 1q/19q codeletion (Spearman’s R = −0.36). No correlations were found between gender, radiotherapy, or chemotherapy.

3.2. Survival Outcomes and Multivariate Analysis of Prognostic Factors for HGG

As illustrated by Kaplan–Meier survival analysis (Figure 2c), a significantly shortened OS was detected in patients with high C5aR1 expression relative to those with low C5aR1 expression (p < 0.001), with a median OS of 14.933 months (95% CI: 12.8–18.133) vs. 64.433 (95% CI: 44.633–94.5). Furthermore, the AUC of the time-dependent ROC curve for C5aR1 expression was 0.724, 0.73, and 0.708 at the 1-, 2-, and 3-year follow-up, respectively (Figure 2d). Univariate Cox regression analysis for OS in Figure 3a revealed that high C5aR1 levels correlated with poorer survival (HR: 3.984, 95% CI: 2.831–5.607, p < 0.001). In multivariate Cox regression analysis (Figure 3a), C5aR1 showed an independent prediction ability for OS (HR: 1.503, 95% CI: 1.022–2.209, p < 0.05), as did grade, IDH status, and chemotherapy. Conversely, age, MGMT promoter, and 1q/19q codeletion status did not.
We further investigated the effect of C5aR1 expression on prognosis using a subgroup analysis stratified by clinical traits (Figure 3b). These findings suggested that C5aR1 was a risk variable for OS among all subgroups, including age, gender, MGMT promoter status, radiotherapy, and chemotherapy (p < 0.001).

3.3. Immune Cell Fraction Analyses and GO/KEGG Pathway Analysis for Patients with High- and Low-C5aR1 Expression

We investigated the variation in the fraction of the 22 immune cell types between patients with high and low C5aR1 (Figure 4a). Eight types of immune cells, including T cells CD8, T cells CD4 memory resting, monocytes, and macrophages M0, M1, and M2, showed greater fractions in the high-C5aR1 group, with significant differences compared to the low-C5aR1 group (p < 0.001). Among immune cells, the M2 macrophages were the most abundant, especially in the high-C5aR1 group. We moreover identified the DEGs between patients with high- and low-C5aR1 expression. The 30 most significantly enriched KEGG and GO terms, following the GO and KEGG enrichment analyses of the identified DEGs, are shown in Figure 4b and Figure 4c, respectively.

3.4. Radiomics Features Selection

Figure A1a shows that the median ICC value of radiomics features was 0.926. There were 91 features with ICC values ≥ 0.75, accounting for 85% of all features. These features were used for further feature selection. The mRMR method was used to select the 30 highest mRMR-ranked features. The RFE method was employed to select the best feature group. Two optimal features (original_ngtdm_Contrast and original_glcm_Idn) were screened using the mRMR and RFE algorithms (Figure A1).

3.5. Establishment and Validation of Logistic Regression and SVM Models

Two optimal features were selected for the LR and SVM modeling analysis. Figure 5 depicts the performance of the C5aR1 gene expression prediction models in the training and validation cohorts. The SVM model showed satisfactory predictive ability, with an AUC of 0.828 (Figure 5a), and a 10-fold cross-validation returned a similar AUC estimate (Figure 5b). The calibration curve and Hosmer–Lemeshow goodness-of-fit test indicated a relatively high degree of concordance between the SVM model’s prediction and actual observations (p > 0.05, Figure 5c). The decision curve analysis showed that the SVM model was clinically practical (Figure 5d).
As shown in Figure 5e, The LR model also demonstrated satisfactory predictive ability with an AUC of 0.824, and cross-validation showed an AUC of 0.806 (Figure 5f). According to the calibration curve, the LR model exhibited a considerable level of concordance with real observations in relation to the expression level of the C5aR1 gene (p > 0.05, Figure 5g). The LR model also had clinical practicality, as demonstrated by the decision curve analysis (Figure 5h).
The AUC values of the SVM model and the 10-fold internal cross-validation were comparable to those of the LR model (Table 2). The DeLong test was employed to confirm the comparison of ROCs, and the findings indicated a non-significant statistical difference between the two models (p = 0.317 in the training set and p = 0.982 in the validation set). Based on the above results, we used the Rad-score output from the SVM model for further clinical analyses, it being the most optimal between the two models.

3.6. TCIA and TCGA Data Combination

Based on the intersection of the TCIA images and the TCGA clinical data, 182 patients were included in the subsequent analysis. Using 0.2424 as the cutoff value, the patients were categorized into high (n = 119)- and low (n = 63)-Rad-score groups. As shown in Table 3, the Rad-score significantly correlated with age, WHO grade, IDH status, 1p/19q codeletion, MGMT promoter status, and OS (p < 0.001). No significant relationship was detected between the Rad-score and other clinicopathological characteristics.

3.7. Validation of the Prognostic Value of the Radiomics Score

According to Kaplan–Meier curves, a high Rad-score significantly linked to poorer OS (p < 0.001) (Figure 6a). The median survival time was 15.57 and 55.53 months in the high- and low-Rad-score groups, respectively. We also analyzed the association between the Rad-score and immune-related genes (Figure 6b), observing that the Rad-score positively correlated with CD70 (Spearman’s R = 0.62, p < 0.001), CD80 (Spearman’s R = 0.61, p < 0.001), and IDO1 (Spearman’s R = 0.59, p < 0.001).

3.8. C5aR1 Expression Analysis in the SVM and Logistic Regression Model

As shown in Figure 6c, the expression levels of C5aR1 were higher in the high Rad-score group in comparison to the low group. Similar results were also obtained for the LR model (Figure 6d).

4. Discussion

In this study, we first established the prognostic role of C5aR1 in patients with HGG by investigating RNA-seq data. Patients with HGG who also presented with higher C5aR1 gene expression have a worse prognosis. C5aR1 expression was associated with advanced clinicopathological features (old age and high grade), as well as with the status of MGMT promoter, IDH, and 1q/19q codeletion. We further uncovered the biological significance underlying C5aR1 expression in HGG. We then developed and validated a contrast-enhanced T1WI MRI-based radiomics model in order to preoperatively evaluate the expression level of C5aR1 in HGG. Our results showed that a higher Rad-score was linked to a higher C5aR1 expression and poorer survival, and additional associations were observed between the Rad-score and immune-related gene sets. The current study employed radiomic analysis of MRI images to predict the status of C5aR1 expression, which may contribute to making more precise clinical decisions.
Several studies have demonstrated the existence of an association between high C5aR1 expression and poor outcomes in head and neck squamous cell carcinoma [10], gastric cancer [23], breast cancer [24], and glioblastoma [25]. Our current work focused on the field of HGG and we observed that patients with increased C5aR1 expression had worse OS. C5aR1 was an independent prognostic factor, as determined using univariate and multivariate Cox analysis.
To understand the potential biological underpinnings of the prognostic significance of C5aR1, RNA-seq data of gliomas was used to reveal the immunological and functional nature of C5aR1. The two C5aR1 subgroups showed differential immune cell compositions, among which M2 macrophages and monocytes were more enriched in the C5aR1 high-expression phenotype. Infiltration of myelomonocytic cells, specifically macrophages and monocytes, is a common denominator in cancer [26]. The M2 macrophages have been involved in tumor progression in gliomas via an immunosuppressive effect [27,28,29]. Chen et al. discovered that bone marrow-derived monocytes/macrophages infiltrate glioblastomas early in tumorigenesis [30]. Therefore, the effect of C5aR1 on prognosis may be related to the recruitment of M2 macrophages and monocytes [31]. Through GO and KEGG analyses, we further identified that C5aR1 was associated with Alzheimer’s disease-related pathways and tumor development-related pathways, e.g., the MAPK signaling cascade. These findings provide clues regarding the function of C5aR1 in HGG development. Additional research would be warranted to more comprehensively characterize the underlying regulatory mechanisms of C5aR1 in HGG.
Radiomics features have been reported as potential markers for predicting the molecular characteristics of gliomas [32,33]. Several biomarkers, including the status of 1p/19q codeletion (AUC 0.72–0.96) [34,35] and MGMT methylation (AUC 0.88) [36], have been predicted by MRI radiomics. Here, we extracted radiomics features from contrast-enhanced T1WI MRI images to construct a predictive model and calculated a Rad-score to predict C5aR1 expression levels in HGG. Two radiomic texture features (original_glcm_Idn, and original_ngtdm_Contrast), which define the association between adjacent voxels, were selected to construct the model. As a commonly used trait, the gray-level co-occurrence matrix (glcm) provides second-order statistics for grayscale data. The original_glcm_Idn reflects local heterogeneity within tumors, whereas the original_neighboring gray tone difference matrix (ngtdm)_Contrast measures the variation between gray and the average gray values at adjacent distances [37]. Our model used two common classifiers, SVM and LR, that showed similar favorable performances. Our two radiomics models achieved high predictive quality in the training and validation subsets, with AUCs of 0.828 and 0.808 for the SVM model and 0.824 and 0.806 for the LR model, respectively. Based on our observations, we conclude that MRI radiomics is an effective tool for predicting C5aR1 expression.
We further explored the association between the Rad-score and OS in a radiomics cohort with paired MRI and clinical data (n = 182). MRI radiomics have been used to estimate the OS in multiple cancer types, with several results stemming from gliomas. A glioblastoma study by Choi et al. reported that patients with a high risk Rad-score had a shorter OS [38]. Li et al. developed a 14-feature MRI radiomics model that showed the prognostic significance and predictive power of OS in glioma [13]. Two research teams indicated that the performance of radiomics was superior to that of clinical parameters, exploring the possibility of utilizing radiomic features to anticipate progression-free survival [39,40]. In our present work, a high Rad-score was significantly associated with poor prognosis in patients with HGG, which is consistent with previous reports. Altogether, the radiomics model showed great potential for risk stratification based on patient OS.
However, our approach had several limitations. First, the retrospective nature of the study meant that the influence of confounding factors was inevitable. Moreover, this was a single-center study with a small sample size, and the findings required additional verification by large-sample multicenter studies. Third, the patients included herein were recruited from public datasets and data heterogeneity may have been present. In planned follow-up investigations, we will collect our own data to further verify the predictive reliability of our developed models.

5. Conclusions

The expression level of C5aR1 was significantly linked to the prognosis of patients with HGG. Radiomics features based on contrast-enhanced T1WI MRI were linked to prognosis in patients with HGG, highlighting their potential as a robust noninvasive solution for predictively assessing C5aR1 expression in such patients. Our predictive model may be used as an effective, reproducible, and generalizable tool for the noninvasive characterization of gliomas in order to support individualized treatment.

Author Contributions

Conceptualization, Z.W.; Methodology, Z.W.; Software, Z.W.; Validation, Z.W., Y.Y. and Y.Z.; Formal Analysis, Z.W.; Investigation, Y.Y.; Resources, Z.W.; Data Curation, Z.W.; Writing—Original Draft Preparation, Y.Y.; Writing—Review and Editing, Z.W., Y.Y. and Y.Z.; Visualization, Y.Y.; Supervision, Y.Z.; Project Administration, Z.W.; Funding Acquisition, Z.W. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the Fundamental Research Funds for Central Universities (grant number: 2042023kf0036) from Wuhan University.

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki and abides by the TCIA Data Usage Policy; all datasets were de-identified, thus did not require institutional review board approval.

Informed Consent Statement

Patient informed consent was waived due to the retrospective nature of this study.

Data Availability Statement

Publicly available datasets were analyzed in this study. This data can be found in: TCGA (portal.gdc.cancer.gov/, accessed on 1 January 2023); GTEx (https://gtexportal.org/home, accessed on 1 January 2023); and TCIA (https://tcia.at/home, accessed on 1 January 2023).

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

Appendix A

Figure A1. Selection of radiomics features: (a) The evaluation of inter- and intra-observer agreement on ROI extraction by intraclass correlation efficient. (b) Recursive feature elimination method to further select features. (c,d) The importance of the two optimal features to SVM and LR algorithms, respectively.
Figure A1. Selection of radiomics features: (a) The evaluation of inter- and intra-observer agreement on ROI extraction by intraclass correlation efficient. (b) Recursive feature elimination method to further select features. (c,d) The importance of the two optimal features to SVM and LR algorithms, respectively.
Cancers 15 04661 g0a1

References

  1. Stupp, R.; Mason, W.P.; van den Bent, M.J.; Weller, M.; Fisher, B.; Taphoorn, M.J.; Belanger, K.; Brandes, A.A.; Marosi, C.; Bogdahn, U.; et al. Radiotherapy plus concomitant and adjuvant temozolomide for glioblastoma. N. Engl. J. Med. 2005, 352, 987–996. [Google Scholar] [CrossRef] [PubMed]
  2. Tan, A.C.; Ashley, D.M.; López, G.Y.; Malinzak, M.; Friedman, H.S.; Khasraw, M. Management of glioblastoma: State of the art and future directions. CA Cancer J. Clin. 2020, 70, 299–312. [Google Scholar] [CrossRef] [PubMed]
  3. Wen, P.Y.; Weller, M.; Lee, E.Q.; Alexander, B.M.; Barnholtz-Sloan, J.S.; Barthel, F.P.; Batchelor, T.T.; Bindra, R.S.; Chang, S.M.; Chiocca, E.A.; et al. Glioblastoma in adults: A Society for Neuro-Oncology (SNO) and European Society of Neuro-Oncology (EANO) consensus review on current management and future directions. Neuro-Oncology 2020, 22, 1073–1113. [Google Scholar] [CrossRef]
  4. Yang, K.; Wu, Z.; Zhang, H.; Zhang, N.; Wu, W.; Wang, Z.; Dai, Z.; Zhang, X.; Zhang, L.; Peng, Y.; et al. Glioma targeted therapy: Insight into future of molecular approaches. Mol. Cancer 2022, 21, 39. [Google Scholar] [CrossRef]
  5. Louis, D.N.; Perry, A.; Wesseling, P.; Brat, D.J.; Cree, I.A.; Figarella-Branger, D.; Hawkins, C.; Ng, H.K.; Pfister, S.M.; Reifenberger, G.; et al. The 2021 WHO Classification of Tumors of the Central Nervous System: A summary. Neuro-Oncology 2021, 23, 1231–1251. [Google Scholar] [CrossRef]
  6. Pandey, S.; Maharana, J.; Li, X.X.; Woodruff, T.M.; Shukla, A.K. Emerging Insights into the Structure and Function of Complement C5a Receptors. Trends Biochem. Sci. 2020, 45, 693–705. [Google Scholar] [CrossRef]
  7. Mastellos, D.C.; Hajishengallis, G.; Lambris, J.D. A guide to complement biology, pathology and therapeutic opportunity. Nat. Rev. Immunol. 2023. [Google Scholar] [CrossRef]
  8. Ajona, D.; Ortiz-Espinosa, S.; Moreno, H.; Lozano, T.; Pajares, M.J.; Agorreta, J.; Bértolo, C.; Lasarte, J.J.; Vicent, S.; Hoehlig, K.; et al. A Combined PD-1/C5a Blockade Synergistically Protects against Lung Cancer Growth and Metastasis. Cancer Discov. 2017, 7, 694–703. [Google Scholar] [CrossRef]
  9. Ding, P.; Li, L.; Lv, X.; Zhou, D.; Wang, Q.; Chen, J.; Yang, C.; Xu, E.; Dai, W.; Zhang, X.; et al. C5aR1 is a master regulator in Colorectal Tumorigenesis via Immune modulation. Theranostics 2020, 10, 8619–8632. [Google Scholar] [CrossRef]
  10. Gadwa, J.; Bickett, T.E.; Darragh, L.B.; Knitz, M.W.; Bhatia, S.; Piper, M.; Van Court, B.; Bhuvane, S.; Nguyen, D.; Nangia, V.; et al. Complement C3a and C5a receptor blockade modulates regulatory T cell conversion in head and neck cancer. J. Immunother. Cancer 2021, 9, e002585. [Google Scholar] [CrossRef]
  11. Yan, J.; Zhang, B.; Zhang, S.; Cheng, J.; Liu, X.; Wang, W.; Dong, Y.; Zhang, L.; Mo, X.; Chen, Q.; et al. Quantitative MRI-based radiomics for noninvasively predicting molecular subtypes and survival in glioma patients. NPJ Precis. Oncol. 2021, 5, 72. [Google Scholar] [CrossRef] [PubMed]
  12. Kim, J.Y.; Park, J.E.; Jo, Y.; Shim, W.H.; Nam, S.J.; Kim, J.H.; Yoo, R.E.; Choi, S.H.; Kim, H.S. Incorporating diffusion- and perfusion-weighted MRI into a radiomics model improves diagnostic performance for pseudoprogression in glioblastoma patients. Neuro-Oncology 2019, 21, 404–414. [Google Scholar] [CrossRef] [PubMed]
  13. Li, G.; Li, L.; Li, Y.; Qian, Z.; Wu, F.; He, Y.; Jiang, H.; Li, R.; Wang, D.; Zhai, Y.; et al. An MRI radiomics approach to predict survival and tumour-infiltrating macrophages in gliomas. Brain J. Neurol. 2022, 145, 1151–1161. [Google Scholar] [CrossRef]
  14. Sun, Q.; Chen, Y.; Liang, C.; Zhao, Y.; Lv, X.; Zou, Y.; Yan, K.; Zheng, H.; Liang, D.; Li, Z.C. Biologic Pathways Underlying Prognostic Radiomics Phenotypes from Paired MRI and RNA Sequencing in Glioblastoma. Radiology 2021, 301, 654–663. [Google Scholar] [CrossRef] [PubMed]
  15. Choi, Y.S.; Bae, S.; Chang, J.H.; Kang, S.G.; Kim, S.H.; Kim, J.; Rim, T.H.; Choi, S.H.; Jain, R.; Lee, S.K. Fully automated hybrid approach to predict the IDH mutation status of gliomas via deep learning and radiomics. Neuro-Oncology 2021, 23, 304–313. [Google Scholar] [CrossRef] [PubMed]
  16. Wu, Z.; Wang, D.; Zeng, F.; Zhang, Y.; Zhu, G.; Ma, Y.; Song, B.; Lui, S.; Wu, M. High IER5 Gene Expression Is Associated with Poor Prognosis in Glioma Patients. Front. Cell Dev. Biol. 2021, 9, 679684. [Google Scholar] [CrossRef] [PubMed]
  17. Yang, Y.; Duan, M.; Zha, Y.; Wu, Z. CENP-A is a potential prognostic biomarker and correlated with immune infiltration levels in glioma patients. Front. Genet. 2022, 13, 931222. [Google Scholar] [CrossRef] [PubMed]
  18. Vivian, J.; Rao, A.A.; Nothaft, F.A.; Ketchum, C.; Armstrong, J.; Novak, A.; Pfeil, J.; Narkizian, J.; Deran, A.D.; Musselman-Brown, A.; et al. Toil enables reproducible, open source, big biomedical data analyses. Nat. Biotechnol. 2017, 35, 314–316. [Google Scholar] [CrossRef]
  19. Yu, G.; Wang, L.G.; Han, Y.; He, Q.Y. clusterProfiler: An R package for comparing biological themes among gene clusters. OMICS 2012, 16, 284–287. [Google Scholar] [CrossRef]
  20. Newman, A.M.; Liu, C.L.; Green, M.R.; Gentles, A.J.; Feng, W.; Xu, Y.; Hoang, C.D.; Diehn, M.; Alizadeh, A.A. Robust enumeration of cell subsets from tissue expression profiles. Nat. Methods 2015, 12, 453–457. [Google Scholar] [CrossRef]
  21. van Griethuysen, J.J.M.; Fedorov, A.; Parmar, C.; Hosny, A.; Aucoin, N.; Narayan, V.; Beets-Tan, R.G.H.; Fillion-Robin, J.C.; Pieper, S.; Aerts, H. Computational Radiomics System to Decode the Radiographic Phenotype. Cancer Res. 2017, 77, e104–e107. [Google Scholar] [CrossRef] [PubMed]
  22. Xie, J.; Chen, L.; Tang, Q.; Wei, W.; Cao, Y.; Wu, C.; Hang, J.; Zhang, K.; Shi, J.; Wang, M. A Necroptosis-Related Prognostic Model of Uveal Melanoma Was Constructed by Single-Cell Sequencing Analysis and Weighted Co-Expression Network Analysis Based on Public Databases. Front. Immunol. 2022, 13, 847624. [Google Scholar] [CrossRef] [PubMed]
  23. Bao, D.; Zhang, C.; Li, L.; Wang, H.; Li, Q.; Ni, L.; Lin, Y.; Huang, R.; Yang, Z.; Zhang, Y.; et al. Integrative Analysis of Complement System to Prognosis and Immune Infiltrating in Colon Cancer and Gastric Cancer. Front. Oncol. 2020, 10, 553297. [Google Scholar] [CrossRef] [PubMed]
  24. Ou, B.; Liu, Y.; Yang, X.; Xu, X.; Yan, Y.; Zhang, J. C5aR1-positive neutrophils promote breast cancer glycolysis through WTAP-dependent m6A methylation of ENO1. Cell Death Dis. 2021, 12, 737. [Google Scholar] [CrossRef]
  25. Ye, S.; Yang, B.; Zhang, T.; Wei, W.; Li, Z.; Chen, J.; Li, X. Identification of an Immune-Related Prognostic Signature for Glioblastoma by Comprehensive Bioinformatics and Experimental Analyses. Cells 2022, 11, 3000. [Google Scholar] [CrossRef] [PubMed]
  26. Cassetta, L.; Fragkogianni, S.; Sims, A.H.; Swierczak, A.; Forrester, L.M.; Zhang, H.; Soong, D.Y.H.; Cotechini, T.; Anur, P.; Lin, E.Y.; et al. Human Tumor-Associated Macrophage and Monocyte Transcriptional Landscapes Reveal Cancer-Specific Reprogramming, Biomarkers, and Therapeutic Targets. Cancer Cell 2019, 35, 588–602. [Google Scholar] [CrossRef]
  27. Komohara, Y.; Ohnishi, K.; Kuratsu, J.; Takeya, M. Possible involvement of the M2 anti-inflammatory macrophage phenotype in growth of human gliomas. J. Pathol. 2008, 216, 15–24. [Google Scholar] [CrossRef]
  28. Wei, J.; Marisetty, A.; Schrand, B.; Gabrusiewicz, K.; Hashimoto, Y.; Ott, M.; Grami, Z.; Kong, L.Y.; Ling, X.; Caruso, H.; et al. Osteopontin mediates glioblastoma-associated macrophage infiltration and is a potential therapeutic target. J. Clin. Investig. 2019, 129, 137–149. [Google Scholar] [CrossRef]
  29. Xue, N.; Zhou, Q.; Ji, M.; Jin, J.; Lai, F.; Chen, J.; Zhang, M.; Jia, J.; Yang, H.; Zhang, J.; et al. Chlorogenic acid inhibits glioblastoma growth through repolarizating macrophage from M2 to M1 phenotype. Sci. Rep. 2017, 7, 39011. [Google Scholar] [CrossRef]
  30. Chen, Z.; Feng, X.; Herting, C.J.; Garcia, V.A.; Nie, K.; Pong, W.W.; Rasmussen, R.; Dwivedi, B.; Seby, S.; Wolf, S.A.; et al. Cellular and Molecular Identity of Tumor-Associated Macrophages in Glioblastoma. Cancer Res. 2017, 77, 2266–2278. [Google Scholar] [CrossRef]
  31. Pio, R.; Ajona, D.; Ortiz-Espinosa, S.; Mantovani, A.; Lambris, J.D. Complementing the Cancer-Immunity Cycle. Front. Immunol. 2019, 10, 774. [Google Scholar] [CrossRef] [PubMed]
  32. Qi, Y.; Zhao, T.; Han, M. The application of radiomics in predicting gene mutations in cancer. Eur. Radiol. 2022, 32, 4014–4024. [Google Scholar] [CrossRef] [PubMed]
  33. Jian, A.; Jang, K.; Manuguerra, M.; Liu, S.; Magnussen, J.; Di Ieva, A. Machine Learning for the Prediction of Molecular Markers in Glioma on Magnetic Resonance Imaging: A Systematic Review and Meta-Analysis. Neurosurgery 2021, 89, 31–44. [Google Scholar] [CrossRef]
  34. Zhou, H.; Vallières, M.; Bai, H.X.; Su, C.; Tang, H.; Oldridge, D.; Zhang, Z.; Xiao, B.; Liao, W.; Tao, Y.; et al. MRI features predict survival and molecular markers in diffuse lower-grade gliomas. Neuro-Oncology 2017, 19, 862–870. [Google Scholar] [CrossRef]
  35. van der Voort, S.R.; Incekara, F.; Wijnenga, M.M.J.; Kapas, G.; Gardeniers, M.; Schouten, J.W.; Starmans, M.P.A.; Nandoe Tewarie, R.; Lycklama, G.J.; French, P.J.; et al. Predicting the 1p/19q Codeletion Status of Presumed Low-Grade Glioma with an Externally Validated Machine Learning Algorithm. Clin. Cancer Res. Off. J. Am. Assoc. Cancer Res. 2019, 25, 7455–7462. [Google Scholar] [CrossRef]
  36. Li, Z.C.; Bai, H.; Sun, Q.; Li, Q.; Liu, L.; Zou, Y.; Chen, Y.; Liang, C.; Zheng, H. Multiregional radiomics features from multiparametric MRI for prediction of MGMT methylation status in glioblastoma multiforme: A multicentre study. Eur. Radiol. 2018, 28, 3640–3650. [Google Scholar] [CrossRef] [PubMed]
  37. Hu, L.S.; Hawkins-Daarud, A.; Wang, L.; Li, J.; Swanson, K.R. Imaging of intratumoral heterogeneity in high-grade glioma. Cancer Lett. 2020, 477, 97–106. [Google Scholar] [CrossRef]
  38. Choi, Y.; Nam, Y.; Jang, J.; Shin, N.Y.; Lee, Y.S.; Ahn, K.J.; Kim, B.S.; Park, J.S.; Jeon, S.S.; Hong, Y.G. Radiomics may increase the prognostic value for survival in glioblastoma patients when combined with conventional clinical and genetic prognostic models. Eur. Radiol. 2021, 31, 2084–2093. [Google Scholar] [CrossRef]
  39. Bae, S.; Choi, Y.S.; Ahn, S.S.; Chang, J.H.; Kang, S.G.; Kim, E.H.; Kim, S.H.; Lee, S.K. Radiomic MRI Phenotyping of Glioblastoma: Improving Survival Prediction. Radiology 2018, 289, 797–806. [Google Scholar] [CrossRef]
  40. Kickingereder, P.; Neuberger, U.; Bonekamp, D.; Piechotta, P.L.; Götz, M.; Wick, A.; Sill, M.; Kratz, A.; Shinohara, R.T.; Jones, D.T.W.; et al. Radiomic subtyping improves disease stratification beyond key molecular, clinical, and standard imaging characteristics in patients with glioblastoma. Neuro-Oncology 2018, 20, 848–857. [Google Scholar] [CrossRef]
Figure 1. Overall workflow and pipeline of the project. ****, p < 0.0001.
Figure 1. Overall workflow and pipeline of the project. ****, p < 0.0001.
Cancers 15 04661 g001
Figure 2. C5aR1 expression and clinical correlation in HGG according to TCGA. (a) Higher C5a receptor 1 (C5aR1) expression was observed in high-grade glioma (HGG) samples compared with normal tissues. (b) Correlation of C5aR1 expression with clinicopathologic features. (c) Impact of C5aR1 expression on overall survival in Kaplan–Meier curves. (d) Time-dependent receiver operating characteristic (ROC) analysis of C5aR1 expression for 1-, 2-, and 3-year survival prediction. *, p < 0.05; **, p < 0.01; ***, p < 0.001.
Figure 2. C5aR1 expression and clinical correlation in HGG according to TCGA. (a) Higher C5a receptor 1 (C5aR1) expression was observed in high-grade glioma (HGG) samples compared with normal tissues. (b) Correlation of C5aR1 expression with clinicopathologic features. (c) Impact of C5aR1 expression on overall survival in Kaplan–Meier curves. (d) Time-dependent receiver operating characteristic (ROC) analysis of C5aR1 expression for 1-, 2-, and 3-year survival prediction. *, p < 0.05; **, p < 0.01; ***, p < 0.001.
Cancers 15 04661 g002
Figure 3. Prognostic analyses based on the TCGA cohort: (a) Univariate and multivariate Cox regression analysis of C5aR1 and clinicopathologic factors. (b) Subgroup analysis and interaction test for the prognostic value of C5aR1.
Figure 3. Prognostic analyses based on the TCGA cohort: (a) Univariate and multivariate Cox regression analysis of C5aR1 and clinicopathologic factors. (b) Subgroup analysis and interaction test for the prognostic value of C5aR1.
Cancers 15 04661 g003
Figure 4. C5aR1-related immune infiltration analysis and pathway enrichment: (a) The differences in the fraction of 22 immune cell types between the high- and low-C5aR1 expression groups from the TCGA cohort. (b) Significantly enriched KEGG pathways and (c) GO annotations of C5aR1-related genes. NS, no significance; *, p < 0.05; **, p < 0.01; ***, p < 0.001.
Figure 4. C5aR1-related immune infiltration analysis and pathway enrichment: (a) The differences in the fraction of 22 immune cell types between the high- and low-C5aR1 expression groups from the TCGA cohort. (b) Significantly enriched KEGG pathways and (c) GO annotations of C5aR1-related genes. NS, no significance; *, p < 0.05; **, p < 0.01; ***, p < 0.001.
Cancers 15 04661 g004
Figure 5. Radiomics models’ performance in predicting C5aR1 expression for 182 patients with HGG from the TCIA-TCGA intersection cohort: (a,b) ROC curves of the support vector machine (SVM) model in the training and validation cohorts, with (c,d) being the associated SVM calibration and decision curves. (e,f) The logistic regression model ROC curves in the training and validation cohorts, with (g,h) being the accompanying calibration and decision curves.
Figure 5. Radiomics models’ performance in predicting C5aR1 expression for 182 patients with HGG from the TCIA-TCGA intersection cohort: (a,b) ROC curves of the support vector machine (SVM) model in the training and validation cohorts, with (c,d) being the associated SVM calibration and decision curves. (e,f) The logistic regression model ROC curves in the training and validation cohorts, with (g,h) being the accompanying calibration and decision curves.
Cancers 15 04661 g005
Figure 6. Analyses of Rad-score based on the TCIA-TCGA intersection cohort: (a) Kaplan–Meier curve showing the association of high radiomics (Rad)-score with worse overall survival of patients and the median survival time in the high- and low-Rad-score groups. (b) The correlations between Rad-score and immune-related genes. (c,d) Difference in C5aR1 expression between high- and low-Rad-score groups in the SVM and LR models, respectively. *, p < 0.05; **, p < 0.01; ***, p < 0.001; ****, p < 0.0001.
Figure 6. Analyses of Rad-score based on the TCIA-TCGA intersection cohort: (a) Kaplan–Meier curve showing the association of high radiomics (Rad)-score with worse overall survival of patients and the median survival time in the high- and low-Rad-score groups. (b) The correlations between Rad-score and immune-related genes. (c,d) Difference in C5aR1 expression between high- and low-Rad-score groups in the SVM and LR models, respectively. *, p < 0.05; **, p < 0.01; ***, p < 0.001; ****, p < 0.0001.
Cancers 15 04661 g006
Table 1. Clinical characteristics of patients with glioma from TCGA.
Table 1. Clinical characteristics of patients with glioma from TCGA.
VariablesTotal (n = 298)Low Expression of C5aR1 (n = 174)High Expression of C5aR1 (n = 124)p
Age, n (%) <0.001
 ~59197 (66)134 (77)63 (51)
 60~101 (34)40 (23)61 (49)
Gender, n (%) 0.917
 Female120 (40)71 (41)49 (40)
 Male178 (60)103 (59)75 (60)
Grade, n (%) <0.001
 III170 (57)142 (82)28 (23)
 IV128 (43)32 (18)96 (77)
MGMT_promoter_status, n (%) <0.001
 Unmethylated/Unknown118 (40)46 (26)72 (58)
 Methylated180 (60)128 (74)52 (42)
IDH_status, n (%) <0.001
 Wildtype169 (57)62 (36)107 (86)
 Mutant129 (43)112 (64)17 (14)
Chr_1p_19q_codeletion, n (%) <0.001
 Non-codel248 (83)125 (72)123 (99)
 Codel50 (17)49 (28)1 (1)
Radiotherapy, n (%) 0.466
 NO70 (23)44 (25)26 (21)
 YES228 (77)130 (75)98 (79)
Chemotherapy, n (%) 0.078
 NO79 (27)39 (22)40 (32)
 YES219 (73)135 (78)84 (68)
Table 2. The performance of the SVM and LR models.
Table 2. The performance of the SVM and LR models.
Training Cohort Validation Cohort
AUC (95% CI)ACCSPESENpAUC (95% CI)ACCSPESENp
SVM0.828 (0.731–0.924)0.8090.8250.781 0.808 (0.703–0.913)0.8090.8250.781
LR0.824 (0.727–0.921)0.8090.8250.781 0.806 (0.703–0.91)0.7980.8250.75
SVM vs. LR 0.317 0.806
AUC, area under the curve; SVM, support vector machine; LR, logistic regression; ACC, accuracy; SPE, specificity; SEN, sensitivity.
Table 3. Clinical characteristics of patients with glioma from combination of TCIA and TCGA.
Table 3. Clinical characteristics of patients with glioma from combination of TCIA and TCGA.
VariablesTotal (n = 182)Low Radiomics Score (n = 63) High Radiomics Score (n = 119)p
Gender, n (%) 0.033
 Female80 (44)35 (56)45 (38)
 Male102 (56)28 (44)74 (62)
Age, n (%) <0.001
 ~59112 (62)53 (84)59 (50)
 ~6070 (38)10 (16)60 (50)
Grade, n (%) <0.001
 III62 (34)51 (81)11 (9)
 IV120 (66)12 (19)108 (91)
MGMT_promoter_status, n (%) <0.001
Unmethylated/Unknown94 (52)19 (30)75 (63)
Methylated88 (48)44 (70)44 (37)
Chemotherapy, n (%) 0.129
 NO39 (21)9 (14)30 (25)
 YES143 (79)54 (86)89 (75)
Radiotherapy, n (%) 0.059
 NO34 (19)17 (27)17 (14)
 YES148 (81)46 (73)102 (86)
IDH_status, n (%) <0.001
 Wildtype134 (74)24 (38)110 (92)
 Mutant48 (26)39 (62)9 (8)
Chr_1p_19q_codeletion, n (%) <0.001
 Non-codel163 (90)46 (73)117 (98)
 Codel19 (10)17 (27)2 (2)
OS, n (%) <0.001
057 (31)40 (63)17 (14)
1125 (69)23 (37)102 (86)
OS time, Median (Q1, Q3)17.02 (9, 27.55)21.27 (13.85, 38.07)14.93 (7.57, 24.57)<0.001
OS, overall survival.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Wu, Z.; Yang, Y.; Zha, Y. Radiomics Features on Magnetic Resonance Images Can Predict C5aR1 Expression Levels and Prognosis in High-Grade Glioma. Cancers 2023, 15, 4661. https://doi.org/10.3390/cancers15184661

AMA Style

Wu Z, Yang Y, Zha Y. Radiomics Features on Magnetic Resonance Images Can Predict C5aR1 Expression Levels and Prognosis in High-Grade Glioma. Cancers. 2023; 15(18):4661. https://doi.org/10.3390/cancers15184661

Chicago/Turabian Style

Wu, Zijun, Yuan Yang, and Yunfei Zha. 2023. "Radiomics Features on Magnetic Resonance Images Can Predict C5aR1 Expression Levels and Prognosis in High-Grade Glioma" Cancers 15, no. 18: 4661. https://doi.org/10.3390/cancers15184661

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop