Next Article in Journal
Impaired Right Ventricular Calcium Cycling Is an Early Risk Factor in R14del-Phospholamban Arrhythmias
Next Article in Special Issue
Mandible Segmentation of Dental CBCT Scans Affected by Metal Artifacts Using Coarse-to-Fine Learning Model
Previous Article in Journal
Oncogenic Role of miRNA in Environmental Exposure to Plasticizers: A Systematic Review
Previous Article in Special Issue
Influence of the Depth of the Convolutional Neural Networks on an Artificial Intelligence Model for Diagnosis of Orthognathic Surgery
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
JPM
Volume 11
Issue 6
10.3390/jpm11060501
jpm-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Machine Learning to Predict In-Hospital Mortality in COVID-19 Patients Using Computed Tomography-Derived Pulmonary and Vascular Features

by
Simone Schiaffino
1,†,
Marina Codari
2,†,
Andrea Cozzi
3,*,†,
Domenico Albano
4,5,
Marco Alì
6,
Roberto Arioli
7,
Emanuele Avola
8,
Claudio Bnà
9,
Maurizio Cariati
10,
Serena Carriero
8,
Massimo Cressoni
1,
Pietro S. C. Danna
7,
Gianmarco Della Pepa
8,
Giovanni Di Leo
1,
Francesco Dolci
11,
Zeno Falaschi
7,
Nicola Flor
12,
Riccardo A. Foà
10,13,
Salvatore Gitto
3,
Giovanni Leati
13,
Veronica Magni
3,
Alexis E. Malavazos
14,
Giovanni Mauri
15,16,
Carmelo Messina
4,
Lorenzo Monfardini
9,
Alessio Paschè
7,
Filippo Pesapane
17,
Luca M. Sconfienza
3,4,
Francesco Secchi
1,3,
Edoardo Segalini
18,
Angelo Spinazzola
13,
Valeria Tombini
19,
Silvia Tresoldi
10,
Angelo Vanzulli
15,19,
Ilaria Vicentin
19,
Domenico Zagaria
7,
Dominik Fleischmann
2,20 and
Francesco Sardanelli
1,3add Show full author list remove Hide full author list
1
Unit of Radiology, IRCCS Policlinico San Donato, Via Rodolfo Morandi 30, 20097 Milan, Italy
2
Department of Radiology, School of Medicine, Stanford University, 300 Pasteur Drive, Stanford, CA 94305, USA
3
Department of Biomedical Sciences for Health, Università degli Studi di Milano, Via Luigi Mangiagalli 31, 20133 Milan, Italy
4
IRCCS Istituto Ortopedico Galeazzi, Via Riccardo Galeazzi 4, 20161 Milan, Italy
5
Department of Biomedicine, Neurosciences and Advanced Diagnostics, Section of Radiological Sciences, Università degli Studi di Palermo, Via del Vespro 127, 90127 Palermo, Italy
6
Department of Diagnostic Imaging and Stereotactic Radiosurgery, C.D.I. Centro Diagnostico Italiano S.p.A., Via Simone Saint Bon 20, 20147 Milan, Italy
7
Radiodiagnostics, Department of Diagnosis and Treatment Services, Azienda Ospedaliero Universitaria Maggiore della Carità, Corso Giuseppe Mazzini 18, 28100 Novara, Italy
8
Postgraduate School in Radiodiagnostics, Università degli Studi di Milano, Via Festa del Perdono 7, 20122 Milan, Italy
9
Unit of Interventional Radiology, Unit of Radiology, Fondazione Poliambulanza Istituto Ospedaliero, Via Leonida Bissolati 57, 25124 Brescia, Italy
10
Diagnostic and Interventional Radiology Service, ASST Santi Paolo e Carlo, Via Antonio di Rudinì 8, 20142 Milan, Italy
11
Emergency Department, ASST Crema—Ospedale Maggiore, Largo Ugo Dossena 2, 26013 Crema, Italy
12
Unit of Radiology, Ospedale Universitario Luigi Sacco—ASST Fatebenefratelli Sacco, Via Giovanni Battista Grassi 74, 20157 Milan, Italy
13
Unit of Interventional Radiology, Unit of Radiology, ASST Crema—Ospedale Maggiore, Largo Ugo Dossena 2, 26013 Crema, Italy
14
High Speciality Center for Dietetics, Nutritional Education and Cardiometabolic Prevention, IRCCS Policlinico San Donato, Via Rodolfo Morandi 30, 20097 Milan, Italy
15
Department of Oncology and Hematology-Oncology, Università degli Studi di Milano, Via Festa del Perdono 7, 20122 Milan, Italy
16
Division of Interventional Radiology, IEO—Istituto Europeo di Oncologia IRCCS, Via Giuseppe Ripamonti 435, 20141 Milan, Italy
17
Division of Breast Radiology, IEO—Istituto Europeo di Oncologia IRCCS, Via Giuseppe Ripamonti 435, 20141 Milan, Italy
18
Department of General and Emergency Surgery, ASST Crema—Ospedale Maggiore, Largo Ugo Dossena 2, 26013 Crema, Italy
19
ASST Grande Ospedale Metropolitano Niguarda, Piazza dell’Ospedale Maggiore 3, 20162 Milan, Italy
20
Cardiovascular Institute, 265 Campus Drive, Stanford University, Stanford, CA 94305, USA
 
add Show full affiliation list
 
remove Hide full affiliation list
*
Author to whom correspondence should be addressed.
†
These authors contributed equally to this work.
J. Pers. Med. 2021, 11(6), 501; https://doi.org/10.3390/jpm11060501
Submission received: 28 April 2021 / Revised: 31 May 2021 / Accepted: 1 June 2021 / Published: 3 June 2021
(This article belongs to the Special Issue Application of Artificial Intelligence in Personalized Medicine)
Browse Figures
Versions Notes

Abstract

:
Pulmonary parenchymal and vascular damage are frequently reported in COVID-19 patients and can be assessed with unenhanced chest computed tomography (CT), widely used as a triaging exam. Integrating clinical data, chest CT features, and CT-derived vascular metrics, we aimed to build a predictive model of in-hospital mortality using univariate analysis (Mann–Whitney U test) and machine learning models (support vectors machines (SVM) and multilayer perceptrons (MLP)). Patients with RT-PCR-confirmed SARS-CoV-2 infection and unenhanced chest CT performed on emergency department admission were included after retrieving their outcome (discharge or death), with an 85/15% training/test dataset split. Out of 897 patients, the 229 (26%) patients who died during hospitalization had higher median pulmonary artery diameter (29.0 mm) than patients who survived (27.0 mm, p < 0.001) and higher median ascending aortic diameter (36.6 mm versus 34.0 mm, p < 0.001). SVM and MLP best models considered the same ten input features, yielding a 0.747 (precision 0.522, recall 0.800) and 0.844 (precision 0.680, recall 0.567) area under the curve, respectively. In this model integrating clinical and radiological data, pulmonary artery diameter was the third most important predictor after age and parenchymal involvement extent, contributing to reliable in-hospital mortality prediction, highlighting the value of vascular metrics in improving patient stratification.

1. Introduction

Since the inception of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic, chest computed tomography (CT) has been widely used both as a triaging test on emergency department admission [1,2,3]—especially in the case of unavailability of immediate result of reverse transcriptase-polymerase chain reaction (RT-PCR)—and as a diagnostic tool to monitor Coronavirus Disease 2019 (COVID-19) pneumonia [1,2].
The widely recognized systemic prothrombotic profile of COVID-19, swiftly linked with worse outcomes [4,5,6,7,8,9,10,11], directly affects pulmonary vasculature [11,12,13,14,15,16,17,18,19,20]. In hospitalized COVID-19 patients with moderate and severe disease, CT-detected lung parenchymal involvement [21,22] has been associated with varying degrees of pulmonary arterial vascular damage [10,11,12,13,14,15], which disrupts vascular dynamics and increases pulmonary artery (PA) pressure [23,24,25,26,27,28,29]. Of note, bedside ultrasound examinations performed during hospitalization have hinted the prognostic value of indirect signs of impaired pulmonary circulation, such as right ventricular longitudinal strain and increased end-diastolic chamber size [29,30]. However, these parameters are encumbered by operator dependency and potential overlap with other pre-existing causes of right ventricular dysfunction [31]; moreover, the routine performance of echocardiography in the triage of COVID-19 patients during pandemic peaks could prove challenging, due to the high volume of patients referred to emergency departments.
Rather than identifying thrombotic phenomena as a byproduct of pulmonary embolism, ever-increasing evidence points to a direct origin of thrombosis in arterial lung vasculature [14,32,33], highlighting the need for a more accurate characterization of these phenomena in their true location [34].
Diagnosis of pulmonary arterial thrombosis requires contrast-enhanced CT angiography, which implies the intravenous administration of iodinated contrast agents [35]. However, alterations of pulmonary vascular metrics also detectable with unenhanced CT, such as the enlargement of the PA and an increased ratio between diameters of the PA and of the ascending aorta (AA), are known indirect signs of pulmonary hypertension caused by fibrotic or thromboembolic processes [36,37,38,39,40,41,42]. Notably, these pulmonary vascular metrics have been demonstrated to be altered also in COVID-19 patients—when compared to previous values measured on CT scans acquired before the SARS-CoV-2 pandemic—and to carry prognostic implications [25,26,27]. Since unenhanced chest CT is still widely performed in COVID-19 patients, the integration of CT-derived vascular features with other readily available clinical and imaging parameters could improve the stratification of COVID-19 patients on emergency department admission, potentially improving patient management and prognosis.
This multicenter study, conducted in six different hospitals in northern Italy during the first pandemic peak of 2020, aims to evaluate the prognostic power of a machine learning model integrating imaging features of lung parenchyma and vasculature with clinical data of COVID-19 patients routinely retrieved on emergency department admission.

2. Materials and Methods

2.1. Study Population

This multicenter observational study was conducted in six different institutions in northern Italy: Azienda Ospedaliero-Universitaria Maggiore della Carità, Novara (Center 1); ASST Grande Ospedale Metropolitano Niguarda, Milano (Center 2); Fondazione Poliambulanza Istituto Ospedaliero, Brescia (Center 3); ASST Crema—Ospedale Maggiore, Crema (Center 4); ASST Santi Paolo e Carlo, Milano (Center 5); and IRCCS Istituto Ortopedico Galeazzi, Milano (Center 6). Approval for this retrospective study was obtained from the Ethics Committee of the coordinating center IRCCS Policlinico San Donato (Comitato Etico IRCCS Ospedale San Raffaele, protocol code COVID19-TCretro, protocol number 77/INT/2020, approved 5 May 2020). Study-specific informed consent was waived due to the retrospective nature of the study.
We included patients hospitalized in each of these institutions during the study period (21 February to 30 April 2020) with RT-PCR-confirmed SARS-CoV-2 infection and unenhanced chest CT performed on emergency department admission. Authors from each center reviewed their own institutional electronic databases to retrieve patients’ clinical data, including: demographics, symptoms, partial pressure of oxygen in arterial blood (PaO2), comorbidities, smoking history, body mass index, and white blood cells, lymphocytes, platelets, and lactate dehydrogenase values. For outcome assessment, censoring was applied on 1 June 2020, when all patients had either been discharged or had died during hospitalization.

2.2. Image Acquisition and Analysis

Unenhanced chest CT scans were performed in all six centers in supine position, during a single inspiratory breath-hold condition when possible. Table 1 shows the technical characteristics of the scanners and acquisition parameters.
Radiologists with 7 to 32 years of experience blindly reviewed CT images from their own institution, assessing lung parenchyma and the presence of pleural effusion and mediastinum lymph nodes with short-axis diameters larger than 1 cm [43]. Lung parenchymal involvement was qualitatively assessed by radiologists on their own institutional digital imaging and communications in medicine (DICOM) viewer, considering the presence of parenchymal involvement signs, (i.e., ground-glass opacities (GGOs), consolidations, and crazy paving pattern) and the volumetric extent of parenchymal involvement, assessed according to Chung et al. [21]: 0% (absent, 0); 1–25% (minimal, 1); 26–50% (mild, 2); 51–75% (moderate, 3); and over 75% (severe, 4). As described by Wells et al. [44], PA and AA maximum diameters were measured on a single axial slice selected at the level of pulmonary arterial main trunk bifurcation (Figure 1).

2.3. Data Preprocessing and Initial Feature Selection

To build a predictive model of in-hospital mortality, we selected among available features those with no more than 25% missing data. In these selected features, residual missing data were imputed with the mean value of the respective feature. Data were randomly split into a training/validation set (85%) and a testing set (15%). In each set, all features were rescaled to have null mean and unitary variance.
To prevent overfitting, we used training/validation data to perform a two-step feature selection process. First, we removed highly correlated features by thresholding the feature correlation matrix, setting the absolute Pearson’s correlation threshold value at 0.7. Then, we used the least absolute shrinkage and selection operator (LASSO) to compute feature importance, i.e., the absolute value of the LASSO regression coefficients [45,46]. The best α value was determined using a ten-fold cross-validation process. Selected features were used to develop several machine learning models using both support vector machines (SVM) and multilayer perceptrons (MLP).

2.4. Support Vector Machines

We initialized the SVM classifier with linear kernel, C value = 1, and balanced class-weights using all selected features. Then, we performed a systematic hyperparameter grid-search to find: the optimal SVM kernel among linear, radial basis function, polynomial, and sigmoid kernels; the best C values among 100 values linearly sampled in the range 1–8; and the best γ value (for nonlinear kernels) among ten logarithmically-sampled (base 10) values in the range 10−10–10−1. Each candidate model (n = 310) was validated using a ten-fold cross-validation process, accounting for 31,000 fits. The best model was defined as the one that maximized the average F1 score in validation data. F1 score was selected as a scoring metric to consider class imbalance. Finally, the best model was refitted on the entire training/validation set.
Given this initial model and its optimal C (Copt) and γ (γopt) hyperparameters, we performed an importance-based backward feature selection. We optimized SVM hyperparameters for each feature subset using a second systematic grid search strategy, further fine-tuning C and γ values in narrower ranges. The new search-grid was defined by selecting 25 C values linearly sampled in the range (Copt ± 2), while 50 γ values were logarithmically sampled in the range ( 10 log ( γ opt − 1 ) , 10 log ( γ opt + 1 ) ) Each one of the 1250 candidate models was validated using a ten-fold cross-validation process. Again, the best model (i.e., the one that maximized the average F1 score in validation data) was finally refitted over the entire training/validation set.

2.5. Multilayer Perceptrons

After selecting all features with a non-zero LASSO coefficient, we further selected features by keeping the most important ones that allowed us to meet the rule of thumb of at least ten events in our training/validation set for each feature included in our model [47,48].
The general MLP architecture is summarized in Table 2. We initialized the hidden unit numbers N and M values to 7 and 5, respectively. To prevent dying rectified linear unit (ReLU) issues, we used LeakyReLU [49] as activation function (α = 0.03), while weights were initialized using the method proposed by He et al. [50]. To improve model generalizability and prevent overfitting, we added a dropout layer after each fully connected layer with a dropout rate equal to 0.2. The last layer of the perceptron comprised a single unit with a sigmoid activation function to encode patient mortality. The initial learning rate was set to 0.01, being then decreased during training using an exponential decay schedule of 1000 steps and a base of 0.5. We set binary cross-entropy as the loss function, adopted the Adam optimizer [51], and set a default batch-size value of 32. The F1 score, the area under the curve (AUC) at receiving operator characteristic (ROC) analysis, accuracy, precision, and recall were selected as the performance metrics. The training stopped if the loss function did not decrease for 25 epochs, then the best weights corresponding to the loss function minimum were restored. The maximum number of epochs was set to 1000.
Given this model and its hyperparameters, we performed a systematic grid-search ten-fold cross-validation to optimize the following hyperparameters: the batch size in (4, 16, 32, 64); the dropout rate in (0.1, 0.2, 0.4); the starting learning rate in (0.1, 0.01, 0.001); the number of hidden units in the second and third layers (N) in (5, 7, 10); and the number of hidden units in the fourth and fifth layers (M) in (3, 5, 7). Each one of the 324 candidate MLPs was validated using a ten-fold cross-validation process (3240 fits). The best model (i.e., the one that maximized the average F1 score in validation data) was finally refitted over the entire training/validation set.

2.6. Statistical Analysis

Due to their nonparametric distribution, assessed with the Shapiro–Wilk test, continuous variables were reported as median with interquartile range (IQR), while categorical variables as total number and percentage. The Mann–Whitney U test was used to compare means from different groups for continuous variables during univariate explorative analysis. Statistical analyses were performed using SPSS v.26.0 (IBM SPSS Inc., Chicago, IL, USA), and p values < 0.05 were considered statistically significant. All SVMs and MLPs were developed using python v.3.7.7 [52], sklearn v.0.23.1 [53], and tensorflow-gpu v.2.2.0 [54] on a laptop with an Intel Core i7-6500 CPU (2.50 GHz), 8 GB of RAM, and a Nvidia GeForce 940MX GPU.

3. Results

3.1. Population Characteristics

The study population included 897 COVID-19 patients (608 males, 68%; median age 66 years, IQR 55–77 years), hospitalized in six different centers from 21 February to 30 April 2020. A total of 270 patients came from Center 1, 197 from Center 2, 194 from Center 3, 144 from Center 4, 80 from Center 5, and 12 from Center 6.
Symptoms, comorbidities, height, weight, and laboratory parameters were assessed on emergency department admission and are reported in Table 3, alongside CT-derived lung and vascular features. The main symptoms observed on emergency department admission were fever (500/897 patients, 56%), cough (277/897 patients, 31%), and dyspnea (229/897 patients, 26%). During hospitalization, which lasted a median of 7 days (IQR 4–13 days), 133/897 patients (15%) were admitted to intensive care units (ICU). Low molecular weight heparin (LMWH) was administered to 296/897 patients (33%): to 88/296 (30%) at therapeutic dosage, to 177/296 (60%) at prophylactic dosage, while for the remaining 31/296 (10%) dosage data was not available. At censoring for outcome assessment (June 1, 2020), we found that 229/897 (26%) patients had died during hospitalization, while the remaining 668/897 (74%) had been discharged.

3.2. Explorative Univariate Analysis of Pulmonary Vascular Features

The overall median PA maximum diameter was 28 mm (25–30 mm) and median AA 34 mm (32–37 mm). Patients who died during hospitalization showed significantly higher median PA maximum diameter (29.0 mm, IQR 26.0–32.0 mm) compared to patients who survived (27.0 mm, IQR 25.0–30.0 mm, p < 0.001), and significantly higher median AA maximum diameter (36.6 mm, IQR 34.0–39.0 mm, versus 34.0 mm, IQR 31.0–36.0 mm, p < 0.001).

3.3. Support Vector Machines and Multilayer Perceptrons

After discarding features with a percentage of missing data higher than 25%, 14 features were selected for further processing (Table 3). After this selection step, residual missing data were clustered in 223 patients, who had almost only input data and were therefore also removed. Only 674 out of the initial 897 patients (75%) were therefore used to develop all machine learning models. The percentage of missing data ranged from 0% to 5% in the remaining features (median 0.4%, IQR 0.1–3.2%). After the initial random database split, our training/validation dataset was composed of 572 patients, 160 of whom (28%) died during hospitalization, while the testing set accounted for 102 patients, 30 of whom died during hospitalization (29%). No features were excluded after Pearson’s correlation matrix thresholding (Figure 2). Obtained LASSO coefficients for all initial features are reported in Figure 3.
Fourteen SVM models were developed using different feature subsets. The best SVM model took 10 features as inputs (Table S1). The best model had a radial basis function kernel, with C equal to 5.924 and γ equal to 6.06 × 10−5. After the model was refitted on the entire training/validation dataset, the final model performances on testing data were: an F1 score of 0.632, an AUC of 0.747, a precision of 0.522, and a recall of 0.800. All SVMs had radial basis function kernels, their other hyperparameters and performance being reported in Table S1, while the ROC curve of the best SVM model is depicted in panel a of Figure 4.
Due to the high number of events in the training/validation set, MLPs were built using all features with a non-null LASSO coefficient (n = 10). Therefore, all developed MLPs have the same input features as the best SVM model. Among the 324 models resulting from the systematic hyperparameter grid search, the best MLP had the following hyperparameters: N and M equal to 7 and 3 respectively; a batch-size of 4; a dropout rate equal to 0.1; and a starting learning rate equal to 0.01. After refitting the best model on the entire training/validation dataset, the final performance on testing data were: an F1 score of 0.618, an AUC of 0.844, a precision of 0.680, and a recall of 0.567. The ROC curve of the best MLP is depicted in panel b of Figure 4.

4. Discussion

In this retrospective multicenter study, we developed a predictive model of in-hospital mortality for COVID-19 patients, using clinical and radiological data acquired on emergency department admission. The CT data of 674 patients from six hospitals in northern Italy were used to extract pulmonary parenchymal and vascular features. While machine learning models centered on clinical and imaging data have been proposed to aid both COVID-19 diagnosis and severity stratification [55,56,57,58,59], the inclusion of vascular features stems from an ever-larger corpus of observations that link vascular (particularly endothelial) impairment in COVID-19 [4,5,6,7,8,9] to pulmonary thromboembolism [11,12,13,16,17,18,19,20,23] and gross damage to pulmonary arterial vessels [25,26,27].
In a preliminary study on a limited sample size [26] we already observed that COVID-19 patients have a PA maximum diameter greater than that measured in previous CTs performed for non-cardiovascular reasons. In that study, enlarged PA diameter was also associated to death during hospitalization [26]. These observations were confirmed by the present study already through the explorative univariate analysis of pulmonary vascular features, where we found that patients who died during hospitalization had a significantly higher PA maximum diameter. Further, as shown in Figure 3, the LASSO regression coefficient for the PA maximum diameter was the third highest among investigated predictors of in-hospital death, after age and the extent of lung parenchymal involvement. It is worth noting that lung consolidation, appearing in the later stages of COVID-19 pneumonia, seems to be associated with better prognosis, as highlighted by the negative LASSO regression coefficient. This apparently counterintuitive result, already hinted at by a few studies [60,61], can be explained by considering that extensive parenchymal involvement in COVID-19 patients most frequently leads to in-hospital death, leaving no time for pulmonary damage progression to consolidation. Indeed, consolidation was present in almost 50% of patients from our cohort but was sparsely distributed and was associated with GGOs in 91% of cases, pointing out how hospital admission occurred at relatively early stages of COVID-19 pneumonia.
The prognostic role of the PA maximum diameter also shows how the early microvascular damage caused by SARS-CoV-2 infection may represent one of the most insidious aspects of severe COVID-19 presentations [10]. Notably, SARS-CoV-2-induced microthrombosis in distal branches of the PA can start the disruption of the already fragile coagulation profile of these patients [5,8,9], but can be difficult to detect even on contrast-enhanced CT scans [23]. Conversely, the PA diameter is an indirect metric of the thromboembolic profile of COVID-19 patients that can be easily obtained from unenhanced CT scans. Its integration into a reliable prognostic model further highlights how features extracted from CT scans performed without the administration of a contrast agent carry a substantial prognostic potential in COVID-19 patients, as already hinted at by other studies [62,63].
This study was performed at the early stage of the SARS-CoV-2 pandemic, when prophylactic LMWH was not yet routinely administered. This represents the main limitation of our study, preventing the investigation of the impact of appropriate treatment—targeting vascular damage—on COVID-19 patients’ prognosis. Moreover, shortages in ICU beds affecting all hospitals in northern Italy during the first pandemic peak [64] frequently restricted ICU admission to relatively younger patients without comorbidities, preventing us from assessing ICU admission predictive models, since these would have been biased by such patient selection. Of note, this logistical stress on healthcare systems also precluded the inclusion of young patients (which were prevalently treated at home), hindered data collection on admission, and likely impacted the prognosis of the included patients: all these aspects must be carefully considered when evaluating the generalizability of our results. From a technical point of view, the main limitation of our study is the lack of a longitudinal and independent test set, as we randomly sampled test patients from the original multicenter cohort. Since we aimed first to maximize the heterogeneity of the training set as much as possible, using retrospective data collected from all centers, a robust generalizability assessment will need to be conducted prospectively on an independent test set. Although commonly reported in published literature [25,37,38,39,44], another technical limitation is represented by the manual measurement of the diameters of great vessels on axial scans: a better approach could be represented by acquiring vessel area measurements on reformatted oblique reconstructions considering the vessel axis [42]. Moreover, we focused on a single feature selection technique (i.e., the LASSO) and on few machine learning models (i.e., SVMs and MLPs) to develop the in-hospital mortality predictor, with a comparably small batch size in MLPs, although supported by results of systematic hyperparameter search. Finally, the use of the mean feature value as an imputation technique may also lead to suboptimal performance. Indeed, the use of multiple imputations or machine learning based imputation techniques could improve the predictive power of developed classifiers.
In conclusion, our study shows how a predictive model integrating simple clinical and radiological data acquired on admission, such as the PA/AA ratio on unenhanced CT scans, allowed to predict in-hospital death in COVID-19 patients, highlighting the major role of pulmonary vascular involvement in the stratification of COVID-19 patients.

Supplementary Materials

The following are available online at https://www.mdpi.com/article/10.3390/jpm11060501/s1, Table S1: Input features, hyperparameters, and performance of the support vector machines developed as a result of the LASSO-importance-based backward feature selection.

Author Contributions

Conceptualization, F.S. (Francesco Sardanelli), S.S., M.C. (Marina Codari), and A.C.; methodology, S.S., A.C., M.C. (Marina Codari), G.D.L., and F.S. (Francesco Sardanelli); formal analysis, M.C. (Marina Codari), A.C., and S.S.; investigation, S.S., M.C. (Marina Codari), A.C., D.A., M.A., R.A., E.A., C.B., M.C. (Maurizio Cariati), S.C., M.C. (Massimo Cressoni), P.S.C.D., G.D.P., G.D.L., F.D., Z.F., N.F., R.A.F., S.G., G.L., V.M., A.E.M., G.M., C.M., L.M., A.P., F.P., L.M.S., F.S. (Francesco Secchi), E.S., A.S., V.T., S.T., A.V., I.V., D.Z., D.F., and F.S. (Francesco Sardanelli); resources, C.B., M.C. (Maurizio Cariati), L.M.S., A.S., A.V., D.F., and F.S. (Francesco Sardanelli); data curation, S.S., M.C. (Marina Codari), A.C., D.A., M.A., R.A., E.A., S.C., M.C. (Massimo Cressoni), P.S.C.D., G.D.P., G.D.L., F.D., Z.F., N.F., R.A.F., S.G., G.L., V.M., A.E.M., G.M., C.M., L.M., A.P., F.P., F.S. (Francesco Secchi), E.S., S.T., and D.Z.; writing—original draft preparation, S.S., M.C. (Marina Codari), and A.C.; writing—review and editing, S.S., M.C. (Marina Codari), A.C., V.M., D.F., M.A., M.C. (Massimo Cressoni), N.F., F.P., L.M.S., F.S. (Francesco Secchi), and F.S. (Francesco Sardanelli); visualization, M.C. (Marina Codari) and A.C.; supervision, F.S. (Francesco Sardanelli), D.F., C.B., M.C. (Maurizio Cariati), L.M.S., A.S., and A.V.; project administration, A.C.; funding acquisition, F.S. (Francesco Secchi). All authors have read and agreed to the published version of the manuscript.

Funding

This study was partially supported by Ricerca Corrente funding from Italian Ministry of Health to IRCCS Policlinico San Donato.

Institutional Review Board Statement

The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Ethics Committee of IRCCS Policlinico San Donato (Comitato Etico IRCCS Ospedale San Raffaele, protocol code COVID19-TCretro, protocol number 77/INT/2020, approved 5 May 2020).

Informed Consent Statement

Patient consent was waived due to the retrospective nature of the study.

Data Availability Statement

We will be able to make available deidentified individual participant data that underlie the results reported in this article, beginning 12 months and ending 24 months following article publication. We will consider requests from researchers who will provide a methodologically sound proposal and will share data needed to achieve the aims declared in the approved proposal. Proposals may be submitted up to 36 months following article publication. After 24 months the data will be available in the data warehouse of the coordinating center but without investigator support other than deposited metadata. Information regarding submitting proposals and accessing data may be obtained by email inquiry to the first/corresponding author at andrea.cozzi1@unimi.it.

Conflicts of Interest

S. Schiaffino declares to have received travel support from Bracco Imaging and to be member of speakers’ bureau for General Electric Healthcare. D. Fleischmann declares to have received a research grant from Siemens AG, to be a shareholder of Segmed, Inc., and to have ownership interests in Ischemia View, Inc. F. Sardanelli declares to have received grants from or to be a member of the speakers’ bureau/advisory board for Bayer Healthcare, Bracco Group, and General Electric Healthcare. All other authors declare that they have no conflicts of interest and that they have nothing to disclose.

References

  1. Akl, E.A.; Blažić, I.; Yaacoub, S.; Frija, G.; Chou, R.; Appiah, J.A.; Fatehi, M.; Flor, N.; Hitti, E.; Jafri, H.; et al. Use of Chest Imaging in the Diagnosis and Management of COVID-19: A WHO Rapid Advice Guide. Radiology 2021, 298, E63–E69. [Google Scholar] [CrossRef]
  2. Rubin, G.D.; Ryerson, C.J.; Haramati, L.B.; Sverzellati, N.; Kanne, J.P.; Raoof, S.; Schluger, N.W.; Volpi, A.; Yim, J.-J.; Martin, I.B.K.; et al. The Role of Chest Imaging in Patient Management during the COVID-19 Pandemic: A Multinational Consensus Statement from the Fleischner Society. Radiology 2020, 296, 172–180. [Google Scholar] [CrossRef] [Green Version]
  3. Sverzellati, N.; Milanese, G.; Milone, F.; Balbi, M.; Ledda, R.E.; Silva, M. Integrated Radiologic Algorithm for COVID-19 Pandemic. J. Thorac. Imaging 2020, 35, 228–233. [Google Scholar] [CrossRef]
  4. Tang, N.; Li, D.; Wang, X.; Sun, Z. Abnormal coagulation parameters are associated with poor prognosis in patients with novel coronavirus pneumonia. J. Thromb. Haemost. 2020, 18, 844–847. [Google Scholar] [CrossRef] [Green Version]
  5. Nadkarni, G.N.; Lala, A.; Bagiella, E.; Chang, H.L.; Moreno, P.R.; Pujadas, E.; Arvind, V.; Bose, S.; Charney, A.W.; Chen, M.D.; et al. Anticoagulation, Bleeding, Mortality, and Pathology in Hospitalized Patients with COVID-19. J. Am. Coll. Cardiol. 2020, 76, 1815–1826. [Google Scholar] [CrossRef]
  6. Fogarty, H.; Townsend, L.; Ni Cheallaigh, C.; Bergin, C.; Martin-Loeches, I.; Browne, P.; Bacon, C.L.; Gaule, R.; Gillett, A.; Byrne, M.; et al. COVID-19 coagulopathy in Caucasian patients. Br. J. Haematol. 2020, 189, 1044–1049. [Google Scholar] [CrossRef]
  7. Fauvel, C.; Weizman, O.; Trimaille, A.; Mika, D.; Pommier, T.; Pace, N.; Douair, A.; Barbin, E.; Fraix, A.; Bouchot, O.; et al. Pulmonary embolism in COVID-19 patients: A French multicentre cohort study. Eur. Heart J. 2020, 41, 3058–3068. [Google Scholar] [CrossRef]
  8. Masi, P.; Hékimian, G.; Lejeune, M.; Chommeloux, J.; Desnos, C.; Pineton De Chambrun, M.; Martin-Toutain, I.; Nieszkowska, A.; Lebreton, G.; Bréchot, N.; et al. Systemic Inflammatory Response Syndrome Is a Major Contributor to COVID-19–Associated Coagulopathy. Circulation 2020, 142, 611–614. [Google Scholar] [CrossRef]
  9. Gu, S.X.; Tyagi, T.; Jain, K.; Gu, V.W.; Lee, S.H.; Hwa, J.M.; Kwan, J.M.; Krause, D.S.; Lee, A.I.; Halene, S.; et al. Thrombocytopathy and endotheliopathy: Crucial contributors to COVID-19 thromboinflammation. Nat. Rev. Cardiol. 2021, 18, 194–209. [Google Scholar] [CrossRef]
  10. Teuwen, L.-A.; Geldhof, V.; Pasut, A.; Carmeliet, P. COVID-19: The vasculature unleashed. Nat. Rev. Immunol. 2020, 20, 389–391. [Google Scholar] [CrossRef]
  11. Lowenstein, C.J.; Solomon, S.D. Severe COVID-19 Is a Microvascular Disease. Circulation 2020, 142, 1609–1611. [Google Scholar] [CrossRef]
  12. Varga, Z.; Flammer, A.J.; Steiger, P.; Haberecker, M.; Andermatt, R.; Zinkernagel, A.S.; Mehra, M.R.; Schuepbach, R.A.; Ruschitzka, F.; Moch, H. Endothelial cell infection and endotheliitis in COVID-19. Lancet 2020, 395, 1417–1418. [Google Scholar] [CrossRef]
  13. Ackermann, M.; Verleden, S.E.; Kuehnel, M.; Haverich, A.; Welte, T.; Laenger, F.; Vanstapel, A.; Werlein, C.; Stark, H.; Tzankov, A.; et al. Pulmonary Vascular Endothelialitis, Thrombosis, and Angiogenesis in Covid-19. N. Engl. J. Med. 2020, 383, 120–128. [Google Scholar] [CrossRef]
  14. Lax, S.F.; Skok, K.; Zechner, P.; Kessler, H.H.; Kaufmann, N.; Koelblinger, C.; Vander, K.; Bargfrieder, U.; Trauner, M. Pulmonary Arterial Thrombosis in COVID-19 with Fatal Outcome. Ann. Intern. Med. 2020, 173, 350–361. [Google Scholar] [CrossRef]
  15. Carsana, L.; Sonzogni, A.; Nasr, A.; Rossi, R.S.; Pellegrinelli, A.; Zerbi, P.; Rech, R.; Colombo, R.; Antinori, S.; Corbellino, M.; et al. Pulmonary post-mortem findings in a series of COVID-19 cases from northern Italy: A two-centre descriptive study. Lancet Infect. Dis. 2020, 20, 1135–1140. [Google Scholar] [CrossRef]
  16. Léonard-Lorant, I.; Delabranche, X.; Séverac, F.; Helms, J.; Pauzet, C.; Collange, O.; Schneider, F.F.; Labani, A.; Bilbault, P.; Molière, S.; et al. Acute Pulmonary Embolism in Patients with COVID-19 at CT Angiography and Relationship to d-Dimer Levels. Radiology 2020, 296, E189–E191. [Google Scholar] [CrossRef] [Green Version]
  17. Grillet, F.; Behr, J.; Calame, P.; Aubry, S.; Delabrousse, E. Acute Pulmonary Embolism Associated with COVID-19 Pneumonia Detected with Pulmonary CT Angiography. Radiology 2020, 296, E186–E188. [Google Scholar] [CrossRef] [Green Version]
  18. Monfardini, L.; Morassi, M.; Botti, P.; Stellini, R.; Bettari, L.; Pezzotti, S.; Alì, M.; Monaco, C.G.; Magni, V.; Cozzi, A.; et al. Pulmonary thromboembolism in hospitalised COVID-19 patients at moderate to high risk by Wells score: A report from Lombardy, Italy. Br. J. Radiol. 2020, 93, 20200407. [Google Scholar] [CrossRef]
  19. Schiaffino, S.; Giacomazzi, F.; Esseridou, A.; Cozzi, A.; Carriero, S.; Mazzaccaro, D.P.; Nano, G.; Di Leo, G.; Spagnolo, P.; Sardanelli, F. Pulmonary thromboembolism in coronavirus disease 2019 patients undergoing thromboprophylaxis. Medicine 2021, 100, e24002. [Google Scholar] [CrossRef]
  20. De Cobelli, F.; Palumbo, D.; Ciceri, F.; Landoni, G.; Ruggeri, A.; Rovere-Querini, P.; D’Angelo, A.; Steidler, S.; Galli, L.; Poli, A.; et al. Pulmonary Vascular Thrombosis in COVID-19 Pneumonia. J. Cardiothorac. Vasc. Anesth. 2021. [Google Scholar] [CrossRef]
  21. Chung, M.; Bernheim, A.; Mei, X.; Zhang, N.; Huang, M.; Zeng, X.; Cui, J.; Xu, W.; Yang, Y.; Fayad, Z.A.; et al. CT Imaging Features of 2019 Novel Coronavirus (2019-nCoV). Radiology 2020, 295, 202–207. [Google Scholar] [CrossRef] [Green Version]
  22. Bernheim, A.; Mei, X.; Huang, M.; Yang, Y.; Fayad, Z.A.; Zhang, N.; Diao, K.; Lin, B.; Zhu, X.; Li, K.; et al. Chest CT Findings in Coronavirus Disease-19 (COVID-19): Relationship to Duration of Infection. Radiology 2020, 295, 685–691. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  23. Grillet, F.; Busse-Coté, A.; Calame, P.; Behr, J.; Delabrousse, E.; Aubry, S. COVID-19 pneumonia: Microvascular disease revealed on pulmonary dual-energy computed tomography angiography. Quant. Imaging Med. Surg. 2020, 10, 1852–1862. [Google Scholar] [CrossRef]
  24. Lins, M.; Vandevenne, J.; Thillai, M.; Lavon, B.R.; Lanclus, M.; Bonte, S.; Godon, R.; Kendall, I.; De Backer, J.; De Backer, W. Assessment of Small Pulmonary Blood Vessels in COVID-19 Patients Using HRCT. Acad. Radiol. 2020, 27, 1449–1455. [Google Scholar] [CrossRef]
  25. Esposito, A.; Palmisano, A.; Toselli, M.; Vignale, D.; Cereda, A.; Rancoita, P.M.V.; Leone, R.; Nicoletti, V.; Gnasso, C.; Monello, A.; et al. Chest CT-derived pulmonary artery enlargement at the admission predicts overall survival in COVID-19 patients: Insight from 1461 consecutive patients in Italy. Eur. Radiol. 2021, 31, 4031–4041. [Google Scholar] [CrossRef]
  26. Spagnolo, P.; Cozzi, A.; Foà, R.A.; Spinazzola, A.; Monfardini, L.; Bnà, C.; Alì, M.; Schiaffino, S.; Sardanelli, F. CT-derived pulmonary vascular metrics and clinical outcome in COVID-19 patients. Quant. Imaging Med. Surg. 2020, 10, 1325–1333. [Google Scholar] [CrossRef]
  27. Aoki, R.; Iwasawa, T.; Hagiwara, E.; Komatsu, S.; Utsunomiya, D.; Ogura, T. Pulmonary vascular enlargement and lesion extent on computed tomography are correlated with COVID-19 disease severity. Jpn. J. Radiol. 2021, 39, 451–458. [Google Scholar] [CrossRef] [PubMed]
  28. Eslami, V.; Abrishami, A.; Zarei, E.; Khalili, N.; Baharvand, Z.; Sanei-Taheri, M. The Association of CT-measured Cardiac Indices with Lung Involvement and Clinical Outcome in Patients with COVID-19. Acad. Radiol. 2021, 28, 8–17. [Google Scholar] [CrossRef] [PubMed]
  29. D’Andrea, A.; Scarafile, R.; Riegler, L.; Liccardo, B.; Crescibene, F.; Cocchia, R.; Bossone, E. Right Ventricular Function and Pulmonary Pressures as Independent Predictors of Survival in Patients with COVID-19 Pneumonia. JACC Cardiovasc. Imaging 2020, 13, 2467–2468. [Google Scholar] [CrossRef]
  30. Li, Y.; Li, H.; Zhu, S.; Xie, Y.; Wang, B.; He, L.; Zhang, D.; Zhang, Y.; Yuan, H.; Wu, C.; et al. Prognostic Value of Right Ventricular Longitudinal Strain in Patients With COVID-19. JACC Cardiovasc. Imaging 2020, 13, 2287–2299. [Google Scholar] [CrossRef]
  31. Fukui, M.; Cavalcante, J.L. Relation among Right Ventricular Dysfunction, Lung Damage, and Mortality in Patients With COVID-19. JACC Cardiovasc. Imaging 2020, 13, 1858–1859. [Google Scholar] [CrossRef]
  32. Saba, L.; Sverzellati, N. Is COVID Evolution Due to Occurrence of Pulmonary Vascular Thrombosis? J. Thorac. Imaging 2020, 35, 344–345. [Google Scholar] [CrossRef]
  33. Wichmann, D.; Sperhake, J.-P.; Lütgehetmann, M.; Steurer, S.; Edler, C.; Heinemann, A.; Heinrich, F.; Mushumba, H.; Kniep, I.; Schröder, A.S.; et al. Autopsy Findings and Venous Thromboembolism in Patients With COVID-19. Ann. Intern. Med. 2020, 173, 268–277. [Google Scholar] [CrossRef]
  34. Zhou, H.; Zhang, G.; Deng, X.; Jin, B.; Qiu, Q.; Yan, M.; Wang, X.; Zheng, X. Understanding the current status of patients with pulmonary hypertension during COVID-19 outbreak: A small-scale national survey from China. Pulm. Circ. 2020, 10. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  35. Rouhezamin, M.R.; Haseli, S. Diagnosing Pulmonary Thromboembolism in COVID-19: A Stepwise Clinical and Imaging Approach. Acad. Radiol. 2020, 27, 896–897. [Google Scholar] [CrossRef] [PubMed]
  36. Matsushita, S.; Matsuoka, S.; Yamashiro, T.; Fujikawa, A.; Yagihashi, K.; Kurihara, Y.; Nakajima, Y. Pulmonary arterial enlargement in patients with acute exacerbation of interstitial pneumonia. Clin. Imaging 2014, 38, 454–457. [Google Scholar] [CrossRef] [PubMed]
  37. Shen, Y.; Wan, C.; Tian, P.; Wu, Y.; Li, X.; Yang, T.; An, J.; Wang, T.; Chen, L.; Wen, F. CT-Base Pulmonary Artery Measurement in the Detection of Pulmonary Hypertension. Medicine 2014, 93, e256. [Google Scholar] [CrossRef]
  38. Yagi, M.; Taniguchi, H.; Kondoh, Y.; Ando, M.; Kimura, T.; Kataoka, K.; Furukawa, T.; Suzuki, A.; Johkoh, T.; Hasegawa, Y. CT-determined pulmonary artery to aorta ratio as a predictor of elevated pulmonary artery pressure and survival in idiopathic pulmonary fibrosis. Respirology 2017, 22, 1393–1399. [Google Scholar] [CrossRef]
  39. Terzikhan, N.; Bos, D.; Lahousse, L.; Wolff, L.; Verhamme, K.M.C.; Leening, M.J.G.; Felix, J.F.; Gall, H.; Ghofrani, H.A.; Franco, O.H.; et al. Pulmonary artery to aorta ratio and risk of all-cause mortality in the general population: The Rotterdam Study. Eur. Respir. J. 2017, 49, 1602168. [Google Scholar] [CrossRef] [Green Version]
  40. Iyer, A.S.; Wells, J.M.; Vishin, S.; Bhatt, S.P.; Wille, K.M.; Dransfield, M.T. CT Scan-Measured Pulmonary Artery to Aorta Ratio and Echocardiography for Detecting Pulmonary Hypertension in Severe COPD. Chest 2014, 145, 824–832. [Google Scholar] [CrossRef] [Green Version]
  41. Remy-Jardin, M.; Ryerson, C.J.; Schiebler, M.L.; Leung, A.N.C.; Wild, J.M.; Hoeper, M.M.; Alderson, P.O.; Goodman, L.R.; Mayo, J.; Haramati, L.B.; et al. Imaging of Pulmonary Hypertension in Adults: A Position Paper from the Fleischner Society. Radiology 2021, 298, 531–549. [Google Scholar] [CrossRef] [PubMed]
  42. Turner, V.L.; Jubran, A.; Kim, J.B.; Maret, E.; Moneghetti, K.J.; Haddad, F.; Amsallem, M.; Codari, M.; Hinostroza, V.; Mastrodicasa, D.; et al. CTA pulmonary artery enlargement in patients with severe aortic stenosis: Prognostic impact after TAVR. J. Cardiovasc. Comput. Tomogr. 2021. [Google Scholar] [CrossRef]
  43. Sardanelli, F.; Cozzi, A.; Monfardini, L.; Bnà, C.; Foà, R.A.; Spinazzola, A.; Tresoldi, S.; Cariati, M.; Secchi, F.; Schiaffino, S. Association of mediastinal lymphadenopathy with COVID-19 prognosis. Lancet Infect. Dis. 2020, 20, 1230–1231. [Google Scholar] [CrossRef]
  44. Wells, J.M.; Washko, G.R.; Han, M.K.; Abbas, N.; Nath, H.; Mamary, A.J.; Regan, E.; Bailey, W.C.; Martinez, F.J.; Westfall, E.; et al. Pulmonary Arterial Enlargement and Acute Exacerbations of COPD. N. Engl. J. Med. 2012, 367, 913–921. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  45. Tibshirani, R. Regression Shrinkage and Selection via the Lasso. J. R. Stat. Soc. Ser. B 1996, 58, 267–288. [Google Scholar] [CrossRef]
  46. Muthukrishnan, R.; Rohini, R. LASSO: A feature selection technique in predictive modeling for machine learning. In Proceedings of the 2016 IEEE International Conference on Advances in Computer Applications (ICACA), Coimbatore, India, 24 October 2016; pp. 18–20. [Google Scholar]
  47. Peduzzi, P.; Concato, J.; Feinstein, A.R.; Holford, T.R. Importance of events per independent variable in proportional hazards regression analysis II. Accuracy and precision of regression estimates. J. Clin. Epidemiol. 1995, 48, 1503–1510. [Google Scholar] [CrossRef]
  48. Peduzzi, P.; Concato, J.; Kemper, E.; Holford, T.R.; Feinstein, A.R. A simulation study of the number of events per variable in logistic regression analysis. J. Clin. Epidemiol. 1996, 49, 1373–1379. [Google Scholar] [CrossRef]
  49. Maas, A.L.; Hannun, A.Y.; Ng, A.Y. Rectifier Nonlinearities Improve Neural Network Acoustic Models. In Proceedings of the 30th International Conference on Machine Learning, Atlanta, GA, USA, 16–21 June 2013. [Google Scholar]
  50. He, K.; Zhang, X.; Ren, S.; Sun, J. Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification. In Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile, 7–13 December 2015; pp. 1026–1034. [Google Scholar]
  51. Kingma, D.P.; Ba, J. Adam: A Method for Stochastic Optimization. arXiv 2014, arXiv:1412.6980. [Google Scholar]
  52. Van Rossum, G.; Drake, F.L. Python 3 Reference Manual; CreateSpace: Scotts Valley, CA, USA, 2009. [Google Scholar]
  53. Pedregosa, F.; Varoquaux, G.; Gramfort, A.; Michel, V.; Thirion, B.; Grisel, O.; Blondel, M.; Prettenhofer, P.; Weiss, R.; Dubourg, V.; et al. Scikit-learn: Machine Learning in Python. J. Mach. Learn. Res. 2011, 12, 2825–2830. [Google Scholar]
  54. Abadi, M.; Agarwal, A.; Barham, P.; Brevdo, E.; Chen, Z.; Citro, C.; Corrado, G.S.; Davis, A.; Dean, J.; Devin, M.; et al. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. arXiv 2016, arXiv:1603.04467. [Google Scholar]
  55. Cury, R.C.; Megyeri, I.; Lindsey, T.; Macedo, R.; Batlle, J.; Kim, S.; Baker, B.; Harris, R.; Clark, R.H. Natural Language Processing and Machine Learning for Detection of Respiratory Illness by Chest CT Imaging and Tracking of COVID-19 Pandemic in the US. Radiol. Cardiothorac. Imaging 2021, 3, e200596. [Google Scholar] [CrossRef] [PubMed]
  56. Tezza, F.; Lorenzoni, G.; Azzolina, D.; Barbar, S.; Leone, L.A.C.; Gregori, D. Predicting in-Hospital Mortality of Patients with COVID-19 Using Machine Learning Techniques. J. Pers. Med. 2021, 11, 343. [Google Scholar] [CrossRef]
  57. Roberts, M.; Driggs, D.; Thorpe, M.; Gilbey, J.; Yeung, M.; Ursprung, S.; Aviles-Rivero, A.I.; Etmann, C.; McCague, C.; Beer, L.; et al. Common pitfalls and recommendations for using machine learning to detect and prognosticate for COVID-19 using chest radiographs and CT scans. Nat. Mach. Intell. 2021, 3, 199–217. [Google Scholar] [CrossRef]
  58. Driggs, D.; Selby, I.; Roberts, M.; Gkrania-Klotsas, E.; Rudd, J.H.F.; Yang, G.; Babar, J.; Sala, E.; Schönlieb, C.-B. Machine Learning for COVID-19 Diagnosis and Prognostication: Lessons for Amplifying the Signal While Reducing the Noise. Radiol. Artif. Intell. 2021, e210011. [Google Scholar] [CrossRef]
  59. Signoroni, A.; Savardi, M.; Benini, S.; Adami, N.; Leonardi, R.; Gibellini, P.; Vaccher, F.; Ravanelli, M.; Borghesi, A.; Maroldi, R.; et al. BS-Net: Learning COVID-19 pneumonia severity on a large chest X-ray dataset. Med. Image Anal. 2021, 71, 102046. [Google Scholar] [CrossRef] [PubMed]
  60. Hu, Q.; Guan, H.; Sun, Z.; Huang, L.; Chen, C.; Ai, T.; Pan, Y.; Xia, L. Early CT features and temporal lung changes in COVID-19 pneumonia in Wuhan, China. Eur. J. Radiol. 2020, 128, 109017. [Google Scholar] [CrossRef]
  61. Xiong, Y.; Sun, D.; Liu, Y.; Fan, Y.; Zhao, L.; Li, X.; Zhu, W. Clinical and High-Resolution CT Features of the COVID-19 Infection. Investig. Radiol. 2020, 55, 332–339. [Google Scholar] [CrossRef] [PubMed]
  62. Schiaffino, S.; Albano, D.; Cozzi, A.; Messina, C.; Arioli, R.; Bnà, C.; Bruno, A.; Carbonaro, L.A.; Carriero, A.; Carriero, S.; et al. CT-derived Chest Muscle Metrics for Outcome Prediction in Patients with COVID-19. Radiology 2021. [Google Scholar] [CrossRef]
  63. Pediconi, F.; Rizzo, V.; Schiaffino, S.; Cozzi, A.; Della Pepa, G.; Galati, F.; Catalano, C.; Sardanelli, F. Visceral adipose tissue area predicts intensive care unit admission in COVID-19 patients. Obes. Res. Clin. Pract. 2021, 15, 89–92. [Google Scholar] [CrossRef]
  64. Grasselli, G.; Pesenti, A.; Cecconi, M. Critical Care Utilization for the COVID-19 Outbreak in Lombardy, Italy. JAMA 2020, 323, 1545–1546. [Google Scholar] [CrossRef] [Green Version]
Jpm 11 00501 g001 550
Figure 1. Example of axial slice selected at the level of pulmonary arterial main trunk bifurcation. The solid and dashed lines identify the maximum diameters of the pulmonary artery and ascending aorta, respectively.
Figure 1. Example of axial slice selected at the level of pulmonary arterial main trunk bifurcation. The solid and dashed lines identify the maximum diameters of the pulmonary artery and ascending aorta, respectively.
Jpm 11 00501 g001
Jpm 11 00501 g002 550
Figure 2. Feature correlation matrix, the color gradient representing the Pearson’s correlation coefficient value. GGOs: ground glass opacities; PA: pulmonary artery; AA: ascending aorta.
Figure 2. Feature correlation matrix, the color gradient representing the Pearson’s correlation coefficient value. GGOs: ground glass opacities; PA: pulmonary artery; AA: ascending aorta.
Jpm 11 00501 g002
Jpm 11 00501 g003 550
Figure 3. Coefficient of the least absolute shrinkage and selection operator (LASSO) regression in training/validation data. PA: pulmonary artery; AA: ascending aorta; GGOs: ground glass opacities.
Figure 3. Coefficient of the least absolute shrinkage and selection operator (LASSO) regression in training/validation data. PA: pulmonary artery; AA: ascending aorta; GGOs: ground glass opacities.
Jpm 11 00501 g003
Jpm 11 00501 g004 550
Figure 4. Receiver operator characteristic curves of the best support vector machine (a) and multilayer perceptron (b) models.
Figure 4. Receiver operator characteristic curves of the best support vector machine (a) and multilayer perceptron (b) models.
Jpm 11 00501 g004
Table
Table 1. Center-specific technical characteristics of the CT scanners and acquisition parameters.
Table 1. Center-specific technical characteristics of the CT scanners and acquisition parameters.
CenterLocationVendorModelAcquired SlicesSlice Thickness (mm)Tube Voltage (kVp)
Azienda Ospedaliero-Universitaria Maggiore della CaritàNovaraPhilips HealthcareIngenuity Core1281120
ASST Grande Ospedale Metropolitano NiguardaMilanoSiemens HealthineersSomatom Definition Edge1281.5120
Fondazione Poliambulanza Istituto OspedalieroBresciaGeneral Electric HealthcareLightSpeed RT 16161.25120
ASST Crema—Ospedale MaggioreCremaCanonAquilion CXL641.5135
General Electric HealthcareRevolution EVO641.25120
ASST Santi Paolo e CarloMilanoGeneral Electric HealthcareLightSpeed RT 16161.25120
IRCCS Istituto Ortopedico GaleazziMilanoSiemens HealthineersSomatom Definition AS 64641.5120
Table
Table 2. Multilayer perceptron summary architecture.
Table 2. Multilayer perceptron summary architecture.
LayerNumber of Hidden UnitsTrainable Parameters
Dense_1Number of selected features (f)f × (f + 1)
Dropout_1Number of selected features (f)0
Dense_2NN × (f + 1)
Dropout_2N0
Dense_3NN × (N + 1)
Dropout_3N0
Dense_4MM × (N + 1)
Dropout_4M0
Dense_5MM × (M + 1)
Dropout_5M0
Dense_61M + 1
N: number of hidden units in the second and third layer of the multilayer perceptron; M: number of hidden units in the fourth and fifth layer of the multilayer perceptron.
Table
Table 3. Demographic, clinical, and imaging characteristics of the two sets of patients.
Table 3. Demographic, clinical, and imaging characteristics of the two sets of patients.
VariableVariable TypeOverall
(897 Patients)		Training/Validation Set
(572 Patients)		Test Set
(102 Patients)
Demographics
SexCategorial608 M/289 F389 M/183 F66 M/36 F
Age (years)Continuous66.2 (55.1–76.5)66.9 (55.9–77.0)66.7 (52.7–79.2)
Comorbidities
Cardiovascular diseasesDichotomic433 (48%)320 (56%)53 (52%)
DiabetesDichotomic151 (17%)113 (20%)19 (19%)
Oncological historyDichotomic76 (8%)52 (9%)10 (10%)
Chronic kidney insufficiencyDichotomic52 (6%)45 (8%)3 (3%)
Outcome
Deceased patientsDichotomic229 (26%)160 (28%)30 (29%)
CT findings and features
Lung parenchymaGround-glass opacitiesDichotomic681 (76%)504 (93%)90 (92%)
ConsolidationsDichotomic434 (48%)271 (50%)53 (54%)
Crazy paving patternDichotomic194 (22%)162 (30%)36 (37%)
Extent of parenchymal involvement *Discrete2 (1–3)2 (1–3)2 (1–3)
Bilateral parenchymal involvementDichotomic631 (70%)502 (93%)86 (88%)
Vascular featuresPA diameter (mm)Continuous28.0 (25.0–30.0)28.0 (25.0–30.1)28.0 (25.0–30.3)
AA diameter (mm)Continuous34.0 (32.0–37.0)35.0 (32.0–37.0)34.0 (31.0–37.4)
PA/AA ratioContinuous0.81 (0.73–0.89)0.81 (0.73–0.90)0.81 (0.74–0.89)
All descriptive statistics are reported as median and interquartile ranges or frequencies for continuous and dichotomic/discrete variables, respectively. M: males; F: females; PA: pulmonary artery; AA: ascending aorta. * Semi-quantitatively assessed from 0 to 4, according to Chung et al. [21], as follows: 0% = 0 (absent); 1–25% = 1 (minimal); 26–50% = 2 (mild); 51–75% = 3 (moderate); and over 75% = 4 (severe).
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Schiaffino, S.; Codari, M.; Cozzi, A.; Albano, D.; Alì, M.; Arioli, R.; Avola, E.; Bnà, C.; Cariati, M.; Carriero, S.; et al. Machine Learning to Predict In-Hospital Mortality in COVID-19 Patients Using Computed Tomography-Derived Pulmonary and Vascular Features. J. Pers. Med. 2021, 11, 501. https://doi.org/10.3390/jpm11060501

AMA Style

Schiaffino S, Codari M, Cozzi A, Albano D, Alì M, Arioli R, Avola E, Bnà C, Cariati M, Carriero S, et al. Machine Learning to Predict In-Hospital Mortality in COVID-19 Patients Using Computed Tomography-Derived Pulmonary and Vascular Features. Journal of Personalized Medicine. 2021; 11(6):501. https://doi.org/10.3390/jpm11060501

Chicago/Turabian Style

Schiaffino, Simone, Marina Codari, Andrea Cozzi, Domenico Albano, Marco Alì, Roberto Arioli, Emanuele Avola, Claudio Bnà, Maurizio Cariati, Serena Carriero, and et al. 2021. "Machine Learning to Predict In-Hospital Mortality in COVID-19 Patients Using Computed Tomography-Derived Pulmonary and Vascular Features" Journal of Personalized Medicine 11, no. 6: 501. https://doi.org/10.3390/jpm11060501

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop