Next Article in Journal
Whipple Disease Presenting as Isolated Transverse Myelitis with Permanent Neurological Damage in a Patient with Systemic Lupus Erythematosus: A Case Report of a Difficult Diagnosis with a Literature Review
Previous Article in Journal
An Aminoglycoside-Sparing Regimen with Double Beta-Lactam to Successfully Treat Granulicatella adiacens Prosthetic Aortic Valve Endocarditis—Time to Change Paradigm?
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Brief Report

A Vulnerability Index to Assess the Risk of SARS-CoV-2-Related Hospitalization/Death: Urgent Need for an Update after Diffusion of Anti-COVID Vaccines

Health Search, Italian College of General Practitioners and Primary Care, 50142 Florence, Italy
Hygiene Unit, San Martino Policlinico Hospital-IRCCS for Oncology and Neurosciences, 16132 Genoa, Italy
Genomedics S.r.l., 50141 Firenze, Italy
Italian College of General Practitioners and Primary Care, 50142 Florence, Italy
Department of Health Sciences, University of Genoa, 16132 Genoa, Italy
Author to whom correspondence should be addressed.
Infect. Dis. Rep. 2024, 16(2), 260-268;
Submission received: 15 November 2023 / Revised: 11 March 2024 / Accepted: 12 March 2024 / Published: 15 March 2024


Background: There are algorithms to predict the risk of SARS-CoV-2-related complications. Given the spread of anti-COVID vaccination, which sensibly modified the burden of risk of the infection, these tools need to be re-calibrated. Therefore, we updated our vulnerability index, namely, the Health Search (HS)-CoVulnerabiltyIndex (VI)d (HS-CoVId), to predict the risk of SARS-CoV-2-related hospitalization/death in the primary care setting. Methods: We formed a cohort of individuals aged ≥15 years and diagnosed with COVID-19 between 1 January and 31 December 2021 in the HSD. The date of COVID-19 diagnosis was the study index date. These patients were eligible if they had received an anti-COVID vaccine at least 15 days before the index date. Patients were followed up from the index date until one of the following events, whichever came first: COVID-19-related hospitalization/death (event date), end of registration with their GPs, and end of the study period (31 December 2022). To calculate the incidence rate of COVID-19-related hospitalization/death, a patient-specific score was derived through linear combination of the coefficients stemming from a multivariate Cox regression model. Its prediction performance was evaluated by obtaining explained variation, discrimination, and calibration measures. Results: We identified 2192 patients who had received an anti-COVID vaccine from 1 January to 31 December 2021. With this cohort, we re-calibrated the HS-CoVId by calculating optimism-corrected pseudo-R2, AUC, and calibration slope. The final model reported a good predictive performance by explaining 58% (95% CI: 48–71%) of variation in the occurrence of hospitalizations/deaths, the AUC was 83 (95% CI: 77–93%), and the calibration slope did not reject the equivalence hypothesis (p-value = 0.904). Conclusions: Two versions of HS-CoVId need to be differentially adopted to assess the risk of COVID-19-related complications among vaccinated and unvaccinated subjects. Therefore, this functionality should be operationalized in related patient- and population-based informatic tools intended for general practitioners.

1. Background

To categorize the risk of clinical progression of SARS-CoV-2 infection and to assess the risk of COVID-19-related complications, we recently developed and validated a score (i.e., hospitalization/death), namely, the Health Search (HS)-CoVulnerabiltyIndex (VI)d [1]. It combines regulatory recommendations [2] with other clinical and demographic information registered in electronic health records (EHRs). Such a tool, which fairly predicts (pseudo-R2 = 60%; AUC = 80%; and slope (1.01; p < 0.770) and intercept (0.12; p < 0.169) calibration) COVID-19-related complications for a 30-month event horizon, empowers General Practitioners (GPs) to prioritize the most susceptible patients to receive the initial anti-COVID vaccine or any subsequent booster doses. Before the development of the HS-CoVId, similar tools were available in Italy and other countries, but they were intended for use in hospital/specialist settings and/or tailored to clinicians with a particular expertise [3,4,5]. On the other hand, there was no similar score for primary care physicians, whose patients’ datasets are certainly characterized by greater dimensions and major heterogeneity.
This tool, once implemented in software intended for GPs, besides being country-specific must be compliant with local regulatory and policy decisions. The HS-CoVId is mainly based on these criteria along with other clinical information embedded in EHRs. Prior COVID-19 infections and profession-related categories are tailoring factors which have to be accounted for by GPs as well. It is therefore crucial for GPs to verify the placement of their individual patients in the related risk stratum and compare them with other patients classified into different subgroups, thus allowing the physicians to prioritize certain treatment and hospitalization in positive patients and vaccination in those at high-risk for respiratory complications but who are still uninfected.
For these reasons, each prediction score needs to be regularly updated, given the overtime changes of biological, therapeutic, and environmental factors which could lead to algorithm mis-calibration [6]. The speed of this update strictly depends on disease evolution which, in the case of SARS-CoV-2, as with other infectious diseases, might even be performed on a daily basis. Indeed, the HS-CoVId was based on data stemming from pre-vaccine era, and the sensible reduction in hospitalization rates and fatal events among vaccinees [7,8] made us to question whether the score was currently mis-calibrated for risk prediction. As a matter of fact, when we attempted to apply the individual score to those who were immunized with an anti-COVID vaccine in 2021 (i.e., the Italian GPs were actively involved in the vaccination campaign by administering vaccines in their own clinics and/or in vaccination hubs), its predictive ability sensibly differed between vaccinated and unvaccinated subjects. As shown in Figure 1, while the HS-CoVId still performs well in unvaccinated subjects, there is a clear overestimation of risk among immunized subjects. This evidence is largely consistent with findings on the effectiveness of anti-COVID vaccines, along with available studies on their favorable safety profiles [9].
As in several countries, GPs have a key role in influenza vaccination campaigns, which has been recently “re-defined” as influenza-anti-COVID campaigns by the healthcare authorities [8]. They are indeed providing GPs with further doses of anti-COVID vaccines for older and at-risk subjects to be co-administered with flu vaccination. The HS-CoVId re-calibration was therefore needed to provide GPs with a reliable tool to prioritize anti-COVID vaccines and/or its booster doses.

2. Methods

2.1. Data Source

The Health Search Database (HSD) is a comprehensive, longitudinal database that contains electronic health records (EHRs) of roughly 1 million adults. The HSD was established in 1996 by the Italian College of General Practitioners and Primary Care, with the aim to conduct clinical research in the primary care setting. This database includes demographic and clinical data, which are linked through a unique encrypted code that tracks various aspects of a patient’s health, such as drug prescriptions, lifestyle factors, clinical investigations, hospitalizations, and deaths. The diagnoses and prescribed medications are coded using the International Classification of Diseases, 9th Revision, Clinical Modification (ICD-9-CM) and Anatomical Therapeutic Chemical (ATC) systems, respectively. The other variables are registered using regional coding systems. The study included 800 general practitioners who met the quality criteria and served approximately 1.2 million patients. These GPs, homogenously distributed across Italy, attended specific courses for data entry according to the HSD methodology for data collection. The HSD is currently listed in the EMA RWD official catalog [10] and it has been previously adopted for epidemiological research including the development and validation of prediction scores [1,11,12].

2.2. Study Design and Data Analyses

To re-calibrate the HS-CoVId, we adopted the same methodology leading to the previous version of the score, which is described elsewhere [1]. This analysis was compliant with the TRIPOD statements [13]. In brief, we formed a cohort of individuals aged ≥15 years and diagnosed with COVID-19 (ICD-9-CM: 460/30; 480.9/60) by any laboratory method or clinically between 1 January and 31 December 2021 in the HSD. The date of COVID-19 diagnosis was the study index date. These subjects were eligible if they had received an anti-COVID vaccine at least 15 days before the index date. Those younger than 15 years old and with no registered anti-COVID vaccination in the 15 days preceding COVID-19 diagnosis were excluded. The selected individuals were monitored from the index date until the earliest of the following events occurred: hospitalization or death related to COVID-19 (event date), discontinuation of their registration with their GPs, or the end of the study period on 31 December 2022. For the purpose of this study, COVID-19-related hospitalizations were defined as those in which the terms “SARS-CoV-2”, “COVID*”, or “coron*” were reported in the code description field and the hospitalization took place in departments such as “intensive”, “respiratory”, or “emergency”. Fatal cases were those occurring within 30 days of the diagnosis of SARS-CoV-2 infection. In line with the prior study, each record was then manually reviewed by an expert clinician to verify the veracity of the event definition [1]. All available demographic and clinical determinants forming the score stemmed from systematic evidence [14,15] and official documents issued by the Italian Health Authorities [8]; every covariate is operationally described in the primary study on the creation of the HS-CoVId [1].
Descriptive statistics were presented in the form of means with standard deviations (SDs) and proportions with 95% confidence intervals (CIs) for continuous and categorical variables, respectively. The incidence rate of COVID-19-related hospitalizations and deaths was calculated by dividing the number of events by the person-months accumulated during the follow-up period. A multivariate Cox regression model was used to estimate the regression coefficients for each covariate, with the effect size expressed as an adjusted hazard ratio (aHR) along with a 95% CI. A patient-specific score was derived by combining these coefficients linearly, resulting in the HS-CoVId score. The 30-day predicted risk of experiencing the study outcomes was calculated as a function of the cumulative baseline hazard and the linear predictor, which is the sum of the product of the predictor values for the individual patient and the beta coefficients for each risk factor. To validate the score, we calculated pseudo-R2 and AUC as overall performance and discrimination measures, respectively. We also calibrated the score by comparing the predicted vs. observed risks of COVID-19-related hospitalizations and deaths. We provided optimism-corrected pseudo-R2, AUC, and calibration slope by bootstrapping 200 random samples from the study cohort [6,16]. To make the score more easily interpretable and applicable for general practitioners, we categorized it into different subgroups, including a low/intermediate-risk category and a high-risk category, using cut-off points based on the predicted risk of COVID-19-related complications as determined by Cox’s methods [14].
Given the reduced size of the study cohort, we conducted a secondary analysis to verify the robustness of the results. We ran the primary model again in the entire cohort of vaccinees (N = 124,320) irrespective of the COVID-19 diagnosis. By doing so, we were able to identify more cases of hospitalizations/deaths, thereby ensuring the analysis power among immunized patients. For this analysis, we re-calculated pseudo-R2 and AUC as performance and discrimination measures, respectively. In addition, we determined the sensitivity and specificity with a precise level of uncertainty, utilizing our increased analytical power. The calibration slope and optimism-corrected measures were re-calculated as well [6,16]. We adopted the same selection criteria to form another validation cohort using the 2023 update (i.e., up to June 2023) of the HSD. By doing so, a temporal validation of the algorithm was performed as a proxy of the external validation test [6].
The scientific Internal Review Board of the Italian College of General Practitioners and Primary Care approved this study.

3. Results

A total of 2192 patients were included in the study, with 51.7% of them being female and a mean age of 52.3 years (SD 18.4). During the follow-up period, 57 cases of COVID-19-related hospitalizations and/or deaths were recorded, resulting in an overall incidence rate of 1.4 per 100 person-years (95% CI: 1.1–1.8). The Cox model revealed coefficients related to age, sex, and clinical characteristics that pertained to highly vulnerable and/or severely disabled patients, as per regulatory indications, and/or those with other vulnerabilities, such as other risk factors like diabetes with no complications and hypertension, coded in the EHRs. Among vaccinees, they were linearly combined to form the HS-CoVId, which was then categorized in deciles in order to evaluate its prediction accuracy.
The model explained 62% (95% CI: 55–78%) of variance in the occurrence of hospitalizations/deaths; the discrimination accuracy was higher than 80% (AUC 82% (95% CI: 75–79%)); in terms of calibration, over a 30-day follow-up, both calibration intercept (0.3 (95% CI: −0.25–0.83)) and slope (1.00 (95% CI: 0.81–1.20)) did not reject the equivalence hypothesis (p-value equal to 0.951 and 0.286, respectively). When the explained variance, discrimination and calibration measures, and related 95% CI were calculated after bootstrapping 200 samples, the results were consistent with those reported above (Table 1). Along this line, the secondary analysis on the entire cohort of 124,320 vaccinees with 2123 events (incidence rate equal to 1.9 (95% CI: 1.8–2.0) person-years) provided results which were largely consistent with those described above. The model explained 48% (95% CI: 46–50%) of variance in the occurrence of hospitalizations/deaths; the discrimination accuracy was higher than 80% (AUC 88% (95% CI: 87–90%)); in terms of calibration, over a 30-day follow-up, the calibration slope (1.00 (95% CI: 1.00–1.04)) did not reject the equivalence hypothesis (p-value equal to 0.436). Similar findings were obtained for the optimism-corrected estimates after bootstrapping (Table 2).
According to the Cox methodology, we identified low/moderate-risk (n = 90,743 (73%)) and high-risk (n = 33,577 (27%)) categories, with a sensitivity and specificity of 92% (95% CI: 88–95%) and 70% (95% CI: 69–70%), respectively. In the first semester of 2023 (N = 126,161; mean age: 61 (SD: 19.1), 54.1% female), 35,269 (28.0%) and 90,892 (72%) patients were classified as low/moderate and high risk, respectively. The related sensitivity, specificity, pseudo-R2, and AUC were 68% (95% CI: 68–70%), 87% (95% CI: 86–89%), 90.0% (95% CI: 86.2–93%), and 67.6% (95% CI: 67.3–68.0%), respectively.

4. Discussion

With this analysis, we updated the HS-CoVId which was previously developed and validated in 2020 during the pre-vaccine era. The score was tested for its prediction accuracy among patients who had received an anti-COVID vaccine in 2021, and all validation measures were consistent with those calculated with the primary version of the score [1]. As such, these findings confirm the need for a score and its regular update to predict the risk of COVID-19-related complications in primary care. Indeed, while similar evidence-based tools were available in the medical literature, they were based on in-hospital settings [3,4] or were not trained in actual COVID-19 infections identified early by GPs [5]. This update, in addition to providing further evidence of anti-COVID vaccine effectiveness (i.e., observed out of predicted risks of COVID-19-related hospitalization/death were sensibly reduced) [17,18], shows that HS-CoVId-based estimation of risk is currently able to differentiate between vaccinated and unvaccinated subjects.
Three other prognostic scores for severe respiratory failure have been developed in Italy [3,4,19]. The PREDI-CO score, developed on 1113 in-patients, demonstrated excellent discrimination ability (AUC 0.89) in predicting severe respiratory failure by combining clinical information and laboratory findings [4]. The Brixia score, designed for experienced radiologists, showed good agreement (κ 0.82) between chest X-ray findings and severe in-hospital progression of COVID-19 [19]. Cecconi et al. [3] provided a reliable (C 0.845) tool to predict the risk of clinical deterioration in hospitalized patients using respiratory rate, blood gas parameters, history of coronary heart disease, C-reactive protein, and serum creatinine levels.
Nevertheless, these scores did not meet general practice’s needs, given the required determinants to retrieve an estimation along with the wider clinical heterogeneity of patients being cared for by GPs. In this context, practical implementations of our results in general practice are even more crucial. In GPs’ informatic tools, the score might be automatically provided by combining demographic and clinical features according to the pre-established programmed criteria. The HS-CoVId would allow the view of two different individualized scores whether anti-COVID-19 vaccination is present or not. Specifically, two main implementations of the updated version of the HS-CoVId should be considered. First, the HS-CoVId might be operationalized into a Clinical Decision Support System (CDSS), with which GPs can visualize a patient-based dashboard indicating the individual risk. This approach can easily be implemented in primary care settings, as electronic medical charts are currently widely available and mandatory for GPs to use when providing care (D.M. 4 March 2009 [G.U. n. 146 del 26 June 2009]; DPCM March 26, 2008 [G.U. 28 May 2008, n. 124]). Additionally, according to a population-based approach, GPs can create a list of “high-risk” patients for whom they can plan immunization schedules and/or explore other treatment options. For example, GPs may choose to contact patients who do not schedule vaccination appointments (primary or booster injections) spontaneously based on their individual HS-CoVId score.
In Italy, this algorithm might support the GPs during the next influenza vaccination campaign in which the public health authority recommends the co-administration of influenza and COVID vaccines for those at risk (as per the combination of risk factors for progression of the infectious disease) of developing respiratory-related complications. On average, Italian GPs have 10.3 encounters with patients per year. The majority of these encounters are due to older adults, especially those aged 85 and above, who may have up to 20 or more encounters per patient/year. According to the data from the Health Search reports [20], a GP with a maximum of 1500 patients would expect to have 420 patients classified as high-risk vaccinees, requiring frequent evaluations. Considering the importance of vaccination campaigns for the National Health Service (NHS), the expected workload from these decision tools should be deemed acceptable and cost-effective for both GPs and the NHS.
The combination of patient- and population-based tools embedding the HS-CoVId might therefore effectively optimize the immunization procedures, given their proven need in clinical decision-making, especially in case of reduced availability of vaccines, delay of vaccine deliveries, vaccine hesitancy in at-risk populations, and/or issues in organization of primary care clinics and their interaction with in-hospital departments for vaccination. The population-based tool performing on the EHR-based HS-CoVId would automatically stratify patients according to their vulnerability; within those classified as highly vulnerable in case of a contagion, further staging might be obtained for those aged 60 years or older, on their level of frailty levels as assessed by our Primary Care Frailty Index [21].
Furthermore, such a system would sustain GPs to participate in networks and create a real-time epidemiological observatory. Through these data, GPs and public health authorities would be able to plan, monitor, and implement prevention strategies, given the absence of epidemiologic observatory for most vaccinations in adults.
This study has some limitations. First, the fact that the multivariate model was based on a few cases of COVID-19-related hospitalizations/deaths (n = 57) provided some unstable estimates. For this reason, we validated the score using bootstrapping instead of cohort split-samples [22,23], as well as performed a secondary analysis on the entire cohort of vaccinees irrespective of the COVID-19 diagnosis. Reassuringly, the results stemming from bootstrapping and the secondary analysis were always consistent with those reported for the primary analysis. Second, we were unable to validate the score using an external (independent) population. Reassuringly, the prediction accuracy of the score was largely consistent with its previous version, which was externally validated [1]; similar findings were gathered when the algorithm was applied to the first semester of 2023 on a temporal validation cohort. That being said, when these algorithms are developed through a representative data source with the aim to apply this score in the same setting, the internal validity is sufficient [24]. In this respect, the HS-CoVId might be adopted in similar settings (e.g., countries with similar primary care role), but its re-calibration (i.e., external validation) would be necessary to demonstrate the prediction accuracy in these different populations. Third, given that vaccination hubs were largely involved in the first phase of the vaccination campaign, GPs could have under-registered some injections, thus increasing the presence of false-negatives for vaccinees. Reassuringly, the level of sensitivity of vaccine exposure was sufficient to obtain different calibration among vaccinees.
In Italy, GPs are actively included in the anti-COVID vaccination campaign and will soon cover further key roles to fight the pandemic and its current transition to an endemic phase. The use of anti-COVID vaccines is moving toward a personalized approach [25] with major focus and prioritization on most vulnerable and/or frail patients. The use of vaccines, and their updated versions against new variants, should be therefore tailored on individual subjects. To do so, GPs need to be equipped with regularly updated evidence-based scoring systems embedded in their informatic tools.

Author Contributions

F.L. and E.M. conceived and designed the research and performed statistical and other analyses and drafted the manuscript. I.C., I.G., A.R., A.D., G.I. and C.C. drafted and critically revised the manuscript. All authors have read and agreed to the published version of the manuscript.


This study was supported by an unconditional grant from Moderna. The corresponding author had full access to all the data in the study and had final responsibility for the decision to submit it for publication.

Institutional Review Board Statement

This study adhered to the principles of research ethics, which included compliance with the Declaration of Helsinki and the TRIPOD statements ( (accessed on 30 April 2023)). The study protocol was approved by the scientific Internal Review Board of the Italian College of General Practitioners and Primary Care. The retrospective nature of the study and the type of data were such that no approval from an ethics committee was required.

Informed Consent Statement

Not applicable.

Data Availability Statement

Data are contained within the article.

Conflicts of Interest

The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: F.L., E.M. and I.C. provided consultancies in protocol preparation for epidemiological studies and data analyses for Moderna, AstraZeneca, and Pfizer; and I.G., A.R. and C.C. provided clinical consultancies for Moderna, AstraZeneca, and Pfizer. A.D. was previously employed by Seqirus and received consultancy and/or speaker fees from Seqirus and SD Biosensor. G.I. provided consultancies and/or received grants for conducting experimental and/or observational studies for GSK, Sanofi Pasteur, MSD, and Pfizer.


  1. Lapi, F.; Domnich, A.; Marconi, E.; Rossi, A.; Grattagliano, I.; Lagolio, E.; Medea, G.; Sessa, A.; Cricelli, I.; Icardi, G.; et al. Predicting the risk of severe COVID-19 outcomes in primary care: Development and validation of a vulnerability index for equitable allocation of effective vaccines. Expert Rev. Vaccines 2021, 21, 377–384. [Google Scholar] [CrossRef] [PubMed]
  2. Season IM of HP and C of IR for. Available online: (accessed on 2 March 2023).
  3. Cecconi, M.; Piovani, D.; Brunetta, E.; Aghemo, A.; Greco, M.; Ciccarelli, M.; Angelini, C.; Voza, A.; Omodei, P.; Vespa, E.; et al. Early Predictors of Clinical Deterioration in a Cohort of 239 Patients Hospitalized for Covid-19 Infection in Lombardy, Italy. J. Clin. Med. 2020, 9, 1548. [Google Scholar] [CrossRef] [PubMed]
  4. Bartoletti, M.; Giannella, M.; Scudeller, L.; Tedeschi, S.; Rinaldi, M.; Bussini, L.; Fornaro, G.; Pascale, R.; Pancaldi, L.; Pasquini, Z.; et al. Development and validation of a prediction model for severe respiratory failure in hospitalized patients with SARS-CoV-2 infection: A multicentre cohort study (PREDI-CO study). Clin. Microbiol. Infect. 2020, 26, 1545–1553. [Google Scholar] [CrossRef] [PubMed]
  5. DeCaprio, D.; Gartner, J.; Burgess, T.; Garcia, K.; Kothari, S.; Sayed, S.; McCall, C.J. Building a COVID-19 vulnerability index. J. Med. Artif. Intell. 2020, 3. Available online: (accessed on 20 June 2022). [CrossRef]
  6. Steyerberg, E.W.; Vickers, A.J.; Cook, N.R.; Gerds, T.; Gonen, M.; Obuchowski, N.; Pencina, M.J.; Kattan, M.W. Assessing the performance of prediction models: A framework for traditional and novel measures. Epidemiology 2010, 21, 128–138. [Google Scholar] [CrossRef] [PubMed]
  7. Cunningham, A.L.; McIntyre, P.; Subbarao, K.; Booy, R.; Levin, M.J. Vaccines for older adults. BMJ 2021, 372, n188. [Google Scholar] [CrossRef] [PubMed]
  8. Ministry of Health. Aggiorn delle Misure di Prev della Trasm di SARS-CoV-2. Available online: (accessed on 20 April 2021).
  9. Safety of COVID-19 Vaccines|European Medicines Agency. Available online: (accessed on 24 January 2024).
  10. EMA. EMA RWD Catalogue. Available online: (accessed on 3 March 2024).
  11. Dentali, F.; Fontanella, A.; Cohen, A.T.; Simonetti, M.; Parretti, L.; Marconi, E.; Parretti, D.; Gussoni, G.; Campanini, M.; Agnelli, G.; et al. Derivation and Validation of a Prediction Model for Venous Thromboembolism in Primary Care. Thromb. Haemost. 2020, 120, 692–701. [Google Scholar] [CrossRef] [PubMed]
  12. Grande, G.; Vetrano, D.L.; Mazzoleni, F.; Lovato, V.; Pata, M.; Cricelli, C.; Lapi, F. Detection and Prediction of Incident Alzheimer Dementia over a 10-Year or Longer Medical History: A Population-Based Study in Primary Care. Dement. Geriatr. Cogn. Disord. 2020, 49, 384–389. [Google Scholar] [CrossRef] [PubMed]
  13. Moons, K.G.M.; Altman, D.G.; Reitsma, J.B.; Ioannidis, J.P.A.; Macaskill, P.; Steyerberg, E.W.; Vickers, A.J.; Ransohoff, D.F.; Collins, G.S. Transparent Reporting of a multivariable prediction model for Individual Prognosis or Disagnosis (TRIPOD): Explanantion and Elaboration. Ann. Intern. Med. 2015, 162, W1–W73. [Google Scholar] [CrossRef] [PubMed]
  14. Parohan, M.; Yaghoubi, S.; Seraji, A.; Javanbakht, M.H.; Sarraf, P.; Djalali, M. Risk factors for mortality in patients with Coronavirus disease 2019 (COVID-19) infection: A systematic review and meta-analysis of observational studies. Aging Male 2021, 23, 1416–1424. [Google Scholar] [CrossRef] [PubMed]
  15. Huang, Y.; Li, W.; Macheret, F.; Gabriel, R.A.; Ohno-Machado, L. A tutorial on calibration measurements and calibration models for clinical prediction models. J. Am. Med. Inform. Assoc. 2020, 27, 621–633. [Google Scholar] [CrossRef] [PubMed]
  16. Steyerberg, E.W.; Vergouwe, Y. Towards better clinical prediction models: Seven steps for development and an ABCD for validation. Eur. Heart J. 2014, 35, 1925–1931. [Google Scholar] [CrossRef] [PubMed]
  17. El Sahly, H.M.; Baden, L.R.; Essink, B.; Doblecki-Lewis, S.; Martin, J.M.; Anderson, E.J.; Campbell, T.B.; Clark, J.; Jackson, L.A.; Fichtenbaum, C.J.; et al. Efficacy of the mRNA-1273 SARS-CoV-2 Vaccine at Completion of Blinded Phase. N. Engl. J. Med. 2021, 385, 1774–1785. [Google Scholar] [CrossRef] [PubMed]
  18. Baden, L.R.; El Sahly, H.M.; Essink, B.; Kotloff, K.; Frey, S.; Novak, R.; Diemert, D.; Spector, S.A.; Rouphael, N.; Creech, C.B.; et al. Efficacy and Safety of the mRNA-1273 SARS-CoV-2 Vaccine. N. Engl. J. Med. 2021, 384, 403–416. [Google Scholar] [CrossRef] [PubMed]
  19. Borghesi, A.; Maroldi, R. COVID-19 outbreak in Italy: Experimental chest X-ray scoring system for quantifying and monitoring disease progression. Radiol. Medica 2020, 125, 509–513. [Google Scholar] [CrossRef] [PubMed]
  20. Available online: (accessed on 30 June 2023).
  21. Vetrano, D.L.; Zucchelli, A.; Onder, G.; Fratiglioni, L.; Calderón-Larrañaga, A.; Marengoni, A.; Marconi, E.; Cricelli, I.; Aprile, P.L.; Bernabei, R.; et al. Frailty detection among primary care older patients through the Primary Care Frailty Index (PC-FI). Sci. Rep. 2023, 13, 3543. [Google Scholar] [CrossRef] [PubMed]
  22. Debray, T.P.A.; Vergouwe, Y.; Koffijberg, H.; Nieboer, D.; Steyerberg, E.W.; Moons, K.G.M. A new framework to enhance the interpretation of external validation studies of clinical prediction models. J. Clin. Epidemiol. 2015, 68, 279–289. [Google Scholar] [CrossRef] [PubMed]
  23. Wynants, L.; Van Calster, B.; Collins, G.S.; Riley, R.D.; Heinze, G.; Schuit, E.; Albu, E.; Arshi, B.; Bellou, V.; Bonten, M.M.J.; et al. Prediction models for diagnosis and prognosis of COVID-19: Systematic review and critical appraisal. BMJ 2020, 369, m1328. [Google Scholar] [CrossRef] [PubMed]
  24. Ramspek, C.L.; Jager, K.J.; Dekker, F.W.; Zoccali, C.; van Diepen, M. External validation of prognostic models: What, why, how, when and where? Clin. Kidney J. 2021, 14, 49–58. [Google Scholar] [CrossRef] [PubMed]
  25. Valdés-Fernández, B.N.; Duconge, J.; Espino, A.M.; Ruaño, G. Personalized health and the coronavirus vaccines—Do individual genetics matter? Bioessays 2021, 43, 2100087. [Google Scholar] [CrossRef] [PubMed]
Figure 1. Calibration plot showing expected vs. observed risk of COVID-19-related hospitalization/death over 1-month follow-up, according to the HS-CoVId in vaccinated (A) and unvaccinated (B) subjects. LOWESS (Locally Weighted Scatterplot Smoothing).
Figure 1. Calibration plot showing expected vs. observed risk of COVID-19-related hospitalization/death over 1-month follow-up, according to the HS-CoVId in vaccinated (A) and unvaccinated (B) subjects. LOWESS (Locally Weighted Scatterplot Smoothing).
Idr 16 00021 g001
Table 1. Measurements of prediction accuracy for the HS-CoVId with a 30-day follow-up for the overall cohort and after bootstrapping.
Table 1. Measurements of prediction accuracy for the HS-CoVId with a 30-day follow-up for the overall cohort and after bootstrapping.
Overall Cohort
(N = 2195)
(n = 200 Samples)
Explained variation
Pseudo-R20.619 (0.555–0.767)0.575 (0.474–0.711)
AUC0.823 (0.751–0.893)0.829 (0.770–0.930)
Slope1.000 (0.805–1.195)1.010 (0.800–1.179)
Intercept0.293 (−0.246–0.832)-
Table 2. Measurements of prediction accuracy for the HS-CoVId with a 30-day follow-up for the cohort of vaccinees and after bootstrapping.
Table 2. Measurements of prediction accuracy for the HS-CoVId with a 30-day follow-up for the cohort of vaccinees and after bootstrapping.
Overall Cohort
(N = 124,320)
(n = 200 Samples)
Explained variation
Pseudo-R20.478 (0.460–0.504)0.467 (0.445–0.492)
AUC0.883 (0.869–0.897)0.889 (0.874–0.905)
Slope1.000 (0.962–1.038)0.996 (0.948–1.040)
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Lapi, F.; Marconi, E.; Domnich, A.; Cricelli, I.; Rossi, A.; Grattagliano, I.; Icardi, G.; Cricelli, C. A Vulnerability Index to Assess the Risk of SARS-CoV-2-Related Hospitalization/Death: Urgent Need for an Update after Diffusion of Anti-COVID Vaccines. Infect. Dis. Rep. 2024, 16, 260-268.

AMA Style

Lapi F, Marconi E, Domnich A, Cricelli I, Rossi A, Grattagliano I, Icardi G, Cricelli C. A Vulnerability Index to Assess the Risk of SARS-CoV-2-Related Hospitalization/Death: Urgent Need for an Update after Diffusion of Anti-COVID Vaccines. Infectious Disease Reports. 2024; 16(2):260-268.

Chicago/Turabian Style

Lapi, Francesco, Ettore Marconi, Alexander Domnich, Iacopo Cricelli, Alessandro Rossi, Ignazio Grattagliano, Giancarlo Icardi, and Claudio Cricelli. 2024. "A Vulnerability Index to Assess the Risk of SARS-CoV-2-Related Hospitalization/Death: Urgent Need for an Update after Diffusion of Anti-COVID Vaccines" Infectious Disease Reports 16, no. 2: 260-268.

Article Metrics

Back to TopTop