Next Article in Journal
Bloodstream Infection Due to Coagulase-Negative Staphylococci: Impact of Species on Prevalence of Infective Endocarditis
Next Article in Special Issue
In Vitro Susceptibility of Aztreonam-Vaborbactam, Aztreonam-Relebactam and Aztreonam-Avibactam Associations against Metallo-β-Lactamase-Producing Gram-Negative Bacteria
Previous Article in Journal
Chemical Characterization and Antifungal Activity of Blue Tansy (Tanacetum annuum) Essential Oil and Crude Extracts against Fusarium oxysporum f. sp. albedinis, an Agent Causing Bayoud Disease of Date Palm
Previous Article in Special Issue
Laboratory Surveillance, Quality Management, and Its Role in Addressing Antimicrobial Resistance in Africa: A Narrative Review
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Antibiotics
Volume 12
Issue 9
10.3390/antibiotics12091452
antibiotics-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Systematic Review

Predicting Extended-Spectrum Beta-Lactamase and Carbapenem Resistance in Enterobacteriaceae Bacteremia: A Diagnostic Model Systematic Review and Meta-Analysis

by
Tristan T. Timbrook
1,2,* and
McKenna J. Fowler
1
1
Department of Pharmacotherapy, University of Utah College of Pharmacy, Salt Lake City, UT 84112, USA
2
BioMérieux, 69280 Marcy l’Etoile, France
*
Author to whom correspondence should be addressed.
Antibiotics 2023, 12(9), 1452; https://doi.org/10.3390/antibiotics12091452
Submission received: 15 August 2023 / Revised: 12 September 2023 / Accepted: 14 September 2023 / Published: 17 September 2023
(This article belongs to the Special Issue Strategies for Combatting Multidrug-Resistant and Extensively Drug-Resistant Bacteria: Current Challenges and Future Prospects)
Browse Figures
Versions Notes

Abstract

:
Enterobacteriaceae bacteremia, particularly when associated with antimicrobial resistance, can result in increased mortality, emphasizing the need for timely effective therapy. Clinical risk prediction models are promising tools, stratifying patients based on their risk of resistance due to ESBL and carbapenemase-producing Enterobacteriaceae in bloodstream infections (BSIs) and, thereby, improving therapeutic decisions. This systematic review and meta-analysis synthesized the literature on the performance of these models. Searches of PubMed and EMBASE led to the identification of 10 relevant studies with 6106 unique patient encounters. Nine studies concerned ESBL prediction, and one focused on the prediction of carbapenemases. For the two ESBL model derivation studies, the discrimination performance showed sensitivities of 53–85% and specificities of 93–95%. Among the four ESBL model derivation and validation studies, the sensitivities were 43–88%, and the specificities were 77–99%. The sensitivity and specificity for the subsequent external validation studies were 7–37% and 88–96%, respectively. For the three external validation studies, only two models were evaluated across multiple studies, with a pooled AUROC of 65–71%, with one study omitting the sensitivity/specificity. Only two studies measured clinical utility through hypothetical therapy assessments. Given the limited evidence on their interventional application, it would be beneficial to further assess these or future models, to better understand their clinical utility and ensure their safe and impactful implementation.

1. Introduction

More than 2.8 million antibiotic-resistant infections occur in the United States each year, and more than 35,000 people die as a result [1,2]. Resistant pathogens of major global health concern on the CDC’s urgent and serious threat lists include carbapenem-resistant Enterobacteriaceae (CRE) and extended-spectrum-beta-lactamase-producing Enterobacteriaceae (ESBL-E) [1]. The use of broad-spectrum antibiotics has resulted in the development of resistance in many bacterial pathogens [3]. Thus, the use of broad-spectrum agents, such as carbapenems, is often avoided when possible [4]. This conundrum of ineffective antimicrobial medications and resistant microorganisms places a burden on patients and healthcare providers [5]. Identifying the risk factors for resistant Gram-negative infections is imperative in guiding appropriate treatment interventions and reducing the utilization of broad-spectrum antibiotics [6]. Moreover, employing these initiatives to accurately predict antimicrobial resistance and assign targeted effective treatment is crucial to the recovery of patients with resistant Gram-negative bloodstream infections (GN-BSIs) [7].
Currently, published models exist that assess clinical risk factors for predicting whether a patient has a resistant Gram-negative bloodstream infection [8]. These models often assess factors such as hospitalization, previous colonization, previous antibiotic therapy, age, catheter use, indwelling hardware, surgery, etc. [6]. The application of prediction models can support antimicrobial stewardship efforts by aiding the selection of appropriate treatments and reducing the selection pressure from antibiotics that leads to resistant pathogens [6]. The prediction of resistance in these organisms can be enhanced through evaluation of the performance of published prediction models. This could enhance the understanding of the clinical performance of these models, and their utility in healthcare institutions for effective clinical decision-making. A review of the risk factors and scoring systems used to predict ESBL-E and CRE infections was recently published [8]. However, systematic evaluations and the potential pooled performance of such tools are lacking.
Given the increasing threat from antibiotic-resistant infections, and the gaps in the literature regarding the prediction of ESBL-E and CRE bloodstream infections, this diagnostic model systematic review aims to evaluate the performance of the available models in the literature. This could inform healthcare institutions’ decision-making processes via evaluation of the integration of such models into their clinical procedures to reduce healthcare costs and improve patient outcomes. Moreover, this study could facilitate the adoption of high-performing models to further enhance effective clinical decision-making and antimicrobial stewardship efforts. This diagnostic model systematic review aims to synthesize the available literature to evaluate the performance and discriminatory capabilities of the published models used in predicting ESBL and carbapenemase-producing Enterobacteriaceae bloodstream infections.

2. Results

2.1. Study Selection

The literature search resulted in 2279 studies that met our keyword query (Figure 1). After the removal of duplicates, we searched 1811 studies, and this resulted in five studies identified for inclusion. Searching the bibliographies of the included studies identified an additional four studies for inclusion. One additional study was included from a conference proceeding. In total, data were extracted from 10 studies for analysis, with 6106 unique patient encounters [9,10,11,12,13,14,15,16,17,18].

2.2. Study and Model Characteristics

The characteristics of the studies included can be found in Table 1. The studies were primarily conducted in the United States (70%). Nearly all the studies had a retrospective cohort design (80%), with two retrospective case–control designs, and no prospective studies. The sample sizes varied, ranging from 145 to 1288 patients. The majority of studies (90%) were ESBL models with a resistance incidence ranging from 5.7% to 33.3%, and a median of 12.7%. A single study was of a CRE model, and did not report resistance incidence, as it was a case–control study. Among the 10 studies, 19 analyses of models were evaluated, with four (21.1%) being derivation-based, five (26.3%) being derivation- and internal-validation-based, and ten (52.6%) being external-validation-based.
The demographic details across the studies were heterogeneously reported. Patients predominantly had a median or mean age in the mid-60s, with the female representation ranging from 45.4% to 56.9% (Supplementary Table S1). The racial and ethnic distributions demonstrated substantial diversity, with African American individuals constituting up to 72.5% of patients in some cohorts, while, in others, White individuals comprised up to 47.2%, though the majority of the studies did not report race or ethnicity. Diabetes was consistently highlighted as a significant comorbidity, with prevalence rates as high as 40.1%. Other notable comorbidities included congestive heart failure (up to 50.2%) and end-stage renal disease (23.9%). The source of infection was consistently highest for urinary tract infections, ranging from 31.7% to 58.6%. Although the organisms evaluated varied across studies, E. coli was the dominant causative organism, observed in proportions ranging from 56% to 77.2%, followed by K. pneumoniae, with figures fluctuating between 13.5% and 40%, except for the single carbapenem-resistance-prediction study, which focused solely on K. pneumoniae.
Though the models displayed variations in the risk factors for resistance, certain factors consistently emerged across studies (Table 2). Regarding demographics, age was a predictor in two studies. Hospitalization or admission histories were also common predictors, including prior hospitalizations, recent emergency department visits, and having received care in high-burden regions or abroad. A variety of medical histories were frequently used as predictors, including ESBL-E infection or colonization, in addition to comorbidities or medical exposures, such as COPD, emphysema, ventilator dependence, and urological diseases, and the presence of chronic indwelling hardware. Procedures or interventions were predictors in two studies, and included invasive procedures and GI/GU procedures. Medication-use history was a consistent predictor with antibiotic exposure, both in terms of the duration (days or weeks), and with specific mentions of antibiotic classes, such as β-lactams and fluoroquinolones.

2.3. Results of Model Performance in Individual and Pooled Studies

Where reported, the discrimination performance based on the sensitivity and specificity of derivation, internal validation, and external validation studies for ESBL models can be found in Figure 2. The derivation performance ranged in sensitivity from 53 to 85%, and in specificity from 93 to 95%, with medians of 55% and 94%, respectively. The derivation and internal validation performance ranged from 43 to 88% for sensitivity and 77 to 99% for specificity, with medians of 43% and 90%, respectively. With the exception a single study that performed derivation and an external validation (Tumbarello; external validation sensitivity 73%, and specificity 95%), subsequent studies performing external validations had a performance that ranged from 7 to 37% for sensitivity, and from 88 to 96% for specificity, with medians of 32% and 90%, respectively. One external validation study did not report the sensitivity and specificity performances, but did report the AUROCs for the Augustine, Goodman 2016, Goodman 2019, and Tumbarello models, which were 0.71 (95% CI 0.65–0.78), 0.59 (95% CI 0.52–0.66), 0.74 (95% CI 0.67–0.81), and 0.76 (95% CI 0.69–0.82), respectively. Two models were externally evaluated in more than one study, and had a pooled discrimination performance, with AUROCs of 0.71 (95% CI 0.66 to 0.75) and 0.65 (95% CI 0.56 to 0.73) (Figure 3). Only three studies reported calibration evaluations, all reporting an adequate fit either from calibration plots, Hosmer–Lemeshow testing, or both. No studies published their model equations.

2.4. Clinical Utility

Only two studies (20%) evaluated the potential therapy impacts of model use. In Cwengros et al., using the Lee model, 10% were overtreated with ceftriaxone-susceptible isolates, and 69% were undertreated for resistant ones. Using a severity-of-illness-stratified Augustine model, Cwengros et al. reflected 12% overtreated, and 63% undertreated. Finally, using the Augustine model without severity-of-illness stratification, Cwengros et al. showed results of 11% overtreated and 71% undertreated. Similarly, Andrews et al. observed that 6% of patients with susceptible isolates were overtreated via the Lee model, and that 93% of resistant isolates were undertreated. Using the Augustine model without severity-of-illness stratification, Andrews et al. noted results of 4% overtreated and 68% undertreated. Both the study by Cwengros and that by Andrews et al. evaluated the total carbapenem consumption per 1000 patient days, with similar results.

3. Discussion

Due to the increasing burden of antimicrobial resistant infections, tools are needed to improve appropriate antimicrobial use, and achieve the antimicrobial stewardship goals of providing the right drug to the right patient at the right time. In our systematic review of the literature, we found 10 studies with 19 analyses of various models for ESBL and carbapenem resistance among Enterobacteriaceae bloodstream infections. While ESBL derivation studies noted a moderate performance, with the median sensitivity and specificity of 55% and 94%, subsequent studies with external validations reflected a lower median sensitivity and specificity of 32% and 90%, respectively. Two ESBL models with multiple external validations showed a pooled performance with AUROCs of 65% and 71%. Clinical utility evaluations were limited to two studies that reflected a low overtreatment in ceftriaxone-susceptible isolates with the use of models, but a substantial (>60%) undertreatment in ceftriaxone-resistant ones. Given these findings, while some models may limit overtreatment, there is still a critical need to refine and improve current models, to address the substantial risk of undertreatment, ensuring optimal patient care, and advancing antimicrobial stewardship goals.
Endorsed by Infectious Diseases Society of America guidelines, the local validation of risk prediction models, and thorough reporting, are paramount to understanding their performance and potential clinical utility [19]. In our review, less than half of the studies were external validations. Notably, where these validations were performed, they demonstrated an inferior performance compared to the original studies—a decrease which is often seen when models undergo external validation [20].
Our understanding of a model’s generalizability and reproducibility is facilitated by external validation [21]. Specifically, this process can clarify the model’s discrimination (the ability to correctly classify patients into risk groups) and calibration (the alignment between the observed vs. predicted risks across predicted risk ranges) [22]. A significant limitation we observed was that none of the derivation models disclosed their equations, impeding their use for external calibration assessments by frontline clinicians. These studies opted to transform the equations into risk scores for easier external application. Yet, this omission also prohibits the often necessary re-calibration of the risk model’s intercept for differences in baseline risk, particularly when considering the observed heterogeneity in ESBL rates. Given such variations in baseline risk and predictor performance, the external application of models will generally require a model rebuild, rather than reliance on risk scores.
Only three of the studies we analyzed reported calibration performance evaluations, a key metric that is crucial to understanding how prediction models perform across diverse risk levels. The nuances of the demographics and health backgrounds in these studies present a significant variable. For instance, despite differing demographics, such as the elderly veteran population, in the Madrid-Morales study, its discrimination performance was unexpectedly comparable to the Cwengros study. Such similarities in discrimination, despite potential age-related susceptibilities in predicting resistance or varying comorbidities such as COPD, suggest that, while the discrimination might align, the calibration could vary significantly, due to differences in the distribution of predictors across studies. In essence, two models might classify risk similarly (discrimination), but could differ in how accurately those risk predictions match the actual outcomes (calibration) when applied to different populations. This underlines the critical importance of calibration evaluations, which provide insight into how well predictions align with actual outcomes across different risk strata and varied predictor distributions. Given these observations, the comprehensive reporting of calibration evaluations becomes paramount for ensuring the robust application of prediction models across diverse settings and populations.
In our review, only two of the reviewed studies reported methods of sample-size consideration for their prediction model, and these indicated a planned adherence to historical minimal recommendations of 10 events per variable (EPV) for modeling sample size requirements [23]. Surprisingly, no study undertook formal sample-size calculations for prediction modeling, and one study did not meet the minimal EPV recommendations. Given the elements noted above regarding the overall methodological approach of current studies, we strongly advocate for future studies to adhere to the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) Reporting Guidelines. Additionally, it is imperative that future studies employ the available tools for sample size evaluations, and consistently evaluate and report the calibration performance of their models [24,25,26].
Clinical utility evaluations of predictive models are important in understanding the utility of a model beyond its discrimination and calibration performance; however, few studies in our review evaluated clinical utility [27]. As noted, in the two studies evaluating clinical utility, only the hypothetical undertreatment, overtreatment, and carbapenem use per 1000 patient days was evaluated. However, the collaborators of the Augustine study recently published a separate subsequent study evaluating the implementation of their risk prediction scoring on patient management [28]. For their implementation, they made the prediction score and algorithm available in print, on their internal website for providers, and through a mobile app. They noted a significant decrease in the time taken to administer appropriate antimicrobial therapy from 78 h to 46 h after the implementation of the risk prediction score (p = 0.04). This study offers a promising testament to the potential of locally derived, internally validated, and clinically implemented ESBL bloodstream infection risk scoring tools.
However, the landscape of resistance prediction is not confined to these tools alone. Of particular note, genotypic prediction of resistance from diagnostic testing has been shown to outperform some ESBL risk-prediction models [11]. There are myriad factors to weigh when selecting a tool for clinical utility and local validation, as delineated elsewhere [8]. In light of such advancements, institutions should remain adaptive, continuously evaluating a broad spectrum of tools. These evaluations should align with their unique patient demographics and institutional needs, and the evolving epidemiology of resistance [29,30].
An additional important consideration in our review is the inclusion of only a single study focusing on predicting carbapenem resistance in Enterobacteriaceae bloodstream infections. The limited representation of CRE is noteworthy for several reasons. Firstly, CRE is an extremely clinically challenging infection to manage, associated with significant morbidity and mortality [31]. Secondly, CRE detections in bloodstream infections are much more infrequent in the United States, where the majority (70%) of the models included in our systematic review were developed, possibly reflecting differences in clinical practice and research priorities versus countries with higher detection rates. Finally, the singular representation of CRE in our results may reflect a publication bias or a gap in the existing research that warrants future studies. However, despite its limited representation, the inclusion of the CRE prediction model in Enterobacteriaceae bloodstream infections serves to highlight the current state of the literature on prediction model tools to support patient management and antimicrobial stewardship, along with the need for additional research in this area.
There are several limitations to our review. The selection of studies was limited to studies evaluating risk modeling for ESBL and carbapenem resistance in Enterobacteriaceae in the bloodstream, which may limit practical applications, as it requires a positive blood culture and rapid organism identification method, to determine eligibility for the tool and provide risk calculation before automated susceptibility testing, in order to appropriately benefit from these approaches [32,33]. Models have been developed elsewhere for general resistance to empiric therapy for both sepsis and bloodstream infections, and future diagnostic prediction model systematic reviews should evaluate the performance of available models for these clinical scenarios, to inform clinical practice [34,35]. As noted previously in the discussion, quality indicators for performance, including calibration and the clinical utility of models, were missing from the majority of studies, limiting the interpretability of the performance and useability of the models [36,37,38]. Additionally, the external validation studies were limited, further challenging the evaluation of model performance. Future studies should focus on the external validation of existing models and clinical utility determinations. Similarly, only 30% of the studies took place outside of the United States, which makes the generalizability or transportability of such approaches and related performance to non-US countries unclear [39]. Finally, our study was limited to only grey literature (e.g., conference proceedings) for two leading infectious disease conferences, which may have led to the omission of relevant research from other conferences, particularly outside of infectious diseases.

4. Materials and Methods

This systematic review and meta-analysis was reported in accordance with the Transparent Reporting of Multivariable Prediction Models for Individual Prognosis or Diagnosis: Checklist for Systematic Reviews and Meta-Analyses (TRIPOD-SRMA) (see Supplementary Table S2) [40]. This study was not preregistered.

4.1. Study Selection

We performed a literature search using PubMed and EMBASE, using the time period from each database’s inception to 3 September 2023, and including English-language studies. A search strategy for keywords related to Gram-negative AMR and prediction models was employed (Supplementary Table S3). In addition to searching these databases, we hand-searched infectious disease major conference proceedings (ECCMID and IDWeek) related to our topic for the last two years, covering the period of 2022–2023, to capture early-stage data that might still be pending full publication.
The eligible studies included cohort and case–control observational studies evaluating a population of adult patients (≥18 years old) with Enterobacterales BSI and risk prediction models, including risk scores. Both the derivation and validation model types were eligible for inclusion. The outcome of the risk-prediction models was AMR including ESBL and CRE. The timing for the model included models used at the suspected onset of bloodstream infection for both community-onset and healthcare-acquired infection. The setting of the models was for use in hospital-admitted or emergency department patients. Studies with multivariable models evaluating the association of predictors with AMR, but which did not attempt to develop the model for prediction (e.g., the predictive performance of the model was not evaluated), were excluded.
Two reviewers (M.F. and T.T.T.) independently assessed the identified records using the eligibility criteria above. Duplicates were initially removed, before articles were reviewed. A first-pass review of titles and abstracts was performed, with selection disagreements resolved through consensus. A second-pass review of full-text articles was performed, with the additional step of resolving final discordances though consensus. A bibliography review of the included articles (i.e., snowballing) was performed to identify additional articles for inclusion.

4.2. Data Extraction

Extracted data from the final selection of articles were included in the review into a standardized Microsoft Excel grid, based on guidance from the CHARMS checklist [41]. The PICO framework (population, index model, comparator model, outcome[s]) information was extracted for each study, using a standardized form [42]. Additionally, the source of the data, the sample size, the number of participants with AMR, the handling of missing data, the selection of predictors, the predictive performance (the overall model fit, discrimination, and calibration including standard errors or confidence intervals) and, where present, measures of clinical utility (e.g., net benefit, decision curve analysis) were reviewed. No authors were contacted for missing data.

4.3. Statistical Analysis

The study characteristics, including the model performance, were reported with descriptive statistics, or through figures. Meta-analysis using random effects modeling was performed for models where a specific model had been externally validated [43]. All the data analyses, including the meta-analyses, were performed using R version 4.0.1 and ‘metafor’ for analysis [44].

5. Conclusions

In conclusion, our review reveals that a multitude of resistance-in-Enterobacteriaceae bloodstream prediction models exist in the literature, with a minority of studies concerning the external validation of existing models. The models showed a moderate discrimination performance, with a lower performance in external validations. A local external validation of the performance of the model discrimination and calibration is needed in order to understand the use of these models in specific settings, along with potential clinical utility evaluations versus the baseline appropriate prescription in Enterobacteriaceae bacteremia. Due to the scarcity of reports on the interventional use of these models, further clinical evaluation is required, to determine their clinical utility and safety. Evaluating these or newer models will strengthen the assessment of the efficacy of such approaches and their impact on patient outcomes, thus informing healthcare institutions when it comes to potentially improving therapy management, reducing healthcare costs, and enhancing patient outcomes in Enterobacteriaceae bloodstream infections.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/antibiotics12091452/s1, Table S1: Patient characteristics; Table S2: TRIPOD-SRMA Checklist for Reporting Systematic Reviews of Prediction Model Studies; Table S3: Keyword search.

Author Contributions

Conceptualization, T.T.T. and M.J.F.; methodology, T.T.T. and M.J.F.; formal analysis, T.T.T.; writing—original draft preparation, T.T.T.; writing—review and editing, T.T.T. and M.J.F. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Data Availability Statement

The data used for all analyses and analytic code are available from the corresponding author upon reasonable request.

Conflicts of Interest

T.T.T. is an employee and shareholder of bioMérieux, an in vitro diagnostics company.

References

  1. CDC. Antibiotic Resistance Threats in the United States; U.S. Department of Health and Human Services: Atlanta, GA, USA, 2019.
  2. van Duin, D.; Paterson, D.L. Multidrug-Resistant Bacteria in the Community: An Update. Infect. Dis. Clin. N. Am. 2020, 34, 709–722. [Google Scholar] [CrossRef]
  3. Serwecińska, L. Antimicrobials and Antibiotic-Resistant Bacteria: A Risk to the Environment and to Public Health. Water 2020, 12, 3313. [Google Scholar] [CrossRef]
  4. Cizman, M.; Plankar Srovin, T. Antibiotic consumption and resistance of gram-negative pathogens (collateral damage). GMS Infect. Dis. 2018, 6, Doc05. [Google Scholar] [CrossRef]
  5. Smith, R.A.; M’Ikanatha N, M.; Read, A.F. Antibiotic resistance: A primer and call to action. Health Commun. 2015, 30, 309–314. [Google Scholar] [CrossRef] [PubMed]
  6. Palacios-Baena, Z.R.; Giannella, M.; Manissero, D.; Rodríguez-Baño, J.; Viale, P.; Lopes, S.; Wilson, K.; McCool, R.; Longshaw, C. Risk factors for carbapenem-resistant Gram-negative bacterial infections: A systematic review. Clin. Microbiol. Infect. 2021, 27, 228–235. [Google Scholar] [CrossRef] [PubMed]
  7. Lodise, T.P.; Zhao, Q.; Fahrbach, K.; Gillard, P.J.; Martin, A. A systematic review of the association between delayed appropriate therapy and mortality among patients hospitalized with infections due to Klebsiella pneumoniae or Escherichia coli: How long is too long? BMC Infect. Dis. 2018, 18, 625. [Google Scholar] [CrossRef]
  8. Li, C.; Claeys, K.C.; Justo, J.A.; Heil, E.L. Heil. No Crystal Ball? Using Risk Factors and Scoring Systems ot Predict Extended-Spectrum Beta-Lactamase Producing Enterobacterales (ESBL-E) and Carbapenem-Resistant Enterobacterales (CRE) Infections. Curr. Infect. Dis. Rep. 2022, 24, 147–158. [Google Scholar] [CrossRef]
  9. Andrews, S.R.; Timbrook, T.T.; Fisher, M.A.; Tritle, B.J. Evaluation of a Rapid Diagnostic Test and Scoring Tools for Predicting Ceftriaxone Resistance in Enterobacterales Bloodstream Infections. In Proceedings of the IDWeek 2023, Boston, MA, USA, 11–15 October 2023. [Google Scholar]
  10. Augustine, M.R.; Testerman, T.L.; Justo, J.A.; Bookstaver, P.B.; Kohn, J.; Albrecht, H.; Al-Hasan, M.N. Clinical Risk Score for Prediction of Extended-Spectrum β-Lactamase-Producing Enterobacteriaceae in Bloodstream Isolates. Infect. Control Hosp. Epidemiol. 2017, 38, 266–272. [Google Scholar] [CrossRef]
  11. Cwengros, L.N.; Mynatt, R.P.; Timbrook, T.T.; Mitchell, R.; Salimnia, H.; Lephart, P.; Pogue, J.M. Minimizing Time to Optimal Antimicrobial Therapy for Enterobacteriaceae Bloodstream Infections: A Retrospective, Hypothetical Application of Predictive Scoring Tools vs Rapid Diagnostics Tests. Open Forum Infect. Dis. 2020, 7, ofaa278. [Google Scholar] [CrossRef]
  12. Goodman, K.E.; Lessler, J.; Cosgrove, S.E.; Harris, A.D.; Lautenbach, E.; Han, J.H.; Milstone, A.M.; Massey, C.J.; Tamma, P.D.; Antibacterial Resistance Leadership, G. A Clinical Decision Tree to Predict Whether a Bacteremic Patient Is Infected with an Extended-Spectrum β-Lactamase-Producing Organism. Clin. Infect. Dis. 2016, 63, 896–903. [Google Scholar] [CrossRef]
  13. Goodman, K.E.; Lessler, J.; Harris, A.D.; Milstone, A.M.; Tamma, P.D. A methodological comparison of risk scores versus decision trees for predicting drug-resistant infections: A case study using extended-spectrum beta-lactamase (ESBL) bacteremia. Infect. Control Hosp. Epidemiol. 2019, 40, 400–407. [Google Scholar] [CrossRef] [PubMed]
  14. Holmgren, A.; Ljung, A.; Bremell, D. An easy-to-use scoring system for predicting bacteraemia with third-generation cephalosporin-resistant Enterobacterales in a low-resistance setting. Infect. Dis. 2020, 52, 242–248. [Google Scholar] [CrossRef]
  15. Lee, C.H.; Chu, F.Y.; Hsieh, C.C.; Hong, M.Y.; Chi, C.H.; Ko, W.C.; Lee, C.C. A simple scoring algorithm predicting extended-spectrum β-lactamase producers in adults with community-onset monomicrobial Enterobacteriaceae bacteremia: Matters of frequent emergency department users. Medicine 2017, 96, e6648. [Google Scholar] [CrossRef]
  16. Madrid-Morales, J.; Sharma, A.; Reveles, K.; Velez-Mejia, C.; Hopkins, T.; Yang, L.; Walter, E.; Cadena, J. Validation of Available Extended-Spectrum-Beta-Lactamase Clinical Scoring Models in Predicting Drug Resistance in Patients with Enteric Gram-Negative Bacteremia Treated at South Texas Veterans Health Care System. Antimicrob. Agents Chemother. 2021, 65. [Google Scholar] [CrossRef] [PubMed]
  17. Tumbarello, M.; Trecarichi, E.M.; Bassetti, M.; De Rosa, F.G.; Spanu, T.; Di Meco, E.; Losito, A.R.; Parisini, A.; Pagani, N.; Cauda, R. Identifying patients harboring extended-spectrum-beta-lactamase-producing Enterobacteriaceae on hospital admission: Derivation and validation of a scoring system. Antimicrob. Agents Chemother. 2011, 55, 3485–3490. [Google Scholar] [CrossRef]
  18. Weston, G.; Jahufar, F.; Sharma, N.; Su, C.; Bellin, E.; Ostrowsky, B. Derivation of a Model to Guide Empiric Therapy for Carbapenem-Resistant Klebsiella pneumoniae Bloodstream Infection in an Endemic Area. Open Forum Infect. Dis. 2020, 7, ofaa070. [Google Scholar] [CrossRef] [PubMed]
  19. Metlay, J.P.; Waterer, G.W.; Long, A.C.; Anzueto, A.; Brozek, J.; Crothers, K.; Cooley, L.A.; Dean, N.C.; Fine, M.J.; Flanders, S.A.; et al. Diagnosis and Treatment of Adults with Community-acquired Pneumonia. An Official Clinical Practice Guideline of the American Thoracic Society and Infectious Diseases Society of America. Am. J. Respir. Crit. Care Med. 2019, 200, e45–e67. [Google Scholar] [CrossRef]
  20. Siontis, G.C.; Tzoulaki, I.; Castaldi, P.J.; Ioannidis, J.P. External validation of new risk prediction models is infrequent and reveals worse prognostic discrimination. J. Clin. Epidemiol. 2015, 68, 25–34. [Google Scholar] [CrossRef] [PubMed]
  21. Debray, T.P.; Vergouwe, Y.; Koffijberg, H.; Nieboer, D.; Steyerberg, E.W.; Moons, K.G. A new framework to enhance the interpretation of external validation studies of clinical prediction models. J. Clin. Epidemiol. 2015, 68, 279–289. [Google Scholar] [CrossRef]
  22. Ramspek, C.L.; Jager, K.J.; Dekker, F.W.; Zoccali, C.; van Diepen, M. External validation of prognostic models: What, why, how, when and where? Clin. Kidney J. 2021, 14, 49–58. [Google Scholar] [CrossRef]
  23. Peduzzi, P.; Concato, J.; Kemper, E.; Holford, T.R.; Feinstein, A.R. A simulation study of the number of events per variable in logistic regression analysis. J. Clin. Epidemiol. 1996, 49, 1373–1379. [Google Scholar] [CrossRef] [PubMed]
  24. Collins, G.S.; Reitsma, J.B.; Altman, D.G.; Moons, K.G. Transparent reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): The TRIPOD statement. J. Clin. Epidemiol. 2015, 68, 134–143. [Google Scholar] [CrossRef]
  25. Pate, A.; Riley, R.D.; Collins, G.S.; van Smeden, M.; Van Calster, B.; Ensor, J.; Martin, G.P. Minimum sample size for developing a multivariable prediction model using multinomial logistic regression. Stat. Methods Med. Res. 2023, 32, 555–571. [Google Scholar] [CrossRef] [PubMed]
  26. Snell, K.I.E.; Archer, L.; Ensor, J.; Bonnett, L.J.; Debray, T.P.A.; Phillips, B.; Collins, G.S.; Riley, R.D. External validation of clinical prediction models: Simulation-based sample size calculations were more reliable than rules-of-thumb. J. Clin. Epidemiol. 2021, 135, 79–89. [Google Scholar] [CrossRef]
  27. Steyerberg, E.W.; Vergouwe, Y. Towards better clinical prediction models: Seven steps for development and an ABCD for validation. Eur. Heart J. 2014, 35, 1925–1931. [Google Scholar] [CrossRef] [PubMed]
  28. Haimerl, B.J.; Encinas, R.; Justo, J.A.; Kohn, J.; Bookstaver, P.B.; Winders, H.R.; Al-Hasan, M.N. Optimization of Empirical Antimicrobial Therapy in Enterobacterales Bloodstream Infection Using the Extended-Spectrum Beta-Lactamase Prediction Score. Antibiotics 2023, 12, 1003. [Google Scholar] [CrossRef] [PubMed]
  29. Pogue, J.M.; Heil, E.L. The Clinical Impact of a Negative Molecular beta-Lactamase Gene Test for Enterobacteriaceae: Let’s Not Let Perfect Be the Enemy of Really Good. J. Clin. Microbiol. 2020, 58. [Google Scholar] [CrossRef] [PubMed]
  30. Pogue, J.M.; Heil, E.L.; Lephart, P.; Johnson, J.K.; Mynatt, R.P.; Salimnia, H.; Claeys, K.C. An Antibiotic Stewardship Program Blueprint for Optimizing Verigene BC-GN within an Institution: A Tale of Two Cities. Antimicrob. Agents Chemother. 2018, 62. [Google Scholar] [CrossRef]
  31. Tamma, P.D.; Aitken, S.L.; Bonomo, R.A.; Mathers, A.J.; van Duin, D.; Clancy, C.J. Infectious Diseases Society of America 2023 Guidance on the Treatment of Antimicrobial Resistant Gram-Negative Infections. Clin. Infect. Dis. 2023, ciad428. [Google Scholar] [CrossRef]
  32. Lagace-Wiens, P.R.; Adam, H.J.; Karlowsky, J.A.; Nichol, K.A.; Pang, P.F.; Guenther, J.; Webb, A.A.; Miller, C.; Alfa, M.J. Identification of blood culture isolates directly from positive blood cultures by use of matrix-assisted laser desorption ionization-time of flight mass spectrometry and a commercial extraction system: Analysis of performance, cost, and turnaround time. J. Clin. Microbiol. 2012, 50, 3324–3328. [Google Scholar] [CrossRef]
  33. Idelevich, E.A.; Schule, I.; Grunastel, B.; Wullenweber, J.; Peters, G.; Becker, K. Rapid identification of microorganisms from positive blood cultures by MALDI-TOF mass spectrometry subsequent to very short-term incubation on solid medium. Clin. Microbiol. Infect. 2014, 20, 1001–1006. [Google Scholar] [CrossRef] [PubMed]
  34. MacFadden, D.R.; Coburn, B.; Shah, N.; Robicsek, A.; Savage, R.; Elligsen, M.; Daneman, N. Decision-support models for empiric antibiotic selection in Gram-negative bloodstream infections. Clin. Microbiol. Infect. 2019, 25, 108.e1–108.e7. [Google Scholar] [CrossRef]
  35. Lodise, T.P.; Bonine, N.G.; Ye, J.M.; Folse, H.J.; Gillard, P. Development of a bedside tool to predict the probability of drug-resistant pathogens among hospitalized adult patients with gram-negative infections. BMC Infect. Dis. 2019, 19, 718. [Google Scholar] [CrossRef] [PubMed]
  36. Moons, K.G.; Altman, D.G.; Reitsma, J.B.; Ioannidis, J.P.; Macaskill, P.; Steyerberg, E.W.; Vickers, A.J.; Ransohoff, D.F.; Collins, G.S. Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): Explanation and elaboration. Ann. Intern. Med. 2015, 162, W1–W73. [Google Scholar] [CrossRef]
  37. Moons, K.G.M.; Wolff, R.F.; Riley, R.D.; Whiting, P.F.; Westwood, M.; Collins, G.S.; Reitsma, J.B.; Kleijnen, J.; Mallett, S. PROBAST: A Tool to Assess Risk of Bias and Applicability of Prediction Model Studies: Explanation and Elaboration. Ann. Intern. Med. 2019, 170, W1–W33. [Google Scholar] [CrossRef] [PubMed]
  38. Steyerberg, E.W.; Vickers, A.J.; Cook, N.R.; Gerds, T.; Gonen, M.; Obuchowski, N.; Pencina, M.J.; Kattan, M.W. Assessing the performance of prediction models: A framework for traditional and novel measures. Epidemiology 2010, 21, 128–138. [Google Scholar] [CrossRef] [PubMed]
  39. Moons, K.G.; Kengne, A.P.; Grobbee, D.E.; Royston, P.; Vergouwe, Y.; Altman, D.G.; Woodward, M. Risk prediction models: II. External validation, model updating, and impact assessment. Heart 2012, 98, 691–698. [Google Scholar] [CrossRef] [PubMed]
  40. Snell, K.I.E.; Levis, B.; Damen, J.A.A.; Dhiman, P.; Debray, T.P.A.; Hooft, L.; Reitsma, J.B.; Moons, K.G.M.; Collins, G.S.; Riley, R.D. Transparent reporting of multivariable prediction models for individual prognosis or diagnosis: Checklist for systematic reviews and meta-analyses (TRIPOD-SRMA). BMJ 2023, 381, e073538. [Google Scholar] [CrossRef] [PubMed]
  41. Moons, K.G.; de Groot, J.A.; Bouwmeester, W.; Vergouwe, Y.; Mallett, S.; Altman, D.G.; Reitsma, J.B.; Collins, G.S. Critical appraisal and data extraction for systematic reviews of prediction modelling studies: The CHARMS checklist. PLoS Med. 2014, 11, e1001744. [Google Scholar] [CrossRef]
  42. Schardt, C.; Adams, M.B.; Owens, T.; Keitz, S.; Fontelo, P. Utilization of the PICO framework to improve searching PubMed for clinical questions. BMC Med. Inform. Decis. Mak. 2007, 7, 16. [Google Scholar] [CrossRef]
  43. Damen, J.A.A.; Moons, K.G.M.; van Smeden, M.; Hooft, L. How to conduct a systematic review and meta-analysis of prognostic model studies. Clin. Microbiol. Infect. 2023, 29, 434–440. [Google Scholar] [CrossRef]
  44. Viechtbauer, W. Conducting Meta-Analyses inRwith themetaforPackage. J. Stat. Softw. 2010, 36, 1–48. [Google Scholar] [CrossRef]
Antibiotics 12 01452 g001
Figure 1. The study selection procedure.
Figure 1. The study selection procedure.
Antibiotics 12 01452 g001
Antibiotics 12 01452 g002
Figure 2. The discrimination performance of ESBL prediction models studies.
Figure 2. The discrimination performance of ESBL prediction models studies.
Antibiotics 12 01452 g002
Antibiotics 12 01452 g003
Figure 3. The pooled discrimination performance of external validations. (a) Augustine 2017 [10] ESBL model. (b) Lee 2017 [15] ESBL model.
Figure 3. The pooled discrimination performance of external validations. (a) Augustine 2017 [10] ESBL model. (b) Lee 2017 [15] ESBL model.
Antibiotics 12 01452 g003
Table 1. Study characteristics.
Table 1. Study characteristics.
First Author,
Year		CountryDesignModel TypeSample SizeResistance TypeResistance
Incidence
Andrews 2023 [9]USARetrospective cohortExternal validation of Lee 2017 and Augustine 2017356ESBL11.5%
Augustine 2017 [10]USARetrospective cohortDerivation and internal validation910ESBL4.6%
Cwengros 2020 [11]USARetrospective cohortExternal validation of Lee 2017 and Augustine 2017451ESBL16%
Goodman 2016 [12]USARetrospective cohortDerivation and internal validation1288ESBL15%
Goodman 2019 [13]USARetrospective cohortDerivation and internal validation1288ESBL15%
Holmgren 2020 [14]SwedenRetrospective cohortDerivation625ESBL9%
Lee 2017 [15]TaiwanRetrospective cohortDerivation1141ESBL5.7%
Madrid-Morales 2021 [16]USARetrospective cohortExternal validation of Augustine, 2017; Goodman, 2016 and 2019; and Tumbarello, 2011145ESBL13.8%
Tumbarello 2011 [17]ItalyRetrospective case–controlDerivation and external validation339; 510ESBL33.3%; 20%
Weston 2020 [18]USARetrospective case–controlDerivation and internal validation341CRENA
Table 2. Model characteristics.
Table 2. Model characteristics.
StudyDemographicsHospitalization/Admission HistoryMedical HistoryProcedure/Intervention HistoryMedication Use HistoryOther
Augustine 2017 [10]--ESBL-E infection or colonizationGI/GU procedureNumber of prior BL/FQ courses-
Goodman 2016 [12]Age, ≥43 yearsHospitalization in ESBL high-burden regionESBL-E infection or colonization, chronic indwelling vascular hardware-Days of antibiotic exposure
Goodman 2019 [13]-Hospital care abroad,
prior hospitalization		COPD, emphysema, ventilator dependence, indwelling hardware, MDRO colonization or infection-Weeks of antibiotic exposureSource of infection
Hömgren 2019 [14]-Hospital care abroadESBL-E infection or colonization---
Lee 2017 [15]-Prior hospitalization, recent ED visitsUrological diseases, diabetes mellitusInvasive procedureAntibiotic exposureNursing home residents
Tumbarello 2011 [17]Age, ≥70 yearsRecent hospitalization, admission from healthcareCharlson comorbidity index, urinary catheterization-Previous therapy with β-lactams and/or fluoroquinolones-
Weston 2020 [18]-Admitted >3 daysPrior CRE culture, liver disease, mechanical ventilation-Proton pump inhibitor, antibiotic exposureAdmission from SNF, no prior culture
NOTE. BL: β-lactams; COPD: chronic obstructive pulmonary disease; CRE: carbapenem-resistant Enterobacteriaceae; ED: emergency department; ESBL-E: extended-spectrum β-lactamase–producing Enterobacteriaceae; FQ: fluoroquinolones; GI: gastrointestinal; GU: genitourinary; MDRO: multidrug-resistant organisms; SNF: skilled nursing facility.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Timbrook, T.T.; Fowler, M.J. Predicting Extended-Spectrum Beta-Lactamase and Carbapenem Resistance in Enterobacteriaceae Bacteremia: A Diagnostic Model Systematic Review and Meta-Analysis. Antibiotics 2023, 12, 1452. https://doi.org/10.3390/antibiotics12091452

AMA Style

Timbrook TT, Fowler MJ. Predicting Extended-Spectrum Beta-Lactamase and Carbapenem Resistance in Enterobacteriaceae Bacteremia: A Diagnostic Model Systematic Review and Meta-Analysis. Antibiotics. 2023; 12(9):1452. https://doi.org/10.3390/antibiotics12091452

Chicago/Turabian Style

Timbrook, Tristan T., and McKenna J. Fowler. 2023. "Predicting Extended-Spectrum Beta-Lactamase and Carbapenem Resistance in Enterobacteriaceae Bacteremia: A Diagnostic Model Systematic Review and Meta-Analysis" Antibiotics 12, no. 9: 1452. https://doi.org/10.3390/antibiotics12091452

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop