Next Article in Journal
Antimicrobial Defined Daily Dose in Neonatal Population: Validation in the Clinical Practice
Previous Article in Journal
In Vitro Bacterial Competition of Staphylococcus aureus, Streptococcus agalactiae, and Escherichia coli against Coagulase-Negative Staphylococci from Bovine Mastitis Milk
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Direct Detection of KPC Peak from Positive Blood Cultures Using MALDI-TOF MS: Are We There Yet?

by
Natália Kehl Moreira
1,2,3,*,
Camila Mörschbächer Wilhelm
1,3,
Aymê Duarte Echevarria
3,4,
Fabiana Caroline Zempulski Volpato
1,3,
Priscila Lamb Wink
1,3,
Afonso Luís Barth
1,3 and
Juliana Caierão
1,2
1
Programa de Pós-Graduação em Ciências Farmacêuticas, Faculdade de Farmácia, Universidade Federal do Rio Grande do Sul, Porto Alegre 90610000, Rio Grande do Sul, Brazil
2
Laboratório de Pesquisa em Bacteriologia Clínica, Universidade Federal do Rio Grande do Sul, Porto Alegre 90610000, Rio Grande do Sul, Brazil
3
Laboratório de Pesquisa em Resistência Bacteriana, Hospital de Clínicas de Porto Alegre, Porto Alegre 90035007, Rio Grande do Sul, Brazil
4
Graduação em Biomedicina, Universidade Federal de Ciências da Saúde de Porto Alegre, Porto Alegre 90050170, Rio Grande do Sul, Brazil
*
Author to whom correspondence should be addressed.
Antibiotics 2023, 12(3), 601; https://doi.org/10.3390/antibiotics12030601
Submission received: 23 February 2023 / Revised: 14 March 2023 / Accepted: 15 March 2023 / Published: 17 March 2023
(This article belongs to the Section Mechanism and Evolution of Antibiotic Resistance)

Abstract

:
Detecting carbapenemase-associated carbapenem resistance is a subject of major clinical and epidemiological concern as it influences therapeutic choice. Matrix-assisted laser desorption ionization time-of-flight mass spectrometry (MALDI-TOF MS) has been proposed as a means to assess bacterial resistance mechanisms. We aimed to detect the KPC enzyme directly from positive blood cultures using MALDI-TOF MS. To do so, 102 clinical Enterobacteria were evaluated, including 59 blaKPC positives. Proteins were extracted using formic acid, isopropyl alcohol, and water (17:33:50) and spotted onto a steel target plate using the double-layer sinapinic acid technique. Two parameters were considered: (i) the visual detection of a clear peak with the expected KPC m/z and (ii) the evaluation of the relative intensity of the ions in the peak. A peak was observed in 56/59 blaKPC-positive isolates (94.9% sensitivity), with no false-positive results (100% specificity). When considering intensity, with a cut-off ≥120 (a.u.), sensitivity was 94.9% and specificity was 95.3%. We proposed a “buffer” zone, with intermediate values of intensity (115 to 125) reaching 100% sensitivity and specificity. The detection of KPC peaks directly from positive blood cultures using MALDI-TOF MS is feasible and rapid, which may improve appropriate patient therapy and antimicrobial stewardship.

1. Introduction

Antimicrobial resistance is considered by the World Health Organization (WHO) as one of the top ten global public health threats facing humanity [1]. Carbapenem-resistant Enterobacterales (CRE) are one of the main clinical challenges, as they are associated with significant morbidity and mortality, making them priority pathogens for research and the development of new antimicrobials [2,3,4]. When causing bloodstream infections, these bacteria are associated with a 75% increase in in-hospital mortality [5]. Surely, early appropriate antibiotic use is necessary to improve the prognosis of bloodstream infections, as the median survival rate decreases by 7.6% per hour of delay in implementing proper treatment for septic patients [6,7]. Thus, in the context of microbiology laboratories, rapid bacterial identification and detection of antimicrobial resistance from blood cultures play a significant role in guiding the most appropriate antibiotic therapy, aiming to improve patient outcome [8] and highlight antimicrobial stewardship [9,10].
In some cases, detecting the mechanism of resistance is necessary to establish an optimized treatment [11]. When it comes to CRE, carbapenemase production is the main mechanism, and Klebsiella pneumoniae carbapenemase (KPC) is the most widespread enzyme in many regions [2,12]. Several approaches have been described for the detection of carbapenemases [13,14,15,16,17]. Traditional microbiological methods such as the combined disc test using carbapenem and carbapenemase inhibitors, such as phenylboronic acid (PBA) and ethylenediaminetetraacetic acid (EDTA), remain convenient as they are easy and cheap to perform and allow to discriminate enzyme type; however, it is not fast enough, as it takes overnight incubation in order to provide results [16]. The carbapenem inactivation method (CIM) and its derivatives, mCIM and eCIM, also require a long turnaround time (20–22 h). On the other hand, although rapid (up to 4 h), colorimetric tests (Carba-NP and Blue Carba) do not discriminate by enzyme type and can also be subjective as they rely on visual observation to determine positivity for hydrolysis [17,18,19]. Polymerase chain reaction (PCR) is a fast and reliable way to detect and discriminate enzymes, but it is unable to detect novel carbapenemase genes. Besides, the correlation of the PCR result with the phenotypic expression of the enzyme may be an issue, as high Ct (threshold cycle) values can miss low-level expression of carbapenemase genes [13,20,21]. Moreover, a considerable portion of clinical laboratories do not include PCR as part of their routine. Another possibility for rapid carbapenemase detection are the immunochromatographic tests, which may detect specific enzymes directly from blood cultures in around 30 min or less [22,23]. However, their costs may be an issue for many laboratories [24].
Matrix-assisted laser desorption ionization time-of-flight mass spectrometry (MALDI-TOF MS) has proven to be an important tool in clinical microbiology laboratories, providing accurate and very rapid bacterial identification. Recently, MALDI-TOF MS has been proposed to assess bacterial resistance mechanisms [25,26,27,28] with a low cost per analysis [17]. Current studies have demonstrated the possibility of predicting carbapenemase production based on the detection of the hydrolyzed carbapenem molecule using MALDI-TOF MS. Although reliable, this detection is laborious, as it demands many extraction steps [14,28,29,30]. Besides, unspecific hydrolysis of the antibiotic may be a concern, compromising specificity; on the other hand, enzymes with reduced hydrolytic activity, such as some oxa-type carbapenemases, could generate false-negative results, reducing sensitivity [31,32].
In order to detect some specific peak that could be related to KPC production, a few authors have proposed that the presence of the KPC enzyme can be predicted at 11,109 Da, regarding the protein p019. However, this method relies on genetic context, as the p019 gene, present in blaKPC-2-producing K. pneumoniae, belongs to the specific ST258 clone. Therefore, negative results should be complemented by confirmatory tests [33,34,35].
Considering that, the direct detection of β-lactamases by MALDI-TOF MS seems to be a more reliable option. Indeed, recently, methodologies have been developed and evaluated for direct detection of specific β-lactamases, including KPC, using MALDI-TOF MS with promising results [36,37,38,39]. However, a significant number of studies are available on this issue, especially considering the direct detection from positive blood bottles, excluding the culture step. Indeed, to the best of our knowledge, there is a single study that used MALDI-TOF MS to detect KPC from blood cultures [37], which found 100% sensitivity and specificity. However, it evaluated a reduced number of blood culture bottles (39 samples, with only 16 KPC-producing bacteria). To strengthen the usefulness of this methodology, we evaluated an optimized MALDI-TOF MS protocol aiming to detect the KPC enzyme directly from positive blood cultures.

2. Results and Discussion

We evaluated 102 isolates, including 59 Enterobacterales carrying blaKPC gene (58 blaKPC and 1 blaKPC/blaNDM-1), identified as K. pneumoniae (n = 43), Serratia marcescens (n = 11), Escherichia coli (n = 3), and Enterobacter cloacae complex (n = 2) (Table 1). Besides, it was evaluated as 43 blaKPC negative Enterobacterales, with 24 of them carrying other carbapenemase genes: 19 blaNDM-1, 4 blaOXA-48-like, and 1 blaNDM-1/blaOXA-48-like, mainly K. pneumoniae (n = 13), but also E. cloacae complex (n = 9), E. coli (n = 1), and Klebsiella oxytoca (n = 1), as shown in Table 2. The blaKPC-positive isolates had MICs of meropenem ranging from 16 to >256 µg/mL, while the isolates producing carbapenemase other than KPC presented MICs ranging from ≤0.5 to 256 µg/mL.
Moreover, nineteen isolates that had negative results for carbapenemase genes (i.e., KPC, NDM-1, GES, OXA-48, IMP, and VIM) by the HRM-qPCR were also included. They were identified as K. pneumoniae (n = 11), E. cloacae complex (n = 3), K. oxytoca (n = 3), and Klebsiella. aerogenes (n = 2) (Table 2), with MICs varying from ≤0.5 to 64 µg/mL.
Statistical results showed a non-normal distribution by Shapiro-Wilk normality test (p-value < 0.001). Then, the Mann-Whitney U test confirmed a statistically significant difference (p-value < 0.001), considering the presence of KPC peaks when both groups of bacteria (blaKPC-positive and negative) were compared. The area under the curve (AUC) of the receiver operating characteristic (ROC) curve for this peak was 0.997, which enabled us to believe the detection of KPC peak is capable of significantly discriminating populations of KPC-producing bacteria.
We first analyzed results by visual observation after spectra were baseline-subtracted and smoothed. Considering this, 56 out of 59 blaKPC-positive isolates presented a positive peak in at least one of the triplicates, providing a 94.9% sensitivity. KPC peaks were found at 28,661–28,736 m/z, with an arithmetic mean ± SD of 28,698 ± 23 (95% CI). If two or three replicates presented a spectrum, the m/z average of the peak was considered. For three isolates (7595F, 8219F, and 8649F), a clear KPC peak was not found (Figure 1 and Table 1). Of note, all blaKPC-positive isolates presented meropenem MIC ≥ 16 µg/mL. On the other hand, for the 43 isolates negative to blaKPC by HRM-qPCR, the peak was absent (100% specificity) (Table 2).
Previous studies showed different m/z values for KPC peaks. Moreira et al. presented peaks at m/z 28,685–28,691 [39]. As determined by Yoon et al., the molecular mass of the intact KPC-2 polypeptide estimated for blaKPC-2 transformants was 28,718 Da; however, they observed KPC peaks at m/z 28,708–28,728 corresponding to m/z of different KPC variants [38]. Indeed, the mass of KPC peak varies according to the enzyme subtype, which may justify the m/z range observed in our experiments as well as in other studies [38]. In this context, one limitation of our study is that we did not discriminate the KPC subtypes, as the HRM-qPCR detects all recognized KPC genes [40].
The second variable we evaluated was intensities of KPC peaks. For blaKPC-positive isolates, it ranged from 115 to 2229 (a.u.), with a median value of 364 (Figure 2 and Table 1). On the other hand, blaKPC-negative isolates, despite not having any peaks by visual observation, presented a spectrum with an uneven background at 28,661–28,736 m/z and very low intensity (9–123; a median value of 38 [a.u.]) (Figure 2 and Table 2).
Considering, as suggested elsewhere [39], that intensity values of 120 (a.u.) or higher indicate a KPC-positive isolate, the method presented a sensitivity of 94.9% and a specificity of 95.3%. Indeed, using this cut-off, 3 false-negative results were observed, which were the same isolates classified as false negative when analysis was performed by visual observation: K. pneumoniae isolates 7595F, 8219F, and 8649F (Figure 1 and Table 1), with intensities of 117, 119, and 115 (a.u), respectively. Of note, these values of intensity are borderline considering 120 (a.u.) as the cutoff.
The phenotypic combined disc test confirmed these false-negative isolates as serine-β-lactamases producers, and MIC values were 128 µg/mL or higher. Besides, KPC gene sequencing revealed a closer match with approximately 100% identity in an overlap of 798 nucleotides with the registration under the name “beta-lactamase KPC-2 [Klebsiella pneumoniae]”. Thus, the reasons for the intensities just below 120 (a.u.) remain to be elucidated.
On the other hand, among the 43 blaKPC-negative Enterobacterales, we classified 2 isolates as false positives (Figure 1 and Table 2) if the cut-off of 120 (a.u.) was applied: a K. pneumoniae (8238F) not carrying carbapenemase genes, with MIC of 1 μg/mL, and a blaNDM-1-positive E. cloacae complex (8396F), presenting MIC of 4 μg/mL. These isolates had intensities of 123 and 121 (a.u.), respectively, even after repetition was performed.
As described above, the intensity values of both false negatives and false positives were very close to the intensity cut-off of 120 (a.u.). For this reason, based on our experience and considering what has been published for carbapenem hydrolysis assays (i.e., intermediate values of logRQ) [27,41], we propose an intermediate value of intensity (i.e., a “buffer” zone), between 115 and 125 (a.u.), in which results would be considered indeterminate or inconclusive. Therefore, applying this “buffer” zone, 7 isolates were classified as indeterminate or inconclusive. These isolates included those 5 bacteria with false-positive and negative results considering the cut-off of 120 (a.u.), but also another 2 isolates, both S. marcescens (8848F and 9221F), with intensities of 125 and 121 (a.u.), respectively. Excluding these inconclusive results, the detection of KPC isolates, considering the intensity of KPC peak, would present 100% sensitivity and specificity. It must be highlighted that other studies did not have proposed intermediate values to evaluate the intensity of the KPC peak [37,38].
We are aware that the visual determination of the presence or absence of the KPC peak may be operator-dependent. That is the reason why we suggested the use of the cut-off value of peak intensity. We believe that it is necessary to better differentiate KPC positive and negative isolates. Indeed, establishing a non-ambiguous way to evaluate the MALDI-TOF results for KPC detection, such as a cut-off value of peak intensity, is important to avoid misleading results from the technique. It should be noted, however, that our results do not demonstrate significant issues using the visual analysis of the spectra. Even so, it must be considered that the cut-off value should be standardized for each laboratory to overcome inter-laboratory variations in spectra.
The rapid identification of carbapenem-resistant organisms is critical not only for the initiation of the proper antimicrobial regimen but also for stopping their spread. The preliminary screening for carbapenemase producers in clinical specimens is traditionally based on phenotypic tests, whereas confirmation tests are mainly based on molecular assays [13]. Conventional phenotypic methods have some important disadvantages, mainly the fact that they are time-consuming, as they require bacterial growth in solid culture medium (18–24 h) plus overnight incubation for test reading (Figure 3). Furthermore, they can be difficult to interpret, and sensitivity/specificity vary between different species [13,16,24,42].
In times where speed in releasing results is increasingly demanded from clinical microbiology laboratories, a few methods, such as the detection of carbapenem hydrolysis by MALDI-TOF MS and Carba-NP (Figure 3), can be performed directly from positive blood cultures, improving the time of carbapenem resistance detection. It may enable the establishment of proper treatment in approximately two and a half hours [14,27,42]. The hydrolysis assay and Carba-NP are not able to determine the carbapenemase type, which may be especially needed when using the new combinations of beta-lactam and beta-lactamase inhibitors [11]. Despite having excellent sensitivity and specificity, immunochromatographic tests are not included in Figure 3 due to the high cost per sample of about 15€, which is considered expensive for some laboratories since the MALDI-TOF detection method can be performed for less than 1€ per sample [17]. Therefore, the strength of our study was to allow the detection of KPC (the most widely disseminated carbapenemase type) from positive blood cultures in less than 1 h from positivity of blood cultures, which would be valuable to improve patient outcome and antimicrobial stewardship.

3. Materials and Methods

3.1. Bacterial Isolates and Carbapenemase Characterization

A total of 102 clinical Enterobacterales, recovered between 2018 and 2021 from patients attending a tertiary hospital in Porto Alegre, southern Brazil, were evaluated. The study was approved by the local research ethical committee (CAAE 167 31638920800005327). Bacteria were identified by MALDI-TOF MS (Bruker Daltonics, Billerica, MA, USA).
Carbapenemase genes were detected by multiplex high-resolution melting real-time PCR (HRM-qPCR) [40]. Briefly, bacterial DNA was extracted by thermal lysis. To do so, the bacteria (from the solid culture medium) were placed in a sterile Eppendorf tube with 500 µL of TE buffer, which was maintained at 80 °C in a dry bath and at −20 °C until freezing. The material was centrifuged, and the supernatant was quantified by Nanodrop Nucleic Acid Quantification (Thermo Fisher Scientific, Waltham, MA, USA) prior to HRM qPCR (MeltDoctor™ HRM Master Mix, Thermo Fisher Scientific). The qPCR multiplex used specific primers, as published elsewhere [40], for the following carbapenemase genes: blaKPC, blaNDM, blaOXA-48-like, blaIMP, blaGES, and blaVIM.
Susceptibility to meropenem (Sigma-Aldrich, St. Louis, MO, USA) was determined by broth microdilution (0.5 to 256 µg/mL), according to ISO 20776-1 [43] and interpreted using the European Committee on Antimicrobial Susceptibility Testing (EUCAST) breakpoints [44]: isolates presenting MIC > 8 μg/mL were considered resistant to meropenem. Escherichia coli ATCC 25922 was included for quality control.
Isolates for which the blaKPC gene was detected but peak/intensity was not identified were evaluated phenotypically by the enzymatic inhibition test according to the EUCAST guideline [45]. Briefly, a 0.5-McFarland suspension was prepared for each isolate and inoculated on Mueller-Hinton Agar (Sigma-Aldrich). Meropenem discs (OXOID, Hampshire, UK), 10 µg, were used as substrate; in two of them, 10 µL of ethylenediaminetetraacetic acid (EDTA) (NEON, São Paulo, Brazil) 0.1 M or phenylboronic acid (PBA) (Sigma-Aldrich) 40 mg/mL solution was added. After 18 to 24 h of incubation at 35–37 °C, an increase in the inhibition zone of at least 5 mm comparing discs with and without PBA defined the isolate as a KPC producer.
False-negative isolates classified as blaKPC-positive by HRM-qPCR had their amplicons submitted to Sanger sequencing to confirm the presence of blaKPC. The results obtained by sequencing were compared with the GenBank database using the “National Center for Biotechnology Information Computer Blast” program (https://blast.ncbi.nlm.nih.gov/Blast.cgi, accessed on 13 December 2022).

3.2. Blood Culture Samples

Bacteria were cultured on Mueller-Hinton agar and incubated overnight. To simulate positive blood cultures, 1 mL of a bacterial suspension containing approximately 107 colony forming units (CFU) was inoculated into an aerobic blood culture bottle (BacT/ALERT® FA Plus REF 410851, bioMérieux, Marcy-l’Étoile, France), which had previously been inoculated with 4 mL of human blood [37]. Thereby, a final bacterial concentration of approximately 2 × 106 UFC/mL per bottle was achieved. The bottle was incubated following the manufacturer’s instructions (BacT/ALERT®, bioMérieux) until positivity was achieved.

3.3. Protein Extraction

Once positive, a bacterial pellet was obtained by successive centrifugation steps [37]. Briefly, 1.4 mL of bottle solution was transferred to an eppendorf tube, which was centrifuged at 1450 rpm (200× g) for 5 min. The supernatant was collected and centrifuged at 12,210 rpm (14,170× g) for 1 min, and the pellet was washed with distilled water (900 µL), vortexed for 30–60 s, and centrifuged at 13,000 rpm (16,060× g) for 2 min. The bacterial pellet was suspended in 300 μL of distilled water and vortexed for 30 s at room temperature. Then, 900 μL of absolute ethanol (Dinamica, São Paulo, Brazil) was added, vortexed for 30–60 s, and centrifuged at 13,000 rpm (16,060× g) for 2 min. The supernatant was discarded, and the pellet was re-suspended in 100 µL of extraction solvent (formic acid—isopropyl alcohol—water, 17:33:50 v/v) (formic acid and isopropyl alcohol from Sigma-Aldrich, Missouri, EUA). The suspension was vortexed for 30–60 s and centrifuged for 2 min at 13,000 rpm (16,060× g). The clean supernatant extract was kept at room temperature for further analysis [39].

3.4. Target Spot Loading

The extracted protein was spotted onto a steel target plate using a double-layer sinapinic acid (SA) method [37]. The first layer was composed of 0.7 μL of SA-saturated solution (10 mg/mL SA in absolute ethanol) (sinapinic acid from Sigma-Aldrich, Missouri, EUA). For the second layer, protein extract was mixed 1:1 with 10 mg/mL of SA solution in acetonitrile (30:70 v/v) (Merck, Darmstadt, Germany) and 0.1% trifluoroacetic acid (Sigma-Aldrich) in water. One microliter of this sample/matrix mixture was deposited onto a spot containing the first layer. The sample was left to dry at room temperature and then analyzed by MALDI-TOF MS. Each sample was analyzed in triplicate, loaded once into three different spots (i.e., three spectra per sample).

3.5. Spectra Acquisition

Linear MALDI-TOF spectra were obtained in the positive ion mode of the Microflex LT mass spectrometer (Bruker Daltonics, Bremen, Germany) with flexControl 3.4 software (Bruker Daltonics). The parameters were configured as previously described: mass range of 17,000 Da to 50,000 Da; spectrometer ion source 1, 20.08 kV; ion source 2 18.16 kV; lens 6.03 kV; pulsed ion extraction, 550 ns; detection gain of 2803 V; sample rate; and electronic setting of 0.50 GS/s. Laser frequency is 60 Hz, and laser power ranges between 60 and 85%. Each spectrum was obtained after 100 shots per spot. Data were automatically acquired using autoXecute mode, and spectra were analyzed using FlexAnalysis 3.4 software (Bruker Daltonics) [38]. Before each run, the spectrometer was calibrated using Protein Standard II Calibration Mix (Bruker Daltonics).

3.6. Data Analysis

We searched for a peak closer to the expected size of the KPC (28,643–28,731 Da), as determined elsewhere [39]. Two parameters were considered. First, the visual presence of the KPC peak was checked. After the relative ion intensity [arbitrary units (a.u.)] was considered after the spectra were baseline subtracted and smoothed, the arithmetic mean ± standard deviation (SD) were calculated, as were sensitivity and specificity, considering the results of HRM-qPCR [40] as a reference. The Shapiro–Wilk test was conducted to test normality, and the Mann–Whitney test was conducted to express differences between the groups. Still, the area under the curve (AUC) of the receiver operating characteristic (ROC) curve was also performed to confirm that the peak found corresponded to blaKPC-positive isolates. All statistical tests were performed with a 95% confidence interval. Statistical analyses were performed using SPSS (PASW Statistics, version 18.0.3).

4. Conclusions

The widespread presence of CRE, especially KPC, has generated an urgent need for the development of rapid and reliable methods for detecting carbapenem resistance. In this context, the potentials of MALDI-TOF MS have been explored. Our study presented an optimized protocol to detect KPC peaks using MALDI-TOF MS directly from positive blood cultures. It proved to be an accurate and rapid method to discriminate between spectra of Enterobacterales that produce and do not produce KPC. In order to make the analysis unambiguous, we proposed a cut-off value for the intensity of the KPC peak, as we believe the visual observation of the peak may be subjective in some instances. Applying this methodology, the detection of carbapenem resistance would be very unexpansive and rapid, reducing the turnaround time of the exam for up to 1 day and potentially contributing to the establishment of adequate therapy for patients with infections caused by CRE.

Author Contributions

Conceptualization, visualization supervision, funding acquisition, and project administration, A.L.B. and J.C.; methodology, N.K.M. and C.M.W.; validation, N.K.M.; formal analysis and investigation, N.K.M., C.M.W., F.C.Z.V., A.D.E. and P.L.W.; writing—original draft preparation, N.K.M.; writing—review and editing, N.K.M., C.M.W., A.L.B. and J.C. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by Fundação de Amparo à Pesquisa do Estado do Rio Grande do Sul (Programa Pesquisador Gaúcho, edital 05/2019), Fundo de Incentivo à Pesquisa e Eventos do Hospital de Clínicas de Porto Alegre (Projeto 2020-0233), Instituto Nacional de Pesquisa em Resistência Antimicrobiana (INPRA) (INCT/CNPq 465718/2014-0 and INCT/FAPERGS 17/2551-0000514-7) and a master’s scholarship from Coordenação de Aperfeiçoamento de Pessoal de Nível Superior.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

The data presented in this study are available on request from the corresponding author.

Conflicts of Interest

The authors have no financial interest or personal relationships that may have influenced the work reported in this article.

References

  1. WHO 10 Global Health Issues to Track in 2021. Available online: https://www.who.int/news-room/spotlight/10-global-health-issues-to-track-in-2021 (accessed on 20 March 2022).
  2. Lee, C.-R.; Lee, J.H.; Park, K.S.; Kim, Y.B.; Jeong, B.C.; Lee, S.H. Global Dissemination of Carbapenemase-Producing Klebsiella Pneumoniae: Epidemiology, Genetic Context, Treatment Options, and Detection Methods. Front. Microbiol. 2016, 7, 895. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  3. Logan, L.K.; Weinstein, R.A. The Epidemiology of Carbapenem-Resistant Enterobacteriaceae: The Impact and Evolution of a Global Menace. J. Infect. Dis. 2017, 215, S28–S36. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  4. Shrivastava, S.R.; Shrivastava, P.S.; Ramasamy, J. World Health Organization Global Priority List of Antibiotic-Resistant Bacteria to Guide Research, Discovery, and Development of New Antibiotics. J. Med. Soc. 2018, 32, 76. [Google Scholar] [CrossRef]
  5. Perez, F.; Bonomo, R.A. Carbapenem-Resistant Enterobacteriaceae: Global Action Required. Lancet Infect. Dis. 2019, 19, 561–562. [Google Scholar] [CrossRef]
  6. Kumar, A.; Roberts, D.; Wood, K.E.; Light, B.; Parrillo, J.E.; Sharma, S.; Suppes, R.; Feinstein, D.; Zanotti, S.; Taiberg, L.; et al. Duration of Hypotension before Initiation of Effective Antimicrobial Therapy Is the Critical Determinant of Survival in Human Septic Shock. Crit. Care Med. 2006, 34, 1589–1596. [Google Scholar] [CrossRef]
  7. Kumar, A.; Ellis, P.; Arabi, Y.; Roberts, D.; Light, B.; Parrillo, J.E.; Dodek, P.; Wood, G.; Kumar, A.; Simon, D.; et al. Initiation of Inappropriate Antimicrobial Therapy Results in a Fivefold Reduction of Survival in Human Septic Shock. Chest 2009, 136, 1237–1248. [Google Scholar] [CrossRef]
  8. Tariq, T.M.; Rasool, E. Emerging Trends of Bloodstream Infections: A Six-Year Study at a Paediatric Tertiary Care Hospital in Kabul. J. Coll. Physicians Surg. Pak. 2016, 26, 887–891. [Google Scholar]
  9. Rice, L.B. Antimicrobial Stewardship and Antimicrobial Resistance. Med. Clin. N. Am. 2018, 102, 805–818. [Google Scholar] [CrossRef]
  10. Nauclér, P.; Huttner, A.; van Werkhoven, C.H.; Singer, M.; Tattevin, P.; Einav, S.; Tängdén, T. Impact of Time to Antibiotic Therapy on Clinical Outcome in Patients with Bacterial Infections in the Emergency Department: Implications for Antimicrobial Stewardship. Clin. Microbiol. Infect. 2021, 27, 175–181. [Google Scholar] [CrossRef]
  11. Noval, M.; Banoub, M.; Claeys, K.C.; Heil, E. The Battle Is on: New Beta-Lactams for the Treatment of Multidrug-Resistant Gram-Negative Organisms. Curr. Infect. Dis. Rep. 2020, 22, 1. [Google Scholar] [CrossRef]
  12. Bush, K.; Bradford, P.A. Epidemiology of β-Lactamase-Producing Pathogens. Clin. Microbiol. Rev. 2020, 33, e00047-19. [Google Scholar] [CrossRef]
  13. AlTamimi, M.; AlSalamah, A.; AlKhulaifi, M.; AlAjlan, H. Comparison of Phenotypic and PCR Methods for Detection of Carbapenemases Production by Enterobacteriaceae. Saudi J. Biol. Sci. 2017, 24, 155–161. [Google Scholar] [CrossRef] [Green Version]
  14. Hrabák, J. Detection of Carbapenemases Using Matrix-Assisted Laser Desorption/Ionization Time-of-Flight Mass Spectrometry (MALDI-TOF MS) Meropenem Hydrolysis Assay. Sepsis 2015, 1237, 91–96. [Google Scholar] [CrossRef]
  15. Nordmann, P.; Poirel, L.; Dortet, L. Rapid Detection of Carbapenemase-Producing Enterobacteriaceae. Emerg. Infect. Dis. 2012, 18, 1503–1507. [Google Scholar] [CrossRef] [Green Version]
  16. Tamma, P.D.; Opene, B.N.A.; Gluck, A.; Chambers, K.K.; Carroll, K.C.; Simner, P.J. Comparison of 11 Phenotypic Assays for Accurate Detection of Carbapenemase-Producing Enterobacteriaceae. J. Clin. Microbiol. 2017, 55, 1046–1055. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  17. Tamma, P.D.; Simner, P.J. Phenotypic Detection of Carbapenemase-Producing Organisms from Clinical Isolates. J. Clin. Microbiol. 2018, 56, e01140-18. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  18. Nordmann, P.; Poirel, L. Epidemiology and Diagnostics of Carbapenem Resistance in Gram-Negative Bacteria. Clin. Infect. Dis. 2019, 69, S521–S528. [Google Scholar] [CrossRef] [Green Version]
  19. Perez, K.K.; Olsen, R.J.; Musick, W.L.; Cernoch, P.L.; Davis, J.R.; Land, G.A.; Peterson, L.E.; Musser, J.M. Integrating Rapid Pathogen Identification and Antimicrobial Stewardship Significantly Decreases Hospital Costs. Arch. Pathol. Lab. Med. 2013, 137, 1247–1254. [Google Scholar] [CrossRef] [Green Version]
  20. Byun, J.-H.; Kim, Y.A.; Kim, M.; Kim, B.; Choi, J.Y.; Park, Y.S.; Yong, D. Evaluation of Xpert Carba-R Assay v.2 to Detect Carbapenemase Genes in Two Hospitals in Korea. Ann. Lab. Med. 2020, 40, 209–215. [Google Scholar] [CrossRef]
  21. Lutgring, J.D.; Limbago, B.M. The Problem of Carbapenemase-Producing-Carbapenem-Resistant-Enterobacteriaceae Detection. J. Clin. Microbiol. 2016, 54, 529–534. [Google Scholar] [CrossRef] [Green Version]
  22. Hamprecht, A.; Vehreschild, J.J.; Seifert, H.; Saleh, A. Rapid Detection of NDM, KPC and OXA-48 Carbapenemases Directly from Positive Blood Cultures Using a New Multiplex Immunochromatographic Assay. PLoS ONE 2018, 13, e0204157. [Google Scholar] [CrossRef] [Green Version]
  23. Wink, P.L.; Martins, A.S.; Inamine, E.; Dalmolin, T.V.; Barth, A.L. Rapid Detection of the Main Carbapenemases in Brazil Directly from Spiked Blood Culture Using the RESIST-3 O.K.N. Immunoassay. Braz. J. Microbiol. 2019, 50, 657–662. [Google Scholar] [CrossRef] [PubMed]
  24. Baeza, L.L.; Pfennigwerth, N.; Greissl, C.; Göttig, S.; Saleh, A.; Stelzer, Y.; Gatermann, S.G.; Hamprecht, A. Comparison of Five Methods for Detection of Carbapenemases in Enterobacterales with Proposal of a New Algorithm. Clin. Microbiol. Infect. 2019, 25, 1286.e9–1286.e15. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  25. Burckhardt, I.; Zimmermann, S. Susceptibility Testing of Bacteria Using Maldi-Tof Mass Spectrometry. Front. Microbiol. 2018, 9, 1744. [Google Scholar] [CrossRef] [Green Version]
  26. Florio, W.; Baldeschi, L.; Rizzato, C.; Tavanti, A.; Ghelardi, E.; Lupetti, A. Detection of Antibiotic-Resistance by MALDI-TOF Mass Spectrometry: An Expanding Area. Front. Cell. Infect. Microbiol. 2020, 10, 572909. [Google Scholar] [CrossRef] [PubMed]
  27. Akyar, I.; Kaya Ayas, M.; Karatuna, O. Performance Evaluation of MALDI-TOF MS MBT STAR-BL Versus In-House Carba NP Testing for the Rapid Detection of Carbapenemase Activity in Escherichia Coli and Klebsiella Pneumoniae Strains. Microb. Drug Resist. 2019, 25, 985–990. [Google Scholar] [CrossRef]
  28. Papagiannitsis, C.C.; Študentová, V.; Izdebski, R.; Oikonomou, O.; Pfeifer, Y.; Petinaki, E.; Hrabák, J. Matrix-Assisted Laser Desorption Ionization–Time of Flight Mass Spectrometry Meropenem Hydrolysis Assay with NH4 HCO3, a Reliable Tool for Direct Detection of Carbapenemase Activity. J. Clin. Microbiol. 2015, 53, 1731–1735. [Google Scholar] [CrossRef] [Green Version]
  29. Oviaño, M.; Bou, G. Matrix-Assisted Laser Desorption Ionization–Time of Flight Mass Spectrometry for the Rapid Detection of Antimicrobial Resistance Mechanisms and Beyond. Clin. Microbiol. Rev. 2018, 32, e00037-18. [Google Scholar] [CrossRef] [Green Version]
  30. Oviaño, M.; Sparbier, K.; Barba, M.J.; Kostrzewa, M.; Bou, G. Universal Protocol for the Rapid Automated Detection of Carbapenem-Resistant Gram-Negative Bacilli Directly from Blood Cultures by Matrix-Assisted Laser Desorption/Ionisation Time-of-Flight Mass Spectrometry (MALDI-TOF/MS). Int. J. Antimicrob. Agents 2016, 48, 655–660. [Google Scholar] [CrossRef] [PubMed]
  31. Oueslati, S.; Nordmann, P.; Poirel, L. Heterogeneous Hydrolytic Features for OXA-48-like β-Lactamases. J. Antimicrob. Chemother. 2015, 70, 1059–1063. [Google Scholar] [CrossRef] [Green Version]
  32. Bakthavatchalam, Y.; Anandan, S.; Veeraraghavan, B. Laboratory Detection and Clinical Implication of Oxacillinase-48 like Carbapenemase: The Hidden Threat. J. Glob. Infect. Dis. 2016, 8, 41. [Google Scholar] [CrossRef] [PubMed]
  33. Youn, J.-H.; Drake, S.K.; Weingarten, R.A.; Frank, K.M.; Dekker, J.P.; Lau, A.F. Clinical Performance of a Matrix-Assisted Laser Desorption Ionization–Time of Flight Mass Spectrometry Method for Detection of Certain Bla KPC -Containing Plasmids. J. Clin. Microbiol. 2016, 54, 35–42. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  34. Gaibani, P.; Galea, A.; Fagioni, M.; Ambretti, S.; Sambri, V.; Landini, M.P. Evaluation of Matrix-Assisted Laser Desorption Ionization–Time of Flight Mass Spectrometry for Identification of KPC-Producing Klebsiella Pneumoniae. J. Clin. Microbiol. 2016, 54, 2609–2613. [Google Scholar] [CrossRef] [Green Version]
  35. Lau, A.F.; Wang, H.; Weingarten, R.A.; Drake, S.K.; Suffredini, A.F.; Garfield, M.K.; Chen, Y.; Gucek, M.; Youn, J.-H.; Stock, F.; et al. A Rapid Matrix-Assisted Laser Desorption Ionization–Time of Flight Mass Spectrometry-Based Method for Single-Plasmid Tracking in an Outbreak of Carbapenem-Resistant Enterobacteriaceae. J. Clin. Microbiol. 2014, 52, 2804–2812. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  36. Espinosa, R.F.; Rumi, V.; Marchisio, M.; Cejas, D.; Radice, M.; Vay, C.; Barrios, R.; Gutkind, G.; Di Conza, J. Fast and Easy Detection of CMY-2 in Escherichia Coli by Direct MALDI-TOF Mass Spectrometry. J. Microbiol. Methods 2018, 148, 22–28. [Google Scholar] [CrossRef] [PubMed]
  37. Figueroa-Espinosa, R.; Costa, A.; Cejas, D.; Barrios, R.; Vay, C.; Radice, M.; Gutkind, G.; Di Conza, J. MALDI-TOF MS Based Procedure to Detect KPC-2 Directly from Positive Blood Culture Bottles and Colonies. J. Microbiol. Methods 2019, 159, 120–127. [Google Scholar] [CrossRef]
  38. Yoon, E.-J.; Lee, E.H.; Hwang, D.H.; Lee, H.; Baek, J.-H.; Jeong, S.H. Direct Detection of Intact Klebsiella Pneumoniae Carbapenemases Produced by Enterobacterales Using MALDI-TOF MS. J. Antimicrob. Chemother. 2020, 75, 1174–1181. [Google Scholar] [CrossRef]
  39. Moreira, N.K.; Wilhelm, C.M.; Wink, P.L.; Barth, A.L.; Caierão, J. MALDI-TOF Mass Spectrometry for Direct KPC Detection among Enterobacterales. Braz. J. Microbiol. 2022, 53, 1907–1913. [Google Scholar] [CrossRef]
  40. Monteiro, J.; Widen, R.H.; Pignatari, A.C.C.; Kubasek, C.; Silbert, S. Rapid Detection of Carbapenemase Genes by Multiplex Real-Time PCR. J. Antimicrob. Chemother. 2012, 67, 906–909. [Google Scholar] [CrossRef] [Green Version]
  41. Jung, J.S.; Popp, C.; Sparbier, K.; Lange, C.; Kostrzewa, M.; Schubert, S. Evaluation of Matrix-Assisted Laser Desorption Ionization-Time of Flight Mass Spectrometry for Rapid Detection of -Lactam Resistance in Enterobacteriaceae Derived from Blood Cultures. J. Clin. Microbiol. 2014, 52, 924–930. [Google Scholar] [CrossRef] [Green Version]
  42. Meier, M.; Hamprecht, A. Systematic Comparison of Four Methods for Detection of Carbapenemase-Producing Enterobacterales Directly from Blood Cultures. J. Clin. Microbiol. 2019, 57, e00709-19. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  43. ISO Clinical Laboratory Testing and in Vitro Diagnostic Test Systems—Susceptibility Testing of Infectious Agents and Evaluation of Performance of Antimicrobial Susceptibility Test Devices—Part 1: Broth Micro-Dilution Reference Method for Testing the in Vitro Activity of Antimicrobial Agents against Rapidly Growing Aerobic Bacteria Involved in Infectious Diseases. 2019. Available online: https://www.iso.org/standard/70464.html (accessed on 12 October 2022).
  44. EUCAST Breakpoint Tables for Interpretation of MICs and Zone Diameters. Version 12.0. 2022. Available online: Http://Www.Eucast.Org (accessed on 19 August 2022).
  45. EUCAST Guidelines for Detection of Resistance Mechanisms and Specific Resistances of Clinical and/or Epidemiological Importance 2017. Version 2.01. 2017. Available online: http://www.eucast.org/ (accessed on 19 August 2022).
Figure 1. False negative (7595F, 8219F, and 8649F) and false positive (8238F and 8396F) results, considering the visual analysis of the spectra (positive control: 7606F) and the intensities of the peak with the expected m/z for KPC, compared with HRM-qPCR. I: intensity; a.u.: arbitrary units; MIC: meropenem minimal inhibitory concentration.
Figure 1. False negative (7595F, 8219F, and 8649F) and false positive (8238F and 8396F) results, considering the visual analysis of the spectra (positive control: 7606F) and the intensities of the peak with the expected m/z for KPC, compared with HRM-qPCR. I: intensity; a.u.: arbitrary units; MIC: meropenem minimal inhibitory concentration.
Antibiotics 12 00601 g001
Figure 2. Box plot showing the median (line in the middle) and interquartile range calculated from the intensities of the 28,698 Da peak in the blaKPC-positive and negative isolates. Dots above the whiskers indicate the outliers. The x and y axes show the HRM-qPCR result and intensity (arbitrary units—a.u.), respectively.
Figure 2. Box plot showing the median (line in the middle) and interquartile range calculated from the intensities of the 28,698 Da peak in the blaKPC-positive and negative isolates. Dots above the whiskers indicate the outliers. The x and y axes show the HRM-qPCR result and intensity (arbitrary units—a.u.), respectively.
Antibiotics 12 00601 g002
Figure 3. Turnaround time of available alternative methods to detect carbapenemase production among Enterobacterales. CARB: carbapenem; CP: carbapenemase producer without definition of type; Non-CP: non-carbenemase producer. The figure was partly generated using Servier Medical Art, provided by Servier, licensed under a Creative Commons Attribution 3.0 unported license.
Figure 3. Turnaround time of available alternative methods to detect carbapenemase production among Enterobacterales. CARB: carbapenem; CP: carbapenemase producer without definition of type; Non-CP: non-carbenemase producer. The figure was partly generated using Servier Medical Art, provided by Servier, licensed under a Creative Commons Attribution 3.0 unported license.
Antibiotics 12 00601 g003
Table 1. Characteristics of blaKPC-positive bacteria used to inoculate blood cultures for KPC detection and results of spectra analysis by direct inspection and determination of intensity.
Table 1. Characteristics of blaKPC-positive bacteria used to inoculate blood cultures for KPC detection and results of spectra analysis by direct inspection and determination of intensity.
IDSpeciesKPC Peak m/zi (a.u.)Meropenem MIC (µg/mL)
7023FE. coli28,71356364
7431FK. pneumoniae28,727351256
7434FK. pneumoniae28,66366364
7437FE. cloacae complex28,67195164
7440FK. pneumoniae28,7261366256
7442FK. pneumoniae28,726994128
7462FK. pneumoniae28,71753064
7464FK. pneumoniae28,688883>256
7467FK. pneumoniae28,724683128
7480FK. pneumoniae28,721434128
7481FK. pneumoniae28,729298256
7502FK. pneumoniae28,710360>256
7514FK. pneumoniae28,720200128
7590FK. pneumoniae28,684584>256
7592FK. pneumoniae28,6632229128
7593FK. pneumoniae28,721149128
7594FK. pneumoniae28,665450128
7595FK. pneumoniaeAbsent117>256
7599FK. pneumoniae28,715202128
7605FK. pneumoniae28,71322464
7606FK. pneumoniae28,67949764
8156FK. pneumoniae28,73613916
8219FK. pneumoniaeAbsent119128
8285F *K. pneumoniae28,68736464
8467FK. pneumoniae28,721502>256
8649FK. pneumoniaeAbsent115>256
8799FS. marcescens28,6771228>256
8817FK. pneumoniae28,71566864
8829FS. marcescens28,688134256
8848FS. marcescens28,717125>256
8871FS. marcescens28,674173>256
8882FK. pneumoniae28,73152032
8884FE. coli28,704264256
8910FS. marcescens28,67224564
8946FS. marcescens28,674210256
8974FE. coli28,668394256
8982FS. marcescens28,666746128
9077FK. pneumoniae28,69023716
9079FS. marcescens28,70913116
9100FK. pneumoniae28,677298>256
9169FE. cloacae complex28,715298>256
9173FK. pneumoniae28,72237532
9176FS. marcescens28,699331128
9221FS. marcescens28,67712132
9236FK. pneumoniae28,677593128
9274FK. pneumoniae28,661414>256
9279FK. pneumoniae28,702310>256
9283FK. pneumoniae28,702796256
9288FK. pneumoniae28,66146064
9289FK. pneumoniae28,704393128
9294FK. pneumoniae28,72585516
9295FK. pneumoniae28,67053832
9297FK. pneumoniae28,720322256
9299FK. pneumoniae28,670173128
9300FS. marcescens28,725270128
9301FK. pneumoniae28,664372256
9304FK. pneumoniae28,676442256
9305FK. pneumoniae28,70633632
9310FK. pneumoniae28,707202128
i: intensity; a.u.: arbitrary units; * blaKPC + blaNDM-1; absent: intensity lower than 120 (a.u.) and there is no visual presence of the peak.
Table 2. Characteristics of blaKPC-negative bacteria used to inoculate blood cultures for KPC detection and results of spectra analysis by direct inspection and determination of intensity.
Table 2. Characteristics of blaKPC-negative bacteria used to inoculate blood cultures for KPC detection and results of spectra analysis by direct inspection and determination of intensity.
IDSpeciesHRM-qPCRKPC Peak m/zi [a.u.]Meropenem MIC (µg/mL)
7282FK. pneumoniaeNegativeAbsent964
7452FK. pneumoniaeNegativeAbsent4264
7523FK. pneumoniaeblaNDM-1Absent688
8113FK. oxytocaNegativeAbsent6516
8143FK. pneumoniaeNegativeAbsent3532
8144FK. pneumoniaeNegativeAbsent368
8152FK. pneumoniaeNegativeAbsent204
8155FK. pneumoniaeNegativeAbsent4416
8158FK. pneumoniaeNegativeAbsent54≤0.5
8165FK. aerogenesNegativeAbsent804
8215FE. cloacae complexNegativeAbsent344
8238FK. pneumoniaeNegativeAbsent1231
8300FK. oxytocaNegativeAbsent284
8311FK. pneumoniaeNegativeAbsent54≤0.5
8314FK. oxytocaNegativeAbsent164
8333FK. pneumoniaeblaNDM-1Absent11264
8348FK. pneumoniaeblaOXA-48likeAbsent318
8355FE. cloacae complexblaNDM-1Absent208
8378FE. cloacae complexblaNDM-1Absent248
8382FK. pneumoniaeblaNDM-1Absent94
8387FE. coliblaNDM-1Absent23≤0.5
8389FE. cloacae complexblaNDM-1Absent3116
8396FE. cloacae complexblaNDM-1Absent1214
8400FK. pneumoniaeNegativeAbsent9532
8411FK. pneumoniaeblaNDM-1Absent59128
8412FK. pneumoniaeblaNDM-1Absent3464
8414FE. cloacae complexblaNDM-1Absent284
8420FE. cloacae complexNegativeAbsent1012
8424FE. cloacae complexblaNDM-1Absent148
8432FK. pneumoniaeblaNDM-1Absent838
8435FK. aerogenesNegativeAbsent3832
8449FK. pneumoniaeblaNDM-1Absent328
8471FK. pneumoniaeblaNDM-1Absent191
8474FK. pneumoniaeblaOXA-48likeAbsent10716
8478FE. cloacae complexNegativeAbsent618
8481FK. pneumoniaeblaOXA-48likeAbsent5016
8485FK. oxytocablaNDM-1Absent11≤0.5
8486FE. cloacae complexblaNDM-1Absent2732
8889FE. cloacae complexblaNDM-1Absent4764
8890FE. cloacae complexblaOXA-48likeAbsent128
9110FK. pneumoniaeNegativeAbsent3964
9112FK. pneumoniaeblaNDM-1 + blaOXA-48likeAbsent97256
9291FK. pneumoniaeblaNDM-1Absent162
i: intensity; a.u.: arbitrary units; absent: intensity lower than 120 (a.u.) and there is no visual presence of the peak.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Moreira, N.K.; Wilhelm, C.M.; Echevarria, A.D.; Volpato, F.C.Z.; Wink, P.L.; Barth, A.L.; Caierão, J. Direct Detection of KPC Peak from Positive Blood Cultures Using MALDI-TOF MS: Are We There Yet? Antibiotics 2023, 12, 601. https://doi.org/10.3390/antibiotics12030601

AMA Style

Moreira NK, Wilhelm CM, Echevarria AD, Volpato FCZ, Wink PL, Barth AL, Caierão J. Direct Detection of KPC Peak from Positive Blood Cultures Using MALDI-TOF MS: Are We There Yet? Antibiotics. 2023; 12(3):601. https://doi.org/10.3390/antibiotics12030601

Chicago/Turabian Style

Moreira, Natália Kehl, Camila Mörschbächer Wilhelm, Aymê Duarte Echevarria, Fabiana Caroline Zempulski Volpato, Priscila Lamb Wink, Afonso Luís Barth, and Juliana Caierão. 2023. "Direct Detection of KPC Peak from Positive Blood Cultures Using MALDI-TOF MS: Are We There Yet?" Antibiotics 12, no. 3: 601. https://doi.org/10.3390/antibiotics12030601

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop