Next Article in Journal
Deep Learning Approach to Automatize TMTV Calculations Regardless of Segmentation Methodology for Major FDG-Avid Lymphomas
Previous Article in Journal
Prognostic Nutritional Index and Lung Immune Prognostic Index as Prognostic Predictors for Combination Therapies of Immune Checkpoint Inhibitors and Cytotoxic Anticancer Chemotherapy for Patients with Advanced Non-Small Cell Lung Cancer
Previous Article in Special Issue
Differentiation of Benign and Malignant Breast Lesions Using ADC Values and ADC Ratio in Breast MRI
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:

Quantification and Classification of Contrast Enhanced Ultrasound Breast Cancer Data: A Preliminary Study

Georgios S. Ioannidis
Michalis Goumenakis
Ioannis Stefanis
Apostolos Karantanas
1,2,4 and
Kostas Marias
Computational BioMedicine Laboratory (CBML), Foundation for Research and Technology—Hellas (FORTH), 70013 Heraklion, Greece
Department of Radiology, Medical School, University of Crete, 71003 Heraklion, Greece
Department of Electrical & Computer Engineering, Hellenic Mediterranean University, 71410 Heraklion, Greece
Department of Medical Imaging, University Hospital, 71003 Heraklion, Greece
Author to whom correspondence should be addressed.
Diagnostics 2022, 12(2), 425;
Submission received: 30 December 2021 / Revised: 2 February 2022 / Accepted: 3 February 2022 / Published: 6 February 2022
(This article belongs to the Special Issue Advancement in Breast Diagnostic and Interventional Radiology)


This study aimed to investigate which of the two frequently adopted perfusion models better describes the contrast enhanced ultrasound (CEUS) perfusion signal in order to produce meaningful imaging markers with the goal of developing a machine-learning model that can classify perfusion curves as benign or malignant in breast cancer data. Twenty-five patients with high suspicion of breast cancer were analyzed with exponentially modified Gaussian (EMG) and gamma variate functions (GVF). The adjusted R2 metric was the criterion for assessing model performance. Various classifiers were trained on the quantified perfusion curves in order to classify the curves as benign or malignant on a voxel basis. Sensitivity, specificity, geometric mean, and AUROC were the validation metrics. The best quantification model was EMG with an adjusted R2 of 0.60 ± 0.26 compared to 0.56 ± 0.25 for GVF. Logistic regression was the classifier with the highest performance (sensitivity, specificity, Gmean, and AUROC = 89.2 ± 10.7, 70.0 ± 18.5, 77.1 ± 8.6, and 91.0 ± 6.6, respectively). This classification method obtained similar results that are consistent with the current literature. Breast cancer patients can benefit from early detection and characterization prior to biopsy.

1. Introduction

Breast cancer constitutes the most common neoplasm in women and accounts for 30% of all recently diagnosed cancer in women [1]. Early detection and treatment are the most significant factors for reducing mortality and improving quality of life [2]. Mammography is the modality of choice for screening, which has been proven to reduce mortality due to breast cancer [3,4]. However, it exhibits low sensitivity (30–48%) in dense breasts [3], resulting in misdiagnosis and a high rate of false negative cases [5]. In addition, mammography exhibits limited diagnostic sensitivity (65.2%) for small lesions (≤1.0 cm) compared to ultrasound (85.1%), regardless of breast density [6]. Consequently, the majority of cancer-free women pay a high price in terms of false positive results, especially in the United States [7].
On the contrary, contrast enhanced ultrasound (CEUS) has been found to improve diagnostic efficacy compared to mammography or unenhanced ultrasound [8,9,10] because it also evaluates blood perfusion from tumor-induced neovascularity [11]. Published studies have shown that CEUS increases accuracy in distinguishing benign from malignant breast lesions having both qualitative [12] and quantitative [13] features. Furthermore, CEUS can improve the diagnosis of breast cancer in early stages because it can accurately differentiate benign from malignant lesions [14], and it can help to prevent unnecessary biopsies because of its high negative predictive value (100% on BIRADS III and 89.2% on BIRADS IV) [15]. Moreover, a recent systematic review of 51 studies with 4875 patients reported an high overall sensitivity reaching 88%.
Aside from CEUS and mammography, dynamic contrast-enhanced MRI (DCE-MRI) is a popular and important tool for breast lesion characterization [16]. The large number of freely accessible breast cancer datasets led to an increase in studies on breast lesion classifications [17,18,19,20,21,22], which exhibit high accuracy and AUROC (above 80%) [23,24,25]. Additionally, in a recent radiomic analysis including contrast enhanced mammography data, the authors presented a high AUROC varying from 89% to 96% for classifying benign and malignant breast lesions [26]. By conducting bibliography research on computed tomography (CT), it is evident that no efforts have been devoted to breast lesion classification but only to breast tissue classification (i.e., fatty, glandular, or dense) [27].
CEUS is a perfusion imaging technique that uses perflubutane as a contrast agent. Vascular perfusion can be visualized in real time through the contrast effect of 2–3 μm perflubutane microbubbles [28,29]. Therefore, the dynamic scan can produce parametric maps of hemodynamic parameters via pharmacokinetic modeling. Hemodynamic parameters or parametric maps are limited because of the high cost of the commercially available products and the lack of freely accessible quantification software. Therefore, the majority of studies aiming to classify breast lesions as benign or malignant utilized qualitative or semantic tumor characteristics such as the shape, diameter, and boundaries, as presented in a recent systematic review [11].
The aim of this study was two-fold. Firstly, we investigated which semi-quantitative perfusion model better described the behavior of CEUS curves. Secondly, we built a machine-learning framework that used quantitative and statistical characteristics of each CEUS curve to classify breast lesions as benign or malignant. To the best of our knowledge, we are the first to develop a classification model directly from perfusion curves with the aim of characterizing each image voxel as benign or malignant.

2. Materials and Methods

2.1. Study Population

From June 2019 to May 2021, we performed CEUS on a total of 27 breast lesions (BI-RADS IV) in 27 women at our local hospital. However, 2 patients were excluded due to patient motion during dynamic contrast enhanced ultrasound examination. Thus, 25 patients were included in the study. After each patient underwent ultrasound-guided biopsy, we determined that 14 had histopathologically proven breast cancer and the rest had benign lesions, mainly fibroadenomas. Patient characteristics can be found in Table 1.
The inclusion criteria were women who underwent conventional US for (a) a screening procedure or (b) the characterization of a known palpable lesion or (c) a detectable finding from the US that required biopsy. The exclusion criteria were women with (a) known metastatic breast cancer diagnosed by biopsy or (b) US findings that had not been confirmed by biopsy.

2.2. Imaging Protocol

This study was conducted at the US department of “Venizeleio” General Hospital with an iU22 Ultrasound System (Philips, Bothell, WA, USA). A CEUS study was performed involving women with a suspicious ultrasound breast finding, for which a biopsy was required. The sample was sent for pathological examination, and the result were compared to the contrast enhancement pattern of the lesion. The pathology report served as the ground truth for the development of the CEUS classification model.

2.3. Data Pre-Processing

As raw DICOM CEUS data are usually stored in a multi-channel video format, the first step was to convert videos into grayscale using the luminance algorithm [30]. The next step was to extract B-mode and CEUS sequences with automated video-cropping techniques using the DICOM tag “SequenceOfUltrasoundRegions”. Subsequently, videos were temporally sub-sampled to 1 s resolution to facilitate the quantitative model-fitting process. In order to avoid motion artifacts, videos were registered in the temporal domain using the pyStackReg library (accessed date: 10 October 2021) for Python [31]. Lastly, a clinical expert delineated the suspicious regions from which dynamic signal curves were extracted. The pre-processing steps are summarized in Figure 1.

2.4. CEUS Quantification, Parametric Mapping

In order to extract quantitative voxel-by-voxel markers from the perfusion curves, two functions were used to fit each curve. The first was the exponentially modified Gaussian (EMG) function, presented in Equation (1), and the second was the Gamma variate function (GVF), presented in Equation (2):
F ( t ) = a c 2 π 2 d exp ( b t d + c 2 2 d 2 ) [ d | d | e r f ( b t 2 c + c 2 d ) ]
where e r f (t) is the Gaussian error function, e r f ( t ) = t t e x 2 d x .
G ( t ) =   A   t a   exp ( t b )
In both equations, the unknown parameters a , b, c, and d for Equation (1) and A, a, and b for Equation (2) do not have physiological meaning; thus, optimization was performed in the range of real numbers using the Levenberg–Marquardt algorithm [32].
After fitting Equations (1) and (2) to the perfusion curves per voxel, a variety of semi-quantitative parameters could be computed using the first derivative of each of the functions such as: wash-in, wash-out, time to peak (TTP), and time to maximal slope (TMSP). More precisely, wash-in and wash-out describe the rate of change of contrast’s agent inflow and outflow, respectively. Mathematically, this is described as the maximum and minimum value of the derivative, respectively. TTP is the time required for Equations (1) and (2) to reach their maximum value. TMSP reflects the time point of maximum wash-in. Computationally, TMSP is the time required for the first derivative of Equation (1) to reach its maximum value. These equations have been used to describe the dispersion of a bolus as it passes through a series of compartments, mainly for perfusion modeling [33,34,35,36]. Further information and graphical representation of the quantitative markers for EMG can be found in [37,38].
Aside from the aforementioned imaging markers (wash-in, wash-out, TTP, and TMSP), we calculated the area under the perfusion curve (AUC) and the mean slope of increase (MSI). Assuming C t ( t ) to be the perfusion curve and t 0 to be the final time of the baseline, MSI was computed by the following formula:
M S I = 1 N t 1 = t 0 t N = T T P C t ( t i + 1 ) C t ( t i )

2.5. Goodness of Fit

The criterion chosen to assess the goodness of fit between model function and data (CEUS curves) was the adjusted R2 ( R ¯ 2 ), a generalized metric that is based on R-squared ( R 2 ) . This metric is suitable for the purpose of this study since it accounts for both the number of temporal points of the curve (N) and the number of explanatory variables (p) of the model [39]. R ¯ 2 is given in Equation (4) and its values range from 0 to 1.
R ¯ 2 = 1 ( 1 R 2 ) N 1 N p 1

2.6. Machine-Learning Pipeline

2.6.1. Feature Extraction

Since our dataset was limited in the number of enrolled patients, classification analysis was performed on a voxel-by-voxel basis using the perfusion curves from each ROI delineated by the expert. For the differentiation between benign and malignant tissue types from each ROI, two sets of features depending on the fitting equation were extracted. The first feature set (EMG set) included both statistical features and semi-quantitative metrics stemming from the EMG-fitted curve, as previously described. Thus, the first set of features (EMG set) included wash-in, wash-out, TTP, TMSP, AUC, MSI, mean, median, max, and standard deviation(SD). The GVF set was the same as the first and differed only in that the first 6 features were computed from the fitted GVF function on CEUS curves.

2.6.2. Feature Selection

In order to make our model more robust and reliable, the minimum redundancy maximum relevance feature-selection algorithm from the pymrmr library [40] was used to identify the most relevant patterns in the training set.
Minimum redundancy maximum relevance (MRMR) is a supervised feature-selection algorithm (i.e., uses both the input features and output class labels). The aim of MRMR is to find the set of features that best matches the output class labels while minimizing redundancy among selected features. In order to find the best match between the features and the output labels, MRMR usually deploys the mutual-information framework. Further information can be found in [40].

2.6.3. Classification

Differentiation between benign and malignant breast lesions was achieved using a variety of classifiers obtained from the scikit-learn library [41] such as: quadratic discriminant analysis (QDA), Gaussian naïve Bayes (GaussianNB), AdaBoost, random forest, k-nearest neighbors (KNeighbors; k = 3), and logistic regression. We used a support vector machine (SVM) with the radial basis function kernel (RBF).
In the context of perfusion curves differentiation (benign or malignant), all classifiers were trained in a 10-fold cross-validation scheme on the extracted curve features. Data stratification was applied on a patient basis across folds, avoiding sample selection bias and overfitting of models.

2.6.4. Model Evaluation Metrics

In order to evaluate the classification performance, the standard deviations of several metrics (for each fold) were calculated on the unseen testing sets. The metrics assessing performance included s e n s i t i v i t y = T P T P + F N , s p e c i f i c i t y = T N F P + T N , and geometric mean G m e a n = s e n s i t i v i t y × s p e c i f i c i t y . TP, TN, FP, and FN stand for true positive, true negative, false positive, and false negative, respectively. The geometric mean aggregates both sensitivity and specificity, and it is suitable for imbalanced datasets. Moreover, the area under the receiver operating characteristic (ROC) curve (AUROC) was calculated. The ROC curve is a two-dimensional graph in which the y-axis indicates the true-positive rate and the x-axis indicates the false-positive rate. It has been extensively used to evaluate medical decision-making and machine-learning systems. Please note that AUROC differs from AUC, which is the area under the perfusion curve.

3. Results

3.1. Goodness of Fit

The mean value ± the standard deviation of the goodness of fit metric R ¯ 2 calculated from all voxels inside the suspicious areas for the two models, EMG and GVF, was 0.60 ± 0.26 and 0.56 ± 0.25, respectively.
Wash-in- and AUC-produced parametric maps after voxel-by-voxel fitting to the CEUS perfusion data using Equations (1) and (2) are presented in Figure 2 and Figure 3. The parametric maps of a benign fibroadenoma patient are shown in Figure 4 and Figure 5.

3.2. Machine Learning

3.2.1. Feature Selection

The best selected features from the EMG feature set were wash-in, AUC, SD, max, TTP, and mean. In addition, the best selected features from the GVF set were WIN, AUC, SD, Max, mean and wash-out.

3.2.2. Classification Results

The metrics for the classification for benign and malignant tissue types are summarized in Table 2 for the EMG feature set and in Table 3 for the GVF feature set.

4. Discussion

In this work, two functions were used to quantify CEUS perfusion signals into meaningful imaging markers with the goal of building a machine-learning model that can classify the perfusion curves as benign or malignant.
Concerning the quantification of the CEUS curves, the model that better described CEUS perfusion was found to be the exponentially modified Gaussian function (EMG) according to the adjusted R2 criterion. This is a logical outcome since the EMG function has more degrees of freedom (four parameters) in fitting the CEUS data than the gamma variate function (three parameters). In addition, GVF model was found to be more sensitive to noise compared to EMG as it failed to quantify a non-negligible number of voxels. This could be attributed either to high level of noise in the temporal data or to numerical errors in the computation of the derivative, which is prerequisite for obtaining the wash-in parameter. This can be easily observed in the wash-in values (Figure 2C,E and Figure 4C,E) where several voxels appear without color.
Quantitative models aim to describe physiology and use appropriate simplifications such as the number of compartments to derive a mathematical model. On the contrary, semi-quantitative methods are data-driven and do not attempt to model the underlying physiology. The major advantage of using semi-quantitative models for breast perfusion evaluation is the exclusion of the arterial input function (AIF). In addition, semi-quantitative models are less prone to numerical errors, having a one-step process as opposed to a more complex workflow [38]. In such acquisitions where the imaging field of view is focused on the lesion, and the area of the artery is frequently non-visible. This is a different approach from the widely used perfusion models such as the extended Tofts model, Patlak’s model, etc. [42,43], where the AIF is mandatory for quantification.
Regarding the classification analysis, the EMG feature set (Table 2) expectedly performed better than the GVF feature set (Table 3) according to the AUROC metric. This metric was preferred over the others because it is indicative of the separability between classes because it takes into account sensitivity and specificity. Keeping that in mind, the logistic regression classifier exhibited the best performance with both feature sets. The geometric mean was incorporated in our study since it is a more suitable metric for handling unbalanced datasets [44,45]. Our machine-learning method is based on training classifiers with features obtained from every voxel inside the breast lesion, with 22,446 benign and 65,762 malignant voxels.
CEUS has the potential to improve the diagnostic efficacy of mammography. Notably, the results of our study on breast cancer classification are in line with a recent review of 51 CEUS-based studies that reported a mean sensitivity of 0.88. and a mean AUROC of 0.91 [11].
To the best of our knowledge, this is the first study to distinguish benign and malignant tissue types using CEUS perfusion curves on a voxel-by-voxel basis. A similar approach was investigated by Ta et al., who reported an ACC of 100% in a cohort of 10 rat tumors [46]. As a result, our study lacks a point of comparison. There are similar studies, such as those on region-based quantitative classification analysis; Kapetas et al. reported an AUROC of 0.812% in a cohort of 65 patients [47], Janu et al. reported an AUROC of 78% in a cohort of 230 patients [48], and Park et al. reported an AUROC of 0.841 in a cohort of 98 patients [49].
Although this study exhibited promising results, there is room for improvement. For example, a larger patient cohort would make our analysis more statistically robust. In addition to the dynamic data that were used, a larger patient cohort would introduce more spatial and textural information to the model such as radiomics.

5. Conclusions

In conclusion, we presented a novel approach based on dynamic CEUS signal curves for the classification of benign and malignant breast lesions. The EMG feature set exhibited the highest performance (AUROC 91%) in distinguishing malignant and benign lesions on a voxel-by-voxel basis. This framework has the potential to evolve into an objective diagnostic support tool using the dynamic signal characteristic of CEUS, reducing unnecessary biopsies in breast cancer screening programs.

Author Contributions

G.S.I., CEUS quantification software, classification analysis, manuscript writing; M.G., data curation, ROI delineation, manuscript writing and revision; I.S., CEUS imaging platform for ROI measurements development; A.K. and K.M., supervision and manuscript revision. All authors have read and agreed to the published version of the manuscript.


G.S.I. and I.S. acknowledge the support by the Stavros Niarchos Foundation within the framework of the project ARCHERS (“Advancing Young Researchers’ Human Capital in Cutting Edge Technologies in the Preservation of Cultural Heritage and the Tackling of Societal Challenges”).

Institutional Review Board Statement

All patient recruitment and sample collection was performed with the appropriate human subjects protocol approval from the Institutional Review Board of Venizeleio General Hospital of Heraklion, Crete (session: 9, date: 26 June 2019).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

The data are available upon a reasonable request to the corresponding author.

Conflicts of Interest

The authors declare no conflict of interest.


  1. Siegel, R.L.; Miller, K.D.; Jemal, A. Cancer statistics, 2019. CA Cancer J. Clin. 2019, 69, 7–34. [Google Scholar] [CrossRef] [Green Version]
  2. Harbeck, N.; Gnant, M. Breast cancer. Lancet 2017, 389, 1134–1150. [Google Scholar] [CrossRef]
  3. Hooley, R.J.; Greenberg, K.L.; Stackhouse, R.M.; Geisel, J.L.; Butler, R.S.; Philpotts, L.E. Screening US in patients with mammographically dense breasts: Initial experience with Connecticut public act 09-41. Radiology 2012, 265, 59–69. [Google Scholar] [CrossRef]
  4. Lee, S.H.; Yi, A.; Jang, M.J.; Chang, J.M.; Cho, N.; Moon, W.K. Supplemental screening breast us in women with negative mammographic findings: Effect of routine axillary scanning. Radiology 2018, 286, 830–837. [Google Scholar] [CrossRef] [PubMed]
  5. Sprague, B.L.; Stout, N.K.; Schechter, C.; Van Ravesteyn, N.T.; Cevik, M.; Alagoz, O.; Lee, C.I.; Van Den Broek, J.J.; Miglioretti, D.L.; Mandelblatt, J.S.; et al. Benefits, harms, and cost-effectiveness of supplemental ultrasonography screening for women with dense breasts. Ann. Intern. Med. 2015, 162, 157–166. [Google Scholar] [CrossRef] [Green Version]
  6. Chen, H.L.; Zhou, J.Q.; Chen, Q.; Deng, Y.C. Comparison of the sensitivity of mammography, ultrasound, magnetic resonance imaging and combinations of these imaging modalities for the detection of small (≤2 cm) breast cancer. Medicine 2021, 100, e26531. [Google Scholar] [CrossRef] [PubMed]
  7. Jacobsen, K.K.; O’Meara, E.S.; Key, D.; Buist, D.S.M.; Kerlikowske, K.; Vejborg, I.; Sprague, B.L.; Lynge, E.; Von Euler-Chelpin, M. Comparing sensitivity and specificity of screening mammography in the United States and Denmark. Int. J. Cancer 2015, 137, 2198–2207. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  8. Shao, S.H.; Li, C.X.; Yao, M.H.; Li, G.; Li, X.; Wu, R. Incorporation of contrast-enhanced ultrasound in the differential diagnosis for breast lesions with inconsistent results on mammography and conventional ultrasound. Clin. Hemorheol. Microcirc. 2020, 74, 463–473. [Google Scholar] [CrossRef]
  9. Xiao, X.; Dong, L.; Jiang, Q.; Guan, X.; Wu, H.; Luo, B. Incorporating Contrast-Enhanced Ultrasound into the BI-RADS Scoring System Improves Accuracy in Breast Tumor Diagnosis: A Preliminary Study in China. Ultrasound Med. Biol. 2016, 42, 2630–2638. [Google Scholar] [CrossRef]
  10. Xiao, X.; Ou, B.; Yang, H.; Wu, H.; Luo, B. Breast Contrast-Enhanced Ultrasound: Is a Scoring System Feasible? A Preliminary Study in China. PLoS ONE 2014, 9, e105517. [Google Scholar] [CrossRef]
  11. Zhou, S.C.; Le, J.; Zhou, J.; Huang, Y.X.; Qian, L.; Chang, C. The Role of Contrast-Enhanced Ultrasound in the Diagnosis and Pathologic Response Prediction in Breast Cancer: A Meta-analysis and Systematic Review. Clin. Breast Cancer 2020, 20, e490–e509. [Google Scholar] [CrossRef] [PubMed]
  12. Yuan, Z.; Quan, J.; Yunxiao, Z.; Jian, C.; Zhu, H.; Liping, G. Diagnostic value of contrast-enhanced ultrasound parametric imaging in breast tumors. J. Breast Cancer 2013, 16, 208–213. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  13. Saracco, A.; Szabó, B.K.; Aspelin, P.; Leifland, K.; Wilczek, B.; Celebioglu, F.; Axelsson, R. Differentiation between benign and malignant breast tumors using kinetic features of real-time harmonic contrast-enhanced ultrasound. Acta Radiol. 2012, 53, 382–388. [Google Scholar] [CrossRef] [PubMed]
  14. Du, Y.R.; Wu, Y.; Chen, M.; Gu, X.G. Application of contrast-enhanced ultrasound in the diagnosis of small breast lesions. Clin. Hemorheol. Microcirc. 2018, 70, 291–300. [Google Scholar] [CrossRef] [PubMed]
  15. Zhang, X.L.; Guan, J.; Li, M.Z.; Liu, M.J.; Guo, Y.; Zheng, Y.L.; Yang, Z.; Yang, J.Y. Adjunctive targeted contrast-enhanced ultrasonography for the work-up of Breast Imaging Reporting and Data System category 3 and 4 lesions. J. Med. Imaging Radiat. Oncol. 2016, 60, 485–491. [Google Scholar] [CrossRef]
  16. Schnall, M.D.; Blume, J.; Bluemke, D.A.; DeAngelis, G.A.; DeBruhl, N.; Harms, S.; Heywang-Köbrunner, S.H.; Hylton, N.; Kuhl, C.K.; Pisano, E.D.; et al. Diagnostic architectural and dynamic features at breast MR imaging: Multicenter study. Radiology 2006, 238, 42–53. [Google Scholar] [CrossRef]
  17. Ikeda, D.M.; Hylton, N.M.; Kinkel, K.; Hochman, M.G.; Kuhl, C.K.; Kaiser, W.A.; Weinreb, J.C.; Smazal, S.F.; Degani, H.; Viehweg, P.; et al. Development, standardization, and testing of a lexicon for reporting contrast-enhanced breast magnetic resonance imaging studies. J. Magn. Reson. Imaging 2001, 13, 889–895. [Google Scholar] [CrossRef]
  18. Fusco, R.; Sansone, M.; Filice, S.; Carone, G.; Amato, D.M.; Sansone, C.; Petrillo, A. Pattern Recognition Approaches for Breast Cancer DCE-MRI Classification: A Systematic Review. J. Med. Biol. Eng. 2016, 36, 449. [Google Scholar] [CrossRef] [Green Version]
  19. Fusco, R.; Sansone, M.; Filice, S.; Granata, V.; Catalano, O.; Amato, D.M.; Di Bonito, M.; D’Aiuto, M.; Capasso, I.; Rinaldo, M.; et al. Integration of DCE-MRI and DW-MRI Quantitative Parameters for Breast Lesion Classification. BioMed Res. Int. 2015, 2015, 237863. [Google Scholar] [CrossRef]
  20. Tzacheva, A.A.; Najarian, K.; Brockway, J.P. Breast cancer detection in gadolinium-enhanced MR images by static region descriptors and neural networks. J. Magn. Reson. Imaging 2003, 17, 337–342. [Google Scholar] [CrossRef]
  21. Agner, S.C.; Soman, S.; Libfeld, E.; McDonald, M.; Thomas, K.; Englander, S.; Rosen, M.A.; Chin, D.; Nosher, J.; Madabhushi, A. Textural Kinetics: A Novel Dynamic Contrast-Enhanced (DCE)-MRI Feature for Breast Lesion Classification. J. Digit. Imaging 2011, 24, 446. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  22. Schlossbauer, T.; Leinsinger, G.; Wismuller, A.; Lange, O.; Scherr, M.; Meyer-Baese, A.; Reiser, M. Classification of Small Contrast Enhancing Breast Lesions in Dynamic Magnetic Resonance Imaging Using a Combination of Morphological Criteria and Dynamic Analysis Based on Unsupervised Vector-Quantization. Investig. Radiol. 2008, 43, 56. [Google Scholar] [CrossRef]
  23. Amit, G.; Ben-Ari, R.; Hadad, O.; Monovich, E.; Granot, N.; Hashoul, S. Classification of Breast MRI Lesions Using Small-Size Training Sets: Comparison of Deep Learning Approaches. In Proceedings of the SPIE 10134, Medical Imaging 2017: Computer-Aided Diagnosis, Orlando, FL, USA, 11–16 February 2017; p. 101341H. [Google Scholar]
  24. Zheng, Y.; Baloch, S.; Englander, S.; Schnall, M.D.; Shen, D. Segmentation and Classification of Breast Tumor Using Dynamic Contrast-Enhanced MR Images. Med. Image Comput. Comput. Assist. Interv. 2007, 10, 393. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  25. McLaren, C.E.; Chen, W.P.; Nie, K.; Su, M.Y. Prediction o Malignant Breast Lesions from MRI Features: A Comparison of Artificial Neural Network and Logistic Regression Techniques. Acad. Radiol. 2009, 16, 842. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  26. Wang, S.; Mao, N.; Duan, S.; Li, Q.; Li, R.; Jiang, T.; Wang, Z.; Xie, H.; Gu, Y. Radiomic Analysis of Contrast-Enhanced Mammography with Different Image Types: Classification of Breast Lesions. Front. Oncol. 2021, 11, 629. [Google Scholar] [CrossRef]
  27. Nelson, T.R.; Cerviño, L.I.; Boone, J.M.; Lindfors, K.K. Classification of breast computed tomography data. Med. Phys. 2008, 35, 1078–1086. [Google Scholar] [CrossRef] [Green Version]
  28. Shima, H.; Okuno, T.; Nakamura, T.; Noro, A.; Noma, M.; Sato, M.; Kaga, T.; Mituzuka, Y.; Kamei, K.; Imayoshi, Y.; et al. Comparing the extent of breast cancer tumors through contrast-enhanced ultrasound vs B-mode, opposed with pathology: Evergreen study. Breast Cancer 2021, 28, 405–413. [Google Scholar] [CrossRef]
  29. Miyamoto, Y.; Ito, T.; Takada, E.; Omoto, K.; Hirai, T.; Moriyasu, F. Efficacy of Sonazoid (perflubutane) for contrast-enhanced ultrasound in the differentiation of focal breast lesions: Phase 3 multicenter clinical trial. Am. J. Roentgenol. 2014, 202, W400–W407. [Google Scholar] [CrossRef]
  30. Kanan, C.; Cottrell, G.W. Color-to-Grayscale: Does the Method Matter in Image Recognition? PLoS ONE 2012, 7, e29740. [Google Scholar] [CrossRef] [Green Version]
  31. Thévenaz, P.; Ruttimann, U.E.; Unser, M. A pyramid approach to subpixel registration based on intensity. IEEE Trans. Image Process. 1998, 7, 27–41. [Google Scholar] [CrossRef] [Green Version]
  32. Marquardt, D.W. An algorithm for least-squares estimation of nonlinear parameters. J. Soc. Ind. Appl. Math. 1963, 11, 431–441. [Google Scholar] [CrossRef]
  33. Sourbron, S.; Luypaert, R.; Morhard, D.; Seelos, K.; Reiser, M.; Peller, M. Deconvolution of bolus-tracking data: A comparison of discretization methods. Phys. Med. Biol. 2007, 52, 6761–6778. [Google Scholar] [CrossRef]
  34. Ioannidis, G.S.; Christensen, S.; Nikiforaki, K.; Trivizakis, E.; Perisinakis, K.; Hatzidakis, A.; Karantanas, A.; Reyes, M.; Lansberg, M.; Marias, K. Cerebral CT Perfusion in Acute Stroke: The Effect of Lowering the Tube Load and Sampling Rate on the Reproducibility of Parametric Maps. Diagnostics 2021, 11, 1121. [Google Scholar] [CrossRef] [PubMed]
  35. Madsen, K.; Nielsen, H.B.; Tingleff, O. Methods for Non-Linear Least Squares Problems; University of Denmark: Copenhagen, Denmark, 2004. [Google Scholar]
  36. Madsen, M.T. A simplified formulation of the gamma variate function. Phys. Med. Biol. 1992, 37, 1597–1600. [Google Scholar] [CrossRef]
  37. Ioannidis, G.S.; Nikiforaki, K.; Karantanas, A. Correlation of DWI and DCE MRI Markers for the Study of Perfusion of the Lower Limb in Patients with Peripheral Arterial Disease. In Proceedings of the 2019 IEEE 19th International Conference on Bioinformatics and Bioengineering (BIBE), Athens, Greece, 28–30 October 2019; IEEE: Piscataway, NJ, USA, 2019; pp. 433–438. [Google Scholar]
  38. Ioannidis, G.S.; Maris, T.G.; Nikiforaki, K.; Karantanas, A.; Marias, K. Investigating the Correlation of Ktrans with Semi-Quantitative MRI Parameters Towards More Robust and Reproducible Perfusion Imaging Biomarkers in Three Cancer Types. IEEE J. Biomed. Heal. Inform. 2019, 23, 1855–1862. [Google Scholar] [CrossRef] [PubMed]
  39. Ioannidis, G.S.; Marias, K.; Galanakis, N.; Perisinakis, K.; Hatzidakis, A.; Tsetis, D.; Karantanas, A.; Maris, T.G. A correlative study between diffusion and perfusion MR imaging parameters on peripheral arterial disease data. Magn. Reson. Imaging 2019, 55, 26–35. [Google Scholar] [CrossRef] [PubMed]
  40. Peng, H.; Long, F.; Ding, C. Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy. IEEE Trans. Pattern Anal. Mach. Intell. 2005, 27, 1226–1238. [Google Scholar] [CrossRef] [PubMed]
  41. Pedregosa, F.; Varoquaux, G.; Gramfort, A.; Michel, V.; Thirion, B.; Grisel, O.; Blondel, M.; Müller, A.; Nothman, J.; Louppe, G.; et al. Scikit-learn: Machine Learning in Python. J. Mach. Learn. Res. 2012, 12, 2825–2830. [Google Scholar]
  42. Patlak, C.S.; Blasberg, R.G.; Fenstermacher, J.D. Graphical Evaluation of Blood-to-Brain Transfer Constants from Multiple-Time Uptake Data. J. Cereb. Blood Flow Metab. 1983, 3, 1–7. [Google Scholar] [CrossRef] [PubMed]
  43. Sourbron, S.P.; Buckley, D.L. On the scope and interpretation of the Tofts models for DCE-MRI. Magn. Reson. Med. 2011, 66, 735–745. [Google Scholar] [CrossRef]
  44. Bekkar, M.; Kheliouane Djemaa, D.; Akrouf Alitouche, D. Evaluation Measures for Models Assessment over Imbalanced Data Sets. J. Inf. Eng. Appl. 2013, 3, 27–38. [Google Scholar]
  45. Batuwita, R.; Palade, V. A New Performance Measure for Class Imbalance Learning. Application to Bioinformatics Problems. In Proceedings of the 2009 International Conference on Machine Learning and Applications, Miami, FL, USA, 13–15 December 2009; IEEE: Piscataway, NJ, USA, 2009; pp. 545–550. [Google Scholar]
  46. Ta, C.N.; Kono, Y.; Barback, C.V.; Mattrey, R.F.; Kummel, A.C. Automating tumor classification with pixel-by-pixel contrast-enhanced ultrasound perfusion kinetics. J. Vac. Sci. Technol. B Nanotechnol. Microelectron. Mater. Process. Meas. Phenom. 2012, 30, 02C103. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  47. Kapetas, P.; Clauser, P.; Woitek, R.; Wengert, G.J.; Lazar, M.; Pinker, K.; Helbich, T.H.; Baltzer, P.A.T. Quantitative Multiparametric Breast Ultrasound: Application of Contrast-Enhanced Ultrasound and Elastography Leads to an Improved Differentiation of Benign and Malignant Lesions. Investig. Radiol. 2019, 54, 257–264. [Google Scholar] [CrossRef]
  48. Janu, E.; Krikavova, L.; Little, J.; Dvorak, K.; Brancikova, D.; Jandakova, E.; Pavlik, T.; Kovalcikova, P.; Kazda, T.; Valek, V. Prospective evaluation of contrast-enhanced ultrasound of breast BI-RADS 3–5 lesions. BMC Med. Imaging 2020, 20, 66. [Google Scholar] [CrossRef] [PubMed]
  49. Park, A.Y.; Kwon, M.; Woo, O.H.; Cho, K.R.; Park, E.K.; Cha, S.H.; Song, S.E.; Lee, J.H.; Cha, J.; Son, G.S.; et al. A prospective study on the value of ultrasound microflow assessment to distinguish malignant from benign solid breast masses: Association between ultrasound parameters and histologic microvessel densities. Korean J. Radiol. 2019, 20, 759–772. [Google Scholar] [CrossRef]
Figure 1. Data pre-processing workflow.
Figure 1. Data pre-processing workflow.
Diagnostics 12 00425 g001
Figure 2. Wash-in parametric map calculated with EMG (C) and GVF (E) of a cancer patient. (A) ROI in B-mode, (B) CEUS mode at time to peak, and (D) EMG and GVF fitted to the mean ROI signal over time.
Figure 2. Wash-in parametric map calculated with EMG (C) and GVF (E) of a cancer patient. (A) ROI in B-mode, (B) CEUS mode at time to peak, and (D) EMG and GVF fitted to the mean ROI signal over time.
Diagnostics 12 00425 g002
Figure 3. AUC parametric map calculated with EMG (C) and GVF (E) of a cancer patient. (A) ROI in B-mode, (B) CEUS mode at time to peak, and (D) EMG and GVF fitted to the mean ROI signal over time.
Figure 3. AUC parametric map calculated with EMG (C) and GVF (E) of a cancer patient. (A) ROI in B-mode, (B) CEUS mode at time to peak, and (D) EMG and GVF fitted to the mean ROI signal over time.
Diagnostics 12 00425 g003
Figure 4. Wash-in parametric map calculated with EMG (C) and GVF (E) of a benign case. (A) ROI in B-mode, (B) CEUS mode at time to peak, and (D) EMG and GVF fitted to the mean ROI signal over time.
Figure 4. Wash-in parametric map calculated with EMG (C) and GVF (E) of a benign case. (A) ROI in B-mode, (B) CEUS mode at time to peak, and (D) EMG and GVF fitted to the mean ROI signal over time.
Diagnostics 12 00425 g004
Figure 5. AUC parametric map calculated with EMG (C) and GVF (E) of a benign case. (A) ROI in B-mode, (B) CEUS mode at time to peak, and (D) EMG and GVF fitted to the mean ROI signal over time.
Figure 5. AUC parametric map calculated with EMG (C) and GVF (E) of a benign case. (A) ROI in B-mode, (B) CEUS mode at time to peak, and (D) EMG and GVF fitted to the mean ROI signal over time.
Diagnostics 12 00425 g005
Table 1. Patient characteristics.
Table 1. Patient characteristics.
Total patients25
Age (in years)
Histopathological grades
  Benign lesions14
  Malignant lesions11
  Number of benign voxels22,446
  Number of malignant voxels65,762
Table 2. Classification metrics ± standard deviation per classifier using EMG feature set.
Table 2. Classification metrics ± standard deviation per classifier using EMG feature set.
ClassifiersSensitivity SpecificityGmeanAUROC
QDA69.7 ± 20.888.5 ± 12.076.8 ± 10.989.7 ± 5.4
GaussianNB69.0 ± 22.190.7 ± 11.277.2 ± 12.589.8 ± 7.4
AdaBoost 87.4 ± 11.962.6 ± 21.572.2 ± 11.487.9 ± 9.7
Random forest 88.3 ± 11.870.3 ± 17.577.6 ± 9.287.1 ± 9.8
KNeighbors85.4 ± 11.555.6 ± 15.167.9 ± 9.176.7 ± 9.2
Logistic regression89.2 ± 10.770.0 ± 18.577.1 ± 8.691.0 ± 6.6
SVM88.1 ± 11.468.6 ± 18.676.7 ± 11.1 87.9 ± 10.8
Table 3. Classification metrics ± standard deviation per classifier using GVF feature set.
Table 3. Classification metrics ± standard deviation per classifier using GVF feature set.
ClassifiersSensitivity SpecificityGmeanAUROC
QDA70.4 ± 21.583.8 ± 18.674.2 ± 12.587.6 ± 7.1
GaussianNB67.8 ± 23.087.6 ± 19.374.2 ± 14.488.8 ± 6.6
AdaBoost 89.2 ± 11.357.1 ± 20.169.5 ± 10.686.6 ± 9.8
Random forest 90.4 ± 10.160.8 ± 25.871.9 ± 14.386.5 ± 11.8
KNeighbors86.6 ± 9.852.9 ± 15.866.3 ± 8.776.1 ± 7.5
Logistic regression88.5 ± 13.366.3 ± 22.474.6 ± 11.889.0 ± 11.0
SVM89.2 ± 10.356.2 ± 25.168.1 ± 15.985.8 ± 9.2
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Ioannidis, G.S.; Goumenakis, M.; Stefanis, I.; Karantanas, A.; Marias, K. Quantification and Classification of Contrast Enhanced Ultrasound Breast Cancer Data: A Preliminary Study. Diagnostics 2022, 12, 425.

AMA Style

Ioannidis GS, Goumenakis M, Stefanis I, Karantanas A, Marias K. Quantification and Classification of Contrast Enhanced Ultrasound Breast Cancer Data: A Preliminary Study. Diagnostics. 2022; 12(2):425.

Chicago/Turabian Style

Ioannidis, Georgios S., Michalis Goumenakis, Ioannis Stefanis, Apostolos Karantanas, and Kostas Marias. 2022. "Quantification and Classification of Contrast Enhanced Ultrasound Breast Cancer Data: A Preliminary Study" Diagnostics 12, no. 2: 425.

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop