MfdcModel: A Novel Classification Model for Classification of Benign and Malignant Breast Tumors in Ultrasound Images

Abstract

Automatic classification of benign and malignant breast ultrasound images is an important and challenging task to improve the efficiency and accuracy of clinical diagnosis of breast tumors and reduce the rate of missed and misdiagnosis. The task often requires a large amount of data to train. However, it is difficult to obtain medical images, which contradicts the large amount of data needed to obtain good diagnostic models for training. In this paper, a novel classification model for the classification of breast tumors is proposed to improve the performance of diagnosis models trained by small datasets. The method integrates three features from medical features extracted from segmented images, features selected from the pre-trained ResNet101 output by principal component analysis (PCA), and texture features. Among the medical features that are used to train the naive Bayes (NB) classifier, and the PCA-selected features are used to train the support vector machine (SVM) classifier. Subsequently, the final results of boosting are obtained by weighting the classifiers. A five-fold cross-validation experiment yields an average accuracy of 89.17%, an average precision of 90.00%, and an average AUC value of 0.95. According to the experimental results, the proposed method has better classification accuracy compared to the accuracy obtained by other models trained on only small datasets. This approach can serve as a reliable second opinion for radiologists, and it can also provide useful advice for junior radiologists who do not have sufficient clinical experience.

1. Introduction

Breast cancer is one of the worst diseases threatening women’s health, ranking first among female malignant tumors [1]. Global cancer statistics in 2020 showed that there were about 2.3 million new female breast cancer cases worldwide each year. Additionally, breast cancer is one of the most commonly diagnosed cancers in women [2]. Clinical medical research showed that early detection of breast cancer will be of great significance for helping breast cancer patients’ recovery and survival [3]. There are many breast imaging screening techniques, such as X-ray [4], CT [5], MRI [6], ultrasonic [7], etc. Due to the advantages of ultrasonic, such as safe, non-invasive, non-radiation, it is suitable for any age and is not affected by the type of breast glands. What is more, it can also observe the lesions at multiple angles and in real-time dynamics [4,7]. Therefore, breast ultrasound examination is one of the most basic methods for diagnosing breast lesions and has been widely used in breast cancer screening [7,8].

With the continuous development of computer technology, the application of computer-aided technology in medical imaging has been greatly developed and used [9]. Breast tumor recognition and benign and malignant classification prediction models based on ultrasound images have become a research hotspot in the field of computer-aided diagnosis (CAD). The main research contents of ultrasound image classification include ultrasound image pre-processing, feature extraction, feature selection, benign/malignant tumor discrimination, and disease prediction modules. Among them, feature extraction is one of the most important steps. The features extracted in traditional methods are often extracted texture features or combined with other features [10]. Moon et al. [11] extracted as textural, morphological, and descriptor features, and provided these features to a classifier to classify tumors. Uniyal et al. [12] generated malignancy maps based on estimated cancer likelihood from ultrasound radiofrequency (RF) time series to classify malignant breast and benign tumors. Flores et al. [13] improved the accuracy of ultrasound classification of breast tumors by analyzing different morphological and textural features. Liang et al. [14] estimated the edges by using the Laplacian of Gaussian (LoG) method. The box-counting method was employed to estimate the texture images and extract the values of the extracted features, such as average, standard deviation, skewness, and kurtosis. Finally, we used logistic regression to classify breast ultrasound as benign or malignant tumors. Wei et al. [15] proposed an effective method of combining textural features and morphological (the compactness, the elliptic compactness, the mean, and the variance of the radial distance spectrum) features to improve the accuracy of breast ultrasound image classification. The extracted texture features were used for the training of an SVM classifier, and the morphological features were used for the training of an NB classifier. Menon et al. [16] extracted multi-feature fusion such as histogram features, morphological features, and textural features from breast ultrasound images for the classification of breast ultrasound images. Wei-Chung Shia et al. [17] extracted the histogram pyramid of directional gradient descriptors and used the method to obtain feature vectors. The correlation-based feature selection method was used to identify malignant breast tumors. Pomponiu et al. [18] extracted features based on the histogram directional gradient (HDG) and used SVM to classify the tumors.

Due to the reliance on custom hand-crafted features, the traditional method lacks robustness, though it has made great achievements in assisting breast cancer diagnosis of breast cancer. In addition, hand-crafted feature selection is very cumbersome and subjective, so it is difficult to improve the accuracy of the classification of benign and malignant breasts by using these methods. Moreover, due to the limitations of the ultrasound imaging mechanism, ultrasound images have high noise and low grayscale contrast. The method using only traditional hand-crafted features cannot accurately classify benign and malignant breast tumors.

Over the last ten years, a number of deep convolutional neural network (CNN) models, such as ResNet [19], Inception [20], InceptionRestv2 [21], Exception [22], AlexNet [23], and GoogLeNet [20], have been proposed for application to object detection and classification. CNN-based methods are widely used to extract high-dimensional abstract features from ultrasonic images, resulting in high-performance analysis in breast cancer diagnosis fields. For example, Zeimarani et al. [24] used CNNs with several hidden layers and applied regularization techniques to classify tumors. Kong et al. [25] designed three CNN models to fuse multi-view classification information. Zhuang et al. [26] proposed an image decomposition method to obtain fuzzy enhancement, bilateral filtered images, and raw ultrasound images as inputs presented to a pre-trained VGG16 model for feature fusion. Chiang et al. [27] proposed a fast and effective computer-aided detection system based on a 3-D convolutional neural network (CNN) and priority candidate aggregation. Shen et al. [28] proposed to consist of the following two subnets: an encoder-decoder network for segmentation and a lightweight multi-scale network for classification. Qi et al. [29] proposed a state-of-the-art method with multi-scale kernels and the method skips connections to diagnose breast tumors with ultrasound images. Wei et al. [30] used the trained deep residual network model to classify breast ultrasound malignant tumors by using sequential minimal optimization (SMO) linear SVM. The accuracy of the method exceeds that of the average radiologist. M.I.Daoud et al. [31] extracted features from the CONV5, FC6, and FC7 layers of the AlexNet network, merged the features, and then combined migration learning to distinguish benign and malignant breast tumors. Yap et al. [32] investigated the use of patch-based LeNet U-Net transfer learning in FCN-AlexNet and provided a comprehensive evaluation of the most representative lesion detection methods.

According to the above analysis, although traditional machine learning methods can explain the learned features well, however, the disadvantage is that manual feature extraction is difficult and not necessarily perfect. Compared with traditional machine learning methods, deep learning methods often have better classification results, but the datasets required for deep learning for training are very large. As we all know, medical image datasets involve patient privacy. What is more, the labeling of medical ultrasound images often adds to the burden of work for physicians. All of the above reasons lead to the quantity of the datasets not being large enough. However, if only a small dataset is used to train a deep learning model, it can easily lead to overfitting, which indicates better performance on the training set, but the performance on the test set may decrease. This will result in poor generalization of the trained model, making it difficult to apply the model to clinical applications. Therefore, this paper proposes a method for combing the features extracted by the traditional hand-crafted and deep learning ways that uses the pre-trained deep residual network.

This paper focuses on improving the accuracy of benign and malignant breast tumor classification in ultrasound images under the condition of small datasets. The proposed method is based on a complementary weighted fusion of two different classifiers with global deep network features, texture features, and morphological features. The rest of this paper will cover the following contents. Section 2, Section 3 and Section 4 describe the materials, methods, results, and discussion. A conclusion of the research work in this paper is presented in Section 5.

2. Methods

In this paper, we propose an automatic classification method that aims to improve classification accuracy from a small dataset. The proposed method is based on a complementary weighted fusion of two different classifiers with global deep network features, texture features, and morphological features. Figure 1 shows a flow chart of the implementation.

First, the original ultrasound images were denoised. Then, using the pre-processed data, three different features were extracted, and two different classifiers were trained. Two classification results were obtained by the two classifiers. After obtaining the classification results, the final results were obtained by weighting the classifiers. The details of how to obtain the results are as follows:

• The steps to obtain the features by deep learning as Branch 1 shown in Figure 1. First, using breast tumor ultrasound images and classification labels from pre-trained ResNet101, such features are obtained. Subsequently, the least correlated features are selected from these extracted features by PCA;
• The steps to obtain features based on texture features as Branch 2 shown in Figure 1. First, features of the whole image are extracted from the original breast tumor ultrasound image. Subsequently, the least correlated features are selected from these extracted features by PCA;
• The steps to obtain features based on medical features as Branch 3 shown in Figure 1. The segmentation results are obtained using the K-means algorithm. Then, morphological features of breast tumors are obtained from the segmentation results. Subsequently, the least correlated features are selected from these extracted features by SPSS;
• The steps to concatenate the features selected by deep learning with the selected texture features to obtain a final score using the support vector machine classifier;
• The steps to concatenate the selected deep learning features and the selected texture features to obtain a final score using the NB classifier;
• They integrated all the above by using weighted classifier to obtain the final result of the improved results.

This section describes the main methods used in the proposed multi-features integrated a weighted double-classifier learning.

2.1. Image Pre-Processing

In this study, due to the different sizes of breast tumors, the original images were of different sizes. Therefore, all images were resized to 256 × 256 pixels. To protect patient privacy, all irrelevant information such as text titles, indicators, related marks, etc., are removed first. At the same time, due to the mechanism and characteristics of ultrasonic imaging, breast ultrasound images often exhibit features such as low contrast, low resolution, high speckle noise, and blurred boundaries between tissues and anatomical structures, which will inhibit the computer’s feature extraction of ultrasound images. So, all images were pre-processed by the speckle reducing anisotropic diffusion (SRAD) filter [16] and histogram equalization method [16]. All images were pre-processed by the speckle reducing anisotropic diffusion (SRAD) filter and histogram equalization method. Among them, the ultrasonic image is processed by SRAD, as shown in Equation (1) as follows:

$$\left\{\begin{array}{c}\frac{\partial L\left(i,j;t\right)}{\partial t}=div\left[c\left(q\right)\nabla L\right]\left(i,j;t\right)\\ L\left(i,j;0\right)=L_0\left(i,j\right),\left(\frac{\partial L\left(i,j;t\right)}{\partial \frac{\to }{n}}\right){|}_{\partial \Omega }=0\end{array}\right.$$

(1)

L₀(i,j) and l(i,j;t) represent the initial image and output image respectively, ∇ represents the Laplacian operator, div represents the divergence operator, ∂Ω represents the boundary of the image Ω continuous domain, and $\frac{\to }{n}$ represents the external method of ∂Ω vector. c(q) represents the diffusion coefficient. The pre-processed image is obtained by equalizing the histogram of the image processed by the SRAD algorithm.

2.2. Feature Extraction

The proposed method contains features of the breast tumor shape and features of the whole image. Features of the breast tumor regions: In breast tumor ultrasound image diagnosis, doctors often diagnose benign or malignant breast tumors based on several widely recognized morphological features. Based on these features, we extracted some features for the diagnosis of breast tumors. These features are shown in Figure 2, including convexity, area of breast tumor, area of convex hull, elongation, the smallest rectangular box, area of box, center distance, and elliptical variance. etc. Features of the whole image: These features are extracted including deep learning features and texture features. Typically, the different deep learning features and texture features can reflect the whole breast tumor image. The main texture features extracted are energy, contrast, correlation coefficient, entropy, differential moment, inverse differential moment, total mean, and total variance.

The shape and texture of the tumor often play an important role when diagnosing breast tumors. Breast tumors tend to show a wide variation in morphology, benign breast tumors whose borders are clear and smooth, usually have a more regular shape. In contrast, malignant breast tumors, whose borders are unclear and irregular, usually have an irregular shape. In terms of other features, in general, benign breast tumors have a clearer texture with high echo than malignant breast tumors. Meanwhile, malignant breast tumors tend to be characterized by hypoechoic and a high degree of calcification. In fact, experts also consider these shape features and textural features when diagnosing breast tumors. The visual depiction of morphological features is extracted according to Equations (2)–(11), whose details are shown as follows:

Convexity: This feature is defined as the ratio of the convex hull to tumor circumference. The convex hull is the shape drawn around the outline of the tumor.

$$F_{Conv}=\frac{P_c}{P_n}$$

(2)

Solidity: This feature is used to describe the convexity or concavity of the tumor [33]. When the lesion has an irregular shape, the solidity value is less than 1. Generally, benign breast tumors have a more regular shape, and the area of the convex bundle is closer to the area of the tumor.

$$F_{Sol}=\frac{A_n}{A_c}$$

(3)

Elongation: This feature refers to the ratio between the length and width of the smallest rectangular box of the tumor. The value of elongation is generally between 0 and 1. When the shape of the lesion is close to square or round, the elongation value is close to 1.

$$F_{Elo}=\frac{W_n}{L_n}$$

(4)

Compactness: This feature is referring to the ratio of the tumor area to the square of the circumference.

$$F_{Com}=\frac{4\pi A_n}{P_n{}^2}$$

(5)

Rectangularity: This feature is referred to the ratio of the area of the tumor to the area of the smallest circumscribed rectangle of the tumor [34].

$$F_{ec}=\frac{A_n}{A_r}$$

(6)

Roundness: This feature is the ratio of the area of the breast tumor to the square of the minimum convex perimeter of the breast tumor boundary.

$$F_{oud}=\frac{4\pi A_n}{P_c{}^2}$$

(7)

Center distance: The center distance is the distance between the center of mass of the breast tumor and the center of mass of the least convex body. The center distance value for benign tumors is smaller than that for malignant tumors.

Convex Perimeter: This feature is the perimeter of the convex hull of a breast tumor [35].

$$F_{CP}={\displaystyle \sum }_{i=1}^{N-1}\left|c{x}_i-c{x}_{i+1}\right|$$

(8)

Convex Area: This feature refers to the area of the tumor surrounded by the convex hull.

Perimeter: This feature represents the circumference of the breast tumor.

$$F_{Pn}={\displaystyle \sum }_{i=1}^{N-1}\left|x_i-x_{i+1}\right|$$

(9)

Area: This feature is referred to the total pixels in the area of the breast tumor.

Rectangular box perimeter: This feature is the circumference of the smallest rectangular box surrounding the breast tumor.

Rectangular box area: This feature is the area of the smallest rectangular box surrounding the breast tumor. The area of the rectangular box refers to all pixels that occupy the rectangular box.

Elliptic compactness: This feature is referred to the ratio of the fitted ellipse to the circumference of the tumor contour. The smaller the value of elliptical compactness, the greater the likelihood that the breast tumor is malignant. The core of the ellipse fitting method lies in fitting the ellipse to the tumor outline points.

$$F_{EC}=\frac{\pi \left(a+b\right)}{F_{Pn}}$$

(10)

The mean of the radial distance spectrum is as follows: This feature is used to quantify the correlation of tumor margins by defining the distance from the edge of the tumor to the center of the tumor [15]. The logarithmic amplitude spectrum of the radial distance is obtained by performing a Fourier transform and logarithmic operation on the radial distance spectrum. Obtain the mean value based on the logarithmic amplitude spectrum of the radial distance.

$$F_D=\sqrt{{(x_1-x_0)}^2+{(y_1-y_0)}^2}$$

(11)

The variance of the radial distance spectrum is as follows: This feature is the variance value obtained from the logarithmic amplitude spectrum of the radial distance.

Where P_c represents the circumference of a convex hull, P_n is the circumference of the tumor, and A_n is the area of the tumor. A_c is the area of the convex hull; W_n is the width of the tumor. L_n is the length of the tumor, A_r is the area of the smallest rectangular box, cx₁…cx_n represents the boundary coordinates of the breast tumor, x₁…x_n represents the boundary coordinates of the breast tumor, a represents the semi-approximate axis of the fitted ellipse, and b represents the semi-body axis of the fitted ellipse, (x₁, y₁) represents the coordinates of the edge point of the breast tumor, and (x₀, y₀) represents the coordinate of the center point of the breast tumor.

Ultrasound images of benign breast tumors and malignant breast tumors often have major differences. Their features such as hidden features and texture features are also different.

Deep learning features: Deep residual network is an extremely effective deep learning method. Its network structure is mainly composed of a stack of various basic modules, including input layer, convolutional layer, activation function, batch standardization, identity mapping, global mean pooling, an output layer, etc. [19]. Extracting features uses the ResNet101 model pre-trained on the ImageNet datasets to extract high-level features for classification purposes. The network structure of ResNet101 consists of 101 weighted layers. It contains 33 3-layer residual blocks, which end with an average pooling layer and a fully connected layer, classified as a SoftMax function, and executed after each convolution before activation batch normalization [36] to improve the learning rate.

The pre-trained ResNet101 convolutional neural network was subjected to migration learning to extract the depth features in the ultrasound images. As shown in Figure 3, after feeding the preprocessed breast ultrasound image set into the input layer and analyzing the network, all image features are extracted and aggregated in the averaging pooling layer. Extracting features from the average pooling layer, as shown in Equation (12), converting them into feature vectors, and finally sending them to a suitable classifier for prediction as follows:

$$Y_{GAP}\left(1,1,i_{ch}\right)=averag{e}_{iro,ico}\left(X_{GAP}\left(i_{ro},i_{co},i_{ch}\right)\right)$$

(12)

where X_GAP represents the input features map of the average pooling layer, Y_GAP represents the output features map of the average pooling layer, i_ro, i_co, i_ch represent the index of the row, column, and channels in X_cov, respectively. Additionally, X_cov represents the input feature map of the convolutional layer.

Some hidden features are extracted from the pool layer of the pre-trained ResNet101. This allows these hidden features to be used to reflect features other than the morphological and texture features of the original image.

Texture features: The extraction of these features is based on the LBP, HOG, and GLCM. Texture is a visual feature that reflects the phenomenon of homogeneity in an image, and it reflects the property of surface structure organization arrangement with slow or periodic changes on the surface of an object. LBP [37] is an operator used to describe the local texture features of an image. It has significant advantages such as rotation invariance and gray-level invariance [38]. HOG [39] is a descriptor used in computer vision and image processing that constitutes features by calculating and counting the histogram of gradient directions in local regions of an image. GLCM [40] extracts the relationship between pixel pairs and obtains the partial eigenvalues of the matrix by calculating the co-occurrence matrix to represent certain texture features of the image. The main texture features extracted are energy, contrast, correlation coefficient, entropy, differential moment, inverse differential moment, sum average, and sum variance. These features were extracted according to Equations (13)–(22).

Energy:

$$T_1={\displaystyle \sum }_i{\displaystyle \sum }_j{\left[g\left(i,j\right)\right]}^2$$

(13)

Contrast:

$$T_2={\displaystyle \sum }_i{\displaystyle \sum }_j{\left(i-j\right)}^{2}g\left(i,j\right)$$

(14)

Correlation coefficient:

$$T_3=\frac{{{\displaystyle \sum }}_i{{\displaystyle \sum }}_{j}ijg\left(i,j\right)-{{\displaystyle \sum }}_{i}i{{\displaystyle \sum }}_{j}g\left(i,j\right){{\displaystyle \sum }}_{i}j{{\displaystyle \sum }}_{j}g\left(i,j\right)}{\sqrt{{{\displaystyle \sum }}_i{(i-{{\displaystyle \sum }}_{i}i{{\displaystyle \sum }}_{j}g\left(i,j\right))}^2{{\displaystyle \sum }}_{j}g\left(i,j\right){{\displaystyle \sum }}_j{(j-{{\displaystyle \sum }}_{i}j{{\displaystyle \sum }}_{j}g\left(i,j\right))}^2{{\displaystyle \sum }}_{i}g\left(i,j\right)}}$$

(15)

Entropy:

$$T_4=-{\displaystyle \sum }_i{\displaystyle \sum }_{j}g\left(i,j\right){log}_2\left[g\left(i,j\right)\right]$$

(16)

Differential moments:

$$T_5={\displaystyle \sum }_i{\displaystyle \sum }_j{\left(i-\mu \right)}^{2}g\left(i,j\right)$$

(17)

Inverse differential moments:

$$T_6={\displaystyle \sum }_i{\displaystyle \sum }_j\frac{g\left(i,j\right)}{1+{(i-j)}^2}$$

(18)

Sum average:

$$T_7={\displaystyle \sum }_{i+j}{\displaystyle \sum }_i{\displaystyle \sum }_{j}ig\left(i,j\right)$$

(19)

Sum variance:

$$T_8={\displaystyle \sum }_{i+j}{\displaystyle \sum }_i{\displaystyle \sum }_j{\left(i-T_7\right)}^{2}g\left(i,j\right)$$

(20)

Sum entropy:

$$T_9=-{\displaystyle \sum }_{i+j}{\displaystyle \sum }_i{\displaystyle \sum }_{j}g\left(i,j\right){log}_2\left[g\left(i,j\right)\right]$$

(21)

Difference in variance:

$$T_{10}={\displaystyle \sum }_{k=2}^{2N}{\displaystyle \sum }_{i=1}^{2N}{\displaystyle \sum }_{j=1}^{2N}g\left(i,j\right),k=\left|i-j\right|=0,1,\cdots ,N-1$$

(22)

where i and j represent horizontal and vertical coordinates respectively, g(i,j) is the GLCM of each breast tumor image.

After extracting features of the breast tumor region and features of the entire image, although the morphological features of breast tumor are partially extracted based on the doctor’s prior knowledge, there may still be useless features. Meanwhile, the features of the whole image may contain some useless features, so the features above still need to be filtered before the subsequent training of the classifier.

2.3. Feature Selector

The core of the feature selector is to develop a criterion. This criterion allows us to measure the importance of each feature. Additionally, all features are classified and simplified. In this way, the redundant features are removed to obtain the best subset of features. The selected features remain valid in the classification. In this paper, PCA is used as a feature selector. Deep learning features and texture features are fed to the feature selector to obtain the main features. Morphological features were sent to SPSS for statistical analysis to obtain the features with the lowest correlation.

In order to select the least relevant features to achieve better classification. For morphological features, statistical analysis of benign and malignant parameters was performed by using the Man–Whitney test [41]. The default significance level is 0.05, and the confidence interval defaults to 95%. Finally, except that the significance of solidity and rectangularity are 0.81 and 0.08, respectively, which are greater than 0.05. Therefore, there is no significant difference, and other features are all significant. In the proposed method, the morphological features with significant differences are retained.

When combining texture features and deep learning features, there is a correlation between texture features and deep learning features, which easily leads to the weak generalization ability of the model. For this reason, it is necessary to perform feature dimensionality reduction on high dimensional texture features to reduce the number of feature attributes and ensure that the attributes are independent of each other. PCA [42] is a common approach used to reduce feature size. The details are as follows:

• Obtain m-dimensional eigenvectors of n ultrasound images, the m-dimensional eigenvectors form a feature matrix Xm×n calculate the average value of each m-dimensional eigenvector in the feature matrix, and then subtract all m-dimensional eigenvectors from their corresponding Average, obtain the feature matrix after de-averaging;
• Calculate the covariance matrix of the feature matrix after de-averaging;
• Diagonalize the eigenvalues and eigenvectors of the covariance matrix so that all elements except the diagonal are 0, assigning the energy to the principal directions;
• Sort the eigenvectors by the size of the eigenvalues;
• Retain the largest former ω eigenvalues, where, ω < m;
• Transform the data in the feature matrix into a new space constructed by feature vectors and obtain n feature vectors after dimension reduction.

The features extracted by ResNet101 and the texture features were combined into a vector after PCA dimensionality reduction, as shown in Figure 4. Additionally, the fusion vector with these features was fed into the next classifier for training and testing.

Both sets of features are prepared for training. Then, texture features and deep learning features were used in the SVM classifier for learning and morphological features were used in the NB for learning. Then, after that, the final results are obtained by weighting the results obtained from both sets of classifiers to decide whether the breast tumor is benign or malignant.

2.4. Weighting and Training Double-Classifier

In this paper, radial basis functions are applied as kernel functions in a support vector machine and five-fold cross-validation is performed using a grid search method to automatically find the optimal parameters. Then, the NB classifier is trained by using the selected morphological features to obtain the classification results, and the SVM is trained by using the selected texture features and deep learning features to obtain the classification results.

As for features extracted from the shape of a K-means segmentation, previously, 16 features were extracted based on the output of K-means, and 14 features were obtained after filtering. These features were used to train an NB classifier.

As for features extracted from the pooling layer of ResNet101 and texture features, previously, we extracted 2048 features and 5620 texture features based on the pooling layer output of ResNet101 and obtained 140 features after filtering. These features are used to train an SVM classifier.

Thus far, two sets of classification results have been obtained. In addition, weighted classifier is used to integrate these two sets of classification results to obtain the final results. A single classifier does not sufficiently learn the relevant features of breast ultrasound images, and the complementary between two or multiple classifiers can increase the classification effect. The classifiers used to learn breast ultrasound features are divided into parametric and nonparametric classifiers. Parametric classifiers are often used to learn high-dimensional features and nonparametric classifiers are often used to learn low-dimensional features. According to Occam’s razor theory [43], the following are the results and the weighted matrix of the linearly weighted fusion sub-classifier (Equation (23)).

$$\overrightarrow{S}={\displaystyle \sum }_{i=1}^2{W}_i\overrightarrow{S_i}={\displaystyle \sum }_{i=1}^2\left[\begin{array}{ccc}{p}_{i1}& \cdots & 0\\ ⋮& \ddots & ⋮\\ a_{31}& \cdots & p_{im}\end{array}\right]\left[\begin{array}{c}{S}_{i1}\\ ⋮\\ S_{im}\end{array}\right]=\left[\begin{array}{c}{\displaystyle {\displaystyle \sum }_{i=1}^2}{S}_{i1}{p}_{i1}\\ ⋮\\ {\displaystyle {\displaystyle \sum }_{i=1}^2}{S}_{im}{p}_{im}\end{array}\right]$$

(23)

where, $\overrightarrow{S_i}$ represents the result of the sub-classifier, and W_i represents the weighting matrix of the corresponding sub-classifier.

$$S_{\lambda }=\underset{0\le j\le m}{max}\left\{{\displaystyle \sum }_{i=1}^2{S}_{ij}{P}_{ij}\right\}={\displaystyle \sum }_{i=1}^2{S}_{ik}{P}_{ik}$$

(24)

Based on the maximum rule, such as Equation (24), where P_ij is the recognition rate at which the i-th picture is recognized as the j-th category; j ∈ {0,1}. The benign and malignant state with the highest score is the final recognition result. $S_{\lambda }$ represents the score after the fusion of the two classifiers. When 0 < $S_{\lambda }$ < 0.5, the tumor is considered to be a benign tumor. When 0.5 < $S_{\lambda }$ < 1, the tumor is considered to be a malignant tumor.

3. Experiments and Results

During the experiments, to avoid overfitting, the breast ultrasound dataset was split into five for five-fold cross-validation. Divide the breast ultrasound dataset into five equal parts. Use the first 20% as the test set and the other parts as the training set to obtain accuracy. In turn, use the second 20% as the test set, the remaining part is used as the training set, and a total of 5 results are obtained. Take the average of these five results as the result of the model. In this study, all experimental results were also derived from the arithmetic mean after five-fold cross-validation. The results of feature selection, classification of breast tumor ultrasound images, and the classification of breast tumor ultrasound images by dual weighted classification are presented. These results are described and analyzed in detail in this section.

3.1. Image Acquisition

The 2D breast ultrasound images involved in this study were obtained from the Department of Ultrasonics, First Affiliated Hospital of Fujian Medical University, Fujian Province, China, with instrument models including Philips, Siemens, etc. The other color ultrasound diagnostic equipment has a probe frequency of 12 MHz. The study has passed the medical ethics review, complying with the principles of medical ethics and with consent from the patients involved. Ultrasound breast images of women aged from 14 to 80 years, collected between 2018 and 2021, were compared to pathological findings with a clear diagnosis. Among these, 418 were benign breast tumors and 373 were malignant breast tumors. Figure 5 shows the sample ultrasound images with benign and malignant breast tumors. In addition, the diagnosis of the images was verified by biopsy. The region of interest (ROI) of each breast ultrasound image was manually annotated by a radiologist with extensive experience and verified by another radiologist who has more than 10 years of breast ultrasound diagnosis experience. To protect the privacy of patients, the patient information around each breast ultrasound image has been removed.

3.2. Evaluation Indicators

To quantitatively assess the classification performance, we used the following five metrics: (1) Acc (Accuracy) = (TP + TN)/(TP + TN + FP + FN); (2) Pre (Precision) = TP/(TP + FN); (3) Rec (Recall) = TN/(TN + FP); (4) F1 = (2 × Precision × Recall)/(Precision + Recall); (5) AUC (area under the ROC curve). Here, TP (true positive) and TN (true negative) represent the number of correctly classified positive and negative samples, respectively, while FP (false positive) and FN (false negative) represent the number of incorrectly classified negative and positive samples, respectively. In the classification of breast tumors, positive samples were malignant tumors and vice versa.

3.3. Classification Results Based on the Selected Features

Since the texture features and the dimensionality of the deep residual network extracted are too large. PCA is used to perform feature selection on the eigenvector matrix so as to filter out unimportant features. Using statistical analysis to analyze morphological features and select useful features to reduce the time required for feature extraction. To provide a more visual indication of the effectiveness of the selected features. As shown in

Table 1. Time consuming for before and after feature selection.

Experiments	Features Selection	Time (s)	Acc (%)
1	N	0.4034	87.90
2	Y	0.0521	89.17

, we verified this by using the required classification accuracy and time before and after feature selection.

As shown in Table 1, the features that were filtered played a more active role in the classification. The overall accuracy obtained after feature selection increased by 1.27%; meanwhile, the training time was reduced to 0.0521 s. In clinical application, it can also improve the efficiency of the doctor’s diagnosis.

3.4. Classification Results Obtained Based on the Weighting of the Double-Classifier

Considering the differences among the performance of different classifiers, a weight is assigned to each classifier to make the weighted classifier method more reasonable. In general, the weight assigned to each classifier is related to its performance. Figure 6 shows the analysis of weighted fusion. When accuracy, precision, recall, F1, and AUC values are taken into account, the highest accuracy is achieved when the weight is 0.9. In this condition, the model achieves the best results.

To demonstrate that the deep learning feature extraction network (ResNet101) involved in the designed experiments has better performance in extracting breast ultrasound image features compared to other methods. We have designed experiments to classify breast ultrasound images by using popular neural networks such as ResNet18 [19], ResNet50 [19], ResNet101 [19], GoogleNet [21], AlexNet [23], and InceptionRestv2 [21] respectively. The results are shown in

Table 2. Classification results based on different CNNs.

Methods	Acc (%)	Pre (%)	Rec (%)	F1 (%)
ResNet18	81.53	83.58	75.68	79.43
ResNet50	77.07	73.75	79.73	76.62
ResNet101	83.44	84.29	79.73	81.95
GoogleNet	79.62	75.68	81.48	78.47
AlexNet	81.53	80.00	81.08	80.54
InceptionResNetv2	66.23	62.26	81.48	70.59
Xception	80.89	85.14	76.83	80.77
Efficientnetb0	78.98	74.32	79.71	76.92

. ResNet101 has the highest breast ultrasound classification accuracy of 83.44%. Its accuracy was 1.91% higher than the second-ranked ResNet18 model.

3.5. Classification Results Based on Single Breast Tumor Features Combined with Different Classifiers

To find the most suitable classifier for the proposed features, we have designed experiments in which different classifiers learn the extracted features separately and compared them. As for texture features with high dimensionality, we chose SVM [44], KNN [45], DT [46], and LDA [47] classifiers for learning texture features. As for morphological features with low dimensionality, we chose SVM and NB classifiers for learning. As for the features extracted by ResNet101, which have high dimensionality, we chose SVM [44], and SoftMax classifiers for learning texture features. The average values of the classification results of the above experiments after five-fold cross-validation are shown in

Table 3. The classification results with a single feature combined with different classifiers.

Features	Classifiers	Acc (%)	Pre (%)	Rec (%)	F1 (%)	AUC
Texture	SVM	80.80	83.10	79.73	81.38	0.82
	KNN	80.89	79.73	79.73	79.73	0.77
	DT	64.99	62.03	66.22	64.05	0.50
	LDA	75.16	87.23	55.41	67.77	0.88
Morphological	SVM	63.69	60.24	67.57	63.69	0.69
	NB	73.83	69.23	78.26	73.47	0.72
ResNet101	SVM	86.62	84.42	87.83	86.09	0.85
	SoftMax	83.44	84.29	79.73	81.95	0.83

. From Table 3, we can see that the SVM classifier has a better learning effect for texture features extracted from breast ultrasound images. The NB classifier has a better learning effect for morphological features. The SVM classifier has a better learning effect for extracting deep residual network features from breast ultrasound images.

In this study, we chose to combine texture features with deep residual network features, then feed the combined features into SVM for learning, and finally use the NB classifier to learn the morphological features and weight the classification results of both for scoring. It will improve the final classification result. We designed comparison experiments in which each feature is fed into the SVM classifier separately for learning. Specifically, the texture features after feature selection and the deep learning features after feature selection were first fed into the SVM classifier for learning, and then the texture features were classified by a support vector machine, and the results of the three were weighted and scored. The average of the experimental results after the five-fold cross-validation is shown in

Table 4. The classification results with different features combined with different classifiers.

Features	Classifiers	Acc (%)	Pre (%)	Rec (%)	F1 (%)	AUC
Texture + ResNet101	SVM	89.17	90.14	86.49	88.28	0.95
Morphological	NB
Texture	SVM	87.26	82.14	93.24	87.34	0.93
ResNet101 + Morphological	NB
Texture	SVM	83.44	88.00	89.19	88.59	0.92
Morphological	NB
ResNet101	SVM

. The results show that the method proposed in this paper can improve the classification accuracy of breast tumor ultrasound images.

To demonstrate the effectiveness of the proposed method for the classification of benign and malignant breast tumors in ultrasound images. The method proposed in this article was compared with related methods published in recent years under the same small datasets. Figure 7 shows the ROC values of the proposed method in this study and the comparison methods, including (1) C.YL et al. trained a logistic classifier with average, standard deviation, skewness, and kurtosis [14]; (2) Wei et al. trained SVM classifier and NB classifier with combining textural features and morphological features [15]; (3) Wei.CS et al. used the trained deep residual network model to classify breast ultrasound malignant tumors by using sequential minimal optimization (SMO) linear SVM [30]; (4) M.I.Daoud et al. extracted features from the CONV5, FC6, FC7 layers of the AlexNet network, and merged the features [31]. (5) Moon proposed ensemble VGGNet, ResNet, and DenseNet on US images [48]. (6) Francisco had extracted 137 morphological and texture features, and these features were trained by the LDA classifier [49]. The accuracy, precision, recall, F1 value, and AUC value of the area of the ROC curve are shown quantitatively for the proposed method and other existing methods for classifying in the same breast ultrasound datasets. The classification results of the above experiments are shown in

Table 5. The performance comparison of the proposed method and the related existing methods.

Experiments	Acc (%)	Pre (%)	Rec (%)	F1 (%)	AUC
C.YL [14]	78.98	78.08	77.02	77.55	0.84
Wei [15]	80.89	88.89	75.68	81.75	0.89
Wei.CS [30]	84.71	86.76	79.73	83.10	0.91
M.I.Daoud [31]	80.89	78.48	83.78	81.04	0.91
Moon [48]	86.62	85.33	86.49	85.91	0.92
Francisco [49]	80.89	78.95	81.08	80.00	0.89
Ours	89.17	90.14	86.49	88.28	0.95

.

By comparing with the other four methods, this method has the highest total accuracy. The method proposed in this study has 2.55% higher accuracy, 4.81% higher precision, 2.37% higher F1 value, and 0.03 higher AUC value of the area of the ROC curve than the second-ranked Moon [48]. In conclusion, the method has better results on small datasets compared with the above methods published in recent years. The method provides better results in the diagnosis of breast tumors.

4. Discussions

Breast ultrasound is widely used in the diagnosis of breast cancer due to its versatility, safety, and high sensitivity. However, breast ultrasound examination is a time-consuming task. Therefore, designing a CAD system for auxiliary analysis of breast ultrasound images is necessary and it has important clinical application value. The classification of breast tumors is a challenging task due to the large variation in breast tumor shape and irregular and blurred breast tumor boundaries. However, traditional single deep network models and single classifier learning features are not effective in identifying breast ultrasound image classification.

To address the lack of robustness of traditional hand-crafted features for the classification of breast tumor lesions under small datasets and improve the sensitivity for clinical application when using small datasets. In this paper, a novel classification model and a multifeatured ensemble double-classifier learning method are proposed. The multi-features mainly include global depth residual network features, texture features, and morphological features. By comparing the computation time before and after feature selection, it can be seen that the PCA used in this study can effectively select the extracted features, reducing computation time and improving accuracy. Multi-features mainly include global deep residual network features, texture features, and morphological features. The deep residual network model, ResNet101, is used to extract high-dimensional image features from the pooling layers of the trained network to describe the latent features of the image. By comparing with different deep learning algorithms, as shown in Table 2, it can be seen that our Resnet101 model for extracting global information has better results. Meanwhile, in order to better feature learning, a non-parametric NB classifier is trained to work with a parametric SVM classifier that exploits the complementarity of the classifiers. Through the training results of various classifiers, as shown in Table 3, the two classifiers, SVM and NB, have better learning abilities after weighting. At the same time, for the selection of weights, it can be seen from Figure 6 that the weights do not have a linear relationship with the experimental effect. For example, if the weight value is set to 0.8, the AUC value decreases instead. When the weight value is 0.9, a five-fold cross-validation experiment yielded an average accuracy of 89.17%, an average precision of 90.00%, and an average AUC value of 0.95. In addition, compared with several existing methods, the classification model designed in this study has better performance under the same dataset. Finally, although the method proposed in this study can better solve the problem of breast tumor classification in ultrasound images, the breast ROI area of ultrasound images involved in this study is manually cropped, and the research on automatic detection of breast tumors in ultrasound images will be research in our next work.

5. Conclusions

This paper aims to address the problem of obtaining a method that is more effective in self-diagnosis and has higher accuracy. The method obtained double-classifier complementary weighted fusion for automatic classification of breast ultrasound images with the combination of global deep residual network features, texture features, and morphological features. As a result, the accuracy was 89.17%, the precision was 90.00%, and the AUC of the datasets was 0.95. In conclusion, the method proposed in this paper successfully solves the problem of low accuracy due to the use of small samples. The experimental results show that the method proposed in this paper achieves good accuracy on the test set.