1. Introduction
Breast disease is the most serious problem for women’s health today and has become a serious global public health problem [
1]. Studies have shown the importance of early screening for the prognosis and treatment of breast cancer [
2]. Ultrasound (US) examination is widely used for the diagnosis of internal organs and superficial organs due to its non-invasiveness, non-radiation use and convenience, compared with CT, MRI or X-ray [
3]. In particular, it has been proved to be especially effective in the diagnosis of breast cancer [
4]. Meanwhile, ultrasound examinations rely on clinical experience, which affects the quality of ultrasound images and the accuracy of diagnosis. Therefore, objective quality evaluation of ultrasound images has clinical significance to guarantee the standardization of ultrasound examination. In addition, to achieve automation of ultrasound examination, the ultrasound scanning robot has emerged as a research trend. In our previous research, an autonomous ultrasound scanning robot was designed [
5]. The quality assessment of ultrasound images is considered as a visual servo that can be used as control feedback for autonomous scanning robots. There are many reasons for the poor quality of ultrasound images, such as noise in signal transmission, artifacts, low echoes due to poor contact between the probe and the skin, and dragging blurring due to probe movement. To address these issues, this study focuses on the quality assessment of breast ultrasound images.
Image quality assessment has been intensively studied in the field of nature images. Generally, IQA is divided into subjective assessment, which is judged by experts, and objective assessment, which is computed with mathematical algorithms. Subjective IQA can be time-consuming, labor-intensive, costly, error-prone, and inconsistent [
6]. The objective assessment can be further divided into three categories based on the availability of reference images: (i) full reference (FR-IQA), where there is a perfect reference image for comparison with the test image; (ii) reduced reference (RR-IQA), which contains partial information of the reference image, and (iii) no reference (NR-IQA), where no information regarding the reference image is available for the assessment [
7]. FR-IQA calculates the statistical difference between the sample image and the reference image, such as PSNR [
8], RMSE [
9], intensity [
10] or SSIM [
11]. However, FR-IQA and RR-IQA have limited practical applications since the reference image is usually not available, especially for medical images. In contrast, NR-IQA methods do not require any information from a reference image and are well-suited for practical applications.
Many researchers have focused on the quality assessment of medical images, including MRI [
6,
12,
13], retinal fundus images [
14], video endoscopy [
15,
16], CT [
17], and ultrasound images. Medical ultrasound images are known to present with a lot of noise and artifacts, low resolution, blurred boundaries, and low contrast, which bring many difficulties to the quality assessment of images. Traditional NF-IQA methods for ultrasound images generally extract image features first and then adopt SVM, SVR, and other methods with machine learning for quality regression or classification [
13]. In recent years, some researchers have introduced confidence mapping into autonomous robotic scanning tasks to evaluate the quality of ultrasound images. Confidence mapping is a method to quantitatively evaluate the uncertainty caused by the attenuation and shadow in ultrasound images [
18,
19]. However, the confidence map only evaluates the contact condition between the probe and human skin, indicating the reliability of the ultrasound imaging at each pixel, and does not consider the significance of the image content for diagnosis. Therefore, the confidence mapping method has a big gap between it and the clinical standard of ultrasound image quality assessment. Recently, end-to-end deep learning methods have been demonstrated to have great potential for IQA. Most researchers working on clinical ultrasound image analysis have proposed to assess the image quality based on whether key anatomical structures can be detected. Wu et al. [
20] proposed a computer-assisted fetal ultrasound image quality assessment algorithm, which is composed of L-CNN and C-CNN. The function of L-CNN is to find the ROI (Region of Interest) in the fetal abdominal region of the ultrasound image, and C-CNN detects whether there are key structures, such as gastric vesicles and umbilical veins, in the obtained ROI to evaluate the quality of the ultrasound image. Lin et al. [
21] proposed a MF R-CNN (Multi-task Fast Regional Convolutional Neural Network) for the quality assessments of fetal head ultrasound images, which can detect and identify the key anatomical structures of the fetal head, and then evaluate the quality of ultrasound images. Camps et al. [
22] proposed a DenseNet classification to automatically assess the quality of trans-perineal ultrasound images of the male pelvic region based on the visibility of the prostate. Similarly, Antico et al. [
23] trained a DenseNet to classify knee ultrasound images into two categories based on the visibility of cartilage. In breast ultrasound images, the anatomical structures such as the subcutaneous fat layer, breast glands, pleura and other anatomical structures are not clear, and the breast morphology of different patients is very different due to individuality. Therefore, it is difficult to assess the quality of breast ultrasound images through the extraction and detection of specific anatomical structures.
There are some other studies which have converted quality assessments to classification or regression by using doctors’ subjective evaluations of samples as labels. In Zhang’s work, ultrasound images were artificially added with noise, subjectively scored, and regressed with DCNN [
24]. In our previous work, doctors classified breast ultrasound images into five levels according to subjective evaluation, and a BCNN assessment network was trained [
25]. These works conducted global quality assessment of ultrasound images; however, for medical ultrasound images, doctors usually pay more attention to image content, especially those images with lesions. This research is an extension of our previous work [
25], and a soft-reference-based global–local hybrid breast ultrasound IQA framework is proposed. The breast ultrasound images were first classified into two groups that were either with or without lesions, then for the ultrasound images without lesions, the global image quality was calculated, and for the ultrasound images with lesions, the lesion was segmented and the image quality of the local region was evaluated.
The main contributions of this paper are listed as follows:
An integrated global–local breast ultrasound IQA framework is established, in which the images without lesions are evaluated globally, and the images with lesions are evaluated locally (i.e., with a focus on the lesion area).
A soft-reference IQA method based on lesion segmentation is proposed, in which the segmented lesion is taken as a reference image and transforms the medical ultrasound image assessment problem, from no reference to reduced reference.
A comprehensive evaluation index for medical ultrasound images with lesions is proposed, including pixel-level features MSE (Mean Squared Error), PSNR (Peak Signal-to-Noise Ratio) and semantic-level features CR (Contrast Ratio) and SSIM (Structural Similarity).
The rest of this paper is organized as follows.
Section 2 describes the dataset construction, which includes the image collection and data labeling and the IQA framework.
Section 3 presents the implement details of the network and the results of the experiments. Finally, the conclusion of this study is presented in
Section 4.