A Semi-Unsupervised Segmentation Methodology Based on Texture Recognition for Radiomics: A Preliminary Study on Brain Tumours

Abstract

Because of the intrinsic anatomic complexity of the brain structures, brain tumors have a high mortality and disability rate, and an early diagnosis is mandatory to contain damages. The commonly used biopsy is the diagnostic gold standard method, but it is invasive and, due to intratumoral heterogeneity, biopsies may lead to an incorrect result. Moreover, some tumors cannot be resectable if located in critical eloquent areas. On the other hand, medical imaging procedures can evaluate the entire tumor in a non-invasive and reproducible way. Radiomics is an emerging diagnosis technique based on quantitative medical image analyses, which makes use of data provided by non-invasive diagnosis techniques such as X-ray, computer-tomography (CT), magnetic resonance (MR), and proton emission tomography (PET). Radiomics techniques require the comprehensive analysis of huge numbers of medical images to extract a large and useful number of phenotypic features (usually called radiomics biomarkers). The goal is to explore and obtain the associations between features of tumors, diagnosis and patients’ prognoses to choose the best treatments and maximize the patient’s survival rate. Current radiomics techniques are not standardized in term of segmentation, feature extraction, and selection, moreover, the decision on suitable therapies still requires the supervision of an expert doctor. In this paper, we propose a semi-automatic methodology aimed to help the identification and segmentation of malignant tissues by using the combination of binary texture recognition, growing area algorithm, and machine learning techniques. In particular, the proposed method not only helps to better identify pathologic tissues but also permits to analyze in a fast way the huge amount of data, in Dicom format, provided by non-invasive diagnostic techniques. A preliminary experimental assessment has been conducted, considering a real MRI database of brain tumors. The method has been compared with the segmentation software’s tools “slicer 3D”. The obtained results are quite promising and demonstrate the potentialities of the proposed semi-unsupervised segmentation methodology.

1. Introduction

In the last years, a novel approach called radiomics, aimed at solving the issues of precision medicine and how it can be performed, has emerged as promising tool to support physician’s diagnosis [1,2,3,4,5]. Radiomics is based on multimodality medical images and it is a non-invasive, fast, and low in cost methodology. Currently, radiomics techniques have been applied with efficacy to different kinds of pathologies [6,7,8,9,10,11], requiring at first an accurate analysis of dozens of medical images for each patient, provided by standard diagnostic tools such as echography (ECO), X-ray computer aided tomography (CT), and magnetic resonance (MR) [12,13,14]. This first step requires an expert operator and much time. After the identification of a suspect pathology, the operator has to perform a manual delineation of the lesions, which is called segmentation [15]. These first two steps, which are mandatory for any radiomics procedure, are very time-consuming and subjective to the operators experience. In radiomics research, a large amount of radiological image data are processed in order to extract a large a set of quantitative image features [16,17]. The features are then analyzed to identify imaging biomarkers [18], meaningful patterns, analogies, and other markers that can support the diagnosis of a given pathology. Recently radiomics has been applied to identify oncological pathologies, in particular, it can be used to help discriminate between histological tumor subtypes, predict treatment response and overall survival and, consequently, support individualized and specific therapy for each patient. Radiomics seems to be the correct way to implement precision medicine, applying specific and customized therapy for each patient [19,20]. With radiomics, researchers have discovered a way to identify new biomarkers, by collecting quantitative features, especially from oncology medical images. In particular, researchers demonstrated that these images, obtained with non-invasive diagnostic tools (MR, CT, PET), could contain more information than those that an expert radiologist could identify. The information not visible to the naked eyes of a physician represents a powerful tool that can help correctly identify the pathology, reduce clinical errors, help train inexperienced clinicians, and promote telemedicine. Hence, radiomics represent an efficient clinical decision support system (DSS). Medical imaging, image processing techniques, automated image analysis, and machine learning techniques have recently reached significant advances [21,22], and the results seem to be very promising, especially in radiomics. Despite the advancements reached by radiomics, it remains a methodology strongly dependent on the operator’s experience. Up to now, it is not easy to obtain an automatic or a semi-automatic procedure. Moreover, radiomics is not standardized; many methodologies are customized to different pathologies. Some researchers have tried to apply machine learning techniques as in [22] or semi-unsupervised segmentation technique [23]. Texture features identification, a well-known technique, is widely used in biomedical applications [24,25,26,27,28]. Texture features can help clinicians to identify suspicious areas better. The features are extracted from gray-level co-occurrence and run-length matrix, and is then tested and compared with a set of healthy brain tissue samples. A support vector machine classifier provides the identification of tissue by using a suitable dataset previously-stored [29,30,31,32,33]. In this work during the first phase, where the operators must identify the pathology by analyzing lots of MRI data, a 2D texture analysis was performed on magnetic resonance images (MR) and combined with a growing area algorithm to perform the segmentation [25,34]. Meanwhile, the correlation between texture features and brain areas are calculated; when significant differences among the groups are found, a warning is provided. The results showed in scientific literature demonstrated that texture features including entropy, gray level and run length non-uniformity in brain tissues were significantly different for malignant or suspicious tissues. The paper is organized as follows. Section 2 presents a detailed description of the proposed semi-automatic segmentation methodology. In particular, a texture analysis combined with a growing area algorithm has been used to segment the suspected area. Then, a support vector machine algorithm aims to detect suspect tissues, helping the operator correctly identify pathological lesions and at the same time reducing the interpretation error. This phase can also be an efficient and handy tool for young tech operators and a new class of clinicians. Another very useful tool is the segmentation algorithm, which helps the operators to segment the malignant tissues (i.e., tumor delineation) with a high degree of accuracy and a strong time reduction, permitting the evaluation of different patients in a reduced amount of time. The innovations with respect to the other state of the art methods are: the integration between segmentation algorithm and machine learning, which permit a better efficiency and accuracy during the segmentation phase. The human operator has to provide only the initial seed, then the segmentation procedure proceeds unsupervised. The time reduction and the accuracy provided by the proposed methodology are better with respect to the standard methodologies commonly considered in radiomics. The goal is to obtain a mask to isolate only the malignant tissue and then apply the radiomics features. In Section 3 an experimental validation considers a clinical database of real patients with brain tumor lesions. Finally, Section 4 reports the conclusions and proposes ideas for future studies.

2. Description of the Methodology

The workflow of radiomics procedure is reported in Figure 1. The first step is the acquisition of Dicom data. Then, the second phase consists in the identification of suspect pathological tissue, its segmentation and creation of a 3D mask of the segmented area, the so called region of interest (ROI).

Then the features of ROI are extracted, and the data are post-processed in order to remove redundant or unused data, and suitable prognostic algorithms are applied. The segmentation and mask extraction is the most time-consuming, boring, and difficult phase, especially for an inexperienced operator.

To simplify the segmentation and mask extraction phase, a combination of the growing area [34] and pattern texture recognition [27] algorithms are considered. Moreover, the method is integrated with a machine learning algorithm based on a support vector machine (SVM) to discern malignant from healthy tissues [29,30] and to serve as a region membership criterion for the growing area algorithm. For pattern recognition, a local binary pattern (LBP) is considered. LBP is a technique that computes a local representation of image texture, and it is usually applied to grayscale images. Another similar technique that can be used for this goal is the census transform [35]. Local binary patterns and Census are quite similar. They both encode the local region by establishing the relationship between neighbor pixels to obtain information concerning the features. However, recently LBP and its variants have been successfully applied in various applications, such as texture classification, segmentation, face recognition, object detection, and other interesting practical applications. Census transform has only been, up to now, principally used to investigate stereo correspondence problems and some recent works [36] demonstrated that LBP is superior to census. For these reason we decided to use LBP instead to census transform. The local texture representation is made by comparing each pixel of the image with their surrounding neighbors. In particular, for each pixel of the Dicom image, a number N of neighbors pixels are selected, a $(2N+1)\times (2N+1)$ square matrix is composed of the selected pixel in the center and the surrounding pixels all around. If the signal intensity of the central pixel is greater than or equal to one of its ${(2N+1)}^2-1$ neighbors, the value of the neighbor is set to the logic value 0 otherwise it is set to 1. Then the number of transitions $N_t$ from 0 to 1 and 1 to 0 between all the neighbors is estimated. A low number of transitions $N_t$ mean a homogeneous area, while a high number of transitions represent an area with high variations. It is worth noticing that the LBP does not identify suspicious tissues but only areas with similar spatial distributions/textures. For the application at hand, aimed at identifying brain pathologies, a number $N=1$ seems to be reasonable, considering that the spatial resolution of an MRI pixel is about 3 mm, with $N=1$, a matrix of nine elements, which identifies an area of about a 1 cm², is considered. The operator manually chooses a seed (the starting pixel) located in an area considered suspect by the operator. Application of the LBP process from Dicom images permits the identification of homogeneous areas. The LBP procedure is summarized in Figure 2.

$$\bigcup _{i=1}^I\left[F_i\right]=ROIi=1,2,3\cdots ,I$$

(1)

$$RMC\left(\left[F_i\right]\right)=\left\{\begin{array}{c}TRUEif{N}_t^{F_i}\simeq N_t^{F_j}\\ FALSEif{N}_t^{F_i}\ne N_t^{F_j}\end{array}\right.$$

(2)

$${\left[F_i\right]}_{M\times N}=\left[\begin{array}{ccc}{f}_{(m-1,n-1)}& f_{(m-1,n)}& f_{(m-1,n+1)}\\ f_{(m,n-1)}& f_{(m,n)}& f_{(m,n+1)}\\ f_{(m+1,n-1)}& f_{(m+1,n)}& f_{(m+1,n+1)}\end{array}\right]\begin{array}{c}ifRMC\left(\left[F_i\right]\right)=TRUE\left[F_i\right]\in ROI\\ ifRMC\left(\left[F_i\right]\right)=FALSE\left[F_i\right]\notin ROI\end{array}$$

(3)

Then it is possible to use machine learning techniques to classify healthy tissues from malignant ones. To properly apply classification, the creation of a dataset is mandatory. To form the database, we used a set of real patients’ MRI images. In particular, from the dataset, samples of healthy brain tissue and tumoral tissue (diffuse brain glioma and meningioma) have been taken from different regions of the brain. The dataset construction has been supported by a qualified neuroradiologist with more than 15 years of experience who also provided the correct diagnosis of the malignant tissues. A support vector machine (SVM) classifier has been used not only to classify healthy regions from suspicious tissues but also to drive the growing region area algorithm. SVMs are machine learning methodologies based on statistical learning theory. Thanks to their high accuracy, simplicity, and good promotion, they have been widely used by the scientific community for different practical applications [31]. SVMs require a limited dataset for the training phase, and they provide a very good classification results. The combination between LBP and SVM permits extracting the region or area of interest (ROI-AOI). In particular, the LBP permits the extraction of the features used by the SVM aimed at identifying the healthy from ill brain tissues.

The segmentation algorithm aimed to extract the area with suspicious pathologies is based on a customized version of the region growing algorithm (RGA). This customized algorithm works as follows: starting from the initial pixel called seed and manually selected by the user, the algorithm determines whether neighboring pixels can be added to the region of interest. In a standard RGA, adjacent pixels are added to the area depending on a region membership criterion (RMC), which is usually based on the pixels’s intensity threshold value, aimed at identifying a given variation or by using a gradient provided by the operator. Unfortunately, this approach does not work well for the problem at hand since most brain pathologies present very low contrast with respect to healthy brain tissues, and they are quite difficult to identify and localize with this standard technique. In the presented algorithm, a set of pixels is added to the ROI only if the LBP identifies a similar texture, following the pattern recognition approaches. The RMC criteria adopted in this work are described and summarized by relations (1)–(3), where the segmented region of interest (ROI) is the union of pixel regions that satisfy the RMC as indicated in relation (2). If a pixel region cannot be added to the ROI because it does not satisfy the RMC, another direction is chosen, and the process is repeated until all the directions are explored. In the end, areas with similar textures are identified. It is worth noticing that the customized growing area algorithm only isolates a similar area, but it does not provide any information concerning the nature of tissues belonging to the segmented areas. To classify the tissue typologies, a machine learning technique based on the SVM algorithm is applied. The SVM classifies tissues into two distinct categories: health and pathologic. The SVM is previously trained with a suitable database that contains different samples of healthy tissues and brain pathologies. Once applied, the SVM marks areas containing health and pathologic tissues with a red and green color, respectively.

3. Experimental Assessment

In this section, the segmentation methodology proposed in the previous section is experimentally assessed by considering real MRIs provided by medical archives. A selected set of scenarios have been chosen from a big MRIs medical database containing different kinds of brain pathologies. The considered database comprises pre-surgical MR images of diffuse gliomas (according to the revised 2016 WHO classification [37] and meningiomas, and in particular: −364 diffuse gliomas: 118 low-grade tumors (GII e GIII), 33 oligodendrogliomas; the other are glioblastomas, −150 meningioma (75 GI, 60 GII, 15 GIII). The images have been retrospectively collected from both 1.5 T and 3 T scanners (GE Optima MR450w 1.5 T, Waukesha, WI, USA; Ingenia, 1.5 T, Philps, The Netherlands; Signa Excite, MAGNETOM Skyra, 3 T Siemens Healthcare, Erlangen, Germany). Conventional MRI protocol comprised the following: axial T1-weighted (T1w) fast spin-echo (FSE), axial T2-weighted (T2w) fast relaxation fast spin echo-propeller sequence (FRFSE-propeller), and axial T2w fluid-attenuated inversion recovery imaging (FLAIR). Section thickness between 4 and 5 mm, intersection gap (1 mm) and FOV (240 × 240 mm). After IV (gadobutrol, 0.1 mmol/kg) contrast-agent injection, a 3D fast-spoiled gradient-echo (FSPGR) sequence was acquired, 1 mm isotropic voxel). DWI was performed in the axial plane with an echo-planar sequence before injection of the contrast agent (gadobutrol, 0.1 mmol/kg) with a section thickness of 4 mm, intersection gap 1 mm, and FOV 240 × 240 mm. The b-values were 0 and 1000 s/mm² with diffusion gradients encoded in the x, y, and z directions to generate three sets of diffusion-weighted images. In particular, the experimental validation has been carried on considering three scenarios related to different brain pathologies. For the sake of comparison, the segmentations with the proposed methodology are compared with the ones obtained with 3D Slicer, a segmentation software tool widely known and commonly used by radiologists. Concerning the dataset used to train the SVM classifier aimed at identifying pathologies from healthy brain tissues, a set of 100 healthy and 100 pathological tissues samples have been manually collected by an expert clinical operator. The pathological as well as healthy tissues have been selected and diagnosed by expert neuroradiologist. In particular, each tissue sample is a matrix of $3\times 3$ elements with a spatial resolution of about 1 cm², the same considered by LBP and RMC algorithms. For the sake of comparisons, the proposed segmentation algorithm has been compared with other state of the art segmentation algorithm provided by Slicer 3D. A growing area methodology based on single, multiple seeds, and boundaries (aimed at manually limit the pathological area) have in particular been considered. In addition, a segmentation algorithm based on a manually imposed threshold has been considered. Unfortunately, all state of the art segmentation methodologies, provided by slicer 3D, were not able to properly segment the pathological area since the contrast between healthy and suspect tissues was too small. We must operate the segmentation with a slicer 3D manually.

3.1. First Scenario

In the first experiment, a patient with two brain pathologies is considered. We decided to test T1-weighted after contrast adjacentinjection T1W + C images, instead of T2W, for their low intrinsic contrast, except for those lesions that present contrast enhancement. An example of an MRI (axial T1W + C fast field echo (FFE) sequence) related to the first patient is reported in Figure 3. Two pathologic areas can be seen: one in the left parietal lobe (meningioma) identified with a dotted green line, the other in the right frontal lobe (low-grade glioma) identified with a dotted red line. To assess the methodology in realistic conditions the segmentation has been performed by an inexperienced operator (an engineer without any clinical experience in brain anatomy and pathologies); only a few pieces of information concerning how to recognize suspected areas have been provided by an expert clinical operator, which also diagnosed the patient’s pathology. It is worth noticing that the inexperienced operator, in the beginning, was not able to properly identify the low grade glioma because the pathological area presented a very low contrast in T1W + C the images with respect to the healthy tissue; the help of an expert physician was mandatory to identify the suspected area where the pathology belonged and properly set the initial seed. For the sake of comparison, the inexperienced operator made the segmentation by using the well-known Slicer 3D software, a segmentation tool widely used by researchers and radiologists that operate in the field of radiomics. Slicer 3D provides some unsupervised techniques that permit to easily identify and segment areas of interest. However, in this experiment, both threshold, and growing area from seed wizards, provided by Slicer 3D, do not provide useful results because the contrast between pathologies and healthy tissues is quite low. The segmentation was performed manually, and the 3D masks related to the two pathologies were retrieved. The manual segmentation with Slicer 3D software required about 40 min. The results are reported in Figure 4a,b, respectively. It is worth noticing that Slicer 3D permits the extraction of the 3D mask of the two pathologies but does not provide any information concerning tissues nature; the inexperienced operator thought that the pathology located in the left parietal lobe of the brain was a malignant tumour.

The validation continues by considering the proposed methodology based on the combination of the growing area algorithm and pattern recognition schema that requires only the indication of a pixel called seed and is located in a suspected area. For the considered scenario, the operator considers two different seeds, one for each suspected area. Then the growing area algorithm combined with the pattern recognition is started. The procedure is summarized in Figure 5a,b. In particular, Figure 5a reports the two seeds manually, indicated by the operator in the suspected areas, while Figure 5b reports the result of segmentation in the considered MRI slice. The whole procedure requires about three minutes to complete the whole segmentation.

With respect to the results obtained with slicer 3D software, the proposed tool provides information concerning the nature of identified areas. In particular, with reference to Figure 5b, the malignant tissues are identified with a red color area while the benignant areas, which do not represent a thread for the patient’s health is marked with a green color. The identification of tissue typologies is provided thanks to the dataset and machine learning technique based on the SVM algorithm, and the diagnosis was confirmed by the specialist who followed the patient.

3.2. Second Scenario

In the second experiment, a patient with a diffuse low grade glioma located on the left parietal lobe of the brain was considered. Also, in this case, the contrast between healthy and malignant tissue is not so evident, as it can be noticed from the MRI axial slice of the T1W + C FFE sequence, reported in Figure 6, the tumor located on the left parietal lobe of the brain is highlighted with a dotted red line. The segmentation of pathology, as in the previous case, is firstly performed with the slicer 3D tool. This scenario is a little bit more simple with respect to the previous one. In addition, in this case, the segmentation is performed manually since the growing area and the threshold tools of slicer 3D were not effective. The results of the segmentation procedure obtained with slicer 3D are reported in Figure 7a,b. In particular, Figure 7a reports the three-dimensional reconstruction of the pathology while Figure 7b shows only the reconstruction of one axial slice. The whole segmentation procedure required about 20 min; also in this case, the software tool does not provide any indication related to the tissue nature. Then the segmentation is performed with the proposed method that requires only the indication of a pixel (the seed) located in the suspected area. The seed is provided by the operator, then the growing area algorithm based on texture recognition and the SVM classifier is applied. The semi-unsupervised procedure requires less than two minutes, and it does not require further operator actions. The results are reported in Figure 8a,b. Figure 8a shows the seed indication, while Figure 8b reports the segmentation result. For the sake of comparison, the results obtained with the two considered segmentation tools are compared and reported in Figure 9. The right side of Figure 9 reports the segmentation obtained with the proposed method; the suspected area is highlighted with a red color that identifies, thanks to the SVM classifier, the possibility of a tumor. The right side of Figure 9 presents the segmentation manually provided with the Slicer 3D software tool. The suspected area is indicated with a light green area, and no indication related to the tissue’s nature is provided. As can be noticed from the results shown in Figure 9 the two segmentation procedures provide very similar results; the difference between the suspected area is less than 5%.

3.3. Third Scenario

In the last experiment, a patient with an extended lesion with undefined borders located in the left frontal lobe was considered. An MRI axial section related to the T1W + C FFE sequence is reported in Figure 10; the pathology is highlighted with a dotted red line. In this scenario, the pathology is quite evident due to its dimensions, despite the low contrast between the healthy and pathologic tissue, the inexperienced operator immediately identified the suspected area. Also, in this case, the low contrast between healthy and pathologic tissue makes the use of slice 3D semi unsupervised tools, namely threshold and dual seeds growing area, useless. Therefore, the segmentation is performed manually. The whole procedure, due to the extension of pathology, required about half an hour. The results of slicer 3D segmentation are reported in Figure 11a,b. In particular, Figure 11b reports the 3D volume of the pathology. Also, in this case, the segmentation process does not provide any information concerning the tissue nature. Then, the segmentation is performed considering the proposed method. The results are reported in Figure 12a,b. In particular, Figure 12a reports the first phase of the procedure with the initial seed, inserted manually by the operator, highlighted in red color. The segmentation result is reported in Figure 12b. As it can be noticed, the suspected area is identified with a satisfactory degree of accuracy, moreover, indications concerning the possible nature of the pathology are provided by the SVM classification, which marks the area with a red color. The results of the two segmentation methodologies are compared in Figure 13; both tools permit to identify with a satisfactory degree of accuracy the suspected area. The proposed methodology performed the segmentation quickly and without further actions of the human operator, saving time and permitting expert clinicians to manage more patients. An added value of the method is that it permits inexperienced operators or students to learn from the indications provided by the SVM classification how to correctly identify different pathologies. The following

Table 1. Results comparisons between the results obtained with the two segmentation tools.

Slicer 3D	Scenario	Tissue Identification	Identified Area [cm3]
	Patient 1	NO	$26.4$
	Patient 2	NO	$18.2$
	Patient 3	NO	$45.8$
Proposed Method
	Patient 1	YES	$23.8$
	Patient 2	YES	$19.5$
	Patient 3	YES	$43.2$

summarizes and compares the results obtained with the two segmentation tools for all the three considered scenarios. As can be noticed from the data reported in Table 1, the considered methodology provides very promising results. In particular, the segmented area obtained with the semi-unsupervised proposed method is quite similar to the area manually selected with the Slicer 3D tool.

4. Conclusions

In this work, a novel approach for the semi unsupervised segmentation of brain pathologies for radiomics applications has been presented and experimentally assessed with MRI, in DICOM format, provided by a dataset of real patients. In particular, the proposed method focused on the first two phases of the radiomics process; the first phase is aimed at pathology segmentation by using a combination between a growing area algorithm and texture analysis. In the second phase, a machine learning technique based on a support vector machine is aimed to provide a tissues identification, which can help inexperienced operators to make the correct pathology diagnosis or help expert operators to process more patients. The obtained preliminary results were compared with other state-of-the-art segmentation techniques, and they demonstrated the potentialities and advantages of the proposed method. Future works will be aimed at improving the identification and segmentation techniques, extending the methodology to other pathologies thanks to the huge amount of clinical data actually presented in our database, and integrating the proposed algorithm into existing segmentation tools such as Slicer 3D.