Next Article in Journal
Relative Entropy Method Applied for the Fatigue Life Distribution of Carbon Fiber/Epoxy Composites
Next Article in Special Issue
Influences of Different Architectures on the Thermodynamic Performance and Network Structure of Aircraft Environmental Control System
Previous Article in Journal
The Entropy Universe
Previous Article in Special Issue
Causal Geometry
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Computational Complexity Reduction of Neural Networks of Brain Tumor Image Segmentation by Introducing Fermi–Dirac Correction Functions

by
Yen-Ling Tai
1,
Shin-Jhe Huang
1,2,
Chien-Chang Chen
1,2,3 and
Henry Horng-Shing Lu
3,4,5,*
1
Bio-Microsystems Integration Laboratory, Department of Biomedical Sciences and Engineering, National Central University, Taoyuan City 32001, Taiwan
2
Chronic Disease Research Center, National Central University, Taoyuan City 32001, Taiwan
3
Shing-Tung Yau Center, National Chiao Tung University, Hsinchu 30010, Taiwan
4
Institute of Statistics, National Chiao Tung University, Hsinchu 30010, Taiwan
5
Institute of Statistics, National Yang Ming Chiao Tung University, Hsinchu 30010, Taiwan
*
Author to whom correspondence should be addressed.
Entropy 2021, 23(2), 223; https://doi.org/10.3390/e23020223
Submission received: 16 December 2020 / Revised: 29 January 2021 / Accepted: 7 February 2021 / Published: 11 February 2021
(This article belongs to the Collection Feature Papers in Information Theory)

Abstract

:
Nowadays, deep learning methods with high structural complexity and flexibility inevitably lean on the computational capability of the hardware. A platform with high-performance GPUs and large amounts of memory could support neural networks having large numbers of layers and kernels. However, naively pursuing high-cost hardware would probably drag the technical development of deep learning methods. In the article, we thus establish a new preprocessing method to reduce the computational complexity of the neural networks. Inspired by the band theory of solids in physics, we map the image space into a noninteraction physical system isomorphically and then treat image voxels as particle-like clusters. Then, we reconstruct the Fermi–Dirac distribution to be a correction function for the normalization of the voxel intensity and as a filter of insignificant cluster components. The filtered clusters at the circumstance can delineate the morphological heterogeneity of the image voxels. We used the BraTS 2019 datasets and the dimensional fusion U-net for the algorithmic validation, and the proposed Fermi–Dirac correction function exhibited comparable performance to other employed preprocessing methods. By comparing to the conventional z-score normalization function and the Gamma correction function, the proposed algorithm can save at least 38% of computational time cost under a low-cost hardware architecture. Even though the correction function of global histogram equalization has the lowest computational time among the employed correction functions, the proposed Fermi–Dirac correction function exhibits better capabilities of image augmentation and segmentation.

1. Introduction

Deep learning methods at present are playing an indispensable role in the field of computer vision. The relevant fruitful achievements also facilitate and change the fashion of synergy between clinical diagnosis and computerized assistance. The scope of application covers the demands of computer-guided pathological inspection [1,2,3,4], brain neural circuit mapping and tracking [1,5,6,7,8,9], specific tissue detection in image-based datasets [5,10,11,12,13], and other clinical applications. Among these applications, neural network (NN)-based recognition methods that are capable of detecting life-threatening abnormalities from image-based datasets especially attract the attention of both scientific and engineering participants [2,11,14,15,16,17] and gradually replace conventional approaches. Within these techniques, the emergence of fully convolutional neural networks (FCN) has successfully acquired more attention, and the FCN-based methods have further elevated the performance of convolutional neural networks (CNNs) in the field of modern medical image recognition and segmentation [15,18,19,20]. In applications of clinical practice, on the other hand, the medical image datasets are often sparse, so the technical development in these fields is leading by the NN models that are suitable for dealing with small-size datasets. Thus, starting from FCN, over V-net [15], U-net [21,22,23], and the auxiliary of feature extraction blocks [24,25,26], then to the attention U-net [27,28] and dimension fusion U-net (D-Unet) [29], the relevant methods gradually become the developing core in medical image analysis. Meanwhile, due to the extremely high mortality [1,13], the modality investigations of malignant brain tumors have also affected the mainstream techniques of the brain tumor image segmentation [1,30] and the procedures of image-guided surgery.
However, for pursuing the performance of model prediction, continuing to increase model depths and kernel numbers would become the burden of hardware architecture in the points of view of power control and the maintenance of computational complexity. For instance, the consumption of GPU memory and the computational complexity are all proportional to the size and number of convolutional kernels. Large numbers of parameters in training procedures also challenge the memory sizes of GPUs, the design of algorithmic schemes, and power requirements [14]. Additionally, the labor intervention and time cost in the stage of data labeling also obstruct the progression of technical development. Thus, the development of deep learning methods would step into the inevitable situations of high-cost hardware and high computational complexity configuration [14,16]. The combination of contour-based models and statistical-learning-based methods offer alleviant approaches by utilizing prior information [3,31,32,33,34] to reduce the high-cost scheme caused by the deep learning schemes. For instance, the statistical models of the random walks are used for tracking and segmenting thin and elongated image components by assigning seed points on the ends of these components [35]. Region-based contour models combined with statistical classification processes have been utilized to solve the problems of intensity inhomogeneity that occurred within image blocks [36,37,38] by arranging curve functions on the blocks of interest. Nevertheless, the manual intervention of the combinatorial approaches also causes different problems in artificially predefining labels of seeded image components [14,39,40,41]. Physical-based methods also offer another track to deal with these undergoing problems. The data density functional theory (DDFT) successfully extracts the intensity features from brain tumor image datasets by estimating the morphological heterogeneity in energy spaces [42,43] so that it may remove the undesired manual intervention. However, the architecture that can bridge the deep learning procedures and DDFT has not appeared yet.
Therefore, inspired by the merits of the physical framework of DDFT and also considering the advantages of deep learning methods, we introduced the Fermi–Dirac distribution to be a correction function in the following procedures. By employing the theoretical scheme of DDFT, we mapped the three-dimensional image space into a noninteracting physical system and treated each image voxel as a physical particle. Under this condition, the Fermi–Dirac distribution was used to analyze the particle behavior and classify these particle clusters according to their intrinsic energies. Thus, the particle clusters formed a specific morphological structure according to their energy property in the physical system. The estimates were then mapped back to the image space, and the image voxels were decomposed by considering their morphological heterogeneity. In the article, we compared the performance of the Fermi–Dirac correction function in Unet-based architectures. The experimental results validated the superiority of the proposed Fermi–Dirac correction function under the purpose of computational complexity reduction.

2. Datasets and Methods

2.1. Datasets

For fair comparisons, we employed the open datasets from the multimodal brain tumor segmentation challenge (BraTS) 2019 [44,45] to test the proposed algorithmic schemes. The type of datasets includes T1, T1-CE (T1-weighted contrast-enhanced), T2-weighted, and T2-FLAIR (Fluid-Attenuated Inversion Recovery). The datasets were skull-stripped, interpolated to the same resolution, and labeled by experienced neuro-radiologists. There are four labeled types for the tumor anatomic structures, which comprise the enhancing tumor (ET), the peritumoral edema, the necrotic and nonenhancing tumor core (TC), and the background. There are 300 testing sets and 35 validation sets, and 155 pieces of images in each individual set. The dimensions of each image size are 240 pixels both for height and width, and all of the images are in axial views.

2.2. Theoretical Scheme of Fermi–Dirac Correction Function

According to the theoretical framework of DDFT, an arbitrary high-dimensional image space can be isomorphically mapped into a pseudo-physical space with the same dimensionality once the data length is fixed [42]. Then, image voxel components are correspondingly mapped as particle-like clusters. Thus, there exists a bijective relation between the voxel intensity distribution and the Fermi hypersurface function. In other words, the voxel intensity can be mapped as a corresponding energy value within the pseudo-physical space. The voxel intensity distribution becomes a function of the Fermi hypersurface function under this theoretical framework, and vice versa. As the Fermi hypersurface function has an adequately theoretical definition in the pseudo-physical space [42], we could safely introduce appropriate physical properties to deal with the encountering situations. For instance, the Hamiltonian curves and the Lagrangian density functional could be exploited to measure the most-possible-cluster numbers and cluster boundaries, respectively [42,46]. Thus, we can delineate the morphological structure of images by applying these properties of energy under the framework of DDFT. For image segmentation, we introduced a specific particle distribution in terms of energy properties, named Fermi–Dirac distribution, to analyze and pre-decompose the image clusters according to their morphological structures.
Under the DDFT framework, the pseudo-physical space is spanned onto a noninteracting (or weakly interacting) system so that we can establish required conditions by considering each cluster to be a subsystem and each particle (mapped from a voxel in the image space) to be in a single-particle state. Additionally, as the locations and the corresponding intensity distribution of voxels within an image space are all fixed, there is no interaction between voxels, and this space can be treated as a stationary and “frozen” system. In other words, the clusters will not exchange particles and energy with their surroundings in the noninteracting system; thus, we can exploit an additional condition of zero temperature into this system. Thus, the probability of observing the number of particles n in the single-particle state can be estimated by introducing the perspective of thermodynamic properties:
P n = e n ε n ε F / ε b Z ,  
where the parameters ε , ε F , and ε b are the energy of the single-particle state, the Fermi energy at zero temperature, and the correction energy for dimensional balance, respectively. The symbol Z represents a partition function for the subsystem of interest:
Z = n e ε ε F / ε b n .  
The grand potential G V can be then defined by designating the particles to be pseudo-fermions or pseudo-bosons, respectively:
G V = ε b ln n = 0 1 Z = ε b ln 1 + e ε ε F / ε b ,   for   pseudo fermions ε b ln n = 0 Z = ε b ln 1 e ε ε F / ε b ,   for   pseudo bosons   .
Then, the corresponding particle distribution functions for each particle type can be expressed formally:
n ^ = G V ε F = 1 e ε ε F / ε b + 1 ,   for   pseudo fermions 1 e ε ε F / ε b 1 ,   for   pseudo bosons   .  
The expressions of these distributions are the well-known Fermi–Dirac (FD) distribution and Bose–Einstein (BE) distribution, respectively. Equation (4) lists their representations in our proposed pseudo-physical system. It should be emphasized that the n takes 0 and 1 for pseudo-fermions and 0, 1, 2, …, for pseudo-bosons. Theoretically, the maximum of n in different particle types (fermions or bosons) represents the maximum capacity of particles a single-particle state can contain.
We found an interesting phenomenon by simultaneously comparing the theoretical form of the FD distribution to that of the sigmoid function and the z-score normalization. The coarse form of the FD distribution is exactly the sigmoid function, so the pseudo-particle distribution will be squeezed into the range of 0 , 1 as expected. On the other hand, as the form of the mathematical kernel of the FD distribution,   ε ε F / ε b , is also similar to the z-score normalization, we found a route to determine well the parameter definitions of pseudo-energy by comparing their parameters. Due to the voxel intensity in the image space and the energy in the pseudo-physical space being bijective, we correspondingly assigned these parameters, ε , ε F , and ε b , to be the intensity value distribution of each image, the global mean intensity of each testing set, and the minus global standard deviation of intensity of each testing set, respectively. Thus, the FD distribution exhibits its own mathematical merits for data processing by fusing the characteristics of the sigmoid function and the z-score normalization. Mathematically, we can clarify the difference between the FD and the BE distributions by examining their behavior under extreme conditions. When the value of ε ε F in the kernel approaches zero, it also means that the intensity approaches its mean value in the subsystem, the value of the FD distribution will reach 0.5, whereas that of the BE distribution will approach infinity. Thus, the BE distribution intrinsically reveals its own anti-behavior compared to the sigmoid function. When the kernel is much larger than 1, both FD and BE distributions will have the same mathematical form n ^ e ε ε F / ε b . This form is also a well-known Maxwell–Boltzmann (MB) distribution. Figure 1 illustrates each mathematical behavior of the mentioned distributions with different values of minus global standard deviation ε b .
As the bosons are allowed to have the same energy value, it would lead to a consequence that all of the voxels are classified in the same cluster and become indistinguishable. On the other hand, the intrinsic mathematical form of the BE distribution also causes itself to be an infinite value while the voxel intensity approaches its mean value; thus, these inconvenient properties make the BE distribution hardly a correction function for data normalization and data filtering in the data processing. A similar situation would happen to the MB distribution. Additionally, it is also impracticable to estimate the mean intensity of each subsystem technically due to the undesired computational complexity; thus, we utilized the global mean intensity of the whole system to conquer this issue. Therefore, by considering both the theoretical and the technical merits, the FD distribution is more suitable as a correction function in the data preprocessing procedures.

2.3. Experimental Framework

To test the performance of the proposed FD correction function, we employed the conventional preprocessing methods, a z-score normalization function, a Gamma correction function, and a three-dimensional (3D) global histogram equalization function [47], for the performance comparison and used a null-preprocess as a baseline. As mentioned above, the mathematical structure of the FD correction function was established by a coarse skeleton and a kernel embedded in the skeleton. Theoretically, the skeleton is a sigmoid function, and the kernel function has the same form as the z-score normalization function. Thus, we adopted the z-score normalization function as one of the preprocessing ways for a fair comparison of performance and then investigated the differences between these functions. On the other hand, we also adopted the Gamma correction function due to its performance being adjustable by assigning an appropriate parameter. Thus, we can also compare the performance of conventional supervised correction functions to the proposed FD correction function. The way to determine the optimized Gamma parameter directly relies on a large number of experiments. We extracted the Gamma parameters in a set of 0.2 ,   0.4 ,   0.6 ,   0.8 ,   1.0 ,   1.2 ,   1.5 ,   2 . According to the performance of image segmentation, we adopted the value of 0.6 as the Gamma parameter due to it offering the best performance in the image segmentation. Finally, we compared the capability between the global histogram equalization function and our functions on the topic of image augmentation.
Thus, five preprocessing scenarios were employed for the performance comparison under a specific neural network model. To achieve this purpose, we utilized the D-Unet-based structure for the brain tumor image segmentation. Figure 2 illustrates the relevant framework. D-Unet is established based on the conventional U-net structure, and thus, its structure is suitable for dealing with small-size datasets. Furthermore, a delicate function component named the dimension-transform-block [29] is employed in the encoding procedure of D-Unet, then all of the two- and three-dimensional extracted features would be fused through these blocks. As illustrated in Figure 2, two dimension-transform-blocks (indicated as the red columns) were used in the proposed D-Unet scheme. Meanwhile, to reduce the undesired computational cost, only the fused features and the two-dimensional features are fed into the decoding procedure. Thus, the D-Unet structure can simultaneously preserve the significant high-dimensional features and offer an acceptable computational cost. Under the framework of the employed D-Unet structure, the dimensions of image sizes were scaled down to 160 pixels both for height and width, and the input types of each image included four channels, T1, T1-CE, T2-weighted, and T2-FLAIR. The adopted numbers of filters are indicated above each layer, and only one channel, the whole tumor (WT), would be output for the performance comparison. The output activation layer was a sigmoid function. The number of epochs and batch size were set to 30 and 32, respectively, for all preprocessing scenarios. The optimizer and the loss function were Adam and the three-dimensional soft dice loss function [48,49], respectively. The analytic form of the soft dice loss function is [15,48,49,50]:
D L = 1 2 i N p i g i + ϵ i N p i 2 + i N g i 2 + ϵ .
The number N is the total data length and the symbol ϵ = 10 5 was adopted to avoid D L diverging. The parameters p i 0 , 1 and g i 0 , 1 are respectively the binary predicted segmentation and ground truth labeling. Ref. [50] offers the source codes of the soft dice loss. The initial value of the learning rate of all scenarios is set to 0.00015. The relevant model parameters and functions are collected and listed in Table 1.

3. Results

All of the experiments were executed with the hardware specification of i9-9980XE CPU @ 3 GHz, 18 cores, and one GPU of NVIDIA GeForce RTX 2080 Ti. Figure 3 illustrates the visualized comparison between the employed preprocessing methods and the proposed FD correction function. To theoretically inspect the capability of the FD correction function, two values of ε b were used to construct the kernel. The corresponding values for FD 1 and FD 2 correction functions were ε b = 1 and ε b = minus the global standard deviation of intensity of each testing set, respectively. In the words, the FD 2 correction function is a standard FD correction function. To clarify the main difference between FD 1 and FD 2 correction functions, their analytical forms are:
FD 1 = 1 e ε ε F + 1   and   FD 2 = 1 e ε ε F / | ε b | + 1 .
By comparing to the results of other preprocessing methods, all of the proposed FD-type correction functions can remove the insignificant components of the brain tissue images, especially for the cases of T2 and T2-FLAIR images. Meanwhile, the FD 2 correction function can further enhance the filtered image components, and so did the 3D global histogram equalization function. The z-score normalization and the Gamma correction functions exhibit similar preprocessing results.
The performance comparison of all preprocessing scenarios is listed in Table 2. The dice score of WT without any preprocessing is the lowest one as expected, and thus, we used its corresponding computational time as the baseline. All dice scores of the employed preprocessing methods are similar, and the FD-type correction functions present slightly high scores. Even though the dice score of the z-score normalization function reaches a good level, its computational time is much higher than that of baseline. It might be that the execution efficiency of this method was not optimized. On the other hand, the Gamma correction function also exhibits good performance in both the dice score and the corresponding computational time. However, it costed a large number of experiments and labor intervention to choose the optimized gamma parameter. In our experiments, the final optimized parameter of the Gamma correction function was 0.6. Although the dice score of the 3D global histogram equalization function is the lowest one compared to all employed methods, its computational time reveals the superior performance. Table 3 represents a confusion-matrix table to show the comparison of accuracy, sensitivity (recall), and precision between the proposed FD-type correction functions and the conventional correction methods. The definition of each indicator is as follows [51,52]:
Accuracy = TP + TN TP + TN + FP + FN ,
Recall Sensitivity = TP TP + FN ,  
Precision = TP TP + FP .  
The factors TP, TN, FP, and FN are the true positive, true negative, false positive, and false negative, respectively. The total summation of these four factors is around 812,535 with a small residual error. Then, the total number of tumors is defined as TP + FN and has a value of about 21,191.
Thus, we further compared the performance between the 3D global histogram equalization function and the proposed FD-type correction functions, wherein the tumor labels were categorized into WT, TC, and ET in detail. The output channels of the D-Unet became three branches. Table 4 and Table 5 respectively list the performance comparison and the corresponding confusion-matrix table. We preserved all of the model parameters for a fair comparison but replaced the output layer with a softmax activation function for multi-label detections. We also presented the corresponding dice scores of each label for the training stage and the validation stage. As expected, the dice scores of the validation stage are much lower than those of the training stage. All computational times in this experiment are comparable, but the required time costs were all raised due to the multi-label detections. In this experiment, all of the dice scores of the FD-type correction functions are higher than that of the 3D global histogram equalization function. We might deduce these reasons by utilizing their visualized results, as illustrated in Figure 4.
All types of image sets were processed with the 3D global histogram equalization function and the proposed FD-type correction functions, and the preprocessed images are illustrated in Figure 4a. The image inclinations resulted from the augmentation of affine transformations. Then, the corresponding results of brain tumor image segmentation are illustrated in Figure 4b. As the tumor recognition of ET type is typically hard, the corresponding dice scores are quite low, as listed in Table 3, for all employed preprocessing methods. The visualized results of the ET type illustrated in Figure 4b reflect this fact. We can recognize that the image structures of ET type are meticulous compared to TC and WT, and thus, that it would cause difficulty in the ET tumor image recognition. It should also be emphasized that the segmented results of TC and WT types are similar in all employed methods in the experiments. Then, the FD 2 correction function shows better performance in recognizing the ET tumor edges than the other two methods. Therefore, the results validate that the FD 2 correction function, i.e., the FD correction function with a kernel of z-score normalization function, exhibits superior performance in both computational time reduction and brain tumor image segmentation by comparing other conventional preprocessing methods.

4. Discussion and Conclusions

We theoretically reconstructed the Fermi–Dirac distribution function to be a correction function for brain tumor image segmentation. The skeleton form and the kernel of the proposed Fermi–Dirac correction function are respectively a sigmoid function and a z-score normalization function. Thus, the Fermi–Dirac correction function inherits the merits of these functions. It is suitable for dealing with intensity normalization and filtering insignificant image components simultaneously. We also explained the mathematical inconvenience of the Bose–Einstein and Maxwell–Boltzmann distribution functions in imaging processing.
We then compared the performance of the proposed correction function with other conventional preprocessing methods, such as the standard z-score normalization, the Gamma correction, and the three-dimensional global histogram equalization. We also changed the normalization factor of the kernel to inspect the mathematical properties of the proposed Fermi–Dirac correction functions. The experimental consequences validated that the proposed correction function can have better computational cost than those of conventional and supervised methods. Then, the proposed correction function would inhibit the insignificant image components and enhance the filtered image segments. As the experimental results also showed that the three-dimensional global histogram equalization can offer a similar computational capability as the proposed correction method, we further fed those methods into a complete comparison. The experimental results validate the superiority of the proposed Fermi–Dirac correction function.
In further performance comparison, the proposed Fermi–Dirac correction function exhibits its superior computational and recognizing capability for the brain tumor image segmentation with the multimodal types of ET, TC, and WT. Therefore, the proposed Fermi–Dirac correction function can not only reduce the computational complexity but also reinforce the image component recognition and segmentation.

Author Contributions

Conceptualization, C.-C.C. and H.H.-S.L.; methodology, Y.-L.T. and S.-J.H.; software, Y.-L.T. and S.-J.H.; validation, C.-C.C. and Y.-L.T.; formal analysis, Y.-L.T. and S.-J.H.; investigation, Y.-L.T. and S.-J.H.; data curation, C.-C.C. and Y.-L.T.; writing—original draft preparation, Y.-L.T. and C.-C.C.; writing—review and editing, C.-C.C. and H.H.-S.L.; project administration, C.-C.C. and H.H.-S.L.; funding acquisition, C.-C.C. and H.H.-S.L. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the Ministry of Science and Technology, Taiwan, grant number MOST 108-2221-E-008-081-MY3 and 107-2118-M-009-006-MY3.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Publicly available datasets were analyzed in this study. This data can be found here: https://www.med.upenn.edu/cbica/brats2019/data.html.

Acknowledgments

This work was also supported in part by the Higher Education Sprout Project of National Chiao Tung University and the Yushan Scholar Program from the Ministry of Education, Taiwan.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Bauer, S.; Wiest, R.; Nolte, L.; Reyes, M. A survey of MRI-based medical image analysis for brain tumor studies. Phys. Med. Biol. 2013, 58, R97–R129. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  2. Zhang, W.; Li, R.; Deng, H.; Wang, L.; Lin, W.; Ji, S.; Shen, D. Deep convolutional neural networks for multi-modality isointense infant brain image segmentation. Neuroimage 2015, 108, 214–224. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  3. Gurusamy, R.; Subramaniam, D.V. A Machine Learning Approach for MRI Brain Tumor Classification. CMC Comput. Mat. Contin. 2017, 53, 91–108. [Google Scholar]
  4. Wang, S.; Zhang, Y.; Liu, G.; Phillip, P.; Yuan, T.F. Detection of Alzheimer’s Disease by Three-Dimensional Displacement Field Estimation in Structural Magnetic Resonance Imaging. J. Alzheimers Dis. 2016, 50, 233–248. [Google Scholar] [CrossRef]
  5. Qiu, A.; Brown, T.; Fischl, B.; Ma, J.; Miller, M.I. Atlas Generation for Subcortical and Ventricular Structures with Its Applications in Shape Analysis. IEEE Trans. Image Process. 2010, 19, 1539–1547. [Google Scholar] [CrossRef] [Green Version]
  6. Kim, J.; Duchin, Y.; Shamir, R.R.; Patriat, R.; Vitek, J.; Harel, N.; Sapiro, G. Automatic localization of the subthalamic nucleus on patient-specific clinical MRI by incorporating 7 T MRI and machine learning: Application in deep brain stimulation. Hum. Brain Mapp. 2019, 40, 679–698. [Google Scholar] [CrossRef]
  7. Kreshuk, A.; Straehle, C.N.; Sommer, C.; Koethe, U.; Knott, G.; Hamprecht, F.A. Automated segmentation of synapses in 3D EM data. In Proceedings of the 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro, Chicago, IL, USA, 30 March–2 April 2011; pp. 220–223. [Google Scholar]
  8. Vasilkoski, Z.; Stepanyants, A. Detection of the optimal neuron traces in confocal microscopy images. J. Neurosci. Methods 2009, 178, 197–204. [Google Scholar] [CrossRef] [Green Version]
  9. Wang, Y.; Narayanaswamy, A.; Tsai, C.L.; Roysam, B. A Broadly Applicable 3-D Neuron Tracing Method Based on Open-Curve Snake. Neuroinformatics 2011, 9, 193–217. [Google Scholar] [CrossRef]
  10. Park, S.H.; Gao, Y.; Shen, D. Multiatlas-Based Segmentation Editing with Interaction-Guided Patch Selection and Label Fusion. IEEE Trans. Biomed. Eng. 2016, 63, 1208–1219. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  11. Xu, Y.; Wang, Y.; Yuan, J.; Cheng, Q.; Wang, X.; Carson, P.L. Medical breast ultrasound image segmentation by machine learning. Ultrasonics 2019, 91, 1–9. [Google Scholar] [CrossRef] [PubMed]
  12. Zarpalas, D.; Gkontra, P.; Daras, P.; Maglaveras, N. Gradient-Based Reliability Maps for ACM-Based Segmentation of Hippocampus. IEEE Trans. Biomed. Eng. 2014, 61, 1015–1026. [Google Scholar] [CrossRef]
  13. Rajinikanth, V.; Satapathy, S.C.; Fernandes, S.L.; Nachiappan, S. Entropy based segmentation of tumor from brain MR images—A study with teaching learning based optimization. Pattern Recognit. Lett. 2017, 94, 87–95. [Google Scholar] [CrossRef]
  14. Ma, C.; Luo, G.; Wang, K. Concatenated and Connected Random Forests with Multiscale Patch Driven Active Contour Model for Automated Brain Tumor Segmentation of MR Images. IEEE Trans. Med. Imaging 2018, 37, 1943–1954. [Google Scholar] [CrossRef]
  15. Milletari, F.; Navab, N.; Ahmadi, S. V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation. In Proceedings of the Fourth International Conference on 3D Vision (3DV), Stanford, CA, USA, 25–28 October 2016; pp. 565–571. [Google Scholar]
  16. Smistad, E.; Falch, T.L.; Bozorgi, M.; Elster, A.C.; Lindseth, F. Medical image segmentation on GPUs—A comprehensive review. Med. Image Anal. 2015, 1, 1–18. [Google Scholar] [CrossRef] [PubMed]
  17. Chen, L.; Papandreou, G.; Kokkinos, I.; Murphy, K.; Yuille, A.L. DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs. IEEE Trans. Pattern Anal. Mach. Intell. 2018, 40, 834–848. [Google Scholar] [CrossRef] [PubMed]
  18. Dasgupta, A.; Singh, S. A fully convolutional neural network based structured prediction approach towards the retinal vessel segmentation. In Proceedings of the IEEE 14th International Symposium on Biomedical Imaging, Melbourne, Australia, 18–21 April 2017; pp. 248–251. [Google Scholar]
  19. Kim, S.; Luna, M.; Chikontwe, P.; Park, S.H. Two-Step U-Nets for Brain Tumor Segmentation and Random Forest with Radiomics for Survival Time Prediction. In Proceedings of the International MICCAI Brainlesion Workshop, Shenzhen, China, 17 October 2019; pp. 200–209. [Google Scholar]
  20. Pei, L.; Vidyaratne, L.; Rahman, M.M.; Iftekharuddin, K.M. Context aware deep learning for brain tumor segmentation, subtype classification, and survival prediction using radiology images. Sci. Rep. 2020, 10, 1–11. [Google Scholar] [CrossRef] [PubMed]
  21. Pei, L.; Vidyaratne, L.; Rahman, M.M.; Shboul, Z.A.; Iftekharuddin, K.M. Multimodal Brain Tumor Segmentation and Survival Prediction Using Hybrid Machine Learning. In Proceedings of the International MICCAI Brainlesion Workshop, Shenzhen, China, 17 October 2019; pp. 73–81. [Google Scholar]
  22. Rehman, M.U.; Cho, S.; Kim, J.H.; Chong, K.T. BU-Net: Brain Tumor Segmentation Using Modified U-Net Architecture. Electronics 2020, 9, 2203. [Google Scholar] [CrossRef]
  23. Ronneberger, O.; Fischer, P.; Brox, T. U-Net: Convolutional Networks for Biomedical Image Segmentation. In Lecture Notes in Computer Science; Navab, N., Panizo, L., Islam, M.M.M., Varela, M.L.R., Ali Mohamad, N., Xiao, Z., Eds.; Springer: Cham, Switzerland, 2015; Volume 9351, pp. 234–241. [Google Scholar]
  24. Li, X.; Wang, W.; Hu, X.; Yang, J. Selective Kernel Networks. In Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, 15–20 June 2019; pp. 510–519. [Google Scholar]
  25. Hu, J.; Shen, L.; Sun, G. Squeeze-and-Excitation Networks. In Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–22 June 2018; pp. 7132–7141. [Google Scholar]
  26. Woo, S.; Park, J.; Lee, J.Y.; Kweon, I.S. CBAM: Convolutional block attention module. In Proceedings of the 15th the European Conference on Computer Vision (ECCV), Munich, Germany, 8–14 September 2018; pp. 3–19. [Google Scholar]
  27. Abraham, N.; Khan, N.M. A Novel Focal Tversky Loss Function with Improved Attention U-Net for Lesion Segmentation. In Proceedings of the 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), Venice, Italy, 8–11 April 2019; pp. 683–687. [Google Scholar]
  28. Schlemper, J.; Oktay, O.; Schaap, M.; Heinrich, M.; Kainz, B.; Glocker, B.; Rueckert, D. Attention gated networks: Learning to leverage salient regions in medical images. Med. Image Anal. 2019, 53, 197–207. [Google Scholar] [CrossRef] [PubMed]
  29. Zhou, Y.; Huang, W.; Dong, P.; Xia, Y.; Wang, S. D-UNet: A dimension-fusion U shape network for chronic stroke lesion segmentation. IEEE/ACM Trans. Comput. Biol. Bioinform. 2019, 1. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  30. Myronenko, A. 3D MRI Brain Tumor Segmentation Using Autoencoder Regularization. In Lecture Notes in Computer Science; Crimiet, A., Navab, N., Panizo, L., Islam, M.M.M., Varela, M.L.R., Ali Mohamad, N., Eds.; Springer: Cham, Switzerland, 2019; Volume 11384, pp. 311–320. [Google Scholar]
  31. Dawngliana, M.; Deb, D.; Handique, M.; Roy, S. Automatic brain tumor segmentation in MRI: Hybridized multilevel thresholding and level set. In Proceedings of the 2015 International Symposium on Advanced Computing and Communication (ISACC), Silchar, India, 14–15 September 2015; pp. 219–223. [Google Scholar]
  32. Cai, H.; Yang, Z.; Cao, X.; Xia, W.; Xu, X. A New Iterative Triclass Thresholding Technique in Image Segmentation. IEEE Trans. Image Process. 2014, 23, 1038–1046. [Google Scholar] [CrossRef]
  33. Feng, Y.; Zhao, H.; Li, X.; Zhang, X.; Li, H. A multi-scale 3D Otsu thresholding algorithm for medical image segmentation. Digit. Signal Prog. 2017, 60, 186–199. [Google Scholar] [CrossRef] [Green Version]
  34. Li, Y.; Bai, X.; Jiao, L.; Xue, Y. Partitioned-cooperative quantum-behaved particle swarm optimization based on multilevel thresholding applied to medical image segmentation. Appl. Soft. Comput. 2017, 56, 345–356. [Google Scholar] [CrossRef]
  35. Dong, X.; Shen, J.; Shao, L.; Gool, L.V. Sub-Markov Random Walk for Image Segmentation. IEEE Trans. Image Process. 2016, 25, 516–527. [Google Scholar] [CrossRef] [Green Version]
  36. Li, C.; Ding, Z.; Gatenby, J.C.; Metaxas, D.N.; Gore, J.C. A Level Set Method for Image Segmentation in the Presence of Intensity Inhomogeneities with Application to MRI. IEEE Trans. Image Process. 2011, 20, 2007–2016. [Google Scholar]
  37. Pratondo, A.; Chui, C.-K.; Ong, S.-H. Integrating machine learning with region-based active contour models in medical image segmentation. J. Vis. Commun. Image R. 2017, 43, 1–9. [Google Scholar] [CrossRef]
  38. Wang, L.; Pan, C. Robust level set image segmentation via a local correntropy-based K-means clustering. Pattern Recognit. 2014, 47, 1917–1925. [Google Scholar] [CrossRef]
  39. Norouzi, A.; Rahim, M.S.M.; Altameem, A.; Saba, T.; Rad, A.E.; Rehman, A.; Uddin, M. Medical Image Segmentation Methods, Algorithms, and Applications. IETE Tech. Rev. 2014, 31, 199–213. [Google Scholar] [CrossRef]
  40. Singh, P.; Bhadauria, H.S.; Singh, A. Automatic brain MRI image segmentation using FCM and LSM. In Proceedings of the 3rd International Conference on Reliability, Infocom Technologies and Optimization, Noida, India, 8–10 October 2014; pp. 1–6. [Google Scholar]
  41. Natarajan, P.; Krishnan, N.; Kenkre, N.S.; Nancy, S.; Singh, B.P. Tumor detection using threshold operation in MRI brain images. In Proceedings of the 2012 IEEE International Conference on Computational Intelligence and Computing Research, Coimbatore, India, 18–20 December 2012; pp. 1–4. [Google Scholar]
  42. Chen, C.-C.; Juan, H.-H.; Tsai, M.-Y.; Lu, H.H.-S. Unsupervised Learning and Pattern Recognition of Biological Data Structures with Density Functional Theory and Machine Learning. Sci. Rep. 2018, 8, 1–11. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  43. Chen, C.-C.; Tsai, M.-Y.; Kao, M.-Z.; Lu, H.H.-S. Medical Image Segmentation with Adjustable Computational Complexity Using Data Density Functionals. Appl. Sci. 2019, 9, 1718. [Google Scholar] [CrossRef] [Green Version]
  44. Menze, B.H.; Jakab, A.; Bauer, S.; Kalpathy-Cramer, J.; Farahani, K.; Kirby, J.; Burren, Y.; Porz, N.; Slotboom, J.; Wiest, R.; et al. The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS). IEEE Trans. Med. Imaging. 2015, 34, 1993–2024. [Google Scholar] [CrossRef] [PubMed]
  45. Bakas, S.; Akbari, H.; Sotiras, A.; Bilello, M.; Rozycki, M.; Kirby, J.S.; Freymann, J.B.; Farahani, K.; Davatzikos, C. Advancing The Cancer Genome Atlas glioma MRI collections with expert segmentation labels and radiomic features. Nat. Sci. Data 2017, 4, 170117. [Google Scholar] [CrossRef] [Green Version]
  46. Huang, S.-J.; Wu, C.-J.; Chen, C.-C. Pattern Recognition of Human Postures Using Data Density Functional Method. Appl. Sci. 2018, 8, 1615. [Google Scholar] [CrossRef] [Green Version]
  47. Amorim, P.; Moraes, T.; Silva, J.; Pedrini, H. 3D Adaptive Histogram Equalization Method for Medical Volumes. In Proceedings of the 13th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, Funchal, Portugal, 27–29 January 2018; Volume 4, pp. 363–370. [Google Scholar]
  48. Kang, D.; Park, S.; Paik, J. SdBAN: Salient Object Detection Using Bilateral Attention Network with Dice Coefficient Loss. IEEE Access 2020, 8, 104357–104370. [Google Scholar] [CrossRef]
  49. Eelbode, T.; Bertels, J.; Berman, M.; Vandermeulen, D.; Maes, F.; Bisschops, R.; Blaschko, M.B. Optimization for Medical Image Segmentation: Theory and Practice When Evaluating With Dice Score or Jaccard Index. IEEE Trans. Med. Imaging 2020, 39, 3679–3690. [Google Scholar] [CrossRef] [PubMed]
  50. Source Codes of D-Unet and Dice Score. Available online: https://github.com/SZUHvern/D-UNet (accessed on 21 October 2020).
  51. Fawcett, T. An Introduction to ROC Analysis. Pattern Recognit. Lett. 2006, 27, 861–874. [Google Scholar] [CrossRef]
  52. Chicco, D.; Jurman, G. The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation. BMC Genom. 2020, 21, 6-1–6-13. [Google Scholar] [CrossRef] [PubMed] [Green Version]
Figure 1. The model estimations of proposed distributions. In the model estimations, the global mean value is ε F = 125 and the value ε 1 , 255 . (a) Difference comparison of the three proposed distributions, and the minus global standard deviation is ε b = 1 . (b) Behavior of the Fermi–Dirac (FD) distribution with different ε b values. (c,d) The same situations but with Bose–Einstein (BE) and Maxwell–Boltzmann (MB) distribution, respectively.
Figure 1. The model estimations of proposed distributions. In the model estimations, the global mean value is ε F = 125 and the value ε 1 , 255 . (a) Difference comparison of the three proposed distributions, and the minus global standard deviation is ε b = 1 . (b) Behavior of the Fermi–Dirac (FD) distribution with different ε b values. (c,d) The same situations but with Bose–Einstein (BE) and Maxwell–Boltzmann (MB) distribution, respectively.
Entropy 23 00223 g001
Figure 2. The framework of the D-Unet-based structure for the brain tumor image segmentation. The original image sizes are 240 pixels, and we rescaled the input sizes to 160 pixels to match the requirements of the adopted neural network models. The third-dimensional value of 160 × 160 × 4 used in the 2D Unet represents four-type images, include T1, T1-weighted contrast-enhanced (T1-CE), T2-weighted, and T2-fluid-attenuated inversion recovery (FLAIR), and so does that used in the 3D feature extraction procedure. The fourth dimension value of 160 × 160 × 4 × 1 in the 3D feature extraction procedure represents the number of trials. The dimension-transform-blocks were used to fuse the two- and three-dimensional features in the encoding procedure, and only fused- and two-dimensional features were used for the information decoding. Thus, these procedures can offer a trade-off between high-dimensional information and computational complexity.
Figure 2. The framework of the D-Unet-based structure for the brain tumor image segmentation. The original image sizes are 240 pixels, and we rescaled the input sizes to 160 pixels to match the requirements of the adopted neural network models. The third-dimensional value of 160 × 160 × 4 used in the 2D Unet represents four-type images, include T1, T1-weighted contrast-enhanced (T1-CE), T2-weighted, and T2-fluid-attenuated inversion recovery (FLAIR), and so does that used in the 3D feature extraction procedure. The fourth dimension value of 160 × 160 × 4 × 1 in the 3D feature extraction procedure represents the number of trials. The dimension-transform-blocks were used to fuse the two- and three-dimensional features in the encoding procedure, and only fused- and two-dimensional features were used for the information decoding. Thus, these procedures can offer a trade-off between high-dimensional information and computational complexity.
Entropy 23 00223 g002
Figure 3. The figure shows the visualized results from the employed preprocessing methods and the proposed FD-type correction functions. Two values of ε b were individually employed to estimate the kernel, and the corresponding results are illustrated in the second and third columns. The proposed FD-type correction functions exhibit the capability not only for image intensity normalization but also for image component filtering. The following columns list the preprocessed results using the other employed methods.
Figure 3. The figure shows the visualized results from the employed preprocessing methods and the proposed FD-type correction functions. Two values of ε b were individually employed to estimate the kernel, and the corresponding results are illustrated in the second and third columns. The proposed FD-type correction functions exhibit the capability not only for image intensity normalization but also for image component filtering. The following columns list the preprocessed results using the other employed methods.
Entropy 23 00223 g003
Figure 4. The visualized comparison between the 3D global histogram equalization function and the proposed FD-type correction functions. (a) The preprocessing results dealing with these functions, and (b) the predicted results of the tumor image segmentation categorized into ET, TC, and WT, and their corresponding ground truths.
Figure 4. The visualized comparison between the 3D global histogram equalization function and the proposed FD-type correction functions. (a) The preprocessing results dealing with these functions, and (b) the predicted results of the tumor image segmentation categorized into ET, TC, and WT, and their corresponding ground truths.
Entropy 23 00223 g004aEntropy 23 00223 g004b
Table 1. Model parameters and relevant functions.
Table 1. Model parameters and relevant functions.
Parameter/FunctionValue/Method
Number of epochs30
Batch size32
Learning rate
(Initial value)
0.00015
Loss function3D soft dice loss function
OptimizerAdam
Table 2. Performance comparison between the conventional preprocessing methods and the proposed FD-type correction functions. This table only lists the dice scores of the whole tumor (WT). Numbers in bold show the best records.
Table 2. Performance comparison between the conventional preprocessing methods and the proposed FD-type correction functions. This table only lists the dice scores of the whole tumor (WT). Numbers in bold show the best records.
Preprocessing MethodDice Score (WT only)Computational Time (min)
Null 0.718391
z-score Normalization0.9296142
3D Global Histogram Equalization0.914884
Gamma Correction 10.931993
FD 1 correction 0.943188
FD 2 correction0.934790
1 The gamma parameter = 0.6.
Table 3. The confusion-matrix table. It shows the comparison of accuracy, sensitivity (recall), and precision between the proposed FD-type correction functions and the conventional correction methods. Numbers in bold show the best records.
Table 3. The confusion-matrix table. It shows the comparison of accuracy, sensitivity (recall), and precision between the proposed FD-type correction functions and the conventional correction methods. Numbers in bold show the best records.
Preprocessing MethodValidation Stage (WT Only)
TPTNFPFNAccuracyRecallPrecision
Null 15,306789,342200358860.99010.710.85
z-score Normalization19,512789,570177616800.99570.900.90
3D Global Histogram Equalization19,293789,400194718990.99520.890.89
Gamma Correction 118,603790,100124825620.99530.860.92
FD 1 correction 19,180790,010133920120.99590.890.91
FD 2 correction19,471789,500184117210.99560.900.89
1 The gamma parameter = 0.6.
Table 4. Further performance comparison between the 3D global histogram equalization function and the proposed FD-type correction functions. The dice scores of the tumor components categorized as WT, tumor core (TC), and enhancing tumor (ET) are individually presented for the comparison. Numbers in bold show the best records.
Table 4. Further performance comparison between the 3D global histogram equalization function and the proposed FD-type correction functions. The dice scores of the tumor components categorized as WT, tumor core (TC), and enhancing tumor (ET) are individually presented for the comparison. Numbers in bold show the best records.
Preprocessing MethodDice ScoreComputational
Time (min)
Training StageValidation Stage
3D Global Histogram EqualizationWT:0.92200.8002138
TC:0.94190.7688
ET:0.91420.6365
FD 1 correctionWT:0.94910.8337140
TC:0.97570.7976
ET:0.95590.6802
FD 2 correctionWT:0.93360.8433141
TC:0.97730.8041
ET:0.96060.6848
Table 5. The confusion-matrix table. Numbers in bold show the best records.
Table 5. The confusion-matrix table. Numbers in bold show the best records.
Preprocessing Method Validation Stage
TPTNFPFNAccuracyRecallPrecision
3D Global Histogram EqualizationWT:18,910789,880146522810.99530.890.89
TC:7818801,309149019180.99580.800.84
ET:3141807,9147007800.99810.800.82
FD 1 correctionWT:18,949789,874147022420.99540.890.93
TC:7627801,691110721100.99600.780.87
ET:3007808,1564599140.99830.770.87
FD 2 correctionWT:19,240789,955134019520.99590.910.93
TC:7966801,84195817720.99660.820.89
ET:3153808,2004157680.99980.800.88
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Tai, Y.-L.; Huang, S.-J.; Chen, C.-C.; Lu, H.H.-S. Computational Complexity Reduction of Neural Networks of Brain Tumor Image Segmentation by Introducing Fermi–Dirac Correction Functions. Entropy 2021, 23, 223. https://doi.org/10.3390/e23020223

AMA Style

Tai Y-L, Huang S-J, Chen C-C, Lu HH-S. Computational Complexity Reduction of Neural Networks of Brain Tumor Image Segmentation by Introducing Fermi–Dirac Correction Functions. Entropy. 2021; 23(2):223. https://doi.org/10.3390/e23020223

Chicago/Turabian Style

Tai, Yen-Ling, Shin-Jhe Huang, Chien-Chang Chen, and Henry Horng-Shing Lu. 2021. "Computational Complexity Reduction of Neural Networks of Brain Tumor Image Segmentation by Introducing Fermi–Dirac Correction Functions" Entropy 23, no. 2: 223. https://doi.org/10.3390/e23020223

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop