StoolNet for Color Classification of Stool Medical Images

Abstract

The color classification of stool medical images is commonly used to diagnose digestive system diseases, so it is important in clinical examination. In order to reduce laboratorians’ heavy burden, advanced digital image processing technologies and deep learning methods are employed for the automatic color classification of stool images in this paper. The region of interest (ROI) is segmented automatically and then classified with a shallow convolutional neural network (CNN) dubbed StoolNet. Thanks to its shallow structure and accurate segmentation, StoolNet can converge quickly. The sufficient experiments confirm the good performance of StoolNet and the impact of the different training sample numbers on StoolNet. The proposed method has several advantages, such as low cost, accurate automatic segmentation, and color classification. Therefore, it can be widely used in artificial intelligence (AI) healthcare.

1. Introduction

The advantages of artificial intelligence (AI) have brought many achievements to the healthcare field in both industry and academia [1]. For example, in recent years, as a research hotspot, convolutional neural networks (CNN) have been used in healthcare to automatically diagnosis diseases [2]. For AI healthcare, clinical examination is critical in diagnosing and reducing diseases. Digestive system diseases are terribly harmful to people’s health. For instance, among the different kinds of cancers, the incidence and mortality of stomach cancer rank fifth and third, respectively, in the world [3]. As an important clinical examination, stool examination can effectively diagnose and prevent digestive system diseases. Actually, stool color can reflect lots of patients’ conditions in the digestive system. Unfortunately, the current stool color examination heavily depends on laboratorians’ sophisticated professional skills.

Some researchers have studied human stools based on biotechnology [4]. However, an accurate automatic analysis system for stool color based on computer technologies is still absent [5]. Currently, many clinical diagnosis technologies are designed based on digital medical images, so it is significant to improve the quality of medical images [6] to accurately diagnose diseases based on medical images [7]. X-ray [8], computed tomography [9], and ultrasonic images [10] can generate many important medical images that can be used for disease diagnosis.

Stool examination has two modes, namely appearance examination and microscopic examination. Color and trait are two important attributes in stool appearance examination. The results of color examination can be used for disease diagnosis, such as dysentery, enteritis, rectal diseases, and tumors. Sharp et al. [11] compared different examination methods of stool microbial detection. Tchiotsop et al. [12] detected intestinal parasites in microscopic stool samples using multi-scale wavelet transform.

Most laboratorians still use manual stool examination. The smell of stool samples is extremely unpleasant, so some laboratorians are not willing to do stool examinations. In addition, in manual examinations, laboratorians need to be close to the samples, which definitely increases the risk of cross-infection. Although stool examination is pretty important, it requires some professional skills. Piekkala et al. [13] invited 52 patients to join their experiments. The patients were asked to test fecal calprotectin at home by themselves monthly and the entire experiments lasted for six months. Of the patients, aged between five and 18 years, 67% completed the study and 197 home tests were performed. Of these, 15% of the tests failed due to technical reasons.

In many fields of healthcare, computer vision technologies have been used to solve different medical problems. Rundo et al. [14] proposed an electrocardiogram (ECG) pipeline using an innovative bio-inspired system implemented by the cellular neural network (CNN). Gao et al. [15] propose a corner detection method based on hierarchical textures for 2D images and the classification results were better than those of the state-of-the-art classification methods based on brain medical images. Chen and You [16] applied traditional classification methods to classify benign and malignant thyroid tumors on ultrasonic images. Opbroek et al. [17] proposed an investigative kernel learning method and a novel image weighting method to reduce the differences between training and testing data. They found that combining image weighting and kernel learning could give a small additional improvement in classification performance on brain tissue.

In recent years, CNNs have achieved remarkable success in many important tasks of computer vision, such as classification [18], detection [19], and recognition [20]. The features of CNN are self-learned, so they are more recognizable and more robust than hand-crafted features. However, the major disadvantage of deep learning is that it requires a large number of training data, so the advantages of CNN were not fully shown until ImageNet [21] was completed. After the dataset came out, CNN gained remarkable precision compared to traditional methods [22,23,24]. Razzak et al. surveyed the overview, challenges, and future of deep learning for medical image processing. They also claim that CNN has been used in the interdisciplinary fields of computer science and healthcare [25]. Lee et al. [26] used CNN to automatically classify the tympanic membrane images. CNN was applied to create a detector that is able to automatically detect the anatomical structure presented in a brain magnetic resonance image [27]. Khawaldeh et al. [28] used CNN to judge whether brain medical images are healthy. Anatomically constrained neural networks (ACNNs) [29] were proposed to incorporate prior knowledge of CNN-based segmentation. This framework is a new regularization model that is trained end-to-end and encourages models to follow the global anatomical properties of the underlying anatomy. Hachuel et al. [30] proposed a deep learning method for human stool recognition and characterization in macroscopic images and they achieved good performance.

Although AI healthcare is a research hotspot, the stool appearance examination based on AI is absent. If AI technologies, including image processing, pattern recognition, deep learning, and so on, can be applied in this field, the capital cost, labor cost, and safety will be greatly improved. Because stool samples contain patients’ private information, it is difficult to collect a large number of samples. Due to the scarcity of a stool classification dataset, we tried to collect a dataset for scientific research with the consent of patients, their families, and their hospitals. The collected samples have high research values that are marked by professional doctors.

In the proposed methodology framework, the images are preprocessed first to localize and segment the region of interest (ROI). After that, the preprocessed images are input into the classifier, an effective shallow CNN named StoolNet. The background was already masked, so StoolNet could learn enough features with a shallow structure. Sufficient experiments confirm the effectiveness of the preprocessing method, the performance of StoolNet, and the impact of different training sample numbers on StoolNet.

The main contributions of this paper are as follows:

• The current stool color examination heavily depends on the medical laboratorians’ professional skills. We first focus on this important factor and propose a lightweight, practical, and efficient automatic color classification method to remarkably alleviate burdens on medical laboratorians.
• The developed preprocessing method can automatically and accurately segment the effective stool region. The designed model, namely StoolNet, can converge well and automatically and accurately classify the stool color.

In this paper, Section 2 introduces the whole framework including the preprocessing method and StoolNet. The experimental results are shown in Section 3. Conclusions and future works are discussed in Section 4.

2. Proposed Method

2.1. Overview of the Proposed Method

The proposed framework is based on digital image processing and CNN, so it not only solves the problem of cross-infection in manual mode but also greatly reduces the heavy burden of manual stool examination. In this paper, neither chemical nor physical processing methods are necessary to deliberately enlarge the stool characteristics. In order to reduce costs, stool samples are classified directly based on the features in original images.

The whole framework consists of two parts, preprocessing and classification. The original images are preprocessed by an automatic segmentation algorithm. The classification mechanism is named StoolNet and is an effective shallow CNN. The experiments show that the accuracy of StoolNet can reach 100% after only a few iterations. And the whole framework is shown in Figure 1.

2.2. Preprocessing Stage

Due to the particularity of stools, sample shapes cannot be fixed. The irregular shapes are very likely to lead to incorrect and inaccurate ROI localization and degrade classification accuracy. An effective and efficient method [31] was developed to segment the stool region from the original images. Six features were tested and represented in grayscale maps, as shown in Figure 2. According to our sufficient investigation and analysis, the discrimination between stool and background is relatively high in saturation maps.

The main purpose of preprocessing is to get the ROI correctly and accurately. Whether the ROI is correct and accurate directly affects the performance of the latter classification algorithm. It is a simple, quick, and efficient way to segment ROI based on a threshold. The threshold segmentation operation is

$$\mathrm{f}\left(\mathrm{x},\mathrm{y}\right)=\{\begin{array}{c}1,\mathrm{g}(\mathrm{x},\mathrm{y})<\mathrm{T}\\ 0,\mathrm{g}(\mathrm{x},\mathrm{y})\ge \mathrm{T}\end{array}$$

(1)

where $\mathrm{f}\left(\mathrm{x},\mathrm{y}\right)$ labels the foreground and background pixels and $\mathrm{g}\left(\mathrm{x},\mathrm{y}\right)$ represents the pixel values in the original image. $\mathrm{T}$ is a threshold value to binarize the image. The pixels with lower, higher, or equal values than $\mathrm{T}$ are classified as background or foreground pixels.

An adaptive algorithm is employed to optimize the threshold value for accurate segmentation. Once the optimal threshold is obtained, the difference between the background and foreground regions are maximized. The goal is to maximize inter-class variance with

$${\mathrm{w}}_0=\frac{{\mathrm{N}}_0}{\mathrm{M}\times \mathrm{N}}$$

(2)

where ${\mathrm{w}}_0$ is the proportion of the number of foreground pixels to the total number of all the pixels in the image. ${\mathrm{N}}_0$ is the number of foreground pixels. The size of the image is $\mathrm{M}\times \mathrm{N}$ according to

$${\mathrm{w}}_1=\frac{{\mathrm{N}}_1}{\mathrm{M}\times \mathrm{N}}$$

(3)

where ${\mathrm{w}}_1$ is the proportion of the number of background pixels to the total number of all the pixels in the image. ${\mathrm{N}}_1$ is the number of background pixels. Also, the size of the image is $\mathrm{M}\times \mathrm{N}$ according to

$${\mathrm{N}}_1+{\mathrm{N}}_0=\mathrm{M}\times \mathrm{N}$$

(4)

where ${\mathrm{N}}_0$ and ${\mathrm{N}}_1$ should satisfy Formula 4, i.e., their sum should be equal to the total number of all the pixels in the image. In

$${\mathrm{w}}_0{+\mathrm{w}}_1=1$$

(5)

where ${\mathrm{w}}_0$ and ${\mathrm{w}}_1$ should satisfy Equation (5), their sum should be equal to 1. In

$${\mathsf{\mu }=\mathrm{w}}_0{\times \mathsf{\mu }}_0{+\mathrm{w}}_1{\times \mathsf{\mu }}_1$$

(6)

where $\mathsf{\mu }$ is the average grayscale of the whole image, ${\mathsf{\mu }}_0$ and ${\mathsf{\mu }}_1$ are the average values of foreground and background pixels, respectively. In

$${\mathrm{g}=\mathrm{w}}_0{\times (\mathsf{\mu }}_0{-\mathsf{\mu })}^2{+\mathrm{w}}_1{\times (\mathsf{\mu }}_1{-\mathsf{\mu })}^2$$

(7)

where $\mathrm{g}$ is the inter-variance between foreground and background regions, it can be given as

$${\mathrm{g}=\mathrm{w}}_0{\times \mathrm{w}}_1{\times (\mathsf{\mu }}_0{-\mathsf{\mu }}_1{)}^2$$

(8)

where Equation (6) is substituted into Equation (7) to obtain Equation (8).

For all possible values of $\mathrm{T}$, we selected the value that maximizes g in Equation (8). The algorithm is performed to extract the ROI candidates from the six color components, as shown in Figure 3. Saturation is the best feature for foreground segmentation among the six color features.

The target is labeled with the foreground region in the binary image, shown in Figure 4.

2.3. StoolNet

There are four different colors for stool medical examination, namely brown, black, yellow, and green. An effective shallow CNN is carefully designed for color classification and is named StoolNet. The structure of the proposed StoolNet is shown in Figure 5.

The input of StoolNet has been preprocessed, so background information is eliminated. The input is resized to $200\times 200$. The sizes of convolutional kernels are $5\times 5$ and the stride is 1. All of the activation functions in StoolNet are ReLU. The ReLU formula is

$$\mathrm{f}\left(\mathrm{x}\right)=\{\begin{array}{c}0, \mathrm{x}\le 0\\ \mathrm{x}, \mathrm{x}>0\end{array}$$

(9)

The dropout method is used in every layer. The loss function is the cross-entropy and the stochastic gradient descent (SGD) is used to update weights. The loss function is defined as

$$\mathrm{L}=-\frac{1}{\mathrm{M}}{\displaystyle \sum }_{\mathrm{k}=1}^{\mathrm{M}}{\displaystyle \sum }_{\mathrm{i}=0}^{\mathrm{N}-1}{\mathrm{y}}_{\mathrm{i},\mathrm{k}}{\mathrm{logp}}_{\mathrm{i},\mathrm{k}}$$

(10)

where $\mathrm{M}$ and N are the numbers of samples and classes, respectively. ${\mathrm{y}}_{\mathrm{i},\mathrm{k}}$ is the true label of the $\mathrm{k}$-th sample if the sample has the $\mathrm{i}$-th label ${\mathrm{y}}_{\mathrm{i},\mathrm{k}}$ = 1, otherwise it is ${\mathrm{y}}_{\mathrm{i},\mathrm{k}}$ =0. ${\mathrm{p}}_{\mathrm{i},\mathrm{k}}$ is the probability of the $\mathrm{i}$-th label of the $\mathrm{k}$-th sample.

A benefit of the preprocessing stage is that the background is masked and, hence, the input of StoolNet has more discrimination than its corresponding original image. Since the number of classes is only four, a shallow structure is enough to achieve a satisfactory classification performance. The sufficient experiments in Section 3 will prove that the accuracy would not always be higher when the depth is deeper. Therefore, a suitable and reasonable depth is a key question for classification tasks. In addition, another advantage of a shallow structure is low computational complexity. Considering the balance between the accuracy and the computational complexity, StoolNet is a good choice for meeting these requirements. Accordingly, this indicates that StoolNet can be applied to several healthcare commercial applications.

3. Experiments

The experimental environment was configured with an Intel Xeon E-2136 CPU @3.30GHz, 16GB internal storage, NVIDIA Quadro P4000 GPU, and a Windows 10 64-bit operating system. The resolution of images in the self-collected dataset was $640\times 480$ pixels. The proposed method and testing were implemented using a Tensorflow deep learning framework [32]. The preprocessing codes were programmed in MATLAB R2016a, Mathworks, US. The network related codes, including the sample random selection for training sets and testing sets, were written in Python. The Python compiler was Spyder. Important to training our network was the learning rate set at 0.001, max iteration set at 120, and batch size set at 32.

Stool color classification is not an open problem and there is no public database. It is very difficult to collect a stool image dataset because of privacy problems. Even though stool color classification is very important for healthcare, few researchers foray into this field because of the particularity of stool samples. We collected a stool images dataset, with the collaboration of a hospital and medical institution, containing 110 images. All sample images were carefully labeled by professional doctors, so we believe the conclusions were drawn from reliable medical results. The images were rotated three times and the total number was 440 images. According to our study, this article is the first paper on stool color classification. Since stool color classification is a particular field, StoolNet cannot be compared with other methods, but sufficient experiments can solidly confirm the effectiveness and efficiency of the proposed method.

The proportion of data used to train the network was set to three ratios, 75%, 50%, and 25% and the rest of the data were used for testing. In order to reduce the effect of the training sample quality, the experiments were repeated five times. In each experiment session, the training and testing samples were randomly selected in a prescribed ratio. Figure 6 shows the accuracy and loss curves in training sets and testing sets. Because the network is an effective shallow CNN and the inputs of the network are preprocessed, StoolNet can converge to get a good classification result in several epochs in three prescribed ratios.

The accuracy of Group 5 in (d) of Figure 6 is fairly constant until 10 epochs and the main reason is that CNN is a probability model. However, there is not a remarkable change in loss until 10 epochs, as shown in Figure 7. CNN is a probability model containing a certain randomness, so it is possible that the weights are not properly adjusted, which results in fairly constant loss and accuracy.

When the training sample proportion is not large enough, the speed of learning features are slower. It can be concluded that the fewer samples used to train the network, the slower the convergence speed will be. In order to confirm the effect of the preprocessing method, the same experiments are performed on the dataset that is not preprocessed. The results are shown in Figure 8. No matter how many samples are used for training, it is difficult to converge and the oscillation is remarkably violent.

Table 1. Numbers of convergence epochs.

	Group 1	Group 2	Group 3	Group 4	Group 5	Average
25% training	48	18	18	26	27	27.4
25% training *	18	26	18	21	16	19.8
50% training	20	38	30	22	24	26.8
50% training *	10	13	11	24	24	16.4
75% training	26	25	13	28	41	26.6
75% training *	8	13	8	7	7	8.6

shows the numbers of convergence epochs on training sets with different training sample proportions and their averages. Experiments with and without preprocessing were both performed. * means the dataset was preprocessed. Our good preprocessing method can greatly accelerate the convergence speed. The numbers of convergence epochs were recorded when the accuracy reached 100%.

StoolNet can achieve remarkable performance in training sets, but different training sample proportions can greatly impact the results in testing sets, as shown in (b), (d), and (f) of Figure 6 and in (b), (d), and (f) of Figure 7.

Table 2. Accuracies in testing sets.

	Group 1	Group 2	Group 3	Group 4	Group 5	Average
25% training	0.6250	0.7500	0.8750	0.6250	0.7500	0.7250
25%training *	0.8125	0.8125	0.7188	0.7500	0.7188	0.7625
50% training	0.9375	0.9375	0.8125	0.8750	0.9375	0.9000
50%training *	0.9375	0.8438	0.8125	0.8750	0.8438	0.8625
75% training	0.8750	1.0000	0.9375	0.8750	0.8125	0.9000
75%training *	0.9688	1.0000	0.9688	0.9375	1.0000	0.9750

intuitively reveals the impact of training sample proportions on the accuracy. * means the dataset was preprocessed. The preprocessing method can remarkably improve the accuracy. Thus the preprocessing method in this paper is necessary to improve the performance of StoolNet.

The average accuracies of five groups with different training sample proportions were calculated and are shown in Figure 9. The network can learn more information and features with more training samples. Thus, the larger the training sample number is, the lower the epoch number is.

Figure 10 reveals the relationship between the network depth and its classification ability. The results show that “deeper is not always better”, so the depth of StoolNet was set to two in terms of accuracy and efficiency.

StoolNet was compared with the method proposed in [30]. The results are shown in Figure 11. Five groups were tested on the preprocessed images. Lightweight StoolNet was able to converge in each group, while the method in [30] could not converge well. This is because the structure of the convolutional layers of StoolNet is simpler than that of the method in [30], so StoolNet has a better converge performance on a small training set.

Actually, most medical equipment for collecting stool images can properly control illumination. However, illumination conditions could be different and could change the color appearance of stool. Since the structure of StoolNet is light, a small- or medium-sized dataset under certain illumination can be conveniently used to train StoolNet. We would like to simulate the effect on different illumination scales. The illumination intensity is enlarged or reduced if the illumination scale is larger or less than 1, as shown in Figure 12.

There are two stages of the proposed method, namely preprocessing and StoolNet. Preprocessing is based on the inter-variance between the foreground and the background and is slightly affected by illumination. The foreground and background pixels are labeled as 1 and 0 in the segmentation mask images. The dissimilarity between the original segmentation mask image, denoted as ${\mathrm{I}}_1$, and another one under different illumination, denoted as ${\mathrm{I}}_2$, is measured with the Hamming distance, denoted as D, and computed as follows:

$$\mathrm{g}\left(\mathrm{x},\mathrm{y}\right)=\{\begin{array}{c}0, {\mathrm{I}}_1\left(\mathrm{x}, \mathrm{y}\right)={\mathrm{I}}_2\left(\mathrm{x},\mathrm{y}\right)\\ 1, {\mathrm{I}}_1\left(\mathrm{x},\mathrm{y}\right)\ne {\mathrm{I}}_2\left(\mathrm{x},\mathrm{y}\right)\end{array}$$

(11)

where $\left(\mathrm{x},\mathrm{y}\right)$ is the coordinate and

$$\mathrm{D}=\frac{{{\displaystyle \sum }}_{\mathrm{x}=1}^{\mathrm{m}}{{\displaystyle \sum }}_{\mathrm{y}=1}^{\mathrm{n}}\mathrm{g}\left(\mathrm{x},\mathrm{y}\right)}{\mathrm{m}\times \mathrm{n}}$$

(12)

where $\mathrm{m}$ and $\mathrm{n}$ are the width and height of the image, respectively.

Table 3. Hamming distances between the original segmentation mask image and the others under different illuminations scales.

Illumination Scale	Yellow	Brown	Black	Green
0.5	0.0043	0.0072	0.0298	0.0072
0.6	0.0035	0.0060	0.0310	0.0058
0.7	0.0033	0.0054	0.0194	0.0052
0.8	0.0026	0.0047	0.0212	0.0045
0.9	0.0029	0.0048	0.0139	0.0047
1.1	0.0026	0.0040	0.0112	0.0040
1.2	0.0025	0.0035	0.0086	0.0040
1.3	0.0039	0.0050	0.0159	0.0055
1.4	0.0067	0.0079	0.0198	0.0080
1.5	0.0131	0.0156	0.0410	0.0189

shows the average Hamming distances between the original segmentation mask image and the others under different illuminations. The dissimilarities are all small, so the preprocessing is slightly affected by illumination change.

The proliferated samples with a certain illumination scale are conveniently used to train StoolNet.

Table 4. Accuracies in different illumination environments (unit %).

Illumination Scale	Group 1	Group 2	Group 3	Group 4	Group 5	Average
0.5	90.63	100.00	93.75	93.75	96.88	95.00
0.6	96.88	96.88	100.00	93.75	93.75	96.25
0.7	100.00	93.75	100.00	93.75	100.00	97.50
0.8	96.88	93.75	96.88	90.63	96.88	95.00
0.9	93.75	96.88	93.75	96.88	94.79	95.21
1.1	94.27	93.75	96.88	96.88	96.88	95.73
1.2	96.88	94.27	95.31	93.75	100.00	96.04
1.3	100.00	100.00	90.63	96.88	96.88	96.88
1.4	93.75	96.88	100.00	93.75	93.75	95.63
1.5	96.88	100.00	100.00	100.00	87.50	96.88

shows that all the trained StoolNets under different illumination conditions have good classification performance, so the StoolNet model is robust in different illumination environments.

4. Conclusions and Future Work

This paper concerns stool color classification, which has not been extensively studied but is a very important field in healthcare. We collected stool images to establish a dataset for appearance detection. The accuracy of the developed classification algorithm was shown to reach 97.5%. Sufficient experiments revealed the relationship between the training sample proportion and accuracy, the relationship between the training sample proportion and the epoch, as well as the relationship between network depth and classification ability. The proposed framework can basically meet the requirements of business and application, but more experiments should be designed to test the robustness of the algorithm in future works. In addition, we will try to continue collecting more stool images to enlarge the dataset scale and further improve our method.