Jawbone Segmentation with Trabecular Bone Preservation from Cone Beam CT Images

Abstract

To segment jaw structures in both open and closed mouth states from cone beam CT images and to preserve the true structures of trabecular bones. Segmentation algorithms of the mandible and maxilla were designed based on their different structures. We detected edges in volume and segment edges of the mandible and maxilla whether the mouth of a patient was open or closed. The internal structures of the mandible and maxilla were preserved by a morphological method with different parameters, respectively. An axial plane corresponding to the bottom of the hard palate was identified so that the bone structures of the maxilla above this plane were removed. Finally, the mandibular surface was smoothed by a simple thresholding method, and the maxillary surface was optimized by a geodesic active contours method. The 3D jaw model was segmented automatically by our proposed procedure with high accuracy. The average dice coefficients of the mandible and maxilla were equal to 0.9709 and 0.9420, respectively. The proposed jaw segmentation algorithm is automatic and knowledge-driven. The segmented jaws truly demonstrate the cortical and trabecular bone structures. It may potentially assist doctors in diagnosis and surgical planning.

1. Introduction

In medical image analysis, a good view of an organ is essential for pathological diagnosis and treatment. In oral medicine, the bone mineral density and structures of a jaw are linked to primary operational success such as tooth implantation [1,2]. They can be assessed through X-ray-based radiographic images, such as cone beam CT (CBCT) images, which have low radiation dose and a high resolution [3]. CBCT images are frequently applied to maxillofacial surgical planning [4]. Analysis of a true 3D model that is based on CBCT data is accurate and reliable and is bound to replace traditional orthodontic planning [5,6,7,8]. Image segmentation helps doctors to better understand and analyze the region of interest (ROI) [9]. An appropriate patient model of the jaw structure is necessary for maxillofacial surgical planning [10]. To generate a 3D jaw model, we propose an automatic algorithm to segment the mandible and maxilla with trabecular bone preservation from CBCT images.

The jaw segmentation from CBCT images is challenging. As CBCT scanners have a low-level radiation dose, CBCT images have a weak signal–noise ratio and are frequently subject to artifacts [10]. The jaw structures are formed by the hard cortical bone and its inner spongy trabecular bone structures [11]. Although the cortical bone can represent the shape of the jaw, the trabecular bone structures can affect surgical success. Therefore, it is imperative that the segmented jaw model contains the true trabecular bone structures. The bone mineral densities of cortical bone and teeth are significantly higher than those of the trabecular bones. In addition, the bone mineral density of the cervical vertebra is also at a high level. Hence, it is hard to segment the jaw structures only by thresholding or region growing methods. The cortical bone of the maxilla consists of thin fibrous bones, and its bone mineral density is lower than that of the mandible [1,12]. As such, the segmentation of the maxilla is more difficult than that of the mandible. It is better to separate them and design different segmentation algorithms that are based on their specific structures. When the mouth is open, the mandible and maxilla are disconnected, and it is easy to separate them. When the mouth is closed, some teeth of the mandible and maxilla touch each other, and it is challenging to accurately separate them.

There are two categories of jaw segmentation methods that are based on 2D images and 3D volume respectively [13,14]. Some papers propose semi-automatic segmentation [15], segmentation that is based on histogram equalization [16,17], or 2D slice-by-slice segmentation methods [9,11]. Most of these methods only segment the cortical bone and the principal trabecular bones, ignoring internal non-bony structures of the jaw. A statistical shape model was proposed for the mandible segmentation [10,18]. This kind of algorithm could generate an accurate mandible with trabecular bones only when the mouth is open, and these algorithms do not work for the maxilla segmentation. In our previous work [14], we segmented the mandible and maxilla with trabecular bone preservation. However, the selection of maxilla ROI was rough, and the algorithm was not adjusted for the different structures of the mandible and maxilla.

The mandible and maxilla have different structures. In our proposed method, we designed slightly different algorithms that were based on the edges, region growing, and morphological operations in 3D volume. Our 3D jaw model illustrates not only the jaw shape but also the internal trabecular bone structures.

2. Materials and Methods

The proposed jaw segmentation algorithm is based on jaw edges. The procedure is summarized in the second row in Figure 1. The first and third rows in Figure 1 show the axial binary images at levels on the volumes that are marked by yellow and blue, respectively, in each step. The segmentation procedure consists of four steps: (I) edge detection, (II) separation of mandible and maxilla, (III) preservation of jaw structures, and (IV) model optimization.

2.1. Step I: Edge Detection

Noise in the CBCT images is a little serious, leading to many incorrect or weak edges that are detected by some edge detectors. Figure 2a shows an axial CBCT image with mandible foramina. Figure 2b shows the enlarged portion of the mandible foramen that is enclosed by the yellow square in Figure 2a. The edges of the foramen are not clear. Figure 2c shows the enlarged portion that is enclosed by the red square in Figure 2a, which illustrates that background noises are serious, and the gray value distribution of background is not uniform. Thus, the Canny edge detector is used to detect edges [19]. This method uses two thresholds to detect strong and weak edges. The strong edges and the weak edges that are connected to strong edges are the detected results. The Canny edge detector is less likely to be fooled by noise than other detectors [20].

The two thresholds that are used by the Canny detector are normalized to [0, 1] in this paper. After many experiments, the higher the threshold of the Canny edge detector is selected in range of 0.15–0.25, and the lower threshold is a classic threshold—0.4 times of the higher threshold. Figure 3a shows the Canny edges of Figure 2a with the higher threshold 0.2. The edges contain not only the mandible, the maxilla, and the spine, but also the trachea, artifacts, and the circular boundaries of CBCT field of view (FOV). The intensity values of the mandible, the maxilla, and the spine are at a high level in original CBCT images but the intensity values of the trachea, artifacts, and the circular boundaries of FOV are at a low level in original CBCT images. Therefore, an appropriate threshold T can remove or reduce these redundant edges. The T can be calculated by an automatic threshold method, such as the Otsu’s method [21]. There are four steps to obtain edges as follows:

• Detect edges in each axial image by the Canny edge detector and generate a volume that is composed of the edges;
• Dilate the volume by a spherical structuring element with 3 pixels radius in three-dimensions to ensure that the edges between each axial image are connected in three-dimensions;
• Calculate a threshold T by the Otsu’s method that is based on pixel values of whole original CBCT images so that the between-class variance achieves maximum;
• Remove voxels of the dilated volume whose intensity values of the original CBCT images are less than the threshold T.

It is important to emphasize that pixels that are outside the FOV are removed before calculating T. Figure 3b indicates the result of Figure 2a that was processed by this step.

2.2. Step II: Separation of Mandible and Maxilla

The maxilla is more difficult to segment than the mandible. The mandibular surface is formed with high bone mineral density as shown in Figure 4c. Its edges are more obvious and are easier to detect. However, the edges of the maxillary alveolar bones are not as clear as those of the mandible. Figure 4b shows the maxillary alveolar bones around the maxillary molar. The edges are too blurred to distinguish between the jaw and background. Due to the different structures of the mandible and maxilla, we propose different algorithms to segment them.

In this step, we propose to segment the mandible and maxilla that resulted from the last step. When the mouth is open, the mandible and maxilla are not connected in space and can be easily separated by a region growing method. When the mouth is closed, the mandible and maxilla are connected in space, and it is difficult to separate them accurately. Whether the mouth is open or closed, the mandible and maxilla that resulted from Step I can be segmented as follows.

The volume the result from Step I is projected to a sagittal image as shown in Figure 5a. The red point is the closest pixel to the left boundary of the sagittal image. The axial images are 2 cm below and above the red point that is selected to find the initial seeds of mandible and maxilla, respectively, as the blue and green lines that are marked in Figure 5a, and the corresponding axial images are as shown in Figure 5b,c. The pixels that are closest to the top boundary of these two axial images are selected as the initial seeds of mandible and maxilla, which are marked as the blue and green points in Figure 5b,c, and their sagittal projections are marked as blue and green points in Figure 5a. Once the initial seeds are selected, the mandible and maxilla are segmented by a 3D region growing method that is based on 26-adjacency (in a 3 × 3 × 3 cube) [22]. If the mandible and maxilla are not intersecting, the mouth is considered as open and the segmented results of mandible and maxilla by the region growing method are used in next Step.

If the mandible and maxilla are intersecting, according to the principle of the region growing method, the grown results are the same and contain both the mandible and maxilla as shown in Figure 6a. It is difficult to separate the mandible and maxilla accurately in this case. We proposed to find an axial image that is marked in Figure 6a to roughly separate the mandible and maxilla. This axial image should contain parts of both the mandible and maxilla, and the maxilla should not connect with the mandible. All voxels are then removed, as shown below this axial image in Figure 6b. The remainder voxels consist of the maxilla and the mandibular ramus. The maxilla is not connected with the mandibular ramus. As Figure 6c shows, the maxilla can be segmented by a 3D region growing method using the same initial seeds of the maxilla as shown in Figure 5c. The mandible is generated by the merged mandible and maxilla minus the maxilla, as shown in Figure 6d.

This axial image localization is critical for the segmentation of the mandible and the maxilla when the mouth is closed. The axial image detection algorithm is based on a slice-by-slice region growing method. In each slice, the axial image is segmented by region growing method that isbased on 8-adjacency. When the i-th axial image of the volume has been segmented, all the segmented pixels and their 8-adjacency pixels in the (i + 1)-th axial image which satisfy the region growing rule are set as the initial seeds of the (i + 1)-th axial image.

The axial images of the initial seeds of the mandible and maxilla in the 3D region growing are set as the first and last slice. The voxels above the first slice and below the last slice are removed as shown in Figure 7b. The volume, as shown in Figure 7c,d, is generated by a slice-by-slice region growing method. For Figure 7c, the initial seeds of the first slice are all the pixels of the first slice. For Figure 7d, the initial seed is same as that of maxilla in 3D region growing as shown in Figure 5c. As shown in Figure 7d, a part of the mandibular ramus is not segmented because the maxilla in these axial images is not connected with the mandible. The volume in Figure 7e is generated by the volume in Figure 7c minus the volume in Figure 7d. We propose to evenly divide the volume of Figure 7e into left and right volumes, then find the bottom axial images of these two volumes respectively, and finally set the upper of these two bottom axial images as the axial image that separates the mandible and maxilla, as shown in Figure 6a marked by the yellow line.

In this step, whether the mouth is open or closed, the algorithm can separate the mandible and maxilla of the volume that resulted from Step I.

2.3. Step III: Preservation of Jaw Structures

The mandible and maxilla that resulted from the last step contain no details of the internal structures, as shown in the axial images of Step II in Figure 1. This step will present our preservation algorithm of the internal structures of the jaw. The preservation algorithms for the mandible and maxilla structures are different because of their respective structural differences and are introduced respectively.

2.3.1. Preservation of Mandibular Structures

In Step I, the mandible is dilated. The borders are enclosed for almost all the axial images. The lost internal structures can be thought as holes in the axial images. To preserve them, the hole-filling morphological operation is used [23]. Here, the hole-filling operation is applied in each axial image of the mandible. For a binary image, a hole is the set of zero-valued pixels and not connected to the image boundary. If the borders of the mandible are not closed, the missing internal structures cannot be filled by the hole-filling operation. To further ensure that the borders of the mandible in each axial image are closed, the mandible that resulted from Step II is dilated first by a spherical structuring element with a 3-pixel radius. The hole-filling operation is performed and then followed by an erosion operation with the same structuring element.

2.3.2. Preservation of Maxillary Structures

As shown in Figure 8a, the edges of the maxilla that were obtained by Step II are not connected. The reason is that some edges of the maxilla are weak as shown in Figure 4b and the Canny edge detector suppresses them. The edges of the maxilla need to be enhanced. On each axial image of the maxilla that resulted from Step II, the following procedure is executed:

• Set the GH as strong edges from Step II as shown in Figure 8a. The red square is the minimum enclosing rectangle of GH, and its width and height are C and R;
• Expand this region to the yellow square enclosing the red square and all the edges that are detected by Canny edge detector. If the yellow square is beyond the image boundary, the yellow square is shrunk to the image boundary;
• Detect the weak edges GL by the Canny detector with low level thresholds as shown in Figure 8b. The higher threshold of the Canny detector can be selected in the range of 0.03–0.05, and the lower threshold is 0.4 times of the higher threshold. In Figure 8b, the higher threshold is 0.04;
• Mark all the weak edges in GL that are connected to GH based on 8-connectivity as valid edges as shown in Figure 8c.

As shown in Figure 8c, the internal structures are missing. The edges are not as closed as the mandible and the improved edges may have false edges in the axial images. If the hole-filling operation is processed on the axial images, some false structures may be preserved. The internal structures of maxilla can be preserved well if the hole-filling operation is processed on the sagittal images. First, to enclose all the holes of the maxilla in the sagittal images except for those on the top axial image, set all the pixels in the top axial image of the maxilla to 1. Second, dilate the maxilla by a spherical structuring element with a 3-pixel radius. Third, the hole-filling operation is applied on each sagittal image. Finally, the preserved volume is eroded by a spherical structuring element with a 3-pixel radius.

After edge enhancement, some false positive edges are added as shown in Figure 8c. Since the edges are detected on each axial image, most of these false positive edges are discontinuous between two adjacent axial images. After the hole-filling operation, these false positive edges are only thin lines in three-dimensions, and, unlike the maxillary structure, is a connected region. Therefore, these false positive edges are removed by morphological open operations with 1 × 1 × 5, 1 × 5 × 1 and 5 × 1 × 1 line structuring elements respectively.

2.4. Step IV: Model Improvement

After the last step, the mandible and the maxilla are roughly segmented. In Step I and III, the dilation operations are used several times. Although the dilation result is filtered in Step I and the opposite erosion operations are applied with the same structuring element in Step III, false positive voxels are still added to the surface of the segmented jaw and they should be removed or reduced. Due to the different structures of the mandible and maxilla, the jaw model improvement algorithms are different and introduced respectively.

2.4.1. Model Improvement of Mandible

As the intensity values of the mandibular surface are high, the surface voxels are calibrated by a thresholding method as follows:

• Set I1 as the segmented mandible from Step III;
• Erode I1 by a spherical structuring element with a 3 pixels radius, and obtain I2 and calculate the mandibular surface I3 = I1 − I2;
• Remove the voxels in I3 with the corresponding intensity values in the original CBCT images that are lower than T; T is the threshold that was computed in Step I. Let I1 = I2 + I3;
• To smooth the volume, the morphological opening operation is applied on I1 by a spherical structuring element with a 3-pixel radius, and then I1 is the improved mandible.

2.4.2. Model Improvement of Maxilla

Our ROI of the maxilla is immediately below the hard palate. As shown in Figure 9, the red point is the top pixel of the maxilla in an axial image, the green point is the bottom pixel of the maxilla vertically below the red point, and H is the distance between these two points. We calculate H in each axial image of the maxilla that was yielded from Step III from the bottom slice to the top slice. If H is larger than 2cm, this slice and its above slices are removed.

As the intensity values of the maxillary surface are not all at a high level as those of the mandible, the improvement method of the mandibular surface is not applicable to the maxillary surface. The geodesic active contours (GAC) method [24] is used to improve the maxillary contour in each axial image. The original images of the GAC method are the original CBCT images, the images that were processed after Step III are the initial contours of the evolution. The maximum number of iterations in the evolution is 20. To smooth the volume, the morphological opening operation by a spherical structuring element with a 3-pixel radius is performed after GAC processing.

3. Results

To evaluate the performance of the algorithm, six groups of CBCT datasets were randomly collected and manually segmented by two experts. The CBCT datasets were acquired using the MEYER SS_X9010DPro-3DE CBCT system (HEFEI MEYER OPTOELECTRONIC TECHNOLOGY INC., China), whose voxel size is 0.2658 × 0.2658 × 0.2515 mm [3] and FOV is 127 mm × 127 mm × 80 mm. The proposed method was implemented using MATLAB codes on a Lenovo desktop (Win 10, Intel Core i7-9700K 3.60 GHz CPU, 16 GB RAM). The precision, recall, dice coefficient, Hausdorff distance (HD), and the root mean square symmetric surface distance (RMSD) were used to verify the accuracy of the proposed algorithm.

Table 1. The evaluation metrics of the mandible and maxilla that were segmented by the proposed algorithm (values indicate the mean ± standard deviation).

	Mandible	Maxilla
Precision	0.9709 ± 0.0093	0.9249 ± 0.0124
Recall	0.9710 ± 0.0081	0.9599 ± 0.0125
Dice Coefficient	0.9709 ± 0.0026	0.9420 ± 0.0065
HD/mm	5.2174 ± 1.3466	7.3647 ± 4.3813
RMSD/mm	0.3887 ± 0.1011	0.6489 ± 0.1275

presents the evaluation metrics of the mandible and maxilla that were segmented by the proposed algorithm. These mean and standard deviation values illustrate that the segmented results have a high level of accuracy and the accuracy is stable. As mentioned in the Introduction section, the structure of the maxilla is more complex than that of the mandible, and the accuracy of the maxilla is lower than the mandible.

Figure 10 and Figure 11 show the segmentation results of the proposed algorithm, the corresponding ground truth, and the superimposing volume of the mandible and maxilla, respectively. The volumes are visualized by the 3D plugin of ImageJ software. In the superimposing volumes, the blue voxels are the background that were wrongly classified into the jaw (false positives), and red voxels are the jaw wrongly that were classified into the background (false negatives). When the mouth is closed, the algorithm can also segment the mandible and maxilla as shown in Figure 12.

4. Discussion

The proposed algorithm can be mainly in three aspects: (1) The algorithm segments both the mandible and maxilla; (2) The segmented jaw contains not only the cortical bone but also the trabecular bone structures; and (3) The algorithm works well whether the mouth is in an open or closed state. The previous approaches did not address all these aspects. The 2D image-based methods [9,11,15,16,17] did not segment the whole trabecular bone structures. The 3D volume-based methods [10,18] did not segment the maxilla and could not work when the mouth was in a closed state.

The trabecular bone structures are too complicated to segment manually. Therefore, an approximate method was designed to quantify the accuracy of the trabecular bone structure. First, erode the ground truth volume twice by the spherical structuring element with a three pixel radius and obtain the trabecular ground truth (TGT). Second, remove voxels of the TGT whose intensity values in the original CBCT images are larger than a threshold. Third, calculate the accuracy of the trabecular bone as defined as follow.

$$Accuracy=\frac{TGT \cap Segmented Jaw}{TGT}$$

The average accuracy values of the mandible and maxilla are 0.9941 ± 0.0053 and 0.9982 ± 0.0044, where the threshold is same as T, as calculated in Step I. These data illustrate that the trabecular bone structures are preserved accurately.

After jaw segmentation, the structures of the trabecular bones can be observed by thresholding. Figure 13 shows some sections of the segmented mandible thresholding by T that was calculated in Step I. Figure 13e–h are the enlarged red square portions of Figure 13a–d.

To clinically use CBCT for dental diagnosis and treatment planning, the potential applications of the proposed algorithm includes the following: (1) The proposed algorithm can automatically segment the structures of the mandible and maxilla, which currently has to be done manually by experts (taking around 12 h); (2) The proposed algorithm is convenient for dentists to observe the 3D structures of the mandible and maxilla alone without muscle, cervical vertebra, and the airways. For example, if a patient needs implantation in the mandible, the dentist can only observe the 3D mandible by the proposed algorithm, rather than all the tissues from the CBCT images; and (3) The proposed algorithm can provide a 3D model of the mandible and maxilla which can be transferred to an STL file for 3D printing.

5. Conclusions

Nowadays, more and more people are concerned about dental health and aesthetics. Due to its low radiation dose, CBCT has become popular and is widely used in dental diagnosis and surgical planning. For dental implantation, orthodontic and other surgical procedures, the jaw is the major ROI. According to the evaluation metrics, our proposed jaw segmentation algorithm performs well. The proposed algorithm can automatically and precisely segment the jaw structure whether the mouth is open or closed. The results can provide a good visual of the jaw structure and help with dental diagnosis and preoperative planning. The proposed algorithm potentially helps dentists diagnose and perform surgical planning.