Automated Defect Analysis System for Industrial Computerized Tomography Images of Solid Rocket Motor Grains Based on YOLO-V4 Model

Abstract

As industrial computerized tomography (ICT) is widely used in the non-destructive testing of a solid rocket motor (SRM), the problem of how to automatically discriminate defect types and measure defect sizes with high accuracy in ICT images of SRM grains needs to be urgently solved. To address the problems of low manual recognition efficiency and data utilization in the ICT image analysis of SRM grains, we proposed an automated defect analysis (ADA) system for ICT images of SRM grains based on the YOLO-V4 model. Using the region proposal of the YOLO-V4 model, a region growing algorithm with automatic selection of seed points was proposed to segment the defect areas of the ICT images of grains. Defect sizes were automatically measured based on the automatic determination of defect types by the YOLO-V4 model. In this paper, the image recognition performance of YOLO-V4, YOLO-V3, and Faster R-CNN models were compared. The results show that the average accuracy (mAP) of the YOLO-V4 model is more than 15% higher than that of the YOLO-V3 and Faster R-CNN models, the F1-score is 0.970, and the detection time per image is 0.152 s. The ADA system can measure defect sizes with an error of less than 10%. Tests show that the system proposed in this paper can automatically analyze the defects in ICT images of SRM grains and has certain application value.

1. Introduction

In the life cycle of an SRM, inner bore cracking, internal slag inclusion, and interlayer debonding are the main defects affecting the structural integrity of grains. In order to study the effects and causes of defects, determining the types and sizes of defects in the grains is important. Industrial computerized tomography (ICT) is widely used in the non-destructive testing of SRMs. The manual-based interpretation of defective features in the ICT images of grains may result in reduced reliability due to fatigue [1]. Automatically recognizing the types and sizes of defects in the ICT images of grains is the key to fast and highly reliable quality assessments.

Identifying defects in the ICT images of grains is a problem for image target detection. The current mainstream target detection networks can be divided into two categories: the two-stage target detection network and one-stage target detection network [2]. Using the former, a series of candidate regions is first extracted and then classified. Region-convolutional neural network (R-CNN) algorithms, such as Fast R-CNN [3], Faster R-CNN [4], and Mask R-CNN [5], are the most representative. Using the latter, the target detection problem is regarded as a regression problem with a convolutional neural network (CNN) as a regressor and the whole image to be detected as a candidate region. The CNN is directly input to regress the location information of the target in the image to be detected. The YOLO [6] and SSD [7] are the most representative algorithms. The detection speed of the one-stage target detection network is much higher than that of the two-stage detection network. In [8], Bochkovskiy et al. proposed the YOLO-V4 model. So far, scholars have made many efforts to improve the YOLO-V4 model. Wu et al. [9] proposed to reduce the training cost of the YOLO-V4 model by using a channel pruning algorithm. Dewi et al. [10] used different generative adversarial networks (GAN) to generate more image data for the training of the YOLO-V4 model and achieved a higher quality recognition accuracy. Chie et al. [11] proposed a network scaling method, which can modify the depth, width, resolution, and structure of the network model. These improvement measures reduce the training cost of the YOLO-V4 model, which makes the application of the YOLO-V4 model in grain ICT image datasets feasible.

Due to the high costs of the ICT images of grains, deep learning-based methods have not been maturely applied in the existing studies. In the main method, different defect types are classified and identified by different image region features, which are subject to greater human interference and inefficiency [12,13,14]. For target detection networks, it is relatively easy to determine the target type; however, measuring the target size requires segmenting the target area. Using the Mask R-CNN, the segmentation requirement can be achieved. However, its detection time is long due to the two-stage network structure. In addition, the training cost of the Mask R-CNN is high, which is not suitable for the ICT images of grains [15,16].

In order to segment defect areas in the framework of the one-stage target detection network, we developed an automated defect analysis (ADA) system based on the YOLO framework to determine the types and measure the size of the defects in ICT images of grains. With the addition of post-processing operations in ICT images of grains, defect areas can be automatically segmented, and defect sizes can be measured. We compared the detection effect of the YOLO-V4 model with that of the YOLO-V3 and Faster R-CNN models, and the measured dimensional errors were counted to prove the effectiveness of the system.

Section 2 introduces the ICT image dataset of the grains. Section 3 presents the proposed ADA system. The experimental results and discussion are present in Section 4. The paper ends with the conclusions in Section 5.

2. Materials

In this study, the images of grains with defects obtained by ICT were tested. The defect areas in images were labeled with rectangular boxes, and XML documents were generated. We used the YOLO-V4 model to train the dataset and compared its precision (P), recall rate (R), average accuracy (mAP), and other evaluation indicators with those of the YOLO-V3 and Faster R-CNN models.

2.1. Defect Types of Grains

Stress concentration is the main reason for defects in grains. When defects occur, the stress concentration tends to expand the size and number of defects of the same type rather than generate different types of defects. Therefore, most defects are of a single defect type in ICT images of grains. Among the defect types of an SRM, internal slag inclusion occurs at the production stage, while inner bore cracking and interlayer debonding tend to occur during cooling, storage, and transportation [17]. The defect classification image is shown in Figure 1, with the three types marked as Slag, Crack, and Debonding, respectively. LabelMe was used as the image labeling software. The annotation file was saved in XML format.

2.2. Image Augmentation

High-quality images can reduce the loss of the target areas. The ICT images of grains contain much noise. In order to reflect the characteristics of defects and prepare images for annotation, we adopted a bilateral filter to reduce the noise in the ICT images and maintain the edge information of the defects in the images [18]. Figure 2 shows image augmentation in the pre-processing. The template of the bilateral filter can be expressed as follows:

$$w\left(i,j,k,l\right)=exp\left(-\frac{{\left(i-k\right)}^2+{\left(j-l\right)}^2}{2{\sigma }_d^2}-\frac{{\Vert I\left(i,j\right)-I\left(k,l\right)\Vert }^2}{2{\sigma }_r^2}\right)$$

(1)

where i, j, k, and l are the spatial positions of pixels. σ_d and σ_r are the space standard deviation and range template standard deviation, respectively. I is the image pixel value. In the bilateral filtering algorithm used in this paper, σ_d = 30, σ_r = 70, and the filtering radius, r = 2.

The effect of the image augmentation is shown in Figure 3. Figure 3 shows that the image is smoother, and the gray distribution is more concentrated after augmentation, making the defect features more prominent.

2.3. Dataset

According to the ICT test standard of an SRM, there are unified requirements for the sizes of images generated after ICT scanning of an SRM, which are 512 × 512, 1024 × 1024, and 4096 × 4096, respectively. With the increase in image size, the computational effort of YOLO-V4 increases, and the training cost of the model increases accordingly. Therefore, the size of 512 × 512 was selected in this paper.

In order to achieve better model training effects, we collected 600 ICT images of SRM propellants with defects. There are 200 images of Crack, Slag, and Debonding, respectively. The numbers of images in the training and test datasets are shown in

Table 1. Numbers of images in the training and test datasets.

Defect Types	Training Dataset	Test Dataset
Crack	150	50
Slag	150	50
Debonding	150	50
Total	450	150

.

3. Methodologies

The ADA system was proposed to identify the types and measure the sizes of the defects in the ICT images of grains. Figure 4 shows the structure of the ADA system. The system mainly consists of the pre-processor for datasets, the YOLO-V4 framework for defect recognition, and the post-processor based on region growing. First, the images of grains were collected by ICT to form the dataset of the ADA system, and then these data were annotated manually. Next, the whole dataset was divided into the training dataset used to train the YOLO-V4 model and the test dataset used to evaluate the performance of the trained model. Finally, we used the region-growing method with automatic selection of seed points to segment the defect areas and measure the defect sizes with the addition of post-processor operation based on the gray distribution characteristics of the calibrated defect area.

3.1. YOLO-V4 Framework

In a YOLO model, a target detection problem is treated as a regression problem. The image to be detected is used as a candidate region, and the image is fed into the CNN to generate the boundary coordinates and probabilities for each class. The YOLO model mainly consists of the Backbone, Neck, and Dense Prediction. Among them, the Backbone is composed of the CNN, which completes the task of extracting feature information from images. The Neck fuses the feature maps generated by different layers in the CNN so that the deeper image information of the image can be discovered. Dense Prediction detects the targets in the feature maps and generates the prediction results of the model. The structure of the YOLO model is shown in Figure 5.

The YOLO-V4 model is an optimized YOLO model. In the YOLO-V4 model, the Backbone-DarkNet53 network in the YOLO-V3 model is optimized as the Cross-Stage-Partial-connections (CSP) DarkNet53 network. The feature mapping of the base layer is first divided into two parts by CSP, and then they are merged by a cross-stage hierarchy. Therefore, the learning ability of the CNN can be improved, making the YOLO-V4 model lightweight while maintaining accuracy. Moreover, the computational cost can be effectively reduced. The CSP Darknet53 network has a higher detection accuracy than the CSP ResNeXt-50 [19]. In the YOLO-V4 model, the activation function of the DarknetConv2D is optimized from LeakyReLU to Mish to improve the accuracy of model classification. Mish is an activation function similar to ReLU and Swish, but it performs better in different datasets [20]. Its formula is shown below:

$$y=x\times \tanh \left(ln\left(1+e^x\right)\right)$$

(2)

where x is the independent variable; and y is the dependent variable.

In the YOLO-V4 model, the Backbone is the CSP Darknet53 network, the Neck consists of SPP and PAN, and the Head is still the head of the YOLO-V3 model. Its structure is shown in Figure 6.

3.2. Post-Processor

The trained YOLO-V4 model can effectively calibrate defect types and ranges. However, determining defect types alone cannot meet the actual demand for the defect analysis of the ICT images of grains. Measuring defect sizes is equally important for the safety evaluation of an SRM. Therefore, we measured defect sizes by adding the post-processing operation based on the region proposal.

The defect parts in the grains need to be segmented to measure the defect sizes. The gradient magnitude of an image can reflect the clarity of the edge [21]. It can be calculated as follows:

$$G=\sqrt{G_x^2+G_y^2}$$

(3)

where Gx and Gy are the gradients in the x and y directions of the image, respectively. G is the gradient amplitude of the image. The gradient amplitudes of the ICT images of the grains with different defects were calculated after image augmentation. The results are shown in Figure 7. The defects in the ICT images of grains have clear edges and are suitable for image segmentation using the region growing method. The basic idea of the region growing algorithm is to select a seed point in the target area and then merge the seed point with the pixel points in its neighborhood that can meet the growth conditions. The merged points are used as the new seed points to grow outward until they reach the boundary conditions [22]. It can be seen that the key to the region growing method is the selection of seed points, growth conditions, and boundary conditions [23].

The automatic selection of seed points has always been a key issue in applying the restricted region growing method in engineering. In this paper, we analyzed the gray values in the rectangular boxes of the calibrated target areas based on the defect areas detected by the YOLO-V4 model. Figure 8 shows the variations in the image grayscales along the diagonals in the rectangular boxes. According to different defect types, the maximum gray value can be selected as the seed point in the defect distribution regions of Debonding and Crack, and the minimum gray value can be selected as the seed point in the defect distribution region of Slag.

Region growing can be performed according to appropriate growth criteria when the seed point and boundary conditions are determined. Figure 8 shows that the grayscale difference in pixels in each defect area is very small. Therefore, the grayscale difference less than a certain threshold should be selected as the region growth criterion of defect parts in the ICT images of grains. The specific growth criterion is as follows: in the next four neighborhood pixels to be analyzed, the point with the smallest absolute difference from the average gray value in the segmented region is selected. I_i is the gray value of the four pixels in the neighborhood, where i = 1, 2, 3, 4. µ is the mean gray value of the segmented region. For the pixel to be analyzed, first determine whether it is within the boundary. After the conditions within the boundary are satisfied, the absolute difference between I_i and μ is calculated. If the absolute difference is minimized, the pixel is merged with the segmented region. If not, the pixel is removed. The specific algorithm flow is shown in Figure 9.

Since most cracks are connected to the inner holes of the grains, the boundaries of the cracks are not closed, making it difficult to segment the crack defect area using the region growth method. The size of an inner hole of a grain changes with the change in the location of ICT detection. The diameter of the inner hole should be obtained to complement the inner hole boundary with a portion of that circle to close the crack defect area. In the ICT image of the grain of an SRM, the center of the inner hole of the grain is coincident with the center of the image. Referring to the method for automatically selecting seed points, we proposed an algorithm for automatically measuring inner hole diameters, as shown in Algorithm 1.

Algorithm 1. Algorithm for automatically measuring inner hole diameters

Begin:

(1) Record the grayscale change curves of the images in the four directions of A, B, C, and D, as shown in Figure 10;

(2) Take the horizontal coordinates of the points in the image grayscale change curves where the grayscales change at the beginning, as the measurements of the inner hole radii, noted as rA, rB, rC, and rD;

(3) Select the significant level α = 0.05, the number of measurements n = 4, and the critical value D (α, n) = 0.829 to reject the outliers among ri (i = 1, 2, 3, 4) according to the Dixon criterion [24];

(4) Calculate the average of the remaining inner hole radius measurements as the estimate of the inner hole radius, rm.

If the crack direction coincides with one of the directions A, B, C, or D, the measurement of the inner hole radius will be larger than the real value. Therefore, the Dixon criterion is used to eliminate the abnormal value by calculating the Q-value, which is shown as:

$$Q_n=\frac{x_n-x_{n-1}}{x_n-x_1}$$

(4)

where Q_n is the Q value of the nth data; x_n is the dataset that arranges the data from smallest to largest; and n is the serial number of the data value. The confidence level α and sample size n were chosen as 0.05 and 4. The calculated Q_n is compared with the critical value in the Dixon coefficient table. If Q_n > Q(0.05, 4), the value is considered to be abnormal and should be excluded. The Dixon coefficient table is shown in

Table 2. The Dixon coefficient.

n	α = 0.05	α = 0.01
3	0.970	0.994
4	0.829	0.926

.

If the ICT image of a grain contains a crack defect, an inner hole circle with a radius of r_m is added to the image to fill the crack boundary, and then the crack area is segmented using the region growing algorithm.

The YOLO-V4 model is used to calibrate the defect area, and the region growing algorithm is used to segment the image. The segmented defects are calibrated with the smallest rectangular boxes. The size of each defect is obtained by multiplying the pixel points occupied by the length and width of the smallest rectangular box with the actual scale, respectively. The measurement results are shown in Figure 11.

4. Experimental Results

In order to verify the effectiveness of the ADA system proposed in this paper, we analyzed the defect detection performance. The experiments focused on two aspects: (i) the detection performance of the trained YOLO-V4 model; and (ii) the accuracy of the defect size measurements in post-processing operations. This section presents the results of the experiments with real datasets.

The processor, Intel Core i7-11800H, with the main frequency of 2.30 GHz, the memory of 16 GB, and the GPU of NVIDIA RTX 3060 was used to train the detection model. The algorithm in this paper was developed using Microsoft Visual Studio 2015 and Python 3.6.

4.1. Results of the YOLO-V4 Model

The YOLO model was trained by loss functions, such as bounding box regression loss (L_IoU), confidence loss (L_confidence), and classification loss (L_class) [19]. Figure 12 shows the training loss images of the YOLO-V4 model in this paper, indicating that the dataset established in this paper can support the training of the model due to the obvious characteristics of the three typical defects in the ICT images of the SRM grains. The defect detection results of the ICT images of the grains based on the YOLO-V4 model are shown in Figure 13.

4.2. Comparison of Target Detection Algorithms

In a target detection algorithm, the F1-score and mAP are two widely used metrics. The F1-score is proposed based on precision and recall. It is the summed average of precision and recall and can reflect the performance of the target detection algorithm more objectively [25]. Its calculation formula is shown below:

$$F_1=\frac{2\times P\times R}{P+R}$$

(5)

where F1 is the F1-score; P is the precision; and R is the recall rate.

The average accuracy for a particular category is represented by AP, and mAP is the average value of AP. The value of mAP reflects the detection ability of the training model in all categories [26]. Therefore, the F1-score and mAP are used as the indices to measure the performance of the target detection algorithm in this paper. The performance parameters of three different target detection algorithms, Faster R-CNN, YOLO-V3, and YOLO-V4, trained on the same training dataset and tested on the same test dataset are shown in Table 2.

Table 3. Performance parameters of three target detection algorithms.

Model	mAP (%)	F1-Score	T (s)
Faster R-CNN	0.490	0.976	0.352
YOLO-V3	0.497	0.993	0.146
YOLO-V4	0.572	0.970	0.152

shows that the YOLO-V3 algorithm has the highest F1-score value of 0.993. The differences in the F1-score values of the YOLO-V4, Faster R-CNN, and YOLO-V3 algorithms are not significant. The mAP value of the YOLO-V4 algorithm is more than 15% higher than that of the YOLO-V3 and Faster R-CNN algorithms. The Faster R-CNN algorithm has the longest detection time, while the detection times of the YOLO-V3 and YOLO-V4 algorithms do not differ much. Therefore, the YOLO-V4 algorithm has superior performance in detecting defects in the CT images of SRM grains.

4.3. Accuracy of Defect Size Measurement

Through the post-processing operation in the ADA system, the sizes of the defects in the grain ICT images can be obtained. A certain error exists in the measured defect sizes due to the loss caused during the region growing and the smallest rectangular calibration. The actual defect size is obtained by manual measurement. The manual measurement of defect size varies depending on the type of defect. The actual defect size is its maximum radial dimension for slag and debonding defects. For crack defects, the actual defect size is the length of the line connecting the beginning and end of the crack. The actual defect size should be compared with the long side size of the smallest rectangular box used in the automatic measurement. The accuracy of the algorithm is measured as the percentage error, and the percentage error was calculated by the following formula:

$$E=\left|\frac{r_m-r_t}{r_t}\right|$$

(6)

where E is the percentage error; r_m is the value of the measured defect size; and r_t is the value of the actual defect size. The sizes of the defects in the 450 images of the datasets in this paper were measured. Figure 14 shows the percentage error distribution, indicating a percentage error of the defect sizes measured by the ADA system of below 10%.

5. Conclusions

In this paper, we proposed an ADA system to detect defects and measure defect sizes in the ICT images of SRM grains. The system identifies defects based on the YOLO-V4 model. The defect sizes are automatically measured through the region growth algorithm with the automatic selection of seed points. The comparison with the YOLO-V3 and Faster R-CNN models proves that the YOLO-V4 model can well meet the requirements of defect recognition in the ICT images of grains. The error of the proposed method was found to be less than 10% by calculating the percentage error of size measurement. This system provides a new research idea for image segmentation in the framework of the one-stage target detection network. It can effectively solve the problem of requiring manual measurement of defect sizes in engineering. In addition, it can improve the efficiency of the ICT image analysis of SRM grains.

In future research, the inefficient training and detection of the YOLO-V4 model for large-sized images should be solved first. In particular, reducing training costs is important for defect detection in ICT images of solid rocket motor grains. Secondly, the method used in this paper to achieve the automatic measurement of defect size is relatively complicated. In addition, there is still room for improvement in measurement accuracy. Achieving the automatic measurement of defect size with higher accuracy and simpler methods is still a problem worth studying.