A Novel Robotic-Vision-Based Defect Inspection System for Bracket Weldments in a Cloud–Edge Coordination Environment

Abstract

Arc-welding robots are widely used in the production of automotive bracket parts. The large amounts of fumes and toxic gases generated during arc welding can affect the inspection results, as well as causing health problems, and the product needs to be sent to an additional checkpoint for manual inspection. In this work, the framework of a robotic-vision-based defect inspection system was proposed and developed in a cloud–edge computing environment, which can drastically reduce the manual labor required for visual inspection, minimizing the risks associated with human error and accidents. Firstly, a passive vision sensor was installed on the end joint of the arc-welding robot, the imaging module was designed to capture bracket weldments images after the arc-welding process, and datasets with qualified images were created in the production line for deep-learning-based research on steel surface defects. To enhance the detection precision, a redesigned lightweight inspection network was then employed, while a fast computation speed was ensured through the utilization of a cloud–edge-computing computational framework. Finally, virtual simulation and Internet of Things technologies were adopted to develop the inspection and control software in order to monitor the whole process remotely. The experimental results demonstrate that the proposed approach can realize the faster identification of quality issues, achieving higher steel production efficiency and economic profits.

1. Introduction

Arc welding is an extremely important technology primarily embraced by the automobile manufacturing industry [1,2,3]. The adoption of arc-welding robots in the production process has yielded many benefits, including enhanced productivity, cost savings, and improved efficiency [4,5]. The arc-welding robot usually needs to be supplied with a stable voltage and current in the process of joining parts, but this is not easy to achieve in practice. Consequently, steel surface defects cannot be avoided, and their quality inspection is particularly important. The post-manufacturing inspection of parts is usually performed at checkpoints due to the open-loop production process. Often, experienced workers examine steel surface defects with their eyes, leading to judgment subjectivity.

With the increasing demand for automatic inspection, the vision-based inspection system has opened a window of opportunity for automation-assisted detection. The vision-based inspection system has created opportunities for automation-assisted detection due to the increasing demand for automatic inspection [6,7,8,9]. As the computing power of hardware devices develops, the vision-based inspection system using detection algorithms with high accuracy is becoming a popular topic. Early machine vision algorithms focused on morphological characteristics for identifying key issues. Because the geometrical features of various defects are selected manually, some important feature information may be ignored in the process of feature extraction. Currently, convolutional neural networks (CNNs) [10], a type of promising deep learning model that relies on the network itself for automatic feature extraction, have been successfully applied to challenging visual tasks like image classification [11,12,13,14], object detection [15,16,17,18], image segmentation [19,20,21], etc. In pursuit of a better detection performance, the network has been improved according to various real-world scenarios and has become increasingly complex, meaning that higher device computing power is required. If this cannot be provided in actual manufacturing industries, the speed of defect detection will be affected, reducing production efficiency.

Edge computing, with the advantage of being closer to the data source, is proposed and usually used on-site to provide a faster response speed. Researchers have demonstrated that combining deep-learning-based approaches with edge computing can help to develop diverse application systems [22,23]. Edge computing can also be used in combination with cloud computing that provides on-demand services for intensive calculation [24,25,26,27,28]. In order to complete the inspection task in a hash industrial production environment, a more sophisticated system can be created by integrating advanced information technologies such as the Internet of Things (IoT), virtual simulation and cloud–edge computing. In this work, the framework of a robotic-vision-based defects inspection system is proposed, making use of advanced information technologies in a cloud–edge computing environment to attain a fast speed for computing performance. In the stage of steel surface defect inspection, the improved YOLOv5 object detection algorithm is applied to inspect multi-scale defects, in which a YOLO-SPD-Conv module, consisting of the space-to-depth layer and non-striped convolution layer, is added to further retain discriminant information on small defects.

The following structure is applied in this paper: Section 2 presents the related work. The framework of the robotic-vision-based defect inspection system for bracket weldments is described in Section 3. The details of the surface defect detection algorithm for bracket weldments are explained in Section 4. The detection performance verified on the edge through experiments, the cloud–edge coordination inspection and the control software developed are described in Section 5. Finally, Section 6 discusses the conclusion and future work.

2. Related Works

As problems have emerged about manual intervention during the manufacturing process, the introduction of automation systems has assisted workers considerably in production lines. Lee et al. [29] developed a laser-vision-based inspection system for welding defects on automobiles’ shock absorbers. Because of noise in industrial environments, however, it is difficult to apply the laser vision sensor in production lines. Therefore, the open-loop solution is applied to the system, in which the laser sensor is fixed and the shock absorber is moved. To solve these problems, Nguyen et al. [30] developed an improved laser-vision-based inspection system for small beads, in which a complementary metal oxide semiconductor (CMOS) camera is also applied to capture laser stripes. With this system, it has been proven that a higher detection accuracy can be ensured through the laser–camera mechanism. Chu et al. [31] developed an automated-vision-based system for non-destructive post-welding quality inspection. A new image-processing technique was proposed that can extract weld joint profiles and feature points and measure bead size and detect defects. The 3D surface of the weld can also be reconstructed to monitor weld quality online. Nevertheless, the above detection algorithms involved in these systems cannot meet changing production demands that require a variety of defect features to be considered during algorithm design.

Senthil Kumar et al. [32] proposed a system to classify various defects on butt joints. The specific algorithm process is divided into the following two steps: Firstly, the regions of interest in defect images are localized and segmented. Then, a back-propagation neural network is utilized as the classifier. However, this method heavily relies on the manual definitions of defect features; thus, it can be subjective and time-consuming. Weimer et al. [33] explored an alternative method that uses deep convolutional neural networks (CNNs). Unlike manual approaches, deep CNNs autonomously generate features by learning from large training datasets. The results of that study are promising, as the use of deep CNNs has been shown to significantly improve defect detection accuracy while keeping false alarm rates low. Sassi et al. [34] proposed a smart system for quality control assessment in industry using deep learning techniques. It successfully detects welding defects on fuel injector assemblies with an accuracy of 97.22% using approximately 7 million trained network parameters and a limited number of images. It demonstrates that deep neural networks can replace conventional human quality inspection efforts. Shin et al. [35] proposed a non-destructive testing method for detecting and predicting porosity defects in real time during the welding of zinc-coated steel. A deep neural network is utilized that can analyze the welding voltage signal without requiring any additional device in the proposed system. A high-strength steel sheet used in the automotive industry was employed to measure the process signals. Feature variables were extracted through preprocessing, and their correlation with weld porosity was analyzed. As detection algorithms become increasingly complex, with new data being continuously accumulated during operations, the computing power requirement of the system are becoming stricter. There is an urgent need for powerful computational frameworks.

To support inspection systems, computing environments are widely developed for customers’ different requirements [36,37,38]. Li et al. [39] developed an automatic pipeline inspection system that can overcome data accessibility and security limitations by deploying a cloud-based framework. The pipeline defect detection cloud system was proposed with role encryption for video collection and access security and a hybrid information method for defect detection. Huang et al. [40] introduced a unified framework for deep neural networks used for image defect detection and classification in edge computing. Transfer learning and data augmentation are included in the framework, resulting in a better accuracy with small samples. Implementing this approach in a rolling manufacturing system, it reduced computational time and satisfied real-time requirements. Chu et al. [41] also developed an inspection application in an edge-computing environment. A novel method that combines supervised and unsupervised learning was proposed to reduce the frequency of retraining and optimize the CNN-based detection model. On this basis, few annotated data points and iterative times are needed for the edge, which greatly improves the system’s efficiency. Vater et al. [26] combined edge and cloud computing to optimize the manufacturing of electric motors. A cloud–edge architecture was proposed for the efficient processing of vast amounts of data, which highlights its advantage over pure-cloud solutions. Wang et al. [42] also developed an inspection system in a cloud–edge computing environment in which the final model was applied to detect defects at the edge. The Faster Region-based CNN [38] model was selected and trained, considering the fact that two-stage methods can yield excellent results in the cloud.

CNN-based algorithms are not only suitable for image classification but can also be improved for object detection. In particular, desirable detection results can be gained using the two-stage method, in which a large number of candidate boxes are first obtained and then the CNN is used to evaluate whether they are in an image. However, most industrial sectors are production-oriented, with the pursuit of higher production efficiency. A faster detection speed may be considered in real-time applications. Additionally, open-loop production, where the image sensors are fixed and the objects to be detected are moved at checkpoints, may be less efficient. Therefore, more effort needs to be devoted to deploying a more efficient automation-assisted inspection system in the production line.

3. Framework of a Robotic-Vision-Based Defect Inspection System for Bracket Weldments

The proposed robotic-vision-based system framework has four primary elements, including smart terminals, an edge-computing center, cloud-computing center, and control center. The proposed system framework is presented in Figure 1.

• Smart terminals: The smart terminals mainly include the hardware for image acquirement (the industrial camera and the light source) and the arc-welding robot. To achieve instant messaging, functional interfaces and multi-protocol technologies are applied for data interaction and transmission, such as MODBUS, WIFI, and 4G edge gateway. In this work, the imaging module is designed as follows: the CMOS industrial camera is chosen, characterized by its image independence from the connecting requirements, and the light-emitting diode (LED) is selected for illumination. To simplify the inspection process, the image acquirement hardware is installed on the end arm of the arc-welding robot. The data source transferred to the edge-computing center usually includes the images of the bracket weldments that need to be inspected, and the bracket weldments come from the arc-welding-robot-based production line.
• Edge-computing center: The edge-computing center can provide the on-site services, such as historical data queries that need to be uploaded to the cloud-computing center, providing a fast response to operations. The edge devices are mainly composed of the Raspberry-Pi-powered board and the edge gateway. To realize a fast response in defect inspection, the Raspberry-Pi-powered board is equipped with the computational resource as the edge server and connected to the CMOS industrial camera. Firstly, image data are transmitted from the edge gateway to the edge-computing center through the transfer interface and protocol. Then, the images are pre-processed in the edge-computing center and are collected through the imaging module. Finally, the processed images are uploaded to the cloud-computing center for historical data storage. In this way, the edge-computing center can effectively save time and relieve stress on the cloud, and a fast response can be efficiently completed.
• Cloud-computing center: Considering that images are captured from different shooting distances, different detection models may be needed. Therefore, the cloud server is responsible for storing the detection models, except for the processed images. In addition, sufficient computing resources can be provided by the cloud-computing center, which ensures efficiency to train and update the detection models. In this way, the detection models will be trained and updated in the cloud and then distributed to the edge for defect inspection. As a result, defect inspection results are obtained locally, in close proximity to the data source, within a short time. At the same time, the inspection results will be transmitted to the part of the control center used for further parameter adjustments. Additionally, information from the training process can be saved as images or files and sent to the control center.
• Control center: The control center mainly provides the management service for the entire inspection process, including the control devices, control strategy and control software. After the devolved model has been implemented in the edge-computing center, the inspection results will be shown at the control software’s interface. The control strategy is formed based on the state data collected from the control devices (e.g., the robot controller), which examine welding parameters such as the voltage, current, and wire feed speed, determining whether they are within the recommended range. The above control process can be visualized by means of Unity 3D and programmatically adjusted according to the actual industrial situation using C#. In this way, the large number of fumes and toxic gases generated during arc welding can effectively be avoided.

4. Surface Defect Detection Approach Based on YOLOv5

In this part of the study, the one-stage method that involves evaluating a small number of predefined boxes in cross-scale feature maps was applied due to its fast detection speed. Meanwhile, the edge server could further improve the detection speed. In the proposed system, accuracy is another important evaluation criterion of system performance. In the actual industrial situation, some challenges may increase the difficulty of accurate identification and location. Evolved from you-only-look-once (YOLO)-based object detection algorithms [43,44], the YOLOv5 object detection algorithm [45], one of the most accurate object detection algorithms, was applied and trained on the created datasets. Moreover, the improved module is proposed to further improve the detection precision.

4.1. Data Pre-Processing

Considering that welding images are collected vertically, their rotation operation can make the final model more robust in terms of direction. Therefore, the rotation operation is introduced, and the function is defined as:

$$\left[\begin{array}{c}{x}_i^{\prime }\\ y_i^{\prime }\end{array}\right]=\left[\begin{array}{c}cos\theta \\ sin\theta \end{array}\right.\left.\begin{array}{c}-sin\theta \\ cos\theta \end{array}\right]\left[\begin{array}{c}{x}_i\\ y_i\end{array}\right]$$

(1)

where ($x_i,$ $y_i$) represents the original input image pixel, ($x_i^{\prime }$, $y_i^{\prime }$) represents the output image pixel after being rotated, and θ represents the rotation angle.

In addition, one way to eliminate the effect in which the defect area’s bounding box deviates from its original position is to rotate the original bounding box. Therefore, the bounding box parameters of the image annotation files need to be adjusted according to the origin of the coordinates. The rotation function is defined as:

$$\left\{\begin{array}{c}{x}_{min}=min(x_1^{\prime }, x_2^{\prime }, x_3^{\prime }, x_4^{\prime })\\ y_{min}=min(y_1^{\prime }, y_2^{\prime },y_3^{\prime }, y_4^{\prime })\\ x_{max}=max(x_1^{\prime }, x_2^{\prime }, x_3^{\prime }, x_4^{\prime })\\ y_{max}=\max \left(y_1^{\prime }, y_2^{\prime },y_3^{\prime }, y_4^{\prime }\right)\end{array}\right.$$

(2)

where the parameters of $x_1^{\prime }$ and $y_1^{\prime }$ represent transformed coordinates, and the subscript number represents the four points of the bounding box.

The rotation verification of one image is shown in Figure 2. After the rotation operation, the image is rotated, and the coordinates of the bounding box where the label is stored in XML files are adjusted so that the bounding box always falls on the target.

4.2. Surface Defect Detection

Evolved from YOLO-based deep learning methods, the version called YOLOv5 can offer highly accurate detection for objects. In this work, the lightweight YOLOv5s model was used to realize the detection of multi-scale and multi-type steel surface defects (Figure 3). Three primary parts are involved in the structure of the YOLOv5s model: the backbone, the neck and the head. Firstly, the backbone network is a core part of the model used to extract features of the input image based on the Convolutional Block Sequences (CBS) and the Cross-Stage Partial (CSP) structures. The CBS component is composed of Conv (convolution with stride two), BatchNorm (batchnormalization) and SiLU (a special Swish activation). The Resx represents multiple bottlenecks of the Resnet neural networks [46]. The CSP structure is similar to the bottleneck structure, but their specific operations on tensors are different. In particular, the shortcut pattern is concatenation in the former case but addition in the latter. Then, multiple-scale features are mainly fused from different feature pyramid levels in the neck network. Finally, the head is responsible for defect detection through the loss function (see Section 4.3 for details).

In practice, one type of defect, the end hole, may be quite small. Image scaling, one machine-vision-based method, is usually adopted to eliminate the effect of multi-scale defects, but size differences are inevitable when images are captured from fixed shooting distances. Another issue is where the defect is similar to its background, since the weakness of class separation occurs in steel bracket weldments. If the above issues are not well addressed, it will be more difficult to accurately locate and identify defects.

Inspired by Spatially Partitioned Depthwise Convolution (SPD-Conv) [47], the YOLO-SPD-Conv module is proposed to improve the detection precision, wherein the space-to-depth layer and non-striped convolution layer are added to CBS and CSP structures. The feature map is divided into smaller partitions, and each partition is processed separately. In this way, more detailed features can be extracted from the input data, leading to improved accuracy in the task of defect recognition. Therefore, seven YOLO-SPD-Conv modules are added to further retain as much discriminant information as possible (Figure 4).

4.3. Training Process

In this section, several training tricks are introduced, and the training parameters of our model are presented in Algorithm 1. The Warmup and Cosine learning rate strategy is adopted to accelerate training. The multi-scale training strategy is introduced to reduce the memory occupation and computational burden on the GPU. In this case, the training sizes are the random values of 0.5 × 3072~1.5 × 2048 pixels. With 32 times at most during down-sampling, all integers are multiples of 32.

In addition, the mean squared error is used to obtain a high IoU during the process of bounding box regression. The Adam algorithm is applied as an optimizer to achieve gradient normalization as well as loss minimization during backpropagation. The loss function is mainly composed of three parts, as shown in Equation (3). Binary cross entropy is adopted as the loss equation for classification (Lcls) and the object (Lobj) in Equation (4). CIoU loss is defined as shown in Equation (5) and used for the location (Lloc):

$$Loss={\lambda }_1{L}_{cls}+{\lambda }_2{L}_{obj}+{\lambda }_3{L}_{loc}$$

(3)

where ${\lambda }_1, {\lambda }_2$ and ${\lambda }_3$ represent equilibrium coefficients.

$$L_{BCE}=-\left(ylog\right(p\left(x\right)+(1-y)log(1-p(x\left)\right)\left)\right)$$

(4)

where $p\left(x\right)$ represents the output of the model, and y is the real label.

$$L_{CIOU}=1-IOU+\frac{{\rho }^2(b,b^{gt})}{c^2}+\alpha \upsilon$$

(5)

where IoU is the intersection-over-union, ${\rho }^2(b,b^{gt})$ represent the Euclidean distance of the center points of the prediction box and the real box, c represents the angular distance of the minimum closure area that can contain both the prediction box and the real box, and α and υ are the aspect ratio.

Algorithm 1 Training process of the proposed approach

Step 1: Import the necessary libraries and tools for image processing, object detection and deep learning, such as the Warmup and Cosine learning rate strategy and multi-scale training strategy;

Step 2: Define the object detection parameters, such as the confidence threshold, non-maximum suppression (NMS) threshold, input size, etc.;

Step 3: Load the dataset (Section 5) for training with labeled images and annotations;

Step 4: Split the dataset into training and validation sets at 85% and 15%, respectively;

Step 5: Use the multi-scale training strategy to process the dataset by resizing the images to the input size of the model, scaling and normalizing the pixel values;

Step 6: Train the proposed YOLOv5s object detection model (Section 6) on the dataset, using the Adam optimizer and loss function;

Step 7: Use the Warmup and Cosine learning rate strategy to accelerate training;

Step 8: Monitor the training progress by evaluating the loss and mAP score on the validation set;

Step 9: Fine-tune the model by adjusting the learning rate and other hyperparameters;

Step 10: Once the model is trained and optimized, use it to perform object detection for the new images of the bracket weldments;

Step 11: Apply NMS to remove overlapping object detections and refine the output;

Step 12: Display the final detection results for the original images, including the bounding boxes, class labels and confidence scores.

4.4. Evaluation Metrics

To better evaluate the algorithm, several indexes are analyzed as follows. First, the basic index, the intersection-over-union (IoU), is calculated to assess the quality of the bounding box. Second, mean accuracy precision (mAP) is used to comprehensively evaluate the detection accuracy. Finally, frames per second (FPS) is calculated to evaluate the model’s inference speed:

$$\mathit{IoU}=\frac{B_{predict}\cap B_{truth}}{B_{predict}\cup B_{truth}}$$

(6)

where the bounding box is predicted through B predict, and B truth represents the ground truth bounding box. The predicted box is more exact when the IOU value is higher. If the predicted object in A is not in the ground truth set B, then it is marked as a false positive:

$$\mathit{mAP}=\frac{{\sum }_{i=1}^{n}A{P}_i}{n}$$

(7)

where n is the number of categories that need to be detected, and AP_i is the average precision for each category i.

$$\mathit{FPS}=\frac{framNum}{framTime}$$

(8)

where framNum is the number of frames captured within a set time period, and framTime is the duration of that time period in seconds.

5. Experimental Results and Analysis

As described in this section, some raw data on bracket weldments were gathered and tested on the edge through experiments. The Raspberry-Pi-powered board is equipped with an Intel i7-10750 2.60 GHz processor and an NVIDIA GeForce GTX 1660ti GPU (NVIDIA company, Santa Clara, CA, USA) for parallel computation. The experiments were implemented in the Python 3.80 environment and run on the Ubuntu 16.0 LTS. The cloud–edge coordination inspection and control software was developed combining IoT and virtual simulation technologies. When analyzing the industrial scenario, the workstation, with an Intel i7-7700 3.60 GHz processor and an NVIDIA GeForce RTX 2080 Ti GPU, serves as the cloud platform. The 128 GB memory of the workstation satisfies the requirements for the storage of image data and model training.

5.1. Dataset Creation

The bracket plays an important role in supporting the automobile body. If there are surface defects in the weld seams on the robot-joined bracket, the normal driving of the automobile will be influenced. Therefore, bracket weldments were chosen in this work to verify the effectiveness of the proposed approach. The hardware setup for dataset creation is presented in Figure 5. The robot controller was connected to the Eston arc-welding robot, on which a 600-million-pixel CMOS industrial camera was installed for image capture, and the light controller was used to control the on–off functions of the P-RLG-80-90-WS LED light source. Considering the rigid material of steel bracket weldments that can lead to an exposure problem when capturing images, the camera and light source were equipped with polarizers. Furthermore, a dome was designed to protect the image acquirement hardware from strong welding sparks and large-area spatters.

On this basis, the dataset was created, and some high-resolution images are presented, in which the structure of the weld seam can be observed in the fusion zone, as presented in Figure 6. Each of the images revealed the property of a weld seam obtained at various arc energy levels. Seven types of weld seams, including normal, penetration, welding offset, missing solder, not beautiful, end hole and undercut seams, can be observed, as they are commonly observed in working conditions.

Additionally, the sample size was further expanded so that each sample was rotated eight times, by 45 degrees each time, generating eight rotated images from the original one. The percentages of training and validation sets were 85% and 15%, respectively. The numbers of samples are shown in

Table 1. The numbers of samples.

Defect Type	Number
Normal	1272
Undercut	1296
Welding offset	1272
Penetration	1272
Not beautiful	1272
End hole	1248
Missing solder	1272

.

5.2. Edge Experiment and Result Analysis

Two experiments were conducted to illustrate the performance of the proposed method in terms of detection accuracy and computational speed, respectively.

5.2.1. Detection Result Analysis

In order to widely verify the applicability of the proposed approach, previous machine-vision-based methods may be inappropriate in this case. Therefore, the typical two-stage method of Faster R-CNN [48] and one-stage method of Single-Shot Multi-Box Detection (SSD) [49] are adopted in this paper. The ResNet-based networks are one of the most commonly used convolutional networks that can be used as backbone networks in the Faster R-CNN and SSD object detection algorithms. As a result, Faster R-CNN+ResNet-50 and SSD+ResNet-50 are compared with the YOLOv5s and the proposed detection model in terms of detection precision. The experimental results of the mean average precision (mAP@50) are shown in

Table 2. Experimental results of different models.

Models	Feature Extraction Network	mAP (@0.5)
Faster RCNN+ResNet-50	ResNet-50	86.40%
SSD+ResNet-50	ResNet-50	87.40%
YOLOv5s	CSPNet	90.40%
YOLOv5s	CSPNet+YOLO-SPD-Conv (proposed)	98.60%

. The mAP@0.5, a commonly used measurement, considers the precision and recall when the IOU is more than 50%. The mAP of the proposed approach is improved by at least 5.0%. The training parameters were set as shown in

Table 3. Setting of training parameters.

Models	Faster RCNN + ResNet-50	SSD + ResNet-50	YOLOv5s	Proposed Method
Image size	640 × 640	640 × 640	640 × 640	640 × 640
Learning rate	0.001	0.001	0.01	0.01
Optimizer	Adam	Adam	Adam	Adam
Weight decay	0.0001	0.0001	0.0005	0.0005
Batch size	8	16	16	16
Number of epochs	100	100	100	100

.

Some of the inspection results identified are presented in Figure 7, Figure 8 and Figure 9. Theoretically, the Faster R-CNN+ResNet-50 is better than the one-stage methods in terms of its detection effect. However, its performance may vary depending on the specific dataset and detection task. In this work, the end hole defect is small, while the other defects are similar to the background, which make it difficult to distinguish them from nuts, dirt or other objects (Figure 7a). As a result, the defects were detected with low accuracy (Figure 7b). Compared with the Faster R-CNN+ResNet-50, the detection performance of SSD+ResNet-50 improved a little, but there are still some problems, such as false detections and missing detections. The oval hole was incorrectly detected as a penetration defect (Figure 8a), the normal weld seam was mistakenly identified as a missing solder defect (Figure 8b), and the detection of the welding offset was missed (Figure 8c). To solve these problems, our approach applies parallel convolutions to extract features on different scales that can effectively avoid the loss of local information, which improves the accuracy of defect objection. In order to avoid such a situation in which the candidate boxes are either too large to cover the defect completely or too small, causing the overlapping problem, appropriate aspect ratios and scales are obtained using the Adam algorithm to obtain higher-quality candidate boxes and more accurate defect localization (Figure 9).

5.2.2. Detection Speed

As presented in

Table 4. Comparison of inference time (ms) and inference speed (FPS).

Models	Inference Time (ms)	Inference Speed (FPS)
Faster RCNN	118	8
SSD	68	15
YOLOv5s	57	18
Proposed approach	47	21

and Figure 10, the fastest detection models are the proposed one and the SSD, which have a speed of 20 FPS, among all the models, but the inference time per image is only 47 ms for our approach. On the basis of the detection results’ analysis, when detection applicability is combined with the model’s inference speed, SSD and our approach have relatively fast detection speeds, both reaching 20 FPS. This demonstrates that the one-stage method has the potential to be applied to this industrial scenario. The proposed model can achieve better detection results on the created dataset. As a result, our proposed approach is a suitable choice for real-time applications.

5.3. Ablation Study

Ablation experiments were conducted on the created dataset. The original YOLOv5s model was trained as the benchmark, adding the YOLO-SPD-Conv module to verify the detection performance. Some of the results identified are shown in Figure 11, before and after adding the module. They demonstrate that this technology, using dilated convolutions to increase the receptive field of the detection model, can enhance the ability to capture more context and information from the image of the bracket weldment.

At the same time, the training loss and mAP@50 were calculated to further prove that the utilization of the YOLO-SPD-Conv module can improve overall inspection performance. As shown in Figure 12, overall, the training loss after adding the YOLO-SPD-Conv module is lower than that without the module. This means that the training loss is lower, and the model performance is better. As the number of epochs increase, our approach’s mAP also increases until reaching approximately 90, where the mAP remains stable above 95%. In conclusion, it is evident that the YOLO-SPD-Conv module can improve the models’ sensitivity and specificity and enhance the detection accuracy.

5.4. Cloud–Edge Coordination Inspection and Control Software

The following information can be seen in the software: the model training information in the cloud, the detection information at the edge and the virtual and physical robot welding parameter information on-site. The operation process of the cloud–edge collaboration inspection and control software is shown in Figure 13.

The cloud-computing center mainly trains and updates the model using the uploaded images and then devolves it to the edge. The edge-computing center mainly carries out real-time defect inspection using the devolved model. At the same time, the control center provides control strategies (e.g., the panel or voice prompts) so that the operator can make parameter adjustments to the virtual arc-welding robot. The control strategies are set in advance: for example, if the voltage is too high, panel or voice prompts are given to indicate that it should be lowered so as to reduce the risk of “penetration”. If the wire feed speed is too low, it should be increased to prevent “undercut”. Finally, the welding parameters of the physical robot can be adjusted by combing virtual robot operation and expert experience. In this way, the software can achieve the closed-loop management that can improve the production efficiency.

A flow chart for the program is presented in Figure 14. Firstly, the cloud–edge coordination inspection and control software need to be initialized. Then, data acquisition is implemented, through which real-time data from the sensors (cameras, voltage sensors, wire feed speed sensors) attached to the welding equipment is obtained. To transmit the collected image data for edge computing, a GigE-based transmission standard is adopted to realize image signal transmission. The edge computing is responsible for preprocessing and analyzing the captured data locally through the edge devices. The analyzed welding defect data and control commands are sent to the cloud for further processing through the HTTP standard protocol. In addition, to ensure the encrypted transmission of the image data, control commands and model updates to protect the integrity and privacy of the information, the WiFi module, one of the network connectivity solutions, is employed to establish connections between the edge devices and cloud. After that, control strategies and decision making are executed to notify the operator about the detected defects and suggest corrective actions and trigger control commands to optimize the welding process in real time. Data visualization is performed to generate visualizations and prompts for monitoring and optimization. Finally, the program ends or loops back to the data acquisition stage for continuous monitoring and analysis.

6. Conclusions

With the increasing demand for reliable and effective defect inspection methods in various industrial sectors, which are embraced with advanced information technologies, a robotic-vision-based defect inspection system for bracket weldments was proposed for a cloud–edge coordination environment. Instead of public datasets, accurate and accessible datasets were built in this work. In this way, the YOLOv5 object detection algorithm was used to inspect multi-scale defects in which a YOLO-SPD-Conv module was added to further retain discriminant information on small defects. Furthermore, a control software for defect inspection was developed for the arc-welding-robot-based production line. By enabling feedback and corrections during the manufacturing process, it helps to improve overall manufacturing efficiency. As a result, the proposed system’s cost-effectiveness and reduced inspection time make it an attractive option for manufacturers and industries requiring efficient quality control solutions. In this work, however, we mainly focused on the defect inspection and classification of brackets in automobiles. More types of datasets should be gathered and tested in the future.