Assessment of Laser Joining Quality by Visual Inspection, Computer Simulation, and Deep Learning

Featured Application

Polymer joining is an important technology in automotive and industrial applications. Lasers are an attractive heat source for polymer joining because they are more controllable, productive, and easy to automate and cause minimal collateral damage to heat-affected zones. However, the assessment of quality evaluation remains a problem for reliable products in mass production. Laser polymer joining requires a non-destructive evaluation, and recent developments in deep learning have begun a new era for this requirement. We believe that deep learning technology can be used to provide state-of-the-art polymers that can improve quality assessment in the non-destructive polymer joining process.

Abstract

Polymer joining results are evaluated and compared in different ways, such as visual inspection, computer simulation, and deep learning analysis, to assess the joining quality. For the experiments, energies in the range of 3 to 5 J/mm were used from preliminary experimental data. A total of 15 welding experiment schedules were performed. Weld defects due to a lack of fusion were detected in some regions of specimens treated with a low-power laser region (3 J/mm), where a lack of fusion, in turn, occurred due to underheating. Bubble-shaped weld defects were observed in some specimens treated with a high-power laser region (5 J/mm); melting occurred due to the overheating of the specimen. Computer simulations were used to trace the boundaries of the fusion zone, and yielded results similar to the visual inspection ones. In the lower-energy region, the energy may not be sufficient to fuse the specimen, whereas the high-energy region may have sufficient energy to break down the polymer chains. A novel deep learning algorithm was used to statistically evaluate the weld quality. Approximately 1700–1900 samples were collected for each condition, and the pre-trained quality evaluation indicated a highly reliable (>98%) welding classification (fail or good). According to the results of this study, welding quality assessments based on visual inspection, computer simulation, and DL-based inspection yield similar results.

1. Introduction

Polymer materials are widely used in the interiors and exteriors of automobiles because they are lighter than metallic materials. Their excellent strength and moldability at low temperatures represent additional advantages of polymers, as this reduces production costs and enhances design flexibility [1,2]. Researchers have developed various techniques for producing components from polymer materials. Adhesive and mechanical fastening are typical bonding methods for polymers, but the recent development of direct thermal energy for joining has reduced costs and provided eco-friendly products. Various joint configurations were compared to distinguish the bonding quality, and the use of chromium interlayers allowed an increase in adhesion forces. Laser beams, ultrasonic waves, and radio frequency (RF) are also commonly used for polymer joining. Joining techniques have improved significantly over the past few decades [1,2,3,4,5,6].

A laser is a very versatile, controllable, and easy to use tool for the polymer joining process. In particular, direct thermal energy deposition provides strong joins, and there is little possibility of contamination during the joining process [2,5]. Furthermore, joining using thermal energy can be easily applied to various thermoplastic polymeric materials during the fabrication of automotive parts, biomedical devices, and electrical devices [2,3,4,5]. The advantages of direct joining include lower costs compared to adhesives and the eco-friendly nature of the technology, as no chemicals are used.

Lasers are also a preferred thermal energy source, as they are readily available in the industry, easy to use, and highly precise [3,4,5,6]. Lasers are also less sensitive to materials and can be used for most types of polymer joining, such as joining polymers to other polymers, glass, or other materials. They also have the advantage of good sealing, which produces better esthetic results after joining compared to other joining processes, such as ultrasonic and adhesive joining [2]. In laser joining, the thermal energy imparted by the laser beam is absorbed in the interface between transparent and opaque materials. The top layer is transparent to the laser, while the bottom layer is opaque [2,7]. As the most absorptive wavelength can be used for laser joining, the joining efficiency can be maximized without damaging the surrounding area. After the energy has been absorbed on the boundary, the joining region is heated to over the glass transition temperature and forged by an external pressing force [8,9].

In the era of the 4th industrial revolution, artificial intelligence (AI) technology using deep learning (DL) is being applied in various fields [10,11,12,13,14,15,16,17,18]. A computer vision-based DL algorithm has been used for autonomous vehicle driving, games, and medical applications. Recently, this technology has also been used to improve the production quality in manufacturing applications [11,12,13,14]. In DL for product quality monitoring, the greater the amount of correctly classified training data, the more accurate the results are.

A system for monitoring porosity in laser welding processes was developed using a DL algorithm [10], and real-time weld pool data were collected. DL was also used to predict the hardness distribution with the laser heat treatment of AISI H13 tool steel [11]. During the fabrication of sensor systems using laser welding processes, DL-based computer vision systems have been used to capture signals indicating the welding status [10]. A monitoring system to determine the degree of penetration during laser welding has also been introduced [10].

Convolution neural networks (CNNs) are the most commonly used networks for image-based DL. CNNs, as multilayer perceptrons, have layers in which each neuron is connected to all the neurons in the next layer. CNNs analyze hierarchical patterns in data and then generate smaller, simpler patterns. These networks also employ encoder-decoder structures to preserve the prominent features of images [15]. The successful use of CNN-based DL algorithms for product quality monitoring has been reported by many researchers.

This paper reports the results of a comparative analysis of the joining of polycarbonate (PC) and acrylonitrile butadiene styrene (ABS) polymeric materials with different characteristics using a diode laser. The weld quality was evaluated by (1) visual inspections, (2) computer simulations, and (3) DL techniques, and the results were then compared.

2. Experiments

2.1. Experimental Setup

A diode laser was used as the heat source in the experiments because these lasers are widely used for polymer bonding due to their excellent efficiency and ease of control. As shown in Figure 1, a laser beam resonated by the diode laser (c) was transmitted through fiber optics. The specimen was placed on the working table controlled by a computer numerical control (CNC) machine (a), and integrated control was achieved using a laptop computer (b). The delivered laser beam was focused on the specimen through a focusing lens installed at the head of the CNC.

The laser beam was transferred to the CNC head, and the distance between the workpiece and focus lens (FL = 50 mm) could be adjusted to obtain the desired beam size. In the experiments, the focused spot diameter was set to 5 mm based on the preliminary experimental results. A 3-axis-controlled CNC machine (SME3030; SME Trading) was used for the precise transfer of specimens by G-code, and both the laser and specimen transfers were controlled using the laptop. The laser generated a continuous waveform (CW) with a wavelength of 980 nm, and the output was set in the range of 3~17 Watts. The welding speed was set in the range of 60~250 mm/min depending on the output power. A total of 15 combinations of laser power and welding speed were used to achieve an input energy per unit length of 3~5 J/mm; the detailed welding schedules are summarized in

Table 1. Matrix of welding schedule.

Input Energy (J/mm)		Power (Watt)	Welding Speed (mm/min)
#1-1	3	3	60
#1-2	3	5.5	110
#1-3	3	8	160
#1-4	3	10.5	210
#1-5	3	13	250
#2-1	4	5	75
#2-2	4	7.5	112.5
#2-3	4	10	150
#2-4	4	12.5	187.5
#2-5	4	15	225
#3-1	5	7	84
#3-2	5	9.5	114
#3-3	5	12	144
#3-4	5	14.5	174
#3-5	5	17	204

.

The irradiated laser beam passed through the transparent PC material and was absorbed on the boundary by the ABS material (through transmission). The absorbing layer was heated to the glass transition temperature, which is approximately 135 and 110 °C for PC and ABS, respectively [19]. Then, the absorbed heat energy on ABS transferred to the contact PC layer, which resulted in a crosslinking of polymer chains at the contact boundary.

For the experiments, the specimens on the joining jig were prepared, and the materials were cut to dimensions of 42 × 100 × 6.5 mm for ABS and 42 × 100 × 3 mm for PC. The specially designed jig shown in Figure 2 was used to hold the weld specimens. Precision mold springs and wrench bolts were used to maintain a precise and constant pressing force on the specimens via pressure plates made of aluminum. The mold springs had an inner diameter of 6 mm, an outer diameter of 12 mm, a free length of 40 mm, and a spring constant of 0.7 kg_f/mm. The wrench bolts were tightened in four places to yield a force of 22.4 kg_f in total.

2.2. Computer Simulation

A computer simulation can be effectively used to track the fusion boundary based on the temperature distribution. By calculating the temperature distribution, the weld quality can be estimated indirectly. The heat transfer module of the COMSOL Multiphysics software (ver. 5.3a; COMSOL Inc.) was used and the results were validated based on prior research, where the simulated temperature distribution was compared to the actual temperatures measured by thermocouples and infrared (IR) cameras.

For the simulation, a blocked domain of dimensions 20 × 10 × 2 mm was used, and a linear heating source with a Gaussian-shaped laser beam was assumed to be as described in Equation (1).

$$Q\left(x,y,z\right)=Q_0\left(1-R_c\right)·\frac{A_c}{\pi {\sigma }_x{\sigma }_y}{e}^{-\left[\frac{{\left(x-x_0\right)}^2}{2{\sigma }_x{}^2}+\frac{{\left(y-y_0\right)}^2}{2{\sigma }_y{}^2}\right]}·e^{-A_{c}z},$$

(1)

where Q₀ is the peak power, R_c is the reflectance, σ_x and σ_y are the beam waists in the x and y directions, and A_c is the skin depth/absorption layer thickness.

As described in Equation (1), the irradiating laser beam was assumed as a 2D ellipsoidal distribution, with A_c of skin depth with respect to the z-direction. The skin depth can be analytically calculated for the metallic material, but it was found from computer simulation in this research. The reflectance R_c value is also an important value, and it was found by the experimental data analysis by using an integrating sphere.

To trace the temperature distribution, multiple probe points were assigned to various locations. The simulation conditions and schedules were identical to the experimental ones which are described in Table 1. As the governing equation for the simulation domain, a general 3D heat transfer equation was used, as described as Equation (2).

$$\rho C_p\left(\frac{\partial T}{\partial t}+{\mathrm{u}}_{trans}·\nabla T\right)+\nabla ·\left(q+q_r\right)=-\alpha \mathrm{T}:\frac{dS}{dt}+Q,$$

(2)

$$q=-k\nabla T or h\left(T_{ext}-T\right),$$

(3)

where Equation (3) is a conductive (κ) or convective (h) heat transfer through the surrounding bodies. In Equation (2), ρ is the density of the polymers; C_p is the specific heat capacity at constant stress; T is the absolute temperature; u_trans is the velocity vector of translational motion; q and q_r are the heat flux caused by conduction and radiation, respectively. α is the coefficient of thermal expansion, S is the second Piola-Kirchhoff stress tensor, and Q contains additional laser heat sources which can be described using Equation (1).

2.3. Deep Learning Experiment

Given the importance of reliable training for DL, large amounts of data were collected prior to the DL process. To construct the dataset, a series of images of specimens welded under conditions of 3, 4, and 5 J/mm was acquired. A charge-coupled device (CCD) camera (STC-SC33POE; Sentech) with a resolution of 640 × 480 was installed and a GigE connection was used for image collection. A linear translation stage (DDSM100/M; Thorlabs) was used for precise specimen translation (Figure 3). The specimens were transferred on a linear translation stage at a rate of 0.8 mm/s, and image data were acquired every 100 ms with 10 ms of exposure time. For each condition, approximately 1700–1900 sets of data were collected, and up to 28,500 images were used in the DL process.

The images collected for each condition were divided into two categories, “Good” and “Fail”, for the quality assessment. The acquired images had a resolution of 648 × 494, which was re-adjusted to 227 × 227 to match the input format of the DL algorithm (AlexNet). As shown in Figure 5, the straightness of the edges in the weld beads was used to assess the weld quality as “Good” or “Fail” by visual inspection. The DL algorithm was implemented in MATLAB (ver. 2019b; MathWorks Inc.) using the DL toolbox and an NVIDIA GPU (RTX 2070).

The MATLAB Parallel Computing Toolbox was used for implementation. The parallel computation was performed using CUDA (ver. 10.2; NVIDIA) to improve the speed of the algorithm by exploiting parallelism. The DL-based image processing algorithm was in turn based on an AlexNet CNN. A special program was coded to recognize videos and images, even if the image size or position was randomly placed. The AlexNet CNN consisted of 5 convolutional layers and 3 fully connected layers. For this research, the AlexNet was modified and the details of the DL process structure of DL are described in Figure 4.

The modified DL architecture consists of 8 layers, which include 5 convolutional layers and 3 fully connected layers. It uses Rectified Linear Units (ReLU) for lower error and faster convergence. The maximum number of repetitions was set at 30 and the mini-batch size was set to 128 from the preliminary DL research. Based on a series of training processes, it took less than a second to evaluate the welding quality when the algorithm was pretrained on 28,500 photographs using a spatial gray-level dependence matrix (SGDM). Overall, the predictive accuracy reached approximately 99% through repeated learning.

The pretrained network, as described in Figure 4, was adapted to our welding bead data. To apply the developed deep learning algorithm, we created a custom training loop using automatic differentiation. In a commercial deep learning toolbox provided by MATLAB and the initialization process was conducted for the weights and biases, which are given by the internal function—the “WeightsInitializer” and “BiasInitializer” properties of the layers, respectively. To minimize the loss function for the convergence, the standard gradient descent algorithm was used to update the network parameters, such as weights and biases, and basic mathematical equations.

In addition, to expedite the converging time and increase accuracy, the stochastic gradient descent with momentum algorithm was used to evaluate the gradient and update the parameters using a subset of the training welding data. The mini-batch size and the maximum number of epochs were optimized for accuracy. The basic governing equation is described in Equation (4):

$${\theta }_{i+1}={\theta }_{i-\alpha }\nabla E\left({\theta }_i\right)+\gamma \left({\theta }_i-{\theta }_{i-1}\right),$$

(4)

where i is the iteration number, α > 0 is the learning rate, θ is the parameter vector, E(θ) is the loss function, and γ determines the contribution of the previous gradient step to the current iteration. In the standard gradient descent algorithm, the gradient of the loss function, ∇E(θ), is evaluated using the entire training set, and the standard gradient descent algorithm uses the entire data set at once.

3. Results

3.1. Welding Experiment and Visual Inspection Results

The experiments were performed based on the welding schedule presented in Table 1, and the results are summarized in Figure 5. The welding was inspected visually for melting, overheating, or a lack of fusion caused by underheating. As described in the previous chapter, the welding experiments were performed with input energies in the range of 3~5 J/mm, but in some areas welding failure occurred due to under or overheating.

Typically, excessive heating resulted in bubble defects in the central region, while underheating resulted in a lack of fusion in the edge boundary region. The results of the visual inspections are summarized in Figure 6. As shown in the graph, a lack of fusion (experiment #1-1) due to underheating was observed under the low-power, low-speed conditions in the 3 J/mm region. However, in the 5 J/mm area, some overheating occurred under the high-power, high-speed conditions (experiment #3-5).

3.2. Computer Simulation

For the computer simulation, we followed the procedure used in a previous study. The results of the simulation are summarized in Figure 7. The output and welding speed of the laser beam are described in Table 1. The focused spot diameter was set to 5 mm, which is identical to the actual welding condition, and the absorption of the laser heat source was set to 60%, considering the PC transmission (90%) and ABS absorption efficiency (70%). The input heat source was assumed to be a 3D ellipsoidal Gaussian distribution with a skin depth in z-direction, as described as Equation (1).

To quantitatively analyze the results, the temperature distribution throughout the sample was traced from 0.25 mm underneath the surface at a position offset by 2.5 mm from the direction of travel of the laser (Figure 8). As shown in Figure 8, we predicted good results under the 4 and 5 J/mm conditions, where the temperature was observed to vary from 120 to 180 °C in all sections. However, as shown in Figure 8c, under low-power and low-speed conditions (experiment #1-1), 3 Watt–60 mm/min, a lack of fusion was found as the maximum temperature was 100 °C. At high power and high-speed transfer (experimental condition #3-5), 17 Watt–204 mm/min, the probability of welding failure was predicted to be high, as the maximum temperature would be at least 190 °C.

3.3. Deep Learning Algorithm and Process

Welding quality was evaluated under each condition using the DL algorithm. Figure 9a–c show cases with defects and those with a good end-product along with the experimental schedule described in Table 1. The welding quality was assessed based on the stochastic results, however the general trend of weld quality was identical to that of visual inspection. The detailed analysis is described in the following.

Under the low-energy condition where the heat input was set to 3 J/mm (experiments #1-1~1-5), the welding quality was not identical between the laser powers of 3 and 5.5 W. To obtain quantitative results, DL was performed on photographs of the weld surface. The results in each case (3, 5.5, 8, 10.5, and 13 W) were classified as fail or good, with the accuracy of >97% and >99%, respectively.

Under the mid-energy condition where the heat input was set to 4 J/mm (experiments #2-1~2-5), the welding quality was more acceptable than at lower energies regardless of the power levels. The laser power was increased from 5 to 15 W in increments of 2.5 W, and the quality of the laser welding was evaluated based on the images, which were classified as good or fail using our DL algorithm with an accuracy >99% or >94%, respectively.

Under the high-energy condition where the heat input was set to 5 J/mm (experiments #3-1~3-5), the laser power was increased from 7 to 14.5 W in increments of 2.5 W. The welding quality was acceptable except for experiments #3-4 and #3-5, where extremely high peak powers generated bubbles on the welded parts. Good and fail corresponded to accuracies of over 99% and 91%, respectively.

As a summary of the DL results, the weld quality was statistically assessed, as shown in the bubble chart in Figure 10. While deterministically concluded in Figure 6, the results can be statistically estimated with a varying reliability. By providing statistical data to operators, one can select the best region for the welding where a high level of success can be achieved.

4. Conclusions and Discussion

This paper reports an assessment of laser polymer joining by comparing visual inspection, computer simulation, and DL-based prediction. Based on the results of the preliminary research, these three modalities were used to increase the accuracy of the laser welding quality assessments.

For the experiments, energies in the range of 3~5 J/mm were used. A total of 15 welding experiments were performed, and the laser power and welding speed were varied. Weld defects due to a lack of fusion were detected in some regions of specimens treated with the low-power laser (3 J/mm), where a lack of fusion occurred due to underheating. Bubble-shaped weld defects were observed in some specimens treated with the high-power laser (5 J/mm); melting occurred due to the overheating of the specimen.

Computer simulations were used to trace the boundaries of the fusion zone and yielded results similar to the visual inspection. In the lower-energy region, the energy may not be sufficient to fuse the specimen, whereas the high-energy region may have sufficient energy to break down the polymer chains.

Our results were then reevaluated using a novel DL algorithm which was modified from the AlexNet algorithm. Using a data acquisition (DAQ) system with a CCD camera, image data were acquired every 100 ms with 10 ms of exposure time, and approximately 1700~1900 samples were collected for each condition, with 28,500 images in total were used for training. The pre-trained quality evaluation indicated a highly reliable (>98%) welding classification (fail or good). Specifically, the accuracy of the fail classification was over 99%, and that of the good classification was greater than 97% in the 3 J/mm condition. The accuracy was similar between the 4 and 5 J/mm conditions. Failures were mainly due to pores or bubbles at the joining boundary. However, in some cases light reflected by scratches on the surface led to an incorrect classification as joining failure, which reduced the reliability of our DL algorithm-based quality evaluation.

According to the results of this study, welding quality assessments based on visual inspection, computer simulation, and DL-based inspection yield similar results. In conclusion, a visual inspection can be replaced with image-based DL and computer simulation for similarly reliable quality inspections but with significant cost and labor savings.