Optimization of the Weld Pool Boundary Calculated by the LSTM-Based Measurement Method in GTAW

Abstract

The shape of the weld pool surface contains a lot of important features to reflect the quality of the weld. However, it is difficult to obtain the weld pool boundary precisely. In this paper, a boundary extension method is designed first to optimize the boundary profile of the interpolated weld pool surface calculated by the long-short-term memory (LSTM)-based measurement method. Experimental results show that after boundary extension, the errors of the left and right part of the boundary of the weld pool are slightly improved. The weld width error in the X direction is reduced to 2.43%, and the weld width error in the Y direction is slightly increased to 8.68%. Then the robustness of the LSTM-based model is analyzed by studying the phenomena of missing points and more points. To solve the problem caused by the missing/adding the first few imaging points, a forward-reverse united reconstruction optimization method is then designed. After optimization by forward-reverse united reconstruction optimization method, the boundary error of the reconstructed weld pool surface is obviously reduced. The weld width error is reduced to 1.62% in the X direction and 3.94% in the Y direction.

1. Introduction

The weld pool surface is the most important feature for the skilled-welder, adjusting the welding parameters such as feeding speed, welding speed, the posture of the welding torch in real time, and also the key and main sensing information to realize the automation and intelligentization of Gas Tungsten Arc Welding (GTAW). However, the interference of the super bright arc and high temperature significantly limit the application of the vision-based measurement methods. At present, there are four categories of vision-based measurement methods that have been reported to successfully observe the shape of the weld pool: (1) Shape from shading [1,2]; (2) shape from polarization [3]; (3) binocular stereo vision [4,5]; and (4) structured light based 3D vision. Among all these methods, the structured light-based 3D vision method dexterously avoids the interference of the super bright arc and the high temperature existing during welding, however, how to extract the 3D reflection points on the weld pool surface from the 2D imaging points on the imaging plane quickly and precisely becomes the key point.

Many researchers around the world have made efforts to solve this problem. Song et al. from the University of Kentucky [6] proposed two algorithms to achieve this task, namely edge-point algorithm (EPA) and one-point algorithm (OPA) respectively. The updating ways of the 3D reflection points in the two algorithms are both based on slope calculation, but have different assumptions for their initial positions. On this basis, an additional assumption was proposed by Zhang et al. [7] that the z-coordinates of the 3D reflection points in row or column satisfy quadratic polynomial. A dual-plane system was also developed [8] to measure the 3D weld pool surface, however, the diffusion of the laser points causes difficulty in an image-processing algorithm.

In our previous works, a novel measurement method of 3D weld pool surface in GTAW was developed based on the structured light-sensing system [9]. This method tactfully considers the 2D imaging points on the imaging plane as time series data and introduces long short-term memory (LSTM) neural network to achieve the mapping calculation from the 2D imaging points to the 3D reflection points on the weld pool surface. A sorting step is added to the sample points before training, which makes the LSTM-based measurement method completely skip the point recognition stage and greatly simplifies the calculation process. In GTAW experimental tests, the weld width errors $w_{err}^x$ and $w_{err}^y$ of the reconstructed weld pool surface reach 13.36% and 2.17%, respectively.

Although the proposed LSTM-based measurement method [9] of 3D weld pool surface has achieved decent results preliminarily, it is noted that $w_{err}^x$ is obviously higher than $w_{err}^y$. On the other hand, there are some discrepancies like “missing points” or “more points” that may cause the floating oxides on the weld pool or the scattering of the reflected laser points to be omitted. These phenomena may decrease the accuracy of the LSTM-based measurement method and increase the error of the weld pool boundary.

In this paper, we focus on the optimization of the LSTM-based measurement method [9] to obtain an idea weld pool boundary with minor errors. The paper is organized as follows: In Section 2, the causes of boundary errors of the reconstructed weld pool surface are analyzed in detail. To reduce the boundary errors, Section 3 introduces an optimization method of boundary extension. In Section 4, the phenomena of “missing points” and “more points” that may increase the boundary errors are explained in detail. The robustness of the LSTM-based measurement method and the optimization method of boundary extension is analyzed in Section 5. In view of the deficiency that the LSTM-based model shows, with poor robustness for initial input imaging points (on the upper left side of the reflection image) and low reconstruction accuracy for initial output reflection points, a forward-reverse united reconstruction optimization method is proposed in Section 6. Section 7 presents conclusions of this study.

2. Causes of Boundary Errors

Figure 1 is the experimental system used in this paper. A 100 mW laser (COHERENT, Wilsonville, OR, USA) with a wavelength of 660 nm is selected to project a pattern of 17 × 17 dot-matrix rays. The Miller GTAW torch (ITW Inc., Glenview, IL, USA) keeps stationary to easily obtain a complete weld pool surface such that the accuracy and the robustness of the LSTM-based measurement method could be tested by comparing with the reconstructed weld pool. The 304 L stainless steel is chosen as the base metal. Because the GTAW torch is not moving in this study, their is no need to use filler metal. The welding current is 70 A, the welding voltage is 10.5 V, and the flow rate of the Ar shielding gas is 10 L/min.

A reconstructed weld pool surface calculated by the LSTM-based measurement method is shown in Figure 2a. The actual cross-section of this weld is also shown in Figure 2b; the width of its X and Y directions are 4.94 mm and 5.07 mm. The comparison of these two boundaries is presented in Figure 2c. Since the difference between the actual weld width in the X direction and Y direction is very small, it can be considered reasonably that the actual weld pool boundary is a circle with a diameter equal to 5 mm. In order to represent, in detail, the difference in different directions, the boundary of the weld pool is divided into four parts: upper, lower, left, and right parts according to the four reflection points located at the boundary of the rows of the dot-matrix.

The boundary error of the weld pool is defined as:

$$b^i=\left|r_m-r_p^i\right|,\hspace{1em}i=1,\dots ,n$$

(1)

where $r_m=2.5 \mathrm{mm}$, which is the approximate radius of the actual weld pool; $r_p^i$ is the distance from the sampling point to the center of the reconstructed weld pool surface, which is $\left(0,0\right)$. A total of $3600$ points are sampled from the boundary of the reconstructed weld pool surface in polar coordinates at equal angles ($2\times \pi ÷ 3600$), and $n$ is the number of sampling points on the upper, lower, left, and right boundaries.

The average boundary error and maximum boundary error are defined as:

$$b_{mean}=\frac{1}{n}{\sum }_{i=1}^n{b}^i,\hspace{1em}i=1,\dots ,n$$

(2)

$$b_{max}=max\left(b^i\right),\hspace{1em}i=1,\dots ,n$$

(3)

The relative average boundary error and relative maximum boundary error are defined as:

$$b_{mean}^r=\frac{b_{mean}}{r_m}\times 100\%$$

(4)

$$b_{max}^r=\frac{b_{max}}{r_m}\times 100\%$$

(5)

Error statistics of the upper, lower, left, and right boundaries of the reconstructed weld pool surface are shown in

Table 1. Error statistics of the upper, lower, left, and right boundaries of the reconstructed weld pool surface.

Loction	${\mathit{b}}_{\mathit{m}\mathit{e}\mathit{a}\mathit{n}}$ $\left(\mathbf{m}\mathbf{m}\right)$	${\mathit{b}}_{\mathit{m}\mathit{e}\mathit{a}\mathit{n}}^{\mathit{r}}$ $\left(\mathbf{\%}\right)$	${\mathit{b}}_{\mathit{m}\mathit{a}\mathit{x}}$ $\left(\mathbf{m}\mathbf{m}\right)$	${\mathit{b}}_{\mathit{m}\mathit{a}\mathit{x}}^{\mathit{r}}$ $\left(\mathbf{\%}\right)$
Upper boundary	0.11	4.4	0.31	12.4
Lower boundary	0.11	4.4	0.27	10.8
Left boundary	0.13	5.2	0.25	10.0
Right boundary	0.32	12.8	0.58	23.2

. It can be found that the errors of left and right boundaries are large, especially that the error of the right boundary is very significant. However, the errors of the upper and lower boundaries are relatively small.

Table 2. The weld width comparison between the reconstructed weld pool surface and the measured weld.

${\mathit{w}}_{\mathit{r}}^{\mathit{x}}$ $\left(\mathbf{m}\mathbf{m}\right)$	${\mathit{w}}_{\mathit{p}}^{\mathit{x}}$ $\left(\mathbf{m}\mathbf{m}\right)$	${\mathit{w}}_{\mathit{e}\mathit{r}\mathit{r}}^{\mathit{x}}$ $\left(\mathbf{\%}\right)$	${\mathit{w}}_{\mathit{r}}^{\mathit{y}}$ $\left(\mathbf{m}\mathbf{m}\right)$	${\mathit{w}}_{\mathit{p}}^{\mathit{y}}$ $\left(\mathbf{m}\mathbf{m}\right)$	${\mathit{w}}_{\mathit{e}\mathit{r}\mathit{r}}^{\mathit{y}}$ $\left(\mathbf{\%}\right)$
4.94	4.28	13.36	5.07	4.96	2.17

shows the weld width comparison between the reconstructed weld pool surface and the measured weld. It can be seen from the table that the weld width error $\left(w_{err}^y=\frac{\left|w_r^y-w_p^y\right|}{w_r^y}\right)$ of the reconstructed weld pool surface in the Y direction is small, while the error $w_{err}^x$ in the X direction is large, which is consistent with the above law of the boundary errors.

The reason for the large error in X direction and small error in Y direction of the reconstructed weld pool surface is mainly related to the oblique incidence of the dot-matrix laser, which is the inherent characteristic of this structured light-sensing system. The rectangular dot-matrix laser is obviously elongated along the incident direction (X direction), while the change along its vertical direction (Y direction) is small. The dot-matrix laser finally projected on the substrate surface is trapezoidal, as shown in Figure 3.

The elongation of the dot-matrix laser along the X direction will lead to the increase of the line spacing of laser spots, especially on the right side. In the Y direction, there are little changes, so that the laser spots can still keep small column spacing. The detailed difference can be seen in Figure 4. In Figure 4a, it is clear that the outermost row of the incident ray is far away from the boundary of the weld pool that will cause a big error. While in the Y direction, the incident laser lines have small spacing and keep high density. The outermost incident rays are close to the boundary of the weld pool such that the errors are small, as shown in Figure 4b.

3. Optimization Method of Boundary Extension

The 3D weld pool surface obtained from the direct interpolation of the reflection points is non-holonomic, as shown in Figure 2a, and then the boundary of the reconstructed surface is actually not matched with the actual boundary of the weld pool, as shown in Figure 2c. To obtain a complete reconstruction weld pool surface, it is necessary to develop the optimization method of boundary extension.

Zhang [7] has done some research before to realize the boundary extension of the interpolated weld pool surface by fitting it with an analytic model, as shown in Figure 5. To guarantee the accuracy of the fit boundary, the reflection points at the ends of rows and their most adjacent reflection points reflected from the weld pool surface are both used as fit points. The reconstructed weld pool boundary is obtained by fitting with the model in Figure 5a and across through boundary points in Figure 5b. However, this fit method does not consider the curvature variation of the weld pool surface near the boundary, so the accuracy of this fit method is not very satisfactory. In this section, an optimization method of boundary extension of the interpolated weld pool surface is proposed firstly, based on the variation tendency of the curved surface near the boundary.

The schematic diagram of this method is shown in Figure 6. An interpolation is conducted at first based on the X and Y coordinates of reflection points (dots in red) to obtain the X and Y coordinates of interpolation points (dots in black) in the coverage area (aera in yellow), as shown in Figure 6a. Then a local quadratic polynomial fitting is carried out on the Z coordinate of the interpolation points and reflection points in each row or column respectively; in addition, many more interpolation points (Z > 0) can be created even beyond the original yellow area by using the new-earned fitting equation, and a epitaxial surface (area in red) can be achieved, as shown in Figure 6b. A detailed local quadratic polynomial fitting is shown in Figure 6c; it can be seen that the red points are the original reflection points, and the black points are the interpolation points obtained by using the X and Y coordinates of reflection points. The blue points, which epitaxial the original red points, are achieved by using the new-earned fitting equation. The green points are also the extended points, but these points are abandoned because the Z coordinates are less than 0.

Finally, the extended weld pool surface can be obtained by drawing the surface directly through the updated grid data of the interpolation points. In this method, the interpolation points in each row and column are all fitted with a local quadratic polynomial. The extended weld pool surface with high fineness can be obtained by increasing the density of the grids.

Figure 7 shows the extended weld pool surface of Figure 1 optimized by the boundary extension method. The boundary comparison between the measured weld and the reconstructed weld pool surface after boundary optimization is shown in Figure 7b.

The boundary of the weld pool is still divided into four parts according to the four reflection points in Figure 2: the upper, lower, left, and right boundaries. The error statistics of these four boundaries are shown in

Table 3. The error statistics of the four boundaries.

Loction	${\mathit{b}}_{\mathit{m}\mathit{e}\mathit{a}\mathit{n}}$ $\left(\mathbf{m}\mathbf{m}\right)$	${\mathit{b}}_{\mathit{m}\mathit{e}\mathit{a}\mathit{n}}^{\mathit{r}}$ $\left(\mathbf{\%}\right)$	${\mathit{b}}_{\mathit{m}\mathit{a}\mathit{x}}$ $\left(\mathbf{m}\mathbf{m}\right)$	${\mathit{b}}_{\mathit{m}\mathit{a}\mathit{x}}^{\mathit{r}}$ $\left(\mathbf{\%}\right)$
Upper boundary	0.22	8.8	0.35	14.0
Lower boundary	0.32	12.8	0.50	20.0
Left boundary	0.19	7.6	0.54	21.6
Right boundary	0.10	4.0	0.31	12.4

. It can be observed that the b mean of the right boundary is obviously reduced, while the errors of the upper, lower, and left boundaries are increased.

The comparison between the weld width of the measured weld and the reconstructed weld pool surface after boundary optimization is shown in

Table 4. The comparison between the weld width of the measured weld and the reconstructed weld pool surface.

${\mathit{w}}_{\mathit{r}}^{\mathit{x}}$ $\left(\mathbf{m}\mathbf{m}\right)$	${\mathit{w}}_{\mathit{p}}^{\mathit{x}}$ $\left(\mathbf{m}\mathbf{m}\right)$	${\mathit{w}}_{\mathit{e}\mathit{r}\mathit{r}}^{\mathit{x}}$ $\left(\mathbf{\%}\right)$	${\mathit{w}}_{\mathit{r}}^{\mathit{y}}$ $\left(\mathbf{m}\mathbf{m}\right)$	${\mathit{w}}_{\mathit{p}}^{\mathit{y}}$ $\left(\mathbf{m}\mathbf{m}\right)$	${\mathit{w}}_{\mathit{e}\mathit{r}\mathit{r}}^{\mathit{y}}$ $\left(\mathbf{\%}\right)$
4.94	5.06	2.43	5.07	5.51	8.68

. It can be seen that the width error of the reconstructed weld pool surface in the X direction $w_{err}^x$ is obviously reduced from 13.36% to 2.43%, while the width error in the Y direction $w_{err}^y$ is increased from 2.17% to 8.68%.

The reconstructed weld pool surface after boundary optimization is more comparable with the measured weld, in that the $w_{err}^x$ and the $w_{err}^y$ both decreased below 10%. In the next section, the phenomena of “missing points” and “more points”, which often occur in the actual process, are discussed and detailed to analyze the robustness of the LSTM-based model.

4. Phenomena of “Missing Points” and “More Points”

The phenomena of “missing points” and “more points” encounter is an actual process that may affect the accuracy of this LSTM-based measurement. For example, the projected laser may scatter by metal vapor, which will increase the size of the imaging points and decrease the contrast. When two large-sized imaging points are crossed, they may be identified as a single point during the image processing. If the contrast of an image point is too low, it may be directly lost in the image processing, as shown in Figure 8.

If floating oxides are formed on the weld pool surface, the laser rays projected on the oxides will be diffusely reflected instead of specularly reflected, and the corresponding reflection points will be lost. Figure 9 shows three reflection images and their image processing results taken continuously in the 70 A current test (the frame rate of the camera is 26 Hz, and the first frame corresponds to the time of 15.46 s after arcing). It can be observed that the floating oxides usually drift on the weld pool surface at a certain position from the Figure 9a. In actual sensing, it is found that the number of missing concentrated points is generally less than 4, that is, floating oxides with a large area are rarely produced, as shown in Figure 9b.

If the brightness of some particular locations is too high, the image-processing algorithm may mistakenly identify them as imaging points, which will cause the phenomenon of more points, as shown in Figure 10. This phenomenon is obvious when the size of the weld pool is small.

In addition, the imaging points may be split into two points when scattered by metal vapor or interrupted by floating oxides with small sizes. If two highlighted pixel blocks are extracted in image processing, it will cause the phenomenon of more points, as shown in Figure 11. However, this situation occurs rarely in the actual process.

5. Robustness Analysis of the LSTM-Based Model

The phenomena of missing points and more points have great influence on the reconstruction of weld pool surface for the analytic reconstruction algorithms [6], which need to ensure the definite correspondence between the projected points and the reflection points. Although the proposed LSTM-based measurement method avoids the complicated procedure of point recognition, it is of great significance to analyze the influence of missing points and more points on the LSTM-based model after boundary extension by the proposed optimization method. In this paper, the phenomenon of missing points is divided into missing scattered points and missing concentrated points corresponding to Figure 8 and Figure 9 respectively. For the phenomena of more points, the extra points that scattered randomly are mainly considered.

A reflection image reflected by a simulation surface (created through MATLAB, version 2013a, Mathworks, Natick, MA, USA), which contained 140 reflection points, is taken as an example; the reflection image is shown in Figure 12a and the reconstructed surface (after boundary extension by the proposed optimization method) of the sample without missing points and more points is shown in Figure 12b. The simulated surface height $h_r$, radius of curvature $R_r$, weld width $w_r^x$ in the X direction, and weld width $w_r^y$ in the Y direction are shown in

Table 5. Target surface parameters of the sample.

${\mathit{h}}_{\mathit{r}} \left(\mathbf{m}\mathbf{m}\right)$	${\mathit{R}}_{\mathit{r}} \left(\mathbf{m}\mathbf{m}\right)$	${\mathit{w}}_{\mathit{r} }^{\mathit{x}} \left(\mathbf{m}\mathbf{m}\right)$	${\mathit{w}}_{\mathit{r}}^{\mathit{y}} \left(\mathbf{m}\mathbf{m}\right)$
0.3320	29	8.75	8.75

.

The characteristics of the weld pool surface reconstructed by the LSTM-based model are shown in

Table 6. Reconstructed surface parameters of samples without missing points and more points.

${\mathit{h}}_{\mathit{p}}$ $\left(\mathbf{m}\mathbf{m}\right)$	${\mathit{h}}_{\mathit{e}\mathit{r}\mathit{r}}$ $\left(\mathbf{\%}\right)$	${\mathit{R}}_{\mathit{p}}$ $\left(\mathbf{m}\mathbf{m}\right)$	${\mathit{R}}_{\mathit{e}\mathit{r}\mathit{r}}$ $\left(\mathbf{\%}\right)$	${\mathit{w}}_{\mathit{p}}^{\mathit{x}}$ $\left(\mathbf{m}\mathbf{m}\right)$	${\mathit{w}}_{\mathit{e}\mathit{r}\mathit{r}}^{\mathit{x}}$ $\left(\mathbf{\%}\right)$	${\mathit{w}}_{\mathit{p}}^{\mathit{y}}$ $\left(\mathbf{m}\mathbf{m}\right)$	${\mathit{w}}_{\mathit{e}\mathit{r}\mathit{r}}^{\mathit{y}}$ $\left(\mathbf{\%}\right)$
0.3342	0.66	28.27	2.52	8.83	0.91	8.84	1.03

, including the surface height $h_p$; the radius of curvature $R_p$; the width $w_p^x$ in the X direction; the width $w_p^y$ in the Y direction; and their relative errors $h_{err}$, $R_{err}$, $w_{err}^x$ and $w_{err}^y$. It can be observed that the LSTM-based model has quite high accuracy in the case of no missing points and more points.

Then three samples with the phenomena of missing points and more points, such as scattered missing 15% points, concentrated missing 15 points (about 10%), and random adding 15 points (about 10%) are used to test the robustness of this LSTM-based model. The samples and their reconstructed surfaces (after boundary extension by the proposed optimization method) are also shown in Figure 13. The points in blue represent the missing/adding points.

The detailed reconstructed surface characteristics of the three samples are shown in

Table 7. The detailed reconstructed surface characteristics of the three examples.

Samples	${\mathit{h}}_{\mathit{p}}$ $\left(\mathbf{m}\mathbf{m}\right)$	${\mathit{h}}_{\mathit{e}\mathit{r}\mathit{r}}$ $\left(\mathbf{\%}\right)$	${\mathbf{R}}_{\mathbf{p}}$ $\left(\mathbf{m}\mathbf{m}\right)$	${\mathit{R}}_{\mathit{e}\mathit{r}\mathit{r}}$ $\left(\mathbf{\%}\right)$	${\mathit{w}}_{\mathit{p}}^{\mathit{x}}$ $\left(\mathbf{m}\mathbf{m}\right)$	${\mathit{w}}_{\mathit{e}\mathit{r}\mathit{r}}^{\mathit{x}}$ $\left(\mathbf{\%}\right)$	${\mathit{w}}_{\mathit{p}}^{\mathit{y}}$ $\left(\mathbf{m}\mathbf{m}\right)$	${\mathit{w}}_{\mathit{e}\mathit{r}\mathit{r}}^{\mathit{y}}$ $\left(\mathbf{\%}\right)$
Scattered missing 15% points	0.3302	0.54	26.81	7.55	8.15	6.86	8.45	3.43
Concentrated missing 15 points	0.3342	0.66	27.01	6.86	8.83	0.91	8.46	3.31
Random adding 15 points	0.3518	5.96	28.82	0.62	9.41	7.54	9.47	8.23

. It can be observed that the $h_{err}$ are 0.54% and 0.66% for the scattered missing 15% points and concentrated missing 15 points, which have no difference with the ideal result (0.66%) in Table 6. However, the $h_{err}$ of the random adding 15 points is increased to 5.96%. The $R_{err}$ are 7.55% and 6.86% for the scattered missing 15% points and concentrated missing 15 points, which are obviously increased according to the ideal result (2.52%), while the $R_{err}$ is 0.62% for the random adding 15 points. This is because the LSTM-based model has been well trained to reconstruct the curved weld pool surface when the reflection points are enough; the $R_{err}$ will increase when the reflection points are lacking. The $w_{err}^x$ and the $w_{err}^y$ are both increased for the three samples with the increase of the $h_{err}$ and the $R_{err}$. However, all the errors below 10% prove that the LSTM-based model has an acceptable robustness.

Figure 14 shows the reconstruction results of missing/adding the first few imaging points, including missing the 2nd and 3rd points, missing the 5th, 7th, 8th, and 10th points, and one more point at the beginning, which corresponds to missing scattered points, missing concentrated points, and more points randomly. It can be observed intuitively from the reconstructed surface that the phenomenon of missing/adding the first few imaging points will have a great influence on the reconstruction results. The reason for this phenomenon is that the accumulated information of historical state of the LSTM-based model is less at the beginning, the model is still in the exploration stage, and the characteristics of the sample have not yet been mastered.

The parameters of the reconstructed surface of the three examples are shown in

Table 8. The parameters of the predicted surface of the three examples.

Samples	${\mathit{h}}_{\mathit{p}}$ $\left(\mathbf{m}\mathbf{m}\right)$	${\mathit{h}}_{\mathit{e}\mathit{r}\mathit{r}}$ $\left(\mathbf{\%}\right)$	${\mathit{R}}_{\mathit{p}}$ $\left(\mathbf{m}\mathbf{m}\right)$	${\mathit{R}}_{\mathit{e}\mathit{r}\mathit{r}}$ $\left(\mathbf{\%}\right)$	${\mathbf{w}}_{\mathbf{p}}^{\mathbf{x}}$ $\left(\mathbf{m}\mathbf{m}\right)$	${\mathit{w}}_{\mathit{e}\mathit{r}\mathit{r}}^{\mathit{x}}$ $\left(\mathbf{\%}\right)$	${\mathit{w}}_{\mathit{p}}^{\mathit{y}}$ $\left(\mathbf{m}\mathbf{m}\right)$	${\mathit{w}}_{\mathit{e}\mathit{r}\mathit{r}}^{\mathit{y}}$ $\left(\mathbf{\%}\right)$
Missing the 2nd, 3rd points	0.3234	2.59	19.09	34.17	8.64	1.26	8.41	3.89
Missing the 5th, 7th, 8th, 10th points	0.3260	1.81	14.43	50.24	8.51	2.74	11.61	32.69
One more point at the beginning	0.3352	0.96	17.49	39.69	8.83	0.91	8.72	0.34

. It can be observed that the $R_p$ of the reconstructed surface by using the LSTM-based model is significantly worsened, which is directly related to the large errors on the lower left side of the reconstructed surface, as shown in Figure 14b,d,f.

6. Forward-Reverse United Reconstruction Optimization Method

A forward-reverse united reconstruction optimization method is designed to increase the accuracy of the reconstructed weld pool surface. The formal LSTM-based model is trained following the order from left to right and from the top to bottom, which is consistent with the natural order of the image-processed 2D points obtained in practice [9], as shown in Figure 15a. In this forward-reverse united reconstruction optimization method, another LSTM model is also trained with the image-processed points with a total reverse order, as shown in Figure 15b. Because the LSTM model can obtain an accurate reconstruction especially, except for the first few points, we proposed to adopt the second halves of the output reflection points from both LSTM models to unite a complete sample. Then through surface interpolation and boundary extension, the well-reconstructed weld pool surface can be obtained.

Figure 16 shows the forward-reverse united optimization results of the three above mentioned cases: missing the 2nd and 3rd points, missing the 5th, 7th, 8th, and 10th points, and one more point at the beginning. It is obviously that the reconstructed surface on the lower left side is obviously improved.

The parameters of the reconstructed surfaces of the three samples are shown in

Table 9. The predicted surface parameters of the sample in the three cases.

Samples	${\mathit{h}}_{\mathit{p}}$ $\left(\mathbf{m}\mathbf{m}\right)$	${\mathit{h}}_{\mathit{e}\mathit{r}\mathit{r}}$ $\left(\mathbf{\%}\right)$	${\mathbf{R}}_{\mathbf{p}}$ $\left(\mathbf{m}\mathbf{m}\right)$	${\mathit{R}}_{\mathit{e}\mathit{r}\mathit{r}}$ $\left(\mathbf{\%}\right)$	${\mathit{w}}_{\mathit{p}}^{\mathit{x}}$ $\left(\mathbf{m}\mathbf{m}\right)$	${\mathit{w}}_{\mathit{e}\mathit{r}\mathit{r}}^{\mathit{x}}$ $\left(\mathbf{\%}\right)$	${\mathit{w}}_{\mathit{p}}^{\mathit{y}}$ $\left(\mathbf{m}\mathbf{m}\right)$	${\mathit{w}}_{\mathit{e}\mathit{r}\mathit{r}}^{\mathit{y}}$ $\left(\mathbf{\%}\right)$
Missing the 2nd and 3rd points	0.3341	0.63	27.76	4.28	8.76	0.11	9.17	4.80
missing the 5th, 7th, 8th, 10th points	0.3341	0.63	26.79	7.62	8.77	0.23	9.17	4.80
one more point at the beginning	0.3352	0.96	27.14	6.41	8.79	0.46	9.13	4.34

. It can be observed that the accuracy of the weld pool surface increased clearly. The $R_{err}$ of all three samples decreased to below 10% and the other errors such as the $h_{err}$, the $w_{err}^x$, and the $w_{err}^y$ are both below 5%.

Figure 17a is the weld pool surface of Figure 2a reconstructed after the boundary optimization and the forward-reverse united reconstruction optimization method. The boundary comparison between the measured weld and the reconstructed weld pool surface output by the optimized LSTM model is shown in Figure 17b. For convenience of comparison, the boundary of the weld pool is still divided into four parts: upper, lower, left, and right parts according to the four reflection points as in Figure 2. The error statistics of each part of the boundary are shown in

Table 10. The error statistics of each part of the boundary.

Loctions	${\mathit{b}}_{\mathit{m}\mathit{e}\mathit{a}\mathit{n}}$ $\left(\mathbf{m}\mathbf{m}\right)$	${\mathit{b}}_{\mathit{m}\mathit{e}\mathit{a}\mathit{n}}^{\mathit{r}}$ $\left(\mathbf{\%}\right)$	${\mathit{b}}_{\mathit{m}\mathit{a}\mathit{x}}$ $\left(\mathbf{m}\mathbf{m}\right)$	${\mathit{b}}_{\mathit{m}\mathit{a}\mathit{x}}^{\mathit{r}}$ $\left(\mathbf{\%}\right)$
Upper boundary	0.20	8.0	0.32	12.8
Lower boundary	0.13	5.2	0.23	9.2
Left boundary	0.14	5.6	0.26	10.4
Right boundary	0.09	3.6	0.31	12.4

. It can be observed that the errors of the four parts of the weld pool boundary are controlled in a small range after optimization, especially the error of the lower boundary is obviously reduced.

The comparison between the weld width of the measured weld and the weld width of the reconstructed weld pool surface after optimization is shown in

Table 11. The comparison between the weld width of the measured weld and the weld width of the recon-structed weld pool surface after robustness optimization.

${\mathit{w}}_{\mathit{r}}^{\mathit{x}}$ $\left(\mathbf{m}\mathbf{m}\right)$	${\mathit{w}}_{\mathit{p}}^{\mathit{x}}$ $\left(\mathbf{m}\mathbf{m}\right)$	${\mathit{w}}_{\mathit{e}\mathit{r}\mathit{r}}^{\mathit{x}}$ $\left(\mathbf{\%}\right)$	${\mathit{w}}_{\mathit{r}}^{\mathit{y}}$ $\left(\mathbf{m}\mathbf{m}\right)$	${\mathit{w}}_{\mathit{p}}^{\mathit{y}}$ $\left(\mathbf{m}\mathbf{m}\right)$	${\mathit{w}}_{\mathit{e}\mathit{r}\mathit{r}}^{\mathit{y}}$ $\left(\mathbf{\%}\right)$
4.94	5.02	1.62	5.07	5.27	3.94

. It can be observed that the forward-reverse united reconstruction optimization method has obviously improved the accuracy of the weld pool boundary.

7. Conclusions

In this paper, a boundary extension method was designed first to optimize the boundary profile of the interpolated weld pool surface calculated by a LSTM-based weld pool measurement method specialized for the structured light-sensing system. Then the phenomea of missing points and more points, which are observed in physical welding, were analyzed. The robustness of the LSTM-based method to the phenomena was also studied. To solve the problem caused by the missing/adding the first few imaging points, a forward-reverse united reconstruction optimization method was designed at last. The following are concluded.

(1)

A boundary extension method is proposed to realize the boundary optimization of the weld pool effectively. After boundary extension, the errors of the left and right part of the boundary of the weld pool are obviously reduced.

(2)

The LSTM-based method has good robustness in the cases of missing scattered points (no more than 15%), missing concentrated points (no more than 10%), and more points randomly (no more than 10%), but shows certain inaccuracy in the phenomenon of missing/adding first few imaging points.

(3)

The designed forward-reverse united reconstruction optimization method effectively overcomes the inaccurate prediction of the missing/adding first few imaging points. After optimization by this method, the boundary error of the reconstructed weld pool surface is obviously reduced. The weld width error is reduced to 1.62% in the X direction and 3.94% in the Y direction.