Adjacent Frame Difference with Dynamic Threshold Method in Underwater Flash Imaging LiDAR

Abstract

During the underwater LiDAR imaging process, the images achieved by the conventional constant threshold adjacent frame difference (AFD) method normally loses the distance information of targets. This is mainly due to the Gaussian distribution of the laser light intensity field, which leads to the inhomogeneous intensity distribution in the frame from the target acquired by intensity charge-coupled devices (ICCD). In order to overcome this issue, the novel dynamic threshold adjacent frame difference (DTAFD) method was proposed in this paper. The DTAFD method modifies the intensity threshold following the pixel intensities in the different parts of the single frame intensity image acquired by ICCD. After the detailed theoretical demonstration of the DTAFD method, with the purpose of verifying its feasibility, the self-developed range-gated flash imaging LiDAR has been employed to construct the distance images of the rectangular and circular shaped targets at different distances. The distance between the rectangular target and the LiDAR system is 25.7 m, and the circular target is 70 cm further away from the rectangular target. The full distance information of these two targets is obtained by the DTAFD method with an effectively suppressing noise and the PSNR is increased from $6.95\pm 0.0426$ dB to $7.62\pm 0.0264$ dB. The experimental results indicate that the DTAFD method efficiently solves the AFD method’s drawback on the target information loss caused by the unequal optical field distribution, which makes it more suitable for the scenarios with uneven laser distribution such as the underwater imaging environment.

1. Introduction

With the growing demands for ocean resource developments, underwater imaging systems are gaining more and more attention [1,2,3,4]. Due to its fast-imaging speed, high resolution, simplified scanning system and suppression on the back-scattering interference during the laser transmission, the underwater range-gated flash imaging LiDAR have become widely used in the field of underwater imaging [5,6,7]. However, since the flash imaging LiDAR employs an area charge-coupled device (area-CCD) as the receiver in the system [8], a fast and accurate 3D reconstruction algorithm is particularly important for the range-gated flash imaging LiDAR to accurately acquire target distance information and display it in real time.

The adjacent frame difference (AFD) method is a new three dimensional (3D) reconstruction algorithm based on the range-gated method [9]. In this method, the distance information of the target is obtained by thresholding the intensity difference between two adjacent frames achieved by the intensified-charge-coupled device (ICCD). Compared with the trapezoid-range-intensity profile [10], triangular-range-intensity profile methods [11] and the centroid method [12], the availability of the AFD method does not rely on the strict correlation between the range-gate and laser pulse widths. Meanwhile, it also presents many advantages such as fast imaging speed, large imaging range, and simple data processing.

However, an inhomogeneous light intensity field is distributed on the target surface during the underwater imaging process due to the Gaussian distribution of the laser light intensity field, which leads to the loss of the distance information of the target when the AFD method based on the constant threshold is applied for 3D image reconstruction. Although this problem can be solved by using a flat top beam shaper [13,14,15,16] or by further expanding the beam. The expensive flat beam shaper will significantly add the cost of the LiDAR system. If the laser beam is further expanded, the detection range of the LiDAR will be significantly reduced.

In order to overcome this issue, a novel dynamic threshold adjacent frame difference (DTAFD) method has been presented in this paper. In the DTAFD method, the intensity threshold is modified based on the pixel intensities in the different parts of the single frame intensity image. This method is further implemented in the underwater imaging using our self-developed underwater flash LiDAR imaging system based on the range-gated technology.The experimental results show that, using this DTAFD method, the 3D information of the targets can be obtained quickly and accurately. It indicates that the DTAFD method can efficiently solve the AFD method’s drawback on the target information loss caused by the unequal optical field distribution. So the DTAFD method is more suitable for the applications in the complex working environment with an uneven light intensity field.

2. Dynamic Threshold Adjacent Frame Difference

The AFD method based on a constant threshold has been precisely explained in our previous work [9]. The block diagrams of this method are demonstrated in Figure 1. The judge equation for determining the validity of a pixel in the AFD method is:

$$\left\{\begin{array}{cc}\Delta I_{(m,n)}^k>{\delta }^{+}\hfill & (\mathrm{Front}\mathrm{edge})\hfill \\ \Delta I_{(i,j)}^k<{\delta }^{-}\hfill & (\mathrm{Back}\mathrm{edge}).\hfill \end{array}\right.$$

(1)

Normally, the constant threshold will be set as one fifth of the max pixel intensity value, and $\left|{\delta }^{+}\right|=\left|{\delta }^{-}\right|$ [9]. As shown in Figure 1, due to the constant thresholds ${\delta }^{+}$ and ${\delta }^{-}$, which are highlight by the red color, when the light distribution of the original intensity image acquired by ICCD is uneven, the proper threshold value becomes difficult to choose. The distance information of the target will lose entirely with a high threshold value. On the other side, the poor peak signal-to-noise ratio (PSNR) will also have negative impacts on the target distance information if the threshold is too low.

With the purpose of overcoming these issues, a novel dynamic threshold adjacent frame difference (DTAFD) method has been proposed in this paper. In this method, the threshold value is no longer a constant value, but modified following the pixel intensities in the different parts of the single frame intensity image acquired by ICCD. Briefly speaking, the threshold value is bigger in regions where the light intensity field is strong and smaller at locations where the light intensity field is weak. This method not only enables the effective noises suppression but also guarantees the acquisition of complete distance information of the target.

The block diagrams of the DTAFD method are shown in Figure 1, the global constant threshold is changed to a dynamic threshold that varies with the pixel intensity value of the original intensity image. For the front edge imaging, the condition for determining the validity of the pixel is:

$$\Delta I_{(m,n)}^k>\epsilon \times I_k^{(m,n)},$$

(2)

where $\Delta I_{(m,n)}^k$ is the value of the ${(m,n)}^{}$th pixel in the $k^{}$th intensity difference image of the two adjacent frames, $I_{(m,n)}^k$ is the value at ${(m,n)}^{}$th pixel in the previous intensity image of the two adjacent frames, $\epsilon$ is the threshold percentage. So, the distance value of the corresponding element $D_{f-D}^{(m,n)}$ in the distance matrix $D_{f-D}$ is:

$$D_{f-D}^{(m,n)}=\frac{c[\Delta \tau +k\Delta {\tau }_{step}]}{2},$$

(3)

where c is the light speed, $\Delta \tau$ is the initial delay time, k is the frame number, $\Delta {\tau }_{step}$ is the delay step. $\Delta \tau +k\Delta {\tau }_{step}$ is the delay time corresponding to the $k^{}$th frame intensity image. If $0<\Delta I_{(m,n)}^k<\epsilon \times I_k^{(m,n)}$, then the ${(m,n)}^{}$th pixel is invalid, the value of the ${(m,n)}^{}$th element in corresponding distance matrix $D_{f-D}$ will be set as infinity.

At the back edge imaging, the condition change as:

$$\Delta I_{(i,j)}^k<-\epsilon \times I_{k+1}^{(j,k)},$$

(4)

where $\Delta I_{(i,j)}^k$ is the value of the ${(i,j)}^{}$th pixel in the $k^{}$th intensity difference image of the two adjacent frames, $I_{(i,j)}^{k+1}$ is the value at ${(i,j)}^{}$th pixel in the latter intensity image of the two adjacent frames, $\epsilon$ is the threshold percentage, to determine the threshold value at different pixels. So, the distance value of the corresponding element $D_{b-D}^{(i,j)}$ in the distance matrix $D_{b-D}$ is:

$$D_{b-D}^{(i,j)}=\frac{c[\Delta \tau +k\Delta {\tau }_{step}]}{2}.$$

(5)

If $-\epsilon \times I_k^{(i,j)}<\Delta I_{(i,j)}^k<0$, then the ${(i,j)}^{}$th pixel is invalid, the value of the ${(i,j)}^{}$th element in the corresponding distance matrix $D_{b-D}$ will be set as infinity.

3. Experiment and Results

The experiment was carried out using the self-developed underwater imaging flash LiDAR system. The optical image of this system is shown in Figure 2. A Pilot-3 air-cooled pulsed laser with a center wavelength at 532 nm is used as the irradiation source with the 5 ns pulse width and the 1 kHz pulse repeat frequency, respectively. The averaged energy per pulse is 2 mJ. The laser divergence angle is adjusted by a collimation system. The size of the ICCD is $1292\times 964$, the bit-depth of the ICCD is 8. When the laser emits a pulse laser, the trigger delay system starts timing, and the delay time and range-gate width are set by the time delay controller to generate a precisely delayed trigger signal. This range-gate signal is further used for triggering the ICCD to realize the selection of the underwater imaging area, and to acquire the laser signal reflected by the target.

Two flat panels were used as targets for the measurements, as shown in Figure 3. The first target is a rectangle that is a $40\mathrm{cm}$ long and $30\mathrm{cm}$ width, which is 25.7 m away from the LiDAR system. The second target is circular shaped with a diameter of 40 cm, which is 70 cm further away from the first target. The angle between the rectangular target plane and the laser transmission direction is $41.4^{\circ }$. In the experiment, the used range-gate width is 20 ns and the delay step is 1 ns.

During the underwater imaging process, the intensity distribution of the images acquired by ICCD is not uniform due to the uneven distribution of the laser light intensity field irradiated on the target surface. Figure 4a–f present the intensity images acquired by ICCD at different delay time, respectively. From these images, it can be found that this nonuniform intensity phenomenon is more obvious for the front rectangular target, which has a higher echo signal intensity on the right side compared with the one on the left side. To performing the quantitative analysis of the shortcomings of the AFD method, the averaged intensity difference between the darker area (labeled as A in Figure 4) and brighter area (labeled as B in Figure 4) of the front target will be compared in the following part. The averaged intensity of area A or B is obtained by the following equation:

$$I_{\mathrm{ave}}=\frac{\sum I_i}{N},$$

(6)

where $I_{\mathrm{ave}}$ is the averaged intensity, $I_i$ is the intensity of each pixel at region A or B, N is the number of pixels in the region of A or B. Meanwhile, since the results from the front and back edges of the range-gated imaging are based on the same theory, the front edge of the range-gate imaging as the examples for further discussions.

As shown in Figure 5, the blue and green lines represent the grey value of areas A and B in the intensity difference image at different delay times over five measurements, respectively. It would be found that the averaged grey value between areas A and B exhibits significant fluctuations at different delay times, which is presented by the black curve in the same figure. This fluctuation of the curve makes it difficult to determine the constant threshold for the conventional AFD method, which is further used to determine the validity of the pixels. Thus, using the constant threshold AFD method, it is difficult to reconstruct the images including all pixels in areas A and B with a high PSNR.

The averaged PSNR of the distance information obtained by the constant threshold AFD method in five measurements is shown in Figure 6. As shown in Figure 6, the PSNR increases gradually as the constant threshold is varied from 3 to 7. As the constant threshold changes from 7 to 10, the PSNR decreases. The maximum PSNR is $6.95\pm 0.0429$ dB when the constant threshold value is selected as 7. If the constant threshold is set to a lower value (such as the threshold is 4 in Figure 6), although the rectangular target is relatively intact, the noises are more obvious. The missing parts of the rectangular target will further increase if the threshold is set to a higher value (such as the threshold is 10 in Figure 6). These results indicate that, using the conventional constant threshold AFD method, it is difficult to acquire the entire target distance information with a high PSNR.

The DTAFD method is considered as the effective solution to this issue by setting a proper $\epsilon$. The averaged PSNR curve of the distance information obtained by the DTAFD method with different $\epsilon$ in five measurements is shown in Figure 7. The PSNR increases gradually as the $\epsilon$ is varied from 1 to 15%. As the $\epsilon$ changes from 15 to 35%, the PSNR decreases. The maximum PSNR is $7.65\pm 0.0264$ dB when the $\epsilon$ value is set as $15\%$. So, if the $\epsilon$ is set too small, although the complete distance information of the target can be obtained, there are more noises (as shown in Figure 7 with $\epsilon =10\%$). If $\epsilon$ is set too large, the number of missing pixels gradually increases. Compared to the highest PSNR achieved by the constant threshold AFD method, the PSNR increased $10\%$ by the DTAFD method. This experimental result demonstrates that the main issues during the 3D image reconstruction process caused by the constant threshold AFD method would be simply compensated for by the DTAFD method.

Figure 8 is the averaged grey value of area A (a) and B (b) in the intensity difference image at different delay times in five measurements. The corresponding $\epsilon$ of $15\%$. The blue curves in Figure 8a,b represent the averaged grey values of areas A and B in the intensity difference images at different delay times in five measurements, and the green curve shows the dynamic thresholds in the corresponding area. As shown, the dynamic threshold is less than the intensities in the rising part of the blue curves in Figure 8a,b, but greater than the intensities in the falling part of the blue curve. Therefore, according to Equation (2), all valid pixels of parts A and B can be extracted by the DTAFD method simultaneously. This is why the max PSNR of the distance information is achieved when the $\epsilon =15\%$.

4. Discussion

As shown in Figure 9, Figure 9a shows the averaged distance information achieved by the AFD method with the constant threshold of seven in five measurements. Figure 9b shows the averaged distance information achieved by the DTAFD method with the $\epsilon$ of $15\%$ in five measurements. The values of column ${646}^{}$th (the vertical symmetry axis of the distance information of the targets) pixels of Figure 9a,b are shown in Figure 9c. Figure 9d shows the values of ${482}^{}$th row (the horizontal symmetry axis of the distance information of the targets). The yellow line in Figure 9c,d represents the real position of the rectangular target, and the green line represents the real position of the circular target. As shwon in the Figure 9c, the distance information of the rectangular target and the circular target obtained by the DTAFD method are $25.71\pm 0.114$ m and $26.47\pm 0.117$ m, respectively. Compared to the ones achieved by the AFD method, which are $25.73\pm 0.118$ m and $26.53\pm 0.120$ m, respectively. Their differences are only $0.08\%$ and $0.26\%$. The rectangular target is not orthogonal to the optical axes. The rectangular target is rotated by an angle along the vertical symmetry axis, making the distance of the rectangle target smaller on the left and lager on the right. The distance difference between the left edge and right edge of the rectangular target is 30 cm. As shown in Figure 9d, the angle between the rectangular target and the laser propagation direction is about $41.4^{\circ }$. According to our previous work [9] and Figure 9d, the depth resolution is 0.15 m.

The structural similarity index (SSIM) between Figure 9a,c is 0.7845, the SSIM figure is shown in Figure 10. As shown in Figure 10, the SSIM index is smaller at the noises parts and the missing parts of the rectangular target. At the right side of the rectangular target and the circular part, the SSIM index is larger. The SSIM figure fully demonstrated that the disadvantages of the constant threshold AFD method are compensated by the DTAFD method. These results clarified that the DTAFD method can obtain complete and accurate distance information about the targets and effectively suppress the noises.

5. Conclusions

In this work, in order to overcome the issue about the distance information loss of targets from the constant threshold AFD method caused by the uneven light intensity field distribution in underwater imaging, a new DTAFD 3D reconstruction method has been proposed. Unlike the conventional AFD method with a constant threshold, this novel DTAFD method employed dynamic thresholds which varies with the pixel intensity value of the intensity image acquired by ICCD. Through the underwater imaging experiment for the rectangular and circular targets 70 cm apart at 25.7 m distance and using the self-developed range-gated imaging flash LiDAR, the qualities of 3D images reconstructed by the constant threshold AFD and DTAFD methods were compared. Compared with the image reconstructed by the AFD method, that achieved by the DTAFD method not only restored the full distance information of the target, but also presented a high PSNR. These experimental results indicate that the developed DTAFD method would effectively overcome the shortcomings of the AFD method, which makes it more appropriate for flash imaging LiDAR under the situation of uniform light distribution.