# Template-Guided Hierarchical Multi-View Registration Framework of Unordered Bridge Terrestrial Laser Scanning Data

^{1}

^{2}

^{3}

^{4}

^{*}

^{†}

## Abstract

**:**

## 1. Introduction

- (1)
- A marker-free multi-view registration framework is proposed to hierarchically align unordered bridge terrestrial laser scanning data.
- (2)
- A template-based initial pose estimation method is proposed to recover the overlaps of unordered PCD, which avoids extensive pairwise matching and improves the efficiency.
- (3)
- To group scans with high overlaps into the same block, a graph partition algorithm based on the overlaps and scanning locations is utilized to construct scan-blocks.

## 2. Research Background

#### 2.1. Registration of Bridge PCD

#### 2.2. Multi-View Registration

## 3. Methodology

#### 3.1. Template-Guided Initial Pose Estimation

#### 3.1.1. Acquisition of Side View Geometric Features

_{x}and δ

_{z}(their calculation methods will be introduced later), and then it is converted to a binary image with each grid corresponding to a pixel. If the number of points within a grid is more than one, the grey value of the corresponding pixel is set to 1; Otherwise, it is set to 0. The binary image of the PCD in Figure 2a is shown in Figure 2b.

#### 3.1.2. Unified Scale of the Template and Binary Images

_{x}and δ

_{z}are determined using the following two formulas.

_{x}and N

_{y}are the width and height of the minimum bounding box of the non-blank region in the template.

#### 3.1.3. Image Matching

#### 3.1.4. Correction of False Matching

_{i}, and its areas intersecting with all the other scans are calculated based on the locations obtained in the template matching step. The n scans with the largest intersection areas are selected, their projected images denoted as I

_{1}, I

_{2}, …, I

_{n}are utilized to calculate the redundancy score S

_{i}for the ith scan using Formula (3).

_{i}, respectively; ($\circ $) denotes the Hadamard product. The scores for all the scans are then statistically analyzed to calculate the coefficient of variation. A higher coefficient of variation indicates a greater discreteness of scans, meaning false matching may be present. The top 20% scans with the highest scores are selected as potentially mismatched PCDs, awaiting further validation.

#### 3.2. Overlap-Based Scan-Block Construction

_{ij}between the nodes v

_{i}and v

_{j}is defined as

_{j}and B

_{i}in Equation (5) represent the bounding boxes of two scans.

_{i}represents the ith scan-block, ${\overline{A}}_{i}$ the complement of A

_{i}, k the total number of blocks, and cut(A

_{i}, A

_{j}) the cut between A

_{i}and jth scan-block A

_{j}, i.e.,

_{i}) represents the sum of degrees of each node within A

_{i}, i.e.,

#### 3.3. Pairwise Coarse Registration by Optimization Algorithms

_{i}in P is paired with its nearest neighbour q

_{j}in Q, forming a point pair (p

_{i}, q

_{j}). The set of point pairs is represented as H = {(p

_{i}, q

_{j})}

_{1}

^{k}, where k is the number of point pairs. If the distance between p

_{i}and q

_{j}is less than a threshold δ, the point pair is considered as a true correspondence. The MCS problem [30] aims to find the transformation matrix corresponding to the maximum number of true correspondences, i.e.,

**R**and

**T**, and each particle is described by speed v

_{ij}and position x

_{ij}, which are updated by Equations (10) and (11).

_{ij}(t) and v

_{ij}(t + 1) are the particle speeds at times t and t + 1, respectively; x

_{ij}(t) and x

_{ij}(t + 1) are the particle positions at times t and t + 1, respectively; p

_{ij}is the historical optimal solution of the current particle; and p

_{gj}is the historical optimal solution of the swarm; c

_{1}and c

_{2}are acceleration constants and both set to 2; w is the inertia weight and is set to 0.8; r

_{1}and r

_{2}are random numbers in the closed interval [0,1].

#### 3.4. Fine Registration and Pose Optimization

## 4. Experiments and Analysis

#### 4.1. Datasets Description and Evaluation Criteria

^{r}and translation error e

^{t}of all transformations among the scans [35], which are as follows:

^{e}and t

^{e}represent the estimated rotation matrix and translation vector, respectively, and R

^{g}and t

^{g}are the those of the ground truth. In addition, the successful registration rate (SRR) is also utilized and defined by

_{s}the number of successfully aligned scans. A scan is considered successfully aligned when its rotation error and translation error are both less than the specified thresholds σ

^{r}and σ

^{t}, respectively.

#### 4.2. Results of Template-Guided Initial Pose Estimation

#### 4.3. End-to-End Performance Evaluation

^{r}and σ

^{t}to 100 mdeg and 100 mm, respectively. The experimental setup utilized a hybrid programming approach, with the template-guided initial pose estimation step implemented in Python and the remaining parts in C++. Our method was tested on a laptop with 32 GB RAM and an Intel Core i7-7700K CPU.

#### 4.4. Discussion

## 5. Conclusions

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Conflicts of Interest

## References

- Ministry of Transport of the People’s Republic of China. Statistical Bulletin on the Development of the Transport Industry in 2022; Ministry of Transport of the People’s Republic of China: Beijing, China, 2023. (In Chinese)
- Yang, Y.B.; Chen, J.; Xu, H. Normalized formula for removing damping effect in recovering bridge mode shapes using a moving and a stationary vehicle. J. Sound Vib.
**2023**, 573, 118219. [Google Scholar] [CrossRef] - Xu, H.; Liu, Y.H.; Chen, J.; Yang, D.S.; Yang, Y.B. Novel formula for determining bridge damping ratio from two wheels of a scanning vehicle by wavelet transform. Mech. Syst. Signal Pract.
**2024**, 208, 111026. [Google Scholar] [CrossRef] - Bhowmik, B.; Tripura, T.; Hazra, B.; Pakrashi, V. Real time structural modal identification using recursive canonical correlation analysis and application towards online structural damage detection. J. Sound Vib.
**2020**, 468, 115101. [Google Scholar] [CrossRef] - Kaewunruen, S.; Sresakoolchai, J.; Ma, W.; Phil-Ebosie, O. Digital twin aided vulnerability assessment and risk-based maintenance planning of bridge infrastructures exposed to extreme conditions. Sustainability
**2021**, 13, 2051. [Google Scholar] [CrossRef] - Rashidi, M.; Mohammadi, M.; Sadeghlou Kivi, S.; Abdolvand, M.M.; Truong-Hong, L.; Samali, B. A decade of modern bridge monitoring using terrestrial laser scanning: Review and future directions. Remote Sens.
**2020**, 12, 3796. [Google Scholar] [CrossRef] - Kaewunruen, S.; AbdelHadi, M.; Kongpuang, M.; Pansuk, W.; Remennikov, A.M. Digital Twins for Managing Railway Bridge Maintenance, Resilience, and Climate Change Adaptation. Sensors
**2023**, 23, 252. [Google Scholar] [CrossRef] - Date, H.; Yokoyama, T.; Kanai, S.; Hada, Y.; Nakao, M.; Sugawara, T. Efficient registration of laser-scanned point clouds of bridges using linear features. Int. J. Autom. Technol.
**2018**, 12, 328–338. [Google Scholar] [CrossRef] - Zhang, H.; Zhu, Y.; Xiong, W.; Cai, C.S. Point Cloud Registration Methods for Long-Span Bridge Spatial Deformation Monitoring Using Terrestrial Laser Scanning. Struct. Control Health Monit.
**2023**, 2023, 2629418. [Google Scholar] [CrossRef] - Cheng, G.; Liu, J.; Cui, N.; Hu, H.; Xu, C.; Tang, J. Virtual trial assembly of large steel members with bolted connections based on point cloud data. Automat. Constr.
**2023**, 151, 104866. [Google Scholar] [CrossRef] - Liu, J.; Cui, N.; Zhou, X.; Li, D.; Cheng, G.; Zeng, Y.; Cao, Y. Intelligent inspection method for dimensional quality of houses based on 3D laser scanning. J. Archit. Civ. Eng.
**2022**, 71–80. (In Chinese) [Google Scholar] - Wu, H.; Yan, L.; Xie, H.; Wei, P.; Dai, J. A hierarchical multiview registration framework of tls point clouds based on loop constraint. ISPRS J. Photogramm.
**2023**, 195, 65–76. [Google Scholar] [CrossRef] - Dong, Z.; Yang, B.; Liang, F.; Huang, R.; Scherer, S. Hierarchical registration of unordered TLS point clouds based on binary shape context descriptor. ISPRS J. Photogramm.
**2018**, 144, 61–79. [Google Scholar] [CrossRef] - Weber, T.; Hänsch, R.; Hellwich, O. Automatic registration of unordered point clouds acquired by Kinect sensors using an overlap heuristic. ISPRS J. Photogramm.
**2015**, 102, 96–109. [Google Scholar] [CrossRef] - Zhao, L.; Xiang, Z.; Chen, M.; Ma, X.; Zhou, Y.; Zhang, S.; Hu, C.; Hu, K. Establishment and Extension of a Fast Descriptor for Point Cloud Registration. Remote Sens.
**2022**, 14, 4346. [Google Scholar] [CrossRef] - Deng, X.; Wang, S.; Zhuang, Y.; Fan, Y.; Zhou, Y. Evaluation of new target structure and recognition for point cloud registration and coordinates transformation of China’s large double-span bridge. J. Eng. Appl. Sci.
**2023**, 70, 135. [Google Scholar] [CrossRef] - Singh, S.K.; Banerjee, B.P.; Raval, S. Three-dimensional unique-identifier-based automated georeferencing and coregistration of point clouds in underground mines. Remote Sens.
**2021**, 13, 3145. [Google Scholar] [CrossRef] - Ge, X.; Zhu, Q. Target-based automated matching of multiple terrestrial laser scans for complex forest scenes. ISPRS J. Photogramm.
**2021**, 179, 1–13. [Google Scholar] [CrossRef] - Ge, X.; Hu, H.; Wu, B. Image-guided registration of unordered terrestrial laser scanning point clouds for urban scenes. IEEE Trans. Geosci. Remote Sens.
**2019**, 57, 9264–9276. [Google Scholar] [CrossRef] - Dong, Z.; Liang, F.; Yang, B.; Xu, Y.; Zang, Y.; Li, J.; Wang, Y.; Dai, W.; Fan, H.; Hyyppä, J. Registration of large-scale terrestrial laser scanner point clouds: A review and benchmark. ISPRS J. Photogramm.
**2020**, 163, 327–342. [Google Scholar] [CrossRef] - Guo, Y.; Sohel, F.; Bennamoun, M.; Wan, J.; Lu, M. An accurate and robust range image registration algorithm for 3D object modeling. IEEE Trans. Multimed.
**2014**, 16, 1377–1390. [Google Scholar] [CrossRef] - Theiler, P.W.; Wegner, J.D.; Schindler, K. Keypoint-based 4-points congruent sets–automated marker-less registration of laser scans. ISPRS J. Photogramm.
**2014**, 96, 149–163. [Google Scholar] [CrossRef] - Tang, Y.; Feng, J. Hierarchical multiview rigid registration. Comput. Graph. Forum
**2015**, 34, 77–87. [Google Scholar] [CrossRef] - Remondino, F.; Spera, M.G.; Nocerino, E.; Menna, F.; Nex, F. State of the art in high density image matching. Photogramm. Rec.
**2014**, 29, 144–166. [Google Scholar] [CrossRef] - Schenk, T. Digital Photogrammetry; Springer: Berlin/Heidelberg, Germany, 2009. [Google Scholar]
- Fras, M.K.; Vezocnik, R.; Gvozdanovic, T.; Kogoj, D. Complete automation of the relative orientation of a stereopair. Geod. Vestn.
**2008**, 52, 241–253. [Google Scholar] - Tjahjadi, M.E. Photogrammetric area-based least square image matching for surface reconstruction. Elektrika
**2017**, 1, 22–26. [Google Scholar] - Shi, J.; Malik, J. Normalized cuts and image segmentation. IEEE Trans. Pattern Anal. Mach. Intell.
**2000**, 22, 888–905. [Google Scholar] - Anon. sklearn.cluster. KMeans. Available online: https://scikit-learn.org/stable/modules/generated/sklearn.cluster.KMeans.html (accessed on 14 January 2024).
- Chin, T.-J.; Suter, D. The Maximum Consensus Problem: Recent Algorithmic Advances; Springer Nature: Berlin/Heidelberg, Germany, 2022. [Google Scholar]
- Besl, P.J.; McKay, N.D. Method for registration of 3-D shapes. Proc. SPIE
**1992**, 1611, 586–606. [Google Scholar] [CrossRef] - Lu, F.; Milios, E. Globally consistent range scan alignment for environment mapping. Auton. Robot.
**1997**, 4, 333–349. [Google Scholar] [CrossRef] - Anon. Leica ScanStation P40/P30—High-Definition 3D Laser Scanning Solution. Available online: https://leica-geosystems.com/products/laser-scanners/scanners/leica-scanstation-p40--p30 (accessed on 14 January 2024).
- Anon. User Manuals and Quick Start Guides for the Focus Laser Scanner. Available online: https://knowledge.faro.com/Hardware/Focus/Focus/User_Manuals_and_Quick_Start_Guides_for_the_Focus_Laser_Scanner (accessed on 14 January 2024).
- Wei, P.; Yan, L.; Xie, H.; Huang, M. Automatic coarse registration of point clouds using plane contour shape descriptor and topological graph voting. Autom. Constr.
**2022**, 134, 104055. [Google Scholar] [CrossRef]

**Figure 3.**An example of image matching: (

**a**) The relative positions of a scan with respect to the registration template. (

**b**) The relative positions between different scans.

**Figure 8.**Two bridges in the bridge datasets: (

**a**) Cuntan Yangtze River Bridge. (

**b**) Huanghuayuan Jialing River Bridge.

**Figure 9.**Initial pose estimation results of two bridges: (

**a**) Cuntan Yangtze River Bridge. (

**b**) Huanghuayuan Jialing River Bridge.

**Figure 10.**Registration results of the bridge dataset: (

**a**) Cuntan Yangtze River Bridge. (

**b**) Huanghuayuan Jialing River Bridge.

Dataset | Scanners | Scans | Pts (Billion) |
---|---|---|---|

Cuntan Yangtze River Bridge | Leica P40 (Leica, Wetzlar, Germany) | 29 | 1.52 |

Huanghuayuan Jialing River Bridge | Faro S350 (Faro, Lake Mary, FL, USA) | 20 | 1.68 |

Dataset | Rotation Error (mdeg) | Translation Error (m) | Time (min) | ||
---|---|---|---|---|---|

Δx | Δy | Δz | |||

Cuntan Yangtze River Bridge | 19.1 | 0.66 | 3.62 | 1.88 | 6.08 |

Huanghuayuan Jialing River Bridge | 19.7 | 0.64 | 5.16 | 2.35 | 4.43 |

Dataset | Rotation Error (mdeg) | Translation Error (mm) | SSR (%) | Time (min) | ||
---|---|---|---|---|---|---|

Average | RMSE | Average | RMSE | |||

Cuntan Yangtze River Bridge | 0.96 | 0.67 | 28.04 | 14.13 | 100 | 50.6 |

Huanghuayuan Jialing River Bridge | 0.74 | 0.66 | 43.25 | 28.48 | 100 | 40.5 |

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |

© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Xiong, G.; Cui, N.; Liu, J.; Zeng, Y.; Chen, H.; Huang, C.; Xu, H.
Template-Guided Hierarchical Multi-View Registration Framework of Unordered Bridge Terrestrial Laser Scanning Data. *Sensors* **2024**, *24*, 1394.
https://doi.org/10.3390/s24051394

**AMA Style**

Xiong G, Cui N, Liu J, Zeng Y, Chen H, Huang C, Xu H.
Template-Guided Hierarchical Multi-View Registration Framework of Unordered Bridge Terrestrial Laser Scanning Data. *Sensors*. 2024; 24(5):1394.
https://doi.org/10.3390/s24051394

**Chicago/Turabian Style**

Xiong, Guikai, Na Cui, Jiepeng Liu, Yan Zeng, Hanxin Chen, Chengliang Huang, and Hao Xu.
2024. "Template-Guided Hierarchical Multi-View Registration Framework of Unordered Bridge Terrestrial Laser Scanning Data" *Sensors* 24, no. 5: 1394.
https://doi.org/10.3390/s24051394