Enhancing the Stability and Placement Accuracy of BIM Model Projections for Augmented Reality-Based Site Management of Infrastructure Projects

Abstract

The utilization of Building Information Modeling (BIM) on-site has been limited due to the lack of an appropriate medium for visualizing and accessing models and project data intuitively in the field. AROS, an Augmented Reality-based site management system, was developed to allow the visual projection of BIM models and relevant data directly to field personnel. Detailed field experiments with inspection experts revealed specific issues with the stability and accurate placement of model projections in AROS. Projection stability was improved by reducing the number of triangle meshes of the model to relieve the need for processing power. Investigations revealed that simplification rates of 40% and 20% were optimal for rectilinear and curvilinear components, respectively. Projection placement was improved by implementing a hybrid of target anchoring methods. Specifically, ARWorldMap was used to add additional anchor points, which were identified from the topologies of the structure and surrounding planes. Post-evaluations demonstrated increased stability and reductions in displacement errors. The formalizations provide measures for using AR and BIM models when applying these technologies to large-scale civil infrastructure projects.

1. Introduction

The construction industry is unique in that the implementation of physical works is conducted primarily in the field, and thus site management is a key to the successful delivery of projects [1]. A construction project involves multiple stakeholders, including engineers, contractors, and vendors, who must make evaluations and decisions to maximize productivity while minimizing potential cost overruns and delays, as well as accidents and quality defects [2,3]. Thus, it is essential to provide project managers with relevant project information in a timely and effective manner.

Building Information Modeling (BIM) provides a fundamental basis for improving site management, as it enables a way to manage and integrate project data with design information in the form of 3D models [4]. It can also be used to represent the physical work in a virtual environment, enabling simulations of construction sequences to identify interferences, safety hazards, and potential delays [5]. The utility of BIM has been further improved by the introduction of Common Data Environments (CDE) [6], which provide a platform to share simulations and analyses of BIM models performed by multiple project participants.

Until now, accessing BIM directly in the field has been limited [7,8], with one of the reasons being attributed to the lack of an appropriate medium that allows access to the models readily and easily for personnel on-site [9]. It is also difficult to obtain and compare changing field conditions via BIM models. Indeed, [10] noted that existing BIM-based site management systems had not provided adequate solutions for intuitively utilizing BIM models in the field.

Augmented Reality (AR), which allows the projection of 3D models and relevant information onto real-world scenes, has been explored by researchers in the construction domain to bridge this gap [11]. AR takes advantage of our natural ability to visually acquire knowledge and information [12]. In construction, AR can provide project information directly on-site, via tablets or head-mounted displays, and thus overcome the limitations associated with conventional BIM-based tools reliant on desktops [13]. Specifically, AR has the potential as a technological medium to access and share the information stored in BIM models.

The authors of this study previously developed AROS (Augmented Reality-based On-Site), a system that enables site management by linking BIM with AR technology [14]. AROS is embedded with functionalities customized for field inspection needs, especially for quality and safety checks. Unlike existing AR studies that mainly focused on the architectural field [15,16], AROS is customized for infrastructure projects. However, AROS and its functionalities have not been tested thoroughly by its potential users, nor for its robustness to meet the hardware and software restrictions of display devices.

This study introduces the on-site experiments conducted using AROS to determine its efficacy and practical applicability. The experiments identified two technical issues to improve the projection stability and placement accuracy of the BIM models: (1) the need for model simplification and (2) the need for advanced anchoring techniques.

This paper describes how these issues were addressed, improved, and verified. The results provide practical approaches to how model projections should be calibrated for BIM-based AR systems to work accurately and effectively.

The rest of this paper is structured as follows. Section 2 introduces existing studies where AR has been used for site management, as well as state-of-the-art techniques for AR development. Section 3 provides a summary of the AROS system, while Section 4 explains the methodology used in this study to test and validate AROS in the field. Section 5 presents the two technical issues identified by the field experts, while Section 6 and Section 7 describe the technical approaches adopted to resolve them, respectively. Finally, Section 8 presents a discussion of the results and their limitations.

2. Research Background

2.1. AR Application Cases for Construction Site Management

Virtual Reality (VR) and Augmented Reality (AR) are types of Mixed Reality (MR) technologies that allow interactions of digital artifacts in conjunction with the real world through 3D digital projections [17]. In VR, users acquire 3D digital projection and related information in a fully immersed virtual environment. VR allows users to experience prearranged scenarios within which they can interact separately from the real world [18]. Conversely, AR superimposes 3D artifacts into the real world without setting up a separate virtual environment, allowing physical and digital objects to coexist and interact in real-time [19]. This superimposition between digital objects and the real world promotes intuitive information delivery [20].

In the construction industry, VR has been primarily used for educating personnel about safety hazards, quality checking, and installation procedures before their actual site deployment [21]. AR has been used to directly provide relevant information in the field by superimposing 3D models and relevant field data. Existing studies have shown that AR approaches can reduce times for information searches by as much as 50% compared to conventional management methods [10]. AR has also been shown to improve the effectiveness of progress monitoring, equipment maintenance, as well as safety and quality inspections [22]. The following provides summaries of existent AR systems in these areas:

• Progress monitoring

HoloBuilder (Holobuilder, Inc., Lake Mary, FL, USA) developed a construction monitoring service that enables collaborations between headquarters and field workers by providing images and videos of the site as an AR-based 360-degree visualization. To support the work efficiency of construction managers, [23] utilized an AR-based simulation system to provide information on design, construction, and maintenance processes. Furthermore, [24] developed a real-time 4D AR system that can provide AR-based 4D BIM information to construction managers to identify issues and address them before construction. In addition, [25] developed a machine learning-based object detection system to enable the monitoring of construction progress using an AR device.

2.

Equipment inspection and maintenance

VIRNECT (VIRNECT, Co., Ltd., Seoul, Korea) developed an AR device to project the operations, maintenance, and assembly procedures of equipment at a construction site; an example of the use of this technology for quality and maintenance. SK Telecom (SK Telecom, Co., LTD., Seoul, Korea) developed a technology that uses a smartphone embedded with Trimble’s Site Vision to provide on-site construction information and 3D blueprints. Additionally, Deutsche Bahn (Deutsch Bahn AG, Berlin, Germany) provided AR-based step-by-step repair processes to improve the efficiency of maintenance work at railway facilities.

3.

Safety management

AR has also been explored for safety management. Schiavi et al. [26] developed an AR system that provides visualization of step-by-step tasks and safety guidelines to enable safe work processes and procedural execution at construction sites. Zhou et al. [27] developed an AR-based system for the automatic evaluation of structural safety by examining displacements between tunnel constructions. Kim et al. [28] developed an AR-based system that projects risk information onto smart glasses so that workers are aware of risk factors to avoid before accidents occur. In addition, XYZ Reality (XYZ Reality Ltd., London, UK) developed an AR-based remote sensing system to improve worker safety and ensure increased productivity by 40% compared to conventional productivity management methods. Furthermore, DEEP.FINE (DEEP.FINE Co., Ltd., Seoul, Korea) has been demonstrated to reduce inspection time and cost by up to 60% through a remote safety inspection service.

These studies show that AR has been explored for its potential to improve construction-related tasks. However, many of the commercial systems have been developed for general use purposes and are not tailored for specific work types or field-related needs. Furthermore, they do not provide empirical evidence of their practical applicability and utility on-site.

This study focused on field testing AROS via inspection experts to determine their robustness for conducting safety and quality tasks for infrastructure projects. Infrastructure projects such as bridges provide unique challenges, as their virtual models are large in scale and traverse over long distances. As discussed in Section 5, such challenges require formalizing approaches to ensure stable and accurate placement projections of the 3D models and related project data.

2.2. AR technology Overview

Fundamentally, AR is a technology that projects digital artifacts onto a display surface and aligns the coordinates of the artifact with objects in the real world. With increased interest in the medium as a technology to bridge between virtual and real domains, various hardware and software developments are continuously upgraded.

AR hardware devices enable immersion and a sense of presence by providing visual information in the real world. Hardware is namely divided into wearable devices, such as head-mounted displays (HMDs), and AR-enabled handheld devices, such as smartphones or tablet computers. Although HMDs have their advantages, wearing them for significant amounts of time outdoors has not been readily adopted in the construction industry. HMDs have limited battery capacity and low brightness, and need to be separately acquired [29]. Comparatively, handheld devices are owned by personnel on-site and regularly used in the field, and thus their usage is less prone to resistance. Therefore, AROS has been developed and optimized for tablet devices.

AR content is created using AR software development kits (SDK) that provide various resources and libraries for 2D/3D positioning tracking and graphics processing. AR SDK provides a development environment for Android and iOS, and the functions can be linked to game engines such as Unity’s Unity3D or Epic Games’ Unreal engine. Representative SDKs include the Vuforia SDK by PTC, ARKit by Apple, and ARCore by Google.

Projecting a virtual model onto a real-world scene can be classified into marker-based and markerless methods. Marker-based methods use 2D images with easy-to-extract visual features (e.g., QR codes) and overlay the virtual objects using them as location-specific markers. Markerless methods do not rely on markers, but create virtual objects by tracking extracted feature points of objects within the real world or using location-tracking technologies, such as Global Positioning Systems (GPS) and Radio Frequency Identification (RFID).

Table 1. Various Augmented Reality (AR) Model Projection Methods.

Projection Method	Object Target	3D Model Target	Image Target	Spatial Target
Examples of results by target method
Methodology	Extracting feature points after a 360° shooting using an object scanner	Set 3D CAD file as a target using Model Generator	Determine feature points of 2D photo data and recognize them as targets	Recognize planes that can project models

shows exemplary target methods for each of the projection methods. The ‘Image Target’ is a well-known marker-based method, whereas ‘Object Target,’ ‘3D Model Target,’ and ‘Spatial Target’ are common markerless methods.

2.3. AR and BIM Integration Approaches

Early attempts at AR adoption in the construction industry [30,31] used models created from 3D CAD drafting tools or 3D rendering software (e.g., Blender, 3ds Max, Maya, etc.). These models were subsequently converted into standard vector file formats, such as VRML or STL, for projection using legacy AR devices. However, these approaches only allowed the projection of the 3D graphical components of the models and were consequently limited in their intractability and data accessibility. Thus, AR use cases were restricted to simple projections for equipment arrangement [30,31], interference checks [32], and engineering education [33].

The increased adoption of BIM in the industry has made it natural to use BIM models as a direct source of AR content [22]. Meanwhile, game development engines, once a tool exclusively used for creating animations and 3D gaming scenes, have been extended and applied to engineering disciplines. Consequently, engineering CAD models and BIM models can be readily imported into these engines and transformed into formats acceptable for AR environments. More importantly, these transformations retain the semantic definitions and attributes of the individual elements in BIM models, thereby enabling project data to be displayed over real-world scenes via AR devices. Such advances have enabled BIM models to be used with AR devices as an interactive tool in the field for progress monitoring [24,25], safety inspections [26,28], and quality management [16,27].

The two representative game engines include Unity 3D and Unreal. Both engines provide add-ins within Autodesk Revit for the conversion of BIM models to AR-acceptable model formats: ‘Reflect’ for Unity 3D and ‘Datasmith’ for Unreal Engine.

Table 2. Comparison of Unity 3D and Unreal engine (Adapted from Ref. [34]).

Game Engine	Unity 3D	Unreal Engine
Programming language	C#	C++
Developer community (versatility)	Large	Small
Strengths	Suitable for rapid development of cross-platform apps, Short 3D model production period	Realistic graphics and performance, low programming difficulty
Weaknesses	Not open source	High development difficulty
BIM to AR integration tool	Reflect	Datasmith
OS for BIM to AR integration tool	Windows OS, Mac OS, Android, iOS	Windows OS

provides a summary of the characteristics of both engines. Unity3D has a larger user base and is suited for rapid 3D model development, whereas Unreal is suitable for models requiring high-end graphical performance [34]. Moreover, Unreal’s Datasmith can only be run on Windows, whereas Unity’s Reflect runs on both Windows and Mac environments. This study used Unity3D’s Reflect for BIM-to-AR integration, using an iPad Pro as the primary device for AR implementation.

3. AROS Overview

3.1. AROS Overview and Functions

Figure 1 provides an overview of AROS. AROS has been developed to be deployed to handle the needs of safety and quality inspections for infrastructure projects. AROS allows inspections to be performed directly in the field, as well as remotely from the site office. For “field inspection,” AROS provides a series of functions and user interfaces customized for direct use by site inspectors. “Remote inspection” provides a set of functions for off-site management that enables continuous inspections despite restrictions, such as weather or site inaccessibility. Specifically, point clouds acquired through laser scans and virtual models are projected side-by-side, allowing inspections between as-built and as-planned models.

Table 3. Main functions of AR on-site (AROS).

No.	Function	Description	Examples of the Operation Screen
1	Checklist provisions via BIM model projection	Project BIM-based AR model that stores information on checklist and inspection results when selecting individual elements
2	BIM element marker	Remark elements that have inspection issues or are insufficiently inspected and display elements with colors
3	Dimension measurement	Provide an AR-based measurement tool to measure and detect the length of elements
4	Progress monitoring	Project models by data for comparison of plans and current progress
5	BIM model viewer	Instantly examine the designed BIM models on the screen
6	BIM model and LiDAR analysis	Simultaneously project the point cloud data collected with LiDAR and BIM-based AR model in the same environment

shows the main functions of AROS and their examples. These functions were developed based on surveys and interviews with site management experts [14] and included (1) checklist provisions via BIM model projection, (2) BIM element marker, (3) dimension measurement, (4) progress monitoring, (5) BIM model viewer, and (6) BIM model and LiDAR analysis. The last function was developed exclusively for off-site inspections, whereas the other functions can be used for both field and remote inspections.

Field inspections, the focus of this study, are conducted using a pre-entered checklist in AROS. The inspector checks and marks the individual elements using functions (1) through (5) and notates elements with safety or quality issues. These results are first saved in the virtual model in AROS, then subsequently updated in the original BIM model. This was achieved by encoding a conversion script in Dynamo to save the data in a JSON file format. Saving the information directly in the BIM model allows (1) sharing of the inspection results by all project stakeholders and (2) redisplaying the results in the subsequent inspection runs. AROS displays marked elements in red to ensure that inspectors can check whether related issues have been resolved.

3.2. AROS Implementation and Projection Method

AROS has been optimized for use in Apple’s iPad Pro and was developed using ARKit, a Software Development Kit (SDK) optimized for iOS. Unity 3D, a popular game engine, was used to convert BIM models into graphic formats suitable for projections in AROS. Specifically, Revit model files are converted in Unity 3D as “prefab” formats using the ‘Reflect’ plug-in, an add-on in Unity 3D for BIM to AR model conversion. The conversion objectifies the elements and structure of the BIM model to ensure that the integrity of its data (i.e., geometry and attribute information) is retained. The material attributes (e.g., texture, color, etc.) of the individual elements are also retained, allowing them to be realistically rendered in the AR model projection.

The ‘Image Target’ method, as introduced in Table 1, was adopted as the model anchoring and projection method. Although markerless methods have the advantage of not needing a separate marker, these approaches require extracting and a comparison of model feature points that may not always be available on-site. In Jeon et al. [35], projection using the ‘Image Target’ method, more specifically Quick Resource (QR) codes, was verified to show high stability and minimum displacement errors.

4. Research Methodology

This study aimed to evaluate and improve the practical applicability and robustness of AROS. Figure 2 illustrates the overall process divided into two distinct steps: the existing prototype is referred to as AROS_α, while the improved version resulting from this study is referred to as AROS_β.

First, AROS_α was deployed on a bridge construction site by experienced safety and quality inspectors who used the functionalities introduced in Section 3. They were also asked to specifically evaluate the stability and locational accuracy of the model projections. Subsequently, a Focus Group Interview (FGI) was conducted with the inspectors to identify and collect potential technical issues with the system. The FGI enables people with similar backgrounds to discuss and exchange expert opinions based on their professional experiences, while a mediator helps to provide focus on individual issues [36].

The group consisted of eleven experts in safety and quality management with an average of 13 years of experience in engineering and construction firms, as well as public agencies.

Next, a causal analysis was performed to determine the reasons behind these issues, and approaches to resolve them were investigated and formalized. Finally, these improvements were updated in AROS_β and verified by deploying them on an identical site and quantitatively measuring their improvements.

5. Utilization Assessment and FGI Results of AROS_α

A single-spanned bridge of 20.39 m in length and 5.12 m in height was selected as the target for our study. The bridge was selected as it would provide a single target to test the functionalities of AROS_α. The bridge was first converted into a BIM model using 2D drawings, then converted into an AR model.

Figure 3 shows the results of the utilization assessment and FGI for the individual functions provided in AROS_α. The assessment was conducted by requiring each expert to evaluate the individual functions with respect to their practical applicability in the field. They were asked to assign scores using a 5-point Likert scale, where 1 indicates “very low” and 5 indicates “very high.” The figure shows the average and standard deviations of the scores. The standard deviations were calculated as a value around 1 and represent the spread of the individual scores.

The functions ‘Checklist provisions via BIM model projection’, ‘dimension measurement,’ and ‘BIM model and LiDAR analysis’ ranked highest, with mean values over 4.0; the functions with the highest utility. The results revealed that the experts deemed these functions to have the highest utility in common and periodic inspection tasks. The lowest score was recorded in the ‘BIM model viewer,’ with a mean value of 3.36. Therefore, separately displaying a BIM model is useful, but not on a frequent basis. The differences in the standard deviations were not large (i.e., 0.72–0.96), confirming that the results represented a consensus between the experts as to the utility of the individual AR functions.

Discussions during the FGI revealed that, despite the high utility of the main functions, the experts noted potential shortcomings that would limit their use in the field. Specifically, they noted that the overall quality of the BIM model projections needed to be improved. This problem was further divided into two separate technical issues, and their details are discussed further in the following sections.

5.1. Technical Issue 1: Projection Stability Degradation

Site management for infrastructure typically involves a large footprint that covers a wide and linearly long area. Therefore, inspection times can be substantive and involve frequently moving around the base of a large structure. Thus, the experts emphasized that the AR projections should account for such structures and provide stability during these inspections. However, as shown in Figure 4a, the projection of the bridge did not always overlap correctly with the actual structure. Moreover, lags occurred when inspectors moved from positions, and in severe cases, the projections disappeared from the screen altogether (Figure 4b). Such issues occurred when the distance between the inspectors and the bridge was over 15 m and after 40 min of using AROS_α.

5.2. Technical Issue 2: Placement Accuracy Degradation

When using AROS_α to conduct safety and quality inspections, the experts concurred that the model projections needed to accurately overlap with the real structure with a variance tolerance of around +/−0.1 m. To confirm whether or not AROS_α satisfied this requirement, we performed an experiment in which models were projected, and their variances were measured relative to the real structure. Each measurement was taken from the structure at 3-m intervals, starting at 3 m and stopping at 15 m. At each interval, measurements were taken ten times along the longitudinal (x–z plane) and transverse (x–y plane) axes. The variances in the longitudinal direction were measured as the difference between the projected model and the real structure in the x–z plane, while variances for the transverse direction were measured in the x–y plane.

The results are presented in

Table 4. Projection errors of AROS_α with distance.

Distance	3 m		6 m		9 m		12 m		15 m		Mean (Total)
Standard Plane	x–z	x–y	x–z	x–y	x–z	x–y	x–z	x–y	x–z	x–y	x–z	x–y
1	0.180	0.140	0.150	0.100	0.150	0.100	0.180	0.120	0.150	0.120	0.162	0.116
2	0.150	0.100	0.130	0.100	0.120	0.100	0.180	0.100	0.100	0.070	0.136	0.094
3	0.130	0.100	0.130	0.090	0.120	0.090	0.150	0.100	0.100	0.100	0.126	0.096
4	0.110	0.050	0.140	0.090	0.110	0.110	0.230	0.120	0.210	0.100	0.160	0.094
5	0.130	0.100	0.120	0.100	0.120	0.110	0.120	0.150	0.120	0.120	0.122	0.116
6	0.130	0.100	0.150	0.120	0.130	0.130	0.170	0.090	0.120	0.140	0.140	0.116
7	0.140	0.140	0.120	0.120	0.090	0.100	0.190	0.125	0.200	0.140	0.148	0.125
8	0.100	0.140	0.100	0.130	0.090	0.100	0.100	0.140	0.180	0.100	0.114	0.122
9	0.090	0.140	0.100	0.120	0.100	0.110	0.170	0.050	0.230	0.090	0.138	0.102
10	0.090	0.130	0.060	0.130	0.120	0.110	0.070	0.090	0.220	0.105	0.112	0.113
Mean	0.125	0.114	0.120	0.110	0.115	0.106	0.156	0.109	0.163	0.109	0.136	0.109
Mean (total)	0.120		0.115		0.111		0.132		0.136		0.123

. The total mean of projection errors for longitudinal axes (x–z plane) was 0.136 m, which exceeded the determined minimum requirement of 0.1 m, and all mean values per interval were also exceeded. Specifically, the maximum mean error of longitudinal axes per interval occurred at 15 m with 0.163 m, while the minimum value was 0.115 m at 9 m. For the transverse axes (x–y plane), the total mean error was 0.109 m. The maximum value was 0.114 m at 3 m, and the minimum was 0.109 at 12 and 15 m. In summary, both axes did not meet the requirement, and it was confirmed that the error was longitudinally larger.

A causal analysis revealed that the root of this issue was related to the limitations of the ‘Image Target’ method, the primary approach used to anchor projections in AROS_α. To overcome this limitation, an additional anchoring approach was investigated and tested to be used in conjunction. Specifics of this solution and the details of the validation are provided in Section 6.

6. Improving Projection Stability

6.1. Causal Analysis of Projection Stability Degradation

Polygon meshes are one of the main approaches for representing 3D objects in computer graphics and visualizations [37]. Meshes represent a 3D object using an interlinked collection of polygons (i.e., triangles or rectangles), which in turn are composed of edges and vertices. The simple and repetitive representation format makes computation highly efficient, especially when using Graphical Processing Units (GPUs) to handle graphic displays in computing devices.

AROS_α uses triangle meshes to represent and project 3D objects of interest (prefab format). The total number of vertices for a given object determines its resolution. According to Euler’s formula, 2n triangles are created when the number of vertices is n [38]. As the number of vertices increases, an object can be expressed more accurately, but it requires additional computational power to process and display. So, an excessive increase in the number of triangle meshes can result in hardware overload, which occurs more frequently on mobile devices which are limited in their graphics specifications.

Therefore, it is necessary to set the appropriate number of vertices according to the device and purpose of use. The process of increasing and decreasing the resolution of 3D meshes is referred to as ‘refinement’ and ‘simplification,’ respectively (Figure 5).

Based on investigations into the specifications of iPad Pro and Unity 3D, the recommended range for optimal rendering performance in terms of mesh numbers was between 300 and 1500 (Unity3D (version 2020.3) recommends this range, assuming that the latest high-end hardware is used in mobile devices (https://docs.unity3d.com/560/Documentation/Manual/ModelingOptimizedCharacters.html (accessed on 17 October 2022))). The two most representative high-end mobile devices are Apple’s iPad Pro (4th generation, A12Z Bionic chip with 64-bit architecture, 6 GB RAM) and Samsung’s Galaxy S8 Ultra (Qualcomm Snapdragon Gen 1 SM8450, 16 GB RAM). As a result of comparing the two devices, the rendering performance of the iPad Pro was superior, and LiDAR was also embedded, so this study utilized an iPad Pro as AR implementation hardware). However, the number of triangle meshes for the bridge model included over 2200 meshes (red box in Figure 6) and exceeded these specifications. This was identified as a primary cause for deterioration in projection stability.

6.2. Mesh Simplification Method

Mesh simplification involves a decrease in the number of triangle meshes using an edge reduction algorithm that decomposes and merges the triangles (Figure 7). Meshes can be either manually or automatically simplified. Manual simplification uses 3D computer graphics software, such as Autodesk’s 3DS Max and Robert McNeel & Associates’ Rhinoceros 3D, to select portions of multiple triangle meshes and combine them into a single mesh. This is a precise method, but it is intensive in terms of time and labor.

By contrast, automatic simplification uniformly reduces all triangle meshes of a 3D model and thus is consistent and less time-consuming. However, it has the disadvantage of not being able to simplify a specific part within a given 3D object.

Civil structures are large and consist of multiple subparts, so using a manual method can be inefficient. Moreover, considering that civil structures are perpendicular and planar in nature, which reduces the risk of losing relevant details during simplification, the automatic approach was deemed appropriate. Unity’s Pixyz software was employed, which provides a ‘decimate’ function for mesh simplification and allows direct conversion to AR file formats.

6.3. Optimal Simplification Rate Selection

In a 3D object, the number of meshes for its shape representation differs based on the existence of curvilinear subparts. For rectilinear subparts, a relatively small number of triangles are needed for its representation. For curvilinear parts, a larger number of triangles are densely included to express the individual curvatures. Since automatic simplification uniformly reduces the number of triangle meshes in a 3D object, there is the risk of losing geometric details for these curvilinear components. Such losses in detail may make the model unusable for performing safety or quality inspections. Thus, determining an optimal simplification rate, which reduces mesh sizes while minimizing the loss of detail, needs to be investigated.

Table 5. Change in details of the AR model with a change in the simplification rate.

No.	Simplification Rate	Changes in Model Shape According to Simplification Rate		Description
		Rectilinear Elements	Curvilineal Elements
1	0% (original)	(a) No. of triangle meshes: 374	(b) No. of triangle meshes: 1821	-
2	10%	(a) No. of triangle meshes: 347	(b) No. of triangle meshes: 1637	No change in overall and detailed geometry
3	20%	(a) No. of triangle meshes: 309	(b) No. of triangle meshes: 1455	No change in overall and detailed geometry
4	30%	(a) No. of triangle meshes: 269	(b) No. of triangle meshes: 1274	Damage to the elements with curve
5	40%	(a) No. of triangle meshes: 231	(b) No. of triangle meshes: 1091	Damage to the elements with curve
6	50%	(a) No. of triangle meshes: 191	(b) No. of triangle meshes: 910	Damage to the overall elements

shows the changes in the shape of the model according to the simplification rate. The level of simplification was adjusted from 10% (=90% of triangle meshes of the existing model) to 50% (=50% of triangle meshes of the existing model) with 10% increments. It was confirmed that rectilinear objects in the AR model do not get lost, even if the simplification rate was increased to 40%. In addition, there was no change in the shapes of the overall model, and details were observable up to a simplification rate of 20% (row 3 in Table 5).

However, for curvilinear components, changes in the shape of the model were observed from a simplification rate of 30% (row 4, column (b) in Table 5). At 50%, the degradation of the geometric details was observed even in the rectilinear components, revealing that simplification rates should not exceed this threshold.

These experiments revealed that the optimal simplification rates needed to differ for rectilinear and curvilinear components. Specifically, the optimal simplification rates for rectilinear and curvilinear components were observed to be 40% (=60% of triangle meshes of the existing model) and 20% (=80% of triangle meshes of the existing model), respectively.

6.4. Validation of the Improvement in Projection Stability

The aforementioned optimal simplification rates were applied to the bridge of our case study. As a result, 231 meshes were required for rectilinear elements (reduced by 143 compared to the original model), and 1455 meshes were required (reduced by 366 compared to the original model) to represent the curvilinear components. In total, 509 triangle meshes were reduced from the original AR model, resulting in a reduced model of 1686 meshes.

Figure 8 shows the results of projecting the modified bridge model in AROS_β. Compared to the images in Figure 4a, which showed unstable projections in AROS_α, the model overlapped correctly and was stable without any perceivable lags. Furthermore, the stable projection lasted more than an hour on the AROS_β screen without disappearing.

Nevertheless, variances between the real structure and the projected model were still visually present, indicating that securing the projection stability of the model is possible through simplification, but the placement accuracy was not improved to the degree required to achieve precise overlap.

7. Improving Placement Accuracy

7.1. Causal Analysis of Projection Stability Degradation

‘Image Target,’ the model positioning method used by AROS_α, employs a QR code that contains the size, projection location, and directional coordinates of the projected AR model. When AROS_α recognizes the QR code as the anchor point, a real-sized model is projected at the specified location and direction.

Proper calibration is achieved by first matching the model projection with the real structure, then placing a QR code in a fixed location that locks in these coordinates. Subsequent projections should align correctly when the QR code is recognized by AROS_α. Any change in the location or direction of the user will result in the projected model undergoing appropriate adjustments to maintain the previously specified location. Hence, the model should be correctly projected as long as the QR code is correctly and consistently recognized.

However, this also implies that model projection is highly dependent on the QR code. Although accurate projection is possible within relatively close distances (e.g., 1–3 m), such may not be the case where recognition of the QR code becomes difficult. Using a single QR code may not be practical at construction sites where inspection needs to be performed in different locations and flexibly performed. Existing AR studies recommend that QR codes be installed in multiple locations and scanned periodically to refresh the coordinates of the model [39]. Unfortunately, it is often not feasible to place QR codes in permanent locations, as physical changes continuously occur throughout the site. Therefore, a complementary approach was needed in which these limitations could be addressed.

7.2. Hybrid Target Method

This study used a session state world-tracking approach to supplement the ‘Image Target’ method of AROS_α. Specifically, the ARKit platform provides ‘ARWorldMap,’ a technique that uses the details of the user’s physical space to determine the device’s position and orientation, as well as any anchor objects added to the session that can represent detected real-world features or virtual content. ARWorldMap first identifies the physical features around the user, after which the information is used to anchor the AR model to an identical location in the same direction (https://developer.apple.com/documentation/arkit/arworldmap (accessed on 5 May 2022)). The physical features and topographical objects are obtained from point clouds collected from the LiDAR attached to the iPad device. In effect, these are used to create multiple anchor points. Moreover, ARWorldMap stores spatial measurement data on the device, so the anchors can be used for quick re-projections in the same location.

Figure 9 shows ARWorldMap integrated with the ‘Image Target’ method to improve placement accuracy. The AR model is first created using a QR code that contains the projection information. Simultaneously, ARWorldMap is executed, and topographical objects are identified using LiDAR. If the user moves, the LiDAR continues searching for characteristic points that will serve as new anchors. This prevents even the smallest displacement of the model, thus ensuring that it stays fixed even if the QR code is not visible on the AROS_β screen.

7.3. Validation of the Improved Placement Accuracy

Placement accuracy improvements were validated by comparing the differences in the amount of projection errors occurring between AROS_α versus AROS_β. As discussed in Section 5.2, the mean error needed to be under 0.1 m, the criterion specified by the inspectors as the maximum allowable tolerance.

Table 6. Projection errors of AROS_β by 3 m intervals.

Distance	3 m		6 m		9 m		12 m		15 m		Mean (Total)
Standard Plane	x–z	x–y	x–z	x–y	x–z	x–y	x–z	x–y	x–z	x–y	x–z	x–y
1	0.030	0.040	0.030	0.020	0.010	0.000	0.170	0.020	0.120	0.000	0.072	0.016
2	0.020	0.020	0.020	0.020	0.030	0.010	0.180	0.010	0.090	0.030	0.068	0.018
3	0.030	0.020	0.010	0.020	0.030	0.010	0.090	0.010	0.070	0.020	0.046	0.016
4	0.030	0.020	0.020	0.020	0.010	0.010	0.220	0.020	0.200	0.000	0.096	0.014
5	0.030	0.000	0.015	0.010	0.010	0.000	0.050	0.020	0.060	0.010	0.033	0.008
6	0.020	0.010	0.030	0.020	0.020	0.000	0.160	0.010	0.075	0.020	0.061	0.012
7	0.010	0.010	0.030	0.000	0.015	0.010	0.180	0.010	0.190	0.020	0.085	0.010
8	0.020	0.020	0.020	0.010	0.020	0.010	0.060	0.010	0.170	0.020	0.058	0.014
9	0.030	0.000	0.025	0.010	0.000	0.010	0.160	0.020	0.200	0.010	0.083	0.010
10	0.020	0.020	0.020	0.020	0.010	0.010	0.065	0.010	0.190	0.020	0.061	0.016
Mean	0.024	0.016	0.022	0.015	0.016	0.007	0.134	0.014	0.137	0.015	0.066	0.013
Mean (total)	0.020		0.019		0.011		0.074		0.076		0.040

shows the results of the longitudinal and transverse measurements. The mean errors along the longitudinal (x–z plane) and transverse (x–y plane) axes were 0.066 and 0.013 m, respectively; thus, both axes were under the target threshold.

For the transverse axis, the mean values at all intervals satisfied the requirement of being under 0.1 m. In the longitudinal axis, mean values under 9 m also satisfied the threshold. However, projections at 12 and 15 m exceeded the requirement, showing mean errors of 0.134 and 0.137 m, respectively.

The minimum error was observed at 9 m in both axes (longitudinal and transverse). The causal analysis revealed that this was due to the correlation between the projection distance and the number of point clouds collected by LiDAR. The number of point clouds proportionally increased as the distance increased from 3 to 9 m. With more point cloud data points, ARWorldMap was able to recognize more topographical features and provide more legitimate anchor points. However, at projection distances of more than 9 m, the extracted point clouds were too far, and thus ARWorldMap was not as effective in identifying the useful features.

The differences in the amount of error between the longitudinal and transverse axes also resulted from the quality of the point clouds. As the x–y plane of the transverse axes used the point clouds of the ground surface as an anchor, it was possible to recognize a continuous and consistent number of anchors, thereby maintaining high accuracy. Conversely, the point clouds used in the longitudinal axes (x–z plane) were segmented, and thus the number of legitimate anchors was smaller.

In summary, the results revealed that the anchors extracted from the bridge’s x–z plane were most effective at a projection distance of up to 9 m. At distances over 9 m, projection errors rapidly increased as the anchors were lost. Accordingly, inspections conducted at around 9 m would provide the most optimal placement accuracy in using AROS_β.

Table 7. Delta values of the projection error between AROS_α and AROS_β.

Distance	3 m		6 m		9 m		12 m		15 m		Mean (Total)
Standard Plane	x–z	x–y	x–z	x–y	x–z	x–y	x–z	x–y	x–z	x–y	x–z	x–y
1	−0.150	−0.100	−0.120	−0.080	−0.140	−0.100	−0.010	−0.100	−0.030	−0.120	−0.090	−0.100
2	−0.130	−0.080	−0.110	−0.080	−0.090	−0.090	−0.000	−0.090	−0.010	−0.040	−0.068	−0.076
3	−0.100	−0.080	−0.120	−0.070	−0.090	−0.080	−0.060	−0.090	−0.030	−0.080	−0.080	−0.080
4	−0.080	−0.030	−0.120	−0.070	−0.100	−0.100	−0.010	−0.100	−0.010	−0.100	−0.064	−0.080
5	−0.100	−0.100	−0.105	−0.090	−0.110	−0.110	−0.070	−0.130	−0.060	−0.110	−0.089	−0.108
6	−0.110	−0.090	−0.120	−0.100	−0.110	−0.130	−0.010	−0.080	−0.045	−0.120	−0.079	−0.104
7	−0.130	−0.130	−0.090	−0.120	−0.075	−0.090	−0.010	−0.115	−0.010	−0.120	−0.063	−0.115
8	−0.080	−0.120	−0.080	−0.120	−0.070	−0.090	−0.040	−0.130	−0.010	−0.080	−0.056	−0.108
9	−0.060	−0.140	−0.075	−0.110	−0.100	−0.100	−0.010	−0.030	−0.030	−0.080	−0.055	−0.092
10	−0.070	−0.110	−0.040	−0.110	−0.110	−0.100	−0.005	−0.080	−0.030	−0.085	−0.051	−0.097
Mean	−0.101	−0.098	−0.098	−0.095	−0.099	−0.099	−0.022	−0.095	−0.026	−0.094	−0.070	−0.096
Mean (total)	−0.100		−0.096		−0.100		−0.058		−0.060		−0.083

and Figure 10 show the quantitative improvement in the placement accuracy of AROS_β compared to AROS_α. The introduction of additional anchors using ARWorldMap decreased the placement error of AROS_α by a mean of −0.083 m. The improvement in the longitudinal and transverse axes were −0.070 and −0.096 m, respectively, revealing better performance in the transverse axis. The highest rate of improvement was observed at 3 and 6 m distances, where the use of QR codes is most effective. The results reinforce the validity of using a hybrid target method to improve projection placement accuracy.

8. Discussion

8.1. Summary of the Results

Two technical issues relating to projection stability and placement accuracy were resolved through the adoption of novel projection techniques. Projection stability was resolved using an edge reduction algorithm to simplify the triangle meshes used for model representation. Systematic testing revealed that simplification rates of 40% and 20% were optimal for rectilinear and curvilinear components, respectively. Applying these rates to the bridge BIM model resulted in reducing its size from 2195 to 1686 meshes, and its projection remained stable without lags and remained stationary for over an hour.

Such findings provide a more detailed approach to mesh simplification techniques. Existing approaches have devised ways to simplify mesh representations with the goal of improving the battery time of AR mobile devices [40,41] or reducing the computation load for model rendering [42]. These studies focus on indiscriminately compressing large mesh models so that AR-related operations may function as normal. In comparison, this study differentiated the simplification ratios according to component shape types to ensure that AR-based on-site management operations can be correctly performed, while accounting for hardware performance restrictions.

Placement accuracy was improved using point clouds and ARWorldMap functions to identify and leverage additional anchor points. The validation revealed that the mean errors of displacement along the longitudinal (x–y plane) and transverse (x–z plane) were improved by −0.070 and −0.096 m, respectively. As a result, the mean errors in AROS_β were 0.066 m for the longitudinal (x–z plane) axes and 0.013 m for the transverse (x–y plane) axes, which satisfied the required tolerance of 0.1 m. The experiments also revealed that a projection distance of 9 m was most suitable for on-site inspections.

The optimal projection distance of civil structures is an issue that has not been addressed in depth in the existing research. Previous approaches to securing placement accuracy were mainly investigated for indoor work [43,44], so projection distance was not considered as a significant factor in improving placement accuracy. On the other hand, the inspection of civil structures is focused on outdoor work, and due to the massive size of the target, it is necessary to secure adequate placement accuracy in long-distance projection compared to indoor work.

Thus, the results derived from the empirical tests performed in this study provide concrete findings that can be exclusively utilized for AR deployment on large-scale, outdoor civil engineering structures.

8.2. Limitations and Future Work

Despite these improvements, the following limitations are recognized in this study:

First, to focus on the identification of the technological issues, the experts’ validation was conducted on a completed bridge rather than a bridge under construction. The experiments did not reveal all potential problems that may occur while managing a large-scale infrastructure site. The existence of materials, temporary facilities, and equipment on such sites can create occlusions and blind spots, hampering projections and lines of sight. Furthermore, the structure itself will change as construction progresses, creating issues for the ideal placement of the QR codes. Such factors need to be accounted for in the future development of AROS.

Secondly, the length of the bridge used in the experiment was 20 m, which is relatively small compared to common civil infrastructure bridges. The number of triangle meshes of a larger bridge will likely be higher, so securing projection stability solely using the specified simplification criteria may not be feasible. This may be solved by varying the Level of Detail (LOD) by projection distance, as shown in Figure 11. For projections over 50 m, the entire model will be projected, so models with low LOD will be projected, thereby significantly increasing the simplification rate. Conversely, close-up projections should have higher levels of LOD, thus showing details of the model with decreased simplification rates.

This approach requires storing in advance BIM models with different LODs in AROS and displaying the appropriate model depending on the projection distance. Further studies are required to determine and formalize the appropriate LODs in relation to project distances.

Finally, limitations were observed when using ARWorldMap for improved placement accuracy. Projection time lags and intermittent loss of stored anchors occurred when traveling distances of over 15 m. These issues occurred due to the burden of having to collect and process the point cloud data required by ARWorldMap. As mentioned in Section 6.1, AROS requires significant computing power for processing the triangle meshes, leaving little free space to process the point cloud data. [45] identified such limitations in using ARWorldMap on tablets or smartphones.

One solution would be to segment the scan regions and thereby reduce the amount of point cloud data needed per anchoring cycle. Therefore, any future research includes devising a suitable segmentation approach optimized for large civil structures. For tasks that do not require exact projections, e.g., projecting an inspection checklist, opting to neutralize ARWorldMap and rely solely on the QR code should also relieve the requirement for processing power.

9. Conclusions

This study investigated the technical problems that occurred in the field application of AROS and implemented techniques to resolve these issues. Focus group interviews revealed that the functions provided in AROS had high utilization rates for safety and quality inspection-related tasks. However, the model projections were unstable, revealing cases of miscalibration and projection losses, i.e., showing limitations in their projection stability and placement accuracy.

Projection stability was improved by reducing the number of triangle meshes of the model to relieve the need for processing power. Specifically, simplification rates were investigated to formalize the optimal percentages depending on the shape compositions of the model. The results revealed that simplification rates of 20 and 40% were optimal for models with and without any curvilinear components, respectively. Post-evaluations revealed that these rates allowed stable model projections for prolonged inspection times in AROS.

Placement accuracy was improved by implementing a hybrid of target anchoring methods. Specifically, ARWorldMap was used to add additional anchor points, which were identified from the topologies of the structure and surrounding planes. Post-evaluations revealed that mean displacement errors along the longitudinal (x–y plane) and transverse (x–z plane) were improved by +0.069 and +0.097 m, respectively, satisfying the target accuracy threshold. The highest accuracy was observed at a projection distance of 9 m.

However, these solutions need to be tested on bridges of a larger scale and under construction, as well as incorporating LOD and model segmentation approaches to relieve the burden of model rendering in handheld devices. These issues are planned for future work.

This study focused on verifying the practical applicability of using AR in the field, which has not been rigorously documented or previously demonstrated. Expert evaluations revealed the importance of projection stability and accuracy in making AR a feasible inspection tool, specifically for infrastructure projects. In this regard, this study contributed to accelerating the use of AR technology in the construction industry by formalizing ways to use optimal simplification rates and a hybrid target method to improve model projections in the field.