Analysis of Maintenance Techniques for a Three-Dimensional Digital Twin-Based Railway Facility with Tunnels

Abstract

In accordance with the paradigm of the 4th and Industrial Revolution, the introduction of building information modeling is expected in all areas related to railroad construction, operation and management, along with the establishment of a metaverse platform that combines big data, the Internet of Things, and artificial intelligence. The performance of tasks related to the safety and maintenance of railway facilities is aided by the use of digital systems free from physical and temporal constraints. Three-dimensional (3D) modeling and other 4th industrial technologies, such as unmanned aerial vehicles (UAVs) and light detection and ranging (LiDAR), are increasingly implemented in many types of infrastructure. With respect to railroads, the use of these methods to monitor tunnel spaces has been hindered by the limitations of modeling with UAV and inadequate Global Positioning System reception. Here, we conducted the domestic application of 4th industrial technologies to a railway tunnel; we addressed these problems using a BLK360, a fixed LiDAR device that combines two-dimensional panoramic images and a 3D point cloud method. The outcomes of this research will benefit railway operation managers by providing a platform combining a two-dimensional panoramic virtual reality (VR) image and a 3D model developed from a 3D scan framework for the maintenance of existing railway facilities (tunnels). Our approach was optimized for the maintenance and operational management of railroad facilities, as demonstrated for tunnels, because it continuously acquires time-series data that is appropriate for the maintenance of the corresponding space. In the future, this approach can be used for test tracks and operational lines.

1. Introduction

The Korean government’s Digital New Deal (2022) proposes to establish a new innovation roadmap for the railway industry as a promising future technology that can change the paradigm of the railway sector [1]. The planned approach combines systems such as big data, the Internet of Things, and artificial intelligence. It will be realized through the Metaverse, a key technology in the digitization of railway facility maintenance, resulting in a paradigm change in the railway industry. In addition to new technologies, excavation and construction are needed; thus, the Ministry of Land, Infrastructure and Transport and the Ministry of Science and Technology established the ‘Railway Building Information Modeling (BIM) 2030 Roadmap’ and the ‘Metaverse New Industry Leading Strategy’, respectively. These projects will require the construction of a BUS platform [2,3,4]. Previously, the maintenance of existing railway facilities (e.g., railway lines, tracks, bridges, and tunnels) was performed by crews that directly inspected and collected data on-site. However, in recent years, real-time data related to railways have been collected digitally and chronologically. These data can be used to establish a platform system that forms the basis of railway operating system digitization, thereby avoiding physical and temporal constraints [1]. For example, digital twins can be constructed through 3D scanning of spaces of interest [5,6,7,8,9,10,11,12]. In previous studies of the 3D modeling of railway tunnels, the tunnel modeling method typically involved a photogrammetry method [11,12]. Thomas M. et al. [11] describe the expression of the visual information of a 3D model in the limited space of a railroad tunnel, and Leanne A. et al. [12] describe the physical structure inside a tunnel with photogrammetry techniques, obtained using a digital single-lens reflex (DSLR) camera. Problems that arise in tunnels have been examined using the tunnel scan technique. However, previous studies have not obtained a railway tunnel digital twin dataset but rather reviewed existing research. Thomas M et al. [11] verified that tunnel scans could be implemented using the LiDAR technique, while Leanne A et al. [12] revealed that they could be achieved with simple photogrammetry. The current railway tunnel surveying method is a 3D-point-cloud-based terrestrial LiDAR method that records geospatial coordinates as individual point data. Because it enables rapid surveying compared to the total station method, it has the advantage of being able to acquire the inner cross-section of the tunnel in a timely manner with minimal errors and the occurrence of over-excavation. However, when the internal section is enlarged and checked, the point cloud is composed of data points, and therefore a process combined with photogrammetry is required. The survey method used in this study was a terrestrial LiDAR method that was capable of image scanning and produced a model from a combination of 2D images and 3D point clouds. Geometric information was obtained through precise 3D scanning equipment, and relatively precise 3D scanning results were obtained.

The United States is actively introducing BIM into the public sector, with major contributions by the private sector. Singapore’s Building and Construction Authority has been supporting industrial facilities since 2010, with the application of BIM being required to improve construction productivity by 25% within 1 year. In the UK, to improve and advance productivity in the construction industry, which constitutes approximately 7% of the gross domestic product, the British government has required the application of BIM to public construction projects since 2011; a BIM level 2 application has been required since April 2016. China is actively introducing unmanned aerial vehicles (UAVs) into railway monitoring methods. In India, railway track assessments using UAV images, as well as image enhancement and analysis, have been adopted. Although people were previously employed to directly monitor railway track conditions, a new framework has been developed through survey analysis using UAV.

The 13-km-long Osong Railway Comprehensive Test Track, the first test track constructed in Korea, is a test-bed for the performance of field tests on railroad vehicles and facilities. This space allows the development of a model to ensure consistent integrated safety management and national digital maps. Among digital transformations in the railroad industry that have been implemented in connection with national policies, digitization allows preemptive railroad maintenance, along with efficient test track operation and management. Thus, there is a need to construct various types of railway facilities using a 3D digital twin technique and then to use them to secure basic data for the maintenance and operation of a national test-bed. In this study of a 13 km test track, an approximately 4 km section of tunnel was targeted. The 2D floor VR video and 3D model pluggable platform developed from the 3D scan framework for the maintenance of existing railway facilities (tunnels) was loaded to enable its general use by railway operation managers. The remainder of this paper is structured as follows: the scan results of the test tunnel are presented, the current lidar technology is compared and analyzed, and the future plan is presented by establishing a digital twin space.

2. Materials and Methods

2.1. Research Site

The study site was selected among the facilities of the first comprehensive railway test track in Korea, located across Osong-eup, Heungdeok-gu, Cheongju-si, Chungcheongbuk-do, and Jeondong-myeon, Sejong Special Self-Governing City. (Figure 1). The comprehensive railway test track was opened on 15 March 2019; it consists of a test track with a total length of 12.990 km that extends from Osong Station, passes the Osong Base entry line, runs in parallel with the Gyeongbu High-Speed Line, and then runs along the train station after adjoining the Gyeongbu Line. Among the nine bridges in the comprehensive railroad test track, six of their tunnels were examined in this study: test 1 tunnel (551 m), test 2 tunnel (229 m), test 3 tunnel (1245 m), test 4 tunnel (271 m), test 5 tunnel (905 m), and test 6 tunnel (1020 m). The total length of the tunnel sections is 2.265 km (17.4% of the total section).

2.2. Experimental Methods

2.2.1. Scan Equipment and Shooting Method

The 3D digital twin scan required for tunnel scanning was conducted using a Leica BLK360(Leica, Zuerich, Swiss) (i.e., BLK360) as a fixed LiDAR (Figure 2). The BLK360 collects 3D point cloud data and is effective in the analysis of objects for which precise measurements of indoor and outdoor spaces are needed (e.g., buildings and facilities). The BLK360 can achieve a scan speed of 360,000 points per second; it can perform laser scanning in the range of 360° horizontally and 300° vertically. The maximum scan distance is 60 m, and high-speed image shooting can support high dynamic range images of 150 million pixels. The BLK360 is equipped with an automatic leveling device, allowing the control of physical tilt during scanning. It can be used to assess the inside and outside of a tunnel through highly realistic images obtained using a virtual reality (VR) workflow and two-dimensional (2D) panoramic images.

Scanning was conducted via duplicate scans at regular intervals of 5 m from the start to the end of the tunnel, with a length of ~5 min per scan. It proceeded in the form of overlapping and matching at the QR code installation distance. With respect to railway tunnels, continuous scanning was ensured by attaching a self-made QR code to each 5 m interval to ensure a continuous connection between the scanned sections and to control for poor Global Positioning System (GPS) reception (Figure 3). The QR code deployed in this study was an assisted alignment technique, which is conceptually similar to a QR code but can be viewed as a kind of 2D barcode. This technique is based on the AprilTags system that was developed at the University of Michigan (USA) and is used as a visual reference system. Its use is essential in places where there are uniform and repetitive structures, such as railway tunnels. In the case of 3D scanning, when the previous scan position and the new scan position are very similar, it may be difficult to capture data, and misalignment and alignment errors may occur. Using a QR code makes such a scan possible, and it is important to check the scan results in real time on a monitor. The scan could be conducted in the exact location by recognizing the QR code in a 2D panoramic image of the BLK360.

2.2.2. Capture Software

The capture software was Matterport Capture (Matterport, Sunnyvale, CA, USA), connected to the BLK360 via Wi-Fi. This software allows for the evaluation of scan results in real time on the display of a tablet or mobile device. It enables the degree of overlap and accuracy of the scan to be determined in advance, thereby permitting correction and rescanning (Figure 4). After the scan is complete, the final virtual 3D model of the railroad tunnel can be evaluated through image upload and processing.

2.2.3. 3D Digital Twin Program

The 3D digital twin program was provided by Matterport Cloud (Matterport, Sunnyvale, CA, USA). The scan data collected via Matterport Capture software was uploaded to the Matterport Cloud.

Within this program, the tunnel space in the comprehensive railroad test track is presented as a virtual 3D model platform identical to the real tunnel (Figure 5), with a conversion of the data into factors such as space details, schematic floor plans, OBJ files, and BIM. The data can be downloaded and used for the desired application. Additionally, using the editing function (e.g., labeling, measurement, and tagging) developed for the 3D model, further analysis is possible; data can be recorded that is related to the space in the tunnel.

To scan the total length of the tunnel (i.e., approximately 4 km of the entire 13 km section) the shooting time was 8 h, the equipment rental cost was 2 million won, two people were required for shooting (for measuring and checking the distance), and the scan data verification method was used to check the measurement results in real time on a tablet. The editing tool maintenance cost was 1.1 million won per year, and the editing time was 3 h.

3. Results and Discussion

3.1. Scan Results

The 3D models of all six tunnels in the railway comprehensive test track, as derived by Matterport Cloud software, are shown in Figure 5 and Figure 6. The 3D model of each tunnel was produced as a 3D space platform that combined a 2D-panoramic-image-based spherical VR video and a 3D point cloud. Because information and data (e.g., digital tags, repairs, and maintenance) can be documented in 2D VR images and 3D models, this approach can be used as a continuous maintenance technique for various railway tunnels (Figure 7). Figure 7 shows the unique method implemented in ‘Matterport’, a 3D digital twin production program. This monitoring technique can be implemented by converging 2D VR images and 3D models. Through this, it is possible to realistically obtain railroad facilities with 2D images and check their structure and physical form with 3D models, enabling multifaceted image monitoring.

Specifically, the structure and flat/sectional characteristics of all sections of a railway tunnel can be assessed using the 3D model, which can also be used as a review method for the comprehensive management of railway facilities (e.g., distribution boards and power lines) (Figure 7;

Table 1. Dimensions, shapes, and conditions of test tracks.

	Tunnel 1	Tunnel 2	Tunnel 3	Tunnel 4	Tunnel 5	Tunnel 6
Tunnel shape	straight	straight	roundabout	straight	straight	sharp curve
Line type	monorail	monorail	monorail	double track	monorail	monorail
Excavation method	NATM tunnel
Tunnel extension length (m)	551	229	1245	271	905	1020
Tunnel diameter (m)	7.40	7.44	7.28	14.54	7.23	7.40
Tunnel height (m)	6.90	6.94	6.86	9.16	7.85	6.86
Iconographic form	cobblestone	cobblestone	cobblestone	cobblestone	gravel, concrete roadbed	cobblestone

).

3.2. Comparative Analysis of LiDAR Techniques

The LiDAR devices used in this study were: (1) GeoSlam ZEB-REVO Horizon, a Simultaneous Localization and Mapping (SLAM)-type mobile handheld LiDAR; (2) a 2D panoramic VR and 3D modeling-based Matterport Pro2 scanner; and (3) a fixed LiDAR Leica A BLK360 imaging scanner. This study was conducted to derive optimal spatial platform construction equipment and techniques for railway tunnels via a comparison of the results obtained with each of the 3D digital twin scanners (

Table 2. Comparison of 3D digital twin equipment.

Designation	GeoSlam ZEB-REVO Horizon	Matterport Pro2	Leica BLK360
Photo
Summary	Portable handheld 3D scanner (indoor/outdoor)	Appropriate for most indoor spaces and limited outdoor spaces (for indoor use)	Appropriate for construction design and built projects (indoor/outdoor combined use)
3D sensor	LiDAR-based	Structured light (infrared) sensor	LiDAR-based
Result	3D point cloud	2D panoramic image + 3D point cloud	2D panoramic image + 3D point cloud
Device connection	Wi-Fi	Wi-Fi	Wi-Fi
Relative accuracy	Up to 6 mm	-	Up to 4 mm
Image quality	25 MP	134.2 MP	34 MP
Points per second	300,000	-	360,000

).

When using the handheld (mobile) LiDAR, based on the SLAM method, data errors were confirmed when the same type of internal structure (e.g., a tunnel) was repeatedly scanned. Data distortion occurred because of accumulated errors, so that this device was inappropriate for application in tunnels. The Matterport Pro2 (Matterport, USA) allows indoor space shooting, thereby enabling 2D panoramic image shooting and 3D space detection with a structured-light (infrared ray)-based sensor. However, when the center of the tunnel was scanned, the GPS reception rate decreased, and data errors occurred. Furthermore, when outdoor areas such as the entrance of the tunnel were scanned, the infrared power of sunlight was weak; thus, nearby objects could not be scanned. The BLK360 can acquire 2D panoramic VR images in addition to 3D point cloud data, unlike other fixed LiDARs; therefore, it is appropriate for the realization of real-life colors and textures. Accordingly, the BLK360 produced reliable visual and precise results throughout the railway facilities and structures, supporting its use in the collection and documentation of information regarding railway tunnels and surrounding environmental spaces.

This study did not perform a quantitative verification based on the LiDAR design algorithm. It was a preliminary study that applied various equipment to tunnels, with the aim of identifying and applying the optimal lidar equipment technology in spaces where GPS signals are not caught, i.e., tunnels. Therefore, quantitative parameters such as sigmas, coverage percentages, and drift errors were not obtained. These will be reviewed in future studies.

As a result of the comparisons made in this study, a 3D point cloud was generated using a SLAM-based lidar scan. It was difficult to confirm the area element for an object when the section was enlarged. However, because the 3D digital twin produced by Matterport Capture can combine real-life 2D images (RGB) with 3D point cloud data and extract them in the obj data format, i.e., a 3D model, it can be implemented in an almost similar way to a real space realization.

3.3. Measurement Method and Utilization Plan

The railway tunnel 3D digital twin space platform produced through Matterport Capture creates a virtual 3D model through a virtual space realization. It provides a VR workflow through 2D panoramic images (4K). The width, height, and spacing of the railway tunnel can be assessed through the precise measurement function, both in 2D VR images and as a 3D model (Figure 8). In contrast to time-consuming manual measurement, data related to the space of interest can be obtained without visiting the site; there are neither temporal nor spatial limitations. Tunnel measurement information can also be documented by capturing dimensional data.

In particular, cracks generated inside the railroad tunnel, crack repair sections, and leak occurrence sections can be assessed using VR images. Both the crack length (m) and the leak area can be assessed accurately and completely, thereby facilitating railroad tunnel maintenance. This novel approach can be used in tunnel repair and maintenance (Figure 9).

The versatility of this method also allows the resulting 3D digital twin space to be transmitted as a link through online messengers, such as SNS and text messages, thereby permitting maintenance personnel in the facility to easily review data related to the desired section without visiting the site.

In this study, a model that guarantees integrated safety management and produces national digital maps was developed, and a platform combining 2D panoramic VR video and a 3D model was developed from the 3D scan framework for the maintenance of existing railway facilities (tunnels). It was considered feasible for the model to be used universally by railroad operation managers.

4. Conclusions and Discussion

This study was conducted to contribute to the construction of a BIM model by establishing a high-resolution 3D digital twin for a railway comprehensive test track; the resulting test-bed will allow various field tests on railway vehicles and facilities within a national infrastructure. During the maintenance of existing railway facilities, trained professional maintenance personnel move the facilities and record them on paper, resulting in human error. Safety needs to be confirmed, even in very trivial parts of the system, and the results of this study will enable human error to be reduced through 3D scanning and increase the precision of records based on location. This study has established a new maintenance technique. Specifically, we examined proactive maintenance and efficient test-vessel operation and management, as applied to railway tunnels. Our results can be summarized as follows:

First, in terms of the scanning method, real-time monitoring and shooting through the BLK360 scanner was a robust approach for a tunnel connecting indoor and outdoor spaces. An advantage of the BLK360 scanner is that it can perform LiDAR-based high-precision space scanning at locations where GPS reception is weak, such as in a tunnel. However, because ≥ 5 min are required for the BLK360 to scan each area, problems may occur in terms of matching 2D panoramic images and 3D point clouds in relation to the movements of track cars, railroad cars, and maintenance crews. Thus, in terms of 2D and 3D image accuracy and noise removal, an important requirement is the ability to scan while controlling elements that hinder scanning; noise processing is an inherent consideration.

Second, in terms of the scan result, both the color and texture of the actual tunnel are reproduced in the 2D panoramic image, leading to the generation of realistic data that are appropriate for maintenance determinations. The results can be easily shared with maintenance crews through SNS and other messaging systems. The BLK360-based scanning approach is also highly useful for digital-twin-based facility maintenance; time-series data can be stored and managed by uploading detailed maintenance data regarding cracks and leaks in the tunnel to the Matterport platform. The 3D digital twin results produced by methods such as LiDAR scanning can be derived from BIM-type application softwares, based on precisely measured metadata; in the future, this approach will have practical applications for railway facilities.

Third, in terms of the potential for using 3D digital space, temporal and spatial limitations will presumably be overcome with digital stereoscopic images and the accumulation of related time-series data for recording. In particular, when problems such as cracks and water leaks occur in railway tunnel spaces, it will be possible to effectively respond by testing solutions through before/after comparisons in terms of facility maintenance; this approach will facilitate environmental impact assessments of future railroad projects. The application of this technique to railway tunnels will ultimately enable the implementation of a metaverse-based platform. Additionally, the accumulated data can be validated through digital twin construction via continuous 3D scanning.

It is necessary to perform quantitative analyses based on the program design algorithm for lidar technology in the future. In addition, there are plans to introduce new technologies, which may help shorten the measurement time.