Investigating Surface Fractures and Materials Behavior of Cultural Heritage Buildings Based on the Attribute Information of Point Clouds Stored in the TLS Dataset

Abstract

To date, the potential development of 3D laser scanning has enabled the capture of high-quality and high-precision reality-based datasets for both research and industry. In particular, Terrestrial Laser Scanning (TLS) technology has played a key role in the documentation of cultural heritage. In the existing literature, the geometric properties of point clouds are still the main focus for 3D reconstruction, while the surface performance of the dataset is of less interest due to the partial and limited analysis performed by certain disciplines. As a consequence, geometric defects on surface datasets are often identified when visible through physical inspection. In response to that, this study presents an integrated approach for investigating the materials behavior of heritage building surfaces by making use of attribute point cloud information (i.e., XYZ, RGB, reflection intensity). To do so, fracture surface analysis and material properties are computed to identify vulnerable structures on the existing dataset. This is essential for architects or conservators so that they can assess and prepare preventive measures to minimize microclimatic impacts on the buildings.

1. Introduction

1.1. General Background

Over the past decade, cultural heritage documentation using Terrestrial Laser Scanning (TLS) technology has significantly improved for both research and industry. This is not only due to the wide availability of technology but also to the ease of use, which makes creating the dataset efficient and practical [1]. As such, it now becomes feasible to provide significant contributions to heritage building performance analysis. In this regard, the heritage building surfaces require great attention to maintain a proper lifecycle, especially relating to climate and environmental issues such as energy consumption, safety, materials, and indoor environment.

According to De Wilde [2], the concept of building performance refers to three principal definitions, which are engineering (e.g., building’s function), process (e.g., the construction process), and aesthetic (e.g., visual representation). These principles can then be viewed as the behavior of buildings in relation to their use. This includes architectural performances that rely on the integration between the observer, buildings, and surroundings. In the context of heritage buildings, at least two potential factors are generally identified as causative aspects of vulnerable building structures, namely human (e.g., war [3,4] and accidents [5,6]) and the environment (e.g., natural disaster [7,8] and aging [9,10]). These factors have a massive impact, not only on the physical (i.e., chemical, biological mechanism) [11] and functional performance of the heritage building, but also on the financial performance (i.e., maintenance cost) [12]). Specifically for climate change, due to the natural weathering of materials and lack of regular monitoring [11], geometric defects such as cracks [13], the loss of thermal performance [14], and structural deformation [15] are often unavoidable in the surface morphology of heritage buildings.

According to Sesana et al. [11], the gradual changes in climate can significantly cause various threats on the indoor environment of heritage buildings. For example, mechanical degradation often occurs based on freeze-thaw cycles, salt crystallization cycles, and physical damage on hygroscopic wooden materials and masonry buildings. There is also chemical and biological degradation that affects materials made of silk, wall decorations, furniture fabrics, and paper due to insects and mold activity. These degradations particularly happen in warmer climates with more humid environment, such as southern European countries, the British Isles, and Scandinavian countries. In our view, Indonesia is more likely to be one of the affected countries according to the relative humidity levels of its warm climate. Therefore, precautions must be taken in the first place to prevent substantial decay to the surface of cultural heritage buildings.

Thanks to the development of advanced laser scanning technology, one can generate high-quality and high-precision reality-based datasets in a matter of minutes. As a product of laser scanning datasets, the point cloud contains attribute information as part of recorded data structures, typically characterized by position information (XYZ), color information (RGB), and reflection intensity (I) [16]. Each piece of information serves a different task. For example, RGB color can be used to capture a specific area based on its value. Kobayashi et al. [17] further employed this color information by calculating Hue (H) and Saturation (S) values in order to identify road signs in Japan. Zhan et al. [18] performed color-based segmentation based on colorimetric similarity and spatial proximity to segment ancient Chinese architecture, while Alshawabkeh [19] used it to automatically detect and quantify façade linear features.

In addition to the color information, the reflection intensity (I) also contributes to the analysis of the performance of the surface dataset. In principle, the intensity consists of the return strength value of the laser beam corresponding to the reflectivity of the surface being scanned [20,21]. This is why the intensity value is highly dependent on the material and surface characteristics of the dataset. This information can be used not only to classify and filter lidar points [22], but also to develop a model study for detecting crack information in building walls [23] and pavement lines [24], and for monitoring in building construction [25,26]. As for the position information (XYZ), this can be used as an index to align and select the coordinate location of the color and intensity values in the dataset.

Therefore, leveraging the potential application of attribute information stored in point cloud data (i.e., geometric and radiometric properties) provides further opportunities to explore features relevant to the assessments of heritage building surfaces. The potentiality of TLS datasets is also relevant for observation through wavelength or electromagnetic (EM) waves. Each TLS device contains a different wavelength depending on the laser material, optical system, and excitation method used [27]. In this regard, the greater the wavelength, the lower the electromagnetic energy carried by the laser beam. According to the electromagnetic spectrum, laser scanners usually lie in the range between 700 nm–2 μm, while the solar energy spectrum can start from 250 nm [28]. Given that the wavelength band of laser scanners belongs to the category of near infrared light (NIR), in principle it corresponds to low-frequency radiation close to the visible red hue [29]. This information provides us with insights to explore materials behavior for the light/solar spectrum retrieved from the dataset. In this regard, materials behavior refers to the functions that affect and characterize the quality of materials in the dataset, both externally (i.e., physical deformation) and internally (i.e., intrinsic defects) [30]. These functions can consist of temperature, orientation of fabric, and strain rates. In the context of heritage buildings, investigating the material behavior of surface datasets plays an important role, not only to identify the morphological characteristics of building surfaces, but also to calculate the affected areas caused by environmental impacts or external functions.

Drawing on the above considerations, this study presents non-destructive methods for investigating the surface degradation characteristics of existing heritage buildings. Given the large number of cultural heritage buildings in Indonesia, this method may attract and help architects or conservators to efficiently conduct preliminary assessments. This study will specifically select one heritage building in Malang City, East Java, as an exploratory case. The proposed method consists of two main features of integrated performances: fracture surface analysis and material properties, each of which will supply information regarding the surface morphological characteristics of the existing dataset. For example, fracture surface analysis is used to identify the structural deformation of the dataset, while material properties are computed to identify the performance behavior of surface materials. Furthermore, the novel method proposed in this work provides several contributions on the current performance assessment of cultural heritage buildings, as follows:

• The proposed method allows one to detect the fracture distribution on the existing surface dataset. This can be done by computing geometric properties of the point cloud (i.e., XYZ) through normal values, dip angle, and dip direction of the surface plane in the dataset. This can help architects or conservators to further identify potential cracks from the surface dataset.
• The proposed method contributes to the detection of vulnerable surfaces in the existing dataset, especially with regard to the materials behavior of the dataset. This can be done by calculating radiometric properties (i.e., RGB, I) of point clouds through the albedo, reflectance, and transmittance values of the TLS dataset. In so doing, architects or conservators can identify the performance qualities of certain areas in the dataset.
• The integrated analysis between fractures and materials behavior permits us to not only analyze the surface performance of the dataset in relation to microclimatic impacts in the indoor environment but also to calibrate the resulting simulations conducted between fracture analysis and materials point distribution.

This work ultimately aims to develop a computational framework for performance assessment of heritage building surfaces. The present study is structured into five sections. Section 1 will describe a general overview of the study, discussing the potential of point cloud attribute information relevant to the current issues in cultural heritage documentation. It will be followed by a discussion of existing works related to the method presented in this work. Section 2 will describe a detailed computational procedure of the proposed method in this study. It mainly consists of three main parts, namely fracture surface analysis, material properties, and the integrated analysis between selected fracture zones and materials behavior. A dataset collection consisting of 3D scanning tools and selected heritage building dataset will then be presented in Section 3. Subsequently, the results and discussion of the study will be described comprehensively in Section 4. Lastly, Section 5 will present concluding remarks and suggestions for future research.

1.2. Related Works

Various studies have been conducted to identify the aforementioned fractures and materials degradation, ranging from conventional methods and tools (e.g., mechanical probes [31], electronic sensors [32,33]) to those using the latest technology, such as 3D laser scanning and machine learning approaches [34]. This study, however, focuses on detecting the surface deformation characteristics of heritage buildings based on TLS datasets due to their ease of use, time and cost considerations, the existence of high-quality datasets, non-contact measurements, and the interdisciplinary fields of research areas. Over the past decade, there has been a growing number of works using TLS datasets to investigate material degradations and physical deformation (i.e., cracks) of heritage buildings. For example, Cabaleiro et al. [35] developed an automatic detection method for identifying and analyzing the cracks in timber beams based on lidar datasets. Specifically, they formulated Alpha-shaped algorithms to control the growth of cracks over time so that they could map the potential cracks that could be harmful for the building structure. Jiang et al. [36] proposed a series of extraction methods for detecting wall cracks on earthquake-damaged buildings. Their method consists of three main steps, namely point cloud wall segmentation based on a triangular irregular network (TIN), image generation, and wall crack extraction. Similarly, Laefer et al. [37] also employed TLS datasets to determine the surface cracks on masonry structures. Although these works predominantly focus on geometric analysis of the dataset, they also have relevant features for further investigation.

However, only a few studies have been found to specifically identify the structural deformation and materials behavior of the building based on attribute point cloud information. For example, Armesto-González et al. [38], Guldur and Hajjar [39], and Olsen et al. [40] similarly examined surface degradation of buildings, mainly based on the intensity information. Several works have also been identified to utilize RGB color information for crack detection of the building surface. For example, Kim et al. [41] investigated cracks on concrete structures using an RGB-D camera based on the angle of view, Sanchez and Bairan [42] performed crack pattern analysis on concrete elements based on RGB images and orientation kernels, and Vashpanov [43] and Barazeetti and Scaioni [44] conducted crack measurements using a high-resolution digital camera with photogrammetry techniques. These studies, however, only employ image-based methods, which depend on the camera lens, focal length, and the quality of the pixel size taken during the dataset collection [44]. In the context of cultural heritage documentation, the aforementioned studies mostly perform partial assessments on geometric detection (i.e., cracks), often without calibrating the thermal and optical performance of materials behavior. This consequently makes the performance assessments of surface datasets slow and largely dependent on physical inspection, especially when it comes to geometric defects (e.g., cracks, materials decay). In addition, research that specifically focuses on an integrated approach to the assessment of surface performance between structural deformation and materials behavior based on attribute point cloud information has not yet been further explored. This study, therefore, develops a computational framework to examine the performance of heritage building surfaces (i.e., fracture analysis and materials behaviors) based on the geometric and radiometric information stored in TLS datasets.

2. Methods

As introduced in the previous section, this study aims to investigate the surface morphology of existing cultural heritage buildings by analyzing the material properties and fracture planes in the surface dataset. A series of computational procedures was developed to achieve this goal (see Figure 1). In general, it consists of three main steps, namely pre-processing of the dataset, exploratory data analysis that include fracture surface analysis and material properties, and lastly, integrated analysis between fracture plane points and material properties.

In order to perform specific tasks in each predetermined step, the proposed method is supported by several digital tools. For example, the raw dataset was gathered using Leica RTC360 to collect high-resolution point clouds. It was equipped with a Cyclone FIELD 360 to perform dataset modelling, coloring, and transformation from scanner to the workstation in any designated format. In addition, Cloud Compare (CC) [45] was employed to perform dataset preprocessing, such as removing outliers (unnecessary point clouds), calculating surface normal using the Hough Normal plugin [46] on CC, enabling the scalar field feature, and performing dataset subsampling. LAStools [47] was used to georeference datasets. Python [48] and R [49] were used to calculate fracture points and material properties in the dataset, respectively. Additionally, Grasshopper and Rhinoceros were employed for the dataset visualization. Furthermore, a detailed procedure of each step is discussed below.

2.1. Step 01—Dataset Pre-Processing

After collecting a raw point cloud dataset, several tasks are first required to prepare and select the relevant information in the dataset. These tasks include georeferencing, which aims to locate the position coordinates of the dataset to a real-world geographic location. The georeferencing is conducted with the lasinfo and las2las modules in LASTools. First, the ground control points are retrieved from the Differential Global Positioning System (DGPS) from the four corners of the building. The coordinate is in UTM, with all units (x,y,z) in meter. Because the point cloud is referenced to local coordinates —or the offset is 0,0,0— major translation is applied with the lasinfo module to adjust the header information. Lastly, minor translation, which translates point accuracy to mm accuracy is conducted with the las2las module. This is an important step in determining the correct orientation so that the fracture plane can be calculate at a later stage. Outlier removal and noise reduction are also performed to remove irrelevant point clouds as well as to cut the selected portions of the dataset. Given that the raw TLS dataset includes only typical attribute information such as XYZ, RGB, and intensity (I), the surface normal (Nx, Ny, Nz) of the dataset are first calculated using the Hough Normal plugin [46] on CC. This plugin is based on a convolutional neural network. It is useful, not only to select a normal from the accumulator, but also to deal with density variation based on a local density estimation. A detailed feature and procedure of this plugin can be found in Boulch and Marlet [46].

After computing the surface normal of the dataset, intensity correction can be performed. In this regard, intensity correction aims to correct the point cloud radiometric information from erroneous measurements caused by environmental parameters (i.e., equipment sensitivity [50], sensor noise interference [51], atmospheric pollution [38]) during the scan. In fact, it is extremely difficult to compensate for all these variables due to some local constraints, such as climatic condition (moisture and temperature pressures) and the tools’ default settings from the manufacture. For example, the equipment calibration plays an important part in determining the performance evaluation and reducing the uncertainty in point clouds [52]. However, factors such as the end users (audience) of 3D scanning technology targeted in this study, most of whom are architects and heritage conservators, are less applicable (less priority) to equipment calibration due to the prerequisite knowledge of 3D processing datasets. This is why this study focuses on the application of geometric acquisition parameters, namely angle of incidence, which is relevant to the investigation of the surface characteristics of the dataset. The corrected intensity dataset allows us to identify better values of materials attached to the object’s surface dataset.

Furthermore, due to the high-resolution datasets received from TLS, the resulting points often reach very dense sizes. Thus, this study performs a subsampling procedure to manage point density during data analysis. As part of an exploratory research, this study subsamples the density of point clouds into a spatial distance of 5 cm. This is because, first, this study focuses on exploiting the attribute information of the dataset, especially the radiometric properties of the point cloud. This means that the surface morphology analysis of the dataset does not require the Triangular Irregular Network (TIN) method to convert to the 3D mesh model. Accordingly, the subsampling procedure becomes a relevant step to also reduce the computational cost during the simulation. Second, this study develops a computational workflow to build an integrated analysis between surface fracture detection and materials behavior. This workflow includes some processing tools of architecture, such as Rhino and Grasshopper, which unfortunately have limited capacity to handle such high-density point clouds. After that, the subsampling point can be readily used for exploratory data analysis in the second step.

As depicted in Figure 1, the detailed computational procedures of the dataset pre-processing can be found in our previous work [53].

2.2. Step 02—Exploratory Data Analysis of Attribute Point Cloud Information

After establishing the required dataset from the pre-processing procedure, this step focuses on exploring the surface characteristics of the existing building dataset based on the geometric and radiometric properties stored in the point cloud data. It aims not only to identify the materials behavior of the heritage building surface but also to detect potentially cracked surfaces in the dataset. For specific investigations on material properties, a detailed methodical framework can be found in our previous studies [53,54]. This study focuses on the integration of surface analysis between material properties and surface planes in the context of cultural heritage buildings.

• Fracture surface analysis

Cracks or fractures are found in both old and new structures [35], even more so in heritage buildings because of their historical nature. They represent the accumulation of damage during normal use and affect the aesthetics and durability of the building and play important roles in risk assessment of the building [36]. By definition, a fracture is a failure zone and any type of brittle discontinuities where the crack adheres to the plane [55]. In this regard, the cracks often appear as an irregular arrangement. The irregularity can then be recognized on the normal plane as a reference. Given that the fracture varies in a plane, it depends not only on the high or low intensity but also on the large or small volumetric size of the fracture cavity [56]. Thus, identifying the fracture character plays an important role in recognizing the geometric shape and distribution of the fracture [57,58].

In order to determine the fracture distribution, the spatial geometries of the plane, such as dip angle and dip direction, needs to be defined first. In this regard, dip angle is defined between the horizontal plane and the discontinuity plane, while the dip direction refers to the bearing angle (a clockwise direction with respect to north of the projection of the discontinuity plane in the horizontal plane angle of planar features) [59,60,61]. These criteria are used to characterize the distinctive features of the plane, whether it is a fracture zone or not. The fracture zone will undoubtedly make the plane less homogeneous, although random patterns appear in some locations. This results in a non-homogeneous distribution of point clouds, which tends to vary across the plane, suggesting fracture and non-fracture zone clusters. In principle, fractures can be measured through a direct (i.e., scan lining) or an indirect (e.g., remote sensing) method. Because the indirect method provides more benefit in terms of non-destructive measurement and time consumption, this study collected the dataset using a lidar measurement, especially with TLS. This technique is proven to be able to capture a large amount of the dataset at a high level of resolution [1].

In the case of surface objects, fractures attached to the plane are represented as a lineament. This lineament is perpendicular to the plane on the object’s surface. Thus, a normal vector is required to determine the direction of fracture lineament. The magnitude of the fracture direction is then calculated based on the distribution of the bearing angle value. However, the raw point cloud datasets need to be first transformed into angular coordinates because they are still Cartesian coordinates, which do not have angular aspects.

On the other hand, angular coordinates consist of the radial distance, azimuth, and vertical elevation as a set of components. Here, the term of azimuth values is used to define the fracture characterization of the plane. It refers to the magnitude of the angular direction to the normal vector. In standard practice, the azimuth is measured from true north; nevertheless, this study uses this term for the calculation of dip angle and dip direction, as also mentioned in several references [59,60,61]. Furthermore, two crucial components obtained from normal vectors are the magnitude of dip angle and the dip direction. In order to calculate these components, the normal vector of the fracture plane ($h$) and the horizontal plane ($h_n$) must first be defined based on $bx+cy-z+a=0$ and $x-z$ plane for y = 0 as $\overrightarrow{h}=\left(b,c,-1\right)$ and $\overrightarrow{h_n}=\left(0,1,0\right)$. The dot product of these two vectors is $\overrightarrow{h}\times \overrightarrow{h_n}=\left| \overrightarrow{h_n}\right|\left|\overrightarrow{h} \right|\cos \alpha$. Furthermore, the relationship between these two normal vectors can be described as $bX+cY-Z+a=0$ with $X-Z$ for $Y=0$ [62]. By applying the dot product of two vectors [63], the magnitude of angle ($\alpha$) can be computed as follows:

$$\cos \alpha =\raisebox{1ex}{$\left|\overrightarrow{h_n}\times \overrightarrow{h}\right|$}\!\left/ \!\raisebox{-1ex}{$\left|\overrightarrow{h_n}\right|\left|\overrightarrow{h}\right|$}\right.=\left|\raisebox{1ex}{$c$}\!\left/ \!\raisebox{-1ex}{$\sqrt{b^2+c^2+1}$}\right.\right|,$$

(1)

According to Equation (1), the dip angle (DA) can be solved using the following equation:

$$DA={\cos }^{-1}\left|\raisebox{1ex}{$c$}\!\left/ \!\raisebox{-1ex}{$\sqrt{b^2+c^2+1}$}\right.\right|,$$

(2)

After computing the dip angle of the surface dataset, the magnitude of dip direction can be further investigated. In principle, it has a reference in the north direction and is perpendicular to the strike of the plane, which is the horizontal plane. Equation (3) is applied to convert the strike orientation to the dip direction.

$$strike={\tan }^{-1}\raisebox{1ex}{$u_y$}\!\left/ \!\raisebox{-1ex}{$u_x$}\right.+Q,$$

(3)

where, $u_y$ and $u_x$ are the vector components of $\left|u\right|$, $\left|u\right|$ is the three-dimensional vector space in the Cartesian system, and $Q$ is an angle, in degrees, that ensures $\alpha$ lies in the correct quadrant between $0$ to 360°, as listed in

Table 1. The Quadrant parameters in Equation (3).

${\mathit{u}}_{\mathit{x}}$	${\mathit{u}}_{\mathit{y}}$	$\mathit{Q}$ (…°)
$\ge 0$	$\ge 0$	0
$<0$	$\ge 0$	180
$<0$	$<0$	180
$\ge 0$	$<0$	360

[64].

Furthermore, each location coordinate of the point cloud is proceeded to retrieve the azimuth value of the normal vector (see schematic diagram in Appendix A). Subsequently, the distribution of these azimuth values can be plotted to identify fracture and non-fracture zones.

• Materials behavior

The resulting corrected dataset is calculated based on two material qualities: thermal and optical. Thermal properties correspond to the response of surface materials regarding thermal conductivity over time [65], while optical properties refer to the interaction of materials with electromagnetic radiation through the propagation vector or intensity [66]. Both qualities can be used to determine the performance of surface materials through the radiometric information of the point cloud. For example, areas that absorb less energy indicate high albedo values. After identifying these areas, we are able to substitute the surface material that has a 70% lower insolation value with a high albedo value. This strategy is indeed useful for measuring the thermal impact of microclimate issues on cultural heritage buildings. As for the optical qualities, they play an important role in determining the morphological characteristics of the surface material. In this regard, reflectance value can be used to identify the gloss and roughness of the materials in the dataset.

Furthermore, this study computes several parameters from the aforementioned qualities, such as the albedo, reflectivity, and translucency values of the material dataset, respectively. In this regard, albedo is defined as a fraction of solar radiation reflected by the surface with the incidence of radiation on it [67]. Meanwhile, the material reflectivity corresponds to the reduction in reflected electromagnetic power caused by the absorbent surfaces or materials [68], and translucency refers to the diffuse transmission of light [69]. Specifically for translucency, it corresponds to the transmittance values of material, where opaque materials have a zero-transmittance value because they are unable to penetrate light. Computing these properties through RGB and intensity values stored in existing point cloud datasets allow us to map a surface material catalogue. This catalogue can further be used, not only to describe the surface characteristics (e.g., roughness and glossiness) of heritage buildings, but also to take precautions regarding microclimatic conditions, such as reducing heat gains on high reflectivity materials [70].

2.3. Step 03—Integrated Analysis between Material Properties and Fracture Points

This section develops an integrated analysis between surface fractures and materials behavior of the heritage building. The aims are not only to further investigate the performances of the surface morphology of the dataset in relation to the identified areas of surface material distributions, but also to conduct a comparative analysis between the overlapping areas resulting from the two performed steps (i.e., Step 2A and Step 2b). This will provide us information regarding specific areas that include values for both fracture points and materials properties. In doing so, this study will superimpose the value range map of certain material properties (i.e., albedo, reflectance, transmittance) into the selected fracture dataset. Thus, we can detect whether the performance rates of the indicated area are low or high, depending on the size of the overlapping areas. For example, high albedo values usually refer to a smooth or light surface with little structural deformation, while low ones indicate the opposite. Furthermore, the reflectance value can designate the level of surface reflectivity of the dataset, which also indicates the state of surface temperature during the observation [71]. For architects or conservators, this integrated analysis will help them to further study the surface characteristics and current condition of the heritage building, especially related to thermal performance, structural deformation, and materials degradation of the surface dataset.

3. Dataset Collection

3.1. Selected Heritage Building Dataset

The point cloud dataset of the building was collected using 3D laser scanning technology. This technology enables us not only to gather high precision datasets, as compared to ALS (Aerial Laser Scanning), but also to capture specific areas outside and inside the building. In this regard, the selected dataset was gathered using a Leica RTC360. As a lightweight and high-speed 3D portable scanner, this tool can perform a measuring rate of up to two million points per second with an automated targetless field registration (based in VIS technology). This enables one to captured high-quality scans of colored 3D point clouds in under 2 min. A detailed specification of this tool is presented in

Table 2. A detailed specification of the 3D scanner [72].

Parameters		Performance Specification Unit
Performance	Data acquisition	<2 min for complete full dome scan and spherical HDR image at 6 mm @ 10 m
	Real time registration	Automatic point cloud alignment based on Visual Inertia System (VIS)
Scanning	Distance measurement	High dynamic time of flight enhanced by Waveform Digitizing Technology (WFD)
	Laser class	1, 1550 nm (invisible)
	Field of view	360° (horizontal)/300° (vertical)
	Range	Minimum 0.5–130 m
	Resolution	Three user selectable settings (3/6/12 mm @ 10 m)
	Accuracy	Angular accuracy 18″
		Range accuracy 1.0 mm + 10 ppm
	Range noise	0.4 mm @10 m, 0.5 mm @20 m
Imaging	Camera	36 MP 3-camera system captures
	Speed	1 min for full spherical HDR image
Environmental	Operating temperature	−5 °C to +40 °C
	Dust/humidity	Solid particle/liquid ingress protection IP54

.

Furthermore, this study selects one spot captured from a single scan dataset. This is not only due to the dataset correction requirements, but also to the very dense point clouds produced by the TLS.

3.2. Selected Heritage Building Dataset

This study selected the Church of St. Virgin Mary of Mount Carmel, often called the Malang Cathedral Church (see Figure 2), as an exploratory case for implementing the method proposed in Figure 1. This church is located on Ijen Street, Malang City, East Java, Indonesia (7°58′06.11″S, 112°37′24.68″E, with ellipsoid reference WGS84). The Cathedral church was historically built on 11 February 1934, designed by architect Rijksen en Estourgie, constructed by NV Bouwundig Bureau Siten en Louzada, and then renovated on 27 July 2002 [73]. As one of the cultural heritage buildings in Malang, this church presents a typical Neo-Gothic style with combined modern materials such as steel and natural stone, and some materials imported directly from Europe. The 3D visualization of the dataset is freely accessible via this link https://s.ub.ac.id/gerejaijen (accessed on 17 November 2021) [74].

This study specifically selected the Altar room as a scanning dataset (see Figure 3). This room occupies the upper part of the trunk of the cross. The selected area contains 35 million points after being deducted from the original dataset. In order to maintain computational time and cost during the analysis, the trimming dataset was then subsampled. The subsampling dataset ultimately yielded around 121.386 points. The selected area consists of walls, tiles, windows, and some furniture, such as chairs, desks, and lamps).

Furthermore, to investigate the dataset analysis in more detail, the selected portion of the dataset is divided into four small parts, which consist of North wall (A/(a)), North roof (A/(b)), South wall (B/(a)), and South roof (B/(b)) (see Figure 3).

3.3. Selected Materials of the Heritage Building

The selected area predominantly consists of four materials, namely concrete coated with white paint (wall), marble tile (floor), gypsum (ceiling), and modified bitumen (roof). Sections A/(a) and B/(a) most likely correspond to wall and floor materials, while sections A/(b) and B/(b) refer to wall, ceiling, and roof. We have also identified wood material that is mostly distributed in the furniture of the altar. In principle, each material contains different performance thresholds of radiometric properties (see

Table 3. Material specification of the heritage building. Collected from various sources [75,76,77,78,79].

No	Elements	Material Types	Material Properties
			Albedo	Reflectance	Transparency
1.	Wall	Concrete coated with white paint	0.9	0.72	Opaque
2.	Floor	Marble tile	0.6	0.45	Opaque
3.	Ceiling	Gypsum	0.85	0.7	Opaque
4.	Roof	Bitumen roof	0.2	0.25	Opaque

). Values below this threshold can be categorized as low performance surfaces, and vice-versa.

To our knowledge, there are three different approaches to detect damages in a structure: heuristic feature-extraction methods, deep learning-based, and change detection [80]. Here, our approach falls into the first category, which detects damages based on the surface normal. This study found few references that specifically address fracture detection directly based on the distribution of point distances. Most references convert the collected point clouds into a 3D mesh model [81,82] before analyzing the fracture pattern. However, this approach not only requires a highly computational power to handle a dense dataset, but it also cannot be directly integrated into the proposed workflow of material properties in this study due to the requirement of point attributes. Thus, this study currently can only refer to similar fracture patterns and behaviors obtained from the experiment [83,84]. In this regard, the threshold is set based on the surface fracture plane 75°, assuming that the potential crack of this heritage building is caused by gravity and some lateral loads.

In general, a crack can start to propagate when certain parameters, such as stress intensity factors, reach a critical point. This case often causes a curved crack path, especially when the material consists of inhomogeneities surfaces or has different material interfaces due to holes, inclusions, microscopic cracks, etc. [85]. However, the fracture behavior is highly dependent on the material. For example, concrete cracks will usually exist when a strong pressure state is gradually distributed from micro-cracks to macro-cracks, causing a softening behavior [86] or decreasing the bearing capacity [87]. Cacks in marble tile are indicated by several defects, such as local chipping along cracking, short and long linear cracks, broken corners, local depressions, and nicks along edges [88]. These cracks are usually caused by a dull sound from the tapped slabs due to dysconnectivity to the substrate and bowing due to decohesion of calcite grains [89]. The crack in gypsum is caused by the high strain rate [90]. Last, cracks in the bituminous roof can be characterized by edge cracks that are usually caused by circumferential cracks, starting at the periphery, and gradually moving inward with radial peaks and valleys [91].

4. Result and Discussion

Having established the selected dataset, this section discusses the implementation of the proposed workflow. The analysis results are presented in three main parts, namely fracture surface analysis, materials behavior, and comparative analysis between fracture zones and materials behavior.

4.1. Fracture Surface Analysis

Figure 3 shows the point distribution of the azimuth values from normal vectors of each part of the dataset. The North and South roofs have bimodal distributions, while the North and South walls have multimodal distributions. In this regard, the azimuth on the North and South sides of the heritage building is between northeast and southwest. This makes the 0°–90° azimuth consistently dominate the trend in the histogram and, thus, we interpret it as a non-facture plane. The highest peak indicates that the point corresponds to the non-fracture plane, while the second and third largest peaks correspond to the fracture plane. In this regard, we interpret that the plane has at least two fracture clusters with different azimuth orientations. The estimated azimuth values, which range between 0° and −75° on the North wall, −75° on the North roof, 0° and 75° on the South wall, and 75° on the South roof, enable us to identify the fracture azimuthal orientation.

Figure 4 shows a pair of conjugate fractures with north–northeast (0° up to 75°) and north–northwest (0° up to −75°) directions. In principle, the conjugate joint fractures are relatively common in natural fractures. They usually consist of two or more separated parts, induced by compressive external factors. In the context of the building, conjugate fractures are often caused, not only by the compressive strength of the surrounding plane, but also by the material qualities of the building surfaces. Although parallel fractures do exist, they are relatively uncommon. For example, homogenous materials tend to form a set of parallel fractures, while heterogeneous materials produce erratic natural fracture patterns.

Figure 5 displays the distribution of fracture zones contained in each part of the building. The trend shows that the points identified in the fracture category are less in quantity compared to those found in the surface plane without fracture (refer to Figure 3). Each peak in the histogram corresponds to a particular normal vector azimuth. Another interesting pattern is the tail of the azimuth distribution in Figure 4, which shows a variety of point cloud distributions. This indicates that the surface morphology of the building is quite heterogeneous due to the unevenness of the azimuth cluster of normal vector values and, thus, it refers to the fracture plane.

According to Figure 5, some fracture directions are identified. For example, Figure 5A/(a) depicts the pattern of the conjugate fractures. This can be observed through the cluster of fracture point clouds that seem unevenly separated, although the orientation pattern is partly towards the north and northeast. Figure 5B/(a) also shows the characteristics of this pattern through the distribution of the azimuth, but to a much lesser extent. In general, the gradual pattern of conjugate fractures is also often caused by natural forces such as earthquakes. On the other hand, there seems to be another fracture direction that refers to the characteristics of parallel fractures. This can be found in Figure 5A/(b),B/(b), which present the northwest and northeast directions, respectively. In this regard, the azimuth clusters of Figure 5A/(b),B/(b) are relatively parallel, and their density is less because their loading is dominant in only one direction, especially regarding the structural load of the building itself.

4.2. Materials Behavior

This step presents a mapping of the material property on the existing surfaces of the selected dataset in parallel with the calculation of the point density for each identified surface. This information allows one to measure the specific performance of particular areas in the existing dataset. The property values are set to a range between 0.0–1.0 so that a clear comparison of material identification can be drawn from one surface to another. A predefined range is divided into four different scales to identify a significant percentage from the resulting identification.

According to Figure 6, the general trend depicts a similar pattern found between the albedo and reflectance values for all parts of the dataset. Interestingly, these properties not only illustrate the same areas but also show a similar point density for all range of the values. For example, values of albedo and reflectance ranged between 0–0.25 are consistently identified as the least amount of point density. To be more precisely, the point density of these properties only consists of less than 8% from the total points in each part of the dataset. For sections A/(a) and B/(a), this range of values mainly covers corner and furniture areas such as wooden tables, leather benches, and candles. These objects are assumed to correspond to non-continuous surfaces, which contain multiple separated geometries. Although the scanner position is close to this area, the resulting footprint emitted from the Gaussian beam is relatively scattered and, thus, the distribution of energy is not normally allocated. Moreover, the object properties seem to have a darker color that absorbs higher energy and primarily includes rough textures, which is why the surface material of these objects can be categorized as non-reflective surfaces that show a diffuse reflection and include more noise datasets.

Meanwhile, the largest density of points for albedo and reflectance values are identified differently in each part of the dataset. For example, Part A/(a) refers to a range of 0.5–0.75 (around 45% from the total points), while Part B/(a) is identified at a range of 0.75–1.0 (around 37% from the total points), and Part B/(b) and A/(b) are similarly located at a range of 0.25–0.5, with 53% and 51–55% of point density, respectively. Some areas identified by these values are influenced by the surface brightness of the dataset during scanning. This specifically can be observed in Part A/(a), with a range of 0.5–0.75. Interestingly, the tube (fluorescent) light located vertically on the corner of the wall makes its surroundings brighter than other areas (see Figure 7). This is why a red curve pattern is spotted along the North wall as a boundary of the shade light. In this case, the laser beam captures a different energy from different surface brightness. The intensity values located in the surrounding of the tube (fluorescent) light is higher than others.

When these results are compared with the elements and material types presented in Table 3, section A/(a) depicts that the densest albedo values are illustrated in the range of 0.5–0.75, with a point density of 45%. This figure predominantly fulfills the ideal value for floor materials (marble tile) with an albedo of 0.6, while walls with concrete materials are shown in the range of 0.75–1.0 for the ideal albedo value. A similar trend is shown for the ideal reflectance values of floor materials (marble tile). The range of 0.5–0.75 contains the greatest point density and is then followed by a range of 0.75–1.0.

A slightly different pattern is shown in section B/(a). The ideal albedo value is illustrated in the range of 0.75–1.0, with a point density of 51% and is followed by a range of 0.5–0.75. In this regard, the identified areas for the ideal albedo are dominated by the wall element, while the floor element dominates the second largest point density. Although the trend of point density looks similar with the reflectance values, the largest ideal point for reflectance lies in the range of 0.25–0.5, with only around 9.4% of point density. The area identified in this range is predominantly assigned to marble tile. As for the transmittance values, all sections relatively illustrate a similar pattern. The point densities for all sections are predominantly concentrated in the ranges of 0.0–0.25 and 0.25–0.5. This indicates that the surface materials of the existing dataset are similarly categorized as opaque.

According to the material property distribution plotted in Figure 6, we can further calculate the total (Δ) albedo modification in the existing dataset according to its point density. This is essential to identify specific parts that might need further actions due to microclimatic impacts (see Figure 8). To do so, the total identified areas are multiplied by the upper range of albedo values. This then yields the total increment (Δ) of albedo in a certain range of the dataset. For example, Sections A/(b) and B/(b) contain a similar pattern regarding high and albedo values. The trend shows that the highest albedo corresponds to the upper range of 0.5, which is around 0.267 or 53.4% for A/(b), while it is around 0.259 or 51.8% for B/(b). This pattern is consecutively followed by a range of upper values of 0.75 (A/(b)—0.2775 or 27% and B/(b)—0.27 or 36%), 1.0 (A/(b)—0.0617 or 6.17% and B/(b)—0.066 or 6.6%), and 0.25 (A/(b)—0.008575 or 3.43% and B/(b)—0.014 or 5.6%). As for A/(a) and B/(a), the only similar pattern of total (Δ) albedo is found in the upper value range of 0.25 with 0.01825 or 7.3% and 0.014 or 5.6%, respectively. Considering that the densest point is located at low albedo values, further treatment is required to reduce the heat absorption in those areas. For example, the Church management team or the heritage organization could install thermal insulation or modify the existing surface material to do a preventive action for specific areas. As for sections A/(a) and B/(a), the low albedo values are more visible around the furniture, not only because of the disjointed geometries, but also because of the dark color and rough material surfaces.

4.3. Comparative Analysis between Fracture Analysis and Material Behaviors of the Dataset

This section discusses the point distribution of materials based on the fracture datasets identified previously in Figure 5. The resulting material properties (i.e., albedo, reflectance, transmittance) can principally be used to calibrate the surface performance of the fracture datasets. This study selects the albedo only to demonstrate the feasibility of this proposed method. Accordingly, we can understand the potential feature of integrating fracture analysis and material behaviors for cultural heritage buildings.

Figure 9 exemplifies several interesting trends regarding the overlap between albedo (green datasets) and surface fracture (red datasets). For example, the densest albedo values vary in each section of the fracture dataset. This can be observed in sections A/(a) and B/(b) in the range of 0.75, which shows the greatest overlapping albedo points, while Sections A/(b) and B/(a) are identified in the range 0.5 and 1.0, respectively. At a glance, these identified ranges show a performance contradiction between the thermal and surface properties of the material dataset. This is because a higher albedo value indicates a smoother or lighter surface of the dataset, while the fracture dataset itself designates a higher surface porosity or high irregular surface morphology. Accordingly, by having high albedo values on identified surfaces, fractures are quite contradictory, such as in sections B/(a) (at a range of 1.0) and B/(b) (at a range of 0.75). This often occurs on flat surfaces with bright colors, due to the limited parameters used for dataset correction. In order to avoid this ambiguity, we can focus on areas with less overlapping albedo points as they clearly consist of low surface performances. Furthermore, Figure 9 also shows that areas with low albedo values not only provide us with significant information regarding vulnerable surfaces in the existing heritage building, but also strengthen our hypothesis regarding fracture datasets.

According to the results presented in this study, our proposed method may provide a significant contribution to the current framework for assessing the performance of cultural heritage buildings. In particular, the integrated analysis between surface fractures and materials behavior has confirmed its potential application relevant to existing studies, especially related to issues of structural deformation and material degradation. The incorporation of thermal performance analysis retrieved from the materials properties into the surface geometric studies completes the current assessment features for heritage buildings, which mostly focuses on physical deformation [35], single material observations [41], and lab-based measurement methods [42].

Although this study acknowledges some further considerations on simulation accuracy and point density of the dataset, this work presents a potential method for conducting the initial performance assessment of heritage buildings in Indonesia. This also includes the other 57 cultural heritage buildings [92] that may also require periodic cultural monitoring and assessments. Thus, corresponding architects or conservators can take a preventive action to minimize the environmental impacts on existing heritage building surfaces.

5. Conclusions

This study has discussed a computational framework of using 3D laser scanning technology in examining the surface performance of the existing heritage church in Malang, Indonesia. In particular, the presented study investigates attribute information (i.e., geometric and radiometric properties) stored in point cloud data, not only to identify fracture and non-fracture points, but also to calculate and map the material properties (i.e., albedo, reflectance, transmittance) of the existing surface dataset. The resulting findings ultimately allow architects or conservators to detect and analyze the microclimatic impact of existing building surfaces at an early stage before geometric defects physically occur. Several concluding remarks of the findings are specifically summarized as follows:

• Dataset preprocessing tasks such as georeferencing, outliers removal, intensity corrections, and dataset subsampling play a crucial part in this study, which is not only useful for filtering relevant information from the raw point cloud data, but also for minimizing erroneous results during the dataset measurement. However, there are some aspects to consider, such as the trade-off during the dataset subsampling of whether to have a dense dataset or maintain computational time and costs. In this regard, this study ultimately applies a spatial distance of 5 cm to enable one to perform the integrated workflow between fracture analysis and materials behavior. In addition, this study selects the angle of incidence as the correction parameter for intensity values as it is closely related and relevant to the main scope of the study. In this case, the distance effects are assumed to be maintained by the scanner due to an automatic brightness-reducer at a certain distance.
• Computing the dip angle and dip direction of the surface dataset leads us to identify fracture and non-fracture zones in the dataset. In this regard, fracture points can be detected, not only from the geometric planes, but also from the materials behavior of the surface dataset. Nevertheless, additional on-site measurements (i.e., image-based methods) may be required, not only to calibrate and validate the simulation results, but also to identify more environmental parameters that may be relevant to fortify our hypothetical results.
• It is worth noting that the workflow developed in this study only applies to a single scan dataset. This is mainly because the correction parameters (i.e., angle of incidence) require a single reference point for the georeferenced coordinate. Otherwise, each point cloud will contain multiple reference points due to multiple scanner locations after merging the dataset. This will thus create confusion for detecting the true normal value of each point of the original dataset.
• This study detects at least two kinds of fractures, namely conjugate fractures and parallel fractures. These fractures are detected through uneven distribution of azimuth clusters. This uneven distribution occurs due to several factors, such as heterogeneous materials, disjointed geometries, the structural load of the heritage building, and natural forces such as earthquakes.
• The areas identified for thermal performances, to some extent, are not always parallel to the fracture zone. This specifically happens on flat areas with homogenous materials and minimal crack propagation because the crack (e.g., holes, porosity) itself can act as an isolator that breaks the heat flux distribution.
• The total (Δ) albedo modification provides essential information regarding the heat absorption of the surface dataset.
• Comparing the albedo and fracture values in the same dataset enables us to identify and confirm the initial surface performance analysis.

While the present study shows encouraging results, further research could be of further benefit to include several aspects. For example, it is important to integrate the dataset measurement collected by TLS with a thermal camera in order to calibrate and verify temperature parameters when calculating the material properties of the dataset. Future studies may consider the use of a deep learning approach to perform a fully automated detection for the heritage building assessment and the use of additional parameters (e.g., distance range during the acquisition geometry, surface temperature) to identify very small fractures or particular cracks. Our present work will be developed into a software application to perform real-time detection of defects using vision sensors, including drones. The work will also be extended to cover other models that can detect other defects in construction, such as structural movements, spalling, and corrosion. Our long-term vision includes plans to create a large, open-source database of different building and construction defects, which will support world-wide research on the condition assessment of built assets.