Utilization of 3D Digital Technologies in the Documentation of Cultural Heritage: A Case Study of the Kunerad Mansion (Slovakia)

Abstract

Preservation of cultural heritage plays an important role in society, significantly contributing to its sustainability, which is important for human development and quality of life. The issue of cultural heritage digitization has recently been the subject of increasing research and is part of the initiatives of the European Commission for the Promotion of Cultural Heritage. Therefore, it is necessary to define the stages of the digitization process, which is crucial for cultural heritage preservation. The aim of this article was to examine currently used technologies and their role in the digitization workflow, and to present a detailed historical building information model (HBIM) developed during the digitization project at the historic Kunerad Mansion. To meet these goals, we will describe the best practices for each discussed technology. Special attention is paid to data processing at the very end of the object modeling process. Additionally, particular findings alert the reader to the need to create object element databases in terms of effective point cloud modeling and present the advantages of the technologies examined over traditional ways of preserving cultural heritage. In addition to predicting procedures, cooperation at a multidisciplinary level is needed in a narrower context. On the whole, the article provides some useful suggestions for both practitioners and stakeholders.

1. Introduction

The construction of new buildings includes modern methods that allow the creation of 3D digital models and complete drawing documentation in a form that covers all the necessary steps from idea to implementation. Tools used are mostly CAD (computer assisted design) [1] for civil engineers and BIM (building information modeling) [2] software, designed mostly for architects.

The situation is different in the case of old and historical buildings that are part of the cultural heritage of the given country. Many of them are damaged by weather, natural disasters, traffic impacts, air pollution, or destructive human activity. In many cases they require partial or complete reconstruction. Detailed printed documentation of the building is usually not available. During the reconstruction process, historical photographic documentation is used but, in such cases, it must be detailed enough. It is also possible to draw blueprints from detailed painted pictures. CAD and BIM [3] software cannot be used for the abovementioned purpose, but offers reverse engineering functions to support the reconstruction process. With the help of several modern technologies within the processes of reverse engineering, a very accurate 3D digital model can be created from an existing real object. At the end of the process, printed documentation can be generated from the created model. In such case it is possible to archive the current state of the cultural monument and, additionally, all steps of the reconstruction phase, where the model is updated according to completed work, new discoveries, or unintentional changes in the structure. The resulting 3D digital models can also be used for presentation and educational purposes and, in the case of an unexpected destruction of the building, they also provide a source of information for potential virtual models. They can be moved into virtual reality and enable virtual tours not only for small VR devices but also for large CAVE (computer assisted virtual environment) projection devices [4].

The SWA (School–Work Alternation) project [5], a training project focused on cultural heritage, deals with the acquisition of technical and methodological tools for knowledge access, preservation and enhancement of cultural heritage, with special regard to diagnostic tools and information and communication technologies. It also assesses the technical–professional and crosscutting skills that students have acquired or are expected to have.

The European Commission is active in the preservation of cultural heritage, not only in its physical but 3D digital forms. One of the digital platforms is the Europeana [6], which allows people to discover historic sites and hidden gems across Europe. It provides access to more than 51 million items (including images, text, sound, video and 3D materials) from the collections of more than 3700 libraries, archives, museums, galleries and audiovisual collections across the Europe [7].

Research in 3D scanning and reconstruction covers the problem of cultural heritage digitization. The authors of [8] present creation of a 3D model using photogrammetric and geodetic measurements, data transfer to AutoCAD and Adobe Photoshop and production of a final photorealistic digital model in 3D StudioMAX. However, they do not deal with the creation of printed construction documentation.

A case study of the Museum of King Jan III’s Palace at Wilanów [9] describes the creation of a 3D model using terrestrial laser scanning (TLS) and photogrammetry. Since the building was in a good, fully functional state, it was not suitable for reconstruction activities (undertaken when no printed documentation is available and thus must be produced from an incomplete building).

A paper dealing with the digitization of the medieval castle of Haut-Andlau (Alsace, France) [10] compares geodetic, TLS and photogrammetry techniques, but provides no clear recommendation, since specific limitations were observed for each discussed method. The building consisted only of stone walls; there were no windows, doors or other decorated architectural style.

The Batawa project (Toronto, ON, Canada) [11] was very large project covering 600 hectares of an old manufacturing city with many buildings and the result provides a digital BIM archive of that area. Source data were drawings and paper copies of hand-drawn and computer-generated plans and documents with a black line, paper, digitized photographs (some from 1939) and digital and paper texts. Furthermore, TLS was used and a topographic study was created in AutoCAD Civil 3D. Other applications used in the project were SketchUp and Navisworks. The final BIM model was created in Autodesk Revit.

Data for the case study of the historical building Kurşunlu Khan in Turkey [12] were photographs, textual data on architectural and spatial characteristics, and 2D digital drawings and alterations created during construction and reconstructions. Another source was point clouds from TLS. The tool for 2D drafting was AutoCAD 2011, while for BIM modeling it was Revit Architecture 2013.

The authors of [13], who used the Mobile Mapping System to document cultural heritage, revealed that despite its great potential, human intervention is still needed to achieve the desired results, with technical and methodological limitations causing skepticism about the mobile mapping system for three-dimensional surveys.

Digital 3D modeling and visualization technologies have been widely used for more than 30 years. Despite the enormous efforts made to establish information technologies and especially 3D technologies for digital 3D modeling and visualization, the current situation is still ambiguous and the knowledge and skills in practice concerning 3D technologies pose a major challenge [14,15].

Since literary sources do not clearly identify exact reverse engineering [16,17] methods recommended for the creation of a 3D model of cultural heritage, it is necessary to consider the utilization of several methods in a specific case study.

This paper demonstrates the potential of reverse engineering methods such as terrestrial laser scanning (TLS), terrestrial (ground) (DSLR), aerial (UAV) photogrammetry, BIM, as-built documentation for cultural heritage conservation, and scientifically examines the impact and assessment of these modern technologies’ benefits throughout the process. This is important, since the combination of these technologies can contribute to saving time, maintaining security and providing high-precision digital archiving.

The advantages are explored over traditional ways of preserving cultural heritage. There may be limitations in both methods of data acquisition, so space and lighting conditions need to be considered. For photogrammetry, it is necessary to consider a sufficient degree of overlap to avoid deformation of the corners of the object. With TLS, it is important to optimize the number of scan positions to avoid unnecessary extension of the registration process of scans in the postprocessing phase and thus the achievement of an excessively dense point cloud.

The aim of this article was to examine currently used technologies and their role in the digitization workflow, and present a detailed historical building information model (HBIM) developed during the digitization project at the historic Kunerad Mansion (Slovakia).

This work is organized as follows: in Section 2, the subject of the case study and scientific problem are discussed. Then, in Section 3, the used methodology is presented, detailing three methods. TLS is described in Section 4, including the knowledge base, planning and realization phases. Section 5 deals similarly with photogrammetry, including ground and aerial data processing. In Section 6 follows data integration from TLS and both photogrammetry methods. The BIM is described in Section 7 with the resulting 3D model and as-built documentation. Finally, we draw conclusions in Section 8.

2. Subject of Case Study and Scientific Problem

The Kunerad Mansion National Cultural Monument was chosen for the case study and impact and benefit assessment. The castle was built in 1914 by Count Valentín Balestrém (1860–1920). The construction was inspired by the Art Nouveau architecture style of castles. During the Slovak National Uprising in 1944, it served as headquarters for the Second Partisan Staff of Milan Rastislav Štefánik. It was burned down on 25 September 1944. After the end of World War II, it was confiscated and became the property of the state forests department. Subsequently, in the years 1945–1948, it underwent extensive reconstruction according to the project of Dipl. Acad. Arch. Ferdinand Čapek and became a medical institute for the treatment of respiratory diseases. In 1959, the manor became part of the Rajecké Teplice spa. On 25 May 1967, it was declared a national cultural monument together with the adjoining park. During the repair of the roof on 11 March 2010, the manor house was engulfed in a large fire. Since then, it has remained closed and dilapidated. On 20 October 2018, eight years after the devastating fire, the dilapidated mansion burned down again (Figure 1) [18].

At the very beginning of the case study process, our first goal was to determine the level of detail (LOD). LOD defines the amount and degree of building information that needs to be placed in a BIM model. The determination of the LOD was complicated, as the monument was in a desolate state and was surrounded by lush vegetation. Based on the inspection and technical capabilities of the technologies we had at our disposal, the LOD value was determined to be 300.

One of the set goals was to create a 3D model and subsequently a photorealistic model, which is one of the basic pillars for visualization of unique architecture resulting from the requirements of the monument office. This 3D model is to be used for presentation purposes as well as to document the state of the building before the actual reconstruction of the monument. It was necessary to digitally record all the necessary visual aspects.

The next goal was to create a HBIM model of the building from the outputs obtained from modern technologies used during the process, where the identified materials of the building were documented and described, as well as the information related to the building itself.

The final goal was to create new detailed project documentation in the shortest possible time. It is important to note that the original project documentation did not exist and in order to follow the rules of the monument office, it was necessary to visualize as much of the historical details of the cultural monument as possible.

3. Methods

The used methodology is divided into 5 phases: planning, data collection, data processing, data integration and 3D model construction. Three methods were used within the methodology: TLS, photogrammetry and BIM. Each of the defined methods was included in the individual phases of the methodology according to the implemented project research. Some phases also interacted, while the as-built documentation was created from a geometric model. The scheme of the methodology is shown in Figure 2.

4. 3D Laser Terrestrial Scanning: Knowledge Base

Utilization of 3D laser scanners is nowadays one of the most accurate, fastest and most efficient ways of obtaining 3D data (models) of existing objects. Obtained data are the source of information for real objects’ transformation into an accurate 3D digital copy, which can capture spatial geometry and can be used for various types of analysis and further processing on a computer [19,20,21].

Terrestrial laser scanning, also known as terrestrial LiDAR (light detection and ranging) or topographic LiDAR, works by the XYZ coordinate system for many points on the ground. Laser pulses are projected to these points and the device–target distance is computed.

Laser scanning provides high accuracy when recording real-world objects. The scan result is a point cloud that represents a 3D image of the scanned objects. This technology is used in many areas of industry, but also in architecture [22].

The 3D model obtained by this process helps architects in the renovation planning of buildings and structures and brings the possibility of testing various design variants. These options are very effective in civil engineering too [23]. Designers can access data from different parts of the world and work simultaneously on new projects without the need for traveling and manual measurement in the case of potential changes evaluation.

Three-dimensional laser scanning is one of the reverse engineering techniques, which is also used in the digitization of cultural heritage objects. Reverse engineering (RE) is a way by which is possible to obtain the shape and the geometry of a given object and thus create its 3D digital model based on a physical model. The scientific and technical development of these methods brings many advantages applicable in almost the entire field of protection and visualization of cultural heritage. The possibilities of using the methods are, for example, digital twin creation, computer-aided repair, new educational tools (virtual tours and virtual museums) and much more [24].

According to the authors of [25], research and new technologies in the field of image processing, computer graphics and virtual reality have significantly progressed in creation of 3D computer images of real objects.

4.1. Selection of Appropriate Digitization Technology

There are several sources [26,27] that indicate frequently used types of digitization technologies. They enable the acceleration of the selection of a suitable technology, based on defined selection criteria. The chosen technology can influence some important parameters such as the results of digitization, time, overall price, etc.

Figure 3 shows the selection of digitization technology depending on the criteria of accuracy range and object size. Since the accuracy is an important criterion when creating project documentation, the dependence of these criteria is relevant to us. The object was measured at the beginning of the process with a Leica DISTO D510 laser distance meter, where approximate measures of 53.94 m × 27.35 m × 11.610 m (width × depth × height) were determined. Based on the specified size of the object and the accuracy we had to achieve, we have chosen the most suitable digitization technologies that met our requirements:

• Terrestrial laser scanning;
• Terrestrial (ground) photogrammetry;
• Aerial photogrammetry.

4.2. Terrestrial Laser Scanning: Planning Phase

This phase can also be called the preliminary phase of the 3D laser scanning process. It is necessary to obtain all available information about the object and its surroundings. The following requirements, listed in

Table 1. Requirements for the implementation of the case study and the planned estimate.

Case Study Requirements	Description	Planning Estimation
Project location	It is necessary to consider the influence of weather in given locality, availability, density of vegetation around the object, lighting and area conditions.	Kunerad Castle 2106, 013 13 Kunerad. The building has an access road; there is vegetation in the area that will not interfere with data collection and there is no electricity or lighting in the building.
Project type	It can determine the required level of detail, cost, time, etc.	Reverse engineering, cultural heritage, LOD 300.
Time required for the project	Thorough planning and estimation of time consuming can contribute to efficiency in solving a project.	Outside–7 h 0 Floor–7 h 1 Floor–5 h 2 Floor–6 h Basement–10 h Total: 35 h. The need to divide the scanning process into several days.
Scanning accuracy	Scanning accuracy is determined by the type of scanner.	<2 mm
Scanning object	The subjects of scanning are objects and areas respectively scanning scenery.	Outside area of the object, 2 floors with ground floor and basement.

, had to be considered.

Planned estimation of study requirements is always performed after the physical inspection of the object, which is necessary before the start of the scanning. The inspection was realized in October 2019. For high efficiency, it is important to plan the correct placement of reference points, estimate the number of scan positions and plan their correct placement to achieve the expected results. It can be very helpful if there is project documentation for the object, from which it is possible to draw the mentioned facts. In our case, however, it was necessary to create project documentation and therefore we created only a so-called site sketch, which is helpful for the registration and postprocessing process when we can get quick information about the relationship between the scanner positions. Depending on the size of the object and its complexity, the scanner positions were determined for each scenario separately. As mentioned above, the cultural heritage site was in a dilapidated state after fires and weather conditions (collapsed parts of the floor, collapsed roof, unstable parts, etc.) and it was also necessary to adjust the scanning positions accordingly. From the position of the scanner, the fact that reconstruction work will take place during the digitization was also considered. Other influences that could affect the scan quality were the dusty environment and safety.

4.3. Terrestrial Laser Scanning: Realization Phase

To create 2D project documentation, it is necessary to set two important parameters, resolution and quality. These two parameters are the most important for the whole process, because they affect the level of detail captured and the time required to obtain data from one scanner position, which is adequately reflected in the total time required to obtain all data and help in the correct process of registering individual scans in postprocessing. The first parameter, resolution, determines the point distance, which then determines the level of detail. By increasing this parameter, we will achieve the capture of a larger number of points and, at the same time, reduce their distance. Otherwise, lowering this parameter results in a smaller number of points being captured while increasing the distance between them. For our purposes, based on the required level of detail and time saving in calculating the available distance to the object of interest, a resolution of 1/4 was chosen. This represented the distance of the point 6.136 mm at 10 m from the scanner.

The quality setting primarily determines the measurement speed and ‘noise’ reduction level, i.e., increased quality value increases the measurement time that the scanner spends on each scan point, while performing multiple measurements to confirm the information and then averaging the result. In addition to the above, this parameter also uses a noise reduction algorithm that determines whether the differences in the scan points are an accurate representation of detail or noise. The algorithm compares the scanned points at a certain distance to each other and determines whether their difference is in the tolerance specified by the quality setting. If not, the scan point is removed, resulting in noise reduction. The setting of this parameter also largely depends on the scanning conditions, such as interior or exterior spaces, weather conditions, etc. To meet the condition of saving time, we tried to achieve optimal conditions and thus achieve the quality setting parameter—2×.

Based on preliminary on-site measurements, it was necessary to determine the distance between the scanner positions and the distance of the reference points from the scanner position.

Table 2. Comparison of quality and resolution parameters and average scan times.

Quality Settings	Reference Point Distance	5 m	15 m	30 m	Average Scan Time
	Resolution Setting				with RGB (min)	without RGB (min)
2×	1/4	1150	120	25	5:01	2:47
	1/8	235	22	-	3:40	1:26
3×	1/4	1175	126	27	6:48	4:34
	1/8	239	25	-	4:07	1:53
4×	1/4	1390	135	25	10:23	8:09
	1/8	200	27	-	5:01	2:47

shows a comparison of the settings of various parameters (quality and resolution) and the resulting achieved number of scan points on reference objects (spheres) at a given setting. Green values represent higher accuracy, yellow values represent lower accuracy and gray values represent low (<20) or insufficient number of acquired scan points required in the registration process. The minimum value for achieving green values is 80 captured scan points. The following columns represent the average time required to record data from a single scan position when scanning with RGB and when scanning without RGB. Each area of scenery that was scanned (see Table 1) required specific settings depending on the conditions. When scanning exterior scenery, it is always necessary to consider weather conditions at a given time. These measurements also show that in ideal weather it is sufficient to use the quality setting 2× at a resolution 1/4 and the location of reference points and targets at a distance up to 10 m.

Scanning was performed in November 2019 with the terrestrial laser scanner Faro Focus X 330, with the possibility of scanning to a distance of 330 m and accuracy of 2 mm.

To achieve the highest possible accuracy, a combination of 140 mm reference balls (spheres) and targets (checkboards) was used. When using this type of terrestrial laser scanner, it is possible to scan without the use of reference spheres and targets, which can ultimately save time and money, but at the expense of scanning accuracy of the object. To achieve higher accuracy by scanning without reference balls and targets, it would be necessary to place the scanning positions at a very short distance, which would require a much larger number of final scanner positions and much more time spent on site. On the other hand, the use of reference balls and targets will only slightly prolong the time spent on site compared to the aforementioned previous case, and will also simplify the registration process in the software environment.

Ball positions are used in data processing, where the software automatically recognizes them as reference elements. Based on them, it automatically combines scans from different angles and thus creates a complex point cloud of the scanned object. It is also important to note that there had to be at least three common reference points between two scan positions. The harmonization and coordination of individual scans within the postprocessing is enabled by the integrated GPS receiver.

The most challenging scenery in terms of terrestrial laser scanning within the entire building was the basement of the castle. There were several reasons for this, such as the unavailability of electric power, very poor lighting conditions, high humidity, areas of fragmentation, transitions between light and dark parts of the building, the occurrence of dangerous shafts, surface condensation, falling plaster, etc. It was necessary to reduce the time required to obtain data. The scanner settings were adjusted and scanning was performed without RGB to ensure that the time required for data collection was minimal. Due to the fragmentation of the premises, 61 scanning positions were performed in the mansion basement. Figure 4 shows the correspondence view and the specific locations of the scan positions after the individual scans were registered in the postprocessing. Correspondence is the relationship between the same common reference object in multiple scans. Faro Scene software (version 2019.1, FARO Technologies, Inc., Sarasota, FL, USA) uses these objects to create correspondences. The result can then be visualized and evaluated in the Correspondence View, the 3D View or the Structure View tool. Postprocessing took place also in the FARO SCENE software environment, where the individual scenarios were registered separately in individual clusters. Then, these clusters were registered in the final point cloud. Finally, filters were applied in the software environment, and unwanted points in the point clouds of individual scenarios were removed. During the scanning itself, cleaning work also took place on the object and its surroundings, which led to minor time plan changes and various noises. These externalities had to be filtered out in the final process, as shown in Figure 5. It is important to mention that during the use of the technology of terrestrial laser scanning, so-called death spots arise where the laser beam does not hit the measured point on the object. These places must be added later in the following photogrammetry process. The resulting point cloud in this phase met the requirement of LOD 300 in terms of shape, orientation and location.

5. Photogrammetry: Knowledge Base

Photogrammetry is the art, science and technology of obtaining reliable information about physical objects and the environment through processes of recording, measuring and interpreting photographic images and patterns of recorded radiant electromagnetic energy and other phenomena [28]. Photogrammetry is divided according to:

• Positions;
• Method of evaluation;
• Number of pictures taken;
• Evaluations.

Figure 6 shows the scheme of photogrammetry organization.

Modern photogrammetry technologies, which are used to create a variety of environments and tools, have undergone tremendous development in recent decades. During this period, they have passed several milestones from the visual side through the quality of textures to large-scale 3D models. The basic principle of photogrammetry [29], which also results from the above scheme, is convergent imaging. When a photogrammetric image is taken, a central projection is created. The relationship between the subject and its captured image at the time of exposure is given by a photogrammetric beam passing through the center of the projection.

The shape of these rays is determined mainly by the elements of the internal orientation [30] and their position in space is determined by six rays from the external orientation. With respect to the elements of internal and external orientation and the sufficient offset, it is possible to create a point cloud representing the digitized object. Further processing of these points results in the creation of a triangulated irregular network (TIN) model, which forms the basis for further postproduction to achieve required details. Finally, the obtained texture based on the quality of the image is applied. It is important to note that the quality of the data will significantly affect the result. The main advantages of photogrammetry are high accuracy and massive collection of special data.

Like laser scanning, photogrammetry can be used in reverse engineering. Sufficient photographs in ground and aerial photogrammetry form a good spatial model, but also more detailed information about the individual details of the object [31]. The technology can also be used in hard-to-reach and inaccessible places. Created photorealistic 3D models (PR3DM) [32] find further application in the field of construction, architecture, medicine and also in the gaming industry. Within the already mentioned architectural [33] and cultural heritage digitization [34], sufficient interoperability is ensured, which allows integral use of photogrammetry in the digitization process as one of the basic tools.

5.1. Photogrammetry General Principles: Planning Phase

During the digital processing of Kunerad manor house using the photogrammetric method, it was necessary to consider that the 3D model itself had to be created in combination with laser scanning. This meant that the model was emphasized not only in terms of visual details but also in terms of accuracy by the qualitative output of data from laser scanning. This implies that in order to achieve the required outputs from the model, basic recommendations have to be followed, based on the previous experience and knowledge of the authors:

• When capturing the subject, there is no relationship between the size of the subject and the number of shots. If there are enough photos available, more details can be captured and there is no need to return to the place and take a new picture of the object.
• If possible, capturing should be performed at the highest possible resolution and without using zoom during shooting. Our advice is to create captured images in RAW format in terms of the larger range of adjustments.
• Time, lighting, weather, haze and spatial conditions need to be considered so that the object can be captured from every possible angle and under consistent lighting conditions in the digitization process. This data can be used in combination with laser scanning.
• Try to avoid shadows, because the RGB information is lost during the 3D model creation process and the model shape is incorrectly rendered.
• When applying the convergent capturing method, it is necessary to ensure a sufficient degree of image overlap. If the overlap level does not reach the value of at least 80%, the detailing process may not be accurate. To ensure sufficient overlap between the individual pictures and adjust for their distortion, the horizontal angle should be in the range (−45°, 45°) and the vertical angle (−30°, 30°) [35].
• Pictures should not show any signs of blur due to hand movement or instability of the body and should also be taken with minimal noise. If there is the need for focus only on the subject, shooting is then performed in automatic mode. In manual mode, we gain control over the basic parameters of the camera (ISO, F, f) and a balance is sought between them through the exposure triangle.
• Capturing should be in the loop and with the same distance around the whole object if possible due to the spatial arrangement. The shooting distance should copy the shape of the subject throughout the loop. From the point of view of the vertical displacement, these loops should be made at intervals, respecting the value of the overlap. The aim is to achieve an even level of detail throughout the building.
• Try to avoid panoramic shots because with these the depth information of each point in the picture is not obtained. Therefore, it is necessary to take pictures at small distances.
• When using unmanned aerial vehicles (UAV), it is necessary to perform a control flight over and next to the building in the initial phase. Based on this flight, places where additional detailed scanning is required are identified.
• Evaluate the situation in terms of setting the UAV selection of individual modes, both automatic and manual. It is also possible to create a combination of individual modes when it is necessary to consider the time of flight, weather conditions, space conditions or the phase of reconstruction carried out at the time.
• In terms of flight speed in automatic flight mode, set the shutter speed and autofocus points so that pictures are not blurred.
• For accurate parameterization of the object, it is suitable to use alignment points with known spacing distances, or to use a measuring tape or other tool of known size.
• In order not to degrade the images, it is necessary to avoid moving objects, dust and smoke during the whole digitization process.
• The subject should be as large as possible in the photo.

According to the recommendations, it can be stated that during the processing of all available pictures, the process of postproduction is minimized, such as adjusting white balance, noise reduction, over/under exposure, and other settings. Another piece of advice is to take some time for the sorting process, when you will create separate folders that contain slides to align the object and the remaining folders to create textures.

The result of the interconnection and achieving LOD 300 is a sufficient acquisition of information about the object such as shape, articulation and structure RGB.

5.2. Terrestrial (Ground) Photogrammetry: Realization Phase

It is possible to determine dimensions, positions and properties of objects and phenomena from photographic images produced by ground photogrammetry. Capturing is done from a stable position on the ground. Secured elements of the internal and external orientation of the images allow determining the 3D position of each point [36].

After the 3D laser scanning phase, terrestrial photogrammetry of the Kunerad manor house was started (December 2019). Capturing was performed on a DSLR CANON EOS 60D with an APS-C sensor, the resolution of 18 Mpx, and EPS 18–75 mm lens. The output format was RAW format for subsequent postprocessing to achieve image consistency (exposure, shadows, overexposure or underexposure). By the combination of requirements for accuracy and spatial arrangement from the project definition, the maximum distance of 5 m from the object was specified. At the maximum resolution and distance, a pixel size value of 2 mm was determined. This parameter guaranteed sufficient additional points to the laser scan point cloud.

From the basic setting parameters, the ISO value was fixed at 100 throughout the shooting, the aperture was F8.0, and the shutter speed changed during the position and orientation of the camera.

The ISO 100 value guaranteed that the sensor would not be too sensitive to light, and at the higher value the remaining two parameters would not be compensated for and the images would not produce higher noise intensity, which would result in a significant deterioration of the 3D model [37]. The aperture number was based on the spatial arrangement, as mentioned above. The relationship between the depth of field and the distance to the object was also considered, which means that the shortest distance from the object could be at a maximum 2.5 m. The hyperfocal distance, based on the well-known parameters of the camera, lens, and distance, was 1.93 m, which ensured that all images would be sufficiently focused.

Due to weather conditions, the capturing was performed during cloudy weather and in almost no wind. These conditions ensured that the lighting of the object would be diffuse, without significant sharp shadows. During the photo shoot, image checks were performed, because, as already mentioned, the photo shoot took place during the winter and in the proximity of the building. Moreover, there was snow on the building in certain places. Snow in combination with daylight affected the camera’s basic parameters. Therefore, it was necessary to change these values and at the same time to follow mentioned recommendations when taking photos [38].

From the point of view of photography limitations, the same problem occurred as for the laser scanning, when both reconstruction and cleaning work took place during the process. It was necessary to avoid the mechanisms present during the capturing and at the same time monitor them so that they were not captured in the images.

The combination of data from laser scanning and ground photogrammetry was ensured by targets that were on the object and in its proximity range.

The total number of images created during the day with the emphasis on even coverage of the object was 1500. Photography was performed both by hand and using a monopod. Figure 7 shows the detail from ground photogrammetry.

5.3. Aerial Photogrammetry: Realization Phase

Aerial photogrammetry [39] deals with the interpretation and evaluation of digital images that have been created from aircraft or other UAVs. The term UAV photogrammetry defines a remote measurement platform, which can be partially dependent or independent, does not contain a pilot and is equipped with photogrammetric measurement systems. These include small and medium video cameras, thermal or infrared camera systems and LIDAR aeronautical systems. Within the UAV standard, it is possible to monitor and record the position and direction of sensors that are applied in a local or global coordinate system.

In our case, to create a 3D model of the Kunerad manor it was necessary to implement not only ground but aerial photogrammetry as well. The DJI Mavic 2 PRO drone with an integrated Hasselblad L1D-20c camera with resolution of 20 Mpx and 1 inch CMOS sensor was used for this phase. Captured photos were saved in DNG format for further postprocessing.

To achieve the same pixel size (GSD) [40] as with the terrestrial photogrammetry, the maximum distance between the UAV and the object was determined to be 20 m. Again, the spatial conditions were considered, where part of the forest was in this radius. For this reason, the object was captured in manual mode for the outer perimeter and from above in automatic mode.

The photography was performed in a fully automatic mode, since the histogram images were evaluated as uniform during the control flight. A total of eight images was taken from each side, from above the building, and for details at the terraces.

Since it was not possible to combine two methods of photogrammetry in one day, the weather forecast was monitored for upcoming days and the lighting conditions were evaluated for the same parameters. It was not necessary to change the flight modes and the way of flying, because the wind conditions were very favorable (approx. 5 m/s).

During the flight, attention was paid to the degree of capture details on terraces, tower, external staircase, main entrance and individual details of the architecture. The essential part was capturing of common goals for the combination of the two methods of digitization.

The total number of images created during the flight, with the emphasis on even coverage of the object, was 670. Figure 8 shows the UAV positions in which the images were taken.

5.4. Photogrammetry Data Processing

Having outputs from ground and aerial photogrammetry, image processing was started. For each of the methods, it was necessary to check the output quality of each image. The inspection was performed in Lightroom software (version 8.0, Adobe, San Jose, CA, USA). Overexposed and underexposed images, exposure, shadows, chromatic aberration and profile correction were monitored. Edited images created the base dataset for generating high-quality model textures. Produced slides were then sorted and the subset was used to align the object and the rest to texture the model. The processing of images into the 3D model was carried out in the RealityCapture software environment [41,42]. After importing folders containing slides to align the object, the first component that formed a point cloud was created. Another component included folders containing image information for textures. Both components were then merged into the form of a point cloud (60.7 mil. points), representing the object’s shapes. The maximum deviation reprojection was set at 2.0 pixels, with a resulting component average re-projection deviation of 0.36 pixels. This parameter represented the LOD for the individual object details. The secondary task was to supplement the information from already mentioned dead spots. The point cloud model reached the LOD 300 and formed the basis for additional information involvement, such as cracks, deformations, etc.

Based on Figure 9, it can be stated that if the images were completely output, the point cloud also represented the quality of the obtained data. This means that the quality of the output model for further work depends on the quality of the result obtained at the beginning.

6. Integration of Terrestrial Scanning, Terrestrial Photogrammetry and Aerial Photogrammetry

The next processing step was the integration of point clouds from both 3D laser scanning, ground scanning and aerial photogrammetry. Point cloud outputs from the laser scanner contained so-called dead spots. This means that the model was not complete, since the scanning laser beam did not receive all the information for each location in the object during the scanning process. This was caused by both the angle and the position of the scanner. However, these dead spots can be supplemented with information from ground and aerial photogrammetry. Based on this fact, there were two cloud points that needed to be combined. Exported point clouds data were imported into the open source CloudCompare environment, where the process of cleaning and alignment was performed. During the importing, the position of data was adjusted to the local coordinate system with smaller coordinates [43]. Trimming of point clouds of gross errors was performed by the segmentation function. The tool used allowed us to choose segmentation of individual entities through polygons or rectangles. If the rotation was used, it was necessary to define the polygon, since it is available only for 2D. Individual entities were represented by surrounding vegetation, fencing, mechanisms (in photogrammetry) and paved areas. Subsequently, the statistical outlier removal (SOR) noise filter was applied to each point cloud [44]. The SOR algorithm calculates the average distance to each nearest neighbor point selected. It evaluates as the noise points those points that are further than the average distance and n-times the standard deviation. In this case, the basic settings defined by the software were sufficient. Segmented point clouds could then be combined (interconnected). The reason for this was the difference in the coordinate systems of the software from which data were originally exported. CloudCompare has several ways to align boundaries, including bounding box alignment, manual point-to-point transformation and automatic alignment. Mutual points between laser scanning and photogrammetry had already been defined, so we applied alignment by using these points. To precisely specify individual common points, it is advisable that each of them carries an RGB value for each point, which will facilitate the search for points. The result of the alignment of the individual point clouds is shown in Figure 10. The histogram shows that the largest part consisted of the interconnection points with the smallest distance. The increasing distance was mostly formed by data from photogrammetry, where the laser device was unable to digitize the object. The created point cloud was exported in the structured E57 format and returned to the PR3DM generation process.

The exported point cloud was reimported into the RealityCapture software environment. The interconnection created a sufficiently detailed model represented by the mean projection deviation of 0.35 pixels for mesh formation. It should be noted that in the case of 3D model creation, the point cloud from laser scanning is used without RGB and from photogrammetry with RGB for texturing the model (function provided by the software). The point cloud forms the mesh that needs to be optimized. Performing this step is necessary since the mesh itself also contains invalid points resulting from too thick a point cloud. In this process, the predefined distance between two points was 2 mm, which, despite sufficient detail, affected the size of the object itself. With the integrated simplification function, the degree of optimization of the mesh model is considered until there is a significant loss of details. The resulting number of triangles that contained the model was 29.7 mil. Finally, unwrap was performed and the texture was applied in 4 K resolution and texture quality at the level of 80%, which gives the object a photorealistic look. The texture quality value informs us about the maximum detail that can be obtained from the images.

The result is a 3D model that met the required output quality criteria, including LOD 300, color and photorealism. The final model, shown in Figure 11, serves mainly for presentation purposes as the digital twin of the real object, but also defined the basis for further analysis and required outputs.

7. Building Information Modeling: Knowledge Base

Building information modeling (BIM) is used to design and document building and infrastructure designs. Every detail of the building is modeled in BIM. The model can be used for analysis to explore design options and to create visualizations that help stakeholders understand what the building will look like before it is built. The model is then used to generate the design documentation for construction [45].

As mentioned, digitization brings benefits [46] in time and cost savings. The same is true for BIM modeling. In our case, we used HBIM.

HBIM can be divided into two categories. The first category is the simplified model that contains elements only defined as geometric shapes. The second category includes modeling using repetitive parametric elements. The problem of creating HBIM models is the absence of these elements and libraries, but also the mapping of the real state and identification in the creation of these libraries. A combination of these methods is used to make a HBIM.

This type of documentation is significantly more accurate than traditional, manually created documentation (using measurement tape or a laser distance meter), since the elements are captured in 3D space. In the case of historic buildings, it is very important that all structures, including those that lie outside the section plane in 2D drawings, are included in the project documentation.

7.1. Building Information Modeling: Planning Phase

The planning was divided into the three stages. During the first stage it was necessary to achieve the required LOD, since it determines the quality of the other two stages and respective goals. There are several levels of LOD in the field of cultural heritage, depending on the purpose for which they are used [47]. In our case, the requested level was closely related to its utilization in HBIM model creation and project documentation.

The second stage comprised the HBIM model’s creation with the required LOD. An exported point cloud was created in the REVIT software, as it proved to be an effective tool with number of libraries that can be used in the creation of BIM [48] and in facility management (FM). In the next phase, segmentation took place for individual objects from the point cloud model. This segmentation was divided according to standard primitives integrated in the software environment.

Subsequently, the individual segments were modeled to the required level. In this process, there were two modeling subprocesses, namely the already mentioned simple shapes or using modeling tools. During this process, information from the texture of the component (cracks, deformations, etc.) was added to the model.

The final goal was to create a new project documentation of the existing state with the capture of the necessary details for the monument office and architects, to serve as a basis for the development of a project for the reconstruction of the Kunerad Mansion.

7.2. Building Information Modeling: Realization Phase

For the creation of the HBIM model, a point cloud from terrestrial laser scanning and photogrammetry was registered and cleaned. Since the target was the historical building, there were various ornaments on the facade, decorative casings, columns and other structures. The scan-to-BIM method was used, i.e., a point cloud served directly as the basis for the software in which the HBIM model was created. It was not appropriate to create the mesh or solid model from a point cloud in the CAD environment, since such modeling would only be time consuming and would not bring any benefits. Such a procedure is suitable for buildings such as production halls or industrial buildings. As mentioned several times, the emphasis was on both the accuracy of the data and the reliable capture of the existing condition of the object. Therefore, in cases when the basic dimensions of the object were directly modeled, it was no longer possible to simplify the structures either in the dimensions or in the direction (i.e., there are no parallel walls in the building).

To add more information to the HBIM, two more geomagnetic and ferromagnetic scanners were used during object scanning and photography. One scanned and determined the material composition of individual structures, the other, ferromagnetic one, scanned and determined the distribution and diameter of the reinforcement of load-bearing walls and ceiling boards (decks). Due to the poor technical condition of the aboveground floors of the building and factors such as humidity, and building materials containing metal, substrates with cavities or tiles, the required data could not be obtained.

Due to the high year-round humidity, it was not even possible to scan the basement using a geomagnetic scanner. Nevertheless, during scanning with a non-humidity-sensitive ferromagnetic scanner, the results in specific areas of the basement were very skewed. We assume that the entire measured area of the basement was made of reinforced concrete in combination with steel elements, and the data were distorted due to the high content of ferromagnetic metals. Therefore, invasive methods for the material composition determination were finally chosen (performed during the reconstruction). To conclude, at the time of the HBIM model creation, it was not possible to bring this information into the design of the model itself.

The system families of software were fully used in the creation of reinforced concrete slabs and floors. These constructions were relatively easy to identify from the scan. This was because of the fact that most of them were exposed by fire and the accuracy of technologies used in construction at beginning of the last century.

A special group of objects examined during the creation of the HBIM model was formed by walls, where it was not possible to determine their material composition, for the same reason as already mentioned. Based on experience of similar projects, it was possible to estimate with significant certainty the material composition of individual walls, and a new wall type was created for each wall thickness. The walls of the building were made of solid fired bricks and in the combination of their connections they differed only in the thickness of the plaster.

From the architectural point of view, high-precision shapes were required for vaults, so it was not possible to create a group for repetitive elements, as each vault differed in details. Therefore, vaults were created by the model-in-place system. This is a problem of HBIM where the ideal model is not achieved, but the process is reversed, and the model is an abstraction of reality [49].

Windows were defined as a group of families. Despite the fact that they were destroyed and only parts of them were preserved, it was possible to create a window with casing on a facade frontage. Casings were categorized as the wall additions and fully parametric. The door constructions were not sufficiently preserved and therefore it was not possible to capture them sufficiently into the HBIM.

The degree to which groups of objects were created reflects the level of detail (LOD).

Table 3. Level of detail of individual object structures.

Type of Construction/LOD	LOD 100	LOD 200	LOD 300	LOD 350	LOD 400	LOD 500
Wall			x
Floor			x
Roof		x
Windows				x
Doors			x
Railings balusters			x
Columns			x

shows the level of detail for individual modeled structures. The roof and door structures were devastated by the fire and therefore it was not possible to reliably capture their shapes. The pillars and railings were made of reinforced concrete, fire resistant, and could be modeled reliably. In the case of windows, it was possible to model them in a higher LOD, because during extinguishing of several fires in the building, some parts of their construction were revealed. In particular, the components of the wooden parts’ connection (expansion joints for window linings), point-precise places of windows’ attachment to the main structure and the interconnection of the wooden peripheral part of the window and the masonry structure were preserved.

A fully usable HBIM model using parametric families was a good choice, especially for elements and structures such as railings, windows and various ornaments on the facade.

After the creation of the geometric HBIM model, the creation of as-built documentation followed, in which the building object was captured in the actual state. As-built documentation captures changes in the design compared to the planned documentation. This type of documentation contains the exact dimensions of the object as in the reality. The content of the documentation is:

• Changes in individual units in the dispositional solution of the building, including changes in individual parts of structures, materials and location;
• Unexpected complications that occurred during the construction and appropriate solutions applied, including notes and dates of resolved changes;
• Attached with all related shop drawings and appendices [50].

In our case study of the Kunerad Mansion, where the mentioned documentation of construction drawings was not available, new, accurate as-built documentation was created from the HBIM model as the result of the basic specified goal. This documentation represents a digital model and is recognized as the main initial documentation. As shown in Figure 12, the documentation is highly accurate, because errors from manual measurements were eliminated. There are visible architectural elements, such as vaults, as well as the basic characteristic dimensions of the building, including the floor level.

Finally, the HBIM model of the real condition was created, together with the drawing documentation. The final situation shown in Figure 13 represents a built-up area of 736.97 m² and including the area of the terrace, the overall area is 1012.67 m². The combination of the photorealistic model and the west view from the project documentation demonstrates the achievement of a high degree of accuracy in object modeling.

Within the creation of the HBIM model, it was also possible to complete the phase that represented the reconstructed state of some parts of the manor. As the example, we present a reconstructed baluster in Figure 14, while Figure 15 shows parts of the baluster with dimensions and numbers of pieces. This information was also used for the investor and other professions involved in the reconstruction.

The HBIM model can be exported to IFC format. IFC is the universal interchange file used between individual software packages that serve only as browsers. Many of them are also available online. By using IFC, it is possible to constantly inspect the project not only in the design phase, but also in the reconstruction phase, and there is a possibility to inspect the construction from anywhere in the world. Construction companies can make notes to the HBIM/IFC model to intervene and solve problems with investors and architects. Another use is in the management of the building where the model is, whether in HBIM or IFC format, a valuable source of information.

8. Conclusions

Preservation of cultural heritage is one of the basic pillars of a country’s history and intellectual wealth. In addition to storytelling, modern technology is currently contributing to this effort to pass information from generation to generation. Cultural heritage buildings and nearby objects are often notable parts of a city and transport infrastructure. It is important to mention that the terrestrial laser scanning and photogrammetry approach has a significant impact on other research and application areas. For example, laser scanning is one of the fundamental technologies for smart mobility solutions, which integrate operation of transport systems, overall sustainability [51], and safety of traffic. Vehicles with a specific level of automation require precise information about nearby objects to make real-time decision during driving, too [52]. Approaches and technologies mentioned in this paper are substantial for creating data which can be further used within the intelligent, digital infrastructure.

The presented paper demonstrates the utilization of the specific methods of terrestrial laser scanning, terrestrial (ground) and aerial photogrammetry and BIM. The methodology of digitization is divided into five steps: planning, data collection, data processing, data integration and 3D modeling.

The individual partial goals of the Kunerad manor digitization project were reached. Results were obtained by specific steps and integrated into the solution. The main part of the result was a 3D photorealistic model, created by the combination of TLS, ground and air photogrammetry. It has become a digital twin of the real object and is used mainly for presentation purposes. It is important to mention that the defined level of detail (LOD) of 300 was reached for each method applied during the project. Another part of the result was a HBIM model, using parametric families, especially for elements and structures such as railings, windows and various ornaments on the facade. Finally, as-built documentation was created reflecting the current state of the manor, and changes in individual units of structure, materials and locations.

Based on the implemented case study and experience gained by authors during the project, some advantages and disadvantages were identified regarding the methods and workflow used.

The advantages of the presented TLS approach are high accuracy and detailed object capture, which did not require return and additional measurement. Disadvantages, on the other hand, are expensive technology, high financial risk if the technology is damaged during the project, and limited ability to use equipment in a highly dusty environment, which is typical for construction sites.

For the photogrammetry, advantages come with the use of less expensive technology, higher quality of details presented by textures, and ability to capture whole object completely (terrestrial and aerial scanning). Disadvantages include the time factor, as the photogrammetry approach requires more time to digitize objects (capture details and specific individual parts), that the process needs to be performed by a person sufficiently trained in the basic photogrammetry principles, and higher inaccuracy.

The following is a recommendation summary applicable for similar projects:

(1)

If a monument is devastated by fire, a documentary safety analysis of the monument is always required;

(2)

Interior spatial conditions can be a limiting factor in the case of a devastated object and it is necessary to consider which method to use for data collection (TLS or photogrammetry);

(3)

A 3D model obtained from aerial photogrammetry helps in the planning of reconstruction works in terms of machines and mechanisms’ location;

(4)

A BIM model becomes a necessary part of construction management for its monitoring but also for other works associated with reconstruction or renovation;

(5)

A HBIM model allows understanding of the state of the monument, the decay of individual materials, and identifies cracks and deformations.

The digitization of the building contributed not only to archiving with an emphasis on high accuracy, but also to the future direction in the field of NFT (non-fungible tokens). The creation of the HBIM model of the Kunerad manor created a basis for individual professions, so that they did not clash in the design and, finally, the pricing object was parameterized.

The outputs processed by proposed methodology fulfilled the achievement of research tasks with a comprehensive view of the issue. In conclusion, we can state that the methodology and chosen procedures can be applied to similar cultural heritage buildings in order to preserve them, whether in physical or digital form, for future generations.