Sustainable Restoration of Ancient Architectural Patterns in Fujian Using Improved Algorithms Based on Criminisi

Abstract

Based on current manual restoration methods, a better algorithm for restoring images based on sample blocks is proposed, along with a sustainable restoration technique for digital virtualization, with the aim of preserving and restoring the priceless art of ancient architectural motifs. The paper uses curve fitting to pre-process the restored photos by re-constructing their damaged borders and filling in the structural information that is absent with the aid of an enhancement of the Criminisi method. The repaired photos have improved edges that were previously blurry, fractured, and over-extended. In order to increase the dependability of the priority calculation when restoring photos and make it possible to acquire a more precise restoration order, we rewrote the priority calculation formula for restoration blocks in the Criminisi algorithm. The purpose was to enhance the aesthetics of the photographs and provide a viable and sustainable restoration technique for the restoration of ancient architectural motifs in Fujian. The Criminisi algorithm with deep learning is used in the thesis to fully restore the content, color, and texture of the architectural photographs, bringing the murals as close to their original state as is practical. In order to improve the blurry, broken, and over-extended edges of the restored images, the broken edges of the images are first repaired through image pre-processing. Then, adjustment factors are added to the priority calculation to increase the weight of the data items, resulting in a more accurate priority order while preventing the priority values from degrading quickly in the later stages of restoration. The PSNR values of the restored images were calculated and compared to those of the Criminisi method, demonstrating that the revised algorithm produces better restoration results and can effectively improve restoration efficiency while lowering restoration costs and ensuring pattern restoration sustainability. By retaining as much of the structural information of the original image as possible in the design of the network model and allocating larger weights to the structural part, this process also uses style migration in deep learning to restore the texture and color of the mural. As a result, the final image is as similar to the original image as possible in terms of content and as similar as possible to the style image in terms of color and texture. A better solution is proposed based on the Criminisi algorithm. By comparing the experimental results of the three sets of building images, the PSNR values of the priority improvement algorithm (30.26, 38.06, 39.56) were significantly better than those of the Criminisi algorithm (27.59, 37.06, 37.59), using the peak signal-to-noise ratio (PSNR) values as a reference standard. In order to determine the appropriate restoration sequence and enhance the quality of picture repair, the broken edges of the pattern are strengthened. The algorithm’s matching criteria can be applied in subsequent work to improve sample-matching accuracy and produce better sustainable restoration results for ancient architectural patterns in Fujian. It no longer requires specialized professional knowledge to reproduce the color of faded architectural photos; instead, a style migration approach is employed to recover the color and texture of architectural images. This study proposes the use of a texture synthesis method and a layered processing method through which the PSNR values of the resulting restored images calculated are superior and significantly higher than those of the sample-based method and the variational framework of synthetic images with regular texture components. We achieved the creation of an updated Criminisi algorithm-based solution that improves the quality of image restoration by fortifying the pattern’s frayed edges and determining the optimum repair order. These two techniques can be combined to improve the sustainability of restoration of faded architectural photographs for issues such as pattern breakage, color loss and fading. To achieve better restoration results for the historic architectural patterns in Fujian, the accuracy of sample matching can be increased, starting with the algorithm’s matching criterion.

1. Introduction

The sustainable restoration and recovery of motifs has long been a research priority in the field of ancient cultural studies. Traditional manual restoration methods rely on researchers’ and professional painters’ experience and skill, as well as their knowledge of historical materials, to repair missing areas, which is not only time-consuming but also extremely demanding. Previously, restoration work on the motifs was performed directly on the motifs, increasing the risk of irreversible damage to the motifs. The most distinctive Dunhuang designs, for example, are rich and colorful, with a variety of Buddhist stories, natural landscapes, pavilions, and other buildings, as well as various animals and birds, flowers and plants, flying images, and scenes of ancient working people’s lives and production; they are a reproduction of ancient customs and traditions and of great historical and cultural value (Appendix A and Appendix B). As the remaining historic subjects have a long history and have undergone a great deal of natural erosion as well as human damage, much of which is very critical, it is vital to rescue and restore them to sustainability, as shown in Figure 1.

Image restoration is the process of replacing missing parts of an image by referring to the content and information of other areas of the image for erroneous and defective image areas [1]. The goal is to preserve the image’s integrity, to ensure that it conforms to the human cognitive psyche, to ensure that the restored area’s boundaries intersect smoothly and naturally with the known area, to ensure that the image expresses a coherent form, and to minimize restoration marks. Image restoration techniques are widely used and have a high research value in a variety of applications such as medical imaging, special object removal, pattern restoration, image compression, and so on. Image sustainable restoration algorithms are classified as partial differential equation-based, image decomposition-based, or texture synthesis-based [2]. The Bertamlio Sapiro Caselles Ballester (BSCB) model, developed by Bertalmio et al. [3], uses the texture synthesis technique to repair the texture part and the BSCB method to repair the structure part; and Chan et al. [4] developed the total variation (TV) image restoration model and the curve image restoration model, which have better restoration effects on small-scale broken images.

The priority of the repair in the Criminisi algorithm will directly determine the result of the repair. In view of the phenomenon that data items in traditional algorithms will rapidly drop to zero, proposed image restoration based on NSCT in 2013 [5]. The texture synthesis-based restoration method and the image decomposition-based restoration method are more effective at restoring digital images with large defective areas. The image decomposition restoration method begins by breaking down the image into structural and textural components. The fundamental idea behind the texture synthesis picture restoration method is to identify the image block from the unbroken area that most closely mimics the lost data in order to promote culture in a sustainable way. The Criminisi algorithm is the most prominent among them. Based on this, many academics have suggested various improvement algorithms to address its shortcomings. Wu et al. [6] proposed an improved algorithm to determine the best matching block by introducing geometric distance. Veepin et al. [7] improved the algorithm’s sensitivity in processing image texture details by introducing a modulation factor to adjust the priority order.

Wang et al. [8] proposed an improved algorithm for adaptively selecting the matching block size. This algorithm produces better results because it creates a block selection mechanism by counting the number of gray levels in the image. This algorithm improves the data items in the original algorithm. The traditional Criminisi method has defects such as error repair accumulation, unreasonable patch priority design, low accuracy, and search-matching block errors. In 2015, Liu et al. [9] introduced structure tensor theory to construct a local structure measurement function and optimized match block priority and designed a new matching criterion. This algorithm greatly improves the fidelity of the image structure. When the image has a relatively large missing area, the traditional Criminisi algorithm lacks a global analysis of the image. In 2017, Vahid et al. [10] used approximate matrices to overcome this shortcoming. The basic idea is to first use singular value decomposition to obtain an approximate matrix of the damaged image and then use this matrix to re-construct the target area. The approximate matrix here is actually the grayscale matrix of the original image, in which the target area is approximated during the entire rank reduction process, thereby improving the quality of image restoration.

The paper’s main focus is on restoring ancient motifs in all of their characteristics, including finding minor damage spots, fixing damage, and restoring the color and texture of the motifs in three stages. Throughout the procedure, the Criminisi image restoration method was used. During sustainable picture restoration, the Criminisi algorithm resolves the issues of blurry edges and the incapability to precisely estimate the maximum priority repair blocks. The quality of the image sustainability restoration is enhanced by utilizing particular pre-processing techniques and data clustering [11] to restore the image’s damaged edges and by choosing the right restoration procedure.

2. Method Introduction

2.1. Criminisi Introduction

The analysis of the optimization algorithm [12] can lead to the enhancement of the restoration of ancient architectural motifs. The Criminisi method uses the image structure data as a reference for the image restoration order, allowing the restoration process to rationalize the restoration order based on the image data.

The principle of the Criminisi image restoration algorithm is illustrated in Figure 1. In Figure 2, Ι represents the entire image to be restored, Ω is the damaged area, δΩ is the edge of the damaged area, Φ is the range of known information, Ψ_p is the damaged block centered on the edge pixel point p, n_p is the normal vector at point p, and $\nabla I_p^{\perp }$ is the iso-illumination line vector at point p, where Φ = I − Ω.

The order in which the broken sections are filled is crucial to the Criminisi algorithm procedure. The implementation step is: calculate the priority.

$$P\left(p\right)=C\left(p\right)D\left(p\right)$$

(1)

$$C\left(p\right)=\frac{{{\displaystyle \sum }}_{q\in {\mathsf{\Psi }}_p\cap \mathsf{\Phi }}C\left(p\right)}{\left|{\mathsf{\Psi }}_p\right|}$$

(2)

$$D\left(p\right)=\frac{\left|\nabla I_p^{\perp }·n_p\right|}{a}$$

(3)

where C(p) is a confidence term representing the reliable information within the target block Ψ_p, and |Ψ_p| represents the area of Ψ_p with values ranging from C(p) = 0, p ∈ Φ; C(p) = 1, p ∈ 1 − Φ. D(p) is a data term representing the iso-illumination intensity function of the broken edge δΩ at each iteration, and α is a normalization factor taken as 255. In the second step, the SSD matching criterion is used to find the best match block ${\mathsf{\Psi }}_{\widehat{p}}$ and replicate it, replacing the information in the target block ${\mathsf{\Psi }}_{\widehat{p}}$. The matching criteria for the target block and the best matching block are as follows:

$${\mathsf{\Psi }}_{\widehat{q}}=arg\underset{{\mathsf{\Psi }}_q\in \mathsf{\Phi }}{\min }d({\mathsf{\Psi }}_{\widehat{p}},{\mathsf{\Psi }}_q)$$

(4)

where $d\left({\mathsf{\Psi }}_{\widehat{p}} ,{\mathsf{\Psi }}_q\right)$ is the sum of the squares of the color differences of the corresponding known pixels in the target block ${\mathsf{\Psi }}_{\widehat{p}}$ and the sample block ${\mathsf{\Psi }}_q$. Definition is as follows:

$$d\left({\mathsf{\Psi }}_{\widehat{p}} ,{\mathsf{\Psi }}_q\right)=\sum \left[{\left(I_R-I^{\prime }{ }_R\right)}^2+{\left(I_G-I^{\prime }{ }_G\right)}^2+{\left(I_B-I^{\prime }{ }_B\right)}^2\right]$$

(5)

I, I ′ correspond to the pixel points in blocks ${\mathsf{\Psi }}_{\widehat{p}}$ and ${\mathsf{\Psi }}_q$, respectively. In the third step, after completing the previous step, the confidence C(p) is updated to:

$$C\left(p\right)=C\left(\widehat{p}\right)\forall q\in \cap \mathsf{\Omega }$$

(6)

Replace the confidence level of the repaired point p with the confidence level of the new center point $\widehat{p}$ of the block to be repaired. In the fourth step, repeat the above steps until the broken area Ω = 0.

2.2. Pre-Processing

The Bezier curve is obtained by interactively determining a set of control polygon fixed points to obtain the desired curve form [13,14,15]. The curvature of the curve is changed when the endpoints at either end are moved; if the middle point is moved, the curve moves uniformly between the start and end points. This paper focuses on pre-processing images with first-order (linear) Bessel curves and second-order Bessel curves. If only two fixed points p₀ and p₁ are given, then the first order Bezier curve is represented as a straight line between two points. This can be defined as:

$$B\left(t\right)=p_0+\left(p_1-p_0\right)t=\left(1-t\right)p_0+t{p}_1 t\in \left[0, 1\right]$$

(7)

The second-order Bezier curve path is determined by the given data points p₀, p₂ (start and end points) and the control point p₁ and is defined as follows:

$$B\left(t\right)={\left(1-t\right)}^2{p}_0+2t\left(1-t\right)p_1+t^2{p}_2 t\in \left[0, 1\right]$$

(8)

The use of Bessel curves to pre-repair broken edges can effectively improve the blurred edges and broken discontinuities that occur after image repair and achieve edge re-construction effects.

2.3. Priority Improvement

In the priority formula, D(p) represents the structural information of the image. The damaged area becomes smaller as it is repaired, and the angle between $\nabla I_p^{\perp }$ and np becomes larger and more vertical.

The initial Criminisi algorithm restores the image according to the priority of the pixels at the restoration boundary, whose priority formula is given in Equation (1). C(p) decreases sharply as the restoration proceeds, while D(p) changes more slowly except for peaks at places with strong structure, thus making the trend of P(p) almost the same as that of C(p) and causing the priority order to be disrupted, thus affecting the final restoration result. Secondly, since the iso-illuminated curvature K(p) is a good description of the local features of the image, and the smoothing term L(p) is a good restoration of the image edges, this paper introduces both of them into the priority calculation formula. Finally, the weights of the data term and the confidence term are adjusted by introducing dynamic adjustment factors. In order to solve the above problem, change C(p) and D(p) to addition, and introduce weights α and β. The improved priority calculation formula is as follows:

$$P\left(p\right)=\alpha ·C\left(p\right)+\left(\beta -\lambda \right)·D\left(p\right)·\left(D\left(p\right)+\frac{1}{k}\left(p\right)+L\left(p\right)\right)$$

(9)

where $\alpha +\beta =1$. In the restoration process, the priority calculation is improved by weakening the C(p) weights and increasing the D(p) weights so that the information-rich image edges can be restored first, successfully avoiding the situation where the priority is 0 and achieving better restoration results in both texture and structure of the image. In this paper, we take α = 0.6 and β = 0.4 in the experiment.

The dynamic adjustment factor λ is defined as λ = n/N + γ, where n is the number of current iterations, and γ is the correction factor. N is usually taken as 30, 40; in this paper N = 30; γ is usually taken as 0.1, 1; in the experiment, γ needs to be adjusted according to the different types of restored images.

3. Results

The effects of painting pigments and natural corrosion on the motifs’ age will undoubtedly cause the colors to fade and fall off, and corrosion will also cause the once-intricate textures to blur. It is necessary to restore both the colors and the intricate textures in order to fully restore the historical themes and ensure its cultural survival. Merely fixing the damaged parts will not bring them back to their previous appearance.

This restoration process, if used manually, requires the painter to have a more comprehensive understanding of the historical material and, in addition to being time-consuming, relies heavily on the restorer’s personal subjective aesthetic, with results varying greatly from person to person. Therefore, there is a strong need to automate the restoration process using digital image processing and fractal-based image recognition [16].

3.1. Studies Related to Image Style Migration and Restoration

In order to achieve automated recovery of color and texture from faded and weathered patterns, it is necessary for the computer to truly understand the representation of the style and content of the image.

In this field of research, computer science methods have a long history, with early work involving non-photorealistic rendering (NPR) and texture synthesis for the sustainable dissemination of culture. In the field of non-photorealistic rendering, there are several approaches to style migration: stroke-based rendering (SBR), image analogy (IAR), and image filtering (IF). The SBR method uses a specific style of virtual strokes to style the digital image [17], and the process of conversion is performed by iteratively placing the virtual strokes on the real image to obtain a non-real image with a stroke style. This approach is extremely effective for a particular style, but the design of strokes is only specific to a particular style and is less flexible, and subsequent work has built on this by proposing different strokes for different regions [18,19]. The image analogy approach is to understand the image style by learning the mapping function between the original image and its corresponding style map and then applying the learned style map to the input image to be processed to achieve the purpose of style migration. This approach does not require a specific style for migration, but it requires a large number of style-specific images corresponding to the original image and its content structure to train the mapping relationship, and the learned style mapping is limited to low-dimensional features, which cannot effectively extract style and content. For example, this method is used to obtain a cartoon-style output, and the computation time for each image is short enough to be applied to video streams [20]. However, the design of the filter operator does not allow for the simulation of arbitrary styles, limiting its use to scenarios.

Texture synthesis addresses the difficulty of separating style and content extraction. There are two approaches to this type of research: parametric texture modeling with summary statistics and non-parametric texture modeling with Markov random fields (MRFs), both of which are based on statistical distributions. Using parametric texture modeling based on statistical distributions by obtaining information about the image statistical distribution from the target texture image, Julesz [21] modeled the texture as an Nth order statistic to be used for texture migration of the target image. Subsequent studies by Heeger et al. [22] used the response of the filter operator to analyze texture features rather than directly analyze pixels, and Portilla et al. [23] further used gradient descent with a multi-scale directional filter operator to obtain more refined results based on this. Aujol [24] proposed a method to synthesize textures by searching for similar domains in the original image by replacing them with similar domains in the texture map, based on the assumption that each individual pixel in a texture image is influenced by features in its spatial domain. Based on this method, Chan et al. [25] used fixed fields to speed up the search of similar fields and thus improve the efficiency of the algorithm.

Another problem that needs to be solved for style migration is how to reduce a given feature representation to an image, i.e., image re-construction. This problem has two solutions: slow image re-construction based on online image optimization and fast image re-construction based on offline model optimization. The slow image re-construction algorithm based on online image optimization optimizes the objective function by performing gradient descent in the image pixel space, i.e., iteratively optimizing all the pixel values starting from random noise to achieve a target result close to the expected one. This method is widely used in deep learning-based generative models, such as for digital image re-construction, but it is computationally intensive due to its per-pixel-value approach and cannot produce high-definition images. The fast image re-construction algorithm based on offline model optimization, on the other hand, is pre-trained with a large amount of data to produce a model that can be used to produce a specific re-construction result for a specific feature input, achieving offline and fast results.

With the advancement of deep learning methods in recent years, the migration of styles from other paintings to natural images via convolutional neural networks has been accomplished [26]. This method uses a Gram matrix of feature responses from each layer of the convolutional network to represent the input image style features. In this method, the authors represent the image content as a Gram matrix of feature responses from one layer of the pre-trained convolutional neural network and the image style as a superposition of Gram matrices of feature responses from each layer. In this paper, the final result is obtained by iteratively reducing the distance between the target image and the content and style maps in their respective corresponding content and style feature spaces. The experimental results of this paper show that convolutional neural networks can be used to separate style and content for arbitrary images without requiring a specific style dataset for training, greatly reducing the difficulty of style migration and improving its efficiency. This process, due to the use of high-level convolutional features, will cause a certain degree of loss of information at the bottom level of the image such as edges, lines, and other information. In order to solve this problem, the process of stylized migration considers both the pixel space and feature space, combining the filtering results of the original input image and the results of network model generation to construct a new loss function, which can solve the problem of loss of low-level image information to a certain extent [27,28].

3.2. Convolutional Neural Network-Based Style Patterns

The paper uses deep learning-style transfer as a method to restore pattern color and texture, and Figure 3 depicts the paper’s overall network model. The imitation of patterns by contemporary painters is where the style drawing originates. Re-constructing the input content image using self-encoding yields the desired image. The network model’s goal is to transfer the texture and style of the style map to the target map and save the target map’s structural information so that the final target map combines the traits of the two input images, i.e., the content is similar to the content map, and the style is similar to the style map. The style map is derived from a modern painter’s copy of the pattern. Image inpainting and balancing algorithms [29,30] may be used in the sustainable restoration of ancient building segmentation in future implementations.

The target graph is obtained by self-encoding its re-construction of the input content graph. The purpose of this network model is to migrate the style and texture of the style map to the target map, and to preserve the structural information of the target map, so that the final target map obtained has the characteristics of both input images, i.e., close to the content map in terms of content and close to the style map in terms of style.

The analysis of non-linear equations [31] and DY-type conjugate gradient methods [32] can provide further insight into the lack of restoration information for accurate algorithm enhancement, providing useful support for improving problems such as restoration ghosting [33] of ancient architectural motifs in Fujian. The final image is obtained by iteratively iterating the model through breakage repair of copies of the original and modern painter’s patterns, using sustainable re-construction of slow images based on online image optimization, and optimizing to reduce pattern loss.

$$\begin{gathered} I^{*}=\arg \underset{1}{\min }{L}_{total}(I_c,I_s,I) \\ =\arg \underset{1}{\min }\alpha L_c(I_c,I)+\beta L_s\left(I_S,I\right) \end{gathered}$$

(10)

Equation (10) is the loss between the content representation of the input pattern image and the target image, and L_S is the loss between the pattern image copied by the artist and the target image in the Gram matrix-based style representation. $\alpha$ and $\beta$ are the weighting parameters for content loss and style loss, which are used to adjust the weighting of style and content loss throughout the training process.

The squared Euclidean distance between the feature vectors of two images in a convolutional neural network’s first layer is defined as follows:

$$L_c={{\displaystyle \sum }}_{l\in \left\{l_c\right\}}\Vert {\mathsf{\Gamma }}^{\prime }\left(I_c\right)-{\mathsf{\Gamma }}^{\prime }\left(I\right)\Vert$$

(11)

where Equation (11) represents the activation representation of the image at the neural network layer, and $\left\{l_c\right\}$ represents the set of network layers in the pre-trained network used to calculate the content loss, in this case, the set of layers of the VGG16 network used.

The style loss I_S calculates the square of the Euclidean distance between the target image and the style image in the style representation, and the calculation of the style representation based on the Gram matrix is shown in Equation (12). The I_S is calculated as shown in Equation (12):

$$I_s={{\displaystyle \sum }}_{I\in \left\{I_s\right\}}{\Vert G\left({\mathsf{\Gamma }}^{\prime }{\left(I_s\right)}^{\prime }\right)-G\left({\mathsf{\Gamma }}^{\prime }{\left(I\right)}^{\prime }\right)\Vert }^2$$

(12)

In Equation (12), G is the Gram matrix used in the Gram style representation calculation and is the set of VGG16 network layers used to calculate the style loss.

4. Results of Criminisi’s Improved Algorithm

4.1. Results and Analysis of Criminisi Algorithm Improvements

In three parts, the paper focuses on the restoration of all aspects of the ancient motifs, including the detection of minor damage areas, repair of the damage, and restoration of the motifs’ colors and textures. The Criminisi image restoration algorithm was used throughout the process [34]. The Criminisi algorithm is being improved to address the issues of blurred edges and the inability to accurately calculate the maximum priority repair block during the restoration process. The quality of image restoration is improved by pre-processing the original broken image using specific technical means to strengthen the image’s broken edges in order to calculate the best restoration sequence. In the following work, the algorithm’s matching criterion will be used to improve sample-matching accuracy in order to achieve better restoration results.

In order to verify the restoration effect of the improved algorithm in this paper, Matlab R2016a is used as the experimental platform to conduct simulation experiments and analyze and compare the experimental results. The evaluation is a combination of subjective perception and objective evaluation, where the objective evaluation takes the value of peak signal-to-noise ratio (PSNR) as the reference standard, and the larger the PSNR value is, the better its restoration effect is. The image restoration results shown in Figure 4 are mainly for the improvement of priority. It was found through experiments that when α is taken as 0.2 and β as 0.8, the restoration effect is shown in Figure 4c.

The repair and restoration of motifs has always been an important research topic in the field of ancient cultural studies. Traditional manual restoration methods rely on the experience and skill of researchers and professional painters, as well as knowledge of historical materials, to repair missing areas to achieve the sustainable dissemination of ancient architectural motifs. This method of restoration is not only time-consuming but also extremely demanding on the worker. Earlier restoration work had been carried out directly on the motifs themselves, increasing the likelihood of irreversible damage. As can be seen from the experimental results shown, the original algorithm did not work well for restoration. In Figure 5b, there is a significant edge break discontinuity after sustainable repair.

There are also three similar problems in Figure 6b. However, from the restoration results and the PSNR values indicated in Figure 6c and Figure 7c, the algorithm in this paper solves the problem well and achieves good restoration results.

The faded, moldy, peeling, and cracked images can be repaired by pre-processing the frescoes before restoration and filtering out the distracting information from the severely damaged images using the background removal, followed by color restoration of the optimized frescoes using the improved Criminisi algorithm scheme. The results of the three sets of comparative tests show that the resultant image is smoother, retains some useful color information, and is more uniform when fusing the colors, avoiding unclear edges and giving better results after restoration. The results are also acceptable when segmenting more severely damaged colored paintings. We have analyzed the color pigments commonly used in antiquity and the causes and processes of their discoloration or fading and have organized and divided the fresco colors in an RGB color model based on color correspondence. The choice of different color spaces determines the outcome of the segmentation of the colored images and, as different combinations of color spaces are used for different purposes, their characteristics are constantly changing as a result. In practice, a variety of color spaces can be selected, and when color distributions that can produce singularities are encountered, a better restoration method is used for the color part, so that the restoration results are in good agreement with human vision.

We compared the different peak signal-to-noise ratios using the data in Figure 5, Figure 6 and Figure 7. Figure 8 shows the comparison of the peaks for the three models. As shown in Figure 8, the density network has the highest peak signal-to-noise ratio, indicating that our proposed model has the best segmentation.

Similarly, the final loss function was calculated using Equation (2) with a slightly larger value to retain more of the original building images in order to retain more sustainable structural information. To complete the restoration of the building image and compare it to the artist’s copy, the thesis combines the earlier sections of subtle breakage detection, manual mask addition, breakage repair, and color and texture restoration. The result is the graphic in Figure 8. The effectiveness of the thesis method is demonstrated by the fact that the sustainable restoration of the architectural image by the textual method yields a very close match to the artist’s copy, and even the patterns in the corresponding areas are well repositioned.

4.2. Analysis of the Results of the Style Recovery

The result of the migration is heavily influenced by the choice of L_c and L_s used for the {l_c} and {l_s} calculations. For the choice of {l_c}, the use of the feature representation in the lower layer of VGG16 preserves more structural information, such as edge information, while the choice of the higher layer preserves more structural information about the image as a whole, and the choice of {l_s} is generally the set of all layers in order to make the stylistic information more comprehensive.

The lowest layer of feature responses was used in the thesis for the restoration of the color and texture of the building images because we only wanted to alter the color and texture aspects of the building images and did not want to produce a final image that would significantly change the structure and lines of the building images themselves. The resultant plots created by choosing the feature responses of the various layers were compared in the studies to better visualize this selection process. Figure 9 and Figure 10 illustrate how the disparity between the structure of the produced maps and the original input building image increases with the number of layers selected.

Figure 9 and Figure 10 display the restoration of the detail region from the composite image and the original image, respectively. The results (c–f) will be impacted by the response characteristics of different layers because the restoration region is so small (f). The accuracy has gradually improved, while the number of selected layers has gradually increased. As a result, the results are further analyzed using a quantitative method. Figure 9 and Figure 10 display the PSNR and SSIM values for each method’s repair outcomes.

Additionally, when calculating the final loss function using the formula in Figure 3 to preserve more of the original architectural image, the value is slightly increased. This is in order to retain more structural information in the original architectural image. The thesis then completes the restoration of the architectural image, compares it to the artist’s copy work, and integrates the minor damage detection, manual mask addition, damage repair, and color and texture restoration previously carried out. Figure 7 displays the final graph. As can be seen, when the text method is used to restore an architectural image, the restoration outcome can be very similar to the copying result of the artist, and even the pattern of the corresponding area can be more effectively transferred, demonstrating the method’s efficacy in our paper.

Figure 9 and Figure 11 demonstrate how the PSNR value of the repair result image obtained using the texture synthesis method and the layered processing method in this section is significantly higher than that of the sample-based method and the variational framework for synthetic images with regular texture components. The sample-based method’s closeness is primarily due to the matching error in the complex texture image. Other than the sample-based method, there is little difference in the PSNR of the restoration results for the actual true color image. The technique suggested in this synopsis produced the highest PSNR value.

With the exception of the sample-based method, the SSIM value is still very comparable to the values of other methods for synthetic images with regular texture components, as shown in Figure 9 and Figure 10. The text in this section has been removed to make room for the actual true color image. Compared to other methods, this method has produced better results.

The PSNR values of the resultant restored images by the texture synthesis method and the layered processing method in this sub-section are much greater than those of the sample-based method and are close to those of the variational framework-based method, as shown in Figure 9, for synthetic images with predominantly regular texture components, owing to the matching errors that occur with the sample-based method in complex texture images. There is little difference in the PSNR of the restoration results for actual true-color images between the methods other than the sample-based method, with the proposed method achieving the highest PSNR value. The SSIM values for synthetic images with predominantly regular texture components are still very close to those of all methods except the sample-based method, while the paper’s method achieves better results than all other methods for text removal from actual true-color images.

In practice, the image to be restored frequently lacks a corresponding copy, and a computer method is used to generate the corresponding restoration results to assist the restoration practitioner. To demonstrate the efficacy of the thesis method in this case, several images of the building to be restored were chosen for restoration without corresponding copies, and the results are shown in Figure 10 below, where the images to be restored and the selected style images have the same semantic information. In Figure 9, however, the image to be restored and the style image have no semantic similarities at all, but they are from the same genre and time period and have similar color textures.

To recover faded, blurred textures from architectural images, a deep learning-style, migration-based approach is used. The combination of image re-construction and content and style extraction enables better migration of style textures from the painter’s copy of the architectural image work into the architectural image to be recovered, resulting in better results. The paper’s results of architectural image processing also demonstrate the method’s effectiveness by displaying the final results that can be obtained by the thesis for the restoration of architectural images.

We hope that the restoration of the color and texture of the architectural image will only affect the color and texture, and the resulting image is used in the paper because it is not anticipated to significantly alter the structure and lines of the architectural image itself. To accomplish this, the lowest characteristic response is used. In the experiment, the outcomes of selecting the distinctive responses of various layers were compared in order to more clearly illustrate this selection process. Figure 9 and Figure 11 show that the difference between the final image and the original input architectural image increases with the number of layers selected.

Using the data in Figure 9 and Figure 10, we compared the various peak signal-to-noise ratios. Figure 12 depicts a comparison of the four models’ peaks.

The density network has the highest peak signal-to-noise ratio, as shown in Figure 12, indicating that our proposed model has the best segmentation.

We used the data in the right-hand side of Figure 9 and Figure 10 to compare the different SSIMs. The results of the SSIM comparison between the two models are shown in Figure 13. The SSIM values for the four models are lowest for the c chart in Figure 8. c is 2.51% lower than d, d is 3.20% lower than e, and e is 0.3% lower than f for the Phoenix Chart. c is 7.671% lower than d, d is 0.44% lower than e, and e is 0.5% lower than f for the Dragon Pattern.

4.3. The Value and Effect of Restoration of Ancient Architectural Patterns in Fujian

Using artificial intelligence color image segmentation and restoration technology for a few ancient building patterns in Fujian, we can observe the learning effect from Criminisi, priority improvement, and deep learning-style transfer in artificial intelligence. From Figure 14, we can see that the combination of the two methods can mean the restoration of the original ancient architectural patterns in Fujian is better. A1, B1, C1, D1 are the original images of ancient buildings, and A2, B2, C2, D2 are the restoration of ancient architectural patterns using Criminisi, priority improvement, and deep learning-style transfer method. We used Criminisi and priority improvement to repair the shape and texture of the pattern and used the deep learning-style transfer to restore the fade, noise, and texture to restore the color to a certain extent, while retaining the artistic effect of the painting.

To enhance the impact of the integration of various deep learning methods on color and shape virtualization on historic architectural patterns, this paper proposes the artificial intelligence-based Criminisi, priority improvement, and deep learning-style transfer method for color and pattern virtualization [35]. The ancient architectural patterns of buildings in Fujian are the focus of this paper, which also examines the scientific issues and key technologies related to the color virtualization of these patterns. The issue of how to more effectively describe and extract the structure, texture, and color information contained in the ancient architectural patterns in Fujian is resolved in this paper. It also addresses how to design a useful computer restoration method suitable for the virtual color restoration of ancient architectural patterns. Then, in order to test solutions for the issues of color fading and blurred texture, a method based on artificial intelligence for color and pattern restoration was conducted. Combining image re-construction with content and style extraction can more effectively transfer the artist’s copy of the pattern’s style and texture to the mural image that needs to be restored, restore it in a highly realistic and accurate manner, and retain the texture structure of the input mural image as well as the artistry of painting. Figure 14 shows how old buildings in Fujian were image-segmented and sampled using artificial intelligence for color and image restoration.

5. Discussion and Conclusions

This paper improves the algorithm by pre-processing the restored image, using curve fitting to re-construct its broken edges, filling in the missing structural information and improving the blurred, broken, and over-extended edges of the restored image; secondly, it redefines the priority calculation formula for the restored blocks in the Criminisi algorithm, increasing the weights of the data items to improve the reliability of the priority calculation when restoring the image; thus, a more accurate restoration order is obtained. The restoration of ancient motifs can be restored to their original appearance as far as possible to achieve their sustainable development, and the experimental results show that the method used in the paper can obtain better restoration results.

Ancient architectural motifs have a unique aesthetic and research value. However, due to their age, they are inevitably subject to varying degrees of natural and human damage, and their sustainable restoration has been an important research topic in the field of ancient cultural studies. Traditional manual restoration methods, which rely on the experience of researchers and painting techniques, are time-consuming and place high demands on the restorer. With the development of computer technology, digital image processing has provided a powerful tool in this field. By digitally capturing and digitally restoring patterns, it is possible to provide guided restoration solutions for sustainable physical restoration, reducing the difficulty of restoration, improving restoration efficiency, and reducing reliance on human labor. The thesis uses a deep learning approach to restore the content, color, and texture of the patterns in a comprehensive manner, restoring them to their original state as far as possible. Future applications of auto-encoder-extreme learning techniques [36] may involve the environmentally responsible restoration of historic structures. Optimization algorithms [37] have proven to be useful in solving real life bargaining problems, which may have a strong relation to image restorative procedures.

In order to better preserve and achieve sustainable restoration of precious ancient architectural pattern art, an improved solution based on the Criminisi algorithm is proposed for the problems of blurred edges and inability to accurately calculate the maximum priority restoration blocks in the pattern restoration process. Through comparative analysis of the experimental results, the algorithm in this paper achieves good restoration results. It strengthens the broken edges of the pattern and achieves the purpose of calculating the best repair order as a way to improve the image repair quality. Three dimensional modelling [38] of the building structure may be implemented to assist in the evaluation of the image repair and restorative effects.

In the following future work, the matching criterion of the algorithm can be used to improve the sample matching accuracy and obtain better and sustainable restoration results for the ancient architectural patterns in Fujian. With the advancement of modern computer technology in hardware and software, the digital image processing method has become a powerful tool for this subject’s research work. It can provide guiding restoration solutions for solid restoration through digital collection and digital restoration of patterns, which greatly reduces the difficulty of repair and restoration, improves efficiency, and reduces reliance on manpower. With the current deep learning method’s maturity, it can automatically learn the structure, texture, and color information of patterns from a large number of its own patterns and apply it to the repair, greatly reducing the requirements for researchers and achieving relatively perfect repair results.

The method focuses on the sustainable virtual restoration of ancient Fujian architectural motifs using an improved algorithm based on Criminisi. For a more accurate reflection of the method’s general applicability, selected pictures of historic structures in Fujian were chosen for sensory evaluation. The effects of color restoration, the degree of image blurring, the degree of color modification, structural deformation, and distortion phenomena were assessed and taken into account. The method put forth in this paper is fairly robust in terms of both sensory and objective evaluation indicators. The objective of image processing is to optimize images, and for the color-restored mural images, a series of optimization techniques are used to enhance the color restoration’s visual impact. This will serve as a solid foundation and make a significant contribution to the restoration of ancient architectural motifs in the future. Deep learning techniques must be used for the efficient conservation of historic architectural motifs, and restoration methods must be widely used.