How to Enhance Perception of Reassembled but Incomplete Works of Ancient Art? Eye-Tracking Study of Virtual Anastylosis
Round 1
Reviewer 1 Report
The article illustrates the results of an experiment using eye-tracking to evaluate viewers' engagement with the anastylosis of a monument. The authors examine how people look at different elements of the proposed reconstructions, assessing the impact of the various displayson the global appreciation of the object. The experiment uses the digital image of a column displayed on a monitor against a black background as a case study. Using the time spent on each element of the image, the results of the experiments offer an instrument to estimate the success of restoration work in its preparatory phase.
Although the article offers a convincing argument, there are some elements that I would like the authors to reflect on.
The authors ground their argument on the assumption that the visual perception of a digital element is equal to that of an object in the real world. Therefore, they assume that the results of their experiment, carried out in a carefully curated environment, are also significant in a real-world scenario (characterised by the variety of conditions they also describe in the final part of their article). The difference between the two experiences, tackled in the work, for example, of Haskins et al. and Hayhoe, lacks any critical consideration in the paper. This jeopardises the reliability of the findings in a real-world setting. The article will significantly benefit from an explanation of how, despite the differences, the experiment can help understand visitor experience.
In section 2. Research goals, authors claim that a decision made during the process of anastylosis can lead to “manage cultural tourism in a more sustainable way” (l. 154). This is either overstated or needs a better explanation.
The aim of the work is “to find out how different approaches to cavities in reassembled historical monuments affect the visual reactions of non-professional viewers” (l.145-147). That is, understanding the impact of missing elements in appreciating the artefact. The authors move from the standpoint that anastylosis is also an aesthetic procedure, which is correct, but they seem to underestimate the statical issues connected with reconstructing a fragmentary artefact. This significantly affects how they prepare the experiment and understand the results.
- First, the authors use integrations characterised by different luminance levels in preparing the images to show during the experiment. Is this a purely aesthetic consideration, or does it have some relation with the possible materials to be used in the actual reconstruction?
- Second, in discussing their results, the authors claim: “This study demonstrates that adding new elements during the reassembly is less preferable than leaving an empty cavity.” (l. 445). Again, it seems that the statical reasons behind the integration of missing elements are not considered when designing the intervention. Is adding an external scaffolding to support the pieces better than integrating the void? Furthermore, as I will discuss later, this claim appears to contradict the data emerging from the experiment.
In selecting the volunteers, the authors claimed to have chosen them in such a way as to secure homogeneity (l.234). What do they mean by homogeneity, and why was this relevant? Is not the diversity of samples an added value when testing a hypothesis? This needs to be further elaborated. Also, they mention the gender and age of the participants. How is this relevant to the results of the study? I suppose these data are reported to document the equality in participation rather than suggesting a difference in viewing among the genders/age groups. If this is the case, this needs to be rephrased.
In the paragraph starting at line 250, the authors describe the calibration of the tools. Elements such as ”five-point calibration” or “average error” need an explanation. It is also important to describe how they impact the overall results.
In describing the experiment, the authors say that participants were shown separator boards with instructions. Can you add some extra information on what these instructions were? Consider also adding an image. Can you also offer some insight into how these instructions inform the way of looking at pictures? In discussing your results, you seem to suggest that they impact what participants look at. Therefore, they are crucial for understanding the experiment and the validity of its results.
On page 9, the value ‘one way ANOVA is introduced without any explanation.
In line 402, you define the abbreviation TVD, but you use it for the first time in line 389.
The last part of the experiment consisted in collecting data through a questionnaire. Can you offer more information on the question asked? Can you explain how they related to the data you captured with eye tracking?
Data show that volunteers exposed to case B looked at the capital (designated as the intended focal point of the observation) significantly after than the others. They also show that they return to observe the void more often than visitors exposed to other cases did. Do these two elements not signify that they are distracted/attracted by the hole? If so, isn’t this contradicting the restoration principle that claims that the intervention should not distract from the appreciation of the aesthetic quality of the artefact? These data also seem to contradict the authors’ claim: "This is important because it tells an expert who wants to preserve the authenticity of historical buildings that leaving an empty cavity does not have to hinder the reception of the monument in its entirety."
Author Response
REVIEWER 1
•The review helped us improve the manuscript.
•The corrected file is in the attachment.
The article illustrates the results of an experiment using eye-tracking to evaluate viewers' engagement with the anastylosis of a monument. The authors examine how people look at different elements of the proposed reconstructions, assessing the impact of the various displayson the global appreciation of the object. The experiment uses the digital image of a column displayed on a monitor against a black background as a case study. Using the time spent on each element of the image, the results of the experiments offer an instrument to estimate the success of restoration work in its preparatory phase.
Although the article offers a convincing argument, there are some elements that I would like the authors to reflect on.
The authors ground their argument on the assumption that the visual perception of a digital element is equal to that of an object in the real world. Therefore, they assume that the results of their experiment, carried out in a carefully curated environment, are also significant in a real-world scenario (characterised by the variety of conditions they also describe in the final part of their article). The difference between the two experiences, tackled in the work, for example, of Haskins et al. and Hayhoe, lacks any critical consideration in the paper. This jeopardises the reliability of the findings in a real-world setting. The article will significantly benefit from an explanation of how, despite the differences, the experiment can help understand visitor experience.
•This is a very important aspect. Experiences are different, what we write about. We have not previously cited the publication Haskins, Mentch, Botch, et al. (2020), but for two others: Gulhan, D., Durant, S. & Zanker, J.M. (2021) oraz Rusnak (2022). We will add a new reference. This is an interesting article . We will try to better explain the essence of comparing several stimuli in the same environment.
In section 2. Research goals, authors claim that a decision made during the process of anastylosis can lead to “manage cultural tourism in a more sustainable way” (l. 154). This is either overstated or needs a better explanation.
•„However, this paper is by no means limited to a typical case study…” The whole paragraph refers to the promotion of the use of biometric tools to understand observers' reactions to heritage. Knowing the way people look can better plan how to share specific places.
The aim of the work is “to find out how different approaches to cavities in reassembled historical monuments affect the visual reactions of non-professional viewers” (l.145-147). That is, understanding the impact of missing elements in appreciating the artifact. The authors move from the standpoint that anastylosis is also an aesthetic procedure, which is correct, but they seem to underestimate the statical issues connected with reconstructing a fragmentary artifact. This significantly affects how they prepare the experiment and understand the results.
•Thank you. The process of anastylosis is complex and difficult to evaluate. We focus on a very narrow aspect. ET can only be used to answer questions like these. In a hypothetical situation, we examine one aspect: the presence or absence of a supplement. Anastylosis has more problems than aesthetics, as we state in the introduction.
“The scientific discussion that has accompanied individual pieces of anastylosis focused primarily on technical issues, such as the minimum amount of original material necessary to perform anastylosis (ICOMOS 1964), the type of appropriate support structures for the original elements (Torun and Ercan 2013, 38; Vacharopoulou 2006, 78), and finally whether to fill the cavities or not, and if so, what material should be used for the filling (Torun and Ercan 2013, p. 29,37-38; Vacharopoulou 2006, p. 82).”
- First, the authors use integrations characterized by different luminance levels in preparing the images to show during the experiment. Is this a purely aesthetic consideration, or does it have some relation with the possible materials to be used in the actual reconstruction?
•In this selection, students are asked to choose the most appropriate example for ET research in their opinion. In 2021, a study on different luminances of fillings was conducted (Rusnak 2021).
- Second, in discussing their results, the authors claim: “This study demonstrates that adding new elements during the reassembly is less preferable than leaving an empty cavity.” (l. 445). Again, it seems that the statical reasons behind the integration of missing elements are not considered when designing the intervention. Is adding an external scaffolding to support the pieces better than integrating the void? Furthermore, as I will discuss later, this claim appears to contradict the data emerging from the experiment.
•Thank you for pointing this out. Yes, this statement is contradictory. This error at line 446 "less" should be "more" (We are VERY sorry).
In selecting the volunteers, the authors claimed to have chosen them in such a way as to secure homogeneity (l.234). What do they mean by homogeneity, and why was this relevant? Is not the diversity of samples an added value when testing a hypothesis? This needs to be further elaborated. Also, they mention the gender and age of the participants. How is this relevant to the results of the study? I suppose these data are reported to document the equality in participation rather than suggesting a difference in viewing among the genders/age groups. If this is the case, this needs to be rephrased.
•Thank you. The use of the word homogeneous itself may be misplaced. To express our intention more clearly, we have modified this fragment.
In the paragraph starting at line 250, the authors describe the calibration of the tools. Elements such as ”five-point calibration” or “average error” need an explanation. It is also important to describe how they impact the overall results.
•The description has been expanded to briefly explain to the reader what the role of calibration is.
In describing the experiment, the authors say that participants were shown separator boards with instructions. Can you add some extra information on what these instructions were? Consider also adding an image. Can you also offer some insight into how these instructions inform the way of looking at pictures? In discussing your results, you seem to suggest that they impact what participants look at. Therefore, they are crucial for understanding the experiment and the validity of its results.
•Thank you for your comment. Hopefully, the additional illustration (currently No. 3) will fill in the gap.
On page 9, the value ‘one way ANOVA is introduced without any explanation.
•This has been supplemented.
In line 402, you define the abbreviation TVD, but you use it for the first time in line 389.
•Thank you for your attention. In its present form, the abbreviations appear earlier.
The last part of the experiment consisted in collecting data through a questionnaire. Can you offer more information on the question asked? Can you explain how they related to the data you captured with eye tracking?
•Yes. Of course. In order to make this section more understandable, we've added a bit of text.
Data show that volunteers exposed to case B looked at the capital (designated as the intended focal point of the observation) significantly after than the others. They also show that they return to observe the void more often than visitors exposed to other cases did. Do these two elements not signify that they are distracted/attracted by the hole? If so, isn’t this contradicting the restoration principle that claims that the intervention should not distract from the appreciation of the aesthetic quality of the artefact? These data also seem to contradict the authors’ claim: "This is important because it tells an expert who wants to preserve the authenticity of historical buildings that leaving an empty cavity does not have to hinder the reception of the monument in its entirety."
•They looked at the hole and the head for longer. AOI TOP observation time has hardly changed (see Table 2). However, the involvement in viewing the entire structure has increased significantly. An explanation has been developed.
•Thank you for your time and help.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
Le conclusioni dovrebbero essere ampliate e approfondite. Un Il primo tema è quello di riferire la strumentazione utilizzata ad un panorama più ampio di strumenti neurocognitivi, e rispetto a questo panorama per chiarire la importanza di quello usato. Un secondo tema è quello di commentare criticamente la limitazioni dell'esperimento condotto, in relazione all'interferenza con altri oggetti, i possibili campioni di osservatori, il tipo di oggetto Studiato. Un terzo tema è quello di delineare le possibili sistematizzazioni del studio effettuato, ad esempio estendendo le considerazioni conclusive a diversi oggetti di arte antica, a diversa illuminazione e ambiente condizioni, ecc.
Comments for author File: Comments.pdf
Author Response
•The review helped us to refine the manuscript. The corrected file is in the attachment.
The first theme is to refers the instrumentation used to a wider panorama of neurocognitive instruments, and with respect to this panorama to clarify the importance of the one used.
•General characteristics and types of tools are given. The need for this type of research has been described in previous articles (Rusnak …) that we refer to in the text. If necessary, we will describe it again in some form, but we don't want to copy it.
A second theme is to critically comment on the limitations of the experiment conducted, in relation to the interference with other objects, the possible samples of observers, the type of object being studied.
•Limitations – We refer to this both in the introduction and at the end of the manuscript.
•Participants – have been characterized precisely in the description, on the charts. This description is required by the research methodology (Holmvist et al.)
•Type of objects- The article concerns the column, but it is only a pretext for deeper considerations, other objects of art (frescoes, mosaics, sculptures…) are also given.
A third theme is to outline the possible systematizations of the study carried out, for example by extending the concluding considerations to different objects of ancient art, to different lighting and environmental conditions, etc.
•Thank you for your feedback. ET tests do not allow you to answer broadly posed questions. In our opinion, this is a major limitation of this tool. The authors, being aware of this, try to answer the following research questions in small steps. “However, there are many factors that contribute to the perception of anastylosis and they cannot all be studied at the same time – to the contrary, they ought to be tackled one by one.” We can't compare objects of different shapes in different environments - it doesn't allow us to draw conclusions. In order to be able to discuss the results, only one aspect can be changed. Luminance variability was the subject of research presented by the author in 2021 in Heritage Science.
We appreciate you reading our article.
Author Response File: 
Author Response.docx
Reviewer 3 Report
Brief summary of the paper
The article investigates the reactions of a homogeneous audience of non-experts in the field of architecture to the sight of different types of restoration of an architectural element. With an eye tracker various parameters are recorded. The aim of the study is to understand how the general public reacts to the different solutions proposed, in order to provide a tool for restorers to evaluate the aesthetic and final result of restoration choices. The detailed results of the experiment are compared, and a critical summary is provided for each one, based on the factors taken into consideration and a critical interpretation
General concept comments
The manuscript is presented in a well-structured manner. The state-of-the art is an in-depth analysis and relevant and sufficient references are provided. The research aim is clearly declared, in line with the latest publications on the same subject in the academic world. The methodology is well described and provides sufficient technical details for the repetition of the experiment. The results are well analysed and exemplified in clear graphical diagrams.
The authors clearly express the factors considered in the development of the research and the reasons that led to a simplification of the analysed variables.
The pictures, tables and graphs facilitate the interpretation of the data and provide a valuable tool for understanding the data.
The conclusions should be expanded, proposing study methodologies that include more and more factors in the production of the data (such as light, the use of different materials, the use of different techniques in filling in gaps), as the authors themselves said to be a limitation of the experiment considered in the manuscript.
The bibliography is extensive, relevant and up-to-date with respect to the academic landscape.
As a general comment, I am puzzled by the use of the expression "virtual anastylosis" when in fact the contribution speaks of different ways of filling gaps, taking an architectural element (a single column) as an example. The etymology of the word 'anastylosis' refers to the reassembly of a building or artefact using its fragments. The operation of anastylosis as described by the cited Venice Charter (ICOMOS 1964) is "the reassembling of existing but dismembered parts" of a building. The Venice Charter itself states later that "the material used for integration should always be recognisable and its use should be the least that will ensure the conservation of a monument and the reinstatement of its form". I therefore believe that the contribution refers more to the integration of gaps or missing elements that are necessary for anastylosis. Generalising the manuscript to integrations in architectural restoration, the experiment would find a wider application in all architectural restoration and also in the restoration of works of art (such as frescoes or mosaics). This is a much-discussed field of research among specialists in the field, who have limited tools to evaluate the aesthetic aspect of their design choices prior to realisation, without damaging the artefact. The use of the eye-tracker and virtual images to analyse the way a general public observes can therefore make a valuable contribution to the topic.
I would therefore recommend limiting the focus of research to the filling of gaps using virtual simulation environments, better defining the possible fields of application, e.g:
- - architectural restoration: anastylosis, restoration of historical and monumental buildings
- - restoration of works of art: mosaics, frescoes, statuary complexes
- - museum exhibitions: study of lighting, background
In relation to this, the method of creating the three-dimensional model is poorly defined in the article. In order to achieve an even more accurate aesthetic evaluation, I suggest evaluating the use of reality-based three-dimensional models, acquired with digital sensing and colour control methodologies. Although the use of AR environments or 3D models during the experiment makes data collection much more difficult (as rightly stated in the manuscript), it is always possible to use 3D models for the production of high-detail, reality-adherent rendering images of a real cultural asset. Moreover, the lighting conditions of a render can be easily modified and recorded as experiment data. Furthermore, this can also be used for entire buildings, without the presence of people or extraneous objects that could divert attention from the subject, a difficulty made explicit in the article. In this regard, I suggest referring to the vast bibliography on the application of digital methodologies to Cultural Heritage and some specific references on colour control:
Gaiani, M.; Apollonio, F.I.; Ballabeni, A.; Remondino, F. Securing Color Fidelity in 3D Architectural Heritage Scenarios. Sensors 2017, 17, 2437. https://doi.org/10.3390/s17112437
Scopigno, R.; Cignoni, P.; Pietroni, N.; Callieri, M.; Dellepiane, M., Digital Fabrication Techniques for Cultural Heritage: A Survey. Computer Graphics Forum, 2017, 36, 6-21. https://doi.org/10.1111/cgf.12781
Salleh, S.Z.; Razak, B.A, Bibliometric and content analysis on publications in digitization technology implementation in cultural heritage for recent five years (2016–2021). Digital Applications in Archaeology and Cultural Heritage, 2022, 25, https://doi.org/10.1016/j.daach.2022.e00225
Specific comments
Some specific comments are listed here.
- - The two paragraphs from lines 101-107 and 112-119 would be much more comprehensible if they were written as a list;
- - Line 138 does not make explicit which factors have been considered and which have been excluded for justified reasons. These are analysed in detail in section 6.2, general remarks. I suggest that such considerations regarding the design of the experiment be made explicit at the outset and not only in section 6.2;
- - At lines 195 and 209 two specific software are named, but without providing the full name and software version (e.g. Adobe Photoshop release 24.0.1);
- - At line 330 and 361 provide the acronym of Average Fixation Duration and Time to First Fixation, as done in the other paragraphs, while at line 398 explicit the acronym TVD (and not at line 402);
- - A brief description of the use of the one-way Analysis of Variance should be provided at the beginning of the section analysing the results;
- - At line 446 “less” should be “more”;
- - References: The entire bibliographical references section must be rechecked and adapted to the editorial rules of the journal. The citation method is not uniform, and this also applies to in-text citations. Some examples on lines 125, 455.
Comments for author File: Comments.pdf
Author Response
Brief summary of the paper
The article investigates the reactions of a homogeneous audience of non-experts in the field of architecture to the sight of different types of restoration of an architectural element. With an eye tracker various parameters are recorded. The aim of the study is to understand how the general public reacts to the different solutions proposed, in order to provide a tool for restorers to evaluate the aesthetic and final result of restoration choices. The detailed results of the experiment are compared, and a critical summary is provided for each one, based on the factors taken into consideration and a critical interpretation
General concept comments
The manuscript is presented in a well-structured manner. The state-of-the art is an in-depth analysis and relevant and sufficient references are provided. The research aim is clearly declared, in line with the latest publications on the same subject in the academic world. The methodology is well described and provides sufficient technical details for the repetition of the experiment. The results are well analysed and exemplified in clear graphical diagrams.
The authors clearly express the factors considered in the development of the research and the reasons that led to a simplification of the analysed variables.
The pictures, tables and graphs facilitate the interpretation of the data and provide a valuable tool for understanding the data.
•Thank you.
The conclusions should be expanded, proposing study methodologies that include more and more factors in the production of the data (such as light, the use of different materials, the use of different techniques in filling in gaps), as the authors themselves said to be a limitation of the experiment considered in the manuscript.
•There has been a note added. There is a need for this type of research, and new directions should be explored.
The bibliography is extensive, relevant and up-to-date with respect to the academic landscape.
•Thank you.
As a general comment, I am puzzled by the use of the expression "virtual anastylosis" when in fact the contribution speaks of different ways of filling gaps, taking an architectural element (a single column) as an example. The etymology of the word 'anastylosis' refers to the reassembly of a building or artefact using its fragments. The operation of anastylosis as described by the cited Venice Charter (ICOMOS 1964) is "the reassembling of existing but dismembered parts" of a building. The Venice Charter itself states later that "the material used for integration should always be recognisable and its use should be the least that will ensure the conservation of a monument and the reinstatement of its form". I therefore believe that the contribution refers more to the integration of gaps or missing elements that are necessary for anastylosis. Generalising the manuscript to integrations in architectural restoration, the experiment would find a wider application in all architectural restoration and also in the restoration of works of art (such as frescoes or mosaics). This is a much-discussed field of research among specialists in the field, who have limited tools to evaluate the aesthetic aspect of their design choices prior to realisation, without damaging the artefact. The use of the eye-tracker and virtual images to analyse the way a general public observes can therefore make a valuable contribution to the topic.
•This is all very good advice we added a new key word: restoration. We extended explanations.
I would therefore recommend limiting the focus of research to the filling of gaps using virtual simulation environments, better defining the possible fields of application, e.g:
- architectural restoration: anastylosis, restoration of historical and monumental buildings
- restoration of works of art: mosaics, frescoes, statuary complexes
- museum exhibitions: study of lighting, background
•We agree with the suggestion. We've made a few modifications that talk about that. The mention of the museum aspect was already in the manuscript. In addition, we changed the key word exposition to museum exhibition.
In relation to this, the method of creating the three-dimensional model is poorly defined in the article. In order to achieve an even more accurate aesthetic evaluation, I suggest evaluating the use of reality-based three-dimensional models, acquired with digital sensing and colour control methodologies. Although the use of AR environments or 3D models during the experiment makes data collection much more difficult (as rightly stated in the manuscript), it is always possible to use 3D models for the production of high-detail, reality-adherent rendering images of a real cultural asset.
Moreover, the lighting conditions of a render can be easily modified and recorded as experiment data.
•Simulation of natural conditions seems attractive, but from a methodological point of view, the presence of additional distractors will make interpretation impossible. The author's dream is to gather experience on anastylosis in such a way as to be able to apply for a grant that resolves subsequent issues in a more comprehensive way.
Furthermore, this can also be used for entire buildings, without the presence of people or extraneous objects that could divert attention from the subject, a difficulty made explicit in the article. In this regard, I suggest referring to the vast bibliography on the application of digital methodologies to Cultural Heritage and some specific references on colour control:
Gaiani, M.; Apollonio, F.I.; Ballabeni, A.; Remondino, F. Securing Color Fidelity in 3D Architectural Heritage Scenarios. Sensors 2017, 17, 2437. https://doi.org/10.3390/s17112437
Scopigno, R.; Cignoni, P.; Pietroni, N.; Callieri, M.; Dellepiane, M., Digital Fabrication Techniques for Cultural Heritage: A Survey. Computer Graphics Forum, 2017, 36, 6-21. https://doi.org/10.1111/cgf.12781
Salleh, S.Z.; Razak, B.A, Bibliometric and content analysis on publications in digitization technology implementation in cultural heritage for recent five years (2016–2021). Digital Applications in Archaeology and Cultural Heritage, 2022, 25, https://doi.org/10.1016/j.daach.2022.e00225
•Thank you very much for providing the literature.
Specific comments (•All comments are accurate - thank you!)
Some specific comments are listed here.
- - The two paragraphs from lines 101-107 and 112-119 would be much more comprehensible if they were written as a list;
- - Line 138 does not make explicit which factors have been considered and which have been excluded for justified reasons. These are analysed in detail in section 6.2, general remarks. I suggest that such considerations regarding the design of the experiment be made explicit at the outset and not only in section 6.2;
- - At lines 195 and 209 two specific software are named, but without providing the full name and software version (e.g. Adobe Photoshop release 24.0.1);.
- - At line 330 and 361 provide the acronym of Average Fixation Duration and Time to First Fixation, as done in the other paragraphs, while at line 398 explicit the acronym TVD (and not at line 402);
- - A brief description of the use of the one-way Analysis of Variance should be provided at the beginning of the section analysing the results;
- - At line 446 “less” should be “more”;
- - References: The entire bibliographical references section must be rechecked and adapted to the editorial rules of the journal. The citation method is not uniform, and this also applies to in-text citations. Some examples on lines 125, 455.
• We appreciate you reading our article.
Author Response File: Author Response.docx
Round 2
Reviewer 1 Report
Dear Authors,
Thank you for the accurate responses to previous feedback. My only suggestion before the final submission is to double-check the language.
