Investigating the Relationship between Users’ Behavioral Intentions and Learning Effects of VR System for Sustainable Tourism Development

Round 1

Reviewer 1 Report

The paper seems written well. The developed tourism system and the hanting game seem useful as education contents to learn the rusin of Saint Pauls's. However, I can not understand the merit of the paper as a research paper. For example, there are not any technological aspects about the developed tourrism system and the hanting game. I can understand PLS-MGA is one of the useful method to evaluate systems based on user experiments( questionnaire survey ) . If the authors want to insist the merit of applying PLS-MGA, I think the authors should compare it with other analysis methods in the another experiment besides your tourism system and hanting game.

Author Response

Reviewer 1

The paper seems written well. The developed tourism system and the hanting game seem useful as education contents to learn the rusin of Saint Pauls's. However, I can not understand the merit of the paper as a research paper. For example, there are not any technological aspects about the developed tourrism system and the hanting game. I can understand PLS-MGA is one of the useful method to evaluate systems based on user experiments( questionnaire survey ) . If the authors want to insist the merit of applying PLS-MGA, I think the authors should compare it with other analysis methods in the another experiment besides your tourism system and hanting game.

 

Q1: I can not understand the merit of the paper as a research paper. For example, there are not any technological aspects about the developed tourrism system and the hanting game.

ANS 1: Thanks very much for your kind work and consideration on publication of our paper. We would like to express our great appreciation to reviewer. We have added contents from lines 407-418, the details are as follows:

In the first stage, digital data were collected, and Photoshop and Illustrator were used for the visual interface design. Subsequently, 4DKanKan cameras, 3DMAX, and the C# programming language of Unity3D were used to design VR and AR interactive content. Then, relevant 3D scene data and interface images were uploaded to a network server for execution on computers or smartphones. The operation methods included two types:

1. Users could download the APP to their smartphones to watch and learn at any time in the Ruins of St. Paul’s scenic area; they could also use VR devices to experience at home.
2. The subjects could use computer-related equipment (mouse, keyboard) in the laboratory and operate VR in a fixed space for experiential learning or scan a QR code to operate on their smartphones. Relevant data was sent back to the server for recording.

 

Q2: I think the authors should compare it with other analysis methods in the another experiment besides your tourism system and hanting game.

ANS 2: Thanks reviewer for good suggestion, and we have added contents from lines 575-582, the details are as follows:

4.5. ANOVA Comparison Test for Different Groups

Table 8 shows the ANOVA comparison test for the VR tour group and the AR treasure-hunting group. The findings indicate significant differences in three variables between the two groups. The AR treasure-hunting group in this study scored significantly higher than the VR tour group in behavioral intentions (BI), learning effects (LE), and users’ learning satisfaction (ULS). Scheffe's test data also indicated AR>VR.

Table 8. ANOVA Comparison Test for Different Groups

	VR tour group VS AR treasure hunting group
	VR tour group (n=30)		AR treasure hunting group (n=30)		F-value	Scheffe's test
Variables	M	SD	M	SD
Behavioral intentions (BI)	71.41	8.22	87.51	5.41	8.22***	AR>VR
Learning effects (LE)	80.25	7.16	88.56	4.21	12.52***	AR>VR
Users’ learning satisfaction (ULS)	72.51	6.22	86.32	5.71	13.24***	AR>VR

***p<0.001

Author Response File: Author Response.pdf

Reviewer 2 Report

Dear authors thank you for interesting read on service design, behavioral intentions, and learning effects for sustainable tourism development. The findings of this study indicate that the majority of users accepted the proposed innovative mode of operation and found it entertaining. Augmented reality is not constrained by space or time; however, virtual reality devices that operate too slowly, switch too frequently, or have too many functional interfaces may cause operational issues. This study identified and modified the influencing factors and issues of the proposed system in order to continue expanding the applications of 4DKanKan to other cultural attractions and museums in the future. While paper is well written I found several issues that should be addressed:

 

1. The purpose sounds like a task and should be rephrased and better connected to the research gap in the field.

 

2. Virtual reality and augmented reality should go before metaverse section in theoretical part

 

3. Readers of the manuscript would really benefit from limitations of the study which this paper lacks.

Author Response

Reviewer 2

Dear authors thank you for interesting read on service design, behavioral intentions, and learning effects for sustainable tourism development. The findings of this study indicate that the majority of users accepted the proposed innovative mode of operation and found it entertaining. Augmented reality is not constrained by space or time; however, virtual reality devices that operate too slowly, switch too frequently, or have too many functional interfaces may cause operational issues. This study identified and modified the influencing factors and issues of the proposed system in order to continue expanding the applications of 4DKanKan to other cultural attractions and museums in the future. While paper is well written I found several issues that should be addressed:

 

Q1: The purpose sounds like a task and should be rephrased and better connected to the research gap in the field.

ANS 1: Thanks very much for your kind work and consideration on publication of our paper. We would like to express our great appreciation to reviewer. We have added contents from lines 116-123, the details are as follows:

This study's specific objectives are as follows:

1. Analyze the feasibility and acceptance of applying VR interaction and design

models for developing cultural tourism experience systems.

2. Explore the system and information quality, and user satisfaction of cultural

tourism experience systems for users.

3. Construct a PLS multigroup structural model to explore and analyze the

degree of influence and explanatory power of system quality, information quality, behavioral intention, and learning effects among themselves.

 

Q2: Virtual reality and augmented reality should go before metaverse section in theoretical part.

ANS 2: Thanks reviewer for good suggestion, and we have changed Augmented Reality and Virtual Reality before Metaverse from lines 125-188. The details are as follows:

2.1. Virtual and Augmented Reality

At present, virtual reality (VR) and augmented reality (AR) are widely applied in simulations, including applications related to cultural heritage, archaeology, museums, games, medicine, military services, and entertainment [13]. Tussyadiah et al. (2018) found that the increased sense of presence can enhance the enjoyment of users when operating VR [14]. In terms of the use of AR, dynamic language prompts in digital content can improve the willingness of users to pay for services, especially when the enhanced environment creates a strong sense of presence. In addition, the effectiveness of AR has also been validated in many educational studies [15, 16]. As AR technology becomes more common, many AR functions have been developed simultaneously with situational learning theory, inquiry learning theory, and collaborative learning theory. Moreover, AR-based systems are more suitable for lower-level cognitive learning tasks. Based on the interview data, this study proposed a three-stage context-aware ubiquitous learning approach (CAUL) with the functions of tag searches, image tracking, and panoramic recognition by combining 3D animation with wearable AR devices [17].

Bozzelli et al. (2019) implemented ArkaeVision to integrate AR and VR. Their main goal was to use virtual technology to create sustainable cultural resources as well as provide users with a new way to experience cultural heritage. Through a user-centered extension, Bozzelli et al. (2019) designed exploration games using 3D environments that included virtual reconstructions of ancient buildings, science fiction elements, and engaging story content. The platform used VR to explore the temple of Hera II Paestum and used AR to explore the tomb of the diver. Users could actively generate motivation and obtain a good experience in ArkaeVision through gamified participation. The ArkaeVision project also provided higher benefits to cultural education in the learning process [18]. The gallery of the Victoria and Albert Museum in London, U.K. has created a virtual museum with interactive VR and AR applications, and it encourages users to use VR and AR to manipulate 3D artifacts. Sylaiou et al. (2010) found that when users switch between virtual spaces and real spaces, they perceive a stronger sense of immediacy, and that the cognitive impact of VR and AR is positively correlated [19]. Jung et al. (2016) explored the impact of using VR and AR on visitor experiences in mining museums. The research results show that in a mixed environment containing VR and AR elements, social contact exists in the four experience domains of education, aesthetics, entertainment, and escape. Except for the aesthetics domain, these domains can be verified and extended to predict the overall tourism experience rating and re-visit intention [20]. The above findings suggest that today’s games, events, and exhibitions are increasingly focused on placing users in a single space for multi-device experiences. After tricking the brain with images, users interact with virtual objects or environments through a variety of technologies, such as stereo displays, motion tracking devices, input devices, and mobile computer platforms, for visual perception, motion sensing, and interactions.

2.2. Metaverse

The metaverse is a popular topic in the science and technology field right now, mainly due to the COVID-19 pandemic, which has given many companies the opportunity to nurture and develop immersive experience technologies. It provides a 3D space that allows users to conduct natural activities in a virtual environment, including working, playing, socializing, and other virtual experiences [21]. Recently, a number of scholars have begun to study issues related to the metaverse. Jaung (2022) used a virtual forest space to engage college students in recreational activities so as to understand the potential interactions between technology, society, and ecosystems. Jaung’s results show that the users enjoyed experiencing leisure activities in the virtual forest, and this interaction mode affected their interaction with nature [22]. Blockchain technology is a key component of the metaverse and represents an opportunity for developing countries to expand their market and promote a sharing economy [23]. Based on blockchain’s professional authentication, Gemiverse Limited Corporation developed the Gemiverse travel platform and set three development stages for users to experience the platform. Its provision of the travel platform has not only brought new opportunities for tourism but also provided a trustworthy platform for both travel agencies and tourists [24]. In order to discover the gradual change in the form of tourism, Martins et al. (2022) conducted a project based on Amiais, a village in central Portugal, which used the Second Life metaverse to reconstruct the village. This project has introduced the area to the whole world, improved the environment of cultural heritage and intangible cultural heritage, and promoted tourism in rural areas [25]. From the above literature, it could be found that the metaverse is still developing. Moreover, through blockchain technology, multi-development can be achieved by incorporating games, learning, and incentives to promote cultural value, user experience, and sustainability.

 

Q3: Readers of the manuscript would really benefit from limitations of the study which this paper lacks.

ANS 3: Thanks reviewer for good suggestion, and we have changed Augmented Reality and Virtual Reality before Metaverse from lines 754-766. The details are as follows:

5.3. Research Limitations

This study's limitations pertain to samples and sites. The main experimental activity involved inviting users to the laboratory for VR operations. Although there were also external observations of AR user operations, the number of users was limited by available personnel and time. Therefore, in the future, the experiment and observation time can be extended. In addition to the preset experimental subjects, research can also be conducted to investigate the users’ differences among various ethnic groups, such as foreign tourists and local residents. Furthermore, the memorial archway of the Ruins of St. Paul covers a wide area; some intangible aspects of cultural heritage around this site were not included in the virtual environment due to the limitations of this experiment. In the future, we can make additions to make the system richer and more complete. Future projects can also extend the content to include other cultural heritage sites in Macao for virtual content design and make exploration and comparison through this model.

Author Response File: Author Response.pdf

Reviewer 3 Report

 

Thank you for giving me the opportunity to review this paper. Overall, the paper focused on an interesting topic. Below some comments which I hope that the author(s) will find useful for further improving this study.

- I found the title of the paper broader than expected. I suggest using a title which is more focused on the actual goal of this study. I also suggest avoiding abbreviations such as PLS-MGA in the title.

- The abstract should also reflect the methodological aspects of this study (e.g. research design, sample, data collection and analysis methods followed).

- The keywords seem quite generic – some more specific keywords could be also used (e.g. 4DKanKan technology, Ruins of Saint Paul’s)

- In the introduction, at the last paragraph, the author(s) explain(s) that the purpose of this research was to use a 4DKanKan camera to take 3D images of the Ruins of Saint Paul’s in Macao and its surrounding area to construct a virtual scene. Beyond the stating the research purpose, the author(s) should also add 1-2 paragraphs in the introduction to acknowledge other relevant studies in the field and explain how this work differentiates from and extends prior research. 

- In the background of the study, the author(s) covered various aspects related to the topic of their study. However, I felt that this theoretical background should result in the problem statement driving this study as well as in some clear research questions. Towards this end, I suggest adding a section after “Background” and before “Research Method” e.g. titled as “Problem statement & Research Questions”.

- Importantly, in the theoretical background of their study the author(s) should introduce, present,  and contextualize the “research framework of behavioral intentions and learning effects of sustainable tourism development” which they investigate in this study. They should also explain the need and the motivation behind investigating this framework.  

I would suggest renaming section 3.1 into “Procedures”. Then, the prior paragraph (right after “3. Research method”) could have the sub-head “Research design”. I would also suggest adding a sub-section dedicated on the participants/sample of the study.

-In the data analysis section, beyond presenting the statistical tests deployed, the author(s) also present their findings. I think that the paper would benefit by presenting separately the data analysis from the findings. The findings should be presented in a separate section on their own. Also, the findings could be structured in sub-sections according to the research questions guiding this study.

-Finally, the discussion could be also structured according to the research questions guiding this study.  

 

Author Response

Reviewer 3

Thank you for giving me the opportunity to review this paper. Overall, the paper focused on an interesting topic. Below some comments which I hope that the author(s) will find useful for further improving this study.

Q1 - I found the title of the paper broader than expected. I suggest using a title which is more focused on the actual goal of this study. I also suggest avoiding abbreviations such as PLS-MGA in the title.

ANS1: Thanks very much for your kind work and consideration on publication of our paper. We would like to express our great appreciation to reviewer. We have revised the title of the paper to” Applying Multigroup Analysis to Partial Least Square Structural Equation Modeling in the Service Design, Behavioral Intentions, and Learning Effects of Sustainable Tourism Development” from lines 2-5.

 

Q2 - The abstract should also reflect the methodological aspects of this study (e.g. research design, sample, data collection and analysis methods followed).

ANS2: Thanks reviewer for good suggestion, and we have added contents from lines 30-34, the details are as follows:

We collated and analyzed relevant data and text materials through systematic testing, observation, operation processes, and semi-structured interviews. The PLS multigroup structural model was used to explore and analyze the degree of influence and explanatory power of system quality, information quality, behavioral intention, and learning effects among themselves.

 

Q3 - The keywords seem quite generic – some more specific keywords could be also used (e.g. 4DKanKan technology, Ruins of Saint Paul’s)

ANS 3: Thanks reviewer for good suggestion, and we have revised the keywords to virtual reality; partial least squares multigroup analysis; sustainable education; behavioral intention; learning effects from lines 42-43.

 

Q4 - In the introduction, at the last paragraph, the author(s) explain(s) that the purpose of this research was to use a 4DKanKan camera to take 3D images of the Ruins of Saint Paul’s in Macao and its surrounding area to construct a virtual scene. Beyond the stating the research purpose, the author(s) should also add 1-2 paragraphs in the introduction to acknowledge other relevant studies in the field and explain how this work differentiates from and extends prior research. 

ANS4: Thanks reviewer for good suggestion, and we have added contents from lines 100-107, the details are as follows:

The 3D virtual scene construction camera, 4DKanKan, was jointly launched by the China-Germany Artificial Intelligence Institute and ZHUHAI 4DAGE Technology. It eliminates the need for operators to possess complex professional knowledge. 3D virtual scenes are collected using a one-click method, and later modeling is uniformly processed using a one-click method in the cloud. For example, the Cloud Airshow was launched at the 13th China International Aviation & Aerospace Exhibition in 2021, and a model of the Longmen Grottoes in Luoyang, Henan Province, has been built for the reconstruction and restoration [12].

 

Q5 - In the background of the study, the author(s) covered various aspects related to the topic of their study. However, I felt that this theoretical background should result in the problem statement driving this study as well as in some clear research questions. Towards this end, I suggest adding a section after “Background” and before “Research Method” e.g. titled as “Problem statement & Research Questions”.

ANS5: Thanks reviewer for good suggestion, and we have added contents from lines 266-276, the details are as follows:

2.5. Problem Statement and Research Questions

According to the research purpose and relevant literature, this study's research problem statements are presented as follows:

Question 1: How well do users accept the VR system that integrates cultural tourism experience content? What are the potential system issues that may need correcting?

Question 2: To determine whether users are satisfied with their operating experience regarding the quality and information content of the virtual system. What are the factors affecting their satisfaction?

Question 3: Construct a PLS multigroup structural model to explain whether users are satisfied with their operating experiences regarding system quality and information quality. What are the issues affecting their satisfaction.

 

Q6 - Importantly, in the theoretical background of their study the author(s) should introduce, present,  and contextualize the “research framework of behavioral intentions and learning effects of sustainable tourism development” which they investigate in this study. They should also explain the need and the motivation behind investigating this framework.

ANS6: Thanks reviewer for good suggestion, and we have added contents from lines 309-338, the details are as follows:

This study's hypotheses are shown in Figure 1. According to the literature review, Information Quality (IQ) refers to the degree to which users perceive the usefulness, understandability, completeness, timeliness, and correctness of the information provided by a system’s content information. Delone and Mclean (1992) mentioned that good information quality has a positive impact on user adoption. Yoo (2020) also used the information quality model to explore AR and consumers’ degree of satisfaction with information quality; their study illustrated the usability and ease of learning to measure the behavioral intention of information quality adoption. Therefore, this study proposes that H1: Good SQ positively impacts BI. System quality (SQ) refers to the degree to which users perceive the usefulness, understandability, completeness, timeliness, and correctness of the information a system provides. The system quality indicators proposed by Delone and Mclean (1992) were used by the present study to evaluate system adoption. Yoo (2020) also used this model to explore AR/VR and consumers’ satisfaction with the system, usability, ease of learning, convenience of access, system efficiency, and reaction time to measure system quality and whether good system quality can help users’ behavior of adoption. Therefore, this study proposes that H2: Good IQ positively impacts BI. Behavioral Intention (BI) refers to the subjective perception that consumers may use an AR/VR system in the future. BI can be used to explore the relationship between attitude and behavior under various usage situations. When there are changes in behavioral attitude, subjective range, and perceptual behavior control, behavioral outcomes will change [40, 41, 42]. Fujita Stank and Thompson (1994) suggested that improving learning satisfaction has impacts on improving learning effects. Moreover, the strength of behavioral intention (BI) can positively impact learning effects [47]. Therefore, this study proposes H3: BI positively impacts LE. Involvement refers to the degree to which users believe that AR/VR systems are related to personal information needs and prior knowledge. Zaichkowsky (1985) discussed the degree to which VR systems were related to personal information needs [51]. Therefore, this study proposes the hypothesis that H4: Involvement has a mediating effect on H3. The overall study hypotheses are illustrated in Figure 1. The research framework of behavioral intentions and learning effects of sustainable tourism development.

 

Q7 - I would suggest renaming section 3.1 into “Procedures”. Then, the prior paragraph (right after “3. Research method”) could have the sub-head “Research design”. I would also suggest adding a sub-section dedicated on the participants/sample of the study.

ANS7: Thanks reviewer for good suggestion, and we have added and modified relevant contents as follows:

1. In line 296, we have modified 3.1. Research design and process to 3.2 Procedures.
2. In line 278, we have added 3.1 Research design under 3 Research Method.
3. We have added 3. Research Subjects from lines 339-347, the details are as follows:

This study adopted random sampling recruitment methods. The main research subjects of this experimental system were adults over 18 years of age (including tourists to Macao or students from Macao). Regarding the VR experience, research subjects were invited to the digital media studio of the Macao Polytechnic University for the operation. Regarding the AR experience, subjects were required to download their own APP for operation. The study’s researchers accompanied the research subjects to mitigate issues that might have occurred during the experiment and they observed the operation process.

 

Q8 - In the data analysis section, beyond presenting the statistical tests deployed, the author(s) also present their findings. I think that the paper would benefit by presenting separately the data analysis from the findings. The findings should be presented in a separate section on their own. Also, the findings could be structured in sub-sections according to the research questions guiding this study.

ANS8: Thanks reviewer for good suggestion. In addition to 4.1. Narrative statistics, we disassemble the analysis parts of 4. Data Analysis as follows:

1. Add 4.2. System Quality Evaluation and Analysis in line 467.
2. Add 4.3. Information Quality Evaluation and Analysis in line 510.
3. Add 4.4. Evaluation and Analysis of Users’ Learning Satisfaction in line 547.
4. Modify the title numbers of 4.6. PLS Structural Model Analysis in line 584 and 4.7.

Involvement Degree Interference Effect Verification in line 601.

 

Q9 - Finally, the discussion could be also structured according to the research questions guiding this study.

ANS9: Thanks reviewer for good suggestion, and we have added and modified relevant contents as follows:

1. In line 640, we have added 5.1.1. Actual impact of system quality and information quality on users.
2. In line 656, we have added 5.1.2. Good system design can yield higher value and improve users’ satisfaction.
3. We have added 5.1.3. PLS Multigroup Comparative Analysis of the Impact of Different Types of Participation on learning effects from lines 675-689, the details are as follows:

This study's results indicate that the participation of groups with different levels of involvement in PLS multigroup comparative analysis could mediate learning effects. The test results also show that the relationships between SQ, IQ, BI, and LE variables were significantly different due to the involvement of different types. Therefore, from this research model, it is evident that different types of involvement had significant differences in learning effect needs. The model test results show that LE was significantly and positively affected by BI; its variance interpretation ability was the low involvement group sample (28.7%) and the high involvement group sample (57.2%), respectively. The hypotheses based on this research model were all verified. This indicates that the higher the degree of correlation between the user’s information needs for the AR/VR system and their prior knowledge, the better the learning effects. This is consistent with the research on the correlation between the higher the user's involvement, the better the effectiveness and the personal information needs, as proposed by Zaichkowsky (1985) [51].

Author Response File: Author Response.pdf

Round 2

Reviewer 3 Report

Thank you for giving me the opportunity to review the revised version of this work. I would also like to thank the author(s) for taking into consideration my previous comments. At this stage, the paper is quite improved and I have only some minor comments, as follows:

#1 First, there has been a misunderstanding with regard to my comment on the title of your paper. My suggestion was to completely remove any reference to your statistical analysis from the title. So, please completely remove the “Applying Multigroup Analysis to Partial Least Square Structural Equation Modeling” and revise your title accordingly in a way that captures your research goal but without mentioning the undertaken statistical analysis.  E.g. “Investigating the relations between the design of a VR system, users’ behavioral intentions, and learning effects for Sustainable Tourism Development”

#2 Please rename section 2.5 into “Research Questions and Hypotheses” and revise the first sentence as follows: “According to the research purpose and relevant literature, this study's research questions are presented as follows:”. I would also suggest revising your research questions, to increase their readability, as follows:

RQ1: How well do users accept the VR system that integrates cultural tourism experience content?

RQ2: Which are the main factors affecting users’ satisfaction with their operating experience regarding the quality and information content of the virtual system?

RQ3: Whether the system quality and information quality can predict users’ behavioral intentions and learning effects?

3    #3 In section 2.5 right after the RQs, move Figure 1 as well as the text you have added in lines 304-333.  

#4 Please rename section 4 into “Data analysis and Findings”

Author Response

Reviewer 3

Thank you for giving me the opportunity to review the revised version of this work. I would also like to thank the author(s) for taking into consideration my previous comments. At this stage, the paper is quite improved and I have only some minor comments, as follows:

Q1 First, there has been a misunderstanding with regard to my comment on the title of your paper. My suggestion was to completely remove any reference to your statistical analysis from the title. So, please completely remove the “Applying Multigroup Analysis to Partial Least Square Structural Equation Modeling” and revise your title accordingly in a way that captures your research goal but without mentioning the undertaken statistical analysis.  E.g. “Investigating the relations between the design of a VR system, users’ behavioral intentions, and effects learning for Sustainable Tourism Development”

ANS1: Thank you for reviewer’s suggestion. Authors have change the title to “Investigating the Relationship between Users' Behavioral Intentions and Learning Effects of VR System for Sustainable Tourism Development”.

 

Q2 Please rename section 2.5 into “Research Questions and Hypotheses” and revise the first sentence as follows: “According to the research purpose and relevant literature, this study's research questions are presented as follows:”. I would also suggest revising your research questions, to increase their readability, as follows:

RQ1: How well do users accept the VR system that integrates cultural tourism experience content?

RQ2: Which are the main factors affecting users’ satisfaction with their operating experience regarding the quality and information content of the virtual system?

RQ3: Whether the system quality and information quality can predict users’ behavioral intentions and learning effects?

ANS2: Thanks reviewer for good suggestion, and we have modified the section 2.5 into “Research Questions and Hypotheses” and Contents of RQ1, RQ2, RQ3.

 

Q3 In section 2.5 right after the RQs, move Figure 1 as well as the text you have added in lines 304-333.  

ANS3: Thanks reviewer for good suggestion, and we have moved the contents of lines 304-333 and Figure 1 to section 2.5.

 

Q4 Please rename section 4 into “Data analysis and Findings”

ANS4: Thanks reviewer for good suggestion, and we have modified the section 4 into “Data analysis and Findings”.

Author Response File: Author Response.pdf