Chatbot Service Quality: An Experiment Comparing Two Countries with Different Levels of Digital Literacy

Comments：

Response:

This study discusses an interesting new topic, tourists' digital literacy. The use of innovative and revolutionary new technologies in the tourism industry not only improves the quality of services, but also changes the way tourists consume, especially in terms of the acceptance and use of digital technologies by tourists, so the topic of tourists' digital literacy may be important and necessary. I only provide a few comments for the editors and authors' reference. 

First of all, thanks for your considerations with our paper.

We the authors have tried to response all the comments.

#1：Introduction

The authors could present more information about tourists' digital literacy, such as the importance of tourists' digital literacy in areas such as tourism service experience and evaluation and emphasize the need and urgency of research on tourists' digital literacy.

Agree. We have clarified the gap and the aim of the study in the introduction section. Changes in page 2:

If we take into account that this technology is increasingly present in the tourism sector (Tussyadiah, 2020) and that more and more companies are opting to use them in different services offered to their customers, such as making or changing bookings, which implies a reduction in costs for the company and time for the tourist (Fan et al., 2022), it is necessary to know the users' opinion on the matter. To this end, some authors have been using some models of technology acceptance, such as the UTATU model to know if they are willing to use it (Gopinath and Kasilingam, 2023). These works on technology uptake use digital literacy to look at the level of technology acceptance, however a different level of digital literacy may also imply different perceptions of perceived quality, which may further hinder uptake in certain regions (Castañeda et al., 2020; Cheng et al., 2022), however, this is the first work, to the best of our knowledge, that attempts to determine the factors that affect the introduction of chatbots into the tourism sector, studying the quality of service offered by an already established technology such as email versus a new one such as the chat bot. The results can help tourism managers to carry out a correct implementation of these technologies, since it takes into account the differences in the perception of the quality of service offered by each one for subjects with different levels of digital literacy.

New references:

Castañeda, J. A., Frías-Jamilena, D. M., Rodríguez-Molina, M. A., & Jones, A. (2020). Online Marketing Effectiveness-the influence of information load and digital literacy, a cross-country comparison. Electronic Markets, 30, 759-773.

Cheng, C., Gearon, E., Hawkins, M., McPhee, C., Hanna, L., Batterham, R., & Osborne, R. H. (2022). Digital health literacy as a predictor of awareness, engagement, and use of a national web-based personal health record: population-based survey study. Journal of Medical Internet Research, 24(9), e35772.

Fan, X., Jiang, X., and Deng, N. (2022). Immersive technology: A meta-analysis of augmented/virtual reality applications and their impact on tourism experience. Tourism Management, 91, 104534.

Gopinath, K., and Kasilingam, D. (2023). Antecedents of intention to use chatbots in service encounters: A meta‐analytic review. International Journal of Consumer Studies.

Tussyadiah, I. (2020). A review of research into automation in tourism: Launching the Annals of Tourism Research Curated Collection on Artificial Intelligence and Robotics in Tourism. Annals of Tourism Research, 81, 102883.

#2：Literature review

The authors presented a definition of digital literacy but did not provide more information on the definition and structural dimensions of tourists' digital literacy. Therefore, I suggest that the authors conceptualize tourists' digital literacy based on the literature review.

Agree. Thanks for your comment. We have tried to highlighted this concept in tourism related literature. Changes in the page:

In the tourism sector, some studies have used this construct to explain the perceived ease of use of some techonoligies in their travels (Liu et al., 2017), by including in the acceptance technology models (Bae and Han, 2020), linking the DL with the motivation to obtain autonomy during travel (Tussyadiah and Wang, 2016) or to eplain the perception of the congruency between digital app presentations and their actual experiences (Xiong and Zhang, 2024). DL has been used over the years within these technology acceptance models, such as TAM or UTAUT in the tourism sector (Xiong and Zhang, 2024).

New references;

Bae, S. Y., & Han, J. H. (2020). Considering cultural consonance in trustworthiness of online hotel reviews among generation Y for sustainable tourism: An extended TAM model. Sustainability, 12(7), 2942.

Liu, Y., Li, K. J., Chen, H., & Balachander, S. (2017). The effects of products’ aesthetic design on demand and marketing-mix effectiveness: The role of segment prototypicality and brand consistency. Journal of Marketing, 81(1), 83-102.

Tussyadiah, I. P., & Wang, D. (2016). Tourists’ attitudes toward proactive smartphone systems. Journal of Travel Research, 55(4), 493-508.

Xiong, S., & Zhang, T. (2024). Enhancing tourist loyalty through location-based service apps: Exploring the roles of digital literacy, perceived ease of use, perceived autonomy, virtual-content congruency, and tourist engagement. Plos one, 19(1), e0294244.

#3：Conclusion and limitation

The authors need to clarify and add to the theoretical contributions of this study.

Again, thanks for your comments.

We have clarified this issue in the conclusion sector. Changes in the page 11:

In addition to these conclusions on the managerial level, the findings of the study also hold theoretical implications. While previous works have already addressed the issue of ESQ (Hanna, 2018; Zouari and Abdelhedi, 2021), this study constitutes an advancement in that it compares two technologies: one that is fully embedded among the majority of clients (email) and another that is still in the process of being implemented by some firms in the sector (chatbot). This, together with the division of the sample into two publics that differ in their levels of DL, rendered it possible to demonstrate how DL acts as a moderator of the perceived quality of a new technology vs. one that is well established. Furthermore, these differences are demonstrated for each of the six dimensions of quality taken into account in the study. Regarding the theoretical contributions to the specific field of tourism, this research is, to the best of our knowledge, the first to capture these differences in perceptions of a chatbot system vs. another technology, which is of particular interest, given the recent boom in studies on the implementation of AI in the tourism sector (e.g., Bigné and Matamura, 2023). Other works in the sector have used this type of technology in their studies, but not to compare the acceptance of two technologies with different levels of acceptance, such as email and chatbot. (e.g. Bae and Han, 2020; Xiang and Zhang., 2024).

Comments

Response

The content of the manuscript must be improved. Please consider the following remarks:

First of all, thanks for your valuable comments which with no doubt have improved our paper.

The content of the abstract section is quite fine. However, there is a sentence, and I am curious whether it is literally and scientifically correct. In the sentence “Finally, all the participants rated the responsiveness of the chatbot higher than that of email,” The word “all” should be checked, because if all the respondents had such an idea, the data gathering—I mean participant selection—and consequently the data used were biased. Otherwise, please correct the sentence.

Agree. We have changed this sentence by eliminating the word all, thanks for your appreciation.

We want to refer that it is found that “participants rated the responsiveness of the chatbot higher than that of email”.

Keywords should be selected based on their importance and frequent repetition in the text. Hence, the keywords "international tourism” and “cross-cultural analysis” should be eliminated or modified as they have not been used in the text.

Agree. Thanks for your valuable comment.

We have erased these two keywords and replace for:

Keywords: Technology acceptance; Digital literacy; e-Service Quality; Automated services; Technology in tourism

Accessibility to the data set and the declaration of authors should be moved to the conclusion section.

Agree.

We have moved it at the end of the mentioned section.

The research gap and the aim of the study should be highlighted, and their coherence should be supported.

Agree. We have clarified the gap and the aim of the study in the introduction section. Changes in page 2:

If we take into account that this technology is increasingly present in the tourism sector (Tussyadiah, 2020) and that more and more companies are opting to use them in different services offered to their customers, such as making or changing bookings, which implies a reduction in costs for the company and time for the tourist (Fan et al., 2022), it is necessary to know the users' opinion on the matter. To this end, some authors have been using some models of technology acceptance, such as the UTATU model to know if they are willing to use it (Gopinath and Kasilingam, 2023). These works on technology uptake use digital literacy to look at the level of technology acceptance, however a different level of digital literacy may also imply different perceptions of perceived quality, which may further hinder uptake in certain regions (Castañeda et al., 2020; Cheng et al., 2022), however, this is the first work, to the best of our knowledge, that attempts to determine the factors that affect the introduction of chatbots into the tourism sector, studying the quality of service offered by an already established technology such as email versus a new one such as the chat bot. The results can help tourism managers to carry out a correct implementation of these technologies, since it takes into account the differences in the perception of the quality of service offered by each one for subjects with different levels of digital literacy.

New references:

Castañeda, J. A., Frías-Jamilena, D. M., Rodríguez-Molina, M. A., & Jones, A. (2020). Online Marketing Effectiveness-the influence of information load and digital literacy, a cross-country comparison. Electronic Markets, 30, 759-773.

Cheng, C., Gearon, E., Hawkins, M., McPhee, C., Hanna, L., Batterham, R., & Osborne, R. H. (2022). Digital health literacy as a predictor of awareness, engagement, and use of a national web-based personal health record: population-based survey study. Journal of Medical Internet Research, 24(9), e35772.

Fan, X., Jiang, X., and Deng, N. (2022). Immersive technology: A meta-analysis of augmented/virtual reality applications and their impact on tourism experience. Tourism Management, 91, 104534.

Gopinath, K., and Kasilingam, D. (2023). Antecedents of intention to use chatbots in service encounters: A meta‐analytic review. International Journal of Consumer Studies.

Tussyadiah, I. (2020). A review of research into automation in tourism: Launching the Annals of Tourism Research Curated Collection on Artificial Intelligence and Robotics in Tourism. Annals of Tourism Research, 81, 102883.

It is expected to see the hypothesis considered in this study based on the provided research model.

Agree.

We have included sub-hypothesis for each dimension. Changes in the results section have been made consequently. Changes in the manuscript. Page 5:

H1: DL moderates the user’s perception of the perceived quality of a new technology compared to a mature one.

H1A: the perception of security will differ from a new technology compared to a mature one.

H1B: the perception of empathy will differ from a new technology compared to a mature one.

H1C: the perception of reliability will differ from a new technology compared to a mature one.

H1D: the perception of competence will differ from a new technology compared to a mature one.

H1E: the perception of responsiveness will differ from a new technology compared to a mature one.

H1F: the perception of information quality will differ from a new technology compared to a mature one.

In Section 3.2, it has been mentioned that “mean age (p-value = 0.54) and the percentage of women (p-value = 0.58)"  should be less than or equal to 0.05 or 0.01 to decide whether to reject or not to reject a hypothesis. Please explain the reason for using this parameter or modify it.

The test that we have carried out is a test of mean differences to check that both groups are stable. In this test the null hypothesis states that there is equality of means. It has been specified in the text.

Changes in the manuscript. Page 6:

These data were balanced across the two technologies under analysis, with no differences in either the mean age (p-value=0.54) or in the percentage of women (p-value=0.58) (where H0 means that there are no differences in the average by groups). By country, no differences were identified in the socio-demographic variables, with p-values of 0.69 and 0.11, respectively, for age and gender.

There are some sentences like “Graph Error! No text of specified style in the document” in Section 4 should be corrected.

Agree.

We have made changes. It was probably a problem with the format.

It seems the respected authors used Excel software to test the hypothesis and for the descriptive statistics. It is highly recommended to use scientific statistical software to analyze the data.

We used Excel for the graphs. To conduct the ANOVA analysis, we used STATISTICA. We have clarified this in the paper. Thanks for your comment. Changes in the manuscript, page 6:

Once the data had been standardized to avoid the influence of cultural response bias, the adequate distribution of individuals across the different groups had been confirmed (taking into account the gender and age variables), and the internal consistency of the measurement scales had been verified, the ANOVA analysis could be performed using the statistic software STATISTICA.

Figure numbers and titles are not provided.

Regarding this issue we can see the figure numbers and titles. We do not know if there is a problem with the sending document.

Here you can find the summary:

Figure 1. Dimensions of ESQ.

Table 1. Sample data.

Graph 1. Interaction effect of technology type and country on user evaluation of ESQ.

Graph 2.1 Perception of security

Graph 2.2 Perception of empathy

Graph 2.3. Perception of reliability

Graph 2.4. Perception of competence

Graph 2.5 Perception of responsiveness

Graph 2.6. Perception of information quality

Additionally, some mistakes have been found in the figure 1, that the authors have fixed.

There are some arrows on the graphs that were not explained

Agree.

We have included the following explanation in the text in order to clarify the meaning of these arrows. Page number 7:

In addition, arrows have been added to highlight the differences between each of the groups.