Qualitative Analyses of e-Learning Implementation and Hybrid Teaching during the COVID-19 Pandemic at Spanish Universities

Round 1

Reviewer 1 Report

 This paper needs to be revised for several reasons:

 1.    TITLE and ABSTRACT.  The title seems to be quite long, the second part (‘new insights into hybrid teaching’) might not be necessary. The abstract should better specify the research subjects as described in the paper: ‘79 university students’ (line 78) and 13 teachers, 5 of them ‘occupy university management posts’ (line 90).  In the data analyses (tables) they are consistently represented as US (students), TE (teachers), and TM (Teachers with management posts) as expressed in line 154. However, the abstract states that ‘content analysis was done of three discussion groups made up of 13 teachers (5 occupy university management posts) using 79 open questionnaires completed by students’ (lines 17-19). Were the 13 teachers using 79 open questionnaires completed by students? This description is not accurate and can be confusing. They should rephrase this part, the three groups (US, TE and TM) need to be clearly stated and specified in the abstract.

2.     LITERATURE REVIEW. This section seems to be quite short and adopt a very general approach. It should clearly explain the research need based on previous literature and identify the main objectives. For example, the authors claim that ‘the present study objective is to analyse the actual feasibility of having an e-learning methodology in the hybrid teaching context at university to know the real possibilities of the teaching methodology and format, and to look closely at the related psycho-educational implications’ (lines 70-75). What do they mean by ‘feasibility’ and ‘real possibilities’? What ‘teaching methodology and format’ do they refer to? Which ‘related psycho-educational implications’? To me, this study delves into the Spanish student’s and teachers’ perceptions (benefits and challenges) toward a hybrid teaching model through a qualitative analysis. The research need and objectives need to be clearly formulated in the paper.

3.    PARTICIPANTS, data representation. There is no need to include both numbers and percentages, and one decimal place is sufficient. For example ‘In this study, 79 university students (64 women/81.01% and 15 men/18.99%)’ (line 78). This may seem redundant. And in ‘According to their training, 39 were Early Childhood Education 81 students (49.37%), 38 studied Primary Education (48.10%) and two studied both degrees 82 at the same time (2.53%)’.(lines 81-84). This pattern is repeated throughout the paper.  

4.    PARTICIPANTS. Further clarification is required about the sampling method. How were they selected? ‘This study included 13 teachers (8 women/61.54% and 5 men/38.46%), who also came from different public and private university faculties and centres’ (line 87). The authors later explain that they were invited by email but no details about the number/type of target institutions/participants are provided. This is necessary to evaluate the scope of the present study and the representativeness of the results (read last comment about this issue)

5.    PARTICIPANTS. Some statements are not clear enough, for example ‘There were not statistically significant differences in the distribution of the criteria of interest variables training and gender in the students group and in the teachers group’ (lines 94-96). What do the authors mean by ‘interest variables training and gender’?

6.    DATA COLLECTION. Some statements are not clear enough and can be even contradictory or at least misleading:  ‘Then three discussion groups were set up, two with teachers and a third teaching+management group’ (line 110). So, the study was based on three discussion groups made up of only teachers? Or did they include students, teachers and teachers in management positions? No discussion group with students (US)? Read first comment for a related question. The authors need to clarify the number and composition of the different discussion groups and be consistent throughout the paper.

7.    DATA COLLECTION. Terminology. The authors need to be cautious about the use of certain terms such as ‘e-learning, blended learning, hybrid learning, online teaching’, etc., which they seem to use sometimes interchangeably (lines 128-133)

8.    DATA COLLECTION. The authors need to explain the use of dichotomous questions in the interviews as these are usually considered unsuitable for qualitative data collection since they may restrict respondents to a limited set of answer-options. For example ‘Do you think that students and centres have enough means (technological resources, devices and infrastructures) to give hybrid teaching?’ (lines 130-131) How did the participants respond to this type of YES/NO question?

9.    METHOD. The authors should clarify whether they used a content analysis (lines 17, 141, 143, 477) or a thematic analysis (lines 135, 151, 160). To me this seems more a thematic analysis (TA), not a content analysis but the authors seem to interchange both terms.

10.  RESULTS. This section needs to be strengthened with more references to previous findings. In their thematic analysis, the authors make reference to different sets of codes and themes without properly relating them with recent works related with the same context. For example, they insist on the participants’ perceptions about the lack of proper training as in: ‘could explain this lack of specific training in using certain methodologies’ (199-200),  ‘the students detected certain training insufficiency in their early training’ (203) and also on the need for more online interaction as in ‘but with no possibility of synchronous interaction’ (234),  ‘classes with more interaction are more practical’ (502), and ‘lack of interpersonal contact’ (598). These themes have been recently investigated in several works such as:

Lack of proper training in ERT (Emergency Response Teaching) due to COVID-19:

García-Sampedro, M., Peña-Suárez, E., & Rodríguez-Olay, L. (2021). Online Education during the COVID 19 lockdown and school closures in Spain. Teachers’ perceptions. Aloma: revista de psicologia, ciències de l'educació i de l'esport, 39(2), 43-51.

Llerena-Izquierdo, J., & Ayala-Carabajo, R. (2021, February). University teacher training during the COVID-19 emergency: the role of online teaching-learning tools. In International Conference on Information Technology & Systems (pp. 90-99). Springer, Cham.

 Need of online interaction (teacher-student and peer-to-peer) and impact on students’ motivation

Belda-Medina, J. (2021). Enhancing Multimodal Interaction and Communicative Competence through Task-Based Language Teaching (TBLT) in Synchronous Computer-Mediated Communication (SCMC). Education Sciences, 11(11), 723.

Díaz-Noguera, M. D., Hervás-Gómez, C., De la Calle-Cabrera, A. M., & López-Meneses, E. (2022). Autonomy, motivation, and digital pedagogy are key factors in the perceptions of Spanish higher-education students toward online learning during the COVID-19 pandemic. International Journal of Environmental Research and Public Health, 19(2), 654.

 11.  LIMITATIONS. The authors should clearly state the research limitations and clarify some statements, for example ‘Some bias in the data obtained and analyzed could affect the feasibility of the conclusions’ (617-618). Which ones (participant or research bias)? And what type (acquiescence, confirmation, question-order, etc)?  And ‘We could consider that our sample represents the reality analyzed.’ (620) How representative of the whole context can it be with a sample size of 79 students and 13 teachers?  

Author Response

Generally speaking, before the first submission the manuscript was revised by a native English translator. Regarding other aspects reported in the review, we tried to contextualize better the study and clarified the objectives. We also have included some current and relevant references along the manuscript to improve theoretical and empirical background on the topic. Moreover, we have included clarifications regarding some methodological aspects.

1.- TITLE and ABSTRACT.  The title seems to be quite long, the second part (‘new insights into hybrid teaching’) might not be necessary. The abstract should better specify the research subjects as described in the paper: ‘79 university students’ (line 78) and 13 teachers, 5 of them ‘occupy university management posts’ (line 90).  In the data analyses (tables) they are consistently represented as US (students), TE (teachers), and TM (Teachers with management posts) as expressed in line 154. However, the abstract states that ‘content analysis was done of three discussion groups made up of 13 teachers (5 occupy university management posts) using 79 open questionnaires completed by students’ (lines 17-19). Were the 13 teachers using 79 open questionnaires completed by students? This description is not accurate and can be confusing. They should rephrase this part, the three groups (US, TE and TM) need to be clearly stated and specified in the abstract.

Response: We agree with the reviewer. We have proposed a new title more descriptive to gain adequacy. We also have rephrased the part of the abstract related with participants of the study and we have revised this aspect along the content of the manuscript.

2.- LITERATURE REVIEW. This section seems to be quite short and adopt a very general approach. It should clearly explain the research need based on previous literature and identify the main objectives. For example, the authors claim that ‘the present study objective is to analyse the actual feasibility of having an e-learning methodology in the hybrid teaching context at university to know the real possibilities of the teaching methodology and format, and to look closely at the related psycho-educational implications’ (lines 70-75). What do they mean by ‘feasibility’ and ‘real possibilities’? What ‘teaching methodology and format’ do they refer to? Which ‘related psycho-educational implications’? To me, this study delves into the Spanish student’s and teachers’ perceptions (benefits and challenges) toward a hybrid teaching model through a qualitative analysis. The research need and objectives need to be clearly formulated in the paper.

Response: We have included some modifications in the content of the introduction to gain adequacy in relation with the objectives stated (page 1-2 of the manuscript). We also have adjusted and reformulated the objectives in the paper (page 2-3 of the manuscript). These changes have also been incorporated into the abstract (page 1).

3.- PARTICIPANTS, data representation. There is no need to include both numbers and percentages, and one decimal place is sufficient. For example ‘In this study, 79 university students (64 women/81.01% and 15 men/18.99%)’ (line 78). This may seem redundant. And in ‘According to their training, 39 were Early Childhood Education 81 students (49.37%), 38 studied Primary Education (48.10%) and two studied both degrees 82 at the same time (2.53%)’.(lines 81-84). This pattern is repeated throughout the paper.

Response: We partially agree with reviewer. Numbers and percentages are commonly reported in the participants sections in the literature. Percentages give complementary information regarding sample distribution. In any case, the modifications suggested were applied (page 3 of the manuscript).

4.- PARTICIPANTS. Further clarification is required about the sampling method. How were they selected? ‘This study included 13 teachers (8 women/61.54% and 5 men/38.46%), who also came from different public and private university faculties and centres’ (line 87). The authors later explain that they were invited by email but no details about the number/type of target institutions/participants are provided. This is necessary to evaluate the scope of the present study and the representativeness of the results (read last comment about this issue)

Response: It is a good appreciation. Further clarification about sampling method was included (page 3 of the manuscript). We also have included more details about target institutions/participants (page 4 of the manuscript). Regarding the scope of the study and representativeness, given the objectives of the study and starting from a qualitative approach, our interest focused on understanding the phenomenon of interest through the actors involved to learn more about the reality studied. In this sense, as we discussed in the paper, the adequacy of a sample and its size are related with the ability of data to provide a rich approach to the phenomenon studied and objectives of the study.

5.- PARTICIPANTS. Some statements are not clear enough, for example ‘There were not statistically significant differences in the distribution of the criteria of interest variables training and gender in the students group and in the teachers group’ (lines 94-96). What do the authors mean by ‘interest variables training and gender’?

Response: These clarifications have been added to the manuscript (page 4 of the manuscript).

6.- DATA COLLECTION. Some statements are not clear enough and can be even contradictory or at least misleading:  ‘Then three discussion groups were set up, two with teachers and a third teaching+management group’ (line 110). So, the study was based on three discussion groups made up of only teachers? Or did they include students, teachers and teachers in management positions? No discussion group with students (US)? Read first comment for a related question. The authors need to clarify the number and composition of the different discussion groups and be consistent throughout the paper.

Response: These methodological clarifications have been added to the manuscript (page 4 of the manuscript).

7.- DATA COLLECTION. Terminology. The authors need to be cautious about the use of certain terms such as ‘e-learning, blended learning, hybrid learning, online teaching’, etc., which they seem to use sometimes interchangeably (lines 128-133).

Response: We agree with the reviewer. We have adjusted the terms used to be consistent and to gain adequacy (page 4 of the manuscript).

8.- DATA COLLECTION. The authors need to explain the use of dichotomous questions in the interviews as these are usually considered unsuitable for qualitative data collection since they may restrict respondents to a limited set of answer-options. For example ‘Do you think that students and centres have enough means (technological resources, devices and infrastructures) to give hybrid teaching?’ (lines 130-131) How did the participants respond to this type of YES/NO question?

Response: In the questionnaire instructions, students were invited to develop their answers and not to limit their answers to a simple YES/NO form. In fact, this was the reason for excluding 29 students from the initial sample.

This reminder was also made in the introduction to the focus groups. However, the dialogue between the participants did not lead to such type of answers.

9.- METHOD. The authors should clarify whether they used a content analysis (lines 17, 141, 143, 477) or a thematic analysis (lines 135, 151, 160). To me this seems more a thematic analysis (TA), not a content analysis but the authors seem to interchange both terms.

Response: We agree with the reviewer. To gain consistency and methodological adequacy we have referred to thematic analysis   (page 1, 5 and 12 of the manuscript).

10.- RESULTS. This section needs to be strengthened with more references to previous findings. In their thematic analysis, the authors make reference to different sets of codes and themes without properly relating them with recent works related with the same context. For example, they insist on the participants’ perceptions about the lack of proper training as in: ‘could explain this lack of specific training in using certain methodologies’ (199-200),  ‘the students detected certain training insufficiency in their early training’ (203) and also on the need for more online interaction as in ‘but with no possibility of synchronous interaction’ (234),  ‘classes with more interaction are more practical’ (502), and ‘lack of interpersonal contact’ (598).

Response: The decision to report results without an interpretative process was not arbitrary. We adopted the model used in our previous related manuscript published in the same journal, and proposed by another reviewer in the revision. We have conducted the interpretation of the results in the discussion section. Thus, we have included the references proposed by the reviewer in the discussion   (pages 13 and of the manuscript).

11.- LIMITATIONS. The authors should clearly state the research limitations and clarify some statements, for example ‘Some bias in the data obtained and analyzed could affect the feasibility of the conclusions’ (617-618). Which ones (participant or research bias)? And what type (acquiescence, confirmation, question-order, etc)?  And ‘We could consider that our sample represents the reality analyzed.’ (620) How representative of the whole context can it be with a sample size of 79 students and 13 teachers?

Response: We have included some clarifications in the limitations about research bias associated with sampling (page 15 of the manuscript). Regarding representativeness, the reviewer has to consider that our interest was focused on understanding the phenomenon through the actors involved and learn more about the reality studied. In fact, in our text we did not specifically say that our sample is a representative sample. We said our sample allows us to analyze the reality of interest. Our qualitative approach responses to the research objectives stated that involve to look closely how the e-learning implementation and hybrid teaching in Spanish universities was and to obtain information that could be useful for High education system in Spain. Not necessarily involved extrapolation to other realities. Therefore, the emphasis is not on the probabilistic representation but on the quality of the information we can obtain from the participants. In this sense, we discussed in the paper about saturation because the adequacy of a sample and its size are related with the ability of data to provide a rich approach to the phenomenon studied and objectives of the study, and cannot be detached from the study characteristics that influence saturation.

Reviewer 2 Report

I have read in other papers related to this topic the term Emergency distance education. It might be interesting to name it also in the introduction, because this term and the theoretical framework related to it is more suitable for this situation. I strongly recommend to review this possibility to improve the manuscript

Author Response

1.- I have read in other papers related to this topic the term Emergency distance education. It might be interesting to name it also in the introduction, because this term and the theoretical framework related to it is more suitable for this situation. I strongly recommend to review this possibility to improve the manuscript.

Response: We have included content related with the term Emergency Remote Teaching (ERT) in the introduction section (page 2 of the manuscript).

Reviewer 3 Report

This study used a qualitative research approach to discuss the implementation of online learning in Spanish universities before, during, and after the COVID-19 lockdown.

1. Although the research design and procedure are introduced in detail, this manuscript lacks the content of the literature review. It is suggested that the author can introduce existing related studies to highlight the importance of this study.

2. Before the outbreak of the Covid-19 pandemic, the effectiveness of blended learning was discussed by many researchers. Such studies are already abundant. Many countries around the world have been forced to implement online education in different ways because of the Covid-19 pandemic. Due to the urgent implementation of remote and home-based online learning, some teachers have no or lack online teaching experience, which affects the teaching effectiveness. For students, in the case of online learning at home, they need to have enough learning abilities such as self-directed learning. All these challenges the effect of students' online learning. According to page 2, lines 70 of this manuscript, “the present study objective is to analyze the actual feasibility of having an e-learning methodology in the hybrid teaching context at university to know the real possibilities of the teaching methodology and format, and to look closely at the related psycho-educational implications.” In the early stage of implementing online learning, too many factors interfered with this result. Therefore, it seems that this study is not very convincing to explore the feasibility of online learning in this context.

3. The study collected data from students, teachers, and administrators about online learning during the pandemic, as well as their evaluation of teaching prior to online learning. It would be more attractive if the manuscript could summarize the views of three different groups and provide related implications for different groups. Now in the post-pandemic era, we hope to learn from the implementation of online learning or blended learning to provide some reference for the implementation of remote online learning in similar emergency situations, rather than discussing whether the implementation of online learning or blended learning is effective.

Author Response

Generally speaking, before the first submission the manuscript was revised by a native English translator. Regarding other aspects reported in the review, we tried to contextualize better the study and clarified the objectives. We also have included some current and relevant references along the manuscript to improve theoretical and empirical background on the topic. Moreover, we have included clarifications regarding some methodological aspects.

1.- Although the research design and procedure are introduced in detail, this manuscript lacks the content of the literature review. It is suggested that the author can introduce existing related studies to highlight the importance of this study.

Response: We have included some modifications in the content of the introduction to gain adequacy in relation with the objectives stated (page 1-2 of the manuscript). We also have included content related with the term Emergency Remote Teaching (ERT) (page 2 of the manuscript).

2.- Before the outbreak of the Covid-19 pandemic, the effectiveness of blended learning was discussed by many researchers. Such studies are already abundant. Many countries around the world have been forced to implement online education in different ways because of the Covid-19 pandemic. Due to the urgent implementation of remote and home-based online learning, some teachers have no or lack online teaching experience, which affects the teaching effectiveness. For students, in the case of online learning at home, they need to have enough learning abilities such as self-directed learning. All these challenges the effect of students' online learning. According to page 2, lines 70 of this manuscript, “the present study objective is to analyze the actual feasibility of having an e-learning methodology in the hybrid teaching context at university to know the real possibilities of the teaching methodology and format, and to look closely at the related psycho-educational implications.” In the early stage of implementing online learning, too many factors interfered with this result. Therefore, it seems that this study is not very convincing to explore the feasibility of online learning in this context.

Response: We have adjusted and reformulated the objectives in the paper (page 2-3 of the manuscript).

3.- The study collected data from students, teachers, and administrators about online learning during the pandemic, as well as their evaluation of teaching prior to online learning. It would be more attractive if the manuscript could summarize the views of three different groups and provide related implications for different groups. Now in the post-pandemic era, we hope to learn from the implementation of online learning or blended learning to provide some reference for the implementation of remote online learning in similar emergency situations, rather than discussing whether the implementation of online learning or blended learning is effective.

Response: Given the rationale of the study and the objectives that we tried to clarify we do not agree with the reviewer’s proposal. Otherwise, we thank the reviewer for his/her useful comment because it has gave us the opportunity to clarify a methodological aspect included in our study. We have clarified that data collection strategy allowed mixed triangulation (page 4 of the manuscript). Our interest was focused on understanding the phenomenon through the different actors involved to learn more and broadly about the reality studied. The research objectives stated involve to look closely how the e-learning implementation and hybrid teaching in Spanish universities was and to obtain information that could be useful for High education system in Spain and allows universities to examine what is needed to work more effectively nurturing future possibilities in online and remote teaching and learning contexts. In this sense, we tried to analyze the possibilities, benefits and challenges of e-learning as a methodology per se, and hybrid teaching as a teaching format that could gain protagonism in universities, in a context without the possibility of combining them with others methodologies and/or teaching models.

Round 2

Reviewer 1 Report

Thanks for the revised version.

 

Author Response

We want to thank the reviewer for his/her helpful suggestions and comments.

Reviewer 3 Report

First, the part of explaining the research motivation and value have not been supplemented.

Second, there have been many investigations on online learning during the epidemic. There is nothing new in the research, and the conclusions of the research can be known without investigation.

Author Response

The study contextualization was extended to clarify the research motivation and value (pages 2-3 of the manuscript).

On the other hand, as we stated in one of our previous responses in the first revision of the manuscript, our interest was focused on understanding the phenomenon through the actors involved and learn more about the reality studied to obtain information that could be useful for High education system in Spain. Due to lack of related research, our study is trying to extend evidence in the Spanish Higher education system regarding all the characteristics elements that conceptualized e-learning as a teaching method, not before and during lockdown only but also during Hybrid teaching period, considering the perspective and experience from different protagonists like students and teachers. We obtained empirical evidence from a systematic research process that support the daily experience from the university, obtaining useful and certainly contrastable conclusions. Likewise, we have adjusted the conclusions obtained.

Round 3

Reviewer 3 Report

English language should be proofread. There is no other additional comment. 

Author Response

Finally, the manuscript has been revised by a native English proofreader before being submitted (we include the certificate in the cover letter).