Understanding the Impact of a Learning Management System Using a Novel Modified DeLone and McLean Model

Round 1

Reviewer 1 Report

This research would acquire more relevance and impact if it noted some of the emerging trends about the impact of a learning management system.

This research needs deep literature (Serbia, Rusia, Argentina, Indonesia... what about the other countries) on the problem statement and research questions.

So, related literature should be in its section and extended to bring out the limitations of the present research, leading to how the proposed research model-study overcomes these. 

The text needs to explain more clearly and in enough depth the research approach of the study. The methods should be adequately described and show how the methods used will provide the answers to the questions. 

 

An interrogative and argumentative voice is lacking to make it more interesting.

Author Response

Response to Reviewer 1 Comments

 

Point 1: This research would acquire more relevance and impact if it noted some of the emerging trends about the impact of a learning management system.

 

Response 1: Thank you for the review suggestions. We have added some of the emerging trends regarding the impact of a learning management system (Paragraph 8 in the Introduction).

Facing the 21st century, UNESCO, through the journal "The International Commission on Education for the Twenty-First Century," recommends continuous-lifelong education based on the four pillars of the learning process: learning to know, learning to do, learning to be, and learning together [3]. To realize the four pillars of education in the current era of information globalization, teachers as learning agents need to master and apply Information and Communication Technology in learning. This statement from UNESCO became important during the pandemic to continue developing a learning management system, especially in the Indonesian government program, namely PPG. As technology continues to advance and the importance of digital learning grows, there are several emerging trends in the impact of LMS: personalized learning [4], gamification [5], mobile learning [6], social learning [7], analytics [8], and data-driven insights [9]. Overall, the emerging trends in LMS design focus on creating a more engaging and personalized learning experience that is accessible, collaborative, and data-driven.

 

Point 2: This research needs deep literature (Serbia, Rusia, Argentina, Indonesia... what about the other countries) on the problem statement and research questions.

 

Response 2: Several supporting sentences for deep literature have been added at the end of each section of the cited literature in the Introduction.

In Serbia and Russia, the program is done by implementing digital competency to develop vocational teacher professional digital competency. Through the program, teachers are qualified to teach in a local, national, or international work environment following the European Union's policy on online learning [5]. A specific model of LSM for Serbia and Russian teacher are difficult to find. Therefore, there should be initiative among the educated in terms of developing the required LMS.

In the Philippines, the program was realized through practicum teaching experience. In Philippine education, all teacher education curricula at all higher education institutions include practical teaching as one of the requirements for obtaining a Bachelor of Education degree [6]. In their research, they found that most of the issues and challenges were connected to what can be described as the pre-service teacher training program in the Philippines, as reflected in the practicum experience of the pre-service student teachers.

In Argentina (USA), a teacher training program is conducted to use digital technology for mathematics teachers. It was done to refine the needs of future teachers and the role of technology in advancing mathematics embodied in project modeling [7]. However, students may need to integrate LMS with real-world experience, for example, in a remote laboratory [1]. The integrated infrastructure may revalue student and teacher communication in face-to-face teaching and avoid capitalizing on the digital competencies of teaching hybridization.

In Australia, the teacher training program is carried out by teaching and learning in the Australian context during practicum and how this learning experience constructs their identity as teachers through the activity theory framework [8]. In addition, using technology to transform online learning and teaching with the technological pedagogical content knowledge (TPACK) framework [9]. However, a few common problems can arise when using an LMS in Australia. Some key issues and potential solutions are technical, lack of training, inadequate content, limited collaboration, and accessibility [2]. Overall, the successful implementation of an LMS requires a collaborative effort between institutions, faculty, and students. By addressing these common problems, institutions can create a more effective and efficient learning environment for their students.

 

Despite the benefits of implementing the LMS, it has some common problems. One of them is the high level of student absenteeism compared to face-to-face learning. The assumption of this problem is the level of user acceptance of technology, including the user's willingness to use technology. The differentiating factor, in this case, is the level of knowledge and technology application of different users. In addition, there are research findings that reveal that online learning program management can be realized effectively if the teacher carries out his duties professionally so that students get a sense of satisfaction and are loyal to the learning activity itself, which in the end, the goals of the teaching-learning process can be fulfilled [10].

 

Point 3: So, related literature should be in its section and extended to bring out the limitations of the present research, leading to how the proposed research model-study overcomes these.

 

Response 3: We have added some of the limitations obtained from the cited literature to offer that our proposed research model-study overcomes those limitations.

Some of the existing related literature found several limitations from the previous studies: lack of personalization, limited interaction, technical issues, cost, and limited flexibility. Therefore, this study aims to understand these causes successfully using the information systems (IS) model derived from a novel modified DeLone and McLean (D&M). Our proposed model sees LSM users and developers as equal entities.               

 

Point 4: The text needs to explain more clearly and in enough depth the research approach of the study. The methods should be adequately described and show how the methods used will provide the answers to the questions.

 

Response 4: We have added and shown how a modified M&C model would provide answers to questions.

• In the Proposed Model and Hypotheses section:

The modification of the D&M model illustrates that system benefit or advantage variables will affect the impact variables, results, and benefits provided by the system on user needs and the success of the PPG LMS.

• In the Method section:

The D&M modification model can cut the path from system quality, information quality, and service quality dimensions to net benefits more quickly when compared to the original model, which must first pass use and user satisfaction. The modified D&M can be used to answer a variety of questions regarding the efficacy and productivity of the PPG LMS. The evaluation may incorporate metrics such as system dependability, information correctness, reaction time, and user feedback to answer the question. The model can be used to analyze the system's net benefits. This review may incorporate cost savings, time savings, and increased efficiency to address the question. Nevertheless, the M&C model may be used to evaluate the success of a PPG LMS and provide answers to numerous concerns concerning its efficacy and efficiency. The metrics and methodologies used to evaluate the quality criteria can be adjusted to the precise topic posed.

 

 

Point 5: An interrogative and argumentative voice is lacking to make it more interesting.

 

Response 5: Thank you for the suggestion. We add some sentences to make it more interesting in the Introduction and Discussion section.

Overall, the emerging trends in LMS design focus on creating a more engaging and per-sonalized learning experience that is accessible, collaborative, and data-driven, which is considered as convoluted problems.

 

Therefore, this study aims to promote a shortest path yet novel modification DeLone and McLean (D&M). The model is use to discover the user’s understanding to the impact of using LMS.

 

The modified D&M model also can solve the limitation of LMS in previous work, such as lack of personalization, limited interaction, technical issues, cost, and limited flexibility. This proposed model could be implemented in future research related to the user experience of LMS or other content management systems.

 

 

Additional references for revised version:

[1]         F. Lerro et al., “Integration of an e-learning platform and a remote laboratory for the experimental training at distance in engineering education,” in 2012 9th International Conference on Remote Engineering and Virtual Instrumentation (REV), Jul. 2012, pp. 1–5, doi: 10.1109/REV.2012.6293119.

[2]         D. Weaver, C. Spratt, and C. S. Nair, “Academic and student use of a learning management system: Implications for quality,” Australas. J. Educ. Technol., vol. 24, no. 1, Jan. 2008, doi: 10.14742/ajet.1228.

[3]         J. Delors, “The International Commission on Education for the Twenty-First Century,” 1996.

[4]         H. Xie, H.-C. Chu, G.-J. Hwang, and C.-C. Wang, “Trends and development in technology-enhanced adaptive/personalized learning: A systematic review of journal publications from 2007 to 2017,” Comput. Educ., vol. 140, p. 103599, Oct. 2019, doi: 10.1016/j.compedu.2019.103599.

[5]         G. Tuparov, D. Keremedchiev, D. Tuparova, and M. Stoyanova, “Gamification and educational computer games in open source learning management systems as a part of assessment,” in 2018 17th International Conference on Information Technology Based Higher Education and Training (ITHET), Apr. 2018, pp. 1–5, doi: 10.1109/ITHET.2018.8424768.

[6]         H. Hamidi and A. Chavoshi, “Analysis of the essential factors for the adoption of mobile learning in higher education: A case study of students of the University of Technology,” Telemat. Informatics, vol. 35, no. 4, pp. 1053–1070, Jul. 2018, doi: 10.1016/j.tele.2017.09.016.

[7]         S. A. Aljawarneh, “Reviewing and exploring innovative ubiquitous learning tools in higher education,” J. Comput. High. Educ., vol. 32, no. 1, pp. 57–73, Apr. 2020, doi: 10.1007/s12528-019-09207-0.

[8]         M. Cantabella, R. Martínez-España, B. Ayuso, J. A. Yáñez, and A. Muñoz, “Analysis of student behavior in learning management systems through a Big Data framework,” Futur. Gener. Comput. Syst., vol. 90, pp. 262–272, Jan. 2019, doi: 10.1016/j.future.2018.08.003.

[9]         C. Guan, J. Mou, and Z. Jiang, “Artificial intelligence innovation in education: A twenty-year data-driven historical analysis,” Int. J. Innov. Stud., vol. 4, no. 4, pp. 134–147, Dec. 2020, doi: 10.1016/j.ijis.2020.09.001.

[10]       A. N. Bahasoan, Wulan Ayuandiani, Muhammad Mukhram, and Aswar Rahmat, “Effectiveness of Online Learning In Pandemic Covid-19,” Int. J. Sci. Technol. Manag., vol. 1, no. 2, pp. 100–106, Jul. 2020, doi: 10.46729/ijstm.v1i2.30.

[11]       M. Gawlik-Kobylinska, “The Four-Dimensional Instructional Design Approach in the Perspective of Human-Computer Interactions.,” in APPIS, 2018, pp. 146–156.

 

Author Response File: Author Response.pdf

Reviewer 2 Report

My congratulations, I found it very interesting and of current interest to know the aspects related to a current and increasingly used teaching, original to know if the evaluation of e-learning systems is effective.

Aspects that I recommend for improvement:

On the other hand I note the lack in the sample of the participants , there is no reference to the total number of participants by age and gender and it would be interesting to consider this.

This could be substantiated by more auditing.

 

Author Response

Response to Reviewer 2 Comments

 

Point 1: On the other hand I note the lack in the sample of the participants , there is no reference to the total number of participants by age and gender and it would be interesting to consider this.

 

This could be substantiated by more auditing.

 

Response 1: Thank you for the review suggestions. We have added information about the gender and age distribution of the participants.

Of the existing participants, 124 were female and 115 male, aged around 25 – 32 years.  

 

Author Response File: Author Response.pdf

Reviewer 3 Report

Dear Author(s),

the article is nicely written, however, there are several issues that should be taken into account.

1) Between the lines 25-26 please insert an explanation info on what basis you picked up the countries such as Nigeria, Russia, Serbia, etc.

2) If you did your research in Indonesia, please provide this information in the Introduction.

3) Also, in the Introduction, add the aim, research questions, and hypotheses.

4) Figure 1 is blurred. Please provide a better resolution of a picture.

5) In the line 103, the abbreviation IS should be explained.

6) The lines 294-297 are in italics. Why? If italics are not necessary, please use a plain text.

7) In the lines 8-9 the aim of the paper is "this study aims to evaluate the success of learning management systems using the modified DeLone and McLean model", whereas in the Discussion "This paper promotes a new modification of Delene and McLean's success model". It is a bit beyond the goal of the paper. Please rewrite/specify better the aim.

8) The discussion should involve literature positions, which are new and relevant for the study. Please refer to the recent positions; for instance, you may refer to the issue of content and communication design in the context of human-computer interactions described in many literature positions (e.g. Gawlik-Kobylinska, M. (2018, December). The Four-Dimensional Instructional Design Approach in the Perspective of Human-Computer Interactions. In APPIS (pp. 146-156).

9) Please provide the information about the study's limitations. 

I would suggest reconsidering the paper after major revision.

Author Response

Response to Reviewer 3 Comments

 

Point 1: Between the lines 25-26 please insert an explanation info on what basis you picked up the countries such as Nigeria, Russia, Serbia, etc.

 

Response 1: The basis from picked those countries is to know the implementation differences of LMS in other continents in the world.

 

Point 2: If you did your research in Indonesia, please provide this information in the Introduction.

 

Response 2: Thank you for the reminder, we have added information about it.

Therefore, it is essential to know the aspects that influence the acceptance of LMS users to improve the student learning experience in the Indonesia Teacher Training Program (PPG).

 

Point 3: Also, in the Introduction, add the aim, research questions, and hypotheses.

 

Response 3: Thanks for the suggestion, in manuscript we already add the aim.

Therefore, this study aims to promote the shortest path yet novel modification DeLone and McLean (D&M). The model is used to discover the user’s understanding of LMS's impact. Our proposed model sees LMS users and developers as equal entities.

 

Point 4: Figure 1 is blurred. Please provide a better resolution of a picture.

 

Response 4: All done, we change the Figure with the new one.

Figure 1. Main Page of PPG's LMS.

 

Point 5: In the line 103, the abbreviation IS should be explained.

 

Response 5: We already state the abbreviation of IS in the last paragraph of the Introduction.

Therefore, this study aims to understand these causes successfully using the information systems (IS) model derived from a novel modified DeLone and McLean (D&M).

 

Point 6: The lines 294-297 are in italics. Why? If italics are not necessary, please use a plain text.

 

Response 6: Sorry we make a mistake. We already rewrite in normal style (not italic). Thanks for the correction.

 

Point 7: In the lines 8-9 the aim of the paper is "this study aims to evaluate the success of learning management systems using the modified DeLone and McLean model", whereas in the Discussion "This paper promotes a new modification of Delene and McLean's success model". It is a bit beyond the goal of the paper. Please rewrite/specify better the aim.

 

Response 7: Thank you for the review suggestions. We already rewrite the aim in the Discussion section.

This paper evaluates the promote the shortest path yet novel modification DeLone and McLean (D&M). The model is used to determine the user's comprehension of the LMS's effects.

 

Point 8: The discussion should involve literature positions, which are new and relevant for the study. Please refer to the recent positions; for instance, you may refer to the issue of content and communication design in the context of human-computer interactions described in many literature positions (e.g. Gawlik-Kobylinska, M. (2018, December). The Four-Dimensional Instructional Design Approach in the Perspective of Human-Computer Interactions. In APPIS (pp. 146-156).

 

Response 8: Thank you for the review suggestions. We added more literature as suggested. The previous was 26, and now it is 37, including your recommendation paper.

Additionally, it demonstrates that students understand the value of the material offered through the LMS and are content with the present level of quality available through the LMS [11].

Additional references for revised version:

[1]         F. Lerro et al., “Integration of an e-learning platform and a remote laboratory for the experimental training at distance in engineering education,” in 2012 9th International Conference on Remote Engineering and Virtual Instrumentation (REV), Jul. 2012, pp. 1–5, doi: 10.1109/REV.2012.6293119.

[2]         D. Weaver, C. Spratt, and C. S. Nair, “Academic and student use of a learning management system: Implications for quality,” Australas. J. Educ. Technol., vol. 24, no. 1, Jan. 2008, doi: 10.14742/ajet.1228.

[3]         J. Delors, “The International Commission on Education for the Twenty-First Century,” 1996.

[4]         H. Xie, H.-C. Chu, G.-J. Hwang, and C.-C. Wang, “Trends and development in technology-enhanced adaptive/personalized learning: A systematic review of journal publications from 2007 to 2017,” Comput. Educ., vol. 140, p. 103599, Oct. 2019, doi: 10.1016/j.compedu.2019.103599.

[5]         G. Tuparov, D. Keremedchiev, D. Tuparova, and M. Stoyanova, “Gamification and educational computer games in open source learning management systems as a part of assessment,” in 2018 17th International Conference on Information Technology Based Higher Education and Training (ITHET), Apr. 2018, pp. 1–5, doi: 10.1109/ITHET.2018.8424768.

[6]         H. Hamidi and A. Chavoshi, “Analysis of the essential factors for the adoption of mobile learning in higher education: A case study of students of the University of Technology,” Telemat. Informatics, vol. 35, no. 4, pp. 1053–1070, Jul. 2018, doi: 10.1016/j.tele.2017.09.016.

[7]         S. A. Aljawarneh, “Reviewing and exploring innovative ubiquitous learning tools in higher education,” J. Comput. High. Educ., vol. 32, no. 1, pp. 57–73, Apr. 2020, doi: 10.1007/s12528-019-09207-0.

[8]         M. Cantabella, R. Martínez-España, B. Ayuso, J. A. Yáñez, and A. Muñoz, “Analysis of student behavior in learning management systems through a Big Data framework,” Futur. Gener. Comput. Syst., vol. 90, pp. 262–272, Jan. 2019, doi: 10.1016/j.future.2018.08.003.

[9]         C. Guan, J. Mou, and Z. Jiang, “Artificial intelligence innovation in education: A twenty-year data-driven historical analysis,” Int. J. Innov. Stud., vol. 4, no. 4, pp. 134–147, Dec. 2020, doi: 10.1016/j.ijis.2020.09.001.

[10]       A. N. Bahasoan, Wulan Ayuandiani, Muhammad Mukhram, and Aswar Rahmat, “Effectiveness of Online Learning In Pandemic Covid-19,” Int. J. Sci. Technol. Manag., vol. 1, no. 2, pp. 100–106, Jul. 2020, doi: 10.46729/ijstm.v1i2.30.

[11]       M. Gawlik-Kobylinska, “The Four-Dimensional Instructional Design Approach in the Perspective of Human-Computer Interactions.,” in APPIS, 2018, pp. 146–156.

 

Point 9: Please provide the information about the study's limitations.

 

Response 9: The study’s limitation is a limited focus on technology. We add the describe of limitation in the conclusion.

However, the modified D&M only focuses on the technological aspect, these research can not provide the non-technological aspect, such as user behavior and external factors. The model developed in this work can serve as a springboard with additional non-technological aspects for future research.

 

Author Response File: Author Response.pdf

Reviewer 4 Report

INTRODUCTION

 

Line 23: should be teachers' not teacher's

Li 38: what is meant by Argentina (USA)?

Li 70: LSM should be LMS

 

THEORETICAL FRAMEWORK

 

Li 103: Delone should be DeLone

Li 119: The box in Figure 3 should be 'User Satisfaction'

Li 122: X should be 3

 

METHOD and RESULT ANALYSIS

 

Li 138: Delete 'item'

Li 154: It can be implied from this that the finalised questionnaire contained 36 items. However, such information should be explicitly stated, together with more details of the questionnaire, including how the 36 items were mapped to the six variables.

 

Sole reliance on a short, 'snapshot' questionnaire is a very limiting way to address H4, H5 & H6 in particular. Why were no user interviews conducted against which to triangulate and help validate the questionnaire responses? Some attempt appears to have been made to assess face validity of the questionnaire items, but far more detail is needed on how this was conducted (representativeness of the sample, questions asked, analysis of consensus, etc). The reliance on linear regression analysis is outdated: it does not address the construct validity and criterion validity of the outcomes and fails to realise the many benefits of a mixed-methods approach.

 

DISCUSSION and CONCLUSION

 

Li 342: should be 'strategies'

 

It's not made clear what is the significance of the minor modification made to the DeLone and McLean Model. The study appears to be no more than an attempt to fit limited data into an existing theoretical framework, with a nominal reconceptualisation of Net-Benefit. It is claimed [Li 367] that "this study can assist educational institutions in identifying system characteristics ... that should be improved". However, no specific advice is given as to how this might be achieved.

 

The discussion reveals a rather simplistic belief in 'education as instruction' – a transactional process of information delivery. While this perspective may be slightly more relevant to STEM disciplines it is not in many others, where a more nuanced focus is on the interpretation and creation of information within societal contexts. Hence, research into how Learning Management Systems might be employed to support collaborative learning would be of more practical use to educators than the quantification of factors around a theoretical model.

 

Author Response

Response to Reviewer 4 Comments

 

Point 1: INTRODUCTION: Line 23: should be teachers' not teacher's.

 

Response 1: Thank you for the correction. We already changet it to teachers.

One standard human development program is teachers training and development in this context.

 

Point 2: INTRODUCTION: Li 38: what is meant by Argentina (USA)?

 

Response 2: The point is to show that Argentina is in the Americas, but it is wrong to write (USA). So that in the manuscript we have omitted it, so it becomes Argentina only. Thanks for the correction.

In Argentina, a teacher training program is conducted to use digital technology for mathematics teachers.

 

Point 3: INTRODUCTION: Li 70: LSM should be LMS

 

Response 3: Thank you for the correction. We already changed it to LMS.

Our proposed model sees LMS users and developers as equal entities.

 

Point 4: THEORETICAL FRAMEWORK: Li 103: Delone should be DeLone

 

Response 4: Thank you for the correction. We already changed it to DeLone.

One model that can be used to measure it is the updated DeLone and McLean (D&M) model [11].

 

Point 5: : THEORETICAL FRAMEWORK: Li 119: The box in Figure 3 should be 'User Satisfaction'

 

Response 5: Thank you for the correction. We already changed it to User Satisfaction in the Figure 3.

Figure 3. Modified D&M Model.

 

Point 6: THEORETICAL FRAMEWORK: Li 122: X should be 3

 

Response 6: Thank you for the correction. We already changed it to Figure 3.

From Figure 3, the net benefit is not the final construct or the key to the success of an IS..

 

Point 7: METHOD and RESULT ANALYSIS: Li 138: Delete 'item'

 

Response 7: Thank you for the correction. We already delete it.

The instrument will be evaluated, arranged into a questionnaire, and given to the respondent.

 

Point 8: METHOD and RESULT ANALYSIS: Li 154: It can be implied from this that the finalised questionnaire contained 36 items. However, such information should be explicitly stated, together with more details of the questionnaire, including how the 36 items were mapped to the six variables.

 

Response 8: Thank you for the review. We already add the more details about the 36 items were mapped to the six variables in Table 4.

Table 4. Questionnare for PPG LMS user.

No.	Variable	Item
1	System Quality	I find the PPG LMS easy to use
2		I find it easy to get the PPG LMS do what I want
3		The PPG LMS is flexible to interact with
4		Learning to operate the PPG LMS was easy for me
5		I fine the PPG LMS website design is interesting
6		It is easy for me to do exercise using technology (PPG LMS)
7		User privacy are provided in PPG LMS
8		PPG LMS give notification if there something happen with the system.
9	Information Quality	The information generated by the PPG LMS is correct
10		The information generated by the PPG LMS is useful for its purpose
11		The PPG LMS generates information in a timely manner
12		I trust the information output of the PPG LMS
13		The information in PPG LMS provide me with the opportunity to practice
14		Easy to find information that I need
15	Service Quality	There is adequate technical support from the system’s provider
16		The overall infrastructure in place is adequate to support the PPG LMS
17		The PPG LMS can be relied on to provide information as when needed
18		The output of the PPG LMS is complete for study processes
19	Use	Using the PPG LMS enables me accomplish tasks more quickly
20		Using the PPG LMS has improved my study performance
21		Using the PPG LMS has made my study easier
22		I like doing exercise in PPG LMS
23		I think exercise done on PPG LSM are beneficial
24		I find the PPG LMS useful in my study
25		Easy to submit the tasks
26	User Satisfaction	I am satisfied with the functions of the PPG LMS
27		I am satisfied with the learning material provided  in PPG LMS
28		the video quality are excelent
29		The PPG LMS has eased work processes
30		I am generally satisfied using the PPG LMS
31	Net Benefits	The PPG LMS will help overcome the limitations of the paper-based system
32		Using the PPG LMS will cause an improvement in my teaching performance
33		The PPG LMS facilitates easy access to teaching materials
34		The PPG LMS will enhance communication among user
35		PPG LMS use will cause improved decision making
36		PPG LMS help me complated my task efectively

 

Point 9: METHOD and RESULT ANALYSIS: Sole reliance on a short, 'snapshot' questionnaire is a very limiting way to address H4, H5 & H6 in particular. Why were no user interviews conducted against which to triangulate and help validate the questionnaire responses? Some attempt appears to have been made to assess face validity of the questionnaire items, but far more detail is needed on how this was conducted (representativeness of the sample, questions asked, analysis of consensus, etc). The reliance on linear regression analysis is outdated: it does not address the construct validity and criterion validity of the outcomes and fails to realise the many benefits of a mixed-methods approach.

 

Response 9: Thank you for the review. A simple interview is also conducted to get additional facts.

 

Point 10: DISCUSSION and CONCLUSION: Li 342: should be 'strategies'

 

Response 10: Thank you for the correction. We already changed it to strategies.

The strategies could be different if the developer implements the original DeLone and McLane Models.

 

Point 11: DISCUSSION and CONCLUSION: It's not made clear what is the significance of the minor modification made to the DeLone and McLean Model. The study appears to be no more than an attempt to fit limited data into an existing theoretical framework, with a nominal reconceptualisation of Net-Benefit. It is claimed [Li 367] that "this study can assist educational institutions in identifying system characteristics ... that should be improved". However, no specific advice is given as to how this might be achieved.

 

Response 11: Thank you for the suggestions. We add some specific advice to be achieved in Discussion section.

A periodic assesment using this modifed model should be conducted to evaluate the fitness of LMS with the user needs. Once it is unsuitable a developmen based on the gaps should be performed.

 

Point 12: DISCUSSION and CONCLUSION: The discussion reveals a rather simplistic belief in 'education as instruction' – a transactional process of information delivery. While this perspective may be slightly more relevant to STEM disciplines it is not in many others, where a more nuanced focus is on the interpretation and creation of information within societal contexts. Hence, research into how Learning Management Systems might be employed to support collaborative learning would be of more practical use to educators than the quantification of factors around a theoretical model.

 

Response 12: Thank you for the suggestion. Excellent input for the development of our research on how Learning Management Systems, we add your suggestion to our paper for our future research perfection in Conclusion section.

This research viewpoint may be slightly more applicable to STEM fields, but it is not applicable to many other fields, such as those that place a more nuanced emphasis on the interpretation and production of information within the context of social settings. Hence, future research into how LMS may be used to enhance collaborative learning would be of greater practical benefit to educators than the measurement of components surrounding a theoretical model would be of help to them.

 

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

Overall, the research process and the results could be explained more thoroughly to make the study more transparent and informative.

I would like to know more about the ethical issues of this research.

 Author(s) need to mention ethical issues for their study and the relations between science and society. I propose to add the following reference:

Petousi, V., & Sifaki, E. (2020). Contextualizing harm in the framework of research misconduct. Findings from a discourse analysis of scientific publications, International Journal of Sustainable Development, 23(3/4), 149-174, https://doi.org/10.1504/IJSD.2020.10037655 

I applaud all the efforts of the author(s) for this research and the revised version of this manuscript. 

 

 

Comments for author File: Comments.pdf

Author Response

Response to Reviewer 1 Comments

 

Point 1: Overall, the research process and the results could be explained more thoroughly to make the study more transparent and informative.

 

Response 1: We already add more details on the research process and the result, as requested by reviewers.

Data collection is essential to this research flow, providing the information necessary to answer research questions and test hypotheses. A general overview of the data collection process in research is developed data instrument, for example, as in Table 4. In the table, we used 36 questions for data collection. The instrument is used to collect data from participants, and it can take many different forms depending on the research question and the type of data being collected.

The next step is data analysis. Data analysis aims to identify patterns, relationships, and trends within the collected data and to use these findings to answer the research question or test the research hypothesis. In this study, the analysis uses T-test and R Square. The primary function of a t-test is to determine whether there is a significant difference between two groups. R-Square is used to determine the usefulness of a regression model.

The last process is data interpretation. Data interpretation is used to analyze and make sense of the data collected during the research process. Data interpretation aims to draw conclusions and make inferences from the data that can be used to answer the research question or test the research hypothesis. In this study, we proposed modifed DeLone and McLean model to interpret the data.

 

Point 2: I would like to know more about the ethical issues of this research.

Author(s) need to mention ethical issues for their study and the relations between science and society.

 

I propose to add the following reference: Petousi, V., & Sifaki, E. (2020). Contextualizing harm in the framework of research misconduct. Findings from a discourse analysis of scientific publications, International Journal of Sustainable Development, 23(3/4), 149-174, https://doi.org/10.1504/IJSD.2020.10037655.

 

Response 2: Thanks for the review and suggestion. We add the information about the ethical issue of our research and also cited your suggestion reference.

The research plan, including the questionnaire, has been presented in front of the research reviewer at Universitas Negeri Malang. Even though it is not following with the issue of ethical clearance, the acceptance of the university reviewer means that this study follows ethical conduct to avoid harm and research misconduct [11].

 

Additional references for the revised version:

[1]         F. Lerro et al., “Integration of an e-learning platform and a remote laboratory for the experimental training at distance in engineering education,” in 2012 9th International Conference on Remote Engineering and Virtual Instrumentation (REV), Jul. 2012, pp. 1–5, doi: 10.1109/REV.2012.6293119.

[2]         D. Weaver, C. Spratt, and C. S. Nair, “Academic and student use of a learning management system: Implications for quality,” Australas. J. Educ. Technol., vol. 24, no. 1, Jan. 2008, doi: 10.14742/ajet.1228.

[3]         J. Delors, “The International Commission on Education for the Twenty-First Century,” 1996.

[4]         H. Xie, H.-C. Chu, G.-J. Hwang, and C.-C. Wang, “Trends and development in technology-enhanced adaptive/personalized learning: A systematic review of journal publications from 2007 to 2017,” Comput. Educ., vol. 140, p. 103599, Oct. 2019, doi: 10.1016/j.compedu.2019.103599.

[5]         G. Tuparov, D. Keremedchiev, D. Tuparova, and M. Stoyanova, “Gamification and educational computer games in open source learning management systems as a part of assessment,” in 2018 17th International Conference on Information Technology Based Higher Education and Training (ITHET), Apr. 2018, pp. 1–5, doi: 10.1109/ITHET.2018.8424768.

[6]         H. Hamidi and A. Chavoshi, “Analysis of the essential factors for the adoption of mobile learning in higher education: A case study of students of the University of Technology,” Telemat. Informatics, vol. 35, no. 4, pp. 1053–1070, Jul. 2018, doi: 10.1016/j.tele.2017.09.016.

[7]         S. A. Aljawarneh, “Reviewing and exploring innovative ubiquitous learning tools in higher education,” J. Comput. High. Educ., vol. 32, no. 1, pp. 57–73, Apr. 2020, doi: 10.1007/s12528-019-09207-0.

[8]         M. Cantabella, R. Martínez-España, B. Ayuso, J. A. Yáñez, and A. Muñoz, “Analysis of student behavior in learning management systems through a Big Data framework,” Futur. Gener. Comput. Syst., vol. 90, pp. 262–272, Jan. 2019, doi: 10.1016/j.future.2018.08.003.

[9]         C. Guan, J. Mou, and Z. Jiang, “Artificial intelligence innovation in education: A twenty-year data-driven historical analysis,” Int. J. Innov. Stud., vol. 4, no. 4, pp. 134–147, Dec. 2020, doi: 10.1016/j.ijis.2020.09.001.

[10]       A. N. Bahasoan, Wulan Ayuandiani, Muhammad Mukhram, and Aswar Rahmat, “Effectiveness of Online Learning In Pandemic Covid-19,” Int. J. Sci. Technol. Manag., vol. 1, no. 2, pp. 100–106, Jul. 2020, doi: 10.46729/ijstm.v1i2.30.

[11]   E. Sifaki and V. Petousi, “Contextualising harm in the framework of research misconduct. Findings from discourse analysis of scientific publications,” Int. J. Sustain. Dev., vol. 23, no. 3/4, p. 149, 2020, doi: 10.1504/IJSD.2020.10037655.

[12]       M. Gawlik-Kobylinska, “The Four-Dimensional Instructional Design Approach in the Perspective of Human-Computer Interactions.,” in APPIS, 2018, pp. 146–156.

 

Author Response File: Author Response.pdf

Reviewer 3 Report

Dear Author(s),

Please add research questions, hypotheses and methods to the Introduction. The purpose of this is to help the reader better understand the content.

As a reviewer, I must evaluate the research design, questions, hypotheses and methods clearly stated.

Could you please explain the basis for choosing the countries you mentioned in the introduction? Is it random? It should be stated at the beginning.

Please fix small errors: Line 34 "LSM" instead of "LMS"

Author Response

Response to Reviewer 3 Comments

 

Point 1: Please add research questions, hypotheses and methods to the Introduction. The purpose of this is to help the reader better understand the content.

 

As a reviewer, I must evaluate the research design, questions, hypotheses and methods clearly stated.

 

Response 1: Thanks for the review, we already add the research questions, hypotheses and methods to the Introduction.

This study raises the main question of whether the proposed model can indicate the user experience. This study is divided into several chapters to understand the content better. The theoretical framework explains the LMS theory, the default of the DeLone and McLean model and each modification, and the research hypotheses (H1-H9). The next chapter explains the research method, which consists of data collection, analysis, and interpretation. We present the result in a statistical report for each hypothesis. In the discussion, we promote Net-Benefit and its influential factors as a consequence of the modified DeLone and McLean model. The final chapter concluded the result and recommended it for future research.

 

Point 2: Could you please explain the basis for choosing the countries you mentioned in the introduction? Is it random? It should be stated at the beginning.

 

Response 2: Thankyou for remind as the comment. The first ssentence in the paragraph 8 is confusing. We realized that we have to change the following sentence “The basis from picked those countries is to know the implementation differences of LMS in other continents in the world” into “The LMS are globally use but the implementation is unique based on its user characteristic”

 

Point 3: Please fix small errors: Line 34 "LSM" instead of "LMS".

 

Response 3: Thanks for your correction. We already changed it to LMS.

A specific model of LMS for Serbia and Russian teacher are difficult to find.

 

Author Response File: Author Response.pdf

Round 3

Reviewer 3 Report

Dear Authors,

Thank you very much for the improvements. 

- removed for peer-review -