Next Article in Journal
Adaptive Kernel Graph Nonnegative Matrix Factorization
Next Article in Special Issue
Engagement with Optional Formative Feedback in a Portfolio-Based Digital Design Module
Previous Article in Journal
Sharing Augmented Reality between a Patient and a Clinician for Assessment and Rehabilitation in Daily Living Activities
Previous Article in Special Issue
How a Rubric Score Application Empowers Teachers’ Attitudes over Computational Thinking Leverage
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Information
Volume 14
Issue 4
10.3390/info14040207
information-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Identification of Critical Parameters Affecting an E-Learning Recommendation Model Using Delphi Method Based on Expert Validation

by
Abubaker Salem Mohamed Shibani
1,2,
Masnizah Mohd
1,*,
Ahmad Tarmizi Abdul Ghani
1,
Mohamad Shanudin Zakaria
1 and
Sumaia Mohammed Al-Ghuribi
1,3
1
Faculty of Information Science and Technology, Universiti Kebangsaan Malaysia (UKM), Bangi 43600, Selangor, Malaysia
2
Artificial Intelligence Department, Faculty of Information Technology, Sebha University, Sabha P.O. Box 18758, Libya
3
Department of Computer Science, Faculty of Applied Sciences, Taiz University, Taiz 6803, Yemen
*
Author to whom correspondence should be addressed.
Information 2023, 14(4), 207; https://doi.org/10.3390/info14040207
Submission received: 17 February 2023 / Revised: 20 March 2023 / Accepted: 22 March 2023 / Published: 28 March 2023
(This article belongs to the Special Issue Information Technologies in Education, Research and Innovation)
Browse Figures
Review Reports Versions Notes

Abstract

:
E-learning is an innovative strategy for enhancing teaching and learning in digital environments with the goal of enhancing education. In the same context, recommendation models have been developed for predicting the user’s learning preferences. A task that has become urgently necessary is enhancing the learning process by designing recommendation models for e-learning software that then helps users choose the most pertinent learning materials (contents) from a wide number of sources. The general consensus is that designing a recommendation model for e-learning is influenced by parameters that are related to e-learning, and much effort has been exerted to determine those parameters. However, no agreement has been reached as to what constitutes such parameters. Keeping this issue in mind, this study aims to identify the parameters that should be considered when generating e-learning recommendations in developing countries. On the basis of the relevant literature, with the use of the Delphi method and with aid from e-learning experts, this paper identifies ten critical parameters related to e-learning. The results show that perceived ease of use is the most critical parameter out of the ten e-learning-related parameters, while user preference is the parameter that contributes least to e-learning.

1. Introduction

E-learning is an innovative method for enhancing teaching and learning in virtual environments with the aim of raising educational standards. With the cooperation of the virtual environment, e-learning has, in recent decades, become a strategy that has attracted researchers’ attention when aiming to foster teaching and learning [1,2,3]. E-learning refers to the use of the Internet to provide and enhance a student’s learning environment, methods and strategies, allowing students who have an Internet connection to access it anytime and anywhere with any available device [4]. E-learning can accommodate a more adaptable behaviour compared to traditional learning, which supports only one educational model in a particular classroom setting while giving students the same course materials without considering their individual needs, preferences or traits [5,6]. An e-learning website is a unified collection of interactive online services that provide users and learners with the information and knowledge needed to improve their educational outcomes; E-learning websites are considered a common source of delivering subject-related material to learners. Learners interact with these websites to reduce workload and engage in learning tasks efficiently, effectively and satisfactorily [7]. Developed and developing countries have embraced and adopted e-learning for teaching and learning in order to create a learning environment and deal with the shortcomings of conventional learning [8]. E-learning can be entirely dependent on a learning management system, or can be combined with traditional teaching–learning methods such as chalkboards [9,10]. These online learning systems can be helpful in promoting innovative pedagogical approaches [11].
In the same context, a recommendation model has been developed to predict the user’s learning preferences [12,13,14]. Designing recommendation models for e-learning software that aids users in selecting the most relevant learning materials (contents) from a variety of sources has become critically important in improving the learning process [9,15]. Recommendation models are employed in the context of e-learning to predict the user’s interests through the analysis of user data and the extraction of important information, in order to provide users with the best e-learning services [16]. The analysis of user data and the extraction of information for creating future predictions are the fundamental functions of recommendation models [9,17,18,19,20].
Determinants of e-learning have been discussed largely in the literature. For example, Ref [15] argued that identifying the parameters that influence e-learning is the first step toward developing a recommendation model for e-learning. In this vein, numerous studies have discussed important parameters that influence e-learning, yet they have limitations. Despite there being agreement that designing a model for e-learning is influenced by some e-learning-related parameters, and despite the many efforts that have been made to identify those determinants, no agreement has been reached as to what constitutes such parameters [21,22]. Furthermore, little is known about the contribution of some parameters to e-learning, such as interface design [23], website quality [24] and learners’ experience [25]. Much of the current research has focused on the technical aspects, such as technical support, and ignores the other aspects related to e-learning users [15]. Thus, more research is needed to validate the parameters that impact e-learning [17]. In this context, critical parameters that affect e-learning need to be validated to design recommendation models that can make e-learning systems user-friendly. Poorly designed recommendation models can make e-learning systems less user-friendly [26]. Accordingly, an important task is to identify critical parameters that affect e-learning, especially based on the knowledge of experts and users’ perspective [26,27,28]. On the basis of the discussion above, the current study aims to identify the critical parameters that should be considered in an e-learning recommendation model from an expert’s perspective. Such critical parameters can help future research design more effective recommendation systems for e-learning.
The importance of addressing the determinants of e-learning increases in the context of developing countries [21], because the adoption of e-learning in developing countries is still in its infancy or has not yet been launched [22]. E-learning can decrease the cost of education by cutting out traditional costs (such as those for classrooms and textbooks) to the point at which developing countries can afford it. The deployment of e-learning in developing countries, however, may initially encounter infrastructural issues that prevent it from supporting the state-of-the-art in education [29]. Beyond basic education, developing countries can use e-learning to acquire skills, which is crucial for nations looking to boost employment and competitiveness. This approach will also make them more appealing to foreign investment, while also fostering a business and entrepreneurial culture that is tailored to and meets local needs [30]. Therefore, the main problems need to be understood, and the parameters that affect e-learning in developing countries that may result in the success of e-learning in these regions need to be identified [31].
The remainder of this paper is organised as follows: Section 2 reviews the literature, and Section 3 presents the methodology. Section 4 highlights the Delphi method iteration results, and Section 5 presents the conclusions of the study.

2. Related Work

This study aims to identify the parameters that influence e-learning in developing countries. The current literature argues that e-learning is influenced by many parameters [22]. For example, some studies, such as [32,33], have mentioned that learners’ attitudes are an important e-learning parameter because student attitudes can contribute to the relatively successful transition from face-to-face learning to e-learning [33]. Similarly, Ref [34,35] mentioned that students’ knowledge level significantly affects e-learning outcomes by enhancing students’ acceptance and benefits from e-learning. Researchers have also asserted that one of the distinguishing characteristics of personalised e-learning is its usefulness and capacity to identify conceptual learning issues and provide the next learning directions [29].
In the same vein, flexibility is considered a key differentiator in favour of e-learning design [36,37]. Alshehri et al. [38] found that the preference of students/users is one of the most important parameters influencing students’ use of e-learning. In addition, Ref [39] found that perceptions of usefulness and enjoyability are critical parameters in students’ willingness to accept and use e-learning. Abdous in [40] argued that self-efficacy is important to ease students’ transition into becoming self-directed learners by clarifying the expectations, roles and responsibilities of online learning. Furthermore, self-efficacy, according to [40], is crucial in order to facilitate students’ transition towards becoming self-directed learners by outlining the goals, expectations and tasks of online learning. Al Mulhem in [41] found that content quality has a positive and significant effect on students’ satisfaction with e-learning.
Information quality has also been identified as an important parameter in e-learning [42,43]. As such, Ref [43] mentioned that the perceived ease of use is always a strong predictor of attitudes towards e-learning, which in turn, enhance students’ willingness to adopt e-learning. Hammad et al. [30] examined the accessibility of 11 e-learning websites from 11 countries in the Middle East and found several content accessibility issues, which hinder students from benefitting from the e-learning websites. Aning and Baharum [44] mentioned that the design and implementation of the e-learning system should be simple to reduce the complexity of utilising the application. Similarly, Ref [24] mentioned that the e-learning website quality must be adequately taken into consideration to avoid negative consequences for students and to enhance students’ trust in e-learning. Other parameters have also been mentioned in the literature as crucial determinants of e-learning, such as rating similarity [45,46], the preference for user similarity [9], service delivery [47], learners’ experience [25,48], learning ability [49], technical support [50] and reputation [51,52]. In sum, the literature presents various parameters that influence e-learning. The 20 parameters that were extracted from the related literature on e-learning studies are presented in Table 1.
This study will review the existing literature that focuses on the parameters that influence students’ use of e-learning resources. The reviewed literature will predominantly include studies on university e-learning from three developing countries (Libya, Yemen and Iraq).

3. Methodology

This research aims to identify the critical parameters that affect e-learning. We conducted a structured literature review of academic articles selected from a large pool of articles on e-learning. For this, refs. [75,76,77] suggested the Delphi method, which is commonly used in higher education research. It is defined as a structured cyclical process for collecting and condensing expert knowledge or feedback in a particular area of interest through a series of, most typically, three iterations of research [78,79]. Dawood et al. [80] mentioned that the Delphi method is used to deal with the uncertainty iteration of some variables’ ‘grey zone’, thus ensuring a satisfactory analysis result. Delphi is used to gather the anonymous opinions of professionals on a vague subject or challenging issue. A typical Delphi strategy involves three iterations of data collection and of the analysis methods used [81,82].
The gathered information is then summarised and sent back to the original panel of experts for approval and evaluation after each Delphi iteration. The participants then have the option to add more comments during this process until the third iteration is over [78]. Participants’ mental models are validated when they use additional feedback information to reconstruct them, leading to higher quality judgments [83]. The Delphi method was utilised in this study to identify the best list of critical parameters that affect e-learning by acquiring the consensus of experts on the appropriateness of the preselected parameters and in order to remove the unfit parameters based on the procedure proposed by [76], as depicted in Figure 1. The Delphi method’s steps are described below.

3.1. Step 1: Select Session Originator

The primary researcher in this study served as the sessions’ originator for the comments made by the panel of experts. The researcher had the necessary knowledge. He distributed the questionnaires to experts, monitored their adherence to the instructions and the presentation of their opinions, gathered the responses, and identified shared and divergent points of view. The process kept going in order to create a consensus and gradually move towards synthesis.

3.2. Step 2: Select Experts

The suggested sample size varies greatly across the literature. Scholars suggest that the minimum and maximum admissible sample size for experts is between four and eight participants, respectively [76,84,85,86]. Participants with the necessary competency were selected using a purposive sample strategy [83]. Seven experts (three from Libya, two from Yemen and two from Iraq) were chosen to ensure that they offered a variety of perspectives and the requisite information about e-learning in developing countries. This sample size is large enough to allow for the development of various viewpoints on the subject under study, while being manageable in terms of translating participant responses to Likert scale questions. The researchers ensured the anonymity of the seven experts throughout the research. The participants were qualified for inclusion in the sample if they met the following requirements:
  • Knowledge of technology-based learning or self-regulated learning through teaching or research experience.
  • Skills in developing digital learning applications.
  • Staff from the university administration taking part in strategic e-learning innovation decisions.

3.3. Step 3: The Delphi Iteration

Three iterations of the Delphi method with seven e-learning experts were conducted to fulfil the study’s goal. All the experts had expertise in managing, creating and teaching online courses for an average of eight years. As a result, they were highly experienced in e-learning systems and were familiar with their difficulties. An online survey method was used to conduct the three iterations of the Delphi method online. During the period between 15 June to 6 September 2022, data were collected for the three iterations. The data for the first iteration were collected between 15 June to 6 July 2022. The data for the second iteration were collected between 21 July 2022 to 7 August 2022. The data for the third iteration were collected between 20 August to 6 September 2022. In the first iteration, the experts were asked to identify and select the most critical parameters for e-learning. A five-point Likert scale was used in order to ask the experts to express how far they agreed with the most critical parameters that affect e-learning. The participants were asked to express their opinion and feelings by selecting critical parameters with respect to each parameter’s influence on e-learning. Numeric values from 1 to 5 were assigned to each choice of the Likert scale, where 5 was assigned to ‘Strongly agree’ and 1 to ‘Strongly disagree’ [7]. The first iteration produced the most critical candidate parameters for e-learning and eliminated the least critical parameters based on the opinion of the seven experts. With the use of the findings from the first iteration, the experts were asked again to give their agreement about the most critical parameters that affect e-learning by using a five-point Likert scale in the second iteration. The candidate parameters were summarised after the completion of the first and second iterations and sent back to the experts for approval and evaluation. In the third iteration, the experts were asked for their agreement ratings regarding the contribution of the parameters derived from the results of the first and second iterations. Furthermore, the researcher conducted personal online interviews with the experts to understand the reasoning behind any variations in their responses between the iterations. The results of each iteration are as follows:

3.3.1. Iteration 1 Result

Twenty parameters were extracted from the literature on e-learning studies, as shown in Table 1. The experts were asked to state how far they agreed with the importance of the most critical parameters related to e-learning by using a five-point rating scale survey. The goal was to obtain expert consensus on the parameters governing e-learning to promote users’ trust. The responses were evaluated statistically by using the Statistical Package for the Social Sciences (SPSS version 27.0). Every parameter’s mean, frequency distribution and percentage were calculated. Table 2 shows the results from the first iteration of the Delphi survey for each parameter.
The results show that the mean score of 20 parameters was between 2.143 and 4.857. On the basis of the mean value, P16 and P19 were the most influential parameters with 4.857, while P6 was the least influential parameter with a mean value of 2.143. Inherently, for the first iteration, a cut-off mean value was imposed where only items with a mean value of 3.5 and above were examined. The rejected 7 parameters had a mean value of <3.5, and the accepted 13 parameters had a mean value of ≥3.5. Consequently, 7 parameters (P1–P7) were discarded, and 13 selected parameters were finalised for iteration 2. A cut-off mean value was automatically set for the first iteration, allowing only items with a mean value of 3.5 and higher to be considered, which means that the responses were in the range of ‘Agree’ to ‘Strongly agree’, as recommended by [76].
The majority of the experts strongly agree that e-learning should be prioritised on the perceived ease of use (85.7%), interface design (85.7%), content quality (71.4%), flexibility (57.1%), perceived accessibility (57.1%), website quality (57.1%), enjoyment (42.9%) and user preferences (42.9%). Experts also agree with the inclusion of perceived system usefulness (57.1%), self-efficacy (57.1%), information quality (42.9%), service delivery (42.9%) and perceived reputation (42.9%).

3.3.2. Iteration 2 Result

In total, 13 parameters from the first iteration were extracted and used in the second iteration, as shown in Table 3. With the use of a five-point rating scale survey, the experts were once more requested to express their agreement on the most critical parameters that affect e-learning. The findings indicate that all the parameters’ average scores were between 3.143 and 5.000. Perceived ease of use had a mean value of 5.000, making it the most influential parameter, while service delivery had a mean value of 3.143, making it the least influential parameter. Similar to the first iteration, a cut-off mean value of 3.5 and above was used to determine which parameters would advance to the third iteration and which ones would be eliminated. The findings indicate that the majority of experts strongly concurred that the following parameters should be taken into consideration in e-learning: perceived ease of use (100%) followed by perceived accessibility (85.7%), flexibility (71.4%), interface design (71.4%), enjoyment (57.1%), perceived system usefulness (42.9%), website quality (42.9%), content quality (42.9) and self-efficacy (42.9%). Furthermore, they agreed that user preference (71.4%) is an important parameter for e-learning.
The sum calculation of disagreement and agreement regarding the parameter results from experts in the second iteration of the Delphi method shows that 3 parameters (service delivery, information quality and perceived reputation) received the lowest score on the Likert scale from experts, while 10 parameters (user preferences, website quality, perceived system’s usefulness, flexibility, enjoyment, self-efficacy, content quality, perceived ease of use, interface design and perceived accessibility) had a mean value of 3.5 and above, as shown in Figure 2.

3.3.3. Iteration 3 Result

In the third iteration, ten parameters were extracted after the completion of the first and second iterations and were sent back to experts for approval and evaluation. In the third iteration, the experts were asked for their agreement on the parameters. All the experts strongly and totally agreed that the parameters of perceived system’s usefulness, flexibility, user preferences, enjoyment, self-efficacy, content quality, perceived ease of use, interface design, website quality and perceived accessibility should have an important role in the development of e-learning.
Furthermore, to rate the contribution of each parameter to e-learning, as extracted in the third iteration, the experts ranked them based on the mean values obtained in the second iteration. Table 4 shows that among the ten parameters, perceived ease of use has the greatest contribution to e-learning, while user preference has the least contribution.

4. Findings

This study was conducted to identify the critical parameters that influence e-learning in developing countries, with ten parameters selected based on expert validation. This study concentrated on experts’ perspectives about the most critical parameters that affect e-learning and sought to collect their agreement about the critical parameters that influence the effectiveness, productivity and user acceptance of e-learning websites, as perceived by undergraduate students in developing countries. The experts were chosen from three developing countries (i.e., Libya Yemen and Iraq), and the study was conducted utilising the three-iteration Delphi method. The data gathered from the three iterations of the Delphi method demonstrated that the experts identified ten critical parameters that influence e-learning. In the third iteration, the parameters were ranked based on their contribution and significance towards e-learning.
The findings suggest that the ten critical parameters that need to be taken into account in e-learning are perceived system usefulness, flexibility, user preferences, enjoyment, self-efficacy, content quality, perceived ease of use, interface design, website quality and perceived accessibility. Usefulness is defined as ‘the users believe that e-learning can help improve their academic performance, by simplifying the entire learning process and completing learning-related tasks in particular’ [58,87]. The results indicate that the recommendation system should be seen as useful to encourage user acceptance and confidence. Flexibility is the ability to adjust to unique or changing requirements. Thus, the adaptability of the recommendation model to learning is crucial. Flexibility is the ability to modify one’s information systems to suit shifting conditions and includes necessary skills, such as activity coordination, change management and programming [88]. Another crucial component is user preference. User preference includes the user’s thoughts on a variety of societal issues, products, services, friends, works, marketing and more [59]. The goal of a recommendation system is to transform information about users and their preferences into predictions of future potential interests; thus, user preferences are typically closely related to the suggestions made by e-learning systems. Although recommendation e-learning systems are not the only places in which user preferences may be seen, they generate vast amounts of data and a strong demand for user preferences, which makes mining and discovering more elements of user preference possible [60]. The enjoyment parameter describes how much using a computer system is perceived as personally enjoyable in and of itself, independent of the technology’s primary function [57,62]. The user’s satisfaction is essential because it will encourage and motivate the user to embrace and trust the suggested e-learning model. An e-learning recommendation model must have a fun component to lure and motivate the user to use it [89,90]. Self-efficacy is another crucial parameter. It is defined as a person’s belief in their capacity to perform tasks successfully. Self-efficacy represents what people believe they are capable of doing based on their abilities or skills, but it is not a measure of one’s competence in and of itself [63,65]. It is essential to ensure that every user has self-efficacy in the recommendation system because advanced technology has a tendency to demotivate and discourage users [91]. Higher levels of computer self-efficacy may enable a person to perform more difficult computer-based tasks compared to individuals with lower levels of computer self-efficacy. Computer self-efficacy reveals how much a user trusts their judgement [92]. A person’s level of self-efficacy with computers is determined by how much they believe they can complete a variety of computer-related tasks. The degree to which the evaluation is constrained to just a few distinct computer-related tasks is indicated by the computer self-efficacy generalisability [64]. People with higher computer self-efficacy generalisability should be more adept at using a variety of software applications and hardware systems than people with lower computer self-efficacy generalisability [66]. Content quality has been also regarded as essential parameter in the same perspective. The usefulness of the information that is provided to the user is measured by the quality of the content. Quality is typically viewed as subjective, which is why different consumers and uses may experience different levels of information quality [58]. Thus, the quality of content must be acknowledged in e-learning [41]. Perceived ease of use is the next parameter, and it is related to a technical system’s display and accessibility. In addition, it conveys an understanding of the perceived ease of use, as a level of trust is created when using a particular technology does not need much effort [69]. To guarantee user acceptability, e-learning must be designed in a user-friendly way. A user-friendly revolutionary technology will typically be positively appreciated by its initial users. The other parameter is the interface design, which is the process by which designers develop software or digital device interfaces with a focus on aesthetics or style [72]. Creating an intuitive and entertaining user interface is the goal of designers. The adoption of e-learning by users is typically enhanced by using an appealing interface [73]. Another crucial parameter is the website’s quality. A website’s quality can be described in terms of the dependability, effectiveness, responsiveness and other aspects of the service, and prior research indicated that these aspects are essential for providing customers with satisfying services in a new recommendation system for technology acceptance [10]. Positive user experiences increase the likelihood that they will return to a website or general online portal [12,93]. Perceived accessibility is the final parameter, defined as the ease with which the user may live a satisfying life with the aid of e-learning as an addition to objective metrics and notions of accessibility [39]. User confidence and acceptability will increase when the perceived accessibility of e-learning is high [94].

5. Conclusions and Future Work

E-learning is a strategy for improving teaching with the support of electronic resources. Designing a recommendation model for e-learning is an important task to predict users’ learning preferences and recommend to them the most relevant learning materials from a various number of sources. Such a recommendation model is influenced by parameters related to e-learning. This study aims to identify the critical parameters that affect the e-learning process in developing countries from the perspective of experts. This study reflects a first step in the endeavour to develop and design more effective recommendation systems for e-learning in developing countries. A structured literature review was conducted on academic articles selected from a large pool of articles on e-learning, and the Delphi method was applied with e-learning experts. The experts were chosen from three developing countries: Libya, Yemen and Iraq. The study was then performed utilising the three-iteration Delphi method. The data gathered from the three iterations of the Delphi method demonstrated that the experts identified ten critical parameters that influence e-learning. In the third iteration of Delphi, the parameters were ranked based on their contribution and significance to e-learning. The findings indicate that, among the ten parameters, perceived ease of use has the greatest impact on e-learning, while user preference has the lowest impact, as indicated by the perspective of the experts. For future work, this study will involve additional experts and developing countries in order to produce an accurate recommendation model for the e-learning field. Moreover, the authors plan to conduct more studies in order to verify the findings via a student survey and the application of the suggested solutions.

Author Contributions

Conceptualization, methodology, formal analysis, validation, A.S.M.S.; writing—original draft preparation, review and editing, A.S.M.S., M.M., A.T.A.G., S.M.A.-G.; supervision, funding acquisition M.M., M.S.Z. All authors have read and agreed to the published version of the manuscript.

Funding

This study was supported by the Fundamental Research Grant Scheme (FRGS/1/2021/ICT02/UKM/02/1) from the Ministry of Higher Education, Malaysia.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Acknowledgments

The authors want to thank all the experts that contribute in this study.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Patra, I.; Hashim Alghazali, T.A.; Sokolova, E.G.; Prasad, K.; Pallathadka, H.; Hussein, R.A.; Shanan, A.J.; Ghaneiarani, S. Scrutinizing the effects of e-learning on enhancing EFL learners’ reading comprehension and reading motivation. Educ. Res. Int. 2022, 2022, 4481453. [Google Scholar] [CrossRef]
  2. Shin, J.L.K.; Yunus, M.M. A Systematic Review of E-Learning in Teaching And Learning of Speaking Skills. Int. J. Acad. Res. Bus. Soc. Sci. 2021, 11, 725–740. [Google Scholar]
  3. Berestova, A.; Burdina, G.; Lobuteva, L.; Lobuteva, A. Academic Motivation of University Students and the Factors That Influence It in an E-Learning Environment. Electron. J. e-Learn. 2022, 20, 201–210. [Google Scholar] [CrossRef]
  4. Chaudhary, K.; Gupta, N. E-learning recommender system for learners: A machine learning based approach. Int. J. Math. Eng. Manag. Sci. 2019, 4, 957. [Google Scholar] [CrossRef]
  5. Chen, F.-H. Sustainable education through e-learning: The case study of ilearn2. 0. Sustainability 2021, 13, 10186. [Google Scholar] [CrossRef]
  6. Nortvig, A.-M.; Petersen, A.K.; Balle, S.H. A Literature Review of the Factors Influencing E-Learning and Blended Learning in Relation to Learning Outcome, Student Satisfaction and Engagement. Electron. J. e-Learn. 2018, 16, 46–55. [Google Scholar]
  7. Wahit, F.B.; Mohd, M. Evaluation on usability of enhancement e-learning of PTPL College Sabah with social networking elements. Procedia Technol. 2013, 11, 1096–1102. [Google Scholar] [CrossRef] [Green Version]
  8. Muhammad, A.H.; Siddique, A.; Youssef, A.E.; Saleem, K.; Shahzad, B.; Akram, A.; Al-Thnian, A.-B.S. A hierarchical model to evaluate the quality of web-based e-learning systems. Sustainability 2020, 12, 4071. [Google Scholar] [CrossRef]
  9. Dash, G. Pandemic induced e-learning and the impact on the stakeholders: Mediating role of satisfaction and moderating role of choice. Athens J. Educ. 2022, 9, 1–22. [Google Scholar] [CrossRef]
  10. Ali, S.; Hafeez, Y.; Humayun, M.; Jamail, N.S.M.; Aqib, M.; Nawaz, A. Enabling recommendation system architecture in virtualized environment for e-learning. Egypt. Inform. J. 2022, 23, 33–45. [Google Scholar] [CrossRef]
  11. Tahir, S.; Hafeez, Y.; Abbas, M.A.; Nawaz, A.; Hamid, B. Smart Learning Objects Retrieval for E-Learning with Contextual Recommendation based on Collaborative Filtering. Educ. Inf. Technol. 2022, 27, 8631–8668. [Google Scholar] [CrossRef]
  12. Ahmed, A.J.; Mohammed, F.H.; Majedkan, N.A. An Evaluation Study of an E-Learning Course at the Duhok Polytechnic University: A Case Study. J. Cases Inf. Technol. 2022, 24, 1–11. [Google Scholar] [CrossRef]
  13. Osman, N.A.; Mohd Noah, S.A.; Darwich, M.; Mohd, M. Integrating contextual sentiment analysis in collaborative recommender systems. PLoS ONE 2021, 16, e0248695. [Google Scholar] [CrossRef] [PubMed]
  14. Al-Ghuribi, S.M.; Noah, S.A.M. Multi-criteria review-based recommender system–the state of the art. IEEE Access 2019, 7, 169446–169468. [Google Scholar] [CrossRef]
  15. Shahbazi, Z.; Byun, Y.-C. Agent-Based Recommendation in E-Learning Environment Using Knowledge Discovery and Machine Learning Approaches. Mathematics 2022, 10, 1192. [Google Scholar] [CrossRef]
  16. Aberbach, H.; Jeghal, A.; Sabri, A.; Tairi, H. E-learning Recommendation Systems: A Literature Review. In Proceedings of the International Conference on Digital Technologies and Applications, Fez, Morocco, 28–30 January 2022; Springer: Berlin/Heidelberg, Germany, 2022; pp. 361–370. [Google Scholar]
  17. Ooge, J.; Kato, S.; Verbert, K. Explaining Recommendations in E-Learning: Effects on Adolescents’ Trust. In Proceedings of the 27th International Conference on Intelligent User Interfaces, Helsinki, Finland, 22–25 March 2022; pp. 93–105. [Google Scholar]
  18. Jamil, N.; Noah, S.M.; Mohd, M. Collaborative item recommendations based on friendship strength in social network. Int. J. Mach. Learn. Comput. 2020, 10, 437–443. [Google Scholar] [CrossRef]
  19. Saat, N.I.Y.; Noah, S.A.M.; Mohd, M. Towards serendipity for content-based recommender systems. Int. J. Adv. Sci. Eng. Inf. Technol. 2018, 8, 1762–1769. [Google Scholar] [CrossRef]
  20. Al-Ghuribi, S.M.; Noah, S.A.M. A comprehensive overview of recommender system and sentiment analysis. arXiv 2021, arXiv:2109.08794. [Google Scholar]
  21. Elshaer, I.A.; Sobaih, A.E.E. FLOWER: An Approach for Enhancing E-Learning Experience Amid COVID-19. Int. J. Environ. Res. Public Health 2022, 19, 3823. [Google Scholar] [CrossRef]
  22. Umar, M.; Ko, I. E-Learning: Direct Effect of Student Learning Effectiveness and Engagement through Project-Based Learning, Team Cohesion, and Flipped Learning during the COVID-19 Pandemic. Sustainability 2022, 14, 1724. [Google Scholar] [CrossRef]
  23. Hasani, L.M.; Santoso, H.B.; Isal, R.Y.K. Designing alternative interface design of E-learning modules based on felder-silverman learning styles and user centered design approach. In Proceedings of the 2019 International Conference on Advanced Computer Science and Information Systems (ICACSIS), Bali, Indonesia, 12–13 October 2019; IEEE: Piscataway, NJ, USA, 2019; pp. 459–464. [Google Scholar]
  24. Pham, L.; Limbu, Y.B.; Bui, T.K.; Nguyen, H.T.; Pham, H.T. Does e-learning service quality influence e-learning student satisfaction and loyalty? Evidence from Vietnam. Int. J. Educ. Technol. High. Educ. 2019, 16, 7. [Google Scholar] [CrossRef] [Green Version]
  25. Regmi, K.; Jones, L. A systematic review of the factors–enablers and barriers–affecting e-learning in health sciences education. BMC Med. Educ. 2020, 20, 91. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  26. Rahayu, N.W.; Ferdiana, R.; Kusumawardani, S.S. A systematic review of ontology use in E-Learning recommender system. Comput. Educ. Artif. Intell. 2022, 3, 100047. [Google Scholar] [CrossRef]
  27. Vlasenko, K.V.; Volkov, S.V.; Lovianova, I.V.; Sitak, I.V.; Chumak, O.O.; Semerikov, S.O.; Bohdanova, N.H. The criteria of usability design for educational online courses. In Proceedings of the 1st Symposium on Advances in Educational Technology (AET 2020), Kyiv, Ukraine, 12–13 November 2020; Volume 1, pp. 461–470, ISBN 978-989-758-558-6. [Google Scholar]
  28. Malanga, A.C.M.; Bernardes, R.C.; Borini, F.M.; Pereira, R.M.; Rossetto, D.E. Towards integrating quality in theoretical models of acceptance: An extended proposed model applied to e-learning services. Br. J. Educ. Technol. 2022, 53, 8–22. [Google Scholar] [CrossRef]
  29. Al-Fraihat, D.; Joy, M.; Sinclair, J. Evaluating E-learning systems success: An empirical study. Comput. Hum. Behav. 2020, 102, 67–86. [Google Scholar] [CrossRef]
  30. Hammad, M.; Alnabhan, M.; Doush, I.A.A.; Alsalem, G.M.; Al-Alem, F.A.; Al-Awadi, M.M. Evaluating usability and content accessibility for e-learning websites in the Middle East. Int. J. Technol. Hum. Interact. 2020, 16, 54–62. [Google Scholar] [CrossRef]
  31. Alshehri, A.; Rutter, M.; Smith, S. Assessing the Relative Importance of an E-Learning System’s Usability Design Characteristics Based on Students’ Preferences. Eur. J. Educ. Res. 2019, 8, 839–855. [Google Scholar] [CrossRef] [Green Version]
  32. Rafiq, F.; Hussain, S.; Abbas, Q. Analyzing students’ attitude towards e-learning: A case study in higher education in Pakistan. Pak. Soc. Sci. Rev. 2020, 4, 367–380. [Google Scholar] [CrossRef]
  33. Johnson, J.B.; Reddy, P.; Chand, R.; Naiker, M. Attitudes and awareness of regional Pacific Island students towards e-learning. Int. J. Educ. Technol. High. Educ. 2021, 18, 1–20. [Google Scholar] [CrossRef]
  34. Al-Chalabi, H.K.M.; Hussein, A.M.A.; Apoki, U.C. An Adaptive Learning System Based on Learner’s Knowledge Level. In Proceedings of the 2021 13th International Conference on Electronics, Computers and Artificial Intelligence (ECAI), Pitesti, Romania, 1–3 July 2021; IEEE: Piscataway, NJ, USA, 2021; pp. 1–4. [Google Scholar]
  35. Botelho, M.G.; Agrawal, K.R.; Bornstein, M.M. An systematic review of e-learning outcomes in undergraduate dental radiology curricula—Levels of learning and implications for researchers and curriculum planners. Dentomaxillofacial Radiol. 2019, 48, 20180027. [Google Scholar] [CrossRef]
  36. Roy, S.; Bhattacharya, S.; Das, P. Identification of e-learning quality parameters in Indian context to make it more effective and acceptable. Proc. Eng. Sci. 2020, 2, 209–222. [Google Scholar] [CrossRef]
  37. Adams, D.; Simpson, K.; Davies, L.; Campbell, C.; Macdonald, L. Online learning for university students on the autism spectrum: A systematic review and questionnaire study. Australas. J. Educ. Technol. 2019, 35, 111–131. [Google Scholar] [CrossRef] [Green Version]
  38. Alshehri, A.; Rutter, M.J.; Smith, S. An implementation of the UTAUT model for understanding students’ perceptions of learning management systems: A study within tertiary institutions in Saudi Arabia. Int. J. Distance Educ. Technol. 2019, 17, 1–24. [Google Scholar] [CrossRef]
  39. Salloum, S.A.; Alhamad, A.Q.M.; Al-Emran, M.; Monem, A.A.; Shaalan, K. Exploring students’ acceptance of e-learning through the development of a comprehensive technology acceptance model. IEEE Access 2019, 7, 128445–128462. [Google Scholar] [CrossRef]
  40. Abdous, M.H. Well begun is half done: Using online orientation to foster online students’ academic self-efficacy. Online Learn. 2019, 23, 161–187. [Google Scholar] [CrossRef] [Green Version]
  41. Al Mulhem, A. Investigating the effects of quality factors and organizational factors on university students’ satisfaction of e-learning system quality. Cogent Educ. 2020, 7, 1787004. [Google Scholar] [CrossRef]
  42. Taat, M.S.; Francis, A. Factors Influencing the Students’ Acceptance of E-Learning at Teacher Education Institute: An Exploratory Study in Malaysia. Int. J. High. Educ. 2020, 9, 133–141. [Google Scholar] [CrossRef] [Green Version]
  43. Ji, Z.; Yang, Z.; Liu, J.; Yu, C. Investigating users’ continued usage intentions of online learning applications. Information 2019, 10, 198. [Google Scholar] [CrossRef] [Green Version]
  44. Aning, A.; Baharum, A. A preliminary study of identifying the visualization pattern of E-learning website for HEIs in Malaysia using card sorting method. TEST Eng. Manag. 2020, 82, 11948–11955. [Google Scholar]
  45. Wan, S.; Niu, Z. A hybrid e-learning recommendation approach based on learners’ influence propagation. IEEE Trans. Knowl. Data Eng. 2019, 32, 827–840. [Google Scholar] [CrossRef]
  46. Tarus, J.K.; Niu, Z.; Mustafa, G. Knowledge-based recommendation: A review of ontology-based recommender systems for e-learning. Artif. Intell. Rev. 2018, 50, 21–48. [Google Scholar] [CrossRef]
  47. Uppal, M.A.; Ali, S.; Gulliver, S.R. Factors determining e-learning service quality. Br. J. Educ. Technol. 2018, 49, 412–426. [Google Scholar] [CrossRef]
  48. Zhang, Q. Enhanced Recommender Systems Through Cross-Domain Knowledge Transfer. Ph.D. Thesis, Master's Thesis. University of Technology Sydney, Sydney, Australia, 2018. [Google Scholar]
  49. Khlifi, Y. An Advanced Authentication Scheme for E-evaluation Using Students Behaviors Over E-learning Platform. Int. J. Emerg. Technol. Learn. 2020, 15, 90–111. [Google Scholar] [CrossRef]
  50. Encarnacion, R.F.E.; Galang, A.A.D.; Hallar, B.J.A. The impact and effectiveness of e-learning on teaching and learning. Online Submiss. 2021, 5, 383–397. [Google Scholar]
  51. Hinz, V.T.; Pimenta, M.S. Integrating Reputation to Recommendation Techniques in an e-learning Environment. In Proceedings of the 2018 17th International Conference on Information Technology Based Higher Education and Training (ITHET), Olhao, Portugal, 26–28 April 2018; IEEE: Piscataway, NJ, USA, 2018; pp. 1–6. [Google Scholar]
  52. Jayachitra, T.A.; Jagannarayan, N. An Empirical Study on Student’s Learning Through e-Learning Modules Offered by Corporate Through Colleges in Mumbai. 2021. Available online: https://thinkindiaquarterly.org/index.php/think-india/article/view/19895 (accessed on 16 February 2023).
  53. Eroglu, M.; Ozbek, R. The investigation of the relationship between attitudes towards e-learning and self-directed learning with technology of secondary school students. Int. Online J. Educ. Sci. 2018, 10, 297–314. [Google Scholar]
  54. Elkahky, A.M.; Song, Y.; He, X. A multi-view deep learning approach for cross domain user modeling in recommendation systems. In Proceedings of the 24th International Conference on World Wide Web, Florence, Italy, 18–22 May 2015; pp. 278–288. [Google Scholar]
  55. Luo, N.; Zhang, Y.; Zhang, M. Retaining learners by establishing harmonious relationships in e-learning environment. Interact. Learn. Environ. 2019, 27, 118–131. [Google Scholar] [CrossRef]
  56. Liu, H.K.J. Correlation research on the application of e-learning to students’ self-regulated learning ability, motivational beliefs, and academic performance. EURASIA J. Math. Sci. Technol. Educ. 2016, 12, 1091–1100. [Google Scholar] [CrossRef]
  57. Al-Rahmi, W.M.; Yahaya, N.; Aldraiweesh, A.A.; Alamri, M.M.; Aljarboa, N.A.; Alturki, U.; Aljeraiwi, A.A. Integrating technology acceptance model with innovation diffusion theory: An empirical investigation on students’ intention to use E-learning systems. IEEE Access 2019, 7, 26797–26809. [Google Scholar] [CrossRef]
  58. Kumar, P.; Saxena, C.; Baber, H. Learner-content interaction in e-learning-the moderating role of perceived harm of COVID-19 in assessing the satisfaction of learners. Smart Learn. Environ. 2021, 8, 5. [Google Scholar] [CrossRef]
  59. Vedavathi, N.; Anil Kumar, K. An efficient e-learning recommendation system for user preferences using hybrid optimization algorithm. Soft Comput. 2021, 25, 9377–9388. [Google Scholar] [CrossRef]
  60. Pariserum Perumal, S.; Sannasi, G.; Arputharaj, K. An intelligent fuzzy rule-based e-learning recommendation system for dynamic user interests. J. Supercomput. 2019, 75, 5145–5160. [Google Scholar] [CrossRef]
  61. Muthuprasad, T.; Aiswarya, S.; Aditya, K.; Jha, G.K. Students’ perception and preference for online education in India during COVID-19 pandemic. Soc. Sci. Humanit. Open 2021, 3, 100101. [Google Scholar] [CrossRef]
  62. Pradhana, R.A. Exploring Students Experience in Online Speaking Class Using Role-Play Technique. J. Engl. Lang. Educ. 2021, 6, 93–102. [Google Scholar]
  63. Latip, M.S.A.; Tamrin, M.; Noh, I.; Rahim, F.A.; Nur, S.; Latip, N.A. Factors Affecting e-Learning Acceptance among Students: The Moderating Effect of Self-efficacy. Int. J. Inf. Educ. Technol. 2022, 12, 116. [Google Scholar] [CrossRef]
  64. Bruijns, B.A.; Vanderloo, L.M.; Johnson, A.M.; Adamo, K.B.; Burke, S.M.; Carson, V.; Heydon, R.; Irwin, J.D.; Naylor, P.-J.; Timmons, B.W. Change in pre-and in-service early childhood educators’ knowledge, self-efficacy, and intentions following an e-learning course in physical activity and sedentary behaviour: A pilot study. BMC Public Health 2022, 22, 244. [Google Scholar] [CrossRef] [PubMed]
  65. Bubou, G.M.; Job, G.C. Individual innovativeness, self-efficacy and e-learning readiness of students of Yenagoa study centre, National Open University of Nigeria. J. Res. Innov. Teach. Learn. 2020, 15, 2–22. [Google Scholar] [CrossRef]
  66. Rahmawati, R.N. Self-efficacy and use of e-learning: A theoretical review technology acceptance model (TAM). Am. J. Humanit. Soc. Sci. Res. 2019, 3, 41–55. [Google Scholar]
  67. Kurniawan, Y.; Candra, S.; Tungka, L.Y. E-Learning: MOOC User Intention Analysis Using TAM and TTF with Social Motivation Factor and MOOC Features. In Digital Literacy and Socio-Cultural Acceptance of ICT in Developing Countries; Springer: Berlin, Germany, 2021; pp. 101–117. [Google Scholar]
  68. Rajasekaran, V.A.; Kumar, K.; Susi, S.; Mohan, Y.; Raju, M.; Hssain, M.W. An Evaluation of E-Learning and User Satisfaction. Int. J. Web-Based Learn. Teach. Technol. 2022, 17, 11. [Google Scholar] [CrossRef]
  69. Vlachogianni, P.; Tselios, N. Investigating the impact of personality traits on perceived usability evaluation of e-learning platforms. Interact. Technol. Smart Educ. 2021, 19, 202–221. [Google Scholar] [CrossRef]
  70. Cinquin, P.-A.; Guitton, P.; Sauzéon, H. Online e-learning and cognitive disabilities: A systematic review. Comput. Educ. 2019, 130, 152–167. [Google Scholar] [CrossRef] [Green Version]
  71. Kaakour, S.; Ali, A.A.; Mostapha, N. Tam extension in e-learning system applicable in private universities in lebanon. BAU J. -Soc. Cult. Hum. Behav. 2022, 3, 2. [Google Scholar] [CrossRef]
  72. Ramadhan, A.; Hidayanto, A.N.; Salsabila, G.A.; Wulandari, I.; Jaury, J.A.; Anjani, N.N. The effect of usability on the intention to use the e-learning system in a sustainable way: A case study at Universitas Indonesia. Educ. Inf. Technol. 2022, 27, 1489–1522. [Google Scholar] [CrossRef]
  73. Nordin, H.; Singh, D.; Mansor, Z.; Yadegaridehkordi, E. Impact of Power Distance Cultural Dimension in E-Learning Interface Design Among Malaysian Generation Z Students. IEEE Access 2022, 10, 64199–64208. [Google Scholar] [CrossRef]
  74. Chopra, G.; Madan, P.; Jaisingh, P.; Bhaskar, P. Effectiveness of e-learning portal from students’ perspective: A structural equation model (SEM) approach. Interact. Technol. Smart Educ. 2019, 16, 94–116. [Google Scholar] [CrossRef]
  75. Ilyas, M. Determining Critical Success Factors for Quality and Accreditation through Delphi Technique. Int. J. High. Educ. 2019, 8, 148–158. [Google Scholar] [CrossRef] [Green Version]
  76. Muhammad, A.; Shaikh, A.; Naveed, Q.N.; Qureshi, M.R.N. Factors affecting academic integrity in E-learning of Saudi Arabian Universities. An investigation using Delphi and AHP. IEEE Access 2020, 8, 16259–16268. [Google Scholar] [CrossRef]
  77. Wei, W.; Chen, S.C.; Qiu, Y. Understanding Hospitality and Tourism Students’ Emotional Intelligence Performance in the E-learning Environment: A Delphi Approach. J. Hosp. Tour. Educ. 2022, 35, 73–87. [Google Scholar] [CrossRef]
  78. Willems, J.; Sutton, K.; Maybery, D. Using a Delphi process to extend a rural mental health workforce recruitment initiative. J. Ment. Health Train. Educ. Pract. 2015, 10, 91–100. [Google Scholar] [CrossRef]
  79. Driessen, S.; Ponds, R.; van Alphen, B.; Nederstigt, A.; Deckers, K.; Sobczak, S. Treating Symptoms of Posttraumatic Stress in People with Dementia: Expert Consensus Using the Delphi Method. Clin. Gerontol. 2023, 1–15. [Google Scholar] [CrossRef]
  80. Dawood, K.A.; Sharif, K.Y.; Ghani, A.A.; Zulzalil, H.; Zaidan, A.; Zaidan, B. Towards a unified criteria model for usability evaluation in the context of open source software based on a fuzzy Delphi method. Inf. Softw. Technol. 2021, 130, 106453. [Google Scholar] [CrossRef]
  81. Gossler, T.; Sigala, I.F.; Wakolbinger, T.; Buber, R. Applying the Delphi method to determine best practices for outsourcing logistics in disaster relief. J. Humanit. Logist. Supply Chain Manag. 2019, 9, 438–474. [Google Scholar] [CrossRef]
  82. Naisola-Ruiter, V. The Delphi technique: A tutorial. Res. Hosp. Manag. 2022, 12, 91–97. [Google Scholar] [CrossRef]
  83. Mirata, V.; Hirt, F.; Bergamin, P.; van der Westhuizen, C. Challenges and contexts in establishing adaptive learning in higher education: Findings from a Delphi study. Int. J. Educ. Technol. High. Educ. 2020, 17, 32. [Google Scholar] [CrossRef]
  84. Youzbashi, A.; Pajhohi, S. Identification of Professors’ Professional Competencies in e-learning environments (Delphi Techniques). Inf. Commun. Technol. Educ. Sci. 2019, 9, 5–25. [Google Scholar]
  85. Lim, H.M.; Ng, C.J.; Teo, C.H.; Lee, P.Y.; Kassim, P.S.J.; Nasharuddin, N.A.; Yong, P.V.C.; Sellappans, R.; Yap, W.H.; Lee, Y.K. Prioritising topics for developing e-learning resources in healthcare curricula: A comparison between students and educators using a modified Delphi survey. PLoS ONE 2021, 16, e0253471. [Google Scholar] [CrossRef] [PubMed]
  86. Bruijns, B.A.; Johnson, A.M.; Tucker, P. Content development for a physical activity and sedentary behaviour e-learning module for early childhood education students: A Delphi study. BMC Public Health 2020, 20, 1600. [Google Scholar] [CrossRef] [PubMed]
  87. Weerathunga, P.R.; Samarathunga, W.; Rathnayake, H.; Agampodi, S.; Nurunnabi, M.; Madhunimasha, M. The COVID-19 pandemic and the acceptance of E-learning among university Students: The Role of Precipitating Events. Educ. Sci. 2021, 11, 436. [Google Scholar] [CrossRef]
  88. Al Rawashdeh, A.Z.; Mohammed, E.Y.; Al Arab, A.R.; Alara, M.; Al-Rawashdeh, B. Advantages and disadvantages of using e-learning in university education: Analyzing students’ perspectives. Electron. J. E-Learn. 2021, 19, 107–117. [Google Scholar] [CrossRef]
  89. Zakaria, M.S. A Preliminary Review on Mobile Users Trust-based Recommendation Systems. In Proceedings of the 2020 IEEE Conference on e-Learning, E-Management and E-Services (IC3e), Kota Kinabalu, Malaysia, 17–19 November 2020; IEEE: Piscataway, NJ, USA, 2020; pp. 76–81. [Google Scholar]
  90. Gurban, M.A.; Almogren, A.S. Students’ Actual Use of E-Learning in Higher Education During the COVID-19 Pandemic. SAGE Open 2022, 12, 21582440221091250. [Google Scholar] [CrossRef]
  91. Bagunaid, W.; Chilamkurti, N.; Veeraraghavan, P. AISAR: Artificial Intelligence-Based Student Assessment and Recommendation System for E-Learning in Big Data. Sustainability 2022, 14, 10551. [Google Scholar] [CrossRef]
  92. Ithriah, S.; Ridwandono, D.; Suryanto, T. Online Learning Self-Efficacy: The Role in E-Learning Success. In Proceedings of the Journal of Physics: Conference Series, Moscow, Russian, 20–21 October 2020; IOP Publishing: Bristol, UK, 2020; Volume 1569, p. 022053. [Google Scholar]
  93. Nugroho, M.A.; Rahmawati, D.; Novitasari, B.T. The influence of website quality on e-learning usage continuity. J. Adv. Res. Dyn. Control. Syst. 2019, 11, 382–388. [Google Scholar]
  94. Aquino, K.C.; BuShell, S. Device usage and accessible technology needs for post-traditional students in the e-learning environment. J. Contin. High. Educ. 2020, 68, 101–116. [Google Scholar] [CrossRef]
Information 14 00207 g001 550
Figure 1. Delphi method steps.
Figure 1. Delphi method steps.
Information 14 00207 g001
Information 14 00207 g002 550
Figure 2. Summary of the total agreement and disagreement with parameters of iteration 2.
Figure 2. Summary of the total agreement and disagreement with parameters of iteration 2.
Information 14 00207 g002
Table
Table 1. eLearning parameters.
Table 1. eLearning parameters.
NoParametersParameter DefinitionRelated Works
1Learners’ attitudes toward e-learningNegative or positive attitudes and attitudes that directly give the behaviours of individuals and imply that learners have positive or negative feelings about their participation in e-learning activities[33,53]
2Knowledge levelThe nature and depth of knowledge, skill and ability in a particular subject, which is considered one of the main features that the adaptation process relies on, to provide the right materials that match the learner’s level[29,34,35]
3Rating similarityThe similarity measure between rating values that use learners’ ratings, which would help other learners acquire appropriate learning materials[45,46,48]
4User similarityThe product between the similarity of interest and a learning style[9,54]
5Learners’ experienceThe learners’ background and experience in using e-learning[25,55]
6Learning abilityThe process by which students can control their actions and guide their personal learning behaviours[49,56]
7Technical supportThis is concerned with support for providing learners with proper, timely assistance in an effective and efficient fashion[38,50]
8Perceived system’s usefulnessThe degree to which a student expects an increase in performance as a result of adopting an e-learning environment[29,57,58]
9FlexibilityThe ability to react to changes in customer learning needs and requirements quickly[36,37]
10User preferenceA mechanism to express users’ interests in items and seamlessly collected clickstream data for inferring users’ interests or preferences for e-learning[59,60,61]
11EnjoymentThe sensation and perception of using the computer as enjoyable, apart from any probable and predictable learning performance consequences[57,62,63]
12Self-efficacyAn individual’s perception of their ability to use computers in accomplishing a learning task[63,64,65,66]
13Content qualityThe availability of materials and services that are directly related to student learning outcomes[41,58]
14Perceived reputationThe beliefs or opinions that are generally held about someone or something, which is one of the measures that can influence one’s interest or one’s behaviour towards the use of a particular technology[52,67]
15Information qualityUsers’ overall judgment and evaluation of the quality of information, assessed by the degree of accuracy, informativeness, timeliness and relevancy of information provided by the website.[42,68]
16Perceived ease of useThe degree to which a person believes that using information technology will be free of effort[69]
17Perceived accessibilityThe ease of living a pleasant life with the assistance of the e-learning system and refers to the person’s perceptions, experiences and expectations of accessibility[39,70,71]
18Service deliveryThese learning services should be delivered in an effective, predictable, reliable and customer-friendly manner as users seek or provide data, handle their affairs or complete tasks.[47]
19Interface designInvolves designs of surfaces or look of screens, buttons, icons, images, text and all visual elements that a learner interacts with during e-learning[47,72,73]
20Website qualityThe degree to which the users believe that the learning website is easy to navigate and are able to interact with it consistently[7,74]
Table
Table 2. Summary of parameters of iteration 1.
Table 2. Summary of parameters of iteration 1.
NoParameters Strongly DisagreeDisagreeNeutralAgreeStrongly AgreeMean
1Learners’ attitudes towards e-learning 3 (42.9%)3 (42.9%) 1 (14.3%)2.857
2Knowledge level2 (28.6%)2 (28.6%) 1 (14.3%)2 (28.6%)2.857
3Rating similarity 4 (57.1%)1 (14.3%)2 (28.6%) 2.714
4User similarity 3 (42.9%)2 (28.6%)2 (28.6%) 2.857
5Learners’ experience3 (42.9%) 1 (14.3%)2 (28.6%)1 (14.3%)2.714
6Learning ability2 (28.6%)2 (28.6%)3 (42.9%) 2.143
7Technical support 3 (42.9%) 2 (28.6%)2 (28.6%)3.429
8Perceived system’s usefulness 4 (57.1%)3 (42.9%)4.429
9Flexibility 1 (14.3%)2 (28.6%)4 (57.1%)4.429
10User preferences 2 (28.6%)2 (28.6%)3 (42.9%)4.143
11Information quality1 (14.3%)1 (14.3%) 3 (42.9%)2 (28.6%)3.571
12Self-efficacy 4 (57.1%)3 (42.9%)4.429
13Content quality 2 (28.6%)5 (71.4%)4.714
14Reputation1 (14.3%)1 (14.3%) 3 (42.9%)2 (28.6%)3.571
15Enjoyment 1 (14.3%)3 (42.9%)3 (42.9%)4.286
16Perceived ease of use 1 (14.3%)6 (85.7%)4.857
17Perceived accessibility 3 (42.9%)4 (57.1%)4.571
18Service delivery1 (14.3%)1 (14.3%) 3 (42.9%)2 (28.6%)3.571
19Interface design 1 (14.3%)6 (85.7%)4.857
20Website quality 2 (28.6%)1 (14.3%)4 (57.1%)4.286
Table
Table 3. Summary of parameters of iteration 2.
Table 3. Summary of parameters of iteration 2.
NoParametersStrongly DisagreeDisagreeNeutralAgreeStrongly AgreeMean
1Service delivery2 (28.6 %)1 (14.3%) 2 (28.6 %)2 (28.6%)3.143
2Information quality1 (14.3%)2 (28.6 %) 2 (28.6 %)2 (28.6%)3.286
3Perceived reputation 1 (14.3%)2 (28.6 %) 2 (28.6 %)2 (28.6%)3.286
4User preferences 2 (14.3%)5 (71.4%) 3.714
5Website quality 2 (28.6%)2 (28.6%)3 (42.9%)4.142
6Perceived system’s usefulness 1 (14.3%)3 (42.9%)3 (42.9%)4.429
7Flexibility 2 (28.6%)5 (71.4%)4.714
8Enjoyment 3 (42.9%)4 (57.1%)4.517
9Self-efficacy 1 (14.3%)3 (42.9%)3 (42.9%)4.429
10Content quality 1 (14.3%)3 (42.9%)3 (42.9%)4.429
11Perceived ease of use 7 (100%)5.000
12Interface design 2 (14.3%)5 (71.4%)4.714
13Perceived accessibility 1 (14.3%)6 (85.7%)4.857
Table
Table 4. Summary of parameters of iteration 3.
Table 4. Summary of parameters of iteration 3.
ParametersMeanRanking
Perceived ease of use5.0001
Perceived accessibility 4.8572
Flexibility, interface design 4.7143
Enjoyment 4.5174
Perceived system’s usefulness, content quality, self-efficacy4.4295
Website quality 4.1426
User preferences3.7147
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Shibani, A.S.M.; Mohd, M.; Ghani, A.T.A.; Zakaria, M.S.; Al-Ghuribi, S.M. Identification of Critical Parameters Affecting an E-Learning Recommendation Model Using Delphi Method Based on Expert Validation. Information 2023, 14, 207. https://doi.org/10.3390/info14040207

AMA Style

Shibani ASM, Mohd M, Ghani ATA, Zakaria MS, Al-Ghuribi SM. Identification of Critical Parameters Affecting an E-Learning Recommendation Model Using Delphi Method Based on Expert Validation. Information. 2023; 14(4):207. https://doi.org/10.3390/info14040207

Chicago/Turabian Style

Shibani, Abubaker Salem Mohamed, Masnizah Mohd, Ahmad Tarmizi Abdul Ghani, Mohamad Shanudin Zakaria, and Sumaia Mohammed Al-Ghuribi. 2023. "Identification of Critical Parameters Affecting an E-Learning Recommendation Model Using Delphi Method Based on Expert Validation" Information 14, no. 4: 207. https://doi.org/10.3390/info14040207

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop