The Factors Affecting Acceptance of E-Learning: A Machine Learning Algorithm Approach

Abstract

The Covid-19 epidemic is affecting all areas of life, including the training activities of universities around the world. Therefore, the online learning method is an effective method in the present time and is used by many universities. However, not all training institutions have sufficient conditions, resources, and experience to carry out online learning, especially in under-resourced developing countries. Therefore, the construction of traditional courses (face to face), e-learning, or blended learning in limited conditions that still meet the needs of students is a problem faced by many universities today. To solve this problem, we propose a method of evaluating the influence of these factors on the e-learning system. From there, it is a matter of clarifying the importance and prioritizing construction investment for each factor based on the K-means clustering algorithm, using the data of students who have been participating in the system. At the same time, we propose a model to support students to choose one of the learning methods, such as traditional, e-learning or blended learning, which is suitable for their skills and abilities. The data classification method with the algorithms multilayer perceptron (MP), random forest (RF), K-nearest neighbor (KNN), support vector machine (SVM) and naïve bayes (NB) is applied to find the model fit. The experiment was conducted on 679 data samples collected from 303 students studying at the Academy of Journalism and Communication (AJC), Vietnam. With our proposed method, the results are obtained from experimentation for the different effects of infrastructure, teachers, and courses, also as features of these factors. At the same time, the accuracy of the prediction results which help students to choose an appropriate learning method is up to 81.52%.

1. Introduction

The Covid-19 pandemic is spreading rapidly around the world and shows no signs of stopping. With different infection modes, there were nearly 20 million infections and more than 0.73 million deaths at the time of writing [1]. Moreover, the epidemic has affected all fields of economics and politics, and the social life of all people around the world [2]. The social separation and isolation required to avoid the spread of the epidemic has formed new habits and activities of life through the internet. Education is one of the fields greatly influenced as it is necessary for many people and in each different country. Students are limited in their ability to go to school and to participate in collective activities, so learning, research, and exchange activities are mainly done according to the e-learning or blended learning methods [3,4]. This approach has provided universities with an effective method of training in combining e-learning with traditional training [4], depending on the content and resources that the universities can provide for courses of e-learning at different levels. Favorable features for educational activities include online interactive activities, and data storage and assessment, which can be conducted on an internet platform so that students and faculty members can complete their tasks without being face to face [3,5].

However, e-learning has limitations: teachers and students must have certain skills, knowledge and experience in technology and pedagogical skills to complete their course. The infrastructure of the e-learning system needs to be synchronous, efficient, and secure. This is required to support teacher and student interaction, to store data, and evaluate the effectiveness of the course [6]. Therefore, the problem of how to have a good e-learning system is established. One of the interesting approaches is evaluating systems based on the influencing factors [7,8].

Katerina Kabassi et al. [9] evaluated the learning management system for blended learning in Greek using six influencing factors: platform, student, teacher, design, courses, environment and technology. Then, they used statistical methods to evaluate and analyze their system. Cheol-Rim Choi et al. [10] assessed the quality of the multimedia content of the e-learning system using four groups of factors: system quality, information quality, service quality, and the ability to evaluate the system based on calculating the weight of attributes using the analytic network process (ANP) technique. However, the weight initialized for attributes needs to be calculated carefully because it affects the evaluation results. Said A. Salloum et al. [5] used four groups of factors affecting the efficiency of e-learning systems: technology innovativeness, knowledge sharing, quality and trust. The data samples were collected from 251 students and statistical methods were used to evaluate the system.

Therefore, the method of sampling data from students in e-learning using affecting factors is scientifically grounded. In this research, we propose a method of evaluating e-learning systems based on data gathered from feedback of students using an explicit factor model (EFM) with clustering algorithm. We suggest separating factors into four groups such as: student, teacher, infrastructure and course, including sixteen features. The data clustering method used was the K-means machine learning algorithm. It is used to divide data into clusters. Thereby, it could help managers to evaluate and compare factors affecting the system to make appropriate decisions and build a better system.

The e-learning and blended learning systems are used by universities, and they are providing students with both online and traditional learning methods. Courses may be in online, traditional, or blended form depending on each institution. Normally, students often need help to get information on what form is suitable for their skills and abilities. This is solved by assistants or professors. However, this requires a lot of resources and sometimes does not really work out well. One interesting approach is to use data mining methods [11,12] which, from the available data, use an analytic process to give information about a problem in the future.

With the development of machine learning techniques and artificial intelligence, we propose a method to help students choose an appropriate learning method based on classification algorithms. It is called the studying method selection (SMS) model. According to this method, training data has been labelled and collected from previous students. Using the analytical SMS model, learners will receive results that they could use to select the best form of study. The most important aspect of this approach is how to show a set of features for systems. It is also a concerning problem for researchers, as in the following cases:

Alan Y.K. Chan et al. [8] evaluated online courses based on four main factors: online courses, learning effectiveness, evaluation methods and evaluation results. The student interaction information from the system based on the above factors is stored in the database. It is applied to system analysis and evaluation. The evaluation is based on the statistical method, aiming to recognize interactions between system factors. Said A. Salloum et al. [5] surveyed more than 280 students with a behavior intention model to predict the students’ intention to use e-learning with five related hypotheses. Sujit Kumar Basa et al. [13] have identified the main elements of e-learning to provide a framework to assess the uneven and difficult implementation of e-learning in initial and continuing educational institutions. Thereby, they are providing information to both learners and administrators. Their research is based on eight factors: institutional, technical, resource, training, competency, infrastructural, attitudinal and social integration. F. Martin et al. [8] have built a framework to evaluate the quality of an online course with seven factors: institutional support, technology infrastructure, course design, learner and instructor support, course assessment and evaluation, learning effectiveness, faculty and student satisfaction.

The training system is also built by effective learners. Therefore, feedback and assessment of students about the training system and capacity of the institution are useful. It helps to plan the policy and improve the quality of system, and following the above researches, identify influence factors based on the system purpose. They also affect the results, methodology and the type of evaluation. Because of that, we suggest four factors including sixteen features, which affect e-learning systems. Based on the features, a classification model will be made for forecasting the learning method. It helps the students choose a suitable method with their abilities and conditions.

In summary, we suggest four affected factors including sixteen features. Base on that, data samples that are collected from students who have taken the courses aim to solve two different problems: firstly, to evaluate the influence measurement of factors and features based on the clustering method by the K-mean algorithm. Secondly, to build a prediction model using classification techniques to support the students in choosing an appropriate learning method. The experiment is on the collected data samples from 303 students studying at the Academy of Journalism and Communication (AJC), Vietnam, with the prediction result being up to 81.52% accurate from the studying method selection model (SMSM). It is meaningful for AJC to enhance training qualities. Specially, to provide training in journalistic, publication, and political courses at any location of the nation.

2. Literature Review

2.1. Affected Factors

The evaluated results about e-learning systems are used so that students and management can develop the system effectively. The methods based on constituted factors of the system have applied for researches and showing their feasibility.

According to

Table 1. Evaluation Blended learning base on affecting factors.

Research	Factors/Attributes/Features	Method/Measurement/Tools
Alberto Rivas et al. [15] (2020)	Course viewed, Course module viewed, Discussion viewed, Course module instance list viewed, The status of the submission has been viewed, A submission has been submitted, Summary of the questionnaire attempt, Attempted visualized questionnaire, The attempt has begun, Attempt sent, Grade user report viewed, Course user report seen	Decision tree/Random forest Extreme gradient boosting/Multilayer perceptron
Said A.salloum et al. [14] (2019)	Content quality, information quality, and system quality, Self-efficacy, Subjective norm, enjoyment, accessibility, playfulness, ease of use, usefulness, Attitude towards use, behavioral intention	Statistics/Convergent validity, Discriminant validity measurement/SmartPLS 3.2.7 tool
Seyyed Mohsen Azizi et al. [17] (2019)	Attitude, Subject Norm, Behavioral Control, attitudinal beliefs, normative beliefs, and control beliefs, Ease of Use, Usefulness	Statistics/Convergent validity, Discriminant validity measurement/SmartPLS 3.2.7 tool
Sinan Keskin et al. [7] (2019)	Learning Environment Preferences, Self-directed learning, Learner control, Motivation towards e-learning, Rehearsal, Organization, Elaboration, Test anxiety, Task value, Self-efficacy	Non-linear correlation between the variables/OVERALS analysis tool in SPSS 21
Cheol-Rim Choi et al. [18] (2019)	System quality, Information quality, Service quality, Attractiveness,	Weights of the quality attributes/ANP
F. Martin et al. [8] (2018)	Institutional Support, Technology Infrastructure, Course Design, Learner and Instructor Support, Course Assessment and Evaluation, Learning effectiveness, Faculty and Student Satisfaction	overview
Sujit Kumar Basa et al. [13] (2017)	institutional, technical, resource, training, competency, infrastructural, attitudinal and social integration	Technology Acceptance Model/Theory of Planned Behavior
Katerina Kabassi et al. [9] (2016)	Student, Teacher, Design, Courses, Environment, Technology	Statistical
Dilrukshi Gamage et al. [16] (2014)	Technology, Pedagogy, Motivation, Usability, Content/Material, Support for Learners, Assessment, Future Directions, Collaboration, Interactivity Does	Statistical/SPSS
Alan Y.K. Chan et al. [8] (2003)	online courses, learning effectiveness, evaluation methods and evaluation results	Web Mining techniques

, the factors are usually used to assess e-learning, and blended learning systems such as: student or their activity on systems, shown in [7,8,9,14]. The teacher information and affectivity of them are shown in [8,9]. The course with content and features affectivity are shown in [8,9,15,16]. The infrastructure with component, structure, affectivity and ability are shown in [7,9,13,15,17,18]. Therefore, we also suggest a set of affecting factors aiming to evaluate the e-learning system. These features are separated into four groups such as: students, teacher, infrastructure and course. It is shown in Figure 1 below:

With the features illustrated in

Table 2. The Features Affect E-learning System.

Category	Feature	Illustration
Student	SF1	Year of birth
	SF2	Gender
	SF3	Place of residence
	SF4	Year of grade
	SF5	Field of studying
	SF6	Average of outcome
Teacher	TF7	Influence of teacher
Infrastructure	IF8	Preparing lecture and material
	IF9	Hardware, internet quality
	IF10	Influence of learning management system (LMS) and storage service
	IF11	Influence of regulation environment
Courses	CF12	Suitableness of course in e-learning
	CF13	Effect of e-learning method
	CF14	Level of course
	CF15	Influence of assessment
	CF16	Learning methods

, we implemented the EFM model which is illustrated by Figure 2 to evaluate affecting measurement. By the measurement results, the manager improved the e-learning system. The features are also applied for SMSM aim to provide information for students by the model in Figure 3.

2.2. Artificial Intelligence Methodologies in Education

Nowadays, the artificial intelligence methodologies applied in education are more and more popular [4,19,20,21,22]. Ammar Almasri et al. [21] have used data mining techniques in education (EDM) to build an understandable model. It helps decision makers enhance the performance of the students. The novel of their approach is combination of clustering and classifying from data over 13 academic years of 1062 graduates. The classification algorithms MLP, NB, J48, ETM are used for the experiment. The accuracy, precision, recall and f1—score measurement are used to evaluate the model with the highest classification results up to 96.96%. Irene Pasina et al. [22] have used the hierarchal clustering algorithms to group students based on their learning style. Their experiment in engineering education is limited in Saudi Arabia.

It is shown that the data mining approach is more and more applied to education research. The popular algorithm could be found in research such as the K-mean clustering algorithm [23]; the classify algorithms are applied to build an understandable model, such as decision tree (DT), RF, extreme gradient boosting (EGB) in [15,21,24,25].

3. Evaluated E-Learning System Using Clustering Method

With the factors that affect that the E-learning system, we propose a method to calculate the influence measurement of factors based on clustering method such as Figure 2 below:

In Figure 2, we use the model to exploit an affectation measurement of factors with the e-learning system. Thereby, we have a basis for investing in and upgrading the system. The raw data obtained from the students’ assessment of the system is transformed to the dataset (D). It consists of N rows such as vector type $V\left(f_1,f_2,\dots ,f_N\right)$ with N as a number of features. The dataset D will be applied to find a number K, which is a number that the dataset D could divide into K clusters using D, K, and K-mean algorithm. We also obtained K clusters such as $Cluster\left(C_1,C_2,\dots ,C_K\right)$ with $C_i=V_i\left(f_{1i},f_{2i},\dots ,f_{Ni}\right)$ and N as number of features. Then, we got the per factor vector from their features by Equation (1). It shows the influence measurement of the factors and helps management to make a strategy decision to invest the factor or features. Our algorithm suggestion is illustrated by Algorithm 1 below:

Algorithm 1 calculated vector of influence factors

Initialization: D, a data set of N features.

1. getting K, number of clusters [26]

2. cluster D to K Cluster $\left(D_1,D_2,\dots ,D_k\right)$ by K Mean

3. calculated a Vector $Facto{r}_p({\mathrm{f}}_{p1},{\mathrm{f}}_{p2},\dots ,{\mathrm{f}}_{pK})$ by Equation (1)

Return $Facto{r}_p({\mathrm{f}}_{p1},{\mathrm{f}}_{p2},\dots ,{\mathrm{f}}_{pK})$

Equation (1) calculated factors value.

$$\begin{gathered} Facto{r}_p({\mathrm{f}}_{p1},{\mathrm{f}}_{p2},\dots ,{\mathrm{f}}_{pK})=(\frac{{\displaystyle \sum _{j=1}^M{f}_{j1}^p}}{M},\frac{{\displaystyle \sum _{j=1}^M{f}_{j2}^p}}{M},\dots ,\frac{{\displaystyle \sum _{j=1}^M{f}_{jK}^p}}{M}) \\ with{f}_{j1}^p\in C_{1}andj=MisnumberfeatureofFactorp \end{gathered}$$

(1)

4. Studying Method Selection Model (SMSM)

4.1. Classifying Model

The SMSM is divided into two phases. Firstly, the training phase, in which the data training with labels is used for the training model and obtaining the classifier (CF). The data labels are collected from the students who have finished their course and evaluated the system based on feature suggestions. After data transformation, we obtained a data set (DT) with labels which are the learning method for training. The technical 10-fold cross-validation (10CV) is applied to an algorithm such as RF, NB, SVM, MP and KNN in order to find a better algorithm. By the algorithm selected, we built a classifier to predict the learning method when a student is required by their information input.

Secondly, the monitoring phase, after student’s data is transformed, the mode uses the classifier (CF) to predict a label. The obtainable prediction result is learning method label such as e-learning, traditional, or blended learning. It helps student to choose an appropriate studying method for a course via SMSM model. This result also helps institutions better support students as well as diversify their forms and improve their training quality.

4.2. Algorithms and Evaluated Measurement

4.2.1. Algorithms

In the first phase, some commonly used classification algorithms are: DT, RF, EGB and MP in [15,21,24,25]. Therefore, we suggest five algorithms to experiment on such as RF, KNN, SVM and NB algorithm, with the aim of choosing the appropriate algorithm for the second phase.

4.2.2. Evaluated Measurement

With the classification model, some measurements are used in evaluating data classification result, by factors such as: accuracy, precision, recall, f1-score, ROC in [15,21] and our previous research in [24,25]. Thereby, we also suggest accuracy, precision, recall, f1-score, and ROC measurement to evaluate classifying model results.

5. Experiment and Results

5.1. Data Collection

We used the survey samples method to collect data from students who have attended an online and offline training system at AJC. The illustration of student information based on features with their values is in

Table 3. Collected Data by Features of Factors.

Category	Feature	Clustering	Classifying
Student	SF1	Integer	Integer
	SF2	Male; Female	1, 0
	SF3	City, Province, other	3, 2, 1
	SF4	Grade: 1, 2, 3,4	1, 2, 3, 4
	SF5	Journalistic and publication, political, other	3, 2, 1
	SF6	A, B, C, D, E	4, 3, 2, 1
Teacher	TF7	Five likert scale	1, …, 5
Infrastructure	IF8	Five likert scale	1, …, 5
	IF9	Five likert scale	1, …, 5
	IF10	Five likert scale	1, …, 5
	IF11	Five likert scale	1, …, 5
Courses	CF12	Five likert scale	1, …, 5
	CF13	Five likert scale	1, …, 5
	CF14	Knowledge, basic, field	1, 2, 3
	CF15	Five likert scale	1, …, 5
	CF16-label	e-learning, traditional, blended	E, T, B

below:

We collected data samples from 303 students. They are from first year to fourth year studying at AJC. The number of collected data samples is 679 based on three different levels training: knowledge, basic and field. The AJC provided forms of learning for student such as e-learning, traditional and blended learning. This will be the data label used for classification model.

5.2. Environment

We used Google form to collect data samples from AJC students. The Waikato environment for knowledge analysis (WEKA) tool version 3.7.8 was used for clustering and classifying [27]. The default parameters of the machine learning algorithms on this WEKA version were applied during the experimentation. The number cluster K = 5 is found and applied for EFM.

5.3. Experiment on EFM Result

After collecting and transforming data, we used the method of selecting a number of clusters [26] to get the appropriate number of clusters K = 5. The obtained experiment on EFM results is illustrated in

Table 4. The Experiment on EFM Results.

SF	General	C1 17%	C2 18%	C3 13%	C4 36%	C5 16%
SF1	19.7387	20	19.1282	20.85	20.1207	19.087
SF2	Female	Female	Female	Female	Female	Female
SF3	City	Other	City	City	City	City
SF4	Grade2	Grade2	Grade1	Grade3	Grade2	Grade1
SF5	Other	Political	Other	Political	Journal	Other
SF6	B	C	C	B	B	B
TF7	3.9899	3.8056	4.1026	3.5	4.1207	4.087
IF8	4.2663	3.9167	4.3333	4.2	4.3448	4.413
IF9	4.608	4.4444	4.7949	4.5	4.6034	4.6304
IF10	4.2513	4.0833	4.4872	4.2	4.2069	4.2609
IF11	3.7273	3.3889	4.0256	3.45	3.7539	3.8261
CF12	4.1608	3.8056	4.4615	4.15	4.0517	4.3261
CF13	3.598	3.4444	3.6667	3.5	3.8103	3.4348
CF14	Basic	Basic	Knowledge	Field	Basic	Knowledge
CF15	3.9045	3.5833	4.1026	3.8	3.9483	3.9783
Method	Blended	Blended	Blended	Blended	E-Learning	Blended

with five clusters such as: C1, C2, C3, C4 and C5 below:

From the clustering results in Table 4, we could see that the blended learning method was chosen by the majority of students in all different levels of training. This is according to the effectiveness of blended learning. The traditional combination with online methods in blended learning for the whole course or each part of the course helps learners to complete their course more effectively. The results in Table 4 are also shown that journalistic and publication students who have better capacities will choose e-learning to implement their courses. From factors affecting the system, the manager also sees the influence level of three factors by the same measure: teacher, infrastructure, and course, which are calculated by Equation (1) and shown in Figure 4 below:

In Figure 4 it is shown that the infrastructure factor affects the e-learning system the most. This is also consistent with the practice; because if the infrastructure is not good it will adversely affect the teaching process, learning psychology, and the effectiveness of course evaluation. In addition, the manager can see each feature influencing in different factors. For the infrastructure factor, the specific level of influence for each feature is shown in Figure 5 below:

Figure 5 above also shows that the quality of the internet connection, hardware devices, and student learning equipment (IF9) are the most affected the e-learning system. The next is the LMS system and information storage services (IF10). The continuum feature is preparing lectures and material (IF8), then finally the policy environment set by the universities. This result provided information for managers to make policies and strategies to build a better system infrastructure or more features.

To perform an experiment with the same method on features of the course factor, we obtained the results in Figure 6 below:

As shown in Figure 6, the CF12 property has a higher value than the rest. It has proved that the feature of the suitableness of the content and nature of the course to the system are evaluated as higher. It is also shown that, depending on the nature and content requirements of the subject, it is possible to build the course according to the appropriate method at each training level (knowledge, basic, field).

5.4. Experiment on SMSM Result

In this session, we experimented on classification with five algorithms such as: NB, KNN, SVM, MP and RF. The method 10CV was also applied in the experimental process. The model results are evaluated by accuracy, precision, recall, f1—score and ROC measurement. It is expressed in

Table 5. Experiment on SMSM Results.

Algorithm	Accuracy	Precision	Recall	F1	ROC
NB	53.25%	0.491	0.532	0.497	0.607
KNN	77.45%	0.772	0.774	0.773	0.796
SVM	58.64%	0.759	0.586	0.44	0.508
MP	63.92%	0.63	0.639	0.629	0.719
RF	81.52%	0.813	0.815	0.812	0.911

below:

As the results in Table 5 show, the results of the RF algorithm are the highest, such as: 81.52% in accuracy; 0.812 in f1-score and 0.91 in ROC measurement. With accuracy of the model by algorithm expressed in Figure 7, we used RF for predicted learning method label with monitoring data input on secondly phases. The classification performance results are different when we use different algorithms, on different datasets, or depending on the number of different features of the same dataset. For example, in the research in [15], the highest accuracy with RF algorithm is 75.5%, which was performed on the dataset that the research built.

5.5. Discussion

Each learning service system is oriented towards learners. Therefore, information received from learners is more important in building and managing the system. Based on each different purpose, the studies used appropriate methods based on the dataset collected. The data clustering method in this study is based on the proposed set of features used to render explicit and evaluate the effects of the system components. The experimentation results show that the infrastructure factor had the greatest impact on the system. In addition, the feature clearly shows influence based on its value. The order of precedence for features is at different levels, among which, hardware and internet quality is considered the most important property. Next is LMS and storage service and preparing lecture and material properties features. The last one is the influence of regulation environment property. The next important factor is the teacher; this is an indispensable factor because the reputation, orientation, skills and knowledge of the teachers help students get the best results in the learning and research process. Therefore, the experimentation results obtained show that the teacher is a greater influence than the course factor. This information is really useful, especially for universities in the transition phase of training model.

In the classification problem, the characteristics of the data always affect the classification results. Depending on the purpose of the problems, they propose different factors that affect different systems. Therefore, the number and value of the attributes will be one of the factors determining the model results. The question of how to best complete a course is always posed to each student. It depends on many influencing factors, but this information is not always fully provided by the student. Therefore, the SMSM model predicts the suitable method for a subject that students are preparing to choose to achieve the accuracy prediction result of 81.52% providing practical meaning. This gives students the confidence in their choice based on the data of previous learners. Moreover, the model also helps training institutions reduce the burden of consulting and answering students’ questions about related learning issues.

Using the RF classification algorithms, but on different datasets, the research [15] aims to understand the factors that influence the university learning process. The number of times students have accessed the resources made available on VLE platforms has been identified as a key factor affecting student performance.

The prediction accuracy results by our model are higher than that of research [15]. The comparison result is shown in

Table 6. The Comparison Results.

Model/System	Feature	Algorithm	Accuracy
System in [15]	5 type of events/39 features	DT	70.5%
		RF	78.1%
		EGB	76.5%
		MP	78.2%
SMSM	4 type factor/16 features	RF	81.52%

. The predicted results of the SMSM model gave 3.32% higher accuracy than the results proposed by research [15].

6. Conclusions and Future Work

Building a strategy, operating, and managing online learning systems are some of the most important problems of each university. Choosing and properly assessing the influence of factors will help managers to have appropriate strategies. In our research, we have proposed four influencing factors including sixteen features, using data clustering algorithms to analyze elements and features to help managers enhance training qualities and systems effectively. At the same time, we used data classification techniques to build a model to support students in choosing the learning methods, such as e-learning, traditional, or blended leaning, that are suitable for their course content and learning capacity from the datasets of previous learners with accuracy of result of up to 81.52%.

The research is still limited because the experimental data has only been from AJC. The classification algorithms have used default parameters. Therefore, in the next studies, we will add more features, use the evaluation methods of feature selection, use advanced techniques such as deep learning and perform the steps of optimization of parameters and algorithms for higher results.

Beside the limitations, the research has proposed a set of features to enhance effective e-learning in order to build and improve the quality of the education system by current popular techniques. At the same time, we proposed research on facilities to improve the performance of training institutions based on successful application of artificial intelligence.