E-Learning Platform Assessment and Selection Using Two-Stage Multi-Criteria Decision-Making Approach with Grey Theory: A Case Study in Vietnam

Abstract

Education has changed dramatically due to the severe global pandemic COVID-19, with the phenomenal growth of e-learning, whereby teaching is undertaken remotely and on digital platforms. E-learning is revolutionizing education systems, as it remains the only option during the ongoing crisis and has tremendous potential to fulfill instructional plans and safeguard students’ learning rights. The selection of e-learning platforms is a multi-criteria decision-making (MCDM) problem. Expert analyses over numerous criteria and alternatives are usually linguistic terms, which can be represented through grey numbers. This article proposes an integrated approach of grey analytic hierarchy process (G-AHP) and grey technique for order preference by similarity to ideal solution (G-TOPSIS) to evaluate the best e-learning website for network teaching. This introduced approach handles the linguistic evaluation of experts based on grey systems theory, estimates the relative importance of evaluation criteria with the G-AHP method, and acquires e-learning websites’ ranking utilizing G-TOPSIS. The applicability and superiority of the presented method are illustrated through a practical e-learning website selection case in Vietnam. From G-AHP analysis, educational level, price, right and understandable content, complete content, and up-to-date were found as the most impactful criteria. From G-TOPSIS, Edumall is the best platform. Comparisons are conducted with other MCDM methods; the priority orders of the best websites are similar, indicating the robust proposed methodology. The proposed integrated model in this study supports the stakeholders in selecting the most effective e-learning environments and could be a reference for further development of e-learning teaching-learning systems.

1. Introduction

The rapid evolution of Information and Communication Technologies (ICT), specifically, the Internet, has caused a massive boom in online training. E-learning is increasingly popular because of its flexibility and convenience in terms of time and place. E-learning platforms were born as a revolution in teaching and learning, becoming an inevitable trend of the times. The world of online learning is not expected to slow down anytime soon, as more and more people have the means to pursue an education through the Internet and because of one eminent cause: the COVID-19 pandemic. Under the influence of the COVID-19 pandemic, network teaching has been widely established to protect students’ learning rights and ensure the implementation of teaching plans [1,2]. In Vietnam, as the country moves toward the 4.0 revolution, the use of modern technology in education is becoming highly significant. A plethora of e-learning platforms is strongly developed. The first COVID-19 epidemic in Vietnam occurred in late January 2020, and since then, the Vietnamese government has taken a firm stance and closed all educational institutions. According to the Vietnam Economic Times, prior to COVID-19, students in Vietnam used e-learning platforms to take English or soft skill courses, and the e-learning paradigm had been associated with those courses for over a decade [3]. The pandemic has provided an excellent opportunity for e-learning platforms to rethink their ICT infrastructure to grow more sustainable.

Network teaching differs from traditional education due to the peculiarities of the Internet [4]. Even though they may come from across the nation, teachers and students in network teaching have a dynamic engagement. Furthermore, both in terms of quantity and sharing, network teaching materials provide substantial assistance for students. Time savings, cost savings, improved interaction, and higher flexibility are all advantages of network teaching over traditional education. Because of these benefits, network teaching has become increasingly popular, resulting in a surge in the number of e-learning websites [5]. Many factors, such as technological characteristics, user-friendly online platforms, class activities, and assessments, may influence the success of online learning [6,7,8]. The design and delivery of online courses significantly impact students’ satisfaction, studying, and retention. Students and website developers have paid close attention to the quality of e-learning websites. As a result, the quality assessment of e-learning websites from users’ standpoint should be investigated further. Selecting an appropriate e-learning website will directly impact user performance and promote network teaching quality. Toward this end, relevant and practical approaches for selecting e-learning websites are required.

Since various evaluation criteria must be recognized when selecting a specific e-learning website with the best performance for online education, the problem of selecting and ranking e-learning websites can be modeled as a multi-criteria decision-making (MCDM) problem, necessitating the development and adoption of a methodology for website selection. As the most efficient and pragmatic MCDM methods, analytical hierarchy process (AHP) and technique for order preference by similarity to an ideal solution (TOPSIS) are among the most common techniques used by plenty of scholars. They are well-known for their flexibility and ability to decompose a decision problem that reflects subjective judgments for both quantitative and qualitative criteria when evaluating the alternatives. AHP has been used in conjunction with other decision science and operations management models because of its capacity to transform qualitative data. Indeed, combining AHP with TOPSIS has proven to be highly beneficial in making more successful selections [9,10,11,12,13,14]. Exact numbers are used to represent a judgment or a score in classical AHP and TOPSIS approaches that have been proved to cause imprecise information and reduce judgments’ accuracy in many real-world cases. Thus, the use of fuzzy logic with MCDM techniques is gaining popularity as a way to handle the ambiguity inherent in expert judgments [15,16,17,18]. Another approach that helps in addressing ambiguity in inputs is the grey systems theory. While the main downside of fuzzy sets is that modeling the discrete character of the data can be challenging, especially if there are insufficient data, grey systems theory is frequently a preferable choice in situations where there is a significant level of uncertainty, and identifying membership functions is challenging [19].

This paper proposes a hybrid MCDM model that combines grey analytic hierarchy process (G-AHP) and grey technique for order preference by similarity to ideal solution (G-TOPSIS) for e-learning website evaluation and selection. More specifically, G-AHP is used to determine the relative importance of criteria and sub-criteria, and then, G-TOPSIS is utilized to prioritize the alternatives. A comprehensive set of evaluation criteria on e-learning websites were recognized through a literature review and experts to apply the integrated approach. A real case study for Vietnam is conducted to validate the proposed method. This work also aims to provide e-learning service providers, system developers, and online research researchers with significant insights into network teaching.

The structure of this paper is ordered as follows. Section 2 summarizes MCDM methodologies applied to e-learning websites selection and critical evaluation criteria used in the studies. In Section 3, materials and methods are expounded upon in detail. In Section 4, a realistic e-learning website selection case in Vietnam is demonstrated. Section 5 presents a comparative analysis of methods to validate the results. Section 6 contains concluding remarks.

2. Literature Review

There have been increasingly significant studies on e-learning website evaluation and selection over the years, and a plethora of MCDM methods toward this problem have been put forward in the literature.

To assess educational websites, Hwang et al. [20] suggested an integrated group-decision technique that includes AHP, Decision Support System (DSS), fuzzy theory, and grey system theory. Their research made a significant contribution by assisting students and instructors in searching for appropriate educational and instructive Internet resources. They proposed that using an integrated strategy for evaluating educational websites is extremely suited and trustworthy. Büyüközkan et al. [21] developed an evaluation framework based on fuzzy VIKOR (VIseKriterijumska Optimizacija I KOmpromisno Resenje) for performance assessment of e-learning websites, aiming to support e-learning service providers and system developers. Tzeng et al. [22] suggested a hybrid MCDM model in which DEMATEL (decision-making trial and evaluation laboratory) was utilized to determine e-learning program requirements’ dependent and independent relationships. Then, in line with the subjective assessment of the environment, the AHP method and fuzzy set theory methodology were utilized. Their empirical testing findings showed that the suggested approach could successfully evaluate e-learning programs, even when many evaluation criteria are entangled. For assessing the quality of e-learning websites, an axiomatic design-based technique for fuzzy group decision making was proposed in [23], then fuzzy TOPSIS was used to validate the outcomes. It was offered in a case study concentrating on Turkish e-learning websites. Lin [16] adopted an evolution model that blended triangular fuzzy numbers with AHP to create a unique FAHP assessment methodology. To prioritize the relative weights of course website quality characteristics, the FAHP technique was developed. They contended that the FAHP assessment approach is capable of providing a realistic reference to system designers looking to improve the efficacy of course websites.

Tseng et al. [24] used the analytic network process (ANP) in conjunction with fuzzy set theory to assess the efficacy of teaching and training in an e-learning system. The goal of their research was to assist managers and decision makers. The findings suggested that the FANP technique is a straightforward, appropriate, and successful way for assessing the efficacy of e-learning systems, particularly when the context of interdependent measurements is included. They contended that the suggested FANP framework might be used in a variety of scenarios to assess the efficiency of e-learning systems. Bhuasiri et al. [25] found several characteristics that contribute to the success of e-learning systems in developing nations. They then used AHP with the Delphi approach to analyze the relative relevance of elements across two stakeholder groups, which included ICT specialists and faculty members. Finally, essential success variables were identified and prioritized based on their significance. Jain et al. [26] adopted the weighted distance-based approximation (WDBA) method for identifying the best e-learning platforms that provides several important advantages over the current ones. The authors utilized the TOPSIS method to verify the results. Garg et al. [27] suggested a fuzzy complex proportional assessment (COPRAS) approach for assessing and choosing e-learning websites for programming languages. A matrix method was deployed in [28] and a computational and quantitative model based on weighted Euclidean distance approximation and complex ratio evaluation in [29] for a similar problem of solving e-learning website evaluation.

The study of Khan et al. [30] described the use of a newly created multi-criteria decision-making (MCDM) technique, namely the Proximity Indexed Value (PIV) method, for ranking and selecting e-learning websites. Compared to other MCDM approaches, PIV is a computationally simpler solution that minimizes the rank reversal problem. The PIV method’ s applicability and usefulness have been proven using two illustrative scenarios relevant to selecting e-learning websites. Muhammad et al. [31] focused on identifying and prioritizing factors related to the design quality of e-learning systems through a hierarchical quality model with AHP. Jaukovic Jocic et al. [32] proposed an integrated approach on the basis of MCDM methods and symmetry principles for e-learning website selection. The pivot pairwise relative criteria importance assessment (PIPRECIA) method was utilized to determine the significance levels of criteria, and the interval-valued triangular fuzzy additive ratio assessment (ARAS) method was employed to rank the alternatives, i.e., e-learning courses. In the most recent study of the field, Gong et al. [5] proposed a new integrated MCDM approach based on linguistic hesitant fuzzy sets (LHFSs) and the TODIM (an acronym in Portuguese of interactive and multi-criteria decision making) method for choosing the best e-learning platform.

In this paper, a hybrid model that combines G-AHP and G-TOPSIS is proposed for the first time to evaluate and select e-learning websites. To the best of our knowledge, the proposed integrated approach has not been reported elsewhere. Furthermore, there have been limited studies on e-learning website evaluation in the Vietnamese context [33,34,35]. Thus, the paper’ s contributions are presented as follows:

• This paper presents an effective evaluation model for e-learning websites, which contains a comprehensive set of indicators through literature review and experts’ opinions.
• The grey systems theory is employed to express experts’ uncertain and complicated evaluation on e-learning websites.
• A practical case study of evaluating e-learning websites in Vietnam is solved for the first time. A comparative analysis of other MCDM methods is conducted to test the robustness of the proposed model. The evaluation based on distance from average solution (EDAS) and the complex proportional assessment (COPRAS) methods integrated with the grey systems theory are employed for this comparative analysis. The EDAS method was propounded by Ghorabaee et al. [36] has superior advantages compared to other methods for classification, which lies in its accurate efficiency and fewer mathematical calculations, while the COPRAS method is an appropriate method to process the information in a reasonable and efficient way [37].
• The paper is directed toward providing a helpful guideline to e-learning service providers, system developers, and researchers related to web research.

3. Materials and Methods

In this study, grey theory, AHP, and TOPSIS methods are integrated and utilized to determine the most important criteria and the best e-learning sites in the Vietnamese context regarding their sustainability with considering uncertain judgments.

3.1. Grey Theory

In 1989, Julong-Deng [38] introduced the grey system theory in order to handle uncertain conditions in mathematical modeling. The degree of the information in the grey theory can be divided into three categories, including “white system”, “black system”, and “grey system”, if the information is “fully known”, “unknown, and “partially known”, respectively. The grey system theory concept is depicted in Figure 1.

A grey number is denoted as $⨂x=\left[\underset{\_}{x},\overline{x}\right],$ where $\underset{\_}{x}$ is the lower bound, and $\overline{x}$ is the upper bound of the membership function. The exact value of the grey number is unknown, but the interval range within which the value lies is known.

Let $⨂x_1=\left[{\underset{\_}{x}}_1,{\overline{x}}_1\right]$ and $⨂x_2=\left[{\underset{\_}{x}}_2,{\overline{x}}_2\right]$ represent two grey numbers; $k$ represents a positive real number, and $L$ represents the grey number length. The basic arithmetic operations are shown in Equations (1)–(6).

$$⨂x_1+⨂x_2=\left[{\underset{\_}{x}}_1+{\underset{\_}{x}}_2, {\overline{x}}_1+{\overline{x}}_2\right]$$

(1)

$$⨂x_1-⨂x_2=\left[{\underset{\_}{x}}_1-{\overline{x}}_2, {\overline{x}}_1-{\underset{\_}{x}}_2\right]$$

(2)

$$⨂x_1\ast ⨂x_2=\left[\min \left({\underset{\_}{x}}_1{\underset{\_}{x}}_2,{\underset{\_}{x}}_1{\overline{x}}_2,{\overline{x}}_1{\underset{\_}{x}}_2,{\overline{x}}_1{\overline{x}}_2\right),\max \left({\underset{\_}{x}}_1{\underset{\_}{x}}_2,{\underset{\_}{x}}_1{\overline{x}}_2,{\overline{x}}_1{\underset{\_}{x}}_2,{\overline{x}}_1{\overline{x}}_2\right)\right]$$

(3)

$$⨂x_1/⨂x_2=\left[\min \left({\underset{\_}{x}}_1/{\underset{\_}{x}}_2,{\underset{\_}{x}}_1/{\overline{x}}_2,{\overline{x}}_1/{\underset{\_}{x}}_2,{\overline{x}}_1/{\overline{x}}_2\right),\max \left({\underset{\_}{x}}_1/{\underset{\_}{x}}_2,{\underset{\_}{x}}_1/{\overline{x}}_2,{\overline{x}}_1/{\underset{\_}{x}}_2,{\overline{x}}_1/{\overline{x}}_2\right)\right]$$

(4)

$$k⨂x_1=k\left[{\underset{\_}{x}}_1,{\overline{x}}_1\right]=\left[k{\underset{\_}{x}}_1,k{\overline{x}}_1\right]$$

(5)

$$L\left(⨂x_1\right)=\left[{\overline{x}}_1-{\underset{\_}{x}}_1\right]$$

(6)

The degree of grey possibility between two grey numbers $⨂x_1$ and $⨂x_2$ is shown in Equation (7).

$$P\left\{⨂x_1\le ⨂x_2\right\}=\frac{\max \left(0,L^{*}-\max \left(0,{\overline{x}}_1-{\underset{\_}{x}}_2\right)\right)}{L^{*}}$$

(7)

where $L^{*}=L\left(⨂x_1\right)+L\left(⨂x_2\right)$.

3.2. Grey Analytical Hierarchy Process (G-AHP)

Grey analytical hierarchy process (G-AHP) integrates grey theory and AHP to reduce the subjective judgments in the decision-making process. The linguistic scale and grey number used in the G-AHP model is presented in

Table 1. The linguistic scale with grey number of the G-AHP model.

Level	Linguistics Scale	Grey Number
1	Equivalent Importance (EI)	[1,2]
3	Medium Importance (MI)	[2,4]
5	Strong Importance (SI)	[4,6]
7	Very Strong Importance (VSI)	[6,8]
9	Extreme Importance (EMI)	[8,10]

. The steps of G-AHP are presented as follows [39].

Step 1: Build the decision tree and the pair-wise comparisons matrix based on experts’ judgments. Then, develop the integrated grey comparison matrix using the geometrical aggregation, as can be seen in Equation (8).

$$D=\left[\begin{array}{ccc}⨂x_{11}& \cdots & ⨂x_{1n}\\ ⋮& ⋮& ⋮\\ ⨂x_{m1}& \cdots & ⨂x_{mn}\end{array}\right]=\left[\begin{array}{ccc}\left[{\underset{\_}{x}}_{11},{\overline{x}}_{11}\right]& \cdots & \left[{\underset{\_}{x}}_{1n},{\overline{x}}_{1n}\right]\\ ⋮& ⋮& ⋮\\ \left[{\underset{\_}{x}}_{m1},{\overline{x}}_{m1}\right]& \cdots & \left[{\underset{\_}{x}}_{mn},{\overline{x}}_{mn}\right]\end{array}\right]$$

(8)

Step 2: Compute the normalized grey comparison matrix using Equations (9)–(11).

$$D^{*}=\left[\begin{array}{ccc}⨂x_{11}^{*}& \cdots & ⨂x_{1n}^{*}\\ ⋮& ⋮& ⋮\\ ⨂x_{m1}^{*}& \cdots & ⨂x_{mn}^{*}\end{array}\right]=\left[\begin{array}{ccc}\left[{\underset{\_}{x}}_{11}^{*},{\overline{x}}_{11}^{*}\right]& \cdots & \left[{\underset{\_}{x}}_{1n}^{*},{\overline{x}}_{1n}^{*}\right]\\ ⋮& ⋮& ⋮\\ \left[{\underset{\_}{x}}_{m1}^{*},{\overline{x}}_{m1}^{*}\right]& \cdots & \left[{\underset{\_}{x}}_{mn}^{*},{\overline{x}}_{mn}^{*}\right]\end{array}\right]$$

(9)

$${\underset{\_}{x}}_{ij}^{*}=\frac{2{\underset{\_}{x}}_{ij}}{{{\displaystyle \sum }}_{i=1}^m{\underset{\_}{x}}_{ij}+{{\displaystyle \sum }}_{i=1}^m{\overline{x}}_{ij}}$$

(10)

$${\overline{x}}_{ij}^{*}=\frac{2{\overline{x}}_{ij}}{{{\displaystyle \sum }}_{i=1}^m{\underset{\_}{x}}_{ij}+{{\displaystyle \sum }}_{i=1}^m{\overline{x}}_{ij}}$$

(11)

where $⨂x_{ij}$represent the pairwise comparison from a group of decision makers with respect to the $i\mathrm{th}$ criterion over the $j\mathrm{th}$ criterion.

Step 3: Compute the grey weight of each criterion using Equation (12).

$$⨂w_i=\frac{{{\displaystyle \sum }}_{j=1}^n⨂x_{ij}^{*}}{n}$$

(12)

where $n=\left\{1, 2, \dots ,N\right\}$ denotes the set of criteria.

Step 4: Conduct the whitenization of the grey weight using Equation (13). The whited value of an interval grey weight is a crisp number whose possible value lies between the upper and lower bound of the interval grey weight.

$$M_i=\left(1-\lambda \right){\underset{\_}{w}}_i+\lambda {\overline{w}}_i$$

(13)

where $\lambda$ represents the whitening coefficient and $\lambda \in \left[0, 1\right]$. In this paper, the value of λ is selected as 0.5 ($\lambda =0.5$).

3.3. Grey Technique for Order of Preference by Similarity to Ideal Solution (G-TOPSIS)

Grey technique for order preference by similarity to ideal solution (G-TOPSIS) integrates grey theory and TOPSIS to reduce the subjective judgments in the decision-making process. The steps of the G-TOPSIS model are presented below [40].

Step 1: Suppose that $S=\left\{S_1,S_2,\dots ,S_m\right\}$ is a discrete set of $m alternatives$, which are ranked by a discrete set $C=\left\{C_1,C_2,\dots ,C_n\right\}$ of $n$ criteria. In this paper, the grey weights are calculated by the G-AHP model.

Step 2: Based on the grey linguistic scale with grey numbers in

Table 2. The linguistics scale with grey numbers of the G-TOPSIS model.

Linguistics Scale	Grey Number
Very Poor (VP)	[0,1]
Poor (P)	[1,3]
Medium Poor (MP)	[3,4]
Fair (F)	[4,5]
Medium Good (MG)	[5,6]
Good (G)	[6,9]
Very Good (VG)	[9,10]

, assess the ratings of the alternatives in each of the criteria. Suppose that there are $k$ experts, and the value of alternative $h$ in the criterion $g$ is calculated in Equation (14).

$$⨂G_{hg}=\frac{1}{k}\left(⨂G_{hg}^1+⨂G_{hg}^2+\dots +⨂G_{hg}^k\right)$$

(14)

Step 3: Transfer the linguistic scale into grey numbers. Then, build the grey decision matrix using Equation (15).

$$E=\left[\begin{array}{cc}\begin{array}{ccc}⨂G_{11}& ⨂G_{12}& \cdots \\ ⨂G_{21}& ⨂G_{22}& \cdots \end{array}& \begin{array}{c}⨂G_{1n}\\ ⨂G_{2n}\end{array}\\ \begin{array}{ccc}⋮& ⋮& ⋮\\ ⨂G_{m1}& ⨂G_{m2}& \cdots \end{array}& \begin{array}{c}⋮\\ ⨂G_{mn}\end{array}\end{array}\right]$$

(15)

where $⨂G_{hg}$ is the important of alternative $h$ in the criterion $g$.

Step 4: Construct the normalized grey decision matrix that the values will be in the range $\left[0,1\right]$ after normalization process, using Equation (16).

$$E^{*}=\left[\begin{array}{cc}\begin{array}{ccc}⨂G_{11}^{*}& ⨂G_{12}^{*}& \cdots \\ ⨂G_{21}^{*}& ⨂G_{22}^{*}& \cdots \end{array}& \begin{array}{c}⨂G_{1n}^{*}\\ ⨂G_{2n}^{*}\end{array}\\ \begin{array}{ccc}⋮& ⋮& ⋮\\ ⨂G_{m1}^{*}& ⨂G_{m2}^{*}& \cdots \end{array}& \begin{array}{c}⋮\\ ⨂G_{mn}^{*}\end{array}\end{array}\right]$$

(16)

If the criteria of benefit set, Equation (17) is used for normalization.

$$G_{gh}^{*}=\left[\frac{{\underset{\_}{G}}_{hg}}{G_g^{max}},\frac{{\overline{G}}_{hg}}{G_g^{max}}\right] \mathrm{where} G_g^{max}=ma{x}_{1\le h\le m}\left\{{\overline{G}}_{hg}\right\}$$

(17)

If the criteria of cost set, Equation (18) is used for normalization.

$$G_{gh}^{*}=\left[\frac{G_g^{min}}{{\overline{G}}_{hg}},\frac{G_g^{min}}{{\underset{\_}{G}}_{hg}}\right] \mathrm{where} G_g^{min}=mi{n}_{1\le h\le m}\left\{{\underset{\_}{G}}_{hg}\right\}$$

(18)

Step 5: Develop the weighted normalized grey decision matrix using Equation (19).

$$X=\left[\begin{array}{cc}\begin{array}{ccc}⨂X_{11}& ⨂X_{12}& \cdots \\ ⨂X_{21}& ⨂X_{22}& \cdots \end{array}& \begin{array}{c}⨂X_{1n}\\ ⨂X_{2n}\end{array}\\ \begin{array}{ccc}⋮& ⋮& ⋮\\ ⨂X_{m1}& ⨂X_{m2}& \cdots \end{array}& \begin{array}{c}⋮\\ ⨂X_{mn}\end{array}\end{array}\right] \mathrm{where} ⨂X_{hg}=⨂G_{hg}^{*}\times ⨂w_g$$

(19)

where $⨂w_g$ denotes the grey weights of each criterion.

Step 6: Compute the values of the positive ideal solution (PIS) and the negative ideal solution (NIS) using Equations (20) and (21), respectively.

$$S^{max}=\left\{\left[ma{x}_{1\le h\le m}{\underset{\_}{X}}_{h1},ma{x}_{1\le h\le m}{\overline{X}}_{h1}\right],\left[ma{x}_{1\le h\le m}{\underset{\_}{X}}_{h2},ma{x}_{1\le h\le m}{\overline{X}}_{h2}\right],\dots ,\left[ma{x}_{1\le h\le m}{\underset{\_}{X}}_{hn},ma{x}_{1\le h\le m}{\overline{X}}_{hn}\right]\right\}$$

(20)

$$S^{min}=\left\{\left[mi{n}_{1\le h\le m}{\underset{\_}{X}}_{h1},mi{n}_{1\le h\le m}{\overline{X}}_{h1}\right],\left[mi{n}_{1\le h\le m}{\underset{\_}{X}}_{h2},mi{n}_{1\le h\le m}{\overline{X}}_{h2}\right],\dots ,\left[mi{n}_{1\le h\le m}{\underset{\_}{X}}_{hn},mi{n}_{1\le h\le m}{\overline{X}}_{hn}\right]\right\}$$

(21)

Step 7: Compute the degree of grey possibility between the ideal solution $S^{max}$ and the alternative set $S_h=\left\{S_1,S_2,\dots ,S_m\right\}$ using Equation (22).

$$P\left\{S_h\le S^{max}\right\}=\frac{1}{n}{\displaystyle \sum }_{g=1}^{n}P\left\{⨂X_{hg}\le ⨂G_g^{max}\right\}$$

(22)

Sorting the value of the grey possibility degree in increasing order. The optimal alternative is selected with the lowest value of the degree of greyness.

3.4. Research Framework

With the research problem defined and critical assessment criteria determined through literature review and experts’ opinions, the study framework in this paper is divided into two main phases of employing the MCDM methodologies (defining research methods), as shown in Figure 2. The G-AHP model is employed in the first step to determine the grey weights of the assessment criterion. The G-TOPSIS model is then used in the second step to determine the ranking of e-learning websites. The suggested methodology is tested using a case study of the top six e-learning websites in Vietnam. This study proposes the integrated MCDM model, which deals with expert linguistic assessments based on grey system theory.

4. Results Analysis

This section focuses on the problem of evaluating and selecting the best e-learning websites in Vietnam. A realistic case study is presented, and criteria and alternatives are analyzed and discussed.

4.1. A Case Study in Vietnam

The proposed model in this paper is tested through a case study of top e-learning websites in Vietnam, which are Unica (EW-01), Edumall (EW-02), Hocmai.vn (EW-03), Kyna.vn (EW-04), Tuyensinh247.com (EW-05), and Moon.vn (EW-06), as can be seen in

Table 3. The list of e-learning websites used in this study.

No	E-Learning Website	Symbol	Website (accessed on 15 November 2021)
1	Unica	EW-01	https://unica.vn/
2	Edumall	EW-02	https://edumall.vn/
3	Hocmai.vn	EW-03	https://hocmai.vn/
4	Kyna.vn	EW-04	https://kyna.vn/
5	Tuyensinh247.com	EW-05	https://tuyensinh247.com/
6	Moon.vn	EW-06	https://moon.vn/

. Along with the literature review for factors, the criterion system and considered alternatives were verified through interviews with six experts with at least ten years of working experience in network teaching. The experts are ICT faculty teaching in e-learning sites, IT specialists, IT managers, as well as researchers who have had experience and worked with e-learning issues. Finally, three assessment aspects and 15 evaluation criteria were selected. The summary of criteria considered from the previous studies is shown in

Table 4. Summary of criteria considered from the previous studies.

Aspect	Criteria	Gong et al. [5]	Hwang et al. [20]	Büyüközkan et al. [23]	Tseng et al. [24]	Jain et al. [26]	Khan et al. [30]	Muhammad et al. [31]	Jaukovic Jocic et al. [32]	Yuen [41]	Singh et al. [42]
Website quality	Design		v		v			v			v
	Navigation	v	v	v		v				v	v
	Response rate		v		v						v
	Impression score										v
	User-friendliness	v	v	v	v	v	v	v		v	v
	Interactivity	v		v		v		v	v
	Connectivity				v			v	v	v	v
	Security	v	v	v		v
Content quality	Right and understandable content	v	v	v	v	v	v	v	v		v
	Complete content	v	v	v		v	v	v	v		v
	Up-to-date		v		v			v	v		v
	Ethical and legal issues				v
Customers’ needs	Variety of educational level							v			v
	Price		v		v
	Personalization	v		v		v	v			v

.

4.2. Estimation of Grey Weights with the G-AHP Model

In this section, the G-AHP model is used to calculate the grey weight of the evaluation criteria. A total of three main aspects are considered, including website quality, content quality, and customer’s needs, which are decomposed into 15 criteria. The criteria and their definition as used in the paper are presented in

Table 5. The criteria and their definition as used in the paper.

Aspect	Criteria	Description
C1. Website quality	C11. Design	The appearance of the site to generate a positive impression to hold the user’s sight once they arrive on the site.
	C12. Navigation	The directions for accessing the relevant information to the user; an easy navigation system can reduce search time and increase the satisfaction level.
	C13. Response rate	Relates to the time taken for loading the information requested by the user. Too much time to download the data from the sites makes users switch to other websites.
	C14. Impression score	Calculated by counting the number of times a user views advertising that appears on a webpage.
	C15. User-friendliness	Denotes the site’ s simplicity, which gives users the ease of understanding needed to use the site correctly, enhancing their satisfaction levels.
	C16. Interactivity	Refers to the interaction between learners and e-learning technologies, and it relates to the level of involvement of learners participating in the learning process.
	C17. Connectivity	A site’s social links with other social networking sites, such as Facebook, Twitter, and YouTube, allow users to connect with people worldwide and boost prospective users.
	C18. Security	Website security is a crucial concern. A digital certificate is required to insert sensitive information on websites securely.
C2. Content quality	C21. Right and understandable content	Credibility, clarity, and brevity are all part of this criterion. When utilizing educational websites, authority is critical since high-quality material must be guaranteed, ensuring that the learning objectives are satisfied. The text should be simple to comprehend, clear, and concise.
	C22. Complete content	Accuracy and coverage are included, ensuring that the material is accurate: current, factual, thorough, precise, and complete. It also considers the presence of tests, quizzes, and examinations to determine whether or not suitable assessment methods are in place.
	C23. Up-to-date	Signifies if the quantity of information on the site is up-to-date. If the material is stagnant and not updated, the user will lose interest. As a result, continuously updating the material encourages users to return to the site and enhances its quality.
	C24. Ethical and legal issues	When building and developing an online program, it is crucial to understand specific legal and ethical issues related to online learning settings.
C3. Customers’ needs	C31. Variety of educational level	It includes the users who are pursuing either standard or higher education or conducting research on any topic.
	C32. Price	Whether the cost of e-learning programs on a specific website is worthwhile in terms of their teaching materials and methods.
	C33. Personalization	This dimension states a level of individualization from customers’ requirements, making the website more attractive for e-learners.

.

The following procedure demonstrates how to determine the weight (weight of eigenvector) of the three main aspects (website quality (C1), content quality (C2), and customer’s needs (C3)) and the calculation of the consistency ratio. The initial comparison matrix with linguistic variables of the G-AHP model are presented in

Table 6. The initial comparison matrix of the G-AHP model.

Aspect	Linguistics Variables									Aspect
	EMI	VSI	SI	MI	EI	MI	SI	VSI	EMI
C1						1	3	2		C2
C1				1	1	2	2			C3
C2		1	2	3						C3

below.

The linguistics variables with grey number are converted to the crisp number to check the consistency ratio (CR) of the performance rating from expert’ s judgments. The crisp matrix of the three main aspects is shown in

Table 7. The crisp matrix of the G-AHP model.

Aspect	Website Quality (C1)	Content Quality (C2)	Customers’ Needs (C3)
Website quality (C1)	1.0000	0.1947	0.4870
Content quality (C2)	5.1369	1.0000	4.0964
Customers’ needs (C3)	2.0536	0.2441	1.0000
Total	8.1905	1.4388	5.5833

.

To obtain the priority vector of the three main aspects, the normalized matrix of the G-AHP model is created by dividing each value in a column of the matrix by its column sum. The priority vector is then computed by averaging the row elements in the normalized matrix, as illustrated in

Table 8. The normalized matrix of the G-AHP model.

Aspect	Website Quality (C1)	Content Quality (C2)	Customers’ Needs (C3)	Priority Vector
Website quality (C1)	0.1221	0.1353	0.0872	0.1149
Content quality (C2)	0.6272	0.6950	0.7337	0.6853
Customers’ needs (C3)	0.2507	0.1697	0.1791	0.1998
Total	1.0000	1.0000	1.0000	1.0000

.

The largest eigenvector (${\lambda }_{max}$) is calculated to get the consistency index (CI), the random index (RI), and the consistency ratio (CR), as follows.

$$\begin{gathered} \left[\begin{array}{ccc}1.0000& 0.1947& 0.4870\\ 5.1369& 1.0000& 4.0964\\ 2.0536& 2.2441& 1.0000\end{array}\right]\times \left[\begin{array}{c}0.1149\\ 0.6853\\ 0.1998\end{array}\right]=\left[\begin{array}{c}0.3456\\ 2.0940\\ 0.6030\end{array}\right]; \\ \left[\begin{array}{c}0.3456\\ 2.0940\\ 0.6030\end{array}\right]/\left[\begin{array}{c}0.1149\\ 0.6853\\ 0.1998\end{array}\right]=\left[\begin{array}{c}3.0085\\ 3.0556\\ 3.0176\end{array}\right] \end{gathered}$$

In this paper, a total of three main aspects is analyzed. Hence, we get $n=3$. Consequently, ${\lambda }_{max}$ and CI are calculated as follows:

$${\lambda }_{max}=\frac{3.0085+3.0556+3.0176}{3}=3.0272$$

$$CI=\frac{{\lambda }_{max}-n}{n-1}=\frac{3.0272-3}{3-1}=0.0136$$

such that $n=3,$ we get $RI=0.58$, and the consistency ratio (CR) is calculated as follows:

$$CR=\frac{CI}{RI}=\frac{0.0136}{0.58}=0.0235$$

From the result, $CR=0.0235<0.1$. As a result, the pairwise comparison matrix is consistent, and the G-AHP model output is acceptable. Then, using the same formula, the other criteria are computed.

Table A1. The normalized grey comparison matrix used in the G-AHP model.

Criteria	C11		C12		C13		C14		C15
C11. Design	0.0323	0.0323	0.0319	0.0487	0.0124	0.0232	0.0087	0.0155	0.0290	0.0506
C12. Navigation	0.0137	0.0209	0.0207	0.0207	0.0046	0.0073	0.0068	0.0109	0.0063	0.0100
C13. Response rate	0.0387	0.0724	0.0788	0.1260	0.0279	0.0279	0.0253	0.0437	0.0122	0.0188
C14. Impression score	0.0870	0.1550	0.0788	0.1260	0.0266	0.0460	0.0417	0.0417	0.0638	0.1104
C15. User-friendliness	0.0244	0.0426	0.0788	0.1260	0.0569	0.0877	0.0144	0.0250	0.0383	0.0383
C16. Interactivity	0.0173	0.0302	0.0521	0.0793	0.0403	0.0650	0.0262	0.0446	0.0319	0.0516
C17. Connectivity	0.0173	0.0302	0.0521	0.0793	0.0403	0.0650	0.0173	0.0315	0.0284	0.0460
C18. Security	0.0274	0.0512	0.0788	0.1260	0.0904	0.1327	0.1699	0.2571	0.0161	0.0230
C21. Right and understandable content	0.0976	0.1658	0.0328	0.0585	0.0627	0.1004	0.0186	0.0371	0.0161	0.0230
C22. Complete content	0.0217	0.0355	0.0496	0.0928	0.0845	0.1265	0.0186	0.0371	0.0284	0.0460
C23. Up-to-date	0.0690	0.1172	0.0067	0.0103	0.0627	0.1004	0.0186	0.0371	0.0319	0.0552
C24. Ethical and legal issues	0.0256	0.0456	0.0625	0.1062	0.0532	0.0836	0.0525	0.0954	0.1158	0.1968
C31. Variety of educational level	0.1381	0.2103	0.1251	0.1680	0.1064	0.1672	0.1699	0.2617	0.1788	0.2601
C32. Price	0.1381	0.2103	0.0227	0.0337	0.1064	0.1672	0.1699	0.2617	0.1788	0.2601
C33. Personalization	0.0120	0.0203	0.0101	0.0170	0.0090	0.0157	0.0155	0.0262	0.0135	0.0211
Criteria	C16		C17		C18		C21		C22
C11. Design	0.0361	0.0631	0.0399	0.0696	0.0229	0.0429	0.0210	0.0356	0.0785	0.1282
C12. Navigation	0.0088	0.0134	0.0097	0.0148	0.0060	0.0095	0.0381	0.0678	0.0192	0.0360
C13. Response rate	0.0145	0.0234	0.0160	0.0258	0.0077	0.0112	0.0299	0.0480	0.0191	0.0285
C14. Impression score	0.0316	0.0536	0.0492	0.0895	0.0059	0.0089	0.1209	0.2417	0.0970	0.1939
C15. User-friendliness	0.0251	0.0406	0.0310	0.0502	0.0606	0.0865	0.1793	0.2559	0.0719	0.1164
C16. Interactivity	0.0338	0.0338	0.0230	0.0372	0.0119	0.0182	0.0353	0.0538	0.0210	0.0320
C17. Connectivity	0.0338	0.0547	0.0372	0.0372	0.0551	0.0779	0.0180	0.0282	0.0330	0.0571
C18. Security	0.0675	0.1029	0.0174	0.0246	0.0364	0.0364	0.0941	0.1480	0.0124	0.0180
C21. Right and understandable content	0.0675	0.1029	0.1420	0.2230	0.0265	0.0417	0.1077	0.1077	0.2329	0.4150
C22. Complete content	0.0910	0.1387	0.0564	0.0974	0.1748	0.2539	0.0224	0.0399	0.0864	0.0864
C23. Up-to-date	0.0910	0.1387	0.0171	0.0273	0.0265	0.0417	0.0210	0.0333	0.0123	0.0176
C24. Ethical and legal issues	0.0056	0.0084	0.0171	0.0273	0.0058	0.0089	0.0356	0.0634	0.0204	0.0320
C31. Variety of educational level	0.1378	0.2121	0.1054	0.1588	0.1557	0.2194	0.0220	0.0356	0.0194	0.0320
C32. Price	0.1378	0.1961	0.2008	0.2829	0.1962	0.2764	0.0220	0.0356	0.0194	0.0320
C33. Personalization	0.0134	0.0223	0.0251	0.0469	0.0270	0.0476	0.0159	0.0224	0.0129	0.0189
Criteria	C23		C24		C31		C32		C33
C11. Design	0.0131	0.0223	0.0160	0.0286	0.0049	0.0075	0.0055	0.0084	0.0371	0.0630
C12. Navigation	0.0955	0.1474	0.0044	0.0075	0.0040	0.0053	0.0222	0.0328	0.0284	0.0477
C13. Response rate	0.0133	0.0213	0.0076	0.0119	0.0054	0.0084	0.0060	0.0095	0.0416	0.0721
C14. Impression score	0.0536	0.1072	0.0099	0.0180	0.0051	0.0079	0.0057	0.0089	0.0371	0.0630
C15. User-friendliness	0.0331	0.0574	0.0044	0.0075	0.0047	0.0069	0.0053	0.0077	0.0425	0.0661
C16. Interactivity	0.0116	0.0177	0.0907	0.1368	0.0051	0.0079	0.0062	0.0089	0.0354	0.0589
C17. Connectivity	0.0651	0.1042	0.0309	0.0495	0.0076	0.0114	0.0048	0.0067	0.0185	0.0347
C18. Security	0.0417	0.0657	0.0925	0.1425	0.0053	0.0075	0.0048	0.0067	0.0179	0.0315
C21. Right and understandable content	0.1546	0.2455	0.0385	0.0686	0.0975	0.1578	0.1093	0.1769	0.1122	0.1581
C22. Complete content	0.2340	0.3367	0.0612	0.0960	0.0868	0.1431	0.0973	0.1604	0.1070	0.1570
C23. Up-to-date	0.0478	0.0478	0.1469	0.1934	0.1406	0.2023	0.1473	0.2200	0.1587	0.2068
C24. Ethical and legal issues	0.0056	0.0074	0.0227	0.0227	0.0049	0.0075	0.0052	0.0075	0.0049	0.0076
C31. Variety of educational level	0.0076	0.0109	0.0971	0.1479	0.0322	0.0322	0.2624	0.3352	0.1414	0.1899
C32. Price	0.0078	0.0117	0.1090	0.1582	0.2341	0.2990	0.0361	0.0361	0.0051	0.0093
C33. Personalization	0.0054	0.0070	0.0700	0.1090	0.1950	0.2618	0.0910	0.1653	0.0234	0.0234

shows the normalized grey comparison matrix utilized in the G-AHP model (Appendix A).

The relative grey weights from the G-AHP model is given in

Table 9. The relative grey weights from the G-AHP model.

Criteria	Grey Weights		Influence Level
C11. Design	0.0260	0.0426	3.4%
C12. Navigation	0.0192	0.0301	2.5%
C13. Response rate	0.0229	0.0366	3.0%
C14. Impression score	0.0476	0.0848	6.6%
C15. User-friendliness	0.0447	0.0676	5.6%
C16. Interactivity	0.0295	0.0451	3.7%
C17. Connectivity	0.0306	0.0476	3.9%
C18. Security	0.0515	0.0782	6.5%
C21. Right and understandable content	0.0878	0.1388	11.3%
C22. Complete content	0.0813	0.1232	10.2%
C23. Up-to-date	0.0665	0.0966	8.2%
C24. Ethical and legal issues	0.0292	0.0480	3.9%
C31. Variety of educational level	0.1133	0.1627	13.8%
C32. Price	0.1056	0.1514	12.8%
C33. Personalization	0.0359	0.0550	4.5%

. Based in the results, for example, the grey weight of criteria design (C11) has the lowest weight (pessimistic value) at 0.0260 and the highest weight (optimistic value) at 0.0426. As the same concept, the grey weight of criteria Navigation (C12) has the lowest weight at 0.0192 and the highest weight of 0.0301. Other criteria have the same explanation. These grey weights will be converted into script value (the influence level of criteria) through the whitenization of the grey weight. Consequently, the obtained weights are visualized in Figure 3. As it can be seen, variety of educational level (C31), price (C32), right and understandable content (C21), complete content (C22), and up-to-date (C23) were found as the most impactful criteria with weights of 13.8%, 12.8%, 11.3%, 10.2%, and 8.2%, respectively.

4.3. Ranking Alternatives with the G-TOPSIS Model

In this section, the linguistics with grey number in the G-TOPSIS model is used to describe the performance ranking of six e-learning websites (alternatives) in Vietnam, which are Unica (EW-01), Edumall (EW-02), Hocmai.vn (EW-03), Kyna.vn (EW-04), Tuyensinh247.com (EW-05), and Moon.vn (EW-06). The decision tree for evaluation of e-learning websites is visualized in Figure 4.

In this paper, the relative grey weight of criteria is calculated from the G-AHP model. According to the G-TOPSIS procedures, the optimal alternative is determined with the lowest value of the grey possibility degree. According to the G-TOPSIS procedure, the normalized grey decision matrix used in the G-TOPSIS model is shown in

Table A2. The normalized grey decision matrix used in the G-TOPSIS model.

E-learning Website	Symbol	C11		C12		C13		C14		C15
Unica	EW-01	0.6038	0.8679	0.5769	0.8654	0.6038	0.8679	0.8333	1.0000	0.8333	1.0000
Edumall	EW-02	0.8491	1.0000	0.8269	1.0000	0.8491	1.0000	0.6389	0.8611	0.5000	0.7778
Hocmai.vn	EW-03	0.3962	0.5849	0.2692	0.4038	0.2830	0.4151	0.6111	0.8333	0.6667	0.8889
Kyna.vn	EW-04	0.1509	0.2642	0.1154	0.2500	0.1509	0.2642	0.3889	0.5556	0.3889	0.5556
Tuyensinh247.com	EW-05	0.1321	0.3208	0.1346	0.3269	0.5094	0.6226	0.3611	0.6389	0.6111	0.8056
Moon.vn	EW-06	0.4906	0.6038	0.2115	0.3269	0.2075	0.3208	0.4722	0.6389	0.4722	0.6389
E-learning Website	Symbol	C16		C17		C18		C21		C22
Unica	EW-01	0.8182	1.0000	0.7632	0.9737	0.4407	0.5763	0.4576	0.5932	0.4151	0.5472
Edumall	EW-02	0.4545	0.7273	0.6053	0.8947	0.8644	1.0000	0.8644	1.0000	0.8679	1.0000
Hocmai.vn	EW-03	0.5758	0.8182	0.5526	0.8158	0.3729	0.5424	0.3729	0.5424	0.3962	0.5849
Kyna.vn	EW-04	0.2424	0.4242	0.2895	0.5000	0.3051	0.4407	0.4237	0.5593	0.1132	0.2453
Tuyensinh247.com	EW-05	0.2121	0.5152	0.7895	1.0000	0.1864	0.4068	0.5424	0.6780	0.4717	0.6038
Moon.vn	EW-06	0.0909	0.2727	0.4474	0.7368	0.4576	0.5932	0.3220	0.4237	0.1698	0.3019
E-learning Website	Symbol	C23		C24		C31		C32		C33
Unica	EW-01	0.7308	1.0000	0.6111	0.8704	0.6038	0.8679	0.3889	0.4667	0.5946	0.7838
Edumall	EW-02	0.5577	0.6731	0.8519	1.0000	0.8491	1.0000	0.5000	0.7778	0.3243	0.5946
Hocmai.vn	EW-03	0.4615	0.6154	0.4444	0.6111	0.3962	0.5849	0.3784	0.5185	0.7297	1.0000
Kyna.vn	EW-04	0.2115	0.3654	0.5185	0.6667	0.4528	0.6038	0.7000	1.0000	0.2162	0.3784
Tuyensinh247.com	EW-05	0.5769	0.7115	0.4444	0.5741	0.5094	0.6226	0.3784	0.4667	0.6757	0.8378
Moon.vn	EW-06	0.3846	0.5385	0.0556	0.1667	0.0943	0.2453	0.2979	0.4000	0.2973	0.3784

(Appendix A). Following that, the values of the grey positive and negative ideal solutions are calculated in

Table 10. The values of the grey positive and negative ideal solutions from the G-TOPSIS model.

Criteria	${\mathit{S}}^{\mathit{m}\mathit{a}\mathit{x}}$		${\mathit{S}}^{\mathit{m}\mathit{i}\mathit{n}}$
C11. Design	0.0220	0.0426	0.0034	0.0113
C12. Navigation	0.0159	0.0301	0.0022	0.0075
C13. Response rate	0.0195	0.0366	0.0035	0.0097
C14. Impression score	0.0397	0.0848	0.0172	0.0471
C15. User-friendliness	0.0373	0.0676	0.0174	0.0376
C16. Interactivity	0.0241	0.0451	0.0027	0.0123
C17. Connectivity	0.0242	0.0476	0.0089	0.0238
C18. Security	0.0445	0.0782	0.0096	0.0318
C21. Right and understandable content	0.0759	0.1388	0.0283	0.0588
C22. Complete content	0.0706	0.1232	0.0092	0.0302
C23. Up-to-date	0.0486	0.0966	0.0141	0.0353
C24. Ethical and legal issues	0.0248	0.0480	0.0016	0.0080
C31. Variety of educational level	0.0962	0.1627	0.0107	0.0399
C32. Price	0.0739	0.1514	0.0315	0.0605
C33. Personalization	0.0262	0.0550	0.0078	0.0208

.

Following that, the results of the degree of grey possibility between the ideal solution $S^{max}$ and the alternative set of six e-learning websites, $S_h=\left\{S_1,S_2,\dots ,S_6\right\}$, are shown as follows.

• Unica (EW-01), $P\left\{S_1\le S^{max}\right\}=0.6918;$
• Edumall (EW-02), $P\left\{S_2\le S^{max}\right\}=0.6074;$
• Hocmai.vn (EW-03), $P\left\{S_3\le S^{max}\right\}=0.8492;$
• Kyna.vn (EW-04), $P\left\{S_4\le S^{max}\right\}=0.9452;$
• Tuyensinh247.com (EW-05), $P\left\{S_5\le S^{max}\right\}=0.8619;$
• Moon.vn (EW-06), $P\left\{S_6\le S^{max}\right\}=0.9510$.

Then, sorting the value of the grey possibility degree in increasing order, the optimal e-learning website is determined with the lowest value of the degree of greyness. The final ranking order of all alternatives from the G-TOPSIS model is visualized in Figure 5. Based on the results, the top three potential e-learning websites are Edumall (EW-02), Unica (EW-01), and Hocmai.vn (EW-03), prioritized as first, second, and third with the possibility of 0.6074, 0.6918, and 0.8492, respectively.

5. Comparative Analysis of Methods

In this section, a comparative analysis of methods is conducted to demonstrate the applicability and rationality of the presented G-AHP and G-TOPSIS models. In this paper, the ranking of e-learning websites is evaluated by TOPSIS [43], G-EDAS [44], and G-COPRAS [45] models. The ranking results of compared methods are presented in

Table 11. Ranking results of compared methods.

E-Learning Website	Symbol	G-AHP G-TOPSIS		G-AHP TOPSIS		G-AHP G-EDAS		G-AHP G-COPRAS
		Value	Ranking	Value	Ranking	Value	Ranking	Value	Ranking
Unica	EW-01	0.6918	2	0.5674	2	0.6265	2	0.8371	2
Edumall	EW-02	0.6074	1	0.8204	1	0.7635	1	1.0000	1
Hocmai.vn	EW-03	0.8492	3	0.4047	4	0.4636	3	0.6744	3
Kyna.vn	EW-04	0.9452	5	0.3573	5	0.3241	5	0.5562	5
Tuyensinh247.com	EW-05	0.8619	4	0.4416	3	0.4558	4	0.6644	4
Moon.vn	EW-06	0.9510	6	0.1350	6	0.2336	6	0.4458	6

and visualized in Figure 6. The comparison results suggest that the rating of the e-learning websites in Vietnam has given the same result as the model proposed in this paper, which is among the integrated models of G-AHP and G-TOPSIS, G-AHP and G-EDAS, and G-AHP and G-COPRAS. The ranking of the G-AHP and TOPSIS models is slightly different from that of the proposed model. The difference is between Tuyensinh247.com (EW-05) and Hocmai.vn (EW-03). In short, the priority orders of the best websites are similar when the applied methods reach common rankings; Edumall got the highest rank. These demonstrate the efficacy of the proposed methodology for evaluating and selecting e-learning websites. As a result of the above comparative analysis, it is possible to conclude that the ranking order of the e-learning websites is reliable and can be a useful guideline for users or decision makers in determining the suitable online platform, especially in the COVID-19 pandemic.

6. Conclusions

As the globe entered the knowledge economy, online education via e-learning platforms emerged worldwide as an inevitable development trend. Despite its status as a developing country, Vietnam has swiftly caught up with this trend and is now a viable market for domestic and global investors. Identifying critical factors and evaluating e-learning websites are essential to e-learning service providers and system developers and researchers to improve online educational platforms as well as to e-learners who pursue their knowledge via the Internet. Regarding the problem of assessing e-learning platforms, the uncertainty of the evaluation environment existing during the decision-making process needs to be addressed. For managerial implications, this study has conducted an effective method for e-learning website evaluation and selection. Successful e-learning implementations require technology awareness, motivation, and a change in learners’ behaviors. Researchers and practitioners will find several recommendations helpful in assisting the implementation of e-learning systems in developing countries.

In the present study, a hybrid model that integrates G-AHP and G-TOPSIS is proposed for the first time to evaluate and select e-learning websites. To the best of our knowledge, the proposed integrated approach has not been reported elsewhere. A comprehensive set of indicators through literature review and experts’ opinions was developed to employ the integrated model. More specifically, G-AHP is adopted to determine the significance levels of criteria and sub-criteria, and then G-TOPSIS is utilized to prioritize the websites. The suggested grey MCDM approach has the advantage of processing the uncertain evaluations manifested by grey numbers to generate a more accurate and robust ranking for the alternatives. A practical case study of evaluating e-learning websites in Vietnam is solved for the first time. A comparative analysis of other MCDM methods is performed to test the robustness of the proposed model. The paper is directed toward providing a helpful guideline to e-learning service providers, system developers, and researchers related to web research.

The following study directions are suggested for future studies. First, the proposed model is restricted to a small number of experts. It is advised that a new technique be implemented to handle e-learning website selection issues in large groups in the future. Comprehensive research can be carried out by including other evaluation criteria to enhance the proposed model, such as cutting-edge technologies and those following the COVID-19 pandemic. Applying other novel and robust MCDM techniques (i.e., DEA, WASPAS, VIKOR) [46,47,48] are recommended in terms of methodologies. For handling uncertainty in the decision-making process, spherical fuzzy sets theory [49] is one of the novel methods that can signify avenues for researchers to obtain more robust results.