Next Article in Journal
Monitoring and Evaluation of High-Speed Railway Turnout Grinding Effect Based on Field Test and Simulation
Next Article in Special Issue
Machine Learning Ensemble Modelling for Predicting Unemployment Duration
Previous Article in Journal
Marine-Originated Materials and Their Potential Use in Biomedicine
Previous Article in Special Issue
A Dynamic Grid Index for CkNN Queries on Large-Scale Road Networks with Moving Objects
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:

Effects of the Hybrid CRITIC–VIKOR Method on Product Aspect Ranking in Customer Reviews

Saif Addeen Ahmad Alrababah
1,* and
Keng Hoon Gan
Department of Information Systems, Faculty of Prince Hussein bin Abdullah for Information Technology, Al Al-Bayt University, Mafraq 25113, Jordan
School of Computer Sciences, Universiti Sains Malaysia, USM, Penang 11800, Malaysia
Author to whom correspondence should be addressed.
Appl. Sci. 2023, 13(16), 9176;
Submission received: 6 July 2023 / Revised: 27 July 2023 / Accepted: 2 August 2023 / Published: 11 August 2023
(This article belongs to the Special Issue Data Analysis and Mining: New Techniques and Applications)


Product aspect ranking is critical for prioritizing the most important aspects of a specific product/service to assist probable customers in selecting suitable products that can realize their needs. However, given the voluminous customer reviews published on websites, customers are hindered from manually extracting and characterizing the specific aspects of searched products. A few multicriteria decision-making methods have been implemented to rank the most relevant product aspects. As weights greatly affect the ranking results of product aspects, this study used objective methods in finding the importance degree of a criteria set to overcome the limitations of subjective weighting. The growing popularity of online shopping has led to an exponential increase in the number of customer reviews available on various e-commerce websites. The sheer volume of these reviews makes it nearly impossible for customers to manually extract and analyze the specific aspects of the products they are interested in. This challenge highlights the need for automated techniques that can efficiently rank the product aspects based on their relevance and importance. Multicriteria decision-making techniques can address the issue of product aspect ranking. These techniques seek to offer a methodical strategy for assessing and contrasting various product attributes based on various criteria. The subjective nature of determining weights for each criterion raises serious issues because it might lead to bias and inconsistent ranking outcomes. The CRITIC–VIKOR method was adopted in the product aspect ranking process. The statistical findings based on a benchmark dataset using NDCG demonstrate the superior performance of the method of using objective weighting to reasonably acquire subjective weighting results. Also, the results show that the product aspects ranked by using CRITIC–VIKOR could be considered guidelines for probable customers to make a wise purchasing decision.

1. Introduction

Customers can freely share their thoughts and experiences regarding goods and services anytime and anyplace thanks to the social web. By giving customers knowledge about goods and services that other people have used, online reviews, like other electronic word of mouth (eWOM) methods, help to reduce the uncertainty associated with online shopping [1]. Therefore, the use of surveys to collect the opinions of customers about products and subsequently measure the degree of customer satisfaction may no longer be needed for organizations, especially since such information is already available online given the explosive growth of social networks [2,3]. Moreover, online user feedback can be accessed in real time and does not involve any costs. Without the need for expensive survey initiatives, firms can now access a multitude of easily available consumer reviews to learn more about how well their products function, pinpoint areas for improvement, and make wise business decisions. Utilizing this beneficial resource enables businesses to communicate with their clients directly and anticipate their wants and preferences. Additionally, the openness of consumer reviews on e-commerce websites encourages accountability and transparency, raising the platform’s general credibility and dependability.
Nonetheless, the massive growth of online opinion data has overwhelmed users and firms such that manually tracking these online reviews has become a dreadful task [1]. For instance, for TripAdvisor, which is considered one of the best review platforms to find information regarding accommodation, hotels, and other services, the number of online reviews on this platform has grown from 200 million in 2014 to more than 600 million in 2017 [4].
The valuable knowledge extractable from high-volume online reviews that can benefit customers and firms has inspired researchers to design automatic review approaches. Most of their studies have focused on techniques to identify product features and the sentiments of users regarding each aspect based on NLP and statistical techniques; however, the ranking and prioritization of critical aspects that can considerably affect the decisions of customers and firms are rarely investigated [5,6,7]. Some of the studies have performed a ranking process for candidate product aspects based on the statistical information of their occurrences as a complementary task to the extraction process. Nonetheless, the significance of these aspects differs in their influence on the satisfaction of customers toward a product. For instance, some aspects of the iPhone, such as “battery” and “usability”, are considered more important than “usb” and “button”. Furthermore, customers search for quality information in website reviews. Guiding customers to focus on important product aspects will allow them to make wise purchasing decisions.
In order to help them make informed purchasing decisions, buyers actively seek out reliable information in online evaluations. They wish to concentrate on the factors that are crucial and significantly affect their enjoyment. Therefore, it becomes crucial to have a solid and trustworthy process for methodically rating and prioritizing important factors. Customers can make informed selections and choose products that match their unique needs and tastes by receiving clear instructions on the significance of various product characteristics. Additionally, organizations are better equipped to prioritize product developments and spend resources efficiently when they comprehend the relative importance of various components. Businesses can strategically improve their products and obtain an advantage in the market by concentrating on the factors that have the biggest effects on consumer happiness.
A novel framework was proposed [8] that combined sentiment analysis and multicriteria decision making (MCDM) to rank multiple alternatives based on three extraction criteria: frequency-based product aspects, opinion-related product aspects, and domain-related aspects. They particularly investigated the classical subjective technique for order preference by using the similarity to ideal solution (TOPSIS) method [9], which is based on the shortest distance to the positive ideal solution and the farthest distance from the negative solution. The best values of the decision matrix’s criteria are all contained in a positive ideal solution, whereas the worst values are all contained in a negative ideal solution [10].
In typical MCDM methods, a weight needs to be assigned to a criterion to indicate its relative importance in the decision-making process. Criteria weights considerably influence the final ranking of decision alternatives. Moreover, most MCDM methods assign criteria weights by using crisp values based on human judgments [11,12,13,14], which means that the relative importance of each criterion depends on the subjective preferences of decision makers. However, subjective weights cannot accurately evaluate a criterion because of the uncertainty of human judgments, and assessment errors are usually inevitable [11,14]. Furthermore, despite the widespread belief that the decision criteria in MCDM problems are independent of one another, this is frequently not the case in many situations [14].
This research aims to improve the ranking process of product aspects by enhancing the criteria weighting task in MCDM. In this proposed scheme, objective weighting (i.e., a method independent of decision-making judgments) was prioritized, and mathematical models for handling known data in a decision matrix were used to automatically determine the criteria weights [11,15]. Among the many multiple-objective weighting approaches explored in the literature, the criteria importance through intercriteria correlation (CRITIC) method [16] was selected as the focal point of this research. This method provides a robust and objective measure for determining the importance of each criterion by leveraging intercriteria correlations. By capturing the relationships and dependencies between criteria, CRITIC enhances the accuracy and reliability of the criteria weighting process.
To further advance the ranking process, CRITIC was investigated alongside VIKOR, a well-known multicriteria decision-making approach initially developed by [17] to address the challenge of finding a compromise solution when confronted with multiple criteria. Combining the strengths of CRITIC and VIKOR allows for a comprehensive and effective product aspect ranking process that takes into account both objective criteria weights and the need for compromise in decision making. By adopting objective weighting methods and leveraging the synergy between CRITIC and VIKOR, this research seeks to enhance the overall ranking process of product aspects. The proposed approach aims to provide more accurate and reliable results that align with the needs and preferences of customers and facilitate informed decision making for both customers and firms.
The remainder of this manuscript is organized as follows: Section 2 reviews the literature by dividing it into three subsections. Types of criteria weighting are briefly reviewed in the first subsection with some examples of each type, and then in the second subsection, the CRITIC objective weighting approach is discussed in detail. The third subsection introduces the VIKOR approach as one of the popular MCDM ranking techniques. Section 3 proposes the overall framework of ranking product aspects and explains in detail the deployment of objective weighting in the VIKOR approach. Section 4 presents the evaluation procedure of the proposed framework and presents the discussion and results. Finally, Section 5 concludes the manuscript and anticipates future research.

2. Literature Review and Theory

2.1. Types of Criteria Weighting Tasks in MCDM

Criteria weighting is considered an essential stage in MCDM approaches, indicating the relative importance of each participating criterion in the decision-making process. The two weighting approaches for handling a criteria set in MCDM are the subjective and objective approaches [11,16]. Subjective weighting depends on the judgments of decision makers who assign crisp values to each criterion and rank all criteria based on experience and knowledge. Then, mathematical methods are applied to evaluate the importance of each criterion. Several subjective weighting methods have been proposed in the literature, such as the simple attribute weighting technique [17], direct rating [18], and analytical hierarchy process (AHP) [19]. AHP, a subjective weighting method based on pairwise comparisons, is often used to evaluate the preferences of decision makers regarding a set of criteria, after which these preferences undergo a ranking process. AHP allows decision makers to decompose a decision problem into a hierarchy, such as a goal level, criteria level, or subcriteria level (lowest in the hierarchy). Pairwise comparisons between the pairs of criteria on the same level are performed on a scale of 1 to 9 [19]. Although AHP has gained acceptance from many researchers, it is constrained by limitations such as uncertainties in the judgments of decision makers [20]. Furthermore, the hierarchical structure of AHP requires the criteria to be independent of one another, which is not always true in real-world applications [21].
An advanced version of the AHP method proposed by [22] is the analytical network process (ANP). In ANP, the goal, criteria, and alternatives are formulated as a network structure to process the interdependencies among the criteria during the weighting process [13]. An essential step in the ANP procedure is for the decision maker to determine in advance the relative importance of the criteria. ANP also requires many more pairwise comparisons [23]. Furthermore, in the criteria weighting process, fuzzy set theory is incorporated into the judgments of decision makers, allowing them to quantitatively express the criteria as linguistic variables (i.e., ordinary, important, and very important) instead of using crisp values [24,25,26,27]. AHP approaches are considered expert-dependent methods because the weights are determined by a decision group of  K  persons [28], and the final weight  w j  for each criterion  j  is obtained by aggregating the fuzzy weights provided by the decision makers, as shown in Equation (1). Additionally, fuzzy weighting approaches have inert limitations that have been derived from the traditional assumption that the evaluation criteria must be considered independently.
w j = i = 1 k w j k
Objective weighting is highly independent of the judgments of decision makers, and the criteria weights can be determined automatically using mathematical models that utilize the known data in the decision matrix [11,15]. Furthermore, in contrast to subjective approaches, objective approaches are more suitable for situations in which reliable subjective weights cannot be obtained [23]. The fuzziness of human judgments can be eliminated using the aforementioned approaches. Various objective measures have been proposed to mathematically evaluate the criteria weights. Examples include the entropy method of Shannon [29], which computes weights based on the degree of divergence in the performance of a criterion; the equal weight method [15], which considers the criteria as having importance; the criteria importance method that uses intercriteria correlation (i.e., the CRITIC method) [30], which computes the weights of the criteria based on correlation analysis to incorporate both the contrast intensity and conflict between the criteria; and the preference selection index method [31], which is based on the degree of convergence in the performance rating of each criterion. Recently, the research study of [32] drew attention to the fact that both subjective and objective methods ignore the significance of criterion value and introduced SODOSM, which combines subjective and objective approaches to overcome this drawback.
Subjective weighting methods based on human assessment of the importance of the criteria may generate inaccurate values [33] because they depend on the knowledge and experience of decision makers while processing the decision-making problem. Moreover, the vagueness of the judgments of decision makers complicates the adoption of decisions in real-life situations [34]. The ranked criteria in subjective weighting take the form of benefit criteria (i.e., better performance is preferred, but the criteria must be mutually dependent), which creates a conflict for the decision maker when attempting to decide the relative importance of the criteria. The analysis presented above underscores the suitability of mathematically determining criteria weights through objective approaches. Objective methods offer a more robust and reliable means of establishing the importance of criteria. In this study, the CRITIC weighting method, representing an objective approach, was investigated alongside the VIKOR approach to prioritize the most important product aspects for potential customers. By employing objective approaches, this research seeks to overcome the limitations of subjective weighting and provide a more accurate and unbiased ranking of product aspects.

2.2. CRITIC Weighting Method

The CRITIC method generates objective weights for a criteria set based on the divergence in the scores of the alternatives in the decision matrix [35]. By analyzing the contrast intensity and conflict within the decision problem data, the CRITIC method derives robust and objective weights. It provides a systematic and reliable means of determining the importance of each criterion, taking into account the inherent variations and conflicts present in the decision data. It offers a more objective perspective, free from biases and subjectivity, which enhances the accuracy and validity of the criteria weighting process [30]. In particular, the contrast intensity of criterion  j  is measured by the standard deviation ( σ j ) after normalizing the performances of the alternatives. Equation (2) is employed for criteria normalization.
r i j = x i j x j x j + x j .
Then, the measure of conflict (or deviation) between the corresponding criterion  j  and the other criteria is applied by using Equation (3)
k = 1 n ( 1 p j k ) ,
where  p j k  is the correlation coefficient between criteria  j  and  k , which is computed by using Equation (4):
p j k = i = 1 m r i j r j ¯ · ( r i k r k ¯ ) i = 1 m r i j r j ¯ 2 · i = 1 m r i k r k ¯ 2 .
The assumption underlying this method is that interdependent criteria can potentially lead to misleading results [30]. This arises from the challenge of distinguishing decision alternatives when the criteria are similar to each other. To address this, a deviation measure is employed to assess the degree of discordance between a particular criterion and the remaining criteria, enabling a more accurate evaluation of the alternatives. Considering the interdependence among criteria and utilizing deviation measures, the proposed method offers a comprehensive and nuanced approach to criteria weighting. This helps overcome the limitations of traditional methods that may overlook the intricate relationships between criteria, ultimately leading to more reliable and meaningful rankings of product aspects.
The amount of information emitted by criterion  C j  is determined by composing both conflict and contrast intensity measures, as shown in Equation (5). The greater the value of  C j , the larger the amount of information contained in criterion  j , and thus, the higher the relative importance of the criterion in the decision-making problem. The normalized objective weight of criterion  j  is calculated by using Equation (6).
C j = σ j . k = 1 n ( 1 p j k ) .
w j = C j k = 1 n C k .
Previous studies have attempted to combine the CRITIC method with other subjective or objective approaches for weight assessment. For instance, [36] designed a hybrid CRITIC–VIKOR method to evaluate the ranking of initial public offerings. Moreover, [37] introduced CRITIC into COCOSO for use in a fuzzy DM environment. Additionally, [38] designed a hybrid PF CRITIC–COCOSO method for the 5G industry. To the best of our knowledge, no research has previously investigated the usability of the CRITIC approach for solving problems related to product aspect ranking.

2.3. VIKOR Method

The VIKOR method was originally developed by [39] as an MCDM approach for finding a compromise solution in the existence of multiple criteria. It has been utilized in many domains for ranking several alternatives based on a variety of decision-making criteria. For instance, the fuzzy-VIKOR multicriteria decision-making approach was utilized in the study of [40] to evaluate, select, and rank the best wheat suppliers in Jordan. In the study of [41], a novel methodology was introduced for the risk management of subsea pipelines by integrating CRITIC with VIKOR. The two MCDM methods, VIKOR and WASPAS, are used in the study of [42] on a silent diesel generator according to several criteria. Overall, the main concept of VIKOR is to use a multicriteria ranking index to select the compromise solution based on its closeness to the ideal solution [43]. The ranking index in VIKOR is derived by considering both the maximum group utility and the minimum individual regret of the opponent [43]. In this manner, the closeness of each alternative to the feasible solution can be determined.

VIKOR Procedure

  • Determine the best  r j +  and the worst  r j  value for each evaluation criterion j, where  r j + = m a x ( r i j )  and  r j = m i n ( r i j ) .
  • Compute  S i , which represents the distance of the aspect to the positive ideal solution, and the value of  R i , which expresses the distance of the i aspect from the negative ideal solution, as shown in Equations (7) and (8), respectively [44].
    S i = j = 1 n w j ( r j + r i j ) / ( r j + r j )
    R i = max   [ w j ( r j + r i j ) / ( r j + r j ) ]
  • Compute the value of  Q i , which represents the rating value for each alternative. Its formulation is given by Equation (9):
    Q i = v ( S i S ) S + S + ( 1 v ) ( R i R ) R + R ,
    where  S = m i n   ( S i ) S + = m a x   ( S i ) R = m i n   ( R i ) , and  R + = m a x   ( R i ) . Furthermore, v is the weight of the maximum group utility, and (1 − v) is the weight of the minimum individual regret. The value of v is usually set to 0.5 [45].
  • Finally, rank the alternatives based on  Q i . The lower the value of  Q i , the better the alternative.

3. Methodology

The research design of this study is mainly based on the aspect extraction and ranking introduced in the study of [8], which is mainly based on subjective weighting of the criteria. The main contribution of this research is the deployment of objective weights on the ranking criteria in MCDM to allow the weighting process to be independent of the human weighting process. In particular, the CRITIC objective weighting method is integrated into the ranking domain of the product aspects to enhance the prioritization of extracted aspects in customer reviews.
Figure 1 shows the complete design of the ranking method for the product aspects. The structure of the framework is categorized into aspect extraction/retrieval and aspect ranking/prioritization. In this research, the extraction approach by [46] was employed in the aspect extraction stage. In particular, sentiment analysis techniques were used to extract the candidate product aspects from customer reviews based on two main criteria. First, the candidate aspects should be opinion-driven and discussed frequently in customer reviews. Second, these aspects should be relevant to the product domain. Simultaneous with the aspect extraction stage, MCDM was utilized in the aspect ranking stage to rank the extracted aspects based on the evaluation criteria. A decision matrix Dk was built to manage the extracted information from the customer reviews. Eventually, the decision matrix would contain the candidate product aspects that were extracted in the first stage and data on the numerical performance of these aspects according to the extraction criteria. Figure 2 presents the structure of the decision matrix. Once the decision matrix has been built, the numerical performance of the aspects should be normalized using vector normalization, as part of the VIKOR process, to be standard and comparable. After the normalization stage, the criteria weighting stage will be applied. In this research, the CRITIC method was used as part of the VIKOR approach to apply the weighting process, allowing the criteria weights to be determined mathematically, further eliminating the fuzziness of human judgments.

4. Results and Discussion

The performance evaluation of the proposed hybrid CRITIC–VIKOR technique placed a strong emphasis on evaluating the retrieved product attributes gleaned from Internet reviews. Based on the suggested strategy, this study sought to rank the most crucial product features. The performance of the suggested method was compared with the subjective VIKOR method in order to offer a thorough analysis. This made it possible to compare and contrast the results of the ranking for the product attributes that were extracted. For evaluation, the first benchmark dataset introduced by Bing Liu [5], consisting of consumer reviews of four electrical devices, was used to conduct the tests. Based on actual consumer input, these datasets were used to assess the effectiveness and performance of the suggested method. The research aimed to confirm the superiority and efficacy of the suggested hybrid CRITIC–VIKOR technique in producing precise and significant rankings of product aspects, allowing potential buyers to make informed selections. The details of the datasets are shown in Table 1.
The second benchmark dataset that was used in the evaluation is the SemEval 2016 Task 5 dataset of laptop reviews [47]. The original laptop reviews contain about 2375 sentences. Among these sentences, we focused on the sentences that have opinions, as subjective information is the focus of sentiment analysis. Accordingly, the total number of opinionated sentences in laptop reviews is 2039. The normalized discounted cumulative gain at top k (NDCG@k) measure was used in the evaluation procedure because it is well known to be a superior tool for evaluating ranking quality [48,49]. By combining graded relevance judgments, NDCG@k provides various advantages over other ranking metrics. The capability of NDCG@k to handle many levels of relevance judgments is one of its main advantages. NDCG@k takes graded relevance into account as a measure of utility, in contrast to other ranking metrics like mean average precision (MAP), which solely deal with binary relevance (either “relevant” or “irrelevant”). Because of its versatility, the ranking quality may be evaluated in a more nuanced manner, offering a truer representation of the preferences and pleasure of the user. This study intends to measure the genuine efficacy of the suggested hybrid CRITIC–VIKOR approach in rating product characteristics by using NDCG@k as the assessment metric. The usage of NDCG@k improves the accuracy and thoroughness of the evaluation, allowing for a more thorough examination of the performance of the suggested method in contrast to other ranking strategies. The measure of NDCG accumulated at a particular rank k is defined in Equation (10):
N D C G @ k = 1 Z i = 1 k 2 t ( i ) 1 log ( 1 + i ) ,
where  t ( i )   refers to the relative importance of the candidate product aspect at position i, and Z is a normalization term derived from the perfect ranking at the top-k aspects. The evaluation approach introduced by [50] was used to determine the importance of each aspect. Typically, the evaluation approach for aspect importance is based on human judgments. Thus, three annotators were invited to evaluate the importance of each aspect according to three levels of importance, namely, “unimportant”, “ordinary”, and “important,” represented numerically by “1”, “2”, and “3”, respectively.
One of the main tasks of the CRITIC method is to compute the correlation coefficient of the evaluation criteria to discover the relative importance of a corresponding criterion based on the degree of its conflict with the other criteria. Based on Equation (4) mentioned previously, Table 2, Table 3, Table 4 and Table 5 present the linear correlation coefficients for each pair of criteria used in this research.
These results showed a high correlation between the frequency- and opinion-based criteria. On the one hand, a candidate aspect that is mentioned frequently in customer reviews will most likely be highly viewed positively or negatively. On the other hand, the correlations between the aspect relevancy criterion and the other criteria are much lower because the performance scores of this criterion depend on the correlation measure between the product name and the candidate aspect in the WordNet lexicographer files. Then, on the basis of the correlation coefficients between different pairs of criteria, the objective weights of the evaluation criteria were determined using the CRITIC method for all of the product datasets used in this study (Table 6). Meanwhile, the weights  w j  of the criteria  j = 1 , , n  in the subjective VIKOR method for all products were determined on the basis of human judgments. The weight values for the criteria were as follows: frequency-based criterion =  α , opinion-based criterion =  β , and aspect relevancy criterion =  γ . Here,  α > β > γ , and  j = 1 n ,   w j = 1 .
In this study, the CRITIC method was employed, assuming that the criteria are independent of each other. As a result, the weighting approach focused on evaluating the degree of divergence in the performance scores of each criterion, disregarding the information related to other criteria. Upon conducting the analysis, it was observed that the frequency-based criterion exhibited a higher level of intrinsic information (as presented in Table 6). Consequently, this criterion received comparatively higher weight values in comparison to the remaining criteria. On the other hand, the aspect relevancy criterion obtained relatively lower weights, indicating its minimal significance in the decision-making process. These findings suggest that, within the context of this research, the aspect relevancy criterion played a less influential role in determining the ranking of product aspects. By considering the individual performance of each criterion and assigning weights accordingly, the CRITIC method facilitates a comprehensive assessment of criteria importance. This approach ensures that criteria with greater significance are assigned higher weights, thereby enabling a more precise and meaningful ranking of product aspects. For example, Table 7 compares the findings of the proposed CRITIC–VIKOR methodology with the top 15 candidate aspects for the product “Canon G3” using the subjective VIKOR method.
In Table 7, the ranked aspects from positions 1 to 5 recorded high performance ratings in all of the ranking criteria; hence, the ranking performances of the two approaches are equal until rank 6. Position 6 marked a turning point in the ranking outcomes of candidate aspects. The potential product aspect “price” was found at this position by using the CRITIC–VIKOR approach, but the candidate aspect “software” was found via subjective VIKOR. This is due to the fact that the “Aspect Relevancy” criterion’s relative weight in CRITIC–VIKOR is higher than that determined with human judgment in subjective VIKOR, which caused the aspect’s weight to be near the ideal response. Additionally, the CRITIC–VIKOR approach gave priority to genuine product aspects like “resolution,” which are not among the top 15 aspects in subjective VIKOR, giving the suggested approach an edge.
Figure 3, Figure 4 and Figure 5 show the comparative results of the product aspect rankings using the proposed hybrid CRITIC–VIKOR method and the subjective VIKOR method in terms of NDCG@5, NDCG@10, and NDCG@15. The superior performance of CRITIC–VIKOR in identifying the most relevant product aspects can be attributed to the divergence relationship established among the evaluation criteria by this hybrid method instead of focusing on complementary relationships.
The somewhat similar performance of the subjective VIKOR and CRITIC–VIKOR methods, especially for NDCG@5, can be explained by almost the same assumption used for the conflict relationships between pairs of criteria and the dominance of the frequency-based criterion in the aspect ranking process. However, the proposed CRITIC–VIKOR, which considered independence relationships during evaluation, outperformed subjective VIKOR in the other weighting criteria. On average, CRITIC–VIKOR outperformed subjective VIKOR in terms of NDCG@5, NDCG@10, and NDCG@15 by over 3.97, 20.42, and 18.32%, respectively.
In summary, the suggested hybrid CRITIC–VIKOR strategy outperforms other current approaches and highlights the significance of interdependent and complementary interactions among the evaluation criteria in product aspect ranking. The outstanding performance of CRITIC–VIKOR can be due to its capacity for locating the most pertinent product details within customer evaluations while concurrently weighing the weight of each option across all parameters. The use of objective weighing utilizing the CRITIC approach is a crucial component of CRITIC–VIKOR’s effectiveness. By doing so, the performance is guaranteed to be equivalent to subjective weighting approaches while upholding the basic MCDM tenet that each criterion is taken into account separately. The hybrid CRITIC–VIKOR approach achieves a fair and trustworthy assessment of product features by adding objective weighting based on the CRITIC method.
By considering the interconnectedness and convergence of the criteria, the hybrid CRITIC–VIKOR approach goes beyond traditional weighing approaches. Its improved performance compared to other weighting systems is a result of the thorough evaluation of relationships it takes into account. Making educated purchasing decisions is made possible for customers by the capacity to accurately and relevantly capture the complex interplay of criteria. Overall, the suggested hybrid CRITIC–VIKOR strategy leverages the interdependence of assessment criteria to offer an efficient and reliable solution for product aspect ranking. It provides a useful framework for guiding customers in choosing products that are in line with their needs and preferences by integrating objective weighting and taking both convergence and interdependence relationships into consideration.

5. Conclusions

Given the enormous amount of data available online, rating product attributes based on user reviews has become a big difficulty. Traditional methods for multiple criteria decision making (MCDM), which primarily rely on the subjective weighting of choice criteria, can be complicated and ineffective since they heavily rely on the perspectives of decision makers. In order to improve the ranking process of the product aspects mentioned in customer reviews on the Web, this study suggests a novel hybrid methodology that combines the objective weighting CRITIC method with the MCDM VIKOR method. The suggested technique aims to lessen the restrictions associated with subjective assessments by introducing objective weighting, which is mostly dependent on the performance of the criteria.
The hybrid CRITIC–VIKOR technique has been shown to perform better in empirical assessments and comparisons with the subjective VIKOR method in terms of ranking the extracted product aspects to prioritize the most influential aspects that have a gear impact on the customers’ decisions. When objective weighting is used, the assessment is more impartial and objective, allowing buyers to make defensible choices based on the relative weighting of various product attributes. The hybrid CRITIC–VIKOR strategy is successful at handling the difficulties involved in ranking product characteristics because it makes extensive use of decision information and incorporates objective weighting. This strategy improves the validity and usefulness of the rankings produced from customer evaluations by decreasing the reliance on subjective judgments and adopting a more methodical approach to criteria weighting.
This study, despite its merits, has a shortcoming that point the way for further investigation. Only reviews of electronic devices were used to build the suggested model, but future research could expand its application by looking at additional product categories. This research motivates future work to utilize other MCDM approaches to prioritize important product aspects like COPRAS, MOORA, ARAS, and other methods. These methods could be used and evaluated to highlight the pros and cons of each method in ranking the product aspects. Moreover, the proposed approach, CRITIC–VIKOR, considers the independent relationships among the criteria in the ranking process, which motivates more future work in applying weighting approaches that consider the complementary relationships among the criteria.

Author Contributions

Methodology, S.A.A.A.; Software, S.A.A.A.; Writing—original draft, S.A.A.A.; Writing—review & editing, K.H.G.; Supervision, K.H.G. All authors have read and agreed to the published version of the manuscript.


This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

In this study, we analyzed the publicly archived dataset of Bing Liu entitled “Customer Review Datasets (5 products)” available on the following link: (accessed on 5 July 2023).

Conflicts of Interest

The authors declare no conflict of interest.


  1. Aghakhani, N.; Oh, O.; Gregg, D.G.; Karimi, J. Online Review Consistency Matters: An Elaboration Likelihood Model Perspective. Inf. Syst. Front. 2021, 23, 1287–1301. [Google Scholar] [CrossRef]
  2. Bressmann, T. Self-inflicted cosmetic tongue split: A case report. J. Can. Dent. Assoc. 2004, 70, 156–157. [Google Scholar] [PubMed]
  3. Liu, B. Sentiment analysis and opinion mining. In Synthesis Lectures on Human Language Technologies; Springer: Berlin/Heidelberg, Germany, 2012; Volume 5, pp. 1–167. [Google Scholar]
  4. D’Acunto, D.; Tuan, A.; Dalli, D.; Viglia, G.; Okumus, F. Do consumers care about CSR in their online reviews? An empirical analysis. Int. J. Hosp. Manag. 2020, 85, 102342. [Google Scholar] [CrossRef]
  5. Hu, M.; Liu, B. Mining Opinion Features in Customer Reviews. In Proceedings of the 19th National Conference on Artificial Intelligence, San Jose, CA, USA, 25–29 July 2004; Volume 4, pp. 755–760. [Google Scholar] [CrossRef] [Green Version]
  6. Popescu, A.M.; Etzioni, O. Extracting product features and opinions from reviews. In Natural Language Processing and Text Mining; Springer: Berlin/Heidelberg, Germany, 2007; pp. 9–28. [Google Scholar] [CrossRef]
  7. Quan, C.; Ren, F. Unsupervised product feature extraction for feature-oriented opinion determination. Inf. Sci. 2014, 272, 16–28. [Google Scholar] [CrossRef]
  8. Alrababah, S.; Gan, K.; Tan, T.-P. Product aspect ranking using sentiment analysis and TOPSIS. In Proceedings of the 3rd International Conference on Information Retrieval and Knowledge Management Product, Malacca, Malaysia, 23–24 August 2016; pp. 124–128. [Google Scholar]
  9. Hwang, C.-L.; Yoon, K. Multiple Attribute Decision Making; Springer: Berlin/Heidelberg, Germany, 1981. [Google Scholar]
  10. Trung, D.D.; Thinh, H.X. A multi-criteria decision-making in turning process using the MAIRCA, EAMR, MARCOS and TOPSIS methods: A comparative study. Adv. Prod. Eng. Manag. 2021, 16, 443–456. [Google Scholar] [CrossRef]
  11. Wang, T.C.; Lee, H.D. Developing a fuzzy TOPSIS approach based on subjective weights and objective weights. Expert Syst. Appl. 2009, 36, 8980–8985. [Google Scholar] [CrossRef]
  12. Ali, N.H.; Sabri, I.A.; Noor, N.M.; Ismail, F. Rating and Ranking Criteria for Selected Islands using Fuzzy Analytic Hierarchy Process (FAHP). Int. J. Appl. Math. Inform. 2012, 6, 57–65. [Google Scholar]
  13. Jato-Espino, D.; Castillo-Lopez, E.; Rodriguez-Hernandez, J.; Canteras-Jordana, J.C. A review of application of multi-criteria decision making methods in construction. Autom. Constr. 2014, 45, 151–162. [Google Scholar] [CrossRef]
  14. Sotoudeh-Anvari, A. The applications of MCDM methods in COVID-19 pandemic: A state of the art review. Appl. Soft Comput. 2022, 126, 109238. [Google Scholar] [CrossRef]
  15. Roszkowska, E. Rank ordering criteria weighting methods—A comparative overview 2. Optim. Stud. Ekon. 2013, 5, 14–33. [Google Scholar] [CrossRef]
  16. Wang, J.J.; Jing, Y.Y.; Zhang, C.F.; Zhao, J.H. Review on multi-criteria decision analysis aid in sustainable energy decision-making. Renew. Sustain. Energy Rev. 2009, 13, 2263–2278. [Google Scholar] [CrossRef]
  17. Edwards, W.; Barron, F.H. SMARTES and SMARTER: Improved simple methods for multiattribute utility measurement. Organ. Behav. Hum. Decis. Process. 1994, 60, 306–325. [Google Scholar] [CrossRef]
  18. Bottomley, P.A.; Doyle, J.R. A comparison of three weight elicitation methods: Good, better, and best. Omega 2001, 29, 553–560. [Google Scholar] [CrossRef]
  19. Saaty, T.L. Decision making with the analytic hierarchy process. Int. J. Serv. Sci. 2008, 1, 83. [Google Scholar] [CrossRef] [Green Version]
  20. Saeed, A.; Moghadam, K. Comparison of AHP and FAHP for Selecting Yard Gantry Cranes in Marine Container Terminals. J. Persian Gulf 2012, 3, 59–70. [Google Scholar]
  21. Kabir, G.; Sadiq, R.; Tesfamariam, S. A review of multi-criteria decision-making methods for infrastructure management. Struct. Infrastruct. Eng. 2014, 10, 1176–1210. [Google Scholar] [CrossRef]
  22. Saaty, T.L. Fundamentals of the analytic network process. In Proceedings of the ISAHP 1999, Kobe, Japan, 12–14 August 1999; pp. 1–14. [Google Scholar]
  23. Jahan, A.; Mustapha, F.; Sapuan, S.M.; Ismail, M.Y.; Bahraminasab, M. A framework for weighting of criteria in ranking stage of material selection process. Int. J. Adv. Manuf. Technol. 2012, 58, 411–420. [Google Scholar] [CrossRef]
  24. Kelemenis, A.; Askounis, D. A new TOPSIS-based multi-criteria approach to personnel selection. Expert Syst. Appl. 2010, 37, 4999–5008. [Google Scholar] [CrossRef]
  25. Mohammadi, A.; Mohammadi, A.; Aryaeefar, H. Introducing a new method to expand TOPSIS decision making model to fuzzy TOPSIS. J. Math. Comput. Sci. 2011, 2, 150–159. [Google Scholar] [CrossRef] [Green Version]
  26. Chang, T.; Wang, T. Using the fuzzy multi-criteria decision making approach for measuring the possibility of successful knowledge management. Inf. Sci. 2009, 179, 355–370. [Google Scholar] [CrossRef]
  27. Chen, C.-T. Extensions of the TOPSIS for group decision-making under fuzzy environment. Fuzzy Sets Syst. 2000, 114, 1–9. [Google Scholar] [CrossRef]
  28. Nădăban, S.; Dzitac, S.; Dzitac, I. Fuzzy TOPSIS: A General View. Procedia Comput. Sci. 2016, 91, 823–831. [Google Scholar] [CrossRef] [Green Version]
  29. Jee, D.-H.; Kang, K.-J. A method for optimal material selection aided with decision making theory. Mater. Des. 2000, 21, 199–206. [Google Scholar] [CrossRef]
  30. Diakoulaki, D.; Mavrotas, G.; Papayannakis, L. Determining objective weights in multiple criteria problems: The critic method. Comput. Oper. Res. 1995, 22, 763–770. [Google Scholar] [CrossRef]
  31. Maniya, K.; Bhatt, M.G. A selection of material using a novel type decision-making method: Preference selection index method. Mater. Des. 2010, 31, 1785–1789. [Google Scholar] [CrossRef]
  32. Albahri, O.S.; Al-Obaidi, J.R.; Zaidan, A.A.; Albahri, A.S.; Zaidan, B.B.; Salih, M.M.; Qays, A.; Dawood, K.A.; Mohammed, R.T.; Abdulkareem, K.H.; et al. Helping doctors hasten COVID-19 treatment: Towards a rescue framework for the transfusion of best convalescent plasma to the most critical patients based on biological requirements via ml and novel MCDM methods. Comput. Methods Programs Biomed. 2020, 196, 105617. [Google Scholar] [CrossRef]
  33. Cui, Z.X.; Yoo, H.K.; Choi, J.Y.; Youn, H.Y. Multi-criteria group decision making with fuzzy logic and entropy based weighting. In Proceedings of the 5th International Conference on Ubiquitous Information Management and Communication—ICUIMC ’11, Seoul, Republic of Korea, 21–23 February 2011; pp. 1–70. [Google Scholar] [CrossRef]
  34. Abdullah, L.; Otheman, A. A New Entropy Weight for Sub-Criteria in Interval Type-2 Fuzzy TOPSIS and Its Application. Int. J. Intell. Syst. Appl. 2013, 5, 25–33. [Google Scholar] [CrossRef] [Green Version]
  35. Akram, M.; Ramzan, N.; Deveci, M. Linguistic Pythagorean fuzzy CRITIC-EDAS method for multiple-attribute group decision analysis. Eng. Appl. Artif. Intell. 2023, 119, 105777. [Google Scholar] [CrossRef]
  36. Yalçın, N.; Ünlü, U. A multi-criteria performance analysis of initial public offering (IPO) firms using critic and vikor methods. Technol. Econ. Dev. Econ. 2018, 24, 534–560. [Google Scholar] [CrossRef]
  37. Peng, X.; Huang, H. Fuzzy decision making method based on cocoso with critic for financial risk evaluation. Technol. Econ. Dev. Econ. 2020, 26, 695–724. [Google Scholar] [CrossRef] [Green Version]
  38. Peng, X.; Zhang, X.; Luo, Z. Pythagorean fuzzy MCDM method based on CoCoSo and CRITIC with score function for 5G industry evaluation. Artif. Intell. Rev. 2020, 53, 3813–3847. [Google Scholar] [CrossRef]
  39. Opricovic, S. Multi-Criteria Optimization of Civil Engineering Systems. Ph.D. Thesis, Faculty of Civil Engineering, Belgrade, Serbia, 1998. [Google Scholar]
  40. Magableh, G.M. Evaluating Wheat Suppliers Using Fuzzy MCDM Technique. Sustainability 2023, 15, 10519. [Google Scholar] [CrossRef]
  41. Li, X.; Han, Z.; Yazdi, M.; Chen, G. A CRITIC–VIKOR based robust approach to support risk management of subsea pipelines. Appl. Ocean Res. 2022, 124, 103187. [Google Scholar] [CrossRef]
  42. Vaid, S.K.; Vaid, G.; Kaur, S.; Kumar, R.; Sidhu, M.S. Application of multi-criteria decision-making theory with VIKOR-WASPAS-Entropy methods: A case study of silent Genset. Mater. Today Proc. 2021, 50, 2416–2423. [Google Scholar] [CrossRef]
  43. Jati, H. Comparison of University Webometrics Ranking Using Multicriteria Decision Analysis: TOPSIS and VIKOR Method. Word J. Int. Linguist. Assoc. 2012, 71, 1663–1669. [Google Scholar]
  44. Baghla, S.; Bansal, S. Effect of normalization techniques in VIKOR method for network selection in heterogeneous networks. In Proceedings of the 2014 IEEE International Conference on Computational Intelligence and Computing Research, IEEE ICCIC 2014, Coimbatore, India, 18–20 December 2014. [Google Scholar] [CrossRef]
  45. Akram, M.; Muhiuddin, G.; Santos-Garcia, G. An enhanced VIKOR method for multi-criteria group decision-making with complex Fermatean fuzzy sets. Math. Biosci. Eng. 2022, 19, 7201–7231. [Google Scholar] [CrossRef] [PubMed]
  46. Alrababah, S.; Gan, K.; Tan, T.-P. Domain-independent approach for mining opinionated product features using WordNet lexicographer files. J. Inf. Sci. 2017, 43, 769–785. [Google Scholar] [CrossRef]
  47. Pontiki, M.; Galanis, D.; Papageorgiou, H.; Androutsopoulos, I.; Manandhar, S.; Al-Smadi, M.; Al-Ayyoub, M.; Zhao, Y.; Qin, B.; De Clercq, O.; et al. SemEval-2016 Task 5: Aspect Based Sentiment Analysis. In ProWorkshop on Semantic Evaluation (SemEval-2016); Association for Computational Linguistics: Stroudsburg, PA, USA, 2016; pp. 19–30. [Google Scholar]
  48. Järvelin, K.; Kekäläinen, J. Cumulated gain-based evaluation of IR techniques. ACM Trans. Inf. Syst. 2002, 20, 422–446. [Google Scholar] [CrossRef]
  49. Wang, Y.; Wang, L.; Li, Y.; He, D.; Chen, W.; Liu, T.-Y. A Theoretical Analysis of NDCG Ranking Measures. In Proceedings of the 26th Annual Conference on Learning Theory, Princeton, NJ, USA, 12–14 June 2013; pp. 1–30. [Google Scholar]
  50. Zha, Z.J.; Yu, J.; Tang, J.; Wang, M.; Chua, T.S. Product aspect ranking and its applications. IEEE Trans. Knowl. Data Eng. 2014, 26, 1211–1224. [Google Scholar] [CrossRef]
Figure 1. Product aspect ranking in the hybrid CRITIC–VIKOR.
Figure 1. Product aspect ranking in the hybrid CRITIC–VIKOR.
Applsci 13 09176 g001
Figure 2. Structure of the decision matrix.
Figure 2. Structure of the decision matrix.
Applsci 13 09176 g002
Figure 3. Performance of aspect ranking in terms of NDCG@5.
Figure 3. Performance of aspect ranking in terms of NDCG@5.
Applsci 13 09176 g003
Figure 4. Performance of aspect ranking in terms of NDCG@10.
Figure 4. Performance of aspect ranking in terms of NDCG@10.
Applsci 13 09176 g004
Figure 5. Performance of aspect ranking in terms of NDCG@15.
Figure 5. Performance of aspect ranking in terms of NDCG@15.
Applsci 13 09176 g005
Table 1. Descriptions of the review datasets.
Table 1. Descriptions of the review datasets.
Product DescriptionTotal Review SentencesTotal Opinionated Aspects
Digital camera 1: Nikon Coolpix 430014859
Digital camera 2: Canon G317269
Cell phone: Nokia 661026176
Mp3 player: Creative Labs Nomad Jukebox Zen Xtra 40 GB721117
Table 2. Correlation coefficients for each pair of criteria in the Canon G3 reviews.
Table 2. Correlation coefficients for each pair of criteria in the Canon G3 reviews.
Frequency-BasedOpinion-BasedAspect Relevancy
Aspect Relevancy0.300.391
Table 3. Correlation coefficients for each pair of criteria in the Nikon Coolpix 4300 reviews.
Table 3. Correlation coefficients for each pair of criteria in the Nikon Coolpix 4300 reviews.
Frequency-BasedOpinion-BasedAspect Relevancy
Aspect Relevancy0.400.391
Table 4. Correlation coefficients for each pair of criteria in the Nokia 6610 reviews.
Table 4. Correlation coefficients for each pair of criteria in the Nokia 6610 reviews.
Frequency-BasedOpinion-BasedAspect Relevancy
Table 5. Correlation coefficients for each pair of criteria in the Creative Labs Nomad Jukebox Zen Xtra 40 GB reviews.
Table 5. Correlation coefficients for each pair of criteria in the Creative Labs Nomad Jukebox Zen Xtra 40 GB reviews.
Frequency-BasedOpinion-BasedAspect Relevancy
Aspect Relevancy0. 380.311
Table 6. Objective weights of the evaluation criteria in the CRITIC method.
Table 6. Objective weights of the evaluation criteria in the CRITIC method.
ProductsRanking Criteria
Frequency-BasedOpinion-BasedAspect Relevancy
Nikon Coolpix 43000.590.320.09
Canon G30.690.190.12
Nokia 66100.620.270.11
Creative Labs Nomad Jukebox Zen Xtra 40 GB0.620.310.07
Table 7. The top 15 aspects of the “Canon G3” product as determined by Subjective VIKOR and CRITIC–VIKOR.
Table 7. The top 15 aspects of the “Canon G3” product as determined by Subjective VIKOR and CRITIC–VIKOR.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Alrababah, S.A.A.; Gan, K.H. Effects of the Hybrid CRITIC–VIKOR Method on Product Aspect Ranking in Customer Reviews. Appl. Sci. 2023, 13, 9176.

AMA Style

Alrababah SAA, Gan KH. Effects of the Hybrid CRITIC–VIKOR Method on Product Aspect Ranking in Customer Reviews. Applied Sciences. 2023; 13(16):9176.

Chicago/Turabian Style

Alrababah, Saif Addeen Ahmad, and Keng Hoon Gan. 2023. "Effects of the Hybrid CRITIC–VIKOR Method on Product Aspect Ranking in Customer Reviews" Applied Sciences 13, no. 16: 9176.

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop