Explaining the Paradox of World University Rankings in China: Higher Education Sustainability Analysis with Sentiment Analysis and LDA Topic Modeling

Abstract

The development of the World University Rankings (WURs) has produced the following paradox. On the one hand, the WURs are often criticized for their ranking methodology and logic. On the other hand, the WURs are growing in influence worldwide. Universities are caught in a vicious cycle of pursuing indicators, which has a negative impact on the sustainability of higher education. In order to explain the development paradox of WURs, this research uses sentiment analysis and Latent Dirichlet Allocation (LDA) topic modeling to understand how the WURs thrive amid controversy by analyzing the emotion and cognition in 18,466 Chinese public comments on the WURs. The research found that (1) although the Chinese public has mixed feelings about the WURs, the overall sentiment is positive; (2) the Chinese public perceives the WURs through four main perspectives: standpoint cognition, dialectical cognition, interest cognition, and cultural cognition; and (3) the public is more concerned about whether their standpoints are met, whether their interests are reflected, and whether their individual experiences are verified but rarely think about the problems of ranking from a dialectical perspective. The need for ranking has always existed but the issue of ranking has often been ignored, leading to the development paradox of rankings.

1. Introduction

The pressures of globalization have forced higher education to take part in the marketplace [1], knowledge has become a commodity with geopolitical implications [2], and countries have positioned the development of universities as a means of state-building and enhancing national global competitiveness [3]. In this context, a clear perception of the international level and relative position of universities is urgently needed. The WURs demystify universities in a simple and accessible way, attempting to explain what makes a world-class university through a series of metrics and giving the results of the WURs based on the metrics [4]. The understanding of what constitutes a world-class university, as reflected in the WURs, is gradually being recognized by governments, university policymakers, businesses, students, and parents. In particular, the influential WURs, such as Ranking’s Academic Ranking of World Universities (ARWU), QS World University Rankings (QS-WUR), Times Higher Education World University Rankings (THE-WUR), and U.S. News & World Report Best Global Universities Rankings (U.S. News-GUR), have become a reference point for governments to assess the effectiveness of universities, for university policymakers to formulate university-development plans, for businesses to select partner institutions, and for students and parents to choose a university [5]. The WURs reinforce ordinal comparative relationships in space and time, making universities, as resource-dependent entities, inevitably caught up in national and international competition [6]. More and more higher-education institutions (HEIs) are using the WURs to measure the achievement of “world-class status” in their mission statements and for the development of the long-term strategic goals in the world’s leading universities [7]. In a sense, the WURs have awakened international discussion on higher education, fostered competition among global universities [8], promoted the development of research universities, and provided stakeholders with a general reference standard for assessment. However, with the widespread use of the WURs as a tool [9] and criterion, universities are also at risk of conscious or unconscious isomorphism. University policymakers are constantly adapting to the rules of the game of WURs by reshaping priorities to fit the university-ranking criteria. The “academic arms race” of universities to achieve higher rankings has led to a high turnover of talent, reinforcing the “Matthew Effect.” Although the WURs exert their influence and appeal, they also cast a shadow over the higher-education system [10]. Universities are internally trapped in a vicious cycle of pursuing progress in their indicators, and stakeholders are finding that the original intention of using the WURs as an external safeguard mechanism to promote good university development has been diverted. As the WURs place more emphasis on research and reputation indicators, the quality of teaching is replaced by simple indicators such as the student–teacher ratio. This leads to a certain degree of neglect of the quality of education, which affects the experience of university students in the process of receiving higher education. This impact is not only limited to the higher-education system but may also have some negative consequences for the economy. The importance of economic growth cannot be ignored [11], and some studies have shown that the contribution of education quality to economic growth increases with the level of education quality [12] and that imbalances in education quality may affect economic growth. The unintended consequences of the continued embedding of ranking discourse in the higher-education system need more attention.

The development paradox of WURs has taken shape. On the one hand, a growing number of researchers believes that the WURs are biased in terms of ranking values and are somewhat flawed in their methodology and that the unintended negative consequences of the WURs for the higher-education system even outweigh their potential value [13,14,15,16,17]. On the other hand, the WURs are still widely used as a tool and continue to exert influence on the higher-education system [18,19,20,21]. Many scholars believe that the creation of this paradox may be related to the new public management and neoliberal reforms in the field of higher education [22,23,24,25]. With the transition of higher-education institutions to a market model and the commercialization of knowledge production, universities are being asked to respond to the public demand for disclosure of their performance [26]. The audit culture under neoliberalism, which is mainly characterized by accountability and the pursuit of transparency and efficiency, has gradually permeated the higher-education system, and university rankings as a manifestation of the audit culture have become an important tool to meet public concerns about higher education [27]. The WURs establish the legitimacy of the rankings by claiming that “rankings are inevitable,” “they are needed by the public,” and “they reflect reality” [28]. These claims may seem reasonable and in line with objective reality, but it is an evasive answer. In fact, the role of rankings in understanding the higher-education system is undisputed. What is controversial is the ranking methodology, logic, and values used in the rankings, which are instead ignored in the claims made by the World University Ranking agencies. As the rankers have not responded to the public’s need for more nuanced rankings, the public has been forced to compromise and continue to refer to the controversial rankings to aid decision-making. This compromise also deepens the development paradox of ranking. In China, at the end of the 20th century, the Chinese government proposed to build several world-class and advanced-level universities. To analyze the position of Chinese universities in the global higher-education system, the Institute of Higher Education of Shanghai Jiao Tong University published the ARWU in 2003. Next, other national institutions published their World University Rankings one after another. Quacquarelli Symonds (QS) and Times Higher Education (THE) collaborated to publish the THE-QS World University Rankings in 2004, but the partnership ended in 2010. QS published the QS-WUR separately, and THE published the THE-WUR separately. U.S. News & World Report (U.S. News) published U.S. News-GUR in 2014 [29]. Along with the emergence of several WURs, these rankings once provided a reference for Chinese policymakers to assess the level of universities in the world. Many university leaders are also making development plans based on the list. The deepening influence of the WURs, to a certain extent, has guided the direction of university development in China. However, with the continuous penetration of the ranking discourse into the Chinese higher-education system, unsuitable evaluation systems have been formed in many universities, such as the excessive pursuit of article-publication targets in the introduction of teachers or the evaluation of projects. Policymakers are also aware of the negative impact on the sustainability of higher education in the context of the development paradox of WURs. The Ministry of Education and the Ministry of Science and Technology of China jointly issued the “Opinions on Regulating the Use of SCI Paper-related Indicators in Higher Education Institutions and Establishing Correct Evaluation Guidance” in 2020. In this policy, universities, university-competent departments, and subordinate institutions are required not to publish rankings of SCI papers and ESI-related indicators and not to adopt or spread rankings of other institutions with SCI papers and ESI as core indicators. The policy attempts to counteract the negative consequences of the WURs, reverse the current chaos of Chinese universities blindly pursuing index progress, and create a good and sustainable environment for higher education [30]. However, the WURs are still widely reported by individuals, companies, news media, and official accounts. The Chinese public continues to be concerned about the WURs, despite their method and logic being controversial. The development paradox of WURs still exists in China. Previous research explaining the development paradox of WURs has mostly been analyzed in the institutional context of new public management and neoliberal reforms, arguing that this paradox is the result of a compromise in the context of the times and the institution. The responses of organizations or units such as governments, businesses, and universities to such isomorphic pressures are often discussed, and the public as a broader audience of the WURs is rarely discussed. Therefore, it is necessary to explain the development paradox by analyzing the Chinese public’s emotions and perceptions of the WURs, to clarify why the WURs can flourish amid controversies, and to provide some factual basis for higher-education policymakers to formulate policies to deal with the negative effects of the WURs. Some questions that need to be answered:

(1) How does the public perceive the WURs?

(2) Why does the public ignore the shortcomings of the WURs and maintain a positive attitude toward them?

(3) Is it only because the theories, methods, and logic of the rankings are unreasonable that the public has negative attitudes toward them?

(4) Which part of the rankings do people who are objective about the rankings tend to pay more attention to?

(5) How do the WURs continue to influence the higher-education system in the process of being disputed by the public?

China has the largest higher-education system in the world [31] and the Chinese higher-education system has been influenced by the WURs for about 20 years. In addition, compared to developed countries such as the United States, China has a smaller number of world-class universities. In 2015, the State Council issued the General Plan for the Coordinated Promotion of the Construction of World-Class Universities and First-Class Disciplines, which made new arrangements for the construction of higher education in China in the new era. The plan calls for accelerating the establishment of some world-class universities and first-class disciplines. In this context, the Chinese public maintains higher expectations for the country’s higher-education construction. The WURs are often discussed by the Chinese public because they provide a visual representation of a university’s relative position in the international arena. Therefore, it is representative to explain the development paradox by studying the Chinese public’s sentiment and perception of the WURs. Emotions and cognition are internalized mental representations of the individual, and after external stimuli, the individual’s emotional and cognitive units are activated to varying degrees, which then influence the individual’s behavior [32]. Commentary is one of the expressions of individual behavior. The tone and attitude of commentary texts can express the emotional information output by the individual after the interaction between the stimulus source and the internal psychological system. The angle of entry and the way of thinking of the text can express the underlying cognitive processes of the individual in the face of external stimuli. In order to answer the above questions, this paper attempts to further explain the development paradoxes by analyzing the texts of Chinese public comments on the WURs and mining the public’s cognitive performance under different emotions. Firstly, Python code was written to crawl and filter the comments to get the initial corpus. Secondly, the public comments were divided into positive, objective, and negative parts based on the sentiment probability obtained from sentiment analysis. In this section, it is possible to analyze the overall sentiment of the public towards the WURs. Finally, the topics under different emotions were mined based on LDA topic modeling, and the public’s cognitive perspectives on the WURs were further summarized and extracted. After combining the performance of public perceptions under different emotions, the development paradox of WURs is explained.

2. Materials and Methods

2.1. Data Sources

In this study, we wrote crawler code using Python 3.8 to crawl public comments on information reports related to the four internationally renowned World University Rankings (ARWU, QS-WUR, THE-WUR, U.S. News-GUR) in Jinritoutiao for the period from 2018 to 2022. ARWU focuses on using research indicators to rank universities’ academic standards, QS-WUR places a strong emphasis on academic reputation and employer reputation, THE-WUR places more emphasis on research impact and university reputation, and U.S. News-GUR places more emphasis on research reputation and indicators related to research impact. Despite the different focus of these rankings, there is a commonality in that they are all ranked based on internationally comparable and size-comparable metrics, and the results are made available to the public. Such commonality makes the rankings accessible to a wider audience and is the basis for collecting more research data. Then, these collected comments were filtered and culled, and a total of 18,466 comment texts were obtained. We selected the above four WURs as representatives to explore public sentiment and cognition based on the following two principles. Principle one is continuity—the list of WURs must be published continuously for many years. Principle two is authority—the selected WURs must have high recognition and strong influence in the international arena. We chose to capture the comment information of Jinritoutiao instead of Weibo and Zhihu, which are commonly used for knowledge-network analysis in China, for the following two reasons. The first reason is that Weibo is more suitable for discussion of hot topics, in which the public’s comments on the WURs are mostly focused on the month in which the list is released, and it is difficult to reflect the public’s emotions and cognitions in other months; Zhihu is more focused on the interaction of questions, and the public’s comments are limited by the questions, so it is impossible to observe the public’s more diverse cognitions. The second reason is that Jinritoutiao, as one of the top 10 media websites in China [33], has a large user base and provides a search-engine service that allows the public to retrieve relevant ranking lists and participate in interactions at any time.

2.2. Research Methods

This study mainly used sentiment analysis and LDA topic modeling in text data-mining technique to extract relevant knowledge from the unstructured information of 18,466 comments to meet the purpose of the research. The specific research process is shown in Figure 1.

2.2.1. Sentiment Analysis

Sentiment analysis is often used to analyze subjective texts with emotional overtones, to uncover the emotional tendencies embedded in that text. This study imported SnowNLP in Python to perform sentiment analysis on public comments. SnowNLP is a natural-language-processing database in Python that is often used to determine the sentiment of Chinese comments because it can process Chinese text content at high speed and has a database that has been trained and can be used directly. The sentiment analysis used by SnowNLP is based on the theory of the Naive Bayes classification algorithm. The core formula is shown in Formula (1):

$$\mathrm{P}\left(\mathrm{C}|\mathrm{W}\right)=\frac{\mathrm{P}\left(\mathrm{W}|\mathrm{C}\right)\mathrm{P}\left(\mathrm{C}\right)}{\mathrm{P}\left(\mathrm{W}\right)}$$

(1)

P(C) is the a priori probability obtained by computing it on the already-labeled training data. P(W|C) is the probability of the occurrence of a word in different sentiment categories. P(W) is the probability of occurrence of each word. P(C|W) is the posterior probability used to determine the sentiment classification to which the word belongs.

Emotional probabilities obtained from sentiment-tendency judgments on text using SnowNLP are in [0,1]. The closer the sentiment probability is to 0, the higher the probability that the comment is a negative sentiment; the closer the sentiment probability is to 1, the higher the probability that the comment is a positive sentiment. This research identified sentiment-probability values of [0,0.4) as negative comments, sentiment-probability values of [0.4,0.6] as objective comments, and sentiment-probability values of (0.6,1] as positive comments. Further visualization of the sentiment probability of public comments was carried out to facilitate an understanding of the Chinese public’s sentiment toward the WURs.

2.2.2. LDA Topic Modeling

The research of Chinese public cognitions of the WURs was explored using the LDA topic-modeling analysis method of the text-mining technique. LDA topic modeling is an unsupervised machine-learning technique that can be used to mine large collections of documents for implied topic information [34]. The core formula is shown in Formula (2):

$$\mathrm{P}\left({\mathrm{w}}_{\mathrm{j}}{|\mathrm{D}}_{\mathrm{i}}\right)={\displaystyle \sum }_{\mathrm{k}=1}^{\mathrm{K}}\mathrm{P}\left({\mathrm{w}}_{\mathrm{j}}{|\mathrm{T}}_{\mathrm{k}}\right)\mathrm{P}\left({\mathrm{T}}_{\mathrm{k}}{|\mathrm{D}}_{\mathrm{i}}\right)$$

(2)

LDA topic modeling contains three levels: document, topic, and word. The method can extract the topic model from the document and reflect the potential topics of the document through the probability distribution of the words. $\mathrm{P}\left({\mathrm{w}}_{\mathrm{j}}{|\mathrm{D}}_{\mathrm{i}}\right)$ denotes the probability of feature word ${\mathrm{w}}_{\mathrm{j}}$ appearing in document ${\mathrm{D}}_{\mathrm{i}}$. This probability value is the product of the probability of the feature word and the probability of the topic-feature word, i.e., the product of the probability of ${\mathrm{w}}_{\mathrm{j}}$ appearing in topic ${\mathrm{T}}_{\mathrm{k}}$ and the probability of topic ${\mathrm{T}}_{\mathrm{k}}$ appearing in document ${\mathrm{D}}_{\mathrm{i}}$. K is the number of topics.

The score of topic coherence determines whether the words corresponding to a high probability under the generated topic are semantically consistent. It is often used as an indicator to determine the optimal number of topics for the LDA topic model [35]. The core formula is shown in Formula (3):

$$\mathrm{C}\left(\mathrm{W}\right)={\displaystyle \sum }_{{(\mathrm{w}}_{\mathrm{m}}{,\mathrm{w}}_{\mathrm{n}})\in \mathrm{W}}\mathrm{score}\left({\mathrm{w}}_{\mathrm{m}}{,\mathrm{w}}_{\mathrm{n}},\mathsf{ϵ}\right)$$

(3)

W is a series of characteristic words under a topic; ${\mathrm{w}}_{\mathrm{m}}$ and ${\mathrm{w}}_{\mathrm{n}}$ are the feature words within the topic; $\mathsf{ϵ}$ is the smoothing factor; and $\mathrm{score} \left(\right)$ is a function that measures the degree of semantic closeness of a feature word within a topic. A higher score for topic coherence means that the model is more effective.

In this study, LDA topic-modeling analysis was conducted on a corpus containing different sentiment information. Firstly, the number of topics implied in the comments was determined by the score of topic coherence. Secondly, the number of topics was selected and the LDA topic-modeling results were obtained. Thirdly, social public-comment themes were identified through meaning construction on the generated topics and the characteristic words under each topic. Finally, themes and sentiment corpus were combined to analyze how the public perceives the WURs in different emotions, and the kind of cognitions that influence the public’s behavior and attitude leading to the paradox of the development of WURs was further analyzed.

3. Results

3.1. Emotional-Analysis Results

This study analyzed 18,466 texts of Chinese public comments on the WURs by SnowNLP. The results show that there were 10,286 comments with sentiment-probability values of (0.6, 1]. Among these comments, the public had a more strongly positive sentiment towards the WURs, such as “excellent,” appearing 944 times; “fame follows merit,” appearing 325 times; and “accurate,” appearing 184 times. There were 3006 comments with sentiment-probability values of [0.4,0.6]. In these comments, the public sentiment towards the WURs was more moderate, although there was a tendency to shift towards positive or negative attitudes in the comments. However, the WURs were still evaluated from an objective perspective, with few words of strong emotion. There were 5174 comments with sentiment-probability values of [0,0.4). In these comment texts, the public had a relatively strong negative sentiment towards the WURs, such as “untrustworthy,” appearing 262 times; “fool,” appearing 176 times; and “fabricate,” appearing 105 times.

In order to analyze the change in comment frequency under different sentiment probabilities, we set the start value of sentiment probability to 0, the stop value to 1, and the step value to 0.01 and counted the frequency of comment text for every 0.01 increase in sentiment-probability value starting from 0. The visualization results are shown in Figure 2. It was found that at a step size of 0.01, there were comments greater than 100 under each sentiment value, which indicates that the public attitudes towards the WURs were varied. Furthermore, when combining the number of positive, objective, and negative comments, as well as the number of comments higher than 200 for every 0.01 increase in sentiment value after a sentiment-probability value of 0.9, it was observed that the public maintained a positive sentiment toward the WURs in general.

3.2. LDA Topic-Modeling Results

3.2.1. Public Cognitive Perspective on WURs under the Positive Emotion

The topic coherence of the positive-comment word database was calculated for different numbers of topics and the topic-coherence curve was obtained, as shown in Figure 3. It was found that the trend in topic-consistency score changed from a sharp increase at the beginning to a leveling-off. The higher the topic-consistency score, the better the LDA modeling fit. However, to avoid the negative effects of over-fitting for the study, considering that when the number of topics was 24 there was also a relatively high topic-consistency score, the number of topics was determined to be 24.

Combined with the positive corpus, the output LDA topic-modeling results were subjected to meaning construction to obtain

Table 1. LDA topic-modeling results on positive comments.

Cognitive Perspective	Theme	Topic Number and Topic Feature Words
Standpoint cognition	National position	Topic 1: school, undergraduate, ranking, famous university, Beijing
		Topic 3: first, world, ranking, Asia, Chinese universities
		Topic 7: Tsinghua University and Peking University (THU and PKU), transcend, come on, elite, proud
		Topic 11: education, development, China, identify, science
		Topic 16: top, place, 100, promote, list
		Topic 18: THU, PKU, University of Science and Technology of China (USTC), excellent, future
		Topic 23: fame follows merit, Harbin Institute of Technology (HIT), excellent, Tongji University (Tongji), Wuhan University (WHU)
Dialectical cognition	Ranking indicator	Topic 2: student, international, Western, overseas student, geographical situation
		Topic 14: ranking, country, learning, contribution, standard
	University value	Topic 4: talent, train, Britain, America, THU and PKU
	University development	Topic 6: field, effort, disparity, scientific research, C9
		Topic 15: subject, not bad, building, system, performance
		Topic 21: science and engineering, social sciences, influence, train talent, stress
	Ranking result	Topic 8: major, institute, excellent, Massachusetts Institute of Technology (MIT), developed
		Topic 12: accurate, world, understand, total, ranking list
		Topic 19: Russia, Japan, most, comprehensiveness, learn
		Topic 20: internal, research, level, ranking, scientific research
	National development	Topic 22: China, university, science and technology, quality, global
	Objective reality	Topic 24: achievement, graduation, enroll, grade cutoff point, deserve
Interest cognition	Personal interest	Topic 5: excellent, ratio, Soochow University (SUDA), WUR, area
		Topic 9: Shenzhen University (SZU), college entrance examination, 211, investment, reliable
		Topic 10: world, comprehensive, Huazhong University of Science and Technology (HUST), reach, top 100
		Topic 17: University, Southern University of Science and Technology (SUSTech), ranking, principal, many years
	Regional interest	Topic 13: Fudan University (FU), Shanghai Jiao Tong University (SJTU), follow with interest, progress, future

.

The public’s positive commentary behavior on the WURs was driven by three cognitive perspectives: standpoint, dialectic, and interest.

From the perspective of standpoint cognition, the positive attitude of the Chinese public towards the WURs was mainly based on the national position. The Chinese public had a positive sentiment toward the WURs when the results satisfied the Chinese public’s national stance and brought about a sense of national pride and strengthened national confidence. For example, topic 1 was about the Chinese public believing that THU’s undergraduate education is of a high standard internationally. Topic 3 showed the pride of the Chinese public when THU’s ranked first in Asia. Topic 7 was about the proud mentality of the Chinese public when the ranking of THU and PKU continued to improve. Topic 11 was about the identification of the Chinese public with the development of education in China. Topic 16 addressed the national confidence generated by the increasing number of Chinese universities in the top 100 of the WURs. Topics 18 and 23 reflected the expectations of the public for the continued progress of the country’s high-level universities.

From a dialectical–cognitive perspective, the theme of the Chinese public’s perception of the WURs is diverse. For example, in topics 2 and 14, the Chinese public analyzed the indicators of the WURs and found that the level of internationalization and academic excellence of universities are important indicators of the rankings. Topic 4 addressed the Chinese public’s perceived value of a university for talent development. Topics 6, 15, and 21 were about the Chinese public’s awareness of the gap that still exists between the overall level of development of Chinese universities and that of developed countries through their access to the WURs. They hope to strengthen the discipline construction and promote the development of the fields of applied and social sciences. In topics 8, 12, 19, and 20, the public analyzed the results of the WURs in a general and dialectical way and concluded that MIT has been number one on the QS list for many years, that the rankings are generally accurate, that many universities in Russia and Japan are also excellent, and that the level of research in China is improving. Topic 22 was about the Chinese public’s opinion that the country should make further efforts to improve its scientific and technological capabilities. In topic 24, it was shown that the Chinese public believes that reference to the WURs is the result of objective reality. Due to the information asymmetry and the fact that many employers refer to the WURs when recruiting, the public is forced to refer to the WURs to assist in their choice of institutions for better development.

From the perspective of interest cognition, the Chinese public’s attitude toward the WURs depends on the extent to which personal and regional interests are met. The public had a positive attitude towards the WURs when the interest was met. In topics 5, 9, 10, and 17, the public recognized the ranking results of SUDA, SZU, HUST, and SUSTech. Combined with the positive corpus, we saw that the words “alma mater,” “study,” and “children” were often found in the comments under these topics, indicating that the public pays attention to the results of the university rankings that are relevant to them. If the ranking results meet demand, the public will become optimistic about the WURs. It is a reflection of personal interest. In topic 13, the public expected FU and SJTU—the universities located in Shanghai—to keep up the good work. It is an expression of regional interest.

3.2.2. Public Cognitive Perspective on WURs under the Objective Emotion

The topic coherence of the objective-comment word database was calculated for different numbers of topics and the topic-coherence curve was obtained, as shown in Figure 4. As can be seen, the trend in the topic-consistency score changed from a sharp increase at the beginning to a leveling-off and then a sharp decrease. The number of topics for the LDA modeling of objective comments was determined to be 17.

Combined with the objective corpus, the output LDA topic-modeling results were subjected to meaning construction to obtain

Table 2. LDA topic-modeling results on objective comments.

Cognitive Perspective	Theme	Topic Number and Topic Feature Words
Standpoint cognition	National position	Topic 4: get on the list, global, university, total, ranking list
		Topic 11: ranking, Zhejiang University (ZJU), Nanjing University (NJU), Tianjin University (TJU), Nankai University (NKU)
		Topic 16: THU, Western, talent, train, America
Dialectical cognition	Ranking result	Topic 1: ranking, internal, America, university, front
		Topic 2: ranking, Britain, University of Oxford (Oxford), University of Cambridge (Cambridge), University
		Topic 5: THU and PKU, ranking, Yale University (Yale), Princeton University (Princeton), exactly
	Ranking discourse	Topic 9: ranking, foreign, country, America, understand
		Topic 17: identify, ranking, standard, world, rank
	Ranking indicator	Topic 10: Renmin University of China (RUC), subject, should, train, social sciences
	Ranking development	Topic 12: ranking, QS, ARWU, THE, effort
		Topic 13: U.S. News, ranking, make better, world ranking, America
Interest cognition	Personal interest	Topic 3: university ranking, HIT, objectivity, disparity, school
		Topic 6: excellent, major, education, Shanghai University (SHU), ZJU
		Topic 7: ranking, Sun Yat-sen University (SYSU), low, university, really
		Topic 8: PKU, ranking, authority, USTC, persuasiveness
		Topic 14: university, WHU, fame follows merit, fabricate, fair
Cultural cognition	Symbolic culture	Topic 15: China, first, think, foreign country, Nobel Prize

.

The public’s objective commentary behavior on the WURs was driven by four cognitive perspectives: standpoint, dialectic, interest, and culture.

From the perspective of standpoint cognition, the objective attitude of the Chinese public towards the WURs was mainly based on the national position. In topics 4 and 11, the Chinese public was still interested in the number of Chinese universities on the list and the results of the ranking of China’s top universities. Topic 16 was about the Chinese public’s concerns about the transnational brain drain and the accumulation of Chinese human capital.

From a dialectical cognitive perspective, the Chinese public’s objective perception of the WURs behavior was mainly based on ranking results, ranking discourse, ranking indicators, and ranking development. Topics 1, 2, and 5 were about public judgments on the overall and individual ranking results, yielding the conclusion that US universities are generally ranked highly, British universities are ranked highly in the British published rankings, and the level of Yale and Princeton is underestimated. Topics 9 and 17 were about the importance of the public’s awareness of the power of discourse on rankings by comparing the results of different country rankings. In topic 10, the public felt that the ranking indicators lacked emphasis on social-science outcomes, resulting in a low ranking for RUC, which is known for its social-science research. In topics 12 and 13, the public thought that WUR institutions should respond adequately to public needs and improve the credibility of the WURs.

From the perspective of interest cognition, the Chinese public objectively perceived the WURs mainly out of personal interest. In topics 3, 6, and 8, the main content was the public’s expectation of the continued development of stakeholder universities. Topics 7 and 14 were about the public questioning the lower-ranking results of stakeholder universities.

From a cultural-cognition perspective, the Chinese public’s objective perception of the WURs was mainly due to the influence of symbolic culture. Topic 15 addressed that the Nobel Prize is a typical symbol that represents the world’s top level of scholarship and contribution in the relevant field. The Chinese public regretted that China rarely wins Nobel Prizes when they know that Chinese universities are at the global level based on the WURs.

3.2.3. Public Cognitive Perspective on WURs under the Negative Emotion

The topic coherence of the negative-comment word database was calculated for different numbers of topics and the topic-coherence curve was obtained, as shown in Figure 5. As can be seen, the trend in the topic-consistency score changed from a sharp increase at the beginning to a levelling-off. The optimal number of topics for the LDA modeling of negative comments was determined to be 29.

Combined with the negative corpus, the output LDA topic-modeling results were subjected to meaning construction to obtain

Table 3. LDA topic-modeling results on negative comments.

Cognitive Perspective	Theme	Topic Number and Topic Feature Words
Standpoint cognition	National position	Topic 2: fool, ranking, Britain, country, suppress
		Topic 8: ranking, THE, low, list, QS
		Topic 9: why, ranking, America, Britain, influence
		Topic 12: ranking, untrustworthy, internal, QS, foreign
		Topic 13: ranking, reliability, not have, list, ridiculous
		Topic 14: university ranking, global, Chinese universities, quantity, belittle
		Topic 18: ranking, Western, standard, feel, value
		Topic 22: ranking, America, U.S. News, meaningless, standard
		Topic 27: ranking list, world, China, discourse power, chaos
	University standpoint	Topic 3: buy a list, ranking, brag, rule, list
		Topic 19: ranking, participate in, NJU, Lanzhou University (LZU), refuse
		Topic 20: get on the list, sprayer, disparity, fool, do not take it seriously
Dialectical cognition	Ranking indicator	Topic 7: ranking, foreign country, uncertain, English, Chinese
		Topic 16: list, international, talent, scale, quality
		Topic 21: fool, RUC, ranking, ratio, too low
		Topic 28: science and engineering, paper, ranking, error, culture
	University value	Topic 25: think, consult, unwanted, without effect, science and technology
Interest cognition	Personal interest	Topic 4: spend money, university, ranking, SYSU, rank
		Topic 6: Shanghai, place, FU, score, not as good as
		Topic 10: NJU, ranking, garbage, query, withdraw
		Topic 15: WHU, 985, against, famous university, reason
		Topic 24: Xi’an Jiaotong University (XJTU), only, ranking, fool, American and British
Cultural cognition	Symbolic culture	Topic 1: ranking, list, position, SUSTech, too unrealistic
		Topic 5: fabricate, HIT, ranking, SZU, impossible
		Topic 11: unrealistic, casual, place, SZU, Hunan University (HNU)
		Topic 17: GUR, not authoritative, ranking, University of California- Berkeley (Cal), science
		Topic 23: money, bad, uncertain, Hunan, Xiangtan University (XTU)
		Topic 29: SUSTech, ranking, lie, Sichuan University (SCU), SZU
	Institutional culture	Topic 26: ranking, subject, evaluate, fame, Ministry of Education

.

The public’s negative commentary behavior on the WURs was driven by four cognitive perspectives: standpoint, dialectic, interest, and culture.

The Chinese public’s negative perception of the WUR behavior from the perspective of standpoint cognition was mainly based on the national position and the school’s position. The public that perceived the WURs based on a national stance believed that the WURs reinforce the elite Anglo-Saxon research-university model, which emphasizes the role of the market in resource allocation [36], as elaborated in topics 2, 9, 12, 13, 18, and 22. This leaves other types, such as those with characteristics of the Roman system—emphasizing the role of government action in higher education—to perform poorly in the WURs, as discussed in topics 8 and 14. The Chinese public was also aware of the importance of establishing and improving a WUR system with the power of Chinese discourse, as described in topic 27. The public that perceived the WURs from the standpoint of the university recognized that universities are forced to compete in the rankings and that they cannot even make a decision regarding their own ranking position. In fact, the fairness of rankings often gives way to profit, and the public expects universities not to be overly obsessed with their ranking position but to do well for themselves, as described in topics 3 and 20. Topic 19 addressed the public’s understanding and support for the move by Nanjing University and Lanzhou University to announce their withdrawal from the WURs.

From a dialectical–cognitive perspective, the Chinese public’s negative perception of WUR behavior was mainly based on ranking indicators and university values. In topics 7, 16, 21, and 28, the Chinese public questioned the tendency of the WURs indicator to emphasize English-speaking universities over other non-native-English-speaking universities, quantity over quality, science over social sciences, and research over teaching. The Chinese public also recognized that rankings can only be used as a reference and that the value of universities is development and innovation, as elaborated in topic 25.

From the perspective of interest cognition, the Chinese public viewed the ranking of world universities negatively mainly out of personal interest. As discussed in topics 4, 6, 10, 15, and 24, when the ranking results were not in line with the public’s interests, the public questioned the credibility of the WURs and thought that the rankings are just fooling and cheating.

From the perspective of cultural cognition, the Chinese public viewed the WURs mainly due to the influence of symbolic and institutional culture. The WURs are internationally oriented rankings of universities in different cultural contexts, which means that the public’s recognition of the WURs is influenced by the individual experiences of different international contexts and cultural environments [37]. The negative attitude of the public towards the rankings stems from the shock of the ranking results on the symbolic culture in the previous individual experience. This symbolic culture includes the history of the university, “Project 985,” “Project 211,” and the Nobel Prize. In topics 1 and 23, the former questioned the high ranking of SUSTech, which has a relatively short history, and the latter questioned the low ranking of XTU, which has a long history. In topics 5, 11, and 29, the public questioned the ranking position of SUSTech and SZU, which are not “Project 985” and “Project 211” universities, as being higher than some old “Project 985” universities. “Project 985” and “Project 211” were born at the end of the 20th century in China, and they are the national key construction of higher-education-system projects. These two construction projects aim to focus on building some universities and disciplines at the advanced global level. Universities that are included in the construction projects of “Project 985” and “Project 211” mean that they have more funding and more favorable development conditions. In this context, whether a university is “Project 985,” “Project 211,” or “double not” is often simplified as the basis for judging the construction level of Chinese universities, forming a symbolic culture that is implemented in most Chinese public perception. In topic 17, the public questioned the slightly lower ranking of Berkeley, a college that has won multiple Nobel Prizes. For example, some members of the Chinese public found it questionable that Berkeley was only ranked 32nd in the 2022 QS-WUR, a lower ranking position than some colleges and universities that have never won a Nobel Prize. Instead of referring to the WURs, the public was more inclined to refer to the Ministry of Education’s subject-evaluation results, as elaborated in topic 26. This stems from the fact that the Chinese government dominates the allocation of higher-education resources, and the long-established culture of the higher-education system has led the Chinese public to trust the results of government-led assessments.

4. Discussion

Sentiment analysis is often used to observe public opinion, and the results can provide an important basis for online-opinion management, commodity production, service-industry strategy formulation, etc. LDA topic modeling is commonly used for the classification and clustering of massive text and can enable topic recognition of unexpected events, image processing and classification, the discovery of knowledge communities, etc. The modeling has been widely used in various fields, such as text mining and computer vision, and continues to play an important role. In this research, public comments were classified by the sentiment-analysis method. Then, the LDA topic modeling was applied to obtain the topics of public comments under different emotions. Finally, the public’s perspective on the perception of the WURs was outlined. The results of the research show that although the Chinese public has mixed feelings about the WURs, they are generally positive about them. That the rankings will continue to exist is not only a politicized rhetorical act of the global higher-education elite but also an objective reality that we have to face. Although the rankings are often criticized by groups of various classes with different identity characteristics, the credibility of the WURs remains and the influence on the higher-education system will continue, judging from the positive attitude maintained by the public as a whole.

Despite the drawbacks of the WURs themselves, a part of the public was still willing to be optimistic about them after perceiving the external stimulus from the cognition of position, discernment, and interest. The WURs satisfy the public’s need for competition between university institutions in different countries [38]. When the public saw that their country had made progress or dominated in the ranking competition, the resulting strong sense of national pride and national self-confidence made the public ignore the problems of the ranking and thus develop a positive attitude toward the WURs. In addition, the public was concerned about the ranking results of universities that have a stake in them, and they expected these universities to improve in the rankings because it may provide benefits to the public in terms of further education and employment [39]. When the public saw the rising ranking of universities with which they have a stake, they also ignored the disadvantages of the ranking and maintained a positive attitude. A part of the public understood that no ranking is perfect and without flaws, but they also recognized that some parts of the rankings are valuable. These members of the public dialectically viewed certain aspects of the rankings they believed are available in the context of objective reality and made slightly more positive comments. Of these, dialectical thinking about ranking indicators accounted for 8.33%.

When the public accepted the external stimulus of the World University Rankings, they combined their accumulation and personal preferences to awaken the units of position cognition, dialectical cognition, interest cognition, and cultural cognition, and interact with the emotional unit. When the public maintained a relatively stable mood, the reaction was more rational, such as objective comment behavior. The public that maintained an objective attitude toward the WURs tended to understand the WURs from the perspective of dialectical cognition and interest cognition. Under dialectical cognition, the public was more concerned about the indicator system, overall results, discourse reflection, and development direction of the ranking. Of these, dialectical thinking about ranking indicators accounted for 5.88%. Under the cognition of interests, the public paid attention to the ranking results of universities with interests, but the overall sentiment was relatively objective, even if the ranking results impacted the public interest.

The public that maintained a negative attitude toward the WURs was more likely to believe that the results of the ranking were different from the public’s position, interests, and long-held cultural cognitions. Of course, the problems of ranking itself was considered dialectically by the public. The public, which tended to be dialectically cognitive, thought about the rationality of the ranking indicators. They argued that the rankings are problematic because of the erroneous use of underdeveloped ranking indicators and the use of simple inappropriate alternative indicators to compare complex higher-education institutions, which are inherently open to criticism. The results of research in non-English languages, the quality of research, the results of social sciences, the social impact of universities, and the quality of teaching should not be ignored or simply replaced by a system of indicators [40]. However, those who had a negative attitude toward the ranking of world universities from the perspective of dialectical cognition were always in the minority. For example, dialectical thinking about ranking indicators accounted for 13.79%. More people thought negatively about the ranking of world universities based on their standpoint cognition, interest cognition, and cultural cognition. The public that looked at the WURs from a national standpoint found that the rankings have become a competitive game. No matter how much ranking institutions try to emphasize the objectivity of their rankings in their mission statements, the truth is that it is impossible for rankings to remain a neutral value [41,42]. The country that holds the power of the ranking discourse is the one that truly has the upper hand in the game [43,44]. For this reason, in order to resist being defined and manipulated in different ways by different rankings, and to counteract the passive participation in international rankings, many countries are designing their own international university rankings and thus competing for the power of international discourse in the rankings [45,46,47,48]. This has further led to an increasing number of WURs. Since the impact of rankings on higher education is already inevitable, it is pointless to discuss whether rankings should continue to exist. To counteract the unintended negative consequences of rankings, joining the ranking system, understanding the rules of ranking, and creating a new ranking of world universities seem to be a solution. This indirectly leads to a paradox in the development of rankings, and universities are caught in a situation in which they are overwhelmed by a variety of evaluation criteria. On this basis, part of the public also understood the situation of universities from the standpoint of universities and expressed its understanding and support for some universities to announce their withdrawal from WURs. The public with negative attitudes toward the WURs based on cognitions of interest was more inclined to focus on the negative impacts that rankings can bring to themselves. A part of the public that maintained a negative attitude toward the WURs perceived them from the perspective of cultural cognition. The Chinese public compared and analyzed the ranking results of relevant universities with individual experiences derived from the symbolic culture given by the state or accumulated reputation, and once the ranking results conflicted with individual experiences, the public had negative emotions toward the WURs. This also shows that even though the impact of the WURs on the higher-education system continues to deepen, universities with a certain identity attributed by the state or with a high reputation in a certain field are less affected by the WURs. From this perspective, universities with certain identity characteristics or certain reputational characteristics can avoid the isomorphic pressure placed on them by the WURs to a certain extent. Especially in some countries—where the state dominates the higher-education system—the public was more likely to give a higher level of trust to a state-recognized university than to a school that is ranked higher in WURs. Because of the identity characteristics given by the state, such as Project 985 and Project 211, the symbolic feature is difficult to obtain by universities due to the policy and background of the times [49]. The symbols obtained by reputation accumulation such as Nobel Prize winners or highly cited scholars have become the target of many universities to enhance their reputation and counter the unintended negative consequences of the WURs. A market for research stars has emerged [50], where scholars with high reputations in certain fields are often appointed to adjunct positions at other schools, and some schools even promise high packages with low commitments to get those scholars to come and teach at their universities on a full-time basis [51,52]. The increasing mobility of elite talent has intensified the vertical hierarchy of university levels worldwide, and the Matthew effect has become more pronounced.

5. Conclusions

This research analyzed 18,466 Chinese public comments on the WURs using the sentiment-analysis method and the LDA topic-modeling method in the text data-mining technique. Combining the results of sentiment analysis and LDA topic modeling, the complex emotions and different cognitive perspectives of the public on the WURs were summarized and the development paradox of WURs was explained.

Although the WURs are often controversial, the public still has a positive attitude toward them in general. Whether the WURs satisfy the public’s standpoint and interest affects the public’s sentiment and attitude toward them. In addition, once the public’s cultural cognitions are shaken, they maintain a negative attitude toward the WURs. The public rarely thinks dialectically about the problems with World University Ranking indicators, and the public is even barely exploring ranking methods. Although these are the main bases of criticism of the WURs by many higher-education researchers [53], the public is more inclined to pay attention to the WURs from the perspective of standpoint cognition, interest cognition, and cultural cognition, and the WURs are more often used as a tool by the public. Unlike some researchers who focus on ranking methods and logical regrets, the public is more concerned with whether standpoints are met, interests are represented, and individual experiences are validated. The public rarely criticizes the theories and methods of ranking, even if they perceive the WURs dialectically. This has led to the development paradox of ranking, as the problems with rankings are often ignored, whereas the public demand for the WURs is always present.

There are some limitations in this research. Firstly, the research only used the text of Chinese public comments on relevant WURs in the media platform Jinritoutiao as the data source for analysis, which may affect the generalizability of the results. Secondly, to explore the development paradox of the WURs, this research discussed how the Chinese public perceives the WURs under different emotions based on sentiment analysis and LDA topic modeling. However, the approach of using large-scale text-data mining easily ignores the individual characteristics of comments. In this research, valuable identity traits are neglected, publics with different identity traits have different cognitive perspectives and emotions regarding the WURs, and the behavioral mechanisms of the WURs are not similar. In addition, this research only analyzed the Chinese public’s comments on four internationally renowned WURs in a further attempt to explain the development paradox of the WURs. However, it is worth noting that this research does not distinguish the Chinese public’s comments on different WURs. Each World University Ranking has its value orientation and ranking methodology, which may lead to subtle differences in the Chinese public’s cognitive perspectives on different categories of WURs.

Further research is needed to resolve these limitations. Firstly, further expansion of textual data resources is required, such as policy texts and news texts from government and school official websites. Secondly, it is necessary to code identity traits for each piece of textual information and explore how stakeholders with different identity profiles feel about the WURs, from initial ambivalent feelings to having to participate in the ranking game. In addition, it is necessary to divide the public comments on different WURs. Furthermore, the common and individual characteristics of the development paradox of the formation of different WURs should be explored.