An Integrated Decision-Making Model for Analyzing Key Performance Indicators in University Performance Management

Abstract

University performance has an important effect on the social influence of universities. With increasing emphasis placed on higher education, it is important to improve and optimize university performance management. However, the performance of university management is affected by numerous indicators in practice, and it is difficult for administrators to optimize all of them because of resource restriction. To address this concern, in this paper, we design a novel integrated model by combining linguistic hesitant fuzzy sets (LHFSs) with the decision-making trial and evaluation laboratory (DEMATEL) method to identify key performance indicators (KPIs) for improving the level of university performance management. Specifically, the LHFSs are utilized to express the hesitant and vague interrelationship assessment of performance indicators provided by experts. A modified DEMATEL is adopted to visualize the causal relationship between performance indicators and determine critical ones. Moreover, we introduce a gray relation analysis (GRA)-based method to derive experts’ weights when their weight information is unknown. Finally, a comprehensive university in Shanghai, China, is employed as an example to illustrate the practicability and availability of the proposed linguistic hesitant fuzzy DEMATEL model.

1. Introduction

In the past few decades, great changes have taken place in the construction and size of higher education, especially in China [1,2]. With the reform of the national education system, the modification of the modern higher education system, educational models, and teaching methods promoted the diversified development of modern universities [3,4]. To enhance the competitiveness of a university, problems on how to enhance overall strength of the university, improve the teaching quality, and cultivate highly qualified talent have attracted the attention of university administrators [5,6]. As evidence of the achievement of organizational goals, performance measurement has emerged in university performance management [7,8,9]. A growing number of researchers are focusing on university performance measurement [10,11,12]. However, the performance of university management is affected by several indicators, and it is unrealistic to improve them all simultaneously due to the restriction of resources. Hence, it is imperative to identify key performance indicators (KPIs) to measure and evaluate the performance of universities [13].

In prior research, KPIs for university performance management were often identified using prepared questionnaires and expert interviews [14,15]. Moreover, some researchers focused on university performance measurement using the balanced scorecard (BSC) method [16,17]. However, these research methods are vulnerable to the subjectivity of humans and cannot explore the causal interrelationships among university performance indicators. The decision-making trial and evaluation laboratory (DEMATEL) method initiated by Gabus and Fontela [18] is a powerful approach to extract the relationships and the interdependence among elements. It is superior to conventional cause–effect analysis techniques because it visualizes the structure of complex causal relationships through matrices and digraphs [19,20]. Given its capabilities, the DEMETAL method has been broadly used to deal with causal relationships among complex factors in various research fields [21,22,23]. Thus, it is promising to adopt the DEMATEL method to analyze the interrelationships among indicators and identify KPIs for university performance management.

Due to the complex interrelationships of indicators, as well as the ambiguity of human thinking, experts tend to express their opinions with linguistic terms in the process of performance indicator evaluation [24]. Moreover, they usually provide their interrelationship assessment information with hesitancy or anonymity. Linguistic hesitant fuzzy sets (LHFSs) were presented by Meng et al. [25] to deal with complex decision-making information and the hesitancy of decision-makers. This method can not only deal with the qualitative evaluations of experts but also reflect their hesitancy, uncertainty, and fuzziness [26,27]. Thus, LHFSs are able to express the vague information of decision-makers more accurately in practical situations. Recently, the LHFS method was employed in various different areas to address the qualitative preferences of decision-makers. For example, Dong et al. [28] presented a linguistic hesitant fuzzy (LHF)–VIKOR (in Serbian: ViseKriterijumska Optimizacija I Kompromisno Resenje) method for selecting transportation systems. Wu et al. [29] proposed an integrated model by combining the cloud model with LHFSs for the risk assessment of a seawater-pumped hydro storage project. Meng et al. [30] introduced an extend model using LHFSs for evaluating corporate environmental performance. On the basis of the linguistic hesitant fuzzy rock engineering system connection cloud, Gao et al. [31] established a stability evaluation model of surrounding rock stability.

Considering the advantages of LHFSs and the DEMATEL method in the decision-making process, this paper aims to combine them to develop an integrated model to analyze the interrelationships of performance indicators and identify KPIs for university performance management. In summary, this study makes the following valuable contributions: (1) we introduce the use of LHFSs to deal with the uncertain and vague assessment information provided by experts about the interrelationships of performance indicators; (2) we propose a modified DEMATEL method to investigate the causal relationships of indicators and obtain KPIs for university performance improvement; (3) we construct a gray relation analysis (GRA)-based method to acquire experts’ weights when their weight information is unknown. Finally, a practical example of a university in Shanghai, China, is provided to demonstrate the application and effectiveness of the proposed LHF–DEMATEL model.

The remainder of this paper is arranged as follows: previous studies related to university performance management and the DEMATEL method are reviewed in Section 2. In Section 3, a hybrid decision-making model using LHFSs and DEMATEL is put forward to identify KPIs for university performance management. In Section 4, a practical case is presented to validate the effectiveness and advantages of the proposed LHF–DEMATEL model. Finally, Section 5 draws the conclusions of this paper and provides further research directions.

2. Literature Review

2.1. University Performance Management

Over the past few years, performance management in universities has been an important research topic and has received great attention from researchers. For example, Philbin [17] analyzed and identified the KPIs of a multidisciplinary university institute in the United Kingdom using the balanced scorecard method. Acquaah et al. [32] put forward a disproportionate stratified random sampling technique for identifying the KPIs impacting performance management implementation in public universities in Uganda. Chang et al. [33] used the multivariate analysis of covariance to evaluate university students’ knowledge management performance. By using the fuzzy analytic hierarchy process (AHP), Wang and Liu [34] assessed the performance of scientific research project management in a university. Amin et al. [35] conducted a questionnaire survey on a public university in Malaysia to measure the impact of human resource management practices on organizational performance. The multicriteria constructivist methodology for decision support was adopted by Valmorbida et al. [13] to monitor and manage the performance of the Federal Technological University of Paraná-Brazil. Karuhanga [36] applied principal component and cluster analysis techniques for the performance measurement of public universities in Uganda. Thanassoulis et al. [37] integrated AHP with data envelopment analysis (DEA) to evaluate higher-education teaching performance. By integrating AHP and a technique for order preference by similarity to an ideal solution (TOPSIS), Xu et al. [38] established a model to assess teaching performance in a smart campus. By combing a standardized u-control chart with the ABC analysis method, Carlucci et al. [39] proposed an integrated approach to evaluate the teaching and course quality in an Italian public university.

2.2. The DEMATEL Method

In the literature, the DEMATEL technique has been adopted by researchers to analyze the interrelationships among system elements in various fields [40]. For instance, Tsai [41] utilized the fuzzy DEMATEL method to identify key success factors of the auto lighting aftermarket industry. Ding and Liu [42] presented a two-dimensional uncertain linguistic DEMATEL model to find significant success factors in emergency management. Raj et al. [43] investigated the critical barriers to the adoption of Industry 4.0 technologies in the manufacturing sector by using the gray DEMATEL method. Abdullah et al. [44] analyzed the interrelationships and dependence of criteria in sustainable solid waste management using an interval-valued intuitionistic fuzzy DEMATEL method. Liu and Ming [45] employed the rough DEMATEL method to analyze and evaluate requirements for a smart industrial product service system. Jiang et al. [46] presented a large-group linguistic Z-DEMATEL approach for identifying KPIs in hospital performance management. Chauhan et al. [22] utilized a hybrid model on the basis of interpretive structural modeling (ISM) and DEMATEL to select a sustainable supply chain for agricultural produce in India. A hesitant fuzzy DEMATEL model was used by Kumar et al. [47] to analyze the factors related to the adoption of sustainable supply chain practices. By integrating a DEMATEL-based analytic network process, Hsu et al. [48] investigated the critical selection criteria for local middle and top management in multinational enterprises.

An extensive review of the related literature shows that researchers made great efforts in measuring and improving university performance management. However, no known current study, as per available data, has considered the interaction between performance indicators in university performance management. On the other hand, many fuzzy extensions of the DEMATEL method have been proposed to analyze the interrelationships among performance indicators and determine KPIs in different complex systems. However, these modified DEMATEL methods are not efficient in expressing the hesitant assessment information of decision-makers. In order to address the above issues, we extend the DEMATEL method with LHFSs in this study and develop an LHF–DEMATEL model to evaluate the relationships of performance indicators and identify KPIs for university performance management.

3. The Proposed LHF–DEMATEL Model

In this section, a hybrid model using LHFSs and DEMATEL is proposed to analyze the interrelationships of performance indicators and to identify KPIs for university performance management. First, the LHFSs are utilized to deal with the fuzzy assessment information of experts regarding the interrelationships of performance indicators. Second, a GRA-based weighting method is adopted to derive the weights of experts. Finally, a modified DEMATEL method is employed to analyze the interrelationships among indicators and identify KPIs. Figure 1 shows the detailed steps of the proposed LHF–DEMATEL approach in the form of a flowchart.

Suppose that $F=\left\{F_1,F_2,\dots ,F_n\right\}$ is a set of university performance indicators identified for future analysis, and $E=\left\{E_1,E_2,\dots ,E_m\right\}$ is a set of experts. The experts are invited to give their evaluation information on the interdependence among indicators. Let $H^k={\left(L{H}_{ij}^k\right)}_{n\times n}\left(k=1,2,\dots ,m\right)$ be the linguistic interrelationship evaluation matrix of the k-th expert, where $L{H}_{ij}^k=\{\langle h_k,l{h}_k\rangle |h_k\in S,$$k=1,2,\dots ,l_{LH}\}$ is an LHFS provided by E_k on the comparison of performance indicators $F_i$ and $F_j$. Note that the basic concepts and definitions of LHFSs can be found in [19,25]. The procedure of the proposed LHF–DEMATEL model is introduced below.

Stage 1. Determine expert weights on the basis of the GRA method.

The GRA method proposed by Deng [49] is an effective method to choose the alternative with the highest gray relational grade to a reference sequence. In this study, the GRA method is adopted to determine the weights of experts when their weight information is unknown.

Step 1: Calculating the distance matrices between $H^k$ and $H^{*}$.

The ideal matrix $H^{*}={\left(L{H}_{ij}^{*}\right)}_{n\times n}$ is the average matrix of the m matrices $H^k\left(k=1,2,\dots ,m\right)$. As a result, the distance matrix between $H^k$ and $H^{*}$ can be represented as

$$D\left(H^k,H^{*}\right)=\left[\begin{array}{llll}0& d_{12}& \dots & d_{1n}\\ d_{21}& 0& \dots & d_{2n}\\ ︙& ︙& \dots & ︙\\ d_{n1}& d_{n2}& \dots & 0\end{array}\right],$$

(1)

where $d_{ij}$ is calculated using Equation (2) according to the LHF distance defined in [28].

$$\begin{array}{cc}{d}_{ij}=\hfill & \{\frac{1}{2}[\frac{1}{l_{L{H}_{ij}^k}}{\displaystyle \sum _{\left(h_k,l{h}_k\right)\in L{H}_{ij}^k}\underset{\left(h_q,l{h}_q\right)\in L{H}_{ij}^q}{\min }\left|\frac{\frac{f\left(h_k\right)}{\left|l{h}_k\right|}\left({\displaystyle {\sum }_{k^{\prime }=1}^{\left|l{h}_k\right|}{k}^{\prime }}{r}_{k^{\prime }}^k\right)}{{\displaystyle {\sum }_{k^{\prime }=1}^{\left|l{h}_k\right|}{k}^{\prime }}}-\frac{\frac{f\left(h_k\right)}{\left|l{h}_q\right|}\left({\displaystyle {\sum }_{p=1}^{\left|l{h}_q\right|}p}{r}_p^{*}\right)}{{\displaystyle {\sum }_{p=1}^{\left|l{h}_q\right|}p}}\right|}\hfill \\ \hfill & +\frac{1}{l_{L{H}_{ij}^q}}{\displaystyle \sum _{\left(h_q,l{h}_q\right)\in L{H}_{ij}^{*}}\underset{\left(h_k,l{h}_k\right)\in L{H}_{ij}^k}{\min }}\left|\frac{\frac{f\left(h_k\right)}{\left|l{h}_q\right|}\left({\displaystyle {\sum }_{p=1}^{\left|l{h}_q\right|}p}{r}_p^q\right)}{{\displaystyle {\sum }_{p=1}^{\left|l{h}_q\right|}p}}-\frac{\frac{f\left(h_k\right)}{\left|l{h}_k\right|}\left({\displaystyle {\sum }_{k^{\prime }=1}^{\left|l{h}_k\right|}{k}^{\prime }}{r}_{k^{\prime }}^k\right)}{{\displaystyle {\sum }_{k^{\prime }=1}^{\left|l{h}_k\right|}{k}^{\prime }}}\right|]\},\hfill \end{array}$$

(2)

where $r_p^{*}\in l{h}_q\left(p=1,2,\dots ,\left|l{h}_q\right|\right)$ indicates p-th possible membership degree of the q-th linguistic term in $L{H}_{ij}^{*}$, and p is a position weight related to $r_p^q$.

Step 2: Obtaining the gray relation coefficients to the ideal matrix.

The gray relation coefficient matrix ${\xi }_k={\left({\xi }_{ij}^k\right)}_{n\times n}\left(k=1,2,\dots ,m\right)$ of the k-th expert can be calculated by

$${\xi }_{ij}^k=\frac{\underset{1\le i\le n}{\min }\underset{1\le j\le n}{\min }{d}_{ij}+\varsigma \underset{1\le i\le n}{\max }\underset{1\le j\le n}{\max }{d}_{ij}}{d_{ij}+\varsigma \underset{1\le i\le n}{\max }\underset{1\le j\le n}{\max }{d}_{ij}},$$

(3)

where $\zeta$ is the distinguishing coefficient satisfying $\zeta \in \left[0,1\right]$.

Step 3: Determining the weights of experts.

The weight of each expert $w_k$ is calculated by

$$w_k=\frac{{\overline{\xi }}_{ij}^k}{{\displaystyle \sum _{k=1}^m{\overline{\xi }}_{ij}^k}},$$

(4)

where ${\overline{\xi }}_{ij}^k$ is the average gray relation coefficient of $H^k$ and can be computed by

$${\overline{\xi }}_{ij}^k=\frac{1}{n\times n}{\displaystyle \sum _{i=1}^n{\displaystyle \sum _{j=1}^n{\xi }_{ij}^k}}.$$

(5)

Stage 2. Analyze the interrelationships of indicators with the DEMATEL.

In this stage, a modified DEMATEL technique is employed to interpret the interrelationships of performance indicators for identifying KPIs. The application steps of the modified DEMATEL method are listed below.

Step 4: Constructing the group interrelationship evaluation matrix.

By using the linguistic hesitant fuzzy weighted averaging (LHFWA) operator [25], the group interrelationship evaluation matrix $LH={\left(L{H}_{ij}^{}\right)}_{n\times n}$ can be obtained by

$$\begin{array}{c}L{H}_{ij}^{}=LHFWA\left(L{H}_{ij}^1,L{H}_{ij}^2,\dots ,L{H}_{ij}^m\right)=\underset{\left(s_{\theta \left(1\right)},lh\left(s_{\theta \left(1\right)}\right)\right)\in L{H}_{ij}^1,\dots ,\left(s_{\theta \left(m\right)},lh\left(s_{\theta \left(m\right)}\right)\right)\in L{H}_{ij}^m}{\cup }\\ \left(s_{{\displaystyle {\sum }_{k=1}^m{w}_{L{H}_{\left(k\right)}}}\theta \left(k\right)},\underset{r_{\left(1\right)}\in lh\left(s_{\theta \left(1\right)}\right),\dots ,r_{\left(m\right)}\in lh\left(s_{\theta \left(m\right)}\right)}{\cup }\left(1-{\displaystyle \prod _{k=1}^m{\left(1-r_{\left(k\right)}\right)}^{w_{L{H}_{\left(k\right)}}}}\right)\right),\end{array}$$

(6)

where $L{H}_{ij}$ is composed of g linguistic hesitant fuzzy sets, denoted by $L{H}_{ij}=\left\{L{H}_t|t=1,2,\dots ,g\right\}$ and $L{H}_t=\{\langle h_t,l{h}_t\rangle |h_t\in S,t=1,2,\dots ,l_{L{H}_t}\}.$

Step 5: Generating the direct relation matrix.

On the basis of the group interrelationship evaluation matrix LH, the direct relation matrix $Z={\left(z_{ij}\right)}_{n\times n}$ can be obtained, in which $z_{ij}$ is determined by

$$z_{ij}=\frac{{\displaystyle {\sum }_{t=1}^g{\Delta }^{-1}\left(h_t,\overline{l{h}_t}\right)}}{g},$$

(7)

where $\overline{l{h}_t}$ indicates the average possible membership degree in $l{h}_t$, and ${\Delta }^{-1}$ is a linguistic conversion function [19].

Step 6: Calculating the normalized direct relation matrix.

After acquiring the direct relation matrix $Z$, the normalized direct relation matrix $X={\left(x_{ij}\right)}_{n\times n}$ is obtained by

$$X=\frac{Z}{\max \left\{\underset{1\le i\le n}{\max }{\displaystyle \sum _{j=1}^n{z}_{ij},\underset{1\le j\le n}{\max }{\displaystyle \sum _{i=1}^n{z}_{ij}}}\right\}}.$$

(8)

Step 7: Computing the total relation matrix.

The total relation matrix $T={\left(t_{ij}\right)}_{n\times n}$ can be derived by

$$T=\underset{u\to \infty }{\lim }\left(X+X^2+X^3+\dots +X^u\right)=X{\left(I-X\right)}^{-1}.$$

(9)

where I is an $n\times n$ identify matrix.

Step 8: Construct the causal diagram of indicators.

The vectors R and C, which denote the sum of the rows and columns respectively, are acquired using Equations (10) and (11).

$$R={\left(r_i\right)}_{n\times 1}={\left({\displaystyle \sum _{j=1}^n{t}_{ij}}\right)}_{n\times 1},$$

(10)

$$C={\left(c_j\right)}_{1\times n}={\left({\displaystyle \sum _{i=1}^n{t}_{ij}}\right)}_{1\times n},$$

(11)

where $R$ represents the total influence that indicator $F_i$ exerts to the other indicators, while $C$ stands for the total influence that indicator $F_i$ receives from the other indicators.

Finally, a causal diagram can be constructed by mapping the ordered pairs of $\left(R+C,R-C\right)$, where the horizontal axis $R+C$ is named “Prominence” and the vertical axis $R-C$ is named “Relation”. In the causal diagram, “Prominence” represents the strength of influences that are given and received of the indicators, whereas “Relation” indicates the net effect contributed by the indicators to the system. Generally, if $R-C>0$, the indicator should be group under the cause group; if $R-C<0$, the indicator should be group under the effect group [40,50].

4. Case Study

In this section, the performance analysis of a university in Shanghai is provided to illustrate the flexibility and effectiveness of our proposed LHF–DEMATEL model.

4.1. Implementation of the Proposed Model

The considered university has 10 academic disciplines of engineering, science, medicine, management, economics, philosophy, humanities, law, education, and arts. Currently, the university registers over 41,000 students and it has 4200 academic staff for teaching or research. Currently, it has 47 programs for academic master’s degrees, 18 programs for professional master’s degrees, 26 programs for master’s degrees in engineering, 30 programs for academic doctoral degrees, three programs for professional doctoral degrees, and 25 postdoctoral research stations. To further improve the efficiency and the quality of management, and to enhance the core competitiveness of the university, it is of the utmost importance for administrators to determine KPIs for university performance management.

Through expert interviews and a literature review [15,37,51], 14 performance indicators $\left\{F_1,F_2,\dots ,F_{14}\right\}$ were identified to have a significant influence on the university performance management, as shown in

Table 1. Performance indicators considered in the case study.

Measures	Indicators	Description
Construction of teacher group	Quantity and structure (F1)	Number of professors and associate professors, lecturers
	Professional abilities (F2)	The abilities of teachers involve classroom teaching, educational and scientific research, curriculum resource development and utilization, academic communication
Teaching condition and use	Teaching infrastructure (F3)	Teaching infrastructure includes laboratories, libraries, campus networks, school buildings, outpatient expenses
	Funding input (F4)	Expenditures related to teaching expenses, including fundings for teaching staff, academic research, teaching equipment
Professional curriculum construction	Number of key disciplines (F5)	The number of key disciplines in which universities use their limited resources for certain disciplines to achieve breakthroughs in talent and technology
	Quality of newly established specialty (F6)	The qulaity of newly established speciality, mainly refers to the enrollment of new majors, employment rate
Construction of study style	Construction of study style (F7)	Comprehensive performance of all teachers and students in learning
	Guidance and service level (F8)	Specific training of professionals such as entrepreneurship guidance and employment guidance
Teaching achievements	Social recognition (F9)	The satisfaction of companies to the graduates’ knowledge reserve, working skills, comprehensive quality
	Employment rate of students (F10)	Comprehensive quality evaluation of employment rate, salary, professional matching, and career expectation
	Teachers’ scientific research level (F11)	Quantity and quality of academic papers, project topics
	Information infrastructure achievements (F12)	Achievements in the construction of information-based teaching equipment and information resources
	Teacher satisfaction (F13)	The satisfaction of teachers to the management system, curriculum arrangement, welfare
	Student satisfaction (F14)	The satisfaction of students to the infrastructure construction, learning conditions, teaching capacity

. Furthermore, an expert panel consisting of five domain experts $E=\left\{E_1,E_2,\dots ,E_5\right\}$ was established to assess the interrelationships among performance indicators. The following linguistic term set $S$ was adopted to evaluate the direct influence of the 14 performance indicators:

$$\begin{array}{cc}S=\hfill & \{s_1=Noinfluence,s_2=VeryLowinfluence,s_3=Lowinfluence,s_4=Mediuminfluence,s_5=Highinfluence,\hfill \\ \hfill & s_6=VeryHighinfluence,s_7=ExtremelyHighinfluence\}.\hfill \end{array}$$

As a result, the linguistic evaluations of the five experts on the direct influence of each pair of performance indicators were obtained. For example, the linguistic evaluation matrix $H^1={\left(L{H}_{ij}^1\right)}_{14\times 14}$ of expert E₁ is displayed in

Table 2. Individual interrelationship evaluation matrix of the first expert.

Indicators	F1	F2	F3	F4	…	F13	F14
F1	/	{(s6,0.8)}	{(s3,0.6) (s4,0.4)}	{(s6,0.5)}	…	{(s2,0.6) (s4,0.4)}	{(s6,0.7) (s7,0.9)}
F2	{(s6,0.9) (s7,0.8)}	/	{(s4,0.3)}	{(s6,0.4)}	…	{(s3,0.7) (s6,0.4)}	{(s7,0.8)}
F3	{(s4,0.4)}	{(s3,0.5) (s6,0.4)}	/	{(s4,0.6)}	…	{(s4,0.9)}	{(s3,0.6) (s5,0.6)}
F4	{(s5,0.7)}	{(s6,0.7)}	{(s6,0.6)}	/	…	{(s6,05) (s7,0.6)}	{(s6,0.9)}
F5	{(s5,0.9)}	{(s6,0.7)}	{(s5,0.4)}	{(s6,0.8)}	…	{(s5,0.8)}	{(s6,0.9)}
F6	{(s5,0.3)}	{(s7,0.8)}	{(s3,0.5)}	{(s6,0.5)}	…	{(s5,0.6)}	{(s6,0.7)}
F7	{(s4,0.4)}	{(s5,0.6)}	{(s2,0.3)}	{(s5,0.4)}	…	{(s6,0.9)}	{(s6,0.9)}
F8	{(s3,0.4)}	{(s7,0.8)}	{(s1,0.5)}	{(s7,0.8)}	…	{(s6,0.8)}	{(s7,0.8)}
F9	{(s6,0.3)}	{(s4,0.4)}	{(s5,0.7)}	{(s6,0.9)}	/	{(s7,0.7)}	{(s6,0.7) (s7,0.8)}
F10	{(s3,0.4)}	{(s5,0.3)}	{(s2,0.3)}	{(s4,0.9)}	…	{(s6,0.8)}	{(s7,0.8)}
F11	{(s5,0.6)}	{(s7,0.8)}	{(s4,0.7)}	{(s5,0.3)}	…	{(s6,0.8)}	{(s7,0.7)}
F12	{(s2,0.4)}	{(s2,0.5)}	{(s4,0.3)}	{(s4,0.3)}	…	{(s6,0.4)}	{(s7,0.9)}
F13	{(s6,0.9)}	{(s7,0.7)}	{(s4,0.5)}	{(s3,0.4)}	…	/	{(s4,0.3)}
F14	{(s2,0.5)}	{(s2,0.2)}	{(s3,0.7)}	{(s4,0.5)}	…	{(s4,0.7) (s5,0.9)}	/

.

Below, the implementation steps of our proposed LHF–DEMATEL model are described.

Step 1: According to Equations (1) and (2), the distance matrices of the five experts $D\left(H^k,H^{*}\right)\left(k=1,2,\dots ,5\right)$ can be obtained. For example, the distance matrix of the first expert $D\left(H^1,H^{*}\right)$ is shown in

Table 3. Distance matrix of the first expert.

Indicators	F1	F2	F3	F4	F5	F6	F7	F8	F9	F10	F11	F12	F13	F14
F1	0.000	0.166	0.149	0.166	0.291	0.014	0.180	0.163	0.134	0.211	0.106	0.151	0.406	0.029
F2	0.220	0.000	0.126	0.257	0.020	0.034	0.040	0.163	0.000	0.071	0.237	0.340	0.123	0.043
F3	0.006	0.291	0.000	0.011	0.274	0.080	0.046	0.063	0.114	0.080	0.091	0.080	0.086	0.303
F4	0.114	0.017	0.197	0.000	0.191	0.089	0.031	0.006	0.003	0.229	0.326	0.051	0.017	0.194
F5	0.126	0.051	0.083	0.006	0.000	0.054	0.046	0.137	0.006	0.237	0.157	0.091	0.054	0.066
F6	0.183	0.160	0.031	0.037	0.297	0.000	0.117	0.100	0.231	0.046	0.106	0.057	0.091	0.134
F7	0.071	0.057	0.154	0.031	0.057	0.017	0.000	0.011	0.051	0.006	0.209	0.046	0.171	0.031
F8	0.029	0.469	0.080	0.529	0.117	0.269	0.146	0.000	0.103	0.209	0.160	0.129	0.246	0.046
F9	0.174	0.080	0.260	0.354	0.111	0.149	0.551	0.057	0.000	0.000	0.034	0.083	0.091	0.054
F10	0.011	0.054	0.040	0.063	0.057	0.014	0.126	0.126	0.020	0.000	0.023	0.063	0.117	0.060
F11	0.089	0.266	0.137	0.251	0.186	0.091	0.046	0.049	0.086	0.083	0.000	0.126	0.103	0.069
F12	0.089	0.177	0.023	0.351	0.020	0.054	0.086	0.154	0.086	0.097	0.006	0.000	0.209	0.177
F13	0.206	0.014	0.014	0.034	0.209	0.226	0.137	0.054	0.069	0.100	0.060	0.203	0.000	0.211
F14	0.011	0.071	0.169	0.074	0.006	0.063	0.394	0.057	0.146	0.169	0.094	0.031	0.037	0.000

.

Step 2: Using Equation (3), we can establish the gray coefficient relation matrices ${\xi }_k={\left({\xi }_{ij}^k\right)}_{14\times 14}\left(k=1,2,\dots ,5\right)$ of five experts. Taking the first expert as an example, the results are presented in

Table 4. The gray relation coefficient relation matrix ${\xi }^1$.

Indicators	F1	F2	F3	F4	F5	F6	F7	F8	F9	F10	F11	F12	F13	F14
F1	1.000	0.625	0.650	0.625	0.486	0.951	0.605	0.629	0.673	0.566	0.723	0.646	0.405	0.906
F2	0.556	1.000	0.687	0.517	0.932	0.889	0.873	0.629	1.000	0.794	0.538	0.448	0.692	0.866
F3	0.980	0.486	1.000	0.960	0.501	0.775	0.858	0.814	0.707	0.775	0.751	0.775	0.763	0.477
F4	0.707	0.942	0.583	1.000	0.590	0.757	0.898	0.980	0.990	0.547	0.458	0.843	0.942	0.587
F5	0.687	0.843	0.769	0.980	1.000	0.836	0.858	0.668	0.980	0.538	0.637	0.751	0.836	0.808
F6	0.601	0.633	0.898	0.881	0.481	1.000	0.702	0.734	0.544	0.858	0.723	0.828	0.751	0.673
F7	0.794	0.828	0.641	0.898	0.828	0.942	1.000	0.960	0.843	0.980	0.569	0.858	0.617	0.898
F8	0.906	0.370	0.775	0.343	0.702	0.507	0.654	1.000	0.728	0.569	0.633	0.682	0.529	0.858
F9	0.613	0.775	0.515	0.438	0.712	0.650	0.333	0.828	1.000	1.000	0.889	0.769	0.751	0.836
F10	0.960	0.836	0.873	0.814	0.828	0.951	0.687	0.687	0.932	1.000	0.923	0.814	0.702	0.821
F11	0.757	0.509	0.668	0.523	0.598	0.751	0.858	0.850	0.763	0.769	1.000	0.687	0.728	0.801
F12	0.757	0.609	0.923	0.440	0.932	0.836	0.763	0.641	0.763	0.740	0.980	1.000	0.569	0.609
F13	0.573	0.951	0.951	0.889	0.569	0.550	0.668	0.836	0.801	0.734	0.821	0.576	1.000	0.566
F14	0.960	0.794	0.621	0.788	0.980	0.814	0.412	0.828	0.654	0.621	0.745	0.898	0.881	1.000

.

Step 3: Using Equations (4) and (5), the weights of the five experts are derived as $w_1=0.200,w_2=0.205,w_3=0.200,w_4=0.200,w_5=0.195$.

Step 4: Applying Equation (6), the five linguistic interrelationship evaluation matrices $H^k\left(k=1,2,\dots ,5\right)$ are aggregated to obtain the collective interrelationship evaluation matrix $LH={\left(L{H}_{ij}\right)}_{14\times 14}$.

Step 5: By Equation (7), the direct relation matrix $Z={\left(z_{ij}\right)}_{14\times 14}$ is established as provided in

Table 5. The direct relation matrix Z.

Indicators	F1	F2	F3	F4	F5	F6	F7	F8	F9	F10	F11	F12	F13	F14
F1	0.000	1.000	0.643	0.868	0.928	0.985	0.812	0.930	1.000	0.697	0.993	0.568	0.815	0.948
F2	0.710	0.000	0.510	0.892	1.000	1.000	0.585	0.840	0.982	0.926	0.873	0.641	0.703	0.987
F3	0.326	0.750	0.000	0.619	0.788	0.769	0.395	0.446	0.433	0.329	0.842	0.628	0.765	0.892
F4	0.722	0.888	0.960	0.000	0.842	0.899	0.464	0.625	0.596	0.469	0.893	0.899	0.891	0.969
F5	0.634	0.833	0.527	0.905	0.000	0.909	0.470	0.652	1.000	1.000	1.000	0.525	0.819	0.932
F6	0.585	0.845	0.445	0.674	0.765	0.000	0.532	0.412	0.902	0.924	0.866	0.488	0.828	0.986
F7	0.297	0.745	0.397	0.615	0.451	0.822	0.000	0.601	0.984	0.805	0.739	0.442	0.869	0.936
F8	0.322	0.583	0.333	0.543	0.520	0.704	0.794	0.000	0.886	0.836	0.684	0.568	0.705	0.987
F9	0.639	0.580	0.413	0.688	0.660	0.695	0.641	0.380	0.000	0.953	0.348	0.583	0.862	0.911
F10	0.284	0.500	0.284	0.685	0.654	0.518	0.554	0.781	1.000	0.000	0.407	0.404	0.842	1.000
F11	0.603	0.751	0.498	0.721	1.000	0.740	0.597	0.595	0.905	0.511	0.000	0.452	0.858	0.869
F12	0.417	0.581	0.528	0.751	0.718	0.641	0.482	0.467	0.694	0.389	0.707	0.000	0.812	0.907
F13	0.715	0.958	0.503	0.403	0.729	0.945	0.560	0.505	0.559	0.281	0.867	0.397	0.000	0.678
F14	0.357	0.333	0.303	0.531	0.506	0.392	0.775	0.504	0.760	0.574	0.499	0.420	0.710	0.000

.

Step 6: Via Equation (8), the normalized direct relation matrix $X={\left(x_{ij}\right)}_{14\times 14}$ is constructed as represented in

Table 6. The normalized direct relation matrix X.

Indicators	F1	F2	F3	F4	F5	F6	F7	F8	F9	F10	F11	F12	F13	F14
F1	0.000	0.083	0.054	0.072	0.077	0.082	0.068	0.078	0.083	0.058	0.083	0.047	0.068	0.079
F2	0.059	0.000	0.043	0.074	0.083	0.083	0.049	0.070	0.082	0.077	0.073	0.053	0.059	0.082
F3	0.027	0.063	0.000	0.052	0.066	0.064	0.033	0.037	0.036	0.027	0.070	0.052	0.064	0.074
F4	0.060	0.074	0.080	0.000	0.070	0.075	0.039	0.052	0.050	0.039	0.074	0.075	0.074	0.081
F5	0.053	0.069	0.044	0.075	0.000	0.076	0.039	0.054	0.083	0.083	0.083	0.044	0.068	0.078
F6	0.049	0.070	0.037	0.056	0.064	0.000	0.044	0.034	0.075	0.077	0.072	0.041	0.069	0.082
F7	0.025	0.062	0.033	0.051	0.038	0.069	0.000	0.050	0.082	0.067	0.062	0.037	0.072	0.078
F8	0.027	0.049	0.028	0.045	0.043	0.059	0.066	0.000	0.074	0.070	0.057	0.047	0.059	0.082
F9	0.053	0.048	0.034	0.057	0.055	0.058	0.053	0.032	0.000	0.079	0.029	0.049	0.072	0.076
F10	0.024	0.042	0.024	0.057	0.055	0.043	0.046	0.065	0.083	0.000	0.034	0.034	0.070	0.083
F11	0.050	0.063	0.042	0.060	0.083	0.062	0.050	0.050	0.075	0.043	0.000	0.038	0.072	0.072
F12	0.035	0.048	0.044	0.063	0.060	0.053	0.040	0.039	0.058	0.032	0.059	0.000	0.068	0.076
F13	0.060	0.080	0.042	0.034	0.061	0.079	0.047	0.042	0.047	0.023	0.072	0.033	0.000	0.057
F14	0.030	0.028	0.025	0.044	0.042	0.033	0.065	0.042	0.063	0.048	0.042	0.035	0.059	0.000

.

Step 7: With Equation (9), the total relation matrix $T={\left(t_{ij}\right)}_{14\times 14}$ is acquired as shown in

Table 7. The total relation matrix T.

Indicators	F1	F2	F3	F4	F5	F6	F7	F8	F9	F10	F11	F12	F13	F14
F1	0.147	0.276	0.188	0.258	0.276	0.288	0.231	0.238	0.306	0.245	0.282	0.196	0.286	0.326
F2	0.196	0.188	0.171	0.250	0.271	0.278	0.205	0.223	0.293	0.253	0.262	0.194	0.267	0.316
F3	0.134	0.202	0.099	0.186	0.210	0.213	0.152	0.155	0.199	0.163	0.215	0.159	0.221	0.251
F4	0.190	0.249	0.199	0.172	0.251	0.262	0.188	0.199	0.253	0.208	0.257	0.207	0.271	0.303
F5	0.185	0.245	0.167	0.244	0.187	0.263	0.190	0.203	0.285	0.250	0.263	0.180	0.267	0.302
F6	0.169	0.230	0.149	0.211	0.229	0.175	0.181	0.171	0.260	0.229	0.237	0.164	0.250	0.285
F7	0.139	0.211	0.137	0.195	0.194	0.227	0.129	0.175	0.253	0.210	0.215	0.152	0.240	0.268
F8	0.136	0.194	0.129	0.185	0.193	0.213	0.188	0.124	0.241	0.208	0.206	0.158	0.223	0.266
F9	0.161	0.195	0.136	0.196	0.204	0.213	0.176	0.156	0.171	0.216	0.182	0.159	0.234	0.260
F10	0.127	0.179	0.119	0.187	0.194	0.190	0.162	0.177	0.238	0.134	0.176	0.139	0.223	0.255
F11	0.170	0.222	0.153	0.213	0.245	0.233	0.184	0.183	0.258	0.197	0.169	0.161	0.250	0.275
F12	0.142	0.191	0.143	0.197	0.205	0.205	0.160	0.158	0.220	0.169	0.205	0.110	0.226	0.254
F13	0.167	0.223	0.142	0.175	0.210	0.232	0.169	0.164	0.215	0.166	0.222	0.144	0.166	0.241
F14	0.117	0.147	0.107	0.156	0.162	0.160	0.162	0.140	0.197	0.160	0.163	0.125	0.191	0.152

.

Step 8: By utilizing Equations (10) and (11), the sums of rows and columns are calculated respectively. The values of $R+C$ and $R-C$ are determined, as displayed in

Table 8. Sum of influences given and received for each performance indicator.

Indicators	R	C	R + C	R − C
F1	3.542	2.179	5.721	1.363
F2	3.368	2.951	6.319	0.417
F3	2.558	2.038	4.596	0.520
F4	3.210	2.824	6.033	0.386
F5	3.231	3.031	6.262	0.200
F6	2.939	3.154	6.093	−0.215
F7	2.744	2.476	5.221	0.268
F8	2.663	2.465	5.128	0.197
F9	2.658	3.389	6.046	−0.731
F10	2.500	2.807	5.306	−0.307
F11	2.912	3.052	5.964	−0.140
F12	2.584	2.247	4.831	0.337
F13	2.636	3.314	5.950	−0.678
F14	2.136	3.754	5.890	−1.617

. Then, the causal diagram of the 14 performance indicators is drawn, as shown in Figure 2.

4.2. Discussion of the Results

As depicted in Figure 2, the 14 indicators can be divided into a cause group and an effect group according to the value of $R-C$. The indicators contained in the cause group are F₁, F₂, F₃, F₄, F₅, F₇, F₈, and F₁₂, and the remaining indicators belong to the effect group. A larger value of $R-C$ denotes the higher influence of that indictor on the others.

4.2.1. Cause Indicator Analysis

For the cause indicators, they have a net influence on the whole university performance management, and their performance can seriously affect the university. F₁ has the highest value of $R-C$, which means that it exerts a more important influence on the university performance than it receives from other indicators. Therefore, F₁ is a KPI for the performance management of this university in Shanghai.

With respect to F₂, it has the largest value of $R+C$, and its net effect value $R-C$ is the third highest among all the indicators. These indicate that F₂ has a great impact on other indicators in the university performance management. Consequently, the improvement of F₂ will greatly enhance the efficiency of whole system. Thus, F₂ can be identified as a KPI in the given case. Similarly, F₄ and F₅ are KPIs for the performance management of this university in Shanghai.

Although the $R-C$ value of F₃ ranks second among the indicators, both R and C values are not sufficiently high. The low value of $R+C$ indicates that F₃ cannot have a significant impact on the improvement in university performance. Hence, F₃ is not a KPI for the performance management of this university in Shanghai.

4.2.2. Effect Indicator Analysis

For the effect indicators, they are impacted by other indicators. The $R+C$ value of F₆ ranks third among all the indicators. Moreover, both R and C values of F₆ are fairly high, although their $R-C$ value is negative. In other words, the relatively low value of $R-C$ cannot dispute the reality that it has a remarkable impact on the whole system. Thus, F₆ can be recognized as a KPI in this considered case. Similarly, F₁₁ is a KPI for the performance management of this university in Shanghai.

In addition, F₉ has high value of $R+C$, but its value of $R-C$ is −0.731, revealing that it has no significant effect on the other indicators. Moreover, the adjustment of other indicators can lead to the improvement of F₉. Therefore, F₉ is not a KPI for the university performance management. Meanwhile, F₁₃ and F₁₄ are not KPIs for the performance management of this university in Shanghai.

5. Conclusions

In this study, we proposed a new LHF–DEMATEL model to analyze and identify KPIs for university performance management. In this model, the LHFSs were utilized to deal with the uncertain and vague assessment information provided by experts about the interrelationships of performance indicators, and a modified DEMATEL method was employed to identify KPIs for university performance management. Moreover, a GRA-based method was proposed to derive experts’ weights with unknown weight information. Finally, an empirical example was adopted to validate the effectiveness of our proposed LHF–DEMATEL approach. Six KPIs were determined for the considered application, including “quantity and structure of teachers”, “professional abilities”, “funding input”, “number of key disciplines”, “quality of newly established specialty”, and “teachers’ scientific research level”.

In future research, we will focus on the following directions: first, other advanced uncertainty theories, such as unbalanced fuzzy linguistic term sets, can be adopted to handle the uncertainties of experts’ evaluations in the future. Second, in the process of evaluating the interrelationships among performance indicators, experts’ opinions are diverse and may be conflicting. Hence, it is promising to develop a method to eliminate conflicting opinions in university performance management. In addition, the approach developed in this research is general and can be used for identifying KPIs in other fields, and other factors such as intensive human capital losses in the higher education [52,53] can be considered in the university performance management.