Performance Evaluation of Urban Water Environment Treatment PPP Projects Based on Cloud Model and OWA Operator

Abstract

The public-private partnership (PPP) model has become one of the marketization models for water environment treatment projects. Evaluating the performance of these projects is vital for their long-term success. Performance evaluations can inform the government when allocating expenditures for the operation and maintenance of services and can guide the private sector’s operation and maintenance management of projects. By attending to the specific characteristics of urban water environment treatment PPP projects (UWETP-PPP), this study developed a performance evaluation system and corresponding performance evaluation model comprised of eight first-level indicators and fifty second-level indicators. This model was used to evaluate a water environment treatment and ecological restoration project located in Xuchang, China. The results generated by the performance evaluation model indicated that this project was satisfactory and used the PPP model with a very high level of success, which accurately reflected real-world assessments of the project and verified the effectiveness of the model. This research provides guidance for the government in designing a performance evaluation mechanism that implements the specific characteristics of PPP projects. It also provides practical value for the operation management and performance improvement of PPP projects in China.

1. Introduction

Water is an indispensable substance for people’s lives [1]. However, with the rapid development of industrialization and urbanization in China, water ecology in China has been seriously damaged [2]. Problems such as water environmental pollution and water ecological damage have become increasingly prominent [3,4]. Water pollution has become one of the crucial factors restricting Chinese economic and social development [5,6]. The treatment of the urban water environment is a typical infrastructure issue and vital public service, one that involves significant public welfare attributes, large investments, strong professionalism, and high-level requirements for operation and maintenance. Water environment treatment projects are thus an important part of public infrastructure construction projects in China. However, the traditional investment model used to fund such projects puts increased financial pressure on the government, making it difficult to achieve good results in the operation of services. The PPP model is an effective way to solve these problems [7]. It helps to relieve financial pressure on the government, improve the quality of public products, and realize tripartite benefits for the government, the private sector, and the public [8,9].

Performance evaluations are an important basis for the government to provide monetary compensation to the private sector for PPP projects, and are the main basis for evaluating the quality and efficiency of the public goods and services provided by the private sector. However, the current performance evaluation system for water environment treatment PPP projects in China has limitations. Moreover, compensation contracts signed by the government and the private sector have not been closely linked with performance evaluations, further exacerbating the “heavy construction, light management” phenomenon. Therefore, a sound method for evaluating the performance of urban water environment treatment PPP projects (UWETP-PPP) is urgently needed for the industry’s development and will directly affect the sustainability of water environment treatment PPP projects. According to performance objectives, performance evaluations gaging the cost, output, governance effect, and other aspects in the life cycle of PPP projects is the key to their success, and to improving the service level of social public infrastructure [10]. Performance evaluations that can be effectively carried out and evaluation results that can be effectively used are important means to measure the success of a project [11,12]. Furthermore, since the government uses performance evaluation results as the basis for the payment of operating subsidies, and to effectively supervise and motivate the behaviors of the private sector, a sound performance evaluation model will help to realize the goal of improving the quality and efficiency of public services, and will lay a more stable foundation for the sustainable development of PPP projects [13]. There have been some performance evaluations of water environment treatment projects and PPP projects, and researchers have made some progress in the construction of indicator systems and evaluation methods. In performance-based incentive PPP projects, the success of the project is more dependent on the project performance management system and key performance indicators (KPIs). Only when the performance level is scientifically and reasonably evaluated can the project have a positive incentive effect, thus promoting the private sector to improve the operation quality and efficiency of public infrastructure. The existing evaluation index system and model provides a powerful support role for this study, but the evaluation index, index weights, and the evaluation model, did not specifically reflect the water environment governance project concluding water environment, water ecology, water infrastructure, municipal infrastructure, public satisfaction. The evaluation system and model do not consider the differences between qualitative and quantitative indicators, complexity, and ambiguity. Water environment treatment projects are different from other infrastructure projects. They are vital to public welfare and require a high degree of professionalism to effectively operate and maintain. At the same time, PPP projects, as cooperative efforts between the government and the private sector, involve many participants, as well as diversified project portfolios and revenue modes. To evaluate the performance of water environmental treatment projects under the PPP model, it is necessary to consider the government, the private sector, and the public, while also fully respecting the opinions of project participants. A scientific, practical, and universally recognized performance evaluation index system for UWETP-PPP has not been established. However, problems existing in the implementation of such projects have been highlighted in the literature.

Due to the complexity of the project, it is difficult to quantify indicators. Furthermore, there is no standard reference, and it is difficult to evaluate. In general, evaluators tend to use vague language to express their opinions, but this information is difficult to quantify. In the process of data processing, how to ensure that the processed data information objectively reflects the judgment of the evaluation subject and avoid the distortion of evaluation information to the greatest extent is an urgent problem to be solved; how to avoid the deviation of evaluation results by not fully studying the fuzziness and uncertainty of index values. The main research methods for performance evaluation include the Analytic Hierarchy Process [14,15], TOPSIS [15,16], Principal Component Analysis [17,18], and the Neural Network [19,20], but these methods cannot solve the specific problem of UWETP-PPP. In order to make up for the gap in the existing research, it is necessary to conduct performance evaluation research based on the applicability of the performance evaluation model to water environment treatment PPP projects in China. Therefore, this study developed a UWETP-PPP performance evaluation index system. This index system adopted the Ordered Weighted Averaging (OWA) operator weighting method to determine the weight of performance evaluation indicators. Then, by utilizing the cloud model to deal with the uncertainty and fuzziness of the evaluation index values, a UWETP-PPP performance evaluation model was established, and performance evaluation results for UWETP-PPP were calculated by combining specific cases.

The structure of this study is as follows. Section 2 includes a literature review that summarizes research on PPP project performance evaluation index systems and evaluation models. Section 3 introduces the research methods used in this study, including the construction of the performance evaluation system and the corresponding performance evaluation model for UWETP-PPP. Section 4 presents a case study of the applicability of the performance evaluation model to a real-world UWETP-PPP. Section 5 and Section 6 include the discussion, conclusions, and implications of this study.

2. Literature Review

In the process of executing a PPP project, the supervision and overall performance evaluation of the project can determine its success [21,22]. Yu et al. [23] believed that appropriate evaluation indicators and rating methods should be selected for project performance evaluation, and these methods should reflect real-world project evaluation results. Eadie et al. [24] pointed out that PPP project performance evaluations have a significant positive impact on improving project efficiency. Therefore, for PPP projects, it is extremely urgent to design a relatively perfect and scientific performance evaluation system, and to construct an effective and practical performance evaluation model.

2.1. PPP Project Performance Evaluation Index System

A performance evaluation Key Performance Index (KPI) conceptual framework for transit PPP projects has been established. A structured questionnaire and the confirmatory factor analysis method were employed to build a performance evaluation index, which included the satisfaction of the public, government departments, project stakeholders, the project’s schedule, and the complexity of the design [25,26]. Mladenovic et al. [27] summarized a set of important performance indicators for the technical and financial levels of PPP projects based on performance appraisal standards of the public sector and private enterprise. Yuan et al. [28] screened the operational performance indicators of public rental PPP projects through confirmatory factor analysis and the structural equation model; these indicators mainly included housing allocation and recovery efficiency, project spatial distribution, living environment, project financial status, etc. Toor and Ogunlana [29] asserted that it is becoming more important to measure project performance with regards to safety, effective utilization of resources, stakeholder satisfaction, and conflict incidence of public projects, and they incorporated these measures into their indicator system. Liu et al. [30] put forward a performance evaluation system oriented toward the whole life cycle of stakeholders; their system included the project’s critical success factors, the roles and responsibilities of the public sector, the choices of the concessionaire, risk management, different types of cost and time efficiency, and the introduction of value into the system in order to allow the public and the private sector to further improve their performance throughout the whole life cycle of the project. Liu et al. [31] discussed the feasibility of implementing a PPP project life cycle performance evaluation that considers five aspects: the satisfaction of key stakeholders, the project delivery process, the capability of public institutions and private sectors, and the contribution of stakeholders to the project. Negishi et al. [32] constructed an indicator system assessing building technology, the end user, and the external system, and then proposed a whole life cycle performance evaluation framework. In addition, based on the research of Yuan et al. [25], Xiong et al. [33] constructed a performance target system for projects using the reliability of quality, the achievement of budget targets, and the provision of good public services. Using system dynamics, an adjustment model of stakeholder satisfaction was proposed to balance satisfaction among stakeholders. Song et al. [34] identified the influencing factors for the early termination of 11 PPP projects, including government decision-making errors and payment defaults, through multiple case studies. These studies provide some ideas and reference points for constructing a performance evaluation index system of UWETP-PPP.

2.2. PPP Project Performance Evaluation Model

Neely et al. [35] constructed a performance evaluation method utilizing a balanced scorecard and performance prism based on the project performance management process. Mladenovic et al. [27] discussed how project KPI meets the performance objectives of stakeholders. They proposed a two-layer evaluation method for project performance. The first layer evaluates the ultimate goal of the project from the perspective of each stakeholder; namely, the profitability of the private sector, the efficiency and value of the public sector, and the level of service provided to users. The second layer adjusts and weights the realization of specific stakeholder objectives to form a comprehensive evaluation method for PPP project performance. Based on the theory of efficiency, economy, effect, and equity, Cong and Ma [36] constructed a performance evaluation index system for old reconstruction building PPP projects, established an orderly weighted index model, and conducted performance evaluation in conjunction with the cloud model. Luo et al. [13] constructed an index system for shale gas PPP projects accounting for five aspects—including economic benefit, the internal process of the project, innovation and environmental protection, sustainable development, and stakeholder satisfaction—and evaluated the performance of actual projects using AHP and matter element analysis. The above research provides a theoretical basis for the construction of a PPP project performance evaluation model.

2.3. Summary of Research Status

In summary, in terms of PPP project performance evaluation, the methods widely used by scholars include the value for money (VFM) evaluation method and the KPI method, while a few scholars have also adopted the balanced scorecard method and the performance prism model. Scientific and reasonable evaluation methods can reflect project performance as accurately and effectively as possible. Different types of projects should choose different evaluation methods. The research perspectives selected in these studies are also rich, including the perspectives of stakeholders, sustainable project development, and project success. The existing research perspectives provide some references for the development of this study.

Although research on PPP project performance evaluation is mature, most scholars have studied all PPP projects rather than PPP projects in a specific industry. Specifically, scholars have rarely studied the performance of water environment treatment PPP projects. In view of the current dearth of research, the index systems for PPP project performance evaluations have not fully considered the unique characteristics of water environment treatment projects. Considering the complexity and diversity of water environment treatment projects, the evaluation indexes of existing studies are not applicable to them. Though the existing evaluation index systems and models have provided strong support for this study, they still fail to meet the specific requirements for an accurate and efficient performance evaluation of water environment treatment PPP projects. Therefore, it is necessary to construct a performance evaluation system that fully considers both the specific characteristics of water environment treatment projects and the multi-dimensional, multi-source, and multi-agent characteristics of PPP projects to assess the value for money of UWETP-PPP.

3. Methodology

Performance evaluation involves a series of methods, techniques, and tools used to objectively and accurately evaluate project performance. Scientific and reasonable evaluation methods can reflect project performance as precisely and effectively as possible. When evaluating the performance of a project, it is necessary to choose a targeted and feasible evaluation method.

Through a literature analysis, technical specifications, and in-depth interviews, this study preliminarily identifies the indicators for a performance evaluation of UWETP-PPP. In order to reduce the difficulty of modeling and to improve the efficiency of the evaluation, the relative importance index method was used to screen the performance evaluation indicators initially selected. The relative importance index quantifies the qualitative opinions given by experts, avoids the ambiguity of qualitative problems and the influence of subjective consciousness, and meets the needs of the screening evaluation indexes used in this study. This method obtains data through questionnaire surveys or expert interviews and can intuitively judge the importance of indicators. When processing data, it is necessary to ensure that the processed data and information objectively reflect the judgment of the evaluation subject and avoid distortions of evaluation information to the maximum extent. If the fuzziness and uncertainty of the index value are not fully studied, the evaluation results may be biased. The cloud model is very good at dealing with fuzziness and randomness. Therefore, this study uses the cloud model to measure the evaluation index value of indicators.

Due to the complexity and diversity of UWETP-PPP, evaluating their performance involves complex data types from multiple sources, multiple dimensions, and multiple agents. Some indicators are quantitative and can be directly measured. Other indicators are difficult to quantitatively describe, but these qualitative indicators are still very important for constructing an accurate performance evaluation. Therefore, when selecting methods for this study, the advantages of each method in processing qualitative data were considered before being used to carry out the research.

3.1. The Construction of the Performance Evaluation Index System

3.1.1. Selection of the Performance Evaluation Index

Based on the related literature and the guidance of experts on project construction and operation, including all aspects of water environment treatment, the performance evaluation index was preliminarily determined, as shown in

Table 1. Preliminary construction of indicator system.

Evaluation Object	First-Level Indicator	Second-Level Indicator	Source
Performance evaluation of UWETP-PPP A	SPV corporate governance B1	Institutional framework C1-1	[37,38,39]
		Institutional mechanisms C1-2
		Safety management C1-3
		Financial management C1-4
		Human resource management C1-5
	River embankment B2	Embankment crest and flood prevention roads C2-1	[39,40,41,42,43]
		Levee slope and berm C2-2
		Embankment body C2-3
		Embankment crest and flood prevention roads C2-4
		Embankment shoulder C2-5
		River protection works C2-6
		Anti-flood wall C2-7
		Embankment ancillary facilities C2-8
	Water conservancy facilities B3	The main structure C3-1	[39,44]
		Expansion joint C3-2
		Overall status of the hoist C3-3
		Operating status of electromechanical equipment C3-4
		Safety signsC3-5
		Gate bearing and supporting device C3-6
		Surface of strobeC3-7
		Gate chamberC3-8
		Daily maintenance recordC3-9
	River water body B4	Water surface cleaning degreeC4-1	[39,45,46]
		Water transparencyC4-2
		No peculiar smell C4-3
		Ammonia nitrogen contentC4-4
		Total phosphorus content C4-5
		Chemical oxygen demandC4-6
		Pollution source control around shoreline C4-7
		Coastline signsC4-8
	Landscape facilities B5	Rest and recreation facilities C5-1	[39,47,48,49,50]
		Landscape walkway and near water terrace C5-2
		Landscape gallery, landscape pavilion, landscape sculptureC5-3
		The green barrier and cold protection facilities C5-4
		Accessibility facilitiesC5-5
		Garden irrigation facilityC5-6
		Sanitary fixture C5-7
		Boundary stoneC5-8
		Lighting facilityC5-9
		Guard barC5-10
		Garden pathC5-11
		SignboardC5-12
	Garden plants B6	Growth trend C6-1	[39,47,48,49,50,51,52,53]
		Pruning C6-2
		Pest controlC6-3
		Weed control C6-4
		Replanting C6-5
		Irrigate C6-6
		FertilizationC6-7
		Loosen soilC6-8
		Tree hole C6-9
	Bridge B7	Bridge deck gap C7-1	[39,54,55,56,57]
		Telescopic deviceC7-2
		Protecting facilitiesC7-3
		Pedestrian path C7-4
		Drainage facilityC7-5
		Abutment C7-6
		Bridge bearingC7-7
	Public satisfaction index B8	Completeness and comfort of rest and entertainment facilities C8-1	[39,58]
		Hygiene situationC8-2
		Pleasure in surrounding activities C8-3
		Diversity and smoothness of supervision or complaint channelsC8-4
		Comprehensive quality of service personnelC8-5
		Complaint implementation statusC8-6
Meet the needs of life and leisureC8-7
Completeness of safety facilities and signs C8-8
Convenience of lifeC8-9
Degree of coordination between facilities and environmentC8-10
The comfort of the surroundingsC8-11

. It can be seen from the table that the preliminary index system for water environment treatment PPP projects includes eight first-level indicators and sixty-nine second-level indicators. First-level indicators include special purpose vehicles (SPV), corporate governance, river embankments, water conservancy facilities, river water body, landscape facilities, garden plants, bridges, and public satisfaction. Due to the subjectivity of identifying these indicators, experts with extensive experience both in the field of water environment treatment and in PPP projects were invited to screen and revise the indicators.

3.1.2. Questionnaire Survey

To test the reliability of these preliminary indicators, it was necessary to conduct a questionnaire survey about them prior to constructing a final performance evaluation index system for UWETP-PPP. Questionnaire questions were designed according to the initial performance evaluation index and individual scales used in other relevant literature. The questionnaire asked participants to rate index items according to a five-point Likert scale, which ranked the options “Extremely important”, “Very important”, “Moderately important”, “Slightly important”, and “Not at all important” from highest to lowest. The questionnaire was comprised of four parts: the first part provided background information about UWETP-PPP and the purpose of the questionnaire. The second part included basic information about the individuals involved in the survey, although it did not violate personal privacy to maintain the feasibility of the survey results. The third part asked for the respondents’ judgments of the importance of the indicators, which included sixty-nine second-level indicators under eight first-level indicators. The fourth part requested the opinions or the suggestions of the respondents. According to the feedback provided in response to this section, the survey was credible and reliable and the questionnaire was rationally constructed. In addition, according to the different opinions offered by the experts, it was possible to check for omissions in the indicators and to fill necessary gaps.

The targeted respondents of this survey were from government departments, employees of SPV companies, experts in the PPP project field, employees of a design group, and employees of an investment group. A total of 30 questionnaires were issued and all received responses. The characteristics of the sample data obtained through statistical analysis are shown in

Table 2. Characteristics of the questionnaire data.

Characteristics	Distribution	Frequency	Percentage
Type of enterprise	Relevant government department	2	6.67%
	Research institution	6	20.00%
	Investment corporation	5	16.67%
	Design enterprise	4	13.33%
	SPV project company	5	16.67%
	Construction enterprise	5	16.67%
	Other	3	10.00%
Number of PPP projects involved	0–1	5	16.67%
	2–3	16	53.33%
	4–5	7	23.33%
	Above 6	2	6.67%
Years of working experience	Under 1 year	3	10.00%
	2–3 years	8	26.67%
	4–5 years	14	46.67%
	Above 6 years	5	16.67%

.

3.1.3. The Relative Importance Index Value of the Evaluation Index

According to the characteristics of the performance index system and previous studies, the Relative Importance Index (RII) method was adopted in this study for screening. The RII was calculated according to the following equation to evaluate and measure the importance of the selected evaluation indexes:

$$RI{I}_x=100\times \frac{Q_{x1}\times 1+Q_{x2}\times 2+Q_{x3}\times 3+Q_{x4}\times 4+Q_{x5}\times 5}{5\times Q}$$

(1)

$RI{I}_x$ = the importance index value of indicator X; $Q$ = the total number of questionnaires; $Q_{x1}$ = the number of people with an evaluation score of “1” for indicator X; $Q_{x2}$= the number of people with an evaluation score of “2” for indicator X; $Q_{x3}$ = the number of people with an evaluation score of “3” for indicator X; $Q_{x4}$ = the number of people with an evaluation score of “4” for indicator X; $Q_{x5}$ = the number of people with an evaluation score of “5” for indicator X.

The larger the $RI{I}_x$ value is, the more important the indicator is, and the more it should be retained. This study took 80 as the reference value, and indicators < 80 were deleted. It is appropriate to take 80 as the benchmark value because doing so ensures that the selected indicators are important and that important indicators have been selected [39,59]. It is assumed that the survey data conforms to the normal distribution because the normal distribution is a natural distribution. According to the central limit theorem, if the data reaches a certain amount, it will conform to the normal distribution. It can be seen that 95% of the data are within the interval ($\mu -2\sigma ,\mu +2\sigma$), where $\mu$ represents the mean value of the questionnaire data and $\sigma$ is the standard deviation of the data. The calculation equation of variation degree of data $\lambda$ is as follows:

$$\lambda =\frac{\sigma }{\mu }$$

(2)

In Equation (2), $\lambda$ indicates the degree of dispersion of the results. The greater the value of $\sigma$, the greater the degree of dispersion of the data, and thus the greater the difference between respondents’ opinions. In this study, 0.3 was taken as the reference value, and $\lambda$ indexes with a value of greater than 0.3 were deleted [59].

3.1.4. Determine the Performance Evaluation Index System

Through the questionnaire, the evaluation value of index importance for UWETP-PPP was determined. The data was calculated by Equations (1) and (2), and indicators with an RII value greater than 80 were retained. The final performance evaluation index system for UWETP-PPP is shown in Figure 1.

3.2. The Construction of the Performance Evaluation Model

3.2.1. Performance Evaluation Index Value

To create a performance evaluation model for UWETP-PPP, the first and most important thing is to determine the performance evaluation index value. Data can be obtained in three ways: expert evaluation, instrumental monitoring, and public surveys. This study adopted the method of the cloud model to conduct dimensionless and standardized treatment of the indicators. This method can solve the problem of the fuzziness and randomness of the evaluation index value. Qualitative and quantitative descriptions of UWETP-PPP can be combined by using the expected value $Ex$, entropy value $En$, and super-entropy $He$. The expected value $Ex$ is the most representative, signifying the expected value of the spatial distribution of the performance level. The entropy value $En$ measures the uncertainty of the performance level and represents the degree of dispersion. Super-entropy $He$ refers to the entropy of entropy and measures the uncertainty of entropy, which can comprehensively reflect the fuzziness and randomness of entropy.

Due to the characteristics of the quantitative performance evaluation index and qualitative performance evaluation index in the performance evaluation system, the values of the two indexes must be determined using different methods. The uncertain reasoning of the cloud is applicable to the determination of the quantitative index, while the determination of the qualitative index uses forward and reverse cloud generators.

(1)

Determination of the quantitative performance evaluation index values based on cloud uncertainty reasoning

In the performance evaluation of UWETP-PPP, let $B=\left\{B_1,B_2,\dots ,B_m\right\}$ be the set of first-level indicators and let $C_i=\left\{C_{i1},C_{i2},\dots ,C_{ij}\right\} \left(i=1,2,\dots ,m;j=1,2,\dots ,n\right)$ be the set of second-level indicators. The detailed steps for determining the quantitative performance evaluation index value based on cloud uncertainty reasoning are as follows:

Step 1: Make the comments set according to the quantitative performance evaluation index of UWETP-PPP.

Based on existing professional knowledge and accumulated experience, the evaluation comments on the secondary indicators of UWETP-PPP are expressed by $L_t$, which is denoted as $L=\left\{L_1,L_2,\dots ,L_t\right\}$.

Step 2: Build a cloud model for the evaluation set of quantitative performance evaluation indicators.

The cloud model characteristics of quantitative performance evaluation comments $L_t$ of UWETP-PPP can be expressed as $M_L\left(E{x}_{ij}{}^{L_t},E{n}_{ij}{}^{L_t},H{e}_{ij}{}^{L_t}\right)$, which can be obtained according to existing literature and consulting experts.

Step 3: Determine the comments on PPP projects’ quantitative performance evaluation indexes for UWETP-PPP.

It is assumed that the attribute value of the quantitative performance evaluation index of UWETP-PPP is denoted by $x_{ij}$, and the forward cloud generator is used to calculate the membership degree ${\delta }_{ij}$ of $x_{ij}$ to $M_L\left(E{x}_{ij}{}^{L_t},E{n}_{ij}{}^{L_t},H{e}_{ij}{}^{L_t}\right)$. If ${\delta }_{ij}<{\delta }_{ij}{}^{L_e}$, then the corresponding comment of the second-level indicator $C_{ij}$ of PPP project performance evaluation is $L_e$.

Step 4: Establish the evaluation set of the quantitative performance evaluation index score and the corresponding cloud model of UWETP-PPP.

Set $Q$ as the evaluation set of the quantitative performance evaluation index score, which can be denoted as $Q=\left\{Q_1,Q_2,\dots ,Q_k\right\}$. The digital characteristics of the cloud model corresponding to the rating set $Q$ of UWETP-PPP are $M_Q\left(E{x}_{ij}{}^{Q_k},E{n}_{ij}{}^{Q_k},H{e}_{ij}{}^{Q_k}\right)$.

Step 5: Determine the quantitative performance evaluation index value $v_{ij}$ of UWETP-PPP.

The quantitative performance evaluation index value of UWETP-PPP is determined according to the uncertainty of the cloud model. The determined form is if $v_{ij}\in M_t$, then the result is $v_{ij}\in Q_k$ ($t$ and $k$ have a one-to-one correspondence).

(2)

Determine the qualitative performance evaluation index value based on the forward and reverse cloud generator

On the theoretical basis of the Delphi method, the forward and reverse cloud generator can be used to determine the qualitative performance evaluation index value of UWETP-PPP. The specific steps are as follows. First, the qualitative indicators are scored by experts, and then the digital characteristics $M\left(E{x}_{ij},E{n}_{ij},H{e}_{ij}\right)$ of the cloud model are analyzed using the reverse cloud generator. Next, the normal cloud image is obtained using the forward cloud generator. The thickness of the cloud image obtained from the analysis was observed, and experts were informed of the results. After that, scores were repeated until the thickness of the cloud image reached the ideal value, and then the final digital characteristics $M^{\prime }\left(E{x}_{ij}{}^{\prime },E{n}_{ij}{}^{\prime },H{e}_{ij}{}^{\prime }\right)$ of the cloud model were obtained, in which $E{x}_{ij}{}^{\prime }$ represented the qualitative performance evaluation index value $v_{ij}$.

3.2.2. Performance Evaluation Model Based on OWA Operator

According to the performance evaluation system and the evaluation value of UWETP-PPP constructed above, the OWA operator weighting method was adopted to calculate the weights of first-level indicators and second-level indicators of the performance evaluation of UWETP-PPP. The OWA operator can reduce or even completely eliminate subjective influences, and the weight of indicators can be measured according to the location characteristics of data [60]. Then, the index evaluation value calculated by the cloud model and the index weight calculated by the OWA operator can be combined to obtain the performance evaluation results. Set $B=\left\{B_1,B_2,\dots ,B_m\right\}$ as the first-level index set for the performance evaluation of UWETP-PPP and $C_i=\left\{C_{i1},C_{i2},\dots ,C_{ij}\right\} \left(i=1,2,\dots ,m;j=1,2,\dots ,n\right)$ as the second-level index set. The detailed steps are as follows:

Step 1: Score by experts.

Some experts were selected to score the first-level and second-level indicators of the performance evaluation of UWETP-PPP, and the dataset of the scoring results $\left\{e_1,e_2,\dots ,e_d\right\}$ was obtained. The data were sorted from largest to smallest, starting from 0, and the ranking results $g_0\ge g_1\ge \cdots \ge g_k\ge \cdots g_{d-1}$ were obtained.

Step 2: Position empowerment.

According to the ranking results, the scoring data set $\left\{g_1,g_2,\dots ,g_d\right\}$ of the first-level and second-level indicators of the performance evaluation of UWETP-PPP was given a weight, and the weight vector ${\beta }_{k+1}$ was obtained as follows:

$${\beta }_{k+1}=\frac{C_{d-1}^k}{{\displaystyle \sum _{i=0}^{d-1}{C}_{d-1}^i}}=\frac{C_{d-1}^k}{2^{d-1}},k=0,1,\cdots ,d-1$$

(3)

Step 3: Determine the absolute weight.

The weighting vector ${\beta }_{k+1}$ was used to weight the scoring dataset $\left\{g_1,g_2,\dots ,g_d\right\}$, and the absolute weight values ${\eta }_i{}^{\prime }$ and ${\eta }_{ij}{}^{\prime }$ of the first-level indicators $B_i$ and second-level indicators $C_{ij}$ of the performance evaluation of UWETP-PPP were obtained as follows:

$${\eta }_i{}^{\prime }\left({\eta }_{ij}{}^{\prime }\right)={\displaystyle \sum _{k=1}^d{\beta }_k}{g}_k,k\in \left[1,d\right]$$

(4)

Step 4: Determine the relative weight.

According to the absolute weight of the first-level index $B_i$ of the performance evaluation of UWETP-PPP, the relative weight value ${\eta }_i$ and ${\eta }_{ij}$ of the first-level index $B_i$ and the second-level index $C_{ij}$ of the performance evaluation were calculated:

$${\eta }_i\left({\eta }_{ij}\right)=\frac{{\eta }_i{}^{\prime }}{{\displaystyle \sum _{i=1}^m{\eta }_i{}^{\prime }}},i=1,2,\dots ,m$$

(5)

Thus, the weight results of all levels of indicators in the performance evaluation of UWETP-PPP can be obtained.

Step 5: Determine the performance evaluation results.

According to the evaluation values and weights of the quantitative and qualitative indicators of the performance evaluation of UWETP-PPP, the performance evaluation results of project $X$ were calculated as follows:

$$X={\displaystyle \sum {\displaystyle \sum {\nu }_{ij}}}{\eta }_{ij}{\eta }_i$$

(6)

${\nu }_{ij}$ is the performance evaluation value obtained from the second-level performance evaluation index of UWETP-PPP, and ${\eta }_{ij}$ is its corresponding weight.

The framework of the research method is shown in Figure 2.

Firstly, according to the characteristics of UWETP-PPP, the performance evaluation indicators were initially identified using a literature analysis, technical specifications, and in-depth interviews. Through the questionnaire survey, the relative importance index method was adopted to determine the final performance evaluation indicator system. Then, cloud uncertainty reasoning was used to determine the evaluation index value of the quantitative index, and positive and reverse cloud generators were used to determine the evaluation index value of the qualitative index. Next, the OWA operator weighting method was used to calculate the index weight, and finally the performance evaluation model of UWETP-PPP was constructed. Accordingly, the construction of the indicator system and the evaluation model have been improved in the performance evaluation method, which provides methodological references for the performance evaluation of other projects and can be promoted in other fields.

4. Case Study

4.1. Case Description

This research used a UWETP-PPP in Xuchang, China, as a case study to demonstrate the validity of the proposed performance evaluation model. The water environment treatment and ecological restoration project of Xuchang, China, is one of the key construction projects of Xuchang, China, which is one of forty-five pilot cities for water ecological civilization construction. The research region of the UWETP-PPP in Xuchang is shown in Figure 3 [58]. This project is a typical water environment treatment PPP project, and it can serve as a valuable demonstration of the validity of the model. Using this project as an example while carrying out performance evaluation research provides a reference for evaluating similar projects. In addition, the authors of the present study are deeply involved in the project, and can therefore obtain real and reliable information regarding it.

In 2014, the water environment treatment and ecological restoration project in Xuchang, China, was launched. The water ecological comprehensive management project was divided into the Xueyin River comprehensive treatment project, the Qingni River Basin comprehensive treatment project, and the Qingyi River comprehensive treatment project. The construction period was two years, from June 2014 to June 2016. After the construction was completed, the project entered the franchise period, which will last for 15 years. The local government authorized the Xuchang Water Construction Investment and Development Limited Company to sign a contract with a party from the private sector, who won the bid to jointly fund the establishment of the project company. The project company is responsible for the investment, financing, construction, operation, and maintenance of the project. The Xuchang Municipal Government pays fees to the project company according to the availability of services and annual operation, as well as for maintenance fees for the services provided by the project company.

4.2. Data Sources and Pretreatment

4.2.1. Data Sources

According to the performance evaluation index system of UWETP-PPP constructed above, the index data sources can be divided into three parts:

The first part is expert evaluation data. The performance evaluation value for SPV governance, some indicators of river water such as water conservancy facilities, river water, garden plants, bridges and garden facilities, and other qualitative indicators should be scored by invited experts. Experts from water conservancy, municipal administration, garden, finance, law, and other related industries should be invited to respond to questionnaires to obtain evaluation data.

The second part is monitoring data. Data related to water quality quantitative indexes for river water mainly include COD, NH₃-N, and TP, which can be obtained by monitoring water quality sensors.

The third part is public satisfaction. Evaluation data regarding the public satisfaction index should be obtained by inviting the public to respond to the questionnaire.

4.2.2. Performance Evaluation Data Processing Based on the Cloud Model

(1)

Determine the quantitative performance evaluation index value based on cloud uncertainty reasoning

With the help of expert experience and the Standard for Environmental Quality of Surface Water (GB 3838-2002), as well as other norms and standards, the qualitative evaluation set and the cloud model can be determined according to the calculation results obtained above. For example, the natural attribute value of the “ammonia nitrogen (NH₃-N)” index is 0.43. Let the evaluation set of the quantitative performance evaluation index corresponding to the qualitative evaluation index be {very poor, poor, medium, good, excellent}, then the corresponding digital characteristics of the cloud model are (2.00, 1.85/6, 0.1), (1.50, 1.85/6, 0.1), (1.00, 1.85/6, 0.1), (0.50, 1.85/6, 0.1), (0.15, 1.85/6, 0.1) for the ammonia nitrogen (NH₃-N) indicator. The cloud models of other quantitative performance evaluation indicators are similar.

The qualitative comments corresponding to the quantitative performance evaluation index scores of the cloud model are {extremely low, very low, medium, very high, extremely high}, and the corresponding digital characteristics of the cloud model are (15, 80/6, 0.2), (35, 80/6, 0.2), (55, 80/6, 0.2), (75, 80/6, 0.2), and (95, 80/6, 0.2), respectively. Furthermore, qualitative descriptions of the water environment treatment and ecological restoration project in Xuchang, China, were transformed into a percentage system. Taking “ammonia nitrogen (NH₃-N)” as an example, the indefinite reasoning process is as follows:

• If ammonia nitrogen (NH₃-N) is “very poor”, then the score is “extremely low”;
• If ammonia nitrogen (NH₃-N) is “poor”, then the score is “very low”;
• If ammonia nitrogen (NH₃-N) “medium”, then the score “medium”;
• If ammonia nitrogen (NH₃-N) is “good”, then the score is “very high”;
• If ammonia nitrogen (NH₃-N) is “excellent”, then the score is “extremely high”.

Repeat the above steps to obtain other quantitative performance evaluation index values for the water environment treatment and ecological restoration project in Xuchang, China, as shown in

Table 3. Performance evaluation index value and weight of water environment treatment and ecological restoration project in Xuchang.

Index	C1-1	C1-2	C1-3	C1-4	C2-1	C2-2	C2-3	C2-4	C2-5
Index value	85.13	84.25	80.13	82.79	78.81	75.63	80.13	79.46	75.15
Weight	0.2371	0.1934	0.3681	0.2014	0.2979	0.2175	0.2135	0.1320	0.1391
Index	C3-1	C3-2	C3-3	C3-4	C3-5	C3-6	C4-1	C4-2	C4-3
Index value	71.35	84.31	87.69	75.67	80.17	84.39	85.36	88.69	78.43
Weight	0.2113	0.1176	0.1141	0.0690	0.3903	0.0977	0.1321	0.1200	0.1080
Index	C4-4	C4-5	C4-6	C4-7	C5-1	C5-2	C5-3	C5-4	C5-5
Index value	64.50	84.17	78.34	69.74	86.14	85.79	82.34	87.63	88.76
Weight	0.2120	0.2130	0.1378	0.0771	0.0898	0.3141	0.1420	0.0936	0.0782
Index	C5-6	C5-7	C5-8	C5-9	C6-1	C6-2	C6-3	C6-4	C6-5
Index value	81.31	84.33	79.68	90.13	91.35	90.16	94.78	89.46	86.79
Weight	0.1072	0.0319	0.0762	0.0670	0.1401	0.1945	0.1529	0.1536	0.1299
Index	C6-6	C7-1	C7-2	C7-3	C7-4	C7-5	C8-1	C8-2	C8-3
Index value	91.49	84.17	80.11	79.64	86.75	89.18	91.14	86.79	90.15
Weight	0.2290	0.2796	0.2125	0.1240	0.1836	0.2003	0.1334	0.1127	0.1224
Index	C8-4	C8-5	C8-6	C8-7	C8-8
Index value	79.13	84.79	82.43	93.17	89.96
Weight	0.1068	0.1235	0.1137	0.1548	0.1327

.

(2)

Determine the qualitative performance evaluation index value based on the forward and reverse cloud model

The qualitative performance evaluation index values for the water environment treatment and ecological restoration project in Xuchang, China, need to be evaluated and determined by adopting a forward and reverse cloud generator according to the Delphi method. Due to existing conditions, 20 experts and 50 members of the public were selected to score the qualitative index. In order to ensure the professionalism of experts, experts who were closely related to the content of this study were selected. The selection principles for experts included the following: the richness of their practical experience in UWETP-PPP; if they are now, or have ever been, directly involved in the management and operation of UWETP-PPP; and the depth of their understanding and research on the performance evaluation of UWETP-PPP. At the same time, in order to reach a relatively objective and neutral conclusion, the need to balance the positions of the respondents, including PPP experts from different departments such as the government, enterprises, and academic institutions, were considered when selecting samples. To use the expert consultation method, the number of experts should be statistically significant, and, in order to facilitate statistics, the number of experts should not be too large. So, 20 experts were selected as respondents for this study. The evaluation of public satisfaction with UWETP-PPP is a part of public participation, and the effects of UWETP-PPP are evaluated through assessing the intuitive feelings of the public. The principle of public selection was to select people who live near the project and have a certain degree of education or are well educated. Ultimately, 50 members of the public were selected, which was also statistically significant. The public satisfaction score is reported in the form of a percentage system, and the specific scoring criteria are as follows. A score of 60 or less indicates that the performance level is extremely low, which means that the main goal has not been achieved; a score between 60 and 70 indicates that the level is very low, which means that only a small part of the main goal has been achieved; a score of 70 to 80 is at a medium level, which means that some of the main goals have been achieved; a score of 80 to 90 indicates that the level is very high, which means that the key goals have been achieved; a score of 90 or more corresponds to an extremely high level, which means that the all of the expected goals have been achieved or that more than the expected goals have been achieved.

In this study, the index of “operating status of electromechanical equipment” was used as an example to analyze and collate the scoring results of the 20 experts. Forward and reverse cloud generators were respectively used to obtain the digital characteristics of the cloud model, which were (75.67, 0.64, 0.04), and the obtained mathematical expectation $E_x=75.67$ was the index value. The specific process was as follows. First, 20 experts scored the “operating status of electromechanical equipment” index for the water environment treatment and ecological restoration project in Xuchang, China. Initially, the scoring gap was very large, and the characteristic values obtained were also very large. The cloud map was thick and foggy. Then, the experts were informed of the initial variations in the scoring and they scored the index again. The digital eigenvalues began to decrease, and the cloud image gradually thinned and began to present a normal cloud shape. At that time, the new scoring results were reported back to the experts. The eigenvalues decreased again, and the thickness of the cloud image significantly decreased, showing a relatively obvious normal distribution. Other qualitative performance evaluation index values for the project were determined in the same manner. For public survey data, due to the randomness of the public, the performance evaluation index values were determined by one round of scoring. The above steps were repeated to obtain the qualitative performance evaluation index values, as shown in Table 3.

4.3. Results of Performance Evaluation

The OWA operator weighting method was used to determine the weight of each index for the water environment treatment and ecological restoration project in Xuchang, China. Taking a first-level performance evaluation indicator as an example, five experts were selected to score indicator B₁ to obtain the dataset {8.0, 7.0, 7.5, 7.0, 7.5}. The dataset was sorted, resulting in the new dataset {8.0, 7.5, 7.5, 7.0, 7.0}. When the data was put into Equation (4) for calculation, the absolute weight value of indicator B₁ was calculated as 7.46. Similarly, the absolute weight values of B₂, B₃, B₄, B₅, B₆, B_7, and B₈ were 7.84, 6.02, 9.43, 8.06, 9.85, 6.13, and 7.90, respectively. Finally, according to Equation (5), the relative weights of the first-level performance evaluation indicators were determined: $w_{B1}=0.1190$, $w_{B2}=0.1251$,$w_{B3}=0.0960$, $w_{B4}=0.1504$, $w_{B5}=0.1286$, $w_{B6}=0.1571$, $w_{B7}=0.0978$, and $w_{B8}=0.1260$. The steps to determine the weights of the second-level performance evaluation indicators were the same as above, and the results are shown in Table 3.

According to the performance evaluation index values of the second-level indicators, their corresponding weights, and Equation (6), the final performance evaluation score of the Xuchang Water Environment treatment PPP Project is 83.48, and the performance evaluation grade is very high. Based on the real-world performance of the project, the government considers that it has been smoothly carried out, achieved its relevant goals, and avoided substandard treatment, as well as high financial expenditure pressure. Therefore, the government also considers the project’s performance to be at a very high level. This project is a model for other water environment management projects, and experts and scholars have visited it many times. Since the project has been shown to be successful overall, the performance evaluation grade assigned by this study is consistent with actual determinations about the project’s performance, which verifies the effectiveness of the model.

5. Discussion

Through a literature review, a questionnaire, expert interviews, and the relative importance index method, this study determined a performance evaluation index system for UWETP-PPP, including eight first-level indicators and a total of fifty second-level indicators. Among them, river water bodies (B₄) and garden plants (B₆) are two indicators with relatively larger weights, indicating that these two indicators are relatively more important. The color, odor, and clarity of the water can be used as indicators reflecting the water quality of the river. This is consistent with the findings of Bengraine and Marhaba [61], who determined the impact of these factors on water quality. Bad odors affect mental wellness and health, and poor color indicates low water quality. The river water body is an important part of water environment treatment projects, and it should be used as an index with greater weight in evaluating the performance of such projects. Plants can regulate the temperature and humidity of the air and bring physical and psychological comfort to the public. The design and maintenance of plants needs to prioritize aesthetics to meet people’s ornamental needs [62]. Plants, as an intuitive part of public satisfaction, are also one of the most important indicators for evaluating water environment treatment projects.

After comparing the existing research on the performance evaluation of PPP projects, Forrer et al. [63] proposed a framework based on six dimensions: cost, benefit, social and political influence, skills, collaboration and performance measurement. Grossman [64] proposed a performance evaluation model that aimed to accurately evaluate PPP projects in a complete and comprehensive way, evaluating indicators such as quality of life, return on investment, and management ability. Cappellaro and Ricci [65] established a PPP performance evaluation framework based on four dimensions: finance, investment, process, and results. Previous studies on the performance of PPP projects primarily considered the social, environmental, and economic benefits of the projects. Considering the complexity and diversity of UWETP-PPP, the evaluation indexes of the existing studies are not suitable to be applied to UWETP-PPP. It is necessary to account for both the characteristics of PPP projects and contract characteristics to carry out targeted research on the establishment of a performance evaluation index system of UWETP-PPP. Therefore, this study constructed a performance evaluation system of UWETP-PPP, which enriches the research on how to effectively evaluate water environment treatment projects and PPP projects. Scholars’ focus on all types of PPP projects, which makes their research wide ranging, but it does not consider the unique attributes of specific industries. This study took water environment treatment PPP projects as the evaluation object, which not only expands the research on water environment treatment, but also enriches the research on PPP project performance.

Performance evaluation is an important means for improving the effectiveness of project implementation. Performance evaluation regulates all aspects of project construction and implementation, helps to analyze what is going right and what is going wrong in the project process, summarizes the reasons for successes and failures, and clarifies the responsibilities of relevant managers. Therefore, performance evaluations of the PPP model in water environment treatment projects play an important role in improving project quality and management. The performance evaluation index system and evaluation model constructed in this study provide a basis for the government to determine monetary compensation for project partners according to the performance of the project during the operation period of the UWETP-PPP, and it also provides a reference for conducting performance evaluations of actual water environment treatment PPP projects in the future. A sound performance evaluation system and model are conducive to government supervision and can also promote the active operation of projects by the private sector. As a result of accurate and objective performance evaluations, project efficiency is significantly improved, contract risks are reduced, and sustainable development is supported for PPP projects.

6. Conclusions

In recent years, UWETP-PPP have entered the fast track in terms of their development. However, the imperfect system for evaluating the performance of UWETP-PPP has directly affected the smoothness of their development long term. Therefore, this study, based on the specific characteristics of UWETP-PPP, constructed a water environment treatment performance evaluation system and evaluation model for PPP projects, enriching the theory of project performance evaluation, providing a foundation and basis for performance incentive mechanism design, aiding government regulation, and providing theoretical support to the private sector. The performance evaluation developed in this study is therefore helpful for promoting the goal of improving the quality and efficiency of public services and laying a solid foundation for the sustainable development of UWETP-PPP in the future.

In view of the problems caused by imperfect performance evaluation systems and the lack of constraints in project operation and maintenance, this study first identified performance evaluation indicators based on the specific characteristics of UWETP-PPP through a literature analysis, consideration of technical specifications, in-depth interviews, and a questionnaire survey. The relative importance index method was used to determine the final performance evaluation system. Then, cloud of uncertainty reasoning was used to determine the quantitative indexes of the evaluation indexes, and positive and reverse cloud generators were used to determine the qualitative index of the evaluation indexes. Using empowerment, the OWA operator method was used to calculate the index weight, the index of the cloud model calculation value, and OWA operator performance evaluation results were eventually generated. Finally, the proposed performance evaluation system was applied to a real-world water environment treatment and ecological restoration project in Xuchang, China. The evaluation grade of the proposed performance evaluation system was essentially consistent with actual assessments of the project, verifying the effectiveness of the model.

The main contributions of this study are as follows: Firstly, the relative importance index method was used to quantify the qualitative opinions of experts, making it possible for experts to intuitively judge the relative importance of indicators and allowing for the selection of key evaluation indicators in order to build a systematic performance evaluation system for UWETP-PPP. Secondly, due to the advantage of the cloud model in dealing with the uncertainty and fuzziness of the evaluation index value, the cloud model was used to measure the evaluation index value of the qualitative index, ensuring that the processed data objectively reflected the judgment of the evaluation subject. Thirdly, the OWA method was used to obtain the weight of indicators only related to position, ensuring strong objectivity and obtaining the index weight of the UWETP-PPP performance evaluation.

In this study, a performance evaluation model for UWETP-PPP projects was built, and the stated research results were achieved. However, this study still needs to engage in in-depth discussions and conduct systematic research on the following issues. UWETP-PPP is a typical performance-based payment contract. The rationality of the payment amounts is based on accurately measuring and evaluating project performance. The premise of a performance evaluation is that it has constructed a scientific and effective index system and can obtain accurate measurement data. However, UWETP-PPP performance monitoring data is not unified across space and time. All kinds of assets evolve differently over time, and performance targets change with the progress of society and civilization. If a performance evaluation model cannot effectively integrate performance monitoring data, the index system cannot be updated, and it will not be able to reflect the actual performance of the project. This study only considers qualitative and quantitative indicators, but it does not consider data fusion or allow for dynamic updates. Therefore, future research should explore the data fusion method for data that is from multiple sources, multi-dimensional, multi-spatial, and temporal, and build a performance evaluation system that can be dynamically adjusted to reflect the dynamic changes of UWETP-PPP.