Evaluating the Eco-Efficiency of Wastewater Treatment Plants: Comparison of Optimistic and Pessimistic Approaches

Abstract

The assessment of wastewater treatment plant (WWTP) performance has gained the interest of water utilities and water regulators. Eco-efficiency has been identified as a powerful indicator, as it integrates economic and environmental variables into a single index. Most previous studies have employed traditional data envelopment analysis (DEA) for the evaluation of WWTP eco-efficiency. However, DEA allows the selection of input and output weights for individual WWTPs for the calculation of eco-efficiency scores. To overcome this limitation, we employed the double-frontier and common set of weights methods to evaluate the eco-efficiency of a sample of 30 WWTPs in Spain. The WWTPs were ranked based on eco-efficiency scores derived under several scenarios including best- and worst-case scenarios; this approach to performance assessment is reliable and robust. Twenty-six of the 30 WWTPs were not classified as eco-efficient, even under the most favorable scenario, indicating that these facilities have substantial room for the reduction of costs and greenhouse gas emissions. The ranking of WWTPs varied according to the scenario used for evaluation, which has notable consequences when eco-efficiency scores are used for regulatory purposes. The findings of this study are relevant for water regulators and water utilities, as they demonstrate the importance of weight allocation for eco-efficiency score estimation.

1. Introduction

Wastewater treatment is essential for the protection of human health and environmental sustainability [1]. According to the United Nations Children’s Fund–World Health Organization [2], in 2017 96% of the entire population of developed regions had access to wastewater treatment, carried out in wastewater treatment plants (WWTPs), which are complex and resource-intensive facilities [3]. As the number of WWTPs has increased worldwide and sustainability issues have become more relevant for policy- makers, WWTP performance assessment has gained the interest of WWTP managers and water authorities [4,5,6]. Two main methodologies have been employed for such assessment: decision support systems [7,8,9] and the integration of key performance indicators into a synthetic index of performance [10,11,12]. Data envelopment analysis (DEA) has been employed widely to evaluate the efficiency of WWTPs due to its positive features, which include: (i) the integration of several inputs and outputs into a synthetic efficiency index; (ii) the lack of restriction on the measurement units of inputs and outputs; (iii) the lack of requirement for functional relationships between inputs and outputs, and iv) the ability to estimate the efficiency of units relative to a given dataset [3,13].

The performance and efficiency of WWTPs are related closely to the pollutant removal efficiency, resource consumption, and environmental impacts such as greenhouse gas (GHG) emissions [14]. Thus, as an extension of efficiency assessment, the eco-efficiency of WWTPs has recently been assessed [14,15,16,17,18]. Schaltegger and Sturm [19] first defined eco-efficiency as the ratio between the value added and the environmental impact. This concept entails the production of more goods and services with fewer resources and a minimal environmental impact [20]; the prefix “eco” represents ecological and economic dimensions [21]. For WWTPs, eco-efficiency entails the removal of more pollutants from wastewater while incurring fewer economic costs and emitting fewer GHGs [16,17,18].

From a methodological point of view, DEA has been used to evaluate the eco-efficiency of WWTPs by considering the removal of pollutants as desirable outputs, economic costs as inputs, and GHG emissions as undesirable outputs [16]. Thus, DEA enables integration of the three dimensions of eco-efficiency (service value, resource consumption, and environmental impacts) into a synthetic index of performance [22]. However, the use of DEA for eco-efficiency evaluations also has limitations. DEA allows selection of the weights of (desirable and undesirable) inputs and outputs for the estimation of eco-efficiency scores. Hence, the same variable may be weighted differently to ensure the most favorable result. From a policy perspective, this practice may mean that evaluation results are not accepted by all WWTPs. An additional limitation related to different weight allocation is that more than one WWTP can be considered to be eco-efficient, with no further discrimination possible [23]. To our knowledge, these limitations have not been considered in the few previous studies of the evaluation of WWTP eco-efficiency.

To overcome the limitation of different weights allocation in DEA, Sexton et al. [24] proposed the incorporation of common weights via the cross-efficiency and common set of weights (CSW) methods [25]. Subsequently, Wang et al. [26] and Jahed et al. [27] simplified the cross-efficiency method to the double-frontier DEA method, which measures the efficiency of units from optimistic and pessimistic perspectives (i.e., in best- and worst-case scenarios), thereby integrating uncertainty into the assessment. With the CSW approach, the same weight is allocated to the variables (inputs and outputs) for all units evaluated, resulting in objective evaluation. Several DEA models [28,29,30] have been developed based on the CSW approach. Chen et al. (2018) [31] extended the CSW model to create a double-frontier CSW model for evaluation from optimistic and pessimistic points of view with the integration of undesirable outputs. In other words, this approach enables the estimation of WWTP eco-efficiency with consideration of common weights for all WWTPs evaluated from optimistic and pessimistic perspectives. It improves the discrimination power of eco-efficiency assessment and ranking of WWTPs.

The objectives of this paper are twofold. The first objective is to evaluate and compare the eco-efficiency of a sample of WWTPs considering four scenarios: (i) optimistic with different weights for the WWTPs, (ii) pessimistic with different weights for the WWTPs, (iii) optimistic considering CSW, and (iv) pessimistic considering CSW. This approach enables analysis of the impact of common weight allocation on the efficiency of WWTPs. The second objective is to rank the WWTPs based on their optimistic and pessimistic CSW eco-efficiency scores. This information is very relevant for policy-makers and water authorities, as it permits the benchmarking of WWTP performance based on eco-efficiency scores integrating uncertainty. To our knowledge, the CSW method and double-frontier DEA have not been applied to evaluate WWTP eco-efficiency in a previous study. This paper contributes to the literature on WWTP performance assessment by providing a robust method for the evaluation of WWTP eco-efficiency and objective ranking of facilities, which can facilitate regulation and support decision-making by WWTP managers and water authorities.

2. Methodology

2.1. Double-Frontier DEA for Optimistic and Pessimistic Assessments of Eco-Efficiency

Assume that $n$ units (WWTPs in this study) are using $m$ inputs to produce $s$ desirable outputs and $h$ undesirable outputs. The input, desirable output, and undesirable output values for $WWT{P}_j$ $\left(j=1,\dots ,n\right)$ are denoted by $x_{ij}$ $\left(i=1,\dots ,m\right)$, $y_{rj}$ $\left(r=1,\dots ,s\right)$, and $z_{fj}$ $\left(f=1,\dots ,h\right)$, respectively.

According to Färe et al. [32] and Charnes et al. [33], an optimistic eco-efficiency score for a specific $WWT{P}_0$, assuming constant variable returns to scale and strong disposability, can be computed by solving the following Model (1):

$$\begin{gathered} Max {\theta }_0^w={\displaystyle \sum }_{r=1}^s{u}_r{y}_{r0} \\ s.t. \\ {\displaystyle \sum }_{r=1}^s{u}_r{y}_{rj}-{\displaystyle \sum }_{f=1}^h{w}_f{z}_{fj}-{\displaystyle \sum }_{i=1}^m{v}_i{x}_{ij}\le 0, j=1,\dots ,n \\ {\displaystyle \sum }_{i=1}^m{v}_i{x}_{i0}+{\displaystyle \sum }_{f=1}^h{w}_f{z}_{j0}=1 \\ {u}_r, v_i, w_f\ge 0, r=1,\dots ,s; i=1,\dots ,m; f=1,\dots ,h \end{gathered}$$

(1)

where ${\theta }_0^w$ is the optimistic eco-efficiency score for $WWT{P}_0$. ${\theta }_0^w$ ∈ $\left(0, 1\right]$ and a WWTP is eco-efficient if ${\theta }_0^w$ equals unity and inefficient if 0 ≤ ${\theta }_0^w$ < 1. The difference between the eco-efficiency score and the value of 1 represents the input and undesirable output savings required for the WWTP to be eco-efficient; $w_f$ is the decision variable, similar to $u_r$ and $v_i$ [31].

A pessimistic eco-efficiency score can be computed by solving the following Model (2), which is similar to Model (1):

$$\begin{gathered} Min {\theta }_0^{w´}={\displaystyle \sum }_{r=1}^s{u}_r{y}_{r0} \\ s.t. \\ {\displaystyle \sum }_{r=1}^s{u}_r{y}_{rj}-{\displaystyle \sum }_{f=1}^h{w}_f{z}_{fj}-{\displaystyle \sum }_{i=1}^m{v}_i{x}_{ij}\ge 0, j=1,\dots ,n \\ {\displaystyle \sum }_{i=1}^m{v}_i{x}_{i0}+{\displaystyle \sum }_{f=1}^h{w}_f{z}_{j0}=1 \\ {u}_r, v_i, w_f\ge 0, r=1,\dots ,s; i=1,\dots ,m; f=1,\dots ,h. \end{gathered}$$

(2)

Contrary to Model (1), Model (2) seeks the most unfavorable weight for each unit. If ${\theta }_0^{w´}=1,$ then $WWT{P}_0$ is pessimistically inefficient; if ${\theta }_0^{w´}$ > 1, it is pessimistically non-inefficient, which does not necessarily imply that it is optimistically efficient [26].

2.2. Double-Frontier CSW DEA: Global Eco-Efficiency Assessment

Model 1 is a traditional DEA model that allocates favorable weights for each WWTP. Hence, optimal weights differ among WWTPs and more than one WWTP can be determined to be eco-efficient, i.e., ${\theta }_0^w=1$. To set common weights for evaluation of the eco-efficiency of all WWTPs, the following optimistic CSW model was employed [31,34]. All WWTPs were treated as a whole, denoted as $WWT{P}_W$. The Model (3) has $m$ inputs denoted by $\sum _{j=1}^n{x}_{ij} \left(i=1,\dots ,m\right)$, $s$ desirable outputs obtained by $\sum _{j=1}^n{y}_{rj} \left(r=1,\dots ,s\right)$, and $h$ undesirable outputs obtained by $\sum _{j=1}^n{z}_{fj} \left(f=1,\dots ,h\right)$.

$$\begin{gathered} Max {\theta }_W^o={\displaystyle \sum }_{r=1}^s{u}_r\left({\displaystyle \sum }_{j=1}^n{y}_{rj}\right) \\ s.t. \\ {\displaystyle \sum }_{r=1}^s{u}_r{y}_{rj}-{\displaystyle \sum }_{f=1}^h{w}_f{z}_{fj}-{\displaystyle \sum }_{i=1}^m{v}_i{x}_{ij}\le 0, j=1,\dots ,n \\ {\displaystyle \sum }_{i=1}^m{v}_i \left({\displaystyle \sum }_{j=1}^n{x}_{ij}\right)+{\displaystyle \sum }_{f=1}^h{w}_f \left({\displaystyle \sum }_{j=1}^n{z}_{fj}\right)=n \\ {u}_r, v_i, w_f\ge 0, r=1,\dots ,s; i=1,\dots ,m; f=1,\dots ,h, \end{gathered}$$

(3)

where ${\theta }_W^o$ is the optimistic eco-efficiency of $WWT{P}_W,$ considering a set of common weights $\left(u_r^{o*}, v_i^{o*}, w_f^{o*}\right)$ $\left(r=1,\dots ,s;i=1,\dots ,m;f=1,\dots ,h\right)$ obtained by solving the equation.

The optimist CSW eco-efficiency of each WWTP, i.e., of $WWT{P}_j$, was obtained as follows:

$${\theta }_j^{o*}=\frac{{\sum }_{r=1}^s{u}_r^{o*}{y}_{rj}}{{\sum }_{i=1}^m{v}_i^{o*}{x}_{ij}+{\sum }_{f=1}^h{w}_f^{o*}{z}_{fj}}$$

(4)

$WWT{P}_j$ is optimistically CSW eco-efficient if ${\theta }_j^{o*}=1;$ otherwise, it is optimistically CSW non–eco-efficient. In other words, even in the best-case scenario, $WWT{P}_j$ is not eco-efficient.

The optimistic CSW model (Models (3) and (4)) overcomes the shortcomings of each WWTP, as the most suitable weights of inputs, desirable outputs, and undesirable outputs can be allocated. However, some information may be lost by considering only the optimistic perspective. Following the global eco-efficiency assessment approach (Model (3)), global eco-efficiency was minimized to obtain another set of common weights for the pessimistic measurement of WWTP eco-efficiency. According to Chen et al. [31], the pessimistic eco-efficiency of $WWT{P}_W$ is estimated as follows Model (5):

$$\begin{gathered} Min {\theta }_W^p={\displaystyle \sum }_{r=1}^s{u}_r\left({\displaystyle \sum }_{j=1}^n{y}_{rj}\right) \\ s.t. \\ {\displaystyle \sum }_{r=1}^s{u}_r{y}_{rj}-{\displaystyle \sum }_{f=1}^h{w}_f{z}_{fj}-{\displaystyle \sum }_{i=1}^m{v}_i{x}_{ij}\ge 0, j=1,\dots ,n \\ {\displaystyle \sum }_{i=1}^m{v}_i \left({\displaystyle \sum }_{j=1}^n{x}_{ij}\right)+{\displaystyle \sum }_{f=1}^h{w}_f \left({\displaystyle \sum }_{j=1}^n{z}_{fj}\right)=n \\ {u}_r, v_i, w_f\ge 0, r=1,\dots ,s; i=1,\dots ,m; f=1,\dots ,h, \end{gathered}$$

(5)

where ${\theta }_W^p$ is the pessimistic eco-efficiency of $WWT{P}_W$ considering a set of common weights $\left(u_r^{p*}, v_i^{p*}, w_f^{p*}\right)$ $\left(r=1,\dots ,s;i=1,\dots ,m;f=1,\dots ,h\right)$, obtained by solving the equation. Unlike in Model (3) (optimistic scenario), the eco-efficiency of $WWT{P}_W$ is minimized in relation to the others, within a range of no less than 1 (i.e., ${\theta }_W^p\ge 1),$ in Model (5).

The pessimistic CSW eco-efficiency of each WWTP, i.e., of $WWT{P}_j$ (${\theta }_j^{p*}$), was obtained as follows:

$${\theta }_j^{p*}=\frac{{\sum }_{r=1}^s{u}_r^{p*}{y}_{rj}}{{\sum }_{i=1}^m{v}_i^{p*}{x}_{ij}+{\sum }_{f=1}^h{w}_f^{p*}{z}_{fj}}$$

(6)

According to Chen et al. [31], $WWT{P}_j$ is pessimistically CSW inefficient if ${\theta }_j^{p*}=1$; otherwise, it is pessimistically CSW non-inefficient.

Optimistic and pessimistic eco-efficiency estimations represent the best- and worst-case scenarios for the $WWT{P}_j$ evaluated. Following Wang and Lan (2013) [35], the Hurwicz criterion was adopted to integrate the optimistic and pessimistic CSW eco-efficiency scores, as follows:

$${\phi }_j^{c*}=\left(1-\zeta \right)\ast \frac{{\theta }_j^{p*}}{ma{x}_1\le j\le n\left({\theta }_j^{p*}\right)}+\zeta \ast {\theta }_j^{o*},$$

(7)

where $\zeta$ ranges from 0 to 1 and represents the degree of decision-makers’ optimism. If $\zeta =1$, decision-makers are evaluating WWTPs using only the optimistic perspective; if $\zeta =0$, they are adopting only the pessimistic perspective.

3. Sample and Variables

Empirical application of the models discussed previously was conducted in this study with a sample of 30 WWTPs in Spain. All of these plants used conventional activated sludge processes for biological treatment, and thus removed mainly organic matter and suspended solids (SS) from wastewater; they had no specific biological process for the removal of nitrogen or phosphorus. The WWTPs´ operational capacities ranged from 22,000 to 555,000 m³/year.

Variable selection was based on previous studies [16,17,36]. The removal of SS and organic matter (in kilograms/year) was taken as two desirable outputs; this unit of measurement allows for the integration of influent and effluent characteristics [18,37]. Nevertheless, if information is available, it would be interesting to include as desirable outputs other pollutants such as nitrogen and phosphorus, whose discharge in water bodies might involve a negative environmental impact. According to the definition of eco-efficiency [38], inputs should reflect resource consumption. Considering the available statistical data, four inputs were considered in this study: (i) maintenance costs, including those for equipment and machinery maintenance; (ii) sludge and waste management costs; (iii) staff costs, including employee salaries and social charges; and (iv) other costs, including laboratory, administration, and reagent costs (see

Table 1. Main statistics of the variables employed for the estimation of WWTP eco-efficiency.

	Inputs (€/year)				Desirable Outputs (kg/year)		Undesirable Outputs (kg CO2eq/year)
	Maintenance Costs	Sludge Costs	Staff Costs	Other Costs	Organic Matter Removed	Suspended Solids Removed	Scope 2 Greenhouse Gas Emissions
Average	1845.68	1670.69	13,679.58	4885.53	418.77	161.06	23,250
SD	1855.47	1913.42	11,313.44	1259.58	226.68	83.02	30,481
Maximum	5780.26	6704.00	48,657.87	8169.61	1108.00	388.50	111,675
Minimum	90.00	100.00	1347.15	3346.93	82.00	30.00	415

Sample description.

).

Water utilities contribute to the release of GHGs into the atmosphere through the use of energy for water and wastewater treatment [39]. Energy consumption is an important parameter to consider when evaluating the environmental impacts of WWTPs [40]. Following previous studies [17,18], we selected indirect GHG emissions produced by WWTPs [in kilograms of CO₂ equivalent (eq)/year] as an undesirable output. We considered scope 2 GHG emissions resulting from the generation of electricity, heat, and steam purchased by the WWTPs from utility providers [41]. Indirect GHG emissions were estimated based on the methodology proposed by the World Resources Institute [42], which involves the following steps. First, data on the WWTPs’ electric energy consumption (in kWh/year) were collected. Second, considering the Spanish electricity production mix and potential 100-year global warming coefficients, a GHG emission factor was estimated (average, 308 g CO₂-eq/kWh of electricity produced). Finally, indirect GHG emissions (in kilograms CO₂-eq/year) were calculated by multiplying each WWTP’s annual electricity consumption by the estimated emission factor. In addition to GHG, odor emissions from WWTPs have aroused the interest of both the scientific community and environmental authorities [43,44] because long-term exposure to these gases can have a negative effect on human well-being. Moreover, they can also involve economic cost for housing surrounding WWTPs [45]. For this reason, it would be very interesting to include odor emission as an undesirable output when the eco-efficiency of WWTPs is evaluated. Unfortunately, this information was not available for the sample of facilities analyzed in this study, but it will be of great relevance to integrate this variable for future research on this topic.

4. Results and Discussion

Following the methodological approach described previously, four estimations of eco-efficiency scores were computed: (i) optimistic eco-efficiency scores considering different weights for each WWTP (${\theta }_0^w$); (ii) pessimistic eco-efficiency scores considering different weights for each WWTP (${\theta }_0^{w´}$); (iii) optimistic eco-efficiency scores considering common weights (${\theta }_j^{o*}$); and (iv) pessimistic eco-efficiency scores considering common weights (${\theta }_j^{p*}$). Results for each of the 30 WWTPs evaluated are shown in

Table 2. Eco-efficiency scores of wastewater treatment plants (WWTPs) for the four scenarios evaluated.

	$\mathbf{Optimistic}\left({\mathit{\theta }}_0^{\mathit{w}}\right)$	$\mathbf{Pessimistic}\left({\mathit{\theta }}_0^{\mathit{w}´}\right)$	$\mathbf{Optimistic}\mathbf{CSW}\left({\mathit{\theta }}_{\mathit{j}}^{\mathit{o}*}\right)$	$\mathbf{Pessimistic}\mathbf{CSW}\left({\mathit{\theta }}_{\mathit{j}}^{\mathit{p}*}\right)$
WWTP1	0.715	1.668	0.715	3.386
WWTP2	0.318	1.992	0.314	2.936
WWTP3	0.442	3.120	0.362	3.131
WWTP4	1.000	2.611	0.652	4.061
WWTP5	0.726	1.665	0.726	3.549
WWTP6	0.567	4.277	0.567	4.593
WWTP7	0.708	1.000	0.210	1.000
WWTP8	0.773	1.793	0.740	3.353
WWTP9	0.174	1.000	0.124	1.000
WWTP10	0.439	2.790	0.439	3.534
WWTP11	1.000	5.775	1.000	10.241
WWTP12	0.364	1.178	0.364	1.840
WWTP13	0.741	3.285	0.457	4.428
WWTP14	0.418	1.358	0.207	2.049
WWTP15	1.000	8.171	0.920	8.943
WWTP16	0.518	1.067	0.488	2.304
WWTP17	0.975	4.139	0.941	6.052
WWTP18	0.720	1.200	0.169	1.702
WWTP19	0.640	4.311	0.474	4.543
WWTP20	0.292	1.000	0.102	1.000
WWTP21	0.674	1.677	0.215	1.720
WWTP22	0.618	3.765	0.516	4.115
WWTP23	0.755	2.782	0.472	3.103
WWTP24	0.535	1.745	0.535	2.784
WWTP25	0.786	3.600	0.701	6.214
WWTP26	0.266	1.000	0.261	1.506
WWTP27	1.000	7.544	1.000	7.595
WWTP28	0.344	1.445	0.344	1.957
WWTP29	0.153	1.000	0.153	1.000
WWTP30	0.323	1.821	0.236	2.036
Average	0.599	2.659	0.480	3.522
SD	0.257	1.891	0.269	2.326

. For ease of interpretation, pairs of scores are compared.

Figure 1 shows optimistic (${\theta }_0^w$) and optimistic CSW (${\theta }_j^{o*}$) eco-efficiency scores for the 30 WWTPs. As expected from a theoretical perspective, the scores ranged from 0 to 1 and ${\theta }_0^w\ge {\theta }_j^{o*}$. In optimistic (favorably weighted) score calculation, 4 (13.3%) of the 30 WWTPs were categorized as eco-efficient. Thus, 86.7% (26 of 30) of the WWTPs were not eco-efficient, even in the most favorable scenario. The average eco-efficiency score was 0.6, meaning that the WWTPs could potentially improve their performance by 40% on average. By contrast, in score calculation with common weighting, two WWTPs (WWTP11 and WWTP27) were categorized as eco-efficient; these plants were identified as having the best performance. The average optimistic CSW eco-efficiency score was 0.480, 12% lower than the average optimistic eco-efficiency score and substantially divergent. The average amplitude was 0.119 and the maximum value, for WWTP18, was 0.551. This high amplitude reveals that the allocation of common weights to data from all WWTPs had a notable impact on the eco-efficiency classification of this WWTP, penalizing its performance measure. This finding illustrates the importance of common weighting in the assessment of WWTP eco-efficiency when the results are used to benchmark performance for regulatory purposes. By contrast, the low amplitude between the optimistic and optimistic CSW eco-efficiency scores indicates that the scores do not depend on variable weighting, i.e., WWTP performance assessment is not influenced by weight allocation.

The results of this study are consistent with previous ones evaluating the eco-efficiency of WWTPs. Thus, Dong et al. [16] and Gómez et al. [17], using a DEA-tolerance model, estimated an average eco-efficiency score of 0.62 and 0.45 for a sample of Chinese and Spanish WWTPs, respectively. Similar results were reported by [14] for a sample of 281 Chinese WWTPs since its average eco-efficiency was 0.54. Molinos-Senante et al. [15] concluded that most of the WWTPs evaluated in their study have significant room to improve eco-efficiency. In conclusion, past research on this topic, which focused on Chinese and Spanish WWTPs, reported average eco-efficiency scores between 0.45 and 0.62, which is consistent with the average eco-efficiency score for the optimistic scenario (0.6) and for the CSW optimistic scenario (0.48) estimated in this study.

Figure 2 shows pessimistic (${\theta }_0^{w´}$) and pessimistic CSW (${\theta }_j^{p*}$) eco-efficiency scores for the 30 WWTPs. In the pessimistic scenario, 5 (16.7%) of the 30 WWTPs were classified as eco-inefficient (${\theta }_0^{w´}=1)$. In the pessimistic CSW scenario, WWTP26 was no longer classified as eco-inefficient (${\theta }_j^{p*}>1$). These findings indicate that the use of different weights for individual WWTPs leads to the underestimation of eco-efficiency under the pessimistic scenario, as the weights in Model (5) are allocated to guarantee the least favorable result for each WWTP [31]. Figure 2 also shows that ${\theta }_j^{p*}\ge {\theta }_0^{w´}$, i.e., WWTPs are classified as less eco-inefficient when common weights are allocated to all facilities, because the flexibility of allocating different weights to different facilities renders decision-makers’ evaluations overly pessimistic.

As it was stated in the methodology, optimist CSW and pessimistic CSW eco-efficiency scores represent the best- and worst- case scenarios for each WWTP evaluated. The same happens for no CSW eco-efficiency scores, i.e., ${\theta }_0^w$ and ${\theta }_0^{w´}$. For simplicity, only the results from models assuming CSW are presented. From a policy perspective, the assessment of variation in WWTP performance based on the evaluation method employed is important. Figure 3 shows the variability in eco-efficiency, i.e., the difference between the best-case (optimistic CSW, $\zeta =1$) and worst-case (pessimistic CSW, $\zeta =0$) scenarios, based on Equation (7). The interval length reflects the eco-efficiency stability of the WWTPs under the assessed scenarios. Only one facility (WWTP11) was classified as eco-efficient in all scenarios, and this facility was thus ranked first among all WWTPs (Figure 4); the performance of WWTP11 was not impacted by the method used to estimate the eco-efficiency score. This WWTP is a small facility treating about 292,000 m³ (i.e., 2780 people-equivalent) wastewater annually. Another characteristic contributing to its eco-efficiency is the absence of a sewage sludge dehydration system using electrical energy, as this WWTP uses drying beds for this purpose. WWTP11 emits 6.97 tons CO₂-eq/year, which is equivalent to 0.024 kg CO_2-eq/m³; the average value for the 30 WWTPs was 0.27 Kg CO_2-eq/m³. This finding demonstrates the importance of reducing the use of electrical energy, which entails scope 2 GHG emissions, to improve the eco-efficiency of WWTPs.

Figure 3 also highlights the performance of WWTP27, the only plant other than WWTP11 that was classified as eco-efficient in the best-case scenario. In the worst-case scenario, the eco-efficiency score for this WWTP was 0.74, indicating that it can improve its performance by 36%. This example illustrates the importance of considering several scenarios when assessing the eco-efficiency of WWTPs. Figure 3 also shows that some WWTPs (e.g., WWTP9, WWTP14, and WWTP20) had very low eco-efficiency scores even in the best-case scenario. Eco-efficiency scores for these facilities were very stable under all scenarios ($\zeta =0, \zeta =0.5, \zeta =1$), but were < 0.2, indicating extremely poor performance. WWTP20, the facility with the lowest eco-efficiency score, i.e., is ranked last among WWTPs. It is the second smallest facility in terms of the volume of wastewater treated [91,250 m³/year (1960 people-equivalent)]. Although WWTP20 meets the legal standards for effluent quality, it has some operational problems, as its efficiency of organic matter and SS removal is 62% and 57%, respectively, whereas average values in this sample were 75% and 85%, respectively. This low performance in the generation of desirable outputs negatively impacted the eco-efficiency classification of WWTP20.

Other facilities (e.g., WWTP2, WWTP3, WWTP13, WWTP15, WWTP19, WWTP29, and WWTP30) showed very stable performance, with almost constant eco-efficiency scores across all assessments. By contrast, the eco-efficiency of other WWTPs (e.g., WWTP1, WWTP4, WWTP5, WWTP8, WWTP17, and WWTP24) varied greatly depending on the scenario considered; the $\zeta$ value allocated clearly influenced eco-efficiency score estimation. The most extreme case is that of WWTP8, whose eco-efficiency scores in the best- and worst-case scenarios were 0.74 and 0.33, respectively (55% variation).

From a policy perspective, the consideration of differences in eco-efficiency scores based on the scenario employed for analysis is especially relevant when regulation is based on facility benchmarking. Figure 4 shows the ranking of WWTPs based on eco-efficiency scores under the three scenarios evaluated ($\zeta =0, \zeta =0.5, \zeta =1).$ Other than WWTP11, consistently ranked first, the WWTPs changed ranks according to the $\zeta$ value (i.e., optimistic or pessimistic scenario). Notably, the ranking of WWTP8 varied between 5th ($\zeta =1$) and 14th ($\zeta =0$); WWTP13 was ranked 16th under the pessimistic scenario and 8th under the optimistic scenario. These findings reveal the importance of considering several scenarios or conducting a sensitivity analysis when evaluating the eco-efficiency of WWTPs for regulatory purposes. Otherwise, biased results might result in missed opportunities to incentivize WWTPs to change their operations to improve eco-efficiency.

5. Conclusions

WWTPs are complex, resource-intensive facilities, and the improvement of their sustainability is a relevant goal for water regulators and water utilities. Among several methodological approaches employed for assessment, DEA is among the most frequently used, as it provides an index integrating environmental and economic variables, thereby enabling the holistic assessment of WWTP performance. However, DEA is limited by factors such as the selection of different weights for inputs and outputs for individual WWTPs. To overcome this limitation, double-frontier CSW DEA was used in this study to assess the eco-efficiency of a sample of WWTPs in Spain. This approach enabled the estimation of eco-efficiency scores under several scenarios, providing a more reliable and robust performance assessment to inform policy-makers.

Only 1 of the 30 WWTPs evaluated was ranked first under all scenarios evaluated, and was thus identified as having the best economic and environmental performance. Twenty-six of the 30 WWTPs were not classified as eco-efficient, even under the most favorable scenario. Thus, most of the WWTPs evaluated have substantial room for reduction of their costs and GHG emissions. Additionally, the assessment illustrated that the allocation of common weights to the inputs and outputs to all WWTPs notably impacted their eco-efficiency scores, which might have important consequences when performance assessment is used for regulatory purposes. In contrast to traditional DEA, which involves differential weighting to yield the most favorable results, the double-frontier CSW DEA approach used in this study is a reliable and robust approach to the ranking of WWTPs based on eco-efficiency scores, and can be employed successfully by policy-makers to regulate and enhance WWTP performance.

From a policy perspective, the four DEA models applied in this study have illustrated the importance of selecting robust approaches to evaluate the performance of WWTPs when results are used for benchmarking purposes. The moderate–low eco-efficiency scores reported by most of the WWTPs assessed revealed that the (waste)water authority should introduce specific policies aimed to improve the economic and environmental performance of WWTPs. In doing so, special emphasis should be placed on reducing GHG emissions as the Sustainable Development Goal 11.6 defined by the United Nations focused on “reducing the adverse per capita environmental impact of cities”. In this context, it is fundamental moving to a low-carbon urban water cycle.