Economic, Ecological and Social Analysis Based on DEA and MCDA for the Management of the Madrid Urban Public Transportation System

Abstract

The aim of this research is to help public transport managers to make decisions on the type of buses that should compose their public transport fleet, taking into account economic, environmental and social criteria from the point of view of sustainability. This paper fills a knowledge gap by including the social dimension of sustainability in addition to the economic and environmental dimensions. The original nature of this study lies in analyzing complementarities in the structuring of an efficiency and multicriteria problem. Our research analyzes Madrid public bus system data; the problem is structured in a comparative way between two analytical methods, a Data Envelopment Analysis (DEA) and an ELimination Et Choice Translating REality (ELECTRE) III. Our research results show that two main groups of vehicles could play a part in part the theoretical solution. The main conclusions of this research are that (a) plug-in and induction electric vehicles are not comparable to GNC and diesel–hybrid vehicles in terms of cost, pollution and service; and (b) the ELECTRE III model provides more information in solving this problem than the DEA model.

1. Introduction

Policy makers are under strong political and social pressure to define a management strategy for urban public transport systems to comply with both environmental regulations [1,2,3,4] and public spending constraints. This is an important problem because less polluting buses are more expensive [5,6,7] and it is quite difficult for policy makers to decide what type of vehicles to use for road public transport systems. Therefore, the motivation of this research is to help policy makers to make balanced decisions to manage sustainable public transport systems. [1,2,3,4,5,6,7].

The originality of this study is in the proposed comparison between a multicriteria decision-aiding model (MCDA) and a Data Envelopment Analysis (DEA) model to help public transport managers to make decisions on the type of buses that should compose their public transport fleet from the point of view of sustainability. This is a novel approach to filling a current research gap, taking into account the three dimensions of sustainability: economic, environmental and social [8,9,10]. From a methodological point of view, this study proposes a comparison between two rankings, one obtained with a multicriteria decision-aiding model (MCDA) and the other with a Data Envelopment Analysis (DEA) model.

This is a relevant topic for research because urban transport is one of the primary sources of pollution in urban areas, and citizens are demanding that cities use a fleet of sustainable vehicles [11,12,13,14]. By 2050, urban air pollution will become the leading environmental cause of mortality worldwide, followed by polluted water and lack of sanitation infrastructure [11]. The concentration of population in large cities and the intensive use of urban road transport are the most important air pollution problems, reaching alert levels that put health at serious risk.

On the one hand, in 2018 approximately 55.3% of the world’s population lived in urban areas. By 2030, these urban areas are expected to host 60% of people worldwide, and one in three people will live in cities with at least half a million inhabitants [15]. In Europe, in particular, although urban and suburban areas cover about 20% of the European Union, they host about 75% of the European population [16].

On the other hand, urban road transport accounts for 40% of all C02 emissions, and up to 70% of other transport pollutants [12]. The urban population of the EU is potentially exposed to very high levels of air pollutants, with levels of nitrogen oxides (NO₂), particulate matter with a diameter of 10 μm or less (PM10), particulate matter with a diameter of 2.5 μm or less (PM2.5), and carbon dioxide (CO₂) all exceeding EU standards [13]. During 2018–2019, the daily maximum limit for PM10 was exceeded by 15%, the daily maximum limit for PM2.5 by 4%, the annual limit for NO₂ by 4%, and the annual CO₂ target by 34% [14].

These worrying levels of air quality have provoked major international organizations to support initiatives to mitigate the negative effects caused by urban road transport. The 2030 Agenda for Sustainable Development in its objective 11.6 “reduce the environmental impact of cities” aims to “by 2030, reduce the adverse per capita environmental impact of cities, including by paying special attention to air quality and municipal and other waste management”. In total, the 2030 Agenda includes the 17 Sustainable Development Goals to be followed by public and private organizations to defend the environment and achieve a cleaner world before 2030 [8].

The European Union has defined numerous regulations with the objective of achieving air quality levels that do not cause risks to human health or significant negative impacts on the environment. The European Commission, in its European directives on air quality, establishes standards for pollution levels [1]. These standards are concentrations of air pollutants in ambient air in terms of NO₂, particulate matter (PM10 and PM2.5) and CO₂ that Member States must not exceed. EU Member States must maintain air pollutant levels below these air quality standards [1] and take measures to reduce pollutants when standards are exceeded [2].

In addition, the COVID-19 pandemic was a serious additional human health problem with significant economic and social impacts. Actions taken by governments across Europe in the early 2020s to manage the pandemic caused a temporary shutdown of many of the economic activities that generate air pollutant emissions, which had a positive effect on air quality [17]. In particular, according to data from [14], nitrogen dioxide concentrations were significantly reduced in April 2020, regardless of meteorological conditions. The level of reduction varied considerably between cities and countries, in some cases with reductions exceeding 60%. Concentrations of PM10 were lower in Europe as well, with reductions of up to 30% in some countries.

Despite this improvement in air quality levels due to both the efforts of Member States and the positive effect of COVID-19, air pollution-related health problems are considered an increasingly serious challenge by European citizens [18]. The management of the COVID-19 pandemic caused an economic shutdown that had a positive effect on air pollution rates. Nevertheless, COVID-19 has become a new pressure element for governments to influence the way they manage pollution problems because certain studies are beginning to link the development and severity of the disease to air pollution [19,20,21].

The European Commission supports Member States in taking appropriate measures, and has launched several initiatives to increase cooperation in reducing air pollution [22]. The European Commission takes a vigilant stance with regard to pollution levels in major European cities, sending formal requests to national and local governments of those cities with dangerous levels of local air pollution when limit values set by EU legislation on ambient air quality are exceeded [3], as has been the case in Madrid since 2018 [4].

One of the main challenges for local public administrations is the difficulty in making decisions to solve air quality problems in an effective way. In general, cities are faced with two main issues: deciding what kind of measures are appropriate, and determining how to evaluate their effects.

Regarding the first issue, what kind of measures are appropriate to keep pollution levels under control in large cities, local public administrations define specific air quality plans. In cities such as Madrid, Paris or London, local administrations have carried out different measures which have in common that 50% of them are related to urban road transport in general. Most of these measures refer to urban road transport in particular [23]. The role of urban public road transport to mitigate local air pollutants is relevant [24] and of growing importance in the political priorities of Europe’s most important municipalities [25] by implementing (1) measures to modify the mobility patterns of citizens by reducing the use of private vehicles in favor of public transport use, and (2) measures to modify the management policies of public transport systems in order to replace polluting buses with non-polluting buses. The emissions from public road transportation depend on the type of fuel technologies used in vehicles. Most urban transportations systems use a combination of several types of buses with different fuel technologies: diesel engines, compressed natural gas (CNG) engines, diesel–hybrid engines, plug-in electric motors and electric induction motors [26].

Regarding the second issue, how to evaluate the effects of measures to keep pollution levels under control in large cities, policy makers have a high level of uncertainty about how best to assess the costs and benefits of measures to reduce pollution in cities [23]. While a few years ago the evaluation of implemented measures was mainly examined in terms of economic costs, today cities focus on measures that ensure sustainable development in economic, environmental and social terms [9,10].

Madrid is among the European cities with the most serious pollution problems. Therefore, this research analyzed Madrid’s public bus system data, creating a Multiple Criteria Decision Analysis model and comparing it to a Data Envelopment Analysis model to identify which urban public transport bus technologies perform better in terms of efficiency and in terms of ranking. The comparison was the basis for subsequent reflection on the strengths and weaknesses of the DEA and multi-criteria methods (the latter an outranking method, ELECTRE III) to enable a set of recommendations to be drawn up for managers of public transportation.

The case study of this major European city is representative, and the research conclusions can be applied to other large cities which have similar bus alternatives and the common goal of reducing air pollution from public transportation systems [27].

The municipal authorities of the city of Madrid are under strong social and political pressure to improve its air quality efficiently and in the shortest possible time frame. This environment hinders the possibility of achieving the theoretical objectives due to budgetary constraints and limitations in the autonomy and recharging of electric vehicle batteries. The constraints that make it difficult to implement measures to achieve the theoretical targets cause policy makers to change their actions and make subjective political decisions to satisfy public opinion and Member State demands. Often, these new subjective decisions may appear to be contradictory or in direct conflict with achieving the theoretical objective based on technical and objective criteria.

The aim of this research is to help public transport managers to make decisions on the type of buses that should compose their public transport fleet, taking into account economic, environmental and social criteria from the point of view of sustainability; therefore the research question is:

-

What are the best vehicle types, taking into account their account cost, environmental, and social criteria, and without allowing trade-offs between the criteria?

Before answering the research question, the efficiency of each of the analysed urban public bus fuel alternatives (diesel, compressed natural gas, diesel–hybrid, plug-in electric, and induction electric) in terms of cost, pollution and service should be clarified. Furthermore, it should be clarified, on the one hand, whether the different bus technologies can really be compared with each other in terms of cost, pollution and service, and, on the other hand, which analytical model (DEA or ELECTRE) provides more relevant information in searching for a solution to this problem.

The rest of this paper is organized as follows. Section 2 provides a review of the literature. Section 3 presents the data sources and methodological description, and section explains the DEA and ELECTRE III models and formulas. The question of the similarity and dissimilarity approach to the problem in terms of MCDA and DEA is addressed here as well. Section 4 includes the results and discussion of the two models. Section 5 offers the main conclusions and recommendations of the study, including the implications of the results for management, the research limitations, and potential futures lines of research.

2. Literature Review

2.1. Bus Assessment: Fuel Consumption Cost and Emissions

Recent studies have proposed models for analyzing vehicle technologies in terms of economic and environmental sustainability.

Table 1. Literature review of bus assessments based on fuel consumption, cost, and emissions.

Methodology	Case Study	Added Value
Emissions and energy consumption	China	Large-scale promotion of electric vehicles should be designated for the long-term [28].
Emissions and fuel Consumption comparison	China	Diesel–hybrid buses reduce emissions and fuel consumption compared to Euro IV and V diesel buses [29].
Emissions comparison	Romania and Madrid	Replacing effects: evaluates the effects on emissions of replacing conventional city buses with electric buses [30,31].
Emissions and cost comparison	Madrid	Plug-in electric buses are the most sustainable alternative in economic and environmental terms [5].
	Madrid	Diesel–hybrid, plug-in electric, and induction electric urban public buses are acceptable in terms of sustainability [6].
	Metrobus	Analyzes public transport real driving emissions [7].
	Bangalore, India	Replacing effects: the replacement of diesel buses by electric bus reduces air pollution and noise pollution [32].
	Finland and California	Alternative bus powertrains can significantly improve energy efficiency. Diesel–hybrid buses are already a cost effective solution for public transportation [33].
Emissions and cost comparison Nonlinear regression models	Tehran	Evaluates scenarios based on environmental, traffic and economic analysis to predict reductions in commute times, fuel consumption, and emissions [34].

shows selected studies on bus assessment based on economic costs and pollutant emissions. However, there are no studies that simultaneously include economic, environmental and social assessment criteria.

Air pollution travels very long distances, making it both a transboundary and local problem [35]. In addition, there is growing political, media and public interest in air quality issues, and increased public support for action. [14].

The growing international political and social pressure for countries and cities to reduce their air pollution levels is the main reason for defining new environmental and transport and logistics management policies. To the strict vigilance of the European Commission must be added the growing pressure from citizens who are increasingly aware of the dangers of pollution, demanding that Member States adopt air quality plans to reduce current levels of air pollution in the long term and as a matter of urgency.

Understanding the gap between EU air quality standards and the exceedances occurring in major European cities is one of the main concerns of the European Commission when reviewing air quality policies in the EU. The review of air quality policy has shown that it is not appropriate to amend the Air Quality Directive, and that the strategy should focus on achieving compliance with the current standards by 2030 at the latest.

2.2. Bus Selection Based on Multi-Criteria Decision Models (MCDM) and Data Envelopment Analysis (DEA)

Public administrations are therefore being forced to continue innovating and investing in sustainable alternative energies for urban public road transport [36]. The selection of the type of buses to be used in public transport is based on the efficiency of the vehicle technologies, on budgetary issues, and on the degree of acceptance by citizens of the management decisions of policy makers from the point of view of economic costs, environmental costs, and social benefits. It is quite difficult for policy makers to decide what type of vehicles to use in their public transport systems because the most environmentally and socially sustainable technologies are the most expensive. Social and political pressure is growing, which makes it more difficult for policy makers to make balanced decisions. The more environmentally sustainable the technology of a public transport vehicle, the more acceptable this technology is to citizens. Recent studies support the use of both plug-in electric and induction electric vehicles for their sustainability [5,6].

However, electric vehicles have significant limitations. On the one hand, electric vehicles are much more expensive because they have a higher purchase price and require additional facilities to recharge the batteries, which are not always affordable in economic terms due to the budgetary limitations of public administrations. On the other hand, it takes a long time to build this infrastructure, making it even more difficult for cities to achieve the objectives required by the European Commission and society at large.

This paper proposes a comparison between a multi-criteria decision-aiding model (MCDA) and a Data Envelopment Analysis (DEA) model to help public transport managers make decisions on the type of buses that should compose their public transport fleet, taking into account economic, environmental and social criteria from the point of view of sustainability.

Multi-criteria decision models are a suitable tool to assist bus fleet managers to make decisions related to the evaluation of vehicle performance from different perspectives, because these models may include criteria related to economic, environmental, social and technological aspects.

Table 2. Literature review of bus selection based on multi-criteria decision models.

MCDM	Case Study	Application
AHP and Delphi	Madrid	Ranking of vehicles based on four economic criteria and three environmental pollution criteria [5].
ELECTRE TRI and Delphi	Madrid	Classification of vehicles into acceptable and non-acceptable ones based on four economic criteria and three environmental pollution criteria [6].
12-indicator MCA method	Sweden	Creation of a 12-indicator MCA method [37].
	Sweden	Application of a 12-indicator MCA method to assess several bus technologies [38].
Intuitionistic Fuzzy Choquet Integral	Istanbul	Bus selection based on RTFV analysis [39].
Graph theory and matrix approach	China and USA	RTFV bus analysis in urban areas [40].
Fuzzy AHP TOPSIS	Ankara	RTFV electric bus analysis [41].
	Taiwan	Selecting low pollutant emission buses [42].
Fuzzy AHP-VIKOR	Ankara	Bus selection based on RTFV analysis [43].
Fuzzy TOPSIS PSI	Taiwan	Bus selection in urban areas based on RTFV [44].
AHP-VIKOR and TOPSIS	Taiwan	RTFV bus analysis in urban areas [45].
AHP	Delhi	Analysis of conventional fuel vs. CNG buses [46].
Intuitionistic fuzzy sets, FMCDM	India	Developing a methodology for identifying the best option for selection of alternative fuels for sustainable urban transportation [47].

RTFV stands for road transportation fuels and vehicles.

provides an overview of publications on bus selection based on multi-criteria decision models.

Table 3. Literature review of bus selection based on DEA.

Methodology	Case Study	Added Value
Comparative efficiency analysis using DEA.	Spain	Use of principal components to reduce dimensionality by obtaining highly significant factors [48].
DEA-based clustering algorithm.	Thessaloniki	Use of bootstrapping techniques to check robustness of DEA results. The DEA-based clustering method provides the readjustment for the combinations of inputs and outputs a DMU may employ in order to achieve a higher level of efficiency [49].
Combination of a network DEA model with the Banker, Charnes, and Cooper (BCC) model.	Seoul	The model can reflect the non-storable nature of public transportation services by sequentially considering transportation services provided by operators and consumed by users [50].
Study of the technical efficiency and environmental efficiency using the slack-based measures and non-separable slack-based measures models with and without considerations of desirable and undesirable outputs.	Taipei	Empirical results show that technical efficiency was affected by environmental pollution constraints from 2007 to 2011 [51].
Use of data envelopment analysis and globally efficient frontier production functions.	USA	The analysis implies that the magnitude of scale economies depends on the output specification [52].
Comparison of DEA and Stochastic Frontier Analysis.	Athens	Introduction of exogenous factors measured by dummy variables [53].
Comparison of two DEA models, CCR and VRS, with different outputs models.	Norway	An extensive conclusion about ways of improving efficiencies in the bus industry [54].
A series two-stage data envelopment analysis (DEA) approach integrated with bootstrapping.	Athens and Pyreus	Bootstrapping in the current research serves as a tool to fully rank the assessed routes, thus overcoming the limitations of conventional DEA models [55].

shows relevant publications on bus selection using DEA models.

This model fills a current research gap by taking into account the three dimensions of sustainability (economic, environmental and social). Therefore, the results allow policy makers to decide how to replace more heavily-polluting cheaper buses with less polluting but more expensive ones in a gradual and balanced way without reducing social sustainability.

The main objective of this research is to identify which urban public transport bus technologies are more convenient and better-performing in terms of economic, environmental and social sustainability. This theoretical objective is obtained based on minimizing the economic cost and pollutant emission criteria and maximizing the social criterion of service by kilometres travelled by each vehicle. However, the theoretical decision obtained cannot always be easily implemented by municipalities, either because of budgetary constraints or because of the lack of sufficient time to implement them to meet the limited deadlines set by social pressure and the tightening of international standards.

In fact, the best technology according to an economic criterion may conflict with the best technology according to an environmental or social criterion. The complexity arising from the interaction between economic, environmental, and social aspects results in a high degree of uncertainty. In this sense, multicriteria decision-aiding (MCDA) theory is an important tool for providing solutions to problems of uncertainty. Our study proposes to compare the recommendations proposed by the MCDA with the results obtained from a more classical approach such as DEA. Uncertainty makes it important to provide managers and decision makers with robust recommendations.

The MCDA and DEA approaches are presented as two complementary methodologies in decision making. The basic DEA models allow each alternative to be able to achieve efficiency by presenting itself in its best possible light. This often leads to efficiency being achieved using criteria or combinations of criteria that are of little relevance to managers. MCDA allows the decisionmaker’s preferences to be introduced, which is fundamental in a problem where the social impact is high. Moreover, these preferences can be introduced in such a way that there is no trade-off between the criteria.

This article analyses the notion of complementarity between the problem statement in terms of MCDA and DEA, and points out the distinctive elements of each approach in the resolution and search for solutions. It is this analysis that makes it possible to complement the solutions obtained by providing the manager or decision maker with robust recommendations obtained by strengthening or weakening the results obtained with each method separately.

This paper aims to fill a knowledge gap by analyzing complementarities in the structuring of an efficiency and multicriteria problem. In fact, in our case, the problem is structured in a complementary way between DEA and ELECTRE III.

3. Materials and Methods

3.1. Materials

The proposed multicriteria decision model is based on 2020 Madrid urban public road transport data published by Madrid City Council [56,57,58], which has been compiled by the authors and assessed by a panel of twenty experts to identify the criteria, factors and weights included in the model.

The expert panel identified the structure of the problem as well as the model building, and the research team applied multi-criteria decision techniques as well as efficiency techniques to provide a final recommendation.

The first and second steps of the research were the identification and the structuring of the problem. A panel of twenty experts was selected to help in this task. The construction of the criteria and the definition of the alternatives were agreed upon between experts and research team. Data were compiled by the authors based on the 2020 Madrid urban public road transport data published by Madrid City Council [57,58].

The selection of experts is a fundamental phase in any decisionmaking process. In our research, a panel of twenty experts was formed. All experts were professionals or scholars in the transport industry (45%), energy in transport (35%), or both (20%) with at least three years’ experience. By domain of expertise, nine experts (45%) belonged to the Transport domain, seven experts (35%) belonged to the Energy domain, and four experts (20%) belonged to the Transport and Energy domain. In terms of experience, six experts (30%) had from three to five years of experience and fourteen experts (70%) had more than five years of experience.

The experts agreed on the selection of the criteria as well as the background information for the proposal of the criteria weights (in the case of ELECTRE III) or the restriction intervals of the criteria weights (in the case of DEA).

Diviz software [59,60] was used to implement ELECTRE III, and Frontier Analyst (Banxia software) was used to carry out the efficiency analysis.

3.1.1. Alternatives

In carrying out our analyses we elected to follow [31]. The Number 21 bus line was chosen as a representative sample of the urban bus lines in the city of Madrid for three reasons: (a) it carries out a long-term route that crosses the entire capital, combining light traffic areas with heavy traffic areas in the same proportion; (b) it has a route with an average distance of 9.21 km per journey that is close to the average of all routes; (c) buses of all the technologies included in this study are used to cover this line. Five vehicle options have been considered in this study, depending on the fuel energy used by each alternative of the main types of buses available on the market: CNG, diesel, diesel–hybrid, plug-in electric, and induction electric.

3.1.2. Criteria and Dataset

Madrid’s public road transport network combines vehicles with different alternative fuel types and different levels of sustainability, economic performance and environmental impact. On the one hand, economic criteria related to the cost of service provision such as depreciation, traction, maintenance and operating costs must be taken into account. On the other hand, environmental criteria should be included to assess emissions of NOx, CO₂, and PM pollutants. Finally, some service indicators should also be included.

In general terms, depreciation costs are calculated on the basis of two variables: acquisition cost and durability. In the case of electric induction buses, a complementary asset cost has to be included, the price of the specific infrastructure needed to charge the battery at the terminals. Therefore, the formula for the depreciation costs can be expressed as:

Depreciation costs = [Vehicle purchase price (€)/Vehicle service life (years)] × Number of vehicles + Infrastructure

(1)

In order to calculate traction costs, two variables must be taken into account: the energy consumption of each type of vehicle and the price of the fuel it uses. In relation to the energy consumption of each type of vehicle, the amount of fuel consumed per kilometre driven must be taken into account on the one hand, and on the other, the price of fuel, which is different for each type of fuel and varies according to market prices. Therefore, the formula for the traction costs can be expressed as:

Traction costs = Consumption (litres per km) × Fuel price (€ per km) × Kilometres travelled

(2)

Maintenance costs include expenses related to the upkeep and repair of vehicles and components present throughout the operation, including costs related to ensuring that the vehicle operates in a safe, reliable, comfortable and environmentally sustainable manner. Maintenance costs have a strong relationship with the service lifetime the vehicles, because upkeep and repair of vehicles tend to become more intensive over time, especially once the warranty period expires. The formula for calculating maintenance costs is defined below:

Maintenance costs Alternative X = Average maintenance costs (€ per km) × Kilometres travelled

(3)

The average maintenance cost per kilometre for one year of service (see

Table 4. Data for cost variables.

Alternative/DMU	Depreciation Costs (€)	Traction Costs (€)	Maintenance Costs (€/km)	Operating Costs (€)
Diesel	479,167	475,797	0.4193	4,225,560
GNC	555,833	333,557	0.4845	4,225,560
Diesel–hybrid	651,667	380,661	0.5425	4,225,560
Plug-in electric	1,420,834	168,918	0.2280	4,320,045
Induction electric	960,500	214,517	0.2280	5,079,285

Source: Compiled by the authors based on data from Madrid Transport Co. [57,58].

) was calculated based on data from the Madrid Transport Company [57,58] and information related to the cost of maintaining the batteries for diesel–hybrid and electric vehicles [61,62]. Diesel–hybrid vehicles have increased maintenance costs compared to a normal diesel engine due to the mechanical complexity of the electric components. However, maintenance costs for plug-in and induction electric vehicles are halved on average thanks to the reduced mechanical difficulty of maintenance and the drastic reduction in preventive costs.

The operating costs were calculated for one year of service for each alternative, taking into account the salaries of the staff required for each type of vehicle and the total hours of service, which are different depending on the technology of each alternative. Unlike diesel, CNG, and diesel–hybrid vehicles, which make the journeys in the same time, induction electric buses need more time to make the same route due to the time they must spend recharging their batteries [63]. The formula for calculating the operating costs can be expressed as:

Operating cost Alternative X = Average operating costs (€ per hour) × Hours in service

(4)

The Table 4 summarises the cost data considered according to engine type.

The difficulty in comparing pollutant emissions from vehicles using different technologies is that each type emits different particulate pollutants and there are no tables of equivalence of the damage caused to the environment by different particulate pollutants. Hence, the NOx, CO₂, and PM pollutant emissions of each type of vehicle were analyzed individually, and the NOx, CO₂, and PM emissions of each alternative for one year of service were assessed separately as three independent criteria.

Quantifying CO₂ emissions is a practical solution to show the damage caused to the environment by different types of buses. Each type of vehicle was assigned an emission value in its type approval; therefore, it is possible to calculate the emissions depending on the fuel used and the carbon footprint.

A carbon footprint is the total amount of generated greenhouse gases expressed as a carbon dioxide equivalent. Cutting CO₂ emissions is the key solution to tackling climate change; therefore, this research includes the carbon dioxide equivalent emissions of different types of public road transport vehicles. Although electric vehicles emit no direct emissions when in operation, they do emit pollutants indirectly by consuming electricity, and consequently it is possible to calculate their indirect carbon dioxide equivalent emissions by establishing an emissions factor of the electricity mix used. The previous emissions factor depends on the energy source used to produce the electricity. Renewable sources or those with low CO₂ emissions have a low or zero mix factor.

Table 5. Pollutant emissions.

Alternative/DMU	NOx (kg)	PM (kg)	CO2 (kg)
Diesel	1138	3.9	1,589,471
GNC	1044	3.4	1,833,113
Diesel–hybrid	910	3.1	1,264,616
Plug-in electric	-	-	724,612
Induction electric	-	-	926,059

Source: Compiled by the authors based on data from Madrid Transport Co. [57,58].

presents the data for pollutant emissions.

The selection of the service indicator is based on a review of the literature; see Table 3. The two variables most commonly used in the literature as outputs are vehicle kilometres travelled and number of people transported [7,30,31,42,47]. As the capacity of the buses is the same regardless of the type of drivetrain, only the kilometres travelled by each vehicle was considered for output.

Table 6. Data for service indicators.

Alternative/DMU	Output: Km/Nº Bus
Diesel	50,443.39
GNC	50,443.39
Diesel–hybrid	50,443.39
Plug-in electric	38,318.97
Induction electric	43,415.78

Source: Compiled by the authors based on data from Madrid Transport Co. [57,58].

shows the data on service indicators.

3.1.3. Criteria Weights

A previous study [5] addressed the problem of constructing a vehicle ranking taking into account only environmental and economic criteria. On that occasion, a simpler approach was chosen; the problem was structured hierarchically and AHP was subsequently applied. Here, the weights used in the AHP, rather than being obtained through pairwise comparison, were obtained after applying a Delphi to the panel of experts. Two criteria were established, one economic and one environmental. The economic criterion was further divided into four subcriteria: depreciation, traction cost, maintenance cost, and operating cost. The environmental criteria were divided into three subcriteria: NOx emissions, particulate matter emissions, and CO₂ emissions.

Two rounds were needed for a convergence of opinion on the importance of the criteria and subcriteria in the model. The consultation instrument in the first and second round was a questionnaire that included the criteria and subcriteria being assessed by the panel of experts according to their importance to achieving the goal of sustainability. In the first round, the panel members received a dossier containing notable studies, European regulations on environmental impact, and the strategic guidelines of private and public bodies, as well as the following key information on the fuel alternatives assessed in this study: service life of vehicles, bus purchase prices, fuel and battery prices, range and charging times, emissions factors of local pollutants and particles, maintenance costs, resources in terms of staffing and fleet size, engine performance, and type approval emissions factors for local pollutants and particles. To assess the environmental subcriteria, additional information was provided to enable proper assessment on the basis of danger to human health according to the parameters set by the WHO [13].

For the second round, the results of the first round were provided to the experts for re-evaluation or to confirm their opinions, including (a) mean values for the whole responses, (b) standard deviations for the total dataset, (c) individual responses in the former round, and (d) the interquartile range (IQR).

The coefficient of variation can be used to determine when consensus has been reached [64]. According to [65], a coefficient of variation between 0 and 0.5 is acceptable to consider consensus achieved and hence terminate the process.

Table 7. Consensus on weights obtained after application of the Delphi method.

Elements	Nomenclature	Unit	Weighting	SD	Q1	Q3	IQR	CV1	CV2	CV1-CV2
Criteria
Economic criterion	Economic	EUR	0.6	0.129	6	6	0	0.16	0.13	0.03
Environmental criterion	Environm	Kg	0.4	0.194	4	4	0	0.28	0.19	0.09
Economic sub-criteria
Depreciation	Deprec	EUR	0.2	0.474	1	2	1	0.57	0.45	0.12
Traction cost	Tract	EUR	0.2	0.474	1	3	2	0.47	0.47	0.00
Maintenance cost	Mainten	EUR	0.2	0.474	1	3	2	0.45	0.47	0.02
Operating cost	Operat	EUR	0.4	0.296	3	5	2	0.44	0.30	0.14
Environmental sub-criteria
NOx emissions	NOx	Kg	0.5	0.228	4	6	2	0.23	0.23	0.00
Particular matter emissions	PM	Kg	0.3	0.279	3	4	1	0.37	0.28	0.09
CO2 emissions	CO2	Kg	0.2	0.354	2	2	1	0.34	0.35	0.01

SD = standard deviation; Q1 = first quartile; Q3 = third quartile; IQR = interquartile range; CV1 = coefficient of variation 1st round; CV2 = coefficient of variation 2nd round.

shows weights obtained by consensus after application of the Delphi method.

In this case, we introduced an additional criterion, the service criterion. Moreover, instead of working with eight criteria and five alternatives (the ratio between the number of alternatives/DMUs and the number of criteria would dramatically worsen the discriminant capacity of the DEA), we first carried out a dimensionality reduction by applying a principal component analysis to the group of economic and environmental criteria separately. In this way, the only information about the weights retained from [5] was the proportion of importance established by the experts between the cost and environmental criteria.

3.2. Methods

We present here the basic fundamentals of the DEA and ELECTRE III analytical methods.

3.2.1. DEA

The DEA is a non-parametric efficiency measurement technique. An efficiency frontier is obtained from the set of observations considered without the estimation of any production function. DEA attempts to optimise the efficiency target for each unit analysed to create an efficiency frontier based on the Pareto criterion, which allows multiple inputs and outputs to be used without imposing a functional form on the data or making assumptions about inefficiency. The efficient frontier is the benchmark against which the relative performance of different Decision-Making Units (DMUs), in our case types of engines, are measured. The DMUs that form the efficient frontier use a minimum amount of inputs to produce the same amount of outputs. The measure of efficiency is represented by the distance from the DMU to the efficiency frontier.

A basic DEA model called CCR was first applied in 1978 [66]. The relative efficiency of a DMU is calculated as the ratio between the weighted sum of outputs and the weighted sum of inputs. The weights are determined by linear programming, such that the DMU maximises its efficiency value; for more details on DEA methods, see [67].

Let $E_0$ be the efficiency score of the observed $DM{U}_0$. Let $y_{j0}$, $j=1,\dots ,s$ be the outputs and $x_{i0}$, $i=1,\dots ,m$ the inputs used to compute the efficiency, and $n$ the total number of DMUs ($r=1,\dots ,n$). Then, the relative efficiency of $DM{U}_0$ is calculated as:

$$E_0=Max\frac{{{\displaystyle \sum }}_{j=1}^s{u}_j{y}_{j0}}{{{\displaystyle \sum }}_{i=1}^m{v}_i{x}_{i0}}$$

(5)

where $u_j$ and $v_j$ are the non-negative weights. If the condition that the efficiency be less than or equal than 1 is imposed, the classical CCR model under the Constant Return to Scale (CRS) assumption is obtained:

$$\begin{array}{l}{E}_0=Max\frac{{{\displaystyle \sum }}_{j=1}^s{u}_j{y}_{j0}}{{{\displaystyle \sum }}_{i=1}^m{v}_i{x}_{i0}}\\ s.t.:\\ \frac{{{\displaystyle \sum }}_{j=1}^s{u}_j{y}_{jr}}{{{\displaystyle \sum }}_{i=1}^m{v}_i{x}_{ir}}\le 1 for r=1,\dots ,n\\ u_j, v_j\ge 0 \forall i,j\end{array}$$

(6)

The transformation for linear fractional programming selects a concrete weight solution for which ${\displaystyle {{\displaystyle \sum }}_{i=1}^m}{v}_i{x}_{i0}=1$, leading to the following linear problem:

$$\begin{array}{l}{E}_0=Max{\displaystyle {\displaystyle \sum }_{j=1}^s}{u}_j{y}_{j0}\\ s.t.:\\ {\displaystyle {\displaystyle \sum }_{i=1}^m}{v}_j{x}_{i0}=1\\ {\displaystyle {\displaystyle \sum }_{j=1}^s}{u}_j{y}_{jr}-{\displaystyle {\displaystyle \sum }_{i=1}^m}{v}_j{x}_{ir}\le 0 r=1,\dots ,n\\ u_j, v_j\ge 0 \forall i,j\end{array}$$

(7)

DEA solves $n$ different LP problems for a set of $n$ DMUs. A $DM{U}_{0 }$ will be efficient if $E_0=1$ with non-zero weights. The CRS model assumes that if an efficient DMU increases its inputs by a constant factor, its outputs are expected to increase by the same factor; [68] proposed a version of the CCR model with variable returns to scale, commonly known as the BCC model, with the difference the introduction of a convexity condition.

3.2.2. ELECTRE III

ELECTRE III is an outranking method focused on ranking the so-called ‘problematic: alternatives’ (enterprises in our case) ordered from best to worst by means of pairwise comparisons [69]. For readers interested in ELECTRE methods and MCDA, see [70] for more details.

Let $\mathit{A}=\left\{{\mathit{A}}_{\mathbf{1}},\dots ,{\mathit{A}}_{\mathit{m}}\right\}$ be the set of alternatives.

Let $\mathit{C}=\left\{{\mathit{C}}_{\mathbf{1}},\dots ,{\mathit{C}}_{\mathit{n}}\right\}$ be a coherent family of criteria

Let ${\mathit{g}}_{\mathit{i}}\left({\mathit{a}}_{\mathit{j}}\right)$ be the value of the criterion ${\mathit{g}}_{\mathit{i}}$ for the alternative ${\mathit{a}}_{\mathit{j}}$

Let ${\mathit{w}}_{\mathit{i}}$ be the weight of the criterion ${\mathit{C}}_{\mathit{i}}$

ELECTRE III makes use of outranking relations. An outranking relation, where alternative a outranks alternative b (denoted by a S b), expresses the fact that there are sufficient arguments to decide whether a is at least as good as b and there are no essential reasons to refute this [71]. An outranking degree S(a, b) between a and b will be computed in order to ‘measure’ or to ‘evaluate’ this assertion. These outranking relations consider three basics binary relations: preference, indifference (ELECTRE III considers that indifference is not necessarily transitive), and incomparability.

If we consider two alternatives, a and b, four situations may occur:

-

aSb and not bSa, i.e., aPb (a is strictly preferred to b).

-

bSa and not aSb, i.e., bPa (b is strictly preferred to a).

-

aSb and bSa, i.e., aIb (a is indifferent to b).

-

Not aSb and not bSa, i.e., aRb (a is incomparable to b).

ELECTRE methods are based on the principles of concordance and discordance. The concordance principle states that if a is demonstrably as good as or better than b according to a sufficiently large weight of criteria, then this is considered to be evidence in favor of a outranking b. The discordance principle states that if b is very strongly preferred to a on one or more criteria, then this is considered to be evidence against a outranking b.

ELECTRE III is divided into two phases. First, the outranking relationship between the alternatives is constructed and then exploited.

To construct the outranking relationship, ELECTRE IIII uses other parameters in addition to the criteria weights to model the decisionmaker’s preferences. These parameters are the preference, indifference and veto thresholds. Criteria can be increasing or decreasing; in the following, and without lack of generality, we will consider the criteria to be increasing.

The introduction of thresholds expands the set of binary relations beyond indifference, preference and incomparability; for more detail see [72].

The indifference threshold indicates the largest difference between the performance of the alternatives on a given criterion that makes the two performances indifferent to the decision maker. Let ${\mathit{q}}_{\mathit{i}}$ be the indifférence threshold for criterion i. Alternative b is weakly preferred to alternative a in terms of criterion i if ${\mathit{g}}_{\mathit{i}}\left(\mathit{b}\right)>{\mathit{g}}_{\mathit{i}}\left(\mathit{a}\right)+{\mathit{q}}_{\mathit{i}} \left({\mathit{g}}_{\mathit{i}}\left(\mathit{a}\right)\right)$.

The preference threshold indicates the largest difference between the performances of the two alternatives, such that one is preferred over the other for the criterion under consideration. Let ${\mathit{p}}_{\mathit{i}}$ be the preference threshold for criterion i. Alternative b is strictly preferred to alternative a in terms of criterion i if ${\mathit{g}}_{\mathit{i}}\left(\mathit{b}\right)>{\mathit{g}}_{\mathit{i}}\left(\mathit{a}\right)+{\mathit{p}}_{\mathit{i}}\left({\mathit{g}}_{\mathit{i}}\left(\mathit{a}\right)\right)$.

The veto threshold for a criterion is the difference between in performance between the two alternatives above which it seems reasonable to reject any credibility about the outranking of one alternative by the other alternative, even when all other criteria are in line with this outranking. Let ${\mathit{v}}_{\mathit{i}}$ be the veto threshold for criterion i. Alternative a cannot outrank alternative b if the performance of b exceeds that of a by an amount greater than the veto threshold, i.e., if ${\mathit{g}}_{\mathit{i}}\left(\mathit{b}\right)\ge {\mathit{g}}_{\mathit{i}}\left(\mathit{a}\right)+{\mathit{v}}_{\mathit{i}}\left({\mathit{g}}_{\mathit{j}}\left(\mathit{a}\right)\right)$.

ELECTRE III constructs a partial concordance index per criterion, an overall concordance index, and a discordance index. Once these indices have been defined, the credibility index can be defined as well.

Partial Concordance index measures the strength of support, given the available evidence, that a is at least as good as b considering one specific criterion. Global Concordance index measures the strength of support, given the available evidence, that a is at least as good as b considering all criteria. For each criterion, Discordance index measures the strength of the evidence against the hypothesis that a is at least as good as b. Credibility index measures the strength of the claim that alternative a is at least as good as alternative b.

Once the decision-maker has defined the parameters that represent the indifference ${\mathit{q}}_{\mathit{i}}$, preference ${\mathit{p}}_{\mathit{i}}$ and veto ${\mathit{v}}_{\mathit{i}}$ thresholds, and the weight of each criterion ${\mathit{w}}_{\mathit{i}},$ the expressions that allow obtaining the indices of partial concordance, global concordance, discordance and credibility are the following.

Partial concordance index por each criterion $\mathit{i}$:

$$C_i\left(a,b\right)=\{\begin{array}{c}0, if g_i\left(b\right)\ge g_i\left(a\right)+p_i\left(g_i\left(a\right)\right)\\ 1, if g_i\left(b\right)\le g_i\left(a\right)+q_i\left(g_i\left(a\right)\right)\\ \frac{g_i\left(a\right)+p_i\left(g_i\left(a\right)\right)-g_i\left(b\right)}{p_i\left(g_i\left(a\right)\right)-q_i\left(g_i\left(a\right)\right)}, otherwise\end{array}$$

(8)

Global concordance index:

$$C\left(a,b\right)=\frac{{\displaystyle \sum }{w}_i{C}_i\left(a,b\right)}{{\displaystyle \sum }{w}_i}$$

(9)

Discordance index for each criterion:

$$D_i\left(a,b\right)=\{\begin{array}{c}0, if g_i\left(b\right)\le g_i\left(a\right)+p_i\left(g_i\left(a\right)\right)\\ 1, if g_i\left(b\right)\ge g_i\left(a\right)+v_i\left(g_i\left(a\right)\right)\\ \frac{g_i\left(b\right)-g_i\left(a\right)-p_i\left(g_i\left(a\right)\right)}{v_i\left(g_i\left(a\right)\right)-p_i\left(g_i\left(a\right)\right)}, otherwise\end{array}$$

(10)

If no veto threshold $v_i$ is specified, $D_i\left(a,b\right)=0$ for all pairs of alternatives.

Credibility index:

$$S\left(a,b\right)=\{\begin{array}{c}C\left(a,b\right), if D_i\left(a,b\right)\le C\left(a,b\right)\forall i\\ C\left(a,b\right){\displaystyle {\displaystyle \prod }_{D_i\left(a,b\right)>C\left(a,b\right)}}\frac{1-D_i\left(a,b\right)}{1-C\left(a,b\right)}\end{array}, otherwise$$

(11)

If no veto threshold ${\mathit{v}}_{\mathit{i}}$ is specified, S(a, b) = C(a, b) for all pairs of alternatives.

The second phase of ELECTRE III consists of the exploitation of the pairwise outranking indices through bottom-up and top-down distillations. The distillation procedures each calculate a complete pre-order. Each pre-order takes into account the behaviour of each alternative when outranking or being outranked by the other alternatives. These two procedures can lead to two different complete pre-orders. The final partial pre-order is obtained as the intersection of these two complete pre-orders [69].

3.2.3. Comparison between DEA and MCDM: State of the Art

Several authors [73,74,75,76,77] have pointed out the associations between DEA and MCDA. However, we must point out here that there are features which are distinctive to DEA or to MCDA as well. These similarities and dissimilarities are a primary subject of our article.

The similarity between a DEA model and an ELECTRE III model can be found in the following problem statement: a DEA problem can be modelled symmetrically through an ELECTRE III model where inputs are criteria to be minimised and outputs are criteria to be maximised. Obviously, the mathematical foundations behind a DEA and an ELECTRE III are different, but the statement of the problem has a symmetrical approach with both methods.

The dissimilarity between the two methods is to be found mainly in the aspect of objectivity versus subjectivity. ”The emphasis in MCDA is on modelling values; the approach is one which seeks to make explicit and to manage subjectivity rather than eliminate it. In contrast … DEA has tended to be data driven with an emphasis on ‘objectivity’.” [73].

ELECTRE III needs to introduce information about the weights of the criteria, i.e., it incorporates information about the value judgements of the decision maker. The original version of DEA allows full flexibility of weights. Consequently, no information on the value judgments of the decision maker is required. Although the more advanced versions try to solve this problem (and consequently the problem of obtaining unrealistic weights) through restrictions on the weights, this information is never treated as precisely as in ELECTRE III.

We want to highlight the question of trade-offs between criteria. DEA is a compensatory method, whereas ELECTRE III is not. In this sense, the incorporation of certain information regarding the value judgments of the decision maker through restrictions on the weights should be taken with caution, in the sense that its meaning will not be strictly the same as in ELECTRE III, which is based on outranking relations.

Our article follows a line of work the most representative article of which is [73], and has its antecedents in the work of [78]. In [73], the question of the complementarity of the two methods is raised from a fundamentally theoretical point of view. There are few works where the interest in using this idea in a real and complex situation can be tested. With our work, we hope to contribute to filling this gap.

Most of the articles in which DEA and multi-criteria decision methods are used together deal with how multi-criteria decision methods can be used to improve the discriminant power of DEA and how to incorporate the judgements of the decision maker. It is therefore a matter of modifying the DEA model by incorporating elements of multi-criteria decision making, and not of using the methods in a complementary way.

Table 8. MCDM used as a tool to improve DEA models.

MCDM	Ref.	MCDM Method	Contribution
Shang et al. (1995)	[79]	AHP	AHP is used to derive weight restrictions in DEA.
Vieira Junior, H. (2008)	[80]	MACBETH	The judgment matrix MACBETH is used to derive virtual weight restrictions in DEA
Bhattacharyya, A. et al. (2014)	[81]	TOPSIS	TOPSIS method is employed to rank efficient DMUs.
Bagherikahvarin, M et al. (2016).	[82]	PROMETHEE II	Stability intervals based on PROMETHEE II are used as absolute weight restrictions in DEA and the unicriterion net flow score of the PROMETHEE II matrix is used instead of the initial DEA evaluation matrix.
Alves Pereira, M. et al. (2020)	[83]	Choquet Integral	Choquet integral is used to take into account criteria interactivity in an additive DEA model

shows the most relevant literature on MCMD contributions to improving DEA models.

The main article where a complementary application of DEA and multicriteria decision aid methods to a real-life situation can be seen is [84]. In this article, the performance of biogas plants in Austria was analysed by comparing results obtained with DEA and ELECTRE TRI/IRIS. As DEA provides a final solution that classifies DMUs into efficient and non-efficient, the authors proposed the use of ELECTRE TRI, which is a method of the ELECTRE family that classifies alternatives into predefined classes. To use IRIS while retaining the spirit behind DEA, no information about criteria weights was introduced at the outset. IRIS implements a methodology developed by [85] that is based on the ELECTRE TRI method, but which does not require precise values for some input parameters (criteria weights and cutting level). Our approach is different; the two main criticisms of DEA are that it is often not sufficiently discriminating and that it is not able to incorporate the judgements of the decision-maker. To overcome these pitfalls, we propose to carry out a DEA with a super-efficiency model that introduces restrictions to the weights which are derived from the judgements of the expert panel. In this way, a ranking (the result of the super-efficiency model) with greater discriminant capacity is obtained as a final solution thanks to the introduction of preferential information (restrictions on the weights). It seems to us that the natural choice of the decision-aiding method should therefore be a method that provides a solution in the form of a ranking, thus favouring the comparison of the results of the two methods; this justifies the choice of ELECTRE III.

4. Results and Discussion

In the presentation of the results, we will follow the same structure as described in the methodology. First, we will analyse the results obtained with DEA, then those obtained with ELECTRE III, and finally we will compare them. Prior to the application of the methods proposed for comparison, two preliminary remarks must be made.

(1)

Due to the large number of variables in relation to the number of observations, we have elected to summarise the information by calculating the principal components of the cost variables on the one hand and the environmental emissions on the other. In this way, we will have three final variables under study: a service indicator, a cost indicator and a pollutant emissions indicator. This need to summarise variables is present in the application of DEA. If the number of variables (inputs + outputs) is large in relation to the number of DMUs, many more efficient DMUs will be obtained because there is a greater chance that a DMU will be very good in one variable and bad in the rest. In the case of ELECTRE III, this problem does not exist. However, for a better comparison of the results we have decided to work in both cases with the same type of information (the service indicator and the two principal components).

(2)

As noted in Section 3.1. Materials (for more details see [5]), a panel of experts studied the relative importance of cost versus environmental emissions variables and concluded that this ratio was 60–40%. In the analyses carried out below, this importance given to the two blocks of inputs/criteria is taken into account. However, we study the sensitivity of the methods to variations in the weights as well.

4.1. Principal Component Analysis

We first carried out a principal component analysis of the cost variables (see Table 4). The results show that we can summarize this information as one principal component (see

Table 9. Total Variance Explained for cost variables.

	Initial Eigenvectors
Component	Total	% of Variance	% Cumulative
1	3.032	75.799	75.799
2	0.734	18.355	94.154
3	0.200	4.993	99.147
4	0.034	0.853	100.000

). This one principal component explains almost 76% of the total information. As in the case of the cost variables, the environmental pollution variables (see Table 5) can be summarised in a single principal component that explains almost 79% of the total variance (see

Table 10. Total Variance Explained for pollutant emissions.

	Initial Eigenvectors
Component	Total	% of Variance	% Cumulative
1	2.358	78.600	78.600
2	0.640	21.341	99.941
3	0.002	0.059	100.000

).

Table 11. Data for ELECTRE III.

Alternative/DMU	Output/Criterion Service	Input Cost/Criterion Cost	Input Pollutants/Criterion Pollutants
Diesel	50,443.39	0.85340426	0.80302158
GNC	50,443.39	0.58025198	0.74712464
Diesel-hybrid	50,443.39	0.74682808	0.17908404
Plug-in electric	38,318.97	−1.09082212	−0.06781797
Induction electric	43,415.78	−1.0896622	−1.66141228

presents the final values of the variables used in the analyses. We are now in a position to apply DEA and the ELECTRE III. Table 11 presents the data for the ELECTRE III analysis.

DEA cannot use negative values. Therefore, we need to make corresponding translations of the values in the principal components so that the values are greater than or equal to zero. After this transformation, the data for the DEA model are as shown in

Table 12. Data for DEA.

Alternative/DMU	Output/Criterion Service	Input Cost/Criterion Cost	Input Pollutants/Criterion Pollutants
Diesel	50,443.39	1.94422638	2.46443386
GNC	50,443.39	1.67107411	2.40853691
Diesel-hybrid	50,443.39	1.83765021	1.84049631
Plug-in electric	38,318.97	0	1.84049631
Induction electric	43,415.78	0.00115992	0

.

4.2. DEA Results

We ran a BCC super-efficiency approach in order to provide a clearer ranking with a higher discriminant power over the DMUS. The super-efficiency approach by [86] builds upon the basic DEA by leaving out one DMU i at a time. A DMU is considered super-efficient, i.e., has an efficiency score larger than 1, if a DMU increases its input vector proportionally while preserving efficiency. This approach is especially useful when DEA delivers many efficient units, as it allows for further differentiation via a more accurate ranking. As our aim is to compare the results of DEA with the ranking obtained with ELECTRE III, this seems to be the most appropriate model.

As for the choice of a BCC model, the reason was the transformation required for the input values obtained from the principal component analysis. The original CCR model was designed not to have negative values; therefore, we had to choose a model that was invariant to translations. In [87], the authors show that “The envelopment form of the input (output)-oriented BCC model is translation invariant with respect only to outputs (inputs) and to non-discretionary inputs (outputs).”

Thus, we started with a DEA in which the input and output weights are left completely free and then introduced restrictions to the weights following the opinions of the expert panel. The latter allowed the decisionmaker’s preferences to be introduced into the DEA, making the results more meaningful when comparing them with those of ELECTRE III.

Table 13. BCC-DEA with no restrictions on weights.

Alternative/DMU	Score	Efficient	Condition
Diesel	89.3%	False	Red
GNC	110.2%	True	Green
Diesel–hybrid	131.0%	True	Green
Plug-in electric	100.0%	False	Amber
Induction electric	1000.0%	True	Green

presents the results of a BCC DEA with no restrictions on weights.

We see that GNC, Diesel–hybrid and Induction electric engines are efficient. In the case of the plug-in electric, this should always be taken with caution because its reference DMU for comparison will be the induction electric motor.

Table 14. Potential improvements for plug-in electric compared to induction electric BCC-DEA, with no restrictions on weights.

Variable	Actual	Target	Potential Improvement
i1 (Cost)	0.00	0.00	0.00%
i2 (Pollutants)	1.84	0.00	−99.95%
o1 (Service)	38,318.97	43,415.78	13.30%
Peers: Diesel-hybrid

presents the potential improvements for plug-in electric when compared it to induction electric.

If we look at Table 14, “Potential Improvements” we see that the plug-in electric would have to emit much less pollutant gases than the induction electric, and would have to improve its service. Compared to the other engine types, however, the plug-in electric performs very well. This is a characteristic of DEA: a DMU is efficient if there is no other DMU that can improve in one variable without getting worse in another.

If we introduce restrictions to the weights (these restrictions are “Assurance Regions”) such that the cost criterion is around 60% of the importance of the inputs and the emissions criterion is around 40%, then the following results are obtained; the results of the BCC-DEA with these restrictions on weights are presented on

Table 15. BCC-DEA with restrictions on weights (cost are $\cong$ 60%, emissions $\cong$ 40%).

Alternative/DMU	Score	Efficient	Condition
Diesel	87.8%	False
GNC	98.3%	False
Diesel–hybrid	1000.0%	True
Plug-in electric	65.0%	False
Induction electric	1000.0%	True

.

Now, only the diesel–hybrid and the induction electric engines are efficient. The reason for this result lies in the following comparisons:

-

The diesel is compared to the diesel–hybrid. From this comparison, it follows that for the diesel to be efficient it would have to improve its costs by 5.15% relative to the cost of the diesel–hybrid and by 25.20% relative to emissions while maintaining its service levels in km/bus delivered (potential improvements are shown in

Table 16. Potential improvements for diesel compared to diesel–hybrid (BCC-DEA) with restrictions on weights (cost are $\cong$ 60%, emissions $\cong$ 40%)).

Variable	Actual	Target	Potential Improvement
i1 (Cost)	1.94	1.84	−5.15%
i2 (Pollutants)	2.46	1.84	−25.20%
o1 (Service)	50,443.39	50,443.39	0.00%
Peers: Diesel–hybrid

).

-

The plug-in electric is in turn compared to the induction electric. In this case, in order to be efficient it would have to improve its mileage by 13.30% and reduce its emissions by 99.95% compared to induction electric (potential improvements are shown in

Table 17. Potential improvements for electric plug-in compared to induction electric (BCC-DEA) with re-strictions on weights (cost are $\cong$ 60%, emissions $\cong$ 40%)).

Variable	Actual	Target	Potential Improvement
i1 (Cost)	0.00	0.00	0.00%
i2 (Pollutants)	1.84	0.00	−99.95%
o1 (Service)	38,318.97	43,415.78	13.30%
Peers: Induction electric

).

4.3. ELECTRE III Results

ELECTRE III requires a number of parameters to be introduced to reflect preferential information. These parameters consist of weights and the indifference, preference and veto thresholds. In order to be able to compare the DEA and ELECTRE III data more realistically, we did not introduce any thresholds.

In ELECTRE III, the criteria weights are central to the analysis. We present different situations to study how the results vary according to the weights assigned. ELECTRE III clearly differentiates between CNG and diesel–hybrid (tied with each other) and plug-in and induction electric (tied with each other). Both groups are ranked at the top of the ranking (i.e., the only “bad” one is diesel); however, it is clear that these two groups of vehicles are not comparable to each other. These results were obtained by giving a weighting of 20% to 30% to the service variable and weighting cost and emissions such that cost was 60% and emissions were 40% of the remainder; these results are presented on Figure 1.

If we increase the importance of service to more than 40%, CNG and diesel–hybrid appear at the top of the ranking, tied with each other. These results are presented in Figure 2.

4.4. Discussion: Comparison between DEA and ELECTRE III Results

The ELECTRE III results seems much more logical, because they reflects with greater nuance the reality of the different types of engines, while the sensitivity analyses show more varied results depending on the importance given to the weights. We tested giving, for example, more importance to environmental factors than to cost factors, and the results were different from those shown here. It is clear that it is diesel engines that need to be eliminated, as both approaches show this.

Introducing the weighting restrictions derived from the information provided by the experts and weighting total service parameter at less than or equal to 40%, ELECTRE III tells us that we cannot compare electric vehicles with CNG to diesel–hybrids. Both groups, however, are acceptable from the point of view of the variables considered.

In the case of the efficiency study, the result is apparently more accurate. The plug-in electric motor is not efficient in comparison with the induction motor. However, both the induction electric and the diesel–hybrid (and, practically speaking, the CNG) appear to be equally efficient, some due to cost and others due to environmental aspects.

The main reason for the differences in the results between DEA and ELECTRE III lie, logically, in the mathematical bases on which each of the methods are founded. In the case of DEA, the best DMUs are those for which we cannot observe an objectively better DMU (or a combination of DMUs). The best DMUs, according to ELECTRE III, are those attaining the highest ranks in a given number of criteria and that do not have any weaknesses. Another fundamental difference between the two approaches is that DEA is a compensatory method between inputs and outputs separately. This does not happen in ELECTRE III, as we have pointed out above; in order to occupy a good position in the ranking, the DMU/alternative has to obtain good values on a set of important criteria while not being bad on the others.

In [84], one of the conclusions obtained is that “…some differences between the two analyses can also be attributed to different degrees of discrimination. … The difference between the approaches can be diminished as the number of categories increases, as the discrimination among DMUs would increase in the MCDA analysis (ELECTRE TRI in this case).”

In order to make a more realistic comparison, we chose ELECTRE III, which provides a ranking, and not ELECTRE TRI, which provides a classification into categories. However, as we can see in the solutions, the results provided by ELECTRE provide more information than those provided by DEA. This is due to the fact that in the ELECTRE methods the decisionmaker’s preferences can be modelled through thresholds and vetoes. This type of preferential information leads, for example, to consider CNG vehicles and D-H vehicles on one side and P-E and IE on the other as equivalent. In the case of DEA with restrictions, D-H is more efficient than CNG and IE is more efficient than P-E. Finally, something that DEA is not able to take into account is the possible incomparability between DMUs; this is a fundamental aspect of modelling the problem. In our case, the introduction of incomparability thanks to the ELECTRE III model shows that GNC and D-H are not comparable with P-E and IE, while DEA with restrictions considers D-H and IE equally efficient.

Our conclusion, based on the results obtained in our study, is that ELECTRE III and DEA can be used in a complementary way, providing a solution that is much richer and more nuanced.

5. Conclusions and Recommendations

The originality of this study is based on (a) the proposition of a model that fills a current research gap by taking into account the social dimension of sustainability along with economic and environmental dimensions, based on the United Nations circle of sustainability method; (b) the combination of the ELECTRE III multiple criteria decision method and the DEA method to enrich the results and help policy makers to make balanced decisions when identifying which urban public transport vehicles perform better in economic, environmental and social terms; and (c) the exploration of similarities and dissimilarities between the ELECTRE III and DEA models.

The most relevant conclusions of this research are the following:

-

That plug-in electric and induction electric vehicles cannot be directly compared to GNC and diesel–hybrid vehicles in terms of cost, pollution and service.

-

That the ELECTRE III model provides more relevant information towards a solution to this problem than the DEA model in assisting policy makers to make balanced decisions. In this work, the ELECTRE III model was compared to the DEA model in order to enrich the results. The ELECTRE III model and the DEA model are two different methods that offer two different solutions to the problem. In this case, the ELECTRE III model offers a more logical solution than the DEA model; thus, the ELECTRE III model is the preferred method to support the research questions and recommendations in this research.

It is not surprising that the conclusions reported by DEA and ELECTRE III are different, as the two methods have different mathematical bases. The best DMUs according to the DEA are those for which a better DMU (or a combination of them) cannot be objectively observed. The best DMUs (or alternatives) from the point of view of ELECTRE III are those that are better in several criteria without being particularly bad in the others.

In our view, the most interesting recommendation for managers is the one proposed by ELECTRE III; however, it is undoubtedly very well complemented by the solution proposed by DEA. Although there is an obvious similarity in the problem statements with ELECTRE III and DEA, the dissimilarities of each approach provide the relevant nuance to strengthen or weaken a given recommendation.

In relation to the efficiency, in terms of cost, pollution and service, of the analyzed urban public bus fuel alternatives, it was determined that: 1. The diesel urban public buses are not efficient in terms of cost, pollution and service; 2. The compressed natural gas urban public buses are efficient in terms of cost, pollution and service; 3. The diesel–hybrid urban public buses are efficient in terms of cost, pollution and service; 4. The plug-in electric urban public buses are not efficient in terms of cost, pollution and service; and 5. The induction electric urban public buses are efficient in terms of cost, pollution and service.

Plug-in electric and induction electric vehicles cannot be compared to GNC and diesel–hybrid vehicles in terms of cost, pollution and service when we do not allow trade-offs between criteria and the indifference ordering relationship may not be transitive.

This research shows that the ELECTRE III model provides more useful information for decision makers than the DEA model. However, the two models complement each other well and DEA provides different nuances for a better understanding of the performance of the types of bus engines studied than either method alone.

Based on our research results, the answer to the previously posed research question is:

-

ELECTRE III clearly differentiates between, on the one hand, “CNG and diesel–hybrid” (tied with each other) and “plug-in and induction electric” (tied with each other) on the other hand. Both groups appear at the top of the ranking (i.e., the only “bad” option is diesel); however, it is clear that these two groups of vehicles are not comparable to each other. These results were obtained by giving service a total importance of less than or equal to 40%, and by weighting cost and emissions such that cost is 60% and emissions 40% of the remainder. If we increase the importance of service so that it is greater than 40%, CNG and diesel–hybrid appear at the top of the ranking, tied with each other.

5.1. Implications for Management

The results show that two categories of vehicles have been created that cannot be compared with each other in terms of cost, pollution and service. Both groups are part of the theoretical solution to the problem, and include types of vehicles that are recommended from the point of view of the three variables used in the model (economic, environmental and social criteria), although for different reasons. On the one hand, the plug-in and induction electric vehicles group is recommended based on their low environmental costs. On the other hand, the CNG and diesel–hybrid vehicles group is recommended based on their better economic costs. There is no reduction in the level of service in either group; however, diesel vehicles are not recommended according to [6] and are consistently rejected as a fuel technology by both the ELECTRE and the DEA model.

The analytical model presented here includes economic, environmental and social sustainability variables in order to make support decision making for the management of sustainable urban public road transport systems. This model offers a useful theoretical solution for policy makers seeking to define the most appropriate transport management strategy to improve air quality in large cities while satisfying both social and political requirements.

According to the theoretical solution obtained in this study, the main recommendation to policy makers is to define a management plan for the urban public bus transport system that seeks to improve air quality, focusing on creating a 100% green bus fleet in the short term while minimizing economic costs and maximizing the level of service to citizens. It is suggested that this renovation plan for polluting buses be carried out in two phases.

-

The first phase envisions the replacement of diesel buses by efficient and less polluting buses. The new bus fleet should include buses with CNG, diesel–hybrid, plug-in electric and/or induction electric technology. Urban public transport vehicles with more environmentally sustainable technologies are in demand by citizens; however, they are much more expensive because they have a much higher purchase price and require complementary facilities for their operation which are not always affordable in a short term, given the budgetary constraints of public administration. During this first phase, these four types of buses should be used for urban public transport service until any budgetary and technical constraints that prevent the replacement of all diesel buses by electric buses in the short term can be resolved. To replace diesel buses, plug-in or induction buses should be purchased according to the characteristics of the city in question. In developed cities, it is more efficient to use plug-in buses than induction buses because the cost of building the infrastructure for recharging the batteries of plug-in electrics is lower than the cost of recharging the batteries of induction buses. However, electric induction buses could be more efficient for urban public transport systems in developing cities, because charging infrastructure could be built under the pavement at a lower cost.

-

The second phase advances as the technical and budgetary restrictions that limit the adoption of electric buses are resolved. In this phase, CNG and diesel–hybrid buses are progressively replaced by plug-in electric and induction electric buses until the bus fleet has only vehicles that are 100% free of polluting emissions. The costs of acquisition and use of electric vehicles can be significantly reduced due to economies of scale without reducing service levels, thus achieving a new Pareto optimum.

Finally, to reduce the social and political pressure that public administrations bear with regard to air quality in large cities, it would be convenient to define complementary action plans for citizens to get involved in reducing environmental pollution. If citizens substitute for the use of private vehicles with the use of public transportation, both the air quality and the income of the locality in question would improve, allowing for faster replacement of the most polluting and least efficient buses.

5.2. Research Limitations and Future Lines of Research

The main research limitation here (which could be an inspiration for future lines of research as well) is that n variables (public opinion, budget restrictions, deadline) could be added to the models to further enrich the analysis. Future research that identifies user opinion, and the impact of the public on voting decisions would be useful for both policy makers and the research community.

Another limitation is that social sustainability could be better measured with a service indicator relative to the number of passengers per bus; however, capacity restrictions due to COVID-19 do not allow this to be applied without negatively affecting the results.

In future lines of research, it would be interesting to incorporate buses based on combustion cell technology into the model, as these represent a new sustainable alternative for urban public transport systems.