Next Article in Journal
Power Prediction of Airborne Wind Energy Systems Using Multivariate Machine Learning
Next Article in Special Issue
Climate Change Mitigation in Households between Market Failures and Psychological Barriers
Previous Article in Journal
Adaptive Armature Resistance Control of Virtual Synchronous Generators to Improve Power System Transient Stability
Previous Article in Special Issue
Model of Optimization of Wind Energy Production in the Light of Legal Changes in Poland
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:

Tools for Measuring Energy Sustainability: A Comparative Review

Rafael Ninno Muniz
Stéfano Frizzo Stefenon
William Gouvêa Buratto
Ademir Nied
Luiz Henrique Meyer
Erlon Cristian Finardi
Ricardo Marino Kühl
José Alberto Silva de Sá
6 and
Brigida Ramati Pereira da Rocha
Electrical Engineering Graduate Program, Department of Electrical Engineering, Federal University of Pará (UFPA), Belém 66075-110, Brazil
Electrical Engineering Graduate Program, Department of Electrical Engineering, Santa Catarina State University (UDESC), Joinville 89219-710, Brazil
Electrical Engineering Graduate Program, Department of Electrical Engineering & Telecommunications, Regional University of Blumenau (FURB), Electrical Engineering, Blumenau 89030-000, Brazil
Electrical Engineering Graduate Program, Department of Electrical Engineering, Federal University of Santa Catarina (UFSC), Florianópolis, 88040-900, Brazil
Institute of Systems and Computer Engineering, Research & Development of Brazil (INESC P&D Brasil), Santos 11055-300, Brazil
Center of Natural Sciences and Technology, Pará State University (UEPA), Belém 66095-100, Brazil
Operations and Management Center of the Amazon Protection System (CENSIPAM), Defense Ministry of Federal Government, Belém 66617-420, Brazil
Author to whom correspondence should be addressed.
Energies 2020, 13(9), 2366;
Submission received: 15 April 2020 / Revised: 3 May 2020 / Accepted: 6 May 2020 / Published: 9 May 2020
(This article belongs to the Special Issue Sustainable Energy Reviews)


This paper is intended to perform a comparative and qualitative review among eight tools to measure energy sustainability. Therefore, it was necessary to create a theoretical and conceptual framework based on four criterias of selection and six categories of comparison. In this work, the conceptual bases that supported the research and the methodology created to carry out the comparative review will be presented. This analysis was based on the intrinsic concepts of energy sustainability of each of the reviewed tools with a critical qualitative analysis. Some conclusions shown through the conceptual framework developed that it was possible to apply an innovative methodology to qualitatively compare different tools to measure sustainability. The importance of this reflects the difficulty of conceptualizing the subjectivity of sustainable development, as shown throughout the paper, where it is often not possible to obtain a measurable result since the measured phenomenon is too complex to reduce it to a numerical value.

1. Introduction

The indicators calculated based on the performance of energy systems provide important measures for the economic, social, and environmental development of a nation since human activities and sustainability problems are intrinsically related to the use of energy. Energy indicators are tools that assist in formulating sustainable development policies. They measure progress towards sustainability. The more in-depth knowledge of these methods allows them to be applied in consideration of their main characteristics, advantages, and limitations of each tool, which leads to more legitimate and reliable results [1].
Sustainability and sustainable developments are terms widely used in scientific literature and public policies, even though they do not have a consensus on a standard nomenclature and metric, due to the vast diversity of existing concepts. Despite this, there is a general understanding of the need to seek a balance in the relationship between human beings and the environment and their complex interaction dynamics [2,3,4,5].
The term sustainable also has different translations such as longevity in German (nachhaltend) and durability in French (durabilité) and also Dutch (duurzaamheid), which leads to different possible interpretations [6]. In this context, the term reflects a solution of the scarcity of natural resources, and the search for a continuous and perpetual use of these resources, which suggests a connection with the concept of energy due to the crises of the energy system that have occured since ancient times [7,8]. Thus, sustainable development can be defined as access to achieve sustainability, which is considered the final long-term goal [9,10]. Sustainability consists of a final objective defined employing scientific criteria, which is measured and accompanied by results generated by the use of sustainable development strategies [11].
To define energy sustainability, it is necessary to initially introduce the concept of sustainability. Sustainability is a characteristic of a system that allows it to remain, at a certain level, for a certain period. This concept becomes a principle according to which the use of natural resources to satisfy present needs cannot compromise the satisfaction of the needs of future generations. The concept of sustainability meets a set of interdependent variables, which must be able to integrate social, energetic, economic, and environmental issues [12,13,14]. Based on this concept, energy sustainability is interconnected with the use of renewable energy sources, and, in this way, it is possible to meet the main need about energy sustainability, which is the guarantee that future generations will not be affected since the resources are not exhausted and are available for free. Sustainability in addition to being related to clean energy generation is also related to social, economic, and environmental issues [15,16,17,18].
Energy sustainability cannot be understood as the energy used to achieve sustainable development [19]. This is the biophysical perspective of the human use of energy to enable the achievement of goals by considering environmental changes [20]. With regard to energy sustainability, the change from fossil fuels to renewable sources cannot be sustained over time. To obtain sustainability, it is necessary to reduce the useful energy supplied, which may occur by improving systems for more efficient methods, i.e., less energy is required to perform the same work [21,22,23].
Since energy is a vital input for the social and economic development of any nation, setting the goal of achieving energy sustainability is a major challenge for all countries [24]. While the debate over the proper definition of sustainable development continues, the fact remains that societies depend heavily on fossil energy sources, which has a negative impact on environment and health [25,26,27,28]. In this context, it is essential to seek an alliance between energy and sustainability, to assist countries in the development of sustainable energy policies, through the use of sustainable indicators and indices as tools to communicate energy issues for the political field and for the general public [29].
The purpose of this research was to conduct a comparative review between several indices and energy sustainability indicators chosen in the international review. This paper is organized in the following segments. In Section 2, the literature review is presented. Section 3 presents the methodology used in the research. Section 4 offers an analysis of the results with a detailed discussion. Lastly, the last section presents the conclusions reached.

2. Literature Review

Sustainable development and energy problems are inherent elements for the governments of all nations and, intrinsically, converge on the routes for the energy industry [30]. These elements constitute tools for the well-being of society. Development is not a consequence, but rather a state of things that are sought through time, regardless of the development paradigm assumed in each context [31,32,33].

2.1. Energy and the Dimensions of Sustainability

Sustainability is an act of equilibrium between social, environmental, and economic dimensions of human needs [34]. These balanced dimensions are constantly changing because of the growing human population and the fulfillment of their consumption needs [35,36,37]. Therefore, since the dimensions of sustainability are not independent of each other, the constant balance of sustainability makes it a dynamic concept and not a static state [38]. Due to the dynamic characteristic of sustainability, new ideas always emerge [39,40,41]. Additionally, for the three fundamental dimensions of sustainability (economic, social, and environmental), there are other definitions of sustainable dimensions in the literature, according to the need for the phenomenon to be measured. The institutional dimension and the technical dimension were incorporated to broaden the vision of sustainability [42,43,44], which totalled five dimensions that will be discussed in this paper [34].
The environmental dimension of sustainability focuses on natural biological processes, health, and ecosystem functionality as well as on continuous productivity with minimal environmental impacts [45]. This dimension seeks to reduce the negative impacts of the extraction of natural resources for energy production, consumption by society, and increase of the positive impacts. Global, national, and regional commitments show the important role of energy in sustainable development to reduce energy-related environmental degradation such as global warming, deforestation, air, land, and water pollution [46,47,48,49].
The economic dimension of sustainability evaluates whether the supply of energy is profitable and affordable or not [50]. Cost-effectiveness is necessary to ensure that energy investments are economically viable to encourage reinvestment in the system that promotes sustainability. Accessibility also ensures that the energy supplied is not only physically available, but accessible to society. The two complement each other to ensure the sustainability of the supply system. If the energy provided is very costly, society cannot consume it and, thus, the suppliers do not get any return for their investment [51,52]. Economic sustainability is achieved if the system offers a profitable and affordable energy supply in the present and the future. This is possible considering the cost of recovering the power supply potential, the operational costs imposed on users, and the need for financial support for the system [53].
In addition to the economic context, there is also a social context to assess the level of sustainability. The economic analysis includes the disparity in wages and costs of services between countries, which may mislead sustainability studies. Thus, the analysis of the level of sustainability may not be based entirely on the economic criteria. Different sustainability studies must go beyond economic analysis due to the intrinsic and complex nature of economic markets. For example, a photovoltaic system installed in Europe, but produced abroad, has an energy burden for society in terms of energy [54,55,56].
The social dimension of sustainability assesses the distributive effect of energy in society. It measures the acceptability, accessibility of the energy supply, and access to energy services for all segments of society [57]. It results in two main subdivisions: scope (extension) and inclusion. The scope defines the total area of society that is physically covered by energy services. Interconnected systems tend to exclude some areas either because they are in remote geographic locations or because investments and maintenance have a high cost for network extension [58].
Electrical exclusion can also occur due to the unequal supply of energy services. Parts of a society may be physically covered by energy services. However, they are still financially excluded from consuming these services. This type of exclusion comes from the unequal distribution of wealth in a country where a smaller percentage of the population has a greater share of its wealth [59]. About 2 billion people do not have access to modern energy services in the world today, and most of them still meet their essential energy needs such as cooking and heating from natural resources like burning wood and residual biomass [60]. These important social issues related to the use of energy, including poverty, quality of life, education, health, income inequality, and social justice, are themes that should be considered under the social dimension of sustainable development [61]. To improve access to electricity, researchers have assessed how electrical systems can be improved to reliably serve the population [62,63,64,65].
The institutional dimension of sustainability deals with the impacts and effects of institutions on social relations and how they can change the behavior of different social players. Institutions are influenced by players and their activities. On the other hand, institutions considerably influence players by shaping their interests and behaviors [66,67,68]. There is a mutual relationship between institutions and players in such a way that social values can shape institutional structures and, in response, institutional values appear as a measure for the rational assessment of players. Consequently, institutions act as an entity between actors and structures, which create new value orientations or reference points that influence the actors and their behavior [69].
From an economic point of view, the institutional dimension of sustainability can be defined as the capacity of an organization to produce results of value so that it acquires sufficient inputs to continue production at a constant or increasing rate [70]. The institutional scenario is a critical component of sustainability in which development policies are conceived, financed, implemented, and administered [69]. For the increasing public-private participation, institutions play an important role as a hub of governance and they balance the activities of the various dimensions of sustainability [71].
The energy crisis of the 1970s was the main driving force for expanding energy-oriented institutional capacities, particularly in the public sector. Government organizations at the national and local level develop their capacities both in decision-making and in managing energy-related issues. During the 1980s, government institutions became more experienced in managing energy-related activities with the adoption of energy policies, energy planning at national, regional, and local levels, and energy management in the industrial, building (commercial/residential), and transport arenas [72]. In addition to increasing public-private participation, institutions play an important role as a governance hub and balance the activities of the various dimensions of sustainability. With a focus on the sustainability of the energy system, a good institutional structure can play a key role in facilitating the implementation of energy plans and policies at international, national, regional, and local levels [73,74,75,76,77,78].
The technical dimension of sustainability measures the capacity of the system to provide energy to society reliably, efficiently, and with clean and renewable sources [79]. Technical sustainability is achieved if energy planning can meet the current and future energy needs of society [80]. This is possible if the power supply is reliable, efficient, and based on renewable energy [81] with locally available support services for the maintenance and execution of energy systems [82,83,84].

2.2. Data, Indicators, and Indices

The etymology of the word indicator comes from the Latin word indicare, which means to reveal, make public, and estimate. An indicator or set of indicators can transform basic statistical information to provide a deeper understanding of a problem or dimension and help develop a clear picture of the entire system, including its interrelationships and commitments [85]. When placed numerically, they are measured or derived from quantitative and/or qualitative measurements, which can be standardized for comparison with information from other areas or regions [86].
Indicators and indices are different, even though they may have a direct relationship. They are often used as synonyms, which can confuse several areas of application. An indicator is a tool that allows obtaining the information about a given reality, which can be individual data or a set of information [87]. A good indicator should be simple to understand, have statistical quantification with coherent logic, and effectively communicate the state of the observed phenomenon [88]. On the other hand, an index is defined as the final aggregate value of an entire calculation procedure in which the indicators are also used as variables that compose such an index, which is also referred to as high category indicators [89,90].
Figure 1 presents a visual representation known as the information pyramid, which shows how the amount of information relates to the increased condensation [88,89]. In Figure 1a, the existing relationship between primary data, indicators, and indices are presented. In Figure 1b, the relationship between the information and the intended public can be observed.
In Figure 1a, the top of the pyramid corresponds to the maximum degree of data aggregation, while the base represents the disaggregated data. The various assessment tools have their particularities. Some do not work with indexes or intermediate indicators because they use only primary and aggregate data to compose the final index, while others use all the steps of the information pyramid because they need a large amount of primary data to obtain the indicators, which are subsequently aggregated to obtain the final index [89].
Figure 1b shows the need to design indicators that provide relevant information for a specific target audience. This does not mean that different audiences receive different information, but that information must be written in a language that resonates according to the target audience. Scientists seek knowledge and understanding of the world, but legislators need to access information that helps them eleborate on viable policies, while the general public needs to know whether the goals and objectives are being met or not. Each audience has its language in terms of communication, and the criteria for using a specific indicator or index is whether it can communicate to the final users what they need to know [88,89,90,91].
The measurability of the sustainability is the key to implementing sustainable development [92]. Indicators and indices gain great importance and recognition as tools to formulate public policies and provide information of performance in areas such as economy, environment, society, and technology [93]. The indicators are adopted by countries and companies for their ability to summarize, focus, and condense the complexity of the dynamic environment. Thus, it can be a manageable amount of meaningful information [94]. The purpose of the tools for measuring sustainability is to provide decision-makers with a comprehensive and integrated assessment of the systems of nature and society such as medium and long-term perspectives to help them determine what actions should or should not be taken [95]. Energy sustainability indicators capture the complexity of energy systems and offer new insights about what can be communicated to the general public [96,97,98,99,100].
Some several indicators and indices exist for measuring energy and its relationship with sustainable development. The methods that created these indicators work as tools to measure sustainable development. The use of indicators is popular in many fields, especially economic indicators, indicators of poverty, health, environmental, and, more recently, energy indicators [91]. The theoretical framework created considered access to energy, energy poverty, energy consumption, energy security, the energy market, development, and energy integration, as some of the concepts used for the revised tools [101,102,103]. These concepts were chosen after conducting a literature review that considered recent research on indicators and indices of energy sustainability and also tools already consolidated in the scientific literature.

2.3. Energy Sustainability Indicators

The Latin American Energy Organization (OLADE), the Economic Commission for Latin America and the Caribbean (ECLAC), and the German Technical Cooperation Agency (GTZ) have presented eight indicators to capture energy sustainability including three indicators from the economic dimension, three indicators from the environmental dimension, and two indicators from the social dimension. The Energy Sustainability Indicators (ESI) are presented in Table 1. This set of indicators is of great historical importance, as they are part of the pioneers promoting the union between energy and sustainability, and serving as the basis for many types of research and development of other indicators and indices of energy sustainability [104]. In this methodology, all indicators have the same weight. There is no difference between them, even though they are related to different dimensions [105].
The ESI was applied in Latin America and the Caribbean to understand the positioning of energy sustainability in these regions [104] as well as used in Mexico to analyze local energy policies [105]. However, no aggregate index was calculated even though the indicators capture important aspects of the energy system under study. A critique of the methodology shows that these indicators were chosen arbitrarily, not being used to provide an international comparative framework [44].

2.4. Energy Indicators for Sustainable Development

Specifically designed by the International Atomic Energy Agency (IAEA) to measure energy supply for sustainable development, it is a project of energy indicators in cooperation with several international organizations, of which are: United Nations Department of Economic and Social Affairs (UNDESA), European Statistical Office (Eurostat), European Environment Agency (EEA), and the International Energy Agency (IEA) was initiated in 1999 [61].
The original name was Indicators for Sustainable Energy Development (ISED) and was subsequently modified, since some of the experts considered that the term “sustainable energy development” referred only to renewable energies, which limits choices related to energy. In a social dimension, the theme is equity with the sub-theme being accessibility, wherein it can be defined as “share of households (or population) without electricity or commercial energy, or heavily dependent on non-commercial energy.” In addition to the social context, the economic and environmental context must also be assessed.
In the first phase, the project was comprised of a set of 41 indicators adapted to measure sustainability in energy systems [86]. In the second phase, this original set of indicators was reduced and the Energy Indicators for Sustainable Development (EISD) consist of a set of 30 group indicators, classified in the dimensions of social, economic, and environmental sustainability [61].
The indicators represent an instrument for policymakers to evaluate and design programs and strategies and monitor progress toward a more sustainable future at a regional level. The Energy Indicators for Sustainable Development (EISD) can assist countries in their efforts to assess the progress made in implementing sustainable energy development strategies and identify areas in which measures and specific policies should be targeted. Case studies developed in several countries, including Brazil, demonstrate the advantages of using this integrated approach in the formulation and implementation of EISD [106,107,108,109].
The set of indicators of the EISD have their scope in the explanation of sustainable access to energy, but it has fundamental flaws, such as those found in the ESI indicators discussed earlier. The interpretation of changes in a large number of indicators can make it impracticable to compare the performance of countries or regions with this set of indicators [44]. A study using EISD was conducted for Brazil and, according to its authors [108], some flaws in the set of indicators may be shown, especially in the face of the inequalities in income that occur in the country and the existing regional differences because the lack of access to electricity mainly affects poorer regions. Because of this situation, concerning accessibility and energy viability, the fact that the lack of access to the electrical grid is mainly a regional problem, it is indirectly linked to the concentration of the population in rural areas and the conditions with poor distribution of income among the various regions of the country [110].

2.5. Energy Development Index

The Energy Development Index (EDI) was elaborated on by the IEA to better understand the role that energy plays in human development. It focuses on the transition made by a country or region to the use of modern fuels and energy services. It does not refer to whether the transition is sustainable, based on the economic, social, or environmental conditions of the country [111]. The composition of EDI is shown in Table 2.
The indicators that compose the EDI seek to quantify the availability of modern fuels, the consumption of these fuels, and how this consumption supplies basic needs and human development by segregating the energy development at domestic and community levels [102].
EDI was created to measure the transition to modern fuels in a country or region. It is calculated as the arithmetic average between four indicators that represent this transition, represented as output variables measured only in the consumption stage, without the inclusion of any time dimension [96]. The EDI shows the production of the country under analysis, but it does not show whether the effort channeled to modernize the fuels is correct or not. Since the limited supply of energy can restrict consumption to values lower than demand, EDI fails to address sustainability subjectively, and this is currently one of the biggest problems of developing countries [44]. A country can quickly increase its production or import energy resources and achieve a high level of EDI, while it can indent the index due to the depletion of its local resources or weaken its internal economy through increased imports.

2.6. Multidimensional Energy Poverty Index

Multidimensional Energy Poverty Index (MEPI) is a more recent metric for measuring access to energy. It focuses on depriving modern energy as opposed to accessing energy, and it captures both the incidence and intensity of energy poverty. It consists of six indicators in five dimensions, comprising the basic services of cooking, lighting, appliances, entertainment/education, and communication [112].
The MEPI methodology is derived from the literature on measures of multidimensional poverty of the Oxford Poverty and Human Development Initiative (OPHI) institution, which defends the need to concentrate on human poverty, considering the lack of opportunities and choices to live a basic human life [113,114]. Essentially, the MEPI captures the set of energy deprivations that can affect a person who is now identified as poor in energy if the combination of privations faced by such a person exceeds a predefined threshold.
The MEPI index focuses on energy services and is based on data related to energy deprivations, as opposed to deriving information indirectly through variables that are presumed to be correlated (e.g., energy or electricity consumption). Additionally, it captures both the incidence (number of poor people in energy) and the intensity (how much these people are poor in energy) [115,116,117].
In the research [112], the dimensions, indicators, and variables of the MEPI index are discussed. In this case, in the dimension, there is an indicator, a variable, and an evaluative item. In the dimension of the way the kitchen is used, the indicators are internal pollution and technology used in which it is evaluated whether the cooking is carried out with internal or external fire and what type of fuel is used. Lastly, the evaluation item can be answered simply as yes or no for the specific use of the technology, or it can be explanatory in terms of defining which fuel is used.
Specifically related to energy poverty, MEPI is a composite index that measures the deprivation of energy access [112]. This metric shows the communication to some countries that are poor in energy due to the direct measurement of useful energy needs but does not assign reasons for the cause or give suggestions on how the situation can be reversed [115]. Energy poverty is a dynamic and complex concept [118], and, although MEPI captures the reliability of the energy supply as part of deprivation, it does not deal with the issue of sustainability directly.

2.7. Sustainable Energy Development Index

The Sustainable Energy Development Index (SEDI) was created more recently to fill the gap between environmental indicators and the dimension of energy sustainability [44]. It comprises an index consisting of eleven indicators, distributed in five dimensions: technical, economic, social, environmental, and institutional. The study was initially carried out in 20 countries considered to be developing and then in 30 developing countries. A comparison of the results with the Human Development Index (HDI), the EDI, and the Environmental Performance Index (EPI) was also performed [43].
Due to the gap between sustainable development and energy development, the SEDI index was considered a proposal to measure sustainable energy development. However, as with the other revised tools, SEDI is not immune to the inherent weaknesses of data masking, where strong performances of some variables tend to conceal the weaknesses of other variables [44]. The value of SEDI should be understood as a relative value, whose total implication at the level of each country should be derived from the analysis of the various dimensions of sustainability. Therefore, if a country studies the performance of individual dimensions under the SEDI methodology, it will be able to identify political pathways to improve the weaker dimensions, which can help optimize the socio-economic benefits derived from access to sustainable energy.

2.8. Sustainable Development of Energy, Water, and Environment Systems Index

The Sustainable Development of Energy, Water, and Environment Systems Index (SDEWES) is a composite index created to assess the sustainable development of local energy systems [119]. It consists of a single set of seven dimensions and 35 main indicators and was initially applied to a sample of 22 port cities in the Mediterranean, based on three energy scenarios.
The scenarios considered the reductions in final energy consumption per capita for a given area of the city, based on intelligent energy measurements [120]. More recent studies have applied the proposed index to 12 cities in Southeastern Europe and 26 global cities, including Rio de Janeiro, to compare the performance related to SDEWES [121,122]. As a result of the application of the SDEWES index, this paper presents the classification of the best sustainable practices of the sampled cities, separated by dimensions. These best practices are focused on action-oriented measures that can be implemented by other cities to improve the performance of dimensions. The actions range from buildings based on cogeneration with combined heat and energy cycle, zero energy buildings, sustainable water management, large urban forests, urban planning, and incentive policies for research, development, and innovation [123,124,125,126,127].
The SDEWES emphasizes the need to implement best practices across a wide horizon of dimensions that will collectively serve for the sustainable development of energy, water, and environmental systems in cities [119]. It provides a foundation for cities to progress to smarter energy systems and, at the same time, cities need to be compared based on the magnitude, efficiency, and intensity of energy use and CO2 emissions [128]. The SDEWES Index assesses the performance of local energy systems based on an integrated approach. The approach of a composite index includes the environmental, social, and technological context of local energy systems. The results indicate that well-articulated political efforts are needed to bring the local energy systems closer to sustainable development [129].
In the application of SDEWES Index dimensions and indicators, only two evaluation levels are presented including the dimension and the indicator. In the dimension, the concepts are broader such as when discussing energy consumption, in which there are several indicators, such as energy consumed in buildings, transportation, or per person. In addition to these metrics, other metrics can be used, such as how much energy is used for heating or cooling [121].

2.9. Regulatory Indicators for Sustainable Energy

Regulatory Indicators for Sustainable Energy (RISE) was created in 2014 with 28 indicators and 85 sub-indicators, divided into four categories and three dimensions, with pilot methodology initially applied in 17 countries [130]. In 2016, the methodology received some modifications, and RISE became defined as a set of 27 indicators and 80 sub-indicators in three dimensions. Its application had coverage in 111 countries, representing 97% of the world population, which provides an instrument to develop energy policies and regulations in each of the three dimensions used: sustainable access to modern energy, energy efficiency, and renewable energy [131].
In December 2018, a fourth dimension was incorporated (clean cooking), and 12 other countries were added to the overall study, which totaled 133 countries categorized into 32 indicators. Since this new dimension is a pilot, the methodology of calculating the indicators remained the same as the previous one with three dimensions [132].
Table 3 shows the dimensions and indicators of RISE in 2018. As a result of a partnership between the United Nation (UN), World Bank, and IEA, RISE uses the goals set out in the Millennium Development Goals (MDG) and in the efforts of the Sustainable Energy for All (SEforALL) initiative, which has led several stakeholders to commit to achieving three goals up to 2030. This was done to ensure universal access to modern energy services, which doubles the rate of energy efficiency improvement and doubles the share of renewable energy in the global energy matrix [133].
RISE ranks countries in a green zone of strong performance, a yellow zone of medium performance, and a red zone of poor performance. Each indicator targets an important policy element or regulatory regime to mobilize the investment, such as the establishment of planning processes and institutions, the introduction of dedicated incentives or support programs, and the assurance of funding energy access programs, and encouraging renewable energies. RISE indicators provide a comprehensive view of government support for sustainable energy and actions taken to transform this support into reality [131].
While many of the countries with the greatest impact on global sustainable energy outcomes are developing or develop strong policies and regulations, the results on access to energy are often distorted, especially in countries with particularly low electrification rates [44]. In the case of Brazil, although there is a high electrification rate, RISE classified the access to electricity with strong performance (100%). However, it did not consider the reality of isolated electrical systems, which correspond to more than half of the national territorial area and comprise thousands of people with no access to electricity [57]. This shows the need to regionalize the indicators for each reality. At the same time, it shows the difficulty of having an indicator to a country or continent.

2.10. Synthetic Index of Sustainable Energy Development

The Synthetic Index of Sustainable Energy Development (SISED) was created in 2016, based on the ESI methodology, which was proposed by the World Economic Forum in 2002. SISED was planned to be a synthetic index consisting of three indices from a set of indicators to promote sustainable energy policies to European countries [134]. The index is based on 33 variables, grouped into nine categories, comprising three distinct dimensions: energy supply safety (SES), competitive energy market (CEM), and environmental protection (EP). The variables were chosen according to the guidelines of the International Atomic Energy Agency (AIEA) [86] and are described in Table 4.
The SES dimension considers three categories: energy dependence, which shows the dependence on imports to meet the country’s energy needs, and considers that diversifying the energy supply is a determinant factor for safety. This is the primary energy production category, which assesses the future insertion of energy sources about the current availability of energy resources, and the power consumption category, which considers that reducing the amount of energy consumed can make society more independent when concerning the energy supply [135].
The CEM dimension comprises three categories. Energy efficiency is one of the foundations for seeking sustainable development, which has, in its energy intensity, its measurement form and relates the energy consumption of an economy with its global energy efficiency. The energy market category has the resource productivity variable as a way of measuring the availability of national resources with the corresponding production. The category of competitive prices has a direct relationship with efficiency gains in the economy because, theoretically, the liberalization of the electricity market aims to increase competition, which decreases the final price for consumers.
The Environmental Protection dimension (EP) has three other categories. The category of renewable energy sources is of fundamental importance for sustainable development, ranging from environmental protection to safety and diversification of energy supplies [136]. The category of greenhouse gas emissions, which evaluates the impact on the atmosphere of energy-related activities. The energy tax category, which aims to facilitate a change to renewable energy sources in the primary energy matrix.
Although the indicators discussed above have been created for a specific purpose, in some specific aspects, they have omitted the dimension of energy sustainability, responsible for the alliance between the technical dimension and other dimensions of sustainability (environmental, economic, social, and institutional). While the MEPI tries to capture the level of energy poverty, EDI has the dimension of access to energy, and SISED has environmental protection with a focus on energy security. They are not specifically designed to capture the dimension of sustainability. A country with a high index of human development and a high rate of access to modern energy may have little or no self-sufficienct natural resources. Even the domestic resources of rich countries could exhaust so quickly that it is uncertain how future supplies will be sustained. Therefore, high EDI does not necessarily indicate a sustainable energy future [44].
Conversely, a country with a low EDI or low MEPI can still have a moderate level of energy sustainability. In the current context in which sustainable development cannot be separated from energy concepts, the lack of energy sustainability needs to complement existing indices and indicators to provide a holistic view of the energy issues by environmental issues.
In general, the indices and indicators presented in this paper can be good complements to the energy metric tools on the path of sustainable development. Indicator systems, in any sphere, have become an important element in determining public and social development. When energy sustainability indicator systems are recognized and accepted, they become important components in the formulation of energy policies, which initiates an effective process of changes in priorities and behavior of the social players. In this sense, it is important to develop, test, and implement tools that capture all the complexity of concepts, such as energy and sustainability, without reducing the significance of each element that is part of any evaluation model.

3. Methodological Procedures

The research is qualitative in that it assigns numerical values to the observations of this investigation, considering that this research has an inductive and interpretive approach. Initially, a literature review on energy sustainability indicators was carried out, through documentary research in articles and documents of the institutions that created the studied methods in addition to critical studies on each tool [137].
Because of the review, eight tools were selected based on four selection criteria: (1) historical importance and origin (authors and institution), (2) the concept of sustainability contained in the tool, (3) the characteristics of each method, and (4) its theoretical and practical applicability. After selecting the tools, they were compared through six categories and six subcategories, as can be seen in Figure 2.
After criterion selection, a criterion comparison method is used. The comparison criteria are grouping, scope, sphere, data (subcategories typology and aggregation), participation, and interface (subcategories complexity, presentation, openness, and educational potential). Each of the categories and subcategories used provides a guideline for comparison between the different methodologies of the indicators and energy sustainability indices revised. Figure 2 is the representation of the methodological structure used.
As previously presented in Figure 1, the base of the pyramid represents the largest volume of data to be analyzed. These data are evaluated to obtain an indicator and then an index. In addition to the information categories presented in Figure 1, Figure 2 shows the existence of different types of information and this information is responsible for forming the indexes.
The research carried out is descriptive and exploratory with a predominantly qualitative approach [138]. For this purpose, only secondary data from bibliographic and documentary research was used, which served as a basis for categorizing the tools in the comparative study. It was initially organized by grouping with representation in group indicators and composite indices, which follow a chronological order of creation of the tools. Subsequently, based on the comparison criteria listed above, each analytical category is described separately, to show the results achieved in comparing the tools.
Referring to the grouping category, three major divisions can be found in the literature [109] such as individual indicators, which provide specific values that are easy to interpret and less susceptible to synthesis errors, when compared to other indicators of groups or compound indices. However, they are one-dimensional and, therefore, can omit vital information inherent to the degree of complexity of the concepts measured, such as sustainability and energy. The second major division includes disaggregated individual group or panel indicators, corresponding to a variety of indicators to deal with issues that require the use of many component variables of different dimensions. The third major division includes compound or aggregate indices, which are simple and easy to interpret as the one-dimensional indicators. At the same time, they are condensed to a set of variables that capture the various dimensions of the theme in question, which take advantage of the points and avoid the weaknesses of the previous categories [112].
The scope category comprehends the dimensions of sustainability encompassed by the evaluation tool and how they are used. It is based on what is measured, while the sphere category analyzes the field of application of the tool and which regions the system is designed for. The tools were classified according to geographic boundaries (global, continental, national, regional, or local).
The data category analyzes the typology of the data used by the tool and how they are treated in the evaluation, which entails the degree of data aggregation relative to the information pyramid shown in Figure 1. The typology refers to the use of qualitative and/or quantitative information [139]. The degree of aggregation of a system of indicators will be observed by the location of its indices, indicators, and data in the information pyramid [89].
The participation category analyzes the form and intensity of the participation of different social players in the development of indicator systems under analysis. It assists in the strategy of ordering knowledge and information processing, with the distinction between models based on a top-down approach, which uses aggregated data to predict purposes, while the bottom-up approach uses disaggregated data for exploration purposes [140].
In a top-down approach, guidance is given primarily by specialists, who perform the fragmentation of the larger system to improve understanding the composition of the subsystems. In the bottom-up approach, the largest weight is given to the players involved in the process, including the target audience. The most basic elements are described in detail and are later associated with a larger subsystem within a given level, while repeating the process until completing the highest level of the proposed objective [141]. Both approaches are extreme points of the same line, and each evaluation tool analyzed must be inserted into this context, which can carry characteristics of the two approaches. This results in a hybrid approach [142].
The interface category analyzes the tools reviewed by the user’s perspective. It considers the ease of observing and interpreting the results obtained. It represents how the tool communicates with the public as well as the degree of complexity and transparency of the tool, its presentation structure, and its pedagogical potential in terms of environmental education. This category is related to the degree of ease to interpret the results obtained in the clear and objective visualization of these results, with the view to better guide decision making. It also has a great influence on the ability to change social behaviors and act as a pedagogical tool in environmental issues. To detail the interface analysis category, four subcategories were divided.
  • Complexity: The main characteristic of this subcategory is to evaluate the level of difficulty of the calculations involved in obtaining the studied indicators and indices to show a high, medium, or low degree of complexity. Objectively evaluating the degree of complexity of an isolated tool is difficult. However, a comparative study is possible to hand out a simplified evaluation.
  • Presentation: It evaluates the visual presentation of the process to the end-user such as chart types, tables, maps, etc.
  • Openness: Subcategory directly related to complexity and presentation, which seeks to show at what degree the implicit values of the evaluation system are disclosed to the public. The system will have greater openness and possibility of observing information in the tool interface, including the weights attributed to the data analyzed, at the different levels of the information pyramid (see Figure 1).
  • Educational Potential: Verifies the learning capacity of the concepts involved in sustainability and energy by the user. Analyzes the pedagogical potential of application for the target audience or civil society.
The objective of this subdivision is to consider distinct and complementary aspects of the evaluation tools.

4. Data Analysis and Results

In group indicators, it is more appropriate to capture complex information than individual indicators due to the greater number of parameters used to measure complex issues such as human development and sustainability, which are multi-dimensional [143,144,145]. On the other hand, compound indices reduce the information set that would have been provided by a myriad of group indicators to a level that makes the analysis convenient and provides a uniform scale over which the comparative performance of the indicators can be measured [44]. Compound indices were created as a commitment between the simplicity of individual indicators, which are one-dimensional, and need to realize the multi-dimensional nature of some issues. They represent an attempt to overcome the shortcomings of the simple indicators and, at the same time, produce a result that condenses the information with easy-to-interpret metrics, which is a sort of composition between the simple and group indicators.

4.1. Grouping

Table 5 presents in chronological order the creation and the revised tools, with indicators and indices classified, according to grouping categories, their respective initiators, and references. As observed in the results of the grouping category, the group indicators were the pioneer methods used by the revised tools initially in 1997 with the ESI and soon after 1999 with the EISD.
The EDI appears as a compound index only in 2004, which is then followed by other tools. In 2014, RISE was created as a set of group indicators in which it resumes the use of pioneer methodologies, but with a modernized approach and interface. The choice of the best grouping method to measure energy sustainability depends on the opinion and experience of each user or researcher. Therefore, the knowledge of the qualitative characteristics of the tools is as important as the quantitative results, which are delivered.

4.2. Scope

Regarding the scope category, which analyses the dimensions in which the tool operates, the revised tools start with two dimensions (EDI) and vary between the three basic dimensions of sustainable development (ESI and EISD). For five dimensions inserting new scope criteria, such as the social and institutional dimensions (SEDI), it also completely alters the basic dimensions with scopes adapted to capture complex themes, such as cooking and lighting (MEPI), energy consumption and climate (SDEWES), access to energy and renewable sources (RISE), and environmental protection (SISED). Some dimensions are created according to the application and use of each tool. The scope category is summarized in Table 6.
A pertinent observation is a tool that uses multiple scopes. The importance of this dimension within the system is proportional to the number of dimensions used. Thus, each dimension of the SEDI has one-fifth weight in the final value of the index because it is composed of five dimensions in total. With this, an excess of dimensions can reduce the relative importance of them within the evaluation tool and impair the validity of the results. On the other hand, a system with few dimensions ends up having a limited relevance in the issue of capturing the complexity to be measured while something observed in the EDI has only two dimensions.

4.3. Sphere

The sphere category is associated with the range of the field of application tools. ESI and SISED have great range and can be applied at global, continental, national, regional, and local levels. The MEPI and SDEWES are more restricted to a local application, as presented in Table 7. Analyzing this item implies knowing the potential of a comparative use of the tool, which may or may not be applied between different scenarios, alone or in comparison with other systems.

4.4. Participation

In the top-down approach, a set of criteria and indicators initially generated is used by a team of experts who adapt and modify this set according to the local situation. In the bottom-up approach, local actors are actively involved in a participatory manner by proposing criteria and indicators based on their perception of the individual situation [146,147].
Although they are fundamentally different approaches, there is a possibility to use the elements of each of the previous approaches without creating conflicts. Top-down processes defined by experts may not have legitimacy for stakeholders, while a participatory approach by local actors can increase the legitimacy of the sets of criteria and indicators. In this way, a hybrid approach allows the improvement of mutual learning and the sharing of experiences to develop adaptive strategies over the entire process, which ends up resulting in the development of more practical and meaningful indicators [148].
In the participation category, the type of approach determines the players involved in the choice of indicators. This has a direct impact on the method used to measure energy sustainability because of the implicit concept of sustainable energy development of a tool that becomes apparent through its dimensions and its indicators. Therefore, evaluation systems reflect the values or beliefs of certain players about energy sustainability. Furthermore, according to Table 8, among the revised tools, there was a tree hybrid approach (MEPI, SDEWES, RISE, and SISED), and five top-down approaches (ESI, EISD, EDI, and SEDI). None of them exclusively uses the bottom-up approach.

4.5. Data

In Table 9, we have the comparative analysis according to data criterion, subdivided by the typology and aggregation. Referring to the data category, in most of the tools, the data worked is quantitative due to the need to reconcile the various existing tools to be compatible with each other [88]. Nonetheless, some tools can associate the concept of sustainable development with the quality of the system, which constitutes subjective and non-quantifiable quantities.
According to Table 9, we have five tools whose typology is uniquely quantitative (ESI, EISD, EDI, SEDI, and SISED) and three others with mixed typology, which work quantitatively and qualitatively (MEPI, SDEWES, and RISE). None are exclusively qualitative.
In the subcategory of data aggregation, the location of indices and indicators in the information pyramid (Figure 1) is the condition of the degree of aggregation. The tools whose grouping results in a general composite index have a maximum degree of aggregation such as EDI, MEPI, SEDI, SDEWES, and SISED. The existence of a highly aggregated index creates a condition to use intermediate indicators to compose the general index (EDI, SEDI, and SDEWES), intermediate indicators and sub-indicators (MEPI) or sub-indices, intermediate indicators, and sub-indicators (SISED). In turn, the tools that do not have a general aggregate index are composed only of indicators, such as ESI, or are formed by indicators and sub-indicators such as the case of EISD and RISE.

4.6. Interface

The interface category is also an important element in evaluating the tools. The efficiency of the results obtained by the indicator system is directly related to the capacity that the tool has in contributing to the increased critical consciousness of the players and users involved. Therefore, the interface category involved in the results regarding the data availability and visual presentation is of fundamental importance in the final process of what will be later released. Table 10 shows the interface category, its subcategories’ complexity, presentation openness, and educational potential.
With the advent of computer graphics and the advancement provided for data analysis, the tools have greatly evolved their visual presentation over time. This is clear when we observe the eight revised tools within the sub-categories: complexity, presentation, openness, and educational potential. As shown in Table 10, the ESI, EISD, and EDI tools contribute in a historical way to the development of methodologies in elaborating energy sustainability indicators but do not allow interaction with the end-user. In turn, newer tools such as SDEWES and RISE have interactive websites with rich visual material, richly illustrated reports, and elaborated methodologies that offer a high degree of openness.
Regarding the subcategory complexity, it was verified that half of the revised tools perform calculations as simple average and linear normalization to reach the final value of the index or indicator (ESI, EDI, SEDI, and RISE). Two tools have a medium complexity with decomposition methods (EISD) or Z-score (SISED) to reach the final value. Two other tools work with high complexity in the elaboration of the final result. The MEPI uses probabilistic functions and analysis of uncertainties because the weight of the indicators is attributed to qualities beyond quantities. The SDEWES has high complexity because it depends on five stages to construct the final index.
The complex themes covered by the tools must be understood and communicated simply and objectively, where interaction with the user and the results delivered to the general public need visual presentations, which facilitate the interpretation by considering the concepts of energy and sustainable development. This helps and impacts directly on the formulation of public policies based on the perception of the players involved, regarding the change in social behavior.
The potential educational subcategory verifies the ability to represent the results to the public, which are achieved by the tool under analysis. It also addresses how much this educational potential can serve to inform the class of scientists in addition to the general public. Thus, it is possible to compare this subcategory with the information pyramid (Figure 1). ESI and EISD have historical value proven by the pioneering in the application and theoretical basis of great educational value, which are characteristics that serve to educate scientists. However, it does not present an interface of interaction with the end-user. EDI has great pedagogical value for scientists because its concept is very close to HDI, which also has educational potential for the general public, even if it does not interact with the end-user.
MEPI has an innovation in the approach for the concept of energy deprivation. SEDI allows the comparison between countries in a simplified way and SISED makes it possible to direct the results toward the elaboration of public policies, which makes these tools have a great pedagogical value for both scientists and the general public, even though none of them have an interactive interface with the general public. This lack of interactivity was overcome by SDEWES and RISE. Both tools have a high educational value and allow a high degree of interaction with the end-user, which also serves as a pedagogical potential for scientists to develop new tools based on their visual interface.

5. Conclusions

The objective of this work was to conduct a conceptual review of eight tools to measure energy sustainability and to elaborate on a qualitative comparison between them. A conceptual framework was created based on selection and comparison criteria. The categories used in the comparison follow the general assumption that the revised tools should represent socio-environmental aspects relevant to society, which have a practical and efficient measurement process help the manager of energy policies in the decision-making process.
Energy sustainability indices and indicators serve as a basis for learning about society, the economy, the environment, institutions, technologies, and the interactions between these areas. When carefully chosen and effectively communicated, it can provide information in a politically neutral manner, contribute to engaging the government and citizens in a shared debate on the meaning of sustainability and energy, with the final purpose of developing common political objectives. As shown in this paper, the effective problem in measuring energy sustainability is related to the use of one or more tools that capture the entire complexity of the system, without reducing the significance of each of the scopes used.
In this context, a considerable challenge is to overcome the existing limitations in the use of predominantly quantitative methodologies. The concept of sustainable development is related to different dimensions, which are not necessarily associated with measurable physical quantities. Examples of this are the social and institutional dimensions because, even if it is possible to associate these dimensions with quantitative indicators, this association suffers limitations according to the variable that is sought to observe. In recent years, different systems have been looking to work with the dimensions of sustainability qualitatively and this approach almost always reveals aspects that are imperceptible in quantitative analysis. The great challenge regarding using the qualitative approach is being able to formulate tools that do not sacrifice the advantages of using quantitative indicator systems.
Some important points highlighted in the research show that the interface of the tools is also an important element within an evaluation system. The theoretical discussion of this work revealed that the effectiveness of the tool is related to the ability to increase the critical awareness of the actors involved, which is promoted by the level of user interaction with the tool. It was shown that the most current tools already have online interactivity and some allow real-time updating, according to the database used. The aggregation of data itself is usually carried out using mathematical models that contribute to simplifying the inherent complexity of the evaluated system.
Based on the comparative and qualitative approach of the work, it can be said that the implicit concept of a tool’s energy sustainability becomes apparent through its dimensions and selected indicators. The approach in terms of participation determines the actors involved in choosing the indicators. Therefore, an evaluation system reflects the values or beliefs of certain actors to the concept of sustainable development and it is important to verify which systems are used in selecting these indicators and their use.
The concepts related to sustainable development may not be quantitatively associated with measurable physical quantities, and, thus, a qualitative analysis helps to overcome these limitations. The results of the research showed characteristics such as dimensions used, typology and data aggregation, degree of openness of the tools, complexity in use, educational potential, and participation of the actors in each tool studied. This is one of the reasons for the emergence of new qualitative indicators, especially in the last decade. Increasingly, there will be a need for qualitative approaches to assist in interpreting complex natural phenomena and the various dimensions that comprise sustainability. The present research meets these needs and seeks to show a possible way to fill this gap.
Lastly, some suggestions for future research include using the proposed conceptual framework to compare new tools, perform a quantitative analysis of the evaluated tools and correlate the results with the qualitative analysis, expand the qualitative analysis proposed for some important concepts in the energy sustainability assessment processes and the actors that participate in it, such as legitimacy, culture, and power, evaluate the selected tools based on the proposed analysis categories in the perception of its users, and validate the tools with a practical application in different local and regional realities.

Author Contributions

R.N.M. was initially responsible for conducting the research. R.M.K., J.A.S.d.S. and B.R.P.d.R. reviewed the theoretical and technical concepts of the article. S.F.S., A.N., E.C.F. and L.H.M. reviewed the text and its organization. W.G.B. adjusted references and the template. All authors have read and agreed to the published version of the manuscript.


This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.


The authors would like to thank CAPES (Coordination of Higher Level Personnel Enhancement) for supporting research through the Pro-Integration Project, which integrates the High Amazonian Studies Center (NAEA) and the Graduate Program in Electrical Engineering (PPGEE) of the Federal University of Pará (UFPA), the Graduate Program in Natural Sciences (PPGCN) of Pará State University (UEPA), the Space Research Institute of the Regional Center of the Amazon (INPE), and the Operations and Management Center of the Amazon Protection System (CENSIPAM). One of the authors would like to acknowledge FAPESC under project code 2019TR84 for its financial support in this research.

Conflicts of Interest

The authors declare no conflict of interest.


  1. Lindsey, T.C. Sustainable principles: Common values for achieving sustainability. J. Clean. Prod. 2011, 19, 561–565. [Google Scholar] [CrossRef]
  2. Barbosa, G.S.; Drach, P.R.; Corbella, O.D. A conceptual review of the terms sustainable development and sustainability. Int. J. Soc. Sci. 2014, 3, 1–15. [Google Scholar]
  3. Eissmann, J.C.; Stefenon, S.F.; Arruda, P.A. Strategic management as a tool for corporate governance: A case study. ESPAC 2017, 38, 22. [Google Scholar]
  4. Agostinho, F.R.; Rocca, G.A.; Ferreira, F.S.; Stefenon, S.F. Study on the financial viability in technological updating of a manufacturing plant: Use of highly efficient electric motors and LED lighting. ESPAC 2017, 38, 5. [Google Scholar]
  5. Harlow, J.; Golub, A.; Allenby, B. A review of utopian themes in sustainable development discourse. Sustain. Dev. 2013, 21, 270–280. [Google Scholar] [CrossRef]
  6. Du Pisani, J.A. Sustainable development historical roots of the concept. Environ. Sci. 2006, 3, 83–96. [Google Scholar] [CrossRef]
  7. Schlör, H.; Fischer, W.; Hake, J.F. Methods of measuring sustainable development of the German energy sector. Appl. Energy 2013, 101, 172–181. [Google Scholar] [CrossRef]
  8. Bolis, I.; Morioka, S.N.; Sznelwar, L.I. When sustainable development risks losing its meaning. delimiting the concept with a comprehensive literature review and a conceptual model. J. Clean. Prod. 2014, 83, 7–20. [Google Scholar] [CrossRef]
  9. Costanza, R.; Patten, B.C. Defining and predicting sustainability. Ecol. Econ. 1995, 15, 193–196. [Google Scholar] [CrossRef]
  10. Geissdoerfer, M.; Savaget, P.; Bocken, N.M.; Hultink, E.J. The circular economy–A new sustainability paradigm? J. Clean. Prod. 2017, 143, 757–768. [Google Scholar] [CrossRef] [Green Version]
  11. Moldan, B.; Janoušková, S.; Hák, T. How to understand and measure environmental sustainability: Indicators and targets. Ecol. Indic. 2012, 17, 4–13. [Google Scholar] [CrossRef]
  12. Marinescu, C.; Cicea, C. Measuring renewable energy influence on economic growth: Evidence from a panel of European countries. Multidiscip. Sci. GeoConf. Surv. Geol. Min. Ecol. Manag. SGEM 2015, 3, 407–414. [Google Scholar]
  13. Ntanos, S.; Skordoulis, M.; Kyriakopoulos, G.; Arabatzis, G.; Chalikias, M.; Galatsidas, S.; Batzios, A.; Katsarou, A. Renewable energy and economic growth: Evidence from European countries. Sustainability 2018, 10, 2626. [Google Scholar] [CrossRef] [Green Version]
  14. Apergis, N.; Payne, J.E. Renewable energy consumption and economic growth: Evidence from a panel of OECD countries. Energy Policy 2010, 38, 656–660. [Google Scholar] [CrossRef]
  15. Badami, M.; Fambri, G.; Mancò, S.; Martino, M.; Damousis, I.G.; Agtzidis, D.; Tzovaras, D. A decision support system tool to manage the flexibility in renewable energy—Based power systems. Energies 2020, 13, 153. [Google Scholar] [CrossRef] [Green Version]
  16. Tsai, W.T. Energy sustainability from analysis of sustainable development indicators: A case study in Taiwan. Renew. Sustain. Energy Rev. 2010, 14, 2131–2138. [Google Scholar] [CrossRef]
  17. Lee, M.; Park, J.; Na, S.-I.; Choi, H.S.; Bu, B.-S.; Kim, J. An analysis of battery degradation in the integrated energy storage system with solar photovoltaic generation. Electronics 2020, 9, 701. [Google Scholar] [CrossRef]
  18. Khorasany, M.; Azuatalam, D.; Glasgow, R.; Liebman, A.; Razzaghi, R. Transactive energy market for energy management in microgrids: The monash microgrid case study. Energies 2020, 13, 2010. [Google Scholar] [CrossRef]
  19. Vivien, F.-D.; Nieddu, M.; Befort, N.; Debref, R.; Giampietro, M. The hijacking of the bioeconomy. Ecol. Econ. 2019, 159, 189–197. [Google Scholar] [CrossRef]
  20. Georgescu-Roegen, N. Energy analysis and economic valuation (1979). From bioeconomics to degrowth georg. New Econ. Eight Essays 2011, 45, 114–141. [Google Scholar]
  21. Hall, C.A.S.; Lambert, J.G.; Balogh, S.B. EROI of different fuels and the implications for society. Energy Policy 2014, 64, 141–152. [Google Scholar] [CrossRef] [Green Version]
  22. Lambert, J.G.; Hall, C.A.S.; Balogh, S.; Gupta, A.; Arnold, M. Energy, EROI and quality of life. Energy Policy 2014, 64, 153–167. [Google Scholar] [CrossRef] [Green Version]
  23. Fizaine, F.; Court, V. Energy expenditure, economic growth, and the minimum EROI of society. Energy Policy 2016, 95, 172–186. [Google Scholar] [CrossRef]
  24. Jebaraj, S.; Iniyan, S. A review of energy models. Renew. Sustain. Energy Rev. 2006, 10, 281–311. [Google Scholar] [CrossRef]
  25. Kaczmarczyk, M.; Sowiżdżał, A.; Tomaszewska, B. Energetic and environmental aspects of individual heat generation for sustainable development at a local scale—A case study from Poland. Energies 2020, 13, 454. [Google Scholar] [CrossRef] [Green Version]
  26. Haider, H.; Hewage, K.; Umer, A.; Ruparathna, R.; Chhipi-Shrestha, G.; Culver, K.; Holland, M.; Kay, J.; Sadiq, R. Sustainability assessment framework for small-sized urban neighbourhoods: An application of fuzzy synthetic evaluation. Sustain. Cities Soc. 2018, 36, 21–32. [Google Scholar] [CrossRef]
  27. Maiolo, M.; Carini, M.; Capano, G.; Piro, P. Synthetic sustainability index (SSI) based on life cycle assessment approach of low impact development in the Mediterranean area. Cogent Eng. 2017, 4, 1410272. [Google Scholar] [CrossRef]
  28. Suganthi, L.; Samuel, A.A. Energy models for demand forecasting—A review. Renew. Sustain. Energy Rev. 2012, 16, 1223–1240. [Google Scholar] [CrossRef]
  29. Solarin, S.A.; Al-Mulali, U. Influence of foreign direct investment on indicators of environmental degradation. Environ. Sci. Pollut. Res. 2018, 25, 24845–24859. [Google Scholar] [CrossRef]
  30. Meyar-Naimi, H.; Vaez-Zadeh, S. Sustainable development based energy policy making frameworks, a critical review. Energy Policy 2012, 43, 351–361. [Google Scholar] [CrossRef]
  31. Longe, O.M.; Ouahada, K. Mitigating household energy poverty through energy expenditure affordability algorithm in a smart grid. Energies 2018, 11, 947. [Google Scholar] [CrossRef] [Green Version]
  32. Bassi, A.M. Evaluating the use of an integrated approach to support energy and climate policy formulation and evaluation. Energies 2010, 3, 1604–1621. [Google Scholar] [CrossRef] [Green Version]
  33. Philippidis, G.; M’barek, R.; Ferrari, E. Is ‘bio-based’ activity a panacea for sustainable competitive growth? Energies 2016, 9, 806. [Google Scholar] [CrossRef] [Green Version]
  34. IUCN; UNEP; WWF. World conservation strategy. Environ. Policy Law 1980, 6, 102. [Google Scholar] [CrossRef]
  35. Hussain, I.; Ullah, M.; Ullah, I.; Bibi, A.; Naeem, M.; Singh, M.; Singh, A.D. Optimizing energy consumption in the home energy management system via a bio-inspired dragonfly algorithm and the genetic algorithm. Electronics 2020, 9, 406. [Google Scholar] [CrossRef]
  36. Stefenon, S.F.; Nied, A. FEM applied to evaluation of the influence of electric field on design of the stator slots in PMSM. IEEE Lat. Am. Trans. 2019, 17, 590–596. [Google Scholar] [CrossRef]
  37. Stefenon, S.F.; Oliveira, J.R.; Coelho, A.S.; Meyer, L.H. Diagnostic of insulators of conventional grid through LabVIEW analysis of FFT signal generated from ultrasound detector. IEEE Latin Am. Trans. 2017, 15, 884–889. [Google Scholar] [CrossRef]
  38. Seghezzo, L. The five dimensions of sustainability. Environ. Politics 2009, 18, 539–556. [Google Scholar] [CrossRef] [Green Version]
  39. Olmos-Gómez, M.C.; Luque Suárez, M.; Ferrara, C.; Olmedo-Moreno, E.M. Quality of higher education through the pursuit of satisfaction with a focus on sustainability. Sustainability 2020, 12, 2366. [Google Scholar] [CrossRef] [Green Version]
  40. Stepanova, N.; Gritsenko, D.; Gavrilyeva, T.; Belokur, A. Sustainable development in sparsely populated territories: Case of the Russian Arctic and far east. Sustainability 2020, 12, 2367. [Google Scholar] [CrossRef] [Green Version]
  41. Yu, S.H.; Liang, W.C. Exploring the determinants of strategic Corporate social responsibility: An empirical examination. Sustainability 2020, 12, 2368. [Google Scholar] [CrossRef] [Green Version]
  42. Ilskog, E. Indicators for assessment of rural electrification—An approach for the comparison of apples and pears. Energy Policy 2008, 36, 2665–2673. [Google Scholar] [CrossRef]
  43. Bhattacharyya, S.C. Energy access programmers and sustainable development: A critical review and analysis. Energy Sustain. Dev. 2012, 16, 260–271. [Google Scholar] [CrossRef]
  44. Iddrisu, I.; Bhattacharyya, S.C. Sustainable energy development index: A multi-dimensional indicator for measuring sustainable energy development. Renew. Sustain. Energy Rev. 2015, 50, 513–530. [Google Scholar] [CrossRef] [Green Version]
  45. Maffia, A.; Pergola, M.; Palese, A.M.; Celano, G. Environmental impact assessment of organic vs. integrated olive-oil systems in Mediterranean context. Agronomy 2020, 10, 416. [Google Scholar] [CrossRef] [Green Version]
  46. Kalinowska, A.; Szopińska, M.; Chmiel, S.; Kończak, M.; Polkowska, Ż.; Artichowicz, W.; Jankowska, K.; Nowak, A.; Łuczkiewicz, A. Heavy metals in a high arctic fiord and their introduction with the wastewater: A case study of adventfjorden-longyearbyen system, svalbard. Water 2020, 12, 794. [Google Scholar] [CrossRef] [Green Version]
  47. Zhao, H.; Wang, Y.; Zhan, Y.; Xu, G.; Cui, X.; Wang, J. Practical model for energy consumption analysis of beam pumping motor systems and its energy-saving applications. IEEE Trans. Ind. Appl. 2018, 54, 1006–1016. [Google Scholar] [CrossRef]
  48. Sundström, N.; Wigström, O.; Lennartson, B. Conflict between energy, stability, and robustness in production schedules. IEEE Trans. Autom. Sci. Eng. 2017, 14, 658–668. [Google Scholar] [CrossRef]
  49. Chen, G.; Zhang, L.; Arinez, J.; Biller, S. Energy-efficient production systems through schedule-based operations. IEEE Trans. Autom. Sci. Eng. 2013, 10, 27–37. [Google Scholar] [CrossRef]
  50. Schau, E.M.; Traverso, M.; Lehmann, A.; Finkbeiner, M. Life cycle costing in sustainability assessment—A case study of remanufactured alternators. Sustainability 2011, 3, 2268–2288. [Google Scholar] [CrossRef] [Green Version]
  51. Américo, J.P.; Cabral, S.H.L.; Stefenon, S.F.; Salvador, M.A.; Oliboni, C.R.P.; Madruga, G.G. Um estudo qualitativo das técnicas utilizadas para medição e localização de descargas parciais em transformadores de potência. ESPAC 2017, 38, 14–23. [Google Scholar]
  52. Galli, F.P.; Stefenon, S.F.; Américo, J.P. Análise de curto-circuitos transitórios em linhas de transmissão utilizando o software UDW. ESPAC 2017, 38, 11–21. [Google Scholar]
  53. Filer, J.E.; Delorit, J.D.; Hoisington, A.J.; Schuldt, S.J. Optimizing the environmental and economic sustainability of remote community infrastructure. Sustainability 2020, 12, 2208. [Google Scholar] [CrossRef] [Green Version]
  54. Gómez-Camacho, C.E.; Ruggeri, B. Energy sustainability analysis (ESA) of energy-producing processes: A case study on distributed H2 production. Sustainability 2019, 11, 4911. [Google Scholar] [CrossRef] [Green Version]
  55. Algunaibet, I.M.; Pozo, C.; Galán-Martín, Á.; Guillén-Gosálbez, G. Quantifying the cost of leaving the Paris agreement via the integration of life cycle assessment, energy systems modeling and monetization. Appl. Energy 2019, 242, 588–601. [Google Scholar] [CrossRef]
  56. Lovins, A.B. Energy efficiency, taxonomic overview. Encycl. Energy 2004, 2, 283–402. [Google Scholar]
  57. Vafadarnikjoo, A.; Ahmadi, H.B.; Hazen, B.T.; Liou, J.J.H. Understanding interdependencies among social sustainability evaluation criteria in an emerging economy. Sustainability 2020, 12, 1934. [Google Scholar] [CrossRef] [Green Version]
  58. Setyowati, A.B. Mitigating energy poverty: Mobilizing climate finance to manage the energy trilemma in indonesia. Sustainability 2020, 12, 1603. [Google Scholar] [CrossRef] [Green Version]
  59. Etukudor, C.; Couraud, B.; Robu, V.; Früh, W.G.; Flynn, D.; Okereke, C. Automated negotiation for peer-to-peer electricity trading in local energy markets. Energies 2020, 13, 920. [Google Scholar] [CrossRef] [Green Version]
  60. Eras-Almeida, A.A.; Fernández, M.; Eisman, J.; Martín, J.G.; Caamaño, E.; Egido-Aguilera, M.A. Lessons learned from rural electrification experiences with third generation solar home systems in Latin America: Case studies in Peru, Mexico, and Bolivia. Sustainability 2019, 11, 7139. [Google Scholar] [CrossRef] [Green Version]
  61. Vera, I.; Langlois, L. Energy indicators for sustainable development. Energy 2007, 32, 875–882. [Google Scholar] [CrossRef]
  62. Abouzeid, A.K.; El-Hag, A.; Assaleh, K. Equivalent salt deposit density prediction of silicone rubber insulators under simulated pollution conditions. Electr. Power Comp. Syst. 2018, 46, 1123–1133. [Google Scholar] [CrossRef]
  63. Polisetty, S.; El-Hag, A.; Jayram, S. Classification of common discharges in outdoor insulation using acoustic signals and artificial neural network. High Volt. 2019, 4, 333–338. [Google Scholar] [CrossRef]
  64. Ibrahim, A.; Dalbah, A.; Abualsaud, A.; Tariq, U.; El-Hag, A. Application of machine learning to evaluate insulator surface erosion. IEEE Trans. Instrum. Meas. 2020, 69, 314–316. [Google Scholar] [CrossRef]
  65. Anjum, S.; Jayaram, S.; El-Hag, A.; Jahromi, A.N. Detection and classification of defects in ceramic insulators using RF antenna. IEEE Trans. Dielectr. Electr. Insul. 2017, 24, 183–190. [Google Scholar] [CrossRef]
  66. Marcelino, L.; Sjöström, J.; Marques, C.A. Socio-problematization of green chemistry: Enriching systems thinking and social sustainability by education. Sustainability 2019, 11, 7123. [Google Scholar] [CrossRef] [Green Version]
  67. Davidescu, A.A.M.; Roman, M.; Strat, V.A.; Mosora, M. Regional sustainability, individual expectations and work motivation: A multilevel analysis. Sustainability 2019, 11, 3331. [Google Scholar] [CrossRef] [Green Version]
  68. Jarosch, L.; Zeug, W.; Bezama, A.; Finkbeiner, M.; Thrän, D. A Regional socio-economic life cycle assessment of a bioeconomy value chain. Sustainability 2020, 12, 1259. [Google Scholar] [CrossRef] [Green Version]
  69. Lorek, S.; Spangenberg, J.H. Sustainable consumption within a sustainable economy—Beyond green growth and green economies. J. Clean. Prod. 2014, 63, 33–44. [Google Scholar] [CrossRef]
  70. Brinkerhoff, D.W.; Goldsmith, A.A. Promoting the sustainability of development institutions: A framework for strategy. World Dev. 1992, 20, 369–383. [Google Scholar] [CrossRef]
  71. Brown, D.R. Evaluating institutional sustainability in development programmers: Beyond dollars and cents. J. Int. Dev. 1998, 10, 55–69. [Google Scholar] [CrossRef]
  72. Poggi, F.; Firmino, A.; Amado, M. Assessing energy performances: A step toward energy efficiency at the municipal level. Sustain. Cities Soc. 2017, 33, 57–69. [Google Scholar] [CrossRef] [Green Version]
  73. Lesse, C.S.; Stefenon, S.F.; Américo, J.P.; Corso, M.P.; Nied, A. Avaliação de falhas em redes de distribuição de energia elétrica através da corrente de curto-circuito. Rev. De Eng. E Tecnol. 2019, 11, 127–134. [Google Scholar]
  74. Ferreira, F.C.S.; Silva, L.R.; Ferreira, R.F.; Benitez, J.C.G.; Stefenon, S.F.; Silva, M.P.; Yamaguchi, C.K. Solar energy microgeneration in Serrana Meso-Region: Project of a grid-connected photovoltaic power system in Lages/SC. Int. J. Innov. Educ. Res. 2018, 6, 206–225. [Google Scholar] [CrossRef]
  75. Garcia, D.S.; Madruga, G.G.; Américo, J.P.; Oliveira, J.R.; Stefenon, S.F. A feasibility study on the application of photovoltaic panels as supplementary power generation for residences in Serra Catarinense. Bus. Manag. Rev. 2017, 5, 8–20. [Google Scholar]
  76. Folster, L.P.; Madruga, G.G.; Ferreira, F.C.S.; Stefenon, S.F. Estudo sobre a eficiência no sistema de iluminação em Salas de AULA (UNIPLAC). ESPAC 2016, 37, 24–35. [Google Scholar]
  77. Godinho, J.M.; Rocca, G.A.D.; Silva, F.C.F.; Stefenon, S.F.; Garcia, A. Estudo da eficiência energética e viabilidade econômica voltado à iluminação do centro de ciências exatas e tecnológicas da UNIPLAC. ESPAC 2017, 42, 6–29. [Google Scholar]
  78. Souza, F.G.; Rocca, G.A.D.; Ferreira, F.C.S.; Stefenon, S.F.; Arruda, P.A. Análise de viabilidade econômica da substituição de lâmpadas comuns por econômicas e tecnologia LED em residências. ESPAC 2017, 38, 17–34. [Google Scholar]
  79. Wang, L.; Watanabe, T. The development of straw-based biomass power generation in rural area in Northeast China—An institutional analysis grounded in a risk management perspective. Sustainability 2020, 12, 1973. [Google Scholar] [CrossRef] [Green Version]
  80. Corso, M.P.; Stefenon, S.F.; Couto, V.F.; Cabral, S.H.L.; Nied, A. Evaluation of methods for electric field calculation in transmission lines. IEEE Lat. Am. Trans. 2018, 16, 2970–2976. [Google Scholar] [CrossRef]
  81. Stefenon, S.F.; Américo, J.P.; Meyer, L.H.; Grebogi, R.B.; Nied, A. Analysis of the electric field in porcelain pin-type insulators via finite elements software. IEEE Lat. Am. Trans. 2018, 16, 2505–2512. [Google Scholar] [CrossRef]
  82. Stefenon, S.F.; Freire, R.Z.; Coelho, L.d.S.; Meyer, L.H.; Grebogi, R.B.; Buratto, W.G.; Nied, A. Electrical insulator fault forecasting based on a wavelet neuro-fuzzy system. Energies 2020, 13, 484. [Google Scholar] [CrossRef] [Green Version]
  83. Stefenon, S.F.; Branco, N.W.; Nied, A.; Bertol, D.W.; Finardi, E.C.; Sartori, A.; Meyer, L.H.; Grebogi, R.B. Analysis of training techniques of ANN for classification of insulators in electrical power systems. IET Gener. Transm. Distrib. 2020, 14, 1591–1597. [Google Scholar] [CrossRef]
  84. Stefenon, S.F.; Grebogi, R.B.; Freire, R.Z.; Nied, A.; Meyer, L.H. Optimized ensemble extreme learning machine for classification of electrical insulators conditions. IEEE Trans. Ind. Electron. 2020, 67, 5170–5178. [Google Scholar] [CrossRef]
  85. Hák, T.; Janoušková, S.; Moldan, B. Sustainable development goals: A need for relevant indicators. Ecol. Indic. 2016, 60, 565–573. [Google Scholar] [CrossRef]
  86. Vera, I.; Langlois, L.; Rogner, H.H.; Jalal, A.I.; Toth, F.L. Indicators for sustainable energy development: An Initiative by the international atomic energy agency. Nat. Resour. Forum 2005, 29, 274–283. [Google Scholar] [CrossRef]
  87. Hatefi, S.M.; Torabi, S.A. A common weight MCDA-DEA approach to construct composite indicators. Ecol. Econ. 2010, 70, 114–120. [Google Scholar] [CrossRef]
  88. Mitchell, G. Problems and fundamentals of sustainable development indicators. Sustain. Dev. 1996, 4, 1–11. [Google Scholar] [CrossRef]
  89. Siche, R.; Agostinho, F.; Ortega, E.; Romeiro, A. Índices versus indicadores: Precisões conceituais na discussão da sustentabilidade de países. Ambiente Soc. 2007, 10, 137–148. [Google Scholar] [CrossRef]
  90. Shields, D.J.; Šolar, S.V.; Martin, W.E. The role of values and objectives in communicating indicators of sustainability. Ecol. Indic. 2002, 2, 149–160. [Google Scholar] [CrossRef]
  91. Patlitzianas, K.D.; Doukas, H.; Kagiannas, A.G.; Psarras, J. Sustainable energy policy indicators: Review and recommendations. Renew. Energy 2008, 33, 966–973. [Google Scholar] [CrossRef]
  92. Asatani, K.; Takeda, H.; Yamano, H.; Sakata, I. Scientific attention to sustainability and SDGs: Meta-analysis of academic papers. Energies 2020, 13, 975. [Google Scholar] [CrossRef] [Green Version]
  93. Narula, K.; Reddy, B.S. Three blind men and an elephant: The case of energy indices to measure energy security and energy sustainability. Energy 2015, 80, 148–158. [Google Scholar] [CrossRef]
  94. Campos, L.M.S.; Heizen, D.A.M.; Verdinelli, M.A.; Cauchick Miguel, P.A. Environmental performance indicators: A study on ISO 14001 certified companies. J. Clean. Prod. 2015, 99, 15–286. [Google Scholar] [CrossRef]
  95. Bebbington, J.; Brown, J.; Frame, B. Accounting technologies and sustainability assessment models. Ecol. Econ. 2007, 61, 224–236. [Google Scholar] [CrossRef]
  96. Singh, R.K.; Murty, H.R.; Gupta, S.K.; Dikshit, A.K. An overview of sustainability assessment methodologies. Ecol. Indic. 2012, 15, 281–299. [Google Scholar] [CrossRef]
  97. Mayer, F.D.; Brondani, M.; Vasquez Carrillo, M.C.; Hoffmann, R.; Silva Lora, E.E. Revisiting energy efficiency, renewability, and sustainability indicators in biofuels life cycle: Analysis and standardization proposal. J. Clean. Prod. 2020, 252, 119850. [Google Scholar] [CrossRef]
  98. Van Schoubroeck, S.; Van Dael, M.; Van Passel, S.; Malina, R. A review of sustainability indicators for biobased chemicals. Ren. Sustain. Energy Rev. 2018, 94, 115–126. [Google Scholar] [CrossRef]
  99. Gan, X.; Fernandez, I.C.; Guo, J.; Wilson, M.; Zhao, Y.; Zhou, B.; Wu, J. When to use what: Methods for weighting and aggregating sustainability indicators. Ecol. Indic. 2017, 81, 491–502. [Google Scholar] [CrossRef]
  100. Rösch, C.; Bräutigam, K.R.; Kopfmüller, J.; Stelzer, V.; Lichtner, P. Indicator system for the sustainability assessment of the German energy system and its transition. Energ. Sustain. Soc. 2017, 7, 1–13. [Google Scholar] [CrossRef] [Green Version]
  101. Kanchiralla, F.M.; Jalo, N.; Johnsson, S.; Thollander, P.; Andersson, M. Energy end—Use categorization and performance indicators for energy management in the engineering industry. Energies 2020, 13, 369. [Google Scholar] [CrossRef] [Green Version]
  102. Lu, J.; Ren, L.; Yao, S.; Qiao, J.; Strielkowski, W.; Streimikis, J. Comparative review of corporate social responsibility of energy utilities and sustainable energy development trends in the baltic states. Energies 2019, 12, 3417. [Google Scholar] [CrossRef] [Green Version]
  103. Gomaa, M.R.; Rezk, H.; Mustafa, R.J.; Al-Dhaifallah, M. Evaluating the environmental impacts and energy performance of a wind farm system utilizing the life-cycle assessment method: A practical case study. Energies 2019, 12, 3263. [Google Scholar] [CrossRef] [Green Version]
  104. Suding, P.H.; La Vega, F.F.; Altomonte, H.; Pistonesi, H. Energy and Sustainable Development in Latin America and the Caribbean: Approaches to Energy Policy; OLADE, ECLAC, GTZ: Quito, Ecuador, 1997. [Google Scholar]
  105. Sheinbaum-Pardo, C.; Ruiz-Mendoza, B.J.; Rodríguez-Padilla, V. Mexican energy policy and sustainability indicators. Energy Policy 2012, 46, 278–283. [Google Scholar] [CrossRef]
  106. Pereira, A.O.; Soares, J.B.; de Oliveira, R.G.; de Queiroz, R.P. Energy in Brazil: Toward sustainable development? Energy Policy 2008, 36, 73–83. [Google Scholar] [CrossRef]
  107. Streimikiene, D.; Ciegis, R.; Grundey, D. Energy indicators for sustainable development in Baltic States. Renew. Sustain. Energy Rev. 2007, 11, 877–893. [Google Scholar] [CrossRef]
  108. Schaeffer, R.; Szklo, A.S.; Cima, F.M.; Machado, G. Indicators for sustainable energy development: Brazil’s case study. Nat. Resour. Forum 2005, 29, 284–297. [Google Scholar] [CrossRef]
  109. Mandelli, S.; Barbieri, J.; Mattarolo, L.; Colombo, E. Sustainable energy in Africa: A comprehensive data and policies review. Renew. Sustain. Energy Rev. 2014, 37, 656–686. [Google Scholar] [CrossRef]
  110. Arens, S.; Schlüters, S.; Hanke, B.; von Maydell, K.; Agert, C. Sustainable residential energy supply: A literature review—Based morphological analysis. Energies 2020, 13, 432. [Google Scholar] [CrossRef] [Green Version]
  111. Wang, B.; Li, H.-N.; Yuan, X.-C.; Sun, Z.-M. Energy poverty in China: A dynamic analysis based on a hybrid panel data decision model. Energies 2017, 10, 1942. [Google Scholar] [CrossRef] [Green Version]
  112. Nussbaumer, P.; Bazilian, M.; Modi, V. Measuring energy poverty: Focusing on what matters. Renew. Sustain. Energy Rev. 2012, 16, 231–243. [Google Scholar] [CrossRef] [Green Version]
  113. Alkire, S.; Foster, J. Counting and Multidimensional Poverty Measurement; OPHI Working Paper No. 32; Oxford Poverty and Human Development Initiative (OPHI): Oxford, UK, 2008. [Google Scholar]
  114. Alkire, S.; Santos, M.E. Acute Multidimensional Poverty: A New Index for Developing Countries; OPHI Working Paper No. 38; Oxford Poverty and Human Development Initiative (OPHI): Oxford, UK, 2010. [Google Scholar]
  115. Acharya, R.H.; Sadath, A.C. Energy poverty and economic development: Household-level evidence from India. Energy Build. 2018, 183, 785–791. [Google Scholar] [CrossRef]
  116. Pablo, Q.S.; Paloma, T.L.P.; Francisco, J.T. Energy poverty in Ecuador. Sustainability 2019, 11, 6320. [Google Scholar] [CrossRef] [Green Version]
  117. Nussbaumer, P.; Nerini, F.F.; Onyeji, I.; Howells, M. Global insights based on the multidimensional energy poverty index (MEPI). Sustainability 2013, 5, 2060–2076. [Google Scholar] [CrossRef] [Green Version]
  118. Meyer, S.; Laurence, H.; Bart, D.; Middlemiss, L.; Maréchal, K. Capturing the multifaceted nature of energy poverty: Lessons from Belgium. Energy Res. Soc. Sci. 2018, 40, 273–283. [Google Scholar] [CrossRef]
  119. Kilkiş, Ş. Composite index for benchmarking local energy systems of Mediterranean port cities. Energy 2015, 92, 622–638. [Google Scholar] [CrossRef]
  120. Stefenon, S.F.; Silva, M.C.; Bertol, D.W.; Meyer, L.H.; Nied, A. Fault diagnosis of insulators from ultrasound detection using neural networks. J. Intell. Fuzzy Syst. 2019, 37, 6655–6664. [Google Scholar] [CrossRef]
  121. Kilkiş, Ş. Sustainable development of energy, water and environment systems index for Southeast European cities. J. Clean. Prod. 2016, 130, 222–234. [Google Scholar] [CrossRef]
  122. Kilkiş, Ş. Application of the sustainable development of energy, water and environment systems index to world cities with a normative scenario for Rio de Janeiro. J. Sustain. Dev. Energy Water Environ. Syst. 2018, 6, 559–608. [Google Scholar] [CrossRef]
  123. Kilkiş, Ş. Benchmarking South East European cities with the sustainable development of energy, water and environment systems index. J.Sustain. Dev. Energy Water Environ. Syst. 2018, 6, 162–209. [Google Scholar] [CrossRef]
  124. Marquez Ballesteros, M.J.; Mora Lopez, L.; Lloret Gallego, P.; Sumper, A.; Sidrach de Cardona, M. Measuring urban energy sustainability and its application to two spanish cities: Malaga and Barcelona. Sustain. Cities Soc. 2019, 45, 335–347. [Google Scholar] [CrossRef]
  125. Kumar, P.; Johnson, B.A.; Dasgupta, R.; Avtar, R.; Chakraborty, S.; Kawai, M.; Magcale-Macandog, D.B. Participatory approach for more robust water resource management: Case study of the Santa Rosa sub-watershed of the Philippines. Water 2020, 12, 1172. [Google Scholar] [CrossRef] [Green Version]
  126. Ariyawansha, T.; Abeyrathna, D.; Kulasekara, B.; Pottawela, D.; Kodithuwakku, D.; Ariyawansha, S.; Sewwandi, N.; Bandara, W.; Ahamed, T.; Noguchi, R. A novel approach to minimize energy requirements and maximize biomass utilization of the sugarcane harvesting system in Sri Lanka. Energies 2020, 13, 1497. [Google Scholar] [CrossRef] [Green Version]
  127. Chantasiriwan, S. Increased energy efficiency of a backward-feed multiple-effect evaporator compared with a forward-feed multiple-effect evaporator in the cogeneration system of a sugar factory. Processes 2020, 8, 342. [Google Scholar] [CrossRef] [Green Version]
  128. Steinle, S.; Zimmerlin, M.; Mueller, F.; Held, L.; Suriyah, M.R.; Leibfried, T. Time-dependent flexibility potential of heat pump systems for smart energy system operation. Energies 2020, 13, 903. [Google Scholar] [CrossRef] [Green Version]
  129. Dui, N.; Urbaniec, K.; Huisingh, D. Components and structures of the pillars of sustainability. J. Clean. Prod. 2015, 88, 1–12. [Google Scholar] [CrossRef]
  130. The World Bank. Readiness for Investment in Sustainable Energy. A Tool for Policymakers; World Bank Group: Whashington, DC, USA, 2014. [Google Scholar]
  131. Banerjee, S.G.; Moreno, A.; Sinton, J.; Primiani, A.; Seong, J. Regulatory Indicators for Sustainable Energy. A Global Scorecard for Policy Makers; World Bank Group, ESMAP, Climate Investment Funds, Sustainable Energy for All: Washington, DC, USA, 2016. [Google Scholar]
  132. ESMAP. Regulatory Indicators for Sustainable Energy; World Bank Group: Washington, DC, USA, 2018. [Google Scholar]
  133. Modi, V.; McDade, S.; Lallement, D.; Saghir, J. Energy and the Millennium Development Goals; United Nations Development Programme and World Bank: New York, NY, USA, 2006. [Google Scholar]
  134. García-Álvarez, M.T.; Moreno, B.; Soares, I. Analyzing the sustainable energy development in the EU-15 by an aggregated synthetic index. Ecol. Indic. 2016, 60, 996–1007. [Google Scholar] [CrossRef]
  135. Pardo, J.E.; Mejías, A.; Sartal, A. Assessing the importance of biomass-based heating systems for more sustainable buildings: A case study based in Spain. Energies 2020, 13, 1025. [Google Scholar] [CrossRef] [Green Version]
  136. Kasburg, C.; Stefenon, S.F. Deep learning for photovoltaic generation forecast in active solar trackers. IEEE Lat. Am. Trans. 2019, 17, 2013–2019. [Google Scholar] [CrossRef]
  137. Van Bellen, H.M. Indicadores de sustentabilidade: Um levantamento dos principais sistemas de avaliação. Cadernos eBAPe Br 2004, 2, 1–14. [Google Scholar] [CrossRef]
  138. Stefenon, S.F.; Steinheuser, D.F.; Klaar, A.C.R.; de Souza, K.E.; Júnior, A.G.; Venção, A.T. Application of active methodologies in engineering education through the integrative evaluation at the universidade do planalto catarinense, Brazil. Interciencia 2019, 44, 408–413. [Google Scholar]
  139. Arruda, M.P.; Lima, L.C.; Arruda, R.; Stefenon, S.F.; Klaar, A.C.R. Metodologias ativas para promover autonomia: Reflexões de professores do ensino superior. ESPAC 2017, 38, 1–13. [Google Scholar]
  140. Mougouei, F.R.; Mortazavi, M.S. Effective approaches to energy planning and classification of energy systems models. Int. J. Energy Econ. Policy 2017, 7, 127–131. [Google Scholar]
  141. Nakata, T. Energy-economic models and the environment. Prog. Energy Combust. Sci. 2004, 30, 417–475. [Google Scholar] [CrossRef]
  142. Tomczak, S.K. Comparison of the financial standing of companies generating electricity from renewable sources and fossil fuels: A new hybrid approach. Energies 2019, 12, 3856. [Google Scholar] [CrossRef] [Green Version]
  143. Ruano, M.A.; Cruzado, M.G. Use of education as social indicator in the assessment of sustainability throughout the life cycle of a building. Eur. J. Eng. Educ. 2012, 37, 416–425. [Google Scholar] [CrossRef]
  144. Poveda, C.A. The theory of dimensional balance of needs. Int. J. Sustain. Dev. World Ecol. 2017, 24, 97–119. [Google Scholar] [CrossRef]
  145. Barnes, D.F.; Singh, B.; Shi, X. Modernizing Energy Services for the Poor: A World Bank Investment Review; World Bank Group: Washington, DC, USA, 2010. [Google Scholar]
  146. Allen, C.; Smit, I.; Wallach, W. Artificial morality: Top-down, bottom-up, and hybrid approaches. Ethics Inf. Technol. 2005, 7, 149–155. [Google Scholar] [CrossRef]
  147. Ghersi, F. Hybrid bottom-up/top-down energy and economy outlooks: A review of IMACLIM-S experiments. Front. Environ. Sci. 2015, 3, 74. [Google Scholar] [CrossRef] [Green Version]
  148. Khadka, C.; Vacik, H. Comparing a top-down and bottom-up approach in the identification of criteria and indicators for sustainable community forest management in Nepal. For. Int. J. For. Res. 2012, 85, 145–158. [Google Scholar] [CrossRef] [Green Version]
Figure 1. Information pyramid [88,89].
Figure 1. Information pyramid [88,89].
Energies 13 02366 g001
Figure 2. Methodological structure.
Figure 2. Methodological structure.
Energies 13 02366 g002
Table 1. Dimensions and meanings of ESI indicators.
Table 1. Dimensions and meanings of ESI indicators.
EconomicEnergy autarkyOwn energy resources ensure energy supply over a long period and are measured by the degree of dependence on imports from one country
Soundness in the face of external changesMeasures a country’s economic vulnerability due to changes in international market conditions
Energy productivityThis indicator is the inverse of the energy intensity and measures the output per unit of energy
EnvironmentalUse of renewablesMeasures the share of renewable energy in the energy matrix
Scope of fossil resources and firewoodMeasures the depletion of fossil fuels and firewood, considering the reserve-production ratio
The relative purity of energy useMeasures carbon dioxide emissions from energy, with low emission levels indicating high sustainability
SocialCoverage of basics energy needsIt captures whether families use enough amounts of useful energy
Electric power coverageA high percent of electrified households
Table 2. Composition of Energy Development Index (EDI).
Table 2. Composition of Energy Development Index (EDI).
EDIResidentialShare of the population with access to electricity
Modern fuel share of the residential sector
CommunitarianPer capita consumption of public sector electricity
Share of productive use of energy in total final consumption
Table 3. Dimensions and indicators of Regulatory Indicators for Sustainable Energy (RISE).
Table 3. Dimensions and indicators of Regulatory Indicators for Sustainable Energy (RISE).
DimensionIndicators-Policies and Regulations
Electricity AccessExistence and implementation of electrification planGrid electrificationAffordability of electricityUtility creditworthiness
Scope of electrification planMini-gridsUtility transparency and monitoringStandalone systems
Clean CookingPlanningScope of planningStandards and labelingIncentives and attributes
Energy EfficiencyNational energy efficiency planningMandates and incentives: utilitiesMinimum energy performance standardsTransport energy efficiency
Energy efficiency entitiesMandates and incentives: large consumersEnergy labeling systemFinancing mechanisms for energy efficiency
Information provided to electricity consumersMandates and incentives: public entitiesBuilding energy codesCarbon pricing and monitoring
Types of electricity rate structures
Renewable EnergyLegal framework for renewable energyIncentives and regulatory support for renewable energyNetwork connection and useCounterparty risk
Planning for renewable energy expansionAttributes of financial and regulatory incentivesCarbon pricing and monitoring
Table 4. Dimensions, categories, and variables of the SISED index [134].
Table 4. Dimensions, categories, and variables of the SISED index [134].
Security of Energy Supply (SES) EvaluationEnergy Dependence Import of fuels, oil, or natural gas
Differentiation of the type of primary energy productionRenewable energies, gas, petroleum, nuclear energy
Energy consumptionGas consumption, consumption of petroleum products, electricity use, consumption of solid fuels
Competitive Energy Market (CEM)Energy EfficiencyEnergy used with regard to losses
Power marketProductivity and resource management
Competitive prices of energyPrice of electricity, gas, and fossil fuelsEnergy storage for resource managementLarge energy consumersEnergy planningInvestment in new energy sources
Environmental Protection (EP)Renewable Energy SourcesShare of Renewable Energy Sources in electrical energy production, primary energy production, and final energy consumption
Emitted CO2Energy intensity of total Greenhouse Gas (GHG) emissions per capita Share of energy industry emissions and on total GHG emissions
Taxes applied to energyThe tax rate applied to energy
Table 5. Summary table of revised energy sustainability assessment tools.
Table 5. Summary table of revised energy sustainability assessment tools.
ToolAcronymGroupingInitiatorYear CreationReference
Energy Sustainability IndicatorsESIGroup IndicatorsOLADE, ECLAC, GTZ1997[104]
Energy Indicators for Sustainable DevelopmentEISDGroup IndicatorsIAEA1999[61,86,107]
Energy Development IndexEDIComposite IndexIEA, OECD2004[111]
Multidimensional Energy Poverty IndexMEPIComposite IndexOPHI2012[112]
Sustainable Energy Development IndexSEDIComposite IndexResearchers2012[44]
Regulatory Indicators for Sustainable EnergyRISEGroup IndicatorsWorld Bank2014[130,131,132]
Sustainable Development of Energy, Water, and Environment SystemsSDEWESComposite IndexSDEWES Centre2015[119,123]
Synthetic Index of Sustainable Energy DevelopmentSISEDComposite IndexResearchers2016[134]
Table 6. The comparative framework according to the scope criterion.
Table 6. The comparative framework according to the scope criterion.
DimensionDimension Name
ESI3Economic, environmental, social
EISD3Economic, environmental, social
EDI2Residential, community
MEPI5Cooking, lighting, household appliances, entertainment/education, communication
SEDI5Economic, environmental, social, institutional, technical
RISE4Electricity access, clean cooking, energy efficiency, renewable energy
SDEWES7Energy consumption and climate, penetration of energy and CO2 saving measures, renewable energy potential and utilization, water and environmental quality, CO2 emissions and industrial profile, city planning and social welfare, R&D, innovation and sustainability policy
SISED3Security of energy supply, competitive energy market, environmental protection
Table 7. The comparative framework according to the sphere criterion.
Table 7. The comparative framework according to the sphere criterion.
Table 8. The comparative framework according to the participation criterion.
Table 8. The comparative framework according to the participation criterion.
ESItop-downExperts determine the final index without interference from social actors.
EISDtop-downExperts determine the final index without interference from social actors.
EDItop-downExperts determine the final index without interference from social actors.
MEPIhybridThe final index is determined by specialists with the participation of social actors.
SEDItop-downExperts determine the final index without interference from social actors.
RISEhybridSocial actors participate in the selection of indicators but do not have access to their weights, being restricted to specialists.
SDEWEShybridExperts determine the indicators and final index, but social actors can interact with dynamic tables and are encouraged to use the results of the index to assess the performance of a specific city.
SISEDtop-downExperts determine the final index without interference from social actors.
Table 9. The comparative table according to data criterion.
Table 9. The comparative table according to data criterion.
ToolTypology of DataData Aggregation
IndexSub-indicesIndicatorsSub-IndicatorsAggregate DataPrimary Data
Table 10. The comparative table according to the interface criterion, with its four subdivisions.
Table 10. The comparative table according to the interface criterion, with its four subdivisions.
ComplexityPresentationOpennessEducational Potential
ESILow—is a set of group indicators, with simple independent calculations and linear normalization.It does not have a general numerical index because it is a set of group indicators. The visual presentation depends on each specialist, varying in comparative charts of bar, dispersion, or radar.High—displays the data used with its respective sources. It also shows the weights and normalization methods of each indicator used.It has its educational value associated with historical value, as it is a pioneering tool in energy-based sustainability analysis. It has no interaction interface with the end-user.
EISDMedium—comprises a set of indicators, where each is calculated independently through its methods, ranging from a simple average to a decomposition method.Since it is a set of group indicators, it does not have a general numeric index. Each application of the methodology allows obtaining area, line, bar, and dispersion graphs for visual presentation of the results.High—with the primary data and aggregates referenced as well as the method chosen to calculate each of the indicators.It is a tool geared to facilitate the elaboration of energy policies geared toward sustainability. It does not have an interactive interface with the user, and is used only by specialists.
EDILow—elaborated through four indicators, where each is expressed by the simple arithmetic average, with aggregation index varying between 0 and 1. Follows the logic used to obtain HDI.It has a general numerical index that compares countries through the bar and scatter charts. It also compares the general index obtained with HDI.Medium—it presents the primary and aggregate data, but it does not make clear the value of the weights of the indicators used to calculate the final index.It has high pedagogical value because it is an approach similar to HDI where it relates to energy. It has no interface for end-user but has several richly illustrated reports.
MEPIHigh—due to the weights being attributed to qualities beyond quantities, probabilistic functions are defined to calculate the indicators that compose the final index, as well as analysis of uncertainties through the Monte Carlo method.It has a general numerical index. It uses maps, tables, bar charts, scatters plots, and probabilistic density functions for visual presentation of the results.High openness—it shows the primary data, aggregates, and details the weights of the indicators used to calculate the final index.High educational potential, being a tool of analysis differentiated from the others because its methodology is based on the concept of energy poverty and its intensity.
SEDILow—The general index is obtained by the simple average of five indicators referring to the dimensions worked. The aggregation technique follows the same logic used in EDI and MEPI.Compares the general numerical index obtained with other tools and uses tables, bar, radar, and scatter charts to show the results.High—providing information about the required data, data source, and the weight of each indicator used to calculate the final index.High educational potential, with the possibility of comparing countries from different regions in a simple way. It does not allow user interaction with the tool.
RISELow—The score of the indicators is weightless and simple arithmetic averages. It allows the user to adjust the most relevant weights of each indicator through the website of the project.It does not use a general numeric index, but rather a set of indicators. The visual part is well elaborated and shown in graphics (pizza, bars, radar, scatter) with a very interactive website.A high degree of openness. Displays data sources and allows user interaction in the form of an online tool with access to data, spreadsheets, and graphics.High educational potential with a strong degree of user interaction with the tool. It also produces illustrated reports available in an open form on the website.
SDEWESHigh—Five stages lead to the construction of the index: selection, data collection, normalization by the Min-Max method, and aggregation of data.It uses a general numerical index to rank with a visual presentation in the form of bar, radar, and spatial mapping charts. It has a dynamic website with a great presentation of the data and results achieved.High openness with a detailed presentation of the input data of each indicator used and their respective weights.An innovative tool with high educational potential for sustainable urban planning. Contributed to the formulation of public policies.
SISEDMedium—The final synthetic index is obtained by the arithmetic average of three indices, which are calculated by the Z-score method of each one.It uses a general numerical index and three intermediate indices with a visual presentation in bar and radar charts (polar).A high degree of openness. The tool shows the data used, its sources, and respective weights involved in the calculations of the indices.High educational potential because it works directly with comparative results aimed at the elaboration of public policies.

Share and Cite

MDPI and ACS Style

Ninno Muniz, R.; Frizzo Stefenon, S.; Gouvêa Buratto, W.; Nied, A.; Meyer, L.H.; Finardi, E.C.; Marino Kühl, R.; Silva de Sá, J.A.; Ramati Pereira da Rocha, B. Tools for Measuring Energy Sustainability: A Comparative Review. Energies 2020, 13, 2366.

AMA Style

Ninno Muniz R, Frizzo Stefenon S, Gouvêa Buratto W, Nied A, Meyer LH, Finardi EC, Marino Kühl R, Silva de Sá JA, Ramati Pereira da Rocha B. Tools for Measuring Energy Sustainability: A Comparative Review. Energies. 2020; 13(9):2366.

Chicago/Turabian Style

Ninno Muniz, Rafael, Stéfano Frizzo Stefenon, William Gouvêa Buratto, Ademir Nied, Luiz Henrique Meyer, Erlon Cristian Finardi, Ricardo Marino Kühl, José Alberto Silva de Sá, and Brigida Ramati Pereira da Rocha. 2020. "Tools for Measuring Energy Sustainability: A Comparative Review" Energies 13, no. 9: 2366.

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop