Nationwide Evaluation of Urban Energy System Resilience in China Using a Comprehensive Index Method

Abstract

The carbon peak and carbon neutrality goals for China signify a critical time of energy transition in which energy resilience is a vital issue. Therefore, a comprehensive evaluation of urban energy system resilience (UESR) is important for establishing a theoretical foundation. To this end, in this paper, 309 Chinese cities were evaluated using a comprehensive UESR assessment framework composed of 113 indices that measured vulnerability and capabilities of resistance and restoration. The results showed that China’s UESR is distributed unevenly and that cities in the eastern region generally have higher resilience than those in other regions. The minimum and maximum UESR results corresponded to Tibet and Shandong, respectively, at the provincial level and Rikaze and Weifang, respectively, at the city level. Regression analysis showed a positive correlation among UESR, carbon dioxide emissions, and GDP.

1. Introduction

On 22 September 2020, President Xi Jinping announced that China would adopt more forceful policies and measures to reach the peak of carbon dioxide emissions by 2030 and to achieve carbon neutrality by 2060; these goals are referred to as the 3060 targets [1]. Energy structure transformation is key to achieving the 3060 targets. The main approaches include reducing the proportion and total amount of fossil fuel consumption, developing renewable energy, reforming the power system, and developing clean and green industries. These approaches assist in building resilient energy systems, as energy system resilience refers to the ability to maintain the essential functions and services of the energy system, ensure stable energy supply and demand with controllable fluctuations, and quickly adapt to new conditions when disruption occurs. Therefore, the 3060 targets, which involve all aspects of energy production, transmission, distribution, consumption, and storage, provide an important opportunity to enhance energy system resilience.

Cities are the macroscopic consumption unit of national energy systems and are responsible for 70% of global greenhouse gas emissions; thus, they should play an important role in this energy transition [2]. When cities meet various urban energy demands related to citizens’ daily lives and provide other infrastructures with enabling functions, a plethora of threats with natural, technical, or human causes might jeopardize the security of their energy systems, leading people to realize that urban energy system resilience (UESR) is becoming increasingly important in the process of urban development [3,4,5].

Billions of dollars in resilience investment are being mobilized globally, creating demand for a rigorous and decision-oriented resilience measurement [6]. However, the evaluation of UESR has not received much attention or research despite its importance. On the one hand, current research on the evaluation of urban resilience has mainly addressed disturbances due to climate change and natural disasters on cities [7,8], while UESR has been rarely studied. As a means of evaluation, the comprehensive index method has been applied to evaluate resilience at the community [6,7,8,9], region [10], city [11,12,13], and country [14,15] levels. For example, resilient city research for China has proposed a set of indicators such as networks and transportation [9,10]. However, the energy sector is usually not considered the major focus of urban resilience [9,10,11,12,13]. On the other hand, though energy system resilience has been defined by many researchers [14,15,16,17,18,19,20], and the quantification thereof is an important branch of energy system resilience research, there is still no consensus on a suitable and comparable evaluation methodology, and the mainstream quantitative methods have limitations of broad applicability and comparability for various cities. Apart from comprehensive index methods, [21] divided the evaluation methods into two categories: quantitative and qualitative. The quantitative methods are mainly time-dependent matric methods and consider resilience to be capacities of resistance, absorption, and restoration [22,23,24]. The metrics assess the system performance, which is ad hoc, i.e., system- or event-specific and backed by historical data [25,26,27,28]. The complexity and computability of the models and the requirement for historical data limit the broad applicability and comparability of these methods, especially across hundreds of cities. Besides, very few such qualitative methods have been applied to study at the city level. Though a dynamic energy balance-based model has been proposed to measure UESR, this methodology also requires input data and cannot sufficiently providing resilience enhancement strategies at the regional and national levels [29]. Qualitative methods have been less studied; these mainly include checklists and questionnaires [30], the matrix scoring system [31], and the analytic hierarchy process [32]. Case studies to verify feasibility are few as well. In summary, a broadly applicable and comparable quantitative method for evaluating energy system resilience of various cities has not hitherto existed.

To fill this knowledge gap, in this paper, a comprehensive index method is proposed to semi-quantitatively evaluate baseline UESR, which involves the capacities of resistance and restoration combined with vulnerability assessment. To do so, the system boundary of the urban energy system was clarified and UESR was defined; based on the definition, the capacities of resistance and restoration were qualitatively evaluated by three dimensions, namely the multifarious capabilities of the energy system within a city (CE), the interdependencies between other basic city subsystems and the energy system (CI), and the comprehensive vulnerabilities of cities and energy (CV); and these three dimensions were quantitively evaluated by 113 indices, which were selected through a relatively thorough literature review under a set of selection principles. The applicability and comparability of the comprehensive index method are demonstrated through case studies of 309 cities in China.

2. Materials and Methods

The resilience discussion herein is proposed to be constrained to high-impact rare events (HR events), also called black swan events [4,33]. The system boundary is constrained on the city level, which represents an adequate unit for policy implementation and is convenient for the overall management of practical events in terms of China’s existing realities.

2.1. Characterization of Urban Energy System (UES)

The system boundary for an UES can be clarified, as in the working paper of the cross-center UKERC Energy 2050 project [17]. The energy resources, energy carriers, energy technologies, energy infrastructures (physical and virtual), and surrounding supporting facilities in a city are collectively referred to as the UES. Energy resources include fuels, such as coal, charcoal, gasoline, diesel, natural gas, biogas, uranium, and hydrogen, and natural energy sources, such as hydropower, geothermal power, solar power, and wind power. Energy carriers work in terms of electricity, heat, and cold in addition to fuels. Energy technologies are related to centralized power plants, distributed energy systems, and (micro)grids. Supporting facilities incorporate monitoring and protection devices, electric energy storage supporting equipment, etc. Generally, the UES can also be traced through the energy flow through production, transmission, distribution, conversion, consumption, and storage within a city’s physical boundaries, while part of production, i.e., exploration, exploitation, transportation, and processing, usually occurs outside the UES.

2.2. Definition of UESR

In accordance with the essence of the definitions, UESR can be defined as the ability of a UES to resist HR events’ impacts, so as to maintain essential functions and services and ensure energy supply and demand within controllable fluctuations, and to quickly restore full energy production. With higher UESR, a UES has a greater capacity to handle foreseeable and/or unforeseeable impacts. From the time dimension, UESR requires the UES to reduce the probability of risk occurrence through measures of risk mitigation in the pre-event stage; diminish the direct and indirect impacts and shorten the duration when an HR event occurs; and withstand various sequential impacts, accommodate and recover from degradation, adapt to new conditions, and learn lessons for future mitigation strategies in the post event stage. In short, for UESs, resilience signifies the capacities of resistance and restoration.

When an HR event occurs, higher resistance helps the UES suffer less performance decline, and higher restoration helps the UES undergo quicker adaptation to new conditions, as shown in Figure 1. The height of the blue-shaded triangle is negatively related to resistance capacity, representing the decrease in system performance. The base of the blue-shaded triangle is negatively related to restoration capacity, representing the restoration of the system performance. As the reverse of the blue-shaded area depicts the simplified resilience level, resilience can be determined as follows:

$$Resilience=Resistance\times Restoration$$

(1)

To evaluate the capacities of resistance and restoration, three dimensions are proposed: CE, CI, and CV. CE refers to the comprehensive quality of UESs, including robustness, diversity, flexibility, and availability: (1) robustness refers to the condition of hardware and its ability to resist external impacts to reduce the physical influence of disasters and prevent widespread grid outages and energy supply failures. Hardware refers to grid lines, transformers, energy practitioners, and power generation capacity in this framework. Energy reserves of various fuels play an important role in energy feedstock cutoff. Technological and financial feasibilities should also be considered, e.g., improving energy supply stability and enriching the fuel stock. (2) Diversity consists of energy generation and consumption as well as enterprise productive capacity. To evaluate energy diversity, the Shannon–Weaver index is applied, since it is widely preferred for variety and balance [34]. The Shannon–Weaver index is defined as [35,36]:

$$D=-{\displaystyle \sum }_i{p}_{i}ln\left(p_i\right)$$

(2)

where $p_i$ represents the share of energy source $i$ in the mix of energy generation/consumption for an energy system. The higher the value of $D$ is, the more diverse a system is evaluated to be. (3) Flexibility is based primarily on the view of the UES as a complex and flexible integrated system that includes organizational, technical, and administrative factors. The system should have the ability to take precautions, study disaster prediction, and obtain the latest information before an event so that rational planning and allocation can be performed in advance in terms of equipment, technology, organization, personnel, resources, and capital. This quality enables the system to flexibly adapt to new internal and external conditions and find a new stable state when an HR event is about to end or after a long period of time following the event. Thus, many aspects at the system-management level are inspected. Evaluation of practice includes demonstration projects, energy savings, and equipment decommission. (4) Availability refers to the ability to adjust the system based on resource availability and financial feasibility. Resource exploitation and processing are considered for coal, petroleum, and other fuels. Financial feasibility is evaluated in terms of the fixed and current assets of energy industries.

CI involves basic city subsystems that closely interact with the energy system. The interdependencies between critical infrastructures should be taken into consideration since a powerful countermeasure of energy sector that does not explore potential synergies between other pertinent sectors may exacerbate the vulnerability or reduce the overall UESR [37,38,39]. Thus, CI refers to the capability of a city to cope with hazardous events, including interdependencies between UESs and other societal sectors, such as water, transportation, ecology, emergency services, medical services, and information and telecommunications [40,41]. Water systems are critical in an emergency, and they interact with energy systems via water flow, sewage discharge, cooling water, and circulating water. The transportation system is powered mainly by gasoline, diesel, natural gas and electricity; moreover, the accessibility of the transportation system plays a key role in emergency situations. Ecological systems can provide effective buffering, such as vegetation management and green open space [42]. Emergency services, medical services, and information and telecommunications are high priorities for energy supply and are essential for urban system restoration [43,44].

CV refers to the number of objects with regard to the basic urban conditions in the city and the energy infrastructures in the energy system, that could possibly be affected by hazard [45,46,47]. City vulnerability takes demographic, economic, and architectural factors into consideration. Energy vulnerability is associated mainly with pipeline and gas stations of various fuels. District heat and electricity consumption have direct impacts on urban residents’ daily lives when HR events occur.

According to the above, the greater the CE or CI, the faster the system performance is restored; the greater the CE or the smaller the CV, the less the system performance decreases. The evaluation of resilience, i.e., the UES’s capacities of resistance and restoration, is converted into the evaluation of CE, CI, and CV as shown in Figure 1 [48].

2.3. Index Selection

Comprehensive index methods have become a standard approach to simplifying governmental and organizational policy making, decision making, performance appraisal, and progress tracking at all levels [48]. This study proposes a comprehensive index method, providing each dimension with a series of indices for evaluation. In the early stage of developing the comprehensive index framework, a large number of proposed indices by other researchers and database were collected based on a literature review and data research. The index selection procedure is depicted in Figure 2. To organize a consistent UESR framework, indices must first suit the scope of UES. To this end, hundreds of primary indices were obtained. These primary indices were then classified according to the meaning and category into three dimensions: CE, CI, and CV. Each index was described in accordance with the referred literature as closely as possible. Following that, a set of selection principles was examined to evaluate the index’s systematism, unicity, feasibility, objectivity, and representation. To describe the overall dimension, the index set should systematically reflect every subsystem and be neither too detailed nor too general [49]. Unicity means that repeated indices should be removed. Feasibility refers to the availability of data from reliable sources with no obvious errors and the operability of quantitative methods and statistical approaches. To be objective, indices should conform to objective facts and not be interfered with subjective values. Representation means that limited indices should describe a dimension as comprehensively as possible. Indices that met the five selection principles were retained, and those that did not meet any principle were deleted. Detailed primary index selection records are shown in

Table A1. Aggregated index selection for CE (note: ✓ indicates compliance with the selection principle and ✗ indicates noncompliance; selection principles: systematism (S), unicity (U), feasibility (F), objectivity (O), and representation (R)).

No.	Primary Index	Ref.	S	U	F	O	R	Result
1	Energy feedstock	[52]	✓	✓	✗	✓	✓	Deleted
2	Energy not supplied	[53]	✓	✓	✗	✗	✓	Deleted
3	Energy storage	[54]	✓	✓	✓	✓	✓	Retained
4	Hydrophobic coating on equipment	[55]	✗	✓	✗	✓	✗	Deleted
5	Key replacement equipment stockpile	[55]	✗	✓	✗	✓	✗	Deleted
6	Redundant power lines	[55]	✓	✓	✗	✓	✗	Deleted
7	Reinforced concrete versus wooden distribution poles	[55]	✗	✓	✗	✗	✓	Deleted
8	Siting infrastructure	[55]	✗	✓	✗	✓	✗	Deleted
9	Underground, overhead, undersea distribution/cable lines	[56,57]	✓	✓	✗	✓	✓	Deleted
10	Unique encrypted passwords for utility “smart” distribution	[55]	✗	✓	✗	✗	✗	Deleted
11	Workers employed	[52,55,58]	✓	✓	✓	✓	✓	Retained
12	Communication/control systems/control centers	[59]	✗	✓	✗	✓	✓	Deleted
13	Electrical protection and metering	[59]	✗	✓	✗	✓	✗	Deleted
14	Equipment positioning	[55]	✗	✓	✗	✗	✗	Deleted
15	Flow paths, line flow limits	[60]	✗	✓	✗	✓	✗	Deleted
16	Gen/load bus distribution	[60]	✗	✓	✗	✓	✗	Deleted
17	Reserve/spare capacity	[57,61,62]	✓	✓	✓	✓	✓	Retained
18	Substations (switchyards)—overhead lines and underground cables are interconnected	[59]	✗	✓	✗	✓	✗	Deleted
19	Ancillary service	[54]	✗	✓	✗	✗	✓	Deleted
20	Function-altered hazard rate of component after certain maintenance	[63]	✗	✓	✗	✓	✓	Deleted
21	Net ability—measures the aptitude of the grid in transmitting power from generation to load buses efficiently	[60]	✓	✓	✗	✓	✓	Deleted
22	Path redundancy—assesses the available redundancy in terms of paths in transmitting power from generation to a load bus based on entropy	[60]	✓	✓	✗	✗	✓	Deleted
23	Viability of investments	[52]	✗	✓	✗	✗	✓	Deleted
24	Coefficient of variation of the frequency index of sags	[64]	✗	✓	✗	✓	✓	Deleted
25	Bulk electric system reliability performance indices	[65]	✓	✓	✓	✓	✓	Retained
26	Derated power—rated power multiplied by the reliability of the plant	[66]	✗	✓	✗	✓	✓	Deleted
27	Energy efficiency/intensity	[62,67,68,69,70]	✓	✓	✓	✓	✓	Retained
28	Failure rate	[63]	✗	✓	✓	✓	✗	Deleted
29	Resilience index—parameter that quantifies the potential probability of malfunction of the system	[71]	✓	✓	✓	✓	✓	Retained
30	Resilience index—derived from robustness, resourcefulness, and recovery; ranges from 0 (low resilience) to 100 (high resilience)	[30,72,73]	✓	✓	✓	✓	✓	Retained
31	Survivability—evaluates the aptitude of the network to assure the possibility of matching generation and demand in case of failures or attacks	[60]	✓	✓	✗	✗	✓	Deleted
32	System average interruption duration/frequency index	[74]	✗	✓	✗	✓	✓	Deleted
33	Load loss damage index—damage caused by fire to the electrical system	[75]	✗	✓	✗	✓	✗	Deleted
34	Transmission lines available	[76]	✓	✓	✓	✓	✓	Retained
35	Functional zones—generation, transmission, and distribution	[52]	✓	✓	✓	✓	✓	Retained
36	Operator training	[55]	✗	✓	✗	✗	✓	Deleted
37	Mutual assistant agreements	[55]	✓	✓	✗	✗	✓	Deleted
38	Transformers—connecting parts of the network operating at different voltages	[59]	✓	✓	✓	✓	✓	Retained
39	Tree-trimming metrics	[55,57]	✓	✓	✗	✗	✓	Deleted
40	Adequacy—the ability of the system to supply customer requirements under normal operating conditions	[52]	✓	✓	✓	✓	✗	Deleted
41	Congestion control	[77]	✗	✓	✗	✓	✓	Deleted
42	Customer average interruption duration index—sustained outage metric; measures average duration of sustained outage per customer	[74]	✗	✓	✗	✓	✓	Deleted
43	Economy—achieving the best profits by adjusting the power system operation mode to minimize line losses, making full use of equipment, ensuring the security of the power system, and meeting utility users’ demand	[68]	✓	✓	✗	✗	✓	Deleted
44	Fairness—consists of the fulfillment rate of contracts and standard deviation indexes	[68]	✗	✓	✗	✗	✗	Deleted
45	Interrupted energy assessment rate	[65]	✗	✓	✗	✓	✓	Deleted
46	Security—the dynamic response of the system to unexpected interruptions; relates to the system’s ability to endure them	[52]	✓	✓	✗	✗	✓	Deleted
47	Transmission losses	[56]	✓	✓	✓	✓	✓	Retained
48	Cost of interruption—social, commercial, industrial, etc.	[56]	✓	✓	✗	✗	✓	Deleted
49	Impact factor on the population—share of the population affected by the power loss	[78]	✓	✗	✗	✓	✓	Deleted
50	Long-distance transmission costs	[56]	✓	✗	✓	✓	✓	Deleted
51	Noise	[56]	✗	✓	✗	✗	✓	Deleted
52	Performance-based regulation reward/penalty structure	[65]	✗	✓	✗	✗	✓	Deleted
53	Price of electricity	[56]	✓	✓	✓	✓	✓	Retained
54	Value of lost load—value of unserved energy; customers’ value of the opportunity cost of outages or benefits forgone through interruptions in electricity supply	[61]	✓	✓	✗	✗	✓	Deleted
55	Fuel nodes with the most links are the most interconnected and serve as hubs	[79]	✓	✓	✗	✓	✗	Deleted
56	Flow between nodes takes place on links (roads, electric power transmission lines, water mains, etc.)	[79,80,81]	✗	✓	✗	✗	✓	Deleted
57	Elements of the energy network that can receive fuels from storage facilities, pipeline interconnections, or production areas	[79,81]	✓	✓	✓	✓	✓	Retained
58	Primary energy supply—includes the systems and processes used to supply a primary energy resource to its point of conversion into the final energy product of interest	[52]	✓	✓	✓	✓	✓	Retained
59	Storage facilities/nodes, intermediate storage	[80,81]	✓	✓	✓	✓	✓	Retained
60	Emergency procedures/emergency shutdown system	[82]	✓	✓	✗	✗	✓	Deleted
61	Response to equipment outages—degree to which the system is able to continue to reliably operate in the event of equipment downtime	[52]	✗	✓	✗	✗	✓	Deleted
62	Adaptive capacity—degree to which the system is capable of self-organization for recovery of system performance levels	[83]	✓	✓	✗	✗	✓	Deleted
63	Ability of the system to provide sufficient throughput to supply final demand	[52]	✓	✓	✓	✓	✓	Retained
64	Information security—the degree to which information assets in the system are secure against threats	[52]	✓	✓	✓	✓	✓	Retained
65	Physical security—the degree towhich physical assets in the systemare secure against threats	[52]	✓	✓	✗	✗	✓	Deleted
66	Absorptive capacity—degree to which a system can automatically absorb the impacts of perturbations and minimize consequences with little effort	[83]	✓	✓	✗	✗	✓	Deleted
67	Connectivity loss—the average reduction in the ability of sinks to receive flow from sources	[78]	✓	✗	✓	✓	✓	Deleted
68	Energy processing and conversion—relates to production of the final energy product	[52]	✓	✓	✓	✓	✓	Retained
69	Flexibility—the degree to which the system can adapt to changing conditions	[52]	✓	✓	✗	✗	✓	Deleted
70	History—the degree to which the system has been prone to disruption in the past	[52]	✓	✓	✓	✓	✓	Retained
71	Intermittency—the degree to which the system lacks constant levels of productivity	[52]	✗	✓	✗	✗	✗	Deleted
72	Network resiliency—measured by its ability to keep supplying and distributing fuels in spite of damage to pipelines, import terminals, storage, and other sources	[79]	✓	✓	✗	✗	✓	Deleted
73	Response to demand fluctuations—the extent to which the system is able to adapt to changes in the quantity of energy demanded or location of demand	[52]	✓	✓	✓	✓	✓	Retained
74	Systemic impact—impact that a disruption has on system productivity; measured by evaluating the difference between a targeted system performance level and the actual system performance	[80,83]	✓	✓	✗	✗	✗	Deleted
75	Impacts on interdependent systems—the degree to which a disruption in the system might feasibly cause damage to interdependent systems	[52]	✓	✓	✗	✗	✓	Deleted
76	Optimal resilience costs—resilience costs for a system when the optimal recovery strategy (minimizing the combined system impact and total recovery effort costs) is employed	[83]	✗	✓	✗	✗	✓	Deleted
77	Recovery-dependent resilience costs—resilience costs of a system under a particular recovery strategy	[83]	✗	✓	✗	✗	✓	Deleted
78	Diversity of import fuels	[67]	✗	✓	✗	✓	✓	Deleted
79	Natural gas strategic reserve	[84]	✓	✓	✓	✓	✓	Retained
80	Import levels—the degree to which primary energy supply relies on resources originating outside of the system	[17,52,62,81,85,86,87,88,89,90,91,92]	✓	✓	✓	✓	✓	Retained
81	Industrial aspects—vulnerability indicator	[85]	✓	✓	✗	✗	✓	Deleted
82	Vulnerability—proportional to the reliance on imported gas from countries in geopolitical conflict	[85]	✗	✓	✗	✓	✓	Deleted
83	Ability to expand facilities—the degree to which the system can be easily and cost-effectively expanded	[52]	✓	✓	✗	✗	✓	Deleted
84	Pipeline capacity used	[79]	✓	✓	✗	✓	✓	Deleted
85	Resiliency—ability to supply gas to customers willing to pay the clearing price, even in the face of supply constraints	[84]	✗	✓	✗	✗	✓	Deleted
86	Restorative capacity—ability of a system to be repaired easily; these repairs are considered to be dynamic	[83]	✓	✓	✗	✗	✓	Deleted
87	Total recovery effort— efficiency with which a system recovers from a disruption, measured by analyzing the amount of resources expended during the recovery process	[83]	✓	✓	✗	✗	✓	Deleted
88	Sector coordination—the degree to which coordination between stakeholders within the sector results in an effective exchange of information, alerting stakeholders of emerging threats and mitigation strategies	[52]	✓	✓	✓	✓	✓	Retained
89	Price/price volatility	[52,84]	✓	✓	✓	✓	✓	Retained
90	Intelligent institutional leadership with heightened sensitivity and/or preparedness for rapid and pervasive changes	[93]	✗	✓	✗	✗	✓	Deleted
91	Diversity of electricity generation	[16,17,31,34,62,86,87,88,89,90,91,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108]	✓	✓	✓	✓	✓	Retained
92	Diversity of imports of embodied electricity	[34]	✗	✓	✗	✓	✓	Deleted
93	Diversity of electricity consumption	[34]	✓	✓	✓	✓	✓	Retained
94	Renewable energy electricity, mainly wind and solar power	[109,110,111]	✓	✓	✓	✓	✓	Retained
95	Share of buildings with low thermal insulation in the total building stock	[112]	✓	✓	✗	✗	✓	Deleted
96	Share of renewables in total heating energy	[112]	✓	✓	✗	✓	✓	Deleted
97	Share of fossil fuels in total energy consumption	[112]	✓	✗	✓	✓	✓	Deleted
98	Share of electricity produced by renewables in total electricity consumption	[8,112]	✓	✗	✓	✓	✓	Deleted
99	Nonrenewable fuel used in generation	[62]	✓	✗	✓	✓	✓	Deleted
100	Generation efficiency	[62]	✓	✗	✓	✓	✓	Deleted
101	Distribution efficiency—transmission and distribution losses and the amount of electricity consumed by energy industry	[62]	✓	✗	✗	✓	✓	Deleted
102	Carbon intensity of generation	[17,49,62,87,91,98,113]	✓	✗	✗	✓	✗	Deleted
103	Redundant power for use	[62]	✓	✓	✓	✓	✓	Retained
104	Existence and monitoring of officially approved electrification plan	[114]	✗	✓	✗	✗	✗	Deleted
105	Framework for grid electrification	[114]	✗	✓	✗	✗	✓	Deleted
106	Framework for minigrids	[114]	✗	✓	✗	✗	✓	Deleted
107	Framework for standalone systems	[114]	✓	✓	✗	✗	✓	Deleted
108	Consumer affordability of electricity	[110,114]	✓	✓	✗	✗	✓	Deleted
109	Utility transparency and monitoring	[114]	✗	✓	✗	✗	✗	Deleted
110	Utility creditworthiness	[114]	✗	✓	✗	✗	✗	Deleted
111	Information provided to consumers about electricity usage	[114]	✓	✓	✗	✗	✓	Deleted
112	Financing mechanisms for energy efficiency	[114]	✓	✓	✗	✗	✓	Deleted
113	Energy efficiency entities	[114]	✗	✓	✗	✗	✓	Deleted
114	Incentives from electricity rate structures	[114]	✗	✓	✗	✗	✓	Deleted
115	Incentives and mandates: large consumers/public sector/utilities	[114]	✗	✓	✗	✗	✓	Deleted
116	Minimum energy efficiency performance standards	[114]	✓	✓	✗	✗	✓	Deleted
117	Energy labeling systems	[114]	✓	✓	✗	✗	✓	Deleted
118	Building energy codes	[114]	✓	✓	✗	✗	✓	Deleted
119	Carbon pricing and monitoring	[95,114,115,116,117]	✓	✓	✗	✗	✓	Deleted
120	Legal framework for renewable energy	[114]	✓	✓	✗	✗	✓	Deleted
121	Planning for renewable energy expansion	[114]	✓	✗	✗	✗	✓	Deleted
122	Incentives and regulatory support for renewable energy	[114]	✓	✓	✗	✗	✓	Deleted
123	Attributes of financial and regulatory incentives for renewable energy	[114]	✓	✓	✗	✗	✓	Deleted
124	Network connection and pricing	[114]	✓	✗	✗	✗	✓	Deleted
125	Counterparty risk of renewable energy	[114]	✗	✓	✗	✗	✗	Deleted
126	Maximized availability of operational power supply	[118]	✗	✓	✗	✗	✓	Deleted
127	Replacement inventories of equipment and supplies	[110,118]	✓	✓	✗	✗	✓	Deleted
128	Maximized provision target power supply level of restoration	[118]	✓	✓	✗	✗	✓	Deleted
129	Largest single source of supply	[17]	✓	✓	✓	✓	✗	Deleted
130	Energy portfolios—price volatility	[17]	✓	✗	✓	✓	✓	Deleted
131	Statistical probability of supply interruption in network industries (gas and electricity)	[17]	✗	✓	✗	✓	✓	Deleted
132	Expected number of annual hours in which energy is unserved	[17]	✓	✓	✗	✗	✓	Deleted
133	Value/level of unserved energy	[17]	✓	✓	✓	✓	✓	Retained
134	Energy storage capacity and/or stocks by fuel and market	[17]	✓	✗	✓	✓	✓	Deleted
135	Redundancy in network architecture	[17]	✗	✓	✗	✗	✓	Deleted
136	Expected probability of interruption for long-term planning and design	[119]	✗	✓	✓	✗	✓	Deleted
137	Expected energy not served per interruption	[119]	✗	✓	✓	✗	✓	Deleted
138	Expected outage duration per interruption for short-term operational planning	[119]	✗	✓	✓	✗	✓	Deleted
139	Expected energy loss	[24]	✗	✗	✓	✗	✓	Deleted
140	Collapse ratio	[24]	✗	✓	✓	✓	✗	Deleted
141	Recovery ratio	[24,110]	✗	✓	✓	✓	✗	Deleted
142	Energy cost stability	[120]	✓	✗	✓	✓	✓	Deleted
143	Stability of energy generation	[120]	✓	✓	✗	✗	✓	Deleted
144	Peak load response	[120]	✗	✓	✓	✓	✓	Deleted
145	Market concentration on supply	[120]	✗	✓	✗	✗	✓	Deleted
146	CO2eq emissions	[120]	✗	✗	✓	✓	✓	Deleted
147	Fuel use	[120]	✓	✗	✓	✓	✓	Deleted
148	Employment	[120]	✓	✗	✓	✓	✓	Deleted
149	Levelized costs (incl. capital, operational/maintenance, fuel costs)	[120]	✓	✓	✗	✓	✓	Deleted
150	Technological maturity	[120]	✓	✓	✗	✗	✓	Deleted
151	Technological innovation ability	[120]	✓	✓	✗	✗	✓	Deleted
152	Energy demand and consumption	[8,121]	✓	✗	✓	✓	✓	Deleted
153	Flexibility of grid	[8,121]	✗	✓	✗	✗	✓	Deleted
154	Urban energy supply systems for increasing shares of renewable energy	[121,122]	✓	✗	✓	✓	✓	Deleted
155	Reduced end-use energy demand	[111,121,122]	✓	✓	✓	✓	✓	Retained
156	Energy monitoring	[8,121]	✗	✓	✗	✗	✗	Deleted
157	Reduced reliance on energy	[16,62,123,124,125]	✓	✗	✓	✓	✓	Deleted
158	Energy source diversity	[16,62,111,123,125,126,127]	✓	✗	✓	✓	✓	Deleted
159	Energy storage capabilities	[124,125,126]	✓	✗	✓	✓	✓	Deleted
160	Redundancy of critical capabilities	[62,126,128,129]	✓	✓	✗	✗	✓	Deleted
161	Preventative maintenance on energy systems	[110,126,129]	✗	✓	✗	✗	✓	Deleted
162	Sensors, controls, and communication links to support awareness and response	[125,126,129]	✓	✓	✗	✗	✗	Deleted
163	Protective measures against external attack	[123,126,128]	✓	✓	✗	✗	✓	Deleted
164	Design margin to accommodate range of conditions	[124,126,129,130,131]	✗	✓	✗	✗	✓	Deleted
165	Limited performance degradation under changing conditions	[16,124,126,129,130]	✓	✓	✗	✗	✓	Deleted
166	Operational system protection, e.g., pressure relief, circuit breakers	[126,129]	✓	✓	✗	✗	✓	Deleted
167	Installed/ready redundant components	[16,31,49,90,126,128,129,132,133,134,135]	✓	✓	✗	✓	✓	Deleted
168	Ability to isolate damaged systems/components (automatic/manual)	[62,126,129]	✓	✓	✗	✗	✓	Deleted
169	Capability for independent local/subnetwork operation	[126,128]	✓	✓	✗	✗	✓	Deleted
170	System flexibility for reconfiguration and/or temporary system installation	[16,125,126,128,130]	✓	✓	✗	✗	✓	Deleted
171	Capability to monitor and control portions of system	[124,126,129]	✓	✓	✗	✗	✓	Deleted
172	Fuel flexibility	[16,31,62,99,128,130,136,137]	✓	✗	✓	✗	✓	Deleted
173	Capability to reroute energy from available sources	[16,126,128,129,130]	✓	✓	✗	✗	✗	Deleted
174	Investigate and repair malfunctioning controls or sensors	[129]	✗	✓	✗	✗	✓	Deleted
175	Energy network flexibility to reestablish service by priority	[16,126,129]	✓	✓	✗	✗	✓	Deleted
176	Backup communication lighting, power systems for repair/recovery operations	[126,129]	✓	✓	✗	✓	✓	Deleted
177	Flexible network architecture to facilitate modernization and new energy sources	[16,126,128,130]	✓	✓	✗	✗	✓	Deleted
178	Sensors and data collection and visualization capabilities to support system performance trending	[62,126,128,129]	✓	✓	✗	✗	✓	Deleted
179	Ability to use new/alternative energy sources	[16,125,130]	✓	✓	✗	✗	✓	Deleted
180	Updating system configuration/functionality based on lessons learned	[16,126,128,129,130]	✓	✓	✗	✗	✓	Deleted
181	Phasing out obsolete or damaged assets and introducing new assets	[123,126,128,129,130,133,138,139]	✓	✓	✗	✗	✓	Deleted
182	Integrating new interface standards and operating system upgrades	[126,128,129]	✓	✗	✗	✗	✓	Deleted
183	Updating response equipment/supplies based on lessons learned	[128]	✓	✓	✗	✗	✓	Deleted
184	Capabilities and services prioritized based on criticality or performance requirements	[124]	✓	✓	✗	✗	✓	Deleted
185	Internal and external system dependencies identified	[124,125,140]	✗	✓	✗	✗	✗	Deleted
186	Design, control, operational, and maintenance data archived and protected	[124,129]	✓	✓	✗	✓	✓	Deleted
187	Vendor information available	[124]	✗	✓	✗	✓	✗	Deleted
188	Control systems operational and protected with antivirus and other safeguards	[124,126,129]	✓	✓	✗	✗	✓	Deleted
189	Operating environment forecasts captured in planning scenarios	[123,124,126,129]	✓	✓	✗	✗	✗	Deleted
190	Response/recovery plans established and distributed	[124,126,129]	✓	✓	✗	✗	✗	Deleted
191	Environmental condition forecast and event warnings broadcast	[62,125,129]	✓	✓	✗	✗	✓	Deleted
192	System status, trends, and margins available to operators, managers, and customers	[62,110,125,126,128,129]	✓	✓	✗	✗	✓	Deleted
193	Critical system data monitored; anomalies alarmed	[62,126,128,129]	✓	✗	✗	✗	✓	Deleted
194	Operational/troubleshooting/response procedures available	[126,129]	✓	✓	✗	✗	✓	Deleted
195	Status/trend limits trigger safeguards and isolate components to stop cascade effect	[62,125,126]	✓	✓	✗	✗	✓	Deleted
196	Status/response/mitigation information transmitted effectively and efficiently to stakeholders/decision makers	[124]	✓	✓	✗	✗	✓	Deleted
197	Information and communications coordinated throughout supply chain	[126]	✓	✗	✗	✗	✓	Deleted
198	Information available to authorities and crews regarding customer/community needs/status	[128,129]	✓	✓	✗	✗	✓	Deleted
199	Recovery progress tracked, synthesized, and available to decision makers and stakeholder	[128,129]	✓	✓	✗	✗	✓	Deleted
200	Design, repair parts, and substitution information available to recovery teams	[126]	✓	✓	✗	✗	✓	Deleted
201	Location, availability, and ownership of energy, hardware, and services for restoration teams	[126]	✓	✓	✗	✗	✓	Deleted
202	Resource needs, sources, and authorities available to decision makers	[128]	✓	✗	✗	✗	✓	Deleted
203	Information regarding centralized facilities and distribution of essential supplies and services available to community	[128]	✓	✗	✗	✗	✓	Deleted
204	Coordinating information and communications among recovery organizations	[128]	✓	✓	✗	✗	✓	Deleted
205	Initiating event, incident point of entry, and associated vulnerabilities and impacts identified	[123,125,126,128,129]	✓	✓	✗	✗	✓	Deleted
206	Event data and operating environment forecasts utilized to anticipate future conditions/events	[125,126,128,129]	✓	✓	✗	✗	✓	Deleted
207	Updated information about energy resources, alternatives, and emergent technologies available to managers and stakeholders	[16,125,128,129]	✓	✗	✗	✗	✓	Deleted
208	Design/operation/maintenance information updated consistently with system modifications	[16,126,129]	✓	✓	✗	✗	✓	Deleted
209	Consumer/stakeholder awareness of energy alternatives, cost/benefits, and implementation requirements	[16,124,125]	✓	✓	✗	✗	✓	Deleted
210	Community impacts, priorities, interdependencies updated to capture lessons learned	[124,128,129]	✓	✓	✗	✗	✗	Deleted
211	Response plans updated with lessons learned	[125,126,128,129]	✓	✓	✗	✗	✓	Deleted
212	Understood performance trade-offs of organizational goals	[123,125]	✗	✓	✗	✗	✗	Deleted
213	Broad-based operational and maintenance training	[126,129]	✗	✓	✗	✗	✓	Deleted
214	Periodic operator, management, and community drills	[126,128,129]	✗	✓	✗	✗	✓	Deleted
215	Developed individual expertise in energy impacts, techniques, and alternatives (energy-informed culture)	[124]	✓	✓	✗	✓	✓	Deleted
216	Awareness of and focusing of effort on identified critical assets and services	[124,126,128]	✗	✓	✗	✗	✗	Deleted
217	Decision-making protocol or aid to determine proper course of action	[125,126,128]	✗	✓	✗	✗	✓	Deleted
218	Operators and managers utilizing critical thinking and maintain proactive posture to recognized and arrest events	[125,126]	✓	✓	✗	✗	✓	Deleted
219	Community response to mitigate impact, e.g., demand curtailment	[124,126,128]	✓	✓	✗	✓	✓	Deleted
220	Utilizing data and decision-making aids to quickly select recovery options	[128]	✓	✓	✗	✗	✓	Deleted
221	Recovery crew managing incremental recovery with available equipment	[126]	✓	✓	✗	✗	✓	Deleted
222	Community members utilizing available resources and improvised to meet local needs	[16,124,125,128]	✗	✓	✗	✗	✓	Deleted
223	Community members managing constrained energy resources responsibly and consistent with public guidance	[16,124,128]	✗	✓	✗	✗	✓	Deleted
224	Documentation and review of management response and decision-making processes	[125,126,128]	✗	✓	✗	✗	✗	Deleted
225	Periodic revisitation of organizational risk tolerance and mission priorities, adjusting as necessary	[124,125]	✗	✓	✗	✗	✓	Deleted
226	Integration of lessons learned and best practices from internal and external sources	[125,126,128,129]	✓	✓	✗	✗	✓	Deleted
227	Customers and stakeholders taking action to implement more resilient energy solutions	[16,124,125,126,129]	✓	✗	✗	✗	✓	Deleted
228	Identification of stakeholders (internal and external)	[126,128]	✗	✓	✗	✗	✓	Deleted
229	Use of scenario-based war gaming to develop understanding of system dependencies and interactions	[125,126,128,131]	✓	✓	✗	✗	✓	Deleted
230	Robust risk analysis and decision support capabilities to facilitate response	[123,124,125,126,128,129]	✓	✗	✗	✗	✓	Deleted
231	Decreased overall reliance on energy or specific sources of energy	[123,124]	✓	✓	✗	✓	✓	Deleted
232	Priorities and policies established for event response	[123,124,125,126,128,129]	✗	✓	✗	✗	✓	Deleted
233	Priorities and operating limits mitigating disruption to energy needs for key community functions	[123,126,128]	✗	✓	✗	✗	✓	Deleted
234	Predefined protective actions limiting external influences in physical, information domains	[124,125,126]	✗	✓	✗	✗	✓	Deleted
235	Agile operational management enabling rapid and effective response under changing conditions	[125,126]	✓	✓	✗	✗	✓	Deleted
236	Individuals and organizations implementing response plans	[124,125,126,128]	✗	✗	✗	✗	✓	Deleted
237	Individuals and organizations taking action in response to observations and/or direction from authorities	[124,128]	✗	✓	✗	✗	✓	Deleted
238	Recovery organizations and communities following contingency recovery plans	[124,125,128]	✓	✓	✗	✗	✓	Deleted
239	Community stakeholders participating in establishment of energy priorities and coordination of restoration actions	[124,126,128]	✓	✓	✗	✗	✓	Deleted
240	Shelters and other centralized services increasing efficiency and control of scarce energy resources to meet critical needs	[126]	✗	✗	✗	✗	✗	Deleted
241	Public/private entities coordinating to deliver aid to affected parties	[128]	✗	✓	✗	✗	✓	Deleted
242	Proactive neighborhood assistance, volunteerism, and compliance with energy response manager direction	[128]	✗	✓	✗	✗	✓	Deleted
243	Reallocation of human resources to better address adverse events	[128]	✓	✓	✗	✗	✓	Deleted
244	Local governments and stakeholders staying informed about threats, changing environment, and protective methods and technologies	[123,124,125,126,128,129]	✓	✓	✗	✗	✓	Deleted
245	Local governments and stakeholders collaborating to develop, prioritize, and implement energy portfolio improvement	[16,123,124,125,126,128,129]	✓	✓	✗	✗	✓	Deleted
246	Incentives for customers and stakeholders to implement more resilient energy solutions	[16,62,123,124,125,126,128,129]	✓	✓	✗	✗	✓	Deleted
247	Energy-informed culture leading to collective decisions and investments which continually improve energy effectiveness	[16,62,126,128]	✗	✓	✗	✗	✓	Deleted
248	Accurate estimation of weather location and severity	[57]	✓	✗	✗	✗	✓	Deleted
249	Energy consciousness of the public and consumption behavior/demand-side management	[8,31,57,69,70,94,99,101,104,113,133,139,141,142,143,144,145,146,147,148,149,150,151,152,153,154]	✓	✓	✓	✓	✓	Retained
250	Fast topology reconfiguration	[57]	✓	✓	✗	✗	✓	Deleted
251	Automated protection and control actions: load and generation rejection, system separation, etc.	[57]	✓	✓	✗	✗	✓	Deleted
252	Monitoring—development of situation awareness, advanced visualization and information systems	[57]	✓	✗	✗	✗	✓	Deleted
253	Ensured communications functionality	[57]	✓	✗	✗	✗	✓	Deleted
254	Microgrids	[57,155,156]	✓	✓	✓	✓	✓	Retained
255	Advanced control and protection schemes	[57,110]	✓	✗	✗	✗	✓	Deleted
256	Disaster assessment and priority setting	[57]	✓	✗	✗	✗	✓	Deleted
257	Risk assessment and management for evaluating and preparing for the risk introduced by such events	[57,122]	✓	✗	✗	✗	✓	Deleted
258	Black-start capabilities installed	[57]	✓	✓	✗	✗	✓	Deleted
259	Repair crew member mobilization	[57]	✓	✓	✗	✗	✓	Deleted
260	Installation of DER or other onsite generation units	[57]	✓	✓	✗	✓	✓	Deleted
261	Coordination with adjacent networks, and repair crews	[57]	✓	✓	✗	✗	✓	Deleted
262	Upgrading poles and structures with stronger, more robust materials	[57]	✗	✗	✗	✗	✓	Deleted
263	Elevating substations and relocating facilities to areas less prone to flooding	[57]	✗	✓	✗	✗	✓	Deleted
264	Redundant transmission routes via additional transmission facilities	[57]	✓	✓	✗	✗	✓	Deleted
265	Available energy sources/generation methods	[110]	✓	✗	✓	✓	✓	Deleted
266	Number of service connections able to handle entire load	[110]	✗	✓	✗	✗	✓	Deleted
267	Damage assessment methods	[110]	✓	✓	✗	✗	✓	Deleted
268	Scenario/contingency planning	[110]	✓	✓	✗	✗	✓	Deleted
269	Local availability of tools/expertise to address damage	[110]	✗	✓	✗	✗	✓	Deleted
270	Load shedding and load factor	[110]	✓	✓	✗	✗	✓	Deleted
271	Estimated lifespan of generation plant	[110]	✓	✓	✗	✓	✓	Deleted
272	Fortification and robustness (physical security)	[62,89,96,98,143,157,158,159]	✓	✗	✓	✓	✓	Deleted
273	Operational system protection, e.g., system relief, circuit breakers	[31]	✓	✗	✗	✗	✓	Deleted
274	Diversification of energy supply—fuel mix, multisourcing, type of generation	[16,17,31,62,86,87,88,89,90,91,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108]	✓	✗	✓	✓	✓	Deleted
275	Spatially distributed generation (and critical facilities)	[31,95,96,99,109,138,139,141,160,161,162,163]	✓	✗	✗	✗	✓	Deleted
276	Energy production near point of use (colocation of supply and demand)	[96,164,165]	✓	✓	✗	✗	✓	Deleted
277	On-site energy production (photovoltaics, micro-combined heat and power, trigeneration, thermal panels, small wind turbines mounted at the corners of the roof)	[16,70,99,102,147,148,149,150,158,159,161,166,167,168,169,170,171,172,173,174,175]	✓	✗	✗	✓	✓	Deleted
278	Solar absorption cooling	[176,177]	✗	✓	✗	✓	✓	Deleted
279	Large wind turbines located outside the built-up area	[162,178,179]	✗	✓	✗	✓	✓	Deleted
280	Large solar thermal collectors	[149,178]	✗	✓	✗	✓	✓	Deleted
281	Smart microgrids fed by microturbines and solar panels (photovoltaics, building integrated photovoltaics) and storage facilities	[62,104,109,136,138,141,142,144,151,152,158,180,181,182,183]	✗	✓	✗	✓	✓	Deleted
282	Building-integrated photovoltaic/thermal for recovery of heat loss form photovoltaics and building integrated photovoltaics	[180]	✗	✓	✗	✓	✓	Deleted
283	Ground source heat pumps	[149,150,178,184,185]	✗	✓	✗	✓	✓	Deleted
284	Waste heat or biomass-fueled combined heat and power plants	[138,178,186]	✓	✓	✓	✓	✓	Retained
285	Biofuel energy (food waste, second generation cellulosic biofuels, third generation using algae, etc.)	[139,182,184,187,188,189,190]	✓	✓	✓	✓	✓	Retained
286	Biomass supply chain, wood pellet systems	[101,139]	✗	✓	✗	✓	✓	Deleted
287	Interdependency and interconnection of infrastructures and their networks	[95,96,99,115,159,160,165,191]	✓	✓	✓	✓	✓	Retained
288	Regular maintenance	[31,33,88,96]	✓	✓	✗	✓	✓	Deleted
289	Generation, transmission, and distribution efficiency (leakages, etc.)	[62,86,87,98,192]	✓	✗	✓	✓	✓	Deleted
290	Age of the fleet (feeder lines, etc.)	[62,193]	✗	✓	✗	✓	✓	Deleted
291	Type of feeder lines (overhead/underground cables; looped/interconnected or radial configuration)	[49,95,146,158,159,193,194]	✗	✓	✗	✓	✓	Deleted
292	Natural gas distribution: continuous (grid) vs. discontinuous (propane tanks)	[195]	✗	✓	✗	✓	✓	Deleted
293	Alternative and safer energy sources for critical infrastructure such as parking gates, traffic lights, subway, etc.	[96,191]	✓	✓	✗	✓	✓	Deleted
294	Intelligent ICT infrastructure and cybersecurity thereof for maintaining grid operation	[31,33,49,96,133,158,191,196,197]	✓	✓	✗	✓	✓	Deleted
295	Flexible network architecture	[31]	✗	✓	✗	✗	✓	Deleted
296	Number of configuration of nodes and links in the transmission and distribution grid	[17,22,198]	✓	✓	✓	✓	✓	Retained
297	Backup energy sources and stocks of energy	[17,33,96]	✓	✗	✓	✓	✓	Deleted
298	Energy storage facilities involving electro-chemical batteries, flow batteries, hydrogen, etc.	[16,49,70,86,90,109,138,144,146,199]	✗	✓	✗	✓	✓	Deleted
299	Distributed storage	[95,158]	✓	✗	✗	✓	✓	Deleted
300	Connectivity of generation and storage infrastructure	[88,89,200]	✗	✓	✗	✗	✓	Deleted
301	Backup data of the utility infrastructure (information networks, data sharing, etc.)	[31,157]	✗	✓	✗	✗	✓	Deleted
302	Spare capacity and reserve margins—resources, transmission lines, etc.	[31,49,62,98,100,191,201,202]	✓	✗	✗	✗	✓	Deleted
303	Vehicle-to-grid and vehicle-to-community selling of surplus power	[70,150,203]	✗	✓	✗	✗	✓	Deleted
304	Parks and open space, bioswales, etc. (attention to regular trimming of trees)	[193,204,205,206,207,208,209,210,211,212,213,214,215,216,217,218]	✓	✗	✓	✓	✓	Deleted
305	Indigenous (native) vs. invasive plants	[138,208]	✗	✓	✗	✓	✗	Deleted
306	Deciduous trees for cold climate	[168]	✗	✓	✗	✓	✗	Deleted
307	Xeriscape for hot and arid climates	[207,219]	✗	✓	✗	✓	✗	Deleted
308	Urban agriculture (vacant lands, marginal lands, etc.)	[220]	✗	✓	✗	✗	✗	Deleted
309	Green area ration	[213]	✓	✗	✓	✓	✓	Deleted
310	Green wall (vegetative covering, green façade)	[213,221,222,223]	✗	✓	✗	✗	✗	Deleted
311	Green roof (living roof)	[138,206,215,219,224,225,226,227]	✗	✓	✗	✗	✗	Deleted
312	Rainwater harvesting, decentralized water harvesting systems	[137,147,204,228]	✗	✓	✗	✗	✗	Deleted
313	Water conservation	[147,219]	✗	✓	✗	✗	✗	Deleted
314	Heat recovery and energy generation from sewage	[204,229]	✗	✓	✗	✓	✓	Deleted
315	Separation of used water into grey and black flows	[219]	✗	✓	✗	✓	✗	Deleted
316	Removing and recovering ammonium and phosphate from wastewater	[219]	✗	✓	✗	✓	✗	Deleted
317	Waterscape as a natural heat sink	[209,215,230]	✗	✓	✗	✗	✗	Deleted
318	Roof ponds	[99,122,136,231]	✗	✓	✗	✗	✗	Deleted
319	Redesign and refurbishment (retrofit)	[113,115,139,148,149,151,164,207,219,232,233,234,235]	✗	✓	✗	✗	✗	Deleted
320	Glazing	[113,115,139,148,149,151,164,207,219,232,233,234,235]	✗	✓	✗	✗	✗	Deleted
321	Net zero- and net positive-energy buildings	[148,163,235,236]	✓	✓	✗	✗	✓	Deleted
322	Insulation and dynamic insulation of buildings	[104,109,139,141,147,148,149,152,153,159,168,176,214,219,233,235,237,238,239]	✗	✓	✗	✗	✓	Deleted
323	Cut-off air conditioning waste heat discharge	[223]	✗	✓	✗	✗	✗	Deleted
324	Net zero-energy neighborhoods	[148]	✗	✓	✗	✗	✓	Deleted
325	Pooling of the built environment (shared walls)	[148,217]	✗	✓	✗	✗	✓	Deleted
326	District energy systems—using low-temperature heat from renewable sources and industrial waste heat	[87,137,138,151,184]	✓	✓	✗	✗	✓	Deleted
327	Infrastructure for active transportation modes	[136,138,164,168,196,220,240,241,242,243,244]	✗	✓	✗	✗	✗	Deleted
328	Modal split	[87,241]	✗	✓	✗	✗	✗	Deleted
329	Size of cars	[196]	✗	✓	✗	✗	✗	Deleted
330	Fuel efficiency of cars	[115,196,243]	✗	✓	✗	✗	✗	Deleted
331	Supporting promotion of hybrid vehicles and installing electric vehicle plug-ins in locations where multiple use can be achieved	[31,70,99,136,137,138]	✓	✓	✓	✓	✓	Retained
332	Enhancing energy efficiency through innovation and technology (building, industry, transportation)	[31,62,69,94,96,99,117,143,144,147,150,164,165,180,184,186,228,237,241,243,245]	✓	✓	✗	✗	✓	Deleted
333	Energy conservation	[139]	✓	✗	✓	✓	✓	Deleted
334	Energy self sufficiency	[91,99,160]	✗	✓	✓	✓	✗	Deleted
335	Energy cycling	[70,142]	✗	✓	✗	✗	✓	Deleted
336	Waste management and waste incineration	[86,108,147,184]	✓	✗	✓	✓	✓	Deleted
337	Environmental and socioeconomic impacts of energy system	[86,98,99,108]	✗	✓	✗	✗	✓	Deleted
338	Reducing energy footprint of water production, treatment, and distribution	[95,116,138,192,228,229,246,247]	✗	✓	✗	✗	✓	Deleted
339	Provision of less energy-intensive rainwater harvesting systems in buildings	[228]	✗	✓	✗	✗	✓	Deleted
340	Water and energy resource coupling	[109]	✗	✓	✗	✗	✓	Deleted
341	Reducing energy footprint of wastewater collection, treatment, and discharge	[138]	✗	✓	✗	✗	✓	Deleted
342	Reducing water footprint of energy production and transmission	[95,116,192,246,247]	✓	✓	✗	✗	✓	Deleted
343	Improving the efficiency of energy production by enhancing water quality	[187]	✓	✓	✗	✗	✓	Deleted
344	Understanding the water intensity of fuels used for electricity generation	[247]	✗	✓	✗	✗	✓	Deleted
345	Less water-intensive technologies for cooling purposes in thermoelectric plants	[95,192,246]	✗	✓	✗	✗	✓	Deleted
346	Use of natural gas for steamed turbines and combined cycle plants	[192,246]	✓	✓	✗	✗	✓	Deleted
347	Use of wet cooling towers instead of once-through cooling	[246]	✓	✓	✗	✗	✓	Deleted
348	Knowing groundwater implications of energy (technologies, extraction, etc.)	[86,187,229]	✗	✓	✗	✗	✓	Deleted
349	Scenario-based energy planning and risk management	[31,133,229]	✗	✗	✗	✗	✓	Deleted
350	Risk communication and energy response of urban governance	[96]	✗	✗	✗	✗	✓	Deleted
351	Community involvement in and/or ownership of renewable energy generation	[96]	✓	✓	✗	✗	✓	Deleted
352	Institutional coordination on water, food, health, and energy nexus	[116]	✓	✓	✗	✗	✓	Deleted
353	Reliance on nuclear energy	[31,154]	✓	✓	✓	✓	✓	Retained
354	Regular publication of energy planning documents and statistics	[99]	✗	✓	✗	✗	✓	Deleted
355	Market competitiveness and investment risk of decentralized renewable energy	[99,139,150,239]	✗	✓	✗	✗	✓	Deleted
356	Requirement for suppliers to source a proportion of electricity from renewables	[239]	✗	✓	✗	✗	✓	Deleted
357	Legal and regulatory frameworks to encourage technological development and transition towards energy resilience	[161,180,248]	✗	✓	✗	✗	✓	Deleted
358	Measures against electricity theft	[249]	✗	✓	✗	✗	✓	Deleted
359	Attracting private sector’s investment in low-carbon development	[95,115,116,117]	✗	✓	✗	✗	✓	Deleted
360	Financial and nonfinancial mechanisms and incentives for promoting green products and renewable energy technologies and enhancing affordability	[95,115,116,117]	✗	✓	✗	✗	✓	Deleted

,

Table A2. Aggregated index selection for CI.

No.	Primary Index	Ref.	S	U	F	O	R	Result
1	Train transportation	[250]	✓	✓	✓	✓	✓	Retained
2	Emergency organization and infrastructure in place and critical functions identified	[44,118]	✓	✓	✗	✗	✓	Deleted
3	Waste and disposal	[41,120,122]	✓	✓	✓	✓	✓	Retained
4	Land use requirement	[120]	✗	✓	✗	✗	✓	Deleted
5	Level of public resistance/opposition	[120]	✗	✓	✗	✗	✗	Deleted
6	Market size—domestic/potential export	[120]	✓	✓	✗	✗	✓	Deleted
7	Permeable pavement and bioswales	[121]	✓	✓	✗	✗	✓	Deleted
8	Urban tree canopy	[121]	✗	✓	✗	✗	✗	Deleted
9	Water demand and consumption	[8,121,122,251,252]	✓	✓	✓	✓	✓	Retained
10	Water-efficient landscaping	[8,41,121]	✗	✓	✗	✗	✗	Deleted
11	Protection of water-sensitive lands	[121]	✓	✓	✓	✗	✗	Deleted
13	Water quality and quantity monitoring	[121,252]	✗	✓	✓	✓	✗	Deleted
14	High-efficiency irrigation	[8,121]	✗	✓	✓	✓	✗	Deleted
15	High-frequency schedule for public transportation	[41,42,121]	✗	✓	✓	✗	✗	Deleted
16	Principle arterial miles per square mile	[121]	✗	✓	✓	✗	✗	Deleted
17	Vehicle ownership	[8,10,121,251,253]	✓	✓	✓	✓	✓	Retained
18	Parks	[8,121]	✗	✓	✓	✓	✗	Deleted
19	Forest conservation	[8,121]	✗	✓	✓	✗	✗	Deleted
20	Waste management	[8,121]	✓	✗	✓	✓	✓	Deleted
21	Provision of open space for shelter	[8,121,122]	✓	✓	✓	✓	✓	Retained
22	Percentage of vacant rental units	[121]	✗	✓	✗	✓	✗	Deleted
23	Number of hotels/motels per square mile	[8,121]	✗	✓	✓	✓	✗	Deleted
24	Evacuation route	[8,121]	✗	✓	✓	✗	✗	Deleted
25	Building insulation, layout, and orientation	[121]	✗	✓	✗	✓	✗	Deleted
26	Reducing air infiltration and thermal bridging	[121]	✗	✓	✗	✓	✓	Deleted
27	Natural ventilation	[121]	✗	✓	✗	✗	✗	Deleted
28	Preservation of housing	[121]	✗	✓	✗	✗	✗	Deleted
29	Building codes	[121]	✗	✓	✓	✓	✗	Deleted
30	Housing age	[121]	✗	✓	✓	✓	✗	Deleted
31	Generating and making use of information	[121]	✓	✓	✗	✗	✓	Deleted
32	Geospatial information and communication technology	[121]	✓	✓	✗	✗	✓	Deleted
33	Volunteered geographic information	[121]	✗	✓	✗	✗	✓	Deleted
34	Visualization technologies	[121]	✗	✓	✗	✗	✓	Deleted
35	Alerts and emergency notification systems	[121]	✓	✓	✗	✗	✓	Deleted
36	Embracing e-commerce	[121]	✗	✓	✗	✗	✓	Deleted
37	Biodiversity	[8,121]	✗	✓	✗	✗	✓	Deleted
38	Restoration of hydrologic flows	[8,121]	✗	✓	✗	✗	✓	Deleted
39	Conservation of ecologically vulnerable areas	[121,254]	✗	✓	✗	✗	✓	Deleted
40	Proximity of different habitats	[121]	✗	✓	✗	✗	✓	Deleted
41	Erosion rates	[121]	✓	✓	✗	✗	✓	Deleted
42	Urban green commons	[121,122]	✓	✓	✓	✓	✓	Retained
43	Culture of cooperation	[121]	✗	✓	✗	✗	✗	Deleted
44	Balance demographic distribution	[121]	✗	✗	✓	✓	✓	Deleted
45	Aging population	[121]	✗	✓	✗	✗	✓	Deleted
46	Responsive health systems	[121]	✓	✓	✓	✓	✓	Retained
47	Health coverage and access	[8,121,253]	✓	✓	✓	✓	✓	Retained
48	Road density	[10,45,251]	✓	✓	✓	✓	✓	Retained
49	Distribution of fire stations	[45]	✓	✓	✗	✓	✓	Deleted
50	Distribution of police stations	[45]	✓	✓	✗	✓	✓	Deleted
51	Distribution of civil air defense facilities	[45]	✓	✓	✗	✓	✓	Deleted
52	Distribution of emergency shelters	[45]	✓	✓	✗	✓	✓	Deleted
53	Land types	[45]	✗	✓	✗	✗	✗	Deleted
54	College students	[251]	✗	✓	✓	✓	✓	Deleted
55	Hospital distribution	[10,45]	✓	✓	✓	✓	✓	Retained
56	Medical rescue capability	[10,45,251]	✓	✓	✓	✓	✓	Retained
57	Ecological restoration capacity—green coverage ratio	[10,45,251]	✓	✓	✓	✓	✓	Retained
58	Social security	[45]	✓	✓	✓	✓	✓	Retained
59	Gas supply pipeline	[10]	✓	✓	✓	✓	✓	Retained
60	Drainage pipeline	[10,41]	✓	✓	✓	✓	✓	Retained
61	Internet users	[10,251]	✓	✓	✓	✓	✓	Retained
62	Mobile phone users	[41,251,253]	✓	✓	✓	✓	✓	Retained
63	Medical insurance coverage	[251,253]	✓	✓	✓	✓	✓	Retained
64	Unemployment insurance coverage	[251]	✗	✓	✓	✓	✗	Deleted

and

Table A3. Aggregated index selection for CV.

No.	Primary Index	Ref.	S	U	F	O	R	Result
1	Human health impact—the degree to which a disruption in the system might feasibly harm the health of employees or the public	[52]	✓	✓	✓	✓	✓	Retained
2	Electricity consumption per capita	[112]	✓	✓	✓	✓	✓	Retained
3	Climate resilience	[120]	✗	✓	✗	✗	✓	Deleted
4	Noise pollution	[120]	✗	✓	✗	✓	✓	Deleted
5	Aesthetic/functional impact	[120]	✗	✓	✗	✗	✓	Deleted
6	Mortality and morbidity due to air pollution	[120]	✗	✓	✗	✗	✓	Deleted
7	Accident fatalities	[120]	✗	✓	✓	✓	✓	Deleted
8	Ecosystem damages due to acidification and eutrophication caused by pollution from electricity production	[120]	✗	✓	✗	✗	✓	Deleted
9	Seismic risk	[45]	✗	✓	✓	✓	✓	Deleted
10	Flood risk	[45,122]	✗	✓	✓	✓	✓	Deleted
11	Meteorological hazard	[45]	✗	✓	✓	✓	✓	Deleted
12	Geological hazard risk	[45]	✗	✓	✓	✓	✓	Deleted
13	Hazard of industrial disaster	[45]	✗	✓	✗	✗	✓	Deleted
14	Population density	[45,251]	✓	✓	✓	✓	✓	Retained
15	Demographic structure	[45,251,253]	✓	✓	✓	✓	✓	Retained
16	Demographic change	[45,251]	✓	✓	✓	✓	✓	Retained
17	Distribution of important buildings	[45]	✓	✓	✓	✓	✓	Retained
18	GDP per capita	[10,45,251]	✓	✓	✓	✓	✓	Retained
19	Affected elements and components	[110]	✗	✓	✓	✓	✓	Deleted
20	Number of households affected	[110]	✗	✓	✓	✓	✓	Deleted

(Appendix A). The deletion of each index was related to its original meaning as it underwent the index selection process. There were two main reasons for deleting indices. Unicity is part of the reason, as most scholars generally attach great importance to output of renewable energy, application of distributed energy system, energy sources, energy diversity, etc. Feasibility was the main reason, because some indices were difficult to quantify, some were not suitable for too many measurement objects because the quantization process was too tedious or the quantization workload was large, and some did not apply to China’s actual situation. Therefore, 113 indices were finally retained for the UESR assessment index framework, as shown in Figure 3.

The selected 113 indices are quantitatively measured and equally weighted, and they can be assigned differently to satisfy various assessment purposes through a dialogue process between decision makers and stakeholders.

2.4. Normalization of the Indices and Calculation of UESR

Indicators were divided into positive and negative indicators according to their supporting or inhibiting effects on resilience [50]. The higher the negative indicators, the lower the corresponding criteria and resilience, such as the share of imported electricity, daily water consumption per capita, and railway access index. All other indicators are positive. Min–max normalization is used to process the original data as follows.

For positive indicators:

$$y_{ij}=\frac{x_{ij}-min\left(x_{ij}\right)}{max\left(x_{ij}\right)-min\left(x_{ij}\right)}$$

(3)

For negative indicators:

$$y_{ij}=\frac{max\left(x_{ij}\right)-x_{ij}}{max\left(x_{ij}\right)-min\left(x_{ij}\right)}$$

(4)

where $x_{ij}$, $y_{ij}$ represent the original and normalized data, respectively; $max\left(x_{ij}\right)$ is the maximum value of this indicator; and $min\left(x_{ij}\right)$ is the minimum value of this indicator;

$$CI={\displaystyle \sum }_{i=1}^n{I}_i\times {\omega }_i$$

(5)

$$CE={\displaystyle \sum }_{i=1}^n{E}_i\times {\omega }_i$$

(6)

where $I_i$ and $E_i$ represent the normalized value of index $i$ for CI and CE, respectively, and ${\omega }_i$ represents the weight of index $i$. According to the universal risk evaluation model, CV is determined as follows [47]:

$$V^2={\displaystyle \sum }_{i=1}^n{V}_i\times {\omega }_i$$

(7)

where $V_i$ represents the normalized value of index $i$ for city vulnerability or energy vulnerability. Then, resilience is determined as:

$$Resilience=\frac{\left({\sum }_{i=1}^n{E}_i\times {\omega }_i\right)\times \left({\sum }_{i=1}^n{I}_i\times {\omega }_i+{\sum }_{i=1}^n{E}_i\times {\omega }_i\right)}{{({\sum }_{i=1}^n{V}_i\times {\omega }_i)}^{\frac{1}{2}}}$$

(8)

Based on data survey, statistics, and analysis, the UESR of a city can be obtained by substituting these 113 parameters into Equation (8).

3. Results

The energy resilience of 309 Chinese cities is shown in Figure 4. The entire country was divided into four regions according to the National Bureau of Statistics of China [51], namely, the western region (107 cities), the central region (81 cities), the eastern region (87 cities), and the northeastern region (34 cities). Several cities were more resilient than the surrounding areas. There were four types for different reasons. First, provincial capital cities generally had better political resources, management levels, and economic development advantages compared with their surrounding cities and thus had stronger comprehensive city strength and better performance in CI and CE. This applied to Changchun of Jilin, Harbin of Heilongjiang, Taiyuan of Shanxi, Kunming of Yunnan, and Fuzhou of Fujian. Second, Zhangjiakou of Hebei is close to the capital, Beijing, and serves as an important satellite city. It is located in the coal transport corridor, has abundant wind energy resources, has developed a number of microgrid projects, and has few energy-consuming industries, all of which made it a relatively energy-resilient city. Third, Zhuhai of Guangdong has relatively small population density, industrial density, and economic size in Guangdong province, resulting in low CV. As CE and CI were not significantly different, Zhuhai’s resilience value was higher. Fourth, Shenzhen of Guangdong was more resilient within the province because of its better performance in energy diversity, microgrid projects, and development of nuclear power.

3.1. Regional Level

In general, a majority of the 309 cities, especially those in the northeastern and western regions, had relatively low energy resilience. In contrast, UESR in the eastern region was generally higher. The average resilience (R) result of the eastern region was more than twice that of the northeastern and western regions. The resilience variance (S²) of the eastern region was nearly an order of magnitude higher than that of the other three regions. The most evenly distributed cities were located in the central region. The differences in CV among the four regions were not significant in terms of average, maximum, minimum, or variance, with the eastern region only slightly higher than the other three regions. From the perspective of CE, there were no obvious distribution characteristics. The eastern region had the highest average. The central region had the lowest variance. The situations of the western and northeastern regions were similar. The highest CI average occurred in the eastern region as well. The statistics of the evaluation results are shown in

Table 1. Statistics of the evaluation results.

Region	Resilience	S2	CV	CE	CI
Nationwide	0.32	0.022	0.36	0.20	0.36
Western	0.24	0.0053	0.35	0.16	0.34
Central	0.28	0.0028	0.35	0.18	0.36
Eastern	0.50	0.022	0.38	0.28	0.40
Northeastern	0.22	0.0035	0.37	0.16	0.33

. The detailed data and evaluation results can be seen in Tables S1–S4 of the Supplementary Materials.

3.2. Provincial Level

Among the evaluated 27 provinces/autonomous regions:

• The highest average resilience occurred in Shandong (0.69), and the lowest, in Tibet (0.039). The distribution of resilience development was most balanced in Qinghai, with the lowest variance (0.000050) and the smallest range (0.020), and least balanced in Yunnan, with the second-highest variance (0.0046) and the largest range (0.26).
• The highest average CV occurred in Shandong (0.40), and the lowest, in Guizhou (0.32). The distribution of CV was most balanced in Tibet, with the lowest variance (0.000098) and the smallest range (0.028), and least balanced in Guangdong, with the highest variance (0.0046) and the largest range (0.24).
• The highest average CE occurred in Shandong (0.36), and the lowest, in Tibet (0.049). The distribution of CE was most balanced in Qinghai, with the lowest variance (0.000057) and the smallest range (0.018), and least balanced in Ningxia, with the highest variance (0.0019) and the second-largest range (0.12).
• The highest average CI occurred in Jiangsu (0.41), and the lowest, in Tibet (0.26). The distribution of CI was most balanced in Hainan, with the lowest variance (0.000045) and the smallest range (0.016), and least balanced in Guangdong, with the highest variance (0.0038) and the largest range (0.25).

3.3. City Level

• Among the 309 cities, 107 (35%) had higher energy resilience than the national average, while 202 (65%) had lower energy resilience than the national average.
• The four municipalities, Tianjin, Shanghai, Chongqing, and Beijing, ranked 88th, 84th, 71st, and 48th in resilience, respectively. All municipalities were above the average level, not only for resilience but for CV, CE and CI. Beijing ranked first in CI and CV.
• The minimum, median, and maximum resilience results corresponded to Rikaze, Yingkou, and Weifang, respectively. Detailed comparisons of these three cities are shown in Figure 5 and Figure 6. The numbered acronyms on the left in Figure 6 correspond to the indices in Figure 3. The levels of the three cities’ CV varied little. Rikaze had an obvious advantage in energy vulnerability, but its city vulnerability was due mainly to a large number of civil protection units in the city, such as historic sites, temples, and repositories of ancient books, pictographs, and other cultural relics. Its city competitiveness (index Fl 13-20), including the city’s external connectivity, software and hardware environment, knowledge and information development level, and infrastructure construction, was in a disadvantageous position as well. These data were obtained from the Yearbook of China’s Cities sponsored by the Sustainable City Committee of the China Research Society of Urban Development. According to the editor, the evaluation indices mainly reflected the competitiveness of cities in transforming from quantitative growth to qualitative sustainable development. To improve the resilience of Rikaze, this sustainable competitiveness should be comprehensively considered. Additionally, the reliability of the power supply can be improved, and the line loss rate of power enterprises can be reduced. Electricity conservation could be further advocated and executed, and new energy vehicles and enhanced transportation accessibility could be promoted. In terms of energy diversity, the use of natural gas and heat supply also lagged. However, this is related to the local climate and residents’ habits and customs, which are difficult to change in the short term and require long-term adjustment and planning.
• For Yingkou, the main means of improving resilience would include promoting and practicing electricity conservation; improving the management of State Grid Liaoning Power Co., Ltd., among the major power grid companies in the country; and improving the diversity of power generation. With the current Huaneng Yingkou Thermal Power plant as the dominant plant, the city could develop microgrid projects, distributed energy systems, etc., to develop capacity other than thermal power generation.
• As the comparison of financial feasibility was based on provincial data, Weifang’s advantages in both the fixed assets and current assets of the energy industry benefit from Shandong’s advantages among provinces, as do the decommissioning of thermal power units and the achievement of energy savings. In addition, according to the China Electric Power Industry Annual Development Report, State Grid Shandong Power Co., Ltd., has relatively better comprehensive management on the supply side in its industry, so cities in Shandong also scored high on this series of indices. This implies that financial and managerial resilience can be improved at the provincial level.

3.4. Regression Analysis

Since the resilience of UESs is a critical issue in the current energy transition toward the 3060 targets, it is interesting to understand the relation among a city’s energy system resilience, carbon dioxide emissions (megaton) and GDP (10¹⁰ RMB).

By the weighted least squares method (weight = 1/resid²), the following binary nonlinear regression equation is obtained, and the model fits the evaluation results well.

$${\mathrm{RESILIENCE}}_{\mathrm{i}}=-0.049111+0.177735\mathrm{CO}2{\mathrm{E}}_{\mathrm{i}}{}^{0.204}+0.045861{\mathrm{lnGDP}}_{\mathrm{i}}+{\mathrm{e}}_{\mathrm{i}}$$

(9)

$$\mathrm{t}=(705.8698***)(749.1603***)(484.5519***)$$

$$\mathrm{R\_squared}=0.9999,\mathrm{n}=309$$

where *** means at 1% significant level. The empirical results showed a positive correlation between resilience and carbon dioxide emissions, suggesting that there should be a balance among loss of resilience, reduction in carbon dioxide emissions, and increase in GDP. For an example, in Yingkou, a reduction in carbon dioxide emissions of one million tons would sacrifice resilience by 0.0073 and drop the city 12 places in the ranking, and an increase in GDP of 22,949.87 million RMB would enhance resilience to maintain the original position. Therefore, in the process of achieving the 3060 targets, to ensure the safety and sustainability of a city and allow its resilience to fluctuate within reasonable limits, how to appropriately allocate the carbon dioxide emission reduction quota to each city is critical. Based on the evaluation framework of this study, the options for both reducing emissions and enhancing resilience vary from city to city. Generally, feasible alternatives include advancing the financial feasibility of the energy sector, promoting, and practicing energy conservation, and improving the management of power enterprises.

4. Conclusions

With the ambitious 3060 targets, China is looking forward to an unprecedented energy transition. As a core part of energy transition and sustainability, resilience must be given serious attention, especially when extreme events have occurred more frequently in recent years.

To this end, this paper implemented a nationwide comprehensive assessment of the resilience of UESs in China. The results showed that the current capabilities of Chinese UESs to handle exogenous extreme events are very uneven, and that cities in the eastern region generally have higher resilience than those in other regions. The minimum, median, and maximum UESR results corresponded to Rikaze, Yingkou, and Weifang, respectively. Regression analysis of 309 cities’ resilience evaluation results showed a positive correlation among UESR, carbon dioxide emissions, and GDP. When the details of this evaluation are combined and the differences lucubrated at the urban/provincial levels, each city should develop a tailored plan to reduce carbon emissions, ensure reasonable changes in UESR, and flexibly utilize economic instruments.

The aim of this study was to establish a benchmark to understand the complicated correlations and challenges of energy transition. The findings of this study may assist municipal and provincial decision makers with unique insights for enhancing overall UESR. Moreover, continual assessments of the UESR of these cities in future years could offer policy makers much more valuable information on energy transition and urban development.

The proposed indicators mainly suit China’s current reality, and different, specific indices should be adopted when the assessments are applied to cities in other countries. The results do not contain value or other judgments.