Comprehensive Success Evaluation Framework for Socio-Natural Disaster Recovery Projects

Abstract

There is an ample amount of studies attempting to evaluate the success of recovery projects. However, they mostly focused on unilateral success indicators and ignored the continual lifespan and various stakeholders. This study has focused on addressing these obstacles by proposing a comprehensive success evaluation framework for recovery projects following socio-natural disasters through identifying the success dimensions and their effectual parameters. Triangulation of the results of quantitative and qualitative methods and the logic of qualitative comparative analysis were used to answer the research questions. System thinking and the concept of resiliency and sustainability, known as key performance indicators, structured the success measurements. The effectual elements on the success, critical success factors, were grouped according to the continuous life-cycle of these projects. Finally, the framework for success evaluation of socio-natural disaster recovery projects illustrates interactions among the deducted critical success factors and key performance indicators. The proposed framework may serve as more efficient guidelines to set and follow the recovery goals, comprehensively considering a wide range of stakeholders and long-term recovery. The results of this study can be subject to further research by using different methods to enhance the comprehensivity of the framework. The authors will conduct further research to verify the suggested framework by implementing case studies.

1. Introduction

From 2006 to 2016, the world has experienced more than four thousand natural hazards representing at least one natural hazard that has threatened human societies every day [1]. A recent report by the Center for Research on Epidemiology of Disasters (CRED) reveals that in 2020, the world experienced 389 natural hazards, which have shown an increase in number and higher economic loss compared to 2000–2019 [2]. However, it is essential to differentiate “natural hazards” and “disasters” as the latter results from the exposure of the former in a vulnerable society. According to CRED, a disaster is “a situation or event that overwhelms local capacity, necessitating a request at the national or international level for external assistance; an unforeseen and often sudden event that causes great damage, destruction and human suffering” [3]. Natural hazards such as earthquakes, floods and drought suddenly happen without adequate warning can influence human physical (injuries, deaths) or mental (psychological issues) well-being along with the loss of infrastructures and services, economic loss, environmental, organizational, cultural impacts and so forth [4,5]. Although the term “natural disasters” has been used frequently in the recent literature, declaring that all types of disasters originate from human activities, this research uses “socio-natural disasters”, which both refer to humanity role in the creation of socio-natural disasters and differentiates this group of catastrophe with the others [4,6].

The complex and dynamic nature of Socio-Natural Disaster Recovery (SNDR) projects, besides the involvement of various stakeholders, require a dynamic response and timely updates to answer upcoming changes [7]. The greater the ability to manage the complexity in an SNDR project, the higher the project’s chance of success [8]. However, there is no agreement upon the recovery project’s ending point [9,10,11,12]. While traditional project management usually evaluates projects by examining the project’s outcome (time, cost, and quality) rather than the process [13], SNDR projects should be assessed throughout more holistic and flexible sets of indicators to answer the complexity and the long-term requirements [14,15].

As witnessed in reality, a significant number of recovery projects are reported as successfully conducted as they have been assessed after the construction phase, overemphasizing the tangible outcomes and neglecting the long-term measurements such as public satisfaction [16,17]. A few illustrative examples are given here. Although the Bam recovery project, Iran, 2003, has been carefully monitored and documented by the World Bank, this mid-term evaluation did not assess long-term recovery results and overemphasized the budget management and physical recovery [18]. On the other hand, some published studies discussing the long-term recovery of Bam’s neighbourhoods have shown the low people satisfaction of the recovery results, which varied greatly across the location of their houses [19]. Moreover, the public was less content with the short-term recovery response due to the shortage of shelters and long-living duration in transitional houses [20]. The recovery project after the Wenchuan earthquake, China, 2008 is another illustration representing the various indicators applied for the project’s success measuring. The reconstruction has been expedited to be completed earlier than the planned schedule, and most survivors were satisfied with the outcomes [21]. However, long-term recovery has not been properly addressed, causing unaffordable infrastructures for low-income households [22] and the shut down of small businesses a few years after the project’s completion [23].

To tackle the difficulties mentioned above, the successful recovery and its measurements should be defined. While the scholars have put various cycles and steps forward, measurements for evaluating SNDR project’s success should consider both tangible and intangible outcomes varying from the number of houses and reconstructed infrastructures to economic growth and enhanced social welfare [24]. Moreover, evaluating SNDR project’s success depends on stakeholders perspectives that can differ from local citizens to governments as their needs vary [25]. Critical Success Factors (CSFs) performing as the project’s influential factors have been widely studied [10,26]. Focusing on the reconstruction phase, one study categorized CSFs into three groups based on the reconstruction phases, planning, design, and construction [27], while another study sorted out them based on the reconstruction project’s types varying from housing to public projects [10]. However, both previous studies simplified the recovery after a disaster by emphasizing the reconstruction phase and neglecting long-term recovery.

CSFs have varied significantly across the studies. Although Platt’s study has shown a relationship between the size of disaster (death × economic loss/Gross Domestic Product (Gross Domestic Product, GDP)) and recovery speed, reconstruction duration that evaluates physical recovery cannot suitably measure the long-term recovery [28]. Recovery evaluation is affected by the simultaneous occurrence of effectual factors [29]. A detailed study on the recovery after the Bam earthquake has suggested that the success of the recovery projects heavily relies on the well-defined relationships between the involved organizations in recovery projects [30]. Although a clear organizational framework plays a vital role in the success of the recovery project, as stated by the last study, the conclusions have been obtained from one case as the Bam reconstruction project has mostly suffered from the lack of an integrated recovery policy framework. As He described, governments’ decisions directly impact the success of recovery plans [31]. Recovery decisions taken by governments should consider people’s needs and culture and try to avoid the marginalization of special groups [7,11,32,33]. Housing, as one of the most well-studied topics, must address the needs and culture of locals [5,33,34,35,36]. Furthermore, the designers need to spend enough time to learn about the hit area culture and people lifestyle [12,13,32]. Housing reconstruction can be conducted through various methods such as owner-driven reconstruction, donor-driven reconstruction, and contractor-driven reconstruction [7,8,11]. Owner-driven reconstruction enables the residents to build the houses by themselves [13] by using authorized materials, techniques, and consultations for residents [13,37]. Some scholars believe that owner-driven housing may lead to successful recovery projects [38,39]. This method showed more effectiveness in rural areas [7]; however, the safety of owner-built houses is a critical issue because of the owners’ limited knowledge about safe construction and their financial capacity [38].

On the other hand, regarding political strategies in the SNDR projects, the terms “Top-down approach” and “Bottom-up approach” are prevalently seen in disaster recovery documents [9,28,33]. The top-down method has been identified as a formal concentrated way focusing on the government benefits and neglecting people’s real needs; however, the bottom-up approach pays attention to empowering disaster victims [40]. It is necessary to consider that effective communication between governments and residents, measured by public anxiety reduction and increase in public trust, can lead to approved planning, improved disaster recovery, better information flow, and improvement of trust and cohesion among stakeholders [25,41,42]. Strong social networks and improved communication, which have been nominated as one of the disaster planning goals [43], had great impacts on the success of post-disaster recovery in the Sri Lanka earthquake, 2004 [44] and Hanshin-Awaji earthquake, Japan, 1995 [16].

The unilateral perspective toward success evaluation is also prevalent in practical cases. A study by Gapanati and Mukherji showed that India’s recovery success evaluation heavily depended on the speed of financial assistance distribution [11]. This evaluation criterion imposed by inflexible timelines for project completion based on the World Bank regulations has been observed as an obstacle to achieving intangible recovery outcomes. A theoretical framework for successful management of post-disaster recovery projects has been proposed by Meding et al., which attempts to link disaster management, strategic management, and project management concepts [45]. This framework applies to a wide range of projects; however, there is still room for development as the proposed model focuses on physical recovery and neglects the pre-disaster stage. Additionally, the framework built based on the surveys in two countries may not be easily generalized, which motivates the need for an alternative approach. A post-implementation evaluation of the permanent housing project from the residents perspective in the Eastern Anatolia Region of Turkey proposed by Yilmaz et al., is a well-structured framework for success evaluation in a specific topic [46]. The proposed framwork aimed at project evaluation after implementation and has specificly focused on people satisfaction of the newly built houses. The research has applied great literature review and field work; however, the extracted indicators can limtedly appraise one aspect of multidimentional topics in the SNDR projects. Even though the last-mentioned studies attempted to offer solutions for success evaluation in SNDR, this paper addresses the following questions which were somehow neglected in the literature. How can we comprehensively evaluate the success of SNDR projects? Moreover, which factors can be influential on the success of SNDR projects?

This study attempts to answer the questions mentioned above by proposing a success evaluation framework consisting of a clear definition of the successful recovery measurements and how influential factors can be used to manage SNDR projects. Triangulation of the results obtained from critical literature review, factors interrelationship analysis, and Delphi survey help the authors to identify causes of success and measurements of successful SNDR projects. The interactions between the last two mentioned groups of elements have then been assessed under the logic of Qualitative Comparative Analysis (QCA), resulting in the comprehensive framework for the success evaluation of SNDR projects. The research applies systematic consideration to place the parameters in the right place. Additionally, the continual lifespan enabled the evaluation framework to assess the success in different life-cycle stages. The success definition is also adjustable based on each project’s requirements and features, providing the opportunity to be applied in various types of disaster recovery projects. The study’s framework is open to further research and can be examined by its application in cases studies with distinctive characteristics.

2. Materials and Methods

As mentioned by Yilmaz et al., only the establishment of CSFs is not sufficient to form success evaluation strategies. However, the identification of CSFs may lead to establishing measurements evaluating the project’s success [46]. This paper has modified the methodology applied by the last-mentioned research and passed several steps to present the comprehensive success evaluation framework for SNDR projects. As seen in Figure 1 this study’s methodology applies triangulation of the results obtained from the three methods to propose a comprehensive success evaluation framework. QCA, seeing the combination of the effectual factors and their interrelationships on the success, has been chosen to assist the research in establishing the success evaluation framework.

2.1. Critical Literature Review

The research has been initiated by exploring academic resources for selecting the most suitable English written references published after 2003. As numerous publications have been found when searching “disaster recovery “AND “natural disasters” in academic search engines such as Science Direct, ProQuest, and Springer, the authors have used the critical literature review. This method, which extensively and critically evaluates the resources, does not simply focus on the narration of the literature. The result obtained by this methodology might be the starting point for the rest of the research as it produces a hypothesis or a conceptual framework [47]. Therefore, “post-natural disaster recovery” AND “post-disaster reconstruction” refined the results, and the authors selectively chose the most relevant publications. The authors tried to select the resources, which discuss different types of natural hazards and, at the same time, include global cases. Finally, the study comprised 135 resources from 2003–2021, selected from different countries and a wide range of disaster types. “International Journal of Disaster Risk Reduction”, “International Journal of Disaster Resilience in the Built Environment”, “ Open House International”, “ Habitat Internationa”, “ World Development”, “Natural Hazards”, “Natural Hazards Review”, and “Disaster Prevention and Management” own the highest number of the resources in this study. Additionally, 40 per cent of the resources have been chosen from books, dissertations, and conference papers.

2.2. System Thinking and Factors Interrelationship Analysis

The potential indicators have been extracted throughout a detailed review of the selected literature and were codified based on system thinking in NVivo2020. Traditional thinking methods break a project into parts and neglect interrelationships, while system thinking attempts to track patterns of changes [48]. Application of system thinking assisted researchers in answering the complexity in SNDR projects triggered by the engagement of various subjects. As shown by the context analysis, the use of system thinking in SNDR projects is prevalent [36,49]. This study set social, environmental, economic, government, and project management as the sub-systems of SNDR projects. Each sub-system has then been divided into groups consisting of several factors. The codification has been conducted through a double round process to ensure that the assigned sub-systems and groups cover all the factors. Present-day science indicates that SNDR projects have a complex nature, requiring balancing pre-disaster preparedness and post-disaster relief. Therefore, a continuous life-cycle for the SNDR project, consisting of five phases, pre-disaster stage, post-disaster immediate response, planning and design, procurement, construction and completion, and continual development, was applied to evaluate the factors interrelationships. Two rounds of factor-interrelationship analysis were conducted to determine the extracted influential factors’ accuracy and redundancy, and relationships between the factor have been illustrated [50]. The frequency of the extracted factors throughout critical literarure review and the total number of each factor’s links according to the factors’ interrelationship analysis has then been calculated.

2.3. Delphi Survey

Delphi is a group communication method used to solve complex issues by gaining the experts’ opinion throughout rounds of questionnaires and sharing the last rounds’ feedback anonymously with those experts [51]. A two rounded Delphi method was selected for this research because it is a rigorous method for obtaining the opinion of a geographically dispersed group of experts and a practical solution to save time [52,53]. The identified success factors can be validated and refined by taking advantage of unbiased judgments because of anonymous panellists. Additionally, sharing last rounds results with the panellists provides the research with an excellent opportunity to reach a reliable consensus [54]. A panel consisting of ten selected experts, who have (1) a PhD degree in the relevant fields (2) at least one academic journal paper about SNDR projects, and (3) more than three years experience in disaster-related fields, have ranked the importance of the factors by a five-degree Likert scale. The panellists are a selection of well-known researchers in disaster recovery, and some have experience working in another country. Moreover, their diversified research area (from planning and architecture to management) ensures the reliability of their judgment. In general, the panellists far exceeded these minimum requirements. The demographic information of the panel members has been illustrated in

Table 1. Demographic information of the panel members.

Panellist Age	Panellist Study Major	Panellist Working Experience	Country
30–39	Architecture and disaster studies	10–15 years	Iran, Canada
More than 50	Disaster management	More than 16 years	Iran, Australia
30–39	Built environment planning	5–9 years	China, Canada
30–39	Disaster management	5–9 years	Iran
30–39	Disaster management	10–15 years	Iran
30–39	Civil engineer	4 years	USA
More than 50	Earth and Environmental Sciences	More than 16 years	UK
Less than 30	Disaster management	4 years	UK, China
40–49	Architecture and disaster studies	5–9 years	Iran, Japan
More than 50	Housing reconstruction after disasters	10–15 years	Sweden

.

The Delphi questionnaire has been aligned based on the applied life-cycle and the identified sub-systems introduced in Section 2.2. Additionally, the panellists have been asked to modify the factors, and their anonymous comments have been attached to the second round of the questionnaire alongside the first round’s results. The consensus emerges if the mean value is equal to or more than four OR if more than 75% of respondents give the factor the same rating. The indicators, which met the defined criteria for consensus, have been put aside and have not been evaluated later.

2.4. Triangulation; Comparison of the Last Methods Results

The authors applied triangulation methodology to gain refined perspectives of the influential factors in SNDR project’s success, which are well-known for their intricate nature. This methodology benefits from applying multiple research methods at the same time to understand the construct [55]. Reliability check of different resources and accuracy test of the researcher’s interpretation are some of the advantages of this method resulting from the simultaneous use of qualitative and quantitative methods [56]. However, the application of triangulation can be costly and time-consuming [55]. In this research, the authors have chosen the Delphi method to analyze the research quantitively, while critical literature review alongside the factors interrelationship analysis qualitatively evaluates the data. Later, triangulation categorized the results of the last-mentioned stages and set acceptance criteria for each method to select the top-ranked factors.

2.5. Qualitative Comparative Analysis (QCA)

The last step results have been used as input for further interpretation to structure the success evaluation framework. QCA attempts to compare cases were first introduced by Charles Ragin in 1987 [57]. QCA, which uses advantages of traditional qualitative and quantitative research methods within-case comparison, has been nominated as the third way to do social research [58]. Generally, QCA tries to define which factors/combination of factors might affect the results [58]. The application of QCA in SNDR projects may enable the researchers to identify where, when, and under what circumstances the project can be managed successfully. However, three necessary steps should be passed before the application of QCA. First, outcome(s) should be determined. The outcome is an objective or result (desirable or undesirable) of the procedure. Second, conditions, the characteristics affecting the outcomes, should be defined [29]. The term condition is somewhat similar to the variable used in quantitative methods [29,57]. Finally, cases should be carefully selected to include different variety of outcomes within cases. Random case selection commonly happens in large-N research, may bring heterogeneous cases into the sample and should be avoided in QCA [59]. Subsequently, the required data for the conditions and outcome(s) will be collected. Through the process of calibration and sub-set analysis, combinations of conditions leading to the outcome(s) can be analyzed [29].

Note that this paper delves into recognizing two fundamental elements of QCA project’s, conditions and outcomes to answer the two research questions. Outcomes have been supposed as SNDR project’s success measurements, named as Key Performance Indicators (KPIs), while conditions refer to causes of success or CSFs. Furthermore, two sets of tables, conditions and outcomes were generated based on the guidelines published by Befani [57]. This step’s results are the foundation of the rest of the research to conduct a qualitative comparative analysis of the selected cases. The case selection and analysis of data using QCA are not considered here and are left as an area for future study.

3. Findings

The identified factors throughout the critical literature review and factors interrelationship analysis have been listed in

Table 2. Influential factors resulted from literature review and factors interrelationship analysis adapted/reprinted from ref. [50].

Life-Cycle Stage	Code	Factors	Code	Factors
Pre-disaster stage	PRE1	recognition of residents and business information	PRE6	emergency management plan
	PRE2	special fund and resources (SFR) for disaster	PRE7	emergency response training
	PRE3	climate monitoring	PRE8	community participation
	PRE 4&5	vulnerability protection of the built environment
Post-disaster immediate response stage	Post 1	people’s basic needs (food, sanitation, and security)	Post 9	debris cleaning
	Post 2	temporary school	Post 10	consideration of secondary hazards
	Post 3	rescue and medical aid	Post 11	immediate leadership and coordination
	Post 4	psychological support	Post 12	immediate infrastructure restoration
	Post 5	criminal behaviour prevention	Post 13	assistance from other countries or areas
	Post 6	residents’ social network and trust	Post 14	damage assessment
	Post 7	assistance from NGOs	Post 15	site investigation
	Post 8	quick and fair allocation of SFR	Post 16	fast provision of safe shelters
Planning and design Stage	Pla & de 1	consideration of local culture	Pla & de 8	integrated recovery plan (simplification of reconstruction procedure)
	Pla & de 2	consideration of local climate	Pla & de 9	designers’ professionalism
	Pla & de 3	consideration of community needs	Pla & de 10	site selection
	Pla & de 4	budget for reconstruction	Pla & de 11	reasonable housing design
	Pla & de 5	environment protection plan	Pla & de 12	resilient infrastructure design
	Pla & de 6	property right protection	Pla & de 13	enforcement of standards
	Pla & de 7	pre-established plans revision
Procurement, construction, and completion stage	Pro & con 1	use of local labours	Pro & con 9	availability of construction materials
	Pro & con 2	use of local materials	Pro & con 10	contractors competence
	Pro & con 3	use of local construction methods	Pro & con 11	skilful labours
	Pro & con 4	cost control	Pro & con 12	logistic management
	Pro & con 5	waste management	Pro & con 13	safety control
	Pro & con 6	use of recyclable materials	Pro & con 14	quality control
	Pro & con 7	supervision on reconstruction	Pro & con 15	on-time completion and delivery
	Pro & con 8	rapid construction method
Continual development stage	ConDev 1	development and recovery of livelihood	ConDev 6	information management system
	ConDev 2	improvement of public capabilities (awareness) to cope with natural hazards	ConDev 7	hazard warning and protection systems
	ConDev 3	local business recovery	ConDev 8	house condition evaluation after PNDR
	ConDev 4	sustainable environment	ConDev 9	infrastructure condition evaluation after PNDR
	ConDev 5	updated regulations and standards based on lessons learnt

. Later, highly-ranked factors from each method (critical review, factors interrelationship analysis, and Delphi) have been identified to be compared by triangulation. To choose the top factors in each phase, the authors establish three criteria. Twenty-five top influential factors have been listed based on the literature’s frequency analysis, and the parameters with equal or more than four links formed the second part. Finally, the factors that have reached consensus in the Delphi survey have been selected as the third part of this comparison. Figure 2 shows the scenario explained above.

Seven factors have appeared in all the methods utilized in triangulation. “Vulnerability protection of the built environment”, “emergency management plan”, “quick and fair allocation of special fund and resources”, “immediate leadership and coordination”, “integrated recovery plan (simplification of reconstruction procedure)”, “site selection”, and “reasonable housing design” have gained consensus from all the applied triangulation methods (named as 3/3 in Figure 2). However, the rest of the influential factors can be grouped in seven clusters; “simultaneously ranked by literature review and factors interrelationship analysis”, “simultaneously ranked by literature review and Delphi”, “simultaneously ranked by Delphi and factors interrelationship analysis”, “ranked by literature review”, “ranked by factors’ interrelationship analysis”, “ranked by Delphi”, and “non-ranked factors”.

The criteria voted in two sections simultaneously (named 2/3 in Figure 2) show relatively higher importance than those selected by only one method (named 1/3 in Figure 2). Similarly, the non-ranked factors (named 0/3 in Figure 2) represent the lowest degree of importance among all the identified effectual factors. This analysis is the basis for creating measurable success factors and the definition of successful SNDR projects.

4. Discussion

This study concentrates on the success dimensions and their effectual factors in SNDR projects. The vital parameters affecting SNDR project’s success that are regarded as CSFs are called “conditions”. Meanwhile, a successful SNDR project’s ultimate goal(s) (KPIs) can be defined as “outcome(s)”.

4.1. Structuring Success Evaluation Framework

According to the triangulation comparison results, this section structured the KPIs and CSFs to establish a success evaluation framework of SNDR projects. A time-based life-cycle, proposed by Bahmani and Zhang, has been applied to form the CSFs [50]. The CSFs have been grouped into short-term, mid-term, and long-term periods. As most of the current literature pointed out the importance of alleviating vulnerabilities and systems quick ability to bounce back to normalcy after disasters [11,60,61], the definition of sustainability and resiliency alongside system thinking set KPIs.

Table 3. Table of KPIs (outcomes).

Systematic Outcomes	No.	KPIs (Outcomes)	Indices
Social recovery	A1	Resilient Society	Social connections
			Phycological support (suicide, mental disorders report)
			Safety (criminal behaviour)
			Life satisfaction
			Satisfaction of recovery process
			Population growth
			Equity among population
Environment recovery	A2	Sustainable and resilient built environment	Safety of construction methods and technologies
			Homeless people
			Unoccupied houses
			Population per capita in houses
			Hygiene water accessibility
			Electricity accessibility
			Roads improvement
			Public transportation
			Number of schools, hospitals
			Evacuation facilities
	A3	Sustainable natural environment
			Environmental-friendly construction
Economy recovery	A4	Resilient economy	Number of new businesses initiation or restoration of old ones
			GDP growth rate
			Employment growth rate
			Household income growth rate
Policy recovery	A5	Sustainable policy development	Simplified procedures
			Regular revision of plans
			Simultaneous consideration of pre-and post-disaster plans

and

Table 4. Table of CSFs (conditions).

Time-Based Framework	No.	CSFs (Conditions)	Indices
Short-term	a1	Sufficient answer to people’s basic needs *	Food availability
			Sanitation level
			Starting time for rescue and site investigation
			Beneficiary selection
			Sufficient and on-time budget allocation
			Immediate leadership & coordination
			NGOs assistance
			Other areas assistance
	a2	Availability of shelters and schools	Shelter availability
			School availability
	a3	Immediaterestoration of environment and infrastructures	Damage assessment
			Debris cleaning
			Climate monitoring
Mid-term	a4	Reasonable housing design **	Consideration of local culture
			Consideration of local climate
			Consideration of community needs
			Use of local construction methods
			Professionalism of designers
			Site selection (distance to facilities and safety)
			Consideration of standards
	a5	Resilient infrastructures design	Consideration of local climate
			Professionalism of designer
			Consideration of standards
	a6	Project management ***	Be on planned budget
			Immediate leadership and coordination
			On-time completion and delivery
			Quality control
			Supervision on reconstruction
			Materials price control
			Labour force’s price control
			Contractors competence
			Availability of skilful labour force
			Logistic management
			Rapid construction methods
			Availability of materials
Long-term	a7	Improvement of public capabilities	Emergency response training
			Safe construction training
			Livelihood development
	a8	Community engagement level	Decision-making
			Planning
			Construction
	a9	Local economy improvement	Recovery/development plans for local business
			Use of local materials and labour force
	a10	Integrated recovery policies	Pre-established plans
			Emergency management plans
			Updated regulations and standards based on lessons learned
			Enforcement of standards
			Property right legislation
	a11	Vulnerability protection of the built environment	Regular maintenance
			Information management system
			Hazard warning and protection systems
			Consideration of secondary hazards

* Sufficient answer to people’s basic needs covers emergency needs and fund and assistance management. ** Reasonable housing design includes indicators covering community and technical design. *** Project management consists of planning and construction management.

separately list the KPIs and CSFs. As shown in those tables, each factor is accompanied by multiple indices to measure that factor. The criteria ranked by none of the triangulation methods (Figure 2) formed indices, and the rest of the factors may be used as main KPIs/CSFs. However, some factors that have gained agreement in at least one methodology have been grouped as indices of the foremost parameters to simplify the model. Additionally, according to the literature, some extra indices were added to the framework [29,32,38,62,63].

4.1.1. Key Performance Indicators; Systematic Outcomes

SNDR projects should be able to reduce the possibility of upcoming risks and susceptibilities. Additionally, developing capabilities throughout resiliency improvement should be planned [32]. While sustainability evaluates results, the process can be assessed by resiliency [38]. One more difference between resiliency and sustainability is lifetime measures; while sustainability looks forward to reducing vulnerabilities within the long-term future, resiliency focuses on systems capability to return to normalcy in a short period [64]. However, instead of oversimplifying the concept of resiliency to come back to the pre-disaster level, which may generate the previous vulnerabilities, increasing communities disaster resiliency has been suggested [65]. Recent studies have usually seen resiliency as the following phases; preparedness, absorption, recovery, and adaption [66]. Robustness and rapidity, and the system’s ability to come back to customary conditions, have also been seen as resiliency functions [67].

In this study, the holistic perspective, gained by the system thinking approach, has been combined with the definition of sustainability and resilience to set the SNDR project’s goals. A suggested framework for assessing a community’s success reaching resiliency in some studies consisted of social, economic, institutional, infrastructure, and community capacities [62]. However, in this research, system thinking logic divided the KPIs into four stages; social recovery, environment recovery, economic recovery, and policy recovery. Subsequently, each group consists of primary outcomes based on the definition of resiliency or sustainability or both. The next paragraphs provide a brief explanation of each KPIs that appeared in Table 3.

• Resilient Society: Different indicators can measure this outcome, referring to a community’s ability to return to normal situations [68]. The multiplication of the different factors can be used to evaluate social resilience, such as annual population growth, annual growth of Gross Domestic Product (GDP), and Gini index of equality [28]. Another study found sex ratio, per capita GDP, percentage of ethnic minorities, and medical facilities as the most influential characteristics on social resiliency [69]. However, in this study, some different indicators have been set. As one of the objectives of SNDR projects is restructuring community ties and social networks [9], it can be counted as an indicator of a resilient society. Although psychological support and criminal behaviour protection are non-ranked factors based on triangulation, some scholars counted them essential recovery steps [60]. Residents satisfaction of life and the recovery process, population growth and equity among the population (especially in participation and aids allocation) are the other indicators assisting analysis of this KPI.
• Sustainable and resilient built environment: This outcome can be measured considering indicators from two possible groups; housing and infrastructure. Sustainable reconstruction has often been considered as a systematic methodology for development focusing on long-term impacts. Houses, where people spend most of their time, should be sustainably designed to stand against socio-natural disasters [63]. Records of homeless people after the completion time of the recovery project and unoccupied newly-built houses alongside people’s accessibility to water and electricity in houses have set this outcome’s indicators. On the other hand, reconstruction of structures, especially infrastructures, applies modern technologies to enhance infrastructures resilient to withstand a defined level of future disasters [70,71]. The improvement or decline in public transportation, roads, and public social well-being service centres have been structured to evaluate the group of infrastructures.
• Sustainable natural environment: One aspect of sustainable building is the perception of its impacts on the natural environment [72]. As a result, the degree of environmental pollution after SNDR project’s has been established as one indicator. In addition, strategies to mitigate the negative effects of reconstruction on the natural environment [27], generally called environmentally friendly construction, measure this outcome’s score.
• Resilient economy: A resistant economy, which can tolerate fluctuations and negative disaster effects, can be assessed through several indicators. Numerous studies utilized GDP as an indicator measuring society’s resiliency [28,73]; however, to collect accurate data, the authors established other indicators. The growth rate of employment, household income, and increase/reduction in the number of new business initiations or restoration of the old business has set the indicators of this KPI.
• Sustainable policy development: As proposed by Okamoto and Ishikawa, sustainability is divided into three groups: environmental sustainability, cultural sustainability, and social sustainability [74]. Taking the concept of “social sustainability”, which expresses the restoration of decision-making, this research has modified this group’s name to “sustainable policy development” to cover wider topics. The sustainable policy can be appraised by planning strategies. Generally, these strategies should be followed in pre-disaster and post-disaster recovery plans, which might be established and utilized in a cyclic program [50]. Furthermore, the researchers have chosen simplification of the procedures, especially in permission obtainment [4,75] and consideration of planning revision [76] as the indices of this outcome.

4.1.2. Critical Success Factors (CSFs); Time-Based Conditions

The influential factors, also named CSFs, are assumed to affect recovery project’s success after socio-natural disasters. As shown in Table 4, time-based objectives are depicted based on the continuous life-cycle of SNDR projects. Each stage of this time-based table includes criteria and indices, easing the measurement of the factors. The following paragraphs briefly explain the conditions and indices in each group, referring to the time-based framework.

• Short-term conditions: Community’s urgent needs and the fast revitalization of critical infrastructures are the conditions in this group. Based on the Delphi results, people’s basic needs and rescue and medical aids have reached the panel consensus. Referring to the literature, to restore the lifeline service, people’s basic needs must be answered [77,78]. Additionally, the importance of emergency aids and recovery funding, which is counted as a potential subject affecting SNDR project’s’ success, has been focused on in some studies [78,79]. On the other hand, restoration of the suffered infrastructure and its effect on social recovery has gained attention in the literature [80]. In addition, following the Delphi results, half of the panel ranked it as a crucial influential factor. Therefore, damage assessment, debris cleaning, and climate monitoring have formed the indices of this condition.
• Mid-term conditions: CSFs related to housing, infrastructures, and project management have been grouped and structured in the mid-term division. This group, which implies the importance of design and construction phases, has been investigated by several researchers. Both literature and the triangulation results have mentioned consideration of local culture, climate, and residents needs, alongside the competence of designers and safe and acceptable site selection [27,28,81]. Similarly, resilient infrastructures design consists of technical indicators, which has gained acceptance from two applied methods in triangulation. Finally, project management, structured based on the relevant recognized indicators, consists of planning and construction control groups. Time, cost, and quality management, alongside the immediate leadership and supervision on reconstruction, are the indicators of planning control. On the other hand, the group of construction control delves into addressing on-site topics such as material and methods, labour force, and logistics management.
• Long-term conditions: The effects of improvement of social well-being, economy, environment, decision-making process and legislations, and buildings have been evaluated in this stage. People as the main consumers of the recovery outcomes should be informed and trained. The necessity of considering community involvement, especially in developing countries [9,32], has been formed the other important CSF affecting the recovery outcomes. On the other hand, as having a stable occupation is the main element of each family’s livelihood recovery, providing carriers for residents and hiring them can facilitate local economy recovery [9,42]. Integrated recovery policies consist of structuring and revising the pre-established emergency and recovery plans, establishing property right legislation, and enforcing the set standards. Vulnerability protection of the built environment, agreed by all the triangulation methods, consists of applying a hazard warning system, considering secondary hazards, using an information management system, and conducting regular maintenance.

4.1.3. Interactions between CSFs and KPIs

Although the research has already recognized the KPIs (outcomes) and CSFs (conditions), not all the conditions may directly affect the outcomes (Some may not have theoretical links) [29]. Furthermore, it is crucial to formulate the interactions between CSFs and KPIs before going further to avoid adding more complexity to the framework [82]. The data collected through context analysis is the main tool to determine the theoretical links between the conditions and outcomes. The details of the background behind the theoretical links are given in the following paragraphs.

According to the definition of successful recovery projects after socio-natural disasters, SNDR project’s success evaluation has five aspects presented in Table 3. Furthermore, the success evaluation framework of SNDR projects can be depicted as shown in Figure 3. As each KPI may be influenced by the identified conditions CSFs, five primary models for success evaluation can be considered. Each model has focused on the assessment of the project outcome based on the specific assigned KPI. Finally, those five models interactions can comprehensively assess the and long-lasting success of the SNDR projects. The presented model of the interactions between the identified KPIs and CSFs depicts the general relationship in SNDR projects regardless of the project’s content. This general framework can be specified for different types of socio-natural disasters and variable scope.

• KPI A1: As the measurement of resilient society is based on the people’s social ties, their satisfaction, and population growth, the authors considered the CSFs that can have a direct impact on reaching a resilient society. Furthermore, the CSFs related to housing, infrastructures, people empowerment, and policy-making majorly impact resilient society [83,84]. Although most CSFs can influence resilient society, there is no theoretical link between this KPI and two CSFs, project management and local economic improvement.
• KPI A2: Indicators, which appraise sustainable and resilient built environments, can be divided into two groups, housing and infrastructure conditions. Therefore, theoretical links have been found between the CSFs pointing to these groups. The role of social networks in housing recovery is widely accepted [85,86]. Public knowledge of safe construction is also another factor affecting the built environment [87]. Furthermore, except for three CSFs, a1, a2 and a6, related to emergency response and economic recovery, the rest of the conditions have been assumed to affect this outcome directly.
• KPI A3: Sustainable natural environment, which has two indicators, environmental-friendly construction and the amount of environmental pollution, have relationships with the effects of reconstruction and procurement, damage assessment and debris cleaning, buildings maintenance, and recovery plans [4,17]. Moreover, the existing literature shows that CSFs pertinent to community needs and empowerment, facility design, and economic recovery dominantly do not contribute to forming a sustainable natural environment. As a result, four of the CSFs have been recognized having a straight impact on this outcome.
• KPI A4: Resilient economy and restoration of the business have been directly linked to the fast restoration of the infrastructures, especially the transportation system [76]. Moreover, livelihood development and business recovery plans have undeniable logical effects on a resilient economy. However, there exists a wide variety of conditions that do not affect the resilient economy. In general, those CSFs discuss social and technical aspects.
• KPI A5: Apparently, people and planners are two stakeholders that can greatly impact sustainable policy development. In addition, the role of a trade-off between the project goals, time, cost, and quality, planning, and management is the other aspect influencing this KPI. However, the CSFs debating technical, economic, and environmental topics can be excluded from the list of influential CSFs on this outcome.

5. Conclusions

A considerable number of disaster recovery projects after socio-natural disasters fail to address different stakeholders needs, although they might be successful enough in some aspects. Hence, it is necessary to comprehensively identify success dimensions and the parameters affecting them to evaluate SNDR projects. However, there is a lack of clarity due to limited studies on these topics. This paper’s key idea is to use triangulation methodology and QCA to establish a practical success evaluation framework for SNDR projects by considering various systematic success dimensions and a time-based set of success causes. The following paragraphs express the study’s pathways:

• First, an extensive literature review and detailed generated codes enabled the researchers to extract and revise the effectual factors in previous studies. To structure the influential factors of the recovery projects, this research applies continuous lifespan, which tends to link the post-disaster recovery activities to the pre-disaster preparedness level. This lifespan will provide recovery projects with higher resiliency as the post-disaster stage of one catastrophe might be the pre-disaster phase of the next one. Additionally, the recovery projects have not been seen as a single unit. Their connections with the natural environment, decision-making zone, and business sector have been considered to form the systematic KPIs.
• Second, the importance of the revised influential factors has been appraised within a two-round Delphi survey. Twenty-two factors have gained consensus and the rest, except four, have been ranked as important indicators. Later, triangulation of the critical review results, factors interrelationship, and Delphi survey compared the top-listed recognized factors based on the selection criteria.
• Third, the authors grouped the prioritized elements applying the QCA method while considering the definition of resiliency and sustainability and applying the systematic groups and the continual life-cycle. Finally, eleven CSFs (conditions) and five KPIs (outcomes) have structured the fundamental interaction model between the outcomes and conditions.
• Fourth, before going through the next phases of QCA, the established framework should be simplified as not all the recognized CSFs directly affect each KPI. Furthermore, theoretical links based on the published literature assisted the authors in simplifying the connections among the outcomes and conditions.

The results have been proposed as a framework to comprehensively evaluate the success of recovery projects without overemphasizing or neglecting some aspects. As the research attempts to consider SNDR projects like a complex system including various fields, the recognized outcomes in this framework can be viewed separately or together to examine the project’s degree of success. The set of KPIs aims at different stakeholders’ perspectives toward the recovery process. Some of them point to public opinion, while others might need decision-makers participation or official report to be assessed. This research framework’s parameters can be utilized to evaluate any SNDR project. However, it is essential to modify the proposed model based on the context of cases as some of the factors might be missing or unmeasurable depending on the situation of that project.

The authors will apply this framework in multiple case studies within a specific context to show how combinations of different CSFs have facilitated reaching the specific KPIs. The proposed framework can also be used as a checklist to identify the real project process’s vital parameters. The checklist, which is a management facilitator, can measure the project management effectiveness.

The current findings suggest several avenues for future research. Socio-natural disaster recovery projects are naturally complex, and a combination of elements might generate different results. Furthermore, future research should focus on examining the various cases to define the most important influential factors. Application of other methods to analyze the instances within different contexts can optimize the results to propose project management guiltiness for SNDR projects within different regions.