Next Article in Journal
Can the Opening of High-Speed Railway Restrain Corporate Financialization?
Next Article in Special Issue
An Integrated MCDM Model for Sustainable Course Planning: An Empirical Case Study in Accounting Education
Previous Article in Journal
Finite Element Analysis of Continuously Reinforced Bonded Concrete Overlay Pavements Using the Concrete Damaged Plasticity Model
Previous Article in Special Issue
Multi-Scale Remaining Useful Life Prediction Using Long Short-Term Memory
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

Expanding Fundamental Boundaries between Resilience and Survivability in Systems Engineering: A Literature Review

Department of Mechanical and Industrial Engineering, University of Massachusetts-Lowell, Lowell, MA 01854, USA
*
Author to whom correspondence should be addressed.
Sustainability 2023, 15(6), 4811; https://doi.org/10.3390/su15064811
Submission received: 20 January 2023 / Revised: 21 February 2023 / Accepted: 3 March 2023 / Published: 8 March 2023
(This article belongs to the Special Issue Operations Research: Optimization, Resilience and Sustainability)

Abstract

:
The pressures of an everchanging world have impacted the ways in which service-based systems operate, along with their forms and boundaries. Resilience and survivability have been treated interchangeably when readying a system to remain true to its functions despite disturbances. Some situations prove the concepts may not always be the equivalent of the other, not even the consequence of the other. There may come scenarios where system components fail to adhere to certain predefined thresholds and cross a breaking point. It is therefore proposed in this study that systems can be survivable, instead of resilient, when they comply in time with the resurgence property. This property signifies the systematic behavior of overcoming a certain stagnation period and, after a time range, return as a transformed system with new functions and challenges. Through this study, it was detected that the symmetries between resilience and survivability are only superficial if systems suffer breakages after misconceiving the true causes of failure. Still, a lack of consensus among scientists and practitioners remains an issue when applying resilience and survivability in their own problems. Although workful, pushing to achieve a greater consensus would signify optimal performance in multifaceted systems involving technical, social, and economic challenges.

1. Introduction

Resilience and survivability engineering are two domains addressing system infrastructure, operational sustainability, and risk management. These concepts have gained more pertinence today as the world faces economic and political instabilities triggered by war, climate change, inflation, collective distress, and the aftermaths of the COVID-19 pandemic [1,2]. Several organizations such as hospitals, courts, businesses, factories, and laboratories have been solidifying their operations against internal disturbances with applied resilience or survivability, understanding that uncertainties from the outside world can have a massive influence on general performance [3,4,5,6,7,8,9,10]. Since before the onset of the pandemic, several theories have arisen to suggest alternative ways to architect systems that can withstand and even leverage adverse events [9,10,11,12,13]. Those theories have used elements already approached by earlier resilience and survivability theorists in their endeavor to provide solutions that can improve system endurance. Some have highlighted resilience as a desired behavior in the process of attaining survivability on complex ecological systems [14] and satellite systems [15].
Although the resilience and survivability paradigms have been extensively explored, there are still questions left unanswered and discoveries left for future studies [16]. There has been a slight agreement among scholars on what resilience and survivability mean in terms of universal practice. Even numerical formulations have consistently varied across fields to fit specific scenarios involving approaches built on probabilistic and deterministic models [17,18,19,20]. Both qualitative and quantitative tools have been in demand, as they assist in managing resources when learning about the needs of selected stakeholders and the system of interest. Current approaches to quantifying community-based and operational resilience can be classified as either objective or subjective assessments; the discussion of which tool should be used in which situation is still ongoing [21,22]. Special attention has been paid to strengthening our tool selection criteria because the current methods to account for resilience or survivability can be qualitative-objective, qualitative-subjective, quantitative-objective, or quantitative-subjective in nature.
Agreeing on overarching definitions for both domains would help practitioners find common ground among diverse disciplines. It would also minimize confusion when putting theoretical formulations into effect. With new complexities in systems design, more universal frameworks should be articulated to support product reliability, service delivery, and optimization [23]. However, unification among disciplines concerning resilience and survivability techniques has been deemed irrational for the required data types gauging system performance [24]. Diversity in formulation can be favorable when it unifies viewpoints to a shared goal instead of creating segregation. Ma [14] posed a few unifying definitions for ecological applications where associations with networked computer systems exist. It was proposed to view resilience as a subpart of survivability where other three vectors play a key role in formalizing a definition for survivability [14]. Other authors have addressed these concepts differently in the aim of unifying definitions. Sterbenz et al. [10] examined survivability as a subset of resilience in which challenges related to component and system failures are captured. The views from these authors openly demonstrate that there have been several attempts to deliver integral definitions in present times.
Perceptions surrounding resilience and survivability have continued to shift in non-engineering fields, widening the toolset for evaluating new scenarios. Hence, resilience and survivability have been proven to be user-specific and tied to fixed circumstances, meaning their indicators will resemble progress (or deterrence) to only one sector (or perhaps only a few members of that sector) of the industry. The same logic applies to sectors of a community. This feature poses a disadvantage in the sense that one sector of the industry can merely benchmark its resilience against its own from past experiences until new factors are incorporated in the analysis. In other words, user-specific quantifications of resilience and survivability are useful for within-sectors comparisons, but not necessarily for between-sectors comparison across time periods.
Resilience and survivability have accumulated an assortment of interpretations in the literature and in private practice. Most interpretations derive from special circumstances or assumptions surrounding an effort. An effort can intend to supply definitions, gather perspectives, put together a set of metric equations, or apply the knowledge on the topics to world problems. Considering recent events exposing systems’ vulnerability, this literature review article intends to (1) expand on the interplay between resilience and survivability by gathering views from engineering and non-engineering fields; (2) discover similarities and differences between them; and (3) underline resurgence and survivability as properties requiring more attention in value-delivering, socio-technical systems.
The rest of the article has been organized as follows: Section 2 presents the methodology for a systematic search of the literature; Section 3 presents the results of the literature search and classifies the articles according to different constructs to address the first two objectives of this article; Section 4 further addresses objective two by adding other key insights and mathematical formulations; and finally, the article concludes by proving the case that resilience and survivability should be treated differently at times.

2. Methods

A systematic literature review (SLR) was conducted to find and categorize previous works in a way that is unbiased and repeatable, as proposed by Kitchenham and Charters [25] and applied by Barbosa et al. [26]. The entire review can be summarized in three steps: planning with a selection rule logic, conducting an extraction from the literature pool, and reporting the findings [25].
Publications were retrieved from two major searchers: Google Scholar and Science Direct. The keywords associated with the search are resilience, survivability, engineering, and systems. Some other publications were found with the use of Research Rabbit to learn about references existing before and after a specific written work. As a research tool, the application facilitates the graphical connection between authors as they cite one another from diverse fields. It can also simplify the selection process by introducing popular citations and articles into the search network. Further, relying on this tool is vital to organize ideas and identify gaps between them.
The reference sorting was conducted through categorical sorting, setting two classifications in function of the COVID-19 pandemic, three main subclasses, and a selection rule. If the reference was published before the pandemic, it was sorted into the pre-pandemic category, or bucket; otherwise, it was deposited into the post-pandemic bucket (i.e., every publication between 2020 and 2022). It was assumed theories would change once the pandemic factor was included across the board. Within each bucket, three more classifications were fashioned: definition or literature review, math formulation, and industrial application. Only one bucket was left for miscellaneous current projects that contribute to the general research aims: others. The bucket classification model allowed for effective reference management and revision. As a review policy, each bucket had a maximum capacity of 60 references and a minimum of 5.
The rule dictating the bucket selection concerns the number of times a keyword appears in the abstract. To be considered a potential source, the publication had to present either the word resilience or survivability at least two times to evidence the authors’ commitment to the topic; only four references went into the source bucket without directly complying with the selection rule. Careful attention was paid to those references sharing any experience in the service sector.
Figure 1 delineates the categorical sorting mechanism linking the two main classifications and their subclasses. Figure 2 shows in greater detail the decision process which guided the bucket selection, considering the criteria below and the SLR stages. Three criteria in form of questions were used to add references:
Criterion 1: Does the reference contain either the term resilience, survivability, engineering, or systems in its title or abstract?
Criterion 2: Does the reference define or quantify resilience or survivability in a local or broad sense?
Criterion 3: How much of the reference is dedicated to explaining the importance of resilience and/or survivability in practice? At least two instances.
In this article, the definitions for resilience and survivability were crafted by tabulating views from the first ten references found in chronological order (i.e., order in which articles were added to the research buckets) and pairing other references according to their likeness to the first ten references. The pairing activity was done twice for resilience and survivability; at the same time, it allowed for the saving of similar views from other authors in the ten reviewed texts. To make this strategy less subjective, it was useful to identify some keywords that would click one definition with the next. It is important to clarify that the ten reviewed references come from the first classification labeled “definitions or literature review”, which suggests that references stored in this exclusive bucket are to be used to build up specific definitions by condensing the existent literature.
A second clarification was that definitions were paired from before the pandemic to the afterday. In other words, it is the research’s interest to see how granulated or focused resilience and survivability have become in terms of ideation and practicality. The same logic applies to the mathematical reviews for the two terms; in this case, the references looked up are those placed in the bucket themed “mathematical formulation”. When studying the math behind the concepts, the analysis went from pre- to post-pandemic publications. All references under “mathematical formulation” and “industrial application” share pairable definitions that were tabulated in accordance with the initial ten definitions.

3. Literature Search Results

The number of references included in the compendium of definitions and quantification techniques equals 132. The largest number of publications ranges from 2020 to 2022—the estimated duration of the pandemic—while the shortest concentration illustrates the insights from the past century, specifically the late nineties. Most of the references gathered demonstrate the wide exploration dedicated to resilience as a topic of general interest over the last twenty-five years, but only 3% managed to insert survivability into their studies. Even lower is the percentage of publications comparing the two subjects in a single study since 2010.
From the initial stages of the categorical sort, there were some frequencies suggesting a constant split between those articles centered in defining and those in applying. Within a sample of 50 references, a 23% chance of encountering applications that favor organizational resilience and welfare was observed. The lowest chance within that sample—nearly 5%—was concerned with some applications to ensure resilience in airports. Interestingly, in those 50 references, 56% forwarded frameworks with broad definitions and, within that 56%, there was a 57.14% chance that one reference was addressing the survivability conundrum. If considering only resilience, that probability drops to 42.86%. This detail reveals there are still sectors trying to outline what survivability means to them within their own contexts, preferences, and needs, and how it fits into making systems more resilient. The frequencies found stress the necessity for deeper studies on survivability.
Consider the pie graph for theme distribution (Figure 3), as it projects the proportions of references discussing both resilience and survivability in a single study and those covering only the survivability tract. It demonstrates that publications involving resilience and survivability in a single study scarcely reach a reasonable percentage. Furthermore, according to Figure 4, 53.79% of the references gathered have shown the pertinence of either resilience, survivability or both theories in the systems branch, which converges several sectors of the industry uncovering new strategies for disruption. The systems branch involves the advancements made in fields where a collection of units interdepends on one another to achieve a common purpose. The feature of interdependence is faithfully tied to the systems engineering mindset. Then, 19% of the references certify the overall effectiveness of the theories in natural and human sciences, reuniting views from economists, ecologists, environmentalists, physicians, and psychologists from different colleges of the US and other countries. Figure 3 and Figure 4 superpose on each other in the sense that most engineering and systems engineering articles focus on resilience.
Some of the references had a share of authors participating in the study who were not necessarily in the same region. If this was the case, each author was counted as one in accordance with the country he or she represented. Therefore, a sole reference could have two or three authors from different states, and each was tallied separately and joined the pool of authors who research in representation of a common college or region. Figure 5 supplies more details pertaining to the spread and concentration of research efforts.
By country, most references gathered come from colleges and industrial members in the US and the UK. Statistically, the references from the US comprise 40% of the total, while 8.5% come from the UK. Since the US covered such a reasonable percentage of all the references gathered, it was essential to see the spread of the efforts through the nation in twenty-five years. It was estimated that 10.5% of the initial 40% of references originated in the state of California, and another 10.5% were completed in Pennsylvania. In addition, it was seen that Ohio and Massachusetts have kept an active record in publishing works about resilience and survivability, demonstrated by their respective percentages of 9.3% and 8.1%. The other states record a count below 5% of the total; consult Figure 6 for more details on the spread and concentration of research efforts in a single area. No tendency was perceived on whether more references come from the East or West Coast of the US. As noted from the 10.5% tie that serves as a contrast between the East and West Coasts, both sides have contributed to the overall understanding of either resilience or survivability in terms of conceptualization and application. No trends implicate that one side of the nation has dedicated efforts to one theme over the other or to conceptualization more than application.
Table 1 and Table 2 number a set of keywords, ten for resilience and nine for survivability, that allude to certain definitions fashioned from the literature. Each definition proposes a singular view on either resilience or survivability, depending on the objectives and the circumstances surrounding each effort. The tables then allocate those definitions to ten selected authors from the pre-pandemic category, or bucket. More suitably, insights from the top ten references in the pre-pandemic bucket (i.e., the first ten articles appearing in the bucket) were grouped in accordance with the predefined keywords. The definitions to either resilience or survivability vary across a spectrum beginning with the elasticity/flexibility property and ending with the perception of failure. The format used in Table 1 and Table 2 to arrange definitions places a level of pertinence over the different aspects of failure. For instance, by acquainting oneself with the causes of failure, it will be easier to work on a disturbed system and foster its resilient and survivable properties. Failure may be caused by not complying with the elasticity property (i.e., keywords 1 and 2 in Table 1), which is basic in resilience theory. Another cause of failure may be prompted by not keeping the rate of value delivery or not having a recovery plan (i.e., keywords 3 and 4 in Table 1). On the survivability side, sources of failure may be not equipping the system to resurge once it has crossed its breaking point (i.e., keyword 1 in Table 2), delaying the recovery time or overlooking essential component functions.
Echoing the words of Madni and Jackson [27], on one hand, resilience is “the elasticity that allows a system to return to its original form, position, or configuration after being bent, compressed or stretched” (p. 185). On the other hand, the survivable character of a system is implied to be the umbrella for resilience as a way of withstanding and continuing to exist despite any disruption [27]. Provided that a system suffers changes in its lifecycle, much to its transformation, authors in this review have missed the state related to resurgence as a conditional property for a system to be survivable and not resilient. Loss of resilience is due to fact that the system has resurged as something entirely different, and it did not retract to what it was before, therefore violating the elasticity property.
Although there is no scientific convention regarding a breaking point or the resurgence property, the top ten references intend to picture survivability as the ability of a system to remain operational and deliver its output in accordance with some design thresholds. The recovery time must abide by these thresholds, as well as the system components undergoing changes. Richards et al. [34] compare the survivability phenomenon to some “special case of value robustness with a finite condition on disturbance duration”, with the caveat that survivable systems do not tolerate changes tending to infinity (p. 6). Then again, authors have faintly claimed the notion that crossing the breaking point can result from a disturbance or a change. Several of the top ten references have agreed with the thought that failure can be subject to how system designers trade off functionality and resources to attain best performance. Keywords 7 and 8 in Table 2 portray failure as an expression of slowness and unprofitability, not as a result of crossing a breaking point. It is interesting to take notice of the double perception of failure seen in the last two rows of Table 2: the first latches itself onto the probability of mission success, which takes the entire system as the full producer of service value; the second circulates the notion of individual parts sharing the blame for threatening the acceptable levels of service delivery.
Both resilience and survivability share points of debate and development. It is proposed in this study to combine the theories summarized in Table 1 and Table 2 to accommodate other scenarios than those already discussed in the literature. These scenarios should reflect the advantages of the resurgence property in system performance.

4. Discussion

In light of the definitions from the systematic review, there are several aspects concerning the two domains in question that this section addresses. Some of these are the comparable properties between resilience and survivability, the symmetries between resilience and survivability, the mathematical approaches for each, the subjects of breaking point and resurgence, and current gaps.
Tariq et al. [22] inform about two definitions that have inspired resilience researchers from the National Academy of Science in the US. The first is the general ability to plan for, as well as absorb, recover from, and adapt more successfully to actual or potential adverse events. The second concerns the capacity of exposed systems or communities to resist, buffer, accommodate, and repair itself from the influences of a hazard in a timely and efficient manner, including by means of the preservation and restoration of its essential structures and functions. However, in practical terms, hazards can vary from situation to situation. Community-based resilience and survivability are even harder to measure, as they equate certain intangibles depending on aspirations, expectations, and motivations from various members of the society. For this reason, resilience and survivability have been deemed more of a static value than that of a dynamic value fluctuating throughout time.
Cutter et al. [42] made the argument that resilience excuses those in authority from leading a social system out of its own vulnerability to hazards. Attached to this thought is the common knowledge that communities are bound to resist hazards at their own expense, and some find it geographically tough to overcome the issues impairing their survival. This makes the concept of resilience more a subjective matter, leaving researchers with the task of uncovering a scientifically objective rule to relay resilience to future audiences. Subjectivity (inflicted with bias) could make research studies misleading and polemical, and subjectively, resilience can be evaluated as synonymous with survivability in any situation.
Bearing that in mind, Figure 7 displays some symmetries between the concepts of interest. It also features comparable properties that should merge the concepts with equal intentions. Even though these properties may seem equal, it is fair to remember that these may happen at different points in time while systems dealing with complexities and pressures to achieve success have not been able to keep the minimum expectation and start to decline [43,44].

4.1. Comparable Properties between Resilience and Survivability

The first two comparable properties listed in Figure 7 point out that both resilience and survivability seek to lower costs and risks at a regional or global level. Both concepts have been described—especially resilience—as something the system does for the better, rather than something the system possesses [32,45]. Researchers have recently been examining where common ground may exist between domains to claim a definition that attaches diverse components into an intercommunicating system [46]. Every component may face unexpected events in times of uncertainty, which stirs the urgency to strategize and operationalize each component against internal and external shocks. In addition, engineers have notoriously been working on safety and quality in delivering service through high levels of adaptive capacity and risk tolerance [47].
Capable systems should therefore anticipate, learn, monitor, and respond to adverse events: these should be the four centric actions backed up by resilience [48] and survivability engineering. It has been suggested that further actions need dedication for system performance, such as noticing, planning, adapting, and rebounding; characteristics such as robustness and graceful extensibility should gain major focus in system architecture as well [28,45,49]. Graceful extensibility, in particular, allows a system to behave beyond its innate capacity and yield an acceptable performance [28].
To act upon the four centric actions, the system must acknowledge what the significant factors are capacitating its operations to such acceptable performance. It should receive inputs from its key actors and past experiences. The reality is that anticipation mechanisms do not record disruption as effectively as they ought to. Some corrective actions do not amend fractures in the system as fast as desired, creeping towards the undesirable breaking point. In general practice, some systems (or system components) are abandoned to failure with no recovery plan that could save their operations.
Several systems have not been properly prepared to address the onslaughts of natural disasters, internal attacks, and underdevelopment. In a socio-technical sense, tools have not been emplaced to cope with uncertainties and complexities. Awareness of the interconnection between system components (i.e., knowing that small changes may foreshadow certain effects from one link to the next) can be meaningful when performing the four centric actions. Grecco et al. [50] mention a critical cause bearing the answer as to why systems fall short of becoming resilient (or survivable): there is no true top-down commitment from those who manage the system or a sustainable culture that prioritizes service reliability.

4.2. Symmetries between Resilience and Survivability—Keywords 10 (Table 1) and 7-8-9 (Table 2)

Resilience and survivability exhibit a level of affinity that enables an overlap and overshadowing between the concepts. Sterbenz et al. [10] propose to view survivability as a major subset of resilience disciplines, and that subset particularly deals with “the design and engineering of systems that continue to provide service in the face of challenges” (p. 1247). Survivability, in this case, has a subset in which fault tolerance favors, but does not guarantee, resilience in every system.
One of the symmetries between resilience and survivability, number 10 in Table 1 and 7-8-9 in Table 2, has to do with how these theories perceive the causalities of failure. Survivability theory concerns itself with numerous correlated failures in an unbounded system that has no central authority or tightly coupled components, meaning some failures will prompt subjugated components to failure [10]. Reason [51] had referred to those failures as latent instead of active; latent failures are the result of induced errors (i.e., human-made decisions) that have cut through all the safety defenses in a process. Inherently, the organizational processes affect its own defenses through willful or accidental mistakes that trace a latent failure pathway, hurting the system through deficient decision-making [51].
According to the literature, resilience theory regards an occurrence as a failure if the levels of absorption and adaptation declined without surrendering any needed component or function. Madni and Jackson [27] specify that failure, in a systematic sense, is merely the product of a “web of interactions and adaptations that characterizes complex system behavior” (p. 181). It can be implied that failure may not always be due to poor design or unforeseen malfunctions; it clearly can encompass other circumstantial systemic behaviors [27].
Despite the level of affinity to resilience, survivability theory adds to the understanding of failure when it states that failure attaches to the minimum acceptable performance and value. System performance can generate value when those overseeing the system manage to balance functions with specifications and resources with demand. The definitions proposed in this work allow for the logic that tradeoffs cannot be achieved at times until the system has survived its breaking point and gained the experience to offer such features of balance. Lipson and Fisher [38] contribute another crucial piece in the technical perception of failure while also mentioning the benefits of tradeoff: each component of the system can undergo a local failure with the potential of compromising the entire infrastructure. Besides voicing the dangers in which accessible computerized systems can be exposed to, Lipson and Fisher [38] proclaim the most essential assumption for survivability when describing that “no component is immune from compromise, accidents, or failure” (p. 34). That premise alone applies not only to technical but also to executive problems across various businesses where induced failures from one component to the next can rekindle greater fires that could obstruct the mission.

4.3. Symmetries between Resilience and Survivability—Keywords 5 and 7 (Table 1) and 5 (Table 2)

Keyword numbers 5 and 7 in Table 1 can couple with keyword number 5 in Table 2, although there can be a slight difference. The idea of having four non-functional requirements (NFRs) was first adopted in this study from Tahoori et al. [12] when they conclude that “the main NFRs […] introduced by researchers are changeability […], versality […], survivability […], and robustness […]” (p. 619). The authors hinted that survivability is, in fact, the balance of three system parameters: perturbation type, output parameter, and system parameter. In theory, if the perturbation type is a disturbance, and the output parameter has no change, and the system parameter has a change, then the system is said to be manifesting a survivable demeanor in which extramural uncertainties instill the need for rethinking value [12].
Of course, in the literature, the assertation that system outputs will remain unaltered in the face of a disturbance (i.e., an earthquake, a cyberattack, enforced regulative policies, etc.) can evoke a debate when accumulating views from diverse sectors. In transportation research, Morlok and Chang [52] addressed survivability as a token of system flexibility, where infrastructure has degraded to the point of only satisfying a fraction of the demand. Makhutov and Gadenin [53] bridge survivability with a certain loss of functionality and “process and operational flaws […] and damage […] that violate the existing rules and norms” (p. 298). Load-bearing components can be subject to stresses that detract from the underlining system functionality and debilitate its strengths, as discussed in reliability theory [53].
Keyword 5 in Table 2 named four specific NFRs, as it can be proposed that the permutation of two, three, or four of these requirements (placing careful attention on the order of the requirements) may result in another parameter variant. The slight difference between keywords 5 in Table 1 and Table 2 is that one encourages resolving survivability issues by finding a convenient leveling of the NFRs, while the latter suggests that at most all the four-A features should be capacitated to optimal function, reserving the leveling matter to keyword 7.
As disclosed in Figure 7, the four-A features in a system are anticipation, adaptation, absorption, and amelioration. This study identified Faturechi and Miller-Hooks [29] as precursors of the thought in which survivability and resilience conceptually differ but complement one another. Resilience increments over time while the system-makers gain knowledge about the “strength and weaknesses measured by risk, vulnerability, reliability, robustness, […] survivability, and adaptability […]” (p. 3). Therefore, in light of that argument, the interventions made in a system should be configurated in a balanced manner, making sure that the performance indicator being monitored can aid the system in predicting disruption, buffering impacts, and strengthening its current components. Balanced configurations to intervene should be decided bearing in mind pre- and post-disaster circumstances in the effort of making a threat assessment.
On that account, Small et al. [9] had incorporated “ilities” as essential criteria for system analysis as they “are the key consideration in the cost and value of systems” (p. 7). Small et al. [9] also enlisted all the possible “ilities”, requiring notice for resilience evaluation (and threat assessment): availability, reliability, survivability, producibility, supportability, and others. Richards et al. [34] proposed the usage of the “ility space” where the authors graphed the interrelationships between three analytical elements: flexibility, adaptability, and serviceability. It is further encouraged to map survivability against other “ilities” to minimize confusion in the definition of those elements in the design and testing processes [34,54].

4.4. Symmetries between Resilience and Survivability—Keywords 1-2 (Table 1) and 2-3 (Table 2)

Keywords 1 and 2 in Table 1 match keywords 2 and 3 in Table 2. The “ility space” better serves its analytical purpose if mapped as either a passive or active survivability effort. Philosophically, passive survivability stands by the same principle as static resilience: enforce some resistance to architectural shifts and harm. Under the passive classification, survivability is then argued as a trait the system has, as opposed to being active [35]. Passive survivability tools are employed to support the resistance and stimulate defensive barriers. Active ones, however, aid in repairing the system if there has been any breakage or contact with detrimental factors. Uday and Marais [13], quoting the inferences of Rose [55] and Whitson and Ramirez-Marquez [56], framed static resilience as maintaining systemic functions, while dynamic resilience turns to recovery. When plotting situation performance versus time, static resilience would be present during system deterioration as the system protects itself from breaking; immediately after, dynamic resilience would help the system escalate to normal performance outputs. Such phenomena precipitate the intervention of modifying anything in the system, if advisable, given the factors threatening the current system design (or arrangement).

4.5. Expanded Definition Tables with Aligned References

Table 3 and Table 4 ingress important data from Table 1 and Table 2, as they also aggregate the quotes of late and fresh precursors altogether. For review purposes, it is indispensable to call attention to keywords 8 and 9 in Table 3 and the number of precursors it has amassed in the third column titled Aligned References (the word precursor would be in relation to this study, not the top ten authors necessarily). The total of 20 aligned references denotes an ongoing movement in research that underlines the validity of judgement and experience in resilience efforts. In other words, the more subjective and custom-made the resilience tools are to individual and collective needs, the more effective they will be at making progress.
Bruneau et al. [57], for example, explain how the resilience of essential services like water, power, emergency care, and micro-organizations in the local region improves community resilience in general. Nemeth and Herrera [31] exhort further research in resilience engineering to be oriented towards answering these questions: can petit scales of resilience build a stronger one, how can resilience engineering export lessons across fields, and do all components share the same scale? Inquiries like these evoke the need that a clearer delineation for systems engineering should be made in tandem with the analytical tools to ensure resilient behavior on a larger scale. In addition, the dimensions those systems should possess, and their scalabilities, have shown to be a promising breakthrough in resilience research (i.e., keyword 6 in Table 1). Hazelrigg and Saari [58] explore more on subjectivity, common system preference, and beliefs guiding decision-makers to put first what they esteem valuable through the conveyance of nine fundamental principles of systems engineering and design.
As stated earlier, in and of itself, subjectivity constitutes a major risk in systems analysis when it separates views on resilience and survivability even more. Jones [21] argues about the lingering confusion within the discipline of resilience measurement in classifying their own tools as either objective or subjective; as many views are provided in the literature, the blurrier the measurement becomes in practice. The subjective–objective continuum offers a mechanical exercise to map approaches according to who defines and who gauges resilience: the discrepancies between internal and external evaluators [21].
Table 3. Resilience keywords and aligned authors.
Table 3. Resilience keywords and aligned authors.
Resilience
View (and Keywords)Top 10 ReferencesAligned References
(1) Elasticity or (2) flexibility propertyMadni and Jackson [27]; Uday and Marais [13,24]; Woods [28]Darling Rasmussen et al. [59]; Dinh et al. [60]; Herrman et al. [61]; McCubbin [62]; Pan et al. [63]; Woods [64]
(3) Static and (4) dynamic resilienceUday and Marais [13,24]Dinh et al. [60]; Oehmen and Kwakkel [65]; Rose [55]; Sharma et al. [66]; Whitson and Ramirez-Marquez [56]; Wied et al. [67]
(5) Four-A featuresFaturechi and Miller-Hooks [29]; Jones [21]; Small et al. [9]Folke et al. [68]; Ham [69]; Hollnagel [70]; Hollnagel et al. [48]; Izadi et al. [71]; Plotnek and Slay [72]; Righi et al. [73]
(6) Dimensions of a resilient systemHosseini et al. [30]Bruneau et al. [57]; Cimellaro et al. [74]; Dinh et al. [60]; Doorn et al. [75]; Izadi et al. [71]; McCubbin [62]; Oehmen and Kwakkel [65]; Peñaloza et al. [76]; Rabbani et al. [77]; Ranasinghe et al. [78]; Shirali et al. [79]; Wied et al. [67]; Yu et al. [46]; Zuo [23]
(7) Openness to tradeoffsFaturechi and Miller-Hooks [29]; Hosseini et al. [30]; Jones [21]Ayyub [80]; Bruneau et al. [57]; Bruneau and Reinhorn [81]; Cheng et al. [82]; Cimellaro et al. [74]; Jasiūnas et al. [83]; Ma [14]; Said et al. [84]; Sharma et al. [66]; Wang et al. [85]
Heuristics based on (8) experience and (9) judgementNemeth and Herrera [31]; Patriarca et al. [32]; Jones [21] Bruneau et al. [57]; Bruneau and Reinhorn [81]; Cimellaro et al. [74]; Darling Rasmussen et al. [59]; Feldman [86]; Fiksel [87]; Herrman et al. [61]; Lee et al. [88]; Meerow et al. [89]; Patterson et al. [90]; Pillay [91]; Pooley and Cohen [92]; Righi et al. [73]; Rubio-Romero et al. [93]; Rudd et al. [94]; Schafer et al. [95]; Southwick et al. [96]; Tariq et al. [22]; Zautra et al. [97]; Zohuri et al. [98]
(10) Failure as the inability to absorb disruptions or adapt to changesMadni and Jackson [27]Hollnagel et al. [48]; Oehmen and Kwakkel [65]; Wied et al. [67]; Yu et al. [46]
In the matter of survivability, Table 4 proves the ongoing efforts to further elucidate the matter of what makes a system survivable. More specifically, the popular movements seek to provide insights on what essential means (or channels) of service call for protection and to what extent, in conjunction with the optimal arrangement of NFRs. The tally of aligned references for keywords 4 and 5 in Table 4 records a sum of six references each, which suggests a fair diffusion of theories addressing the “ilities” and NFRs.
Table 4. Survivability keywords and aligned authors.
Table 4. Survivability keywords and aligned authors.
Survivability
View (and Keywords)Top 10 ReferencesAligned Reference
(1) Resurgence propertyNo reference.No reference.
(2) Active and (3) passive survivabilityKnight and Sullivan [33]; Richards et al. [34]; Richards et al. [35]Linger et al. [99]; Richards et al. [20,100]
(4) Minimum essential set of means to provide autonomous performanceEllison et al. [36]; Firesmith [37]Ellison et al. [101]; Linger et al. [99]; Mead [102]; Mead et al. [103]; Redman et al. [104]; Westmark [105]
(5) One of four non-functional requirements (i.e., viability, flexibility, dependability, survivability, or other variants)Lipson and Fisher [38]Ivanov and Dolgui [106]; Makhutov and Gadenin [53]; Morlok and Chang [52]; Peshkov [107]; Westmark [105]; Woolley et al. [108]
(6) Measured performance over timeHellmann et al. [39]Bulian et al. [109]; Woodard et al. [110]
“Failure” dependent on (7) how fast and (8) how much value or net benefit the system can deliver to its stakeholdersEllison et al. [36]; Knight et al. [40]; Yaghlane et al. [41]Mead et al. [103]; Yaghlane et al. [41]
(9) “Failure” as a threat that affects value deliveryFaturechi and Miller-Hooks [29]; Lipson and Fisher [38]; Yaghlane et al. [41]Ellison et al. [101]; Levitin et al. [111]; Mead et al. [103]; Woodard et al. [110]
In industrial and systems engineering, the concepts of resilience and survivability as metrics to sustain an improvement may highly apply to engineering controls in any project undergoing a systematic problem-solving approach. In practice, sustaining a change can be challenging without a performance metric or economic trend proving the change has worked for the better. Another practical term more commonly used to explain the overall interest of reaching a level of resilience and survivability would be reliability, which validates the thought that resilient or survivable systems tend to be more reliable. Other terms found in the literature and practice resembling the roles and interpretations of resilience or survivability are flexibility and viability. The interchangeability of words to concretize the influences of unsustainable systems everywhere assures that a common interest exists to understand these roles more deeply.

4.6. Industrial Applications Revolving Survivability

In past efforts, survivability has been framed in terms of what a specific situation or stakeholder requires. Mead [102], on the one hand, supported that survivability holds a close relation with service quality and should be treated as a factor for dependability, a critical aspect of user-centered quality. On the other hand, Sharma et al. [66] identified other major outlooks for survivability in buyer–supplier transactions considering the effects of the pandemic. The stepwise weight assessment ratio analysis (SWARA) allowed for thorough assessments pointing to viability as the principal factor for achieving survivability. Viability refers to how swift or reactive an entity can be at replanning or reconfiguring itself as it faces crises [66].
According to the Beer’s viability system model (VSM), viability has its biological systems connotations, and these can be translated into sociocultural goals to achieve survivability in open systems [112,113]. Viability had been approached by Tahoori et al. [12], who posed a fairly similar definition of survivability in terms of value. Special attention therefore should be paid to systemic infrastructures to bridge the gaps between sustainable factors and redesign for viability [66]. Examining survivability in small enterprises in the Polish rubber products industry, Czainska et al. [114] discussed ideas supporting sustainable resource management as a strategy to fortify business relations under distress; the resources identified for survivable systems generating value include human capital, economy, material, and technology.

4.7. Industrial Applications Revolving Resilience

Studies providing empirical evidence on what works best in terms of recovering from crises have been lacking since the COVID-19 outbreak. An empirical study suggested a link between resilience indexes and service quality dimensions, defining resilience as the ability to face unforeseen events given a set of shock management techniques [115]. Madni and Jackson [27] had emphasized the need to develop safety levels as modern systems continue to grow in complexity, claiming that significant system failures likely occur because of latent organizational factors. The notion that resilience engineering is an iterative tradeoff between utility, cost, schedule, safety, and performance was then introduced. At an organizational level, Huang and Farboudi Jahromi [116] share five strategies to enhance resilience in the manufacturing and service industries under external shocks. The authors recommend that enterprises conduct transformative efforts favoring market orientation, supply chain optimization, strategic alliances, innovation, and business models (i.e., ways of doing business with customers and other key stakeholders in the field).
Reflecting on resilience engineering research over twenty years up to 2018, Patriarca et al. [32] observe that its various advances have been insufficient to determine some practical implications. Yet, Pawar et al. [117] distinguish some advances in research that target sociotechnical issues, disasters, restoration, and optimization. Resilience engineering therefore has served as a large source of tools for risk management. For example, Arcuri et al. [47] applied systems thinking to tackle issues in healthcare delivery resilience, specifically employing the functional resonance analysis method (FRAM) to predict and aid demands for access to essential services through mobile river systems (i.e., a river-type ambulance network) in Brazil. The two major findings described were, first, safety could be challenged when informal organizational support nets fail, and second, the system was trying to operate for a cause for which it was not designed [47]. There may be systems with not enough capacity, self-protection mechanisms, equipment, or well-planned strategies to generate desirable outputs under crises. Belhadi et al. [118] add further insights into the impacts of the COVID-19 pandemic on the automotive and airline supply chain, concluding that uncertainty phenomena and the interconnectedness of components play a key role in determining the appropriate tools needed for resilient system outputs in the short run.
Multi-criteria decision models (MCDM) have served as practical tools in resilience engineering. Zarei et al. [44] developed an MCDM to gather impressions from a wide spectrum of workers occupying positions with varied educational levels. The purpose of the study was to weight and rank organizational performance indicators: openness to learning, culture, and emergency preparedness [44]. Through the fuzzy AHP approach by Zarei et al. [44], it was proven that resilience can be perceived as a socio-technical matter. Huang et al. [119] proposed an MCDM framework to survey resilience at three international airports in Taiwan, discovering that “most experts consider resistance capability […] to be the critical dimension for airport resilience” (p. 8). Resistance capability requires training workers to respond to risks as they encourage airport users (to follow safety practices) and assign special resources for prevention [119].
Seventeen indicators for supply chain resilience were included in a review study by Singh et al. [120], affirming that “little consideration has been given to performance measurement of supply chain(s)” (p. S106). These indicators (i.e., agility, flexibility, robustness, redundancy, visibility, information sharing, among others) basically fit three categories found in the literature: the anticipation phase, the resistance phase, and the recovery phase [120]. Through the advocacy of production innovation in manufacturing, Romero et al. [121] confirmed a statement made by Singh et al. [120] in their attempt to defend the need for resilience in supply chain networks. Such statement conveyed the notion that short product life cycles and variability in stakeholders’ requirements lead to disruption.
Further insights have recommended that addressing loss of skills in a tech-dependent workplace can resolve the matter of underperformance. For that reason, production innovation has been proposed as another tool to manage disruptions more proactively and to certify resilience throughout certain capacities, allowing a successful reconfiguration of the elements making up the system of interest [121]. It has been therein implied that resilience in sub-systems will induce evenly widespread resilience across the whole entity.
Contemporary resilience research has focused on policymaking efforts, economic sustainability, and community resilience. Highlighting the impact on economic development, Chacon-Hurtado et al. [122] stated that transportation infrastructures nurture both the local and regional economies by tracing numerous trade nodes where accessibility can still be improved. As a result, Chacon-Hurtado et al. [122] provide a set of metrics to gauge accessibility to procuring growth in social capital and industrial diversity. Those two elements, along with livability and road quality, are primary aspects of resilience targeted to development and fiscal shocks. Other critical architectures that impact development either directly or indirectly involve energy distribution points [123,124], communication networks [10,125], and satellite arrangements and space-based missions [11,15].
Describing resilience as a group’s aptitude for withstanding social, political, and environmental change, Payne et al. [126] provide proof that resilience is indeed quantifiable “at an individual dimension and overall”, arriving at the conclusion that perceptions will continue to vary from place to place (p. 3). Findings suggest a certain complexity as to how social groups view their own strengths and how they conceptualize and grade their own community resilience. Each group will have its own context-specific variables as per its experience in a specific time [126].

4.8. Mathematical Interpretations and Applications

The theories espoused in the previous sections have set the principles to quantify the intensities in which engineering controls and policies can ensure resilience and survivability. For this reason, Henry and Ramirez-Marquez [127] and Meyer [128] reviewed multiple measures and formulae to estimate system resilience, detailing the conceptual bases on which the quantitative approaches were conceived. Resilience can be mathematically ideated in function of time where the states of the system can reflect deterrence or betterment. In any case, it is imperative to (1) draw boundaries of the scope of the component of the system (or the system of the systems) urging an intervention, and (2) determine a figure of merit (FOM) parameter. These should be the two principal steps for any system analysis, whether it be simple or convoluted [127].
FOMs refer to quantifiable system outputs that fluctuate through time. They are the numerical parameters any system analyzer would want as large as possible. Throughput, profit, and utilization can exemplify kinds of FOMs. Equation (1) is a generalized formulation for FOM proposed by Ferris [129]. Table 5 includes the equation, which was found in a second publication by Ferris [130], as well as the mathematical approaches used to tackle system resilience.
In terms of applicability and frequency, as shown in Table 5, there have been many efforts approaching survivability with Markov chain modeling and other models such as the stochastic reward net model and the deterministic minimum attack difficulty path. Uncertainty can be modeled well in systems analyses through probabilistic parameters and regression. Much subjectivity, nonetheless, can creep into some of the models, especially those revisiting historical data or surveying opinions from stakeholders.
Table 5. Quantification techniques breakdown with aligned authors.
Table 5. Quantification techniques breakdown with aligned authors.
Quantification Techniques
Theme ApproachAligned References
ResilienceProbabilistic parameters (i.e., system recoverability importance, system disruption importance, and system recovery time importance)Uday and Marais [24]
Resilience Matrices and/or loss of resilience integral (i.e., in terms of quality or flow capacity) Pant et al. [131]; Pumpuni-Lenss et al. [132]
ResilienceFOM parameters Ferris [129,130]
ResilienceMCDM models (i.e., fuzzy hybrid)Zarei et al. [44]
ResiliencePre- and post-disaster restoration models (i.e., expected outage duration, expected energy not served, probability curve of line fault)Shi et al. [133]
ResilienceJoint contagion model and crisis severity indexChih et al. [134]
SurvivabilityManeuvering intensity analysis and parameters Hohoniants et al. [135]
SurvivabilityAttack signature formulation and generation (i.e., complex computer systems) and profiles of all functionalities of a system during specific time intervalsKrings et al. [18]
SurvivabilityGrey relational analysis and network entropy differenceZhao et al. [136]
SurvivabilityState nodes and intrusion analysis (i.e., minimum attack difficulty path)Zhang et al. [137]
SurvivabilitySystem steady state and/or transient behavior (i.e., Markov chain modeling)Bisikalo et al. [138]; Hohoniants et al. [17]; Liu and Trivedi [19]; Rumawas and Asbjørnslett [139]; Saleh and Chowdhury [140]; Trivedi and Xia [141]
SurvivabilityProbabilities and the stochastic reward net model (Markov chain-based)Chang et al. [142]
SurvivabilityProbabilities, utility loss, and availability threshold (and other variants for policymaking)Richards et al. [20,100]; Zhao et al. [143]
SurvivabilityRegression Le Thanh et al. [144]
Survivability/ResilienceOptimization problem (i.e., network service availability and total demand, plus Lagrangean relaxation)Ríos et al. [145]; Santiváñez and Melachrinoudis [146]; Wang et al. [147]
Equation (1), along with Equations (2) and (3), manages to equilibrate the objective aspect of the math formulation with the subjective aspect of the themes in question. Equation (2), as proposed by Zhang et al. [137], contributed to the analysis of the minimum attack difficulty path, suggesting that any system remains as strong as its weakest link. Translated into computer networks (almost all services depend on a computerized system today), their survivability is thought as the “minimal cost function to compromise the system with relation to all possible intrusion scenarios” (p. 3213). Equation (3), at least from the minus onto the integral, was first proposed by Bruneau et al. [57] and requoted by Pant et al. [131] as loss of resilience. The equation was rewritten below as a subtraction to 1 to complement the loss with the area of success. In other words, the subtraction would yield the estimation for resilience from time sub-0 to time sub-1.
Stressing the effectiveness of the trade–space method for design proposals and their comparisons, Equation (1) smooths the difficulty of having conflicting approaches for system resilience. In fact, every variable in the equation is a point of disparity where stakeholders could come to an agreement. This benefit is not exclusive to one sector of the industry or to a particular field. FOM is conveniently fitting to any system in any greenfield (i.e., building new systems in a new climate) or brownfield (i.e., modifying existent systems) development project [130]. It must be clarified that FOMs, in turn, can convert into measures of resilience when they signal the value outputted by the system, or v i j , where i tallies performance attributes up to n in the trade-space analysis, and j truncates the system’s lifecycle into m equal timeslots at t = 0. Each attribute i is expected to yield a performance level P in every timeslot j under environmental or operational condition k. Likewise, each attribute i will be assigned a weight w relative to its cruciality in deliveries of v under k. In summary, Equation (1) can directly quantify resilience “in terms of achievable performance during a sequence of epochs t j t t j + 1 ” (p. 1578).
Equation (2) searches for the minimum quantification of the complement for the product of the flipside of each attack difficulty D in each case m M of vulnerability total I. Therein, D can be expressed from a 0 to 1 scale, satisfying the assumption that the closer to 1, the more difficult it will be for any attacker (or disturbance) to penetrate the system. Another assumption worth mentioning lies in the context defining survivability as the measure of the lowest intrusion success [137]. This formulation could be imagined as an organized assault that was issued on a certain infrastructure, by an attacker who learned the easiest path (or most vulnerable node) into the system.
Equation (3), grading a system’s quality from 0 to 100%, estimates resilience as the area under the curve X between two points in time. Loss of resilience, also known as the resilience triangle, can measure “technical, organizational, social, and economic aspects of community behavior” ([85], p. 96).
F O M = i = 1 n j = 0 m 1 ( w i k v i k ( P i j ) ( t j + 1 t j ) )
S u r = min [ 1 i = 1 I m ( 1 D m i ) ] m = 1 , ,     M
R e s = 1 t 0 t 1 [ 100 X ( t ) ]   d t  
Pant et al. [131] forward support metrics that leverage the subjectivity of weights when speaking about resource allocation and planning for resilient interconnected infrastructures, argued in the study as tasks for static resilience. The authors reasserted the ethical crossroads between decision-making and resilience and the mathematical connection between three key resilience indicators: maximum loss of functionality, time to recover, and time-averaged level of operationality. Informed decision-making can induce the chance of lower recovery rates from sector to sector, as it can strengthen the economy by placing investments where propriety demands (i.e., debate on investments in transportation or manufacturing or another area).
Although the subject of thresholds was not inferred in the study by [131], they do find value in the process of planning decision spaces when deciding on activities and investment schema. Breaking point formulations were not found in their study either. Richards et al. [100] and Zhao et al. [143], however, fairly have relayed the subject of thresholds as design constraints and performance control analyzers in every system’s life.

4.9. Breaking Point and Resurgence Features

Acquiring a nature much more aligned with what was originally proposed by ecologists, resilience and survivability can conceptually part from each other as systems resurge as something completely new. In other words, systems can technically break (fail to deliver the expected value) and slowly come back with new parameters and capabilities, and that can become the new survivability phenomenon.
Woodard et al. [110] began a useful exploration on survivability, leaning towards graceful degradation as the ultimate manifestation of such phenomenon. Graphically, it is seen immediately after a disruption surprises the system, which fluctuates downwards without touching a breaking point. In addition, Belhadi et al. [118] used a graphical delineation of resilience that upholds the math proposed in this article. The study appears to be mostly empirical, but it also leverages statistical analyses to draw informed conclusions. It captures the measure of resilience in the integral (i.e., area under the curve) of the equation describing the performance curve between two event times. Contrariwise, the loss of performance can be measured by subtracting the resulting area from what is called the permanent-level performance times the recovery time [118]. Although the idea of touching a “minimum level” is considered, there has not been as much discussion concerning surpassing a breaking point in resilience or survivability theory as in psychology and materials science.
When discussing complex systems in engineering, where many disciplines combine to procure resilience and survivability, a worthy example can be found in a group of citizens who have withstood either internal disruptions (i.e., from the government) or external disruptions (i.e., from mother nature) and, after many years, the group has reached a breaking point. Once reaching that breaking point, the hesitant attitude towards their situation withers more and more until their character as a group changes completely. It can take several years for that group to realize that their situation needs to change (for the better), and here the resurgence property takes effect over the group. While resilience points to the elasticity property testing the patience of the group, the survivability factor denotes their ability to come through the adversity by assuming different postures and channels to solve their problems.
Under these circumstances, survivability will be foreshadowed by the repercussions of a broken system and the overall system upturn through the resurgence phenomenon. The group coming out of the resurgence period is not the same as the original group that showed resilience for an allotted time. Therefore, the elasticity or flexibility property was lost during the resurgence phase. In other words, there is no “bouncing back” to the initial state if the new parameters (i.e., the patience of the group can be seen as a parameter, for the sake of the example) remain strictly unchanged. Hence, a parameter may be a direct reference to the performance level or system output. Parameters such as collective patience or economic yield help measure resilience and survivability at specific points in the lifetime of the system. Then, it could be theoretically legitimate to establish the following difference between resilience and survivability: the former ensures a certain bend in the system that should be able to return to its first form, while the latter sees to a reconfiguration of the entire system in terms of expectations and functions after the resurgence period.
System components undergoing breaking points, such as a natural systematic demeanor for resilience, have been exposed (or at least alluded to) in the literature during the past decade. For instance, Boin et al. [148] refute the prescriptions for organizational resilience altogether, calling them highly impractical and challenging for crisis managers and planners at the time of use. According to the claims made by the authors, an organization tends to “rally its resources and partners in creative ways”, suggesting the likelihood of seeing more improvised protocols than planned solutions (p. 433). Power crises and blackouts (i.e., the cases of the state of California and Puerto Rico) bespeak the truth that breaking points exist and happen more often than what is known. If a system survives its breaking point at time t, the sole measure to monitor would be survivability, since the resilience expectation failed.
Broccardo et al. [149] have widely contributed to system analysis by investigating resilience using the PEER (from the Pacific Earthquake Engineering Research Center) framework through the interactions between the time of recovery and the interarrival time of future events. The most valuable piece of knowledge is that resilience can wear out in civil applications. In other words, some events or series of events can trigger a non-acceptable loss of resilience for a certain timespan; hence, the system can be categorized as not resilient [149]. There is a chance to have recovery breaking points in the near or distant future [149].
In Figure 8, the shaded area underneath the minimum value threshold or Period B is proposed in this study to be the resurgence feature of a system, where its performance P has hit the unacceptable region with a slight probability of coming back up.
Figure 8 clearly illustrates a version of keyword 6 in Table 2, which interprets the survivability of a system as a measure describing performance over time. Bruneau et al. [57], Cimellaro et al. [74], Wang et al. [85], Jasiūnas et al. [83], and Cheng et al. [82] have introduced graphical versions of Figure 8 in their respective studies without any threshold or shaded downturns. The scientific contributions made by Richards et al. [100] also inspired the parametrization of survivability with illustrative addiments, still taking into account the three registered types of survivability. Type-3 survivability, also known as resilience enhancement, concerns itself with keeping newly survivable systems resilient after their resurgence. The resurgence phenomenon is avoided when having adequate anticipation mechanisms and corrective actions improving hazardous issues in time. This would benefit a system, as it is kept above the breaking point. Expanding on the premise, survivable systems essentially have been resilient before, but they did not manage to remain afloat for much longer.
S u r = t T L P ( t )   d t t T L V m i n ( t )   d t
P = P e r f o r m a n c e   c u r v e V m i n = M i n i m a l   A c c e p t a b l e   V a l u e   t h r e s h o l d T L = T o t a l   l i f e t i m e
O p t   S u r = P m a x V m i n T L t
Equations (4) and (5) explain survivability as the integral of performance P since time t, extending possibly through the lifetime TL of the system under assessment, minus the integral of the minimum value Vmin region from time t to an indefinite horizon. Optimum survivability is proposed to be the constant value of maximum performance minus the threshold value of minimal acceptable performance, divided by the time range or duration of the system. This second metric can only be quantified for historical purposes after a system dies beyond repair. It is valid to distinguish that system output could be in terms of cash or units of production; it will all depend on the accorded FOM.
Interdependency between sets of systems and its effect in resilience engineering have proven to be of interest in the literature as well. Hickford et al. [150] reviewed the science behind resilience engineering when applied to interdependent critical structures or systems that affect one another. Affecting one another can be twofold, in the sense that a change in a random system could have a positive or a negative effect on another random system attached to the first. This is distinctively revealing for resilience and survivability engineering because one breaking point could precipitate another breaking point on a linked system component or network. It was nonetheless denounced as bad practice in regard to keeping a structure reliable and hazard-free, or just the gap between knowing and doing [150]. Another interesting statement by Hickford et al. [150] says that “the accuracy and relevance of [resilience engineering] models is dependent on good quality data”, and quality data could be unreachable in developing countries where there are no databanks or storing mechanisms. By extension, the poor status of a country deters system resurgence.

4.10. Why Continue Resilience and Survivability Studies?

Payne et al. [126] elaborated on reasons that make measuring resilience critical. The first reason points precisely to the benefit of benchmarking a community against itself or another experiencing the same external influences; tracking the resilience history can help a community recognize what activities foster progress. The second reason leans towards the benefit of making smarter choices and setting priorities. The third reason hints toward the effective participation of community members (or system components) in order to empower other members into acting on and improving their own individual resilience. Costella et al. [43] might have added an additional reason why minding resilience is important. Resilience engineering—as survivability engineering—upholds systems designs that not only assure high levels of performance, but also foster self-assessments on health and safety management. Self-assessments and control mechanisms are paramount to achieving optimum conditions for systems delivery and developing a culture centered on “good practices”.

5. Conclusions

Although the literature brought together a set of insights, there is much more to say about resilience than about survivability in a partial, sector-specific manner. The subjectivity embedded into current models has allowed practitioners to strengthen the systems they oversee, but it has brought discrepancies among scientific communities from different disciplines. One principal takeaway, which has been stressed on many occasions, concerns the lack of agreement clouding the authentic definition of resilience and survivability. It was found in this study that resilience and survivability share some symmetries that may imply some level of interchangeability between them. For example, they both address the well-being of an infrastructure struggling against internal and external influences, while adjusting parameters in system design to guarantee optimum performance. However, some scenarios were described with the purpose of ratifying their differences when tensions overpower strengths and systems start to lose resilience. The main complementarity between them is the subject of resurgence after systems break.
Another takeaway recognizes that systems failures are assumed to be contained in the minimal value region from where systems can resurge or come back. Resilience breaking points contribute to a rising theory claiming that resilience may not always be reflected in the performance curve of a system. Rather, non-resilient systems can have a “second chance” at delivering value, but they would appear with renovated characteristics and performance thresholds. Under such circumstances, the interplay between resilience and survivability begins at these breaking points where resurgence takes place.
A third takeaway is that system analyses require an assortment of measures which facilitate the progress or deterrence of any system in question. For resilience and survivability, there is a tendency to suggest either stochastic models, multicriteria decision-making models, or a set of deterministic equations. The equation for FOM smooths the conflicting views for resilience by scaling inputs.
Proposing addiments to the graphical resilience version by Bruneau et al. (2003) [57] makes room for further analyses and discoveries. Future research could be oriented towards making qualitative analyses to identify resurgence and resurgence probabilities and even develop models to forecast resurgence events amid uncertainty. Other future endeavors could concentrate on expanding the survivability conception in social, economic, and technological aspects of struggling countries where engineered systems often fail. Resilience has been exploited more in those disciplines involving applications for developing countries, but it has not been employed in the same context proposed in this study. Another study worth advising, more associated with engineering, would be probing controls for safety/security administration and resilience enhancement (i.e., Type-3 survivability) for those systems recovering from hazards or downturns. Along those lines, the hierarchy of engineering controls, well known in the safety domain, could be the focal point of discussions on how it can aid the performance of complex systems and work with resilience heuristics.
This study has certain limitations regarding the scope and the number of references included in the keywords/views analysis. Generally, the study preferred those references with an engineering background and skimmed over complementary references from other fields. It is therefore understood that due to how popular resilience and survivability have become in present times, many suitable heuristics and strategies were left out. Further limitations are tied to the assumption that resilience and survivability, as design requisites for optimum performance, need more neutrality in their formulations in the effort of relaxing the conflict between views. This assumption can generate even more division, as it is of common belief that systems will always work when confined within preference and user context, disregarding the point that system-makers may not always know what benefits their own systems or how to deal with problems happening in their settings.
Both resilience and survivability must go beyond a safety checklist; they should be understood as parameters of good practice to be monitored throughout the lifespan of a system. Such parameters should be continually updated to match problematics emerging from an ever-changing environment. Further research can suggest definitions and math formulations attemperated to our time, always bearing in mind that shareable views on resilience and survivability are desirable.

Author Contributions

Conceptualization, K.M. and D.C.; methodology, K.M.; formal analysis, K.M.; investigation, K.M.; writing—original draft preparation, K.M.; writing—review and editing, K.M. and D.C.; visualization, K.M.; supervision, D.C. All authors have read and agreed to the published version of the manuscript.

Funding

The publication of this research was funded by the Scientific Caribbean Foundation, Inc., in San Juan, Puerto Rico.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Acknowledgments

The authors would like to acknowledge the valuable input of each member of the H.O.M.E. Lab at UMass-Lowell in shaping this research, as well as the support of Juan Arratia in providing funding.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Chaigneau, T.; Coulthard, S.; Daw, T.M.; Szaboova, L.; Camfield, L.; Chapin, F.S., III; Gasper, D.; Gurney, G.G.; Hicks, C.C.; Ibrahim, M.; et al. Reconciling well-being and resilience for sustainable development. Nat. Sustain. 2022, 5, 287–293. [Google Scholar] [CrossRef]
  2. Roostaie, S.; Nawari, N.; Kibert, C. Sustainability and resilience: A review of definitions, relationships, and their integration into a combined building assessment framework. Build. Environ. 2019, 154, 132–144. [Google Scholar] [CrossRef]
  3. Fairbanks, R.J.; Wears, R.L.; Woods, D.D.; Hollnagel, E.; Plsek, P.; Cook, R.I. Resilience and resilience engineering in health care. Jt. Comm. J. Qual. Patient Saf. 2014, 40, 376–383. [Google Scholar] [CrossRef] [PubMed]
  4. Jacobson, S.; Hall, S.; Swisher, J. Discrete-event simulation of health care systems. In Patient Flow: Reducing Delay in Healthcare Delivery; Springer: New York, NY, USA, 2006; Volume 2, pp. 273–309. [Google Scholar]
  5. Matyas, D.; Wills, P.; Dewitt, B. Imagining Resilient Courts: From COVID to the Future of Canada’s Judicial System; University of Cambridge: Cambridge, UK, 2021. [Google Scholar]
  6. Matyas, D.; Wills, P.; Dewitt, B. Imagining Resilient Courts: From COVID-19 to the Future of Canada’s Court System. Can. Public Policy 2021, 48, 186–208. [Google Scholar] [CrossRef] [PubMed]
  7. Mitri, M.; Abi Fadel, F.; Juvelekian, G. Resilience in Health Care: Surviving a Coinciding Pandemic, a Major Deadly Disaster, and an Economic Collapse: What Did We Learn? Chest 2021, 160, 1986–1988. [Google Scholar] [CrossRef]
  8. Ruhl, J.; Cosens, B.; Soininen, N. Resilience of Legal Systems: Toward Adaptive Governance. In Multisystemic Resilience; Oxford University Press: Oxford, UK, 2021; pp. 509–552. [Google Scholar]
  9. Small, C.; Parnell, G.; Pohl, E.; Goerger, S.; Cottam, B.; Specking, E.; Wade, Z. Engineering resilience for complex systems. In Disciplinary Convergence in Systems Engineering Research; Springer: Berlin, Germany, 2018; pp. 3–15. [Google Scholar]
  10. Sterbenz, J.P.; Hutchison, D.; Çetinkaya, E.K.; Jabbar, A.; Rohrer, J.P.; Schöller, M.; Smith, P. Resilience and survivability in communication networks: Strategies, principles, and survey of disciplines. Comput. Netw. 2010, 54, 1245–1265. [Google Scholar] [CrossRef]
  11. Castet, J.-F.; Saleh, J.H. On the concept of survivability, with application to spacecraft and space-based networks. Reliab. Eng. Syst. Saf. 2012, 99, 123–138. [Google Scholar] [CrossRef]
  12. Tahoori, M.; Gheidar-Kheljani, J.; Gavarashki, M.H.K. Design for Viability of Complex Engineered Systems under Uncertainty. Ind. Eng. Manag. Syst. 2017, 16, 619–631. [Google Scholar] [CrossRef]
  13. Uday, P.; Marais, K. Designing resilient systems-of-systems: A survey of metrics, methods, and challenges. Syst. Eng. 2015, 18, 491–510. [Google Scholar] [CrossRef]
  14. Ma, Z. Towards a unified definition for reliability, survivability and resilience (I): The conceptual framework inspired by the handicap principle and ecological stability. In Proceedings of the 2010 IEEE Aerospace Conference, Big Sky, MT, USA, 6–13 March 2010. [Google Scholar]
  15. Ross, A.M.; Stein, D.B.; Hastings, D.E. Multi-attribute tradespace exploration for survivability. J. Spacecr. Rocket. 2014, 51, 1735–1752. [Google Scholar] [CrossRef] [Green Version]
  16. Clédel, T.; Cuppens, N.; Cuppens, F.; Dagnas, R. Resilience properties and metrics: How far have we gone? J. Surveill. Secur. Saf. 2020, 1, 119–139. [Google Scholar] [CrossRef]
  17. Hohoniants, S.; Chopa, D.; Kilmeninov, O.; Loishyn, A.; Horbachov, K. Development of the Survivability Indicators Forecasting Method of the Special Purpose System Executive Element Based on Analytical and Stochastic Simulation of a Conflict Situation. East.-Eur. J. Enterp. Technol. 2021, 3, 14–23. [Google Scholar]
  18. Krings, A.; Harrison, W.; McQueen, M.; Matthews, S. Shifting the Focus of Survivability: Back to the Basics; University of Idaho: Moscow, ID, USA, 2001. [Google Scholar]
  19. Liu, Y.; Trivedi, K.S. Survivability quantification: The analytical modeling approach. Int. J. Perform. Eng. 2006, 2, 29. [Google Scholar]
  20. Richards, M.G.; Ross, A.M.; Shah, N.B.; Hastings, D.E. Metrics for evaluating survivability in dynamic multi-attribute tradespace exploration. J. Spacecr. Rocket. 2009, 46, 1049–1064. [Google Scholar] [CrossRef] [Green Version]
  21. Jones, L. Resilience isn’t the same for all: Comparing subjective and objective approaches to resilience measurement. Wiley Interdiscip. Rev. Clim. Change 2019, 10, e552. [Google Scholar] [CrossRef] [Green Version]
  22. Tariq, H.; Pathirage, C.; Fernando, T. Measuring community disaster resilience at local levels: An adaptable resilience framework. Int. J. Disaster Risk Reduct. 2021, 62, 102358. [Google Scholar] [CrossRef]
  23. Zuo, M. System reliability and system resilience. Front. Eng. Manag. 2021, 8, 615–619. [Google Scholar] [CrossRef]
  24. Uday, P.; Marais, K.B. Resilience-based system importance measures for system-of-systems. Procedia Comput. Sci. 2014, 28, 257–264. [Google Scholar] [CrossRef] [Green Version]
  25. Kitchenham, B.; Charters, S. Guidelines for Performing Systematic Literature Reviews in Software Engineering. 2007. Available online: https://www.elsevier.com/__data/promis_misc/525444systematicreviewsguide.pdf (accessed on 14 February 2023).
  26. Barbosa, G.; de Souza, É.F.; dos Santos, L.B.R.; da Silva, M.; Balera, J.M.; Vijaykumar, N.L. A Systematic Literature Review on prioritizing software test cases using Markov chains. Inf. Softw. Technol. 2022, 147, 106902. [Google Scholar] [CrossRef]
  27. Madni, A.M.; Jackson, S. Towards a conceptual framework for resilience engineering. IEEE Syst. J. 2009, 3, 181–191. [Google Scholar] [CrossRef]
  28. Woods, D.D. Four concepts for resilience and the implications for the future of resilience engineering. Reliab. Eng. Syst. Saf. 2015, 141, 5–9. [Google Scholar] [CrossRef]
  29. Faturechi, R.; Miller-Hooks, E. Measuring the performance of transportation infrastructure systems in disasters: A comprehensive review. J. Infrastruct. Syst. 2015, 21, 04014025. [Google Scholar] [CrossRef]
  30. Hosseini, S.; Barker, K.; Ramirez-Marquez, J.E. A review of definitions and measures of system resilience. Reliab. Eng. Syst. Saf. 2016, 145, 47–61. [Google Scholar] [CrossRef]
  31. Nemeth, C.P.; Herrera, I. Building change: Resilience Engineering after ten years. Reliab. Eng. Syst. Saf. 2015, 141, 1–4. [Google Scholar] [CrossRef]
  32. Patriarca, R.; Bergström, J.; Di Gravio, G.; Costantino, F. Resilience engineering: Current status of the research and future challenges. Saf. Sci. 2018, 102, 79–100. [Google Scholar] [CrossRef]
  33. Knight, J.C.; Sullivan, K.J. On the Definition of Survivability; Technical Report CS-TR-33-00; Department of Computer Science, University of Virginia: Charlottesville, VA, USA, 2000. [Google Scholar]
  34. Richards, M.G.; Hastings, D.; Rhodes, D.; Weigel, A. Defining survivability for engineering systems. In Proceedings of the 5th Conference on Systems Engineering Research, Hoboken, NJ, USA, 14–16 March 2007. [Google Scholar]
  35. Richards, M.G.; Hastings, D.E.; Ross, A.M.; Rhodes, D.H. Design principles for survivable system architecture. In Proceedings of the 2007 1st Annual IEEE Systems Conference, Waikiki Beach, Honolulu, HI, USA, 9–13 April 2007. [Google Scholar]
  36. Ellison, R.J.; Fisher, D.A.; Linger, R.C.; Lipson, H.F.; Longstaff, T. Survivable Network Systems: An Emerging Discipline. 1997. Available online: https://apps.dtic.mil/sti/pdfs/ADA341963.pdf (accessed on 1 February 2022).
  37. Firesmith, D.G. Common Concepts Underlying Safety Security and Survivability Engineering. 2003. Available online: https://apps.dtic.mil/sti/pdfs/ADA421683.pdf (accessed on 1 February 2022).
  38. Lipson, H.F.; Fisher, D.A. Survivability—A new technical and business perspective on security. In Proceedings of the 1999 Workshop on New Security Paradigms, Caledon Hills, ON, Canada, 22–24 September 1999. [Google Scholar]
  39. Hellmann, F.; Schultz, P.; Grabow, C.; Heitzig, J.; Kurths, J. Survivability of deterministic dynamical systems. Sci. Rep. 2016, 6, 1–12. [Google Scholar] [CrossRef] [Green Version]
  40. Knight, J.C.; Strunk, E.A.; Sullivan, K.J. Towards a rigorous definition of information system survivability. In Proceedings of the DARPA Information Survivability Conference and Exposition, Washington, DC, USA, 22–24 April 2003. [Google Scholar]
  41. Yaghlane, A.B.; Azaiez, M.N.; Mrad, M. System survivability in the context of interdiction networks. Reliab. Eng. Syst. Saf. 2019, 185, 362–371. [Google Scholar] [CrossRef]
  42. Cutter, S.L.; Barnes, L.; Berry, M.; Burton, C.; Evans, E.; Tate, E.; Webb, J. A place-based model for understanding community resilience to natural disasters. Glob. Environ. Change 2008, 18, 598–606. [Google Scholar] [CrossRef]
  43. Costella, M.F.; Saurin, T.A.; de Macedo Guimarães, L.B. A method for assessing health and safety management systems from the resilience engineering perspective. Saf. Sci. 2009, 47, 1056–1067. [Google Scholar] [CrossRef]
  44. Zarei, E.; Ramavandi, B.; Darabi, A.H.; Omidvar, M. A framework for resilience assessment in process systems using a fuzzy hybrid MCDM model. J. Loss Prev. Process Ind. 2021, 69, 104375. [Google Scholar] [CrossRef]
  45. Woods, D.D. Resilience is a verb. In Domains of Resilience for Complex Interconnected Systems; International Risk Governance Center: Lausanne, Switzerland, 2018; Volume 2, pp. 167–172. [Google Scholar]
  46. Yu, D.J.; Schoon, M.L.; Hawes, J.K.; Lee, S.; Park, J.; Rao, P.S.C.; Siebeneck, L.K.; Ukkusuri, S.V. Toward general principles for resilience engineering. Risk Anal. 2020, 40, 1509–1537. [Google Scholar] [CrossRef]
  47. Arcuri, R.; Bellas, H.C.; de Souza Ferreira, D.; Bulhões, B.; Vidal, M.C.; Rodrigues de Carvalho, P.V.; Jatobá, A.; Hollnagel, E. On the brink of disruption: Applying Resilience Engineering to anticipate system performance under crisis. Appl. Ergon. 2022, 99, 103632. [Google Scholar] [CrossRef] [PubMed]
  48. Hollnagel, E.; Woods, D.; Leveson, N. Resilience engineering in a nutshell. Resil. Eng. Perspect. 2008, 1, 1–4. [Google Scholar]
  49. Woods, D.D. Essential characteristics of resilience. In Resilience Engineering; CRC Press: Boca Raton, FL, USA, 2017; pp. 21–34. [Google Scholar]
  50. Grecco, C.H.; Vidal, M.C.; Santos, I.J.; Carvalho, P.V. A method to assess safety and resilience in radiopharmaceuticals production process. Work 2012, 41, 5839–5843. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  51. Reason, J. Understanding adverse events: Human factors. BMJ Qual. Saf. 1995, 4, 80–89. [Google Scholar] [CrossRef] [Green Version]
  52. Morlok, E.K.; Chang, D.J. Measuring capacity flexibility of a transportation system. Transp. Res. Part A: Policy Pract. 2004, 38, 405–420. [Google Scholar] [CrossRef]
  53. Makhutov, N.; Gadenin, M. Integrated assessment of the durability, resources, survivability, and safety of machinery loaded under complex conditions. J. Mach. Manuf. Reliab. 2020, 49, 292–300. [Google Scholar] [CrossRef]
  54. De Weck, O.L.; Ross, A.M.; Rhodes, D.H. Investigating relationships and semantic sets amongst system lifecycle properties (ilities). In Proceedings of the Third International Engineering Systems Symposium, Delft, The Netherlands, 18–20 June 2012. [Google Scholar]
  55. Rose, A. Economic resilience to natural and man-made disasters: Multidisciplinary origins and contextual dimensions. Environ. Hazards 2007, 7, 383–398. [Google Scholar] [CrossRef]
  56. Whitson, J.C.; Ramirez-Marquez, J.E. Resiliency as a component importance measure in network reliability. Reliab. Eng. Syst. Saf. 2009, 94, 1685–1693. [Google Scholar] [CrossRef]
  57. Bruneau, M.; Chang, S.E.; Eguchi, R.T.; Lee, G.C.; O’Rourke, T.D.; Reinhorn, A.M.; Shinozuka, M.; Tierney, K.; Wallace, W.A.; von Winterfeldt, D. A framework to quantitatively assess and enhance the seismic resilience of communities. Earthq. Spectra 2003, 19, 733–752. [Google Scholar] [CrossRef] [Green Version]
  58. Hazelrigg, G.A.; Saari, D.G. Toward a theory of systems engineering. J. Mech. Des. 2022, 144, 011402. [Google Scholar] [CrossRef]
  59. Darling Rasmussen, P.; Storebø, O.J.; Løkkeholt, T.; Voss, L.G.; Shmueli-Goetz, Y.; Bojesen, A.B.; Simonsen, E.; Bilenberg, N. Attachment as a core feature of resilience: A systematic review and meta-analysis. Psychol. Rep. 2019, 122, 1259–1296. [Google Scholar] [CrossRef] [PubMed]
  60. Dinh, L.T.; Pasman, H.; Gao, X.; Mannan, M.S. Resilience engineering of industrial processes: Principles and contributing factors. J. Loss Prev. Process Ind. 2012, 25, 233–241. [Google Scholar] [CrossRef]
  61. Herrman, H.; Stewart, D.E.; Diaz-Granados, N.; Berger, E.L.; Jackson, B.; Yuen, T. What is resilience? Can. J. Psychiatry 2011, 56, 258–265. [Google Scholar] [CrossRef] [Green Version]
  62. McCubbin, L. Challenges to the Definition of Resilience. 2001. Available online: https://files.eric.ed.gov/fulltext/ED458498.pdf (accessed on 1 May 2022).
  63. Pan, S.; Yan, H.; He, J.; He, Z. Vulnerability and resilience of transportation systems: A recent literature review. Phys. A: Stat. Mech. Its Appl. 2021, 581, 126235. [Google Scholar] [CrossRef]
  64. Woods, D. Engineering organizational resilience to enhance safety: A progress report on the emerging field of resilience engineering. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, San Francisco, CA, USA, 16–20 October 2006. [Google Scholar]
  65. Oehmen, J.; Kwakkel, J. Risk, uncertainty, and ignorance in engineering systems design. In Handbook of Engineering Systems Design; Springer: Cham, Switzerland, 2021; pp. 1–31. [Google Scholar]
  66. Sharma, M.; Luthra, S.; Joshi, S.; Kumar, A. Developing a framework for enhancing survivability of sustainable supply chains during and post-COVID-19 pandemic. Int. J. Logist. Res. Appl. 2022, 25, 433–453. [Google Scholar] [CrossRef]
  67. Wied, M.; Oehmen, J.; Welo, T. Conceptualizing resilience in engineering systems: An analysis of the literature. Syst. Eng. 2020, 23, 3–13. [Google Scholar] [CrossRef] [Green Version]
  68. Folke, C.; Carpenter, S.R.; Walker, B.; Scheffer, M.; Chapin, T.; Rockström, J. Resilience thinking: Integrating resilience, adaptability and transformability. Ecol. Soc. 2010, 15, 20. [Google Scholar] [CrossRef]
  69. Ham, D.-H. Safety-II and resilience engineering in a nutshell: An introductory guide to their concepts and methods. Saf. Health Work 2021, 12, 10–19. [Google Scholar] [CrossRef]
  70. Hollnagel, E. Resilience engineering: A new understanding of safety. J. Ergon. Soc. Korea 2016, 35, 185–191. [Google Scholar] [CrossRef]
  71. Izadi, M.; Hosseinian, S.H.; Dehghan, S.; Fakharian, A.; Amjady, N. A critical review on definitions, indices, and uncertainty characterization in resiliency-oriented operation of power systems. Int. Trans. Electr. Energy Syst. 2021, 31, e12680. [Google Scholar] [CrossRef]
  72. Plotnek, J.J.; Slay, J. Power systems resilience: Definition and taxonomy with a view towards metrics. Int. J. Crit. Infrastruct. Prot. 2021, 33, 100411. [Google Scholar] [CrossRef]
  73. Righi, A.W.; Saurin, T.A.; Wachs, P. A systematic literature review of resilience engineering: Research areas and a research agenda proposal. Reliab. Eng. Syst. Saf. 2015, 141, 142–152. [Google Scholar] [CrossRef]
  74. Cimellaro, G.P.; Reinhorn, A.M.; Bruneau, M. Framework for analytical quantification of disaster resilience. Eng. Struct. 2010, 32, 3639–3649. [Google Scholar] [CrossRef]
  75. Doorn, N.; Gardoni, P.; Murphy, C. A multidisciplinary definition and evaluation of resilience: The role of social justice in defining resilience. Sustain. Resilient Infrastruct. 2019, 4, 112–123. [Google Scholar] [CrossRef] [Green Version]
  76. Peñaloza, G.A.; Formoso, C.T.; Saurin, T.A. Safety performance measurement systems based on resilience engineering. In Proceedings of the 25th Annual Conference of the International Group of Lean Construction (IGLC), Heraklion, Greece, 9–12 July 2017. [Google Scholar]
  77. Rabbani, M.; Yazdanparast, R.; Mobini, M. An algorithm for performance evaluation of resilience engineering culture based on graph theory and matrix approach. Int. J. Syst. Assur. Eng. Manag. 2019, 10, 228–241. [Google Scholar] [CrossRef]
  78. Ranasinghe, U.; Jefferies, M.; Davis, P.; Pillay, M. Resilience engineering indicators and safety management: A systematic review. Saf. Health Work 2020, 11, 127–135. [Google Scholar] [CrossRef]
  79. Shirali, G.A.; Motamedzade, M.; Mohammadfam, I.; Ebrahimipour, V.; Moghimbeigi, A. Assessment of resilience engineering factors based on system properties in a process industry. Cogn. Technol. Work 2016, 18, 19–31. [Google Scholar] [CrossRef]
  80. Ayyub, B.M. Systems resilience for multihazard environments: Definition, metrics, and valuation for decision making. Risk Anal. 2014, 34, 340–355. [Google Scholar] [CrossRef]
  81. Bruneau, M.; Reinhorn, A. Overview of the resilience concept. In Proceedings of the 8th US National Conference on Earthquake Engineering, San Francisco, CA, USA, 18–22 April 2006. [Google Scholar]
  82. Cheng, Y.; Elsayed, E.A.; Huang, Z. Systems resilience assessments: A review, framework and metrics. Int. J. Prod. Res. 2022, 60, 595–622. [Google Scholar] [CrossRef]
  83. Jasiūnas, J.; Lund, P.D.; Mikkola, J. Energy system resilience–A review. Renew. Sustain. Energy Rev. 2021, 150, 111476. [Google Scholar] [CrossRef]
  84. Said, S.; Bouloiz, H.; Gallab, M. A new structure of sociotechnical system processes using resilience engineering. Int. J. Eng. Bus. Manag. 2019, 11, 1847979019827151. [Google Scholar] [CrossRef]
  85. Wang, Z.; Nistor, M.S.; Pickl, S.W. Analysis of the definitions of resilience. IFAC-Pap. 2017, 50, 10649–10657. [Google Scholar] [CrossRef]
  86. Feldman, R. What is resilience: An affiliative neuroscience approach. World Psychiatry 2020, 19, 132–150. [Google Scholar] [CrossRef]
  87. Fiksel, J. Sustainability and resilience: Toward a systems approach. Sustain. Sci. Pract. Policy 2006, 2, 14–21. [Google Scholar] [CrossRef]
  88. Lee, J.; Siahpour, S.; Jia, X.; Brown, P. Introduction to resilient manufacturing systems. Manuf. Lett. 2022, 32, 24–27. [Google Scholar] [CrossRef]
  89. Meerow, S.; Newell, J.P.; Stults, M. Defining urban resilience: A review. Landsc. Urban Plan. 2016, 147, 38–49. [Google Scholar] [CrossRef]
  90. Patterson, E.S.; Woods, D.D.; Cook, R.I.; Render, M.L. Collaborative cross-checking to enhance resilience. Cogn. Technol. Work. 2007, 9, 155–162. [Google Scholar] [CrossRef]
  91. Pillay, M. Resilience engineering: An integrative review of fundamental concepts and directions for future research in safety management. Open J. Saf. Sci. Technol. 2018, 7, 129–160. [Google Scholar] [CrossRef] [Green Version]
  92. Pooley, J.A.; Cohen, L. Resilience: A definition in context. Aust. Community Psychol. 2010, 22, 30–37. [Google Scholar]
  93. Rubio-Romero, J.C.; Pardo-Ferreira, M.d.C.; Rojas, M.M.; López-Arquillos, A.; Suarez-Cebador, M. Resilience engineering: Concepts of the new paradigm. In Engineering Digital Transformation; Springer: Berlin, Germany, 2019; pp. 133–140. [Google Scholar]
  94. Rudd, G.; Meissel, K.; Meyer, F. Measuring academic resilience in quantitative research: A systematic review of the literature. Educ. Res. Rev. 2021, 34, 100402. [Google Scholar] [CrossRef]
  95. Schafer, D.; Abdelhamid, T.; Mitropoulos, P.; Howell, G. Resilience engineering: A new paradigm for safety in lean construction systems. In Proceedings of the 16th Annual Conference of the International Group for Lean Construction, Manchester, UK, 16–18 July 2008. [Google Scholar]
  96. Southwick, S.M.; Bonanno, G.A.; Masten, A.S.; Panter-Brick, C.; Yehuda, R. Resilience definitions, theory, and challenges: Interdisciplinary perspectives. Eur. J. Psychotraumatology 2014, 5, 25338. [Google Scholar] [CrossRef] [Green Version]
  97. Zautra, A.J.; Hall, J.S.; Murray, K.E. Resilience: A new definition of health for people and communities. In Handbook of Adult Resilience; The Guilford Press: New York, NY, USA, 2010; pp. 3–29. [Google Scholar]
  98. Zohuri, B.; Moghaddam, M.; Mossavar-Rahmani, F. Business Resilience System Integrated Artificial Intelligence System. Int. J. Theor. Comput. Phys. 2022, 3, 1–7. [Google Scholar]
  99. Linger, R.C.; Ellison, R.J.; Longstaff, T.A.; Mead, N.R. The survivability imperative: Protecting critical systems. Crosstalk 2000, 13, 12–15. [Google Scholar]
  100. Richards, M.G.; Ross, A.M.; Hastings, D.E.; Rhodes, D.H. Multi-Attribute Tradespace Exploration for Survivability; Massachusetts Institute of Technology, Engineering Systems Division: Cambridge, MA, USA, 2009. [Google Scholar]
  101. Ellison, R.; Linger, R.; Lipson, H.; Mead, N.; Moore, A. Foundations for survivable systems engineering. J. Def. Softw. Eng. 2002, 10–15. [Google Scholar]
  102. Mead, N.R. Requirements Engineering for Survivable Systems. Networked Systems Survivability. Available online: https://resources.sei.cmu.edu/asset_files/TechnicalNote/2003_004_001_14153.pdf (accessed on 1 February 2022).
  103. Mead, N.R.; Ellison, R.; Linger, R.C.; Lipson, H.F.; McHugh, J. Life-cycle models for survivable systems. In Proceedings of the Third Information Survivability, Workshop (ISW-2000), Boston, MA, USA, 24–26 October 2000. [Google Scholar]
  104. Redman, J.; Warren, M.; Hutchinson, W. System survivability: A critical security problem. Inf. Manag. Comput. Secur. 2005, 13, 182–188. [Google Scholar] [CrossRef]
  105. Westmark, V.R. A definition for information system survivability. In Proceedings of the 37th Annual Hawaii International Conference on System Sciences, Big Island, HI, USA, 5–8 January 2004. [Google Scholar]
  106. Ivanov, D.; Dolgui, A. Viability of intertwined supply networks: Extending the supply chain resilience angles towards survivability. A position paper motivated by COVID-19 outbreak. Int. J. Prod. Res. 2020, 58, 2904–2915. [Google Scholar] [CrossRef] [Green Version]
  107. Peshkov, V. Organization of work on the development of measures to ensure the survivability of buildings exposed to flooding. IOP Conf. Ser. Mater. Sci. Eng. 2020, 1011, 011001. [Google Scholar] [CrossRef]
  108. Woolley, A.; Ewer, J.; Lawrence, P.; Travers, A.; Deere, S.; Whitehouse, T.; Galea, E. A modelling and simulation framework to assess naval platform integrated survivability. In Proceedings of the Maritime/Air Systems & Technologies Conference, Amsterdam, The Netherlands, 21–23 June 2016. [Google Scholar]
  109. Bulian, G.; Cardinale, M.; Dafermos, G.; Lindroth, D.; Ruponen, P.; Zaraphonitis, G. Probabilistic assessment of damaged survivability of passenger ships in case of grounding or contact. Ocean Eng. 2020, 218, 107396. [Google Scholar] [CrossRef]
  110. Woodard, M.; Marashi, K.; Sarvestani, S.S.; Hurson, A.R. Survivability evaluation and importance analysis for cyber–physical smart grids. Reliab. Eng. Syst. Saf. 2021, 210, 107479. [Google Scholar] [CrossRef]
  111. Levitin, G.; Dai, Y.; Xie, M.; Poh, K.L. Optimizing survivability of multi-state systems with multi-level protection by multi-processor genetic algorithm. Reliab. Eng. Syst. Saf. 2003, 82, 93–104. [Google Scholar] [CrossRef]
  112. Beer, S. The viable system model: Its provenance, development, methodology and pathology. J. Oper. Res. Soc. 1984, 35, 7–25. [Google Scholar] [CrossRef]
  113. Rezaee, Z.; Azar, A.; Erz, A.M.B.; Nayeri, M.D. Application of viable system model in diagnosis of organizational structure. Syst. Pract. Action Res. 2019, 32, 273–295. [Google Scholar] [CrossRef]
  114. Czainska, K.; Sus, A.; Thalassinos, E.I. Sustainable Survival: Resource Management Strategy in Micro and Small Enterprises in the Rubber Products Market in Poland during the COVID-19 Pandemic. Resources 2021, 10, 85. [Google Scholar] [CrossRef]
  115. Annarelli, A.; Battistella, C.; Nonino, F. A framework to evaluate the effects of organizational resilience on service quality. Sustainability 2020, 12, 958. [Google Scholar] [CrossRef] [Green Version]
  116. Huang, A.; Farboudi Jahromi, M. Resilience building in service firms during and post COVID-19. Serv. Ind. J. 2021, 41, 138–167. [Google Scholar] [CrossRef]
  117. Pawar, B.; Park, S.; Hu, P.; Wang, Q. Applications of resilience engineering principles in different fields with a focus on industrial systems: A literature review. J. Loss Prev. Process Ind. 2021, 69, 104366. [Google Scholar] [CrossRef]
  118. Belhadi, A.; Kamble, S.; Jabbour, C.J.C.; Gunasekaran, A.; Ndubisi, N.O.; Venkatesh, M. Manufacturing and service supply chain resilience to the COVID-19 outbreak: Lessons learned from the automobile and airline industries. Technol. Forecast. Soc. Change 2021, 163, 120447. [Google Scholar] [CrossRef]
  119. Huang, C.-N.; Liou, J.J.; Lo, H.-W.; Chang, F.-J. Building an assessment model for measuring airport resilience. J. Air Transp. Manag. 2021, 95, 102101. [Google Scholar] [CrossRef]
  120. Singh, C.S.; Soni, G.; Badhotiya, G.K. Performance indicators for supply chain resilience: Review and conceptual framework. J. Ind. Eng. Int. 2019, 15, 105–117. [Google Scholar] [CrossRef] [Green Version]
  121. Romero, D.; Stahre, J.; Larsson, L.; Rönnbäck, A.Ö. Building Manufacturing Resilience through Production Innovation. In Proceedings of the 2021 IEEE International Conference on Engineering, Technology and Innovation (ICE/ITMC), Cardiff, UK, 21–23 June 2021. [Google Scholar]
  122. Chacon-Hurtado, D.; Losada-Rojas, L.L.; Yu, D.; Gkritza, K.; Fricker, J.D. A proposed framework for the incorporation of economic resilience into transportation decision making. J. Manag. Eng. 2020, 36, 04020084. [Google Scholar] [CrossRef]
  123. Erker, S.; Stangl, R.; Stoeglehner, G. Resilience in the light of energy crises—Part I: A framework to conceptualise regional energy resilience. J. Clean. Prod. 2017, 164, 420–433. [Google Scholar] [CrossRef]
  124. Erker, S.; Stangl, R.; Stoeglehner, G. Resilience in the light of energy crises—Part II: Application of the regional energy resilience assessment. J. Clean. Prod. 2017, 164, 495–507. [Google Scholar] [CrossRef]
  125. Jin, L.; Zhang, G.; Wang, J.; Zhu, H.; Duan, W. Quantitative survivability analysis using probability model checking: A study of cluster-based vehicle networks with dual cluster heads. China Commun. 2020, 17, 206–219. [Google Scholar] [CrossRef]
  126. Payne, P.; Kaye-Blake, W.; Kelsey, A.; Brown, M.; Niles, M. Measuring rural community resilience: Case studies in New Zealand and Vermont, USA. Ecol. Soc. 2021, 26, 2. [Google Scholar] [CrossRef]
  127. Henry, D.; Ramirez-Marquez, J.E. Generic metrics and quantitative approaches for system resilience as a function of time. Reliab. Eng. Syst. Saf. 2012, 99, 114–122. [Google Scholar] [CrossRef]
  128. Meyer, K. A mathematical review of resilience in ecology. Nat. Resour. Model. 2016, 29, 339–352. [Google Scholar] [CrossRef]
  129. Ferris, T.L. A resilience measure to guide system design and management. IEEE Syst. J. 2019, 13, 3708–3715. [Google Scholar] [CrossRef] [Green Version]
  130. Ferris, T.L. Measurement of Resilience and the Time Value of Resilience. IEEE Syst. J. 2020, 15, 1578–1585. [Google Scholar] [CrossRef]
  131. Pant, R.; Barker, K.; Zobel, C.W. Static and dynamic metrics of economic resilience for interdependent infrastructure and industry sectors. Reliab. Eng. Syst. Saf. 2014, 125, 92–102. [Google Scholar] [CrossRef]
  132. Pumpuni-Lenss, G.; Blackburn, T.; Garstenauer, A. Resilience in complex systems: An agent-based Approach. Syst. Eng. 2017, 20, 158–172. [Google Scholar] [CrossRef]
  133. Shi, Q.; Liu, W.; Zeng, B.; Hui, H.; Li, F. Enhancing distribution system resilience against extreme weather events: Concept review, algorithm summary, and future vision. Int. J. Electr. Power Energy Syst. 2022, 138, 107860. [Google Scholar] [CrossRef]
  134. Chih, Y.-Y.; Hsiao, C.Y.-L.; Zolghadr, A.; Naderpajouh, N. Resilience of Organizations in the Construction Industry in the Face of COVID-19 Disturbances: Dynamic Capabilities Perspective. J. Manag. Eng. 2022, 38, 04022002. [Google Scholar] [CrossRef]
  135. Hohoniants, S.; Repilo, I.; Tytarenko, O.; Kokoiko, A.; Golovchenko, O. Improving a method for determining the maneuvering intensity of the executive element of a special-purpose system. East. -Eur. J. Enterp. Technol. 2021, 5, 75–83. [Google Scholar]
  136. Zhao, G.; Wang, H.; Wang, J. A novel quantitative analysis method for network survivability. In Proceedings of the First International Multi-Symposiums on Computer and Computational Sciences (IMSCCS), Hangzhou, China, 20–24 June 2006. [Google Scholar]
  137. Zhang, L.-J.; Wang, W.; Guo, L.; Yang, W.; Yang, Y.-T. A survivability quantitative analysis model for network system based on attack graph. In Proceedings of the 2007 International Conference on Machine Learning and Cybernetics, Hong Kong, China, 19–22 August 2007. [Google Scholar]
  138. Bisikalo, O.V.; Kovtun, V.V.; Kovtun, O.V.; Romanenko, V.B. Research of safety and survivability models of the information system for critical use. In Proceedings of the 2020 IEEE 11th International Conference on Dependable Systems 2020, Services and Technologies (DESSERT), Kyiv, Ukraine, 14–18 May 2020. [Google Scholar]
  139. Rumawas, V.; Asbjørnslett, B.E. Survivability of ships at sea: A proposed model to account for human factors in a safety critical system. Int. J. Marit. Eng. 2014, 156, A137–A147. [Google Scholar] [CrossRef]
  140. Saleh, S.A.; Chowdhury, M.R. Survivability Analysis of Impacts of Load-Side Activities on Power Systems. IEEE Trans. Ind. Appl. 2022, 58, 1869–1878. [Google Scholar] [CrossRef]
  141. Trivedi, K.S.; Xia, R. Quantification of system survivability. Telecommun. Syst. 2015, 60, 451–470. [Google Scholar] [CrossRef]
  142. Chang, X.; Lv, S.; Rodríguez, R.J.; Trivedi, K. Survivability model for security and dependability analysis of a vulnerable critical system. In Proceedings of the 2018 27th International Conference on Computer Communication and Networks (ICCCN), Hangzhou, China, 30 July–2 August 2018. [Google Scholar]
  143. Zhao, X.; Fan, Y.; Qiu, Q.; Chen, K. Multi-criteria mission abort policy for systems subject to two-stage degradation process. Eur. J. Oper. Res. 2021, 295, 233–245. [Google Scholar] [CrossRef]
  144. Le Thanh, H.; Ngoc, T.D.; Trung, T.T. How to improve the survivability of environmentally innovative firms: The case of Vietnam’s SMEs. J. Clean. Prod. 2022, 362, 132223. [Google Scholar] [CrossRef]
  145. Ríos, M.; Marianov, V.; Gutierrez, M. Survivable capacitated network design problem: New formulation and Lagrangean relaxation. J. Oper. Res. Soc. 2000, 51, 574–582. [Google Scholar] [CrossRef]
  146. Santiváñez, J.A.; Melachrinoudis, E. Embedding Network Resilience through Locational Decisions. In Proceedings of the IIE Annual Conference, Nashville, TN, USA, 30 May–2 June 2015. [Google Scholar]
  147. Wang, J.; Gao, F.; Ip, W. Measurement of resilience and its application to enterprise information systems. Enterp. Inf. Syst. 2010, 4, 215–223. [Google Scholar] [CrossRef]
  148. Boin, A.; Comfort, L.K.; Demchak, C.C. The rise of resilience. Des. Resil. Prep. Extrem. Events 2010, 1, 1–12. [Google Scholar]
  149. Broccardo, M.; Galanis, P.; Esposito, S.; Stojadinovic, B. Probabilistic resilience assessment of civil systems: Analysis and validity of the PEER framework. In Safety and Reliability of Complex Engineered Systems: ESREL 2015; CRC Press: Boca Raton, FL, USA, 2015; Volume 331. [Google Scholar]
  150. Hickford, A.J.; Blainey, S.P.; Ortega Hortelano, A.; Pant, R. Resilience engineering: Theory and practice in interdependent infrastructure systems. Environ. Syst. Decis. 2018, 38, 278–291. [Google Scholar] [CrossRef] [Green Version]
Figure 1. Bucket classification model.
Figure 1. Bucket classification model.
Sustainability 15 04811 g001
Figure 2. Selection rule logic.
Figure 2. Selection rule logic.
Sustainability 15 04811 g002
Figure 3. Distribution of references according to the theme.
Figure 3. Distribution of references according to the theme.
Sustainability 15 04811 g003
Figure 4. Distribution of references according to the application.
Figure 4. Distribution of references according to the application.
Sustainability 15 04811 g004
Figure 5. Distribution of references according to the country.
Figure 5. Distribution of references according to the country.
Sustainability 15 04811 g005
Figure 6. Distribution of references according to US state.
Figure 6. Distribution of references according to US state.
Sustainability 15 04811 g006
Figure 7. Venn diagram of resilience vs. survivability.
Figure 7. Venn diagram of resilience vs. survivability.
Sustainability 15 04811 g007
Figure 8. Graphical depictions for resilience, survivability, and resurgence.
Figure 8. Graphical depictions for resilience, survivability, and resurgence.
Sustainability 15 04811 g008
Table 1. Definitions for Resilience.
Table 1. Definitions for Resilience.
Resilience
View (and Keywords)DefinitionTop 10 References
(1) Elasticity or (2) flexibility property[…] the property of a system to return to its original state after any deformation.Madni and Jackson [27]; Uday and Marais [13,24]; Woods [28]
(3) Static and (4) dynamic resilience[…] the ability of a system to continue delivering value even through disruption and/or repair itself after suffering harm.Uday and Marais [13,24]
(5) Four-A features[…] the capacity or intelligence the system must have to expect, transform from, withstand, and mitigate a disruption. Faturechi and Miller-Hooks [29]; Jones [21]; Small et al. [9]
(6) Dimensions of a resilient system[…] the collection of attributes and activities that enables any system to be adaptable, reliable, agile, effective, flexible, and responsive according to a certain recovery level in an expected time range.Hosseini et al. [30]
(7) Openness to tradeoffs[…] the measure of a system’s ability to maintain its vital functions while it degrades within acceptable parameters and may recover soon without incurring in major costs and risks. Faturechi and Miller-Hooks [29]; Hosseini et al. [30]; Jones [21]
Heuristics based on (8) experience and (9) judgement[…] the ability of either an individual, a firm, a group such as communities, flora, and fauna to find stability and cope with traumatic events in a way that embraces challenges and changes as these fortify their environment. Nemeth and Herrera [31]; Patriarca et al. [32]
[…] subjective embodiment of requisites needed to accomplish a mission, comply with standards and requirements, pay a service, and/or fulfill a certain purpose.Jones [21]; Patriarca et al. [32]
(10) Failure as the inability to absorb disruptions or adapt to changes[…] simply a different way to see failure; failure can tell how resilient a system is by gauging levels of absorption and adaptation.Madni and Jackson [27]
Table 2. Definitions for Survivability.
Table 2. Definitions for Survivability.
Survivability
View (and Keywords)DefinitionTop 10 References
(1) Resurgence property[…] the point where a system has gone below its breaking point, but after some indefinite time, it comes back as something completely new with entirely redefined properties and tolerances.No reference.
(2) Active and (3) passive survivability[…] the ability of a system to diminish the impact of a disturbance on value delivery and repair itself in an effective manner or the innate capability to cope and resist uncertainties and threats coming from the outside world before they happen; systems can be reactive or proactive before a disruption. Knight and Sullivan [33]; Richards et al. [34]; Richards et al. [35]
(4) Minimum essential set of means to provide autonomous performance[…] the degree to which a system protects its essential services from malicious harm and accidents; essential services keep delivering value despite either accidental or malicious harm affecting the system.Ellison et al. [36]; Firesmith [37]
(5) 1 of 4 non-functional requirements (i.e., viability, flexibility, dependability, survivability, or other variants)[…] the tradeoff among several quality attributes which can be either functional or non-functional requirements to complete a mission. Lipson and Fisher [38]
(6) Measured performance over time[…] a stability-based measure where a fraction of initial states of perturbation give rise to evolutions that stay within a desirable regime of performance that extends to a period. Hellmann et al. [39]
“Failure” dependent on (7) how fast and (8) how much value or net benefit the system can deliver to its stakeholders[…] a measurable tradeoff between functionality and resources which complies with an acceptable service value, service transitions, service environments, service specifications, service options, and probabilities of mission breakthrough.Ellison et al. [36]; Knight et al. [40]; Yaghlane et al. [41]
(9) “Failure” as a threat that affects value delivery[…] an emergent property that cannot be achieved at an atomic level by only addressing a single failing component; each component will fight for its own survival, but the intended system function must be met.Faturechi and Miller-Hooks [29]; Lipson and Fisher [38]; Yaghlane et al. [41]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Martínez, K.; Claudio, D. Expanding Fundamental Boundaries between Resilience and Survivability in Systems Engineering: A Literature Review. Sustainability 2023, 15, 4811. https://doi.org/10.3390/su15064811

AMA Style

Martínez K, Claudio D. Expanding Fundamental Boundaries between Resilience and Survivability in Systems Engineering: A Literature Review. Sustainability. 2023; 15(6):4811. https://doi.org/10.3390/su15064811

Chicago/Turabian Style

Martínez, Kenneth, and David Claudio. 2023. "Expanding Fundamental Boundaries between Resilience and Survivability in Systems Engineering: A Literature Review" Sustainability 15, no. 6: 4811. https://doi.org/10.3390/su15064811

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop