Next Article in Journal
Rethinking Suicide in Rural Australia: A study Protocol for Examining and Applying Knowledge of the Social Determinants to Improve Prevention in Non-Indigenous Populations
Next Article in Special Issue
The Impact of Activity-Based Mobility Pattern on Assessing Fine-Grained Traffic-Induced Air Pollution Exposure
Previous Article in Journal
Self-Selected Pacing during a 24 h Track Cycling World Record
Previous Article in Special Issue
Investigation on the Injury Severity of Drivers in Rear-End Collisions Between Cars Using a Random Parameters Bivariate Ordered Probit Model
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:

Risk Assessment in Urban Large-Scale Public Spaces Using Dempster-Shafer Theory: An Empirical Study in Ningbo, China

School of Civil and Transportation Engineering, Ningbo University of Technology, Ningbo 315211, China
College of Transportation Engineering, Tongji University, Shanghai 200082, China
Intelligent Transport System (ITS) R & D Center, Shanghai Urban Construction Design and Research Institute (Group) Co., Ltd., Shanghai 200082, China
School of Economics and Management, Chang’an University, Xi’an 710064, China
Department of Civil and Environmental Engineering, University of Waterloo, Waterloo, ON N2L 3G1, Canada
Author to whom correspondence should be addressed.
Int. J. Environ. Res. Public Health 2019, 16(16), 2942;
Submission received: 12 July 2019 / Revised: 13 August 2019 / Accepted: 14 August 2019 / Published: 16 August 2019


Urban Large-scale Public Spaces (ULPS) are important areas of urban culture and economic development, which are also places of the potential safety hazard. ULPS safety assessment has played a crucial role in the theory and practice of urban sustainable development. The primary objective of this study is to explore the interaction between ULPS safety risk and its influencing factors. In the first stage, an index sensitivity analysis method was applied to calculate and identify the safety risk assessment index system. Next, a Delphi method and information entropy method were also applied to collect and calculate the weight of risk assessment indicators. In the second stage, a Dempster-Shafer Theory (DST) method with evidence fusion technique was utilized to analyze the interaction between the ULPS safety risk level and the multiple-index variables, measured by four observed performance indicators, i.e., environmental factor, human factor, equipment factor, and management factor. Finally, an empirical study of DST approach for ULPS safety performance analysis was presented.

1. Introduction

Urban Large-scale Public Spaces (ULPS) are important carriers of urban culture and economic development, which are also an indispensable part of the cultural and economic life of the citizens. With the continuous increase of urban population and the higher frequency of social interaction, a variety of large-scale activities have promoted the construction of ULPS (e.g., theaters, stadiums, railway stations, subway stations, commercial streets, and supermarkets, etc.). According to the rough statistics in the ULPS all over the world from 2000 to 2018 shown in Table 1 [1], many kinds of risk incidents such as fire accidents and trampling incidents often occur. Therefore, the risk assessment issue in large-scale public spaces has become a hot topic at present [2].
Urban large-scale public spaces safety assessment has played a crucial role in the theory and practice of urban sustainable development. It is necessary to identify potential safety hazards in large-scale public spaces, so that relevant management departments would take measures to avoid risks promptly. It is of great significance to carry out research on safety risk assessment in ULPS, as follows: (1) to evaluate the current situation of ULPS and promote the construction of safety guarantee system; (2) to prevent the occurrence of various risk events and carry out active safety control; and (3) to promote the spread of risk knowledge in ULPS, and raise public awareness of risks. In addition, it also has important theoretical value and social significance for promoting the development of urban disaster prevention and mitigation.
Due to ULPS’s own architectural structure, spatial layout, personnel and wealth distribution, ULPS are vulnerable to various internal and external risks, such as fire, flood, explosion, and crowding. A large number of facts [1,2] have proved that, although the ULPS have a low probability of occurrence, it will inevitably cause irreparable casualties or property losses in the event of accidents. Hence, it is very helpful to identify and evaluate the possible risks in ULPS, which will not only help to assess the current situation of large-scale public places and promote the construction of safety guarantee system, but also prevent the occurrence of various risky incidents and carry out active safety regulation, and improve the level of information about risk knowledge of large-scale public spaces and enhance public awareness of risks.
Unfortunately, at present, although urban managers attach great importance to the safety risk assessment of ULPS, there are many shortcomings in the current research, such as risk assessment index system, advanced and feasible assessment methods. On the one hand, ULPS are important areas of urban culture and economic development, which are characterized by high-density gathering, high mobility, and high concentration, potential safety risks are associated with operations and production activities in large public spaces. On the other hand, the safety risk assessment of ULPS is a complex process involving multiple factors, multiple indicators, and uncertainties, many indicators are often difficult to accurately quantify and compare the risk analysis.
To fill this gap, we have divided the whole evacuation process for a ULPS into two periods. For the first period, we proposed a two-level risk assessment index system for the ULPS, which is calculated and identified by an index sensitivity analysis method; and next, a Delphi method and information entropy method were also applied to collect and calculate the weight of risk assessment indicators. For the second period, we have employed a Dempster-Shafer Theory (DST) method to integrate multi-source uncertainty information and apply this method to the security risk assessment of large-scale public spaces, which also helps calculate joint information from the sets of mass sources and conduct evidence fusion for all levels of risk indicators.
This research makes the following two contributions. Firstly, a scientific and practical risk assessment indicator system (RAIS) for ULPS was proposed, which includes four first-level indicators and 20 second-level indicators. Secondly, a Dempster-Shafer Theory (DST) with evidence fusion technique was employed to analyze the interaction between the RAIS and risk level in the ULPS. Conclusively, an empirical study of DST approach for ULPS safety performance analysis was verified and presented.
The remainder of this paper is organized as follows. Section 2 reviews the existing studies on risk analysis method and visualization of risk analysis. Section 3 introduces the Dempster-Shafer theory, identifies the risk assessment index system and determines the weight of risk assessment indicators. Section 4 describes a numerical example and presents the risk assessment results. Section 5 discusses the results obtained from the model. Section 6 draws the conclusions.

2. Related Studies

2.1. Risk and Risk Assessment Analysis

Risk analysis is related to the survival of humans and loss of property on ULPS, and during public place risk assessment, “risk” is associated with a number of factors or indicators [3], such as environmental factors, human factors, infrastructure factors, management factors and known as a public place safety outcome. In the field of urban large-scale public spaces safety, the “risk assessment” [4] is defined as identify and prevent any risk(s) associated with a decision and evaluating all possible outcomes and potential impacts of risk. In general, there are three steps for risk assessment analysis [5]. First, the risk influencing factors are analyzed; second, a reasonable and scientific risk assessment index system is established; third, a scientific risk assessment method is selected to form a comprehensive evaluation process based on the above two steps. It is the capacity to see problems as they arise, deal with them and try to prevent them from happening again.
There are a number of different risk analysis methods for researchers to choose from [6,7,8]. At present, they can be divided into three categories, i.e., qualitative risk assessment methods, semi-quantitative risk assessment methods, and quantitative risk assessment methods. For the qualitative risk assessment methods [9,10,11], which are easy to operate, the evaluation process and results are simple to express, such as (a) questionnaire survey method; (b) collective discussion method; (c) expert investigation method; (d) safety checklist method; and (e) risk assessment matrix (RAM). However, these methods rely more on the experience of the evaluators and lack depth in describing the risk of the system. The evaluation results of different types of evaluation objects are not comparable. Meanwhile, they have strict requirements on the professionalism of the personnel involved in the risk assessment.
For the semi-quantitative risk assessment methods, such as (a) fault tree analysis [12,13,14]; (b) event tree analysis [15]; (c) preliminary hazard analysis [16,17]; (d) failure mode and effect analysis [18,19]; and (e) index evaluation method [20,21], they are easy to implement and have stronger objectivity than qualitative analysis, but need a lot of preparatory work. For example, these methods can be used to evaluate the probability of accidents and risk analysis, but preparatory work such as (a) estimating the exposure time of people in a dangerous environment, (b) evaluating the severity of accidents, (c) determining the corresponding scores of different factors, and (d) drawing the evaluation conclusions, are needed to be prepared in advance.
As for the quantitative risk assessment methods, they can reflect the risk levels by mathematical calculation, such as (a) fuzzy comprehensive evaluation method [13]; (b) factor analysis [22,23]; (c) analytic hierarchy process [24]; (d) cluster analysis [25,26]; (e) regression analysis [27,28,29]; (f) Bayesian analysis [30,31,32]; (g) logit model [33,34,35]; (h) time series analysis method [36,37]; and (i) Dempster-Shafer theory [38,39]. These methods have an objective and direct evaluation through the observed parameters, so as to find out the dangerous and harmful factors in the evaluation system, and then put forward the corresponding solutions in technology and management to realize safety management.

2.2. DST and Applications

Dempster-Shafer theory (DST) is a general framework for reasoning with uncertainty by Dempster [38] in 1967, and the theory was later developed by Shafer [39] into a general framework for modelling epistemic uncertainty. Previously, risk assessment of ULPS has many characteristics such as multiple indicators and multiple knowledge areas, meanwhile, the risk assessment process has a lot of uncertainties and is difficult to strictly quantify. Fortunately, DST provides an opportunity to combine multiple evidence sources. This is because DST is an established technique that maintains a mechanism of multiple indicators inputs and integrated outputs for the evaluation of risk analysis.
DST is a technique that can handle all the available evidence from different sources and arrive at a degree of belief (represented by a mathematical object called belief function) [40,41,42,43]. Previously, DST is mainly used in artificial intelligence, pattern recognition, data fusion technology, expert system and decision analysis, and its application in risk assessment has gradually increased in recent years. For example, Sun et al. developed an alternative methodology for the risk analysis of information systems security under the DST of belief functions. Zeng et al. [44] proposed a technique for traffic incident detection, which combined multiple multi-class probability support vector machines using DST approach. Rassafi et al. [45] employed an DST approach for road safety assessment modelling as a complex multi attribute decision analysis problem to deal with unavoidable uncertainties such as ignorance and vagueness. Through the application of DST, we can analyse the multiple-index variables related to safety situations in urban large-scale public spaces. Analysis of risk level with reference to risk assessment indicators will be helpful in understanding safety performance of ULPSs.

2.3. Visualization of Risk Analysis Studies

From the web of science database, 1997–2019, we have searched the main research countries, authors, and research institutions in risk analysis studies, our search topics are “traffic safety” and “risk analysis”, and we found that there were a total of 1400 + records, the number of authors was 4515, the number of research institutions was 1488, and the number of countries was 91. Next, the visualization of keywords in risk analysis studies, 1997–2019, were created by VOSviewer software [46], as shown in Figure 1.
The keywords are an important part of a research paper, which carries important information about the fields of interests. A total of 5000+ keywords appeared in all the collected literature on traffic safety and risk analysis, see Figure 1. Figure 1a is the density of main research keywords and Figure 1b is the keywords co-occurrence network of risk analysis studies. It can be clearly seen that the research theme of risk assessment has roughly formed four clusters in Figure 1b, and there is a significant correlation between the keywords in each cluster.
The visualization of keywords in risk analysis studies indicates that previous studies [8,13,16,19,20,25,26] that focused on risk assessment in ULPS were undertaken at an aggregate level. Systematization is the key requirement in safety risk assessment. However, systematization firstly means the evaluation system is systematic, which reflects the performance of specific models; secondly, the evaluation system should comprehensively cover both qualitative and quantitative indicators. Most ULPS safety risk studies [8,13,16,19,20,25,26] rely on questionnaire survey data but do not provide much information about the index system of risk assessment. In addition, little is known about the underlying factors that affect the risk analysis of ULPS. Furthermore, there is a paucity of research discerning the interrelationships between the multiple-index system and risk assessment, and the contributory indicators and multiple-index weight at a disaggregated level using Delphi method and entropy method.

3. Method

3.1. Dempster-Shafer Theory (DST)

3.1.1. Framework of Discernment (FoD)

DST is a finite nonempty set of hypotheses as the frame of discernment (FoD) Θ . Let Θ be the universe: the set representing all possible states of a system under consideration. The power set 2 Θ is the set of all subsets of Θ , including the empty set ϕ . For example, if Θ = { a , b } , then 2 Θ = { ϕ , { a } , { b } , Θ } .
The elements of the power set can be taken to represent propositions concerning the actual state of the system, by containing all and only the states in which the proposition is true.

3.1.2. Basic Belief Assignment (BBA)

The theory of evidence assigns a belief mass to each element of the power set. Formally, a function m : 2 Θ [ 0 , 1 ] is called a basic belief assignment (BBA), which has two properties. First, the mass of the empty set is zero.
m ( Φ ) = 0
Second, the masses of the remaining members of the power set add up to a total of 1.
A 2 Θ m ( A ) = 1
The mass m(A) of A, a given member of the power set, expresses the proportion of all relevant and available evidence that supports the claim that the actual state belongs to A but to no particular subset of A. The value of m(A) pertains only to the set A and makes no additional claims about any subsets of A, each of which have, by definition, their own mass.
From the mass assignments, the upper and lower bounds of a probability interval can be defined. This interval contains the precise probability of a set of interest (in the classical sense), and is bounded by two non-additive continuous measures called belief (or support) and plausibility.
The belief bel(A) for a set A is defined as the sum of all the masses of subsets of the set of interest.
b e l ( A ) = B | B A m ( B )
The plausibility pl(A) is the sum of all the masses of the sets B that intersect the set of interest A.
p l ( A ) = B | B A m ( B )
The two measures are related to each other as follows.
p l ( A ) = 1 b e l ( A ¯ )
And conversely, for finite A, given the belief measure bel(B) for all subsets B of A, we can find the masses m(A) with the following inverse function.
m ( A ) = B | B A ( 1 ) | A B | b e l ( B )
where |A−B| is the difference of the cardinalities of the two sets.

3.1.3. Dempster’s Combination Rule

Generally, the detailed combination rule is to calculate the distance between the two sets of masses mi and mj, and then modify the obtained mass function by the calibration coefficient. Finally, the combination rule is used for evidence fusion in the following manner.
Step1: The Θ is a finite nonempty set of hypotheses as the FoD, the masses mi and mj are the base degrees of belief (or confidence, or trust) for the frame of discernment Θ , the distance between the two sets of masses mi and mj is calculated by Equation (7).
d i j = ( 1 / 2 ) × ( m i m j ) T × D ( m i m j )
where D is a matrix of 2N×2N, the element in the matrix is, D(Ai, Aj)=(|AiAj|)/(|AiAj|), i = 1, 2, … , 2N, and dij indicates the difference between the two sets of masses, dij∈[0,1].
The similarity between the sets of masses mi and mj is Sij, as shown in Equation (8).
S i j = 1 d i j
The MASS function M m i ( R j ) is compared in pairs, and the distance between the evidences is calculated to obtain the evidence similarity matrix Sim.
S i m = [ 1 s 12 s 1 n s 21 1 s 2 n s n 1 s n 2 1 ]
The degree to which evidence m i is supported by other evidence is described as Sup( m i ).
S u p ( m i ) = j = 1 , j i n S i j
The credibility of the evidence m i is C r d 1 ( m i ) .
C r d 1 ( m i ) = S u p ( m i ) m a x { S u p ( m k ) 1 k n }
Due to the large number of professional fields covered by safety risk assessment in ULPS, experts in various fields have strong professional knowledge background and authority. In order to make the evaluation results more accurate, comprehensive consideration of each expert’s professional background and other factors, we give each expert the same weight [33,34] for the RAIS, the sum of weights is r = 1 n λ r = 1 . From this, the credibility of expert i is C r d 2 ( m i ) .
C r d 2 ( m i ) = S u p ( λ i ) m a x { S u p ( λ k ) 1 k n }
Step 2: According to the credibility of the evidence and the credibility of the experts, the calibration coefficient α i of the evidence is obtained.
α i = μ C r d 1 ( m i ) + ( 1 μ ) C r d 2 ( m i )
Supposing μ = 0.5 , it means that the credibility of evidence and the credibility of experts are of equal importance.
According to the calibration of coefficient α i , we adjust the MASS function of the indicator B i shown in Equations (14) and (15).
m ( B ) = α M ( B )
m ( Θ ) = α M ( Θ ) + 1 α
After modification by Equations (14) and (15), a new MASS function m B i ( R j ) is obtained, then the Dempster’s rule of combination is used to merge the m experts’ comments on the second-level index B i , and the new MASS function m B i ( R j ) is also obtained corresponding to each second-level index B i .
Step3: Dempster’s orthogonal rule of combination is determined by Equations (16) and (17).
m ( A ) = K 1 A 1 A 2 A n = A m 1 ( A 1 ) m 2 ( A 2 ) m n ( A n )
K = 1 A 1 A 2 A n = φ m 1 ( A 1 ) m 2 ( A 2 ) m n ( A n ) = A 1 A 2 A n φ m 1 ( A 1 ) m 2 ( A 2 ) m n ( A n )
where K is a measure of the amount of conflict between the two mass sets, if K = 0, it means that all evidence is completely contradictory, and the Dempster’s combination rule cannot be applied; Conversely, if K≠0, then the fusion of a set of evidence m 1 , m 2 , , m n becomes the orthogonal sum, m is new evidence produced by the combination, and it also is a MASS function too. Note that m = m 1 m 2 m 3 m n , which represents the combination of m 1 , m 2 , and m n , carries the joint information from the sets of masses sources.
According to the principle of maximum membership degree, the MASS function of the second-level indicator and its weight are linearly weighted, and the risk level of the first-level indicator can be obtained, see Equation (18).
F = i = 1 n ω i m B i ( R j )
Similarly, the MASS function M A i ( R j ) corresponding to each level indicator A i can be obtained, and the risk evaluation level of the large-scale public space is determined.

3.2. Risk Assessment Index System (RAIS)

3.2.1. Preliminary RAIS

The safety of the urban large-scale public spaces will be affected by various factors during the operation, especially when the high-density pedestrian flow is gathering. RAIS has played a crucial role in the risk assessment of ULPS’s development, while the scientific knowledge and comprehensiveness of index system will directly affect the accuracy of safety risk assessment in the ULPS. The principles [20,21] for selecting the preliminary RAIS in ULPS are as follows,
(1) Principle of scientific
All indicators in the RAIS can objectively reflect the risk factors faced by ULPS. The division of indicators has the basis of scientific theory for reference and can truly reflect the characteristics of security risks in ULPS.
(2) Principle of comprehensiveness
All indicators in the RAIS can comprehensively reflect the specific situation of security risks in ULPS. The selection of the preliminary risk indicators in ULPS not only includes the internal and external risk factors of the buildings, but also includes the surrounding environment, transportation facilities, system management, and other factors. Generally speaking, the selection of RAIS covers all factors affecting the security of ULPS.
(3) Principle of accuracy
All indicators in the RAIS are interrelated and independent, so as to ensure that the same type of evaluation indicators is not repeated, and indicators that have weak influence on security risks in ULPS should be eliminated to avoid statistical difficulties, calculation redundancy and credibility reduction.
(4) Principle of operability
All indicators in the RAIS need to be clear and easy to understand, and easy to conduct questionnaires or data collection. Principle of operability is to ensure the authenticity, effectiveness and operability of the risk assessment.
At the same time, based on the above principles and previous literatures [13,16,19,20,21,22,23,25,26,27,43,44,45], this study built a risk assessment index system including 4 first-level indicators, i.e., environmental factors, human factors, infrastructure factors and management factors, and 31 second-level indicators, as shown in Table 2.

3.2.2. Sensitivity Analysis of Indicators

The risk assessment indicators established in the initial stage covered too much information, which not only caused the information redundancy but also increased the difficulty of risk assessment and reduced the accuracy of the evaluation results. Therefore, the indicators should be screened.
The Delphi method was used to analyze the sensitivity of evaluation indicators. In the process of sensitivity analysis, we take the factors that have a greater impact on the risk assessment as the sensitivity index, our purpose is to find the sensitive indicators and delete the non-sensitive indicators.
Let E ¯ i p as the ith indicator of the degree of acceptance in the expert group p , let E ¯ i as the ith indicator of the total average degree of acceptance in the expert group n . The formula is as follows:
E ¯ i p = 1 k j = 1 5 E j n i j p
E ¯ i = 1 n p = 1 n E i p
where n i j p is the number of experts who are considered to be j -level in the expert group p ; E j is the corresponding value of the j -level importance of a certain indicator. The importance of indicators is divided into five levels: very unimportant (E1 = 1), unimportant (E2 = 2), generally important (E3 = 3), important (E4 = 4), very important (E5 = 5). Through the calculations of Equations (19) and (20), the total average acceptance of each indicator in all expert groups is calculated. The result of the sensitivity analysis is shown in Figure 2.
After removing the indicator with the value less than 4.0, the final 20 indicators are obtained, as shown in Figure 3.

3.3. Determination of the Weight of Risk Assessment Indicators

In the process of risk assessment, the importance of various indicators in the evaluation system is different. Hence, it is necessary to establish the corresponding weights of indicators for ULPS. The entropy weight method is used to calculate the index weights, that is, firstly, the Delphi method [20,21] is used to collect the weight information by experts; secondly, the value of the ranking matrix of expert evaluation is calculated; thirdly, the entropy value is calculated by the entropy decision process, and final weights of indicators are obtained by the entropy weight method [20,21,22,30].

3.3.1. Collection of Expert Opinions for RAIS

The weight information is collected using Delphi method by experts. It is assumed that m experts are invited to participate in the weight information survey of the safety risk assessment indicators in LSPS, the survey requires that the experts hired to sort the evaluation index sets according to their rich knowledge, professionalism and practical experience. Five indicates that the index is "most important" and four indicates "important", and the importance decreases in turn until 1, allowing experts to have the same value for multiple indicators. The ranking matrix obtained by m experts is set to R.
R = ( a 11 a 12 a 1 n a 21 a 22 a 2 n a m 1 a m 2 a m n )
where a m n is the evaluation value of the mth expert for the nth indicator.

3.3.2. Calculation of Indicator Weight

To determine the entropy value, the ranking matrix is first transformed into the membership matrix [20,21]. The membership function of ranking transformation is defined as S ( G )
S ( G ) = λ x n ( G ) I n x n ( G )
x n ( G ) = N G N 1 , λ = 1 I n ( N 1 ) , N = n + 2
We define P ( G ) as membership function, as shown in Equation (24),
P ( G ) = S ( G ) / ( N G N 1 ) 1
P ( G ) = I n ( N G ) I n ( N 1 )
where G is the ranking value given by the experts, N is the index value after standardization conversion, and n is the number of indicators.
The ranking number of each index is brought into the Equation (25), and the sorting matrix M can be converted into the membership degree matrix M = ( q i j ) m × n , and q i j is called the membership degree of the ranking number G. Taking the column vectors in the membership matrix M as q j = ( q 1 j , q 2 j , , q m j ) T , and the average value q j ¯ of the membership in the vector is obtained, as shown in Equation (26).
q j ¯ = 1 m i = 1 m q m j
Therefore, the mean square deviation S j of the membership in the vector can be obtained.
S j = 1 m i = 1 m ( q m j q j ) 2
The comprehensive evaluation value of each index by m experts is recorded as σ j , as shown in Equation (28).
σ j = q j ¯ ( 1 S j )
Thereby, the risk evaluation vector θ = ( σ 1 , σ 2 , , σ n ) of index σ j can be obtained.
We normalize the weights ω j of indicators, as shown in Equation (29).
ω j = σ j / j = 1 n σ j
Let W = ( ω 1 , ω 2 , , ω n ) be the weight vector of risk assessment index system U = { u 1 , u 2 , , u n } , ω j > 0 ( j = 1 , 2 , , n ) , and j = 1 n ω j = 1 . Each value in W corresponds to the weight of each grade of indicators, and the larger the value, the stronger the impact of the index on the safety risk assessment of large-scale public spaces.

4. Case Study

Tian-yi Square, the largest commercial plaza in Ningbo City, was completed by the end of 2001, the total investment of the project was 1.25 billion RMB. It has 167,000 square meters of shops, 20,000 square meters of parking lots, 64,000 square meters of green space, 6,000 square meters of water areas and 1,000 square meters of performing stage. In the case study, we take the Tian-yi Square, one of the biggest large-scale public spaces in Ningbo, as an example, the DST method is proposed to evaluate the safety risk of ULPS, in order to verify the practicability and effectiveness of the risk assessment method.

4.1. Data Collection

We developed an expert questionnaire based on the RAIS given above. Experts from various research fields such as traffic engineering, traffic safety, municipal engineering and risk management were invited to evaluate the risk assessment indicators. According to the collected data, scores of RAIS by experts as shown in Table 3.

4.2. Calculation of Index Weight for RAIS

Experts’ decision-making is an important part of index weight calculation and risk assessment for the ULPS. In order to reduce the influence of different experts on the evaluation results, firstly, the assumption is that we fully believe and respect the scoring results of the experts, secondly, we employ as many experts from different fields as possible to score together and take the average, according to their rich knowledge, professionalism and practical experience. In addition, in the actual calculation process, we remove a maximum score, while removing a minimum score, and then calculate the average of the remaining index scores. According to the scores of expert survey results (see Table 2), the ranking matrix of expert opinions can be obtained.
M = ( 2 2 3 1 3 2 2 1 4 2 3 1 2 3 1 1 3 1 3 2 ) M 1 = ( 5 2 3 1 4 4 2 3 1 5 2 3 4 1 5 3 2 4 3 5 5 4 3 2 4 ) M 2 = ( 3 4 1 5 2 3 1 2 5 4 2 4 2 4 5 3 3 4 5 2 1 4 2 4 5 )
M 3 = ( 2 5 3 4 1 1 3 3 4 2 1 2 4 4 2 2 4 3 5 1 1 2 3 4 2 ) M 4 = ( 1 4 3 2 2 2 2 4 3 1 2 1 4 3 2 1 2 4 3 1 2 3 5 4 1 )
The ranking membership matrix M of each grade of indicators can be obtained by Equations (21)–(24).
M = ( 0.861 0.861 0.683 1.000 0.683 0.861 0.861 1.000 0.431 0.861 0.683 1.000 0.861 0.683 1.000 1.000 0.683 1.000 0.683 0.861 )
M 1 = ( 0.387 0.898 0.774 1.000 0.613 0.613 0.898 0.774 1.000 0.387 0.898 0.774 0.613 1.000 0.387 0.774 0.898 0.613 0.774 0.387 0.387 0.613 0.774 0.898 0.613 ) M 2 = ( 0.774 0.613 1.000 0.387 0.898 0.774 1.000 0.898 0.387 0.613 0.898 0.613 0.898 0.613 0.387 0.774 0.774 0.613 0.387 0.898 1.000 0.613 0.898 0.613 0.387 )
M 3 = ( 0.898 0.387 0.774 0.613 1.000 1.000 0.774 0.774 0.613 0.898 1.000 0.898 0.613 0.613 0.898 0.898 0.613 0.774 0.387 1.000 1.000 0.898 0.774 0.613 0.898 ) M 4 = ( 1.000 0.613 0.774 0.898 0.898 0.898 0.898 0.613 0.774 1.000 0.898 1.000 0.613 0.774 0.898 1.000 0.898 0.613 0.774 1.000 0.898 0.774 0.387 0.613 1.000 )
The mean square deviation S j of the membership matrix M can be obtained by Equations (25) and (26).
S = ( 0.158 , 0.101 , 0.129 , 0.055 ) ; S 1 = ( 0.205 , 0.112 , 0.079 , 0.089 , 0.111 ) ;
S 2 = ( 0.092 , 0.152 , 0.130 , 0.111 , 0.229 ) ; S 3 = ( 0.050 , 0.194 , 0.064 , 0.191 , 0.050 )
S 4 = ( 0.050 , 0.133 , 0.123 , 0.091 , 0.050 )
Then, the evaluation vector σ i of each grade of indicators is obtained by Equation (27).
σ = ( 0.592 , 0.767 , 0.618 , 0.918 ) ; σ 1 = ( 0.487 , 0.725 , 0.654 , 0.851 , 0.424 )
σ 2 = ( 0.766 , 0.613 , 0.749 , 0.424 , 0.491 ) ; σ 3 = ( 0.911 , 0.575 , 0.694 , 0.516 , 0.892 )
σ 4 = ( 0.892 , 0.726 , 0.526 , 0.697 , 0.911 )
Finally, the weight vector ω i of each grade indicator is obtained by Equation (28).
ω = ( 0.200 , 0.259 , 0.230 , 0.310 ) ; ω 1 = ( 0.155 , 0.231 , 0.208 , 0.271 , 0.135 )
ω 2 = ( 0.252 , 0.201 , 0.246 , 0.139 , 0.161 ) ; ω 3 = ( 0.254 , 0.160 , 0.193 , 0.144 , 0.249 )
ω 4 = ( 0.238 , 0.193 , 0.140 , 0.186 , 0.243 )
Results of the weights of primary indicators and secondary indicators are calculated by Equations (21)–(28), as shown in Table 4.

4.3. Evidence fusion Process of Security Risk Assessment

The membership matrix R n × j of security risk level can be obtained by forward generator in Normal Cloud (NC) Model [49,50] from Table 2 and Table 3. In this study, the NC model was used to evaluation the membership matrix of security risk level. The steps to construct a NC model were as follows.
Setp 1: NC model: Let U be a quantitative domain and C a qualitative concept on U . If a certain value x U , then x is a random implementation of C . The determinacy of x to C is a random number with a stable tendency μ ( x ) : U →[0,1], ∀ x U , x→ μ ( x ) . Then, the distribution of x on the domain U is called a cloud, and each x is called a cloud droplet. Three parameters are used to characterize the cloud model, which are the expectation Ex, the entropy En, and the super-entropy He.
Setp 2: NC generator: If the definition of x in the cloud satisfies x ~ N(Ex, En’) and En’ ~ N(En, He2), the determinacy of x to C satisfies the following.
μ ( x ) = E x p ( x E x ) 2 2 ( E n ) 2
The NC model is widely used in solving probabilistic and ambiguous problems, which combines the normal distribution function with the bell-shaped membership function. All the cloud models used in this study referred to the NC model. By definition, the quantitative data obtained from cloud digital eigenvalues were forward NC generators, while those obtained from the quantitative data were backward NC generators. In this study, the forward NC generator was used to solve the problem of risk level.
Setp 3: Cloud synthesis: The synthesis of clouds was to combine the clouds of the same nature with a parent cloud. The parent cloud C (Ex, En, He) was synthesized using n child clouds Cn (Exn, Enn, Hen), expressed as follows.
C = C 1 C 2 C 3 C n
E x = i = 1 n E x i × E n i × w i i = 1 n E n i × w i , E n = i = 1 n E n i × w i , H e = i = 1 n H e i × E n i × w i i = 1 n E n i × w i
where “◦” refers to the process of cloud synthesis and wi to the weight of the ith child cloud.
The process of the NC model followed three steps:
Step 1: Determining the digital features of the cloud
Let X = (x1, x2, …, x j , …, xn-1) be the threshold vector of an index in which x j denotes the threshold of the index at jth R degree. As the synthetic clouds were conducted in the same domain, the index needed to be standardized before calculating the digital features. Using the bigger-is-better and the smaller-is-better rules, the standardizations were as shown in Equations (33) and (34), respectively.
Bigger is better,
x j = x j min { x j } max { x j } min { x j } ( j = 1 , 2 , , n 1 )
Smaller is better,
x j = max { x j } x j max { x j } min { x j } ( j = 1 , 2 , , n 1 )
where x j * is the standardized value of x j , and max { x j } and min { x j } are the maximum and minimum values for the threshold j, respectively.
The R degree j (j = 1, 2, … , n-1) is indicated by NCs in the semi-rising and semi-descending states, leading to the three digital features in Equation (35). Then, the other grades are indicated by NCs in full states (see Equation (36)), and their corresponding digital features are calculated as follows.
E x x 1 = x 1 , E n x 1 = E n x 2 E x x n = x n 1 , E n x n = E n x n 1
{ E x x j = ( x j 1 + x j ) / 2 E n x j = ( x j 1 x j ) / 6 H e = 0.01 ( j = 2 , , n 1 )
Step 2: Establishing the template cloud model
Synthesizing all the indexes of child clouds under the R degree criterion in n grade results in the parent cloud, which is called the template cloud or standardized cloud and uses a standard cloud map to assess the crowd state in the metro station. For example, a certain facility has three indexes, and each of their child cloud is denoted as Rj, Sj, and Tj. Thus, the parent cloud Uj was synthesized as Uj = Rj ◦ Sj ◦ Tj (j = 1, 2, …, n), and can be referred from Equation (32).
Step 3: Establishing the candidate cloud model
(a). Establish a forward cloud generator CGXj, according to the digital features of the R degree.
(b). Standardize the actual values x collected by the index and marked as x*. Let the standardization results of index in x ≤ min { x j } be 0 and index in x ≥ min { x j } be 1.
(c) Input the standardized values x* into the forward NC generators, please find detailed information from Refs. [49,50].
(d) Output of the cloud generator, μ X j (j = 1, 2, …, n), represents the degree of x belonging to Xj. The fuzziness and randomness nature of the NC model indicates that, instead of a determined number, μ X j is a random number with a stable tendency. The membership matrix R n × j of security risk level is consisted of these μ X j for each X j .
After normalizing all μ X j , Equation (37) shows the weights W X j for each X j .
W X j = μ X j j = 1 n μ X j
The MASS function M m i ( R j ) of various indicators are obtained with these weights W X j . Take the secondary indicator “Architectural layout (B2)” factor as an example, as follows, the others are listed in Appendix A.
R B 1 = ( 1.000 0.020 0.000 0.000 0.000 0.000 0.020 0.018 0.000 0.000 0.017 0.995 0.000 0.000 0.000 1.000 0.020 0.000 0.000 0.000 0.017 0.995 0.000 0.000 0.000 )
Then, after normalization, the MASS function M m i ( R j ) of various indicators are obtained by normalized processing technology.
M B 1 = ( 0.980 0.020 0.000 0.000 0.000 0.000 0.522 0.478 0.000 0.000 0.145 0.855 0.000 0.000 0.000 0.980 0.020 0.000 0.000 0.000 0.145 0.855 0.000 0.000 0.000 )
When obtaining expert evidence comments, due to the large differences in the research fields and work experience of experts, there may be great conflicts between their evidence comments.
Therefore, in order to solve the above problems and make the results of the evidence fusion more accurate, we use the distance function as in Equation (7) to measure the degree of conflict between pieces of evidence, and the calibration coefficients are used to revise the evidence comments from experts.
Next, the evidence similarity matrix Sim is obtained by Equations (7)–Equation (9).
S i m = ( 1.000 0.151 0.036 1.000 0.036 0.151 1.000 0.530 0.151 0.530 0.036 0.530 1.000 0.036 1.000 1.000 1.000 0.036 1.000 0.036 0.036 0.530 1.000 0.036 1.000 )
And, the credibility vector C r d 1 of each expert evidence is obtained by Equations (10)∓(11).
C r d 1 = ( 0.763 , 0.850 , 1.000 , 0.763 , 1.000 )
The credibility vector C r d 2 of experts are then obtained by expert weights
C r d 2 = ( 1.000 , 1.000 , 1.000 , 1.000 , 1.000 )
The calibration coefficient of the evidence is obtained by Equation (13), and the MASS function of the indicators is revised by Equation (14), the revised MASS function m B i ( R j ) is obtained.
m B 1 = ( 0.865 0.017 0.000 0.000 0.000 0.118 0.000 0.483 0.442 0.000 0.000 0.075 0.016 0.984 0.000 0.000 0.000 0.000 0.865 0.017 0.000 0.000 0.000 0.119 0.016 0.984 0.000 0.000 0.000 0.000 )
Finally, the experts’ MASS function is merged by the D-S evidence fusion theory given in Equations (10)–(14), and the result is m B 1 .
The secondary indicator MASS function is obtained by Equations (7)–(14), as shown in Table 5.
According to the weight of secondary indicator and the MASS function, which are linearly weighted by the Equation (17). The result of risk level value is,
F A 1 = ( 0.845 , 0.155 , 0.000 , 0.000 , 0.000 , 0.000 ) F A 2 = ( 0.861 , 0.139 , 0.000 , 0.000 , 0.000 , 0.000 )
F A 3 = ( 0.503 , 0.143 , 0.354 , 0.000 , 0.000 , 0.000 ) F A 4 = ( 0.194 , 0.678 , 0.128 , 0.000 , 0.000 , 0.000 )
By Equations (7)–(14), the results of the MASS function of the primary indicators is obtained by D-S evidence fusion, as shown in Table 6.
According to the weight of primary indicator and the MASS function, which are linearly weighted by the Equation (17). The result of risk level value of Tian-yi Square is,
F = ( 0.459 , 0.311 , 0.230 , 0.000 , 0.000 , 0.000 ) .

4.4. Outcome of the Risk Assessment

(1) The security risk level for the primary indicators can be obtained by the principle of maximum membership, as shown in Table 7.
In Table 7, for the environmental factors, the value of the excellent level is 0.845, and the value of the good level is 0.155. Overall, the risk assessment level of environmental factors is excellent. For the human factors, the value of the excellent level is 0.861, and the value of the good level is 0.139. Overall, the risk assessment level of human factors is excellent. For the infrastructure factors, the value of the excellent level is 0.503, and the value of the good level is 0.143. Overall, the risk assessment level of infrastructure factors is also excellent. For the management factor, the value of the excellent level is 0.194, and the value of the good level is 0.678. Overall, the risk assessment level of management factors is good.
(2) The overall security risk level result of Tian-yi Square is obtained by the principle of maximum membership, that is, the value of the excellent level is 0.459, the value of the good level is 0.311, and the value of the satisfactory level is 0.230. Therefore, the risk assessment level of Tian-yi Square is safety. The reason is that, Ningbo city, one of the three economic centers in Zhejiang province, is located in the southeast coastal area and belongs to the subtropical monsoon climate, at the same time, Tian-yi Square is one of the famous public places in Ningbo city, the surrounding environment is comfortable, the transportation is convenient, and society is harmonious. Hence, the overall safety risk assessment is excellent.
However, due to the early completion of Tian-yi Square, many facilities and equipment cannot be updated in time, and there are some defects in large-scale event management, store management, and staff training, so the evaluation level of infrastructure factors and management factors is only good, not excellent.

5. Discussions

(1) Enhance the relevance of RAIS
The original of the risk assessment indicator system (RAIS) fully considers the factors affecting the environment, human, infrastructure and management. However, in order to improve the pertinence and applicability of the RAIS, considering the gaps in risk factors faced by different cities and different public places, the sensitivity analysis method is used to measure the entire primary RAIS in the process of index preparation. Finally, all the indicators involved in risk assessment are valuable and applicable.
The risk indicators covered in the established primary RAIS are diversified and multi-faceted, but in the end, the indicators used in the risk assessment questionnaire are screened and representative. Therefore, it is very important to improve the accuracy of risk assessment indicator system and reduce the impact of the low sensitivity of indicators to the overall risk analysis.
(2) Universality of risk assessment methods
The Dempster-Shafer Theory is used to analyze risk assessment in urban large-scale public spaces, which can effectively integrate multiple evidence. Regardless of how many secondary indicators are set in the entire risk assessment indicator system or how many experts are employed to participate in the risk assessment, the Dempster-Shafer theory can be used to combine the expert opinions and the final risk assessment results for urban large-scale public spaces is obtained. Hence, the Dempster-Shafer theory for the risk assessment has universality.
(3) Division of evaluation level
The Delphi method was used by the experts to evaluate the risk and divide the risk assessment criteria into five levels according to the principle of equidistance: excellent (8 ~ 10), good (6 ~ 8), satisfactory (4 ~ 6), fair (2 ~ 4), and poor (0 ~ 2). It is different from the two-level evaluation method that divides the evaluation level into only "good and bad", and which is also different from the percentage system and the thousand-point system of the unlimited-level evaluation level. That is, reasonably dividing the evaluation level into five levels, the real feedback expert’s risk assessment for the urban large-scale public space is also conducive to the fusion of D-S evidence.

6. Conclusions

The main purpose of this work is to identify potential safety hazards in large-scale public spaces as early as possible, so that relevant management departments can promptly take measures to avoid risks. Taking Tian-yi Square in Ningbo as an example, this paper analyzes the scientific and feasible safety risk assessment index system from four aspects: environmental factors, human factors, infrastructure factors, and management factors. The risk assessment method using Dempster–Shafer theory is not limited to a single risk event, but also a comprehensive assessment method of the multiple risk factors faced by urban large-scale public spaces. The main results are described as follows.
A robust risk indicator system for assessing the security risk of large-scale public spaces were selected by Delphi method and information entropy method, which included four first-level indicators and twenty second-level indicators. The first-level indicators were environmental factors, human factors, infrastructure factors, and management factors. There was five second-level indicators belonging to the environmental factors, which were architectural layout, weather, external traffic, public health, and social stability; and there was also five second-level indicators belonging to the human factors, which were crowd characteristics, negligence behavior, macroscopic fundamental diagram, safety awareness, and panic. Meanwhile, there was five second-level indicators belonging to the infrastructure factors, which were exit, guide sign, alarm system, firefighting system, broadcasting and monitoring system; and there was also five second-level indicators belonging to the management factors, which were personnel training, public health management, event organization and management, emergency evacuation management, and infrastructure management.
Data were collected in Ningbo, using an expert questionnaire survey approach based on the RAIS. The survey results showed that the risk indicator system for the ULPS assessment process is scientific and reasonable. In the risk index system, twelve variables were found to be statistically significant, which were ranked by weights: emergency evacuation management, personnel training, crowd characteristic, macroscopic fundamental diagram, public health management, exit, event organization and management, broadcasting monitoring system, public health environment, negligence behavior, weather, alarm system, infrastructure management, panic, external traffic, guide sign, safety awareness, firefighting system, architectural layout, and social stability. In addition, in the calculation process of weights, we fully considered the expert’s knowledge background and opinions, and reduce the uncertainty in the assessment, which was characterized by strong explanatory and high precision.
A Dempster-Shafer Theory with evidence fusion technique was employed to analyze the interaction between the RAIS and risk level in the ULPS. The results from the DST approach indicated that three variables were found to be excellent level, which were ranked by importance: environmental factors, human factors, infrastructure factors. Only one variable was found to be good level, which were management factors. The results from the value of the MASS function indicated that three indicators were found in higher risk level, which were guiding signs, alarm system, and personnel training. Simultaneously, eleven indicators were found in a higher safety level, which were weather, external traffic, public health environment, social stability, crowd characteristic, negligence behavior, macroscopic fundamental diagram, panic, exit, broadcasting monitoring system, public health management. The findings of this study provided insight into the factors associated with environmental factors, human factors, infrastructure factors, and management factors in the ULPS.
In the risk assessment of large-scale public spaces, we used the DST approach to conduct a multi-level risk assessment, and the risk grading of each evaluation index was also obtained. Taking Tian-yi Square as a case study, the results were consistent with the actual operation of large-scale public spaces, indicating that the DST approach has certain theoretical guiding significance and practical value.
However, there are some limitations in this study. Firstly, the survey was only conducted in Ningbo city, but studies based on multiple cities could help better understand and capture more risk factors affecting large-scale public spaces. Secondly, an update questionnaire survey and risk assessment index system could be conducted to capture more meaningful factor with risk assessment work. We have to accept that there are other indicators in the risk assessment of large-scale public spaces, such as building characteristic index (e.g. building density, height, fire resistance, ventilation, etc.) in Environmental Factors (A1), temporary shelter, emergency shelter, basement shelter, and flame retardant equipment in Infrastructure Factors (A3), and evacuation training, safety education in Management Factors (A4). Hence, it will be encouraged to build a risk assessment indicators database for large-scale public spaces in the future study. Thirdly, risk management was defined as a procedure to control the level of risk and to mitigate its effect. Hence, the generally steps of risk management in the ULPS could be captured and described, such as risk identification, risk analysis, risk response and etc.
Furthermore, the extension of this work should examine the different expert weights and experience scores for the RAIS, especially for unobserved heterogeneity across more experts or staff or passengers or tourists. To solve this problem, in the future study, we will establish a historical database to quantify risk assessment indicators to reduce the impact of expert subjective factors on the assessment results. Recent work only provided the framework of discernment DST model [38,39,40,41,42,43,44]. Under this framework, risk assessment effects on environmental factors, human factors, infrastructure factors, and management factors across various passengers (or tourists) in the large-scale public spaces can be estimated, we are expecting more research results to emerge by standing upon the shoulders of ours.

Author Contributions

Conceptualization, J.Z.; Data curation, J.Z. and Y.W.; Methodology, J.Z. and Y.W.; Resources, S.D. and M.Z.; Supervision, X.M. and J.Z.; Writing-review & editing, J.Z. and Y.W.; Funding acquisition, J.Z and X.M.


This study was supported by the Natural Science Foundation of Zhejiang Province (No. LQ19E080003; LY17E080013), Philosophy and Social Science Foundation of Zhejiang Province (No. 18NDJC107YB), Educational Science Planning of Zhejiang Provincial Education Department (No. 2018SCG103), Natural Science Foundation of Ningbo Municipality (No. 2018A610127), Humanities and Social Science Research Program of Ministry of Education in China (No. 16XJCZH002) and the Fundamental Research Funds for the Central Universities (Grant Number 310823170657 and 300102238501).

Conflicts of Interest

The authors declare that there are no conflicts of interest regarding the publication of this paper.


The following list of symbols is used in this research.
Θ Frame of discernment
m(A)Basic probability assignment function
bel(A)Belief function, which is based on the basic probability assignment in the FoD
pl(A)Plausibility function, which is based on the basic probability assignment in the FoD
d i j Distance between two evidences, mi and mj
DA matrix of 2N×2N
SimSimilarity matrix of evidence
α i Calibration coefficient, which is used to correct the original matrix
KNormalized constant
FLinear weighting of the mass function and weight of each indicator
kNumber of experts in each group
nNumber of expert groups hired to participate in the review
kNumber of experts in each group
nNumber of expert groups hired to participate in the review
GRanking value given by the experts
NIndex value after standardization conversion
q i j Membership degree
nNumber of indicators
ω j Index weight coefficient
ExExpectation, that is the central value of the spatial distribution of cloud droplets in the universe
EnEntropy, that is a measure of the ambiguity of a qualitative concept, reflecting the range of cloud drip values that can be accepted by the concept space
HeSuper-entropy, that is a measure of the uncertainty of entropy, reflecting the degree of dispersion of the cloud droplets and the thickness of the cloud.
Process of cloud synthesis
W i Weight of the child cloud

Appendix A

Table A1. Results of original matrix and normalization
Table A1. Results of original matrix and normalization
IndicatorsOriginal MatrixNormalized MatrixIndicatorsOriginal MatrixNormalized Matrix
Architectural layout1.0000.0200.0000.0000.0000.9800.0200.0000.0000.000Weather1.0000.0200.0000.0000.0000.9810.0190.0000.0000.000
External traffic0.0170.9950.0000.0000.0000.0160.9840.0000.0000.000Public health1.0000.0200.0000.0000.0000.9810.0190.0000.0000.000
Social stability1.0000.0200.0000.0000.0000.9810.0190.0000.0000.000Crowd characteristics0.0170.9950.0000.0000.0000.0160.9840.0000.0000.000
Negligence behavior1.0000.0200.0000.0000.0000.9810.0190.0000.0000.000Macroscopic fundamental diagram1.0000.0200.0000.0000.0000.9810.0190.0000.0000.000
Safety awareness0.0170.9950.0000.0000.0000.0160.9840.0000.0000.000Panic1.0000.0200.0000.0000.0000.9810.0190.0000.0000.000
Exit1.0000.0200.0000.0000.0000.9810.0190.0000.0000.000Guide sign0.0000.0001.0000.0000.0000.0000.0001.0000.0000.000
Alarm system0.0000.0001.0000.0000.0000.0000.0001.0000.0000.000Firefighting system0.0000.0200.0180.0000.0000.0000.5220.4780.0000.000
Broadcasting and monitoring system1.0000.0200.0000.0000.0000.9810.0190.0000.0000.000Personnel training0.0000.0200.0180.0000.0000.0000.5220.4780.0000.000
Public health management1.0000.0200.0000.0000.0000.9810.0190.0000.0000.000Event organization and management1.0000.0200.0000.0000.0000.9810.0190.0000.0000.000
Emergency evacuation management1.0000.0200.0000.0000.0000.9810.0190.0000.0000.000Infrastructure management1.0000.0200.0000.0000.0000.9810.0190.0000.0000.000
Table A2. Results of similar matrix and modified matrix
Table A2. Results of similar matrix and modified matrix
IndicatorsSimilar MatrixModified MatrixIndicatorsSimilar MatrixModified Matrix
Architectural layout1.0000.1510.0361.0000.0360.8650.0170.0000.0000.0000.118Weather1.0001.0001.0001.0001.0000.9810.0190.0000.0000.0000.000
External traffic1.0000.0360.0360.0360.0360.0090.5150.0000.0000.0000.477Public health1.0001.0001.0001.0001.0000.9810.0190.0000.0000.0000.000
Social stability1.0001.0001.0001.0001.0000.9810.0190.0000.0000.0000.000Crowd characteristics1.0000.0360.0360.0360.0360.0090.5150.0000.0000.0000.477
Negligence behavior1.0000.0200.0000.0000.0000.9810.0190.0000.0000.0001.000Macroscopic fundamental diagram1.0001.0001.0001.0001.0000.9810.0190.0000.0000.0000.000
Safety awareness1.0000.5301.0000.5300.0360.0160.9580.0000.0000.0000.026Panic1.0001.0000.0360.0361.0000.9810.0190.0000.0000.0000.000
Exit1.0000.0361.0000.0361.0000.9810.0190.0000.0000.0000.000Guide sign1.0000.4781.0000.4780.4930.0000.0001.0000.0000.0000.000
Alarm system1.0001.0001.0000.4780.4780.0000.0001.0000.0000.0000.000Firefighting system1.0001.0000.5300.5301.0000.0000.5220.4780.0000.0000.000
Broadcasting and monitoring system1.0001.0001.0001.0001.0000.9810.0190.0000.0000.0000.000Personnel training1.0000.4780.5301.0000.1510.0000.5220.4780.0000.0000.000
Public health management1.0001.0001.0001.0000.0360.9810.0190.0000.0000.0000.000Event organization and management1.0000.0200.0000.0000.0000.8650.0170.0000.0000.0000.119
Emergency evacuation management1.0000.0360.0360.1511.0000.8650.0170.0000.0000.0000.119Infrastructure management1.0000.1510.1511.0000.0360.8480.0170.0000.0000.0000.135


  1. Wikipedia. List of Human Stampedes: World Crowd Disaster Map. Available online: (accessed on 3 July 2019).
  2. Generic Risk Assessment Model for Indoor and Outdoor Space Spraying of Insecticides; Department of Control of Neglected Tropical Diseases; World Health Organization: Geneva, Switzerland, 2018.
  3. Cornett, M.M.; Saunders, A. Financial Institutions Management: A Risk Management Approach (8th Edition); McGraw-Hill Education: New York, NY, USA, 2013. [Google Scholar]
  4. Mohaghegh, Z.; Kazemi, R.; Mosleh, A. Incorporating organizational factors into probabilistic risk assessment (PRA) of complex socio-technical systems: a hybrid technique formalization. Reliab. Eng. Syst. Safe. 2009, 94, 1000–1018. [Google Scholar] [CrossRef]
  5. Parry, G.W. The characterization of uncertainty in probabilistic risk assessments of complex systems. Reliab. Eng. Syst. Safe. 1996, 54, 119–126. [Google Scholar] [CrossRef]
  6. Chen, F.; Song, M.; Ma, X. Investigation on the injury severity of drivers in rear-end collisions between cars using a random parameters bivariate ordered probit model. Int. J. Environ. Res. Public Health. 2019. [Google Scholar] [CrossRef]
  7. Ma, C.; Hao, W.; Xiang, W.; Yan, W. The impact of aggressive driving behavior on driver-injury severity at highway-rail grade crossings accidents. J. Adv. Transport. 2018. [Google Scholar] [CrossRef]
  8. Guo, Y.; Li, Z.; Liu, P.; Wu, Y. Modeling correlation and heterogeneity in crash rates by collision types using full Bayesian random parameters multivariate Tobit model. Accid. Anal. Prev. 2019, 128, 164–174. [Google Scholar]
  9. Brener, N.D.; Collins, J.L.; Kann, L.; Warren, C.W.; Williams, B.I. Reliability of the youth risk behavior survey questionnaire. Am. J. Epidemiol. 1995, 141, 575–580. [Google Scholar] [CrossRef]
  10. Zou, P.; Zhang, G.; Wang, J. Understanding the key risks in construction projects in China. Int. J. Proj. Manag. 2007, 25, 601–614. [Google Scholar] [CrossRef]
  11. Guo, Y.; Zhou, J.; Wu, Y.; Li, Z. Identifying the factors affecting bike-sharing usage and degree of satisfaction in Ningbo, China. PloS one. 2017. [Google Scholar] [CrossRef]
  12. Shalev, D.M.; Tiran, J. Condition-based fault tree analysis (CBFTA): A new method for improved fault tree analysis (FTA), reliability and safety calculations. Reliab. Eng. Syst. Safe. 2007, 92, 1231–1241. [Google Scholar]
  13. Yazdi, M.; Zarei, E. Uncertainty handling in the safety risk analysis: an integrated approach based on fuzzy fault tree analysis. J. Fail. Anal. Pre. 2018, 18, 392–404. [Google Scholar] [CrossRef]
  14. Wu, P.; Meng, X.; Song, L.; Zuo, W. Crash Risk Evaluation and crash severity pattern analysis for different types of urban junctions: fault tree analysis and association rules approaches. Transportation Research Record. 2019, 2673, 403–416. [Google Scholar] [CrossRef]
  15. Mineo, S.; Pappalardo, G.; D’urso, A.; Calcaterra, D. Event tree analysis for rockfall risk assessment along a strategic mountainous transportation route. Environmental earth sciences. 2017, 76, 620–630. [Google Scholar] [CrossRef]
  16. Wu, Y.; Abdel-Aty, M.; Lee, J. Crash risk analysis during fog conditions using real-time traffic data. Accid. Anal. Prev. 2018, 114, 4–11. [Google Scholar] [CrossRef] [PubMed]
  17. Cordeiro, F.; Bezerra, B.; Peixoto, A.; Ramos, R. Methodological aspects for modeling the environmental risk of transporting hazardous materials by road. Transport. Res. D-Tr. E. 2016, 44, 105–121. [Google Scholar] [CrossRef]
  18. Liu, H.; You, J.; Duan, C. An integrated approach for failure mode and effect analysis under interval-valued intuitionistic fuzzy environment. Int. J. Prod. Eco. 2019, 207, 163–172. [Google Scholar] [CrossRef]
  19. Faiella, G.; Parand, A.; Franklin, B.; Chana, P.; Cesarelli, M.; Stanton, N.A.; Sevdalis, N. Expanding healthcare failure mode and effect analysis: A composite proactive risk analysis approach. Reliab. Eng. Syst. Safe. 2018, 169, 117–126. [Google Scholar] [CrossRef] [Green Version]
  20. Zhou, J.B.; Chen, H.; Yan, B.; Shi, X.; Li, X.W. Risk assessment of operation safety in freeway tunnels: An evaluation approach using multiple safety indices. J. Transp. Saf. Secur. 2014, 6, 93–116. [Google Scholar] [CrossRef]
  21. Chen, H.; Zhou, J.; Wang, J.; Li, X.; GAN, Z. Safety evaluation indexes and method for traffic environment of highway tunnels. J. Chang’an Univ., Nat. Sci. Ed. 2013, 33, 54–61. [Google Scholar]
  22. Chen, F.; Wang, J.; Deng, Y. Road safety risk evaluation by means of improved entropy TOPSIS–RSR. Saf. Sci. 2015, 79, 39–54. [Google Scholar] [CrossRef]
  23. Tang, J.; Liang, J.; Han, C.; Li, Z.; Huang, H. Crash injury severity analysis using a two-layer stacking framework. Accid. Anal. Prev. 2019, 122, 226–238. [Google Scholar] [CrossRef]
  24. Kokangül, A.; Polat, U.; Dağsuyu, C. A new approximation for risk assessment using the AHP and Fine Kinney methodologies. Saf. Sci. 2017, 91, 24–32. [Google Scholar] [CrossRef]
  25. Shen, L.; Lu, J.; Long, M.; Chen, T. Identification of accident blackspots on rural roads using grid clustering and principal component clustering. Math. Probl. Eng. 2019, 2019, 1–12. [Google Scholar] [CrossRef]
  26. Zou, Y.; Zhong, X.; Tang, J.; Ye, X.; Wu, L.; Ijaz, M.; Wang, Y. A copula-based approach for accommodating the underreporting effect in wildlife-vehicle crash analysis. Sustainability. 2019. [Google Scholar] [CrossRef]
  27. Rhee, K.A.; Kim, J.K.; Lee, Y.I.; Ulfarsson, G.F. Spatial regression analysis of traffic crashes in Seoul. Accid. Anal. Prev. 2016, 91, 190–199. [Google Scholar] [CrossRef] [PubMed]
  28. Shi, Q.; Abdel-Aty, M.; Lee, J. A Bayesian ridge regression analysis of congestion’s impact on urban expressway safety. Accid. Anal. Prev. 2016, 88, 124–137. [Google Scholar] [CrossRef] [PubMed]
  29. Ma, C.; Hao, W.; Wang, A.; Zhao, H. Developing a coordinated signal control system for urban ring road under the vehicle-infrastructure connected environment. IEEE Access. 2018, 6, 52471–52478. [Google Scholar] [CrossRef]
  30. Zong, F.; Chen, X.; Tang, J.; Yu, P.; Wu, T. Analyzing traffic crash severity with combination of information entropy and Bayesian network. IEEE Access. 2019, 7, 63288–63302. [Google Scholar] [CrossRef]
  31. Guo, Y.; Wu, Y.; Lu, J.; Zhou, J. Modeling the unobserved heterogeneity in e-bike collision severity using full Bayesian random parameters multinomial logit regression. Sustainability 2019. [Google Scholar] [CrossRef]
  32. Guo, Y.; Essa, M.; Sayed, T.; Haque, M.M.; Washington, S. A comparison between simulated and field-measured conflicts for safety assessment of signalized intersections in Australia. Transport. Res. C-Emer. 2019, 101, 96–110. [Google Scholar] [CrossRef]
  33. Chen, F.; Chen, S.; Ma, X. Analysis of hourly crash likelihood using unbalanced panel data mixed logit model and real-time driving environmental big data. J. Safety. Res. 2018, 65, 153–159. [Google Scholar] [CrossRef]
  34. Ma, X.; Chen, S.; Chen, F. Multivariate space-time modeling of crash frequencies by injury severity levels. Anal. Methods. Accid. Res. 2017, 15, 29–40. [Google Scholar] [CrossRef]
  35. Chen, F.; Chen, S. Injury severities of truck drivers in single-and multi-vehicle accidents on rural highways. Accid. Anal. Prev. 2011, 43, 1677–1688. [Google Scholar] [CrossRef]
  36. Yanmaz-Tuzel, O.; Ozbay, K. A comparative Full Bayesian before-and-after analysis and application to urban road safety countermeasures in New Jersey. Accid. Anal. Prev. 2010, 42, 2099–2107. [Google Scholar] [CrossRef] [PubMed]
  37. Lord, D.; Mannering, F. The statistical analysis of crash-frequency data: a review and assessment of methodological alternatives. Transport. Res. A-Pol. 2010, 44, 291–305. [Google Scholar] [CrossRef]
  38. Dempster, A. Upper and lower probabilities induced by a multivalued mapping. In Classic Works of the Dempster-Shafer Theory of Belief Functions; Springer: Berlin, Germany, 2008; pp. 57–72. [Google Scholar]
  39. Shafer, G. A Mathematical Theory of Evidence; Princeton University Press: Princeton, NJ, USA, 1976. [Google Scholar]
  40. Wikipedia. Dempster-Shafer theory. Available online: (accessed on 5 July 2019).
  41. Smets, P. Advances in the Dempster-Shafer Theory of Evidence; John Wiley & Sons: New York, NY, USA, 1994; pp. 5–34. [Google Scholar]
  42. Denoeux, T. A neural network classifier based on Dempster-Shafer theory. IEEE Trans. Syst. Man. Cybern. A. Syst. Hum. 2000, 30, 131–150. [Google Scholar] [CrossRef]
  43. Sun, L.; Srivastava, R.; Mock, T. An information systems security risk assessment model under the Dempster-Shafer theory of belief functions. J. Manage. Inform. Syst. 2006, 22, 109–142. [Google Scholar] [CrossRef]
  44. Zeng, D.; Xu, J.; Xu, G. Data fusion for traffic incident detection using DS evidence theory with probabilistic SVMs. J. Comput. 2008, 3, 36–43. [Google Scholar] [CrossRef]
  45. Rassafi, A.; Ganji, S.; Pourkhani, H. Road safety assessment under uncertainty using a multi attribute decision analysis based on Dempster-Shafer theory. KSCE. J. Civil. Eng. 2018, 22, 3137–3152. [Google Scholar] [CrossRef]
  46. Ma, C.; Yang, D.; Zhou, J.; Feng, Z.; Yuan, Q. Risk riding behaviors of urban e-bikes: a literature review. Int. J. Env. Res. Pub. He. 2019. [Google Scholar] [CrossRef]
  47. Production Safety Law of the People’s Republic Of China, no. 13, Order of the President of the People’s Republic of China; Ministry of Emergency Management of the People’s Republic of China: Beijing, China, 2018; Volume 13, pp. 1–32.
  48. Department of Defense Standard Practice System Safety (MIL-STD-882E); Department of Defense: Washington, WA, USA, 2012; pp. 9–20.
  49. Li, D.; Liu, C.; Gan, W. A new cognitive model: Cloud model. Int. J. Intell. Syst. 2009, 24, 357–375. [Google Scholar] [CrossRef]
  50. Peng, H.; Wang, J. A multicriteria group decision-making method based on the normal cloud model with zadeh’sz-numbers. IEEE. Trans. Fuzzy. Syst. 2018, 26, 3246–3260. [Google Scholar] [CrossRef]
Figure 1. The keywords in risk assessment studies, 1997–2019. (a) The density of main research keywords; (b) the keywords co-occurrence network of risk analysis studies.
Figure 1. The keywords in risk assessment studies, 1997–2019. (a) The density of main research keywords; (b) the keywords co-occurrence network of risk analysis studies.
Ijerph 16 02942 g001
Figure 2. Result of sensitivity analysis of indicators.
Figure 2. Result of sensitivity analysis of indicators.
Ijerph 16 02942 g002
Figure 3. Result of the final risk assessment index system for Urban Large-scale Public Spaces (ULPS).
Figure 3. Result of the final risk assessment index system for Urban Large-scale Public Spaces (ULPS).
Ijerph 16 02942 g003
Table 1. Statistical results of casualty incidents in the Urban Large-scale Public Spaces (ULPS) across the world (2000–2018) [1].
Table 1. Statistical results of casualty incidents in the Urban Large-scale Public Spaces (ULPS) across the world (2000–2018) [1].
9 March 2000Jiaozuo, ChinaTheatre74/Fire
25 March 2000Durban, South AfricaBallroom13150Fire
9 May 2001GhanaStadium100/Crowd
3 September 2001Shaanxi, ChinaYuquan Park166Crowd
20 February 2002EgyptTrain350/Fire
19 February 2003South KoreaTrain120387Arson
5 February 2004Beijing, ChinaLantern Show3715Crowd
11 March 2004SpainStation2001800Terrorist Attack
1 September 2004RussiaSchool300/Terrorist Attack
10 June 2005Shantou, ChinaHotel3115Fire
7 July 2005London, UKMetro52700Terrorist Attack
12 September 2006Yemen/51200Crowd
18 July 2007Jinan, China/25170Rainstorm
3 August 2008IndiaTemple16247Crowd
5 July 2009Urumqi, China/184939Terrorist Attack
15 November 2010Shanghai, ChinaApartment8571Fire
15 January 2011IndiaTemple102100Crowd
18 March 2012EgyptGathering3137Crowd
28 October 2013Beijing, ChinaSquare540Terrorist Attack
1 March 2014Kunming, ChinaTrain Station29143Terrorist Attack
13 November 2015FranceStadium197/Terrorist Attack
21 May 2016Dalian, ChinaMarket3/Fire
25 May 2017Henan, ChinaSenior Apartment396Fire
14 May 2018BengalRamadan Rally1050Crowd
25 August 2018Harbin, ChinaHotel2023Fire
9 September 2018LuandaStadium57Crowd
17 March 2019Northern Ireland.Hotel3/Human Crush
26 June 2019MadagascarStadium16101Human Crush
Table 2. Preliminary risk assessment index system.
Table 2. Preliminary risk assessment index system.
First-Level IndicatorsSecond-Level IndicatorsExplanations
Environmental factors (A1)Architectural layout (B11)These indicators reflect the relationship between various environmental factors and safety risks in UPLS, and reflect the impact of random factors on safety risk assessment.
Weather (B12)
External traffic (B13)
Internal traffic (B14)
Public health (B15)
Social stability (B16)
Human factors (A2)Crowd characteristics (B21)These indicators reflect the relationship between human factors and risk level in UPLS, and reflect the impact of human factors on risk assessment level of UPLS.
Negligence behavior (B22)
Macroscopic fundamental diagram (B23)
Age and gender (B24)
Safety awareness (B25)
Physical fitness (B26)
Panic (B27)
Trip purpose (B28)
Infrastructure factors (A3)Exit (B31)These indicators reflect the relationship between infrastructure factors and risk level of UPLS, and reflect the impact of static factors on risk assessment level of UPLS.
Guide sign (B32)
Power system (B33)
Lighting system (B34)
Fume exhaust system (B35)
Drainage system (B36)
Alarm system (B37)
Firefighting system (B38)
Broadcasting and monitoring system (B39)
Management factors (A4)Personnel training (B41)These indicators reflect the relationship between the management mechanism of public places and risk level of public places, and reflect the impact of dynamic factors on risk assessment level of UPLS.
Public health management (B42)
Security risk management (B43)
Event organization and management (B44)
Fire safety management (B45)
Emergency evacuation management (B46)
Store management (B47)
Infrastructure management (B48)
Table 3. Scores of Risk Assessment Index System (RAIS) by experts.
Table 3. Scores of Risk Assessment Index System (RAIS) by experts.
IndicatorsThe importance Value of RAIS aThe level Value of RAIS b, d
E1 cE2E3E4E5E1E2E3E4E5
Environmental factor23423/////
Human factor22231/////
Equipment factors32313/////
Management factors11112/////
Architectural layout5423586797
External traffic3344378989
Public health environment11132910889
Social stability4555410910109
Crowd characteristic33231798810
Negligence behavior4143498988
Macroscopic fundamental diagram1224298989
Safety awareness5545476768
Guide sign5324256564
Alarm system3343355566
Firefighting system4445466776
Broadcasting monitoring system1221210109910
Personnel training1221265768
Public health management4212389887
Infrastructure management3444596787
Event organization and management2333487768
Emergency evacuation management2121186687
Note: a denotes the importance value of risk assessment index system (RAIS), which is from 1–5. b denotes the level value of risk assessment index system (RAIS), which is from 1–10. c means the Expert 1. d a five-level risk assessment standard is proposed by the standard [47,48], the description of which is excellent, good, satisfactory, fair and poor. Next, the use of 10-scores scale that defines criterion for the hazard, is generally preferable to quantitative analysis, which are excellent (8 ~ 10], good (6 ~ 8], satisfactory (4 ~ 6], fair (2 ~ 4] and poor (0 ~ 2].
Table 4. Results of the weights of indicators in Tian-yi Square.
Table 4. Results of the weights of indicators in Tian-yi Square.
First-Level IndicatorsWeights Second-Level IndicatorsWeights
Environmental factor0.200Architectural layout0.155
External traffic0.208
Public health environment0.271
Social stability0.135
Human factor0.259Crowd characteristic0.252
Negligence behavior0.201
Macroscopic fundamental diagram0.246
Safety awareness0.139
Equipment factors0.230Exit0.254
Guide sign0.160
Alarm system0.193
Firefighting system0.144
Broadcasting monitoring system0.249
Management factors0.311Personnel training0.238
Public health management0.193
Infrastructure management0.140
Event organization and management0.186
Emergency evacuation management0.243
Table 5. Results of Mass function of secondary indicators.
Table 5. Results of Mass function of secondary indicators.
Secondary IndicatorsR1R2R3R4R5 Θ
Architectural layout0.0020.9980.0000.0000.0000.000
External traffic1.0000.0000.0000.0000.0000.000
Public health environment1.0000.0000.0000.0000.0000.000
Social stability1.0000.0000.0000.0000.0000.000
Crowd characteristic1.0000.0000.0000.0000.0000.000
Negligence behavior1.0000.0000.0000.0000.0000.000
Macroscopic fundamental diagram1.0000.0000.0000.0000.0000.000
Safety awareness0.0000.9990.0010.0000.0000.000
Guide sign0.0000.0001.0000.0000.0000.000
Alarm system0.0000.0001.0000.0000.0000.000
Firefighting system0.0000.9950.0050.0000.0000.000
Broadcasting monitoring system1.0000.0000.0000.0000.0000.000
Personnel training0.0000.5770.4230.0000.0000.000
Public health management1.0000.0000.0000.0000.0000.000
Infrastructure management0.0020.9980.0000.0000.0000.000
Event organization and management0.0020.9980.0000.0000.0000.000
Emergency evacuation management0.0000.8860.1140.0000.0000.000
Table 6. Results of Mass function of primary indicators.
Table 6. Results of Mass function of primary indicators.
First-Level IndicatorsR1R2R3R4R5 Θ
Environmental factors1.0000.0000.0000.0000.0000.000
Human Factors1.0000.0000.0000.0000.0000.000
Infrastructure factors0.0000.0001.0000.0000.0000.000
Management factor0.0001.0000.0000.0000.0000.000
Table 7. Results of Mass function of First-level indicators.
Table 7. Results of Mass function of First-level indicators.
Primary IndicatorsExcellent a,bGoodSatisfactoryFairPoor
Environmental factors0.8450.1550.0000.0000.000
Human factors0.8610.1390.0000.0000.000
Infrastructure factors0.5030.1430.3540.0000.000
Management factors0.1940.6780.1280.0000.000
Note: a The criterion for judging risk level come from Production Safety Law of the People’s Republic of China [47], Detailed rule for the management and control system of electricity enterperise work safety risk classification [47], and DoD standard which is approved for use by all Military Departments and Defense Agencies within the Department of Defense (DoD) [48]. b Following the standard [47,48], in our study, a five-level risk assessment standard is proposed, the description of which is excellent, good, satisfactory, fair and poor. Next, the use of 10-scores scale that defines criterion for the hazard, is generally preferable to quantitative analysis, which are excellent (8~10], good (6~8], satisfactory (4~6], fair (2~4] and poor (0~2].

Share and Cite

MDPI and ACS Style

Zhou, J.; Mao, X.; Wang, Y.; Zhang, M.; Dong, S. Risk Assessment in Urban Large-Scale Public Spaces Using Dempster-Shafer Theory: An Empirical Study in Ningbo, China. Int. J. Environ. Res. Public Health 2019, 16, 2942.

AMA Style

Zhou J, Mao X, Wang Y, Zhang M, Dong S. Risk Assessment in Urban Large-Scale Public Spaces Using Dempster-Shafer Theory: An Empirical Study in Ningbo, China. International Journal of Environmental Research and Public Health. 2019; 16(16):2942.

Chicago/Turabian Style

Zhou, Jibiao, Xinhua Mao, Yiting Wang, Minjie Zhang, and Sheng Dong. 2019. "Risk Assessment in Urban Large-Scale Public Spaces Using Dempster-Shafer Theory: An Empirical Study in Ningbo, China" International Journal of Environmental Research and Public Health 16, no. 16: 2942.

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop