Next Article in Journal
Underutilized Fig (Ficus carica L.) Cultivars from Puglia Region, Southeastern Italy, for an Innovative Product: Dried Fig Disks
Next Article in Special Issue
Experimental Study on Downhole Acoustic Wave Propagation Characteristics in Curved Drill String
Previous Article in Journal
Vibration Characteristics of Concrete Pump Trucks with Multiple Postures and Multiple Conditions Based on the Secondary Development of HyperWorks
Previous Article in Special Issue
Frequent Alarm Pattern Mining of Industrial Alarm Flood Sequences by an Improved PrefixSpan Algorithm
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

A Risk-Data-Based Human Reliability Analysis for Chemical Experiments with Hazardous Processes

School of Safety Engineering, Beijing Institute of Petrochemical Technology, Beijing 102617, China
*
Author to whom correspondence should be addressed.
Processes 2023, 11(5), 1484; https://doi.org/10.3390/pr11051484
Submission received: 19 April 2023 / Revised: 10 May 2023 / Accepted: 11 May 2023 / Published: 13 May 2023
(This article belongs to the Special Issue Risk Assessment and Reliability Engineering of Process Operations)

Abstract

:
In recent years, chemical experiment accidents have frequently occurred, resulting in injuries and fatalities among researchers. It is crucial to address this issue to improve laboratory safety. Based on many publications, it is clear that human error makes a major contribution to many laboratory accidents which contain hazardous processes. However, there is limited research focusing on human error in laboratory safety, and there is also a lack of effective measures to assess Human Error Probability (HEP) for experimental process safety. Therefore, we propose an improved Cognitive Reliability and Error Analysis Method (CREAM) which is based on risk data to assess the HEP during hazardous processes in chemical experiments. The proposed method adjusts nine Common Performance Conditions (CPCs) in conventional CREAM to make them suitable to describe chemical experiments. Then, in contrast to the traditional approach, this study uses the definition of risk as the support to collect CPC data from the perspectives of possibility and severity, so as to improve the rationality of the data and decrease the subjectivity of expert judgment. Afterwards, the weight value of each CPC is calculated through Gray Relation Analysis (GRA) based on the collected risk data of each CPC. Meanwhile, the collected risk data are used to determine the fuzzy degrees of each CPC, the activated fuzzy If-Then rules, and the corresponding rule weights. Finally, the CPCs’ membership degrees, the CPCs’ weights, and If-Then rule weights are integrated together to acquire the HEP by defuzzification. In short, the proposed method changes the CPCs to ensure they are suitable, and then it innovatively uses risk data as the source to directly and indirectly determine the CPC’s fuzzy degree, the CPC’s importance weight, and the If-Then rule weight by fuzzy theory and GRA for collecting final HEP results. This method was tested on a selected chemical experiment, “preparation of active ferrous sulfide”, which contains hazardous processes. Through the proposed method, the HEP of each procedure in the selected risky chemical experiment could be determined, and among the procedures, the highest HEP was 1.51 × 10−3. In addition, with the HEP results, several subtasks with a high risk of human error could be identified. The results matched the real situations.

1. Introduction

In recent years, many accidents have happened during hazardous chemical experiments. On 21 July 2022, an accident of catalyst leakage occurred at a research laboratory of Petro China, which led to the death of one person and three people being injured. On 27 April 2022, at Central South University, an accident happened in a research laboratory during a chemical- and material-science-related experiment, which caused one PhD candidate’s death. The laboratory is an important site for scientific and technological innovation activities, which many young researchers are working in, so the safety issue should not be neglected [1,2]. Nowadays, research on laboratory safety attracts increasing attention [3,4], and many researchers share their findings and opinions. With the current focus on safety management and accident causation analysis, Keckler et al. developed a safety management method to enhance laboratory safety by using questionnaire data [5]. Frederico et al. found that improving safety culture can strengthen laboratory safety management [6]. Peng et al. used machine learning to improve the accuracy and solve the limitations of subjective evaluation in safety management [7]. Recently, many researchers have realized that human error is a considerable issue in laboratory safety. Ma et al. conducted accident analysis and causation research, finding that most defects in the laboratories and most root causes of laboratory accidents are human-related [8]. Furthermore, in identifying and analyzing the causes of laboratory fire and explosion accidents, it has also been recognized that human error is a major problem [9]. Due to the importance of human error, Zhao et al. recently published a study which views humans as a key element for laboratory safety assessment; moreover, Yang et al. assigned the human factor a significant role in the laboratory safety management framework [10,11]. With the deepening of laboratory safety research, human error has been acknowledged to be the main cause of laboratory accidents, and its influence should be considered [12,13]. Although people have realized that human error is a key element in triggering accidents during hazardous experiments, there is limited research on the quantification of human error in laboratory safety. Without effective human error quantification analysis, people may find it difficult to realize how human error impacts laboratory safety and it may be hard to manage and to avoid human errors. Therefore, HEP quantification is a crucial and necessary research point in the domain of laboratory safety.
Human Reliability Analysis (HRA) is widely used to quantitatively estimate HEP. Considering this study, it is also a reasonable choice to deal with HEP in conducting hazardous chemical experiments. So far, dozens of HRA methods have been designed and published, and those HRA methods can be divided into three generations. The first generation HRA methods include the Technique for Human Error Rate Prediction (THERP), Standardized Plant Analysis Risk-Human Reliability Analysis (SPAR-H), Success Likelihood Index Method (SLIM), and Human Error Assessment and Reduction Method (HEART) [14,15,16,17,18,19], and they focus on task characteristics. Typical methods in the second generation are CREAM and A Technique for Human Error Analysis (ATHEANA) [20,21]. They consider the influence of the environment on human reliability apart from task characteristics. However, the ATHEANA method is for nuclear industries, time-consuming, and weak in quantification, so this study does not consider that method. With in-depth research on safety psychology, cognitive science, and system safety science, the newest generation of human reliability methods includes Phoenix and the Integrated Human Event Analysis System (IDHEAS) [22,23], which have been published and applied in areas such as the oil and gas industries and nuclear power plants [24,25]. The above-mentioned HRA methods are mostly designed for specific domains such as the oil and gas industry, the petrochemical industry, and the nuclear industry. For this study, CREAM was selected as the main method for further research, since it is not designed for a specific domain, and it has strong functions for quantification.
The first publication of CREAM was in 1998, and it has received many expansions. As human factor data is always insufficient, fuzzy CREAM has become an effective research direction. Since 2006, fuzzy CREAM has been applied in the areas of oil and gas, the nuclear industry, railway transportation, aviation transportation, and maritime activities, with useful results [26,27,28]. So far, many improvements and expansions have been made based on the fuzzy CREAM framework. In 2009, relevant scholars endowed fuzzy CREAM with the function of probabilistic safety calculation through fuzzy min-max operators [29]. In addition, due to the dependence between CPCs, a Bayesian network was integrated with fuzzy CREAM to build a probability safety assessment model [30,31,32]. However, CPCs have different weights of influence on human reliability. Some researchers have promoted the credibility of analysis results based on fuzzy product-sum operator to combine CPCs’ weight and CPCs’ fuzzy degree together [33,34,35,36]. In addition, as If-Then rules activated by CPCs have weight, Zhang et al. integrated If-Then rule weights into the analytical process of fuzzy CREAM [37]. Furthermore, type-II fuzzy theory has been introduced to fuzzy CREAM to analyze CPC membership or weights and it can improve the quality of assessment results [38,39]. Although the above research promotes the efficiency of fuzzy CREAM, there is still a high level of subjectivity in CPC data. The quality and credibility of CPC data must be guaranteed because the CPC data act as a fundamental guarantee for the fuzzy degree of membership and probabilistic safety calculation.
This research aims to improve the collection of CPC data and to make sure the fuzzy CREAM process is more reasonable. Through analysis of the literature, it was found that some researchers have used the concept of risk to collect data from the perspectives of possibility and severity to describe performance-shaping factors (i.e., CPCs) [40]. Moreover, as the definition of risk is widely acknowledged, using it to collect CPCs’ performance data can be easily understood by people. Considering this study, it is worth adopting the definition of risk to determine a CPC’s performance data, and then using the CPC’s risk data to determine the CPC’s fuzzy degree, the CPC’s weight, and the If-then rule weight activated directly or indirectly by each CPC for the final calculation of the HEP.
Nonetheless, in contrast to the data of facilities and hardware, which have complete information, the recording of CPC data is always insufficient; namely, it is partially clear and partially unclear. Therefore, it is also worth introducing gray relation theory to deal with this situation. Gray relation theory was designed by Professor Deng Julong in the 1980s and has been applied in many areas [41,42], especially for GRA which has been successfully used for identifying important failure modes and key performance-shaping factors for human errors [43]. For this study, the collected risk data of each CPC can be processed by GRA for determining the importance weight of each CPC. The weight value of each CPC is integrated with the CPC’s fuzzy degree as determined by the CPC’s risk data through product-sum operator. In addition, as CREAM is designed for general industries but our study focuses on chemical experiments that contain risky processes, therefore, the CPCs should receive some changes to make sure they are suitable to describe any chemical experiment task.
According to the description above, this study proposes an improved CREAM to effectively and reasonably estimate the HEP for chemical experiments with hazardous processes. This study firstly adjusts nine CPCs involved in the CREAM to make them match chemical experiments. Then, this study employs the concept of risk to describe the performance of each CPC from the aspect of possibility and severity, so as to enhance the rationality and to decrease the subjectivity of CPC data. Afterwards, the risk data of each CPC, the GRA method, fuzzy calculation, and defuzzification are used together to estimate the HEP for the whole chemical experiment process. In summary, the proposed method provides a feasible passage to estimate the HEP for risky chemical experiments; moreover, traditional fuzzy data which are mainly determined by experts’ subjective evaluations are replaced by the risk data (possibility and severity) to decrease the subjective level and increase data credibility; in addition, CPCs’ fuzzy degrees, CPCs’ weights, and activated If-Then rule weights are effectively integrated for reasonably estimating the HEP value. Those together form the contributions of this study.
This research takes the case of ferrous sulfide preparation as the example to validate the proposed method. The remainder of this paper is structured as follows. Section 2 introduces the basic procedures of the proposed approach. Section 3 elaborates on the methodologies used and their principles. Section 4 uses the test case to verify our proposed method. Section 5 contains a discussion of the strengths and limitations of this study, and Section 6 presents our results and conclusions.

2. Research Procedure

This research provides an improved CREAM approach for estimating the HEP while carrying out chemical experiments with hazardous processes. The main procedure of the proposed method contains ten steps, which are presented in Figure 1.
The explanation of each step is as follows:
Step 1. Selecting a typical chemical experiment that has risky processes and then adjusting the CPCs in CREAM to make sure each CPC is appropriate to the experiment.
Step 2. Using Hierarchical Task Analysis (HTA) to decompose the selected chemical experiment into several subtasks and to describe them.
Step 3. According to the HTA result, and based on the concept of “Risk”, collecting performance data for each CPC in each subtask from the aspect of possibility and consequence.
Step 4. According to the collected risk data for each CPC, analyzing the CPC’s weight value through Gray Relation Analysis (GRA).
Step 5. Constructing a fuzzy degree of membership function, and then determining each CPC’s fuzzy degree in each subtask according to each CPC’s risk data.
Step 6. Based on the fuzzy CREAM, and according to the CPCs’ fuzzy degrees in each subtask, identifying the activated If-Then rules.
Step 7. With the activated If-Then rules, this study adopts fuzzy product-sum operator to combine a CPC’s weight values and a CPC’s fuzzy degrees together to determine the initial fuzzy membership degree of each COCOM.
Step 8. Based on fuzzy T-Norm theory, calculating the weights of each activated If-Then rule by multiplying each corresponding CPC’s fuzzy degrees together.
Step 9. Using fuzzy product-sum operator to integrate the initial fuzzy membership degree of each COCOM and each corresponding activated If-Then rule to determine the final fuzzy degree of each COCOM.
Step 10. With the final fuzzy degree of each COCOM, this study uses a defuzzification method (the center of area method is used here) to calculate the HEP data of each subtask in the selected risky chemical experiment.

3. Methods and Principles

We propose an approach for human reliability assessment and HEP estimation for chemical experiments with hazardous processes. The main methodologies and their corresponding principles are illustrated in this section.

3.1. CREAM

CREAM includes nine types of CPCs: adequacy of organization, working conditions, adequacy of human–computer interface, availability of programs and plans, number of targets achieved concurrently, available time, time slot of day, adequacy of training and experience, and quality of member cooperation [44]. These are adjusted due to the many differences between laboratory and industry systems, such as task objectives, the work experience of personnel, and the work environment. The adjusted CPCs are listed in Table 1. It should be noted that CPC9 is not used when the experiment only needs one person to carry it out.
Table 1 shows that each CPC has impacts on system human reliability, which are “improving,” “reducing,” and “insignificant,” with corresponding scores of +1, 0, and −1. Based on this, we analyze the CPCs based on the human error impact index, calculated by Equation (1).
H = X R e . X I m .
where H represents the total impact of all CPCs on the task. X R e . and X I m . are the numbers of CPCs that reduce and improve reliability, respectively. Figure 2 shows the relationship between the CPC and the COCOM. The horizontal and vertical coordinates indicate X R e . and X I m . , respectively.
CREAM calculates human error probabilities in accordance with the COCOM, which has scrambled, opportunistic, tactical, and strategic control modes, each corresponding to an error probability interval. Table 2 provides error probability intervals under different COCOM modes [45,46].

3.2. HTA Method

As shown in Figure 1, we use the HTA method to build task models for the selected case tasks, and then for human reliability analysis. The HTA task analysis method is used to hierarchically describe task goals and their subgoals, and it is often used to complete task analysis during human reliability analysis. Through structured analysis, this method disassembles the task to determine the goals of main tasks and subtasks. Goals are connected according to the corresponding logical relationship, so as to build the task model.

3.3. Collecting CPC Data Based on Risk Concept

CPC data is collected on the basis of the concept of risk (the combination of possibility and severity), so as to decrease subjective impacts caused by experts who only rate according to their experience. In our study, as shown in Figure 3, we used a Likert scale to collect the data for each CPC from the aspects of likelihood or severity.
The Likert scale is set from 0.0 to 10.0, indicating possibility or severity of consequences from low to high. In addition, in order to ensure the data quality, the experts invited for collecting risk data in this study all had a master’s degree with at least three years of experience conducting chemical experiments that include risky processes.

3.4. CPC Weight Calculation

As shown in Figure 1, this research adopted the GRA method to solve the issue of weight differences among the CPCs in each subtask. In accordance with the collected risk data and in combination with the GRA procedure, we built a gray matrix for the collected possibility and severity data, as seen in Equation (2):
G x = X 1 , 1 X 1 , 2 X 1 , n X 2 , 1 X 2 , 2 X 2 , n X m , 1 X m , 1 X m , n
The matrix contains m groups of data sequences ( 1 m i , i is the number of experts), each including n data points ( 1 n j , where j is the number of CPCs), and X m , n is the gray data for the n CPCs selected for the m experts in the task.
The reference sequence is determined according to G x in Equation (2), and it is generally the maximum or minimum data of a sequence. We select the maximum data as Equation (3):
X m a x = X m a x 1 , X m a x 2 , , X m a x n
where X m a x n is the maximum value of group n (in this research, n = 8). The gray relation coefficient of element k of group j data in the reference sequence is calculated as Equation (4),
G k j = min ( min X max X c ) + max ( max X m a x X c ) X m a x k X c j   k   + δ × max ( max X m a x X c )
where G k j is the gray relation coefficient of element k of group j data; min ( min X max X c ) and max ( max X m a x X c ) are the minimum and maximum values, respectively, of the absolute difference between the reference sequence and all comparison sequences; X m a x k is element k in the reference sequence; X c j   k is element k of sequence j in the decision data; and δ 0 , 1 is the distinguishing coefficient, where normally we set δ = 0.5 . The gray relation degree of data sequence j is determined as Equation (5),
G j = 1 n k = 1 n W k G k j
where G j is the gray relation degree of data sequence j; W k is the weight of expert k (experts have equal weight in this research), where G k j is calculated by Equation (4). The gray relation degree of each CPC based on the definition of risk in data sequence j can be obtained by Equation (6),
G j R . = G j P oss . G j C ons .
where G j R . is the gray relation degree of each CPC; G j P oss . and G j C ons . are the respective gray relation degrees for possibility and consequences of data sequence j. The weight data of each CPC corresponding to the sequence j can be acquired by normalization ( Q j R . ) as Equation (7).
Q j R . = G j R . n = 1 8 G j R .

3.5. HEP Calculation

We use risk data as the input to the fuzzy CREAM analytical process. The CPC risk data are acquired by multiplying possibility and severity data. The risk data of each CPC are described by a trapezoidal membership function as Equation (8),
h y = 0 y a y a b a a y b 1 b y c   d y d c c y d 0 y d
where h y is the trapezoidal membership function; y indicates risk data, and a, b, c, and d are parameters of the fuzzy degree of membership function. Reference is made to the research of Zhang et al. (2018) for the fuzzy degree of membership function in this research [34], which is used to determine the fuzzy degree of membership for each CPC, to activate If-Then rules to obtain statistics on CPC membership of each subtask, and to determine the COCOM according to Equation (1) and Figure 2. Then, with the CPC weight data, the initial membership degree of COCOM can be calculated by Equation (9),
G j co . = n = 1 8 M j n × Q j R .
where G j co . is the initial membership degree of COCOM; Q j R . is calculated using Equation (7), M j n indicates the fuzzy degree of membership of the nth CPC in subtask j under If-Then rules, and the fuzzy weight of m types of If-Then rules that exist in subtask j is calculated as Equation (10),
Q j m = M j 1 × M j 2 × M j n
where Q j m is the fuzzy weight of m types of If-Then rules that exist in subtask j; M j n indicates CPC membership of subtask j. Then, the COCOM membership of subtask j is calculated as Equation (11),
D j co . = G j co . × Q j m
where D j co . is the COCOM membership of subtask j; G j co . and Q j m are calculated using Equations (9) and (10), respectively. The end result of the HEP is acquired according to the Center of Area (COA) defuzzification method as Equation (12),
log 10 H E P = i = 1 n z L i z U i h i z z d z i = 1 n z L i z U i h i z d z
where log 10 H E P is the common logarithm of the HEP; z indicates membership of each COCOM,   h i z is the expression of the corresponding COCOM fuzzy degree of membership function, and z U i and z L i are the respective upper and lower limits of the membership function.

4. Experimental Verification

We selected the experiment of active ferrous sulfide preparation for model verification. Laboratory supplies include high-risk chemicals, such as sodium sulfide nonahydrate, ethyl alcohol, and ferrous sulfide, and other sources of hazard that can easily cause accidents, including electrical equipment (heating plates, centrifugal machines, and stirrers) and glass apparatuses (beakers and conical flasks). As the experiment has a high risk coefficient with much human–machine interaction, we selected it to verify the proposed model of human reliability analysis in a laboratory.
Through HTA analysis, the goals, tasks, subgoals, and subtasks of the ferrous sulfide preparation experiment were determined, with results as shown in Table A1 of Appendix A. According to the results of the HTA analysis, the experiment was divided into 12 main tasks and 26 subtasks. Subtask m of main task n was defined as “Tn.m.” The data of T10.3 (“checking activity in fume cupboard, quickly pouring samples from vial onto filter paper in a pile, and observing sample combustion or not”) were selected for illustration. As this is a solo experiment, CPC9 (quality of member cooperation) was not considered in the process of analysis. A survey of experts’ decisions was designed on the basis of 26 subtasks and eight CPCs. Four experts with experience in this experiment were invited to make judgements on the CPCs of each task from the perspectives of severity and possibility. Each expert held a master’s degree or above, with sufficient experience in experiments and in the management of ferrous sulfide preparation. They completed the risk data collection of the experiment task with accurate cognition of basic information and experimental risk. Using the subtask T10.3 as an example, four experts’ decision data can be seen in Table 3. The data for other subtasks can be similarly determined.
Table 3 shows that there were certain differences in their subjective judgments. Therefore, the issue of data incompleteness and non-uniqueness must be addressed. The gray relation degrees of possibility and consequences were calculated by Equations (4) and (5), respectively, and the gray relation degree of risk was calculated by Equation (6). In the end, the GRA weight result of each CPC was acquired after normalization through Equation (7). The gray relation degree weight of each CPC in T10.3 is shown in Table 4.
Risk data were brought into the CPC fuzzy degree of membership function. Fuzzy sets and corresponding membership were used to interpret the risk data. The fuzzy degree of membership function was described according to parameters a, b, c, and d in Equation (8) and used to interpret the risk data. For instance, fuzzy sets of CPC1, “adequacy of organization/management system,” are very efficient [a, b, c, d] = [0, 0, 20, 30], efficient [a, b, c, d] = [20, 30, 40, 60], inefficient [a, b, c, d] = [40, 60, 60, 90], or deficient [a, b, c, d] = [60, 90, 100, 100]. All CPC membership functions are shown in Figure 4.
The data of the fuzzy degree of membership for each task on every CPC were calculated using Equation (8), as seen in Figure 4. See Table 5 for the fuzzy degree of membership of T10.3.
It was found that in T10.3, five CPCs cover two levels, so it can activate 25 = 32 If-Then rules to determine the COCOM. One example of an activated If-Then rule is CPC1 is Efficient (0.33), CPC2 is Advantageous (1), CPC3 is Adequate (0.88), CPC4 is Acceptable (0.57), CPC5 is Fewer than capacity (1), CPC6 is Adequate (0.93), CPC 7 is Day-time (1), and CPC8 is Inadequate (0.17).
Based on Equation (9) and the CPC weight data in Table 4, the initial membership degree of COCOM for T10.3 can be collected. Using the presented If-Then rule as an example, its COCOM degree is: 0.1061 × 0.33 + 0.1430 × 1 +   + 0.1959 × 0.17 = 0.68 with Tactical. Then, as each activated If-Then rule has the rule weight, Equation (10) is selected. Still using the presented If-Then rule as the example, its rule weight is 0.33 × 1 × × 0.17 = 0.026. For other activated rules, their weights can be determined in the same way. Afterwards, the final membership degree of each COCOM can be collected using Equation (11). Here, using the presented If-Then rule in T10.3 as an example, its final COCOM is 0.68 × 0.018 = 0.01224 (Tactical). For all activated If-Then rules in T10.3 and other subtasks, their final membership degrees of COCOMs can be calculated similarly. Table 6 shows the final COCOM degrees.
With the final membership degree of each COCOM, we can calculate the corresponding HEP data. According to the HEP interval of each COCOM (see Table 2), the fuzzy membership functions of each COCOM are shown as Figure 5. Then, using Equation (12), the HEP of each task was acquired after COA defuzzification and logarithmization, and the corresponding results of each task are shown in Figure 6.
According to the final results in Figure 6, the HEP of each subtask can be calculated. The task during which human errors will most likely appear in this experiment is T11.1 (“The liquid after separation, the liquid after water washing, and the liquid after alcohol washing need to be put into the corresponding waste bucket”) with HEP = 1.51 × 10−3. For the HEPs for other subtasks, Table A1 in the Appendix A gives their results.
Overall, the HEPs of subtasks T10.3, T11.1, and T11.2 were the top three highest. These subtasks not only have sources of hazards with high risk but are also involved in the complicated procedure of combustion and waste disposal. In addition, the HEPs of subtasks T1.1, T2.1, T3.1, T4.1, and T9.2 were moderately high. In these tasks, there are sources of hazard with high risk, such as heating plates, sodium sulfide, and ammonium ferrous sulfate, as well as many human–machine interaction activities. It also is found that the HEPs in other subtasks are low, and that those subtasks have fewer sources of hazards and low complexity of human operation. The results of human reliability analysis conform to the human error situation of this experiment, so the proposed model has effective results.

5. Discussion

According to the description above, it can be seen that the method provided by this study is applicable and effective for HEP estimation for chemical experiments that contain hazardous processes. This method has attempted to consider the characteristics of chemical experiment processes, so as to provide some modifications for the nine CPCs in CREAM. Moreover, based on the definition of risk, this proposed method uses the risk data of each CPC for further calculating the CPCs’ fuzzy degrees, the CPCs’ weights, and If-Then rule weights. Through this method, experts can judge each CPC from more dimensions (possibility and severity), rather than just experts’ subjective judgments. In addition, compared with many previous publications, this study designs a fuzzy operating approach that can absorb the influence of CPCs’ degrees, CPCs’ weights, and If-Then rule weights together during HEP calculation.
However, this study still has limitations. Although some changes have been made to the nine CPCs, more considerations are still required to make sure each CPC is suitable to describe the characteristics of dangerous chemical experiment processes. Using risk data instead of traditional fuzzy data can increase data credibility and decrease subjectivity, but the uncertainty issues are still existing, so some data processing methods should be involved to decrease the uncertainty level of the risk data. Furthermore, software should be designed to allow the whole method to be conducted and presented by computer. In summary, the proposed method is an effective attempt to deal with human-related safety issues for risky chemical experiments, but more specific efforts are still necessary.

6. Conclusions

This study provides an effective method to estimate the HEPs for chemical experiments with hazardous processes. The proposed approach firstly adjusts nine CPCs to make sure they can match the characteristics of chemical experiments. Next, for improving the rationality and for reducing the subjectivity of the CPC data, the method uses the definition of risk as the support to collect data to express the performance level of each CPC. Then, this study innovatively chooses those collected risk data as the basis to directly and indirectly determine the CPCs’ fuzzy degrees, the CPCs’ weight values, and the weight value of each activated If-Then rule by fuzzy membership function, GRA, and fuzzy calculation. Afterwards, this method combines the CPC’s degree, the CPC’s weight, and the If-Then rule weight together by using fuzzy calculating operator to determine the final fuzzy degree of each COCOM. Finally, the COA method is selected to give defuzzification to the COCOM’s fuzzy degrees for assessing the HEP results. Those together form the risk-data-based HRA method for operation safety during chemical experiments.
Based on the analysis results, we verified that the model can calculate the HEP of the experiment of the selected test case, “active ferrous sulfide preparation”. With the proposed method, the highest HEP was 1.51 × 10−3 for the subtask “The liquid after separation, the liquid after water washing, and the liquid after alcohol washing need to be put into the corresponding waste bucket”. Therefore, some appropriate measures should be taken. For instance, setting up a risk identifier to remind everyone involved in the dangerous experiment of safety awareness and to ask them to be in compliance with experimental regulations; setting up a work safety checklist to make sure the correct operations are carried out; and assigning a staff member to supervise safety when carrying out an experiment that has risky processes.
Through the approach designed in this research, quantitative human error data can be acquired to enhance people’s awareness of human risk and improve laboratory management. Meanwhile, it can help the management team to prevent human error and reduce unsafe behavior in a more targeted way. However, some further development of this method is still required. The design of the CPCs for chemical experiments with hazardous processes should be deeply analyzed to ensure they are closely related to the task processes and task characteristics. Additionally, the source data used in this study still have a level of uncertainty, so solutions such as type-II fuzzy membership function can be selected to replace traditional fuzzy membership function for HEP estimation in the future.

Author Contributions

Conceptualization, R.Z.; methodology, R.Z.; software, J.G.; validation, R.Z., J.Z. and J.G.; formal analysis, J.G.; investigation, J.Z., H.C., Q.Z. and Z.Z.; resources, J.G. and J.Z.; data curation, H.C., Q.Z. and Z.Z.; writing—original draft preparation, R.Z. and J.G.; writing—review and editing, R.Z.; visualization, J.G.; supervision, R.Z.; project administration, R.Z.; funding acquisition, R.Z. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the National Natural Science Foundation of China (Grant No. T2192933), the R&D Program of Beijing Municipal Education Commission (KM202210017002), and the University Research Training (URT) project of the Beijing Institute of Petrochemical Technology (Project No. 2023J00076).

Data Availability Statement

Not applicable.

Acknowledgments

The authors would like to thank those who gave valuable help in improving this work.

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A

As mentioned above, this study analyzed the experimental tasks using the HTA method, and the analysis results are shown in Table A1 in Appendix A.
Table A1. Task analysis of active ferrous sulfide preparation experiment.
Table A1. Task analysis of active ferrous sulfide preparation experiment.
Main TaskSubtaskHEP
T1. Preparation before experimentT1.1 Wearing personal protective equipment1.77 × 10−4
T1.2 Preparing chemicals, instruments, and equipment required for experiment1.36 × 10−4
T2. Preparation of deoxygenated and deionized waterT2.1 Using conical flask to hold deionized water and heating with heat plate in fume cupboard at 200 °C1.89 × 10−4
T2.2 Placing conical flask into basin for cooling1.30 × 10−4
T3. Preparation of sodium sulfide aqueous solutionT3.1 Powering-on balance, debugging, and weighing a certain amount of sodium sulfide solid to beaker1.51 × 10−4
T3.2 Adding water to beaker, rotor, covering with sealing film, and placing onto magnetic stirrer to stir1.30 × 10−4
T4. Preparation of ferrous sulfate aqueous solutionT4.1 Powering-on balance, debugging, weighing a certain amount of ammonium ferrous sulfide and ascorbic acid to beaker1.68 × 10−4
T4.2 Adding a certain amount of deoxygenated and deionized water to the beaker, using glass rod to stir1.30 × 10−4
T5. Preparation of ferrous sulfide aqueous solutionT5.1 Slowly pouring sodium sulfide aqueous solution into ammonium ferrous sulfide aqueous solution, and gently stirring1.30 × 10−4
T5.2 Covering with sealing film and putting into refrigerator for cooling for half an hour1.30 × 10−4
T6. Solid–liquid separationT6.1 Taking solution out of refrigerator and evenly pouring into centrifuge tube1.30 × 10−4
T6.2 Placing centrifuge tube in centrifuge, powering on centrifuge, setting parameters, and executing centrifugation1.30 × 10−4
T7. Water washing three timesT7.1 Pouring liquid out of centrifuge tube and adding appropriate amount of deoxygenated and deionized water1.30 × 10−4
T7.2 Stirring with glass rod in centrifuge tube to realize water washing1.30 × 10−4
T7.3 Placing into centrifuge, setting up parameters, and starting centrifugation1.30 × 10−4
T8. Alcohol washing three timesT8.1 Pouring liquid out of centrifuge tube and adding appropriate amount of ethyl alcohol1.30 × 10−4
T8.2 Stirring with glass rod in centrifuge tube to realize ethyl alcohol washing1.30 × 10−4
T8.3 Placing into centrifuge, setting up parameters, and starting centrifugation1.30 × 10−4
T9. Vacuum dryingT9.1 Opening centrifuge tube and putting into glovebox with airlock chamber1.30 × 10−4
T9.2 Using motor to extract air from airlock chamber, and drying in vacuum environment1.57 × 10−4
T10. Package inspection and verificationT10.1 Taking out sample to glovebox and grinding into black powder1.30 × 10−4
T10.2 Putting a small sample into vial and taking to fume cupboard; packaging reminder of samples1.30 × 10−4
T10.3 Checking activeness in fume cupboard, quickly pouring sample from vial onto filter paper as a pile, and observing sample combustion or not1.03 × 10−3
T11. Waste disposalT11.1 The liquid after separation, the liquid after water washing, and the liquid after alcohol washing need to be put into the corresponding waste bucket1.51 × 10−3
T11.2 Sample might contain original sample after combustion; placing into fume cupboard to wait until complete reaction in air and placing separately into corresponding sites of solid waste4.38 × 10−4
T12. Instrument cleaningT12.1 Cleaning instrument and using diluted sulfuric acid to soak area where some parts cannot be cleaned away1.30 × 10−4

References

  1. Li, X.H.; Zhang, L.Y.; Zhang, R.R.; Yang, M.; Li, H. A semi-quantitative methodology for risk assessment of university chemical laboratory. J. Loss Prev. Process Ind. 2021, 72, 104553. [Google Scholar] [CrossRef]
  2. Gao, X.; Abdul, R.; Hizaddin, H.; Buthiyappan, A.; Bello, M. Bayesian networks based laboratory retrofitting towards inherent safety: A risk-based implementation framework. J. Loss Prev. Process Ind. 2023, 83, 105036. [Google Scholar] [CrossRef]
  3. Love, T.; Roy, K.; Sirinides, P. A National study examining safety factors and training associated with STEM education and CTE laboratory accidents in the United States. Saf. Sci. 2023, 160, 1–13. [Google Scholar] [CrossRef]
  4. Menard, A.D.; Trant, J. A review and critique of academic lab safety research. Nat. Chem. 2019, 12, 1–9. [Google Scholar] [CrossRef]
  5. Keckler, M.; Anderson, K.; McAllister, S.; Rasheed, J.; Noble-Wang, J. Development and implementation of evidence-based laboratory safety management tools for a public health laboratory. Saf. Sci. 2019, 117, 205–216. [Google Scholar] [CrossRef]
  6. Frederico, M.; Ana, R.; Kewin, R. Safety culture maturity in a civil engineering academic laboratory. Saf. Sci. 2021, 134, 105076. [Google Scholar]
  7. Peng, T.; Li, C.; Zhou, X.B. Application of machine learning to laboratory safety management assessment. Saf. Sci. 2019, 120, 263–267. [Google Scholar] [CrossRef]
  8. Ma, L.H.; Ma, X.X.; Zhang, J.W.; Yang, Q.; Wei, K. A methodology for dynamic assessment of laboratory safety by SEM-SD. Int. J. Environ. Res. Public Health. 2021, 18, 6545. [Google Scholar] [CrossRef] [PubMed]
  9. Ma, L.H.; Ma, X.X.; Xing, P.F.; Yu, F.Y. A hybrid approach based on the HFACS-FBN for identifying and analysing human factors for fire and explosion accidents in the laboratory. J. Loss Prev. Process Ind. 2021, 75, 104675. [Google Scholar] [CrossRef]
  10. Zhao, J.; Cui, H.; Wang, G.; Zhang, J.; Yang, R. Risk assessment of safety level in university laboratories using questionnaire and Bayesian network. J. Loss Prev. Process Ind. 2023, 83, 105054. [Google Scholar] [CrossRef]
  11. Yang, J.; Xuan, S.; Hu, Y.; Liu, X.; Bian, M.; Chen, L.; Lv, S.; Wang, P.; Li, R.; Zhang, J.; et al. The framework of safety management on university laboratory. J. Loss Prev. Process Ind. 2022, 80, 104871. [Google Scholar] [CrossRef]
  12. Bai, M.Q.; Liu, Y.; Qi, M.; Roy, N.; Shu, C.M.; Khan, F.; Zhao, D.F. Current status, challenges, and future directions of university laboratory safety in China. J. Loss Prev. Process Ind. 2021, 74, 104671. [Google Scholar] [CrossRef]
  13. Bye, A. Future needs of human reliability analysis: The interaction between new technology, crew roles and performance. Saf. Sci. 2023, 158, 105962. [Google Scholar] [CrossRef]
  14. Asma, Z.; Naser, H.; Salman, F.; Mehdi, J. Applying analytic hierarchy process and failure likelihood index method (AHP-FLIM) to assess human reliability in critical and sensitive jobs of a petrochemical industry. Heliyon 2022, 8, e09509. [Google Scholar]
  15. Kumar, M.; Sivaprakasam, R.; Prabu, V. Application of fuzzy HEART and expert elicitation for quantifying human error probabilities in LPG refuelling station. J. Loss Prev. Process Ind. 2017, 48, 186–198. [Google Scholar] [CrossRef]
  16. Ribeiro, A.; Sousa, A.; Duarte, J.; Frutuoso, P. Human reliability analysis of the Tokai-Mura accident through a THERP–CREAM and expert opinion auditing approach. Saf. Sci. 2016, 87, 269–279. [Google Scholar] [CrossRef]
  17. Ahn, S.; Kurt, R.; Akyuz, E. Application of a SPAR-H based framework to assess human reliability during emergency response drill for man overboard on ships. Ocean Eng. 2022, 251, 111089. [Google Scholar] [CrossRef]
  18. Erdem, P.; Akyuz, E. An interval type-2 fuzzy SLIM approach to predict human error in maritime transportation. Ocean Eng. 2021, 232, 109161. [Google Scholar] [CrossRef]
  19. Aliabadi, M. Human error analysis in furnace start-up operation using HEART under intuitionistic fuzzy environment. J. Loss Prev. Process Ind. 2020, 69, 104372. [Google Scholar] [CrossRef]
  20. Zhou, Q.J.; Wong, Y.D.; Loh, H.S.; Yuen, K.F. A fuzzy and Bayesian network CREAM model for human reliability analysis—The case of tanker shipping. Saf. Sci. 2018, 105, 149–157. [Google Scholar] [CrossRef]
  21. Forester, J.; Bley, D.; Cooper, S.; Lois, E.; Siu, N.; Kolaczkowski, A.; Wreathall, J. Expert elicitation approach for performing ATHEANA quantification. Reliab. Eng. Syst. Saf. 2004, 83, 207–220. [Google Scholar] [CrossRef]
  22. Ekanem, N.; Mosleh, A.; Shen, S.H. Phoenix—A model-based human reliability analysis methodology: Qualitative analysis procedure. Reliab. Eng. Syst. Saf. 2015, 145, 301–315. [Google Scholar] [CrossRef]
  23. Liao, H.F.; Groth, K.; Adams, S. Challenges in leveraging existing human performance data for quantifying the IDHEAS HRA method. Reliab. Eng. Syst. Saf. 2015, 144, 159–169. [Google Scholar] [CrossRef]
  24. Ramos, M.; Droguett, E.; Mosleh, A.; Moura, M. A human reliability analysis methodology for oil refineries and petrochemical plants operation: Phoenix-PRO qualitative framework. Reliab. Eng. Syst. Saf. 2019, 193, 106672. [Google Scholar] [CrossRef]
  25. Yin, Z.J.; Li, Z.Z.; Liu, Z.P.; Yang, D.F.; Zhang, J.H.; Long, L.; Zhang, Y.J.; Gong, B.Y. Collection of IDHEAS-based human error probability data for nuclear power plant commissioning through expert elicitation. Ann. Nucl. Energy 2023, 181, 109544. [Google Scholar] [CrossRef]
  26. Zhou, Q.J.; Wong, Y.; Xu, H.; Thai, V.; Loh, H.S.; Yuen, K.F. An enhanced CREAM with stakeholder-graded protocols for tanker shipping safety application. Saf. Sci. 2017, 95, 140–147. [Google Scholar] [CrossRef]
  27. Konstandinidou, M.; Nivolianitou, Z.; Kiranoudis, C.; Markatos, N.C. A fuzzy modeling application of CREAM methodology for reliability. Reliab. Eng. Syst. Saf. 2006, 91, 706–716. [Google Scholar] [CrossRef]
  28. Ung, S.T. A weighted CREAM model for maritime human reliability analysis. Saf. Sci. 2015, 72, 144–152. [Google Scholar] [CrossRef]
  29. Marseguerra, M.; Zio, E.; Librizzi, M. Human reliability analysis by fuzzy “CREAM”. Risk Anal. 2007, 27, 137–154. [Google Scholar] [CrossRef]
  30. Fan, H.; Enshaei, H.; Jayasinghe, S. Human error probability assessment for LNG bunkering based on fuzzy Bayesian network-CREAM model. J. Mar. Sci. Eng. 2022, 10, 333. [Google Scholar] [CrossRef]
  31. Ghasemi, F.; Ghasemi, A.; Kalatpour, O. Prediction of human error probability during the hydrocarbon road tanker loading operation using a hybrid technique of fuzzy sets, Bayesian network, and CREAM. Int. J. Occup. Saf. Ergon. 2021, 28, 1342–1352. [Google Scholar] [CrossRef]
  32. Wang, C.; Liu, Y.P.; Yu, C.; Zheng, Y.Y.; Wang, G.R. Dynamic risk analysis of offshore natural gas hydrates depressurization production test based on fuzzy CREAM and DBN-GO combined method. J. Nat. Gas Sci. Eng. 2021, 91, 103961. [Google Scholar] [CrossRef]
  33. Ung, S.T. Human error assessment of oil tanker grounding. Saf. Sci. 2018, 104, 16–28. [Google Scholar] [CrossRef]
  34. Ung, S.T. Evaluation of human error contribution to oil tanker collision using fault tree analysis and modified fuzzy Bayesian network based CREAM. Ocean Eng. 2019, 179, 159–172. [Google Scholar] [CrossRef]
  35. Elidolu, G.; Akyuz, E.; Arslan, O.; Arslanoğlu, Y. Quantitative failure analysis for static electricity-related explosion and fire accidents on tanker vessels under fuzzy bow-tie CREAM approach. Eng. Fail. Analysi. 2022, 131, 105917. [Google Scholar] [CrossRef]
  36. Wang, N.; Du, X.; Zhang, M.; Xu, C.S.; Lu, X.Y. An improved weighted fuzzy CREAM model for quantifying human reliability in subway construction: Modeling, validation, and application. Hum. Factors Ergon. Manuf. Serv. Ind. 2020, 30, 248–265. [Google Scholar] [CrossRef]
  37. Zhang, R.Y.; Tan, H.; Afzal, W. A modified human reliability analysis method for the estimation of human error probability in the offloading operations at oil terminals. Process Saf. Prog. 2021, 40, 84–92. [Google Scholar] [CrossRef]
  38. Xiaoqing, C.; Liu, X.W.; Qin, Y. An extended CREAM model based on analytic network process under the type-2 fuzzy environment for human reliability analysis in the high-speed train operation. Qual. Reliab. Eng. Int. 2020, 37, 284–308. [Google Scholar]
  39. Chuan, L.; Xu, Q.; Huang, Y. An HFM-CREAM model for the assessment of human reliability and quantification. Qual. Reliab. Eng. Int. 2022, 38, 2372–2387. [Google Scholar]
  40. Liu, P.; Lv, X.; Qiu, Y.P.; He, J.D.; Tong, J.J.; Zhao, J.; Li, Z.Z. Identifying key performance shaping factors in digital main control rooms of nuclear power plants: A risk-based approach. Reliab. Eng. Syst. Saf. 2017, 167, 264–275. [Google Scholar] [CrossRef]
  41. Deng, J. Introduction to grey system theory. J. Grey Syst. 1982, 1, 1–24. [Google Scholar]
  42. Chen, L.L. Research of the safety path of university laboratory basing on the analysis of grey correlation degree. J. Intell. Fuzzy Syst. 2020, 40, 1–8. [Google Scholar] [CrossRef]
  43. Zhang, R.Y.; Meng, H.X.; Ge, J.; Tan, H. A method for identifying the key performance shaping factors to prevent human errors during oil tanker offloading work. J. Mar. Sci. Eng. 2022, 10, 688. [Google Scholar] [CrossRef]
  44. Hollnagel, E. Cognitive Reliability and Error Analysis Method (CREAM); Elsevier: Amsterdam, The Netherlands, 1998. [Google Scholar]
  45. Boroun, R.; Tahmasebi, B.Y.; Mosavianasl, Z.; Shirali, G. Fuzzy Logic in HEART and CREAM methods to assess human error and find an optimum method using a hierarchical fuzzy system: A case study in a steel factory. Int. J. Occup. Hyg. 2022, 13, 105–119. [Google Scholar] [CrossRef]
  46. Abbassinia, M.; Kalatpour, O.; Motamedzade, M.; Soltanian, A.; Mohammadfam, I. Dynamic human error assessment in emergency using fuzzy Bayesian CREAM. J. Res. Health Sci. 2022, 20, e00468. [Google Scholar] [CrossRef] [PubMed]
Figure 1. The flow chart of this study.
Figure 1. The flow chart of this study.
Processes 11 01484 g001
Figure 2. Relationship between CPC and COCOM.
Figure 2. Relationship between CPC and COCOM.
Processes 11 01484 g002
Figure 3. Likert scale used in this study.
Figure 3. Likert scale used in this study.
Processes 11 01484 g003
Figure 4. Fuzzy membership functions of each CPC.
Figure 4. Fuzzy membership functions of each CPC.
Processes 11 01484 g004
Figure 5. Fuzzy membership function of COCOM.
Figure 5. Fuzzy membership function of COCOM.
Processes 11 01484 g005
Figure 6. Human error probability of each task.
Figure 6. Human error probability of each task.
Processes 11 01484 g006
Table 1. CPCs adjusted according to the laboratory setting.
Table 1. CPCs adjusted according to the laboratory setting.
CPCCPC LevelEffectsEffects Index
Adequacy of organization/managementVery efficientImproved+1
EfficientNot significant0
InefficientReduced−1
DeficientReduced−1
Lab conditionsAdvantageousImproved+1
CompatibleNot significant0
IncompatibleReduced−1
Availability of experimental support conditionSupportiveImproved1
AdequateNot significant0
TolerableNot significant0
InappropriateReduced−1
Rational task orderAppropriateImproved1
AcceptableNot significant0
InappropriateReduced−1
Complexity of taskFewer than capacityImproved1
Matching current capacityNot significant0
More than capacityReduced−1
Available time of taskAdequateImproved1
Temporarily inadequateReduced−1
Continuously inadequateReduced−1
Time of dayDay-timeNot significant0
Night-timeReduced−1
Adequacy of training and safety awarenessAdequate, high experienceImproved+1
Adequate, limited experienceNot significant0
InadequateReduced−1
Quality of member cooperation (experiments with only one person need not consider this CPC)Very efficientImproved+1
EfficientNot significant0
InefficientNot significant0
DeficientReduced−1
Table 2. HEP and logarithm of HEP for each COCOM.
Table 2. HEP and logarithm of HEP for each COCOM.
COCOMHEPlog10 HEP
Strategic(0.00005, 0.01)(−5.3, −2)
Tactical(0.01, 0.1)(−3, −1)
Opportunistic(0.01, 0.5)(−2, −0.3)
Scrambled(0.1, 1)(−1, 0)
Table 3. Experts’ judgements for the subtask T10.3 (possibility/consequences).
Table 3. Experts’ judgements for the subtask T10.3 (possibility/consequences).
CPC1CPC2CPC3CPC4CPC5CPC6CPC7CPC8
Expert15.8/7.26.9/9.28.2/8.86.0/7.86.7/6.15.7/4.35/1.07.6/9.1
Expert24.0/3.03.0/2.05.0/3.05.0/2.02.0/2.04.0/3.02.0/2.05.0/4.0
Expert34.0/5.06.0/5.06.0/5.09.0/7.06.0/4.06.0/5.06.0/5.09.0/6.0
Expert43.0/7.09.0/6.01.0/6.03.0/5.01.5/6.02.0/7.01.0/7.03.0/8.0
Table 4. Weights of CPCs (taking T10.3 as an example).
Table 4. Weights of CPCs (taking T10.3 as an example).
CPC1CPC2CPC3CPC4CPC5CPC6CPC7CPC8
0.10610.14300.14080.14450.08750.10190.07990.1959
Table 5. Fuzzy degree of membership of each CPC (taking T10.3 as an example).
Table 5. Fuzzy degree of membership of each CPC (taking T10.3 as an example).
No.CPCFuzzy Degree of Membership
1Adequacy of organization/managementVery efficient (0.67), Efficient (0.33)
2Lab conditionsAdvantageous (1)
3Availability of experimental support conditionSupportive (0.12), Adequate (0.88)
4Rational task orderAppropriate (0.43), Acceptable (0.57)
5Complexity of taskFewer than capacity (1)
6Available time for taskAdequate (0.93), Temporarily inadequate (0.07)
7Time of dayDay-time (1)
8Adequacy of training and safety awarenessAdequate, limited experience (0.83), Inadequate (0.17)
Table 6. COCOMs’ final membership degrees.
Table 6. COCOMs’ final membership degrees.
TaskOrderStrategicTacticalOpportunisticScrambled
T1.110.93250.067500
T1.221000
T2.130.91570.084300
T2.241000
T3.150.97560.024400
T3.261000
T4.170.94370.056300
T4.281000
T5.191000
T5.2101000
T6.1111000
T6.2121000
T7.1131000
T7.2141000
T7.3151000
T8.1161000
T8.2171000
T8.3181000
T9.1191000
T9.2200.96040.039600
T10.1211000
T10.2221000
T10.3230.40470.595300
T11.1240.28330.716700
T11.2250.62440.375600
T12.1261000
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Zhang, R.; Ge, J.; Zhang, J.; Cui, H.; Zhang, Q.; Zhang, Z. A Risk-Data-Based Human Reliability Analysis for Chemical Experiments with Hazardous Processes. Processes 2023, 11, 1484. https://doi.org/10.3390/pr11051484

AMA Style

Zhang R, Ge J, Zhang J, Cui H, Zhang Q, Zhang Z. A Risk-Data-Based Human Reliability Analysis for Chemical Experiments with Hazardous Processes. Processes. 2023; 11(5):1484. https://doi.org/10.3390/pr11051484

Chicago/Turabian Style

Zhang, Renyou, Jun Ge, Jinchao Zhang, Huanhuan Cui, Qinhao Zhang, and Zexing Zhang. 2023. "A Risk-Data-Based Human Reliability Analysis for Chemical Experiments with Hazardous Processes" Processes 11, no. 5: 1484. https://doi.org/10.3390/pr11051484

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop