Next Article in Journal
Active Commuting, Physical Activity, and Sedentary Behaviors in Children and Adolescents from Spain: Findings from the ANIBES Study
Previous Article in Journal
Assessment of the Risk of Oral Cancer Incidence in A High-Risk Population and Establishment of A Predictive Model for Oral Cancer Incidence Using A Population-Based Cohort in Taiwan
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:

Comprehensive Evaluation System of Occupational Hazard Prevention and Control in Iron and Steel Enterprises Based on A Modified Delphi Technique

School of Public Health, North China University of Science and Technology, Tangshan 063210, China
Hebei Province Key Laboratory of Occupational Health and Safety for Coal Industry, North China University of Science and Technology, Tangshan 063210, China
Authors to whom correspondence should be addressed.
Int. J. Environ. Res. Public Health 2020, 17(2), 667;
Submission received: 21 November 2019 / Revised: 17 January 2020 / Accepted: 19 January 2020 / Published: 20 January 2020
(This article belongs to the Section Public Health Statistics and Risk Assessment)


The study designs a comprehensive evaluation system for the prevention and control of occupational hazards, calculates its weight coefficient, and provides a potential strategic and effective tool for the scientific evaluation of occupational hazards in the iron and steel enterprises. The system was established through induction and analysis of relevant literature, personal interview, theoretical analysis, Delphi expert consultation, and special group discussions. Using an improved analytical hierarchy process fuzzy comprehensive evaluation model and on the basis of the improved Delphi expert investigation, the weight of the operability comprehensive evaluation index system is constructed. A three-level index system is established on the basis of harmful factors of occupational activities, health status of employees, protection facilities of occupational hazards, occupational health management, and so on. The index system structure is 4-20-95, and the weight coefficients of the four dimensions are 0.2516, 0.2428, 0.2550, and 0.2506. The recovery rate of the questionnaire was 82.5%, 100.0%, and 100.0%. The effective rates were 75.0%, 100.0%, and 100.0%. Conversely, the expert authority coefficients of the four dimensions are 0.875, 0.769, 0.832 and 0.800. Results show that the consistency factors of the four dimensions are statistically significant. Cronbach’s α coefficient, standardized Cronbach’s α coefficient, and split-half reliability of the comprehensive evaluation index system are 0.959, 0.950, and 0.810, respectively. After factor analysis, four common factors were extracted on the basis of expert opinions, and the cumulative variance was 63.1%. The comprehensive evaluation system for the prevention and control of occupational hazards in the iron and steel enterprises proposed by the study is relatively complete and reasonable.

1. Introduction

The iron and steel industry is a relatively important basic industry for national economies worldwide, such as China, Japan, India, America, and Russia. Out of these countries, it is particularly not the least prominent for China [1], and relevant data have showed that the Chinese output of crude steel reached 808 million tons in 2016, which accounted for 49.6% of the worldwide production [2]. At present, China has nearly 1000 large and medium-sized iron and steel plants, in which a large number of labor employees are assiduously occupied with contributing to the booming and flourishing development of China’s iron and steel industry [3]. However, the industry can produce many occupational hazards during its production processes in combination with the large population base exposed to such occupational hazards [2,4,5,6,7,8]. This scenario is more likely to result in numerous occupational diseases and potential economic losses [9,10,11,12]. In addition, with the increasingly stringent environmental policy, especially under the growing smog pressure at the national level, the resultant prevention and control effect on occupational hazards in iron and steel foundries has become a key indicator for the evaluation of its development status [1,13]. Li suggested to implement various countermeasures to solve the problem of prevention and control of occupational diseases in Anshan from the aspects of legislation, capital investment and law enforcement [14]. Xing established an index system for the prevention and control of occupational hazards in the Bohai Bay support base of CNPC and applied it [15]. He used analytic hierarchy process, comprehensive fuzzy evaluation and Delphi based on field investigation and expert consultation to build seven indicators of occupational hazard risk factors. The assessment results showed that the hazard level was the seventh level (the lowest hazard). Zhang e used Delphi and the least squares method to construct the regression model of the occupational disease prevention performance index, and finally formed the occupational disease prevention performance system, and made a comprehensive evaluation of nine regions in Shandong Province [16]. In view of the abovementioned issues, formulating a specific system for appraising occupational hazards in iron and steel facilities in a scientific and comprehensive manner is imperative to mitigate predominant and recognized hazardous threats. The system will be of great practical significance in reducing the morbidity and mortality of occupational hazard-induced diseases [17]. However, to date, a specific study on a comprehensive evaluation system for hazardous working conditions in the Chinese context remains lacking. Therefore, based on the full consideration of the mechanisms and features of occupational hazards, the study aims to establish a comprehensive evaluation system for hazardous working environments from certain aspects, such as inherent occupational hazards, offset factors of occupational hazards and workers’ health conditions. In addition, we collected data on advanced experience from domestic and foreign assessment involving workplace occupational hazards [17,18,19,20,21]. Ultimately, the study intends to focus on formulating a comprehensive, dynamic, objective, effective as well as scientific system to mirror the status on the prevention and control of occupational hazards in the iron and steel industry based on the modified Delphi technique.

2. Materials and Methods

2.1. Selection of Consultation Experts

The selection of consultation experts plays a pivotal role when using the Delphi method to construct the index system [22], hence, we took the occurrence of unresponsiveness and withdraw into account, as well multi-disciplinary specialty across related disciplines. We invited 40 experts who are professionals engaged in the field from colleges and universities, heads, and managers of occupational health hazards from iron and steel enterprises, scholars from centers for disease control and prevention, and researchers from occupational hazard detection companies, to carry out consultation.
The study gains from the insights of 40 experts who are teachers in colleges and universities who have been engaged in relevant research for more than five years and directors and managers of occupational health in iron and steel enterprises, occupational disease prevention and control institutes, disease prevention and control centers, occupational hazard testing companies, and other relevant units. The selected number of experts is 40 to meet the minimum standard deviation under normal distribution and random sampling.
When using the Delphi method to build an index system, the selection of consulting experts plays a crucial role [22]. Therefore, we consider the occurrence of slow response and withdrawal, as well as interdisciplinary disciplines. We invited 40 university professional and technical personnel who have been engaged in relevant research for more than 5 years, the person in charge of occupational health hazards of iron and steel enterprises, managers, scholars of the Center for Disease Prevention and Control, and researchers of the company for occupational disease hazard detection for consultation. The number of experts selected is 40 to meet the minimum standard deviation under normal distribution and random sampling.

2.2. Selected Principles of the System

The selected principles of the comprehensive evaluation system mainly include a system with a reliable source, a system with concise operation, an all-round reflection on the various aspects of subjects, a combination of subjective and objective indicators, an integration of static and dynamic indicators, mutual independence among indicators, and a system for evaluation.
See Figure 1 for the evaluation system flow.
Figure 1 shows the whole process of the fuzzy evaluation system, and concludes that the overall situation of occupational hazard prevention and control in iron and steel enterprises needs empirical stage: the overall situation of prevention and control needs multiple levels; each level contains multiple indicators, and the weight is not easy to determine; the indicators in each level are fuzzy.

2.3. Selected Criterion of the System

Tutors and experts who invariably research on occupational and environmental health were consulted on the basis of the relevant literature, steel-related and other industry-related documents as well as national laws, regulations, and outline. In-depth face-to-face personal interviews were conducted on site, where data were generated through the following criteria: importance, operability, authenticity, and sensitivity, including the mean as well as the coefficient of variation (CV) of at least two cited indices at >7 and ≤0.25 were required. Moreover, we finally proceeded to make a primary option for the items in the system corresponding to selected principles and criteria.

2.4. Delphi Survey

Delphi expert consultation was utilized to collect and develop items regarding the proposed comprehensive evaluation system using the following iterative inquiry [23,24]: consultation → feedback → disposal of data → re-consultation → re-feedback → re-disposal of data. Finally, the process was repeated until the standpoints of each expert are basically in accordance with the reliability of the proposed scheme or conclusion is relatively satisfied [25].
The study was conducted for a total of three rounds of email-based Delphi survey, each of which continued for 1.5-month for total of 4.5-month (15 March 2016 to 30 July 2016). The first-round equivalent of the pilot survey of the Delphi process was used to assess the framework of system. Data such as the aim and meaning of the study, personal baseline information of experts, and a preliminary system framework, which was structured through the literature review as well as face-to-face interviews with experts. After collating and summarizing the results in round 1, the second-round items of the Delphi technique were filtered by discarding and adding items according to the proposals of the expert panel. The second-round Delphi survey primarily aims to verify and improve the system. On the basis of considering the scoring situations and opinions of the invited experts, we formulated the third-round items of the Delphi method, which was principally utilized to determine the weighting coefficients of the system during this process. Out of the three-round Delphi consultation, the invited scholars and specialists were required to rate the significance and operability of index system in round 1. A 10-point Likert-type scale was used ranging from 1 (non-significant, non-operable) to 10 (very significant, very operable) [26]. In rounds 2 and 3, the experts were expected to rate the significance, operability, sensitivity, as well as authenticity of the index system using the 10-point Likert-type scale and analytical hierarchy process (AHP) ranging from 1 (non-significant, non-operable, non-sensitive, and non-valid) to 10 (very significant, very operable, very sensitive, and very valid) [27]. Afterward, the positivity of the panelists was appraised by analyzing the recovery and response rates of the questionnaires [28]. Furthermore, the coefficients of expert authority were assessed based on the degree of familiarity as well as the quantification scores of judgment basis and influence degree for the system. The consensus coefficient of expert opinions, which is in general defined as coordination coefficient W, indicated the degree of consistency among the experts and consistency of scoring results during the Delphi course [29,30].

2.5. Determining the Weighting Coefficients for the Designed System

The weight of the comprehensive evaluation system was calculated using the AHP-fuzzy comprehensive evaluation model, which proceeds as follows [27,31,32]. Using the improved AHP method, we determine the weighting coefficients of the primary index. Then, the weight of the primary index is taken as the membership degree of the fuzzy comprehensive evaluation model. Finally, the weighting coefficients of the index is determined on the basis of fuzzy modeling.

2.6. Statistical Methods

The database was established using Excel 2007 software, and the reliability and validity of the index system were quantitatively evaluated using the following methods: recovery rate, response rate, Cronbach’s alpha, and factor analysis. All analyses were performed using SAS (Version 9.3; SAS Institute, Cary, NC, USA).

2.7. Ethical Approval

This method has been approved by the ethics committee of North China University of Science and Technology.

3. Results

3.1. Baseline Information of Consultants

At the inceptive stage, a total of 40 consultation experts were invited to participate in the study. However, only 30 questionnaires were considered valid in the third round. Table 1 shows that 30 unique panel members overlapped the course of the three-round Delphi process with 83.3% of the participants aged over 40 years old.
All experts had over 5-year length of service, out of which 66.7% experts had worked for more than 20 years. All panel members were trained with an educational background of bachelor’s degree or above, where 66.7% of the subjects obtained a graduate’s degree. All panelists had been awarded an intermediate or higher professional title, and experts who persistently concentrated their study in the field of occupational health-related work comprised 93.3% of the sample.

3.2. Panel Experts’ Responses to Questionnaires

A total of three rounds of the Delphi survey were conducted. A total of 40 questionnaires were distributed in round 1, out of which 33 questionnaires obtained feedback from panel members and a recovery rate of 82.5%. In the final round, only 30 questionnaires were deemed valid with a response rate of 75.5%. This rate met the requirements among respondents when three questionnaires were excluded due to incomplete or missing data. In the final two rounds, the recovery and response rates of the questionnaires reached 100%.

3.3. Authority Degree for Expert Panel

Table 2 shows that the expert familiarity and authority coefficients of the four main indicators are greater than 0.65 and 0.75, respectively. In general, if the authority coefficient of each index participant is greater than 0.7 [20], then the project of the index system can be regarded to be with high credibility. That is, the higher the expert authority score, the higher the project consensus proposed. Apparently, based on this argument, the projects collected in the research can be considered relatively credible with high consistency.
The expert’s familiarity with the comprehensive evaluation index system and basis for judgments are two factors that determine an expert’s authority. Familiarity with the indicator system is divided into six levels, namely, very unfamiliar, less familiar, general, more familiar, familiar, and very familiar. The basis for expert judgment of the indicator system is evaluated from four aspects, namely, theoretical analysis, practical experience, understanding, and intuition of local and abroad peers. The sums of the judgment coefficient are equal to 0.6 and 0.8, which indicate that the judgment basis has little influence on the judgment of experts and that the judgment basis has medium influence on the judgment of experts, respectively. If the sum of the discrimination coefficients is equal to 1, it means that the judgment basis has a great influence on the expert judgment.

3.4. Consensus in Opinions from Consultant Experts

Table 3 presents the consistency coefficients and test results of the selected items during the three rounds of the Delphi process. The table indicates that the consistency coefficients of the other indicators were statistically significant except for operability of indicators and third-party regulatory situation. In the first round of consultation, results were small with differences being statistically non-significant. Furthermore, they established a consensus on the index system to varying degrees among experts by enhancing the number of consultations, which suggests that coincident with the continuous improvement of the system and according to the remarks of the panel members, a relative coordination was observed regarding the views shared in the responses, which is necessary for the understanding of the results with a relatively high degree of credibility.

3.5. Retaining, Revising, or Discarding of the System During the Delphi Enquiry Process

After completing round 1, most experts decided that the items under third-party supervision and occupational health organizations and regulations should be incorporated into occupational health management as primary indicators. In total, two out of 19 secondary indicators and nine out of 95 tertiary indicators were discarded with mean scores between 6 and 7. Furthermore, three out of 19 secondary indicators and 27 out of 95 tertiary indicators were revised. Meanwhile, other participants proposed that three new secondary items and 10 novel tertiary items should be introduced to the system. In round 2, none of items were revised or deleted, whereas one new proposed item was added to the secondary indicators (data not shown). In round 3, the importance, operability, authenticity, and sensitivity of all items reached mean scores above 7 with a CV of <0.25, which indicates agreement on all items among experts and the successful formulation of the system.

3.6. Quantitative Determination for the Weighting Coefficients of the System

The single weighting coefficients of the comprehensive evaluation system were calculated using the formula: P i = Y i / i = 1 j Y i , Y i = K W i R i j V . In this study, Y i = K W i R i j V = 1 [ 0.3997 0.0359 0.3997 0.1647 ] R i j [ 9.5 7.5 5.5 3.5 1.5 ] .
Table 4 shows that we can obtain the single and hybrid weighting coefficients of the system, where the argument of Rij was denoted as a judgment matrix that corresponds to the percentage of experts who ranked a certain rate for a given number of indicators. In addition, the hybrid weighting coefficients were finally calculated using the product method based on a single weighting coefficient [21].

3.7. System Reliability Evaluation

For the comprehensive evaluation system, the next step of analysis is calculating the reliability coefficient (also known as structural validity), namely, Cronbach’s α and split-half reliability, which are used to measure the internal consistency of the system. The result of this coefficient is in general higher than 0.7 [17,33]. Thus, the scale is considered to have high internal consistency.
Structural validity aims to examine the relationship between the test scores and indicators. The selection of indicator data and test scores are collected simultaneously, which is the external standard for measure test effectiveness, usually the behavior we want to predict.
The expected use and advantage of the evaluation tool is that it can get a more scientific and trustworthy quantitative result by processing the fuzzy and difficult to quantify indicators through the complex digital operation of fuzzy evaluation; it can combine the qualitative and quantitative indicators organically, and the result is in line with the actual situation; it can get a vector result by processing the fuzzy comprehensive evaluation model of fuzzy mathematics, and Not a point value. In this way, it will contain rich information, which can not only accurately describe the evaluated object, but also further process and get reference information.
Table 5 provides the reliability test results of the system, which suggests that although the value of Cronbach’s α coefficient standardized by the item under personal protection in the secondary index was 0.695, Cronbach’s α coefficient and split-half reliability reached more than 0.7, which indicates that the item was considered to have an acceptable level of internal consistency. In addition, the three values of the indicators that reflected the reliability of the remainder of the items and comprehensive evaluation system were all above 0.7, which further demonstrated that these consistent items correctly reflect the field that the system aims to measure.

3.8. Validity of Comprehensive Evaluation System

For the questionnaire survey, validity is typically more significant than reliability [21]. In this study, validity, specifically, the effectiveness, accuracy, and correctness of the evaluation system, refers to the extent that the designed system can reflect the objective authenticity of occupational hazard prevention and control in the iron and steel enterprises.
Finally, the study aims to examine the structural validity of the 20 secondary indicators through factor analysis, which can reflect the structural validity of the evaluation system to a certain extent. Table 6 displays the elective results of factor analysis using principal component extraction for the original data. In general, factor interpretability is readily achieved with factor rotation [34]. The study was performed with equamax rotation to yield rotated factors. Observing the initial eigenvalues manifested that the common factors of the rotated and unrotated factor loadings in the respective setting were four and six, respectively, which were linked to the set of items. The eigenvalues for the first four factors were greater than 1, which accounts for 63.1% of the variance in the twenty items.
Table 7 depicts the loadings of the component matrix rotated with equamax, of which loadings greater than 0.400 are specified as strongly correlated among twenty items [31]. Thus, the study infers that the indexes of D1–D11 load highly on factor one, which indicates the factor of occupational health management. Indicators D1 and D7–D9 load highly on factor two, which are representative of the factor third-party supervision.
Indices A1–A3 load highly only on factor three and is indicative of the factor harmful factors in occupational activities. Indexes B1–B4 load highly only on factor four, which represents the factor workers’ health conditions. Factor five is composed of indicators D5 and D10–D11, which indicates the factor of occupational health organizations and regulations. Indices C1–C2 consisted mostly of factor six, which pertains to the factor protection facilities against occupational hazards. Based on the aforementioned analysis, cross-loading associations were observed among the factors. However, the study found that the loadings of items D5 and D11 in factor one was greater than that of factor five, and items D7–D9 in factor one was greater than that of factor two. Although the loadings of item D10 in factor five was greater than that of factor one, both reached more than 0.7 with close proximity to each other. Therefore, factors two and five can be combined as factor one, and the discriminant validity of the four primary indexes remains relatively high after merging. This result is consistent with the comments of consultants, who stated theoretical support that the reliability of the system was fairly high in the current study.

4. Discussion

Currently, the Delphi technique is widely applied for policy- and decision-making to reach an agreement on significant questions or opinions. The advantage of this technique is that experts without psychological pressure are not influenced by the outside environment when making judgments based on academic experience and theoretical knowledge [14,32,33]. As such, consultants can maximize their creativity to guarantee that superior viewpoints are received by putting ideas together [14]. By doing so, on the basis of the three-round Delphi survey procedure and to the best of our knowledge, the current work is the first to comprehensively build an evaluation system of occupational hazard prevention and control for the iron and steel enterprises. The study relied on literature review, synthetical analysis, field epidemiological investigation, and face-to-face interviews. During the formation of the comprehensive evaluation system, the system not only embodied the integrity, hierarchy, and rationality of occupational hazard prevention and control in the iron and steel industry, but also reflected the concept and thought of system theory, that is, the formation of the four primary indicators and development of twenty secondary indicators according to the four primary indicators to which they belonged. Finally, each secondary index was divided into a certain number of tertiary indicators of 95. The indices used to appraise the reliability of the Delphi survey mainly include numbers of consultations, numbers of panelists, representative of panelists, enthusiasm of panelists, authority as well as the consistent opinions of panelists [14,18,21]. In the present work, the respective description was presented as follows. ① For the numbers of experts: previous studies provided recommendations of the appropriate number of participants, which ranged from 15 to 50 after eliminating the underlying dropouts [14,21]. An excessive number of participants in the Delphi survey can potentially increase the burden as well as result in difficulties in terms of quality control during the consultation period. Conversely, if the participants are scarce, the formulation of the system will become unstable [34]. Thus, this study aimed to select 40 eligible scholars and specialists based on mathematical statistics theory, document literature, and eliminating possible dropouts during the survey period as respondents in the first round. A total of 33 researchers returned the questionnaires through email, where three failed to provide completed questionnaires due to busy schedules. Thus, we considered these participants irretrievable. As such, 30 subjects remained for the remainder of the survey. ② Representative of panelists: Participants with master’s degree or above selected by the study accounted for 66.7% of the sample. 93.3% of the participants had reached at least 10 years of tenure in occupational health-related professions. All of them had intermediate and above job titles. Out of the 30 subjects, 80% possessed higher job titles and all of them worked in the field of occupational health, public health, health education, epidemiology and biostatistics, health management, and health economics. In total, 93.3% of the participants are engaged in occupational health research. Furthermore, previously cited results implied that the selected experts are essential elements of good representation in terms of sound professional quality. ③ Enthusiasm of panel members: Given that the recovery and response rate of the questionnaires are equal or greater than 70%, the panelists are regarded to be with high enthusiasm and the questionnaires are of high quality [26]. The recovery and response rates of the consultation questionnaires in the first round of the Delphi process were 82.5% and 75.0%, respectively. A total of 56.7% of the respondents provided remarks for the modification of certain parts of the indicators, which indicated that the quality of the response questionnaires was considerably high, and the index system in the first round was dramatically immature. Nevertheless, the recovery and response rates of the consultation questionnaires for rounds 2 and 3 are 100.0%, where 6.7% of the consultants proposed additional comments on the index system in the second round, and none of them gave further advice in the third round. This tendency exhibits that scholars and specialists eventually acquired consensus on the items in round 2. ④ Authority of experts: A linear relationship was observed between the authority of experts and precision of the consultation results in a preceding report [34]. In the current work, the authority of the remaining three primary indicators were equal or greater than 0.8 in addition to workers’ health conditions with a value of experts’ authority calculated as 0.769. Result reveals that the experts were substantially adept in their respective research fields. ⑤ Opinion consistency of consultation experts: With the increasing number of the Delphi process, the study finds coordination among the experts’ viewpoints regarding the indicators, which were scored from four aspects, namely, importance, operability, authenticity, and sensitivity and more or less enhanced. The W value for indicators had a tendency to lean toward 0.5 with a significantly statistical difference in round 3. Thus, we concluded that the experts’ results in scoring the index system ultimately achieved consistency. Moreover, several large-scaled Delphi surveys implemented in the health domain concluded that the consistency coefficient W in the last round was generally prone to fluctuate by approximately 0.5, which indicated that the results of the present study were in accordance with domestic and foreign research [34,35].
In terms of confirming the weighting coefficients for the integrated assessment system, a modified AHP-fuzzy comprehensive evaluation methodology, which is means of empowerment along with subjective and objective combination, was adopted in the current study by referring to previous related studies [36,37,38,39]. The results were relatively congruent with theoretical and practical conditions. We find that an improved hybrid approach was ultimately utilized to adequately avert the disadvantage of the single when calculating the weighting coefficients of the system. The abovementioned result also suggested that the improved AHP-fuzzy combined method was well received as a novel model that provided investigators with not only a bulk of information regarding feasibility, rationality, and accuracy in counting the weights under study, but also a type of methodological strategy for subsequent studies.
In summary, the novel and innovatively constructed system of the study was implicated to be good feasibility, reasonability, and scientificity and can be used to comprehensively assess the preventive and control status of occupational hazards in the iron and steel industry. Moreover, it can easily identify and detect unsubstantial links responsible for the prevention and control of hazardous occupations in the iron and steel industry during the evaluation process. Finally, niche-targeting strategies for addressing this issue can be required to initiate. Notably, however, further empirical research using the comprehensive evaluation system for occupational hazards in the iron and steel industry is required to appraise the reliability and validity of the system and ultimately, improve it.

5. Conclusions

Based on Delphi expert consultations, the method for determining the weight of the comprehensive evaluation index system using the improved AHP-fuzzy comprehensive evaluation model is scientific and reliable. The comprehensive evaluation index system for the prevention and control of occupational hazards in the iron and steel industry is relatively inclusive, reasonable, and has high reliability and validity. In addition, the BPNN neural network model can overcome the difficulties and defects of the fuzzy comprehensive evaluation model in the empirical research stage and has a strong application prospect for comprehensive evaluation.

Author Contributions

Formal analysis, S.Z.; Funding acquisition, J.Y.; Investigation, Z.C., J.W. and C.L.; Project administration, X.Z.; Supervision, X.L.; Writing—original draft, Y.S. All authors have read and agreed to the published version of the manuscript.


This research was funded by the National Key R&D Program of China (No.2016YFC0900605).

Conflicts of Interest

The authors declare no conflict of interest.


  1. Xuan, Y.; Yue, Q. Scenario analysis on resource and environmental benefits of imported steel scrap for China’s steel industry. Resour. Conserv. Recycl. 2017, 120, 186–198. [Google Scholar] [CrossRef]
  2. Invest in China. China Crude Steel Output Rises Slightly in 2016; Investment Promotion Agency of Ministry of Commerce: Beijing, China, 2017; Available online: (accessed on 8 February 2017).
  3. Sheng, Y.; Song, L. The technical efficiency of China’s large and medium iron and steel enterprises: A firm-level analysis. In The Chinese Steel Industry’s Transformation; Edward Elgar Publishing: Troutham, UK, 2012. [Google Scholar]
  4. Chang, E.-E.; Wei-Chi, W.; Li-Xuan, Z.; Hung-Lung, C. Health risk assessment of exposure to selected volatile organic compounds emitted from an integrated iron and steel plant. Inhal. Toxicol. 2010, 22, 117–125. [Google Scholar] [CrossRef] [PubMed]
  5. Tsai, J.-H.; Lin, K.-H.; Chen, C.-Y.; Lai, N.; Ma, S.-Y.; Chiang, H.-L. Volatile organic compound constituents from an integrated iron and steel facility. J. Hazard. Mater. 2008, 157, 569–578. [Google Scholar] [CrossRef] [PubMed]
  6. Yu, B.; Li, X.; Qiao, Y.; Shi, L. Low-carbon transition of iron and steel industry in China: Carbon intensity, economic growth and policy intervention. J. Environ. Sci. 2015, 28, 137–147. [Google Scholar] [CrossRef] [PubMed]
  7. Tong, J.; Wang, Y.; Yuan, J.; Yang, J.; Wang, Z.; Zheng, Y.; Chai, F.; Li, X. Effect of Interaction Between Noise and A1166C Site of AT1R Gene Polymorphism on Essential Hypertension in an Iron and Steel Enterprise Workers. J. Occup. Environ. Med. 2017, 59, 412–416. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  8. Zeng, L.; Chai, D.-L.; Li, H.-J.; Lei, Z.; Zhao, Y.-M. Assessment of personal noise exposure of overhead-traveling crane drivers in steel-rolling mills. Chin. Med. J. 2007, 120, 684–689. [Google Scholar] [CrossRef]
  9. Xu, Z.; Brown, L.M.; Pan, G.W.; Liu, T.F.; Gao, G.-S.; Ston, B.J.; Cao, R.M.; Guan, D.-X.; Shen, J.H.; Yan, Z.S.; et al. Cancer risks among iron and steel workers in Anshan, China, Part II: Case-control studies of lung and stomach cancer. Am. J. Ind. Med. 1996, 30, 7–15. [Google Scholar] [CrossRef]
  10. Xu, Z.; Pan, G.-W.; Liu, L.-M.; Brown, L.M.; Guan, D.-X.; Xiu, Q.; Sheng, J.-H.; Stone, B.J.; Dosemeci, M.; Fraumeni, J.F.; et al. Cancer risks among iron and steel workers in Anshan, China, part I: Proportional mortality ratio analysis. Am. J. Ind. Med. 1996, 30, 1–6. [Google Scholar] [CrossRef]
  11. Chen, G.Y.; Cao, H.X.; Li, F.; Cai, X.B.; Ao, Q.H.; Gao, Y.; Fan, J.G. New risk-scoring system including non-alcoholic fatty liver disease for predicting incident type 2 diabetes in East China: Shanghai Baosteel Cohort. J. Diabetes Investig. 2016, 7, 206–211. [Google Scholar] [CrossRef]
  12. Zhao, X.L. Research on Independent Innovation of State-Owned Enterprises from the Perspective of National Innovation System. Ph.D. Thesis, Shandong University, Jinan, China, 2013. (In Chinese). [Google Scholar]
  13. Wang, X.; Lin, B. Factor and fuel substitution in China’s iron & steel industry: Evidence and policy implications. J. Clean. Prod. 2017, 141, 751–759. [Google Scholar]
  14. Li, J. Study on Prevention and Control of Occupational Diseases in Anshan City, Liaoning Province. Master’s Thesis, Shenyang Normal University, Shenyang, China, 2019. [Google Scholar]
  15. Xing, S.L. Assessment of Occupational Hazards in Support Base of CNPC. Master’s Thesis, North China University of Science and Technology, Qinhuangdao, China, 2013. [Google Scholar]
  16. Zhang, L. Research on the Performance Evaluation and Analysis of Occupational Disease Prevention and Control. Ph.D. Thesis, Shandong University, Jinan, China, 2014. [Google Scholar]
  17. Chang, Y.-K.; Chuang, K.-Y.; Tseng, J.-M.; Lin, F.-C.; Su, T.-S. Hazardous workplace review program in Taiwan. Ind. Heal. 2013, 51, 432–442. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  18. Conway, S.H.; Pompeii, L.A.; Casanova, V.; Douphrate, D.I. A qualitative assessment of safe work practices in logging in the southern United States. Am. J. Ind. Med. 2017, 60, 58–68. [Google Scholar] [CrossRef] [PubMed]
  19. Leiter, M.P.; Robichaud, L. Relationships of occupational hazards with burnout: An assessment of measures and models. J. Occup. Heal. Psychol. 1997, 2, 35–44. [Google Scholar] [CrossRef]
  20. Ezejiofor, T.I.N.; Ezejiofor, A.N.; Orisakwe, O.E.; Nwigwe, H.C.; Osuala, F.O.; Iwuala, M.O. Anicteric hepatoxicity: A potential health risk of occupational exposures in Nigerian petroleum oil refining and distribution industry. J. Occup. Med. Toxicol. 2014, 9, 3. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  21. Sun, Z.Q. Medical Statistics, 3rd ed.; People’s Medical Publishing House: Beijing, China, 2009. (In Chinese) [Google Scholar]
  22. Brown, B.B. Delphi Process: A Methodology Used for the Elicitation of Opinions of Experts; RAND Corp.: Santa Monica, CA, USA, 1968. [Google Scholar]
  23. Hasson, F.; Keeney, S.; McKenna, H. Research guidelines for the Delphi survey technique. J. Adv. Nurs. 2000, 32, 1008–1015. [Google Scholar]
  24. Chang, L.-C.; Chen, Y.-C.; Wu, F.L.; Liao, L.-L. Exploring health literacy competencies towards patient education programme for Chinese-speaking healthcare professionals: A Delphi study. BMJ Open 2017, 7. [Google Scholar] [CrossRef] [Green Version]
  25. Wojcieszek, A.M.; Reinebrant, H.E.; Leisher, S.H.; Allanson, E.; Coory, M.; Erwich, J.J.; Frøen, J.F.; Gardosi, J.; Gordijn, S.; Gülmezoglu, M.; et al. Characteristics of a global classification system for perinatal deaths: A Delphi consensus study. BMC Pregnancy Childbirth 2016, 16, 223. [Google Scholar] [CrossRef] [Green Version]
  26. Blaschke, S.; O’Callaghan, C.C.; Schofield, P. Identifying opportunities for nature engagement in cancer care practice and design: Protocol for four-round modified electronic Delphi. BMJ Open 2017, 7. [Google Scholar] [CrossRef] [Green Version]
  27. Topuz, E.; Van Gestel, C.A. An approach for environmental risk assessment of engineered nanomaterials using Analytical Hierarchy Process (AHP) and fuzzy inference rules. Environ. Int. 2016, 92, 334–347. [Google Scholar] [CrossRef]
  28. Diamond, I.R.; Grant, R.C.; Feldman, B.M.; Pencharz, P.B.; Ling, S.C.; Moore, A.M.; Wales, P.W. Defining consensus: A systematic review recommends methodologic criteria for reporting of Delphi studies. J. Clin. Epidemiol. 2014, 67, 401–409. [Google Scholar] [CrossRef]
  29. Qiu, Y.; Shen, J.; Lu, H. The Development of Criteria for the Selection of Chinese Peer Educators in HIV Management: A Delphi Study. Scientifica 2016, 2016, 1–4. [Google Scholar] [CrossRef] [PubMed]
  30. Rankin, G.; Rushton, A.; Olver, P.; Moore, A. Chartered Society of Physiotherapy’s identification of national research priorities for physiotherapy using a modified Delphi technique. Physiotherapy 2012, 98, 260–272. [Google Scholar] [CrossRef] [PubMed]
  31. Li, M.; Du, Y.; Wang, Q.; Sun, C.; Ling, X.; Yu, B.; Tu, J.; Xiong, Y. Risk assessment of supply chain for pharmaceutical excipients with AHP-fuzzy comprehensive evaluation. Drug Dev. Ind. Pharm. 2016, 42, 676–684. [Google Scholar] [CrossRef] [PubMed]
  32. Arrow, K.J. Social Choice and Individual Values, 2nd ed.; Yale University Press: New York, NY, USA, 1963. [Google Scholar]
  33. Chongsuvivatwong, V. Analysis of Epidemiological Data Using R and Epicalc; Book Unit, Faculty of Medicine, Prince of Songkla University: Songkla, Thailand, 2008. [Google Scholar]
  34. Gonzalez, R. Applied Multivariate Statistics for the Social Sciences, 6th ed.; Routledge: New York, NY, USA, 2009. [Google Scholar]
  35. Nieuwenhuijsen, K.; Verbeek, J.H.A.M.; De Boer, A.G.E.M.; Blonk, R.W.B.; Van Dijk, F.J.H. Validation of performance indicators for rehabilitation of workers with mental health problems. Med Care 2005, 43, 1034–1042. [Google Scholar] [CrossRef] [PubMed]
  36. Chang, Y.-K.; Su, T.-S.; Ouyang, Y.; Tseng, J.-M. Employee impact and attitude analysis for GHS implementation in Taiwan. Ind. Health 2013, 51, 353–363. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  37. Parente, F.J.; Anderson-parente, J. Delphi inquiry systems. In Judgmental Forecasting; Wright, G., Ayton, P., Eds.; Wiley: Chichester, UK, 1987. [Google Scholar]
  38. O’Neill, N.; Dogar, O.; Jawad, M.; Keller, I.; Kanaan, M.; Siddiqi, K. Which Behavior Change Techniques May Help Waterpipe Smokers to Quit? An Expert Consensus Using a Modified Delphi Technique. Nicotine Tob. Res. 2016. [Google Scholar] [CrossRef] [Green Version]
  39. Wu, J.-H.; Wei, W.; Zhang, L.; Wang, J.; Damasevicius, R.; Li, J.; Wang, H.-D.; Wang, G.-L.; Zhang, X.; Yuan, J.-X.; et al. Risk Assessment of Hypertension in Steel Workers Based on LVQ and Fisher-SVM Deep Excavation. IEEE Access 2019, 7, 23109–23119. [Google Scholar] [CrossRef]
Figure 1. Flow chart of evaluation system.
Figure 1. Flow chart of evaluation system.
Ijerph 17 00667 g001
Table 1. Experts’ demographics.
Table 1. Experts’ demographics.
VariablesCategoriesN (%)
<300 (0.0)
30–395 (16.7)
40–4914 (46.7)
50–5910 (33.3)
≥601 (3.3)
Years of working
5–92 (6.7)
10–198 (26.7)
≥2020 (66.7)
Educational level
College degree or lower0 (0.0)
Bachelor’s degree10 (33.3)
Master’s degree9 (30.0)
Doctorate11 (36.7)
Job title
Intermediate3 (10.0)
Vice-senior3 (10.0)
Senior24 (80.0)
Professional field *
Health management2 (6.7)
Health economics1 (3.3)
Occupational and environmental health28 (93.3)
Epidemiology and biostatistics5 (16.7)
Public health6 (20.0)
Health education3 (10.0)
Clinical medicine1 (3.3)
Maternal and child health care1 (3.3)
Main service *
Administrative management9 (30.0)
Teaching and research22 (73.3)
Clinical and health services2 (6.7)
Notes: * This option can pertain to multiple choices, in which the proportion is defined as number of N (%).
Table 2. Scores of degree of authority of expert consultation.
Table 2. Scores of degree of authority of expert consultation.
ItemsCoefficients of Experts’ FamiliarityJudgment CoefficientsCoefficients of Authority Degree
A Harmful factors in occupational activities0.8200.9300.875
B Workers’ health conditions0.6670.8700.769
C Protection facilities against occupational hazards0.7470.9170.832
D Occupational health management0.7270.8730.800
Table 3. Results of consistency and test for expert consultation.
Table 3. Results of consistency and test for expert consultation.
W χ W 2 dfPW χ W 2 dfP
FirstA Harmful factors in occupational activities0.446280.84921<0.050.15899.42421<0.05
B Workers’ health conditions0.23598.50014<0.050.11347.51314<0.05
C Protection facilities against occupational hazards0.258116.16215<0.050.13159.03015<0.05
D Occupational health management0.214180.30728<0.050.182153.07028<0.05
E Third party supervision0.10538.55312<0.050.03412.18712>0.05
F Occupational health organizations and regulations0.06837.23118<0.050.06032.39618<0.05
SecondA Harmful factors in occupational activities0.429231.74018<0.050.343185.27518<0.05
B Workers’ health conditions0.239100.58514<0.050.18778.47514<0.05
C Protection facilities against occupational hazards0.409159.99613<0.050.346134.81713<0.05
D Occupational health management 0.382744.95365<0.050.297579.83765<0.05
ThirdA Harmful factors in occupational activities0.526284.08318<0.050.465251.30118<0.05
B Workers’ health conditions0.308129.10314<0.050.22795.44714<0.05
C Protection facilities against occupational hazards0.483188.52413<0.050.445173.72913<0.05
D Occupational health management0.448887.04566<0.050.395770.59866<0.05
W χ W 2 dfPW χ W 2 dfP
SecondA Harmful factors in occupational activities0.435234.73618<0.050.444239.99318<0.05
B Workers’ health conditions0.23297.47414<0.050.20184.45014<0.05
C Protection facilities against occupational hazards0.453190.71513<0.050.348135.84113<0.05
D Occupational health management0.361703.97465<0.050.319622.14465<0.05
ThirdA Harmful factors in occupational activities0.456246.03918<0.050.414223.44318<0.05
B Workers’ health conditions0.246103.47414<0.050.273114.73314<0.05
C Protection facilities against occupational hazards0.466182.22413<0.050.389151.17813<0.05
D Occupational health management0.374736.29066<0.050.324641.33366<0.05
Notes: The first round of the Delphi process failed to consult the authenticity and sensitivity of indicators.
Table 4. Weighting coefficients of comprehensive evaluation systems.
Table 4. Weighting coefficients of comprehensive evaluation systems.
Primary IndicatorWeighting CoefficientSecondary IndicatorWeighting CoefficientTertiary IndicatorWeighting CoefficientHybrid Weighting Coefficient
A Harmful factors in occupational activities0.2516A1 Harmful factors during the production process0.3399A1.1 Qualified rate of dust monitoring points0.16850.0144
A1.2 Qualified rate of hazardous gas monitoring points0.16780.0143
A1.3 Qualified rate of high temperature monitoring points0.17570.0150
A1.4 Qualified rate of noise monitoring points 0.17630.0151
A1.5 Qualified rate of thermal radiation monitoring points0.16310.0140
A1.6 Qualified rate of wind speed monitoring points0.14860.0127
A2 Harmful factors during the labor process0.3275A2.1 Labor organization and system0.21220.0175
A2.2 Occupational psychological stress0.17960.0148
A2.3 Labor intensity0.22330.0184
A2.4 Incorrect posture0.18810.0155
A2.5 Tool usage0.19670.0162
A3 Harmful factors in the working environment0.3326A3.1 Natural environment0.18190.0152
A3.2 Equipment layout0.19470.0163
A3.3 Daylighting and artificial lighting0.19520.0163
A3.4 Ventilation conditions0.22800.0191
A3.5 Air conditioning0.20030.0168
B Workers’ health conditions0.2428B1 Workers’ basic situation0.2485B1.1 Living habits0.31040.0187
B1.2 Health habits0.32120.0194
B1.3 Average exposure length of service0.36840.0222
B2 Workers’ exposure characteristics0.2596B2.1 Number of exposures 0.35220.0222
B2 B2.2 Level of exposure0.32330.0204
B2.3 Exposure time and frequency0.32450.0205
B3 Workers’ quality0.2604B3.1 Educational level0.49400.0312
B3.2 Professional skill0.50600.032
B4 Workers’ knowledge of, attitude toward, and practice in terms of occupational hazards0.2315B4.1 Awareness rate0.36580.0206
B4.2 Belief formation rate0.28120.0158
B4.3 Behavior formation rate0.35300.0198
C Protection facilities against occupational hazards0.2550C1 Personal
0.4969C1.1 Usage rate of personal protective equipment0.16930.0214
C1.2 Protective properties and performance of protective equipment0.17000.0215
C1.3 Routine maintenance of protective equipment0.16850.0213
C1.4 Maintenance replacement cycle0.16580.021
C1.5 Protective equipment release records0.15710.0199
C1.6 Appropriate types of protective equipment release0.16930.0214
C2 Engineering protection0.5031C2.1 Setting rate of detoxification purification facilities 0.17520.0225
C2.2 Setting rate of noise and vibration control facility0.17210.0221
C2.3 Setting rate of non-ionizing radiation protection facility 0.16550.0212
C2.4 Setting rate of heatstroke prevention, cold-proof, and moisture-proof0.16260.0209
C2.5 Setting rate of high temperature and thermal radiation protection facilities 0.17020.0218
C2.6 Setting rate of other harmful factors in protection facility 0.15440.0198
D Occupational health management0.2506D1 Occupational health surveillance0.0919D1.1 Development of occupational health surveillance program0.19440.0045
D1.2 Completion rate of occupational health surveillance file 0.19840.0046
D1.3 Examination rate of occupational health0.21670.005
D1.4 Physical examination of pre-service, on-the-job training, and leave 0.21260.0049
D1.5 Post-care medical follow-up examination0.17790.0041
D2 Occupational hazard monitoring0.0915D2.1 Coverage rate of occupational hazard monitoring points0.16920.0039
D2.2 Detection rate of occupational hazard factors0.17770.0041
D2.3 Monitoring rate of personal dose0.16790.0039
D2.4 Filing and reporting of monitoring results0.17160.0039
D2.5 Improvement rate of supervision and inspection institutions0.15650.0036
D2.6 Intact rate of monitoring equipment0.15720.0036
D3 Materials and equipment management0.0856D3.1 Adoption of new technologies, techniques, and new materials0.19560.0042
D3.2 Adoption of equipment and materials for the production of occupational hazards0.19890.0043
D3.3 Leading suppliers of raw materials for enterprises0.19320.0041
D3.4 Chinese instructions of harmful equipment, hazardous chemicals, and radioactive materials0.20520.0044
D3.5 Public hazards of new technologies, processes, and new materials0.20710.0044
D4 Auxiliary health facilities0.0822D4.1 Setting rate of production health room0.34330.0071
D4.2 Setting rate of living room0.33390.0069
D4.3 Setting rate of women’s health room0.32280.0066
D5 Emergency rescue facilities and warning sign configuration0.0910D5.1 Establishment of emergency response agencies0.14100.0032
D5.2 Contingency plans0.14190.0032
D5.3 Relief supplies0.13890.0032
D5.4 Emergency drills0.13950.0032
D5.5 Setting rate of alarm device0.14840.0034
D5.6 Setting rate of warning labels0.1460.0033
D5.7 Maintenance of emergency rescue facilities0.14440.0033
D6 Supervision of regulatory agencies0.0985D6.1 Rectification supervision0.49330.0122
D6.2 Penalties0.50670.0125
D7 Early prevention0.0911D7.1 Technical review and approval of construction projects0.25620.0058
D7.2 Declaration rate of declaration management project with occupational hazard factors0.24360.0056
D7.3 Occupational safety production license0.24420.0056
D7.4 Pre-evaluation rate of construction projects with occupational hazard factors0.25600.0058
D8 Protection and management during the operation process0.0923D8.1 Notification rate of occupational hazards0.33970.0079
D8.2 Establishment and daily management of occupational health institutions0.33610.0078
D8.3 Training rate of occupation health0.32420.0075
D9 Occupational hazard post-treatment situation0.0881D9.1 Investigating and treating rate of occupational hazard accidents0.25300.0056
D9.2 Qualified rate of physical examination0.24510.0054
D9.3 Detection rate of occupational disease0.24870.0055
D9.4 Coverage rate of work-related injury insurance0.25330.0056
D10 Occupational health organizations0.0927D10.1 Occupation health management organizations and full- or part-time management staff0.34110.0079
D10.2 Occupational health professional and technical monitoring institutions and specialized or concurrent vocational and technical personnel0.32090.0075
D10.3 Occupational health leading bodies and constituent personnel0.33800.0078
D11 Occupational health regulations0.0952D11.1 Management system of occupational hazard protection equipment and facilities0.07360.0018
D11.2 Management system of labor employment and health care files0.07220.0017
D11.3 Management system of the workplace0.07040.0017
D11.4 Management system of occupation disease diagnosis0.06870.0016
D11.5 Management system of occupational hazard daily monitoring0.07190.0017
D11.6 Evaluation system of occupational hazards0.07060.0017
D11.7 Notification system of occupational hazards0.07240.0017
D11.8 Inspection system of occupational hazards0.07100.0017
D11.9 Education and training system of occupational health0.06910.0016
D11.10 Management system of protective supplies0.07080.0017
D11.11 Management system of equipment maintenance0.07170.0017
D11.12 Declaration system of occupational disease harm0.07160.0017
D11.13 Evaluation system of construction projects with occupation disease harm0.07310.0017
D11.14 Three simultaneous management systems0.07290.0017
Table 5. Test results of reliability.
Table 5. Test results of reliability.
ItemsCronbach’s α CoefficientStandardized Cronbach’s α CoefficientSplit-Half Reliability
A Harmful factors in occupational activities0.8240.8230.782
A1 Harmful factors during the production process0.7070.7050.742
A2 Harmful factors during the labor process0.8510.8400.825
A3 Harmful factors in the working environment0.7990.7820.737
B Workers’ health conditions0.7730.7700.821
B1 Workers’ basic situation0.7930.7880.717
B2 Workers’ contact characteristics0.8580.8610.860
B3 Workers’ quality0.7310.7380.737
B4 Workers’ knowledge of, attitude toward, and practice in terms of occupational hazards0.7450.7570.725
C Protection facilities against occupational hazards0.7750.7740.775
C1 Personal protection0.7140.6950.740
C2 Engineering protection0.8710.9190.881
D Occupational health management0.7650.7830.746
D1 Occupational health surveillance0.8990.9040.874
D2 Occupational hazard monitoring0.8950.9030.827
D3 Materials and equipment management0.7790.7860.893
D4 Auxiliary health facilities0.9160.9180.961
D5 Emergency rescue facilities and warning signs configuration0.7390.7900.740
D6 Supervision of regulatory agencies0.8870.8870.887
D7 Early prevention0.8010.8070.703
D8 Protection and management during operation process0.7020.7040.782
D9 Occupational hazard post-treatment situation0.8440.8620.802
D10 Occupational health organizations0.7070.7690.769
D11 Occupational health regulations0.9460.9440.947
Comprehensive evaluation index system0.9590.9500.810
Table 6. Principal component of factor analysis.
Table 6. Principal component of factor analysis.
ComponentInitial EigenvaluesExtracted Sums of Squared LoadingsRotation Sums of Squared Loadings
Total% of VarianceCumulative %Total% of VarianceCumulative %Total% of VarianceCumulative %
Table 7. Component matrix rotated by equamax.
Table 7. Component matrix rotated by equamax.
A1 Harmful factors during the production process0.391−0.0960.4970.3050.155−0.065
A2 Harmful factors during the labor process−0.1570.0030.6610.3220.0440.029
A3 Harmful factors in the working environment−0.056−0.0580.8300.0370.0870.055
B1 Workers’ basic situation0.081−0.1640.1900.4930.015−0.026
B2 Workers’ contact characteristics0.035−0.0820.1540.709−0.051−0.018
B3 Workers’ quality0.1130.1680.1800.8170.095−0.077
B4 Workers’ knowledge of, attitude toward, and practice in terms of occupational hazards−0.2270.183−0.1180.630−0.1190.280
C1 Personal protection−0.038−0.2180.0150.000−0.0630.765
C2 Engineering protection0.1520.232−0.083−0.046−0.0520.685
D1 Occupational health surveillance0.6610.426−0.0030.0650.219−0.020
D2 Occupational hazard monitoring0.5650.308−0.0430.2480.074−0.038
D3 Materials and equipment management0.5080.3700.398−0.0150.320−0.159
D4 Auxiliary health facilities0.4370.340−0.108−0.032−0.096−0.016
D5 Emergency rescue facilities and warning sign configuration0.4930.3680.167-0.0930.4140.262
D6 Supervision of regulatory agencies0.532−0.138−0.1400.0380.3580.103
D7 Early prevention0.5420.4480.0680.178−0.1580.286
D8 Protection and management during operation process0.7270.7140.0820.287−0.0710.134
D9 Occupational hazard post-treatment situation0.5310.408−0.0340.0310.188−0.120
D10 Occupational health organizations0.7460.3860.167−0.0890.8040.132
D11 Occupational health regulations0.8290.1180.0520.0170.7760.102

Share and Cite

MDPI and ACS Style

Song, Y.; Chen, Z.; Zhang, S.; Wang, J.; Li, C.; Li, X.; Yuan, J.; Zhang, X. Comprehensive Evaluation System of Occupational Hazard Prevention and Control in Iron and Steel Enterprises Based on A Modified Delphi Technique. Int. J. Environ. Res. Public Health 2020, 17, 667.

AMA Style

Song Y, Chen Z, Zhang S, Wang J, Li C, Li X, Yuan J, Zhang X. Comprehensive Evaluation System of Occupational Hazard Prevention and Control in Iron and Steel Enterprises Based on A Modified Delphi Technique. International Journal of Environmental Research and Public Health. 2020; 17(2):667.

Chicago/Turabian Style

Song, Yang, Zhe Chen, Shengkui Zhang, Jiaojiao Wang, Chao Li, Xiaoming Li, Juxiang Yuan, and Xiujun Zhang. 2020. "Comprehensive Evaluation System of Occupational Hazard Prevention and Control in Iron and Steel Enterprises Based on A Modified Delphi Technique" International Journal of Environmental Research and Public Health 17, no. 2: 667.

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop