Application of the FMEA Tool in an Accredited Testing Laboratory in the Context of the ISO/IEC 17025:2017 Standard

Abstract

Organizations and laboratories tend to integrate systems based on risk management. Risk management helps to optimize laboratory processes and information flow, increase valid and reliable results, and make better decisions. This paper describes the development and present trends in risk management related to standard ISO/IEC 17025. This paper focuses on risk assessment in the accredited testing laboratories using the FMEA tool. In the basic eight defined laboratory areas, risks were identified and evaluated, and methods were proposed to minimize them.

1. Introduction

Risk identification is the step of identifying what could happen, and where, why, and how laboratory error can occur. It is the most important step of risk management. It is focused on minimizing possible risks and using efficient project and change management approaches to maximize the opportunities for successful completion of the project that could affect the entire laboratory activity and the achievement of the desired goals [1,2]. Nowadays, besides for risk management, business continuity management (BCM) has also become an important topic in organizations because of the COVID-19 pandemic. BCM is focused on strategies to deal with disruptive events such as floods, terrorist attacks, and pandemics [3].

Testing, as well as calibration, is a daily practice of more than 60,000 laboratories worldwide. The main objective is to ensure valid and reliable results for customers [4]. The Slovak National Accreditation Service states that there are more than 200 testing laboratories and 50 accredited calibration laboratories in the Slovak Republic. The accredited laboratories are competent and meet the requirements to provide impartial, confidential, and internationally valid results [5]. Personnel of the laboratory follows the risk management tools, procedures, and requirements of ISO/IEC 1725:2017 [6] (Figure 1). Personnel of the laboratory must first assess the possible occurrence of errors and outline the steps necessary for their detection and prevention before they may cause any undesirable event or severity. It is a process that minimizes the possibility of errors; thus, we can ensure the capability of laboratory test results. This process does not provide complete risk elimination, but it reduces the risk to an acceptable level. It is necessary to more specifically identify and then discuss how the specificities of the organization can be accounted for in order to comply with the specific items of ISO/IEC 17025: 2017 [7,8].

This standard helps laboratories demonstrate that they are operating competently, and they are producing technically valid results. It is ensured confidence in their work at the national as well as international levels. It also helps to facilitate cooperation between laboratories and other organizations, and the results of the accredited laboratories are acceptable mutually between the countries [6]. Therefore, highly regarded accredited laboratories reduce costs and minimize technical barriers in international trade by eliminating the need for retesting and calibration because they use the reliable data of other countries. This international standard guarantees the quality of production, testing, and calibration. It brings global confidence [9]. Up to now, research applying BCM in an accredited testing laboratory has not been realized, even though it is already widely used in the industry. It can be a future topic because BCM has the potential to play a decisive role in flexibility, velocity, and collaboration between personnel and customers, as well as response and recovery capabilities [10].

The standard applies to all laboratories regardless of the number of personnel. The standard specifies the general requirements for laboratories to enable them to demonstrate they operate competently in the area of testing and calibration [11]. The purpose of this paper is risk assessment in the accredited testing laboratory using the FMEA tool. The accredited testing laboratory provides authorized emission measurements, inspection activity of automatic monitoring emissions systems, and the calibration of emissions analyzers.

2. Methodology and Theoretical Requirements for Risk Management

2.1. Methodology of Risk Management in Accredited Laboratories

The methodology of risk management is illustrated in Figure 1. It is a process of prediction. It is time to think about what errors may occur. This process involves the assessment of the frequency of these errors, how often they occur, and the monitoring of undesirable events they cause. The final step of this process is to eliminate the risks to an acceptable level.

Various risks occur in laboratories because the laboratory is a workplace. In the laboratory, there are samples, chemicals, combustible materials, mechanical tools and machines, various pressure vessels, chambers, and others. Appropriate handling should be taken to avoid incorrect results, accidents, or damage to health [7,12].

Laboratories need to have risk management because each laboratory has its weak point where errors may occur. Each laboratory must try to eliminate errors to have the best results for tests and calibrations. Risk management should be part of laboratory processes [7,13].

According to Mascia et al., frequently, steps that involve human intervention are the weak links in the process. Risk analysis, therefore, gives considerable benefit to analytical validation, assessing and avoiding failures due to human error, potential imprecision in applying protocols, uncertainty in equipment function, and imperfect control of materials [14].

The management decides on risk assessment based on ISO 9000. It gives some freedom, and first of all it increases the responsibility of accepted measures. Organizations also have Integrated Management Systems to support business objectives and maintain them for a long time (business continuity) (Figure 2) [15].

2.2. International Standard ISO/IEC 17025:2018 and Theoretical Requirements for Risk Management of Risk Management in Accredited Laboratories

International Standard ISO/IEC 17025: 2017 is important for all types of laboratories that practice testing, calibration, or sampling, regardless of whether they are public, private, or part of industrial enterprises. The requirements of risk and opportunity management are incorporated into the Standard ISO/IEC 17025:2017 (

Table 1. Identification of risk of ISO/IEC 17025:2018, [16].

ISO/IEC 17025: 2017	Risk, Requirements
4.1.4; 4.1.5	Risk to impartiality
7.8.6	Level of risk associated with a decision rule used to make a statement of conformity to a specification (such as false accept and false reject and statistical assumptions)
7.10	Action taken for nonconforming work based upon risk level established by the laboratory
8.5	Actions to address risks and opportunities
8.7	Updated risks and opportunities when corrective action is taken
8.9	Management review includes results of risk identification
Examples of Particular Risks of ISO/IEC 17025:2017
Confidence between employer and laboratory personnel; relations between personnel involved in testing and calibration process; results falsification; unsatisfactory technical conditions of equipment; incorrect calibration of machines and equipment; unsatisfactory climatic conditions in the laboratory during testing; incorrectly selected methods; protocol error; non-updating risk register and protocol; incorrect procedures of testing and calibration; inappropriate shipping and storage conditions; sample damage; no internal audit; poor communication with customers; insufficient staff training; no goals achievement; information system failure; bad working machines; non-compliance with laws and standards; faulty equipment for measuring climate changes in the laboratory chamber.

) [16].

Laboratories can only deal with risks that are detected in time. Mapping a checklist from the pre-analytical through the analytical to the post-analytical testing phase can perform detection of risks in time.

2.3. Failures in the Phases of the Laboratory Testing Process

Errors can occur in any phase of the laboratory testing process. The phases are as follows: pre-analytic, analytic, and post-analytic phases (Figure 3). It is necessary to identify in which phase errors occur. Each phase of the laboratory process is affected by different risks. The risk relates to the unknown outcomes of the future event with the assumption that these outcomes will be undesirable. The prerequisite of a safe and competent laboratory is its management by managers who can apply the theory of risk and reduce it to an acceptable level [17,18].

The pre-analytical phase is the first phase of the laboratory testing process. According to the literature review of risk management, it is the most dangerous because studies have shown that 46% to 68.2% of laboratory errors occur in the pre-analytical phase. The pre-analytical phase represents all the procedures, processes, and analyses performed until the laboratory receives the sample. It includes an overall analysis of client demand [19]. The majority of errors come from the pre-analytical phase because of human dependence. The pre-analytical phase must include strict control measures related to communication between the laboratory and customers to avoid problems with customer demands [20]. It is therefore advisable to focus the effort on risk reduction at this stage, thereby maximizing the reliability of the entire testing process [21]. Errors that may occur in the pre-analytical phase are inappropriate test requirements, errors during order placement, use of incorrect containers, inadequate sampling and sample transportation, inappropriate sample volume, and sorting and labeling errors [22] (Figure 3). As reported by Carraro et al., the most common mistakes in the pre-analytical phase are missing samples, lost applications for sample testing, incorrect or missing sample identification, contamination, insufficient sample size, inappropriate packaging, and inappropriate shipping and storage conditions [21].

The second phase is the analytical phase. Unlike the pre-analytical phase, approximately 7% to 13% of laboratory errors occur in the analytical phase. This phase includes what are commonly considered “the real” laboratory testing or diagnostic procedures that eventually produce results. These activities are the main laboratory processes. All personnel of the laboratory must be competent, and each procedure must be documented. Errors that may occur in the analytical phase are equipment and instrument failure, failure in quality control, and incorrect testing or calibration procedures (Figure 3) [23].

The post-analytical phase is the final phase of the laboratory testing process. In the post-analytical phase occur 18.5% to 47% of errors. The post-analytical activities within the laboratory include verification of the results, their processing in the information system, and communication with clients. Problems may occur in the process of verifying the results when an inappropriate statistical method is discovered. If the distribution of data obtained from tests is not normal, it is necessary to use non-parametric tests (Figure 3) [24].

There are more studies that analyze testing and calibration activities performed in each phase from the pre-pre-analytical to the post-post-analytical phase. The goal of any risk management process is to identify, assess, mitigate, and eliminate the risks to an acceptable level. Risk management is a methodological discipline that involves the leadership coordination and management of all situations where risk may exist [25].

The output of the risk identification process is an updated, written list (register, catalog, database) of all identified factors/risks, and these data are further used as input for risk analysis and evaluation. Several management tools could be used to identify risks, such as FMEA, FTA, Ishikawa diagram, risk matrix, brainstorming, and others [26,27,28].

3. Failure Mode and Effect Analysis (FMEA)

FMEA is a common process analysis tool for identifying all possible or potential failures and the consequences of those failures. The purpose of the FMEA is to take actions to eliminate or reduce failures, starting with the highest-priority ones. It has been widely used in high-risk industries to evaluate and mitigate process weaknesses [29]. FMEA is used to prevent the occurrence of failures. This prevention may reduce the risk of harm to both customers and staff [30]. FMEA is particularly useful in evaluating a new process before implementation and in assessing the impact of a proposed change on an existing process. For example, incorporating FMEA in maintenance plan optimization can help organizations identify and prioritize the failure modes with the biggest impact. It saves time and costs while adding more value. Risk reduction occurs through the development of a preventive action plan to promote process improvement: immediate removal of the risk source when the pieces of equipment were increased; a change in the probability of certain risks when the selection process for new employees is initiated. This tool can bring benefits to future risk management and general process improvement within the laboratory environment [31]. FMEA is described as a process for identifying what could be wrong, why a failure could happen, what the consequences would be of each failure, and how to eliminate failures. FMEA can be complicated and time-consuming, as there is a number of individual laboratory activities with risks [32,33].

3.1. FMEA Method

FMEA is usually divided into two main phases and particular steps:

• Verbal phase

Using verbal methods, most often “brainstorming”, the following activities are realized:

Risk identification means, what is the likelihood that failure will occur? After making notes, possible failure modes are identified, and after that the possible consequence, what the consequences of that failure could be are discussed.

Because of the pandemic situation, society has changed, and there can be new potential risks to be discussed. A company’s success and society’s development are interrelated. Organizational integrity means that operations follow a clear set of values that meet societal expectations [34].

2.

Numerical phase

This phase is focused on risk calculation using a risk priority number (RPN). For each failure mode, an RPN is given by multiplying the severity score, occurrence score, and detection score. This allows for a focus on the highest scoring, highest risk problems first. Equation (1) is shown as follows:

RPN = SEV . Occur . Det

(1)

where (

Table 2. FMEA rating.

	Value	Severity of Effect	Likelihood of Detection	Probability of Occurrence
Safety/Regulatory/Legal Zone	10	May result in safety issue or regulatory violation without warning	Absolutely uncertain that failure will be detected	1 in 2
	9	May result in safety issue or regulatory violation with warning	Very remote chance that failure will be detected	1 in 10
Warranty/ Field Failure Zone	8	Primary function is lost or seriously degraded	Remote chance that failure will be detected	1 in 50
	7	Primary function is reduced, and customer is impacted	Very low chance that failure will be detected	1 in 250
	6	Secondary function is lost or seriously degraded	Low chance that failure will be detected	1 in 1000
	5	Secondary function is reduced, and customer is impacted	Moderate chance that failure will be detected	1 in 5000
	4	Loss of function or appearance such that most customers would return product or stop using service	Moderately high chance that failure will be detected	1 in 10,000
	3	Loss of function or appearance that is noticed by customers but would not result in a return or loss of service	High chance that failure will be detected	1 in 50,000
	2	Loss of function or appearance that is unlikely to be noticed by customers and would not result in a return or loss of service	Very high chance that failure will be detected	1 in 250,000
	1	Little to no impact	Almost certainty that failure will be detected	1 in 1 million

) [35]:

• SEV = Severity (1 = Least Severe, 10 = Most Severe),
• Occur = Probability of Occurrence (1 = Least Likely, 10 = Most Likely),
• Det = Probability of Detection (1 = Most Likely, 10 = Least Likely).

The FMEA process consists of the following steps: preparation and analysis.

• Step I—Preparation

It is very important to understand the process of production, namely, what the company is producing. All production or manufacturing is divided into particular steps. For each process step, there is a description of relations, brainstorming, and discussions. It may occur that some selected parts will be analyzed.

• Step II—Analysis

This step involves defining all potential process failures, their effects, and their causes. The important part of the analysis is risk assessment in terms of the probability of severity, occurrence, and detection (see Table 2). After that, RPN is defined.

RPN is an essential number that indicates the highest risk problems. There is another very important indicator, which can change the risk priority. The risk substance must be considered. Risks with RPNs of 9 and 10, which represent damage to health, death, loss of customers, or income loss, may be given higher priority even though their RPN number is lower than the numbers of other risks.

• Step III—Risk Minimization

Discussions of each failure mode and preventive actions specify who will put these measures into place and set up deadlines. The maintenance and operations team needs to ensure these items are completed as soon as possible to reduce costs.

3.2. Practical Application of FMEA

In this paper, the FMEA tool was applied to work out eight risk areas in the testing and calibration laboratory (

Table 3. Risk areas.

	Areas
1.	Economic circumstances in the laboratory
2.	Legislative changes
3.	Changes in technical standards
4.	Personal security
5.	Technical support
6.	Spatial security, working conditions, information systems
7.	Management activities and management interventions
8.	Process risks

). Laboratory personnel using brainstorming identified and defined potential risks. They considered legislative requirements, customer requirements, and potential losses with no-meet-mentioned requirements. The risk areas were selected by the organization and the FMEA analysis team, which was composed of senior internal supervisors. A total of 8 experts came from four different departments, including measurement and calibration management, economics, quality control, and the laboratory director. Each expert had more than 10 years of experience in measuring and implementing services and knew the structure and principles of FMEA. Based on their long-term experience and knowledge of possible critical conditions for the laboratory, these laboratory leaders determined eight risk areas (Table 3).

The following elements were also considered by the FMEA tool:

• The impact on performing laboratory,
• The probability of risk occurs in the laboratory,
• The possibility of risk detection in the laboratory.

According to the resulting RPN number, the risk effect is evaluated according to

Table 4. RPN evaluation.

Evaluation	Measure	RPN
High risk	Necessary intervention in the process is required	>150
Moderate risk	Process control is required	121–150
Low risk	No special measures required	<121

and the recommended action is taken. Table 4 is determined by the organization and analysis team based on the characteristics of the process being analyzed and other organizational factors, such as budget, customer requirements, applicable legal regulations, etc.

4. Results and Discussion

In each risk area (Table 3) potential brainstorming considering legislative requirements identified risks, customer requirements, and potential losses with no-meet mentioned requirements. Each risk brings consequences that must be defined and RPN evaluated. The next step is to the find causes of the risk. Each failure mode can have one or more causes of the risk. To identify causes, it is common to use the Ishikawa diagram. To identify a probability of risk occurrence, it is common to use Fault Tree Analysis (FTA). The recognition of potential risks and hazards in laboratory environments and activities is based on checklists, walk-through observations, and interviews with working individuals in laboratories.

Table 5. FMEA—identification, analysis, risk evaluation.

Areas	Risk	Potential Effect of Failure	SEV	Current Process Controls
				Potential Cause	Occur	Prevention	Detection	Det	RPN
2.	Suspension or revocation of accreditation	Sales recession, customer loss, restriction in operation	9	Not meeting conditions of impartiality (4.1)	3	Multiple inspections, quality manual, regulations in employment contracts	Quality manager, double check by two correctors	4	108
				Result interpretations according to own purpose	3			3	135
4.	Confidential break of laboratory test data (4.2)	Accidental or intentional transmission of data (to the customer or to the third person)	10	Attitude of employees to partial and final results	2	Training of personal data protection, retraining of employees	-	7	140
5.	External influences—energy sources, weather, etc. (6.3)	Inability to perform the measurement	7	Supply voltage fluctuations, influence of different electromagnetic fields	4	Verification of power supply, external power supply, location of the measuring vehicle of the frequency fields	Verification of the measure before measurement	1	28
				Bad weather—storm, rain	6	-	Responsible person	6	252
7.	Failure to update the risk list (8.5)	Errors in laboratory processes	7	Person responsible for risk management in the laboratory	5	-	-	7	245

shows examples of risk identification, analysis, and risk assessment.

The corrective actions were determined to be high risks, and also employees were responsible for meeting implementation deadlines and new requirements. Once action was taken, the new RPN numbers were recalculated, and the FMEA tool was reworked (

Table 6. FMEA—recommended actions.

Areas	Risk	Potential Effect of Failure	RPN	Recommended Actions			Action Results
				Recommended Actions	Responsible	Target Completion Date	Actions Taken	SEV	Occur	Det	RPN
2.	Suspension or revocation of accreditation	Sales recession, customer loss, restriction in operation	108	Multiple checks, trainings, knowledge of procedures	Quality manager	April 2022	Established training system	9	2	4	72
			135				Increased control level		3	3	81
4.	Confidential break of laboratory test data (4.2)	Accidental or intentional transmission of data (to the customer	140	Make a list of employees who have the right to see the test results	Executive manager	March 2022	List of responsible employees	10	1	5	50
5.	External influences—energy sources, weather, etc. (6.3)	Inability to perform the measurement	28	Improving communication with the customer	Measuring technician	March 2022	Established communication manual	7	3	1	21
			252	Weather monitoring	Measuring technician	March 2022	Established weather monitoring system		5	5	175
7.	Failure to update the risk list (8.5)	Errors in laboratory processes	245	Identify the person responsible for updates and implement the updated procedure	Quality manager	February 2022	Established procedure for updating risks	7	3	6	126

).

In total, 65 risks and 83 potential causes were identified in eight areas.

Table 7. Number of risks in each area according to the RPN value before and after the recommended actions.

Areas	Before Recommended Actions			After Recommended Actions
	High Risk	Moderate Risk	Low Risk	High Risk	Moderate Risk	Low Risk
1.	-	-	7	-	-	7
2.	1	1	4	-	1	5
3.	1	2	2	-	1	4
4.	7	-	3	2	2	6
5.	2	-	24	1	1	24
6.	2	-	3	-	-	5
7.	1	-	3	-	1	3
8.	2	2	16	-	1	19

shows the number of risks in each area according to the evaluated RPN based on Table 4.

Table 5 is an example of risks from each area. The proposed measures (Table 6) should lead to a reduction in RPN values. High-risk values in area 4 are caused by an insufficient representation of employees, incapacity for work, retirement, and leaving of authorized employees to the competition. The high-risk value in area 5 is caused by instrument damage during transport, and the use of non-updating evaluation software. Measures to improve detection are costly and ineffective. Thus, it is better to focus on cause probability because causes generate risks.

A strong vision of risk assessment requires the FMEA to be updated over time. It is easier to achieve it with appropriate computer support such as Design of Experiment (DoE), Monte Carlo (MC), FMEA, or QPR process guide [36,37,38].

With the application of measures, the RPN number should decrease compared to the original RPN number. If this does not happen, it is necessary to go back and repeat the procedure from the recommended action step. In our case, the RPN numbers decreased to the acceptable laboratory level by using appropriate measures and FMEA reassessment.

The most important limitations of the proposed approach in contrast to other FMEA are as follows: The initial output of an FMEA is the prioritizing of failure modes. This helps identify the most important failure modes to be addressed. If action is not implemented and evaluated for effectiveness, the failure mode will not be eliminated. In addition, further action outside the scope of the FMEA may be needed. Rating scales should be meaningful to everyone in the organization. The generic rating scales might be confusing to some teams. Management will not be able to compare risks to prioritize activities between teams. Future directions of methodological development referring to other scientific FMEA developments are formed, for example, based on a combination of the FMEA and RM-risk matrix methodologies, a novel risk assessment methodology called partial risk map (PRISM). The PRISM risk assessment method is more generic and can be applied in different operational fields as well, where the risk assessment is based on similar rating factors to the FMEA. Similar to the FMEA method, PRISM applies three risk assessment factors (probability of occurrence, severity of consequences, and degree of undetectability). Since the PRISM methodology defines and visualizes the phenomena of partial risk, the method describes well all the potentially existing hidden risks that are not taken into consideration by the RPN. In the future, we consider new logistics failures, such as the failure modes and failure reasons for the logistics system under the COVID-19 pandemic or a new area of environmental impact [34,39,40,41].

5. Conclusions

To achieve effective risk management in the laboratory, it is necessary to apply risk management in each step (phase) of the laboratory testing process. The output of the risk identification in the laboratory is a risk database, which contains all potential risks that may happen during laboratory processes. This database is the first step in having effective risk management integrated into accredited testing laboratories. Each accredited testing laboratory must integrate risk management in accordance with the requirement of the ISO 1725 standard. FMEA is one of these databases. It is a risk analysis tool to identify, prioritize, and minimize potential failures that may occur. The possibility of using the FMEA tool for risk assessment in an accredited testing laboratory has been demonstrated, taking into account the requirements of the ISO/IEC 17025 standard and the specifics of the organization as a characteristic of the analyzed process and other organizational factors such as the budget, customer requirements, applicable legal regulations, etc. Based on long-term monitoring, the organization determined eight risk areas in which potential risks were defined. Selected risks were treated and re-evaluated, resulting in a reduction of high risk from 16 to 3. FMEA should be always updated and should constantly reflect current risks and consequent measures. Complaints, changes in legislation, and standards are reactions to improve risk management. Prevention is better than a cure.