Validating Safety in Human–Robot Collaboration: Standards and New Perspectives

Abstract

Human–robot collaboration is currently one of the frontiers of industrial robot implementation. In parallel, the use of robots and robotic devices is increasing in several fields, substituting humans in “4D”—dull, dirty, dangerous, and delicate—tasks, and such a trend is boosted by the recent need for social distancing. New challenges in safety assessment and verification arise, due to both the closer and closer human–robot interaction, common for the different application domains, and the broadening of user audience, which is now very diverse. The present paper discusses a cross-domain approach towards the definition of step-by-step validation procedures for collaborative robotic applications. To outline the context, the standardization framework is analyzed, especially from the perspective of safety testing and assessment. Afterwards, some testing procedures based on safety skills, developed within the framework of the European project COVR, are discussed and exemplary presented.

1. Introduction

The concept of human–robot collaboration (HRC), based on the synergic work of robots and humans, questions the traditional paradigm of physical barriers separating machines and workers, lowering fences and closing distances. This paradigm shift was enabled by two main technological factors: the integration of safety-related features in the robot architectures and control systems and the use of multi-modal interfaces for more intuitive, aware and safer human–robot interaction (HRI) [1,2]. Moreover, thanks to the advances in artificial intelligence, robots can adapt their tasks and behaviors, becoming suitable to act in unstructured scenarios and interacting with unskilled and undertrained users.

The statistics show that, in the last years, several robot fields are increasing their markets, such as: robots for domestic tasks, entertainment robots, logistic robots, robots for public environments, defense applications, inspection and maintenance robots, professional cleaning, field robots, powered human exoskeletons, medical robots, construction and demolition [3]. Focusing just on the medical and healthcare domain, a non-comprehensive list includes: surgical robotics, boosting the transition from open surgery to laparoscopic and other minimally invasive surgical procedures [4]; bionic prostheses, enabling wearers to autonomously carry out their daily activities; the category of “RACA” robots, including a variety of devices, either wearable or static, to perform rehabilitation, assessment, compensation and alleviation (RACA); and caregiver robots, which are expected to play a fundamental role in aging societies [5]. In the agri-food sector, autonomous mobile robots provided by navigation and perception technologies represent a breakthrough, with the potential of performing mass phenotyping, crop management, and selective harvesting [6].

The importance of robots was underlined by the recent pandemic, as it is recognized that robot adoption can be extremely useful to support social distancing. Accordingly, robotic-assisted surgery has become a safe alternative, protecting the patient, the operating team, and saving resources for people with COVID-19 [7]. Due to their safe nature and possibility of “learning from demonstration”, collaborative robots can be implemented as sterilizable physical mediators between patients and clinicians, for example, for mass swabbing and physiotherapy treatments; at the same time, autonomous robots can perform UV sterilization of the environments and optimize internal logistics of health facilities [8]. Furthermore, from the perspective of social distancing, robotics can enable pandemic-sustainable travel, tourism and hospitality, performing environment sterilization, measuring body temperatures, welcoming and entertaining [9].

The aforementioned examples are characterized by different levels of HRI, spanning from teleoperation to workspace sharing and synergistic co-control. Taking a step back, one can observe that boundaries between industrial and service robotics are becoming blurred [10] and HRC is assuming a broader role, becoming applicable to a wide variety of robot applications in which the close interaction between human and robots is envisaged. However, the implementation of robots sharing space with humans, either to increase productivity rates, relieve us from heavy and repetitive tasks or mitigate certain hazard sources, always has to consider the mechanical hazards associated with HRC. Furthermore, the implementation and observance of safety is closely related to overall task performance, as safer interaction enables lower barriers and can help robots to reach their full potential.

In such a rapidly evolving landscape, in which industrial and service robotics become increasingly closer, identifying, interpreting and fulfilling the applicable standards for robot safety is critical. Based on the experience of the authors with robot end-users, manufacturers and integrators, the promotion of cobot technologies in different domains is slowed down by the path towards ensuring safety. This is due to both the limited knowledge about relevant standards and the absence of clear procedures to prove the compliance in these standards [11,12]. To address this need, a novel cross-domain validation approach is proposed in this paper, based on testing procedures made available through an online toolkit.

1.1. Safety in HRC

Increasing safety in collaborative robot operations is a goal pursued at different levels. Collaborative robots are intrinsically safer, but the implementation of external sensing strategies further improves safety, and safety-related systems can be used to enable HRC with traditional robots [13]. The research community is addressing several issues related to sensing, robot hardware and safeguarding measures; the main drivers concern increasing the perception of both robots and humans, lowering the potential damage due to mechanical features, control strategies or external protections, providing user-friendly programming strategies [1]. Aiming at increasing collaboration synergy and safety, human–robot interfacing can be enhanced by implementing different technologies, such as robot integrated vision, voice recognition, head-mounted displays for augmented reality, haptic-based devices and even the detection of bio-signals [2].

Safety-related aspects have a relevant impact in the design of HRC application, as hazard identification, risk assessment and the identification of risk mitigation strategies become even more crucial. In [14], a typical process is described, demonstrating that an appropriate modeling of safety aspects can reduce both the design time and the final workspace required by the layout. In a recent review [15], it is shown how the research outputs concerning safety and ergonomics are increasing, in the last years, with the rise of collaborative robotics, identifying four main clusters of interest: contact avoidance, contact detection and mitigation, physical ergonomics, cognitive and organizational ergonomics. Even if the two former categories are more widely addressed, the authors claim that ergonomic issues will play a major role when the advances in contact avoidance strategies will push the limits of HRI, leading to a complementarity of these aspects towards safe and efficient HRC.

Concerning the evaluation of safety in HRC, some approaches use objective measures to plan and evaluate the performance of applications featuring “speed and separation monitoring” collaborative scenarios [16,17]. Unfortunately, they also demonstrate the difficulties in identifying the point in time during a robot’s trajectory where a specific algorithm is the least safe, requiring either a simulation or a test with the completed system.

Even if these approaches can provide a certain measure of the HRC safety, they are not aimed at the validation of robotic applications with reference to directives and standards. Furthermore, they are targeting robot experts dealing with HRC scenarios in the industrial domain, while the ambition of the present work is to meet the needs of a diverse user audience, operating in different domains with the potential for implementing cobots in a wide range of tasks.

1.2. Paper Contribution

The risks related to an autonomous robot acting in a public environment are similar to the ones typical of a driverless mobile robot in industry. In the same way, similar risks characterize a manipulator for collaborative assembly and one implemented for delicate health diagnostic procedures. This leads the consideration that the specific safety-related testing procedures could rely on a common base of knowledge and experience. It is also worth observing that, even in the fields characterized by higher levels of expertise, close HRC brings new challenges for safety verification and validation. Accordingly, the analogue hazards make the guidance for clear, step-by-step testing procedures a common need of both experienced and new users.

The aim of this paper is to outline emerging necessities and possible trends concerning safety testing procedures for cobot applications, based on an overview of the landscape of relevant standards for the validation of safety, and to describe a new cross-domain approach based on safety skills and testing protocols. In this context, we refer to “safety skills” as the capability of a robotic system to reduce a specific risk and to “protocols” as step-by-step instructions for executing validation measurements. It is a belief of the authors, indeed, that clearer safety assessment processes allow cobots to be used with more confidence in more situations; this can be a valuable boost to increase the variety of cobots on the market and the variety of services cobots can offer to the general population. As a remark, the proposed protocols are not intended to substitute standards; they represent instead a tool to guide and facilitate the users in the identification of relevant standards and their fulfillment.

The paper is organized as follows. In Section 2, the standard framework is addressed; building on the overview of all the relevant standards, some recent publications dealing with detailed test procedures, are described and gaps with user needs are highlighted. Section 3 deals with the proposed cross-domain approach, which is based on the definition of safety skills and the development of testing protocols, while in Section 4 some exemplary cases of protocol application are reported.

2. Robot Safety: The Regulatory Framework

2.1. Overview

The main regulation in the European community dealing with robot safety is the Machinery Directive 2006/42/EC [18]. It is translated in all the national languages and transposed into laws by each member country. A robot falls in the scope of the Machinery Directive, as it is composed of “linked parts or components, at least one of which moves” and being actuated by a drive system. A programmable robot supplied by a robot manufacturer is regarded by the Machinery Directive to be a “partly completed machinery”. This means that the robot itself is not CE-marked for Machinery Directive, but that all information needed by integrators to ensure safety is provided by the robot manufacturer. To be considered as a “completed” machinery, it must be designed or integrated for a specific application. In industrial robotics, the concept of machinery applies to robotics applications or robotic cells and whereby the integrator is considered to be the manufacturer. In other fields, such as robots for consumers, assistive robotics or medical robotics, the intended use must be clearly defined.

The Medical Device Regulation EU 2017/745 [19] applies to rehabilitation and, more generally, to healthcare-related activities. It replaced the Medical Device Directive, which focused mainly on the design of the system, providing more safety considerations over the entire device lifetime [20]. The Medical Device certification is strongly based on an “intended use” and valid only for this “intended use” for which clinical evidence must be provided. So, in accordance with the Machinery Directive, a generic robot without his specific application cannot be certified. It is worth noting that, for a robotic medical device, the Machinery Directive must also be observed. Other directives may apply depending on the field of use, such as the General Product Safety Directive [21], when a robotic device is made available on the consumer market, or a type-approval regulation when a robot becomes a vehicle (this category is outside the scope of this paper). Finally, there are more general relevant directives, such as the Low Voltage Directive [22], the Electromagnetic Compatibility Directive [23], and the Radio Equipment Directive [24].

Standards are technical reference documents, representing the consensus on the state of the art and compliance to them is not mandatory. Technical committees (TCs) are in charge of developing standards and the main committee within ISO for industrial and service robotics (excluding toys and military applications) is the ISO TC 299 “Robotics”. Robotics ISO standards from this group mainly address vocabulary, performance and safety. Concerning safety, they describe solutions gathering a certain level of consensus within the specific community. Indeed, robot manufacturers typically take part in standardization committees, along with integrators, end-users and stakeholders representing public health and health insurance. “Harmonized” standards are recognized by a European standard organization as fully compliant with the relevant directive or regulation; thus, their application provides the so-called “presumption of conformity”. The most relevant standards for robot safety from the perspective of HRC are listed in Table 1.

Safety standards are categorized into: type A, providing the basic design principles and valid for all machines; type B, which are generic standards concerning either specific aspects of safety or devices for safeguarding; and type C, covering particular classes of machines (i.e., robots) and having precedence over type A and B. Accordingly, for robots belonging to the categories of industrial robots or personal care robots, type C standards are the main reference. The list reported in Table 1 is not comprehensive of all the robotics-relevant type B standards, as it is limited to standards dealing with HRC, either explicitly or not. An up-to-date list of all the robotic standards, which can be filtered by application domain and device type, can be found in COVR Toolkit [25].

The Machinery Directive sets out the “Essential health and safety requirements relating to the design and construction of machinery”. Fulfilling these requirements is based on an “iterative process of risk assessment and risk reduction” for which one can rely on the standard ISO 12100 [26] (type A). It should be noted that the medical device safety is addressed by a dedicated standard for risk management, the ISO 14971 [27]. The Machinery Directive promotes the integration of safety in all design stages, giving priority to inherent safety, followed by protective measures, and, lastly, by organizational measures, such as user training and personal protective equipment.

The ISO 10218 is a type C standard for industrial robotics, divided into two parts: the first [28] dealing with robots considered as “partly completed machinery” and the second [29] addressing integrated applications, which are the machineries to be considered in compliance with the Machinery Directive. Even though harmful HRI had already been addressed in the first versions, intended physical interaction has been introduced in 2009 in both 10218-1 and -2, by describing “hand-guiding” (HG) and “power and force limiting” (PFL) operating modes. The possibility to avoid safeguards by the use of distance sensing was introduced with “safety-rated monitored stop” (SRMS) and “speed and separation monitoring” (SSM). The safety requirements for these collaborative modalities are currently under development for the upcoming revision of ISO 10218 (to be published in 2022), as will be addressed in the following paragraph.

Technical specifications (TS) and technical reports (TR) are different ISO deliverables. TS address work still under development but with a chance to be included in an international standard, while TR contain other kinds of information, for example, the perceived state of the art of a specific topic. ISO/TS 15066 [30] represented a milestone among HRC standardization. Besides including information regarding collaborative robot system design, hazard identification, risk assessment and the requirements for the applications, it provides a more detailed description of the collaborative modes SRSM, SRSS, PFL, HG and limit values for quasi-static and transient contact forces are illustrated.

Table 1. List of the most relevant standards dealing with safety requirements in HRC.

ID	Standard	HRC Relevance	Type	Harmonized
ISO 12100:2010 [26]	Safety of machinery—general principles for design—risk assessment and risk reduction	Risk assessment	A	Yes
ISO 13855:2010 [31]	Safety of machinery—positioning of safeguards with respect to the approach speeds of parts of the human body	Suitable for SSM	B	Yes
ISO 18497:2018 [32]	Agricultural machinery and tractors—safety of highly automated machinery	Safety requirements, verification	B	Yes
ISO 13851:2019 [33]	Safety of machinery—two-hand control devices—principles for design and selection	Suitable for HG	B	Yes
ISO 10218-1:2011 [28]	Robots and robotic devices—safety requirements for industrial robots—part 1: robots	HRC operation requirements	C	Yes
ISO 10218-2:2011 [29]	Robots and robotic devices—safety requirements for industrial robots—part 2: robot systems and integration	HRC operation requirements	C	Yes
ISO 13482:2014 [34]	Robots and robotic devices—safety requirements for personal care robots	Guidance for safety (lifecycle)	C	Yes
ISO 3691-4: 2020 [35]	Industrial trucks—safety requirements and verification—part 4: driverless industrial trucks and their systems	Human detection	C	Yes
IEC 80601-2-78: 2020 1 [36]	Medical electrical equipment—part 2-78: particular requirements for basic safety and essential performance of medical robots for rehabilitation, assessment, compensation or alleviation	Safe HRI with RACA robots	n.a.	No
ISO/TS 15066:2016 [30]	Robots and robotic devices—collaborative robots	Focused on HRC	TS	n.a.
ISO/TR 23482-1:2020 [37]	Application of ISO 13482—part 1: safety-related test methods	Testing procedures for ISO 13482	TR	n.a.
ISO/TR 20218-1:2018 [38]	Robotics—safety design for industrial robot systems—part 1: end-effectors	Requirements for end-effectors	TR	n.a
ISO/TR 20218-2:2017 [39]	Robotics—safety requirements for industrial robots—part 2: manual load/unload stations	Req. for load/unload stations	TR	n.a

¹ IEC 80601-2-78 belongs to a series of particular standards that can modify, replace or delete requirements of the general standard IEC 60601-1:2020 (Medical electrical equipment—part 1: general requirements for basic safety and essential performance).

Table 1. List of the most relevant standards dealing with safety requirements in HRC.

ID	Standard	HRC Relevance	Type	Harmonized
ISO 12100:2010 [26]	Safety of machinery—general principles for design—risk assessment and risk reduction	Risk assessment	A	Yes
ISO 13855:2010 [31]	Safety of machinery—positioning of safeguards with respect to the approach speeds of parts of the human body	Suitable for SSM	B	Yes
ISO 18497:2018 [32]	Agricultural machinery and tractors—safety of highly automated machinery	Safety requirements, verification	B	Yes
ISO 13851:2019 [33]	Safety of machinery—two-hand control devices—principles for design and selection	Suitable for HG	B	Yes
ISO 10218-1:2011 [28]	Robots and robotic devices—safety requirements for industrial robots—part 1: robots	HRC operation requirements	C	Yes
ISO 10218-2:2011 [29]	Robots and robotic devices—safety requirements for industrial robots—part 2: robot systems and integration	HRC operation requirements	C	Yes
ISO 13482:2014 [34]	Robots and robotic devices—safety requirements for personal care robots	Guidance for safety (lifecycle)	C	Yes
ISO 3691-4: 2020 [35]	Industrial trucks—safety requirements and verification—part 4: driverless industrial trucks and their systems	Human detection	C	Yes
IEC 80601-2-78: 2020 1 [36]	Medical electrical equipment—part 2-78: particular requirements for basic safety and essential performance of medical robots for rehabilitation, assessment, compensation or alleviation	Safe HRI with RACA robots	n.a.	No
ISO/TS 15066:2016 [30]	Robots and robotic devices—collaborative robots	Focused on HRC	TS	n.a.
ISO/TR 23482-1:2020 [37]	Application of ISO 13482—part 1: safety-related test methods	Testing procedures for ISO 13482	TR	n.a.
ISO/TR 20218-1:2018 [38]	Robotics—safety design for industrial robot systems—part 1: end-effectors	Requirements for end-effectors	TR	n.a
ISO/TR 20218-2:2017 [39]	Robotics—safety requirements for industrial robots—part 2: manual load/unload stations	Req. for load/unload stations	TR	n.a

¹ IEC 80601-2-78 belongs to a series of particular standards that can modify, replace or delete requirements of the general standard IEC 60601-1:2020 (Medical electrical equipment—part 1: general requirements for basic safety and essential performance).

2.2. A New Trend in Standardization?

If in domains characterized by well-established validation practices and reference standards, such as the industrial field, the presence of collaborative robots brings new challenges in the assessment of human–robot interaction, the increasing spread of robots in environments shared with humans generates new application scenarios, involving new types of users. An inexpert end-user can harbor the false perception that the mere implementation of a collaborative robot is sufficient to ensure safety, due to its low weight, ease of use and even friendly appearance. As a result, these two parallel trends—the spread of robots among humans and the increase of end-users new to the robot world—increasingly pose the need of clear procedures for testing robots sharing their workspaces with humans. There are recent signals that the standardization world is moving to meet this need and provide the required procedures. In

Table 2. HRI safety-related test procedures reported in the cited standards.

Standard	Test	Section	Description
ISO/DIS 10218-2: 2020 [40]	Power- and force-limited robot applications—pressure and force measurements	Annex N	The required measuring device for the measurements is described, which has to incorporate a spring and a soft pad, whose hardness and stiffness, respectively, must comply with specific values depending on the body part involved in the potential contact. The setup, measurement and data analysis procedures are then described to test both quasi-static (clamping) and transient contact force events.
ISO 3691-4: 2020 [35]	Tests for detection of persons	§ 5.2	The truck travels towards a cylindrical test piece with defined dimensions, placed in different positions and poses along the path. Test success corresponds to a full truck stop before contact (in case of contactless detection means) or to a contact with a limited interaction force (in case of bumper-based detection), observed over three repetitions.
	Stability tests	§ 5.3	The worst-case conditions must be replicated (loaded, unloaded, lift height, slope, turn, forward direction, backward direction, floor/ground slope) and be stable, or otherwise, stability can be proved by calculations.
ISO/TR 23482-1: 2020 [37]	Physical hazard characteristics	§ 6	It includes voltage at user-accessible parts, acoustic noise and surface temperature.
	Physical hazard characteristics (for mobile robots)	§ 7	This category includes the tests of injury parameters in collision, performed using a dummy with features as per the US code of Federal Regulations and the tests of force control for intended and unintended contact with a robot, performed with a contact piece simulating the human body part and a force transducer and a pressure sensor as sensing equipment. In both cases, the acquired impact force values are compared with the ones listed in the ISO/TS 15066.
	Physical hazard characteristics (for restraint-type physical ass. robots)	§ 8	The physical stress or strain to the user is tested by means of a setup composed by a force sensor, a dummy, a cuff with force/pressure sensing capability and a manipulator moving the cuff. The combination of tangential traction forces and continuous repeated rubbing contact acquired is compared with reference curves referred to the generation of blisters in human skin.
	Static stability characteristics	§ 11	The robot is placed on a test plane with a slope, if necessary for the type of vehicle transporting a dummy, and the stability is observed.
	Dynamic stability characteristics with respect to moving parts (mobile robot)	§ 12	The robot is placed on a test plane with a slope, if necessary with a dummy simulating shifting loads. The worst-case directions are considered, and the robot moves, maximizing the generated dynamic forces and, if appropriate, moving loads or dynamic passengers are also simulated.
	Dynamic stability characteristics with respect to travel (for mobile robot)	§ 13	Replicating the same working conditions, with the use of dummies if necessary, stability tests are performed on a flat surface (breaking and acceleration tests), on an inclined surface (maximum speed, acceleration and braking on downward slope, upward slope acceleration, downward slope full turn, crossing, pivot turn) and in relation to potential steps and gaps in the environment.
	Safety-related control functions (universal)	§ 14	Different tests consider the electro-sensitive protective equipment (ESPE), performed with human-like objects to be detected, the operation in slippery environments, and electro-magnetic immunity.
	Response to safety-related obstacles on the ground (mobile robot)	§ 15	These tests include the assessment of: - the distance of protective stop, performed towards a wall, a cylindrical post or a dummy, with different travel directions, facing angle and position of test piece, and measuring the distance after stop; - the stopping distance before concave terrain moving towards it (similar procedure).
	Safety-related localization and navigation	§ 16	With a pre-defined environment map and programmed path, an obstacle is positioned in different positions along the robot path. Jerky robot movements, unexpected stops or other potentially hazardous movements are detected.
	Reliability of autonomous decisions and actions (universal)	§ 17	These tests are related to the autonomous action of identifying an object, interpreting user commands, choosing a strategy to minimize the collision risk, etc. The document describes only the test for object identification.
ISO 18646-1:2016 [41]	Rated speed	§ 5	The aim of the test is to determine the wheeled robot rated speed for travel-related tasks. In a cycle consisting in acceleration, constant speed, and deceleration, two sensors detect the passage of the robot at constant speed and the rated speed is obtained by calculation.
	Stopping characteristics	§ 6	The robot moves in a straight line up to the rated speed; afterwards a stop command is initiated. Once the robot is fully stopped, stopping distance and times are obtained by the acquisitions.
ISO 18646-2:2019 [42]	Obstacle detection	§ 6	This is a static test in which six different obstacles are positioned at specific distances from the robot, corresponding to the maximum and minimum acquisition ranges declared by the manufacturer, with different orientations with respect to the line of sight.
	Obstacle avoidance	§ 7	This test determines the ability of the robot of reaching a goal position avoiding moving obstacles along its path. The test is performed by commanding moving obstacles to move along a path, normally causing collision with the robot, which is, in turn, commanded to autonomously move along a path. The test is repeated with different obstacle trajectories. The test is successful if the robot reaches the goal position in all the trails, and a time “delay factor” is calculated.

, the tests currently available in these standards are listed.

The new version of the ISO 10218-2, currently under development, is expected to give greater attention to collaborative applications. As a “draft international standard” (DIS), the ISO/DIS 10218-2 [40] was published in late 2020 and is currently under evaluation. Collaborative applications are identified as characterized by one or more of three technologies: “hand-guided controls” (HGC), “speed and separation monitoring” (SSM), and “power and force limiting” (PFL). Specific risk assessment is envisaged for potential human–robot contact conditions, as well as related passive safety design measures and active risk reduction measures. To the latter category belongs the safety function “monitored-standstill”, corresponding to the capability of performing the SRMS collaborative mode, no longer addressed. Furthermore, new annexes are specifically dedicated to HRC. Among these, Annex L describes how to calculate the protective separation distance in SSM and Annex M reports the limits for quasi-static and transient contact. These annexes incorporate the information previously provided by the ISO/TS 15066 [30]. The most relevant from the perspective of safety verification and validation testing is Annex N, dedicated to pressure and force measurements in PFL robotic applications.

The recently published ISO 3691-4 [35] provides some testing procedures. It applies to driverless industrial trucks, which are powered trucks, designed to work automatically. Examples of trucks falling in this category are all-automated guided vehicles, autonomous mobile robots, bots, automated guided carts, tunnel tuggers, under carts, etc., whereas trucks guided by rails or other mechanical means or controlled remotely are excluded. Therefore, industrial mobile platforms and several types of mobile service robots are included. From the perspective of the safety assessment, the relevant section is the one related to the “verification of safety requirements and/or protective measures”, in which testing procedures are specified, including tests for the detection of persons and stability tests.

As a technical report, the ISO/TR 23482-1 [37] “describes test methods which are guidelines to verify compliance to the requirements of 13482”, which, in turn, describes “safety requirements for personal care robots”. In the TR, test conditions are provided as general rules and practices for the performance of all the tests. Afterwards, a list of several tests is provided, organized depending on the related hazard, along with a detailed description of the test principle, apparatus, procedure, and pass/fail criteria.

Even if the ISO 18646 standard series deals principally with robot performance criteria and assessment, this aspect is closely related to safety, especially in the case of performance related to safety-related functionalities. Accordingly, some performance testing procedures are reported in Table 2, extracted from the ISO 18646-1 [41] and the ISO 18646-2 [42].

2.3. Analysis of the Gaps

Whereas, 20 years ago, industrial robots were almost solely used for high volume manufacturing, today the same robot manipulator can be used for manufacturing, logistics, rehabilitation, or even agricultural applications. This can lead to uncertainty with respect to safety and applicable standards, especially due to the domain-specific organization of international safety standards, through two specific ways. On the one hand, in situations where innovations occur faster than standardization, there are not always relevant domain-specific standards for the application in question. On the other hand, combinations of device types can lead to conflicting issues from different safety standards. An example is a mobile platform equipped with a robot manipulator. The forces specified during contact are different when considering an industrial manipulator (in the ISO/TS 15066 [30]) or an autonomous truck (ISO 3691-4 [35]). Further challenges arise when considering what separation distances to apply, as the ISO/TS15066 [30] also specifies that the approach speed of humans needs to be taken into account, whereas the ISO 3691-4 [35] does not.

While adherence to standards is not legally binding, they do represent the state of the art and can be extremely helpful for considering the safety of collaborative robotics applications. A streamlined approach that offers robotics stakeholders the means to conceptually talk about the safety of their system, regardless of the specific domain, would be helpful here. Furthermore, it would be extremely helpful to the robotics community if the same approach were to extend to methods for validating the implemented risk reduction measures. This approach should, however, respect the fact that there are a wide variety of technical means available for implementing the safety.

3. Safety Skills and Testing Protocols in a Cross-Domain Perspective

In the consolidated practice, boundaries between different robotic domains are well-defined and recognized. One typical example is that HRI with robotic medical devices can also generate physical benefits, balancing the risks associated to their operation, while this is not applicable for other types of robots. However, this approach can represent a limit when similar hazards have to be addressed in different domains; in these cases, indeed, cross-domain fertilization can provide an extra gear to define the best practices for the verification of safety requirements. This section proposes a safety verification approach based on the definition of cross-domain safety skills and the development of testing protocols, drafted on the basis of practical needs and updated by expert consultations and by the examination and addressment of real application cases. In Section 3.1, the safety skill concept and the skill-based approach is described, whereas the testing protocol structure and the list of the protocols are addressed in Section 3.2.

3.1. The Skill-Based Approach

When considering the safety of collaborative robotic systems, it is essential to start with a risk assessment, whereby hazards specific to a concrete application are identified and risk reduction measures (RRM) are chosen. The operation methods for collaborative robotics applications, such as the SRMS, SSM, PFL, HG, as currently defined by the ISO 10218, can be implemented through a variety of technical means and offer the planner a level of abstraction to discuss and consider the safety of the application. These definitions may become challenging when other device types and domains rather than manufacturing are considered, such as industrial robotics for rehabilitation or mobile robots for agriculture.

To overcome this challenge, the concept of safety skills has been proposed in [43]. Safety skills are defined as an abstract representation of the ability of the robot system to reduce some risk, i.e., to deploy suitable RRM. The implementation of the protective measure can be executed in a number of ways. A safety skill therefore indicates what kind of protection is desired and is independent of the execution of how this protection is delivered. A further characteristic of safety skills is that they can be validated for specific applications at a system level.

A two-pronged approach towards the definition of cross-domain safety skills was applied. In addition to an identification of existing operating modes from available standards, an analysis of possible hazards and risk reduction methods for a large variety of applications featuring a combination of different robotic device types for six domains (manufacturing, rehabilitation, agriculture, civil, logistics, and consumer/home) was executed. While the latter analysis was quite large, it cannot be considered comprehensive, as robotics are increasingly being used for new applications in an even wider variety of domains than those analyzed. Nevertheless, the concept of cross-domain safety skills is explicitly conceived to deal with the novelty of future robotics applications and provide guidance for how to consider their safety. The safety skills that were identified and associated and known operating modes from existing standards are listed in

Table 3. Safety skills identified and corresponding operating modes and/or testing procedures from existing standards.

Icon	Safety Skill	Corresponding Operating Modes and/or Testing Procedures with Standard Reference
	Maintain safe distance (MSD)	ISO/TS 15066 [30]: SRMS (§5.5.2), SSM (§5.5.4). ISO/TR 23482-1 [37]: response to safety-related obstacles on the ground (§15). ISO 3691-4 [35]: tests for detection of persons (§5.2).
	Maintain dynamic stability (DYS)	ISO 3691-4 [35]: stability tests (§5.3). ISO/TR 23482-1 [37]: static stability characteristics (§11), dynamic stability characteristics with respect to moving parts (§12), dynamic stability characteristics with respect to travel (§13), response to safety-related obstacles on the ground (§15).
	Limit physical interaction energy (LIE)	ISO/TS 15066 [30]: PFL (§5.5.5), (HG, §5.5.3). IEC-80601-2-78 [36]: overtravel end stops (§201.9.2.3.2), movement beyond pre-set limits for individual. Patient movement (§201.9.2.3.101), movement or force/torque of RACA robot (§201.12.101), mechanical hazards associated with misalignment (§201.9.101). ISO/TR 23482-1 [37]: physical hazard characteristics (for mobile robots), (§7)
	Limit range of movement (LRM)	ISO 10218-1 [28]: speed control (§5.6), axis and space limiting (§5.12). IEC-80601-2-78 [36]: movement beyond pre-set limits for individual patient movement (§201.9.2.3.101), overtravel end stops (§201.9.2.3.2) (ISO/TS 15066 [30]: HG, §5.5.3).
	Maintain proper alignment (MPA)	IEC-80601-2-78 [36]: mechanical hazards associated with misalignment (§201.9.101)
	Limit restraining energy (LRE)	ISO/TR 23482-1 [37]: test of physical hazard characteristics (for restraint-type physical assistance robots), (§8). IEC 80601-2-78 [36]: accuracy of controls and instruments and protection against hazardous outputs (§201.12), mechanical hazards associated with misalignment (§201.9.101), movement beyond pre-set limits for individual patient movement (§201.9.2.3.101).

.

3.2. Protocols for Skill-Based Validation of Applications

As per their definition and scope, skills are abstract concepts applying to different collaborative operation scenarios. Their proper and effective application depends on several operation features, such as the domain in which they are tested, the design of the robotized operation, and the type of robotic system involved. Verifying and validating the application of a safety skill in a specific scenario corresponds to assessing the application’s safety features from the perspective of the specific skill, providing evidence of the effectiveness of the safety measures implemented with reference to the relevant standards. Such an assessment is not a trivial task, as it requires, in order: (i) a comprehensive knowledge of the applicable regulation landscape, (ii) critical awareness of the relevant physical metrics and performance data to be measured, (iii) technical knowledge of the most appropriate testing equipment and methodologies, (iv) the production of clear, complete and self-explanatory reports.

A suitable methodology to validate safety skills relies on the application of testing protocols, developed with the specific scope of providing a step-by-step guide on how to execute a validation measurement to check the safety of a robotics application. To exploit the potential of the cross-domain approach, protocols have to be general procedures, applicable in several domains and individuated only by two fundamental variables, that are the safety skill to be validated and the robotic device involved in the specific task to be used for the validation. To maximize their impact, protocols have to pursue the following scopes:

-

increasing the familiarity in the robotics community with possible measuring techniques;

-

informing protocol users of what aspects of their risk analysis and system behavior are relevant for the validation.

In fact, the risk assessment, conducted as per the relevant standards, is a fundamental, preliminary step for the validation of a safety skill. It is necessary, indeed, to define the conditions in which protocol procedures must be applied; these conditions depend on the specific installation, environment, task, user awareness, safeguarding measures provided by the manufacturer, prescribed protective measures, and so on.

A set of safety skill validation protocols is available in COVR Toolkit [25]; they are designed for the system-level evaluation of the safety performance. To cope with the aforementioned requirements, the structure of a protocol is organized to guide the user in all the steps of the safety skill validation (see

Table 4. General structure of safety skill validation protocols.

Section	Contents
Introduction	Relevant regulations and standards Scope and limitations
Concept and objectives	Characterization of hazards to consider for skill validation Target metrics: physical, measurable quantities on which depends the validation (or not) of a skill in relation to a specific risk reduction level
Conditions	System, environment, possible subsystems, other relevant aspects
Setup	Based on the measurement device indicated for the specific safety skill
Procedure	Step-by-step testing procedure Practices for data analysis Formats for reporting and documenting the validation of the skill
Annexes	Record sheets and eventual further relevant information

). Aiming at enhancing clarity of contents and usability, relevant stakeholders and protocol users can provide feedback during the whole protocol lifecycle. As a result, the available protocols are proven testing procedures, intended to be dynamically and periodically updated according to the increasing usage feedbacks. They are indeed published once a “Protocol Readiness Level” 7 (PRL, defining the level of protocol advancement on a nine-level scale) is achieved and they are open for further user feedbacks, collected by means of specific questionnaires.

The development of a protocol depends on the analysis of the meaningful combinations of safety skills and robot device type, as reported in

Table 5. Identification of protocols as a combination of device types and safety skills.

Device Type		MSD	DYS	LIE	LRM	MPA	LRE
Robotic arm	ROB	1, 2 *	-	1, 2	1, 2, 3, 4, 5 *	-	1
Mobile platform	MOB	1	1	1	1	-	-
Exoskeleton	EXO	-	-	1	1	1	1 *
Gripper **	GRI	-	-	1	1	-	-
Weight support	WSU	-	-	1 *	1 *	1 *	1 *
Balance Trainer	BAT	-	-	1 *	1 *	1 *	1 *
Mobile robot	MRO	-	1	1	-	-	-

NOTE: Safety skills acronyms (MSD, DYS, LIE, LRM, MPA, LRE) are explained in Table 3. The numbers represent the different protocols developed per each combination of device type and skill. * Still not published (PRL ≤ 6). ** Considered separately as characterized by a specific motion (closing jaws).

. The list of robotic device types reported in the table is based on the systems typical of the industrial practice, whose implementation and use are extended to several application domains (i.e., robotic arms in assistive robotics or mobile robots in the agricultural domain) and on the robotic systems typically used in the rehabilitation field. This set represents a first nucleus of devices, but further systems can be included to develop more validation protocols. In some cases, for the same device–skill combination, several protocols were developed, considering the use of different measuring equipment. Likewise, there can be specific operation conditions, which can variate depending on task design or the context of use, leading to different testing approaches. These aspects result in the development of several protocols for the same device–skill combination, individuated by the increasing numbers in Table 5.

The list of protocols currently available in COVR Toolkit is reported in

Table 6. Summary table of the safety skill validation protocols available on COVR Toolkit (PRL ≥ 7).

ID 1	Conditions	Object	Main Reference	Metrics	Sensing Device(s)	Data Analysis	Repetitions
ROB-MSD-1	Indoor/factory	Human entering robot workspace	ISO 13855 [31] ISO/TS 15066 [30]	Safety Distance Ss	String potentiometer, triggering actuator	Ss ≤ SP or Xs ≤ XH	5
ROB-LIE-1	Indoor/factory	Moving obstacle (human body part)	ISO/TS 15066 [30] DGUV FBHM 080 [44]	Transient contact force and pressure	Force and pressure (biofidelity to the specific body part)	Filtering, correction 2, comparison with limits in [30]	3
ROB-LIE-2	Indoor/factory	Stationary obstacle (human body part)	ISO/TS 15066 [30] DGUV FBHM 080 [44]	Transient and quasi-static contact force and pressure	Force and pressure (biofidelity to the specific body part)	Filtering, comparison with limits in [30]	3
ROB-LRE-1	Robotic device, single human joint	Pre-defined torque, human body part	IEC 80601-2-78 [36]	Effective torque	Force sensor	Obtain torque and compare with limits	2 per 3 sets
ROB-LRM-1	Indoor/ factory	Vertical plane (workspace limit)	ISO 10218-2 [29] ISO 10218-1 [28]	Robot structure point Pcross crosses plane [T/F]	Light barrier or target plane + camera or similar	Check Pcross does not cross the plane	3 (change angle)
ROB-LRM-2	RACA, 3D mov., robot control	Pre-defined ROM, human body part	IEC 80601-2-78 [36]	ROM exit [T/F], potential collision [T/F]	Optoelectronic measurement or motion tracking system	Acquire robot motion and compare to ROM
ROB-LRM-3	RACA, 3D mov., co-control	Pre-defined ROM, human body part	IEC 80601-2-78 [36]	ROM exit [T/F], potential collision [T/F]	Optoelectronic measurement or motion tracking system	Acquire robot motion (external co-guiding) and compare to ROM
ROB-LRM-4	RACA, 3D mov., co-control	Pre-defined ROM, human body part	IEC 80601-2-78 [36]	ROM exit [T/F], potential collision [T/F]	Instrumented limb (angular encoders) attached to the end-effector	Obtain robot motion (external co-guiding) and compare to ROM
MOB-LRM-1	Indoor	Forbidden area		Footprint observation: entrance in forbidden area [T/F]	Position tracker or video-camera or light curtain or laser scanner or similar	Acquire robot motion from top or process the acquisition	3
MOB-DYS-1	Indoor	Accelerations, stops, payload, floor tilt	EN 1525 [45] ISO 3691-4 [35]	ROLL, PITCH angles below limits [T/F]	Video-camera, inclinometer	Apply safety factor, compare with limits	5
MOB-LIE-1	Indoor/ factory	Stationary obstacle (human body part)	ISO/TS 15066 [30] DGUV FBHM 080 [44]	Quasi-static contact force and pressure	Force and pressure (biofidelity to the specific body part)	Filtering, comparison with limits in [30]	3
MOB-MSD-1	Indoor/outdoor	Stationary obstacle	EN 1525 [45] ISO 3691-4 [35]	Distance from detected obstacle after stop	Distance measuring system, ground markers.	Compare with limit in risk assessment	3
EXO-LIE-1	Exoskeleton	Stationary/moving obstacle (therapist body part)	ISO/TS 15066 [30] DGUV FB HM080 [44]	Transient and quasi-static contact force and pressure	Force and pressure (biofidelity to the specific body part)	Filtering, correction (trans.), comparison with limits in [30]	3
EXO-LRM-1	Segments of an exoskeleton	Swinging limb segments	IEC 80601-2-78 [36]	Joint flexion/extension angles	Electro-goniometer	Filtered values must be >0° and comply with subject limits
EXO-MPA-1	Exoskeleton/ Restrain-type RACA	Limb segments	IEC 80601-2-78 [36]	Force and torque in joint(s), misalignment	Optoeletronic measurement or similar, instrumented limb	Calculate joint forces and torqes and compare with limits
GRI-LIE-1	Any gripper	Stationary/moving obstacle (human body part)	ISO/TS 15066 [30] DGUV FBHM 080 [44]	Transient and quasi-static contact force and pressure	Force and pressure (biofidelity to the specific body part)	Filtering, correction (transient), comparison with thresholds in [30]	3
GRI-LRM-1	Any gripper	Obstacle (human body part)		Minimum gap after closure (pre-defined)	Gauge blocks or similar	Observe object position after (failed) pick	5
MRO-DYS-1	Indoor	Accelerations, stops, payload, floor tilt	EN 1525 [45] ISO 3691-4 [35] ISO 18646-1 [41]	ROLL, PITCH angles below limits [T/F]	Inclinometer	Apply safety factor, compare with limits	5
MRO-LIE-1	Indoor/ factory	Stationary obstacle (human body part)	ISO/TS 15066 [30] DGUV FBHM 080 [44]	Quasi-static contact force and pressure	Force and pressure (biofidelity to the specific body part)	Filtering, comparison with thresholds in [30]	3

¹ The code identifying each protocol has the structure XXX-YYY-Z, in which XXX refers to the device type, YYY to the safety skill and Z to protocol variant (see Table 5). ² Adaptation for the assessment of transient contact according to the methodology in [46].

. In the table, some details are specified, in order to enable the observation of the main similarities and differences. With this perspective, the following considerations arise:

• Protocol applicability conditions are general and valid in different scenarios. The main advantage of basing protocol development on a cross-domain perspective is the possibility of reducing their quantity, still meeting the specific needs of a wide variety of application cases.
• The assessed metrics can be based on measured values or on Boolean variables. This is mainly related to the required testing equipment, as in the case of video cameras, the observation of the test enables analysis only on a threshold-based assessment.
• Regardless of the test metrics, safety skills are validated based on the verification of the compliance with specific thresholds, which are provided by specific standards, or, if not available, determined by the manufacturer. None of the thresholds are proposed by protocol designers, in compliance with the protocol concept and aims.
• To maximize the usability of protocols and, consequently, further shorten the distances between the users and the safety validation, where possible, several kinds of testing equipment and methodologies are suggested.
• All the LIE validation protocols are based on the use of sensing devices with the same basic principle, which is acquiring normal force and pressure. Furthermore, they all can be characterized as “biofidel”, referring to the capability of reproducing the biomechanical behavior of the human body part potentially involved in the contact. This approach represents the state-of-the-art for the assessment of human–robot physical interaction, which is expected to be adopted by the relevant ISO standards. As shown in Section 2.2, this is indeed one of the new aspects introduced in the ISO/DIS 10218-2 [40] and a similar device is also envisaged by the ISO/TR 23482-1 [37].

4. Examples of Protocol Use and Application

In this Section, three application cases of the validation protocols are shown. It is worth mentioning that the process of developing validation protocols involved a clear specification of the use-case and protocol user needs, input from the experience of the protocol developers from previous measurements, and several feedback loops from users within the COVR consortium and from the projects sponsored through the cascade funding mechanism within the overall COVR project framework. The application cases hereafter reported refer to:

• an experimental campaign, belonging to the in-house trials performed by the COVR partners, aimed at obtaining the maximum permissible velocities of a collaborative robot in some areas of the workspace;
• the validation of a mobile robot for a retail environment;
• the validation of a rehabilitation robotic device.

4.1. Admissible Velocities of a Robot Arm in a Shared Workspace

The test procedure described in the protocol ROB-LIE-1 “Test robot arm for collision with movable object (Impact)” was used to perform a series of impact tests in a pre-defined set of positions, in order to obtain an indication of the maximum permissible operational velocities of a Universal Robots UR10 in different areas of its working volume, with reference to specific human–robot contact scenarios. The protocol is aimed at testing the contact force and pressure affecting the operator in an unintended contact scenario with a robot with PFL functions activated, to be then compared with the force and pressure limits provided by ISO TS/15066 [30]. The contact scenario identified in the risk assessment is reproduced to obtain the force and pressure values related to the transient contact phase. The robot program from the real application is executed and the measurement takes place along the position in space where the risk analysis specifies that contact is likely to occur. The robot collides with the sensing equipment, which consists in a biofidelic load-cell sensor and a foil pressure sensor. Data is then post-processed and the peak value is obtained for the transient contact phase (t_c ≤ t ≤ t_c + 0.5 s, with t_c representing the time in which contact occurs).

The tests carried out considered the contact of a test piece with the sternum, the abdominal region and the back of the hand at three different points (see Figure 1a) in the robot workspace. The measurements were executed at different velocities for each position (with a 50 mm/s increment for linear trajectories and 5 deg/s for the point-to-point motions). The following table reports the setup of each of the tests, performed using a GTE CBSF-75 Basic with adjustable spring and impact pad as force sensor. The obtained force values were then compared to the limits in the ISO/TS 15066 [30] to identify the permissible robot velocities for each position with the tested configurations. The results related to the tests performed in CNR-STIIMA laboratories are also reported in

Table 7. Setup details and permissible velocities validated by the application of ROB-LIE-1 protocol.

Contact Point	Body Region	Spring Stiffness [N/mm]	Damping Material [44] [SH]	Permissible Values TR 1		Validated Permissible Velocities
				Force [N]	Pressure [N/cm2]	x (PtP 1) [deg/s]	y (Linear) [mm/s]	z (Linear) [mm/s]
P1	Sternum	25	70	280	240	30	200	unlikely
P2	Abdomen	10	10	220	220	30	200	unlikely
P2	Back Hand	75	70	280	400	20	150	300
P3	Abdomen	10	10	220	220	45	400	unlikely
P3	Back Hand	75	70	280	400	30	300	100

¹ TR: Transient contact; PtP: Point-to-Point.

.

4.2. Stockbot: A Mobile Robot for a Retail Environment

The Stockbot (pal-robotics.com/robots/stockbot, accessed on 15 April 2021) mobile robot from Pal-Robotics (Figure 2a) is a mobile robot intended for the implementation in retail environments for stocking purposes during normal activity in shops. The MOB-MSD-1 “Test Mobile Platform to Maintain a Separation Distance” protocol was used to validate the ability of the robot of maintaining a safety distance from humans and obstacles. The protocol MOB-MSD-1 describes the validation procedure for a mobile robot sharing an environment with humans and objects encountered along its path, aiming at avoiding contact by keeping a safety distance in any condition. This is obtained by setting up a test using a cylindrical object as a dummy, placed vertically on the floor along the robot path. During the test, the robot, with enabled safety functions and after a warm-up phase, moves towards the obstacle and is expected to stop automatically before impacting with the obstacle. The distance between the robot and the obstacle is then acquired (Figure 2b) and compared to a safety distance, which is pre-defined depending on the application requirements and specified in the risk assessment. The test must then be repeated with different-sized obstacles.

In the Stockbot case study, a wide variety of cylindrical dummies were tested, considering the following colors and materials: matt black, bright red, mirror, cardboard, aluminum, transparent plastic, and two different dimensions. They were placed at different heights for testing by using a post with sliding guides. Tests were performed in an operating environment exactly like in real scenarios. The protocol was applied with a main modification: in order to assess the Stockbot advanced functionality of creating an alternative path when an obstacle is encountered, the robot was not expected to stop during the test, but the minimum distance from the obstacle was calculated, post-processing the acquisitions from an HTC Vive tracking system (Figure 2c), which is recognized as a suitable 3D tracking system [47]. Besides the application of the MOB-MSD-1, the robot was validated also considering other hazards (impacts, stability vs. steps or slopes) and the protocol-based test campaign enabled to successfully validate the use of the robot in shared public environments.

4.3. Achilles: A Robotic Device for Ankle Rehabilitation

The protocol ROB-LRE-1 was co-developed and executed in an earlier version with the partners Amsterdam UMC (location VUmc), TU Delft and LUMC (all from NL) to validate the safety of two haptic rehabilitation robots, Achilles and Wristalyzer (Figure 3) from Moog B.V. (www.moognetherlands.nl, accessed on 15 April 2021), used for diagnostics of joint hyper-resistance and quantification of passive stiffness versus reflex torque in neurological patients. They are used for the ankle and wrist joint, respectively, and can apply a torque to the human joint in two different scenarios:

• to counteract a maximum torque applied by the human subject, maintaining a set position to determine the subject’s capabilities;
• to generate a motion of the attached body segments within pre-defined physiologically safe torque limits.

The applied protocol includes two tests to validate (i) that the torques actually applied by the robot match those set in the robot control (“torque mode” test) and (ii) that the pre-set maximum allowed torques are not exceeded (“position mode” test). In the torque mode test, a rod is connected to the robot and then connected to the floor via a non-elastic rope with a force sensor and a spring (Figure 4). The robotic device must be set to apply an increasing torque according to a linear incremental ramp profile; torque values from the robot software are then compared to those obtained by computing the force sensor acquisitions. In the position mode test, a non-elastic rope, to which weights can be added though a pulley system, is attached to the robotic device (Figure 4). By computing the data obtained by a force sensor, the torques generated by the weights and imposed on the robot are obtained and compared to the values acquired by the robotic device. A spring dampens the force peak that may result from placement of the test weights. The safety of Achilles was validated by applying the protocol: in the torque mode test, the reliability of the torques computed by the robotic devices was proven, while in the position mode test it was observed that robot torque application was stopped before reaching the pre-set maximum torque.

5. Conclusions

This paper deals with safety-related testing procedures in human–robot collaboration. With HRC, the authors refer to all those robotic application scenarios, whether industrial or not, in which close human–robot interaction is envisaged during robot operation. The authors claim that the potential of the latest frontiers in HRC, along with the spread of robots in a variety of domains and applications, is currently limited by the difficulties faced when dealing with safety validation of the specific robotic application. This is due to the very articulated regulation and standard framework, combined with the (still) scarce availability of step-by-step testing procedures.

To address this topic, this paper first addresses the standards and European regulation within the context of safety for HRC. Likewise, an overview of all the testing procedures provided in the latest standards, useful for safety verification and validation, is provided. However, based on the experience of the authors with real applications, safety validation issues arising due to the cross-fertilization of different domains pose new challenges, whose solution often requires the consideration of non-domain-specific robotic standards and best practices as a whole. By addressing as a whole the industrial and non-industrial specific issues, analogies are highlighted, providing the foundation for an innovative cross-domain perspective.

Accordingly, a new approach is proposed, based on the definition of a selected set of safety skills that are applicable to a wide variety of robotic applications, regardless of the implementation domain. The cross-domain safety skill concept has been translated into testing “protocols”, each defined by the robotic device implemented in the task and the validated safety skill. Protocols build on standards and best practices and are aimed at providing robotic users, either non-expert or expert but dealing with new HRC challenges, with guidelines for the identifications of relevant standards and the testing procedures to fulfill their requirements. The first set of protocols was published and is freely available within the COVR Toolkit [25], and it is constantly updated based on the feedback from users and experts. As examples of protocol use, some real cases are reported in the paper, belonging to different application domains.

The proposed methodology aims at guiding new users in their approach to safety in HRI, the verification procedures and the relevant standards and at providing a different perspective to experts dealing with HRC in different fields.