Challenges in Virtual Reality Training for CBRN Events

Abstract

The contemporary geopolitical environment and strategic uncertainty shaped by asymmetric and hybrid threats urge the future development of hands-on training in realistic environments. Training in immersive, virtual environments is a promising approach. Immersive training can support training for contexts that are otherwise hard to access, dangerous, or have high costs. This paper discusses the challenges for virtual reality training in the CBRN (chemical, biological, radioactive, nuclear) domain. Based on initial considerations and a literature review, we conducted a survey and three workshops to gather requirements for CBRN training in virtual environments. We structured the gathered insights into four overarching themes—the future of CBRN training, ethical and safety requirements, evaluation and feedback, and tangible objects and tools. We provide insights on these four themes and discuss recommendations.

1. Introduction

The current political landscape and the presence of unconventional threats posed by both government and non-government actors highlight the pressing need for military personnel or civilian first responders to perform practical training in realistic settings. However, factors such as available infrastructure, costs for training, and physical risks for trainees can limit the frequency and extent of training sessions. As preparing for potential threats is essential for enhancing readiness and security, a limit in training frequency can hinder the desired level of preparedness of civilian and military first responder organisations.

Virtual reality (VR)-based training solutions are a promising technology to reduce training costs and decrease the risk of personal and support controlled and replicable training. VR is described by the Encyclopaedia Britannica [1], as ”the use of computer modelling and simulation that enables a person to interact with an artificial three-dimensional (3-D) visual or other sensory environment.” The advancements in 3D modelling, rendering, and game-engine technology as well as improvements in VR hardware—the “second wave of VR” [2]—provide promising benefits to improve and extend current training curricula. Therefore, in recent years, VR training solutions have been used in various contexts such as assembly training [3], medicine [4], emergency and safety training [5], social skills training [6], military training [7,8], and many other areas.

For this work, especially interesting are VR training approaches applied for first responders, e.g., police training [9,10,11], medical first responder training [12], firefighters [13], disaster risk reduction and civil protection [14,15], and training for chemical, biological, radioactive, and nuclear (CBRN) events, e.g., [16,17,18].

In prior work [19], we have presented challenges for the future development of VR training for CBRN responders. VR for CBRN training offers the potential to enhance and accelerate learning and can serve as a valuable supplement to existing disaster preparedness and CBRN training curricula and programs [20]. It supports the development of various skills, such as technical expertise, operational response, improvements in the command structure, and effective collaboration among multiple teams in dynamic scenarios. The use of VR is particularly promising because it allows for repeated training sessions without concerns about material wear or infrastructure interruptions. Moreover, VR can eliminate the hazards associated with real-world CBRN (live agent) training, such as contamination risks. Additionally, VR training facilitates comprehensive after-action review, enabling trainers to (semi-)automatically evaluate performance and provide detailed feedback [21]. VR also offers the possibility to explore novel collaborative and remote training methods where trainees and trainers can connect virtually from different locations. This is especially interesting for multinational collaborations and collaborations between different entities (e.g., CBRN specialists can train together with medical responders, police, firefighters, etc.).

Although VR training for CBRN responders is a promising direction, there still seems to be a gap in knowledge regarding the requirements and needs of trainers, trainees, and other stakeholders for the successful implementation of VR CBRN training.

This work presents the results of an in-depth requirements analysis for VR CBRN training. We provide insights from the workshops and observations conducted in the requirements phase. We contribute towards a more in-depth understanding of stakeholder needs to implement and perform CBRN training in virtual environments.

The remainder of this paper is structured as follows: In Section 2, we discuss related work on non-immersive and immersive virtual training solutions. In Section 3, we describe the multi-methods approach which consists of an observation, an online survey, and workshops. Section 4 presents the results of the multi-methods approach structured along the topics gathered from a literature review. Section 4 provides recommendations for VR CBRN training. Section 5 addresses the limitations and provides an outlook for future work. Lastly, Section 6 provides a summary and conclusions.

2. Related Work

In related work, there are many approaches that use VR technology to train military personnel [18]. While the majority of VR simulators are designed for the Air Force and Navy [18], there is a scarcity of solutions catering specifically to CBRN specialists [16,17,22]. We refer to Kako et al. [23] for a systematic review of disaster education and training for CBRN.

When discussing related work, we differentiate between non-immersive (Section 2.1) or fully immersive (Section 2.2) virtual training for CBRN responders.

2.1. Non-Immersive CBRN Training

A non-immersive training tool, primarily designed for civilian usage, is the Advanced Disaster Management Simulator (ADMS) [24]. The ADMS offers a wide range of training possibilities and is targeted towards various user groups such as police, firefighters, medical responders, disaster management personnel, and airport staff. While the ADMS includes flight and vehicle/driving simulation features, its most relevant application in the context of CBRN training lies in emergency management and simulation, specifically, command, control, coordination, and communication in critical disaster situations.

Another example of a non-immersive application is the Virtual Battle Space (VBS) simulation [25]. Although VBS is not directly aimed at CBRN training, VBS emphasises commander training, communication training and mission planning, as well as communication, which are also important skills for reaction to CBRN incidents. With a similar intent, the Enhanced Dynamic Geo-Social Environment (EDGE) [26] uses a desktop-based gaming platform to train first responders’ communications and coordination with a strong focus on cross-disciplinary operations. In fact, the tool allows trainees to play different roles in emergency and military organisations, learning each other’s procedures. Among the different threat options available (e.g., bomb threats, arson, etc.) CBRN incidents are also available.

Focusing on clinical training, Heinrichs et al. [27] proposed a simulator for training in acute-care medicine. The simulator included three scenarios, one of which was designed specifically to train emergency department teams and hospital staff in managing mass casualties following CBRN incidents.

2.2. Immersive CBRN Training

Within the realm of CBRN training research, various immersive training platforms have been explored. One notable example is XVR Simulation [28], a commercial platform that enables first responder organisations to train across different aspects such as operational skills, team communication, strategic decision-making, and coordination. XVR Simulation offers a range of modalities, including classroom use, 2D screens, and fully immersive VR environments.

Another significant effort in the field of immersive VR training for CBRN responders comes from the LINKS Foundation and the Technical University of Turin. Their projects, VR4CBRN and VR4CBRN2, focused on developing and investigating several aspects, including a virtual instructor agent, remote multi-user support, 3D models of CBRN equipment, and a user interface for training assessment by the trainer. Lorenzis et al. provide a detailed account of the immersive CBRN VR training platform developed within these projects [29].

With a stronger focus on the requirements for immersive VR in CBRN, Mossel et al. offer an extensive overview of the state-of-the-art VR technologies for CBRN responder training [18]. Their work also includes a requirement analysis conducted in collaboration with CBRN stakeholders, which identified key elements for successful VR training in this domain. These requirements encompass full immersion in the training scenario, 3D object interaction, natural walking, and simulation of stress and exhaustion, as well as support for collaborative training. Building on these findings, Göllner et al. [17] present a novel immersive multi-user VR training system for CBRN that integrates additional requirements and incorporates a gas mask for enhanced realism and immersion. The requirements identified in [17] include (1) communication (e.g., team communication, radio communication to the commander, etc.), (2) manipulation (e.g., transportation of weapons and equipment, map consulting), (3) specific manipulations (e.g., reading measurement results, use of binoculars, sampling contamination on a flat surface, packaging sample, etc.), (4) movement (e.g., going, running, lying on the ground, etc.), (5) specific movement (mount/dismount vehicle), and (6) customisation of scenario content and parameters (e.g., chemical hazard agents, weather - wind and visibility, etc.). For the complete list, refer to p.10, Table 1 in [17]. The system and the requirements proposed by Göllner et al. underwent a positive evaluation involving 13 participants from the National Defence Academy and Competence Center NBC Defence (Austria), receiving higher ratings in terms of acceptance, applicability, and potential integration into current CBRN training frameworks.

In the study conducted by Schonauer et al. [30], a novel approach combining virtual reality and augmented reality (AR) was employed to enhance CBRN preparedness training. The researchers utilised VR technology to recreate a virtual training environment, while AR was employed to seamlessly integrate real tools and equipment into the simulation. Building on the requirements set in [17,18], this system not only accommodates multiple trainees simultaneously but also incorporates full body tracking and enables locomotion through real walking.

Among immersive VR platforms, Lamberti et al. [31] describe a VR system for CBRN training which was designed together with the CBRN experts from Terzo Stormo – Aeronautica Militare di Villafranca di Verona. The system was evaluated by 30 CBRN experts and received positive feedback in terms of usability and effectiveness. Similarly, in Altan et al. [32], 16 CBRNe experts evaluated three immersive serious games for CBRNe training in terms of usability, presence, and technology acceptance. The results showed that users positively rated the VR games on all three measures. Additionally, qualitative feedback revealed that participants would like to use VR games in their training frameworks.

While the works presented so far are mainly designed based on requirements collected from military personnel, several works have also been conducted in the context of CBRN in disaster risk reduction and civil protection. In this area, the works of [14,15] (conducted under the Prodige project) are particularly significant. In [14,15], an immersive VR training platform for civil protection training is presented. The system was tested with different civilian organisations (local police, firefighters, and first responders) in large field trials. In particular, both works provided an important contribution by collecting training requirements from civilian operators which were missing in the studies mentioned so far.

Although related work has already addressed CBRN training in VR, there is still a research gap towards the requirements of stakeholders (trainees, trainers, decision makers, etc.) for successfully developing, deploying, and integrating VR CBRN training in the future. Thus, we tackle this research gap by performing an in-depth requirements analysis which is reported in the remainder of this paper.

3. Method

We used a multi-methods approach to gather requirements for CBRN training. Thus, we conducted the following activities: (1) an observation, (2) an online survey, and (3) workshops with stakeholders.

The following topics of interest were derived from the screening of literature, discussions, and brainstorming sessions. These areas were used to shape questions which were used as the basis for data analysis:

• Future of CBRN training: This area includes questions regarding the general usage of VR training, as well as the advantages and disadvantages of VR training. The aim is to understand users’ feelings about VR, and what expectations they have of this technology.
• Evaluation and feedback: An important aspect and a strong benefit of digital training, especially VR training, is the possibility to provide in-depth feedback to trainers and trainees. Replaying actions that trainees have taken, showing trajectories, recording and showing communication structures, etc., can provide useful feedback for learning and transfer of learning. The goal is to identify how feedback is currently given to trainees and which elements of the evaluation can be replicated in the VR system.
• Ethical and safety requirements of a CBRN training: ethical aspects, e.g., the well-being of trainees are important to consider, especially in mandatory training. Since exposure to VR can be challenging for some, it is important to identify which regulations are already in place to safeguard the trainees and reproduce them also in the VR system. Moreover, novel ethical issues can emerge from the use of VR, such as the exposure to visually realistic casualties or cybersickness symptoms that can manifest while using VR.
• Tangible objects/tools in VR: When training goes beyond providing information or learning processes but requires learning special skills (e.g., handling machinery, taking probes, etc.), the use of real objects, real tools and tangible materials is a promising direction. Here, it is needed to identify which instruments are crucial for successful CBRN training, and if/how these instruments and tools should be replicated in VR.

3.1. Observation of CBRN Training

Observations were conducted at the training facilities of the Joint NBC Defence School, in Rieti (Italy). The overall goal was to gather insight into the current training modalities (e.g., scenarios, tools, equipment used, etc.) but also into logistics (venues, spaces, etc.).

At the end of the training centre visit, questions regarding the future of CBRN training, evaluation and feedback, ethical and safety requirements, and the integration of tangible objects/tools in VR were discussed with trainers and staff at the training centre in an open discussion. The questions were derived from an analysis of related work conducted before the observation, as described in the prior section.

Participants from the Italian Military Airforce attended the discussion and brought their relevant experience, having already developed 6 CBRN VR simulations (which in April 2022 started to be used in their CBRN courses) in the framework of the VR4CBRN initiative, funded by the Italian Ministry of Defense.

3.2. Online Survey

As part of the process to collect user requirements, a survey was created and shared among relevant partners in the CBRN domain. The survey aimed to collect views, opinions, and information from current CBRN training practices and, thus, to gain an overview of current practice and what unfulfilled needs there are. The results of the survey were crucial to understanding the current perspective among CBRN practitioners and identifying their perception of novel technologies such as virtual reality (VR), augmented reality (AR), and mixed reality (MR). The results further served as a useful basis to prepare the content of the workshop.

The survey received 15 full and 11 incomplete responses for a total of 26 responses in July and August 2022. Of these, 13 identified as males and 2 as females and 11 preferred not to answer. Regarding the organisation of origin, only 22 responses were received. Of these, 7 people belonged to a military organisation, 4 belonged to a private company, 2 were in university education, 4 belonged to the police force, 1 belonged to a customs office, and 4 did not answer. Regarding the level of education of the participants, out of 22 responses, 7 people had a secondary degree, 1 had a bachelor’s degree, 12 people had a master’s degree, and 2 people held a doctorate. Concerning the experience with CBRN training, 11 respondents had conducted CBRN exercises more than 10 times, 5 respondents between 1 and 5 times, and 3 respondents between 5 and 10 times. Lastly, 1 respondent had attended a single training exercise and only 1 had never attended one.

When asked about their experience with VR, 12 respondents reported no experience with this technology, 9 reported having used it before, and 5 did not provide an answer. Those who had experienced VR before were asked how often they had used it, with 6 respondents reporting having used VR between 1 and 10 times, 2 between 11 and 50 times, and 1 only once.

3.3. Workshops

Workshops were conducted with members of the NATO HFM-NMSG-354 panel, the NBC School, and students of the International Master CBRNe courses of the University of Rome Tor Vergata. The goal of the workshops was to gather valuable inputs, in terms of wants and needs, for the development of VR-supported CBRN training. During the workshops, participants had the opportunity to reflect on VR training scenarios and define requirements for the training systems.

Overall, 30 people participated in the workshops. At the NATO HFM-NMSG-354 panel workshop, 12 people (9 male, 3 female) participated. At the NATO HFM-NMSG-354 panel, participants had a military background or a background in software development. At the NBC School workshop, 5 people (all male) participated. All participants were CBRN specialists with a military background. Finally, at the last workshop, with students and alumni from the University of Rome Tor Vergata, 13 people participated (10 male, 3 female).

The first workshop was focused on a discussion of the legal, ethical, diversity, and safety requirements of CBRN training. Members of the NATO HFM-NMSG-354 panel took part in the event to discuss the future development and implementation of XR for CBRN training. The brainstorming was carried out using the World Café method. World Café is a type of brainstorming which encourages the sharing of ideas in a relaxed, informal, and creative atmosphere. Two groups (N = 6) were set up to discuss ideas and opinions regarding the legal, ethical, diversity, and safety requirements of CBRN training.

The second workshop was carried out at the NBC School in Rieti (Italy). The workshop was divided into three core sessions. In the first session, participants took part in a World Café brainstorming. Two groups were created to discuss and share an opinion on (1) the legal, ethical, diversity, and safety requirements of CBRN training, and (2) CBRN training: current practices and how to improve it. The second session engaged participants in a gamestorming activity [33]. In the “make-a-world” activity, participants should create an ideal version of “the perfect CBRN VR training”. Two different methods were used to carry out this session: (1) a multi-user VR co-design tool developed by AIT [34]; (2) building blocks and art supplies. Participants were divided into two groups and assigned to one of the methods. After 45 min, the groups exchanged places so that everybody could try a different approach to the game. One of the main objectives of this session is to allow all participants to familiarise themselves with and try out a VR application. This allows users to better understand the current possibility of the technology and provide better contributions during the last session of the workshop. The last session of the workshop was a wants and needs analysis. During this part, moderators and participants discussed the VR training tool and which elements should have been used (needs) and which elements would be nice to have but are not imperatives (wants). Each participant is encouraged to discuss how they feel about each element and to list the pros and cons. Lastly, every participant is forced to choose the three features from the wants and needs list that the participant thinks will make the system usable for CBRN training.

The third workshop was carried out online with students and alumni of the International Master Courses in Protection against CBRNe events of the University of Tor Vergata (UNITOV). Eight participants took part in this workshop. The participants belonged to different civilian organisations such as first responders and civil protection from different European countries. A World Café and a wants and needs session were performed as the online setting limited the possibility of performing a gamestorming exercise. The methodology and the content of the World Café and wants and needs sessions were the same as those already described for the earlier workshops.

3.4. Data Analysis

Data was gathered through an observation (see Section 3.1), an online survey (see Section 3.2), and three workshops (see Section 3.3). During the observation and the workshops, researchers took notes and pictures. The data from the online survey was exported as a CSV file.

Subsequently, two researchers (the first and second author of this paper) analysed the data and structured/clustered the data along the topics of interest: (1) future of CBRN training, (2) evaluation and feedback, (3) ethical and safety requirements of a CBRN training, and (4) tangible objects/tools in VR, which were identified through a review of related work (deductive category application, (cf. [35], p. 5). Initially, both researchers independently reviewed the data and assigned the gathered data to the identified categories following [35]. Subsequently, subcategories for each category were identified (e.g., the category identified from the literature “future of VR training” developed the subcategory “reasons against usage”). After each reviewer has coded the data independently, both researchers met and compared the analysis and the developed subcategories. Finally, the categories and data were discussed and merged into a common set of subcategories. Finally, an (internal) research report was produced that summarised the findings which provides the basis for this article.

4. Results

In this section, the results from the three activities, observation, online survey, and workshops, are presented.

4.1. Observation

The observation of the training facility allowed the gathering of a first-hand understanding of how (real-world) CBRN training is developed/implemented, assessing key users’ needs and requirements, understanding which requirements are suitable for virtualisation, and reflecting on how this can be achieved. The observation provided a great opportunity to reflect on which aspects of current training should be replicated with VR, and what elements can VR support in making them more immersive and/or more realistic. It emerged that VR training could be targeted for two conditions: (1) Training simple operations that can be suitable for many actors (e.g., firefighters, police, military) with a possible focus on awareness training as the main takeaway from the exercise. Awareness training means raising awareness to the general handling of CBRN threats and preparing people for (standard) responses to CBRN events (identifying a threat, identifying dangerous situations, using protective equipment, decontamination procedures, etc.). (2) Training complex operations that cannot be trained or are not be trained as often as needed due to limited access to real critical infrastructure, the need to conduct large-scale exercises with many actors, environments that are not built yet, etc. One example could be a poison gas attack in a subway station with several casualties.

Procedures of sampling, analysis, documentation, and decontamination should be included in both VR conditions as they are part of the core practices that every CBRN trainee must learn.

4.2. Survey Results

4.2.1. Future of CBRN Training

To understand how participants imagine the future of CBRN, the survey asked which threats participants believe CBRN training should focus on in the future.

Chemical threats are considered to be the most important ones, followed closely by radiological threats and environmental contamination by CBRN release. Biological threats (of intentional origin) are considered somewhat important, while nuclear threats and nuclear outbreaks (of unintentional origin) are only of limited importance for the participants in the survey.

Having identified which future threats should be addressed, participants were asked to identify which CBRN areas should be included in a VR training system. The majority of respondents agreed on including physical protection (including individual protection and collective protection), hazard management (including decontamination), and detection, identification, and monitoring. At a lower respondent rate, but still considered somewhat important, are the medical countermeasures and support and information management (including warning and reporting) areas.

Lastly, participants were asked to imagine how VR could be integrated into future CBRN training. According to respondents, VR could be useful to practice skills during basic training exercises and to refresh the knowledge of already experienced CBRN specialists without the need for using operational resources.

VR as a form of basic training can be one of the first steps to putting theory into practice and training basic standard operating procedures before conducting full-scale exercises or responding to real CBRN emergencies. VR training could be used at the early stage to provide basic knowledge about threats and to teach the most important operational procedures in case of CBRN threats. VR training could also be used periodically to refresh knowledge, especially for re-certification.

VR Usage in CBRN

We asked participants how VR, AR, and MR might be useful in the CBRN domain in the future. The majority of the respondents recognised the (potential) positive impact of VR technologies, especially in replicating dangerous scenarios and providing ’immersive’ situational awareness training that cannot be (easily) replicated in a ’real world’ environment. VR would help to visualise/simulate a realistic environment for training purposes and could offer training on risks and threats in a safe training environment, e.g., working with firearms, explosives, dangerous dogs, fire, and other risks. Furthermore, VR could provide a basic level of training which could provide solid preparation before going into practical training. However, in terms of basic skill training, VR will not replace real training and a real environment.

Participants also highlighted the possibility of reducing the cost of the training. On the one hand, it would be a cheaper alternative to large-scale live exercises. On the other hand, it may save costs on training, and thus, allow more regular training to be carried out.

Only two respondents had a negative opinion regarding VR technology. Noteworthy is the comment that—according to one participant—VR and MR cannot effectively reproduce the same stress as an exercise in a live-agent environment.

Objectives and Limitations of Current CBRN Training

To better understand the current practices in CBRN training, survey participants were asked what the main objectives of their training are. Training skills and procedures are considered to be the main objectives of a CBRN exercise, followed by teamwork and communication.

Participants were also asked to address the current limitations of CBRN training. Half of the respondents indicated financial aspects as the biggest drawback of current CBRN training. Additionally, logistics and realism of training are two other factors that limit the training practice. Seven answers indicated time as a limiting factor for training.

Having identified the reasons that might limit CBRN training, participants were asked to describe which situations and scenarios are difficult to train with current tools. The two areas where respondents feel improvement is necessary are radiological and nuclear scenarios and simulation of contaminated areas. Respondents mentioned the difficulty of simulating nuclear scenarios due to the risks involved and the legal limitations that regulate the use of real agents to simulate contaminated areas.

Reasons against Usage

Lastly, participants were asked what are possible reasons for not using a VR system for CBRN training. Respondents consider the potentially high price of the system as the main drawback of using a VR system. Some mentioned how VR training is not comparable to the real training experience and the possible limitation of such a system in terms of training options as VR cannot provide all aspects of training.

Furthermore, respondents consider the realism of training as an important component of CBRN training. Respondents identified the potential drawback that the absence of personal protective equipment (PPE) will reduce the knowledge of working with PPE and understanding and appreciating the limits on mobility in PPE. In particular, understanding and experience with PPE are crucial tools trainees have to learn in the CBRN field.

4.2.2. Evaluation and Feedback

An important aspect of training is the evaluation phase. Participants were asked how current training progress is evaluated and documented in their organisation. The received answers show that training is assessed through continuous evaluation of the trainees and by keeping a record of every development. A few respondents mentioned the use of cameras and forms to document the progress. Unfortunately, none of the answers provided a detailed explanation of the methods currently in use.

To further understand how evaluation is carried out, participants were asked which evaluation measures are currently used. Five elements emerged as recurrent in evaluation practices: (1) user feedback, (2) delegate feedback, (3) practitioner forums, (4) collaborative projects/programs, and (5) international working groups. Other responses mentioned written and practical evaluation as another form of evaluation which combines quantitative and qualitative methods.

A crucial element to characterise successful training is feedback. Participants in the survey were asked to list what type of feedback is given during and after the training. During training, verbal feedback is usually given to correct “on the spot” wrong behaviour. Immediate verbal feedback is used to guide and develop the learning process. After training, trainers focus their feedback on the learning objectives and provide mostly verbal feedback to trainees. The recap of learning objectives followed by a performance evaluation of the trainees as a group is an important debriefing aspect. Sometimes also written tests are used to evaluate the training progress and trainee performance.

4.3. Workshops

4.3.1. Future of CBRN Training

Future Training

Participants indicated that the current objectives of CBRN training are highly dependent on the type of exercise. This can be summed up in three categories:

• Tabletop exercise to train the ability to understand and properly manage a specific scenario as the details unfold;
• Live agent and drills are specific exercises/activities such as detection and decontamination;
• Full-scale exercises enable a complete view of the emergency, including all operators and response/recovery procedures.

Additionally, participants mentioned that CBRN training also has a strong focus on the knowledge of tools and equipment that should be employed during a CBRN incident; see Section 4.3.4.

According to the participants in the workshops, a VR system would bring with it several benefits and advantages compared to current practices. Training scenarios could be easily replicated in VR and new scenarios can be easily added to the system. Furthermore, complex scenarios, such as a nuclear explosion, can be addressed more realistically in VR compared to real-life training. Participants also suggested a system that allows trainers and trainees to join and conduct exercises remotely. According to participants, the possibility of collaborating over distance will increase interoperability between CBRN specialists and strengthen the relations among the CBRN institutions.

Participants also raise the need for a stronger focus on communication. From their point of view, CBRN training should teach how to effectively communicate among team members and especially how to establish and maintain interagency communication during CBRN threats. Participants suggested translating verbal communications among the trainees into text so that trainers can evaluate the effectiveness of the communication, which is currently not possible in standard training. Moreover, some participants would like to receive better training on how to deal with the press, the public, and other agencies that may receive information over social media, since this aspect is currently not tackled but is an emerging issue in current society.

Another advantage mentioned by the participants is the opportunity to control and modify several variables of the training. Elements such as weather and wind direction should be under the direct control of the trainer and changes should be possible also in the simulation. Similarly, it should be possible to simulate equipment malfunctioning to force trainees into communicating more often with partners.

Participants imagined future CBRN training having a stronger focus on diversity and inclusion. This would allow taking into account the diversity of the people involved in emergencies. CBRN threats should be understood not only from an operational perspective but should also bear in mind societal differences among the population affected by these threats.

As a further benefit, monitoring and evaluation of trainees could be conducted more objectively since the systems allow trainers to replay and analyse in detail every action performed during the exercise; see Section 4.3.3.

Although participants were overall quite optimistic about the opportunities; they also recognised that some limitations may occur, especially when it comes to the physical sampling of soil and liquid or the haptic feeling of materials.

Player Interaction

Since most of the work of a CBRN specialist involves manual operations (i.e., using equipment and handling samples), it is necessary for them to be able to conduct the virtual training without the use of controllers that could reduce the realism of the interaction. Thus, hand tracking should be implemented to allow for free-hand interaction with the virtual environment and the objects.

Role Play

In terms of roleplay, three aspects need to be taken into account: (1) interaction with virtual agents, (2) interaction with teammates, and (3) interaction with the command-and-control headquarters. Interaction with virtual agents is mostly relevant when evacuating civilians (and sometimes for enemy contact). Here, basic interaction mechanisms are needed (e.g., telling people to leave). Regarding interactions with virtual agents, one can rely here on a standard from agent-based simulations and game development.

According to participants, each person involved in the training should be able to play the same role they have in real life. However, it should also be possible, especially for the incident commander, to play different roles to experience all the different points of view during a CBRN incident. For interacting with teammates during performing the scenarios, it is important that real-time communication is possible. When communicating with the headquarters we envision that simulated communication, e.g., with bots, will be an important aspect in the future. However, at the current stage, participants argued for using a “Wizard of Oz” approach, as there are possibly too many technical difficulties and risks for the simulation of this part.

Virtual agents should be able to interact with the trainees and behave as close as possible to humans. They should display realistic psychological traits and the actions they perform should also be realistic. According to some participants, the realism of virtual agents should be used to assess decision-making skills when trainees are interacting with the population. To increase the diversity of the VR training, participants suggested including more realistic psychological behaviour of the virtual agents to display different and more realistic reactions to the events that are happening around them. Additionally, participants noted that issues might emerge during first aid situations when confronted with people from different cultures. For some participants, CBRN training should brief trainees on intercultural communication.

4.3.2. Ethics and Safety

Ethics and Diversity

From an ethical point of view, a VR training system should refrain from creating a dramatisation of events, such as an excessive representation of injured people or the reactions of virtual agents (e.g., exaggerated screaming from the injured people, too intense renditions of pain, etc.). Given the potential immersive capabilities of a VR system, the level of stress during the training should be monitored or limited to avoid triggering post-traumatic stress disorder (PTSD) symptoms or overwhelming feelings that could compromise the success of the training exercise.

Concerning diversity, participants in the discussion agreed that a VR system should take into account the ergonomics aspects of the training equipment such as a headset, controllers, and CBRN operative tools. In particular, the size of the equipment should be suitable for all body types. Another point raised during the discussion concerns avatars. The system should be able to have customisable avatar representations in terms of gender and skin colour. For all the participants, avatar representation is quite important and should be considered when designing the system.

Although participants reported that no gender-related issues are present in their current training, participants recognised that diversity issues may arise from the use of specific equipment. Participants mentioned that current CBRN equipment is designed mainly for males and does not take into account the differences between the female body. In particular, the gas mask can be difficult to fit if the person has long hair or if the head size is too small.

Safety

Regarding safety issues, participants do not expect any direct physical concern from the use of a VR training system. In the context of military training, a doctor is always available during training and should also be present during a VR exercise. On the topic of psychological safety, workshop participants argued for a realistic representation of casualties. However, participants suggested that the system should have different levels of realism to gradually expose trainees to more realistic scenarios.

It is important that a VR training system should account for possible health issues that might arise. For example, cybersickness should be mitigated and instructions should be given to trainees on how to reduce it. Other general remarks were expressed regarding general protection from obstacles and external objects when wearing a VR headset. From a military perspective, the system should consider the current regulations concerning the equipment weight and the lifting limits imposed.

Overall, participants agreed that a training system should be designed by considering the diversity of the modern world while keeping trainees safe from the negative consequences of overexposure to stressful events.

Data Protection and Classification

A VR training system should satisfy all the civilian and military requirements in terms of data protection and user privacy. In addition, the military environment requires further regulations regarding access to training data. According to the participants, sharing training data and CBRN procedures is possible as long as anonymity is ensured during and after the data collection. However, some participants stressed the importance of sharing only certain aspects of training procedures (e.g., scenarios) while keeping more sensitive security issues, such as type of equipment and facilities, classified. Furthermore, training progress and evaluation data could reveal vulnerabilities of organisations and, thus, might need classification.

Some trainers also pointed out that if biosignals are collected during the training, these should not be shared with the trainee. According to the trainers, sharing estimations of stress or objective measurements of physical strain might compromise the performance of the trainee in the future.

4.3.3. Evaluation and Feedback

The current evaluation of trainees’ knowledge is achieved by assessing theoretical and practical aspects. The theoretical knowledge is evaluated using a multiple-choice questionnaire and trainees receive a grade which determines the success or failure of the training. Regarding practical knowledge, the evaluation is intended to demonstrate the ability of trainees to carry out the tasks learned by the end of the training period. Different from the assessment of theoretical knowledge, this evaluation is not graded. This evaluation focuses on five aspects: detection, identification, monitoring, personal protection, and hazard management.

To support current training practice, it is important for a new training system to provide two modes—training mode and evaluation mode.

• In training mode, the focus should be put on guidance and immersive evaluation features as the instructor will interrupt the training to provide feedback during and at the end of the exercise. During the training, the trainer should be able to pause the training and provide feedback to the trainees when mistakes are made. If necessary, the trainer should be able to rewind the simulation to a previous state before the error was made. Furthermore, automated feedback, e.g., visualisations of containment, count of errors, etc., or visual guidance for certain procedures can and should be available in training mode.
• Differently, in an evaluation, the trainer will not stop the exercise and only afterwards will they provide comments together with a final score. Therefore, in evaluation mode, the focus should be put on scoring and a detailed after-action review. Here, no help should be provided, but the focus is on documenting and scoring of the performance of the trainees. For example, the system can perform automatic scoring of certain tasks, but this should not compromise the evaluation provided by the trainer.

Lastly, it should be possible for trainers to modify the simulated environment while the training is in progress.

VR Scenarios

VR training should target two conditions:

• Training at a basic level, i.e., training simple operations that can be suitable for both military and civilian operators, with a possible focus on awareness training.
• Training at a specialist level, i.e., complex operations/scenarios that could not otherwise be realistically reproduced.

The VR scenarios should be as realistic as possible with the possibility of having realistic injuries and wounds depending on the type of CBRN exposure. Additionally, it should be possible to select different levels of difficulty so that the exposure to a scenario could be gradual and trainees have enough time to learn the basic procedure before tackling very realistic scenarios.

Participants reported that if VR training is highly realistic it might be more efficient to make them shorter than full-scale exercises (which can last up to 5 h) to account for the higher level of stress placed on the operators, who have to deal also with the higher mental load of the VR interaction. It is important that trainees can self-assess their own needs in training and work up to the competencies and confidence to respond to all aspects of an emergency.

Interface for Trainers

The users in the workshops highlighted the need for an interface where the trainer can keep control of what is happening in the virtual environment. The interface should allow for a complete recording of the training session with the possibility of inserting tags during the exercise to remember when mistakes were made or moments the trainer wishes to replay when the training session is finished. The interface imagined by the users can integrate standard data sheets for the communication of strategic information which is commonly used in standard training to report which activities have been carried out so far and what were their results. Additionally, the interface should allow the trainers to have control over variables such as the weather so that they could modify them during the exercise.

The feedback phase of CBRN training is currently divided between training and evaluations. According to the participants in the workshop, it is important that the VR training system maintains this distinction and allows the trainer to choose between one or the other mode.

Feedback to the Trainees

A trainer should always be available to provide feedback and assistance to the trainees. This should be achieved by communicating directly with the trainees through the VR system or by interrupting the simulation. It should be possible to record the entire training exercise so that the trainer can review the actions and provide detailed feedback at the end of the training. The VR system should provide the trainer with quantitative data regarding the execution of the task (number of errors, time to finish it, etc.). However, the feedback to trainees should take into account the observations from the trainer and not be based only on quantitative data. Furthermore, the system can perform automatic scoring of certain tasks, but this should not compromise the evaluation provided by the trainer.

4.3.4. Tangible Objects and Tools

For the trainees, it is important to hold instruments and become comfortable with their shape and weight. According to the participants in the workshops, for VR training it is not necessary to have real equipment available; a replica of the shape and weight of an instrument will be enough to meet the training goals. Companies that produce the instruments could be involved in the training procedures so that they could supply realistic virtual replicas of their tools. The simulation should then superimpose the instrument on the necessary interface (e.g., buttons, screen) to replicate its functioning.

Moreover, some participants would like to have real objects in the simulation using MR techniques, where the real full functional tool is integrated. However, due to expected development efforts, this should be considered only when it is important for trainees to learn how to use a real object.

Additionally, to add realism to the training, it should be possible to simulate issues with the tools (reading issues, broken instruments, etc.) and failures with the protective equipment. The possibility to update tools’ 3D models coherently with the new versions released seems to also be a relevant aspect. Lastly, according to participants, the system should allow the possibility to tackle the same CBRN threat using different instruments. This should be performed to verify the interoperability of instruments and procedures.

Still, future research is needed to see if and what elements can be made tangible with sufficient effort, and which can still be digital only, without interfering with the purpose of the training.

CBRN VR Mask

When asked about which features are necessary to successfully integrate PPE and especially a VR gas mask in the training, users raised the need for it to be as close as possible to the real experience.

While wearing the mask, a narrow field of view in the VR simulation should be present to simulate the same perspective one would have when wearing a real gas mask. Additionally, the mask should allow for radio communication between the trainees and the trainers and the voice should be distorted and reproduce a similar sound quality when wearing a gas mask. According to the participants, these features are important to increase the realism of the simulation and provide trainees with a realistic feeling of the mask.

5. Recommendations for CBRN Training in VR

This paper sought to support the identification of core functions, logistics, and structure and interface needs for the development of CBRN training in VR.

5.1. Future of CBRN Training

A VR system can provide several benefits and advantages compared to current CBRN training practices: (1) basic training scenarios can be easily replicated in VR, (2) new scenarios can be easily added to the system, and (3) complex scenarios can be addressed more realistically compared to real-life training. Additionally, a VR system can allow collaboration over distance, which will increase interoperability between CBRN specialists and strengthen the relations among the CBRN institutions. This is especially relevant on a European level to strengthen collaboration between European entities that deal with CBRN threats.

We identified that a VR system should take ergonomics aspects of the training equipment, such as a headset, controllers, and CBRN operative tools, into account, as equipment should be suitable for all body types. In terms of core system functions, trainees should be able to conduct the virtual training without the use of controllers, as this could reduce the realism of the interaction. Thus, hand tracking should be implemented to allow free-hand interaction with the virtual environment and objects. Participants envisioned using tangible representations of tools and equipment, however, replicas of real tools would be sufficient.

We documented a need for trainers to keep control of what is happening in the virtual environment. Participants highlighted the importance of three variables that will be controlled by the trainer: weather conditions, incident location, and possible access point to the contaminated area. Similarly, it should be possible to simulate equipment malfunctioning to force trainees to communicate with partners.

Regarding roleplay and other people, (1) interaction with virtual agents, (2) interaction with teammates, and (3) interaction with the command-and-control headquarters are the most relevant points for interaction and communication. Especially for virtual agent design and avatar design, customisable representations in terms of gender, skin colour, etc., should be provided to support diversity in avatars and virtual agents, which was seen as a crucial aspect.

Summary—Future of CBRN Training in VR

• Benefits and advantages of VR training:

  –

  basic training scenarios can be easily replicated in VR;

  –

  new scenarios can be easily added;

  –

  complex scenarios can be addressed more realistically.

• Trainers should be able to control the virtual environment:

  –

  weather conditions;

  –

  incident location;

  –

  access points to the contaminated area.

• Roleplay and most relevant points for interaction and communication:

  –

  interaction with virtual agents;

  –

  interaction with teammates;

  –

  interaction with the command-and-control headquarters.

• A VR system can enable collaboration over a distance.

5.2. Ethics and Safety

From an ethical point of view, a VR training system should not create an over-dramatisation of events to avoid triggering trauma or overwhelming feelings that could compromise the success of the training exercise. Furthermore, the level of stress should be monitored to detect possible stress-related negative effects.

Nevertheless, participants argued, in general, for a realistic representation of casualties; the system should have different levels of difficulty and/or realism to gradually expose trainees to more realistic scenarios.

Moreover, general safety concerns for immersive applications including possible health issues (e.g., cybersickness), as well as general protection from obstacles and external objects, should be taken into account.

An additional element to consider in terms of data protection, classification, and safety is the vulnerability exposed by evaluation/performance data. It is likely that in cases where data shows detailed performance indicators and in particular a lack of performance, the data itself should become classified. Any data that can show a specific vulnerability is sensitive and should be classified because it can reveal to “enemies” potential vulnerabilities to be attacked. This does raise an interesting research question as an automatic evaluation system is powerful and has many advantages but at the same time generates sensitive data and must, thus, be handled carefully. This is not only true in the military but also in the civilian context. For example, acquiring the information that firefighters are lacking in certain skills or expertise could be used for malicious purposes should these data fall into the wrong hands (e.g., a terrorist group).

Summary—Ethics and Safety

• Avoid over-dramatisation of events.
• Monitor possible health issues (e.g., cybersickness).
• Importance of protection from obstacles and external objects.
• Classify vulnerability exposed by evaluation/performance data.

5.3. Evaluation and Feedback

Trainers should have access to an interface allowing them to control the virtual environment, and provide options for recording and analysing the training session. It is important to provide two modes—training mode and evaluation mode.

In training mode, the focus should be put on guidance and the provision of additional didactic features (e.g., visualisations). Trainers should be able to interrupt the training to provide feedback during the exercise if required. Thus, features are needed that allow the rewinding of the simulation to an earlier stage.

In evaluation mode, the focus should be put on scoring and providing data for a detailed after-action review. In the evaluation phase, the system should support an automatic scoring of certain tasks combined with support for qualitative assessment of the trainer.

Summary—Evaluation and Feedback

• Two modes are needed:

  –

  training mode;

  –

  evaluation mode.

• Trainers need an interface to control the virtual environment, and options for recording and analysing the training session.

5.4. Tangible Objects and Tools

Tangible equipment—trainees need to handle devices while performing CBRN operations, the use of replicas during VR training is, therefore, important.

To further increase the level of immersion and realism of the training, a real gas mask that can be integrated into training is seen as beneficial. When wearing a gas mask, a narrow field of view should be present and radio communication between trainers and trainees should reproduce a similar sound quality as when wearing a gas mask during real-life training.

Summary—Tangible Objects and Tools

• Use of hand tracking to allow free-hand interaction with the virtual environment and objects.
• Use of replicas during VR training is important.
• VR gas mask is seen as beneficial.

6. Limitations and Future Work

In this work, we used a multi-method approach by combining observations, an online survey, and workshops to strive towards capturing an accurate picture of the wants and needs for CBRN training in VR. Although we used a multi-method approach and involved a broad range of stakeholders, there are some limitations.

First, CBRN training and CBRN responses are differently organised in each country and also differently organised in each institution (e.g., firefighters, medical responders, military, police, civil protection). We involved stakeholders with different backgrounds and nationalities, however, many participants involved had a military background. Thus, in future work, we will put a stronger focus also on the needs of civilian responders.

A second limitation is that in this work no actual implementation of a VR training system for CBRN responders was evaluated. However, we used immersive prototyping techniques in some workshops to provide a realistic impression of VR and immersive environments. Still, a user evaluation of an actual training system would provide additional insights. Currently, a prototype of a VR CBRN training system is being implemented and will be used to evaluate the overall user acceptance and the design of certain features.

7. Conclusions

In this work, we discussed the challenges of developing harmonised virtual-reality-based CBRN training. By standardising CBRN training on a European level, interoperability between different actors (military and civilian) and European nationalities will be increased. Based on initial considerations and a literature review, we conducted an observation, a survey, and three workshops to gather requirements for CBRN training in virtual environments. We structured the gathered insights along five overarching themes: the future of CBRN training, evaluation and feedback, logistics and organisational aspects, ethical and safety requirements, tangible objects and tools. We provide insights into these five themes and provide recommendations.