Cognitive Implementation of Metaverse Embedded Learning and Training Framework for Drivers in Rolling Stock

Abstract

Public safety is prime concern in rail industry and driver training on hazard perception is crucial. Additionally, a new driver’s skill set determines the productivity and quality of existing driver training methods. Apprentice train drivers are required to complete massive hours under supervision of experienced drivers to attain the required skill sets causing productivity issues. Traditional driver training is paper based, and assessments are individually evaluated without any scientific rigor, resulting in quality challenges. This paper proposes a Metaverse embedded learning and training framework for drivers in rolling stock. The framework includes driver vision analysis by eye tracking and pupil dilation focusing on enhancing the productivity and quality of driver training and hazard detection for drivers in rolling stock. Metaverse embedded training and learning enhances experiential learning with unique benefits. In this paper, a metaverse-based training framework is proposed for train drivers to enhance productivity, quality, and safety aspects through case studies including: (i) driver sightline studies and (ii) vision analysis. The studies developed quantifying driver hazard perceptions and related comprehension rates based on eye tracking and vision studies. In conclusion, the overall savings on cost and time are 95% effective using Metaverse-based training method compared to traditional methods. Stakeholders need to supervise on driver tasks, knowledge retention, damage control due to the occurrence of hazards. The framework substantially reduced hazards to 50% with saving up to 3696 man-hours. The assessment was completely automated to provide real time assessment thus providing 93% more positive results compared to traditional methods.

1. Introduction

The rail industry supplies a major transport medium within Australia, and its drivers are required to have high skill sets as key human capital within the industry. However, driver training is mainly done on-the-job or on trains, which adds to related risks. Drivers require specific training for different environmental factors and situations that may occur during operations [1]. To allow them the opportunity to create the cognitive links within learning environments, an environmental training system must be created so that they may associate the different track conditions—from straightaways to tight and complex curves—with the psychomotor skills and experience required to navigate them within a virtual environment (VE) embedding Metaverse [2,3].

To increase the retention rate and learning efficiency of VR simulations, a full-size, modular mock-up of control surfaces was created with low-cost rapid prototyping options, allowing users to experience the full haptic responses of their actions, increasing their information retention rate and allowing them to be fully immersed within the simulation by ensuring that sight, sound and touch were all simulated accurately [4].

The goal of this manuscript is to create a proven system that allows professions that encounter high-risk environments to have a low-cost portable option to train their operators on how to perform in both standard and emergency operations. In addition, required competency tests can be done at regular intervals to ascertain performance and safe practices in Metaverse-based platform [5]. Traditionally, driver training is part of human capital development. It consists of two major components, which are mainly learning and assessment [6]. However, traditional driver training practices are paper-based and do not adequately cover certain complexities. Moreover, unsafe practices add safety risks to rolling stock stakeholders. These methods are limited in terms of efficiency, quality and safety. The lack of a qualitative assessment framework and requirements for continuous improvement to training, result in economic burdens on stakeholders. Based on the current gaps identified from existing literature this research contributions are as follows:

• Developed a metaverse based driver training framework encapsulating real world training synchronous digital environment with unbiased automated assessments and driver training indices for productivity, quality and safety enhancements.
• Validated the conceptual framework with distinct case studies to evaluate driver vision analysis and driver sightlines studies.
• Developed a mathematical model to validate the training effectiveness using the driver training framework.

1.1. Current Rail Driver Training

The rail industry is primarily a craft industry, relying on primitive instructive practices to organize and schedule daily training operations. Due to the complex tasks involved in the rolling stock, highly skilled training is required by each individual and involves substantial time, high costs and strict assessment measures. Anecdotally, training offered as basic paper-based and assessment frameworks are paper trails. The current method of driver training involves hours of PowerPoint lectures from experienced trainers, which, while sufficient, are slow and limited due to the 2D nature of the training format, causing issues such as low levels of comprehension that often result in errors. However, it is necessary for the workforce to possess high levels of skills to ensure that practices are safe, productive and of good quality. This need to ensure that the workforce is highly skilled has resulted in increased operational costs and rail delays. The current comprehension rate of traditional training varies widely depending on the training style used. These can be as low as 10% to 20% for passive learning styles like presentations, to 75% for practical experiences [7]. However, due to inaccessible equipment or scheduling issues for practical experiences, a workaround is needed to assess comprehension rates.

The use of AVR tools under Metaverse provides comparative advantages to reduce the amount of passive learning and shift the learning style to a portable, accessible and practical teaching solution, increasing the overall comprehension rates [8,9]. The scope of this manuscript is to develop AVR-based driver training models and will further establish the efficacy of a developed model through case studies in the rolling stock.

1.2. Current Rail Hazard Perception

In the workplace, accident avoidance highly depends on people’s perception of available information regarding hazards. However, the success rate of such procedures is not very high, as they consist of standards that refer to procedures to efficiently follow safety signs, such as conditional features and warning designs along with the targeted users. Further, based on recent research, it has been brought to light that warning systems based on technology may be much better in terms of communicating safety information based on display dynamics—for instance, signs based on sensors may be able to provide elderly workers with the necessary cognitive support required to compensate for attention deficiency caused by the ageing process.

Research has also been conducted on the usability Metaverse enabled platform to improve the safety of the workplace [10,11]. The research conducted was in the domain of transportation, which focused on the advantages of using such technology due to the benefit it provides in identifying hazards to warn people in time. In addition, the research done regarding the impact of safety signs was restricted by various aspects because the study used a Metaverse-based set-up to test the technology, which gave a platform to assess behavioural compliance in dangerous situations with complete control of experimental conditions [12,13]. The major objectives of the study are (i) to assess the impact of Augmented Reality (AR) warnings on behavioural compliance and hazard risk behaviours, and (ii) to analyze the user experience while interacting with given signs and a prototype of the VE [10,14].

2. Proposed Framework

The rail industry is a critical transport industry that requires constant monitoring and upskilling. This industry is spread across regional and metropolitan areas of Australia, employing thousands of Australians. Among them, train drivers play a crucial role in helping people move from one location to another. They contribute immensely to the social and economic outcomes of the Australian economy. The previous chapter explained that train drivers require knowledge in the areas of safety, risk analysis, hazard perception and decision-making to operate trains. However, current driver training methods focus on theoretical tasks to build hazard perception and decision-making skills, and these methods have been deemed low quality and have resulted in productivity losses.

The dependency on work-based training also prolongs the skill development process of drivers and requires constant supervision. Long supervision combined with high trainer costs made driver training programs cumbersome. These irregularities in current driver training programs reduce the productivity of both the trainer and trainees. Another critical issue with a competency-based training program is the risk of biased trainer judgements. Since assessment judgements are solely dependent on the expertise of the trainer, there is potential for a few drivers to fall through the cracks and endanger the community.

Conventional training methods are generally less effective, with low comprehension rates. Traditional driver training also focusses on technical and cognitive skills and knowledge development. However, it does not reinforce the importance of ergonomics and work postures. The safety and productivity of drivers can be potentially improved with advanced posture recognition analysis. These gaps in current driver training programs adversely affect the productivity, quality and safety of the driver. Figure 1 shows that the disparities in conventional training can be addressed by implementing a Metaverse (Driver Training Framework) DTF that utilizes transformative technologies (e.g., AVR, finite element analysis and motion capture) to help drivers address the aforementioned knowledge and skills gap. The implementation of innovative training methods provides real-time feedback and enhance the productivity of the training and trainee. Several assessments showed improvements in driver trainees’ reaction time, situational awareness, decision-making, information retention rate and posture.

3. Metaverse-Embedded Driver Studies

Current driver training techniques adopted by the rolling stock were reviewed for gaps. Train cabin designs around the world vary depending on the standards followed by different countries and organizations. Therefore, based on the study conducted, a set of scenarios that are relevant to driver training was identified. The immersive environment and high-risk scenarios were developed in consultation with rolling stock professionals. A tram cabin was considered to analyze the driver’s line of sight. Drivers are required to anticipate every action possible within an environment to ensure safe travel. It is implicit that drivers should have direct visibility of the path forward; hence, cameras and mirrors were provided for better vision. Utilizing immersive technology, we tracked the movement of the participants and analyzed their situational awareness. The objective of this study was to identify the key factors that affect driver performance in terms of attention and visualization. The participants were subjected to a questionnaire survey to obtain general information, and a visual test was carried out to create basic driver profiles. In the next stage, the participants were exposed to various simulations in a virtual set-up.

3.1. Attention and Hazard Perception

A detailed study was conducted on the rolling stock to identify high-risk situations faced by train drivers in Australia. Hazard perception scenarios ranging from low-to high-risk environments were developed based on information received from the industry. Eye-tracking equipment was utilized to analyze and monitor the participants’ fixations. The eye-tracking unit was calibrated to capture the visual perception of the driver and obtain qualitative data from the experiment. This data enabled a qualitative analysis of drivers’ attention and visual perception of hazards and helped to explain the attention distribution of train drivers in relation to the environment and control panel. However, the use of the eye tracking equipment was not necessary if the driver’s eyes were fixated on the region of interest, meaning that they were paying attention to the region. The objective of collecting the fixation data was to review the drivers’ attention and spatial awareness.

Attention can be broadly classified into different categories, such as sustained attention, selective attention, divided attention and alternative attention, depending on their function. The most fundamental level among all the defined types of attention is sustained attention, which decreases when facing repetitive scenarios and hence, increases the possibility of accidents [15]. Another category of attention, selective attention, can be termed as the ability to be attentive while being occupied by different stimuli. It is called selective attention because the person must stay attentive even while being occupied and avoid attention interception. Figure 2 highlights the sequence involved in observing the attentiveness of drivers through eye tracking. The duration of fixation on points of interest, time to first eye movement towards the hazard, dwell time and number of times the attention goes to the critical area are the factors that signify attention behaviour. Using eye-tracking data, we can analyze drivers’ eye movements, understand their situational awareness and engagement and find a correlation between gaze data and situational awareness. Situational awareness was crucial in studying the hazard perception and reaction time of the participants. The information about where the driver’s attention was and where they were supposed to look determined their attention profile. Every train driver must anticipate movements in their surroundings based on the actions taking place in the scenario.

3.2. Pupil Dilation

The opening in the pigmented area of the iris containing the sphincter and dilator muscles is known as the pupil. It is believed that the pupil dilates due to the cognitive workload on the body [16,17]. As a voluntary motion, the pupil dilates when there is a surge in brain activity due to an increased cognitive load. Unlike a neutral scene or stimulus, which does not lead to a dilation of the pupil, a high-risk scenario can lead to positive or negative valence [18].

By analyzing the pupil diameter, we aimed to determine if the participants were triggered by risky scenarios and evaluated their reaction time. With the aid of an eye-tracking device, the movement of the pupil could be analyzed and used to confirm if the participants were attentive to the situation. Cognitive studies helped determine the preparedness of the participants to face real-life scenarios. The eye data captured from the device were analyzed to help determine the first trigger that confirmed their perception of the scenario. The quicker they responded to the scene, the better their perception of the risk. As shown in Figure 3, fatigue is categorized into task-related fatigue and sleep-related fatigue for the accurate analysis of the train drivers’ behaviour in this study.

3.3. Comprehension Assessment

Research has shown that novice drivers more commonly fail to anticipate and mitigate crash hazards compared to experienced drivers [19]. Risk assessment and perception training are a few techniques that have been used to improve hazard anticipation and mitigation skills among new drivers. Novice drivers are prone to greater risk compared to experienced drivers due to a lack of experience, immaturity and eagerness to take risks. In Australia, there are approximately 23,000 level crossings, and the majority of these are road crossings. Level crossings are further classified as (i) active level crossings, (ii) passive level crossings, (iii) occupational or accommodation crossings, and (iv) maintenance crossings. Out of these types of crossings, active level crossings contain automated warning systems, such as boom gates, to restrict the flow of traffic. The remaining level crossings do not have automated warning systems and are therefore classified as high-risk environments for both train drivers and the individuals involved in the environment. The ignorance of the people crossing railway crossings can lead to serious injuries or even death. Therefore, these circumstances need to be considered high-risk scenarios. It is necessary that novice train drivers be prepared for such situations.

3.3.1. Knowledge Retention Rate and Hazard Perception Using Metaverse

Train drivers are required to anticipate hazards and locate threats that can lead to catastrophic accidents. Studies have shown that improvements can be made to conventional driver training to improve hazard perception and knowledge retention rate [20]. Conventional training methods are highly dependent on the theoretical knowledge gained from the training. The information retention rate and effectiveness of such programs are low due to the lack of immersive cognitive mapping. The eye-tracking system, combined with the immersive environment, will enable drivers to analyze their hazard perception skills and help them to capture information quickly and efficiently. One of the objectives of this experiment was to examine the hazard perception and information retention rate of the participants using traditional and Metaverse-based training. VR headsets are considered an effective training tool compared to simulators due to the immersive experience provided by the HMDs. Agrawal et al. indicated that Metaverse-based training helps improve hazard perception skills among novice drivers [19]. While an experienced driver has gained skills and knowledge through continuous exposure in real environments, a novice driver undergoing Metaverse embedded training can obtain the skill set within a shorter time frame. Driver training is a method used to improve the performance of a driver by imparting the skills required to safely and efficiently drive a train.

The participants selected for the experiment included a novice group who had no practice or previous experience in Metaverse embedded driver training. In this experiment, the improvements in productivity and information retention rate due to the inclusion of an immersive environment were studied. The inclusion of audiovisual instructions, combined with the immersive experience, assisted in understanding the change in driver performance. A series of scenarios were developed based on the input provided by experienced train drivers and industry experts. Events that occur during a shunting process in a train yard, taking actions based on the audio instructions provided and operating trains based on visual signals were considered in the scenarios. The participants were instructed to take action, and their response rates were measured for analysis. To deliver a comprehensive vision analysis, HP G2 Reverb Omnicept was utilized. This device was chosen due to its state-of-the-art sensor system that could measure eye movement, gaze, pupil size and pulse and seamlessly transfer data to the HP Omnicept platform for further analysis.

3.3.2. Hazard Reaction Time

The hazard reaction time of the participants was determined based on the press of the button in the Metaverse-based simulation during critical circumstances. The eye-tracking data, combined with the reaction to the scenario, determined their efficiency and attention towards the hazard. If a participant did not respond to the potential hazard, real-time feedback was provided to the participant.

4. Metaverse Embedded Training Framework Results

This manuscript shows the studies and results achieved by the proposed DTF model for enhancing driver productivity, safety and quality. The model incorporates unique AVR-based techniques to enhance outcomes. Case studies were performed in three areas to evaluate the efficacy of the developed Metaverse-based DTF: (i) sightline studies for enhanced productivity, (ii) driver vision analysis for enhanced driver attention.

4.1. Driver Sightline

Drivers mainly use far-off vision and proceed to regularly look over both sides in a closed field of vision. To account for the dangers train drivers are most afraid of, vision requirements are defined to improve driver vision of pedestrians and cyclists. When a requirement with regard to visibility is defined, it is implied that the driver has direct visibility without the use of an on-board camera or mirrors. Far-off images outside the field of vision are considered important to the driver, so these are anticipated when the vehicle is moving, and drivers can clearly see the signals. Figure 4 summarizes the requirements of this field of vision.

The post has a standard defined height, and therefore, the cabins had to be designed accordingly (see Figure 5). Rail network systems vary across the world, and therefore, testing and design standards would have to be developed accordingly. This is commonly done using traditional design and prototype testing methods, but this, in turn, would ramp up costs and alter production timelines significantly.

4.2. Analysis of Traditional and Metaverse-Based Studies

All the drivers successfully spotted the strategically placed pedestrian on the new and improved cabin design. This was reported when a significantly wider frontal field of view was achieved by reducing pillar size. Only a small percentage of drivers were able to spot the pedestrian in the old design, pointing to the potential blind spot in the cabin design. The tests were successful, and they were able to make the one significant oversight that eluded the design team. Nonetheless, with full support from both the drivers and shareholders, the entire system allowed the company to proceed with the construction of a revolutionary design without physical prototyping. A comparison of the two designs is shown in Figure 6a,b.

Figure 7a,b and Figure 8 show a comparative benchmarking study. The Metaverse-based simulations for prototyping were compared with the traditional prototype manufacturing approach for a tram design. Costs and man-hours were significantly reduced throughout the entire design, testing and validation processes. Figure 7a shows the calculated costs of operations to construct the prototypes and indicated savings of A$480,000 for an A-class tram.

Another prominent issue with trains and trams is multiple stakeholder involvement, including involvement from government organizations, driver associations, manufacturers, maintenance contractors, main and subcontractors and relevant councils. This prolongs the decision-making process and results and inflates costs. Figure 8 indicates a substantial reduction in related man-hours of up to 3696 man-hours.

Table 1. Percentage of savings using Metaverse-based prototyping.

Criteria	Savings (%)
Tram prototype	94.4
Stakeholder meeting	50
Testing	100
Rejigging	93.8
Approval	93.3

depicts the overall savings achieved from Metaverse-based simulations. The overall savings and effectiveness were found to be around 95% compared to traditional methods of prototyping. Stakeholder meetings were reduced by 50%, while approval and assessment strategies were over 93% effective for Metaverse-based practices. Testing was completely eradicated, as Metaverse-based simulations allowed quick changeovers and visualizations.

4.3. Driver Vision Analysis for Hazard Perception

4.3.1. Simulation Scenarios

A simulation was generated with a total of 12 different scenarios or events occurring during a train shunting process in the train yard. The 12 different scenarios simulated included a variety of hazards or distractions. The different scenarios included (i) track closure or changed driving conditions due to construction work, (ii) alerts flashing on different display screens of the driver’s cockpit, (iii) persons crossing the train tracks or in the vicinity of the train tracks, (iv) traffic control signals, (v) driver visual aids such as mirrors, (vi) oncoming train traffic, and (vii) road vehicle movement along rail crossing. The variables that defined the three driving conditions included (i) weather conditions, (ii) natural and street lighting, (iii) water over the windscreens, (iv) movement of train wipers, which altered visibility, and (v) clarity under each of the driving conditions. The test environments (i) a morning (ii) night and (iii) evening

The 12 different hazard scenarios are detailed below in the order that the participant experienced in the simulation:

• A-pillar—This event or hazard scenario corresponded to the hazards that result from the design of the train interior. As the focus of this simulation was on the driver’s field of view, the A-pillar, which holds the windscreen in position, was an obstruction to the driver’s vision and hence, was considered a hazard. The driver must look around the A-pillar to ensure that nothing is hidden from sight behind it. As the A-pillar is part of the train body, it was always at a fixed distance from the driver.
• Screen 2—Similar to the above scenario, in this case, one of the many on-board screens was required, as the driver needed to be on constant lookout for prompts relating to situations that required action from the train driver. This screen was mounted on one of the interior panels of the train at a fixed distance of one metre from the driver.
• Screen 1—This corresponded to another one of the many on-board screens where the driver needed to be on constant lookout for prompts. This screen was mounted on one of the interior panels of the train at a fixed distance of 1 m from the driver. The scenario was named after the designated name of the hazard in the simulation set-up.
• Left mirror—This scenario corresponded to an external mirror mounted on the left side of the train at a fixed distance of two metres from the train. The driver was expected to constantly monitor the external mirror to be aware of any approaching hazard (e.g., another train, construction equipment, road vehicle or pedestrian). Being closer to the driver by a distance of one metre compared to the right mirror, the left mirror obstructed the driver’s view more.
• Signal 1—The train driver was expected to be on the lookout for the signals, notice the signal or signals relevant to their section of the track and carry out required operations, such as reducing the speed or bringing the train to a complete halt.
• Moving vehicle—In this scenario, the train driver was again expected to make a note of vehicular movement in their field of view and be on the lookout for the vehicle’s movement to ensure that the vehicle did not run into the train’s path, assuming that the vehicle’s operator has not sighted the train moving adjacent to the vehicle.
• Right mirror—This was one of the visual aids that was mounted exterior to the train. Similar to the left mirror, the right mirror, in combination with the A-pillar, offered further obstruction to the driver’s view. However, as this mirror was a metre further away from the driver compared to the left mirror, it caused slightly less obstruction to the view and was rated as slightly less hazardous.
• Emergency stop lever—In this scenario, the driver had to visually spot and locate the stop lever so that they are aware of its location relative to their own hand position in case it needs to be operated if an emergency event were to occur. As part of their behaviour to improve their situational awareness, the train driver was expected to visually locate and spot the emergency stop lever to ensure that they were aware of its location relative to their own position from time to time.
• Distant construction—This scenario involved a construction activity at a significant distance from the rail tracks, such that it did not pose any risk to the train’s operation. In this scenario, the driver was expected to take note of the construction activity and be cautious but was not necessarily required to slow down the train due to the distance of the construction activity from the train.
• Close-by construction—In this scenario, a construction activity close to the rail tracks was simulated. This scenario required the driver to obey yard driving rules and slow the train down for the safety of the personnel involved in the construction activity, and the level of safety risk was significant.
• Pedestrian—This scenario simulated a person walking next to the train tracks. Similar to the scenario where a vehicle was moving in the proximity of the train, the driver was expected to make a note of the person and track their movement around the train to ensure their safety.
• Signal 2—This was a scenario where a signal in the form of a visual prompt was given to the driver or operator of the train requiring them to attend to one of the on-board systems to ensure an operational requirement was met. This visual prompt for signalling to the driver was mounted on one of the interior panels of the train and was at a fixed distance of two metres from the driver’s seat.

Figure 9 and Figure 10 show a compilation of a series of snapshots taken from the train driver simulation during the morning driving condition, showing the display that was visible through the headsets of the AVR system to the drivers undertaking the simulation. Starting from the left, the following are observed: the distant construction activity, the left mirror and A-pillar, a nearby pedestrian, Signal 1, close-by construction activity next to the train tracks, the train tracks, a distant pedestrian to the right of the tracks and a moving vehicle. A pedestrian is also observed in the image on the left side of the train and is covered partially by the blind spot created by the left mirror and the A-pillar combined.

Of the above 12 hazards, six were presented from within the train. These were the A-pillar, Screen 1, Screen 2, left mirror, right mirror and emergency stop lever. The remaining six hazards were presented in the simulation, namely Signal 1, the moving vehicle, distant construction, close-by construction, the pedestrian and Signal 2. These were external hazards or hazard indicators, and the drivers were expected to be alert and on the lookout for such hazards.

The variables that defined the three separate driving conditions included the weather conditions, ambient lighting (i.e., natural and street or yard lighting), water over the windscreens and exterior mirrors, and wiper movement on the train’s windscreen. All these driving conditions altered the visibility and clarity of driver vision. The morning environment presented clear weather and good natural lighting, whereas the night environment presented a mixture of lighting conditions, including limited lighting in some areas of the train yard and bright lights in other areas of the train yard, in addition to the lighting from the train headlamps. The evening rain environment presented poor driving conditions, such as poor visibility due to water on the train windscreen and exterior mirrors and reduced visibility due to rain and poor lighting.

4.3.2. Simulation Metrics

Five key metrics were measured to record the train drivers’ responses during the different simulation scenarios. These five key metrics were:

• Average fix instance duration (FID)—the average time (in seconds) the eyes were focused on a particular object in a single instance
• Total fixation count (FC)—the number of times the person looked at an object
• Total fixation length (FL)—the total time (in seconds) spent looking at a particular object
• Average TFF—the time (in seconds) taken to first spot an object once the simulation starts
• Distance to first fixation (DFF)—the distance (in metres) from an object when spotted.

These metrics collectively defined the response of a driver to a particular event. This was from the time the event was presented to them to the time when the driver had to respond to the event in the desired manner. The collected metric values were exported from the system in a suitable format for further reading and processing.

4.3.3. Results

Figure 11 shows the response of all the 20 drivers to scenarios where there was a hazard created due to the closing of the tracks because of construction activities. FID, FC and FL were plotted as response curves for each of the drivers. Figure 11 shows that although the total FC for individual drivers may be high, it was not necessary that they have a high total FL. The variation in the FID and, in turn, the FL, strongly indicated that while some drivers fixated on the same hazard for a relatively long time, others were capable of looking at other monitors of indicators. Hence, they were able to take note of other activities in their site of view that were essential from a safety viewpoint.

Similar to Figure 11, Figure 12 and Figure 13 show the response of all 20 drivers to the hazard scenario arising from the close-by construction activity in the night and evening environments with poor operating conditions. Comparing Figure 11, Figure 12 and Figure 13, it is observed that the drivers responded very differently to the same hazard under varying driving conditions. Across all three driving conditions, the value of FID was fairly constant and there was a reduction in FL value. This value reduced as the driving conditions deteriorated from clear visibility in the morning to very poor visibility in the evening under heavy-rain conditions. This was a direct indication of the increased level of alertness of the drivers in paying attention to and scanning the field of view for other potential hazards and visual inputs as driving conditions deteriorated.

Figure 12 compares the TFF with the distance at first fixation (DTFF) for a close-by construction hazard in the morning scenario for all 20 drivers. It was observed that when the average TFF was shorter, the DTFF was higher. However, it was not always the case, which indicated that some drivers were constantly looking at different monitors for other hazards even before they responded to a particular hazard.

Figure 13 shows the response of an individual driver across all scenarios in the morning driving environment. It was observed that the response to each of the hazards varied significantly, indicating that the driver perceived each hazard differently and responded to them with varying degrees of attention.

Figure 11, Figure 12 and Figure 13 represent one of the 12 hazard scenarios that the drivers were put through as part of the simulation to gain an understanding of how the simulation and various metrics derived from them can be used to understand and gauge the driver responses under different driving conditions.

To gain a better understanding of driver responses, values for the three metrics—FID, FC and FL—were averaged for all the 20 drivers across the 12 hazard scenarios for the three different driving conditions. The results are summarized in

Table 2. Average values of driver responses for all hazards across all driving conditions.

	Morning Driving Condition			Night Driving Condition			Evening Rain Driving Condition
	FID (s)	FC (#)	FL (s)	FID (s)	FC (#)	FL (s)	FID (s)	FC (#)	FL (s)
A-pillar	5.32	3.00	15.57	6.55	4.85	31.91	6.70	4.95	33.38
Screen 2	12.08	3.95	47.93	9.03	6.00	54.43	7.36	5.10	37.5
Screen 1	4.94	3.00	14.82	5.08	4.85	24.52	6.16	5.75	35.76
Left mirror	1.95	10.85	21.15	2.19	12.00	26.42	2.04	9.70	19.77
Signal 1	2.39	2.00	4.79	2.99	3.00	8.97	4.11	3.95	16.26
Moving vehicle	1.61	6.05	9.77	1.41	4.25	6.00	1.64	3.00	4.93
Right mirror	1.60	8.95	14.31	1.91	9.65	18.40	1.95	12.1	23.78
Emergency stop lever	1.88	7.70	14.61	2.48	8.90	22.13	2.03	10.95	22.36
Distant construction	2.42	4.10	9.89	1.60	2.70	4.80	1.05	1.90	2.11
Close-by construction	2.68	7.05	18.91	2.09	5.00	11.26	1.78	4.00	6.34
Pedestrian	3.21	2.00	6.42	2.97	3.00	8.91	2.43	3.60	9.66
Signal 2	3.24	3.00	9.72	2.75	4.35	12.05	2.15	5.10	37.5

.

4.3.4. Comparison Based on Distance from a Hazard

Based on the generated data, driver responses can also be gauged by comparing the trends between the time taken to first identify the hazard or TFF and the distance from the hazard when it was first identified or the DFF. Based on the general definition of both TFF and DFF, it can be generally concluded that the longer the time taken is to identify the hazard, the shorter the distance is from the hazard.

Figure 14 compares the TFF and the DFF for the close-by construction hazard in the morning driving environment for all 20 drivers. The general trend observed suggests that the shorter the TFF, the higher the corresponding DFF, and this is true for most cases. A driver operating the train at a higher speed with a low TFF value may still have a lower DFF compared to a driver operating the train at a lower speed with a higher TFF value. This means that a quicker response (TFF) by a driver operating at higher speeds may still result in the hazard being identified at a closer distance (DFF) from it compared to a driver operating at lower speeds with a slower response. Similar to Figure 14a–c show individual driver responses based on TFF and DFF values for the night and evening rain driving environments, respectively.

When placed in order of increasing difficulty, the three driving conditions discussed here are the morning driving condition, night driving condition and evening rain driving condition. From the plots in Figure 14a–c, it is observed that the average time for a driver to first catch sight of a hazard (i.e., the TFF) increases as the difficulty level of driving conditions increases.

Table 3. Average TFF and DFF values for all hazard scenarios under all driving conditions.

	Avg. Time to First Fixation, TFF (Seconds)			Avg. Distance to First Fixation, DFF (Metres)
	Morning	Night	Evening Rain	Morning	Night	Evening Rain
A-pillar	0.74	0.56	0.56	1.00	1.00	1.00
Screen 2	3.09	3.31	3.37	1.00	1.00	1.00
Screen 1	3.54	3.22	3.26	1.00	1.00	1.00
Left mirror	5.23	4.99	5.05	2.00	2.00	2.00
Signal 1	5.97	5.87	4.53	2.00	2.00	2.00
Moving vehicle	6.99	8.28	8.43	27.00	29.05	19.80
Right mirror	9.12	8.12	8.61	3.00	3.00	3.00
Emergency stop lever	3.31	2.98	2.94	1.00	1.00	1.00
Distant construction	16.82	18.25	24.94	32.15	29.60	24.20
Close-by construction	46.60	61.04	64.17	21.30	22.75	23.55
Pedestrian	48.25	52.88	48.49	33.65	32.05	21.85
Signal 2	55.02	51.62	56.36	35.15	32.45	31.75

shows that the average TFF for all the 20 drivers is 46.60 s under the morning driving condition, 61.04 s under the night driving condition and 64.17 s under the evening rain driving condition. The average DFF for all the 20 drivers is 21.3 m under the morning driving condition, 22.75 m under the night driving condition and 23.55 m under the evening rain driving condition. Assuming that each driver has similar responses for all three driving conditions, it can be observed from the above plots that there is a 38% variation between the minimum and maximum average TFF values, whereas the variation for the average DFF is only 10%, indicating that the drivers varied the operating speed significantly based on the driving conditions to identify hazards and ensure safe operations. Slowing down the operating speed in difficult driving conditions led to an increased average TFF and was also accompanied by a slight increase in average DFF values, indicating that the drivers responded positively to more difficult driving conditions to ensure safety.

The A-pillar was an important safety concern for many train and tram drivers as it obstructs drivers’ views to a great extent. Based on the analysis of the data collected from the simulations, the significance that drivers attribute to the A-pillar was confirmed. As driving conditions deteriorated, it can be noted from Table 3 that the average values of all three parameters (i.e., FID, FC and FL) increased. There was a significant increase in the values of FID and FC and a sharp increase in the value of FL as conditions deteriorated. The TFF value decreased as driving conditions deteriorated. The TFF value decreased with deteriorating driving conditions, as the risk posed by the A-pillar was further compounded as visibility became poorer.

Screens 1 and 2 were interior to the train cabin and were mainly tasked with displaying routine information about the train’s performance to the driver. The data show that, on average, the drivers paid more attention to Screen 1 than Screen 2 in deteriorating driving conditions. The TFF values varied slightly for the two screens, as it was most likely that the drivers looked at Screen 2 before looking at Screen 1. However, it can be noted from Table 3 that the TFF for Screen 1 was slightly shorter than for Screen 2 for the more difficult night and evening rain driving conditions, thus reinforcing the finding that drivers generally paid more attention to Screen 1.

Next, comparing the data on the left and right mirrors, it was observed that the drivers generally paid more attention to the left mirror than the right mirror. This was mainly because the left mirror, together with the left A-pillar, significantly obstructed the drivers’ view compared to the right mirror. However, in the worst condition of driving (i.e., in the evening in heavy rain), the drivers paid significantly more attention to the right mirror. It is worth noting that the left mirror (2 m) was closer to the driver compared to the right mirror (3 m). Based on this observation, under the worst driving conditions, the proximity of the left mirror to the driver was of less significance than the right mirror.

The distance between the train and Signal 1 was very short, and for this reason, Signal 1 was considered by the train drivers as a more significant hazard than Signal 2, which was further away from the train. In the case of Signal 1, the drivers paid more attention to it under all driving conditions with increased FID and FC values, resulting in an associated increase in FL value under deteriorating driving conditions. Conversely, for Signal 2, the drivers spent less time looking at it on each instance but looked at it more often, leading to a decrease in FID values and an increase in FC and FL values as the driving conditions changed from a clear morning environment to a hazy evening rain environment. Based on the distance from the signals, the drivers’ TFF decreased with deteriorating conditions for Signal 1, whereas for Signal 2, the drivers took a significantly long time to first fixate on it, leading to a significantly shorter distance between the train and Signal 2.

The other two hazards that were generated during the simulation were the two construction scenarios—one close to the tracks and the other further away from them. Both of these construction activities did not directly affect the train’s operations, and hence, both were deemed as low risk. Comparing the average values of FID, FC and FL for both construction activities across the three driving conditions, it was observed that the corresponding values for the distant construction site were much lower than those for the close-by construction site. This observation indicated that the distance from the train was considered a key factor by the drivers in perceiving the level of risk. The average TFF values suggested that the drivers located the hazard due to the construction activities later as the driving conditions worsened due to poor visibility and also as a result of them focusing more on the vicinity of the tracks.

4.3.5. Individual Driver Responses

Figure 15a to Figure 16c capture Driver 1’s responses in terms of each of the five metrics for an individual driver across all hazard scenarios in all the three driving conditions. The driver’s response to each of the hazards under varying driving conditions varied significantly, indicating that the driver perceived each hazard differently and responded to them at varying levels. Figure 15a compares the response of Driver 1 towards all hazard scenarios across the three driving conditions. Generally, it was noted that the FID was relatively lower for the evening rain driving condition but had a higher FC, as shown in Figure 15b, indicating that, on average, Driver 1 looked at the hazards for a shorter time on each fixation instance. The trends for Driver 1 corroborate the earlier observation based on average FID and FC values for all 20 drivers—that during bad driving conditions, drivers generally paid more attention to high-risk hazards while paying less attention to hazards that posed less risk, such as the close-by and distant construction activities.

Figure 16a compares Driver 1 responses in terms of total FL. It should be noted that FL is directly proportional to FC and FID. The trend of the driver’s response in terms of FL corresponds with the trends for FC and FID. A-pillar, both the left and the right mirrors and the emergency stop lever had most of the driver’s attention along with Signal 1, especially during the evening rain driving condition. The least amount of attention was paid to the construction activities during the evening rain driving condition, as the driver was aware that these posed minimum risk and there were more hazards in the vicinity of the train that required more attention.

Figure 16b shows that most of the hazards were identified by the driver at similar times after the start of the simulation. The relatively significant variation in the TFF values for the two construction activities indicated that the driver was more focused on looking for hazards in the vicinity of the train and tracks, and the two construction activities did not fall in the vicinity of the zone that the driver deemed important and of high risk. In terms of DFF values, Figure 16c shows that most hazards were seen by the driver for the first time at similar distances from the train. Due to the attention given to other hazards during the evening rain driving condition, the driver recorded lower values of DFF for the moving vehicle and the pedestrian compared to during the other two driving conditions.

4.3.6. Business Impact

The business impact theme addresses how the changes in organizational behaviour arising from the training affect the overall business. It encompasses the business impact of the training at the organizational level. Further, in the hierarchy of the themes, it influences training effectiveness and is affected by attributes that are organizational in nature. Thus, this theme is not influenced by any attributes that are related directly to the individual who underwent the training. In dealing with the business impact theme, it was determined whether the organization had achieved their goals through the training. The business impact that a successful training program is expected to have on an organization is attributed to outcomes such as (i) increased sales, (ii) better customer satisfaction, (iii) increased market share, (iv) fewer operation errors, and (v) reduced downtime. The metrics that are attributed to the business impact of training are routinely monitored as these are the key performance indicators (KPIs) of the organization’s health. However, the attributes that need to be monitored at the operational level are (i) non-optimal methods of operation, (ii) erroneous operation methods (often leading to varying degrees of downtime), and (iii) equipment faults. The main indicators that are monitored to measure the successful outcome of a training program being evaluated are (i) improved quality, (ii) reduced cost, (iii) reduced time, and (iv) higher revenue.

Lower Operation Cost

Effective technical competency training, when implemented and integrated into processes, leads to significant improvement in system efficiency. Hence, this leads to a significant reduction in costs. Cost reduction is considered achieved if there are (i) reductions in waste material, (ii) improved equipment handling and maintenance, (iii) improved processes and procedures, (iv) lowered inventory, and (v) improved production and manufacturing time, among others. In the case of operating cost, this is a KPI that is monitored to ensure the health of an organization. Therefore, it is logically one of the key indicators to look for when assessing the business impact of a training program. This is considered when a training program aims to improve the efficiency of the business or organization’s function. Operating cost, in this context, does not have to be considered from a manufacturing perspective but from any process within an organization, as the cost is associated with all functions.

Improved Quality

Improved quality is one of the attributes that influences the business impact from an effective training program perspective. Effective training leads to fewer errors and improvement in the performance of tasks and operations. These, in turn, lead to better overall outcomes. Improved quality leads to positive customer feedback and customer satisfaction. By accounting for various quality factors, it is possible to establish the impact an effective training program has on an organization.

Improved Efficiency

The improved efficiency attribute aims to determine the processes, procedures and performance that have improved through the effective training of staff and employees. Efficiency attribute is considered improved if there are (i) reduced turnaround times, (ii) reduced waste, (iii) reduced maintenance downtime, (iv) improved safety, and (v) reduced accidents, among others. There is a clear correlation between a training program’s business impact and indicators of improved efficiency.

Higher Revenue

Greater or improved training effectiveness leads to lower operational cost, improved quality and improved efficiency. These, in turn, increase an organization’s earnings or improves their profit margins. The main purpose of an organization’s existence has been identified as to increase revenue. Therefore, their dedication to effective training aims to help make processes, procedures and products more robust. Higher or increased revenue as a result of an effective training program is the most sought-after business impact by organizations. This attribute is quantified by calculating the ROI to ensure that the outcomes that resulted from the training program justify the cost involved.

Quantitative Data for Business Impact

The quantitative data for business impact, as shown in

Table 4. Quantitative data for business impact.

Attribute	Description	Source
Operating cost	Any decrease in operating cost as a direct result of improved processes due to the knowledge or skill acquired from training	Operations Department or Finance Department of the organization or business
Improved quality	Any improvement in the quality of the deliverables, such as manufactured goods, products or services, as a direct result of improved processes due to knowledge or skill acquired from training	Operations Department or Quality Department of the organization or business
Improved efficiency	Any reduction in operating cost and time as a direct result of improved processes due to knowledge or skill acquired from training	Operations Department of the organization or business
Increased revenue	Any increase in the revenue as a direct result of improved processes due to knowledge or skill acquired from training	Finance Department of the organization or business

, were generated differently compared to the data from the rest of the previous themes. In this case, relevant data needed to be obtained from various business reports that were routinely generated by different functions within the business.

5. Conclusions

This research established a novel Metaverse DTF for train drivers. Further, this framework was validated through a series of driver studies, including (i) driver sightline studies and (ii) vision analysis.

• Based on the study that was conducted, responses of an individual driver followed similar trends over all the 12 hazards across the three driving conditions. In the analysis of the response of all 20 drivers, it was also observed that the drivers paid more attention to known risks such as the A-pillar and the left and right mirrors, which were known to obstruct their view.
• Out of 12 hazards, drivers paid about 40% more attention to both internal and external hazards during the night driving condition, whereas they paid about 32% lower attention during the evening rain driving condition.
• The number of hazards that were observed and the measures taken to resolve them have significant improvements by employing Metaverse-based driver training. Themes were introduced and subdivided into their key attributes. The employee response theme consisted of (i) training content, (ii) training design, (iii) training delivery, (iv) training appropriateness, and (v) training encouragement. The main attributes of the acquired knowledge theme were (i) targeted gain in knowledge, (ii) suitability of the gained knowledge, (iii) change in attitude, and (iv) acquired knowledge. Finally, the business impact theme needed to be monitored, and the main indicators were (i) improved quality, (ii) reduced cost, (iii) reduced time, and (iv) higher revenue.
• The overall savings in terms of cost and time are 95% effective using Metaverse-based training method compared to traditional methods. Stakeholder meetings are reduced by 50%, while tasks that required assessments were automated providing over 93% effectiveness using Metaverse-based practices. Assessments is completely eradicated, as Metaverse-based simulations allowed quick changeovers and visualization capabilities.
• There is a 38% variation between the minimum and maximum average TFF values (time taken to first spot a hazard causing object once the simulation starts), whereas the variation for the average DFF (distance from a hazard causing object when spotted.) is only 10%. This indicated that the drivers varied the operating speed significantly based on the driving conditions to identify hazards and ensured safe operations.