Gender Characteristics on Gaze Movement in Situation Awareness

Abstract

In large systems, such as nuclear power plants, the operator’s situation awareness is vital to the system’s safety. Since gaze movement is closely related to situation awareness, various studies have evaluated it through gaze movement. The number of female workers is increasing even in large systems, such as nuclear power plants, so it is relevant to compare and analyze the situation awareness and gaze movement characteristics of men and women. In this study, an experiment was conducted to compare and analyze men’s and women’s situation awareness and gaze movement characteristics by making a simulator for emergency scenarios in nuclear power plants. Gaze entropy was used as a measure to indicate gaze movement, while the Situation Awareness Rating Technique (SART) was utilized to measure situation awareness. A total of 20 engineering college students (10 male, 10 female) participated in the experiment. Loss of coolant accident (LOCA), steam generator tube rupture (SGTR), steam line break (SLB), and loss of voltage (LOV) were the nuclear power plant accident situations used as task scenarios for the experiment. For all accident scenarios, the SART score did not show a significant difference between men and women. Shannon entropy, dwell time entropy, and heat map entropy did not show a significant difference between men and women, but Markov entropy was found to be significantly higher in women. In conclusion, there was no significant difference between men and women in awareness of accident situations. In addition, there was no significant difference between men and women in the ratio of viewing the necessary information elements in the situation awareness process. However, it was found that women had more gaze movements between necessary information elements than men.

1. Introduction

In a system where safety is essential, such as a nuclear power plant, it is vital to judge the system state in an emergency operation correctly. The judgment of the system state depends on the operator’s situation awareness (SA), which is the perception of environmental elements within time and space, the comprehension of their meaning, and the immediate projection of their status [1,2]. This definition describes the perceptual aspect of information processing, the mental activities that process the perceived information, and the resulting mental activity [3].

While SA is not equivalent to performance, a higher SA is generally correlated with higher performance. High SA has improved safety in aviation control rooms [4] and nuclear power plant control rooms [5,6,7] by reducing mistakes and accidents. Endsley [1] reported that SA in a complex system causes 88% of human errors. Grech et al. [8] analyzed 177 marine accidents and found that 71% of human errors were SA-related. Schulz et al. [9] showed that 81.5% of human errors in anesthesia were related to SA. Therefore, it can be seen that the operator’s SA is crucial for a system’s safety. In this context, studies on SA have been conducted from various perspectives.

In particular, since humans receive most of the information through sight, gaze movement has been used as a crucial indicator in SA. Gaze movement analysis has been used to analyze SA in various fields such as civil aviation cockpit settings [3], nuclear power plant control rooms [10], aviation [11], and health care [12]. As described above, although some studies have been conducted on the characteristics of gaze movement in SA, there has been no study on the gender characteristics of gaze movement in the SA process. As the number of female workers increases even in large systems, such as nuclear power plants, it is relevant to compare and analyze men and women’s SA and gaze movement characteristics

Although it is not a study on SA, several research results show a difference in the visual characteristics of men and women during information processing. Choi [13] said that there are differences in the visual perception characteristics of men and women in spatial exploration. Kim [14] said that there are differences in the characteristics between men and women from various viewpoints concerning human information processing. Graham et al. [15] stated that, from a cognitive information processing’s perspective, men tend to pay more attention to core information consistent with personal goals, whereas women tend to pay attention to various information types. Therefore, there may be differences according to gender in visual attention to the same object. From this fact, it can be hypothesized that there may be a gender difference in gaze movement in the SA of the system state. Therefore, a study was conducted to compare and analyze men and women’s SA and gaze movement characteristics by making a simulator for emergency scenarios in nuclear power plants.

2. Methods

2.1. Participants

Twenty undergraduate engineering students with no prior experience in control room operations participated in the experiment. They were selected among students from the same engineering major and received prior instruction on accident scenarios and experimental methods before the experiment. Their average age was 24.0 years (SD = 1.72 years). The average age of men was 25.4 years (SD = 0.97), while the average age of women was 22.6 years (SD = 0.97). Participants were required to have normal or corrected-to-normal vision in both eyes.

2.2. Apparatus

2.2.1. Simulator

A desktop computer-based simulator of a nuclear power plant system consisting of three screens was used in this experiment. The three screens consisted of the main steam system screen where the radioactivity level of the secondary system was displayed; the reactor coolant system screen displayed three variables: reactor power (Rx Power), reactor vessel water level (RV water level), and pressure of the primary system (PZR Pr.). The residual heat removal system screen displayed two variables: containment building radiation (CTMT Rad.) and containment building pressure (CTMT Pr.). The display also provides all the necessary information for operators to monitor the system’s state (Figure 1). Flow lines connect the different components of the system (i.e., valves, pump, and tanks), with arrows to indicate flow direction. Gauge values adjacent to each of the valves indicate flow rate, pressure, and temperature.

2.2.2. Eye-Tracker

This experiment utilized Tobii Pro Glasses 2 as an eye-tracking system. Tobii Pro Glasses 2 is a wearable eye-tracking tool that allows researchers to capture truly objective and deep insights into human behavior in any real-world environment. Tobii Pro Glasses 2 consists of a head unit, a recording unit, and controller software. The gaze sampling frequency of Tobii Glass 2 is 100 Hz. Corneal reflection and dark pupil tracking techniques were used in this system. The average binocular accuracy associated with the eye-tracking system is 0.62° (SD = 0.23) in optimal condition (lighting condition: 300 lux, distance to target: 1.5 m, gaze angle: <15°, black target/white background). Accuracy was defined as the average difference between the fixation target location and the measured gaze location on the screen. The precision was 0.05° (SD = 0.10). Precision was defined as the ability of the eye-tracker to reproduce the same gaze point measurement reliably and is calculated via the root mean square (RMS) from the successive data points. The detected gaze, defined as the percentage of gaze samples reported by the eye-tracker during fixation on the target, was 99% (SD = 1.7).

To collect accurate eye-tracking data, Tobii Pro Glasses 2 must be calibrated individually for each participant. During the calibration process, the participant had to wear the head unit while focusing on the center of the calibration target. The calibration target was printed on the supplied calibration cards and calibration stickers. The calibration card was held flat against a wall (otherwise straight) towards the participant. The distance between the participant and the calibration card was between 0.75 and 1.25 m (2.5 and 4.1 feet).

While the participant was looking at the calibration target, he/she tapped the calibrate icon in the bottom right-hand corner of the live-view window. Once the calibration was complete, the results were displayed in the calibration results box.

Participants were asked to make themselves comfortable before the calibration and the session to ensure that they would stay within these bounds. In the experiment with Tobii Glasses 2, no chinrest was used.

In order to analyze the gaze movement, it was analyzed that saccade occurred when the eye movement angle was greater than 100 degrees in 1 s, and fixation occurred when the gaze stayed at a specific point for more than 60 ms.

2.3. Experiment Tasks and Procedure

2.3.1. Experiment Tasks

A participant judges the four accident situations for the experiment through six indicators (Rx power, Rv water level, PZR Pr., CTMT Rad., CTMT Pr., Sec. Rad.) on simulator screens. The changes of the six indicators in four accident situations are shown in

Table 1. Changes of six indicators in four accident situations.

	LOCA	SGTR	SLB	LOV
Rx Power	↓	↓	↓	↓
RV water level	↓	↓	-	-
PZR Pr.	↓	↓	↓	↓
CTMT Rad.	↑	-	-	-
CTMT Pr.	↑	-	↑	-
Sec. Rad.	-	↑	-	-

.

In the four accident situations, the screen’s reactor power (RX Power) starts to drop when an accident occurs. At this time, if the reactor water level (Rv water level) decreases, the containment building radiation (CTMT Rad.) increases, the containment pressure (CTMT Pr.) increases, and the LOCA is judged to have occurred. On the other hand, if the Rv water level decreases and the radioactivity of the secondary system (Sec. Rad) increases, it is judged that SGTR has occurred. If the containment pressure rises, it is judged as an SLB situation. Finally, if there is no change in the other variables other than a drop in the pressure of the primary system, it is considered an LOV situation.

2.3.2. Procedure

The experiment was conducted in a quiet experimental space, and a computer, a simulator, three monitors, and an eye tracker were used as experimental equipment. The three monitors for the experiment were LCD monitors with 24-inch screen size, with the distance between the experimental monitor and the subject being 60 cm.

The experimenter explained the experimental scenario and method to the subject before the experiment. In addition, the subject was instructed to understand the experimental method through the preliminary experiment fully. The experiment was started at the same time as the experimenter’s “start” signal and ended simultaneously as the subject’s “end” signal when the subject had finished judging the accident situation.

The participants judged the accident situation by viewing three screens presenting information on the accident status of the nuclear power plant within 2 min. In the four accident scenarios, the number of trials is 1 time; that is, the task of judging the accident situation 4 times per subject was performed. In order to measure the gaze movement during the experiment for judging an accident, the participants wore an eye tracker throughout the experiment. When the participants judged through the change of indicator variables presented on the screen that there was an accident situation, the experiment was stopped, and the questionnaire was filled out.

The questionnaire was composed of three parts. The first part was to determine the current accident situation, the second part was to ask whether the accident situation was judged after recognizing the change of the variables, and the third part was to measure SA, composed of SART evaluation items with a 7-point Likert scale. In order to eliminate the effect of experiment order, accident situations were presented at random for each subject. The experiment scene is shown in Figure 2.

2.4. Data Processing and Statistical Analysis

2.4.1. Variables

The independent variables in this experiment were gender and the four accident scenarios, which were loss of coolant accident (LOCA), steam generator tube rupture (SGTR), steam line break (SLB), and loss of voltage (LOV). LOCA refers to a loss of coolant that exceeds the coolant replenishment capacity because of the reactor coolant replenishment system. SGTR is caused by damage to pipes because of defects in the steam generator. It refers to an accident in which the heat transfer pipe in the steam generator is damaged, along with the protective wall between the reactor coolant system and the secondary system. SLB refers to an accident in which a pipe located in a containment building of the main steam system is broken. LOV refers to an accident that results in loss of offsite power (LOOP). At this time, an emergency diesel generator cannot be used.

In this experiment, participant performance and gaze entropy were used as dependent variables. Participant performance had two major components: the SA score and the judgment accuracy of the accident scenario type. The judgment accuracy of the accident scenario type was determined by whether the participants correctly identified the accident type. Participant judgments were classified as either correct or incorrect. The percentage of correct judgments was calculated for each participant, while the SA scores were measured by the Situation Awareness Rating Technique (SART), a validated SA measurement technique [4].

2.4.2. Gaze Entropy

A gaze plot and a heat map that express the overall movement of the gaze can be utilized in gaze movement analysis. Gaze plots and heat maps effectively analyze the overall pattern of gaze movement, but they have the disadvantage of being difficult to measure quantitatively [16]. In order to solve this problem, a method for calculating entropy from a gaze plot and a heat map using the concept of information theory has been studied. Entropy in information theory is the average information or uncertainty associated with a choice [17]. This concept of entropy has been adopted to measure system complexity [18,19]. In this context, entropy has been used since the early 1980s to analyze the complexity of gaze movement.

Nevertheless, it is only recently that the concept of gaze entropy has been used in earnest [20]. There are few cases of studying the characteristics of gaze entropy in SA. Merwe et al. [3] studied whether gaze tracking could be a means to determine SA in flight situations and confirmed that pilots’ gaze entropy increased when SA was low. Bhavsar et al. [21] studied a method to quantitatively measure SA using gaze entropy during an abnormal situation in a chemical power plant. They found a significant difference in gaze entropy between the group that succeeded and the group that did not succeed in the task. From the above studies, it was found that gaze entropy tends to decrease when SA is high.

Methods for using entropy for gaze movement analysis can be primarily divided into two: using a gaze plot and using a heat map [18]. The entropy types derived from the gaze plot are Shannon entropy, dwell time entropy, and Markov entropy. The value derived from the heat map is called heat map entropy. This study used Shannon entropy, Markov entropy, dwell time entropy, and heatmap entropy to analyze a participant’s gaze movement. The four gaze entropy measures have complementary characteristics in eye movement analysis, but there was no case in which all four entropy measures were used to analyze SA and gaze movement.

In this study, all four gaze entropies were used to analyze the gaze movements of women and men in SA. The concept and calculation method of the four gaze entropies are as follows.

Shannon entropy is the method that utilizes the number of times the eye stays in the Area Of Interest (AOI) [20,22]. Five AOIs, which are indicators that provide information necessary to determine the four accident situations, Rx Power, RV water level, PZR Pr., CTMT Rad., and CTMT Pressure, were defined.

Shannon entropy is defined as follows [20,22]:

$$H=-{\displaystyle \sum }_{i=1}^n{P}_i{\log }_2{P}_i$$

(1)

In this equation, P_i is the ratio of the relative fixation count divided by the fixation count of the ith AOI by the total fixation count of all AOIs, and n is the number of all AOIs. Therefore, if the value of Shannon entropy obtained from the gaze plot is high, the user’s gaze is distracted; if it is low, it means that the user’s gaze is focused on specific AOIs. However, Shannon entropy does not take into account the eye movement pattern between AOIs.

Markov entropy is determined using a Markov chain model, a stochastic model describing a sequence of possible events where the probability of each event depends only on the state attained in the previous event. Markov entropy is obtained using the state space at a specific point in time and a transition matrix representing the probability of transition between states. Markov entropy is calculated using the time the eye stays on the AOIs and the transition probability between the AOIs [17,23]. It is defined as follows [20,23]:

$$H=-{\displaystyle \sum }_{i=1}^n{P}_i{\displaystyle \sum }_{j\ne 1}{P}_{ij}{\log }_2{P}_{ij}$$

(2)

Here, the state set is defined as the set of initial probabilities of each AOI, and the transition matrix is defined as a matrix of transition probabilities between AOIs. The initial probability P_i of the ith AOI is a value obtained by dividing the fixation count of the ith AOI by the total fixation count of all AOIs. The transition probability P_ij from the ith AOI to the jth AOI is obtained by dividing the number of movements from the ith AOI to the jth AOI by the total number of movements between all AOIs. Markov entropy has the advantage of including information on the direction of eye movement [21]. Markov entropy, like Shannon entropy, has a limitation in that it cannot take into account information about how long the eye stayed at each AOI.

Dwell time entropy is a method to reflect the time stayed on each AOI, and it has the same calculation formula as Shannon entropy. However, P_i is defined as the ratio of the fixation time of the ith AOI to the total fixation time of all AOIs [16].

Heat map entropy is a method of calculating entropy using a Gaussian distribution under the assumption that the heat map has a Gaussian distribution from the point where an eye stays the longest to the point where the eye stays the least. Gu et al. [24] defined the joint probability distribution function of the heat map by assigning different weights according to the fixation duration.

$${\tilde{f}}_{XY}\left(x,y\right)={\displaystyle \sum }_{f=1}^{f_{num}}{d}_f\times \frac{1}{2\pi {\sigma }^2}exp-(\frac{{(x-x_f)}^2+{(y-y_f)}^2}{2{\sigma }^2})$$

(3)

Here, σ is a standard deviation, meaning visual angle in eye tracking, and refers to the range of pixels that the user can perceive when viewing the screen; $f_{num}$ is the number of fixations; and $d_f$ is the weight of each fixation distribution. Using this combined probability distribution function, heatmap entropy can be calculated as follows:

$$H=-{{\displaystyle \sum }}_{xy}{\tilde{f}}_{XY}\left(x,y\right)·\log {\tilde{f}}_{XY}\left(x,y\right)$$

(4)

2.4.3. Situation Awareness

SA is an essential concept in explaining the behavior of the person operating the system. It has been used to evaluate operating behavior in aircraft control, air traffic control, large systems such as chemical processes and nuclear power plants, and tactical and strategic systems such as firefighting and military operation command centers [25].

In particular, the main human–machine interface of the main control room of a computer-based next-generation nuclear reactor has been designed to improve the operator’s SA, which has been used as one of the operator’s performance evaluation criteria in the design suitability evaluation experiment [25]. Situation Awareness Rating Technique (SART), Situation Awareness Global Assessment Technique (SAGAT), and Situation Awareness Control Room Inventory (SACRI) are usual evaluation methods of SA.

SART is a method to evaluate SA through evaluation items according to the degree of attention on system design, the scale of attention on distribution, and the understanding of the situation [26]. SART is one of the self-evaluation methods proposed by Taylor [26], and it was first used in the US Air Force’s air control system. Since then, SART has evaluated SA in many fields such as flight, cyber security, military operations, submarines, virtual reality, and nuclear power plant control rooms [27]. SART consists of three items: demands on attentional resources (D), supply of attentional resources (S), and understanding of the situation (U). SA score is calculated as SA = U − (D − S). The detailed items of the demands on attentional resources include instability, complexity, and variability, and the detailed items of the supply of attentional resources include arousal, spare mental capacity, concentration, and division of attention. The detailed items of the understanding of the situation include information quantity, information quality, and familiarity [4] (

Table 2. SART evaluation form.

		Low						High
		1	2	3	4	5	6	7
Demand	Instability of Situation
	Variability of Situation
	Complexity of Situation
Supply	Arousal
	Spare Mental Capacity
	Concentration
	Division of Attention
Understanding	Information Quantity
	Information Quality
	Familiarity

).

SAGAT and SACRI are other situation awareness measurement methods. SAGAT evaluates SA through comprehensive SA requirements, system function, and external environment characteristics after stopping the experiment at a randomly selected time point [28]. SACRI is a method that combines signal detection theory with the evaluation method of SAGAT. Like SAGAT, it is a method to stop the experiment in the middle of the experiment and evaluate the questionnaire [25].

Compared to the methods of SAGAT and SACRI, SART has the advantage of not interfering with the experiment, because it is a method to evaluate questionnaire items after the experiment is finished. In addition, SART has the advantage that it can be used quickly and easily through minimal prior training [4]. However, SART has a disadvantage in that it relies on individual memory, because the evaluation items are evaluated after the end of the experiment. Nevertheless, it has been an effective method in relatively short experimental scenarios or situations [4].

This study aimed to comparatively analyze the gaze movements of men and women through an experiment to judge the accident situation in a short experimental scenario. In addition, it was essential to evaluate SA quickly and easily through little training because subjects without prior experience in the nuclear system were used. Therefore, SART was used as the evaluation method of SA.

3. Results

3.1. Accident Situation Judgment According to Gender

The rate of correctly judging the accident situations of LOCA, SGTR, SLB, and LOV was found to be 80% for both men (SD = 30.9) and women (SD = 23.0). Therefore, it can be seen that there is no significant difference in the judgment ability of men and women in the task of judging the accident situation from the nuclear power system screen.

As a result of ANOVA on time taken to judge the accident situation, there was no significant difference between men and women (F = 0.009, p = 0.923). Figure 3 is a graph showing the average judgment time between men and women by accident situation. It can be seen that men’s judgment time is slightly longer in LOCA, and women’s judgment time is slightly larger in the rest of the accident situations.

3.2. Situation Awareness According to Gender

Analysis of variance (ANOVA) was performed to identify a significant difference in SA according to gender and accident scenarios. There was no statistically significant difference in the SA score according to gender (F = 0.995, p = 0.322). Additionally, the difference in SA scores according to accident scenarios did not show a statistically significant difference (F = 0.396, p = 0.756), and the interaction effect between gender and accident scenario did not appear statistically significant (F = 0.268, p = 0.849).

The average score of SA for all accident scenarios was 20.5 (SD = 5.2) for men and 19.4 (SD = 4.5) for women. In the LOCA, the male SA score was higher than the female SA score (men = 20.8 ± 4.2, women = 18.5 ± 4.0). In the SGTR, the male and female SA scores were almost equal (men = 19.5 ± 7.6, women = 19.8 ± 4.4). In SLB, the male SA score was slightly higher than the female SA score (men = 19.8 ± 3.6, women = 19.1 ± 5.0). In the LOV, the male SA score was higher than the female SA score (men = 21.9 ± 5.0, women = 20.1 ± 5.2) (Figure 4).

3.3. Gaze Entropy According to Gender

Looking at ANOVA results of gaze entropy according to accident scenario, Shannon entropy (F = 0.702, p = 0.554), dwell time entropy (F = 0.427, p = 0.734), heat map entropy (F = 1.136, p = 0.340), and Markov entropy (F = 1.035, p = 0.382) showed no statistically significant difference.

According to gender, the mean of Shannon entropy (F = 0.118, p = 0.732), dwell time entropy (F = 0.021, p = 0.886), and heat map entropy (F = 0.953, p = 0.332) showed no statistically significant difference. However, there was a significant difference in Markov entropy (F = 5.263, p = 0.025). Figure 5 shows the average male and female gaze entropy by accident scenario. There was not much difference between men and women in Shannon entropy, dwell time entropy, and heat map entropy. However, in the case of Markov entropy, the entropy of women was greater than that of men in all accident scenarios.

The interaction effect between gender and accident scenario according to Shannon entropy (F = 0.356, p = 0.785), dwell time entropy (F = 0.379, p = 0.768), heat map entropy (F = 0.479, p = 0.698), and Markov entropy (F = 0.534, p = 0.660) was not statistically significant in all cases.

3.4. Gaze Movement According to Gender

Through gaze entropy analysis, it was confirmed that there was a significant difference in Markov entropy between men and women, and through this, it was possible to predict that women’s gaze movement was more frequent than men’s. Figure 6 is a gaze plot for the main steam system screen of one male and one female participant. As seen in the figure, women move their gaze between various components more frequently than men.

For further understanding, an ANOVA was performed on the fixation time and the visit counts to AOIs according to gender. The results showed that the fixation time was not statistically significant (F = 0.485, p = 0.488). However, the visit count to AOIs was statistically significant (F = 8.106, p = 0.006). Figure 7 shows the fixation time and the visit count. As can be seen in the figure, women’s fixation time and visit count are larger than men’s, but the difference is significant only in visit count.

4. Conclusions and Discussion

The SA and gaze movement characteristics of men and women were analyzed in this study. SART was used to measure SA, while gaze entropy was used to indicate gaze movement. As a result, there were no gender differences in SA for nuclear power plant accidents. This means that the performance of men and women in judging the accident situation is not significantly different. This study used Shannon entropy, dwell time entropy, Markov entropy, and heat map entropy to analyze gaze movement in the SA process. Men and women did not show significant differences in Shannon entropy, dwell time entropy, and heat map entropy in the SA process. This means that the ratio of staring at the indicators set as AOI in the SA process does not show a significant difference between men and women. On the other hand, there was a significant difference between males and females in the Markov entropy, which considers the number of gazes at AOIs and the number of visual transitions between AOIs. Since the entropy of women is higher, it can be seen that women move their eyes between indicators to judge the accident situation more than men. This means that women move their gaze more frequently than men in order to acquire necessary information. From the point of view of information processing, men pay more attention to core information that meets individual goals, whereas women tend to pay attention to various information types [15,29,30]. Therefore, it is judged that women frequently move their gaze between AOIs to collect various information types.

The results of this study will need to be considered when designing an information system that judges the state of a system given the recent trend of increasing women’s employment. In particular, a system that requires a female worker’s occupation should be designed so that important information can be delivered to the worker and unnecessary gaze movement between system components does not occur.

Finally, this study has some limitations. The experiments were conducted on LOCA, SGTR, SLB, and LOV. Although it is composed of typical accidents that occur in nuclear power plants as much as possible, this accident situation is insufficient to cover all accident situations. There is a possibility that it may have influenced the generalization of the experiment. Thus, it will be necessary to conduct experiments on various other accident situations in the future.

Additionally, this study was conducted on engineering college students. Although they were sufficiently trained in this experiment, the sample’s representativeness decreased, possibly affecting the experiment’s external validity. In future experiments, the experiments could be conducted on the operators of nuclear power plants.