Analysis of Human Factors in Typical Accident Tests of Certain Type Flight Simulator

Abstract

With the improvement of modern aviation equipment manufacturing technology, there are relatively few failures due to the unreliability of the aircraft. However, human factors which resulted in air crashes and unsafe events are raised. In this paper, for many typical accident scenarios of a particular plane, the flight simulation verifications of the pilot’s workload and behavior are carried out on the certain transport category airplane, namely the six-degrees-of-freedom full-motion flight simulator. The subjective and physiological evaluation methods combine to analyze the human factors of pilots in the sudden typical accident scene during a flight mission. The study uses eye trackers, professional heart rate monitors, cameras, and other equipment to collect the pilot’s physiological information during the flight mission, and allows the pilots to fill in the subjective evaluation scale, establishing a subjective and objective evaluation index system. Thus, the human factors of pilots in typical fault situations are analyzed. The analysis shows the combined personal and accurate evaluation method, with the test equipment and environment proposed by this paper being feasible for the human factor evaluation in the accident or incident of transport category airplanes. It will benefit aviation stakeholders in determining the proper action to decrease the workload to an acceptable level.

1. Introduction

Since the invention of aircraft, human beings have accumulated many experiences in aircraft design. With scientific and industrial development, airplanes with a complex system and an interactive platform with the pilot have formed. The developed cockpit, from the original instrument to the modern one with its highly integrated system, as well as the degree of automation, are each increasing. However, aviation safety is still a major problem restricting the development of aviation. Therefore, aviation safety has become a critical factor in the design of civil aircraft.

Furthermore, aircraft airworthiness standards have also emerged, along with flight safety. In aircraft design, designers should continually improve the automation and reliability of aircraft systems. Statistics show that the probability of flight accidents caused by aircraft equipment errors has dropped by about 70% over the past century. However, the human factor, another safety variable associated with aircraft accidents, has gradually become a bottleneck restricting the further improvement of flight safety in the development of the aviation industry [1,2,3]. Human factors refer to the study of human influence and roles in the system of the “human–machine environment”. In real life, industrial and agricultural production and transportation have safety production problems. To analyze these problems systematically, we must start with the “human–machine environment”, in which “human” refers to human factors.

Compared with aviation accidents caused by aircraft mechanical failures, the proportion of aircraft accidents caused by human factors has gradually increased. According to statistics, the investigation of flight accidents in the past 20 years shows that the proportion of flight accidents caused by human error factors is as high as 70–80% of the total flight accidents. In contrast, aviation accidents caused by human factors are more fatal those caused by aircraft failures [4,5]. Taking an example of CAAC data in decades, CAAC has had about 32 flight accidents, 18 of which were caused by the crew, accounting for 56.25% of total accidents. Therefore, human factors research is of great significance in improving aviation safety. The safety of civil aviation is realized in the interaction of human–machine and natural factors. Personal mistakes are inevitable daily and integral to behavior [6,7,8]. Some investigations have found that the flight crew will make two to nine errors every 1 h during the execution of the mission, and each error may lead to different consequences. The human errors of flight crews have become a significant hidden danger affecting the safety of civil aviation flights [9,10]. Therefore, new ideas and trends in human error research mainly include cognitive models and computer-based information technology. These technologies have been fully developed abroad and widely used, such as in the nuclear industry, automobile transportation industry, and other specific operation industries, focusing on load analysis, cognitive mechanism research, and error evaluation of front-line operators. This technology is the promising research direction of human error research. This paper mainly analyzes the load and behavior of pilots through the evaluation system established by the subjective evaluation method, the objective evaluation method, and the principal component analysis in the case of failure during the flight of aircraft, and adopts the data analysis method of the subjective analysis and objective analysis, aiming at finding out the deficiencies in the design layout of the cockpit or in the flight crew operation process that lead to the fact that it is easy for the pilot to make mistakes.

2. Research Methods

Whether the selection of evaluation indicators is appropriate plays a decisive role in the rationality of comprehensive evaluation [11,12,13]. There is not an established method of general indicator system at present, but the following principles should be followed when selecting indicators: (1) Testability: The indicators are required to have clear physical meanings, can be measured by subjective or objective means, and have effective and stable data sources. (2) Comprehensiveness: The indicator system should cover the main characteristics of the evaluation object from multiple aspects. (3) Simplification: Indicators should be few rather than many, and simple rather than complex. They should not only comprehensively summarize the main nature of the evaluation object, but also make the evaluation simple and feasible, saving time and cost. (4) Comparability. The evaluation indicators should be able to represent some commonness of the evaluation objects and distinguish the differences between the evaluation objects so that the evaluation objects can be compared.

2.1. Subjective Evaluation Method

The subjective evaluation method is an analytical method that concludes by analyzing the results of the subjects who filled in the scales by filling in the corresponding scales after completing the relevant homework tasks [9,14,15]. This paper uses the NASA-TLX scale for subjective evaluation. The NASA-TLX scale is a multi-dimensional scale mainly used for the pilots’ self-assessment of the load status of the flight after the flight mission. It is one of the most widely used tools due to its many advantages, such as low cost and easy management. The NASA-TLX scale is divided into six dimensions: cognitive load, physical load, time requirement, performance level, effort level, and frustration level.

As the task difficulty increases, the amount of target information increases, and the operator must process the increasing information within a limited time, which leads to a gradual increase in the pilots’ load level, expressed as a score on the NASA-TLX scale significant changes.

2.2. Objective Evaluation Method

Subjective evaluation is sensitive to assessing the mental load of the entire process, but unpredictable to the extremes of cognitive load. It is also susceptible to individual differences such as memory loss, contextual influences, and cultural differences. It should use continuous objective measurement methods. Measurements of mental load at each stage are used to estimate peak load and thus to evaluate overall load, so subjective evaluation methods are now usually used in combination with physiological measurements [16,17,18]. Physiological measurement technologies reflect changes in pilots’ workload levels. Since the physiological indicators are real-time, this method can also obtain real-time changes in the pilot’s workload. However, because many factors unrelated to workload may also cause changes in a particular physiological index, a single physiological index evaluation method often has limitations. The subjective evaluation method, the Bedford subjective evaluation scale method, and the objective evaluation method have their advantages and regulations, and a single evaluation method cannot achieve the best results. The comprehensive evaluation method has become the development trend in evaluating workload.

2.3. Principal Component Analysis

Principal component analysis (PCA) is a popular algorithm for data compression and data visualization which is widely used in science and industry. However, when studying the use of PCA to analyze the workload of pilots, problems were found as follows:

(1)

The principal components obtained by the PCA algorithm have no corresponding actual meaning. Since each principal component is a linear combination of the original variables, it is difficult to explain the principal component in general, especially those components that are not the first principal component. Using standard PCA to analyze the workload of pilots will bring a lot of inconveniences, and the results can only be explained in the sense of data statistics. Moreover, when dealing with practical problems, the attributes of the principal component often have positive and negative attributes, which reduces the interpretability of the actual meaning of the principal component.

(2)

When the flight task takes a long time, and there are a number of physiological signals and flight data to be processed is large, it takes a long time to calculate the workload of pilots using standard PCA. As more and more physiological signals of pilots can be collected and applied in the experiment and the data dimension is becoming higher and higher, the standard PCA needs a long calculation time, which also brings inconvenience to the research.

In order to solve the above problems, PCA can be considered as a regression optimization problem, called sparse principal component analysis (SPCA), being an algorithm that can solve the interpretative problem and calculation speed of pilot physiological signal principal component analysis. The main reason is that the main part of the data can be highlighted by the coefficients of sparse principal components (the coefficients of most principal components are 0). In this way, the principal components can be more easily explained, and the calculation speed of SPCA for large coefficient data sets is faster.

The algorithm of pilot physiological signal SPCA (Sparse principal component analysis) can be described by the following steps [19,20]:

(1) Select a benchmark. Choose a datum {e_v} and calculate the coordinates (x_iv) for each x_i:

$$x_i\left(t\right)={\displaystyle \sum _{\nu }{x}_{i\nu }}{e}_{\nu }\left(t\right)\begin{array}{cc}& \end{array}i=1,\cdots ,n$$

(1)

(2) Subset. Calculate the sample variance ${\widehat{\sigma }}_v^2=\widehat{\upsilon ar}\left(x_{iv}\right)$. Let $\widehat{I}$ denote the k group index ν corresponding to the largest variance, where k can be pre-specified or chosen based on the dataset.

(3) Dimensionality reduction. Apply PCA (Principal component analysis method) on the selected k-dimension subset to reduce the dimensionality of dataset $\left\{x_{i\nu },\nu \in \widehat{I},i=1,\cdots ,n\right\}$ to obtain the feature vector ${\widehat{\beta }}^j= {\widehat{\beta }}_{\nu }^j,j = 1,\cdots ,k$.

(4) Threshold noise reduction. Filter out noise in the estimated eigenvectors by hard thresholding:

$${\widehat{\rho }}_{\nu }^{*j}{}^{}=\eta H\left({\widehat{\rho }}_{\nu }^j,\delta \right)$$

(2)

The problematic threshold derives from $\eta H\left(x,\delta \right)=xI\left\{\left|x\right|\ge \delta \right\}$. Another method is soft thresholding $\eta H\left(x,\delta \right)=\mathrm{sgn}\left(x\right)\left(\left|x\right|-\delta \right)$, but the complex entry is chosen for this study because it preserves more of the amplitude of the physiological signal. The point choice can be performed by iterative correction or using $\delta ={\widehat{T}}_j\sqrt{2\ln k}$ to estimate ${\widehat{T}}_j$.

(5) Reconstruction. Return to the original signal domain and let

$${\widehat{\beta }}_j\left(t\right)={\displaystyle \sum }_{\nu }{\widehat{\beta }}_{\nu }^{*j}{e}_{\nu }\left(t\right)$$

(3)

3. Test Plan

3.1. Test Equipment

3.1.1. Flight Simulator

According to the FAA FFS standard, the Flight Simulation Technology Research Center of Shanghai University of Engineering Technology (上海工程技术大学) designed and manufactured a certain transport category airplane six-degree-of-freedom full-motion flight simulator, as shown in Figure 1. The real-time multi-channel sound fusion algorithm and frequency response curve compensation algorithm based on four-level parameter conversion is creatively proposed. The main frequency points and characteristic influence parameters in the acceleration power spectral density are extracted by the finite element analysis method. The simulation error of the sound system and the dynamic special effect reaches Level D, realizing the quantitative simulation of the sound and dynamic system simulation of the simulator. A partition parallel processing method based on modular architecture and multithreading technology is proposed. A dual-redundant architecture based on raster graphics and partial rendering technology has been established, a design method of a multi-channel voice communication system based on a “digital matrix” has been invented, and the modeling and simulation of navigation, integrated display, communication, and other core avionics software have been independently completed, solving the problem of the fine modeling of complex wind shear environment.

3.1.2. Physiological Information Equipment

(1)

Glasses eye tracker

This experiment used the Tobii Glasses 2 eye tracker (Figure 2) to measure the pilot’s attention distribution and information perception activities during flight tasks. Tobii Glasses eye tracker is a video-based non-invasive eye detection and tracking technology with a sampling frequency of 50 Hz, and the interval between frames is 20 ms. It calculates the number of frames each glance takes.

(2)

Heart rate watch

This experiment used the Mio Alpha heart rate watch (Figure 3). Based on photoelectric pulse volume wave technology, the watch can measure pilots’ heart rates during flight missions with extremely low interference to reflect the workload fluctuations during the mission.

(3)

RGB-D Camera

In this experiment, the Intel Real sense D455 camera (Figure 4) was used to monitor the pilot’s behavior, and RGB images and depth information were used to detect the pilot’s hand manipulation activities.

3.2. Task Settings

Based on the previous accident scenarios of B737–800, the following four single tasks and one task with double faults are determined. These accident scenarios provide reference and reference value for the study of human factors in the case of sudden typical faults in flight missions and for subsequent aircraft airworthiness certification.

The pilot executes the set flight tasks:

(1)

Type 1 blind descent (Task1): The visibility is about 800 m, the cloud ratio height is 60 m (cloud ratio height: the distance from the runway to the clouds above), and the decision height is 60 m. If the pilot can see the runway clearly at this height, he can land, and if not, he can go around.

(2)

Radio Altitude Fault (Task2): First of all, there will be a significant difference between the radio altimeter values on both sides; secondly, the automatic driving mode of two adjacent channels cannot be used; thirdly, in the process of approaching, the pilot’s flight indicator on one side shows an accidental loss, and the radio altimeter value is displayed incorrectly; finally, when the aircraft takes off, during the approach or go-around process, there may be an abnormal warning prompt. After the aircraft intercepted the glide path, the wireless altimeter would be set to malfunction, and the pilot would handle it according to the operating procedures.

(3)

Wind Shear (Task3): Wind shear is an atmospheric phenomenon and the change of wind vector (wind direction, wind speed) in the horizontal and/or vertical distance in the air. The pilot approached at 300 ft with mild wind shear, and the pilot handled the procedure according to the procedure.

(4)

Fuel Imbalance (Task4): Modern civil aircraft fuel stores in the aircraft’s wing and central wing box. When the difference between the fuel quantity of the left- and right-wing fuel tanks reaches a certain amount, the aircraft may have a fuel imbalance. The cold tank is activated, the takeoff is normal, and the fuel is not equal on the three sides. The pilot checks the QRH and handles it according to the procedure.

(5)

Fuel Imbalance and Radio Altitude Fault (Task5). After the aircraft intercepted the glide path and set the fuel inequity and radio altimeter fault, the pilot handled it according to the QRH.

3.3. Subjects

The test flight crew of this experiment are pilots from Shanghai Airlines: PF-Wang Lei (the company’s captain), PNF-Wang Cheng (co-pilot for the company).

The two pilots are B737-800 pilots in active service, their working hours are the same day, and their physical conditions fully meet the requirements of normal working conditions.

4. Experiment Process

The initial information for this experiment is set, as shown in

Table 1. Experiment setup initial information.

Category	Information
Airport	ZSSS
Track	36R
Landing ILS	IWB110.300
Ground Wind	0.0°/0.0 kt
Runway Visual Range	3000 m
Airport Temperature	15 °C
Period	0:00
Scenes	summer
Aircraft Gross Weight	52,653.0 kg
Zero fuel weight	45,000.0 kg
Watch Speed	0.0 kt
Latitude and Longitude	N31°11.01′/E121°20′
High Air Pressure	21 ft
True/Magnetic Heading	357.15°/2.66°
Weather	Experiments 4 and 5 are moderate rain

. Each experimental operation follows the flow of each practical task. After the investigation, the pilots filled out the NASA-TLX scale.

5. Data Analysis

5.1. Subjective Evaluation Analysis

Figure 5 shows the comparison of NASA TLX among various subjects. Among the five tasks, the fuel imbalance task has the lowest subjective evaluation score, and the wind shear task has the highest score. These include the scores of Type 1 blind landing, radio altitude fault, wind shear fluctuation, fuel imbalance, and RA and fuel imbalance. These two scored lower than the other three tasks, and there is no significant difference between the two.

5.2. Objective Analysis

5.2.1. Statistical Comparison of Gaze Time between Subjects

As can be seen from Figure 6, the Q1 and Q3 values represent the lower and upper, respectively, and Q2 values represent the median of the yellow line. The fluctuations between the Q1 values of the five tasks are small, the difference between the Q2 values of tasks 1, 2, 4, and 5 is slight, and the Q2 value of task 3 is relatively higher. The Q3 and upper limit values of the five tasks are quite different. The Q3 value and the upper limit value of task 3 are much higher than those of tasks 1, 2, 4, and 5. There are two groups of abnormal values in the experimental data of task 3. It can also be seen from the five tasks that the overall time of the Wind Shear is longer than that of the other four tasks, indicating that the subjects have the longest fixation time during the Wind Shear task.

5.2.2. Statistical Comparison of Pupil Diameter between Subjects

Figure 7 shows that the difference between the upper limit of the five tasks is slight, and they all fluctuate around 6 mm. The Q3 values of the five tasks all fluctuate about 5 mm, and the fluctuation degree is not significant. However, Q2 values of Task 1, Task 4, and Task 5 are much higher than those of Task 2 and Task 3, and Q3 values of Task 1, Task 2, and Task 4 are much higher than those of Task 3 and Task 5. The result cannot count the lower limit of pupil diameter under the Wind Shear, Fuel Imbalance and Radio Altitude Fault tasks, so it is expressed in 0 mm.

5.2.3. Statistical Comparison of Saccade Time between Subjects

Figure 8 shows that the Q2 values of the time required for saccades among the five tasks distribute around 4 ms, and the Q1 values are spread around 2 ms. The overall parameter value of each task fluctuates little, and the lower limit of the five tasks is the same. However, the Q3 value and the upper limit of task 2 are much higher than those of the other four tasks, and the Q3 value and the upper limit of the other four tasks are not much different. Meanwhile, the scanning time between subjects is the longest under the Radio Altitude Fault task, while the opposite is valid under the Wind Shear task.

5.2.4. Statistical Analysis of the Overall Action Volume among Flight Crew Subjects

As shown in Figure 9, the lower limit of the five tasks is evenly distributed, and the fluctuations are small, but the Q1 value, Q2 value, Q3 value, and the upper limit of tasks 1 and 2 are far from tasks 3, 4, and 5. The gap between task 1 and task 2 is tiny. In tasks 3, 4, and 5, the difference between the Q1 values and the three is more negligible, while the difference between the Q2 value, Q3 value, and the upper limit of tasks 3, 4, and 5 is more prominent. The overall motor capacity of the unit subjects under the two motions of Wind Shear and Fuel Imbalance is the highest of the five tasks.

5.2.5. Heart Rate Trends across Subjects

The continuous change of the center rate of each flight subject is shown below.

(1) Type 1 blind descent

As we all know, the regular human heart rate is 60 to 100 beats per minute. In flight subject 1, by observing the heart rate change chart (Figure 10), we can find that the pilot’s heart rate is high when the pilot encounters a fault, before gradually stabilizing. It is relatively gentle, and the change of heart rate when there is a fault in the future trends downward.

(2) Radio altitude fault

By observing the heart rate statistics chart (Figure 11), the subject’s heart rate changes sharply when the radio altitude fault occurs for the first time, and the increasing range is extensive, but the heart rate returns to normal quickly. In the subsequent simulated flight failure tasks, the heart rate changes are more gradual than the first time, and all are below the normal range.

(3) Wind shear

From the analysis of the heart rate chart is shown in Figure 12, during the period of 11:12–11:20, the subjects’ heart rates slightly varied and were average. However, from 11:20 to 11:28, the changes of the subjects’ heart rates were pronounced, and the subjects’ heart rates rose sharply after encountering a malfunction. However, after 11:28, when the malfunction occurred again, the subjects’ heart rates changed significantly more slowly than the first time.

(4) Fuel imbalance

From the above statistical analysis of heart rate, the subjects’ heart rates changed significantly when they encountered the fault for the first time, and the heart rate rose sharply between 14:45–14:55 (as shown in Figure 13). The rising value far exceeded the regular human standard heart rate. It took longer for the maximum heart rate to return to normal than for other tasks and was well above average.

(5) Fuel Imbalance and Radio Altitude Fault

The analysis of the heart rate chart, the Figure 14 shows that compared to the performance in other tasks, the heart rate change rate of the subjects after the initial failure is relatively flat. Still, the heart rate value during the period of 15:16–15:18 is far higher. It is higher than the standard human heart rate and takes a long time to return to the average heart rate. Overall, the subjects’ heart rates remained above average for extended periods and decelerated more slowly than the subjects’ performances on other tasks.

6. Case Analysis

In the five experiments performed, the radio altimeter fault was a simple and routine fault with short flight times. However, the current simulator is not designed with light and sound alarms for radio altimeter faults, but only automatically disconnects the autopilot. The radio altimeter fault causes the pilot to deliberately look for the cause of the autopilot disconnection, that is, to constantly scan the cockpit to find the reason. Therefore, in the five experiments, the statistical comparison results show that during the radio altimeter fault, the pilot has the longest scan time of the cockpit (as shown in Figure 8), and the red line is the highest. It shows that after the radio fault occurs, the pilot spends more time finding the cause of the fault, and cannot quickly find the radio altimeter. It is possible to lose situational awareness during this time. Even a scene of the captain and the co-pilot did not monitor or manipulate the aircraft in these experiments. They all looked up on the overhead panel to find faults (as shown in Figure 15, the specific operation object at the time point 0:31.14), which is extremely dangerous, and it is easy to lose sight of the outside world and dangerously approach or even collide with other aircraft or obstacles. In addition, during the test, the pilot needs to do a lot of work, the time is tight, and the task is heavy. If they cannot find the fault of the radio altimeter quickly, it is easy to cause an accident. Please find out the five moments with the most significant amount of pilot operation activity in this topic, as shown in Figure 16. The abnormally high peak value is caused by noise or other personnel invading the screen, which is not considered here. It can see from the picture that the pilots are looking for the cause of the fault while losing forward vision for 31.14 s. This behavior is hazardous during the approach phase.

Through experimental analysis, it is found that during the approach phase, the pilot repeatedly glanced to see the cause of the fault, and it was easy to lose situational awareness.

7. Conclusions

Safety is the eternal theme of civil aviation work and the top priority of civil aviation work. The safety of civil aviation is realized through the interaction between human and natural factors. In sum, in the era of aircraft design and manufacturing technology becoming perfect, the first way to reduce accident rate should be to reduce human error by eliminating or controlling the various possible factors that lead to human factor errors, which is also an important reason for our extensive research on aviation human factors. Therefore, human factors have an important impact on aviation safety, and their importance is beyond doubt. It can make communication between pilots and controllers more reasonable and enable the aviation safety management team to grow. In the aviation process, only through appropriate measures can we ensure the quality and efficiency of aviation safety management and provide passengers with quality services. This paper comprehensively evaluates and analyzes the pilot’s load and behavior during the errors. We found out that there are deficiencies in the cockpit design layout and/or the flight crew operation process that lead to the pilots being prone to make mistakes.

By analyzing the load and behavior of pilots in the case of failure during the flight, the study aims to find out if there are deficiencies in the design layout of the cockpit or in the flight crew operation process that cause pilots to make mistakes easily. The study also provides a reference for the future civil aircraft design department to avoid its design defects or deficiencies, that is, to design “aircrafts suitable for pilots” instead of “aircrafts requiring pilots to adapt”, making the civil aircraft cockpit more humanized, in addition to highly automated.

In future research work, we will accumulate more fault cases to explore more complete fault scenario-operating procedures for pilot handling, reduce the occurrence of unsafe events, and maximize flight safety.