Next Article in Journal
Investigation on the Reynolds Number Effect of a Flying Wing Model with Large Sweep Angle and Small Aspect Ratio
Next Article in Special Issue
Dynamic Prediction of Air Traffic Situation in Large-Scale Airspace
Previous Article in Journal
Configuration Stability Analysis for Geocentric Space Gravitational-Wave Observatories
Previous Article in Special Issue
Identification and Quantification of Contributing Factors to the Criticality of Aircraft Loss of Separation
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Assessment of Potential Conflict Detection by the ATCo

1
Department of Aerospace Systems, Air Transport and Airports, Universidad Politécnica de Madrid (UPM), 28040 Madrid, Spain
2
ATM Research and Development Reference Centre (CRIDA), 28022 Madrid, Spain
*
Author to whom correspondence should be addressed.
Aerospace 2022, 9(9), 522; https://doi.org/10.3390/aerospace9090522
Submission received: 21 July 2022 / Revised: 20 August 2022 / Accepted: 13 September 2022 / Published: 17 September 2022
(This article belongs to the Special Issue Advances in Air Traffic and Airspace Control and Management)

Abstract

:
The main goal of this article is to analyse the probability of detecting potential conflicts by the Air Traffic Controller (ATCo). The ATCo ensures the safety of aircraft and one of its main functions is collision avoidance. Collision avoidance is known as separation provision and this term means assuring the safe distance between each aircraft by sides, vertical and longitudinal minimums of separation. The air traffic controller must ensure a high level of airspace capacity. The work performance is related to high demands on individual characteristics, knowledge, skills and, of course, air traffic characteristics. In addition to analysing the probability of detecting potential conflicts, the study of the most influential factors on this safety event is considered of special relevance since the ATCo represents the last executive section of the air traffic control system and failure to detect potential conflicts could lead to a possible infringement of the minimum separation distances between aircraft or even a collision. In order to carry out this approach, Bayesian Networks will be used due to their high predictive capacity. In addition, a dual approach based on knowledge and real operational data provided by an ANSP will be used. These data are one of the great advantages of this study compared to those included in the current literature.

1. Introduction

Current knowledge of the determinants of the occurrence of aircraft-to-aircraft safety incidents is limited, as is the industry’s current ability to predict them [1].
Studies and models analysing or determining the safety of aircraft operations in airspace, and the loss of separation between aircraft, have so far focused on human intervention and human error as the main cause of unsafe situations resulting from loss of separation between aircraft [2,3,4].
Loss of separation incidents between aircraft generally have very low frequencies of occurrence, which makes it very difficult to have a sufficient volume of data to allow relevant statistical analysis. As the SESAR safety reference material points out [5]: “Due to the high level of safety achieved in ATM, relatively little data is available on negative safety outcomes; consequently, it is increasingly difficult to achieve more safety benefits using standard safety approaches”.
Because of this, most studies have been devoted to ex-post analysis and the reconstruction of events leading to loss of separation between aircraft in the air [3]. Some studies have developed analytical models, in which the probability of incidents is predicted from the expected trajectories of future aircraft, calculated from the respective flight plans [4]. However, there are very few references that exploit operational data on a massive scale, i.e., the large volume of data from all those airspace operations that have not resulted in loss of separation [6,7]; nor that allow the incorporation of expert information into predictive models or use data from scenarios or situations with similar characteristics [8]. It is important to report and assess less serious security incidents. Less serious incident data and statistics can provide comprehensive and valuable information, considering that such data are more frequent and the reports generated are simpler and more accessible.
To address all these shortcomings, a combination of reactive, proactive and predictive analytics is required to exploit the potential of safety-related data to provide information on potential hazards and operational risks. In this context, the statistical models used and the safety analyses performed must evolve from the analysis of reactive methods towards a more predictive perspective, which allows predicting the likelihood of future safety events.
It is for this reason that it was decided to develop the model presented here. To do this, it was first necessary to carry out a bibliographical analysis of the state of the art of the statistical models used to date and their predictive capacity.
In recent decades, significant research has been carried out on the use of modern statistical techniques to predict incidents and precursors of safety in other modes of transport, particularly road transport. One such technique that has proven to be highly effective is Bayesian Networks (BN). The relevant academic literature provides interesting examples of the types of aviation-related problems to which Bayesian Networks can be applied. Some examples are [9,10,11,12].
Still in the context of aviation, research is available on predictive modelling or BN to predict safety events, or to help predict safety performance. Some recent studies that provide relevant results are [13,14,15,16,17].
All of these examples of methodological advances in research take an innovative statistical approach. They use Bayesian Networks to develop statistical models for estimating and predicting safety events based on the intrinsic characteristics of the route and sector scenario, traffic and its evolution, and airspace management. These models allow the assessment, determination and prediction of the safety performance of a given airspace and once developed, calibrated and validated, could be extended to other key performance areas in air traffic management, such as capacity, efficiency, etc.
Therefore, one of the most comprehensive alternatives is to employ BN-based models. These models address two common problems associated with predictive safety models: (1) the consideration of regression to the mean (RTM) on the one hand; and (2) the lack of data when there is an insufficient historical period or a very low number of occurrences on the other hand [18]. Regression to the mean is a common bias when assessing a network in terms of accident or safety, as a point or element of the network may have a high number of occurrences in a year and yet be compatible with an acceptable probability distribution for the occurrences. The BN model will allow better estimation of the safety of a part of the air transport system, taking into account not only the number of safety-related occurrences at that location, but also the occurrences observed in similar environments, naturally incorporating expert knowledge of the causes that could have produced them [18].
In all the cases analysed, the ATCo behaviour is difficult to predict or model. In fact, most accidents are the result of human error caused by poor performance or distraction. This is why special relevance is attributed to the ATCo and his performance in detecting potential conflicts in order to avoid the progression of a possible loss of separation [19].
Furthermore, given that these factors and their effects can be very diverse depending on the different airspace considered, it is recommended to calibrate the existing models for each airspace sector, with the aim of developing new, more accurate tools that predict more reliably the safety levels of the airspace considered [14].

1.1. Principles of Bayesian Network Analysis

Bayesian Networks (BN) are statistical tools that represent a set of associated uncertainties on the basis of conditional independence relationships between them.
A BN is a directed acyclic graph in which each node represents a random variable that has an associated conditional probability function. The structure of the BN provides information about the conditional dependence and independence relationships between the variables. These relationships simplify the representation of the joint probability function as the product of the conditional probability functions of each variable [20].
Let U = {X1, X2,..., Xn} be a set of random variables. Formally, a Bayesian Network for U is a pair B = <G, T> in which:
-
G is a directed acyclic graph in which each node represents one of the variables X1, X2, …, Xn, and each arc represents direct dependence relationships between the variables. The direction of the arcs indicates that the variable “pointed to” by the arc depends on the variable at its origin.
-
T is a set of parameters that quantify the network. It contains the probabilities PB(xi|pxi) for each possible value xi of each variable Xi and each possible value pxi of PXi, where the latter denotes the set of parents of Xi in G. Thus, a Bayesian network B defines a unique joint probability distribution over U.
It is important to note that the topology or structure of the network not only provides information on the probabilistic dependencies between variables, but also on the conditional independencies of a variable or set of variables given one or more other variables. Each variable is independent of variables that are not its descendants in the network, given the state of its parent variables.
The inclusion of independence relations in the graph structure itself makes Bayesian networks a good tool for compact knowledge representation (the number of required parameters is reduced). In addition, they provide flexible methods of reasoning based on the propagation of probabilities along the network according to the laws of probability theory.

The Main Reasons for Selecting Bayesian Network Methods

The main reasons why Bayesian Networks have been selected are explained below:
-
Bayesian networks are very useful for capturing and analysing causality and influence relationships. They are very effective for diffusing uncertainty and updating systems with new data. They are also applicable when the structure of the system is too complex. They provide an intuitive and efficient way to represent a large field, making it feasible to model complex systems.
-
Bayesian networks are primarily used to update the probability distribution of the states of hypothetical variables (variables that cannot be directly observed). This probability distribution helps decision-makers determine the appropriate course of action.
-
Bayesian networks provide a convenient and consistent way of expressing uncertainty in uncertainty models and are increasingly used to express knowledge. They are used for qualitative and quantitative modelling of uncertainty and its causes.
-
Due to the conditional dependence of the variables in the network, BNs provide the ability to predict or diagnose (i.e., they can determine impact and causes). BNs are used to model multidirectional forward and backward uncertainties.
BNs can perform qualitative cause-and-effect assessments and can quantitatively update the probability distribution of unobservable variables.
-
Qualitative analysis: Given a scene, the BN graphically represents the causal relationship between the different elements of the scene.
-
Quantitative analysis: Updating the probability distribution. Given the hypothetical variables representing the possible actions and the prior probability distribution, the BN provides the function of updating this probability distribution when new data and information are acquired.

2. Methodology

The proposed methodology aims to obtain a model capable of characterising and predicting the detection of potential conflicts between en-route aircraft (level flight as well as climb and descent) by ATCo. The content of this study can be structured in three blocks:
-
Development of the Bayesian Network Model (BNM). The development of the BNM starts with a characterisation of the safety event in terms of safety dimensions (precursors) and their aggregation. The result of this descriptive analysis, together with prior statistical knowledge, serves to structure the model that could provide statistical representations of the frequency and severity of the safety event.
-
Calibration, tuning and sensitivity analysis of the Bayesian Network Model (BNM). The model proposed in the previous task is fitted and calibrated on real data. The explanatory power of the model and/or of each independent variable and mixed effects is quantified. The sensitivity analysis considering mixed effects allows characterising the safety performance in terms of not only the independent dimensions, but also of their combinations, identifying the prior thresholds of these dimensions that would reduce the frequency of a safety event.
-
Most influential factors in the Bayesian Network Model (BNM). Application of the model to the defined case study or scenario to quantify the most influential factors.

3. Development of the Bayesian Network Model (BNM)

This section will focus on the development of the Bayesian Network aimed and assess the probability of conflict detection by the ATCo.

3.1. Construction of the BN

This network models the ATCo’s ability to detect potential conflicts as a result of short- and medium-term effects. The high-level structure of the network is presented in Figure 1. For this purpose, mainly two concepts are considered:
ATCo alert status. This alert status is assessed on the basis of the workload of the air traffic controller during the hour in which the Closest Point of Approach (CPA) of the aircraft pair occurs. The workload shall consider the traffic in that hour, the trajectories and conditions of this traffic, as well as the actions taken by the ATCo during that period of time [21]. This concept takes into account the medium-term effects that influence the ATCo’s performance. This idea is represented by the blue box at the top left of Figure 1 with the label “Hourly ATCo workload & Situational awareness”. It should be noted that as these are real operational data collected by an ANSP, psychometric factors relating to ATCo are not available. However, there are other variables of great relevance to its behaviour, such as the ATC actions it performed on the traffic.
Scenario and traffic conditions at the time of the event. This concept considers the short-term effects that influence ATCo performance. They are derived from the relative situation of each pair of aircraft at the time at which ATCo searches for potential conflicts. This time t is considered to be the 5 min period in which the event occurs. To account for short-term effects, each hour will be divided into 12 periods of 5 min, and the variables representing the situation of the aircraft pairs will be calculated for each of the 5 min periods. This idea is represented by the blue box at the bottom left of Figure 1 labelled “Traffic complexity at the time of conflict”. The scenario conditions are subdivided into three variables:
o
Traffic conditions.
o
Aircraft performance.
o
Not expected aircraft in the sector.
Thus, the probability of detecting the conflict will depend on the ATCo alert status throughout the hour, and on the complexity of the overall situation in short periods of 5 min, when aircraft pair events occur.
Finally, the probability of the conflict being detected by the ATCo shall be assessed for each aircraft pair and marked with a label “YES/NO–1/0” in the corresponding file. The two possible branches of results are foreseen, as shown in Figure 1.
The outputs of this network could lead to further analysis, e.g., in case the potential conflict is not detected by the controller, the final vertical and horizontal distance in the CPA of the aircraft pair could be assessed [22]. On the other hand, if the ATCo has detected the conflict, the conflict resolution could be assessed [23,24].
Thus, the construction of the network is based on:
-
Network entry: It shall consist of the four inputs already mentioned, composed of the ATCo workload plus the three variables characterising the complexity of the scenario (traffic conditions, aircraft performance and unexpected aircraft).
-
Training data: The network is trained with the conditions of each record.
-
Network exit: The output will be the probability that the ATCo detects the potential conflict. Note that the conflict will be considered detected whenever the air traffic controller takes any action on any of the two aircraft involved.
This high-level approach is incorporated into a single network (Figure 1) in which, ultimately, four main variables directly feed into the probability of a potential conflict being detected by the ATCo. These variables are represented by the four areas of analysis that are considered to be of most relevance in determining whether a controller successfully detects the conflict. Each of the above areas is discussed separately below:

3.1.1. ATCo Alert Status Analysis Area

The state of alertness will depend directly on the workload of the controller during the hour of operation. In this sense, both “saturation” and “relaxation” can be a problem.
Therefore, it is a matter of having an assessment of the workload during the hour in which the event occurs, estimated by the actions performed by the ATCo during that time.
This area of analysis could be treated as a separate network, the construction of which has been based on the following elements:
-
Network entry: Global parameters of the hour, referring to traffic density and ATCo actions during the hour.
-
Training data: Considering the sector conditions, it is trained to identify the ATCo actions in the hour, and in particular the authorisations to resolve a conflict (flight level change, direct to, reroute).
-
Network exit: The workload level, defined in four states:
o
Level 1: Number of high-resolution actions, and high total number of actions.
o
Level 2: Number of medium resolution actions, and total number of actions high.
o
Level 3: Number of medium resolution actions and medium total number of actions.
o
Level 4: Low number of total actions.

3.1.2. Scenario Conditions Analysis Area

This is the assessment of the scenario at the moment when the second aircraft of the pair enters the sector, as this would be the moment from which a conflict is assessed and detected.
This is implemented through three areas of analysis where each output variable would constitute a direct input to determine the probability of the ATCo detecting the conflict. However, the type of input data for each of these can be described in a general way:
-
Network entry: Average of the scenario parameters in the 5 min period identified in time.
-
Parameter calculation: Considering the scenario conditions, training is carried out to evaluate the three parameters that characterise the scenario and which are described below:
o
Traffic conditions: defined by parameters such as instantaneous demand (count of number of aircraft entering the sector), the percentage of regulated aircraft and the distribution of flight levels. All these variables are evaluated at 5 min time intervals.
o
Aircraft performance: Defined by vertical and horizontal speed distributions.
o
Unexpected aircraft in the sector: Both out-of-flow and unexpected aircraft are considered; however, as it was not possible to obtain data related to these flows, only the first variable will be used.
The detailed calculation of the scenario parameters is discussed below.
Traffic conditions: It considers the situation of the demand.
-
Instantaneous demand: Ratio of instantaneous demand to average maximum instantaneous demand on the day in question (maximum shall be 1).
-
% Regulated aircraft: % of regulated aircraft with respect to the total in the period of time in question on a scale of 0–1. The fact that the aircraft are regulated means that the flow will be continuous.
-
% Aircraft changing FL in the period: % of aircraft changing FL relative to the total in the time period in question on a scale of 0–1.
Performances: Considers the difference in aircraft performances at time t.
-
Speed distribution: Complexity occurs when aircraft have differences in speed. Therefore, the speed difference between the two aircraft is compared:
o
% Aircraft with a speed difference greater than 50 knots: % on a scale of 0 to 1.
o
Vertical speed distribution: Complexity occurs when ACs climb or descend with different speeds. Therefore, the difference in speeds between the two aircraft is compared.
o
% of AC with non-zero vertical speed difference: % on a scale of 0–1.
Unplanned aircraft: Considers the complexity contributed by unexpected aircraft or aircraft operating outside the standard flows in the 5 min period.
o
% Unexpected AC: % of aircraft that have changed EOBT time against IOBT.
Figure 2 shows the detailed schematic of the proposed Bayesian network. In this case, it is a three-level distributed network. It represents the selected input variables and the causal relationships between them. The causal relationships, i.e., the arrows connecting the variables, have been established directly by the model and reviewed and completed by expert judgement.

4. Calibration, Fitting and Sensitivity Analysis of the Bayesian Network Model (BNM)

One of the objectives of this project is to try to identify the cause-effect relationship that exists between certain variables or situations, which directly or indirectly affect safety and the detection of potential conflicts by the ATCo.
Therefore, a wide range of factors that may be related to the occurrence of a loss of separation between aircraft have been identified, regardless of whether or not the data are available in later phases of the project.
This study has been developed in three hierarchical levels as the level of detail of the possible cause is deepened [25].
Thus, the different areas of analysis are placed at the first level. These areas try to group the different precursors or potential causal factors according to their nature and, in turn, each causal factor is divided into different parameters that characterise the event.
However, in this section we intend to develop the third level parameters, i.e., the input, training and output variables, in more depth.
Therefore, Table 1 includes a description of the variables of the model as well as an explanation of the discretisation criterion followed for each of them. Discretisation consists of converting the continuous variables into variables grouped by intervals. This step is necessary since most algorithms are optimised for discrete variables.
The discretisation of the variables can be based on statistical characterisation or expert knowledge. The discretisation must ensure that no information is lost or too many states are considered [26].
A possible example of discretisation for each variable is also included for information purposes, as it is considered more visual. However, this discretisation could vary for the same variable depending on the scenario in which it is found.

4.1. Parametric Learning

Figure 3 shows the probabilities observed directly from the data for the network parameters, with particular reference to the Santiago Sector (LECMSAN).
After looking at the directly observed probabilities of the Santiago Sector data, it is worth noting that at the exit node, 95% of the records are in State 1, i.e., they are conflicts that have been detected by the controller. Even so, it should be noted that the ATM system has a considerable number of safety barriers between this detection by the ATCo of a potential conflict and a possible loss of separation.
Another aspect to highlight is that most of the cases of the ATC Workload variable are in State 3 and 4, specifically 82% of them, which indicates that in this sector there is a high workload for the ATCs.
Finally, with regard to the distribution of vertical entry speeds in the sector, it should be noted that in 99% of the cases, the median of this variable per hour is 0, which indicates that in this sector practically all flights are stabilised.

4.2. Sensitivity Analysis

This section shows the results after performing a sensitivity analysis [27] on the Bayesian network. This analysis is carried out with the intention of identifying which variables have a greater influence on the output variable, i.e., conflict detection.
For the sensitivity analysis, the variable “Conflict detected” has been selected as the target variable. Figure 4 shows the results of the sensitivity analysis carried out on the Santiago Sector network.
The variables closest to the target node (on the right-hand side of Figure 4) that are most sensitive to the target node are: “ATC workload”, followed by “No planned AC”, “Performance” and “Traffic conditions”, respectively.
As for the input variables most sensitive to this node, the most important are the percentage of regulated aircraft (“% AC regulated”) and the vertical and horizontal speed distributions at sector entry time (“Proportion Vertical Speed Diff”, “Proportion Horizontal Speed Diff”). The relationship with the “Proportion Vertical Speed Diff” has a great impact due to the “Proportion Horizontal Speed Diff”. Vertical Speed Diff” has a great impact due to the reasons explained in the previous section.
On the other hand, it should also be noted that less sensitive variables are the hourly distribution of flight levels (“Entry FL Distribution”), aircraft over declared capacity (“AC over capacity”), and the mix of actions performed by the ATCo (“Action Mix”). Therefore, it could be considered to remove these variables from the Bayesian network created in order to calibrate it.

5. Discussion: Most Influential Factors in the Bayesian Network Model (BNM)

Once the sensitivity analysis had been carried out, it was considered interesting to evaluate the most influential factors from another perspective [28].
Tornado Analysis measures the impact of each variable or combination of variables and states of the model on a target forecast.
Thus, Figure 5 shows the tornado diagram of network 3 of the LECMSAN sector. It shows the ten variables or combinations of variables that are most sensitive to a change in the output node “Conflict_detected”.
Firstly, it is observed that the combination of variable states most sensitive to a change in the exit node is given by a low aircraft vertical speed difference (“Proportion_Vertical_Speed_Difference” = s1_below_0_35) and that the vertical speed distribution at the sector entry is negative (“Entry_Vertical_Speeds_Distribution” = s1_below_0”).
The number of unplanned aircraft in the sector (“No_Planned_AC” = s3_0_35_up) is also very sensitive to change. Moreover, for the first three cases, an inverse relationship with the change in the output variable is observed. That is, an increase in the output variable implies a decrease in any of these three combinations of variable states.
To summarise, there are variables and states that are repeated throughout the graph, thus establishing themselves as highly relevant variables vis à vis the output node (“Conflict_detected”). These variables refer to the vertical (“Proportion_Vertical_Speed_Difference”) and horizontal (“Proportion_Horizontal_Speed_Difference”) speeds of the pair of aircraft, their performances (“Performances”) and, finally, the number of unplanned aircraft in the sector (“No_Planned_AC”).
As the sensitivity of the variables decreases, the symmetry is lost; in this case, it reflects that a 10% increase in the output variable will have a greater influence than a decrease in the output variable.

Forward Analysis

The last of the analyses carried out on this potential conflict detection probability network is the Forward Analysis. Forward analysis is used to predict the effects, i.e., the level of uncertainty (output-node-child), by establishing the probability distribution of the input nodes.
The forward analyses that have been carried out are based on changing the configuration of the parent nodes of each network. This new configuration consists of establishing specific conditions for two different scenarios. The first scenario taken into account is the case in which the value of the hourly AC inputs is less than 20% of the sector capacity, and the second case for a value of the hourly inputs higher than 90%.
Through data processing, we have obtained the configurations that the parent nodes of the networks would have if these scenarios were to occur. By changing these configurations, the forward analysis allows us to see how these changes propagate to all the nodes in the network.
Comparing Figure 6 and Figure 7, it can be seen that the intermediate nodes have undergone notable changes, as has happened, for example, for the variable representing the entry of aircraft into the sector every 5 min (“Instantaneous_Demand”). Logically, in the case of inflows above 90%, the instantaneous demand has increased in states 3 and 4 compared to the case of inflows below 20%.
Other variables that have also modified the configuration of their states are the distribution of flight levels at the sector entry (“Entry_FL_Distribution”), or the actions that the ATCo performs per hour (“ATC_actions_hour”), for the case of 90% these actions have increased.
However, no variation is observed in the output node (“Conflict_detected”). This is because it has had a variation in its states of 2 per thousand, but the graph shown is not able to capture this variation due to the rounding of the percentages. This order of magnitude in the output node variation is consistent with the rate of error variation for a skilled person performing a complex task. This rate is on the order of one per thousand.

6. Validation

The basic concept in the validation is the goodness of fit of the model. The proposed model is, in effect, a BN structure.
Inspired by classical validation techniques, two different use cases were defined to test the goodness-of-fit of the network. Each use case refers to a specific sector of the Spanish airspace (LECMSAN and LECBCCC, respectively). Furthermore, each use case was divided into two scenarios; these scenarios represent extreme conditions of sector occupancy rates, therefore both high levels of sector occupancy (90%, 80% and 70% occupancy) and low levels (40%, 30% and 20% occupancy) will be taken into account.
For each scenario, a representative dataset was separated from the learning data and subsequently used to test the predictive ability of the BN.
The predicted results of the BN (predicted success probability of the ATM barrier) are compared with the actual ones to determine the validity of the statistical approach (Table 2).
The ATM safety barrier concerning the potential conflict identification by the ATCO is divided into three columns. The first refers to the percentage value provided by the data, the second to the value predicted by the BN model, and the third reflects the difference between the two values or the error.
Thus, the most important way of determining whether a model is good is ensuring that the difference between the actual and the predicted is as small as possible, i.e., the error is as small as possible.
In addition, the box plot has been selected as appropriate for this purpose, as this diagram is suitable for comparing the range and distribution of numerical data sets, illustrated by a box and a central line in the middle. The chart is suitable for comparing the range and distribution in numerical data sets.
The first thing to note, which can already be seen in Figure 8, is that the errors are very small (less than 1.5%), considering that they are expressed as a percentage.
Furthermore, the plot reflects little dispersion of the data, there are no outliers and the extreme values are close to the mean. It is for this reason that the model is considered tested and valid.

7. Conclusions

This article has dealt with the highly relevant issue of analysing the probability that the ATCo will detect a potential conflict and thus prevent it from becoming a possible Separation Minima Infringement.
For this purpose, a model based on data and knowledge was developed using Bayesian Inference techniques due to its high statistical capacity in low probability events. In this way, the Bayesian network is constructed by defining all the variables considered influential for the defined safety event.
The differentiating element of this study in comparison with previous analyses is the large number of variables identified for which real data have been obtained to support the study, specifically the variables related to ATCo performance, since they are not easily accessible.
Thus, after carrying out the parametric analysis, it was decided to perform a sensitivity analysis on the potential conflict estimation network by the ATCo in order to clarify which precursors have a greater explanatory capacity for this safety event.
In this way, conclusions can be drawn, among which the high influence of the ATCo’s workload on the detection of potential conflicts stands out. In other words, in more saturated sectors or with a higher traffic complexity [28], it is less likely that the ATCo has the capacity to detect all potential conflicts, thus having to resort to other ATM safety barriers to avoid the progression of an accident.
As explained above, the complexity of the sector is another aspect to be taken into account. This is why there are variables such as the % of regulated ACs or the differences in vertical and horizontal speeds of the ACs at the entrance of the sector that also show a highly explainability on the target node.
It was also intended to evaluate the influence of combinations of variables on the influence of potential conflict detection. For this purpose, tornado analysis was used. This analysis shows, for example, the high influence of the difference between the difference in vertical velocities and their distribution of aircraft at the entrance to the sector in question.
Finally, a forward analysis is developed to assess the change in the distribution of the exit node that would result from a change in the entry nodes. To this end, two scenarios are selected, one with high occupancy and one with low occupancy, in order to be able to compare the output distribution of the safety event. Thus, it is observed that although there are significant changes in the intermediate variables of the model, these are not transmitted to the distribution of the output variable.
It is considered that a very complete model has been developed, which incorporates variables that had not been taken into account until now and that allow clarifying some of the most notable variables when analysing the performance of the ATCo in detecting potential conflicts.

Author Contributions

Conceptualization, R.D.-A.J. and V.F.G.C.; methodology, R.D.-A.J. and R.M.A.V.; software, F.P.M.; validation, M.Z.S. and C.E.V.G.; formal analysis, R.D.-A.J.; investigation, R.D.-A.J.; resources, V.F.G.C.; data curation, R.D.-A.J.; writing—original draft preparation, R.D.-A.J.; writing—review and editing, R.M.A.V.; visualization, F.P.M.; supervision, C.E.V.G.; project administration, R.D.-A.J.; funding acquisition, C.E.V.G. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by SESAR Joint Undertaking under the European Unions’s Horizon 2020 research and innovation program, grant agreement number 892542.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

The data used to support the findings of this study are available from CRIDA Data WareHouse (DWH), upon request.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. FARO Consortium. D2.1 Project Scope. SESAR 3 Joint Undertaking under the European Union’s Horizon 2020. Available online: https://faro-h2020.eu/downloads/ (accessed on 20 July 2022).
  2. Eurocontrol. ESARR Advisory Material/Guidance Document (EAM/GUI) EAM 2/GUI 8; Eurocontrol: Brussels, Belgium, 2005. [Google Scholar]
  3. Valdés, R.A.; Comendador, F.G. New European Union Approach to Civil Aviation Accident Investigation and Prevention. J. Aircr. 2011, 48, 1396–1404. [Google Scholar] [CrossRef]
  4. Kuchar, J.; Yang, L. A review of conflict detection and resolution modeling methods. IEEE Trans. Intell. Transp. Syst. 2000, 1, 179–189. [Google Scholar] [CrossRef]
  5. SESAR Joint Undertaking. SESAR 2020 Safety Reference Material, Brussels, Belgium. 2018. Available online: https://www.sesarju.eu/sites/default/files/documents/transversal/SESAR2020%20Safety%20Reference%20Material%20Ed%2000_04_01_1%20(1_0).pdf (accessed on 20 July 2022).
  6. Nazeri, Z.; Barbara, D.; Jong, K.D.; Donohue, G.; Sherry, L. Contrast-Set Mining of Aircraft. In Proceedings of the Industrial Conference on Data Mining, Leipzig, Germany, 16–18 July 2008; Springer: Berlin/Heidelberg, Germany, 2008; pp. 313–322. [Google Scholar]
  7. Oztekin, A. A Data-Driven Quantitative Framework for Safety Assessment of Air Traffic Control System. In Proceedings of the 15th AIAA Aviation Technology, Integration, and Operations Conference, Dallas, TX, USA, 22–26 June 2015. [Google Scholar] [CrossRef]
  8. Verstraeten, J.; van Baren, G.B.; Wever, R. The Risk Observatory: Developing an Aviation Safety. J. Saf. Stud. 2016, 2, 91. [Google Scholar] [CrossRef]
  9. Ayra, E.S.; Ríos Insua, D.; Cano, J. Bayesian Network for Managing Runway Overruns. J. Aerosp. Inf. Syst. 2019, 16, 546–558. [Google Scholar]
  10. Miaou, S.P.; Lord, D. Modeling traffic crash-flow relationships for intersections: Dispersion. J. Transp. Res. 2003, 1840, 31–40. [Google Scholar]
  11. Valdés, R.M.A.; Comendador, V.F.G.; Sanz, L.P.; Sanz, A.R. Prediction of aircraft safety incidents using Bayesian inference and hierarchical structures. Saf. Sci. 2018, 104, 216–230. [Google Scholar] [CrossRef]
  12. Hauer, E.; Harwood, D.W.; Council, F.M.; Griffith, M.S. Estimating Safety by the Empirical Bayes Method: A Tutorial. Transp. Res. Rec. J. Transp. Res. Board 2002, 1784, 126–131. [Google Scholar] [CrossRef]
  13. Valdés, R.A.; Comendador, V.F.G.; Castan, J.A.P.; Sanz, A.R.; Sanz, L.P.; Nieto, F.J.S.; Aira, E.S. Bayesian inference in Safety Compliance Assessment under. Saf. Sci. 2019, 116, 188–195. [Google Scholar]
  14. Heydari, S.; Miranda-Moreno, L.F.; Lord, D.; Fu, L. Bayesian Methodology to Estimate and Update SPF Parameters under Limited Data Conditions: A Sensitivity Analysis. Accid. Anal. Prev. 2014. [Google Scholar] [CrossRef] [PubMed]
  15. Acarbay, C.; Kiyak, E. Risk mitigation in unstabilized approach with fuzzy Bayesian bow-tie analysis. Aircr. Eng. Aerosp. Technol. 2020, 92, 1513–1521. [Google Scholar] [CrossRef]
  16. Bujor, A.; Ranieri, A. Assessing Air Traffic Management Performance Interdependencies through Bayesian Networks: Preliminary Applications and Results. J. Traffic Transp. Eng. 2016, 4, 34–48. [Google Scholar] [CrossRef]
  17. Comendador, V.F.G.; Valdés, R.M.A.; Diaz, M.V.; Parla, E.P.; Zheng, D. Bayesian Network Modelling of ATC Complexity Metrics for Future SESAR Demand and Capacity Balance Solutions. Entropy 2019, 21, 379. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  18. CANSO. Incidents Investigation Toolbox. 2021. Available online: https://canso.fra1.digitaloceanspaces.com/uploads/2021/04/CANSO-Incidents-Investigation-Toolbox.pdf (accessed on 10 February 2022).
  19. Santiesteban Rojas, J.C.; Utria Pérez, D.; Hernández Reyes, C.E. Definition of Bayesian Networks and their applications. Vinculando J. 2012. Available online: https://www.bu.edu/sph/files/2014/05/bayesian-networks-final.pdf (accessed on 20 July 2022).
  20. Netjasov, F.; Crnogorac, D.; Pavlović, G. Potential safety occurrences as indicators of air traffic management safety performance: A network based simulation model. Transp. Res. Part C Emerg. Technol. 2019, 102, 490–508. [Google Scholar] [CrossRef]
  21. ICAO. Application of Separation Minima; ICAO: Montreal, QC, Canada, 2020. [Google Scholar]
  22. FARO Consortium. D4.1 Safety Performance Functions Methodology. 2021. Available online: https://faro-h2020.eu/wp-content/uploads/2021/12/D41-Safety_Performance_Functions_Methodology_v00.01.01_Public.pdf (accessed on 20 July 2022).
  23. Jurado, R.D.A.; Comendador, V.F.G.; Suárez, M.Z.; Moreno, F.P.; Gallego, C.E.V.; Valdes, R.M.A. Safety performance functions to predict separation minima infringements in en-route airspace. In Aircraft Engineering and Aerospace Technology; Emerald Publishing Limited: Bingley, England, 2022; p. 10. ISSN 1748-8842. [Google Scholar] [CrossRef]
  24. FARO Consortium. D6.2—Validation Report. SESAR 3 Joint Undertaking under the European Union’s Horizon 2020. Available online: https://faro-h2020.eu/downloads/ (accessed on 20 July 2022).
  25. Buselli, I.; Oneto, L.; Dambra, C.; Gallego, C.V.; Martınez, M.G.; Smoker, A.; Ike, N.; Martino, P.R.; Pejovic, T. Natural Language Processing and Data-Driven Methods for Aviation Safety and Resilience: From Extant Knowledge to Potential Precursors. In Proceedings of the 11th SESAR Innovation Days, Virtual, 7–9 December 2021. [Google Scholar]
  26. Sensitivity Analysis in Bayesian Networks (and Influence Diagrams). 2016. Available online: https://support.bayesfusion.com/docs/GeNIe/bn_sensitivitybn.html (accessed on 15 December 2021).
  27. Borgonovo, E. Sensitivity Analysis: An Introduction for the Management Scientist; International Series in Operations Research & Management Science; Springer: New York, NY, USA, 2019. [Google Scholar]
  28. Pejovic, T.; Netjasov, F.; Crnogorac, D. Relationship between Air Traffic Demand, Safety and Complexity in High-Density Airspace in Europe. In Risk Assessment in Air Traffic Management; IntechOpen: London, UK, 2020. [Google Scholar] [CrossRef] [Green Version]
Figure 1. Conflict detection assessment network.
Figure 1. Conflict detection assessment network.
Aerospace 09 00522 g001
Figure 2. Conflict identification BN.
Figure 2. Conflict identification BN.
Aerospace 09 00522 g002
Figure 3. Probabilities directly observed from data, LECMSAN.
Figure 3. Probabilities directly observed from data, LECMSAN.
Aerospace 09 00522 g003
Figure 4. Sensitivity Analysis.
Figure 4. Sensitivity Analysis.
Aerospace 09 00522 g004
Figure 5. Tornado diagram.
Figure 5. Tornado diagram.
Aerospace 09 00522 g005
Figure 6. Forward analysis LECMSAN below 20% of its capacity.
Figure 6. Forward analysis LECMSAN below 20% of its capacity.
Aerospace 09 00522 g006
Figure 7. Forward analysis LECMSAN greater than 90% of its capacity.
Figure 7. Forward analysis LECMSAN greater than 90% of its capacity.
Aerospace 09 00522 g007
Figure 8. Box plot of the validation error.
Figure 8. Box plot of the validation error.
Aerospace 09 00522 g008
Table 1. Model variables, description and discretisation criterion.
Table 1. Model variables, description and discretisation criterion.
ParameterDescriptionDiscretisation CriteriaDiscretisation Example
OccupancyOccupation Sector data at the AC entry hourIt is divided into 4 intervals that represent the percentage of occupation in the sector at the AC entry hour respect to the maximum value detected in the sector. The values of the intervals depend on the sector data.Value < 40%
40% < value < 65%
65% < value < 85%
85% < value
EntryEntries Sector data at the AC entry hourIt is divided into 4 intervals that represent the percentage of entries in the sector respect to the sector entry declared value at the AC entry hour. The values of the intervals depend on the sector data.Value < 55%
55% < value < 65%
65% < value < 80%
80% < value
Instant demandAc entry data in the sector in the 5 min periodIt is divided into 4 intervals that represent the percentage respect to the maximum instantaneous demand value. The values of the intervals depend on the sector data.Value < 60%
60% < value < 90%
90% < value < 120%
120% < value
AC above capacityPercentage of aircraft exceeding declared capacity per hourIt is divided into two states, when the percentage of AC above capacity is equal to 0 and when it is greater than 0Value = 0
Value > 0
Entry FL DistributionCalculation of the median flight level with which AC enter the sector at the entry hourIt is divided into three intervals that represent the entry FL distribution in the sectorValue < 350
350 < Value < 360
Value > 360
Horizontal Speeds Differences DistributionCalculation of the median horizontal speeds with which AC enter the sector at the entry hourIt is divided into three intervals that represent the entry Horizontal Speed Difference Distribution in the sectorValue < 35
35 < Value < 55
Value > 55
Entry Vertical Speeds DistributionCalculation of the median vertical speeds with which AC enter the sector at the entry hour. It is calculated in absolute valueIt is divided into two states, when the median of entry Vertical speed is 0 and when it is greater than 0. This discretisation is intended to represent the complexity for the controller, modelling the AC that are ascending or descending.Value = 0
Value > 0
Changes in FL distributionCalculation of the median percentage of AC that change their FL in the sector at the entry hourIt is divided into three intervals, that depends on the sector data.Value = 0
0% < Value < 15%
Value > 15%
Delay DistributionCalculation of the median time delayed of AC at the entry hour into the sectorIt is divided into four intervals that represent the delay time (seconds) in the sector. These intervals depend on the sector dataValue = 0
0 < Value < 240
240 < Value < 420
420 < Value
% Regulated ACCalculation of the percentage of AC that have some regulation at the entry hourIt is divided into four intervals. These intervals depend on the sector dataValue < 15%
15% < Value < 25%
25% < Value < 35%
35% < Value
% Delayed ACCalculation of the percentage of AC that have some delay at the entry hourIt is divided into four intervals. These intervals depend on the sector dataValue < 35%
35% < Value < 50%
50% < Value < 65%
65% < Value
ATCo actions hourCalculation of the total actions that the controller has given in one hour in the sectorIt is divided into three intervals, which are intended to represent the workload of ATCo according to the action they have undertaken. These intervals depend on the sector dataValue < 75
75 < Value < 100
Value > 100
Action MIXSum of resolution actions of any kind. This variable considers flight level changes, vectors and directs.Resolution actions are added, and the same three categories are definedValue < 0.5
0.5 < Value < 1
Value > 1
% AC that change FL in 5 minDistribution of aircraft by number of flight levels that changeIt is divided into three intervalsValue = 0%
0% < Value < 70%
70% < Value
Proportion Vertical Speed DifferenceCalculation of the percentage of AC with vertical speed difference other than 0, on a scale of 0–1It is divided into three intervals that represent the complexity when ACs are ascending or descending with different speeds.Value < 0.35
0.35 < Value < 0.7
0.7 < Value
Proportion Horizontal Speed DifferenceCalculation of the percentage of AC with horizontal speed difference greater than 50 knots: on a scale of 0–1It is divided into three intervals that represent the complexity when ACs have speed differenceValue < 0.35
0.35 < Value < 0.7
0.7 < Value
ATC WorkloadCombination of total actions with resolution actionsThis output node is divided into four states, as defined in the first sectionsLevel 1
Level 2
Level 3
Level 4
Traffic ConditionsRelative situation of each aircraft pairThis output node is divided into three states, in order to model the traffic conditions at the conflict momentValue < 0.5
0.5 < Value < 0.75
0.75 < Value
No Planned ACUnexpected aircraft or aircraft operating outside standard flows in the 5 min periodThis output node is divided into three states from 0 to 1Value < 0.28
0.28 < Value < 0.35
0.35 < Value
PerformanceConsiders the difference in aircraft performances at time tThis output node is divided into three states from 0 to 1Value < 0.35
0.35 < Value < 0.7
0.7 < Value
Conflict identificationIt is considered that there is a detected conflict according to the conditions set out in the first section of this nodeIt is divided into 2 states, 0 (when there is not a detection of the conflict) and 1 (when the conflict is detected)Value = 0
Value = 1
Table 2. Bayesian Network validation.
Table 2. Bayesian Network validation.
Use Case 1Use Case 2
OccupancyActual DataModel OutputsErrorActual DataModel OutputsError
Scenary 1-High Occupancy90%93.527%93.860%0.333%97.198%95.915%1.283%
80%95.165%93.856%1.309%96.706%95.815%0.891%
70%92.374%93.845%1.471%96.642%95.824%0.818%
Scenary 2-Low Occupancy40%94.584%93.959%0.625%96.381%95.917%0.464%
30%93.434%93.948%0.514%95.946%95.913%0.033%
20%95.235%94.153%1.082%95.879%95.918%0.039%
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Delgado-Aguilera Jurado, R.; Gómez Comendador, V.F.; Zamarreño Suárez, M.; Pérez Moreno, F.; Verdonk Gallego, C.E.; Arnaldo Valdés, R.M. Assessment of Potential Conflict Detection by the ATCo. Aerospace 2022, 9, 522. https://doi.org/10.3390/aerospace9090522

AMA Style

Delgado-Aguilera Jurado R, Gómez Comendador VF, Zamarreño Suárez M, Pérez Moreno F, Verdonk Gallego CE, Arnaldo Valdés RM. Assessment of Potential Conflict Detection by the ATCo. Aerospace. 2022; 9(9):522. https://doi.org/10.3390/aerospace9090522

Chicago/Turabian Style

Delgado-Aguilera Jurado, Raquel, Víctor Fernando Gómez Comendador, María Zamarreño Suárez, Francisco Pérez Moreno, Christian Eduardo Verdonk Gallego, and Rosa María Arnaldo Valdés. 2022. "Assessment of Potential Conflict Detection by the ATCo" Aerospace 9, no. 9: 522. https://doi.org/10.3390/aerospace9090522

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop