Flight Departure Time Prediction Based on Deep Learning
Round 1
Reviewer 1 Report
The authors present a machine learning model for predicting the departure time of flights, with application towards initiatives such as airport collaborative decision-making (A-CDM) as well as tactical schedule adjustments. In general, the paper is well written, and the results appear to indicate an improvement in predictive capabilities. However, I have a few concerns related to the features used as inputs, the process through which the model was trained, as well as how real-life operators could use such a predictive model.
I would recommend a minor revision -- please see below for my comments.
(1.) Page 3: There should be some discussion of airline-related factors as well, in addition to flight information, airport, and weather factors. For example, delays due to crew scheduling problems also comprise a significant portion of root and propagated delays within the air transportation system. I understand that it might not be possible to obtain such data related to airline factors (e.g., crew scheduling, maintenance, etc.), it would still be good to acknowledge this major impact factor.
(2.) Table 2: I'm a bit confused on how this table is organized and described. For example, I would expect "departure time of the outbound flight" to be a value of time, but then the description says "total number of flights that need to take off during the actual departure time"? There is some inconsistency here, and should be clarified.
(3.) Page 5: It would be good if the authors could provide a citation/reference to these facts regarding min-max scaling, particularly a reference that notes it can achieve similar scaling results as Z-score normalization for Gaussian data.
(4.) Page 5: The parameter adjustment procedure should be much better documented, as this process could significantly affect the characteristics of the resultant network model.
(5.) Section 4 (Conclusion) -- It is clear that the learning technique used and trained by the authors do show good results in terms of predictions on the validation set. However, it is much less clear how the predicted value (i.e., flight plan deviation) should be used by the various stakeholders in this process? First, the authors should provide some quantification of the uncertainty around these results -- for example, say that at 0100Z the predicted flight plan deviation time is 1 hour -- what is the confidence of this effort? Will the deviation have a mean of 1 hour, with some standard deviation-based range around it? Second, how will stakeholders make use of this data? For example, if an airline notes that the projected departure delay for an aircraft is 1-2 hours -- what actions could these algorithms propose, and/or how could they be helpful for the purposes of decision support?
Author Response
please see the attachment
Author Response File: 
Author Response.docx
Reviewer 2 Report
Line 69, RMSE is used before it is defined. It is currently formally defined on line 218. Please define RMSE and related terms before using them.
Line 113, please add a paragraph overviewing the content of section 2
When discussing the weather factor in Section 2.1, it is unclear if referring to weather at the runway surface or weather across different altitude layers. Weather at the surface is often very different than the weather at higher altitudes. While weather factors are not leveraged (lines 146-147,) please elaborate and be more explicit to prevent confusion.
Line 154, the table is difficult to read due to formatting. It appears the 3 columns are equal width, recommend decreasing the width of the “Ranges” column and increasing the width of “Description” to improve readability. Also experiment with left justification for “Feature” and “Description.”
Line 156, be consistent with heading names. This is currently all lower case while section 2.1 capitalizes the first letter of each word.
Line 170, what version of SK-learn and Python were used? Be specific to facilitate traceability and reproducibility. Similarly, on line 198 be specific with the Keras version, as Python 3.7 is already explicitly stated.
Section 2.3 overviews the GRU and LSTM neural networks but the section assumes the reader is familiar with the high level concepts for neutral networks and deep learning. Readers of the Aerospace journal may not be familiar with these concepts, specifically the AI / ML challenge of time series dependence. Please introduce the neural network and deep learning concepts in Section 2.3 and update the literature review accordingly.
Line 208, please add a paragraph overviewing the content of section 3
Line 209, be consistent with heading names. Choose a consistent naming convention.
Lines 210-215, please provide more details on the test and validation sets. Understanding the test and validation sets are fundamental to determining the contribution of the manuscript. Basic statistics such as size or quantity are recommended at a minimum. Stating the ratio of 6:2:2 is helpful but insufficient. To address this feedback, I recommend more than adding a couple sentences about the tests and it is likely a few paragraphs, perhaps even with a table, would be required to sufficiently describe the test and validation sets. As this manuscript emphasizes deep learning and to the point of including it in the title, a high level of rigor is required to describe the fundamental test and validation sets.
Lines 239-244, please improve graph formatting as the axis are difficult to read, particularly Figure 3.
Due to lack of details on the test and validation tests, along with poor readability of the axes, it is difficult to review the results and conclusions.
Author Response
Please see the attachment
Author Response File: Author Response.docx
Reviewer 3 Report
I have three major concerns of this paper.
First, the writing, explanation and presentation of the paper can be significantly improved. For example, the factors to be considered in the model are not clearly explained. Not only the reason why these factors are selected are not explained, but also the meanings of these factors are not clear.
Secondly, it is not clear how the model can be applied in practice. One issue of applying this type of prediction model is that the factors themselves need to be forecasted, which is not easy. At least, the time horizon of the application of the proposed prediction model needs to be specified.
Thirdly, the performance of this prediction model has been compared with other neural network models in the paper, but it is not compared with any other non-neural network model or any other methods.
Author Response
Please see the attachment
Author Response File: Author Response.docx
Round 2
Reviewer 2 Report
Page 1, line 22: Please provide citation to support statement "total amount of civil aviation passengers and cargo has risen sharply." The conclusion also mentions that COVID-19 has impact aviation operations and it is unclear if the ZSNJ overlaps with the COVID-19 pandemic or not.
Page 2, line 70: RMSE and MSE are used before being defined. Please define the acronyms at first use. The RMSE and MAE are defined in the abstract, but they also need to be first defined in the main text too.
Page 2, line 70: RMSE and MSE are used in the introduction, but the results section also uses MAPE. Please include MAPE when discussing RMSE and MSE.
Page 7, table 3: Please update column spacing so "neurons" are not cut in half. Either have "nerons" on the second line or expand the width of the first column.
Page 7, line 294: Please define the ZSNJ airport code. For example, recommend something like "...flight operational data of Nanjing Lukou International Airport (ICAO airport code: ZSNJ) from 2019 to 2020. ZSNJ is one of the busiest civil airports in China and is the primary airport serving Nanjing." Also does the dataset encompass all 24 months of 2019 and 2020? Please be specific on dataset temporal scope.
Page 7, Section 3.1: Recommend including a map or diagram of ZSNJ and the surrounding region.
Page 7, Section 3.1: Please add statement regarding summary statistics of the dataset. For example, how many flight hours or number of operations were included in the ZSNJ dataset from 2019 to 2020? How does the scale and quantity of the ZSNJ dataset compared to others used in the aviation literature?
Page 7, Line 306: What are the units for the 320 and 400? I don't understand what "data volume" is referring to in this context.
Page 8, Line 314: What are the units for the 80 and 100? I don't understand what "data volume" is referring to in this context.
Page 8, Line 322: What are the units for the 80 and 100? I don't understand what "data volume" is referring to in this context.
Page 8, line 329: What is "AQ1033?" It is used before being defined and the manuscript doesn't describe the relationship between AQ1033 and ZSNJ.
Page 8, figure 2: Please update legend to plain text. For example, "Training loss" instead of "train_loss."
Figures 2-9: Please use consistent formatting for figures. The text size and font is not consistent.
Page 13, line 408: Improve comma use in clause...Replace text with "For example, airlines can use model..."
Author Response
Please see the attachment
Author Response File: Author Response.docx
Reviewer 3 Report
First, since all these factors are changing all the time, it would be better for the author to explain a little more how this model can be applied dynamically in practice.
Secondly, the other Neural Network model and RF model applied in this paper can be described in more details. The value of machine-learning based models can be demonstrated better through comparing with other classical prediction models in this field.
Author Response
Please see the attachment
Author Response File: Author Response.docx
