Identification and Modeling Method of Longitudinal Stall Aerodynamic Parameters of Civil Aircraft Based on Improved Kirchhoff Stall Aerodynamic Model
Round 1
Reviewer 1 Report
The authors presented parameter estimation results for a post-stall longitudinal model, however the associated uncertainties were not presented. Therefore, it is not possible to assess the quality of the results. Also, the authors did not conduct validation tests to assess the predictive capability of the model, which is a standard procedure in aircraft system identification (see refs [30] and [32]).
The authors increased the number of independent terms in the pitching moment formulation, thus improving somewhat the fit of experimental data (as shown in Fig. 4), but it does not mean that the final model is better, because the extra terms might be modeling noise or other random components of data, instead of the deterministic component of flight dynamics. Standard errors for each parameter and Validation tests would definitely help in this regard. Hence, it is crucial that the authors present such results to support the claims in the paper.
The main result of the paper is estimation of higher order terms CmX1, CmX2, CmX3, which the authors modeled as a function of the X variable. However, without estimation of standard errors, I suspect the plots in Fig.6 and Fig. 8 simply represent random identification results of an over-parameterized model. For example, it is well-known that the time constant tau-2 is only identifiable for very dynamic stall maneuvers (see, for example, Ref [30]), which is not the case in the present paper. Hence, the identification results are questionable.
Overall, the text is excessively verbose, sometimes repeating the same concept many times. The first half of the paper (Sections 1 and 2) should be more concise.
More specific comments about format and contents are given below: The authors refer to "the state-space model" in general... but the formulation is nonlinear. The original literature about Kirchhoff`s theory employs the term "state-space", because it is related to the 1st order ODE that governs the dynamics of X. However, this reviewer finds this terminology misleading. I suggest "nonlinear model" for the model in Eqs (1) and (19), whereas the "baseline model" (when X=1) could be called "linear model".
The Introduction is too long and the authors keep going back and forth when citing references, and describing the problems. The same issue is related more than once. The Introduction must be made more concise. At the end of Introduction, it is helpful to add a paragraph about the structure of the paper.
The authors criticize splines modeling in: "[...]spline function does not have a definite physical meaning, and its order and knots are different for different aircrafts and flight states, which need to be determined according to the actual fitting. This limits the application of spline function fitting." However, on the same page, the authors themselves use the concept of splines in "Next, the correction coefficients are modeled by spline function". It sounded like a contradiction to me.
On page 11, lines 424-425, the authors claim that it is necessary to provide a smooth transition between low and high Angles of attack. However, the classical Kirchhoff`s formulation already provides this transition. There is no need for this "switch point" in Fig. 10. More information about the test aircraft used should be provided. Ideally, a picture showing the overall aerodynamic configuration.
"making the identification results cannot be combined with the baseline model to form the full flight envelope aerodynamic model" Not accurate. When X=1, the model reduces smoothly to the conventional, linear-in-the-parameters formulation of stability and control derivatives. "Second, its pitching moment model cannot accurately describe the aircraft stall pitching moment characteristics." There is no pitching moment in Kirchhoff's theory, only lift. "In order to carry out airworthiness certification of civil aircraft, it is necessary to establish an accurate high angle of attack (AOA) stall aerodynamic model, which requires flight test data reflecting the aircraft's stall characteristics" This is not entirely correct. For example, the FAA Advisory Circular 120-109A (Stall Prevention and Recovery Training) requires models with "sufficient fidelity", but not necessarily from flight test data. Can the authors provide a reference to support this? "simple handling and good safety" What does this mean? "it has two major defects: First, the structure of the state-space model is obviously different from that of the baseline model. As a result, some parameters of steady aerodynamic characteristics obtained by state-space model identification cannot be compatible with the baseline model [1]." I don't see any mention about this in Ref [1]. As mentioned in a previous comment, when X=1, the model reduces smoothly to the conventional form.
"In references [5-8], the existing state-space model has poor fitting to the measurement values of pitching moment coefficients, which fully illustrates this problem." In my opinion, the matching plots for Cm in References 5,6 and 8 are perfectly acceptable in the context of system identification results. For example, the main unsteady loops due to hysteresis are clearly represented. Perhaps the authors should clarify what is considered a "poor fitting". "the form of drag and pitch moment correction terms in the state-space model is too simple" The terms CmX*(1-X) and CDX*(1-X) are purely empirical, they are not part of Kirchhoff's theory of separated flow. Other empirical terms can be evaluated and added by the analyst. See, for example, Ref. [9]: “Stall Model Identification of a Cessna Citation II from Flight Test Data Using Orthogonal Model Structure Selection,” paper AIAA 2021-1725. https://doi.org/10.2514/6.2021-1725 where the authors analyzed other empirical regressors, such as X, (1-X), max(1/2,X), and ((1+sqrt(X)) /2)^2, in addition to conventional terms (flow angles, body rates, surface deflections). The authors also included second order terms, i.e.products of these terms as candidate regressors for the aerodynamic model. "Obviously, it is difficult to accurately describe the nonlinear variation of pitching moment during stall with the added CmX(1-X) correction term only." Once again, I beg to differ. The matching plots for Cm in References 5,6 and 8 are perfectly acceptable in the context of system identification results. "In order to solve the problem of incompatibility between the baseline model and the identification results of stall aerodynamic parameters obtained through the state-space model" As mentioned in a previous comment, when X=1, the model reduces smoothly to the conventional form. "Finally, a longitudinal stall aerodynamic modeling method used for mathematical simulation of aircraft stall process is developed" A modeling method used for simulation? Please clarify. Figure 1 flow "seperation" --> separation "which constitutes a typical state-space model [28,30]" The addition of X-related terms does not make the model "state-space". This is related to the 1st order ODE that governs the dynamics of X. "when the AOA is small and the air flow on the upper surface of the wing is not separated, the aerodynamic characteristics of the aircraft are linear and steady" This is not accurate... even at low AOA, the "baseline model" still predicts unsteady condition terms, such as derivatives related to pitch rate, or alpha-dot. "When the AOA increases, the value of Cmq(alpha) will change greatly compared with that of Cmq_cr" Greatly? This really depends on the range of AOA analyzed. Kirchhoff`s theory is for unsteady or quasi-steady stalls, but does not account for very large variations of AOA (say, 30, 40, or 50 degrees). Authors should substantiate these sentences with data or literature. There are 2 obvious errors in Eq (19): CLq multiplies alpha-dot, and Cm-alpha-dot multiplies "q". Also, in Eq (1), CLq multiplies alpha-dot. "then close the throttle" please explain what "close" means
Author Response
Attached is our reply to the reviewer's comments.
Author Response File: 
Author Response.docx
Reviewer 2 Report
The authors address an extension and refinement on a state-space based stall model focusing on longitudinal movement.
The following points should be considered prior to publication:
1) Overall: The paper is quite lengthy. It should be checked if some information could be presented in a more concise form.
2) A nomenclature would help to have all symbols and abbreviations at one place while explanations in the text should be kept.
3) Stall flow physics: Regarding Fig. 1 and the corresponding text the stall type refers to trailing-edge stall (flow separation starts at the trailing-edge progressing upstream to the leading-edge until flow separation on the entire upper surface occurs). There should be a clear reasoning and categorization on the type of stall which is considered here. Also, the wording, e.g. "and vortex breakage occurs" should be set in the correct aerodynamic context. What is meant by "vortex breakage" ? The value of X refers to the location of flow separation related to an airfoil section as shown in Fig. 1 (?).
4) Some more information on the "conventional layout large twin-engine jet civil aircraft" may be of interest.
5) The text should be checked for clear wording and typos.
For example: Line 586: Ma0.275~0.30 -> Ma = 0.275 ... 0.30.
5) Conclusions: The section "conclusions" should be readable by itself, i.e. symbols and abbrevaitions used should be explained. For exmaple, starting with "When X=0.95 ..." one does not know the meaning of X (stall onset position on airfoil section ?).
Author Response
Attached is our reply to the reviewer's comments.
Author Response File: Author Response.docx
Round 2
Reviewer 1 Report
In this revised manuscript, the authors have presented the standard deviations for the system identification results. Typically, in aircraft system identification, reasonable standard deviations are around 5% or 10% of the value of identified parameter (although this is simply one of a number of requirements for a good model). However, some standard deviations presented are excessively high (same order of magnitude of the parameters themselves, or higher), which is unacceptable in system identification. It also compromises the main contribution of the paper, which is the data fit improvement shown for Cm in Fig. 4. In my opinion, it is clear that: 1) the model structure used is too complex to be estimated from the type of maneuvers used; and 2) the data fit improvement shown for Cm is due to over-fitting or over-parameterization of the model. This is an interesting paper, and it would be great to have it published, but additional work is required. Additional details are provided below.
The standard deviations shown in Table 1 are very high, indicating that the model is unreliable. For tests 1 thru 5 (in Table 1), the standard deviations (e.g. for a1, tau1, tau2) are on the same order of magnitude of the estimates, or much higher. For a1, on average, the std dev is 37% of the estimate. For tau1 and tau2, the standard deviation is orders of magnitude higher than the estimate. The time constant tau2 was negative for some cases, which is unlikely to be true. This lack of identifiability for some parameters raises concerns about the other parameters. The problem usually propagates to other parameters in the model because there is always some correlation between them, so the other parameters become unreliable as well.
For tests 6 thru 15 (still in Table 1), the authors stated that the standard deviations were greatly reduced, but I don't agree. The standard deviations were reduced for most parameters, but critical parameters such as a1, tau1 and tau2 are still poorly estimated. For a1, on average, the std dev is 50% of the estimate. For tau1, averaging the results yields tau1=12 +/- 80 (for 1 std dev) or tau1=12+/-160 (for 2 std dev, or 95% confidence). For tau2, the average is tau2=3+/-10 (1 sigma) or tau2=3+/-20 (2 sigma), i.e. the standard deviation is one or two orders of magnitude higher than the estimate.
In system identification, when the standard errors are too large, it means that:
1) the data do not contain information about that part of the model, or
2) this part of the model is correlated with another parameter that cannot be properly estimated; or
3) that parameter is not relevant to the modeling problem. However, a1 and tau2 are crucial to Kirchhoff's theory, so (1) or (2) above are probably the cause.
In Ref. [30], which is probably the best reference on the topic of this paper, it is stated:
1) on page 418: "Determination of both the time constants tau1 and tau2 requires highly dynamic stall maneuvers".
2) on page 419: "Flight data with quasi-steady stall would enable estimation of the hysteresis time constant tau2 only."
In the present work at analysis, the authors used quasi-steady stalls only. For this reason, the uncertainties for tau_1 are so high. The lack of identifiability probably contaminated estimation of tau_2 as well, which is typically well estimated from quasi-steady stalls.
It should be noted that tau_2 influences how much aerodynamic hysteresis is developed, or how low the value for X can get. If tau2 is unreliable, all subsequent analysis in the paper that used the minimum value for X, i.e. Xmin, is unfortunately unreliable as well. Section 4.2, especially Fig. 6, must be reconsidered. The uncertainty levels for CDX and CmX1 in Figs. 6a and 6b preclude any type of fitting. The authors are fitting data with high uncertainty, so many types of outcomes are possible.
The authors did not mention how the standard deviations were obtained, but they are most likely related to the diagonal of the covariance matrix. In the output error method the uncertainty estimates are always optimistic (also called Cramer-Rao Lower Bound). See [30], page 111: "The Cramer–Rao lower bound indicates the theoretically maximum achievable accuracy of the estimates". If these uncertainties are already too high, then the model is certainly unreliable.
My suggestions to the authors are:
1) Start with a simpler model, remove tau1, CmX2 and CmX3 from the formulation;
2) (Optional) Concatenate the quasi-stall maneuvers, so that you can obtain a single set of average results directly (but leave a couple stall tests for validation, one abrupt and one smooth, for example);
3) Check the standard deviations and goodness of fit. If all standard deviations are ok (below 10%), increase model complexity with CmX2, check std dev, add CmX3, check std dev.
I believe that (1-X), (1-X)^2 and (1-X^3) are highly correlated, i.e. very similar to each other, so the addition of CmX2 or CmX3 will affect other parameters in the model. But do this as a build up approach and check at each step. If desired, check with tau_1, but there is very little chance of success using quasi-steady stalls only.
4) I don't think it is possible to obtain models for CDX, CmX1,2,3 as a function of X itself. If the authors decide to pursue this, then you need to show that: a) this modeling is necessary (compare with and without, and provide some statistical test of this modeling against a simple constant model); and b) that the curves obtained actually pass through the confidence intervals of each individual point (in this case, assuming that the authors are still treating each stall separately, and not concatenating the maneuvers).
Additional comments:
Page 10, line 380
Eq (22) is not the cost-function for the maximum likelihood method. Please check.
Please try to avoid using the word "obvious" so often. I suggest "clear", "evident", "easily noticed".
Author Response
The reply to the reviewer is attached.
Author Response File: Author Response.docx
Reviewer 2 Report
Authors have addressed the concerns and points raised by the reviewer to a certain extent.
Some doubling of text/sentences should be avoided in the final manuscript used for publication.
Author Response
Thanks to the reviewer's comments, some doubling texts/sentences have been removed from the manuscript.
