Aircrews, Rules and the Bogeyman: Mapping the Benefits and Fears of Noncompliance
Round 1
Reviewer 1 Report
66-67 - This is too vague an introduction to previous work. More detail needs to be provided to give the background and context, noting it is in the next section.
68 - These should be defined and cited in the context of the literature, such as (Moppett and Shorrock, 2018).
68-69 - Subjective sentiment, there are plenty in aviation safety that believe in the goal of an accident rate of zero (I do not, but that is beside the point, you have made a generalisation that is not true).
69 - There are plenty of pragmatic safety professionals and scientists that have noted the realities of the subject, and this is very true in some Defence contexts, with UAS in particular.
81 - Bogeyman does not need to be defined. If you used the term Baba Yaga fair enough, but in an English language journal this is not needed.
66-83 - I love the narrative flare here, it makes for compelling reading, but I don't think it is an accurate description of pragmatic safety science. There are many accounts of non-adherence to procedures that have resulted in many fatal outcomes. A classic example is the cross feed value for the Metroliner, where pilots in a rush would fill one wing with the required fuel quantity and then once inside open the cross feed value and hope that the quantity would balance to the opposite wing tank before the aircraft left the ground. There were a number of fatalities that resulted from this. It would be good to get a more objective literature review that correctly contextualizes the topic.
143 - Grounded theory is not an analysis tool, it is the methodology. What type of coding did you use, as grounded theory has diverged considerably?
151 - Clearly explain the difference between the 12 interviewees in the previous paragraph and the four noted here.
167 - While I do not doubt that all of the interviewees were male, gendered terms should be avoided.
166-172 - Given this is semistructured, it is essential to have some example follow up questions, and how these varied by interviewee type etc.
177-190 - These are great examples of gamifying regulations, and there has been work done on this in UAS operations, and I am sure in many other aspects of aviation.
193-195 - More details are needed. I suspect there will be more later, but for the narrative flow you should define case 1 to 5 here and then you can refer to them as such later (retrospectively, there are not more later).
195-197 - It is essential that the concept of the "researcher as instrument" be considered here. Some subjective information about how the researchers would answer their own questions would help understand their conclusions. I suspect they are not objective given the choice of the word bogeyman.
200-201 - What was the verification process? How does this impact the findings?
203-204 - Can you confirm this was inductive open coding?
204 - [References] Glaser and Charmaz do not agree with each other on the nature of grounded theory. Given these are your two references, you need to include more information about your approach to grounded theory and where you fall in the landscape. I was going to say spectrum, but there is not just a line of grounded theory.
205-207 - Retrospectively, I feel like this is written to demonstrate the methodological "work-as-prescribed", but does not match the methodological "work-as-done". With a lack of coding structure and evolution, showing saturation, and specifically evidence of the inductive open coding claimed, it is hard to support this is the actual approach.
209-213 - This does not conform to any of the standard grounded theory methodologies.
215-216 - There is insufficient information in general here. While qualitative, Glaser believed grounded theories were objective, and hence conformed to the scientific method. As such, reproducibility applies, and you have not included sufficient information in your methodology for me to reproduce your work. Again, I am sure some of the missing details will be below (retrospectively, they are not), but they are needed here in the methodology. You need to have more information about the nature of follow up questions, also details for the case studies, and much more information about the coding and codes (nodes and themes). Was the coding informed by literature, or was it pure open coding? It would be interesting to see some memos to note how the codes were named and defined and if this influenced by the researcher's expertise or the literature.
Details are also required regarding the development of the coding structure and the coding process. In what order where things coded, and how frequently was the coding structure revised?
229-230 - Is this the correct formatting for the journal for a large quote? Asking only because it does not look pretty.
231 - Again the white space after the quote looks unpleasant. I am hoping you can use italics, without quotes, indented, and not centered.
236-237 - You have not established who each interviewee is in regards to rolls etc. This is another thing missing from the methodology.
256 - This is a wonderful example where the research as instrument needs to be detailed. I read this paragraph including the quote as being supportive of the interviewees' narratives. However, I read the comments and cannot believe that the interviewees have constructed a personal narrative that supports the non-adherence to procedures. As in, they have thrown out the spirit of the law while appealing to the letter of the law, which is not at all the intent, and is why regulations are needlessly complex. But in the end, how the statements and sentiments have been read by the authors and abstracted needs more comments (hence why memos are essential in qualitative research and in particular grounded theory).
600 - "procedures important" should be "procedures as important"?
602 - I think the safety record of military aviation in general is considered less impressive relative to commercial air transport, so not sure how the military nature intensifies this.
606-607 - Again, this is language that appears to support creative non-adherence to procedures, which is risky behavior.
613 - To me the data as presented suggests that employees cannot be trusted in their judgement.
615 - Indented not indented, there is little consistency with the paragraphs.
619-623 - I see the paradox here, but it would be good to point to a clear example, because many of those given before do not lead me to empathize with those doing the deviations, which I too want to label as violations.
625-629 - I do not see the point here.
634-636 - Again I see a need here for the spirit of the regs to be consisdered when working around them. So while there is a need for operational freedom, work arounds that do not adhere to the spirit of the reg are just violations.
641-642 - This is addressed by safety culture, which is organizational wide.
643-652 - I think you need to include this in the introduction as part of the impact/significance. There is clearly a problem that needs a solution.
692-693 - Then how are there values in the spider diagram?
740-742 - The output of a grounded theory study should be theory (a framework etc). Where is that?
755 - Sample size is not a problem if saturation was achieved. However, the representativeness of the sample needs to be addressed as there are no details about the interviewees.
757 - You have provided no proof of saturation. This would come through the evolution of the coding structure as you progress through the case studies and transcripts.
769 - I really like the paper. There is clearly an underlying issue, and that needs to be studied to solve the problem. However, I think there are many fundamental aspects missing for this to be a true grounded theory study, the most obvious of which is the theory, three themes don't make a theory. If you want my advice about proceeding, I would recommend dropping the need to call this a grounded theory study, and use a more general qualitative thematic analysis with triangulation. This avoids many of the issues I have pointed out about the methodology. If you do want to call this a grounded theory, the methodology needs a lot of work.
Refs
2 - Missing information
3 - This bracket is never closed.
15 - Missing information
17 - Missing information
28 - Confirm, potentially missing information
29 - Confirm, potentially missing information
34 - Missing information
Author Response
Please see the attachment
Author Response File: 
Author Response.docx
Reviewer 2 Report
I have no significant comments. In order to improve the quality of the article, I would recommend the authors to address the following minor comments:
Clearly identify the main objective of the study in the abstract and in the Introduction.
Fix description of Figure 1, which has reduced readability.
As a former transport pilot, I support the academic and scientific discussion on the mentioned issue, which aims to strengthen the safety and security agenda in the performance of operations, at an acceptable level of risks and compliance with issued orders and rules for achieving goals.
I recommend the presented manuscript for publication after minor corrections.
Author Response
Please see attachment
Author Response File: Author Response.docx
Reviewer 3 Report
This study presents an interesting topic that requires further inquiry. The review of the literature is written in the style of a term paper in that authors summarize one study at a time. Yet, it is very comprehensive. The methodology of the study is also well described. The instruments and their administration met the guidelines for protecting human subjects. The researchers have the needed skills to administer the interviews and to interpret findings. Results are clearly described and the inductive logic used to produce these results in a qualitative study are explicit. Conclusions are supported by data and include quotes to illustrate points made. The strategies to minimize the researchers’ biases are adequately described. Authors made interesting connections between their findings and the existing literature.
I have a few suggestions to improve this manuscript:
1. Line 44 – isn’t = is not;
2. Target organization (Line 84) = it may be included in the methodology section as “population of interest”;
3. Line 106 – NATO = North Atlantic Treaty Organization;
4. Line 135 = “The current study, building on the previously found enduring differences between the way work is done and prescribed, provides more insight into the reasons these differences remain. This study shows how the achievement of 5GAF behaviour and culture is hampered due to the enduring differences between the way work is done and prescribed” - this statement seems like a finding of the study (???);
5. Line 245 – Interviews – I would suggest a different heading – Procedures (?). Then the authors can describe / explain the “interviews” as well as the “documents study” (with two subheadings);
6. Line 724 - Hale & Borys = Hale AND Borys;
7. Line 739 – Limitations and Strengths – I would suggest a different heading (like the one in line 769;
8. I would suggest the “Limitations and Strengths” section will be placed after the conclusion section of the manuscript;
9. References – Lines 812 – Is this resource a paper? Presentation? Year??.
Most importantly, this project is a worthwhile endeavor. Congratulations on the impressive research study. I recommend this scholarly manuscript should be accepted for publication after authors have addressed my comments.
Author Response
Please see attachement
Author Response File: Author Response.docx
Round 2
Reviewer 1 Report
You use the term climate on page 17, and you mean culture, as in safety culture or just culture.
Otherwise, much improved from the previous version.
Reviewer 2 Report
I recommend the manuscript for publication.
Reviewer 3 Report
