Research on MASS Collision Avoidance in Complex Waters Based on Deep Reinforcement Learning
Round 1
Reviewer 1 Report (Previous Reviewer 1)
Thank you for the extended revision. I have just got one recommendation: The section from line 73 to 86 could possibly be removed. It does not add much to the paper.
Author Response
Thank you very much for your constructive comments, which will help us greatly improve the quality of manuscripts. We have removed lines 73 to 86.
Reviewer 2 Report (Previous Reviewer 2)
This paper proposed the MASS's collision avoidance decision model based on the Dyna-DQN model.
The mathematical model is properly formulated and verified based on the numerical experiments as case study.
The revision points are properly modified.
Author Response
Thank you sincerely for your careful review and comments!
This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.
Round 1
Reviewer 1 Report
The paper describes an approach for path planning of autonomous ships, using variants of reinforcement learning. 3 examples of different complexity using actual AIS data are presented and discussed.
In terms of presentation of the paper, I have got the following recommendations:
1) In general the paper needs a review in terms of language. Expressions such as (just examples, there are more) "unmanned development of ships", "multi-ship at sea", "when ships meet nearby are easy to fail", "millions of odors", ... should be reconsidered. This will substantially improve the readability of the paper.
2) Also grammar should be checked, because currently the interpretation of the paper is difficult, e.g. "the COLREGS need to be considered, the target ship is assumed to keep course and speed in the set multi-ship encounter scenario, and only altering is considered a collision avoidance measure.", ...
3) There is a whole page related to hull design, CAD and NURBS (Lines 99 to 133) that seem to be completely unrelated to the rest of the paper.
4) The section from line 142 to 299 is poorly structured. It starts with a list of generic terms (AG, VO, FL, ...) that are not put into context. Only later it becomes clear that these items are being discussed one after each other. A bit of sub-structure might help here.
5) The section headers 3.1, 3.1.1, 3.2 seem to be messed up.
In terms of technical content, I have got the following recommendations:
6) In section 1 the authors analyse the state of the art and a whole range of algorithms at length. In line 248 conclusions are drawn about the main problems of decision making for MASS collision avoidance. However, the analysis before does not really relate to these conclusions.
7) I consider reference [35] and the conclusions drawn from it, particularly critical.
7a) I am not sure whether [35] is a good reference to justify that MASS should deviate from the COLREGs. There are other papers, e.g.,
Porathe, Thomas. (2019). Maritime Autonomous Surface Ships (MASS) and the COLREGS: Do We Need Quantified Rules Or Is “the Ordinary Practice of Seamen” Specific Enough?. TransNav, the International Journal on Marine Navigation and Safety of Sea Transportation. 13. 511-518. 10.12716/1001.13.03.04.
that particularly address this topic and that reach other conclusions. Their main argument being that MASS must behave predictably. Not all ships are equipped with AIS and e.g. based on a radar signal it is not obvious whether a ship will follow MASS-specific COLREGs or not.
7b) The MASS-specific variation of the COLREG, that aims at "MASS will avoid all other ships" also substantially simplifies the approach, because then there is no need to consider the reactions of other boats. They are just moving obstacles. It should be analysed in depth how the proposed approach would work under the existing COLREGs.
8) It is said that "The action space is the difference between the own ship's course that can be safely navigated in this encounter situation and the course of the last moment." It is not clear whether "last moment" refers to the previous time instance, or to the "Last Moment Manoeuvre for Collision Avoidance". If the latter is meant, then the action space should be reconsidered, because the MASS's actions cannot be on the edge of forcing a right-of-way boat into a "last moment manoeuvre".
9) In section 3.2.3 it is mentioned that there is a "finite convex area of two-dimensional sea-level", that seems to correspond to the navigable area. It has to be justified why this area is convex.
10) Figure 9 does not really provide much information and could be omitted.
11) MMSI is "Maritime Mobile Service Identities" not "Identify"
12) The results are not compared to any other method. It is thus not clear along which metrics the paper intends to make a progress over existing methods. Such comparison would be required.
Reviewer 2 Report
This paper proposed the MASS's collision avoidance decision model based on the Dyna-DQN model.
The mathematical model is properly formulated and verified based on the numerical experiments as case study.
The citation format in References should be standardized as shown in the guidance.
For example,in paper 30 the volume number and page number are missing.
Reviewer 3 Report
In the manuscript jmse-2261657, the authors made an interesting attempt to better planning of collision-avoidance paths of autonomous ships using reinforcement learning. Despite at first glance, the article seems to be valuable in increasing ship safety, the findings, research motivation, and scientific soundness are doubtful. Some specific comments are as follows:
1. The research motivation is blurry and should be clearly stated. Please reorganize the Introduction, to clearly present the research motivation by highlighting the existing research gap and the need to bridge this gap. Currently, the Introduction is too extensive, and the literature background mixed with the statistics about the present level of maritime safety, while eventually, the authors’ contribution remains unclear.
2. Two paragraphs in lines 99-133 look like copied from a totally different paper, and the citations do not match, similar to the description of the figure (line 114): “Figure 1 shows the processes of defining the ship hull NURBS model [10]. ” meanwhile, Fig. 1 presents EMSA’s statistics. This is unacceptable and shows that the authors did not even read the entire manuscript before sending it. The paragraphs are copied from the paper: Zhu, K.; Shi, G.; Liu, J.; Shi, J. Fast High-Precision Bisection Feedback Search Algorithm and Its Application in Flattening the NURBS Curve. J. Mar. Sci. Eng. 2022, 10, 1851. https://doi.org/10.3390/ jmse10121851
3. The literature review on the subject of the paper is very poor. Better literature background allows for a better presentation of the authors’ contribution and the novelty of their solution. Please check recently published papers in the field of collision avoidance of autonomous ships, application of reinforcement learning in maritime transportation, or intelligent solutions in ship collision avoidance. For instance (doi numbers provided), there exist many papers dealing with ship’s maneuverability in collision-avoidance that may be used by MASS (like geometrical concepts of critical area/ship domains), utilization of AIS in maritime safety analyses, MASS path-planning with nautical chart integration, COLREG and autonomous shipping (mentioned by the authors), mixed traffic conditions: 10.1016/j.oceaneng.2022.112378, 10.1016/j.neunet.2022.04.008, 10.1016/j.ress.2023.109195, 10.1016/j.ress.2021.107806, 10.1016/j.oceaneng.2020.107709, 10.1016/j.oceaneng.2022.112029, 10.3390/su142416516, 10.1016/j.oceaneng.2022.113120, 10.1016/j.oceaneng.2018.03.092, 10.1007/s00773-021-00825-x, etc., etc.
4. The paper lacks an extensive discussion of findings and identified limitations of the proposed method.
5. The conclusions are extremely short, which raises a question about the rationale of the study.
