Ship-Collision Avoidance Decision-Making Learning of Unmanned Surface Vehicles with Automatic Identification System Data Based on Encoder—Decoder Automatic-Response Neural Networks

Abstract

Intelligent unmanned surface vehicle (USV) collision avoidance is a complex inference problem based on current navigation status. This requires simultaneous processing of the input sequences and generation of the response sequences. The automatic identification system (AIS) encounter data mainly include the time-series data of two AIS sets, which exhibit a one-to-one mapping relation. Herein, an encoder–decoder automatic-response neural network is designed and implemented based on the sequence-to-sequence (Seq2Seq) structure to simultaneously process the two AIS encounter trajectory sequences. Furthermore, this model is combined with the bidirectional long short-term memory recurrent neural networks (Bi-LSTM RNN) to obtain a network framework for processing the time-series data to obtain ship-collision avoidance decisions based on big data. The encoder–decoder neural networks were trained based on the AIS data obtained in 2018 from Zhoushan Port to achieve ship collision avoidance decision-making learning. The results indicated that the encoder–decoder neural networks can be used to effectively formulate the sequence of the collision avoidance decision of the USV. Thus, this study significantly contributes to the increased efficiency and safety of maritime transportation. The proposed method can potentially be applied to the USV technology and intelligent collision-avoidance systems.

1. Introduction

With the increasing popularity of the automatic identification system (AIS) equipment and the development of shore-based and spaceborne AIS equipment, AIS data have become a popular data source for big data analysis and machine learning in the marine industry. Therefore, several studies have investigated AIS data. The maturity of the algorithms, such as the machine learning and big data mining algorithms, has resulted in novel solutions to the previous problems. Currently, researchers can easily access and analyze the AIS data to improve the intelligence of maritime autonomous surface ships (MASSs). Obtaining seamen-collision avoidance experience from the massive AIS data is particularly important; therefore, the big-data-related technologies should urgently develop methodologies to satisfy their requirements. The development of intelligent/unmanned surface vehicles is becoming the trend in the shipbuilding industry. Therefore, intelligent collision avoidance technologies must be investigated to resolve navigation problems. Previously, relevant experts and scholars have used various mechanisms, including expert systems [1], automatic collision avoidance systems [2], fuzzy theory [3], and dynamic route-planning systems [4]. Furthermore, they have studied the collision avoidance of ships at sea.

Many scholars have improved their research methods regarding intelligent ship collision-avoidance research with the continuous development of machine-learning and artificial intelligence technologies. Kuwata [4] proposed an automatic collision-avoidance algorithm for unmanned boats, ensuring that they can avoid both static and dynamic obstacles and comply with the rules based on the velocity obstacle principle. Montiel et al. [5] proposed a bacterial evolutionary algorithm and a bacterial potential field combined with the artificial potential field method to obtain safe paths under dynamic conditions. In addition, Pietrzykowski [6] considered a real-time implementation of an automotive collision-mitigation system and evaluated possible risks, and Guo et al. [7] proposed a deep reinforcement learning method to realize the intelligent route planning of unmanned ships in unknown environments. Simsir et al. [8] utilized an artificial neural network to predict the future locations of vessels, and Azzeddine et al. [9] proposed an adaptive ship-safety domain using spatial risk functions to identify collision and grounding risks. Jansson and Gustafsson [10] presented a probabilistic framework to design and analyze the existing collision-avoidance algorithms that enabled online computation of the risk in the case of faulty intervention and the consequences of different actions. Zaccone and Martelli [11,12] presented a collision-avoidance algorithm based on an ad hoc modified version of the rapidly exploring random tree (RRT*) algorithm. They proposed an automatic system to detect static and moving obstacles and plan an evasive route, which could significantly improve ship safety. Chiang and Tapia [13] proposed an RRT-based motion planner for unmanned surface vehicles (USVs) for identifying long-term International Regulations for Preventing Collisions at Sea (COLREGS) compliant trajectories. Fang et al. [14] utilized a real-time simulator based on the six-dimensional (6D) maneuvering model group (MMG) model to simulate the ship’s motion and simplify the complex collision avoidance cases in fast-time simulations with multi-ship encounter conditions for the safety of ship navigation in heavy traffic areas. Hu et al. [15] proposed a route-planning method based on the hierarchical multi-objective particle swarm optimization algorithm (H-MOPSO) to generate collision-free paths that comply with the COLREGS. Tsou et al. [16] considered the COLREGS and navigational practices and proposed a path-planning system based on the ant colony algorithm. The planned path considered economy and safety at the same time. Lazarowska [17] proposed a path-planning method for unmanned ships based on the ant colony optimization (ACO) algorithm for a marine dynamic environment. This solution can be applied to a ship’s decision support system or obstacle intelligent detection and collision avoidance system. Montiel et al. [5] proposed a bacterial potential field method based on the bacterial potential field (BPF) that combines the bacterial evolutionary algorithm (BEA) and the artificial potential field method artificial potential field (APF). It plans a safe path under the circumstances. Zhang et al. [18] described a new adaptive collision avoidance algorithm based on state-behavior-reward-state-behavior (SARSA) in strategy reinforcement learning to deal with trajectory changes caused by external conditions. Perera et al. [19] considered COLREGS using artificial neural networks, fuzzy logic, and Bayesian networks and proposed a model for autonomous navigation and the collision avoidance of ships based on intelligent guidance systems.

The conventional methodology is to dismantle the collision-avoidance decision, determine the waypoint, and use control technology to avoid collisions. However, this study proposes the usage of encoder–decoder neural networks by utilizing the sequence-to-sequence (Seq2Seq) model to learn the manner in which appropriate collision-avoidance decisions can be generated based on successful collision-avoidance cases.

2. Materials and Methods

In this section, we first describe encoder–decoder training data preparation. The deep encoder -decoder automatic-response neural networks require a considerable amount of training data and repeated training to activate the deep network units. The process includes obtaining more data samples, separately obtaining the best part of each sample, continuously integrating and optimizing decision-making, and surpassing the superiority of the sample’s decision-making. However, this process is always based on the anthropomorphic characteristics of the training samples, and sailing at sea would not conflict with the actual navigation rules. This study uses the AIS trajectory data obtained in 2018 from Zhoushan Port (Figure 1) as the source for identifying the encounter data and extracting the handling-behavior data of ships. The flexible multi-sequence mapping samples of the ship encounters are obtained as Seq2Seq training data for the encoder–decoder machine learning. The extracted successful ship collision-avoidance cases from the AIS big data can be divided into two steps: (a) ship AIS-encounter-trajectory-data identification and (b) ship key feature-point (KFP) extraction from the ship-encounter trajectories.

2.1. Trajectory Data Identification of Ship Encounters

Currently, there are two quantification sources based on which the ship-encounter azimuthal map can be obtained.

(1) The radar-based collision avoidance steering diagram is mainly used to provide guidance to the ship’s pilot when a target ship is detected solely by the radar and the pilot cannot see the target ship [20,21]. The portion-angle division is as follows: 030°–90°–150°–210°–292.5°–330° (Figure 2).

(2) COLREGS define the azimuth of the target ship (005°–112.5°–247.5°–355°; Figure 3). Rule 13 states: A vessel shall be deemed to be overtaking when coming up with another vessel from a direction more than 22.5 degrees abaft her beam. Rule 14 states: When two power-driven vessels are meeting on reciprocal or nearly reciprocal courses so as to involve risk of collision each shall alter her course to starboard so that each shall pass on the port side of the other.

Furthermore, Based on COLREGs and combined with the arc illumination range of ship lights to establish the ship-collision avoidance encounter azimuthal map (002°–005°–044°–108°–203°–252°–306°–355°), as shown in Figure 4.

The difference with respect to the heading and speed for establishing the ship collision-avoidance behavioral modes is obtained based on the encounter azimuthal maps of two ships approaching collision and the distance to the closest point of approach (DCPA), time to the closest point of approach (TCPA), distance, difference of heading and speed (

Table 1. Ship-encounter pattern class characteristics.

Encounter Pattern	Distance (n mile)	Time to the Closest Point of Approach (TCPA) (s)	Distance to the Closest Point of Approach (DCPA) (n mile)	Encounter Situation	Difference of Heading ΔC (°)
A1-A1	Dis < 6	TCPA > 0	DCPA < 1	Head On	174 < ΔC < 186
A2-A1	Dis < 6	TCPA > 0	DCPA < 1	Head On	174 < ΔC < 186
A2-A2	Dis < 6	TCPA > 0	DCPA < 1	Head On	174 < ΔC < 186
B1-D	Dis < 6	TCPA > 0	DCPA < 2	Crossing
B1-A	Dis < 6	TCPA > 0	DCPA < 2	Crossing
B2-D	Dis < 6	TCPA > 0	DCPA < 2	Crossing
B2-A	Dis < 6	TCPA > 0	DCPA < 2	Crossing
C1-A	Dis < 4	TCPA > 0	DCPA < 3	Overtaking	S1 < S2cos(ΔC)
C1-B1	Dis < 4	TCPA > 0	DCPA < 3	Overtaking	S1 < S2cos(ΔC)
C1-D2	Dis < 4	TCPA > 0	DCPA < 3	Overtaking	S1 < S2cos(ΔC)
C2-A	Dis < 4	TCPA > 0	DCPA < 3	Overtaking	S1 < S2cos(ΔC)
C2-B1	Dis < 4	TCPA > 0	DCPA < 3	Overtaking	S1 < S2cos(ΔC)

). Subsequently, the appropriate training data are extracted based on the ship-encounter pattern (12 types).

The relative motion of the ship with respect to DCPA and TCPA can be calculated as follows:

$$\{\begin{array}{l}{V}_{xO}=V_O\sin {\phi }_O\\ V_{yO}=V_O\cos {\phi }_O\end{array}$$

(1)

$$\{\begin{array}{l}{V}_{xT}=V_T\sin {\phi }_T\\ V_{yT}=V_T\cos {\phi }_T\end{array}$$

(2)

$$\{\begin{array}{l}{V}_{xR}=V_{xT}-V_{xO}\\ V_{yR}=V_{yT}-V_{yO}\end{array}$$

(3)

$$VR=\sqrt{V_{xR}^2+V_{yR}^2}$$

(4)

$${\phi }_{\mathsf{R}}=\arctan \frac{V_{xR}}{V_{yR}}+\alpha$$

(5)

$$\alpha =\{\begin{array}{cc}{0}^{\circ }\hfill & \hfill V_{xR}\ge 0, VyR\ge 0 \\ {180}^{\circ }\hfill & \hfill V_{xR}<0, VyR<0\\ {180}^{\circ }\hfill & \hfill V_{xR}>0, VyR<0\\ {360}^{\circ }\hfill & \hfill V_{xR}<0, VyR>0\end{array}$$

(6)

$$DCPA=S\sin ({\phi }_{\mathsf{R}}-C-\pi )$$

(7)

$$TCPA=S\cos ({\phi }_{\mathsf{R}}-C-\pi )/V_R$$

(8)

Here, V_xO and V_yO are the components of the own ship’s speed vector on the x- and y-axes, respectively, V_xT and V_yT are the components of the target ship’s speed vector on the x- and y-axes, respectively, V_R is the relative speed, φ_R is the relative course, and α is the angle compensation coefficient.

The ship-encounter situation data were extracted based on the AIS trajectories of the ship, as shown in Figure 5.

2.2. Key Feature-Point Extraction from the Ship-Encounter Trajectories

In a meeting situation, some ship handling behaviors are hidden in the trajectory, occupying 15% of the overall trajectory points [22]. Therefore, a large amount of redundant data will be present with respect to the overall ship trajectory. In this study, the KFPs are extracted based on the AIS encounter trajectory to recognize the ship handling behavior as training data for the encoder–decoder neural networks.

We improved the multiple-scale sliding window algorithm when considering the ship drift angle to perform KFP extraction. The sliding window algorithm can be used to solve the problem of filtering the sub-elements in the array sequence. This algorithm can convert the original nested loop problem into a one-time single loop problem, which can considerably reduce the time when processing big data. The receptive field of the sliding window in the AIS trajectory KFP extraction algorithm constructed in this study is a rectangle, which always contains only three trajectory points and only judges whether the second point in the receptive field is retained as a feature point in the trajectory at a certain time. In this study, we combined the principle of the compression algorithm, ship drift angle deviation, position deviation, and AIS spatiotemporal characteristics to improve the performance of the online sliding window KFP extraction algorithm.

The KFP extraction process is illustrated in Figure 6 and Figure 7. For more details, please refer to our previous study [23].

The sliding window algorithm was utilized to extract KFPs from the ship-encounter situation data presented in Section 2.1. Figure 8 and Figure 9 show the before and after extraction of the single ship trajectory. The ship-handling-behavior data were extracted, as shown in Figure 10.

2.3. Decoder–Encoder Automatic-Response Neural Networks

In traditional ship-collision avoidance research, a ship collision-avoidance behavior pattern library has to be established in advance; this library must contain most of the maritime encounter patterns at sea stored on a database. Subsequently, the computer matches the current navigation status with the historical data and generates the corresponding ship collision-avoidance decision based on the candidate set. This method has four shortcomings. (a) It requires a large database capacity, (b) the current encounter situation must exist in historical data, (c) it needs strong database search capabilities for support, and (d) it cannot distinguish the quality of the data source. The optimality of the collision-avoidance decision is considerably limited by the quality of the data source; furthermore, the quality of this decision cannot avoid that of the data source itself. The computer only uses the storage and index functions during the collision-avoidance decision-making process and does not fully utilize the advantages of artificial intelligence. Therefore, this can be referred to as automatic collision avoidance. An intelligent collision-avoidance decision-making system should be able to learn, analyze the current state, and perform an in-depth study of the ship collision-avoidance mechanism associated with the ship-encounter data when a situation not recorded in the database is encountered. In addition, it must make relatively reasonable decisions similar to those made by the officers in the ship.

2.3.1. Sequence-to-Sequence (Seq2Seq) Model

The Seq2Seq model is a similar translation response model that translates a sequence into another sequence using two sets of recurrent neural networks (RNNs). The structure of the Seq2Seq model is presented in Figure 11.

The Seq2Seq model implements step-by-step encoding and decoding of the input and output sequences through a semantic vector state C, which can completely retain all the information from the input sequence to the output sequence.

(1) Encoder stage

The encoder converts a variable-length input sequence into a fixed-length vector, obtains the output for each hidden layer, summarizes the output, and generates a state vector C after a series of non-linear transformations.

$$C=q(E_0,E_1,E_2,E_3,\dots ,E_t)$$

(9)

where t is the number of KFPs in the input sequence. The output h_t of the last hidden layer can also be used as the semantic vector C, and E₁, …, E_t denote the input data of the target ship trajectory.

(2) Decoder stage

In the decoder, the output sequence Y₂, …, Y_t−1 is the own ship response sequence for the target ship and the fixed semantic vector C will be used to predict the next output word Y_t.

$$Y_t=\arg \max P(Y_t)={\displaystyle \coprod _{i=1}^{T}p(Y_t|\{Y_1,Y_2,Y_3,\dots ,Y_{t-1}\})}$$

(10)

With the continuous evolution of neural networks, their structure is becoming increasingly complex such that they can include complex data structures, simplify the preprocessing of the input data, and make the input information more complete and without any losses. In this study, the overall target-ship KFP sequence is considered the input and the overall own-ship KFP sequence is considered the output. The encoder–decoder network encodes the target ship sequence to obtain a space state vector and performs decoding to generate the corresponding collision avoidance decision sequence (Figure 12).

Among the two recurrent neural networks (RNN) networks, the first one mainly considers the input of the time-series data sequence to generate a state vector, whereas the other one generates a time-series data sequence through the generated state vector. The two networks cooperate with each other to learn ship collision-avoidance decisions.

2.3.2. Bidirectional Long Short-Term Memory Recurrent Neural Networks (Bi-LSTM RNN) Structure

The AIS data include standard time-series data; the KFP sequence extracted from the trajectory is considered time-series data. The optimal effect of processing time-series data in neural networks can be obtained using the RNN. Therefore, each encoder and decoder in this study are independent Bi-LSTM RNN networks. Accordingly, the forget gate in the LSTM unit [24,25,26] (Figure 13) stores historical data, remembers the common behavioral patterns, and forgets unique behaviors, improving the universality of the model. Thus, it is more stable and effective when data are being inputted. This study, which is based on RNN, adds LSTM units to the time-series data to solve the gradient disappearance problem, and introduces a bidirectional structure to enhance the contextual correlation [27].

Finally, the seq2seq model and Bi-LSTM RNN are combined to construct the encoder-decoder as shown in Figure 14.

3. Results

The encoder–decoder was implemented in Python within the machine-learning framework of Google’s TensorFlow. The overall network training structure, which was automatically generated via TensorBoard, is depicted in Figure 15.

As shown in Figure 16, the ship encounter data obtained by screening are disassembled into the trajectory of the target ship and the collision avoidance decision corresponding to this ship, which is used as the input value of the encoder–decoder for training.

Figure 16 shows the overall process of identifying the encounter data from the AIS data and extracting KFP. Finally, the data are converted into training data for the encoder–decoder.

From 20 sets of training data (Figure 17), it was found that the automatic-response system based on the encoder–decoder has great potential in terms of the ability to reason about collision avoidance decisions. Figure 18 (1) shows the crossing encounter situation in which AIS trajectory data are extracted from the AIS data obtained from Zhoushan Port, and Figure 18 (2) shows the KFPs extracted using the sliding window algorithm. Figure 18 (3) shows the normalized training data, and Figure 18 (4) shows the training data after sequence decomposition and translation rotation transformation. Figure 18 shows the training and learning processes.

As shown in Figure 19, the reasoning task (task 1) in the single meeting situation shows a certain potential. It can be grounded on the basis of the reasoning of the position relationship, and the results of the generative automatic-response model proposed here are excellent. This shows that the overall structure based on the Seq2Seq model can be directly applied for structured learning, and the Bi-LSTM RNN can integrate the above knowledge to learn relevant information from the ship trajectory sequence, such as position and relationship information, when generating collision avoidance decisions, which can be triggered correctly.

From Figure 19, it can be seen that the decision learning for ship collision avoidance converges very quickly, and the effect of collision avoidance is quickly achieved. Figure 20 presents the results of each type of decision-making to avoid target ships.

4. Conclusions

In this study, we propose the usage of encoder–decoder neural networks by utilizing the Seq2Seq model to learn how to generate appropriate collision-avoidance decisions based on the historic successful collision-avoidance cases. The Seq2Seq model can be conveniently used to develop training data and facilitate collision-avoidance actions. Thus, the proposed method could considerably improve the optimization efficiency, resulting in the increased realistic nature of the generated trajectories. Moreover, if the system encounters an unknown situation, conventional machine learning can be used to generate an average value without considering the authenticity of the ship navigation. The encoder–decoder is dependent on real data structures to generate an anthropomorphic decision, which may not be the optimal collision-avoidance solution; however, it can ensure the effectiveness and authenticity of the collision-avoidance strategy.

Thus, the proposed method employs big data training. Furthermore, it has various applications and exhibits high versatility, achieving timely strategy generation, timely response, and enhanced collision-avoidance security. Therefore, it can be used to generate intelligent ship-collision avoidance decision-making strategies and perform dynamic route planning and other relevant tasks.