Sparse Bayesian Relevance Vector Machine Identification Modeling and Its Application to Ship Maneuvering Motion Prediction

Abstract

In order to establish a sparse and accurate ship motion prediction model, a novel Bayesian probability prediction model based on relevance vector machine (RVM) was proposed for nonparametric modeling. The sparsity, effectiveness, and generalization of RVM were verified from two aspects: (1) the processed Sinc function dataset, and (2) the tank test dataset of the KRISO container ship (KCS) model. The KCS was taken as the main research plant, and the motion prediction models of KCS were obtained. The $\epsilon$-support vector regression and $\upsilon$-support vector regression were taken as the compared algorithms. The sparsity, effectiveness, and generalization of the three algorithms were analyzed. According to the trained prediction models of the three algorithms, the number of relevance vectors was compared with the number of support vectors. From the prediction results of the Sinc function and tank test datasets, the highest percentage of relevance vectors in the trained sample was below 17%. The final prediction results indicated that the proposed nonparametric models had good prediction performance. They could ensure good sparsity while ensuring high prediction accuracy. Compared with the SVR, the prediction accuracy can be improved by more than 14.04%, and the time consumption was also relatively lower. A training model with good sparsity can reduce prediction time. This is essential for the online prediction of ship motion.

1. Introduction

1.1. Background and Motivation

The research on maritime autonomous surface ships (MASS) is widely discussed [1]. More intelligent ships are being used for motion control and sea transportation. In recent years, with the construction of MASS, the accuracy of the model directly affects the performance of the motion control. The sparse modeling method can avoid overfitting to a certain extent and reduce the prediction time [2], which can lay the foundation of online prediction. Ship motion prediction is of great significance for the design of course-keeping, path-following, and adaptive controllers, and it can also improve the control effect [3,4]. Therefore, the development of an efficient and sparse modeling method is of great importance for identification modeling and overcoming the effect of ship type, etc.

1.2. Related Work

System identification has been widely used in many research fields, which solves the modeling problem. It includes parameter and nonparametric identification. In the field of parameter identification of ship motion models, some studies have been conducted using classical and intelligent algorithms. Classical algorithms, such as least square (LS) [5], extended Kalman filter (EKF) [6,7], gradient descent (GD) [8], and recursive prediction error (RPE) [9], were used for the parameter estimation of MMG, coupled with heave-pitching motion models. In the parameter estimation results, these algorithms ensured certain estimation accuracy. As for intelligent algorithms, neural network (NN) [10] and support vector machine (SVM) [11,12] were frequently applied to the parameter estimation of ship motion models. For machine learning algorithms, a certain amount of data is required as support to obtain accurate parameter estimation results. A sensitivity analysis [13,14], as a way to simplify models, reduced the complexity of parameter identification.

As for nonparametric identification modeling, intelligent identification algorithms are widely used. With the rapid development of artificial intelligence, NNs with different structures have been proposed. Recurrent NN [15,16,17], self-designed fully connected NN [18], and long short-term memory NN [19] were applied to the maneuvering motion prediction of KCS, KVLCC2, and DTMB 5415, respectively. Moreover, compared to the traditional NN, the emerging NN can obtain high prediction accuracy. Based on statistical learning theory, SVM uses the kernel method to avoid dimension disaster. It includes $\epsilon$-SVR and $\upsilon$-SVR when it is used for regression. When $\epsilon$-SVR [20] and $\upsilon$-SVR [21] were applied to the parameter identification, precise ship motion prediction results were obtained. However, the setting of hyperparameters greatly affects the regression accuracy of SVR. Zhang et al. [22] combined the grey wolf optimizer with $\epsilon$-SVR. Based on the full-scale trial data, the hyperparameters of $\epsilon$-SVR were tuned and more accurate prediction models are obtained. The supervised learning algorithm, fast noisy input Gaussian process (GP) [23], was used to conduct online motion prediction for container ships and unmanned surface ships. Local weighted learning (LWL) was widely used in robot control fields and is an effective identification algorithm for ship motion prediction [24,25]. As a deep learning algorithm, delayed deep cycle reservoir with regular jumps (DDCRJ) [26] was used to predict ship heave motion.

Table 1. Summary of related work.

System Identification	Algorithm Types	Algorithms	Advantages	Disadvantages
Parameter identification	Classical algorithms	LS [5]	(1) Each step takes less time; (2) Simple structure; (3) Mathematical reasoning, etc.	(1) The influence of the initial value; (2) Parameters drift; (3) Convergence speed.
		EKF [6,7]
		GD [8]
		RPE [9]
	Intelligent algorithms	NN [10] SVM [11,12]	(1) Generalization of parameters; (2) Learning ability; (3) Convergence speed, etc.	(1) Training time; (2) The influence of the data sample; (3) Complex structure.
Nonparametric identification	Intelligent algorithms	NN [15,16,17,18,19]	(1) Generalization of prediction model; (2) Prediction accuracy; (3) Wide range of applications.	(1) The influence of the data sample; (2) Long training time; (3) Hyperparameter tuning.
		SVM [20,21,22]
		GP [23]
		LWL [24,25]
		DDCRJ [26]

shows a summary of relevant work.

Most algorithms used in Table 1 were applied to three degrees of freedom (3 DOF) ship motion, response, and coupled heave-pitching motion models. For the current ship motion controller design, most scholars used the ship response and 3 DOF motion models. Therefore, there are many studies on the 3 DOF ship motion model. With the development of MASS, the prediction of their plane motion is also necessary. SVM has featured in a lot of research in the fields of both parameter identification and nonparametric identification. It has led to good research results. However, although SVM is a relatively sparse model, the computational complexity will increase significantly as the training sample size becomes larger. RVM has many of the features of SVM and avoids the main limitations of SVM. In addition, compared to the state-of-the-art ship motion prediction models, there are relatively few studies on algorithmic sparsity.

Sparsity encompasses feature sparsity and model sparsity. Feature sparsity refers to the sparsity of feature vectors. Model sparsity refers to the sparsity of model weights. This study aims to explore the model sparsity of the three algorithms while ensuring prediction accuracy. The obtained regression model has a certain sparsity if the weights of the regression function have more zeros. Sparsity can significantly increase model capacity and performance without a proportional increase in computation. RVM combines the kernel method with Bayes’ theorem to obtain a relatively sparse prediction model. Taking RVM as an example, the number of relevance vectors (RVs) in the prediction model is small (i.e., the weights of the nonparametric model obtained by RVM have more zeros). RVM only has one hyperparameter to be tuned. Therefore, it does not require an optimization algorithm to tune the hyperparameter. It can quickly obtain a suitable hyperparameter by trial and error, and also guarantee good prediction accuracy. The related work on RVM is discussed as follows.

Sparse models are more and more widely used in many research fields, which can avoid overfitting in the modeling process. RVM has been applied in many fields [27,28,29]. Based on RVM, Tian et al. [30] proposed a diagnosis method for high-pressure hydrogen leakage. In some specific cases, the proposed model took less time. Moreover, the proposed diagnostic model had an accuracy rate of more than 95%. Chang et al. [31] used an empirical model to predict the tunnel convergence. The prediction residuals were modified by RVM. The probability model for tunnel convergence monitoring was obtained. Compared to the prediction results of BPNN and GP, the proposed algorithm had good prediction performance.

1.3. Contributions and Structure

Based on the descriptions in Section 1.1 and 1.2, studies of the sparsity and prediction performance of RVM and SVR are conducted. This research proposes a sparse ship motion prediction model based on RVM. The sparse model removes many redundant variables, simplifying the prediction models while retaining the most significant information in the dataset. Two different datasets, the Sinc function dataset and tank test dataset of KCS, are taken as illustrative examples to analyze the sparsity, effectiveness, and generalization of RVM, $\epsilon$-SVR, and $\upsilon$-SVR. The novelty of the research lies in the following three points:

(1)

Unlike most existing machine learning algorithms, RVM is a sparse model proposed under the fast sequential sparse Bayesian framework. This can be used to obtain a sparse ship motion prediction model.

(2)

Based on the Bayesian framework, a regularization parameter is not used in the derivation of RVM. This reduces the complexity of model learning. Additionally, the obtained ship motion prediction models can avoid overfitting.

(3)

The kernel function of RVM does not need to satisfy the Mercer condition. RVM can flexibly choose the kernel function to obtain an accurate ship motion prediction model. For the setting of the hyperparameter, RVM only needs to set the kernel function parameter.

The structure of the research is organized as follows: Section 2 describes the framework of a 3 DOF ship maneuvering motion model and gives the framework of a nonparametric identification model based on RVM. Section 3 introduces Bayesian inference, sparse Bayesian learning, and the derivation and principle of RVM. In Section 4, the research introduces two illustrative examples to demonstrate the sparsity, effectiveness, and generalization of the nonparametric identification model. Section 5 summarizes the conclusions and future research prospects.

2. Ship Maneuvering Motion Model

In order to describe the ship motion, earth-fixed coordinates and body-fixed coordinates are adopted. Figure 1 shows these two coordinate systems.

The main objective is to obtain a 3 DOF nonparametric identification model of KCS. Therefore, the 3 DOF motion model of KCS is established [32]. The Abkowitz model is widely used in identification modeling research. Although its derivation process is relatively complex, it can accurately describe the dynamic maneuvering characteristics of the ship [33]. Moreover, identification modeling simplifies the calculation of ship hydrodynamic derivatives, so as to establish an accurate mathematical model of ship motion based on data samples. Therefore, the Abkowitz model is used in this study. Equation (1) lists the expression of the 3 DOF motion model for KCS.

$$\left\{\begin{array}{c}m\left(\dot{u}-vr-x_G{r}^2\right)=X\\ m\left(\dot{v}+ur+x_G\dot{r}\right)=Y\\ I_{\mathrm{z}}\dot{r}+m{x}_G\left(\dot{v}+ur\right)=N\\ \dot{\psi }=r\\ \dot{x}=u\cos \left(\psi \right)-v\sin \left(\psi \right)\\ \dot{y}=u\sin \left(\psi \right)+v\cos \left(\psi \right)\end{array}\right.$$

(1)

where $m$ is the mass of ship; $x_G$ is the gravity center of ship; $X$, $Y$, and $N$ are force and moment; $I_{\mathrm{z}}$ is the added moment of inertia; $u$, $v$, $r$, and $\psi$ are surge velocity, sway velocity, yaw rate, and yaw angle, respectively; $x$ and $y$ are displacement in ${\mathrm{X}}_0$ and ${\mathrm{Y}}_0$ direction.

The Equation (1) is expressed by the state space equation to obtain Equation (2).

$$\left[\begin{array}{ccc}m-X_{\dot{u}}& 0& 0\\ 0& m-Y_{\dot{v}}& m{x}_G-Y_{\dot{r}}\\ 0& m{x}_G-N_{\dot{v}}& I_{zz}-N_{\dot{r}}\end{array}\right]\left[\begin{array}{c}\dot{u}\left(t\right)\\ \dot{v}\left(t\right)\\ \dot{r}\left(t\right)\end{array}\right]=\left[\begin{array}{c}{f}_X\left(t\right)+{\tau }_X\left(t\right)\\ f_Y\left(t\right)+{\tau }_Y\left(t\right)\\ f_N\left(t\right)+{\tau }_N\left(t\right)\end{array}\right]$$

(2)

where $X_{\dot{u}},Y_{\dot{v}},Y_{\dot{r}},N_{\dot{v}}$, and $N_{\dot{r}}$ are ship hydrodynamic derivatives; $f_1$, $f_2$, and $f_3$ are the nonlinear functions related to force and moment; ${\tau }_X$, ${\tau }_Y$, and ${\tau }_N$ are forces and moment coming from environmental disturbances.

Nonparametric identification modeling considers the input, output, and dynamic process of the system, without considering the internal structure of the model. According to the motion model of KCS established by Equation (2), the mapping relationship between the ship motion is shown in Equation (3).

$$\left\{\begin{array}{c}\dot{u}\left(t_{n+1}\right)={\mathrm{R}\mathrm{V}\mathrm{M}}_1\left(u\right(t_n),v(t_n),r(t_n),\delta (t_{n+1}\left)\right)\\ \dot{v}\left(t_{n+1}\right)={\mathrm{R}\mathrm{V}\mathrm{M}}_2\left(u\right(t_n),v(t_n),r(t_n),\delta (t_{n+1}\left)\right)\\ \dot{r}\left(t_{n+1}\right)={\mathrm{R}\mathrm{V}\mathrm{M}}_3\left(u\right(t_n),v(t_n),r(t_n),\delta (t_{n+1}\left)\right)\end{array}\right.$$

(3)

where $\delta$ is the rudder angle and ${\mathrm{R}\mathrm{V}\mathrm{M}}_1$, ${\mathrm{R}\mathrm{V}\mathrm{M}}_2$, and ${\mathrm{R}\mathrm{V}\mathrm{M}}_3$ are mapping function.

3. Methodology

3.1. Bayesian Inference and Sparse Bayesian Learning

Suppose the system observation data set is $D$. $\theta$ is the system parameter. Then, the likelihood function of the system is $p\left(D\right|\theta )$, and the prior distribution of parameters is $p\left(\theta \right)$. According to Bayes’ theorem, the posterior distribution $p\left(D|\theta \right)$ of the system is Equation (4).

$$p\left(\theta |D\right)=\frac{p\left(\theta \right)p\left(D\right|\theta )}{p\left(D\right)}$$

(4)

where $p\left(D\right)$ is the marginal likelihood function.

Bayesian inference uses probability operations to fit observed data. $x_p$ is the new data to be predicted. According to Bayesian inference, the probability distribution of the predicted points is calculated as Equation (5).

$$p\left(x_p|D\right)=\int p\left(x_p,\theta |D\right)\mathrm{d}\theta =\int p\left(x_p|\theta ,D\right)p\left(\theta |D\right)\mathrm{d}\theta$$

(5)

Sparse Bayesian learning combines the prior information of unknown parameters with sample information to obtain the assumption of the prior probability distribution [34]. Then, the posterior information is obtained according to Bayes’ theorem. Finally, the obtained posterior information is used to update the prior distribution and sample information. After the above multiple iterations, more and more accurate sample information is obtained. In general, the model of sparse learning can be expressed as Equation (6).

$$y_s\left(x\right)={\mathsf{\Phi }}_s\left(x_s\right)w_s$$

(6)

where $y_s\left(x\right)$ is the regression value of sparse model, ${\mathsf{\Phi }}_s\left(x_s\right)$ is the kernel functions matrix, $w_s$ is the weight, and $x_s$ is the training sample.

The basic idea of sparsity is to use the weighted idea to represent a test sample after the training sample is mapped linearly or nonlinearly. Since the test samples are only related to the training samples with similar information, most of the weighted values are 0 to achieve the purpose of sparsity.

3.2. Relevance Vector Machine

In the 1990s, the rise of SVM led to research into the kernel method. The kernel method is regarded as the third revolution of pattern analysis method after classical statistical linear analysis, NN, and decision tree nonlinear analysis [35]. Therefore, the sparsity of SVM after the introduction of the kernel method has captured the interest of scholars. With the in-depth study of SVM, the defects of SVM have been gradually revealed.

RVM is a supervised learning algorithm proposed by Tipping [36]. Based on Bayesian inference and the sparse Bayesian learning idea, it has many features of SVM and avoids its main limitations. Compared with land vehicles, ships have the characteristics of delayed response to maneuvering. Therefore, there are complex mapping relationships between data. The sparse prediction model can reduce this effect and improve the generalization performance. Given a set of the KCS motion dataset as {${\mathrm{x}}_{\mathrm{n}},t_n$| n = 0, 1, 2,…, N}, N is the number of data samples. According to probability theory, $t_n$ can be expressed as Equation (7). The regression function $f\left(\mathbf{x}\right)$ is shown in Equation (8).

$$t_n=y\left({\mathrm{x}}_{\mathrm{n}},\mathit{w}\right){+\epsilon }_n,{\epsilon }_n~\mathcal{N}(0,{\mathsf{\sigma }}^2)$$

(7)

$$f\left(\mathbf{x}\right)={\sum }_{n=1}^N{\omega }_{n}K(\mathbf{x},{\mathrm{x}}_{\mathrm{n}})+b$$

(8)

where $\mathit{w}$ is a vector of the weight parameters, ${\epsilon }_n$ is the random noise, $\mathcal{N}$ is the normal distribution, and $b$ is the bias.

Supposing that $t_n$ is a mutually independent distribution, the likelihood estimation probability of the training dataset is shown in Equation (9).

$$\mathcal{P}\left(\mathit{t}\right|\mathit{w},{\mathsf{\sigma }}^2)={\left(2\mathsf{\pi }{\mathsf{\sigma }}^2\right)}^{-\frac{N}{2}}\mathrm{e}\mathrm{x}\mathrm{p}(-\frac{{‖\mathit{t}-\mathsf{\Phi }\mathit{w}‖}^2}{2{\mathsf{\sigma }}^2})$$

(9)

where $\mathit{t}={(t_1,t_2\dots ,t_N)}^{\mathrm{T}}$, $\mathsf{\Phi }$ is the $N\times$($N+1$) kernel functions matrix, i.e., $\mathsf{\Phi }={\left[\mathsf{\phi }\left({\mathrm{x}}_1\right),\mathsf{\phi }\left({\mathrm{x}}_2\right),\dots ,\mathsf{\phi }\left({\mathrm{x}}_{\mathrm{n}}\right)\right]}^{\mathrm{T}}$, $\mathsf{\phi }\left({\mathrm{x}}_i\right)=\left[1,K\left({\mathrm{x}}_i,{\mathrm{x}}_1\right),K\left({\mathrm{x}}_i,{\mathrm{x}}_2\right)\dots K\left({\mathrm{x}}_i,{\mathrm{x}}_n\right)\right]$.

If the maximum likelihood estimation of weights is directly conducted, overfitting may occur. To avoid overfitting, it is necessary to constrain $\mathit{w}$ in the RVM regression model. Suppose that $\mathit{w}$ is mutually independent and it follows $\mathcal{N}(0,{\mathsf{\sigma }}^2)$. The expression of prior probability is shown in Equation (10).

$$\mathcal{P}\left(\mathit{w}\right|\mathit{\alpha })={\prod }_{i=0}^N\mathcal{N}\left({\omega }_i\right|0,{{\alpha }_i}^{-1})$$

(10)

where $\mathit{\alpha }$ is a vector of $N$ + 1 hyperparameters.

According to the prior probability and maximum likelihood estimation, the posterior probability of $\mathit{w}$ can be obtained. The expression of posterior probability is shown in Equation (11). Maximize $\mathcal{P}\left(\mathit{t}|\mathit{\alpha },{\mathsf{\sigma }}^2\right)$ to obtain Equation (12).

$$\mathcal{P}\left(\mathit{w}|\mathit{t},\mathit{\alpha },{\mathsf{\sigma }}^2\right)=\frac{\mathcal{P}\left(\mathit{t}|\mathit{w},{\mathsf{\sigma }}^2\right)\mathcal{P}\left(\mathit{w}|\mathit{\alpha }\right)}{\mathcal{P}\left(\mathit{t}|\mathit{\alpha },{\mathsf{\sigma }}^2\right)}=-{\left(2\pi \right)}^{-\frac{N+1}{2}}{\left|\mathsf{\Sigma }\right|}^{-\frac{1}{2}}\mathrm{e}\mathrm{x}\mathrm{p}(-\frac{1}{2}{(\mathit{w}-\mathit{\mu })}^{\mathrm{T}}{\mathsf{\Sigma }}^{-\frac{1}{2}}(\mathit{w}-\mathit{\mu }\left)\right)$$

(11)

$$\mathcal{P}\left(\mathit{t}|\mathit{\alpha },{\mathsf{\sigma }}^2\right)={\left(2\mathsf{\pi }\right)}^{-\frac{\mathrm{N}}{2}}{\left|\mathsf{\Omega }\right|}^{-\frac{1}{2}}\exp \left(-\frac{1}{2}{\mathit{t}}^{\mathbf{T}}\mathsf{\Omega }\mathit{t}\right)$$

(12)

The expression of posterior mean and covariance is shown in Equation (13).

$$\left\{\begin{array}{c}\mathit{\mu }={\mathsf{\sigma }}^{-2}\Sigma {\mathsf{\Phi }}^{\mathrm{T}}\mathit{t}\\ \Sigma ={\left({\mathsf{\sigma }}^2{\mathsf{\Phi }}^{\mathrm{T}}\mathsf{\Phi }+\mathbf{A}\right)}^{-1}\end{array}\right.$$

(13)

where $\mathbf{A}=\mathrm{d}\mathrm{i}\mathrm{a}\mathrm{g}({\alpha }_0,{\alpha }_1\dots {\alpha }_{\mathrm{N}})$, representing the diagonal matrix about $\mathit{\alpha }$. $\mathsf{\Omega }={\mathsf{\sigma }}^2\mathbf{I}+\mathsf{\Phi }{\mathbf{A}}^{-1}{\mathsf{\Phi }}^{\mathbf{T}}$.

The expressions for hyperparameter updating are shown in Equations (14) and (15).

$${\alpha }_i^{New}=\frac{{\gamma }_i}{{\mu }_i^2}$$

(14)

$${\mathsf{\sigma }}_i^{New}=\frac{{‖\mathit{t}-\mathsf{\Phi }\mathit{\mu }‖}^2}{N-\sum _{i=0}^N{\gamma }_i}$$

(15)

where ${\gamma }_i=1-{\alpha }_i{\mathsf{\Sigma }}_{ii}$. ${\mathsf{\Sigma }}_{ii}$ is $i$th diagonal element of the posterior covariance matrix $\mathsf{\Sigma }$.

Supposing that the test sample is $x^{\mathrm{*}}$, Equation (16) lists the predicted value $y^{\mathrm{*}}$. The prediction variance ${\mathsf{\sigma }}^{\mathrm{*}}$ is shown in Equation (17). Equation (18) lists the probability distribution of prediction value $t^{\mathrm{*}}$.

$$y^{\mathrm{*}}={\mathit{\mu }}^{\mathrm{T}}\mathsf{\Phi }\left({\mathbf{x}}^{\mathrm{*}}\right)$$

(16)

$${\mathsf{\sigma }}_{\mathrm{*}}^2={\mathsf{\sigma }}_{\mathrm{M}\mathrm{P}}^2+{\mathsf{\Phi }\left({\mathrm{x}}^{\mathrm{*}}\right)}^{\mathrm{T}}\mathsf{\Sigma }\mathsf{\Phi }\left({\mathrm{x}}^{\mathrm{*}}\right)$$

(17)

$$\mathcal{P}\left(t^{\mathrm{*}}|\mathit{t},{\mathit{\alpha }}_{\mathrm{M}\mathrm{P}},{\mathsf{\sigma }}_{\mathrm{M}\mathrm{P}}^2\right)=\int \mathcal{P}\left(t^{\mathrm{*}}|\mathit{w},{\mathsf{\sigma }}_{\mathrm{M}\mathrm{P}}^2\right)\mathcal{P}\left(\mathit{w}|\mathit{t},{\mathit{\alpha }}_{\mathrm{M}\mathrm{P}},{\mathsf{\sigma }}_{\mathrm{M}\mathrm{P}}^2\right)d\mathit{w}=\mathcal{N}\left(t^{\mathrm{*}}\right|y^{\mathrm{*}},{\mathsf{\sigma }}_{\mathrm{*}}^2)$$

(18)

where ${\mathit{\alpha }}_{\mathrm{M}\mathrm{P}}$ and ${\mathsf{\sigma }}_{\mathrm{M}\mathrm{P}}^2$ are the optimal parameter estimates of $\mathit{\alpha }$ and ${\mathsf{\sigma }}^2$.

Remark 1.

RVM combines kernel methods and Bayes’ theorem to obtain a sparse probabilistic prediction model. During the derivation of the algorithm, it does not add regularization coefficients, but uses the form of a probabilistic description. In addition, as a machine learning algorithm with strong sparsity and generalization, it has been proven to have certain universality [29,30,31].

4. Illustrative Examples and Discussion

Both $\epsilon$-SVR and $\upsilon$-SVR show good prediction performance in the field of ship motion prediction [20,21]. However, SVR needs to determine hyperparameters, such as the penalty factor ($C$). RVM is developed based on SVR. It uses Bayes’ theorem to obtain the probability prediction model without determining $C$. Because RVM has only one hyperparameter to be tuned, it does not require an optimization algorithm to tune the hyperparameter, unlike SVR. The tuning of hyperparameters depends on the published research results and the swarm intelligence optimization algorithm. This study chooses the former. RVM’s feature is sparsity, which can improve prediction performance and efficiency. In the aspect of regression, the prediction model obtained by RVM is generally sparser than that obtained by SVR. Therefore, the speed can be improved when predicting test datasets. The structure of this research for RVM identification modeling is shown in Figure 2.

In order to demonstrate the effectiveness, generalization, and sparsity of the proposed algorithm, two illustrative examples are used in this research. One is the Sinc function that is often used in the fields of digital signal processing and Fourier transform theory and mathematics. The other is the KCS motion datasets published by the SIMMAN workshop in 2020 [37]. The SIMMAN workshop is used to assess simulation methods for ship maneuvering. The technical route of this research is shown in Figure 3.

4.1. Prediction Results of the Sinc Function Dataset

To validate the sparsity and prediction accuracy of the RVM, the Sinc function dataset with added interference is used as the verification sample. The interference is white noise with a mean of 0 and a variance of 0.02. Equation (19) lists the expression of the Sinc function. When $x$ = 0, define Sinc(0) = 1. Take the independent variable $x$ as the input data and the dependent variable $y$ as the output data. The independent variable and dependent variable are scalars. The value range of $x$ is [−20, 20]. The sampling interval is 0.2. The number of data samples is 201. A processed data sample is shown in Figure 4.

$$y=\mathrm{S}\mathrm{i}\mathrm{n}\mathrm{c}\left(x\right)=\frac{\mathrm{s}\mathrm{i}\mathrm{n}\left(x\right)}{x}$$

(19)

In order to reasonably divide the training and test datasets, this research takes the odd numbered data samples as the training dataset and the even numbered samples as the test dataset. The number of training and test datasets is 101 and 100, respectively. In general, the training and test sets of the Sinc function dataset are divided in a 70%/30% ratio. Because three algorithms all have small sample learning ability, this study divides the Sinc function dataset into training and test datasets in the 50%/50% ratio. The training dataset is different from the test dataset. Figure 4 shows that the processed data sample has certain noise interference. For $\epsilon$-SVR, the number of support vectors (SVs) depends on the setting of the insensitive factor ($\epsilon$). In general, the smaller the $\epsilon$, the higher the prediction performance. However, a decrease in $\epsilon$ will cause an increase in the number of SVs and low calculation efficiency. For $\upsilon$-SVR, the setting of the parameter $\upsilon$ can adaptively adjust the number of SVs without adjusting the $\epsilon$ [21]. To compare the experiment results, the three algorithms are set to different hyperparameter values [20,38]. In addition, compared with SVR, RVM only needs to set the kernel function parameter ($\gamma$). In terms of tuning hyperparameters, RVM also shows certain advantages. For the three algorithms, this research mainly designed three different experiments (as shown in

Table 2. Settings for three different experiments.

Algorithms	Experiment (1)	Experiment (2)	Experiment (3)
RVM	$\gamma =0.1$	$\gamma =0.5$	$\gamma =1.0$
ε-SVR	$C={10}^4,\gamma =0.1,\epsilon =0.01$	$C={10}^5,\gamma =0.5,\epsilon =0.1$	$C={10}^5,\gamma =0.8,\epsilon ={10}^{-5}$
υ-SVR	$C={10}^4,\gamma =0.1,\upsilon =0.5$	$C={10}^5,\gamma =0.5,\upsilon =0.3$	$C={10}^5,\gamma =0.8,\upsilon =0.2$

). The experiment results of three algorithms are shown in Figure 5.

Figure 5a shows that the prediction results of three experiments are smooth and fit the Sinc(x) test dataset for RVM well. In contrast, since three hyperparameters need to be set, the prediction accuracy of SVR is restricted. As can be seen from Figure 5b,c, when the hyperparameters are set differently, the prediction results of SVR differ greatly.

Table 3. The number of RVs and SVs for Sinc function dataset.

Algorithms	Experiment (1)	Experiment (2)	Experiment (3)	Average Percentage c
RVM a	6	12	17	11.56%
$\epsilon$-SVR b	99	72	101	89.77%
$\upsilon$-SVR b	69	80	91	79.21%

^a: The proposed algorithm; ^b: Algorithm for comparison; ^c: Average percentage of RVs and SVs in training sample.

shows the number of RVs and SVs in three experiments. To test the sparsity of the trained prediction model, the number of RVs and SVs in experiment (2) is compared. The comparison result is shown in Figure 6.

Figure 6 shows that the number of RVs is less than the number of SVs in experiment (2). In the trained model, it can be seen from Table 3 that the number of RVs is far less than the number of SVs for the three experiments. The average percentage of RVs is 11.56%. The average percentage of SVs is at least 6.8 times that of RVs. In addition, the highest percentage of RVs in the training sample is 16.83%. Compared with $\epsilon$-SVR and $\upsilon$-SVR, the results verify the sparsity of RVM, i.e., RVM can attain sparser prediction models. In terms of time consumption, RVM spends a total of 0.286 s on training and predicting. However, SVR takes relatively more time, especially $\upsilon$-SVR. The mean square error (MSE) is used as the evaluation criteria to verify the prediction performance.

Table 4. MSEs of prediction results for the three experiments.

Algorithms	Experiment (1)	Experiment (2)	Experiment (3)
RVM	0.0338	0.0305	0.0305
$\epsilon$-SVR	0.0368	0.0431	0.0846
$\upsilon$-SVR	0.0348	0.0417	0.1000
Percentage error for $\epsilon$-SVR	8.88%	41.31%	177.38% *
Percentage error for $\upsilon$-SVR	2.96%	36.72%	227.87% *

* Due to the tuning of three hyperparameters, the settings of the hyperparameters have certain challenges. The prediction results are overfitting.

lists the MSEs of prediction results for the three experiments. As can be seen from Table 4, the three algorithms all have good prediction effect in the Sinc function dataset, but the prediction accuracy obtained by RVM is better compared with SVR.

Remark 2. In this section, the research utilizes the prediction results of the Sinc function dataset to verify the performance of RVM. In the setting of the hyperparameters, the research designs different experiments according to the role of hyperparameters. In the process of experiments, the settings of $C$, $\epsilon$, and $\upsilon$ have a great effect on the prediction accuracy and time consumed. After comparison, the prediction model obtained by RVM is relatively sparse (the number of training samples is 101). In Section 4.2, this research increases the number of training samples to further verify the sparsity and prediction performance.

4.2. Prediction Results of Tank Test Dataset for KCS

In Section 4.1, using the Sinc function dataset, the sparsity and prediction performance of RVM are verified by the three experiments. The KCS is taken as the main research plant to further verify the sparsity and generalization of RVM. By dividing the training dataset and selecting the hyperparameters reasonably, the 3 DOF motion prediction models of KCS are obtained. The tank test data for the KCS is from the SIMMAN 2020 workshop [37]. It has some noise interference, so there is no need to add the noise interference.

Table 5. Main particulars of KCS.

Main Particulars	Values
Length between perpendiculars	6.0702 m
Breadth	0.8498 m
Mean draft	0.2850 m
Block coefficient	0.6505
Displacement volume	0.9565 m3

shows the main particulars of KCS.

In the SIMMAN 2020 workshop, the tank tests of KCS include the −10°/−10°, 20°/20°, −20°/−20° zigzag maneuvers, and the −35° turning maneuver. Due to the effect of sensors and other factors, the test datasets have noise interference [37]. In addition, the training dataset should reflect more dynamic information of the KCS. Therefore, a small part of the −10°/−10° and −20°/−20° zigzag maneuver data is used to construct the training dataset. The 20°/20° zigzag maneuver data and −35° turning maneuver data are taken as test datasets. The number of training and test datasets is shown in

Table 6. Training and test datasets.

Data Samples	Maneuvers	Number of Data Samples
Training dataset	−10°/−10° zigzag maneuver	183
	−20°/−20° zigzag maneuver	282
Test dataset	20°/20° zigzag maneuver	618
	−35° turning maneuver	700

.

As can be seen from Table 6, the number of training datasets is 465. This is about five times the number of training datasets in the Sinc function dataset. This research further verifies the sparsity of RVM from the perspective of KCS motion prediction. It is essential to determine the hyperparameters of the three algorithms to make the motion prediction results more accurate. These hyperparameters are set according to previous research [20,21,39]. For the prediction of $u$, $v$, and $r$, the hyperparameters of $\epsilon$-SVR, $\upsilon$-SVR, and RVM are set to different values. The setting values of the hyperparameters are shown in

Table 7. Setting of the hyperparameters for $u$, v and r.

Algorithms	$$\mathit{u}$$	$$\mathit{v}$$	$$\mathit{r}$$
RVM	$\gamma =0.09$	$\gamma =0.08$	$\gamma =0.01$
$\epsilon$-SVR	$C={10}^3$ $\gamma ={10}^{-2}$ $\epsilon ={10}^{-3}$	$C=4\times {10}^3$ $\gamma ={10}^{-2}$ $\epsilon ={10}^{-2}$	$C=150$ $\gamma ={10}^{-2}$ $\epsilon ={10}^{-2}$
$\upsilon$-SVR	$C={10}^3$ $\gamma ={10}^{-2}$ $\nu =0.5$	$C={10}^3$ $\gamma ={10}^{-4}$ $\nu =0.5$	$C=400$ $\gamma ={10}^{-3}$ $\nu =0.5$

.

Remark 3.

Compared to $\epsilon$-SVR and $\upsilon$-SVR, RVM only has one parameter to tune, and no optimization algorithm is needed for further optimization. This also reflects the advantages of RVM from the perspective of hyperparameter tuning. For SVR, the settings of $C$, $\epsilon$, and $\upsilon$ have a great influence on the results of ship motion prediction. Sometimes, the larger the $C$, the less accurate the prediction will be.

According to the training datasets and the setting of hyperparameters, the motion prediction models of KCS are obtained. The test datasets are −35° turning maneuver data and 20°/20° zigzag maneuver data, respectively. The prediction models trained by the three algorithms are compared to verify the sparsity. For the prediction models of $u$, $v$, and $r$, the number of RVs and SVs is shown in

Table 8. The number of RVs and SVs for the tank test dataset.

Algorithms	$$\mathit{u}$$	$$\mathit{v}$$	$$\mathit{r}$$
RVM	3	5	4
ε-SVR	264	37	93
υ-SVR	251	239	239

. Figure 7 shows the prediction results of −35° turning maneuver data.

Table 9. MSEs of prediction results for −35° turning maneuver.

Algorithms	$$\mathit{u}$$	$$\mathit{v}$$	$$\mathit{r}$$
RVM	$6.40\times {10}^{-4}$	$1.53\times {10}^{-4}$	0.2373
$\epsilon$-SVR	$2.72\times {10}^{-3}$	$2.48\times {10}^{-4}$	0.3076
υ-SVR	$3.54\times {10}^{-3}$	$1.87\times {10}^{-4}$	0.3702

shows the MSEs of motion prediction results for the −35° turning maneuver.

RVM constructs a learning machine based on Bayes’ theorem rather than the structural risk minimization principle. As the number of training samples increases, it can be seen from Table 8 that the number of RVs is far less than the number of SVs. This further verifies the sparsity of the proposed algorithm. For the trained model obtained by $\upsilon$-SVR, $\upsilon$ is set as 0.5 [21]. However, the sparsity of $\upsilon$-SVR is worse than that of $\epsilon$-SVR. In Figure 7, the length of the absolute residual represents the prediction error. Moreover, only the absolute residuals of the ship motion prediction are presented. Figure 7a,b show that the prediction error of the three algorithms is small. As can be seen from Figure 7c, the three algorithms all have some prediction errors. For the prediction results of RVM, the maximum absolute residual is 0.52°. In the later stage of yaw angle prediction, the three algorithms all have certain deviations from Figure 7d. In the prediction results of the KCS motion trajectory, the degree of coincidence between the results predicted by the RVM and the tank test data is higher. For the time required to predict $u$, $v$, and $r$, RVM, $\epsilon$-SVR, and $\upsilon$-SVR took the total of 0.749 s, 4.307 s, and 3.985 s, respectively.

Table 9 shows the MSEs of the prediction results. For the prediction results of $r$, the percentage errors for $\epsilon$-SVR and $\upsilon$-SVR are 29.62% and 56.01%, respectively. Using the proposed algorithm, the accuracy of other prediction results can be improved by more than 22.22%. The generalization of RVM is demonstrated from the data prediction results. The 20°/20° zigzag maneuver data of the KCS is taken as another data sample to further validate the generalization of RVM. Although there are some similarities between the 20°/20° zigzag maneuver data and the −20°/−20° zigzag maneuver data, there are some differences, such as the opposite excitation of the input rudder angle. The hyperparameter settings of the three algorithms are shown in Table 7. The prediction results of the 20°/20° zigzag maneuver are shown in Figure 8.

Table 10. MSEs of prediction results for 20°/20° zigzag maneuver.

Algorithms	$\mathit{u}$	$\mathit{v}$	$\mathit{r}$
RVM	$3.27\times {10}^{-6}$	$1.78\times {10}^{-4}$	$0.0709$
$\epsilon$-SVR	$3.55\times {10}^{-5}$	$2.03\times {10}^{-4}$	$0.0814$
$\upsilon$-SVR	$3.70\times {10}^{-5}$	$2.06\times {10}^{-4}$	$0.0813$

shows the MSEs of the prediction results for the 20°/20° zigzag maneuver.

It can be seen from Figure 8 that the three algorithms have high accuracy for motion prediction of the 20°/20° zigzag maneuver due to certain similarities between the test data and the training data. In Figure 8c,e, the prediction results of RVM have certain deviations in the later stage. However, as can be seen from Table 10, the overall prediction results of the KCS motion based on RVM are better than SVR. For the final prediction result, the accuracy can be improved by more than 14.04% with the proposed algorithm. For the time required to predict $u$, $v$, and $r$, RVM, $\epsilon$-SVR, and $\upsilon$-SVR took the total of 0.746 s, 4.152 s, and 3.952 s, respectively. In addition, the three algorithms have high accuracy for $\psi$. The first overshoot angle of the KCS is 33.56°. The first overshoot angle predicted by the proposed algorithm is 32.28°.

RVM is an algorithm that combines kernel methods and Bayes’ theorem. Under the structure of a priori parameter, the automatic relevance determination is used to remove the irrelevant sample points to obtain a sparse model. The sparse model simplifies the model and can avoid overfitting to some extent. The reason is that it retains the explanatory variable most relevant to the response variable. In the prediction of −35° turning maneuver data and 20°/20° zigzag maneuver data, the prediction models obtained by RVM have good sparsity and prediction accuracy.

Remark 4.

In this section, the research utilizes the prediction results of the tank test dataset to further verify the sparsity, effectiveness, and generalization of RVM. From the perspective of the trained model, as the number of training samples increases, the RVM can still maintain a certain degree of sparsity. Although there are certain prediction errors, the overall prediction curve of RVM fits the tank test data curve well.

5. Conclusions

The main finding of this research is that RVM based on sparse Bayes’ theorem is proposed to obtain a sparse and efficient ship motion prediction model. The final prediction results indicate that the proposed RVM can ensure good sparsity while ensuring prediction accuracy. Good sparsity of the trained model can reduce the prediction time, which is essential for the online prediction of ship motion. Therefore, the proposed scheme can provide a reference for online ship motion prediction. The experimental verification is performed based on the Sinc function dataset and tank test dataset of KCS, and the results are as follows:

(1)

The Sinc function dataset is used as a simulation example to verify the sparsity and prediction accuracy of RVM. Based on the different hyperparameter settings of RVM, ε-SVR, and υ-SVR, three experiments are conducted. The MSE of prediction results is lower than 0.0305 for RVM. The highest percentage of RVs in the training sample is below 17%. The prediction results of three experiments verify the sparsity, effectiveness, and generalization of RVM.

(2)

Compared with the training dataset for the Sinc function, there are more training samples. However, although the number of training datasets has increased, the proposed scheme can still maintain sparsity. After dividing the training dataset and selecting the hyperparameters reasonably, the maneuvering motion prediction of the KCS is conducted. The accuracy of prediction results can be improved by more than 14.04%. The prediction performance of RVM is further verified.

(3)

The prediction results of RVM are compared with those of ε-SVR and υ-SVR. From the prediction results of two illustrative examples, all three algorithms have good prediction results, but the prediction performance of RVM is better than that of SVR. The overall prediction curve of RVM fits the tank test data curve well. Moreover, less time is also required. In addition, compared with the prediction models obtained by SVR, the prediction models obtained by RVM are sparser.

RVM has only one hyperparameter to adjust. Compared with SVR, the structure is simple. In terms of kernel function selection, RVM is not satisfied by the Mercer condition, so it has a wider selection of kernel function. It can also achieve a sparse prediction model and has the ability to predict quickly. However, for some more complex datasets, the prediction performance of RVM with a single kernel function remains to be studied. Ship motion prediction is of great significance for the design of course-keeping, path-following and adaptive controllers. In future, we will further improve the performance of the proposed algorithm and focus on ships’ autonomous berthing and unberthing and path-following controllers using the improved algorithm.