Solving Fractional Order Differential Equations by Using Fractional Radial Basis Function Neural Network

Abstract

Fractional differential equations (FDEs) arising in engineering and other sciences describe nature sufficiently in terms of symmetry properties. This paper proposes a numerical technique to approximate ordinary fractional initial value problems by applying fractional radial basis function neural network. The fractional derivative used in the method is considered Riemann-Liouville type. This method is simple to implement and approximates the solution of any arbitrary point inside or outside the domain after training the ANN model. Finally, three examples are presented to show the validity and applicability of the method.

1. Introduction

The symmetry design of the system includes integer partial differential equations and fractional-order partial differential equations. Fractional differential equations (FDEs) can be applied for modelling many problems in real-life. Lately, fractional calculus has received much consideration of the researchers due to it’s numerous applications in diverse scientific fields such as fluid flow, dynamical system control theory, signal processing, diffusion transport, electric network, etc. [1,2,3,4]. In most cases, it is hard to gain analytical solutions of these problems. Therefore, numerical approaches have to be applied for approximating the solution of FDEs [5,6,7,8,9,10,11,12,13,14,15].

According to our primary studies, over the past decades, various numerical methods have been proposed for solving these problems. For example finite difference [16,17], Adomian decomposition [18], monotone iterative technique [19], random walk [20], operational matrix method [21,22], etc. Although these techniques provide good approximations to the solution, but most of them give series expansions in the neighborhood of initial conditions are used [23].

In recent years, artificial neural network (ANN ) methods have been established as a powerful technique to solve partial and ordinary differential problems [24,25,26].

Lagaris et al. [27] proposed an artificial neural network method to solve boundary and initial value problems. Pakdaman et al. [28] applied neural network and Broyden-Fletcher-Goldfarb-Shanno (BFGS) optimization technique to solve linear and nonlinear FDEs. Pang et al. [29] presented fractional physics-informed neural networks to solve space-time fractional advection-diffusion equations. Jafarian et al. [30] employed ANN model for approximate polynomial solution of fractional order Volterra integro differential equations. Qu et al. [31] discussed about a neural network algorithm based on Legendre’s polynomial to solve fractional diffusion equations. In another study, the neural network training and cosine basis functions with adjustable parameters have been presented by Qu and Liu for solving fractional order differential equations [32].

One of the advantages of solving equations ANN is the simple implementation and approximation of the solution of any desired point inside or outside the domain after training the ANN model. In this paper, we apply fractional radial basis function neural network (FRBF-NN) to solve fractional order initial value problems.

This paper is organized as follows: Section 2 lists basis definitions and fundamental issues of fractional calculus and structure of RBF neural network are given. Section 3 describes the proposed algorithm for solving FDEs. Section 4 presents three examples to show the effectiveness of the proposed method. Finally, Section 5 includes the concluding remarks.

2. Preliminaries

In this section, we intend to give some definitions and concepts related to fractional calculus and RBF neural network, which are used further in this article.

2.1. Fractional Derivative Definition

An important benefit of the fractional derivative over the ordinary derivative is that it extends the search region around the test point in specified boundaries [1].

The Riemann-Liouville differential operator $D^{\nu }$, for a function $f:(0,\infty )\to R$ that $\nu >0$ is defined as [33]:

$$D^{\nu }f\left(x\right)=\frac{1}{\mathsf{\Gamma }(n-\nu )}\frac{{\partial }^n}{\partial x^n}{\int }_0^x{(x-t)}^{n-\nu -1}f\left(t\right)dt,$$

(1)

that $n=\left[\nu \right]+1$, $\left[\nu \right]$ is the integer part of $\nu$ and $\mathsf{\Gamma }(.)$ denotes the Gamma function.

2.2. Structure of RBF-Neural Network

The RBF neural networks, can be used function approximation problems.

RBF networks architecture consists of three layers:

• The input layer
• The hidden layer includes radial basis functions:

The nonlinear input space is represented by a nonlinear mapping to a higher dimensional space in the hidden layer.

• The output layer consists of linear neurons:

Linear regression is performed to predict the desired goals by output layer. Architecture of RBF networks is indicated in Figure 1. Suppose $X\in R^{m_1}$ be the input vector, the RBF network’s overall mapping, $s:R^{m_1}\to R$ is given as:

$$y=\sum _{i=1}^M{w}_i{\varphi }_i(\parallel x-x_i\parallel )+b,$$

(2)

where M is the number of hidden layer neurons, $x_i\in R^{m_1}$ are the RBF neural network centers, $w_i$ refer to the weights between the output neuron and the hidden layer, ${\varphi }_i$ is the ith hidden neuron kernel and b refers to the bias term of the output neuron.

Without losing the generality, only one neuron is considered in the output layer. Some of the kernel functions that use in RBF neural networks are: inverse multiquadrics, multiquadrics and Gaussian functions [34]. The Gaussian function is the most commonly used kernel because of its versatility [35].

$${\varphi }_i(\parallel x-x_i\parallel )=exp(-\frac{\parallel x-x_i{\parallel }^2}{{\sigma }^2}),$$

(3)

that $\sigma$ refers to the spread of the Gaussian kernel.

3. Description of the Method

Here, we describe the fractional RBF neural network to approximate FDEs. The RBF-NN standard learning algorithms are based on classic gradient descent algorithm. By applying fractional derivatives, more information can be obtained in comparison with the classic derivative, which only results the tangent at a point of the given function [36].

Consider a FDE of order $\nu$,

$$D_x^{\nu }y\left(x\right)=f(x,y\left(x\right)),$$

where $y\left(x_0\right)=y_0,x\in R^{m_1}$.

The overall mapping, at the nth learning iteration is written as follows:

$$y\left(n\right)=\sum _{i=1}^M{w}_i\left(n\right){\varphi }_i(x,x_i)+b\left(n\right),$$

where M is the number of neurons, weights $w_i\left(n\right)$ and $b\left(n\right)$ are adjusted at each iteration. The approximate solution $y\left(n\right)$ satisfies the initials conditions.

The error function can be written as:

$$E\left(n\right)=\frac{1}{2}{(Y\left(n\right)-y\left(n\right))}^2=\frac{1}{2}\sum _k{e}_k^2\left(n\right),$$

where $Y\left(n\right)$ is the nth iteration’s output. $e\left(n\right)=Y\left(n\right)-y\left(n\right)$ is error between exact solution and approximate solution, and k is the number of output layer nourons.

The equation of weight update applying convex combination of fractional and classical gradient is written as [37]:

$$w_i(n+1)=w_i\left(n\right)-\alpha \eta {▽}_{w_i}E\left(n\right)-(1-\alpha ){\eta }_{\nu }{▽}_{w_i}^{\nu }E\left(n\right),$$

(4)

where mixing parameter $\alpha \in [0,1]$, ${\eta }_{\nu }$ and $\eta$ prefer to the step sizes of fractional and classical gradients, and

$$-{▽}_{w_i}E\left(n\right)={\varphi }_i(x,x_i)e_k\left(n\right),$$

(5)

$$-{▽}_{w_i}^{\nu }E\left(n\right)={\varphi }_i(x,x_i)e_k\left(n\right)\frac{w_i^{1-\nu }}{\mathsf{\Gamma }(2-\nu )}.$$

(6)

Using (5) and (6), Equation (4) can be reduced to:

$$\mathsf{\Delta }{w}_i\left(n\right)=e_k\left(n\right)\{\alpha \eta +(1-\alpha ){\eta }_{\nu }{w}_i^{(1-\nu )}\left(n\right)\}{\varphi }_i(x,x_i).$$

(7)

Also the update formula for $b\left(n\right)$ is as follows:

$$\mathsf{\Delta }b\left(n\right)=e_k\left(n\right)\{(1-\alpha ){\eta }_{\nu }{b}^{(1-\nu )}\left(n\right)+\alpha \eta \}.$$

(8)

4. Numerical Results

This section provides three numerical examples in order to verify the powerfulness and applicability of the proposed strategy. It is worth mentioning that for obtaining the best results during training, the parameters $\alpha$, $\nu$, $\eta$ and ${\eta }_{\nu }$ of FRBF-NN for each example are empirically selected. We use just the real part of the update sentence beacause of avoiding complex values.

The experiments are repeated 50 times for 1000 epoch iterations. The values of weights and bias were randomly initialized per round. All the mathematical computations were conducted using MATLAB-R2019b.

Example 1.

Let us consider the following FDE

$${}_0^x{D}_C^{0.5}u\left(x\right)+u\left(x\right)={\displaystyle \frac{2x^{1.5}}{\mathsf{\Gamma }\left(2.5\right)}}+x^2,0\le x\le 1,$$

where u(0) = 0. The analytical solution of this example is $u\left(x\right)=x^2$.

The learning rates were taken to be $\eta ={10}^{-2}$ and ${\eta }_{\nu }=3\times {10}^{-2}$. The parameters $\alpha$ and $\nu$ were set to $0.5$ and $0.6$ respectively. For training phase, a total of 101 and 1001 values of x were applied ranging from 0 to 1.

Table 1. Numerical results for Example 1.

$\mathbf{\Delta }\mathit{x}$	Number of Kernels	MSE (Training)	MSE (Testing)	Approximation Accuracy
0.01	35	$7.34\times {10}^{-5}$	$7.41\times {10}^{-5}$	99.40%
0.001	70	$1.88\times {10}^{-5}$	$2.21\times {10}^{-5}$	99.66%

reports the numerical results of the proposed method. Figure 2 plots the mean squared error for $\mathsf{\Delta }x=0.01$. Figure 3 and Figure 4 show the approximated output of present method compared to the actual output for the training and testing data.

Example 2.

We consider the following FDE as:

$${\displaystyle \frac{\partial u(x,y,t)}{\partial t}}=d(x,y){\displaystyle \frac{{\partial }^{1.8}u(x,y,t)}{\partial x^{1.8}}}+e(x,y){\displaystyle \frac{{\partial }^{1.6}u(x,y,t)}{\partial y^{1.6}}}+q(x,y,t),$$

where $x,y\in (0,1)$ for $t\in [0,T_{end}]$ that $T_{end}=1$,

$$\begin{array}{c}\hfill d(x,y)={\displaystyle \frac{\mathsf{\Gamma }\left(2.2\right)x^{2.8}y}{6}},\\ \hfill e(x,y)={\displaystyle \frac{2x{y}^{2.6}}{\mathsf{\Gamma }\left(4.6\right)}},\\ \hfill q(x,y,t)=x^3{y}^{3.6},\end{array}$$

in which the initial conditions are: $u(0,y,t)=u(x,0,t)=0,u(x,y,0)=x^3{y}^{3.6},u(1,y,t)=y^{3.6}{e}^{-t}$ and $u(x,1,t)=x^3{e}^{-t}$ for $t\ge 0$.

The analytical solution of this problem is $u(x,y,t)=x^3{y}^{3.6}{e}^{-t}$. The function $u(x,y,t)$ is approximated using FRBF-NN. The learning rates were taken to be $\eta ={10}^{-2}$ and ${\eta }_{\nu }=3\times {10}^{-2}$. The parameters $\alpha$ and $\nu$ were set to $0.6$ and $0.6$, respectively.

Table 2. Numerical results for Example 2.

$\mathbf{\Delta }$t	$\mathbf{\Delta }$x = $\mathbf{\Delta }$y	Number of Kernels	MSE (Training)	MSE (Testing)	Approximation Accuracy
0.1	0.1	150	$1.20\times {10}^{-4}$	$7.71\times {10}^{-5}$	99.40%
0.05	0.05	400	$2.56\times {10}^{-5}$	$1.11\times {10}^{-5}$	99.75%

lists the numerical results obtained of the proposed method. Figure 5 displays the mean squared error plot for training phase. Figure 6 represents The absolute error surfaces for test data points in $t=0.16$ and train data points in $t=0$.

Example 3.

Finally, we consider the following FDE

$${\displaystyle \frac{\partial u(x,t)}{\partial t}}+{\displaystyle \frac{{\partial }^{\alpha }u(x,t)}{\partial t^{\alpha }}}={\mu }_1{\displaystyle \frac{{\partial }^{2}u(x,t)}{\partial x^2}}-{\mu }_2{\displaystyle \frac{\partial u(x,t)}{\partial x}}+f(x,t),$$

that $0<x<1$, $0\le t\le T_{end}=1$ and

$$\begin{array}{c}\hfill f(x,t)=10(-2+14x-18x^2+4x^3)(t+1)+10\left(1+{\displaystyle \frac{t^{1-\alpha }}{\mathsf{\Gamma }(2-\alpha )}}\right)x^2{(1-x)}^2,\end{array}$$

with the initial conditions $u(x,0)=10{(1-x)}^2{x}^2,u(1,t)=u(0,t)=0$ and $u(x,1)=20x^2{(1-x)}^2$.

The analytical solution of this example is $u(x,t)=10x^2{(1-x)}^2(1+t)$. The learning rates were taken to be $\eta ={10}^{-2}$ and ${\eta }_{\nu }={10}^{-2}$. The parameters $\alpha$ and $\nu$ were set to $0.4$ and $0.7$, respectively.

Table 3. Numerical results for Example 3.

$\mathbf{\Delta }$t	$\mathbf{\Delta }$x	Number of Kernels	MSE (Training)	MSE (Testing)	Approximation Accuracy
0.01	0.01	120	$8.37\times {10}^{-5}$	$5.31\times {10}^{-5}$	99.44%

displays the numerical results obtained for this example. Figure 7 shows the mean squared error plot for training phase. Figure 8 represents the absolute error surfaces for train and test data points.

5. Conclusions

This paper presented a method to approximate FDEs by using artificial neural network model. The presented technique is the convex combination of the fractional and classical gradient descents. By applying fractional derivatives, more information can be obtained compared with the classical derivative. Fractional derivative has the potential to do what integer-order derivative cannot. An important benefit of the fractional derivative over the ordinary derivative is that it extends the search region around the test point in specified boundaries. To test the effectiveness of the proposed method we used it to solve three FDEs. The results shown that implementation of this method is easy and it has high accuracy when applied to solve FDEs.