An Efficient Method for Solving Two-Dimensional Partial Differential Equations with the Deep Operator Network

Abstract

Partial differential equations (PDEs) usually apply for modeling complex physical phenomena in the real world, and the corresponding solution is the key to interpreting these problems. Generally, traditional solving methods suffer from inefficiency and time consumption. At the same time, the current rise in machine learning algorithms, represented by the Deep Operator Network (DeepONet), could compensate for these shortcomings and effectively predict the solutions of PDEs by learning the operators from the data. The current deep learning-based methods focus on solving one-dimensional PDEs, but the research on higher-dimensional problems is still in development. Therefore, this paper proposes an efficient scheme to predict the solution of two-dimensional PDEs with improved DeepONet. In order to construct the data needed for training, the functions are sampled from a classical function space and produce the corresponding two-dimensional data. The difference method is used to obtain the numerical solutions of the PDEs and form a point-value data set. For training the network, the matrix representing two-dimensional functions is processed to form vectors and adapt the DeepONet model perfectly. In addition, we theoretically prove that the discrete point division of the data ensures that the model loss is guaranteed to be in a small range. This method is verified for predicting the two-dimensional Poisson equation and heat conduction equation solutions through experiments. Compared with other methods, the proposed scheme is simple and effective.

1. Introduction

A partial differential equation (PDE) is a kind of equation that deals with the derivatives of unknown variables to time and space variables. These equations can describe natural phenomena (e.g., hydrodynamics [1] and electromagnetism [2]). In addition, it also describes practical applications (e.g., solving population problems [3] and urban traffic problems [4]). People often pay more attention to the analysis and solution of complex PDE models. In recent decades, the research of numerical methods for solving PDEs has never been interrupted. There are finite difference methods [5], finite element methods [6], spectral methods [7], boundary element methods [8], finite volume methods [9], Monte Carlo methods [10], and adaptive mesh refinement. With the development of the research, many new numerical methods appear (e.g., the fast curvilinear finite difference method [11], multiscale RBF collocation method [12], Haar wavelet collocation method [13], and sinc-collocation method [14]). However, we often need to analyze and design PDE solution algorithms via numerical methods. The calculation cost and resource consumption will increase as PDEs become more and more complex. Meanwhile, we can simplify PDEs by using the variational method, separation of variables method, Green function method, transformation method, and deep learning method. These make it possible to solve PDEs quickly and accurately.

With the rapid development of artificial intelligence technology, people pay more and more attention to using deep learning methods to analyze complex problems. A mesh-free algorithm called the deep Galerkin method [15] is proposed to approximate high-dimensional PDEs. In [16], the authors propose a scheme that embeds constraints in the Physics-informed neural networks of loss functions to predict PDEs with specific conditions. In addition, some PDE models have successfully solved practical application problems. For instance, Physics-informed neural networks [16,17] are used to predict the fluid flow in porous media [18] and simulate heat transfer [19]. Although these methods have successfully solved various problems, they are time-consuming, laborious, and not universal. When changing conditions, these strategy models need to be retrained.

The latest application of deep learning in PDEs comes from the universal approximation theory that shows that a neural network can accurately approximate any continuous nonlinear function or operator [20]. Based on this theory, Lu et al. [21] recently proposed a new neural network architecture called the Deep Operator Network (DeepONet). The architecture is composed of branch and trunk networks, which solve a class of partial differential equations through learning operators (maps between infinite-dimensional function spaces). To demonstrate the capability and effectiveness of DeepONet, some researchers have also analyzed the error and convergence and proved theoretically it can overcome the disaster of dimension [22]. Recent theories suggest that DeepONet has successfully solved various problems. In [23], it is used as a learning operator to predict the multiscale bubble dynamics. In [24], DeepONet is used to predict the linear unstable waves in the high-speed boundary layer according to the data provided in advance. In addition, DeepM&Mnet predicts a two-dimensional electric pair flow field [25] and predicts the coupled flow and finite rate chemical reaction after impacting normal [26] by integrating multiple parallel DeepONets. We consider that the high-dimensional problems most widely exist in a real scenario. Therefore, we focus on solving two-dimensional PDEs by using DeepONet.

For the original model, the branch network receives a function, which is a vector consisting of m discrete points, and the trunk network input is the information of spatial and time variables. However, when we use it to solve the two-dimensional PDEs, the input function is expressed as a matrix, and the spatial variables increase to two. At this time, the input function cannot be the branch network input, and the variable information of the trunk network input also increases. To cope with these challenges, Cai et al. [25] use a fully connected network and a pre-trained DeepONet to predict the 2D electroconvection problem by minimizing the total loss. In [22,27], the input function can be regarded as an image and the branch network can be replaced by a convolutional neural network (CNN) [28] to learn the mapping between two variables. However, these methods increase the complexity of the model and lack complete theoretical support.

To deal with model redundancy, we designed an efficient framework to solve two-dimensional PDEs based on DeepONet. First, starting with the input function, the input matrix is reshaped into a vector. The input function can be regarded as the input of the branch network. Second, because the two-dimensional PDEs contain more location and time information, we add spatial and time variable information as the input of the trunk network. Finally, according to the data form of the branch and trunk networks, we expand the data set required by DeepONet to approximate the solutions of PDEs. Firstly, the reduced dimension input function and the added spatial and temporal variable information form a binary. This part represents the input information of DeepONet. Then, the numerical solution of the corresponding variable is obtained according to the numerical solver, and these three parts together form a data set. In addition, we also designed a method to randomly select the location information from the corresponding time variables to consist of the data set. The proposed method can predict the solution of PDEs efficiently and accurately without increasing the complexity of the model. Moreover, it can solve a class of PDEs when only the input function is changed, so it is universal. All told, the main contributions of this paper are as follows.

• To obtain enough two-dimensional PDE data for training, we design a method to construct binary functions in the classical function space and obtain the corresponding PDE solutions through the difference method to form point-value data sets. To prove the DeepONet output is closer to the actual value, we analyze the constructed function theoretically that can be applied for model training.
• To solve the challenge of the form of the input data, we rely on DeepONet to design a new data dimensionality reduction algorithm. This method flattens the matrix data representing the function into vectors for inputting the branch network and randomly selects the location and time information as the trunk network input.
• The proposed strategy is performed on the two-dimensional Poisson equation and the heat conduction equation experiments. Compared with other methods, we verify that it is accurate and efficient in predicting two-dimensional PDEs.

This paper is organized as follows. The proposed method, data set, and training scheme are introduced in Section 2. Next, we formalize the problem and theoretically prove the relationship between the discrete points and model accuracy. The fourth part shows the experimental results of the proposed method on two-dimensional Poisson equations and heat conduction equations. Finally, we summarize this paper.

2. Methodology

In this section, we first review the work on deep operator networks and present a method for solving two-dimensional PDEs based on it. Next, we explain the source of the data set and the proposed training strategies.

2.1. Deep Operator Network Architecture

Here, we introduce the background of DeepONet. It is a simple neural network architecture that consists of two subnetworks. One is a branch network, which encodes the discrete points of the input function. The other one is a trunk network, which encodes the coordinate position of the output function. Each subnetwork is a feed-forward neural network (FNN). This network predicts PDE solutions by learning nonlinear operators from input to output. Generally, we define many points $[x_1,x_2,\cdots ,x_m]$ in domain f. $[f\left(x_1\right), f\left(x_2\right), \cdots , f\left(x_m\right)]$ represent the function f and the branch network inputs. The trunk network inputs are the time and space variables $x,t$, and two network outputs must be the same dimensional vectors (Figure 1) to conduct a dot-product operation. If $\Psi$ is an operator that maps from input to solution, the final output $\Psi \left(f\right)(x,t)$ is the solution of the PDEs.

When solving a one-dimensional partial differential equation, the input function is input into a branch network as a finite number of equidistant discrete points. The trunk network inputs are spatial variables and time variable information. However, when we solve two-dimensional PDEs, the input function is expressed as a matrix that cannot be the branch network inputs. Therefore, we propose a scheme to overcome this problem. For the branch network input, we convert the matrix representing the two-dimensional function into a vector (Figure 2). For the trunk network input, we extend it to two spatial variables and the time variable information. The proposed approach also uses a bias term for the output. Algorithm 1 represents the dimensionality reduction process of the input function. It means taking out each row of the matrix and appending it to the end of the previous row.

Algorithm 1 Function dimensionality reduction

input function $f(x,y)$ output discrete points vector F represents the function $f(x,y)$ 1: using matrix F to represent function $f(x,y)$ 2: for each row of the matrix do 3:     append to the previous line 4: end for 5: thus, matrix F becomes vector F

2.2. The Data Set and Training Scheme

The data set consists of two parts: input and output. Due to input functions being essential for model training, we designed a random sampling input functions method. It is divided into two steps. In step one, we obtain univariate functions in the Gaussian random field (GRF).

$$R\sim Q(0,k_l(x_1,x_2))$$

(1)

$$Y\sim Q(0,k_l(x_1,x_2))$$

(2)

where $R,Y$ are different input signals, k is the radial-basis function (RBF) kernel, $l>0$ is a length-scale parameter, and $k_l(x_1,x_2)=exp(-||x_1-x_2{||}^2/2l^2)$. If l is larger, the function R and Y is smoother. In step two, we randomly choose two univariate functions for a linear combination. $c_1,c_2$ are constants.

$$\begin{array}{c}\hfill F=c_{1}R+c_{2}Y\end{array}$$

(3)

The variable information of the trunk network is location points randomly selected according to the range of independent variables. In addition, the most important is to obtain PDE solutions to train the model. Because it is difficult to gain the analytical solution, we consider using finite difference methods to obtain the numerical solution. A training point is a triple $(F,(x,y,t),\Psi (F\left)\right(x,y,t\left)\right)$. It may correspond to multiple training data points at different positions for specific input. For example, if the training data points are 36,000 and the number of functions is 200, each input function includes 180 training data points at different locations. Because the data points are randomly selected, they have different positions in various inputs. Then, we consider two methods of combining the data sets. In one, we randomly form triplets with two spatial variables and a time variable $x,y,t$; the input function F; and the result value $\Psi \left(F\right)(x,y,t)$. In the other, we randomly sample points in each period and form triples $(F,(x,y,t),\Psi (F\left)\right(x,y,t\left)\right)$. Algorithms 2 and 3 display the two training methods.

Algorithm 2 Training strategy one

input input function F, independent variable $x,y,t$ output triples $(F,(x,y,t),\Psi (F\left)\right(x,y,t\left)\right)$ 1: A random combination of x, y, t 2: set $x_j=[x_1,\cdots ,x_m],y_i=[y_1,\cdots ,y_m],t_k=[t_1,\cdots ,t_n].j,i=1,2,\cdots ,m.k=1,2,\cdots ,n.$ 3: for training points do 4:     randomly select $x_j$ 5:     randomly select $y_i$ 6:     randomly select $t_k$ 7: end for 8: form the matrix $(x_j,y_i,t_k)$ and change into $(x,y,t)$ 9: Processing of PDE solution 10: obtain PDE solutions through solvers 11: transform into the matrix $\Psi \left(F\right)(x,y,t)$ 12: combine with matrix $(x,y,t)$ and get two tuples $\left(\right(x,y,t),\Psi (F\left)\right(x,y,t\left)\right)$ 13: add F and get triples $(F,(x,y,t),\Psi (F\left)\right(x,y,t\left)\right)$

Algorithm 3 Training strategy two

input input function F, independent variable $x,y,t$ output triples $(F,(x,y,t),\Psi (F\left)\right(x,y,t\left)\right)$ 1: Processing of time period 2: set $t_k=[t_1,\cdots ,t_n],k=1,2,\cdots ,n.$ 3: A random combination of $x,y,t$ 4: set $x_j=[x_1,\cdots ,x_m],y_i=[y_1,\cdots ,y_m].j,i=1,2,\cdots ,m.$ 5: for n times do 6:     for training points do 7:         randomly select $x_j$ 8:         randomly select $y_i$ 9:     end for 10: end for 11: combine $t_k$ to form the matrix $(x_j,y_i,t_k)$ and change into $(x,y,t)$ 12: Processing of PDE solution 13: obtain PDE solutions through solvers 14: transform into the matrix $\Psi \left(F\right)(x,y,t)$ 15: combine with $(x,y,t)$ and get two tuples $\left(\right(x,y,t),\Psi (F\left)\right(x,y,t\left)\right)$ 16: add F and get triples $(F,(x,y,t),\Psi (F\left)\right(x,y,t\left)\right)$

3. Problem Formulation

In this part, we demonstrate an influence on model accuracy with the number of discrete points. We keep the two-dimensional input function sampling space from the GRF and design a method to sample it. Because the sampling space remains unchanged and the input function is a two-dimensional function, it is necessary to extend the proof. Suppose that the two-dimensional heat conduction problem with the source term is described as follows:

$$\left\{\begin{array}{c}\frac{\partial s}{\partial t}=g[s(x,y,t),f(x,y),(x,y)]\hfill \\ IC:s(x,y,0)=s_0\hfill \\ BC:0\hfill \end{array}\right.$$

(4)

$f(x,y)=f_1\left(x\right)+f_2\left(y\right)\in V$ (compact set of Banach space), and the domain of s is $[n,k]\times [n,k]\to {\mathbb{R}}^k$. $\Psi$ is an operator that is from f to s, (${\Psi }_f(x,y,t)=s$). ${\Psi }_f$ satisfies

$${\Psi }_f(x,y,t)=s_0+\int \int \int g[{\Psi }_f(l,p,h),f(l,p),(l,p)]dldpdh$$

and choose $(m+1)\times (m+1)$ points in the domain.

$$x_j=n+\frac{j(k-n)}{m},j=1,2,\cdots ,m.$$

$$y_i=n+\frac{i(k-n)}{m},i=1,2,\cdots ,m.$$

where $x_j,y_i\in [n,k].$ Define the function $f_m(x,y)=f_m\left(x\right)+f_m\left(y\right)$.

$$f_m\left(x\right)=f\left(x_j\right)+\frac{f\left(x_{j+1}\right)-f\left(x_j\right)}{x_{j+1}+x_j}(x-x_j)$$

$$f_m\left(y\right)=f\left(y_i\right)+\frac{f\left(y_{i+1}\right)-f\left(y_i\right)}{y_{i+1}+y_i}(y-y_i)$$

$$f_m(x,y)=f\left(x_j\right)+\frac{f\left(x_{j+1}\right)-f\left(x_j\right)}{x_{j+1}+x_j}(x-x_j)+f\left(y_i\right)+\frac{f\left(y_{i+1}\right)-f\left(y_i\right)}{y_{i+1}+y_i}(y-y_i)$$

$\mathsf{{\rm Y}}$ is an operator that is from $f(x,y)$ to $f_m(x,y)$. $P_m={\mathsf{{\rm Y}}}_m\left(f\right)|f\in V$, V is a compact set of the Banach space; V is compact and $\mathsf{{\rm Y}}$ is also compact. Hence, $Z_m:=V\cap P_m$ also remain compact. According to Lemma 1, $Z:={\bigcup }_{i=1}^{\infty }{Z}_i$ is also compact. $\Psi$ is a continuous implicit operator, and $\Psi \left(Z\right)$ is a compact set. Suppose $g(s,f,(x,y\left)\right)$ satisfies the Lipschitz condition, $s,f\in \Psi \left(Z\right)\times Z$, there exists a constant c such that

$$||g(s_1,f,(x,y))-g(s_2,f,(x,y)){\left|\right|}_2⩽c||s_1-s_2{\left|\right|}_2$$

$$||g(s,f_1,(x,y))-g(s,f_2,(x,y)){\left|\right|}_2⩽c||f_1-f_2||$$

To achieve this condition, g is differentiable about f and s on $\Psi \left(Z\right)\times Z$. For $f\in V,f_m\in P_m$, $c_1,c_2$ and $s(m,V)$ are constant such that

$$\underset{x,y\in [n,k]}{Max}|f(x,y)-f_m(x,y)|⩽(c_1+c_2)s(m,V)$$

(5)

$(c_1+c_2)s(m,V)\to 0$ when $m\to \infty$ According to Theorem 1, we have $s\sim \frac{1}{m^2{l}^2}$. Based on the above deduction, we obtain Theorem 2.

Lemma 1.

$Z:={\bigcup }_{i=1}^{\infty }{Z}_i$ is compact.

Proof. 

For any $\epsilon >0$, by (5), there exists an $m_0$, such that:

$$\left|\right|f-{\mathsf{{\rm Y}}}_m\left(f\right){||}_c<\frac{\epsilon }{4},\forall f\in V,\forall m>m_0$$

$Z_{m_0}$ is a compact set, and there exists a $\delta >0$ such that:

$$|(x_1,y_1)-(x_2,y_1)| <\delta ⟹|f(x_1,y_1)-f(x_2,y_1)| <\frac{\epsilon }{2}$$

$$|(x_1,y_1)-(x_1,y_2)| <\delta ⟹|f(x_1,y_1)-f(x_1,y_2)| <\frac{\epsilon }{2}$$

$\forall f\in Z_{m_0},\forall x_1,x_2,y_1,y_2\in [n,k]$. All the variables satisfy the above conditions and $f\in Z$. If $f\in Z_{m_0}$, we have

$$|f(x_1,y_1)-f(x_2,y_1)| <\frac{\epsilon }{2}<\epsilon$$

$$|f(x_1,y_1)-f(x_1,y_2)| <\frac{\epsilon }{2}<\epsilon$$

otherwise, $f\in {\bigcup }_{i=m_0+1}^{\infty }{P}_i$; suppose $f={\mathsf{{\rm Y}}}_m\left(k\right),m>m_0,k\in V$,

$$|f(x_1,y_1)-f(x_2,y_1)| = |s(x_1,y_1)-k(x_1,y_1)+k(x_1,y_1)-k(x_2,y_1)+k(x_2,y_1)-s(x_2,y_1)|$$

$$\le |{\mathsf{{\rm Y}}}_m\left(k\right)(x_1,y_1)-k(x_1,y_1)| + |k(x_1,y_1)-k(x_2,y_1)| + |{\mathsf{{\rm Y}}}_m\left(k\right)(x_2,y_1)-k(x_2,y_1)|$$

$$\le 2|{\mathsf{{\rm Y}}}_m{\left(k\right)-k|}_c+|k(x_1,y_1)-k(x_2,y_1)|\le 2\times \frac{\epsilon }{4}+\frac{\epsilon }{2}=\epsilon .$$

Similarly, $|f(x_1,y_1)-f(x_1,y_2)| \le \epsilon .$ Z is uniformly bounded and successive, so it is pre-compact through the Arzelà–Ascoli theorem. Define ${\left\{z_i\right\}}_{i=1}^{\infty }\subset Z$, which is a sequence that converges to $z_0\in c[n,k]$. If an m exists such that $\left\{z_i\right\}\subset Z_m$, we have $z_0\in Z_m\in Z.$ Otherwise, we have a subsequence $\left\{Y_{i_n}\left(v_{i_n}\right)\right\}$ of $\left\{z_i\right\}$, such that $v_{i_n}\subset V$, when $n\to \infty ,i_n\to \infty$, and we have

$$\left|\right|v_{i_n}-z_0{||}_c=\left|\right|v_{i_n}-{\mathsf{{\rm Y}}}_{i_n}\left(v_{i_n}\right)+{\mathsf{{\rm Y}}}_{i_n}\left(v_{i_n}\right)-z_0{||}_c$$

$$\le \left|\right|v_{i_n}-{\mathsf{{\rm Y}}}_{i_n}\left(v_{i_n}\right){||}_c+\left|\right|{\mathsf{{\rm Y}}}_{i_n}\left(v_{i_n}\right)-z_0{||}_c$$

$$\le \kappa (i_n,V)+\left|\right|{\mathsf{{\rm Y}}}_{i_n}\left(v_{i_n}\right)-z_0{||}_c.$$

It means $z_0=\underset{n\to \infty }{lim}{v}_{i_n}\in V\subset Z$. Hence, Z is closed. □

Theorem 1.

$R\left(s\right),Y\left(c\right)\sim Q(0,k_l(x_1,x_2))$, using a piecewise linear interpolation of $R\left(s\right),Y\left(c\right)$ with m points will convergence with order $O\left(\frac{2}{{\left(ml\right)}^2}\right)$.

Proof. 

Suppose that $R\left(s\right),Y\left(c\right)$ are functions that sample in the Gaussian random field.

$$R\left(s\right)=\sqrt{2}{\left(\pi \right)}^{\frac{1}{4}}({\int }_{{\mathbb{R}}^{+}}\sqrt{l}cos\left(ks\right)e^{-\frac{l^2{k}^2}{8}}dW\left(k\right)-\sqrt{l}sin\left(ks\right)e^{-\frac{l^2{k}^2}{8}}dB\left(k\right))$$

$$Y\left(c\right)=\sqrt{2}{\left(\pi \right)}^{\frac{1}{4}}({\int }_{{\mathbb{R}}^{+}}\sqrt{l}cos\left(kc\right)e^{-\frac{l^2{k}^2}{8}}dW\left(k\right)-\sqrt{l}sin\left(kc\right)e^{-\frac{l^2{k}^2}{8}}dB\left(k\right))$$

$W,B$ are independent standard Brownian motions [29]. Setting $T=lk$ and applying linear interpolations ${\Pi }_1,{\Pi }_2$ in $[s_i,s_{i+1}],[c_i,c_{i+1}]:$

$$H[{(R(s)-{\Pi }_{1}R(x))}^2]=2{(\pi )}^{\frac{1}{2}}[(\int {((I-{\Pi }_1)cos(\frac{T}{l}s))}^2+\int {((I-{\Pi }_1)sin(\frac{T}{l}s))}^2)e^{-\frac{T^2}{l}}dT]$$

$$\le {(\pi )}^{\frac{1}{2}}{(S_{i+1}-S_i)}^4\int {(\frac{T}{l})}^4{e}^{-\frac{T^2}{l}}dT=24\pi {(\frac{S_{i+1}-S_i}{l})}^4.$$

Similarly, $H[{(Y(c)-{\Pi }_{1}Y(c))}^2]\le 24\pi {(\frac{S_{i+1}-S_i}{l})}^4.$ By the linear interpolation error, we have:

$$|(I-{\Pi }_1)g\left(s\right)| =\frac{{(b-a)}^2|f^{{}^{″}}\left({\epsilon }_1\right)|}{2}$$

$$|(I-{\Pi }_2)g\left(s\right)| =\frac{{(b-a)}^2|f^{{}^{″}}\left({\epsilon }_2\right)|}{2}$$

Applying the Borel–Cantelli lemma, we have

$$|R\left(s\right)-{\Pi }_{1}R\left(s\right)+Y\left(c\right)-{\Pi }_{2}Y\left(c\right)|\le \frac{k[{(S_{i+1}-S_i)}^{2-ϵ}+{(C_{i+1}-C_i)}^{2-ϵ}]}{l^2}$$

k is a positive number of a Gaussian random variable with finite variance. Applying the piecewise linear interpolation of $R\left(s\right)$ and $Y\left(c\right)$ with m points, we can obtain convergence with order $O(\frac{2}{{\left(ml\right)}^2})$. □

Theorem 2.

Suppose that m is a positive integer, making $c(k-n)(c_1+c_2)s(m,V)e^{c(k-n)}<\epsilon$, for $d,e\in [n,k]$, function $f\in V$, there exists $W_1\in {\mathbb{R}}^{n\times (m+1)}$, $b_1\in {\mathbb{R}}^{(m+1)}$, $W_2\in {\mathbb{R}}^{k\times n}$, such that:

$$\left|\right|(\Psi )(d,e)-(W_2·\sigma (W_1·{[f(x_1,y_1),\cdots ,f(x_m,y_m)]}^T+b_1)+b_2){||}_2<\epsilon .$$

Proof. 

For $f\in V,f_m\in P_m$, by (5) and the Lipschitz condition, such that:

$$\left|\right|\left({\Psi }_f\right)(d,e)-\left({\Psi }_{f_m}\right)(d,e){||}_2\le c\int \int |\left({\Psi }_f\right)(x,y)-\left({\Psi }_{f_m}\right)(x,y)|dxdy$$

$$+c\int \int |f(x,y)-f_m(x,y)|dxdy\le c\int \int |\left({\Psi }_f\right)(x,y)-\left({\Psi }_{f_m}\right)(x,y)|dxdy$$

$$+c{(k-n)}^2(c_1+c_2)s(m,V)$$

According to the Gronwall inequality, we obtain

$$\left|\right|\left({\Psi }_f\right)(d,e)-\left({\Psi }_{f_m}\right)(d,e){||}_2\le c{(k-n)}^2(c_1+c_2)s(m,V)e^{\int \int 1dxdy}$$

$$\le c{(k-n)}^2(c_1+c_2)s(m,V)e^{c{(k-n)}^2}$$

Define $L_m=(f(x_1,y_1),f(x_1,y_2),\cdots ,f(x_m,y_m))\in {\mathbb{R}}^{m+1}$, $f\in V$, and $L_m$ is a compact set. There is a mapping relationship between $L_m$ and $P_m$. We define a vector-valued function on $L_m$, such that:

$$\rho (f(x_1,y_1),f(x_1,y_2),\cdots ,f(x_m,y_m))=\left({\Psi }_{f_m}\right)(d,e)$$

For any $\epsilon >0,$ when $m\to \infty ,$$c{(k-n)}^2(c_1+c_2)s(m,V)e^{c{(k-n)}^2}<\epsilon .$ Applying the universal approximation theorem of a neural network for high-dimensional functions:

$$\left|\right|\rho (f(x_1,y_1),f(x_1,y_2),\cdots ,f(x_m,y_m))-(W_2·\sigma (W_1·{[f(x_1,y_1),\cdots ,f(x_m,y_m)]}^T+b_1)+b_2){||}_2$$

$$\le \epsilon -c{(k-n)}^2(c_1+c_2)s(m,V)e^{c{(k-n)}^2})$$

Thus, we have:

$$\left|\right|\left({\Psi }_f\right)(d,e)-(W_2·\sigma (W_1·{[f(x_1,y_1),\cdots ,f(x_m,y_m)]}^T$$

$$+b_1)+b_2{)||}_2\le \left|\right|\left({\Psi }_f\right)(d,e)-\left({\Psi }_{f_m}\right)(d,e){||}_2+\left|\right|\left({\Psi }_f\right)(d,e)$$

$$-(W_2·\sigma (W_1·{[f(x_1,y_1),\cdots ,f(x_m,y_m)]}^T+b_1)+b_2){||}_2$$

$$\le c{(k-n)}^2(c_1+c_2)s(m,V)e^{c{(k-n)}^2})+\epsilon$$

$$-c{(k-n)}^2(c_1+c_2)s(m,V)e^{c{(k-n)}^2})=\epsilon$$

□

The number of discrete points only affects the smoothness of the PDE curve. The more discrete points, the smoother the curve. With the increase in discrete points, the mesh segmentation becomes finer. The amount of data required by the model will continue to increase, but it has no impact on the accuracy. We can always make the model achieve a better effect.

4. Experimental Results

In this section, we conduct experiments using the proposed method. For the data sets, we obtain functions and solutions through the GRF and the numerical solver of the PDEs. In the prediction aspect, we exhibit the prediction ability of the proposed method based on the two-dimensional Poisson and heat conduction equation. In addition, we analyze the experimental results by the losses and compare the predictions with numerical solutions. The loss function is the mean square error (MSE). As shown in the following,

$$MSE=\frac{1}{n}\sum _{i=1}^n{(Y_i-\widehat{Y_i})}^2$$

n is the number of data, $\widehat{Y_i}$ is the predicted value, and Y is the true value. This indicator can directly show the prediction effect.

4.1. Experimental Setup

The underlying framework for implementing DeepONet is TensorFlow. We train DeepONet using the Windows 10 operating system, with a 32 G memory, an NVIDIA GeForce RTX 3080 Ti graphics card, and Python as the programming language. In addition, the hyperparameters are set as follows. The learning rate is 0.001, the epoch is 20,000, the activation function is ReLU, and the optimizer is Adam. The width and depth of the branch and the trunk network are 40 and 2.

4.2. Poisson Equation

The two-dimensional Poisson equation is described as:

$$(\frac{{\partial }^{2}s}{\partial x^2}+\frac{{\partial }^{2}s}{\partial y^2})=f(x,y),x,y\in [0,1]$$

with zero boundary conditions and a fixed f. We aim to predict $s(x,y)$ over the domain $[0,1]$ for any $f(x,y)$. The objective is to learn an operator from function f to s. The selected function space is the GRF, the number of discrete points is 20 × 20, the number of training and test functions is 200/40, and the number of training data points of each function is 120. They are the solutions of the PDEs, which can be obtained by the five-point difference method, as shown in Figure 3.

The training result is shown in Figure 4. The loss will decrease with the iterations’ increase and fluctuate in a small range until stable. The best test loss is $8.04\times {10}^{-6}$. Next, we use the trained model to predict the PDE. We choose the input function $f=-5e^{\frac{[{(x-\frac{1}{4})}^2+{(y-\frac{1}{4})}^2]}{4}}$, and the numerical solution and the prediction of DeepONet are shown in Figure 3 and Figure 5.

Compared with Figure 3, the prediction result of DeepONet is almost the same as the numerical result. Moreover, the models also show good generalization. When the input functions are $f_1=xsin\left(\pi x\right)+ysin\left(\pi y\right)$ and $f_2=-xsin\left(\pi x\right)-ycos\left(\pi y\right)$, the numerical solution and the prediction are shown in Figure 6. The prediction effects are excellent when the functions are in the same function space.

Figure 6a,b are the DeepONet prediction results, and Figure 6c,d are the PDE numerical solutions. The results indicate that DeepONet can accurately predict the two-dimensional Poisson equation. Compared with the traditional numerical solution methods, the DeepONet framework is flexible because it is a pre-trained model. As long as the corresponding input functions are set, DeepONet can predict this kind of PDE.

4.3. Heat Conduction Equation

The two-dimensional heat conduction PDE with the source term $f(x,y)$ is described by

$$\frac{\partial s}{\partial t}=a^2(\frac{{\partial }^{2}s}{\partial x^2}+\frac{{\partial }^{2}s}{\partial y^2})+f(x,y),x,y,t\in [0,1]$$

with a zero boundary condition, and the initial condition is

$$s_0=e^{-10[{(x-\frac{1}{2})}^2+{(y-\frac{1}{2})}^2]}$$

where $a=1$ is a constant. We will train DeepONet to learn the implicit operator that is a mapping from $f(x,y)$ to $s(x,y,t)$. According to the dimension reduction algorithm of the input function and the model training method under a multivariable, we show six different experimental results for DeepONet. These experiments include the number of input functions, the number of training points, increasing discrete points, different function spaces, different training methods, and different models. We analyze the experimental results by comparing the prediction and numerical solution. The purpose is to find the best model and accurately predict the two-dimensional heat conduction equation.

4.3.1. Number of Input Functions

The data set size is expressed as the product of the input function and the training data points. One of the factors affecting it is the number of input functions. To maintain the unity of other variables, we set them in

Table 1. The number of input functions is different, and other parameters are unchanged.

Case	Training Data Points	Functions	Discrete Points	Space	MSE
1	180	50/10	20 × 20	GRF	7.64 × 10−4
2	180	60/12	20 × 20	GRF	4.43 × 10−4
3	180	80/16	20 × 20	GRF	2.87 × 10−4
4	180	110/22	20 × 20	GRF	1.59 × 10−4
5	180	150/30	20 × 20	GRF	3.28 × 10−5
6	180	200/40	20 × 20	GRF	2.53 × 10−5
7	180	300/60	20 × 20	GRF	1.46 × 10−5

.

To control the variables, we only change the number of functions. After obtaining the results through the numerical solver, 180 data points are randomly selected. These points are part of the numerical solutions. Each function randomly selects data points at different locations, as shown in Figure 7.

For the different numbers of functions, the training results are different. The losses of the models are shown in Figure 8.

The dotted lines represent the test loss, and the solid lines represent the training loss. The different colors represent the model training status corresponding to the number of input functions. When the number of input functions is small, the model will be under-fitted, and the test loss is far greater than the training loss. With the increase in the number of input functions, the gap between the test and training loss becomes smaller and smaller, and the test loss of the model becomes lower until the number of functions reaches convergence at 300/60. At this time, the optimal loss of the trained model is $1.46\times {10}^{-5}$. Generally, we evaluate the model generalization ability on the test set. When the test loss is low, the model generalization ability is strong. Therefore, the number of functions we choose is 300/60.

We set function $f(x,y)=e^{-\frac{[{(x-\frac{1}{4})}^2+{(y-\frac{1}{4})}^2]}{4}}$, and DeepONet predicts the best results, which are shown in Figure 9. Figure 9a–f show the solution of the two-dimensional heat conduction equation predicted by DeepONet and the process of the PDE changing with time. For comparison, we show the corresponding numerical solution in Figure 10.

For seven cases, the MSE is shown in Table 1. It shows that the larger the number of functions, the smaller the MSE, and the better the model training, but the training time is longer and longer. When the number of functions reaches 200/40, the model loss is ∼${10}^{-5}$ and does not decrease. At this time, the model has reached convergence.

4.3.2. Number of Training Data Points

Training data points are a significant part of the data sets, and their number also determines the size of the data sets. To improve the model training efficiency, we will find appropriate data points. We randomly select 20, 40, 70, 110, 160, 220, and 300 training data points in each corresponding input function. Similarly, we designed

Table 2. The number of training data points is different, and the other parameters are unchanged.

Periods	Training Data Points	Functions	Discrete Points	Space	MSE
6	20	200/40	20 × 20	GRF	1.17 × 10−3
6	40	200/40	20 × 20	GRF	1.24 × 10−4
6	70	200/40	20 × 20	GRF	5.36 × 10−5
6	110	200/40	20 × 20	GRF	2.43 × 10−5
6	160	200/40	20 × 20	GRF	1.81 × 10−5
6	220	200/40	20 × 20	GRF	1.59 × 10−5
6	300	200/40	20 × 20	GRF	1.84 × 10−5

to show the control variables.

According to the number of input functions, periods, and training data points, we can calculate the training times of each data point in the grid. When choosing 40 as the number of training data points, the training time of each data point in the grid is 200 × 40/400 × 6, which is about 3. The specific experimental results are shown in Figure 11. It shows the model loss under different data points, and the colors represent the distinction of the training data points. With the increase in the iterations, the loss change also shows the same trend. When the number of training data points is less than 110, there exists a big gap.

When the number of training data points is more than 110, the test loss gradually reaches the fitting state. It is ${10}^{-5}$. As more and more training data points are selected, the test loss is gradually consistent with the training loss. Especially, the number of iterations exceeds 20,000. When the number of training data points is 220, the test loss is almost the same as the training loss, and the model generalization is the strongest. The best test loss in the different training data points is shown in Table 2.

In Table 2, the test losses become smaller with the increase in the training points. After 70 training points, the best test loss remains at ∼${10}^{-5}$. We select around 220 training points, the 20 × 20 grid size, and the $200\times 220/400\times 6\approx 18$ training times. The model training effect is best when each point is trained about 18 times.

4.3.3. Different Function Spaces

In the previous parts, we mentioned the method of the sampling function. Firstly, we randomly sample univariate functions in the GRF and then a random linear combination between two univariate functions. However, according to the work of [21], the classical function space can also select a Chebyshev polynomial. Therefore, in this part, we also consider sampling the function in this space as the data sets and training the model. According to the comparison of the GRF, we explore which function space is more suitable for sampling two-dimensional functions.

The numbers of the functions and data points are set to 200 and 220. The input function from the Chebyshev polynomial space is as follows: let $k>0$, and $F_i$ are Chebyshev polynomials of the first kind. We define the orthogonal polynomials of degree N as:

$$V_p=\sum _{i=0}^{N-1}{a}_i{F}_i\left(x\right):|a_i|⩽k$$

We obtain the input functions from $V_p$ by randomly sampling $a_i$ from [−k, k]. The model training loss corresponding to the two function spaces is shown in Figure 12. When the sampling space comes from the GRF or Chebyshev polynomials, the blue and red lines in Figure 12 represent the corresponding training and test losses. By comparing the loss changes in the model, we find that the model loss represented by red is slightly greater than that of blue. The experimental results show that under the state of the other same parameters, the generalization error of the model training is small in the GRF.

The prediction results with the same f are shown in Figure 13. It shows that when the function space is Chebyshev polynomials, DeepONet cannot predict the solutions well. Although the generalization abilities of the model are poor in this state, they can improve in the GRF.

4.3.4. Discrete Points

According to the theoretical analysis in part 3, the more discrete points, the more able to represent the input function. We send the discrete input function as a branch network input to DeepONet. As long as the loss of the model is less than $\epsilon$, DeepONet can accurately predict the solution of the two-dimensional partial differential equation. However, when the number of m gains, the mesh and function image become denser and smoother. If the data set of the same size is maintained, the larger m is, there will be fewer training times of each data point, which may lead to a bad result for the model training, as shown in Figure 14a.

Blue, red, and green are the changes in the model training loss when m = 50, 45, and 25. When the same data set is maintained, the model loss will increase to the expansion of several m. However, if the number of m increases, we can improve the number of training data points or input functions (expand the training times of each data point) to reduce the model loss. The results are shown in Figure 14b. While keeping the other parameters unchanged and increasing the training data points, the green line in Figure 14b shows the loss change during the model training. The training and test losses decrease to the best state with the iterations’ increase. Then, it tends to be stable. When increasing the number of input functions, the loss change is shown by the red line in Figure 14b. The test and training losses keep a similar trend, and the optimal test loss is smaller than the blue line. Therefore, we can increase the number of input functions (expand the data set) while increasing the training data points, and it can ensure that the model training is better, as shown in Figure 15.

The blue line indicates the change in the loss when the discrete points increase to 40 × 40. The green line represents the change in the model loss after the data set increases. In comparison, the best loss of the blue lines is greater than that of the green. According to the records in

Table 3. The MSE of different discrete points.

m	Data Points	Functions	MSE
20	220	200/40	1.48 × 10−5
25	320	300/60	1.61 × 10−5
40	600	400/80	5.79 × 10−5
50	600	400/80	6.76 × 10−5

, for different discrete points, the optimal loss of the model will eventually reach ${10}^{-5}$. It shows that when the discrete points gradually increase, the model will always ensure that the optimal MSE is small enough to make the prediction results accurately.

4.3.5. Different Training Method

In part two, we give two training methods. The first is to extend the training method to a two-dimensional PDE according to the work of [21] in Algorithm 2. The second is to divide the period into n equal parts into blocks and randomly select training points in each block for training in Algorithm 3. In this part, we reproduce the two training methods and illustrate that these methods are effective according to the change in the model training loss. We obtain the solution of the PDE corresponding to six time periods and use these two methods to train the model. The results are shown in Figure 16a.

In Figure 16a, red and green represent the change in the model training loss corresponding to the origin and our algorithms. From the figure, we can see that no matter the training or test loss, they almost maintain the same change trajectory, and the loss gradually decreases and tends to fit with the increase in the iterations. Finally, the best test loss remains at ${10}^{-5}$. However, we speculate that the reason may be that there are too few periods to see the advantages. Therefore, we will select ten discrete-time variable training periods, and the change in the model loss is shown in Figure 16b. The experimental results show that the two training algorithms can train the model well. Both can minimize the loss of the model.

4.3.6. Different Models

By the work of [21], we considered two models, FNN and Residual Networks (ResNet), because the input has no specific structure. In this part, we still regard these two models as comparative models. We apply the improved data set to train the two models, and the experimental results are shown in Figure 17.

The red lines indicate the training result of FNN. With the increase in iterations, the training loss decreases continuously. However, the test loss will not change when it reaches a certain extent. The green lines represent the training results of ResNet. We can see that the training and test loss changes are very tortuous and unstable. The training results of the two models are much worse than DeepONet.

The best training error of DeepONet is 1.28 × 10⁻⁵, and the best test error is 3.56 × 10⁻⁵. The best training error of FNN is 6.29 × 10⁻⁵, and the best test error is 1.27 × 10⁻⁴. The best training error of ResNet is 8.98 × 10⁻⁵, and the test error is 3.01 × 10⁻⁴. Through the training of FNN and ResNet, we found that whether the FNN or ResNet model are used, their training results are worse than DeepONet.

In addition, DeepM&Met indicates that a two-dimensional PDE can also be predicted by using a fully connected network and the pre-trained DeepONet. This method mainly solves multi-physical and multiscale problems, but adding an FNN is redundant for a two-dimensional PDE with simple prediction. In addition, the model complexity and prediction time will increase when the branch network is replaced by a convolution neural network. As shown in Figure 18, the prediction accuracy of the method using a CNN to replace the branch network is consistent with our proposed method. However, in terms of the training time and prediction time, our method shows absolute advantages, as shown in

Table 4. Comparison with CNN.

Metrics	Our Method	CNN
Loss	5.34 × 10−5	5.63 × 10−5
Training time	75 s	348 s
Prediction time	0.009 s	0.03 s

.

The above experiments indicate that our proposed method has a strong operator learning ability. When dealing with a two-dimensional PDE, it can predict the corresponding solution by learning the implicit operator. We also analyze various factors affecting the training of the DeepONet model and select the optimal parameters. Although the prediction accuracy is lower than that of the numerical solver and close to the methods based on a CNN, our proposed method saves time and labor in solving two-dimensional PDEs.

5. Conclusions

This paper presents an efficient method based on DeepONet for solving two-dimensional PDEs and takes the two-dimensional Poisson and heat conduction equation as research problems. Moreover, we theoretically analyze the influence of the number of discrete points corresponding to the two-dimensional functions on the model accuracy and verify it through experiments. In addition, many experimental results show that the proposed method can accurately predict two-dimensional PDEs. Compared with the numerical solution, the mean square error of the prediction solution can reach ${10}^{-5}$. Compared with other methods, it is efficient. However, our approach considers many simple nonlinear input functions and ignores some complex function forms. In the future, we would like to improve the algorithm and expand the model to solve more complex partial differential equations.