# Prediction of Discretization of GMsFEM Using Deep Learning

^{1}

^{2}

^{3}

^{4}

^{5}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Preliminaries

## 3. Deep Learning for GMsFEM

- ${g}_{B}^{m,i}$ maps the permeability coefficient $\kappa $ to a local multiscale basis function ${\varphi}_{m}^{{\omega}_{i}}$, where i denotes the index of the coarse block, and m denotes the index of the basis in coarse block ${\omega}_{i}$$${g}_{B}^{m,i}:\kappa \mapsto {\varphi}_{m}^{{\omega}_{i}}(\kappa ),$$
- ${g}_{M}^{l}$ maps the permeability coefficient $\kappa $ to the coarse grid parameters ${A}_{c}^{{K}_{l}}$ ($l=0,\cdots ,12$)$${g}_{M}^{l}:\kappa \mapsto {A}_{c}^{{K}_{l}}(\kappa ).$$

#### 3.1. Network Architecture

#### 3.2. Network-Based Multiscale Solver

## 4. Numerical Results

- in experiment 1, the channel configurations were all distinct, and the permeability coefficients inside the channels were fixed in each sample (see Figure 5 for illustrations), and
- in experiment 2, the channel configurations were randomly chosen among five configurations, and the permeability coefficients inside the channels followed a random distribution (see Figure 6 for illustrations).

- For the multiscale basis function ${\varphi}_{m}^{{\omega}_{i}}$, we built a network ${\mathcal{N}}_{B}^{m,i}$ using
- –
- Input: vectorized permeability pixels values $\kappa $,
- –
- Output: coefficient vector of multiscale basis ${\varphi}_{m}^{{\omega}_{i}}(\kappa )$ on coarse neighborhood ${\omega}_{i}$,
- –
- Loss function: mean squared error $\frac{1}{N}\sum _{j=1}^{N}\left|\right|{\varphi}_{m}^{{\omega}_{i}}({\kappa}_{j})-{\mathcal{N}}_{B}^{m,i}({\kappa}_{j};{\theta}_{B}){\left|\right|}_{2}^{2}$,
- –
- Activation function: leaky ReLu function,
- –
- DNN structure: 10–20 hidden layers, each layer have 250–350 neurons,
- –
- Training optimizer: Adamax.

- For the local coarse scale stiffness matrix ${A}_{c}^{{K}_{l}}$, we build a network ${\mathcal{N}}_{M}^{l}$ using
- –
- Input: vectorized permeability pixels values $\kappa $,
- –
- Output: vectorized coarse scale stiffness matrix ${A}_{c}^{{K}_{l}}(\kappa )$ on the coarse block ${K}_{l}$,
- –
- Loss function: mean squared error $\frac{1}{N}\sum _{j=1}^{N}\left|\right|{A}_{c}^{{K}_{l}}({\kappa}_{j})-{\mathcal{N}}_{M}^{l}({\kappa}_{j};{\theta}_{M}){\left|\right|}_{2}^{2}$,
- –
- Activation function: ReLu function (rectifier),
- –
- DNN structure: 10–16 hidden layers, each layer have 100–500 neurons,
- –
- Training optimizer: Proximal Adagrad.

#### 4.1. Experiment 1

#### 4.2. Experiment 2

## 5. Conclusions

## Author Contributions

## Funding

## Conflicts of Interest

## References

- Efendiev, Y.; Hou, T. Multiscale Finite Element Methods: Theory and Applications; Surveys and Tutorials in the Applied Mathematical Sciences; Springer: New York, NY, USA, 2009; Volume 4. [Google Scholar]
- Hou, T.; Wu, X. A multiscale finite element method for elliptic problems in composite materials and porous media. J. Comput. Phys.
**1997**, 134, 169–189. [Google Scholar] [CrossRef] - Jenny, P.; Lee, S.; Tchelepi, H. Multi-scale finite volume method for elliptic problems in subsurface flow simulation. J. Comput. Phys.
**2003**, 187, 47–67. [Google Scholar] [CrossRef] - Efendiev, Y.; Galvis, J.; Hou, T. Generalized multiscale finite element methods (GMsFEM). J. Comput. Phys.
**2013**, 251, 116–135. [Google Scholar] [CrossRef][Green Version] - Chung, E.; Efendiev, Y.; Hou, T.Y. Adaptive multiscale model reduction with Generalized Multiscale Finite Element Methods. J. Comput. Phys.
**2016**, 320, 69–95. [Google Scholar] [CrossRef][Green Version] - Efendiev, Y.; Leung, W.T.; Cheung, S.W.; Guha, N.; Hoang, V.H.; Mallick, B. Bayesian Multiscale Finite Element Methods. Modeling Missing Subgrid Information Probabilistically. Int. J. Multiscale Comput. Eng.
**2017**. [Google Scholar] [CrossRef] - Cheung, S.W.; Guha, N. Dynamic Data-driven Bayesian GMsFEM. arXiv
**2018**, arXiv:1806.05832. [Google Scholar] [CrossRef] - Krizhevsky, A.; Sutskever, I.; Hinton, G.E. Imagenet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst.
**2012**, 1097–1105. Available online: http://www.informationweek.com/news/201202317 (accessed on 24 March 2019). [CrossRef] - Hinton, G.; Deng, L.; Yu, D.; Dahl, G.E.; Mohamed, A.R.; Jaitly, N.; Senior, A. Approximation Capabilities of Multilayer Feedforward Networks. IEEE Signal Process. Mag.
**2012**, 29, 82–97. [Google Scholar] [CrossRef] - He, K.; Zhang, X.; Ren, S.; Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 770–778. [Google Scholar]
- Cybenko, G. Approximations by superpositions of sigmoidal functions. Math. Control Signals Syst.
**1989**, 2, 303–314. [Google Scholar] [CrossRef] - Hornik, K. Approximation Capabilities of Multilayer Feedforward Networks. Neural Netw.
**1991**, 4, 251–257. [Google Scholar] [CrossRef] - Telgrasky, M. Benefits of depth in neural nets. arXiv
**2016**, arXiv:1602.04485. [Google Scholar] - Liao, H.M.; Poggio, T. Learning functions: When is deep better than shallow. arXiv
**2016**, arXiv:1603.00988v4. [Google Scholar] - Hanin, B. Universal function approximation by deep neural nets with bounded width and relu activations. arXiv
**2017**, arXiv:1708.02691. [Google Scholar] - Zhu, Y.; Zabaras, N. Bayesian deep convolutional encoder–decoder networks for surrogate modeling and uncertainty quantification. J. Comput. Phys.
**2018**, 366, 415–447. [Google Scholar] [CrossRef][Green Version] - Li, Z.; Shi, Z. Deep Residual Learning and PDEs on Manifold. arXiv
**2017**, arXiv:1708.05115. [Google Scholar] - Weinan, E.; Yu, B. The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problems. arXiv
**2017**, arXiv:1710.00211. [Google Scholar] - Khoo, Y.; Lu, J.; Ying, L. Solving parametric PDE problems with artificial neural networks. arXiv
**2017**, arXiv:1707.03351. [Google Scholar] - Cheung, S.W.; Chung, E.T.; Efendiev, Y.; Gildin, E.; Wang, Y. Deep Global Model Reduction Learning. arXiv
**2018**, arXiv:1807.09335. [Google Scholar] - Wang, Y.; Cheung, S.W.; Chung, E.T.; Efendiev, Y.; Wang, M. Deep Multiscale Model Learning. arXiv
**2018**, arXiv:1806.04830. [Google Scholar] - Chung, E.; Efendiev, Y.; Leung, W.T. Generalized multiscale finite element method for wave propagation in heterogeneous media. SIAM Multicale Model. Simul.
**2014**, 12, 1691–1721. [Google Scholar] [CrossRef] - Chung, E.; Efendiev, Y.; Lee, C. Mixed generalized multiscale finite element methods and applications. SIAM Multicale Model. Simul.
**2014**, 13, 338–366. [Google Scholar] [CrossRef] - Efendiev, Y.; Galvis, J.; Li, G.; Presho, M. Generalized Multiscale Finite Element Methods Oversampling strategies. Int. J. Multiscale Comput. Eng.
**2013**, 12, 465–484. [Google Scholar] [CrossRef] - Calo, V.; Efendiev, Y.; Galvis, J.; Li, G. Randomized Oversampling for Generalized Multiscale Finite Element Methods. arXiv
**2014**, arXiv:1409.7114. [Google Scholar] [CrossRef] - Chung, E.T.; Efendiev, Y.; Leung, W.T. Residual-driven online Generalized Multiscale Finite Element Methods. J. Comput. Phys.
**2015**, 302, 176–190. [Google Scholar] [CrossRef] - Chung, E.T.; Efendiev, Y.; Leung, W.T.; Vasilyeva, M.; Wang, Y. Non-local Multi-continua upscaling for flows in heterogeneous fractured media. arXiv
**2018**, arXiv:1708.08379. [Google Scholar] - Glorot, X.; Bordes, A.; Bengio, Y. Deep Sparse Rectifier Neural Networks. In Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics. PMLR, Ft. Lauderdale, FL, USA, 11–13 April 2011; pp. 315–323. [Google Scholar]
- Kingma, D.P.; Ba, J. Adam: A method for stochastic optimization. arXiv
**2014**, arXiv:1412.6980. [Google Scholar] - Chollet, F. Keras. 2015. Available online: https://keras.io (accessed on 1 May 2019).

**Figure 2.**A flow chart in illustrating the idea of using deep learning in the generalized multiscale finite element method (GMsFEM) framework.

**Figure 7.**Exact multiscale basis functions ${\varphi}_{m}^{{\omega}_{1}}$ (

**left**), predicted multiscale basis functions ${\varphi}_{m}^{{\omega}_{1},\mathrm{pred}}$ (

**middle**) and their differences (

**right**) in the coarse neighborhood ${\omega}_{1}$ in experiment 2. The first row and the second row illustrate the first basis function ${\varphi}_{1}^{{\omega}_{1}}$ and the second basis function ${\varphi}_{2}^{{\omega}_{1}}$, respecitvely.

**Figure 8.**Exact multiscale basis functions ${\varphi}_{m}^{{\omega}_{2}}$ (

**left**), predicted multiscale basis functions ${\varphi}_{m}^{{\omega}_{2},\mathrm{pred}}$ (

**middle**) and their differences (

**right**) in the coarse neighborhood ${\omega}_{2}$ in experiment 2. The first row and the second row illustrate the first basis function ${\varphi}_{1}^{{\omega}_{2}}$ and the second basis function ${\varphi}_{2}^{{\omega}_{2}}$, respecitvely.

**Figure 9.**Exact multiscale basis functions ${\varphi}_{m}^{{\omega}_{3}}$ (

**left**), predicted multiscale basis functions ${\varphi}_{m}^{{\omega}_{3},\mathrm{pred}}$ (

**middle**) and their differences (

**right**) in the coarse neighborhood ${\omega}_{3}$ in experiment 2. The first row and the second row illustrate the first basis function ${\varphi}_{1}^{{\omega}_{3}}$ and the second basis function ${\varphi}_{2}^{{\omega}_{3}}$, respecitvely.

**Table 1.**Percentage error of multiscale basis functions ${\varphi}_{1}^{{\omega}_{i}}$ in experiment 1.

Sample | ${\mathit{\omega}}_{1}$ | ${\mathit{\omega}}_{2}$ | ${\mathit{\omega}}_{3}$ | |||
---|---|---|---|---|---|---|

$\mathit{j}$ | ${\mathit{e}}_{{\mathit{L}}^{\mathbf{2}}}$ | ${\mathit{e}}_{{\mathit{H}}^{\mathbf{1}}}$ | ${\mathit{e}}_{{\mathit{L}}^{\mathbf{2}}}$ | ${\mathit{e}}_{{\mathit{H}}^{\mathbf{1}}}$ | ${\mathit{e}}_{{\mathit{L}}^{\mathbf{2}}}$ | ${\mathit{e}}_{{\mathit{H}}^{\mathbf{1}}}$ |

1 | 0.47% | 3.2% | 0.40% | 3.6% | 0.84% | 5.1% |

2 | 0.45% | 4.4% | 0.39% | 3.3% | 1.00% | 6.3% |

3 | 0.34% | 2.3% | 0.40% | 3.1% | 0.88% | 4.3% |

4 | 0.35% | 4.2% | 0.43% | 5.4% | 0.94% | 6.6% |

5 | 0.35% | 3.3% | 0.37% | 3.9% | 0.90% | 6.1% |

6 | 0.51% | 4.7% | 0.92% | 12.0% | 2.60% | 19.0% |

7 | 0.45% | 4.1% | 0.38% | 3.2% | 1.00% | 6.4% |

8 | 0.31% | 3.4% | 0.43% | 5.5% | 1.10% | 7.7% |

9 | 0.25% | 2.2% | 0.46% | 5.6% | 1.10% | 6.2% |

10 | 0.31% | 3.5% | 0.42% | 4.5% | 1.30% | 7.6% |

Mean | 0.38% | 3.5% | 0.46% | 5.0% | 1.17% | 7.5% |

**Table 2.**Percentage error of multiscale basis functions ${\varphi}_{2}^{{\omega}_{i}}$ in experiment 1.

Sample | ${\mathit{\omega}}_{1}$ | ${\mathit{\omega}}_{2}$ | ${\mathit{\omega}}_{3}$ | |||
---|---|---|---|---|---|---|

$\mathit{j}$ | ${\mathit{e}}_{{\mathit{L}}^{\mathbf{2}}}$ | ${\mathit{e}}_{{\mathit{H}}^{\mathbf{1}}}$ | ${\mathit{e}}_{{\mathit{L}}^{\mathbf{2}}}$ | ${\mathit{e}}_{{\mathit{H}}^{\mathbf{1}}}$ | ${\mathit{e}}_{{\mathit{L}}^{\mathbf{2}}}$ | ${\mathit{e}}_{{\mathit{H}}^{\mathbf{1}}}$ |

1 | 0.47% | 4.2% | 0.40% | 1.4% | 0.32% | 1.1% |

2 | 0.57% | 3.2% | 0.31% | 1.4% | 0.30% | 1.1% |

3 | 0.58% | 2.7% | 0.31% | 1.4% | 0.33% | 1.1% |

4 | 0.59% | 3.6% | 0.13% | 1.3% | 0.32% | 1.1% |

5 | 0.53% | 4.0% | 0.51% | 1.6% | 0.27% | 1.0% |

6 | 0.85% | 4.3% | 0.51% | 2.1% | 0.29% | 1.3% |

7 | 0.50% | 2.7% | 0.22% | 1.5% | 0.29% | 1.0% |

8 | 0.43% | 4.5% | 0.61% | 1.9% | 0.35% | 1.1% |

9 | 0.71% | 2.9% | 0.14% | 1.4% | 0.27% | 1.1% |

10 | 0.66% | 4.4% | 0.53% | 1.8% | 0.26% | 1.1% |

Mean | 0.59% | 3.6% | 0.37% | 1.6% | 0.30% | 1.1% |

Sample j | ${\mathit{e}}_{{\mathit{\ell}}^{2}}$ | ${\mathit{e}}_{\mathit{F}}$ |
---|---|---|

1 | 0.67% | 0.84% |

2 | 0.37% | 0.37% |

3 | 0.32% | 0.38% |

4 | 1.32% | 1.29% |

5 | 0.51% | 0.59% |

6 | 4.43% | 4.28% |

7 | 0.34% | 0.38% |

8 | 0.86% | 1.04% |

9 | 1.00% | 0.97% |

10 | 0.90% | 1.08% |

Mean | 0.76% | 0.81% |

Sample j | ${\mathit{e}}_{{\mathit{L}}^{2}}$ | ${\mathit{e}}_{\mathit{a}}$ |
---|---|---|

1 | 0.31% | 4.58% |

2 | 0.30% | 4.60% |

3 | 0.30% | 4.51% |

4 | 0.27% | 4.60% |

5 | 0.29% | 4.56% |

6 | 0.47% | 4.67% |

7 | 0.39% | 4.70% |

8 | 0.30% | 4.63% |

9 | 0.35% | 4.65% |

10 | 0.31% | 4.65% |

Mean | 0.33% | 4.62% |

**Table 5.**Mean percentage error of multiscale basis functions ${\varphi}_{m}^{{\omega}_{i}}$ in experiment 2.

Basis | ${\mathit{\omega}}_{1}$ | ${\mathit{\omega}}_{2}$ | ${\mathit{\omega}}_{3}$ | |||
---|---|---|---|---|---|---|

$\mathit{m}$ | ${\mathit{e}}_{{\mathit{L}}^{\mathbf{2}}}$ | ${\mathit{e}}_{{\mathit{H}}^{\mathbf{1}}}$ | ${\mathit{e}}_{{\mathit{L}}^{\mathbf{2}}}$ | ${\mathit{e}}_{{\mathit{H}}^{\mathbf{1}}}$ | ${\mathit{e}}_{{\mathit{L}}^{\mathbf{2}}}$ | ${\mathit{e}}_{{\mathit{H}}^{\mathbf{1}}}$ |

1 | 0.55 | 0.91 | 0.37 | 3.02 | 0.20 | 0.63 |

2 | 0.80 | 1.48 | 2.17 | 3.55 | 0.27 | 1.51 |

${\mathit{e}}_{{\mathit{\ell}}^{2}}$ | ${\mathit{e}}_{{\mathit{\ell}}^{\mathit{\infty}}}$ | ${\mathit{e}}_{\mathit{F}}$ | |
---|---|---|---|

Mean | 0.75 | 0.72 | 0.80 |

${\mathit{e}}_{{\mathit{L}}^{2}}$ | ${\mathit{e}}_{\mathit{a}}$ | |
---|---|---|

Mean | 0.03 | 0.26 |

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Wang, M.; Cheung, S.W.; Chung, E.T.; Efendiev, Y.; Leung, W.T.; Wang, Y.
Prediction of Discretization of GMsFEM Using Deep Learning. *Mathematics* **2019**, *7*, 412.
https://doi.org/10.3390/math7050412

**AMA Style**

Wang M, Cheung SW, Chung ET, Efendiev Y, Leung WT, Wang Y.
Prediction of Discretization of GMsFEM Using Deep Learning. *Mathematics*. 2019; 7(5):412.
https://doi.org/10.3390/math7050412

**Chicago/Turabian Style**

Wang, Min, Siu Wun Cheung, Eric T. Chung, Yalchin Efendiev, Wing Tat Leung, and Yating Wang.
2019. "Prediction of Discretization of GMsFEM Using Deep Learning" *Mathematics* 7, no. 5: 412.
https://doi.org/10.3390/math7050412