Finite Element-Based Machine Learning Model for Predicting the Mechanical Properties of Composite Hydrogels

Abstract

In this study, a finite element (FE)-based machine learning model was developed to predict the mechanical properties of bioglass (BG)-collagen (COL) composite hydrogels. Based on the experimental observation of BG-COL composite hydrogels with scanning electron microscope, 2000 microstructural images with randomly distributed BG particles were created. The BG particles have diameters ranging from 0.5 µm to 1.5 µm and a volume fraction from 17% to 59%. FE simulations of tensile testing were performed for calculating the Young’s modulus and Poisson’s ratio of 2000 microstructures. The microstructural images and the calculated Young’s modulus and Poisson’s ratio by FE simulation were used for training and testing a convolutional neural network regression model. Results showed that the network developed in this work can effectively predict the mechanical properties of the composite hydrogels. The R-squared values were 95% and 83% for Young’s modulus and Poisson’s ratio, respectively. This work provides a surrogate model of finite element analysis to predict mechanical properties of BG-COL hydrogel using microstructure images, which could be further utilized for characterizing heterogeneous materials in big data-driven material designs.

1. Introduction

In tissue engineering, biocompatible materials (i.e., biomaterials) are employed to generate hydrogels or other scaffolds for use in the repair or replacement of damaged and diseased tissues. To reduce the risk of scar tissue formation at the interface with the host tissues, biomimetic materials that match the physicochemical properties of native tissue are commonly desired. Collagen (COL) is the most abundant protein in mammals [1], which could be reinforced by bioglass (BG) to formulate composite scaffolds with improved mechanical properties for bone tissue engineering applications [2]. Specifically, Kajave et al. [3] showed that incorporating BG into COL reduced the swelling and improved the stability and rheological properties (i.e., yield stress) of COL hydrogels. Gurumurthy et al. [4] reported, in a review paper, that the stiffness of COL scaffolds significantly increased upon adding BGs. The mechanical properties of BG-COL composite scaffolds have been estimated from either a mathematical model or finite element (FE) method [5]. Homogenization methods, such as the double inclusion method, Mori–Tanaka mean field method, and self-consistent approaches, are generally practical for simple microstructures [6,7,8,9]. The representative volume element (RVE) technique in the FE method has been well utilized to estimate the effective properties of composite materials [10,11,12]. Prior work has shown that the mechanical properties of BG-COL scaffolds depend on the concentration, spatial distribution, and particle size of the BG, as well as the fabrication process [13].

Machine learning methods, especially deep learning methods, have been well utilized in solving engineering problems, e.g., predicting atomic and molecular properties, crystal structures and stability, stent expansion, and retinal mechanics [14,15,16,17]. Specifically, deep neural network (DNN) approaches have proven efficient in uncovering unique structures. With sufficient training data, DNN uses the high-dimensional feature vector from the original data and learns the feature vector’s nonlinear relationship with the expected output [18,19]. Ye et al. [20,21] demonstrated that DNN could efficiently provide an accurate mapping between the effective mechanical properties (Young’s modulus and Poisson’s ratio) and the microstructures of composites. A supervised ML was presented by Ford et al. [22] to predict the homogenized elastic properties of two-phase materials. Yang et al. [23] predicted the stress–strain curve of binary composites by using a combination of principal component analysis and convolutional neural networks (CNN). Hamel et al. [24] presented a FE-based evolutionary algorithm to design active composite structures for 4D printing that can achieve target shape shifting responses. Ponnusami et al. [25] introduced a deep learning approach for predicting unidirectional fiber-reinforced composites’ transverse elastic and plastic properties. Wei et al. [26] utilized different machine learning methods, including CNN, support vector regression (SVR), and Gaussian process regression (GPR), to predict the effective thermal conductivity of composite materials. Rong et al. [27] employed 2D CNN to predict the effective thermal conductivity of 3D composites using 2D cross-section images. Kim et al. [28] predicted the transverse mechanical behavior of composites in terms of the stress–strain curves by implementing CNN. They considered the volume fraction (Vf) percentage of the particles as 40%, 50%, and 60%. However, the ML models have not been applied for predicting the mechanical properties of composite hydrogels. The current study is the first attempt to adopt an FE-based ML approach for predicting the mechanical properties of BG-COL. The integration of FE and ML methods, a.k.a. FE-based ML approach, is still in its early stage. The FE input (imaging) and outputs (any filed variables including Young’s modulus and Poisson’s ratio) are used to train the ML models and predict the FE outputs in test cases. This study can serve as a surrogate model for predicting Young’s modulus and Poisson’s ratio of composite hydrogels.

In this work, a CNN regression method was used to predict Young’s modulus and Poisson’s ratio of BG-COL composites. First, 2000 images of BG-COL microstructures were generated. Then, the mechanical properties of the BG-COL composite were calculated using the FE simulation software. These FE obtained data were used to train a CNN regression model for predicting the mechanical properties of BG-COL based on its microstructural image. We demonstrated that our CNN regression model could predict the mechanical properties of BG-COL and hence can aid in overcoming the challenges of predicting these properties with traditional homogenization methods. This work could guide the design of BG-COL and other composite hydrogels.

2. Materials and Methods

The overall workflow of our framework that integrates FE analysis with the CNN model to predict composite material properties is illustrated in Figure 1. Based on observing the microstructural images of BG-COL composite, we first generated 2000 virtual microstructural images of BG-COL with various structures consisting of circular shapes of BG. The microstructural images of BG-COL were then imported into the simulation software, Abaqus/Explicit software version 2019 (Dassault Systems Simulia Corporation, Providence, RI, USA). Then, their effective mechanical properties were extracted by using python script. The data acted as the foundation for developing our DNN prediction approach. Then, by training a CNN regression network, the mechanical properties of the composite were predicted.

2.1. Simulation-Based Datasets

Bioglass particles were circular based on the experimental results and scanning electron microscope (SEM) images, as shown in Figure 1 (depicted by red circles). Periodic two-phase microstructures were created using a synthetic two-dimensional (2D) microstructure generation algorithm we developed using Python (Python Software Foundation, DE, USA). This algorithm can generate non-overlapping circles of varying sizes and Vf (

Table 1. Microstructural images of BG-COL parameters.

Two-Phase Microstructure	Python Parameters	Material Parameters
COL	COL size = 20 × 20 µm	E = 3 kPa, $\mathsf{\nu }=0.49$
BGs	BGs radius = 0.5 to 1.5 µm BGs Vf % = 0.17 to 0.59 BGs number = 30 to 120	E = 76.7 GPa, $\mathsf{\nu }=0.261$ [29]

). Additionally, a python script was developed to calculate the BG-COL material properties from preliminary RVE data. Python code developed in-house was used to organize, create, and link sets necessary for attaining deformable periodic boundary surfaces, which can distort and no longer remain plane, and compute homogenized properties, including Young’s modulus and Poisson’s ratio. Then, the periodic RVE homogenization method concepts were automatically assigned. First, the python code determines the boundary surfaces and RVE dimensions; then, by building nodal sets and applying the boundary conditions and the displacement needed for each property, FE analysis was performed. The interface between the BG and COL was considered to be perfectly bonded for simplicity.

The Young’s modulus (E) and Poisson’s ratio (ν) of the BG were adopted as E = 76.7 GPa and ν = 0.261, respectively [29]. The elastic properties of the COL were tested using our atomic force microscopy (AFM) as E = 3 kPa and ν = 0.49. Two-dimensional generalized plane stress elements (CPS4R) were used for the BG and COL meshing. A uniform displacement of 4 µm, i.e., an average strain of 20% [30], was applied at the boundary surface of the RVE.

To calculate the Young’s modulus and Poisson’s ratio of the BG-COL hydrogel, the reaction forces were computed as the sum of boundary nodal forces along the loading direction. The average stress was calculated as the reaction force divided by the boundary surface area. Then, Young’s modulus of BG-COL was calculated by dividing the stress value by the applied axial strain of 20%. The transverse strain is simply a ratio of the change in height to the original height, and Poisson’s ratio is estimated as the ratio of the transverse strain to the applied axial strain of 20%. The FE modeling framework was validated by the 3D RVEs in [12,31].

2.2. Machine Learning Approach

A CNN regression model was trained to estimate the effective mechanical properties of the BG-COL composite from microstructural images. The TensorFlow and Keras library were used to create the CNN model architecture [32]. CNN is a class of DNN that applies a series of computationally nonlinear layers to analyze visual imagery. CNN can gradually extract representations of images with higher-level generalizations by operating on two functions that produce a third function, which expresses how the shape of one is modified by the other. The current CNN model was selected after trying many architectures and tuning hyperparameters. The tuned graph of the CNN regression used is depicted in Figure 2. It contains convolutional layers (first five layers) merged by fully connected layers (last four layers). The convolution features (or kernels) are memorized hierarchically and comprised of low-level features to build more complex patterns. The input to the CNN is the 2D BG-COL composite gray images of 200 × 200 pixels. A learnable kernel was applied to the input images to extract the convolved feature. This convolved feature is computed through a rectified linear unit (ReLU) [33] function and passed on to the following layers. The respective mechanical properties (Young’s modulus and Poisson’s ratio) are emanated at the outputs layer. The first five layers of architecture involve 3 × 3 convolutional layers using a stride of 2 in both the x and y directions; 2 × 2 maximum pooling layers; and 512, 256, 128, 64, and 32 feature maps, followed by four fully connected layers with 1000, 100, 50, and 10 units. To enhance the precision of the CNN’s prediction, 20% dropout [34] was used in four fully connected layers to prevent overfitting. The Adam optimizer was used to speed up the convergence of the network [35]. Linear activation function and sigmoid in the output layer of the CNN were chosen for Young’s modulus and Poisson’s ratio, respectively, by using functional API. A loss function was used to estimate the differences between CNN’s prediction results and the real mechanical properties of the images. The loss function is minimized via backpropagation by optimizing the CNN parameters, such as biases and kernels, in the convolutional layers and the weights in the fully connected layers, as depicted in Figure 2. The loss function is defined using the mean square error (MSE) since the prediction is a regression problem. MSE is denoted as:

$$MSE=L\left[Y,f\left(X\right)\right]=\frac{1}{n}{{\displaystyle \sum }}_{i=1}^n{\left[Y-f\left(X\right)\right]}^2$$

(1)

where X denotes the input image describing the microstructure of the BG-COL, n represents the batch size, and $f\left(X\right)$ indicates the prediction of the CNN. Y denotes the mechanical properties of BG-COL, which includes both Young’s modulus and Poisson’s ratio. Performing verification to prevent overfitting during every training epoch was also done to select the best model. The CNN model was run on a workstation with the following specifications: Ryzen 9 5950X processor, 128 GB DDR4/2666 MHz memory, and Nvidia GeForce RTX 3090 GPU.

3. Results

The training and testing datasets contain 1600 and 400 images, respectively, and the corresponding mechanical properties (using an 80–20 split). We randomly chose 20% of the dataset as a validation set to assess performance when selecting model architecture and hyperparameters. By 20 repeated epochs, we trained our network, as demonstrated in Figure 3. The training and validation loss converge to a similar value, indicating insignificant overfitting.

The Young’s modulus and Poisson’s ratio of the test set predicted by the CNN plotted against those obtained by the FE simulation are shown in Figure 4. It is clear that our network can efficiently learn and map the microstructure images to the mechanical properties. Additionally, these results indicate that our model works well for testing images. The ranges of Vf were illustrated in different colors to estimate the influence of BG on the mechanical properties. Results demonstrated that Young’s modulus will increase with a larger Vf of BG. In contrast, Poisson’s ratio decreased with a larger Vf. The MSE for Young’s modulus is 0.135, and Poisson’s ratio is 0.000162. The R-squared for Young’s modulus is 0.95, and Poisson’s ratio is 0.83, which shows how well our model fits. Our prediction is similar to the accuracy by Ye et al. [20].

From all 400 testing cases, two representative cases were further delineated regarding the prediction error in Young’s modulus and Poisson’s ratio. The case in Figure 5a with a Vf of 27.73% illustrated that the prediction error of Young’s modulus and Poisson’s ratio is 0.32% and 3.61%, respectively. Moreover, the case in Figure 5b with a larger Vf of 47.85% showed that the prediction error of Young’s modulus and Poisson’s ratio is 1% and 1.7%, respectively. Additionally, the von Mises stress (kPa) distribution in the BG-COL composite demonstrated that the stress concentrations are generally around BG, with the maximum stress magnitude as 3.952 kPa for the case with the Vf of 27.73%, and increased to 11.28 kPa for the case with a larger Vf of 47.85%.

4. Discussion

A FE-based CNN regression network method was used for predicting the effective mechanical properties of BG-COL composite hydrogel. From the microstructure images, we can efficiently and accurately predict the mechanical properties of BG-COL, by considering various BG numbers, sizes, and Vf. In total, 2000 2D RVE microstructural images were generated with various circular shapes of BG and Vf. This RVE of BG-COL was then imported into the FE simulation software to calculate its effective mechanical properties. Subsequently, a python script in Abaqus was developed to extract the effective mechanical properties, including Young’s modulus and Poisson’s ratio. Finally, a CNN regression network was trained and tuned to predict the mechanical properties of the composite with 95% and 83% accuracy for Young’s modulus and Poisson’s ratio, respectively.

The FE input (imaging) and outputs (Young’s modulus and Poisson’s ratio) are used to train the ML models and then predict the FE outputs in 400 test cases. A sufficiently large number of BG-COL images were generated to represent various composite configurations based on our experimental observations. Specifically, we effectively developed a framework to generate geometries, solve the FE computations, and then create a substantial database of input–output pairs. Similarly, Ford et al. [22] used three common two-phase materials (UD-CRTS [36], UHP mortar [37,38], and MPR paste [39]), including fibers of circular shape with the same diameter, but they used different fiber Vf, median sizes, and particles in a robust cementitious matrix and a metallic particulate-reinforced (MPR) cement mortar. They used artificial neural network (ANN) and forest ensemble MLs methods to predict Young’s modulus and Poisson’s ratio of materials by using geometrical features of particles and Vf inputs. CNN regression was chosen in this work considering that our dataset is 200 × 200 gray images with corresponding mechanical properties.

The FE-based CNN method to predict the mechanical properties of the composite hydrogel is still lacking in the literature [22,36,40]. The prediction of BG-COL properties herein was motivated by the work of Ye et al. [20]. They used a CNN regression method to predict the mechanical properties of an artificial composite consisting of two different components (matrix and inclusions). The uniqueness of our dataset is that it is based on the observations of the microstructural images of the BG-COL composites (Figure 1). In addition, we adopted the Vf ranging from 17% to 59% based on the published experiments [3]. The range of our Vf is wider than the one by Kim et al. [28]. Our results have shown that a larger Vf significantly increased Young’s modulus and reduced the Poisson’s ratio. Subjected to the same deformation, a larger Vf is associated with a larger force load and a higher peak stress in the BG-COL (Figure 5). This trend was also observed by Yi et al. [9].

The most critical parameters to achieve the prediction accuracy of 95% and 83% for Young’s modulus and Poisson’s ratio, respectively, were the number of feature maps in the convolutional layers and the number of units in the fully connected layer. A total of nine ML architectures with different layers (number of convolutional layers and feature maps in these layers, and fully connected layers and units in each layer), and different activation functions, including ReLU, sigmoid, and linear, were picked and trained. In the first architecture, our network was created by three convolutional layers with 256, 128, and 32 feature maps in each layer, respectively. The last convolutional layer was then connected to the first fully connected layer as the input in the vector form. Additionally, three fully connected layers with 500, 100, and 10 units were considered. The initial accuracy was 80% for Young’s modulus and 72% for Poisson’s ratio. By adding more layers and feature maps in the convolutional layers and units in the fully connected layers, our network had more trainable parameters and could learn more details and extract more features about images and their following material properties. It must be noted that overfitting can occur in a network with more layers, although more layers can increase the performance to a certain extent. The optimal number of layers and nodes in a given configuration should be decided to avoid falling into the situation of overfitting.

The tuned model in this work consists of learnable kernels applied to the input images for extracting convolved features. These convolved features were computed through a rectified linear unit (ReLU) function and passed on to the following layers. With 512 different kernels (the sizes of 3 × 3, using a stride of 2 in both x and y directions) at the first layer, similar procedures were repeated in layers 2, 3, 4, and 5 with 256, 128, 64, and 32 kernels, respectively. The last convolutional layer was then connected to the first fully connected layer as the input in the vector form. In the fully connected layers, learnable weights were calculated for each layer. We used the dropout technique after every fully connected layer. The last layer was a linear function for Young’s modulus and a sigmoid function for Poisson’s ratio. To minimize the prediction error, the kernels mentioned above were learned and will be activated when a similar feature appears in the input.

This study utilized two-dimensional RVE datasets. However, the framework could be expanded to predict the three-dimensional mechanical properties of BG-COL. Additional properties of the composite hydrogel, such as thermal conductivity, thermal expansion coefficients [27], fatigue life, toughness, and stress–strain curves [23], could be predicted using our framework. Moreover, it must be mentioned that this work did not consider the impact of COL crosslinking, which is typically done for these hydrogels [3]. When the COL and BG are crosslinked, the effect of BG on the mechanical properties of COL can change depending on what COL crosslinking method was used and how much crosslinking was done. For example, COL crosslinking can mask the effect of BG such that the BG incorporation does not impact the mechanical properties of the COL hydrogels.

5. Conclusions

In this work, the FE input (imaging) and outputs (Young’s modulus and Poisson’s ratio) were used to train a CNN regression network for predicting the mechanical properties of BG-COL composite hydrogel, including Young’s modulus and Poisson’s ratio. Randomly distributed spherical BGs with diameters ranging from 0.5 µm to 1.5 µm and volume fractions from 17% to 59% were considered. The mechanical properties of 2000 images of the BG-COL microstructure were calculated in the FE simulation software utilizing an in-house python script. The prediction accuracy was 95% and 83% for Young’s modulus and Poisson’s ratio, respectively. The FE-based ML model is expected to facilitate nondestructive testing and big data-driven design of BG-COL and other composite hydrogels.