Intelligent Generation of Cross Sections Using a Conditional Generative Adversarial Network and Application to Regional 3D Geological Modeling

Abstract

The cross section is the basic data for building 3D geological models. It is inefficient to draw a large number of cross sections to build an accurate model. This paper reports the use of multi-source and heterogeneous geological data, such as geological maps, gravity and aeromagnetic data, by a conditional generative adversarial network (CGAN) and implements an intelligent generation method of cross sections to overcome the problem of inefficient modeling data based on CGAN. Intelligent generation of cross sections and 3D geological modeling are carried out in three different areas in Liaoning Province. The results show that: (a) the accuracy of the proposed method is higher than the GAN and Variational AutoEncoder (VAE) models, achieving 87%, 45% and 68%, respectively; (b) the 3D geological model constructed by the generated cross sections in our study is consistent with manual creation in terms of stratum continuity and thickness. This study suggests that the proposed method is significant for surmounting the difficulty in data processing involved in regional 3D geological modeling.

1. Introduction

Since its convenience as a three-dimensional system [1,2,3], 3D geological modeling has been a hot issue for the prospecting and engineering fields [2,3,4,5,6]. The methods for establishing 3D geological models differ based on the modeling data [4,5,7,8,9], such as GIS-based [6], multi-source data-based [9], borehole-based [8,10], section-based [4,7,11,12] or geophysical data-based [3]. Section maps, integrating the experience of geologists [11,12], are widely used in the areas with sparse borehole data due to their relatively low-cost [13,14,15], playing an important role in the modeling. However, the drawing of cross sections, heavily relies on the experiences of geological expertise [13,16], limiting its quantity in the modeling dataset [17,18], it is necessary to determine the stratum thickness or rock mass morphology through gravity and aeromagnetic inversion [3,19,20]. This process is often inefficient and imposes a heavy workload [21,22].

To improve the efficiency of drawing cross sections, Ming et al. [23] developed GSIS software based on the core method of 3D geological multi-body modeling from netty cross sections with topology to interpolate the cross sections and build a 3D geological model automatically. Although this study can generate interpolated sections, the influence of geological factors is not considered, and the generated sections lack geological constraints. Automatic section generation remains a challenge, despite considerable progress [24].

With the development of artificial intelligence (AI) [25,26], deep learning methods have been demonstrated to be an effective path for inversing underground geological bodies and thereby modeling [27,28,29]. Convolutional neural networks (CNNs), graph neural networks, generative adversarial networks and other models are used in prospecting [30], mapping [31] and modeling [32,33]. Thus far, it has not been possible to realize intelligent generation of cross sections with AI technology.

Here, we report a method of intelligent generation of cross sections based on the CGAN model using geological, gravity and aeromagnetic data. The method can automatically generate cross sections at any position with few manual interventions. The results show that: (a) the accuracy of the proposed method is higher than the GAN and Variational AutoEncoder (VAE) models, achieving 87%, 45% and 68%, respectively. The 3D geological model constructed by the generated cross sections in our study is consistent with manual creation in terms of stratum continuity and thickness.

2. Conditional Generative Adversarial Network

The generative adversarial network (GAN) is a deep learning model that was first proposed by Goodfellow [34]. It is primarily used for unsupervised learning of data characteristics, and it can generate new data after training. The network has received extensive attention since it was proposed, and it has been widely studied and applied in the fields of image and vision. It can generate handwritten instances, natural landscape transformation, facial expression generation, target map switching, and super resolution images. In the fields of voice generation, virus sample generation and other applications, the GAN neural network provides enough simulated sample data to improve the recognition accuracy of the discriminator.

Different from AlexNet [35], VGG [36], GoogLeNet [37], and other single model neural networks, GAN is a neural network model that can generate target data and includes two modules: a generator (G) and a discriminator (D) (see Figure 1). Each module separately constitutes a network. Generator G constantly generates samples that obey the distribution of real data based on random noise, and the discriminator D is used to judge whether the input data are real data. Therefore, the discriminator is essentially a binary classification network. Through continuous iteration and optimization, the final generator G can produce false target data.

Generator G in the GAN network extracts the feature space of the input data by using the convolution operation and then generates the specified size data by using the deconvolution operation based on the feature space. Therefore, the network G is composed of a series of convolution and deconvolution layers. The network model of the generator G is shown in Figure 1.

$$\begin{array}{c}\min \\ G\end{array}\begin{array}{c}\max \\ D\end{array}V(D,G)=E_{x~Pdata(x)}[\log D(x)]+E_{z~Pz(z)}[\log (1-D(G(z)))]$$

(1)

The GAN neural network uses Equation (1) to train generator G and discriminator D [37]. In Equation (1), x represents the real sample, and D(x) represents the probability that x judges it as a real sample through the discrimination network; z represents the noise of the input-generated sample; G(z) represents the sample generated by the noise z of the generated network, and D(G(z)) represents the probability of judging the generated sample as a real sample after passing the discrimination network.

The original GAN neural network generates pseudo data based on random noise, and has the disadvantages of instability, mode collapse, and non-convergence, it is often unable to generate data with specific constraints. Therefore, many researchers have proposed different GAN models with constraint information based on the original GAN neural network, among which conditional generative adversarial network (CGAN) has been the most successful [38]. CGAN uses a condition variable c in the generator G(z, c). When training, x and z both add the condition c to participate in the training.

The objective function of CGAN after adding constraint data c on the basis of the original GAN network can be shown in Equation (2):

$$\begin{array}{c}\min \\ G\end{array}\begin{array}{c}\max \\ D\end{array}V(D,G)=E_{x~Pdata(x)}[\log D(x|c)]+E_{z~Pz(z)}[\log (1-D(G(z|c)))]$$

(2)

The structure of the CGAN model can be shown in Figure 2.

2.1. Convolution Layer

The convolution layer extracts the features of the input image through convolution calculation (Figure 3) and outputs the feature map. The convolution layer consists of a series of fixed size filters (called convolution kernels) that are used to perform convolution operations on image data to generate feature maps [39]. Generally, the calculation of a characteristic diagram can be realized by Equation (3):

$$h_{ij}^k={\displaystyle {\sum }_{i\in M_j}((w^k\times x_{ij})+b_k)}$$

(3)

In Equation (3), k represents the kth layer; h represents the eigenvalue; (i, j) represents the coordinates of pixels in the image; w^k represents the convolution kernel of the current layer, and b_k is the offset. Parameters in convolutional neural networks such as bias (b_k) and the convolution kernel (w^k) are usually trained without supervision [40].

2.2. Leaky-ReLU Activation

After the convolution operation, the Leaky-ReLU activation function is often added to activate neurons by non-linear mapping the characteristic map of the convolution layer output to avoid overfitting and improve the learning ability [41]. This function was originally introduced in the AlexNet model [42]. The Leaky-ReLU activation function (Equation (4)) is used for the output feature mapping of each convolution layer. Compared with the RELU function, Leaky-ReLU keeps negative data in the feature map to activate neurons for the next step of the calculation, and this can improve the robustness of the noise value.

$$f(x)=\{\begin{array}{cc}x& x>0\\ 0.1x& x\le 0\end{array}$$

(4)

2.3. Deconvolution

Deconvolution is a special convolution operation. It first expands the size of the input data by adding 0 according to certain rules, and then generates data with a larger size according to the convolution operation. In fact, the deconvolution operation is implemented by Equation (3) too.

3. Materials and Methods

The methodology of intelligent generation of cross sections is introduced in this section. The architecture of the model, data preparation and data augmentation are described in turn.

3.1. Intelligent Generation of Cross Sections Based on CGAN

In this paper, the intelligent generating algorithm for cross sections based on the CGAN is studied by constructing a training dataset, training model, adjusting the parameters, and other steps. When building the training dataset, the algorithm takes the existing sections as the label data, employs the geological, gravity and aeromagnetic data as the input data to build the dataset for network training. By adjusting the depth and super parameters of the model through experiments, when the generated cross section matches the known section, the model is considered to have converged and can be applied to intelligent generate the cross sections in an unknown area.

Based on the CGAN model, an intelligent generation network model for cross sections was designed (see Figure 4). The model, including two modules, namely, a generator G and a discriminator D, uses Equation (2) to train. The G module is primarily used to generate label samples, while the D module is used to judge whether the generated label samples are real to continuously improve the authenticity of the generated label samples.

In Figure 4, G is trained by continuous input of training data and labels, and can generate false cross sections (S_f). The false cross section and the true cross section (S_t) are input into D to judge whether S_f is true. After the iteration of the specified epochs, the generated cross sections S_f and S_t will converge, and thus G can be used to generate the cross section.

After training, the generator model can be used in the section generation task at the specified location. The geological map, gravity and continuation data, aeromagnetic and continuation data of the known sections are fed into the generator model to generate the modeling sections.

3.2. Data Preparation

The training of the model depends on a large number of training data. To generate sections intelligently, preparation and processing operations are necessary.

For regional 3D geological modeling, the data used for section generation are composed of geological, gravity and aeromagnetic anomaly data for each location point on the section.

3.2.1. Geological Data

Geological data include mineral geological maps and borehole histograms. The stratigraphic units and occurrences in the mineral geological map are used as input data. Stratigraphic units are encoded in one-hot. For example, the stratigraphic sequence developed in a region from top to bottom might be Gaixian (Pt₁gx), Dashiqiao (Pt₁d), Gaojiayu (Pt₁g), Lieryu (Pt₁lr), and Langzishan (Pt₁l). When the exposed stratum on the surface is the Dashiqiao formation, the input stratigraphic unit code is 01000. The occurrence of the stratigraphic unit only considers the dip angle during input, and the dip angle is expressed in radians. Borehole histograms are used for stratum constraints and are encoded in the same way as the stratigraphic unit. If the area has no borehole data, the channel datum is set to 0 for processing.

3.2.2. Geophysical Data

Generally, gravity and aeromagnetic anomaly data are the common geophysical data. Geological surveys and geophysical exploration have been carried out in the area where 3D geological modeling is to be performed. A number of geophysical sections can be collected, or a certain number of geophysical comprehensive interpretation sections can be compiled on the basis of the measured data. These sections should reflect various underground geological conditions in a region. These geophysical data cannot be used in the networks, unless a data gridding operation is performed.

3.2.3. Data Gridding

In order to grid the geophysical data, an interpolation algorithm is needed. In the geology filed, Kriging interpolation is the most used algorithm. According to the sparse degree of the geophysical data and the scale of the study area, a grid size needs to be determined. For example, in the 1:250,000 area, a 300 m × 300 m grid interval can meet the needs of geological research. After interpolation, the geophysical data are transformed into regular data.

3.2.4. Construction of the Input Dataset

We assume that the data vectors of two adjacent sections are a₁ and a₂. The gravity anomaly data corresponding to these two sections are g₁ and g₂. Anomaly data from the aeromagnetic method are denoted by m₁ and m₂. The sections a₁ and a₂ contain multiple section polylines, set as al_1i and al_2j, i 1, 2, …, n, j ∈ 1, 2, …, m. According to the stratigraphic age of the polyline in the section, we take section polylines with the same stratigraphic age P, that is, P_1i==P_2j, to form a section polyline pair (al_1i, al_2j) from top to bottom. When there is a pinch out, the missing section polyline datum is set to 0.

Every section polyline pair (al_1i, al_2j) is interpolated to form 256 points, and then each section polyline is copied 256 times to form a 256 × 256 matrix, recorded as AL_1i and AL_2j.

Then, the same operation is applied to g₁, g₂, m₁ and m₂, and a channel of a 256 × 256 matrix is formed, marked as G₁, G₂, M₁ and M₂.

While generating the modeling section with the input sections of a₁ and a₂, the distance between the generated section and the two input sections will also affect the shape of the section polylines in the modeling. Therefore, this method introduces the distance factor d as a parameter.

$$\{\begin{array}{c}d=\frac{d_1}{d_2}\\ d_1=\sqrt{{(p_{1x}-p_{mx})}^2+{(p_{1y}-p_{my})}^2}\\ d_2=\sqrt{{(p_{1x}-p_{2x})}^2+{(p_{1y}-p_{2y})}^2}\end{array}$$

(5)

The parameter d is calculated as follows (Equation (5)).

The middle points p₁ of section a₁ and p₂ of section a₂ are calculated.

Points p₁ and p₂ are connected, where upon the line p₁p₂ intersects the generated section a_m at p_m.

Length d₁ of p₁p_m and length d₂ of p₁p₂ are calculated.

It is assumed that d = d₁/d₂.

A 256 × 256 matrix D is filled with the value of d as an input channel.

For the interpolated section, once the section line a_m (see Figure 5) is determined, the gravity data g_m corresponding to the section line and aeromagnetic data m_m are known. After the interpolation of gravity and aeromagnetic data formed as a 2 × 256 matrix, the matrix is padded to form a 256 × 256 matrix, marked as G_m and M_m, where m × 1, 2, …, 256. Therefore, G_m, M_m and the previously generated AL_1i, AL_2j, G₁, G₂, M₁ M₂ and D are stacked as input matrices with the shape of 256 × 256 × 9 and recorded as input.

3.2.5. Label Data

In the training stage, a label is required. Different with the classification model, the generation model needs section data as a label. Similar to the preparation process of input data, the section polyline al_mi, corresponding to stratigraphic age in section a_m, is interpolated into 256 points, then padded into a 256 × 256 matrix forming the label data and marked as label.

3.3. Data Augmentation

Since each section contains multiple section polylines, and the data used for training in this method is a single section polyline, polylines of the section map can be extracted as training data (see Figure 6), thus enhancing the data volume and avoiding overfitting.

Figure 6 shows the method of data augment in this study. In each section, the interface of two different geological bodies decides the top or the bottom. So, section polylines are the objects of this method. Every interface in the section is used for the training process, thus augmenting the dataset.

4. Results and Discussion

To verify the generating ability of the proposed method, we conducted experiments in the Benxi–Huanren area in eastern Liaoning Province, China (see Figure 7). This area has been surveyed by certain geological survey projects. The geological, gravity and aeromagnetic data are available.

In this section, the data of training is described firstly, and then the environment of the experiment is listed. Finally, the results are discussed.

4.1. Data and Data Processing

In the study area, we collected 1650 section pairs, including 467 simple stratum sections, 332 complex stratum sections, 420 rock mass sections and 431 fault sections (see

Table 1. The amount of collected data for training.

Type	Section Pairs	Samples
Simple strata	467	4281
Complex strata	332	1672
Rock masses	420	420
Faults	431	431
Total	1650	6804

) in the location of dataset area (see Figure 7). According to the data augment method, the section polylines are extracted from the section pairs. Finally, 6804 samples are prepared. Correspondently, the gravity data, aeromagnetic data, and other data for the 6804 samples are calculated with the instruction of Section 3.2.

In order to evaluate the proposed method, we divided the samples into training, validating and testing datasets with the proportion of 8:2. Here, 20% of the training dataset are used for validation during the training process. Thus, 5443 samples are used for training, 1088 samples of the training dataset are used for validating, and 1361 samples are used for testing.

4.2. Experiments and Results

In order to quantify the difference between the generated section and the real section, this study used the arithmetic mean deviation of section (AMDoS, Equation (6)) as the evaluation standard, where (x_i, y_i) is the coordinates of the ith point on the label section polyline, and (x_i’, y_i’) are the coordinates of the ith point in the generated section polyline.

$$AMDoS={\displaystyle {\sum }_{i=1}^{11}(\sqrt{{(x_i-x_i^{\prime })}^2+{(y_i-y_i^{\prime })}^2}}$$

(6)

Since the number of points in the real section is inconsistent with the number of points in the generated section, when calculating the error value, we calculate the length of the curve, divide the curve into 10 equal parts on average from the starting point of the curve, and calculate the end point of each curve as the error calculation point, including the starting point of the overall curve for 11 points (Equation (6)).

Simultaneously, the coincidence rate of point coordinates (CRoPC, Equation (7)) is calculated, where TP is the point number with the deviation being less than AMDoS, and on the other hand, FP is the deviation more than AMDoS.

$$CR\mathrm{o}PC=\frac{TP}{TP+FP}$$

(7)

If the CRoPC value is more than 0.5, the generated section polyline is considered a positive sample, otherwise a negative sample. So, with this method, we can obtain an accuracy of the proposed model.

The experiments described here run on a computer with the configuration listed in

Table 2. Hardware and software configurations used in the experiment.

Configuration	Value
CPU	Intel Core i5-7300HQ 2.5 GHz
GPU	NVIDIA GeForce GTX 1050Ti with 4GB RAM
Memory	8 GB
Hard disk	1 TB
Operating System	Windows 10
Python Version	3.6.5
Tensorflow Version	Tensorflow-GPU 1.5.0

.

Different super parameters, such as epochs, initial learning rate (ILR), decay rate and batch size, play an important role in the learning effect. Therefore, it is necessary to test the impact of different super parameters on the generator to obtain a relative optimal solution (see Discussion for specific experiments). Finally, with an initial learning rate of 10⁻⁴, two samples in a batch, 10⁻³ decay of learning rate, the model ran 18,000 epochs of training and validating, and achieved a validating accuracy of 92%. The correspondent curves of cross-entropy losses are shown in Figure 8.

Figure 8 shows that at the beginning of training, the loss curve of the discriminator fluctuated, indicating that the training parameters were being adjusted frequently and that this had a significant impact on the generator. When the number of training steps reached 6000, the loss of the discriminator decreased steadily. After a large shock with 10,000 training steps, the loss value of the D dropped to the lowest point after 14,000 steps, and the loss curve of G was relatively stable at this time. Then, the curves of the generator and discriminator began to fluctuate again. Therefore, the trained model after 14,000 steps can be taken for generating modeling sections.

Part of the cross sections generated by the trained CGAN model are shown in Figure 9. It can be seen from the comparison of label section polylines that those generated using the proposed method (Figure 9b,d) are smoother than the label section polylines (Figure 9a,c).

4.3. Influence of Different Super Parameters on the Results

The super parameters affect the performance and accuracy of the proposed model.

Table 3. Performance on different super parameter settings of the proposed method.

Experiments	Epochs	ILR	Batch	Decay	Validation Accuracy
1	18,000	10−3	2	10−3	86%
2	18,000	10−3	2	10−4	87.2%
3	18,000	10−3	2	10−5	85%
4	18,000	10−4	2	10−3	92%
5	18,000	10−4	2	10−4	88%
6	18,000	10−4	2	10−5	89%
7	18,000	10−5	2	10−3	91%
8	18,000	10−5	2	10−4	86%
9	18,000	10−5	2	10−5	85%

compares the accuracy when using different super parameter settings to train the model.

The experimental results show that different super parameters will exhibit different validation accuracies. Due to the limits of the configuration of the executing computer, the batch size is set to 2, and in consideration of the run time, the epoch value is set to 18,000. When the ILR is 10⁻⁴, and decay is 10⁻³, the proposed model achieved an overall accuracy of 92%.

4.4. Comparison with Other Deep Learning Algorithms

In order to test the superiority of the proposed method, the VAE model and the common GAN model are also used for generating cross sections, and the sections generated by the three models are compared (see Figure 10).

AutoEncoder is a generative model [25]. It is suitable for image editing using concept vectors. It maps the input data to a potential vector space through an encoder module, and then decodes it to an output with the same size as the original input through a decoder.

By adding statistical operations to the AutoEncoder network, VAE [44] enables the AutoEncoder network to learn a continuous and highly structured potential space, thus becoming a powerful tool in the field of image generation. The traditional VAE includes two parts: the encoder and decoder. The encoder module is used to collect and train the features of the input data and generate the feature distribution of the training data, and then the decoder module is used to generate the section data to be interpolated based on the feature distribution.

The encoder module is implemented through a convolution operation. After three consecutive convolution operations, a dropout operation is added to discard some data, improve the generalization ability of the model, and finally produce a full connection layer as the output of the encoder module.

The decoder module was created through a four-layer deconvolution operation. Through the deconvolution operation, the potential feature space data extracted by the encoder module is dimensionally restored to finally achieve the purpose of generating the data.

Using the same training and validating dataset, we trained the VAE and GAN models. The statistics of AMDoS, CRoPC and validation accuracy are calculated (see

Table 4. Effect evaluation of different models.

Methods	VAE	GAN	Our Work
Max AMDoS	1568.49	3351.65	1021.61
Min AMDoS	352.84	2015.68	154.23
Max CRoPC	83%	65%	92%
Min CRoPC	37%	32%	44%
Validation accuracy	68%	45%	87%

). A generated section by the three models is compared (see Figure 10).

Table 4 gives the statistical information of the three models’ training processes. The AMDoS value and CRoPC are quite different with the VAE, GAN and our model. The deviations of the GAN model are bigger than those of the other two models. Consequently, the GAN model has the worst performance and 45% validation accuracy. In contrast, our work performs well, both in the deviation value and the validation accuracy of 87%, and the VAE model performs better than the GAN model, but worse than our work.

It can be seen from Figure 10 that the section generated by the GAN model is quite different from the labeled section, seriously deviating from the label. The polylines generated by the VAE and our model are consistent with the label.

5. 3D Geological Modeling and Application

After the training is completed, the network parameters are frozen. Three different application tests (see Figure 7), Yangjiabao, Shuangtaling and Huanren, are carried out in the study area.

The geological, gravity and aeromagnetic data, and few main sections [13] of the test areas are provided. Using the frozen network parameters, cross sections are generated every 50 m from left to right in the test areas, and finally 29 horizontal cross sections in each test area are obtained. Similarly, 24 vertical cross sections in each test area are obtained from top to bottom. The 3D geological models are established on the basis of the generated sections using the modeling method based on cross sections (see Figure 11).

The test results show that, the 3D geological models as illustrated in Figure 11 basically conforms to the geological principles of the study areas, and the model is relatively smooth. Compared with the Yangjiabao area, the geology of the Huanren and Shuangtaling areas are more complex, with structures such fold and fault growing in these two areas. These 3D models also prove that the proposed method can generate section data intelligently, with good stability and practicality.

6. Conclusions

In this study, we considered the key problem of inefficiency of drawing cross sections, and an intelligent method for generating cross sections based on the CGAN model using multi-source and heterogeneous geological data, such as geological, gravity and aeromagnetic anomaly data. After the proposed model was trained, this method achieved an overall accuracy of 87%. Cross sections at the specified location can be generated using the trained model. We tested the method via the establishment of three different 3D geological models in the areas of Yangjiabao, Shuangtaling and Huanren, Liaoning Province. The experimental results show the proposed method can improve the drawing efficiency of the cross sections significantly.

At present, the problem remains that the generated cross sections may not conform to the actual situation when the trained model is used to build a complex section, which has multiple structures and intrusions. To solve this problem, we will try improving the method from two aspects in the following research. First, we can increase the size of the training dataset. The more training data there are, the better the model can update the corresponding model parameters so when applied to new areas, it can generate relatively more accurate sections. Second, the super parameters in the model (such as batch size and the gradient descent algorithm) can be tested to find the optimal network parameters in order to train a more accurate network model.