Application of Artificial Neural Networks for Identification of Lithofacies by Processing of Core Drilling Data

Featured Application

This work offers a groundbreaking application in real-time lithofacies identification through core drilling data. By transforming drilling parameters into an image recognition problem using Deep Artificial Neural Networks (DANNs), it overcomes traditional limitations, delivering rapid and accurate binary classification of lithofacies types with enhanced network training and testing performance, making it invaluable in geosciences and drilling industries.

Abstract

Identifying lithofacies types from core drilling data presents significant challenges, especially given the limited number of physical drilling characteristics available for analysis. Traditional machine learning methods often face issues with poor training and testing due to these limitations. Addressing this, we propose a new method for processing core drilling data to improve the accuracy of deep artificial neural networks (DANNs) in lithofacies recognition. Our approach transforms torque, weight on bit (WOB), and rotational speed data into three square matrices, creating a novel three-channel lithofacies image. This method allows for the application of DANNs by converting the complex lithofacies recognition task into a more standard image recognition problem. The developed method dramatically increases the input vector dimensions, enhancing the richness of the data input. The validation of results revealed that the DANN model trained for merely 3000 iterations successfully predicted lithofacies types of all eight testing samples in a mere 2.85 ms, showcasing superior accuracy. The innovative drilling data processing method proposed in this study enables DANNs to identify lithofacies with increased speed and accuracy. This offers a new direction for other DANNs utilizing drilling data.

1. Introduction

Core drilling technology not only enables the collection of rock samples from deep underground, but also enables the acquisition of highly accurate mineral information [1]. Moreover, the rapid and accurate identification of types of underground lithofacies based on core drilling data can greatly improve key drilling performances, such as core recovery rate, and is also of great significance for the exploration, development, and production of underground resources [2,3,4].

The most typical method applied for identifying lithofacies types during core drilling operations includes the use of basic machine learning approaches like support vector machines (SVMs) for predicting the physical properties of rock formations based on data generated during drilling operations, such as torque, rotational speed, drilling pressure or WOB, and rate of penetration (ROP) [5,6]. However, due to the characteristics of the core drilling operations, cutting rocks causes sensor vibrations, leading to the collection of drilling data that include low-frequency or high-frequency noise [7], or abrupt changes in drilling data due to variations in formation strength [8]. Therefore, conventional methods mostly rely on empirical formulas to identify geological formations based on core drilling data. However, the use of these formulas is decidedly inconvenient because the accuracy of empirical formulas depends entirely upon the similarities between the current geological setting and the setting under which the empirical formulas were derived. Alternatively, lithofacies types can be identified based on a manual analysis of the physical core materials produced during the drilling process [9]. However, this method is not always reliable because the results are influenced by geological expertise [10]. Hence, this is an impractical and tedious task for on-site engineers. Accordingly, these conventional methods are fundamentally inconvenient, tiresome, and expensive, which has increasingly led to the development of more advanced machine learning methods to identify lithofacies types from rock core drilling data [3].

Currently, advanced machine learning methodologies have increasingly supplanted traditional analysis approaches. These innovative techniques have been applied in the calculation of geomechanical parameters [11,12,13], optimization and prediction of ROP [7,14,15,16], and the identification of drilling conditions [17,18], aiding geologists in obtaining a more accurate understanding of subsurface operations. At the same time, some progress has also been made in the use of advanced machine learning for lithofacies identification. For example, Zhang et al. [19] demonstrated how artificial neural networks (ANNs) can be employed to identify lithofacies in subsurface studies using conventional wireline logs, providing a more consistent and objective approach than traditional methods. Chang et al. [20] subsequently identified lithofacies from core drilling data using the Kohonen self-organizing map (SoM) algorithm, which was demonstrated to provide higher prediction accuracy compared to the use of backpropagation neural networks (BPNNs) and standard SoM neural networks. Zhou et al. [21] conducted geostratigraphic unit simulations based on a recursive neural network (RNN) to establish a stratigraphic sequence algorithm for predicting stratigraphic sequences from stratigraphic sequence models and stratigraphic thickness models as input. The work of de Lima et al. [22] conducted lithofacies identification based on core image data processed using a deep convolutional neural network (CNN) trained with millions of core images. Chaki et al. [23], using a multiclass SVM framework based on well logs as predictor variables, demonstrating the superiority of multiclass SVMs over other conventional SVMs in terms of classification accuracy, improved the prediction accuracy to 84.5%. Tewari [3] proposed a heterogeneous ensemble method (HEM) approach for lithofacies identification, which has the capability to handle complex, nonlinear, multidimensional, and imbalanced drilling data, achieved an accuracy of 88.32% on the testing set.

However, even advanced machine learning methods struggle with the difficulty of detecting significant changes in core drilling data features due to the limited dimensionality of input vectors, which usually consist of no more than 10 physical core drilling characteristics, such as the torque, drilling pressure, rotational speed, WOB, and ROP parameters discussed above, and hinder the neural network from extracting important features from the data, resulting in poor training and testing performance. Specifically, this issue causes prediction models to fall easily into local optima, and the stochastic gradient descent algorithm employed in the optimization process may even suffer from gradient disappearance, making it unable to minimize the objective function. These conditions produce high levels of both bias and variance in the trained model, which affect its accuracy and reliability. Here, a relatively high level of bias may lead to the underfitting of the model, which produces large training and testing errors, while a relatively high level of variance may lead to overfitting, which produces very small training errors and large testing errors. Hence, balancing these two features is beneficial for increasing the overall learning effectiveness of the model. Typically, deeper neural networks and more training iterations are used to reduce bias, while increasing the volume of training data and data regularization are applied to reduce variance. In addition, modifying the structure of the neural network can, at times, reduce bias and variance simultaneously.

Other studies have demonstrated that the design of the optimization algorithm employed during training can have a pronounced effect on the levels of bias and variance. For example, Gan et al. [16] proposed an improved iterative bat algorithm, which is a heuristic optimization algorithm inspired by the natural behavior of bats, that applies iterative local search and random inertial weights to enhance the global exploration ability and optimization stability of the algorithm. This algorithm was updated by Gan et al. [7] in subsequent research, and the improved algorithm was demonstrated to provide enhanced global optimization performance by solving the problems of local optima and unstable optimization results. Moreover, the algorithm was applied in actual core drilling operations, and was demonstrated to increase the drilling rate by as much as 34.84%. In addition, other machine learning algorithms such as BPNNs have poor fitting capabilities when dealing with unbounded and nonlinear problems.

The above-discussed issues associated with lithofacies identification based on advanced machine learning algorithms can be potentially addressed via the use of deep convolutional neural networks (DANNs), which have been applied in fields such as speech recognition, visual object recognition, object detection, drug discovery, and genomics [24]. Here, applying an optimal number of hidden layers and neuron units is a crucial consideration in neural network design. A deeper neural network can transform each input feature into a higher and more abstract representation through simple and nonlinear calculations, making the machine learning model more sensitive to subtle changes in data between different samples. This allows for the learning of more complex nonlinear functions and representations of parameter change relationships, without over-emphasizing either bias or variance in the model, particularly when the number of input samples is limited. Hence, a DCNN offers excellent potential for extracting important features from low-dimensional core drilling data. For example, He et al. [13] applied a DCNN to predict the strength parameters of rock cores based on data generated during the core drilling process. The approach demonstrated excellent performance in estimating the unconfined compressive strength of different types of rocks. Moreover, the addition of convolution kernels in the algorithm facilitated the rapid and reliable prediction of the strength parameters of on-site rocks. However, the problem of extracting sample features and fitting sample functions remains to be solved.

The present study addresses the above-discussed issues by proposing a novel preprocessing method that transforms complex and abstract drilling data into intuitive image data, thereby converting the intricate lithofacies recognition problem into a classic image recognition problem under supervised learning. First, time-sequence core drilling data comprising torque, WOB, and rotational speed features are normalized and transformed into square matrices, and the data within the square matrices are then converted into the pixel values of what serves as a three-channel lithofacies image. Second, the lithofacies image data are flattened into individual column vectors for input into a DANN, which increases the input vector dimensions from the small number of dimensions typically employed to tens of thousands of dimensions. Our proposed approach was applied for detecting two lithofacies in a binary classification process, and the results demonstrated that the proposed approach offered more than a 92% accuracy rate. This implies that the preprocessed drilling data can be more effectively used for feature extraction. Accordingly, the proposed method provides a new approach to identifying lithofacies from core drilling data.

2. Methods

The flowchart depicting the drilling data preprocessing method proposed in this study is shown in Figure 1. It consists of four steps: collecting drilling data; matrixing the drilling data and converting it into lithofacies images; transforming the data into single column vectors; and using the high-dimensional column vectors as the input layer to train the DANN.

An example of three-dimensional input data comprising time sequences of m core drilling data points, including torque, WOB, and rotational speed, is presented in Figure 2. Just as objects in images can be accurately recognized based on image parameters [25], types of lithofacies can be recognized based on core drilling parameters. For example, just as a color image contains red, green, and blue data channels, a sample of drilling data can be formulated according to torque, WOB, and rotational speed data channels. As shown in Figure 3, these three data channels can be transformed into three square matrices based on 70 × 70 = 4900 data points, and then combined into a single three-channel data structure analogous to a lithofacies image.

As is typical for image recognition applications, the present work flattens the lithofacies image data into a single column vector of length 14,700, as shown in Figure 4, which is then applied as the input of a DANN composed of N hidden layers. As a result, the original three-dimensional time sequence data have been transformed into 14,700 dimensions, which can effectively address the difficulty of extracting important features from the data due to insufficient parameter dimensions, and thereby greatly enhancing the fitting capacity of the machine learning model.

Forward propagation: The input matrix $X$ of the neural network is defined as follows:

$$X=\left[\begin{array}{cccc}⋮& ⋮& \cdots & ⋮\\ x_1& x_2& \ddots & x_n\\ ⋮& ⋮& \cdots & ⋮\end{array}\right]$$

(1)

here, X is formed of n flattened 14,700-element feature vectors composed of torque, WOB, and rotational speed parameters. Specifically, the input dimension of the neural network applied in the present work is a (14,700, 32) matrix composed of 32 training set samples obtained by labeling drilling data samples according to the different types of lithofacies represented. This yields the following DANN layer structure and the predicted value Y[l] output by the l-th layer:

$$Y^{\left[l\right]}=g^{\left[l\right]}\left(Z^{\left[l\right]}\right)\left\{\begin{array}{l}{g}^{\left[l\right]}=\mathrm{R}\mathrm{e}\mathrm{L}\mathrm{U}\left(\dots \right),\mathrm{i}\mathrm{f}\left(l\ne L\right)\\ g^{\left[l\right]}=\mathrm{S}\mathrm{i}\mathrm{g}\mathrm{m}\mathrm{o}\mathrm{i}\mathrm{d}\left(\dots \right), \mathrm{i}\mathrm{f}\left(l=L\right)\end{array}\right.$$

(2)

Here, L is the total number of layers in the neural network, which must be optimized based on the final bias and variance values, as well as the required training time, $Z^{\left[l\right]}$ is the output of the linear regression unit in the l-th layer, where $Z^{\left[1\right]}=X$, and $g^{\left[l\right]}$ is an activation function. As indicated in (2), the neurons in all hidden layers except the last layer are activated using the rectified linear unit (ReLU) function, which has a relatively high computation speed and effectively mitigates the vanishing gradient problem. Finally, the sigmoid function is applied to activate the neurons in the last layer, which is highly suitable for binary classification problems because it outputs values in the range of (0,1). In addition, the neurons in the input layer are fully connected to the neurons in the hidden layer.

Loss function: The present work applies the cross-entropy loss function, which measures the difference between the probability distribution of the predicted values Y[l] and the probability distribution of the true values T, and is commonly used in classification problems as a method to minimize information entropy. This is defined as follows:

$$J(Y^{\left[l\right]},T)=-\left[T\cdot \log \left(Y^{\left[l\right]}\right)+\left(1-T\right)\cdot \log \left(1-Y^{\left[l\right]}\right)\right]$$

(3)

Here, the true values T ∈ {0,1} represent a binary classification task, where the value of the target lithofacies type is 1 and the value of the non-target lithofacies type is 0.

Parameter update: The training process iteratively yields the network parameters $W^{\left[l\right]}$ and $B^{\left[l\right]}$ of the l-th layer according to the following parameter update processes based on the gradients of $W^{\left[l\right]}$ and $B^{\left[l\right]}$, and the learning rate $\alpha$:

$$W^{\left[l\right]}=W^{\left[l\right]}-\alpha \frac{\partial J(Y^{\left[l\right]},T)}{\partial W^{\left[l\right]}}$$

(4)

$$B^{\left[l\right]}=B^{\left[l\right]}-\alpha \frac{\partial J(Y^{\left[l\right]},T)}{\partial B^{\left[l\right]}}$$

(5)

Here, $\alpha$ is a hyperparameter that can be adjusted to improve the machine learning step size, where the network parameters may converge to a local minimum if α is too small, while the gradients may oscillate around the optimum point if α is too large. The ideal value of α ensures that the gradients of $W^{\left[l\right]}$ and $B^{\left[l\right]}$ decrease steadily. The process of updating parameters $W^{\left[l\right]}$ and $B^{\left[l\right]}$ represents a continuous learning of the features of the drilling data, and eventually establishes a fully trained model capable of accurately predicting the lithofacies type of an unknown data sample.

Sample prediction: As a binary classification problem, the predicted geological label $P_n$ of sample n is defined as follows:

$$P_n=\left\{\begin{array}{c}\begin{array}{cc}1,& \mathrm{i}\mathrm{f} Y_n^{\left[L\right]}\ge 0.5\end{array}\\ \begin{array}{cc}0,& \mathrm{i}\mathrm{f} Y_n^{\left[L\right]}<0.5\end{array}\end{array}\right.$$

(6)

Here, the final predicted value is marked as 1 (i.e., as the target lithofacies type) for any value greater than or equal to 0.5, and the final predicted value is marked as 0 (i.e., as the non-target lithofacies type) for any value less than 0.5. The prediction accuracy of the trained model is then defined according to the true lithofacies type label T_n ∈ {0,1} of the training sample data as follows:

$$\mathrm{A}\mathrm{c}\mathrm{c}\mathrm{u}\mathrm{r}\mathrm{a}\mathrm{c}\mathrm{y}\mathrm{r}\mathrm{a}\mathrm{t}\mathrm{e}=\left(\left.1-\frac{1}{n}\sum _{i=1}^n\left|T_n-P_n\right|\right)\times 100\%\right.$$

(7)

Here, an accuracy rate of 100% represents a perfectly accurate prediction and 0 indicates an entirely incorrect prediction.

The core drilling data employed in the present work were obtained for drilling operations conducted in granite and concrete materials. Granite is a natural rock composed of various minerals such as quartz, feldspar, and mica. It has a high strength and rigidity, with a compressive strength typically between 100–300 MPa and an elastic modulus usually between 50–100 GPa. In contrast, concrete is usually a mixture of cement with other materials such as aggregates and sand. Its strength and elastic modulus are generally less than those of granite, with a compressive strength usually between 20–60 MPa and an elastic modulus usually between 20–40 GPa. The significant differences in the mechanical properties of these materials will ensure that the corresponding drilling data are more easily distinguished by the DANN, which is helpful in a proof-of-concept study like the one presented herein.

The dataset was sourced from data collected on a small-scale indoor core drilling machine equipped with a full range of sensors [26]. The data were recorded at a frequency of 100 Hz, with an interval of 10 ms between each group of collected sensor data. The rotational speeds were set at 400 rpm and 600 rpm to collect different torque and WOB data. Core drilling was conducted for a total of 8 granite samples and 6 concrete samples, which generated a total of 40 sets of drilling data samples comprising 18 sets of granite drilling data samples and 22 sets of concrete drilling data samples. As discussed, each sample set included 4900 data points for torque, WOB, and rotational speed. Accordingly, the experiments included 40 × 4900 × 3 = 588,000 data entries. Among these, 16 sets of granite drilling data samples and 16 sets of concrete drilling data samples were applied for a total of 32 sets of drilling data samples as the training dataset, and 4 sets of granite drilling data samples and 4 sets of concrete drilling data samples were applied for a total of 8 sets of drilling data samples as the testing dataset. Equal numbers of drilling data samples were extracted under the friction and cutting states in all datasets. The classification experiments were conducted on a personal computer with a Core i7-9700K CPU, a NVIDIA GeForce RTX 3060Ti GPU, and 32 GB of RAM.

3. Results and Discussion

Representative distributions of normalized drilling data samples composed of 4900 data points obtained for concrete, concrete, granite, and granite are presented in Figure 5(a1,b1,c1,d1), respectively. The corresponding lithofacies image-type data structures composed of 70 × 70 = 4900 pixels are, respectively, presented in Figure 5(a2,b2,c2,d2). Details regarding the features of the data samples presented in Figure 5 are listed in

Table 1. Features of data samples presented in Figure 5.

Sample Number	Lithofacies Classification	State	Scatter Plot	Lithofacies Image
5	Concrete	Cutting	Figure 5(a1)	Figure 5(a2)
13	Concrete	Friction	Figure 5(b1)	Figure 5(b2)
23	Granite	Cutting	Figure 5(c1)	Figure 5(c2)
31	Granite	Friction	Figure 5(d1)	Figure 5(d2)

. As can be seen from Figure 5, both the normalized drilling data and the lithofacies images of the concrete and granite samples exhibit strong visual similarities that are difficult for human viewers to distinguish. In contrast, DANNs have demonstrated excellent performance at detecting subtle differences between the features of the samples and distinguishing them accordingly.

The loss function values obtained according to Equation (3) are plotted with respect to the number of training iterations in Figure 6 for different learning rates with a 14,700-8-3-1 network structure composed of four layers, including an input layer with 14,700 nodes, two hidden layers with eight and three nodes, respectively, and one node in the output layer. As can be seen, the learning rate has a profound effect on the rate at which the loss function value decreases to zero, which is indicative of the neural network having learned the significant sample features reflecting the lithofacies types of the materials. Details regarding the loss values obtained under different learning rates are listed in

Table 2. Loss values obtained under different learning rates.

Learning Rate	Loss at 20,000 Iterations	Iterations When Loss = 0.05	Training Time (s)
0.0100	0.312837	—	36.260358
0.0060	0.002251	2718	37.891610
0.0040	0.000908	2331	36.654807
0.0020	0.002735	4792	36.648935
0.0010	0.009985	8653	42.522422

. Considering the key factors of time and stability, a learning rate of $\alpha$ = 0.0020 and 20,000 iterations were selected for the model training process in all subsequent experiments.

The effect of network structure on the loss function value is presented in Figure 7, where the abbreviations uniformly follow the 14,700-8-3-1 example discussed above. Details regarding the loss values obtained under different network structures are listed in

Table 3. Loss values obtained under different neural network structures.

Neural Network Structure	Loss at 20,000 Iterations	Iterations When Loss = 0.05	Training Time (s)
14,700-3-1	0.051492	—	27.469723
14,700-8-3-1	0.002734	4792	41.872812
14,700-8-5-3-1	0.031424	13,354	44.347052
14,700-10-10-8-5-1	0.000713	3383	60.278994
14,700-20-10-8-3-1	0.000726	3332	87.465936

. The results demonstrate that the 14,700-8-3-1 network structure provides a relatively ideal learning condition because the training time cost is approximately half that of the most complex model with very nearly the same effectiveness, as indicated by the final loss value obtained after 20,000 iterations. Accordingly, this network structure is capable of effectively representing and detecting the key input data features and exhibits high stability and convergence capability. Therefore, the 14,700-8-3-1 network structure was applied in all subsequent experiments.

For illustrative purposes, we applied both the training and testing datasets in the training process, and the accuracy rates obtained at each iteration according to Equation (7) are presented in Figure 8. As can be seen, the prediction accuracy obtained for both the training and testing datasets steadily increases as the value of the loss function continues to decrease according to the results discussed above, and the model accurately identifies the lithofacies types in both the training and testing datasets after around 3000 iterations. Details regarding the sample types and prediction accuracy obtained for the eight samples in the testing dataset are listed in

Table 4. Prediction accuracy rates obtained for the testing dataset.

Sample Number	Predictive Value	Sample Label	Predicted Label	Accuracy Rate	Identification Time (s)
1	0.000008	0	0	100%	0.00285
2	0.000000	0	0
3	0.000014	0	0
4	0.000001	0	0
5	0.000000	0	0
6	0.000000	0	0
7	0.996510	1	1
8	0.972238	1	1

. To mitigate the risk of overfitting and not properly testing, we applied L2 regularization and incorporated dropout procedures for both the input and hidden layers during the training process. Furthermore, when tested on different segments of the same drilling data, we consistently achieved an accuracy rate of over 92% for this binary classification problem, compared to a prediction accuracy rate of 31.86% for SVM methods [27], 82.43% for multiclass SVM methods [23], 84.5% in multi-agent collaborative learning architecture methods [28], and 88.32% in HEMs [3]. We note from the table that the trained model required only 2.85 ms to classify all eight samples in the testing dataset, which is negligible compared to the 49 s required to collect the 4900 data points for a single sample at the applied sampling frequency. This enables the near real-time analysis of underground lithofacies during the drilling process after data collection.

4. Conclusions

This study proposed a novel core drilling data processing method to address the challenges faced by even advanced machine learning methods in distinguishing core drilling features due to limited physical characteristics. By assembling drilling data to create a lithofacies image, we flattened the lithofacies image data into column vectors for DANN input, expanding the input vector dimensions from a few to tens of thousands. This approach transforms the complex lithofacies recognition task into an image recognition task using a DANN. With a 14,700-8-3-1 neural network structure setup and a 0.0020 learning rate, optimal accuracy is achieved in about 3000 iterations, swiftly recognizing eight drilling data points in just 2.85 ms.

Future research will focus on the classification of multiple lithofacies target types. Therefore, other neural network algorithms such as CNNs will be considered to reduce the learning cost and improve learning efficiency when facing the possibility of larger models.