Artificial Neural Networks (ANNs) for Vapour-Liquid-Liquid Equilibrium (VLLE) Predictions in N-Octane/Water Blends

Abstract

Blends of bitumen, clay, and quartz in water are obtained from the surface mining of the Athabasca Oil Sands. To facilitate its transportation through pipelines, this mixture is usually diluted with locally produced naphtha. As a result of this, naphtha has to be recovered later, in a naphtha recovery unit (NRU). The NRU process is a complex one and requires the knowledge of Vapour-Liquid-Liquid Equilibrium (VLLE) thermodynamics. The present study uses experimental data, obtained in a CREC-VL-Cell, and Artificial Intelligence (AI) for vapour-liquid-liquid equilibrium (VLLE) calculations. The proposed Artificial Neural Networks (ANNs) do not require prior knowledge of the number of vapour-liquid phases. These ANNs involve hyperparameters that are used to obtain the best ANN model architecture. To accomplish this, this study considers (a) R² Coefficients of Determination and (b) ANN training requirements to avoid data underfitting and overfitting. Results demonstrate that temperature has a major influence on ANN vapour pressure predictions, while the concentration of octane, the naphtha surrogate having, in contrast, a lesser effect. Furthermore, the ANN data obtained allows the calculation of octane-in-water and water-in-octane maximum solubilities.

1. Introduction

Canada is the sixth largest oil producer in the world, significantly contributing to the Athabasca oil, tar, and bituminous sands [1]. Northern Alberta holds one of the world’s largest deposits of hydrocarbons, containing more than 175 billion barrels of bitumen [2]. Bitumen from the oil sands is produced by employing approximately 20% surface mining and 80% in situ technologies. Surface mining yields a blend of bitumen, quartz, clay, and water. This is composed of about 85% quartz/clay particles and 15% bitumen plus water [3]. There are three classes of oil sands with different bituminous contents: low-grade (6–8 wt% bitumen), medium-grade (8–10 wt% bitumen), and rich grade (>10 wt% bitumen) oil sands [3]. Bitumen is composed of a complex mixture of hydrocarbons, with its elemental composition including hydrogen, carbon, nitrogen, and metals such as vanadium and nickel [3]. Bitumen can be blended with a solvent to remove water and solids. Usually, this solvent is a locally produced naphtha [4]. Following this process, the naphtha can be reclaimed in a Naphtha Recovery Unit (NRU). In this manner, one is able to reuse it, while minimizing its environmental impact [4,5].

In the NRU feed, hydrocarbons are blended with water. Due to the low hydrocarbon miscibility in water, and the lack of Vapour-Liquid-Liquid Equilibrium (VLLE) data available, our research group has shown that there is presently a lack of predictability of the NRU recovery unit efficiency [5,6,7,8]. Thus, studies considering hydrocarbon/water blends, and their vapour-liquid-liquid equilibrium (VLLE), as shown in Figure 1, are of major significance to improve the design and the operation of these VLL separators.

With the advancement of computational power, ANNs have attracted the attention of the scientific community, in order to solve VLLE and flash calculation problems. Schmitz et al. (2006) used ANNs to predict the number of phases in a ternary ethanol–ethyl acetate–water blend [6]. A total of 58,828 experimental data points were used concurrently with the NRTL model and the Antoine equations. This was conducted to evaluate liquid–liquid equilibrium, vapor–liquid–liquid equilibrium, and vapor–liquid equilibrium, at temperatures ranging from 335.2 K to 373 K, at 760 mmHg [9]. Argatov and Kocherbitov (2019) proposed the generalization of Wilson and NRTL models by employing ANNs and acetic acid–n-propyl alcohol–water blends at 313.15 K [10]. Li et al. applied ANNs to binary blend flash calculations by using a set of 1300 data points [11]. Zhang et al. (2020) used ANNs with data generated from an iterative NVT flash calculation. They suggested that by model training reliability was questionable. Thus, as an alternative, they proposed a self-adaptive algorithm to accelerate the flash calculations. The critical properties of each component were considered as inputs to the neural network, with the output identifying the total number of phases at equilibrium and the molar compositions in each phase [12]. For ANN model training and testing, 90,601 data points obtained from an 8-hydrocarbon blend were employed. A total of 90% of the data were used for training and a 10% was employed for ANN model validation [3].

In previous work, our research group [5] employed Machine Learning (ML) methods to calculate VLE and VLLE by using theoretical data from traditional thermodynamic models such as the NRTL [6,8]. It was extensively shown that flash and other ML calculations using theoretical data or data from simulations were not reliable [10]. In the present work, ANNs are used instead to calculate VLLE. N-octane in water blends in experiments conducted in a CREC-VL-Cell [5] are employed to train and validate the ANNs. Octane in water is used given that it is a good surrogate to represent naphtha in water blends [9]. The proposed ANNs circumvent convergence problems reported in [6], occurring when using classical VLLE thermodynamics equilibrium models and do not require one to know the number of VL or VLL phases prior to equilibrium calculations [6].

2. Materials and Methods

This section describes both octane and water solutions, the CREC-VL-Cell unit used for VLE and VLLE experiments, vapour-liquid and vapour, the data recorded and the Artificial Neural Networks of the present study.

2.1. Materials

Distilled water (18.02 g/mol) was used in all the experimental studies. n-Octane (114.23 g/mol) with a 99.0% purity and a 0% water content was obtained from Sigma-Aldrich (St. Louis, MO, USA).

2.2. CREC-VL-Cell Unit

The Chemical Reactor Engineering Center (CREC, London, ON, Canada) developed a CREC-VL-Cell unit that allows the VLL equilibrium thermodynamic measurements of hydrocarbon–water blends (refer to Figure 2) [6]. The CREC-VL-Cell uses a marine type of impeller (propeller) that ensures close to isothermal conditions and homogenous mixing inside the cell. This special cell design, proposed by the CREC team, allows one to analyze a hydrocarbon–water sample directly, without losses of light volatile components, due to sample transfers [8].

This unit uses a “dynamic heating method” with the temperature of the cell increasing progressively, using a thermal ramp of 1.22 °C/min. As a result, every run provides a large amount of vapour-liquid equilibrium data (10 Hz), with the vapour pressure data being recorded at various temperatures, every 0.01 s.

The proposed dynamic method, as reported in Figure 3, involves the simultaneous recording of temperature and pressure, as the run time is progressing. Additional explanations regarding the cell operation are reported in [5,8]. Data obtained from this dynamic method were validated with static measurements [5,8].

2.3. Data Available and Data Cleansing

Experimental data were obtained in the CREC-VL-Cell of the present study by changing the temperature for a set initial n-octane concentration in water. One should note that for every temperature and initial n-octane in water concentration, the total pressure measured required a correction. To accomplish this, the air partial pressure was discounted from the cell total pressure, as proposed in [5].

Sixty-one (61) runs were developed by using various n-octane/water blends and temperatures. The obtained data were combined into one single file, leading to 4200 run records that were employed to evaluate the proposed ANN models. Additional details regarding the 61 runs developed were reported by our research team in [5].

2.4. Artificial Neural Networks (ANN)

ANN models emulate the human brain [13]. The human brain is able to remember and use earlier experiences as a precedent for different future situations. ANNs include basic elements designated as nodes (also called units) that represent brain neurons, as well as layers that represent an ensemble of nodes [14]. Figure 4 describes an ANN configuration with input and output layers. The input and output layers are connected through hidden layers. The input layers provide the data provided to the ANN model, while the hidden layers process the data, and finally, the output layers yield the ANN model output variables [14,15,16].

Figure 5 further describes coexisting inner processes taking place in between “j” and “j + 1” generic hidden layers. One can observe in Figure 5 that both “forward calculations” and “backward calculations” take place. In the “forward calculation”, data are fed to the “j” hidden layer, processed in this layer, and then moved from the “j” to the “j + 1” hidden layer. In the “backward calculation”, data are fed and processed in the “j + 1” layer and moved back in the opposite direction, from the “j + 1” layer to the “j” hidden layer.

Thus, ANN modeling involves Steps A to D that describe the connection between layers in the forward direction, and Steps F to H that explain the ANN connections in the opposite backward direction, as follows:

• Step A: A random initial weight is assigned to all nodes in the multilayer ANN model structure [17].
• Step B: An $\mathrm{“}x\mathrm{”}$ parameter is defined for each layer connection as $x={\sum }_{i=0}^m{a}_{ij}{w}_{ij}+A, \mathrm{w}\mathrm{i}\mathrm{t}\mathrm{h} \mathrm{“}x\mathrm{”}$ being the sum of the $a_{ij}$ input, $w_{ij}$ representing the weights assigned in the generic “j” hidden layer, “A” denoting a bias parameter, and “m” standing for the layer number of nodes [18].
• $Step C :$ An $\alpha \left(x\right) F\left(x\right)$ activation function is chosen to provide an adequate fitting for the available data [19].
• Step D: The revised data calculated in Step C is transferred to the next “j + 1” hidden layer. This process is repeated until the very last layer or output layer is reached [20].
• Step E: An output error is calculated, using an R² Coefficient of Determination [17].
• Step F: A new set of weights is established by using a stochastic gradient, which is calculated by employing an optimizer function as follows: $\overline{W_{Updated}}=-\nabla \ast C\left(w\right)+\overline{W_{Initial}}$ [19].
• Step G: The revised weights are assigned to the various nodes and layers.
• Step H: The calculation process, as outlined from Steps B to G, is repeated until the algorithm learning rate no longer improves results or the output layer data reaches the acceptable tolerance range for each node [17,19,20].

3. ANN Modeling with Hyperparameters

While the ANN methodology is becoming popular in chemical engineering, there is limited information in the open technical literature, regarding the theoretically based selection of the best architectures for ANNs, in order to obtain VLL equilibrium thermodynamic data.

Given the above, Python version 3.9.5 was used to develop the ANN models of the present study. This coding software was selected given that it is an open-source tool, with a rich library of built-in modules for ML and AI, as well as with a good capability for result visualization. The main library modules used in this study include the following: (a) Scikit-learn (sklearn 0.0), (b) Keras 2.6.0, (c) Pandas 1.1.3, (d) NumPy 1.19.2, and (e) Matplotlib 3.3.2. Pandas and NumPy were used for data processing, Scikit-learn and Keras were applied for ML and AI tasks, and Matplotlib was employed to construct all the plots and graphs of this document.

Table 1. ANN Hyperparameters and their dataset types.

Hyperparameter	Dataset Type	Usual Values
Hidden layer	Integer	1, 2, 3
Units (Neurons)	Integer	10, 50, 100
Activation Function	Equation	ReLu, Tanh, SoftMax
Optimizer	Algorithm	Adam, RMSP
Cost (or Loss) Function	Equation	MSE, MAE
Epoch	Integer	50, 100, 200
Batch Size	Integer	2, 4, 8, 16, 32

reports typical hyperparameters for the ANNs such as hidden layers, nodes, activation functions, optimizer, cost function, Epochs, and Batch sizes. These parameters are of importance as they provide the required accuracy for a wide range of possible computations. Thus, the development of an adequate ANN model must include the proper selection of the number of hidden layers, nodes, and activation functions [19]. On this basis, the determination of a successful ANN model is achieved with a good balance between accuracy and efficiency [12,18]. Table 1 also reports the advisable Epochs and Batch sizes, with Epoch size referring to the number of times that the algorithm repeatedly learns from the available training data [21], and Batch size referring to the number of mini-Epochs or Batches required for a given dataset.

Furthermore, in the present study, in order to establish the most appropriate ANN model, different hyperparameters were combined, creating at least six ANN models by employing different activation functions, as reported in

Table 2. Activation functions—formulas and recommended applications.

Activation Function	Equation	Recommended Applications [21]
ReLU	$\max \left(0, x\right)$ [22]	General purposes
ELU	$\left\{\begin{array}{ll} x & ,x\ge 0\\ \alpha (e^x-1)& , x<0\end{array}; 0<\alpha <1 \right.$ [21]	Classification
Sigmoid	$\frac{1}{1+e^{-x}}$ [12]	Binary classification
Tanh	$Tanh\left(x\right)=\frac{e^x-e^{-x}}{e^x+e^{-x}}$ [20]	Binary classification
SoftMax	$\frac{e^x}{\sum _{j=1}^k{e}^x}$ [12]	Multivariable classification
SoftPlus	$\log \left(e^x+1\right)$ [22]	Function approximation

.

The ELU, ReLU, Sigmoid, SoftMax, SoftPlus, and Tanh activation functions for the ANNs were evaluated by using several numbers of hidden layers, ranging from one to five and various numbers of nodes in every hidden layer, as follows: 5, 10, 50, 100, and 200. Regarding model configuration, the input layer involved two nodes and the output layer encompassed one node. This was the case for all the models considered, where the temperature and hydrocarbon molar fraction were the input variables, and the pressure was the model output variable. Furthermore, model training was developed by using an 8 GB RAM memory, and a RYZEN 5-4000 series processor, with this taking 3.7 h of total computer time.

Table 3. ANN hyperparameters.

Hyperparameter	Value
Optimizer	Adam
Epochs	100
Batch Size	32
Cost (Loss) Function	MSE

describes various ANN hyperparameters adopted in the present study, used to develop ANN calculations.

The Adam Optimizer hyperparameter from Table 3 was considered and evaluated by using Momentum and Root Mean Square Propagation, in order to establish the optimized descent gradients, which allow one to avoid the local minima and to calculate the global minimum [23,24] The ANN models evaluated included Epochs and Batch size hyperparameters. Epochs were set to 50, 100, and 150. It was shown that Epochs exceeding 100 did not improve the R² Coefficient of Determination significantly, as shown in Figure A2 in Appendix B. Thus, Epochs set to 100 were considered for all ANNs calculations. Furthermore, the Batch Size was set to 32. This is a typical batch size used to achieve training stability that requires a low computation time [25,26].

4. Evaluation of Proposed ANN Models

Data from the 61 runs developed in the CREC-VL-Cell led to 4200 records of cells, employed to evaluate the various ANN models considered, as described in Table 2 and Table 3. Runs included conditions with input and output variable ranges, as described in

Table 4. Description of Input and Output Variable Ranges involved in the ANN’s Calculations of the present study.

Data Inputs	Data Outputs
Temperature: 70–110 °C Hydrocarbon molar Fractions: 0 to 1	Pressure: 100–400 KPa Phases: two or three Maximum Solubility for octane in water from 0.00015 to −0.00079 Maximum, solubility for water in octane from: 0.012 to 0.015

.

In this respect, the R² Coefficient of Determination helped to establish the ANN model’s ability to provide adequate predictions. The R² was computed by means of the following equation:

$$\mathrm{Coefficient} \mathrm{of} \mathrm{Determination}={\mathrm{R}}^2=1-\frac{RSS}{TSS}=1-\frac{\sum _i{\left(y_i-f_i\right)}^2}{\sum _i{\left(y_i-\overline{y}\right)}^2},$$

(1)

where $RSS$ is the residual sum of squares, $TSS$ is the total sum of squares, $y_i$ is the output variable experimental data, $f_i$ is the output variable predicted data, and $\overline{y}$ is the predicted data mean [25]. Note that an R² Coefficient of Determination close to 1 is considered to be a good correlation, while a 0 value means that there is a very poor data correlation.

Figure 6 reports a comparative analysis of various ANN models evaluated by using various numbers of hidden layers and different activation functions. This graph shows how the number of hidden layers affects the R² Coefficient of Determination.

It can be noted, as shown in Figure 6, that the ReLU activation function superseded all the others, in most cases, in term of the maximum value of the R² Coefficient of Determination, with this coefficient remaining within the 0.96–0.97 range, when the number of hidden layers increased from 1 to 5, with three hidden layers providing best results.

Furthermore, as shown in Figure 7, training computational times were significantly affected by the number of hidden layers. One interesting observation is that the ANNs with the ReLU activation function were, in most cases, the ones consistently requiring lower computational times.

Figure 8 evaluates ANNs with various activation functions by varying the number of nodes per hidden layer. It shows how this affects the R² Coefficient of Determination. It was observed that the best ANNs were the ones using 50 nodes per hidden layer, with no significant improvement in the Coefficients of Determination when the number of nodes surpassed 50.

In the case of Figure 8, it can be noted that the ANN that used the ReLU activation function displayed the best performance in terms of the R² Coefficient of Determination when compared to other ANN models using the other activation functions.

Finally, Figure 9 evaluates ANNs with various numbers of nodes per hidden layer, in terms of required computational training time.

It can be observed, in Figure 9, that the ANN model with the ReLU activation function consistently required the shortest computational training time, when compared to ANNs with other activation functions.

Thus, on the basis of the above, it can be concluded that the best ANN, in the context of the present study, includes 3 hidden layers, 50 node units per hidden layer, and the ReLU activation function. This conclusion was reached by observing the higher R² Coefficients of Determination obtained and the lower ANN computational training times required, when using these parameters. In this case, the R² Coefficient of Determination was very close to 0.97, and the required computational time was close to 15 s. It is interesting to note that the 0.97 R² Coefficient of Determination superseded the previous R² Coefficient of Determination values obtained via Machine Applications by our research team [6].

Given all this, an ANN designated as 3HL-50N-ReLU ANN (3 hidden layers, 50 nodes, and ReLU activation function) was selected to obtain the best system pressure predictions, as reported in the upcoming sections of this article.

5. Results and Discussions

The present study shows the viability of the ANNs used to predict the system pressure of highly diluted hydrocarbons in water. This is conducted without the need of defining a priori the number of liquid and vapor phases, present in complex hydrocarbon/water blends and found in naphtha recovery processes.

To train the ANN models, a system pressure dataset containing 4220 data points, resulting from various experiments was used. These records were from experiments conducted between 30 °C to 120 °C with octane in water concentrations ranging from 0 to 100 wt.% and with system pressures between 0 to 440 KPa.

To determine the accuracy of the ANN system pressure predictions at various temperatures and octane in water concentrations, the ANN was trained 20 times, while using a 70% randomly selected of the available experimental data. The remaining 30% of the experimental data was employed for validation purposes.

By using this approach, it was found that the average R² Coefficient of Determination for 30% of the experimental data was over 0.98 and that the average computational training time required for each model was 10.6 s. The lowest R² Coefficient of Determination was over 0.979 and the highest one was below 0.984, while the training times were in the range of 10.1 s to 11.5 s, respectively.

Figure 10 compares the predicted and experimentally measured system pressures when using the best-performing ReLU–ANN model, and 30% of the experimentally measured pressure values obtained with the CREC-VL-Cell. This figure displays an R² of 0.982 (average) with 11.1 s of required computational time.

Thus, Figure 10 shows the excellent ability of the proposed ANN model to predict system pressure values with reasonable accuracy and limited computer time.

Once the proposed ANN model was trained and validated, 90% of the available experimental data was chosen randomly to create a restricted dataset. Then, the developed ANN model was retrained with 70% of the points of this restricted dataset. The remaining 30% of the limited-size dataset was employed for ANN validation. It was observed that under these conditions, the average R² Coefficient of Determination remained at 0.9439, with a minimum value of 0.9201 and a maximum value of 0.9583. Given, these good R² Coefficients of Determination obtained with 90% of the available data, it was considered that the original 4220 experimental data points were adequate for the ANN model development of the present study.

Figure 11a–d illustrates the adequacy of the proposed ANN model for predictions, with octane–water concentrations ranging from 0 to 100%, and temperatures ranging from 80 °C to 110 °C. Figure 11a–d report experimental data points [10], as well as pressure predictions obtained from the ANN model developed.

It can be observed that at all four thermal levels (80 °C, 90 °C, 100 °C, and 110 °C), the ANN model predicted the total pressure closely, for the eighteen octane in water concentrations ranging from 0 to 100%, with 7–9% standard deviations. One can thus conclude that the proposed ANN is adequate, without having to know in advance, the number of phases present, as required in our previous study [7].

Table 5. Maximum Chemical Species Solubilities at Different Temperatures.

Temperature	Maximum Water in Octane Solubilities	Maximum Octane in Water Solubilities
80 °C	0.012434	0.00015784
90 °C	0.012871	0.00039511
100 °C	0.014054	0.00039511
110 °C	0.015615	0.00079189

reports the maximum solubility of chemical species, for both water highly diluted in octane and in octane highly diluted in water, using the methodology proposed by Lopez Zamora (2021) [7]. Thus, the maximum solubility is obtained at the intersection point between the characteristic linear increasing pressure at very low and very high molar fractions, and the constant pressure, at all other intermediate molar fractions, as described in Figure 11a–d. It is speculated that at this intersection points, there is a change in the multiphase blend state, which evolves from two LV phases (liquid–vapour) to three LLV phases (liquid–liquid–vapour).

One should note that results reported in Table 4 are in agreement with Lopez Zamora [7], who reported maximum solubilities for water highly diluted in octane and octane highly diluted in water, in the 0.014–0.017 and 0.00015–0.00079 ranges, respectively. These maximum solubilities are much higher (about 20 times) than the ones previously reported by others [27], adding significant value to the ANN calculations and data, reported in the present study.

6. Conclusions

• ANN models can be used effectively to calculate equilibrium system pressures in hydrocarbon–water blends.
• ANN models require the careful selection of ANN hyperparameters such as number of hidden layers, number of nodes per hidden layers, and activation functions.
• Three hidden layers, fifity node units per hidden layer, and the ReLU activation function are best parameters for the ANN model, yielding the highest R² Coefficients of Determination and requiring the shortest computational times.
• The implemented ReLU-ANN requires abundant VLL data for both training and validation. This required data was obtained experimentally using a CREC-VL-Cell, with octane–water concentrations, ranging from 0 to 100%, and temperatures ranging from 30 °C to 110 °C.
• The ANN–ReLU model used 70% randomly selected data from experiments in the CREC-VL-Cell for training, with the remaining 30% being available for ANN validation. This approach led to valuable ANN predictions, where an average R² Coefficient of Determination of 0.982 was obtained.
• The developed ANN-ReLu can be used to successfully predict both the system pressures in the entire range of octane–water compositions, as well as the maximum solubilities of octane highly diluted in water and water highly diluted in octane.