Development of Prediction Models for the Torsion Capacity of Reinforced Concrete Beams Using M5P and Nonlinear Regression Models

Abstract

Torsional strength is related with one of the most critical failure types for the design and assessment of reinforced concrete (RC) members due to the complexity of the associated stress state and low ductility. Previous studies have shown that reliable methods to predict the torsional strength of RC beams are still needed, namely for over-reinforced and high-strength RC beams. This research aims to offer a novel set of models to predict the torsional strength of RC beams with a wide range of design attributes and geometries by using advanced M5P tree and nonlinear regression models. For this, a broad database with 202 experimental tests is used to generate highly reliable and resilient models. To build the models, three independent variables related with the properties of the RC beams are considered: concrete cross-section area (area enclosed within the outer perimeter of the cross-section), concrete compressive strength, and torsional reinforcement factor (which accounts for the type—longitudinal or transverse—amount, and yielding strength of the torsional reinforcement). In contrast to multiple nonlinear regression approaches, the findings show that the M5P tree approach has the best estimation in terms of both accuracy and safety. Furthermore, M5P model predictions are far more accurate and safer than the most prevalent design equations. Finally, sensitivity and parametric studies are used to confirm the robustness of the presented models.

1. Introduction

In several cases, reinforced concrete (RC) structures incorporate members which need to sustain high torsional loading in their critical cross-sections. These cases include common bridges and building structures, where linear members (beams and columns) might need to suffer high torsional effects due to eccentric loadings. Even if torsional effects are combined with other internal forces, an accurate estimation of the torsional strength is required in the critical zones of the members. This is particularly important in assessment projects.

For practice, structural engineers usually base their calculations on the provisions from standards, namely, for the ultimate limit states. Nowadays, standards for RC structures incorporate design rules for torsion based on semi-empirical equations, which can still fail to accurately estimate the torsional strength of RC beams and even provide unsafe predictions, namely for over-reinforced and high-strength RC members [1,2,3]. As a consequence, in recent years, some structural failures attributed to torsional effects are still reported [4]. Hence, alternative and more accurate models for RC members under torsional loading are still needed.

In the last decades, reliable models which allow us to compute, with accuracy, the strength of RC beams under pure torsion have been proposed. The model, so-called skew-bending, theory was firstly proposed by Hsu in 1968 [5] and further developed [6,7,8]. This model was initially established from the observation of the torsional failure pattern in several experiments with RC rectangular beams with small sizes. However, this model gives rise to less accurate and more complex equations for RC beams with cross-sections with larger aspect ratios, with more complex geometrical cross-sections, and for combined torsion. For these reasons, it has been abandoned by most of the research community. Nevertheless, some few current standards still base their provisions for torsion on the skew-bending theory, such as the Russian Building Code [9].

Nowadays, most of the standards for RC structures (such as the American, Canadian, and European ones [10,11,12,13]) base their provisions for torsion on the space truss analogy, which was firstly proposed by Raush in 1929 [14] and further developed mainly in the second half of last century [15,16,17,18]. This physically more consistent model for RC beams in the cracked stage provides simpler equations to compute the torsional strength for a wide range of geometrical RC cross-sections. However, for practical use, different empirical hypotheses were incorporated in the model for the standards in order to simplify the torsional design. As a consequence, the resulting semi-empirical equations can be somewhat different between standards and, as referred before, they can fail to provide accurate and safe predictions for the torsional strength of RC beams. In the past decades, refined models based on the space truss analogy have been proposed, which allow the strength of RC beams under pure and combined torsion to be predicted with high accuracy [19,20,21,22,23,24]. More advanced analytical models have also been proposed in the literature for RC beams under torsion and combined loadings [25,26]. In addition, nonlinear finite element models have also proved to be very reliable to simulate the behavior of RC beams under torsion until failure [27]. Despite these models having been shown to be very reliable when compared with experimental results, some of them require advanced calculation procedures to be implemented in the computer, sometimes with a lot of programming effort, and others already available in the software can be computationally very demanding. Most of them are not easy for practitioners to use for structural design and assessment.

These drawbacks can be solved by developing accurate Machine Learning (ML)-based models as an alternative to the previously referred ones. Several ML techniques have already taken place in recent years due to innovations in computing and have been applied to successfully predict the strength of RC members, including, but not limited to, punching shear in slabs and shear in beams [28,29,30,31,32]. Among such studies, some of them have already focused on the problem of RC members under torsion [33,34,35,36,37] and also strengthened RC beams with torsional loading [38,39,40]. However, since several ML techniques exist and can be applied to solve problems in the field of structural civil engineering [40], one can state that few studies still exist in the literature for a given problem, such as the one related to RC beams under pure torsion. Hence, more studies are needed to help find the best ML models and strategies for a given problem, in this case, for predicting with accuracy the torsional strength of common RC beams with broad and varying design attributes and geometries. Furthermore, in machine learning model definitions, Artificial Neural Network (ANN) [34,35,37,39,40] and Ensembled Trees [38] are examples of computer-aided machine learning approaches, whereas M5P model tree and multiple non-linear regression (MNLR) are examples of explicit machine learning techniques. The first category is referred to as “black-box approaches”, and this indicates that their paradigms rely on a computer-assisted methodology, while the second category is known as “white-box techniques”, and this implies that they provide clear expressions.

This work’s primary innovation and contribution is the development of two unique explicit correlations which can be used easily to accurately forecast the torsion resistance of RC beams with broad and varying design attributes and geometries. To achieve this, a sample experimental database was used to implement M5P and MLNR models using three input parameters: the cross-section area (the area enclosed within the outer perimeter of the cross-section), the concrete compressive strength, and a reinforcement factor which accounts for the type—longitudinal or transverse—amount, and yielding strength of the torsional reinforcement. The newly proposed models were assessed using statistical and graphical criteria, and their performance was compared to that of earlier design building codes and models. Finally, and in order to assess the degree of contribution of each parameter used to run the models, a sensitivity analysis was also developed.

2. Materials and Methods

2.1. M5P Model Tree Techniques

The model trees approach provides the piecewise linear fit of the class and a structural representation of the data. Although they feature a decision tree in a traditional form, as illustrated in Figure 1, they employ linear functions at the leaves rather than discrete class labels. A model tree is employed to anticipate numerical values. A linear regression model is maintained at each leaf to forecast the class value of instances that will reach the leaf. Model trees provide superior compactness and prediction accuracy over standard regression trees [41]. By using rules and regression equations, they directly define the patterns and correlations that are inherent in the data, whereas other advanced machine learning algorithms, such as ANN (Artificial Neural Network) and SVR (Support Vector Regression), hide them.

2.1.1. The M5 Tree Algorithm

Model trees are effective and precise methods for modeling the relationships and patterns in big datasets, despite being simple [43]. In order to forecast continuous variables, Quinlan [44] created a novel form of tree known as the M5 tree. The regression trees created by CART have values at their leaves, but trees created by M5 can include multivariate linear models. One significant distinction between the M5 regression tree and the CART (Classification and Regression Tree) regression tree is that the M5 tree can include multivariate linear models, whereas the CART tree can have values at its leaves. Predictions by M5 are more adaptable under this circumstance. The three main stages of M5 tree growth are tree building, tree pruning, and tree smoothing, as illustrated in Figure 2. The construction of the M5 tree aims to optimize a parameter known as the standard deviation reduction (SDR). A definition of the SDR is the following one:

$$SDR=sd\left(T\right)-{\displaystyle \sum }_{i=1}^n\frac{\left|T_i\right|}{\left|T\right|}sd\left(T_i\right)$$

(1)

where $T$ is the set of cases, $T_i$ is the ith subset of cases that result from the tree splitting based on a set of variables (attributes),$sd\left(T\right)$ is the standard deviation of $T$, and $sd\left(T_i\right)$ is the standard deviation of $T_i$ as a measure of error [42].

2.1.2. The M5P Tree Algorithm

The original M5 tree technique was improved by Wang and Witten [45] and was given the name M5P algorithm to handle enumerated attributes and attribute missing values. Each enumerated attribute is converted into a binary variable before the M5P tree algorithm starts building the tree. Traditional decision trees and the nodes’ potential for linear regression functions are combined in M5P. Firstly, a tree is constructed using a decision-tree induction process. Splitting criteria are employed instead of maximizing information gain at each inner node to reduce intra-subset fluctuation in the class values along each branch. If just a small number of instances are left, or the class values of all examples that reach a node differ only slightly, the splitting process in M5P comes to an end. Secondly, each leaf on the tree is pruned back. With a regression plane, an inner node is transformed into a leaf during pruning [46].

Comparing the M5P algorithm to other algorithms shows advantages. It can be applied to missing values in both continuous and categorical data. When a value is absent, M5P uses a technique called “surrogate splitting”, which asks for another attribute to be divided in place of the original location and uses it in return. Class magnitude is employed in the training portion by M5P as a surrogate attribute in the belief that it should be connected to the attribute that is used for splitting. All missing values are replaced by the average values of the respective characteristics from the training example at the conclusion of the splitting phase. Instead of using an unknown attribute value in the testing phase, the average value of that attribute for all training instances that reach the node is utilized. M5P creates models that are reasonably understandable and small [47].

2.2. Multiple Nonlinear Regression Method

Let us assume that $y$ is the dependent variable and that $x_1, x_2,\dots$ and $x_n$ are the n independent variables in the nonlinear relationship. They have a nonlinear relationship that can be represented as follows:

$$y=a_0{x}_1^{a_1}{x}_2^{a_2}\dots x_n^{a_n}$$

(2)

From Equation (2), the following linearized equation can be constructed by using logarithmic transformation:

$$logy=log{a}_0+a_{1}log{x}_1+a_{2}log{x}_2+\cdots +a_{n}log{x}_n$$

(3)

Similar to the Multi-Linear Regression (MLR) approach, the least squares method can be used to calculate the coefficients $a_0, a_1,\dots$, and $a_n$.

2.3. Model Inputs

The mechanical characteristics (e.g., the torsional strength $T_R$) of composite members such as reinforced concrete beams are influenced by a wide range of variables. These variables include the cross-section area, amount of longitudinal and transverse reinforcement, spacing between bars, concrete compressive strength, steel yielding strength, modulus of elasticity, and more [48,49,50,51]. Therefore, it is necessary to identify the most critical parameters from various independent variables to create an accurate model. In this research, three independent factors comprising the cross-section area ($A_c$, the area enclosed within the outer perimeter of the cross-section), the concrete compressive strength ($f_c$), and a reinforcement factor which accounts for the type—longitudinal or transverse—amount, and yielding strength of the torsional reinforcement $\left(A_l{f}_{yl}{A}_t{f}_{yt}/s\right)$ were considered as predictive variables based on prior research and current design codes [1,2]. The only output of the model is the torsional capacity of the RC beam, $T_R$. In the reinforcement factor, $A_l$ is the total area of longitudinal reinforcement, $A_t$ is the area of one rebar of the transverse reinforcement (stirrups), $s$ is the stirrups’ longitudinal spacing, and $f_{yl}$ and $f_{yt}$ are steel yielding stress of the longitudinal and transverse reinforcement, respectively.

2.4. Database Used

A reasonably recent comprehensive dataset by Bernardo et al. [1] from 21 experimental investigations carried out between 1968 and 2020 is utilized to analyze the torsional strength of the RC beams and create a new model and prediction formulae. The utilized dataset is available and described in detail in [1], including all properties of the RC beams. For this reason, it is not incorporated in this study. For various $T_R$ values between 8.9 kN.m and 521.3 kN.m, the whole collection consists of 202 recorded experimental data samples. Two independent parts of the dataset are randomly selected: training (162) and testing (40). The training dataset is used to train the M5P algorithm, and the testing dataset is used to evaluate it.

Table 1. The input and output parameter ranges for the training and testing sets.

Statistics	Subset	Min	Max	Mean	STD
$A_c$ (m2)	Training	0.04	0.37	0.15	0.10
	Testing	0.05	0.36	0.15	0.09
$f_c$ (MPa)	Training	13.1	109.8	47.1	23.50
	Testing	17.1	96.8	45.1	20.61
$A_l{f}_{yl}{A}_t{f}_{yt}/s$	Training	1.19 × 106	531.30 × 106	37.27 × 106	61.15 × 106
	Testing	1.13 × 106	310.78 × 106	32.35 × 106	57.34 × 106
$T_R$ (kN.m)	Training	8.99	521.33	92.48	101.01
	Testing	12.30	467.26	90.79	95.04

lists the range of output and variables for testing and training datasets.

3. Model Result

3.1. M5P and MLNR Derived Models

The relationship between the input and output parameters in the current study is not always linear, since the distribution of a wide range of values follows a nonlinear trend, such as a power law. When calculating the torsional capacity, the M5P model can be utilized to produce concise, transparent rules that are simple to implement. However, it could suggest that the input and output parameters are linearly related. Log (inputs) and log (outputs) were used to create a model to bypass this limitation (output). Consequently, and based on the results from studies [1,2], it was hypothesized that $T_R$ is a power function that has the following form:

$$T_R=a^{\prime }{\left(f_c\right)}^{b^{\prime }}{(A_c)}^{c^{\prime }}{(A_l{f}_{yl}\frac{A_t{f}_{yt}}{s})}^{d^{\prime }}$$

(4)

where the constants ($a^{\prime }$, $b^{\prime }$, $c^{\prime }$, and $d^{\prime }$) have different values under various circumstances and all other terms were previously defined.

Figure 3 displays the established model tree created using the M5P technique.

Table 2. Coefficients for Equation (4) predicted from the model tree.

Linear Model	Coefficient
	${\mathit{a}}^{\prime }$	${\mathit{b}}^{\prime }$	${\mathit{c}}^{\prime }$	${\mathit{d}}^{\prime }$
LM1	4.5311	0.3108	0.7739	0.1491
LM2	1.8252	0.2765	1.1161	0.2883
LM3	1.9852	0.2866	1.0774	0.272

provides the coefficients for Equation (4) predicted from the M5P algorithm.

The following example illustrates the methodology of using M5P to predict the torsional capacity of RC beams. For example, let us consider a reference beam from the testing dataset with the attributes summarized in

Table 3. Reference beam attributes from the testing dataset.

TR(exp) (kN.m)	${\mathit{f}}_{\mathit{c}}$(MPA)	${\mathit{A}}_{\mathit{c}}$(m2)	${\mathit{A}}_{\mathit{l}}{\mathit{f}}_{\mathit{y}\mathit{l}}\frac{{\mathit{A}}_{\mathit{t}}{\mathit{f}}_{\mathit{y}\mathit{t}}}{\mathit{s}}$
37.48	28.07	0.098	12090616

.

As can be seen from Table 3 and Figure 3, the beam sample belongs to the M5P LM1 prediction equation, since the value of $A_c$ is ≤ 0.1164 and > 0.0625. Therefore, using the coefficients related to LM1, the predicted value for the torsional capacity of the reference beam in Table 3 is 37.29 kN.m, which agrees very well with the experimental torsional capacity (37.48 kNm).

The newly developed equation to predict the torsional resistance of RC beams in this research using an MLNR (Multiple NonLinear Regression) model can be written as follow:

$$T_R=5.1808 {\left(f_c\right)}^{0.1964}{(A_c)}^{1.1755}{(A_l{f}_{yl}\frac{A_t{f}_{yt}}{s})}^{ 0.2459}$$

(5)

3.2. Performance Analysis

In the ML modeling process, the number of the entire dataset used to build a new regression model is a critical task. Frank and Todeschini [52] recommended a minimal proportion of three between the full dataset utilized and the number of included variables for constructing a credible model based on data mining approaches. A value of five is considered a more conservative and safe option. As can be seen in this study, the ratio between experimental data points to the predictors is equal to 202/3 = 67.33.

In order to compare the M5P model with MLNR, performance metrics were used. The Root Mean Square Error (RMSE), Coefficient of Determination (R²), and Mean Estimation Error (MAE) were used to evaluate the predictive performance of M5P and MLNR on a regular basis [53,54]. The three equations below are used to calculate these metrics.

$$R^2=\frac{{{\displaystyle \sum }}_{i=1}^n{(y_i^{obs}-y^{-obs})}^2-{{\displaystyle \sum }}_{i=1}^n{\left(y_i^{obs}-y_i^{pre}\right)}^2}{{{\displaystyle \sum }}_{i=1}^n{\left(y_i^{obs}-y^{-obs}\right)}^2} \in \left[0,1\right]$$

(6)

$$RMSE=\sqrt{\frac{{{\displaystyle \sum }}_{i=1}^n{\left(y_i^{obs}-y_i^{pre}\right)}^2}{n}} \in \left[0,+\infty \right]$$

(7)

$$MAE=\frac{{{\displaystyle \sum }}_{i=1}^n\left|y_i^{obs}-y_i^{pre}\right|}{n}\in \left[0,+\infty \right]$$

(8)

where $n$ denotes the overall number of data points used to calculate the bias, $y_i^{obs}$ and $y_i^{pre}$ are the observed values for the torsional strength and the predicted torsional strength of the RC beams for the i^th observation, respectively, and $y^{-obs}$ is the average of all observed data.

As shown in Figure 4a,b, for both training and testing datasets, the M5P and MNLR models’ results reveal a little scatter around the ideal line between the observed and predicted values of the torsional strength. Model performance measurements are provided in

Table 4. M5P and MLNR model performance in predicting the torsional strength for training and test datasets.

Statistics	MAE		RMSE	R	R2
Training	M5P	8.279	13.288	0.991	0.983
	MNLR	9.435	13.908	0.989	0.979
Testing	M5P	8.224	13.432	0.990	0.981
	MNLR	9.224	14.438	0.980	0.960
Total	M5P	9.521	14.663	0.990	0.981
	MNLR	10.240	15.049	0.980	0.960

for the training and testing datasets to further check the validity of the obtained models.

There is a substantial correlation between the predicted and experimental values when |R| > 0.8, according to Smith [55]. The matching between the observed and predicted values might not be achieved, even if the R value is near 1. In other words, those two variables just vary similarly. It is possible to overcome this constraint by utilizing the coefficient of determination R². When the model’s R² value is close to one, and the model’s RMSE and MAE values are close to zero, the model can be trusted to produce precise results [56]. In terms of accuracy, the models’ performance measures in Table 4 indicate that M5P outperformed the MNLR in both the training and testing sets. For example, and for R², RMSE and MAE values for the training set of the M5P model are 0.983, 13.288, and 8.279, respectively, while the RMSE and MAE values of R² for the testing set are 0.981, 13.432, and 8.224, respectively. It should be noted that the performance of MNLR is also acceptable.

A comparison of the forecasting accuracy of the M5P and MLNR models from this study, with standards and simplified predicting models for the torsional strength of RC beams using the testing dataset, is performed, and the findings are summarized in

Table 5. Accuracy of other models for predicting the torsional strength with M5P and MLNR.

MAE		RMSE	R2	$\overline{\mathit{x}}$	COV %
SNiP18 [9]	23.1680	35.0118	0.9332	1.3584	30.6947
ACI 318-19 [10]	22.3113	36.8349	0.9619	1.2777	26.3644
MC10 [12]	19.9876	32.7077	0.8975	1.2183	44.7848
EC2 [13]	16.4672	24.8102	0.9385	1.1359	23.0495
CSA14 [11]	14.0524	29.5598	0.9399	1.0279	19.2282
Rahal [2]	11.1040	17.0198	0.9705	1.0747	12.1034
MLNR (this study)	9.2240	14.4380	0.960	1.0544	13.9623
M5P (this study)	8.3300	13.4321	0.9807	1.0419	11.6419

. The standard equations from SNiP18 [9], ACI 318-19 [10], MC10 [12], EC2 [13], CSA14 [11], and the simplified equation developed by Rahal in 2013 [2] are among the available torsional strength models investigated in this study. In order to do so, three statistical error measurements, including R², RMSE, and MAE, besides model uncertainty average value $\overline{x}$ and coefficient of variation COV for the calculated ratios $T_{R,exp}/T_{R,pred}$, were utilized for the comparison between the previously developed models and the other referred ones. The lowest RMSE and MAE values with the highest R² are achieved with the M5P model, as shown in Table 5. The mean value of the model uncertainty variable for the M5P is 1.0419, which is close to the optimal value (equal to 1). Moreover, with a COV of 11.6419, the M5P model has the lowest dispersion of all the studied models. The most prominent conservative bias was obtained from the SNiP18 equations, with a mean value of model uncertainty equal to 1.3584, while the broadest dispersion is achieved with the MC10 equation, with the value 44.78%.

The model uncertainty variable between the predicted and observed torsional strength for all the RC beams in the testing dataset was plotted against the predictors used to build each of the models in this study, as shown in Figure 5. These plots help to acquire a better understanding of the standards’ errors. In this regard, the results of the standards were compared to those of the MLNR and M5P models. An adequate model’s errors should be less sensitive or independent of changes in the input parameters involved in the phenomena. Otherwise, it is possible to conclude that such input parameters are either incorrectly included or should be included in the model [55]. As demonstrated in Figure 5, the errors of some of the design codes are sensitive to changes in the concrete compressive strength ($f_c$) or concrete area ($A_c$) values.

For example, the MC10 standard tends to overpredict the torsional strength as the value of $f_c$ or $A_c$ increases, while the ACI 318-19 standard tends to behave in the opposite direction as the value of the same variables increases. Therefore, it is possible to state that those two parameters might not have been correctly included in the standards equations, and more studies should be performed on this. On the other hand, the M5P and MLNR models’ errors are entirely unaffected by these variables. The EC2 standard is less sensitive to the changes in $f_c$ or $A_c$ values when compared to the other standards.

It can also be noticed that some trends in the behavior of the standards equations exist when it comes to relate the prediction of the torsional strength versus reinforcement factor. For example, the status of the equations for some of the standards changes from overprediction to underprediction as the reinforcement factor increases, while the status of the equations from other standards (namely MC10, EC2, and SNiP18) shows the opposite. Consequently, all input variables were better incorporated into the newly suggested models in this study, and the models’ errors were entirely independent of them.

3.3. Parametric and Sensitivity Analyses

The importance of variables in estimating the torsional strength of RC beams is determined by performing a sensitivity analysis. This analysis is carried out by removing each predictor from the database, one at a time, and training and testing the proposed models using the resulting dataset. The R², RMSE, and MAE performance measures from the testing and training database were used to analyze the relevance of each variable on the torsional strength of the RC beams. The results are summarized in

Table 6. Effect of input variables on the performance of the suggested ML models.

Models	Input Variable	Training Set			Testing Set
		MAE	RMSE	R2	MAE	RMSE	R2
M5P	$A_c$$,f_c$$,A_l{f}_{yl}{A}_t{f}_{yt}/s$	8.279	13.288	0.983	9.224	14.438	0.980
	$f_c$$, A_l{f}_{yl}{A}_t{f}_{yt}/s$	36.916	60.554	0.780	29.816	53.699	0.834
	$A_c$$, A_l{f}_{yl}{A}_t{f}_{yt}/s$	11.004	15.688	0.980	12.054	16.699	0.970
	$A_c$$,f_c$	20.362	38.627	0.863	20.347	34.240	0.917
MLNR	$A_c$$,f_c$$, A_l{f}_{yl}{A}_t{f}_{yt}/s$	9.435	13.908	0.979	9.224	14.438	0.980
	$f_c$$, A_l{f}_{yl}{A}_t{f}_{yt}/s$	46.406	72.710	0.609	51.442	78.215	0.638
	$A_c$$, A_l{f}_{yl}{A}_t{f}_{yt}/s$	11.004	17.145	0.973	10.485	16.085	0.972
	$A_c$$,f_c$	20.362	38.627	0.863	20.347	34.240	0.917

. This table depicts all of the input factors that are thought to influence the torsional strength of the RC beams. As can be noticed from Table 6, the area of concrete, $A_c$, is the most sensitive variable impacting the torsional strength of the RC beams when compared to the other input variables. It should also be noticed that leaving out the concrete compressive strength, $f_c$, has little impact on the obtained models’ performance.

4. Conclusions

In this study, two novel models for the reliable prediction of the torsional resistance of RC beams were generated utilizing M5P and MLNR. A representative experimental database based on inputs such as area of concrete, concrete compressive strength, and reinforcement factor was gathered from the published literature in order to construct the relationships.

The following findings are derived from this study:

• Both the M5P and MLNR models had excellent predictive power.
• With a total RMSE value of 14.663, the M5P model exceeds the MLNR model.
• The torsional resistance of the RC beam can be quickly and reasonably estimated using the suggested M5P model.
• The generated M5P correlation was evaluated against the equations of design building codes and other current models. All of these models could not match the developed M5P’s precision.
• The M5P and MLNR models indicated that the area of concrete had the greatest influence on the prediction of torsional resistance.

With only 202 experimental tests, the dataset used to develop the prediction models was relatively small. This represents a limitation of the presented work. The accuracy and dependability of the M5P model can be improved by utilizing a larger dataset. Due to challenges with data collecting, other signs or factors could have gone overlooked. The type and quality of data used greatly influence how well supervised machine learning systems perform. In the future, the authors want to determine if adding a new prediction model or changing the size of the training dataset can increase or decrease the suggested model’s accuracy.