Comparative Analysis of Supervised Machine and Deep Learning Algorithms for Kyphosis Disease Detection

Abstract

Although Kyphosis, an excessive forward rounding of the upper back, can occur at any age, adolescence is the most common time for Kyphosis. Surgery is frequently performed on Kyphosis patients; however, the condition may persist after the operation. The tricky part is figuring out, based on the patient’s traits, if the Kyphosis condition will continue after the treatment. There have been numerous models employed in the past to predict the Kyphosis disease, including Logistic Regression (LR), Naive Bayes (NB), Random Forest (RF), K-Nearest Neighbors (KNN), Support Vector Machine (SVM), Deep Neural Network (DNN), and others. Unfortunately, the precision was overestimated. Based on the dataset received from Kaggle, we investigated how to predict Kyphosis disorders more accurately by using these models with Hyperparameter tuning. While the calculations were being performed, certain variables were modified. The accuracy was increased by optimizing the fit parameters based on Hyperparameter tuning. Accuracy, recall or sensitivity, specificity, precision, balanced accuracy score, F1 score, and AUC-ROC score of all models, including the Hyperparameter tuning, were compared. Overall, the Hyperparameter-tuned DNN models excelled over the other models. The DNN models’ accuracy was 87.72% with 5-fold cross-validation and 87.64% with 10-fold cross-validation. It is advised that when a patient has a clinical procedure, the DNN model be trained to detect and foresee Kyphosis disease. Medical experts can use this study’s findings to correctly predict if a patient will still have Kyphosis after surgery. We propose that deep learning should be adopted and utilized as a crucial and necessary tool throughout the broad range of resolving biological queries.

1. Introduction

Machine learning (ML) involves making predictions based on the data provided to the computer [1], and using this concept of ML, machines can be trained to respond to new information without human interaction. Deep learning (DL) is a type of artificial intelligence and ML that mimics human behavior when used with certain data. It is a type of ML that enables computers to evaluate enormous amounts of data and generate preliminary findings that may subsequently be used to support any response [2].

DL, a subfield of artificial intelligence, works with algorithms that are influenced by the structure and function of the brain. Artificial neural networks with a second hidden layer are used in DL. Cascading neural networks are used to process non-linear data sets. DL is a method that searches through numerous buried data layers to find the key. It can be used to diagnose other illnesses. Due to DL, the healthcare sector may now review data quickly and accurately [3].

There could be a backbone abnormality called Kyphosis (spine). A toddler either has a humpback from birth or is predisposed to developing one due to abnormal medical conditions. There will be anguish and distortion because of the extreme humpback. Kyphosis, a spinal curve that denotes an excessively rounded back, can emerge from stress, infectious disease, aberrant development, genetic disease, and iatrogenic disease on occasion [4]. How Kyphosis is handled depends on several factors, including age, the cause of the curve, and its effects. Kyphosis disease in children should be more thoroughly diagnosed at a young age to prevent unusual spinal and vertebral problems. It is more crucial to recognize that the diagnosis of Kyphosis disease in children at an early stage may prevent issues with deformed spinal vertebrae [5]. The Mayo Clinic has more information on Kyphosis at (https://www.mayoclinic.org/diseases-conditions/kyphosis/symptoms-causes/syc-20374205, accessed on 1 December 2022). Medical diagnosis is a vital yet difficult task that must be conducted correctly and consistently; therefore, automating it would be advantageous [6].

Hyperparameter tuning is finding the perfect model architecture by adjusting the parameters that determine the model architecture, called Hyperparameters. Hyperparameter adjustment is essential to govern ML’s or DL’s general behavior [7]. A model parameter known as a “Hyperparameter” is one whose value affects learning and cannot be inferred from training data. Before beginning the process of training and learning the model, Hyperparameters are externally specified. Finding a set of ideal Hyperparameter values for a learning algorithm and using this tuned algorithm on any data set is called Hyperparameter tuning. A Hyperparameter governs the learning process, and as a result, the values of these parameters directly impact other model parameters such as weights and biases, which in turn affect how well the model performs. By tuning these Hyperparameters, any ML or DL model’s accuracy is frequently increased [8].

By comparing and using various ML algorithms, alternative medical practitioners have already done a respectable job detecting and forecasting diseases. According to Chatter et al., Kyphosis must be treated as some cases are congenital and some are iatrogenic, depending on the patient’s needs and current condition [4]. In [5], Artificial Neural Networks (ANN), Support Vector Machine (SVM), and Random Forest (RF) were utilized to recognize and predict Kyphosis disease. After applying grid search strategies, they found that the ANN performed the best, with 86.42% and 85.19% supporting 10-fold and 5-fold cross-validations, respectively. Singla’s group examined the relationship between front postural stability, round shoulders, and progressive Kyphosis disease [7].

According to [9], artificial intelligence and ML have significantly advanced the field of spine research. In this review, the author covered several decision-support systems, computer-aided diagnoses, and issues. ML performed brilliantly and showed great promise in spine research [10]. ML could assist clinical staff in improving patient care, boosting output, and reducing unfavorable outcomes. ANN is frequently utilized in diagnosis and care system claims [11]. In the study [12], they covered how to forecast the clinical expenses of spinal fusion using the Naive Bayes (NB), SVM, Logistic Regression (LR), Decision Tree (DT), and RF models. They concluded that the RF model performed the best in terms of prediction, with an accuracy of 84.30% compared to the other methods.

Abdullah et al. [13] employed the RF and K-Nearest Neighbors (KNN) models to find spinal abnormalities. They found that, with an accuracy of 85.32%, the KNN model outperformed the RF model. In work [14], they use the SVM, LR, and bagged SVM and LR models, all of which are openly available in the Kaggle repository. The outcomes demonstrated that on the test dataset, SVM, LR, and bagged SVM and LR all performed similarly well, with an accuracy of 86.96%. However, SVM’s superior recall value and miss rate set it apart from the competitors.

Many people undergo Kyphosis surgeries, although the illness may persist after the operation. Therefore, the issue is to predict whether or not the patient will still have Kyphosis after the treatment based on the patient’s many attributes. To classify the disease of Kyphosis, supervised learning systems are the main topic. Thus, in the present study, we apply Logistic Regression (LR), Naive Bayes (NB), Random Forest (RF), K-Nearest Neighbors (KNN), Support Vector Machine (SVM), and Deep Neural Network (DNN) algorithms based on a biomedical dataset to build models to predict the absence or presence of a Kyphosis disease based on historical healthcare and personal records of Kyphosis disease patients after they have undergone surgery.

Even though various models, including LR, NB, RF, KNN, SVM, ANN, etc., have been employed to predict the Kyphosis disease, regrettably, up until recently, it was virtually impossible to increase the accuracy. Therefore, by using these supervised models with Hyperparameter tuning, this work will examine Kyphosis disorder prediction using the dataset received from Kaggle. To choose the most efficient parameters, Hyperparameter tuning was done. While the calculations were being performed, certain variables were modified. The model’s performance is maximized using that set of Hyperparameters. Then, evaluation and comparison of the model outputs were performed.

The study examines the performance of supervised machine learning and deep learning (ML/DL) models in Kyphosis diagnosis across datasets to understand which model performs the best. Accuracy, recall or sensitivity, specificity, precision, balanced accuracy score, F1 score, and AUC-ROC score of all models, including the Hyperparameter tuning, were compared. In this study, we investigated the performance of tuned Hyperparameters and stratified K-fold cross-validation on the most popular ML and DL algorithms: LR, NB, RF, SVM, KNN, and DNN. This research aims to show the biomedical community how ML and DL algorithms have been used to categorize and forecast Kyphosis disease using biomedical data.

The complete article is organized as follows: Section 2 covers the materials and methods, Section 3 covers the implementation results, Section 4 covers the discussion, and Section 5 covers the conclusion and future works.

2. Materials and Methods

The foundation for implementing the advised plan is provided in this section. The preprocessing procedures are discussed in this part, followed by the final classification methods specific to each implementation. In the current study, models are developed using the LR, NB, RF, KNN, SVM, and DNN algorithms to identify people with Kyphosis who have previously received treatment for the condition and have personal records showing when they will need skilled surgery. The effects of the models are then assessed and investigated in detail.

2.1. Data Preparation

The dataset for Kyphosis was downloaded from Kaggle (https://www.kaggle.com/datasets/abbasit/kyphosis-dataset, accessed on 1 December 2022). The records of patients who underwent corrective surgery for Kyphosis problems are included in this dataset on the condition. Age, variety, and starts are three inputs, and one outcome of this choice of knowledge is Kyphosis. In

Table 1. Description of Dataset.

Characteristic	Explanation
Kyphosis	Whether or not the Kyphosis issue persisted after the procedure
Age	Monthly age of the patient
Number	Involved vertebrae within the procedure
Start	The count of the surgery’s first or highest vertebra that was affected

, the dataset’s credits are explained. The data was prepared using the Scikit-Learn library. To improve ML/DL model performance, the data were scaled using the standard scalar function of the Scikit-Learn library.

2.2. Classification Learning Algorithms

A computer program is taught to make new observations or classifications depending on the given data through machine learning, deep learning, and statistical category. The proposed training and result prediction models utilized the learning methods LR, NB, RF, SVM, KNN, and DNN.

Machine learning techniques such as logistic regression are used to solve classification issues. It is a predictive analysis technique that is based on the idea of probability. The output of a categorical dependent variable is predicted by logistic regression. The outcome must therefore have a discrete or absolute value. Rather than delivering the exact answer as 0 or 1, it provides the probability values that fall between 0 and 1. It might be either yes or no, 0 or 1, true or false, etc. Probability is determined in logistic regression using either the logistic function or the sigmoid function. The logistic function, a simple S-shaped curve, converts input into a value between 0 and 1. A logistic regression model mathematically predicts P(Y = 1) as a function of X [15,16], presented in Equation (1). Where $h\Phi \left(x\right)$ define the output of the logistic function, ${\beta }_{1 }$ is the slope, ${\beta }_{ 0}$ is the y-intercept, and X is the independent variable.

$$h\Phi \left(x\right)=\frac{1}{[(1+e^{-({\beta }_{0{ }^{+}}{\beta }_{1{ }^{) }}}\ast X} \mathrm{for} 0\le h\Phi \left(x\right)\ge 1$$

(1)

The NB algorithm is a supervised learning method that addresses classification problems based on the Bayes theorem. As a probabilistic classifier, it makes predictions based on the likelihood of an object [17,18]. The Bayes theorem, often called Bayes’ Rule, is a formula for determining how likely a hypothesis is given certain information. What happens will depend on the conditional probability. The Bayes theorem is expressed as defined in Equation (2).

$$P\left(A|B\right)=\frac{P\left(B|A\right)\ast P\left(A\right)}{P\left(B\right)}$$

(2)

Whereas the probability that a particular hypothesis (A) will happen is expressed as the posterior probability or P(A|B), P(B|A) stands for Likelihood Probability, which quantifies the likelihood that a given hypothesis is true based on the available data; the probability of a theory before viewing the evidence is known as priority probability, or P(A). P(B) stands for the probability of a marginal event.

The well-known ML algorithm RF is a component of the supervised learning approach. It is probably a classifier that uses a collection of selection bushes on different subsets of the provided dataset to increase its predicted accuracy. Instead of relying exclusively on one selection tree, it anticipates the final phrase output using predictions from each tree and most predictions [19,20]. Rather than relying on just one, it uses forecasts from all the trees to predict the result based on which predictions earned the most votes. To improve the predicted accuracy of the dataset, the random forest classifier, as its name suggests, “Contains several decision trees on diverse subsets of the input dataset and takes the average”.

The most popular use of the supervised, linear ML method known as SVM is the best fit for classification problems. This method makes it easier to solve the nonlinearity of the problem using the kernel method [21]. It aids in categorizing the data into two or more groups by using a boundary to distinguish between similar groups. Hence, let’s first update the formulas for how each data point is represented in space, and then consider the equation of a line that will aid in dividing comparable categories, and lastly, consider the formula for the distance between a data point and a line (a boundary separating the categories). SVM, one of the finest classifiers, displays some linearity. It can handle non-linear situations using a non-linear basis function and is backed by solid mathematical intuition [22]. In categories of n-dimensional space, the SVM rule aims to establish a single decision boundary or line. This will enable us to swiftly assign novel data to the appropriate class over the long run [23].

One supervised learning method that is widely used to categorize data depending on how its neighbors are ranked is the K-Nearest Neighbor algorithm. It keeps track of all the available instances and classifies new cases based on a similarity score. The number of nearest neighbors to be considered when choosing the winner by majority vote is indicated by the KNN parameter. Because it is a “lazy learner”, we can apply it when the dataset is labeled, noise-free, and small. Finding the closest point from the given data set to the input point is the definition of “finding nearest neighbors”. Using the majority votes of its K neighbors, the algorithm classifies new cases by storing all existing cases (test data). Data points are first converted into their mathematical values before employing KNN (vectors). The algorithm operates by measuring the separation between these points’ mathematical values. The distance between each data point and the test data is calculated, and the likelihood that the facts are similar to the test data is estimated. For classification, the most likely set of points is utilized. The distance functions that can be used are the Hamming, Minkowski, and Euclidean distances. The KNN approach only saves the dataset during the preparation phase, and when it receives new data, it organizes that data into a collection that is abundant and equal to the incoming information [24].

A deep neural network has many layers and neurons in each layer. When the output of one layer serves as the input for another layer in the forward direction, all layers in a deep neural network will alternately turn on and off. In a deep network, there are three types of data flow: input, output, and sequential [25,26]. The difference between an ANN and a DNN is that the former only comprises one input layer, one output layer, and perhaps one hidden layer. However, the latter ought to have more hidden layers [2,27]. The only neural networks we utilize are feed-forward ones. Three hidden layers have been used in the proposed work to extract a high-quality feature from the dataset.

The Keras model was used to implement the DNN model. A grid of potential discrete Hyperparameter values has been constructed to fit the model. We keep track of the model output for each set and then choose the combination that has delivered the best outcome. The DNN models have recently become quite popular because of their exceptional capacity to learn the underlying structure of the input data vectors and the non-linear input-output mapping [25,28,29].

2.3. Various Performance Measures

Table 2. Confusion Matrix.

Actual	Prediction
		0	1
	0	TN	FP
	1	FN	TP

, with the dimensions “actual” and “predicted”, as well as the scales “True Positives (TP)”, “True Negatives (TN)”, “False Positives (FP)”, and “False Negatives (FN)”, makes up the confusion matrix. The following performance measures assess predictions using ML and DL models for specified categorization problems [5,30,31]. The range of correct predictions is created as a quantitative relation of all made predictions; therefore, classification accuracy should be described. It is equal to (TP + TN)/(TP + FP + FN + TN). Precision, used in document retrievals, is also explained by the diversity of accurate documents that ML and DL models deliver. Precision is equal to (TP + FP)/TP. Because it indicates how successfully a model can recognize positive samples, recall is also examined. The recall equals (TP + FN)/TP.

Specificity is also reviewed to assess a model’s capacity to forecast true negatives for each available category. Specificity is defined as (TN + FP)/TN. The F1 score is calculated using the recall and precision weighted average. The F1 score is influenced equally by recall and precision. It equals 2 × (recall × precision)/(recall + precision). The AUC-ROC statistic measures a model’s ability to identify the categories. Balanced accuracy deals with unbalanced datasets in binary and multiclass classification problems. It is outlined because the average recall was found in each category.

2.4. Proposed Approach

This study illustrates the interaction between the functional design of the proposed predictive model and other significant DL and ML applications, the tuning and control of Hyperparameters, and model testing using K-fold cross-validation. The proposed system has classified people with Kyphosis sickness and healthy individuals. Different ML and DL models for Kyphosis sickness were examined for their results. The design used DL models, DNN, and popular ML models, LR, NB, RF, SVM, and KNN. Figure 1 shows the system’s framework in action.

A dataset for Figure 1 associated with the Kyphosis sickness of the patients was chosen. During the preparation stage, all inconsistencies that might have existed in the dataset when the data was acquired must be eliminated. The next step was to select the testing mode and the classification methods used during implementation. The previously described classification algorithms can now be put to use with the aid of Hyperparameter tuning methods. A model can be trained using existing data, making it capable of fitting the model parameters. Other sensible parameters, referred to as Hyperparameters, cannot be immediately learned using the standard training procedure. Sometimes they are set before the training process itself starts. These factors represent categorical model requirements, such as how complex the model is or how quickly it should learn [32]. The model’s performance can be maximized using that set of Hyperparameters, which minimizes a predetermined loss function and results in better results with fewer errors. When there is a lack of information or information that is not evenly distributed, the K-fold is often used. Sampling also affects K-fold cross-validation. When tackling classification challenges with imbalanced class distributions, stratified K-fold became more popular than K-fold. As a result, stratified K-fold cross-validation was used in the current study to evaluate the models.

To determine which algorithm is the best and delivers the highest level of accuracy when forecasting the outcome, the study will finally assess and contrast each of the produced algorithms. The main steps in the proposed model are summarized below:

(a)

The dataset for the Kyphosis disease was obtained from Kaggle. This dataset on Kyphosis contains information on patients who underwent corrective surgery for the disease. The Kyphosis data set has 81 rows and 4 columns regarding adolescents who underwent spinal fusion surgery. The dataset consists of one outcome, Kyphosis, and three inputs, age, variety, and start. These 81 samples are then divided into a training set, a validation set, and a test set using a ratio of 6:1:2.

(b)

The data were preprocessed using the Scikit-Learn program. The label encoder was used to divide the Kyphosis region into 0s and 1s. To help the ML and DL models work better, the data were scaled using the Standard Scaler function of the Sklearn library.

(c)

A model is trained to produce new observations or classifications based on the data it is provided through the classification process in machine learning and deep learning. The proposed models used the LR, NB RF, SVM, KNN, and DNN algorithms for training and result prediction.

(d)

Certain important parameters, known as Hyperparameters, are difficult to learn using conventional training. These parameters categorize the model’s fundamental features, such as complexity or learning rate. Hyperparameter tuning was used in this study to select the most efficient parameters. With that particular collection of Hyperparameters, the model’s performance is maximized, a preset loss function is minimized, and better outcomes with fewer errors are obtained.

(e)

When one lacks sufficient or evenly distributed information, the K-fold method is often used. Sampling also harms K-fold cross-validation. When tackling classification challenges with imbalanced class distributions, stratified K-fold became more popular than K-fold. As a result, stratified K-fold cross-validation was used in the current study to evaluate the models.

(f)

The study will ultimately evaluate and compare each of the developed algorithms to identify which one is the best and gives the highest level of accuracy when projecting the outcome. Predictions for the given classification problem are evaluated using machine learning and deep learning models utilizing the performance metrics accuracy, recall or sensitivity, specificity, precision, a balanced accuracy score, F1-score, and an AUC-ROC score.

3. Implementation and Results

Version 3.6.9 of the Python programming language was used to implement the models. The DNN model was applied with the aid of the Keras model. DL is the foundation for the deep multilayer perceptron architecture with regularization and dropout used in the developed DNN learning model. By making an analytical inquiry on the selected dataset and using ML/DL models with LR, NB, RF, SVM, KNN, and DNN algorithms supporting stratified K-fold (K = 5 and 10) cross-validation, the outcomes of the examination effort have been reported in this section. The results of the ML and DL models were presented for model evaluation and comparison after Hyperparameter tuning with stratified K-fold cross-validation was performed.

The classification accuracy of the LR model using stratified five-fold cross-validation is shown in Figure 2. It demonstrates fold-3′s superior accuracy. This model validated the stratified 5-fold cross-validation with a mean accuracy of 81.47% and a standard deviation (SD) of 0.0398. After Hyperparameter tuning, the best parameter was chosen at max_iter = 500 and solver = Newton-Cg using 5-fold stratified cross-validation.

The stratified ten-fold cross-validation results of the LR model’s classification accuracy are shown in Figure 3. It demonstrates that fold-1 has the best accuracy. The 10-fold stratified cross-validation was determined to be validated by the LR model’s 82.64% mean accuracy at the time and SD of 0.0625. After Hyperparameter tuning, the best parameter was chosen with max_iter = 1000 and solver = Newton-Cg by stratified 10-fold cross-validation.

The classification accuracy of the NB model using stratified five-fold cross-validation is shown in Figure 4. It demonstrates fold-3′s superior accuracy. This model validated the stratified 5-fold cross-validation with a mean accuracy of 81.47% and a standard deviation (SD) of 0.0398. After Hyperparameter tuning, the best parameter of the Gaussian NB classifier was chosen at prior = none and var_smoothing = 1 × 10⁻⁶ using 5-fold stratified cross-validation.

The stratified ten-fold cross-validation results of the NB model’s classification accuracy are shown in Figure 5. It demonstrates that fold-1 has the best accuracy. The 10-fold stratified cross-validation was determined to be validated by the NB model’s 82.64% mean accuracy at the time and SD of 0.0625. After Hyperparameter tuning, the best parameter of the Gaussian NB classifier was chosen at prior = none and var_smoothing = 1 × 10⁻⁶ using 5-fold stratified cross-validation.

The classification accuracy of the RF model using stratified five-fold cross-validation is shown in Figure 6. It demonstrates fold-3′s superior accuracy. This model validated the stratified 5-fold cross-validation with a mean accuracy of 85.22% and a standard deviation (SD) of 0.0485. After Hyperparameter tuning, the best parameter was chosen at n_estimators = 400 and criterion = Gini using 5-fold stratified cross-validation.

The stratified ten-fold cross-validation results of the RF model’s classification accuracy are shown in Figure 7. It demonstrates that fold-5 has the best accuracy. The 10-fold stratified cross-validation was determined to be validated by the RF model’s 83.89% mean accuracy at the time and SD of 0.0982. After Hyperparameter tuning, the best parameter was chosen with n_estimators = 150 and criterion = Gini by stratified 10-fold cross-validation.

The classification accuracy of the SVM model using stratified five-fold cross-validation is shown in Figure 8. It demonstrates fold-3′s superior accuracy. The stratified 5-fold cross-validation was supported by this model, which had a mean accuracy of 85.22% and an SD of 0.0839. Kernel = rbf, Gamma = 0.1, and C = 3 were chosen as the optimal parameters following Hyperparameter tuning using stratified 5-fold cross-validation.

The classification accuracy of the SVM model using stratified ten-fold cross-validation is shown in Figure 9. It demonstrates that the accuracy of fold-6 is the highest. This model was then discovered to support stratified 10-fold cross-validation with a mean accuracy of 85.14% and an SD of 0.0940. Using stratified 10-fold cross-validation, the optimal Hyperparameter values were kernel = rbf, gamma = 0.1, and C = 8.

The classification accuracy of the KNN model using stratified five-fold cross-validation is shown in Figure 10. It demonstrates that folds 3 and 5 have the most remarkable accuracy. This model supported the stratified 5-fold cross-validation with a mean accuracy of 85.22% and an SD of 0.0740. After Hyperparameter tuning using stratified 5-fold cross-validation, the optimal parameters were determined at n_Neighbor = 9 and p = 2.

The classification accuracy of the KNN model using stratified five-fold cross-validation is shown in Figure 11. It demonstrates that the highest accuracy folds are 2, 4, 5, 6, 7, 9, and 10. The KNN model at the time was discovered to have a mean accuracy of 84.03% and an SD of 0.0535 that supported the 10-fold stratified cross-validation. The best parameters were determined to be n_Neighbor = 11 and p = 2, given Hyperparameter tuning by 10-fold stratified cross-validation.

The classification accuracy of the DNN model using stratified five-fold cross-validation is shown in Figure 12. It demonstrates fold-3’s superior accuracy. This model, which had a mean accuracy of 87.72% and an SD of 0.6666, validated the stratified 5-fold cross-validation. With the following settings: learning rate = 0.0005, epochs = 100, batch size = 10, dropout = 0.2, kernel initialize = “uniform”, inputs = 60, 35, and 25 neurons in 3 hidden layers, severally rectified linear unit activation functions, and sigmoid activation were selected at the final layer with one neuron and the Adam optimizer, the best Hyperparameter tuning parameters were determined.

The classification accuracy of the DNN model using stratified ten-fold cross-validation is shown in Figure 13. It demonstrates that the accuracy of fold-10 is the highest. It was found that this model had a mean accuracy of 87.64% and an SD of 0.0561. The deviation supported the stratified 10-fold cross-validation. With the following settings: learning rate = 0.0005, epochs = 50, batch size = 20, dropout = 0.2, kernel initialize = “uniform”, inputs = 30, 30, and 25 neurons in 3 hidden layers, severally rectified linear unit activation functions, and sigmoid activation were selected at the final layer with one neuron and the Adam optimizer, the best Hyperparameter tuning parameters were determined.

Simulation Results

We conducted a comparative study of every algorithm utilized in this study. The study’s objective was met by creating ML and DL algorithms to recognize Kyphosis disease. Using ML/DL models with learning algorithms; stratified K-fold cross-validation with Hyperparameter tuning has been investigated. Each test underwent 5- and 10-fold cross-validations. The most widely accepted K-fold cross-validation management is often the 5- or 10-fold control.

As indicated in

Table 3. The comparative performance estimation using different ML/DL models based on the stratified five-fold cross-validation.

ML/DL Models	Classification Accuracy in % (with SD)	Recall or Sensitivity	Specificity	Balanced Accuracy Score	Precision	F1-Score	AUC-ROC Score
LR	81.47 (SD = 0.0398)	0.30	0.95	0.63	0.68	0.36	0.63
NB	81.47 (SD = 0.0398)	0.48	0.91	0.70	0.53	0.44	0.70
RF	85.22 (SD = 0.0485)	0.48	0.95	0.72	0.70	0.51	0.72
SVM	85.22 (SD = 0.0839)	0.48	0.95	0.72	0.68	0.51	0.72
KNN	85.22 (SD = 0.0740)	0.53	0.94	0.74	0.63	0.52	0.74
DNN	87.72 (SD = 0.0666)	0.62	0.95	0.79	0.90	0.65	0.79

and represented in Figure 14, the LR model reached 81.47% accuracy with an SD of 0.0398 by undertaking 5-fold stratified cross-validation, followed by the NB model at 81.47% accuracy with an SD of 0.0398, the RF model at 85.22% accuracy with an SD of 0.0485, the SVM model at 85.22% accuracy with an SD of 0.0839, the KNN model at 85.22% accuracy with an SD of 0.0740, and the DNN model at 87.72% accuracy with an SD of 0.0666.

As indicated in Table 3 and Figure 15, the LR model achieved 82.64% accuracy with an SD of 0.0625 by doing 10-fold stratified cross-validation, the NB model gained 82.64% accuracy with an SD of 0.0625, the RF model achieved 83.89% accuracy with an SD of 0.0982, the SVM model gained 85.14% accuracy with an SD of 0.0940, the KNN model achieved 84.03% accuracy with an SD of 0.0535, and the DNN model achieved 87.64% accuracy with an SD of 0.0561. In addition to classification accuracy, Table 3 and

Table 4. The comparative performance estimation using different ML/DL models based on the stratified ten-fold cross-validation.

ML/DL Models	Classification Accuracy in % (With SD)	Recall or Sensitivity	Specificity	Balanced Accuracy Score	Precision	F1-Score	AUC-ROC Score
LR	82.64 (SD = 0.0625)	0.40	0.96	0.68	0.45	0.39	0.68
NB	82.64 (SD = 0.0625)	0.50	0.92	0.71	0.58	0.48	0.71
RF	83.89 (SD = 0.0982)	0.35	0.96	0.65	0.60	0.43	0.65
SVM	85.14 (SD = 0.0940)	0.55	0.94	0.75	0.68	0.54	0.75
KNN	84.03 (SD = 0.0535)	0.50	0.94	0.72	0.55	0.46	0.72
DNN	87.64 (SD = 0.0561)	0.55	0.97	0.76	0.70	0.57	0.76

and Figure 16 and Figure 17 give a comparative analysis of key performance characteristics for the ML and DL models based on stratified 5-fold and 10-fold cross-validation, respectively.

4. Results and Discussion

Based on LR, NB, RF, KNN, SVM, and DNN models, the effects of Hyperparameter tuning on the dataset for Kyphosis disease have been thoroughly examined. The ML/DL model’s performance was maximized using that set of Hyperparameters. We compared these models using several metrics, including accuracy, recall or sensitivity, specificity, and precision. We also considered F1 and AUC-ROC scores. According to our findings, the LR model used 5-fold cross-validation to obtain 81.47% accuracy with an SD of 0.0398, a recall value of 0.30, a specificity of 0.95, a balanced accuracy score of 0.63, a precision of 0.68, an F1-score of 0.36, and an AUC-ROC score of 0.63. With a 0.48 recall value, 0.91 specificities, a 0.70 balanced accuracy score, 0.53 precision, a 0.44 F1-score, and 0.70 AUC-ROC scores, the NB model had an accuracy rate of 81.47% with an SD of 0.0398.

With a recall value of 0.48, a specificity of 0.95, a balanced accuracy score of 0.72, a precision of 0.70, an F1-score of 0.51, and an AUC-ROC score of 0.72, the RF model had an accuracy rate of 85.22% with an SD of 0.0485. With a 0.48 recall value, 0.95 specificities, a 0.72 balanced accuracy score, 0.68 precision, a 0.51 F1-score, and a 0.72 AUC-ROC score, the SVM model had an accuracy rate of 85.22% with an SD of 0.0839. With a 0.53 recall value, 0.94 specificities, a 0.74 balanced accuracy score, 0.63 precision, a 0.52 F1-score, and a 0.74 AUC-ROC score, the KNN model had an accuracy rate of 85.22% with an SD of 0.0740. On the other hand, the DNN model achieved 87.72% accuracy with an SD of 0.0666, a 0.62 recall value, 0.95 specificities, a 0.79 balanced accuracy score, 0.90 precision, a 0.65 F1-score, and 0.79 AUC-ROC scores.

Similarly, with 10-fold cross-validation, the LR model achieved 82.64% accuracy with an SD of 0.0625, a 0.40 recall value, 0.96 specificities, a 0.68 balanced accuracy score, 0.45 precision, a 0.39 F1-score, and 0.68 AUC-ROC scores. With a 0.50 recall value, 0.92 specificities, a 0.71 balanced accuracy score, 0.58 precision, a 0.48 F1-score, and a 0.71 AUC-ROC score, the RF model had an accuracy rate of 83.89% with an SD of 0.0982. With a 0.35 recall value, 0.96 specificities, 0.65 balanced accuracy score, 0.60 precision, 0.43 F1-score, and 0.65 AUC-ROC scores, the RF model had an accuracy rate of 83.89% with an SD of 0.0982. With a 0.55 recall value, 0.94 specificities, 0.75 balanced accuracy score, 0.68 precision, 0.54 F1-score, and 0.75 AUC-ROC scores, the SVM model had an accuracy rate of 85.14% with an SD of 0.0940. With a 0.50 recall value, 0.94 specificities, a 0.72 balanced accuracy score, 0.55 precision, a 0.46 F1-score, and 0.72 AUC-ROC score, the KNN model had an accuracy rate of 84.03% with an SD of 0.0535. At the same time, the DNN model had an 87.64% accuracy rate with an SD of 0.0561 and had a 0.55 recall, 0.97 specificities, a 0.76 balanced accuracy score, 0.70 precision, a 0.57 F1-score, and 0.76 AUC-ROC scores.

Overall, the Hyperparameter-tuned DNN models excelled over the other models. The DNN model was shown to perform better than all other models in classification accuracy, with accuracy validated by stratified 10-fold and 5-fold cross-validation of 87.64% with SD of 0.0561 and 87.72% with SD of 0.0666, respectively. Regarding classification accuracy, recall or sensitivity, specificity, a balanced accuracy score, precision, an F1 score, and an AUC-ROC score that supported stratified 5- and 10-fold cross-validation, the DNN model outperformed a wide variety of other models, as shown in Table 3 and Table 4, respectively. This model likewise received the highest score based on these additional performance metrics.

If we compare our results with the previous work reported by others, we can find some significant accuracy improvements in our proposed model. In Dankwa and Zheng’s work [5], the ANN (3-6-6-1) model surpassed all the other models by attaining 85.19% and 86.42% based on the 5-fold and 10-fold cross-validation, respectively. Our proposed Hyperparameter-tuned DNN model outperformed all other ML models by achieving 87.72% and 87.64% with stratified 5-fold and 10-fold cross-validation, respectively. Our proposed work improves the accuracy by 2.53% and 1.22%, respectively, with the same dataset used in both studies. Compared to ANN, the numerous layers in DNN enable models to learn complicated features and carry out more demanding computational tasks more efficiently.

Furthermore, because DNNs can learn faster, they are more effective in training time. In comparison to other methods presented in this work and the literature, the DNN model is the best ideal for the diagnosis of Kyphosis. This is because DL algorithms can eventually learn from their own mistakes. It can check the accuracy of its estimations or results and make the required corrections. Traditional machine learning algorithms, on the other hand, require varying levels of human input to assess output accuracy.

5. Conclusions

DL methods are significantly more efficient in Kyphosis illness identification and diagnosis than traditional computing technologies. The DNN model outperformed all other ML models regarding classification accuracy, achieving 87.64% and 87.72% with stratified 10-fold and 5-fold cross-validation, respectively. These results exceed the most recent analysis in various academic publications. If we compare the experimental results with the literature with the same dataset, our proposed work improves the accuracy by 2.53% and 1.22% based on the 5-fold and 10-fold cross-validation, respectively. The DNN model is the best suitable for the diagnosis of Kyphosis in comparison to other methods presented in this work and the literature. Compared to ANN, the numerous layers in DNN enable models to learn complicated features and carry out more demanding computational tasks more efficiently. Furthermore, DNN is more effective in training time as they can learn faster.

It is advised that when a patient has a clinical procedure, the DNN model be trained to detect and foresee Kyphosis disease. Medical experts can use this study’s findings to correctly predict if a patient will still have Kyphosis after surgery. We propose that DL should be adopted and utilized as a crucial and necessary tool throughout the broad range of resolving biological queries. Future research can explore more DL methods to increase the accuracy of Kyphosis disease prediction.