Evaluation and Optimization of Biomedical Image-Based Deep Convolutional Neural Network Model for COVID-19 Status Classification

Abstract

Accurate detection of an individual’s coronavirus disease 2019 (COVID-19) status has become critical as the COVID-19 pandemic has led to over 615 million cases and over 6.454 million deaths since its outbreak in 2019. Our proposed research work aims to present a deep convolutional neural network-based framework for the detection of COVID-19 status from chest X-ray and CT scan imaging data acquired from three benchmark imagery datasets. VGG-19, ResNet-50 and Inception-V3 models are employed in this research study to perform image classification. A variety of evaluation metrics including kappa statistic, Root-Mean-Square Error (RMSE), accuracy, True Positive Rate (TPR), False Positive Rate (FPR), Recall, precision, and F-measure are used to ensure adequate performance of the proposed framework. Our findings indicate that the Inception-V3 model has the best performance in terms of COVID-19 status detection.

1. Introduction

The coronavirus disease (COVID-19) [1,2,3,4] is a highly contagious viral disease. Older people and those with comorbidities such as heart and lung problems, high blood pressure, diabetes, and cancers are more susceptible to serious illness or death due to the COVID-19 infection. However, any person may become severely sick or die due to this disease. Common symptoms of the disease include fever, fatigue, and dry cough, headaches, conjunctivitis, sore throat, nasal congestion, muscle pain, skin rash, loss of taste, vomiting, diarrhea, dizziness, irritation, anxiety, nervousness, diminished consciousness, depression, sleep disorder, stroke, brain inflammation, and nerve damage. The indications of severe coronavirus disease include shortness of breath, loss of appetite, chest pain, and high temperature (above 100 °F). COVID-19 is also responsible for respiratory upheaval and weakens the immune system [5].

The majority (approximately 80%) of the patients who have COVID-19 symptoms can overcome the illness without hospitalization. However, approximately 15% of the patients become severely sick and need oxygen and 5% of the patients require intensive care. The symptoms start on average 5–6 days after infection and can last from 1 to 14 days. According to the Centers for Disease Control and Prevention (CDC), people who have contact with coronavirus virus should remain in isolation and quarantine for at least 14 days to potentially stop the spread of the virus, especially when testing is not obtainable. The COVID-19 pandemic has caused a severe setback as the medical infrastructure is not capable to adequately handle the high patient counts and effectively combat this deadly disease. Reverse Transcription-Polymerase Chain Reaction (RT-PCR) is an accurate test which can identify whether a patient is infected by SARS-CoV-2 or not. Though, the test results are only available after 1–2 days, this delay in treatment can lead to loss of life or further spread of the diseases. Moreover, in rural areas the RT-PCR test is not yet widely available. In response, medical practitioners have opted for various other tests such as the Rapid Antigen Test (RAT) [6], chest X-rays, and CT scans in hopes of timely detection of the disease.

The Convolutional Neural Network (CNN) [7,8,9,10,11] is regarded as a deep neural network [12,13] that exhibits robust power in object detection, computer vision, image classification, and segmentation. It has recently been widely applied in order to detect important features from sequencing, gene expression, functional genomics, cell type context, and single cell data in biological and biomedical research [14,15,16,17,18]. RT-PCR tests have a large window of time from administration to results and possess low sensitivity and are therefore unable to provide desired results quickly enough in a pandemic situation such as COVID-19 [19]. Due to the unavailability of special kits needed by this test in some remote areas, RT-PCR tests cannot be easily implemented in COVID-19 detection in such locations [20]. Many existing research studies for COVID-19 detection use machine learning (ML) methods [21,22], pervasive computing, and the consideration of different symptoms of COVID-19 along with non-image-based datasets [23]. Some research article [24] also focused on the forecasting of COVID-19. But the development of CNN methods integrated with image analysis of CT scan and chest X-ray images [25] has been found to be more efficient in detecting COVID-19 status. CNN-based methods [26,27] and weighted extension algorithms have also been applied to detect virus integration sites in the host genomes as well as the trajectory of disease progression in COVID-19 patients [28,29]. In image data analysis, CNN [30] is seen as a tool containing powerful learning capabilities of fetching critical features from the images and discovering important patterns from image set [31,32]. In this paper, we employ different CNN architectures for detecting COVID-19 which uses CT scan and chest X-ray images.

The paper is organized in the following way. Section 2 consists of fundamental concepts and a literature review of existing research regarding COVID-19 detection. Section 3 provides dataset descriptions and Section 4 describes the architecture of CNN. Section 5 goes over the detailed technical aspects of the methods used in this research work. Section 6 describes the results with performance analysis and Section 7 contains the concluding remarks.

2. Related Works

In general, deep learning has expanded into the field of medical science and especially in image-based medical diagnosis. Convolutional neural networks have outperformed other conventional methods of image analysis [33,34,35,36]. Recently, many of these studies are concerned with COVID-19 detection.

COVID-19 detection was implemented by statistical modeling conducted on mined relevant data [37,38,39]. The authors in [40] developed a mathematical model for forecasting coronavirus rates in the population. Such research studies are unable to provide prolific results with higher accuracy in the case of COVID-19 detection. Additionally, CNN accompanied by the CT scan images and chest X-ray images were applied in order to detect COVID-19. Areej A. Wahab Ahmed Musleh and Ashraf Yunis Maghari [34] experimented with a convolutional neural network-based algorithm named CheXNet developed by Stanford University for diagnosing and detecting pneumonia from the chest X-rays. They used 550 chest X-ray images from the Kaggle website and achieved a prediction accuracy of 89.7%. Huda Khaloofi et al. [41] investigated various machine learning approaches to predict COVID-19 using online datasets. They explained the life cycle as well as the spread of COVID-19 disease in Saudi Arabia with associated performance analysis. Brunese et al. [42] employed COVID-19 detection by dint of a deep learning algorithm which was comprised of three phases. According to the authors of this research study, the first phase dealt with the existence of pneumonia from chest X-rays whereas the second phase was applied in order to discriminate COVID-19 from pneumonia. The third phase was concerned with the detection of the body region affected by COVID-19. Their research could achieve 98% accuracy for COVID-19 detection. D. Haritha et al. [43] predicted COVID-19 disease efficiently using the VGG model based on transfer learning. They developed a system for identifying coronavirus disease which used chest X-ray photographs. Suraj Bodapati et al. [44] described a forecasting methodology for the COVID-19 outbreak. Their research study used Long Short-Term Memory (LSTM) models to determine several verified cases and the various death cases. Shah et al. in [45] proposed a model named as CTnet-10 with a view to diagnose COVID-19 using CT scan images. Their proposed CTnet had an accuracy 82.1% and required the lowest amount of time for predicting COVID-19 compared to the prediction time taken by VGG and Inception V3. A Keles et al. in [46] developed inference engines, namely COV19-ResNet, with an accuracy of 97.61% and COV19-CNN having an accuracy 94.28%, both using ResNet for COVID-19 detection. The authors of [47] adopted Artificial Intelligence (AI)-based methods for identifying and analyzing COVID-19 cases. They performed their study using pulmonary CT images. I. D. Apostolopoulos and T. A. Mpesiana [48] applied the approach of transfer learning for identifying COVID-19 disease with an accuracy of 96.78%. The authors in [49] used chest X-ray photographs to perform simulation experiments. They employed the concept of transfer learning and reported a high degree of accuracy, sensitivity, and specificity (96.78%, 98.66%, and 96.46%, respectively) to extract an important biomarker concerning COVID-19 and predict COVID-19. The authors in [50] applied the approaches of deep learning on chest X-ray images and also incorporated the principle of fine tuning to train the model for the purpose of detecting COVID-19. In this research article, they obtained a maximum of 92.6% average accuracy in the process of COVID-19 detection. Wang et al. [25] designed model COVID-Net considering chest X-ray images and this designed model achieved positive predictive value up to 98.9%. The authors of [51] proposed a model endowed with the principles of CNN to achieve an average of 96.8% in the detection of COVID-19. Jain et al. [52] achieved 97.77% accuracy in COVID-19 detection by deep learning on chest X-ray images. A hybrid deep neural network along with CT images and chest X-ray images for the prediction of COVID-19 was mentioned with an accuracy of up to 99% [53].

Existing research studies apply the principles of CNN but only focus heavily on the accuracy for providing acceptable results in the context of COVID-19 detection. However, accuracy is not the only reliable and appropriate performance measuring metric as it may yield misleading outcomes in accordance with the variation in the number of samples [54]. On the other hand, the F-measure is considered a more reliable and effective performance evaluating metric than accuracy for comparing classifiers due to its ability to provide an improved insight into the operations of classifiers [55,56,57,58,59]. RMSE and kappa statistic are powerful performance measuring metrics [60,61,62,63,64,65] and such metrics were not considered in previous papers to reveal the supremacy of their proposed models for COVID-19 detection. The proposed research work here deals with various powerful metrics such as F-measure, RMSE, and kappa statistic in order to detect COVID-19. Consideration of the aforesaid metrics is an integral aspect of this work. Achievement of good recall value indicates lower false negatives which in turn proves that our model is free from over-fitting. This indeed is the major novelty of our proposed work. Employment of the principles of hypermeter tuning, testing on an ample amount of dataset samples, and also the achievement of significant performance of our proposed framework in a lower number of epoch sizes are also important distinctive features of this work.

3. Dataset Description

This communication aims to develop and propose a generalized and robust model for the efficient detection of the disease. The benchmarking of the proposed methodology has been achieved using three different datasets.

The first dataset [66,67] used for validating this research work is comprised of 707 X-ray images and has three class labels: normal, COVID-19, and PNEUMONIA.

The second dataset [68] used in this study is Extensive COVID-19 X-ray and CT scan images and contains 5863 data values. The third dataset [69] provided by Mendeley Data consists of 16,490 images of both X-ray and CT scan images. Sample X-ray images and CT scan images acquired from such three benchmark imagery datasets [66,67,68,69] are displayed in Figure 1 and Figure 2 respectively.

4. Architecture of CNN Model

CNN models are the most used deep learning (DL) methods, specifically in computer vision-related applications. Since CNN possesses both an accurate discrimination ability and good feature generation, this study uses deep CNN-based classification models on Chest X-ray and CT scan images to identify COVID-19. The block diagram of CNN architecture for an input image (Chest X-ray or CT scan image) is presented in Figure 3 below.

A CNN model automatically learns spatial feature hierarchies (horizontal, vertical, lines, etc.) via backpropagation algorithm which uses multiple building blocks, such as convolutional layers, pooling layers, as well as fully connected layers. The different layers in the CNN model are:

A.

Convolutional Layer;

B.

Pooling Layer;

C.

Fully Connected Layer (FC layer).

The CNN architectural model usually consists of alternate layers of convolution and pooling accompanied by one or more FC layers at the end.

A. 

Convolutional Layer: The first layer in the CNN architecture is the convolutional layer. It consists of a set of convolutional kernels (i.e., filters) where each neuron performs as the kernel. Here, the mathematical operation of convolution is accomplished between the input image and a filter of a specific size M × M. Convolution operation of a feature matrix can be expressed as follows:

$${\mathrm{f}}_{\mathrm{d}}^{\mathrm{c}}\left(\mathrm{s},\mathrm{t}\right)={{\displaystyle \sum }}_{\mathrm{k}}{{\displaystyle \sum }}_{\mathrm{a},\mathrm{b}}{\mathrm{i}}_{\mathrm{k}}\left(\mathrm{a},\mathrm{b}\right).{\mathrm{e}}_{\mathrm{d}}^{\mathrm{c}}\left(\mathrm{m},\mathrm{n}\right)$$

(1)

where ${\mathrm{i}}_{\mathrm{k}}\left(\mathrm{a},\mathrm{b}\right)$ denotes an element $\left(\mathrm{a},\mathrm{b}\right)$ of the input image tensor ${\mathrm{I}}_{\mathrm{k}}$, which is element wise multiplied by ${\mathrm{e}}_{\mathrm{d}}^{\mathrm{c}}\left(\mathrm{m},\mathrm{n}\right)$ index of the cth convolutional kernel of the dth layer; here ${\mathrm{f}}_{\mathrm{d}}^{\mathrm{c}}\left(\mathrm{s},\mathrm{t}\right)$ denotes the (s, t) element of the feature matrix concerning cth kernel of dth layer. Finally, the output feature-map ${\mathrm{F}}_{\mathrm{d}}^{\mathrm{c}}$ of the cth convolutional operation on the dth neuron can be represented as:

$${\mathrm{F}}_{\mathrm{d}}^{\mathrm{c}}=[{\mathrm{f}}_{\mathrm{d}}^{\mathrm{c}}\left(1,1\right),\dots ,{\mathrm{f}}_{\mathrm{d}}^{\mathrm{c}}\left(\mathrm{s},\mathrm{t}\right),\dots ,{\mathrm{f}}_{\mathrm{d}}^{\mathrm{c}}\left(\mathrm{S},\mathrm{T}\right)]$$

(2)

In Equation (2), S denotes the total number of rows of the feature matrix and T denotes the total number of columns of the feature matrix.

The resulting output is called the feature map and it provides information about the image such as the corners and edges. The feature map is then fed to further layers to learn several other characteristics of the input image.

The selection of a particular activation function is critical in the decision-making process. There are many unique activation functions, such as ReLU, SWISH, MISH, tanh, sigmoid, maxout, etc., which can be utilized to attain non-linear grouping of features. Equation (3) defines the activation function of a convolved feature-map as follows:

$${\mathrm{T}}_{\mathrm{d}}^{\mathrm{c}}={\mathrm{g}}_{\mathrm{a}}\left({\mathrm{F}}_{\mathrm{d}}^{\mathrm{c}}\right)$$

(3)

In the above equation, ${\mathrm{F}}_{\mathrm{d}}^{\mathrm{c}}$ is the resulting output of a convolution network that is allotted to an activation function ${\mathrm{g}}_{\mathrm{a}}(.)$. It basically adds non-linearity and produces a transformed output ${\mathrm{T}}_{\mathrm{d}}^{\mathrm{c}}$ for the cth layer;

B. 

Pooling Layer: Generally, a pooling layer comes after a convolutional layer. The primary goal of this layer is to reduce the size of the feature map generated by convolution to decrease the computational costs. It is executed by reducing the connections between layers and separately operating on each feature map.

Basically, pooling is a local operation that adds up identical information in the region of the receptive field and results in the dominant response within this local region. This operation is described in Equation (4):

$${\mathrm{Z}}_{\mathrm{d}}^{\mathrm{c}}={\mathrm{g}}_{\mathrm{p}}\left({\mathrm{F}}_{\mathrm{d}}^{\mathrm{c}}\right)$$

(4)

Here, ${\mathrm{Z}}_{\mathrm{d}}^{\mathrm{c}}$ signifies the pooled feature-map of dth layer for cth input feature-map ${\mathrm{F}}_{\mathrm{d}}^{\mathrm{c}}$ and ${\mathrm{g}}_{\mathrm{p}}(.)$ outlines the type of pooling operation.

Different types of pooling functions such as max, average, etc., can be used in CNNs. In case of max pooling, the biggest element is selected from the feature map. The average pooling computes the average of the components in an image portion of predefined size. This layer generally connects the convolutional layer and the FC layer. Here, we have used the max pooling function;

C. 

Fully Connected Layer (FC Layer): This layer contains weights and biases associated with the neurons and is utilized to connect the neurons between two separate layers. These layers are normally located prior to the output layer in a CNN Architecture.

Here, the input image from the preceding layers is flattened and normally fed to the FC layer. The flattened vector then goes through a few more FC layers where the mathematical functions are applied. In this stage, the classification procedure is about to take place. In spite of having some mapping functions, several regulatory units such as dropout and batch normalization are also included to enhance CNN performance. Batch normalization basically unifies the distribution of feature-map values by assigning them to mean zero and unit variance, thereby refining the generalization of the neural network. However, dropout presents regularization within the neural network, which eventually progresses generalization by avoiding several units or connections randomly with a specific probability value.

5. Materials and Methods

This section explores the detailed description of our proposed framework regarding COVID-19 detection. This research study has designed a generalized detection framework, as shown in Figure 4, describing the entire process of this experiment.

The acquirement of the data images from the aforementioned imagery datasets (Dataset 1 (formed on X-ray images), Dataset 2 (formed on X-ray images), and Dataset 3 (formed on CT scan images)) denotes the commencement of the framework proposed in this research work. Such images are passed on to the pre-processing module of the proposed framework that is standardized and allows for the integration of the data from the various datasets to carry out various pre-processing mechanisms for improving the accuracy and performance of our proposed architecture. As a part of image pre-processing, data augmentation was employed in order to improve the amount of data available during training and to incorporate diversity into the training data that helps to overcome the problem of overfitting. The data augmentation technique in this work consists of flip augmentation and rotation augmentation which increase the number of data samples. The state of images after performing augmentation and transformation is shown in the following figures (Figure 5 and Figure 6).

The other preprocessing steps used during model building are reducing the size of images, cropping the image, flipping, rotating, and rescaling which helped in creating variances which lead towards building a stable end-model. After data pre-processing, dataset splitting commences and during this phase, the datasets are divided into two different parts—training datasets and test datasets. Each dataset is divided into training and test datasets using a 70%–30% distribution and K-fold cross-validation (K = 10).

The present research performs the tuning process of three existing models, namely VGG-19, ResNet-50, and Inception-V3 during the training process thereby producing superior results with a view to detect COVID-19. The VGG-19 is a 19-layer convolutional neural network architecture (including 16 convolution layers, 3 fully connected layers, 5 MaxPool layers, and 1 SoftMax layer). The prime objective of developing this network is to increase the number of hidden layers while sinking the number of parameters. In this architecture, a fixed kernel size of 3 × 3 is perceived, and as the progress along the network is achieved, the number of kernels continues enhancing in the hopes of a better feature. ResNet-50 containing 50 layers is a deep residual network. A residual function along with traditional CNN is used by ResNet-50 to solve the problem of gradient dispersion and accuracy degradation. If the process of the addition of a number of layers in our network is continued, during backpropagation, the model will not be able to spread the weight changes due to gradient explosion or dissipation. The ResNet-50 model comprises of five stages, each with a residual block. These residual blocks are responsible for keeping parameters as low as possible. This provides better training and testing errors even if the network depth increases and provides regularization. Hence, ResNet-50 is regarded as the most popular CNN architecture in terms of image classification. Inception-V3 architecture follows the concept of factorized convolutional which reduces computational efficiency. The reduction in the filter size helps in achieving a faster training speed thereby minimizing the number of parameters. This model includes auxiliary classifiers which act as regularizers and are employed to ensure convergence of very deep neural networks. An auxiliary classifier is added in between the layers during training and the losses incurred are added back to the main network. Inception-V3 is effective for smoothing labels of image and acquiring higher accuracy in image classification. As the proposed framework is represented for detecting COVID-19 via chest X-ray images and CT scan images, the training phase deals with such aforementioned models for image classification.

The training phase of this research work includes hyperparameter tuning in the existing models such as VGG-19, Resnet-50, and Inception V3 to achieve adequate results during the testing phase. The first parameter that was tuned is the number of neurons. Three different activation functions namely ReLu, Leaky Relu for avoiding the function from becoming saturated at 0, and the activation function softmax responsible for predicting a multinomial probability distribution were employed and the input values moving from a layer to another layer is kept changing according to the activation function. The two parameters, namely rho and epsilon in adaDelta, were tuned for performance efficiency. The tuning process also involves the usage of optimizers in order to modify the attributes of the neural network such as weights and learning rate thereby enhancing the proficiency of producing minimizing losses. Adam optimizer, momentum-based SGD, and adaDelta were applied while training VGG-19, Resnet-50, Inception-V3, and adam optimizer were found to be the best optimizers.

After the successful completion of the training phase, the performance of each of the trained classifiers incorporated in this research work were evaluated according to various statistical performance metrics including accuracy, kappa statistics, recall, F-measure, etc., and the comparative analysis among these classifiers provides enhanced results with respect to COVID-19 detection.

6. Results and Discussion

The classification models, namely, VGG-19, Inception V3, and ResNet-50, are trained employing the principle of tuning and then tested on three datasets using the Google Colaboratory (Tesla T4, CUDA Version: 11.2). The imagery benchmark datasets used in this research work are divided into two parts, one for training purposes and the other one for testing purposes. A 70%–30% distribution of each of these datasets was employed in this work. A 10-fold cross-validation technique for such datasets is also taken to evaluate the performance of each of such distinct classifiers which are tuned during the training phase. A variety of evaluation metrics are employed to measure the performance in the experiments, including Root-Mean-Square Error (RMSE) [70], Kappa Statistic [71], True Positive Rate (TP-Rate), recall, False Positive Rate (FP-Rate), accuracy, precision, and F-measure. In essence, accuracy, precision, True Positive Rate (TP-Rate), False Positive Rate (FP-Rate), Recall, and F-measure are derived from the confusion matrix [72] of the individual classifier.

It is to be noted that optimal performance for each of the trained classifiers is achieved after 40 epochs though this performance comparison scheme proposed in this research work provides test results of each of these classifiers for 10, 20, and 30 epochs. The Supplementary Materials contain the test results considering 10, 20, and 30 epochs. Performance analysis is comprised of two parts. The first part describes the performance comparisons of the classifiers developed on different evaluation measures such as accuracy, weighted kappa statistic, and root-mean-square error (RMSE). Meanwhile, the second part deals with performance comparisons of each individual classifier with respect to TP-Rate/Recall, FP-Rate, precision, and F-measure. Standard deviation is also presented in the test results. The detailed descriptions of these statistical metrics which are used as tools for measuring performance are provided below.

6.1. Root-Mean-Square Error (RMSE)

The RMSE assessment metric indicates a measure of the difference between the classifier prediction value and the target output value of the classification model. The RMSE system of measurement of a model implies the square root of the mean-squared inaccuracy:

$$RMSE=\frac{\sqrt{{{\displaystyle \sum }}_{k-1}^n{\left(e_{discovered, k}-e_{classifier,k }\right)}^2}}{n}.$$

(5)

Here, $e_{discovered}$ is the target value and $e_{classifier}$ is the predicted value for $\forall k$.

6.2. Kappa Statistic

The kappa statistic denoted by k is a popular evaluation metric in statistics. It indicates the relative measure of reliability within different judges or evaluators. The subsequent equation reckons the value of k as:

$$k=\frac{prob\left(O\right)-prob\left(C\right)}{1-prob\left(C\right)}$$

(6)

Here, $prob\left(O\right)$ is the probability of perceived agreements amongst the evaluators and $prob\left(C\right)$ is the probability of agreements expected by coincidence. If k = 1, then the evaluators have supported each other’s judgment. If k = 0, then the evaluators do not reach agreement.

Using 40 epochs, it is observed that the VGG-19 classifier is responsible for providing accuracy of 87.45%, 85.91%, and 88.76% as presented in

Table 1. Comparison of CNN models using dataset 1 for 40 epochs.

CNN Model	70%–30% Distribution			10-Fold Cross-Validation
	Accuracy (%) (sd)	RMSE (sd)	Weighted Kappa (sd)	Accuracy (%) (sd)	RMSE (sd)	Weighted Kappa (sd)
VGG-19	87.45 ($\pm 0.03)$	0.2285 ($\pm 0.03)$	0.8610 ($\pm 0.03)$	88.35 ($\pm 0.03)$	0.2195 ($\pm 0.03)$	0.8700 ($\pm 0.03)$
ResNet-50	94.38 ($\pm 0.04)$	0.1592 ($\pm 0.03)$	0.9303 ($\pm 0.04)$	97.61 ($\pm 0.03)$	0.1169 ($\pm 0.03)$	0.9726 ($\pm 0.03)$
Inception-V3	95.62 ($\pm 0.04)$	0.1468 ($\pm 0.03)$	0.9427 ($\pm 0.03)$	99.23 ($\pm 0.03)$	0.1107 ($\pm 0.03)$	0.9788 ($\pm 0.04)$

sd: standard deviation.

,

Table 2. Comparison of CNN models using dataset 2 for 40 epochs.

CNN Model	70%–30% Distribution			10-Fold Cross-Validation
	Accuracy (%) (sd)	RMSE (sd)	Weighted Kappa (sd)	Accuracy (%) (sd)	RMSE (sd)	Weighted Kappa (sd)
VGG-19	85.91 ($\pm 0.04)$	0.2439 ($\pm 0.04)$	0.8456 ($\pm 0.04)$	88.12 ($\pm 0.04)$	0.2218 ($\pm 0.03)$	0.8677 ($\pm 0.04)$
ResNet-50	93.92 ($\pm 0.04)$	0.1638 ($\pm 0.04)$	0.9257 ($\pm 0.04)$	97.89 ($\pm 0.05)$	0.1141 ($\pm 0.06)$	0.9754 ($\pm 0.04)$
Inception-V3	96.84 ($\pm 0.05)$	0.1346 ($\pm 0.05)$	0.9549 ($\pm 0.04)$	99.21 ($\pm 0.05)$	0.1109 ($\pm 0.05)$	0.9786 ($\pm 0.05)$

sd: standard deviation.

and

Table 3. Comparison of CNN models using dataset 3 for 40 epochs.

CNN Model	70%–30% Distribution			10-Fold Cross-Validation
	Accuracy (%) (sd)	RMSE (sd)	Weighted Kappa (sd)	Accuracy (%) (sd)	RMSE (sd)	Weighted Kappa (sd)
VGG-19	88.76 ($\pm 0.04)$	0.2154 ($\pm 0.04)$	0.8741 ($\pm 0.04)$	89.34 ($\pm 0.03)$	0.2096 ($\pm 0.04)$	0.8799 ($\pm 0.04)$
ResNet-50	95.64 ($\pm 0.05)$	0.1466 ($\pm 0.05)$	0.9429 ($\pm 0.05)$	96.98 ($\pm 0.04)$	0.1232 ($\pm 0.04)$	0.9663 ($\pm 0.05)$
Inception-V3	96.48 ($\pm 0.03)$	0.1382 ($\pm 0.04)$	0.9513 ($\pm 0.04)$	99.36 ($\pm 0.04)$	0.1094 ($\pm 0.05)$	0.9801 ($\pm 0.05)$

sd: standard deviation.

, respectively, with respect to 70%–30% distribution of employed data and 88.35%, 88.12%, and 89.34% for 10-fold cross-validation according to the results mentioned in these tables.

On the other hand, the accuracy of the ResNet-50 model lies between 93.9% and 95.64% with respect to the 70%–30% distribution and between 96.8% and 97.61% for the 10-fold cross-validation concerning 40 epochs as shown in Table 1, Table 2 and Table 3. The accuracy of Inception-V3 for such three benchmark datasets regarding 40 epochs ranges from 95.62% to 96.48% for the 70%–30% distribution and from 99.2% to 99.36% utilizing the 10-fold cross-validation in accordance with the results presented in these tables. The performance of employed classifiers in this research work in accordance with accuracy for these three datasets regarding 40 epochs is revealed by Figure 7 and it also represents that Inception-V3 possesses supremacy over VGG-19 and Resnet-50 for accuracy taking such employed datasets into consideration. RMSE and kappa statistic are also regarded as powerful metrics to compare various classifiers. To achieve decent performance of a classifier, RMSE should be lowest comparing RMSE of other classifiers taken into account. Previous research articles for COVID-19 detection did not account for RMSE and kappa statistic to prove the superiority of their proposed work. However, this work emphasized RMSE as a comparative measuring metric to differentiate the performance of employed classification models. Table 1, Table 2 and Table 3 reveal the dominance of Inception-V3 as it provides the lowest RMSE comparing the RMSEs provided by VGG-19 and ResNet-50. The lowest RMSE possessed by Inception-V3 for these employed datasets indicates its superiority, as depicted in Figure 8. Table 1, Table 2 and Table 3 denote a variation between 0.8–1.0 showing “almost perfect agreement” for kappa statistic values and exhibiting the superiority of Inception-V3 over VGG-19 and Resnet-50 with respect to these performance-evaluating metrics used in the testing phase. Figure 9 also represents the better performance of Inception-V3 than other two utilized classifiers in this proposed work. Formulated on the kappa statistic values of different classifiers utilized to classify the datasets, Inception-V3 produces a reasonably better outcome compared to ResNet-50 and VGG-19.

The second part of this section explores the performance comparisons of the aforementioned classifiers developed on various statistical performance estimators including TP-Rate/Recall, FP-Rate, precision, and F-measure. TP-Rate/Recall, Precision, FP-Rate, and F-Measure values are evaluated from the confusion matrix of individual classifiers.

6.3. Confusion Matrix

In machine learning, the confusion matrix is a certain tabular representation that shows a classification algorithm’s performance. This offers a more comprehensive analysis compared to accuracy. Each column of the matrix signifies the patterns in the expected data class whereas every row specifies the patterns in the obvious class.

Table 4. A confusion matrix for two-class classifier.

Confusion Matrix		Predicted Class
		Positive	Negative
Actual Class	positive	TP	FP
	negative	FN	TN

below shows a confusion matrix using a two-class classifier with the subsequent cells as:

■

True-Positive (TP) indicates the number of ‘positive’ examples classified as ‘positive.’

■

False-Positive (FP) implies the number of ‘negative’ examples classified as ‘positive.’

■

False-Negative (FN) denotes the number of ‘positive’ examples classified as ‘negative.’

■

True-Negative (TN) means the number of ‘negative’ examples classified as ‘negative.’

A two-class confusion matrix identifies several standard terms. The accuracy (i.e., classification accuracy) is the sum of the correctly classified samples divided by the total number of samples present. The following equation calculates this as:

$$Accuracy=\frac{TP+TN}{TP+T+N+FP+FN}$$

(7)

The precision denotes the ratio of the predicted positive samples discovered to be correct, as calculated using the following equation:

$$Precision=\frac{TP}{TP+FP}$$

(8)

The FP-Rate (i.e., False-Positive Rate) indicates the ratio of negative examples incorrectly classified as positive, as given by the equation:

$$FP-Rate=\frac{FP}{FP+TN}$$

(9)

The TP-Rate (i.e., True-Positive Rate) or recall is the ratio of positive samples discovered correctly, as assessed using the equation:

$$TP-Rate=Recall=\frac{TP}{TP+FN}$$

(10)

A large precision value is occasionally more applicable while sometimes a large recall value is more obvious. However, in the most presentations, the objective is to increase both values.

The cohesive arrangement of these values is the f-measure and usually formulated as a harmonic mean of both the values:

$$F-measure=\frac{2\ast Precision\ast Recall}{Precision+Recall}$$

(11)

Table 5. Performance of CNN models with respect to diverse metrics using dataset 1.

CNN Model	70%–30% Distribution				10-Fold Cross-Validation
	TP-Rate/Recall (%) (sd)	FP-Rate (%) (sd)	Precision (%) (sd)	F-Measure (%) (sd)	TP-Rate/Recall (%) (sd)	FP-Rate (%) (sd)	Precision (%) (sd)	F-Measure (%) (sd)
VGG-19	87.10 ($\pm 0.03)$	12.55 ($\pm 0.03)$	87.06 ($\pm 0.03)$	87.08 ($\pm 0.03)$	88.00 ($\pm 0.03)$	11.65 ($\pm 0.03)$	87.96 ($\pm 0.04)$	87.98 ($\pm 0.03)$
ResNet-50	94.03 ($\pm 0.05)$	5.62 ($\pm 0.05)$	93.99 ($\pm 0.05)$	94.01 ($\pm 0.05)$	97.26 ($\pm 0.05)$	1.39 ($\pm 0.05)$	97.22 ($\pm 0.05)$	97.24 ($\pm 0.05)$
Inception-V3	95.27 ($\pm 0.05)$	4.38 ($\pm 0.05)$	95.23 ($\pm 0.05)$	95.25 ($\pm 0.05)$	98.88 ($\pm 0.04)$	0.77 ($\pm 0.04)$	98.84 ($\pm 0.04)$	98.86 ($\pm 0.04)$

sd: standard deviation. Value in bold denotes the respective measure is shown for classification in figure.

,

Table 6. Performance of CNN models with respect to diverse metrics using dataset 2.

CNN Model	70%–30% Distribution				10-Fold Cross-Validation
	TP-Rate/Recall (%) (sd)	FP-Rate (%) (sd)	Precision (%) (sd)	F-Measure (%) (sd)	TP-Rate/Recall (%) (sd)	FP-Rate (%) (sd)	Precision (%) (sd)	F-Measure (%) (sd)
VGG-19	85.56 ($\pm 0.05)$	14.09 ($\pm 0.05)$	85.52 ($\pm 0.05)$	85.54 ($\pm 0.05)$	87.77 ($\pm 0.04)$	11.88 $\left(\pm 0.05\right)$	87.73 ($\pm 0.04)$	87.75 ($\pm 0.05)$
ResNet-50	93.57 ($\pm 0.06)$	6.08 ($\pm 0.05)$	93.53 ($\pm 0.07)$	93.55 $\left(\pm 0.05\right)$	97.54 $\left(\pm 0.05\right)$	1.11 $\left(\pm 0.05\right)$	97.50 ($\pm 0.04)$	97.52 ($\pm 0.04)$
Inceptn-V3	96.49 ($\pm 0.05)$	3.16 ($\pm 0.06)$	96.45 ($\pm 0.05)$	96.47 ($\pm 0.05)$	98.86 ($\pm 0.05)$	0.79 ($\pm 0.04)$	98.82 ($\pm 0.05)$	98.84 ($\pm 0.03)$

sd: standard deviation. Value in bold denotes the respective measure is shown for classification in figure.

and

Table 7. Performance of CNN models with respect to diverse metrics using dataset 3.

CNN Model	70%–30% Distribution				10-Fold Cross-Validation
	TP-Rate/Recall (%) (sd)	FP-Rate (%) (sd)	Precision (%) (sd)	F-Measure (%) (sd)	TP-Rate/Recall (%) (sd)	FP-Rate (%) (sd)	Precision (%) (sd)	F-Measure (%) (sd)
VGG-19	88.41 ($\pm 0.05)$	11.24 ($\pm 0.05)$	88.37 ($\pm 0.05)$	88.39 ($\pm 0.05)$	88.99 ($\pm 0.05)$	10.66 ($\pm 0.05)$	88.95 ($\pm 0.05)$	88.97 ($\pm 0.05)$
ResNet-50	95.29 ($\pm 0.07)$	4.36 ($\pm 0.05)$	95.25 ($\pm 0.05)$	95.27 ($\pm 0.06)$	96.63 ($\pm 0.05)$	2.02 ($\pm 0.05)$	96.59 ($\pm 0.07)$	96.61 ($\pm 0.05)$
Inception-V3	96.13 ($\pm 0.04)$	3.52 ($\pm 0.05)$	96.09 ($\pm 0.06)$	96.11 ($\pm 0.05)$	99.21 ($\pm 0.04)$	0.64 ($\pm 0.05)$	99.17 ($\pm 0.05)$	99.19 ($\pm 0.04)$

sd: standard deviation. Value in bold denotes the respective measure is shown for classification in figure.

display performances with respect to the aforementioned diverse metrics for datasets 1, 2, and 3, respectively. For evaluating the performance of a classifier, it is desirable to obtain greater precision, recall, TP-rate, and F-measure values but a smaller FP-rate value. The F-measure is generally considered a balanced metric and is often used as the most important single metric. Inception-V3 classifier finds the F-measure values of 95.25%, 96.47%, and 96.11% utilizing 70%–30% distribution for dataset 1, dataset 2, and dataset 3, respectively, and 98.86%, 98.84%, and 99.19% employing 10-fold cross-validation as shown in Table 5, Table 6 and Table 7, respectively. In contrast to this, the F-measure value of ResNet-50 model ranges from 93.55% to 95.27% applying 70%–30% distribution and from 96.61% to 97.52% applying a 10-fold cross-validation, as VGG-19 possesses F-measure values varying between 85.54% and 88.39% for the 70%–30% distribution and from 87.98% to 88.97% for the 10-fold cross-validation as exposed in Table 5, Table 6 and Table 7.

Figure 10 establishes that the Inception-V3 classifier produces a superior performance compared to the other classifiers with respect to the F-measure. The Inception-V3 model has the largest values for accuracy, precision, kappa statistic, TP-Rate/Recall, and F-measure and the smallest values for FP-rate and RMSE. In fact, Inception-V3 overtakes ResNet-50 and VGG-19 classifiers concerning all these performance estimators being applied.

This section explores comparison between the state-of-the-art methods and the proposed framework in accordance with F-measure for the three imagery benchmark datasets employed in this work. Several previous research works have emphasized accuracy as the most accurate paradigm for comparative study and the accuracy level is not only a sufficient and reliable performance measurement tool. Though such current research articles provide enhanced results, as depicted in Figure 11, proposed research work has acquired improvements to their results for the same datasets. Instead of accuracy measurement, the research work proposed in this paper highlights the F-measure as a strong and appropriate performance comparing tool. Figure 11 reveals the superior performance of the proposed framework which possesses better F-measure values 98.86%, 98.84%, and 99.19% for dataset 1 and dataset 2 to some current research techniques employing the aforesaid three imagery benchmark datasets.

7. Conclusions

The research study presents an application of Convolutional Neural Network models in the field of human health, specifically the detection of patients’ COVID-19 status using chest X-ray and CT scan imagery datasets. This work proposes to set and tune parameters, and also implement three models, VGG-19, ResNet-50, and Inception-V3 because these models seem to be efficient at feature extraction as well as image classification. We then validate our results using various performance estimators such as accuracy, RMSE, kappa statistic, F-measure, recall, precision, etc., for various epoch sizes. This research article emphasizes the F-measure as the best evaluating metric obtained from a confusion matrix for comparing the performances of these trained models with some of the current, related research articles and reveals the superiority of the proposed framework. It also accounts for RMSE and kappa statistic to prove its improved and efficient performance for predicting COVID-19 compared to existing research articles in this regard. The Inception-V3 model has an F-measure value of 98.86% using the dataset 1, 98.84% using the dataset 2, and 99.19% using the dataset 3, considering 40 epochs. These resulting outcomes are significantly better (F-measure > 98% (approximately), all three datasets) compared to other models. Considering the three standard datasets, the Inception-V3 model also has the smallest FP-Rate values and the greatest recall values compared to other classifiers. This research work aims to diminish the number of false negatives thereby obtaining a higher recall and builds a classification model which is free from overfitting and a stable model for COVID-19 prediction. The performance analysis based on various diverse metrics including RMSE and kappa statistic of this work also suggests a robust and generic framework for detecting this kind of disease. Though some previous works employed the concept of hyper tuning, they focused heavily on accuracy to detect the status of COVID-19. Consideration of various strong metrics along with classification accuracy in our proposed work show the novelty of this work. Development of an integrated model endowed with the capability of providing an optimal outcome considering the best combination of parameter tuning, concerning both a deep learning model and biological model (from multi-omics data), is the possible future directive.