Using Deep Learning Networks to Identify Cyber Attacks on Intrusion Detection for In-Vehicle Networks^†

Round 1

Reviewer 1 Report

Authors present a study on a new type of NIDS, called OMIDS, which has a good classification performance.

 

1.      For the classification tasks, briefly explain what these datasets are for. How many training data and testing data you use?

2.      “The experimental results show that the classification accuracy of the dataset for HCRL Car-Hacking by the VGG16 and XBoost classifiers (n=50) reached 97.8241% and 99.9995% for the 5-subcategory classification results on the testing data, respectively”. Did we really get 4 digits of accuracy on this number?

3.      Optical neural networks for your consideration to be cited in the references, for instance, H. Zhang, Nat. Commun. 12, Article number: 457 (2021); Nature Communications volume 13, Article number: 1044 (2022).

 

Overall, this paper is comprehensive and the performance is quite good. A minor revision is recommended.

Author Response

We would like to thank the Reviewer 1 for taking the time to read our paper, and providing valuable comments and suggestions. (All revised statements in manuscript were represented in brown color for easy review)

 

Q1)    Authors present a study on a new type of NIDS, called OMIDS, which has a good classification performance. For the classification tasks, briefly explain what these datasets are for. How many training data and testing data you use?

Answer: In the experiments, we used well-known open datasets for the targeted intrusion detection of car hacking developed by the Hacking and Countermeasure Research Lab (HCRL) [24]. The training sample data were obtained from two important data sources for car intrusion experiment: the common HCRL intrusion detection archive [24], and the HCRL Car-Hacking dataset. The HCRL dataset contains 30–40 min of CAN traffic messages. The HRCL dataset was compiled by Culture Makers and the Korea Internet & Security Agency for car security to include synthetic contemporary attack behaviors in real-world in-vehicle network traffic. (https://ocslab.hksecurity.net/Datasets)

The experimental data were divided into a training set and a testing set. The training set contained 175,341 records of CAN messages (68.05%), whereas the testing set contained 82,332 records (31.95%).

Q2)     The experimental results show that the classification accuracy of the dataset for HCRL Car-Hacking by the VGG16 and XBoost classifiers (n=50) reached 97.8241% and 99.9995% for the 5-subcategory classification results on the testing data, respectively”. Did we really get 4 digits of accuracy on this number?

Answer: Yes. To discriminate classification error for few misclassified samples in model testing, 4 digits of accuracy was adopted in our experiment.

Q3)      Optical neural networks for your consideration to be cited in the references, for instance, H. Zhang, Nat. Commun. 12, Article number: 457 (2021); Nature Communications volume 13, Article number: 1044 (2022).

Answer: Add two recent papers as important references in our manuscript.

Zhang, H., Gu, M., Jiang, X.D. et al. An optical neural chip for implementing complex-valued neural network. Nat Commun 12, 457 (2021). https://doi.org/10.1038/s41467-020-20719-7

Zhu, H.H., Zou, J., Zhang, H. et al. Space-efficient optical computing with an integrated chip diffractive neural network. Nat Commun 13, 1044 (2022). https://doi.org/10.1038/s41467-022-28702-0

Author Response File: Author Response.pdf

Reviewer 2 Report

1. On the abstract, you need to provide on more detail reasons why you mentioned " Therefore, it is necessary to develop a new type of NIDS -
 namely, on-the-move Intrusion Dectection System (OMIDS) - to categorise these threats."

2. On the Introduction, you need to provide the chalenges on the implemetation result. The current paper not provide them.

 

3. You mentioned with VGG16. However you not detail enough showing with the contribution of VGG16.

 

4. You need to explore more on "In-Vehicle Detection of Targeted CAN Bus Attacks" on the persfective of implications to the CAN Bus attach.

 

5. Please exolore more on  how you develop and create "General architecture of a basic classifier".

Author Response

We would like to thank the Reviewer 2 for taking the time to read our paper carefully, and providing valuable comments and suggestions.

Q1) On the abstract, you need to provide on more detail reasons why you mentioned " Therefore, it is necessary to develop a new type of NIDS - namely, on-the-move Intrusion Detection System (OMIDS) - to categorise these threats."

Answer: Add the detailed description of the OMIDS on Page 2 as follows. (word limit of an abstract)

The OMIDS focuses on the mobile security issues of in-vehicle network instead of detection for traditional network weak points on the Internet. Notably, the OMIDS detects vulnerabilities of the CAN bus network attacks and categorizes the anomaly traffics within in-vehicle communication protocol with potential attacks exploited and has been a significant tool in securing in-vehicle networks and the related information systems on vehicles.

Also, added Figure 1 Spoofing RPM/GEAR attack of in-vehicle networks to illustrate the detection operations of the OMIDS.

Q2)  On the Introduction, you need to provide the challenges on the implementation result. The current paper not provide them.

 Answer: Highlight the challenges on the implementation result in Sec 1. These revision words are shown as follows.

Considering the increasing role of ECU in the CAN bus systems, the challenges for anomaly detection on the in-vehicle network are summarized as:

(i) As attacks on large-scale networks in the future become more diverse, basic classifiers, such as decision trees (DT), naive Bayes classifiers, LR, k-nearest neighbour classification (k-NN), and SVC, become increasingly unable to effectively obtain sufficient classification results on complex intrusion patterns using an intrusion detection system. Therefore, ensemble learning-based techniques such as boosting [10], random forest [11], gradient boost DT (GBDT) [12], and XGBoost [13-16] have been developed to reduce the bias and variance.

(ii) Practically, most existing ML-based approaches to categorizing the threats for the OMIDS do not work well. Thus, high accuracy of the OMIDS needs high-resolution feature set inputs extracted from attack scenarios of CAN bus, which is not a trivial task.

In other words, the development of an accurate and robust approach for anomaly detection for in-vehicle networks is still a challenge.

 

Q3)  You mentioned with VGG16. However you not detail enough showing with the contribution of VGG16.

 Answer: To highlight the contributions of VGG16 to categorize the threats for in-vehicle networks, contributions were added as follows. (Sec.1)

In summary, the primary contributions of this study are as follows:

• In our experiment, the classification error of the VGG16 model for anomaly detection is 100% for binary classification based on HCRL dataset;
• As shown in Table 10, the proposed VGG16 approach provides higher prediction accuracy for multiclass classification (5 categories) than those of Naïve Bayes and SVC classifiers.

 

Q4). You need to explore more on "In-Vehicle Detection of Targeted CAN Bus Attacks" on the perspective of implications to the CAN Bus attach.

 Answer: Added the description of context in front of Section 2.2

Practically, most vehicles often use the CAN bus for major communication among their components over the IVN. From the intrusion reports [17, 20-21, 26], hackers exploited the vulnerabilities and intruded in-vehicle network to compromise the targeted ECU of the vehicle. Such targeted attack scenarios are often hard to detect by network intrusion detection systems because the specific payload is usually not missed in the experimental data. As shown in Figure 2, a DoS attack injected high-priority of CAN messages (ox000) in a short cycle thru the use of OTA broadcast and delayed the normal message communications.

Also, added Figure 2. DoS attack of in-vehicle networks to illustrate the attack scenario of "In-Vehicle Detection of Targeted CAN Bus Attacks"..

 

 Practically, most vehicles use the CAN bus for communication between their components. Hackers exploited the vulnerabilities and intruded into the in-vehicle network to compromise the targeted ECU of the vehicle. Such targeted attack scenarios are often hard to detect by network intrusion detection systems because the specific payload is usually not contained within their training data sets.

Q5)  Please explore more on how you develop and create "General architecture of a basic classifier".

Answer: Add the detailed description of Figure 3 as follows. (Located at the following text of Figure 3)

As shown in Figure 3, a basic classifier in ML models use the selected feature vectors from the training instances to train the model, determine the hyperparameters of the trained model, and predict the possible categories of test data. However, a single basic classifier does not sufficiently match the data distribution in large-scale networks with complex intrusion patterns.

Author Response File: Author Response.pdf

Round 2

Reviewer 2 Report

Q1) On the abstract, you need to provide on more detail reasons why you mentioned " Therefore, it is necessary to develop a new type of NIDS - namely, on-the-move Intrusion Detection System (OMIDS) - to categorise these threats."

Answer: Add the detailed description of the OMIDS on Page 2 as follows. (word limit of an abstract) The OMIDS focuses on the mobile security issues of in-vehicle network instead of detection for traditional network weak points on the Internet. Notably, the OMIDS detects vulnerabilities of the CAN bus network attacks and categorizes the anomaly traffics within in-vehicle communication protocol with potential attacks exploited and has been a significant tool in securing in-vehicle networks and the related information systems on vehicles. Also, added Figure 1 Spoofing RPM/GEAR attack of in-vehicle networks to illustrate the detection operations of the OMIDS.  

Response: The authors need to elaborate more detail about:

1. Why you confirm that " the OMIDS detects vulnerabilities of the CAN bus network attacks and categorizes the anomaly traffics within in-vehicle communication protocol with potential attacks exploited and has been a significant tool in securing in-vehicle networks and the related information systems on vehicles." Please add them on the paper.

2.  The authros must provide in more detail and systematic sreasons why added "Figure 1 Spoofing RPM/GEAR".  

Q2) On the Introduction, you need to provide the challenges on the implementation result. The current paper not provide them.  

Answer: Highlight the challenges on the implementation result in Sec 1. These revision words are shown as follows.

Considering the increasing role of ECU in the CAN bus systems, the challenges for anomaly detection on the in-vehicle network are summarized as:

(i) As attacks on large-scale networks in the future become more diverse, basic classifiers, such as decision trees (DT), naive Bayes classifiers, LR, k-nearest neighbour classification (k-NN), and SVC, become increasingly unable to effectively obtain sufficient classification results on complex intrusion patterns using an intrusion detection system. Therefore, ensemble learning-based techniques such as boosting [10], random forest [11], gradient boost DT (GBDT) [12], and XGBoost [13-16] have been developed to reduce the bias and variance.

(ii) Practically, most existing ML-based approaches to categorizing the threats for the OMIDS do not work well. Thus, high accuracy of the OMIDS needs high-resolution feature set inputs extracted from attack scenarios of CAN bus, which is not a trivial task. In other words, the development of an accurate and robust approach for anomaly detection for in-vehicle networks is still a challenge.  

Response: The authors must provided in detail :

1. Why you wrote " in the future become more diverse, basic classifiers, such as decision trees (DT), naive Bayes classifiers, LR, k-nearest neighbour classification (k-NN), and SVC, become increasingly unable to effectively obtain sufficient classification results" Please show on the numerical result  with the reasons "become increasingly unable to effectively obtain sufficient classification results""

2. You need to show with the evidences why you wrote "Practically, most existing ML-based approaches to categorizing the threats for the OMIDS do not work well. " 3. You need to show with the evidnces and reasins why you nentioned "for anomaly detection for in-vehicle networks is still a challenge."  

Q3) You mentioned with VGG16. However you not detail enough showing with the contribution of VGG16.  

Answer: To highlight the contributions of VGG16 to categorize the threats for in-vehicle networks, contributions were added as follows. (Sec.1) In summary, the primary contributions of this study are as follows:

1. In our experiment, the classification error of the VGG16 model for anomaly detection is 100% for binary classification based on HCRL dataset;

2. As shown in Table 10, the proposed VGG16 approach provides higher prediction accuracy for multiclass classification (5 categories) than those of Naïve Bayes and SVC classifiers.  

Response;

1. Please add more detail why you mentionedwith " the classification error of the VGG16 model for anomaly detection is 100% for binary classification based on HCRL dataset;"

2. You need to provide in detail why you mentoioned " Table 10, the proposed VGG16 approach provides higher prediction accuracy for multiclass classification (5 categories) than those of Naïve Bayes  

Q4). You need to explore more on "In-Vehicle Detection of Targeted CAN Bus Attacks" on the perspective of implications to the CAN Bus attach.  

Answer: Added the description of context in front of Section 2.2

Practically, most vehicles often use the CAN bus for major communication among their components over the IVN. From the intrusion reports [17, 20-21, 26], hackers exploited the vulnerabilities and intruded in-vehicle network to compromise the targeted ECU of the vehicle. Such targeted attack scenarios are often hard to detect by network intrusion detection systems because the specific payload is usually not missed in the experimental data. As shown in Figure 2, a DoS attack injected high-priority of CAN messages (ox000) in a short cycle thru the use of OTA broadcast and delayed the normal message communications. Also, added Figure 2. DoS attack of in-vehicle networks to illustrate the attack scenario of "In-Vehicle Detection of Targeted CAN Bus Attacks".. Practically, most vehicles use the CAN bus for communication between their components. Hackers exploited the vulnerabilities and intruded into the in-vehicle network to compromise the targeted ECU of the vehicle. Such targeted attack scenarios are often hard to detect by network intrusion detection systems because the specific payload is usually not contained within their training data sets.  

Response:

1. Please provide in detail reasons why you mentioned "most vehicles use the CAN bus for communication between their components. Hackers exploited the vulnerabilities and intruded into the in-vehicle network to compromise the targeted ECU of the vehicle. "

2. Why you mentioned "Such targeted attack scenarios are often hard to detect by network intrusion detection system" Please provided the reasons.  

Q5) Please explore more on how you develop and create "General architecture of a basic classifier".  

Answer: Add the detailed description of Figure 3 as follows. (Located at the following text of Figure 3) As shown in Figure 3, a basic classifier in ML models use the selected feature vectors from the training instances to train the model, determine the hyperparameters of the trained model, and predict the possible categories of test data. However, a single basic classifier does not sufficiently match the data distribution in large-scale networks with complex intrusion patterns.  

Response:

1. Please add more detail arguments why you wrote "Figure 3, a basic classifier in ML models use the selected feature vectors from the training instances to train the model"

2. Please add with detail of " the hyperparameters of the trained model, and predict the possible categories of test data."  

Author Response

We would like to thank the Reviewer 2 for taking the time to read our paper carefully, and providing valuable comments and suggestions.

Q1) On the abstract, you need to provide on more detail reasons why you mentioned " Therefore, it is necessary to develop a new type of NIDS - namely, on-the-move Intrusion Detection System (OMIDS) - to categorise these threats."

Answer: Add the detailed description of the OMIDS on Page 2 as follows. (word limit of an abstract) The OMIDS focuses on the mobile security issues of in-vehicle network instead of detection for traditional network weak points on the Internet. Notably, the OMIDS detects vulnerabilities of the CAN bus network attacks and categorizes the anomaly traffics within in-vehicle communication protocol with potential attacks exploited and has been a significant tool in securing in-vehicle networks and the related information systems on vehicles. Also, added Figure 1 Spoofing RPM/GEAR attack of in-vehicle networks to illustrate the detection operations of the OMIDS.  

Response: The authors need to elaborate more detail about:

1. Why you confirm that " the OMIDS detects vulnerabilities of the CAN bus network attacks and categorizes the anomaly traffics within in-vehicle communication protocol with potential attacks exploited and has been a significant tool in securing in-vehicle networks and the related information systems on vehicles." Please add them on the paper.

Answer: Revise the role description of the OMIDS on Page 2 as follows.

The network-based IDS on in-vehicle network (i.e., OMIDS) play a role of data collector and data analyzer over the CAN bus that identifies security threats and attacks of in-vehicle network systems by using detection of anomalous of CAN bus messages with protocol analyses. [4] Thus, distinct NIDSs on in-vehicle network systems are developing for detecting threats of the CAN bus network attacks within in-vehicle communication protocol associated with potential attacks exploited, categorizes the anomaly messages in securing in-vehicle networks and the related information systems on vehicles. [2-7]

2.  The authors must provide in more detail and systematic reasons why added "Figure 1 Spoofing RPM/GEAR".  

Answer: This sub-paragraph mainly describes the role of the OMIDS. I think it is inappropriate to place the Figure 1 here. Thus redraw the Figure 1 to show the role and deployment strategy of the OMIDS within in-vehicle networking systems as follows

The concept of IDS deployed to the automotive system was first introduced by Hoppe et al. [5] later, IDS deployment strategies were discussed by [6-7] For NIDS to monitor and inspect the communication messages in the CAN network from different sources, it is recommended to be deployed to central gateway. As shown in Fig. 1, the OMIDS was placed to central gateway that monitor activities in the CAN network and identify the attacks.

In summary, we rewrote this paragraph as follows

The network-based IDS on in-vehicle network (i.e., OMIDS) play a role of data collector and data analyzer over the CAN bus that identifies security threats and attacks of in-vehicle network systems by using detection of anomalous of CAN bus messages with protocol analyses. [4] Thus, distinct NIDSs on in-vehicle network systems are developing for detecting threats of the CAN bus network attacks within in-vehicle communication protocol associated with potential attacks exploited, categorizes the anomaly messages in securing in-vehicle networks and the related information systems on vehicles. [2-7]

The concept of IDS deployed to the automotive system was first introduced by Hoppe et al. [5] later, IDS deployment strategies were discussed by [6-7]. For NIDS to monitor and inspect the communication messages in the CAN network from different sources, it is recommended to be deployed to central gateway. [2] As shown in Fig. 1, the OMIDS was placed to central gateway that monitor activities in the CAN network and identify the attacks.

 

-------------------------------------------------------------------------------------------------------------------

Q2) On the Introduction, you need to provide the challenges on the implementation result. The current paper not provide them.  

Answer: Highlight the challenges on the implementation result in Sec 1. These revision words are shown as follows.

Considering the increasing role of ECU in the CAN bus systems, the challenges for intrusion detection on the in-vehicle network are summarized as:

(i) As attacks on large-scale networks in the future become more diverse, basic classifiers, such as decision trees (DT), naive Bayes classifiers, LR, k-nearest neighbor classification (k-NN), and SVC, become increasingly unable to effectively obtain sufficient classification results on complex intrusion patterns using an intrusion detection system. Therefore, ensemble learning-based techniques such as boosting [10], random forest [11], gradient boost DT (GBDT) [12], and XGBoost [13-16] have been developed to reduce the bias and variance.

(ii) Practically, most existing ML-based approaches to categorizing the threats for the OMIDS do not work well. Thus, high accuracy of the OMIDS needs high-resolution feature set inputs extracted from attack scenarios of CAN bus, which is not a trivial task. In other words, the development of an accurate and robust approach for anomaly detection for in-vehicle networks is still a challenge.  

Response: The authors must provide in detail:

1. Why you wrote " in the future become more diverse, basic classifiers, such as decision trees (DT), naive Bayes classifiers, LR, k-nearest neighbor classification (k-NN), and SVC, become increasingly unable to effectively obtain sufficient classification results" Please show on the numerical result with the reasons "become increasingly unable to effectively obtain sufficient classification results""
2. You need to show with the evidences why you wrote "Practically, most existing ML-based approaches to categorizing the threats for the OMIDS do not work well. "
3. You need to show with the evidences and reasons why you mentioned "for anomaly detection for in-vehicle networks is still a challenge."

Answer:

In this sub-paragraph, we are not very thoughtful and rigorous.

We revised the whole sub-paragraph as follows.

Considering the increasing security threats in the CAN bus systems, the challenges for automatic ML-based intrusion detection on the in-vehicle network systems are summarized as:

The idea of machine learning-based IDS deployed to the CAN bus network was first introduced by Kang et al. in 2016.[13] In this case, they used unsupervised pre-training of deep belief networks (DBN) model in detecting any abnormal from normal frequencies of CAN message. Later, Taylor et al. used a support vector machine for binary classification to classify the CAN traffic flows [14]. Recent, Hossain et al. (2020) developed a LSTM to detect the threat predict using sequence data inputs and achieved an overall detection accuracy of 99.995%. [4],

With the proliferation of attacks on in-vehicle network, traditional artificial neural network (ANN) failed to detect in some complex attack cases, such as fuzzy attacks in [15]. In this case, Song et al. (2020) used the deep neural network models, deep convolutional neural network (DCNN) and LSTM, to achieve high accuracy with low error rate for prediction results.

Theoretically, assembling multiple classifiers can reduce false positives and produce more accurate classification results than single classifiers [12]. For example, Rajadurai and Gandhi (2021) used a stacked ensemble model of different base classifiers to build a stronger learner and showed that the stacked ensemble model produced more accurate results than that of a single algorithm. [13] Therefore, ensemble learning-based techniques such as boosting [18], random forest [19], gradient boost DT (GBDT) [20], and XGBoost [21-24] have been developed to reduce the bias and variance.

 (ii) In a supervised ML model, it requires complete labeled data in training process. The difficulties in predicting and generating attack behavior in evaluating the CAN bus system. [2] Practically, existing signature-based approaches for the NIDS are based on behavioral features to categorize the threats. [25] Importantly, high accuracy of the OMIDS needs continuously updating for high-resolution feature set inputs extracted from attack scenarios of CAN bus, which is not a trivial task. Moreover, it cannot efficiently work if an unknown message is abnormal. Thus, the development of an accurate and robust approach for automatic threat detection for in-vehicle network systems is still a challenge.

-------------------------------------------------------------------------------------------------------

Q3) You mentioned with VGG16. However you not detail enough showing with the contribution of VGG16.  

Answer: To highlight the contributions of VGG16 to categorize the threats for in-vehicle networks, contributions were added as follows. (Sec.1) In summary, the primary contributions of this study are as follows:

1. In our experiment, the classification error of the VGG16 model for intrusion detection is 100% for binary classification based on HCRL dataset;
2. As shown in Table 10, the proposed VGG16 approach provides higher prediction accuracy for multiclass classification (5 categories) than those of Naïve Bayes and SVC classifiers.  

Response;

1. Please add more detail why you mentioned with " the classification error of the VGG16 model for anomaly detection is 100% for binary classification based on HCRL dataset;"

Answer: Thanks for reminding us. Revise ‘classification error of the VGG16’ to ‘accuracy of the VGG16’. This paragraph is revised as follows.

In our experiment, the accuracy of the VGG16 model for intrusion detection is 100%/100% (Table 9) for binary classification on the training and testing data.

2. You need to provide in detail why you mentioned " Table 10, the proposed VGG16 approach provides higher prediction accuracy for multiclass classification (5 categories) than those of Naïve Bayes and SVC classifiers  

Answer: This paragraph is revised as follows

As shown in Table 10, the proposed VGG16 approach provides higher prediction accuracy (97.9420% /97.8241%) for multiclass classification (5 categories) than those of Naïve Bayes (91.0095%/91.0273%) and SVC classifier (91.0095%/ 91.4137%) on training and testing data.

-------------------------------------------------------------------------------------------------------Q4). You need to explore more on "In-Vehicle Detection of Targeted CAN Bus Attacks" on the perspective of implications to the CAN Bus attach.  

Answer: Added the description of context in front of Section 2.2

Practically, most vehicles often use the CAN bus for communication among their components over the IVN. From the intrusion reports [17, 20-21, 26], hackers exploited the vulnerabilities and intruded in-vehicle network to compromise the targeted ECU of the vehicle. Such targeted attack scenarios are often hard to detect by network intrusion detection systems because the specific payload is usually not missed in the experimental data. As shown in Figure 2, a DoS attack injected high-priority of CAN messages (ox000) in a short cycle thru the use of OTA broadcast and delayed the normal message communications. Also, added Figure 2. DoS attack of in-vehicle networks to illustrate the attack scenario of "In-Vehicle Detection of Targeted CAN Bus Attacks"..

Practically, most vehicles use the CAN bus for communication between their components. Hackers exploited the vulnerabilities and intruded into the in-vehicle network to compromise the targeted ECU of the vehicle. Such targeted attack scenarios are often hard to detect by network intrusion detection systems because the specific payload is usually not contained within their training data sets. 

Response:

1. Please provide in detail reasons why you mentioned "most vehicles use the CAN bus for communication between their components. Hackers exploited the vulnerabilities and intruded into the in-vehicle network to compromise the targeted ECU of the vehicle. "

Answer:

Thanks for reminding us for ‘imprecise’ wording. Revise ‘most vehicles use the CAN bus’ to ‘modern vehicles use the CAN bus for communication between their components’.

This paragraph is revised as follows.

Modern vehicles initiatives take by the automotive manufacturers have increased the number of ECUs per vehicle. For example, there are nearly 70 ECUs deployed in the modern vehicle. With its increasing applications, CAN bus has become a standard choice for automobiles, as well as for other applications too such as EV batteries, planes, ships, machineries, and many more. [31] Practically, modern vehicles often use the CAN bus for communication among their components.

From the car intrusion reports [4-7, 26, 29-30, 35], hackers exploited the vulnerabilities and intruded the in-vehicle network to compromise the targeted ECU of the vehicle and issue attack commands. As shown in Figure 2, an example of DoS attack injected high priority of CAN messages (ox000) in a short cycle from the compromised ECU node thru the use of OTA broadcasting and delayed the normal message communications.

[31] Precedence Research, Automotive Communication Technology Market Size, Report 2021-2030, https://www.precedenceresearch.com/automotive-communication -technology-market (accessed on 3 July 2022).

 

2. Why you mentioned "Such targeted attack scenarios are often hard to detect by network intrusion detection system" Please provided the reasons.  

Answer:

This explanation for reason is too trivial in the literature. So, delete the text of sub-paragraph for clarity.

The explanation is shown as follows.

In practice, partial messages collect incomplete content within their training data sets, for example, only three fields including ‘the timestamp’, ‘CAN ID’, ‘DLC’, are collected and Data [1]-Data [7] in CAN message are empty. In such situation, targeted attack scenarios are often hard to detect by network intrusion detection systems because the specific payload is missed in the training data.

-------------------------------------------------------------------------------------------------------

Q5) Please explore more on how you develop and create "General architecture of a basic classifier".  

Answer: Add the detailed description of Figure 3 as follows. (Located at the following text of Figure 3) As shown in Figure 3, a basic classifier in ML models use the selected feature vectors from the training instances to train the model, determine the hyperparameters of the trained model, and predict the possible categories of test data. However, a single basic classifier does not sufficiently match the data distribution in large-scale networks with complex intrusion patterns.  

Response:

1. Please add more detail arguments why you wrote "Figure 3, a basic classifier in ML models use the selected feature vectors from the training instances to train the model"
2. Please add with detail of " the hyperparameters of the trained model, and predict the possible categories of test data."  

Answer:

1. Revise ‘basic classifier’ to base classifier for consistency
2. Add the details of (1) the selected feature vectors from the training instances to train the model (2) hyperparameter tuning of the trained model, as follows

As shown in Figure 3, a base (individual) classifier in ML models use the selected feature vectors from the training instances to train the model, determine the hyperparameters of the trained model, and predict the possible categories of test data. In training process, there are important topics for ML model: (i) feature selection. Feature selection is the process of reducing the number of input variables to develop a predictive model using selected feature set to improve the performance of the model by reducing the computational costs of modelling. [37] (ii) hyperparameter tuning. Essentially, the prediction performance of the machine learning network model is influenced quite heavily by the choice of hyperparameters, hence it can corporate the grid search optimisation process to improve the searching efficiency in model development. [38]

Author Response File: Author Response.pdf

Round 3

Reviewer 2 Report

The revisions was done base on our comments.