Privacy-Preserving Tampering Detection in Automotive Systems
Round 1
Reviewer 1 Report
The paper proposes a methodology for data privacy protection and privacy-preserving tampering detection in automotive systems. Specifically, the authors introduce an approach based on Fast Fourier Transform toghether with filters and Forest-based regression.
The paper is well written and quite easy to follow, however, there are some comments:
- it lacks of a good introduction to the problem statement. Indeed, the reader does not understand properly how the privacy-preserving problem and the tampering-dectection are related and, as-is, they appear a bit uncorrelated. Such issue tends to overshadow the paper contribution that is not clear if is just incremental through an empirical study. I would suggest to review the introduction and clarify better how these two problems are related and explain better the contribution.
- the related work section is not exhaustive and a lot of references are quite old. From the privacy-preserving point of view, it would be interesting to see a comparison with some other work for automotive system that appoach a solution through a by-design approach like Cloud2Edge (Grigorescu, Sorin, et al. Sensors 20.19 (2020): 5450).
- the experimental evaluation is not easy to understand. The figures present labels too small and difficult to read. Furthermore, each result is not immediate and could be explained better. Is this solution general enough or it works only on the specific environment the authors evaluate? I would spend some words on that as well.
In any case, I appreciate the paper and with a review addressing such aspects I think it can be accepted.
A minor comment: I notice a typo at row 136: - a space is missing after the word transform: Fourier transform(DFT) -> Fourier transform (DFT)
Author Response
ACKNOWLEDGMENT
------------------
The authors would like to thank the reviewer for the valuable remarks and the opportunity to defend this article. The highly appreciated comments provided us with the chance to improve the quality of this work through the revised manuscript. We sincerely hope that the findings in this research will help the community to design privacy-aware tampering detection techniques. We, therefore, kindly ask the reviewer to take into consideration this work so that it would eventually be a good candidate for publication in the MDPI Electronics Journal.
For clarity, we marked the response to reviewer's comments with "RESPONSE: ".
Reviewer' Comments:
--------------------
The paper proposes a methodology for data privacy protection and privacy-preserving tampering detection in automotive systems. Specifically, the authors introduce an approach based on Fast Fourier Transform together with filters and Forest-based regression.
The paper is well written and quite easy to follow, however, there are some comments:
1. it lacks of a good introduction to the problem statement. Indeed, the reader does not understand properly how the privacy-preserving problem and the tampering-dectection are related and, as-is, they appear a bit uncorrelated. Such issue tends to overshadow the paper contribution that is not clear if is just incremental through an empirical study. I would suggest to review the introduction and clarify better how these two problems are related and explain better the contribution.
RESPONSE:
The reviewer's comment is welcome since it allows us better to explain the utility and applicability of the proposed method. The revised manuscript addresses the reviewer's observation with additional comments that have been added to the paper. Accordingly, the third paragraph in the introduction, was restructured so that it now better explains that, in some cases, tampering detection has to be performed outside of the vehicle, and, in such scenarios, data leaving the vehicle needs to be protected as it can lead to private information leakage. In the fourth paragraph, we further emphasize that a simple, but effective data perturbation/anonymization method can be suitable in the tampering detection scenario.
2. the related work section is not exhaustive and a lot of references are quite old. From the privacy-preserving point of view, it would be interesting to see a comparison with some other work for automotive system that appoach a solution through a by-design approach like Cloud2Edge (Grigorescu, Sorin, et al. Sensors 20.19 (2020): 5450).
RESPONSE:
We understand and agree with the reviewer's concern. We have restructured the "Related Work" section to present the studied approaches better. In addition, the revised manuscript includes more information related to the available data protection methods such as: mentioned encryption, anonymization, and differential privacy (DP) with more recent references. Also, we have added the suggested work related to other possible privacy-preserving approaches.
3. the experimental evaluation is not easy to understand. The figures present labels too small and difficult to read. Furthermore, each result is not immediate and could be explained better.
RESPONSE:
We agree with the reviewer's observation. We have updated all figures with higher resolution, bigger fonts in the revised manuscript. Additionally, we revised the figure descriptions.
4. Is this solution general enough or it works only on the specific environment the authors evaluate? I would spend some words on that as well.
RESPONSE:
In order to address the reviewer's well-justified comment, the last paragraph of Section 2.2 was extended with a discussion on the applicability of the approach to other scenarios as well. Here, we state that its linear complexity makes is suitable to other techniques involving data transmissions outside the vehicle (e.g., smart parking, vehicle to cloud communications). Howeber, for each scenario, the data utility needs to be further analyzed since other applications may require adjusting the method's parameters.
5. A minor comment: I notice a typo at row 136: - a space is missing after the word transform: Fourier transform(DFT) -> Fourier transform (DFT)
RESPONSE:
Thank you for mentioning the typo. This was corrected in the revised material.
Reviewer 2 Report
The authors focus their study in the field of data privacy protection among the components in a vehicle by leveraging the Fast Fourier transform to support the data transformation enabled by filter and additional transformations. Furthermore, the authors adopt a random forest based regression technique to support the statistical analysis for tampering detection in the extreme case of anonymized data.
The provided mathematical analysis is concrete, complete, and correct and the authors have well thought out their main contributions.
The authors have provided a well thought out approach in order to deal with the proposed research problem and they have presented some indicative numerical results in order to show its pure operation and performance. The manuscript is overall well written and easy to follow and the authors have also well thought out their contributions.
The description of the operation of the overall security and trust-based approach that is proposed is detailed and the authors have provided a detailed set of numerical results in order to show the main characteristics of the proposed framework and its main operation. The authors should consider the following suggestions provided by the reviewer in order to improve the scientific depth of their manuscript, as well as they should address the following comments in order to improve the quality of presentation of their manuscript.
Initially, the authors should improve the provided related work in section 1 and 2 which is very limited and not targeted for a journal paper. The authors are encouraged to introduce the related work by using more summative language in order to categorize the existing research works in the examined field and present them to the reader.
Furthermore, the authors should discuss application scenarios, such as Tsiropoulou, E.E., et al. "RFID-based smart parking management system." Cyber-Physical Systems 3.1-4 (2017): 22-41, where the proposed approach can be implemented and solve realistic problems.
In the section of the numerical results, the authors should present some scalability analysis results in order to show the efficiency and robustness of the proposed framework.
In Section 3, the authors should include an additional subsection providing the theoretical analysis of the computational complexity of the algorithmic components that implement the overall framework. The previous comments should be complemented with some additional numerical results showing the real execution time of the proposed framework in order to justify if it can be implemented in a real time or close to real time manner.
The figures of the numerical results need to be improved in terms of their presentation.
Finally, the overall manuscript should be checked for typos, syntax, and grammar errors in order to improve the quality of its presentation.
Author Response
ACKNOWLEDGMENT
------------------
The authors would like to thank the reviewer for the valuable remarks and the opportunity to defend this article. The highly appreciated comments provided us with the chance to improve the quality of this work through the revised manuscript. We sincerely hope that the findings in this research will help the community to design privacy-aware tampering detection techniques. We, therefore, kindly ask the reviewer to take into consideration this work so that it would eventually be a good candidate for publication in the MDPI Electronics Journal.
For clarity, we marked the response to reviewer's comments with "RESPONSE: ".
Reviewer' Comments:
--------------------
The authors focus their study in the field of data privacy protection among the components in a vehicle by leveraging the Fast Fourier transform to support the data transformation enabled by filter and additional transformations. Furthermore, the authors adopt a random forest based regression technique to support the statistical analysis for tampering detection in the extreme case of anonymized data.
The provided mathematical analysis is concrete, complete, and correct and the authors have well thought out their main contributions.
The authors have provided a well thought out approach in order to deal with the proposed research problem and they have presented some indicative numerical results in order to show its pure operation and performance. The manuscript is overall well written and easy to follow and the authors have also well thought out their contributions.
The description of the operation of the overall security and trust-based approach that is proposed is detailed and the authors have provided a detailed set of numerical results in order to show the main characteristics of the proposed framework and its main operation. The authors should consider the following suggestions provided by the reviewer in order to improve the scientific depth of their manuscript, as well as they should address the following comments in order to improve the quality of presentation of their manuscript.
1. Initially, the authors should improve the provided related work in section 1 and 2 which is very limited and not targeted for a journal paper. The authors are encouraged to introduce the related work by using more summative language in order to categorize the existing research works in the examined field and present them to the reader. Furthermore, the authors should discuss application scenarios, such as Tsiropoulou, E.E., et al. "RFID-based smart parking management system." Cyber-Physical Systems 3.1-4 (2017): 22-41, where the proposed approach can be implemented and solve realistic problems.
RESPONSE:
We understand and agree with the reviewer's concern. We have restructured the "Related Work" section to present the studied approaches better. Accordingly, section 2.1 outlines the background, it introduces the terminology and describes well-known (traditional techniques). Conversely, section 2.2 focuses on recent advancements in the field.
According to the reviewer's suggestion, the revised manuscript was extended with additional references and discussions on related techniques. More specifically, section 2.1 was extended with a discussion on differential privacy (lines 145-150), and section 2.2 was extended with a discussion on possible applications of the developed data anonymization methodology (lines 235-239). The added sections include additional related work, including the suggested work by Tsiropoulou, E.E., et al.
2. In the section of the numerical results, the authors should present some scalability analysis results in order to show the efficiency and robustness of the proposed framework.
RESPONSE:
The reviewer's comment is welcome since it allows us better to explain the utility and applicability of the proposed method. The revised manuscript was extended with the scalability analysis included as the last paragraph of Section 4.3. A new figure (Figure 15) was also added to summarize the scalability analysis in terms of execution time. According to the analysis, the approach exhibits a linear behaviour with respect to the number of features and the allocated window size (i.e., the number of samples).
3. In Section 3, the authors should include an additional subsection providing the theoretical analysis of the computational complexity of the algorithmic components that implement the overall framework. The previous comments should be complemented with some additional numerical results showing the real execution time of the proposed framework in order to justify if it can be implemented in a real time or close to real time manner.
RESPONSE:
We thank the reviewer for his comment. With regard to the the computational complexity of the proposed algorithm, the revised manuscript now includes Section 3.5. Additional numerical results showing the execution time have been listed in the scalability analysis, as mentioned above.
4. The figures of the numerical results need to be improved in terms of their presentation.
RESPONSE:
We agree with the reviewer's observation. We have updated all figures with higher resolution, bigger fonts in the revised manuscript. Additionally, we revised the figure descriptions.
5. Finally, the overall manuscript should be checked for typos, syntax, and grammar errors in order to improve the quality of its presentation.
RESPONSE:
Thank you for mentioning these issues. We have further improved the manuscript and tried to identify all typos, syntax/grammar errors.
Round 2
Reviewer 2 Report
The authors have addressed in detail the reviewers’ comments and the quality of presentation, as well as the scientific depth of the paper have been substantially improved. This reviewer has no further concerns regarding the publication of this paper.
