Feature-Enhanced Document-Level Relation Extraction in Threat Intelligence with Knowledge Distillation
Round 1
Reviewer 1 Report
1. Kindly highlight the novelty of the proposed method.
2. Proofread the entire manuscript once.
3. Check the Quality of the figures, all figures must be in HD. Modify figure 1 it must be in the standard format.
4. Check table 2, 5a, and 5b alignment.
5. Use formulas in correct numbering for ex, check 8 and 10.
6. Proposed method? Algorithm?
7. Redraw figure 4 and 5 Quality is low. Check the used place and flow is missing.
8. Improve conclusion and future improvements
Refer:
10.1002/ett.4108
10.1109/TII.2022.3142306
10.1016/j.comcom.2019.12.015
Author Response
Please see the attachment.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
The abstract is poorly written. Very generic statements are used and sometimes have no meaning, such as
"Cyber threat intelligence contains large amounts of knowledge about threat behavior, promoting the transformation of defense from passive to active."
" Due to the lack of publicly avaliable datasets in the domain of threat intelligence..."
The abstract needs to be rewritten in a professional, scientific research manner.
The abstract has several grammatical errors and typos. For example:
...avaliable ==> available
..constructing knowledge graph ==constructing a knowledge graph
...and hack forums. ==> and hacking forums.
...constructed an pair-level document...
many others.
Many informative statements are provided without any citation for example (not limited to, please check all of them):
On February 2022, Ukrainian government agencies and bank- 29 ing websites were targeted by large-scale distributed denial of service (DDoS) attacks, 30 resulting in the offiline of 10 websites at least.
In March, ransomware infections occurred 31 in Rompetrol, a petrol station in Romania. Later, ransomware was launched on Deutsche Windtechnik AG in April, which paralyzed the Remote Control System of about 2000 wind turbines in Germany for a whole day.
Knowledge of threat intelligence originates from security analysis reports, blogs, so- 40 cial media, etc., which provides powerful data support for situational awareness and ac- 41 tive network defense.
......
Abbreviations should be defined at first use, such as BERT line 68, ...
The manuscript is full of typos and grammatical issues. The manuscript requires a sold proofreading and writing enhancement.
Figures 2, 3, 4 and 5 are the core Figures of this manuscript. The authors need to explain them clearly and not just list them without complete elaboration.
Figures 4 and 5 are not readable.
This statement is repeated too many times "Due to the lack of publicly available datasets in the domain of threat intelligence"
The conclusion is simply a summary of what you have done but you need to show the impacts and implications of your study.
The future directions of the authors’ contribution are not provided. I suggest the authors state some explicit future recommendations to improve or upgrade their model by other researchers.
More important results can be adopted in this manuscript, including the classification accuracy, classification overhead, and confusion matrix.
The dataset seems imbalanced; how do you cope with this?
Also, please consider discussing some practical implications and potential applications of the proposed IDS system.
Experimental environment, setup, and tools need to be provided as a separate section to help other researchers to develop similar models
In my opinion, getting 50-53% of the overall F Score is not a competent model. The authors should justify why such results will be important to be published (Provide a strong justification)
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Round 2
Reviewer 2 Report
Figures 4 and 5 are not readable. You need to re-draw both in much higher resolution.
Regarding my last comment in the previous revision, I suggest that authors use some balancing techniques such as SMOT or over/under-sampling, which I think will improve the performance of their system. After that, report on the new performance indicators. I am convinced that your 53% F Score is very low and has a minor contribution to the related research realm.
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Round 3
Reviewer 2 Report
I suggest considering other sample balancing techniques, I think you still can have better performance
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
