Automatic Bug Triaging via Deep Reinforcement Learning
Round 1
Reviewer 1 Report
The authors propose a bug triaging model based on RL, in which feature extraction is done using neural networks. The proposed system (BT-RL) has as output a bug report, online and in real-time. Authors use a bidirectional RNN that integrates multidimensional features that is compared with SVM. Authors stated that their proposal overcame similar studies.
Authors give emphasis that the solution can be applied in real time. A better explanation, or highlighted how the report is generated online and real-time is desirable. Maybe a figure explaining that process in a high level context (no network architecture)
In general, the work presents a methodology that has some contribution, the main steps are explained, but some improvements for a better understanding are recommendable;
The organization of the paper is not easy to understand. Introduction is too long containing data that can be part of other section; The contextualization of the problem and proposed solution is not easy to understand. The location of the section 4 (and maybe 5) is not the most adequate from my point of view. In general, IMO, the paper requires a deep reorganization.
The format and presentation of the paper need to be improved. Tables are divided in different pages, spaces between paragraphs are not right, etc… Figures also not rigt.
The title of section 2 ,should be more highlighted… In general, titles and subtitles are short (abbreviations), and they should be completed for a better explanation
In Eq. (5) justify the range of values of K
Only accuracy was used to performance evaluation, as can be observed in Tables 3 – 7. Why other performance validation metrics that are widely adopted in ML were not used ?
Accuracy results look like lower performance. Are those, the best result foun in the literature? (it is only a curiosity) . I mean, authors show a considerable accuracy increment in percentage, but the base values are low.
A proofreading is recommendable.
Author Response
We sincerely thank you for your valuable feedback that we have used to improve the quality of our manuscript. We have carefully reviewed the comments and have revised the manuscript accordingly, and you could find the point-to-point revise detail in the file of Reply to Reviewer 1.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
The paper is devoted to improving the process of bug triaging. The authors propose two new models for bug triaging: : a deep multisemantic feature (DMSF) fusion model and an online dynamic matching (ODM) model. They were used in experiments with open source datasets (covering 4 projects).
I suggest some refinements of the text: table 1 on page 2 – what do you mean by components? Files, functions their size??? The specification of the last two columns is not clear – needs refinement; lines 61, 62 - need refinement; line 71 – replace repots by reports; lines 91-95 - better comment of the presented numbers and percent is needed; lines161-162 – more substantive explanation of bug reports is needed, e.g. product, component moreover the text is a little bit clumsy. Lines 196-202 – selection of only 25 bugs should be justified and discussed; page 7 - I do not see reference to algorithm 1?? Lines 205-208 - need refinement; line 257 - parametr t is skipped it should appear after word time; page 10 tab. 2 - how do you calculate vocabulary length what do you consider as a word (any sequence of characters or only those from English thesaurus???); caption of table 3 replace acurrary by accuracy, moreover this notion should be commented in a more intuitive way. Caption of tab. 5 replace Tired by Third;
Careful checking of the text is needed to improve English.
The advantage of using your approach is not sufficiently explained. In particular, how many defects in the analysed projects were allocated not correctly and how many of them could be allocated more reasonably and what would be the gain in bug handling time??? In references dominate rather older publication below 2012, I Suggest replacing them by more newer ones including those devoted to bug handling issues, e.g. from Information and software technology journal (Elsevier), Journal of Computer Science and Technology (Springer).
Author Response
We sincerely thank you for your valuable feedback that we have used to improve the quality of our manuscript. We have carefully reviewed the comments and have revised the manuscript accordingly, and you could find the point-to-point revise detail in the file of Reply to Reviewer 2.
Author Response File: Author Response.pdf
Round 2
Reviewer 1 Report
I think authors properly answered all my comments/questions. Thus, the paper can be accepted as the present form.
