Deepsign: Sign Language Detection and Recognition Using Deep Learning

Round 1

Reviewer 1 Report

The paper presents a deep learning-based model that detects and recognizes 16 the words from a person’s gestures. 

The proposed method and its details are well described, allowing a fast implementation. From the current presentation, the novelty of proposed method seems limited but minor contribution has been added on the literature . In addition the paper is well organized. 

Author Response

Thank you for your comments on the review. 

Reviewer 2 Report

Dear Authors,

   Thanks so much for your manuscript submission to MDPI Journal of Electronics. This paper is fairly written, and some of the research study are relatively good. It presented sign language recognition for Indian, the technical approaches are LSTM and GRU, and tested IISL 2020 dataset over a variety of gestures. The authors claimed that their method outperforms all the other available models for commonly used words, increasing the layers of LSTM and applying LSTM followed by GRU may achieve even higher accuracy for recogniting ISL. Besides, the authors also prospected that their approach fits for different datasets under ideal conditions as well as being utilized for interpreting continuous sign language, etc. I noticed that this set of work has some established framework and quantitative evaluations on the proposed scheme and a few state-of-the-arts, while their research study has some problematic issues to be addressed. I may recommend this paper as "Accept with minor edits" in the first round of review, in condition that each of the problems got carefully fixed in the updated version.

   Major aspects suggested for further improvement:

   a) Abstract: the current version only contains ~120 words, the statements are quite generic. Based on the actual load of contributions, I suggest the authors to expand the abstract to ~180 words. Specific details of your approach (i.e., how do you combine LSTM and GRU, what innovations of your method is superior to other approaches?) and keynote quantative results on concluding remarks, need to be supplemented. Thanks a lot!

   b) Introduction: I reviewed the four paragraphs, in my opinion, the authors can be a little bit more specific on the category of sign language based recognition schemes and specifically, common word based sign recognition. Meanwhile, professional papers have a clear statement (in the second last paragraph) on the main summary of contributions on your research study, in 2-4 manifolds (and binded with some specific details), which is suggested for supplementation.

   c) Literal Review (or better be "Related work"): Two problems should be addressed. One is that very few latest work (2020 and 2022) summarized in this version, the other is that the descriptions looks unprofessional and a lot of verbose words contained. My suggestions: any prior work (before 2018) can be condensed and merged to the Introduction section; the state-of-the art work (2019-2022) needs to be re-arranged in 2-3 paragraphs (don't keep the one paragraph per work style!) with emphasis on the classified schemes based on different models. A few more specific details should be addressed on each manifold, the last paragraph (Lines 183-186) should be expanded.

   d) Figures and Tables: the image resolution of some characters in Fig. 1 should be enhanced. The current version contains some blurs, and some transparent distortions should be calibrated. Table 2 and Figure 5 crossed over two pages, please fix that issue in your later edits. Also, I think some figures and tables should take up less space (as MDPI template specified).

   e) Experiments and Results: I suggest that some subsections need to be adjusted. For instance, the quantitative analysis as specified in Figure 6, looks generic and lacks evidence. The authors also missed a subsection of sensitivity analysis or ablation study, which may include variation of parameters and cover the related figures and tables in Sections 3-4. The suggested subsection can be cascaded in before Section 5 (right after sub- section 4.2). PS: I think the valid digits of each numerical score could also be improved, i.e., using 3 valid digits as uniform style; i.e., 0.890, 95.4%, etc.

   f) Discussions: Concise while not very specific. Discussions should not just limited to Table 1. The second paragraph on limitations of study can be further extended. Please consider a major rewrite on this section. 

   g) Conclusions: the current version may also require a major re-write. It may have missed some concluding remarks on the keynote quantitative results; the single paragraph need to include some specific statements. Also, while the authors missed specifications on some opening questions, I suggest the authors split this paragraph, make it separate for a second paragraph (starting from "In future research" at current Line 370) with the summary of research challenges and future study orientations, etc. In a word, please consider reshaping this crucial section, which should also be of significant difference from the Abstract session.

   h) References: a) the authors need to adjust each with the professional standards on citing journal and conference proceedings (by checking the most recent MDPI template) for some citations; b) the authors missed some detailed information on Refs. [1] and [18], regarding those "online available" references, the citation style on volumes and numbers did not comply with the format MDPI template specified; c) a few more latest publications in Years 2019-2022 which are similar / parallel to your study area on word / speech recognition algorithms, typically deep learning based approaches for gesture / sign language recognition, are supposed to be supplemented in your updates.  

   Minor problematic issues to be addressed in the revised version:

   a) Please stop hyphenating a word (which currently appears multiple time at the end of some lines to cross-over two adjacent lines). MS word file of MDPI online template has the options to adjust that. Thanks a lot!

   b) In some sections, the literal quality of English can be improved. I would recommend the peer-reviewed authors to polish the literal aspects of this research article, including grammatical checking and careful proofreading. 
   
   c) Apply uniform font size and style on the characters of each figures, and fix the remaining formatting issues in the proofreading process.

   Once again, wish you the best of luck for paper coming into acceptance. Thank you for your interests on publishing at MDPI Journal of Electronics. Stay well and take care!

Best regards,

Yours sincerely,

Author Response

Point 1: Abstract: the current version only contains ~120 words, the statements are quite generic. Based on the actual load of contributions, I suggest the authors to expand the abstract to ~180 words. Specific details of your approach (i.e., how do you combine LSTM and GRU, what innovations of your method is superior to other approaches?) and keynote quantative results on concluding remarks, need to be supplemented. Thanks a lot!

Response 1: Abstract has been modifide as per the comments

 

Point 2: Introduction: I reviewed the four paragraphs, in my opinion, the authors can be a little bit more specific on the category of sign language based recognition schemes and specifically, common word based sign recognition. Meanwhile, professional papers have a clear statement (in the second last paragraph) on the main summary of contributions on your research study, in 2-4 manifolds (and binded with some specific details), which is suggested for supplementation.

Response 2: Sign language categories have been explained in line no 45 to 50

 

Point 3 :  Literal Review (or better be "Related work"): Two problems should be addressed. One is that very few latest work (2020 and 2022) summarized in this version, the other is that the descriptions looks unprofessional and a lot of verbose words contained. My suggestions: any prior work (before 2018) can be condensed and merged to the Introduction section; the state-of-the art work (2019-2022) needs to be re-arranged in 2-3 paragraphs (don't keep the one paragraph per work style!) with emphasis on the classified schemes based on different models. A few more specific details should be addressed on each manifold, the last paragraph (Lines 183-186) should be expanded.

Response 3: Related work section has been modifided and the updated work on sign language over transformer network from line no 189 to 192 has been added

 

Point 4 : Figures and Tables: the image resolution of some characters in Fig. 1 should be enhanced. The current version contains some blurs, and some transparent distortions should be calibrated. Table 2 and Figure 5 crossed over two pages, please fix that issue in your later edits. Also, I think some figures and tables should take up less space (as MDPI template specified).

Response 4: The figure 1 has been changed with high resolution, figure 5 and table 2 are also set properly

 

Point 5: Experiments and Results: I suggest that some subsections need to be adjusted. For instance, the quantitative analysis as specified in Figure 6, looks generic and lacks evidence. The authors also missed a subsection of sensitivity analysis or ablation study, which may include variation of parameters and cover the related figures and tables in Sections 3-4. The suggested subsection can be cascaded in before Section 5 (right after sub- section 4.2). PS: I think the valid digits of each numerical score could also be improved, i.e., using 3 valid digits as uniform style; i.e., 0.890, 95.4%, etc.

Response 5: The figure 6 and relevent content has been modifide as per comment from line 357 to 359

 

 

Point 6: Conclusions: the current version may also require a major re-write. It may have missed some concluding remarks on the keynote quantitative results; the single paragraph need to include some specific statements. Also, while the authors missed specifications on some opening questions, I suggest the authors split this paragraph, make it separate for a second paragraph (starting from "In future research" at current Line 370) with the summary of research challenges and future study orientations, etc. In a word, please consider reshaping this crucial section, which should also be of significant difference from the Abstract session.

Response 6: As per comment, the conclusion is divided in two paragraph with modification in content .

 

Author Response File: Author Response.pdf

Reviewer 3 Report

-be careful with reference listing. For example different formats are used (see line 76). Please amend and uniform the reference formats 

-line 188: in proposals is not a scientifically sounding expression. Please amend. 

-Results presention is unclear. What are the confusion matrix depicted in Figure 5 reporting? Metrics for tests? Which folds? Everything should be better specified also in the caption 

-Figure 6 is not clear. Please amend e insert more details 

-Line 350, what is the controlling environment? Please rephrase and amend 

-References should be improved and enlarged.

Author Response

Point 1 be careful with reference listing. For example different formats are used (see line 76). Please amend and uniform the reference formats 

Response 1: Reference has been changed as per template in all related work with yellow highlight

 

Point 2: line 188: in proposals is not a scientifically sounding expression. Please amend. 

Response 2: Scientific explaination has been cover in explination of LSTM and GRU with figure 2 

 

Point 3 : Results presention is unclear. What are the confusion matrix depicted in Figure 5 reporting? Metrics for tests? Which folds? Everything should be better specified also in the caption 

Response 3: The caption has been modified and also highlighted in line no 357 to 359

 

Point 4 : Figure 6 is not clear. Please amend e insert more details 

Response 4: The contents are added at line no 357 to 359 and the value also added scientifically,

 

Point 5: Line 350, what is the controlling environment? Please rephrase and amend 

Response 5: The contents have been updated and highlighted at line 367.

 

 

Point 6: References should be improved and enlarged.

Response 6: Added few related work in section 2.

 

Author Response File: Author Response.pdf

Round 2

Reviewer 3 Report

I would like to thank the authors for having addressed my comments. I would only suggest to re-read and edit some english, as well as double checking the references

Author Response

Dear Reviewer/Editor,   All suggested changes are incorporated and the paper has been improved a lot accordingly.

Author Response File: Author Response.pdf