Multimodal Emotional Classification Based on Meaningful Learning

Round 1

Reviewer 1 Report

The manuscript is clearly written and easy to follow. The topic is interesting. However, The accuracy results in table 4 are misleading. When compared to single modalities, bimodality appeared to significantly improve emotion recognition. It merely indicated that the emotion detection of the single modality wasn't well suited by the deep learning technology that was being given. If the appropriate deep/machine learning method is implemented correctly, the accuracy could be significantly higher with a single modality. Numerous relevant academic works are available, such as https://onlinelibrary.wiley.com/doi/full/10.1002/eng2.12189. To prevent readers from misinterpreting the results, it is advised that the authors provide a more thorough explanation of the findings.

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Reviewer 2 Report

1. The main challenges of Multimodal Emotion Recognition (MER)  studied in this paper are not well articulated. The few approaches are already able to justify MER with higher accuracy. The main differences and challenges of this particular MER mission need to be supplemented.

2. In Section 3, the contribution of this paper is not novel enough, CNN, LSTM, and MNN are all methods that are widely used in classification. This paper may achieve a better fusion application of these techniques, but it lacks innovative improvements to these methods.

3. In Section 2, although the deficiencies of each related work are analyzed, a summary of the overall deficiencies of related research work and a statement of the differences in this work are lacking. 

4. In Section 3.1, the description of CNN is very brief and seems to be indistinguishable from the standard CNN method. 

5. How are TP, FP, and FN in section 5.3 defined? No specific explanation can be found in the paper.

6. In Figure 6 and 9 the training cost and training accuracy is somehow similar to audio or bimodal, but the accuracy of text only vs audio only vs bimodal has a vast difference, there is no explanation for the lower accuracy achieved especially for text.

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Reviewer 3 Report

The paper is titled – “Multimodal Emotional Classification Based On Meaningful Learning”. The authors have proposed an approach that uses Meaningful Neural Network for prediction of emotions during conversations. The work seems novel. However, the presentation of the paper needs improvement. It is suggested that the authors make the necessary changes/updates to their paper as per the following comments:

1. Missing references: Several fact-based statements throughout the paper are missing references. For instance, in the Introduction section the authors provide a definition of emotions. The authors are not the first researchers in this field to define what emotions are. So, this definition should have a supporting reference.

2. On page 2, the authors discuss some advances of Deep Learning. However, the references cited to discuss these advances are not recent ones. For instance, [10] was published 16 years ago, [8] was published 8 years ago and so on. Furthermore, recent applications of deep learning such as ambient assisted living, applications for COVID-19 research, etc. are not even mentioned. Cite these two recent papers on deep learning that focus on ambient assisted living (https://doi.org/10.3390/jsan10030039) and COVID-19 (https://doi.org/10.3390/idr13020032) and briefly discuss such recent applications of deep learning.  

3. The challenges and limitations in prior works should be clearly discussed in Section 2.

4. The working of the LSTM cell should be elaborated. To add, the LSTM cell diagram (Figure 1) is difficult to understand. Please provide a high resolution version of this figure. The font-size used to state the specific variables in this Figure, should also be increased.

5. Discussion of results needs improvement. In Table 6, the results are compared with only 4 prior works in this field. However, 14 papers were reviewed in the Literature Review section. Please explain why the results from the remaining 10 papers were not used in this comparison study.

6. A comprehensive proofreading of the paper is necessary: There are several grammatical errors in the paper. To add, there are multiple spelling mistakes as well. For instance, in the title of the paper, “Meaningful” is spelled as “Meagninful”

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Round 2

Reviewer 3 Report

The paper has been significantly updated as per most of my comments from the previous review round. I have a couple of comments at this point:

1.       The newly added references on COVID-19 i.e. references [11] and [22] do not present studies conducted on large-scale data. So, I suggest replacing one of them with this paper - https://doi.org/10.3390/covid2080076 which presents a study conducted on large-scale data

2.       Regarding Response 5, this recent paper proposes a similar work - https://doi.org/10.1109/TAFFC.2022.3141237 using the same dataset, so I suggest including this in the comparison study with similar works using this dataset.

 

3.       For the dataset, i.e. reference [13], the authors have cited the Arxiv version of the paper. The paper was later on published as a conference proceeding - https://aclanthology.org/P19-1050/. Consider citing this version of the paper as it is the peer-reviewed version. 

Author Response

Please see the attachment

Author Response File: Author Response.pdf