Semi-Automatic Approaches for Exploiting Shifter Patterns in Domain-Specific Sentiment Analysis

Round 1

Reviewer 1 Report

The authors compare two approaches to sentiment analysis in Portuguese. I appreciate the author's choice of the problem as it is very relevant in present times, however, please note the following.

1. Can you provide more details about the corpus collection? What annotation guidelines did you use? How did you calculate an agreement between annotators? Did you achieve the full consent of the annotators?

2. Can you provide the statistics for the dataset?

3. Did you use any libraries for preprocessing? If yes, it should be cited.

4. What specific BERT did you use? Cased/uncased? Maybe it should be better to use BERTimbau instead of multilingual BERT?

5. The authors are suggested to provide an Error Analysis section under the qualitative analysis of the predictions from the two models to explain the limitations of each model.

Author Response

"Please see the attachment.

Author Response File: Author Response.docx

Reviewer 2 Report

The paper's heart is symbolic approach based on exploring shifter patterns and rules and compared with deep learning method. Through experiments, the authors show the proposed method is as good as the one founded on deep learning method. In general, the work contains lots of interests. The authors need to improve the following parts to render the paper more perfect:

1. The above assertions need double checking :

“The drawback of these approaches is that they offer no explanation on how the prediction is achieved and hence are often referred to as black-box methods.”

⇨    We can use Local Interpretable Model-Agnostic Explanations (LIME) to interpret the results.

“Many of these patterns are general, not specific to a particular domain, and could be applied across domains and languages. So, these patterns constitute a linguistic knowledge, which is generally useful and could be transferred to other domains.”

⇨    I am not certain about this assertion. If we use Vietnamese language, the results will be very different. Besides, the words whose meaning is vague should be avoided, such as many, generally…

2. It is advisable to combine sections 3, 4, and 5 to form one section because the paper has been divided into numerous sections.
3. Deep learning method (6. Applying a deep learning approach to sentiment analysis) had better be put in a subsection as this method is only to be compared with the main method .
4. It is important to describe clearly datasets in view of their determining characteristics to the paper.
5. The authors need to explain why they have chosen mF1 but other metrics? Have they tried other metrics? If so, what were the results?
6. Some works have combined the two methods are proposed by the authors and got good results. They should cite and discuss them in future works.

7. Are the datasets too small for deep learning methods to manifest their strength? Furthermore, the paper’s scope is “Domain-specific Sentiment Analysis, which is an advantage for the method based on rules.
8. It is not necessary for there to be too many paragraphs.
9. They must explain in English all the terms in Portuguese. For example, in the Table 7. Some shifter patterns that include reversal/inversion.

Author Response

"Please see the attachment.

Author Response File: Author Response.docx

Round 2

Reviewer 2 Report

Authors replied to the queries and added in the article.