The Status and Trend of Chinese News Forecast Based on Graph Convolutional Network Pooling Algorithm
, Jing Peng 3, Tailai Peng 1,2, Rui Chen 1,2, Boyuan Hou 1,2, Xinran Xie 1,2 and Zhe Cui 1,2,*Round 1
Reviewer 1 Report
This paper proposes a methodology to classify trends of Chinese news based on graph convolutional networks.
The topic is relevant and the paper is well written and motivated.
However, I believe section 3 - Results -- needs to be extended and the obtained results should be discussed in more detail:
- Annotation of the dataset: three "taggers" annotated the set of news but there is no information about the degree of agreement between them. A measure of agreement, such as the kappa measure, should be calculated and presented. Moreover, a unique dataset should be created (not 3 distinct ones), through a consensus between the annotators (or a voting mechanism).
- The dataset is not described in detail: how many examples from each class? How was the evaluation performed? (training/test datasets? cross-validation?)
- Only accuracy is presented: precision and recall (and f-measure) are measures which allow a better understanding of the behavior of the classifiers and they should be presented.
- "Typical" text classifiers based on word embeddings obtain a much better performance. So, why to transform text into graphs and, then, classify graphs? (note that your graph approach also uses bert embeddings for the nodes; it is not true that it does not use pre-trained models).
- Related with the previous point, it would be important to analyze the main sources of errors of your approach: when and why does it fail? Are there any classes of errors? Is it a problem of the text to graph transformation? If yes, which kind of information is lost and how to overcome this problem?
- I strongly suggest the authors to make a deeper analysis of the error situations and to try to extend the text to graph transformation in order to try to obtain better results.
Author Response
Dear reviewer1：
Re: Manuscript ID: applsci-1487466 and Title: The Status and Trend of Chinese News Forecast Based on Graph Convolutional Network Pooling Algorithm
Please see the attachment.
Thank you for your letter and the reviewers’comments concerning our manuscript entitled “applsci-1487466”. Those comments are valuable and very helpful. We have read through comments carefully and have made corrections. Based on the instructions provided in your letter, we uploaded the file of the revised manuscript. Revisions in the text are shown using red highlight for additions. The responses to the reviewer's comments are marked in red and presented following.
We would love to thank you for allowing us to resubmit a revised copy of the manuscript and we highly appreciate your time and consideration.
Sincerely. New Years greetings and best wishes！
Xiao Han, et al.
Point 1: Annotation of the dataset: three "taggers" annotated the set of news but there is no information about the degree of agreement between them. A measure of agreement, such as the kappa measure, should be calculated and presented. Moreover, a unique dataset should be created (not 3 distinct ones), through a consensus between the annotators (or a voting mechanism).
Response 1: Your suggestion is very correct. After discussion, we decided to adopt a voting mechanism to vote the data marked by different taggers. The principle is that if the judgments of the three taggers are consistent, we think their marking is true. If two news items are consistent, the other is different, or the labels of the three taggers are different, we will determine the final trend judgment of the news through joint discussion among the authors. We have updated the dataset and described it in Chapter 3.1 of the paper.
Point 2: The dataset is not described in detail: how many examples from each class? How was the evaluation performed? (training/test datasets? cross-validation?)
Response 2: According to the second point, we supplement the detailed data in each category in detail in Section 3.2 of the paper. The evaluation work randomly selects 90% of the data set as a training set and the rest 10% as a test set. Test data will not be used during model training. In the dataset, there are 3573 pieces of data labelled as 0; 1902 pieces of data labelled as 1, and 3906 pieces of data labelled as 2. Thank you for your advice.
Point 3: Only accuracy is presented: precision and recall (and f-measure) are measures which allow a better understanding of the behavior of the classifiers and they should be presented.
Response 3: Regarding the third point, your suggestion is very necessary. We also think that using the Precision, recall rate and F1 value can better illustrate the effect we want to achieve. We redesigned the experimental part, and this part of the content will be reflected in the experimental results Table 1 of the paper .
Point 4: "Typical" text classifiers based on word embeddings obtain a much better performance. So, why to transform text into graphs and, then, classify graphs? (note that your graph approach also uses bert embeddings for the nodes; it is not true that it does not use pre-trained models).
Response 4: Regarding the fourth point, we would like to explain that according to the requirements of our scientific research projects, in addition to categorizing news trend forecasts, we also need to display them visually, so we use knowledge graphs to visualize them. At the same time, you mentioned why not directly use the pre-trained language model, but also use the pre-trained language model to obtain the initial embedding vector. We just used the pre-trained language model to obtain the initial vector. In fact, using any other method to obtain the word embedding vector, such as calling the gensim Chinese word embedding vector method, we can also obtain the initial embedding vector of the node, but it does this. The effect is a little worse than using Bert. We will use gensim to obtain the initial word vector, and the experimental results of our method in the Table 1 in Chapter 3. At the same time, we will make a supplementary explanation in Chapter 3.2 of the paper for your reference.
Point 5: Related with the previous point, it would be important to analyze the main sources of errors of your approach: when and why does it fail? Are there any classes of errors? Is it a problem of the text to graph transformation? If yes, which kind of information is lost and how to overcome this problem?
Response 5: Compared with the use of pre-trained language models, our method is indeed not good enough, but our method can solve the shortcomings of the limited text input length of pre-trained language models. We believe that the pre-trained language model is limited by the length of the text, and there is no good solution for text classification with a length greater than 512. Normally, the text is divided into several texts with a length of 512 before processing. Even if the current new pre-trained language models such as XLNet break through the limitation of text length, it is achieved by increasing the computing perfor-mance and increasing the space occupied by the video memory. The longer the text length, the greater the difficulty of pre-trained and the difficulty increases in proportion to the square. When processing this part of the data in the experiment, we only use the first 512 words as input to the Bert model, and the other parts can only be discarded. Because in Chinese news, the first half of the news likely describes an event, and the follow-up de-scribes the solution or plan of the event. The trend of the description of the event and the solution is the opposite, so the input is divided into sections. The method of the Bert model is not suitable. However, we propose to use graphs to solve this problem to convert texts whose length exceeds the limit into knowledge graphs, extract high-dimensional features and use them to achieve classification, without being restricted by length. In other words, the longer the text length, the more useful triples can be extracted from the knowledge graph we construct, and the relationships between these triples can be used to classify them.
Of course, this also leads to the second problem. Knowledge graphs do lose part of information during the conversion process. Compared with text, knowledge graphs can only extract the main structure of sentences and construct triples. For some attributive adverbials and other information the loss is still serious. Our solution to this is as mentioned in chapter 2.2. In the knowledge graph, we make up 5 virtual nodes, which are keywords, high-frequency words, news characters, news locations, and news organizations, and set the initial vector of these nodes to Zero vector to increase the connectivity and relevance in the knowledge graph. We found that relatively short news text (for example, the text length is less than 100) does not work well for classification. The reason is that the text length is too short, which is not conducive to us extracting enough entities to build triples and knowledge graphs.
Therefore, we decided to add one item to the data set: the length of the main body of the news text. According to the display in the data set, we can clearly see that there are a total of 2524 news items with a text length of no more than 512, and we will add them in Chapter 3.2 of the paper. Thank you for your suggestions.
Point 6: • I strongly suggest the authors to make a deeper analysis of the error situations and to try to extend the text to graph transformation in order to try to obtain better results.
Response 6: Finally, thank you again for your valuable suggestions. We have answered the specific analysis part in the fourth and fifth points. You have provided us with many valuable ideas and experimental verification methods, which will be our most valuable harvest.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
This is the review report of the paper titled "The Status and Trend of Chinese News Forecast Based on 2 Graph Convolutional Network Pooling Algorithm"
I have some comments which have to be addressed,
1- More details are needed regarding the proposed method.
2- Conclusions are too short.
3- Comparison with previous methods is needed on the same used dataset. if any, otherwise I would recommend running some previous methods on your dataset with a suitable reference.
4- training parameters are required to mention.
5- Paper code with a nice demo is important to upload on any public platform.
Author Response
Dear reviewer2：
Re: Manuscript ID: applsci-1487466 and Title: The Status and Trend of Chinese News Forecast Based on Graph Convolutional Network Pooling Algorithm
Please see the attachment.
Thank you for your letter and the reviewers’comments concerning our manuscript entitled “applsci-1487466”. Those comments are valuable and very helpful. We have read through comments carefully and have made corrections. Based on the instructions provided in your letter, we uploaded the file of the revised manuscript. Revisions in the text are shown using red highlight for additions. The responses to the reviewer's comments are marked in red and presented following.
We would love to thank you for allowing us to resubmit a revised copy of the manuscript and we highly appreciate your time and consideration.
Sincerely. New Years greetings and best wishes！
Xiao Han, et al.
Point 1: More details are needed regarding the proposed method.
Response 1: For the first suggestion, we added detailed information about the data set, including key information such as how many instances there are in each category and how many news data have a text length of more than 512. In the dataset, there are 3573 pieces of data labelled as 0; 1902 pieces of data labelled as 1, and 3906 pieces of data labelled as 2. We also counted 2524 news content with a text length of more than 512 and 6857 news content with a text length of more than 512.
Point 2: Conclusions are too short.
Response 2: In response to the second suggestion, we added information about the pros and cons of the experimental methods and pre-training language models we designed, and the reasons we think may cause such results. Compared with the use of pre-trained language models, our method is indeed not good enough, but our method can solve the shortcomings of the limited text input length of pre-trained language models. We believe that the pre-trained language model is limited by the length of the text, and there is no good solution for text classification with a length greater than 512. Normally, the text is divided into several texts with a length of 512 before processing. Even if the current new pre-trained language models such as XL-Net break through the limitation of text length, it is achieved by increasing the computing performance and increasing the space occupied by the video memory. The longer the text length, the greater the difficulty of pre-trained and the difficulty increases in proportion to the square. When processing this part of the data in the experiment, we only use the first 512 words as input to the Bert model, and the other parts can only be discarded. Because in Chinese news, the first half of the news likely describes an event, and the follow-up de-scribes the solution or plan of the event. The trend of the description of the event and the solution is the opposite, so the input is divided into sections. The method of the Bert model is not suitable. However, we propose to use graphs to solve this problem to convert texts whose length exceeds the limit into knowledge graphs, extract high-dimensional features and use them to achieve classification, without being restricted by length. In other words, the longer the text length, the more useful triples can be extracted from the knowledge graph we construct, and the relationships between these triples can be used to classify them.
Of course, this also leads to the second problem. Knowledge graphs do lose part of information during the conversion process. Compared with text, knowledge graphs can only extract the main structure of sentences and construct triples. For some attributive adverbials and other information the loss is still serious. Our solution to this is as mentioned in chapter 2.2. In the knowledge graph, we make up 5 virtual nodes, which are keywords, high-frequency words, news characters, news locations, and news organizations, and set the initial vector of these nodes to Zero vector to increase the connectivity and relevance in the knowledge graph. We found that relatively short news text (for example, the text length is less than 100) does not work well for classification. The reason is that the text length is too short, which is not conducive to us extracting enough entities to build triples and knowledge graphs.
Point 3: Comparison with previous methods is needed on the same used dataset. if any, otherwise I would recommend running some previous methods on your dataset with a suitable reference.
Response 3: In response to the third suggestion, we have added some regulations on the dataset. We decided to adopt a voting mechanism to vote the data marked by different taggers. The principle is that if the judgments of the three taggers are consistent, we think their label is true. If two news items are consistent, the other is different, or the labels of the three taggers are different, we will determine the final trend judgment of the news through joint discus-sion among the authors. After the voting mechanism, they are unified into a dataset. Finally, we apply some baselines to the data obtained in our data set as shown in Table 1.
Point 4: training parameters are required to mention.
Response 4: In response to the fourth recommendation, we will supplement the detailed data in each category in detail in Chapter 3 of the paper. The evaluation work is to randomly select 90% of the data set as the training set, and the remaining 10% as the test set. No test data will be used during model training. In addition, we have changed the inspection indicators from a single accuracy rate to Precision, recall rate and F1 value, which allows you to better judge the pros and cons of the model. Finally, we apply some baselines to the data obtained in our dataset as shown in Table 1.
Point 5: Paper code with a nice demo is important to upload on any public platform.
Response 5: As for the fifth suggestion, because we still need to apply for invention patent and software copyright in China, the code of this project cannot be disclosed for the time being. We will publish the code on GitHub as soon as the invention patent and software copyright are granted. Thank you again for your suggestions. You have provided us with many valuable ideas and experimental verification methods, which will be our most valuable harvest.
Author Response File: Author Response.pdf
Round 2
Reviewer 1 Report
I believe the authors have answered my comments in an adequate way.
Reviewer 2 Report
The authors have addressed all comments
