WenSiM: A Relative Accuracy Assessment Method for Land Cover Products Based on Optimal Transportation Theory

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

Though the work is largely well-written, the article still needs significant improvement in some cases (see my comments below). Considering my observations as follows, I recommend minor revisions are as follows before considering it for publication.

·       Please omit or delete the heading of ‘section 2’ and merge it with ‘introduction’.  I would suggest to write the introduction into three paragraphs, highlighting the basic content of the research field in the first paragraph. Then, review the research progress of the literature in the second paragraph, and in the third paragraph, analyse the limitations of past research and clarify the innovation of your own research.

·       The heading of section ‘3.1’ and ‘3.2’ are same. I think the heading of 3.2 should be ‘Study area’

·       In this study the authors proposed a relative accuracy assessment method for LC products based on optimal transport theory. Is there any limitation of the proposed relative accuracy assessment method? Discuss in the separate section.

·       I suggest to add more reference in the discussion section.

·       Although the language of the manuscript is good, I suggest to check the English grammar to correct some simple linguistic errors.

·       I suggest adding a paragraph that specifies future applications and benefits from the results of this study.

Comments on the Quality of English Language

Moderate editing of English language required.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

Land cover (LC) products are important and widely used in many fields. How to chose the most suitable product is difficult for the common researchers. In this study, WenSiM was proposed to evaluate the accuracy of product. The study objective is good, but there are still some issues that need to be considered and improved.

1. The logic and structure of the article should be further adjusted, for example: how does the first three paragraphs in 2. Related works transition to Figure 1 and the following content,.

2. Please confirm if the words of “all accuracy information” in line 82 is accurate. How can the author ensure that there is no other accuracy information except for spatial position information and global feature information?

3. In Table 1, the time of FROM_GLC10 is in 2017, and the rest of the products is in 2020. Will the time difference between the data have an impact on subsequent results, as the land cover may have undergone significant changes within three years.

4. 3.1 Data preprocessing and 3.2 Data preprocessing are the same.

5. Is the term "Province border" in Figure 2 reasonable?

6. In lines 227-228, how is the sample size determined?

7. In line 254, the accuracy assessment results for LC products, why is there no corresponding description of accuracy assessment methods in 4. Methods?

8. In line 289-290, the study initially divides Beijing into 16 layers, how many samples were selected for each of the 16 layers? What is the basis for selection? How many samples are selected for each class within each layer? What is the basis for selection? Is the selection of sample points random or how?

9. What class does the TL stand in Figure 4?

10. In lines 356-357, what is the sample proportion selected? What other validation datasets refer to?

11. In lines 358, how can the unreasonable evaluation of the classification accuracy obtained be due to the inadequacy of samples, rather than the imbalanced sampling data ratios among different classes?

12. In lines 359-361, will the results be changed through changing the sampling strategy to ensure a larger sample size for small land covers? Can the evaluation results similar to the results of WenSiM also be obtained through changing the sampling strategy?

13. In lines 405-406, there are three questions when choosing the product with the highest accuracy as the reference truth: 1. If we already know which product has the highest accuracy, why do we need to conduct subsequent testing? Can I simply use the data from the product with the highest accuracy? 2. The current evaluation results of WenSiM are based on the product data with the highest accuracy. Can a consistent result be obtained if a different reference data is used? Is the evaluation result of WenSiM related to the accuracy of the reference data? 3. As shown in Table 1, the accuracy of Esri_GLC10 data is only 85.96%. Will the misclassification results of this data also have an impact on subsequent evaluations?

14. The language needs further refinement. We recommend having a native speaker review and edit the manuscript for clarity and coherence.

Comments on the Quality of English Language

The language needs further refinement. We recommend having a native speaker review and edit the manuscript for clarity and coherence.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 3 Report

Comments and Suggestions for Authors

I enjoyed reading this paper. The authors propose an approach built on optimal transport theory for relative accuracy assessment and substantiate its robustness and effectiveness through a case study of a Chinese city. Overall, the paper is well-written, maintaining a smooth flow and thoughtful design throughout. The authors offer clear explanations and provide ample evidence to demonstrate how their proposed approach successfully addresses the limitations inherent in mainstream evaluation methods that demand substantial effort. Based on the merits of the paper, I recommend accepting the manuscript for publication.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Round 2

Reviewer 2 Report

Comments and Suggestions for Authors

Thanks for the author's response. After modification, the content and quality of the paper have been effectively improved. And the paper can be accepted in present form.