A New End-to-End Multi-Dimensional CNN Framework for Land Cover/Land Use Change Detection in Multi-Source Remote Sensing Datasets

Round 1

Reviewer 1 Report

(1)End-to-end framework is a term used in machine learning to realize from input to output as one packaged process. In this paper, change detection is packaged by a process that combines two networks, so it is not wrong to call the proposed method as 'end-to-end'. However, after 188th line in which this word is firstly used in the text, it is not clear why it is used in this paper as a special term expressed in capital letters, such as "End-to-End". If you want to use it as a special term, you should show the reason why the method proposed in this paper could be named as such.

(2) In 3.2, please compare the structure of the proposed method shown in figure 5 with existing network structure in detail. Please describe which network the structure is partially similar to, and what can be expected as a merit compared to the structure of the existing network. If this network proposal is the most important part of this paper, you should claim its value here.

(3) Since 3.2, it is difficult to read because it has too deep section numbers that I have not seen in other papers. The 3.2.1.1 paragraphs are short enough that they should be merged with like numbered short paragraphs to a depth of at least 3.2.1. Also, some subsections are too short, such as 3.3. Sections 3 and 5 should be organized by integrating sections. In 3.2, a lot of papers are cited, of course, since 3.2.1, but these are for explaining the detailed specifications of the proposed method. The readers would be interested in how your proposed network structure relates to existing networks.

(4) In 5., There is not enough discussion about how the network structure of the proposed method is the root cause of the high performance in the three different types of remote sensing datasets.
Also, from 5.2 to 5.5, only the superiority of the proposed method is discussed. Clearly indicate the limits of your research and describe your suggestions for subsequent research.

Author Response

Title: A New End-to-End Multi-Dimensional CNN Framework for Land Cover/Land Use Change Detection in Multi-Source Remote Sensing Datasets

Dear reviewers,

The authors thank you for spending time on this paper and providing invaluable comments and for very insightful questions. They have helped to improve the quality of the paper. The main corrections are listed below:

   (1) The text body is improved.

   (2) The performance proposed method is compared to other methods.

   (3) The improved details about of training dataset.

   (4) The results of some methods checked again.

   (5) Some texts refined and added informative texts.

In the following, the authors answer the individual questions posed by the referee. We have made careful considerations and now reply to the comments one by one. Also, corrected and modified items appear in red in the text.

Reviewer 1:

1. The Abstract is very long. It should reduce to be no more than 250 words.

Answer 1: The authors reduced the abstract (See Abstract).

2. My main concern is about the contributions and goals of the paper, which I think is not clear. In the abstract, the authors talk about a novel CD framework based on Convolutional Neural Network (CNN), but most of the paper is devoted to present results comparing to some frameworks and methods. The authors need to justify the novelty of the framework in terms of the techniques used and the theoretical core of the framework compared to the related works as well as the following works that should be also included in the literature review.

• Carranza-García, M., García-Gutiérrez, J., & Riquelme, J. C. (2019). A framework for evaluating land use and land cover classification using convolutional neural networks. Remote Sensing, 11(3), 274.
• Hu, Y., Zhang, Q., Zhang, Y., & Yan, H. (2018). A Deep Convolution Neural Network Method for Land Cover Mapping: A Case Study of Qinhuangdao, China. Remote Sensing, 10(12), 2053.

Answer 2: Thank you for this comment. The authors described the novelty of the proposed architecture and compared it with other architectures (see Section 3, line 449-463)

3. What is the limitation of the work cited in [32] and what are the differences and advantages of this work compared to [32]?

Answer 3: Thank you for your comment. The authors discussed the mentioned method in disadvantages, its performance in CD (see, Section 5, Line 793-797, 806-810, 821-830).

4. The authors mentioned that there are some change detection studies in [3, 29, 38, 41, 42] used the same datasets of the study but I didn’t found these works in the comparison results. These works should be compared with their work.

Answer 4: Thanks for your comment, you are right. The authors compared the results of our study with those of these methods for each dataset (see, Section 5, line 821-830, 856-866, and 905-908).

5. Even though the paper is organized well in somehow, but there are some sections or subsections are not correct or missing. For example, “3.4.1. Comparison with Ground Truth Data” and I see another subsection “3.4.1. Comparison with Other CD Methods”!? And also the “experimental setup” subsection is missing. You can see a good organization in the paper entitled “A framework for evaluating land use and land cover classification using convolutional neural networks”.

Answer 5: Thank you for your good comment, the authors double-checked the structure of the manuscript. The subsection “experimental setup” is added in Section 5 (see, Section 4, figures 4-5, 8-9 and 16-17, Subsection 3.4.2, Section 5, line 794). The mentioned paper is also cited in the manuscript.

6. In my opinion, the authors change the “accuracy indices” to “evaluation metrics” and “The accuracies of different CD methods” to “The evaluation metrics values of different CD methods”. The authors should explain the aim of every metric. For example, what is the “??” and why it is used?

Answer 6: Thank you for your comment. The authors revised the new version of the paper. Also, the “Pe” is a symbol, however, this issue revised in the new version of the paper (see, Section 3.4.1, table 5, lines: 571, and 574, Subsection 4.3.2, line 711, Subsection 4.2.2, line 654, 657, Subsection 3.4.1, line 562-565, and also, Table 5).

7. How the authors selected the number of neurons, layers, dropout rate, dilation rate, and the size of 3D-dilated convolution layers? The authors should give a systematic way of selecting these configurations. What about using different configurations? Support your answer with results for showing the effect of different configurations and proving whether these configurations are good values for the proposed framework.

Answer 7: Thank you for your comment. These parameters are obtained based on experimental and are knowledge-based. These parameters are defined in a similar approach to the presented method in paper [11] that was based on grid search. However, since using the grid search algorithm is time-consuming, the authors resolved this by the following approach. for tuning hyperparameters, we evaluated the performance of the network using different values for the parameters. In fact, the sensitivity of the network on these parameters is evaluated in low bound for example: if the dropout rate is [0,1] we evaluated only these values: 0.1, 0.5, 0.9). For example, the initial dropout rate is 0.2, then we changed it to 0.1. The authors discussed this issue as future work in section 6. Also, the authors add more detail on how to optimize these parameters (see, Subsection 3.2.5, line 538-547, Section 6, line 509-508).

8. In the subsection “3.3. Optimum Model”, the authors should explain the technique used for selecting the Optimum Model. In other words, explain the method used for tuning the model’s hyperparameters.

Answer 8: Thank you for your good comment. As suggested, the authors added more detail of the Optimum model and tuning parameters (see, Subsection 3.2.4, line 529-533, Subsection 3.3, line 548-554).

9. The authors used the training set to train the model and validation set to evaluate the model as stated in Figure 4. However, evaluating the model using the validation set is not convincing. Evaluating the model should be on a testing set that never seen before and never used in the training process as a validation set. Therefore, the authors should divide the dataset into three sets: training set, validation set, and testing set and explain how many samples in each set.

Answer 9: Thank you for your good comment. As suggested, the authors investigated this issue and added more detail of the training sample (see, Section 3, Figure-4, Section 3, line 529-533, Section 4, Table 6) 

10. The authors should revise the Specificity (%) in Table 6. They seem to be low values except for your framework. And the author should unify the terms used, for example, “KC” in Table 6 and “Kappa” in other Tables.

Answer 10: Thank you for your good comment. The authors checked this and the results are correct. The term Kappa is unified in other tables (see, Section 4, Table 8-12)    

11. I see the reference [69] without a date. Please revise the references and the English writing of the paper.

Answer 11: The authors revised this reference. The manuscript was also carefully reviewed by a native English speaker to improve its grammar and structure.

Author Response File: Author Response.docx

Reviewer 2 Report

The authors proposed a change detection (CD) framework based on the convolutional neural network (CNN) approach. However there are some comments that should be addressed. These comments are written as follows:

1. The Abstract is very long. It should reduce to be no more than 250 words.

2. My main concern is about the contributions and goals of the paper, which I think is not clear. In the abstract, the authors talk about a novel CD framework based on Convolutional Neural Network (CNN), but most of the paper is devoted to present results comparing to some frameworks and methods. The authors need to justify the novelty of the framework in terms of the techniques used and the theoretical core of the framework compared to the related works as well as the following works that should be also included in the literature review.

• Carranza-García, M., García-Gutiérrez, J., & Riquelme, J. C. (2019). A framework for evaluating land use and land cover classification using convolutional neural networks. Remote Sensing, 11(3), 274.
• Hu, Y., Zhang, Q., Zhang, Y., & Yan, H. (2018). A Deep Convolution Neural Network Method for Land Cover Mapping: A Case Study of Qinhuangdao, China. Remote Sensing, 10(12), 2053.

3. What is the limitation of the work cited in [32] and what are the differences and advantages of this work compared to [32]?

4. The authors mentioned that there are some change detection studies in [3, 29, 38, 41, 42] used the same datasets of the study but I didn’t found these works in the comparison results. These works should be compared with their work.

5. Even though the paper is organized well in somehow, but there are some sections or subsections are not correct or missing. For example, “3.4.1. Comparison with Ground Truth Data” and I see another subsection “3.4.1. Comparison with Other CD Methods”!? And also the “experimental setup” subsection is missing. You can see a good organization in the paper entitled “A framework for evaluating land use and land cover classification using convolutional neural networks”.

6. In my opinion, the authors change the “accuracy indices” to “evaluation metrics” and “The accuracies of different CD methods” to “The evaluation metrics values of different CD methods”. The authors should explain the aim of every metric. For example, what is the “??” and why it is used?

7. How the authors selected the number of neurons, layers, dropout rate, dilation rate, and the size of 3D-dilated convolution layers? The authors should give a systematic way of selecting these configurations. What about using different configurations? Support your answer with results for showing the effect of different configurations and proving whether these configurations are good values for the proposed framework.

8. In the subsection “3.3. Optimum Model”, the authors should explain the technique used for selecting the Optimum Model. In other words, explain the method used for tuning the model’s hyperparameters.

9. The authors used the training set to train the model and validation set to evaluate the model as stated in Figure 4. However, evaluating the model using the validation set is not convincing. Evaluating the model should be on a testing set that never seen before and never used in the training process as a validation set. Therefore, the authors should divide the dataset into three sets: training set, validation set, and testing set and explain how many samples in each set.

10. The authors should revise the Specificity (%) in Table 6. They seem to be low values except for your framework. And the author should unify the terms used, for example “KC” in Table 6 and “Kappa” in other Tables.

11. I see the reference [69] without a date. Please revise the references and the English writing of the paper.

Author Response

Title: A New End-to-End Multi-Dimensional CNN Framework for Land Cover/Land Use Change Detection in Multi-Source Remote Sensing Datasets

Dear reviewers,

The authors thank you for spending time on this paper and providing invaluable comments and for very insightful questions. They have helped to improve the quality of the paper. The main corrections are listed below:

   (1) The text body is improved.

   (2) The performance proposed method is compared to other methods.

   (3) The improved details about of training dataset.

   (4) The results of some methods checked again.

   (5) Some texts refined and added informative texts.

In the following, the authors answer the individual questions posed by the referee. We have made careful considerations and now reply to the comments one by one. Also, corrected and modified items appear in red in the text.

Reviewer 2:

1. The end-to-end framework is a term used in machine learning to realize from input to output as one packaged process. In this paper, change detection is packaged by a process that combines two networks, so it is not wrong to call the proposed method as 'end-to-end'. However, after 188th line in which this word is firstly used in the text, it is not clear why it is used in this paper as a special term expressed in capital letters, such as "End-to-End". If you want to use it as a special term, you should show the reason why the method proposed in this paper could be named as such.

Answer 1: Thank you for your good comment. The authors added the main reason for using this term (see, Section 1, line: 308-310).

2. In 3.2, please compare the structure of the proposed method shown in figure 5 with the existing network structure in detail. Please describe which network the structure is partially similar to, and what can be expected as a merit compared to the structure of the existing network. If this network proposal is the most important part of this paper, you should claim its value here.

Answer 2: Thank you for your comment. As recommended, the authors compared the proposed architecture with another state-of-the-art network in terms of visual and statistical accuracies. (see, Section 3, figure 6, and line: 443-458).

3. Since 3.2, it is difficult to read because it has too deep section numbers that I have not seen in other papers. The 3.2.1.1 paragraphs are short enough that they should be merged with like numbered short paragraphs to a depth of at least 3.2.1. Also, some subsections are too short, such as 3.3. Sections 3 and 5 should be organized by integrating sections. In 3.2, a lot of papers are cited, of course, since 3.2.1, but these are for explaining the detailed specifications of the proposed method. The readers would be interested in how your proposed network structure relates to existing networks.

Answer 3: Thank you for your comment. The subsections 3.2.1.1 to 3.2.1.5 is merged (now section 3.2.1). Also, the text of section 5 is improved. (see, Section 3 and Section 5.2).

4. In 5., There is not enough discussion about how the network structure of the proposed method is the root cause of the high performance in the three different types of remote sensing datasets.
   Also, from 5.2 to 5.5, only the superiority of the proposed method is discussed. Clearly indicate the limits of your research and describe your suggestions for subsequent research.

Answer 4: Thank you for your comment. The authors improved the text in section 5. Moreover, we discussed the limitations of the proposed method and provided several suggestions for future works. (see, Section 6, figure 6, and line: 445-458).  

 

Author Response File: Author Response.docx

Round 2

Reviewer 1 Report

A spell miss was seen in newly added sentences.

l. 319 the-of -> of-the

Author Response

Title: A New End-to-End Multi-Dimensional CNN Framework for Land Use/Land Cover Change Detection in Multi-Source Remote Sensing Datasets

Honorable Referees,

The authors thank the referees for spending time on this paper and providing invaluable comments and for very insightful questions. They have helped to improve the quality of the paper. The main corrections are listed below:

   (1) The text body is improved.

   (2) More detail of using different configurations are added.

   (3) The all figures, tables, equations checked again.

   (4) In the confusion matrix, all methods are added.

    (5) Some texts refined and added informative texts.

In the following, the authors answer the individual questions posed by the referee. We have made careful considerations and now reply to the comments one by one. Also, corrected and modified items appear in red in the text.

Reviewer 1:

1. A spell miss was seen in newly added sentences. Line 319 the-of -> of-the

Answer 1: Thank you for your comment. The authors reviewed and revised the new version of the manuscript. The state-of-the-art was also revised (see, Section 3, Line: 323).

Author Response File: Author Response.docx

Reviewer 2 Report

There are still some major comments that need to be addressed by the authors. These comments are as follows:

1. In Table 5, the equation of precision is not correct, it looks like the specificity equation. In addition, is it really that the N is the number of pixels in the image. I think it is the number of samples in the test. As it is used for the Overall Accuracy (OA) = (TP+TN)/N. Also, what is PCC?

2. The authors didn’t explain why the specificity values of the other methods have low values. Moreover, in the discussion of the results, the authors should include the confusion matrices of the proposed method and some other methods used in the comparisons, explaining why some evaluation metrics of compared methods have low values and why the others have high values.

3. For comment 7 in round 1. The authors didn’t address this comment: What about using different configurations? Support your answer with results for showing the effect of different configurations and proving whether these configurations are good values for the proposed framework.

4. In Lines 327-329 “(3) Most of the deep learning methods for CD require a predictor (e.g., image differencing, image ratio, and PCA). However, the proposed method does not need a predictor for what?” What does mean by “for what”. It is not clear, please make it clear.

5. For comment 4 in round 1, "The authors mentioned that there are some change detection studies in [3, 29, 38, 41, 42] used the same datasets of the study but I didn’t found these works in the comparison results. These works should be compared with their work in the Tables and reported results?"; however, I didn’t see the answer in Section 5, line 821-830, 856-866, and 905-908). Please clearly and explicitly answer this comment in the response file and manuscript.

6. On line 398, “data by the cost function. To this end, the sample data is divvied into three categories: (1) training” there is an English mistake in typing the word “divvied”. Eq. (7) has only a single left parenthesis. Please check the English writing, figures, tables, and equations in the whole manuscript.

Author Response

Title: A New End-to-End Multi-Dimensional CNN Framework for Land Use/Land Cover Change Detection in Multi-Source Remote Sensing Datasets

Honorable Referees,

The authors thank the referees for spending time on this paper and providing invaluable comments and for very insightful questions. They have helped to improve the quality of the paper. The main corrections are listed below:

   (1) The text body is improved.

   (2) More detail of using different configurations are added.

   (3) The all figures, tables, equations checked again.

   (4) In the confusion matrix, all methods are added.

    (5) Some texts refined and added informative texts.

In the following, the authors answer the individual questions posed by the referee. We have made careful considerations and now reply to the comments one by one. Also, corrected and modified items appear in red in the text.

Reviewer 2:

1. In Table 5, the equation of precision is not correct, it looks like the specificity equation. In addition, is it really that the N is the number of pixels in the image. I think it is the number of samples in the test. As it is used for the Overall Accuracy (OA) = (TP+TN)/N. Also, what is PCC?

Answer 1: Thank you for your comment. The authors reviewed and revised all the equations in this table. (please see the revised Table 5).

2. The authors didn’t explain why the specificity values of the other methods have low values. Moreover, in the discussion of the results, the authors should include the confusion matrices of the proposed method and some other methods used in the comparisons, explaining why some evaluation metrics of compared methods have low values and why the others have high values.

Answer 2: As suggested, the authors provide more explanation about different metrics. Moreover, the confusion matrix of all methods is added and discussed (see the Result Section, line 709-719, figure 11, figure 14, figure 17, figure 20, figure 23 and figure 26).

3. For comment 7 in round 1. The authors didn’t address this comment: What about using different configurations? Support your answer with results for showing the effect of different configurations and proving whether these configurations are good values for the proposed framework.

Answer 3: Based on your comment, the authors provide more details of using different configurations (see Section 5, Table 7, Line 469-480).

 

4. In Lines 327-329 “(3) Most of the deep learning methods for CD require a predictor (e.g., image differencing, image ratio, and PCA). However, the proposed method does not need a predictor for what?” What does mean by “for what”. It is not clear, please make it clear.

Answer 4: It was a spelling mistake! “for what?” is removed now. In fact, the proposed method does not need any additional process for change detection. However, some methods such as 3D-CNN of GENNET CD methods need an additional process for change detection.

5. For comment 4 in round 1, "The authors mentioned that there are some change detection studies in [3, 29, 38, 41, 42] used the same datasets of the study but I didn’t found these works in the comparison results. These works should be compared with their work in the Tables and reported results?"; however, I didn’t see the answer in Section 5, line 821-830, 856-866, and 905-908). Please clearly and explicitly answer this comment in the response file and manuscript.

Answer 5: The authors compared these methods in Tables 14, and 15 and discussed them in line: 752-764.  

6. On line 398, “data by the cost function. To this end, the sample data is divvied into three categories: (1) training” there is an English mistake in typing the word “divvied”. Eq. (7) has only a single left parenthesis. Please check the English writing, figures, tables, and equations in the whole manuscript.

Answer 6: The authors reviewed and revised the new version of the manuscript.

Author Response File: Author Response.docx