A New Clustering Method to Generate Training Samples for Supervised Monitoring of Long-Term Water Surface Dynamics Using Landsat Data through Google Earth Engine

Round 1

Reviewer 1 Report

 

In this paper, the authors developed a new method to generate training samples and computed the water area for long time series with these samples. However, to improve the quality of this paper, there are several issues need to be clarified or revised as follow:

1. In Section 3.1, why use NDVI instead of NDWI? I think that NDWI may be more appropriate. It seems that NDVI cannot distinguish between water and non-water in Figure 5? Whether the poor performance of SI is caused by the interference of NDVI? Please clarify it.

2. In Figure 5, PA and UA are a pair of indicators. As a binary classification task, UA_w should be equal to PA_NW, but there is a certain difference in your calculation results. Please check that.

3. In Section 3.1, by how much has the accuracy of the samples improved after employing the Fmask algorithm? Please give some quantitative results. In addition, is it possible to use this method to directly map water with two algorithms- RF and SVM, and compare the accuracy of the mapping results? Or is it possible to compare the mapping results using one algorithm with different samples, samples generated by the method in this paper and real samples.

4. What is McNemar's formula? What is the meaning of each metric in Table 2? How are they calculated? I suggest that the authors give a detailed definition for each indicator to facilitate the reader's understanding.

5. The title describes a method with focus on clustering methods, but it does not seem to be sufficient to cover the content of Section 3.2 and 3.3, so it is suggested that the title should be modified to include this part.

6. Existing datasets such as JRC seem to enable the analysis of 3.2 and 3.3 as well? So what different conclusions can be drawn from your approach?

7. Since the results are implemented in the GEE platform, it is suggested that the authors can share the code to increase the contribution of the paper.

8. There are some confusing sentences or phrases in the manuscript, such as Line 338 and Line 508. Please check the article and polish it carefully.

Author Response

Dear Reviewer (#1),

We would like to thank you very much for your time and constructive comments that helped us improve our manuscript. We think that we have addressed all your comments. Please find below our responses to your comments in blue that are also implemented and highlighted in the manuscript.

Please kindly let us know if you have any further comments.

Thank you!

Ebrahim Ghaderpour

On behalf of all the authors

 

1. In Section 3.1, why use NDVI instead of NDWI? I thinkthat NDWI may be more appropriate.It seems that NDVI cannot distinguish between water and non-water in Figure 5? Whether the poor performance of SI is caused by the interference of NDVI? Please clarify it.

As mentioned in the manuscript, the SI feature set consists of 4 indices of MNDWI, AWEI, NDVI, and EVI. The MNDWI index is an improved version of the NDWI index and has shown better performance than NDWI in several studies. In addition, the AWEI index has been obtained in various studies to identify water. Since the study areas included both water and non-water classes, NDVI and EVI indices were used along with water indices. Negative NDVI values ​​have also been used in some studies to identify water areas. This four-index combination has achieved satisfactory performance with more than 90% overall classification accuracy. It is necessary to mention that we tested several combinations of other indices like NDWI, WI2015, ANDWI, GNDVI, NDMI, and WDRVI. A combination of MNDWI, AWEI, NDVI, and EVI could achieve the highest accuracy.

2. In Figure 6, PA and UA are a pair of indicators. As a binary classification task, UA_w should be equal to PA_NW, but there is a certain difference in your calculation results. Please check that.

Thank you for your attention. Figure 6 was checked again, revised, and added to the text. Based on your comment, PA classes were omitted because of their equality with UA of another class.

3. In Section 3.1, by how much has the accuracy of the samples improved after employing the Fmask algorithm? Please give some quantitative results. In addition, is it possible to use this method to directly map water with two algorithms- RF and SVM, and compare the accuracy of the mapping results? Or is it possible to compare the mapping results using one algorithm with different samples, samples generated by the method in this paper and real samples.

The performance evaluation of the proposed method has been conducted by evaluating the trained classification with the evaluation samples. Since a completely different manner was used to collect evaluation data, they can be a good criterion for assessing the performance of a trained classifier that is directly affected by the quality of the training samples. It should be noted that Figure 4 shows the performance of the initial Fmask map using evaluation data. We have also added the classified map obtained from the training of the RF model (using all the features) with direct Fmask map samples (without refinement using the proposed method) to Figure 4. Quantitative results were also included. When the iterative Kmeans-based procedure is not used, the accuracy of the final map is about 78% in all three years. However, these values ​​have reached over 95% every three years after applying the proposed method (Figure 6).

4. What is McNemar's formula? What is the meaning of each metric in Table 2? How are they calculated? I suggest that the authors give a detailed definition for each indicator to facilitate the reader's understanding.

Done. A complete description of different parameters was added to section 2.3.5.

5. The title describes a method with focus on clustering methods, but it does not seem to be sufficient to cover the content of Section 3.2 and 3.3, so it is suggested that the title should be modified to include this part.

Long-term phrase was added to the title to cover long-term change analysis in Sections 3.2 and 3.3.

6. Existing datasets such as JRC seem to enable the analysis of 3.2 and 3.3 as well? So, what different conclusions can be drawn from your approach?

The purpose of Section 3.2 is to investigate the long-term changes in surface water extent in the study sites. Since the proposed methodology can be applied at different times, it allows for the production of water maps over time. In addition, the importance of this part can be seen in the declining trend of some reservoirs in Iran. The correlation between changes in SWA, temperature, and precipitation is discussed in Section 3.3. The JRC reference map provides the SWA using complex methods with different data sources and does not contain temperature and precipitation information. Therefore, it is practically not possible to perform Section 3.3 analyzes using the JRC map. In addition, in the section on 'comparison with other studies,' it is noted that the proposed method has achieved well-competitive accuracy with the JRC map, which uses various data sources and sophisticated machine learning methods. Also, it is impossible to access JRC maps almost in near real-time, and these maps will be available to users after a while. However, monitoring water resources using the proposed method, due to the use of the Fmask algorithm, can be done after providing each image, allowing near real-time processing.

7. Since the results are implemented in the GEE platform, it is suggested that the authors can share the code to increase the contribution of the paper.

We appreciate your comment very much. We added the following at the end of the manuscript:

Software Availability: The computer code for the proposed algorithm will soon be available at https://github.com/ATDehkordi/Sustainability_ICRP

8. There are some confusing sentences or phrases in the manuscript, such as Line 338 and Line 508. Please check the article and polish it carefully.

The manuscript was completely polished.

Reviewer 2 Report

This manuscript proposed an approach for generating reliable training samples through Landsat images, then the samples were used to train SVM and RF supervised classification models, aiming at long-term monitoring of water surface dynamics in the eight Iranian reservoirs. This is an interesting study and is well-organized in general. However, there are several significant issues, including the Methods, results, and the discussion as follows:

1. The novelty of the method in this paper is the iterative-based k-means clustering refinement procedure on the Fmask initial water map to generate reliable training data. So can we say that this study is an improvement of the Fmask algorithm? If so, the simple mechanism of the Fmask algorithm needs to be supplemented in the methods section, rather than a simple article citation.

2. In addition, we consider that the manuscript lacks the description of detailed operational steps of the iterative-based K-means method, including the input and output of the algorithm, the number of classification groups, etc., which are currently not sufficiently discussed in Section 2.3.3.

3. Line 259-260, how are the evaluation indicators calculated? Such as a confusion matrix, OA and kappa coefficient, etc., based on image pixels or water map overlap rate? Additional formulas should be added for clarification.

4. The red box in Fig 4 (b) may be for lack of descriptive text.

5. Line 307-308, RF achieved an overall accuracy of only 0.5% greater than SVM, in my opinion, there is no comparability between RF and SVM, and the discussion in line306-325 is not even necessary. We suggest to focus on the comparison of RF, SVM with Fmask algorithms, such as fig 4(c) and fig 6, to illustrate the advantages of your proposed method.

6. In the discussion of water demand, line 477, reservoir regulation pattern is also an important element for change analysis. For example, multi-year regulation reservoirs and annual regulation reservoirs have different effects on the long-term changes in water surface area.

Author Response

Dear Reviewer (#2),

We would like to thank you very much for your time and constructive comments that helped us improve our manuscript. We think that we have addressed all your comments. Please find below our responses to your comments in blue that are also implemented and highlighted in the manuscript.

Please kindly let us know if you have any further comments.

Thank you!

Ebrahim Ghaderpour

On behalf of all the authors

 

1. The novelty of the method in this paper is the iterative-based k-means clustering refinement procedure on the Fmask initial water map to generate reliable training data. So can we say that this study is an improvement of the Fmask algorithm? If so, the simple mechanism of the Fmask algorithm needs to be supplemented in the methods section, rather than a simple article citation.

In this study, no claim has been made to improve the results of the Fmask algorithm. Fmask is used because it generates a water map of each Landsat scene and protects the proposed method from relying on landcover reference maps that are published with a delay after the images have been acquired. According to investigations, Fmask's initial water map is not free of errors. Therefore, if its samples are used for supervised classification, the classification accuracy would be reduced (as shown in the results). Thus, the proposed method in this study refines the sample points obtained from the initial Fmask map. In other words, the proposed method does not directly improve the Fmask results and is able to remove misclassified samples in an iterative procedure. More details of the Fmask algorithm are added to the text.

2. In addition, we consider that the manuscript lacks the description of detailed operational steps of the iterative-based K-means method, including the input and output of the algorithm, the number of classification groups, etc., which are currently not sufficiently discussed in Section 2.3.3.

Done. Detailed description was added the manuscript.

3. Line 259-260, how are the evaluation indicators calculated? Such as a confusion matrix, OA and kappa coefficient, etc., based on image pixels or water map overlap rate? Additional formulas should be added for clarification.

Done. A complete description of different parameters was added to section 2.3.5.

4. The red box in Fig 4 (b) may be for lack of descriptive text.

Thanks. It was removed.

5. Line 307-308, RF achieved an overall accuracy of only 0.5% greater than SVM, in my opinion, there is no comparability between RF and SVM, and the discussion in line306-325 is not even necessary. We suggest to focus on the comparison of RF, SVM with Fmask algorithms, such as fig 4(c) and fig 6, to illustrate the advantages of your proposed method.

Done. we focused on the improvements obtained by the proposed methodology rather than only comparison of RF and SVM both in results and discussion.

6. In the discussion of water demand, line 477, reservoir regulation pattern is also an important element for change analysis. For example, multi-year regulation reservoirs and annual regulation reservoirs have different effects on the long-term changes in water surface area.

Thank you for your nice suggestion. A paragraph was added to the Discussion based on your comment.

 

Reviewer 3 Report

This review of the article “A New Clustering Method to Generate Training Samples for Supervised Monitoring of Water Surface Dynamics Using Landsat Data Through Google Earth Engine.”

In general, I found the article is very imperative and introduced a new approach for the training sample generation for the water surface extraction. As the manuscript is well written, I would propose a minor revision of the paper.  

The main point of my review is that the paper title suggests a new approach to generating training samples for water surface extraction but includes some parts that fall outside the scope of the paper (Ex – Study of the rainfall and temperature trend in the study basins).   

Minor corrections need to be done

I would recommend to introduce the accuracy assessment results in the abstract.

The citation has missed for the following lines

·        Lines 48-50

·        Lines 59-61

·        Lines 96-98

Table 1 needs to be format as the current format is difficult to read

Site

River

Opening date

Catchment Area

 

Lines 2005 to 2006 repeated

Figure 2 needs to be a bit bigger and needs to improve the visualization quality of the image.

Figure 8 is not required as the same information represented in figure 7, if authors need to provide the spatial variation in water area need to provide the maps for all the water bodies instead of providing it for one water body.

Author Response

Dear Reviewer (#3),

We would like to thank you very much for your time and constructive comments that helped us improve our manuscript. We think that we have addressed all your comments. Please find below our responses to your comments in blue that are also implemented and highlighted in the manuscript.

Please kindly let us know if you have any further comments.

Thank you!

Ebrahim Ghaderpour

On behalf of all the authors

 

- I would recommend to introduce the accuracy assessment results in the abstract.

Done.

- The citation has missed for the following lines

• Lines 48-50
• Lines 59-61
• Lines 96-98

Done.

-Table 1 needs to be format as the current format is difficult to read

Done.

-Figure 2 needs to be a bit bigger and needs to improve the visualization quality of the image.

Done.

-Figure 8 is not required as the same information represented in figure 7, if authors need to provide the spatial variation in water area need to provide the maps for all the water bodies instead of providing it for one water body.

Thanks for your comment. The ZR study site has the steepest downward slope among the study areas, so its changes are easier to observe visually. As a result, Figure 8 shows a visual representation of the dire condition of this reservoir.

 

Reviewer 4 Report

I found the article very interesting. Landsat data allows for the longest period of retrospective analysis. The availability of archival remote sensing data allows for examining the directions and strength of changes that occurred in the analyzed area. They are particularly important in the process of water resources management and in order to maintain the principles of sustainable development.

The article proposes a new automated method for teaching sample generation for supervised monitoring of surface water range changes using Landsat images.

Remote sensing seems to be the only tool for monitoring changes taking place in the vegetation cover at regional, continental and global scales. In particular, satellite remote sensing makes it possible to constantly monitor the earth's surface. The high availability of free Landsat, Sentinel or MODIS data as well as products ready for analysis means that multi-time analysis algorithms are constantly developed.

There are many sensors that record optical data, but this is not made available to the public. Taking into account the distribution method, spatial resolution of the data and the size of the imaged area, as well as the temporal range, images obtained from the Landsat mission are currently the most appropriate material for retrospective environmental analyzes on a regional scale.

I am asking the Authors for an explanation:

- In September 2021, the Landsat 9 satellite was launched. Why did the authors not use the data from this satellite, only 5.7 and 8?

- Due to the spatial resolution of Landsat data, it is problematic to prepare a validation layer. The use of the Fmask algorithm appears to be correct. However, please answer the question whether the Authors considered a different solution to the problem, e.g. using different detection data?

- Currently, data from the Sentinel-2 mission, belonging to the Copernicus program of the European Space Agency (ESA), is being used more and more. The mission consists of two twin satellites placed in the same orbit: Sentinel-2A (2015) and Sentinel-2B (2017). Please clarify whether the Authors have taken into account these satellites?

-Do the Authors see the possibility of implementing the proposed methodology and models for the assessment of other dam reservoirs in the world? What may be the limitations?

- Why did the Authors choose NDVI, MNDWI, EVI and AWEI indicators / indices? Why was GNDVI, WDRVI, NDMI not used?

 

Kind regards.

Author Response

Dear Reviewer (#4),

We would like to thank you very much for your time and constructive comments that helped us improve our manuscript. We think that we have addressed all your comments. Please find below our responses to your comments in blue that are also implemented and highlighted in the manuscript.

Please kindly let us know if you have any further comments.

Thank you!

Ebrahim Ghaderpour

On behalf of all the authors

 

- In September 2021, the Landsat 9 satellite was launched. Why did the authors not use the data from this satellite, only 5.7 and 8?

In this study, a fixed time period (from August to September) is considered to use satellite imagery each year. Since this period mainly coincides with the hottest months of the year (more cloud-free images are available) in the study sites, the Landsat 5, 7, and 8 images were sufficient to provide enough images, and there was no a problem in this regard. The launch time of the L9 satellite almost coincided with the end of the study period in 2021. The first L9 satellite imagery of study areas was also taken in early November 2021, which was not helpful for this study and was out of period. The use of Landsat 9 images is going to be evaluated in future studies.

- Due to the spatial resolution of Landsat data, it is problematic to prepare a validation layer. The use of the Fmask algorithm appears to be correct. However, please answer the question whether the Authors considered a different solution to the problem, e.g. using different detection data?

In order to evaluate the proposed method, evaluation samples were obtained from field visits as well as visual interpretations of images with a high spatial resolution (Google earth and Sentinel-2). The Fmask map is used only for the generation of training samples, and the difference between the process of collecting training and evaluation data can well examine the performance of the proposed method.

- Currently, data from the Sentinel-2 mission, belonging to the Copernicus program of the European Space Agency (ESA), is being used more and more. The mission consists of two twin satellites placed in the same orbit: Sentinel-2A (2015) and Sentinel-2B (2017). Please clarify whether the Authors have taken into account these satellites?

As answered in the previous question, Sentinel-2 images, which have a higher spatial resolution than Landsat images, have been used for visual interpretation in the collecting of evaluation samples along with field and Google Earth images. To study the changes in surface water areas, since the purpose of this study was to examine the long term from 1990 to 2021, Landsat can be considered the only free space mission with the highest spatial resolution. Therefore, for consistency between different years, Landsat images were used in all years. However, the use of Sentinel images will be tested in future studies.

-Do the Authors see the possibility of implementing the proposed methodology and models for the assessment of other dam reservoirs in the world? What may be the limitations?

To better evaluate the proposed method, it was examined in 8 reservoirs which showed satisfactory performance. Due to the diversity of land cover, topography, and climate of the study areas, it can be said that the proposed method can be implemented in other study areas. Also, due to the automatic nature of the method and the applicability in the GEE system, it can be evaluated in other study areas. Despite the primary focus of the present article being on Iran's dams, in response to your question, the proposed method in 2021 (using Landsat 8 images) for the Al-Massira Dam in Morocco was examined, and you can see the results in the below figure. As it turns out, the proposed method has worked well in identifying water and non-water classes.

- Why did the Authors choose NDVI, MNDWI, EVI and AWEI indicators / indices? Why was GNDVI, WDRVI, NDMI not used?

The used indices are among the most widely used indices in the remote sensing community. On the other hand, due to the presence of both water and non-water cover in the study areas, both water and non-water indices have been used. It should be noted that different combinations of spectral indices from MNDWI, AWEI, NDVI, EVI, NDWI, WI2015, ANDWI, GNDVI, NDMI, and WDRVI were evaluated, among which MNDWI, AWEI, NDVI, and EVI could achieve the highest accuracy.

 

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

The authors have addressed all my comments. I have no more questions at this time.

Reviewer 2 Report

Dear Authors,

Many thanks for the improvement made in your initial text.

I consider your paper can be publish in the present form.