Remote Sensing Scene Classification and Explanation Using RSSCNet and LIME

Round 1

Reviewer 1 Report

In the paper, the authors described a two-stage cyclical learning-rate policy for scene classification. I cannot agree that the presented paper is ready for publication - the novelty is very small. The introduction should be updated to the newest research. The proposed method has no theoretical background and no proper mathematical formulation. Moreover, many parts are missing and are no justification like
a) Why was VGG16 used?
b) Why was this architecture used? Some proofs for choosing this one are needed.

Fig. 14 is very simplified and has no purpose. It is confusing, why some figures/charts are created in python and others in excel.

Author Response

We would like to thank reviewers for their helpful suggestions on the manuscript. Accordingly, all changes in the revised manuscript are highlighted in yellow.

Author Response File: Author Response.pdf

Reviewer 2 Report

First of all, I would like to thank the Editorial Board of Applied Sciences for the opportunity to participate in the review of this paper, as well as the fact that the authors have trusted this journal for the dissemination of the results of their research.

The work addresses a topic of great interest such as the application of deep learning techniques based on the use of neural networks to the classification of remote sensing images. It is a subject that has great perspectives, based on the fact that the volume of images that is captured every day is greater and that we need to automate the processes of interpreting them. As the authors point out, the application of these interpretations is diverse, not only in the strictly cartographic field but in other disciplines such as, for example, medical, astronomy, meteorology, etc.

My first comment regarding the work is that, although I consider it to be interesting, I am not sure that Applied Sciences is the most suitable journal for publication since the application component is limited to the classification of images available in a set of datasets, surely the reader would expect to see a more direct application to a real case study. Perhaps in magazines more oriented to Remote Sensing, such as Remote Sensing, or directly in the field of artificial intelligence, such as AI or Big Data and Cognitive Computing, it could have had a more appropriate fit and greater interest by their readers.

Secondly, I must indicate that it has not been particularly easy for me to evaluate, the truth is that the papers in this area usually are not, since the complexity of the methods applied at the end makes their explanation difficult, as well as the analysis. of the results obtained. In the end, it is true that some data is available (in this case, some known and available datasets), that a methodology has been applied (of which not too much detail is known beyond that it provides good results) and that good results are obtained results (in the datasets considered, although with high training percentages - greater than 50% -). In all this, I have three aspects pending that I consider basic for any scientific publication, firstly, to allow the replicability of the process by the reader (it is not only necessary to indicate what we have done but to show the way so that other people can do it ), secondly, to show what is the practical application of this process beyond achieving a correct classification of the datasets considered. Is it possible to classify real images? What are the problems with the process when the image patches are mixed? How does the system deal with problems frequently raised in the images such as the presence of shadows, clouds, vegetation on objects -for example, houses-, etc.? I consider this information to be basic and is not provided to the readers of this paper. Finally, I consider that it is necessary to highlight in a concrete way what the contributions of the method are, beyond improving the well-classified elements in a certain dataset by a certain percentage, I believe that the flexibility of use -different typologies of images-, also it is something fundamental.

Another aspect that is not clear is the classification procedure itself. Is it understood that each of the image portions is classified according to the dominant element in them? Is the system able to identify the elements within the image portion? I believe that the authors should go the extra mile when presenting the results of this work. Undoubtedly, the results achieved, in view of the classification percentages are spectacular, but in the end, the reader is left with the application of a methodology that is scarcely commented (for example, the processes indicated in figure 5 should be detailed) and that it provides spectacular results according to the confusion matrices (it would be good to incorporate the usual indexes for analysis of these tables, such as kappa index and others), but only that. In this sense, the different images that are incorporated into the text clarify very little about the results obtained (for example, in figure 7 for example).

In this sense, I consider that the work should be subjected to a better review especially in everything related to the presentation of the applied methodology, and the concrete results obtained. Likewise, it would be interesting for the authors to enable the necessary procedures (via GitHub, for example) so that readers can replicate these classification experiments.

 

Formal aspects:

L31. Aerial cameras have been on the market for more than 100 years, capturing images with which practically all the existing cartography on our planet has been generated. The only difference is that, at present, images are available from other sensors (for example, satellite for almost 50 years) and through unmanned devices. What is true is that the amount of information available today is unprecedented (and their quality) and that is necessary to improve your own treatment. It is indicated that only 5% of the images captured by current satellites are processed to obtain information, the rest simply remain as a file.

L96. Why is such resampling applied to dataset 1? I think the rest does not apply. Justify the reasons.

L120. Indicate what the GSD of the RSSCNN7 is. The GSD is considered a fundamental parameter in this type of procedure. It would have been interesting to analyze the influence of the GSD with respect to the quality itself in the classification, as well as the size of the object to be classified.

L134. It is considered necessary to incorporate the reference

Simonyan, Karen & Zisserman, Andrew. (2014). Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv 1409.1556.

L152. Comment in detail on the applied methodology, parameters considered, etc.

L201. Comment on the image in detail, in it, appear yellow and green pixels, what do they mean?

L238. Mention is made of the VGG19, ResNet50, and InceptionV3 models, which have not been discussed previously, establishing what are the differences between the methodologies applied by each of the models. It is also considered necessary to include bibliographic references.

L241. was150 … was 150

L290, L375, (and the like). Labels do not appear on the axes of the graphs. What is represented in them?

L367, L403, L420. This figure tries to show the improvement achieved in the process. I think it would be simpler to indicate what real effect it has on the confusion matrix itself.

L431. The importance of prior adjustment is discussed. How can the person applying the methodology know which is the most appropriate adjustment based on the characteristics of the information they wish to extract? Figure 25 clarifies little about the differences between the different methods applied.

Author Response

We would like to thank reviewers for their helpful suggestions on the manuscript. Accordingly, all changes in the revised manuscript are highlighted in yellow.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

Accept.

Reviewer 2 Report

First of all, I would like to thank the authors for their interest in attending to the suggestions that were raised in my previous review. I sincerely believe that the article in its second version has substantially improved its quality.

However, there are two aspects that I consider should be the object of attention by the Editorial Board of this journal:

a) firstly, despite the authors' comments, I still consider that there are more appropriate journals within the journal itself. Editorial MDPI according to the subject of this work,

b) on the other hand, the comment that the authors raise about non-publication of the codes or parameters used in this work until relevant works are published ("best network weights and source code will be considered for release after relevant papers are published."). Does this mean that they do not consider this work to be relevant? In this sense, it must be taken into account that this work applied to standard datasets will lose great interest if it cannot be reproduced by Applied Sciences readers. In short, the article meets the requirements for publication, but in my opinion, it contributes little, and has little interest in the journal's readers, unless the codes and parameters considered are published.

This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.