Submit to Remote Sensing Review for Remote Sensing Propose a Special Issue

Journal Menu

Journal Browser

► Journal Browser

Knowledge Graph-Guided Deep Learning for Remote Sensing Image Understanding

Print Special Issue Flyer
Special Issue Editors
Special Issue Information
Keywords
Published Papers

A special issue of Remote Sensing (ISSN 2072-4292). This special issue belongs to the section "AI Remote Sensing".

Deadline for manuscript submissions: closed (31 December 2021) | Viewed by 27734

Share This Special Issue

Special Issue Editors

Prof. Dr. Yansheng Li

E-Mail Website
Guest Editor

School of Remote Sensing and Information Engineering, Wuhan University, Wuhan 430079, China
Interests: knowledge graph; deep learning; big data mining
Special Issues, Collections and Topics in MDPI journals

Prof. Dr. Yongjun Zhang

E-Mail Website
Guest Editor

School of Remote Sensing and Information Engineering, Wuhan University, Wuhan 430079, China
Interests: digital photogrammetry and remote sensing; computer vision; geometric processing of aerial and space optical imagery; multi-source spatial data integration; integrated sensor calibration and orientation; low-altitude UAV photogrammetry; combined bundle block adjustment of multi-source datasets; LiDAR and image integration; digital city modeling; visual inspection of industrial parts; intelligent extraction of remote sensing information and knowledge modeling
Special Issues, Collections and Topics in MDPI journals

Dr. Chen Chen

E-Mail Website
Guest Editor

Department of Electrical & Computer Engineering, University of North Carolina at Charlotte, Charlotte, NC 28223, USA
Interests: compressed sensing; signal and image processing; pattern recognition; computer vision; hyperspectral image analysis
Special Issues, Collections and Topics in MDPI journals

Special Issue Information

Dear Colleagues,

As one of the most significant achievements in the artificial intelligence (AI) domain, deep learning has achieved tremendous success in remote sensing image understanding, including scene classification, semantic segmentation, and object detection. As is well known, deep learning is a classic data-driven technique and can often be trained in an end-to-end manner. As an inevitable disadvantage, it is incredibly difficult for deep learning to leverage the prior domain knowledge, which means common sense about remote sensing image interpretation and holds a natural generalization property. Despite deep learning’s aforementioned success in remote sensing image understanding, deep learning-based methods are susceptible to noise interference and lack the basic but vital cognition and inference ability. Consequently, deep learning still cannot fully meet the high-reliability demand of remote sensing image interpretation.

As another research hotspot in the field of AI, knowledge graphs work by explicitly representing the domain concepts and their relationships as a collection of triples and have strong knowledge representation capabilities and semantic reasoning capabilities. In order to fully reflect the domain characteristic of remote sensing, how to collaboratively construct a remote sensing knowledge graph with the aid of domain experts and data-driven methods deserves much more exploration. Based on remote sensing knowledge graphs, the semantic reasoning technique becomes very promising to fully leverage the rich semantic information in remote sensing images. Therefore, combining knowledge-driven knowledge graph reasoning and data-driven deep learning would be a promising research avenue to realize intelligent remote sensing image interpretation. The joint technique of knowledge graph reasoning and deep learning benefits assimilates the complementary advantages of these two techniques. It not only makes full use of the low-level and middle-level information mining ability of deep learning but also exerts the high-level semantic reasoning ability of knowledge graph reasoning.

This Special Issue calls for innovative remote sensing image understanding theory and methods by combining deep learning and knowledge graph reasoning. The topics include but are not limited to the following:

Interpretive deep learning methods for remote sensing image understanding, including scene classification, semantic segmentation, and object detection;
Domain knowledge modeling and reasoning for remote sensing image understanding;
Data-driven remote sensing scene graph generation methods;
Crowd-sourced remote sensing knowledge graph construction methods;
High-reliability remote sensing image understanding methods by combining deep learning and knowledge reasoning;
Representation learning of remote sensing knowledge graph for advanced remote sensing image understanding tasks: fine-grained image classification, few-shot image classification, zero-shot image classification, and so forth;
Language-level remote sensing image understanding: image caption, visual question answering, cross-modal retrieval between text and images, and so forth;
Large-scale and long-term remote sensing image understanding methods based on knowledge-guided deep learning.

Dr. Yansheng Li
Prof. Yongjun Zhang
Dr. Chen Chen
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Remote Sensing is an international peer-reviewed open access semimonthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2700 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

high-reliability remote sensing image understanding
knowledge graph construction and reasoning
interpretive deep learning
knowledge graph-guided deep learning

Published Papers (6 papers)

Download All Papers

Order results

Result details

Show export options Show export options

Select all

Export citation of selected articles as:

Research

17 pages, 4036 KiB

Open AccessArticle

The Impacts of Farmland Loss on Regional Food Self-Sufficiency in Yangtze River Delta Urban Agglomeration over Last Two Decades

by Xuelin Duan, Qingxiang Meng, Xufeng Fei, Meng Lin and Rui Xiao

Remote Sens. 2021, 13(17), 3514; https://doi.org/10.3390/rs13173514 - 04 Sep 2021

Cited by 12 | Viewed by 2089

Abstract

Food security is essential for human survival and sustainable development. Due to rapid urbanization and industrialization, the farmland loss in Yangtze River Delta Urban Agglomeration (YRDUA) has threatened food security. Thus, this study intended to quantify the farmland loss and assess its impacts on food security in the YRDUA from 2000 to 2020 at three scales based on the Google Earth Engine platform. Our results show that the area of farmland in YRDUA is decreasing at a rather high speed in the past 20 years and the trend is getting worse. At the urban agglomeration scale, there was a farmland deficit of 0.44 million ha in 2000, followed by larger farmland deficits in 2010 and 2020. At the city scale, Shanghai had the largest scarcity of farmland. At the urban subgroup scale, Subgroup I in the west and Subgroup II in the north always maintained an oversupply of farmland, while Subgroup III in the east and Subgroup IV in the south faced serious food security problems. Our study suggests that farmland must be protected in YRDUA in order to ensure food self-sufficiency and promote regional sustainability. Full article

(This article belongs to the Special Issue Knowledge Graph-Guided Deep Learning for Remote Sensing Image Understanding)

► Show Figures

Figure 1

20 pages, 8278 KiB

Open AccessArticle

Class-Wise Fully Convolutional Network for Semantic Segmentation of Remote Sensing Images

by Tian Tian, Zhengquan Chu, Qian Hu and Li Ma

Remote Sens. 2021, 13(16), 3211; https://doi.org/10.3390/rs13163211 - 13 Aug 2021

Cited by 16 | Viewed by 2745

Abstract

Semantic segmentation is a fundamental task in remote sensing image interpretation, which aims to assign a semantic label for every pixel in the given image. Accurate semantic segmentation is still challenging due to the complex distributions of various ground objects. With the development of deep learning, a series of segmentation networks represented by fully convolutional network (FCN) has made remarkable progress on this problem, but the segmentation accuracy is still far from expectations. This paper focuses on the importance of class-specific features of different land cover objects, and presents a novel end-to-end class-wise processing framework for segmentation. The proposed class-wise FCN (C-FCN) is shaped in the form of an encoder-decoder structure with skip-connections, in which the encoder is shared to produce general features for all categories and the decoder is class-wise to process class-specific features. To be detailed, class-wise transition (CT), class-wise up-sampling (CU), class-wise supervision (CS), and class-wise classification (CC) modules are designed to achieve the class-wise transfer, recover the resolution of class-wise feature maps, bridge the encoder and modified decoder, and implement class-wise classifications, respectively. Class-wise and group convolutions are adopted in the architecture with regard to the control of parameter numbers. The method is tested on the public ISPRS 2D semantic labeling benchmark datasets. Experimental results show that the proposed C-FCN significantly improves the segmentation performances compared with many state-of-the-art FCN-based networks, revealing its potentials on accurate segmentation of complex remote sensing images. Full article

(This article belongs to the Special Issue Knowledge Graph-Guided Deep Learning for Remote Sensing Image Understanding)

► Show Figures

Figure 1

21 pages, 6475 KiB

Open AccessArticle

MSResNet: Multiscale Residual Network via Self-Supervised Learning for Water-Body Detection in Remote Sensing Imagery

by Bo Dang and Yansheng Li

Remote Sens. 2021, 13(16), 3122; https://doi.org/10.3390/rs13163122 - 06 Aug 2021

Cited by 32 | Viewed by 4836

Abstract

Driven by the urgent demand for flood monitoring, water resource management and environmental protection, water-body detection in remote sensing imagery has attracted increasing research attention. Deep semantic segmentation networks (DSSNs) have gradually become the mainstream technology used for remote sensing image water-body detection, but two vital problems remain. One problem is that the traditional structure of DSSNs does not consider multiscale and multishape characteristics of water bodies. Another problem is that a large amount of unlabeled data is not fully utilized during the training process, but the unlabeled data often contain meaningful supervision information. In this paper, we propose a novel multiscale residual network (MSResNet) that uses self-supervised learning (SSL) for water-body detection. More specifically, our well-designed MSResNet distinguishes water bodies with different scales and shapes and helps retain the detailed boundaries of water bodies. In addition, the optimization of MSResNet with our SSL strategy can improve the stability and universality of the method, and the presented SSL approach can be flexibly extended to practical applications. Extensive experiments on two publicly open datasets, including the 2020 Gaofen Challenge water-body segmentation dataset and the GID dataset, demonstrate that our MSResNet can obviously outperform state-of-the-art deep learning backbones and that our SSL strategy can further improve the water-body detection performance. Full article

(This article belongs to the Special Issue Knowledge Graph-Guided Deep Learning for Remote Sensing Image Understanding)

► Show Figures

Graphical abstract

18 pages, 6658 KiB

Open AccessArticle

Subtask Attention Based Object Detection in Remote Sensing Images

by Shengzhou Xiong, Yihua Tan, Yansheng Li, Cai Wen and Pei Yan

Remote Sens. 2021, 13(10), 1925; https://doi.org/10.3390/rs13101925 - 14 May 2021

Cited by 8 | Viewed by 2457

Abstract

Object detection in remote sensing images (RSIs) is one of the basic tasks in the field of remote sensing image automatic interpretation. In recent years, the deep object detection frameworks of natural scene images (NSIs) have been introduced into object detection on RSIs, and the detection performance has improved significantly because of the powerful feature representation. However, there are still many challenges concerning the particularities of remote sensing objects. One of the main challenges is the missed detection of small objects which have less than five percent of the pixels of the big objects. Generally, the existing algorithms choose to deal with this problem by multi-scale feature fusion based on a feature pyramid. However, the benefits of this strategy are limited, considering that the location of small objects in the feature map will disappear when the detection task is processed at the end of the network. In this study, we propose a subtask attention network (StAN), which handles the detection task directly on the shallow layer of the network. First, StAN contains one shared feature branch and two subtask attention branches of a semantic auxiliary subtask and a detection subtask based on the multi-task attention network (MTAN). Second, the detection branch uses only low-level features considering small objects. Third, the attention map guidance mechanism is put forward to optimize the network for keeping the identification ability. Fourth, the multi-dimensional sampling module (MdS), global multi-view channel weights (GMulW) and target-guided pixel attention (TPA) are designed for further improvement of the detection accuracy in complex scenes. The experimental results on the NWPU VHR-10 dataset and DOTA dataset demonstrated that the proposed algorithm achieved the SOTA performance, and the missed detection of small objects decreased. On the other hand, ablation experiments also proved the effects of MdS, GMulW and TPA. Full article

(This article belongs to the Special Issue Knowledge Graph-Guided Deep Learning for Remote Sensing Image Understanding)

► Show Figures

Graphical abstract

22 pages, 2789 KiB

Open AccessArticle

Hyperspectral Image Classification across Different Datasets: A Generalization to Unseen Categories

by Erting Pan, Yong Ma, Fan Fan, Xiaoguang Mei and Jun Huang

Remote Sens. 2021, 13(9), 1672; https://doi.org/10.3390/rs13091672 - 26 Apr 2021

Cited by 15 | Viewed by 4552

Abstract

With the rapid developments of hyperspectral imaging, the cost of collecting hyperspectral data has been lower, while the demand for reliable and detailed hyperspectral annotations has been much more substantial. However, limited by the difficulties of labelling annotations, most existing hyperspectral image (HSI) classification methods are trained and evaluated on a single hyperspectral data cube. It brings two significant challenges. On the one hand, many algorithms have reached a nearly perfect classification accuracy, but their trained models are hard to generalize to other datasets. On the other hand, since different hyperspectral datasets are usually not collected in the same scene, different datasets will contain different classes. To address these issues, in this paper, we propose a new paradigm for HSI classification, which is training and evaluating separately across different hyperspectral datasets. It is of great help to labelling hyperspectral data. However, it has rarely been studied in the hyperspectral community. In this work, we utilize a three-phase scheme, including feature embedding, feature mapping, and label reasoning. More specifically, we select a pair of datasets acquired by the same hyperspectral sensor, and the classifier learns from one dataset and then evaluated it on the other. Inspired by the latest advances in zero-shot learning, we introduce label semantic representation to establish associations between seen categories in the training set and unseen categories in the testing set. Extensive experiments on two pairs of datasets with different comparative methods have shown the effectiveness and potential of zero-shot learning in HSI classification. Full article

(This article belongs to the Special Issue Knowledge Graph-Guided Deep Learning for Remote Sensing Image Understanding)

► Show Figures

Figure 1

22 pages, 8110 KiB

Open AccessArticle

Combining Deep Semantic Segmentation Network and Graph Convolutional Neural Network for Semantic Segmentation of Remote Sensing Imagery

by Song Ouyang and Yansheng Li

Remote Sens. 2021, 13(1), 119; https://doi.org/10.3390/rs13010119 - 31 Dec 2020

Cited by 54 | Viewed by 7869

Abstract

Although the deep semantic segmentation network (DSSN) has been widely used in remote sensing (RS) image semantic segmentation, it still does not fully mind the spatial relationship cues between objects when extracting deep visual features through convolutional filters and pooling layers. In fact, the spatial distribution between objects from different classes has a strong correlation characteristic. For example, buildings tend to be close to roads. In view of the strong appearance extraction ability of DSSN and the powerful topological relationship modeling capability of the graph convolutional neural network (GCN), a DSSN-GCN framework, which combines the advantages of DSSN and GCN, is proposed in this paper for RS image semantic segmentation. To lift the appearance extraction ability, this paper proposes a new DSSN called the attention residual U-shaped network (AttResUNet), which leverages residual blocks to encode feature maps and the attention module to refine the features. As far as GCN, the graph is built, where graph nodes are denoted by the superpixels and the graph weight is calculated by considering the spectral information and spatial information of the nodes. The AttResUNet is trained to extract the high-level features to initialize the graph nodes. Then the GCN combines features and spatial relationships between nodes to conduct classification. It is worth noting that the usage of spatial relationship knowledge boosts the performance and robustness of the classification module. In addition, benefiting from modeling GCN on the superpixel level, the boundaries of objects are restored to a certain extent and there are less pixel-level noises in the final classification result. Extensive experiments on two publicly open datasets show that DSSN-GCN model outperforms the competitive baseline (i.e., the DSSN model) and the DSSN-GCN when adopting AttResUNet achieves the best performance, which demonstrates the advance of our method. Full article

(This article belongs to the Special Issue Knowledge Graph-Guided Deep Learning for Remote Sensing Image Understanding)

► Show Figures

Journal Menu

Journal Browser

Knowledge Graph-Guided Deep Learning for Remote Sensing Image Understanding

Share This Special Issue

Special Issue Editors

Special Issue Information

Keywords

Published Papers (6 papers)

Research

Further Information

Guidelines

MDPI Initiatives

Follow MDPI