remotesensing-logo

Journal Browser

Journal Browser

Multi-Modal and Multi-Task Learning in Photogrammetry and Remote Sensing

A special issue of Remote Sensing (ISSN 2072-4292). This special issue belongs to the section "Remote Sensing Image Processing".

Deadline for manuscript submissions: closed (15 March 2023) | Viewed by 5150

Special Issue Editors

Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100190, China
Interests: remote sensing image interpretation; artificial intelligence; machine learning; computer vision
Special Issues, Collections and Topics in MDPI journals
Institute for Applied Computer Science, Universität der Bundeswehr München, 85577 Neubiberg, Germany
Interests: computer vision; remote sensing; photogrammetry; pattern recognition; machine learning

E-Mail Website
Guest Editor
Institute of Photogrammetry and Remote Sensing, Karlsruhe Institute of Technology (KIT), Englerstra_e 7, 76131 Karlsruhe, Germany
Interests: computer vision; pattern recognition; machine learning; photogrammetry; remote sensing
Special Issues, Collections and Topics in MDPI journals

E-Mail Website
Guest Editor
Institute for Applied Computer Science, Universität der Bundeswehr München, 85577 Neubiberg, Germany
Interests: photogrammetry; computer vision; 3D reconstruction; structure from motion

Special Issue Information

Dear Colleagues,

With the availability of large amounts of remote sensing data from different sensors, multi-modal data processing and analysis techniques have attracted increasing interest in the remote sensing and geoscience communities. However, due to the differences in imaging sensor principle and resolution between different modalities, the appropriate representation of their complementary information remains largely challenging.

In recent years, the great success of deep learning has provided an important opportunity for intelligent information extraction from multi-source data. A large number of scholars have tried to introduce deep learning methods into the field of remote sensing, the performance of which is significantly better than traditional methods. However, the very unique characteristics of remote sensing data, such as large scale, image quality affected by cloudy weather, complex image background, and multi-scale objects, make it difficult for existing deep learning methods to further improve performance. Currently, most remote sensing image interpretation methods are proposed for data of a specific modality and for specific tasks, resulting in a certain bottleneck in the development of multi-modal and multi-task learning in the field of remote sensing. Therefore, on the basis of understanding the characteristics of multi-modal remote sensing images, it is necessary to design a suitable feature extraction structure to make the model have better generalization ability to multi-modal or multi-task learning.

This Special Issue will report cutting-edge models, methods, and system tools tailored for multiple tasks in dealing with multi-modal remote sensing data. It aims at boosting the interpretation of remote sensing data towards more accurate, autonomous, and cost-effective quality levels.

The Special Issue invites authors to submit contributions in (but not limited to) the following topics:

  • Multi-modal data fusion, analysis, and interpretation
  • Multi-temporal remote sensing data analysis
  • Multi-task learning with deep neural networks
  • Multi-modal pre-training algorithms and architectures
  • Image fusion for remote sensing
  • Deep transfer learning methods for remote sensing
  • Weakly supervised, semi-supervised and self-supervised learning methods for remote sensing
  • New datasets, methods, and platforms for image segmentation, object detection and classification

Dr. Xian Sun
Dr. Hai Huang
Dr. Martin Weinmann
Prof. Dr. Helmut Mayer
Guest Editors

Mr. Peijin Wang
Guest Editor Assistant
Affiliation: Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100190, China
E-Mail:wangpj@aircas.ac.cn
Interests: remote sensing image interpretation; machine learning; computer vision

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Remote Sensing is an international peer-reviewed open access semimonthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2700 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • remote sensing data
  • neural networks
  • image fusion
  • image segmentation
  • object detection
  • classification
  • photogrammetry

Published Papers (3 papers)

Order results
Result details
Select all
Export citation of selected articles as:

Research

18 pages, 7486 KiB  
Article
Aircraft-LBDet: Multi-Task Aircraft Detection with Landmark and Bounding Box Detection
by Yihang Ma, Deyun Zhou, Yuting He, Liangjin Zhao, Peirui Cheng, Hao Li and Kaiqiang Chen
Remote Sens. 2023, 15(10), 2485; https://doi.org/10.3390/rs15102485 - 09 May 2023
Cited by 1 | Viewed by 1271
Abstract
With the rapid development of artificial intelligence and computer vision, deep learning has become widely used for aircraft detection. However, aircraft detection is still a challenging task due to the small target size and dense arrangement of aircraft and the complex backgrounds in [...] Read more.
With the rapid development of artificial intelligence and computer vision, deep learning has become widely used for aircraft detection. However, aircraft detection is still a challenging task due to the small target size and dense arrangement of aircraft and the complex backgrounds in remote sensing images. Existing remote sensing aircraft detection methods were mainly designed based on algorithms employed in general object detection methods. However, these methods either tend to ignore the key structure and size information of aircraft targets or have poor detection effects on densely distributed aircraft targets. In this paper, we propose a novel multi-task aircraft detection algorithm. Firstly, a multi-task joint training method is proposed, which provides richer semantic structure features for bounding box localization through landmark detection. Secondly, a multi-task inference algorithm is introduced that utilizes landmarks to provide additional supervision for bounding box NMS (non-maximum suppression) filtering, effectively reducing false positives. Finally, a novel loss function is proposed as a constrained optimization between bounding boxes and landmarks, which further improves aircraft detection accuracy. Experiments on the UCAS-AOD dataset demonstrated the state-of-the-art precision and efficiency of our proposed method compared to existing approaches. Furthermore, our ablation study revealed that the incorporation of our designed modules could significantly enhance network performance. Full article
Show Figures

Graphical abstract

18 pages, 7775 KiB  
Article
Research on A Special Hyper-Pixel for SAR Radiometric Monitoring
by Songtao Shangguan, Xiaolan Qiu and Kun Fu
Remote Sens. 2023, 15(8), 2175; https://doi.org/10.3390/rs15082175 - 20 Apr 2023
Cited by 2 | Viewed by 908
Abstract
The objects presented in synthetic-aperture radar (SAR) images are the products of the joint actions of ground objects and SAR sensors in specific geospatial contexts. With the accumulation of massive time-domain SAR data, scholars have the opportunity to better understand ground-object targets and [...] Read more.
The objects presented in synthetic-aperture radar (SAR) images are the products of the joint actions of ground objects and SAR sensors in specific geospatial contexts. With the accumulation of massive time-domain SAR data, scholars have the opportunity to better understand ground-object targets and sensor systems, providing some useful feedback for SAR-data processing. Aiming at normalized and low-cost SAR radiometric monitoring, this paper proposes a new hyper-pixel concept for handling multi-pixel ensembles of semantic ground targets. The special hyper-pixel in this study refers to low-rise single-family residential areas, and its radiation reference is highly stable in the time domain when the other dimensions are fixed. The stability of its radiometric data can reach the level of 0.3 dB (1σ), as verified by the multi-temporal data from Sentinel-1. A comparison with tropical-rainforest data verified its availability for SAR radiometric monitoring, and possible radiation variations and radiation-intensity shifts in the Sentinel-1B SAR products ere experimentally monitored. In this paper, the effects of seasonal climate and of the relative geometrical states observed on the intensity of the hyper-pixel’s radiation are investigated. This paper proposes a novel hyper-pixel concept for processing and interpreting SAR-image data. The proposed residential hyper-pixel is shown to be useful in multi-temporal-data observations for normalized radiometric monitoring and has the potential to be used for cross-calibration, in addition to other applications. Full article
Show Figures

Figure 1

25 pages, 27339 KiB  
Article
Weakly Supervised Semantic Segmentation in Aerial Imagery via Cross-Image Semantic Mining
by Ruixue Zhou, Zhiqiang Yuan, Xuee Rong, Weicong Ma, Xian Sun, Kun Fu and Wenkai Zhang
Remote Sens. 2023, 15(4), 986; https://doi.org/10.3390/rs15040986 - 10 Feb 2023
Cited by 3 | Viewed by 2163
Abstract
Weakly Supervised Semantic Segmentation (WSSS) with only image-level labels reduces the annotation burden and has been rapidly developed in recent years. However, current mainstream methods only employ a single image’s information to localize the target and do not account for the relationships across [...] Read more.
Weakly Supervised Semantic Segmentation (WSSS) with only image-level labels reduces the annotation burden and has been rapidly developed in recent years. However, current mainstream methods only employ a single image’s information to localize the target and do not account for the relationships across images. When faced with Remote Sensing (RS) images, limited to complex backgrounds and multiple categories, it is challenging to locate and differentiate between the categories of targets. As opposed to previous methods that mostly focused on single-image information, we propose CISM, a novel cross-image semantic mining WSSS framework. CISM explores cross-image semantics in multi-category RS scenes for the first time with two novel loss functions: the Common Semantic Mining (CSM) loss and the Non-common Semantic Contrastive (NSC) loss. In particular, prototype vectors and the Prototype Interactive Enhancement (PIE) module were employed to capture semantic similarity and differences across images. To overcome category confusions and closely related background interferences, we integrated the Single-Label Secondary Classification (SLSC) task and the corresponding single-label loss into our framework. Furthermore, a Multi-Category Sample Generation (MCSG) strategy was devised to balance the distribution of samples among various categories and drastically increase the diversity of images. The above designs facilitated the generation of more accurate and higher-granularity Class Activation Maps (CAMs) for each category of targets. Our approach is superior to the RS dataset based on extensive experiments and is the first WSSS framework to explore cross-image semantics in multi-category RS scenes and obtain cutting-edge state-of-the-art results on the iSAID dataset by only using image-level labels. Experiments on the PASCAL VOC2012 dataset also demonstrated the effectiveness and competitiveness of the algorithm, which pushes the mean Intersection-Over-Union (mIoU) to 67.3% and 68.5% on the validation and test sets of PASCAL VOC2012, respectively. Full article
Show Figures

Graphical abstract

Back to TopTop