Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
7.9
5.0
Journals
Remote Sensing
Special Issues
Multimodality Fusion in Remote Sensing: Data, Algorithms, and Applications
remotesensing-logo
Submit to Remote Sensing Review for Remote Sensing Propose a Special Issue

Journal Menu

► Journal Menu

Journal Browser

► Journal Browser
share announcement

Multimodality Fusion in Remote Sensing: Data, Algorithms, and Applications

A special issue of Remote Sensing (ISSN 2072-4292). This special issue belongs to the section "Remote Sensing Image Processing".

Deadline for manuscript submissions: closed (1 April 2023) | Viewed by 8985

Special Issue Editors

Dr. Ying Qu

E-Mail Website
Guest Editor
1. Faculty of Geographical Science, Beijing Normal University, Beijing 100875, China
2. Department of Electrical Engineering and Computer Science, The University of Tennessee, Knoxville, TN 37996, USA
Interests: super-resolution; unmixing; anomaly detection; classification; deep learning
Dr. Gemine Vivone

E-Mail Website
Guest Editor
Institute of Methodologies for Environmental Analysis, CNR-IMAA, 85050 Tito Scalo, Italy
Interests: statistical signal processing; detection of remotely sensed images; data fusion; tracking algorithms
Special Issues, Collections and Topics in MDPI journals
Dr. Liqiang Zhang

E-Mail Website1 Website2
Guest Editor
Faculty of Geographical Science, Beijing Normal University, Beijing 100875, China
Interests: remote sensing imagery recognition; 3D modeling and reconstruction; deep learning
Special Issues, Collections and Topics in MDPI journals
Dr. Hairong Qi

E-Mail Website
Guest Editor
Department of Electrical Engineering and Computer Science, The University of Tennessee, Knoxville, TN 37996-2100, USA
Interests: image processing; computer vision; machine learning; collaborative information processing in sensor networks
Special Issues, Collections and Topics in MDPI journals

Special Issue Information

Dear Colleagues,

Multimodality fusion plays an essential role in remote sensing since complementary and diverse information of data collected from various sensory systems can be integrated that would result in robust performance with high fidelity. Although a wide spectrum of fusion methods have been well developed through decades of innovations, the tasks associated with remote sensing present unique challenges to multimodality fusion, including, for example, (1) the misalignment between multimodalities due to nonrigid distortions, (2) the spectral variability and spatial heterogeneity due to regional variances, (3) the extremely vast area covered by remote sensors versus the lack of labeled data, etc.  These unresolved problems would largely hinder the application of multimodality fusion in real-world remote sensing applications, such as the fusion of hyperspectral image, multispectral image, panchromatic image, light detection and ranging (LiDAR), and synthetic aperture radar (SAR), etc. 

This Special Issue aims at presenting new benchmark datasets, novel algorithms, and emerging applications that address the challenges of multimodality fusion specifically for remote sensing. We welcome original manuscripts that focused on, but not limited to, one or more of the following topics:

  • Registration and data alignment
  • Pixel-level data fusion
  • Feature-level fusion
  • Decision-level fusion
  • Fusion based representation learning
  • Fusion with misaligned data sources
  • Fusion based spatio-temporal-spectral analysis
  • Fusion models without labeled data or with very few labeled data
  • Adaptive models response to changing environment or changing quality of input data
  • Benchmark datasets and novel evaluation metrics
  • Multimodality fusion for remote sensing applications, including, but not limited to, classification, spectral unmixing, target detection, point cloud processing, change detection, segmentation, etc.

Dr. Ying Qu
Dr. Gemine Vivone
Dr. Liqiang Zhang
Dr. Hairong Qi
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Remote Sensing is an international peer-reviewed open access semimonthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2700 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • multimodality fusion
  • feature fusion
  • representation learning
  • deep learning

Published Papers (5 papers)

Download All Papers
Order results
Result details
Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:

Research

22 pages, 3896 KiB  
Article
Multiband Image Fusion via Regularization on a Riemannian Submanifold
by Han Pan, Zhongliang Jing, Henry Leung, Pai Peng and Hao Zhang
Remote Sens. 2023, 15(18), 4370; https://doi.org/10.3390/rs15184370 - 05 Sep 2023
Viewed by 828
Abstract
Multiband image fusion aims to generate high spatial resolution hyperspectral images by combining hyperspectral, multispectral or panchromatic images. However, fusing multiband images remains a challenge due to the identifiability and tracking of the underlying subspace across varying modalities and resolutions. In this paper, [...] Read more.
Multiband image fusion aims to generate high spatial resolution hyperspectral images by combining hyperspectral, multispectral or panchromatic images. However, fusing multiband images remains a challenge due to the identifiability and tracking of the underlying subspace across varying modalities and resolutions. In this paper, an efficient multiband image fusion model is proposed to investigate the latent structures and intrinsic physical properties of a multiband image, which is characterized by the Riemannian submanifold regularization method, nonnegativity and sum-to-one constraints. An alternating minimization scheme is proposed to recover the latent structures of the subspace via the manifold alternating direction method of multipliers (MADMM). The subproblem with Riemannian submanifold regularization is tackled by the projected Riemannian trust-region method with guaranteed convergence. The effectiveness of the proposed method is demonstrated on two multiband image fusion problems: (1) hyperspectral and panchromatic image fusion and (2) hyperspectral, multispectral and panchromatic image fusion. The experimental results confirm that our method demonstrates superior fusion performance with respect to competitive state-of-the-art fusion methods. Full article
(This article belongs to the Special Issue Multimodality Fusion in Remote Sensing: Data, Algorithms, and Applications)
►▼ Show Figures

Figure 1

23 pages, 5089 KiB  
Article
LPGAN: A LBP-Based Proportional Input Generative Adversarial Network for Image Fusion
by Dongxu Yang, Yongbin Zheng, Wanying Xu, Peng Sun and Di Zhu
Remote Sens. 2023, 15(9), 2440; https://doi.org/10.3390/rs15092440 - 06 May 2023
Cited by 2 | Viewed by 1175
Abstract
Image fusion is the process of combining multiple input images from single or multiple imaging modalities into a fused image, which is expected to be more informative for human or machine perception as compared to any of the input images. In this paper, [...] Read more.
Image fusion is the process of combining multiple input images from single or multiple imaging modalities into a fused image, which is expected to be more informative for human or machine perception as compared to any of the input images. In this paper, we propose a novel method based on deep learning for fusing infrared images and visible images, named the local binary pattern (LBP)-based proportional input generative adversarial network (LPGAN). In the image fusion task, the preservation of structural similarity and image gradient information is contradictory, and it is difficult for both to achieve good performance at the same time. To solve this problem, we innovatively introduce LBP into GANs, enabling the network to have stronger texture feature extraction and utilization capabilities, as well as anti-interference capabilities. In the feature extraction stage, we introduce a pseudo-Siamese network for the generator to extract the detailed features and the contrast features. At the same time, considering the characteristic distribution of different modal images, we propose a 1:4 scale input mode. Extensive experiments on the publicly available TNO dataset and CVC14 dataset show that the proposed method achieves the state-of-the-art performance. We also test the universality of LPGAN by fusing RGB and infrared images on the RoadScene dataset and medical images. In addition, LPGAN is applied to multi-spectral remote sensing image fusion. Both qualitative and quantitative experiments demonstrate that our LPGAN can not only achieve good structural similarity, but also retain richly detailed information. Full article
(This article belongs to the Special Issue Multimodality Fusion in Remote Sensing: Data, Algorithms, and Applications)
►▼ Show Figures

Figure 1

19 pages, 6458 KiB  
Article
Local Adaptive Illumination-Driven Input-Level Fusion for Infrared and Visible Object Detection
by Jiawen Wu, Tao Shen, Qingwang Wang, Zhimin Tao, Kai Zeng and Jian Song
Remote Sens. 2023, 15(3), 660; https://doi.org/10.3390/rs15030660 - 22 Jan 2023
Cited by 16 | Viewed by 2354
Abstract
Remote sensing object detection based on the combination of infrared and visible images can effectively adapt to the around-the-clock and changeable illumination conditions. However, most of the existing infrared and visible object detection networks need two backbone networks to extract the features of [...] Read more.
Remote sensing object detection based on the combination of infrared and visible images can effectively adapt to the around-the-clock and changeable illumination conditions. However, most of the existing infrared and visible object detection networks need two backbone networks to extract the features of two modalities, respectively. Compared with the single modality detection network, this greatly increases the amount of calculation, which limits its real-time processing on the vehicle and unmanned aerial vehicle (UAV) platforms. Therefore, this paper proposes a local adaptive illumination-driven input-level fusion module (LAIIFusion). The previous methods for illumination perception only focus on the global illumination, ignoring the local differences. In this regard, we design a new illumination perception submodule, and newly define the value of illumination. With more accurate area selection and label design, the module can more effectively perceive the scene illumination condition. In addition, aiming at the problem of incomplete alignment between infrared and visible images, a submodule is designed for the rapid estimation of slight shifts. The experimental results show that the single modality detection algorithm based on LAIIFusion can ensure a large improvement in accuracy with a small loss of speed. On the DroneVehicle dataset, our module combined with YOLOv5L could achieve the best performance. Full article
(This article belongs to the Special Issue Multimodality Fusion in Remote Sensing: Data, Algorithms, and Applications)
►▼ Show Figures

Figure 1

15 pages, 4118 KiB  
Communication
Attention Fusion of Transformer-Based and Scale-Based Method for Hyperspectral and LiDAR Joint Classification
by Maqun Zhang, Feng Gao, Tiange Zhang, Yanhai Gan, Junyu Dong and Hui Yu
Remote Sens. 2023, 15(3), 650; https://doi.org/10.3390/rs15030650 - 21 Jan 2023
Cited by 7 | Viewed by 1951
Abstract
In recent years, there have been many multimodal works in the field of remote sensing, and most of them have achieved good results in the task of land-cover classification. However, multi-scale information is seldom considered in the multi-modal fusion process. Secondly, the multimodal [...] Read more.
In recent years, there have been many multimodal works in the field of remote sensing, and most of them have achieved good results in the task of land-cover classification. However, multi-scale information is seldom considered in the multi-modal fusion process. Secondly, the multimodal fusion task rarely considers the application of attention mechanism, resulting in a weak representation of the fused feature. In order to better use the multimodal data and reduce the losses caused by the fusion of different modalities, we proposed a TRMSF (Transformer and Multi-scale fusion) network for land-cover classification based on HSI (hyperspectral images) and LiDAR (Light Detection and Ranging) images joint classification. The network enhances multimodal information fusion ability by the method of attention mechanism from Transformer and enhancement using multi-scale information to fuse features from different modal structures. The network consists of three parts: multi-scale attention enhancement module (MSAE), multimodality fusion module (MMF) and multi-output module (MOM). MSAE enhances the ability of feature representation from extracting different multi-scale features of HSI, which are used to fuse with LiDAR feature, respectively. MMF integrates the data of different modalities through attention mechanism, thereby reducing the loss caused by the data fusion of different modal structures. MOM optimizes the network by controlling different outputs and enhances the stability of the results. The experimental results show that the proposed network is effective in multimodality joint classification. Full article
(This article belongs to the Special Issue Multimodality Fusion in Remote Sensing: Data, Algorithms, and Applications)
►▼ Show Figures

Graphical abstract

20 pages, 2687 KiB  
Article
Sugarcane Biomass Prediction with Multi-Mode Remote Sensing Data Using Deep Archetypal Analysis and Integrated Learning
by Zhuowei Wang, Yusheng Lu, Genping Zhao, Chuanliang Sun, Fuhua Zhang and Su He
Remote Sens. 2022, 14(19), 4944; https://doi.org/10.3390/rs14194944 - 03 Oct 2022
Cited by 6 | Viewed by 1893
Abstract
The use of multi-mode remote sensing data for biomass prediction is of potential value to aid planting management and yield maximization. In this study, an advanced biomass estimation approach for sugarcane fields is proposed based on multi-source remote sensing data. Since feature interpretability [...] Read more.
The use of multi-mode remote sensing data for biomass prediction is of potential value to aid planting management and yield maximization. In this study, an advanced biomass estimation approach for sugarcane fields is proposed based on multi-source remote sensing data. Since feature interpretability in agricultural data mining is significant, a feature extraction method of deep archetypal analysis (DAA) that has good model interpretability is introduced and aided by principal component analysis (PCA) for feature mining from the multi-mode multispectral and light detection and ranging (LiDAR) remote sensing data pertaining to sugarcane. In addition, an integrated regression model integrating random forest regression, support vector regression, K-nearest neighbor regression and deep network regression is developed after feature extraction by DAA to precisely predict biomass of sugarcane. In this study, the biomass prediction performance achieved using the proposed integrated learning approach is found to be predominantly better than that achieved by using conventional linear methods in all the time periods of plant growth. Of more significance, according to model interpretability of DAA, only a small set of informative features maintaining their physical meanings (four informative spectral indices and four key LiDAR metrics) can be extracted which eliminates the redundancy of multi-mode data and plays a vital role in accurate biomass prediction. Therefore, the findings in this study provide hands-on experience to planters with indications of the key or informative spectral or LiDAR metrics relevant to the biomass to adjust the corresponding planting management design. Full article
(This article belongs to the Special Issue Multimodality Fusion in Remote Sensing: Data, Algorithms, and Applications)
►▼ Show Figures

Figure 1

Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:
 
Displaying articles 1-5
Back to TopTop