Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
4.7
2.7
Journals
Entropy
Special Issues
Advances in Image Fusion
entropy-logo
Submit to Entropy Review for Entropy Propose a Special Issue

Journal Menu

► Journal Menu

Journal Browser

► Journal Browser
share announcement

Advances in Image Fusion

A special issue of Entropy (ISSN 1099-4300). This special issue belongs to the section "Multidisciplinary Applications".

Deadline for manuscript submissions: closed (15 December 2022) | Viewed by 71526

Special Issue Editors

Prof. Dr. Jiayi Ma

grade E-Mail Website1 Website2
Guest Editor
Electronic Information School, Wuhan University, Wuhan 430072, China
Interests: machine learning; computer vision; information fusion; image super resolution; hyperspectral image analysis; infrared imaging; image denoising
Special Issues, Collections and Topics in MDPI journals
Dr. Yu Liu

E-Mail Website
Co-Guest Editor
Department of Biomedical Engineering, Hefei University of Technology, Hefei 230009, China
Interests: image fusion; image super-resolution; visual recognition; biomedical image analysis; machine learning; computer vision
Special Issues, Collections and Topics in MDPI journals
Prof. Dr. Junjun Jiang

grade E-Mail Website
Co-Guest Editor
School of Computer Science and Technology, Harbin Institute of Technology, Harbin 150001, China
Interests: image super-resolution; image denoising; video processing; hyperspectral image analysis; image fusion; visual recognition; machine learning
Special Issues, Collections and Topics in MDPI journals
Dr. Zheng Wang

E-Mail Website
Co-Guest Editor
Department of Information and Communication Engineering, The University of Tokyo, Tokyo 113-8656, Japan
Interests: person re-identification; image retrieval; crowd counting; image enhancement; multimedia content analysis
Ms. Han Xu

E-Mail Website
Assistant Guest Editor
Electronic Information School, Wuhan University, Wuhan 430070, China
Interests: computer vision; image fusion; deep learning

Special Issue Information

Dear Colleagues,

Many engineering, medical, remote sensing, environmental, national defense, and civilian applications require multiple types of information. Some examples are multimodality images; images with multiple exposure or focus settings; and images including multispectral, hyperspectral, and panchromatic images, ect. A single type of information can merely represent a part of the scene information, while the combination of multiple types of information can provide a comprehensive characterization. However, the deficiency is that the redundant multiple types of information take up much unnecessary storage space. Thus, the challenge of generating aligned and synthesized results by integrating complementary information has gained significant attention, both from storage space and visual perception viewpoints.

The implementation of information fusion is often hindered by different understandings of information, the definitions of meaningful information for subsequent tasks, the ways of information decomposition, the methods used to distinguish complementary information from redundant information, and the design of fusion rules, ect. Further progress on these issues calls for clearer physical explanations of these methods and definitions. Contributions addressing any of these issues are welcome.

This Special Issue aims to be a forum for new and improved information fusion techniques, which are not restricted to image fusion. Considering that there are inevitable offsets in the information collection process and that source images may sometimes be of low quality, it is likely that fusion techniques involving image registration and image enhancement will gain more popularity. In addition, fusion performance evaluation is also a significant factor in designing a good fusion algorithm. Therefore, studies on image quality assessment techniques, such as image-entropy-based methods, are welcome in this Special Issue.

Prof. Dr. Jiayi Ma
Dr. Yu Liu
Prof. Dr. Junjun Jiang
Dr. Zheng Wang
Ms. Han Xu
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Entropy is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • visible and infrared image fusion
  • medical image fusion
  • multi-exposure image fusion
  • multi-focus image fusion
  • remote sensing image fusion
  • image registration
  • image super-resolution
  • image enhancement and image super-resolution
  • image quality assessment
  • information fusion
  • fusion applications

Published Papers (33 papers)

Download All Papers
Order results
Result details
Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:

Research

16 pages, 3521 KiB  
Article
GRPAFusion: A Gradient Residual and Pyramid Attention-Based Multiscale Network for Multimodal Image Fusion
by Jinxin Wang, Xiaoli Xi, Dongmei Li, Fang Li and Guanxin Zhang
Entropy 2023, 25(1), 169; https://doi.org/10.3390/e25010169 - 14 Jan 2023
Cited by 3 | Viewed by 1458
Abstract
Multimodal image fusion aims to retain valid information from different modalities, remove redundant information to highlight critical targets, and maintain rich texture details in the fused image. However, current image fusion networks only use simple convolutional layers to extract features, ignoring global dependencies [...] Read more.
Multimodal image fusion aims to retain valid information from different modalities, remove redundant information to highlight critical targets, and maintain rich texture details in the fused image. However, current image fusion networks only use simple convolutional layers to extract features, ignoring global dependencies and channel contexts. This paper proposes GRPAFusion, a multimodal image fusion framework based on gradient residual and pyramid attention. The framework uses multiscale gradient residual blocks to extract multiscale structural features and multigranularity detail features from the source image. The depth features from different modalities were adaptively corrected for inter-channel responses using a pyramid split attention module to generate high-quality fused images. Experimental results on public datasets indicated that GRPAFusion outperforms the current fusion methods in subjective and objective evaluations. Full article
(This article belongs to the Special Issue Advances in Image Fusion)
►▼ Show Figures

Figure 1

16 pages, 3594 KiB  
Article
BPDGAN: A GAN-Based Unsupervised Back Project Dense Network for Multi-Modal Medical Image Fusion
by Shangwang Liu and Lihan Yang
Entropy 2022, 24(12), 1823; https://doi.org/10.3390/e24121823 - 14 Dec 2022
Cited by 1 | Viewed by 1413
Abstract
Single-modality medical images often cannot contain sufficient valid information to meet the information requirements of clinical diagnosis. The diagnostic efficiency is always limited by observing multiple images at the same time. Image fusion is a technique that combines functional modalities such as positron [...] Read more.
Single-modality medical images often cannot contain sufficient valid information to meet the information requirements of clinical diagnosis. The diagnostic efficiency is always limited by observing multiple images at the same time. Image fusion is a technique that combines functional modalities such as positron emission computed tomography (PET) and single-photon emission computed tomography (SPECT) with anatomical modalities such as computed tomography (CT) and magnetic resonance imaging (MRI) to supplement the complementary information. Meanwhile, fusing two anatomical images (like CT-MRI) is often required to replace single MRI, and the fused images can improve the efficiency and accuracy of clinical diagnosis. To this end, in order to achieve high-quality, high-resolution and rich-detail fusion without artificial prior, an unsupervised deep learning image fusion framework is proposed in this paper. It is named the back project dense generative adversarial network (BPDGAN) framework. In particular, we construct a novel network based on the back project dense block (BPDB) and convolutional block attention module (CBAM). The BPDB can effectively mitigate the impact of black backgrounds on image content. Conversely, the CBAM improves the performance of BPDGAN on the texture and edge information. To conclude, qualitative and quantitative experiments are tested to demonstrate the superiority of BPDGAN. In terms of quantitative metrics, BPDGAN outperforms the state-of-the-art comparisons by approximately 19.58%, 14.84%, 10.40% and 86.78% on AG, EI, Qabf and Qcv metrics, respectively. Full article
(This article belongs to the Special Issue Advances in Image Fusion)
►▼ Show Figures

Figure 1

20 pages, 4415 KiB  
Article
Water-Air Interface Imaging: Recovering the Images Distorted by Surface Waves via an Efficient Registration Algorithm
by Bijian Jian, Chunbo Ma, Dejian Zhu, Qihong Huang and Jun Ao
Entropy 2022, 24(12), 1765; https://doi.org/10.3390/e24121765 - 02 Dec 2022
Cited by 2 | Viewed by 1485
Abstract
Imaging through the wavy water–air interface is challenging since the random fluctuations of water will cause complex geometric distortion and motion blur in the images, seriously affecting the effective identification of the monitored object. Considering the problems of image recovery accuracy and computational [...] Read more.
Imaging through the wavy water–air interface is challenging since the random fluctuations of water will cause complex geometric distortion and motion blur in the images, seriously affecting the effective identification of the monitored object. Considering the problems of image recovery accuracy and computational efficiency, an efficient reconstruction scheme that combines lucky-patch search and image registration technologies was proposed in this paper. Firstly, a high-quality reference frame is rebuilt using a lucky-patch search strategy. Then an iterative registration algorithm is employed to remove severe geometric distortions by registering warped frames to the reference frame. During the registration process, we integrate JADE and LBFGS algorithms as an optimization strategy to expedite the control parameter optimization process. Finally, the registered frames are refined using PCA and the lucky-patch search algorithm to remove residual distortions and random noise. Experimental results demonstrate that the proposed method significantly outperforms the state-of-the-art methods in terms of sharpness and contrast. Full article
(This article belongs to the Special Issue Advances in Image Fusion)
►▼ Show Figures

Figure 1

20 pages, 8147 KiB  
Article
Infrared and Visible Image Fusion for Highlighting Salient Targets in the Night Scene
by Weida Zhan, Jiale Wang, Yichun Jiang, Yu Chen, Tingyuan Zheng and Yang Hong
Entropy 2022, 24(12), 1759; https://doi.org/10.3390/e24121759 - 30 Nov 2022
Cited by 1 | Viewed by 1262
Abstract
The goal of infrared and visible image fusion in the night scene is to generate a fused image containing salient targets and rich textural details. However, the existing image fusion methods fail to take the unevenness of nighttime luminance into account. To address [...] Read more.
The goal of infrared and visible image fusion in the night scene is to generate a fused image containing salient targets and rich textural details. However, the existing image fusion methods fail to take the unevenness of nighttime luminance into account. To address the above issue, an infrared and visible image fusion method for highlighting salient targets in the night scene is proposed. First of all, a global attention module is designed, which rescales the weights of different channels after capturing global contextual information. Second, the loss function is divided into the foreground loss and the background loss, forcing the fused image to retain rich texture details while highlighting the salient targets. Finally, a luminance estimation function is introduced to obtain the trade-off control parameters of the foreground loss function based on the nighttime luminance. It can effectively highlight salient targets by retaining the foreground information from the source images. Compared with other advanced methods, the experimental results adequately demonstrate the excellent fusion performance and generalization of the proposed method. Full article
(This article belongs to the Special Issue Advances in Image Fusion)
►▼ Show Figures

Figure 1

24 pages, 4408 KiB  
Article
An Improved Human-Body-Segmentation Algorithm with Attention-Based Feature Fusion and a Refined Stereo-Matching Scheme Working at the Sub-Pixel Level for the Anthropometric System
by Lei Yang, Xiaoyu Guo, Xiaowei Song, Deyuan Lu, Wenjing Cai and Zixiang Xiong
Entropy 2022, 24(11), 1647; https://doi.org/10.3390/e24111647 - 13 Nov 2022
Viewed by 1414
Abstract
This paper proposes an improved human-body-segmentation algorithm with attention-based feature fusion and a refined corner-based feature-point design with sub-pixel stereo matching for the anthropometric system. In the human-body-segmentation algorithm, four CBAMs are embedded in the four middle convolution layers of the backbone network [...] Read more.
This paper proposes an improved human-body-segmentation algorithm with attention-based feature fusion and a refined corner-based feature-point design with sub-pixel stereo matching for the anthropometric system. In the human-body-segmentation algorithm, four CBAMs are embedded in the four middle convolution layers of the backbone network (ResNet101) of PSPNet to achieve better feature fusion in space and channels, so as to improve accuracy. The common convolution in the residual blocks of ResNet101 is substituted by group convolution to reduce model parameters and computational cost, thereby optimizing efficiency. For the stereo-matching scheme, a corner-based feature point is designed to obtain the feature-point coordinates at sub-pixel level, so that precision is refined. A regional constraint is applied according to the characteristic of the checkerboard corner points, thereby reducing complexity. Experimental results demonstrated that the anthropometric system with the proposed CBAM-based human-body-segmentation algorithm and corner-based stereo-matching scheme can significantly outperform the state-of-the-art system in accuracy. It can also meet the national standards GB/T 2664-2017, GA 258-2009 and GB/T 2665-2017; and the textile industry standards FZ/T 73029-2019, FZ/T 73017-2014, FZ/T 73059-2017 and FZ/T 73022-2019. Full article
(This article belongs to the Special Issue Advances in Image Fusion)
►▼ Show Figures

Figure 1

18 pages, 4338 KiB  
Article
Infrared and Visible Image Fusion with Significant Target Enhancement
by Xing Huo, Yinping Deng and Kun Shao
Entropy 2022, 24(11), 1633; https://doi.org/10.3390/e24111633 - 10 Nov 2022
Cited by 4 | Viewed by 1799
Abstract
Existing fusion rules focus on retaining detailed information in the source image, but as the thermal radiation information in infrared images is mainly characterized by pixel intensity, these fusion rules are likely to result in reduced saliency of the target in the fused [...] Read more.
Existing fusion rules focus on retaining detailed information in the source image, but as the thermal radiation information in infrared images is mainly characterized by pixel intensity, these fusion rules are likely to result in reduced saliency of the target in the fused image. To address this problem, we propose an infrared and visible image fusion model based on significant target enhancement, aiming to inject thermal targets from infrared images into visible images to enhance target saliency while retaining important details in visible images. First, the source image is decomposed with multi-level Gaussian curvature filtering to obtain background information with high spatial resolution. Second, the large-scale layers are fused using ResNet50 and maximizing weights based on the average operator to improve detail retention. Finally, the base layers are fused by incorporating a new salient target detection method. The subjective and objective experimental results on TNO and MSRS datasets demonstrate that our method achieves better results compared to other traditional and deep learning-based methods. Full article
(This article belongs to the Special Issue Advances in Image Fusion)
►▼ Show Figures

Figure 1

19 pages, 6146 KiB  
Article
Multi-Focus Image Fusion Based on Hessian Matrix Decomposition and Salient Difference Focus Detection
by Xilai Li, Xiaopan Wang, Xiaoqi Cheng, Haishu Tan and Xiaosong Li
Entropy 2022, 24(11), 1527; https://doi.org/10.3390/e24111527 - 25 Oct 2022
Cited by 7 | Viewed by 1279
Abstract
Multi-focus image fusion integrates images from multiple focus regions of the same scene in focus to produce a fully focused image. However, the accurate retention of the focused pixels to the fusion result remains a major challenge. This study proposes a multi-focus image [...] Read more.
Multi-focus image fusion integrates images from multiple focus regions of the same scene in focus to produce a fully focused image. However, the accurate retention of the focused pixels to the fusion result remains a major challenge. This study proposes a multi-focus image fusion algorithm based on Hessian matrix decomposition and salient difference focus detection, which can effectively retain the sharp pixels in the focus region of a source image. First, the source image was decomposed using a Hessian matrix to obtain the feature map containing the structural information. A focus difference analysis scheme based on the improved sum of a modified Laplacian was designed to effectively determine the focusing information at the corresponding positions of the structural feature map and source image. In the process of the decision-map optimization, considering the variability of image size, an adaptive multiscale consistency verification algorithm was designed, which helped the final fused image to effectively retain the focusing information of the source image. Experimental results showed that our method performed better than some state-of-the-art methods in both subjective and quantitative evaluation. Full article
(This article belongs to the Special Issue Advances in Image Fusion)
►▼ Show Figures

Figure 1

20 pages, 9705 KiB  
Article
Tone Image Classification and Weighted Learning for Visible and NIR Image Fusion
by Chan-Gi Im, Dong-Min Son, Hyuk-Ju Kwon and Sung-Hak Lee
Entropy 2022, 24(10), 1435; https://doi.org/10.3390/e24101435 - 09 Oct 2022
Cited by 3 | Viewed by 1926
Abstract
In this paper, to improve the slow processing speed of the rule-based visible and NIR (near-infrared) image synthesis method, we present a fast image fusion method using DenseFuse, one of the CNN (convolutional neural network)-based image synthesis methods. The proposed method applies a [...] Read more.
In this paper, to improve the slow processing speed of the rule-based visible and NIR (near-infrared) image synthesis method, we present a fast image fusion method using DenseFuse, one of the CNN (convolutional neural network)-based image synthesis methods. The proposed method applies a raster scan algorithm to secure visible and NIR datasets for effective learning and presents a dataset classification method using luminance and variance. Additionally, in this paper, a method for synthesizing a feature map in a fusion layer is presented and compared with the method for synthesizing a feature map in other fusion layers. The proposed method learns the superior image quality of the rule-based image synthesis method and shows a clear synthesized image with better visibility than other existing learning-based image synthesis methods. Compared with the rule-based image synthesis method used as the target image, the proposed method has an advantage in processing speed by reducing the processing time to three times or more. Full article
(This article belongs to the Special Issue Advances in Image Fusion)
►▼ Show Figures

Figure 1

16 pages, 5141 KiB  
Article
Fusion of Infrared and Visible Images Based on Three-Scale Decomposition and ResNet Feature Transfer
by Jingyu Ji, Yuhua Zhang, Yongjiang Hu, Yongke Li, Changlong Wang, Zhilong Lin, Fuyu Huang and Jiangyi Yao
Entropy 2022, 24(10), 1356; https://doi.org/10.3390/e24101356 - 24 Sep 2022
Viewed by 1166
Abstract
Image fusion technology can process multiple single image data into more reliable and comprehensive data, which play a key role in accurate target recognition and subsequent image processing. In view of the incomplete image decomposition, redundant extraction of infrared image energy information and [...] Read more.
Image fusion technology can process multiple single image data into more reliable and comprehensive data, which play a key role in accurate target recognition and subsequent image processing. In view of the incomplete image decomposition, redundant extraction of infrared image energy information and incomplete feature extraction of visible images by existing algorithms, a fusion algorithm for infrared and visible image based on three-scale decomposition and ResNet feature transfer is proposed. Compared with the existing image decomposition methods, the three-scale decomposition method is used to finely layer the source image through two decompositions. Then, an optimized WLS method is designed to fuse the energy layer, which fully considers the infrared energy information and visible detail information. In addition, a ResNet-feature transfer method is designed for detail layer fusion, which can extract detailed information such as deeper contour structures. Finally, the structural layers are fused by weighted average strategy. Experimental results show that the proposed algorithm performs well in both visual effects and quantitative evaluation results compared with the five methods. Full article
(This article belongs to the Special Issue Advances in Image Fusion)
►▼ Show Figures

Figure 1

24 pages, 12426 KiB  
Article
Real and Pseudo Pedestrian Detection Method with CA-YOLOv5s Based on Stereo Image Fusion
by Xiaowei Song, Gaoyang Li, Lei Yang, Luxiao Zhu, Chunping Hou and Zixiang Xiong
Entropy 2022, 24(8), 1091; https://doi.org/10.3390/e24081091 - 08 Aug 2022
Cited by 1 | Viewed by 1602
Abstract
With the development of convolutional neural networks, the effect of pedestrian detection has been greatly improved by deep learning models. However, the presence of pseudo pedestrians will lead to accuracy reduction in pedestrian detection. To solve the problem that the existing pedestrian detection [...] Read more.
With the development of convolutional neural networks, the effect of pedestrian detection has been greatly improved by deep learning models. However, the presence of pseudo pedestrians will lead to accuracy reduction in pedestrian detection. To solve the problem that the existing pedestrian detection algorithms cannot distinguish pseudo pedestrians from real pedestrians, a real and pseudo pedestrian detection method with CA-YOLOv5s based on stereo image fusion is proposed in this paper. Firstly, the two-view images of the pedestrian are captured by a binocular stereo camera. Then, a proposed CA-YOLOv5s pedestrian detection algorithm is used for the left-view and right-view images, respectively, to detect the respective pedestrian regions. Afterwards, the detected left-view and right-view pedestrian regions are matched to obtain the feature point set, and the 3D spatial coordinates of the feature point set are calculated with Zhengyou Zhang’s calibration method. Finally, the RANSAC plane-fitting algorithm is adopted to extract the 3D features of the feature point set, and the real and pseudo pedestrian detection is achieved by the trained SVM. The proposed real and pseudo pedestrian detection method with CA-YOLOv5s based on stereo image fusion effectively solves the pseudo pedestrian detection problem and efficiently improves the accuracy. Experimental results also show that for the dataset with real and pseudo pedestrians, the proposed method significantly outperforms other existing pedestrian detection algorithms in terms of accuracy and precision. Full article
(This article belongs to the Special Issue Advances in Image Fusion)
►▼ Show Figures

Figure 1

13 pages, 4084 KiB  
Article
Improving Image Super-Resolution Based on Multiscale Generative Adversarial Networks
by Cao Yuan, Kaidi Deng, Chen Li, Xueting Zhang and Yaqin Li
Entropy 2022, 24(8), 1030; https://doi.org/10.3390/e24081030 - 26 Jul 2022
Cited by 3 | Viewed by 1756
Abstract
Convolutional neural networks have greatly improved the performance of image super-resolution. However, perceptual networks have problems such as blurred line structures and a lack of high-frequency information when reconstructing image textures. To mitigate these issues, a generative adversarial network based on multiscale asynchronous [...] Read more.
Convolutional neural networks have greatly improved the performance of image super-resolution. However, perceptual networks have problems such as blurred line structures and a lack of high-frequency information when reconstructing image textures. To mitigate these issues, a generative adversarial network based on multiscale asynchronous learning is proposed in this paper, whereby a pyramid structure is employed in the network model to integrate high-frequency information at different scales. Our scheme employs a U-net as a discriminator to focus on the consistency of adjacent pixels in the input image and uses the LPIPS loss for perceptual extreme super-resolution with stronger supervision. Experiments on benchmark datasets and independent datasets Set5, Set14, BSD100, and SunHays80 show that our approach is effective in restoring detailed texture information from low-resolution images. Full article
(This article belongs to the Special Issue Advances in Image Fusion)
►▼ Show Figures

Figure 1

14 pages, 5472 KiB  
Article
Multi-Focus Image Fusion Based on Multi-Scale Generative Adversarial Network
by Xiaole Ma, Zhihai Wang, Shaohai Hu and Shichao Kan
Entropy 2022, 24(5), 582; https://doi.org/10.3390/e24050582 - 21 Apr 2022
Cited by 5 | Viewed by 1686
Abstract
The methods based on the convolutional neural network have demonstrated its powerful information integration ability in image fusion. However, most of the existing methods based on neural networks are only applied to a part of the fusion process. In this paper, an end-to-end [...] Read more.
The methods based on the convolutional neural network have demonstrated its powerful information integration ability in image fusion. However, most of the existing methods based on neural networks are only applied to a part of the fusion process. In this paper, an end-to-end multi-focus image fusion method based on a multi-scale generative adversarial network (MsGAN) is proposed that makes full use of image features by a combination of multi-scale decomposition with a convolutional neural network. Extensive qualitative and quantitative experiments on the synthetic and Lytro datasets demonstrated the effectiveness and superiority of the proposed MsGAN compared to the state-of-the-art multi-focus image fusion methods. Full article
(This article belongs to the Special Issue Advances in Image Fusion)
►▼ Show Figures

Figure 1

14 pages, 2682 KiB  
Article
TPFusion: Texture Preserving Fusion of Infrared and Visible Images via Dense Networks
by Zhiguang Yang and Shan Zeng
Entropy 2022, 24(2), 294; https://doi.org/10.3390/e24020294 - 19 Feb 2022
Cited by 6 | Viewed by 1907
Abstract
In this paper, we design an infrared (IR) and visible (VIS) image fusion via unsupervised dense networks, termed as TPFusion. Activity level measurements and fusion rules are indispensable parts of conventional image fusion methods. However, designing an appropriate fusion process is time-consuming and [...] Read more.
In this paper, we design an infrared (IR) and visible (VIS) image fusion via unsupervised dense networks, termed as TPFusion. Activity level measurements and fusion rules are indispensable parts of conventional image fusion methods. However, designing an appropriate fusion process is time-consuming and complicated. In recent years, deep learning-based methods are proposed to handle this problem. However, for multi-modality image fusion, using the same network cannot extract effective feature maps from source images that are obtained by different image sensors. In TPFusion, we can avoid this issue. At first, we extract the textural information of the source images. Then two densely connected networks are trained to fuse textural information and source image, respectively. By this way, we can preserve more textural details in the fused image. Moreover, loss functions we designed to constrain two densely connected convolutional networks are according to the characteristics of textural information and source images. Through our method, the fused image will obtain more textural information of source images. For proving the validity of our method, we implement comparison and ablation experiments from the qualitative and quantitative assessments. The ablation experiments prove the effectiveness of TPFusion. Being compared to existing advanced IR and VIS image fusion methods, our fusion results possess better fusion results in both objective and subjective aspects. To be specific, in qualitative comparisons, our fusion results have better contrast ratio and abundant textural details. In quantitative comparisons, TPFusion outperforms existing representative fusion methods. Full article
(This article belongs to the Special Issue Advances in Image Fusion)
►▼ Show Figures

Figure 1

17 pages, 33679 KiB  
Article
Multiscale Geometric Analysis Fusion-Based Unsupervised Change Detection in Remote Sensing Images via FLICM Model
by Liangliang Li, Hongbing Ma and Zhenhong Jia
Entropy 2022, 24(2), 291; https://doi.org/10.3390/e24020291 - 18 Feb 2022
Cited by 8 | Viewed by 1905
Abstract
Remote sensing image change detection is widely used in land use and natural disaster detection. In order to improve the accuracy of change detection, a robust change detection method based on nonsubsampled contourlet transform (NSCT) fusion and fuzzy local information C-means clustering (FLICM) [...] Read more.
Remote sensing image change detection is widely used in land use and natural disaster detection. In order to improve the accuracy of change detection, a robust change detection method based on nonsubsampled contourlet transform (NSCT) fusion and fuzzy local information C-means clustering (FLICM) model is introduced in this paper. Firstly, the log-ratio and mean-ratio operators are used to generate the difference image (DI), respectively; then, the NSCT fusion model is utilized to fuse the two difference images, and one new DI is obtained. The fused DI can not only reflect the real change trend but also suppress the background. The FLICM is performed on the new DI to obtain the final change detection map. Four groups of homogeneous remote sensing images are selected for simulation experiments, and the experimental results demonstrate that the proposed homogeneous change detection method has a superior performance than other state-of-the-art algorithms. Full article
(This article belongs to the Special Issue Advances in Image Fusion)
►▼ Show Figures

Figure 1

20 pages, 79259 KiB  
Article
Research on Color Image Encryption Algorithm Based on Bit-Plane and Chen Chaotic System
by Jiangjian Xu, Bing Zhao and Zeming Wu
Entropy 2022, 24(2), 186; https://doi.org/10.3390/e24020186 - 26 Jan 2022
Cited by 18 | Viewed by 2599
Abstract
In response to the problems of high complexity and the large amount of operations of existing color image encryption algorithms, a low-complexity, low-operation color image encryption algorithm based on a combination of bit-plane and chaotic systems is proposed that is interrelated with plaintext [...] Read more.
In response to the problems of high complexity and the large amount of operations of existing color image encryption algorithms, a low-complexity, low-operation color image encryption algorithm based on a combination of bit-plane and chaotic systems is proposed that is interrelated with plaintext information. Firstly, three channels of an RGB image are extracted, and the gray value of each pixel channel can be expressed by an eight-bit binary number. The higher- and lower-four bits of the binary gray value of each pixel are exchanged, and the position of each four-bit binary number is scrambled by a logistic chaotic sequence, and all the four-bit binary numbers are converted into hexadecimal numbers to reduce the computational complexity. Next, the position of the transformed image is scrambled by a logistic chaotic sequence. Then, the Chen chaos sequence is used to permute the gray pixel values of the permuted image. Finally, the gray value of the encrypted image is converted into a decimal number to form a single-channel encrypted image, and the three-channel encrypted image is synthesized into an encrypted color image. Through MATLAB simulation experiments, a security analysis of encryption effects in terms of a histogram, correlation, a differential attack, and information entropy is performed. The results show that the algorithm has a better encryption effect and is resistant to differential attacks. Full article
(This article belongs to the Special Issue Advances in Image Fusion)
►▼ Show Figures

Figure 1

20 pages, 24147 KiB  
Article
Synthetic Aperture Radar Image Despeckling Based on Multi-Weighted Sparse Coding
by Shujun Liu, Ningjie Pu, Jianxin Cao and Kui Zhang
Entropy 2022, 24(1), 96; https://doi.org/10.3390/e24010096 - 07 Jan 2022
Cited by 5 | Viewed by 1817
Abstract
Synthetic aperture radar (SAR) images are inherently degraded by speckle noise caused by coherent imaging, which may affect the performance of the subsequent image analysis task. To resolve this problem, this article proposes an integrated SAR image despeckling model based on dictionary learning [...] Read more.
Synthetic aperture radar (SAR) images are inherently degraded by speckle noise caused by coherent imaging, which may affect the performance of the subsequent image analysis task. To resolve this problem, this article proposes an integrated SAR image despeckling model based on dictionary learning and multi-weighted sparse coding. First, the dictionary is trained by groups composed of similar image patches, which have the same structural features. An effective orthogonal dictionary with high sparse representation ability is realized by introducing a properly tight frame. Furthermore, the data-fidelity term and regularization terms are constrained by weighting factors. The weighted sparse representation model not only fully utilizes the interblock relevance but also reflects the importance of various structural groups in despeckling processing. The proposed model is implemented with fast and effective solving steps that simultaneously perform orthogonal dictionary learning, weight parameter updating, sparse coding, and image reconstruction. The solving steps are designed using the alternative minimization method. Finally, the speckles are further suppressed by iterative regularization methods. In a comparison study with existing methods, our method demonstrated state-of-the-art performance in suppressing speckle noise and protecting the image texture details. Full article
(This article belongs to the Special Issue Advances in Image Fusion)
►▼ Show Figures

Figure 1

17 pages, 18231 KiB  
Article
S2A: Scale-Attention-Aware Networks for Video Super-Resolution
by Taian Guo, Tao Dai, Ling Liu, Zexuan Zhu and Shu-Tao Xia
Entropy 2021, 23(11), 1398; https://doi.org/10.3390/e23111398 - 25 Oct 2021
Cited by 4 | Viewed by 1680
Abstract
Convolutional Neural Networks (CNNs) have been widely used in video super-resolution (VSR). Most existing VSR methods focus on how to utilize the information of multiple frames, while neglecting the feature correlations of the intermediate features, thus limiting the feature expression of the models. [...] Read more.
Convolutional Neural Networks (CNNs) have been widely used in video super-resolution (VSR). Most existing VSR methods focus on how to utilize the information of multiple frames, while neglecting the feature correlations of the intermediate features, thus limiting the feature expression of the models. To address this problem, we propose a novel SAA network, that is, Scale-and-Attention-Aware Networks, to apply different attention to different temporal-length streams, while further exploring both spatial and channel attention on separate streams with a newly proposed Criss-Cross Channel Attention Module (C3AM). Experiments on public VSR datasets demonstrate the superiority of our method over other state-of-the-art methods in terms of both quantitative and qualitative metrics. Full article
(This article belongs to the Special Issue Advances in Image Fusion)
►▼ Show Figures

Figure 1

28 pages, 12843 KiB  
Article
Multi-Focus Image Fusion Method Based on Multi-Scale Decomposition of Information Complementary
by Hui Wan, Xianlun Tang, Zhiqin Zhu and Weisheng Li
Entropy 2021, 23(10), 1362; https://doi.org/10.3390/e23101362 - 19 Oct 2021
Cited by 2 | Viewed by 2024
Abstract
Multi-focus image fusion is an important method used to combine the focused parts from source multi-focus images into a single full-focus image. Currently, to address the problem of multi-focus image fusion, the key is on how to accurately detect the focus regions, especially [...] Read more.
Multi-focus image fusion is an important method used to combine the focused parts from source multi-focus images into a single full-focus image. Currently, to address the problem of multi-focus image fusion, the key is on how to accurately detect the focus regions, especially when the source images captured by cameras produce anisotropic blur and unregistration. This paper proposes a new multi-focus image fusion method based on the multi-scale decomposition of complementary information. Firstly, this method uses two groups of large-scale and small-scale decomposition schemes that are structurally complementary, to perform two-scale double-layer singular value decomposition of the image separately and obtain low-frequency and high-frequency components. Then, the low-frequency components are fused by a rule that integrates image local energy with edge energy. The high-frequency components are fused by the parameter-adaptive pulse-coupled neural network model (PA-PCNN), and according to the feature information contained in each decomposition layer of the high-frequency components, different detailed features are selected as the external stimulus input of the PA-PCNN. Finally, according to the two-scale decomposition of the source image that is structure complementary, and the fusion of high and low frequency components, two initial decision maps with complementary information are obtained. By refining the initial decision graph, the final fusion decision map is obtained to complete the image fusion. In addition, the proposed method is compared with 10 state-of-the-art approaches to verify its effectiveness. The experimental results show that the proposed method can more accurately distinguish the focused and non-focused areas in the case of image pre-registration and unregistration, and the subjective and objective evaluation indicators are slightly better than those of the existing methods. Full article
(This article belongs to the Special Issue Advances in Image Fusion)
►▼ Show Figures

Figure 1

18 pages, 3206 KiB  
Article
A New Variational Bayesian-Based Kalman Filter with Unknown Time-Varying Measurement Loss Probability and Non-Stationary Heavy-Tailed Measurement Noise
by Chenghao Shan, Weidong Zhou, Yefeng Yang and Hanyu Shan
Entropy 2021, 23(10), 1351; https://doi.org/10.3390/e23101351 - 16 Oct 2021
Cited by 3 | Viewed by 1733
Abstract
In this paper, a new variational Bayesian-based Kalman filter (KF) is presented to solve the filtering problem for a linear system with unknown time-varying measurement loss probability (UTVMLP) and non-stationary heavy-tailed measurement noise (NSHTMN). Firstly, the NSHTMN was modelled as a Gaussian-Student’s t [...] Read more.
In this paper, a new variational Bayesian-based Kalman filter (KF) is presented to solve the filtering problem for a linear system with unknown time-varying measurement loss probability (UTVMLP) and non-stationary heavy-tailed measurement noise (NSHTMN). Firstly, the NSHTMN was modelled as a Gaussian-Student’s t-mixture distribution via employing a Bernoulli random variable (BM). Secondly, by utilizing another Bernoulli random variable (BL), the form of the likelihood function consisting of two mixture distributions was converted from a weight sum to an exponential product and a new hierarchical Gaussian state-space model was therefore established. Finally, the system state vector, BM, BL, the intermediate random variables, the mixing probability, and the UTVMLP were jointly inferred by employing the variational Bayesian technique. Simulation results revealed that in the scenario of NSHTMN, the proposed filter had a better performance than current algorithms and further improved the estimation accuracy of UTVMLP. Full article
(This article belongs to the Special Issue Advances in Image Fusion)
►▼ Show Figures

Figure 1

11 pages, 692 KiB  
Article
A Multi-Modal Fusion Method Based on Higher-Order Orthogonal Iteration Decomposition
by Fen Liu , Jianfeng Chen , Weijie Tan  and Chang Cai 
Entropy 2021, 23(10), 1349; https://doi.org/10.3390/e23101349 - 15 Oct 2021
Cited by 6 | Viewed by 1899
Abstract
Multi-modal fusion can achieve better predictions through the amalgamation of information from different modalities. To improve the performance of accuracy, a method based on Higher-order Orthogonal Iteration Decomposition and Projection (HOIDP) is proposed, in the fusion process, higher-order orthogonal iteration decomposition algorithm and [...] Read more.
Multi-modal fusion can achieve better predictions through the amalgamation of information from different modalities. To improve the performance of accuracy, a method based on Higher-order Orthogonal Iteration Decomposition and Projection (HOIDP) is proposed, in the fusion process, higher-order orthogonal iteration decomposition algorithm and factor matrix projection are used to remove redundant information duplicated inter-modal and produce fewer parameters with minimal information loss. The performance of the proposed method is verified by three different multi-modal datasets. The numerical results validate the accuracy of the performance of the proposed method having 0.4% to 4% improvement in sentiment analysis, 0.3% to 8% improvement in personality trait recognition, and 0.2% to 25% improvement in emotion recognition at three different multi-modal datasets compared with other 5 methods. Full article
(This article belongs to the Special Issue Advances in Image Fusion)
►▼ Show Figures

Figure 1

15 pages, 8411 KiB  
Article
Learning Two-View Correspondences and Geometry via Local Neighborhood Correlation
by Luanyuan Dai, Xin Liu, Jingtao Wang, Changcai Yang and Riqing Chen
Entropy 2021, 23(8), 1024; https://doi.org/10.3390/e23081024 - 09 Aug 2021
Cited by 2 | Viewed by 1778
Abstract
Seeking quality feature correspondences (also known as matches) is a foundational step in computer vision. In our work, a novel and effective network with a stable local constraint, named the Local Neighborhood Correlation Network (LNCNet), is proposed to capture abundant contextual information of [...] Read more.
Seeking quality feature correspondences (also known as matches) is a foundational step in computer vision. In our work, a novel and effective network with a stable local constraint, named the Local Neighborhood Correlation Network (LNCNet), is proposed to capture abundant contextual information of each correspondence in the local region, followed by calculating the essential matrix and camera pose estimation. Firstly, the k-Nearest Neighbor (KNN) algorithm is used to divide the local neighborhood roughly. Then, we calculate the local neighborhood correlation matrix (LNC) between the selected correspondence and other correspondences in the local region, which is used to filter outliers to obtain more accurate local neighborhood information. We cluster the filtered information into feature vectors containing richer neighborhood contextual information so that they can be used to more accurately determine the probability of correspondences as inliers. Extensive experiments have demonstrated that our proposed LNCNet performs better than some state-of-the-art networks to accomplish outlier rejection and camera pose estimation tasks in complex outdoor and indoor scenes. Full article
(This article belongs to the Special Issue Advances in Image Fusion)
►▼ Show Figures

Figure 1

14 pages, 4741 KiB  
Article
News Video Summarization Combining SURF and Color Histogram Features
by Buyun Liang, Na Li, Zheng He, Zhongyuan Wang, Youming Fu and Tao Lu
Entropy 2021, 23(8), 982; https://doi.org/10.3390/e23080982 - 30 Jul 2021
Cited by 6 | Viewed by 2266
Abstract
Because the data volume of news videos is increasing exponentially, a way to quickly browse a sketch of the video is important in various applications, such as news media, archives and publicity. This paper proposes a news video summarization method based on SURF [...] Read more.
Because the data volume of news videos is increasing exponentially, a way to quickly browse a sketch of the video is important in various applications, such as news media, archives and publicity. This paper proposes a news video summarization method based on SURF features and an improved clustering algorithm, to overcome the defects in existing algorithms that fail to account for changes in shot complexity. Firstly, we extracted SURF features from the video sequences and matched the features between adjacent frames, and then detected the abrupt and gradual boundaries of the shot by calculating similarity scores between adjacent frames with the help of double thresholds. Secondly, we used an improved clustering algorithm to cluster the color histogram of the video frames within the shot, which merged the smaller clusters and then selected the frame closest to the cluster center as the key frame. The experimental results on both the public and self-built datasets show the superiority of our method over the alternatives in terms of accuracy and speed. Additionally, the extracted key frames demonstrate low redundancy and can credibly represent a sketch of news videos. Full article
(This article belongs to the Special Issue Advances in Image Fusion)
►▼ Show Figures

Figure 1

14 pages, 2849 KiB  
Article
Pairwise Elastic Net Representation-Based Classification for Hyperspectral Image Classification
by Hao Li, Yuanshu Zhang, Yong Ma, Xiaoguang Mei, Shan Zeng and Yaqin Li
Entropy 2021, 23(8), 956; https://doi.org/10.3390/e23080956 - 26 Jul 2021
Cited by 2 | Viewed by 1643
Abstract
The representation-based algorithm has raised a great interest in hyperspectral image (HSI) classification. l1-minimization-based sparse representation (SR) attempts to select a few atoms and cannot fully reflect within-class information, while l2-minimization-based collaborative representation (CR) tries to use all of [...] Read more.
The representation-based algorithm has raised a great interest in hyperspectral image (HSI) classification. l1-minimization-based sparse representation (SR) attempts to select a few atoms and cannot fully reflect within-class information, while l2-minimization-based collaborative representation (CR) tries to use all of the atoms leading to mixed-class information. Considering the above problems, we propose the pairwise elastic net representation-based classification (PENRC) method. PENRC combines the l1-norm and l2-norm penalties and introduces a new penalty term, including a similar matrix between dictionary atoms. This similar matrix enables the automatic grouping selection of highly correlated data to estimate more robust weight coefficients for better classification performance. To reduce computation cost and further improve classification accuracy, we use part of the atoms as a local adaptive dictionary rather than the entire training atoms. Furthermore, we consider the neighbor information of each pixel and propose a joint pairwise elastic net representation-based classification (J-PENRC) method. Experimental results on chosen hyperspectral data sets confirm that our proposed algorithms outperform the other state-of-the-art algorithms. Full article
(This article belongs to the Special Issue Advances in Image Fusion)
►▼ Show Figures

Figure 1

23 pages, 5727 KiB  
Article
Robust Vehicle Speed Measurement Based on Feature Information Fusion for Vehicle Multi-Characteristic Detection
by Lei Yang, Jianchen Luo, Xiaowei Song, Menglong Li, Pengwei Wen and Zixiang Xiong
Entropy 2021, 23(7), 910; https://doi.org/10.3390/e23070910 - 17 Jul 2021
Cited by 10 | Viewed by 3428
Abstract
A robust vehicle speed measurement system based on feature information fusion for vehicle multi-characteristic detection is proposed in this paper. A vehicle multi-characteristic dataset is constructed. With this dataset, seven CNN-based modern object detection algorithms are trained for vehicle multi-characteristic detection. The FPN-based [...] Read more.
A robust vehicle speed measurement system based on feature information fusion for vehicle multi-characteristic detection is proposed in this paper. A vehicle multi-characteristic dataset is constructed. With this dataset, seven CNN-based modern object detection algorithms are trained for vehicle multi-characteristic detection. The FPN-based YOLOv4 is selected as the best vehicle multi-characteristic detection algorithm, which applies feature information fusion of different scales with both rich high-level semantic information and detailed low-level location information. The YOLOv4 algorithm is improved by combing with the attention mechanism, in which the residual module in YOLOv4 is replaced by the ECA channel attention module with cross channel interaction. An improved ECA-YOLOv4 object detection algorithm based on both feature information fusion and cross channel interaction is proposed, which improves the performance of YOLOv4 for vehicle multi-characteristic detection and reduces the model parameter size and FLOPs as well. A multi-characteristic fused speed measurement system based on license plate, logo, and light is designed accordingly. The system performance is verified by experiments. The experimental results show that the speed measurement error rate of the proposed system meets the requirement of the China national standard GB/T 21555-2007 in which the speed measurement error rate should be less than 6%. The proposed system can efficiently enhance the vehicle speed measurement accuracy and effectively improve the vehicle speed measurement robustness. Full article
(This article belongs to the Special Issue Advances in Image Fusion)
►▼ Show Figures

Figure 1

21 pages, 4542 KiB  
Article
An Improved Stereo Matching Algorithm for Vehicle Speed Measurement System Based on Spatial and Temporal Image Fusion
by Lei Yang, Qingyuan Li, Xiaowei Song, Wenjing Cai, Chunping Hou and Zixiang Xiong
Entropy 2021, 23(7), 866; https://doi.org/10.3390/e23070866 - 07 Jul 2021
Cited by 2 | Viewed by 1896
Abstract
This paper proposes an improved stereo matching algorithm for vehicle speed measurement system based on spatial and temporal image fusion (STIF). Firstly, the matching point pairs in the license plate area with obviously abnormal distance to the camera are roughly removed according to [...] Read more.
This paper proposes an improved stereo matching algorithm for vehicle speed measurement system based on spatial and temporal image fusion (STIF). Firstly, the matching point pairs in the license plate area with obviously abnormal distance to the camera are roughly removed according to the characteristic of license plate specification. Secondly, more mismatching point pairs are finely removed according to local neighborhood consistency constraint (LNCC). Thirdly, the optimum speed measurement point pairs are selected for successive stereo frame pairs by STIF of binocular stereo video, so that the 3D points corresponding to the matching point pairs for speed measurement in the successive stereo frame pairs are in the same position on the real vehicle, which can significantly improve the vehicle speed measurement accuracy. LNCC and STIF can be used not only for license plate, but also for vehicle logo, light, mirror etc. Experimental results demonstrate that the vehicle speed measurement system with the proposed LNCC+STIF stereo matching algorithm can significantly outperform the state-of-the-art system in accuracy. Full article
(This article belongs to the Special Issue Advances in Image Fusion)
►▼ Show Figures

Figure 1

16 pages, 45388 KiB  
Article
A Foreground-Aware Framework for Local Face Attribute Transfer
by Yuanbin Fu, Jiayi Ma and Xiaojie Guo
Entropy 2021, 23(5), 615; https://doi.org/10.3390/e23050615 - 16 May 2021
Cited by 1 | Viewed by 2228
Abstract
In the context of social media, large amounts of headshot photos are taken everyday. Unfortunately, in addition to laborious editing and modification, creating a visually compelling photographic masterpiece for sharing requires advanced professional skills, which are difficult for ordinary Internet users. Though there [...] Read more.
In the context of social media, large amounts of headshot photos are taken everyday. Unfortunately, in addition to laborious editing and modification, creating a visually compelling photographic masterpiece for sharing requires advanced professional skills, which are difficult for ordinary Internet users. Though there are many algorithms automatically and globally transferring the style from one image to another, they fail to respect the semantics of the scene and are unable to allow users to merely transfer the attributes of one or two face organs in the foreground region leaving the background region unchanged. To overcome this problem, we developed a novel framework for semantically meaningful local face attribute transfer, which can flexibly transfer the local attribute of a face organ from the reference image to a semantically equivalent organ in the input image, while preserving the background. Our method involves warping the reference photo to match the shape, pose, location, and expression of the input image. The fusion of the warped reference image and input image is then taken as the initialized image for a neural style transfer algorithm. Our method achieves better performance in terms of inception score (3.81) and Fréchet inception distance (80.31), which is about 10% higher than those of competitors, indicating that our framework is capable of producing high-quality and photorealistic attribute transfer results. Both theoretical findings and experimental results are provided to demonstrate the efficacy of the proposed framework, reveal its superiority over other state-of-the-art alternatives. Full article
(This article belongs to the Special Issue Advances in Image Fusion)
►▼ Show Figures

Figure 1

21 pages, 14125 KiB  
Article
Pulse Coupled Neural Network-Based Multimodal Medical Image Fusion via Guided Filtering and WSEML in NSCT Domain
by Liangliang Li and Hongbing Ma
Entropy 2021, 23(5), 591; https://doi.org/10.3390/e23050591 - 11 May 2021
Cited by 22 | Viewed by 2439
Abstract
Multimodal medical image fusion aims to fuse images with complementary multisource information. In this paper, we propose a novel multimodal medical image fusion method using pulse coupled neural network (PCNN) and a weighted sum of eight-neighborhood-based modified Laplacian (WSEML) integrating guided image filtering [...] Read more.
Multimodal medical image fusion aims to fuse images with complementary multisource information. In this paper, we propose a novel multimodal medical image fusion method using pulse coupled neural network (PCNN) and a weighted sum of eight-neighborhood-based modified Laplacian (WSEML) integrating guided image filtering (GIF) in non-subsampled contourlet transform (NSCT) domain. Firstly, the source images are decomposed by NSCT, several low- and high-frequency sub-bands are generated. Secondly, the PCNN-based fusion rule is used to process the low-frequency components, and the GIF-WSEML fusion model is used to process the high-frequency components. Finally, the fused image is obtained by integrating the fused low- and high-frequency sub-bands. The experimental results demonstrate that the proposed method can achieve better performance in terms of multimodal medical image fusion. The proposed algorithm also has obvious advantages in objective evaluation indexes VIFF, QW, API, SD, EN and time consumption. Full article
(This article belongs to the Special Issue Advances in Image Fusion)
►▼ Show Figures

Figure 1

19 pages, 21228 KiB  
Article
Unsupervised Exemplar-Domain Aware Image-to-Image Translation
by Yuanbin Fu, Jiayi Ma and Xiaojie Guo
Entropy 2021, 23(5), 565; https://doi.org/10.3390/e23050565 - 02 May 2021
Cited by 3 | Viewed by 2100
Abstract
Image-to-image translation is used to convert an image of a certain style to another of the target style with the original content preserved. A desired translator should be capable of generating diverse results in a controllable many-to-many fashion. To this end, we design [...] Read more.
Image-to-image translation is used to convert an image of a certain style to another of the target style with the original content preserved. A desired translator should be capable of generating diverse results in a controllable many-to-many fashion. To this end, we design a novel deep translator, namely exemplar-domain aware image-to-image translator (EDIT for short). From a logical perspective, the translator needs to perform two main functions, i.e., feature extraction and style transfer. With consideration of logical network partition, the generator of our EDIT comprises of a part of blocks configured by shared parameters, and the rest by varied parameters exported by an exemplar-domain aware parameter network, for explicitly imitating the functionalities of extraction and mapping. The principle behind this is that, for images from multiple domains, the content features can be obtained by an extractor, while (re-)stylization is achieved by mapping the extracted features specifically to different purposes (domains and exemplars). In addition, a discriminator is equipped during the training phase to guarantee the output satisfying the distribution of the target domain. Our EDIT can flexibly and effectively work on multiple domains and arbitrary exemplars in a unified neat model. We conduct experiments to show the efficacy of our design, and reveal its advances over other state-of-the-art methods both quantitatively and qualitatively. Full article
(This article belongs to the Special Issue Advances in Image Fusion)
►▼ Show Figures

Figure 1

17 pages, 10771 KiB  
Article
A Noisy SAR Image Fusion Method Based on NLM and GAN
by Jing Fang, Xiaole Ma, Jingjing Wang, Kai Qin, Shaohai Hu and Yuefeng Zhao
Entropy 2021, 23(4), 410; https://doi.org/10.3390/e23040410 - 30 Mar 2021
Cited by 1 | Viewed by 1993
Abstract
The unavoidable noise often present in synthetic aperture radar (SAR) images, such as speckle noise, negatively impacts the subsequent processing of SAR images. Further, it is not easy to find an appropriate application for SAR images, given that the human visual system is [...] Read more.
The unavoidable noise often present in synthetic aperture radar (SAR) images, such as speckle noise, negatively impacts the subsequent processing of SAR images. Further, it is not easy to find an appropriate application for SAR images, given that the human visual system is sensitive to color and SAR images are gray. As a result, a noisy SAR image fusion method based on nonlocal matching and generative adversarial networks is presented in this paper. A nonlocal matching method is applied to processing source images into similar block groups in the pre-processing step. Then, adversarial networks are employed to generate a final noise-free fused SAR image block, where the generator aims to generate a noise-free SAR image block with color information, and the discriminator tries to increase the spatial resolution of the generated image block. This step ensures that the fused image block contains high resolution and color information at the same time. Finally, a fused image can be obtained by aggregating all the image blocks. By extensive comparative experiments on the SEN1–2 datasets and source images, it can be found that the proposed method not only has better fusion results but is also robust to image noise, indicating the superiority of the proposed noisy SAR image fusion method over the state-of-the-art methods. Full article
(This article belongs to the Special Issue Advances in Image Fusion)
►▼ Show Figures

Figure 1

18 pages, 1247 KiB  
Article
A Generative Adversarial Network for Infrared and Visible Image Fusion Based on Semantic Segmentation
by Jilei Hou, Dazhi Zhang, Wei Wu, Jiayi Ma and Huabing Zhou
Entropy 2021, 23(3), 376; https://doi.org/10.3390/e23030376 - 21 Mar 2021
Cited by 40 | Viewed by 4399
Abstract
This paper proposes a new generative adversarial network for infrared and visible image fusion based on semantic segmentation (SSGAN), which can consider not only the low-level features of infrared and visible images, but also the high-level semantic information. Source images can be divided [...] Read more.
This paper proposes a new generative adversarial network for infrared and visible image fusion based on semantic segmentation (SSGAN), which can consider not only the low-level features of infrared and visible images, but also the high-level semantic information. Source images can be divided into foregrounds and backgrounds by semantic masks. The generator with a dual-encoder-single-decoder framework is used to extract the feature of foregrounds and backgrounds by different encoder paths. Moreover, the discriminator’s input image is designed based on semantic segmentation, which is obtained by combining the foregrounds of the infrared images with the backgrounds of the visible images. Consequently, the prominence of thermal targets in the infrared images and texture details in the visible images can be preserved in the fused images simultaneously. Qualitative and quantitative experiments on publicly available datasets demonstrate that the proposed approach can significantly outperform the state-of-the-art methods. Full article
(This article belongs to the Special Issue Advances in Image Fusion)
►▼ Show Figures

Figure 1

15 pages, 70075 KiB  
Article
An Improvised Machine Learning Model Based on Mutual Information Feature Selection Approach for Microbes Classification
by Anaahat Dhindsa, Sanjay Bhatia, Sunil Agrawal and Balwinder Singh Sohi
Entropy 2021, 23(2), 257; https://doi.org/10.3390/e23020257 - 23 Feb 2021
Cited by 13 | Viewed by 3882
Abstract
The accurate classification of microbes is critical in today’s context for monitoring the ecological balance of a habitat. Hence, in this research work, a novel method to automate the process of identifying microorganisms has been implemented. To extract the bodies of microorganisms accurately, [...] Read more.
The accurate classification of microbes is critical in today’s context for monitoring the ecological balance of a habitat. Hence, in this research work, a novel method to automate the process of identifying microorganisms has been implemented. To extract the bodies of microorganisms accurately, a generalized segmentation mechanism which consists of a combination of convolution filter (Kirsch) and a variance-based pixel clustering algorithm (Otsu) is proposed. With exhaustive corroboration, a set of twenty-five features were identified to map the characteristics and morphology for all kinds of microbes. Multiple techniques for feature selection were tested and it was found that mutual information (MI)-based models gave the best performance. Exhaustive hyperparameter tuning of multilayer layer perceptron (MLP), k-nearest neighbors (KNN), quadratic discriminant analysis (QDA), logistic regression (LR), and support vector machine (SVM) was done. It was found that SVM radial required further improvisation to attain a maximum possible level of accuracy. Comparative analysis between SVM and improvised SVM (ISVM) through a 10-fold cross validation method ultimately showed that ISVM resulted in a 2% higher performance in terms of accuracy (98.2%), precision (98.2%), recall (98.1%), and F1 score (98.1%). Full article
(This article belongs to the Special Issue Advances in Image Fusion)
►▼ Show Figures

Figure 1

22 pages, 5797 KiB  
Article
Exploiting Superpixels for Multi-Focus Image Fusion
by Areeba Ilyas, Muhammad Shahid Farid, Muhammad Hassan Khan and Marcin Grzegorzek
Entropy 2021, 23(2), 247; https://doi.org/10.3390/e23020247 - 21 Feb 2021
Cited by 6 | Viewed by 3153
Abstract
Multi-focus image fusion is the process of combining focused regions of two or more images to obtain a single all-in-focus image. It is an important research area because a fused image is of high quality and contains more details than the source images. [...] Read more.
Multi-focus image fusion is the process of combining focused regions of two or more images to obtain a single all-in-focus image. It is an important research area because a fused image is of high quality and contains more details than the source images. This makes it useful for numerous applications in image enhancement, remote sensing, object recognition, medical imaging, etc. This paper presents a novel multi-focus image fusion algorithm that proposes to group the local connected pixels with similar colors and patterns, usually referred to as superpixels, and use them to separate the focused and de-focused regions of an image. We note that these superpixels are more expressive than individual pixels, and they carry more distinctive statistical properties when compared with other superpixels. The statistical properties of superpixels are analyzed to categorize the pixels as focused or de-focused and to estimate a focus map. A spatial consistency constraint is ensured on the initial focus map to obtain a refined map, which is used in the fusion rule to obtain a single all-in-focus image. Qualitative and quantitative evaluations are performed to assess the performance of the proposed method on a benchmark multi-focus image fusion dataset. The results show that our method produces better quality fused images than existing image fusion techniques. Full article
(This article belongs to the Special Issue Advances in Image Fusion)
►▼ Show Figures

Figure 1

12 pages, 10229 KiB  
Article
Advanced Driving Assistance Based on the Fusion of Infrared and Visible Images
by Yansong Gu, Xinya Wang, Can Zhang and Baiyang Li
Entropy 2021, 23(2), 239; https://doi.org/10.3390/e23020239 - 19 Feb 2021
Cited by 5 | Viewed by 2294
Abstract
Obtaining key and rich visual information under sophisticated road conditions is one of the key requirements for advanced driving assistance. In this paper, a newfangled end-to-end model is proposed for advanced driving assistance based on the fusion of infrared and visible images, termed [...] Read more.
Obtaining key and rich visual information under sophisticated road conditions is one of the key requirements for advanced driving assistance. In this paper, a newfangled end-to-end model is proposed for advanced driving assistance based on the fusion of infrared and visible images, termed as FusionADA. In our model, we are committed to extracting and fusing the optimal texture details and salient thermal targets from the source images. To achieve this goal, our model constitutes an adversarial framework between the generator and the discriminator. Specifically, the generator aims to generate a fused image with basic intensity information together with the optimal texture details from source images, while the discriminator aims to force the fused image to restore the salient thermal targets from the source infrared image. In addition, our FusionADA is a fully end-to-end model, solving the issues of manually designing complicated activity level measurements and fusion rules existing in traditional methods. Qualitative and quantitative experiments on publicly available datasets RoadScene and TNO demonstrate the superiority of our FusionADA over the state-of-the-art approaches. Full article
(This article belongs to the Special Issue Advances in Image Fusion)
►▼ Show Figures

Figure 1

Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:
 
Displaying articles 1-33
Back to TopTop