Data Processing with Artificial Intelligence in Thermal Imagery

A special issue of Journal of Imaging (ISSN 2313-433X). This special issue belongs to the section "AI in Imaging".

Deadline for manuscript submissions: 31 July 2024 | Viewed by 4656

Special Issue Editors


E-Mail Website
Guest Editor
SiMa Technologies Inc. 226 Airport Parkway, Suite 550 San Jose, CA 95110, USA
Interests: computer vision; artificial intelligence; biomedical engineering; remote healthcare; super resolution; convolutional neural networks; machine learning; edge processing
Special Issues, Collections and Topics in MDPI journals

E-Mail Website
Guest Editor
Construction Research Centre, National Research Council Canada, Ottawa, ON K1A 0R6, Canada
Interests: computer vision; image processing; artificial intelligence; deep learning; medical imaging; thermal imaging; spectroscopy; virtual reality; data analytics and risk assessment; electronics/embedded systems
Special Issues, Collections and Topics in MDPI journals

grade E-Mail Website
Guest Editor
Informatics Building School of Informatics, University of Leicester, Leicester LE1 7RH, UK
Interests: deep learning; artificial intelligence; machine learning
Special Issues, Collections and Topics in MDPI journals

Special Issue Information

Dear Colleagues,

Thermal imaging possesses various advantages over the visible light spectrum, allowing us to not only address challenging lighting conditions (e.g., poor lighting [1]), but also reveal information invisible to the naked eye [2]. For this reason, this imaging domain is continuously gaining more popularity across a broad variety of markets, e.g., in the automotive industry for scene understanding [3] and driver monitoring [4]; in the medical field for evaluation of skin conditions [5] or vital sign extraction [6]; and for smart vision in surveillance [7] and border control [8] applications, just to name a few.

At the same time, it is important to note that thermal imagery has different characteristics than visible light data [9]. First, due to the heat flow in objects, thermal images are more blurred with smooth borders between objects and there is an absence of high-frequency components such as edges and textures [10]; frequently, the lack of color data also makes image processing more challenging [11]. Secondly, ranges of thermal sensors are usually shorter than in the case of standard cameras, allowing them to capture only close-proximity scenes. Finally, the resolution of such data is usually lower due to the higher cost of imaging sensors [12].

Although the research in artificial intelligence is progressing at warp speed, only a few studies have focused on imaging domains other than RGB. Furthermore, models are usually designed with visible light spectrum data in mind, assuming that high-frequency components are present in the input data, which are then directly applied to other datasets. However, this frequently leads to worse accuracy [13,14], as such networks cannot capture specific data characteristics, e.g., more distant relationships between object components in thermal images that require bigger receptive fields [15].

Taking this into account, this Special Issue focuses on increasing the community's awareness of the importance of thermal imagery, its benefits and challenges, as well as the need for careful analysis and design of AI solutions with specific data domains in mind. Proposals addressing various research topics are welcome, including, but not limited to:

  • Thermal imaging applications in medicine, automotive, aerospace, robotics, and surveillance industries, among others.
  • AI design for thermal imagery including Neural Architecture Search for domain-specific tasks.
  • Data translation between imaging domains.
  • Thermal data generation using AI.

Reference

  1. Usamentiaga, R., Venegas, P., Guerediaga, J., Vega, L., Molleda, J. and Bulnes, F.G., 2014. Infrared thermography for temperature measurement and non-destructive testing. Sensors, 14(7), pp.12305-12348.
  2. Kwasniewska, A., Ruminski, J. and Szankin, M., 2019. Improving accuracy of contactless respiratory rate estimation by enhancing thermal sequences with deep neural networks. Applied Sciences, 9(20), p.4405.
  3. Weinmann, M., Leitloff, J., Hoegner, L., Jutzi, B., Stilla, U. and Hinz, S., 2014. THERMAL 3D MAPPING FOR OBJECT DETECTION IN DYNAMIC SCENES. ISPRS Annals of Photogrammetry, Remote Sensing & Spatial Information Sciences, 2(1).
  4. Weiss, C., Kirmas, A., Lemcke, S., Böshagen, S., Walter, M., Eckstein, L. and Leonhardt, S., 2022. Head tracking in automotive environments for driver monitoring using a low resolution thermal camera. Vehicles, 4(1), pp.219-233.
  5. Renkielska, A., Kaczmarek, M., Nowakowski, A., Grudziński, J., Czapiewski, P., Krajewski, A. and Grobelny, I., 2014. Active dynamic infrared thermal imaging in burn depth evaluation. Journal of Burn Care & Research, 35(5), pp.e294-e303.
  6. Kwaśniewska, A., Rumiński, J. and Rad, P., 2017, July. Deep features class activation map for thermal face detection and tracking. In 2017 10Th international conference on human system interactions (HSI) (pp. 41-47). IEEE.
  7. Stypułkowski, K., Gołda, P., Lewczuk, K. and Tomaszewska, J., 2021. Monitoring system for railway infrastructure elements based on thermal imaging analysis. Sensors, 21(11), p.3819.
  8. Khaksari, K., Nguyen, T., Hill, B.Y., Quang, T., Perrault, J., Gorti, V., Malpani, R., Blick, E., Cano, T.G., Shadgan, B. and Gandjbakhche, A.H., 2021. Review of the efficacy of infrared thermography for screening infectious diseases with applications to COVID-19. Journal of Medical Imaging, 8(S1), p.010901.
  9. Kwasniewska, A., Ruminski, J., Szankin, M. and Kaczmarek, M., 2020. Super-resolved thermal imagery for high-accuracy facial areas detection and analysis. Engineering Applications of Artificial Intelligence, 87, p.103263.
  10. Baskaran, R., Møller, K., Wiil, U.K. and Brabrand, M., 2022. Using Facial Landmark Detection on Thermal Images as a Novel Prognostic Tool for Emergency Departments. Frontiers in artificial intelligence, 5.
  11. Głowacka, N. and Rumiński, J., 2021. Face with mask detection in thermal images using deep neural networks. Sensors, 21(19), p.6387.
  12. Zhou, H., Sun, M., Ren, X. and Wang, X., 2021. Visible-Thermal Image Object Detection via the Combination of Illumination Conditions and Temperature Information. Remote Sensing, 13(18), p.3656.
  13. Ramanagopal, M.S., Zhang, Z., Vasudevan, R. and Johnson-Roberson, M., 2020. Pixel-wise motion deblurring of thermal videos. arXiv preprint arXiv:2006.04973.
  14. Kwasniewska, Alicja, Maciej Szankin, Jacek Ruminski, Anthony Sarah, and David Gamba. "Improving Accuracy of Respiratory Rate Estimation by Restoring High Resolution Features with Transformers and Recursive Convolutional Models." In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3857-3867. 2021.
  15. Szankin, M., Kwasniewska, A. and Ruminski, J., 2019, June. Influence of thermal imagery resolution on accuracy of deep learning based face recognition. In 2019 12th International Conference on Human System Interaction (HSI) (pp. 1-6). IEEE.

Dr. Alicja Kwasniewska
Dr. M. Hamed Mozaffari
Prof. Dr. Yudong Zhang
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Journal of Imaging is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 1800 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Published Papers (4 papers)

Order results
Result details
Select all
Export citation of selected articles as:

Research

15 pages, 5611 KiB  
Article
Correlated Decision Fusion Accompanied with Quality Information on a Multi-Band Pixel Basis for Land Cover Classification
by Spiros Papadopoulos, Georgia Koukiou and Vassilis Anastassopoulos
J. Imaging 2024, 10(4), 91; https://doi.org/10.3390/jimaging10040091 - 12 Apr 2024
Viewed by 320
Abstract
Decision fusion plays a crucial role in achieving a cohesive and unified outcome by merging diverse perspectives. Within the realm of remote sensing classification, these methodologies become indispensable when synthesizing data from multiple sensors to arrive at conclusive decisions. In our study, we [...] Read more.
Decision fusion plays a crucial role in achieving a cohesive and unified outcome by merging diverse perspectives. Within the realm of remote sensing classification, these methodologies become indispensable when synthesizing data from multiple sensors to arrive at conclusive decisions. In our study, we leverage fully Polarimetric Synthetic Aperture Radar (PolSAR) and thermal infrared data to establish distinct decisions for each pixel pertaining to its land cover classification. To enhance the classification process, we employ Pauli’s decomposition components and land surface temperature as features. This approach facilitates the extraction of local decisions for each pixel, which are subsequently integrated through majority voting to form a comprehensive global decision for each land cover type. Furthermore, we investigate the correlation between corresponding pixels in the data from each sensor, aiming to achieve pixel-level correlated decision fusion at the fusion center. Our methodology entails a thorough exploration of the employed classifiers, coupled with the mathematical foundations necessary for the fusion of correlated decisions. Quality information is integrated into the decision fusion process, ensuring a comprehensive and robust classification outcome. The novelty of the method is its simplicity in the number of features used as well as the simple way of fusing decisions. Full article
(This article belongs to the Special Issue Data Processing with Artificial Intelligence in Thermal Imagery)
Show Figures

Figure 1

16 pages, 1040 KiB  
Article
Thermal Image Processing for Respiratory Estimation from Cubical Data with Expandable Depth
by Maciej Szankin, Alicja Kwasniewska and Jacek Ruminski
J. Imaging 2023, 9(9), 184; https://doi.org/10.3390/jimaging9090184 - 13 Sep 2023
Viewed by 1286
Abstract
As healthcare costs continue to rise, finding affordable and non-invasive ways to monitor vital signs is increasingly important. One of the key metrics for assessing overall health and identifying potential issues early on is respiratory rate (RR). Most of the existing methods require [...] Read more.
As healthcare costs continue to rise, finding affordable and non-invasive ways to monitor vital signs is increasingly important. One of the key metrics for assessing overall health and identifying potential issues early on is respiratory rate (RR). Most of the existing methods require multiple steps that consist of image and signal processing. This might be difficult to deploy on edge devices that often do not have specialized digital signal processors (DSP). Therefore, the goal of this study is to develop a single neural network realizing the entire process of RR estimation in a single forward pass. The proposed solution builds on recent advances in video recognition, capturing both spatial and temporal information in a multi-path network. Both paths process the data at different sampling rates to capture rapid and slow changes that are associated with differences in the temperature of the nostril area during the breathing episodes. The preliminary results show that the introduced end-to-end solution achieves better performance compared to state-of-the-art methods, without requiring additional pre/post-processing steps and signal-processing techniques. In addition, the presented results demonstrate its robustness on low-resolution thermal video sequences that are often used at the embedded edge due to the size and power constraints of such systems. Taking that into account, the proposed approach has the potential for efficient and convenient respiratory rate estimation across various markets in solutions deployed locally, close to end users. Full article
(This article belongs to the Special Issue Data Processing with Artificial Intelligence in Thermal Imagery)
Show Figures

Figure 1

26 pages, 28184 KiB  
Article
The Dangers of Analyzing Thermographic Radiometric Data as Images
by Časlav Livada, Hrvoje Glavaš, Alfonzo Baumgartner and Dina Jukić
J. Imaging 2023, 9(7), 143; https://doi.org/10.3390/jimaging9070143 - 12 Jul 2023
Cited by 1 | Viewed by 1140
Abstract
Thermography is probably the most used method of measuring surface temperature by analyzing radiation in the infrared part of the spectrum which accuracy depends on factors such as emissivity and reflected radiation. Contrary to popular belief that thermographic images represent temperature maps, they [...] Read more.
Thermography is probably the most used method of measuring surface temperature by analyzing radiation in the infrared part of the spectrum which accuracy depends on factors such as emissivity and reflected radiation. Contrary to popular belief that thermographic images represent temperature maps, they are actually thermal radiation converted into an image, and if not properly calibrated, they show incorrect temperatures. The objective of this study is to analyze commonly used image processing techniques and their impact on radiometric data in thermography. In particular, the extent to which a thermograph can be considered as an image and how image processing affects radiometric data. Three analyzes are presented in the paper. The first one examines how image processing techniques, such as contrast and brightness, affect physical reality and its representation in thermographic imaging. The second analysis examines the effects of JPEG compression on radiometric data and how degradation of the data varies with the compression parameters. The third analysis aims to determine the optimal resolution increase required to minimize the effects of compression on the radiometric data. The output from an IR camera in CSV format was used for these analyses, and compared to images from the manufacturer’s software. The IR camera providing data in JPEG format was used, and the data included thermographic images, visible images, and a matrix of thermal radiation data. The study was verified with a reference blackbody radiation set at 60 °C. The results highlight the dangers of interpreting thermographic images as temperature maps without considering the underlying radiometric data which can be affected by image processing and compression. The paper concludes with the importance of accurate and precise thermographic analysis for reliable temperature measurement. Full article
(This article belongs to the Special Issue Data Processing with Artificial Intelligence in Thermal Imagery)
Show Figures

Figure 1

14 pages, 3330 KiB  
Article
Improving Visual Defect Detection and Localization in Industrial Thermal Images Using Autoencoders
by Sasha Behrouzi, Marcel Dix, Fatemeh Karampanah, Omer Ates, Nissy Sasidharan, Swati Chandna and Binh Vu
J. Imaging 2023, 9(7), 137; https://doi.org/10.3390/jimaging9070137 - 07 Jul 2023
Cited by 1 | Viewed by 1423
Abstract
Reliable functionality in anomaly detection in thermal image datasets is crucial for defect detection of industrial products. Nevertheless, achieving reliable functionality is challenging, especially when datasets are image sequences captured during equipment runtime with a smooth transition from healthy to defective images. This [...] Read more.
Reliable functionality in anomaly detection in thermal image datasets is crucial for defect detection of industrial products. Nevertheless, achieving reliable functionality is challenging, especially when datasets are image sequences captured during equipment runtime with a smooth transition from healthy to defective images. This causes contamination of healthy training data with defective samples. Anomaly detection methods based on autoencoders are susceptible to a slight violation of a clean training dataset and lead to challenging threshold determination for sample classification. This paper indicates that combining anomaly scores leads to better threshold determination that effectively separates healthy and defective data. Our research results show that our approach helps to overcome these challenges. The autoencoder models in our research are trained with healthy images optimizing two loss functions: mean squared error (MSE) and structural similarity index measure (SSIM). Anomaly score outputs are used for classification. Three anomaly scores are applied: MSE, SSIM, and kernel density estimation (KDE). The proposed method is trained and tested on the 32 × 32-sized thermal images, including one contaminated dataset. The model achieved the following average accuracies across the datasets: MSE, 95.33%; SSIM, 88.37%; and KDE, 92.81%. Using a combination of anomaly scores could assist in solving a low classification accuracy. The use of KDE improves performance when healthy training data are contaminated. The MSE+ and SSIM+ methods, as well as two parameters to control quantitative anomaly localization using SSIM, are introduced. Full article
(This article belongs to the Special Issue Data Processing with Artificial Intelligence in Thermal Imagery)
Show Figures

Figure 1

Back to TopTop