A Deep Convolutional Neural Network for Detecting Volcanic Thermal Anomalies from Satellite Images

Abstract

The latest generation of high-spatial-resolution satellites produces measurements of high-temperature volcanic features at global scale, which are valuable to monitor volcanic activity. Recent advances in technology and increased computational resources have resulted in an extraordinary amount of monitoring data, which can no longer be so readily examined. Here, we present an automatic detection algorithm based on a deep convolutional neural network (CNN) that uses infrared satellite data to automatically determine the presence of volcanic thermal activity. We exploit the potentiality of the transfer learning technique to retrain a pre-trained SqueezeNet CNN to a new domain. We fine-tune the weights of the network over a new dataset opportunely created with images related to thermal anomalies of different active volcanoes around the world. Furthermore, an ensemble approach is employed to enhance accuracy and robustness when compared to using individual models. We chose a balanced training dataset with two classes, one containing volcanic thermal anomalies (erupting volcanoes) and the other containing no thermal anomalies (non-erupting volcanoes), to differentiate between volcanic scenes with eruptive and non-eruptive activity. We used satellite images acquired in the infrared bands by ESA Sentinel-2 Multispectral Instrument (MSI) and NASA & USGS Landsat 8 Operational Land Imager and Thermal InfraRed Sensor (OLI/TIRS). This deep learning approach makes the model capable of identifying the appearance of a volcanic thermal anomaly in the images belonging to the volcanic domain with an overall accuracy of 98.3%, recognizing the scene with active flows and erupting vents (i.e., eruptive activity) and the volcanoes at rest. This model is generalizable, and has the capability to analyze every image captured by these satellites over volcanoes around the world.

1. Introduction

Thermal satellite imagery has the potential to offer important insights into the thermal behavior of the most active volcanoes around the world, which can help minimize the need for extensive fieldwork. As a result of the continuous improvement of Earth observation technology, a variety of thermal sensors (e.g., AVHRR, MODIS, SEVIRI, GOES and many others) are currently in orbit, providing coverage even for remote and unreachable volcanoes at different temporal resolutions, ranging from minutes to few days. These sensors acquire data over potentially hazardous, high-temperature phenomena, such as volcanic eruptions, with relatively low cost and no risk to the end user. However, the data volume collected by satellite thermal sensors is too large to be readily processed and analyzed on a daily basis and global scale with traditional techniques (i.e., those that employ a threshold). Therefore, an automated and fast approach to accurately detect these thermal anomalies in satellite data is a critical component to any operational volcano monitoring system. Over the last years, Artificial Intelligence (AI) techniques have become the fastest-growing trend in remote sensing data analysis applications, which often involve large amounts of data. The new technologies of Artificial Intelligence, i.e., Machine Learning (ML) and Deep Learning (DL) in particular [1,2], could help the operator to manage these data efficiently. ML techniques are a type of AI in which computers learn from structured data, i.e., representative features are opportunely selected and extracted during the input feature engineering phase. In the specific context of satellite image processing, these values can be the set of pixel spectral intensities in three-bands images (e.g., RGB images) recording the spectral response of the monitored area under investigation [3,4,5,6]. DL models are artificial neural networks capable of analyzing unstructured data, exploiting the spatial and/or temporal relationships between neighboring data to automatically extract significant features.

The effectiveness of ML and DL algorithms in recognizing some volcanic features in optical and radar satellite images has already been tested. Machine learning models were used to identify and map the thermal emissions from active and cooling lava flows [5,7], and to estimate the areal coverage and the total volume of lava fields or deposits [8,9,10,11,12]. Machine learning and deep learning techniques were adopted to detect and characterize the main components of the volcanic plumes during explosive eruptions [13,14,15,16]. Moreover, deep learning algorithms were applied to automatically recognize subtle to intense thermal anomalies exploiting the spatial relationships of the volcanic features [17]. The results from these studies show that the deep learning techniques, compared to traditional machine learning methods, achieve better accuracy in automatic analysis and recognition of the specific features in contexts in which spatial and temporal information aid characterizing satellite images.

Different deep learning models have already been designed to automatically identify objects in different contexts [18,19,20,21]. The most diffused DL models are the deep Convolutional Neural Networks (CNNs) [22,23,24], which have shown great promise in image classification. These data-driven methods are all based on hierarchical feature learning, which can easily be adapted to other specific applications without the need for manual feature extraction [25]. Examples of CNNs are LeNet, AlexNet, VGG16, GoogleNet, ResNet and SqueezeNet [26,27,28,29,30,31]. Deep CNN models are usually trained over datasets containing millions of images across thousands of classes, such as ImageNet (http://www.image-net.org/; [32]). This dataset, in continuous development, is composed of around 14 million annotated images belonging to 1000 different classes. The SqueezeNet trained with the ImageNet dataset achieves a satisfactory balance between training time and classification accuracy [31,33]. Moreover, the SqueezeNet network has the advantage of being among the most compact and fastest networks for classification tasks. Its principal repeating module, the “fire” module, is composed of a “squeeze” convolutional module, followed by an “expand” module, and finally, a softmax function is used for the classification [31].

Although some volcanological applications rely on high volumes of data, a general issue is the limited availability of labeled datasets as ground truth, i.e., the real labels that show the real belonging class of the data [25,34,35]. The performance of deep learning algorithms may be negatively influenced by the limited representative training samples, for example, due to the overfitting problem, where the model is excessively complex relative to the small dataset, and thus, is not being able to generalize to unseen data [2]. However, it is possible to apply pre-trained ImageNet models to small datasets and reduce training time using the technique of transfer learning in a deep learning framework (e.g., [36]). This technique is based on the concept in which a Neural Network already trained with a large dataset and with high performances is subsequently retrained with some images from a different domain and a small number of epochs [2], with the aim of classifying the images under analysis (or other tasks), using the fine-tuning approach [37]. In this way, it is possible to readapt the parameters (weights) of the original model to the new domain, exploiting the potentiality already developed with the previous training data, rapidly and in an efficient way, re-setting all the parameters and re-running all the network with the new dataset and few epochs, instead of learning from scratch [38].

Here, we present a monitoring system to automatically detect volcanic thermal anomalies associated with eruptive activity. In particular, we explore the potential of the SqueezeNet model already trained with the ImageNet dataset (SqueezeNet 1.0 model website) to identify the presence of high temperature volcanic features related to active flows (such as lava flows, pyroclastic flows, warm lahars) and erupting vents (such as lava lake, lava fountains, hot fumaroles) that would appear in the thermal infrared satellite measurements. We use advanced learning techniques, which strongly reduce training times and improve the accuracy level. Thus, we applied the transfer learning technique to readjust the parameters of the pre-trained SqueezeNet model, using a new image dataset containing high temperature volcanic features representative of two classes, those belonging to thermal anomalies related to active flows and vents (i.e., eruptive activity) and those not belonging to thermal anomalies (volcanoes at rest). SqueezeNet’s training time and verification sample accuracy are optimal, using a small number of epochs and a limited number of images for the fine-tuning. In addition, an ensemble approach is used to achieve higher accuracy and robustness compared to individual models. We investigated the thermal features of several active volcanoes worldwide, using the thermal images (infrared) captured by the high spatial resolution ESA Sentinel-2 (from 10 m to 60 m) and NASA & USGS Landsat 8 satellites (from 30 m to 100 m). All the satellite images were obtained via Google Earth Engine (GEE) cloud-based platform. GEE is appealing not only because of its free access, but also because it makes available a powerful application programming interface (API) for processing the satellite data [39]. The SqueezeNet model was developed in Google Colaboratory (Colab), a cloud platform that can easily integrate GEE and machine learning and deep learning applications for data analysis [40].

2. Materials

2.1. Volcanoes

We investigated 9 different volcanoes (see Figure 1): Cumbre Vieja (La Palma, Spain), Etna (Italy), Pico do Fogo (Cape Verde), Volcán de Fuego (Guatemala), Geldingadalir (Iceland), Kīlauea (Hawaii, USA), Klyuchevskaya (Kamchatka, Russia), Pacaya (Guatemala) and Stromboli (Italy). All these volcanoes were characterized by frequent eruptive activity in the 2000s (

Table 1. List of volcanoes and years with eruptive activity between 2013 and 2022 sampled for the study. Details of volcanoes and associated links are available from the US National Museum of Natural History’s Global Volcanism Program (https://volcano.si.edu/).

Volcano	Eruptive Activity between 2013 and 2022
Cumbre Vieja	2021
Etna	2013–2014–2017–2018–2019–2020–2021–2022
Pico do Fogo	2014–2015
Volcán de Fuego	2017–2019–2020–2021
Geldingadalir	2020–2021
Kīlauea	2020–2021
Klyuchevskaya	2020–2021
Pacaya	2020–2021
Stromboli	2014–2019–2020–2021

). We selected these sites for their different volcanic settings and eruptive styles, enabling a more comprehensive evaluation of the capabilities and limitations of infrared remote sensing for detecting and tracking eruptive activity.

Cumbre Vieja, Etna, Volcán de Fuego, Klyuchevskaya and Stromboli are stratovolcanoes with different types of eruptions. Cumbre Vieja is characterized by explosive activity and lava flows. Etna has a continuous degassing activity, with frequent effusive and explosive eruptions. Persistent activity at Volcán de Fuego is distinguished by prolonged periods of lava effusion, frequent yet moderate explosions, and intermittent episodes of paroxysmal activity. Klyuchevskaya is characterized by recurring moderate-volume explosive and effusive eruptions without significant periods of inactivity. Stromboli is dominated by a persistent (strombolian) activity, interspersed with major eruptions and paroxysms. In addition, Pico do Fogo, Geldingadalir, Kīlauea and Pacaya are volcanoes with distinctive landforms. Pico do Fogo is a caldera with continuous activity in the past centuries and recent lava flow eruptions. Geldingadalir is a fissure vent with an effusive activity and sporadic lava fountains. Kīlauea is a shield volcano with a summit caldera, characterized by frequent lava flow eruptions at the summit and flanks alternating with periods of lava lake activity and, finally, Pacaya is a large volcanic complex with strombolian eruptions, lava flow extrusions in the caldera moat, and occasional larger explosive eruptions [from Smithsonian Institution, Global Volcanism Program. Available online: https://volcano.si.edu/, accessed on 18 November 2022].

2.2. Data Source

In this study, the Sentinel-2 and Landsat 8 satellite images were used to identify volcanic thermal anomalies. These two collections of data were chosen because they have a similar range of spectral bands in the portion of the spectrum from the visible to the infrared. Therefore, in this range they could be interchangeable, using the Sentinel-2 or Landsat 8 scenes based on the availability of the data for the specific situation, due to the passage of the satellite in a specific location and at a specific moment.

The Sentinel-2 (S2) mission is part of the Copernicus programme of the European Space Agency (ESA). The mission comprises a constellation of two identical sun-synchronous satellites known as Sentinel-2A (S2A) and Sentinel-2B (S2B), which were launched in 2015 and 2017 respectively (https://sentinel.esa.int/web/sentinel/missions/sentinel-2/). Each satellite in the constellation has a revisit time of 10 days, which collectively leads to a global revisit frequency of 5 days. Both satellites are equipped with a MultiSpectral Instrument (MSI) capable of capturing imagery in 13 bands. The spatial resolution of these bands vary, with a resolution of 10 m for the visible and near-infrared, 20 m for the red edge and shortwave infrared part of the spectrum, and 60 m for the atmospheric bands (“Aerosol,” “Water vapor,” and “Cirrus”).

Landsat 8 (L8) is an Earth observation satellite launched in the frame of a joint mission between the United States Geological Survey (USGS) and National Aeronautics and Space Administration (NASA) in 2013 (https://landsat.gsfc.nasa.gov/satellites/landsat-8/). It carries two sensors in a circular sun-synchronous orbit, namely Operational Land Imager (OLI) and Thermal InfraRed Sensor (TIRS). The first one (OLI) acquires 9 spectral bands, in the visible, near infrared and shortwave infrared (spatial resolution of 30 m). Moreover, it possesses a Panchromatic band at a spatial resolution of 15 m. The second sensor (TIRS) acquires data with 100 m spatial resolution for 2 thermal bands. Landsat 8 data have a high radiometric resolution (16 bits) and are characterized by a repeat cycle of 16 days.

We have acquired all the images via the Google Earth Engine (GEE) platform (https://earthengine.google.com/). Sentinel-2 and Landsat 8 scenes are available with different processing levels, containing raw at-sensor radiance values (L1A) as well as geometrically and radiometrically corrected sensor radiance values (L1B). These processing levels can be used to derive additional Level 2 data products, such as atmospherically corrected surface radiance or surface temperature and emissivity. We have processed the Sentinel-2 images in the bands of Near InfraRed (NIR: B8 with 0.84 μm (S2A)/0.83 μm (S2B) wavelength) and Short-Wave InfraRed (SWIR: B11, with 1.61 μm (S2A, S2B) wavelength, and B12, with 2.20 μm (S2A)/2.19 μm (S2B) wavelength) Since NIR and SWIR Sentinel-2 bands have different spatial resolutions ranging between 10 m and 20 m, all S2 images were automatically resampled to the same spatial resolution of 20 m using the nearest neighbor method. Moreover, we have used the Landsat 8 images in the NIR (B5: 0.85–0.88 μm wavelength) and SWIR (B6: 1.57–1.65 μm wavelength, and B7: 2.11–2.29 μm wavelength) bands. Additionally, the Landsat 8 images were resampled to the higher spatial resolution of 20 m. We have analyzed the Top-Of-Atmosphere (TOA) reflectance data from both sensors, a dimensionless measurement that describes the ratio between reflected and incident solar radiation on a given surface.

3. Method

3.1. Anomalous Volcanic Temperatures

Volcanic activity is typically marked by the transfer of heat to the Earth’s surface and thermal emissions from the erupted products. Thus, an evident indicator of volcanic activity is the high surface temperature compared to the ambient temperature of the ground. Hence, identifying volcanic thermal activity in satellite images is relatively simple; it is sufficient to distinguish the pixels that exhibit thermal emissions directly associated with volcano activity from neighboring pixels that do not [41]. This approach allows not only for the detection of volcanic unrest based on the occurrence of thermal anomalies at a specific time (e.g., [17]), but also for the use of thermal data to retrieve such parameters as the effusion rate of lava flows (e.g., [42,43,44]) and the areal coverage of hot volcanic deposits (e.g., [45]). As a result, thermal satellite observations are an essential component of volcano monitoring and a valuable tool for identifying numerous volcanic features based on their thermal characteristics (e.g., [46]). However, the task of distinguishing volcanic thermal features with respect to the background is made difficult not only by the characteristics of the target and atmospheric conditions (optical sensors are unable to make measurements through the cloud cover) but also by the spatial, temporal, spectral and radiometric resolution at which the remote sensing data are captured and whether the measurements are made during the day or at night (an additional contribution from reflected solar energy is present during the day). Such complications can hamper the detection of subtle temperature anomalies by using low-spatial-resolution sensors or introduce spurious data into time series not attributable to active volcanism (e.g., [47]).

3.2. Theoretical Background

To detect and quantify the thermal emissions from volcanic targets, it is necessary to consider the relation between radiance (i.e., rate of energy flow per unit area) and wavelength. The radiance emitted from a surface, or the reflectance (i.e., a dimensionless measurement that describes the ratio between solar reflected and incident radiation on a given surface), depends on its temperature and radiating efficiency (emissivity) [48]. According to the Stefan–Boltzmann law, the higher the temperature of a surface, the greater the radiance it emits [48]. This applies to all wavelengths of emission but, according to Wien’s displacement law, the peak wavelength of thermal emission also shifts to shorter wavelengths with increasing temperature. Hence, a hotter volcanic surface will radiate more energy compared to a cooler surface, and variations in the radiant flux, which is quantified in terms of radiance per unit of wavelength, are directly related to changes in volcanic activity. This can include phenomena such as a fresh lava flow, an active lava lake or a subtly warm fumarole field [48].

In terms of the temperature of hot volcanic features, fresh basaltic lava may be between 1073 and 1273 K near the vent of the flow [49]. Lava lakes can reach extremely high temperatures up to 1473 K [50]. Gasses emitted from volcanic fumaroles are found to be between ambient temperature and 673 K [51], although temperatures up to 1173 K have been measured [52,53]. In terms of spectral radiance, therefore, the emissions from very hot surfaces (e.g., >900 K) are enhanced in near infrared (NIR, ~0.9–1.1 µm) and shortwave infrared (SWIR, ~1.2–2.5 µm) regions of the spectrum compared with those in the middle infrared (MIR, ~3–5 µm) and longwave (thermal) infrared (TIR, ~7–14 µm) bands.

Various satellites are equipped with sensors that are able to record these infrared radiations and, as such, can be used for detecting and monitoring active volcanism. However, MIR and TIR detections from space will be mainly due to the combination of emissions from both ambient and hot surface components, for which to accurately quantify the emissions related to active volcanism, it is necessary to remove the contributions from the ambient background. Instead, NIR and SWIR signals recorded from space will be primarily due to the temperature of a very hot emitting surface (e.g., fresh lava).

3.3. Three-Step Approach

We propose a satellite thermal monitoring system to automatically detect high-temperature anomalies related to active flows or erupting vents. The monitoring system, based on a deep CNN, can be divided into three steps (Figure 2). The first one builds a balanced volcanic dataset composed of 200 images with high spatial resolution acquired by Landsat 8 and Sentinel-2 in the NIR and SWIR bands, covering the investigated volcanoes. The second step uses the volcanic dataset to re-train the SqueezeNet model applying the transfer learning approach, while the third one combines the different SqueezeNet trained with the Transfer Learning approach, obtaining an Ensemble Classifier, that ingests new NIR and SWIR images as they are available to detect if volcanic thermal anomalies are present. These three steps are described in more detail below.

Firstly, we built the volcanic dataset, sampling 135 images from the Sentinel-2 satellite and 65 images from the Landsat 8 satellite, balancing about the same number of images from each sensor between class 1 and class 0. Thus, a total of 100 images are related to the class with volcanic thermal anomalies and a total of 100 images to the class without anomalies. CNNs require balanced training datasets of images to effectively differentiate into two classes by the presence or absence of volcanic thermal anomalies. We named the first class “Yes Anomaly”, with class ID 1, and the second one “No Anomaly”, with class ID 0. Images with lava flows and active vents belong to the first class, images with volcanoes at rest to the second one. Every image was assigned to the correct class by carefully analyzing the eruptive activity of each volcano both using the volcanological bulletins (e.g., from Smithsonian Institution, Global Volcanism Program, available online: https://volcano.si.edu/, or from INGV National Institute of Geophysics and Volcanology, available online: https://www.ct.ingv.it/) and manually inspecting the scenes. In the same way, we define the “ground truth” labels (i.e., the real labels of the scenes) either examining the volcanological bulletins or manually inspecting the scenes. For representativity of the dataset, we choose images for class 0 with different characteristics, e.g., presence of clouds, snows, sea in the scene, or building areas. We have obtained a final balanced dataset composed of 200 images belonging to the 9 different volcanoes investigated (Cumbre Vieja, Etna, Pico do Fogo, Fuego, Geldingadalir, Kīlauea, Klyuchevskaya, Pacaya and Stromboli), with images acquired during the years 2013–2022 (see Table 1). In the pre-processing phase, we convert the TOA images to the .png format (in the 0–255 scale) to feed the SqueezeNet model. Figure 3 shows an example of images sampled from different volcanoes, belonging to class ID 1 and to class ID 0. The complete dataset used for the Transfer Learning approach is available at the link: https://zenodo.org/record/7944343#.ZGSp8HZBy5c, with the doi: 10.5281/zenodo.7944343.

Subsequently, we used the SqueezeNet model (from the SqueezeNet 1.0 model website), pretrained with the ImageNet dataset, in Google Colab with the PyTorch library [54], and we readjusted this model for the detection of thermally anomalous pixels, setting two final classes for the classification. The volcanic dataset was randomly divided into three independent subsets: a train set, a validation set and a test set. The train set is used, in general, to train the model, the second one to validate the results in the train phase and to fit the hyperparameters. The last set is used to test the model with images not seen in the training phase and to calculate the predictive performances of the model. For each run of the SqueezeNet model, we used 120 images for the train set (60% of the volcanic dataset), 20 images for the validation set (10% of the volcanic dataset) and 60 images for the test set (30% of the volcanic dataset). We used such a small amount of training data to verify the reliability and robustness of the model in transferring knowledge to a new domain (with transfer learning). We fed the train and validation sets to the SqueezeNet 1.0 model in Google Colab, applying the transfer learning approach, running the model with a learning rate alpha of 0.001, a batch size of 25 and 5 epochs for the fine-tuning [2]. The number of 5 epochs was chosen as a good trade-off between complexity of the algorithm and goodness of the model, because a higher number of epochs did not improve the performance much compared to the complexity.

Finally, we ran the trained SqueezeNet model a total of 10 times over the test subset (30% of the volcanic dataset). An ensemble strategy, combining the 10 trained SqueezeNet models, was adopted to enhance the generalization capabilities; that is, the final model outcome is provided by the outcome most voted upon by the 10 runs of the trained model. Ensemble methods use multiple models to achieve better predictive performances than could be obtained from any of the individual models alone [55]. The ensemble result obtained combining the results of the ten SqueezeNet test runs is used as the output for each image classified. The goodness of the classification for the test set is estimated using different model performance indices, i.e., the accuracy index defined as in Equation (1), the precision index as in Equation (2), the recall index as in Equation (3) and the F1 score as in Equation (4), then calculated as percentages [56]:

$$\mathrm{A}\mathrm{c}\mathrm{c}\mathrm{u}\mathrm{r}\mathrm{a}\mathrm{c}\mathrm{y}=\frac{\mathrm{T}\mathrm{P}+\mathrm{T}\mathrm{N}}{\mathrm{T}\mathrm{P}+\mathrm{T}\mathrm{N}+\mathrm{F}\mathrm{P}+\mathrm{F}\mathrm{N}}$$

(1)

$$\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}=\frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{P}}$$

(2)

$$\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}=\frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{N}}$$

(3)

$$\mathrm{F}1\mathrm{s}\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{e}=\frac{2•\mathrm{T}\mathrm{P}}{2•\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{P}+\mathrm{F}\mathrm{N}}=2•\frac{\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}•\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}}{\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}+\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}}$$

(4)

where TP (True Positives) are the images predicted to belong to class 1 that actually do belong to that class, TN (True Negatives) are the images predicted to belong to class 0 that actually do belong to that class, FP (False Positives) are the images incorrectly predicted to belong to class 1 that actually do belong to class 0, and FN (False Negatives) are the images incorrectly predicted to belong to class 0 that actually belong to class 1. The total number of images in the test set is given by the sum of TP, TN, FP and FN.

4. Results

The SqueezeNet model achieved a good balance between training time and classification accuracy for automatically identifying volcanic thermal anomalies. The SqueezeNet network, already trained with the ImageNet dataset, and retrained for five epochs in the transfer learning process, is tested over the test dataset. Ten runs of the SqueezeNet model are used to design the ensemble classifier. For each image, the outcome of the ensemble SqueezeNet is assigned as the most frequent outcome among the trained model runs. Generally speaking, each run provides good performances, as Figure 4 shows, with accuracy values ranging from 91.7% (for one run over the total ten runs) to 100.0% (for two runs over the total ten runs) and an average accuracy of about 96.2% (two images misclassified), with a minimum of zero and a maximum of five misclassified images over the sixty total images for all the ten runs of the trained SqueezeNet. The final accuracy of the ensemble SqueezeNet is then computed taking the mode of each run outcome, resulting in an overall accuracy of 98.3%, with a unique image misclassified over the total. In addition, this final ensemble model reaches a Precision of 100.0%, a Recall of 95.7%, and a F1 score of 97.8%, indicating a high-fidelity model.

It is worth noting that we have inserted in the class 1 dataset not only images with clear active flows, but also images that show a presence of erupting vents. Even though this type of image does not show an active flow in progress, the scenes do show volcanic activity, and these kinds of anomalies could evolve towards different eruptive phenomena, including lava overflows.

In order to investigate the reasons behind single run misclassifications, image samples misclassified by three different fine-tuning representative runs are shown in Figure 5. The first run (R1) is shown in Figure 5a,b, with images labeled as class 1 and classified as belonging to class 0, i.e., FN samples. The second run (R2) is shown in Figure 5c,d, with images labeled as class 0, classified as belonging to class 1, i.e., FP images. Finally, the third run (R3) is shown in Figure 5e,f, with an image labeled as class 1, classified as belonging to class 0 (Figure 5e) and an image belonging to class 0, classified as belonging to class 1 (Figure 5f). These misclassifications are probably due to the heterogeneity of the intraclass scenes, i.e., different kinds of volcanic thermal anomalies and volcanic at rest scenes. Indeed, these are randomly selected as training images and this may produce unbalanced training datasets, i.e., for example more scenes of active vents than lava flows for class 1, or more clear images than cloudy in class 0, thus polarizing the SqueezeNet model performances. That is why an ensemble approach allows for the highest output to be achieved from a small dataset while keeping high generalization capabilities. By considering the outcomes of the ensemble SqueezeNet, only the scene in Figure 5b,e remains misclassified, even though this is a borderline case because the mode results in a 50% probability to belong to one of the classes and a 50% probability to belong to the other one.

5. Discussion

The detection of volcanic thermal anomalies is obviously possible with several methods, ranging from most traditional threshold-based algorithms to most innovative machine learning based techniques. Traditional algorithms are based on the setting of thresholds to cluster the data. These thresholds can be either fixed a priori, obtained with a preliminary study over a group of training data, or selected adopting a statistical approach, calculating mean and standard deviation for each image [57]. In the first case, the analysis can be conducted comparing each pixel of an image with the fixed thresholds, in the second case, with the mean and standard deviation of each entire image, using an interval of confidence centered in the mean and with a radius of n ∙ standard deviations, typically setting n = 2 or n = 3.

ML methods are based on either supervised or unsupervised learning. Labeled datasets, containing sample data that have been tagged with a target parameter, are employed in supervised learning techniques to train machine learning algorithms. The accuracy of the algorithm in interpreting the training data is evaluated using the target. In unsupervised approaches, unlabeled data are fed to the algorithm, which tries to extract common features, schemes or patterns, taking into account only the data similarity.

To make a comparison of the performances of the threshold-based algorithms and ML techniques with the SqueezeNet model, we run them over the test set used in this study (

Table 2. List of satellite images selected for the test set. For each image, date, satellite used, volcano investigated, and presence or not of eruptive activity (the ground truth, i.e., the real labels of the scenes) are shown (date format: day/month/year written as dd/mm/yyyy).

Date (dd/mm/yyyy)	Satellite	Volcano	Activity (Yes: 1, No: 0)	Date (dd/mm/yyyy)	Satellite	Volcano	Activity (Yes: 1, No: 0)
03/05/2013	Landsat 8	Etna	0	03/02/2021	Sentinel-2	Geldingadalir	0
19/05/2013	Landsat 8	Etna	0	07/02/2021	Sentinel-2	Cumbre Vieja	0
02/09/2014	Landsat 8	Etna	0	18/02/2021	Sentinel-2	Etna	1
19/09/2014	Landsat 8	Stromboli	1	21/02/2021	Sentinel-2	Etna	1
28/02/2015	Landsat 8	Fogo	0	25/02/2021	Landsat 8	Fuego	0
07/10/2015	Landsat 8	Etna	0	25/02/2021	Landsat 8	Pacaya	1
03/12/2015	Landsat 8	Etna	1	06/03/2021	Landsat 8	Geldingadalir	0
16/03/2017	Sentinel-2	Etna	1	13/03/2021	Landsat 8	Geldingadalir	0
18/05/2017	Sentinel-2	Etna	0	22/03/2021	Landsat 8	Geldingadalir	1
24/03/2018	Sentinel-2	Etna	0	25/03/2021	Sentinel-2	Pacaya	1
08/05/2018	Sentinel-2	Etna	0	04/05/2021	Sentinel-2	Geldingadalir	1
10/05/2018	Sentinel-2	Etna	0	17/05/2021	Sentinel-2	Etna	0
30/05/2018	Sentinel-2	Etna	0	19/05/2021	Sentinel-2	Pacaya	0
29/06/2018	Sentinel-2	Etna	0	19/05/2021	Sentinel-2	Etna	1
24/12/2018	Sentinel-2	Etna	1	25/05/2021	Landsat 8	Etna	1
14/06/2019	Sentinel-2	Etna	0	27/05/2021	Sentinel-2	Kīlauea	0
19/07/2019	Sentinel-2	Etna	1	13/06/2021	Sentinel-2	Etna	1
27/07/2019	Sentinel-2	Etna	1	16/06/2021	Sentinel-2	Kīlauea	0
18/08/2019	Sentinel-2	Etna	0	12/08/2021	Sentinel-2	Geldingadalir	1
02/04/2020	Sentinel-2	Klyuchevskaya	0	12/08/2021	Sentinel-2	Geldingadalir	1
04/04/2020	Sentinel-2	Klyuchevskaya	0	20/08/2021	Sentinel-2	Stromboli	0
11/06/2020	Sentinel-2	Kīlauea	0	26/08/2021	Sentinel-2	Cumbre Vieja	0
23/07/2020	Sentinel-2	Pacaya	0	30/08/2021	Sentinel-2	Etna	1
16/09/2020	Sentinel-2	Klyuchevskaya	0	09/09/2021	Sentinel-2	Stromboli	0
20/10/2020	Landsat 8	Fuego	0	21/09/2021	Landsat 8	Etna	1
10/11/2020	Sentinel-2	Klyuchevskaya	1	21/09/2021	Sentinel-2	Stromboli	0
21/11/2020	Landsat 8	Geldingadalir	0	03/11/2021	Sentinel-2	Etna	0
28/12/2020	Sentinel-2	Kīlauea	1	05/11/2021	Sentinel-2	Etna	0
18/01/2021	Sentinel-2	Etna	1	28/11/2021	Sentinel-2	Kīlauea	1
24/01/2021	Landsat 8	Geldingadalir	0	08/02/2022	Sentinel-2	Etna	0

). In particular, we used two algorithms reliant on fixed and statistical threshold approaches (with n = 3) [58,59], an unsupervised X-Means ML model [1], and a supervised Random Forest (RF) ML model [60]. This last model was already trained using Sentinel-2 data to get the areal coverage of fresh anomalies, so the comparison with the RF model was made using only the Sentinel-2 images contained in the test set under analysis. Table 2 shows the correspondence between the date of the events in the test set used and the relative volcanoes and satellites for each event, with the ground truth values.

Figure 6 shows the ability to detect the volcanic anomalies for all methods analyzed. The algorithms with fixed and statistical thresholds, the unsupervised and supervised machine learning techniques, and the deep learning model are compared for all the different events in the test set under analysis (Table 2). For each day in the date axis, the presence of a bar of a specific color refers to the detection of the anomaly for that specific date by the relative model, the absence of the bar refers to the absence of thermal anomaly detected on that day. In particular, the dark red bars are the ground truth results (i.e., the real labels), highlighting the real presence of volcanic thermal anomalies on a specific day, between 2013 and 2022. The green bars are related to the threshold-based algorithms, with the light green bars for the fixed threshold method and the dark green bars for the statistical threshold method. The orange bars refer to the unsupervised X-Means model and the yellow bars to the supervised Random Forest one. Finally, the blue bars are related to the SqueezeNet model.

In case of the presence of anomalies (presence of the ground truth dark red bar for a specific day), a good performance of each model analyzed is reached when the ground truth bar is surmounted by all of the five bars related to the outputs of the five models under comparison (fixed threshold, statistical threshold, X-Means, RF, SqueezeNet), e.g., 19 July 2019 and 10 November 2020 cases. Similarly, in case of the absence of anomalies (absence of the ground truth dark red bar for a specific day), a good performance of each model analyzed is obtained when there are no bars related to that specific date, e.g., 19 May 2013 and 29 June 2018 cases. All other cases present some misclassifications. For instance, the 16 September 2020 case study shows absence of volcanic thermal anomalies at Klyuchevskaya volcano. Figure 6 for this case presents bars for the statistical threshold algorithm and the X-Means model, showing that these two models had detected anomalies. In the same way, the 25 May 2021 case study presents a thermal anomaly at Mt. Etna, and this was detected only by fixed and statistical threshold algorithms and SqueezeNet, and not by X-Means (RF is not considered because a Landsat 8 image is used in this case and RF is trained only over Sentinel-2 data). Cases such as these, misclassified by the threshold and ML models and well classified by the DL model, highlight one of the strengths of the DL approaches, i.e., the capability to exploit not only the spectral features but also the spatial ones of the scenes under analysis. In detail, the presence of specific geometric shapes for the thermal anomalies or the background emphasizes the SqueezeNet classification capability; in contrast, the other methods do not consider the spatial features but only the spectral ones, linked to the spectral response of each pixel captured by the satellites [6]. Overall, the SqueezeNet model gives the correct output for each of the test cases, except for the 25 February 2021 case at Pacaya volcano, in which thermal anomalies were present, but were not identified by the model.

Figure 7 shows the accuracy (see Equation (1)) of the different methods for the detection of volcano thermal anomalies with respect to the ensemble SqueezeNet, by using Sentinel-2 and Landsat 8 data (accuracy percentages here rounded to the unit for ease of analysis). It is worth noting that a comparison of the accuracy with the RF model was possible only with the Sentinel-2 data because RF was not trained over the Landsat 8 data. Figure 7a is related to the comparison only for the Sentinel-2 data. The SqueezeNet (blue bar) has an accuracy of 100%. The fixed threshold method (light green bar) has a high accuracy of 98%. Instead, the statistical threshold method (dark green bar) has a lower accuracy of 74% and the X-Means (orange bar) has an accuracy of 60%. The RF model (yellow bar), in this case, has the same accuracy of the SqueezeNet one. However, the proposed model is readily available for S2 and L8 data, allowing for the detection of thermal anomalies reducing the processing dead times, taking advantage of the first available acquisition between the S2-MSI and L8-OLI.

Figure 7b is related to the comparison only with the Landsat 8 data. The threshold-based algorithms have a lower accuracy than the precedent Sentinel-2 data case (Figure 7a), reaching 94% of accuracy for the fixed threshold model and 71% for the statistical threshold model, compared to the precedent 98% and 74%. In contrast, the X-Means accuracy is in growth (65% with respect to 60%). The SqueezeNet model reaches an accuracy of 94% (only one Landsat 8 misclassified image over the total). A similar accuracy is reached by a different model, using only one type of sensor. In contrast, SqueezeNet can be applied to any of the available satellite radiometric scenes captured by the different satellite sensors.

Figure 7c is related to the comparison of both Sentinel-2 and Landsat 8 data considered together. The threshold-based algorithms have a slightly lower accuracy than the Sentinel-2 data case (Figure 7a), reaching 97% accuracy for the fixed threshold method and 73% for the statistical threshold method. Nevertheless, these values are higher than the second case with only Landsat 8 data (Figure 7b). In contrast, the X-Means model has 62% accuracy, showing an opposite trend, with higher accuracy than the first case and lower than the second one. In this last case, the SqueezeNet model reaches 98% accuracy, which is the highest compared to other techniques, corresponding to only one misclassfied Landsat 8 image.

In summary, the SqueezeNet model has 100% accuracy with the Sentinel-2 images, whereas with Landsat 8 images, it achieves almost the maximum accuracy of 94%, similar to the fixed threshold method. However, the SqueezeNet model has the great advantage of being able to use Sentinel-2 and Landsat 8 data indifferently (both with high spatial resolution and with the similar spectral range of data for the infrared bands), achieving the highest accuracy compared to the other methods (98%). In this way, it is possible to detect thermal anomalies in less time, as there is no need to wait for the temporal step between two acquisitions for a single satellite (Sentinel-2 or Landsat 8) because the first image available between the Sentinel-2 and Landsat 8 data can be referenced.

6. Conclusions

In recent years, CNNs have become one of the most popular deep learning methods for image classification across various domains. Thus, we explored the potential of a deep CNN model, which has efficient and robust image processing capabilities, to automatically detect thermal anomalies associated with the eruptive activity of a volcano. The most frequent applications of current remote sensing processing methods based on deep CNNs concern land use classification and feature target recognition [61,62,63], and very limited studies have been conducted for volcano monitoring and forecasting. However, the use of deep learning in volcanology is rapidly growing (i.e., [17]), even if the unavailability of large and labeled training datasets presents a key issue in improving the performance of CNNs for volcanic applications. CNNs have shown the ability to overcome limits related to purely intensity-based approaches in detecting the presence of volcanic activity. Different convolutional neural networks have already been developed and trained with enormous datasets to create a model robust and capable enough to detect objects and classify images with high performances. All of this knowledge can be transferred to a new domain, using the transfer learning approach, fine-tuning the parameters over the new images. In this way, it is possible to readapt a model for the same task of classification or detection in a new domain of images, without wasting time or computational resources.

We used high spatial resolution infrared satellite data, which are fundamental to monitor hot temperature volcanic events, and adopted this transfer learning approach with the pre-trained SqueezeNet CNN. This allowed us to readjust the model more quickly and accurately to the volcanic domain, using a balanced dataset containing infrared images acquired by Sentinel-2 and Landsat 8 satellites. We collected 200 images representative of two different classes, one containing volcanic thermal anomalies (active flows and erupting vents) and the other one not containing thermal anomalies (volcanoes at rest), to fine-tune the weights that the SqueezeNet model had learned with the ImageNet dataset and to test the SqueezeNet model. In this way, we obtained a robust model capable of detecting the presence of active flows or erupting vents in the images under analysis. The comparison of this model with threshold algorithms and other machine learning approaches showed that the SqueezeNet achieves the highest performance score, making it possible to detect the presence of volcanic thermal anomalies with an accuracy of 98.3% using Sentinel-2 and Landsat 8 satellite data. Thus, the application of transfer and ensemble learning approaches greatly reduces computational costs and training times and improves the level of accuracy. However, scenes that contain subtle or thermal anomalies predominantly covered by clouds may be incorrectly classified as “No Anomalies”. This situation further complicates the classification process, posing challenges to accurately distinguish and identify volcanic thermal activity. Future works are needed to improve this aspect by combining our approach with image segmentation techniques.

The computational advances in deep learning frameworks, coupled with the ever-growing satellite constellation and continuing improvements in sensor technology, present unique opportunities for improving our ability to identify renewed volcanic activity using space-based observations. Moreover, virtual constellations of sensors such as Landsat 8 and Sentinel-2, that are in sun-synchronous orbits with different overpass times, provide the opportunity for more frequent and cloud-free surface observations when used together rather than alone. Combined together, the Landsat 8, Sentinel-2A and Sentinel-2B sensors permit a revisit interval of about 2–3 days [64]. Such frequent revisit times at a spatial resolution of 10 m to 30 m are better suited to capturing thermal changes that could indicate the beginning of renewed volcanic activity. Ultimately, the SqueezeNet model could be trained and used to analyze every image captured by these satellites over any volcano around the world, to automatically retrieve an effective and accurate extraction of thermal features and to alert an operator in cases of the presence of active flows and erupting vents. The results demonstrate the potential applicability of the proposed approach to the development of automated thermal analysis systems at the global scale using future data coming from planned future missions, or by also directly processing the data on board the satellites.