Autonomous Satellite Wildfire Detection Using Hyperspectral Imagery and Neural Networks: A Case Study on Australian Wildfire

Abstract

One of the United Nations (UN) Sustainable Development Goals is climate action (SDG-13), and wildfire is among the catastrophic events that both impact climate change and are aggravated by it. In Australia and other countries, large-scale wildfires have dramatically grown in frequency and size in recent years. These fires threaten the world’s forests and urban woods, cause enormous environmental and property damage, and quite often result in fatalities. As a result of their increasing frequency, there is an ongoing debate over how to handle catastrophic wildfires and mitigate their social, economic, and environmental repercussions. Effective prevention, early warning, and response strategies must be well-planned and carefully coordinated to minimise harmful consequences to people and the environment. Rapid advancements in remote sensing technologies such as ground-based, aerial surveillance vehicle-based, and satellite-based systems have been used for efficient wildfire surveillance. This study focuses on the application of space-borne technology for very accurate fire detection under challenging conditions. Due to the significant advances in artificial intelligence (AI) techniques in recent years, numerous studies have previously been conducted to examine how AI might be applied in various situations. As a result of its special physical and operational requirements, spaceflight has emerged as one of the most challenging application fields. This work contains a feasibility study as well as a model and scenario prototype for a satellite AI system. With the intention of swiftly generating alerts and enabling immediate actions, the detection of wildfires has been studied with reference to the Australian events that occurred in December 2019. Convolutional neural networks (CNNs) were developed, trained, and used from the ground up to detect wildfires while also adjusting their complexity to meet onboard implementation requirements for trusted autonomous satellite operations (TASO). The capability of a 1-dimensional convolution neural network (1-DCNN) to classify wildfires is demonstrated in this research and the results are assessed against those reported in the literature. In order to enable autonomous onboard data processing, various hardware accelerators were considered and evaluated for onboard implementation. The trained model was then implemented in the following: Intel Movidius NCS-2 and Nvidia Jetson Nano and Nvidia Jetson TX2. Using the selected onboard hardware, the developed model was then put into practice and analysis was carried out. The results were positive and in favour of using the technology that has been proposed for onboard data processing to enable TASO on future missions. The findings indicate that data processing onboard can be very beneficial in disaster management and climate change mitigation by facilitating the generation of timely alerts for users and by enabling rapid and appropriate responses.

1. Introduction

In recent years, climate change and other environmental issues associated with human activities have received much attention in the scientific literature [1]. Such issues include extreme weather events [2], droughts [3], sandstorms [4], rising sea levels [5], tornados [6], volcanic eruption [7], and wildfires [8]. Wildfires decimate global and regional ecosystems and cause a lot of damage to structures, injuries, and deaths [9,10]. Due to this, it is becoming more and more important to find fires and keep track of their type, size, and effects over large areas [11]. To try to avoid or lessen these effects, early fire detection and fire risk mapping are used [12]. In the past, wildfires were mostly found by people monitoring wide areas from fire observation towers and using simple devices like the Osborne fire finder [13]. Nevertheless, such methods were not very accurate, and their effectiveness could be affected by human fatigue accumulated during long observation periods. On the other hand, alternative sensors designed to detect gasses, flame, smoke, and heat emissions usually need extended measurement times for molecules to approach the sensors. Additionally, since the range of these sensors is small, wide areas can only be covered using a large number of sensors [14]. Rapid advancements in object recognition, deep learning, and remote sensing have given us new ways to find and track wildfire. New materials and microelectronics have also made it easier for sensors to find active wildfires [15,16].

There are three primary classifications of the extensively used technologies that can identify or observe active fire or smoke conditions in real or near-real-time, namely, terrestrial, aerial, and satellite systems. These technologies are typically incorporated with visible, infrared, multispectral, or hyperspectral sensors; once the data have been collected, they can be processed by applicable artificial intelligence (AI) algorithms, usually a machine learning (ML) methodology. These techniques rely on either extracting hand-crafted features or on robust AI methods in order to detect wildfires in their earliest stages and to simulate how smoke and fires behave [15,17,18]. The different types of fire detection methods are shown in Figure 1.

This research work focuses on satellite-based fire detection by including appropriate AI approaches for onboard computation and analysis. Before introducing our proposed solution, a detailed discussion of the satellite-based wildfire detection approach is provided. There have been numerous research efforts to identify wildfires from satellite imagery in recent years, mostly as a result of the vast number of satellites that have been launched and the drop in the associated costs. Specifically, a constellation of satellites (e.g., Planet Lab) was developed for Earth observation (EO) [19]. Satellites can generally be grouped into different categories based on their orbit, each of which has its own set of advantages and disadvantages.

Table 1. Satellite categories, Adapted from [20].

Orbit	Altitude	Advantages
Geostationary Earth Orbit (GEO)	Circular orbit with an altitude of 35,786 km and zero inclination	The satellite does not move at all relative to the ground, Providing a constant view of the same surface area, High temporal resolution.
Low Earth Orbit (LEO)	Altitude of 2000 km or less	Requires the lowest amount of energy for satellite placement; Provides high bandwidth, Low communication latency.
Sun-Synchronous Orbit (SSO)	Nearly polar orbit that passes the equator at the same local time on every pass. Typical Sun-synchronous Earth orbits are 600–800 km.	Satellite will always observe the same scene with the same angle of illumination coming from the Sun, Has high spatial resolution.

shows the most significant categories.

Currently, remote sensing satellites take photos of the Earth, and the images are downlinked to the ground as soon as the satellite makes contact with the ground station network. From here, images can be loaded into machines that extract various forms of actionable knowledge such as wildfire. Downloading imagery is an $O\left(n^2\right)$ problem, which usually provides significant latency when considering critical operations for extreme events management. If time is of the essence for detecting ignitions and thus speeding up the suppression response, it would be much quicker to have the fire mapping analytics right onboard the satellite and only download the vector data (either point or polygon) of the fire with the data already flagged to be forwarded to the appropriate wildland fire dispatcher (based on location). Having the coordinates of the event would allow for satellite managers, or even the satellite itself, to prioritize the transmission of the imagery associated with the AI-generated wildland fire event. The mission architecture would be even more effective when considering a constellation of satellites properly designed for the management of extreme events. Having artificial intelligence onboard the satellites, we would be able to process the data in real-time, and when a wildfire is spotted from one satellite, it will communicate this information to the other satellites in the constellation, thereby enabling trusted autonomous satellite operation (TASO). The most important part of this is to show that the data can be processed and shared with the help of the AI that is onboard, and that only the information that can be used is downlinked rather than all of the data. Preliminary analyses and the results of a mission concept based on a distributed satellite system for wildfire management are reported in [21].

The majority of low Earth polar SSO orbits of EO satellites have precise altitude and inclination estimations to guarantee that the spacecraft observes the very same scenario every time with same angle of light from the Sun, and that on each pass, the shadows appear the same [22]. The spatial resolution of Sun-synchronous satellite data is high, but the temporal resolution is low (LandSat-7/8 [23] has an eight-day repeat cycle, whereas Sentinel 2A/2B [24] has a two-to-three-day repeat cycle at mid-latitudes), while GEO satellites have lower spatial resolution contrasts with their high temporal resolution. As a result, they are ineffective for detecting active wildfires in real-time; rather, they are better suited for much less time-sensitive tasks such as estimating the burnt area [25]. EO satellite systems have been able to find wildfires because they can see a large area. Most satellites that take pictures of the Earth use multispectral imaging sensors and are either in a geostationary orbit or a Sun-synchronous orbit near the polar regions.

Improvements in nanomaterials and microelectronics have made it possible to use CubeSats, which are small spacecraft that orbit close to the Earth. PhiSat-1 (Φ-Sat-1), launched on 3 September 2020 [26,27,28], is a six-unit (6U) European satellite and is the first to show how transmitting down EO data can be made more efficient using onboard intelligence using AI. It is part of the Federated Satellite System (FSSCat), which is made up of two CubeSats [29,30,31,32] carrying AI technologies. The two CubeSats collect data utilising hyperspectral optical equipment and state-of-the-art dual microwaves. They also test intersatellite communications. One of the CubeSats’ hyperspectral cameras takes many pictures of Earth, some of which are cloudy. The Φ-Sat AI chip filters out erroneous cloudy photos before transmitting them to Earth, sending only the usable data. CubeSats are more cost-effective, are smaller than regular satellites, and require less time to launch than traditional satellites. The detailed classification and their parameters are listed in

Table 2. List of some reference remote sensing satellite systems and their characteristics. Adapted from [15].

(Satellite)-Sensor	Spectral Bands	Access to the Data	Spatial Scale	Spatial Resolution	Specs/Advantages/Limitations	Data Coverage	Accuracy Range
Terra/Aqua-MODIS	36 (0.4–14.4 µm)	Registration Required (NASA)	Global	0.25 km (bands 1–2) 0.5 km (bands 3–7) 1 km (bands 8–36)	Easily accessible, limited spatial resolution, revisit time: 1–2 days	Earth	92.75–98.32%
Himawari-8/9—AHI-8	16 (0.4–13.4 µm)	Registration Required/ (Himawari Cloud)	Regional	0.5 km or 1 km for visible and near-infrared bands and 2 km for infrared bands	Imaging sensors with high radiometric, spectral, and temporal resolution. 10 min (Full disk), revisit time: 5 min for areas in Japan/Australia)	East Asia and Western Pacific	75–99.5%
MSG—SEVIRI	12 (0.4–13.4 µm)	Registration Required (EUMETSAT)	Regional	1 km for the high-resolution visible channel, 3 km for the infrared and the 3 other visible channels	Low noise in the long-wave IR channels, tracking of dust storms in near-real-time, susceptibility of the larger field of view to contamination by cloud and lack of dual-view capability, revisit time: 5–15 min	Atlantic Ocean, Europe and Africa	71.1–98%
GOES-16 and 18	16 (0.4–13.4 µm)	Registration Required (NOAA)	Regional	0.5 km for the 0.64 µm visible channel 1 km for other visible/near-IR 2 km for bands > 2 µm	Infrared resolutions allow the detection of much smaller wildland fires with high temporal resolution but relatively low spatial resolution, and delays in data delivery, revisit time: 5–15 min	Western Hemisphere (North and South America)	94–98%
HuanJing (HJ)-1B—WVC (Wide View CCD Camera)/IRMSS (Infrared Multispectral Scanner)	WVC: 4 (0.43–0.9 µm) IRMSS: 4 (0.75–12.5 µm)	Registration Required	Regional	WVC: 30 m IRMSS: 150–300 m	Lack of an onboard calibration system to track HJ-1 sensors’ on-orbit behaviour throughout the life of the mission, revisit time: 4 days	Asian and Pacific Region	94.45%
POES/MetOp—AVHRR	6 (0.58–12.5 µm)	Registration Required (NOAA)	Global	1.1 km by 4 km at nadir	Coarse spatial resolution, revisit time: 6 h	Earth	99.6%
S-NPP/ NOAA-20/NOAA— VIIRS-375 m	16 M-bands (0.4–12.5 µm) 5 I-bands (0.6–12.4 µm) 1 DNB (0.5–0.9 µm)	Registration Required (NASA)	Global	0.75 km (M-bands) 0.375 km (I-bands) 0.75 km (DNB)	Increased spatial resolution, improved mapping of large fire perimeters, revisit time: 12 h	Earth	89–98.8%
CubeSats (data refer to a specific design from [25])	2: MWIR (3–5 µm) and LWIR (8–12 µm)	Commercial access planned	Global	0.2 km	Small physical size, reduced cost, improved temporal resolution/response time, Revisit time: less than 1 h.	Wide coverage in orbit	-

. Currently, most of the data processes are performed on the ground, but there is significant interest in bringing at least some of the computing efforts onboard satellites. The employment of AI algorithms onboard satellites for analysis and segmentation, classification, cloud masking, and potential risk detection shows the potential of satellite remote sensing.

The European Space Agency (ESA) has been a leader in taking the first steps in this direction with the PhiSat-1 satellite. CNNs for detecting volcanic eruptions using satellite optical/multispectral imaging were proposed in [26], with the main goal of presenting a feasible CNN architecture for onboard computing. P. Xu et al. [33] presented an onboard real-time ship detection based on deep learning and utilising SAR data. Predicting, detecting, and monitoring the occurrence of wildfires obviously benefits officials, civilians, and the ecosystem, with advantages in preparedness, reaction times, and damage control. OroraTech which was created in 2018, already has a range of international customers for its own wildfire service, notably SOPFEU Quebec, Forestry Corporation NSW in Australia, and Arauco in Chile. To offer intelligence to contribute to environmental protection and other properties, the system uses sensor data from a range of existing satellites. OroraTech has launched a thermal infrared (TIR) imager on a Spire 6U CubeSat featuring TIR and optical imaging equipment and Edge AI processing in a first step toward vertical integration.

The goal of our study was to see whether AI approaches and onboard computing resources can be used to monitor hazardous events such as wildfire detections by utilising optical/multispectral/hyperspectral satellite imagery. This kind of analyses could be useful to inform Design, Development, Test and Evaluation (DDT&E) activities of future satellite missions such as the ESA Phisat 2 program. In this study, hyperspectral images taken from the PRISMA (PRecursore IperSpettrale della Missione Applicativa) satellite and the following contributions were made:

• A one-dimensional (1D) CNN for detecting wildfires using PRISMA hyperspectral imagery was considered, and promising results are shown for the edge implementation on three different hardware accelerators (i.e., computer hardware designed to perform specific functions more efficiently when compared to software running on a general-purpose central processing unit).
• We demonstrate that AI-on-the-edge paradigms are feasible for future mission concepts using appropriate CNN architectures and mature astrionics technologies to perform time- and power-efficient inferences.

The proposed CNN is described in terms of the constraints imposed by the onboard implementation, meaning that the initial network has been streamlined and adjusted to comply with the intended hardware designs. It is worth noting that the detection of wildfires should be considered as the example test case, and that the proposed methodology (or similar ones) can be successfully applied to other scenarios or tasks, as already discussed and demonstrated in other works [26].

The rest of this article is organised as follows: after the Introduction, Section 2 discusses the methods and analysis, beginning with an overview of the wildfires, followed by information that is more in-depth regarding the description of the study area, and subsequently, information regarding the PRISMA data and the definition of the dataset. The onboard implementation and a description of hardware accelerators are covered in Section 3. The results are covered in Section 4, and the findings and their applicability are covered in depth in Section 5, which is followed by our conclusions in Section 6.

2. Current Detection Methods

A wildfire is a dynamic phenomenon that changes its behaviour over time. The spread of fire is aided by the presence of forest fuel and is carried out by a series of intricate heat transmission and thermochemical processes that control fire behaviour [34]. There are several mathematical models created to characterise wildfire behaviour; each model was built based on the diverse wildfire experiences in various nations. According to the input and environmental parameters, each model differs from the others (fuel indexing [35,36]). Some researchers have been able to incorporate some of these models into simulation programs, or even develop their own ways for mapping the terrain and fire behaviour on monitoring screens for the study and prediction of fire behaviour [37]. The form of a wildfire burning in a steady environment is an ellipse [38]. The environment can change over time, and different portions of a fire may burn in different environments such as humidity levels, wind speed, wind direction, slope, and so on. The heterogeneity of the environment could result in a very complicated fire form [35,39].

F. Tedim et al. [38] made an initial attempt to develop a gravity scale for wildfires that was comparable to the scales used for hurricanes (Saffir–Simpson scale) and tornadoes (Fujita scale). The first four categories are labelled as “normal fires,” or incidents that can generally be put out within the bounds of technology and physical limits. Based on the assessments of recent extreme wildfire incidents and a consolidation of the literature, the three remaining categories are grouped as extreme wildfire events (EWEs; see

Table 3. Classification of wildfires based on fire behaviour and the capacity of control. Adapted from [38].

Fire Category		Real-Time Measurable Behaviour Parameters			Real-Time Observable Manifestations of Extreme Fire Behaviour ((EFB)				Type of Fire and Capacity of Control
		Fireline Intensities (FLI) (kWm−1)	Rate Of Spread (ROS) (m/min)	Flame Length (FL) (m)	Pyrocumulonimbus (PyroCb)	Downdrafts	Spotting Activity	Spotting Distance (m)
Normal Fires	1	<500	<5 a <15 b	<1.5	Absent	Absent	Absent	0	Surface fire Fairly easy
	2	500–2000	<15 a <30 b	<2.5	Absent	Absent	Low	<100	Surface fire Moderately difficult
	3	2000–4000	<20 c <50 d	2.5–3.5	Absent	Absent	High	≥100	Surface fire, torching possible Very difficult
	4	4000–10,000	<50 c <100 d	3.5–10	Unlikely	In some localised cases	Prolific	500–1000	Surface fire, crowning likely depending on vegetation type and stand structure Extremely difficult
Extreme Wildfire Events	5	10,000–30,000	<150 c <250 d	10–50	Possible	Present	Prolific	>1000	Crown fire, either wind- or plume-driven Spotting plays a relevant role in fire growth Possible fire breaching across an extended obstacle to local spread Chaotic and unpredictable fire spread Virtually impossible
	6	30,000–100,000	<300	50–100	Probable	Present	Massive Spotting	>2000	Plume-driven, highly turbulent fire Chaotic and unpredictable fire spread Spotting, including long distance, plays a relevant role in fire growth Possible fire breaching across an extended obstacle to local spread Impossible
	7	>100,000 (possible)	>300 (possible)	>100 (possible)	Present	Present	Massive Spotting	>5000	Plume-driven, highly turbulent fire Area-wide ignition and firestorm development non-organised flame fronts because of extreme turbulence/vorticity and massive spotting Impossible

Note: ^a Forest and shrubland; ^b Grassland; ^c Forest; ^d Shrubland and grassland.

) [38].

Table 4. Mos relevant wildfires that took place in Australia from 2007 to 2021 [42,43,44,45].

Year	Event Name	Affected Area	Burned Area (Approx. Acres)
1 June 2020–1 June 2021	2020–2021 Australian wildfire seasons	Nationwide	617,763
5 September 2019–2 March 2020	2019–2020 Australian wildfire season (Black Summer)	Nationwide	46,030,000
February 2019	Tingha wildfire	New South Wales	57,870
11–14 February 2017	2017 New South Wales wildfires	New South Wales	130,000
January 2016	2016 Murray Road wildfire (Waroona and Harvey)	Western Australia	170,910
25 November–2 December 2015	2015 Pinery wildfire	South Australia	210,000
15–24 November 2015	Perth Hills wildfire complex–Solus Group	Western Australia	24,750
October–November 2015	2015 Esperance wildfires	Western Australia	490,000
29 January–20 February 2015	2015 O’Sullivan wildfire (Northcliffe–Windy Harbour)	Western Australia	244,440
2–9 January 2015	2015 Sampson Flat wildfires	South Australia	49,000
January 2015	2015 Lower Hotham wildfire (Boddington)	Western Australia	129,420
1 August–9 August 2015	2015 Wentworth Falls Winter Fire	New South Wales	2,000
17–28 October 2013	2013 New South Wales wildfires	New South Wales	250,000
18 January 2013	Warrumbungle wildfire	New South Wales	130,000
4 January 2013	Tasmanian wildfires	Tasmania	49,000
27 December 2011–3 February 2012	Carnarvon wildfire complex	Western Australia	2,000,000
7 February–14 March 2009	Black Saturday wildfires	Victoria	1,100,000
30 December 2007	Boorabbin National Park	Western Australia	99,000

shows a list with the most recent and significant wildfire incidents in Australia from 2007 until today. Some of the fires may have been caused by natural disasters or may have been caused directly or indirectly by human recklessness and environmental misuse (particularly the rise in temperature linked to global warming). One of the worst wildfires (Black Saturday wildfires) in Australian history ravaged Victoria. Many people were killed or injured in the wildfire, which ravaged many towns and cities, destroying homes, businesses, schools, and kindergartens [40,41]. From Table 4, it is evident that wildfire events are happening regularly. Since wildfires occur on a regular basis, there is a clear need for wildfire detection. To address this, the recent Australian wildfires were investigated, and an analysis was carried out. The designated area of interest (AOI) is located around 250 km north of Sydney in Ben Halls Gap National Park (BHGNP), which comprises 2500 hectares and is 60 kilometres south of Tamworth and 10 kilometres from Nundle. Because the park is located at a high elevation, it receives a lot of rain and has cool temperatures. However, in late 2019, a combination of high temperatures and wind speeds as well as low relative humidity created the conditions for high-intensity wildfire behaviour to develop. There are different levels of data available, and the differences are reported in Figure 2. As can be observed in the PRISMA image (Figure 3) acquired on 27 December 2019, active wildfires can be spotted across this AOI.

3. PRISMA Mission

A scientific and demonstration mission called PRISMA was launched aboard the VEGA rocket on 22 March 2019. The research based on the HyperSpectral Earth Observer (HypSEO) project [47], which was a product of a partnership between ASI and the Canadian Space Agency, served as the foundation for the early conceptual studies. Due to its ability to capture data globally with a very high spectral resolution and in a wide variety of spectral wavelengths, PRISMA plays an important role in the current and future international setting of EO for both the scientific community and for end users. PRISMA offers the ability to collect, downlink, and preserve the imagery of all panchromatic/hyperspectral channels totalling 200,000 km² daily practically on the entire global region, obtaining 30 km by 30 km square Earth tiles. There are two operational modes for the PRISMA mission: a primary mode as well as a secondary mode. The main method of operation is the gathering of panchromatic and hyperspectral data from specified individual targets as demanded by the end users. The mission will have a setup of continual “background” work in the auxiliary mode of operation that will collect imagery to fully utilise the system’s resources.

One modest class spacecraft makes up the PRISMA space segment. A hyperspectral/panchromatic camera featuring VNIR and SWIR detectors is part of the PRISMA payload. It consists of a medium resolution panchromatic camera (PAN, from 400 nm to 700 nm) with a 5 m resolution and an imaging spectrometer with a 30 m spatial resolution that can acquire in a continuum of spectral bands from 400 nm to 2505 nm (i.e., from 400 nm to 700 nm in VNIR and from 920 nm to 2505 nm in SWIR). The PRISMA hyperspectral sensor makes use of the prism to measure the incoming radiation’s dispersion on 2-D matrix detectors in order to collect many spectral bands from the same ground strip. The 2-D detectors immediately provide the “instantaneous” spectral and spatial dimensions (across track) of the spectral cube, while the satellite motion (pushbroom scanning concept) provides the “temporal” dimension (along track). The following references [48,49,50] contain some works on wildfires using PRISMA.

PRISMA data are made freely accessible for research purposes by the Italian Space Agency (ASI) [51]. In Hierarchical Data Format version 5 (HDF5), the 30 m and 5 m resolution hyperspectral and panchromatic data are given with four choices:

• Level 1, radiometrically corrected and calibrated top of atmosphere (TOA) data.
• Level 2B, Geolocated at-ground spectral radiance product.
• Level 2C, Geolocated at-surface reflectance product.
• Level 2D, Geocoded version of the Level 2C product.

The analysis in this paper was conducted with Level 2D data. Preliminary, direct information can be retrieved by looking at single bands. For instance, smoke can be recognized by looking at the very near infrared (VNIR) bands of the L2D data, while active wildfires can be retrieved by looking at saturated pixels in the short-wave infrared (SWIR) bands. Indeed, in the interval of 2000–2400 nm, the signal easily saturates when looking at active wildfires, as the signal captured from Earth is greater than the signal coming from the Sun (since the wildfires behave as active power emitters). However, analysing the entire spectral signature by means of the convolutional neural network allows us to avoid errors and increase the reliability of the classification.

3.1. Dataset Definition

The AI approach was used to implement automatic segmentation from the obtained image. From Figure 3, three active wildfires can be observed. The southern and the north-east wildfires are the bigger ones, whereas the north-west wildfire is quite small. For the training and validation, the reference pixels must first be labelled. The reference pixels used in this investigation were manually labelled and are shown in Figure 3. The number of labelled pixels selected from the PRISMA image (after investigation of the spectra and looking at the false colour composites) is reported in

Table 5. Number of labelled reference pixels in Australia used for training and testing the CNN [52].

		Pixels per Classes
Wildfire Location	Usage	0 Fire	1 Smoke	2 Burned Areas	3 Vegetation	4 Bare Soil	Total
North-East	Train & Val	58	10	30	50	40	188
South	Test	11	11	9	10	10	51
North-West	Test	5	0	5	5	5	20

. The north-east wildfire was used as the training and validation dataset, while the south and north-west datasets were used as the test datasets. The training set accounted for 70% of the labelled data of the north-east wildfire, while the remaining 30% was chosen for validation. In this research, the training was carried out by utilising the computational capabilities of the ground (i.e., a personal computer equipped with Nvidia RTX2060).

3.2. Automatic Classification with a 1D CNN Approach

The categorisation model utilised in this study was inspired by the model by Hu et al. [53], which is depicted in Figure 4. The input pixel spectrum of the PRISMA data includes the SWIR and VNIR channels. Thus, it is an array with $C=234$ elements (after removal of some useless original data in the input hyper-cube). A 1-D convolutional layer with a kernel of 3, $n_1=112$ filter, the same padding, ReLU activation function, and $l2$ kernel regulariser is the first hidden layer. After the convolutional layer, there is a max pooling layer with a pool size of 2 and a stride of 2 (notice that $n_2=n_1$ in Figure 4). The result of this max pooling is then sent through a flattening layer before being connected to a 128-unit fully connected layer with ReLU activation. The last layer is a dense unit with the SoftMax activation function for multiclass classification. It is worth noting that the values of $C_1$ and $C_2$ in the diagram are easily evaluable and rely on the network’s architecture. The Adam optimiser and the categorical cross-entropy loss function were used to train the model. Python and Keras were used to build the entire network [17,18,54].

4. Astrionics Implementation

The ultimate aim is to build a model that can be uploaded to an onboard astrionics system, so the network complexity, parameter count, and inference execution time must all be optimised. Due to the chip’s restricted elaboration power, the utilisation of a small chip limits the ability to execute the specific classification model, necessitating the development of an accurate model. A prototype for executing the analysis was created in order to evaluate the proposed methodology. To begin, the model was modified to work with the chosen hardware and to detect wildfires onboard.

A significant component of the architecture of many current AI solutions is cloud computing or storage. Several sectors are finding it challenging to apply the technology for real-world use cases due to worries regarding confidentiality, latency, dependability, and bandwidth. Despite its resource restrictions, edge computing can somewhat help to ease these difficulties. The claim that edge and cloud computing are incompatible is untrue; edge computing actually enhances cloud computing. Inflated expectations for edge AI and edge analytics have peaked, according to the Gartner hype cycles for 2019 and 2018 [55]. Although the sector is still in its infancy, software frameworks and hardware platforms will advance with time to deliver value at a reasonable price. Three important AI industry leaders—Intel, Google, and Nvidia—are supporting edge AI by offering hardware platforms and accelerators with compact form dimensions. Although each of the three has benefits and drawbacks, it all depends on the application, budget, and amount of experience that is available.

Table 6. Edge AI hardware comparison [55].

Parameters	Nvidia Jetson Nano	Google Coral USB	Intel Movidius NCS
Inference time	~38 ms	~70–92.32 ms	~225–227 ms
fps	~25	~9–7	~4.43–4.39
CPU usage	47–50%	135%	87–90%
Memory usage	32 %	8.7%	~7%
OS	Ubuntu 18.04 aarch64	Raspbian GNU/License 10 (Buster)	Raspbian GNU/License 9 (Stretch)

presents the comparison of the hardware accelerators [55].

The Intel Movidius Neural Compute Stick (NCS) is a high-performance, affordable USB stick that may be used to implement deep learning inference applications, according to the comparison above. Great AI solutions are provided by the Google Edge TPU. The NVIDIA Jetson Nano, in conclusion, crams a lot of AI power into a little package. We used the Intel Movidius stick and two Nvidia variants, the Jetson and TX2, for our research [56].

4.1. Description of the hardware accelerators

In this section, we provide a description of the three selected astrionics hardware components, with specific reference to the accelerators (i.e., the Movidius Stick in section b, the Jetson Nano in section c, and Jetson TX2 in section d).

4.2. Movidius Stick

The Intel Movidius NCS is a compact fanless deep learning USB drive that is intended to be used for learning AI programming techniques. The Movidius Visual Processing Unit, which is minimal in power but high in performance, drives the stick. It is equipped with an Intel Movidius Myriad 2 Vision Processing Unit. These are the main specifications [26]:

• Supporting CNN profiling, prototyping and tuning workflow;
• Real-time on-device inference (Cloud connectivity not required);
• Features the Movidius Vision Processing Unit with energy-efficient CNN processing;
• All data and power provided over a single USB type-A port;
• Run multiple devices on the same platform to scale performance.

The workflow for executing the software modules on the hardware system is depicted in Figure 5. Prior to running the experiments on the Movidius Stick, the CNN must be translated from its original format (for example, the Keras model) to an OpenVino format, which may be accomplished through the use of the OpenVino library. Because of the Movidius Stick, deep learning coprocessors that are inserted into the USB socket can be inferred more quickly than before. Before transferring the CNN onto the Movidius, it is necessary to optimise the network, which may be accomplished by utilising the OpenVino Intel’s hardware-optimised computer vision library. Intel Distribution, the OpenVino toolkit, is extremely easy to use and is included with the Intel processors. Indeed, once the target CPU has been determined, the OpenCV optimised for OpenVino can handle the rest of the setup. The toolkit supports heterogeneous execution across computer vision accelerators (such as GPU, CPU, and FPGA) using a standard API, in addition to enabling deep learning inference at the edge. It also decreases the time to market by utilising a library of functions and preoptimised kernels, and it includes optimised calls for OpenCV.

Following the model’s implementation on Movidius, it was tested against the PRISMA hyperspectral images. Using the same settings used for the training and validation datasets, the image was processed. Figure 6 depicts the high-level block diagram for the installation and optimisation of OpenVino. The internal data structure or program that a compiler or virtual machine uses to represent source code is known as an intermediate representation (IR).

An IR is made to facilitate additional processing such as optimisation and translation. The model is fed to the Model Optimiser before being delivered to IR, from which we can obtain the .xml and .bin files required to execute OpenVino. The weight and biases are saved in binary form in a .bin file, while the standardised architectural arrangement (and other metadata) is stored in a .xml file, as shown in Figure 7. Model optimisation was performed on a Windows machine and the IR model was implemented on a Windows system with NCS2, as depicted in Figure 8.

4.3. Jetson Nano

The recently released JetPack offers a complete desktop Linux environment for Jetson Nano that is built on Ubuntu. Major open-source frameworks such as TensorFlow, MXNet, Keras, Caffe, PyTorch, and the SDK enable the native installation of frameworks for robotics and computer vision development like OpenCV. Thanks to complete interoperability with all of these frameworks and NVIDIA’s high Calibre platform, it has become simpler than ever before to deploy AI-based inference applications to Jetson. The Jetson Nano makes real-time computer vision and inference possible for a wide range of intricate deep neural network (DNN) models. Advanced AI systems, IoT devices with intelligent edge analytics, and multi-sensor autonomous robots are made possible by these capabilities. Using the ML frameworks, it is even possible to carry out transfer learning while retraining networks locally on the Jetson Nano [55,57,58,59,60]. This procedure is shown in Figure 9, and the implementation procedure is the same as the Intel Movidius Stick.

4.4. Jetson TX2

The Nvidia Jetson series of embedded platforms provides edge devices with server-class AI computation capabilities. When it comes to deep learning inference, Jetson TX2 is twice as energy efficient as its precursor, Jetson TX1, and it performs better than the Xenon server CPU built by Intel. This increase in productivity reframes the potential for moving advanced AI from the cloud to the edge. With support from long short-term memory networks (LSTMs), recurrent neural networks (RNNs), TensorRT libraries, and the NVIDIACUDA Deep Neural Network library (cuDNN), Jetson TX2 accelerates cutting-edge deep neural network (DNN) designs. The conversion of the Keras model to the TensorFlow model is represented in the deployment process in Figure 10, which illustrates this by simply using TensorFlow and TensorFlow TFT integration to show the important distinction. To run NN inferences on their hardware, Nvidia developed the TensorRT NN framework, and the implementation procedure is the same as the Intel Movidius Stick. TensorRT is highly performance-optimised on NVIDIA GPUs. Currently, it is probably the quickest method for running models. NVIDIA’s TensorRT inference acceleration library enables the utilisation of NVIDIA GPU resources to the fullest extent possible at the cutting edge [60,61,62].

5. Results

The results of the classification are shown by means of the segmentation map reported in Figure 11. The classified areas look pretty well-defined, with a very low level of noise.

Table 7. Validation dataset accuracy.

	Precision	Recall	F1 Score
0—Fire	1.00	1.00	1.00
1—Smoke	1.00	1.00	1.00
2—Burned	1.00	1.00	1.00
3—Vegetation	0.92	1.00	0.96
4—Bare soil	1.00	0.92	0.96
Accuracy			0.98
Macro average	0.98	0.98	0.98
Weighted average	0.98	0.98	0.98

summarises the outcomes of the training section conducted over the southern wildfire. Using the validation dataset, the final overall accuracy of the model was 97.83 percent, which was marginally higher than the 96.87 percent reported in Amici et al. [63], where a support vector machine (SVM) was employed to achieve the accuracy.

Only the inference problem is considered when evaluating machine learning implementation on hardware accelerators. This indicates that the training is carried out on a computer with advanced capabilities capable of handling the large amount of data needed for the training. As a result, the training in this paper was in the ground with sophisticated computing capabilities (i.e., a computer with an Nvidia RTX2060), and we describe the findings acquired from the authors’ earlier work [18] in

Table 8. In the three areas indicated, the precision, recall, and F1 scores were calculated. The dataset from north-east Australia was utilised for training, while the others were used for testing.

Wildfire Location	Precision	Recall	F1 Score
Australia, North-East	0.98	0.98	0.98
Australia, South	0.98	0.98	0.98
Australia, North-West	1.00	0.95	0.97

.

The results that were obtained by deploying the CNN into the three hardware accelerators revealed that the performances that were stated in Table 8 were not affected by the deployment of the CNN. As a result, this section describes the deployment performance in terms of the inference time and the amount of power consumed.

• Results on the Movidius: The results of the deployment on the Movidius indicate that the accuracy did not vary in comparison to the values that were presented in

  Table 9. Inference time and power consumption on the three hardware accelerators.

  HW Accelerator	Inference Time (ms)	Power Consumption (W)
  Movidius	5.8	1.4
  Jetson TX2	3.0	4.8 (2.1 GPU only)
  Jetson Nano	3.4	2.6 (2.0 GPU only)

  . At the same time, the inference time was approximately 5.8 milliseconds, and the computing power was 1.4 watts on average.
• Results on the Jetson TX2: The results of the deployment on the Jetson TX2 revealed that the accuracy did not change in comparison to the values that were reported in Table 9. On the other hand, the inference time was approximately 3.0 milliseconds, and the computational power was 4.8 W on average (2.1 W if considering the power consumed by the GPU only). It is important to note that these results are related to the TX2 setup that provided the least inference time and the maximum power consumption. Other configurations can be set up to lower the amount of power that is consumed, so it is important to keep this in mind (and increase the inference time).
• Results on the Jetson Nano: The findings of the deployment on the Jetson Nano demonstrate that the accuracy did not change compared to the values reported in Table 9. On the other hand, the inference time was approximately 3.4 milliseconds, and the computational power was 2.6 watts on average (2.0 W if considering the power consumed by the GPU only). Concerning the Jetson TX2, these findings are associated with the configuration that offers the quickest possible inference time at the expense of the highest possible power consumption.

6. Discussion

In light of the findings discussed above, the following is worth further investigation. The deployment of the hardware accelerators in all of the reported studies used final models with weights given in float 32. As a result, the precision, recall, and F1 results remained unchanged from those achieved during the PC training and testing method. On one hand, this result was possible due to the small dimension of the CNN model, and further weight quantisation was not required (i.e., results in inference time and power consumption were already consistent with the expected and/or required values without additional weights quantisation). Nevertheless, it should be noted that if the model needs to be improved further in terms of weight compression, embedding, or quantisation, the classification performance may suffer. In any case, this investigation’s findings show that the weights’ data format does not need to be changed for the proposed application, and the classification performances were nearly identical while using a PC or a hardware accelerator.

Table 9 shows that the inference timings are perfectly consistent with a real-time early detection service. It should be noted that this time only pertains to the CNN model’s inference time and does not include the pre-processing of the image or the extraction of the spectral signature for the pixel of interest. The inference time reported in Table 9 refers to the analysis of a single pixel. However, it is noteworthy that there is no need to run the classification for all pixels of all detected images. Indeed, a pre-processing based on the saturation of SWIR channels or the usage of specialized indices such as the hyperspectral fire detection index (HFDI) already discussed in [64] can provide a list of candidate pixels where an active fire could be present. Then, the inference can be run on a small neighbourhood of those pixels, so that the analysis can be performed in a few seconds, at most (the reader should note that the acquisition of a scene, considering a multi- or hyper-spectral payload, generally requires few seconds—for PRISMA, it is 4 s). However, these kinds of analyses should be performed along with several other considerations impacting the whole design of the satellite platform. The contribution of this paper is the demonstration of the feasibility of the design of these new mission concepts.

Table 9 also reports the power usage that is generally in line with space missions. When it comes to large platforms like PRISMA, all of the reported solutions are in line with the platform’s total power budget; however, when it comes to CubeSats or small satellites, the Movidius Stick and the Jetson Nano appear to be the most promising options. The maximum power budgets for 1U, 2U, and 3U CubeSats typically fall within the following ranges: 1 to 2.5 Watts, 2 to 5 Watts, and 7 to 20 Watts, accordingly [65]. Therefore, the Jetson Nano and the Intel Movidius are more appropriate for the performance of a CubeSat when taking into consideration the power budget. It is worthy to note that in other works such as in [17], the authors investigated the preliminary generalization performances of the model on other areas of the world with promising outcomes.

As a review of the preceding arguments, this study highlights the potential for future missions to include onboard hardware accelerators to provide early-warning services [66]. We established that this is possible when considering the wildfire analysis that was investigated in this study. Even though we did not take into consideration other important elements impacting on time and power (such as image pre-processing), our results demonstrate the feasibility of the proposed approach. Moreover, if the input data and model complexity are consistent with the ones mentioned here, these conclusions could be applied to other image processing jobs. The comparison of the three hardware solutions revealed that the Jetson Nano is the most promising technology as it offers the best combination of power consumption and inference time (even though the final choice may be influenced by other factors such as the hardware accelerator’s compatibility with the onboard computer, mechanical, and/or electrical interfaces, overall platform dimensions and characteristics, and so on). Furthermore, it is important to remind the reader that this comparison of accelerator technologies is far from complete, as additional boards exist that were not examined in this work for the sake of simplicity (for instance, the Google Coral TPU or FPGA system-on-chips). This work, on the other hand, answers the question of whether or not AI can be used to handle complex data like in hyper-spectral photography, indicating that the current technology is ready and efficient.

DSSs are a relatively new entrant in the realm of small satellite missions. This innovation boosts the value of the mission and paves the way for a variety of new applications. DSSs, or distributed satellite systems, are mission-oriented systems that consist of two or more satellites or modules cooperating with one another to fulfil goals that cannot be accomplished with a single satellite. In recent years, the application of DSS mission architecture has become increasingly widespread. Due to recent advancements in satellite technology and the ability to mass create low-cost small satellites in huge quantities, there has been a resurgence of interest in DSSs such as LEO satellite constellations. The results of the research show that edge computing in space is plausible. Furthermore the findings are promising to enable the TASO in space with a single satellite. Additionally, the same can be implemented and tested for distributed satellite systems. With the use of the Inter Satellite Link (ISL), which connects the satellites that make up the DSS, the system transforms into an intelligent-DSS (i-DSS), which allows for the data to be shared, analysed, and can be employed to provide real-time or near-real-time wildfire management [21,67].

7. Conclusions

The goal of this paper was to examine the performance of hardware accelerators for the edge computing of wildfires for real-time alerts using convolutional neural network models and hyperspectral data analysis. The analysis of the wildfires that occurred in Australia served as a practical example, and the data from PRISMA was utilised in the investigation. Considering three distinct hard-ware accelerators—the Intel Movidius Myriad 2, the Nvidia Jetson Nano, and the Nvidia Jetson TX2—showed that the onboard application is possible in terms of both the inference time and power consumption. These accelerators were employed to show that the onboard application is practicable. In line with other earlier works in the literature, this paper suggests the opportunity to investigate hardware accelerators for onboard edge computing in upcoming space missions in order to improve the services, better manage the entire space-to-ground data-flow, start providing real-time information, and enabling the trusted autonomous satellite operation (TASO), which could be really important in the event of disasters and extreme event management. Other accelerators will be evaluated in future research including field programmable gate arrays (FPGA) and the Google Coral Tensor Processing Unit (TPU). The proposed method will be evaluated in the distributed satellite system (DSS) for real-time disaster management and will increase the area of interest (AOI) coverage and decrease the revisit time. In the future, the methodology will be examined for its applicability to other types of Earth observation (EO) missions, and hardware-in-the-loop experiments will be conducted to demonstrate its effectiveness. Furthermore, transfer learning will be introduced in order to generalise the proposed model for the detection of wildfires and to study its possible applicability to other EO/disaster-relief missions.