MEC: A Mesoscale Events Classifier for Oceanographic Imagery

Abstract

The observation of the sea through remote sensing technologies plays a fundamentalan role in understanding the state of health of marine fauna species and their behaviour. Mesoscale phenomena, such as upwelling, countercurrents, and filaments, are essential processes to be analysed because their occurrence involves, among other things, variations in the density of nutrients, which, in turn, influence the biological parameters of the habitat. Indeed, there is a connection between the biogeochemical and physical processes that occur within a biological system and the variations observed in its faunal populations. This paper concerns the proposal of an automatic classification system, namely the Mesoscale Events Classifier, dedicated to the recognition of marine mesoscale events. The proposed system is devoted to the study of these phenomena through the analysis of sea surface temperature images captured by satellite missions, such as EUMETSAT’s Metop and NASA’s Earth Observing System programmes. The classification of these images is obtained through (i) a preprocessing stage with the goal to provide a simultaneous representation of the spatial and temporal properties of the data and enhance the salient features of the sought phenomena, (ii) the extraction of temporal and spatial characteristics from the data and, finally, (iii) the application of a set of rules to discriminate between different observed scenarios. The results presented in this work were obtained by applying the proposed approach to images acquired in the southwestern region of the Iberian peninsula.

1. Introduction

The impact of climate change on marine ecosystems is often expressed by simplified warming trends [1]. Although this approximation may be valid for oceanic regions, in coastal areas the impact of warming on the ecosystems is far from being homogeneous. This is mainly due to the fact that coastal regions host some of the most biodiverse and variable environments of the ocean.

Near the coast, global drivers are modified by topography and by local atmospheric and oceanographic circulation patterns, including upwelling. Ekman dynamics and large-scale thermocline processes control the coastal upwelling occurring at the eastern boundary upwelling ecosystems (EBUEs) [2,3]. Winds directed towards the Equator drive upwelling which transports deeper, colder, and nutrient-rich waters to the surface, where phytoplankton production is triggered by sunlight [4]. As a result, these areas host the most productive ecosystems in the global ocean [5], playing a major role in the marine primary production and the worldwide fisheries (7% of global marine production and more than 20% of global fish catches), thus providing benefits and means of subsistence for a large percentage of the human population [6].

Upwelling filaments, defined as cold-water tongues with their source in the upwelling zone and extending 50 $\mathrm{k}$$\mathrm{m}$ of width and up to 300 $\mathrm{k}$$\mathrm{m}$ of length [7] have been identified as an effective mechanism of seaward transport from coastal regions to the oligotrophic offshore waters [8]. The significance of this seaward transport of chlorophyll and suspended particulate organic matter has been demonstrated in several studies [9,10,11,12]. Apart from the nutrient load, it was recently shown that upwelled water’s low long-term warming rates may provide thermal refugia, stabilize changes in species distributions, and enhance local biodiversity [13].

According to the related literature, more than 71% of coastal zones are experiencing a net heat gain due to global warming [14]. Yet, it is difficult to systematize the trends observed in different upwelling ecosystems across the global oceans, as positive trends were observed in the coastal areas of Benguela, Peru, northern California, and Canary, while significant negative trends were found along Chile, Somalia, and southern and central California coasts [15]. Therefore, it is surmised that every upwelling ecosystem reacts differently to the changing climate.

Among the world’s EBUEs, the Iberia/Canary Current System (ICCS) is one of the least studied [16]. Several research studies have focused on the western Iberian oceanography [17]. Despite a general circulation similar to other EBUEs, in ICCS the discontinuity imposed by the Gulf of Cádiz and the entrance to the Mediterranean Sea, combined with the seasonality of the large-scale atmospheric circulation, have a profound impact on the regional oceanography. Time scales of a few tens of days explain more than 70% of the variability of the coastal alongshore wind stress, a major factor governing the regional coastal circulation [18]. The region’s continental shelf, with less than 10 $\mathrm{k}$$\mathrm{m}$ wide south of Lisbon, 30–40 km wide off central Portugal and somewhat narrower again off northern Portugal and Galicia, is characterized by a large number of topographical features, such as prominent capes, promontories, and submarine canyons, whose spatial scales are tens to hundreds of kilometers [17]. All the above highlight the importance of sub-seasonal temporal scales and sub-basin spatial scales, which explain the observed oceanographic patterns. In [17], the physical oceanography of the western Iberia system is described and characterized through the main mesoscale features related to that region. They include a succession of mesoscale structures, such as jets, meanders, ubiquitous eddies, upwelling filaments, and countercurrents, superimposed on the more stable variations at seasonal timescales.

1.1. State of the Art

The automatic identification and segmentation of ocean upwelling regions in EBUEs is a topic extensively tackled in previous literature. Traditionally, experts have identified and catalogued upwelling features by subjectively analysing sea-surface temperature (SST) maps of the area of interest. Upwelling indices have been exploited to provide a first guess directing the experts towards the dates and events to be analysed [19], but a visual inspection has always been needed to ascertain the presence of upwelling and to describe it. This procedure is manageable if tens or even hundreds of images are used, but it turns into an unfeasible task as the number of scenes approaches the thousands of images, a typical order of magnitude when the purpose is to investigate climate-related changes. Nowadays, with the growing amount of remote sensing observations, automated techniques have been gaining momentum.

A common issue within this context is the accurate detection of the oceanic front, i.e., the narrow boundary separating the offshore waters from the upwelling area, located close to the shoreline. Among the considered approaches two primary schools of thought emerge: a classical one, exploiting standard computer vision tools, and a more contemporary one, based on machine learning tools, such as convolutional neural networks.

Among the techniques belonging to the first group it is worth mentioning those devoted to the clustering of texture-related features, such as those based on the two-dimensional wavelet transform [20,21] or gray-level co-occurrence matrices and derived texture measurements [22,23]. Along these lines, in [24], the authors present an algorithm to group SST pixels based on a seed region growing approach. The algorithm performs the grouping of pixels in clusters, with the number of clusters not fixed a priori. Following an iterative procedure, each cluster is enlarged by including novel candidates that fulfil a similarity constraint. The process stops as soon as the similarity condition no longer holds. In [25], the authors propose a segmentation method based on a fuzzy-C-means clustering approach, that enables to identify the upwelling area in front of the northwestern African coast, through the analysis of the related SST imagery. In this case, the proposed novelty consists in the introduction of a pre-processing stage where the signal is previously normalized in order to reduce the effect of a non-negligible temperature offset, which varies with the latitude. The cancellation of this spurious trend allows to obtain an accurate separation between offshore and upwelling areas. Alternative approaches address mathematical morphology operators and mobile windows [26,27]. Different approaches take advantage of the amplitude of the SST gradient across the upwelling boundaries, as in [28,29,30]. In [31] the authors describe a modified version of the Canny edge detector [32], implemented to detect the oceanic front boundary. The proposed method has been tested on input imagery that is not provided by real remote sensing campaigns but is actually returned as the output of forecasting models [33,34], in the specific case study related to the South China Sea area.

The recent progresses in the implementation of neural networks architectures encouraged the proposal of novel solutions to the upwelling identification problem, based on a machine learning approach. In [35], the authors exploit neural networks-based algorithms to identify the oceanic boundary and segment the upwelling image subset, while in [36] the authors exploit the currently trending Deep Neural Network (DNN) approach using an adapted version of the GoogleNet network [37]. In the context of remote sensed data, DNNs proved to be useful also to identify other mesoscale oceanic phenomena, such as eddies. For this purpose, the classification has been performed by adapting image processing methods to satellite altimetry data [38]. Although this topic can be connected to the ones introduced in our approach, there are a lot of differences between both the datasets involved and the outcomes of the classification processes: in eddies classification through DNN [39,40], the used data are typically sea-surface height (SSH) data, and the outcomes are edges identifying ocean fronts.

1.2. Proposed Approach

This work introduces Mesoscale Events Classifier (MEC), a new automatic method capable of accepting massive datasets of oceanographic SST imagery as input and returning the classified images as output. It is suitable to point out that the objective pursued through this method slightly differs from the previous approaches: the aim here is not to replicate the implementation of another tool to optimally separate upwelling and non-upwelling areas, but to analyse systematically and in a broader perspective the several mesoscale phenomena that occur within the observed EBUE and to identify and accordingly label the dominant occurring configurations of the system. The output classification labels reflect the different regimes of observable upwelling patterns. The identification of a specific temperature pattern is based on the extraction of quantitative features from the SST maps. Particular attention is devoted to those features that reflect the signal variability (e.g., gradient-based indicators). Indeed the emergence of a certain pattern is usually highly correlated with peculiarities in the temperature spatial arrangement at time fixed (e.g., the presence of abrupt variations in the temperature values within a certain neighbourhood), as well as with the observation of specific temperature trends at fixed locations, providing insights about the flowing of water masses between points at different temperature values. In a previous work devoted to this topic [41], a dedicated tool was developed to extract and visualise the time series of the SST signals related to a given number of fixed locations within an area of interest. Based on this result, a novel step is introduced in the pipeline, with the objective of processing the mentioned signal series to extract quantitative descriptors of the signals trend. The computed quantities are finally used to fulfil the classification task through the implementation of a set of rules that, according to the numerical values of the computed features, assign each set of time series to a specific class.

MEC will be applied to the southern Iberian region, contributing to the understanding of the formation of upwelling filaments and the effects of climate change in this particular EBUE. The metrics currently used in MEC (e.g., the signal variation rate and its deviation from the mean value) are able to identify different types of mesoscale features, but different variables and arrangements still need to be tested, as discussed in the text.

The paper is arranged as follows: Section 2 and Section 3 concern a detailed description of the employed dataset and the related ground truth classification; Section 4 thoroughly reports about the developed processing pipeline and describes a relevant use case; Section 5 and Section 6 conclude the paper by discussing the outcomes of this work and providing a few considerations about future perspectives.

2. SST Satellite Data

The detection and classification of upwelling events is performed by processing SST data acquired by radiometers installed aboard meteorological satellites. In particular, for the analysis described in this paper data from the years 2009 to 2017 from two sources were used: the satellites of the Metop programme of EUMETSAT [42] and the Aqua satellite of NASA [43].

Table 1. Satellite data specifications.

Satellite	Sensor	Resolution	Temperature
		(at Nadir)	Accuracy
Metop-A (2009–2016)	AVHRR	1 $\mathrm{k}$$\mathrm{m}$	$0.01$${}^{\circ }\mathrm{C}$
Metop-B (2017)
Aqua	MODIS	1 $\mathrm{k}$$\mathrm{m}$	$0.005$${}^{\circ }\mathrm{C}$

outlines the main characteristics of these two sources.

Data are collected in the form of either NetCDF-4 files or HDF files (the latter only for older files from Aqua) and processed at level L2P in accordance with the Group for High-Resolution Sea Surface Temperature (GHRSST) data processing specification, (https://www.ghrsst.org/ghrsst-data-services/products/, accessed on 19 January 2023) meaning that the files contain derived SST data at the same resolution and location as source data from the sensor. Both sources cover the entire surface of Earth, but only files that contain data within our region of interest (latitude between 35° and 40° N, longitude between 12° and 6° W) were downloaded, resulting in datasets comprising an average of 2–3 images per day. Moreover, the reliability of the data captured by satellite radiometers is often prone to relevant fluctuations—this may happen, for instance, if the cloud coverage in the area is too dense, resulting in partially unreliable, sometimes even completely, sensor measurements (see Figure 1 for an example). This implies that at the finest possible resolution value, the sensor measurements are often unreliable due to noise, or unavailable because of energy dispersion occurring in the radiation propagation process. Therefore, some images have been pre-emptively selected: in particular, images with less than 15% of the expected amount of data (estimated using the declared spatial resolution and the extension of the area of interest) have been discarded.

3. Types of Patterns

As mentioned in the Introduction, the geomorphological features in the ICCS affect the upwelling jet that runs southwards along the western coast of Portugal, eventually reaching Cape St. Vincent (37°1’30” N, 8°59’40” W). As a consequence, the heterogeneous behaviour of the upwelled water gives rise to a variety of patterns that can be seen in the SST satellite images. In particular, experts visually inspected the SST datasets to determine a set of recurring temperature trends that are consistently observed throughout the considered time span and identified four main patterns relative to four types of mesoscale events:

• A filament of cold water originating from the upwelling jet, going westwards;
• A filament of cold water going southwards, extending the upwelling jet beyond Cape St. Vincent;
• A stream of cool water that bends eastwards from the upwelling jet, overtaking Cape St. Vincent and running along the southern coast of the Iberian peninsula;
• A warm countercurrent originating in the Gulf of Cádiz and running westwards along the southern Iberian coast, eventually reaching Cape St. Vincent and turning northwards.

These four patterns, and the corresponding mesoscale events, will be called E1, E2, E3, and E4, respectively. To be more precise, pattern E3 presents two subcases: E3i, when the difference in temperature between the cool stream and the waters of the Gulf of Cádiz is small, mostly occurring during winter; and E3u, with a more pronounced thermal gradient. In this analysis, we do not distinguish between these two cases. Figure 2 shows some examples of the described patterns, where the images on the right refer to the same coordinates as in Figure 1.

Starting from this classification, a labelled dataset of SST images has been obtained, in which expert oceanographers manually assigned a label “E1”–“E4” to each image in which a clear pattern could be seen (no label was assigned if no pattern could be recognised).

This dataset constitutes the ground truth for all the subsequent analysis; it is a static dataset, meaning that the information about the time at which an image has been generated has mostly not been taken into account when a label was assigned. However, MEC exploits also the dynamic information contained in the SST patterns. To help us visualise this information, the multiple SST signals contained in satellite data with timestamps belonging to a given time interval and related to a given geographical area are extracted from the corresponding data files and arranged in a single 2D time-SST plot, namely a spaghetti plot (see Figure 3d below). This way it is possible to assess the SST temporal trends in the selected area. In the next section the process of the spaghetti plot generation is described more in detail.

All the software dedicated for this analysis has been developed within a Python framework.

4. MEC Explained

4.1. Spaghetti Plot Generation

The data files described in Section 2 are georeferenced, meaning that a satellite image with timestamp t contains several SST readings, each associated with a point on the surface of Earth. In the following we denote by $T(t,\varphi ,\lambda )$ the recorded SST at time t at the point with latitude $\varphi$ and longitude $\lambda$. Note that, if we choose random values for t, $\varphi$, and $\lambda$, it is possible that the corresponding SST value $T(t,\varphi ,\lambda )$ is not defined: this depends on the available data in the satellite files.

Given a target area $A=[{\varphi }_{\min },{\varphi }_{\max }]\times [{\lambda }_{\min },{\lambda }_{\max }]$ and a time interval $\tau =[t_{\mathrm{start}},t_{\mathrm{end}}]$, we illustrate here the steps used to produce a spaghetti plot.

• Choose a resolution $r>0$, such that the numbers $\ell =({\varphi }_{\max }-{\varphi }_{\min })/r$ and $m=({\lambda }_{\max }-{\lambda }_{\min })/r$ are integers (typical values for r lie between 0.01 and 0.25 degrees in latitude and longitude) and divide the area A in $\ell \times m$ squares

  $$a_{i,j}=[{\varphi }_{\min }+ir,{\varphi }_{\min }+(i+1)r]\times [{\lambda }_{\min }+jr,{\lambda }_{\min }+(j+1)r]$$

  with $i=0,\cdots ,\ell -1$ and $j=0,\cdots ,m-1$.
• For each satellite image with timestamp t and for all squares $a_{i,j}$, compute the spatial average of the SST at time t in the square. This step is performed to mitigate the effects of possible outliers and noise within the square. Formally, let $X_{i,j}\left(t\right)$ be the set of points with coordinates $(\varphi ,\lambda )\in a_{i,j}$ that have a recorded SST in the image, i.e.,

  $$X_{i,j}\left(t\right)=\{(\varphi ,\lambda )\in a_{i,j}\mid T(t,\varphi ,\lambda )\mathrm{is}\mathrm{recorded}\};$$

  then for each t, i, and j compute

  $${\overline{T}}_{i,j}\left(t\right)=\frac{1}{\left|X_{i,j}\left(t\right)\right|}\sum _{(\varphi ,\lambda )\in X_{i,j}\left(t\right)}T(t,\varphi ,\lambda )$$

  where $\left|·\right|$ denotes the cardinality of a set. Notice that $X_{i,j}\left(t\right)$ may be empty—in that case, we leave ${\overline{T}}_{i,j}\left(t\right)$ undefined. We choose to not define ${\overline{T}}_{i,j}\left(t\right)$ also when $\left|X_{i,j}\left(t\right)\right|$ is too small, because in that case the lack of a sufficient amount of data within $a_{i,j}$ makes the mean estimation unreliable.
• Let $t_1,\cdots ,t_N$ be the timestamps of the images belonging to the time interval $\tau$. For each square $a_{i,j}$ compute the time series

  $$p_{i,j}=\{(t_k,{\overline{T}}_{i,j}\left(t_k\right))\mid k=1,\cdots ,N\mathrm{such}\mathrm{that}{\overline{T}}_{i,j}\left(t_k\right)\mathrm{is}\mathrm{defined}\}.$$

  (1)
• The spaghetti plot is obtained by plotting all the time series $p_{i,j}$ simultaneously within the same time–temperature coordinate system.

Each square $a_{i,j}$ and the plot of the corresponding series $p_{i,j}$ is colour-coded, so that it is easier to recognise different SST trends in different parts of the area A. The above steps integrate and further detail the generation of spaghetti plots described in [41,44].

After having tested different values of the parameters involved in the spaghetti plot generation process, the following choice represented a satisfactory trade off between the computational effort (more details about this in Section 5) and the classifier resolution requirements, that need to be properly tuned to ensure a correct features extraction process (described in Section 4.2):seemed to produce outputs that better agree with the ground truth:

• Resolution $r=0.25{}^{\circ }$;
• Time interval $\tau =15\mathrm{days}$, which is the typical temporal scale at which the sought mesoscale events appear and fade;
• Data abundance threshold (used in step 2 above): we require $\left|X_{i,j}\left(t\right)\right|\ge 100$ for ${\overline{T}}_{i,j}\left(t\right)$ to be defined (i.e., about 16% of the expected amount of data in a square).

Figure 3 introduces an example that will be discussedwe will follow step by step in the following sections. It represents an event occurred on 13 August 2016 and classified as E4 in the ground truth. Indeed, a pattern that can be linked to an E4 event is recognizable in the spaghetti plot (see Figure 3d): from 8 August to the end of the period, the temperature curves diverge, with the southeastern part of the target area increasing its SST more than the northwestern one.

4.2. Features Extraction

The spaghetti plot representation of an ensemble of SST signals within a given area of interest is a qualitative method that can be exploited in a preliminary stage to visually detect and classify mesoscale events by jointly observing the spatial and temporal SST trend. This approach is prone to human interpretative errors and becomes quickly unfeasible as the size of the considered area increases. A computer-based implementation of the visual classification task must be introduced to make the processing pipeline automatic and replace the mere visual analysis.

In this section, we identify a number of features computed from the SST series exploited in the generation of the spaghetti plots, such as the $p_{i,j}$ defined above, with the purpose of defining a set of rules to be applied to those features. The rules constitute the next step in the classification task and are described in detail in Section 4.3.

The starting point is the set of the series $p_{i,j}$, defined in (1), for all squares $a_{i,j}$ in which the area of interest is divided. For each series $p_{i,j}=\{(t_k,{\overline{T}}_{i,j}\left(t_k\right))\mid k=1,\cdots ,n_{i,j}\}$ the following three statistics are computed:

• The temporal mean of $p_{i,j}$

  $${\mu }_{i,j}=\frac{1}{n_{i,j}}\sum _{k=1}^{n_{i,j}}{\overline{T}}_{i,j}\left(t_k\right);$$
• The standard deviation of $p_{i,j}$

  $${\sigma }_{i,j}=\sqrt{\frac{1}{n_{i,j}}\sum _{k=1}^{n_{i,j}}({\overline{T}}_{i,j}\left(t_k\right)-{\mu }_{i,j}){)}^2};$$

  Figure 3. Event of 13 August 2016 at around 21:40 UTC. (a) SST map at the date of the event; (b) detail of the SST in the reference area for the spaghetti plot (latitude between 37° and 37.75° N, longitude between 9.5° and 8.75° W, resolution 0.25°); (c) reference grid; and (d) generated spaghetti plot.

  Figure 3. Event of 13 August 2016 at around 21:40 UTC. (a) SST map at the date of the event; (b) detail of the SST in the reference area for the spaghetti plot (latitude between 37° and 37.75° N, longitude between 9.5° and 8.75° W, resolution 0.25°); (c) reference grid; and (d) generated spaghetti plot.
• The linear regression coefficient ${\theta }_{i,j}$, defined as the slope of the straight line that better interpolates the points in $p_{i,j}$.

These statistics describe, in first approximation, the behaviour of the SST locally in the square $a_{i,j}$. Additional parameters, such as the coefficients of the quadratic regression curve for the points of $p_{i,j}$, have been considered initially, but they have been discarded since their integration in the classification pipeline did not improve the performance, at the expense of a more complicated model. Notice that, by using the coefficient ${\theta }_{i,j}$ we do not assume that time and SST are linearly correlated: that value simply represents a descriptor of the SST trend in the square $a_{i,j}$.

Due to the fluctuating quality of the SST imagery, the value $n_{i,j}:=\left|p_{i,j}\right|$ appearing in the formulas above is not constant, but depends on the quantity of reliable data in the files within the time period. In particular, higher values of $n_{i,j}$ correspond to a more faithful estimation of the statistics for the SST trend in $a_{i,j}$ in the time period through the computation of ${\mu }_{i,j}$, ${\sigma }_{i,j}$, and ${\theta }_{i,j}$. Therefore, $n_{i,j}$ can be viewed as a reliability index of the classification of the square $a_{i,j}$. As an additional check to avoid the case of unreliable results, no feature is computed for a square $a_{i,j}$ if it contains too few data in the time period.

In Figure 4, three maps show the values of these statistics computed for the case study under investigation. Given the time interval $\tau$ of 15 days, we expect to have $n_{i,j}\approx 30$ SST values defined in each square, therefore in this case we set the threshold of $n_{i,j}>4$ for the statistics to be computed for $a_{i,j}$.

4.3. Classification Rules

For each square $a_{i,j}$ in the area of interest, a score array $e_{i,j}=(e_{i,j}^1,e_{i,j}^2,e_{i,j}^3,e_{i,j}^4)$, with $e_{i,j}^k\in [0,1]$ is computed. The value $e_{i,j}^k$ is an index representing how much an event of type Ek is believed to have occurred inside $a_{i,j}$ at the end of the time period.

Each $e_{i,j}^k$ is computed by applying a set of conditional rules to the statistics described in Section 4.2. In particular, the scores for a square $a_{i,j}$ depend not only on ${\mu }_{i,j}$, ${\sigma }_{i,j}$ and ${\theta }_{i,j}$ but also on the values of $\mu$, $\sigma$ and $\theta$ for the squares in its proximity, with different kinds of neighbourhoods depending on the specific rule (see Figure 5).

The rules are handcrafted to mirror the behaviour of the SST trends during the course of different types of upwelling events, so that the score $e_{i,j}^k$ is increased only if the changing in the values of $\mu$, $\sigma$, and $\theta$ in and nearby $a_{i,j}$ matches the one observed during an event of type Ek.

The following is a detailed description of the classification rules. They are cumulative, i.e., each condition in these lists is tested independently and, if it is verified, increases the score $e_{i,j}^k$ by a fixed amount.

• Increase $e_{i,j}^1$ if:

  (a)

  The SST trend ${\theta }_{i,j}$ inside $a_{i,j}$ is negative;

  (b)

  The SST trend of the eastern neighbouring squares (see Figure 5b) is lower than ${\theta }_{i,j}$;

  (c)

  The SST mean value ${\mu }_{i,j}$ is lower than the averages of the mean values of the SST in the northern and southern neighbouring squares (the two neighbourhoods are tested separately, each increasing the score if the outcome is positive, and an extra increase is awarded if both are verified).

• Increase $e_{i,j}^2$ if:

  (a)

  The SST trend ${\theta }_{i,j}$ inside $a_{i,j}$ is negative;

  (b)

  The SST trend of the northern neighbouring squares (see Figure 5c) is lower than ${\theta }_{i,j}$;

  (c)

  The SST mean value ${\mu }_{i,j}$ is lower than the averages of the mean values of the SST in the eastern and western neighbouring squares (the two neighbourhoods are tested separately, each increasing the score if the outcome is positive, and an extra increase is awarded if both are verified).

• Increase $e_{i,j}^3$ if:

  (a)

  The SST trend ${\theta }_{i,j}$ inside $a_{i,j}$ and the $\theta$-values of all the full neighbourhood squares of $a_{i,j}$ (Figure 5a) are negative;

  (b)

  The SST in the northwestern neighbourhood (see Figure 5d) of $a_{i,j}$ decreases before the one in $a_{i,j}$, and the SST in the southeastern neighbourhood of $a_{i,j}$ either decreases after the one in $a_{i,j}$ or increases (this condition is tested for all the relevant squares in the neighbourhoods separately);

  (c)

  $a_{i,j}$ is warmer, on average, than the squares in its northwestern neighbourhood and colder, on average, than the ones in its southeastern neighbourhood (the two conditions are tested separately, each increasing the score if the outcome is positive, and an extra increase is awarded if both are verified).

• Increase $e_{i,j}^4$ if:

  (a)

  The SST trend ${\theta }_{i,j}$ inside $a_{i,j}$ and the $\theta$-values of all the full neighbourhood squares of $a_{i,j}$ (Figure 5a) are positive;

  (b)

  The SST in the northwestern neighbourhood (see Figure 5d) of $a_{i,j}$ either increases after the one in $a_{i,j}$ or decreases, and the SST in the southeastern neighbourhood of $a_{i,j}$ increases before the one in $a_{i,j}$ (this condition is tested for all the relevant squares in the neighbourhoods separately);

  (c)

  $a_{i,j}$ is warmer, on average, than the squares in its northwestern neighbourhood and colder, on average, than the ones in its southeastern neighbourhood (the two conditions are tested separately, each increasing the score if the outcome is positive, and an extra increase is awarded if both are verified).

• Additional conditions may modify the final scores (these are checked after the previous four points):

  (a)

  If the SST variation ${\sigma }_{i,j}$ is large (namely ${\sigma }_{i,j}\ge 1{}^{\circ }\mathrm{C}$, which is considered a very large value with respect to the typical standard deviation within the chosen time frame—see also Figure 4), then increase either $e_{i,j}^1$, $e_{i,j}^2$ and $e_{i,j}^3$ (if the SST in $a_{i,j}$ decreases) or $e_{i,j}^4$ (if the SST increases);

  (b)

  If $a_{i,j}$ is either globally “cold” (for events E1, E2, and E3) or globally “warm” (for E4), meaning that ${\mu }_{i,j}$ is smaller (respectively, larger) than the average of the $\mu$-values of all the squares in the area of interest, boost the corresponding scores;

  (c)

  If $a_{i,j}$ is too near the coast (within about 75 $\mathrm{k}$$\mathrm{m}$ away from it, that is a threshold to distinguish between inshore and offshore upwelling events, given their typical spatial extension), penalize the scores $e_{i,j}^1$ and $e_{i,j}^2$; if it is too far from the coast (further than about 75 $\mathrm{k}$$\mathrm{m}$ away), penalize $e_{i,j}^3$ and $e_{i,j}^4$.

In the case study that we are following, the application of the rules to the values of the statistics in Figure 4 produces the scores reported in Figure 6.

The final step of the classification process is the production of a heatmap, like the one in Figure 7, which is the result of our case study. It is defined in the following way: for each square $a_{i,j}$, consider the maximum score $e_{i,j}^m=max\{e_{i,j}^1,e_{i,j}^2,e_{i,j}^3,e_{i,j}^4\}$ and, if this score is greater than a certain threshold, mark the square with the “Em” label, otherwise do not assign a label to the square. After some testing, we chose a threshold of 0.6, which produces results that better agree with the ground truth. Each classified square in the heatmap also displays the percentage of available data, which is proportional to the reliability index $n_{i,j}$ described above. Assuming 30 images in a time interval of 15 days, the numbers in the heatmap are given by $100·n_{i,j}/30$. Notice that this assumption is a rough estimate, so numbers above 100 may appear.

5. Results and Discussion

Looking at Figure 7, we observe that the results of our analysis are aligned with the ground truth since the majority of the squares have been classified as “E4” (the red ones), and they are located within the geographical zone where events of type E4 are usually detected. Notably, the squares classified as E4 have a high reliability value $n_{i,j}$, i.e., more high-quality data were available with respect to other squares.

The features computation depends only on properties that are observable inside or in a neighbourhood of the considered square. Indeed, no a priori knowledge is assumed about the geographic and oceanographic properties of the area of interest. Based only on the requirement that a sufficient amount of captured temperature values is available, the algorithm actually returns the prediction of the mesoscale event that may have occurred within every square inside the area of interest. Of course, this approach is not always acceptable since it also provides a classification prediction for regions where the emergence of these phenomena cannot be observed due to the physical properties of the system. To avoid this issue, it is suitable to introduce constraints deriving from theoretical and empirical knowledge about the analysed phenomena. These considerations allow restricting the analysis to the geographical regions that constitute the most likely domains for the development of mesoscale events. Concerning the ICCS EBUE case, cold-water filaments are observed only within well-defined geographical zones off the Portuguese coast while the upwelling phenomena detection is possible only close to the shore. The introduction of such constraints, graphically represented in Figure 8, enables to filter out the unrealistic classifications, resulting, for the presented case study, in the refined classification depicted in Figure 9, where a single dominant event clearly emerges.

In this case, the classifier correctly identifies the event class by returning several suitably labelled squares located in the corresponding geographical domain. It is not unusual to observe SST images series where the classifier response is less univocal. For example, on 16 September 2017 and 7 October 2017 (see, respectively, Figure 10 and Figure 11), two separate clusters differently labelled are observed. In those cases, the output of the classifier apparently conflicts with the ground truth, but this can be explained by the fact that around the end of the observed time window, different types of SST events are detected. On 16 September 2017, two distinct events, E1 and E3, are concurrently detected, while almost the same circumstance occurs on 7 October 2017, where the ground truth label is E4, but traces of E1 are also detected on the previous days. This confirms the correctness of the MEC approach that, employing an analysis based on time series, accordingly identifies multiple typologies of events that develop within the considered time window in the neighbourhood of the main event.

Despite the specific definition of the classification rules, modelled on the characteristic behaviour of the SST in the ICCS, the general applicability of the algorithm is ensured by the formulation of the classifier core. In fact, it can be applied to any marine environment, provided that the classification rules and the geographical constraints identifying the allowed areas are updated accordingly. A further extension of MEC will be achieved by conceiving a set of general classification rules, conceived in such a way that they can be applied to a general marine scenario without requiring any specific fulfilment on the considered area. Future work will address this extension.

Finally, it is suitable to briefly report about the MEC computational performance. Considering that the satellite sensor resolution is about $0.01{}^{\circ }$ (see Table 1), more than ${10}^4$ SST samples are expected to fall within a sea square surface with side size $\delta =1{}^{\circ }$. As mentioned in Section 4, the SST data are projected on a spatial grid with cell side typically selected to be larger than the sensor nominal resolution. The algorithm performance mostly depends on the time required to open the data files, access the stored sensor measurements and assign a signal intensity value to the cell, computed as the mean value of the SST measurements falling within that cell.

Once the resolution value is chosen by fixing the size of the cell side ($r=\delta /d$, with d integer), the number of average operations to be performed roughly amount to $d^2$, which is approximately confirmed by observing the values reported in the second column of

Table 2. Times required for each step of the classifier at different resolutions, for the analysis of an area of $1{}^{\circ }\times 1{}^{\circ }$ for 15 days. All computations were performed on a processor Intel® Core™ i5-8250U @ 1.60 GHz with 8 GB RAM.

Resolution	Spaghetti Plot	Statistics	Application
	Generation	Extraction	of Rules
$0.05$${}^{\circ }$	$261.363$$\mathrm{s}$	$0.114$$\mathrm{s}$	$0.216$$\mathrm{s}$
$0.125$${}^{\circ }$	$44.872$$\mathrm{s}$	$0.026$$\mathrm{s}$	$0.037$$\mathrm{s}$
$0.25$${}^{\circ }$	$14.812$$\mathrm{s}$	$0.010$$\mathrm{s}$	$0.014$$\mathrm{s}$

. The spaghetti plot generation stage thus represents the task with the highest computational burden in the algorithm pipeline, while the number of operations for the remaining pipeline stages (statistics extraction and rules application) grows only linearly with the number of images captured per day. On the other hand, a reasonable choice for d must ensure a suitable execution of the feature extraction stage, so that the grid cell size has to be comparable with the typical size of the sought phenomena (tens of km), which is why $0.25{}^{\circ }$ has been identified as a proper trade off value.

6. Conclusions

This work describes MEC, a system to classify mesoscale events (e.g., upwelling, eddies, thermal fronts), observed as patterns in SST data. It is known that the development of these phenomena shapes the hydrodynamics, habitat, and species distribution of a specific region. Therefore, the detection and tracking of the related features is of fundamental importance in what concerns fisheries management and exploitation, coastal monitoring, and climate change.

MEC represents an attempt to introduce the benefits of a computer-based automation within a data analysis task which at present is, to the best of the authors’ knowledge, extensively manual. The complete automation of such analysis encompasses a few challenges: in the first place the presence of noise, mainly due to clouds and other atmospheric phenomena, which implies that signal variations relevant for the description of the mesoscale features are corrupted or occluded to some extent. This is an important issue to take into account in case gradient-based methods are employed, since these methods are sensitive to the presence of spurious signals and may critically affect the classifier reliability. Within MEC, a preliminary filtering stage ensures that only major quality data, i.e., SST maps containing a percentage of reliable data above a user-defined threshold, are fed to the classifier. Additionally, the extension of the analysis from a single image approach to an entire sequence of consecutive images collected over a given time window allows to tackle problems related to cloud occlusions and ensures an increase in the accuracy in the statistics estimation.

A second challenge is represented by the lack of valid analytical models for the geometric shapes of the structures emerging in a mesoscale event. Indeed, considering any single class of mesoscale events and analysing the corresponding occurrences in the dataset, the conclusion is that the modelling of a specific template is not feasible. This synthesis operation, that would enable the proposal of a pattern-matching approach to the problem, is prevented by the strong and erratic morphological variability, clearly assessed in the data. The approach proposed in MEC once more leverages on the joint analysis of the SST spatial and temporal variability. In the absence of well defined signal patterns the classification task is carried out within an enlarged domain, featuring both space and time parameters, where the clustering process takes simultaneous advantage of the peculiar SST spatial arrangement/configurations and also of the temporal factors that allow to detect the onset of an event and discern its category.

The activities discussed in this paper have been realised within the framework of the EU H2020 project NAUTILOS [45].