Relative Merits of Optimal Estimation and Non-Linear Retrievals of Sea-Surface Temperature from MODIS

Abstract

We compared the results of an Optimal Estimation (OE) based approach for the retrieval of the skin sea surface temperature (SST_skin) with those of the traditional non-linear sea surface temperature (NLSST) algorithm. The retrievals were from radiance measurements in two infrared channels of the Moderate Resolution Imaging Spectroradiometer (MODIS) on the NASA satellite Aqua. The OE used a reduced state vector of SST and total column water vapor (TCWV). The SST and atmospheric profiles of temperature and humidity from ERA5 provided prior knowledge, and we made reasonable assumptions about the variance of these fields. An atmospheric radiative transfer model was used as the forward model to simulate the MODIS measurements. The performances of the retrieval approaches were assessed by comparison with in situ measurements. We found that the OESST reduces the satellite–in situ bias, but mostly for retrievals with an already small bias between in situ and the prior SST. The OE approach generally fails to improve the SST retrieval when that difference is large. In such cases, the NLSST often provides a better estimate of the SST than the OE. The OESST also underperforms NLSST in areas that include large horizontal SST gradients.

1. Introduction

Sea surface temperature (SST) is a critically important geophysical parameter in the Earth System. The Committee on Earth Observation Satellites (CEOS) has declared SST to be an Essential Climate Variable (ECV) that has a high impact on the requirements of the UNFCCC (United Nation Framework Convention on Climate Change) and the IPCC (Intergovernmental Panel on Climate Change) [1]. SST derived from satellite measurements offers the best source of global, repeated fields, along with accompanying uncertainty characteristics. For several decades now, SST has been routinely derived from radiances measured by a range of infrared (IR) or microwave (MW) satellite sensors [2]. The satellite-derived SST is an important element of the global observing system for numerical weather prediction (NWP) and for climate research. The requirements for the accuracy of SST are very demanding, as is the need for error analysis to fulfill the requirements for the generation of climate data records (CDRs) [3]. The concept of a CDR was formally introduced in a National Academy of Sciences report [4] as being “a data set designed to enable study and assessment of long-term climate change, with ‘long-term’ meaning year-to-year and decade-to-decade change. Climate research often involves the detection of small changes against a background of intense, short-term variations…. The production of CDRs requires repeated analysis and refinement of long-term data sets, usually from multiple data sources”. The generation of a CDR of global SST poses stringent requirements on the accuracy of derived SST: An absolute temperature uncertainty of 0.1 K and a decadal stability of ~0.04 K [5].

Over the years since satellite SST retrievals began [2], the most common approach to the retrieval of SST from satellite IR sensors employs algorithms based on the “split-window” technique [6,7,8] where satellite measurements are made in two IR channels typically centered near 11 and 12 μm wavelengths where the cloud-free atmospheric transmissivity is relatively high, but showing a wavelength dependence. The SST is derived by a slightly nonlinear combination of the brightness temperatures (BTs) in these two channels and some baseline sea surface temperature estimate (bsst). This is known as the non-linear SST algorithm (NLSST; ibid). For the Moderate Resolution Imaging Spectroradiometer (MODIS) which is flown on the NASA Aqua and Terra satellites [9] the NLSST has the form:

$$\begin{array}{ll}\mathrm{NLSST}=c1+c2& \ast \mathrm{T}11+c3\ast \left(\mathrm{T}11-\mathrm{T}12\right)\ast \mathrm{bsst}+c4\ast \sec \left(\mathrm{satz}\right)\ast \left(\mathrm{T}11-\mathrm{T}12\right)+c5\ast mirrorside+c6\\ & \ast \mathrm{satz}+c7{\ast \mathrm{satz}}^{2 }\end{array}$$

(1)

where T1 and T12 are brightness temperatures (BTs) in the 11 and 12 μm channels, bsst is the baseline SST, satz is the satellite zenith angle, mirrorside depends on which side of the two-sided, paddle-wheel scan mirror directs radiation to the radiometer, and c1 to c7 are coefficients [10]. Similar formulations exist for other multi-channel IR instruments including the Advanced Very High Resolution Radiometer (AVHRR) [11], the (Advanced) Along-Track Scanning Radiometer ((A)ATSR) [12,13], the Spinning Enhanced Visible and InfraRed Imager (SEVIRI) [14], the Advanced Baseline Imager (ABI) [15], the Advanced Himawari imager (AHI) [16], and the Visible Infrared Imaging Radiometer Suite (VIIRS) [17]. The channel BT difference, T11-T12, is mainly sensitive to the water vapor content of the atmosphere. The coefficients c1 to c7 are derived by regression of the BTs to in situ measurements of SST (e.g., [8,11,18,19]), that are included in the Match-Up Data Bases (MUDBs) [11]. The coefficients may alternatively be derived by regression of channel BTs simulated using radiative transfer modeling and the value of the SST specified as the boundary condition of the model (e.g., [12,20,21]).

Since the NLSST algorithms rely on coefficients that are generally obtained by least-squares regression to in situ data (moored buoy or drifter measurements) they produce good estimates of SST in conditions that are represented by the conditions implicit in the MUDB records used to generate the coefficients. However, when the sea or atmospheric conditions depart from the average state, the NLSST estimates can be quite erroneous. The existence of regional and seasonal biases in the NLSST is well documented [11,22,23]. A good example of this bias is the retrieval over the Northern Atlantic Ocean during Saharan Air Layer (SAL) outbreaks [24]. Typically, the atmosphere in this region is moist and such conditions are dominantly represented in the MUDB used to derive the coefficients of the NLSST formulation. During SAL events, moist air is replaced by a layer of very dry air of continental origin, often including dust aerosols from the Saharan region, and the established coefficients are not representative of these conditions, resulting in larger retrieval errors [25].

One way of reducing the NLSST bias is to select from multiple sets of coefficients those that are specific to different seasons and geographic regions [11,17,19]. Another way is to model explicitly the effects of anomalous atmospheric conditions, including SAL and aerosol effects, using radiative transfer calculations and subtract the modeled bias from the retrieval; this is typically used for retrievals under SAL conditions [25,26,27,28].

In general, in the ocean–atmosphere system, there are fundamentally two leading modes of variability represented in the measurements of the two IR channels used in the NLSST formulation. These are the SST itself and the total atmospheric moisture content (total column water vapor, TCWV) and their variability is captured by the retrieval coefficients derived by linear regression, meaning the NLSST estimates cannot be improved without introducing additional information into the retrieval system. This is accomplished when regional or seasonal NLSST coefficients are used in the retrieval. Prior information on the mean and variability of the SST and atmosphere of a region or a season is then effectively embedded in the coefficients.

More recently methods based on Bayesian approach to inverse problems [29] have been proposed and used to retrieve SST from multichannel IR measurements [30]. The term often used to refer to this approach is Optimal Estimation (OE). In the OE approach, prior knowledge of the state of the ocean and the atmosphere at the time and place of each satellite measurement is used to derive the SST estimates (OESST) and as such could be viewed as an additional level of the coefficient tuning of the NLSST approach.

The OE retrieval of SST has gained popularity in recent years with the promise of improving on the biases inherent in the NLSST approach. A very good agreement between the OE estimates and in situ buoy measurement of average bias of −0.06 K and −0.01 K have been achieved by Merchant et al. (2008) for AVHRR on the Metop-A satellite and by Merchant et al. [31,32] for SEVIRI on Meteosat 9. The statistics of these biases are indeed good, being several times lower than those for corresponding SST derived by NLSST algorithms using global coefficients. Additionally, these retrievals specifically exclude conditions where the OE approach might not work well: Only nighttime measurements were considered, and measurements affected by SAL were excluded.

With an expectation of OE generally outperforming the NLSST retrievals, we focus on comparisons between the two. The OE approach comes with its own set of uncertainties and sources of error which, if not addressed, undermine the expected gain in accuracy or even usefulness of the retrieval. Koner et al. ([33]) proposed an alternative SST retrieval method: The Modified Total Least Squares (MTLS) and found that OE retrieval was worse than the MTLS solution, and sometimes worse than the a priori. They argued that OE is unable to produce the best retrieval for a fairly linear and moderately ill-conditioned problem of SST retrieval, relating it to the estimation of errors (which the MTLS avoids) being difficult in practice. Indeed, the error characteristics are typically assessed by “expert judgment” based on the best available knowledge [32]. Additionally, since the OE requires unbiased priors, care must be taken to remove bias from the priors before the retrieval. In order to remove the degree of arbitrariness from the problem, Merchant et al. (ibid) proposed an OE-based approach to the derivation of the biases and covariance parameters and in application to SST retrieval from SEVIRI measurements achieved improved results over an OE application with non-optimized parameters.

In this work we are not seeking optimal values of the covariance parameters but focus on examining how possible values of these parameters influence the usefulness of the OE method to the SST retrieval from MODIS. We show that in the case of MODIS SST the main factor contributing to the poor OE retrieval is the bias of the prior SST, while all other combined biases play a much smaller role. To assess the relative consequences of factors other than the bias of prior SST on retrieved OESST, we first conducted the OE retrieval with prior SSTs from the temperature measured in situ corrected by the mean skin effect. This eliminates the prior SST bias from the retrieval problem. Since we consider the matching in situ measurements of SST as the reference temperature, any difference between the OESST in this case and the in situ SST is attributable to all the other biases. We found the unresolved biases account on average for about 0.02 K of the OE and in situ temperature difference which could be viewed as a limit on the accuracy of OESST. We then evaluate the accuracy of the OESST with prior SST from ERA5 reanalysis.

We will also address the question: By how much is our knowledge of the SST improved in the OE retrieval in comparison with the a priori knowledge, and how does it compare to the NLSST retrieval?

We briefly review the basic concepts of the Bayesian retrieval in Section 2 and its application to a linear model in Section 3. In Section 4 we describe the application of the OE approach to the retrieval of SST from the MODIS data set used in this study. Section 5 introduces measures used to assess the information gain of the retrieval. In Section 6 we discuss the results of the OE retrieval and identify situations that impair the performance of OE. We compare OE and the NLSST retrievals and suggest when one of these methods is more likely to provide a better estimate of SST than the other in Section 7. Summary and conclusions are given in Section 8.

2. Bayesian Inversion Concepts

Imagine that we have a system that can be described by state vector x, and in order to gain some knowledge about this system, we perform a measurement which results in a vector y. The act of measurement maps the state space into the measurement space. In the presence of a measurement error, the mapping is not exact, i.e., a point in the state space maps into a region in the measurement space that is determined by the probability density function (pdf) of the measurement error ϵ. This mapping can be represented by a forward function F(x) or, in practice, often by a forward model F, and written as:

$$y=F\left(x\right)+\mathsf{ϵ}$$

(2)

Inversely, a measurement of y does not represent a single point but rather a region in the state space that can be described by some pdf. If we have some prior (imperfect) knowledge of the state before the measurement, then that information can be used to constrain the region representing the solution. This prior knowledge can often also be described by a pdf. Bayes’ Theorem shows how the pdf of an imperfect measurement maps back to the state space and combines with the pdf of the imperfect prior knowledge to improve our understanding of the system,

$$P\left(x|y\right)=P\left(y|x\right)P\left(x\right)/P\left(y\right)$$

(3)

where P(x|y) is the pdf of the state being described by vector x provided the measurement is y, P(x) is the pdf of the prior state, P(y|x) is the conditional pdf of obtaining measurement y provided the state vector is x, and P(y) is pdf of obtaining a measurement y. P(y) can be obtained by integrating P(y|x)P(x) over all states x, but is only a normalizing factor and is typically not necessary.

Conceptually, the Bayesian approach can be summarized as follows:

i.

we have some knowledge of the state before the measurement is even made and this prior knowledge is expressed by a prior pdf of the state variables

ii.

we have a forward model that maps the state variable into the measurement space

iii.

we know the pdf of the measurement errors

iv.

we use Equation (3) to calculate posterior pdf by augmenting the prior pdf of the state vector with the measurement.

The result of the Bayesian inversion is thus a pdf of all possible states. This ‘solution’ informs us how the measurement improves our knowledge of the state but to obtain a retrieval that can be more readily compared with other methods, one particular state needs to be selected from all the possible posterior states. This would be an optimal state according to some metric, and often is selected as the most probable state or the expected value state of the posterior pdf.

3. Linear Forward Model and Gaussian PDFs

The distribution of error in many real-world measurements adheres fairly closely to the Gaussian distribution and since the Gaussian formulation is mathematically expedient it is often convenient to use it as an approximation for both the measurement error pdf and the prior pdf.

The forward model F can typically be linearized around some reference state and then Equation (2) becomes

$$y=Kx+\mathsf{ϵ}$$

(4)

where K is a matrix of partial derivatives K_ij = ∂F_i/∂x_j (a Jacobian), sometimes referred to as the kernel.

In the case of a linear F and Gaussian measurement error and prior pdfs, the posterior pdf is also Gaussian with expected value and covariance given by

$$\widehat{x}=x_a+{\left(K^{\mathrm{T}}{S}_{\mathsf{ϵ}}^{-1}K+S_{\mathrm{a}}^{-1}\right)}^{-1}{K}^T{S}_{\mathsf{ϵ}}^{-1}\left(y-F\left(x_a\right)\right)$$

(5)

and

$${\widehat{S}}^{-1}=\left(K^{\mathrm{T}}{S}_{\mathsf{ϵ}}^{-1}K+S_{\mathrm{a}}^{-1}\right)$$

(6)

where x_a and S_a are the expected value and the error covariance matrix of the prior pdf and the S_ϵ is the covariance matrix of the measurement error.

With the above assumptions, Equation (5) indicates that the retrieved state is a sum of the prior state and a correction which is directly related to the difference between the measurement and the model simulated values for the a priori state. In the context of SST retrieval from satellite IR instruments, the measurement is the radiance measured in the spectral channels of the satellite instrument while the model measurement is the simulated value of radiance in those channels calculated by radiative transfer for a selected a priori state.

4. Gain and Averaging Representation

Equation (5) represents a regularized least square problem with S_a⁻¹ acting as the regularization matrix. The regularization is necessary when the inverse problem is underdetermined or ill-posed, i.e., the matrix K^TS_ϵ⁻¹ K has a high condition number, as it often is the case in remote sensing inversion problems. In OE the strength of the regularization and thus the ability to obtain a stable solution is determined by the statistics of the a priori state in relation to the matrix K^TS_ϵ⁻¹ K.

Equation (5) is often written as

$$\widehat{x}=x_a+G\left(y-F\left(x_a\right)\right)$$

(7)

where G, the gain matrix, gives the sensitivity of the retrieval to the observations: $G=\partial \widehat{x}/\partial y.$ Any errors in the measurement are multiplied by the gain and propagated to the solution. Due to the regularization the retrieval obtained by Equation (5) varies from the state that is most consistent with the pure measurement. The retrieved OE state results from the combination of the measurement and the priori information and the ratio at which they are combined depends on the strength of the regularization. In another transformation Equation (5) can be written as

$$\widehat{x}=A{x}_a+\left(I_{\mathrm{n}}-A\right)x_a$$

(8)

where matrix A = GK is the averaging kernel matrix and represents the sensitivity of the retrieved state to the true state ($A=\partial \widehat{x}/\partial x)$ and I_n is an identity matrix of size n, where n is the dimension of the state. The rows of the averaging kernel represent the weighting functions, which determine to what degree the result depends on the components of the true state.

It should be instructive to examine the regularization and the matrices G and A for any retrieval problem as they will inform one about the stability of the OE solution and reveal how much the measurement contributes to the improvement of our knowledge of the state vector as given by the a priori, or if a given measurement contributes to the solution at all. What is more, this can be evaluated even before the measurements or retrievals are made. For example, since the gain matrix is determined by the relative magnitude of K^TS_ϵ⁻¹ K and S_a⁻¹ if the measurement error is large so that K^TS_ϵ⁻¹ K << S_a⁻¹, then the gain will be close to zero and $\widehat{x} \to x_a .$ Inversely, if the measurement is so precise that K^TS_ϵ⁻¹ K >> S_a⁻¹, the a priori information becomes irrelevant. Correspondingly, in terms of the averaging matrix A, if A is close to the identity matrix the retrieval will be mostly determined by the measurement.

What do we know about K, S_ϵ⁻¹ and S_a⁻¹ for the problem of SST retrieval from MODIS 11 and 12 μm channel radiances, and under what condition can we expect the OE retrieval to be worthwhile? Answering these questions is the subject of the next section.

5. Application to SST Retrievals

5.1. Satellite Dataset

We apply the OE retrieval and the above concepts to a subset of MODIS measurements from the Aqua satellite with the intention of establishing the relative merits of the OE approach compared to the NLSST. The MODIS data used in this study are a subset of the MUDB MODIS R2019 data set, where R2019 is the identifier for the set of cloud-screening and atmospheric correction algorithms, sometimes referred to as Version 6 or Collection 6 [10]. Our subset spans two years, 2015 and 2016, from the area of the northern Atlantic Ocean and the Mediterranean Sea selected because of the relatively large number of match-ups; a wide range of SSTs; extreme values of horizontal SST gradients, ranging from the edges of the Gulf Stream to central gyres; and the high variability of atmospheric conditions including those influenced by different continental airmasses advected over the ocean. In addition to in situ SST measurements matched to the SST_skin derived with the NLSST algorithm, the MUDB contains the 11 and 12 μm radiances measured by the instrument in channels 31 and 32, and a number of other fields relevant to the SST retrieval. Each entry in the MUDB includes assessment of the confidence in the retrieved value expressed by the quality flag (Q). The best match-ups have Q = 0 and the slightly worse Q = 1 (these are match-ups that would otherwise be assigned Q = 0 but for the most part correspond to large satellite zenith angles meaning long atmospheric path lengths). Quality flags > 1 typically indicate pixels likely contaminated by radiance from clouds or dense aerosols and are not considered reliable. Here we only consider match-ups with Q = 0 or 1. For the purpose of this study we use a random sample of 30,595 Q = 0 data points (out of a random sample of 100,000 Q = 0 data points, 30,596 fall into the geographical area under consideration) and a random sample of 15,749 Q = 1 data points (out of 50,000).

5.2. Prior Knowledge

Our experience with the NLSST algorithm tells us that the radiances measured in MODIS channels 31 and 32 are most sensitive to the SST and the channel radiance difference is most sensitive to the TCWV [11,19]. With an ideal (error free) measurement of just two quantities one could hope to retrieve at most two components of the state vector, so we limit the representation of the state to a reduced state vector x = [SST, TCWV].

Our prior knowledge is the SST and TCWV fields taken from the European Center for Medium-Range Weather Forecasts (ECMWF) ERA 5 dataset, the fifth generation of ECMWF atmospheric reanalyses of the global climate [34]. Thus, we assume that the prior pdf of x is Gaussian with the mean [SST0, TCWV0] given by ERA5 and some prior variance that we will have to specify.

There are two relevant sea surface temperature fields in the ERA5 data set. First, there is the foundation SST (“sst” in ERA5) defined as temperature at a sufficient depth that there is no diurnal heating and which is taken from the Operational Sea-Surface Temperature and Sea Ice Analysis (OSTIA) analysis [35,36]. Under moderate to high wind speeds, and at night, this SST should best represent the in situ buoy measurements that are also made at depth (typically 20 cm) but it differs from the SST_skin obtained by satellite measurements that are sensitive to the temperature of the top few tens of micrometers of the ocean surface. The average offset has been estimated to be −0.17 K [37]. Thus, we use this value with the ERA5 SST to obtain the prior SST_skin. During daytime, the foundation temperature might not represent the subskin ocean temperature since diurnal heating which can reach several degrees or more [38,39,40] is not included. Using the foundation temperature as a prior in OE, even if corrected for the skin effect, will then necessarily lead to a biased OESST, especially for the daytime retrievals. A solution to this problem would be to correct the foundation temperature for the diurnal signal. To evaluate the diurnal correction, we drew random samples of 50,000 data points for each Q = 0 and Q = 1 subsets of the MUDB and examined the differences between the ERA5 SST and the in situ buoy measurements SSTb (both adjusted to skin temperature using the −0.17 K offset) separating nighttime and daytime measurements. In situ measurements from drifting or moored buoys and from ship mounted radiometers serve to validate the satellite retrieved SST and ultimately the results of the OESST retrieval will be assessed by comparison to these in situ measurements. If there is a bias between the prior SST and the in situ measurements, the OE results also will be biased so it is important to remove the bias before the retrieval.

One would expect the diurnal heating applied to the foundation SST to depend on time of the day or correspondingly on solar zenith angle, but this relationship turns out to be quite weak for the MODIS AQUA dataset. This could be due to the sun-synchronous orbit of the AQUA satellite leading to the MODIS data being acquired over a narrow range of local times (around 01:30 and 13:30 at the Equator). As a result, we will use a simple nighttime and daytime offset to account for the missing diurnal signal of the ERA5 SST foundation temperature. The offsets are calculated as an average of the robust mean differences between ERA5 SST and the in situ measurements of the two random samples drawn from the MUDB set. The nighttime correction to the ERA5 SST is −0.015 K and the daytime correction is −0.220 K.

The ERA5 data also contain a skin temperature variable (“skt”), which we denote here as SKT to differentiate it from the foundation SST. This product is based on a model that describes the difference between skin and foundation temperature as a combination of the cool skin effects and diurnal warming effects [40,41]. Most previous OESST retrieval studies and implementations are based on using the foundation temperature reduced by skin effect as the prior SST. Merchant et al. [30] use a correction for the skin effect of −0.2 K; here, as given above, we follow Donlon et al. [37] and use −0.17 K. In principle, the SKT should be a better prior for the OE retrieval so here we examine the utility of both of these fields.

For the purpose of the forward radiative transfer modeling, we also obtain from the ERA5 the atmospheric profiles of the temperature, water vapor, and ozone. The spatial resolution of the data is 0.4 × 0.4 degree with a temporal resolution of 6 h for the atmospheric profiles and SKT, but daily for the foundation temperature. ERA5 data can be downloaded through the Copernicus Climate Data Store (CDS) infrastructure. More details about the ERA5 SST product can be found in Hirahara et al. (2016) [42].

5.3. Forward Model

As the forward model we use the Radiative Transfer for TOVS (RTTOV) v12.1 [43,44], where TOVS means the TIROS Operational Vertical Sounder, and TIROS means Television InfraRed Observation Satellite. While initially developed to simulate the measurements of TOVS, it is now used to simulate the measurements of many satellite radiometers, including MODIS on Aqua. For every match-up in the MODIS MUDB we find corresponding ECMWF data by interpolating the model fields to the time and position of the matched-up measurement. Using RTTOV we compute MODIS channel 31 and 32 radiance measurements as well as the Jacobian matrix K. The forward model is of course only an approximation and thus is an additional source of error.

5.3.1. Measurement Error Covariance S_ϵ

We followed Merchant et al. [30] in assuming that the measurements in channels 31 and 32 are independent and include the forward modeling errors (also independent) in the combined covariance matrix. If ξ = ξ₃₁ = ξ₃₂ is the uncertainty of radiance measurement in MODIS channels 31 and 32 and ξ_m is the forward model error in these two channels, then

$$S_{\mathsf{ϵ}}=\left[\begin{array}{cc}{e}_{31}^2& 0\\ 0& e_{32}^2\end{array}\right]$$

(9)

where

$$e_{31}^2=e_{32}^2={\mathsf{\xi }}^2+{\mathsf{\xi }}_m^2\sec {(\theta )}^2$$

(10)

and θ is the satellite zenith angle. The MODIS instrument specifications list the NeDT in IR channels at about 0.05 K which, for wavelengths of 11–12 μm, corresponds to 0.007 W/(m² sr μm) in radiance units which we take as the estimate of ξ. Merchant et al. [30] estimate ξ_m to be ~0.15 K for the Advanced Very High Resolution Radiometer (AVHRR) on the first Meteorological Operational satellite (Metop-A) of the European Organisation for the Exploitation of Meteorological Satellites (EUMETSAT). Our comparison of RTTOV simulations of MODIS channel radiances with ERA5 atmospheric profiles and SST taken from in situ buoy measurements with MODIS measurements for a sample of 29,515 quality 0 matchups yields a root mean square of the difference of 0.02 W/m²-μm-sr which we take to represent the ξ_m.

5.3.2. Prior Covariance S_a

While the ERA5 SST and TCWV provide the mean values of the prior SST and TCWV distributions, the prior error covariance must still be specified. While not necessarily true, the SST and TCWV are typically considered independent and thus the error covariance matrix S_a can be represented by a diagonal matrix with variance of prior SST and TWC along the diagonal. Variance, or standard error of either SST or TCWV are not part of the ERA5 data so we need to estimate them separately. As stated above, the ERA5 SST is based on the Operational Sea Surface Temperature and Ice Analysis (OSTIA). OSTIA uses satellite SST data provided by international agencies via the Group for High Resolution SST (GHRSST) to produce daily global SST fields. The performance of OSTIA is assessed by comparison with in situ SST measurements. The average standard deviation of the differences between the OSTIA and in situ SST from Argo floats is ~0.5 K for the area of the Northern Atlantic Ocean although it varies regionally and can be much larger in areas of high SST variability [36]. For the subset of the MUBD dataset considered here we calculated the robust standard deviation of the difference between the ERA5 SST and the in situ temperature measured by buoys and obtained values ~0.5 K, in agreement with the evaluation of Good et al., (2020) [36]. We use robust statistics to avoid contamination by large outliers as these are almost certainly spurious. We take this standard deviation to represent the covariance of the prior SST, e_SST² ₌ 0.5² K². This value is in line with values used by others, e.g., [30].

The error in the prior TCWV is harder to estimate. Chen and Liu [45] compared TCWV derived from ECMWF with radiosonde measurements over the period of 1979 to 2014 and with GPS and Microwave Satellite measurements over the years 2000 to 2014, and 1992 to 2014, respectively. They found RMS differences between the ECMWF and the three types of measurements in range between 7% to 20% in global average with the smallest values for the satellite microwave measurements. Merchant et al. [30] used numerical weather prediction (NWP) forecast fields from the ARPEGE (Action de Recherche Petite Echelle Grande Echelle) operational model of Météo-France and assumed e_TCWV = 0.25 × TCWV.

We settle for e_TCWV = 0.20 × TCWV after Chen and Liu [45] but we will explore how different values of e_TCWV and e_SST affect the retrieval. As stated before, we assume the SST and TCWV to be independent and thus the prior covariance matrix is:

$$S_{\mathrm{a}}=\left[\begin{array}{cc}{e}_{SST}^2& 0\\ 0& e_{TCWV}^2\end{array}\right]$$

(11)

6. Information Content Aspects of the OE Retrieval

6.1. Degrees of Freedom

In the case of two-channel measurements in the SST retrieval, if there were no measurement error, we should be able to determine exactly two independent variables: SST and TCWV. This is a problem with two degrees of freedom. However, given measurement error, the retrieved values will have some uncertainty associated with them and thus the effective number of degrees of freedom will be less than two. This can be seen as a component of the state whose variability is smaller than the measurement error and cannot therefore be distinguished from other states. Often the variability of the state components cannot be directly compared with the measurement error as they are in different spaces, so some transformation is necessary. The aim is to compare the measurement covariance with the prior covariance of the forward model simulated measurement. The effective number of degrees of freedom is the number of singular values of matrix

$$\tilde{K}=S_{\mathsf{ϵ}}^{-1/2}K{S}_{\mathrm{a}}^{1/2}$$

(12)

which are greater than about 1.

The number of degrees of freedom can further be partitioned into those related to the signal (ds) and those related to measurement noise (dn). The ds and dn can be expressed in terms of the singular values of the matrix $\tilde{K}$ as

$$ds={\displaystyle \sum }_{i=1}^m\frac{{\lambda }_i^2}{1+{\lambda }_i^2}$$

(13)

and

$$dn={\displaystyle \sum }_{i=1}^m\frac{1}{1+{\lambda }_i^2}$$

(14)

where λ_i are the singular values of matri $\tilde{K}$x, and m is the number of independent measurements.

6.2. Shannon Information Content

The Shannon information content is a concept in information theory that relates to the entropy of the probability density function. Qualitatively, it can be thought of as a factor by which knowledge of a quantity is improved by making a measurement. In our case, the question is whether the satellite IR measurement improves our knowledge of SST over our prior knowledge. Formally, if P₁(x) is a pdf that describes knowledge before the measurement and P₂(x) describes knowledge after the measurement, and if S is entropy, then the information content of the measurement is the reduction in entropy, H:

$$H=S\left(P_1\right)-S\left(P_2\right)$$

(15)

For Gaussian pdfs

$$S\left(P\right)=\frac{\ln \left(\left|S\right|\right)}{2}+const$$

(16)

where |S| is the determinant of the covariance matrix. And so

$$H=-\frac{\ln \left(\left|{\widehat{S}}^{-1}{S}_a\right|\right)}{2} .$$

(17)

The Shannon information content can also be expressed in terms of the singular values of matrix $\tilde{K}$,

$$H=1/2{\displaystyle \sum }_{i=1}^m\ln \left(1+{\lambda }_i^2\right)$$

(18)

A much more detailed explanation and discussion of the concepts summarized above is given by Rodgers [29].

6.3. Retrieval Potential

The covariance matrices S_a and S_ϵ together with the Jacobian matrix K determine the retrieval process (i.e., how much weight is assigned to prior state vs. the measurement through Equation (8), so once these covariance matrices are known and the Jacobians are calculated, we can glimpse what we can expect from the OE in the particular retrieval problem we are considering. While the interpretation of scalar weight is straightforward it is less so for matrix weights appearing in Equation (8). However, since we are dealing with just two variables it is easy to write out the matrix equation explicitly, thus

$$\left(\begin{array}{c}SS{T}_{OE}\\ TCW{V}_{OE}\end{array}\right)=\left(\begin{array}{c}{A}_{11}SST+\left(1-A_{11}\right)SS{T}_a+A_{12}\left(TCWV-TCW{V}_a\right)\\ A_{22}TCWV+\left(1-A_{22}\right)TCW{V}_a+A_{21}\left(SST-SS{T}_a\right)\end{array} \right)$$

(19)

and since we are interested primarily in SST we need to focus on just two elements of the A matrix A₁₁ and A₁₂.

The values of S_a and S_ϵ laid out in Section 5 represent what we consider the most likely values of these parameters for our case of SST retrieval from MODIS. We will assess the relative contributions of the true and prior state to the OE retrieval for this case by examining the elements of matrix A, but we will also consider how these contributions change if the conditions of the retrieval change. In particular, we will consider two additional scenarios: (a) Poorer/better model or measurement scenario when either the radiative transfer model used or the instrument is less/more accurate which will affect the matrix S_ϵ; and (b) when the a priori state covariance is smaller/larger than in our case (different a priori fields could be used in the retrieval).

Figure 1 shows how the two elements of the averaging matrix respond to changes in S_ϵ and S_a.

S_ϵ will change if either ξ_m or ξ_31,32 change. We will only consider changes in ξ_m as (1) ξ_31,32 come from instrument specification and for given instrument should not be a subject to change; and (2) the effect of changes in either ξ_31,32 or ξ_m on matrix A will follow a similar pattern. On the other hand, ξ_m could change if, for example, a different RTM were to be used to compute the simulated radiances. Figure 1a,b show how A₁₁ and A₁₂ depend on ξ_m, Figure 1c,d show dependence on e_SST, and Figure 1e,f on e_TCWV. The red dots in the plots indicate values for the parameters we will use in this study as described in Section 5. For these values, as can be seen in Figure 1, the results of the OE retrieval will be dominated by the true state with only a small contribution coming from the prior state. This corresponds to only a weak regularization in Equation (5) and the OE solution being quite sensitive to measurement errors.

In Figure 2 we consider a range of different retrieval scenarios. Figure 2a,b show A₁₁ and A₁₂ for a range of ξ_m and e_SST values, Figure 1c shows logarithm of the condition number of the matrix that is to be inverted in Equation (5), and Figure 1d shows the standard deviation of the retrieved SST distribution. The dashed lines are isolines of e_SST; the red line and the red dot are those from Figure 1 representing the values accepted as most likely in our case. As expected, for any value of e_SST, an increase in ξ_m (i.e., a less accurate RTM model) will lead to an increased contribution to the OESST from the prior SST and as the prior SST becomes more accurate, the measurement becomes less relevant. The contribution from the cross term initially increases as the model or measurement becomes less accurate but then drops and the more accurate prior SST lowers the impact of this term on the final retrieval. This contribution will also be small if the prior TCWV is close to the true value independent of the ξ_m or e_SST. Larger values of ξ_m or e_SST also lead to a stronger regularization of the retrieval problem as expressed by the condition number in Figure 2c. The standard variation of the posterior SST distribution is for small e_SST independent of the ξ_m as expected, since the solution is almost entirely determined by the prior state, it then increases with ξ_m as e_SST increases.

The red dots in Figure 2 indicate that for our particular retrieval problem we expect the solution for SST to be driven by measurements with only a weak influence of the prior state. Where the prior state contributes is to the retrieval of the TCWV as can be seen in Figure 3 which shows the weights A₂₂ and A₂₁.

7. The Retrieval Process and Results

7.1. Retrieval with ERA5 “sst” as Prior SST

Figure 4a,b shows histograms of (a) nighttime, and (b) daytime OESST–SSTb for ERA5 SST0 prior with the nighttime or daytime correction. The average statistics of the OESST–SSTb comparison given in the plots (figures in the brackets are robust values) are very good but they are not very different from the SST0–SSTb (see Figure 6a,b in Section 7.2). Figure 4c,d shows corresponding night and daytime histograms for the NLSST–SSTb. The main motivation behind developing new retrieval methods is to improve the accuracy of the retrievals especially for cases where the established methods perform poorly and there is an improvement in the robust statistics for our sample when going from NL to OESST although this is very small for daytime matchups. However, in comparison with NLSST most of the improvement appears to take place near the center of the histograms, i.e, the accuracy of OE is improved mostly for cases when the NL atmospheric correction is already effective. But the tails of the histograms appear to extend further for the OE than the NLSST.

Figure 5 illustrates the fraction of the dataset with absolute value of the SST–SSTb above a given threshold for OESST, NLSST, and also for ERA5 and ERA5 with the night/day correction. For both the (a) nighttime and (b) daytime retrieval, OE does the best job for absolute differences between the retrieval and the in situ temperature below about 0.8 K; however, it does only slightly better than the corrected ERA5 prior. The fraction of retrievals with differences larger than 0.8 K is smallest for the NLSST. In total, about 76% of the OESST data fall outside of the 0.1 K range of CDR target and about 55% fall outside the 0.2 K accuracy of the drifter buoy measurements. The corresponding percentages for the NLSST retrievals are 82% and 65% so with respect to this metric there is an advantage in using OESST rather than the NLSST.

7.2. Using ERA5 “skt” as Prior SST

The ERA5 Skin Temperature variable might seem a more appropriate prior SST than the foundation temperature and it should be able to be used directly without the need for corrections for the skin effect or for diurnal signals (although with the SKT only available four times a day interpolation in time might still be necessary). The SKT, however, is a modeled product that comes with its additional uncertainties [46,47]. Previous comparisons of SKT with skin temperature measured by the shipborne Marine-Atmospheric Emitted Radiance Interferometer (M-AERI, [48]) showed average differences of −0.237 K for nighttime and −0.172 K for daytime measurements [47]. These values are not only larger than the CDR requirement for the SST accuracy, but taking into account that the estimated accuracy of the M-AERI SST_skin is better than 0.05 K indicates a real departure of the ERA5 skin temperature model from the measurements. In Figure 6 we compare the nighttime and daytime differences between ERA5 SST (corrected for the skin effect and with added diurnal correction) and ERA5 SKT and the SSTb in our match-up dataset. Both nighttime and daytime SKT–SSTb differences are larger than SST–SSTb differences which makes the ERA5 SKT a poorer candidate for the SST0 in the OESST retrievals than the ERA5 SST, perhaps a surprising result.

7.3. Geographical Distribution of ERA5 SST0–SSTb

We focus on the large differences between the ERA5 SST0 and the buoy measurements since these contribute most to the OESST–SSTb offset, and so are the retrievals in most need of improvement. Figure 7 shows a map with the locations of match-up points colored by the value of ERA5 SST0–SSTb over contours of SST0 for 1 January 2015, the first day of our dataset. (While the contours of SST0 are for one time instance, the SST0–SSTb are for the entire subset of the matchups.) Panel (a) of Figure 7 shows locations of SST0–SSTb > 1 K differences, and panel (b) the locations of match-ups with negative SST0–SSTb < −1 K. These data points correspond to the tails of the distributions in Figure 4. For comparison, match-ups with |SST0–SSTb| < 0.2 K are shown in panel (c). Different symbols correspond to different types of in situ measurements: Moored buoys, drifting buoys, and ship radiometers.

Figure 7 reveals that the majority of match-ups with large |SST0–SSTb| are found in ocean areas with strong horizontal gradients in SST. Since ERA5 SST represents the temperature of an area of 0.4 × 0.4 degrees in latitude and longitude, it might not represent well any individual buoy, especially those in areas with strong horizontal gradients. In such conditions the OESST might not provide good estimates of the SST. In

Table 1. (a) Statistics of retrieved SST–SSTb differences in bins of SST0–SSTb for Q = 0 nighttime match-ups. Temperature differences are in K. OE StDev is the retrieved standard deviation of OESST distribution and ΔNLSST is the estimate of the error of NLSST obtained by propagation of error through the NLSST equation. (b) As (a), but for daytime.

a
SST0–SSTb Bins	SST0–SSTb			OESST–SSTb				NLSST–SSTb				N Cases
	Mean	RMS	Std	Mean	RMS	Std	OE StDev	Mean	RMS	Std	ΔNL SST
−5.0 > −1.0	−1.597	1.666	0.472	−1.516	1.583	0.456	0.055	−0.067	0.743	0.740	0.284	929
−1.0 > −0.5	−0.697	0.711	0.141	−0.678	0.697	0.161	0.081	−0.225	0.712	0.675	0.304	1057
−0.5 > −0.25	−0.350	0.357	0.070	−0.348	0.364	0.108	0.107	−0.285	0.616	0.546	0.329	2147
−0.25 > −0.1	−0.170	0.175	0.042	−0.170	0.192	0.088	0.117	−0.218	0.534	0.487	0.340	2720
−0.1 > −0.05	−0.075	0.076	0.014	−0.081	0.113	0.079	0.117	−0.201	0.526	0.486	0.339	1126
−0.05 > 0	−0.025	0.029	0.014	−0.034	0.091	0.085	0.122	−0.171	0.522	0.493	0.342	1180
0 > 0.05	0.025	0.029	0.015	0.011	0.080	0.079	0.115	−0.144	0.490	0.490	0.338	1128
0.05 > 0.1	0.075	0.076	0.014	0.057	0.102	0.084	0.118	−0.128	0.469	0.469	0.340	1058
0.1 > 0.25	0.169	0.174	0.043	0.147	0.170	0.085	0.116	−0.111	0.472	0.472	0.340	2453
0.25 > 0.5	0.353	0.360	0.070	0.312	0.328	0.099	0.105	−0.042	0.494	0.494	0.332	2102
0.5 > 1.0	0.680	0.694	0.137	0.609	0.626	0.157	0.083	0.056	0.573	0.573	0.311	1038
1.0 > 5.0	1.824	1.987	0.789	1.687	1.847	0.745	0.063	0.386	0.916	0.916	0.284	569
b
SST0–SSTb Bins	SST0–SSTb			OESST–SSTb				NLSST–SSTb				N Cases
	Mean	RMS	Std	Mean	RMS	Std	OE StDev	Mean	RMS	Std	ΔNL SST
−5.0 > −1.0	−1.627	1.710	0.525	−1.490	1.569	0.492	0.067	0.168	0.715	0.695	0.281	932
−1.0 > −0.5	−0.689	0.703	0.137	−0.649	0.677	0.193	0.102	−0.056	0.780	0.778	0.313	996
−0.5 > −0.25	−0.360	0.367	0.070	−0.345	0.367	0.130	0.122	−0.178	0.627	0.602	0.335	1537
−0.25 > −0.1	−0.170	0.176	0.043	−0.164	0.200	0.113	0.124	−0.147	0.556	0.536	0.338	1757
−0.1 > −0.05	−0.075	0.077	0.015	−0.071	0.126	0.104	0.123	−0.124	0.514	0.499	0.335	666
−0.05 > 0	−0.025	0.029	0.014	−0.019	0.099	0.097	0.121	−0.076	0.581	0.576	0.334	771
0 > 0.05	0.025	0.029	0.015	0.017	0.108	0.107	0.122	−0.092	0.499	0.499	0.335	785
0.05 > 0.1	0.075	0.077	0.014	0.070	0.123	0.101	0.119	−0.055	0.508	0.508	0.335	750
0.1 > 0.25	0.171	0.176	0.042	0.155	0.181	0.111	0.110	−0.016	0.492	0.492	0.329	1874
0.25 > 0.5	0.355	0.362	0.070	0.330	0.351	0.118	0.098	0.085	0.545	0.545	0.317	1711
0.5 > 1.0	0.673	0.686	0.133	0.625	0.646	0.167	0.084	0.239	0.664	0.664	0.293	840
1.0 > 5.0	1.777	1.943	0.786	1.651	1.816	0.754	0.062	0.549	1.075	1.075	0.264	453

we compare the statistics of OESST–SSTb with NLSST–SSTb in bins of SST0–SSTb for night (a) and daytime (b) Q = 0 measurements to see if and under which conditions the NLSST can provide more accurate estimates of the SST than the OE. The StDev for OE retrievals is calculated as the square root of the OESST variance given by Equation (6) with e_SST = 0.5 K. The ΔNLSST is an estimate of NLSST measurement error calculated through the propagation of the uncertainties in the MODIS channel measurements, e₃₁ and e₃₂, through the NLSST equation given in Equation (1). The graphical representation of the mean, RMS, and standard deviation from Table 1 daytime data is shown in Figure 8 (nighttime data follow a very similar pattern and are not shown).

As can be seen in Table 1 and Figure 8 in terms of mean value, OE retrievals are more accurate than those of the NLSST provided the difference between the prior SST and buoy measurements is in the range ± 0.1 K for daytime, with the exception of nighttime negative differences where OE slightly outperforms NLSST up to a difference of −0.25 K and daytime positive differences where the NLSST is better starting bin 0.05 to 0.1 K. The RMS values are consistently smaller for the OE up to ± 1 K difference between SST0 and SSTb. Additionally, the mean value NLSST–SSTb appears to be largely independent of the SST0–SSTb and although RMS does increase with increasing SST0–SSTb, it does so at a slower rate than OESST–SSTb. These results indicate that in areas of the ocean with large SST variability or where strong horizontal SST gradients are prevalent, the OE approach will not necessarily improve on the NLSST if the a priori SST field does not correctly capture the variability. Unlike the diurnal bias, this spatial sampling bias is more difficult to correct, other than improving the spatial resolution of the prior SST0.

7.4. Information Measures

As outlined in Section 5 we calculated the number of degrees of freedom ds and the Shannon information content H for the retrieval with ERA5 SST0 with the diurnal correction and with e_SST = 0.5 K. The average ds is 1.28 with standard deviation of 0.25 indicating that in practice only one element of the state vector can be retrieved.

The average Shannon information content of the retrieval is 4.69 with standard deviation of 0.47 for SST but just 0.30 and 0.30 for TCWV. So, while the measurement does improve our knowledge of the SST with respect to the a priori, it does not contribute significantly to our knowledge of the TCWV.

8. Comparison with NLSST

Table 1 demonstrates that OESST can be quite erroneous compared to buoy measurements if the prior SST0 is similarly biased, and the NLSST can provide more accurate SST estimates in such situations. One of the main reasons for considering the benefit of switching from NLSST to OESST is the prospect of improving on the biases in the NLSST that are due to the coefficients not adequately representing the atmospheric conditions during the measurement.

Table 2. (a) Statistics of retrieved SST–SSTb differences in bins of NLSST–SSTb for Q = 0 nighttime match-ups. OE StDev is the retrieved standard deviation of OESST distribution and ΔNLSST is the estimate of the error of NLSST obtained by propagation of error through the NLSST equation. (b) As (a), but for daytime.

a
NLSST–SSTb Bins	SST0–SSTb			OESST–SSTb				NLSST–SSTb				N Cases
	Mean	RMS	Std	Mean	RMS	Std	OE StDev	Mean	RMS	Std	ΔNL SST
−5.0 > −1.0	−0.240	0.679	0.636	−0.370	0.709	0.604	0.114	−1.415	1.483	0.443	0.329	1009
−1.0 > −0.5	−0.124	0.485	0.469	−0.167	0.469	0.438	0.124	−0.693	0.706	0.138	0.338	2416
−0.5 > −0.25	−0.059	0.507	0.504	−0.080	0.482	0.475	0.110	−0.362	0.369	0.071	0.331	3028
−0.25 > −0.1	−0.045	0.513	0.511	−0.053	0.482	0.479	0.104	−0.174	0.179	0.043	0.329	2416
−0.1 > −0.05	−0.014	0.514	0.514	−0.015	0.478	0.478	0.105	−0.074	0.076	0.015	0.331	850
−0.05 > 0	−0.033	0.600	0.599	−0.033	0.557	0.557	0.102	−0.024	0.028	0.015	0.329	808
0 > 0.05	−0.015	0.577	0.577	−0.018	0.536	0.536	0.097	0.025	0.028	0.028	0.328	739
0.05 > 0.1	−0.014	0.641	0.640	−0.010	0.599	0.599	0.095	0.073	0.075	0.074	0.324	797
0.1 > 0.25	0.025	0.569	0.568	0.031	0.531	0.530	0.098	0.171	0.177	0.177	0.326	1971
0.25 > 0.5	0.069	0.768	0.765	0.081	0.714	0.710	0.098	0.359	0.366	0.366	0.325	2000
0.5 > 1.0	0.046	0.897	0.896	0.068	0.838	0.835	0.094	0.679	0.693	0.693	0.321	1190
1.0 > 5.0	0.508	1.504	1.416	0.539	1.437	1.332	0.081	1.496	1.633	1.633	0.391	292
b
NLSST–SSTb Bins	SST0–SSTb			OESST–SSTb				NLSST–SSTb				N Cases
	Mean	RMS	Std	Mean	RMS	Std	OE StDev	Mean	RMS	Std	ΔNL SST
−5.0 > −1.0	−0.272	0.621	0.559	−0.386	0.651	0.523	0.120	−1.319	1.38	0.363	0.321	585
−1.0 > −0.5	−0.158	0.538	0.515	−0.204	0.524	0.482	0.127	−0.707	0.721	0.142	0.332	1563
−0.5 > −0.25	−0.086	0.523	0.516	−0.107	0.493	0.481	0.113	−0.364	0.371	0.072	0.326	1862
−0.25 > −0.1	−0.053	0.523	0.521	−0.053	0.493	0.487	0.107	−0.173	0.178	0.042	0.326	1508
−0.1 > −0.05	−0.029	0.532	0.531	−0.036	0.499	0.498	0.104	−0.074	0.076	0.015	0.323	585
−0.05 > 0	−0.049	0.552	0.550	−0.038	0.516	0.514	0.106	−0.023	0.028	0.015	0.326	636
0 > 0.05	−0.053	0.535	0.532	−0.051	0.507	0.505	0.102	0.025	0.028	0.028	0.328	514
0.05 > 0.1	−0.094	0.709	0.702	−0.068	0.650	0.646	0.105	0.073	0.074	0.074	0.322	590
0.1 > 0.25	−0.038	0.607	0.605	−0.015	0.562	0.561	0.100	0.173	0.178	0.178	0.318	1622
0.25 > 0.5	−0.004	0.772	0.772	0.022	0.715	0.715	0.101	0.363	0.367	0.370	0.319	1735
0.5 > 1.0	0.023	0.955	0.955	0.069	0.886	0.883	0.100	0.692	0.705	0.705	0.314	1330
1.0 > 5.0	0.146	1.206	1.200	0.232	1.134	1.110	0.089	1.570	1.688	1.688	0.293	542

presents the improvement of OESST over NLSST in bins of NLSST–SSTb similar to Table 1 for (a) night and (b) daytime Q = 0 measurements. Mean, RMS, and standard deviation values from Table 2b are shown in Figure 9.

In most cases the mean value of |OESST-SSTb| is indeed smaller than |NLSST-SSTb| which could be considered an improvement as, on average, the OESST retrievals appear to agree better with the in situ measurements than those of the NLSST algorithm. However, in the majority of cases, SST0 agrees with the in situ SST even better than do the OESST retrievals which puts in question the need of performing the OE retrieval where SST0 values are available and understood to be accurate. The smaller mean values of |OESST–SSTb| (and |SST0–SSTb|) are accompanied by large values of the RMS error indicating large scatter about the mean. The RMS errors are typically significantly smaller for the NLSST–SSTb for all bins up to [−0.5, −0.25] K on the negative side and up to the last bin of [1.0, 5.0] K for the positive differences daytime data, and two last bins for the positive differences nighttime data.

Figure 10 shows the geographical distribution of (a) positive, and (b) negative NLSST–SSTb greater or more negative than ±1 K, respectively. It appears that the retrievals with large positive differences are distributed similarly to the large SST0–SSTb differences, while the large negative differences are distributed quite uniformly throughout the whole domain. This might indicate different causes for the positive and negative biases in the NLSST retrievals: Possibly cloud contamination causing the negative bias and land influence on the coastal atmospheres leading to a positive bias.

As shown in Section 6, the OE cannot work well for situations with large SST0–SSTb. However, since SSTb is not available for all satellite measurements, one cannot rely on this metric to determine if the OE approach is beneficial in any particular case. Figure 11 shows the dependence of OESST–SSTb and NLSST–SSTb on NLSST–SST0. It reveals a strong, almost linear relationship between OESST–SSTb and NLSST–SST0 with a correlation coefficient of −0.64 indicating that NLSST–SST0 might be used to determine whether the OE is likely to improve on the NLSST retrieval. There is also a correlation between NLSST–SSTb and NLSST–SST0 but it is slightly weaker with a correlation coefficient of 0.60.

9. Discussion and Conclusions

New approaches to the retrieval of SST from satellite measurements are introduced to improve on the existing algorithms in terms of increasing the accuracy of the retrievals, eliminating known biases and/or expanding coverage. The main promise of the OE approach is that it would eliminate, or at least reduce, the bias of the NLSST retrievals due to the inability of the NLSST atmospheric correction algorithm to compensate for the effects of atypical atmospheric conditions. However, the OE method comes with its own set of problems; a thorough review is given by von Clarmann et al. [49].

Firstly, the OE approach requires an unbiased prior SST field. In practice, the prior SST comes from either numerical reanalysis, as in our case, or from numerical weather prediction models such as used, for example, by Merchant et al. [30,50] or from operation applications. These prior SST fields are limited in both temporal and spatial resolution, typically being issued 4 times daily and with spatial resolution of tenths of a degree which represents a rather large surface compared to ~1 × 1 km typical pixel size of satellite infrared radiometers or a point buoy measurement. Thus, the prior fields are inherently likely to be biased and so, before OE can be attempted, the biases should be removed. Merchant [32] applies the OE to estimate the prior biases and possible measurement bias before applying the OE to retrieve SST from SEVIRI measurements. One could ask if, after that step, is the OE retrieval still necessary? Here, we estimated the biases directly to apply corrections to the a priori SST field. The results are summarized:

(a)

Through retrievals using situ buoy measurements used as the prior SST in the OE, we estimated that in our application the biases not related to those in the prior SST are on order of 0.02 K, which is small in comparison to the accuracy of buoy measurements of ~0.2 K [51,52,53,54] or the requirement of a CDR of SST of 0.1 K [5]. Newer surface drifters provide more accurate subsurface temperature measurements, with a target of 0.01 K [55,56], but thus far relatively few have been deployed.

(b)

It is relatively simple to estimate the consequences of neglecting diurnal signals that are missing from the foundation SST of the ERA5 data. A significant improvement in daytime OE SST retrievals is achieved by applying a simple diurnal correction, reducing the mean difference between OESST and buoy measurements from −0.25 K to −0.05 K.

(c)

Perhaps, somewhat surprisingly, this simple correction to ERA5 foundation SST provides better prior for the OE retrieval than the ERA5 skin temperature (SKT). The SKT–SSTb bias is larger than the ERA5 SST–SSTb bias even before the diurnal correction, and for both nighttime and daytime matchups.

(d)

Despite the improvement in the mean after application of the diurnal correction, there remain large values of OESST–SSTb. The large disagreements between OESST and SSTb occur mostly in the areas of the ocean where there are large horizontal gradients in the SST, i.e., coastal areas or at the surface expressions of oceanic thermal fronts. The limited spatial resolution of the numerical models providing prior SST fields often fails to adequately represent the spatial variability of SST in those areas. The spatial sampling bias is difficult to correct without an improvement in the resolution of the prior SST. We use the nearest ERA5 SST value for the a priori and more advanced interpolation could improve the estimates but might also introduce undesirable artifacts in areas of strong spatial variability.

(e)

In areas with strong horizontal thermal gradients, the NLSST often provides better estimates of SST than the OE approach.

(f)

In general, OESST improves on NLSST in terms of mean offset from in situ measurements, but not for the RMS, and improvement in the mean is driven mostly by the accuracy of the underlying prior SST with the OE retrieval not departing much from the prior.

(g)

The OESST departure from the prior is on average quite small so if the SST0–SSTb is large then the agreement between the retrieval and the in situ is not much improved.

(h)

On average, for very large differences between NLSST and in situ measurements, the OE provides a better estimate of SST but the OESST retrieval is not as accurate as the corrected prior.

In conclusion, we find that OE can improve estimates of SST from MODIS top-of-atmosphere channel radiance measurements if the biases are effectively removed from the prior SST field. This is not an easy task when it comes to biases resulting from spatial sampling, and often in areas of large SST variability the NLSST provides better retrievals than the OE. We proposed a diagnostic test to determine if the OE approach would be useful by evaluating the difference between the NLSST and the SST0: Cases with large NLSST–SST0 are not likely to lead to improved SST estimated by OE.

One of the main aspirations for the OE was to improve SST retrievals in situations where the atmospheric conditions depart from those implicit in the derivation of the NLSST retrieval coefficients, such as in areas of the oceans affected by large amounts of terrestrial aerosols that frequently move offshore typically from desert areas, and may span ocean basins having marked temporal and spatial variability [57,58,59,60]. The presence of aerosols in the marine atmosphere influences the IR radiation measured by satellite sensors. However, in the OE approach with the reduced state vector [SST, TCWV], this aerosol effect is not accounted for in the differences between the measured and modeled radiances which leads to a bias in the retrieved SST, as does using non-optimised coefficients lead to a bias in the NLSST algorithm in such conditions. Thus, to improve the OESST retrievals under aerosol layers, the OE would need to account explicitly for the presence of aerosols. That would of course also bring new uncertainties into the problem.

In view of these OE limitations, OESST might not always provide the best estimates of the SST. In particular, research related to smaller scale spatial variability of thermal currents, or SST in coastal zones may benefit from utilizing NLSST-generated satellite SST fields. Moreover, given the importance of oceanic fronts on atmospheric circulation especially in relation to extratropical cyclones and atmospheric rivers that develop mostly along the sharp fronts on SST [61,62,63,64] and the role of thermal fronts in marine ecosystems [65], the NLSST seems better suited to the detection of such fronts. This is of even greater importance as the positions of oceanic fronts change on several different time scales [66] and possibly shift in response to climate change [67,68].