How Efficient Is Model-to-Model Data Assimilation at Mitigating Atmospheric Forcing Errors in a Regional Ocean Model?

Abstract

This paper examines the efficiency of a recently developed Nesting with Data Assimilation (NDA) method at mitigating errors in heat and momentum fluxes at the ocean surface coming from external forcing. The analysis uses a set of 19 numerical simulations, all using the same ocean model and exactly the same NDA process. One simulation (the reference) uses the original atmospheric data, and the other eighteen simulations are performed with intentionally introduced perturbations in the atmospheric forcing. The NDA algorithm uses model-to-model data assimilation instead of assimilating observations directly. Therefore, it requires a good quality, although a coarser resolution data assimilating parent model. All experiments are carried out in the South East Arabian Sea. The variables under study are sea surface temperature, kinetic energy, relative vorticity and enstrophy. The results show significant improvement in bias, root-mean-square-error, and correlation coefficients between the reference and the perturbed models when they are run in the data assimilating configurations. Residual post-assimilation uncertainties are similar or lower than uncertainties of satellite based observations. Different length of DA cycle within a range from 1 to 8 days has little effect on the accuracy of results.

1. Introduction

In recent years, the improvement in ocean models and the use of more powerful computers have allowed ocean predictive models to be run at sub-mesoscale resolutions and better represent smaller-scale ocean features such as eddies, fronts, and filaments, e.g., [1] and references therein. Regional ocean forecasting models are usually driven by a continuous supply of time-varying external data, such as atmospheric forcing and lateral boundary conditions. These external data are produced by mathematical models and are available with a certain degree of uncertainty [2,3,4,5]. In addition to uncertainties in the time –varying external data, the errors in the ocean models come from a range of other sources, such as parameterisation of sub-grid oceanic processes, approximations used in numerical schemes, initial conditions and bathymetry.

The downward solar radiation at the sea surface is an essential component of the total heat exchange between the atmosphere and the ocean. Zhang et al. [2] evaluated errors in the shortwave radiation (SWR) from various reanalysis products using satellite and in situ observations. They found that all the reanalyses tend to overestimate the SWR with global (including both land and ocean) mean biases between the reanalysis and surface measurements ranging from 11.25 W/m² to 49.80 W/m². The best performance was demonstrated by ERA-Interim, with the yearly mean bias over the global ocean being −2.85 W/m² and root-mean-squared error (RMSE) being 21.98 W/m² [2]. The RMSE is quite large relative to the yearly and globally averaged SWR over the ocean of 180.0 W/m². The correlation coefficient between the ERA-Interim and surface observations was R = 0.97, showing that the spatial pattern provided by the reanalysis is close to the observations. The uncertainties in the wind data from 15 global and regional reanalysis products were evaluated by [5]. The wind data are shown to be sufficiently reliable on offshore locations with the median BIAS (MB) = −0.1 m/s and median correlation coefficient CORR = 0.89. However, the RMSE ranged from 1 m/s to 3.2 m/s with the median value of 1.35 m/s.

The effect of uncertainties from the lateral boundaries was studied by [6]. They showed that the data assimilating open boundary condition could reduce the system bias by at least 50%, although it was not enough to constrain the mesoscale circulation in the study area. Ocean models are particularly sensitive to wind stress and heat fluxes at the ocean surface as well as to the horizontal resolution of atmospheric data [7,8]. The uncertainties in the ocean model output derived from errors in the atmospheric forcing have been investigated in a number of studies, e.g., [9,10]. Typically, the surface atmospheric fluxes were perturbed in an ensemble of experiments.

While the study of errors in the atmospheric forcing and sensitivity of an ocean model output to the uncertainty in the external input data is important for understanding the propagation of errors in ocean predictions, a practical ocean forecast would benefit from an efficient method of mitigating such errors. The usual way to improve the ocean model forecast is to use one of the many DA methods, which vary in complexity, effectiveness, and computational cost [11,12,13]. The DA methods have both advantages and disadvantages. For example, the assimilation of only SST in an operational model degrades the temperature/salinity profiles [14]. Data assimilation is deemed to be unsuitable for the study of derived quantities, not constrained by observations, i.e., for which observations are not available [15]. Recent research shows that ‘when only SST was assimilated, it had a negative effect on the subsurface layer of most regions, except for some regions in high-latitude regions’ [16]. Therefore the improvement provided by a specific DA scheme cannot be taken for granted and should be assessed for each DA method.

This paper examines the efficiency of a recently developed model-to-model DA method in mitigating errors in heat and momentum fluxes at the ocean surface. The analysis uses a set of 19 numerical simulations, all using the same ocean model and exactly the same DA process. One simulation (the reference) uses the best available atmospheric data, and other eighteen simulations are performed with various perturbations in the atmospheric forcing. A perfect DA would rectify all the consequences of errors in atmospheric forcing, so that all models would produce identical results. The study evaluates how close the NDA algorithm is to the ideal case. The variables under study are temperature, kinetic energy, relative vorticity, and enstrophy at the sea surface. The surface layer is selected as it is mostly influenced by atmospheric forcing. The simulations are carried out in the dynamically active part of South East Arabian Sea, also known as the Lakshadweep Sea.

2. Materials and Methods

The numerical experiments are conducted in the Lakshadweep Sea within the area of 7.5–14.5° N; 68–78° E, see Figure 1. This part of the South East Arabian Sea provides an important supply of proteins to more than 34 million people of the Indian state of Kerala. An accurate description of the physical environment in the area is important to contribute to the improvement of the livelihood of the people there.

All experiments are based on different configurations of the same regional ocean circulation model called LD20. The details of the LD20 model design, configuration, set-up, operation, DA, and validation against satellite and in situ observations are given in [17]. Below is the summary of the main characteristics of the model. The LD20 model uses NEMO v3.6 codebase [18] and is driven by the atmospheric forcing produced by the UK Met Office Unified Model [19]. The lateral open boundary conditions are taken from the global ocean model available via Copernicus Marine Service product GLOBAL_REANALYSIS_PHY_001_030-TDS [20] with the addition of tidal components from the global tidal model TPXO 7.2 [21]. The parent global model has a horizontal resolution of 1/12 degree and 50 depth levels, and it assimilates observations of sea surface temperature (SST), sea surface height, and in situ temperature and salinity profiles. The LD20 model has a horizontal resolution of 1/20 degree and 50 depth levels, which are different from the levels of the parent model to better represent the dynamics of the study area. Bathymetry for LD20 is adopted from GEBCO [22].

The LD20 model is run in two configurations, with and without DA. The DA configuration employs a model-to-model DA process based on Nesting with the Data Assimilation (NDA) algorithm [23]. This algorithm assimilates observations indirectly by taking the 3D data sets from same parent model used in providing a 2D lateral boundary conditions. Therefore, the coarser resolution parent model acts an intermediate processing layer in the NDA algorithm, which spreads actual observations across the coarse grid. The data taken from the parent model are the profiles of temperature, salinity, and u- and v-components of velocity at each parent model grid point. The commonly used DA processes are known to introduce some imbalance between density and velocity fields, so an additional geostrophic balancing step is required (see, for example, [24]). The data taken from the parent model are constraint by the equations of motion, and they therefore do not require dynamic balancing. The water column data from the parent model are more abundant than observations, and, hence, the ratio of the number of the ‘observational’ and model data points is reduced in the NDA algorithm. In the data assimilating configurations, the DA is carried out every 5 days at 00Z.

The base model LD20_DA was run in the DA configuration with undisturbed forcing from 1 January 2015 to 30 December 2016. The atmospheric forcing is provided with 1 h frequency for wind and 3 h frequency for SWR. The models with disturbed forcing were run for one year from 1 January 2016 to 30 December 2016 both in DA and noDA configurations. It took approximately 2–3 weeks for models with disturbed forcing to spin up, hence model results for January were excluded from further analysis.

The set of perturbed atmospheric forcing consists of the nine versions listed in

Table 1. Description of the perturbed atmospheric forcing.

Version No	Description
V0	Undisturbed forcing
V1	F = 1.10 (constant), A = B = 0;
V2	F = 1.05 + 0.05 × rand (time), A = B = 0;
V3	F = 1.05 (constant), A = B = 0;
V4	F = 0.95 (constant), A = B = 0;
V5	F = 1.20 (constant), A = B = 0;
V6	A = 2.0 × randn (time), B = 0, units = m/s, F = 0;
V7	A = 0, B = 2.0 × randn (time), units = m/s, F = 0;
V8	A = 2.0 × randn (time), B = 2.0 × randn (time), units = m/s, F = 0;
V9	F = 1 + 0.20 × randn (time), A = B = 0;

Here, rand (time) is a uniformly distributed random number in the interval (0,1) which changes once a day at 00:01Z; randn (time) is a random number drawn from the normal distribution with standard deviation std = 1, which changes once a day at 00:01Z. The perturbations cover approximately the range of uncertainties reported in [2,5].

. The SWR is modified in versions v1–v5 and v9, while the components of the wind at 10m above the surface are modified in versions v6–v8. The amplitudes of disturbances for both solar radiation and wind speed were taken to be within the range of the actual uncertainty of these variables, as estimated by Zhang et al. [2] and Gualtieri [5]. Solar radiation is modified by a multiplicative factor F, which is uniform across the domain but may vary randomly with time. The wind components are modified by adding a term A to the u-component and term B to v-component, which are uniform across the domain but vary randomly in time.

The errors in the model output in versions V1-V9 without DA and their reduction by the DA process are assessed by comparison with the validated LD20_DA model running with the original undisturbed forcing. The following field variables computed at the sea surface and used for assessing the errors were temperature, kinetic energy, relative vorticity, and enstrophy. The quality of models in representing large-scale ocean features are assessed by calculating area mean deviations (i.e., biases) and their evolution throughout the period of study. Higher resolution ocean models are known to exhibit spatial shift in representing smaller scale features, leading to a so-called ‘double penalty effect’ [25,26]. The model skills in representing meso- and sub-mesoscale structures are assessed by computing RMSE and pixel-to-pixel correlation coefficient (CORR) for each day. The usefulness of RMSE and CORR for assessing the double penalty effect can be illustrated by the following simple example.

Let us consider a street of cyclonic and anticyclonic eddies (Figure 2a) represented by the equation:

Q(x,y) = sin(x) × sin(y)

(1)

where Q(x,y) is any scalar property, e.g., temperature, salinity, etc., distributed over the horizontal plane.

Let the pattern be shifted in the x-direction due to the double penalty effect (Figure 2b):

Qs(x,y) = sin(x + shift) × sin(y)

(2)

The RMSE between Q and Qs varies from 0 to 1, and CORR varies from 1 to −1 as the shift increases from 0 to π as shown in Figure 2c. Therefore, both RMSE and CORR are good estimators for the spatial shift of ocean features and the ‘double penalty effect’.

3. Results and Analysis

3.1. Sea Surface Temperature

The seasonal variability of the area averaged SST in the Lakshadweep Sea is presented in Figure 3 as a summary time series showing the outputs from the reference and ensemble models. The purpose of this figure is to show a ‘cloud’ of curves produced by different experiments to demonstrate a visual spread of results. The separate curves for individual experiments are shown below in Figure 4.

Deviations of individual members of the ensemble from the reference model are shown in Figure 4. For clarity of presentation, the curves in all of the figures below are smoothed with a 5-day moving average.

As expected, the noDA simulations with persistently enhanced SWR (Figure 4(v1–v3,v5)) show the increase in SST throughout most of the year. The simulation with a symmetric spread of errors Figure 4(v9) shows both positive and negative biases. In some cases (see version 5) the bias is as high as 1 °C. Such a level of errors may be unacceptable for some applications as the range of area averaged SST seasonal variability in the Lakshdweep Sea is only about 3 °C (see Figure 4(v5)). The largest errors are recorded at different times for different ensemble members. In experiments Figure 4(v1,v3,v5), the largest errors are in March and November. These experiments are driven by persistent positive perturbations in the SWR, which are strongly correlated in time. The experiment Figure 4(v4) is driven by negative persistent perturbations and shows the largest errors in June and December. The forcing error in SWR for experiment Figure 4(v2) is partly persistent and partly uncorrelated. The distribution of errors over time is similar to experiments Figure 4(v1,v3,v5), demonstrating that a small random component in the forcing does not change the overall behaviour of the model response. Experiment Figure 4(v9) has the largest random perturbations in the SWR within the ensemble, and it shows the largest positive errors in SST similar to but slightly earlier than Figure 4(v1,v3,v5). It also shows large negative errors in June and December similar to experiment Figure 4(v4), but it has an additional large error in May that is not seen in other experiments.

NoDA simulations with randomly perturbed wind Figure 4(v6–v8) consistently show a cooling of the surface. This is due to the nonlinear effect of the wind speed on the latent heat loss, which causes stronger evaporation when the wind perturbation is in the direction of the original wind. The largest errors are common for versions Figure 4(v6–v8) and are recorded in November. The timings for other large errors do not seem to be correlated between the versions.

The models running in the DA mode have the biases significantly reduced down to 0.2 °C or better for all members of the ensemble. The improvement is the greatest for versions Figure 4(v1–v4) with perturbations of SWR not exceeding 10%. The improvement for Figure 4(v5) (driven by forcing with the largest 20% perturbations in SW) is significant but not as spectacular as for other experiments.

The summary of time series of area averaged SST errors produced by the models with and without DA is shown in Figure 5.

The ability of the NDA algorithm to reduce the errors in the spatial distribution of the field variables is demonstrated in Figure 6. The RMSE and CORR time series are calculated using the equations:

$$RMSE\left(t\right)=\sqrt{\frac{1}{N}{{\displaystyle \sum }}_{n=1}^N{\left[SST\left(t,n\right)-SS{T}_{ref}\left(t,n\right)\right]}^2}$$

(3)

$$CORR\left(t\right)=\frac{1}{N}{{\displaystyle \sum }}_{n=1}^N\left(\frac{SST\left(t,n\right)-SST\left(t,n\right)}{\sigma \left(SST\left(t\right)\right)}\right)\left(\frac{SS{T}_{ref}\left(t,n\right)-SS{T}_{ref}\left(t,n\right)}{\sigma \left(SS{T}_{ref}\left(t\right)\right)}\right)$$

(4)

where n = 1, … N is the model grid node index, N is the total number of wet grid nodes at the sea surface (N = 19,147 for the LD20 model), $SST\left(t,n\right)$ and $SS{T}_{ref}\left(t,n\right)$ are the values of SST at a time point t, and grid node index n for the ensemble member and the reference model, respectively, and $\sigma \left(SST\left(t\right)\right)$ and $\sigma \left(SS{T}_{ref}\left(t\right)\right)$ are the standard deviations of SST and SST_ref at a time point t. Similar formulae are used for other variables.

The DA configurations of the experiments are clearly clustered at lower values of RMSE and higher values of CORR than their noDA counterparts to show the improvement introduced by the DA algorithm in reducing the ‘double penalty effect’. The quantitative parameters showing the errors in SST and their reduction due to DA are shown in

Table 2. Uncertainties in SST for experiments with and without DA.

Ver No	Bias_noDA	Bias_DA	RMSE_noDA	RMSE DA	CORR_noDA	CORR_DA	I_Bias	I_RMSE	I_CORR
1	0.25	0.08	0.37	0.10	0.85	0.99	3.1	3.8	1.2
2	0.18	0.06	0.32	0.08	0.85	0.99	3.1	4.2	1.2
3	0.12	0.04	0.28	0.06	0.85	0.99	3.0	4.7	1.2
4	−0.13	−0.04	0.29	0.06	0.84	0.99	3.7	4.8	1.2
5	0.50	0.16	0.59	0.18	0.82	0.98	3.1	3.2	1.2
6	−0.16	−0.03	0.32	0.12	0.83	0.97	5.3	2.6	1.2
7	−0.12	−0.03	0.31	0.12	0.82	0.97	4.4	2.5	1.2
8	−0.26	−0.05	0.39	0.17	0.79	0.95	5.0	2.4	1.2
9	0.00	0.01	0.30	0.11	0.84	0.99	0.5	2.8	1.2

.

The numbers in Table 2 are the time averages of the time series presented in Figure 5 and Figure 6. The improvement ratios shown in the last three columns of Table 2 are calculated as follows: I_Bias = Bias_noDA/Bias_DA; I_RMSE = RMSE_noDA/RMSE_DA; I_CORR = CORR_DA/CORR_noDA and similar for other variables in the following sections. The bias improvement ratio is 3 to 5.3 for all experiments except for version 9, where the bias is slightly worse after DA, although the absolute values are very small (less than 0.1 °C) in most DA configurations. The improvement of RMSE is recorded in all cases, and it is slightly better for experiments with persistent errors in the forcing. The worst residual (after DA) RMSE is 0.18 °C, which is still better than the uncertainty of satellite derived SST (Donlon et al., 2012 [27]). The correlation coefficient is systematically higher in the DA simulations.

The improvement of representing the spatial structure is illustrated in Figure 7 for two typical cases: v1 (persistent perturbation in SWR) and v8 (random wind). A qualitative assessment demonstrates that the configurations with DA (rightmost panels in Figure 7) are nearly identical to the reference case (leftmost panels).

It is has been reported in the literature that some DA schemes are not appropriate for improving the results on derived variables (see, for example, [15]). In the next subsections, we assess how efficient the mode-to-model DA is for mitigating atmospheric forcing errors in the derived variables—kinetic energy, vorticity, and enstrophy.

3.2. Kinetic Energy

The errors in the atmospheric forces translate into errors in currents. The variability of currents and the associated errors are convenient to analyse using scalar rather than vector variables. This sub-section examines the area averaged surface kinetic energy (KE) per unit mass, which is calculated as:

$$KE=\frac{1}{2}\left(u^2+v^2\right)$$

(5)

Figure 8 shows a summary of time series KE for configurations with and without DA.

The time series of KE bias is presented in Figure 9. The DA curves for experiments with perturbed SWR are clearly clustered at small bias values (Figure 9a), while results generated by perturbed wind show a more scattered pattern (Figure 9b).

The ability of the DA process to reduce the shift in the distribution of surface KE is illustrated in Figure 10. For experiments with perturbed SW, the CORR is very close to 1 in DA configurations all year round. For DA experiments with perturbed wind, the CORR is close to 0.9 most of the year but drops to 0.5–0.6 during transitional periods at the start and end of the south-west monsoon period (May and September to October).

Table 3. Uncertainties in surface kinetic energy for experiments with and without DA.

Ver No	Bias_noDA	Bias_DA	RMSE_noDA	RMSE DA	CORR_noDA	CORR_DA	I_Bias	I_RMSE	I_CORR
1	0.0029	0.0007	0.0316	0.0065	0.62	0.98	4.1	4.9	1.6
2	0.0041	0.0005	0.0353	0.0074	0.61	0.98	8.1	4.8	1.6
3	0.0041	0.0003	0.0355	0.0077	0.60	0.97	13.2	4.6	1.6
4	0.0037	−0.0005	0.0357	0.0077	0.60	0.97	−7.3	4.6	1.6
5	0.0051	0.0016	0.0372	0.0086	0.59	0.97	3.2	4.3	1.6
6	0.0090	0.0034	0.0398	0.0182	0.57	0.89	2.6	2.2	1.6
7	0.0064	0.0020	0.0393	0.0169	0.55	0.90	3.1	2.3	1.6
8	0.0107	0.0054	0.0425	0.0226	0.54	0.84	2.0	1.9	1.5
9	0.0047	0.0001	0.0363	0.0085	0.60	0.97	46.2	4.3	1.6

shows the time averages of uncertainties in surface kinetic energy and their reduction by the DA process. The improvement ratios for KE are calculated similar to SST.

The improvements in bias and RMSE for experiments with perturbed SWR are consistently better than that of perturbed wind, exhibiting an improvement ratio between 4.3 and 4.9. Even in the worst performing experiment, v8 (strongly perturbed wind, see Table 1), the RMSE was reduced by half. The improvement in CORR of more than 50% is achieved in all experiments.

3.3. Vorticity

Relative vorticity (VORT) is an important characteristic of ocean currents allowing clear identification of cyclonic (VORT > 0) and anticyclonic (VORT < 0) eddies. Figure 11 shows the evolution of RMSE and CORR for vorticity throughout the year. The calculation and analysis of vorticity bias is excluded as the area averaged vorticity is equal to the contour integral of currents along the boundary (Stokes’s theorem for vector fields). As all experiments use the same boundary conditions, the area averaged vorticities must be equal, and at a high level of accuracy, they are.

The time averages of the curves presented in Figure 11 are shown in

Table 4. Uncertainties in surface vorticity for experiments with and without DA.

Ver No	RMSE_noDA	RMSE DA	CORR_noDA	CORR_DA	I_RMSE	I_CORR
1	7.69 × 10−6	2.5 × 10−6	0.35	0.89	3.0	2.4
2	7.82 × 10−6	2.5 × 10−6	0.34	0.90	2.9	2.6
3	7.84 × 10−6	2.5 × 10−6	0.34	0.89	2.9	2.6
4	8.03 × 10−6	2.5 × 10−6	0.33	0.89	2.9	2.6
5	7.80 × 10−6	2.6 × 10−6	0.34	0.89	2.7	2.5
6	8.06 × 10−6	3.7 × 10−6	0.32	0.79	2.1	2.4
7	8.19 × 10−6	3.6 × 10−6	0.31	0.81	2.2	2.6
8	8.14 × 10−6	4.2 × 10−6	0.32	0.75	1.9	2.3
9	7.91 × 10−6	2.6 × 10−6	0.34	0.89	2.8	2.5

.

The units for RMSE are the same as for vorticity, i.e., 1/s. The coefficient of improvement in CORR for vorticity is greater than for KE, but the CORR value itself is slightly lower for both no DA and DA configurations. The CORR is well improved by the DA process at the onset of the monsoon, but it achieves its lowest values in the middle of the monsoon period for all experiments. The experiments with perturbed SWR show systematically higher values of CORR.

Spatial distributions of VORT for two experiments, one with perturbed SWR and the other with perturbed wind, are shown in Figure 12.

3.4. Enstrophy

The enstrophy (square of relative vorticity) is another useful scalar characteristic of ocean currents and is often used to study the transfer of kinetic energy between larger and smaller scale motions (see, for example, [28]). It is sensitive even to small disturbances of the flow field and, therefore, is a good means to test the model skill. Figure 13 shows the time series of enstrophy bias, RMSE, and CORR.

The BIAS and RMSE for no DA experiments are large in May and September, similar to the seasonal variability of these parameters for kinetic energy. Data assimilation removes the bias very efficiently all year round, but RMSE is still large in the winter months: February to March and November to December. The CORR for enstrophy is improved by DA all year round, but it is still lower than for SST. There is a tendency for CORR to increase towards the end of the year. The time averages of the curves presented in Figure 13 are shown in

Table 5. Uncertainties in surface enstrophy for experiments with and without DA.

Ver No	Bias_noDA	Bias_DA	RMSE_no DA	RMSE DA	CORR_noDA	CORR_DA	I_Bias	I_RMSE	I_CORR
1	1.9 × 10−11	−5.0 × 10−15	1.27 × 10−10	4.3 × 10−11	0.37	0.85	−101.5	2.7	2.2
2	1.9 × 10−11	−4.0 × 10−14	1.29 × 10−10	4.2 × 10−11	0.35	0.85	−28.1	2.5	2.4
3	2.0 × 10−11	−6.1 × 10−14	1.32 × 10−10	4.3 × 10−11	0.34	0.85	−14.6	2.5	2.5
4	2.3 × 10−11	−1.5 × 10−13	1.40 × 10−10	4.3 × 10−11	0.33	0.85	−16.1	2.6	2.6
5	1.8 × 10−11	5.3 × 10−15	1.27 × 10−10	4.5 × 10−11	0.35	0.85	−13.8	2.3	2.4
6	2.3 × 10−11	9.7 × 10−13	1.38 × 10−10	6.6 × 10−11	0.32	0.73	−52.2	1.9	2.3
7	2.4 × 10−11	5.9 × 10−13	1.41 × 10−10	6.4 × 10−11	0.30	0.74	−20.3	2.0	2.5
8	2.5 × 10−11	1.5 × 10−12	1.46 × 10−10	7.6 × 10−11	0.31	0.66	−563.9	1.8	2.1
9	2.2 × 10−11	−1.4 × 10−13	1.33 × 10−10	4.5 × 10−11	0.34	0.85	−15.3	2.4	2.5

.

The DA reduces bias by a large factor, ranging from 15 to more than 500. Improvement of RMSE across all experiments is more uniform. DA performs slightly better in experiments with perturbed SWR than with perturbed wind.

As with other variables presented above, the DA configurations (rightmost panels in Figure 14) produce maps that are difficult to differentiate qualitatively from the reference case (leftmost panels).

4. Discussion

Data assimilation intends to correct errors in the ocean models coming from a range of sources. Different DA method have different efficiency in doing this [11,27]. In order to study the ability of model-to-model DA to rectify errors coming from only one source, namely, atmospheric forcing, the models in all 18 experiments with perturbed forcing (both with and without DA) are identical to the reference model LD20_DA, except for the atmospheric forcing used to drive the models. Therefore, a perfect Data Assimilation would completely eliminate errors in the atmospheric forcing and would give identical results for the reference model (LD20_DA) and for all models with perturbed forcing. In other words, BIAS, RMSE between the perturbed and reference models would be zero, and CORR would be equal to one.

Obviously, none of DA methods are perfect. The analysis in this section is intended to contrast and compare the efficiency of model-to-model DA by comparing the results from the models with perturbed and original (not perturbed) forcing. For further discussion, the experiments are divided into two groups: experiments with forcing versions v1–v5 and v9 have perturbation only in the downward short wave radiation (group SOLAR), and the experiments v5–v8 have perturbation only in the wind components (group WIND). The configurations with model-to-model DA show improvements for all experiments and all estimators of model skill.

The actual residual errors for the SOLAR and WIND groups are evaluated as follows. Let us introduce ‘typical values’ of the parameters used in the previous section. A typical value for a variable in question (e.g., SST, KE, VORT, and Enstr) is equal to the time and area average of the output generated by the reference model. A typical value for a model uncertainty is defined as the group average of model skill parameters in the relevant columns presented in Table 2, Table 3, Table 4 and Table 5, and separately for SOLAR and WIND groups for configurations with DA. The typical values and the units of measurements are shown in

Table 6. Typical values and their uncertainties for perturbed models with DA.

Variable	Typical Value	Typical Bias SOLAR	Typical Bias WIND	Typical RMSE SOLAR	Typical RMSE WIND	Typical CORR SOLAR	Typical CORR WIND
SST (°C)	29.2	0.051	−0.037	0.098	0.14	0.99	0.97
KE (m2/s2)	3.08 × 10−2	4.52 × 10−4	3.64 × 10−3	6.28 × 10−3	1.80 × 10−2	0.98	0.88
VORT (1/s)	N/A	N/A	N/A	2.51 × 10−6	3.82 × 10−6	0.89	0.78
Enstrophy (1/s2)	4.31 × 10−11	−6.63 × 10−14	1.03 × 10−12	4.36 × 10−11	6.91 × 10−11	0.85	0.71

. The correlation coefficient is a non-dimensional number as usual. The uncertainties are calculated against the reference model LD20_DA.

For SST, typical biases for both SOLAR and WIND groups as well as RMSE for the SOLAR group are all below 0.1 °C. The RMSE for the WIND group is slightly higher at 0.14 °C. In all cases, these values of uncertainty are much lower (i.e., better) than those coming from satellite measurements. For example, the accuracy of Operational Sea Surface Temperature and Sea Ice Analysis is around 0.5 °C [29]. The correlation coefficients are high at CORR = 0.97–0.99 and close to the correlation coefficient CORR = 0.97 between SWR from ERA-Interim and surface observations [2]. High correlation indicates a good representation of smaller-scale structures in the temperature field.

The uncertainty in the representation of currents is convenient to discuss in terms of current speed rather than kinetic energy, using the link $KE=\frac{1}{2}{U}^2$, where U is the absolute value of current speed. From Table 6, it follows that the typical value for U is U_typ = 0.25 m/s, and typical biases for SOLAR and WIND groups are 0.02 m/s and 0.06 m/s, respectively. Typical RMSEs are calculated using a formula for uncertainty propagation [30]:

$$RMSE\left(U\right)=\frac{RMSE\left(KE\right)}{U_{typ}}$$

(6)

For SOLAR and WIND groups, the RMSE(U) are 0.025 m/s and 0.073 m/s, respectively. Relative to the typical speed U_typ, these numbers translate into 10% and 29% of uncertainty. The uncertainty for the SOLAR group is well within the average accuracy of satellite measurements of currents at 27% [31], and the WIND group is similar. The correlation coefficients are also better for the SOLAR than the WIND group.

Vorticity and enstrophy are based on the spatial derivatives of current velocities and they are therefore very sensitive to the spatial shift of the circulation patterns. The correlation coefficients are higher for the SOLAR group (0.85–0.89) than for the WIND group (0.71–0.78). Both groups show lower correlations in vorticity and enstrophy fields than in temperature and kinetic energy. Nevertheless, the correlations are sufficiently high to confirm the consistency of current patterns between the reference model and all the experiments with perturbed atmospheric forcing in DA configurations. A visual qualitative analysis of vorticity and enstrophy maps supports this conclusion.

As shown in the analysis above, the efficiency of DA for experiments with perturbations in wind forcing is not as high as for experiments with perturbations in solar radiation. A potential reason for this is the higher temporal variability of wind than that of solar radiation. Below, we test a hypothesis that the efficiency of model-to-model DA depends on the frequency of the DA cycle. In order to test this hypothesis, an additional set of experiments with the WIND group of atmospheric forcing perturbations was carried out with the length of DA cycle of 1, 2, 3 and 8 days. Note that the length of the DA cycle in all previous sections was 5 days. In total, this section analyses the residual uncertainties using the following statistics: BIAS, RMSE, and CORR for 15 DA experiments. All statistics are calculated daily using all wet grid nodes (except the flow relaxation rim) and then averaged from 1 February to 31 December 2016. The results for the kinetic energy of surface currents are shown in

Table 7. Uncertainties in Kinetic Energy against the reference model LD20_DA.

Statistic	Atmospheric Forcing Perturbation Version No	Length of the DA Cycle (Days)					No DA
		1	2	3	5	8
BIAS, m2/s2	6 (U wind only)	0.0042	0.0035	0.0033	0.0034	0.0039	0.0090
	7 (V wind only)	0.0039	0.0022	0.0021	0.0020	0.0020	0.0064
	8 (both U and V wind)	0.0061	0.0055	0.0051	0.0054	0.0058	0.0107
RMSE(KE), m2/s2/ RMSE(U) m/s	6 (U wind only)	0.0214/0.086	0.0188/0.075	0.0177/0.071	0.0169/0.068	0.0194/0.078	0.0398/0.159
	7 (V wind only)	0.0220/0.088	0.0177/0.071	0.0167/0.067	0.0156/0.062	0.0180/0.072	0.0393/0.157
	8 (both U and V wind)	0.0248/0.099	0.0225/0.090	0.0216/0.086	0.0215/0.086	0.0234/0.094	0.0425/0.198
CORR	6 (U wind only)	0.81	0.86	0.87	0.89	0.86	0.57
	7 (V wind only)	0.79	0.87	0.88	0.90	0.87	0.55
	8 (both U and V wind)	0.76	0.81	0.83	0.84	0.81	0.54

. In addition, the RMSE for velocity magnitude is calculated using Equation (6).

For comparison, the last column in Table 7 shows the same statistics of uncertainty but for the free runs (without DA). The BIAS is very small for all assimilation lengths. The RMSE and CORR are marginally sensitive to the length of the assimilation cycle. The 5-day cycle shows slightly better performance than both shorter and longer cycles.

As an example of time evolution of the Pierson correlation coefficient (CORR), the time series for five experiments within the WIND group (v6–v8) are shown in Figure 15.

While time averaged CORR is within a comfortable range of 0.76 to 0.84 for all DA lengths, it is systematically lower in May and September and October. This drop in CORR may be of little practical significance as the magnitudes of deviation from the reference model during these periods are close to their minima, as shown in Figure 16a–c.

The time averaged correlation coefficient is the lowest (worst) at the length of the DA cycle of 1 day and the largest (best) at 5 days. Therefore, the hypothesis that a reduction in the DA cycle can further improve the skill of model-to-model DA does not hold. The reason for such behaviour is not clear.

The efficiency of DA in mitigating errors in atmospheric forcing is best to assess by comparison with the same model but with ‘error-free’ forcing. The error-free forcing is defined here as the best available forcing. However, for practical applications, it is useful to assess the overall skill of DA in reducing uncertainties due to all possible reasons. Below, the model results are compared with two reputable datasets: (i) OSTIA [27], which is largely based on satellite measurements but involves heavy interpolation, particularly at the time of the SW monsoon; and (ii) global ocean reanalysis [20], which, in turn, was validated against in situ observations collected in the World Ocean Database [32].

As shown in Figure 17, the model-to-model DA improves the overall quality of all numerical experiments both against CMEMS reanalysis and OSTIA observations, despite some misfits between these external datasets. The correlation coefficient CORR for experiments without DA is in the range 0.60–0.63 with reference to OSTIA and 0.69–0.73 with reference to CMEMS. In the experiments with DA, its value is higher in the range 0.69–0.74 for OSTIA and 0.90–0.92 for CMEMS.

The temperature and salinity profiles from the model experiments were compared with 399 ARGO float observations available from [33]. Calculations were performed as follows. The profiles from models were bi-linearly interpolated in the horizontal to the location of the ARGO profiles on the same day. The difference in time within the same day was ignored. The ARGO profile data have a higher resolution than the LD20 model and therefore were binned in the vertical in accordance to the extent of the model grid cells centred at the corresponding T-levels of NEMO (see [18]). ARGO data within each bin were averaged, and the differences were computed between observations and models and used to calculate biases and RMSE at each model depth level. The results were then averaged in the vertical to show typical remaining uncertainties after DA. The RMSE and bias for temperature were in the range 0.21–0.24 °C and 0.08–0.12 °C, respectively, for all experiments. The RMSE and bias for salinity were 0.49 and from (−0.13) to (−0.14), respectively.

5. Conclusions

A set of 19 ocean modelling experiments is carried out to examine the effectiveness of recently developed Nesting with the Data Assimilation (NDA) algorithm in the reduction of errors in ocean model output that are generated by errors in atmospheric forcing. The NDA algorithm requires that the parent coarse model, from which the data for model-do-model DA are taken, must use observation-based DA in the first place. The NDA algorithm is computationally very efficient as the processing of large matrices required to calculate the weights at the stochastic downscaling step of the method is carried out only once for a given model configuration. The NDA method does not need an additional step of dynamic balancing of temperature, salinity, and velocity provided by the measurements, as these variables are already balanced in the parent model. Another benefit of the NDA method is that it reduces the so-called ‘double penalty effect’ [17].

When applied to the models driven by disturbed (i.e., containing errors in solar radiation and wind speed) atmospheric forcing, the NDA algorithm significantly reduces biases and RMSE while improving correlation coefficients between the reference and the disturbed models for all of the examined variables: sea surface temperature, kinetic energy, vorticity, and enstrophy.

The residual uncertainties are similar or lower than the uncertainties of satellite observations. The data assimilation works best to compensate for errors in shortwave solar radiation, and it is slightly less efficient at mitigating errors in wind forcing. An additional set of 15 experiments was run from 1–8 day length of the Data Assimilation cycle. The model results are shown to be not sensitive to the length of the DA cycle in this range.