A New Scheme for Capturing Global Conditional Nonlinear Optimal Perturbation

Abstract

Conditional nonlinear optimal perturbation (CNOP) represents the initial perturbation that satisfies a certain physical constraint condition, and leads to a maximum prediction error at the moment of prediction. The CNOP method is a useful tool in studying atmosphere and ocean predictability problems. Generally, the optimization algorithm based on the gradient of the cost function to compute CNOP requires an initial guess. The traditional scheme randomly chooses the initial guess of CNOP within the constraint range and therefore this scheme is called RIG-CNOP. However, the RIG-CNOP scheme reduces the probability of capturing the global CNOP in many cases, such as the prediction model is strongly nonlinear or long-term prediction is performed, or multiple extreme values existed in the cost function. Considering the limitations of the RIG-CNOP scheme, we propose a new initial guess selection scheme. In this scheme, we first pre-analyze a series of random initial guesses, and then, an optimal initial guess is selected. The above process replaces the initial guess selection scheme in the traditional scheme, which is called PAIG-CNOP. Numerical experiments are conducted utilizing the Lorenz-63 model. Also, to compare the performance of the PAIG-CNOP method with the RIG-CNOP method in capturing global CNOP, the CNOP and the maximum cost function value (MCFV) obtained by the filtering method (FM) are used as benchmarks (this value is called FMMCFV in brief). The experimental results show that even the prediction model is strongly nonlinear or the prediction time is long, or the cost function has multiple extreme values, the PAIG-CNOP method can capture the global CNOP with a high probability. The results show that the PAIG-CNOP method has a higher probability of capturing the global CNOP than the RIG-CNOP method. In addition, we use an ensemble-based technique in the computation of gradients, thus avoiding the use of adjoint techniques in the maximization process. Due to the attractive features of the new method, the PAIG-CNOP method is an efficient and useful method for solving CNOP, it can be more easily applied to obtain the global CNOP of operational prediction models.

1. Introduction

Determining the fastest growing of initial perturbations is one of the key issues in studying atmosphere and ocean predictability problems. Lorenz [1] first proposed the linear singular vectors (LSVs) method to achieve this purpose. Since the LSV method was proposed, it has been widely used in atmosphere and ocean predictability studies [2]. For example, it has been applied to research on the predictability of El Niño Southern Oscillation (ENSO) events [3,4,5]. These studies applied singular vector methods to discuss the relationship between ENSO and seasonal variations in predictability, and to investigate the physical mechanisms controlling it. In some studies, singular vectors are also applied to generate initial perturbations for ensemble forecasts [6] and identify target areas in adaptive observations [7,8]. The LSV method is established based on linear theory. It assumes that the evolution of the initial perturbation is linear, which can be controlled approximately by a tangent linear model. This linearity assumption is a reasonable approximation in the nonlinear prediction model, when the prediction time is short and the amplitude of initial perturbation is sufficiently small. However, given real atmospheric and oceanic prediction models are chaotic and nonlinear, a small initial perturbation can be evolved quickly into a large amplitude. Obviously, the LSV theory cannot reveal the evolution of the initial perturbation in the nonlinear prediction model. Considering the limitations of LSV methods applied to nonlinear prediction models, Mu and Duan [9] introduced the concepts of conditional nonlinear optimal perturbation (CNOP). The CNOP describes the initial perturbation that satisfies a certain constraint at the moment of prediction and causes to a maximum prediction error. It also reflects the optimal precursor in weather and climate events. The CNOP method is an extension of the LSV method to nonlinear prediction models. It has been used to explore optimal precursor and spring predictability barrier of ENSO events [10,11,12,13,14], and identify sensitive areas for target observation [15,16,17,18,19,20,21], and generate initial perturbations for ensemble forecasts [22,23,24]. These studies indicate that the CNOP method is a useful tool for atmosphere and ocean predictability and stability or sensitivity studies [25,26].

Although the CNOP method has been widely used in atmosphere and oceanic predictability studies and has achieved certain results, there are some difficulties and challenges in its operational applications. One of the key issues is to calculate the optimal initial perturbation. The existing CNOP solution methods can be divided into two categories: One is the traditional optimization algorithms that require the gradient of the cost function (CF), such as the spectral projected gradient 2 (SPG2) [27], sequential quadratic programming (SQP) [28], and limited memory Broyden–Fletcher–Goldfarb–Shanna (limited memory BFGS) [29]. The other is intelligent optimization algorithms that do not require gradient information, such as genetic algorithms (GA) [30,31], particle swarm optimization algorithms (PSO) [32]. Intelligent optimization algorithms have many advantages in solving CNOP problems, which can overcome the effects of strong nonlinearity of the prediction model and are easy to implement. However, real atmospheric and ocean prediction models are usually high-dimensional, and the cost of using intelligent optimization algorithms to calculate CNOP is unacceptable. For traditional optimization algorithms, CNOP is usually computed by using the gradient descent algorithm based on the adjoint model, which is called ADJ-CNOP [9,10,33,34,35,36]. However, writing code for adjoint models is a very time-consuming job, and many operational prediction models do not have adjoint models. To overcome the above limitations, Wang et al. [37] proposed an ensemble-based method to approximate the adjoint models, which is called EN-CNOP. The EN-CNOP method can avoid the writing of adjoint model codes and equal to the ADJ-CNOP method. In this paper, we use the EN-CNOP method to compute the gradient information of the cost function, while drawing on the experience obtained in formulating the analytical four-dimensional ensemble-variational data assimilation (A-4DEnVar) [38] study to avoid the use of adjoint models. When the gradient information is obtained, the CNOP can be calculated using the SPG2 algorithm.

Generally, as for the optimization algorithm based on the gradient of the cost function to compute CNOP, an initial guess is required. Zheng et al. [32] used the Ikeda model [39,40] in the case of using the RIG-CNOP scheme to select the initial guesses of CNOP. Due to the existence of multiple local minima of the cost-function, the ADJ-CNOP method can only capture the global CNOP with a small probability. To overcome this issue, Pires et al. [41] proposed Quasi-Static Variational Assimilation (QSVA) techniques, where the prediction period is progressively increased with first guesses sequentially provided by previous optimal perturbations. However, as the forecast time increases, the computational cost of this method gradually increases. Tian et al. [42] proposed a two-step optimization strategy based on the EN-CNOP method, and the test results showed that a better first guess could improve the efficiency and accuracy of computing CNOP. However, the two-step optimization strategy actually starts with the RIG-CNOP scheme. Previous studies have shown that it is necessary to find an efficient method to solve the global CNOP problem with low computational cost and can be easily transplanted to other systems. Therefore, we use the EN-CNOP method to compute an adjoint model and propose a new initial guess selection scheme based on pre-analysis of initial guesses to obtain global CNOP, which is called PAIG-CNOP. In this scheme, we first randomly selected a series of initial guesses according to certain rules within the perturbation constraints range, then the initial cost function values (CFVs) of these initial guesses are analyzed to filter out an optimal initial guess. The above process replaces the RIG-CNOP scheme in the EN-CNOP method.

The rest of the paper is organized as follows. In Section 2, The definition of CNOP and the EN-CNOP approach are briefly described, and then the methodology of the new approach is presented. Experimental designing and results using a simple three-variable non-linear Lorenz-63 model [43] are shown in Section 3. A discussion is presented in Section 4. Finally, the conclusion is given in Section 5.

2. Methodology

2.1. Definition of CNOP

CNOP describes the initial perturbation that satisfies some constraint and causes the maximum prediction error at the prediction moment. It is assumed that the nonlinear prediction model for the atmosphere and ocean can be described by the following equation:

$$\left\{\begin{array}{l}\frac{\partial X}{\partial t}+F(X, t)=0\\ X{|}_{t=0}=X_0\end{array}\right.$$

(1)

where $X$ vector is the state variable, including temperature, salinity and current velocity and so on; $t$ is the time variable and $F$ is the nonlinear partial differential operator; $X_0$ is the background state variable at the initial moment. ${\mathrm{M}}_{0\to T}$ is the evolution operator of the state variable from moment 0 to moment $T$. The state variable of Equation (1) at moment $T$ is:

$$X{|}_{t=T}={\mathrm{M}}_{0\to T}(X_0)$$

(2)

Let $x_0$ be the first guess value of CNOP, $x_T$ is the nonlinear evolution of $x_0$ at the moment $T$, given by:

$$x_T={\mathrm{M}}_{0\to T}(X_0+x_0)-{\mathrm{M}}_{0\to T}(X_0)$$

(3)

Choose an appropriate norm $‖·‖$, and note that the conditional nonlinear optimal perturbation cost function is:

$$I(x_0)={‖{\mathrm{M}}_{0\to T}(X_0+x_0)-{\mathrm{M}}_{0\to T}(X_0)‖}^2$$

(4)

The prediction error at prediction moment $T$ is maximized when the cost function finds $x_{0\delta }{}^{*}$ in the constraint range $‖x_{0\delta }^{}‖\le \delta$. $x_{0\delta }{}^{*}$ is the solution to the following optimization problem:

$$I(x_{0\delta }{}^{*})=\underset{‖x_{0\delta }^{}‖\le \delta }{\max }{‖{\mathrm{M}}_{0\to T}(X_0+x_0)-{\mathrm{M}}_{0\to T}(X_0)‖}^2$$

(5)

Then the initial perturbation $x_{0\delta }{}^{*}$ is called CNOP and more detailed description of the formula can be found in Mu et al. [9].

2.2. EN-CNOP

The computational problem of CNOP is essentially solving a constrained maximization problem. Since all existing optimization algorithms are designed for solving the minimization problem, the CNOP cost function is replaced with the inverse of this cost function, then Equation (4) is rewritten as:

$$J(x_0)=1/{‖{\mathrm{M}}_{0\to T}(X_0+x_0)-{\mathrm{M}}_{0\to T}(X_0)‖}^2$$

(6)

The gradient of the above cost function of CNOP initial guess value can be calculated by the following equation:

$$\nabla J(x_0)=\frac{-2{\mathbf{M}}_{0\to T}^{\mathbf{T}}(X_0+x_0)\left[{\mathrm{M}}_{0\to T}(X_0+x_0)-{\mathrm{M}}_{0\to T}(X_0)\right]}{{‖{\mathrm{M}}_{0\to T}(X_0+x_0)-{\mathrm{M}}_{0\to T}(X_0)‖}^4}$$

(7)

where ${\mathbf{M}}_{0\to T}^{\mathbf{T}}(X_0+x_0)$ is the transpose of the Jacobian matrix of ${\mathrm{M}}_{0\to T}$ and can be calculated from the corresponding adjoint model [44]. In order to avoid the preparation of the adjoint model, an ensemble-based approach is used to estimate this term. An ensemble-based approach to obtaining gradient information is as follows. First, a series of uncorrelated initial perturbations ${\tilde{\mathbf{x}}}_i$ are generated, and then an ensemble technique is used to build a statistical model of their corresponding forecast increments ${\tilde{\mathbf{y}}}_i$ at the forecast moment. Finally, drawing on the experience obtained in formulating the A-4DEnVar, we use the background error covariance matrix information to approximate the adjoint models.

First, we select a series of uncorrelated initial perturbations to form the initial perturbation matrix ${\tilde{\mathbf{x}}}_i=({\tilde{x}}_1,{\tilde{x}}_2,\cdots {\tilde{x}}_n)$. The incremental matrix ${\tilde{\mathbf{y}}}_i=({\tilde{y}}_1,{\tilde{y}}_2,\cdots {\tilde{y}}_n)$ at the prediction moment $T$ corresponding to the initial perturbation matrix ${\tilde{\mathbf{x}}}_i$ is obtained from the following equation:

$${\tilde{\mathbf{y}}}_i={\mathrm{M}}_{0\to T}(X_0+x_0+{\tilde{\mathbf{x}}}_i)-{\mathrm{M}}_{0\to T}(X_0+x_0),(i=1,2,\cdots ,n)$$

(8)

If each member of the initial perturbation matrix ${\tilde{\mathbf{x}}}_i$ is small enough compared to $X_0+x_0$, and the integration time is not very long, one can then establish a first-order linear approximate statistical relationship between ${\tilde{\mathbf{x}}}_i$ and ${\tilde{\mathbf{y}}}_i$.

$${\tilde{\mathbf{y}}}_i={\mathbf{M}}_{0\to T}(X_0+x_0){\tilde{\mathbf{x}}}_i,(i=1,2,\cdots ,n)$$

(9)

Then, we use relevant concepts in A-4DEnVar data assimilation scheme [38], the background state variables error covariance matrix at the initial moment is ${\mathbf{B}}_{00}=E\left\{{\tilde{\mathbf{x}}}_i{\tilde{\mathbf{x}}}_i^{\mathbf{T}}\right\}$ where $E\left\{•\right\}$ is the mathematical expectation. Based on the Equation (9), the error covariance matrix of ${\mathbf{B}}_{T0}$ and ${\mathbf{B}}_{0T}$ can be constructed as follows:

$$\left\{\begin{array}{l}{\mathbf{B}}_{T0}=E\left\{\begin{array}{cc}{\tilde{\mathbf{y}}}_i& {\tilde{\mathbf{x}}}_i^{\mathbf{T}}\end{array}\right\}\approx {\mathbf{M}}_{0\to T}(X_0+x_0){\mathbf{B}}_{00}\\ {\mathbf{B}}_{0T}=E\left\{\begin{array}{cc}{\tilde{\mathbf{x}}}_i& {\tilde{\mathbf{y}}}_i^{\mathbf{T}}\end{array}\right\}\approx {\mathbf{B}}_{00}{\mathbf{M}}_{0\to T}^{\mathbf{T}}(X_0+x_0)\end{array}\right.$$

(10)

These two background error covariance matrices actually contain the information of the tangent linear evolution operator matrix ${\mathbf{M}}_{0\to T}(X_0+x_0)$ and the corresponding adjoint operator matrix ${\mathbf{M}}_{0\to T}^{\mathbf{T}}(X_0+x_0)$, respectively.

Finally, the gradient of the CNOP cost function of Equation (7) becomes:

$$\nabla J(x_0)=\frac{-2{\mathbf{B}}_{00}^{-1}{\mathbf{B}}_{0T}\left[{\mathrm{M}}_{0\to T}(X_0+x_0)-{\mathrm{M}}_{0\to T}(X_0)\right]}{{‖{\mathrm{M}}_{0\to T}(X_0+x_0)-{\mathrm{M}}_{0\to T}(X_0)‖}^4}$$

(11)

From Equation (11), it can be seen that the adjoint model is no longer needed. Then, the gradient information obtained from Equation (11) can be used to calculate the minimum value of the cost function Equation (6) using the optimization algorithm SPG2. As a result, the maximum value of the cost function Equation (4) and the corresponding CNOP can be obtained. The scheme for calculating the inverse of the large dimensional matrix can be seen in Appendix B.

2.3. PAIG-CNOP

The traditional scheme of selecting the initial guess is to randomly select one within the constraint range $\delta$, this is called the RIG-CNOP. Zheng et al. [32] proved the limitation of this scheme, and found that the PSO-CNOP method can effectively capture the global CNOP. However, the cost of using PSO-CNOP method to calculate CNOP is considerable. Tian et al. [42] proposed a two-step optimization strategy, and found that a better first guess could improve the efficiency and accuracy of computing CNOP. However, the two-step optimization strategy actually starts with the RIG-CNOP scheme. Considering the limitations of existing methods, it is necessary to find an efficient method to solve the global CNOP problem with low computational cost and can be easily transplanted to other systems. We are inspired by the study of Tian et al. and the population concept in the PSO-CNOP method, we propose a new effective initial guess scheme to capture global CNOP with low computational cost and without using the adjoint model, it is called PAIG-CNOP.

In the new initial guess scheme, in order to ensure that the generated initial guesses have a certain degree of dispersion, the new method needs to produce several random initial guesses according to certain rules within the constraint range $\delta$, and select an optimal initial guess for the EN-CNOP method to compute the CNOP. Figure 1 shows the flow chart of PAIG-CNOP method. The specific implementation scheme is divided into the following four steps:

• Randomly generate P groups of N initial guesses within the constraint range $\delta$, $x_{0ij}^{}\in \delta (i=1,2,\dots ,P;j=1,2,\dots ,N)$. Each initial guess $x_{0ij}^{}$ is generated randomly in the range $\left(-K_{ij},K_{ij}\right)$. where $K_{ij}$ is expressed as:

  $$K_{ij}={\delta }_{\max }-({\delta }_{\max }-{\delta }_{\min })\times \frac{j}{N}(i=1,2,\dots ,P;j=1,2,\dots ,N)$$

  (12)

  and ${\delta }_{\max }={10}^{-i+1}\times \delta ,{\delta }_{\min }={10}^{-i}\times \delta (i=1,2,\dots ,P)$.
• Use Equation (11) to calculate the gradient of each initial guess.

  $$\nabla J(x_{0ij})=\frac{-2{\mathbf{B}}_{00}^{-1}{\mathbf{B}}_{0T}\left[{\mathrm{M}}_{0\to T}(X_0+x_{0ij})-{\mathrm{M}}_{0\to T}(X_0)\right]}{{‖{\mathrm{M}}_{0\to T}(X_0+x_{0ij})-{\mathrm{M}}_{0\to T}(X_0)‖}^4}$$

  (13)
• After obtaining the gradient information of all the initial guesses $x_{0ij}^{}$, we use the SPG2 algorithm to perform only one iteration of optimal search for $x_{0ij}^{}$. Then we can obtain the updated initial guess: $x_{0ij}^{\prime }\in \delta (i=1,2,\dots ,P;j=1,2,\dots ,N)$ and the cost function value (CFV) corresponding to each initial guess value:

  $$I(x_{0ij}^{\prime })={‖{\mathrm{M}}_{0\to T}(X_0+x_{0ij})-{\mathrm{M}}_{0\to T}(X_0)‖}^2$$

  (14)
• Sieve out a point $x_0^{*}$ so that the following equation holds:

  $$I(x_0^{*})=\underset{x_0^{*}\in x_{0ij}^{\prime }}{\max }I(x_{0ij}^{\prime })$$

  (15)
  Then, $x_0^{*}$ is chosen as the optimal initial guess for further optimization calculation.

3. Experiments Based on the Lorenz-63

3.1. The Model

The Lorenz-63 model describes the instability of atmospheric thermal convection with strong nonlinearity between the variables, and this model consists of the following set of three ordinary differential equations.

$$\begin{array}{l}\frac{dx}{dt}=\sigma (y-x)\\ \frac{dy}{dt}=rx-y-xz\\ \frac{dz}{dt}=xy-bz\end{array}$$

(16)

where $\sigma$, r and b are the parameters of the model. Let the left-hand side of Equation (6) be equal to 0, and three fixed points of the equation can be found, whose values do not vary with the time evolution of the model.

$$\left\{\begin{array}{l}O:(x,y,z)=(0,0,0)\\ C_1:(x,y,z)=(-\sqrt{b(r-1)},-\sqrt{b(r-1)},r-1)\\ C_2:(x,y,z)=(\sqrt{b(r-1)},\sqrt{b(r-1)},r-1)\end{array}\right.$$

(17)

Due to the symmetry of the solutions of this set of ordinary differential equations, the points $C_1$ and $C_2$ are equivalent, so only the points $O$ and $C_1$ are taken as the initial background states of the model for numerical experiments [45]. An additional non-equilibrium point $B\left(-3.12346395;-3.12529803;20.69823159\right)$ [46] is taken to ensure the reasonableness of the test. Therefore, a total of 3 points are used as initial background states.

3.2. Experiment Designing

Numerical experiments are conducted using the Lorenz-63 model. We use the CNOP obtained by the filtering method and the filtering method’s maximum cost function value (FMMCFV) as benchmarks, the filtering method [45,47] scheme is as follows. First, the circumscribed cube with ball constraint range $\delta$ is divided into small cubes with side length 0.01 $\delta$. Each small cube vertex is selected as a grid point, and for each of the grid points outside the ball, we connected this point to the center point of the constrained ball, and obtained the intersection point of this line with the sphere, then we made all the grid points inside the ball and these intersection points superimposed on the background state as new mesh points. Finally, we integrated the model from each new mesh points to obtain the cost function value (CFV) of each grid points at the prediction moment. The maximum value of all these is what we need (this value is called FMMCFV in brief), and the corresponding grid point is the global CNOP.

We make comparisons of CNOPs and MCFVs calculated by the RIG-CNOP method and PAIG-CNOP method at different background state points. The gradient calculations are all performed using the EN-CNOP method, then we use the SPG2 as the optimization algorithm. Fifty tests of computing CNOP are performed using the RIG-CNOP method and PAIG-CNOP method, respectively. In each trial, we can use the above two methods to capture the CNOP and calculate the corresponding MCFV. For the MCFV calculated by either method, only if this MCFV is greater than the FMMCFV, we consider that the method captures the global CNOP in this trial. In addition, the values of the three parameters of the Lorenz-63 model are set as follows: $\sigma =10, r=28, b=8/3$ [43]; the fourth-order Runge–Kutta method with a time step of 0.01 time units is used for the calculation of this model. The set constraint range ${‖\delta ‖}_2\le 0.08$ [45]. The PAIG-CNOP method uses the settings $P=4$ and $N=15$. Set the number of ensemble members $n=3$ [37,48] of the initial perturbation matrix in the EN-CNOP method. Notice that in Equation (13) the sampling errors of ${\mathbf{B}}_{T0}$, ${\mathbf{B}}_{0T}$ and ${\mathbf{B}}_{00}$ have no determinant influence since only n = 3 non-colinear perturbation directions are needed to estimate the cost function gradient.

3.3. Result Analysis

In this section, we experimented with three different background points. We take the filtering method’s maximum cost function value (FMMCFV) and CNOP as benchmarks. For each background point the experiments are further divided into two parts.

In the first part, we examine the performance of the RIG-CNOP scheme and the PAIG-CNOP scheme in generating initial guesses. We generate 50 initial guesses using each of these two methods, then we can calculate the cost function value (CFV) corresponding to these initial guesses. We take the maximum (minimum) value and the average value corresponding to all of each of these two methods’ initial guesses CFV for comparison.

In the second part, we use all of these initial guesses as the starting points for the EN-CNOP method to compute CNOP. Then we can obtain the 50 CNOPs for each of the two methods and the maximum cost function values (MCFVs) corresponding to the CNOPs. Using the filtering method’s FMMCFV and CNOP as benchmarks, we can check the accuracy of the RIG-CNOP scheme and the PAIG-CNOP scheme in capturing the CNOPs.

3.3.1. Experimental Results for the Initial Background State of Point O

In the Lorenz-63 model, the initial background state is chosen as point $O(0,0,0)$, and the prediction time $T=150$.

Figure 2 shows the distribution of the initial CFVs of the initial guesses from the PAIG-CNOP method and the RIG-CNOP method in 50 trials, and the FMMCFV calculated by the filtering method is used as the baseline.

Table 1. Statistical analysis of the CFVs corresponding to the initial guesses generated by the different methods at point $O$, $T=150$.

Method	Maximum CFV	Mean CFV
Filtering	52.8287
RIG-CNOP	35.2534	30.5767
PAIG-CNOP	52.8298	51.8793

shows the statistics for the initial guesses selected by the two methods. It can be seen from Figure 2 that the initial CFVs corresponding to the PAIG-CNOP method in 50 trials is overwhelmingly distributed in the vicinity of the FMMCFV baseline, and most of the results of the RIG-CNOP method are much smaller than the FMMCFV. Meanwhile, it can be seen from Table 1 that the mean initial CFVs of 51.879298 for the PAIG-CNOP method are very similar to the FMMCFV 52.828685, while the RIG-CNOP method is only 30.57667. These comparisons demonstrate that the initial guesses obtained by the PAIG-CNOP method are better than the RIG-CNOP method. Surprisingly, the maximum value of the CFV 52.8298 corresponding to the PAIG-CNOP method’s initial guesses is even larger than the FMMCFV 52.8287. Presumably, the initial guesses obtained by PAIG-CNOP method will be closer to the global CNOP and thus it can capture the global CNOP more easily.

Then, we further iterate the initial guesses obtained from the PAIG-CNOP method and RIG-CNOP method in the optimization algorithm SPG2. Since there are multiple global maxima at points near this background point, the CNOPs obtained by the two methods were not list here. The MCFVs distributions of the final solution results of the PAIG-CNOP and RIG-CNOP are shown in Figure 3. The statistical analysis results of MCFVs calculated by above two methods, and the FMMCFV are shown in

Table 2. Statistical analysis of the MCFVs corresponding to the CNOPS generated by the different methods at point $O$, $T=150$.

Method	CFV	Proportion
Filtering	52.8287
RIG-CNOP	52.8291	6%
	31.4637	86%
	Else	8%
PAIG-CNOP	52.8299	94%
	31.4648	6%

. From Figure 3, it is easy to see that most of the MCFVs calculated by PAIG-CNOP in 50 trials are located on the same line as the one presented by the filtering method. According to Table 2, the probability of capturing the global CNOP by the RIG-CNOP method is only 6%, but the PAIG-CNOP method reaches 94%. The results show that the PAIG-CNOP method can greatly improve the probability of capturing the global CNOP in the case where the cost function has multiple extreme values.

3.3.2. Experimental Results for the Initial Background State of Point C₁

The initial background state is $C_1(-6\sqrt{2},-6\sqrt{2},27)$, a longer prediction time $T=400$ is chosen, adopting the same experimental scheme as for point $O$. Figure 4 shows that most of the MCFVs calculated by PAIG-CNOP in 50 trials are almost as consistent as the FMMCFV.

Table 3. Statistical analysis of the CFVs corresponding to the initial guesses generated by the different methods at point $C_1$, $T=400$.

Method	Maximum CFV	Mean CFV
Filtering	0.119260
RIG-CNOP	0.119259	0.117056
PAIG-CNOP	0.119260	0.119247

shows that the initial CFV mean value of 0.119247 for the PAIG-CNOP method is very similar to the FMMCFV 0.119260, while the CFV mean value of the RI-CNOP method is only 0.117056. It can be seen that the initial guesses selected by PAIG-CNOP are closer to the global CNOP than the RIG-CNOP method, which is very beneficial to capture the global optimal CNOP.

Then, the initial guesses of the two methods are substituted into the optimization algorithm for further iterative optimization. From Figure 5, it can be seen that all of the MCFVs calculated by PAIG-CNOP in 50 trials are located on the same line as the baseline. According to

Table 4. Statistical analysis of the MCFVs corresponding to the CNOPS generated by the different methods at point $C_1$, $T=400$.

Method	CFV	CNOP	Proportion
Filtering	0.119260	(8.71285 × 10−3, −5.10324 × 10−2, 6.09900 × 10−2)
RIG-CNOP	0.119260	(8.61310 × 10−3, −5.12034 × 10−2, 6.08607 × 10−2)	58%
	0.118604	(−8.45657 × 10−2, 5.14618 × 10−2, −6.06644×10−2)	42%
PAIG-CNOP	0.119260	(8.61199 × 10−3, −5.12044 × 10−2, 6.08600 × 10−2)	100%

, the PAIG-CNOP method captures the global CNOP with 100% probability in 50 trials, and the captured CNOPs are almost the same as the CNOP obtained by the filtering method. The RIG-CNOP method finds the global optimum with only 58% probability. This shows that the PAIG-CNOP method can effectively capture the global CNOP of the Lorenz-63 model for long prediction time.

3.3.3. Experimental Results for the Initial Background State of Point B

The initial background state is the non-equilibrium point $B\left(-3.12346395;-3.12529803;20.69823159\right)$, the prediction time $T=500$, the other test schemes are the same as the $O$-point test scheme. The nonlinearity of the prediction system is strong at this time. Figure 6 shows that the initial CFVs of the PAIG-CNOP method in 50 trials are all located around in the FMMCFV region, and the initial CFVs generated by the RIG-CNOP method are mostly smaller than the FMMCFV. From

Table 5. Statistical analysis of the CFVs corresponding to the initial guesses generated by the different methods at point $B$, $T=500$.

Method	Maximum CFV	Mean CFV
Filtering	42.1211
RIG-CNOP	42.0998	26.2008
PAIG-CNOP	42.1208	42.0928

, it is shown that the mean initial CFV of the RIG-CNOP method is only 26.2008, which is much smaller than the FMMCFV of 42.1211. The mean initial guess value generated by the PAIG-CNOP method is as high as 42.0928.

After further optimization calculations, the distribution of MCFVs of the two methods is shown in Figure 7. It can be seen that the initial guesses generated by the PAIG-CNOP method eventually converge to the global optimum. In

Table 6. Statistical analysis of the MCFVs corresponding to the CNOPS generated by the different methods at point $B$, $T=500$.

Method	CFV	CNOP	Proportion
Filtering	42.1211	(−2.3557 × 10−2, −5.2657 × 10−2, −5.5428 × 10−2)
RIG-CNOP	42.1211	(−2.3987 × 10−2, −5.2387 × 10−2, −5.5500 × 10−2)	40%
	26.5589	(−4.3248 × 10−2, −6.1176 × 10−2, 2.7256 × 10−2)	12%
	17.2833	(4.3084 × 10−2, 6.0609 × 10−2, −2.9502 × 10−2)	48%
PAIG-CNOP	42.1211	(−2.3978 × 10−2, −5.2392 × 10−2, −5.5498 × 10−2)	100%

, it can be seen that the PAIG-CNOP method captures the global CNOP probability of 100%, while the RIG-CNOP is only 40%. The CNOPs calculated by the PAIG-CNOP method are almost consistent with the filtering method. The results show that PAIG-CNOP can still effectively capture the global CNOP in the case of strong nonlinearity.

4. Discussion

In this section, we discuss limitations and the timeliness of the capturing global CNOP methods mentioned in this paper, including filtering method, RIG-CNOP, PAIG-CNOP, and PSO-CNOP methods, and make a simple test for the non-equilibrium point B. The results of the statistical analysis are shown in

Table A2. Comparison of the maximum number of iterations (max_iter) and CPU time between the 4 different methods when at point $B$, $T=500$.

Method	CFV	Max_Iter	CPU Time(s)
Filtering	42.1211		3.1885 × 104
RIG-CNOP	42.1211	238	14.9192
	26.5589	36	3.9709
	17.2833	35	3.9061
PAIG-CNOP	42.1211	60 + 219	18.5492
PSO-CNOP	42.1211	148	33.1798

in Appendix A.

Duan et al. [49] found that computing CNOP based on filtering methods is very time-consuming. As can be seen in Table A2, even with the simple model of Lorenz-63, the computational cost of the filtering method is very high, which therefore limits the application of capturing CNOP in realistic models.

Zheng et al. [32] showed that the PSO-CNOP method based on an intelligent optimization algorithm can also capture the global CNOP with high probability. However, real atmospheric ocean prediction models are usually high-dimensional, and the cost of using intelligent optimization algorithms to calculate CNOP is unacceptable. From Table A2, we can see that the computer time of PAIG-CNOP method is only half of that of PSO-CNOP method. The PAIG-CNOP method performs competitively compared to the PSO-CNOP method with smaller memory demands and lower computational costs.

Compared with the RIG-CNOP method, we can find that the computational cost of the PAIG-CNOP method is slightly higher from Table A2, but the PAIG-CNOP method can significantly improve the probability of capturing the global CNOP.

In summary, although the filtering method is effective in computing the global CNOP, the computational cost of this method is unacceptable. The PAIG-CNOP method and the PSO-CNOP method both capture the global CNOP with high probability, but the PAIG-CNOP method is less computationally expensive. Although PAIG-CNOP has a slightly higher computational cost compared to RIG-CNOP, the PAIG-CNOP method can efficiently compute CNOPs. Therefore, the PAIG-CNOP method is an efficient and useful method for solving CNOP. To further speed up the computation of the PAIG-CNOP method, the next target of the study is to improve the selection scheme of the initial guess members, then reduce the number of pre-analysis initial guesses.

In addition, all the CNOPs are computed through a simple perfectly identifiable model, i.e., both the prediction model (in which are added the initial perturbations) and the observation model coincide. Under non-perfect models, CNOPs are possibly useful only for small enough prediction periods without much influence of model’s errors [50].To further explore our method, implementation details with a Burgers function model and a real atmospheric or oceanic model will be discussed in future works.

5. Conclusions

For solving the global CNOP problem, the RIG-CNOP scheme chooses the initial guess randomly within the constraint range to obtain the CNOP. However, the RIG-CNOP scheme reduces the probability of capturing the global CNOP in many cases, such as the prediction model is strongly nonlinear or the prediction time is long, or the cost function has multiple extreme values. In the real atmospheric and ocean prediction models, the cost of using intelligent optimization algorithms to calculate CNOP is unacceptable. Therefore, we propose the PAIG-CNOP scheme for capturing the global CNOP, as it can capture the global CNOP with high probability and low computational cost, it also can be easily transplanted to other systems. In this new method, we give a new initial guess setting scheme. First, a series of initial guesses are selected randomly according to certain rules within the perturbation constraints range, then the initial CFVs of these initial guesses are analyzed to filter out an optimal initial guess. At last, we use this optimal initial guess in the EN-CNOP method to calculate the CNOP. Numerical tests are carried out using the Lorenz-63 model. The results show that the PAIG-CNOP method can always capture the global CNOP with high probability. However, the RIG-CNOP method only captures the global CNOP with a small probability. Furthermore, the PAIG-CNOP method has a smaller memory requirement and lower computational cost than the intelligent optimization methods. Meanwhile, we use the PAIG-CNOP scheme in the EN-CNOP method to compute the CNOP, avoiding the use of adjoint techniques in the maximization process. Therefore, the PAIG-CNOP scheme can be easily transplanted to other systems. The above analysis shows that the PAIG-CNOP scheme has good practicality. In conclusion, PAIG-CNOP is an effective method for studying predictability problems in the case of strong nonlinearity. In the future, we believe that the PAIG-CNOP scheme can be applied to obtain the global CNOP for high-dimensional complex nonlinear atmosphere and ocean prediction models.