Let us consider the solar radiative transfer in a rectangular prism of lengths
${L}_{\mathrm{x}}$,
${L}_{\mathrm{y}}$ and
${L}_{\mathrm{z}}$ as shown in
Figure 1. The top and bottom faces of the prism are denoted by
${S}_{\mathrm{t}}$ and
${S}_{\mathrm{b}}$, respectively, while
${S}_{1\mathrm{x}}$ (
$x=0$),
${S}_{2\mathrm{x}}$ (
$x={L}_{\mathrm{x}}$),
${S}_{1\mathrm{y}}$ (
$y=0$), and
${S}_{2\mathrm{y}}$ (
$y={L}_{\mathrm{y}}$) correspond to the lateral faces. The boundary-value problem for the total radiance at point
$\mathbf{r}$ in direction
$\mathbf{\Omega}$ consists of the inhomogeneous differential equation
the top-of-atmosphere boundary condition
and the Lambertian surface boundary condition
At the horizontal boundaries, periodic boundary conditions are assumed, i.e.,
for
${\mathbf{r}}_{i\mathrm{x}}\in {S}_{i\mathrm{x}}$ and
${\mathbf{r}}_{i\mathrm{y}}\in {S}_{i\mathrm{y}}$ with
$i=1,2$. In Equations (
1)–(
4),
$\lambda $ is the wavelength,
${\sigma}_{\mathrm{ext}}(\lambda ,\mathbf{r})$ and
${\sigma}_{\mathrm{sct}}(\lambda ,\mathbf{r})$ the extinction and scattering coefficients, respectively,
$P(\lambda ,\xb7)$ the phase function,
$A(\lambda )$ the surface albedo,
${\mathbf{\Omega}}_{0}=(-{\mu}_{0},{\phi}_{0})$ with
${\mu}_{0}>0$, the solar direction,
${F}_{0}(\lambda )$ the solar flux,
${\mathbf{r}}_{2\mathrm{x}}={\mathbf{r}}_{1\mathrm{x}}+{L}_{\mathrm{x}}\mathbf{i}$ and
${\mathbf{r}}_{2\mathrm{y}}={\mathbf{r}}_{1\mathrm{y}}+{L}_{\mathrm{y}}\mathbf{j}$ with
$(\mathbf{i},\mathbf{j},\mathbf{k})$ being the Cartesian unit vectors. Moreover,
${\mathbf{\Omega}}^{+}$ and
${\mathbf{\Omega}}^{-}$ stand for an upward and a downward direction, respectively,
$\Omega $ is the unit sphere, while
${\Omega}^{+}$ and
${\Omega}^{-}$ denote the upper and lower unit hemispheres, respectively,
An instrument placed at point
${\mathbf{r}}_{\mathrm{m}}$ measures the radiance at the top of the atmosphere in direction
${\mathbf{\Omega}}_{\mathrm{m}}^{0}=({\mu}_{\mathrm{m}}^{0},{\phi}_{\mathrm{m}}^{0})$,
${\mu}_{\mathrm{m}}^{0}>0$. Denoting by
$g(\lambda )$ the slit function of the instrument,
s the slit width, and
${S}_{\mathrm{tm}}$ the footprint of the instrument on the top face
${S}_{\mathrm{t}}$, the spectral signal measured by the instrument at wavelength
${\lambda}_{\mathrm{m}}$ in the spectral interval
$[{\lambda}_{\mathrm{min}},{\lambda}_{\mathrm{max}}]$ is given by
where
is the signal integrated over the field of view of the instrument at wavelength
${\lambda}^{\prime}$,
${\mathbf{\Omega}}_{\mathrm{m}}({\mathbf{r}}_{\mathrm{t}})$ the unit vector along the line connecting the points
${\mathbf{r}}_{\mathrm{t}}\in {S}_{\mathrm{tm}}$ and
${\mathbf{r}}_{\mathrm{m}}$, i.e.,
${\mathbf{\Omega}}_{\mathrm{m}}({\mathbf{r}}_{\mathrm{t}})=({\mathbf{r}}_{\mathrm{m}}-{\mathbf{r}}_{\mathrm{t}})/|{\mathbf{r}}_{\mathrm{m}}-{\mathbf{r}}_{\mathrm{t}}|$ (see
Figure 1), and
and
${A}_{\mathrm{tm}}$ the characteristic function and the area of the instrument footprint, respectively. Assuming that the distance from the top of the atmosphere to the instrument is large, we approximate
$I({\lambda}^{\prime},\mathbf{r},{\mathbf{\Omega}}_{\mathrm{m}}(\mathbf{r}))\approx I({\lambda}^{\prime},\mathbf{r},{\mathbf{\Omega}}_{\mathrm{m}}^{0})$ in Equation (
6).
For an atmosphere consisting of gas molecules and a cloud, we suppose that
The extinction coefficient is then computed as
where
${\sigma}_{\mathrm{ext}}^{\mathrm{cloud}}(\mathbf{r})$ is the extinction coefficient in the cloud,
${\sigma}_{\mathrm{sct}}^{\mathrm{mol}}(\lambda ,z)$ the molecular scattering coefficient due to Rayleigh scattering,
${\sigma}_{\mathrm{abs}}^{\mathrm{gas}}(\lambda ,z)$ the gas absorption coefficient, and
$(x,y,z)$ the Cartesian coordinates of point
$\mathbf{r}$. For the scattering coefficient, we have a similar representation, namely,
where
${\omega}_{\mathrm{cloud}}$ is the single-scattering albedo of the cloud.
The goal of our analysis is to design an approach for accelerating the computation of the spectral signal $\mathcal{I}({\lambda}_{\mathrm{m}})$ at a set of ${W}_{\mathrm{m}}$ spectral points ${\left\{{\lambda}_{\mathrm{m}w}\right\}}_{w=1}^{{W}_{\mathrm{m}}}$ characterizing the instrument. For this purpose, we combine the correlated k-distribution method with some dimensionality reduction techniques.
2.1. Correlated k-Distribution Method
The correlated
k-distribution method [
4,
12] is based on grouping spectral intervals according to absorption coefficient strength. In this section we provide a description of the correlated
k-distribution method, which will enable us to connect this method with dimensionality reduction techniques.
Let
${\left\{{\overline{\lambda}}_{k}\right\}}_{k=1}^{\overline{W}}$ be a discrete set of
$\overline{W}$ equally spaced wavelengths in the interval
$[{\lambda}_{\mathrm{min}}-s/2,{\lambda}_{\mathrm{max}}+s/2]$ with
where
$\Delta \lambda $ the discretization step. The forward-model spectral set
${\left\{{\overline{\lambda}}_{k}\right\}}_{k=1}^{\overline{W}}$ is finer than the measurement spectral set
${\left\{{\lambda}_{\mathrm{m}w}\right\}}_{w=1}^{{W}_{\mathrm{m}}}$; in practice,
$\overline{W}$ is chosen as a multiple of
${W}_{\mathrm{m}}$, i.e.,
$\overline{W}=c{W}_{\mathrm{m}}$ with say,
c being an integer greater than
$20.$ The convolution integral giving the expression of the spectral signal measured by the instrument can be approximated by
As gas absorption has stronger spectral variation than molecular and particulate scattering, we write formally (at a given altitude level
z)
${\mathcal{I}}_{\mathrm{FOV}}(\lambda )={\mathcal{I}}_{\mathrm{FOV}}({\sigma}_{\mathrm{abs}}^{\mathrm{gas}}(\lambda ))$. The accurate technique for computing the integral in Equation (
10) is a line-by-line (LBL) calculation of the gas absorption coefficient versus wavelength. Contrarily, the correlated
k-distribution method is based on the fact that for a homogeneous atmosphere, the transmission within a spectral interval is independent of the LBL variation of
${\sigma}_{\mathrm{abs}}^{\mathrm{gas}}$ with respect to the wavelength, but rather depends only on the distribution of
${\sigma}_{\mathrm{abs}}^{\mathrm{gas}}$ within the spectral interval [
13]. In this regard, let
$F=F({\sigma}_{\mathrm{abs},k}^{\mathrm{gas}})$ be the cumulative density function of
${\sigma}_{\mathrm{abs}}^{\mathrm{gas}}(\lambda )$ in the spectral interval
$[{\overline{\lambda}}_{k}-\Delta \lambda /2,{\overline{\lambda}}_{k}+\Delta \lambda /2]$, and
${\sigma}_{\mathrm{abs},k}^{\mathrm{gas}}(F)$ the quantile function (i.e., the inverse function of the cumulative distribution function). The spectral signal measured by the instrument can then be computed as
where
${\{{F}_{l},{\varpi}_{l}\}}_{l=1}^{{N}_{\mathrm{q}}}$ is a set of
${N}_{\mathrm{q}}$ quadrature points and weights in the interval
$[0,1]$. The
${\sigma}_{\mathrm{abs},k}^{\mathrm{gas}}({F}_{l})$ can be computed by inverting the cumulative density functions of the LBL gas absorption coefficients, or in the case of the “exponential sum fitting of transmittance” method [
14], by solving a nonlinear least squares problem.
In the final step, we define the sets of wavelengths and weights
${\left\{{\lambda}_{w}\right\}}_{w=1}^{W}$ and
${\left\{{\omega}_{w}\right\}}_{w=1}^{W}$ with
$W=\overline{W}{N}_{\mathrm{q}}$, respectively, through the relations
${\lambda}_{w}={\overline{\lambda}}_{k}$ and
${\omega}_{w}=\Delta \lambda \phantom{\rule{0.166667em}{0ex}}{\varpi}_{l}$, where
$w=l+(k-1){N}_{\mathrm{q}}$ for
$k=1,\dots ,\overline{W},\phantom{\rule{0.166667em}{0ex}}l=1,\dots ,{N}_{\mathrm{q}}$. Accordingly, we set
${\sigma}_{\mathrm{abs}}^{\mathrm{gas}}({\lambda}_{w})={\sigma}_{\mathrm{abs},k}^{\mathrm{gas}}({F}_{l})$. Note that
${\left\{{\lambda}_{w}\right\}}_{w=1}^{W}$ contains
$\overline{W}$ groups of
${N}_{\mathrm{q}}$ identical wavelengths. By this construction, Equation (
11) becomes
Note that Equation (
12) is a quadrature rule for the convolution integral (
10) in the case of the correlated
k-distribution method.
The computation of the spectral signal by means of the correlated k-distribution method requires the computation of the absorption coefficients ${\sigma}_{\mathrm{abs}}^{\mathrm{gas}}({\lambda}_{w},z)$ at the set of wavelengths ${\left\{{\lambda}_{w}\right\}}_{w=1}^{W}$, and so, W monochromatic radiative transfer calculations. For applications with $c\ge 20$ and ${N}_{\mathrm{q}}\ge 4$, $W=\overline{W}{N}_{\mathrm{q}}=c{W}_{\mathrm{m}}{N}_{\mathrm{q}}$ can be in the order of hundreds, and a further acceleration is needed. This can be achieved by applying dimensionality reduction techniques on the optical parameters.
2.2. Dimensionality Reduction of Atmospheric Optical Parameters
Following Ref. [
11], the dimensionality reduction problem of the optical parameters can be formulated as follows. For each wavelength
${\lambda}_{w}$, we define an
N-dimensional vector
${\mathbf{x}}_{w}$ by
where
${\sigma}_{\mathrm{abs},k}^{\mathrm{gas}}$ and
${\sigma}_{\mathrm{sct},k}^{\mathrm{mol}}$ are the optical coefficients in the
kth level,
$N=2{N}_{\mathrm{z}}+1$, and
${N}_{\mathrm{z}}$ is the number of altitude levels. For simplicity, the surface albedo
A was not included in
${\mathbf{x}}_{w}$ (we assumed that the variations of
A are negligible over the width of a molecular absorption band). Thus, the wavelength variability of the optical parameters is encapsulated in the vector
${\mathbf{x}}_{w}\in {\mathbb{R}}^{N}$, and there is a one-to-one correspondence between the discrete set of wavelengths
${\left\{{\lambda}_{w}\right\}}_{w=1}^{W}$, and the discrete set of optical parameters
${\left\{{\mathbf{x}}_{w}\right\}}_{w=1}^{W}$. For the
N-dimensional data set
${\left\{{\mathbf{x}}_{w}\right\}}_{w=1}^{W}$,
$\overline{\mathbf{x}}=(1/W){\sum}_{w=1}^{W}{\mathbf{x}}_{w}$ is the sample mean of the data. The goal of a dimensionality reduction technique is to find an
M-dimensional subspace (
$M<N$) spanned by a set of linear independent vectors
${\left\{{\mathbf{a}}_{k}\right\}}_{k=1}^{M}$, such that the centered data
${\mathbf{x}}_{w}-\overline{\mathbf{x}}$ belong to this subspace; that is,
In Equation (
14), the matrix
$\mathbf{A}={\left[{\mathbf{a}}_{k}\right]}_{k=1}^{M}\in {\mathbb{R}}^{N\times M}$, encapsulating the column vectors
${\mathbf{a}}_{k}$, represents the inverse transform from the low-dimensional space to the high-dimensional space, and
${y}_{wk}$ is the
kth component of
${\mathbf{y}}_{w}\in {\mathbb{R}}^{M}$. The vector
${\mathbf{y}}_{w}$ is given by the forward transform from the high-dimensional space to the low-dimensional space,
where
${\mathbf{A}}^{\u2020}={({\mathbf{A}}^{T}\mathbf{A})}^{-1}{\mathbf{A}}^{T}\in {\mathbb{R}}^{M\times N}$ is the pseudoinverse of
$\mathbf{A}$.
An approximate model for estimating the integrated signal of the instrument at wavelength
$\lambda $ is of the form
where
${\mathcal{I}}_{\mathrm{FOV}}^{\mathrm{a}}(\lambda )$ is the integrated signal computed by an approximate radiative transfer model, and
$f(\lambda )$ a correction factor. The correction factor
$f(\lambda )$ can be calculated efficiently and accurately by means of dimensionality reduction techniques applied on the optical parameters. Let us assume that the scalar function
$f({\mathbf{x}}_{w})$ given by Equation (
16) is not too nonlinear in
${\mathbf{x}}_{w}$. The computational process is organized as follows.
$f({\mathbf{x}}_{w})$ is approximated by a second-order Taylor expansion around
$\overline{\mathbf{x}}$; that is,
where
$\nabla f$ and
${\nabla}^{2}f$ are the gradient and the Hessian of
f, respectively.
For
${\mathbf{a}}_{k}=\left|\right|{\mathbf{a}}_{k}\left|\right|{\widehat{\mathbf{a}}}_{k}$ and in view of Equation (
17), the second and third terms in Equation (
18) are written as
and
respectively.
The mixed directional derivatives in Equation (
20) are neglected, while the first- and second-order directional derivatives are approximated by means of central differences; that is,
and
respectively.
For
$h=\left|\right|{\mathbf{a}}_{k}\left|\right|$, Equations (
18)–(
22) yield the computational formula:
From Equation (
23), it is apparent that the computation of
$f({\mathbf{x}}_{w})$ requires
$2M+1$ calls of the exact and approximate models, while the computation of
${\mathcal{I}}_{\mathrm{FOV}}({\lambda}_{w})$ requires, in addition,
W calls of the approximate model. In this regard and taking into account that
$M\ll W$, we are led to a substantial reduction of the computational time.
The most widely used dimensionality reduction techniques are the linear embedding methods which are summarized below.
The principal component analysis (PCA) [
15] performs a dimensionality reduction by projecting the original
N-dimensional data on the
M-dimensional subspace spanned by the dominant singular vectors of the data covariance matrix.
The locality pursuit embedding (LPE) [
16] performs a principal component analysis on local nearest neighbor patches to reveal the tangent space structure on the
M-dimensional subspace.
The locality preserving projection (LPP) [
17] solves a variational problem that optimally preserves the neighborhood structure of the data set.
The locally embedded analysis (LEA) [
18] uses an embedding strategy based on a linear eigenspace analysis to minimize the local reconstruction error.
It should be pointed out that PCA preserves only the global structure of the data, and may fail to preserve the local structure if the data lie on a nonlinear manifold. In contrast, LPE, LPP, and LEA are local linear approaches, which optimally preserve local neighborhood information (the local structure of the data) in a certain sense.
The PCA, LPE, LPP, and LEA methods are illustrated in Algorithms A1–A4 of
Appendix A. The outputs of these algorithms are the linear mapping
$\mathbf{A}$ and the vectors of parameters
${\mathbf{y}}_{w}$,
$w=1,\dots ,W$. Note that in the PCA language, the column vectors
${\mathbf{a}}_{k}$,
$k=1,\dots ,M$ of
$\mathbf{A}$ are the so-called empirical orthogonal functions, while the vectors of parameters
${\mathbf{y}}_{w}$,
$w=1,\dots ,W$ are the principal components.
A last problem which has to be solved is the choice of an approximate model. Two special features of SHDOM facilitate this choice, namely, the use (i) of spherical harmonics and discrete ordinates to represent the radiance field, and (ii) of a sort of successive order of scattering solution method (Picard iteration). In a first approximate model, we choose 2 zenith discrete angles per hemisphere to represent the radiance field; analogous to the one-dimensional radiative transfer, the resulting method is called the four-stream approximation. In a second approximate model, we consider only 2 steps of the method of Picard iteration; the resulting method is called the second-order of scattering approximation.
In summary, the spectral acceleration approach includes two steps. The first step is a pre-processing step, consisting in the computation of the sets of wavelengths ${\left\{{\lambda}_{w}\right\}}_{w=1}^{W}$, the absorption coefficients ${\sigma}_{\mathrm{abs}}^{\mathrm{gas}}({\lambda}_{w},z)$, and the weighting factors ${\left\{{\omega}_{w}\right\}}_{w=1}^{W}$ in the framework of the correlated k-distribution method. The second step is a computational step, consisting of the calculation of
the data vectors ${\mathbf{x}}_{w}$, $w=1,\dots ,W$, the empirical orthogonal functions ${\mathbf{a}}_{k}$, $k=1,\dots ,M$, and the principal components ${\mathbf{y}}_{w}$, $w=1,\dots ,W$ in the framework of dimensionality reduction techniques,
the correction factor
$f({\mathbf{x}}_{w})$,
$w=1,\dots ,W$, and the integrated signal
${\mathcal{I}}_{\mathrm{FOV}}({\lambda}_{w})$,
$w=1,\dots ,W$ according to Equations (
23) and (
16), respectively, and finally,
the spectral signal measured by the instrument
$\mathcal{I}({\lambda}_{\mathrm{mw}})$,
$w=1,\dots ,{W}_{\mathrm{m}}$ by means of Equation (
10).