Complexity Estimation of Cubical Tensor Represented through 3D Frequency-Ordered Hierarchical KLT

Abstract

In this work is introduced one new hierarchical decomposition for cubical tensor of size 2ⁿ, based on the well-known orthogonal transforms Principal Component Analysis and Karhunen–Loeve Transform. The decomposition is called 3D Frequency-Ordered Hierarchical KLT (3D-FOHKLT). It is separable, and its calculation is based on the one-dimensional Frequency-Ordered Hierarchical KLT (1D-FOHKLT) applied on a sequence of matrices. The transform matrix is the product of n sparse matrices, symmetrical at the point of their main diagonal. In particular, for the case in which the angles which define the transform coefficients for the couples of matrices in each hierarchical level of 1D-FOHKLT are equal to π/4, the transform coincides with this of the frequency-ordered 1D Walsh–Hadamard. Compared to the hierarchical decompositions of Tucker (H-Tucker) and the Tensor-Train (TT), the offered approach does not ensure full decorrelation between its components, but is close to the maximum. On the other hand, the evaluation of the computational complexity (CC) of the new decomposition proves that it is lower than that of the above-mentioned similar approaches. In correspondence with the comparison results for H-Tucker and TT, the CC decreases fast together with the increase of the hierarchical levels’ number, n. An additional advantage of 3D-FOHKLT is that it is based on the use of operations of low complexity, while the similar famous decompositions need large numbers of iterations to achieve the coveted accuracy.

1. Introduction

The well-known tensor decompositions Canonical Polyadic Decomposition (CPD), Higher-Order Singular Value Decomposition (HOSVD) [1,2,3], Tensor Train (TT) Decomposition [4], Hierarchical Tucker decomposition (H-Tucker) [5] and some of their modifications [6,7] are based on the calculation of the eigenvalues and eigenvestors of the decomposed tensor. The basic attribute of this group is that they are optimal regarding the minimization of the mean square approximation error derived from the “truncation” of the low-energy components. For the calculation of the retained components, we use iterative methods which need relatively large numbers of mathematical operations to achieve the requested accuracy (Power method [8], Jacoby method [9], etc.).

The hierarchical tensor decompositions based on the H-Tucker algorithm are generalized in publications [7] and [10]. In [7], Vasilescu and Kim introduced a compositional hierarchical tensor factorization that disentangles the hierarchical causal structure of object image formation, but the computational complexity (CC) of the method is not evaluated. In [10], Zniyed et al. offered the TT-based hierarchical decomposition of high-order tensors, for the calculation of which is used the Tensor-Train Hierarchical SVD (TT-HSVD). This approach permits parallel processing, which accelerates the calculations. Unlike the TT-SVD algorithm, the TT-HSVD is based on applying SVDs to matrices of smaller dimensions. As a result, the CC of TT-SVD is O(R²N³) operations higher than that of TT-HSVD (here, R is the rank of the hypercubical tensor of size N).

As an alternative, in this work are presented new hierarchical 3D tensor decompositions, based on the famous statistical orthogonal transforms Principal Component Analysis (PCA) and the Karhunen–Loeve Transform (KLT) [11,12]. The offered decompositions are close to the optimal (which ensure full decorrelation of their components), but do not need iterations and have low computational complexity.

In Section 2, we present the methods for one-dimensional hierarchical adaptive transform based on the PCA (1D-HAPCA) and the Hierarchical Adaptive Karhunen–Loeve Transform (1D-HAKLT). In Section 3, on the basis of these 1D transforms we give the details of the cubical tensor decomposition through separable 3D Frequency-Ordered Hierarchical KLT (3D-FOHKLT), and Section 4 contains the related algorithm. Section 5 presents the comparative analysis of the computational complexity of the new approaches with respect to the well-known hierarchical Tucker decomposition (H-Tucker) and the TT decomposition. Section 6 is the Conclusion.

2. One-Dimensional Hierarchical Adaptive PCA/KLT Transform for a Sequence of Matrices

The detailed description of the algorithm for the one-dimensional adaptive PCA/KLT transform (1D-APCA/AKLT) for a couple of matrices is given below. This is the basis of the hierarchical algorithm for processing the sequence of matrices which represent the corresponding tensor.

Throughout this paper, scalars are represented as x, vectors and matrices as x and X correspondingly, and tensors as $\mathcal{X}$. It is also assumed here that all tensor elements (entries) are nonnegative real numbers x(i,j,k).

2.1. One-Dimensional Adaptive PCA/KLT Transform for a Sequence of Matrices

The transform is executed in the following way.

First, on the elements x_1s and x_2s, which have the same spatial position in the couple of matrices X₁ and X₂ of size N × N, are applied one-dimensional direct and inverse adaptive PCA, i.e., the 1D-APCA [12] is:

$$\begin{gathered} \begin{array}{l}{\mathrm{y}}_{1\mathrm{s}}={(\mathrm{x}}_{1\mathrm{s}}-{\mathrm{m}}_1){\cos \mathsf{\theta }}_{1,2}+{(\mathrm{x}}_{2\mathrm{s}}-{\mathrm{m}}_2){\sin \mathsf{\theta }}_{1,2};\\ {\mathrm{y}}_{2\mathrm{s}}={(\mathrm{x}}_{1\mathrm{s}}-{\mathrm{m}}_1){\sin \mathsf{\theta }}_{1,2}-({\mathrm{x}}_{2\mathrm{s}}-{\mathrm{m}}_2){\cos \mathsf{\theta }}_{1,2};\end{array}\begin{array}{l}{\mathrm{x}}_{1\mathrm{s}}={\mathrm{y}}_{1\mathrm{s}}{\cos \mathsf{\theta }}_{1,2}+{\mathrm{y}}_{2\mathrm{s}}{\sin \mathsf{\theta }}_{1,2}+{\mathrm{m}}_1;\\ {\mathrm{x}}_{2\mathrm{s}}={\mathrm{y}}_{1\mathrm{s}}{\sin \mathsf{\theta }}_{1,2}-{\mathrm{y}}_{2\mathrm{s}}{\cos \mathsf{\theta }}_{1,2}+{\mathrm{m}}_2,\end{array} \\ \mathrm{for}\mathrm{s}=1,2,\dots ,{\mathrm{N}}^2 \end{gathered}$$

(1)

Here, y_1s and y_2s are the corresponding elements in the couple of transformed matrices Y₁ and Y₂, each of size N × N,

$${\mathsf{\theta }}_{1,2}=\mathrm{arctg}{\{2\mathrm{k}}_3/({\mathrm{k}}_1-{\mathrm{k}}_2+\sqrt{{{(\mathrm{k}}_1-{\mathrm{k}}_2)}^2+{4\mathrm{k}}_3^2})\};$$

(2)

$${\mathrm{k}}_1=\mathrm{E}({\mathrm{x}}_{1\mathrm{s}}^2)-{\mathrm{m}}_1^2,{\mathrm{k}}_2=\mathrm{E}({\mathrm{x}}_{2\mathrm{s}}^2)-{\mathrm{m}}_2^2,{\mathrm{k}}_3=\mathrm{E}({\mathrm{x}}_{1\mathrm{s}}{\mathrm{x}}_{2\mathrm{s}})-{\mathrm{m}}_1{\mathrm{m}}_2;$$

(3)

$${\mathrm{m}}_1=\mathrm{E}({\mathrm{x}}_{1\mathrm{s}}),{\mathrm{m}}_2=\mathrm{E}({\mathrm{x}}_{2\mathrm{s}}),$$

(4)

where $\mathrm{m}=\mathrm{E}({\mathrm{x}}_{\mathrm{s}})=(1/{\mathrm{N}}^2){\displaystyle \sum _{\mathrm{s}=1}^{{\mathrm{N}}^2}{\mathrm{x}}_{\mathrm{s}}}$ is the averaging operator.

To calculate the angle θ_1,2 it is necessary to define the extended arctg function. If β₂ = 2k₃ and ${\mathsf{\beta }}_1={\mathrm{k}}_1-{\mathrm{k}}_2+\sqrt{{{(\mathrm{k}}_1-{\mathrm{k}}_2)}^2+{4\mathrm{k}}_3^2}$ are two real numbers, the extended arctg function ε_arctg(β₁, β₂) is mapped to (−π, π] by adding π [13]:

$${\mathsf{\epsilon }}_{\mathrm{arctg}}{(\mathsf{\beta }}_1{,\mathsf{\beta }}_2)=\left\{\begin{array}{l}{\mathrm{arctg}(\mathsf{\beta }}_2/{\mathsf{\beta }}_1{),\mathrm{if}\mathsf{\beta }}_1>0;\\ {\mathrm{arctg}(\mathsf{\beta }}_2/{\mathsf{\beta }}_1)+{\mathsf{\pi },\mathrm{if}\mathsf{\beta }}_1<{0,\mathsf{\beta }}_2\ge 0;\\ {\mathrm{arctg}(\mathsf{\beta }}_2/{\mathsf{\beta }}_1)-{\mathsf{\pi },\mathrm{if}\mathsf{\beta }}_1<{0,\mathsf{\beta }}_2<0;\\ {\mathsf{\pi }/2,\mathrm{if}\mathsf{\beta }}_1={0,\mathsf{\beta }}_2>0;\\ -{\mathsf{\pi }/2,\mathrm{if}\mathsf{\beta }}_1={0,\mathsf{\beta }}_2<0;\\ {0,\mathrm{if}\mathsf{\beta }}_2={\mathsf{\beta }}_1=0.\end{array}\right.$$

(5)

In particular, for ${\mathrm{m}}_1={\mathrm{m}}_2=0$ (valid for 1D-AKLT) and ${\mathsf{\theta }}_{1,2}=\mathsf{\pi }/4$, the relations (1) become as follows:

$$\begin{array}{l}{\mathrm{y}}_{1\mathrm{s}}=(1/\sqrt{2})({\mathrm{x}}_{1\mathrm{s}}+{\mathrm{x}}_{2\mathrm{s}});\\ {\mathrm{y}}_{2\mathrm{s}}=(1/\sqrt{2})({\mathrm{x}}_{1\mathrm{s}}-{\mathrm{x}}_{2\mathrm{s}});\end{array}\begin{array}{l}{\mathrm{x}}_{1\mathrm{s}}=(1/\sqrt{2})({\mathrm{y}}_{1\mathrm{s}}+{\mathrm{y}}_{2\mathrm{s}});\\ {\mathrm{x}}_{2\mathrm{s}}=(1/\sqrt{2})({\mathrm{y}}_{1\mathrm{s}}-{\mathrm{y}}_{2\mathrm{s}}).\end{array}$$

(6)

Hence, in this case the 1D-KLT coincides with the 1D Walsh–Hadamard Transform (1D-WHT).

2.2. One-Dimensional Hierarchical Adaptive KLT for a Sequence of Matrices

The relations above, used for the calculation of 1D-APCA [12], are the kernel for the processing of a sequence of matrices (for example, a sequence of moving images), and do not need iterative calculations. The procedure is executed in the following way. First, the sequence of matrices X_k, each of size N × N for k = 1, 2,…, N is divided into N/2 couples (N = 2ⁿ). For each couple of matrices with elements x_1s(p,q) and x_2s(p,q), when q = 1, 2, …, N/2, in the level p = 1, 2, …, n, is executed the one-dimensional adaptive PCA (1D-APCA), in accordance with Equations (1)–(5). For ${\mathrm{m}}_1(\mathrm{p},\mathrm{q})={\mathrm{m}}_2(\mathrm{p},\mathrm{q})=0$ (valid for the KLT), the direct 1D-AKLT for the couple of matrices q in the level p of the 1D-HAKLT is defined by the relations

$$\begin{array}{c}{\mathrm{y}}_{1\mathrm{s}}(\mathrm{p},\mathrm{q})={\mathrm{x}}_{1\mathrm{s}}(\mathrm{p},\mathrm{q})\cos {\mathsf{\theta }}_{\mathrm{p},\mathrm{q}}+{\mathrm{x}}_{2\mathrm{s}}(\mathrm{p},\mathrm{q})\sin {\mathsf{\theta }}_{\mathrm{p},\mathrm{q}}\\ {\mathrm{y}}_{2\mathrm{s}}(\mathrm{p},\mathrm{q})={\mathrm{x}}_{1\mathrm{s}}(\mathrm{p},\mathrm{q})\sin {\mathsf{\theta }}_{\mathrm{p},\mathrm{q}}-{\mathrm{x}}_{2\mathrm{s}}(\mathrm{p},\mathrm{q})\cos {\mathsf{\theta }}_{\mathrm{p},\mathrm{q}}\end{array}\mathrm{for}\mathrm{s}=1,2,\dots ,{\mathrm{N}}^2,$$

(7)

where s is the sequential number of two elements of the same position in the couple of matrices, q,

$$\begin{gathered} {\mathsf{\theta }}_{\mathrm{p},\mathrm{q}}=\mathrm{arctg}\{2{\mathrm{k}}_3(\mathrm{p},\mathrm{q})/[{\mathrm{k}}_1(\mathrm{p},\mathrm{q})-{\mathrm{k}}_2(\mathrm{p},\mathrm{q})+\sqrt{{({\mathrm{k}}_1(\mathrm{p},\mathrm{q})-{\mathrm{k}}_2(\mathrm{p},\mathrm{q}))}^2+4{\mathrm{k}}_3^2(\mathrm{p},\mathrm{q})}]\}, \\ {\mathrm{k}}_1(\mathrm{p},\mathrm{q})=\mathrm{E}({\mathrm{x}}_{1\mathrm{s}}^2(\mathrm{p},\mathrm{q}));{\mathrm{k}}_2(\mathrm{p},\mathrm{q})=\mathrm{E}({\mathrm{x}}_{2\mathrm{s}}^2(\mathrm{p},\mathrm{q}));{\mathrm{k}}_3(\mathrm{p},\mathrm{q})=\mathrm{E}({\mathrm{x}}_{1\mathrm{s}}(\mathrm{p},\mathrm{q}){\mathrm{x}}_{2\mathrm{s}}(\mathrm{p},\mathrm{q})). \end{gathered}$$

(8)

In the equations, figures and text below are introduced the following symbols: ${\cos \mathsf{\theta }}_{\mathrm{p},\mathrm{q}}={\mathrm{c}}_{\mathrm{p},\mathrm{q}}$ and ${\sin \mathsf{\theta }}_{\mathrm{p},\mathrm{q}}={\mathrm{s}}_{\mathrm{p},\mathrm{q}}$ for p = 1, 2, 3 and q = 1, 2, 3, 4. The execution graph for 1D-HAKLT of three hierarchical levels (N = 8) is shown on Figure 1a. In every level p = 1, 2, 3, and for each couple of neighbor matrices q (fenced in red ellipses) 1D-APCA is executed. In result, the first transformed matrix calculated for each couple q has higher energy than the second matrix, and they are mutually decorrelated. To continue with the operations in the next level of 1D-HAKLT, these matrices are reordered, following the decrease of their energy, i.e.,

$${\mathrm{P}}_{Y_{\mathrm{p},0}}\ge {\mathrm{P}}_{Y_{\mathrm{p},1}}\ge {\mathrm{P}}_{Y_{\mathrm{p},2}}\ge \dots {\mathrm{P}}_{Y_{\mathrm{p},\mathrm{k}}}\ge \dots \ge {\mathrm{P}}_{Y_{\mathrm{p},7}},$$

(9)

where ${\mathrm{P}}_{Y_{\mathrm{p},\mathrm{k}}}={\displaystyle \sum _{\mathrm{i}=1}^8{\displaystyle \sum _{\mathrm{j}=1}^8{[\mathrm{y}}_{\mathrm{p},\mathrm{k}}(\mathrm{i},\mathrm{j})}}{]}^2$ is the energy of matrices Y_p,k with elements ${\mathrm{y}}_{\mathrm{p}.\mathrm{k}}(\mathrm{i},\mathrm{j})$, obtained in result of the execution of 1D-HAKLT in levels p = 1, 2, 3, before the reordering. In result, the reordered matrices E_r in the last level are decorrelated to the highest degree [12].

The reordering in each level p in accordance with Equation (9) needs significant computational resources. For the case when the connections between the outputs Y_p,k for k = 1, 2, …, 8 in the level p and the inputs in the next level p + 1 are fixed, reordering is not performed, and the 1D-HAKLT turns into the one-dimensional Frequency-Ordered Hierarchical KLT (1D-FOHKLT), whose execution graph is shown in Figure 1b. In this case, only the output matrices in the last (third) level are reordered, which does not require mathematical operations. Besides, the energy distribution of matrices Y_p,k is close to that defined by Equation (9). In particular, in accordance with Appendix A, if ${\mathsf{\theta }}_{\mathrm{p},\mathrm{q}}=\mathsf{\pi }/4$ for p = 1, 2, 3 and q = 1, 2, 3, 4, then 1D-HAKLT coincides with the one-dimensional frequency-ordered Fast Walsh–Hadamard Transform (1D-FWHT) for N = 8 [14].

Both algorithms are generalized below for N = 2ⁿ. The algorithm for fast hierarchical 1D-HAKLT is represented through the sequential transformation of the vectors’ elements s in each decomposition level p, in accordance with the relations

$$y_{1,\mathrm{s}}=P_1(2^{\mathrm{n}})G_1(2^{\mathrm{n}}){\mathrm{x}}_{\mathrm{s}}\mathrm{for}\mathrm{p}=1\mathrm{and}\mathrm{s}=1,2,\dots {,\mathrm{N}}^2;$$

(10)

$$y_{\mathrm{p},\mathrm{s}}=P_{\mathrm{p}}(2^{\mathrm{n}})G_{\mathrm{p}}(2^{\mathrm{n}})y_{\mathrm{p}-1,\mathrm{s}}\mathrm{for}\mathrm{p}=2,3,\dots ,\mathrm{n}-1\mathrm{and}\mathrm{s}=1,2,\dots ,{\mathrm{N}}^2;$$

(11)

$$e_{\mathrm{s}}=P_{\mathrm{n}}(2^{\mathrm{n}})G_{\mathrm{n}}(2^{\mathrm{n}})y_{\mathrm{n}-1,\mathrm{s}}.\mathrm{for}\mathrm{p}=\mathrm{n}\mathrm{and}\mathrm{s}=1,2,\dots {,\mathrm{N}}^2.$$

(12)

Here, x_s and e_s are respectively the input and the output column-vector of N components correspondingly, and the column-vectors y_p,s for p = 2, 3,…, n − 1 are the intermediate results. The relation between vectors e_s and x_s, defined on the basis of Equations (10)–(12), is given below:

$$e_{\mathrm{s}}=[{\displaystyle \prod _{\mathrm{p}=1}^{\mathrm{n}}{P}_{\mathrm{p}}{(2}^{\mathrm{n}})G_{\mathrm{p}}{(2}^{\mathrm{n}})}]{\mathrm{x}}_{\mathrm{s}}=T_{\mathrm{HAKLT}}{(2}^{\mathrm{n}}){\mathrm{x}}_{\mathrm{s}}\mathrm{for}\mathrm{s}=1,2,\dots ,{\mathrm{N}}^2,$$

(13)

where

$$T_{\mathrm{HAKLT}}(2^{\mathrm{n}})={\displaystyle \prod _{\mathrm{p}=1}^{\mathrm{n}}{P}_{\mathrm{p}}{(2}^{\mathrm{n}}\left)G_{\mathrm{p}}{(2}^{\mathrm{n}}\right)}$$

(14)

is the matrix for the fast n-level 1D-HAKLT. The sparse transform matrices $G_{\mathrm{p}}(2^{\mathrm{n}})$ of size 2ⁿ × 2ⁿ have a symmetrical block-diagonal structure towards the main diagonal, and are defined by the relations

$$G_{\mathrm{p}}(2^{\mathrm{n}})=\underset{\mathrm{q}=1}{\overset{2^{\mathrm{n}-1}}{\oplus }}{T}_{\mathrm{p},\mathrm{q}}(2)=\mathrm{diag}\{T_{\mathrm{p},1}(2),T_{\mathrm{p},2}(2),\dots ,T_{\mathrm{p},2^{\mathrm{n}-1}}(2)\}\mathrm{for}\mathrm{p}=1,2,\dots ,\mathrm{n},$$

(15)

where $T_{\mathrm{p},\mathrm{q}}(2)=\left[\begin{array}{cc}{\mathrm{c}}_{\mathrm{p},\mathrm{q}}& {\mathrm{s}}_{\mathrm{p},\mathrm{q}}\\ {\mathrm{s}}_{\mathrm{p},\mathrm{q}}& {-\mathrm{c}}_{\mathrm{p},\mathrm{q}}\end{array}\right]$, and ⊕ denotes the direct sum of matrices. The permutation matrices $P_{\mathrm{p}}(2^{\mathrm{n}})$ of size 2ⁿ × 2ⁿ for level p = 1, 2, …, n of 1D-HAKLT are defined on the basis of Equation (9) for k = 0, 1, 2, …, 2ⁿ⁻¹.

The algorithm 1D-FOHKLT is depicted by a relation similar to Equation (13), but the reordering is executed only in the last level, n;

$$e_{\mathrm{s}}=P_{\mathrm{n}}{(2}^{\mathrm{n}})[{\displaystyle \prod _{\mathrm{p}=1}^{\mathrm{n}}{G}_{\mathrm{p}}{(2}^{\mathrm{n}})}]{\mathrm{x}}_{\mathrm{s}}=T_{\mathrm{FOHKLT}}{(2}^{\mathrm{n}}){\mathrm{x}}_{\mathrm{s}}\mathrm{for}\mathrm{s}=1,2,\dots ,{\mathrm{N}}^2,$$

(16)

where $T_{\mathrm{FOHKLT}}{(2}^{\mathrm{n}})=P_{\mathrm{n}}{(2}^{\mathrm{n}})[{\displaystyle \prod _{\mathrm{p}=1}^{\mathrm{n}}{G}_{\mathrm{p}}{(2}^{\mathrm{n}})}]$ is the matrix for 1D-FOHKLT, $P_{\mathrm{n}}(2^{\mathrm{n}})$ is the permutation matrix $P_{\mathrm{n}}(2^{\mathrm{n}})$ for the last level n, and ${\displaystyle \prod _{\mathrm{p}=1}^{\mathrm{n}}{G}_{\mathrm{p}}{(2}^{\mathrm{n}})}$ is the product of n sparse transform matrices $G_{\mathrm{p}}(2^{\mathrm{n}})$ for p = 1, 2, 3, …, n.

Each matrix $G_{\mathrm{p}}(2^{\mathrm{n}})$ is defined as follows:

$$G_{\mathrm{p}}(2^{\mathrm{n}})=\underset{\mathrm{i}=1}{\overset{2^{\mathrm{n}-\mathrm{p}}}{\oplus }}\left[\begin{array}{cc}\underset{\mathrm{j}=1}{\overset{2^{\mathrm{p}-1}}{\oplus }}{\mathrm{c}}_{{\mathrm{p},2}^{\mathrm{p}-1}(\mathrm{i}-1)+\mathrm{j}}& \underset{\mathrm{j}=1}{\overset{2^{\mathrm{p}-1}}{\oplus }}{\mathrm{s}}_{{\mathrm{p},2}^{\mathrm{p}-1}(\mathrm{i}-1)+\mathrm{j}}\\ \underset{\mathrm{j}=1}{\overset{2^{\mathrm{p}-1}}{\oplus }}{\mathrm{s}}_{{\mathrm{p},2}^{\mathrm{p}-1}(\mathrm{i}-1)+\mathrm{j}}& \underset{\mathrm{j}=1}{\overset{2^{\mathrm{p}-1}}{\oplus }}-{\mathrm{c}}_{{\mathrm{p},2}^{\mathrm{p}-1}(\mathrm{i}-1)+\mathrm{j}}\end{array}\right]\mathrm{for}\mathrm{p}=1,2,3,\dots ,\mathrm{n},$$

(17)

where

$$\begin{array}{l}\underset{\mathrm{i}=1}{\overset{2^{\mathrm{n}-\mathrm{p}}}{\oplus }}\left[\begin{array}{cc}\underset{\mathrm{j}=1}{\overset{2^{\mathrm{p}-1}}{\oplus }}{\mathrm{c}}_{{\mathrm{p},2}^{\mathrm{p}-1}(\mathrm{i}-1)+\mathrm{j}}& \underset{\mathrm{j}=1}{\overset{2^{\mathrm{p}-1}}{\oplus }}{\mathrm{s}}_{{\mathrm{p},2}^{\mathrm{p}-1}(\mathrm{i}-1)+\mathrm{j}}\\ \underset{\mathrm{j}=1}{\overset{2^{\mathrm{p}-1}}{\oplus }}{\mathrm{s}}_{{\mathrm{p},2}^{\mathrm{p}-1}(\mathrm{i}-1)+\mathrm{j}}& \underset{\mathrm{j}=1}{\overset{2^{\mathrm{p}-1}}{\oplus }}-{\mathrm{c}}_{{\mathrm{p},2}^{\mathrm{p}-1}(\mathrm{i}-1)+\mathrm{j}}\end{array}\right]=\left[\begin{array}{cc}\underset{\mathrm{j}=1}{\overset{2^{\mathrm{p}-1}}{\oplus }}{\mathrm{c}}_{\mathrm{p},\mathrm{j}}& \underset{\mathrm{j}=1}{\overset{2^{\mathrm{p}-1}}{\oplus }}{\mathrm{s}}_{\mathrm{p},\mathrm{j}}\\ \underset{\mathrm{j}=1}{\overset{2^{\mathrm{p}-1}}{\oplus }}{\mathrm{s}}_{\mathrm{p},\mathrm{j}}& \underset{\mathrm{j}=1}{\overset{2^{\mathrm{p}-1}}{\oplus }}-{\mathrm{c}}_{\mathrm{p},\mathrm{j}}\end{array}\right]\oplus \\ \oplus \left[\begin{array}{cc}\underset{\mathrm{j}=1}{\overset{2^{\mathrm{p}-1}}{\oplus }}{\mathrm{c}}_{{\mathrm{p},2}^{\mathrm{p}-1}+\mathrm{j}}& \underset{\mathrm{j}=1}{\overset{2^{\mathrm{p}-1}}{\oplus }}{\mathrm{s}}_{{\mathrm{p},2}^{\mathrm{p}-1}+\mathrm{j}}\\ \underset{\mathrm{j}=1}{\overset{2^{\mathrm{p}-1}}{\oplus }}{\mathrm{s}}_{{\mathrm{p},2}^{\mathrm{p}-1}+\mathrm{j}}& \underset{\mathrm{j}=1}{\overset{2^{\mathrm{p}-1}}{\oplus }}-{\mathrm{c}}_{{\mathrm{p},2}^{\mathrm{p}-1}+\mathrm{j}}\end{array}\right]\oplus \dots \oplus \left[\begin{array}{cc}\underset{\mathrm{j}=1}{\overset{2^{\mathrm{p}-1}}{\oplus }}{\mathrm{c}}_{{\mathrm{p},2}^{\mathrm{p}-1}(\mathrm{n}-\mathrm{p}-1)+\mathrm{j}}& \underset{\mathrm{j}=1}{\overset{2^{\mathrm{p}-1}}{\oplus }}{\mathrm{s}}_{{\mathrm{p},2}^{\mathrm{p}-1}(\mathrm{n}-\mathrm{p}-1)+\mathrm{j}}\\ \underset{\mathrm{j}=1}{\overset{2^{\mathrm{p}-1}}{\oplus }}{\mathrm{s}}_{{\mathrm{p},2}^{\mathrm{p}-1}(\mathrm{n}-\mathrm{p}-1)+\mathrm{j}}& \underset{\mathrm{j}=1}{\overset{2^{\mathrm{p}-1}}{\oplus }}-{\mathrm{c}}_{{\mathrm{p},2}^{\mathrm{p}-1}(\mathrm{n}-\mathrm{p}-1)+\mathrm{j}}\end{array}\right]\end{array}.$$

(18)

In the level n are rearranged the components of the column-vectors $y_{\mathrm{n},\mathrm{s}}=G_{\mathrm{n}}(2^{\mathrm{n}})y_{\mathrm{n}-1,\mathrm{s}}$, and, respectively, the components of matrices Y_n,k for k = 0, 1, …, N − 1. For this is used the permutation matrix $P_{\mathrm{n}}(2^{\mathrm{n}})$. From the components of the column-vectors $e_{\mathrm{s}}=P_{\mathrm{n}}(2^{\mathrm{n}})y_{\mathrm{n},\mathrm{s}}$ are obtained the matrices E₀, E₁, …, E_N−1. The frequency-ordered matrices E_r are calculated in accordance with the relation between their sequential number r and the corresponding sequential number k, for the matrices Y_n,k. The relation which defines the matrix $P_{\mathrm{n}}(2^{\mathrm{n}})$ is as follows [14]:

• the binary code ${\mathrm{k}}_{\mathrm{n}-1}{,\mathrm{k}}_{\mathrm{n}-2},\dots {,\mathrm{k}}_0$ of the sequential decimal number k = 0, 1, …, 2ⁿ⁻¹ of the component Y_n,k is arranged inversely (i.e., ${\mathrm{k}}_0{,\mathrm{k}}_1,\dots {,\mathrm{k}}_{\mathrm{n}-1}$), as follows:

  $${\mathrm{g}}_{\mathrm{i}}={\mathrm{k}}_{\mathrm{n}-\mathrm{i}-1}\mathrm{for}0\le \mathrm{i}\le \mathrm{n}-1;$$

  (19)
• the so-obtained code ${\mathrm{g}}_{\mathrm{n}-1}{,\mathrm{g}}_{\mathrm{n}-2},\dots {,\mathrm{g}}_0$ is transformed from Gray code into binary code ${\mathrm{r}}_{\mathrm{n}-1}{,\mathrm{r}}_{\mathrm{n}-2},\dots {,\mathrm{r}}_0$, in accordance with the operations

  $${\mathrm{r}}_{\mathrm{n}-1}={\mathrm{g}}_{\mathrm{n}-1},{\mathrm{r}}_{\mathrm{i}}={\mathrm{r}}_{\mathrm{i}-1}\oplus {\mathrm{g}}_{\mathrm{i}}\mathrm{for}0\le \mathrm{i}\le \mathrm{n}-2.$$

  (20)

Here, ⊕ is the symbol for the operation “exclusive OR”. The decimal number $\mathrm{r}={\displaystyle \sum _{\mathrm{i}=0}^{\mathrm{n}-1}{\mathrm{r}}_{\mathrm{i}}}{2}^{\mathrm{i}}$ defines the sequential number of the component E_r for r = 0, 1, …,2ⁿ⁻¹, which before the rearrangement corresponded to the component Y_n,_k, with sequential number $\mathrm{k}={\displaystyle \sum _{\mathrm{i}=0}^{\mathrm{n}-1}{\mathrm{k}}_{\mathrm{i}}}{2}^{\mathrm{i}}$.

An example for the direct 1D-FOHKLT of three hierarchical levels is given in Appendix A.

3. Decomposition of Cubical Tensor of Size N × N × N through 3D-FOHKLT

The decomposition 3D-FOHKLT for a cubical tensor $\mathcal{X}$ of size 8 × 8 × 8, executed in three consecutive stages, is shown on Figure 2.

In the first stage, the tensor $\mathcal{X}$ is divided into eight horizontal slices (mode-1). Then, in accordance with Figure 1, on each couple of matrices is applied 1D-FOHKLT. In result, the sequence of eight transformed matrices is united into one new tensor, $\mathcal{E}$. In the second stage, the input tensor $\mathcal{E}$ is divided into eight lateral slices (mode-2), then the couples of matrices are transformed, and after that united into the tensor $\mathcal{F}$. In the third stage, the input tensor $\mathcal{F}$ is divided into eight frontal slices (mode-3). After applying the 1D-AKLT on each couple of matrices which build the tensor $\mathcal{F}$, the final result is obtained—the general spectral tensor $\mathcal{S}$, with elements $\mathrm{s}(\mathrm{u},\mathrm{v},\mathrm{l})$. The initial tensor $\mathcal{X}$ is restored from the tensor $\mathcal{S}$ through summation of all “weighted” basic tensors ${\mathcal{K}}_{\mathrm{u},\mathrm{v},\mathrm{l}}$ of size 8 × 8 × 8, in accordance with the relation

$$\mathcal{X}=(1/512)[\mathrm{s}\left(0,0,0\right){\mathcal{K}}_{0,0,0}+\mathrm{s}\left(1,0,0\right){\mathcal{K}}_{1,0,0}+\dots +\mathrm{s}\left(7,7,7\right){\mathcal{K}}_{7,7,7}]=(1/8^3){\displaystyle \sum _{\mathrm{u}=0}^7{\displaystyle \sum _{\mathrm{v}=0}^7{\displaystyle \sum _{\mathrm{l}=0}^7\mathrm{s}(\mathrm{u},\mathrm{v},\mathrm{l}){\mathcal{K}}_{\mathrm{u},\mathrm{v},\mathrm{l}}}}}.$$

(21)

Here, $\mathrm{s}(\mathrm{u},\mathrm{v},\mathrm{l})$ are the weight coefficients of the basic tensor, ${\mathcal{K}}_{\mathrm{u},\mathrm{v},\mathrm{l}}$. Each basic tensor is represented as the outer product of three vectors:

$${\mathcal{K}}_{\mathrm{u},\mathrm{v},\mathrm{l}}=k_{\mathrm{u}}(1)\circ k_{\mathrm{v}}(2)\circ k_{\mathrm{l}}(3),$$

(22)

where o is the symbol for the outer product of the two column-vectors ($x\circ y=x.y^{\mathrm{T}}$) and $k_{\mathrm{u}}(1),k_{\mathrm{v}}(2),k_{\mathrm{l}}(3)$ are the basic vectors for the first, second and third stages of 3D-FOHKLT, respectively. These vectors are defined by using the matrix $T_{\mathrm{F}\mathrm{O}\mathrm{H}\mathrm{K}\mathrm{L}\mathrm{T}}(8)$ for 1D-FOHKLT, in accordance with Equation (A6) from Appendix A. The vector $k_{\mathrm{u}}(1)$ for u = 0, 1, …, 7 comprises the elements of the u-th row of the matrix $T_{\mathrm{FOHKLT}}^1\left(8\right)$ for the first stage (t = 1) of 3D-FOHKLT. The vectors $k_{\mathrm{v}}(2)$ and $k_{\mathrm{l}}(3)$ for v, l = 0, 1, …, 7 comprise the elements of rows v and l in the matrices $T_{\mathrm{FOHKLT}}^2\left(8\right)$ and $T_{\mathrm{FOHKLT}}^3\left(8\right)$ for the second (t = 2) and third (t = 3) stages of 3D-FOHKLT, respectively. The basic vectors are defined below for t = 1, 2, 3 and u, v, l = 0, 1, …, 7:

$$k_{\mathrm{u}}(\mathrm{t})={{[\mathrm{k}}_{\mathrm{u},1}^{\mathrm{t}}{,\mathrm{k}}_{\mathrm{u},2}^{\mathrm{t}}{,\mathrm{k}}_{\mathrm{u},3}^{\mathrm{t}}{,\mathrm{k}}_{\mathrm{u},4}^{\mathrm{t}}{,\mathrm{k}}_{\mathrm{u},5}^{\mathrm{t}}{,\mathrm{k}}_{\mathrm{u},6}^{\mathrm{t}}{,\mathrm{k}}_{\mathrm{u},7}^{\mathrm{t}}{,\mathrm{k}}_{\mathrm{u},8}^{\mathrm{t}}]}^{\mathrm{T}};$$

(23)

$$k_{\mathrm{v}}(\mathrm{t})={[{\mathrm{k}}_{\mathrm{v},1}^{\mathrm{t}}{,\mathrm{k}}_{\mathrm{v},2}^{\mathrm{t}}{,\mathrm{k}}_{\mathrm{v},3}^{\mathrm{t}}{,\mathrm{k}}_{\mathrm{v},4}^{\mathrm{t}}{,\mathrm{k}}_{\mathrm{v},5}^{\mathrm{t}}{,\mathrm{k}}_{\mathrm{v},6}^{\mathrm{t}}{,\mathrm{k}}_{\mathrm{v},7}^{\mathrm{t}}{,\mathrm{k}}_{\mathrm{v},8}^{\mathrm{t}}]}^{\mathrm{T}};$$

(24)

$$k_{\mathrm{l}}(\mathrm{t})={[{\mathrm{k}}_{\mathrm{l},1}^{\mathrm{t}}{,\mathrm{k}}_{\mathrm{l},2}^{\mathrm{t}}{,\mathrm{k}}_{\mathrm{l},3}^{\mathrm{t}}{,\mathrm{k}}_{\mathrm{l},4}^{\mathrm{t}}{,\mathrm{k}}_{\mathrm{l},5}^{\mathrm{t}}{,\mathrm{k}}_{\mathrm{l},6}^{\mathrm{t}}{,\mathrm{k}}_{\mathrm{l},7}^{\mathrm{t}}{,\mathrm{k}}_{\mathrm{l},8}^{\mathrm{t}}]}^{\mathrm{T}}.$$

(25)

The general number of the basic tensors in this case is 8³. For example, if u = 3, v = 2 and l = 6, the components of these vectors for each stage are defined by the relations below:

$$\begin{array}{l}{k}_3(1)={[{\mathrm{k}}_{3,0}^1{,\mathrm{k}}_{3,1}^1{,\mathrm{k}}_{3,2}^1{,\mathrm{k}}_{3,3}^1{,\mathrm{k}}_{3,4}^1{,\mathrm{k}}_{3,5}^1{,\mathrm{k}}_{3,6}^1{,\mathrm{k}}_{3,7}^1]}^{\mathrm{T}}=\\ ={[{\mathrm{c}}_{3,3}^1{\mathrm{s}}_{2,1}^1{\mathrm{c}}_{1,1}^1,{\mathrm{c}}_{3,3}^1{\mathrm{s}}_{2,1}^1{\mathrm{s}}_{1,1}^1,-{\mathrm{c}}_{3,3}^1{\mathrm{c}}_{2,1}^1{\mathrm{c}}_{1,2}^1,-{\mathrm{c}}_{3,3}^1{\mathrm{c}}_{2,1}^1{\mathrm{s}}_{1,2}^1,{\mathrm{s}}_{3,3}^1{\mathrm{s}}_{2,3}^1{\mathrm{c}}_{1,3}^1,{\mathrm{s}}_{3,3}^1{\mathrm{s}}_{2,3}^1{\mathrm{s}}_{1,3}^1,-{\mathrm{s}}_{3,3}^1{\mathrm{c}}_{2,3}^1{\mathrm{c}}_{1,4}^1,-{\mathrm{s}}_{3,3}^1{\mathrm{c}}_{2,3}^1{\mathrm{s}}_{1,4}^1]}^{\mathrm{T}};\end{array}$$

(26)

$$\begin{array}{l}{k}_2(2)={[{\mathrm{k}}_{2,1}^2{,\mathrm{k}}_{2,2}^2{,\mathrm{k}}_{2,3}^2{,\mathrm{k}}_{2,4}^2{,\mathrm{k}}_{2,5}^2{,\mathrm{k}}_{2,6}^2{,\mathrm{k}}_{2,7}^2{,\mathrm{k}}_{2,8}^2]}^{\mathrm{T}}=\\ ={[{\mathrm{s}}_{3,3}^2{\mathrm{s}}_{2,1}^2{\mathrm{c}}_{1,1}^2{,\mathrm{s}}_{3,3}^2{\mathrm{s}}_{2,1}^2{\mathrm{s}}_{1,1}^2,-{\mathrm{s}}_{3,3}^2{\mathrm{c}}_{2,1}^2{\mathrm{c}}_{1,2}^2,-{\mathrm{s}}_{3,3}^2{\mathrm{c}}_{2,1}^2{\mathrm{s}}_{1,2}^2,-{\mathrm{c}}_{3,3}^2{\mathrm{s}}_{2,3}^2{\mathrm{c}}_{1,3}^2,-{\mathrm{c}}_{3,3}^2{\mathrm{s}}_{2,3}^2{\mathrm{s}}_{1,3}^2{,\mathrm{c}}_{3,3}^2{\mathrm{c}}_{2,3}^2{\mathrm{c}}_{1,4}^2{,\mathrm{c}}_{3,3}^2{\mathrm{c}}_{2,3}^2{\mathrm{s}}_{1,4}^2]}^{\mathrm{T}};\end{array}$$

(27)

$$\begin{array}{l}{k}_6(3)={[{\mathrm{k}}_{6,1}^3{,\mathrm{k}}_{6,2}^3{,\mathrm{k}}_{6,3}^3{,\mathrm{k}}_{6,4}^3{,\mathrm{k}}_{6,5}^3{,\mathrm{k}}_{6,6}^3{,\mathrm{k}}_{6,7}^3{,\mathrm{k}}_{6,8}^3]}^{\mathrm{T}}=\\ ={[{\mathrm{s}}_{3,2}^3{\mathrm{c}}_{2,2}^3{\mathrm{s}}_{1,1}^3{,-\mathrm{s}}_{3,2}^3{\mathrm{c}}_{2,2}^3{\mathrm{c}}_{1,1}^3{,\mathrm{s}}_{3,2}^3{\mathrm{s}}_{2,2}^3{\mathrm{s}}_{1,2}^3,-{\mathrm{s}}_{3,2}^3{\mathrm{s}}_{2,2}^3{\mathrm{c}}_{1,2}^3,-{\mathrm{c}}_{3,2}^3{\mathrm{c}}_{2,4}^3{\mathrm{s}}_{1,3}^3{,\mathrm{c}}_{3,2}^3{\mathrm{c}}_{2,4}^3{\mathrm{c}}_{1,3}^3{,-\mathrm{c}}_{3,2}^3{\mathrm{s}}_{2,4}^3{\mathrm{s}}_{1,4}^3{,\mathrm{c}}_{3,2}^3{\mathrm{s}}_{2,4}^3{\mathrm{c}}_{1,4}^3]}^{\mathrm{T}}.\end{array}$$

(28)

The vectors $k_{\mathrm{u}}\left(\mathrm{t}\right),k_{\mathrm{v}}\left(\mathrm{t}\right),k_{\mathrm{l}}\left(\mathrm{t}\right)$ are orthonormalized, i.e., for their components are valid the relations:

$${\displaystyle \sum _{\mathrm{m}=1}^8{\mathrm{k}}_{{\mathrm{u}}_1,\mathrm{m}}^{{\mathrm{t}}_1}{.\mathrm{k}}_{{\mathrm{u}}_2,\mathrm{m}}^{{\mathrm{t}}_2}=\left\{\begin{array}{c}{0\mathrm{for}\mathrm{t}}_1\ne {\mathrm{t}}_2;\\ {1\mathrm{for}\mathrm{t}}_1={\mathrm{t}}_2,\end{array}\right.}\mathrm{where}{\mathrm{u}}_1,{\mathrm{u}}_2=0,1,\dots ,7\mathrm{and}{\mathrm{t}}_1,{\mathrm{t}}_2=1,2,3.$$

(29)

The decomposition 3D-HAKLT, shown in Figure 2, could be easily generalized for the cases when N = 16, 32, …, 2ⁿ through uniting the computational principles from Figure 1 and Figure 2 for decompositions of high numbers of levels (n = 4, 5, …). From Equation (29), it follows that the rows of the matrices $T_{\mathrm{FOHKLT}}^{{\mathrm{t}}_1}\left(8\right)$ and $T_{\mathrm{FOHKLT}}^{{\mathrm{t}}_2}\left(8\right)$ for ${\mathrm{t}}_1\ne {\mathrm{t}}_2$ are not correlated. This conclusion could also be generalized for the rows of the matrices $T_{\mathrm{FOHKLT}}^{{\mathrm{t}}_1}{(2}^{\mathrm{n}})$ and $T_{\mathrm{FOHKLT}}^{{\mathrm{t}}_2}{(2}^{\mathrm{n}})$. Hence, after the execution of each stage of 3D-FOHKLT, the so-calculated N slices in the corresponding direction are highly decorrelated [12,15].

In the general case, the spectral coefficients s(u,v,l) are calculated through direct 3D-KLT, in accordance with the relation

$$\mathrm{s}\left(\mathrm{u},\mathrm{v},\mathrm{l}\right)={\displaystyle \sum _{\mathrm{i}=0}^{\mathrm{N}-1}{\displaystyle \sum _{\mathrm{j}=0}^{\mathrm{N}-1}{\displaystyle \sum _{\mathrm{k}=0}^{\mathrm{N}-1}\mathrm{x}\left(\mathrm{i},\mathrm{j},\mathrm{k}\right)\mathrm{k}\mathrm{a}\mathrm{r}(\mathrm{i},\mathrm{u},\mathrm{N})\mathrm{k}\mathrm{a}\mathrm{r}(\mathrm{j},\mathrm{v},\mathrm{N})\mathrm{k}\mathrm{a}\mathrm{r}(\mathrm{k},\mathrm{l},\mathrm{N})}}}\mathrm{for}\mathrm{u},\mathrm{v},\mathrm{l}=0,1,\dots ,\mathrm{N}-1.$$

(30)

The three-dimensional inverse KLT is defined by the relation

$$\mathrm{x}\left(\mathrm{i},\mathrm{j},\mathrm{k}\right)=\frac{1}{{\mathrm{N}}^3}{\displaystyle \sum _{\mathrm{u}=0}^{\mathrm{N}-1}{\displaystyle \sum _{\mathrm{v}=0}^{\mathrm{N}-1}{\displaystyle \sum _{\mathrm{l}=0}^{\mathrm{N}-1}\mathrm{s}\left(\mathrm{u},\mathrm{v},\mathrm{l}\right)\mathrm{k}\mathrm{a}\mathrm{r}(\mathrm{i},\mathrm{u},\mathrm{N})\mathrm{k}\mathrm{a}\mathrm{r}(\mathrm{j},\mathrm{v},\mathrm{N})\mathrm{k}\mathrm{a}\mathrm{r}(\mathrm{k},\mathrm{l},\mathrm{N})}}}\mathrm{for}\mathrm{i},\mathrm{j},\mathrm{k}=0,1,\dots ,\mathrm{N}-1.$$

(31)

Here the 1D-Karhunen–Loeve (KL) functions of the N^th order are kar(i,u,N), kar(j,v,N), and kar(k,l,N).

Each third-order tensor $\mathcal{X}$ of size N × N × N is represented as the weighted sum of N³ 3D-KL functions obtained by using the tensors ${\mathcal{K}}_{\mathrm{u},\mathrm{v},\mathrm{l}}$, each of size N × N × N:

$$\mathcal{X}=(1/{\mathrm{N}}^3)[\mathrm{s}\left(0,0,0\right){\mathcal{K}}_{0,0,0}+\mathrm{s}\left(1,0,0\right){\mathcal{K}}_{1,0,0}+\dots +\mathrm{s}\left(\mathrm{N}-1,\mathrm{N}-1,\mathrm{N}-1\right){\mathcal{K}}_{\mathrm{N}-1,\mathrm{N}-1,\mathrm{N}-1}]=(1/{\mathrm{N}}^3){\displaystyle \sum _{\mathrm{u}=0}^{\mathrm{N}-1}{\displaystyle \sum _{\mathrm{v}=0}^{\mathrm{N}-1}{\displaystyle \sum _{\mathrm{l}=0}^{\mathrm{N}-1}\mathrm{s}(\mathrm{u},\mathrm{v},\mathrm{l}){\mathcal{K}}_{\mathrm{u},\mathrm{v},\mathrm{l}}}}}.$$

(32)

Then, the tensor decomposition based on the 3D-FOHKLT is defined by the relation

$$\mathcal{X}=(1/{\mathrm{N}}^3){\displaystyle \sum _{\mathrm{u}=0}^{\mathrm{N}-1}{\displaystyle \sum _{\mathrm{v}=0}^{\mathrm{N}-1}{\displaystyle \sum _{\mathrm{l}=0}^{\mathrm{N}-1}\mathrm{s}(\mathrm{u},\mathrm{v},\mathrm{l})[{\mathrm{k}}_u(1)\circ {\mathrm{k}}_{\mathrm{v}}(2)\circ {\mathrm{k}}_{\mathrm{l}}(3)]}}}.$$

(33)

Hence, in this case the tensor $\mathcal{X}$ of size N × N × N is decomposed in a way similar to that of the Tucker decomposition [2];

$$\mathcal{X}={\displaystyle \sum _{{\mathrm{r}}_1=1}^{\mathrm{R}}{\displaystyle \sum _{{\mathrm{r}}_2=1}^{\mathrm{R}}{\displaystyle \sum _{{\mathrm{r}}_3=1}^{\mathrm{R}}{\mathrm{d}}_{{\mathrm{r}}_1,{\mathrm{r}}_2,{\mathrm{r}}_3}({\mathrm{a}}_{{\mathrm{r}}_1}\circ {\mathrm{b}}_{{\mathrm{r}}_2}\circ {\mathrm{c}}_{{\mathrm{r}}_3})}}}.$$

(34)

Here, ${\mathrm{d}}_{{\mathrm{r}}_1,{\mathrm{r}}_2,{\mathrm{r}}_3}$ are the eigen values of the tensor $\mathcal{X}$, ${\mathrm{a}}_{{\mathrm{r}}_1},{\mathrm{b}}_{{\mathrm{r}}_2},{\mathrm{c}}_{{\mathrm{r}}_3}$ are the corresponding eigen vectors, and R is the tensor rank. The rank of the three-way tensor $\mathcal{X}$ of size I × J × K is bounded from above in accordance with the inequality, as follows: $\max \{I,J,\mathrm{K}\}\le \mathrm{R}\le \min \{I\times J,I\times \mathrm{K},J\times \mathrm{K}\}$ [3]. If I = J = K = N, then $\mathrm{N}\le \mathrm{R}\le {\mathrm{N}}^2$. The difference between the decompositions from Equations (33) and (34) is the number of their components, which can reach up to N³ and N², respectively (for the maximum value of the rank, R). In the case that the decomposition components number as defined by Equation (33) is not reduced, the 3D-HAKLT is reversible, i.e., the coefficients s(u,v,l) of the tensor $\mathcal{S}$ and its basic tensors ${\mathcal{K}}_{\mathrm{u},\mathrm{v},\mathrm{l}}$ are sufficient to restore the tensor X.

4. Algorithm 3D-FOHKLT for Cubical Tensor Representation

The representation of the tensor $\mathcal{X}$ of size N × N × N, through 3D-FOHKLT based on the spectral tensor $\mathcal{S}$, is executed in correspondence with Figure 2. Each stage of the 3D transform is calculated for N = 8 through 1D-FOHKLT, as shown in Figure 1b.

Algorithm 1: Tensor representation based on the 3D-FOHKLT

Input: Third-order tensor $\mathcal{X}$ of size N × N × N (N = 2n) and with elements x(i,j,k) Output: Spectral tensor $\mathcal{S}$ of n layers, whose elements s(u,v,l) are the coefficients of the 3D-FOНKLT. 1 begin 2 Divide the tensor $\mathcal{X}$ into horisontal slices and compose N/2 couples of matrices $X_{\mathrm{p},\mathrm{q}}^1$ and $X_{\mathrm{p},\mathrm{q}}^2$ of size   N × N, for q = 1, 2, …, N/2; 3 for each couple of matrices $X_{\mathrm{p},\mathrm{q}}^1$ and $X_{\mathrm{p},\mathrm{q}}^2$ when q = 1, 2, …, N/2 and p = 1, 2, …, n, do 4   Calculate the couple of matrices $Y_{\mathrm{p},\mathrm{q}}^1$ and $Y_{\mathrm{p},\mathrm{q}}^2$ obtained through 1D-FOНKLT in      accordance with Equations (7) and (8); 5   Calculate the matrices $P_{\mathrm{n}}(2^{\mathrm{n}})$ and Yn,k for k = 0, 1, …, N−1 and p = n in accordance with Equations (18)–(20); 6   Define the matrices Er for r = 0, 1, …, N−1 in the level p = n by rearranging the matrices Yn,k for      k = 0, 1, …, N−1 on the basis of the vector transform $e_{\mathrm{s}}=P_{\mathrm{n}}(2^{\mathrm{n}})y_{\mathrm{n},\mathrm{s}}$, for s = 1, 2, …, N2; 7   Reshape all rearranged matrices Er into the corresponding tensor $\mathcal{E}$ of size N × N × N. 8  end 9  Divide the tensor $\mathcal{E}$ into lateral slices and compose N/2 couples of intermediate matrices $E_{\mathrm{p},\mathrm{q}}^1$ and $E_{\mathrm{p},\mathrm{q}}^2$     of size N × N, for q = 1, 2, …, N/2; 10  for each couple of matrices $E_{\mathrm{p},\mathrm{q}}^1$ and $E_{\mathrm{p},\mathrm{q}}^2$ when q = 1, 2, …, N/2 and p = 1, 2, …, n, do 11   Calculate the couple of matrices $F_{\mathrm{p},\mathrm{q}}^1$ and $F_{\mathrm{p},\mathrm{q}}^2$ transformed through 1D-FOНKLT in     accordance with Equations (7) and (8); 12   Calculate the matrices $P_{\mathrm{n}}(2^{\mathrm{n}})$ and Fn,k for k = 1, 2, …, N and p = n in accordance with Equations (18)–(20); 13   Define the matrices Fr for r = 0, 1, …, N−1 in the level p = n by rearranging the matrices Fn,k for     k = 0, 1, …, N−1 on the basis of the vector transform $F_{\mathrm{s}}=P_{\mathrm{n}}(2^{\mathrm{n}})f_{\mathrm{n},\mathrm{s}}$, for s = 1, 2, …, N2; 14   Reshape all rearranged matrices Fr in level p = n into the corresponding tensor $\mathcal{F}$ of size N × N × N 15 end 16 Divide the tensor $\mathcal{F}$ into frontal slices and compose N/2 couples of matrices $F_{\mathrm{p},\mathrm{q}}^1$ and $F_{\mathrm{p},\mathrm{q}}^2$ of size   N × N, for q = 1, 2, …, N/2; 17 for each couple of matrices $F_{\mathrm{p},\mathrm{q}}^1$ and $F_{\mathrm{p},\mathrm{q}}^2$ when q = 1, 2,…, N/2 and p = 1, 2, …, n, do 18    Calculate the couple of matrices $S_{\mathrm{p},\mathrm{q}}^1$ and $S_{\mathrm{p},\mathrm{q}}^2$ transformed through 1D-FOНKLT in      accordance with Equations (7) and (8); 19    Calculate the matrices $P_{\mathrm{n}}(2^{\mathrm{n}})$ and Sn,k for k = 1, 2, …, N and p = n in accordance with Equations (18)–(20); 20    Define the matrices Sr for r = 0, 1, …, N−1 in the level p = n by rearranging the matrices Sn,k for      k = 0, 1, …, N−1 on the basis of the vector transform $S_{\mathrm{s}}=P_{\mathrm{n}}(2^{\mathrm{n}}){\mathrm{s}}_{\mathrm{n},\mathrm{s}}$, for s = 1, 2, …, N2; 21    Reshape all rearranged matrices Sr in the level p = n into the spectral tensor $\mathcal{S}$ of size N × N × N 22  end 23  Arrange the coefficients s(u,v,l) of the spectral cubical tensor $\mathcal{S}$ layer by layer in accordance with their    spatial frequencies (u,v,l) - from the lowest (0,0,0), to the highest (N−1, N−1, N−1), for u + v + l = const. 24 end

In result is obtained the spectral tensor S, which comprises n layers of coefficients. These coefficients participate as component weights in the decomposition represented by Equation (33).

The tensor X is restored from the tensor S through inverse 3D-FOHKLT. This transform is based on the algorithm for direct 3D-FOHKLT, wherein is repeatedly performed the inverse 1D-AKLT (with basic operation “butterfly“), defined by the relations:

$$\begin{array}{c}{\mathrm{x}}_{1\mathrm{s}}\left(\mathrm{p},\mathrm{q}\right)=(1/2)[{\mathrm{y}}_{1\mathrm{s}}(\mathrm{p},\mathrm{q})\cos {\mathsf{\theta }}_{\mathrm{p},\mathrm{q}}+{\mathrm{y}}_{2\mathrm{s}}(\mathrm{p},\mathrm{q})\sin {\mathsf{\theta }}_{\mathrm{p},\mathrm{q}}];\\ {\mathrm{x}}_{2\mathrm{s}}\left(\mathrm{p},\mathrm{q}\right)=(1/2)[{\mathrm{y}}_{1\mathrm{s}}(\mathrm{p},\mathrm{q})\sin {\mathsf{\theta }}_{\mathrm{p},\mathrm{q}}-{\mathrm{y}}_{2\mathrm{s}}(\mathrm{p},\mathrm{q})\cos {\mathsf{\theta }}_{\mathrm{p},\mathrm{q}}]\end{array}\mathrm{for}\mathrm{s}=1,2,\dots ,{\mathrm{N}}^2.$$

(35)

Here, the angle θ_p,q is defined by Equation (2), on the basis of the covariance matrix elements $\mathrm{K}(\mathrm{p},\mathrm{q})=\left[\begin{array}{cc}{\mathrm{k}}_1(\mathrm{p},\mathrm{q})& {\mathrm{k}}_3(\mathrm{p},\mathrm{q})\\ {\mathrm{k}}_3(\mathrm{p},\mathrm{q})& {\mathrm{k}}_2(\mathrm{p},\mathrm{q})\end{array}\right]$ for the couple of matrices $X_{\mathrm{p},\mathrm{q}}^1$ and $X_{\mathrm{p},\mathrm{q}}^2$ (in levels p = 1, 2, 3 for q = 1, 2, 3, 4).

Unlike 1D-FOHKLT, the decorrelation of the cubical tensor elements through 3D-FOHKLT is executed in three mutually orthogonal directions, which is its main advantage compared to 1D-FOHKLT.

The algorithm for 3D-HAHKLT is similar to Algorithm 1, given above. The only difference is that the reordering of the matrices in each consecutive level is executed in accordance with Equation (9).

5. Comparative Evaluation of the Computational Complexity of 3D-HAKLT and 3D-FOHKLT

The computational complexity is evaluated on the basis of the needed operations “addition” and “multiplication”.

5.1. Computational Complexity of 3D-HAKLT for a Cubical Tensor of Size N × N × N (N = 2ⁿ)

• The number of additions A_H(n) and multiplications M_H(n) needed to execute the 1D-HAKLT for a couple of matrices of size N × N in one decomposition level is

  $${\mathrm{A}}_{\mathrm{H}}\left(\mathrm{n}\right)=3({\mathrm{N}}^2-1)+2{\mathrm{N}}^2+4=5\times 2^{2\mathrm{n}}+1;$$

  (36)

  $${\mathrm{M}}_{\mathrm{H}}\left(\mathrm{n}\right)=3{\mathrm{N}}^2+4{\mathrm{N}}^2+5=7\times 2^{2\mathrm{n}}+5.$$

  (37)
• For N/2 couples of matrices in one stage of the n-level 3D-HAKLT, these numbers are

  $${\mathrm{A}}_{1\mathrm{H}}^{}\left(\mathrm{n}\right)=[2^{\mathrm{n}-1}(5\times 2^{2\mathrm{n}}+1)+(2^{2\mathrm{n}}-1)(2^{\mathrm{n}}-1)]\mathrm{n};$$

  (38)

  $${\mathrm{M}}_{1\mathrm{H}}^{}\left(\mathrm{n}\right)=[2^{\mathrm{n}-1}(7\times 2^{2\mathrm{n}}+5)+2^{2\mathrm{n}}(2^{\mathrm{n}}-1)]\mathrm{n}.$$

  (39)
• Then, for the three-level 3D-HAKLT is obtained

  $${\mathrm{A}}_{3\mathrm{H}}\left(\mathrm{n}\right)=3\times [2^{\mathrm{n}-1}(5\times 2^{2\mathrm{n}}+1)+(2^{2\mathrm{n}}-1)(2^{\mathrm{n}}-1)]\mathrm{n};$$

  (40)

  $${\mathrm{M}}_{3\mathrm{H}}\left(\mathrm{n}\right)=3\times [2^{\mathrm{n}-1}(7\times 2^{2\mathrm{n}}+5)+2^{2\mathrm{n}}(2^{\mathrm{n}}-1)]\mathrm{n}.$$

  (41)

  Hence,

  $${\mathrm{A}}_{3\mathrm{H}}\left(\mathrm{n}\right)\approx 3\times [2.5\times 2^{3\mathrm{n}}+2^{3\mathrm{n}}]\mathrm{n}=10.5\times 2^{3\mathrm{n}}\mathrm{n};$$

  (42)

  $${\mathrm{M}}_{3\mathrm{H}}\left(\mathrm{n}\right)\approx 3\times [3.5\times 2^{3\mathrm{n}}+2^{3\mathrm{n}}]\mathrm{n}=13.5\times 2^{3\mathrm{n}}\mathrm{n}.$$

  (43)

The computational complexity of 3D-HAKLT, evaluated on the basis of the needed number of operations O_3H(n), is defined by the relation

$${\mathrm{O}}_{3\mathrm{H}}(\mathrm{n})={\mathrm{A}}_{3\mathrm{H}}(\mathrm{n})+{\mathrm{M}}_{3\mathrm{H}}(\mathrm{n})\approx 24\times 2^{3\mathrm{n}}\mathrm{n}.$$

(44)

The normalized value of O_3H(n) for one tensor element (voxel) is defined by the relation

$${\mathrm{O}}_{3\mathrm{H}}^0\left(\mathrm{n}\right)\approx {(1/2}^{3\mathrm{n}})24\times 2^{3\mathrm{n}}\mathrm{n}=24\mathrm{n}.$$

(45)

For 1D-FOHKLT ${\mathrm{A}}_{1\mathrm{H}}^{\mathrm{FO}}\left(\mathrm{n}\right)=2^{\mathrm{n}-1}(5\times 2^{2\mathrm{n}}+1)\mathrm{n}\approx 2.5\times 2^{3\mathrm{n}}\mathrm{n}$ and ${\mathrm{M}}_{1\mathrm{H}}^{\mathrm{FO}}\left(\mathrm{n}\right)=2^{\mathrm{n}-1}(7\times 2^{2\mathrm{n}}+5)\mathrm{n}\approx 3.5\times 2^{3\mathrm{n}}\mathrm{n}.$ Then

$${\mathrm{O}}_{3\mathrm{H}}^{\mathrm{FO}}\left(\mathrm{n}\right)={\mathrm{A}}_{3\mathrm{H}}^{\mathrm{FO}}\left(\mathrm{n}\right)+{\mathrm{M}}_{3\mathrm{H}}^{\mathrm{FO}}\left(\mathrm{n}\right)\approx 18\times 2^{3\mathrm{n}}\mathrm{n}.$$

(46)

The comparison of Equations (44) and (46) shows that the CC of 3D-FOHKLT is reduced approximately (4/3) times, compared to that of 3D-HAKLT.

5.2. Comparison of the Computational Complexity of 3D-HAKLT and 3D-FOHKLT to H-Tucker and TT

The CC of 3D-HAKLT for the transformation of the cubical tensor X of size 2ⁿ, evaluated through the needed number of operations, is ${\mathrm{O}}_{3\mathrm{H}}(24\times 2^{3\mathrm{n}}\mathrm{n}).$ In accordance with [5], the CC of the H-Tucker decomposition is ${\mathrm{O}}_{\mathrm{H}\mathrm{T}}((\mathrm{d}-1){\mathrm{R}}^3+\mathrm{d}\times 2^{\mathrm{n}}\times \mathrm{R})$, where R is the rank, 2ⁿ is the size, and d = 3 is the order of the tensor X. For R = 2ⁿ (the minimum value in the range $2^{\mathrm{n}}\le \mathrm{R}\le 2^{2\mathrm{n}}$), the number of the operations is ${\mathrm{O}}_{\mathrm{H}\mathrm{T}}(2^{4\mathrm{n}+1}+3\times 2^{3\mathrm{n}})={\mathrm{O}}_{\mathrm{H}\mathrm{T}}(2^{3\mathrm{n}}(2^{\mathrm{n}+1}+3)).$ In this case, the relative CCs of 3D-HAKLT and 3D-FOHKLT compared to that of the H-Tucker is defined by the relations:

$${\mathsf{\psi }}_1\left(\mathrm{n}\right)={\mathrm{O}}_{\mathrm{HT}}\left(\mathrm{n}\right)/{\mathrm{O}}_{3\mathrm{H}}\left(\mathrm{n}\right)=2^{3\mathrm{n}}{(2}^{\mathrm{n}+1}+3)/(24\times 2^{3\mathrm{n}}\mathrm{n})={(2}^{\mathrm{n}}+1.5)/12\mathrm{n};$$

(47)

$${\mathsf{\psi }}_2(\mathrm{n})={\mathrm{O}}_{\mathrm{H}\mathrm{T}}(\mathrm{n})/{\mathrm{O}}_{3\mathrm{H}}^{\mathrm{F}\mathrm{O}}(\mathrm{n})=2^{3\mathrm{n}}(2^{\mathrm{n}+1}+3)/(18\times 2^{3\mathrm{n}}\mathrm{n})=(2^{\mathrm{n}}+1.5)/9\mathrm{n}.$$

(48)

The CC of the TT decomposition [4] for the cubical tensor $\mathcal{X}$ of size 2ⁿ is ${\mathrm{O}}_{\mathrm{T}\mathrm{T}}(2^{\mathrm{n}}\mathrm{d}{\mathrm{R}}^3)$, i.e., ${\mathrm{O}}_{\mathrm{T}\mathrm{T}}(3\times 2^{4\mathrm{n}})$. Correspondingly, the relative CC of 3D-HAKLT and 3D-FOHKLT towards the TT decomposition is defined by the relations:

$${\mathsf{\psi }}_3(\mathrm{n})={\mathrm{O}}_{\mathrm{T}\mathrm{T}}(\mathrm{n})/{\mathrm{O}}_{3\mathrm{H}}(\mathrm{n})=(3\times 2^{4\mathrm{n}})/(24\times 2^{3\mathrm{n}}\mathrm{n})=2^{\mathrm{n}}/8\mathrm{n};$$

(49)

$${\mathsf{\psi }}_4(\mathrm{n})={\mathrm{O}}_{\mathrm{T}\mathrm{T}}(\mathrm{n})/{\mathrm{O}}_{3\mathrm{H}}^{\mathrm{F}\mathrm{O}}(\mathrm{n})=(3\times 2^{4\mathrm{n}})/(18\times 2^{3\mathrm{n}}\mathrm{n})=2^{\mathrm{n}}/6\mathrm{n}.$$

(50)

The values of functions ${\mathsf{\psi }}_{\mathrm{i}}(\mathrm{n})$ for i = 1, 2, 3, 4 and n = 2, 3, …, 8 are given in

Table 1. Relative CC of 3D-HAKLT and 3D-FOHKLT compared to H-Tucker and TT.

n	2	3	4	5	6	7	8
${\mathsf{\psi }}_1(\mathrm{n})$	0.23	0.26	0.36	0.57	0.91	1.54	2.68
${\mathsf{\psi }}_2(\mathrm{n})$	0.30	0.35	0.48	0.74	1.21	2.05	3.57
${\mathsf{\psi }}_3(\mathrm{n})$	0.25	0.33	0.50	0.80	1.33	2.28	4.00
${\mathsf{\psi }}_4(\mathrm{n})$	0.33	0.44	0.66	1.06	1.77	3.04	5.33

, and their graphics are shown in Figure 3a,b. From Equations (47)–(50), it follows that the functions ${\mathsf{\psi }}_{\mathrm{i}}(\mathrm{n})$ for i = 1, 2, 3, 4 grow proportionally to 2ⁿ/n.

The obtained results show that for small values of n (in the range from 2 to 5), the decompositions 3D-HAKLT and 3D-FOHKLT have low efficiencies because their CCs are equal or higher than those of H-Tucker and TT.

However, for practical applications (image processing), these values are quite different (for example, n = 8 corresponds to tensor image of size 256 × 256 × 256, which is very small), and in this case the relative CC of the offered decompositions is much lower than those of H-Tucker and TT. An additional advantage is that unlike H-Tucker and TT, the proposed algorithms do not need iterative calculations.

6. Conclusions

In this work are presented new cubical tensor decompositions based on the 3D orthogonal hierarchical transforms HAKLT and FOHKLT. The comparative analysis for the relative CC of the new decompositions compared to those of H-Tucker and TT shows their advantages for tensors of size 256 and more. From these results it follows that 3D-FOHKLT has minimum CC, and is suitable for real-time applications. One more advantage of the offered algorithms is that they do not need iterative calculations, and instead use hierarchical calculations in the three mutually orthogonal directions of the tensor. Unlike the well-known 3D decompositions based on the N-dimensional eigen vectors of the tensor, the kernel of the offered hierarchical transforms is the AKLT of size 2 × 2, used repeatedly in each level. The presented algorithms 3D-HAKLT and 3D-FOHKLT could also be generalized for tensors with three different dimensions $2^{{\mathrm{n}}_1}\times 2^{{\mathrm{n}}_2}\times 2^{{\mathrm{n}}_3}$ for ${\mathrm{n}}_1\ne {\mathrm{n}}_2\ne {\mathrm{n}}_3$. The choice of the offered hierarchical 3D decompositions is made depending on the requirements and limitations of their CC imposed by the corresponding application area.

The future investigations of the 3D-HAKLT and 3D-FOHKLT will be aimed at the evaluation of their characteristics compared to the famous tensor decompositions, in order to define the best settings and to outline the most efficient applications in the 3D compression, filtration, analysis, search and recognition of multidimensional visual information, deep learning, etc.