Efficient Algorithms for Linear System Identification with Particular Symmetric Filters

Abstract

In linear system identification problems, it is important to reveal and exploit any specific intrinsic characteristic of the impulse responses, in order to improve the overall performance, especially in terms of the accuracy and complexity of the solution. In this paper, we focus on the nearest Kronecker product decomposition of the impulse responses, together with low-rank approximations. Such an approach is suitable for the identification of a wide range of real-world systems. Most importantly, we reformulate the system identification problem by using a particular symmetric filter within the development, which allows us to efficiently design two (iterative/recursive) algorithms. First, an iterative Wiener filter is proposed, with improved performance as compared to the conventional Wiener filter, especially in challenging conditions (e.g., small amount of available data and/or noisy environments). Second, an even more practical solution is developed, in the form of a recursive least-squares adaptive algorithm, which could represent an appealing choice in real-time applications. Overall, based on the proposed approach, a system identification problem that can be conventionally solved by using a system of $L=L_1{L}_2$ equations (with L unknown parameters) is reformulated as a combination of two systems of $P{L}_1$ and $P{L}_2$ equations, respectively, where usually $P\ll L_2$ (i.e., a total of $P{L}_1+P{L}_2$ parameters). This could lead to important advantages, in terms of both performance and complexity. Simulation results are provided in the framework of network and acoustic echo cancellation, supporting the performance gain and the practical features of the proposed algorithms.

1. Introduction

Linear system identification problems represent key issues in many important applications [1]. In this context, one of the main challenges is related to the long length of the impulse response to be identified, which is usually the case in applications related to the acoustic environment [2], like echo cancellation, noise reduction, microphone arrays, etc. For example, in echo cancellation [3], such a network/acoustic impulse response can have hundreds/thousands of coefficients (even for a sampling rate of 8 kHz), which significantly increases the difficulty of the system identification problem, especially in terms of the complexity and accuracy of the solution.

An efficient approach that can be exploited in the framework of such high-dimension system identification problems is based on the Kronecker product decomposition of the impulse response, e.g., see [4,5], and the references therein. Such an approach fits very well for the identification of linearly separable (or multilinear) systems, because it involves tensor modeling techniques. In other words, a high-dimension system identification problem is solved based on a combination of low-dimension solutions (i.e., shorter impulse responses), which are tensorized together. This approach can find applications in many important fields, like wireless communication systems [6,7], nonlinear acoustic echo cancellation [8,9], source separation [10,11], array beamforming [12,13], and object recognition [14,15]. Furthermore, by using the decomposition based on the nearest Kronecker product, followed by low-rank approximations, more general forms of impulse responses can be identified, which are not linearly separable [16,17,18,19]. Recent studies on this topic have targeted specific applications, like echo cancellation, noise reduction, adaptive beamforming, microphone arrays, linear prediction, and speech dereverberation, e.g., see [20,21,22,23,24,25,26,27,28] and references therein.

Recently, the identification of linearly separable systems that own particular intrinsic symmetric/antisymmetric properties has been addressed in [29,30]. As compared to the previous approaches, these very recent works exploit a particular “hidden” symmetry feature of the system to be identified, thus leading to improved solutions as compared to their conventional counterparts. In [29], the solution was formulated in terms of an iterative Wiener filter, whereas [30] developed a family of least-mean-square (LMS) adaptive algorithms tailored for such systems.

In this paper, we follow the idea of using such a particular symmetric filter for linear system identification problems. As compared to the works reported in [29,30], we do not limit our solution only to the identification of linearly separable systems, but for more general forms of impulse responses. The resulting contribution is basically twofold. First, an iterative Wiener filter is developed, with improved performance features as compared to the conventional Wiener approach, especially when a small amount of data is available for the estimation of the required statistics. Furthermore, we propose a recursive least-squares (RLS) adaptive algorithm based on a similar framework, which represents a more practical solution (as compared to the Wiener filter) for real-world/real-time applications.

Following this introduction, the system identification problem using the specific model that involves the symmetric filter is presented in Section 2. Next, the solution based on the Wiener filter is developed in Section 3. In Section 4, we propose the RLS-based algorithm following a similar approach. Simulation results are provided in Section 5, in the framework of network and acoustic echo cancellation, thus supporting the theoretical findings and the performance gain of the proposed solutions. Finally, a summarized discussion is included in Section 6, while Section 7 concludes this work, outlining the main ideas and results, together with several perspectives for future works.

2. System Model and Problem Formulation

Let us consider a linear single-input/single-output (SISO) system, whose output at the discrete-time index t is given by

$$\begin{array}{cc}\hfill d\left(t\right)& ={\mathbf{h}}^T\mathbf{x}\left(t\right)+w\left(t\right)\hfill \\ \hfill & =y\left(t\right)+w\left(t\right),\hfill \end{array}$$

(1)

where $d\left(t\right)$ is also known as the desired signal,

$$\begin{array}{cc}\hfill \mathbf{x}\left(t\right)& ={\left[\begin{array}{cccc}x\left(t\right)& x(t-1)& \cdots & x(t-L+1)\end{array}\right]}^T\hfill \end{array}$$

is a vector containing the most recent L time samples of the zero-mean input signal $x\left(t\right)$, the superscript ${}^T$ denotes the transpose operator, $\mathbf{h}$ is the unknown system impulse response (a column vector of length L), and $w\left(t\right)$ is the zero-mean additive noise. It is assumed that we deal with real-valued signals, and that $x\left(t\right)$ and $w\left(t\right)$ are uncorrelated. Then, the objective of the so-called SISO system identification problem is to identify or estimate $\mathbf{h}$, in the best possible way, given the input signal vector, $\mathbf{x}\left(t\right)$, and the output signal, $d\left(t\right)$. From (1), we see that the signal-to-noise ratio (SNR) of the SISO system is

$$\begin{array}{cc}\hfill \mathrm{SNR}& =\frac{{\mathbf{h}}^T{\mathbf{R}}_{\mathbf{x}}\mathbf{h}}{{\sigma }_w^2},\hfill \end{array}$$

(2)

where ${\mathbf{R}}_{\mathbf{x}}=E\left[\mathbf{x}\left(t\right){\mathbf{x}}^T\left(t\right)\right]$ and ${\sigma }_w^2=E\left[w^2\left(t\right)\right]$ are the covariance matrix of $\mathbf{x}\left(t\right)$ and variance of $w\left(t\right)$, respectively, with $E[·]$ denoting mathematical expectation.

Let us assume that $L=L_1{L}_2$. Without loss of generality, we may also assume that $L_1\ge L_2$. Then, the system impulse response can be decomposed as [16]

$$\begin{array}{cc}\hfill \mathbf{h}& =\sum _{l=1}^{L_2}{\mathbf{h}}_{2,l}\otimes {\mathbf{h}}_{1,l},\hfill \end{array}$$

(3)

where ⊗ is the Kronecker product [31], and ${\mathbf{h}}_{1,l}$ and ${\mathbf{h}}_{2,l}$ are short impulse responses (column vectors) of lengths $L_1$ and $L_2$, respectively. Another way to write (3) is

$$\begin{array}{cc}\hfill \mathbf{h}& =\mathrm{vec}\left(\sum _{l=1}^{L_2}{\mathbf{h}}_{1,l}{\mathbf{h}}_{2,l}^T\right)\hfill \\ \hfill & =\mathrm{vec}\left({\mathbf{H}}_1{\mathbf{H}}_2^T\right)=\mathrm{vec}\left(\mathbf{H}\right),\hfill \end{array}$$

(4)

where $\mathrm{vec}(·)$ is the vectorization operation, which consists of converting a matrix into a vector,

$$\begin{array}{cc}\hfill {\mathbf{H}}_1& =\left[\begin{array}{cccc}{\mathbf{h}}_{1,1}& {\mathbf{h}}_{1,2}& \cdots & {\mathbf{h}}_{1,L_2}\end{array}\right],\hfill \\ \hfill {\mathbf{H}}_2& =\left[\begin{array}{cccc}{\mathbf{h}}_{2,1}& {\mathbf{h}}_{2,2}& \cdots & {\mathbf{h}}_{2,L_2}\end{array}\right]\hfill \end{array}$$

are matrices of sizes $L_1\times L_2$ and $L_2\times L_2$, respectively, and $\mathbf{H}$ is the equivalent matrix (of size $L_1\times L_2$) representation of $\mathbf{h}$. However, very often in practice, the column rank of $\mathbf{H}$ is far from its maximum value, i.e., $\mathrm{rank}\left(\mathbf{H}\right)=P\ll L_2$. As a result, (3) can be expressed as

$$\begin{array}{cc}\hfill \mathbf{h}& =\sum _{p=1}^P{\mathbf{h}}_{2,p}\otimes {\mathbf{h}}_{1,p}\hfill \end{array}$$

(5)

and $y\left(t\right)$ in (1) becomes

$$\begin{array}{cc}\hfill y\left(t\right)& ={\left(\sum _{p=1}^P{\mathbf{h}}_{2,p}\otimes {\mathbf{h}}_{1,p}\right)}^T\mathbf{x}\left(t\right).\hfill \end{array}$$

(6)

Let us define the symmetric impulse response [29]:

$$\begin{array}{cc}\hfill {\mathbf{h}}^{+}& =\frac{{\sum }_{p=1}^P\left({\mathbf{h}}_{2,p}\otimes {\mathbf{h}}_{1,p}+{\mathbf{h}}_{1,p}\otimes {\mathbf{h}}_{2,p}\right)}{2}.\hfill \end{array}$$

(7)

We should note that this particular symmetric characteristic is not related to the samples of the impulse response (e.g., like in linear-phase finite-impulse-response filters), but to a specific combination of the two sub-impulse responses ${\mathbf{h}}_{1,p}$ and ${\mathbf{h}}_{2,p}$. Thus, denoting by ${\mathbf{h}}_p({\mathbf{h}}_{1,p},{\mathbf{h}}_{2,p})$ the pth term of the sum from the numerator of (7), we can write

$$\begin{array}{cc}\hfill {\mathbf{h}}_p({\mathbf{h}}_{1,p},{\mathbf{h}}_{2,p})& ={\mathbf{h}}_p({\mathbf{h}}_{2,p},{\mathbf{h}}_{1,p}),\hfill \end{array}$$

(8)

$$\begin{array}{cc}\hfill {\mathbf{h}}_p({\mathbf{h}}_{1,p},{\mathbf{h}}_{1,p})& =2{\mathbf{h}}_{1,p}\otimes {\mathbf{h}}_{1,p},\hfill \end{array}$$

(9)

which define the particular symmetry mentioned before.

It can be shown that

$$\begin{array}{cc}\hfill {\mathbf{h}}_{1,p}\otimes {\mathbf{h}}_{2,p}& ={\mathbf{P}}_{L_1{L}_2}\left({\mathbf{h}}_{2,p}\otimes {\mathbf{h}}_{1,p}\right),\hfill \end{array}$$

(10)

where ${\mathbf{P}}_{L_1{L}_2}$ is an $L\times L$ permutation matrix; see [32] and Appendix A. Substituting the previous expression into (7), we get

$$\begin{array}{cc}\hfill {\mathbf{h}}^{+}& =\mathbf{Q}\sum _{p=1}^P{\mathbf{h}}_{2,p}\otimes {\mathbf{h}}_{1,p}=\mathbf{Q}\mathbf{h},\hfill \end{array}$$

(11)

where

$$\begin{array}{cc}\hfill \mathbf{Q}& =\frac{{\mathbf{I}}_L+{\mathbf{P}}_{L_1{L}_2}}{2},\hfill \end{array}$$

(12)

with ${\mathbf{I}}_L$ being the $L\times L$ identity matrix. As a consequence, another interesting way to express (6) is

$$\begin{array}{cc}\hfill y\left(t\right)& ={\left({\mathbf{h}}^{+}\right)}^T{\mathbf{Q}}^{-T}\mathbf{x}\left(t\right)\hfill \end{array}$$

(13)

and our objective is to identify the symmetric impulse response, ${\mathbf{h}}^{+}$, with a symmetric filter; it is then easy to find an estimate of the original impulse response from the relation in (11).

3. Iterative Wiener Filter

Let the filter $\mathbf{g}$ be an estimate of the impulse response $\mathbf{h}$ and let the symmetric filter:

$$\begin{array}{cc}\hfill {\mathbf{g}}^{+}& =\frac{{\sum }_{p=1}^P\left({\mathbf{g}}_{2,p}\otimes {\mathbf{g}}_{1,p}+{\mathbf{g}}_{1,p}\otimes {\mathbf{g}}_{2,p}\right)}{2}\hfill \\ \hfill & =\mathbf{Q}\mathbf{g}\hfill \end{array}$$

(14)

be an estimate of ${\mathbf{h}}^{+}$, with ${\mathbf{g}}_{1,p}$ and ${\mathbf{g}}_{2,p}$ being estimates of ${\mathbf{h}}_{1,p}$ and ${\mathbf{h}}_{2,p}$, respectively. We can conveniently rewrite (14) as

$$\begin{array}{cc}\hfill {\mathbf{g}}^{+}& =\frac{{\sum }_{p=1}^P\left({\mathbf{g}}_{2,p}\otimes {\mathbf{I}}_{L_1}+{\mathbf{I}}_{L_1}\otimes {\mathbf{g}}_{2,p}\right){\mathbf{g}}_{1,p}}{2}\hfill \\ \hfill & =\sum _{p=1}^P{\mathbf{G}}_{{\mathbf{g}}_{2,p}}{\mathbf{g}}_{1,p}={\underline{\mathbf{G}}}_2{\underline{\mathbf{g}}}_1,\hfill \end{array}$$

(15)

where

$$\begin{array}{cc}\hfill {\mathbf{G}}_{{\mathbf{g}}_{2,p}}& =\frac{{\mathbf{g}}_{2,p}\otimes {\mathbf{I}}_{L_1}+{\mathbf{I}}_{L_1}\otimes {\mathbf{g}}_{2,p}}{2}\hfill \end{array}$$

is a matrix of size $L\times L_1$,

$$\begin{array}{cc}\hfill {\underline{\mathbf{G}}}_2& =\left[\begin{array}{cccc}{\mathbf{G}}_{{\mathbf{g}}_{2,1}}& {\mathbf{G}}_{{\mathbf{g}}_{2,2}}& \cdots & {\mathbf{G}}_{{\mathbf{g}}_{2,P}}\end{array}\right]\hfill \end{array}$$

is a matrix of size $L\times P{L}_1$, and

$$\begin{array}{cc}\hfill {\underline{\mathbf{g}}}_1& ={\left[\begin{array}{cccc}{\mathbf{g}}_{1,1}^T& {\mathbf{g}}_{1,2}^T& \cdots & {\mathbf{g}}_{1,P}^T\end{array}\right]}^T\hfill \end{array}$$

is a vector of length $P{L}_1$. In the same manner, we can rewrite (14) in another useful form as follows:

$$\begin{array}{cc}\hfill {\mathbf{g}}^{+}& =\frac{{\sum }_{p=1}^P\left({\mathbf{I}}_{L_2}\otimes {\mathbf{g}}_{1,p}+{\mathbf{g}}_{1,p}\otimes {\mathbf{I}}_{L_2}\right){\mathbf{g}}_{2,p}}{2}\hfill \\ \hfill & =\sum _{p=1}^P{\mathbf{G}}_{{\mathbf{g}}_{1,p}}{\mathbf{g}}_{2,p}={\underline{\mathbf{G}}}_1{\underline{\mathbf{g}}}_2,\hfill \end{array}$$

(16)

where

$$\begin{array}{cc}\hfill {\mathbf{G}}_{{\mathbf{g}}_{1,p}}& =\frac{{\mathbf{I}}_{L_2}\otimes {\mathbf{g}}_{1,p}+{\mathbf{g}}_{1,p}\otimes {\mathbf{I}}_{L_2}}{2}\hfill \end{array}$$

is a matrix of size $L\times L_2$,

$$\begin{array}{cc}\hfill {\underline{\mathbf{G}}}_1& =\left[\begin{array}{cccc}{\mathbf{G}}_{{\mathbf{g}}_{1,1}}& {\mathbf{G}}_{{\mathbf{g}}_{1,2}}& \cdots & {\mathbf{G}}_{{\mathbf{g}}_{1,P}}\end{array}\right]\hfill \end{array}$$

is a matrix of size $L\times P{L}_2$, and

$$\begin{array}{cc}\hfill {\underline{\mathbf{g}}}_2& ={\left[\begin{array}{cccc}{\mathbf{g}}_{2,1}^T& {\mathbf{g}}_{2,2}^T& \cdots & {\mathbf{g}}_{2,P}^T\end{array}\right]}^T\hfill \end{array}$$

is a vector of length $P{L}_2$.

Now, we can define the error signal between the desired signal, $d\left(t\right)$, and the estimated signal, $\widehat{y}\left(t\right)={\mathbf{g}}^T\mathbf{x}\left(t\right)$, as follows:

$$\begin{array}{cc}\hfill e\left(t\right)& =d\left(t\right)-\widehat{y}\left(t\right),\hfill \end{array}$$

(17)

which can be expressed in two different manners:

$$\begin{array}{cc}\hfill e\left(t\right)& =d\left(t\right)-{\underline{\mathbf{g}}}_1^T{\underline{\mathbf{G}}}_2^T{\mathbf{Q}}^{-T}\mathbf{x}\left(t\right)\hfill \end{array}$$

(18)

$$\begin{array}{cc}\hfill & =d\left(t\right)-{\underline{\mathbf{g}}}_2^T{\underline{\mathbf{G}}}_1^T{\mathbf{Q}}^{-T}\mathbf{x}\left(t\right).\hfill \end{array}$$

(19)

From (18) and assuming that ${\underline{\mathbf{g}}}_2$ is fixed, we can write the mean-squared error (MSE) criterion as

$$\begin{array}{cc}\hfill & \mathcal{J}\left({\underline{\mathbf{g}}}_1|{\underline{\mathbf{g}}}_2\right)=E\left[{\left|d\left(t\right)-{\underline{\mathbf{g}}}_1^T{\underline{\mathbf{G}}}_2^T{\mathbf{Q}}^{-T}\mathbf{x}\left(t\right)\right|}^2\right]\hfill \\ \hfill & ={\sigma }_d^2-2{\underline{\mathbf{g}}}_1^T{\underline{\mathbf{G}}}_2^T{\mathbf{Q}}^{-T}{\mathbf{r}}_{\mathbf{x}d}+{\underline{\mathbf{g}}}_1^T{\underline{\mathbf{G}}}_2^T{\mathbf{Q}}^{-T}{\mathbf{R}}_{\mathbf{x}}{\mathbf{Q}}^{-1}{\underline{\mathbf{G}}}_2{\underline{\mathbf{g}}}_1,\hfill \end{array}$$

(20)

where ${\sigma }_d^2$ is the variance of $d\left(t\right)$ and ${\mathbf{r}}_{\mathbf{x}d}=E\left[\mathbf{x}\left(t\right)d\left(t\right)\right]$ is the cross-correlation vector between $\mathbf{x}\left(t\right)$ and $d\left(t\right)$. From the minimization of $\mathcal{J}\left({\underline{\mathbf{g}}}_1|{\underline{\mathbf{g}}}_2\right)$, we easily get the Wiener filter for ${\underline{\mathbf{g}}}_1$, i.e.,

$$\begin{array}{cc}\hfill {\underline{\mathbf{g}}}_{1,\mathrm{W}}& ={\left({\underline{\mathbf{G}}}_2^T{\mathbf{Q}}^{-T}{\mathbf{R}}_{\mathbf{x}}{\mathbf{Q}}^{-1}{\underline{\mathbf{G}}}_2\right)}^{-1}{\underline{\mathbf{G}}}_2^T{\mathbf{Q}}^{-T}{\mathbf{r}}_{\mathbf{x}d}.\hfill \end{array}$$

(21)

In the same way, from (19) and assuming that ${\underline{\mathbf{g}}}_1$ is fixed, we can write the MSE criterion as

$$\begin{array}{c}\mathcal{J}\left({\underline{\mathbf{g}}}_2|{\underline{\mathbf{g}}}_1\right)=E\left[{\left|d\left(t\right)-{\underline{\mathbf{g}}}_2^T{\underline{\mathbf{G}}}_1^T{\mathbf{Q}}^{-T}\mathbf{x}\left(t\right)\right|}^2\right]\hfill \\ ={\sigma }_d^2-2{\underline{\mathbf{g}}}_2^T{\underline{\mathbf{G}}}_1^T{\mathbf{Q}}^{-T}{\mathbf{r}}_{\mathbf{x}d}+{\underline{\mathbf{g}}}_2^T{\underline{\mathbf{G}}}_1^T{\mathbf{Q}}^{-T}{\mathbf{R}}_{\mathbf{x}}{\mathbf{Q}}^{-1}{\underline{\mathbf{G}}}_1{\underline{\mathbf{g}}}_2,\hfill \end{array}$$

(22)

whose minimization leads to the Wiener filter for ${\underline{\mathbf{g}}}_2$, i.e.,

$$\begin{array}{cc}\hfill {\underline{\mathbf{g}}}_{2,\mathrm{W}}& ={\left({\underline{\mathbf{G}}}_1^T{\mathbf{Q}}^{-T}{\mathbf{R}}_{\mathbf{x}}{\mathbf{Q}}^{-1}{\underline{\mathbf{G}}}_1\right)}^{-1}{\underline{\mathbf{G}}}_1^T{\mathbf{Q}}^{-T}{\mathbf{r}}_{\mathbf{x}d}.\hfill \end{array}$$

(23)

By alternatively iterating between ${\underline{\mathbf{g}}}_{1,\mathrm{W}}$ and ${\underline{\mathbf{g}}}_{2,\mathrm{W}}$ as in [16,33], these two optimal filters will converge to the Wiener solution just after a couple of iterations. The optimal symmetric filter ${\mathbf{g}}_{\mathrm{W}}^{+}$ can be easily constructed and then the Wiener filter ${\mathbf{g}}_{\mathrm{W}}$, which is the optimal estimate of $\mathbf{h}$, follows immediately.

As compared to the conventional Wiener filter, which obtains its solution by solving a linear system of L equations that involves large data structures (e.g., the matrix ${\mathbf{R}}_{\mathbf{x}}$ of size $L\times L$), the proposed iterative Wiener filter combines the solutions of two shorter filters of lengths $P{L}_1$ and $P{L}_2$, with $P\ll L_2$. Thus, it would be more robust and accurate, especially when the estimates of the statistics (i.e., ${\mathbf{R}}_{\mathbf{x}}$ and ${\mathbf{r}}_{\mathbf{x}d}$) are less accurate. Such scenarios happen when only a small amount of data samples is available and/or in noisy conditions (i.e., low SNRs), which raise very challenging practical issues.

4. RLS Adaptive Algorithm

The iterative Wiener filter developed in the previous section owns important advantages as compared to its conventional counterpart. Simulation results provided in Section 5.1 will support these performance features. Nevertheless, this iterative algorithm inherits some inherent limitations of the Wiener filter, like the estimation of the required statistics, the time-invariant framework, and the matrix inversion operation. In practice, especially in real-time applications, the impulse response of the system to be identified could change in time. For example, the acoustic echo paths are time-variant systems, depending on temperature, pressure, humidity, and movement of objects or bodies [2]. This requires good tracking capabilities for the estimation algorithm. In this context, using an adaptive filter represents a more practical solution as compared to the Wiener filter. Consequently, in this section, we propose an RLS-based algorithm, following the approach presented in Section 3.

Let us denote by $\widehat{\mathbf{g}}\left(t\right)$ the impulse response of the adaptive filter at time index t. The conventional RLS algorithm [34,35,36] is defined by the following equations:

$$\begin{array}{cc}\hfill \epsilon \left(t\right)& =d\left(t\right)-{\widehat{\mathbf{g}}}^T(t-1)\mathbf{x}\left(t\right),\hfill \end{array}$$

(24)

$$\begin{array}{cc}\hfill \mathbf{k}\left(t\right)& =\frac{{\widehat{\mathbf{R}}}_{\mathbf{x}}^{-1}(t-1)\mathbf{x}\left(t\right)}{\lambda +{\mathbf{x}}^T\left(t\right){\widehat{\mathbf{R}}}_{\mathbf{x}}^{-1}(t-1)\mathbf{x}\left(t\right)},\hfill \end{array}$$

(25)

$$\begin{array}{cc}\hfill \widehat{\mathbf{g}}\left(t\right)& =\widehat{\mathbf{g}}(t-1)+\mathbf{k}\left(t\right)\epsilon \left(t\right),\hfill \end{array}$$

(26)

$$\begin{array}{cc}\hfill {\widehat{\mathbf{R}}}_{\mathbf{x}}^{-1}\left(t\right)& =\frac{1}{\lambda }\left[{\mathbf{I}}_L-\mathbf{k}\left(t\right){\mathbf{x}}^T\left(t\right)\right]{\widehat{\mathbf{R}}}_{\mathbf{x}}^{-1}(t-1),\hfill \end{array}$$

(27)

where $\epsilon \left(t\right)$ is the a priori error signal, $\mathbf{k}\left(t\right)$ stands for the Kalman gain vector [37], ${\widehat{\mathbf{R}}}_{\mathbf{x}}^{-1}\left(t\right)$ represents an estimate of the inverse of the input signal covariance matrix, and $\lambda$ is the so-called forgetting factor ($0<\lambda \le 1$), which controls the “memory” of the algorithm [38]. As we can notice, the matrix inversion operation is avoided based on the matrix inversion lemma (i.e., the Woodbury formula), which leads to the update from (27). The tracking capability of the algorithm is controlled in terms of the value of the forgetting factor, which is usually chosen as $\lambda =1-1/\left(KL\right)$, with $K\ge 1$. A large value of this parameter (i.e., very close or equal to one) improves the accuracy of the solution, but reduces the tracking capability of the algorithm.

As compared to the LMS-based algorithms, the RLS adaptive filter provides a much faster convergence rate (especially for correlated input signals), while requiring a large computational amount, which is proportional to $\mathcal{O}\left(L^2\right)$. Consequently, for long length filters (e.g., like in echo cancellation), the conventional RLS algorithm becomes prohibitive. On the other hand, there are several “fast” versions of this algorithm, e.g., [39,40], with a computational complexity proportional to the filter length. However, they still face the inherent challenges and limitations related to a long length filter, especially in terms of the convergence rate and tracking capability.

The main feature of the decomposition-based approach presented in the previous sections is that it reformulates the system identification problem based on a combination of shorter filters. Thus, the adaptive filter can also be decomposed similar to (5), thus resulting in

$$\begin{array}{cc}\hfill \widehat{\mathbf{g}}\left(t\right)& =\sum _{p=1}^P{\widehat{\mathbf{g}}}_{2,p}\left(t\right)\otimes {\widehat{\mathbf{g}}}_{1,p}\left(t\right),\hfill \end{array}$$

(28)

where ${\widehat{\mathbf{g}}}_{1,p}\left(t\right)$ and ${\widehat{\mathbf{g}}}_{2,p}\left(t\right)$, with $p=1,2,\dots ,P$, are filters of length $L_1$ and $L_2$, respectively; these represent the estimates of ${\mathbf{h}}_{1,p}$ and ${\mathbf{h}}_{2,p}$, respectively. Furthermore, similar to (14), a symmetric adaptive filter can be defined as

$$\begin{array}{cc}\hfill {\widehat{\mathbf{g}}}^{+}\left(t\right)& =\frac{{\sum }_{p=1}^P\left[{\widehat{\mathbf{g}}}_{2,p}\left(t\right)\otimes {\widehat{\mathbf{g}}}_{1,p}\left(t\right)+{\widehat{\mathbf{g}}}_{1,p}\left(t\right)\otimes {\widehat{\mathbf{g}}}_{2,p}\left(t\right)\right]}{2}\hfill \\ \hfill & =\mathbf{Q}\widehat{\mathbf{g}}\left(t\right),\hfill \end{array}$$

(29)

where the matrix $\mathbf{Q}$ is defined in (12). Following the same development as in (15), the symmetric filter from (29) can be rewritten as

$$\begin{array}{cc}\hfill {\widehat{\mathbf{g}}}^{+}\left(t\right)& =\frac{{\sum }_{p=1}^P\left[{\widehat{\mathbf{g}}}_{2,p}\left(t\right)\otimes {\mathbf{I}}_{L_1}+{\mathbf{I}}_{L_1}\otimes {\widehat{\mathbf{g}}}_{2,p}\left(t\right)\right]{\widehat{\mathbf{g}}}_{1,p}\left(t\right)}{2}\hfill \\ \hfill & =\sum _{p=1}^P{\widehat{\mathbf{G}}}_{{\widehat{\mathbf{g}}}_{2,p}}\left(t\right){\widehat{\mathbf{g}}}_{1,p}\left(t\right)={\widehat{\underline{\mathbf{G}}}}_2\left(t\right){\widehat{\underline{\mathbf{g}}}}_1\left(t\right),\hfill \end{array}$$

(30)

where

$$\begin{array}{cc}\hfill {\widehat{\mathbf{G}}}_{{\widehat{\mathbf{g}}}_{2,p}}\left(t\right)& =\frac{{\widehat{\mathbf{g}}}_{2,p}\left(t\right)\otimes {\mathbf{I}}_{L_1}+{\mathbf{I}}_{L_1}\otimes {\widehat{\mathbf{g}}}_{2,p}\left(t\right)}{2},p=1,2,\dots ,P,\hfill \\ \hfill {\widehat{\underline{\mathbf{G}}}}_2\left(t\right)& =\left[\begin{array}{cccc}{\widehat{\mathbf{G}}}_{{\widehat{\mathbf{g}}}_{2,1}}\left(t\right)& {\widehat{\mathbf{G}}}_{{\widehat{\mathbf{g}}}_{2,2}}\left(t\right)& \cdots & {\widehat{\mathbf{G}}}_{{\widehat{\mathbf{g}}}_{2,P}}\left(t\right)\end{array}\right],\hfill \\ \hfill {\widehat{\underline{\mathbf{g}}}}_1\left(t\right)& ={\left[\begin{array}{cccc}{\widehat{\mathbf{g}}}_{1,1}^T\left(t\right)& {\widehat{\mathbf{g}}}_{1,2}^T\left(t\right)& \cdots & {\widehat{\mathbf{g}}}_{1,P}^T\left(t\right)\end{array}\right]}^T.\hfill \end{array}$$

(31)

On the other hand, based on (16), we can express (29) as

$$\begin{array}{cc}\hfill {\widehat{\mathbf{g}}}^{+}\left(t\right)& =\frac{{\sum }_{p=1}^P\left[{\mathbf{I}}_{L_2}\otimes {\widehat{\mathbf{g}}}_{1,p}\left(t\right)+{\widehat{\mathbf{g}}}_{1,p}\left(t\right)\otimes {\mathbf{I}}_{L_2}\right]{\widehat{\mathbf{g}}}_{2,p}\left(t\right)}{2}\hfill \\ \hfill & =\sum _{p=1}^P{\widehat{\mathbf{G}}}_{{\widehat{\mathbf{g}}}_{1,p}}\left(t\right){\widehat{\mathbf{g}}}_{2,p}\left(t\right)={\widehat{\underline{\mathbf{G}}}}_1\left(t\right){\widehat{\underline{\mathbf{g}}}}_2\left(t\right),\hfill \end{array}$$

(32)

where

$$\begin{array}{cc}\hfill {\widehat{\mathbf{G}}}_{{\widehat{\mathbf{g}}}_{1,p}}\left(t\right)& =\frac{{\mathbf{I}}_{L_2}\otimes {\widehat{\mathbf{g}}}_{1,p}\left(t\right)+{\widehat{\mathbf{g}}}_{1,p}\left(t\right)\otimes {\mathbf{I}}_{L_2}}{2},p=1,2,\dots ,P,\hfill \\ \hfill {\widehat{\underline{\mathbf{G}}}}_1\left(t\right)& =\left[\begin{array}{cccc}{\widehat{\mathbf{G}}}_{{\widehat{\mathbf{g}}}_{1,1}}\left(t\right)& {\widehat{\mathbf{G}}}_{{\widehat{\mathbf{g}}}_{1,2}}\left(t\right)& \cdots & {\widehat{\mathbf{G}}}_{{\widehat{\mathbf{g}}}_{1,P}}\left(t\right)\end{array}\right],\hfill \\ \hfill {\widehat{\underline{\mathbf{g}}}}_2\left(t\right)& ={\left[\begin{array}{cccc}{\widehat{\mathbf{g}}}_{2,1}^T\left(t\right)& {\widehat{\mathbf{g}}}_{2,2}^T\left(t\right)& \cdots & {\widehat{\mathbf{g}}}_{2,P}^T\left(t\right)\end{array}\right]}^T.\hfill \end{array}$$

(33)

At this point, following (18) and (19), and based on (28)–(32), we can express the a priori error from (24) in two equivalent forms, i.e.,

$$\begin{array}{cc}\hfill {\epsilon }_1\left(t\right)& =d\left(t\right)-{\widehat{\underline{\mathbf{g}}}}_1^T(t-1){\widehat{\underline{\mathbf{G}}}}_2^T(t-1){\mathbf{Q}}^{-T}\mathbf{x}\left(t\right)\hfill \\ \hfill & =d\left(t\right)-{\widehat{\underline{\mathbf{g}}}}_1^T(t-1){\mathbf{x}}_{{\widehat{\underline{\mathbf{g}}}}_2}\left(t\right)\hfill \end{array}$$

(34)

and

$$\begin{array}{cc}\hfill {\epsilon }_2\left(t\right)& =d\left(t\right)-{\widehat{\underline{\mathbf{g}}}}_2^T(t-1){\widehat{\underline{\mathbf{G}}}}_1^T(t-1){\mathbf{Q}}^{-T}\mathbf{x}\left(t\right)\hfill \\ \hfill & =d\left(t\right)-{\widehat{\underline{\mathbf{g}}}}_2^T(t-1){\mathbf{x}}_{{\widehat{\underline{\mathbf{g}}}}_1}\left(t\right),\hfill \end{array}$$

(35)

where ${\mathbf{x}}_{{\widehat{\underline{\mathbf{g}}}}_2}\left(t\right)={\widehat{\underline{\mathbf{G}}}}_2^T(t-1){\mathbf{Q}}^{-T}\mathbf{x}\left(t\right)$ and ${\mathbf{x}}_{{\widehat{\underline{\mathbf{g}}}}_1}\left(t\right)={\widehat{\underline{\mathbf{G}}}}_1^T(t-1){\mathbf{Q}}^{-T}\mathbf{x}\left(t\right)$. Based on the expressions from (34) and (35), the cost functions of the RLS-based algorithm can be constructed. Therefore, following the least-squares (LS) optimization criterion [34,35,36], the corresponding cost functions result in

$$\begin{array}{cc}\hfill {\mathcal{J}}_{{\widehat{\underline{\mathbf{g}}}}_2}\left[{\widehat{\underline{\mathbf{g}}}}_1\left(t\right)\right]& =\sum _{i=1}^t{\lambda }_1^{t-i}{\left[d\left(i\right)-{\widehat{\underline{\mathbf{g}}}}_1^T\left(t\right){\mathbf{x}}_{{\widehat{\underline{\mathbf{g}}}}_2}\left(i\right)\right]}^2,\hfill \end{array}$$

(36)

$$\begin{array}{cc}\hfill {\mathcal{J}}_{{\widehat{\underline{\mathbf{g}}}}_1}\left[{\widehat{\underline{\mathbf{g}}}}_2\left(t\right)\right]& =\sum _{i=1}^t{\lambda }_2^{t-i}{\left[d\left(i\right)-{\widehat{\underline{\mathbf{g}}}}_2^T\left(t\right){\mathbf{x}}_{{\widehat{\underline{\mathbf{g}}}}_1}\left(i\right)\right]}^2,\hfill \end{array}$$

(37)

where ${\lambda }_1$ and ${\lambda }_2$ denote the forgetting factors, which are positive parameters smaller (or equal) to one. These two forgetting factors can be chosen according to the corresponding filters’ lengths, i.e., ${\lambda }_1=1-1/\left(KP{L}_1\right)$ and ${\lambda }_2=1-1/\left(KP{L}_2\right)$, with $K\ge 1$.

The cost functions from (36) and (37) suggest a bilinear optimization strategy [41], i.e., one of the systems is considered fixed within the optimization criterion of the other one. First, focusing on (36) and considering that ${\widehat{\underline{\mathbf{g}}}}_2\left(i\right)$ is fixed for $i=1,2,\dots ,t-1$, the corresponding cost function can be developed as

$$\begin{array}{cc}\hfill {\mathcal{J}}_{{\widehat{\underline{\mathbf{g}}}}_2}\left[{\widehat{\underline{\mathbf{g}}}}_1\left(t\right)\right]& =\sum _{i=1}^t{\lambda }_1^{t-i}{d}^2\left(i\right)-2{\widehat{\underline{\mathbf{g}}}}_1^T\left(t\right){\mathbf{p}}_{{\widehat{\underline{\mathbf{g}}}}_2}\left(t\right)+{\widehat{\underline{\mathbf{g}}}}_1^T\left(t\right){\mathbf{R}}_{{\widehat{\underline{\mathbf{g}}}}_2}\left(t\right){\widehat{\underline{\mathbf{g}}}}_1\left(t\right),\hfill \end{array}$$

(38)

where

$$\begin{array}{cc}\hfill {\mathbf{p}}_{{\widehat{\underline{\mathbf{g}}}}_2}\left(t\right)& =\sum _{i=1}^t{\lambda }_1^{t-i}{\mathbf{x}}_{{\widehat{\underline{\mathbf{g}}}}_2}\left(i\right)d\left(i\right)\hfill \\ \hfill & ={\lambda }_1{\mathbf{p}}_{{\widehat{\underline{\mathbf{g}}}}_2}(t-1)+{\mathbf{x}}_{{\widehat{\underline{\mathbf{g}}}}_2}\left(t\right)d\left(t\right),\hfill \end{array}$$

(39)

$$\begin{array}{cc}\hfill {\mathbf{R}}_{{\widehat{\underline{\mathbf{g}}}}_2}\left(t\right)& =\sum _{i=1}^t{\lambda }_1^{t-i}{\mathbf{x}}_{{\widehat{\underline{\mathbf{g}}}}_2}\left(i\right){\mathbf{x}}_{{\widehat{\underline{\mathbf{g}}}}_2}^T\left(i\right)\hfill \\ \hfill & ={\lambda }_1{\mathbf{R}}_{{\widehat{\underline{\mathbf{g}}}}_2}(t-1)+{\mathbf{x}}_{{\widehat{\underline{\mathbf{g}}}}_2}\left(t\right){\mathbf{x}}_{{\widehat{\underline{\mathbf{g}}}}_2}^T\left(t\right).\hfill \end{array}$$

(40)

Similarly, following (37) and considering that ${\widehat{\underline{\mathbf{g}}}}_1\left(i\right)$ is fixed for $i=1,2,\dots ,t-1$, the second cost function results in

$$\begin{array}{cc}\hfill {\mathcal{J}}_{{\widehat{\underline{\mathbf{g}}}}_1}\left[{\widehat{\underline{\mathbf{g}}}}_2\left(t\right)\right]& =\sum _{i=1}^t{\lambda }_2^{t-i}{d}^2\left(i\right)-2{\widehat{\underline{\mathbf{g}}}}_2^T\left(t\right){\mathbf{p}}_{{\widehat{\underline{\mathbf{g}}}}_1}\left(t\right)+{\widehat{\underline{\mathbf{g}}}}_2^T\left(t\right){\mathbf{R}}_{{\widehat{\underline{\mathbf{g}}}}_1}\left(t\right){\widehat{\underline{\mathbf{g}}}}_2\left(t\right),\hfill \end{array}$$

(41)

where

$$\begin{array}{cc}\hfill {\mathbf{p}}_{{\widehat{\underline{\mathbf{g}}}}_1}\left(t\right)& =\sum _{i=1}^t{\lambda }_2^{t-i}{\mathbf{x}}_{{\widehat{\underline{\mathbf{g}}}}_1}\left(i\right)d\left(i\right)\hfill \\ \hfill & ={\lambda }_2{\mathbf{p}}_{{\widehat{\underline{\mathbf{g}}}}_1}(t-1)+{\mathbf{x}}_{{\widehat{\underline{\mathbf{g}}}}_1}\left(t\right)d\left(t\right),\hfill \end{array}$$

(42)

$$\begin{array}{cc}\hfill {\mathbf{R}}_{{\widehat{\underline{\mathbf{g}}}}_1}\left(t\right)& =\sum _{i=1}^t{\lambda }_2^{t-i}{\mathbf{x}}_{{\widehat{\underline{\mathbf{g}}}}_1}\left(i\right){\mathbf{x}}_{{\widehat{\underline{\mathbf{g}}}}_1}^T\left(i\right)\hfill \\ \hfill & ={\lambda }_2{\mathbf{R}}_{{\widehat{\underline{\mathbf{g}}}}_1}(t-1)+{\mathbf{x}}_{{\widehat{\underline{\mathbf{g}}}}_1}\left(t\right){\mathbf{x}}_{{\widehat{\underline{\mathbf{g}}}}_1}^T\left(t\right).\hfill \end{array}$$

(43)

Thus, the minimization of ${\mathcal{J}}_{{\widehat{\underline{\mathbf{g}}}}_2}\left[{\widehat{\underline{\mathbf{g}}}}_1\left(t\right)\right]$ and ${\mathcal{J}}_{{\widehat{\underline{\mathbf{g}}}}_1}\left[{\widehat{\underline{\mathbf{g}}}}_2\left(t\right)\right]$ with respect to ${\widehat{\underline{\mathbf{g}}}}_1\left(t\right)$ and ${\widehat{\underline{\mathbf{g}}}}_2\left(t\right)$, respectively, lead to the normal equations:

$$\begin{array}{cc}\hfill {\mathbf{R}}_{{\widehat{\underline{\mathbf{g}}}}_2}\left(t\right){\widehat{\underline{\mathbf{g}}}}_1\left(t\right)& ={\mathbf{p}}_{{\widehat{\underline{\mathbf{g}}}}_2}\left(t\right),\hfill \end{array}$$

(44)

$$\begin{array}{cc}\hfill {\mathbf{R}}_{{\widehat{\underline{\mathbf{g}}}}_1}\left(t\right){\widehat{\underline{\mathbf{g}}}}_2\left(t\right)& ={\mathbf{p}}_{{\widehat{\underline{\mathbf{g}}}}_1}\left(t\right).\hfill \end{array}$$

(45)

In order to avoid the matrix inversion in (44) and (45), we can use the matrix inversion lemma in (40) and (43), which result in the updates:

$$\begin{array}{cc}\hfill {\mathbf{R}}_{{\widehat{\underline{\mathbf{g}}}}_2}^{-1}\left(t\right)& =\frac{1}{{\lambda }_1}\left[{\mathbf{I}}_{P{L}_1}-{\mathbf{k}}_{{\widehat{\underline{\mathbf{g}}}}_1}\left(t\right){\mathbf{x}}_{{\widehat{\underline{\mathbf{g}}}}_2}^T\left(t\right)\right]{\mathbf{R}}_{{\widehat{\underline{\mathbf{g}}}}_2}^{-1}(t-1),\hfill \end{array}$$

(46)

$$\begin{array}{cc}\hfill {\mathbf{R}}_{{\widehat{\underline{\mathbf{g}}}}_1}^{-1}\left(t\right)& =\frac{1}{{\lambda }_2}\left[{\mathbf{I}}_{P{L}_2}-{\mathbf{k}}_{{\widehat{\underline{\mathbf{g}}}}_2}\left(t\right){\mathbf{x}}_{{\widehat{\underline{\mathbf{g}}}}_1}^T\left(t\right)\right]{\mathbf{R}}_{{\widehat{\underline{\mathbf{g}}}}_1}^{-1}(t-1),\hfill \end{array}$$

(47)

where ${\mathbf{I}}_{P{L}_1}$ and ${\mathbf{I}}_{P{L}_2}$ denote the identity matrices of size $P{L}_1\times P{L}_1$ and $P{L}_2\times P{L}_2$, respectively, and

$$\begin{array}{cc}\hfill {\mathbf{k}}_{{\widehat{\underline{\mathbf{g}}}}_1}\left(t\right)& ={\mathbf{R}}_{{\widehat{\underline{\mathbf{g}}}}_2}^{-1}\left(t\right){\mathbf{x}}_{{\widehat{\underline{\mathbf{g}}}}_2}\left(t\right)\hfill \\ \hfill & =\frac{{\mathbf{R}}_{{\widehat{\underline{\mathbf{g}}}}_2}^{-1}(t-1){\mathbf{x}}_{{\widehat{\underline{\mathbf{g}}}}_2}\left(t\right)}{{\lambda }_1+{\mathbf{x}}_{{\widehat{\underline{\mathbf{g}}}}_2}^T\left(t\right){\mathbf{R}}_{{\widehat{\underline{\mathbf{g}}}}_2}^{-1}(t-1){\mathbf{x}}_{{\widehat{\underline{\mathbf{g}}}}_2}\left(t\right)},\hfill \end{array}$$

(48)

$$\begin{array}{cc}\hfill {\mathbf{k}}_{{\widehat{\underline{\mathbf{g}}}}_2}\left(t\right)& ={\mathbf{R}}_{{\widehat{\underline{\mathbf{g}}}}_1}^{-1}\left(t\right){\mathbf{x}}_{{\widehat{\underline{\mathbf{g}}}}_1}\left(t\right)\hfill \\ \hfill & =\frac{{\mathbf{R}}_{{\widehat{\underline{\mathbf{g}}}}_1}^{-1}(t-1){\mathbf{x}}_{{\widehat{\underline{\mathbf{g}}}}_1}\left(t\right)}{{\lambda }_2+{\mathbf{x}}_{{\widehat{\underline{\mathbf{g}}}}_1}^T\left(t\right){\mathbf{R}}_{{\widehat{\underline{\mathbf{g}}}}_1}^{-1}(t-1){\mathbf{x}}_{{\widehat{\underline{\mathbf{g}}}}_1}\left(t\right)}\hfill \end{array}$$

(49)

are the corresponding Kalman gain vectors. Consequently, the filters’ updates become

$$\begin{array}{cc}\hfill {\widehat{\underline{\mathbf{g}}}}_1\left(t\right)& ={\widehat{\underline{\mathbf{g}}}}_1(t-1)+{\mathbf{k}}_{{\widehat{\underline{\mathbf{g}}}}_1}\left(t\right){\epsilon }_1\left(t\right),\hfill \end{array}$$

(50)

$$\begin{array}{cc}\hfill {\widehat{\underline{\mathbf{g}}}}_2\left(t\right)& ={\widehat{\underline{\mathbf{g}}}}_2(t-1)+{\mathbf{k}}_{{\widehat{\underline{\mathbf{g}}}}_2}\left(t\right){\epsilon }_2\left(t\right).\hfill \end{array}$$

(51)

Finally, the components of the vectors ${\widehat{\underline{\mathbf{g}}}}_1\left(t\right)$ and ${\widehat{\underline{\mathbf{g}}}}_2\left(t\right)$ [see (31) and (33)] are used in (29) to obtain the symmetric filter ${\widehat{\mathbf{g}}}^{+}\left(t\right)$, then the estimate $\widehat{\mathbf{g}}\left(t\right)$ results immediately. Nevertheless, we should note that in some applications (like echo cancellation), the coefficients of the adaptive filter are not really required, because only the error signal is needed. In such cases, the evaluation of $\widehat{\mathbf{g}}\left(t\right)$ based on (29) is not necessary, and the adaptive filter provides only the a priori error signal $\epsilon \left(t\right)={\epsilon }_1\left(t\right)={\epsilon }_2\left(t\right)$.

The proposed RLS algorithm with symmetric filter (RLS-SF) is summarized in

Table 1. RLS algorithm with symmetric filter (RLS-SF).

$$\begin{array}{ccc}& & \mathrm{Data}:\hfill \\ & & \mathbf{x}\left(t\right),d\left(t\right),\mathbf{Q}=\frac{{\mathbf{I}}_L+{\mathbf{P}}_{L_1{L}_2}}{2},{\mathbf{Q}}^{-1}\hfill \\ & & \mathrm{Parameters}:\hfill \\ & & {\delta }_1>0,{\delta }_2>0,{\lambda }_1=1-\frac{1}{KP{L}_1},{\lambda }_2=1-\frac{1}{KP{L}_2},K\ge 1\hfill \\ & & \mathrm{Initialization}:\hfill \\ & & {\widehat{\mathbf{g}}}_{1,p}\left(0\right)={\left[\begin{array}{cc}1& {\mathbf{0}}_{L_1-1}^T\end{array}\right]}^T,p=1,2,\dots ,P\hfill \\ & & {\widehat{\underline{\mathbf{g}}}}_1\left(0\right)={\left[\begin{array}{cccc}{\widehat{\mathbf{g}}}_{1,1}^T\left(0\right)& {\widehat{\mathbf{g}}}_{1,2}^T\left(0\right)& \cdots & {\widehat{\mathbf{g}}}_{1,P}^T\left(0\right)\end{array}\right]}^T\hfill \\ & & {\widehat{\mathbf{g}}}_{2,p}\left(0\right)={\left[\begin{array}{cc}1& {\mathbf{0}}_{L_2-1}^T\end{array}\right]}^T,p=1,2,\dots ,P\hfill \\ & & {\widehat{\underline{\mathbf{g}}}}_2\left(0\right)={\left[\begin{array}{cccc}{\widehat{\mathbf{g}}}_{2,1}^T\left(0\right)& {\widehat{\mathbf{g}}}_{2,2}^T\left(0\right)& \cdots & {\widehat{\mathbf{g}}}_{2,P}^T\left(0\right)\end{array}\right]}^T\hfill \\ & & {\mathbf{R}}_{{\widehat{\underline{\mathbf{g}}}}_2}^{-1}\left(0\right)={\delta }_1^{-1}{\mathbf{I}}_{P{L}_1},{\mathbf{R}}_{{\widehat{\underline{\mathbf{g}}}}_1}^{-1}\left(0\right)={\delta }_2^{-1}{\mathbf{I}}_{P{L}_2}\hfill \\ & & \mathrm{For}\mathrm{discrete}-\mathrm{time}\mathrm{index}-t=1,2,\dots \hfill \\ & & {\widehat{\mathit{G}}}_{{\widehat{\mathbf{g}}}_{2,p}}(\mathit{t}-1)=\frac{{\widehat{\mathbf{g}}}_{2,p}(t-1)\otimes {\mathbf{I}}_{L_1}+{\mathbf{I}}_{L_1}\otimes {\widehat{\mathbf{g}}}_{2,p}(t-1)}{2},p=1,2,\dots ,P\hfill \\ & & {\widehat{\underline{\mathbf{G}}}}_2(t-1)=\left[\begin{array}{cccc}{\widehat{\mathbf{G}}}_{{\widehat{\mathbf{g}}}_{2,1}}(t-1)& {\widehat{\mathbf{G}}}_{{\widehat{\mathbf{g}}}_{2,2}}(t-1)& \cdots & {\widehat{\mathbf{G}}}_{{\widehat{\mathbf{g}}}_{2,P}}(t-1)\end{array}\right]\hfill \\ & & {\widehat{\mathbf{G}}}_{{\widehat{\mathbf{g}}}_{1,p}}(t-1)=\frac{{\mathbf{I}}_{L_2}\otimes {\widehat{\mathbf{g}}}_{1,p}(t-1)+{\widehat{\mathbf{g}}}_{1,p}(t-1)\otimes {\mathbf{I}}_{L_2}}{2},p=1,2,\dots ,P\hfill \\ & & {\widehat{\underline{\mathbf{G}}}}_1(t-1)=\left[\begin{array}{cccc}{\widehat{\mathbf{G}}}_{{\widehat{\mathbf{g}}}_{1,1}}(t-1)& {\widehat{\mathbf{G}}}_{{\widehat{\mathbf{g}}}_{1,2}}(t-1)& \cdots & {\widehat{\mathbf{G}}}_{{\widehat{\mathbf{g}}}_{1,P}}(t-1)\end{array}\right]\hfill \\ & & {\mathit{x}}_{{\widehat{\underline{\mathbf{g}}}}_2}\left(t\right)={\widehat{\underline{\mathbf{G}}}}_2^T(t-1){\mathit{Q}}^{-T}\mathit{x}\left(t\right),{\mathit{x}}_{{\widehat{\underline{\mathbf{g}}}}_1}\left(t\right)={\widehat{\underline{\mathbf{G}}}}_1^T(t-1){\mathbf{Q}}^{-T}\mathbf{x}\left(t\right)\hfill \\ & & \epsilon \left(t\right)=d\left(t\right)-{\widehat{\underline{\mathbf{g}}}}_1^T(t-1){\mathbf{x}}_{{\widehat{\underline{\mathbf{g}}}}_2}\left(t\right)=d\left(t\right)-{\widehat{\underline{\mathbf{g}}}}_2^T(t-1){\mathbf{x}}_{{\widehat{\underline{\mathbf{g}}}}_1}\left(t\right)\hfill \\ & & {\mathbf{k}}_{{\widehat{\underline{\mathbf{g}}}}_1}\left(t\right)=\frac{{\mathbf{R}}_{{\widehat{\underline{\mathbf{g}}}}_2}^{-1}(t-1){\mathbf{x}}_{{\widehat{\underline{\mathbf{g}}}}_2}\left(t\right)}{{\lambda }_1+{\mathbf{x}}_{{\widehat{\underline{\mathbf{g}}}}_2}^T\left(t\right){\mathbf{R}}_{{\widehat{\underline{\mathbf{g}}}}_2}^{-1}(t-1){\mathit{x}}_{{\widehat{\underline{\mathbf{g}}}}_2}\left(t\right)}\hfill \\ & & {\mathbf{k}}_{{\widehat{\underline{\mathbf{g}}}}_2}\left(t\right)=\frac{{\mathbf{R}}_{{\widehat{\underline{\mathbf{g}}}}_1}^{-1}(t-1){\mathbf{x}}_{{\widehat{\underline{\mathbf{g}}}}_1}\left(t\right)}{{\lambda }_2+{\mathbf{x}}_{{\widehat{\underline{\mathbf{g}}}}_1}^T\left(t\right){\mathbf{R}}_{{\widehat{\underline{\mathbf{g}}}}_1}^{-1}(t-1){\mathbf{x}}_{{\widehat{\underline{\mathbf{g}}}}_1}\left(t\right)}\hfill \\ & & {\mathbf{R}}_{{\widehat{\underline{\mathbf{g}}}}_2}^{-1}\left(t\right)=\frac{1}{{\lambda }_1}\left[{\mathbf{I}}_{P{L}_1}-{\mathbf{k}}_{{\widehat{\underline{\mathbf{g}}}}_1}\left(t\right){\mathbf{x}}_{{\widehat{\underline{\mathbf{g}}}}_2}^T\left(t\right)\right]{\mathbf{R}}_{{\widehat{\underline{\mathbf{g}}}}_2}^{-1}(t-1)\hfill \\ & & {\mathbf{R}}_{{\widehat{\underline{\mathbf{g}}}}_1}^{-1}\left(t\right)=\frac{1}{{\lambda }_2}\left[{\mathbf{I}}_{P{L}_2}-{\mathbf{k}}_{{\widehat{\underline{\mathbf{g}}}}_2}\left(t\right){\mathbf{x}}_{{\widehat{\underline{\mathbf{g}}}}_1}^T\left(t\right)\right]{\mathbf{R}}_{{\widehat{\underline{\mathbf{g}}}}_1}^{-1}(t-1)\hfill \\ & & {\widehat{\underline{\mathbf{g}}}}_1\left(t\right)={\widehat{\underline{\mathbf{g}}}}_1(t-1)+{\mathbf{k}}_{{\widehat{\underline{\mathbf{g}}}}_1}\left(t\right)\epsilon \left(t\right)={\left[\begin{array}{cccc}{\widehat{\mathbf{g}}}_{1,1}^T\left(t\right)& {\widehat{\mathbf{g}}}_{1,2}^T\left(t\right)& \cdots & {\widehat{\mathbf{g}}}_{1,P}^T\left(t\right)\end{array}\right]}^T\hfill \\ & & {\widehat{\underline{\mathbf{g}}}}_2\left(t\right)={\widehat{\underline{\mathbf{g}}}}_2(t-1)+{\mathbf{k}}_{{\widehat{\underline{\mathbf{g}}}}_2}\left(t\right)\epsilon \left(t\right)={\left[\begin{array}{cccc}{\widehat{\mathbf{g}}}_{2,1}^T\left(t\right)& {\widehat{\mathbf{g}}}_{2,2}^T\left(t\right)& \cdots & {\widehat{\mathbf{g}}}_{2,P}^T\left(t\right)\end{array}\right]}^T\hfill \\ & & {\widehat{\mathbf{g}}}^{+}\left(t\right)=\frac{{\sum }_{p=1}^P\left[{\widehat{\mathbf{g}}}_{2,p}\left(t\right)\otimes {\widehat{\mathbf{g}}}_{1,p}\left(t\right)+{\widehat{\mathbf{g}}}_{1,p}\left(t\right)\otimes {\widehat{\mathbf{g}}}_{2,p}\left(t\right)\right]}{2}\hfill \\ & & \widehat{\mathbf{g}}\left(t\right)={\mathit{Q}}^{-1}{\widehat{\mathbf{g}}}^{+}\left(t\right)\hfill \end{array}$$

. As we can see, the RLS-SF combines the solutions of two adaptive filters of much smaller lengths (i.e., $P{L}_1$ and $P{L}_2$), as compared to the conventional RLS algorithm that updates a single adaptive filter of length $L=L_1{L}_2$. Because usually $P\ll L_2$, the expected gain of the RLS-SF is twofold, in terms of both performance (i.e., convergence rate/tracking) and computational complexity, which is proportional to $\mathcal{O}\left[{\left(P{L}_1\right)}^2+{\left(P{L}_2\right)}^2\right]$. Simulation results provided in Section 5.2 support these performance features.

The computational complexities of the proposed algorithms developed in Section 3 and Section 4 are similar to those proposed in [16,17], respectively. Basically, in both approaches, the original system identification problem of length $L=L_1{L}_2$ is reformulated as a combination of two shorter filters of lengths $P{L}_1$ and $P{L}_2$, where usually $P\ll L_2$. However, although the solutions developed in [16,17] follow the direct approach based on (5), the current proposed algorithms exploit (11), where the motivation is mainly twofold. First, the intrinsic characteristics of the resulting impulse response can be exploited (i.e., the particular symmetry feature) within the identification process. Second, using the matrix $\mathbf{Q}$ could lead to potential benefits in terms of further exploiting the low-rank features. This is different as compared to the approach used in [16,17], and also represents a step toward our future works on this topic. The final goal would be to further reduce the value of P, which represents an important gain, especially for the identification of acoustic impulse responses.

5. Simulation Results

Simulations are performed in the framework of echo cancellation [2,3]. The reference (or desired) signal $d\left(t\right)$ is obtained based on (1), using different network and acoustic impulse responses of length $L=500$; the sampling rate is 8 kHz. The input signal $x\left(t\right)$ is either an AR(1) process (obtained by filtering a white Gaussian noise through a first-order autoregressive model with a pole at 0.9) or a speech sequence. The output of the echo path (i.e., the echo signal) is corrupted by a background noise, $w\left(t\right)$, which is considered white and Gaussian (with different SNRs). All the experiments were performed by using MATLAB R2018b, running on an Asus GL552VX device (Windows 10 OS), which has an Intel Core i7-6700HQ CPU @ 2.60 GHz, with 4 Cores, 8 Logical Processors, and 16 GB of RAM.

Four echo paths are used in simulations and their corresponding impulse responses are depicted in Figure 1. The impulse response from Figure 1a is the first network echo path from the ITU-T G168 Recommendation [42], which is a cluster of 64 coefficients padded with zeros up to the length $L=500$. The second impulse response from Figure 1b is obtained in a similar manner, by using the fifth network echo path from the same recommendation, i.e., a cluster of 96 coefficients padded with zeros. The impulse responses from Figure 1c,d are two measured acoustic echo paths, with different sparseness degrees.

In order to support the idea behind (5), Figure 2 shows the singular values of $\mathbf{H}$ (normalized with respect to the maximum one), which is the equivalent matrix representation ($L_1\times L_2$) of the impulse response $\mathbf{h}$. In our experiments, because $L=500$, we set $L_1=25$ and $L=20$. The faster these singular values decrease to zero, the farther the column rank of $\mathbf{H}$ is from its maximum value. As we can see in Figure 2a,b, the network echo paths from Figure 1a,b can be very well modeled based on (5), because the corresponding singular values of $\mathbf{H}$ decrease very quickly to zero, so that $P\ll L_2$ in these cases. On the other hand, according to Figure 2c,d, a larger value of P is required in case of the acoustic echo paths from Figure 1c,d. Nevertheless, even in this case, the expected value of P is reasonably smaller as compared to $L_2$.

5.1. Performance of the Iterative Wiener Filter

In this first subsection, the performance of the proposed iterative Wiener filter (developed in Section 3) is investigated, as compared to the conventional Wiener filter. The performance measure is the normalized misalignment (in dB), which is evaluated as $20{\log }_{10}∥\mathbf{h}-{\mathbf{g}}_{\mathrm{W}}∥/∥\mathbf{h}∥$, where $∥·∥$ denotes the Euclidean norm. In case of the iterative Wiener filter, the estimate of the impulse response is obtained as explained in the end of Section 3, while the conventional Wiener solution results in ${\mathbf{g}}_{\mathrm{W}}={\mathbf{R}}_{\mathbf{x}}^{-1}{\mathbf{r}}_{\mathbf{x}d}$.

The statistics ${\mathbf{R}}_{\mathbf{x}}$ and ${\mathbf{r}}_{\mathbf{x}d}$ are required within both the conventional and iterative Wiener filters. These can be estimated by averaging over N available data samples of $x\left(t\right)$ and $d\left(t\right)$, so that the covariance matrix can be evaluated as $(1/N){\sum }_{t=1}^N\mathbf{x}\left(t\right){\mathbf{x}}^T\left(t\right)$, whereas the cross-correlation vector results in $(1/N){\sum }_{t=1}^N\mathbf{x}\left(t\right)d\left(t\right)$. In the experiments reported in this subsection, the input signal is an AR(1) process with a pole at 0.9. The accuracy of the statistics’ estimates reflects the accuracy of the Wiener solution, so that a higher value of N is desirable, i.e., $N\gg L$. However, in practice, only a limited set of data could be available, which raises significant challenges. Because the proposed iterative Wiener filter combines the solution of two shorter filters (of lengths $P{L}_1$ and $P{L}_2$, with $P\ll L_2$), an improved accuracy of the solution is expected in such challenging scenarios, as outlined in the end of Section 3.

In the first experiment, we consider that a large amount of data is available (i.e., $N=10L=5000$ samples) for the identification of the network echo paths from Figure 1a,b. In addition, we set $\mathrm{SNR}=20$ dB, so that the influence of the external noise on the accuracy of the statistics’ estimates is mild. The results are provided in Figure 3, comparing the conventional Wiener filter with the solution obtained by the proposed iterative algorithm using different values of P. As we can notice, the iterative Wiener filter with $P=L_2/10=2$ is able to match the accuracy of the conventional Wiener solution, whereas for $P=3$ it even outperforms its conventional counterpart. This supports the discussion related to Figure 2a,b, where the singular values of $\mathbf{H}$ decrease very quickly to zero, so that the rank of this matrix is much smaller than $L_2=20$.

A similar experiment is performed in Figure 4, for the identification of the acoustic echo paths from Figure 1c,d. The other conditions remain the same, i.e., $N=5000$ and $\mathrm{SNR}=20$ dB. As previously indicated in Figure 2c,d, the rank of matrix $\mathbf{H}$ is larger for the acoustic impulse responses. Nevertheless, a reasonably small value of P (as compared to $L_2$) can still be used even in such scenarios. The results provided in Figure 4 show that a value of $P<L_2/2$ is sufficient for the iterative Wiener filter to achieve the accuracy of the conventional Wiener solution. In addition, a reasonable attenuation of the misalignment is obtained by the iterative algorithm even for a smaller value of its specific parameter, e.g., $P<L_2/3$.

The performance gain of the iterative Wiener filter becomes more apparent in challenging scenarios, e.g., for a small value of N and/or in a low SNR environment [43]. It is known that the accuracy of the conventional Wiener filter is highly influenced in such adverse conditions [34,35,36]. In the following experiments from this subsection, we consider both these challenges, by using only $N=1000$ data samples to estimate the statistics and setting $\mathrm{SNR}=10$ dB.

In this framework, the identification of the network echo paths from Figure 1a,b is analyzed in Figure 5. As we can see, the conventional Wiener filter is not able to provide an accurate solution in this case. On the other hand, the iterative Wiener filter using $P\ll L_2$ still achieves a reasonable attenuation of the misalignment. Moreover, it can be remarked that the iterative algorithm even with $P=1$ clearly outperforms its conventional counterpart. These performances are also supported in Figure 6, where the estimated impulse responses provided by the conventional Wiener filter and the proposed iterative version (using $P=2$) are depicted, as compared to the true network impulse responses, corresponding to the previous experiment from Figure 5. As we can see, the iterative Wiener filter is able to obtain more accurate estimates as compared to its conventional counterpart.

A similar behavior can be observed in Figure 7 and Figure 8, where the identification of the acoustic echo paths from Figure 1c,d is assessed. As we can see in Figure 7, even if a larger value of P should be used here (as compared to the previous experiment from Figure 5), this value is much smaller as compared to $L_2=20$ (e.g., $P=L_2/4$ or $L_2/5$). In fact, the conventional Wiener filter is outperformed by the proposed iterative algorithm even for very small values of P (e.g., $P=3$). Finally, it is important to outline that smaller values of P can be used in these difficult scenarios (presented in Figure 5 and Figure 7), as compared to the experiments reported in Figure 3 and Figure 4, which were performed in more favorable conditions (i.e., $N\gg L$ and good SNR). Figure 8 also supports the performance gain, showing the estimated acoustic impulse responses obtained with the conventional and iterative versions of the Wiener filter. In this figure, the iterative Wiener filter uses $P=4$, whereas the other conditions are the same as in the experiment reported in Figure 7. It can be noticed that more accurate estimates can be obtained by using the proposed iterative Wiener filter, as compared to the solutions provided by the conventional Wiener benchmark.

5.2. Performance of the RLS-SF

This subsection is dedicated to the proposed RLS-SF (from Section 4), which is compared in the following to its conventional counterpart, i.e., the RLS adaptive filter [34,35,36]. The performance measure is the normalized misalignment (in dB), which is now evaluated as $20{\log }_{10}∥\mathbf{h}-\widehat{\mathbf{g}}\left(t\right)∥/∥\mathbf{h}∥$, where $\widehat{\mathbf{g}}\left(t\right)$ represents the adaptive filter estimate at time index t. In order to assess the tracking capabilities of the algorithms, an abrupt change of the echo path is introduced in most of the following experiments, by changing the impulse response in the middle of simulation.

In Figure 9, the identification of the network echo paths from Figure 1a,b is evaluated. Initially, the impulse response from Figure 1a is used for the first 2.5 s. Then, there is an abrupt change of the system and the impulse response from Figure 1b is involved. The forgetting factors of the algorithms are set in order to achieve the same misalignment level, so that we can focus only on the comparisons related to the convergence rate and tracking behavior. Consequently, the conventional RLS adaptive filter uses $\lambda =1-1/\left(KL\right)$, whereas the parameters of the RLS-SF are similarly set (according to the filters’ lengths) to ${\lambda }_1=1-1/\left(KP{L}_1\right)$ and ${\lambda }_2=1-1/\left(KP{L}_2\right)$, with $K=10$, $L=500$, $L_1=25$, and $L_2=20$. As we can see in Figure 9, the initial convergence rate is almost the same for all the algorithms, but the tracking capability of the RLS-SF is much better as compared to the conventional benchmark, for all the values of P that are used in this simulation. Most importantly, we should note that all these values satisfy the condition $P\ll L_2$, which leads to a significant gain in terms of the computational complexity. As expected, for a value of P smaller than the rank of the matrix $\mathbf{H}$ (e.g., $P=2$), the RLS-SF pays with a higher misalignment level as compared to the conventional RLS algorithm. However, even in this case, it fully compensates in terms of the computational complexity and tracking capability.

These results suggest another strategy for compromising between the main performance criteria of the RLS-SF. Because its tracking behavior is better as compared to the conventional RLS algorithm, we could increase the forgetting factors of the RLS-SF, targeting a lower misalignment, while paying with a slightly reduced tracking capability. On the other hand, the forgetting factor of the conventional RLS algorithm could be reduced in order to improve its tracking capability, but this leads to a higher misalignment level. Such a scenario is considered in Figure 9, where the conditions are similar to the previous experiment from Figure 10, but the forgetting factors are set as discussed before. Consequently, the conventional RLS algorithm uses a smaller forgetting factor, i.e., $\lambda =1-1/\left[\right(K/2\left)L\right]$, whereas the forgetting factors of the RLS-SF are increased to ${\lambda }_1=1-1/\left(2KP{L}_1\right)$ and ${\lambda }_2=1-1/\left(2KP{L}_2\right)$, with $K=10$. As expected, the results obtained in Figure 10 show that the RLS-SF outperforms the conventional RLS algorithm in terms of accuracy, reaching a lower misalignment level for most of the values of P used in this experiment. Even when using the smallest value $P=2$, the proposed algorithm is able to match the misalignment of the conventional RLS filter. In addition, the tracking capabilities of the RLS-SF are generally still better as compared to its conventional counterpart.

The next experiment considers a more challenging scenario, wherein the input signal is a speech signal and the SNR is reduced to 10 dB. The other settings remain the same as in the previous simulation. As we can see in Figure 11, the RLS-SF outperforms the conventional RLS algorithm in terms of the main performance criteria, for all the values of $P\ll L_2$. This behavior also supports the results reported in the previous subsection (for the iterative Wiener filter), which indicate that the proposed decomposition-based approach with the symmetric filters performs better (as compared to the conventional benchmarks) especially in challenging conditions, e.g., in noisy environments. The normalized misalignment represents the accuracy of the solution provided by the adaptive filter, i.e., the distance between the true impulse response of the system and its estimate. Consequently, the misalignment level is influenced by the SNR. This can be seen when comparing the results from Figure 10 and Figure 11, where most of the conditions are similar, but the SNRs are different. Although in Figure 10 the misalignment range is between $-30$ to $-20$ dB, in Figure 11 we can see a higher misalignment level, i.e., around $-15$ to $-10$ dB. Nevertheless, in both scenarios, the proposed RLS-SF reaches lower misalignment levels as compared to the conventional RLS filter.

The performance gain is also reflected in the accuracy of the estimated acoustic impulse responses (obtained by the conventional RLS filter and the RLS-SF), as indicated in Figure 12. In this figure, the RLS-SF uses $P=3$, whereas the other conditions are the same as in the experiment reported in Figure 11. As we can see, the accuracy of the estimate provided by the RLS-SF is superior to the estimate of the conventional RLS filter. Even if the estimated impulse response of the echo path is not really required in echo cancellation (where the error signal is transmitted back to the far-end), the results from Figure 12 validate the performance features of the proposed RLS-SF.

The last set of simulations focuses on the identification of the acoustic echo paths from Figure 1c,d. Because the background noise is usually considered stronger in the context of acoustic echo cancellation, while it is also amplified by the microphone of the hands-free device [2], we set $\mathrm{SNR}=10$ dB in all the following experiments. In the simulation provided in Figure 13, the input signal is an AR(1) process and the acoustic impulse response from Figure 1c is considered for the first 2.5 s, then the echo path abruptly changes into the impulse response from Figure 1d. The forgetting factors of the conventional RLS filter and the proposed RLS-SF are set to $\lambda =1-1/\left(KL\right)$ and ${\lambda }_j=1-1/\left(KP{L}_j\right)$, $j=1,2$, respectively. Because the corresponding matrix $\mathbf{H}$ of the acoustic impulse response is closer to full-rank, higher values of P are required in this framework. Nevertheless, as we can see in Figure 13, the RLS-SF using a value of P reasonably smaller as compared to $L_2$ can reach a misalignment level closer to the conventional RLS algorithm. On the other hand, the tracking capability of the RLS-SF (for all the values of P) is better as compared to its conventional counterpart.

In addition, we can further tune the forgetting factors of the RLS-SF in order to improve its accuracy (i.e., to reach a lower misalignment level), while reducing the forgetting factor of the conventional RLS algorithm for a faster tracking. Such a scenario is considered in Figure 14, where these parameters are set to ${\lambda }_j=1-1/\left(2KP{L}_j\right)$, $j=1,2$ and $\lambda =1-1/\left[\right(K/2\left)L\right]$, respectively, by using $K=10$. The other conditions are the same as in the previous experiment reported in Figure 13. As we can see in Figure 14, the initial convergence rate and the tracking reaction is basically the same for all the algorithms, but the proposed RLS-SF (for all the values of P) outperforms the conventional RLS filter in terms of the accuracy of its solution, achieving a lower misalignment level.

Finally, in Figure 15, the robustness of the algorithms is assessed. To this purpose, we consider a variation of the background noise at time 2.5 s, for a period of 0.5 s, by reducing the SNR from 10 dB to 0 dB. Such variations can often happen in acoustic environments, e.g., due to different movements of objects and bodies, the presence of ventilation systems or other electrical devices, etc. In this experiment, the input signal is a speech sequence and the acoustic echo path from Figure 1c is considered. The forgetting factors of the algorithms are set as in the previous simulation (from Figure 14). It is interesting to note that the RLS-SF has the same robustness features despite the value of P. Most importantly, the proposed algorithm reaches a lower misalignment level as compared to the conventional RLS benchmark.

The estimated acoustic impulse responses corresponding to the previous experiment are depicted in Figure 16, where the RLS-SF uses $P=8$. The results are compared to the true impulse response of the acoustic echo path from Figure 1c. It can be seen that a better accuracy of the estimate is obtained by the RLS-SF, as compared to the conventional RLS counterpart.

6. Discussion

The motivation behind the approach discussed in this paper was mainly twofold. First, we have shown how the intrinsic characteristics of the resulting impulse response can be exploited (i.e., the particular symmetry feature) within the identification process. As mentioned before, the identification of linearly separable systems that own particular intrinsic symmetric/antisymmetric properties has been addressed in our recent works [29,30]. In this paper, as compared to these previous works, we did not limit the current solution only to the identification of linearly separable systems, but for more general forms of impulse responses. Second, using the matrix $\mathbf{Q}$ within the identification process could represent a potential benefit in terms of further revealing and exploiting the low-rank features. This is different as compared to the approach used in [16,17], and also represents a step toward other potential future works on this topic. The final goal is to further reduce the value of P, which represents an important gain, especially for the identification of acoustic impulse responses.

The preliminary findings in this direction indicate promising results. For example, let us consider that we are able to find a matrix $\tilde{\mathbf{Q}}$, such that the coefficients of the impulse response $\tilde{\mathbf{Q}}\mathbf{h}$ result in the descending order of their absolute values. Also, let us focus on the singular values of the corresponding matrices, i.e., $\mathbf{H}=\mathrm{ivec}\left(\mathbf{h}\right)$ and $\tilde{\mathbf{H}}=\mathrm{ivec}\left(\tilde{\mathbf{Q}}\mathbf{h}\right)$, where $\mathrm{ivec}(·)$ is the inverse vectorization operation, which converts a vector of length $L=L_1{L}_2$ into a matrix of size $L_1\times L_2$. In case of acoustic impulse responses, these matrices are closer to full-rank. However, the faster their singular values decrease to zero, the better is the approximation that justifies (5). In order to provide a supporting example, the singular values of $\mathbf{H}$ and $\tilde{\mathbf{H}}$ (normalized with respect to the maximum one), together with the corresponding impulse responses, are represented in Figure 17, considering the acoustic impulse response $\mathbf{h}$ from Figure 1c. Clearly, a smaller value of P can be used for the approximation of the permuted impulse response from Figure 17b, because the singular values of $\tilde{\mathbf{H}}$ decrease faster to zero, as compared to the singular values of $\mathbf{H}$.

The previous discussion outlined the main features of the proposed solutions, but it also represents a motivation for future investigations on this approach. This could further lead to potential performance improvements in the framework of low-rank system identification problems, by fully exploiting the intrinsic characteristics and the specific nature of the acoustic impulse responses.

7. Conclusions and Future Works

In this paper, we have followed an efficient approach for linear system identification problems, which basically involves three key items, i.e., (i) the nearest Kronecker product decomposition of the impulse responses, (ii) low-rank approximations, and (iii) a particular symmetric filter. Such an approach fits very well for the identification of long-length impulse responses, which raise significant challenges in many real-world system identification problems.

In this framework, we have developed an iterative Wiener filter, which is able to outperform the conventional Wiener filter (in terms of the accuracy of its solution), especially when only a small amount of data is available to estimate the required statistics and/or in noisy conditions (e.g., low SNRs). For example, when $N=2L$ data samples are available and $\mathrm{SNR}=10$ dB, the conventional Wiener filter fails to provide an accurate solution (i.e., its normalized misalignment is close to 0 dB), while the proposed iterative version provides a reasonable attenuation of the misalignment (around −10 dB).

In addition, an adaptive solution has also been proposed, which results in an RLS algorithm with symmetric filter, namely the RLS-SF. This new adaptive filter provides improved performance features as compared to the conventional RLS benchmark, such as an improved tracking capability and a lower computational complexity. Moreover, the RLS-SF leads to more accurate solutions as compared to the conventional RLS filter, which reflect into lower misalignment levels (e.g., up to 5–10 dB improvement, depending on the SNR).

The proposed algorithms have been tested in the framework of network and acoustic echo cancellation, which represent very challenging system identification problems. Simulation results have indicated the good performance of the iterative Wiener filter and the RLS-SF algorithm, also supporting the theoretical findings behind their development.

In future works, we aim to improve the current solutions based on a different selection of the permutation matrix, which would further allow to better exploit the low-rank features of the systems to be identified. In addition, the development of a Kalman filter based on this approach will also be considered, which could open the path toward other potential applications, e.g., see [44,45], and the references therein.