Time-Varying Pseudoinversion Based on Full-Rank Decomposition and Zeroing Neural Networks

Abstract

The computation of the time-varying matrix pseudoinverse has become crucial in recent years for solving time-varying problems in engineering and science domains. This paper investigates the issue of calculating the time-varying pseudoinverse based on full-rank decomposition (FRD) using the zeroing neural network (ZNN) method, which is currently considered to be a cutting edge method for calculating the time-varying matrix pseudoinverse. As a consequence, for the first time in the literature, a new ZNN model called ZNNFRDP is introduced for time-varying pseudoinversion and it is based on FRD. Five numerical experiments investigate and confirm that the ZNNFRDP model performs as well as, if not better than, other well-performing ZNN models in the calculation of the time-varying pseudoinverse. Additionally, theoretical analysis and numerical findings have both supported the effectiveness of the proposed model.

1. Introduction

The real-time solution to the matrix pseudoinverse (or Moore-Penrose inverse), which commonly occurs in robotics [1], nonlinear systems [2], game theory [3], and other engineering and science domains [4,5,6], has garnered a great deal of attention in recent decades. When a matrix A is taken into consideration, the pseudoinverse $A^{†}$ is the generalization of the inverse matrix $A^{-1}$. For any matrix with entries that are real or complex numbers, the unique pseudoinverse is defined, and it is typically calculated using decomposition techniques, including the singular value decomposition, the QR decomposition, and the full-rank decomposition (FRD) [7,8]. To the best of our knowledge, these techniques typically targeted constant (or time-invariant) matrices. Nevertheless, real-time matrix pseudoinversion issues are frequently encountered. For instance, inverse kinematic issues for online control of redundant robot manipulators can be resolved through the online solution of the time-varying pseudoinverse [9,10]. It is significant to note that the zeroing neural network (ZNN) method effectively enables the real-time solution of the time-varying pseudoinverse using both direct and indirect methods, such as matrix decomposition. For instance, when computing the time-varying pseudoinverse, a ZNN model based on singular value decomposition performs better than a direct ZNN model [11], so it is beneficial to look into ZNN models based on various matrix decomposition techniques. Therefore, the subject of this paper is the calculation of the time-varying pseudoinverse of a given time-varying matrix, based on the FRD method and the ZNN method. Our approach, in particular, complies with the concepts of Propositions 1 and 2, where Proposition 1 is based on [12] (Theorem D.4 (Full rank decomposition)) and Proposition 2 is restated from [12] [Theorem D.5] (Full rank decomposition and pseudoinversion). It is worth mentioning that, for $A\in {\mathbb{R}}^{m\times n}$, the calculation and representation of several generalized inverses is closely related to the next Penrose equations:

$$\begin{array}{cccc}\left(i\right)AXA=A,& \left(ii\right)XAX=X,& \left(iii\right){\left(AX\right)}^{\mathrm{T}}=AX,& \left(iv\right){\left(XA\right)}^{\mathrm{T}}=XA\end{array}$$

(1)

while the pseudoinverse is the matrix X that satisfies all four Penrose equations. Additionally, the following basic symbols from the paper are noteworthy: $I_r$ implies the identity $r\times r$ matrix; ${\mathbf{0}}_{m\times n}$ and ${\mathbf{1}}_{m\times n}$ imply the zero and all-ones $m\times n$ matrices, respectively; $\mathrm{vec}(·)$ implies the vectorization process; ⊗ implies the Kronecker product; $\left(\dot{}\right)$ implies the time derivative; ${∥·∥}_F$ implies the matrix Frobenius norm. Last, the notations sum(.), zeros(.), eye(.), reshape(.), mod(.) and floor(.) in the algorithms of this paper follow the standard MATLAB function concept [13].

Proposition 1.

Let $A\in {\mathbb{R}}^{m\times n}$ be an arbitrary matrix of rank r, $B\in {\mathbb{R}}^{m\times n}$ be the reduced row echelon form of A, and $p\in {\mathbb{N}}^r$ be a vector that indicates the pivot columns of A. A unique FRD $A=CF$ is created from reduced row echelon form by setting the pivot columns of A in C, i.e., $C=A(:,p)\in {\mathbb{R}}^{m\times r}$, and the non-zero rows of B in F, i.e., $F=B(1:r,:)\in {\mathbb{R}}^{r\times n}$.

Proposition 2.

Let $A\in {\mathbb{R}}^{m\times n}$ be an arbitrary matrix of rank r, and $A=CF$ is the FRD of A, then:

$$A^{†}=F^{\mathrm{T}}{\left(F{F}^{\mathrm{T}}\right)}^{-1}{\left(C^{\mathrm{T}}C\right)}^{-1}{C}^{\mathrm{T}}.$$

(2)

Many different approaches for computing the pseudoinverse have been developed over the past few decades, including fast calculation algorithms [14] and error bounds techniques [15]. However, when used for large-scale scenarios, these serial computational techniques (such as iterative algorithms) are ineffective. As matrix size rises, the computing workload increases significantly [16,17]. Thus, to decrease computational complexity and boost computational efficiency, a model with parallel computing capabilities is recommended. As effective parallel computing tools, recurrent neural networks (RNNs) have recently been thoroughly studied and used to address a variety of problems, such as mobile object localization [18], robotic motion tracking [19] and portfolio selection [20,21]. Gradient (or Hopfield) neural networks (GNNs), which are typical RNNs, have been heavily used to solve the constant matrix pseudoinversion problem in real time [22,23]. However, because it is difficult or perhaps impossible for GNNs and these serial methods to take into account the derivative information of the relevant matrices, neither serial methods nor GNNs are useful in calculating the time-varying matrix pseudoinverse.

The ZNN, a unique RNN with strong parallel processing capabilities, was introduced by Zhang et al. [24] to address the aforementioned problems. When it comes to computing the time-varying pseudoinverse, the ZNN approach is currently thought to be state-of-the-art. In particular, ZNNs were initially created to trace the time-varying inverse of a matrix [24]. Nevertheless, later versions of them were dynamic models for computing the time-varying pseudoinverse of full-row/column rank matrices [25,26,27,28]. Subsequent evolution has led to the evolvent of a great number of models for computing various generalized inverses [11], including the Drazin inverse [29], the ML-weighted pseudoinverse [30], and outer inverses [31]. Along these, the ZNN method has been thoroughly investigated and employed to a broad variety of time-varying issues, with its main applications being the approximation of numerous matrix functions [32,33], problems involving quadratic optimization [10], and problems involving linear matrix equation systems [34,35]. To create ZNN dynamics, one must first define an error matrix, $E\left(t\right)\in {\mathbb{R}}^{m\times n}$, for the subjacent issue. The dynamical evolution is applied in the following phase:

$$\dot{E}\left(t\right)=\frac{\mathrm{d}E\left(t\right)}{\mathrm{d}t}=-\lambda \mathcal{H}\left(E\left(t\right)\right).$$

(3)

It is important to note that in (3) a design parameter $\lambda >0$ is used to scale the convergence, and $\mathcal{H}(·):{\mathbb{R}}^{m\times n}\to {\mathbb{R}}^{m\times n}$ indicates the elementwise application of an increasing and odd activation function on the error matrix. In this work, we examine the ZNN evolution (3) by the linear activation function, leading to the next formula:

$$\dot{E}\left(t\right)=-\lambda E\left(t\right).$$

(4)

This paper proposes a ZNN model, dubbed ZNNFRDP, for the calculation of the time-varying pseudoinverse based on FRD.

These are the main contributions of this investigation:

• A new ZNN model, called ZNNFRDP, for the calculation of the time-varying pseudoinverse based on FRD is introduced and investigated. Keep in mind that ZNNFRDP stands for ZNN model based on FRD for the calculation of pseudoinverse.
• Theoretical analysis supported by five numerical experiments show that the ZNNFRDP model performs as well as, if not better than, other well-performing ZNN models in the computation of the time-varying pseudoinverse.

The remainder of the paper’s layout is comprised of Section 2, Section 3, Section 4 and Section 5. In particular, Section 2 defines and analyzes the ZNNFRDP model for computing the time-varying pseudoinverse, while Section 3 demonstrates and investigates the model’s computational effectiveness. The concluding observations and inferences are given in Section 5.

2. Materials and Methods

In this section, the ZNN model, termed ZNNFRDP, that computes the time-varying pseudoinverse based on FRD is introduced and analyzed. Assume that $A\left(t\right)\in {\mathbb{R}}^{m\times n}$ is a differentiable time-varying matrix of constant rank r. In order to guarantee that the ZNNFRDP model applies FRD, the unique FRD presented in Proposition 1 is employed. In particular, consider that $p\in {\mathbb{N}}^r$ is a vector that indicates the pivot columns of $A\left(t\right)$. Then, creating $C\left(t\right)=A\left(:,p\right)\left(t\right)\in {\mathbb{R}}^{m\times r}$ and $F\left(t\right)\in {\mathbb{R}}^{r\times n}$ as indicating in Proposition 1, and setting $Y\left(t\right)={\left(C^{\mathrm{T}}\left(t\right)C\left(t\right)\right)}^{-1}\in {\mathbb{R}}^{r\times r}$, $Z\left(t\right)={\left(F\left(t\right)F^{\mathrm{T}}\left(t\right)\right)}^{-1}\in {\mathbb{R}}^{r\times r}$ and $X\left(t\right)=A^{†}\left(t\right)\in {\mathbb{R}}^{n\times m}$ as indicating in proposition 2, the equations of the next group are satisfied in the case of pseudoinverse based on FRD:

$$\left\{\begin{array}{c}A\left(t\right)=C\left(t\right)F\left(t\right),\hfill \\ \left(C^{\mathrm{T}}\left(t\right)C\left(t\right)\right)Y\left(t\right)=I_r,\hfill \\ Z\left(t\right)\left(F\left(t\right)F^{\mathrm{T}}\left(t\right)\right)=I_r,\hfill \\ X\left(t\right)=F^{\mathrm{T}}\left(t\right)Z\left(t\right)Y\left(t\right)C^{\mathrm{T}}\left(t\right).\hfill \end{array}\right.$$

(5)

The ZNNFRDP model takes into account determining the four unknowns $F\left(t\right),Y\left(t\right),Z\left(t\right)$ and $X\left(t\right)$ of (5). Notice that the elements below the major diagonal of $F\left(t\right)$ are zeros since it is a partitioned matrix of the reduced row echelon form of $A\left(t\right)$. Additionally, $Y\left(t\right)$ and $Z\left(t\right)$ are symmetric invertible matrices because $C\left(t\right)$ and $F\left(t\right)$ are full rank matrices. It is significant to note that when $A\left(t\right)$ is differentiable, $C\left(t\right)$ and $F\left(t\right)$ are also differentiable.

For zeroing (5), we set the next error matrix equations group (EMEG):

$$\left\{\begin{array}{c}{E}_1\left(t\right)=A\left(t\right)-C\left(t\right)F\left(t\right),\hfill \\ E_2\left(t\right)=\left(C^{\mathrm{T}}\left(t\right)C\left(t\right)\right)Y\left(t\right)-I_r,\hfill \\ E_3\left(t\right)=Z\left(t\right)\left(F\left(t\right)F^{\mathrm{T}}\left(t\right)\right)-I_r,\hfill \\ E_4\left(t\right)=X\left(t\right)-F^{\mathrm{T}}\left(t\right)Z\left(t\right)Y\left(t\right)C^{\mathrm{T}}\left(t\right).\hfill \end{array}\right.$$

(6)

Furthermore, the first derivative of EMEG (6) is:

$$\left\{\begin{array}{cc}{\dot{E}}_1\left(t\right)=\hfill & \dot{A}\left(t\right)-\dot{C}\left(t\right)F\left(t\right)-C\left(t\right)\dot{F}\left(t\right),\hfill \\ {\dot{E}}_2\left(t\right)=\hfill & ({\dot{C}}^{\mathrm{T}}\left(t\right)C\left(t\right)+C^{\mathrm{T}}\left(t\right)\dot{C}\left(t\right))Y\left(t\right)+\left(C^{\mathrm{T}}\left(t\right)C\left(t\right)\right)\dot{Y}\left(t\right),\hfill \\ {\dot{E}}_3\left(t\right)=\hfill & \dot{Z}\left(t\right)\left(F\left(t\right)F^{\mathrm{T}}\left(t\right)\right)+Z\left(t\right)(\dot{F}\left(t\right)F^{\mathrm{T}}\left(t\right)+F\left(t\right){\dot{F}}^{\mathrm{T}}\left(t\right)),\hfill \\ {\dot{E}}_4\left(t\right)=\hfill & \dot{X}\left(t\right)-{\dot{F}}^{\mathrm{T}}\left(t\right)Z\left(t\right)Y\left(t\right)C^{\mathrm{T}}\left(t\right)-F^{\mathrm{T}}\left(t\right)\dot{Z}\left(t\right)Y\left(t\right)C^{\mathrm{T}}\left(t\right)\hfill \\ & -F^{\mathrm{T}}\left(t\right)Z\left(t\right)\dot{Y}\left(t\right)C^{\mathrm{T}}\left(t\right)-F^{\mathrm{T}}\left(t\right)Z\left(t\right)Y\left(t\right){\dot{C}}^{\mathrm{T}}\left(t\right).\hfill \end{array}\right.$$

(7)

When $\dot{E}\left(t\right)$ from (4) is replaced with ${\dot{E}}_i\left(t\right),i=1,\dots ,4,$ defined in (7) and the equation is solved in terms of $\dot{F}\left(t\right),\dot{Y}\left(t\right),\dot{Z}\left(t\right),\dot{X}\left(t\right)$, the result below is obtained:

$$\left\{\begin{array}{cc}\hfill & -C\left(t\right)\dot{F}\left(t\right)=-\lambda E_1\left(t\right)-\dot{A}\left(t\right)+\dot{C}\left(t\right)F\left(t\right),\hfill \\ \hfill & \left(C^{\mathrm{T}}\left(t\right)C\left(t\right)\right)\dot{Y}\left(t\right)=-\lambda E_2\left(t\right)-({\dot{C}}^{\mathrm{T}}\left(t\right)C\left(t\right)+C^{\mathrm{T}}\left(t\right)\dot{C}\left(t\right))Y\left(t\right),\hfill \\ \hfill & \dot{Z}\left(t\right)\left(F\left(t\right)F^{\mathrm{T}}\left(t\right)\right)+Z\left(t\right)(\dot{F}\left(t\right)F^{\mathrm{T}}\left(t\right)+F\left(t\right){\dot{F}}^{\mathrm{T}}\left(t\right))=-\lambda E_3\left(t\right),\hfill \\ \hfill & \dot{X}\left(t\right)-{\dot{F}}^{\mathrm{T}}\left(t\right)Z\left(t\right)Y\left(t\right)C^{\mathrm{T}}\left(t\right)-F^{\mathrm{T}}\left(t\right)\dot{Z}\left(t\right)Y\left(t\right)C^{\mathrm{T}}\left(t\right)-F^{\mathrm{T}}\left(t\right)Z\left(t\right)\dot{Y}\left(t\right)C^{\mathrm{T}}\left(t\right)\hfill \\ & =-\lambda E_4\left(t\right)+F^{\mathrm{T}}\left(t\right)Z\left(t\right)Y\left(t\right){\dot{C}}^{\mathrm{T}}\left(t\right).\hfill \end{array}\right.$$

(8)

As shown below, the techniques of Kronecker product and vectorization are used to simplify the dynamic model of (8):

$$\left\{\begin{array}{cc}\hfill & -(I_n\otimes C\left(t\right))\mathrm{vec}\left(\dot{F}\left(t\right)\right)=\mathrm{vec}\left(-\lambda E_1\left(t\right)-\dot{A}\left(t\right)+\dot{C}\left(t\right)F\left(t\right)\right),\hfill \\ \hfill & (I_r\otimes C^{\mathrm{T}}\left(t\right)C\left(t\right))\mathrm{vec}\left(\dot{Y}\left(t\right)\right)=\mathrm{vec}\left(-\lambda E_2\left(t\right)-({\dot{C}}^{\mathrm{T}}\left(t\right)C\left(t\right)+C^{\mathrm{T}}\left(t\right)\dot{C}\left(t\right))Y\left(t\right)\right),\hfill \\ \hfill & (F\left(t\right)F^{\mathrm{T}}\left(t\right)\otimes I_r)\mathrm{vec}\left(\dot{Z}\left(t\right)\right)+(F\left(t\right)\otimes Z\left(t\right))\mathrm{vec}\left(\dot{F}\left(t\right)\right)+(I_r\otimes Z\left(t\right)F\left(t\right))\mathrm{vec}\left({\dot{F}}^{\mathrm{T}}\left(t\right)\right)\hfill \\ & =\mathrm{vec}\left(-\lambda E_3\left(t\right)\right),\hfill \\ \hfill & \mathrm{vec}\left(\dot{X}\left(t\right)\right)-({\left(Z\left(t\right)Y\left(t\right)C^{\mathrm{T}}\left(t\right)\right)}^{\mathrm{T}}\otimes I_n)\mathrm{vec}\left({\dot{F}}^{\mathrm{T}}\left(t\right)\right)-(C\left(t\right)Y^{\mathrm{T}}\left(t\right)\otimes F^{\mathrm{T}}\left(t\right))\mathrm{vec}\left(\dot{Z}\left(t\right)\right)\hfill \\ & -(C\left(t\right)\otimes F^{\mathrm{T}}\left(t\right)Z\left(t\right))\mathrm{vec}\left(\dot{Y}\left(t\right)\right)=\mathrm{vec}\left(-\lambda E_4\left(t\right)+F^{\mathrm{T}}\left(t\right)Z\left(t\right)Y\left(t\right){\dot{C}}^{\mathrm{T}}\left(t\right)\right).\hfill \end{array}\right.$$

(9)

To create a straightforward dynamical model that can quickly calculate the $F\left(t\right)$, $Y\left(t\right)$, $Z\left(t\right)$, and $X\left(t\right)$, (9) must be simplified even more. As a consequence, the Lemma 1 based on [36] regarding vectorization is presented.

Lemma 1.

For $F\in {\mathbb{R}}^{r\times n}$, let $\mathrm{vec}\left(F\right)\in {\mathbb{R}}^{rn}$ denote the matrix F vectorization. What follows holds:

$$\mathrm{vec}\left(F^{\mathrm{T}}\right)=Q\mathrm{vec}\left(F\right),$$

(10)

where $Q\in {\mathbb{R}}^{rn\times rn}$ be a fixed permutation matrix depended solely by the number of rows r and columns n of matrix F.

The next Algorithm 1 shows an algorithmic process for computing the permutation matrix Q in (10) that refers to a $r\times n$ matrix.

Algorithm 1. Matrix Q calculation.

Require: The number of columns n and rows r of matrix $F\in {\mathbb{R}}^{r\times n}$. 1: procedurePermut_Mat_Q($r,n$) 2:     Set $c=$eye$\left(rn\right)$ and $d=$reshape$(1:rn,n,r)$ 3:     return $Q=c(:,$reshape$(d^{\mathrm{T}},1,rn))$ 4: end procedure Ensure: The matrix Q.

Moreover, since all the elements below the major diagonal of $F\left(t\right)$ are zeros, just the elements of $\dot{F}\left(t\right)$ located in the major diagonal and above must be obtained. Therefore, it is significant to use a vector denoted $\dot{\mathbf{f}}\left(t\right)$ to replace $\dot{F}\left(t\right)$, by stacking these elements of $F\left(t\right)$ into $\dot{\mathbf{f}}\left(t\right)$. The dimension of (9) is lessened in this way, while $F\left(t\right)$ is forced to have all of its elements below the major diagonal zeros. Therefore, by using the $w={\sum }_{i=n-r+1}^{n}i$ elements on and above the major diagonal of $F\left(t\right)$, it is possible to obtain the following equation that replaces $\mathrm{vec}\left(\dot{F}\left(t\right)\right)$ in (9):

$$\mathrm{vec}\left(\dot{F}\left(t\right)\right)=G\dot{\mathbf{f}}\left(t\right),$$

(11)

where $G\in {\mathbb{R}}^{rn\times w}$ is an operational matrix which can be calculated applying the algorithmic procedure shown Algorithm 2. Further, the elements on and above the major diagonal of $\mathrm{vec}\left(\dot{F}\left(t\right)\right)$ are piled to create the column vector $\dot{\mathbf{f}}\left(t\right)\in {\mathbb{R}}^w$.

Algorithm 2. Matrix G calculation.

Require: The number of columns n and rows r of matrix $F\in {\mathbb{R}}^{r\times n}$. 1: procedureOpe_Mat_G($r,n$) 2:       Set $w=$sum$(n-r+1:n)$ and $G=$zeros$(rn,w)$ 3:       for $q=1:rn$ do 4:             Set $j=$floor$\left(\frac{q-1}{r}\right)+1$ and $i=$mod$(q-1,r)+1$ 5:             if $i\le j$ then 6:                    Set $G(q,j+$sum$(n-(1:i-1\left)\right)(i>1))=1$ 7:             end if 8:       end for 9:       return G 10: end procedure Ensure: The matrix G.

Additionally, because $Y\left(t\right)$ and $Z\left(t\right)$ are symmetric matrices, we only have to obtain their elements that are placed on and above its major diagonal. Thus, it is significant to use vectors denoted by $\dot{\mathbf{y}}\left(t\right)$ and $\dot{\mathbf{z}}\left(t\right)$ in place of $\dot{Y}\left(t\right)$ and $\dot{Z}\left(t\right)$, respectively, by stacking these elements of $Y\left(t\right)$ and $Z\left(t\right)$ into the vectors $\dot{\mathbf{y}}\left(t\right)$ and $\dot{\mathbf{z}}\left(t\right)$, respectively. The dimension of (9) is lessened in this way, while $Y\left(t\right)$ and $Z\left(t\right)$ are forced to be symmetric matrices. Therefore, using the $h=(r^2+r)/2$ elements on and above the major diagonals of $Y\left(t\right)$ and $Z\left(t\right)$, it is possible to obtain the following equations that replace $\mathrm{vec}\left(\dot{Y}\left(t\right)\right)$ and $\mathrm{vec}\left(\dot{Z}\left(t\right)\right)$ in (9):

$$\mathrm{vec}\left(\dot{Y}\left(t\right)\right)=R\dot{\mathbf{y}}\left(t\right),\mathrm{vec}\left(\dot{Z}\left(t\right)\right)=R\dot{\mathbf{z}}\left(t\right),$$

(12)

where $R\in {\mathbb{R}}^{r^2\times h}$ is an operational matrix which can be calculated applying the algorithmic procedure shown Algorithm 3. Further, the elements on and above the major diagonal of $\mathrm{vec}\left(\dot{Y}\left(t\right)\right)$ and $\mathrm{vec}\left(\dot{Z}\left(t\right)\right)$ are piled to create the column vectors $\dot{\mathbf{y}}\left(t\right),\dot{\mathbf{z}}\left(t\right)\in {\mathbb{R}}^h$.

Using the permutation matrix Q and the operational matrices G and R, (9) can be rewritten as follows:

$$\left\{\begin{array}{cc}\hfill & -(I_n\otimes C\left(t\right))G\mathrm{vec}\left(\dot{\mathbf{f}}\left(t\right)\right)=\mathrm{vec}\left(-\lambda E_1\left(t\right)-\dot{A}\left(t\right)+\dot{C}\left(t\right)F\left(t\right)\right),\hfill \\ \hfill & (I_r\otimes C^{\mathrm{T}}\left(t\right)C\left(t\right))R\mathrm{vec}\left(\dot{\mathbf{y}}\left(t\right)\right)=\mathrm{vec}\left(-\lambda E_2\left(t\right)-({\dot{C}}^{\mathrm{T}}\left(t\right)C\left(t\right)+C^{\mathrm{T}}\left(t\right)\dot{C}\left(t\right))Y\left(t\right)\right),\hfill \\ \hfill & (F\left(t\right)F^{\mathrm{T}}\left(t\right)\otimes I_r)R\mathrm{vec}\left(\dot{\mathbf{z}}\left(t\right)\right)+(F\left(t\right)\otimes Z\left(t\right))G\mathrm{vec}\left(\dot{\mathbf{f}}\left(t\right)\right)+(I_r\otimes Z\left(t\right)F\left(t\right))QG\mathrm{vec}\left(\dot{\mathbf{f}}\left(t\right)\right)\hfill \\ & =\mathrm{vec}\left(-\lambda E_3\left(t\right)\right),\hfill \\ \hfill & \mathrm{vec}\left(\dot{X}\left(t\right)\right)-({\left(Z\left(t\right)Y\left(t\right)C^{\mathrm{T}}\left(t\right)\right)}^{\mathrm{T}}\otimes I_n)QG\mathrm{vec}\left(\dot{\mathbf{f}}\left(t\right)\right)-(C\left(t\right)Y^{\mathrm{T}}\left(t\right)\otimes F^{\mathrm{T}}\left(t\right))R\mathrm{vec}\left(\dot{\mathbf{z}}\left(t\right)\right)\hfill \\ & -(C\left(t\right)\otimes F^{\mathrm{T}}\left(t\right)Z\left(t\right))R\mathrm{vec}\left(\dot{\mathbf{y}}\left(t\right)\right)=\mathrm{vec}\left(-\lambda E_4\left(t\right)+F^{\mathrm{T}}\left(t\right)Z\left(t\right)Y\left(t\right){\dot{C}}^{\mathrm{T}}\left(t\right)\right).\hfill \end{array}\right.$$

(13)

Then, by setting

$$\begin{array}{l}{k}_1\left(t\right)=-(I_n\otimes C\left(t\right))G,k_2\left(t\right)=(I_r\otimes C^{\mathrm{T}}\left(t\right)C\left(t\right))R,k_4\left(t\right)=(F\left(t\right)F^{\mathrm{T}}\left(t\right)\otimes I_r)R,\\ k_3\left(t\right)=(F\left(t\right)\otimes Z\left(t\right))G+(I_r\otimes Z\left(t\right)F\left(t\right))QG,k_5\left(t\right)=-({\left(Z\left(t\right)Y\left(t\right)C^{\mathrm{T}}\left(t\right)\right)}^{\mathrm{T}}\otimes I_n)QG,\\ k_6\left(t\right)=-(C\left(t\right)\otimes F^{\mathrm{T}}\left(t\right)Z\left(t\right))R,k_7\left(t\right)=-(C\left(t\right)Y^{\mathrm{T}}\left(t\right)\otimes F^{\mathrm{T}}\left(t\right))R,\\ \mathbf{K}\left(t\right)=\left[\begin{array}{cccc}{k}_1\left(t\right)& {\mathbf{0}}_{mn\times h}& {\mathbf{0}}_{mn\times h}& {\mathbf{0}}_{mn\times mn}\\ {\mathbf{0}}_{r^2\times w}& k_2\left(t\right)& {\mathbf{0}}_{r^2\times h}& {\mathbf{0}}_{r^2\times mn}\\ k_3\left(t\right)& {\mathbf{0}}_{r^2\times h}& k_4\left(t\right)& {\mathbf{0}}_{r^2\times mn}\\ k_5\left(t\right)& k_6\left(t\right)& k_7\left(t\right)& I_{mn}\end{array}\right],\mathbf{a}\left(t\right)=\left[\begin{array}{c}\mathrm{vec}\left(F\left(t\right)\right)\\ \mathrm{vec}\left(Y\left(t\right)\right)\\ \mathrm{vec}\left(Z\left(t\right)\right)\\ \mathrm{vec}\left(X\left(t\right)\right)\end{array}\right],\\ \dot{\mathbf{a}}\left(t\right)=\left[\begin{array}{c}\mathrm{vec}\left(\dot{F}\left(t\right)\right)\\ \mathrm{vec}\left(\dot{Y}\left(t\right)\right)\\ \mathrm{vec}\left(\dot{Z}\left(t\right)\right)\\ \mathrm{vec}\left(\dot{X}\left(t\right)\right)\end{array}\right],\mathbf{b}\left(t\right)=\left\{\begin{array}{c}\mathrm{vec}\left(-\lambda E_1\left(t\right)-\dot{A}\left(t\right)+\dot{C}\left(t\right)F\left(t\right)\right),\hfill \\ \mathrm{vec}\left(-\lambda E_2\left(t\right)-({\dot{C}}^{\mathrm{T}}\left(t\right)C\left(t\right)+C^{\mathrm{T}}\left(t\right)\dot{C}\left(t\right))Y\left(t\right)\right),\hfill \\ \mathrm{vec}\left(-\lambda E_3\left(t\right)\right),\hfill \\ \mathrm{vec}\left(-\lambda E_4\left(t\right)+F^{\mathrm{T}}\left(t\right)Z\left(t\right)Y\left(t\right){\dot{C}}^{\mathrm{T}}\left(t\right)\right),\hfill \end{array}\right.\end{array}$$

(14)

the ZNN model below, termed ZNNFRDP, is proposed for calculating the time-varying pseudoinverse based on FRD of an arbitrary input matrix $A\left(t\right)\in {\mathbb{R}}^{m\times n}$:

$${\mathbf{K}}^{\mathrm{T}}\left(t\right)\mathbf{K}\left(t\right)\dot{\mathbf{a}}\left(t\right)={\mathbf{K}}^{\mathrm{T}}\left(t\right)\mathbf{b}\left(t\right),$$

(15)

which can be solved successfully in MATLAB using an ode solver. Note that $\dot{\mathbf{a}}\left(t\right),\mathbf{a}\left(t\right)\in {\mathbb{R}}^{(w+2h+mn)}$, $\mathbf{b}\left(t\right)\in {\mathbb{R}}^{(2r^2+2mn)}$ and $\mathbf{K}\left(t\right)\in {\mathbb{R}}^{(2r^2+2mn)\times (w+2h+mn)}$, while ${\mathbf{K}}^{\mathrm{T}}\left(t\right)\mathbf{K}\left(t\right)\in {\mathbb{R}}^{(w+2h+mn)\times (w+2h+mn)}$ is a non-singular mass matrix. In accordance with Theorem 1, the ZNNFRDP model (15) converges to the theoretical solution (TSol) of the time-varying pseudoinverse.

Algorithm 3. Matrix R calculation.

Require: A real symmetric matrix’s column or row number r. 1: procedureOpe_Mat_R(r) 2:       Put $h=(r^2+r)/2$ and $R=$zeros$(r^2,h)$ 3:       for $w=1:r^2$ do 4:             Set $d=$mod$(w-1,r)+1$ and $c=$floor$\left(\frac{w-1}{r}\right)+1$ 5:             if $c\ge d$ then 6:                  Put $R\left(w,d+c\frac{c-1}{2}\right)=1$ 7:            else 8:                 Put $R\left(w,c+d\frac{d-1}{2}\right)=1$ 9:            end if 10:       end for 11:       return R 12: end procedure Ensure: The matrix R.

Theorem 1.

Let $A\left(t\right)\in {\mathbb{R}}^{m\times n}$ be a differentiable matrix of rank r and $p\in {\mathbb{N}}^r$ be a vector that contains the numbers of all pivot columns of $A\left(t\right)$. Starting from any initial price $\mathbf{a}\left(0\right)$, the ZNNFRDP model (15) exponentially converges to the TSol ${\mathbf{a}}^{\ast }\left(t\right)$ at each time $t\in [0,t_f)\subseteq [0,+\infty )$. Furthermore, the TSol of the time-varying pseudoinverse is the last $mn$ elements of ${\mathbf{a}}^{\ast }\left(t\right)$.

Proof. 

The EMEG is declared as in (6) in order to determine the TSol of the time-varying pseudoinverse. The model (8) is developed using the linear ZNN design (4) for zeroing (6). When $t\to \infty$, every equation in the EMEG (8) converges to the TSol starting from any initial price, according to [24] [Theorem 1]. As a result, the ZNNFRDP model (15) also converges to the TSol ${\mathbf{a}}^{\ast }\left(t\right)$ starting from any initial price $\mathbf{a}\left(0\right)$ when $t\to \infty$ since it is really an alternate version of (8). In accordance with (14), the TSol of the time-varying pseudoinverse is the last $mn$ elements of ${\mathbf{a}}^{\ast }\left(t\right)$. Therefore, the proof is finished.    □

Additionally, diagram presentation can be used to describe the above-mentioned ZNNFRDP model process for real-time solution of the time-varying pseudoinverse of matrix $A\left(t\right)$. For the block diagram presentation, the dynamical Equation (15) can be transformed as follows:

$$\dot{\mathbf{a}}\left(t\right)=(I_{(w+2h+mn)}-{\mathbf{K}}^{\mathrm{T}}\left(t\right)\mathbf{K}\left(t\right))\dot{\mathbf{a}}\left(t\right)+{\mathbf{K}}^{\mathrm{T}}\left(t\right)\mathbf{b}\left(t\right),$$

(16)

and the corresponding block diagram of the ZNNFRDP model (16) is shown in Figure 1.

Lastly, the complexity of producing and solving (16) contributes to the ZNNFRDP model’s overall computational complexity. The computational complexity of producing (16) is $\mathcal{O}\left({(w+2h+mn)}^2\right)$ operations since each iteration of the equation has ${(w+2h+mn)}^2$ multiplication and $w+2h+mn$ addition operations. Considering that a linear system of equations is solved at each step by an implicit solver such as ode15s, the complexity of solving (16) is $\mathcal{O}({(w+2h+mn)}^3$ because it involves a $(w+2h+mn)\times (w+2h+mn)$ matrix. Therefore, the ZNNFRDP model’s overall computational complexity is $\mathcal{O}\left({(w+2h+mn)}^3\right)$.

3. Results

The performance of the ZNNFRDP (15) model in five numerical experiments (NEs), which involve calculating time-varying pseudoinverses of rank deficient matrices, is examined in this section. Furthermore, the ZNNFRDP model is compared to two additional ZNN models, namely ZNNP and ZNNSVDP, which are presented in [11]. It is worth mentioning that both the ZNNP and ZNNSVDP models calculate the pseudoinverse of an arbitrary matrix, with the ZNNSVDP model employing singular value decomposition to do so. With the exception of $X\left(0\right)$, where all-ones matrices have been set, the initial prices have been set to matrices with ones on the major diagonal and elsewhere zeros for $F\left(0\right),Y\left(0\right)$ and $Z\left(0\right)$ in all NEs. Furthermore, in the denotation of the input matrix, the functions $\eta \left(t\right)=cos\left(t\right)$ and $\theta \left(t\right)=sin\left(t\right)$ have been used. Last, the MATLAB solver ode15s is employed for the computations in all NEs within the time range $[0,10]$ and with the parameter $\lambda =10$. Because of this, all ZNN models find the real-time solution of the time-varying pseudoinverse of a given matrix $A\left(t\right)$ starting at $t=0$ and ending at $t=10$.

3.1. Experiment 1

Assume the following $3\times 3$ input matrix of rank 2 with $p=[1,2]$:

$$A\left(t\right)=\left[\begin{array}{ccc}3+\eta \left(2t\right)& \theta \left(t\right)+1& \theta \left(t\right)+1\\ 2+\theta \left(2t\right)& \eta \left(t\right)-20& \eta \left(t\right)-20\\ 2+\theta \left(2t\right)& \eta \left(t\right)-20& \eta \left(t\right)-20\end{array}\right].$$

Presuming that $C\left(t\right)=A\left(:,p\right)\left(t\right)$, the findings of the ZNNFRDP model are displayed in Figure 2a–c and Figure 3a,b.

3.2. Experiment 2

Assume the following $4\times 7$ input matrix of rank 3 with $p=[1,2,4]$:

$$A\left(t\right)=\left[\begin{array}{ccccccc}1& 1& 0& 1& 0& 1& 1\\ 1& 1& 0& 0& 1& 0& 1\\ 1& 0& 1& 1& 0& 1& 1\\ 1& 0& 1& 0& 1& 0& 1\end{array}\right]\odot \frac{1}{t+1}.$$

Presuming that $C\left(t\right)=A\left(:,p\right)\left(t\right)$, the findings of the ZNNFRDP model are displayed in Figure 2d–f and Figure 3c,d.

3.3. Experiment 3

Assume the following $4\times 4$ input matrix of rank 1 with $p=1$:

$$A\left(t\right)=[5-\eta \left(\pi t\right),3+\theta \left(\pi t\right),-4-\eta \left(\pi t\right),1+3\theta \left(\pi t\right)]\odot {\mathbf{1}}_{4\times 4}.$$

Presuming that $C\left(t\right)=A\left(:,p\right)\left(t\right)$, the findings of the ZNNFRDP model are displayed in Figure 2g–i and Figure 3e,f.

3.4. Experiment 4

Assume the following $m\times n$ input matrix of rank 1 with $p=1$, $m=20$ and $n=10$:

$$A\left(t\right)=[2+\theta \left(t\right),2+1/2\theta \left(t\right),2+1/3\theta \left(t\right),\cdots ,2+1/n\theta \left(t\right)]\odot {\mathbf{1}}_{m\times n}.$$

Presuming that $C\left(t\right)=A\left(:,p\right)\left(t\right)$, the findings of the ZNNFRDP model are displayed in Figure 3g,h and Figure 4a–c.

3.5. Experiment 5

Assume the following $2\times 3$ input matrix of rank 2 with $p=[1,2]$:

$$A\left(t\right)=\left[\begin{array}{ccc}4& 3& 1\\ 4& 3+{10}^{-20}& 1\end{array}\right]+\theta \left(t\right)\odot {\mathbf{1}}_{2\times 3}.$$

It is worth noting that the matrix A is close to a loss of rank, i.e., its singular values are $7.2111$ and $4.6962·{10}^{-18}$. Such an experiment will show how the ZNNFRDP model converges in the case of ill-conditioned matrices. Presuming that $C\left(t\right)=A\left(:,p\right)\left(t\right)$, the findings of the ZNNFRDP model are displayed in Figure 3i,j and Figure 4d–f.

4. Discussion

This section first presents the performance of the ZNNFRDP model for computing time-varying pseudoinverses before comparing it to the performance of the ZNNP and ZNNSVDP models.

The performance of the ZNNFRDP model for computing time-varying pseudoinverses is investigated through NEs in Section 3.1–Section 3.5, with the results shown in Figure 2 and Figure 4. Particularly, the EMEG convergence in NEs in Section 3.1–Section 3.5 is measured by the Frobenius norm in Figure 2a,d,g and Figure 4a,d, respectively. These figures demonstrate how the EMEG’s $E_i\left(t\right),i=1,2,3$ begin at $t=0$ with a high error value above ${10}^0$ and converge before $t=2$ in a low error value in the range of $[{10}^{-13},{10}^{-2}]$. As opposed to this, the EMEG’s $E_4\left(t\right)$ always begins at $t=0$ with a low error value under ${10}^{-3}$ and converges before $t=2$ in an even lower error value in the range of $[{10}^{-13},{10}^{-4}]$. That is, the EMEG starts from a non-optimal initial price and converges to the zero matrix finally. The trajectories of $F\left(t\right),Y\left(t\right)$ and $Z\left(t\right)$ in NEs in Section 3.1–Section 3.5 are shown in Figure 2b,e,h and Figure 4b,e, respectively, while the trajectories of $X\left(t\right)$ are shown in Figure 2c,f,i and Figure 4c,f, respectively. All of these images show that while the trajectories of $X\left(t\right)$ match those of the TSol, a certain pattern appears in the trajectories of $F\left(t\right),Y\left(t\right),Z\left(t\right)$ and $X\left(t\right)$ before $t=2$.

In general, employing the FRD to compute the pseudoinverse as suggested in (2) will increase rounding errors and result in a loss of precision when using conventional computational methods. However, our study uses the ZNN method, which is considerably different from the conventional computational methods. In fact, for example, we are not calculating the inverse of any matrix by using conventional textbook methods. Numerous papers and books (written in the last two decades) on the ZNN method demonstrate how effective the ZNN design is in attempting to zero out rounding errors by defining an appropriate EMEG (see (6)). It is important to mention that a rounding error is the difference between a rounded-off numerical value and the actual value. Considering that the EMEG’s actual solution is the zero matrix, any EMEG’s element that is not zero during the computations is regarded as a rounding error. As a result, the actual task of the ZNN method is to zero the rounding errors.

It is important to note that the steady-state errors in Figure 2 and Figure 3 are of order ${10}^{-5}$ in the majority of situations because double precision arithmetic ($eps=2.22·{10}^{-16}$) is used by default in MATLAB. Keep in mind that the symbol $eps$ stands for the machine epsilon, an upper bound on the relative approximation error brought on by rounding in floating point arithmetic. Therefore, in order to achieve lower steady-state errors, a higher error tolerance in ode15s must be chosen. Furthermore, the time of convergence, which in the NEs is close to $t=2$, and the value of $\lambda$ are correlated. To decrease the time of convergence, a larger value of $\lambda$ must be chosen [24]. Therefore, the major factor used to rate the ZNN models’ overall performance is the their convergence performance under the same value of $\lambda$.

The performance of the ZNNFRDP, ZNNP and ZNNSVDP models is compared by measuring the errors caused in the Moore-Penrose equations (i)–(iv) from (1) through NEs in Section 3.1–Section 3.5, with the results shown in Figure 3. Particularly, the performance of the ZNNFRDP and ZNNP models is compared through NEs in Section 3.1 and Section 3.2. The results of the ZNNFRDP and ZNNP models are shown in Figure 3a,b, where we observe that in NE in Section 3.1, the errors caused in the Moore-Penrose equations (i)–(iii) and (ii)–(iv), respectively, are shown in Figure 3a,b. These figures demonstrate how these errors begin at $t=0$ with a high value above ${10}^0$ and converge before $t=2$ in a low error value in the range of $[{10}^{-6},{10}^{-3}]$. The overall performance of the ZNNFRDP model in NE in Section 3.1 is almost identical to that of the ZNNP. In NE in Section 3.2, the errors caused in the Moore-Penrose equations (i)–(iii) and (ii)–(iv), respectively, are shown in Figure 3c,d. These figures demonstrate how these errors begin at $t=0$ with a high value above ${10}^0$ and converge before $t=2$ in a low error value in the range of $[{10}^{-10},{10}^{-3}]$. The convergence performance of the ZNNFRDP model in NE in Section 3.2 is slightly better than that of the ZNNP, while their overall performances are almost identical. As a result, the ZNNFRDP model performed slightly better than the ZNNP in NEs in Section 3.1 and Section 3.2.

Additionally, the performance of the ZNNFRDP and ZNNSVDP models is compared through NEs in Section 3.3–Section 3.5. It is important to note that NE in Section 3.4 compares the performances of these models in a high-dimensional input matrix with dimensions $20\times 10$, whereas NE in Section 3.5 compares their performance in an ill-conditioned input matrix. In NE in Section 3.3, the errors caused in the Moore-Penrose equations (i)–(iii) and (ii)–(iv), respectively, are shown in Figure 3e,f. These figures demonstrate how the errors caused in the Moore-Penrose equations (i), (ii) and (iv) begin at $t=0$ with a high value above ${10}^0$ and converge before $t=2$ in a low error value in the range of $[{10}^{-7},{10}^{-3}]$, whereas the error caused in the Moore-Penrose Equation (i) begins at $t=0$ with a lower value and converges before $t=2$ in a lower error value in the range of $[{10}^{-11},{10}^{-5}]$. The convergence performance of the ZNNFRDP model in NE in Section 3.3 is slightly better than that of the ZNNSVDP, while their overall performances are close. In NEs in Section 3.4 and Section 3.5, the errors caused in the Moore-Penrose equations (i)–(iii), respectively, are shown in Figure 3a,b, while the errors caused in the Moore-Penrose equations (ii)–(iv), respectively, are shown in Figure 3a,b. These figures demonstrate how the errors caused in the Moore-Penrose equations (ii) and (iv) begin at $t=0$ with a high value above ${10}^0$ and converge before $t=2$ in a low error value in the range of $[{10}^{-7},{10}^{-3}]$, where the convergence performance of the ZNNFRDP model is slightly worse than that of the ZNNSVDP, while the overall performance of the ZNNFRDP model is slightly better than that of the ZNNSVDP. The error caused in the Moore-Penrose Equation (i) begin at $t=0$ with a lower value and converge before $t=2$ in a lower error value in the range of $[{10}^{-5},{10}^{-2}]$. The convergence and the overall performance of the ZNNFRDP models are almost identical. The error caused in the Moore-Penrose Equation (iii) begin at $t=0$ with a lower value and converge before $t=2$ in a lower error value in the range of $[{10}^{-12},{10}^{-5}]$. The convergence and the overall performance of the ZNNFRDP model are much better than that of the ZNNSVDP. As a result, the ZNNFRDP model performed better than the ZNNSVDP in NEs in Section 3.3–Section 3.5.

Finally, it was demonstrated in Section 2 that the ZNNFRDP model has an overall computational complexity of $\mathcal{O}\left({(w+2h+mn)}^3\right)$, whereas [11] claims that the ZNNP model has an overall computational complexity of $\mathcal{O}\left({\left(mn\right)}^3\right)$ and the ZNNSVDP model has an overall computational complexity of $\mathcal{O}\left({(m^2+n^2+r+mn)}^3\right)$. Therefore, the ZNNFRDP model has a lower overall computational complexity than the ZNNSVDP model, which is an advantage, but a higher overall computational complexity than the ZNNP model, which is a disadvantage.

To summarize, the following can be deduced from this section’s NEs:

• The convergence of the ZNNFRDP model starts from a non-optimal initial price and converge in a small period of time to the zero matrix.
• The errors caused in the Moore-Penrose equations (i)–(iv) and the $F\left(t\right),Y\left(t\right),Z\left(t\right)$ and $X\left(t\right)$ solutions trajectories act consistently due to the EMEGs’ proclivity toward convergence.
• The EMEGs will converge more quickly when the price of the design parameter $\lambda$ is greater.
• The ZNNFRDP model performs as well as, if not better than, the ZNNP and ZNNSVDP models.
• In essence, the time-varying pseudoinverses are calculated with exceptional and efficient performance by the ZNNFRDP model.

5. Conclusions

In this paper, the issue of calculating the time-varying pseudoinverse based on FRD using the ZNN method was examined. Therefore, a novel ZNN model called ZNNFRDP that uses the FRD to compute the time-varying pseudoinverse was proposed and explored in this research. Five NEs show that the ZNNFRDP model’s EMEG has a tendency to converge exponentially to the zero matrix after beginning from a non-optimal initial price. Additionally, they show that the ZNNFRDP model performs as well as, if not better than, other well-performing ZNN models in the computation of the time-varying pseudoinverse. Thus, theoretical analysis and numerical results both supported the effectiveness of the proposed model.

The list below includes some possible research topics.

• To hasten the convergence of the ZNN dynamics, nonlinear activation functions [37,38] may be used.
• It may be possible to use ZNN designs with terminal convergence [39,40] to produce new improved ZNNFRDP models.