Efficient Algorithm for the Computation of the Solution to a Sparse Matrix Equation in Distributed Control Theory

Abstract

In this short communication, an algorithm for efficiently solving a sparse matrix equation, which arises frequently in the field of distributed control and estimation theory, is proposed. The efficient algorithm stems from the fact that the sparse equation at hand can be reduced to a system of linear equations. The proposed algorithm is shown to require significantly fewer floating point operations than the state-of-the-art solution. The proposed solution is applied to a real-life example, which models a wide range of industrial processes. The experimental results show that the solution put forward allows for a significant increase in efficiency in relation to the state-of-the-art solution. The significant increase in efficiency of the presented algorithm allows for a valuable widening of the applications of distributed estimation and control.

1. Notation

The $n\times o$ null matrix and the null vector of dimension n are denoted by ${\mathbf{0}}_{n\times o}$ and ${\mathbf{0}}_n$, respectively. The i-th component of a vector $\mathbf{v}$ is denoted by ${\left[\mathbf{v}\right]}_i$, and the entry $(i,j)$ of a matrix $\mathbf{A}$ is denoted by ${\left[\mathbf{A}\right]}_{i,j}$. The vectorization of a matrix $\mathbf{A}$, denoted herein by $\mathrm{vec}\left(\mathbf{A}\right)$, returns a vector composed of the concatenated columns of $\mathbf{A}$. The Kronecker product of two matrices $\mathbf{A}$ and $\mathbf{B}$ is denoted by $\mathbf{A}\otimes \mathbf{B}$. The cardinality of a set $\chi$ is denoted by $\left|\chi \right|$.

2. Introduction

Sparse matrix equations arise in various real-life problems in a broad range of engineering fields. Thus, given their impact, they have been extensively studied for a long time [1]. In recent developments in the field of distributed estimation and control theory, a sparse matrix equation has arisen, for which, to the best of the knowledge of the authors, an efficient algorithm for its solution has not yet been proposed.

Consider the matrix equation

$$\left\{\begin{array}{cc}{\left[\mathbf{A}\mathbf{X}\mathbf{B}-\mathbf{C}\right]}_{i,j}=0\hfill & ,(i,j)\in \chi \hfill \\ {\left[\mathbf{X}\right]}_{i,j}=0\hfill & ,(i,j)\notin \chi \hfill \end{array}\right.,$$

(1)

where $\mathbf{A}\in {\mathbb{R}}^{n\times n}$, $\mathbf{B}\in {\mathbb{R}}^{o\times o}$, and $\mathbf{C}\in {\mathbb{R}}^{n\times o}$ are known and $\mathbf{X}\in {\mathbb{R}}^{n\times o}$ is the unknown. Let $\mathbf{E}\in {\mathbb{R}}^{n\times o}$ represent a known sparsity pattern, and define the set $\chi$ of integer pairs of the form $(i,j)$ to index the nonzero entries of $\mathbf{E}$ as

$$\left\{\begin{array}{cc}(i,j)\in \chi \hfill & ,{\left[\mathbf{E}\right]}_{i,j}\ne 0\hfill \\ (i,j)\notin \chi \hfill & ,\mathrm{otherwise}\hfill \end{array}\right.,i=1,\cdots ,n,j=1,\cdots ,o.$$

(2)

Equation (1) arises frequently in the field of distributed control and estimation theory. In fact, it is the backbone of the state-of-the-art methods for distributed filter design [2] and controller synthesis [3] for large-scale dynamical systems. However, the solution to (1), presented in these papers, relies on an inefficient closed-form matrix computation. Define a matrix $\mathbf{Z}$, such that the vector $\mathbf{Z}\mathrm{vec}\left(\mathbf{E}\right)$ contains all the nonzero elements of $\mathbf{E}$. The solution to (1) presented in the literature is given by

$$\mathrm{vec}\left(\mathbf{X}\right)={\mathbf{Z}}^T{\left(\mathbf{Z}\left(\mathbf{B}\otimes \mathbf{A}\right){\mathbf{Z}}^{\mathbf{T}}\right)}^{-1}\mathbf{Z}\mathrm{vec}\left(\mathbf{C}\right),$$

(3)

as proven in ([2], Appendix B). Note that (3) requires the multiplication of a matrix of dimension $\left|\chi \right|\times no$ by other dimension $no\times no$, which requires $\mathcal{O}\left(\right|\chi \left|{\left(no\right)}^2\right)$ floating point operations. The number of practical applications, where it is convenient to model the interconnection of complex systems as a whole network, has been steadily increasing [4]. They have emerged in a broad range of engineering fields such as irrigation networks [5,6], power systems networks [7,8], traffic networks [9,10,11,12], and industrial processes [13]. Nevertheless, given the substantial increase in memory and processing power needed to control such systems, the classical control solutions cannot be implemented in practice. That is why, over the past decades, an effort has been made towards the development of distributed and computationally efficient algorithms to handle such systems. In this short communication, an algorithm to obtain an efficient and exact solution to (1) is proposed, which significantly increases the applicability of distributed methods that address the control and estimation problems for large-scale dynamical systems.

3. Materials and Methods

The following result is the basis for the algorithm proposed in this short communication.

Theorem 1.

The system of Equation (1) can be reduced to a system of linear equations of dimension $\left|\chi \right|$,

$$\mathbf{S}\mathbf{x}=\mathbf{r},$$

(4)

where $\mathbf{S}\in {\mathbb{R}}^{\left|\chi \right|\times \left|\chi \right|}$, $\mathbf{r}\in {\mathbb{R}}^{\left|\chi \right|}$, and $\mathbf{x}\in {\mathbb{R}}^{\left|\chi \right|}$ is the vector of the ordered nonzero entries of $\mathrm{vec}\left(\mathbf{X}\right)$.

Proof. 

For every integer pair $(i,j)\in \chi$, the first equation of (1) is given by

$$\sum _{k=1}^n\sum _{l=1}^o{\left[\mathbf{A}\right]}_{i,k}{\left[\mathbf{X}\right]}_{k,l}{\left[\mathbf{B}\right]}_{l,j}-{\left[\mathbf{C}\right]}_{i,j}=0.$$

(5)

Given the second equation of (1), one can rewrite (5) as

$$\sum _{(k,l)\in \chi }{\left[\mathbf{A}\right]}_{i,k}{\left[\mathbf{X}\right]}_{k,l}{\left[\mathbf{B}\right]}_{l,j}-{\left[\mathbf{C}\right]}_{i,j}=0.$$

(6)

Denote by $\mathbf{x}\in {\mathbb{R}}^{\left|\chi \right|}$ the vector of the ordered nonzero entries of $\mathrm{vec}\left(\mathbf{X}\right)$, and let $\mathit{\varphi }={\left[{\varphi }_1{\varphi }_2\right]}^T:{\mathbb{N}}^1\to {\mathbb{N}}^2$ denote the resulting one-to-one map between the entries of $\mathbf{x}$ and the nonzero entries of $\mathbf{X}$. Defining $p:={\mathit{\varphi }}^{-1}(i,j)$ and $q:={\mathit{\varphi }}^{-1}(k,l)$, it follows that satisfying (6) for every integer pair $(i,j)\in \chi$ is equivalent to satisfying $\mathbf{S}\mathbf{x}=\mathbf{r}$, where $\mathbf{S}\in {\mathbb{R}}^{\left|\chi \right|\times \left|\chi \right|}$ and $\mathbf{r}\in {\mathbb{R}}^{\left|\chi \right|}$ are defined such that

$${\left[\mathbf{S}\right]}_{p,q}={\left[\mathbf{A}\right]}_{{\varphi }_1\left(p\right),{\varphi }_1\left(q\right)}{\left[\mathbf{B}\right]}_{{\varphi }_2\left(q\right),{\varphi }_2\left(p\right)},$$

(7)

$p=1,\dots ,\left|\chi \right|,q=1,\dots ,\left|\chi \right|$; and

$${\left[\mathbf{r}\right]}_p={\left[\mathbf{C}\right]}_{\varphi \left(p\right)},p=1,\dots ,\left|\chi \right|;$$

(8)

respectively. It then follows that the solution to (1) is

$${\left[\mathbf{X}\right]}_{i,j}=\left\{\begin{array}{cc}{\left[\mathbf{x}\right]}_{{\varphi }^{-1}(i,j)}\hfill & ,(i,j)\in \chi \hfill \\ 0\hfill & ,(i,j)\notin \chi \hfill \end{array}\right.,$$

(9)

where $\mathbf{x}$ is the solution of (4). □

Corollary 1.

It follows immediately from Theorem 1 that (1) admits: (i) one and only one solution if $\mathbf{S}$ is full rank; (ii) infinitely many solutions if (4) is consistent and $\mathbf{S}$ is not full rank; and (iii) no solution if (4) is inconsistent.

Proposition 1.

Algorithm 1 can be used to obtain the solution to (1), with $\mathcal{O}\left(\right|\chi {|}^3)$ floating point operations.

Proof. 

Algorithm 1 computes $\mathbf{S}\in {\mathbb{R}}^{\left|\chi \right|\times \left|\chi \right|}$ and $\mathbf{r}\in {\mathbb{R}}^{\left|\chi \right|}$ according to (7) and (8), respectively. It, then, follows from Theorem 1 that (1) can be reduced to the solution to $\mathbf{S}\mathbf{x}=\mathbf{r}$. The LU decomposition of $\mathbf{S}$ can be carried out with $\mathcal{O}\left(\right|\chi {|}^3)$ floating point operations, using Gaussian elimination. The type of solution is then assessed according to Corollary 1. If the solution is unique, it follows from Theorem 1 that it is given by (9), which is implemented in Algorithm 1. □

Algorithm 1 Algorithm to efficiently compute the exact solution to (1)

Remark 1.

It is important to point out that each of the three main steps of Algorithm 1 can be carried out in parallel. The first and last steps for loops of Algorithm 1 (which are commented on) can be carried out in parallel, for each j. In this case, the variable p has to be initialized, for each j, with the number of nonzero entries of $\mathbf{E}$ in the first $j-1$ columns. Moreover, the LU decomposition can also be carried out in parallel [14,15].

Remark 2.

There are asymptotically faster algorithms for computing the LU decomposition of a square matrix. For instance, the Strassen algorithm [16] can perform the LU decomposition of $\mathbf{S}$ in $\mathcal{O}\left(\right|\chi {|}^{{log}_{2}7})$ floating point operations. Moreover, the Coppersmith–Winograd algorithm [17] requires only $\mathcal{O}\left(\right|\chi {|}^{2.38})$ floating point operations. However, an improvement in relation to the Strassen algorithm can only be noticed for large values of $\left|\chi \right|$. For more details, see ([18], Section 1.4).

Remark 3.

In the field of distributed estimation and control theory, $\left|\chi \right|$ is the sum of the number of sensor output signals available to each node of the network, and n is the order of the state–space dynamics of the whole system network. Given that, for most applications, each node has a similar order and a similar number of sensor output signals available, then $\left|\chi \right|\approx cn$, where $c\in \mathbb{N}$ is a constant. The system detailed in Section 4 illustrates this claim. It, thus, follows that the ratio of the number of floating points operations required by the proposed algorithm and the number required by the state-of-the-art methods is $\mathcal{O}\left(o^{-2}\right)$.

4. Results

In this section, the solution to the sparse matrix equation (1) is used to design a distributed filter for a large-scale network of N tanks. The computational efficiency of the synthesis with the proposed solution is compared to the state-of-the-art synthesis procedure. This network is widely studied [3,19,20] because it is analogous to a broad range of industrial processes. Consider N interconnected tanks, as shown in Figure 1, where N is an even integer. The water level of tank i is denoted by $h_i$. The network is actuated by $N/2$ pumps, which are controlled by the lower tanks, whose inputs are denoted by $u_i$ for $i=1,\cdots ,N/2$, in accordance with the schematic. Each pump is connected to a three-way valve that regulates the fraction of the flow, held constant, that goes to each of the tanks supplied by the pump. Each tank has a sensor, which measures its water level. Making use of mass balances and Bernoulli’s law, the system dynamics, in the absence of noise, are given by

$$\left\{\begin{array}{c}{A}_i{\dot{h}}_i\left(t\right)=-a_i\sqrt{2g{h}_i\left(t\right)}+a_{\frac{N}{2}+i}\sqrt{2g{h}_{\frac{N}{2}+i}\left(t\right)}+{\gamma }_i{k}_i{u}_i\left(t\right),i=1,\cdots ,N/2\hfill \\ A_i{\dot{h}}_i\left(t\right)=-a_i\sqrt{2g{h}_i\left(t\right)}+(1-{\gamma }_{i-\frac{N}{2}-1})k_{i-\frac{N}{2}-1}{u}_{i-\frac{N}{2}-1}\left(t\right),i=\frac{N}{2}+2,\cdots ,N\hfill \\ A_{\frac{N}{2}+1}{\dot{h}}_{\frac{N}{2}+1}\left(t\right)=-a_{\frac{N}{2}+1}\sqrt{2g{h}_{\frac{N}{2}+1}\left(t\right)}+(1-{\gamma }_{\frac{N}{2}})k_{\frac{N}{2}}{u}_{\frac{N}{2}}\left(t\right),\hfill \end{array}\right.,$$

(10)

where $A_i$ and $a_i$ are the cross sections of tank i and of its outlet hole, respectively; the constant ${\gamma }_i$ represents the fraction of the flow that passes through the valve i to the lower tanks; $k_i$ is the constant of proportionality between the mass flow and the input of pump i; and g denotes the acceleration of gravity.

The values of the constant parameters of the network are presented in

Table 1. Values of the physical constants of the N tanks network.

Constant	Value
$A_i$, i odd	$28{\mathrm{cm}}^2$
$A_i$, i even	$32{\mathrm{cm}}^2$
$a_i$, i odd $\le N/2$	$0.071{\mathrm{cm}}^2$
$a_i$, i even $\le N/2$	$0.057{\mathrm{cm}}^2$
$a_i$, $i>N/2$	$0.040{\mathrm{cm}}^2$
g	$981\mathrm{cm}{\mathrm{s}}^{-2}$
$k_i$	$3.33{\mathrm{cm}}^3{\mathrm{s}}^{-1}{\mathrm{V}}^{-1}$
${\gamma }_i$, i odd	$0.7$
${\gamma }_i$, i even	$0.6$

. The dynamics of the network are nonlinear, thus the system is linearized about its operational point, which is selected to be $h\left(i\right)=15$ cm, $i=1,\dots ,N$. The distributed filter is synthesized with the finite-horizon method ([2], Section 5), which is shown to have better state estimation performance in relation to other distributed filter synthesis methods. Each iteration k of this method computes a new gain, $\mathbf{K}\left(k\right)\in {\mathbb{R}}^{n\times o}$, solving

$$\left\{\begin{array}{cc}{\left[\Lambda (k+1)\mathbf{K}\left(k\right)\mathbf{S}\left(k\right)-\Lambda (k+1)\mathbf{P}\left(k\right|k-1){\mathbf{C}}^T\right]}_{i,j}=0\hfill & ,(i,j)\in \chi \hfill \\ {\left[\mathbf{K}\left(k\right)\right]}_{i,j}=0\hfill & ,(i,j)\notin \chi \hfill \end{array}\right.,$$

(11)

where $\Lambda (k+1)\in {\mathbb{R}}^{n\times n}$, $\mathbf{S}\left(k\right)\in {\mathbb{R}}^{o\times o}$, and $\mathbf{P}\left(k\right|k-1)\in {\mathbb{R}}^{n\times n}$ are defined in [2], with $n=o=N$. Moreover, $\mathbf{C}={\mathbf{I}}_N$ is the output matrix of the network and, given the configuration of the network, the sparsity pattern that defines the set $\chi$ is given by $\mathbf{E}={\mathbf{I}}_N$. Note that (11) has the same form as (1), for which a novel efficient solution is proposed in this short communication.

The goal is to compare the computational efficiency of the gain synthesis using the solution to (11) proposed in this short communication and compare it to the state-of-the-art solution (3). In that sense, the gain synthesis is performed for both alternatives, for a range of N. Figure 2 depicts the elapsed wall-clock time for the distributed filter synthesis, resulting from the average of three simulations of a MATLAB implementation on a single thread of a server with 24GB RAM and 24 Intel Xeon hexa-core CPUs at 2.40GHz. A MATLAB implementation of the distributed filter synthesis, of the application to the N tanks network, and of Algorithm 1, can be found in the DECENTER toolbox available at https://decenter2021.github.io (accessed on 23 June 2021). Figure 2 also depicts the linear regression in the logarithmic scale of the asymptotic evolution, which is considered to be achieved for $N>40$. First, if the solution of (11) is the bottleneck of the gain synthesis procedure, one would expect that the synthesis wall-clock time grows asymptotically with $\mathcal{O}\left(N^5\right)$ for the state-of-the-art solution, and $\mathcal{O}\left(N^3\right)$ for the novel solution proposed in this short communication. It is possible to conclude from Figure 2 that this is, in fact, the case. The experimental complexity is slightly lower than what was projected, since MATLAB makes use of efficient LU decomposition algorithms like the ones pointed out in Remark 2. Second, it is also possible to confirm that the use of the solution proposed in this short communication leads to a ratio of wall-clock synthesis time between the proposed solution and the state-of-the-art solution that evolves with $\mathcal{O}\left(N^{-2.035}\right)$. This improvement significantly widens the applicability of various distributed methods to large-scale systems.

5. Conclusions

In this short communication, an algorithm for efficiently solving the sparse matrix equation (1), which arises frequently in the field of distributed control and estimation theory, is proposed. The state-of-the-art methods for distributed filter design and controller synthesis rely on a very inefficient computation of the solution to (1), requiring $\mathcal{O}\left(\right|\chi \left|{\left(no\right)}^2\right)$ floating point operations. In this letter, an algorithm is proposed to compute the solution to this equation with $\mathcal{O}\left(\right|\chi {|}^3)$ floating point operations. For distributed estimation and control theory applications, the ratio of the number of floating points operations required by the proposed algorithm and the number required by the state-of-the-art methods is $\mathcal{O}\left(o^{-2}\right)$. Furthermore, the algorithm proposed in this short communication was applied to an example representative of real-life large-scale industrial processes, and was compared to the state-of-the-art solution. It is shown that the solution put forward allows for a decrease of the computational complexity of the gain synthesis from $\mathcal{O}\left(N^{4.805}\right)$, which is obtained for the state-of-the-art solution, to $\mathcal{O}\left(N^{2.770}\right)$, where N is the dimension of the network. The significantly greater efficiency achieved with the proposed algorithm allows for the state-of-the-art distributed control and estimation algorithms to be implemented in an online framework and for large-scale dynamical systems, which would otherwise be unfeasible.