Next Article in Journal
Stability Switching in Lotka-Volterra and Ricker-Type Predator-Prey Systems with Arbitrary Step Size
Next Article in Special Issue
A Modified Simulated Annealing (MSA) Algorithm to Solve the Supplier Selection and Order Quantity Allocation Problem with Non-Linear Freight Rates
Previous Article in Journal
A Numerical Approach of Handling Fractional Stochastic Differential Equations
Previous Article in Special Issue
Mathematical Model to Calculate Heat Transfer in Cylindrical Vessels with Temperature-Dependent Materials
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Efficiency of Orthogonal Matching Pursuit for Group Sparse Recovery

1
College of Science, North China University of Science and Technology, Tangshan 063210, China
2
School of Sciences, Tianjin Chengjian University, Tianjin 300384, China
3
School of Mathematical Sciences and LPMC, Nankai University, Tianjin 300071, China
*
Author to whom correspondence should be addressed.
Axioms 2023, 12(4), 389; https://doi.org/10.3390/axioms12040389
Submission received: 10 March 2023 / Revised: 12 April 2023 / Accepted: 14 April 2023 / Published: 17 April 2023
(This article belongs to the Special Issue Applied Mathematical Modeling and Optimization)

Abstract

:
We propose the Group Orthogonal Matching Pursuit (GOMP) algorithm to recover group sparse signals from noisy measurements. Under the group restricted isometry property (GRIP), we prove the instance optimality of the GOMP algorithm for any decomposable approximation norm. Meanwhile, we show the robustness of the GOMP under the measurement error. Compared with the P-norm minimization approach, the GOMP is easier to implement, and the assumption of γ -decomposability is not required. The simulation results show that the GOMP is very efficient for group sparse signal recovery and significantly outperforms Basis Pursuit in both scalability and solution quality.

1. Introduction

The problem of recovering sparse signals from incomplete measurements has been studied extensively in the field of compressed sensing. Many important results have been obtained, see refs. [1,2,3,4,5]. In this article, we consider the problem of recovering group sparse signals. We first recall some basic notions and results.
Assume that x R n is an unknown signal, and the information we gather about x R n can be described by
y = Φ x + e ,
where Φ R m × n is the encoder matrix and y R m is the information vector with a noise level e 2 ϵ . To recover x from y , we use a decoder Δ , which is a mapping from R m to R n , and denote
x * : = Δ ( y ) = Δ ( Φ x + e )
as our approximation of x .
The l 1 -norm minimization, also known as Basis Pursuit [2], is one of the most popular decoder maps. It is defined as follows:
Δ BP ( y ) : = arg min z R n z 1 , s . t . y Φ z 2 ϵ .
Candès and Tao in Ref. [6] introduced the restricted isometry property (RIP) of a measurement matrix as follows:
A matrix Φ is said to satisfy the RIP of order k if there exists a constant δ ( 0 , 1 ) such that, for all k-sparse signals x ,
( 1 δ ) x 2 2 Φ x 2 ( 1 + δ ) x 2 2 .
In particular, the minimum of all δ satisfying the above inequality is defined as the isometry constant δ k .
Based on the RIP of the encoder matrix Φ , it has been shown that the solution to (2) recovers x exactly provided that x is sufficiently sparse [7]. We denote by Σ k the set of all k-sparse signals, i.e.,
Σ k : = { z R n : | supp ( z ) | k } ,
where supp ( z ) is the set of i for which z i 0 and | A | is the cardinality of the set A . Let σ k ( x , · 1 ) be the sparsity index of x from Σ k , which is defined by
σ k ( x , · 1 ) : = inf z Σ k x z 1 .
In Ref. [7], the authors obtained the estimation of the residual error
Δ BP ( y ) x 2 C 0 k σ k ( x , · 1 ) + C 1 ϵ ,
where C 0 , C 1 are constants depending only on the encoder matrix Φ but not on x or e. In many current papers on sparse recovery, the l 1 -norm objective function in (2) has been changed to several other norms.
At about the same time, the research community began to propose that the number of nonzero components of a signal might not be the only reasonable measure of the sparsity of a signal. Alternate notions under the broad umbrella of “group sparsity” and “group sparse recovery” began to appear. For more detail, one can see Ref. [8]. It has been shown that group sparse signals are used widely in EEG [9], wavelet image analysis [10], gene analysis [11], multi-channel image analysis [12] and other fields.
We next recall a result on the recovering of the group sparse signals from Ref. [13]. In that paper, Ahsen and Vidyasagar estimated x from y by solving the following optimization problem:
Δ P ( y ) : = arg min z R n z P , s . t . y Φ z 2 ϵ ,
where · P is the penalty norm. We begin with recalling some definitions from [13]. We denote the set { 1 , 2 , , n } by the symbol [ n ] . The group structure G = { G 1 , G 2 , , G g } is a partition of [ n ] . Here, we suppose that | G i | k for all 1 i g . A signal x R n is said to be group k-sparse if its support set supp(x) is contained in a group k-sparse set, which is defined as follows:
Definition 1.
A subset Λ [ n ] is said to be group k-sparse if Λ = G S : = i S G i for some S { 1 , 2 , , g } and | Λ | k .
We use GkS to denote the set of all group k-sparse subsets of [ n ] .
Definition 2.
We say a norm · on R n is decomposable if x , y R n , satisfy supp ( x ) G S , supp ( y ) G T , with S , T being disjoint subsets of { 1 , 2 , , g } , then one has x + y = x + y .
It is clear that l 1 -norm is decomposable. More examples of decomposable norms can be found in Ref. [13].
In the processing of recovering an unknown signal x R n , the approximation norm on R n will be denoted by · A . We assume that the approximation norm is decomposable.
Definition 3.
If there exists γ ( 0 , 1 ] such that for any x , y R n with supp ( x ) G S , supp ( y ) G T , and S , T are disjoint subsets of { 1 , 2 , , g } , it is true that x + y x + γ y , we say the norm · is γ-decomposable.
Note that when γ = 1 , γ -decomposability coincides with decomposability.
For any index set A [ n ] , let x A denote the part of the signal x consisting of coordinates falling on A . The sparsity index and the optimal decomposition of a signal in R n are defined respectively as follows:
Definition 4.
The group k-sparse index of a signal x R n with respect to the norm · and the group structure G is defined by
σ k , G ( x , · ) : = min Λ GkS x x Λ = min Λ GkS x Λ c .
For x R n and a norm · on R n , if there exist Λ i GkS for i = 0 , 1 , 2 , , s , such that
x Λ 0 c = x x Λ 0 = min Λ GkS x x Λ = σ k , G ( x )
and
x Λ i c = min Λ GkS x j = 0 i 1 x Λ j x Λ , i = 1 , 2 , , s ,
then we call { x Λ 0 , x Λ 1 , , x Λ s } an optimal group k-sparse decomposition of x .
Definition 5.
A matrix Φ R m × n is said to satisfy the group restricted isometry property (GRIP) of order k if there exists a constant δ k ( 0 , 1 ) , such that
1 δ k min Λ GkS min supp ( z ) Λ Φ z 2 2 z 2 2 max Λ GkS max supp ( z ) Λ Φ z 2 2 z 2 2 1 + δ k .
Then, we are ready to state the result of Ahsen and Vidyasagar. We need the following constants:
a : = min Λ GkS min x Λ 0 x Λ P x Λ A , b : = max Λ GkS max x Λ 0 x Λ P x Λ A , f : = k
and
c : = min Λ GkS min x Λ 0 x Λ A x Λ 2 , d : = max Λ GkS max x Λ 0 x Λ A x Λ 2 .
Theorem 1.
Suppose that
1. The norm · A is decomposable.
2. The norm · P is γ-decomposable for some γ ( 0 , 1 ] .
3. The matrix Φ satisfies GRIP of order 2 k with constant δ 2 k .
4. Suppose the “compressibility condition"
δ 2 k < f a γ / b d 2 + f a γ / b d
holds, then
Δ P ( Φ x + e ) x 2 D 1 σ k , G ( x ) A + D 2 ϵ .
Furthermore,
Δ P ( Φ x + e ) x A D 3 σ k , G ( x ) A + D 4 ϵ ,
where
D 1 = b ( 1 + γ ) a γ f · 1 + ( 2 1 ) δ 2 k 1 ( 1 + 2 b d / a γ f ) δ 2 k , D 2 = 2 ( 1 + b d / a γ f ) · 1 + δ 2 k 1 ( 1 + 2 b d / a γ f ) δ 2 k
and
D 3 = b ( 1 + γ ) a γ · 1 + ( 2 d / f 1 ) δ 2 k 1 ( 1 + 2 b d / a γ f ) δ 2 k , D 4 = 2 ( 1 + b d / a γ ) · 1 + δ 2 k 1 ( 1 + 2 b d / a γ f ) δ 2 k .
Recently, Ranjan and Vidyasagar presented sufficient conditions for (2) to achieve robust group sparse recovery by establishing a group robust null space property [14]. They derived the residual error bounds for the l p -norm for p [ 1 , 2 ] .
In compressed sensing, an alternative decoder for (1) is Orthogonal Matching Pursuit (OMP), which was originally proposed by J. Tropp in [15]. It is defined as Algorithm 1:
Algorithm 1 Orthogonal Matching Pursuit (OMP)
Input: measurement matrix Φ , measurement vector y .
Initialization: S 0 = , x 0 = 0 .
Iteration: repeat until a stopping criterion is met at n = n ¯ :
    S n + 1 = S n { j n + 1 } , j n + 1 : = arg max j [ n ] { | Φ * ( y Φ x n ) ) j | } ,
    x n + 1 = arg min z R n { y Φ z 2 , supp ( z ) S n + 1 } .
Output: the n ¯ -sparse vector x # = x n ¯ .
The major advantage of the OMP algorithm is that it is easy to implement, see Refs. [16,17,18,19,20,21]. In [22], under the RIP, T. Zhang proved that OMP with k iterations can exactly recover k-sparse signals and the procedure is also stable in l 2 under measurement noise. Based on this result, Xu showed that the OMP can achieve instance optimality under the RIP, cf. [23]. In [24], Cohen, Dahmen and DeVore extended T. Zhang’s result to the general context of k-term approximation from a dictionary in arbitrary Hilbert spaces. They have shown that OMP generates near-best k-term approximations under a similar RIP condition.
In this article, we will generalize the OMP to recover the group sparse signals. Assume that the signal x is group k-sparse with the group structure G , then the number of non-zero groups of x = ( x G 1 , x G 2 , , x G g ) is no more than k . We define
α : = max j [ g ] | G j | .
It is obvious that α k .
The group support set of x with respect to G is defined as follows:
Gsupp ( x ) : = { j [ g ] : x G j 0 } .
Let | Gsupp ( x ) | be the number of distinct groups on which x is supported. For any i { 1 , 2 , , g } , the subvector (submatrix) x G i ( Φ G i ) can also be written as x [ i ] ( Φ [ i ] ) .
Now we start to propose the Group Orthogonal Matching Pursuit (GOMP) algorithm. Assume that the group structure G is given a prior. Let Ω be the initial feature set of the group sparse signal x with the maximum allowed sparsity M. The GOMP is defined as Algorithm 2:
Algorithm 2 Group Orthogonal Matching Pursuit ( GOMP M ( y ) )
Input: encoding matrix Φ , the vector y , group structure { G i } i = 1 g , a set Ω { 1 , 2 , , g } , maximum allowed sparsity M.
Initialization: x 0 = arg min x ˜ : Gsupp ( x ˜ ) Ω y Φ x ˜ 2 , r 0 = y Φ x 0 , l = 1 , I 0 = Ω .
 
while l k M do
 
    Set i l = arg max i Φ * [ i ] r l 1 2 .
 
    Identity I l = { i 1 , i 2 , , i l } Ω .
 
    Update x l = arg min x ˜ : Gsupp ( x ˜ ) I l y Φ x ˜ 2 , r l = y Φ x l , l = l + 1 .
 
end while
Output: x ^ GOMP = x k .
The GOMP M ( y ) algorithm begins by initializing the residual as r 0 = y , and at the lth ( l 1 ) iteration, we chose one group index which matched r l 1 best. However, the algorithm does not obtain a least-square minimization over the group index sets that have already been selected, but updates the residual and proceeds to the next iteration.
Then, we studied the efficiency of the GOMP. We show the instance optimality and robustness of the GOMP algorithm under the GRIP of the matrix Φ . We formulate them in the following theorem.
Theorem 2.
Suppose that the norm · A is decomposable and 0 < δ 1 . If the GRIP condition δ k α + ( 1 + δ ) δ β k α δ holds with β : = 16 + 15 δ , then for any signal x and any permutation e with e 2 ϵ , the solution x * : = GOMP ( β 1 ) k ( Φ x + e ) obeys
x * x 2 D 1 σ k , G ( x ) A + D 2 ϵ ,
where
D 1 = 1 c [ ( 1 + δ ) ( 1 + 11 + 20 δ ) + 1 ] and D 2 = 1 + δ ( 1 + 11 + 20 δ ) .
Furthermore,
x * x A D 3 σ k , G ( x ) A + D 4 ϵ ,
where
D 3 = d c 2 β α 1 ( 1 + δ ) ( 1 + 11 + 20 δ ) + 1 and D 4 = d ( 2 β α 1 ) ( 1 + δ ) ( 1 + 11 + 20 δ ) .
When e = 0 , from inequalities (6) and (8), we have
x * x 2 D 1 σ k , G ( x ) A
and
x * x A D 3 σ k , G ( x ) A .
This implies the instance optimality of the GOMP. In other words, from the viewpoint of the error, the GOMP is almost the best. The robustness of the GOMP means that the signal can be recovered stably for different dimensions, sparsity levels, numbers of measurements and noise scenarios (which can be observed from the experiments in Section 3). Compared with Basis Pursuit (BP), our proposed algorithm performs far better, and runs extremely faster, especially when it comes large-scale problems.
Based on the observation of Theorems 1 and 2, one can see that the assumption of γ -decomposability is not required for the GOMP. Instead, we developed some new techniques to analyze the error performance of the GOMP.
We remark that the results of Theorem 2 are rather general. They cover some important cases.
Firstly, if n = i = 1 g d i and we take a special partition of [ n ] as
[ n ] = [ 1 , , d 1 G 1 , d 1 + 1 , , d 1 + d 2 G 2 , , n d g + 1 , , n G g ] ,
then the unknown sparse signals appear in a few blocks, i.e., the signals are block-sparse. It is obvious that our results include the case of the block-sparse signals. In practice, there are many forms of block-sparse signal, such as multi-band signal, DNA array, radar pulse signal and the multi-measurement vector problem and so on, see Refs. [25,26,27,28,29,30,31,32].
Secondly, applying Theorem 2 to the case of conventional sparsity, we obtain the following new results on the error estimates of the OMP.
Corollary 1.
(Conventional sparsity) Suppose that Φ satisfies the RIP condition δ k + ( 1 + δ ) δ β k δ with β : = 16 + 15 δ . Then,
OMP ( β 1 ) k ( Φ x + e ) x 2 D 1 σ k ( x ) A + D 2 ϵ ,
and
OMP ( β 1 ) k ( Φ x + e ) x A D 3 σ k ( x ) A + D 4 ϵ ,
where the constants D i are defined in (1.6) and (1.8) for i = 1 , 2 , 3 , 4 .
The remaining part of the paper is organized as follows. In Section 2, based on a corollary of Theorem 3, we prove Theorem 2, while the proof of Theorem 3 will be given in the Appendix B. In the Appendix A, we establish some preliminary results for the proof. In Section 3, we compare the Group Orthogonal Matching Pursuit (GOMP) with the Basis Pursuit (BP) by numerical experiments. In Section 4, we draw the conclusions of our study. Moreover, we mention some other competitive algorithms which we will study in the future.

2. Proof of Theorem 2

The proof of Theorem 2 is based on the estimation of the residual error of the GOMP algorithm. To establish such estimation, we use the restricted gradient optimal constant for group sparse signals.
Definition 6.
Given x ¯ R n and s > 0 , we define the restricted gradient optimal constant ε s ( x ¯ ) as the smallest non-negative value such that
| 2 Φ μ , Φ x ¯ y | ε s ( x ¯ ) μ 2
for all μ R n with | Gsupp ( μ ) | s .
Some estimates about ε s ( x ¯ ) have been given in Ref. [22], such as ε s ( x ¯ ) 2 s Φ * ( Φ x ¯ y ) ,   ε s ( x ¯ ) 2 Φ * ( Φ x ¯ y ) 2 , and so on.
The following theorem is our result of the residual error of the GOMP algorithm. Its proof is quite technical and is given in the Appendix B.
Theorem 3.
Let x ¯ R n and F ¯ = Gsupp ( x ¯ ) . If there exists s, such that
s | F ¯ Ω | + 4 | F ¯ \ Ω | 1 + δ α 1 δ s α ln 20 ( 1 + δ | F ¯ \ Ω | α ) 1 δ s α ,
then when k 4 | F ¯ \ Ω | 1 + δ α 1 δ s α ln 20 ( 1 + δ | F ¯ \ Ω | α ) 1 δ s α , we have
Φ x k y 2 2 Φ x ¯ y 2 2 + 2.5 ε s ( x ¯ ) 2 / ( 1 δ s α ) ,
and
x k x ¯ 2 6 ε s ( x ¯ ) / ( 1 δ s α ) .
Next, we will use Theorem 3 in the case of Ω being empty to prove the following corollary, which plays an important role in the proof of Theorem 2.
Corollary 2.
Let x ¯ R n , k ¯ = | Gsupp ( x ¯ ) | . Let 0 < δ 1 , β = 16 + 15 δ . If the GRIP condition δ k ¯ α + ( 1 + δ ) δ β k ¯ α δ holds, then for k = ( β 1 ) k ¯ , we have
Φ x k y 2 2 ( 11 + 20 δ ) Φ x ¯ y 2 2 .
Proof. 
If δ k ¯ α + ( 1 + δ ) δ β k ¯ α δ , then
1 1 + δ α 1 δ β k ¯ α 1 + δ k ¯ α 1 δ β k ¯ α 1 + δ .
Therefore, we take s = β k ¯ in (10). Since 0 < δ 1 , then for k = ( β 1 ) k ¯ , using the properties of rounding function and logarithm function, we have
k 4 k ¯ ( 1 + δ ) ln ( 20 ( 1 + δ ) ) 4 k ¯ 1 + δ α 1 δ β k ¯ α ln 20 1 + δ k ¯ α 1 δ β k ¯ α .
From Theorem 3 and inequality (11), we conclude
Φ x k y 2 2 Φ ( x ¯ ) y 2 2 + 2.5 ε β k ¯ ( x ¯ ) 2 / ( 1 δ β k ¯ α ) Φ ( x ¯ ) y 2 2 + 2.5 · 4 ( 1 + δ β k ¯ α ) Φ x ¯ y 2 2 / ( 1 δ β k ¯ α ) = Φ ( x ¯ ) y 2 2 + 10 1 + δ β k ¯ α 1 δ β k ¯ α Φ x ¯ y 2 2 Φ ( x ¯ ) y 2 + 10 ( 1 + 2 δ ) Φ x ¯ y 2 2 = ( 11 + 20 δ ) Φ x ¯ y 2 2 ,
where the second inequality holds, since ε β k ¯ ( x ¯ ) 2 1 + δ β k ¯ α Φ x ¯ y 2 .
Now, we prove Theorem 2 with the help of Corollary 2.
Proof of Theorem 2.
Let { x Λ 0 , x Λ 1 , x Λ 2 , , x Λ s } be an optimal group k-sparse decomposition of x . Taking x ¯ = x Λ 0 in Corollary 2.1, we obtain that
Φ x * y 2 11 + 20 δ Φ x Λ 0 y 2 11 + 20 δ Φ x Λ 0 Φ x e 2 11 + 20 δ ( Φ x Λ 0 Φ x 2 + e 2 ) ,
where x * : = GOMP ( β 1 ) k ( Φ x + e ) . Noting that | Gsupp ( x * x Λ 0 ) | β k and ( 1 + δ ) δ β k α δ δ k α , we have
( 1 + δ ) ( 1 δ β k α ) = 1 + δ ( 1 + δ ) δ β k α 1 + δ δ + δ k α 1 .
Combining (12) with (13), we conclude
x * x Λ 0 2 1 1 δ β k α Φ x * Φ x Λ 0 2 1 + δ ( Φ x * y 2 + Φ x Λ 0 y 2 ) 1 + δ ( Φ x * y 2 + Φ x Λ 0 y 2 ) 1 + δ ( 11 + 20 δ + 1 ) Φ x Λ 0 y 2 1 + δ ( 11 + 20 δ + 1 ) ( Φ x Λ 0 Φ x 2 + e 2 ) .
We now estimate the term Φ x Λ 0 Φ x 2 . It is clear that
Φ x Λ 0 Φ x 2 = Φ ( x Λ 0 x ) 2 = Φ ( i = 1 s x Λ i ) 2 = i = 1 s Φ x Λ i 2 i = 1 s Φ x Λ i 2 .
By the GRIP of the matrix Φ , we obtain
Φ x Λ 0 Φ x 2 1 + δ i = 1 s x Λ i 2 .
Using (5) and the decomposability of · A , we have
x x Λ 0 2 i = 1 s x Λ i 2 1 c i = 1 s x Λ i A = 1 c i = 1 s x Λ i A = 1 c x Λ 0 c A = 1 c σ k , G ( x ) A .
Combining inequalities (14)–(17), we obtain
x * x Λ 0 2 ( 1 + δ ) ( 1 + 11 + 20 δ ) σ k , G ( x ) A c + 1 + δ ( 1 + 11 + 20 δ ) e 2 .
Inequalities (17) and (18) imply that
x * x 2 x * x Λ 0 2 + x Λ 0 x 2 1 c [ ( 1 + δ ) ( 1 + 11 + 20 δ ) + 1 ] σ k , G ( x ) A + 1 + δ ( 1 + 11 + 20 δ ) e 2 ,
which leads to the bound in (6).
To derive inequality (8), we adopt a different strategy. Let h = x * x Λ 0 . Without loss of generality, we assume that Gsupp ( x * x Λ 0 ) = { 1 , 2 , , g } and | G 1 | | G 2 | | G g | . We construct a subset S 1 of { 1 , 2 , , g } as follows:
Note that | G 1 | α k . We picked 1 as an element of S 1 ; S 1 contains 2 if and only if | G 1 | + | G 2 | k . Inductively suppose we have constructed the set Ω k 1 = S 1 { 1 , 2 , , k 1 } , then k is an element of S 1 if and only if
i Ω k 1 { k } | G i | k .
By this method, we can construct a unique subset S 1 . Then, by using the same method, we can construct a unique subset:
S 2 { 1 , 2 , , g } S 1 .
Inductively suppose we have constructed subsets S 1 , , S k 1 , we can form a subset { 1 , 2 , , g } i = 1 k 1 S i , then we can construct a unique subset S k { 1 , 2 , , g } i = 1 k 1 S i . In this way, we can decompose { 1 , 2 , , g } as follows:
{ 1 , 2 , , g } = i = 1 l S i .
From the construction, we know that when l 1 , for i l ,
| G S i | > k 2 .
For any 1 i l ,
| G S i 1 | + | G S i | > k .
So we have
i = 1 l | G S i | > k 2 l ,
which is to say, β k α > k 2 l , so l < 2 β α . When l = 1 , we also have l < 2 β α .
Using the Cauchy–Schwarz inequality and the decomposability of · A , we have
h A = Σ i = 1 l h G S i A d Σ i = 1 l h G S i 2 d l ( Σ i = 1 l h G S i 2 2 ) 1 2 = d l h 2 d 2 β α 1 h 2 .
Combining (18) with (19), we have
x * x Λ 0 A d 2 β α 1 x * x Λ 0 2 d c 2 β α 1 ( 1 + δ ) ( 1 + 11 + 20 δ ) σ k , G ( x ) A + d ( 2 β α 1 ) ( 1 + δ ) ( 1 + 11 + 20 δ ) e 2 .
From the above inequality, we conclude
x * x A x * x Λ 0 A + x Λ 0 x A = x * x Λ 0 A + σ k , G ( x ) A [ d c 2 β α 1 ( 1 + δ ) ( 1 + 11 + 20 δ ) + 1 ] σ k , G ( x ) A + d ( 2 β α 1 ) ( 1 + δ ) ( 1 + 11 + 20 δ ) e 2 .
This leads to the bound in (8), and hence we complete the proof of Theorem 2. □

3. Simulation Results

In this section, we test the performance of the GOMP and present the results of our experiments. We first describe the details relevant to our experiments in Section 3.1. We demonstrate the effectiveness of the GOMP in Section 3.2. In Section 3.3, we compare the GOMP with Basis Pursuit to show the efficiency and scalability of our algorithm.

3.1. Implementation

For all experiments, we considered the following model. Suppose that x R N is an unknown N-dimensional signal and we wish to recover it by the given data
y = Φ x + e ,
where Φ R M × N is a known measurement matrix with M N and e is a noise. Furthermore, since M N , the column vectors of Φ are linearly dependent and the collection of these columns can be viewed as a redundant dictionary.
For arbitrary x , y R N , define
x , y = j = 1 N x j y j ,
and
x 2 = j = 1 N | x j | 2 1 / 2 ,
where x = ( x j ) j = 1 N and y = ( y j ) j = 1 N . Obviously, R N is a Hilbert space with the inner product · , · .
In the experiment, we set the measurement matrix Φ to be a Gaussian matrix where each entry is selected from the N ( 0 , M 1 ) distribution and the density function of this distribution is p ( x ) : = 1 2 π M e x 2 M / 2 . We executed the GOMP with the data vector y = Φ x .
To demonstrate the performance of signal-recovering algorithms, we use the mean square error (MSE) to measure the error between the real signal x and its approximant x ^ , which is defined as follows:
MSE = 1 N j = 1 N ( x j x ^ ) 2 .
For the Group Orthogonal Matching Pursuit (GOMP), we let Ω = { 1 , , g } as the initialization. In the experiments, we constructed input signals randomly by the following steps:
i.
Given a sparse level K;
ii.
Produce a group structure { G i } i = 1 g randomly, satisfying # G i K for each index i;
iii.
Randomly select a set S { 1 , , g } , such that # G S : = # { i S G i } = K ;
iv.
Let the set G S be the support, and produce a signal by random numbers from normal distribution N ( 0 , 1 ) .
Some of examples of the randomly constructed group 50-sparse signals in different dimensions can be found in Figure 1.
We set Basis Pursuit (BP) as the baseline for further comparison with the GOMP to analyze the latter’s properties. For the implementation of Basis Pursuit, we used the l 1 Magic toolbox (Open-sourced code: https://candes.su.domains/software/l1magic/#code, accessed on 15 February 2023) developed by E. Candès and J. Romberg in [33].

3.2. Effectiveness of the GOMP

Figure 2 shows the performance of the GOMP for an input signal in dimension N = 512 with group sparsity level K = 50 and number of measurements M = 200 under different noises, where the red line represents the original signal and the black squares represent the approximation. Figure 3 shows the performance of BP. The poor performance of BP can be easily observed when the noise occurs.
In general, the above example shows that the GOMP is very effective for signal recovering, i.e., it can recover the group sparse signal exactly. We will analyze the performance of the GOMP further in terms of the mean square error by a comparison with BP in the next subsection.
Figure 4 describes the situation in dimension d = 1024 under noise e = 0 . It displays the relation between the percentage (of 100 input signals) of the support that can be recovered correctly and the number N of measurements. Furthermore, to some extent, it shows how many measurements are necessary to recover the support of the input group K-sparse signal x R d with high probability. If the percentage equals 100%, it means that support of all the 100 input signals can be found, which implies the support of input signal can be exactly recovered. As expected, Figure 4 shows that when the group sparsity level K increases, it is necessary to increase the number N of measurements to guarantee signal recovery. Furthermore, we can find that the GOMP can recover the correct support of the input signal with high probability.

3.3. Comparison with Basis Pursuit

In order to demonstrate the efficiency and robustness of the Group Orthogonal Matching Pursuit (GOMP), we implement the Group Orthogonal Matching Pursuit (GOMP) and the Basis Pursuit (BP) to recover the group sparse signals for comparison.
We implemented the GOMP and BP for recovering the group sparse signals in dimension N = 1024 with number of measurements M = 500 under different noises to calculate the average mean square error (MSE) and the average running time by repeating the test 100 times. Figure 5 and Figure 6, respectively, show the MSE and running time of the GOMP, implying that the error and the running time of the GOMP will constantly increase with the increased K { 10 , 20 , , 100 } . The figure of the error of the GOMP in terms of MSE is about 10 31 (noiseless) and 10 7 (Gaussian noise), which means that the GOMP can effectively recover the group sparse signal. Meanwhile, the computational complexity of the GOMP is also relatively low according to its running time. Figure 7 and Figure 8 show that the MSE and the running time of the BP have the same trend as the GOMP: the error and the running time increase with the increased sparsity level, without changes to other parameters, while the MSE of BP is higher in figure when compared with the GOMP, which is around 10 7 (noiseless) and 10 5 (Gaussian noise). In order to facilitate comparison, the details of the above results in Figure 5, Figure 6, Figure 7 and Figure 8 also can be found in Table 1 and Table 2.
According to the mean square error, obviously our algorithm outperforms the BP by a lot on both noiseless and noisy data. This trend is expected because the solution of the GOMP is updated by solving least squares problems. What is more, the BP normally cannot exactly recover the real signal since the produced solution may not be 0 on the indices which are not in the support, according to the observation during the experiments (can also be found in Figure 3b. For the GOMP, the solution’s support has been restricted in the group sparse set based on the selected indices of the group structure { G } i = 1 g at each iteration. That also leads to the better performance of the GOMP compared with BP. Although the performance of the GOMP can be affected by the group sparsity level and noise, our method still has pretty low mean square errors and performs better than BP, showing the robustness and effectiveness of our proposed algorithm. We can also observe that the GOMP runs much quicker than the BP, demonstrating the scalability and efficiency of our algorithm.
Further more, when it comes to large-scale group sparse signal recovery, the execution of BP occupies more memory and takes far more time to obtain the solution compared with our method (details can be found in Table 3 and Table 4 for different noise scenarios). This is because the scale of least square problems that the GOMP needs to solve to obtain the solution are bounded by the sparsity level K, as seen in the GOMP defined in Section 1. At the same time, the GOMP still outperforms BP a lot in terms of MSE. In addition, we can also find that the running speed of BP will be slightly slower when the noise occurs, while for the GOMP it will not be affected.

4. Conclusions

We propose the Group Orthogonal Matching Pursuit (GOMP) to recover group sparse signals. We analyze the error of the GOMP algorithm in the process of recovering group sparse signals from noisy measurements. We show the instance optimality and robustness of the GOMP under the group restricted isometry property (GRIP) of the decoder matrix. Compared with the P-norm minimization approach, the GOMP has two advantages. One is its easier implementation, that is, it runs quickly and has lower computational complexity. The other is that we do not need the concept of γ -decomposability. Furthermore, our simulation results show that the GOMP is very efficient for group sparse signal recovery and significantly outperforms Basis Pursuit in both scalability and solution quality.
On the other hand, there are several algorithms that we will study in the future, such as Sparsity Adaptive Matching Pursuit (SAMP) [34], Constrained Backtracking Matching Pursuit (CBMP) [35] and Group-based Sparse Representation-Joint Regularization (GSR-JR) [36], which could be potentially competitive with the GOMP.

Author Contributions

Conceptualization, all authors; methodology, all authors; software, all authors; validation, all authors; formal analysis, all authors; investigation, all authors; resources, all authors; data curation, all authors; writing—original draft preparation, all authors; writing—review and editing, all authors; visualization, all authors; supervision, all authors; project administration, all authors; funding acquisition, all authors. All authors have read and agreed to the published version of the manuscript.

Funding

The work is supported by the National Natural Science Foundation of China Grant No. 11671213 and No. 11701411.

Data Availability Statement

The data presented in this study are available upon request from the corresponding author.

Acknowledgments

The authors would like to thank the referees and the editors for their very useful suggestions, which improved significantly this paper.

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A

In this appendix, we establish three lemmas which will be used in the proof of Theorem 3. The first one is concerned with the reduction of the residuals for the GOMP algorithm.
Lemma A1.
Let ( r k ) k 0 be the residual error sequences generated by the GOMP applied to y ,   F ¯ { 1 , 2 , , g } , and let x ¯ = arg min z : Gsupp ( z ) F ¯ y Φ z 2 . If F ¯ is not contained in I k , then we have
r k + 1 2 2 r k 2 2 1 δ | F ¯ I k | α ( 1 + δ α ) | F ¯ \ I k | max { 0 , r k 2 2 y Φ x ¯ 2 2 } .
Proof. 
The proof of Lemma A1 is based on the ideas of Cohen, Dahmen and DeVore [24]. We may assume that r k 2 y Φ x ¯ 2 . Otherwise, inequality (A1) is trivially satisfied.
Denote
H k + 1 = span { φ i : i G I k + 1 } and F k + 1 = span { φ j : j G i k + 1 } .
It is clear that F k + 1 H k + 1 . Note that
r k + 1 = y Φ x k + 1 = y Φ x k + Φ x k Φ x k + 1 = r k + Φ x k P H k + 1 ( y ) = r k + Φ x k P H k + 1 ( r k + Φ x k ) = r k P H k + 1 ( r k ) ,
which implies
r k + 1 2 2 = r k P H k + 1 ( r k ) 2 2 r k P F k + 1 ( r k ) 2 2 = r k 2 2 P F k + 1 ( r k ) 2 2 .
Now, we estimate P F k + 1 ( r k ) 2 . Since Φ satisfies the GRIP of order α with the isometry constant δ α , we have
j G i k + 1 a j φ j 2 2 ( 1 + δ α ) j G i k + 1 | a j | 2 ,
which results in the following lower estimate
P F k + 1 ( r k ) 2 = sup ψ F k + 1 , ψ 1 | P F k + 1 ( r k ) , ψ | = sup { c j } j G i k + 1 , j G i k + 1 c j φ j 1 | P F k + 1 ( r k ) , j G i k + 1 c j φ j | = sup { c j } j G i k + 1 , j G i k + 1 c j φ j 1 | j G i k + 1 c j r k , φ j | sup { c j } j G i k + 1 , j G i k + 1 | c j | 2 ( 1 + δ α ) 1 | j G i k + 1 c j r k , φ j | = ( 1 + δ α ) 1 2 ( j G i k + 1 | r k , φ j | 2 ) 1 2 ,
where the first inequality is obtained by (A3) and the Cauchy–Schwarz inequality. Using inequality (A4), we can continue to estimate (A2) and conclude
r k + 1 2 2 r k 2 2 ( 1 + δ α ) 1 ( j G i k + 1 | r k , φ j | 2 ) .
Therefore, in view of inequalities (A5) and (A1), it remains to prove that
( 1 + δ α ) 1 Φ * [ i k + 1 ] r k 2 2 1 δ | F ¯ I k | α ( 1 + δ α ) | F ¯ \ I k | ( r k 2 2 y Φ x ¯ 2 2 ) ,
which is equivalent to
r k 2 2 y Φ x ¯ 2 2 | F ¯ \ I k | · Φ * [ i k + 1 ] r k 2 2 1 δ | F ¯ I k | α .
We note that
2 r k 2 2 y Φ x ¯ 2 2 Φ x ¯ Φ x k 2 r k 2 2 y Φ x ¯ 2 2 + Φ x ¯ Φ x k 2 2 = r k 2 2 r k + Φ x k Φ x ¯ 2 2 + Φ x ¯ Φ x k 2 2 2 | r k , Φ x k Φ x ¯ | = 2 | r k , Φ x ¯ | .
This is the same as
r k 2 2 y Φ x ¯ 2 2 | r k , Φ x ¯ | 2 Φ x ¯ Φ x k 2 2 .
On the one hand, by the GRIP of Φ , we have
Φ x ¯ Φ x k 2 2 = i F ¯ Φ [ i ] x ¯ [ i ] i I k Φ [ i ] x k [ i ] 2 2 = i F ¯ \ I k Φ [ i ] x ¯ [ i ] + i F ¯ I k Φ [ i ] ( x ¯ [ i ] x k [ i ] ) i I k \ F ¯ Φ [ i ] x k [ i ] 2 2 ( 1 δ | F ¯ I k | α ) ( i F ¯ \ I k x ¯ [ i ] 2 2 + i F ¯ I k x ¯ [ i ] x k [ i ] 2 2 + i I k \ F ¯ x k [ i ] 2 2 ) ( 1 δ | F ¯ I k | α ) i F ¯ \ I k x ¯ [ i ] 2 2 .
On the other hand, we have
| r k , Φ x ¯ | 2 = | r k , i F ¯ Φ [ i ] x ¯ [ i ] | 2 = | r k , i F ¯ \ I k Φ [ i ] x ¯ [ i ] | 2 = | i F ¯ \ I k r k , Φ [ i ] x ¯ [ i ] | 2 = | i F ¯ \ I k Φ * [ i ] r k , x ¯ [ i ] | 2 .
According to the greedy step of the GOMP, we further derive
| r k , Φ x ¯ | 2 = | i F ¯ \ I k Φ * [ i ] r k , x ¯ [ i ] | 2 ( i F ¯ \ I k Φ * [ i ] r k 2 · x ¯ [ i ] 2 ) 2 Φ * [ i k + 1 ] r k 2 2 · ( i F ¯ \ I k x ¯ [ i ] 2 ) 2 Φ * [ i k + 1 ] r k 2 2 · | F ¯ \ I k | · i F ¯ \ I k x ¯ [ i ] 2 2 ,
where we have used the Cauchy–Schwarz inequality.
Combining inequalities (A7)–(A9), we obtain
r k 2 2 y Φ x ¯ 2 2 | r k , Φ x ¯ | 2 Φ x ¯ Φ x k 2 2 | F ¯ \ I k | · Φ * [ i k + 1 ] r k 2 2 · i F ¯ \ I k x ¯ [ i ] 2 2 ( 1 δ | F ¯ I k | α ) F ¯ \ I k x ¯ [ i ] 2 2 = | F ¯ \ I k | · Φ * [ i k + 1 ] r k 2 2 ( 1 δ | F ¯ I k | α ) ,
and hence inequality (A6) holds. We complete the proof of Lemma A1. □
Corollary A1.
Assume that m i n z : Gsupp ( z ) F ¯ j Φ z y 2 2 Φ x ¯ y 2 2 + q j ( j = 0 , 1 , 2 , , L ) for q 0 q 1 q L 0 and subsets F ¯ Ω = F ¯ 0 F ¯ 1 F ¯ 2 F ¯ L F ¯ . If
k = j = 1 L | F ¯ j \ Ω | 1 + δ α 1 δ s α ln ( 2 μ ) with μ sup j = 1 , 2 , , L 1 q j 1 q j
and
s | F ¯ I k | ,
then
r k 2 2 Φ x ¯ y 2 2 + q L + μ 1 q L 1 .
Proof. 
By Lemma A1, for l = 0 , 1 , 2 , , L , we have either | F ¯ l \ I k | = 0 or inequality (A1) holds. Inequality (A1), along with r k + 1 2 2 r k 2 2 , imply that either | F ¯ l \ I k | = 0 or
max { 0 , r k + 1 2 2 Φ x ¯ y 2 2 q l } [ 1 1 δ s α ( 1 + δ α ) | F ¯ l \ I k | ] max { 0 , r k 2 2 Φ x ¯ y 2 2 q l } exp [ 1 δ s α ( 1 + δ α ) | F ¯ l \ I k | ] max { 0 , r k 2 Φ x ¯ y 2 2 q l } ,
where we use the inequality exp ( x ) > 1 x for x > 0 .
Therefore, for any k k and l = 0 , 1 , 2 , , L , we have either | F ¯ l \ I k | = 0 or
r k 2 2 Φ x ¯ y 2 2 q l exp [ ( 1 δ s α ) ( k k ) ( 1 + δ α ) | F ¯ l \ I k | ] max { 0 , r k 2 2 Φ x ¯ y 2 2 q l } .
We now prove this corollary by induction on L . If L = 1 , then we can set k = 0 and consider μ > 0 . When
k = | F ¯ 1 \ Ω | 1 + δ α 1 δ s α ln ( 2 μ ) ,
we have by inequality (A11)
r k 2 2 Φ x ¯ y 2 2 q 1 exp [ ( 1 δ s α ) k ( 1 + δ α ) | F ¯ 1 \ Ω | ] max { 0 , r 0 2 Φ x ¯ y 2 2 q 1 } exp [ ( 1 δ s α ) k ( 1 + δ α ) | F ¯ 1 \ Ω | ] q 0 ( 2 μ ) 1 q 0 ,
and hence
r k 2 2 Φ x ¯ y 2 2 + q 1 + ( 2 μ ) 1 q 0 .
Note that this inequality also holds when | F ¯ 1 \ I k | = 0 , since
r k 2 2 min z : Gsupp ( z ) F ¯ 1 Φ z y 2 2 Φ x ¯ y 2 2 + q 1 .
Therefore, inequality (A10) always holds when L = 1 . Assume that the corollary holds at L 1 for some L > 1 . This is, with
k = j = 1 L 1 | F ¯ j \ Ω | 1 + δ α 1 δ s α ln ( 2 μ ) ,
we have
r k 2 2 Φ x ¯ y 2 2 + q L 1 + μ 1 q L 2 .
Note that for μ = sup j = 1 , 2 , , L 1 q j 1 q j , we have
r k 2 2 Φ x ¯ y 2 2 q L q L 1 + μ 1 q L 2 q L 2 q L 1 .
Inequality (A11) implies that for | F ¯ L \ I k | 0 ,
r k 2 2 Φ x ¯ y 2 2 q L exp [ ( 1 δ s α ) ( k k ) ( 1 + δ α ) | F ¯ L \ I k | ] max { 0 , r k 2 2 Φ x ¯ y 2 2 q L } .
Combining (A12) with (A13), we conclude that
r k 2 2 Φ x ¯ y 2 2 q L exp [ ( 1 δ s α ) ( k k ) ( 1 + δ α ) | F ¯ L \ I k | ] 2 q L 1 exp [ ( 1 δ s α ) ( k k ) ( 1 + δ α ) | F ¯ L \ Ω | ] 2 q L 1 μ 1 q L 1 .
Again, this inequality also holds when | F ¯ L \ I k | = 0 , since
r k 2 2 min Gsupp ( z ) F ¯ L Φ z y 2 2 Φ x ¯ y 2 2 + q L .
We complete the proof of this corollary. □
The second and third ones are technical lemmas. For F { 1 , , g } , set
x = arg min z : Gsupp ( z ) F Φ z y 2 2 .
Lemma A2.
For all s | F ¯ \ F | , we have
Φ x y 2 2 Φ x ¯ y 2 2 1.5 ( 1 + δ s α ) x ¯ F ¯ \ F 2 2 + 0.5 ε s ( x ¯ ) 2 / ( 1 + δ s α ) .
Proof. 
Let x = x ¯ F ¯ F . Then Φ x y 2 2 Φ x y 2 2 . Therefore,
Φ x y 2 2 Φ x ¯ y 2 2 Φ x y 2 2 Φ x ¯ y 2 2 = Φ x Φ x ¯ + Φ x ¯ y 2 2 Φ x ¯ y 2 2 Φ ( x x ¯ ) 2 2 + | 2 Φ ( x x ¯ ) , Φ x ¯ y | .
We continue to estimate (A14). Note that | Gsupp ( x x ¯ ) | s . By the definition of ε s ( x ¯ ) and the GRIP of Φ , we have
Φ x y 2 2 Φ x ¯ y 2 2 ( 1 + δ s α ) x ¯ F ¯ \ F 2 2 + ε s ( x ¯ ) x ¯ F ¯ \ F 2 ( 1 + δ s α ) x ¯ F ¯ \ F 2 2 + 0.5 ε s ( x ¯ ) 2 / ( 1 + δ s α ) + 0.5 ( 1 + δ s α ) x ¯ F ¯ \ F 2 2 = 1.5 ( 1 + δ s α ) x ¯ F ¯ \ F 2 2 + 0.5 ε s ( x ¯ ) 2 / ( 1 + δ s α ) ,
where we have used the mean value inequality in the second inequality. □
Lemma A3.
For all s | F F ¯ | , we have
( 1 δ s α ) x x ¯ 2 2 2 ( Φ x y 2 2 Φ x ¯ y 2 2 ) + ε s ( x ¯ ) 2 / ( 1 δ s α ) .
Proof. 
Notice that
Φ x y 2 2 Φ x ¯ y 2 2 = Φ x Φ x ¯ + Φ x ¯ y 2 2 Φ x ¯ y 2 2 = Φ x Φ x ¯ 2 2 + Φ x ¯ y 2 2 + 2 Φ ( x x ¯ ) , Φ x ¯ y Φ x ¯ y 2 2 Φ ( x x ¯ ) 2 2 | 2 Φ ( x x ¯ ) , Φ x ¯ y | .
By the definition of ε s ( x ¯ ) and the GRIP of Φ , we derive
Φ x y 2 2 Φ x ¯ y 2 2 ( 1 δ s α ) x x ¯ 2 2 ε s ( x ¯ ) x ¯ x 2 ( 1 δ s α ) x x ¯ 2 2 0.5 ε s ( x ¯ ) 2 / ( 1 δ s α ) 0.5 ( 1 δ s α ) x x ¯ 2 2 = 0.5 ( 1 δ s α ) x x ¯ 2 2 0.5 ε s ( x ¯ ) 2 / ( 1 δ s α ) ,
where we have used again the mean value inequality in the second inequality. □

Appendix B

Proof of Theorem 3.
We prove this theorem by induction on | F ¯ \ Ω | . If | F ¯ \ Ω | = 0 , then
r 0 2 2 = min x ˜ : Gsupp ( x ˜ ) Ω y Φ x ˜ 2 2 Φ x ¯ y 2 2 .
Assume that the inequality holds with | F ¯ \ Ω | p 1 for some integer p > 0 . Then we consider the case of | F ¯ \ Ω | = p . Without loss of generality, we assume for notational convenience that F ¯ \ Ω = { 1 , 2 , , p } and x ¯ G j 2 for j F ¯ \ Ω is arranged in descending order so that x ¯ G 1 2 x ¯ G 2 2 x ¯ G p 2 . Let L be the smallest positive integer, such that for all 1 l < L , we have
i = 2 l 1 p x ¯ G i 2 2 < μ i = 2 l p x ¯ G i 2 2 ,
but
i = 2 L 1 p x ¯ G i 2 2 μ i = 2 L p x ¯ G i 2 2 ,
where μ = 10 ( 1 + δ p α ) / ( 1 δ s α ) . Note that 2 L p , so L log 2 p + 1 . Moreover, if the second inequality is always satisfied for all L 1 , then we can simply take L = 1 .
We define
F ¯ l = { i : 1 i 2 l 1 } ( F ¯ Ω ) , for l = 0 , 1 , 2 , , L .
Then F ¯ Ω = F ¯ 0 F ¯ 1 F ¯ 2 F ¯ L F ¯ . Lemma 3.2 implies that for l = 0 , 1 , , L 1 ,
min x : Gsupp ( x ) F ¯ l Φ x y 2 2 Φ x ¯ y 2 2 + q l ,
where
q l 1 = 1.5 ( 1 + δ p α ) x ¯ F ¯ \ F ¯ l 1 2 2 + 0.5 ε p ( x ¯ ) 2 / ( 1 + δ p α ) = 1.5 ( 1 + δ p α ) i = 2 l 1 p x ¯ G i 2 2 + 0.5 ε p ( x ¯ ) 2 / ( 1 + δ p α ) .
By (A16), we have
q l 1 < 1.5 ( 1 + δ p α ) μ i = 2 l p x ¯ G i 2 2 + 0.5 μ ε p ( x ¯ ) 2 / ( 1 + δ p α ) = μ q l
for p | F ¯ \ F ¯ l | . Thus μ sup j = 1 , 2 , , L 1 q j 1 q j and q 0 q 1 q 2 q L 0 . It follows from Corollary A1 that for
k = j = 1 L | F ¯ j \ Ω | 1 + δ α 1 δ s α ln ( 2 μ ) j = 1 L ( 2 j 1 ) 1 + δ α 1 δ s α ln ( 2 μ ) + L 2 L + 1 1 + δ α 1 δ s α ln ( 2 μ ) ( 2 + L ) 1 + δ α 1 δ s α ln ( 2 μ ) + L 2 L + 1 1 + δ α 1 δ s α ln ( 2 μ ) 1 ,
and
s | F ¯ I k | ,
we have
r k 2 2 Φ x ¯ y 2 2 + q L + μ 1 q L 1 .
Combining (A17) with (A18), we obtain
r k 2 2 Φ x ¯ y 2 2 + 1.5 ( 1 + δ p α ) i = 2 L p x ¯ G i 2 2 + 0.5 ε p ( x ¯ ) 2 / ( 1 + δ p α ) + 1.5 μ 1 ( 1 + δ p α ) i = 2 L 1 p x ¯ G i 2 2 + 0.5 μ 1 ε p ( x ¯ ) 2 / ( 1 + δ p α ) Φ x ¯ y 2 2 + 1.5 ( 1 + δ p α ) μ 1 i = 2 L 1 p x ¯ G i 2 2 + 0.5 ε p ( x ¯ ) 2 / ( 1 + δ p α ) + 1.5 μ 1 ( 1 + δ p α ) i = 2 L 1 p x ¯ G i 2 2 + 0.5 μ 1 ε p ( x ¯ ) 2 / ( 1 + δ p α ) = Φ x ¯ y 2 2 + 3 μ 1 ( 1 + δ p α ) i = 2 L 1 p x ¯ G i 2 2 + 0.5 ( 1 + μ 1 ) ε p ( x ¯ ) 2 / ( 1 + δ p α ) .
If 2 μ 1 ( 1 + δ p α ) i = 2 L 1 p x ¯ G i 2 2 ( 1 + μ 1 ) ε s ( x ¯ ) 2 / ( 1 δ s α ) , then
r k 2 2 Φ x ¯ y 2 2 1.5 ( 1 + μ 1 ) ε s ( x ¯ ) 2 / ( 1 δ s α ) + 0.5 ( 1 + μ 1 ) ε p ( x ¯ ) 2 / ( 1 + δ p α ) 2 ( 1 + μ 1 ) ε s ( x ¯ ) 2 / ( 1 δ s α ) 2.5 ε s ( x ¯ ) 2 / ( 1 δ s α ) ,
where we have used μ 10 in the last inequality.
If 2 μ 1 ( 1 + δ p α ) i = 2 L 1 p x ¯ G i 2 2 > ( 1 + μ 1 ) ε s ( x ¯ ) 2 / ( 1 δ s α ) , then combining inequality (A20) with Lemma A3, we have
( 1 δ s α ) x k x ¯ 2 2 2 ( r k 2 2 Φ x ¯ y 2 2 ) + ε s ( x ¯ ) / ( 1 δ s α ) 6 μ 1 ( 1 + δ p α ) i = 2 L 1 p x ¯ G i 2 2 + ( 2 + μ 1 ) ε s ( x ¯ ) 2 / ( 1 δ s α ) < 10 μ 1 ( 1 + δ p α ) i = 2 L 1 p x ¯ G i 2 2 = ( 1 δ s α ) i = 2 L 1 p x ¯ G i 2 2 ,
which implies that
i = p | F ¯ \ I k | + 1 p x ¯ G i 2 2 i F ¯ \ I k x ¯ G i 2 2 x k x ¯ 2 2 < i = 2 L 1 p x ¯ G i 2 2 .
Therefore, p | F ¯ \ I k | + 1 > 2 L 1 , that is, | F ¯ \ I k | p 2 L 1 . Then we can take I k for Ω and run the GOMP algorithm again. By induction, after another
k 1 = 4 ( p 2 L 1 ) 1 + δ α 1 δ s α ln 20 ( 1 + δ | F ¯ \ I k | α ) 1 δ s α
GOMP iterations, we have
r k + k 1 2 2 Φ x ¯ y 2 2 + 2.5 ε s ( x ¯ ) 2 / ( 1 δ s α ) ,
while s | F ¯ I k | + 4 | F ¯ \ I k | 1 + δ α 1 δ s α ln 20 ( 1 + δ | F ¯ \ I k | α ) 1 δ s α . Since
| F ¯ I k | + 4 | F ¯ \ I k | 1 + δ α 1 δ s α ln 20 ( 1 + δ | F ¯ \ I k | α ) 1 δ s α | F ¯ Ω | + k + 4 ( p 2 L 1 ) 1 + δ α 1 δ s α ln 20 ( 1 + δ | F ¯ \ Ω | α ) 1 δ s α | F ¯ Ω | + 2 L + 1 1 + δ α 1 δ s α ln 2 μ 1 + 4 ( p 2 L 1 ) 1 + δ α 1 δ s α ln 2 μ + 1 | F ¯ Ω | + 4 p 1 + δ α 1 δ s α ln 2 μ = | F ¯ Ω | + 4 | F ¯ \ Ω | 1 + δ α 1 δ s α ln 20 ( 1 + δ | F ¯ \ Ω | α ) 1 δ s α ,
so for
k 0 4 p 1 + δ α 1 δ s α ln 20 ( 1 + δ p α ) 1 δ s α 2 L + 1 1 + δ α 1 δ s α ln ( 2 μ ) 1 + 4 ( p 2 L 1 ) 1 + δ α 1 δ s α ln 20 ( 1 + δ | F ¯ \ I k | α ) 1 δ s α k + k 1
and
s | F ¯ Ω | + 4 | F ¯ \ Ω | 1 + δ α 1 δ s α ln 20 ( 1 + δ | F ¯ \ Ω | α ) 1 δ s α ,
we have
r k 0 2 Φ x ¯ y 2 + 2.5 ε s ( x ¯ ) 2 / ( 1 δ s α ) .
This finishes the induction step for the case | F ¯ \ Ω | = p .
For the second inequality, combining the first inequality with Lemma A3, we have
( 1 δ s α ) x k x ¯ 2 2 2 ( Φ x k y 2 2 Φ x ¯ y 2 2 ) + ε s ( x ¯ ) 2 / ( 1 δ s α ) 6 ε s ( x ¯ ) 2 / ( 1 δ s α ) .
Thus we complete the proof of this theorem. □
When Ω is the empty set, we have the following corollary of Theorem 3.
Corollary A2.
If the GRIP condition 2 δ 31 | F ¯ | α + δ | F ¯ | α 1 holds, then we have for k = 30 | F ¯ | ,
r k 2 2 Φ ( x ¯ ) y 2 2 + 2.5 ε s ( x ¯ ) 2 / ( 1 δ s α )
and
x k x ¯ 2 2 6 1 + δ s α Φ x ¯ y 2 / ( 1 δ s α ) ,
where s = 31 | F ¯ | .
Proof. 
If 2 δ 31 | F ¯ | α + δ | F ¯ | α 1 , then we have
1 + δ α 1 δ 31 | F ¯ | α 1 + δ | F ¯ | α 1 δ 31 | F ¯ | α 2 .
Therefore
s = 31 | F ¯ | = | F ¯ | + 30 | F ¯ | | F ¯ | + 4 | F ¯ | · 2 ln ( 20 · 2 ) | F ¯ | + 4 | F ¯ | 1 + δ α 1 δ s α ln 20 1 + δ | F ¯ | α 1 δ s α .
Combining (A21) with Theorem 3, we obtain the first inequality of Corollary A2 and
x k x ¯ 2 6 ε s ( x ¯ ) / ( 1 δ s α ) .
Notice that for any μ R n with | Gsupp ( μ ) | s , we have
| 2 Φ μ , Φ x ¯ y | 2 Φ μ 2 · Φ x ¯ y 2 2 1 + δ s α μ 2 · Φ x ¯ y 2 ,
and hence
ε s ( x ¯ ) 2 1 + δ s α Φ x ¯ y 2 .
Substituting (A23) into (A22), we obtain the second inequality of Corollary A2. The proof of Corollary A2 is completed. □

References

  1. Foucart, S.; Rauhut, H. A Mathematical Introduction to Compressive Sensing; Springer: Berlin/Heidelberg, Germany, 2013. [Google Scholar]
  2. Cai, T.; Zhang, A. Sparse representation of a polytope and recovery of sparse signals and low-rank matrices. IEEE Trans. Inf. Theory 2014, 60, 122–132. [Google Scholar] [CrossRef]
  3. Günther, M.; Böhringer, S.; Wieczorek, D.; Würtz, R.P. Reconstruction of images from Gabor graphs with applications in facial image processing. Int. J. Wavelets Multiresolut. Inf. Process. 2015, 13, 1550019. [Google Scholar] [CrossRef]
  4. Ye, P.X.; Wei, X.J. Efficiency of weak greedy algorithms for m-term approximations. Sci. China. Math. 2016, 59, 697–714. [Google Scholar] [CrossRef]
  5. Yang, B.; Yang, C.; Huang, G. Efficient image fusion with approximate sparse representation. Int. J. Wavelets Multiresolut. Inf. Process. 2016, 14, 1650024. [Google Scholar] [CrossRef]
  6. Candès, E.; Kalton, N.J. The restricted isometry property and its implications for compressed sensing. C. R. Acad. Sci. Sér. I 2008, 346, 589–592. [Google Scholar] [CrossRef]
  7. Candès, E.J.; Tao, T. Decoding by linear programming. IEEE Trans. Inf. Theory 2005, 51, 4203–4215. [Google Scholar] [CrossRef]
  8. Yuan, M.; Lin, Y. Model selection and estimation in regression with grouped variables. J. R. Stat. Ser. B 2006, 68, 49–67. [Google Scholar] [CrossRef]
  9. Ou, W.; Hämäläinen, M.S.; Golland, P. A distributed spatio-temporal EEG/MEG inverse solver. NeuroImage 2009, 44, 932–946. [Google Scholar] [CrossRef]
  10. Antoniadis, A.; Fan, J. Regularization of wavelet approximations. J. Amer. Stat. Assoc. 2001, 96, 939–967. [Google Scholar] [CrossRef]
  11. He, Z.; Yu, W. Stable feature selection for biomarker discovery. Comput. Biol. Chem. 2010, 34, 215–225. [Google Scholar] [CrossRef]
  12. Mishali, M.; Eldar, Y.C. Blind multiband signal reconstruction: Compressed sensing for analog signals. IEEE Trans. Signal Process. 2009, 57, 993–1009. [Google Scholar] [CrossRef]
  13. Ahsen, M.E.; Vidyasagar, M. Error bounds for compressed sensing algorithms with group sparsity: A unified approach. Appl. Comput. Harmon. Anal. 2017, 42, 212–232. [Google Scholar] [CrossRef]
  14. Ranjan, S.; Vidyasagar, M. Tight performance bounds for compressed sensing with conventional and group sparsity. IEEE Trans. Signal Process 2019, 67, 2854–2867. [Google Scholar] [CrossRef]
  15. Tropp, J.A. Greedy is good: Algorithmic results for sparse approximation. IEEE Trans. Inform. Theory 2004, 50, 2231–2242. [Google Scholar] [CrossRef]
  16. Lin, J.H.; Li, S. Nonuniform support recovery from noisy measurements by orthogonal matching pursuit. J. Approx. Theory 2013, 165, 20–40. [Google Scholar] [CrossRef]
  17. Mo, Q.; Shen, Y. A remark on the restricted isometry property in orthogonal matching pursuit. IEEE Trans. Inf. Theory 2012, 58, 3654–3656. [Google Scholar] [CrossRef]
  18. Dan, W. Analysis of orthogonal multi-matching pursuit under restricted isometry property. Sci. China Math. 2014, 57, 2179–2188. [Google Scholar] [CrossRef]
  19. Xu, Z.Q. The performance of orthogonal multi-matching pursuit under RIP. J. Comp. Math. 2015, 33, 495–516. [Google Scholar] [CrossRef]
  20. Shao, C.F.; Ye, P.X. Almost optimality of orthogonal super greedy algorithms for incoherent dictionaries. Int. J. Wavelets Multiresolut. Inf. Process. 2017, 15, 1750029. [Google Scholar] [CrossRef]
  21. Wei, X.J.; Ye, P.X. Efficiency of orthogonal super greedy algorithm under the restricted isometry property. J. Inequal. Appl. 2019, 124, 1–21. [Google Scholar] [CrossRef]
  22. Zhang, T. Sparse recovery with orthogonal matching pursuit under RIP. IEEE Trans. Inf. Theory 2011, 57, 6215–6221. [Google Scholar] [CrossRef]
  23. Xu, Z.Q. A remark about orthogonal matching pursuit algorithm. Adv. Adapt. Data Anal. 2012, 04, 1250026. [Google Scholar] [CrossRef]
  24. Cohen, A.; Dahmen, W.; DeVore, R. Orthogonal matching pursuit under the restricted isometry property. Constr. Approx. 2017, 45, 113–127. [Google Scholar] [CrossRef]
  25. Eldar, Y.C.; Mishali, M. Robust recovery of signals from a structured union of subspaces. IEEE Trans. Inf. Theory 2009, 55, 5302–5316. [Google Scholar] [CrossRef]
  26. Wright, J.; Yang, A.Y.; Ganesh, A.; Sastry, S.S.; Ma, Y. Robust face recognition via sparse representation. IEEE Trans. Pattern Anal. Mach. Intell. 2009, 31, 210–227. [Google Scholar] [CrossRef]
  27. Van Den Berg, E.; Friedlander, M.P. Theoretical and empirical results for recovery from multiple measurements. IEEE Trans. Inf. Theory 2010, 56, 2516–2527. [Google Scholar] [CrossRef]
  28. Eldar, Y.C.; Kuppinger, P.; Bölcskei, H. Block-sparse signals: Uncertainty relations and efficient recovery. IEEE Trans. Signal Process. 2010, 58, 3042–3054. [Google Scholar] [CrossRef]
  29. Lai, M.J.; Liu, Y. The null space property for sparse recovery from multiple measurement vectors. Appl. Comput. Harmon. Anal. 2011, 30, 402–406. [Google Scholar] [CrossRef]
  30. Zhang, J.; Wang, J.; Wang, W. A perturbation analysis of block-sparse compressed sensing via mixed l2/l1 minimization. Int. J. Wavelets, Multiresolut. Inf. Process. 2016, 14, 1650026. [Google Scholar] [CrossRef]
  31. Chen, W.G.; Ge, H.M. A sharp recovery condition for block sparse signals by block orthogonal multi-matching pursuit. Sci. China Math. 2017, 60, 1325–1340. [Google Scholar] [CrossRef]
  32. Wen, J.M.; Zhou, Z.C.; Liu, Z.L.; Lai, M.J.; Tang, X.H. Sharp sufficient conditions for stable recovery of block sparse signals by block orthogonal matching pursuit. Appl. Comput. Harmon. Anal. 2019, 47, 948–974. [Google Scholar] [CrossRef]
  33. Candès, E.; Romberg, J. l1-Magic: Recovery of Sparse Signals via Convex Programming. Available online: https://candes.su.domains/software/l1magic/#code (accessed on 15 February 2023).
  34. Do, T.T.; Gan, L.; Nguyen, N.; Tran, T.D. Sparsity adaptive matching pursuit algorithm for practical compressed sensing. In Proceedings of the 2008 42nd Asilomar Conference on Signals, Systems and Computers, Pacific Grove, CA, USA, 26–29 October 2008; pp. 581–587. [Google Scholar]
  35. Bi, X.; Leng, L.; Kim, C.; Liu, X.; Du, Y.; Liu, F. Constrained Backtracking Matching Pursuit Algorithm for Image Reconstruction in Compressed Sensing. Appl. Sci. 2021, 11, 1435. [Google Scholar] [CrossRef]
  36. Wang, R.; Qin, Y.; Wang, Z.; Zheng, H. Group-Based Sparse Representation for Compressed Sensing Image Reconstruction with Joint Regularization. Electronics 2022, 11, 182. [Google Scholar] [CrossRef]
Figure 1. Examples of group 50-sparse signals in different dimensions: (a) An example of the group 50-sparse signal in dimension N = 128 . (b) An example of the group 50-sparse signal in dimension N = 256 . (c) An example of the group 50-sparse signal in dimension N = 512 . (d) An example of the group 50-sparse signal in dimension N = 1024 .
Figure 1. Examples of group 50-sparse signals in different dimensions: (a) An example of the group 50-sparse signal in dimension N = 128 . (b) An example of the group 50-sparse signal in dimension N = 256 . (c) An example of the group 50-sparse signal in dimension N = 512 . (d) An example of the group 50-sparse signal in dimension N = 1024 .
Axioms 12 00389 g001
Figure 2. The recovery of an input signal via GOMP in dimension N = 512 with group sparsity level K = 50 and number of measurements M = 200 under different noises: (a) The recovery of an input signal via GOMP under the noise e = 0 . (b) The recovery of an input signal via GOMP under a Gaussian noise e from N ( 0 , 0.1 2 ) .
Figure 2. The recovery of an input signal via GOMP in dimension N = 512 with group sparsity level K = 50 and number of measurements M = 200 under different noises: (a) The recovery of an input signal via GOMP under the noise e = 0 . (b) The recovery of an input signal via GOMP under a Gaussian noise e from N ( 0 , 0.1 2 ) .
Axioms 12 00389 g002
Figure 3. The recovery of an input signal via BP in dimension N = 512 with group sparsity level K = 50 and number of measurements M = 200 under different noises: (a) The recovery of an input signal via BP under the noise e = 0 . (b) The recovery of an input signal via BP under a Gaussian noise e from N ( 0 , 0.1 2 ) .
Figure 3. The recovery of an input signal via BP in dimension N = 512 with group sparsity level K = 50 and number of measurements M = 200 under different noises: (a) The recovery of an input signal via BP under the noise e = 0 . (b) The recovery of an input signal via BP under a Gaussian noise e from N ( 0 , 0.1 2 ) .
Axioms 12 00389 g003
Figure 4. The percentage of the support of 100 input signals correctly recovered as a function of number N of Gaussian measurements for different group sparsity levels K in dimension d = 1024 .
Figure 4. The percentage of the support of 100 input signals correctly recovered as a function of number N of Gaussian measurements for different group sparsity levels K in dimension d = 1024 .
Axioms 12 00389 g004
Figure 5. MSE and running time of the GOMP in dimension N = 1024 with number of measurements M = 500 under the noise e = 0 .
Figure 5. MSE and running time of the GOMP in dimension N = 1024 with number of measurements M = 500 under the noise e = 0 .
Axioms 12 00389 g005
Figure 6. MSE and running time of the GOMP in dimension N = 1024 with number of measurements M = 500 under a Gaussian noise e from N ( 0 , 0.1 2 ) .
Figure 6. MSE and running time of the GOMP in dimension N = 1024 with number of measurements M = 500 under a Gaussian noise e from N ( 0 , 0.1 2 ) .
Axioms 12 00389 g006
Figure 7. MSE and running time of BP in dimension N = 1024 with number of measurements M = 500 under the noise e = 0 .
Figure 7. MSE and running time of BP in dimension N = 1024 with number of measurements M = 500 under the noise e = 0 .
Axioms 12 00389 g007
Figure 8. MSE and running time of BP in dimension N = 1024 with number of measurements M = 500 under a Gaussian noise e from N ( 0 , 0.1 2 ) .
Figure 8. MSE and running time of BP in dimension N = 1024 with number of measurements M = 500 under a Gaussian noise e from N ( 0 , 0.1 2 ) .
Axioms 12 00389 g008
Table 1. The average MSE and running time (repeating 100 times) of the GOMP and BP in 1024 dimension with number of measurements M = 500 under the noise e = 0 .
Table 1. The average MSE and running time (repeating 100 times) of the GOMP and BP in 1024 dimension with number of measurements M = 500 under the noise e = 0 .
Group Sparsity LevelGOMPBP
MSERunning TimeMSERunning Time
K = 10 1.6350 × 10 32 0.0045s8.1887 × 10 08 0.3724 s
K = 20 6.3295 × 10 32 0.0042 s 8.7195 × 10 8 0.5158 s
K = 30 9.0108 × 10 32 0.0047 s 9.3230 × 10 8 0.5358 s
K = 40 1.5460 × 10 31 0.0056 s 1.0408 × 10 7 0.5544 s
K = 50 2.3565 × 10 31 0.0056 s 1.1414 × 10 7 0.4513 s
K = 60 2.7140 × 10 31 0.0054 s 1.2543 × 10 7 0.5099 s
K = 70 3.5189 × 10 31 0.0063 s 1.4186 × 10 7 0.5634 s
K = 80 4.6822 × 10 31 0.0067 s 1.6007 × 10 7 0.5831 s
K = 90 5.5434 × 10 31 0.0077 s 1.8272 × 10 7 0.5946 s
K = 100 6.7451 × 10 31 0.0078 s 2.0194 × 10 7 0.6087 s
Table 2. The average MSE and running time (repeating 100 times) of the GOMP and BP in 1024 dimension with number of measurements M = 500 under a Gaussian noise e from N ( 0 , 0.1 2 ) .
Table 2. The average MSE and running time (repeating 100 times) of the GOMP and BP in 1024 dimension with number of measurements M = 500 under a Gaussian noise e from N ( 0 , 0.1 2 ) .
Group Sparsity LevelGOMPBP
MSERunning TimeMSERunning Time
K = 10 2.1440 × 10 7 0.0043 s1.4580 × 10 5 0.4691 s
K = 20 4.1167 × 10 7 0.0043 s 1.5919 × 10 5 0.5102 s
K = 30 6.1894 × 10 7 0.0050 s 1.6566 × 10 5 0.5228 s
K = 40 8.6262 × 10 7 0.0058 s 1.7790 × 10 5 0.5547 s
K = 50 1.1870 × 10 6 0.0058 s 1.9266 × 10 5 0.5996 s
K = 60 1.3162 × 10 6 0.0057 s 2.0652 × 10 5 0.6134 s
K = 70 1.6356 × 10 6 0.0067 s 2.2859 × 10 5 0.6279 s
K = 80 1.9119 × 10 6 0.0069 s 2.5038 × 10 5 0.6283 s
K = 90 2.1399 × 10 6 0.0076 s 2.8718 × 10 5 0.6265 s
K = 100 2.5990 × 10 6 0.0081 s 3.1227 × 10 5 0.6317 s
Table 3. The average MSE and running time (repeating 100 times) of the GOMP and BP with number of measurements M = 500 and group sparsity level K = 50 under the noise e = 0 .
Table 3. The average MSE and running time (repeating 100 times) of the GOMP and BP with number of measurements M = 500 and group sparsity level K = 50 under the noise e = 0 .
DimensionGOMPBP
MSERunning TimeMSERunning Time
N = 1024 2.3846 × 10 31 0.0057 s 1.1414 × 10 7 0.4513 s
N = 2048 1.1178 × 10 31 0.0114 s 4.4095 × 10 8 1.6611 s
N = 3072 8.5425 × 10 32 0.0195 s 2.7722 × 10 8 5.8170 s
N = 4096 5.9586 × 10 32 0.0413 s 1.9752 × 10 8 11.3636 s
Table 4. The average MSE and running time (repeating 100 times) of the GOMP and BP with number of measurements M = 500 and group sparsity level K = 50 under a Gaussian noise e from N ( 0 , 0.1 2 ) .
Table 4. The average MSE and running time (repeating 100 times) of the GOMP and BP with number of measurements M = 500 and group sparsity level K = 50 under a Gaussian noise e from N ( 0 , 0.1 2 ) .
DimensionGOMPBP
MSERunning TimeMSERunning Time
N = 1024 1.1143 × 10 6 0.0066 s 1.9539 × 10 5 0.4521 s
N = 2048 5.7826 × 10 7 0.0115 s 7.8597 × 10 6 2.6113 s
N = 3072 3.9382 × 10 7 0.0200 s 5.3449 × 10 6 6.3476 s
N = 4096 3.0434 × 10 7 0.0406 s 4.4554 × 10 6 12.1959 s
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Shao, C.; Wei, X.; Ye, P.; Xing, S. Efficiency of Orthogonal Matching Pursuit for Group Sparse Recovery. Axioms 2023, 12, 389. https://doi.org/10.3390/axioms12040389

AMA Style

Shao C, Wei X, Ye P, Xing S. Efficiency of Orthogonal Matching Pursuit for Group Sparse Recovery. Axioms. 2023; 12(4):389. https://doi.org/10.3390/axioms12040389

Chicago/Turabian Style

Shao, Chunfang, Xiujie Wei, Peixin Ye, and Shuo Xing. 2023. "Efficiency of Orthogonal Matching Pursuit for Group Sparse Recovery" Axioms 12, no. 4: 389. https://doi.org/10.3390/axioms12040389

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop