Bias-Corrected Inference of High-Dimensional Generalized Linear Models

Abstract

In this paper, we propose a weighted link-specific (WLS) approach that establishes a unified statistical inference framework for high-dimensional Poisson and Gamma regression. We regress the parameter deviations as well as the initial estimation errors and utilize the resulting regression coefficients as correction weights to reduce the total mean square error (MSE). We also develop the asymptotic normality of the correction estimates under sparse and non-sparse conditions and construct associated confidence intervals (CIs) to verify the robustness of the new method. Finally, numerical simulations and empirical analysis show that the WLS method is extensive and effective.

1. Introduction

With the rapid development of modern computer technology, data collection and processing capabilities are also rapidly increasing. In such a high-dimensional environment, our focuses will become different—for example, what is the relationship between productivity and age in the context of big data [1], how air pollution affects the mortality rate of urban residents [1], and how does the spectrum of raindrops affect mobile dual-polarization radar [2]. Under such conditions, most traditional inference procedures, such as the maximum likelihood method, are no longer valid and yield significantly different results in sparse and non-sparse states. Our attention is thus turned to high-dimensional generalized linear models, such as Poisson regression, Gamma regression, etc., in the expectation that they can solve these problems.

In the face of count data, the Poisson regression model is one of the most commonly used tools, which is a regression model specifically designed to analyze the dependent variable as a count variable based on a Poisson distribution. In contrast, among generalized linear models, the Gamma regression model can usually only be used to model continuous data with response data greater than or equal to zero. This paper establishes a unified inference framework for Poisson regression and Gamma regression through different linkage functions, aiming to build a suitable model for either continuous or discrete data.

High-dimensional Generalized Linear Model (GLM) inferences have been studied by many scholars [3,4,5,6]. Deshpande [7] proposed a debiasing method for constructing CIs. Cai, Athey and Zhu [8,9,10] proposed a more general linear comparison method under the condition of special load vectors. For high-dimensional logistic regression models, Sur and Candès [11] have studied the likelihood ratio test procedure when setting the sparsity $k<1/2, p/n\to k$. Among them, p is the parameter dimension, and n is the number of samples. Ma, Cai, and Li [12] put forward the testing procedure of the global null hypothesis and large-scale simultaneous hypothesis at the time of setting $p\gg n$. Recently, Shi, C et al. [13] imposed certain strict constraints such as bounded individual probability conditions based on the logical link function to perform hypothesis testing. Under the high-dimensional sparse space, Lasso [14] usually leads to a large bias, and the current correction estimation is mainly performed under linear regression [15,16] and logistic regression models. In the work of Ma, R [12], a weighted low-dimensional projection method was proposed, which achieves the effect of debiasing and constructs its CIs by constructing specific weights and a dimensionality reduction process. In the work of Shi [13], a low-dimensional regression coefficient was proposed to reduce the dimensionality from high to medium dimensionality by recursively performing model selection for de-weighting. Cai and Guo [17] developed a weighted link function method that also constructs specific de-weighting and applies it to logistic regression. Generally speaking, the current research is mainly focused on logistic regression [18,19,20], but there are no other GLMs, such as Poisson and Gamma regression.

Motivated by literature such as [12,13,17], different link functions set their corresponding weights. In this paper, we propose a weighted link-specific (WLS) method to correct the bias of the penalty likelihood estimate of the Poisson and Gamma models. Now, we write $\beta$ as the parameter to be estimated in the one-parameter exponential model and $\widehat{\beta }$ as the corresponding penalized likelihood estimate. However, in high-dimensional space, there is a large estimation error between the $\widehat{\beta }$ estimates obtained using the great likelihood method and the true values, so we propose a new bias correction method. Specifically, to optimize the overall mean squared error, we regress the $\widehat{\beta }$, and the obtained regression coefficients are identified as correction weights. Finally, we establish the asymptotic normality of the WLS estimates and construct CIs, and this weight construction method has independent implications for studying other inference problems under high-dimensional GLMs.

Overall, we find that the WLS method proposed in this paper is superior to other methods in terms of interval coverage and power, and the method in this paper performs robustly and has good generalizability in both sparse and non-sparse scenarios.

The rest of this paper is organized as follows. In the second section, under the condition that the design distribution is unknown, we perform bias correction on individual regression coefficients under high-dimensional GLMs. In the third section, for our proposed confidence interval, its related theoretical properties are given, and asymptotic normality is proved. In Section 4, the superiority of our method is demonstrated by comparing the numerical performance of the proposed method with other methods through numerical simulations. In the fifth section, through the analysis of a real data set, the False Discovery Rate (FDR) and its power line chart of the four methods are drawn to indirectly illustrate the powerful performance of the method. The final section discusses the overall superiority of our approach as well as some areas for improvement.

Notation and terminology. Throughout, for a vector $a={\left(a_1,\dots ,a_n\right)}^T\in R^n$, we define the $l_p$ norm: ${\parallel a\parallel }_p={\left({\sum }_{i=1}^n{\left|a_i\right|}^p\right)}^{1/p}$, the $l_0$ norm: ${\parallel a\parallel }_0={\sum }^{n}1\left\{a_i\ne 0\right\}$, and the $l_{\infty }$ norm: ${\parallel a\parallel }_{\infty }={max}_{1\le j\le n}\left|a_i\right|$, and let $a_{-j}\in R^{n-1}$ stand for the subvector of a without the j-th component. For a matrix $A\in R^{p\times q}$, ${\lambda }_i\left(A\right)$ stands for the i-th largest singular value of A and ${\lambda }_{max}\left(A\right)={\lambda }_1\left(A\right), {\lambda }_{min}\left(A\right)={\lambda }_{min\{p,q\}}\left(A\right)$. For a smooth function $f\left(x\right)$ defined on R, we denote $f^{\prime }\left(x\right)=df\left(x\right)/dx$ and $f^{″}\left(x\right)=d^{2}f\left(x\right)/d{x}^2$. For any positive integer n, we denote the set $\{1,2,\dots ,n\}$ as $[1:n]$. For any $a,b\in R$, we denote $I_x(a,b)=B(x;a,b)/B(a,b)$, as the regularized incomplete beta function, where $B(x,a,b)={\int }_0^x{t}^{a-1}{(1-t)}^{b-1}dt$ is the incomplete beta function. We define $\varphi \left(x\right)$ and $\Phi \left(x\right)$ as the density function and the cumulative distribution function(CDF) of the standard Gaussian random variable, respectively. We denote ${\to }_d$ as convergence in distribution. For positive sequences $\left\{a_n\right\}$ and $\left\{b_n\right\}$, we write $a_n=o\left(b_n\right)$, $a_n\ll b_n$ or $b_n\gg a_n$ if ${lim}_n{a}_n/b_n=0$, and write $a_n=O\left(b_n\right),a_n\le b_n$ or $b_n\ge a_n$ if there exists a constant $C$ such that $a_n\le C{b}_n$ for all n, we write $a_n\approx b_n$ if $a_n\le b_n$ and $a_n\ge b_n$.

2. Weighted Link-Specific Method of Generalized Linear Model

In GLM, the standard form of the exponential distribution family is as follows,

$$f(y;\theta ,\varphi )=exp\left\{\frac{\theta y-b\left(\theta \right)}{a\left(\varphi \right)}+c(y,\varphi )\right\},$$

where $a\left(\varphi \right)$ is the known dispersion function, $\theta$ and $\varphi$ denote the natural and dispersion parameters, respectively, while $\theta$ is only related to the expected u, i.e., $\theta =f\left(u\right)$. In addition, for the simplicity of subsequent sample split, we collect $2n$ samples $D={\left\{\left(X_i,y_i\right)\right\}}_{i=1}^{2n}$. In this paper, we mainly focus on Poisson and Gamma distributions and aim to establish a unified statistical inference framework for them in high-dimensional spaces.

2.1. Poisson Regression

For the Poisson distribution, we assume that its observations are independently generated by Formula (1),

$$y_i\mid X_i\sim Poisson\left(f\left(X_i^T\beta \right)\right),X_i\sim P_X,$$

(1)

where $f:R\to (0,1)$ is the link function given in advance, $\beta \in R^p$ is a highly sparse regression vector, and its sparsity is denoted by k and $P_x$ is some probability distribution. In the subsection, we denote the link function as $f\left(x\right)=exp\left(x\right)/(1+exp(x\left)\right)$ and define $l_f\left(\beta \right)$ as the likelihood function,

$$l_f\left(\beta \right)=-\frac{1}{n}\sum _{i=1}^n\left[y_{i}lnf\left(X_i^T\beta \right)-f\left(X_i^T\beta \right)\right],$$

and $l_1$-penalized negative log-likelihood estimator under GLM is as follows,

$$\widehat{\beta }=\underset{\beta }{argmin}\left\{l_f\left(\beta \right)+\lambda {\parallel \beta \parallel }_1\right\},$$

(2)

where $\lambda \approx \sqrt{logp/n}$. Although $\widehat{\beta }$ achieves the best convergence rate [21,22], there is a large error between the penalty likelihood estimator and the real value which could be reduced. Specifically, to optimize the overall MSE, we regress the deviation of parameters on the original estimation error and determine the obtained regression coefficients as the correction weights.

Since the correction method involves a penalized likelihood estimation step and weighting step, we divide the $2n$ samples into two independent data sets $D_1={\left\{\left(X_i,y_i\right)\right\}}_{i=1}^n,$$D_2={\left\{\left(X_i,y_i\right)\right\}}_{i=n+1}^{2n}$ and perform penalized likelihood estimation on $D_1$. The splitting process of samples can better promote the relevant theoretical analysis [17], and the numerical performance obtained is also good enough compared with the non-splitting sample. Perhaps this is not a practical limitation, and so we weighted it on $D_2$.

For a given $j\in [1:p]$, we consider the general form of the bias correction estimate as follows [17,23]:

$${\tilde{\beta }}_j={\widehat{\beta }}_j+u^T{n}^{-1}\sum _i^n{W}_i·X_i\left(y_i-f\left(X_i^T\widehat{\beta }\right)\right),$$

(3)

where we note that $\widehat{R}=\frac{1}{n}{\sum }_{i=1}^n{W}_i·X_i\left(y_i-f\left(X_i^T\widehat{\beta }\right)\right)$, and ${\widehat{\beta }}_j$ is the penalty likelihood estimate defined in (2). $W_i\in R, 1\le i\le n$ and $u\in R^p$ denote the link-specific weight vector and projection direction to be constructed, respectively, such that $\widehat{R}$ is an accurate estimate of the deviation ${\tilde{\beta }}_j-{\beta }_j$. Then, we decompose the error of Formula (3), which can help us to construct the weight vector and projection direction. The model (1) can be re-expressed as

$$y_i=f\left(X_i^T\beta \right)+{\epsilon }_i,E\left({\epsilon }_i\mid X_i\right)=0,$$

(4)

and we carry out Taylor expansion near $X_i^T\widehat{\beta }$ as follows

$$\frac{1}{n}\sum _{i=1}^n{W}_i{X}_i\left(y_i-f\left(X_i^T\widehat{\beta }\right)\right)=\frac{1}{n}\sum _{i=1}^n{W}_i{X}_i{\epsilon }_i+\frac{1}{n}\sum _{i=1}^n{W}_i{f}^{\prime }\left(X_i^T\widehat{\beta }\right)X_i{X}_i^T(\beta -\widehat{\beta })+\frac{1}{n}\sum _{i=1}^n{W}_i{X}_i{\Delta }_i,$$

(5)

where ${\Delta }_i=f^{″}\left(X_i^T\widehat{\beta }+t{X}_i^T(\beta -\widehat{\beta })\right)·{\left[X_i^T(\widehat{\beta }-\beta )\right]}^2,t\in (0,1)$. Combining (3) and (5), we can get the estimated error expression of ${\tilde{\beta }}_j-{\beta }_j$

$$\begin{array}{l}{\tilde{\beta }}_j-{\beta }_j\\ ={\widehat{\beta }}_j-{\beta }_j+u^T{n}^{-1}{\displaystyle \sum _{i=1}^n}{W}_i·X_i\left(y_i-f\left(X_i^T\widehat{\beta }\right)\right)\\ =\left({\widehat{\beta }}_j-{\beta }_j\right)+u^T\frac{1}{n}{\displaystyle \sum _{i=1}^n}{W}_i{X}_i{\epsilon }_i+u^T\frac{1}{n}{\displaystyle \sum _{i=1}^n}{W}_i{f}^{\prime }\left(X_i^T\widehat{\beta }\right)X_i{X}_i^T(\beta -\widehat{\beta })+u^T\frac{1}{n}{\displaystyle \sum _{i=1}^n}{W}_i{X}_i{\Delta }_i\\ =u^T\frac{1}{n}{\displaystyle \sum _{i=1}^n}{W}_i{X}_i{\epsilon }_i+\left(u^T\frac{1}{n}{\displaystyle \sum _{i=1}^n}{W}_i{f}^{\prime }\left(X_i^T\widehat{\beta }\right)X_i{X}_i^T-e_i^T\right)(\beta -\widehat{\beta })+u^T\frac{1}{n}{\displaystyle \sum _{i=1}^n}{W}_i{X}_i{\Delta }_i.\end{array},$$

(6)

where ${\left\{e_i\right\}}_{i=1}^p$ is the standard basis of the Euclidean space $R^p$. In the above Formula (6), the bias can be divided into the sum of three errors. The first part is the random error due to the model error ${\epsilon }_i$, the second part is the residual bias due to the penalized likelihood estimator $\widehat{\beta }$, and the last part is the approximation error due to the nonlinearity of the link function f.

We give two optimization criteria to determine the weight vector and the projection vector. First, the random error in (6) must satisfy asymptotic normality, and its standard error must reach a minimum value. Second, the residual bias and approximation error of the latter two items are negligible compared to the random error.

To satisfy these two conditions, we first construct the weight vector and then the projection direction. The variance of the random error in Formula (6) is

$$Var\left(u^T\frac{1}{n}\sum _{i=1}^n{W}_i·X_i{\epsilon }_i\mid {\left\{X_i\right\}}_{i=1}^n\right)=u^T\frac{1}{n^2}\sum _{i=1}^n{W}_i^2·Var\left({\epsilon }_i\mid X_i\right)X_i{X}_i^{T}u,$$

(7)

and according to Hölder’s inequality, the upper bound of the residual deviation in (6) can be obtained:

$$\left|\left(u^T\frac{1}{n}\sum _{i=1}^n{W}_i{f}^{\prime }\left(X_i^T\widehat{\beta }\right)X_i{X}_i^T-e_j^T\right)(\beta -\widehat{\beta })\right|\le {∥\frac{1}{n}\sum _{i=1}^n{W}_i{f}^{\prime }\left(X_i^T\widehat{\beta }\right)X_i{X}_i^{T}u-e_j∥}_{\infty }{\parallel \beta -\widehat{\beta }\parallel }_1.$$

(8)

According to [8,17], in order to achieve the constraint that the first random error in Formula (6) dominates and the next two terms are relatively negligible, it is sufficient to make the right-hand part in (7) approximately equal to the right-hand part in (8),

$$W_i^2·Var\left({\epsilon }_i\mid X_i\right)X_i{X}_i^{T}u\approx W_i{f}^{\prime }\left(X_i^T\widehat{\beta }\right)X_i{X}_i^{T}u,$$

also known as,

$$W_i^2·Var\left({\epsilon }_i\mid X_i\right)\approx W_i{f}^{\prime }\left(X_i^T\widehat{\beta }\right),$$

(9)

then, the construction of the weighted vector is obtained as $W_i=w\left(X_i^T\widehat{\beta }\right)$ with

$$w\left(z\right)=\frac{f^{\prime }\left(z\right)}{Var\left({\epsilon }_i\mid X_i\right)},$$

(10)

the above weight vector constructed from Formula (10) above ensures that the first random error dominates in Formula (6). Next, we record $\widehat{\Sigma }=\frac{1}{n}{\sum }_{i=1}^{n}w\left(X_i^T\widehat{\beta }\right)·f^{\prime }\left(X_i^T\widehat{\beta }\right)X_i{X}_i^T$ and use the optimization method to estimate the projection direction as

$$\widehat{u}=argmin{u}^T\widehat{\Sigma }u,$$

subject to

$$\begin{array}{l}{∥\widehat{\Sigma }u-X_j∥}_{\infty }\le {∥X_j∥}_2{\lambda }_n,\\ ∥X_j^T\widehat{\Sigma }u-∥X_j∥{}_2^2\mid \le ∥X_j{\parallel }_2^2{\lambda }_n\\ {\parallel Xu\parallel }_{\infty }\le {∥X_j∥}_2{\tau }_n,\end{array},$$

(11)

where ${\lambda }_n\approx \sqrt{logp/n},{\tau }_n\approx \sqrt{logn}$. The constraints in Formula (11) above are used to control the errors in the second and third terms of Formula (6), so this projection could ensure that the deviation ${\tilde{\beta }}_j-{\beta }_j$ is very small [23,24]. Then, the bias correction estimator is

$${\tilde{\beta }}_j={\widehat{\beta }}_j+{\widehat{u}}^T\frac{1}{n}\sum _{i=1}^{n}w\left(X_i^T\widehat{\beta }\right)·\left(y_i-f\left(X_i^T\widehat{\beta }\right)\right)X_i.$$

(12)

2.2. Gamma Regression

For the Gamma distribution, we assume that its observations are independently generated by Formula (13),

$$y_i\mid X_i\sim Gamma\left({\alpha }_0,\beta \right),X_i\sim P_X,$$

(13)

where ${\alpha }_0$ is a known parameter. The correction estimates under Gamma regression are essentially the same as the Poisson regression correction method. First, we determine the link function, assuming that the samples come from (13); then, there is

$$f(y;\beta )=\frac{y^{{\alpha }_0-1}exp(-y/\beta )}{{\beta }^{{\alpha }_0}\Gamma \left({\alpha }_0\right)}.$$

(14)

It is easy to know that the expectation is $u={\alpha }_0·\beta$; now, record $\varphi =1/{\alpha }_0$, then ${\alpha }_0=1/\varphi$ as well as $\beta =u·\varphi$. Then, (14) can be simplified to a standard exponential family of distributions as follows,

$$\begin{array}{ll}f(y;\beta )=& f(y;u,\varphi )=\frac{y^{{\alpha }_0-1}exp(-y/\beta )}{{\beta }^{{\alpha }_0}\Gamma \left({\alpha }_0\right)}\\ & =exp\left\{-\frac{y}{\beta }+\left({\alpha }_0-1\right)lny-{\alpha }_{0}ln\beta -ln\Gamma \left({\alpha }_0\right)\right\}\\ & =exp\left\{-\frac{y}{u\varphi }-\frac{ln\left(u\varphi \right)}{\varphi }+\left(\frac{1}{\varphi }-1\right)lny-ln\Gamma \left(\frac{1}{\varphi }\right)\right\}\\ & =exp\left\{\frac{y·u^{-1}-(-lnu)}{-\varphi }-\frac{ln\varphi }{\varphi }+\frac{1-\varphi }{\varphi }lny-ln\Gamma \left({\varphi }^{-1}\right)\right\}.\end{array}$$

(15)

The link function is $f_1\left(u\right)=\theta =\frac{1}{u}$, but to satisfy the regularity assumption in this paper, the link function is constructed as $f\left(x\right)=\frac{1}{1+x}$. Next, we compute its negative log-likelihood function as follows,

$$l_f\left(\beta \right)=-\frac{1}{n}\sum _{i=1}^n\left\{\frac{y_{i}f\left(X_i^T\beta \right)+ln{f}^{-1}\left(X_i^T\beta \right)}{-\lambda }\right\}.$$

Finally, from Equation (2) to Equation (12), the bias correction estimate can be calculated in the same form as (12).

3. Theoretical Properties

3.1. Asymptotic Normality

In the subsection, we prove the asymptotic normality of the parameters and construct the associated CIs. First, we give a set of mildness conditions [12,13,25] that the link function needs to satisfy. For link function $f:R\to (0,1)$, we have the following:

• The connection function f is a monotone, second-order differentiable concave function on $\mathrm{R}\left(R_{+}\right)$;
• There is a normal constant $C_1,C_2$ such that for all $x\ge 0,f\left(x\right)\le \Phi \left(C_{1}x\right)$, $\Phi \left(x\right)$ is a standard Gaussian distribution, and $max\left\{\frac{f^{\prime }\left(x\right)}{x(1-f(x\left)\right)},x^2{f}^{\prime }\left(x\right)\right\}<C_2$;
• There exists a constant $C_1,C_2$, such that $sup\left|x{f}^{″}(x+w)\right|\le C_1,\left|w\right|<C_2$;
• For $l_f\left(\beta \right)$ defined by Formula (2), there is a constant C, which can make the Hessian matrix expressed as $l_f^{″}\left(\beta \right)=\frac{1}{n}{\sum }_{i=1}^{n}h\left(\beta ;y_i,X_i\right)X_i{X}_i^T$, with $h\left(\beta ;y_i,X_i\right)>0$ and

  $$max\left|logh\left(\beta +b;y_i,X_i\right)-logh\left(\beta ;y_i,X_i\right)\right|\le C\left({\left|X_i^T\beta \right|}^2+{\left|X_i^{T}b\right|}^2+\left|X_i^{T}b\right|\right).$$
  None of the above four conditions is very strict, and a large number of link functions can satisfy them, where 1∼3 are relatively easy to verify, while condition 4 is from Huang [26]. Second, for the random design variables and their distributions, we assume that
• ${\left\{X_i\right\}}_{1\le i\le 2n}$ is an independent and identically distributed sub-Gaussian random vector, that is, there is a constant $C\in R$ that satisfies $E\left[exp\left(v^{T}X\right)\right]\le exp\left({\parallel v\parallel }_2^2{c}^2/2\right),v\in R^p$.

It can be found that if $X_{i1}=1,1\le i\le 2n,\Sigma =E\left(X_i{X}_i^T\right)\in R^{p\times p}$, ${\beta }_1$ is its intercept as well as k is its sparsity level, then there exists a parameter space associated with k as follows.

$$\begin{array}{c}\Theta \left(k\right)=\left\{\theta ={(\beta ,\Sigma ):\parallel \beta \parallel }_0\le k,{\parallel \beta \parallel }_2\le C,\right.\\ \left.M^{-1}\le {\lambda }_{min}(\Sigma )\le {\lambda }_{max}(\Sigma )\le M\right\},\end{array}$$

where $M>1,C>0$.

In a high-dimensional space, the size of sparsity k can easily affect the theoretical distribution of the bias corrector $\tilde{\beta }$, so we build a parameter space $\Theta \left(k\right)$ associated with k. Under this parameter space, we present Theorem 1, which establishes the asymptotic properties of a bias-corrected estimator $\tilde{\beta }$.

Theorem 1. 

Assuming that the conditions 1∼5 are established, $(\beta ,\Sigma )\in \Theta \left(k\right)$, then for any $j\in [1:p]$, if

(1) 

$k<<\frac{n}{lognlogp}$, then we have ${\tilde{\beta }}_j-{\beta }_j=A_n+o\left(p\right)$, and $\frac{\sqrt{n}{A}_n}{v_j^{1/2}}{\to }_{d}N(0,1)$,

(2) 

$k<<\frac{\sqrt{n}}{logp\sqrt{logn}}$, then $\frac{\sqrt{n}\left({\tilde{\beta }}_j-{\beta }_j\right)}{v_j^{1/2}}{\to }_{d}N(0,1)$.

Proof of Theorem 1. 

By the definition of ${\tilde{\beta }}_j$, we have

$$\begin{array}{ll}{\tilde{\beta }}_j-{\beta }_j=& -\left(\frac{1}{n}{\displaystyle \sum _{i=1}^n}w\left(X_i^T\widehat{\beta }\right)f^{\prime }\left(X_i^T\widehat{\beta }\right){\widehat{u}}^T{X}_i{X}_i^T\widehat{\beta }-e_j^T\right)(\widehat{\beta }-\beta )\\ & +\frac{1}{n}{\displaystyle \sum _{i=1}^n}w\left(X_i^T\widehat{\beta }\right){\widehat{u}}^T{X}_i{\epsilon }_i-\frac{1}{n}{\displaystyle \sum _{i=1}^n}{\Delta }_{i}w\left(X_i^T\widehat{\beta }\right){\widehat{u}}^T{X}_i,\end{array}$$

(16)

where we denote

$$A_n=\frac{1}{n}\sum _{i=1}^{n}w\left(X_i^T\widehat{\beta }\right){\widehat{u}}^T{X}_i{\epsilon }_i,$$

and

$$B_n=-\left(\frac{1}{n}\sum _{i-1}^{n}w\left(X_i^T\widehat{\beta }\right)f^{\prime }\left(X_i^T\widehat{\beta }\right){\widehat{u}}^T{X}_i{X}_i^T-e_j^T\right)(\widehat{\beta }-\beta )-\frac{1}{n}\sum _{i-1}^n{\Delta }_{i}w\left(X_i^T\widehat{\beta }\right)u^T{X}_i.$$

In what follows, on the one hand, we show that, in Formula (15), under the conditions of Theorem 1, with probability at least $1-p^{-c}-n^{-c}$,

$$\begin{array}{l}{∥\frac{1}{n}{\widehat{u}}^T\sum _{i=1}^{n}w\left(X_i^T\widehat{\beta }\right)f^{\prime }\left(X_i^T\widehat{\beta }\right)X_i{X}_i^T-e_j^T∥}_{\infty }\le {\lambda }_n,\\ \left|\frac{1}{n}\sum _{i=1}^n{\Delta }_{i}w\left(X_i^T\widehat{\beta }\right){\widehat{u}}^T{X}_i\right|\le {\tau }_n\frac{klogp}{n},\end{array}$$

(17)

furthermore, we have with probability at least $1-p^{-c}-n^{-c}$,

$$\begin{array}{l}\left|\left(\frac{1}{n}\sum _{i=1}^{n}w\left(X_i^T\widehat{\beta }\right)f^{\prime }\left(X_i^T\widehat{\beta }\right){\widehat{u}}^T{X}_i{X}_i^T-e_j^T\right)(\widehat{\beta }-\beta )\right|\\ \le {\lambda }_n{\parallel \widehat{\beta }-\beta \parallel }_1\le \frac{klogp}{n},\end{array}$$

(18)

which along with (16) leads to the upper bound of $B_n$. On the other hand, the asymptotic normality of the stochastic term $A_n$ in (16) can be derived from the following formulation.

Under the conditions of $D_2$ and ${\left\{X_i\right\}}_{i=1}^n$, it can be assumed that:

$$v_j^{-1/2}\frac{1}{\sqrt{n}}\sum _{i=1}^{n}w\left(X_i^T\widehat{\beta }\right){\widehat{u}}^T{X}_i{\epsilon }_i{\to }_{d}N(0,1)$$

The major contribution of Theorem 1 is that it removes many of the strict conditions in high-dimensional GLM inference, such as the bounded individual probability condition and the bounded design condition, greatly relaxing the preconditions for a wider range of situations. □

3.2. Confidence Interval

Under the mild regularity conditions in Section 3.1, ${\tilde{\beta }}_j$ in (12) has the following asymptotic variance:

$$v_j={\widehat{u}}^T\left[\frac{1}{n}\sum _{i=1}^n{w}^2\left(X_i^T\widehat{\beta }\right)Var\left({\epsilon }_i\mid X_i\right)X_i{X}_i^T\right]\widehat{u}.$$

At this point, the variance of the error ${\mathcal{E}}_i$ can be estimated as

$$\widehat{Var}\left({\epsilon }_i\mid X_i\right)=f\left(X_i^T\widehat{\beta }\right)\left(1-f\left(X_i^T\widehat{\beta }\right)\right),$$

then, the variance of ${\beta }_j$ is estimated as follows,

$${\widehat{v}}_j={\widehat{u}}^T\left[\frac{1}{n}\sum _{i=1}^n\frac{{\left[f^{\prime }\left(X_i^T\widehat{\beta }\right)\right]}^2}{\widehat{V}ar\left({\epsilon }_i\mid X_i\right)}{X}_i{X}_i^T\right]\widehat{u}.$$

Based on this, we construct CIs for the regression coefficient ${\beta }_j$ at the $(1-\alpha )$ confidence level as

$$CI\left({\beta }_j,D\right)=\left[{\tilde{\beta }}_j-{\tilde{\rho }}_j,{\tilde{\beta }}_j+{\tilde{\rho }}_j\right],$$

(19)

where ${\tilde{\rho }}_j=\frac{z_{\alpha /2}{\widehat{v}}_j^{1/2}}{\sqrt{n}},z_{\alpha /2}={\Phi }^{-1}(1-\alpha /2)$. Note that it is necessary for sparsity in the confidence interval given in Formula (18): $k<<\frac{\sqrt{n}}{logp\sqrt{logn}}$. In this ultra-sparse state, Formula (18) holds, while for the non-ultra-sparse state, the above formula will not hold with high probability.

4. Simulations

In this section, we evaluate the numerical performance of the proposed method and compare it with other inference methods under high-dimensional GLMs. For the evaluation of CIs, we mainly consider the coverage of regression coefficients and the length of CIs.

CIs for High-Dimensional Poisson and Gamma Regression

We build CIs from Poisson regression and Gamma regression. We set n = 100; let p vary between 400 and 1300, and let the sparsity level k vary between 25 and 35. For the real regression coefficient, given a support set $S,\left|S\right|=k$, we set $\left|{\beta }_j\right|=\psi I_{\{j\in S\}},j=1,\dots ,p$, and the ratio of $\psi$ to $-\psi$ is equal. For the design covariates $X_i$, generated by a multivariate Gaussian distribution, its covariance matrix $\Sigma ={\Sigma }_M$ is a block diagonal matrix consisting of 10 identical unit diagonal Toeplitz matrices, with off-diagonal entries falling from 0.6 to 0.

In the subsection, we construct the confidence interval of the non-zero regression coefficient ${\beta }_2=-\psi$ and the zero regression coefficient ${\beta }_{100}=0$ and set the desired confidence level to $95\%$. We compare three methods separately to evaluate their numerical performance: (i) the WLS method proposed in this paper; (ii) based on the weighted low-dimensional projection method (WLP) proposed by [12]; (iii) finally, the original Lasso estimate is used for comparison. The comparison regression results of zero coefficients and non-zero coefficients of Poisson regression are summarized in

Table 1. Empirical performances of CIs for ${\beta }_2$ under $\Sigma ={\Sigma }_M,\psi =0.5,\alpha =0.05,n=100$ in Poisson.

p	Coverage(%)			Length
	WLS	WLP	Lasso	WLS	WLP	Lasso
k = 20
400	93.0	9.2	10.2	1.12	0.41	0.08
700	91.0	8.5	6.5	1.11	0.41	0.07
1000	96.0	10.2	4.5	1.13	0.40	0.07
1300	93.0	10.0	4.05	0.97	0.41	0.06
k = 25
400	95.0	3.0	10.3	1.15	0.40	0.07
700	90.0	1.9	3.04	1.14	0.41	0.07
1000	98.0	1.2	2.23	1.34	0.41	0.07
1300	91.0	1.0	1.05	0.98	0.40	0.06
k = 35
400	95.0	6.8	9.38	1.15	0.41	0.08
700	95.3	4.1	6.46	0.95	0.41	0.08
1000	96.8	1.2	5.31	1.35	0.41	0.06
1300	93.0	1.1	2.24	0.97	0.40	0.06

and

Table 2. Empirical performances of CIs for ${\beta }_{100}$ under $\Sigma ={\Sigma }_M,\psi =0.5,\alpha =0.05,n=100$ in Poisson.

p	Coverage(%)			Length
	WLS	WLP	Lasso	WLS	WLP	Lasso
k = 20
400	95.0	99.0	16.3	1.11	0.41	0.08
700	99.3	99.2	13.2	1.23	0.40	0.07
1000	99.2	99.5	23.3	1.13	0.41	0.08
1300	99.6	99.8	24.2	1.02	0.40	0.06
k = 25
400	97.0	94.1	11.3	1.15	0.41	0.07
700	96.0	95.2	27.1	1.23	0.40	0.08
1000	99.5	96.1	28.6	1.26	0.41	0.09
1300	96.0	98.2	37.6	1.03	0.41	0.10
k = 35
400	95.0	94.1	21.56	1.16	0.41	0.09
700	96.9	96.4	34.8	1.03	0.41	0.08
1000	98.7	98.6	33.4	1.15	0.41	0.07
1300	99.5	98.9	35.3	1.03	0.40	0.06

, and the comparison regression results of zero coefficients and non-zero coefficients of Gamma regression are summarized in

Table 3. Empirical performances of CIs for ${\beta }_2$ under $\Sigma ={\Sigma }_M,\psi =0.5,\alpha =0.05,n=100$ in Gamma.

p	Coverage(%)			Length
	WLS	WLP	Lasso	WLS	WLP	Lasso
k = 20
400	93.4	6.1	11.12	0.99	0.40	0.08
700	94.2	14.2	8.01	0.84	0.41	0.07
1000	92.5	13.2	3.66	0.85	0.41	0.08
1300	93.2	13.0	1.03	0.86	0.40	0.07
k = 25
400	95.1	3.2	4.23	0.98	0.41	0.07
700	89.3	2.1	2.04	0.84	0.41	0.07
1000	91.7	1.1	3.26	0.87	0.40	0.08
1300	92.3	1.0	2.06	0.89	0.40	0.07
k = 35
400	86.2	7.1	5.07	0.87	0.41	0.08
700	86.3	6.2	1.09	0.85	0.41	0.07
1000	80.4	1.2	1.03	0.78	0.41	0.07
1300	82.3	1.1	1.01	0.80	0.40	0.07

and

Table 4. Empirical performances of CIs for ${\beta }_{100}$ under $\Sigma ={\Sigma }_M,\psi =0.5,\alpha =0.05,n=100$ in Gamma.

p	Coverage(%)			Length
	WLS	WLP	Lasso	WLS	WLP	Lasso
k = 20
400	96.2	99.1	10.02	0.98	0.41	0.08
700	98.3	98.2	24.02	0.97	0.41	0.07
1000	96.3	97.3	27.56	0.89	0.41	0.08
1300	98.7	98.0	26.04	0.87	0.40	0.07
k = 25
400	95.2	95.0	11.25	0.98	0.41	0.08
700	99.2	96.8	17.06	0.91	0.41	0.07
1000	99.3	97.3	26.48	0.94	0.40	0.08
1300	99.6	98.1	35.76	0.95	0.41	0.07
k = 35
400	98.6	96.2	20.0	0.88	0.40	0.07
700	99.2	97.3	26.3	0.90	0.40	0.08
1000	98.6	98.1	51.05	0.80	0.41	0.07
1300	99.3	98.3	35.32	0.85	0.41	0.07

, where each entry represents the average simulation results for 100 rounds.

In the non-sparse space setting, the results in Table 1 and Table 3 show that the coverage obtained by using the WLS method for bias correction is much better than that of the WLP and Lasso methods in both Poisson and Gamma regressions. Although the coverage obtained by the WLP method is longer, it is worthwhile to ensure its accuracy while sacrificing some precision. Moreover, it can be seen that the interval coverage obtained by the WLP and Lasso methods is so small that it fails almost completely, and perhaps there are other limitations in applying them in sparse scenarios.

In the sparse space setting, the results in Table 2 and Table 4 show that both the WLS method and the WLP method can achieve the interval coverage we expect. It can be seen that the interval lengths obtained by the WLP method are generally shorter than those of the WLS method, and their accuracy is higher. However, in our setting, the WLP method produces significantly higher Type II errors than the WLS method, and its precision decreases, while the Lasso method still performs poorly and does not meet our expectations.

In summary, our method is highly adaptable and robust. Under sparse conditions, our method is quite similar to existing comparison methods, but our method is more applicable to some cases where low Type II error rates are retained.

5. Real Data Analysis

We analyze a single-cell RNA-seq dataset in [27], which contains the expression estimates (transcripts per million) for all the 27,723 UCSC-annotated mouse genes of a total of 1861 primary mouse bone marrow-derived dendritic cells spanning over several experimental conditions. The complete dataset was downloaded from the Gene Expression Omnibus with the accession code GSE48968.

5.1. False Discovery Rate and Power Comparison

We apply the WLS method and the four comparison methods to the fit high-dimensional Poisson regression model (1) and Gamma regression model (13), respectively, to obtain FDR plots and power plots for the five methods. Specifically, taking model (1) as an example. WLS regression is the method proposed in this paper. Weighted low-dimensional projection regression (WLP) was proposed in [12], while the remaining “Lasso” regression, “rproj” regression, and “lproj” regression can be found in the relevant software packages of R language software. By comparing Figure 1 and Figure 2, we can see the advantages of our proposed method.

In Figure 1 and Figure 2, we can see that all the methods can control the FDR, but the original Lasso method is conservative and has the weakest efficacy, while methods such as “lproj” and “rproj” have improved but have not increased much, and the power obtained by WLS based on WLP is much higher than other methods. In addition, our proposed WLS results are the most outstanding, showing the great superiority of our approach, which is closely related to the fact that our proposed method has no strict limitation on the data, and this trend becomes increasingly obvious in the Gamma distribution.

5.2. CIs for the Different Stimuli

In the subsection, we use the above data set as a basis to derive the CI produced by different stimuli. This experiment is mainly used to analyre the effect of different stimuli on gene expression. Specifically, we focus on three pathogenic components, namely LPS (a component of Gram-negative bacteria), PIC (virulent double-stranded RNA), and PAM (a synthetic mimic of bacterial lipopeptides), and a set of unstimulated control cells. Sample screening is an important step, and we plot the gene expression profiles of stimulated cells at 0, 1, 2, 4, and 6. For better experimentation, we only consider data at 6 because the variance in this state is the most pronounced. We also remove gene samples with zero counts over $80\%$ and those with significant variance changes, etc. Finally, we fit a high-dimensional Poisson regression model (1) and apply the proposed method to obtain the $95\%$ CIs of each regression coefficient.

Specifically, there are three stimuli in the experiments in this paper: LPS, PIC, and PAM stimuli. Different stimuli are assigned to different sub-datasets: Sdata, Cdata, and Mdata. As an example, consider the PAM stimulus: n = 160, p = 768. The experiments of model (1) are performed on the Mdata data set, and parameter $\beta$ represents the PAM stimulus. We use the WLS method and other comparison methods (WLP, “rproj,” “lproj”) to calculate the confidence interval for each gene under that stimulus, allowing us to compare the performance of different genes under different conditions, as shown in the figure below.

As can be seen in Figure 3, the PAM stimuli obtained by the WLS, WLP, and “lproj” methods all have CI values for one or more genes whose regression coefficients do not cover 0 (i.e., the red part of the interval). This indicates a potential functional response to the stimulus; specifically, for the PAM-stimulated cells, it identifies the specific gene that codes for the protein. However, for the “rproj” method, the regression coefficients for all genes in response to the PAM stimulus cover the value of 0, which is probably related to its specific stimulus.

In the PAM stimulation experiment, we draw the interval length images using four methods. The average interval lengths of all genes produced by WLS, WLP, “lproj”, and “rproj” were calculated to be 1.5, 1.2, 3.5, and 3.5, respectively. The above computational results and images show that the WLP method usually produces shorter CIs and does not cover many genes with a value of 0. On the other hand, the “lproj” method usually highlights similar genes and has a longer CI coverage. Finally, the “rproj” method also tends to produce long CIs and cover zero genes.

6. Discussion

In this paper, we propose a unified framework to perform debiased estimation for distributions with different connectivity functions while constructing confidence intervals under sparse and non-sparse conditions to ensure the validity of interval coverage and coverage length. For technical reasons, we use sample splitting means to establish the relevant theoretical properties. In Van de Geer et al. [28], the random errors (6) are guaranteed to be asymptotically normal if conditions similar to theirs are imposed, and such conditions allow us to establish the relevant theoretical properties without sample partitioning. However, these strict conditions limit the adaptability of the proposed method, so we use sample splitting means to remove other strong assumptions to present our results. Moreover, we cannot guarantee that the proposed method will perform worse without the sample segmentation condition. Therefore, it is of interest to develop a new technical tool to perform inference experiments without sample segmentation.