# Variable Selection for the Spatial Autoregressive Model with Autoregressive Disturbances

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Model and Variable Selection

#### 2.1. The SARAR Model

#### 2.2. Penalized Method

- HARD thresholding penalty function:$${p}_{\lambda}\left(\left|\vartheta \right|\right)={\lambda}^{2}-{\left(\right|\vartheta |-\lambda )}^{2}\mathrm{I}\left(\right|\vartheta |<\lambda ).$$
- ${L}_{1}$ penalty function:$${p}_{\lambda}\left(\left|\vartheta \right|\right)=\lambda \left|\vartheta \right|.$$

- (1)
- Unbiasedness: The resulting estimator is nearly unbiased when the true unknown parameter is large to avoid unnecessary modeling bias;
- (2)
- Sparsity: The resulting estimator automatically sets small estimated coefficients to zero to reduce model complexity;
- (3)
- Continuity: The resulting estimator is continuous in data to avoid instability in the model prediction.

#### 2.3. Main Results

**Theorem**

**1.**

**Theorem**

**2.**

- (i) Sparsity: ${\widehat{\mathit{\theta}}}_{2}=\mathit{0}$;
- (ii) Asymptotic normality:$$\sqrt{n}\left\{\left({\sum}_{n1}\left({\mathit{\theta}}_{10}\right)+\mathsf{\Lambda}\right)\left({\widehat{\mathit{\theta}}}_{1}-{\mathit{\theta}}_{10}\right)+\mathit{d}\right\}\stackrel{d}{\to}N\left\{\mathit{0},{\sum}_{1}\left({\mathit{\theta}}_{10}\right)+{\mathsf{\Omega}}_{1}\left({\mathit{\theta}}_{10}\right)\right\},$$

## 3. Algorithm Design and Implementation

## 4. Numerical Simulation

#### 4.1. Simulation Sampling

#### 4.2. Simulation Results

## 5. Data Example

#### 5.1. The Sample Data

#### 5.2. Spatial Dependence Test

#### 5.3. Model Selection and Estimation

## 6. Summary and Discussion

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Conflicts of Interest

## Appendix A. Assumptions

**Assumption**

**A1.**

**Assumption**

**A2.**

**Assumption**

**A3.**

**Assumption**

**A4.**

**Assumption**

**A5.**

**Assumption**

**A6.**

**Assumption**

**A7.**

**Assumption**

**A8.**

**Assumption**

**A9.**

## Appendix B. Proofs of Theorems 1 and 2

**Lemma**

**A1.**

**Lemma**

**A2.**

**Lemma**

**A3.**

**Proof of**

**Lemma A1.**

**Proof of**

**Lemma A2.**

**Proof of**

**Theorem 1.**

**Proof of**

**Lemma A3.**

**Proof of**

**Theorem 2.**

## References

- Cliff, A.D.; Ord, J.K. Spatial Autocorrelation; Pion Ltd.: London, UK, 1973. [Google Scholar]
- Kelejian, H.H.; Prucha, I.R. A generalized spatial two-stage least squares procedure for estimating a spatial autoregressive model with autoregressive disturbances. J. Real. Estate Financ.
**1998**, 17, 99–121. [Google Scholar] [CrossRef] - Lee, L.F. Asymptotic distributions of quasi-maximum likelihood estimators for spatial autoregressive models. Econometrica
**2004**, 72, 1899–1925. [Google Scholar] [CrossRef] - Arraiz, I.; Drukker, D.M.; Kelejian, H.H.; Prucha, I.R. A spatial Cliff-Ord-type model with heteroskedastic innovations: Small and large sample results. J. Regional. Sci.
**2010**, 50, 592–614. [Google Scholar] [CrossRef] [Green Version] - Anselin, L. Spatial Econometrics: Methods and Models; Kluwer Academic Publishers: Dordrecht, The Netherlands, 1988. [Google Scholar]
- Cressie, N. Statistics for Spatial Data; John Wiley and Sons: New York, NY, USA, 1993. [Google Scholar]
- Akaike, H. Maximum likelihood identification of Gaussian autoregressive moving average models. Biometrika
**1973**, 60, 255–265. [Google Scholar] [CrossRef] - Schwarz, G. Estimating the dimension of a model. Ann. Stat.
**1978**, 6, 461–464. [Google Scholar] [CrossRef] - Foster, D.P.; George, E.I. The risk inflation criterion for multiple regression. Ann. Stat.
**1994**, 22, 1947–1975. [Google Scholar] [CrossRef] - Liang, H.; Li, R. Variable selection for partially linear models with measurement errors. J. Am. Stat. Assoc.
**2009**, 104, 234–248. [Google Scholar] [CrossRef] [Green Version] - Huo, X.; Ni, X. When do stepwise algorithms meet subset selection criteria? Ann. Stat.
**2007**, 35, 870–887. [Google Scholar] [CrossRef] [Green Version] - Tibshirani, R. Regression shrinkage and selection via the lasso. J. R. Statist. Soc. B
**1996**, 58, 267–288. [Google Scholar] [CrossRef] - Fan, J.; Li, R. Variable selection via nonconcave penalized likelihood and its oracle properties. J. Am. Stat. Assoc.
**2001**, 96, 1348–1360. [Google Scholar] [CrossRef] - Zou, H.; Hastie, T. Regularization and variable selection via the elastic net. J. R. Statist. Soc. B
**2005**, 67, 301–320. [Google Scholar] [CrossRef] [Green Version] - Zou, H. The adaptive Lasso and its oracle properties. J. Am. Stat. Assoc.
**2006**, 101, 1418–1429. [Google Scholar] [CrossRef] [Green Version] - Zhang, C.H. Nearly unbiased variable selection under minimax concave penalty. Ann. Stat.
**2010**, 38, 894–942. [Google Scholar] [CrossRef] [Green Version] - Mitchell, T.J.; Beauchamp, J.J. Bayesian variable selection in linear regression. J. Am. Stat. Assoc.
**1988**, 83, 1023–1032. [Google Scholar] [CrossRef] - Raftery, A.E.; Madigan, D.; Hoeting, J.A. Bayesian model averaging for linear regression models. J. Am. Stat. Assoc.
**1997**, 92, 179–191. [Google Scholar] [CrossRef] - Jiang, W.X. Bayesian variable selection for high dimensional generalized linear models: Convergence rates for the fitted densities. Ann. Stat.
**2007**, 35, 1487–1511. [Google Scholar] [CrossRef] [Green Version] - Chen, Y.; Du, P.; Wang, Y. Variable selection in linear models. Wiley Interdiscip. Rev. Comput. Mol. Sci.
**2014**, 6, 1–9. [Google Scholar] [CrossRef] - Steel, M.F. Model averaging and its use in economics. J. Econ. Lit.
**2020**, 58, 644–719. [Google Scholar] [CrossRef] - LeSage, J.P.; Parent, O. Bayesian model averaging for spatial econometric models. Geogr. Anal.
**2007**, 39, 241–267. [Google Scholar] [CrossRef] - LeSage, J.P.; Fischer, M. Spatial growth regressions, model specification, estimation, and interpretation. Spat. Econ. Anal.
**2008**, 3, 275–304. [Google Scholar] [CrossRef] [Green Version] - Cuaresma, J.C.; Doppelhofer, G.; Feldkircher, M. The determinants of economic growth in European regions. Reg. Stud.
**2014**, 48, 44–67. [Google Scholar] [CrossRef] - Cuaresma, J.C.; Doppelhofer, G.; Huber, F.; Piribauer, P. Human capital accumulation and long-term income growth projections for European regions. J. Regional. Sci.
**2018**, 58, 81–99. [Google Scholar] [CrossRef] [Green Version] - Piribauer, P. Heterogeneity in spatial growth clusters. Empir. Econ.
**2016**, 51, 659–680. [Google Scholar] [CrossRef] - Zhu, J.; Huang, H.; Reyes, P.E. On selection of spatial linear models for lattice data. J. R. Statist. Soc. B
**2010**, 72, 389–402. [Google Scholar] [CrossRef] [Green Version] - Liu, X.; Chen, J.; Cheng, S. A penalized quasi-maximum likelihood method for variable selection in the spatial autoregressive model. Spat. Stat.
**2018**, 25, 86–104. [Google Scholar] [CrossRef] - Xie, T.; Cao, R.; Du, J. Variable selection for spatial autoregressive models with a diverging number of parameters. Stat. Pap.
**2020**, 61, 1125–1145. [Google Scholar] [CrossRef] - Wang, H.; Zhu, J. Variable selection in spatial regression via penalized least squares. Can. J. Stat.
**2009**, 37, 607–624. [Google Scholar] [CrossRef] - Zou, H.; Li, R. One-step sparse estimates in nonconcave penalized likelihood models. Ann. Stat.
**2008**, 36, 1509–1533. [Google Scholar] - Nelder, J.A.; Mead, R. A simplex method for function minimization. Comput. J.
**1965**, 7, 308–313. [Google Scholar] [CrossRef] - Wang, H.; Li, R.; Tsai, C.L. Tuning parameter selectors for the smoothly clipped absolute deviation method. Biometrika
**2007**, 94, 553–568. [Google Scholar] [CrossRef] [PubMed] - Case, A.C. Spatial patterns in household demand. Econometrica
**1991**, 59, 953–965. [Google Scholar] [CrossRef] [Green Version] - Harrison, D.H.; Rubinfeld, D.L. Hedonic housing prices and the demand for clean air. J. Environ. Econ. Manag.
**1978**, 5, 81–102. [Google Scholar] [CrossRef] [Green Version] - Pace, R.K.; Gilley, O.W. Using the spatial configuration of the data to improve estimation. J. Real. Estate Financ.
**1997**, 14, 333–340. [Google Scholar] [CrossRef] - Tang, Q. Robust estimation for functional coefficient regression models with spatial data. Statistics
**2014**, 48, 388–404. [Google Scholar] [CrossRef] - LeSage, J.; Pace, R. Introduction to Spatial Econometrics; CRC Press: Boca Raton, FL, USA, 2009. [Google Scholar]
- Horn, R.A.; Johnson, C.R. Matrix Analysis; Cambridge University Press: Cambridge, UK, 1985. [Google Scholar]
- Kelejian, H.H.; Prucha, I.R. On the asymptotic distribution of the Moran I test statistic with applications. J. Econom.
**2001**, 104, 219–257. [Google Scholar] [CrossRef] [Green Version]

${\mathit{\sigma}}^{2}=1$ | n = 30 | n = 60 | n = 180 | ||||||
---|---|---|---|---|---|---|---|---|---|

Method | C | I | mSE | C | I | mSE | C | I | mSE |

${\rho}_{1}=0.7$ | |||||||||

${\rho}_{2}=0.7$ | |||||||||

SCAD | 3.8300 | 0.1100 | 0.4329 | 4.6000 | 0.0100 | 0.0941 | 5.0000 | 0.0000 | 0.0272 |

Hard | 4.2900 | 0.1100 | 0.4961 | 4.6200 | 0.0100 | 0.1048 | 4.9700 | 0.0000 | 0.0276 |

LASSO | 2.7300 | 0.1600 | 0.7452 | 3.0400 | 0.0600 | 0.2209 | 3.4800 | 0.0000 | 0.0635 |

Oracle | 5.0000 | 0.0000 | 0.0909 | 5.0000 | 0.0000 | 0.0300 | 5.0000 | 0.0000 | 0.0090 |

${\rho}_{1}=0.7$ | |||||||||

${\rho}_{2}=0.3$ | |||||||||

SCAD | 3.8900 | 0.2500 | 0.4627 | 4.7300 | 0.0500 | 0.1218 | 5.0000 | 0.0000 | 0.0284 |

Hard | 4.2800 | 0.2900 | 0.4861 | 4.6800 | 0.0700 | 0.1208 | 4.9100 | 0.0000 | 0.0332 |

LASSO | 2.7800 | 0.3200 | 0.5947 | 3.3700 | 0.0600 | 0.1729 | 3.7800 | 0.0000 | 0.0546 |

Oracle | 5.0000 | 0.0000 | 0.1111 | 5.0000 | 0.0000 | 0.0348 | 5.0000 | 0.0000 | 0.0100 |

${\rho}_{1}=0.7$ | |||||||||

${\rho}_{2}=0.0$ | |||||||||

SCAD | 5.1500 | 0.0200 | 0.3423 | 5.7000 | 0.0000 | 0.0983 | 5.9600 | 0.0000 | 0.0273 |

Hard | 5.4000 | 0.0200 | 0.4259 | 5.5700 | 0.0000 | 0.1088 | 5.9400 | 0.0000 | 0.0276 |

LASSO | 4.1900 | 0.0100 | 0.4503 | 4.3800 | 0.0000 | 0.1585 | 4.6000 | 0.0000 | 0.0598 |

Oracle | 6.0000 | 0.0000 | 0.0461 | 6.0000 | 0.0000 | 0.0175 | 6.0000 | 0.0000 | 0.0045 |

${\rho}_{1}=0.0$ | |||||||||

${\rho}_{2}=0.7$ | |||||||||

SCAD | 5.1000 | 0.1600 | 0.4964 | 5.6800 | 0.0200 | 0.1268 | 5.9100 | 0.0000 | 0.0206 |

Hard | 5.1500 | 0.1700 | 0.4975 | 5.7000 | 0.0300 | 0.1288 | 5.9300 | 0.0000 | 0.0271 |

LASSO | 4.0000 | 0.1000 | 0.5633 | 4.1600 | 0.0100 | 0.1472 | 4.7300 | 0.0000 | 0.0487 |

Oracle | 6.0000 | 0.0000 | 0.0761 | 6.0000 | 0.0000 | 0.0260 | 6.0000 | 0.0000 | 0.0071 |

${\rho}_{1}=0.0$ | |||||||||

${\rho}_{2}=0.0$ | |||||||||

SCAD | 5.9800 | 0.0200 | 0.3283 | 6.4600 | 0.0000 | 0.0951 | 6.9200 | 0.0000 | 0.0264 |

Hard | 6.3700 | 0.0200 | 0.3164 | 6.6300 | 0.0000 | 0.1010 | 6.9400 | 0.0000 | 0.0265 |

LASSO | 4.8200 | 0.0000 | 0.4337 | 4.9600 | 0.0000 | 0.1572 | 5.2500 | 0.0000 | 0.0561 |

Oracle | 7.0000 | 0.0000 | 0.0340 | 7.0000 | 0.0000 | 0.0163 | 7.0000 | 0.0000 | 0.0040 |

${\mathit{\sigma}}^{2}=1.5$ | n = 30 | n = 60 | n = 180 | ||||||
---|---|---|---|---|---|---|---|---|---|

Method | C | I | mSE | C | I | mSE | C | I | mSE |

${\rho}_{1}=0.7$ | |||||||||

${\rho}_{2}=0.7$ | |||||||||

SCAD | 3.9500 | 0.1300 | 0.8425 | 4.7100 | 0.0200 | 0.1536 | 5.0000 | 0.0000 | 0.0461 |

HARD | 4.3100 | 0.1300 | 0.8495 | 4.6200 | 0.0200 | 0.1666 | 4.9600 | 0.0000 | 0.0471 |

LASSO | 2.9100 | 0.3400 | 1.6368 | 2.9900 | 0.1400 | 0.4662 | 3.5100 | 0.0000 | 0.1328 |

Oracle | 5.0000 | 0.0000 | 0.1838 | 5.0000 | 0.0000 | 0.0548 | 5.0000 | 0.0000 | 0.0168 |

${\rho}_{1}=0.7$ | |||||||||

${\rho}_{2}=0.3$ | |||||||||

SCAD | 4.0800 | 0.4000 | 0.8675 | 4.7300 | 0.0400 | 0.1829 | 5.0000 | 0.0000 | 0.0462 |

HARD | 4.2200 | 0.3800 | 0.8529 | 4.7100 | 0.0800 | 0.1902 | 4.9000 | 0.0000 | 0.0536 |

LASSO | 2.6700 | 0.2700 | 0.9862 | 3.2100 | 0.0700 | 0.2822 | 3.9700 | 0.0000 | 0.0987 |

Oracle | 5.0000 | 0.0000 | 0.1709 | 5.0000 | 0.0000 | 0.0666 | 5.0000 | 0.0000 | 0.0188 |

${\rho}_{1}=0.7$ | |||||||||

${\rho}_{2}=0.0$ | |||||||||

SCAD | 5.1700 | 0.0700 | 0.7204 | 5.6700 | 0.0200 | 0.1654 | 5.9700 | 0.0000 | 0.0462 |

HARD | 5.2900 | 0.0900 | 0.8144 | 5.6000 | 0.0300 | 0.1816 | 5.9300 | 0.0000 | 0.0491 |

LASSO | 4.1300 | 0.0200 | 0.7540 | 4.2700 | 0.0000 | 0.2520 | 4.7800 | 0.0000 | 0.1036 |

Oracle | 6.0000 | 0.0000 | 0.1110 | 6.0000 | 0.0000 | 0.0394 | 6.0000 | 0.0000 | 0.0103 |

${\rho}_{1}=0.0$ | |||||||||

${\rho}_{2}=0.7$ | |||||||||

SCAD | 4.9600 | 0.2000 | 0.7713 | 5.6900 | 0.0300 | 0.1865 | 5.9000 | 0.0000 | 0.0408 |

HARD | 4.9900 | 0.2100 | 0.8786 | 5.6800 | 0.0400 | 0.1888 | 5.9200 | 0.0000 | 0.0460 |

LASSO | 3.5500 | 0.0900 | 0.8639 | 4.1300 | 0.0100 | 0.2551 | 4.9900 | 0.0000 | 0.0862 |

Oracle | 6.0000 | 0.0000 | 0.1497 | 6.0000 | 0.0000 | 0.0541 | 6.0000 | 0.0000 | 0.0140 |

${\rho}_{1}=0.0$ | |||||||||

${\rho}_{2}=0.0$ | |||||||||

SCAD | 5.9700 | 0.0700 | 0.6812 | 6.5000 | 0.0200 | 0.1697 | 6.9100 | 0.0000 | 0.0446 |

HARD | 6.3000 | 0.0700 | 0.6552 | 6.6100 | 0.0000 | 0.1714 | 6.9300 | 0.0000 | 0.0449 |

LASSO | 4.8200 | 0.0200 | 0.6933 | 4.9200 | 0.0000 | 0.2337 | 5.5200 | 0.0000 | 0.1020 |

Oracle | 7.0000 | 0.0000 | 0.0765 | 7.0000 | 0.0000 | 0.0367 | 7.0000 | 0.0000 | 0.0100 |

${\mathit{\sigma}}^{2}=1$ | n = 36 | n = 64 | n = 196 | ||||||
---|---|---|---|---|---|---|---|---|---|

Method | C | I | mSE | C | I | mSE | C | I | mSE |

${\rho}_{1}=0.7$ | |||||||||

${\rho}_{2}=0.7$ | |||||||||

SCAD | 4.4300 | 0.1000 | 0.2802 | 4.6200 | 0.0000 | 0.1142 | 4.9900 | 0.0000 | 0.0290 |

HARD | 4.5600 | 0.1200 | 0.2998 | 4.6200 | 0.0200 | 0.1316 | 4.9600 | 0.0000 | 0.0343 |

LASSO | 3.2300 | 0.1000 | 0.4090 | 3.3500 | 0.0100 | 0.1982 | 3.8400 | 0.0000 | 0.0573 |

Oracle | 5.0000 | 0.0000 | 0.2415 | 5.0000 | 0.0000 | 0.1121 | 5.0000 | 0.0000 | 0.0206 |

${\rho}_{1}=0.7$ | |||||||||

${\rho}_{2}=0.3$ | |||||||||

SCAD | 4.3400 | 0.2400 | 0.2987 | 4.5500 | 0.1100 | 0.1290 | 4.9800 | 0.0000 | 0.0370 |

HARD | 4.3900 | 0.2800 | 0.3229 | 4.5900 | 0.1500 | 0.1495 | 4.9600 | 0.0000 | 0.0375 |

LASSO | 3.2800 | 0.2700 | 0.3923 | 3.2900 | 0.1200 | 0.1935 | 3.8600 | 0.0000 | 0.0591 |

Oracle | 5.0000 | 0.0000 | 0.2584 | 5.0000 | 0.0000 | 0.1254 | 5.0000 | 0.0000 | 0.0368 |

${\rho}_{1}=0.7$ | |||||||||

${\rho}_{2}=0.0$ | |||||||||

SCAD | 5.5100 | 0.0000 | 0.2082 | 5.6800 | 0.0000 | 0.0939 | 5.9800 | 0.0000 | 0.0264 |

HARD | 5.5900 | 0.0000 | 0.2430 | 5.6700 | 0.0000 | 0.0959 | 5.9600 | 0.0000 | 0.0264 |

LASSO | 4.3200 | 0.0000 | 0.3124 | 4.3500 | 0.0000 | 0.1748 | 4.8500 | 0.0000 | 0.0489 |

Oracle | 6.0000 | 0.0000 | 0.0293 | 6.0000 | 0.0000 | 0.0173 | 6.0000 | 0.0000 | 0.0053 |

${\rho}_{1}=0.0$ | |||||||||

${\rho}_{2}=0.7$ | |||||||||

SCAD | 5.1200 | 0.1100 | 0.3238 | 5.5200 | 0.0000 | 0.1168 | 5.9000 | 0.0000 | 0.0300 |

HARD | 5.2000 | 0.1600 | 0.3712 | 5.5300 | 0.0100 | 0.1240 | 5.9200 | 0.0000 | 0.0307 |

LASSO | 4.0100 | 0.1700 | 0.4305 | 4.3500 | 0.0200 | 0.1922 | 4.7100 | 0.0000 | 0.0717 |

Oracle | 6.0000 | 0.0000 | 0.0616 | 6.0000 | 0.0000 | 0.0396 | 6.0000 | 0.0000 | 0.0121 |

${\rho}_{1}=0.0$ | |||||||||

${\rho}_{2}=0.0$ | |||||||||

SCAD | 6.1900 | 0.0000 | 0.1688 | 6.5600 | 0.0000 | 0.0955 | 6.9300 | 0.0000 | 0.0242 |

HARD | 6.5300 | 0.0100 | 0.1852 | 6.5600 | 0.0000 | 0.1106 | 6.8800 | 0.0000 | 0.0259 |

LASSO | 5.0700 | 0.0000 | 0.3146 | 5.2900 | 0.0000 | 0.1681 | 5.7100 | 0.0000 | 0.0435 |

Oracle | 7.0000 | 0.0000 | 0.0226 | 7.0000 | 0.0000 | 0.0140 | 7.0000 | 0.0000 | 0.0044 |

${\mathit{\sigma}}^{2}=1$ | $\mathit{n}=30$ | $\mathit{n}=60$ | $\mathit{n}=180$ | ||||||
---|---|---|---|---|---|---|---|---|---|

Method | C | I | mSE | C | I | mSE | C | I | mSE |

${\rho}_{1}=0.7$ | |||||||||

${\rho}_{2}=0.7$ | |||||||||

SCAD | 4.1500 | 1.1900 | 2894.0 | 4.5000 | 0.9900 | 4159.8 | 4.7800 | 0.4500 | 5025.6 |

HARD | 1.9300 | 0.3700 | 2153.9 | 2.7700 | 0.4200 | 3697.3 | 4.1700 | 0.2300 | 4870.4 |

LASSO | 0.2500 | 0.1500 | 2110.4 | 0.0000 | 0.0000 | 3449.5 | 0.0000 | 0.0000 | 4762.6 |

${\rho}_{1}=0.7$ | |||||||||

${\rho}_{2}=0.3$ | |||||||||

SCAD | 4.2300 | 0.5700 | 73.698 | 4.4700 | 0.3400 | 105.62 | 4.7800 | 0.0400 | 122.92 |

HARD | 3.9200 | 0.4300 | 70.815 | 4.3700 | 0.3400 | 101.09 | 4.7400 | 0.0500 | 122.92 |

LASSO | 1.7600 | 0.2100 | 79.049 | 0.4900 | 0.1100 | 99.844 | 0.0000 | 0.0000 | 117.50 |

${\rho}_{1}=0.7$ | |||||||||

${\rho}_{2}=0.0$ | |||||||||

SCAD | 4.2600 | 0.4400 | 39.324 | 4.5500 | 0.1900 | 51.2130 | 4.7700 | 0.0100 | 52.666 |

HARD | 4.0500 | 0.4100 | 37.080 | 4.5400 | 0.2000 | 50.641 | 4.8600 | 0.0100 | 52.667 |

LASSO | 1.9700 | 0.1900 | 40.984 | 1.1200 | 0.0600 | 48.713 | 0.0000 | 0.0000 | 49.750 |

${\rho}_{1}=0.0$ | |||||||||

${\rho}_{2}=0.7$ | |||||||||

SCAD | 3.9200 | 0.1000 | 0.7908 | 4.6600 | 0.0000 | 0.5977 | 4.8600 | 0.0000 | 0.5517 |

HARD | 4.3100 | 0.0700 | 0.8395 | 4.7200 | 0.0000 | 0.6243 | 4.8900 | 0.0000 | 0.5618 |

LASSO | 2.4900 | 0.0200 | 0.7809 | 2.8300 | 0.0000 | 0.7512 | 3.1800 | 0.0000 | 0.6659 |

${\rho}_{1}=0.0$ | |||||||||

${\rho}_{2}=0.0$ | |||||||||

SCAD | 3.5500 | 0.0200 | 0.3353 | 4.6600 | 0.0000 | 0.1115 | 4.9900 | 0.0000 | 0.0260 |

HARD | 4.1400 | 0.0200 | 0.4382 | 4.6600 | 0.0000 | 0.1125 | 4.9000 | 0.0000 | 0.0276 |

LASSO | 2.4900 | 0.0000 | 0.4980 | 2.8600 | 0.0000 | 0.1672 | 3.3100 | 0.0000 | 0.0533 |

Method | $\mathit{n}=30$ | $\mathit{n}=60$ | ||||
---|---|---|---|---|---|---|

SD | SDm | SDmad | SD | SDm | SDmad | |

SCAD | ||||||

${\sigma}^{2}$ | 0.2006 | 0.1634 | 0.0325 | 0.1091 | 0.1282 | 0.0154 |

${\rho}_{1}$ | 0.0226 | 0.0227 | 0.0032 | 0.0169 | 0.0131 | 0.0012 |

${\rho}_{2}$ | 0.1106 | 0.1851 | 0.0236 | 0.0565 | 0.0895 | 0.0078 |

${\beta}_{1}$ | 0.1225 | 0.1299 | 0.0152 | 0.1110 | 0.0886 | 0.0079 |

${\beta}_{2}$ | 0.1495 | 0.1374 | 0.0172 | 0.1161 | 0.0890 | 0.0085 |

${\beta}_{5}$ | 0.2074 | 0.1466 | 0.0205 | 0.1111 | 0.0871 | 0.0092 |

HARD | ||||||

${\sigma}^{2}$ | 0.1788 | 0.1498 | 0.0325 | 0.1022 | 0.1251 | 0.0157 |

${\rho}_{1}$ | 0.0222 | 0.0233 | 0.0031 | 0.0165 | 0.0130 | 0.0012 |

${\rho}_{2}$ | 0.0850 | 0.1156 | 0.0134 | 0.0542 | 0.0684 | 0.0062 |

${\beta}_{1}$ | 0.1287 | 0.1297 | 0.0172 | 0.1105 | 0.0874 | 0.0097 |

${\beta}_{2}$ | 0.1816 | 0.1342 | 0.0170 | 0.1157 | 0.0877 | 0.0081 |

${\beta}_{5}$ | 0.1930 | 0.1340 | 0.0207 | 0.1114 | 0.0867 | 0.0084 |

LASSO | ||||||

${\sigma}^{2}$ | 0.2104 | 0.1609 | 0.0350 | 0.1236 | 0.1354 | 0.0173 |

${\rho}_{1}$ | 0.0257 | 0.0252 | 0.0027 | 0.0190 | 0.0137 | 0.0013 |

${\rho}_{2}$ | 0.1242 | 0.1782 | 0.0227 | 0.0686 | 0.0895 | 0.0063 |

${\beta}_{1}$ | 0.1830 | 0.1532 | 0.0246 | 0.0974 | 0.0994 | 0.0108 |

${\beta}_{2}$ | 0.1954 | 0.1570 | 0.0264 | 0.1135 | 0.0961 | 0.0094 |

${\beta}_{5}$ | 0.1921 | 0.1485 | 0.0265 | 0.1161 | 0.0943 | 0.0107 |

Variable | Description |
---|---|

MEDV | The median value of owner-occupied homes. Source: 1970 U.S. Census. |

CRIM | Crime rate by town. Source: FBI (1970). |

ZN | Proportion of a town’s residential land zoned for lots greater than 25,000 square feet. Source: Metropolitan Area Planning Commission (1972). |

INDUS | Proportion nonretail business acres per town. Source: Harrison and Rubinfeld (1978). |

CHAS | Charles River dummy: =1 if tract bounds the Charles River; =0 if otherwise. Source: 1970 U.S. Census. |

NOX | Nitrogen oxide concentrations in pphm (annual average concentration in parts per hundred million). Source: TASSIM. |

RM | Average number of rooms in owner units. Source: 1970 U.S. Census. |

AGE | Proportion of owner units built prior to 1940. Source: 1970 U.S. Census. |

DIS | Weighted distances to five employment centres in the Boston region. Source: Harrison and Rubinfeld (1978). |

RAD | Index of accessibility to radial highways. It was calculated on a town basis. Source: MIT Boston Project. |

TAX | Full value property tax rate ($/$10,000). Source: Massachusetts Taxpayers Foundation (1970). |

PTRATIO | The number of students divided by the number of teachers in town school district. Source: Massachusetts Dept. of Education (1971–1972). |

B | Black proportion of population. Source: 1970 U.S. Census. |

PART | Proportion of population that is lower status = $\frac{1}{2}$ (proportion of adults without some high school education and proportion of male workers classified as laborers). Source: 1970 U.S. Census. |

Terms | Values of Test Statistics | p-Values |
---|---|---|

Moran I | 0.7644 | <2.2e−16 |

LMerr | 186.57 | <2.2e−16 |

LMlag | 190.71 | <2.2e−16 |

SARMA | 228.32 | <2.2e−16 |

**Table 8.**Parameter estimates using quasi-maximum likelihood and penalized estimates via SCAD, HARD, and LASSO under a SARAR model.

Terms | QMLE | SCAD | HARD | LASSO |
---|---|---|---|---|

CRIM | −0.1405 | −0.1240 | −0.1410 | −0.1346 |

ZN | 0.0221 | − | − | − |

INDUS | 0.0280 | − | − | − |

CHAS | −0.0058 | − | − | − |

NOX${}^{2}$ | −0.1037 | −0.0223 | −0.1046 | −0.0560 |

RM${}^{2}$ | 0.1721 | 0.1657 | 0.1643 | 0.1624 |

AGE | −0.0372 | − | − | − |

log(DIS) | −0.2082 | −0.1127 | −0.1851 | −0.1415 |

log(RAD) | 0.1750 | 0.1124 | 0.1680 | 0.1039 |

TAX | −0.1958 | −0.1583 | −0.1757 | −0.1137 |

PTRATIO | −0.1030 | −0.0816 | −0.1071 | −0.0830 |

(B−0.63)${}^{2}$ | 0.0865 | 0.0713 | 0.0827 | 0.0652 |

log(LSTAT) | −0.3998 | −0.4317 | −0.4167 | −0.3900 |

${\rho}_{1}$ | 0.2805 | 0.2695 | 0.2776 | 0.3691 |

${\rho}_{2}$ | 0.4145 | 0.4444 | 0.4107 | 0.2430 |

${\sigma}^{2}$ | 0.1182 | 0.1197 | 0.1190 | 0.1230 |

BIC | 489.81 | 474.18 | 467.36 | 477.00 |

**Table 9.**Parameter estimates using quasi-maximum likelihood and penalized estimates via SCAD, HARD, and LASSO under a classical linear model.

Terms | QMLE | SCAD | HARD | LASSO |
---|---|---|---|---|

CRIM | −0.2537 | −0.2541 | −0.2539 | −0.2420 |

ZN | 0.0047 | − | − | − |

INDUS | 0.0051 | − | − | − |

CHAS | 0.0573 | 0.0565 | 0.0578 | 0.0554 |

NOX${}^{2}$ | −0.2178 | −0.2157 | −0.2158 | −0.1852 |

RM${}^{2}$ | 0.1367 | 0.1383 | 0.1383 | 0.1418 |

AGE | 0.0085 | − | − | − |

log(DIS) | −0.2529 | −0.2570 | −0.2567 | −0.2211 |

log(RAD) | 0.2035 | 0.2013 | 0.2011 | 0.1569 |

TAX | −0.1744 | −0.1708 | −0.1704 | −0.1373 |

PTRATIO | −0.1663 | −0.1667 | −0.1666 | −0.1575 |

(B−0.63)${}^{2}$ | 0.0692 | 0.0689 | 0.0693 | 0.0665 |

log(LSTAT) | −0.5496 | −0.5467 | −0.5464 | −0.5440 |

${\sigma}^{2}$ | 0.1951 | 0.1952 | 0.1952 | 0.1961 |

BIC | 696.27 | 677.68 | 677.67 | 680.24 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Liu, X.; Chen, J.
Variable Selection for the Spatial Autoregressive Model with Autoregressive Disturbances. *Mathematics* **2021**, *9*, 1448.
https://doi.org/10.3390/math9121448

**AMA Style**

Liu X, Chen J.
Variable Selection for the Spatial Autoregressive Model with Autoregressive Disturbances. *Mathematics*. 2021; 9(12):1448.
https://doi.org/10.3390/math9121448

**Chicago/Turabian Style**

Liu, Xuan, and Jianbao Chen.
2021. "Variable Selection for the Spatial Autoregressive Model with Autoregressive Disturbances" *Mathematics* 9, no. 12: 1448.
https://doi.org/10.3390/math9121448