Second-Order Least Squares Estimation in Nonlinear Time Series Models with ARCH Errors

Abstract

Many financial and economic time series exhibit nonlinear patterns or relationships. However, most statistical methods for time series analysis are developed for mean-stationary processes that require transformation, such as differencing of the data. In this paper, we study a dynamic regression model with nonlinear, time-varying mean function, and autoregressive conditionally heteroscedastic errors. We propose an estimation approach based on the first two conditional moments of the response variable, which does not require specification of error distribution. Strong consistency and asymptotic normality of the proposed estimator is established under strong-mixing condition, so that the results apply to both stationary and mean-nonstationary processes. Moreover, the proposed approach is shown to be superior to the commonly used quasi-likelihood approach and the efficiency gain is significant when the (conditional) error distribution is asymmetric. We demonstrate through a real data example that the proposed method can identify a more accurate model than the quasi-likelihood method.

1. Introduction

Dynamic models have been widely applied in the analysis of economic and financial data. Most theories and methods are developed for mean-stationary data generating processes, specifically ARMA processes, although ARIMA processes have also been studied (Koul and Ling 2006; Ling 2003; Ling and McAleer 2003; Meitz and Saikkonen 2008). However, many financial and economic variables exhibit nonlinear behaviour or relationships (e.g., Enders 2010; Franses and Van Dijk 2000). As pointed out by Li et al. (2002), consistent estimation of variance parameters may be misleading or impossible if the conditional mean function is not adequately specified. Therefore, more general and flexible models with time varying mean functions are desirable to capture the nonlinear dynamic behaviour and the structural relationships in the real data.

On the other hand, the autoregressive conditional heteroscedasticity (ARCH) model and its various generalizations have been widely used to analyze economic and financial data. These models allow both the conditional means and variances of a process to jointly evolve over time. The mainstream method for estimation and inference in generalized ARCH (GARCH) models is likelihood based (e.g., Engle 1982; Engle and Gonzalez-Rivera 1991; Weiss 1986), although the estimating function approach is also studied (Li and Turtle 2000). So far most research focuses on the quasi-likelihood method for various generalizations of the ARCH error component while keeping the process mean function very simple, e.g., constant or linear. Therefore from both theoretical and practical points of view, it is important to develop methodologies for mean nonstationary ARCH processes.

In this paper, we consider a model with a fairly general time varying nonlinear mean function and ARCH error that covers both stationary and mean-nonstationary processes. In particular, we propose a so-called second-order least squares (SLS) approach based on the first two conditional moments of the process. We establish the consistency and asymptotic normality for the proposed estimator under general mixing conditions. We demonstrate that this approach is more efficient than the commonly used quasi-likelihood method and the efficiency gain is significant when the conditional error distribution is asymmetric. We also carry out extensive simulations to study finite sample properties of our proposed estimator and compare its performance with other related estimators. Our results show that in most cases the optimal SLS estimator has superior performance over the estimating function estimators based on the same set of moments. Finally, we apply our approach to the empirical example of the U.K. inflation of Engle (1982), which leads to a different model specification than the quasi-likelihood method.

It is worthwhile to note that some researchers have obtained more efficient estimators than the Gaussian quasi-maximum likelihood estimator (QMLE) by assuming various parametric families of error distributions (see a recent survey by Zhu and Li (2015)). However, our approach is based on the first two conditional moments of the process only and does not require any distributional assumptions. The SLS method was first used by Wang (2003, 2004) to estimate the nonlinear measurement error models. Later, it was extended to the nonlinear longitudinal data models by Wang (2007) and to the censored linear models by Abarin and Wang (2009). Wang and Leblanc (2008) showed that under a nonlinear cross-sectional data model, the SLS estimator is asymptotically more efficient than the ordinary least squares estimator when the error term has nonzero third moment, and both estimators are equally efficient otherwise. Further, Kim and Ma (2012) showed that the SLS estimator attains the optimal semiparametric efficiency bound in general. More recently, Rosadi and Filzmoser (2019); Rosadi and Peiris (2014) and Salamh and Wang (2021) used this method in dynamic models. It has also been applied in the optimal design problems by several researchers, e.g., Bose and Mukerjee (2015); Gao and Zhou (2017); Yin and Zhou (2017) and He and Yue (2019).

The paper is organized as follows. In Section 2 we introduce the model and SLS estimator and establish its consistency and asymptotic normality. In Section 3 we derive the optimal SLS estimator and propose a feasible version of it. We also investigate the efficiency gain of the optimal SLS estimator relative to the QMLE and highlight the differences between our approach and that of the estimating functions. In Section 4 we carry out Monte Carlo simulations to study the finite sample behavior of the SLS estimator in the cases of both skewed and leptokurtic conditional error distributions, and compare it with the QMLE and some other related estimators. In Section 5 we apply the SLS approach to an empirical analysis of the U.K. inflation data. Finally, conclusions and discussion are given in Section 6, while regularity assumptions and mathematical proofs are given in the Appendix A.

2. Model and SLS Estimation

Let $\left\{\left(y_t,{\mathit{x}}_t^{\prime }\right)\right\}$ be a sequence of random vectors defined on a complete probability space $\left(\mathsf{\Omega },\mathcal{F},P\right)$ and denote ${\mathit{v}}_t={\left({\mathit{x}}_t^{\prime },y_{t-1},{\mathit{x}}_{t-1}^{\prime },\dots ,y_{t-\tau },{\mathit{x}}_{t-\tau }^{\prime }\right)}^{\prime }$ for some nonnegative integer $\tau$. We consider the model

$$y_t=f_t\left({\mathit{v}}_t,{\mathit{\theta }}_0\right)+{ϵ}_t,t\in \mathbb{Z},$$

(1)

where $f_t:{\mathbb{R}}^{\upsilon }\times \mathsf{\Theta }\to {\mathbb{R}}^1$ are known measurable functions on ${\mathbb{R}}^{\upsilon }$ for each $\mathit{\theta }\in \mathsf{\Theta }\subset {\mathbb{R}}^q$, and continuous on $\mathsf{\Theta }$ uniformly in t a.s.-P. Further, denote the $\sigma$-field ${\mathcal{F}}_{t-1}=\mathcal{F}\left\{{\mathit{x}}_i,y_{i-1},i\le t\right\}$ and assume that ${ϵ}_t={\sigma }_t{\epsilon }_t$ satisfying

$$E\left({\epsilon }_t|{\mathcal{F}}_{t-1}\right)=0\mathrm{a}.\mathrm{s}.\text{-}\mathit{P},E\left({\epsilon }_t^2|{\mathcal{F}}_{t-1}\right)=1\mathrm{a}.\mathrm{s}.\text{-}\mathit{P},{\sigma }_t^2={\varphi }_{00}+\sum _{i=1}^p{\varphi }_{0i}{ϵ}_{t-i}^2,$$

(2)

where ${\varphi }_{00},{\varphi }_{0p}>0$, and ${\varphi }_{0i}\ge 0$ for $i=1,2,\dots ,p-1$. It is easy to see that the linear model with ARCH error is a special case of model (1) and (2). Our main goal is to estimate ${\gamma }_0={\left({\mathit{\theta }}_0^{\prime },{\mathit{\varphi }}_0^{\prime }\right)}^{\prime }\in \mathsf{\Gamma }$ which is a compact subset of ${\mathbb{R}}^{q+p+1}$.

Given the random sample $\left(y_T,{\mathit{x}}_T^{\prime },\dots ,y_{1-p-\tau },{\mathit{x}}_{1-p-\tau }^{\prime }\right)$, let ${ϵ}_t\left(\mathit{\theta }\right)=y_t-f_t\left({\mathit{v}}_t,\mathit{\theta }\right)$, ${\sigma }_t^2\left(\gamma \right)={\varphi }_0+{\sum }_{i=1}^p{\varphi }_i{ϵ}_{t-i}^2\left(\mathit{\theta }\right)$ and ${\mathit{h}}_t\left(\gamma \right)={\left({ϵ}_t\left(\mathit{\theta }\right),y_t^2-f_t^2\left({\mathit{v}}_t,\mathit{\theta }\right)-{\sigma }_t^2\left(\gamma \right)\right)}^{\prime }$. Then the second-order least squares (SLS) estimator is defined as the $\mathcal{F}$–measurable function satisfying

$${\widehat{\gamma }}_T=\underset{\gamma \in \mathsf{\Gamma }}{\mathrm{argmin}}{Q}_T\left(\gamma \right)\mathrm{a}.\mathrm{s}.\text{-}\mathit{P},$$

(3)

where

$$Q_T\left(\gamma \right)=T^{-1}\sum _{t=1}^T{\mathit{h}}_t^{\prime }\left(\gamma \right){\mathit{W}}_t{\mathit{h}}_t\left(\gamma \right)$$

and ${\mathit{W}}_t$ is a nonnegative definite matrix that is measurable, with respect to $\mathcal{F}\left\{{\mathit{v}}_t,\dots ,{\mathit{v}}_{t-p}\right\}$.

Next we establish the consistency and asymptotic normality of ${\widehat{\gamma }}_T$ under the regularity conditions given in Appendix A. The consistency of ${\widehat{\gamma }}_T$ follows from the uniform convergence of $Q_T\left(\gamma \right)$ on $\mathsf{\Gamma }$ to a non-random sequence ${\overline{Q}}_T\left(\gamma \right)$ which have unique minimizer at ${\gamma }_0$ for sufficiently large T.

Theorem 1.

Under Assumptions A1–A3, ${\widehat{\gamma }}_T\stackrel{a.s.}{⟶}{\gamma }_0$, as $T\to \infty$.

Theorem 2.

Under Assumptions A1–A9, ${\mathit{V}}_T^{-1/2}{\overline{\mathit{A}}}_T\left({\gamma }_0\right)\sqrt{T}\left({\widehat{\gamma }}_T-{\gamma }_0\right)\stackrel{d}{⟶}N\left(\mathbf{0},{\mathit{I}}_{q+p+1}\right)$, as $T\to \infty$, where

$${\overline{\mathit{A}}}_T\left({\gamma }_0\right)=2T^{-1}\sum _{t=1}^{T}E\left\{{\nabla }_{\gamma }{\mathit{h}}_t^{\prime }\left({\gamma }_0\right){\mathit{W}}_t{\nabla }_{{\gamma }^{\prime }}{\mathit{h}}_t\left({\gamma }_0\right)\right\}$$

(4)

and

$${\mathit{V}}_T=4T^{-1}\sum _{t=1}^{T}E\left\{{\nabla }_{\gamma }{\mathit{h}}_t^{\prime }\left({\gamma }_0\right){\mathit{W}}_t{\mathit{h}}_t\left({\gamma }_0\right){\mathit{h}}_t^{\prime }\left({\gamma }_0\right){\mathit{W}}_t{\nabla }_{{\gamma }^{\prime }}{\mathit{h}}_t\left({\gamma }_0\right)\right\}.$$

(5)

The proofs are given in the Appendix A.

3. Optimal SLS Estimator

From Theorem 2 the asymptotic covariance (acov) of $\sqrt{T}\left({\widehat{\gamma }}_T-{\gamma }_0\right)$ is given by ${\overline{\mathit{A}}}_T^{-1}\left({\gamma }_0\right){\mathit{V}}_T{\overline{\mathit{A}}}_T^{-1}\left({\gamma }_0\right)$ which depends on the weights ${\mathit{W}}_t$, $t=1,2,\dots ,T$. Therefore it is of interest to find the (asymptotically) optimal estimator, say ${\widehat{\gamma }}_T^o$, which has the smallest asymptotic variance in the class of estimators defined by (3), i.e., for any estimator ${\widehat{\gamma }}_T$ satisfying (3), $\mathrm{acov}\sqrt{T}\left({\widehat{\gamma }}_T-{\gamma }_0\right)-\mathrm{acov}\sqrt{T}\left({\widehat{\gamma }}_T^o-{\gamma }_0\right)$ is nonnegative definite. The following theorem gives the optimal choice of ${\mathit{W}}_t$ to achieve this goal.

Theorem 3.

Suppose ${\mathit{U}}_t=E\left\{{\mathit{h}}_t\left({\gamma }_0\right){\mathit{h}}_t^{\prime }\left({\gamma }_0\right){\mathit{v}}_t,\dots ,{\mathit{v}}_{t-p}\right\}$ is nonsingular a.s.-P, and Assumptions A2, A3, A6–A9 hold with ${\mathit{W}}_t={\mathit{U}}_t^{-1}$. Then, the asymptotically optimal SLS (OSLS) estimator ${\widehat{\gamma }}_T^o$ is obtained by using ${\mathit{W}}_t={\mathit{U}}_t^{-1}$, $t=1,2,\dots ,T$. Further, the corresponding (inverse) optimal covariance matrix is given by

$$\begin{array}{ccc}\hfill {acov}^{-1}\sqrt{T}\left({\widehat{\gamma }}_T^o-{\gamma }_0\right)& =& \hfill T^{-1}\sum _{t=1}^{T}E\left\{{\nabla }_{\gamma }{\mathit{h}}_t^{\prime }\left({\gamma }_0\right){\mathit{U}}_t^{-1}{\nabla }_{{\gamma }^{\prime }}{\mathit{h}}_t\left({\gamma }_0\right)\right\},\hfill \end{array}$$

(6)

$$\begin{array}{ccc}& =& \hfill T^{-1}\sum _{t=1}^{T}E\left\{{\mathit{B}}_t^{\prime }{\mathsf{\Omega }}_t^{-1}{\mathit{B}}_t\right\},\hfill \end{array}$$

(7)

where

$${\mathit{B}}_t^{\prime }=\left(\begin{array}{cc}{\nabla }_{\mathit{\theta }}{f}_t\left({\mathit{v}}_t,{\mathit{\theta }}_0\right)& {\nabla }_{\mathit{\theta }}{\sigma }_t^2\left({\gamma }_0\right)\\ \mathbf{0}& {\nabla }_{\mathit{\varphi }}{\sigma }_t^2\left({\gamma }_0\right)\end{array}\right),$$

(8)

and

$${\mathsf{\Omega }}_t={\sigma }_t^2\left({\gamma }_0\right)\left(\begin{array}{cc}1& {\sigma }_t\left({\gamma }_0\right)E\left({\epsilon }_t^3{\mathit{v}}_t,\dots ,{\mathit{v}}_{t-p}\right)\\ ·& {\sigma }_t^2\left({\gamma }_0\right)\left[E\left({\epsilon }_t^4{\mathit{v}}_t,\dots ,{\mathit{v}}_{t-p}\right)-1\right]\end{array}\right).$$

(9)

However, the OSLS estimator ${\widehat{\gamma }}_T^o$ is infeasible because ${\mathit{U}}_t$ depends on ${\gamma }_0$ and $E\left({\epsilon }_t^j{\mathit{v}}_t,\dots ,{\mathit{v}}_{t-p}\right)$, $j=3,4$. In practice a two-step procedure can be used as follows. First, a first-step consistent estimator of ${\gamma }_0$ is calculated, such as the QMLE or simply the SLS ${\widehat{\gamma }}_T$ using the identity weight matrix. Second, the residuals ${\widehat{\epsilon }}_t$ are calculated and suitable autoregressive models are fitted to ${\widehat{\epsilon }}_t^3$ and ${\widehat{\epsilon }}_t^4$ respectively. Finally, these fitted values are substituted in ${\mathit{U}}_t$ and the second-step estimator is calculated using the estimated optimal weights ${\mathit{W}}_t={\widehat{\mathit{U}}}_t^{-1}$. Under some general conditions this two-step estimator is consistent and, moreover, it has the same asymptotic variance as in (7) if ${\widehat{\mathit{U}}}_t$ is consistent for ${\mathit{U}}_t$. Henceforth, this two-step estimator will be called the feasible optimal SLS estimator (FSLS). Note that fitting the autoregressive models is useful if the errors $\left\{{\epsilon }_t\right\}$ are not i.i.d., otherwise the sample means of ${\widehat{\epsilon }}_t^3$ and ${\widehat{\epsilon }}_t^4$ can be used. Alternatively, if the conditioning set $\left\{{\mathit{v}}_t,\dots ,{\mathit{v}}_{t-p}\right\}$ is reasonably small, one can use nonparametric estimators of the conditional skewness and kurtosis to obtain ${\widehat{\mathit{U}}}_t$. More details about the two-step estimators can be found in White (1996, Section 6.3).

In the rest of this section we investigate the efficiency gain of the OSLS estimator compared to the Gaussian QMLE, which is one of the most popular methods of estimation in GARCH models. The asymptotic properties of the QMLE are studied by Weiss (1986) and Bollerslev and Wooldridge (1992). Specifically for model (1) and (2), the Gaussian QMLE is defined as

$${\widehat{\gamma }}_T^Q=\underset{\gamma \in \mathsf{\Gamma }}{\mathrm{argmin}}{T}^{-1}\sum _{t=1}^{T}log{\sigma }_t^2\left(\gamma \right)+\frac{{ϵ}_t^2\left(\mathit{\theta }\right)}{{\sigma }_t^2\left(\gamma \right)}\mathrm{a}.\mathrm{s}.\text{-}\mathit{P}.$$

(10)

Under similar conditions as Assumptions A2–A9 and similar to the proofs of Theorems 1 and 2, we can show that ${\widehat{\gamma }}_T^Q$ is $\sqrt{T}$-consistent with $\mathrm{acov}\sqrt{T}\left({\widehat{\gamma }}_T^Q-{\gamma }_0\right)$ given by

$$T{\left(\sum _{t=1}^{T}E\left\{{\mathit{B}}_t^{\prime }{\mathsf{\Sigma }}_t^{-1}{\mathit{B}}_t\right\}\right)}^{-1}\left(\sum _{t=1}^{T}E\left\{{\mathit{B}}_t^{\prime }{\mathsf{\Sigma }}_t^{-1}{\mathsf{\Omega }}_t{\mathsf{\Sigma }}_t^{-1}{\mathit{B}}_t\right\}\right){\left(\sum _{t=1}^{T}E\left\{{\mathit{B}}_t^{\prime }{\mathsf{\Sigma }}_t^{-1}{\mathit{B}}_t\right\}\right)}^{-1},$$

(11)

where ${\mathit{B}}_t$ and ${\mathsf{\Omega }}_t$ are defined in (8) and (9) respectively and

$${\mathsf{\Sigma }}_t=\left(\begin{array}{cc}{\sigma }_t^2\left({\gamma }_0\right)& 0\\ 0& 2{\sigma }_t^4\left({\gamma }_0\right)\end{array}\right).$$

Further, similar to the proof of Theorem 3, we can show that

$$\mathrm{acov}\sqrt{T}{\mathit{a}}^{\prime }\left({\widehat{\gamma }}_T^o-{\gamma }_0\right)\le \mathrm{acov}\sqrt{T}{\mathit{a}}^{\prime }\left({\widehat{\gamma }}_T^Q-{\gamma }_0\right)$$

for any $\mathit{a}\in {\mathbb{R}}^{q+p+1}$, and the equality holds if and only if for $t=1,2,\dots ,T$,

$${\mathsf{\Omega }}_t{\mathsf{\Sigma }}_t^{-1}{\mathit{B}}_t\mathit{a}={\mathit{B}}_t{\left(\sum _{t=1}^{T}E\left\{{\mathit{B}}_t^{\prime }{\mathsf{\Omega }}_t^{-1}{\mathit{B}}_t\right\}\right)}^{-1}\left(\sum _{t=1}^{T}E\left\{{\mathit{B}}_t^{\prime }{\mathsf{\Sigma }}_t^{-1}{\mathit{B}}_t\right\}\right)\mathit{a}\mathrm{a}.\mathrm{s}.\text{-}\mathit{P}.$$

(12)

The above general condition can be simplified under specific settings. For example, if the process $\left\{\left(y_t,{\mathit{x}}_t^{\prime },{\sigma }_t,{ϵ}_t\right)\right\}$ is stationary with $E\left({\epsilon }_t^3{\mathit{v}}_t,\dots ,{\mathit{v}}_{t-p}\right)=0$, and $E\left({\epsilon }_t^4{\mathit{v}}_t,\dots ,{\mathit{v}}_{t-p}\right)$ = ${\mu }_4$, then it can be shown that Equation (12) is equivalent to

$${\mathit{a}}_1^{\prime }\left({\mathit{I}}_q-\mathit{C}\right){\nabla }_{\mathit{\theta }}{f}_t\left({\mathit{v}}_t,{\mathit{\theta }}_0\right)=0,{\mathit{a}}_1^{\prime }\left(\frac{{\mu }_4-1}{2}{\mathit{I}}_q-\mathit{C}\right){\nabla }_{\mathit{\theta }}{\sigma }_t^2\left({\gamma }_0\right)=0,$$

(13)

where $\mathit{C}=\left({\mathit{C}}_1+\frac{1}{2}{\mathit{C}}_2\right){\left({\mathit{C}}_1+\frac{1}{{\mu }_4-1}{\mathit{C}}_2\right)}^{-1}$, ${\mathit{C}}_1=E\left\{{\sigma }_t^{-2}\left({\gamma }_0\right){\nabla }_{\mathit{\theta }}{f}_t\left({\mathit{v}}_t,{\mathit{\theta }}_0\right){\nabla }_{{\mathit{\theta }}^{\prime }}{f}_t\left({\mathit{v}}_t,{\mathit{\theta }}_0\right)\right\}$,

${\mathit{C}}_2=E\left\{{\sigma }_t^{-4}\left({\gamma }_0\right){\nabla }_{\mathit{\theta }}{\sigma }_t^2\left({\gamma }_0\right){\nabla }_{{\mathit{\theta }}^{\prime }}{\sigma }_t^2\left({\gamma }_0\right)\right\}$ and ${\mathit{a}}_1$ is the subvector of the first q elements of $\mathit{a}$.

Since it is difficult to quantify the difference between the asymptotic covariance matrices in (7) and (11) in general, in the following we calculate some examples under a simple AR(1) model with ARCH(1) error

$$y_t={\theta }_0{y}_{t-1}+{ϵ}_t,{\sigma }_t^2=1-{\varphi }_0+{\varphi }_0{ϵ}_{t-1}^2,$$

(14)

where the innovations $\left\{{\epsilon }_t={ϵ}_t/{\sigma }_t\right\}$ are i.i.d. with zero mean and unit variance. We consider both symmetric and skewed distributions for ${\epsilon }_t$. In particular, we choose Gamma distribution with various shape parameters and Student’s t distribution with various degrees of freedom to reflect the different degrees of skewness and kurtosis, respectively.

The asymptotic variances of the OSLS and QMLE for various parameter values are given in

Table 1. Asymptotic variances of OSLS, QMLE, and MLE under AR(1)-ARCH(1) model.

		$\mathit{v}\left({\widehat{\mathit{\theta }}}_{\mathbf{0}}\right)$	$\mathit{v}\left({\widehat{\mathit{\varphi }}}_{\mathbf{0}}\right)$	$\mathit{v}\left({\widehat{\mathit{\theta }}}_{\mathbf{0}}\right)$	$\mathit{v}\left({\widehat{\mathit{\varphi }}}_{\mathbf{0}}\right)$	$\mathit{v}\left({\widehat{\mathit{\theta }}}_{\mathbf{0}}\right)$	$\mathit{v}\left({\widehat{\mathit{\varphi }}}_{\mathbf{0}}\right)$	$\mathit{v}\left({\widehat{\mathit{\theta }}}_{\mathbf{0}}\right)$	$\mathit{v}\left({\widehat{\mathit{\varphi }}}_{\mathbf{0}}\right)$
		${\mathit{\theta }}_{\mathbf{0}}=\mathbf{0.2}$		${\mathit{\theta }}_{\mathbf{0}}=\mathbf{0.2}$		${\mathit{\theta }}_{\mathbf{0}}=\mathbf{0.8}$		${\mathit{\theta }}_{\mathbf{0}}=\mathbf{0.8}$
		${\mathit{\varphi }}_{\mathbf{0}}=\mathbf{0.2}$		${\mathit{\varphi }}_{\mathbf{0}}=\mathbf{0.6}$		${\mathit{\varphi }}_{\mathbf{0}}=\mathbf{0.2}$		${\mathit{\varphi }}_{\mathbf{0}}=\mathbf{0.6}$
Gamma (2)	OSLS	0.83	2.89	0.81	1.23	0.24	2.77	0.19	1.23
	QML	1.35	4.48	1.63	2.02	0.42	4.48	0.38	2.03
	ML	0.15	0.25	0.08	0.14	0.04	0.06	0.04	0.13
Gamma (8)	OSLS	0.97	2.08	0.88	0.94	0.31	2.06	0.22	0.94
	QML	1.18	2.52	1.09	1.15	0.38	2.51	0.28	1.14
	ML	0.87	1.44	0.69	0.64	0.27	1.35	0.19	0.62
Gamma (12)	OSLS	1.02	2.00	0.90	0.90	0.32	1.99	0.23	0.90
	QML	1.17	2.30	1.05	1.04	0.37	2.29	0.27	1.04
	ML	0.97	1.58	0.79	0.69	0.30	1.50	0.21	0.72
Gamma (20)	OSLS	1.06	1.93	0.91	0.88	0.34	1.93	0.24	0.88
	QML	1.15	2.11	1.00	0.96	0.37	2.11	0.26	0.96
	ML	1.03	1.67	0.86	0.73	0.33	1.62	0.23	0.75
t (5)	OSLS	1.34	6.32	1.51	2.86	0.37	6.26	0.29	2.86
	QML	1.56	6.32	2.34	2.86	0.41	6.26	0.41	2.86
	ML	1.05	2.44	1.04	1.11	0.29	2.41	0.21	1.11
t (7)	OSLS	1.26	3.30	1.26	1.51	0.37	3.32	0.29	1.51
	QML	1.30	3.30	1.41	1.51	0.38	3.32	0.32	1.51
	ML	1.10	2.31	1.05	1.05	0.32	2.34	0.25	1.05
t (13)	OSLS	1.20	2.31	1.08	1.06	0.37	2.33	0.26	1.06
	QML	1.20	2.31	1.10	1.06	0.37	2.33	0.26	1.06
	ML	1.15	2.12	1.03	0.97	0.36	2.17	0.24	0.97

, where the asymptotic variance of the true maximum likelihood estimator (MLE) is also given as a benchmark. The results show clearly that the efficiency gain of the OSLS over the QMLE is significant in the case of highly skewed distributions such as Gamma, while this is less so in the case of symmetric distributions such as Student’s t. Note that the QMLE and OSLS of ${\varphi }_0$ have the same asymptotic variances under the Student’s t distribution, which is consistent with the theoretical result (13).

In order to see how much loss of efficiency in the QMLE is recovered by the OSLS estimator, next we calculate the relative reduction in the QMLE efficiency-loss (inefficiency)

$$RIEL\left({\mathit{a}}^{\prime }{\gamma }_0\right)=100\times \frac{\mathrm{acov}\sqrt{T}{\mathit{a}}^{\prime }\left({\widehat{\gamma }}_T^Q-{\gamma }_0\right)-\mathrm{acov}\sqrt{T}{\mathit{a}}^{\prime }\left({\widehat{\gamma }}_T^o-{\gamma }_0\right)}{\mathrm{acov}\sqrt{T}{\mathit{a}}^{\prime }\left({\widehat{\gamma }}_T^Q-{\gamma }_0\right)-\mathrm{acov}\sqrt{T}{\mathit{a}}^{\prime }\left({\widehat{\gamma }}_T^M-{\gamma }_0\right)},$$

where ${\widehat{\gamma }}_T^M$ is the true MLE for ${\gamma }_0$. This measure also indicates which estimator approaches the asymptotic variance lower bound faster.

Figure 1 contains a sample of the numerical outputs. First, Figure 1a shows that 45–60% of the inefficiency of ${\widehat{\theta }}_T^Q$ is recovered by ${\widehat{\theta }}_T^o$ when the innovations $\left\{{\epsilon }_t\right\}$ have a heavy tail Student’s t distribution. Further, $RIEL\left({\theta }_0\right)$ declines sharply as the degrees of freedom increases, indicating that as the distribution of $\left\{{\epsilon }_t\right\}$ gets close to the Gaussian, the QMLE improves quickly and it gets close to the OSLS estimator, and both of them approach the variance lower bound. However, the situation in Figure 1b,c is opposite, where $RIEL\left({\theta }_0\right)$ and $RIEL\left({\varphi }_0\right)$ are increasing with the shape parameter of the Gamma distribution. This indicates that the OSLS improves significantly faster than the QMLE as the skewed distribution gets closer to the Gaussian. In other words, the efficiency loss of the QMLE is persistent and therefore it is not desirable under asymmetric conditional error distribution.

4. Simulation Studies

In this section we carry out Monte Carlo simulations to study the finite sample behaviour of the feasible optimal SLS estimator (FSLS) ${\widehat{\gamma }}_T^o$ and compare it with some other related, commonly used estimators.

4.1. Comparison with Quasi-MLE

We first compare the FSLS with the quasi-maximum likelihood estimator (QMLE). Specifically, we generate the data from the AR(1)-ARCH(1) model in (14) with innovations $\left\{{\epsilon }_t\right\}$ drawn from the standardized distributions of different levels of skewness and kurtosis. We consider various sample sizes including $T=$ 10,000 to approximate the asymptotic results. In each simulation, we vary the values of the parameters $({\theta }_0,{\varphi }_0)$ to represent different levels of persistence in the mean and variance components. For each estimator, we calculate the mean estimates $({\widehat{\theta }}_0,{\widehat{\varphi }}_0)$ and the root mean squared errors $(RMSE\left({\widehat{\theta }}_0\right),RMSE\left({\widehat{\varphi }}_0\right))$ based on 3000 independent replications over 4 different pairs of true values $({\theta }_0,{\varphi }_0)$.

Table 2. Simulation results for FSLS and QMLE under AR(1)-ARCH(1) model with ${\epsilon }_t\sim$ Gamma(2,1).

T		${\widehat{\mathit{\theta }}}_{\mathbf{0}}$	$\mathit{RMSE}\left({\widehat{\mathit{\theta }}}_{\mathbf{0}}\right)$	${\widehat{\mathit{\varphi }}}_{\mathbf{0}}$	$\mathit{RMSE}\left({\widehat{\mathit{\varphi }}}_0\right)$	${\widehat{\mathit{\theta }}}_{\mathbf{0}}$	$\mathit{RMSE}\left({\widehat{\mathit{\theta }}}_{\mathbf{0}}\right)$	${\widehat{\mathit{\varphi }}}_{\mathbf{0}}$	$\mathit{RMSE}\left({\widehat{\mathit{\varphi }}}_{\mathbf{0}}\right)$
		(a):${\mathit{\theta }}_{\mathbf{0}}={\mathit{\varphi }}_{\mathbf{0}}=\mathbf{0.2}$				(b):${\mathit{\theta }}_{\mathbf{0}}=\mathbf{0.2}$, ${\mathit{\varphi }}_{\mathbf{0}}=\mathbf{0.6}$
60	QMLE	0.21	0.149	0.30	0.213	0.19	0.152	0.58	0.163
	FSLS	0.21	0.119	0.27	0.172	0.19	0.121	0.59	0.136
100	QMLE	0.20	0.116	0.25	0.169	0.20	0.121	0.58	0.134
	FSLS	0.20	0.091	0.23	0.138	0.20	0.094	0.59	0.108
1000	QMLE	0.20	0.038	0.20	0.063	0.20	0.040	0.60	0.037
	FSLS	0.20	0.029	0.20	0.052	0.20	0.028	0.60	0.030
10,000	QMLE	0.20	0.012	0.20	0.020	0.20	0.013	0.60	0.012
	OSLS	0.20	0.009	0.20	0.016	0.20	0.009	0.60	0.010
		(c):${\mathit{\theta }}_{\mathbf{0}}=\mathbf{0.8}$, ${\mathit{\varphi }}_{\mathbf{0}}=\mathbf{0.2}$				(d):${\mathit{\theta }}_{\mathbf{0}}=\mathbf{0.8}$, ${\mathit{\varphi }}_{\mathbf{0}}=\mathbf{0.6}$
60	QMLE	0.77	0.098	0.29	0.215	0.77	0.097	0.59	0.161
	FSLS	0.78	0.073	0.27	0.178	0.78	0.077	0.59	0.139
100	QMLE	0.78	0.074	0.25	0.172	0.78	0.071	0.58	0.128
	FSLS	0.79	0.054	0.23	0.140	0.79	0.054	0.59	0.106
1000	QMLE	0.80	0.021	0.20	0.062	0.80	0.021	0.60	0.038
	FSLS	0.80	0.016	0.19	0.050	0.80	0.015	0.60	0.030
10,000	QMLE	0.80	0.007	0.20	0.020	0.80	0.007	0.60	0.012
	OSLS	0.80	0.005	0.20	0.016	0.80	0.005	0.60	0.010

contains the summary results for Gamma (2,1) innovations, which show clearly that the FSLS outperforms the QMLE for all sample sizes and panels, while both estimators have the same degree of bias. We have also done the simulations with Student’s $t\left(5\right)$ innovations and the results show that the FSLS has moderate gain of efficiency over the QMLE, which performs fairly well in small samples.

To understand how the values of $({\theta }_0,{\varphi }_0)$, shape parameter, and sample size T affect the relative RMSE of the FSLS compared to QMLE, we use the numerical results of the simulations to fit two regression equations with $RRMSE\left({\widehat{\theta }}_T^o\right)=RMSE\left({\widehat{\theta }}_T^o\right)/RMSE\left({\widehat{\theta }}_T^Q\right)$ and $RRMSE\left({\widehat{\varphi }}_T^o\right)$ as the response variables, respectively. The results in

Table 3. Effect of the shape, sample size, and parameter values on the RRMSEs under Gamma distribution. All coefficients are significant at 0.0001 level.

	Coefficients					${\mathit{R}}^2$	Error df
	Const	Shape	T	${\mathit{\theta }}_0$	${\mathit{\varphi }}_0$
$RRMSE\left({\widehat{\theta }}_T^o\right)$	0.76546	0.02887	$-0.00007$	$-0.02783$	0.02525	0.78692	4785
$RRMSE\left({\widehat{\varphi }}_T^o\right)$	0.79952	0.02078	$-0.00003$	0.01013	0.00962	0.70135	4785

show that the shape parameter has a positive effect on both RRMSE$\left({\widehat{\theta }}_T^o\right)$ and RRMSE$\left({\widehat{\varphi }}_T^o\right)$, while T is negatively associated with both RRMSEs, indicating that the outperformance of the FSLS in large samples is more evident than in small samples if the innovations distribution is skewed. Moreover, the negative sign of ${\theta }_0$ in the RRMSE$\left({\widehat{\theta }}_T^o\right)$ equation indicates that the performance of the QMLE improves quickly as the value of ${\theta }_0$ gets larger.

4.2. Comparison with Estimating Function Estimators

Our approach is related to the estimating function (EF) approach. Following Durairajan (1992), it can be shown that under models (1) and (2) the EF estimator ${\widehat{\gamma }}_T^{EF}$ is obtained by solving the estimating equation

$$\sum _{t=1}^T{\mathit{B}}_t^{\prime }\left(\gamma \right){\mathsf{\Omega }}_t^{-1}\left(\gamma \right)\left(\begin{array}{c}{ϵ}_t\left(\gamma \right)\\ {ϵ}_t^2\left(\gamma \right)-{\sigma }_t^2\left(\gamma \right)\end{array}\right)=\mathbf{0},$$

(15)

where ${\mathit{B}}_t^{\prime }\left(\gamma \right)$ and ${\mathsf{\Omega }}_t\left(\gamma \right)$ are given in (8) and (9) with ${\mathit{\theta }}_0$, ${\gamma }_0$ and $E\left({\epsilon }_t^j{\mathit{v}}_t,\dots ,{\mathit{v}}_{t-p}\right)$, $j=3,4$ replaced by $\mathit{\theta }$, $\gamma$ and $E_{\gamma }\left({\epsilon }_t^j\left(\gamma \right){\mathit{v}}_t,\dots ,{\mathit{v}}_{t-p}\right)$, $j=3,4$ respectively. The EF in (15) is optimal with respect to the so-called Godambe’s information criterion. Moreover, under some regularity conditions similar to Assumptions A2–A9, the EF estimator can be shown to be $\sqrt{T}$-consistent with $\mathrm{acov}\sqrt{T}\left({\widehat{\gamma }}_T^{EF}-{\gamma }_0\right)$ given by Equation (7). However, although the FSLS and EF estimators have the same asymptotic variance, they are distinct in the following aspects. First, the FSLS is an extremum estimator while the EF estimator represents a solution of the optimal estimating Equation (15). Second, if $E_{\gamma }\left({\epsilon }_t^j\left(\gamma \right){\mathit{v}}_t,\dots ,{\mathit{v}}_{t-p}\right),j=3,4$ are known functions of $\gamma$, then the EF estimator can be calculated in one step, while the FSLS remains to be a two-step estimator due the dependence of ${\mathit{W}}_t$ on ${\gamma }_0$. Third, the two estimators may behave differently in finite-sample situations because they have different estimating equations. This can be seen by comparing Equation (15) with the first order condition for the FSLS which can be written as

$$\sum _{t=1}^T{\mathit{B}}_t^{\prime }\left(\gamma \right){\mathit{H}}_t\left(\mathit{\theta }\right){\mathsf{\Omega }}_t^{-1}{\mathit{H}}_t^{\prime }\left(\mathit{\theta }\right)\left(\begin{array}{c}{ϵ}_t\left(\gamma \right)\\ {ϵ}_t^2\left(\gamma \right)-{\sigma }_t^2\left(\gamma \right)\end{array}\right)=\mathbf{0},$$

where

$${\mathit{H}}_t\left(\mathit{\theta }\right)=\left(\begin{array}{cc}1& 2f_t\left({\mathit{v}}_t,\mathit{\theta }\right)-2f_t\left({\mathit{v}}_t,{\mathit{\theta }}_0\right)\\ 0& 1\end{array}\right).$$

Next we calculate some numerical examples to compare the FSLS with four different versions of the EF estimators that are commonly used in practice. First, since ${\epsilon }_t$ are i.i.d., Equation (15) can be written as

$$\sum _{t=1}^T{\mathit{B}}_t^{\prime }\left({\gamma }_1\right){\mathsf{\Omega }}_t^{-1}({\gamma }_2,{\mu }_3,{\mu }_4)\left(\begin{array}{c}{ϵ}_t\left(\gamma \right)\\ {ϵ}_t^2\left(\gamma \right)-{\sigma }_t^2\left(\gamma \right)\end{array}\right)=\mathbf{0}.$$

(16)

Then we calculate four variants of the EF estimator as follows: the estimator EF0 is obtained by taking ${\gamma }_1={\gamma }_2={\widehat{\gamma }}_T^Q$, ${\mu }_3=1/T{\sum }_{t=1}^T{\epsilon }_t^3\left({\widehat{\gamma }}_T^Q\right)$ and ${\mu }_4=1/T{\sum }_{t=1}^T{\epsilon }_t^4\left({\widehat{\gamma }}_T^Q\right)$; EF1 is the same as EF0 except that ${\gamma }_1=\gamma$; EF is the same as EF0 except that ${\gamma }_1={\gamma }_2=\gamma$; and EF2 is obtained by letting ${\gamma }_1={\gamma }_2=\gamma$, ${\mu }_3=1/T{\sum }_{t=1}^T{\epsilon }_t^3\left(\gamma \right)$ and ${\mu }_4=1/T{\sum }_{t=1}^T{\epsilon }_t^4\left(\gamma \right)$. The four variants have the same asymptotic covariance matrix.

Figure 2 shows the relative RMSE of the FSLS over the RMSE of the QMLE and EF estimators, respectively. Figure 2a is based on 4790 simulations with 3000 independent replications where $\left\{{\epsilon }_t\right\}$ are generated from the standardized Gamma distribution with shape parameters 2, 3, 4, 5, 6, and 7 respectively. Similarly, Figure 2b is based on 4050 simulations with $\left\{{\epsilon }_t\right\}$ generated from the standardized Student’s t distribution with df 5, 6, 7, 8, and 9, respectively. In all cases, sample size T varies over a range (30, 40, 50, 60, 70, 80, 90, 100, 500, and 1000) and the RMSE of the estimators are calculated on the parameters grid $\left\{(0.1,0.1),(0.1,0.2),\dots ,(0.1,0.9),(0.2,0.1),\dots ,(0.9,0.9)\right\}$.

The results show clearly that the FSLS outperform the EF estimators for ${\theta }_0$ in almost all cases and for ${\varphi }_0$ in the majority of the cases. In particular, results of EF2 show that replacing the nuisance parameters with highly nonlinear functions of the estimated parameter makes the performance worse. Therefore in practice, a two-step EF estimator such as the EF or EF1 should be recommended. Finally, the QMLE performs reasonably well in the case of symmetric error distributions, while not so well in the case of skewed error distributions.

5. Application

In this section we apply our method to an empirical example of Engle (1982) (see also Enders 2010), who used an AR model with ARCH error to study the wage/price spiral in the U.K. over the period 1958Q2–1977Q2. Specifically, let $p_t$ denote the log of the consumer price index and $w_t$ denote the log of the index of the nominal wage rates. Then $y_t=p_t-p_{t-1}$ and $r_t=w_t-p_t$ are the rate of inflation and real wage, respectively. Engle (1982) first fitted the following equation using the least squares (LS) method

$$\begin{array}{cccccccccccccc}\hfill y_t=& 0.0257& +& 0.334& y_{t-1}& +& 0.408& y_{t-4}& -& 0.404& y_{t-5}& +& 0.0559& r_{t-1}+{ϵ}_t,\\ \hfill & \left(0.006\right)& & \left(0.103\right)& & & \left(0.110\right)& & & \left(0.114\right)& & & \left(0.014\right)& \\ \hfill {\widehat{\sigma }}_t^2=& 8.9\times {10}^{-5},& & & & & & & & & & & \end{array}$$

(17)

where the standard errors are in parentheses. Since the Lagrange multiplier (LM) test for ARCH(1) error was not significant, but for ARCH(4) process was instead, the following conditional variance equation was specified

$${\sigma }_t^2={\varphi }_0+{\varphi }_1(0.4{ϵ}_{t-1}^2+0.3{ϵ}_{t-2}^2+0.2{ϵ}_{t-3}^2+0.1{ϵ}_{t-4}^2),$$

(18)

where the two-parameter variance function with declining weights was chosen to satisfy the nonnegativity and stationarity constraints. Further, Engle (1982) fitted Equations (17) and (18) jointly using the Gaussian quasi-likelihood method, where all coefficients (except the first lag in the inflation rate) were significant at level $0.05$.

Since we could not obtain the wage rates before 1963, we use the data from 1963Q1 through 1982Q1 to compensate for the 19 missing quarters. The data were obtained from the OECD website http://dx.doi.org/10.1787/data-00052-en (accessed on 15 January 2014, see OECD, Main Economic Indicators–complete database). To see if there is major structural difference in our data compared to the data used by Engle (1982), we have done an initial investigation and found no evidence of structural change by replacing the 19 quarters. So our results are to some extent comparable with those in Engle (1982).

We start by fitting the following regression model for inflation using the LS method

$$y_t={\theta }_0+{\theta }_1{y}_{t-1}+{\theta }_2{y}_{t-2}+{\theta }_3{y}_{t-3}+{\theta }_4{y}_{t-4}+{\theta }_5{y}_{t-5}+{\theta }_6{r}_{t-1}+{ϵ}_t$$

(19)

The results are shown in the first part of

Table 4. Fitted Model-I (19) and (20) and Model-II (21) and (22), where standard errors are in parentheses. The superscript ${}^a$ indicates statistical significance at 5% level. $Q1\left(n\right)$ ($Q2\left(n\right)$) is the Ljung-Box statistic for the (squared) standardized innovations and the corresponding p-values are in parentheses. JB is the standard Jarque-Bera test.

Coef.	Model-I				Model-II
	LS	QMLE	FSLS		QMLE	FSLS
	Conditional Mean Equation
${\theta }_0$	${0.071}^a$	${0.079}^a$	${0.049}^a$		${0.067}^a$	${0.063}^a$
	(0.016)	(0.015)	(0.014)		(0.015)	(0.013)
${\theta }_1$	${0.417}^a$	${0.339}^a$	${0.280}^a$		${0.323}^a$	${0.257}^a$
	(0.141)	(0.1)	(0.093)		(0.096)	(0.086)
${\theta }_2$	$-0.039$	$0.004$	0.050		–	–
	(0.095)	(0.086)	(0.081)
${\theta }_3$	$-0.180$	$-{0.235}^a$	−0.158		–	–
	(0.148)	(0.101)	(0.094)
${\theta }_4$	${0.436}^a$	${0.481}^a$	${0.563}^a$		${0.328}^a$	${0.339}^a$
	(0.184)	(0.108)	(0.101)		(0.116)	(0.104)
${\theta }_5$	$-{0.350}^a$	$-{0.310}^a$	$-{0.294}^a$		$-{0.246}^a$	$-{0.234}^a$
	(0.101)	(0.094)	(0.088)		(0.094)	(0.084)
${\theta }_6$	${0.076}^a$	${0.086}^a$	${0.051}^a$		${0.073}^a$	${0.067}^a$
	(0.018)	(0.017)	(0.016)		(0.017)	(0.015)
	Conditional Variance Equation
${\varphi }_0$	$0.0001$	${0.000}^a$	0.000		${0.000}^a$	$0.000$
		(0.000)	(0.000)		(0.000)	(0.000)
${\varphi }_1$	$0.1064$	0.093	0.021		–	–
		(0.134)	(0.126)
${\varphi }_2$	$0.0000$	0.000	0.000		–	–
		(0.077)	(0.072)
${\varphi }_3$	$0.0806$	0.100	0.102		–	–
		(0.146)	(0.137)
${\varphi }_4$	$0.3364$	$0.389$	${0.479}^a$		0.553	${0.556}^a$
		(0.252)	(0.235)		(0.308)	(0.281)
	Diagnostic Statistics of the Standardized Innovations
Q1(5)	0.9 (0.97)	1.0 (0.96)	2.1 (0.83)		2.0 (0.85)	1.9 (0.86)
Q1(10)	4.4 (0.93)	7.4 (0.68)	6.4 (0.78)		8.4 (0.59)	8.5 (0.58)
Q1(15)	12.8 (0.62)	16.1 (0.38)	14.1 (0.52)		19.6 (0.19)	18.7 (0.23)
Q2(5)	3.7 (0.59)	4.4 (0.50)	0.4 (0.99)		6.5 (0.26)	6.4 (0.27)
Q2(10)	5.7 (0.84)	6.9 (0.74)	1.3 (0.99)		8.2 (0.61)	8.1 (0.62)
Q2(15)	9.2 (0.87)	9.8 (0.83)	2.7 (0.99)		9.1 (0.87)	9.4 (0.85)
Skewness	0.78	0.61	1.67		0.73	0.99
Kurtosis	4.07	3.49	8.48		3.73	4.27
JB	${11.0}^a$	5.35	${115}^a$		${8.12}^a$	${16.93}^a$

under Model-I (LS), where the White’s correction for the standard errors are reported in parentheses.

We calculate the Ljung-Box statistic Q for ${\widehat{\epsilon }}_t$ (denoted by Q1) and ${\widehat{\epsilon }}_t^2$ (denoted by Q2) at lags 5, 10, 15, and 20. They are all insignificant at level $0.1$ except for Q2(5), which agrees with Engle (1982) in including four lags in the variance equation. Further, we use the squared residuals from this regression to fit an ARCH(4) model for the conditional variance

$${\sigma }_t^2={\varphi }_0+{\varphi }_1{ϵ}_{t-1}^2+{\varphi }_2{ϵ}_{t-2}^2+{\varphi }_3{ϵ}_{t-3}^2+{\varphi }_4{ϵ}_{t-4}^2$$

(20)

The results are shown in the second part of Table 4 under Model-I (LS). Again, the ARCH(4) model is confirmed by the LM test at significance level 0.05. We report only the LS estimates of the variance function without standard errors because those estimates are only used as starting values to compute the QMLE. The Q1 and Q2 statistics from Model-I (LS) (in the third part of Table 4) indicate that the mean and variance equations are fairly well specified since none of these diagnostics are significant at level 0.1. Therefore we fit the Model-I again using the QMLE, which is more efficient than the LS procedure. Although the diagnostics of the standardized innovations from Model-I (QMLE) do not show serial correlation of the first or second order, all coefficients in the variance function are insignificant except for the constant term. This contradicts the ARCH(4) that we found before to be correctly specified. However, this can be explained by the lack of efficiency in the QMLE due to the moderate level of skewness in the corresponding residuals.

On the other hand, our FSLS estimation yields a significant fourth lag in the variance function in addition to the correct specification as indicated by Q1 and Q2. Accordingly, we use the model fitted by FSLS in stepwise regression algorithm to obtain a reduced model (Model-II in Table 4)

$$y_t={\theta }_0+{\theta }_1{y}_{t-1}+{\theta }_4{y}_{t-4}+{\theta }_5{y}_{t-5}+{\theta }_6{r}_{t-1}+{ϵ}_t$$

(21)

$${\sigma }_t^2={\varphi }_0+{\varphi }_4{ϵ}_{t-4}^2$$

(22)

Note that while the mean equation is identical to that in Engle (1982), only the fourth lag is significant in the variance equation. The above ARCH structure can only be detected by using the full model that is more flexible than the two-parameter variance function in Equation (18). Moreover, the more efficient FSLS estimation yields the ARCH(4) structure, while the QMLE would conclude with a misspecified homoscedastic model.

6. Conclusions and Discussion

Although the ARCH-type models have been extensively studied for decades, most theories and methods are developed for stationary data processes whose conditional mean functions are either ARMA or simple linear functions of covariates. Moreover, recent research has been focusing on generalization of the error component while leaving the mean function to be simple linear form. However, many economic and financial time series are nonlinear and/or nonstationary in the mean, therefore, data transformation is required in order to apply standard methodologies in the analysis.

In this paper, we proposed the second-order least squares (SLS) approach to estimate a flexible and general model with nonlinear and time-varying conditional mean and ARCH conditional variance function. This approach is applicable to both stationary and mean-nonstationary processes and therefore can be used to analyze the transformed as well as the original data. Another advantage of this approach is that it does not require specification of the underlying distribution for the errors. Moreover, the feasible optimal SLS estimator (FSLS) is more efficient than the commonly used QMLE, and the efficiency gain is significant when the conditional error distribution is asymmetric. We have demonstrated through a real data example that the efficiency gain of the proposed approach leads to a more accurate model than the QMLE. The third and fourth conditional moments of the innovation provide useful information that is utilized by the FSLS (through the weight matrix) to gain efficiency over the QMLE. This information is even more important in the case of skewed and/or leptokurtic error distributions. Our simulation studies also show that the SLS approach has better finite sample properties than the estimating function approach based on the same set of conditional moments.

There are some issues remaining to be studied in the future. Some assumptions for the asymptotic theories are sufficient but not necessary. They are adopted here mainly due to the way of the proofs we used. As is common in statistics and econometrics, there are usually different ways to prove an asymptotic theory, and each of them requires a specific set of assumptions. Therefore it is interesting to explore the possibilities of establishing the asymptotic properties for the FSLS estimator under the similar conditions for the QMLE. Indeed, our Monte Carlo simulation studies have shown that the FSLS estimator performs well in the finite sample situations even if the innovation does not have the finite moments of order higher than four. From the application point of view, it is also important to extend the method of this paper to models with more general GARCH errors. This is possible by modifying some assumptions to adapt for the mixing process.