Detection of Interaction Effects in a Nonparametric Concurrent Regression Model

Abstract

Many methods have been developed to study nonparametric function-on-function regression models. Nevertheless, there is a lack of model selection approach to the regression function as a functional function with functional covariate inputs. To study interaction effects among these functional covariates, in this article, we first construct a tensor product space of reproducing kernel Hilbert spaces and build an analysis of variance (ANOVA) decomposition of the tensor product space. We then use a model selection method with the $L_1$ criterion to estimate the functional function with functional covariate inputs and detect interaction effects among the functional covariates. The proposed method is evaluated using simulations and stroke rehabilitation data.

1. Introduction

Functional data can be found in various fields, such as biology, economics, engineering, geology, medicine and psychology. Recently, statistical methods and theories on functional data were widely studied ([1,2,3,4,5,6]). Functional data sometimes have more complicated structures. For example, the motivation data of this paper, stroke rehabilitation data, utilized a collection of 3D video games known as Circus Challenge to enhance upper limb function in stroke patients ([7,8,9]). The patients were scheduled to play the movement game over three months at specified times. At each visit time t, the level of impairment of stroke subject i was measured using the CAHAI (Chedoke Arm and Hand Activity Inventory) score, denoted as $y_i\left(t\right)$, and movements of upper limbs of patients, such as forward circle movement and sawing movement, were also recorded. The movement data at time t and frequency s from the ith patient were denoted as $x_i(t,s)$. Determining a way to model the relationship of $y_i\left(t\right)$ and functional $x_i(t,·)$ is key to studying whether the movements are helpful to the rehabilitation level of the stroke patient or not. Furthermore, there is a question of whether there are interaction effects among the movements on the stroke patient’s rehabilitation. Zhai et al. [9] developed a nonparametric concurrent regression model to study the relationship between functional movements and the CAHAI score. However, they did not consider the interaction effects of functional movements on the CAHAI score. We aim to examine the interaction effects of movements on CAHAI scores and predict the rehabilitation level of stroke patients in this paper.

In this paper, we apply the following nonparametric concurrent regression model (NCRM) to model stroke rehabilitation data:

$$\begin{array}{c}\hfill y_i\left(t\right)=f(t,x_i(t,·))+{ϵ}_i\left(t\right),i=1,\dots ,n,\end{array}$$

(1)

where f is a bivariate functional to be estimated nonparametrically, response $y_i\left(t\right)$ is a function of t, covariate $x_i(t,·)$ is a vector of functional with length q, and ${ϵ}_i\left(t\right)$ is a random error. To explore the interaction effects among components of covariates $x_i(t,·)$, we use the smooth spline analysis of variance (SS ANOVA) method [10,11] to decompose regression function f.

A multivariate function can be decomposed of main effects and interaction effects via the SS ANOVA method ([10,11]). When the dimension of covariates q is large, the decomposed model contains a large number of interaction effects. Even if only the main effects and second-order interaction terms are investigated, the order of the number of decomposition terms is $O\left(q^2\right)$, which leads to a highly complicated model. To model stroke rehabilitation data with $q=3$, there are 22 terms, including the main effects and interaction effects. This challenges the estimation method for the NCRM model. To avoid this shortcoming, Zhai et al. [9] took all functional covariates as a whole and did not consider interaction effects among covariates. Following [12], this paper conducts a model selection method for the NCRM model with all main effects and interaction effects. In this method, the regression function is estimated and significant components of the decomposition are selected simultaneously.

Model selection is a crucial step in building statistical models that accurately capture the relationships between variables ([13,14]). It can choose the most suitable model from a set of candidate models based on certain criteria such as goodness-of-fit, predictive performance, and interpretability. Based on the SS ANOVA approach, model selection is crucial to determine the contribution of each component of the decomposition to the overall variance of the response variable. Several methods have been proposed for selecting models with SS ANOVA, including forward selection, backward elimination, and stepwise regression ([15,16,17,18,19,20]). However, these methods are limited in their ability to handle high-dimensional data and identify complex interactions among variables. Hence, regularization methods such as the $L_1$ penalty have gained popularity in recent years ([12,21,22,23,24,25]), which allow for the selection of sparse and robust models. For example, Zhang et al. [23] developed a nonparametric penalized likelihood method with the likelihood basis pursuit and used it for variable selection and model construction. Lin and Zhang [22] proposed a component selection and smoothing method for multivariate nonparametric regression models by penalizing the sum of component norms of SS ANOVA. Furthermore, Wang et al. [12] developed a unified framework for estimation and model selection methods in nonparametric function-on-function regression, which performs well when using $L_1$ penalty methods for model selection. Dong and Wang [24] proposed a nonparametric method for learning conditional dependence structure in graph models by applying $L_1$ regularization to detect the neighborhoods of edges, where SS ANOVA decomposition is used to depict interaction effects of edges in the graph model. In this paper, we borrowed the $L_1$ regularization idea to build model selection by penalizing the sum of norms of the ANOVA decomposition components for the NCRM model. In addition, Bayesian analysis methods can also be used to study interaction effects; for example, Ren et al. [26] proposed a novel semiparametric Bayesian variable selection model for investigating linear and nonlinear gene–environment interactions simultaneously, allowing for structural identification.

This paper proposes an estimation and model selection approach for the NCRM model (1). Following [12,22], the SS ANOVA decomposition for the tensor product space of the reproducing kernel Hilbert spaces (RKHS) is constructed, and the $L_1$ penalty approach for the components of the decomposition is implemented. We use estimation procedures under either an $L_1$ or a joint $L_1$ and $L_2$ penalty to fit teh NCRM model. We study the interaction effects of the covariate $x_i(t,·)$ in model (1) via ANOVA decomposition of the regression function, where the tensor product RKHS is built based on Gaussian kernels. The decomposition is different from that of Zhai et al. [9], where they took the covariate as a whole variable and did not consider their interaction effects. Based on the decomposition, model selection with the tensor product RKHS is conducted using the $L_1$ penalty method. With regards to the covariate $x_i(t,·)$, the models from Wang et al. [12] are not suitable to analyze the stoke data. In this paper, we apply the proposed method to stroke rehabilitation data and study the relationship of the movements and the patient’s CAHAI score. Besides the main effects, the interaction effect of the movements is also detected.

The remainder of the article is organized as follows. In Section 2, we present the tensor product RKHS with the Gaussian kernel and the SS ANOVA decomposition of the regression function. In Section 3, we show model selection and estimation procedures. The simulation study and application of stroke rehabilitation data are presented in Section 4 and Section 5. We conclude in Section 6.

2. Nonparametric Concurrent Regression Model

For the NCRM model (1), we consider $x_i(t,·)=(x_{i1}(t,·),\dots ,x_{iq}(t,·))$, where $x_{ij}(t,s):\mathcal{S}\to \mathcal{R}$ for any fixed time $t\in \mathcal{T}$ is a function of s within a space denoted by ${\mathcal{X}}_j$, $j\in 1,\cdots ,q$. Generally, t and s can be transformed into $[0,1]$. For simplicity, we let $\mathcal{T}=[0,1]$ and $\mathcal{S}=[0,1]$ and let ${\mathcal{X}}_j\subset L^2[0,1]$, $j=1,\cdots ,q$, which are independent of t. Furthermore, we assume that $y_i\left(t\right)\in \mathcal{Y}\subset L^2[0,1]$ and ${ϵ}_i\left(t\right)$ for $i=1,\dots ,n$ are identically and independently distributed in $L^2[0,1]$ with mean zero and ${\int }_0^{1}E\left[{ϵ}_i{\left(t\right)}^2\right]dt<\infty$. It is shown that the regression function f is a functional function with an independent covariate $x_i(t,·)$. To provide a nonparametric estimation of f, the SS ANOVA decomposition method is used to construct a tensor product space of RKHS to which f belongs.

When f is treated as a function with respect to the first augment $t\in \mathcal{T}$, we consider the Sobolev space [10],

$${\mathcal{H}}^{\left(1\right)}=\left\{f:f\mathrm{and}{f}^{\prime }\mathrm{absolutely}\mathrm{continuous},{\int }_0^1{\left(f^{″}\right)}^{2}dt<\infty \right\},$$

(2)

where ${\mathcal{H}}^{\left(1\right)}$ can be rewritten as

$${\mathcal{H}}^{\left(1\right)}=\left\{1\right\}\oplus \left\{t\right\}\oplus {\mathcal{H}}_2^{\left(1\right)},$$

where $\left\{1\right\}$ is a constant space, $\left\{t\right\}$ is a linear function space with t as an independent variable. ${\mathcal{H}}_2^{\left(1\right)}$ is a smooth function space orthogonal to the constant space and the linear function space. Reproducing kernels (RK) for these three subspaces are ${\mathcal{K}}_0^{\left(1\right)}(t,t^{\prime })=1$, ${\mathcal{K}}_1^{\left(1\right)}(t,t^{\prime })=k_1\left(t\right)k_1\left(t^{\prime }\right)$, and ${\mathcal{K}}_2^{\left(1\right)}(t,t^{\prime })=k_2\left(t\right)k_2\left(t^{\prime }\right)-k_4\left(\right|t-t^{\prime }\left|\right)$, where $k_1$, $k_2$ and $k_4$ are defined as

$$\begin{array}{cc}\hfill k_1\left(x\right)& =x-0.5,\hfill \\ \hfill k_2\left(x\right)& =\frac{1}{2}\left\{k_1^2\left(x\right)-\frac{1}{12}\right\},\hfill \\ \hfill k_4\left(x\right)& =\frac{1}{24}\left\{k_1^4\left(x\right)-\frac{1}{2}{k}_1^2\left(x\right)+\frac{7}{240}\right\}.\hfill \end{array}$$

For functional augments $x(t,·)$, RK and its corresponding RKHS for f as a function of functions in $\mathcal{X}={\mathcal{X}}_1\times \cdots \times {\mathcal{X}}_q$ are constructed as follows. For any $u_j,u_j^{\prime }\in {\mathcal{X}}_j$, we construct a Gaussian kernel as

$$\begin{array}{c}\hfill {\mathcal{K}}_{2,j}^{\left(2\right)}(u_j,u_j^{\prime })=exp\left\{-\frac{{∥u_j-u_j^{\prime }∥}^2}{2}\right\},\end{array}$$

(3)

where ${∥u_j∥}^2={\int }_0^1{u}_j^2\left(s\right)ds$. We can show that when the space ${\mathcal{X}}_j$ is a complete space, ${\mathcal{K}}_{2,j}^{\left(2\right)}$ is a symmetric and strictly positive definite. The unique RKHS ${\mathcal{H}}_{2,j}^{\left(2\right)}$ derived from ${\mathcal{K}}_{2,j}^{\left(2\right)}$ is separable and does not contain any non-zero constants. To construct an SS ANOVA decomposition, we let ${\mathcal{H}}_j^{\left(2\right)}=\left\{1\right\}\oplus {\mathcal{H}}_{2,j}^{\left(2\right)}$. Then, the tensor product space in this paper is ${\mathcal{H}}^{\left(2\right)}={\mathcal{H}}_1^{\left(2\right)}\otimes \cdots \otimes {\mathcal{H}}_q^{\left(2\right)}$ with the following decomposition:

$$\begin{array}{cc}\hfill {\mathcal{H}}^{\left(2\right)}=& {\mathcal{H}}_1^{\left(2\right)}\otimes \cdots \otimes {\mathcal{H}}_q^{\left(2\right)}\hfill \\ \hfill =& \left\{1\right\}\oplus {\mathcal{H}}_{2,1}^{\left(2\right)}\oplus \cdots \oplus {\mathcal{H}}_{2,q}^{\left(2\right)}\oplus \left\{{\mathcal{H}}_{2,1}^{\left(2\right)}\otimes {\mathcal{H}}_{2,2}^{\left(2\right)}\right\}\oplus \cdots \oplus \hfill \\ \hfill & \left\{{\mathcal{H}}_{2,q-1}^{\left(2\right)}\otimes {\mathcal{H}}_{2,1}^{\left(2\right)}\right\}\oplus \cdots \oplus \left\{{\mathcal{H}}_{2,1-1}^{\left(2\right)}\otimes \cdots \otimes {\mathcal{H}}_{2,q}^{\left(2\right)}\right\}.\hfill \end{array}$$

(4)

Decomposition (4) is different from that of Zhai et al. [9] where ${\mathcal{H}}^{\left(2\right)}$ is decomposed of constant space $\left\{1\right\}$ and another RKHS not considering interaction among $x(t,·)$.

Next, we consider the tensor product space $\mathcal{H}={\mathcal{H}}^{\left(1\right)}\otimes {\mathcal{H}}^{\left(2\right)}$ which has the following decomposition:

$$\begin{array}{cc}\hfill \mathcal{H}=& \left(\left\{1\right\}\oplus \left\{t\right\}\oplus {\mathcal{H}}_2^{\left(1\right)}\right)\otimes \left(\left\{1\right\}\oplus \left\{{\oplus }_{j=1}^q{\mathcal{H}}_{2,j}^{\left(2\right)}\right\}\oplus \cdots \oplus \left\{{\mathcal{H}}_{2,1}^{\left(2\right)}\otimes \cdots \otimes {\mathcal{H}}_{2,q}^{\left(2\right)}\right\}\right)\hfill \\ \hfill =& \left\{1\right\}\oplus \left\{t\right\}\oplus {\mathcal{H}}_2^{\left(1\right)}\oplus \left\{{\oplus }_{j=1}^q{\mathcal{H}}_{2,j}^{\left(2\right)}\right\}\oplus \left\{{\oplus }_{j=1}^q\{{\mathcal{H}}_{2,j}^{\left(2\right)}\times \left\{t\right\}\}\right\}\oplus \hfill \\ \hfill & \left\{{\oplus }_{j=1}^q\{{\mathcal{H}}_{2,j}^{\left(2\right)}\times {\mathcal{H}}_2^{\left(1\right)}\}\right\}\oplus \cdots \oplus \left\{{\mathcal{H}}_{2,1}^{\left(2\right)}\otimes \cdots \otimes {\mathcal{H}}_{2,q}^{\left(2\right)}\right\}\oplus \hfill \\ \hfill & \left\{{\mathcal{H}}_{2,1}^{\left(2\right)}\otimes \cdots \otimes {\mathcal{H}}_{2,q}^{\left(2\right)}\times \left\{t\right\}\right\}\oplus \left\{{\mathcal{H}}_{2,1}^{\left(2\right)}\otimes \cdots \otimes {\mathcal{H}}_{2,q}^{\left(2\right)}\otimes {\mathcal{H}}_2^{\left(1\right)}\right\}.\hfill \end{array}$$

There, the null space $\left\{1\right\}\oplus \left\{t\right\}$ stands for the main effect of the parametric form of t, ${\mathcal{H}}_2^{\left(1\right)}$ is the main effect of the non-parametric form of t, ${\mathcal{H}}_{2,j}^{\left(2\right)}$ is the main effect of the non-parametric form of $u_j$, $\left\{t\right\}\otimes {\mathcal{H}}_{2,j}^{\left(2\right)}$ is the linear nonparametric interaction between t and $u_j$, ${\mathcal{H}}_2^{\left(1\right)}\otimes {\mathcal{H}}_{2,j}^{\left(2\right)}$ is the nonparametric nonparametric interaction between t and $u_j$, and so on, ${\mathcal{H}}_2^{\left(1\right)}\otimes {\mathcal{H}}_{2,1}^{\left(2\right)}\otimes \cdots \otimes {\mathcal{H}}_{2,q}^{\left(2\right)}$ is the nonparametric nonparametric interaction between t and u, where $\mathit{u}=(u_1,\dots ,u_q)$. We denote ${\varphi }_1(t,\mathit{u})=1$ and ${\varphi }_2(t,\mathit{u})=k_1\left(t\right)$ as the basis functions of ${\mathcal{H}}_0$. For example, with $q=3$, the RKs corresponding to the above sub-RKHS are

$$\begin{array}{cc}\hfill {\mathcal{H}}_0:=\left\{1\right\}\oplus \left\{t\right\}& ⟷{\mathcal{K}}_0((t,\mathit{u}),(t^{\prime },{\mathit{u}}^{\prime }))=1+k_1\left(t\right)k_1\left(t^{\prime }\right),\hfill \\ \hfill {\mathcal{H}}_1:={\mathcal{H}}_2^{\left(1\right)}& ⟷{\mathcal{K}}_1((t,\mathit{u}),(t^{\prime },{\mathit{u}}^{\prime }))={\mathcal{K}}_2^{\left(1\right)}(t,t^{\prime }),\hfill \\ \hfill {\mathcal{H}}_{1+j}:={\mathcal{H}}_{2,j}^{\left(2\right)}& ⟷{\mathcal{K}}_{1+j}((t,\mathit{u}),(t^{\prime },{\mathit{u}}^{\prime }))={\mathcal{K}}_{2,j}^{\left(2\right)}(u_j,u_j^{\prime }),\hfill \\ \hfill {\mathcal{H}}_{4+j}:=\left\{t\right\}\otimes {\mathcal{H}}_{2,j}^{\left(2\right)}& ⟷{\mathcal{K}}_{4+j}((t,\mathit{u}),(t^{\prime },{\mathit{u}}^{\prime }))=k_1\left(t\right)k_1\left(t^{\prime }\right){\mathcal{K}}_{2,j}^{\left(2\right)}(u_j,u_j^{\prime }),\hfill \\ \hfill {\mathcal{H}}_{7+j}:={\mathcal{H}}_2^{\left(1\right)}\otimes {\mathcal{H}}_{2,j}^{\left(2\right)}& ⟷{\mathcal{K}}_{7+j}((t,\mathit{u}),(t^{\prime },{\mathit{u}}^{\prime }))={\mathcal{K}}_2^{\left(1\right)}(t,t^{\prime }){\mathcal{K}}_{2,j}^{\left(2\right)}(u_j,u_j^{\prime }),\hfill \\ \hfill {\mathcal{H}}_{8+j+l}:={\mathcal{H}}_{2,j}^{\left(2\right)}\otimes {\mathcal{H}}_{2,l}^{\left(2\right)}& ⟷{\mathcal{K}}_{8+j+l}:=((t,\mathit{u}),(t^{\prime },{\mathit{u}}^{\prime }))={\mathcal{K}}_{2,j}^{\left(2\right)}(u_j,u_j^{\prime }){\mathcal{K}}_{2,l}^{\left(2\right)}(u_l,u_l^{\prime }),\hfill \\ \hfill {\mathcal{H}}_{11+j+l}:=\left\{t\right\}\otimes {\mathcal{H}}_{2,j}^{\left(2\right)}\otimes {\mathcal{H}}_{2,l}^{\left(2\right)}& ⟷{\mathcal{K}}_{11+j+l}((t,\mathit{u}),(t^{\prime },{\mathit{u}}^{\prime }))=k_1\left(t\right)k_1\left(t^{\prime }\right){\mathcal{K}}_{2,j}^{\left(2\right)}(u_j,u_j^{\prime }){\mathcal{K}}_{2,l}^{\left(2\right)}(u_l,u_l^{\prime }),\hfill \\ \hfill {\mathcal{H}}_{14+j+l}:={\mathcal{H}}_2^{\left(1\right)}\otimes {\mathcal{H}}_{2,j}^{\left(2\right)}\otimes {\mathcal{H}}_{2,l}^{\left(2\right)}& ⟷{\mathcal{K}}_{14+j+l}((t,\mathit{u}),(t^{\prime },{\mathit{u}}^{\prime }))={\mathcal{K}}_2^{\left(1\right)}(t,t^{\prime }){\mathcal{K}}_{2,j}^{\left(2\right)}(u_j,u_j^{\prime }){\mathcal{K}}_{2,l}^{\left(2\right)}(u_l,u_l^{\prime }),\hfill \\ \hfill {\mathcal{H}}_{20}:={\mathcal{H}}_{2,1}^{\left(2\right)}\otimes {\mathcal{H}}_{2,3}^{\left(2\right)}\otimes {\mathcal{H}}_{2,3}^{\left(2\right)}& ⟷{\mathcal{K}}_{20}((t,\mathit{u}),(t^{\prime },{\mathit{u}}^{\prime }))=\prod _{j=1}^3{\mathcal{K}}_{2,j}^{\left(2\right)}(u_j,u_j^{\prime }),\hfill \\ \hfill {\mathcal{H}}_{21}:=\left\{t\right\}\otimes {\mathcal{H}}_{2,1}^{\left(2\right)}\otimes {\mathcal{H}}_{2,3}^{\left(2\right)}\otimes {\mathcal{H}}_{2,3}^{\left(2\right)}& ⟷{\mathcal{K}}_{21}((t,\mathit{u}),(t^{\prime },{\mathit{u}}^{\prime }))=k_1\left(t\right)k_1\left(t^{\prime }\right)\prod _{j=1}^3{\mathcal{K}}_{2,j}^{\left(2\right)}(u_j,u_j^{\prime }),\hfill \\ \hfill {\mathcal{H}}_{22}:={\mathcal{H}}_2^{\left(1\right)}\otimes {\mathcal{H}}_{2,1}^{\left(2\right)}\otimes {\mathcal{H}}_{2,3}^{\left(2\right)}\otimes {\mathcal{H}}_{2,3}^{\left(2\right)}& ⟷{\mathcal{K}}_{22}((t,\mathit{u}),(t^{\prime },{\mathit{u}}^{\prime }))={\mathcal{K}}_2^{\left(1\right)}(t,t^{\prime })\prod _{j=1}^3{\mathcal{K}}_{2,j}^{\left(2\right)}(u_j,u_j^{\prime }),\hfill \end{array}$$

for $j,l=1,2,3$ and $j<l$, where the left and right parts stand for the tensor product spaces and their corresponding RKs, respectively.

3. Model Selection and Estimation

We let the projection of f onto ${\mathcal{H}}_0$ be ${\sum }_{k=1}^2{d}_k{\varphi }_k(t,\mathit{u})$, $\mathit{u}=(u_1,\dots ,u_q)$, and $\{{\mathcal{H}}_1,\cdots ,{\mathcal{H}}_Q\}$ be the sub-RKHS generated by the tensor product method in Section 2, where Q is a number of sub-RKHS. $L_1$ penalties are applied to coefficients $d_k$ for the space ${\mathcal{H}}_0$ and components of the decomposition of f (projections of f onto ${\mathcal{H}}_j,j=1,\cdots ,Q$). We estimate f by minimizing the following penalized least squares:

$$\begin{array}{c}\hfill \frac{1}{n}\sum _{i=1}^n\left\{{\int }_0^1{(y_i\left(t\right)-f(t,x_i(t,·)))}^{2}dt\right\}+{\lambda }_1\sum _{k=1}^2{w}_{1k}\left|d_k\right|+{\lambda }_2\sum _{v=1}^Q{w}_{2,v}{∥P_{v}f∥}_{\mathcal{H}},\end{array}$$

(5)

where $f\in \mathcal{H}$, $P_v$ is the projection operator onto ${\mathcal{H}}_j$, ${∥·∥}_{\mathcal{H}}$ is a norm induced from $\mathcal{H}$, ${\lambda }_1$ and ${\lambda }_2$ are tuning parameters, and $0\le w_{1k},w_{2,v}<\infty$ are pre-specified weights. We may set $w_{11}=0$ when ${\varphi }_1=1$ to avoid penalty to the constant function.

Since the response function is a stochastic process in the $L^2[0,1]$ space, there exists a set of orthogonal basis functions $\{{\eta }_k\left(t\right),k=1,2,\dots \}$ in $L^2[0,1]$, where $\{{\eta }_k\left(t\right),k=1,2,\dots ,n\}$ is an empirical functional principal component (EFPC) of $\{y_1\left(t\right),\cdots ,y_n\left(t\right)\}$ ([27]). We let ${\nu }_{ik}=<y_i\left(t\right),{\eta }_k\left(t\right)>$ and $L_{ik}f={\int }_0^{1}f(t,x_i(t,·)){\eta }_k\left(t\right)dt$ for $i=1,2,\dots ,n$ and $k=1,\dots ,n$. We assume that $\left\{L_{ik}\right\}$ are bounded linear functionals. With EFPC, functional data can be transformed to scalar data such that modeling and analysis can be conducted by using traditional statistical methods. It can show that the PLS (5) based on functional data $y_i\left(t\right)$ reduces to the following PLS based on scalar data $\left\{{\nu }_{ik}\right\}$:

$$\begin{array}{c}\hfill \frac{1}{n}\sum _{i=1}^n\sum _{k=1}^n\left\{{({\nu }_{ik}-L_{ik}f)}^2\right\}+{\lambda }_1\sum _{k=1}^2{w}_{1k}\left|d_k\right|+{\lambda }_2\sum _{v=1}^Q{w}_{2,v}{∥P_{v}f∥}_{\mathcal{H}}.\end{array}$$

(6)

By Lemma 3.1 in Wang et al. [12], minimizing the PLS (6) is equivalent to minimizing the following PLS:

$$\begin{array}{c}\hfill \frac{1}{n}\sum _{i=1}^n\sum _{k=1}^n\left\{{({\nu }_{ik}-L_{ik}f)}^2\right\}+{\lambda }_1\sum _{k=1}^2{w}_{1k}\left|d_k\right|+{\tau }_0\sum _{v=1}^Q{w}_{2,v}{\theta }_v^{-1}{∥P_{v}f∥}_{\mathcal{H}}^2+{\tau }_1\sum _{v=1}^Q{w}_{2,v}{\theta }_v,\end{array}$$

(7)

subject to ${\theta }_v\ge 0$ for $1\le v\le Q$, where ${\lambda }_1$, ${\tau }_0$, ${\tau }_1$ are tuning parameters.

We let ${\mathcal{H}}^{*}={\mathcal{H}}_1\oplus \cdots \oplus {\mathcal{H}}_Q$. To provide an RK with linear combination of its subspaces RK for the space of ${\mathcal{H}}^{*}$, we define a new inner product in ${\mathcal{H}}^{*}$,

$$\begin{array}{c}\hfill <f,g{>}_{*}=\sum _{v=1}^Q{w}_{2,v}{\theta }_v^{-1}<P_{v}f,P_{v}g>,\end{array}$$

(8)

where $<·,·>$ is the inner product in $\mathcal{H}$. Under the new inner product, the RK of ${\mathcal{H}}_1^{*}$ is

$$\begin{array}{c}\hfill {\mathcal{K}}^{*}((t,\mathit{u}),(t^{\prime },{\mathit{u}}^{\prime }))=\sum _{v=1}^Q{w}_{2,v}^{-1}{\theta }_v{\mathcal{K}}_v((t,\mathit{u}),(t^{\prime },{\mathit{u}}^{\prime })),\end{array}$$

where coefficient ${\theta }_v$ can measure the contribution of the components of the decomposition to the model. Next, we use the reproducing property of the kernel function to transform the infinite-dimensional optimization problem (7) into a finite-dimensional solution problem. We let ${\mathcal{H}}_{1n}=\mathrm{span}\{{\int }_0^1{\mathcal{K}}^{*}((t,x(t,·)),(t^{\prime },x_i(t^{\prime },·))){\eta }_k\left(t^{\prime }\right)d{t}^{\prime },i=1,2,\dots ,n,k=1,2,\dots ,n\}$, which is a subspace of ${\mathcal{H}}^{*}$. Then, any $f\in {\mathcal{H}}^{*}$ can be decomposed as

$$\begin{array}{c}\hfill f=f_0+f_{1n}+\rho ,\end{array}$$

where $f_0\in {\mathcal{H}}_0$, $f_{1n}\in {\mathcal{H}}_{1n}$, and $\rho \in {\mathcal{H}}^{*}\ominus {\mathcal{H}}_{1n}$. We denote

$${\mathcal{K}}_{(t^{\prime },{\mathit{x}}_i(t^{\prime },·))}^{*}(t,\mathit{x}(t,·))={\mathcal{K}}^{*}((t^{\prime },{\mathit{x}}_i(t^{\prime },·)),(t,\mathit{x}(t,·)))$$

as the evaluation function of point $(t^{\prime },{\mathit{x}}_i(t^{\prime },·))$, and $f_1=f_{1n}+\rho$. Then, we can rewrite the PLS (7) as

$$\begin{array}{cc}\hfill & \frac{1}{n}\sum _{i=1}^n\sum _{k=1}^n{\left\{{\nu }_{ik}-u_{ik}-<f_1(t^{\prime },{\mathit{x}}_i(t^{\prime },·)),{\eta }_k\left(t^{\prime }\right)>\right\}}^2+{\tau }_0\sum _{v=1}^Q{w}_{2,v}{\theta }_v^{-1}{∥P_{v}f∥}_{{\mathcal{H}}^{*}}^2+{\lambda }_1\sum _{k=1}^2{w}_{1k}\left|d_k\right|+{\tau }_1\sum _{v=1}^Q{w}_{2,v}{\theta }_v\hfill \\ \hfill =& \frac{1}{n}\sum _{i=1}^n\sum _{k=1}^n{\left\{{\nu }_{ik}-u_{ik}-<<f_1,{\mathcal{K}}_{(t^{\prime },{\mathit{x}}_i(t^{\prime },·))}^{*}{>}_{{\mathcal{H}}^{*}},{\eta }_k\left(t^{\prime }\right)>\right\}}^2+{\tau }_0\sum _{v=1}^Q{w}_{2,v}{\theta }_v^{-1}{∥P_{v}f∥}_{{\mathcal{H}}^{*}}^2+{\lambda }_1\sum _{k=1}^2{w}_{1k}\left|d_k\right|+{\tau }_1\sum _{v=1}^Q{w}_{2,v}{\theta }_v\hfill \\ \hfill =& \frac{1}{n}\sum _{i=1}^n\sum _{k=1}^n{\left\{{\nu }_{ik}-u_{ik}-<f_1,{\int }_0^1{\mathcal{K}}_{(t^{\prime },{\mathit{x}}_i(t^{\prime },·))}^{*}{\eta }_k\left(t^{\prime }\right)d{t}^{\prime }{>}_{{\mathcal{H}}^{*}}\right\}}^2+{\tau }_0\sum _{v=1}^Q{w}_{2,v}{\theta }_v^{-1}{∥P_{v}f∥}_{{\mathcal{H}}^{*}}^2+{\lambda }_1\sum _{k=1}^2{w}_{1k}\left|d_k\right|+{\tau }_1\sum _{v=1}^Q{w}_{2,v}{\theta }_v\hfill \\ \hfill =& \frac{1}{n}\sum _{i=1}^n\sum _{k=1}^n{\left\{{\nu }_{ik}-u_{ik}-<f_{1n},{\int }_0^1{\mathcal{K}}_{(t^{\prime },{\mathit{x}}_i(t^{\prime },·))}^{*}{\eta }_k\left(t^{\prime }\right)d{t}^{\prime }{>}_{{\mathcal{H}}^{*}}\right\}}^2+\hfill \\ & {\tau }_0\sum _{v=1}^Q{w}_{2,v}{\theta }_v^{-1}{∥P_v{f}_{1n}∥}_{{\mathcal{H}}^{*}}^2+{\tau }_0\sum _{v=1}^Q{w}_{2,v}{\theta }_v^{-1}{∥P_v\rho ∥}_{{\mathcal{H}}^{*}}^2+{\lambda }_1\sum _{k=1}^2{w}_{1k}\left|d_k\right|+{\tau }_1\sum _{v=1}^Q{w}_{2,v}{\theta }_v,\hfill \end{array}$$

(9)

where $u_{ik}={\int }_0^1{f}_0(t^{\prime },{\mathit{x}}_i(t^{\prime },·)){\eta }_k\left(t^{\prime }\right)d{t}^{\prime }$. The first equality uses the reproducing property, and the third equality uses the fact that $\rho$ is orthogonal to ${\mathcal{H}}_{1n}$. Minimizing (9) must have $\rho =0$, and we obtain the following representer theorem:

Theorem 1

(Representer Theorem). The solution to PLS (9) is

$$\begin{array}{c}\hfill f(t,\mathit{x}(t,·))=\sum _{j=1}^2{d}_j{\phi }_j\left(t\right)+\sum _{v=1}^Q{w}_{2,v}^{-1}{\theta }_v\sum _{i=1}^n\sum _{k=1}^n{c}_{ik}{\xi }_{ik}(t,\mathit{x}(t,·)),\end{array}$$

(10)

where ${\phi }_1\left(t\right)=1$, ${\phi }_2\left(t\right)=k_1\left(t\right)$, and ${\xi }_{ik}(t,\mathit{x}(t,·))={\int }_0^1{\mathcal{K}}_v((t,\mathit{x}(t,·)),(t^{\prime },{\mathit{x}}_i(t^{\prime },·))){\eta }_k\left(t^{\prime }\right)d{t}^{\prime }$.

From this representer theorem, the PLS (9) reduces to

$$\begin{array}{cc}\hfill & \frac{1}{n}\sum _{i=1}^n\sum _{k=1}^n{({\nu }_{ik}-\sum _{j=1}^2{a}_{ikj}{d}_j-\sum _{v=1}^Q{w}_{2,v}^{-1}{\theta }_v\sum _{j=1}^n\sum _{l=1}^n{c}_{jl}{b}_{ikjl})}^2\hfill \\ \hfill +& {\lambda }_1\sum _{k=1}^2{w}_{1k}\left|d_k\right|+{\tau }_0\sum _{v=1}^Q{w}_{2,v}^{-1}{\theta }_v\sum _{i=1}^n\sum _{k=1}^n\sum _{j=1}^n\sum _{l=1}^n{c}_{ik}{b}_{ikjl}{c}_{jl}+{\tau }_1\sum _{v=1}^Q{w}_{2,v}{\theta }_v,\hfill \end{array}$$

(11)

where $a_{ikj}={\int }_0^1{\phi }_j\left(t\right){\eta }_k\left(t\right)dt$, $b_{ikjl}={\sum }_{v=1}^Q{w}_{2,v}^{-1}{\theta }_v{b}_{ikjl}^v$, $b_{ikjl}^v={\int }_0^1{\xi }_{jl}(t,{\mathit{x}}_i(t,·)){\eta }_k\left(t\right)dt$. We let $\mathbf{\Sigma }={\sum }_{v=1}^Q{w}_{2,v}^{-1}{\theta }_v{\mathsf{\Sigma }}_v$, the $(i+(k-1)n,j+(l-1\left)n\right)$th element of ${\mathsf{\Sigma }}_v$ is $b_{ikjl}^v$. We let ${\mathit{Y}}_k={({\nu }_{1k},\dots ,{\nu }_{nk})}^{\top }$, $\mathit{Y}={({\mathit{Y}}_1^{\top },\dots ,{\mathit{Y}}_n^{\top })}^{\top }$, $\mathit{c}={(c_{11},c_{21},\dots ,c_{nn})}^{\top }$, $\mathit{d}={(d_1,d_2)}^{\top }$, ${\mathit{w}}_2={(w_{2,1},\dots ,w_{2,Q})}^{\top }$, $\mathbf{\Sigma }$ be an $n^2\times n^2$ matrix with $b_{ikjl}$ as the $(i+(k-1)n,j+(l-1\left)n\right)$ element, and T be a $n^2\times 2$ matrix with $a_{ikj}$ as the $(i+(k-1)n,j)$ element. Then, the PLS (11) reduces to

$$\begin{array}{c}\hfill \frac{1}{n}{∥\mathit{Y}-\mathit{T}\mathit{d}-\mathbf{\Sigma }\mathit{c}∥}^2+{\lambda }_1\sum _{k=1}^2{w}_{1k}\left|d_k\right|+{\tau }_0{\mathit{c}}^{\top }\mathbf{\Sigma }\mathit{c}+{\tau }_1{w}_2^{\top }\mathit{\theta },\end{array}$$

(12)

subject to ${\theta }_v\ge 0,v=1,2,\dots ,Q$.

The backfitting algorithm in Wang et al. [12] is applied to solve the PLS (12) as follows (Algorithm 1):

Algorithm 1 Model Selection Algorithm

Set initial value $\mathit{d}={\mathit{d}}^0$, $\mathit{\theta }={\mathit{\theta }}^0$.

repeat

Update c by minimizing $\frac{1}{n}{∥\mathit{Y}-\mathit{T}\mathit{d}-\mathbf{\Sigma }\mathit{c}∥}^2+{\tau }_0{\mathit{c}}^{\top }\mathbf{\Sigma }\mathit{c}$

Calculate ${\mathit{Y}}^{*}=\mathit{Y}-\mathit{R}\mathit{\theta }$, where R is an $n\times Q$ matrix with the v-th column being $w_{2,v}^{-1}{\mathsf{\Sigma }}_v\mathit{c}$

Update d by minimizing $\frac{1}{n}{∥{\mathit{Y}}^{*}-\mathit{T}\mathit{d}∥}^2+{\lambda }_1{\sum }_{k=1}^2{w}_{1k}\left|d_k\right|$

select tuning parameter M by the k-fold cross-validation or BIC method

Update $\mathit{\theta }$ by minimizing $\frac{1}{n}{∥\mathit{Y}-\mathit{T}\mathit{d}-\mathit{R}\mathit{\theta }∥}^2+{\tau }_0{\mathit{c}}^{\top }\mathit{R}\mathit{\theta }$ subject to ${\theta }_v\ge 0$ for $1\le v\le Q$ and ${\mathit{w}}_2^{\top }\mathit{\theta }\le M$

until c, d and $\mathit{\theta }$ converge

return c, d and $\mathit{\theta }$

4. Statistical Properties

In this section, we assume that $\mathcal{X}$ and $\mathcal{Y}$ are complete measurable spaces. We let P be a probability measure on ${\mathcal{X}}^q\times L^2\left(\mathcal{T}\right)$ and $\mathcal{M}=\mathcal{T}\times {\mathcal{X}}^q$. Without the loss of generality, we let the terms ${\lambda }_1{\sum }_{k=1}^2{w}_{1k}\left|d_k\right|$ and ${\lambda }_2{\sum }_{v=1}^Q{w}_{2,v}{∥P_{v}f∥}_{\mathcal{H}}$ in (6) be combined into ${∥f∥}_{\mathcal{H}}$.

We define a loss function,

$$\begin{array}{c}\hfill L(f;\mathit{x},y)={\int }_0^1{(y\left(t\right)-f(t,\mathit{x}(t,·)))}^{2}dt,\end{array}$$

where $y\left(t\right)\in \mathcal{Y}$ and $\mathit{x}\in {\mathcal{X}}^q$. The corresponding L-risk function (Steinwart and Christmann [28]) is

$$\begin{array}{c}\hfill R_{L,P}\left(f\right)=E_P\left[L(f;\mathit{x},y)\right].\end{array}$$

We let $f^{*}=\underset{f\in \mathcal{H}}{argmin}{R}_{L,P}\left(f\right)$, $R_{L,P,\mathcal{H}}^{*}=R_{L,P}\left(f^{*}\right)$, and

$$\begin{array}{c}\hfill f_{P,\lambda }=\underset{f\in \mathcal{H}}{argmin}\{R_{L,P}\left(f\right)+\lambda {∥f∥}_{\mathcal{H}}\}.\end{array}$$

Obviously, $\widehat{f}=f_{D,\lambda }$. We state convergence properties in the following theorem and show its proofs in Appendix B.

Theorem 2.

Assume that $f:\mathcal{M}\to \mathbb{R}$ is measurable for any $f\in \mathcal{H}$, $\mathcal{M}$ is a complete measurable space, and ${\left|P\right|}_2={\int }_{{\mathcal{X}}^q\times L^2\left(\mathcal{T}\right)}{∥y\left(t\right)∥}_2^{2}dP(\mathit{x},y)<\infty$. When $\lambda \to 0$ and ${\lambda }^{6}n\to \infty$ as $n\to \infty$, we have

$$\begin{array}{cc}\hfill & |R_{L,P}\left(\widehat{f}\right)-R_{L,P,H}^{*}|=O_p\left(\lambda \right).\hfill \end{array}$$

Theorem 2 states that as $\lambda$ tends to 0 and ${\lambda }^{6}n$ tends to infinity as n tends to infinity, the function estimate $\widehat{f}$ is L-risk consistent (Steinwart and Christmann [28]).

5. Simulation

In this section, numerical experiments are studied to evaluate the performance of the proposed model selection approach. Functional covariate take ${\mathit{x}}_i(t,·)=(x_{i1}(t,·),x_{i2}(t,·))$, where $x_{ij}(t,·)=cos\left(2\pi \left(x_{ij}^{*}(t,·)\right)\right)$, and $x_{ij}^{*}(t,·)$ follows a Gaussian process with mean function $\mu \left(t\right)=t$. Kernel function for the GP takes the RBF kernel $k_g(s_1,s_2)=exp(-{(s_1-s_2)}^2/2)$ for $j=1$ and the rational quadratic kernel $k_l(s_1,s_2)=1/(1+{(s_1-s_2)}^2)$ for $j=2$. Three functions for $f(t,\mathit{x}(t,·\left)\right)$ are presented as follows: for $t\in$ [0, 1],

$$\begin{array}{cc}\hfill M_1:f(t,\mathit{x}(t,·))=& 1+5cos{\left(2\pi t\right)}^3,\hfill \\ \hfill M_2:f(t,\mathit{x}(t,·))=& 1+0.5t+10{\int }_0^1{x}_1^3(t,s)ds+5{\int }_0^1{x}_2^3(t,s)ds+10{\int }_0^1{x}_1^3(t,s)ds{\int }_0^1{x}_2^3(t,s)ds,\hfill \\ \hfill M_3:f(t,\mathit{x}(t,·))=& 1+5cos\left(2\pi t\right)+10{\int }_0^1{x}_1(t,s)x_2(t,s)ds.\hfill \end{array}$$

We see that $M_1$ has the main effect of t, $M_2$ consists of three main effects and the nonparametric nonparametric interaction of $x_1$ and $x_2$, $M_3$ consists of the main effect of t, and the nonparametric nonparametric interaction of $x_1$ and $x_2$. Random error ${ϵ}_i\left(t\right)$ follows $N(0,{0.22}^2)$ and $N(0,{0.5}^2)$. All simulations are repeated 200 times.

We generate n samples $\{y_i\left(t\right),x_i(t,·):i=1,\dots ,n\}$ as training data, and $n_t=50$ samples $\{{\tilde{y}}_i\left(t\right),{\tilde{x}}_i(t,·):i=1,\dots ,n\}$ as test data. For comparison, we evaluate the performance using the following root mean square error (RMSE) on the test data:

$$\begin{array}{c}\hfill \mathrm{RMSE}=\sqrt{\frac{1}{n_t}\sum _{i=1}^{n_t}{∥f(t,{\tilde{x}}_i(t,·))-\widehat{f}(t,{\tilde{x}}_i(t,·))∥}_2^2},\end{array}$$

where ${∥·∥}_2$ is the norm of $L^2\left(\mathcal{T}\right)$.

The proposed model selection method is used to train the model and predict the test data, denoted by $L_1$. Not considering model selection, we use the $L_2$ penalty method to estimate the NCRM model denoted by $L_2$, which minimizes the following objective function,

$$\begin{array}{c}\hfill \frac{1}{n}\sum _{i=1}^n\left\{{\int }_0^1{(y_i\left(t\right)-f(t,{\mathit{x}}_i(t,·)))}^{2}dt\right\}+\lambda \sum _{v=1}^Q{∥P_{v}f∥}_{\mathcal{H}}^2,\end{array}$$

(13)

where $\lambda$ is the tuning parameter. After model selection, the selected model is estimated with the $L_2$ penalty method, which is denoted by $L_1+L_2$.

Table 1. Average values and standard deviations of RMSEs. (Methods corresponding to bold numbers perform best).

n	$\mathit{\sigma }$	Model	${\mathit{L}}_1$	${\mathit{L}}_2$	${\mathit{L}}_1+{\mathit{L}}_2$
20	0.2	M1	0.2906 (0.1386)	0.3010 (0.0287)	$\mathbf{0.0981}\left(\mathbf{0.0420}\right)$
		M2	$\mathbf{0.8876}\left(\mathbf{0.1147}\right)$	0.9577 (0.0782)	0.9493 (0.0707)
		M3	1.2065 (0.8556)	0.9423 (0.1713)	$\mathbf{0.7108}\left(\mathbf{0.6317}\right)$
	0.5	M1	0.3027 (0.1323)	0.4238 (0.0654)	$\mathbf{0.1205}\left(\mathbf{0.0591}\right)$
		M2	$\mathbf{0.9486}\left(\mathbf{0.1166}\right)$	1.0039 (0.0760)	0.9996 (0.0794)
		M3	1.3906 (0.9134)	1.0044 (0.1952)	$\mathbf{0.8008}\left(\mathbf{0.5486}\right)$
40	0.2	M1	0.1655 (0.0127)	0.2390 (0.0416)	$\mathbf{0.0792}\left(\mathbf{0.0113}\right)$
		M2	$\mathbf{0.7722}\left(\mathbf{0.0517}\right)$	0.8088 (0.0877)	0.7928 (0.0630)
		M3	0.6664 (0.2255)	0.5773 (0.0456)	$\mathbf{0.3968}\left(\mathbf{0.0423}\right)$
	0.5	M1	0.1913 (0.0944)	0.3257 (0.0860)	$\mathbf{0.0913}\left(\mathbf{0.0179}\right)$
		M2	$\mathbf{0.8605}\left(\mathbf{0.0678}\right)$	0.8918 (0.0856)	0.8819 (0.0746)
		M3	0.7824 (0.2060)	0.6869 (0.0420)	$\mathbf{0.4952}\left(\mathbf{0.0764}\right)$
80	0.2	M1	0.1358 (0.0860)	0.1416 (0.0238)	$\mathbf{0.0737}\left(\mathbf{0.0076}\right)$
		M2	$\mathbf{0.6040}\left(\mathbf{0.0620}\right)$	0.6599 (0.0894)	0.6595 (0.0708)
		M3	0.3227 (0.0218)	0.3793 (0.0237)	$\mathbf{0.2635}\left(\mathbf{0.0248}\right)$
	0.5	M1	0.1635 (0.1412)	0.2419 (0.0704)	$\mathbf{0.0844}\left(\mathbf{0.0206}\right)$
		M2	$\mathbf{0.7338}\left(\mathbf{0.0552}\right)$	0.7589 (0.0696)	0.7578 (0.0605)
		M3	0.4080 (0.1488)	0.5511 (0.0475)	$\mathbf{0.3566}\left(\mathbf{0.0470}\right)$

shows the average RMSE and standard deviation in parentheses for these three estimation methods, $L_1$, $L_2$ and $L_1+L_2$. We see that under models M1 and M3, $L_1+L_2$ have the smallest RMSEs among these three estimation methods. Under model M2, $L_1$ has better performance than $L_2$ and comparable results with those of $L_1+L_2$. In addition, for the three different methods, prediction performance improves as the $\sigma$ decreases and the training sample size increases.

To evaluate the performance of model selection by the $L_1$ penalty method, we take three measurement indices in Wang et al. [12], specificity (SPE), sensitivity (SEN) and $F_1$ scores,

$$\begin{array}{c}\hfill \mathrm{SPE}=\frac{\mathrm{TN}}{\mathrm{TN}+\mathrm{FP}},\mathrm{SEN}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}},{\mathrm{F}}_1=\frac{2\mathrm{TP}}{2\mathrm{TP}+\mathrm{FN}+\mathrm{FP}},\end{array}$$

where TP, TN, FP and FN are the numbers of true positives, true negatives, false positives and false negatives, respectively. The non-zero components of the decomposition of the regression function are considered as positive samples. For ${\theta }_v>0$, its estimated value is larger than 0, which is considered a true positive.

Table 2. Average values and standard deviations of SPE, SEN, F1 scores under models M1, M2, M3.

n	$\mathit{\sigma }$	Model	SPE	SEN	F1
20	0.2	M1	0.9956 (0.0291)	0.9800 (0.1404)	0.9783 (0.1421)
		M2	0.9700 (0.0601)	0.7017 (0.1472)	0.7883 (0.1166)
		M3	0.9906 (0.0366)	0.9850 (0.1219)	0.9592 (0.1527)
	0.5	M1	0.9961 (0.0233)	0.9850 (0.1219)	0.9800 (0.1279)
		M2	0.9786 (0.0511)	0.6917 (0.1529)	0.7881 (0.1201)
		M3	0.9889 (0.0386)	0.9850 (0.1219)	0.9553 (0.1543)
40	0.2	M1	1.0000 (0.0000)	1.0000 (0.0000)	1.0000 (0.0000)
		M2	0.9614 (0.0667)	0.9817 (0.0762)	0.9512 (0.0872)
		M3	0.9833 (0.0428)	1.0000 (0.0000)	0.9517 (0.1212)
	0.5	M1	0.9989 (0.0111)	0.9950 (0.0707)	0.9933 (0.0744)
		M2	0.9407 (0.0720)	0.9600 (0.1086)	0.9177 (0.1060)
		M3	0.9844 (0.0417)	1.0000 (0.0000)	0.9550 (0.1178)
80	0.2	M1	0.9983 (0.0175)	1.0000 (0.0000)	0.9958 (0.0424)
		M2	0.9743 (0.0737)	1.0000 (0.0000)	0.9635 (0.0701)
		M3	1.0000 (0.0000)	1.0000 (0.0000)	1.0000 (0.0000)
	0.5	M1	0.9956 (0.0291)	0.9850 (0.1219)	0.9817 (0.1259)
		M2	0.9546 (0.0853)	1.0000 (0.0000)	0.9317 (0.1031)
		M3	0.9983 (0.0135)	1.0000 (0.0000)	0.9950 (0.0406)

shows the sensitivities, specificities, and F1 scores. Overall, the $L_1$ penalty method performs well in different simulation settings. In addition, model selection becomes better with decreasing $\sigma$ and increasing training sample size.

6. Application

The proposed model selection approach is applied to analyze stroke rehabilitation data with 70 stroke survivors ([7]).

The data consist of 34 acute patients with an incidence of stroke less than a month ago and 36 chronic patients with an incidence of stroke more than six months ago. To improve upper limb functions for stroke patients, a convenient home-based rehabilitation system via action video games with 3D-position movement behaviors has been developed [7,8]. The patients played the movement game at scheduled times. For each visit time t, the impairment level of subject i was assessed using a measure called CAHAI (Chedoke Arm and Hand Activity Inventory) score, denoted as $y_i\left(t\right)$, and movements such as forward circular movement and sawing movement were recorded. In this paper, three movements, forward circular movement of the parental limb from the x-axis ($x_{i1}=LA05.lx$), sawing movement of the parental limb from the y-axis ($x_{i2}=LA09.ly$), and the movement of the non-parental limb from the direction of the x-axis ($x_{i3}=LA28.rqx$) are taken as functional covariates. For the purpose of illustrating the proposed method, we use the data from acute patients. During the three-month study period, each acute patient received up to seven assessments, which resulted in 173 observations. CAHAI scores were normalized before analysis.

In this paper, we focus on the interaction effect upon the order of two, and take the following decomposition:

$$\begin{array}{cc}\hfill {\mathcal{K}}_0((t,\mathit{u}),(t^{\prime },{\mathit{u}}^{\prime }))& =1+k_1\left(t\right)k_1\left(t^{\prime }\right),\hfill \\ \hfill {\mathcal{K}}_1((t,\mathit{u}),(t^{\prime },{\mathit{u}}^{\prime }))& ={\mathcal{K}}_2^{\left(1\right)}(t,t^{\prime }),\hfill \\ \hfill {\mathcal{K}}_{1+j}((t,\mathit{u}),(t^{\prime },{\mathit{u}}^{\prime }))& ={\mathcal{K}}_{2,j}^{\left(2\right)}(u_j,u_j^{\prime }),\hfill \\ \hfill {\mathcal{K}}_{4+j}((t,\mathit{u}),(t^{\prime },{\mathit{u}}^{\prime }))& =k_1\left(t\right)k_1\left(t^{\prime }\right){\mathcal{K}}_{2,j}^{\left(2\right)}(u_j,u_j^{\prime })+{\mathcal{K}}_2^{\left(1\right)}(t,t^{\prime }){\mathcal{K}}_{2,j}^{\left(2\right)}(u_j,u_j^{\prime }),\hfill \\ \hfill {\mathcal{K}}_{5+j+l}((t,\mathit{u}),(t^{\prime },{\mathit{u}}^{\prime }))& ={\mathcal{K}}_{2,j}^{\left(2\right)}(u_j,u_j^{\prime }){\mathcal{K}}_{2,l}^{\left(2\right)}(u_l,u_l^{\prime }),\hfill \end{array}$$

for $j,l=1,2,3$ and $j<l$. Readers can also choose various kinds of SS ANOVA decomposition by merging kernel functions according to their own needs. From Section 3, we have

$${\mathcal{K}}^{*}((t,\mathit{u}),(t^{\prime },{\mathit{u}}^{\prime }))=\sum _{v=1}^{10}{w}_{2,v}^{-1}{\theta }_v{\mathcal{K}}_v((t,\mathit{u}),(t^{\prime },{\mathit{u}}^{\prime })),$$

where coefficient ${\theta }_v$ for kernel function ${\mathcal{K}}_v$ provides levels of contribution of ${\mathcal{K}}_v$ to the overall model.

The penalty method with $L_1$ regularization for model selection is applied to stroke rehabilitation data. Parameters $\left\{{\theta }_v\right\}$ are computed, and values larger than 0 are ${\theta }_2=4.157$, ${\theta }_3=0.819$, ${\theta }_4=0.636$, ${\theta }_7=0.592$ and ${\theta }_{10}=0.741$. This shows that the main effects of $x_{i1}(t,·)$, $x_{i2}(t,·)$ and $x_{i3}(t,·)$, the linear nonparametric interaction of t and $x_3(t,·)$ and the nonparametric-nonparametric interaction of $x_2(t,·)$ and $x_3(t,·)$ have nonzero contributions to the CAHAI score. Thus, the three movements, forward circular movements of the parental limb, awing movements of the parental limb and of the non-parental limb, may be helpful to the recovery of stroke patients. In addition, the interaction of awing movements of the parental limb and the non-parental limb, may contribute to the level of daily life dependence or upper limb function impairment. Figure 1 plots the estimates of nonparametric regression functions for four stroke patients. We can see that the regression function in the NCRM model has the same trend as the scores of CAHAI, and on the whole, they all show an increasing trend along with fluctuating trends, which shows that movements may improve upper limb function for stroke patients.

Prediction performance of the proposed method is evaluated using a tenfold cross-validation,

$$\begin{array}{c}\hfill \mathrm{RPE}=\frac{1}{10}\sum _{i=1}^{10}\sqrt{\frac{1}{n_j}\sum _{i\in jthfold}^{ }{∥Y_i\left(t\right)-{\widehat{Y}}_i^{(-j)}\left(t\right)∥}_2^2},\end{array}$$

where ${\widehat{Y}}_i^{(-j)}\left(t\right)$ is the predicted value of $Y_i\left(t\right)$ based on the fitted selected model with the $L_1+L_2$ penalty and the data excluding the jth fold. The RPE for Stoke data is 1.0690, which is smaller than 1.1700 from the method of Zhai et al. [9].

7. Conclusions

For functional data with functional covariate inputs, this paper applies the Gaussian kernel function to construct the tensor product RKHS to model the regression function. This leads to a nonparametric concurrent regression model. The $L_1$ penalty method is used to detect components of the SS ANOVA decomposition of the regression function, which has nonzero contribution to model fit. The backfitting algorithm is developed to estimate the model selection. The proposed method is applied to stroke rehabilitation data, and the results show that besides the main effects, there are interaction effects of the movements on the CAHAI score. This indicates that movements may help improve the level of daily life dependence or impairment of upper limb function of a stroke patient.