# A Method of L1-Norm Principal Component Analysis for Functional Data

^{1}

^{2}

^{3}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Problem Description

#### 2.1. L2-Norm Functional Principal Component Analysis (L2-Norm FPCA)

#### 2.2. L1-Norm Functional Principal Component Analysis (L1-Norm FPCA)

## 3. The Solving Algorithm of L1-Norm Functional Principal Component Weight Functions (L1-FPCA Algorithm)

#### 3.1. Only One Principal Component

**Step 1:**Arbitrarily choose the initial projection direction ${\beta}^{0}(t)$, get ${b}^{0}$ by ${\beta}^{0}(t)={({b}^{0})}^{T}\varphi (t)$, normalize ${b}^{0}$:${b}^{0}=\frac{{b}^{0}}{{\Vert {b}^{0}\Vert}_{2}}$, and set the iteration number $k$ to be 0.

**Step 2:**For all $i\in (1,2,\cdots ,n)$, if $\int {\beta}^{k}(t){x}_{i}(t)dt<0$, i.e., ${({b}^{k})}^{T}{c}_{i}<0$, let ${p}_{i}^{k}=-1$; otherwise ${p}_{i}^{k}=1$.

**Step 3:**Let ${b}^{k}={\displaystyle \sum _{i=1}^{n}{p}_{i}^{k-1}{c}_{i}}$, normalize ${b}^{k}$: ${b}^{k}=\frac{{b}^{k}}{{\Vert {b}^{k}\Vert}_{2}}$, and get the corresponding ${\beta}^{k}(t)$ by ${\beta}^{k}(t)={({b}^{k})}^{T}\varphi (t)$.

**Step 4:**If ${\beta}^{k}(t)\ne {\beta}^{k-1}(t)$, return to step 2. If there is $i$ such that $\int {\beta}^{k}(t){x}_{i}(t)dt=0$, i.e., ${({b}^{k})}^{T}{c}_{i}=0$, then let ${b}^{k}=\frac{({b}^{K}+\Delta b)}{{\Vert {b}^{K}+\Delta b\Vert}_{2}}$ and get the corresponding ${\beta}^{k}(t)$, then return to step 2, where $\Delta b$ is a small non-zero vector. Otherwise, let ${\beta}^{\ast}(t)={\beta}^{k}(t)$, ${b}^{*}={b}^{k}$ and ${\beta}^{\ast}(t)={({b}^{\ast})}^{T}\varphi (t)$, stop.

**Theorem**

**1.**

**Proof.**

#### 3.2. Multiple Principal Components

**Step 1:**Let ${\beta}_{0}(t)=0$, i.e.,${b}_{0}=0$,${\{{c}_{i}^{0}={c}_{i}\}}_{i=1}^{n}$.

**Step 2:**For all $i\in (1,2,\cdots ,n)$, let ${c}_{i}^{1}={c}_{i}^{0}-{b}_{0}({b}_{0}^{T}{c}_{i}^{0})$ and apply the L1-FPCA algorithm to ${c}^{1}=({c}_{1}^{1},{c}_{2}^{1},\cdots ,{c}_{n}^{1})$ to obtain the projection vector ${b}_{1}$ and the corresponding ${\beta}_{1}(t)$.

**Step 3:**For all $i\in (1,2,\cdots ,n)$, let ${c}_{i}^{j}={c}_{i}^{j-1}-{b}_{j-1}({b}_{j-1}^{T}{c}_{i}^{j-1})$ and apply the L1-FPCA algorithm to ${c}^{j}=({c}_{1}^{j},{c}_{2}^{j},\cdots ,{c}_{n}^{j})$ to obtain the projection vector ${b}_{j}$ and the corresponding ${\beta}_{j}(t)$.

**Step 4:**Repeat Step 3 until m projection vectors ${b}_{1},{b}_{2},\cdots ,{b}_{m}$ and corresponding ${\beta}_{1}(t),{\beta}_{2}(t),\cdots ,{\beta}_{m}(t)$ are obtained.

## 4. Numerical Examples

#### 4.1. Simulation

**Model 1**(no contamination): ${x}_{i}(t)=m(t)+{\epsilon}_{i}(t),i=1,2,\cdots ,n$, where error term ${\epsilon}_{i}(t)$ is a stochastic Gaussian process with zero mean and covariance function $\mathrm{cov}(s,t)=(1/2){(1/2)}^{0.9\left|t-s\right|}$ and $m(t)=4t$,$t\in [0,1]$.

**Model 2**(asymmetric contamination): ${y}_{i}(t)={x}_{i}(t)+{c}_{i}M,i=1,2,\cdots ,n$, where ${c}_{i}$ is the sample of the 0–1 distribution with the parameter $q$, and $M$ is the contamination constant.

**Model 3**(symmetric contamination): ${y}_{i}(t)={x}_{i}(t)+{c}_{i}{\sigma}_{i}M,i=1,2,\cdots ,n$, where ${c}_{i}$ and $M$ are defined as in Model 2 and ${\sigma}_{i}$ is a sequence of random variables with values of 1 and −1 with a probability of 1/2 that is independent of ${c}_{i}$.

**Model 4**(partially contaminated): ${y}_{i}(t)=\{\begin{array}{l}{x}_{i}(t)+{c}_{i}{\sigma}_{i}M,t\ge {T}_{i}\\ {x}_{i}(t),\hspace{1em}\hspace{1em}\hspace{1em}t{T}_{i}\end{array},i=1,2,\cdots ,n$, where ${T}_{i}$ is a random number generated from a uniform distribution on [0,1].

**Model 5**(peak contamination): ${y}_{i}(t)=\{\begin{array}{l}{x}_{i}(t)+{c}_{i}{\sigma}_{i}M,T\le t\le {T}_{i}+l\\ {x}_{i}(t),\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}t\notin [{T}_{i},{T}_{i}+l]\end{array},i=1,2,\cdots ,n$, where $l=1/15$ and ${T}_{i}$ is a random number generated from a uniform distribution on $[0,1-l]$.

#### 4.2. Canadian Weather Data

## 5. Concluding Remarks

## Author Contributions

## Funding

## Acknowledgments

## Conflicts of Interest

## References

- Kowal, D.R. Integer-valued functional data analysis for measles forecasting. Biometric
**2019**, in press. [Google Scholar] [CrossRef] [PubMed] - Wagner-Muns, I.M.; Guardiola, I.G.; Sama-ranayke, V.A.; Kayani, W.I. A functional data analysis approach to traffic volume forecasting. IEEE Trans. Intell. Transp. Syst.
**2017**, 19, 878–888. [Google Scholar] [CrossRef] - Ramsay, J.O.; Silverman, B.W. Applied functional data analysis. J. Educ. Behav. Stat.
**2008**, 24, 5822–5828. [Google Scholar] - Yao, F.; Müller, H.G.; Wang, J.L. Functional data analysis for sparse longitudinal data. J. Am. Stat. Assoc.
**2005**, 100, 577–590. [Google Scholar] [CrossRef] - Auton, T. Applied functional data analysis: Methods and case studies. J. R. Stat. Soc.
**2010**, 167, 378–379. [Google Scholar] [CrossRef] - Zambom, A.Z.; Collazos, J.A.; Dias, R. Functional data clustering via hypothesis testing k-means. Comput. Stat.
**2019**, 34, 527–549. [Google Scholar] [CrossRef] - Ferraty, F.; Vieu, P. Nonparametric Functional Data Analysis: Theory and Practice; Springer Science & Business Media: Berlin, Germany, 2006. [Google Scholar]
- Horváth, L.; Kokoszka, P. Inference for Functional Data with Applications; Springer Science & Business Media: Berlin, Germany, 2012; Volume 200. [Google Scholar]
- Tarpey, T.; Kinateder, K.K. Clustering functional data. J. Classif.
**2003**, 20, 93–114. [Google Scholar] [CrossRef] - Ramsay, J.O.; Silverman, B.W. Applied Functional Data Analysis: Methods and Case Studies; Springer: Berlin/Heidelberg, Germany, 2007. [Google Scholar]
- Estévez-Pérez, G.; Vilar, J.A. Functional anova starting from discrete data: An application to air quality data. Environ. Ecol. Stat.
**2013**, 20, 495–517. [Google Scholar] [CrossRef] - Ignaccolo, R.; Ghigo, S.; Giovenali, E. Analysis of air quality monitoring networks by functional clustering. Environmetrics
**2010**, 19, 672–686. [Google Scholar] [CrossRef] - Ferraty, F.; Vieu, P. Nonparametric models for functional data, with application in regression, time series prediction and curve discrimination. Nonparametr. Stat.
**2004**, 16, 111–125. [Google Scholar] [CrossRef] - Febrero, M.; Galeano, P.; González-Manteiga, W. Outlier detection in functional data by depth measures, with application to identify abnormal nox levels. Environmetrics
**2010**, 19, 331–345. [Google Scholar] [CrossRef] - Ratcliffe, S.J.; Heller, G.Z.; Leader, L.R. Functional data analysis with application to periodically stimulated foetal heart rate data ii functional logistic regression. Stat. Med.
**2002**, 21, 1103–1114. [Google Scholar] [CrossRef] [PubMed] - Giraldo, R.; Delicado, P.; Mateu, J. Continuous time-varying kriging for spatial prediction of functional data: An environmental application. J. Agric. Biol. Environ. Stat.
**2010**, 15, 66–82. [Google Scholar] [CrossRef] - Ferraty, F.; Rabhi, A.; Vieu, P. Conditional quantiles for dependent functional data with application to the climatic “el niño” phenomenon. Sankhyā Indian J. Stat.
**2005**, 67, 378–398. [Google Scholar] - Baladandayuthapani, V.; Mallick, B.K.; Young Hong, M.; Lupton, J.R.; Turner, N.D.; Carroll, R.J. Bayesian hierarchical spatially correlated functional data analysis with application to colon carcinogenesis. Biometrics
**2008**, 64, 64–73. [Google Scholar] [CrossRef] [Green Version] - Ramsay, J.O.; Dalzell, C.J. Some tools for functional data analysis. J. R. Stat. Soc.
**1991**, 53, 539–572. [Google Scholar] [CrossRef] - Ramsay, J.O. Functional data analysis. Encycl. Stat. Sci.
**2004**, 4. [Google Scholar] [CrossRef] - Dauxois, J.; Pousse, A.; Romain, Y. Asymptotic theory for the principal component analysis of a vector random function: Some applications to statistical inference. J. Multivar. Anal.
**1982**, 12, 136–154. [Google Scholar] [CrossRef] [Green Version] - Rice, J.A.; Silverman, B.W. Estimating the mean and covariance structure nonparametrically when the data are curves. J. R. Stat. Soc.
**1991**, 53, 233–243. [Google Scholar] [CrossRef] - Levy, A.; Rubinstein, J. Some properties of smoothed principal component analysis for functional data. J. Opt. Soc. Am.
**1999**, 16, 28–35. [Google Scholar] [CrossRef] - Silverman, B.W. Smoothed functional principal components analysis by choice of norm. Ann. Stat.
**1996**, 24, 1–24. [Google Scholar] [CrossRef] - James, G.M.; Hastie, T.J.; Sugar, C.A. Principal component models for sparse functional data. Biometrika
**2000**, 87, 587–602. [Google Scholar] [CrossRef] [Green Version] - Boente, G.; Fraiman, R. Kernel-based functional principal components. Stat. Probab. Lett.
**2000**, 48, 335–345. [Google Scholar] [CrossRef] - Hall, P.; Hosseini-Nasab, M. On properties of functional principal components analysis. J. R. Stat. Soc. Ser. B Stat. Methodol.
**2006**, 68, 109–126. [Google Scholar] [CrossRef] - Benko, M.; Härdle, W.; Kneip, A. Common functional principal components. Ann. Stat.
**2009**, 37, 1–34. [Google Scholar] [CrossRef] - Hörmann, S.; Kidziński, Ł.; Hallin, M. Dynamic functional principal components. J. R. Stat. Soc. Ser. B Stat. Methodol.
**2015**, 77, 319–348. [Google Scholar] [CrossRef] - Kwak, N. Principal component analysis based on l1-norm maximization. IEEE Trans. Pattern Anal. Mach. Intell.
**2008**, 30, 1672–1680. [Google Scholar] [CrossRef] - Nie, F.; Huang, H.; Ding, C.; Luo, D.; Wang, H. Robust principal component analysis with non-greedy ℓ1-norm maximization. In Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence, Barcelona, Spain, 16–22 July 2011. [Google Scholar]
- Markopoulos, P.P.; Karystinos, G.N.; Pados, D.A. Optimal algorithms for L1-subspace signal processing. IEEE Trans. Signal Process.
**2014**, 62, 5046–5058. [Google Scholar] [CrossRef] - Markopoulos, P.P.; Kundu, S.; Chamadia, S.; Pados, D.A. Efficient L1-norm principal-component analysis via bit flipping. IEEE Trans. Signal Process.
**2017**, 65, 4252–4264. [Google Scholar] [CrossRef] - Martin-Clemente, R.; Zarzoso, V. On the link between L1-PCA and ICA. IEEE Trans. Pattern Anal. Mach. Intell.
**2016**, 39, 515–528. [Google Scholar] [CrossRef] - Park, Y.W.; Klabjan, D. Iteratively reweighted least squares algorithms for L1-norm principal component analysis. In Proceedings of the 2016 IEEE 16th International Conference on Data Mining (ICDM), Barcelona, Spain, 12–15 December; IEEE: Piscataway, NJ, USA, 2016. [Google Scholar]
- Markopoulos, P.P.; Dhanaraj, M.; Savakis, A. Adaptive L1-norm principal-component analysis with online outlier rejection. IEEE J. Sel. Top. Signal Process.
**2018**, 12, 1131–1143. [Google Scholar] [CrossRef] - Tsagkarakis, N.; Markopoulos, P.P.; Pados, D.A. On the L1-norm approximation of a matrix by another of lower rank. In Proceedings of the 2016 15th IEEE International Conference on Machine Learning and Applications (ICMLA), Anaheim, CA, USA, 18–20 December 2016; IEEE: Piscataway, NJ, USA, 2016. [Google Scholar]
- Fraiman, R.; Muniz, G. Trimmed means for functional data. Test
**2001**, 10, 419–440. [Google Scholar] [CrossRef] - Yu, F.; Liu, L.; Jin, L.; Yu, N.; Shang, H. A method for detecting outliers in functional data. In Proceedings of the IECON 2017-43rd Annual Conference of the IEEE Industrial Electronics Society, Beijing, China, 29 October–1 November 2017; IEEE: Piscataway, NJ, USA, 2017; pp. 7405–7410. [Google Scholar]

**Figure 1.**Curves generated from Model 1 (without contamination), Model 2 (asymmetric contamination), Model 3 (symmetric contamination), Model 4 (partial contamination) and Model 5 (peak contamination) with n = 200, p = 100, q = 5% and M = 10.

**Figure 2.**The boxplots of the change of the first principal component coefficient for asymmetric contamination (q = 5% and q = 10%; M = 5 and M = 10).

**Figure 3.**The boxplots of the change of the first principal component coefficient for symmetric contamination (q = 5% and q = 10%; M = 5 and M = 10).

**Figure 4.**The boxplots of the change of the first principal component coefficient for partial contamination (q = 5% and q = 10%; M = 5 and M = 10).

**Figure 5.**The boxplots of the change of the first four principal component coefficient for peak contamination (q = 5% and q = 10%; M = 5 and M = 10).

**Figure 6.**Scatter plots of the coefficients of the reconstruction error curves of L1-norm and L2-norm under asymmetric contamination.

**Figure 7.**Scatter plots of the coefficients of the reconstruction error curves of L1-norm and L2-norm under symmetric contamination.

**Figure 8.**Scatter plots of the coefficients of the reconstruction error curves of L1-norm and L2-norm under partial contamination.

**Figure 9.**Scatter plots of the coefficients of the reconstruction error curves of L1-norm and L2-norm under peak contamination.

**Figure 10.**Daily mean temperature curves of 35 observatories in Canada ((

**a**) whole data, (

**b**) normal data).

**Figure 11.**The first principal component weight function for normal data and whole data. (

**a**) L2-norm, (

**b**) L1-norm.

**Figure 12.**The second principal component weight function for normal data and whole data. (

**a**) L2-norm, (

**b**) L1-norm

**Table 1.**The sum of the absolute values of the first principal component weight function coefficient changes for no contamination and asymmetric contamination (5% and 10%).

M = 10 | M = 5 | |||
---|---|---|---|---|

q | L1-norm FPC | L2-norm FPC | L1-norm FPC | L2-norm FPC |

1st FPC | 1st FPC | 1st FPC | 1st FPC | |

5% | 0.17 | 1.13 | 0.13 | 0.91 |

10% | 0.22 | 1.24 | 0.18 | 1.16 |

**Table 2.**The sum of the absolute values of the first principal component weight function coefficient changes for no contamination and symmetric contamination (5% and 10%).

M = 10 | M = 5 | |||
---|---|---|---|---|

q | L1-norm FPC | L2-norm FPC | L1-norm FPC | L2-norm FPC |

1st FPC | 1st FPC | 1st FPC | 1st FPC | |

5% | 0.2 | 1.13 | 0.1 | 0.96 |

10% | 0.25 | 1.17 | 0.23 | 1.04 |

**Table 3.**The sum of the absolute values of the first principal component weight function coefficient changes for no contamination and partial contamination (5% and 10%).

M = 10 | M = 5 | |||
---|---|---|---|---|

q | L1-norm FPC | L2-norm FPC | L1-norm FPC | L2-norm FPC |

1st FPC | 1st FPC | 1st FPC | 1st FPC | |

5% | 1.17 | 13.47 | 0.81 | 10.67 |

10% | 1.70 | 14.77 | 1.24 | 12.29 |

**Table 4.**The sum of the absolute values of the first four principal component weight functions. Coefficient changes for no contamination and peak contamination (5% and 10%).

M = 10 | M = 5 | |||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|

q | L1-norm FPC | L2-norm FPC | L1-norm FPC | L2-norm FPC | ||||||||||||

1st | 2nd | 3rd | 4th | 1st | 2nd | 3rd | 4th | 1st | 2nd | 3rd | 4th | 1st | 2nd | 3rd | 4th | |

5% | 0.5 | 1.5 | 3.9 | 9.8 | 3.9 | 66.3 | 39.2 | 45.2 | 0.2 | 0.7 | 1.3 | 10.1 | 3.9 | 66.8 | 39.9 | 45.7 |

10% | 0.8 | 1.8 | 4.1 | 8.4 | 5.4 | 24.2 | 48.4 | 54.3 | 0.4 | 1.2 | 2.2 | 3.4 | 2.3 | 10.9 | 22.3 | 42.2 |

**Table 5.**The table of the one-sided paired T-test of the coefficients of the reconstruction error curves of L1-norm and L2-norm under asymmetric contamination (Alternative hypothesis: The true difference of the reconstruction error curve coefficients of L1-norm and L2-norm in means was greater than 0.).

q | M = 5 | M = 10 | ||||
---|---|---|---|---|---|---|

t | df | p-Value | t | df | p-Value | |

5% | −2.8447 | 199 | 0.9975 | −2.1651 | 199 | 0.9842 |

10% | −2.2484 | 199 | 0.9872 | −2.5843 | 199 | 0.9948 |

**Table 6.**The table of the one-sided paired T-test of the coefficients of the reconstruction error curves of L1-norm and L2-norm under symmetric contamination (Alternative hypothesis: The true difference of the reconstruction error curve coefficients of L1-norm and L2-norm in means was greater than 0.).

q | M = 5 | M = 10 | ||||
---|---|---|---|---|---|---|

t | df | p-Value | t | df | p-Value | |

5% | −3.8761 | 199 | 0.9999 | −3.34 | 199 | 0.9995 |

10% | −4.7628 | 199 | 1 | −3.5293 | 199 | 0.9997 |

**Table 7.**The table of the one-sided paired T-test of the coefficients of the reconstruction error curve of L1-norm and L2-norm under partial contamination (Alternative hypothesis: The true difference of the reconstruction error curve coefficients of L1-norm and L2-norm in means was greater than 0.).

q | M = 5 | M = 10 | ||||
---|---|---|---|---|---|---|

t | df | p-Value | t | df | p-Value | |

5% | −5.2373 | 199 | 1 | −4.9371 | 199 | 1 |

10% | −7.7896 | 199 | 1 | −5.033 | 199 | 1 |

**Table 8.**The table of the one-sided paired T-test of the coefficients of the reconstruction error curves of L1-norm and L2-norm under peak contamination (Alternative hypothesis: The true difference of the reconstruction error curve coefficients of L1-norm and L2-norm in means was greater than 0.).

q | M = 5 | M = 10 | ||||
---|---|---|---|---|---|---|

t | df | p-Value | t | df | p-Value | |

5% | −6.6502 | 199 | 1 | −6.6212 | 199 | 1 |

10% | −7.6313 | 199 | 1 | −6.8564 | 199 | 1 |

**Table 9.**The sum of absolute change of the coefficients of the first two principal component weighting functions.

The Sum of Absolute Change of the Coefficients | The 1st Function Principal Component Weighting Function | The 2nd Function Principal Component Weighting Function |
---|---|---|

L2-norm | 0.18 | 0.76 |

L1-norm | 0.16 | 0.33 |

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Yu, F.; Liu, L.; Yu, N.; Ji, L.; Qiu, D.
A Method of L1-Norm Principal Component Analysis for Functional Data. *Symmetry* **2020**, *12*, 182.
https://doi.org/10.3390/sym12010182

**AMA Style**

Yu F, Liu L, Yu N, Ji L, Qiu D.
A Method of L1-Norm Principal Component Analysis for Functional Data. *Symmetry*. 2020; 12(1):182.
https://doi.org/10.3390/sym12010182

**Chicago/Turabian Style**

Yu, Fengmin, Liming Liu, Nanxiang Yu, Lianghao Ji, and Dong Qiu.
2020. "A Method of L1-Norm Principal Component Analysis for Functional Data" *Symmetry* 12, no. 1: 182.
https://doi.org/10.3390/sym12010182