An Approach to Canonical Correlation Analysis Based on Rényi’s Pseudodistances

Abstract

Canonical Correlation Analysis (CCA) infers a pairwise linear relationship between two groups of random variables, $\mathit{X}$ and $\mathit{Y}.$ In this paper, we present a new procedure based on Rényi’s pseudodistances (RP) aiming to detect linear and non-linear relationships between the two groups. RP canonical analysis (RPCCA) finds canonical coefficient vectors, $\mathit{a}$ and $\mathit{b}$, by maximizing an RP-based measure. This new family includes the Information Canonical Correlation Analysis (ICCA) as a particular case and extends the method for distances inherently robust against outliers. We provide estimating techniques for RPCCA and show the consistency of the proposed estimated canonical vectors. Further, a permutation test for determining the number of significant pairs of canonical variables is described. The robustness properties of the RPCCA are examined theoretically and empirically through a simulation study, concluding that the RPCCA presents a competitive alternative to ICCA with an added advantage in terms of robustness against outliers and data contamination.

1. Introduction

Canonical Correlation Analysis (CCA) is a statistical technique used to identify and measure associations among two sets of variables; in the following, denoted by ${\mathit{X}}_{q\times 1}$ and ${\mathit{Y}}_{p\times 1}$ ($q\le p).$ It is appropriate in situations where multiple regression would be used but where there are multiple intercorrelated outcome variables. Hence, it allows us to summarize relationships into a lesser number of statistics while preserving the main facets of those relationships. CCA was first considered in [1] and has been widely used in the statistical literature; for example, to summarize relationships between sets of variables, to reduce the dimensionality of data or to transform two sets of variables into a new dataset of uncorrelated variables as a preprocessing step for the multiple linear regression model. More insight about CCA can be found, e.g., in [2,3].

CCA looks for two direction vectors $\mathit{a},\mathit{b}$ (canonical vectors) such that the linear combinations $U={\mathit{a}}^T\mathit{X}$ and $V={\mathit{b}}^T\mathit{Y},$ so called canonical variables, are (linearly) correlated as much as possible. However, if a linear relationship does not exist between the pairs ${\mathit{a}}^T\mathit{X}$ and ${\mathit{b}}^T\mathit{Y},$ CCA could fail in detecting these pairs of canonical vectors. In other words, CCA can only detect linear relations between the canonical variables, but other functional relationships may exist.

The linear restriction is a significant drawback of CCA when analyzing some real data with highly non-linear relationships. For example, Oulai et al. [4] presented a real situation with non-linear relationships between variables regarding the representation of a hydrological process in the delineation of homogeneous regions. In their context, the two groups of variables under consideration were hydrological variables and meteorological and/or graphical characteristics of watersheds, and their non-linear relationship depended essentially on the physiographic characteristics of the watersheds. Additionally, Ref. [5] presented a nice application of non-linear CCA to seasonal climate forecasting. In [6], some real life data with complex non-linear relationships that cannot be properly captivated by classical CCA are also presented. There is an extensive bibliography addressing non-linear CCA. Without wishing to cite all the existing literature on the topic, we would like to mention some interesting works on the subject: [7] (Chapter 6), [8,9,10,11] and references therein.

To shed light on this problem, let us consider the following situation described in [12]: let $\mathit{X}={(X_1,X_2)}^T$ and $\mathit{Y}={(Y_1,Y_2)}^T$ be a pair of random vectors such that

$$\mathit{X}\sim \mathcal{N}(\mathit{0},\mathit{I}),Y_1=X_1^2+Z,Y_2=Z,$$

with $Z\sim {\chi }_1^2$ and independent from $\mathit{X}.$ In this case, $Cov\left(\mathit{X},\mathit{Y}\right)={\mathit{0}}_{2\times 2},$ and so the vectors $\mathit{X}$ and $\mathit{Y}$ are uncorrelated (so they are linearly independent). Consequently, classical CCA cannot detect that, indeed, $Y_1$ is related (although not linearly) to $X_1,$ even if the variables are not fully independent. On the other hand, as the pair $\left(\mathit{X},\mathit{Y}\right)$ does not follow a normal distribution (and therefore uncorrelation does not imply independence), a hidden relationship may exist (and indeed it does exist!) that has not been detected by CCA. Of course, under normality, rejecting any linear correlation using CCA implies independence between both variables.

It is not surprising that CCA fails in the previous example, as CCA focuses on “linear trends”, but the true relation underlying it is quadratic. To overcome this drawback, in a pioneer paper, Yin [12] proposed the use of the Kullback–Leibler divergence and developed a new procedure called Informational Canonical Correlation Analysis (ICCA), aiming to also detect non-linear relationships for linear combinations of the components.

Let $U={\mathit{a}}^T\mathit{X}$ and $V={\mathit{b}}^T\mathit{Y}$ be linear combinations of $\mathit{X}$ and $\mathit{Y}$ defining a pairwise of canonical variables, with $\mathit{a}\in {\mathbb{R}}^q$ and $\mathit{b}\in {\mathbb{R}}^p$. We denote by $f_{UV}(u,v)$ the joint probability density function (PDF) of $(U,V),$ and, by $f_U\left(u\right)$ (resp. $f_V\left(v\right)$), the marginal unidimensional PDF of U (resp. V). From a statistical point of view, both canonical variables U and V would be independent if their joint distribution coincides with the product of the marginal PDF’s, $f_{UV}(u,v)=f_U\left(u\right)\times f_V\left(v\right),$ and, conversely, a strong dependence between U and V would result in a large statistical distance between the joint PDF and the product of the marginals. A suitable divergence should then be adopted to measure such statistical closeness of the two PDFs. The Kullback–Leibler divergence is the most commonly used measure for distinguishing two distributions, and it has a great statistical importance in the field of information theory.

The Kullback–Leibler divergence between $f_{UV}(u,v)$ and $f_U\left(u\right)\times f_V\left(v\right)$ is given as

$$D_{KL}(\mathit{a},\mathit{b}):=D_{KL}(f_{UV},f_U\times f_V)={\int }_{{\mathbb{R}}^2}{f}_{UV}(u,v)ln\frac{f_{UV}(u,v)}{f_U\left(u\right)f_V\left(v\right)}dudv.$$

(1)

The above divergence is not symmetric, so it quantifies the expected inaccuracy excess from using $f_U\times f_V$ as a model when the actual PDF is $f_{UV}.$ That is, the inaccuracy caused by assuming independence between the pair of canonical variables. Consequently, truly independent canonical variables should minimize the Kullback–Leibler divergence in Equation (1); conversely, functionally dependent canonical variables should maximize the divergence. For more details about the Kullback–Leibler divergence, see [13].

In this vein, ICCA aims to identify q pairwise canonical variables ${\mathit{a}}_i\in {\mathbb{R}}^q$ and ${\mathit{b}}_i\in {\mathbb{R}}^p$, $i\le q\le p$ such that $D_{KL}({\mathit{a}}_i,{\mathit{b}}_i)={max}_{\mathit{a},\mathit{b}}{D}_{KL}(\mathit{a},\mathit{b})$. However, the Kullback–Leibler divergence is invariant under linear transformations, and so there are infinitely many ways to define canonical vectors yielding the same objective function. Then, for identification, we constrain the canonical variables to have unit variance. Moreover, once a relationship is identified by a pair of canonical variables, we expect to exclude its effect from the consecutive canonical variables. For such a purpose, we also require that pairs of canonical variables are uncorrelatated with any other pair. That is, ICCA finds q linearly independent pairs of canonical variables with unit variance maximizing (in decreasing order) the corresponding Kullback–Leibler divergence. Mathematically, to compute each pair of variables, we need to solve the optimization problem $D_{KL}({\mathit{a}}_i,{\mathit{b}}_i)={max}_{\mathit{a},\mathit{b}}{D}_{KL}(\mathit{a},\mathit{b})$ subject to ${\mathit{a}}_i^T{\mathbf{\Sigma }}_X{\mathit{a}}_i=$${\mathit{b}}_i^T{\mathbf{\Sigma }}_Y{\mathit{b}}_i=1$ and ${\mathit{a}}_j^T{\mathbf{\Sigma }}_X{\mathit{a}}_i=$${\mathit{b}}_j^T{\mathbf{\Sigma }}_Y{\mathit{b}}_i=0$ for $j=1,\dots ,i-1,$ where ${\mathbf{\Sigma }}_X$ and ${\mathbf{\Sigma }}_Y$ denote the variance–covariance matrices of $\mathit{X}$ and $\mathit{Y},$ respectively. We apply the same for RP.

From Yin’s (2004) reinterpretation of the canonical analysis, several procedures based on divergence and entropy measures have been proposed to reduce the limitations of CCA. For example, Mandal et al. [14] considered $\left(\alpha ,\beta \right)$ divergence measures defined in [15], and Iaci and Sriram [6] used the density power divergence measures defined in [16] as a measure of statistical closeness. In [17], canonical dependence based on the squared-loss mutual information was studied. Other interesting results regarding ICCA can be seen in [18,19,20,21,22,23].

Despite its popularity, the Kullback–Leibler divergence association measure is quite sensitive to outlying observations, as pointed out in [24]. For outliers, we mean data that behave very differently to expectations according to the law modeling the relation. The main purpose of this paper is to extend the ICCA procedure to a wider family of robust methods based on RP divergence, which remains competitive to ICCA in terms of efficiency but provides a more stable estimation of the canonical vectors in the presence of contamination in the data.

The RP family, parameterized by a tuning parameter $\tau$ controlling the trade-off between robustness and efficiency, was considered for the first time in Jones et al. [25]. Later, Broniatowski et al. [26] demonstrated that RP is a proper divergence, positive for any two densities and for all values of the tuning parameter [26,27], and it is null if (and only if) both densities are the same. The theory in [26] for independent and identically distributed random variables was extended to the case of independent but not identically distributed random variables in [28]. They termed this family of pseudodistances as RP because of their similarities with Renyi’s divergence measures Rényi (1961) [29]. Rényi’s pseudodistance has shown promising behavior in other statistical problems, providing robust minimum RP estimators with good asymptotic and robustness properties, and it includes the Kullback–Leibler divergence as a particular case at $\tau =0.$ For example, Toma and Leoni-Aubin [30] considered efficient and robust measures for general parametric models based on RP and, Toma et al. [31] later developed a new criterion for model selection based on the RP. In [27], Castilla et al. introduced a family of Wald-type tests for testing the parameters in linear regression models, and these results were later extended for generalized linear regression models in [32,33]. Wald-type tests based on minimum RP estimators in bidimensional normal populations were considered in [34]. Jaenada et al. [35] introduced and studied the minimum RP estimators under restricted parameter spaces, which are of great statistical interest in many practical applications such as hypothesis testing. Under the name of γ-entropy, Fujisawa and Eguchi [36] applied RP to introduce robust estimators of general parametric families. Motivated for the great performance of the minimum RP estimator on those different statistical models in terms of robustness, we have adopted the RP divergence to extend the ICCA procedure.

The rest of the paper is organized as follows. The Rényi’s Pseudodistance Canonical Correlation Analysis (RPCCA) is introduced in Section 2, and some of its properties are studied. Next, an estimation design for computing the canonical vectors in practice using RPCCA is described in Section 3. In Section 4, the robustness of the RPCCA is theoretically established. Section 5 describes a permutation test to determine the number of significant canonical variables and thereby provide a dimension reduction method. In Section 6, a Monte Carlo simulation study is carried out to empirically evaluate the performance of the RPCCA and compare the proposed method with the ICCA in terms of estimation accuracy and robustness. An example with real data is studied in Section 6.3. Finally, some conclusions are drawn in Section 7.

2. Rényi’s Pseudodistance Canonical Correlation Analysis

Given two multidimensional random variables $\mathit{X}$ and $\mathit{Y},$ the RPCCA aims to identify two direction vectors $\mathit{a}$ and $\mathit{b}$ (the canonical vectors), such that the corresponding canonical variables $U={\mathit{a}}^T\mathit{X}$ and $V={\mathit{b}}^T\mathit{Y}$ are as dependent as possible. Such dependency is measured in terms of RP between their joint distribution and the product of their marginal distributions. The RP of tuning parameter $\tau$ between the joint distribution of the bidimensional random variable $(U,V)$ and the product of their marginals, $f_U\left(u\right)\times f_V\left(v\right)$, is given for $\tau >0$ by (cf. [26]).

$$\begin{array}{ccc}\hfill d_{\tau }\left(\mathit{a},\mathit{b}\right)& =& d_{\tau }\left(f_U\times f_V,f_{UV}\right)\hfill \\ & =& \frac{1}{\tau +1}ln{\int }_{{\mathbb{R}}^2}{f}_U^{\tau +1}\left(u\right)f_V^{\tau +1}\left(v\right)dudv-\frac{1}{\tau }ln{\int }_{{\mathbb{R}}^2}{f}_U^{\tau }\left(u\right)f_V^{\tau }\left(v\right)f_{UV}(u,v)dudv\hfill \\ & & +\frac{1}{\tau \left(\tau +1\right)}ln{\int }_{{\mathbb{R}}^2}{f}_{UV}^{\tau +1}(u,v)dudv.\hfill \end{array}$$

Hence, the RP measures the statistical discrepancy between the joint PDF of the canonical variables, $f_{UV}$ and the marginal PDF’s product $f_U\times 4_V,$ or, in other words, the loss in accuracy that comes with assuming independence.

For $\tau =0$, the RP can be defined as the corresponding limit, $\tau \to 0$, yielding the Kullback–Leibler divergence:

$$d_0\left(\mathit{a},\mathit{b}\right)=\underset{\tau \downarrow 0}{lim}{d}_{\tau }\left(\mathit{a},\mathit{b}\right)=\underset{\tau \downarrow 0}{lim}{d}_{\tau }\left(f_U\times f_V,f_{UV}\right)=D_{KL}(f_{UV},f_U\times f_V).$$

(2)

As earlier discussed, independent canonical variables lead to $d_{\tau }\left(\mathit{a},\mathit{b}\right)=0,$ and, contrarily, strong dependency should result in large RP distances. Then, the RPCCA procedure aims to identify pairwise canonical vectors ${\mathit{a}}_i\in {\mathbb{R}}^q$ and ${\mathit{b}}_i\in {\mathbb{R}}^p$, $i\le q\le p$ such that

$$d_{\tau }({\mathit{a}}_i,{\mathit{b}}_i)=\underset{\mathit{a},\mathit{b}}{max}{d}_{\tau }(\mathit{a},\mathit{b}),$$

and, as before for identification, the canonical variables should have unit variance and be uncorrelated with any previous pairwise of canonical variables:

$${\mathit{a}}_i^T{\mathbf{\Sigma }}_X{\mathit{a}}_i={\mathit{b}}_i^T{\mathbf{\Sigma }}_Y{\mathit{b}}_i=1,\forall i,$$

$${\mathit{a}}_j^T{\mathbf{\Sigma }}_X{\mathit{a}}_i={\mathit{b}}_j^T{\mathbf{\Sigma }}_Y{\mathit{b}}_i=0,\forall j=1,\dots ,i-1,$$

where ${\mathbf{\Sigma }}_X$ and ${\mathbf{\Sigma }}_Y$ are the variance–covariance matrices of $\mathit{X}$ and $\mathit{Y},$ respectively.

Note that, by Equation (2), the ICCA procedure presented in [12] is recovered at $\tau =0,$ and so the RPCCA generalizes ICCA.

Remark 1.

Given the random vectors $\mathit{X}$ and $\mathit{Y},$ RPCCA finds the vectors ${\mathit{a}}_1,{\mathit{b}}_1$ such that ${\mathit{a}}_1^T\mathit{X}$ and ${\mathit{b}}_1^T\mathit{Y}$ are maximally related. This maximal relation is measured via $d_{\tau }({\mathit{a}}_i,{\mathit{b}}_i)$, as previously defined. Once these vectors ${\mathit{a}}_1,{\mathit{b}}_1$ are obtained, the procedure looks for a new pair of vectors ${\mathit{a}}_2,{\mathit{b}}_2$ such that ${\mathit{a}}_1$ and ${\mathit{a}}_2$ are incorrelated, and the same applies for ${\mathit{b}}_1,{\mathit{b}}_2$, and ${\mathit{a}}_2^T\mathit{X}$ and ${\mathit{b}}_2^T\mathit{Y}$ are maximally related. Consequently,

$$d_{\tau }({\mathit{a}}_1,{\mathit{b}}_1)\ge d_{\tau }({\mathit{a}}_2,{\mathit{b}}_2).$$

Next, the procedure looks for ${\mathit{a}}_3,{\mathit{b}}_3$ being incorrelated to ${\mathit{a}}_1,{\mathit{a}}_2$ and ${\mathit{b}}_1,{\mathit{b}}_2,$ respectively, and so on. Hence, it follows that

$$d_{\tau }({\mathit{a}}_i,{\mathit{b}}_i)\ge d_{\tau }({\mathit{a}}_{i+1},{\mathit{b}}_{i+1}),$$

for any $i=1,\dots ,q-1.$ If $d_{\tau }({\mathit{a}}_i,{\mathit{b}}_i)=0$. Then, independence arises and the procedure stops. In practice, we will have an estimation of $d_{\tau }({\mathit{a}}_i,{\mathit{b}}_i)$, and we will stop the procedure if this value does not exceed a certain threshold. This will be applied in Section 5 in order to determine the number of components.

Let us consider, again, the example described in the introduction, where $\mathit{X}={(X_1,X_2)}^T$ and $\mathit{Y}={(Y_1,Y_2)}^T$ satisfy

$$\mathit{X}\sim \mathcal{N}(\mathit{0},\mathit{I}),Y_1=X_1^2+Z,Y_2\sim Z,Z\sim {\chi }_1^2.$$

The true value of the first pair of canonical vectors are then ${\mathit{a}}_1={\mathit{b}}_1={(1,0)}^T.$ Under the described setup, it follows that

$$f_{UV}(u,v)=f_{X_1,Y_1}(x,y)=\frac{1}{\pi }\frac{1}{2}exp\left(-\frac{1}{2}y\right){(y-x^2)}^{-\frac{1}{2}},y>0,x\in \mathbb{R},$$

and

$$f_U\left(u\right)=f_{X_1}\left(x\right)=\frac{1}{\sqrt{2\pi }}exp\left(-\frac{1}{2}{x}^2\right),f_V\left(v\right)=f_{Y_1}\left(y\right)=\frac{1}{2}exp\left(-\frac{1}{2}y\right),y>0.$$

Clearly,

$$f_{UV}(u,v)\ne f_U\left(u\right)\times f_V\left(v\right)$$

and because of the properties of the divergence

$$d_{\tau }({\mathit{a}}_1,{\mathit{b}}_1)>0.$$

The last inequality holds because the RP divergence, $d_{\tau }(·,·),$ only reaches the value zero if both arguments coincide, as discussed in Section 1 (see [26] for more details). In this case, RPCCA should identify a pair ${\mathit{a}}_1,{\mathit{b}}_1$ with a non-zero informational coefficient of canonical correlation defining the canonical variables ${\mathit{a}}_1^T\mathit{X}$ and ${\mathit{b}}_1^T\mathit{Y}$.

For practical use of RPCCA, it is interesting to note that RPCCA is equivariant under invertible linear transformations. This equality does not hold for other extensions of ICCA, but proportionality arises instead.

Proposition 1.

Consider two random variables U and V, and take $R=cU$ and $S=eV,$ where c and e are non-zero real numbers (Indeed, the result also holds if we consider two random vectors $\mathit{U},\mathit{V}$, and consider $\mathit{R}=C\mathit{U}$ and $\mathit{S}=D\mathit{V},$ where C and D are two invertible matrices. In this case, RP is computed considering multidimensional integrals.). Then,

$$d_{\tau }(f_U\times f_V,f_{UV})=d_{\tau }(f_R\times f_S,f_{RS}).$$

Proof. 

By definition,

$$\begin{array}{ccc}\hfill d_{\tau }(f_R\times f_S,f_{RS})& =& \frac{1}{\tau +1}ln\int f_R^{\tau +1}\left(r\right)f_S^{\tau +1}\left(s\right)drds+\frac{1}{\tau (\tau +1)}ln\int f_{RS}^{\tau +1}(r,s)drds\hfill \\ & & -\frac{1}{\tau }ln\int f_R^{\tau }\left(r\right)f_S^{\tau }\left(s\right)f_{RS}(r,s)drds\hfill \\ & =& \frac{1}{\tau +1}ln\int f_U^{\tau +1}\left(u\right){\left(\frac{1}{c}\right)}^{\tau +1}{f}_V^{\tau +1}\left(v\right){\left(\frac{1}{e}\right)}^{\tau +1}cedudv\hfill \\ & & +\frac{1}{\tau (\tau +1)}ln\int f_{UV}^{\tau +1}(u,v){\left(\frac{1}{ce}\right)}^{\tau +1}cedudv\hfill \\ & & -\frac{1}{\tau }ln\int f_U^{\tau }\left(u\right){\left(\frac{1}{c}\right)}^{\tau }{f}_V^{\tau }\left(s\right){\left(\frac{1}{e}\right)}^{\tau }{f}_{UV}(u,v)\left(\frac{1}{ce}\right)cedudv\hfill \\ & =& \frac{1}{\tau +1}ln{\left(\frac{1}{\left|c\right|\left|e\right|}\right)}^{\tau }\int f_U^{\tau +1}\left(u\right)f_V^{\tau +1}\left(v\right)dudv\hfill \\ & & +\frac{1}{\tau (\tau +1)}ln{\left(\frac{1}{\left|c\right|\left|e\right|}\right)}^{\tau }\int f_{UV}^{\tau +1}(u,v)dudv\hfill \\ & & -\frac{1}{\tau }ln{\left(\frac{1}{\left|c\right|\left|e\right|}\right)}^{\tau }\int f_U^{\tau }\left(u\right)f_V^{\tau }\left(v\right)f_{UV}(u,v)dudv\hfill \\ & =& \left(\frac{1}{\tau +1}+\frac{1}{\tau (\tau +1)}-\frac{1}{\tau }\right)ln{\left(\frac{1}{\left|c\right|\left|e\right|}\right)}^{\tau }+d_{\tau }(f_U\times f_V,f_{UV})\hfill \\ & =& d_{\tau }(f_U\times f_V,f_{UV}).\hfill \end{array}$$

□

The next result establishes that the RPCCA is reduced to CCA in the case of normal distributions.

Proposition 2.

In the case of normal distributions, RPCCA coincides with CCA.

Proof. 

Consider normal populations, i.e., assume that the multidimensional random variables $\mathit{X}$ and $\mathit{Y}$ are jointly normally distributed,

$$\left(\begin{array}{c}\mathit{X}\\ \mathit{Y}\end{array}\right)\equiv \mathcal{N}\left(\left(\begin{array}{c}{\mu }_{\mathit{X}}\\ {\mu }_{\mathit{Y}}\end{array}\right),\left(\begin{array}{cc}{\mathbf{\Sigma }}_X& {\mathbf{\Sigma }}_{XY}\\ {\mathbf{\Sigma }}_{YX}& {\mathbf{\Sigma }}_Y\end{array}\right)\right).$$

Therefore, the bidimensional random variable $\left(U,V\right)=\left({\mathit{a}}^T\mathit{X},{\mathit{b}}^T\mathit{Y}\right)$ follows a bidimensional normal distribution whose vector mean is

$$\mu ={\left({\mu }_1,{\mu }_2\right)}^T={(E\left[{\mathit{a}}^T\mathit{X}\right],E\left[{\mathit{b}}^T\mathit{Y}\right])}^T={({\mathit{a}}^T{\mathit{\mu }}_{\mathit{X}},{\mathit{b}}^T{\mathit{\mu }}_{\mathit{Y}})}^T,$$

and the variance–covariance matrix is given by

$$\left(\begin{array}{cc}{\sigma }_1^2& {\sigma }_1{\sigma }_2\rho \\ {\sigma }_1{\sigma }_2\rho & {\sigma }_2^2\end{array}\right)$$

being

$${\sigma }_1^2=Var\left[{\mathit{a}}^T\mathit{X}\right]={\mathit{a}}^T{\mathbf{\Sigma }}_X\mathit{a},{\sigma }_2^2=Var\left[{\mathit{b}}^T\mathit{Y}\right]={\mathit{b}}^T{\mathbf{\Sigma }}_Y\mathit{b}\mathrm{and}\rho =\frac{Cov\left(U,V\right)}{{\sigma }_1{\sigma }_2}=\frac{{\mathit{a}}^T{\mathbf{\Sigma }}_{XY}\mathit{b}}{{\sigma }_1{\sigma }_2}.$$

On the other hand, the marginal densities $f_{{\mu }_1,{\sigma }_1}\left(u\right)$ and $f_{{\mu }_2,{\sigma }_2}\left(v\right)$ of ${\mathit{a}}^T\mathit{X}$ and ${\mathit{b}}^T\mathit{Y},$ respectively, are normal distributions,

$$f_U\left(u\right)\equiv \mathcal{N}\left({\mu }_1,{\sigma }_1^2\right)\mathrm{and}{f}_V\left(v\right)\equiv \mathcal{N}\left({\mu }_2,{\sigma }_2^2\right).$$

We first compute the RP between $f_{{\mu }_1,{\sigma }_1}\left(u\right)\times$$f_{{\mu }_2,{\sigma }_2}\left(v\right)$ and $f_{{\mu }_1,{\mu }_2,{\sigma }_{1,}{\sigma }_2,\rho }(u,v).$ Considering the results obtained in Supplementary Materials (Appendix A) in [6], we have

$${\int }_{{\mathbb{R}}^2}{f}_U{\left(u\right)}^{\tau +1}{f}_V{\left(v\right)}^{\tau +1}dudv=k_1^{\tau }{\left(1+\tau \right)}^{-1},$$

being $k_1={\left(2\pi {\sigma }_1{\sigma }_2\right)}^{-1}$ and

$${\int }_{{\mathbb{R}}^2}{f}_{UV}{(u,v)}^{\tau +1}dudv=k_1^{\tau }{\left(1+\tau \right)}^{-1}{\left(1-{\rho }^2\right)}^{-\frac{\tau }{2}}.$$

On the other hand, it is not difficult to see that

$${\int }_{{\mathbb{R}}^2}{f}_U^{\tau }\left(u\right)f_V^{\tau }\left(v\right)f_{UV}(u,v)dudv=k_1^{\tau }{\left[(1+\tau (1+\rho \left)\right)(1+\tau (1-\rho \left)\right)\right]}^{-1/2}.$$

Based on the previous quantities, we have

$$\begin{array}{ccc}\hfill d_{\tau }(\mathit{a},\mathit{b})& =& \frac{1}{\tau +1}ln{k}_1^{\tau }{(1+\tau )}^{-1}-\frac{1}{\tau }ln{k}_1^{\tau }{k}_1^{\tau }{\left[(1+\tau (1+\rho \left)\right)(1+\tau (1-\rho \left)\right)\right]}^{-1/2}\hfill \\ & & +\frac{1}{\tau (\tau +1)}ln{k}_1^{\tau }{(1+\tau )}^{-1}{(1-{\rho }^2)}^{-\tau /2}\hfill \\ & =& ln\frac{{\left((1+\tau (1+\rho \left)\right)(1+\tau (1-\rho \left)\right)\right)}^{1/2\tau }}{{(1+\tau )}^{1/\tau }{(1-{\rho }^2)}^{1/2(\tau +1)}}.\hfill \end{array}$$

For fixed $\tau$, it can be seen from the previous expression that $d_{\tau }(\mathit{a},\mathit{b})$ depends on $\rho .$ Moreover, it is not difficult to show that $d_{\tau }(\mathit{a},\mathit{b})$ is an increasing function on ${\rho }^2$ for any $\tau$ (see Figure 1 for $\tau =0.1,0.3$ and $0.9$). To show this, it suffices to see that

$$f_{\tau }\left(\rho \right)=\frac{{\left[(1+\tau (1+\rho \left)\right)(1+\tau (1-\rho \left)\right)\right]}^{1/2\tau }}{{(1+\tau )}^{1/\tau }{(1-{\rho }^2)}^{1/2(\tau +1)}},\rho \in (-1,1)$$

is increasing in ${\rho }^2$, and so it will be its logarithm transform. Now, note that

$$(1+\tau (1+\rho ))(1+\tau (1-\rho ))=1+2\tau +{\tau }^2(1-{\rho }^2).$$

So, it suffices to show that the function

$$\frac{{\left(1+2\tau +{\tau }^2(1-{\rho }^2)\right)}^{1/2\tau }}{{(1-{\rho }^2)}^{1/2(\tau +1)}}$$

is increasing in ${\rho }^2.$ Taking derivatives with respect to ${\rho }^2$, we obtain

$$f^{\prime }\left({\rho }^2\right)=\left[\frac{1}{2\tau }{\left(1+2\tau +{\tau }^2(1-{\rho }^2)\right)}^{1/2\tau -1}(-{\tau }^2){(1-{\rho }^2)}^{1/2(\tau +1)}\right.$$

$$-\left.\frac{1}{2(\tau +1)}{(1-{\rho }^2)}^{1/2(\tau +1)-1}{\left(1+2\tau +{\tau }^2(1-{\rho }^2)\right)}^{1/2\tau }\right]\frac{1}{{(1-{\rho }^2)}^{\frac{1}{\tau +1}}}.$$

Thus, it suffices to check the non-negativity of

$$\left[\frac{1}{2\tau }(-{\tau }^2)(1-{\rho }^2)+\frac{1}{2(\tau +1)}\left(1+2\tau +{\tau }^2(1-{\rho }^2)\right)\right].$$

Finally,

$$\begin{array}{cc}\hfill & \left[\frac{1}{2\tau }(-{\tau }^2)(1-{\rho }^2)+\frac{1}{2(\tau +1)}\left(1+2\tau +{\tau }^2(1-{\rho }^2)\right)\right]=\hfill \\ \hfill & \frac{-{\tau }^2(1-{\rho }^2)}{2\tau (\tau +1)}+\frac{1+2\tau }{2(\tau +1)}=\frac{{\tau }^2({\rho }^2+1)+\tau }{2\tau (\tau +1)}>0.\hfill \end{array}$$

and the result holds. Thus, RPCCA is equivalent to classical CCA in the case of random normal variables. □

It can be seen that $d_{\tau }(\mathit{a};\mathit{b})$ is an increasing function on ${\rho }^2$ for any $\tau >0$ under normal distributions; hence, RPCCA also extends CCA with a tuning parameter $\tau$ determining the sharpness of the distance $d_{\tau }(\mathit{a},\mathit{b})$ (or the function $f_{\tau }(·)$ in the proof of Proposition 2).

3. Consistency

We now focus on the practical side of the RPCCA estimation. In practice, the PDFs $f_{UV},$$f_U$ and $f_V$ are unknown; thus, they should be empirically estimated. Likewise, the RPCCA should be formulated for an empirical setup.

The RP, $d_{\tau }(\mathit{a},\mathit{b}),$ can be expressed in terms of expected values as

$$\begin{array}{cc}\hfill d_{\tau }\left(\mathit{a},\mathit{b}\right)=& \frac{1}{\tau \left(\tau +1\right)}ln{E}_{f_{UV}}\left[f_{U,V}{(U,V)}^{\tau }\right]-\frac{1}{\tau }ln{E}_{f_{UV}}\left[f_U^{\tau }\left(U\right)f_V^{\tau }\left(V\right)\right]\hfill \\ \hfill & +\frac{1}{\tau +1}ln{E}_{f_U}\left[f_U{\left(U\right)}^{\tau }\right]E_{f_V}\left[f_V\left(V\right)\right].\hfill \end{array}$$

(3)

This interpretation of $d_{\tau }\left(\mathit{a},\mathit{b}\right)$ makes the definition of its empirical estimator easier. Let $\left({\mathit{X}}_i,{\mathit{Y}}_i\right),i=1,\dots ,n$ be a random sample of size n from the multidimensional random variables $\left(\mathit{X},\mathit{Y}\right).$ Then, an empirical estimator of $d_{\tau }(\mathit{a},\mathit{b})$ is given by

$$\begin{array}{ccc}\hfill \widehat{d_{\tau }^n}\left(\mathit{a},\mathit{b}\right)& =& \frac{1}{\tau \left(\tau +1\right)}ln\left[\frac{1}{n}\sum _{i=1}^n{\widehat{f_{UV}^n}}^{\tau }\left(u_i,v_i\right)\right]-\frac{1}{\tau }ln\left[\frac{1}{n}\sum _{i=1}^n{\widehat{f_U^n}}^{\tau }\left(u_i\right){\widehat{f_V^n}}^{\tau }\left(v_i\right)\right]\hfill \end{array}$$

(4)

$$\begin{array}{ccc}& & +\frac{1}{\tau +1}ln\left[\frac{1}{n}\sum _{i=1}^n{\widehat{f_U^n}}^{\tau }\left(u_i\right)\frac{1}{n}\sum _{i=1}^n{\widehat{f_V^n}}^{\tau }\left(v_i\right)\right].\hfill \end{array}$$

(5)

Here, $\widehat{f_U^n}\left(u\right),$$\widehat{f_V^n}\left(v\right)$ and $\widehat{f_{UV}^n}\left(u,v\right)$ are kernel density estimators of $f_U\left(u\right),$$f_V\left(v\right)$ and $f_{UV}(u,v),$ respectively, given by

$$\widehat{f_U^n}\left(u\right)=\frac{1}{n{a}_n^1}\sum _{i=1}^{n}K\left(\frac{u-u_i}{a_n^1}\right),u\in \mathbb{R},$$

(6)

$$\widehat{f_V^n}\left(v\right)=\frac{1}{n{a}_n^2}\sum _{i=1}^{n}K\left(\frac{v-v_i}{a_n^2}\right),v\in \mathbb{R},$$

(7)

and

$$\widehat{f_{UV}^n}(u,v)=\frac{1}{n{b}_n^1{b}_n^2}\sum _{i=1}^{n}K\left(\frac{u-u_i}{b_n^1}\right)K\left(\frac{v-v_i}{b_n^2}\right).$$

(8)

For the PDF’s estimators, we will use the univariate Gaussian kernel with $a_n^j=1.06n^{-0.2}{s}_j$ and $b_n^j=n^{-1/6}{s}_j$ for $j=1,2,$ and the corresponding sample standard deviations $s_1$ and $s_2.$ This kernel function was proposed in [37] and adopted in many other extensions of ICCA, but other types of kernels could be considered instead, as long as they satisfy the conditions of Lemma 1 below (When the distribution is known up to a parameter value $f_{\theta },$ this information should be taken into account. Hence, the procedure would be the usual procedure in these situations. First, we estimate the parameter of the distribution $\theta$ by $\widehat{\theta }$ and then consider the distribution with the estimated parameters $f_{\widehat{\theta }}$. Next, we use $f_{\widehat{\theta }}$ instead of $\widehat{f}.$). Other interesting results about kernel distributions can be found in [38,39].

Then, the estimated canonical vectors, based on the RP with tuning parameter $\tau$ can be computed as

$$\begin{array}{cc}\hfill & \left({\widehat{\mathit{a}}}_n^{\tau },{\widehat{\mathit{b}}}_n^{\tau }\right)=arg\underset{\mathit{a},\mathit{b}}{max}\widehat{d_{\tau }^n}\left(\mathit{a},\mathit{b}\right),\hfill \\ \hfill & \mathrm{s}.\mathrm{t}.{\left({\mathit{a}}_n^{\tau }\right)}^T{\widehat{\mathbf{\Sigma }}}_{11}{\mathit{a}}_n^{\tau }=1\mathrm{and}{\left({\mathit{b}}_n^{\tau }\right)}^T{\widehat{\mathbf{\Sigma }}}_{22}{\mathit{b}}_n^{\tau }=1,\hfill \end{array}$$

(9)

where ${\widehat{\mathbf{\Sigma }}}_{11}$ and ${\widehat{\mathbf{\Sigma }}}_{22}$ are the empirical estimators of the variance–covariance matrices of $\mathit{X}$ and $\mathit{Y},$ respectively.

We next establish the consistency of the estimated canonical vectors under some regularity conditions. That is, we will prove that the estimated canonical vectors $\left({\widehat{\mathit{a}}}_n^{\tau },{\widehat{\mathit{b}}}_n^{\tau }\right)$ converge for large sample sizes to the true canonical vectors defining the underlying functional relationship. For such a result, it is necessary to present the following lemma whose proof can be found in [40].

Lemma 1.

Let $({\mathit{X}}_i,{\mathit{Y}}_i),i=1,\dots ,n$ be i.i.d. replications of the multidimensional random variables $(\mathit{X},\mathit{Y}).$ Consider a sequence ${\left\{a_n\right\}}_{n\in \mathbb{N}}$ such that $0<a_n$ and ${lim}_{n\to \infty }{a}_n=0.$ Assume

$$\sum _{n=1}^{\infty }{e}^{-\gamma n{a}_n^2}<\infty ,\sum _{n=1}^{\infty }{e}^{-\gamma n{a}_n^4}<\infty ,\forall \gamma >0.$$

Consider a function K of bounded variation (Consider a function $g:{\mathbb{R}}^k\mapsto \mathbb{R}$, and let P be the set of finite partitions of ${\mathbb{R}}^k$ in rectangles $p=\{[x_j,y_j),j=1,\dots ,u_p\}$. Then, g is said to be of bounded variation if $su{p}_{p\in P}\{{\displaystyle \sum _{j=1}^{u_p}}{\displaystyle \sum _{{ϵ}_1,\dots ,{ϵ}_k\in {\{0,1\}}^k}}{(-1)}^{{\sum }_{i=1}^k{ϵ}_i}g({ϵ}_1{x}_{j1}+(1-{ϵ}_1)y_{j1},\dots ,{ϵ}_k{x}_{jk}+$$(1-{ϵ}_k)y_{jk}\left)\right\}<\infty .$) and suppose $f_U\left({\mathit{a}}^T\mathit{x}\right)$ is uniformly continuous in $\mathit{a}$ and $\mathit{x},f_V\left({\mathit{b}}^T\mathit{y}\right)$ is uniformly continuous in $\mathit{b}$ and $\mathit{y},$ and $f_{UV}({\mathit{a}}^T\mathit{x},{\mathit{b}}^T\mathit{y})$ is uniformly continuous in $\mathit{a},\mathit{x},\mathit{b}$ and $\mathit{y}.$ Then,

$$\underset{\mathit{a},\mathit{x}}{sup}|\widehat{f_U^n}\left({\mathit{a}}^T\mathit{x}\right)-f_U\left({\mathit{a}}^T\mathit{x}\right)|\stackrel{a.s.}{⟶}0.$$

$$\underset{\mathit{b},\mathit{y}}{sup}|\widehat{f_V^n}\left({\mathit{b}}^T\mathit{y}\right)-f_V\left({\mathit{b}}^T\mathit{y}\right)|\stackrel{a.s.}{⟶}0.$$

$$\underset{\mathit{a},\mathit{b},\mathit{x},\mathit{y}}{sup}|\widehat{f_{UV}^n}({\mathit{a}}^T\mathit{x},{\mathit{b}}^T\mathit{y})-f_{UV}({\mathit{a}}^T\mathit{x},{\mathit{b}}^T\mathit{y})|\stackrel{a.s.}{⟶}0.$$

Note that the Gaussian kernel functions defined in Equations (6)–(8) satisfy the conditions of Lemma 1. Of course, any other election of the kernel should also satisfy these regularity conditions. Now, let us define for any real value $b>0$ the set of indices such that the observations ${\mathit{a}}^T{\mathit{x}}_i$ and ${\mathit{b}}^T{\mathit{y}}_i,i=1,\dots ,n$ have positive densities

$${\chi }_b=\{i:f_{UV}^{\tau }({\mathit{a}}^T{\mathit{x}}_i,{\mathit{b}}^T{\mathit{y}}_i)\ge b,f_U^{\tau }\left({\mathit{a}}^T{\mathit{x}}_i\right)\ge b,f_V^{\tau }\left({\mathit{b}}^T{\mathit{y}}_i\right)\ge b\}$$

and denote by $n_b$ the number of data outside this set. The next result establishes the consistency of the RPCCA.

Proposition 3.

Suppose the conditions of Lemma 1 hold. Assume $b\to 0$ such that

$$\frac{n_b}{n}\underset{n\to \infty }{\stackrel{P}{⟶}}0,$$

and consider the estimated and true pairs of canonical vectors,

$$\left({\widehat{\mathit{a}}}_n,{\widehat{\mathit{b}}}_n\right)=arg\underset{\mathit{a},\mathit{b}}{max}\widehat{d_{\tau }^n}\left(\mathit{a},\mathit{b}\right)\mathrm{and}({\mathit{a}}^{*},{\mathit{b}}^{*})=arg\underset{\mathit{a},\mathit{b}}{max}{d}_{\tau }(\mathit{a},\mathit{b}).$$

Further, assume that the maximum $({\mathit{a}}^{*},{\mathit{b}}^{*})$ is unique. Then,

$$({\widehat{\mathit{a}}}_n,{\widehat{\mathit{b}}}_n)\underset{n\to \infty }{\stackrel{P}{⟶}}({\mathit{a}}^{*},{\mathit{b}}^{*}).$$

Proof. 

Take $0<ϵ,0<b$ such that $ϵ\underset{n\to \infty }{⟶}0,b\underset{n\to \infty }{⟶}0$ and $ϵb^{-1}\underset{n\to \infty }{⟶}0.$

By identification, we can assume ${\widehat{\mathit{a}}}_n^T{\mathbf{\Sigma }}_{11}{\widehat{\mathit{a}}}_n={\widehat{\mathit{b}}}_n^T{\mathbf{\Sigma }}_{22}{\widehat{\mathit{b}}}_n=1.$ Let us suppose that

$$({\widehat{\mathit{a}}}_n,{\widehat{\mathit{b}}}_n)\underset{n\to \infty }{\stackrel{P}{\nrightarrow }}({\mathit{a}}^{*},{\mathit{b}}^{*}).$$

Hence, there exists a subsequence of $\left\{({\widehat{\mathit{a}}}_n,{\widehat{\mathit{b}}}_n)\right\}$ (that will be denoted by $({\widehat{\mathit{a}}}_n,{\widehat{\mathit{b}}}_n)$ to avoid hard notation) and $({\mathit{a}}_0,{\mathit{b}}_0)$ such that ${\mathit{a}}_0^T{\mathbf{\Sigma }}_{11}{\mathit{a}}_0={\mathit{b}}_0^T{\mathbf{\Sigma }}_{22}{\mathit{b}}_0=1,({\mathit{a}}_0,{\mathit{b}}_0)\ne ({\mathit{a}}^{*},{\mathit{b}}^{*})$ and

$$({\widehat{\mathit{a}}}_n,{\widehat{\mathit{b}}}_n)⟶({\mathit{a}}_0,{\mathit{b}}_0).$$

Now, applying Lemma 1, we know that

$$sup|\widehat{f_U^n}\left({\widehat{\mathit{a}}}_n^T{\mathit{x}}_i\right)-f_U\left({\widehat{\mathit{a}}}_n^T{\mathit{x}}_i\right)|\underset{n\to \infty }{\stackrel{a.s.}{⟶}}0.$$

$$sup|\widehat{f_V^n}\left({\widehat{\mathit{b}}}_n^T{\mathit{y}}_i\right)-f_V\left({\widehat{\mathit{b}}}_n^T{\mathit{y}}_i\right)|\underset{n\to \infty }{\stackrel{a.s.}{⟶}}0.$$

$$sup|\widehat{f_{UV}^n}({\widehat{\mathit{a}}}_n^T{\mathit{x}}_i,{\widehat{\mathit{b}}}_n^T{\mathit{y}}_i)-f_{UV}({\widehat{\mathit{a}}}_n^T{\mathit{x}}_i,{\widehat{\mathit{b}}}_n^T{\mathit{y}}_i)|\underset{n\to \infty }{\stackrel{a.s.}{⟶}}0.$$

Thus, for $\tau >0,$

$$sup|{\widehat{f_U^n}}^{\tau }\left({\widehat{\mathit{a}}}_n^T{\mathit{x}}_i\right)-f_U^{\tau }\left({\widehat{\mathit{a}}}_n^T{\mathit{x}}_i\right)|\underset{n\to \infty }{\stackrel{a.s.}{⟶}}0.$$

$$sup|{\widehat{f_V^n}}^{\tau }\left({\widehat{\mathit{b}}}_n^T{\mathit{y}}_i\right)-f_V^{\tau }\left({\widehat{\mathit{b}}}_n^T{\mathit{y}}_i\right)|\underset{n\to \infty }{\stackrel{a.s.}{⟶}}0.$$

$$sup|{\widehat{f_{UV}^n}}^{\tau }({\widehat{\mathit{a}}}_n^T{\mathit{x}}_i,{\widehat{\mathit{b}}}_n^T{\mathit{y}}_i)-f_{UV}^{\tau }({\widehat{\mathit{a}}}_n^T{\mathit{x}}_i,{\widehat{\mathit{b}}}_n^T{\mathit{y}}_i)|\underset{n\to \infty }{\stackrel{a.s.}{⟶}}0.$$

Hence, for an n large enough,

$${\widehat{f_U^n}}^{\tau }\left({\widehat{\mathit{a}}}_n^T{\mathit{x}}_i\right)=f_U^{\tau }\left({\widehat{\mathit{a}}}_n^T{\mathit{x}}_i\right)+{\Delta }_{1i}=f_U^{\tau }\left({\mathit{a}}_0^T{\mathit{x}}_i\right)+{\delta }_{1i},$$

$${\widehat{f_V^n}}^{\tau }\left({\widehat{\mathit{b}}}_n^T{\mathit{y}}_i\right)=f_V^{\tau }\left({\widehat{\mathit{b}}}_n^T{\mathit{y}}_i\right)+{\Delta }_{2i}=f_V^{\tau }\left({\mathit{b}}_0^T{\mathit{y}}_i\right)+{\delta }_{2i},$$

$${\widehat{f_{UV}^n}}^{\tau }({\widehat{\mathit{a}}}_n^T{\mathit{x}}_i,{\widehat{\mathit{b}}}_n^T{\mathit{y}}_i)=f_{UV}^{\tau }({\widehat{\mathit{a}}}_n^T{\mathit{x}}_i,{\widehat{\mathit{b}}}_n^T{\mathit{y}}_i)+{\Delta }_{3i}=f_{UV}^{\tau }({\mathit{a}}_0^T{\mathit{x}}_i,{\mathit{b}}_0^T{\mathit{y}}_i)+{\delta }_{3i}.$$

(10)

Here, $|{\delta }_{1i}|,|{\delta }_{2i}|,|{\delta }_{3i}|<ϵ.$ Remark that $ln\left(\frac{1}{n}{\sum }_{i=1}^n{\widehat{f_{UV}^n}}^{\tau }({\widehat{\mathit{a}}}_n^T{\mathit{x}}_i,{\widehat{\mathit{b}}}_n^T{\mathit{y}}_i)\right)$ can be written as

$$ln\left(\frac{1}{n}\sum _{i=1}^n{\widehat{f_{UV}^n}}^{\tau }({\widehat{\mathit{a}}}_n^T{\mathit{x}}_i,{\widehat{\mathit{b}}}_n^T{\mathit{y}}_i)\frac{\frac{1}{n}{\sum }_{i=1}^n{f}_{UV}^{\tau }({\mathit{a}}_0^T{\mathit{x}}_i,{\mathit{b}}_0^T{\mathit{y}}_i)}{\frac{1}{n}{\sum }_{i=1}^n{f}_{UV}^{\tau }({\mathit{a}}_0^T{\mathit{x}}_i,{\mathit{b}}_0^T{\mathit{y}}_i)}\right)$$

$$=ln\left(\frac{1}{n}\sum _{i=1}^n{f}_{UV}^{\tau }({\mathit{a}}_0^T{\mathit{x}}_i,{\mathit{b}}_0^T{\mathit{y}}_i)\right)+ln\left(\frac{\frac{1}{n}{\sum }_{i=1}^n{\widehat{f_{UV}^n}}^{\tau }({\widehat{\mathit{a}}}_n^T{\mathit{x}}_i,{\widehat{\mathit{b}}}_n^T{\mathit{y}}_i)}{\frac{1}{n}{\sum }_{i=1}^n{f}_{UV}^{\tau }({\mathit{a}}_0^T{\mathit{x}}_i,{\mathit{b}}_0^T{\mathit{y}}_i)}\right).$$

Now, applying Equation (10), we obtain

$$\frac{\frac{1}{n}{\sum }_{i=1}^n{\widehat{f_{UV}^n}}^{\tau }({\widehat{\mathit{a}}}_n^T{\mathit{x}}_i,{\widehat{\mathit{b}}}_n^T{\mathit{y}}_i)}{\frac{1}{n}{\sum }_{i=1}^n{f}_{UV}^{\tau }({\mathit{a}}_0^T{\mathit{x}}_i,{\mathit{b}}_0^T{\mathit{y}}_i)}=1+\frac{\frac{1}{n}{\sum }_{i=1}^n{\delta }_{3i}}{\frac{1}{n}{\sum }_{i=1}^n{f}_{UV}^{\tau }({\mathit{a}}_0^T{\mathit{x}}_i,{\mathit{b}}_0^T{\mathit{y}}_i)}.$$

The same can be performed for the two other cases. As $|{\delta }_{3i}|<ϵ,$ it follows that

$$\frac{1}{n}\sum _{i=1}^n{\delta }_{3i}\le ϵ.$$

On the other hand,

$$\frac{1}{n}\sum _{i=1}^n{f}_{UV}^{\tau }({\mathit{a}}_0^T{\mathit{x}}_i,{\mathit{b}}_0^T{\mathit{y}}_i)\ge \frac{1}{n}\sum _{i=1}^{n}I(i\in {\chi }_b)f_{UV}^{\tau }({\mathit{a}}_0^T{\mathit{x}}_i,{\mathit{b}}_0^T{\mathit{y}}_i)\ge \frac{n-n_b}{n}b.$$

As $ϵb^{-1}\to 0$ and $\frac{n_b}{n}\to 0,$ we conclude that

$$\frac{\frac{1}{n}{\sum }_{i=1}^n{\delta }_{3i}}{\frac{1}{n}{\sum }_{i=1}^n{f}_{UV}^{\tau }({\mathit{a}}_0^T{\mathit{x}}_i,{\mathit{b}}_0^T{\mathit{y}}_i)}\to 0.$$

The same can be performed for the two other cases. Hence,

$$\begin{array}{ccc}\hfill {\widehat{d}}_{\tau }^n\left({\widehat{\mathit{a}}}_n,{\widehat{\mathit{b}}}_n\right)& =& \frac{1}{\tau +1}ln\left(\frac{1}{n^2}\sum _{i=1}^n{f}_U^{\tau }\left({\mathit{a}}_0^T{\mathit{x}}_i\right)\sum _{i=1}^n{f}_V^{\tau }\left({\mathit{b}}_0^T{\mathit{y}}_i\right)\right)+\frac{1}{\tau +1}o\left(1\right)\hfill \\ & & +\frac{1}{\tau (\tau +1)}ln\left(\frac{1}{n}\sum _{i=1}^n{f}_{UV}^{\tau }({\mathit{a}}_0^T{\mathit{x}}_i,{\mathit{b}}_0^T{\mathit{y}}_i)\right)+\frac{1}{\tau (\tau +1)}o\left(1\right)\hfill \\ & & -\frac{1}{\tau }ln\left(\frac{1}{n}\sum _{i=1}^n{f}_U^{\tau }\left({\mathit{a}}_0^T{\mathit{x}}_i\right)f_V^{\tau }\left({\mathit{b}}_0^T{\mathit{y}}_i\right)\right)-\frac{1}{\tau }o\left(1\right)\hfill \\ & =& {\overline{d}}_{\tau }^n({\mathit{a}}_0,{\mathit{b}}_0)+o\left(1\right),\hfill \end{array}$$

with

$$\begin{array}{ccc}\hfill {\overline{d}}_{\tau }^n({\mathit{a}}_0,{\mathit{b}}_0)& =& \frac{1}{\tau +1}ln\left(\frac{1}{n^2}\sum _{i=1}^n{f}_U^{\tau }\left({\mathit{a}}_0^T{\mathit{x}}_i\right)\sum _{i=1}^n{f}_V^{\tau }\left({\mathit{b}}_0^T{\mathit{y}}_i\right)\right)\hfill \\ & & +\frac{1}{\tau (\tau +1)}ln\left(\frac{1}{n}\sum _{i=1}^n{f}_{UV}^{\tau }({\mathit{a}}_0^T{\mathit{x}}_i,{\mathit{b}}_0^T{\mathit{y}}_i)\right)\hfill \\ & & -\frac{1}{\tau }ln\left(\frac{1}{n}\sum _{i=1}^n{f}_U^{\tau }\left({\mathit{a}}_0^T{\mathit{x}}_i\right)f_V^{\tau }\left({\mathit{b}}_0^T{\mathit{y}}_i\right)\right).\hfill \end{array}$$

Note that ${\overline{d}}_{\tau }^n({\mathit{a}}_0,{\mathit{b}}_0)-d_{\tau }({\mathit{a}}_0,{\mathit{b}}_0)$ is given by

$$\frac{1}{\tau (\tau +1)}ln\left(\frac{1}{n}\sum _{i=1}^n{f}_{UV}^{\tau }({\mathit{a}}_0^T{\mathit{x}}_i,{\mathit{b}}_0^T{\mathit{y}}_i)\right)-\frac{1}{\tau (\tau +1)}ln\left(E_{f_{UV}}\left(f_{UV}^{\tau }({\mathit{a}}_0^T\mathit{X},{\mathit{b}}_0^T\mathit{Y})\right)\right)$$

$$+\frac{1}{(\tau +1)}ln\left(\frac{1}{n^2}\sum _{i=1}^n{f}_U^{\tau }\left({\mathit{a}}_0^T{\mathit{x}}_i\right)\sum _{i=1}^{n}I(i\in {\chi }_b)f_V^{\tau }\left({\mathit{b}}_0^T{\mathit{y}}_i\right)\right)-\frac{1}{(\tau +1)}ln\left(E_{f_U}{f}_U^{\tau }\left({\mathit{a}}_0^T\mathit{X}\right)E_{f_V}\left(f_V^{\alpha }\left({\mathit{b}}_0^T\mathit{Y}\right)\right)\right)$$

$$-\frac{1}{\tau }ln\left(\frac{1}{n}\sum _{i=1}^n{f}_U^{\tau }\left({\mathit{a}}_0^T{\mathit{x}}_i\right)f_V^{\tau }\left({\mathit{b}}_0^T{\mathit{y}}_i\right)\right)+\frac{1}{\tau }ln\left(E_{f_{UV}}\left(f_U^{\tau }\left({\mathit{a}}_0^T\mathit{X}\right)f_V^{\tau }\left({\mathit{b}}_0^T\mathit{Y}\right)\right)\right).$$

As ln is continuous and applying the Strong Law of Large Numbers, it follows

$$ln\left(\frac{1}{n}\sum _{i=1}^n{f}_{UV}^{\tau }({\mathit{a}}_0^T{\mathit{x}}_i,{\mathit{b}}_0^T{\mathit{y}}_i)\right)\underset{n\to \infty }{\stackrel{a.s.}{⟶}}ln\left(E_{f_{UV}}\left(f_{UV}^{\tau }({\mathit{a}}_0^T\mathit{X},{\mathit{b}}_0^T\mathit{Y})\right)\right).$$

We can perform this similarly for the two other lines. We conclude that ${\overline{d}}_{\tau }^n({\mathit{a}}_0,{\mathit{b}}_0)\underset{n\to \infty }{\stackrel{P}{⟶}}{d}_{\tau }({\mathit{a}}_0,{\mathit{b}}_0)$, and hence ${\widehat{d}}_{\tau }^n({\mathit{a}}_0,{\mathit{b}}_0)\underset{n\to \infty }{\stackrel{P}{⟶}}{d}_{\tau }({\mathit{a}}_0,{\mathit{b}}_0).$ On the other hand, $\widehat{d_{\tau }^n}({\widehat{\mathit{a}}}_n,{\widehat{\mathit{b}}}_n)\ge \widehat{d_{\tau }^n}({\mathit{a}}^{*},{\mathit{b}}^{*})$ because $({\widehat{\mathit{a}}}_n,{\widehat{\mathit{b}}}_n)$ is the optimum by definition.

Taking limits,

$$d_{\tau }({\mathit{a}}_0,{\mathit{b}}_0)=\underset{n\to \infty }{lim}\widehat{d_{\tau }^n}({\widehat{\mathit{a}}}_n,{\widehat{\mathit{b}}}_n)\ge \underset{n\to \infty }{lim}\widehat{d_{\tau }^n}({\mathit{a}}^{*},{\mathit{b}}^{*})=d_{\tau }({\mathit{a}}^{*},{\mathit{b}}^{*}).$$

However, $d_{\tau }({\mathit{a}}^{*},{\mathit{b}}^{*})\ge d_{\tau }({\mathit{a}}_0,{\mathit{b}}_0)$ because $({\mathit{a}}^{*},{\mathit{b}}^{*})$ is the optimum. Hence, as $({\mathit{a}}^{*},{\mathit{b}}^{*})$ is the only maximum, we conclude that

$$({\mathit{a}}^{*},{\mathit{b}}^{*})=({\mathit{a}}_0,{\mathit{b}}_0),$$

a contradiction. □

4. Robustness

To motivate the inherent robustness property of the RPCCA procedure, we examine the behavior of the estimated divergence in Equation (4) for small values of the tuning parameter. The presented heuristic argument was first discussed in [6] for the density power divergence generalization of ICCA. Consider the estimated RP

$$\begin{array}{ccc}\hfill \widehat{d_{\tau }^n}(\mathit{a},\mathit{b})& :=& \frac{1}{\tau +1}ln\left[\left(\frac{1}{n}\sum _{i=1}^n{\widehat{f_U^n}}^{\tau }\left(u_i\right)\right)\left(\frac{1}{n}\sum _{i=1}^n{\widehat{f_V^n}}^{\tau }\left(v_i\right)\right)\right]-\frac{1}{\tau }ln\left[\frac{1}{n}\sum _{i=1}^n{\widehat{f_U^n}}^{\tau }\left(u_i\right){\widehat{f_V^n}}^{\tau }\left(v_i\right)\right]\hfill \\ & & +\frac{1}{\tau (\tau +1)}ln\left[\frac{1}{n}\sum _{i=1}^n{\widehat{f_{UV}^n}}^{\tau }(u_i,v_i)\right].\hfill \end{array}$$

and let the tuning parameter be $\tau \downarrow 0.$ Taking limits in the estimated divergence defined in Equation (4), the first term vanishes, and therefore

$$\widehat{d_{\tau }^n}(\mathit{a},\mathit{b})\approx -\frac{1}{\tau }ln\left[\frac{1}{n}\sum _{i=1}^n{\widehat{f_U^n}}^{\tau }\left(u_i\right){\widehat{f_V^n}}^{\tau }\left(v_i\right)\right]+\frac{1}{\tau (\tau +1)}ln\left[\frac{1}{n}\sum _{i=1}^n{\widehat{f_{UV}^n}}^{\tau }(u_i,v_i)\right].$$

We first study the limiting behavior of the first term,

$$l_{\tau }:=-\frac{1}{\tau }ln\left[\frac{1}{n}\sum _{i=1}^n{\widehat{f_U^n}}^{\tau }\left(u_i\right){\widehat{f_V^n}}^{\tau }\left(v_i\right)\right].$$

For $\tau \downarrow 0$, this term is a limit of indeterminate form (0/0). Applying L’Hôpital’s rule, we obtain

$$l_{\tau }\approx \frac{\frac{1}{n}\left[{\sum }_{i=1}^n{\widehat{f_U^n}}^{\tau }\left(u_i\right){\widehat{f_V^n}}^{\tau }\left(v_i\right)ln\left(\widehat{f_U^n}\left(u_i\right)\right)+{\widehat{f_V^n}}^{\tau }\left(v_i\right){\widehat{f_U^n}}^{\tau }\left(u_i\right)ln\left(\widehat{f_V^n}\left(v_i\right)\right)\right]}{\frac{1}{n}{\sum }_{i=1}^n{\widehat{f_U^n}}^{\tau }\left(u_i\right){\widehat{f_V^n}}^{\tau }\left(v_i\right)}.$$

Now, the denominator tends to 1 when $\tau \downarrow 0$ so that

$$l_{\tau }\approx \frac{1}{n}\left[\sum _{i=1}^n{\widehat{f_U^n}}^{\tau }\left(u_i\right){\widehat{f_V^n}}^{\tau }\left(v_i\right)ln\left(\widehat{f_U^n}\left(u_i\right)\right)+{\widehat{f_V^n}}^{\tau }\left(v_i\right){\widehat{f_U^n}}^{\tau }\left(u_i\right)ln\left(\widehat{f_V^n}\left(v_i\right)\right)\right].$$

Similarly, consider

$$m_{\tau }:=\frac{1}{\tau (\tau +1)}ln\left[\frac{1}{n}\sum _{i=1}^n{\widehat{f_{UV}^n}}^{\tau }(u_i,v_i)\right]$$

and consider its L’Hôpital approximation given by

$$m_{\tau }\approx \frac{1}{2\tau +1}\frac{\frac{1}{n}{\sum }_{i=1}^n{\widehat{f_{UV}^n}}^{\tau }(u_i,v_i)ln\widehat{f_{UV}^n}(u_i,v_i)}{\frac{1}{n}{\sum }_{i=1}^n{\widehat{f_{UV}^n}}^{\tau }(u_i,v_i)}\approx \frac{1}{n}\sum _{i=1}^n{\widehat{f_{UV}^n}}^{\tau }(u_i,v_i)ln\widehat{f_{UV}^n}(u_i,v_i).$$

Consequently,

$$\widehat{d_{\tau }^n}(\mathit{a},\mathit{b})\approx \frac{1}{n}\left[\sum _{i=1}^n{\widehat{f_{UV}^n}}^{\tau }(u_i,v_i)ln\widehat{f_{UV}^n}(u_i,v_i)-\sum _{i=1}^n{\widehat{f_U^n}}^{\tau }\left(u_i\right){\widehat{f_V^n}}^{\tau }\left(v_i\right)ln\left(\widehat{f_U^n}\left(u_i\right)\widehat{f_V^n}\left(v_i\right)\right)\right].$$

Note that this approximation is valid for $\tau$ closed to 0, but, for $\tau =0,$

$$d_0(\mathit{a},\mathit{b})=\underset{\tau \downarrow 0}{lim}{\tilde{d}}_{\tau }(\mathit{a},\mathit{b})=\frac{1}{n}\sum _{i=1}^{n}ln\left(\frac{\widehat{f_{UV}^n}(u_i,v_i)}{\widehat{f_U^n}\left(u_i\right)\widehat{f_V^n}\left(v_i\right)}\right)=\frac{1}{n}\left(\sum _{i=1}^{n}ln\widehat{f_{UV}^n}(u_i,v_i)-\sum _{i=1}^{n}ln\widehat{f_U^n}\left(u_i\right)\widehat{f_V^n}\left(v_i\right)\right).$$

This implies that $\widehat{d_{\tau }^n}(f_U\times f_V,f_{UV})$ can be seen as a weighted value of the empirical Kullback–Leibler divergence and the weights depend on ${\widehat{f_{UV}^n}}^{\tau }(u_i,v_i),{\widehat{f_U^n}}^{\tau }\left(u_i\right)$ and ${\widehat{f_V^n}}^{\tau }\left(v_i\right).$ Therefore, if the observations ${\mathit{x}}_i,{\mathit{y}}_i$ or both are outliers, the corresponding density estimations would decrease, so the corresponding weights would not be considered as important as other data on the estimated distance, thus making Renyi’s pseudodistance more robust to outliers than the Kullback–Leibler.

5. Testing to Determine the Number of Pairs

In this section, a dimension reduction algorithm is described for determining the number of significant pairs of canonical vectors: the non-parametric sequential test [41,42].

In the classical approach of CCA, the maximum number of pairs $({\mathit{a}}_i,{\mathit{b}}_i)$ is determined by the greatest index j such that $({\mathit{a}}_j,{\mathit{b}}_j)$ is the first pair satisfying $\rho ({\mathit{a}}_j^T\mathit{X},{\mathit{b}}_j^T\mathit{Y})=0.$ That is, the CCA should run until the best estimated pair leads to linear independence. A natural extension for the RPCCA formulation is then replicated in the CCA dimension reduction algorithm, but using the RP divergence as a measure of dependence.

Let us denote by $d_{\tau }^i$ the maximum value achieved at the i-th iteration,

$$d_{\tau }^i=\underset{{\mathit{a}}_i,{\mathit{b}}_i}{max}{d}_{\tau }\left({\mathit{a}}_i,{\mathit{b}}_i\right),i=1,\dots ,l=min(q,p),$$

such that ${\mathit{a}}_i^T{\mathbf{\Sigma }}_{11}{\mathit{a}}_i=$${\mathit{b}}_i^T{\mathbf{\Sigma }}_{22}{\mathit{b}}_i=1$ and ${\mathit{a}}_j^T{\mathbf{\Sigma }}_{11}{\mathit{a}}_i=$${\mathit{b}}_j^T{\mathbf{\Sigma }}_{22}{\mathit{b}}_i=0.$ The sequence of maximums is decreasing and lower-bounded by $0,$ indicating independence between the estimated canonical variables, $d_{\tau }^1\ge d_{\tau }^2\ge \dots \ge d_{\tau }^l\ge 0.$ Then, a stopping criterion for the maximum number of canonical correlations is naturally determined by the testing problem

$$H_0:d_{\tau }^i=0\mathrm{vs}{H}_1:d_{\tau }^i>0,i=1,\dots ,l.$$

If $H_0$ is not rejected, then all subsequent canonical variables from the i-th onward are not significantly related. Otherwise, the relation is significant, and the maximum number of significant canonical correlations is at least $i.$ It is difficult to obtain the exact sample distribution of $d_{\tau }^i$, but a non-parametric permutation test can be applied, as proposed in [24], for estimating the p-value of the test. Let us explain this procedure with some detail. Suppose there is a relationship between ${\mathit{a}}_i^T\mathit{X}$ and ${\mathit{b}}_i^T\mathit{Y}$ for some vectors ${\mathit{a}}_i,{\mathit{b}}_i$, i.e., $H_0$ does not hold. This means that there exists a function f such that

$$f\left({\mathit{a}}_i^T\mathit{X}\right)\approx {\mathit{b}}_i^T\mathit{Y}.$$

Our procedure will estimate vectors ${\mathit{a}}_i$ and ${\mathit{b}}_i$ and will consider some (near!) vectors ${\widehat{\mathit{a}}}_i$ and ${\widehat{\mathit{b}}}_i,$ respectively. Consequently, we expect that for the sample $({\mathit{X}}_1,{\mathit{Y}}_1),\dots ,({\mathit{X}}_n,{\mathit{Y}}_n),$ we will obtain

$$f\left({\widehat{\mathit{a}}}_i^T{\mathit{X}}_j\right)\approx {\widehat{\mathit{b}}}_i^T{\mathit{Y}}_j,j=1,\dots ,n.$$

This will translate in a large value of $d_{\tau }^n({\widehat{{\mathit{a}}_i}}^T\mathit{X},{\widehat{{\mathit{b}}_i}}^T\mathit{Y})$ and, consequently, the corresponding estimation $\widehat{d_{\tau }^n}({\widehat{{\mathit{a}}_i}}^T\mathit{X},{\widehat{{\mathit{b}}_i}}^T\mathit{Y}).$ Now, if we consider a permutation of the data corresponding to $\mathit{X}$ but maintaining the order for the data corresponding to $\mathit{Y},$ any possible relationship is destroyed because the data corresponding to ${\mathit{X}}_i$ do not correspond to individual i in the sample, so that they have nothing to do with ${\mathit{Y}}_i$. In other words, if we denote the reordered sample for $({\mathit{X}}_1,\dots ,{\mathit{X}}_n)$ by $({\mathit{X}}_1^{*},\dots ,{\mathit{X}}_n^{*}),$ it follows that for any $\mathit{c},\mathit{d}$, then $\widehat{d_{\tau }^n}({\mathit{c}}^T{\mathit{X}}^{*},{\mathit{d}}^T\mathit{Y})\approx 0$ showing independence. Consequently, when the procedure looks for some vectors $\widehat{{\mathit{a}}_i^{*}},\widehat{{\mathit{b}}_i^{*}}$ s.t.

$$\widehat{d_{\tau }^n}({\widehat{{\mathit{a}}_i^{*}}}^T{\mathit{X}}^{*},{\widehat{{\mathit{b}}_i^{*}}}^T\mathit{Y})=max\widehat{d_{\tau }^n}\left({\mathit{a}}^T{\mathit{X}}^{*},{\mathit{b}}^T\mathit{Y}\right),$$

these values are not expected to model a strong relation (because it does not exist), so that we expect $\widehat{d_{\tau }^n}({\widehat{{\mathit{a}}_i^{*}}}^T{\mathit{X}}^{*},{\widehat{{\mathit{b}}_i^{*}}}^T\mathit{Y})\approx 0.$ Hence, if $H_0$ does not hold and a relationship between the canonical variables exists, we expect that for (almost) any permutation

$$\widehat{d_{\tau }^n}({\widehat{{\mathit{a}}_i}}^T\mathit{X},{\widehat{{\mathit{b}}_i}}^T\mathit{Y})>\widehat{d_{\tau }^n}({\widehat{{\mathit{a}}_i^{*}}}^T{\mathit{X}}^{*},{\widehat{{\mathit{b}}_i^{*}}}^T\mathit{Y}).$$

On the other hand, if $H_0$ holds, then there is independence between ${\mathit{c}}^T\mathit{X}$ and ${\mathit{d}}^T\mathit{Y}$ for any $\mathit{c},\mathit{d}.$ Consequently, for the best possible estimated vectors $\widehat{{\mathit{a}}_i},\widehat{{\mathit{b}}_i}$, we will obtain

$$\widehat{d_{\tau }^n}({\widehat{{\mathit{a}}_i}}^T\mathit{X},{\widehat{{\mathit{b}}_i}}^T\mathit{Y})\approx 0.$$

When considering a permutation of the values corresponding to $\mathit{X},$ independence will arise again, and hence, in this case, we expect

$$\widehat{d_{\tau }^n}({\widehat{{\mathit{a}}_i}}^T\mathit{X},{\widehat{{\mathit{b}}_i}}^T\mathit{Y})\approx \widehat{d_{\tau }^n}({\widehat{{\mathit{a}}_i^{*}}}^T{\mathit{X}}^{*},{\widehat{{\mathit{b}}_i^{*}}}^T\mathit{Y}).$$

Of course, the number of possible permutations is $n!$, and this is not affordable for large values of n. Hence, we are going to consider just a subset of randomly chosen permutations. Then, if ${\widehat{d}}_{\tau }^{i,w}$ denotes the value of the index corresponding to the w-th randomly permuted sample, the estimated p-value of the test is given by

$$\frac{1}{R}\sum _{w=1}^R{I}_{\left[{\widehat{d}}_{\tau }^{i,w}>{\widehat{d}}_{\tau }^i\right]},$$

where R denotes the number of permutations considered. Yin [12] used $R=1000$ for a permutation test for ICCA. If the p-value is smaller than a certain significance level, then the null hypothesis $d_{\tau }^i=0$ should be rejected implying a significant relationship for the i-th canonical variables, and the process should be repeated for $i+1.$ Conversely, if the null hypothesis is not rejected, then we should assume that the canonical variables are independent and conclude that there are only i estimated canonical variables exhibiting significant relationships. More details about this dimension reduction method can be seen in [24].

6. Simulation Study

6.1. Computational Methods

Consider $\mathbb{X}=({\mathit{x}}_1,\dots ,{\mathit{x}}_n)$ and $\mathbb{Y}=({\mathit{y}}_1,\dots ,{\mathit{y}}_n)$ as $p\times n$ and $q\times n$ matrices with n observations of the random variables $\mathit{X}$ and $\mathit{Y},$ respectively. The estimation of the i-th pair of canonical vectors ${\widehat{\mathit{a}}}_i^{\tau },{\widehat{\mathit{b}}}_i^{\tau }$ based on the RP with tuning parameter $\tau$ is computed through the constrained maximization problem

$$\begin{array}{cc}\hfill & \left({\widehat{\mathit{a}}}_n^{\tau },{\widehat{\mathit{b}}}_n^{\tau }\right)=arg\underset{\mathit{a},\mathit{b}}{max}\widehat{d_{\tau }^i}\left(\mathit{a},\mathit{b}\right),\hfill \\ \hfill & \mathrm{s}.\mathrm{t}.{\left({\mathit{a}}_i^{\tau }\right)}^T{\widehat{\mathbf{\Sigma }}}_X{\mathit{a}}_i^{\tau }=1\mathrm{and}{\left({\mathit{b}}_i^{\tau }\right)}^T{\widehat{\mathbf{\Sigma }}}_Y{\mathit{b}}_n^{\tau }=1,\hfill \\ \hfill & {\left({\mathit{a}}_i^{\tau }\right)}^T{\widehat{\mathbf{\Sigma }}}_X{\mathit{b}}_j^{\tau }=0\mathrm{and}{\left({\mathit{b}}_i^{\tau }\right)}^T{\widehat{\mathbf{\Sigma }}}_Y{\mathit{b}}_j^{\tau }=0,j=1,\dots ,i-1\hfill \end{array}$$

(11)

where ${\widehat{\mathbf{\Sigma }}}_X$ and ${\widehat{\mathbf{\Sigma }}}_Y$ are the empirical estimators of the variance–covariance matrices of the multidimensional random variables $\mathit{X}$ and $\mathit{Y},$ respectively.

The optimization problem constraints can be simplified by scaling the sample matrices to have zero mean and unit variance as follows:

$$\begin{array}{cc}\hfill \tilde{\mathbb{X}}& ={\widehat{\mathbf{\Sigma }}}_X^{-1/2}(\mathbb{X}-\overline{\mathbb{X}})\hfill \\ \hfill \tilde{\mathbb{Y}}& ={\widehat{\mathbf{\Sigma }}}_{22}^{-1/2}(\mathbb{Y}-\overline{\mathbb{Y}}),\hfill \end{array}$$

(12)

where $\overline{\mathbb{X}}$ and $\overline{\mathbb{Y}}$ denote the corresponding sample mean vectors. From Proposition 1, the RPCCA is invariant under such linear transformations, and, consequently, the problem constraints are transformed into

$$\begin{array}{cc}\hfill & {\left({\mathit{a}}_i^{\tau }\right)}^T{\mathit{a}}_i^{\tau }=1\mathrm{and}{\left({\mathit{b}}_i^{\tau }\right)}^T{\mathit{b}}_n^{\tau }=1,\hfill \\ \hfill & {\left({\mathit{a}}_i^{\tau }\right)}^T{\mathit{b}}_j^{\tau }=0\mathrm{and}{\left({\mathit{b}}_i^{\tau }\right)}^T{\mathit{b}}_j^{\tau }=0,j=1,\dots ,i-1.\hfill \end{array}$$

(13)

For empirical covariance matrices, it may appear as disease degeneration resulting in uninvertible matrices. In those cases, we can skip the scaling transform and apply the estimation algorithm under the original restrictions.

From the transformed canonical vectors, ${\tilde{\mathit{a}}}_i$ and ${\tilde{\mathit{b}}}_i,$ the estimated canonical vectors in the original space can be easily recovered as ${\widehat{\mathit{a}}}_i={\widehat{\mathbf{\Sigma }}}_X^{-1/2}{\tilde{\mathit{a}}}_i$ and ${\widehat{\mathit{b}}}_i={\widehat{\mathbf{\Sigma }}}_Y^{-1/2}{\tilde{\mathit{b}}}_i.$

Here, the constrained optimization is carried out iteratively using the non-linear constrained optimizer optimize from the scipy package in Python, which implements a Sequential Quadratic Programming (SQP) method. The source code for the implementation is publicly available on https://github.com/MariaJaenada/Robust-Canonical-Correlations (Github) (accessed on 29 January 2023).

6.2. Monte Carlo Simulation

We empirically examine the robustness of the RPCCA method through a Monte Carlo simulation. We consider a pair of random vectors, $\mathit{X}=(X_1,\dots ,X_8)$ and $\mathit{Y}=(Y_1,Y_2,Y_3),$ whose components satisfy a linear and a non-linear relationship of the form:

$$Y_1={(2X_1+X_2+X_3)}^2\mathrm{and}{Y}_2=X_2-X_3.$$

(14)

The rest of the variables are independent and they are defined as follows: $X_1,X_2,X_6,X_7$ and $X_8$. They are standard normal variables. $X_3$ comes from a chi-square distribution with 7 degrees of freedom, $X_4$ follows a t-Student distribution with 5 degrees of freedom and $X_5$ comes from a Fisher–Snedecor distribution with 3 and 12 degrees of freedom, respectively. Finally, $Y_3$ comes from a t-Student with 9 degrees of freedom.

The true underlying canonical vectors are then ${\mathit{a}}_1=(0,1,-1,0,0,0,0,0),$${\mathit{b}}_1=(0,1,0)$ and ${\mathit{a}}_2=(2,1,1,0,0,0,0,0),$${\mathit{b}}_2=(1,0,0).$ Note that they are orthogonal, and so are the related variables $Y_1$ and $Y_2.$ Although in the procedure we compute unit-norm vectors, we have considered the description vectors with natural coefficients as they look easier to understand. We named the first canonical vector ${\mathit{a}}_1$ because we empirically detected that the linear relationship is first captured.

We generate a random sample of the pairs $\mathit{X}$ and $\mathit{Y}$ of size $n=100$, and we estimate the pairs of canonical vectors ${\widehat{\mathit{a}}}_i$ and ${\widehat{\mathit{b}}}_i,$$i=1,2,$ such that the random variables $U_i={\mathit{a}}_i^T\mathit{X}$ and $V_i={\mathit{b}}_i^T\mathit{Y}$ are functionally interrelated. To examine the performance of the RPCCA method under contamination, we randomly switch the functional relationships in Equation (14) for an $\epsilon \%$ of the observations, with $\epsilon =5,10,15$ and 20 denoting the contamination proportion. That is, for a random $\epsilon \%$ of the observations, the values of $Y_1$ and $Y_2$ are exchanged, generating orthogonal outliers; the functions defining the $Y_2$ and $Y_1$ are orthogonal to each other. Therefore, this contamination will worsen both relationships at the same time in orthogonal directions. We repeat the simulations over $R=500$ replications and compute averages of the following performance measures: We quantify the accuracy of the estimates with the absolute correlations between the estimated and true canonical variables, $|\rho ({\mathit{a}}_i,{\widehat{\mathit{a}}}_i)|=|\rho ({\mathit{a}}_i^T\mathit{X},{\widehat{\mathit{a}}}_i^T\mathit{X})|$ and $|\rho ({\mathit{b}}_i,{\widehat{\mathit{b}}}_i)|=|\rho ({\mathit{b}}_i^T\mathit{Y},{\widehat{\mathit{b}}}_i^T\mathit{Y})|.$ Additionally, to evaluate the robustness of the method, we compute the $L_2$-norm between the canonical vectors fitted under uncontaminated and contaminated data, $\widehat{\mathit{a}}$ and ${\widehat{\mathit{a}}}^c,$

$$L_2(\widehat{\mathit{a}},{\widehat{\mathit{a}}}^c)=\left|\right|\widehat{\mathit{a}}-{\widehat{\mathit{a}}}^c{\left|\right|}_2$$

as well as the projection of ${\widehat{\mathit{a}}}^c$ into the orthogonal subspace spanned by the uncontaminated estimate, $\widehat{\mathit{a}},$

$$P_2(\widehat{\mathit{a}},{\widehat{\mathit{a}}}^c)=\left|\right|(\mathit{I}-\widehat{\mathit{a}}{\widehat{\mathit{a}}}^T){\widehat{\mathit{a}}}_c^T{\left|\right|}_2.$$

The distance measures $L_2(\widehat{\mathit{a}},{\widehat{\mathit{a}}}^c)$ and $P_2(\widehat{\mathit{a}},{\widehat{\mathit{a}}}^c)$ are smaller the more stable the estimate is, implying that the estimates are not largely affected by the contamination; hence the corresponding method is more robust. Summarizing, the correlations between true and estimated canonical variables $\rho (·,·)$ aim to represent the accuracy of the method, whereas the distance measures between estimated canonical vectors for pure data and for contaminated data, $L_2$ and $P_2$, aim to represent the robustness of the method.

Table 1. RPCCA error measures for the first canonical vector under different values of the tuning parameter $\tau .$

$\mathit{\tau }$	0	0.1	0.2	0.3	0.4	0.5	0.6	0.7	0.8
Pure data
$\rho ({\widehat{\mathit{a}}}_1,{\widehat{\mathit{b}}}_1)$	0.92878	0.96111	0.95505	0.97044	0.97099	0.98082	0.97090	0.98256	0.98606
$\rho ({\mathit{a}}_1,{\widehat{\mathit{a}}}_1)$	0.99908	0.99907	0.99893	0.99877	0.99856	0.99831	0.99799	0.99767	0.99728
$\rho ({\mathit{b}}_1,{\widehat{\mathit{b}}}_1)$	0.99946	0.99951	0.99933	0.99938	0.99936	0.99931	0.99906	0.99916	0.99899
5% contamination
$\rho ({\widehat{\mathit{a}}}_1^c,{\widehat{\mathit{b}}}_1^c)$	0.65796	0.68689	0.71486	0.75556	0.76384	0.80646	0.80876	0.81990	0.81201
$\rho ({\mathit{a}}_1,{\widehat{\mathit{a}}}_1^c)$	0.99801	0.99815	0.99805	0.99786	0.99749	0.99645	0.99353	0.98368	0.96927
$\rho ({\mathit{b}}_1,{\widehat{\mathit{b}}}_1^c)$	0.99548	0.99571	0.99531	0.99538	0.99461	0.99345	0.99084	0.97936	0.96222
$P_2(\widehat{{\mathit{a}}_1},{\widehat{\mathit{a}}}_1^c)$	0.41125	0.38246	0.36111	0.31247	0.30856	0.25109	0.25272	0.22822	0.23050
$P_2(\widehat{{\mathit{b}}_1},{\widehat{\mathit{b}}}_1^c)$	0.35685	0.32477	0.31442	0.27277	0.26949	0.22345	0.22962	0.20914	0.21654
$L_2(\widehat{{\mathit{a}}_1},{\widehat{\mathit{a}}}_1^c)$	0.56677	0.52712	0.49669	0.42880	0.42197	0.33991	0.34109	0.30249	0.30241
$L_2(\widehat{{\mathit{b}}_1},{\widehat{\mathit{b}}}_1^c)$	0.41496	0.37628	0.36696	0.31681	0.31561	0.26241	0.27298	0.24999	0.26391
10% contamination
$\rho ({\widehat{\mathit{a}}}_1^c,{\widehat{\mathit{b}}}_1^c)$	0.41963	0.43878	0.46604	0.49193	0.53443	0.56155	0.57944	0.59613	0.60565
$\rho ({\mathit{a}}_1,{\widehat{\mathit{a}}}_1^c)$	0.99698	0.99714	0.99712	0.99686	0.99563	0.99225	0.98450	0.96631	0.95789
$\rho ({\mathit{b}}_1,{\widehat{\mathit{b}}}_1^c)$	0.99054	0.99018	0.98974	0.98968	0.98719	0.98401	0.97274	0.95167	0.93961
$P_2(\widehat{{\mathit{a}}_1},{\widehat{\mathit{a}}}_1^c)$	0.68137	0.65989	0.63978	0.60585	0.57014	0.53623	0.51301	0.48214	0.47437
$P_2(\widehat{{\mathit{b}}_1},{\widehat{\mathit{b}}}_1^c)$	0.56499	0.54370	0.53006	0.51765	0.48283	0.46007	0.44608	0.43238	0.43151
$L_2(\widehat{{\mathit{a}}_1},{\widehat{\mathit{a}}}_1^c)$	0.94237	0.91248	0.88343	0.83560	0.78292	0.73427	0.69877	0.64896	0.63289
$L_2(\widehat{{\mathit{b}}_1},{\widehat{\mathit{b}}}_1^c)$	0.63783	0.61685	0.60406	0.59298	0.55556	0.53199	0.52065	0.51211	0.51843
15% contamination
$\rho ({\widehat{\mathit{a}}}_1^c,{\widehat{\mathit{b}}}_1^c)$	0.26726	0.29776	0.32650	0.34987	0.40472	0.42547	0.43099	0.43524	0.45246
$\rho ({\widehat{\mathit{a}}}_1,{\widehat{\mathit{a}}}_1^c)$	0.99662	0.99682	0.99670	0.99631	0.99405	0.98763	0.97539	0.95782	0.93472
$\rho ({\widehat{\mathit{b}}}_1,{\widehat{\mathit{b}}}_1^c)$	0.98740	0.98702	0.98627	0.98541	0.98364	0.97541	0.95686	0.93536	0.91078
$P_2(\widehat{{\mathit{a}}_1},{\widehat{\mathit{a}}}_1^c)$	0.83673	0.81093	0.78650	0.75948	0.70320	0.68752	0.68190	0.67188	0.64529
$P_2(\widehat{{\mathit{b}}_1},{\widehat{\mathit{b}}}_1^c)$	0.67519	0.65622	0.64107	0.62355	0.58194	0.57677	0.58054	0.58166	0.57108
$L_2(\widehat{{\mathit{a}}_1},{\widehat{\mathit{a}}}_1^c)$	1.16074	1.12367	1.08848	1.04925	0.96766	0.94157	0.92937	0.90983	0.86243
$L_2(\widehat{{\mathit{b}}_1},{\widehat{\mathit{b}}}_1^c)$	0.75424	0.73561	0.72109	0.70581	0.66064	0.66160	0.67484	0.68431	0.68025
20% contamination
$\rho ({\widehat{\mathit{a}}}_1^c,{\widehat{\mathit{b}}}_1^c)$	0.19852	0.21736	0.22524	0.23800	0.27663	0.28001	0.30514	0.32119	0.31858
$\rho ({\widehat{\mathit{a}}}_1,{\widehat{\mathit{a}}}_1^c)$	0.99661	0.99683	0.99652	0.99593	0.99429	0.99003	0.96500	0.94362	0.90237
$\rho ({\widehat{\mathit{b}}}_1,{\widehat{\mathit{b}}}_1^c)$	0.98527	0.98503	0.98336	0.98251	0.97950	0.97282	0.94210	0.91641	0.86912
$P_2(\widehat{{\mathit{a}}_1},{\widehat{\mathit{a}}}_1^c)$	0.90198	0.90524	0.89112	0.89756	0.85384	0.85092	0.82023	0.80367	0.79283
$P_2(\widehat{{\mathit{b}}_1},{\widehat{\mathit{b}}}_1^c)$	0.72434	0.71723	0.71807	0.72166	0.69817	0.70049	0.69510	0.69435	0.70935
$L_2(\widehat{{\mathit{a}}_1},{\widehat{\mathit{a}}}_1^c)$	0.45211	0.45326	0.44686	0.45094	0.42851	0.42746	0.41497	0.41101	0.41076
$L_2(\widehat{{\mathit{b}}_1},{\widehat{\mathit{b}}}_1^c)$	0.61339	0.61723	0.60407	0.60759	0.59650	0.59591	0.58441	0.59536	0.59885

and

Table 2. RPCCA error measures for the second canonical vector under different values of the tuning parameter $\tau .$

$\mathit{\tau }$	0	0.1	0.2	0.3	0.4	0.5	0.6	0.7	0.8
Pure data
$\rho ({\widehat{\mathit{a}}}_2,{\widehat{\mathit{b}}}_2)$	0.22829	0.19656	0.20033	0.18556	0.18386	0.17334	0.17650	0.16407	0.15426
$\rho ({\mathit{a}}_2,{\widehat{\mathit{a}}}_2)$	0.99687	0.99704	0.99514	0.99387	0.98976	0.96917	0.94815	0.93493	0.88471
$\rho ({\mathit{b}}_2,{\widehat{\mathit{b}}}_2)$	0.99464	0.99490	0.99122	0.98980	0.98375	0.96336	0.93706	0.91885	0.85931
5% contamination
$\rho ({\widehat{\mathit{a}}}_2^c,{\widehat{\mathit{b}}}_2^c)$	0.30303	0.29521	0.27510	0.24815	0.24522	0.21903	0.22096	0.19770	0.20928
$\rho ({\mathit{a}}_2,{\widehat{\mathit{a}}}_2^c)$	0.91167	0.92137	0.91977	0.91112	0.91556	0.90693	0.88127	0.84600	0.82650
$\rho ({\mathit{b}}_2,{\widehat{\mathit{b}}}_2^c)$	0.88049	0.89166	0.88662	0.88904	0.89241	0.88801	0.87192	0.83197	0.80588
$P_2(\widehat{{\mathit{a}}_2},{\widehat{\mathit{a}}}_2^c)$	0.41815	0.38347	0.36114	0.32231	0.31958	0.29261	0.33041	0.32611	0.36974
$P_2(\widehat{{\mathit{b}}_2},{\widehat{\mathit{b}}}_2^c)$	0.44509	0.41272	0.40340	0.36543	0.36662	0.33627	0.36107	0.34812	0.36182
$L_2(\widehat{{\mathit{a}}_2},{\widehat{\mathit{a}}}_2^c)$	0.56221	0.51686	0.48609	0.43221	0.42597	0.38524	0.42792	0.41586	0.46555
$L_2(\widehat{{\mathit{b}}_2},{\widehat{\mathit{b}}}_2^c)$	0.54952	0.50631	0.48727	0.43321	0.43065	0.38858	0.41934	0.39996	0.41901
10% contamination
$\rho ({\widehat{\mathit{a}}}_2^c,{\widehat{\mathit{b}}}_2^c)$	0.31085	0.29651	0.28563	0.28324	0.25836	0.24660	0.25507	0.23689	0.22360
$\rho ({\mathit{a}}_2,{\widehat{\mathit{a}}}_2^c)$	0.76633	0.76784	0.77030	0.77828	0.75962	0.77061	0.78303	0.75727	0.71201
$\rho ({\mathit{b}}_2,{\widehat{\mathit{a}}}_2^c)$	0.77035	0.75135	0.75242	0.75593	0.74394	0.75508	0.74693	0.74538	0.70453
$P_2(\widehat{{\mathit{a}}_2},{\widehat{\mathit{a}}}_2^c)$	0.69844	0.67264	0.64760	0.62480	0.58957	0.56764	0.57146	0.56234	0.60946
$P_2(\widehat{{\mathit{b}}_2},{\widehat{\mathit{b}}}_2^c)$	0.70333	0.68232	0.65851	0.63732	0.60797	0.57351	0.55445	0.54736	0.53981
$L_2(\widehat{{\mathit{a}}_2},{\widehat{\mathit{a}}}_2^c)$	0.94522	0.91180	0.87884	0.84254	0.79453	0.76052	0.75866	0.73627	0.79154
$L_2(\widehat{{\mathit{b}}_2},{\widehat{\mathit{b}}}_2^c)$	0.87344	0.84137	0.80061	0.76733	0.72179	0.67148	0.64907	0.64158	0.63347
15% contamination
$\rho ({\widehat{\mathit{a}}}_2^c,{\widehat{\mathit{b}}}_2^c)$	0.29041	0.26604	0.25341	0.24867	0.22370	0.22678	0.22668	0.21698	0.20791
$\rho ({\widehat{\mathit{a}}}_2,{\widehat{\mathit{b}}}_2^c)$	0.62670	0.62511	0.62775	0.63798	0.63931	0.67347	0.66306	0.63881	0.61437
$\rho ({\widehat{\mathit{b}}}_2,{\widehat{\mathit{b}}}_2^c)$	0.67113	0.65281	0.63298	0.64345	0.64183	0.66804	0.66294	0.66655	0.64393
$P_2(\widehat{{\mathit{a}}_2},{\widehat{\mathit{a}}}_2^c)$	0.86840	0.83480	0.80722	0.78144	0.73384	0.71580	0.71970	0.73208	0.73667
$P_2(\widehat{{\mathit{b}}_2},{\widehat{\mathit{b}}}_2^c)$	0.83973	0.81076	0.78283	0.74695	0.69787	0.68081	0.65889	0.63941	0.61576
$L_2(\widehat{{\mathit{a}}_2},{\widehat{\mathit{a}}}_2^c)$	1.17901	1.13476	1.09703	1.06197	0.99294	0.96654	0.96359	0.97065	0.96509
$L_2(\widehat{{\mathit{b}}_2},{\widehat{\mathit{b}}}_2^c)$	1.04721	1.00181	0.95442	0.89936	0.83020	0.80219	0.77413	0.74676	0.72249
20% contamination
$\rho ({\widehat{\mathit{a}}}_2^c,{\widehat{\mathit{b}}}_2^c)$	0.23981	0.23457	0.22347	0.22122	0.21361	0.20732	0.20959	0.19139	0.20603
$\rho ({\widehat{\mathit{a}}}_2,{\widehat{\mathit{b}}}_2^c)$	0.51483	0.51794	0.52276	0.53520	0.54956	0.53813	0.56693	0.54726	0.56153
$\rho ({\widehat{\mathit{b}}}_2,{\widehat{\mathit{b}}}_2^c)$	0.62369	0.59115	0.56066	0.56134	0.58636	0.58110	0.60681	0.61363	0.63613
$P_2(\widehat{{\mathit{a}}_2},{\widehat{\mathit{a}}}_2^c)$	0.94170	0.92377	0.91740	0.90923	0.86834	0.86758	0.83873	0.82330	0.83708
$P_2(\widehat{{\mathit{b}}_2},{\widehat{\mathit{b}}}_2^c)$	0.89672	0.87587	0.85688	0.82781	0.78143	0.75837	0.72490	0.70158	0.69711
$L_2(\widehat{{\mathit{a}}_2},{\widehat{\mathit{a}}}_2^c)$	0.47541	0.46413	0.46168	0.45820	0.43798	0.43988	0.43203	0.42330	0.43539
$L_2(\widehat{{\mathit{b}}_2},{\widehat{\mathit{b}}}_2^c)$	0.76644	0.73624	0.72641	0.69977	0.67411	0.63579	0.61011	0.61020	0.59094

present all performance measures for the RPCCA method over a grid of tuning parameters ranging from 0 (corresponding to ICCA) to $0.8.$ All methods perform suitably well in terms of accuracy, achieving high absolute correlations between true and estimated canonical variables, even under contaminated scenarios. However, the linear relationship in the first component is captured worse by the ICCA in the presence of contamination, as shown by the lower absolute correlations between the canonical variables, $\rho ({\widehat{\mathit{a}}}_1^c,{\widehat{\mathit{b}}}_1^c).$ Moreover, the RPCCA method with positive values of the tuning parameter produces more stable estimations of the canonical vectors, having smaller $P_2$ and $L_2$ distances between the uncontaminated and contaminated estimated canonical vectors in both components, $(\widehat{{\mathit{b}}_1},{\widehat{\mathit{b}}}_1^c)$ and $(\widehat{{\mathit{b}}_2},{\widehat{\mathit{b}}}_2^c),$ thus demonstrating the advantage in terms of robustness. Although the differences in performance are not impressive, the gain in robustness with very little loss of accuracy with respect to the ICCA makes the RPCCA very attractive.

On the other hand, if the underlying relationship is easily identified, the proposed robust RPCCA performs as good as the ICCA under pure data and outperforms the ICCA in the presence of contamination (Table 1). However, for $\tau >0$, the loss in accuracy in the relationship identification under pure data would be unavoidable (although not very significant); hence, the tuning parameter should be chosen sufficiently close to zero (from the literature, less than 1) to provide an adequate compromise between efficiency loss and robustness gain. Moderate values of the tuning parameter, around 0.3, offer the best compromise producing canonical estimators that are robust against data contamination with a small loss of efficiency with respect to the ICCA in the absence of contamination.

6.3. Real Data Application

We finally illustrate the applicability of our method with real-life data on the heredity of head shape in men. For such a purpose, we use a well-known dataset from Frets [43] that collects the head length and head breadth for the first and second sons for $n=25$ families. Then, the first and second set of variables, $\mathit{X}$ and $\mathit{Y}$, respectively, have the dimension 2 and represent the head length and head breadth of the corresponding son. From the dataset, we want to analyze whether there is a relationship between the head shape among male offspring. The data have been widely used in the literature, and Mardia et al. [2] and Yin [12] analyzed the canonical correlations between the first and second sons’ head shapes using CCA and ICCA, respectively. In their analyses, they found one significant pair of canonical variables with a strong linear relationship. Figure 2 shows the plots of the first (left) and second (right) pair of canonical variables for the head data estimated by RPCCA with $\tau =0$ (top) and $\tau =0.5$ (bottom). As shown, both methods coincide on the first pair of canonical variables (estimate the same observations for the first pair of canonical variables), $x_1$ and $y_1,$ having linear correlation coefficients of $\rho =78.67\%$ ($\tau =0$) and $\rho =76.86\%$ ($\tau =0.5$) as illustrated on the corresponding plots. For the second pair of canonical variables, none of the methods find any clear functional relationship between linear combinations of the variables, and the two procedures considered estimate very different canonical variables without a clear functional relationship between them (as shown in Figure 2). Thus, we also conclude that there is only one pair of canonical variables.

Additionally, to illustrate the advantage of our method in terms of robustness (with a small loss of deficiency), we contaminate a single observation (obs. 24) in both vector variables, generating an outlying observation. Then, we apply RPCCA at $\tau =0$ (corresponding to ICCA) and $\tau =0.5$ with the uncontaminated and the contaminated data.

Table 3. $P_2$ and $N_2$ distances between the estimated canonical vectors under uncontaminated and contaminated data.

$\mathit{\tau }$	0	0.5
$P_2(\widehat{{\mathit{a}}_1},{\widehat{\mathit{a}}}_1^c)$	0.222	0.064
$P_2(\widehat{{\mathit{b}}_1},{\widehat{\mathit{b}}}_1^c)$	0.131	0.059
$L_2(\widehat{{\mathit{a}}_1},{\widehat{\mathit{a}}}_1^c)$	0.224	0.064
$L_2(\widehat{{\mathit{b}}_1},{\widehat{\mathit{b}}}_1^c)$	1.995	0.059

presents $P_2$ and $N_2$ distances between the first pair of canonical vectors (identifying the linear relationship) estimated under uncontaminated and contaminated data, with only one outlying observation. Because the sample size is small, an outlying observation heavily influences the ICCA estimation, whereas the RPCCA method with $\tau =0.5$ shows a great stability in the canonical vector estimation. These results illustrate the advantage of the RPCCA in real-life applications, producing robust estimates of the canonical variables with a small loss of efficiency with respect to the ICCA estimation in the absence of data contamination.

7. Conclusions

We have presented a robust generalization of the ICCA based on RP for identifying linear and non-linear relationships between two sets of variables. We have derived sample versions for estimating the canonical vectors in practice, and we have demonstrated the consistency of such estimators. Further, the robustness advantage of the RPCCA has been examined theoretically and empirically, concluding that the proposed RPCCA offers an appealing alternative to ICCA, competitive in terms of estimation accuracy and more robust against data contamination. The method manages to detect hidden functional relationships between linear combinations of the variables and suitably approximates the true underlying relationships, even under contaminated scenarios. Moreover, a permutation test for determining the number of significant pairs of canonical vectors is presented. Since the RPCCA is a parametric family, a data-driven algorithm for determining optimal values of the tuning parameter is a worthwhile pursuit for future research. Also, the methodology presented here can be extended in future works for identifying relationships between more than two sets of variables. The idea is to consider not only two random vectors but k random vectors as considered in [24] and to look for the linear combinations in all of them so that the RP between the marginal distributions and the whole distribution is as large as possible.