Multivariate Mixed Response Model with Pairwise Composite-Likelihood Method

Abstract

In clinical research, study outcomes usually consist of various patients’ information corresponding to the treatment. To have a better understanding of the effects of different treatments, one often needs to analyze multiple clinical outcomes simultaneously, while the data are usually mixed with both continuous and discrete variables. We propose the multivariate mixed response model to implement statistical inference based on the conditional grouped continuous model through a pairwise composite-likelihood approach. It can simplify the multivariate model by dealing with three types of bivariate models and incorporating the asymptotical properties of the composite likelihood via the Godambe information. We demonstrate the validity and the statistic power of the multivariate mixed response model through simulation studies and clinical applications. This composite-likelihood method is advantageous for statistical inference on correlated multivariate mixed outcomes.

1. Introduction

Clinical research, such as toxicity studies and laboratory examinations, can provide relevant information for measuring the effect of various treatments or experiments on the patients. This type of research needs to jointly analyze different experimental outcomes, while the research outcomes collected during the treatment are correlated and mixed with both categorical and continuous variables. For example, we are studying the efficacy of treatments along with the toxicity and adverse drug reactions simultaneously. In this case, the severity level could be measured as discrete or ordinal data, while the clinical examination results such as the blood test measures are continuous. In the traditional approach, these multiple outcomes are analyzed by different linear models to estimate the effects of the treatments together with the relevant clinical and demographic information. However, this approach ignores the correlation between the outcomes and only provides marginal inferences. Thus, it is desirable to develop a multivariate approach, which can jointly model the multiple mixed-type responses with the treatment and clinical covariates.

In the recent literature, there are two main approaches to build a joint model for mixed response variables. The conditional Gaussian distribution model (CGDM) can decompose the joint distribution of mixed response variables into a combination of the conditional distribution and the marginal distribution. In the bivariate mixed case, the conditional Gaussian distribution model can produce a conditional distribution of the categorical variables given the continuous variables and marginal distribution for the continuous variables. In particular, Cox [1] provided the logistic conditional distribution for binary variables, and Cox and Wermuth [2] extended the model with a probit-type function and showed the potential connection to the latent variable model. Another conditional Gaussian distribution model, referred to as the general location model (GLOM), was proposed by Olkin and Tate [3]. They adopted the opposite factorization, which consists of a conditional normal distribution given the categorical variables and marginal multinomial distribution. Teixeira-Pinto and Normand [4] compared this approach with the models proposed by Sammel et al. [5,6] in a comprehensive review. Yang et al. [7] extended the model to mixed Poisson and continuous response variables through a likelihood-based approach.

The grouped continuous model (GCM) proposed by Anderson and Pemberton [8] and de Leon [9] provides another solution for this problem. The fundamental technique allows the categorical variables to be treated as partitioned continuous latent variables with different nonoverlapping intervals (Poon and Lee [10]; Skrondal and Rabe-Hesketh [11]). This type of transformation allows the latent variables to follow a multivariate Gaussian distribution. Poon and Lee [10] demonstrated maximum likelihood estimation for latent variables with polychoric correlations. As an extension of the grouped continuous model, de Leon [9] proposed the conditional grouped continuous model (CGCM) to build a joint model for the variables mixed with categorical and continuous outcomes. Catalano and Ryan [12], Catalano [13], and Najita et al. [14] applied the conditional grouped continuous model to the studies of fetal toxicity for longitudinal data. Gueorguieva and Agresti [15] proposed that the estimation of correlated mixed response variables can be obtained by the expectation–maximization (EM) algorithm. Zhang et al. [16] extended this to the parameter expanded EM algorithm under the full-likelihood approach. de Leon and Carriére [17,18] developed a general mixed-data model, which aggregates the CGDM and CGCM to jointly analyze correlated nominal, ordinal, and continuous data together.

It remains computationally challenging to estimate the joint distribution of multivariate mixed data. The composite-likelihood method is based on compounded lower-dimensional distributions, which offers an alternative solution to the estimation problem (Lindsay [19], Cox and Reid [20], Varin [21], Varin et al. [22], and Reid et al. [23]). Faes et al. [24] applied this method for longitudinal data with mixed outcomes. In their model, the correlation structure is induced by the random effect, which does not have a closed-form expression. We aim to follow the approach of the CGCM and use the composite-likelihood method to analyze the joint distribution of multivariate mixed-type response variables, where the categorical response variables are modeled by continuous latent variables. The parameters of the mean structure, as well as the correlation among different outcomes, can be estimated simultaneously through the numerical algorithm. The proposed composite-likelihood method consists of three types of bivariate joint densities: two continuous outcomes; two discrete outcomes modeled by two continuous latent variables; and two mixed outcomes with one continuous and one categorical variable.

We provide the numerical algorithm for composite-likelihood estimation and discuss the asymptotical properties of the composite-likelihood estimates. In addition, we derive three composite-likelihood test statistics for joint inference on the multivariate mixed response model. Simulation studies were conducted to examine the empirical performance of the proposed method in comparison with the conventional approaches. The algorithm was applied to the clinical data from a colorectal cancer study. We analyze the effect of the treatment and other clinical factors’ on multiple correlated responses of the patients.

2. Methodology

2.1. Model Setup

Suppose there are n observations ${\mathit{z}}_1,{\mathit{z}}_2,\cdots ,{\mathit{z}}_i,\cdots ,{\mathit{z}}_n$ in a clinical dataset, and each observation contains q multiple outcomes ${\mathit{z}}_i={(z_{i1},z_{i2},\cdots ,z_{ij},\cdots ,z_{iq})}^T$, which are correlated and mixed with continuous and binary variables. Suppose we wish to model the effects of a collection of covariates, and the generalized linear model can be constructed for each outcome as

$$\begin{array}{c}\hfill g_j\left(E\left(z_{ij}\right)\right)={\mathit{x}}_{ij}^T{\mathit{\beta }}_j,\end{array}$$

in which the covariate ${\mathit{x}}_{ij}$ with $i=1,2,\cdots ,n$ and $j=1,2,\cdots ,q$ for different responses can be the same or different, and $g_j$ denotes the link function used for the jth response. If we want to analyze these multivariate mixed-type outcomes simultaneously with the corresponding covariates, the conventional likelihood-based approach needs to identify the multivariate joint densities of all responses. Alternatively, we can set up the pairwise likelihood function for responses $z_{ij}$ and $z_{ik}$ as

$$\begin{array}{c}\hfill L_{jk}\left({\mathit{\theta }}_{jk}\right)=\prod _{i=1}^{n}f(z_{ij},z_{ik}),\end{array}$$

(1)

where ${\mathit{\theta }}_{jk}$ denotes the parameters associated with the pairwise likelihood. The log-likelihood function is given by $l_{jk}\left({\mathit{\theta }}_{jk}\right)=log{L}_{jk}\left({\mathit{\theta }}_{jk}\right)$, and the score function is given by

$$\begin{array}{c}\hfill U_{jk}\left({\mathit{\theta }}_{jk}\right)=\sum _{i=1}^{n}f{(z_{ij},z_{ik})}^{-1}\frac{\partial }{\partial {\mathit{\theta }}_{jk}}f(z_{ij},z_{ik}).\end{array}$$

(2)

According to Lindsay [19] and Cox and Reid [20], the pairwise composite-likelihood function of these q response variables is the product of $\left(\genfrac{}{}{0pt}{}{q}{2}\right)$ paired likelihood functions:

$$\begin{array}{c}\hfill \mathrm{CL}\left(\mathit{\theta }\right)=\prod _{j=1}^{q-1}\prod _{k=j+1}^q{L}_{jk}\left({\mathit{\theta }}_{jk}\right),\end{array}$$

and the score function is constructed by defferentiating the composite log-likelihood function:

$$\begin{array}{c}\hfill U\left(\mathit{\theta }\right)=\frac{\partial }{\partial \mathit{\theta }}log\mathrm{CL}\left(\mathit{\theta }\right).\end{array}$$

(3)

Our multivariate mixed response model estimates the parameters of interest $\mathit{\theta }$ by numerically solving the composite score Function (3) equal to 0 through the Newton–Raphson [25] method. Since the outcomes are mixed with continuous and categorical variables, the score Function (2) can be derived under three different bivariate structures: outcomes with both continuous response variables, outcomes with both binary response variables, and outcomes mixed with one continuous and one binary response variable.

2.1.1. Case 1: Continuous Outcomes

We first discuss the case that both responses $z_{ij}$ and $z_{ik}$ are continuous and assume they follow a bivariate normal distribution:

$$\begin{array}{c}\left(\begin{array}{c}{z}_{ij}\\ z_{ik}\end{array}\right)\sim N_2\left(\left(\begin{array}{c}{\mu }_{ij}\\ {\mu }_{ik}\end{array}\right),\left(\begin{array}{cc}{\sigma }_j^2,& {\rho }_{jk}{\sigma }_j{\sigma }_k\\ {\rho }_{jk}{\sigma }_j{\sigma }_k,& {\sigma }_k^2\end{array}\right)\right),\hfill \end{array}$$

where the mean structure of $z_{ij}$ and $z_{ik}$ is associated with the generalized linear models as ${\mu }_{ij}={\mathit{x}}_{ij}^T{\mathit{\beta }}_j$ and ${\mu }_{ik}={\mathit{x}}_{ik}^T{\mathit{\beta }}_k$. Thus, the pairwise density of $z_{ij}$ and $z_{ik}$ is the same as a bivariate normal density based on Johnson and Wichern [26]:

$$\begin{array}{cc}\hfill f(z_{ij},z_{ik})=& \frac{1}{2\pi {\sigma }_j{\sigma }_k\sqrt{1-{\rho }_{jk}^2}}exp[-\frac{1}{2(1-{\rho }_{jk}^2)}\times \hfill \\ \hfill & \left(\frac{{(z_{ij}-{\mu }_{ij})}^2}{{\sigma }_j^2}-\frac{2\rho (z_{ij}-{\mu }_{ij})(z_{ik}-{\mu }_{ik})}{{\sigma }_j{\sigma }_k}+\frac{{(z_{ik}-{\mu }_{ik})}^2}{{\sigma }_k^2}\right)].\hfill \end{array}$$

The pairwise composite-likelihood function can be constructed by

$$\begin{array}{c}\hfill L_{jk}\left({\mathit{\theta }}_{jk}\right)=\prod _{i=1}^{n}f(z_{ij},z_{ik}),\end{array}$$

with the parameters of interest ${\mathit{\theta }}_{jk}=\{{\mathit{\beta }}_j,{\mathit{\beta }}_k,{\sigma }_j,{\sigma }_k,{\rho }_{jk}\}$, and the log-likelihood function $l_{jk}\left({\mathit{\theta }}_{jk}\right)={\sum }_{i=1}^{n}logf(z_{ij},z_{ik})$, which gives the score function of both continuous responses used in the Equation (2):

$$\begin{array}{ccc}\hfill U_{jk}\left({\mathit{\theta }}_{jk}\right)& =& \frac{\partial l_{jk}\left({\mathit{\theta }}_{jk}\right)}{\partial {\mathit{\theta }}_{jk}}.\hfill \end{array}$$

(4)

2.1.2. Case 2: Binary Outcomes

For binary outcomes, the transformation can be obtained using the threshold value t such that if the latent variable $z_{ij}^{*}\ge t$, then the response $z_{ij}=1$, otherwise $z_{ij}=0$. Without loss of generality, the value of t is set to 0, and we have ${\mu }_{ij}=P(z_{ij}=1)=P(z_{ij}^{*}\ge 0)$. We follow the grouped continuous model settings with the distributional assumptions $z_{ij}\sim Bernoulli\left({\mu }_{ij}\right)$ and $z_{ij}^{*}\sim N({\mu }_{ij}^{*},{\sigma }_j^2)$. Thus, the model can be constructed with the covariates ${\mathit{x}}_{ij}$ as

$$\begin{array}{c}\hfill probit\left({\mu }_{ij}\right)={\mathit{x}}_{ij}^T{\mathit{\beta }}_j.\end{array}$$

Since the mean ${\mu }_{ij}^{*}$ and the variance ${\sigma }_j^2$ are not identifiable at the same time, we follow the rescaling method based on Dunson [27] by setting ${\sigma }_j^2=1$. Therefore, the mean of the latent variable can be simplified as ${\mu }_{ij}^{*}=x_{ij}^T{\mathit{\beta }}_j$.

Since we jointly analyze binary variables $z_{ij}$ and $z_{ik}$, the paired latent variables $z_{ij}^{*}$ and $z_{ik}^{*}$ are generated from a model with the polychoric correlation. Thus, the joint function of $(z_{ij},z_{ik})$ can be derived by following composite-likelihood functions in four cases

$$\begin{array}{ccc}\hfill P(z_{ij}=1,z_{ik}=1)& =& P(z_{ij}^{*}\ge 0,z_{ik}^{*}\ge 0)={\Phi }_2({\mu }_{ij}^{*},{\mu }_{ik}^{*},{\rho }_{jk}),\hfill \\ \hfill P(z_{ij}=1,z_{ik}=0)& =& P(z_{ij}^{*}\ge 0,z_{ik}^{*}<0)={\Phi }_2({\mu }_{ij}^{*},-{\mu }_{ik}^{*},-{\rho }_{jk}),\hfill \\ \hfill P(z_{ij}=0,z_{ik}=1)& =& P(z_{ij}^{*}<0,z_{ik}^{*}\ge 0)={\Phi }_2(-{\mu }_{ij}^{*},{\mu }_{ik}^{*},-{\rho }_{jk}),\hfill \\ \hfill P(z_{ij}=0,z_{ik}=0)& =& P(z_{ij}^{*}<0,z_{ik}^{*}<0)={\Phi }_2(-{\mu }_{ij}^{*},-{\mu }_{ik}^{*},{\rho }_{jk}),\hfill \end{array}$$

where $\pm {\rho }_{jk}$ represents the polychoric correlation under different scenarios and ${\Phi }_2$ denotes the bivariate normal cumulative density function. The equations above can be rewritten as

$$\begin{array}{ccc}\hfill P(z_{ij},z_{ik})& =& {\Phi }_2((2z_{ij}-1){\mu }_{ij}^{*},(2z_{ik}-1){\mu }_{ik}^{*},(2z_{ij}-1)(2z_{ik}-1){\rho }_{jk})\hfill \\ \hfill & =& {\Phi }_2(s_{ij}{\mu }_{ij}^{*},s_{ik}{\mu }_{ik}^{*},s_{ij}{s}_{ik}{\rho }_{jk}),\hfill \end{array}$$

with $s_{ij}=(2z_{ij}-1)$ and $s_{ik}=(2z_{ik}-1)$. The log-likelihood function of paired binary responses is

$$\begin{array}{c}\hfill l_{jk}\left({\mathit{\theta }}_{jk}\right)=\sum _{i=1}^{n}log{\Phi }_2(s_{ij}{\mu }_{ij}^{*},s_{ik}{\mu }_{ik}^{*},s_{ij}{s}_{ik}{\rho }_{jk}).\end{array}$$

The score function can be derived as

$$\begin{array}{c}\hfill U_{jk}\left({\mathit{\theta }}_{jk}\right)=\sum _{i=1}^{n}P{(z_{ij},z_{ik})}^{-1}\frac{\partial }{\partial {\mathit{\theta }}_{jk}}logP(z_{ij},z_{ik}).\end{array}$$

(5)

2.1.3. Case 3: Mixed Outcomes

When one response variable ${\mathit{z}}_{ij}$ is binary, and another response variable ${\mathit{z}}_{ik}$ is continuous, we can model the mixed outcomes together by adopting a latent normal variable $z_{ij}^{*}$ as the conditional grouped continuous model. Thus, they follow a bivariate normal distribution given the polyserial correlation ${\rho }_{jk}$:

$$\begin{array}{c}\hfill \left(\begin{array}{c}{z}_{ij}^{*}\\ z_{ik}\end{array}\right)\sim N_2\left(\left(\begin{array}{c}{\mu }_{ij}^{*}\\ {\mu }_{ik}\end{array}\right),\left(\begin{array}{cc}{\sigma }_j^2,& {\rho }_{jk}{\sigma }_j{\sigma }_k\\ {\rho }_{jk}{\sigma }_j{\sigma }_k,& {\sigma }_k^2\end{array}\right)\right).\end{array}$$

The factorization method can be applied for the joint bivariate normal distribution with a marginal density for the continuous response $P\left(z_{ik}\right)$ and a conditional density for the latent variable $P\left(z_{ij}^{*}\right|z_{ik})$ given the continuous response.

• $z_{ik}$ is the continuous response

  $$\begin{array}{c}{z}_{ik}\sim N({\mu }_{ik},{\sigma }_k^2);\end{array}$$
• $z_{ij}^{*}$ is the latent variable for the binary response $z_{ij}$:

  $$\begin{array}{c}\hfill z_{ij}^{*}|z_{ik}\sim N\left({\mu }_{ij|k}={\mu }_{ij}^{*}+{\rho }_{jk}\frac{{\sigma }_j}{{\sigma }_k}(z_{ik}-{\mu }_{ik}),{\sigma }_{j|k}^2={\sigma }_j^2(1-{\rho }_{jk}^2)\right).\end{array}$$

Let $d_{ij}=-\frac{{\mu }_{ij|k}}{{\sigma }_{j|k}}$, the conditional probability $P\left(z_{ij}^{*}\right|z_{ik})$ can be written as

$$\begin{array}{c}\hfill P\left(z_{ij}^{*}\right|z_{ik})=\left\{\begin{array}{cc}\Phi \left(d_{ij}\right),\hfill & \mathrm{if}{z}_{ij}=0,\hfill \\ \Phi (-d_{ij}),\hfill & \mathrm{if}{z}_{ij}=1.\hfill \end{array}\right.\end{array}$$

The pairwise likelihood function and log-likelihood function is given as follows:

$$\begin{array}{cc}\hfill & L_{jk}=\prod _{i=1}^{n}P\left(z_{ij}^{*}\right|z_{ik})p\left(z_{ik}\right),\hfill \\ \hfill & l_{jk}=\sum _{i=1}^{n}logP\left(z_{ik}\right)+log\mathbf{1}(z_{ij}=0)\Phi \left(d_{ij}\right)+log\mathbf{1}(z_{ij}=1)\Phi (-d_{ij}).\hfill \end{array}$$

The score function can be obtained as

$$\begin{array}{c}\hfill U_{jk}=\sum _{i=1}^n\frac{1}{P\left(z_{ik}\right)}\frac{\partial P\left(z_{ik}\right)}{\partial {\mathit{\theta }}_{jk}}+\left[\frac{\mathbf{1}(z_{ij}=0)}{\Phi \left(d_{ij}\right)}-\frac{\mathbf{1}(z_{ij}=1)}{1-\Phi \left(d_{ij}\right)}\right]\frac{\partial \Phi \left(d_{ij}\right)}{\partial {\mathit{\theta }}_{jk}}.\end{array}$$

(6)

A similar formulation was extended to the longitudinal settings by Najita et al. [14]. More derivation details are included in Appendix A.

The model setting above can simplify the score Function (3) with three types of components as in Equations (4)–(6). We can obtain the maximum composite-likelihood estimate $\widehat{\mathit{\theta }}$ (MCLE) for the parameters $\mathit{\theta }$ by solving the score function numerically. Our numerical algorithm is presented below (Algorithm 1):

Algorithm 1: Multivariate Mixed Response Model Algorithm.

1. Select the initial values ${\mathit{\theta }}^{\left(0\right)}$ of the parameters. Calculate the score function $U\left({\mathit{\theta }}^{\left(0\right)}\right)$ in Equation (3) and the Hessian matrix ${\nabla }_{\theta }U\left({\mathit{\theta }}^{\left(0\right)}\right)$ as the derivative of the score function;  2. Implement the Newton–Raphson method to obtain the updated parameters ${\mathit{\theta }}^{\left(t\right)}$ with $t=0,1,\cdots$; $${\mathit{\theta }}^{(t+1)}={\mathit{\theta }}^{\left(t\right)}-{\left[{\nabla }_{\theta }U\left({\mathit{\theta }}^{\left(t\right)}\right)\right]}^{-1}U\left({\mathit{\theta }}^{\left(t\right)}\right);$$  3. Calculate the score function and Hessian matrix with the updated parameters ${\mathit{\theta }}^{(t+1)}$;  4. Repeat step 2 and 3 until convergence.

2.2. Statistical Inference Using the Composite-Likelihood Method

Cox and Hinkley [28] and Kent [29] presented various hypothesis testing procedures using the full-likelihood function. The composite-likelihood function can be treated as the misspecified-likelihood function. Its asymptotic properties were reviewed and discussed by Varin et al. [22], Reid et al. [23], Jin [30], and Gao and Song [31]. Following this framework, the pairwise composite-likelihood function implemented in our proposed model produces the estimators, which are consistent and asymptotically normally distributed.

The Godambe information [32] G of the parameters $\mathit{\theta }$ for the log composite-likelihood function involves the sensitivity matrix H and the variability matrix J:

$$G\left(\mathit{\theta }\right)=H\left(\mathit{\theta }\right)J^{-1}\left(\mathit{\theta }\right)H\left(\mathit{\theta }\right),$$

where the sensitivity matrix and variability matrix are defined as

$$\begin{array}{c}\hfill H\left(\mathit{\theta }\right)=E_{\theta }\{-{\nabla }_{\theta }U(\mathit{\theta };{\mathit{z}}_i)\}\mathrm{and}J\left(\mathit{\theta }\right)=Va{r}_{\theta }\left\{U(\mathit{\theta };{\mathit{z}}_i)\right\},\end{array}$$

where $U(\mathit{\theta };{\mathit{z}}_i)$ denotes the score function of the ith observation and the total score function $U\left(\mathit{\theta }\right)={\sum }_{i=1}^{n}U(\mathit{\theta };{\mathit{z}}_i)$.

Theorem 1.

Let ${\mathit{\theta }}_0\in {\Re }^d$ denote the true parameter value to set up the multivariate mixed response model. Under regularity conditions, $n\to \infty$, the maximum composite-likelihood estimator $\widehat{\theta }$ is asymptotically normally distributed as

$$\sqrt{n}(\widehat{\mathit{\theta }}-{\mathit{\theta }}_0)\stackrel{d}{\to }{N}_d(\mathbf{0},G^{-1}).$$

Theorem 2.

Under regularity conditions, $n\to \infty$, the maximum composite-likelihood estimator $\widehat{\mathit{\theta }}$ is consistent to ${\mathit{\theta }}_0$ satisfying $\sqrt{n}(\widehat{\mathit{\theta }}-{\mathit{\theta }}_0)=O_p\left(1\right)$.

The sensitivity matrix H and the variability matrix J can be evaluated by the empirical estimates under the maximum composite-likelihood estimators:

$$\begin{array}{c}\hfill H\left(\widehat{\mathit{\theta }}\right)=-\frac{1}{n}\sum _{i=1}^n{\nabla }_{\theta }U(\mathit{\theta };{\mathit{z}}_i){|}_{\widehat{\theta }}\mathrm{and}J\left(\widehat{\mathit{\theta }}\right)=\frac{1}{n}\sum _{i=1}^{n}U(\mathit{\theta };{\mathit{z}}_i)U{(\mathit{\theta };{\mathit{z}}_i)}^T.\end{array}$$

Furthermore, according to Theorem 1, the composite Wald statistic, the composite score statistic, and the composite-likelihood ratio statistic for testing the null hypothesis $H_0$: $\mathit{\theta }={\mathit{\theta }}_0$ are given respectively by

$$\begin{array}{cc}\hfill & W_e=n(\widehat{\mathit{\theta }}-{\mathit{\theta }}_0)G(\widehat{\mathit{\theta }}-{\mathit{\theta }}_0)\hfill \\ \hfill & W_u=n^{-1}U\left({\mathit{\theta }}_0\right)J^{-1}\left(\widehat{\mathit{\theta }}\right)U\left({\mathit{\theta }}_0\right),\hfill \\ \hfill & W=2\{log\mathrm{CL}\left(\widehat{\mathit{\theta }}\right)-log\mathrm{CL}\left({\mathit{\theta }}_0\right)\}.\hfill \end{array}$$

Testing with Nuisance Parameters

Suppose the parameters are partitioned as $\mathit{\theta }=\{\psi ,\lambda \}$ with $\psi \in {\Re }^s$, $\lambda \in {\Re }^p$, and $d=s+p$. The parameter of interest is $\psi$, and $\lambda$ is treated as the nuisance parameter for hypothesis testing. In this setting, the Godambe information matrix and its inverse can be partitioned as

$$\begin{array}{c}\hfill G=\left[\begin{array}{cc}{G}_{\psi \psi }& G_{\psi \lambda }\\ G_{\lambda \psi }& G_{\lambda \lambda }\end{array}\right]\mathrm{and}{G}^{-1}=\left[\begin{array}{cc}{G}^{\psi \psi }& G^{\psi \lambda }\\ G^{\lambda \psi }& G^{\lambda \lambda }\end{array}\right],\end{array}$$

and the inverse of the submatrix pertaining to $\psi$ is given by $G_{\psi \psi ,\lambda }={\left(G^{\psi \psi }\right)}^{-1}=G_{\psi \psi }-G_{\psi \lambda }{G}_{\lambda \lambda }^{-1}{G}_{\lambda \psi }$. According to the asymptotic theorem, the composite Wald statistics under the null hypothesis $H_0$: $\psi ={\psi }_0$ using $\widehat{\lambda }\left({\psi }_0\right)$ is given by

$$\begin{array}{c}\hfill W_e\left({\psi }_0\right)=n(\widehat{\psi }-{\psi }_0)G_{\psi \psi ,\lambda }(\widehat{\psi }-{\psi }_0),\end{array}$$

which has an asymptotic ${\chi }_q^2$ distribution. Similarly, we define the composite score statistics:

$$\begin{array}{c}\hfill W_u\left({\psi }_0\right)=n^{-1}U({\psi }_0,\widehat{\lambda }\left({\psi }_0\right))H^{\psi \psi }{G}_{\psi \psi ,\lambda }{H}^{\psi \psi }U({\psi }_0,\widehat{\lambda }\left({\psi }_0\right)),\end{array}$$

where the matrix $H^{\psi \psi }$ can be obtained by partitioning the inverse sensitivity matrix H. Furthermore, the composite-likelihood ratio statistic can be obtained by

$$\begin{array}{c}\hfill W\left({\psi }_0\right)=2\{log\mathrm{CL}(\widehat{\psi },\widehat{\lambda })-log\mathrm{CL}({\psi }_0,\widehat{\lambda }\left({\psi }_0\right))\},\end{array}$$

with the unrestricted maximum composite-likelihood estimate $\widehat{\mathit{\theta }}=\{\widehat{\psi },\widehat{\lambda }\}$. However, the asymptotic distribution of the composite-likelihood ratio under $H_0$ is given by ${\sum }_{j=1}^q{\lambda }_j{\chi }_{1\left(j\right)}^2$, where ${\chi }_{1\left(j\right)}^2$ are independent ${\chi }_1^2$ variates and ${\lambda }_1,{\lambda }_2\cdots ,{\lambda }_q$ are the eigenvalues of the matrix $H_{\psi \psi ,\lambda }{G}^{\psi \psi }$ with $H_{\psi \psi ,\lambda }=H_{\psi \psi }-H_{\psi \lambda }{H}_{\lambda \lambda }^{-1}{H}_{\lambda \psi }$. There are different adjustments to this nonstandard weighted chi-square distribution (Rotnitzky and Jewell [33], Geys et al. [34], and Pace et al. [35]). For example, we can apply the adjustment by introducing the scaling factor $\overline{\lambda }={\sum }_{j=1}^q{\lambda }_j/q$, then the adjusted composite-likelihood ratio has the same asymptotic distribution as $W_e\left({\psi }_0\right)$ and $W_u\left({\psi }_0\right)$:

$$\begin{array}{c}\hfill \frac{W}{\overline{\lambda }}\stackrel{d}{\to }{\chi }_s^2.\end{array}$$

(7)

Therefore, the composite-likelihood method can simplify the modeling of correlated responses with multiple generalized linear models and allow users to conduct statistic inferences on parameters of interest from different generalized linear models. Moreover, we can select a subset of the parameters and conduct the further inferential assessment in the presence of nuisance parameters.

3. Simulation

Different simulation studies were implemented to show the validity of the multivariate mixed response model. The estimation results from the proposed model are compared with the full-likelihood and marginal approaches, respectively.

3.1. Comparison with Maximum Full-Likelihood Estimation

In the multivariate regression with correlated continuous outcomes, the full-likelihood estimation can be conducted without numerical integration. Thus, we can compare the maximum composite-likelihood estimates with the full-likelihood approach through the simulation study. The simulated samples contain four continuous response variables $z_{i{c}_1},z_{i{c}_2},z_{i{c}_3},$ and $z_{i{c}_4}$, which are generated from Equation (8):

$$\begin{array}{c}\hfill \begin{array}{c}\hfill z_{i{c}_1}={\alpha }_{c_1}+{\beta }_{c_1}{\mathit{x}}_{i{c}_1}+{\gamma }_{c_1}{\mathit{y}}_{i{c}_1}+{\epsilon }_{i{c}_1},\\ \hfill z_{i{c}_2}={\alpha }_{c_2}+{\beta }_{c_2}{\mathit{x}}_{i{c}_2}+{\gamma }_{c_2}{\mathit{y}}_{i{c}_2}+{\epsilon }_{i{c}_2},\\ \hfill z_{i{c}_3}={\alpha }_{c_1}+{\beta }_{c_1}{\mathit{x}}_{i{c}_1}+{\gamma }_{c_1}{\mathit{y}}_{i{c}_1}+{\epsilon }_{i{c}_1},\\ \hfill z_{i{c}_4}={\alpha }_{c_2}+{\beta }_{c_2}{\mathit{x}}_{i{c}_2}+{\gamma }_{c_2}{\mathit{y}}_{i{c}_2}+{\epsilon }_{i{c}_2}.\end{array}\end{array}$$

(8)

The covariates $\{{\mathit{x}}_{i{c}_1},{\mathit{x}}_{i{c}_2},{\mathit{x}}_{i{c}_1},{\mathit{x}}_{i{c}_2}\}\sim N(0,1)$ and $\{{\mathit{y}}_{i{c}_1},{\mathit{y}}_{i{c}_2},{\mathit{y}}_{i{c}_1},{\mathit{y}}_{i{c}_2}\}\sim N(0,0.5)$ are independently simulated. The errors are correlated and generated from a multivariate normal distribution $N_4(\mathbf{0},\Sigma )$, and the variance–covariance matrix $\mathbf{\Sigma }$ is given by

$$\begin{array}{c}\hfill \left[\begin{array}{cccc}{\sigma }_{c_1}^2& {\sigma }_{c_1}{\sigma }_{c_2}{\rho }_{c_1{c}_2}& {\sigma }_{c_1}{\sigma }_{c_3}{\rho }_{c_1{c}_3}& {\sigma }_{c_1}{\sigma }_{c_4}{\rho }_{c_1{c}_4}\\ {\sigma }_{c_1}{\sigma }_{c_2}{\rho }_{c_1{c}_2}& {\sigma }_{c_2}^2& {\sigma }_{c_2}{\sigma }_{c_3}{\rho }_{c_2{c}_4}& {\sigma }_{c_2}{\sigma }_{c_4}{\rho }_{c_2{c}_4}\\ {\sigma }_{c_1}{\sigma }_{c_3}{\rho }_{c_1{c}_3}& {\sigma }_{c_2}{\sigma }_{c_3}{\rho }_{c_2{c}_3}& {\sigma }_{c_3}^2& {\sigma }_{c_3}{\sigma }_{c_4}{\rho }_{c_3{c}_4}\\ {\sigma }_{c_1}{\sigma }_{c_4}{\rho }_{c_1{c}_4}& {\sigma }_{c_2}{\sigma }_{c_4}{\rho }_{c_2{c}_4}& {\sigma }_{c_3}{\sigma }_{c_4}{\rho }_{c_3{c}_4}& {\sigma }_{c_4}^2\end{array}\right].\end{array}$$

In the simulation, the variances are designed as ${\sigma }_{c_1}^2=1$, ${\sigma }_{c_2}^2=1$, ${\sigma }_{c_3}^2=2.25$, and ${\sigma }_{c_3}^2=4$, and an identical correlation $\rho =0.3$ is applied between the errors in the data generating process.

The simulation results (Figure 1) were obtained through 1000 independent replications. The maximum composite-likelihood estimators demonstrate similar performance when compared with the full-likelihood approach. The simulated results also show that the estimates are close to each other, and the maximum likelihood estimators have slightly higher relative efficiency.

3.2. Comparison with the Marginal Approach

For the mixed outcome regression, the full-likelihood approach is computationally challenging, and marginal regression is often resorted to in order to conduct the analysis. We implemented simulation studies to evaluate the performance of our proposed method in comparison with marginal regression. We first tested the overall performance of the point estimates when the outcomes had different levels of dependency and covariates. Next, we focused on the test of the composite statistics. The multivariate mixed response model can provide the statistical inference with nuisance parameters and attains a higher statistical power in terms of dealing with joint inference.

3.2.1. Simulation Settings

We generated the sample data consisting of two binary responses $z_{i{b}_1}$ and $z_{i{b}_2}$ and two continuous responses $z_{i{c}_1}$ and $z_{i{c}_2}$. The binary variables were obtained based on the corresponding latent normal variables $z_{i{b}_1}^{*}$ and $z_{i{b}_2}^{*}$ through the probit link function:

$$\begin{array}{c}\hfill probit\left({\mu }_{i{b}_1}\right)={\mu }_{i{b}_1}^{*},\\ \hfill probit\left({\mu }_{i{b}_2}\right)={\mu }_{i{b}_2}^{*}.\end{array}$$

The simulation studies of the responses are based on Equation (9) associated with covariates ${\mathit{x}}_i=\{{\mathit{x}}_{i{b}_1},{\mathit{x}}_{i{b}_2},{\mathit{x}}_{i{c}_1},{\mathit{x}}_{i{c}_2}\}$ and ${\mathit{y}}_i=\{{\mathit{y}}_{i{b}_1},{\mathit{y}}_{i{b}_2},{\mathit{y}}_{i{c}_1},{\mathit{y}}_{i{c}_2}\}$, respectively:

$$\begin{array}{c}\hfill \begin{array}{c}\hfill z_{i{b}_1}^{*}={\alpha }_{b_1}+{\beta }_{b_1}{\mathit{x}}_{i{b}_1}+{\gamma }_{b_1}{\mathit{y}}_{i{b}_1}+{\epsilon }_{i{b}_1},\\ \hfill z_{i{b}_2}^{*}={\alpha }_{b_2}+{\beta }_{b_2}{\mathit{x}}_{i{b}_2}+{\gamma }_{b_2}{\mathit{y}}_{i{b}_2}+{\epsilon }_{i{b}_2},\\ \hfill z_{i{c}_1}={\alpha }_{c_1}+{\beta }_{c_1}{\mathit{x}}_{i{c}_1}+{\gamma }_{c_1}{\mathit{y}}_{i{c}_1}+{\epsilon }_{i{c}_1},\\ \hfill z_{i{c}_2}={\alpha }_{c_2}+{\beta }_{c_2}{\mathit{x}}_{i{c}_2}+{\gamma }_{c_2}{\mathit{y}}_{i{c}_2}+{\epsilon }_{i{c}_2}.\end{array}\end{array}$$

(9)

We provided different simulation scenarios of the covariates values and three levels of correlation to analyze the response variables with the proposed model. The regression parameters were arbitrarily chosen and set to be fixed values in each simulation study. The errors in Equation (9) follow a multivariate normal distribution $N_4(\mathbf{0},\mathbf{\Sigma })$, and the variance-covariance matrix $\mathbf{\Sigma }$ is given by

$$\begin{array}{c}\hfill \left[\begin{array}{cccc}{\sigma }_{b_1}^2& {\sigma }_{b_1}{\sigma }_{b_2}{\rho }_{b_1{b}_2}& {\sigma }_{b_1}{\sigma }_{c_1}{\rho }_{b_1{c}_1}& {\sigma }_{b_1}{\sigma }_{c_2}{\rho }_{b_1{c}_2}\\ {\sigma }_{b_1}{\sigma }_{b_2}{\rho }_{b_1{b}_2}& {\sigma }_{b_2}^2& {\sigma }_{b_2}{\sigma }_{c_1}{\rho }_{b_2{c}_1}& {\sigma }_{b_2}{\sigma }_{c_2}{\rho }_{b_2{c}_2}\\ {\sigma }_{b_1}{\sigma }_{c_1}{\rho }_{b_1{c}_1}& {\sigma }_{b_2}{\sigma }_{c_1}{\rho }_{b_2{c}_1}& {\sigma }_{c_1}^2& {\sigma }_{c_1}{\sigma }_{c_2}{\rho }_{c_1{c}_2}\\ {\sigma }_{b_1}{\sigma }_{c_2}{\rho }_{b_1{c}_2}& {\sigma }_{b_2}{\sigma }_{c_2}{\rho }_{b_2{c}_2}& {\sigma }_{c_1}{\sigma }_{c_2}{\rho }_{c_1{c}_2}& {\sigma }_{c_2}^2\end{array}\right].\end{array}$$

In the following data generating processes, the values of the variance-covariance parameters are set as ${\sigma }_{b_1}^2=1$, ${\sigma }_{b_2}^2=1$, ${\sigma }_{c_1}^2=16$, and ${\sigma }_{c_2}^2=25$, and the correlation is designed at the levels of low (all $\rho =0.3$), medium (all $\rho =0.5$), and high (all $\rho =0.7$), respectively, to assess the underlying model. Since there is no constraint on the sign of the correlation, the negative correlation can be estimated through our algorithm without further assumptions. Our simulation studies focus on the overall performance of the multivariate mixed response model through independent replications.

3.2.2. Point Estimates

Different simulation scenarios were designed to assess the performance of the underlying model on the point estimates by 1000 independent replications. There are two different sets of simulations for the data generating process, and within each setting, we analyze three levels of correlation, respectively. As shown in

Table 1. The ratio of the mean squared error (MSE) of the multivariate mixed response model (MMR) to the marginal model (GLM). Results based on 1000 independent simulation under two different scenarios and three different levels of correlation.

	Simulation I *			Simulation II †
	Low	Med	High	Low	Med	High
${\alpha }_{b_1}=0.2$	1.00	0.99	0.98	0.97	0.92	0.85
${\beta }_{b_1}=0.3$	0.93	0.81	0.66	1.00	0.99	0.97
${\gamma }_{b_1}=0.3$	0.93	0.81	0.66	0.95	0.85	0.71
${\alpha }_{b_2}=0.2$	1.00	0.98	0.97	0.97	0.92	0.86
${\beta }_{b_2}=0.3$	0.94	0.84	0.69	1.00	0.98	0.95
${\gamma }_{b_2}=0.5$	0.94	0.83	0.70	0.95	0.86	0.71
${\alpha }_{c_1}=0.5$	1.00	1.00	1.00	0.96	0.89	0.79
${\beta }_{c_1}=$ 8	0.89	0.73	0.50	1.00	1.00	1.00
${\gamma }_{c_1}=$ 10	0.90	0.74	0.51	0.93	0.80	0.59
${\sigma }_{c_1}=$ 4	1.01	1.01	1.01	1.01	1.01	1.01
${\alpha }_{c_2}=0.4$	1.00	1.00	1.00	0.97	0.90	0.79
${\beta }_{c_2}=$ 5	0.92	0.77	0.53	1.00	1.00	1.00
${\gamma }_{c_2}=$ 8	0.92	0.75	0.50	0.94	0.80	0.57
${\sigma }_{c_2}=$ 5	1.01	1.01	1.01	1.01	1.01	1.01

* Simulation I: n = 300, and the four responses have different covariates, e.g., ${\mathit{x}}_i$ and ${\mathit{y}}_i$ generated from a normal distribution $N(0,1)$; ^† Simulation II: n = 1000, the responses shared one common covariate ${\mathit{x}}_i$ ∼ $N(0,1)$ and added a different covariate ${\mathit{y}}_i$ ∼ Bernoulli $\left(0.5\right)$.

, the values of the regression parameters and the standard deviation of the continuous response variables are given across all simulation studies. In the first simulation setting, we provide 300 samples, and the mixed response variables are associated with covariates of distinct values. The covariate sets of ${\mathit{x}}_i$ and ${\mathit{y}}_i$ are identically and independently simulated from a normal distribution $N(0,1)$, respectively, in each linear model.

In the second simulation setting, we present the multivariate mixed response model dealing with the common covariate. Regarding the data generating process with 1000 samples, all responses share one common covariate, which was generated from a normal distribution $N(0,1)$, such as ${\mathit{x}}_{i{b}_1}={\mathit{x}}_{i{b}_2}={\mathit{x}}_{i{c}_1}={\mathit{x}}_{i{c}_2}$ in Equation (9). The second covariates ${\mathit{y}}_i$ are from a Bernoulli (0.5), and they are different for each response. This setting represents the scenario in practice when a common factor is included in all of the response models.

In Table 1, we provide the ratio of the mean squared error (MSE) of the proposed method to the marginal approaches. This ratio represents the relative efficiency of the proposed method in comparison with the marginal method under different settings. In most of the simulation settings, the ratio rates of the MSE are well below 1. When the responses are highly correlated and have different covariate sets, our method can reduce MSE by 50%, which indicates a large efficiency gain.

3.2.3. Statistical Test

The test of composite-likelihood statistics can jointly assess the parameters of interest across different generalized linear models, while the conventional methods cannot achieve this. The simulation studies were conducted to measure the type I error rate and the power in comparison with the marginal approaches.

This simulation study was conducted to perform the hypothesis test. The correlated responses were generated based on Equation (9) with all correlation $\rho =0.3$. The parameters of interest are the regression coefficients $\{{\beta }_{b_1},{\beta }_{b_2},{\beta }_{c_1},{\beta }_{c_2}\}$ of the first covariates across four generalized linear models, and the first covariates ${\mathit{x}}_i$ are independently simulated from $N(0,1)$. The regression coefficients of the second covariates ${\mathit{y}}_i$ and other parameters are nuisance parameters with ${\mathit{y}}_i\sim N(0,.5)$. In the simulation study to assess the type 1 error rate, the regression parameters $\{{\beta }_{b_1},{\beta }_{b_2},{\beta }_{c_1},{\beta }_{c_2}\}$ are equal to zero in all generalized linear models, while other parameters have the same values as the previous simulation in Table 1. To assess the power, we fixed the values of the regression parameters as ${\beta }_{b_1}={\beta }_{b_2}=0.1$ for the binary responses and ${\beta }_{c_1}={\beta }_{c_2}=0.3$ for the continuous responses. In comparison, we combined the results of the marginal approaches by applying the Bonferroni adjustment.

Table 2. Type 1 error rate and power under different sample sizes ($N=500$ and $N=1000$).

	Type I Error		Power
	$\mathit{N}\mathbf{=}\mathbf{500}$	$\mathit{N}\mathbf{=}\mathbf{1000}$	$\mathit{N}\mathbf{=}\mathbf{500}$	$\mathit{N}\mathbf{=}\mathbf{1000}$
	Composite-Likelihood Method
	$H_0:{\beta }_{b_1}={\beta }_{b_2}={\beta }_{c_1}={\beta }_{c_2}=0$
Likelihood ratio	0.054	0.043	0.804	0.988
Wald statistics	0.058	0.043	0.800	0.989
Scoring statistics	0.058	0.042	0.798	0.989
	Multiple Test
Bonferroni test	0.051	0.040	0.569	0.902

The likelihood ratio statistic is adjusted as Equation (7), which approximates to a ${\chi }_4^2$.

illustrates the results over 2000 independent replications. The proposed model analyzes all responses simultaneously, and the simulated type I error rates are valid and close to the $0.05$. Through the test of the joint effect of the covariate of interest on all responses, the simulated power is enhanced by our proposed model in comparison with the results from the Bonferroni test. As the sample size increases from 500 to 1000, the composite-likelihood statistics produce increased statistical power from approximately $0.800$ to $0.989$. The overall performance demonstrates that the composite statistics are more powerful than the conventional approach.

4. Data Analysis

In this section, the multivariate mixed response model is applied to the clinical data from a colorectal cancer study. The data consist of clinical observations and demographic information on 743 patients, which are mixed with both categorical and continuous data. Our research interest was to evaluate the effect of treatment and other clinical factors on the toxicity outcomes. We focused on four common toxicity events that are related to colorectal cancer treatment. First, we chose nausea and diarrhea as two categorical responses. They are made up of ordinal data measuring the severity of the toxicity from grade 1 to 4. In our model setting, we only considered the occurrence of nausea and diarrhea for each patient. Therefore, these two responses were designed as binary variables, which are coded as 1 if they occurred and 0 if there is no record during the treatment. The continuous responses include two blood test measures, namely the hemoglobin (HGB) count and the white blood cell (WBC) count. Each patient had several blood examinations during the treatment, and we took the highest value for analysis. The explanatory variables contain the treatment effect, demographic information, tumor status, and genetic markers for each patient. There are two different treatment therapies in this colorectal cancer study. The patient demographic and clinical information, such as age, height, and weight, were collected as continuous variables. The tumor identified in either the colon or rectum was recorded as a binary variable. The study also includes the genetic markers, such as PERF1, PERF2, and KRAS, which are binary variables showing the occurrence of mutation. In total, we needed to jointly estimate 68 parameters for the coefficients of four linear models and the correlation between each outcome.

Table 3. The difference in the effect of treatment between the two treatment therapies. GLM: the generalized linear model; MMR: the multivariate mixed response model.

Regression Parameter	Models
	GLM	MMR
${\mathit{z}}_{b_1}$: occurrence of nausea
Intercept ${\alpha }_{b_1}$	$-0.2685\pm 2.502$	$-0.2793\pm 2.582$
(p value)	(0.833)	(0.832)
Treatment effect ${\beta }_{b_1}$	$-0.2644\pm 0.190$	$-0.2724\pm 0.193$
(p value)	(0.006)	(0.006)
${\mathit{z}}_{b_2}$: occurrence of diarrhea
Intercept ${\alpha }_{b_2}$	$0.6631\pm 2.557$	$0.6741\pm 2.605$
(p value)	(0.611)	(0.612)
Treatment effect ${\beta }_{b_2}$	$-0.6231\pm 0.192$	$-0.6422\pm 0.198$
(p value)	(<0.001)	(<0.001)
${\mathit{z}}_{c_1}$: measures of hemoglobin
Intercept ${\alpha }_{c_1}$	$160.3758\pm 32.989$	$160.3775\pm 32.984$
(p value)	(<0.001)	(<0.001)
Treatment effect ${\beta }_{c_1}$	$-12.492\pm 2.498$	$-12.496\pm 2.454$
(p value)	(<0.001)	(<0.001)
${\mathit{z}}_{c_2}$: measures of white blood cell
Intercept ${\alpha }_{c_2}$	$12.295\pm 9.331$	$12.2946\pm 9.515$
(p value)	(0.010)	(0.011)
Treatment effect ${\beta }_{c_2}$	$-0.1591\pm 0.706$	$-0.1597\pm 0.702$
(p value)	(0.659)	(0.656)

shows the main result of the effect of the treatment, and the complete result is presented in Appendix B. We can observe that the statistical inference on the effect of the treatment through two models was in agreement. The second treatment therapy resulted in lower measures of the hemoglobin and indicates a negative association with the occurrence of nausea and diarrhea, whereas, the effect difference on the measures of white blood cells is insignificant. We can use the composite statistics to jointly assess the overall effect of this therapy on four responses.

Table 4. Estimation Results II: the estimated parameters contain second moments of each outcome.

	Esimated Correlation				Estimated Standard Deviation
	Nausea	Diarrhea	HGB	WBC
Nausea	1.0000	0.3954	0.0736	0.0899	-
Diarrhea		1.0000	0.0351	−0.0126	-
HGB			1.0000	0.0139	16.796
WBC				1.0000	4.7507

provides the standard deviation and the correlation of the four clinical outcomes estimated based on the proposed model.

Using the conventional approach, we cannot make a statistic inference across different linear models. The proposed model is able to test the hypothesis $H_0:{\beta }_{b_1}={\beta }_{b_2}={\beta }_{c_1}={\beta }_{c_2}=0$ based on the asymptotical properties of the composite-likelihood function. The test statistics of the composite Wald statistics under the $H_0$ is approximately $138.5890$, the composite score statistics is $476.975$, and the adjusted composite-likelihood ratio is $264.3069$, which are all greater than the critical value of ${\chi }_4^2$. Therefore, we can reject the null hypothesis and conclude that the two different treatments have a statistically significant difference in terms of patient toxicity response. More specifically, in our estimation results, we infer that there exists a significant difference in terms of the occurrence of nausea and diarrhea and a significant difference in HGB between the two treatments.

5. Discussion

The problem of mixed outcomes is widely discussed in health-related studies. As a result of the computational complexity, most existing methods mainly focus on the case of two outcomes mixed with one continuous variable and one categorical variable. As an extension of the conditional grouped continuous model, we present the multivariate mixed response model to solve high-dimensional mixed multivariate regressions. Our model is constructed using the pairwise composite-likelihood method, such that multiple outcomes are analyzed through different bivariate models simultaneously. Regarding data mixed with continuous and binary responses, our method simplifies the problem of multiple outcomes into three types of scenario, which is both methodologically flexible and analytic appealing. From the simulation studies, the estimators of the proposed model demonstrate a lower MSE than the marginal approaches. The composite statistics also provide increased statistical power for joint hypothesis testing across different generalized linear models, which could make this a favorable approach to analyze clinical data with multiple mixed-type responses.

In addition, the model can be generalized to deal with ordinal and continuous data simultaneously. Under the same setup, the latent variable $z_{ij}^{*}$ is normally distributed with mean ${\mu }_{ij}^{*}={\mathit{x}}_{ij}^T{\mathit{\beta }}_j$ and variance ${\sigma }_j^2=1$. If the latent normal variable $z_{ij}^{*}\in [t_{l-1},t_l)$ with the threshold values $-\infty =t_0<t_1<\cdots <t_l<\cdots <t_{L-1}<t_L=\infty$, the ordinal variable $z_{ij}=l$ with $l=1,2,\cdots ,L$. Thus, the bivariate model of two ordinal outcomes can be formulated by the pairwise probability:

$$\begin{array}{c}\hfill P(z_{ij}=l,z_{ik}=l^{{}^{\prime }})=P(z_{ij}^{*}\in [t_{l-1},t_l),z_{ik}^{*}\in [t_{l^{{}^{\prime }}-1},t_{l^{{}^{\prime }}}))={\int }_{t_{l-1}}^{t_l}{\int }_{t_{l^{{}^{\prime }}-1}}^{t_{l^{{}^{\prime }}}}{\varphi }_2({\omega }_{ij},{\omega }_{ik};{\mu }_{ij}^{*},{\mu }_{ik}^{*},{\rho }_{jk})d{\omega }_{ij}d{\omega }_{ik}.\end{array}$$

The score function can be obtained by taking the derivatives of ${\sum }_{i=1}^n{\sum }_{j=1}^{q-1}{\sum }_{k=j+1}^{q}logP(z_{ij}=l,z_{ik}=l^{{}^{\prime }})$, and the thresholds $t_l$’s need to be estimated with the monotone restriction. The maximum composite-likelihood estimation for mixed outcomes can be conducted through the same approaches as proposed in this paper. Further research will be considered to analyze multivariate outcomes with various distributions.