A Bayesian EAP-Based Nonlinear Extension of Croon and Van Veldhoven’s Model for Analyzing Data from Micro–Macro Multilevel Designs

Abstract

Croon and van Veldhoven discussed a model for analyzing micro–macro multilevel designs in which a variable measured at the upper level is predicted by an explanatory variable that is measured at the lower level. Additionally, the authors proposed an approach for estimating this model. In their approach, estimation is carried out by running a regression analysis on Bayesian Expected a Posterior (EAP) estimates. In this article, we present an extension of this approach to interaction and quadratic effects of explanatory variables. Specifically, we define the Bayesian EAPs, discuss a way for estimating them, and we show how their estimates can be used to obtain the interaction and the quadratic effects. We present the results of a “proof of concept” via Monte Carlo simulation, which we conducted to validate our approach and to compare two resampling procedures for obtaining standard errors. Finally, we discuss limitations of our proposed extended Bayesian EAP-based approach.

1. Introduction

In organizational research, one may be interested in which factors determine an organization or team’s outcome, such as the team’s productivity in terms of the number of sales. For a general framework, see Schneider et al. [1]. The research conducted for the purpose of studying these factors typically face a data structure in which employees are nested in organizations/teams. To analyze such data, multilevel analysis is often used. The term “multilevel analysis” subsumes various approaches, some of which were developed in parallel in very different disciplines, such as economics, psychology, and education sciences. The multilevel analysis involves the use of hierarchical models, models with mixed effects, or multilevel models, terms that are largely used synonymously in the statistical literature. In the following, we will use the term “multilevel model”. Over the past 20 years, these models have established themselves as the standard for analyzing multilevel data, mainly because of their usefulness to examine cross-level interactions and the availability of free software that has been developed and complemented commercial software for performing multilevel analyses. Moreover, a number of influential textbooks have been published (e.g., [2,3,4,5,6]) that make it easier to get started with multilevel analysis.

A research design can be considered a multilevel design if at least one of the variables has a multilevel structure. Following this view and focusing on one level of clustering (i.e., two-level designs), designs can then roughly be differentiated according to whether the dependent variable is measured at the lower or the upper level. Among the designs in which the dependent variable is measured at the upper level, the micro–macro design [6] stands out because it is of great relevance in organizational research. In this design, the dependent variable is predicted by a variable measured at the lower level (e.g., [7]; see also [8]). Croon and van Veldhoven [9] proposed an approach for analyzing data from micro–macro designs. An important feature of their approach is that estimation is carried out by an ordinary least squares regression analysis with Bayesian Expected a Posterior (EAP) estimates as input. Almost one decade after Croon and van Veldhoven [9] published their article, there has been renewed interest in the use of this type of estimation. Despite the criticism of its lack of flexibility [10], the method offers simple computations for an otherwise computationally very demanding class of models. For example, Zitzmann [11] emphasized the method’s favorable statistical properties and showed how the method can be used to estimate multiple indicator two-level measurement models. Zitzmann and Helm [12] extended the method in such a way that it can be used for performing two-level mediation and moderation analyses. Aydin et al. [13] and Aydin et al. [14] discussed the necessity of an extension to three-level models, and Aydin and Algina [15] proposed such an extension.

In this article, we present an extension of Croon and van Veldhoven’s approach for analyzing data from micro–macro designs to interaction and quadratic effects of explanatory variables. Specifically, (a) we define the Bayesian EAPs, (b) present an efficient way for estimating them, and (c) we explain how their estimates can be used to obtain the effects in the model. Moreover, (d) we conduct a “proof of concept” Monte Carlo study in which we also compare the performance of a standard bootstrap procedure for obtaining standard errors with the jackknife proposed by Zitzmann [11] jackknife. Finally, (e) we discuss one possibility to extend the model and a limitation of our proposed method.

2. Nonlinear Extension of Croon and Van Veldhoven’s Model

We begin with the dependent variable, which is measured at the upper level. This may be a measure of productivity, such as a team’s productivity Y. The explanatory variable, on the other hand, is measured at the lower level. For example, it is possible to assess a characteristic of the team leader such as leadership behavior by his or her employees’ perceptions $X_1$ of the team leader’s capacity to set a vision and goals and to support the team to achieve them. To this end, questionnaires or interviews are typically used [9]. One efficient way of assessing the leadership behavior of their team leader is ask employees to rate this characteristic and then average the ratings across the employees in the team [16]. This mean is assumed to reflect the employees’ shared perception of their team leader and should ideally be formed by a latent instead of the common manifest aggregation procedure (see [17] for a detailed discussion). The latent mean ${\xi }_1$ varies between teams but not across employees in a team, whereas the individual deviations ${\delta }_1$ from this mean vary across the employees within a team. Thus, the measurement model for a team leader’s behavior in terms of his or her employees’ shared perception can be formulated as:

$$X_{1ij}={\xi }_{1j}+{\delta }_{1ij}$$

(1)

for an employee $i=1,\dots ,n$ in a team $j=1,\dots ,J$. A central assumption is that employees from the same team differ less in their perceptions from each other than employees from different teams, which implies that the perceptions within a team are not independent of one another. This dependency can be quantified by the Intraclass Correlation (ICC). Formally, the ICC quantifies the proportion of the total variance that is located between the teams, and it can take on values between 0 and 1.

To investigate how leadership behavior ${\xi }_1$ is related to team productivity Y (controlling for a covariate Z), the following regression model can be used, which was proposed by Croon and van Veldhoven [9]:

$$Y_j=\alpha +{\beta }_1{\xi }_{1j}+\gamma Z_j+{\epsilon }_j$$

(2)

where ${\beta }_1$ denotes the effect of the explanatory variable of interest, which describes the effect of the leadership behavior on team productivity. $\gamma$ is the coefficient of the covariate. The effect of the leadership behavior indicates by how many units the productivity of the team will increase on average when the leadership behavior is increased by one unit, while the covariate is kept constant. The covariate is a characteristic that is measured directly at the upper level. One may consider the personality of the team leader as a covariate, for example. The team leader’s personality can be measured by asking the team leader rather than his or her employees.

To extend the model, we consider an additional explanatory variable, which may be a team’s climate. Similar to leadership behavior, this variable can be assessed by employees’ perceptions $X_2$. The latent mean ${\xi }_2$ across the employees in the team reflects the employees’ shared perception of the team, and thus, the measurement model for a team’s climate reads similar to the measurement model in Equation (1). Both explanatory variables may be assumed to interact in such a way that a positive team climate enlarges the effect of leadership behavior. To complete the nonlinear specification, it is further assumed that the explanatory variables have quadratic effects. The resulting model takes the form of the prototype for simultaneously estimating interaction and quadratic effects [18]. The model is expressed by the following regression. For better readability, we omit the employee and team indices:

$$Y=\alpha +{\beta }_1{\xi }_1+{\beta }_2{\xi }_2+{\beta }_3{\xi }_1{\xi }_2+{\beta }_4{\xi }_1^2+{\beta }_5{\xi }_2^2+\gamma Z+\epsilon$$

(3)

The coefficient ${\beta }_3$ denotes the interaction effect of the two explanatory variables, and ${\beta }_4$ and ${\beta }_5$ are their quadratic effects. The interaction effect indicates that the strength and perhaps even the direction of the relation between team productivity and leadership behavior is moderated by the team climate. In a similar vein, the quadratic effects of leadership behavior and team climate describe how the relations of the team productivity with leadership behavior and team climate depend on the levels of leadership behavior and team climate, respectively. Next, we give a detailed description of how this extended nonlinear model can be estimated with the help of Bayesian EAP estimates.

3. Bayesian EAP-Based Model Estimation

Technically speaking, the measurement model in Equation (1) can also be interpreted as decomposing each employee’s perception into two orthogonal or uncorrelated parts: the latent mean across the perceptions of the employees of a team regarding their team leader’s behavior and an individual deviation from this mean. Thus, the model allows the quantification of the differences in employees’ perceptions between and within the teams by two variance components, which we hereafter call the between variance and the within variance.

3.1. Definition of (Adjusted) EAPs

To formally define a Bayesian EAP, we first make some distributional assumptions. Specifically, we assume the employees’ perceptions $X_1$ to have a grand mean of zero (centering) to facilitate the presentation and later estimation. Moreover, we assume that the latent mean ${\xi }_1$ across the perceptions of the employees of the team follows a normal distribution:

$${\xi }_{1j}\sim \mathrm{N}\left(0,{\sigma }_{{\xi }_1}^2\right)$$

(4)

where ∼ reads “is distributed as”, $\mathrm{N}\left(·,·\right)$ stands for the Gaussian or (univariate) normal or distribution, and ${\sigma }_{{\xi }_1}^2$ is the between variance of the employee’s perceptions. The individual deviations ${\delta }_1$ from each latent mean are normally distributed around 0 with a within variance of ${\sigma }_{{\delta }_1}^2$. As the deviations’ variance is the same for all teams, homoscedasticity is inherently assumed.

The latent mean across the perceptions of the employees of the jth team corresponds with the observed mean ${\overline{X}}_{1•j}={\sum }_{i=1}^n{X}_{1ij}/n$, where n is the number of employees surveyed in the team. Note that n does not vary across teams; that is, the same number of employees is surveyed in each team (balanced data). Thus, the team’s likelihood function is:

$${\overline{X}}_{1•j}\sim \mathrm{N}\left({\xi }_{1j},{\sigma }_{{\delta }_1}^2/n\right)$$

(5)

From a Bayesian perspective, Equation (4) can be viewed as a Bayesian prior distribution [19]. As mentioned in other articles of this Special Issue, Bayesian estimates can be obtained from the posterior distribution, which results from combining the prior with the likelihood (i.e., from Bayes’ theorem; [20]). Now, if we combine the prior in Equation (4) with the likelihood in Equation (5), we yield the following (team-specific) normal posterior:

$${\xi }_{1j}\sim \mathrm{N}\left(\frac{{\sigma }_{{\xi }_1}^2}{\mathrm{var}\left({\overline{X}}_{1•j}\right)}{\overline{X}}_{1•j},\frac{{\sigma }_{{\xi }_1}^2}{\mathrm{var}\left({\overline{X}}_{1•j}\right)}{\sigma }_{{\delta }_1}^2/n\right)$$

(6)

where $\mathrm{var}\left(·\right)$ denotes the variance (thus, $\mathrm{var}\left({\overline{X}}_{1•j}\right)$ denotes the variance of the observed mean across the employees’ perceptions). For detailed information about the derivation of this equation, see Appendix A. The mean of this posterior

$${\overline{\xi }}_{1j}=\underset{=\mathit{\omega }}{\underbrace{\frac{{\sigma }_{{\xi }_1}^2}{\mathrm{var}\left({\overline{X}}_{1•j}\right)}}}{\overline{X}}_{1•j}$$

(7)

is called the EAP of the latent mean. $\mathit{\omega }$ is the weighting factor, which can be interpreted as the reliability of the observed counterpart of the latent mean across the perceptions of the employees of a team. The fact that the EAP is the mean of the posterior can be further substantiated by noticing that the setup is similar to the simple normal model in which Equation (4) is the prior for the variable’s mean, and Equation (5) is the likelihood (see also [19] p. 134). Although we made explicit use of the Bayes’ theorem in the definition of the EAP, we will skip the word “Bayesian” for the sake of simplicity. It is also interesting to note that from a non-Bayesian perspective, as indicated by Equation (7), the EAP is simply the prediction from a regression of the latent on the observed mean [21], and this is why the weight can be analytically derived via the ordinary least squares principle.

In models such as Croon and van Veldhoven’s model and our extended nonlinear model, effects can be biased when EAP estimates are used as input in ordinary least squares regression analyses [22]. Therefore, it is necessary to adjust EAPs for other variables in the model. The model in Equation (2) contains also the team leader’s personality Z. Thus, an adjustment needs to be made to the EAP of the latent mean across the perceptions of the employees of a team. The adjusted EAP can be expressed as follows:

$${\overline{\xi }}_{1j}=\underset{=\mathit{\omega }}{\underbrace{\frac{{\sigma }_{{\xi }_1}^2-{\beta }^2\mathrm{var}\left(Z_j\right)}{\mathrm{var}\left({\overline{X}}_{1•j}\right)-{\beta }^2\mathrm{var}\left(Z_j\right)}}}{\overline{X}}_{1•j}+\left(1-\frac{{\sigma }_{{\xi }_1}^2-{\beta }^2\mathrm{var}\left(Z_j\right)}{\mathrm{var}\left({\overline{X}}_{1•j}\right)-{\beta }^2\mathrm{var}\left(Z_j\right)}\right)\beta Z_j$$

(8)

where $\beta$ is the result of a regression of ${\overline{X}}_{1•j}$ on Z. In this equation, $\mathit{\omega }$ is the conditional reliability (i.e., conditioned on the covariate; [23]) (see Appendix B for detailed information about the derivation). Because the nonlinear model in Equation (3) contains the team climate as another explanatory variable, this variable also needs to be taken into account. Thus, one way to express the adjusted EAP is:

$${\overline{\xi }}_{1j}=\left({\overline{X}}_{1•j},{\overline{X}}_{2•j},Z_j\right)\underset{=\mathit{\omega }}{\underbrace{{\mathrm{var}}^{-1}\left({\overline{X}}_{1•j},{\overline{X}}_{2•j},Z_j\right){\left({\sigma }_{{\xi }_1}^2,{\sigma }_{{\xi }_1{\xi }_2},{\sigma }_{{\xi }_{1}Z}\right)}^T}}$$

(9)

where ${\overline{X}}_{2•j}={\sum }_{i=1}^n{X}_{2ij}/n$ is the observed mean across the perceptions of the employees of a team regarding the team’s climate, ${\sigma }_{{\xi }_1{\xi }_2}$ is the between covariance of the employee’s perceptions $X_1$ and $X_2$, and ${\sigma }_{{\xi }_{1}Z}$ is the between covariance of $X_1$ and Z. ${\mathrm{var}}^{-1}\left(·\right)$ gives the inverted covariance matrix, T is the transpose operator, and $\mathit{\omega }$ is a vector containing the weights of ${\overline{X}}_{1•j}$, ${\overline{X}}_{2•j}$, and $Z_j$. The equation indicates once more that the EAP is the prediction from a regression and that the weights are the coefficients in this regression and can thus be derived by ordinary least squares. To adjust the EAP of the latent mean across the perceptions of the employees of a team regarding the team’s climate, an analogous adjustment is made.

What is more interesting is how an adjusted EAP can be defined for the product of the latent means in Equation (3). To tackle this problem, it is instructive to consider one of its solutions in single-level models. Here, products of indicators from two explanatory variables are formed and used as derived “product indicators” of a latent variable whose effect is the interaction effect (i.e., the product indicator approach; e.g., [24,25,26]). Moreover, scholars have pointed out that the measurement models in our extended nonlinear model (see Equation (1)) can be viewed as describing the relation between a latent variable (i.e., the latent mean) and its indicators (i.e., the perceptions of the employees of a team; [27]). This similarity suggests that the EAP of the product of the two latent means can be defined as the prediction from a regression on the observed mean across the products of the employees’ perceptions $X_1$ of their team leaders and $X_2$ of their team, which is given as ${\overline{X_1{X}_2}}_{•j}={\sum }_{i=1}^n{X}_{1ij}{X}_{2ij}/n$.

Because the model in Equation (3) includes also the quadratic effects, an adjustment for the squares of the latent means needs to be made. The adjusted EAP is:

$${\overline{{\xi }_1{\xi }_2}}_j=\left({\overline{X_1{X}_2}}_{•j},{\overline{X_1^2}}_{•j},{\overline{X_2^2}}_{•j}\right)\underset{=\mathit{\omega }}{\underbrace{{\mathrm{var}}^{-1}\left({\overline{X_1{X}_2}}_{•j},{\overline{X_1^2}}_{•j},{\overline{X_2^2}}_{•j}\right){\left({\sigma }_{{\xi }_1}^2{\sigma }_{{\xi }_2}^2+{\sigma }_{{\xi }_1{\xi }_2}^2,2{\sigma }_{{\xi }_1}^2{\sigma }_{{\xi }_1{\xi }_2},2{\sigma }_{{\xi }_2}^2{\sigma }_{{\xi }_1{\xi }_2}\right)}^T}}$$

(10)

where ${\overline{X_1^2}}_{•j}={\sum }_{i=1}^n{X}_{1ij}^2/n$ and ${\overline{X_2^2}}_{•j}={\sum }_{i=1}^n{X}_{2ij}^2/n$. In this equation, the variance of the product of the latent means and the covariances between this product and the squares of the latent means are expressed in terms of the variances and covariances of the latent means applying Theorem 13 of Bohrnstedt and Goldberger [28]. It is important to note that although the nonlinear model contains the latent means and the covariate as variables, these variables need not to be taken into account in the adjustment. This is because the product of the latent means and their squares are uncorrelated with these other variables due to centering. The adjusted EAP of the square of the latent mean across the perceptions of the employees of a team regarding their team leader’s behavior is:

$${\overline{{\xi }_1^2}}_j=\left({\overline{X_1{X}_2}}_{•j},{\overline{X_1^2}}_{•j},{\overline{X_2^2}}_{•j}\right)\underset{=\mathit{\omega }}{\underbrace{{\mathrm{var}}^{-1}\left({\overline{X_1{X}_2}}_{•j},{\overline{X_1^2}}_{•j},{\overline{X_2^2}}_{•j}\right){\left(2{\sigma }_{{\xi }_1}^2{\sigma }_{{\xi }_1{\xi }_2},{\sigma }_{{\xi }_1}^4,2{\sigma }_{{\xi }_1{\xi }_2}^2\right)}^T}}$$

(11)

The adjusted EAP of the square of the latent mean across the perceptions of the employees of a team regarding their team’s climate can analogously be expressed. So far, we have only defined the EAPs. In the next section, we will discuss how these EAPs can efficiently be estimated in order to help applied researchers implement the extended nonlinear model on their own.

3.2. Estimating the Adjusted EAPs

To estimate the adjusted EAPs, we first need to estimate the variances and covariances in the equations. Estimates will be indicated by a hat ($\widehat{}$) symbol. The estimates of the variances and covariances can then be plugged into the equations in order to obtain the EAP estimates. A straightforward way of estimating the variances and covariances is using estimators from the Analysis of Variance (ANOVA) literature (see [9,15]). We begin with the estimation of the variances and covariances of the observed quantities ${\overline{X}}_{1•j}$, ${\overline{X}}_{2•j}$, and $Z_j$. In the case of balanced data (i.e., equal numbers of employees across teams), the estimate of the variance of the observed mean across the perceptions of the employees of a team regarding their team leader’s behavior is given as the mean sum of squares:

$$\widehat{\mathrm{var}}\left({\overline{X}}_{1•j}\right)=\frac{{\sum }_{j=1}^J{\left({\overline{X}}_{1•j}-{\overline{X}}_{1••}\right)}^2}{J-1}$$

(12)

where ${\overline{X}}_{1••}={\sum }_{j=1}^J{\sum }_{i=1}^n{X}_{1ij}/n$ is the grand mean across the employees’ perceptions. Note that Equation (12) is simply the sample variance of ${\overline{X}}_{1•j}$. For the variance of the observed mean across the perceptions of the employees of a team regarding the team’s climate, an analogous estimate is formulated. To estimate the variance of the personality Z of the team leader, one can use:

$$\widehat{\mathrm{var}}\left(Z_j\right)=\frac{{\sum }_{j=1}^J{\left(Z_j-{\overline{Z}}_{•}\right)}^2}{J-1}$$

(13)

The estimate of the covariance between ${\overline{X}}_{1•j}$ and Z is the mean sum of cross products:

$$\widehat{\mathrm{cov}}\left({\overline{X}}_{1•j},Z_j\right)=\frac{{\sum }_{j=1}^J\left({\overline{X}}_{1•j}-{\overline{X}}_{1••}\right)\left(Z_j-{\overline{Z}}_{•}\right)}{J-1}$$

(14)

An analogous estimate is used for the covariance of ${\overline{X}}_{2•j}$ and Z. The covariance between ${\overline{X}}_{1•j}$ and ${\overline{X}}_{2•j}$ is estimated by:

$$\widehat{\mathrm{cov}}\left({\overline{X}}_{1•j},{\overline{X}}_{2•j}\right)=\frac{{\sum }_{j=1}^J\left({\overline{X}}_{1•j}-{\overline{X}}_{1••}\right)\left({\overline{X}}_{2•j}-{\overline{X}}_{2••}\right)}{J-1}$$

(15)

One efficient way for computing all estimates at once is computing the estimate of a covariance matrix:

$$\begin{array}{c}\widehat{\mathrm{var}}\left({\overline{X}}_{1•j},{\overline{X}}_{2•j},Z_j\right)=\hfill \\ \frac{{\sum }_{j=1}^J{\left(\left({\overline{X}}_{1•j},{\overline{X}}_{2•j},Z_j\right)-\left({\overline{X}}_{1••},{\overline{X}}_{2••},Z_{•}\right)\right)}^T\left(\left({\overline{X}}_{1•j},{\overline{X}}_{2•j},Z_j\right)-\left({\overline{X}}_{1••},{\overline{X}}_{2••},Z_{•}\right)\right)}{J-1}\hfill \end{array}$$

(16)

In the estimation of the adjusted EAP of the latent mean across the perceptions of the employees of a team, this estimate can be used in place of the covariance matrix of the observed quantities in Equation (9). However, to estimate the adjusted EAP, one also needs to estimate the variance of the latent mean and its covariances. Estimating these between variances and covariances is less straightforward but still not very difficult. What helps to find estimates is that scholars have noticed that the measurement model of a latent mean is a random-effects model [27], with the latent mean being the team-specific random effect. Due to this equivalence between a measurement model and random-effects model, it is possible to employ the ANOVA estimator for the variance of the random effect to estimate the between variance (see, e.g., [29] for illustrations of how these estimates can be drived using the ANOVA method). The estimate of the between variance in the measurement model for the latent mean across the perceptions of the employees of a team regarding their team leader’s behavior is given by:

$${\widehat{\sigma }}_{{\xi }_1}^2=\frac{{\sum }_{j=1}^J{\left({\overline{X}}_{1•j}-{\overline{X}}_{1••}\right)}^2}{J-1}-\left\{\frac{{\sum }_{j=1}^J{\sum }_{i=1}^n{\left(X_{1ij}-{\overline{X}}_{1•j}\right)}^2}{J\left(n-1\right)}\right\}/n$$

(17)

As the measurement model for the latent mean across the perceptions of the employees of a team regarding the team’s climate is structurally equivalent, the estimate of the between variance ${\sigma }_{{\xi }_2}^2$ looks similar. The estimates of the between covariances are:

$${\widehat{\sigma }}_{{\xi }_1{\xi }_2}=\frac{{\sum }_{j=1}^J\left({\overline{X}}_{1•j}-{\overline{X}}_{1••}\right)\left({\overline{X}}_{2•j}-{\overline{X}}_{2••}\right)}{J-1}-\left\{\frac{{\sum }_{j=1}^J{\sum }_{i=1}^n\left(X_{1ij}-{\overline{X}}_{1•j}\right)\left(X_{2ij}-{\overline{X}}_{2•j}\right)}{J\left(n-1\right)}\right\}/n$$

(18)

$${\widehat{\sigma }}_{{\xi }_{1}Z}=\frac{{\sum }_{j=1}^J\left({\overline{X}}_{1•j}-{\overline{X}}_{1••}\right)\left(Z_j-{\overline{Z}}_{•}\right)}{J-1}-0/n$$

(19)

If we plug these estimates of the between variance and covariances together with the estimate of covariance matrix into Equation (9), then the estimate of the adjusted EAP is:

$${\widehat{\overline{\xi }}}_{1j}=\left({\overline{X}}_{1•j},{\overline{X}}_{2•j},Z_j\right)\underset{=\widehat{\mathit{\omega }}}{\underbrace{{\widehat{\mathrm{var}}}^{-1}\left({\overline{X}}_{1•j},{\overline{X}}_{2•j},Z_j\right){\left({\widehat{\sigma }}_{{\xi }_1}^2,{\widehat{\sigma }}_{{\xi }_1{\xi }_2},{\widehat{\sigma }}_{{\xi }_{1}Z}\right)}^T}}$$

(20)

From a computational perspective, it makes sense to compute this estimate and the estimate of the adjusted EAP of the latent mean across the perceptions of the employees of a team regarding the team’s climate in one step in order to avoid the need for a second matrix inversion and thus to save computing time. To this end, we compute:

$$\left({\widehat{\overline{\xi }}}_{1j},{\widehat{\overline{\xi }}}_{2j}\right)=\left({\overline{X}}_{1•j},{\overline{X}}_{2•j},Z_j\right)\underset{=\widehat{\mathbf{\Omega }}}{\underbrace{{\widehat{\mathrm{var}}}^{-1}\left({\overline{X}}_{1•j},{\overline{X}}_{2•j},Z_j\right)\left(\begin{array}{cc}{\widehat{\sigma }}_{{\xi }_1}^2& {\widehat{\sigma }}_{{\xi }_1{\xi }_2}\\ {\widehat{\sigma }}_{{\xi }_1{\xi }_2}& {\widehat{\sigma }}_{{\xi }_2}^2\\ {\widehat{\sigma }}_{{\xi }_{1}Z}& {\widehat{\sigma }}_{{\xi }_{2}Z}\end{array}\right)}}$$

(21)

Note that $\widehat{\mathbf{\Omega }}$ is the estimate of a weight matrix rather than of a weight vector in this equation. Alternatively, the result of the matrix inversion could first be stored, and then, both EAP estimates could be computed separately using this stored result.

To estimate the adjusted EAPs of the product of the two latent means and their squares, we only need to find estimates of the variances and covariances of the observed counterparts in the equations for these EAPs. The estimation equations for the between variances and covariances have already been discussed. The efficient method for computing estimates of the observed variances and covariances is computing the following covariance matrix:

$$\begin{array}{c}\widehat{\mathrm{var}}\left({\overline{X_1{X}_2}}_{•j},{\overline{X_1^2}}_{•j},{\overline{X_2^2}}_{•j}\right)=\hfill \\ \frac{{\sum }_{j=1}^J{\left(\left({\overline{X_1{X}_2}}_{•j},{\overline{X_1^2}}_{•j},{\overline{X_2^2}}_{•j}\right)-\left({\overline{X_1{X}_2}}_{••},{\overline{X_1^2}}_{••},{\overline{X_2^2}}_{••}\right)\right)}^T\left(\left({\overline{X_1{X}_2}}_{•j},{\overline{X_1^2}}_{•j},{\overline{X_2^2}}_{•j}\right)-\left({\overline{X_1{X}_2}}_{••},{\overline{X_1^2}}_{••},{\overline{X_2^2}}_{••}\right)\right)}{J-1}\end{array}$$

(22)

Together with the estimates of the between variances and covariances, this covariance matrix estimate can be plugged into the equations in order to estimate the adjusted EAPs of the product of the two latent means and their squares. As mentioned, to avoid multiple matrix inversions, it makes sense to estimate these EAPs simultaneously by:

$$\begin{array}{c}\left({\widehat{\overline{{\xi }_1{\xi }_2}}}_j,{\widehat{\overline{{\xi }_1^2}}}_j,{\widehat{\overline{{\xi }_2^2}}}_j\right)=\hfill \\ \left({\overline{X_1{X}_2}}_{•j},{\overline{X_1^2}}_{•j},{\overline{X_2^2}}_{•j}\right)\underset{=\widehat{\mathbf{\Omega }}}{\underbrace{{\widehat{\mathrm{var}}}^{-1}\left({\overline{X_1{X}_2}}_{•j},{\overline{X_1^2}}_{•j},{\overline{X_2^2}}_{•j}\right)\left(\begin{array}{ccc}{\widehat{\sigma }}_{{\xi }_1}^2{\widehat{\sigma }}_{{\xi }_2}^2+{\widehat{\sigma }}_{{\xi }_1{\xi }_2}^2& 2{\widehat{\sigma }}_{{\xi }_1}^2{\widehat{\sigma }}_{{\xi }_1{\xi }_2}& 2{\widehat{\sigma }}_{{\xi }_2}^2{\widehat{\sigma }}_{{\xi }_1{\xi }_2}\\ 2{\widehat{\sigma }}_{{\xi }_1}^2{\widehat{\sigma }}_{{\xi }_1{\xi }_2}& 2{\widehat{\sigma }}_{{\xi }_1}^4& 2{\widehat{\sigma }}_{{\xi }_1{\xi }_2}^2\\ 2{\widehat{\sigma }}_{{\xi }_2}^2{\widehat{\sigma }}_{{\xi }_1{\xi }_2}& 2{\widehat{\sigma }}_{{\xi }_1{\xi }_2}^2& 2{\widehat{\sigma }}_{{\xi }_2}^4\end{array}\right)}}\end{array}$$

(23)

Once all adjusted EAP estimates are computed, these estimates can be used to obtain the coefficients in our extended nonlinear model.

3.3. Using Adjusted EAP Estimates to Estimate the Model

The nonlinear model in Equation (3) is estimated by running an ordinary least squares regression analysis on the estimates of the adjusted EAPs of the latent means, their product, their squares, and the covariate. To this end, the following regression model is specified and estimated:

$$Y_j=\widehat{\alpha }+{\widehat{\beta }}_1{\widehat{\overline{\xi }}}_{1j}+{\widehat{\beta }}_2{\widehat{\overline{\xi }}}_{2j}+{\widehat{\beta }}_3{\widehat{\overline{{\xi }_1{\xi }_2}}}_j+{\widehat{\beta }}_4{\widehat{\overline{{\xi }_1^2}}}_j+{\widehat{\beta }}_5{\widehat{\overline{{\xi }_2^2}}}_j+\widehat{\gamma }{Z}_j+{\epsilon }_j$$

(24)

where the estimates of the coefficients are the usual ordinary least squares estimates, which are assumed to be unbiased for the corresponding coefficients in Equation (3). In other words, when certain conditions are met (e.g., a sufficiently large sample to overcome small-sample bias), the regression analysis will, on average, yield the actual coefficients—an assumption that we will test empirically in the next section of this article by using computer simulations.

Apart from this favorable feature, there is also a significant drawback. It has been emphasized that regression analysis can provide incorrect standard errors, particularly when the data are very unbalanced or homoscedasticity is violated [9]. Some suggestions have been made in order to overcome this limitation. For example, Croon and van Veldhoven [9] suggested the method developed by Davidson and MacKinnon [30] be used in this case. Another suggestion is using resampling procedures such as Zitzmann’s jackknife. For example, for the interaction effect ${\beta }_3$ in the model, the procedure first computes estimates of this effect from R subsamples, each omitting d teams. d is typically much smaller than the number J of teams. In this specific variant of the jackknife, the subsamples are obtained by dividing the indices $\left(1,\dots ,J\right)$ into $R=J/d$ non-overlapping subsets and then using these subsets to create the subsamples. The standard error is then estimated on the basis of the effect estimates by:

$${\widehat{\sigma }}_{{\widehat{\beta }}_3}={\left\{\frac{\left(R-1\right){\sum }_{j=1}^R{\left({\widehat{\beta }}_{3j}-{\overline{\widehat{\beta }}}_3\right)}^2}{R}\right\}}^{1/2}$$

(25)

where ${\overline{\widehat{\beta }}}_3={\sum }_{j=1}^{J/d}{\widehat{\beta }}_{3j}/R$ is the mean across the estimates. This jackknife is computationally very efficient because it can perform well with only 20 subsamples (see [11]). However, some possible problems have been encountered with the use of this procedure (e.g., a standard error that is too large due to a few extreme estimates), and this is why a bootstrap (e.g., [31]) could be an alternative. The bootstrap repeatedly estimates the effect from $R^{*}$ subsamples, which are random samples from the original sample (drawn with replacement). As the size of these samples, one could choose $J-d$ (i.e., the subsample size in the jackknife). Using the estimates, the standard error is:

$${\widehat{\sigma }}_{{\widehat{\beta }}_3}^{*}={\left\{\frac{{\sum }_{j=1}^{R^{*}}{\left({\widehat{\beta }}_{3j}^{*}-{\overline{\widehat{\beta }}}_3^{*}\right)}^2}{R^{*}-1}\right\}}^{1/2}$$

(26)

$R^{*}$ is typically a large number, and Preacher and Hayes [32] recommended at least 1000 subsamples, for example. In the next section, we will present the results of a simulation study, which we conducted to validate our extension of the EAP-based approach to interaction and quadratic effects and to compare the two resampling procedures for obtaining standard errors.

4. Monte Carlo Study

We conducted a Monte Carlo simulation study to find out whether the EAP-based approach performs as expected, meaning that whether it yields unbiased results for the coefficients in the nonlinear model when the sample of teams is sufficiently large. To this end, we assumed the dependent variable Y (i.e., the team’s productivity), the covariate Z (i.e., the team leader’s self-assessed personality), and the two variables $X_1$ and $X_2$ (i.e., the employees’ perceptions of their team leader and their team, respectively) to be standard-normally distributed with a variance of 1. All variables except Y were grand-mean centered, and $X_1$ and $X_2$ both had a multilevel structure (i.e., the employees’ perceptions split into latent team means (i.e., shared perceptions) and individual deviations from these shared perceptions). The between correlation of $X_1$ and $X_2$ (i.e., the correlation between ${\xi }_1$ and ${\xi }_2$) was set to $0.3$ as were the between correlations between the two X variables and Z. We selected this value because $0.3$ lies in the middle of the typical range of empirically observed correlations.

We set the number of teams to 60, 100, or 200 teams. Whereas 60 and 100 teams served to study the behavior of our method in small-sample, 200 teams served as the test case for the absence of bias in large samples and thus for the validity of our approach (asymptotic unbiasedness). The number of employees surveyed in a group was either 10 or 30 employees, and the ICCs of $X_1$ and $X_2$ were both set to 0.1 or 0.3, implying reliabilities of the observed means ${\overline{X}}_{1•j}$ and ${\overline{X}}_{2•j}$ between $0.53$ and $0.93$. For each of the $3\times 2\times 2=12$ data constellations, we generated 1000 data sets from the following nonlinear model:

$$Y=0+0.2·{\xi }_1+0.5·{\xi }_2+0.2·{\xi }_1{\xi }_2+0.2·{\xi }_1^2+0.2·{\xi }_2^2+0.5·Z+\epsilon$$

(27)

and we estimated this model as described above in the popular statistical computing environment R, which is also known as “the R Project for Statistical Computing” [33].

We studied the bias of our approach in recovering the main, interaction, and quadratic effects of the explanatory variables and the coefficient of the covariate in the model. The bias is defined as the deviation of the expected value of an estimate from the true value in the data-generating model. We divided the bias by the true value (e.g., [34]) and multiplied it by $100\%$ in order to obtain the percentage of bias—a measure that helped to judge whether the bias was negligible or not. An absolute value of less than $10\%$ was considered negligible [35]. In addition, we studied the performance of the bootstrap for obtaining the standard errors of the effects with 1000 subsamples. As the criterion for the evaluation of the bootstrap, we used the coverage rate, which is the probability that the $95\%$ Confidence Interval (CI), which is, for example, ${\widehat{\beta }}_3-z_{1-\alpha /2}{\widehat{\sigma }}_{{\widehat{\beta }}_3}\le x\le {\widehat{\beta }}_3+z_{1-\alpha /2}{\widehat{\sigma }}_{{\widehat{\beta }}_3}$ for the interaction effect in the model, captures the true value. A coverage rate of less than $91\%$ (more than $98\%$) was considered to indicate that the standard errors were underestimated (overestimated; [35]). In addition, we compared the bootstrap with Zitzmann’s jackknife. The CI from the jackknife differs only in that Equation (25) is used in place of the bootstrapped standard error.

Table 1. Simulation Study Results: percentage bias and coverage rates.

	No. of Persons	Intraclass	Percentage of	Coverage Rate
No. of Groups	per Group	Correlation	Bias	Bootstrap	Jackknife
Selected Main Effect (${\beta }_1$)
$J=60$	$n=10$	$\mathrm{ICC}=$ 0.1	43.6	100.0	98.0
		$\mathrm{ICC}=$ 0.3	5.2	97.1	95.0
	$n=30$	$\mathrm{ICC}=$ 0.1	16.5	97.5	95.0
		$\mathrm{ICC}=$ 0.3	2.4	96.4	94.3
$J=100$	$n=10$	$\mathrm{ICC}=$ 0.1	20.0	99.7	96.5
		$\mathrm{ICC}=$ 0.3	12.0	96.9	95.5
	$n=30$	$\mathrm{ICC}=$ 0.1	−3.3	95.7	93.9
		$\mathrm{ICC}=$ 0.3	−1.3	94.5	93.1
$J=200$	$n=10$	$\mathrm{ICC}=$ 0.1	−4.3	98.3	94.7
		$\mathrm{ICC}=$ 0.3	0.9	94.7	93.5
	$n=30$	$\mathrm{ICC}=$ 0.1	2.1	96.4	94.3
		$\mathrm{ICC}=$ 0.3	−2.3	95.8	94.6
Interaction Effect (${\beta }_3$)
$J=60$	$n=10$	$\mathrm{ICC}=$ 0.1	−528.7	100.0	99.7
		$\mathrm{ICC}=$ 0.3	−7.9	99.8	97.2
	$n=30$	$\mathrm{ICC}=$ 0.1	48.4	100.0	98.4
		$\mathrm{ICC}=$ 0.3	12.0	99.4	97.0
$J=100$	$n=10$	$\mathrm{ICC}=$ 0.1	25.3	100.0	99.5
		$\mathrm{ICC}=$ 0.3	11.8	98.8	96.5
	$n=30$	$\mathrm{ICC}=$ 0.1	−9.7	99.8	98.0
		$\mathrm{ICC}=$ 0.3	6.0	97.0	94.1
$J=200$	$n=10$	$\mathrm{ICC}=$ 0.1	−39.0	100.0	97.8
		$\mathrm{ICC}=$ 0.3	−2.2	97.6	94.8
	$n=30$	$\mathrm{ICC}=$ 0.1	7.0	98.5	94.8
		$\mathrm{ICC}=$ 0.3	−1.6	95.6	93.8
Selected Quadratic Effect (${\beta }_4$)
$J=60$	$n=10$	$\mathrm{ICC}=$ 0.1	239.2	100.0	99.9
		$\mathrm{ICC}=$ 0.3	−4.2	99.8	97.4
	$n=30$	$\mathrm{ICC}=$ 0.1	1.6	100.0	99.1
		$\mathrm{ICC}=$ 0.3	−11.1	98.7	96.0
$J=100$	$n=10$	$\mathrm{ICC}=$ 0.1	13.3	100.0	99.5
		$\mathrm{ICC}=$ 0.3	0.7	98.7	95.6
	$n=30$	$\mathrm{ICC}=$ 0.1	−35.2	99.9	97.8
		$\mathrm{ICC}=$ 0.3	−8.8	96.8	93.4
$J=200$	$n=10$	$\mathrm{ICC}=$ 0.1	54.5	100.0	98.0
		$\mathrm{ICC}=$ 0.3	0.5	96.9	93.5
	$n=30$	$\mathrm{ICC}=$ 0.1	−11.3	98.3	95.5
		$\mathrm{ICC}=$ 0.3	−0.3	97.2	95.7
Coefficient of the Covariate ($\gamma$)
$J=60$	$n=10$	$\mathrm{ICC}=$ 0.1	−2.7	100.0	97.1
		$\mathrm{ICC}=$ 0.3	−2.1	95.8	94.2
	$n=30$	$\mathrm{ICC}=$ 0.1	−1.6	96.6	94.1
		$\mathrm{ICC}=$ 0.3	−0.2	95.6	94.5
$J=100$	$n=10$	$\mathrm{ICC}=$ 0.1	−0.8	99.4	96.2
		$\mathrm{ICC}=$ 0.3	−1.1	96.1	94.5
	$n=30$	$\mathrm{ICC}=$ 0.1	0.7	95.3	94.1
		$\mathrm{ICC}=$ 0.3	−0.9	94.1	92.6
$J=200$	$n=10$	$\mathrm{ICC}=$ 0.1	0.2	96.8	95.1
		$\mathrm{ICC}=$ 0.3	0.0	95.2	93.9
	$n=30$	$\mathrm{ICC}=$ 0.1	−0.6	96.3	94.9
		$\mathrm{ICC}=$ 0.3	0.5	95.5	94.4

Note. The coverage rate is stated as a percentage.

shows the percentages of bias in estimating the coefficients in the model. As the biases were similar for both main effects and both quadratic effects, the table shows only the biases for the first main effect and the first quadratic effect. In addition, it shows the coverage rates for the bootstrap and Zitzmann’s jackknife. Despite the match between the nonlinear data-generating model and the analysis model, our approach for estimating this model provided substantially biased estimates of the main, interaction, and quadratic effects (i.e., it exhibited biases of more than $10\%$) but not for the coefficient of the covariate. The biases were particularly pronounced when the number of teams was rather small and the ICCs of the two Xs was low. However, these large small-sample biases could have been expected. What is more important for the validation of the approach is that the approach provided unbiased estimates in large samples, which suggests that the approach is indeed an asymptotically unbiased one.

A similar picture emerged with regard to the coverage rates. Some of the coverage rates were too high (i.e., >98%) when the ICCs were low, particularly those of the main, interaction, and quadratic effects. It is interesting to note that the coverage rates for the bootstrap were slightly less accurate than those for Zitzmann’s jackknife. However, they both tended to become close to the nominal level of $95\%$ when the ICCs increased, indicating that the procedures provide correct standard errors.

5. Discussion

Multilevel data with employees nested in organizations/teams are often collected to examine which factors determine an organization/team’s performance. Of great importance in organizational research are micro–macro designs, in which an upper level variable is predicted by a variable measured at the lower level. To analyze the data from such designs, Croon and van Veldhoven [9] proposed an approach in which an ordinary least squares regression analysis is carried out with Bayesian EAP estimates (which we also called EAPs for the sake of simplicity) as input. In this article, we developed and showed an extension of this approach to interaction and quadratic effects of explanatory variables measured at the lower level. After defining the EAPs, we discussed how these EAPs can be estimated and used to obtain interaction and quadratic effects. We conducted a simulation study to validate our extended approach, and we compared two procedures for obtaining standard errors with each other: a standard bootstrap procedure and Zitzmann’s jackknife. The main findings were the following. First and foremost, the results with regard to the bias showed no asymptotic bias, indicating the validity of our approach. Moreover, the bootstrap and the jackknife performed very similarly overall, with a small disadvantage of the bootstrap under challenging conditions. However, because the bootstrap uses much more subsamples than the jackknife (1000 vs. only 20 subsamples), the different procedures differ in terms of the computing effort. As often, a series of different variants of a model or even different variants of different models are estimated, computing effort adds up, which speaks for the use of the jackknife in research practice.

Regarding possible future extensions, it would be interesting to add further dependent variables to the model. However, such a multivariate model is not estimable with ordinary least squares regression analysis of EAP estimates. Thus, multiple univariate models need to be estimated (one per dependent variable), which highlights once more that our approach is a stepwise and thus only a limited information method [9]. However, the fact that it does not estimate the model at once but divides it into simpler submodels can also be seen as a feature ([36,37]; see also [38]). Studying the potential of this type of estimation for estimating complex models is an interesting subject for future research.

A limitation of the use of EAP estimates should nevertheless be mentioned. The approach places relatively high demands on the data. Specifically, the number of teams should be rather large, and the ICCs should not be too low. It is interesting to note that alternatively, the analysis can be carried out with Structural Equation Modeling (SEM) software. However, the demands tend to be even higher with this software, at least when maximum likelihood methods are used [12], which are the default in commercial software such as Mplus [39]. Prominent examples of maximum likelihood methods for nonlinear models are latent moderated structural equations (LMS; [40]) and quasimaximum likelihood (QML; [41]). These methods determine the estimates in such a way that the probability of the data is maximized under the model. They give unbiased results and small standard errors only when the data provide a large amount of information (i.e., many teams, high ICCs). One possible alternative to our approach and the maximum likelihood methods is the Bayesian Markov Chain Monte Carlo (MCMC) method, which iteratively samples from conditional distributions, thereby creating an MCMC chain from which the estimates can be computed. Similar to our approach, this method places less demands on the data than maximum likelihood ([42,43]; see also [44,45]). Furthermore, it enables more flexibility in specifying models [46], and it leads less frequently to estimation problems [47,48]. However, the Bayesian MCMC method can be problematic despite the mentioned advantages. For example, this method comes at the cost of long computing times because a single iteration of the algorithm is slow, and the method requires many iterations to reach convergence (see [49] for a discussion; see also [50]), particularly when the model is empirically hardly identified. See Hecht et al. [51], Merkle et al. [52], Xu and Liao [53] and Yi and Tang [54] for ways to speed up the method. See Hecht et al. [55] and Hecht and Zitzmann [56] for applications to the analysis of longitudinal data. Whether and under which conditions the Bayesian MCMC method outperforms our approach is however an open question, which can best be addressed in extensive simulation work.

To conclude, we showed that Croon and van Veldhoven’s EAP-based approach for analyzing micro–macro multilevel designs can be extended to interaction and quadratic effects, and we hope that the article will contribute to the use of this approach in organizational research. The approach can easily be implemented in any statistics software, such as SPSS, SAS or R. Finally, we would like to stress that the application of this approach is not limited to the organizational context and that it may also be an interesting option in other areas of multilevel research in which latent means are used (e.g., education sciences, psychology; [14,57]).