Next Article in Journal
Decomposing the Driving Factors of Water Use in China
Next Article in Special Issue
Examining Millennials’ Global Citizenship Attitudes and Behavioral Intentions to Engage in Environmental Volunteering
Previous Article in Journal
The 2030 Agenda for Sustainable Development and China’s Belt and Road Initiative in Latin America and the Caribbean
Previous Article in Special Issue
Double-Edged Effects of Socially Responsible Human Resource Management on Employee Task Performance and Organizational Citizenship Behavior: Mediating by Role Ambiguity and Moderating by Prosocial Motivation
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Relationship between Ability-Based Emotional Intelligence, Cognitive Intelligence, and Job Performance

1
Graduate School for International Development and Cooperation, Hiroshima University, Higashihiroshima, Hiroshima 739-8529, Japan
2
School of Business Administration, University of Economics and Business—Vietnam National University, Hanoi 10000, Vietnam
*
Author to whom correspondence should be addressed.
Sustainability 2019, 11(8), 2299; https://doi.org/10.3390/su11082299
Submission received: 24 March 2019 / Revised: 6 April 2019 / Accepted: 9 April 2019 / Published: 17 April 2019
(This article belongs to the Special Issue Psychology of Sustainability and Sustainable Development)

Abstract

:
Based on previous findings, which found that the three facets of ability-based emotional intelligence (EI) have varying effects on job performance, this study investigates the relationship between emotional intelligence, cognitive intelligence (CI), and job performance. The use of a cascade model suggests a progressive pattern, starting from emotion perception, followed by emotional understanding and emotion regulation, with downstream effects on job performance. Considering the advantages and disadvantages of both measurements, we employed the performance-based ability measurement, the Mayer–Salovey–Caruso Emotional Intelligence Test (MSCEIT) and the self-reporting ability EI measurement, Wong Law Emotional Intelligence Scale (WLEIS). Our findings supported the cascade model, but in the case of WLEIS measures, both self-emotion appraisal and others’ emotion appraisal precede emotion regulation, leading to a positive effect on job performance. Moreover, CI moderated the relationship between EI and job performance, such that a decline in CI rendered the relationship more positive. The MSCEIT and WLEIS showed similar results, thus supporting the cascading model and moderating effects.

1. Introduction

Emotional intelligence (EI) has become a major emerging research area in psychological, organizational, and educational fields. Previous research has focused on determining whether EI meets the criteria for general intelligence, and has incremental validity over cognitive intelligence (CI) and personality traits. Upon consensus on the validity of EI, focus has shifted to the relationship between EI and its outcomes, such as job or academic performance, leadership effectiveness, work-life balance, and quality of interpersonal relationships. Among them, EI’s direct effect on job performance has contradictory results. Some studies show a significant positive effect (e.g., [1,2,3,4]), while others indicated an insignificant and/or negligible effect [5]. These inconsistent results could be attributed to the varying effects of EI’s four dimensions: Emotion perception, emotion understanding, emotion usage, and emotion regulation. Kluemper, DeGroot, and Choi [6] found that only the latter two variables had a positive relationship with task performance. The findings supported Joseph and Newman’s [7] cascading meta-analytic model, in which the EI facets follow a progressive pattern (i.e., starting with emotion perception, and followed by emotion understanding and conscious emotion regulation) to explain job performance. This model proposed that among four aspects of EI, emotion regulation played the most important role in determining job performance, and there were no direct effects from emotion perception and emotion understanding on job performance. This argument may explain why, when considering a composite measure of EI as a whole, the relationship can become negligibly small, as it is sometimes found in the literature, especially when the job does not need employees skilled in EI. Therefore, the cascading model, which has deconstructed EI into independent dimensions to evaluate its effect on job performance, has been advocated in the literature.
Unfortunately, this model was developed from performance-based ability evaluations of EI, and thus, it may not correspond with a self-reporting ability measurement. Although developed from the same theoretical ability-based model of EI, emotion perception (perceiving the emotions of our self and others), and emotion understanding (understanding the emotions of our self and others) in MSCEIT, are not similar to self-emotion appraisal (SEA-perceiving and understanding the emotions of our self), and others’ emotion appraisal (OEA-perceiving and understanding the emotions of others) in WLEIS, respectively. Both emotion perception and emotion understanding are included in SEA and OEA of WLEIS. This violates the sequence of progression from emotion perception to emotion understanding of the original cascading model. As such, given the broad usage of self-reported ability tests in research, it is meaningful to examine the possibility of a modified cascading model of EI’s effect on job performance, which can be assessed using such measurements.
In addition, we investigated a compensatory model, in which EI’s effect on performance becomes stronger with reduced CI, similar to the findings of Cote and Miners [8]. Unlike their research, however, only emotion regulation was used to create the interaction with CI in our statistical analysis. This modification to the compensatory model was proposed for three reasons. First, empirical studies reveal that only emotion usage and regulation have a directly positive relationship with job performance [6]. However, a conceptual overlap was observed between emotion usage and other EI dimensions [9], which is why emotion usage was excluded from the compensatory model, as can also be observed in Joseph and Newman’s [7] cascading model. Second, a three-factor model produced a better fit than a four-factor model [10], thereby supporting the elimination of emotion usage. Third, emotion regulation shares the least variance with CI; in fact, it predicted more variance in job performance for jobs requiring substantial emotional labor than CI and personality did [6]. EI, as a whole construct, including four aspects—two of them are not related to job performance—may not be the best indicator for CI compensation. In particular, this research argues that the effect from EI follows a sequence ending with emotional regulation. Thus, we proposed that the compensatory effect comes mainly from the interaction between CI and emotion regulation. This modification serves as a better practical computation for the compensatory model.
Inconsistency in the research results could also be due to the use of different measures in evaluating EI ability. Among frequently used approaches (i.e., performance ability-based models, self- or peer-report measures based on EI’s four-branch model, trait EI model, and “mixed models” of emotional competencies [11]), the mixed model measurements had no impact on job performance if certain covariates were controlled [12]. That was because mixed EI measurements covered other well-established psychological constructs, such as conscientiousness, extraversion, emotional stability, and general self-efficacy [12]. Joseph et al. [12] updated and corrected the conclusions from Joseph and Newman [7] and O’Boyle et al. [11], indicating that mixed EI had a significantly incremental validity, beyond the Big Five and cognitive ability. Those two meta-analysis studies did not control for self-perceptions or for ability-based EI, leading to a sizeable explained variance of mixed EI on job performance. Similar criticism of the trait EI, which belongs to the realm of personality [13] and combined emotion-related self-perceptions and dispositions [14,15], has also been raised. Van der Linden et al. [16] found a large overlap between trait EI measures and general factor of personality (r ~ 0.85). Among available measures, ability-based EI measures seemed to be the most precise in capturing the notion of EI as an intelligence [17]. Therefore, this study utilized ability-based EI measurements to investigate relationship among CI, EI, and job performance.
The MSCEIT-measured EI, based on the ability model, is widely used in the literature owing to data reliability, validity [18], and objectivity. This test has been criticized for reflecting a cultural bias [19], so it may not be well effectively in some regions, such as Africa and Asia. However, there has not been such consensus among researchers. Indeed, MSCEIT has been developed and widely used in North America, where there is high diversity, and research samples usually contain both African and Asian Americans. Similar circumstances are also recorded in research conducted in Europe. Furthermore, the validity, reliability, and invariance of MSCEIT across cultures were supported in recent studies with an Italian sample [20], Spanish sample [21], Romanian sample [22], Chinese sample [23], and a comparison between Pakistani and French samples [24]. However, MSCEIT is time-consuming and expensive, which may cause difficulties for practical application. Keeping the issue of cultural adaptation in mind, this study still valued MSCEIT owing to its data objectivity, which is usually difficult to achieve through a perception questionnaire.
Another measurement, which was also based on an EI ability model is the Wong Law Emotional Intelligence Scale (WLEIS; [3]). This scale is short, easy to operationalize, and also showed invariance across countries, specifically in Singapore and Belgium [25]. Nevertheless, the measure was developed as a self-reported perception questionnaire, which may not be appropriate to evaluate ability due to social desirability bias. Respondents may overestimate their desirable ability and blame other reasons rather than their inability in case of failure [26]. These authors also noted that self-reported answers may capture irrelevant variance in evaluated variables that biases the results.
Certainly, both MSCEIT and WLEIS have their own advantages and disadvantages for assessing EI conceived as a set of abilities. Given the fact that both have been widely used in the literature, the present study employed both the performance-based MSCEIT and self-reported WLEIS to examine the relationship among EI, CI, and job performance. By doing so, it contributes to the literature by integrating a cascading model and a modified compensatory model with both objective and subjective measurements of EI.

2. Theoretical Development

EI’s broad construct, often measured by a mixed-model EI, has been criticized for combining the Big Five personality factors. More specifically, Joseph and colleagues found that the majority variance in mixed EI (62%) is covered by other well-established concepts, whereas only 23% of ability EI is captured by these variables [12]. Ability-based EI also showed acceptable correlation with CI [6] and the general factor of personality [16]. Over the last three decades, progress has been made indicating that EI is an independent and valid construct of intelligence, which encompasses emotions. The MSCEIT and WLEIS, based on the ability model, were used with relatively similar corresponding dimensions. Thus, we combined Caruso, Mayer, and Salovey’s [27] emotion management and emotion facilitation with Wong and Law’s [3] emotion regulation, and emotion usage, respectively. The original terms for emotion perception and understanding of MSCEIT and self-emotion appraisal (SEA), and others’ emotion appraisal (OEA) of WLEIS were maintained because of differences in both conceptual constructs and measurements. More specifically, ‘self-emotion perception’ and ‘self-emotion understanding’ are ‘SEA’, while ‘others’ emotion perception’ and ‘others’ emotion understanding’ are ‘OEA.’

2.1. Cascading Model of EI and Job Performance

Joseph and Newman’s [7] cascading model was based on the theories of emotion, demonstrating that, among EI’s four dimensions, emotion regulation or self-regulation has the biggest effect on job performance. Two mechanisms are at play here. First, positive affective states promote job performance by enhancing collaborative behaviors and developing self-motivation [28,29]. Second, mood suppression attenuates negative or disadvantageous emotions [30,31]. In addition, people skilled at mood suppression also tend to be skilled at positive strategies like cognitive appraisal, leaving them with more cognitive resources for performing their jobs more effectively [7].
Indeed, in an alternative viewpoint based on resource allocation, emotion regulation may have a negative effect on job performance, as it diverts attentional resources from the task at hand [32]. Kanfer, Ackerman, and Heggestad [33] showed that, when emotion regulation strategies do not compete for resources, their relationship with performance is strongest; however, people with a high emotion regulation ability can also choose the approaches more suited to task demands, which would help them in draining less resources and maintaining job performance [6]. Hence, we assumed that a higher emotion regulation ability leads to better job performance.
Based on the model of emotion (in which a stimulus requires attention, then appraisal, and ultimately a response [34]), Joseph and Newman [7] suggested that EI’s effect must follows the sequence of abilities, starting with emotion perception, followed by emotion understanding, and emotion regulation. Emotion perception in the cascading model plays a similar role as attention in the emotion model [7]. Similarly, appraisal is somewhat equivalent to emotion understanding (i.e., understanding of emotional states, emotional differences among people, and emotional situation-appropriateness [35]). The knowledge of emotions implicit in emotion understanding can be conceptualized as accumulated knowledge structures activated by attention [36]. Specifically, emotion perception may influence emotion understanding, leading to a response of emotion regulation. Consequently, Joseph and Newman [7] introduced emotion understanding as a mediating step between emotion perception and emotion regulation.
However, Joseph and Newman [7] also noted that the cascading model only corresponded to MSCEIT measurement. As stated earlier, emotion perception (perceiving emotions of our own and others) and emotion understanding (understanding the emotions of our self and others) in MSCEIT are not similar to SEA (perceiving and understanding the emotions of our self), and OEA (perceiving and understanding the emotions of others) in WLEIS, respectively. Thus, considering the model using MSCEIT as a base, we proposed a modified cascading model for WLEIS, wherein both SEA and OEA precede emotion regulation, which has a positive effect on EI, while simultaneously considering a strong correlation between these two aspects of EI.
The modification is supported by Joseph and Newman’s argument [7], that emotion understanding fully, but sometimes only partially, mediates the relationship between emotion perception and emotion regulation when the process is automatic or without voluntary control. Although a direct relationship with a small coefficient between emotion perception and emotion regulation was found, the authors stated that the mediating effect of emotion understanding accounted for most of the effect in the process. Furthermore, they focused solely on the conscious process of emotion regulation, and they claimed a full mediation of emotion understanding in the process. Nevertheless, both SEA and OEA not only include the dimension of emotion understanding but also contain the emotion perceiving dimension of EI. In WLEIS, emotion perception was measured using knowledge about emotion and observation through actual behaviors of our self (in SEA) and others (in OEA). It can automatically affect the way individuals react or adjust their emotion, even without understanding their own and others’ emotion at that time. This mechanism strengthens the possibility of a direct relationship between both SEA and OEA and emotion regulation.
Hence, we propose two hypotheses corresponding to the two measurements:
Hypothesis 1.
EI has a positive relationship with job performance based on a sequential process, starting with emotion perception, and followed by emotion understanding, emotion regulation, and, ultimately, job performance (MSCEIT case).
Hypothesis 2.
EI has a positive relationship with job performance based on a sequential process, starting with both SEA and OEA, and followed by emotion regulation, and, ultimately, job performance (WLEIS case).

2.2. CI and Emotion Understanding

We noted the important role of CI in explaining job performance. In the original cascading model, the dimensions of emotion understanding partially mediate CI’s influence on job performance. Generally speaking, people with higher CI obtain more job-related knowledge, resulting in better performance [7]; since knowledge as an aspect of CI, strongly affects job performance [37]. In addition to CI’s direct effect on job performance, Trentacosta, Izard, Mostow, and Fine [38] reported a significant correlation between children’s verbal CI and emotion understanding, supporting the original cascading model. Thus, we propose:
Hypothesis 3.
CI has a positive relationship with emotion understanding (MSCEIT case).
In the modified cascading model, using WLEIS, both SEA and OEA, include the emotion understanding dimension. Therefore, we propose a direct relation between CI and both constructs as follows:
Hypothesis 4.
CI has a positive relationship with both SEA and OEA (WLEIS case).

2.3. Compensatory Model between CI and EI

Both CI and EI contribute to job performance; the latter can also explain variance in job performance that cannot be explained by the former [8]. However, as previously discussed, studies on the association between EI and job performance have produced different results. In addition to the cascading model, Cote and Miners [8] suggested another possible explanation for such inconsistency: A compensatory model in which, when a person has high CI, EI’s effect on performance may be limited. They stated that linear effect models of EI, CI, and job performance may be too simplistic and incomplete. The compensatory model proposed a moderating effect of CI on the relationship between EI and job performance, which becomes more positive with a declining CI. This model is consistent with the works of Carroll [39], and Viswesvaran and Ones [40], stating that a high ability (e.g., EI) can be compensatory for the weakness caused by a low ability (e.g., CI) within an individual as long as these abilities contribute to performance.
Cote and Miners [8] suggested that EI may help individuals of high CI with emotional issues, but it will only slightly affect their job performance, which may already be high. The reason may be changing magnitude; that is, the performance of people with high CI may be close to maximum already, and thus, effects on performance are much smaller than on people with a low CI. Consequently, consistent with the compensatory model, we propose the following hypothesis.
Hypothesis 5.
CI moderates the relationship between EI and job performance such that the relationship becomes more positive with declining CI.
However, there is one practical modification in that only the effect of emotion regulation was incorporated into the moderating effect of CI during statistical checking, as discussed in the introduction. Figure 1 and Figure 2 illustrate the conceptual framework for the present study in the case of MSCEIT, and WLEIS, respectively.

3. Methods

3.1. Ethical Statement

This study was approved by the human resource ethics committee of the Center for Business Administration Studies, University of Economics and Business—Vietnam National University, Hanoi. A written informed consent was obtained from each participant after communicating to them that participation was voluntary.

3.2. Sample

A field study was conducted among 191 employees (46.1% male, mean age = 29.01, SD = 4.15, and mean tenure = 4.93, SD = 3.56). The employees were studying as graduate students on weekends at a university in Vietnam. They worked in various companies: 49 electricity and manufacturing factories; 17 banks with 40 different branches; 21 governmental departments; 20 sales businesses; 14 insurance and finance companies; 10 educational services; 8 tourism services, hotels, and restaurants; 7 information technology and telecom; 7 transportation; 5 import-export; 4 housing and real estate; 3 automobile; and 3 consulting services.

3.3. Procedure

A total of 509 employees were approached at the university and asked to participate. We distributed surveys to obtain demographic information, and we also asked for email addresses of their direct bosses and three colleagues. Then, we sent them a link to our online questionnaire with a cover letter to the boss and one randomly chosen colleague of each participant, who evaluated the respective participant’s job performance. To increase the response rate, questionnaires were also mailed to their offices. Both the boss and the peer returned questionnaires for 198 participating employees (38.89%). Those 198 employees proceeded to participate in an online EI assessment. Participants were given US$ 5 for participating in the study.
The MSCEIT was not translated into Vietnamese, as prohibited by its publisher, MHS Inc. With this issue in mind, before administering the test, the research team consulted carefully with all the lecturers. Accordingly, most of the recruited employees were studying at linkage and international programs; they needed to meet a score of 6.5 for the International English Language Testing System-IELTS when enrolled. The rest of the participants who were from standard postgraduate programs also needed to pass the English test of the university before enrollment. The sample MSCEIT was well understood by 10 randomly chosen employees. Furthermore, five English translators provided by the research team were available to support employees during the test. The translators were trained in advance, had excellent English proficiency, and followed an official translation that the research team obtained from a professional linguistic service company. Participants had two choices if having any problems with an English item: Asking for help from the translator or; checking the dictionary by using the Internet. However, the support was negligibly recorded because vocabulary used in the test questions is not difficult and items under each section followed similar structures. Our respondents reported that they mainly managed the test themselves and only checked some words on the Internet. A t-test was conducted to check if any different results of MSCEIT scores would be obtained by international program versus standard program students. We found that no such difference occurred (ps > 0.05) for all aspects of MSCEIT band scores. A copyrighted scoring system (expert consensus) was used; scores were sent to us by the test publisher. Cattell and Cattell’s [41] Culture Fair Intelligence Test (CFIT) Scale 3, Form A, administered as per the guidelines, was used to measure CI. Correct answers were counted, then the raw scores were converted into interpretable normalized standard scores with a mean of 100 and standard deviation of 15, utilizing the transforming scale in the manual. The CFIT is relatively old but has the advantage of not including English literature questions, which could make the scores more reliable, considering the usage of a non-native English speaker sample. Ultimately, data from 191 observations were found valid for analysis. Non-response bias was checked by comparing the final sample’s demographic information with the other employees in the population; the results showed insignificant differences (p > 0.05). The 191 employees received further compensation, which was also valued at USD 5, for their time and effort.

3.4. Measures

To measure EI, the performance-based MSCEIT and self-reported WLEIS, both of which were developed from the ability-based EI model, were used. In this study, Cronbach’s alpha of the perception questionnaire WLEIS was 0.902. For further analyses and theoretical calculation, the EI in WLEIS measurement was divided into three sub-measures: SEA, OEA, and emotion regulation, with Cronbach’s alphas of 0.901, 0.886, and 0.847, respectively. CI was measured using the CFIT [41]. Job performance was evaluated using the items developed by Tsui, Pearce, Porter, and Tripoli [42], which have a Cronbach’s alpha of 0.862. Examples of the five point Likert scale questions are “employee’s quantity of work is higher than average,” “employee’s standards of work quality are higher than the formal standards for this job.” The scale ranged from 0 = strongly disagree to 5 = strongly agree. Obtaining reliable and unbiased performance measures is a live challenge in organizational psychology [43]. Therefore, we tried to reduce the evaluation bias by aggregating evaluations from the supervisor and one colleague of the participant. Although those evaluations were based on a questionnaire, the nature of the questions was to compare the quantity and quality of the tasks performed by the respective participants with those by other employees (boss rating case), and with others including himself/herself (peer rating case). This process might help make the measures more trustworthy and objective. The ratings of each employee from her/his boss and peer had an acceptable level of agreement, as indicated by within-group agreement (rWG = 0.92), intraclass correlation (ICC (1) = 0.69), and inter-rater reliability (ICC (2) = 0.81). These indexes satisfied the criteria suggested by previous studies [44,45,46] for data aggregation; thus, the data were aggregated.

3.5. Controls

We controlled the most common variables that could affect the outcomes namely, age, gender, and tenure. Given the fact that employees in our sample came from diverse organizations, we used a dummy variable to indicate whether the employees needed to work mainly with people in their workplace (so called “type of job”). This is because some people were working for service companies but holding technical positions and vice versa. To some extent, this dummy variable may reflect the emotional labor requirement of the job.
The authors confirm that the data supporting the findings of this study are available within its Supplementary Materials.

4. Results

Table 1 presents the means, standard deviations, and correlation coefficients of variables.
Prior to hypothesis testing, a confirmatory factor analysis (CFA) was conducted to evaluate the measurement model using WLEIS (χ2 = 187.605; df = 129; CMIN/df = 1.454; comparative fit index [CFI] = 0.968; Tucker–Lewis index [TLI] = 0.962; goodness of fit index [GFI] = 0.899; adjusted goodness of fit index [AGFI] = 0.866; root–mean–square error of approximation [RMSEA] = 0.049; P-close = 0.533; SRMR = 0.0483). The validity of the four latent factors (three dimensions of perceived WLEIS and job performance) was also checked. The critical ratio (CR), which was greater than 0.7 in all cases, confirmed the composite reliability of those measurements. Convergent validity was also achieved, as no average variance extracted (AVE) was less than 0.5. Finally, discriminant validity was confirmed, as all AVE values were greater than the maximum shared variance (MSV) and the average shared variance (ASV). The results are reported in Table 2.
Next, we employed the structural equation modeling to check the hypotheses. The model that used the MSCEIT represented a relatively excellent model fit [47], with χ2 = 79.693, df = 74, CMIN/df = 1.077, CFI = 0.993, TLI = 0.990, GFI = 0.947, AGFI = 0.914, RMSEA = 0.020, P-close = 0.968, SRMR = 0.0456; for the perceived WLEIS model, χ2 = 328.253, df = 231, CMIN/df = 1.421, CFI = 0.955, TLI = 0.947, GFI = 0.879, AGFI = 0.843, RMSEA = 0.047, P-close = 0.653, SRMR = 0.0576.
As for the hypotheses, evidence supporting the cascading and compensatory models in both the MSCEIT and WLEIS was found. An examination of the 95% bias-corrected confidence intervals (CIs) from 1000 bootstraps samples was also checked. More specifically, the parameters for emotion perception, emotion understanding, and emotion regulation were 0.355 (95%CI: 0.239–0.463), 0.322 (95%CI: 0.172–0.434), and 0.248 (95%CI: 0.104–0.381), respectively, for the MSCEIT model. Paths from SEA and OEA to emotion regulation and from emotion regulation to job performance had coefficients of 0.418 (95%CI: 0.273–0.534), 0.337 (95%CI: 0.172–0.462), and 0.497 (95%CI: 0.344–0.637), respectively. SEA and OEA also had a correlation of 0.419 with p < 0.01. These results supported both Hypothesis 1 and Hypothesis 2. Hypothesis 5 was also supported by the significant results obtained in the MSCEIT (β = −0.201 (95%CI: −0.352–−0.063)) and WLEIS (β = −0.196 (95%CI: −0.395–−0.009)). Based on the interaction plot (Figure 3), the results revealed that the relationship is stronger when CI decreases.
Additionally, we could not find any significant result to support the association between CI and emotion understanding, as well as SEA or OEA; thus, Hypothesis 3 and Hypothesis 4 were not supported (p > 0.05 in all cases, 95%CIs: −0.198–0.097; −0.121–0.195; −0.102–0.198, respectively). The standardized regression weights are reported in Table 3. Predictive power of the model using MSCEIT was 23.1% with 4.6% variance explained by EI and 8.3% variance explained by CI. The predictive power for the model using WLEIS was 40% with 21.7% variance explained by EI and 8.3% variance explained by CI.

5. Discussion

All three hypotheses were checked for EI in the MSCEIT and WLEIS; both measurements produced similar results. First, a full mediating role of emotion, understanding between emotion perception and emotion regulation, was found in the case of MSCEIT, which was consistent with Joseph and Newman’s [7] final cascading model (for confirmation, we checked the direct relationship from emotion perception to emotion regulation and it was insignificant with p > 0.05). Emotion perception in the MSCEIT was measured by detecting emotions through face tasks, picture tasks, colors, and abstract design. The result could be considered as pure attention in Kintsch’s [36] knowledge activation model, leading to the necessity of emotional understanding in the process. In the case of WLEIS, both SEA and OEA have a positive relationship with emotion regulation, as expected. Second, both cases of measurements obtained a significantly positive association between emotion regulation and job performance. Combining those results, Hypothesis 1 and Hypothesis 2 were supported, providing evidence for the cascading model in terms of the sequential effect of EI dimensions on job performance.
However, the findings indicated that the relationship between emotion regulation and job performance in the WLEIS had a bigger coefficient, compared with that in the MSCEIT (0.497 versa 0.248); this result was possibly caused by features of scale development. Self-reported measurements have been found to have a higher correlation with the mixed model [7], which captures other aspects beyond the ability concept. Unfortunately, this study did not control those, which is why an additional effect might have originated from its overlap with the covariates. This also clarified why the variance of job performance, as explained by EI and measured by WLEIS, was much bigger than that measured by MSCEIT (21.7% compared to 4.6%). The weaker correlation of EI measured by MSCEIT along with its low power of explanation are consistent with findings in previous studies [12].
Furthermore, the findings showed insignificant results for the direct relationship between CI and emotional understanding, as well as SEA and OEA, leaving Hypothesis 3 and Hypothesis 4 unsupported. The results were consistent across MSCEIT and WLEIS, but were contrary to those obtained by Joseph and Newman [7]. Their meta-analysis included certain scales, some of which were based on the mixed EI model and simultaneously utilized self-report methods to measure EI. These constructs are likely to measure existing personality characteristics or emotional competencies, but not intelligence [48]. In contrast, the present study used only the ability-based model to measure emotion as another sphere of intelligence. The ability to perceive and understand emotion might not depend on CI, which leans toward logical thinking and general mental ability. This idea is supported by the fact that EI, which is considered independent of CI, can be learned and improved by experiences or training effort [49,50], especially abilities related to emotional perception and understanding. In other words, individuals with a low CI, can have the same or even higher emotion perception and understanding ability as others, by continuously accumulating knowledge and skills over time. This could lead to an insignificant relationship between CI and emotional understanding, as well as SEA and OEA.
Finally, the compensatory model regarding Hypothesis 5 was supported by the consistent results of both measurements. Basing on the interaction plot (Figure 3), when CI increases, the relationship between EI and job performance becomes less robust. That relationship is also less robust in both cases of high and low CI groups in MSCEIT, compared to the corresponding groups in WLEIS. This is due to a weaker correlation between EI measured by MSCEIT, with job performance, compared to that between EI measured by WLEIS and job performance.

6. Conclusions

6.1. Theoretical Contributions

Supported by both the theoretical foundation and empirical results, this study contributes to the literature by integrating the compensatory model into the cascading one, by examining the association between EI, CI, and job performance. The measurements were carefully chosen to reduce bias related to self-reported constructs. The consistency of the main findings revealed that both measurements of EI are valid. The results confirmed that, among four aspects of EI, emotion regulation plays the most important role in influencing job performance, and that its interaction with CI produced a compensatory effect. This study also contributes to the literature by establishing a modified cascading model, used for self-reporting ability EI measurement, such as WLEIS.
Moreover, we advanced the knowledge of EI by using the MSCEIT and WLEIS together in our research, and by examining EI as respectively evaluated by these measurements, in order to check the robustness of the research results. The largely similar findings confirmed the reliability and validity of the two measurements. However, they cannot be used interchangeably, and researchers may need to consider specific conditions in choosing a suitable scale. If time and budget allow, one may prefer MSCEIT because of its prominent validity and data objectivity. It is considered to be the most accurate measure to evaluate EI as an intelligence in spite of a weak correlation with job performance. WLEIS may be used in well-controlled study conditions. However, we suggest using supervisor or colleague ratings, rather than self-reported perception, although one should note that people with high EI may have some influence on their supervisor and peer evaluation of him or her. Perhaps, their relationship should be controlled to reduce such bias.

6.2. Practical Implications

Our study has practical implications. EI can be included as part of recruitment criteria, and managers consider EI to compensate for low CI, especially in the service industry where EI is important. As EI can be improved through accumulated experiences and training efforts, companies may consider providing EI training, particularly for those with a low CI. The study also has educational implications. In addition to improving cognitive abilities, EI should be included in the curriculum; for example, students could be encouraged to participate in social activities or enhance their arts and social skills [51].

6.3. Limitations and Future Research Direction

This research has several limitations. We only investigated the main parts of the original cascading model, which were related to our studied variables and did not control for the respondents’ personality factors (Big Five), that are also considered to be associated with job performance. Another possible issue is that the respondents’ varying English proficiency levels might have created bias in the responses obtained in the MSCEIT, even though the statistical check did not reveal any problematic issues. In addition, the fact that only two individuals—their boss and one randomly chosen peer—assessed each employee means that their relations with these people might have biased evaluations as mentioned before. High EI employees may be better at selecting colleagues who will have a good opinion of them, or they might impact their supervisors’ expectations and perceptions about their performance. This study lacks in subjective evaluation regardless of our attempt to obtain a more reliable indicator of employees’ job performance, by aggregating data from two evaluation sources rather than from a single self-reported measure. Future research should control supervisor/peer-respondent relations and obtain objective information regarding the performance, such as volume of sale, KPI index, customer satisfaction, and quantity, and quality of creative ideas. Furthermore, as the results of this research were based on cross-sectional study, a causal relationship between EI and job performance cannot be confirmed. Another limitation of this study is that the sample size was relatively small, consequently making it difficult to conduct further sub-group analysis. Therefore, derivation of any practical implication of this study’s results must be made with caution. Although we controlled “type of job,” which may partly capture the effect of emotional labor on job performance, unlike Joseph and Newman’s (2010), this research was unable to eliminate potential heterogeneity from moderators such as emotional labor. Thus, future research should consider job context in the analysis, as it is likely to mitigate potential bias arising from non-random employee assignment to workplaces. Additional control variables should also be included. Besides, all EI dimensions, mechanisms of cascading and compensatory models should be examined to evaluate how EI and CI work together to predict job performance. Other indicators of job performance, such as organizational citizenship behavior, must also be investigated.

Supplementary Materials

The following are available online at https://www.mdpi.com/2071-1050/11/8/2299/s1, S1.File.Dataset.sav.

Author Contributions

Conceptualization, N.N.N. and Y.T.; methodology, N.N.N. and Y.T.; software, N.N.N.; validation, N.N.N., P.T.N., and Y.T.; formal analysis, N.N.N.; investigation, N.N.N. and P.T.N.; resources, N.N.N., P.T.N., and Y.T.; data curation, N.N.N.; writing—original draft preparation, N.N.N.; writing—review and editing, N.N.N., P.T.N., and Y.T.; visualization, N.N.N.; supervision, Y.T.; project administration, N.N.N.

Funding

This research received no external funding.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Farh, C.I.; Seo, M.G.; Tesluk, P.E. Emotional intelligence, teamwork effectiveness, and job performance: The moderating role of job context. J. Appl. Psychol. 2012, 97, 890–900. [Google Scholar] [CrossRef] [PubMed]
  2. Sy, T.; Tram, S.; O’Hara, L.A. Relation of employee and manager emotional intelligence to job satisfaction and performance. J. Vocat. Behav. 2006, 68, 461–473. [Google Scholar] [CrossRef]
  3. Wong, C.S.; Law, K.S. The effects of leader and follower emotional intelligence on performance and attitude: An exploratory study. Leadersh. Q. 2002, 13, 243–274. [Google Scholar] [CrossRef]
  4. Vratskikh, I.; Al-Lozi, M.; Maqableh, M. The impact of emotional intelligence on job performance via the mediating role of job satisfaction. Int. J. Bus. Manag. 2016, 11, 69–91. [Google Scholar] [CrossRef]
  5. Day, A.L.; Carroll, S.A. Using an ability-based measure of emotional intelligence to predict individual performance, group performance, and group citizenship behaviours. Pers. Individ. Differ. 2004, 36, 1443–1458. [Google Scholar] [CrossRef]
  6. Kluemper, D.H.; DeGroot, T.; Choi, S. Emotion management ability: Predicting task performance, citizenship, and deviance. J. Manag. 2013, 39, 878–905. [Google Scholar] [CrossRef]
  7. Joseph, D.L.; Newman, D.A. Emotional intelligence: An integrative meta-analysis and cascading model. J. Appl. Psychol. 2010, 95, 54–78. [Google Scholar] [CrossRef]
  8. Cote, S.; Miners, C.T. Emotional intelligence, cognitive intelligence, and job performance. Adm. Sci. Q. 2006, 51, 1–28. [Google Scholar] [CrossRef]
  9. Palmer, B.R.; Gignac, G.; Manocha, R.; Stough, C.A. Psychometric evaluation of the Mayer-Salovey-Caruso emotional intelligence test version 2.0. Intelligence 2005, 33, 285–305. [Google Scholar] [CrossRef]
  10. Rossen, E.; Kranzler, J.H.; Algina, J. Confirmatory factor analysis of the Mayer–Salovey–Caruso emotional intelligence test V 2.0 (MSCEIT). Pers. Individ. Differ. 2008, 44, 1258–1269. [Google Scholar] [CrossRef]
  11. O’Boyle, E.H., Jr.; Humphrey, R.H.; Pollack, J.M.; Hawver, T.H.; Story, P.A. The relation between emotional intelligence and job performance: A meta-analysis. J. Organ. Behav. 2011, 32, 788–818. [Google Scholar] [CrossRef]
  12. Joseph, D.L.; Jin, J.; Newman, D.A.; O’boyle, E.H. Why does self-reported emotional intelligence predict job performance? A meta-analytic investigation of mixed EI. J. Appl. Psychol. 2015, 100, 298–342. [Google Scholar] [CrossRef]
  13. Śmieja, M.; Orzechowski, J.; Stolarski, M.S. TIE: An ability test of emotional intelligence. PLoS ONE 2014, 9, e103484. [Google Scholar] [CrossRef]
  14. Herpertz, S.; Nizielski, S.; Hock, M.; Schütz, A. The relevance of emotional intelligence in personnel selection for high emotional labor jobs. PLoS ONE 2016, 11, e0154432. [Google Scholar] [CrossRef]
  15. Petrides, K.V.; Furnham, A. Trait emotional intelligence: Psychometric investigation with reference to established trait taxonomies. Eur. J. Pers. 2001, 15, 425–448. [Google Scholar] [CrossRef]
  16. Van der Linden, D.; Pekaar, K.A.; Bakker, A.B.; Schermer, J.A.; Vernon, P.A.; Dunkel, C.S.; Petrides, K.V. Overlap between the general factor of personality and emotional intelligence: A meta-analysis. Psychol. Bull. 2017, 143, 36–52. [Google Scholar] [CrossRef]
  17. MacCann, C.; Joseph, D.L.; Newman, D.A.; Roberts, R.D. Emotional intelligence is a second-stratum factor of intelligence: Evidence from hierarchical and bifactor models. Emotion 2014, 14, 358. [Google Scholar] [CrossRef]
  18. Brackett, M.A.; Mayer, J.D. Convergent, discriminant, and incremental validity of competing measures of emotional intelligence. Pers. Soc. Psychol. Bull. 2003, 29, 1147–1158. [Google Scholar] [CrossRef]
  19. Law, K.S.; Wong, C.S.; Huang, G.H.; Li, X. The effects of emotional intelligence on job performance and life satisfaction for the research and development scientists in China. Asia Pac. J. Manag. 2008, 25, 51–69. [Google Scholar] [CrossRef]
  20. Curci, A.; Lanciano, T.; Soleti, E.; Zammuner, V.L.; Salovey, P. Construct validity of the Italian version of the Mayer–Salovey–Caruso emotional intelligence test (MSCEIT) v2.0. J. Pers. Assess. 2013, 95, 486–494. [Google Scholar] [CrossRef]
  21. Sanchez-Garcia, M.; Extremera, N.; Fernandez-Berrocal, P. The factor structure and psychometric properties of the Spanish version of the Mayer-Salovey-Caruso Emotional Intelligence Test. Psychol. Assess. 2016, 28, 1404–1415. [Google Scholar] [CrossRef]
  22. Iliescu, D.; Ilie, A.; Ispas, D.; Ion, A. Examining the Psychometric Properties of the Mayer-Salovey-Caruso Emotional Intelligence Test. Eur. J. Psychol. Assess. 2013, 29, 121–128. [Google Scholar] [CrossRef]
  23. Mao, W.C.; Chen, L.F.; Chi, C.H.; Lin, C.H.; Kao, Y.C.; Hsu, W.Y.; Hsieh, J.C. Traditional Chinese version of the Mayer Salovey Caruso Emotional Intelligence Test (MSCEIT-TC): Its validation and application to schizophrenic individuals. Psychiatry Res. 2016, 243, 61–70. [Google Scholar] [CrossRef]
  24. Karim, J.; Weisz, R. Cross-cultural research on the reliability and validity of the Mayer-Salovey-Caruso Emotional Intelligence Test (MSCEIT). Cross Cult. Res. 2010, 44, 374–404. [Google Scholar] [CrossRef]
  25. Libbrecht, N.; Beuckelaer, A.D.; Lievens, F.; Rockstuhl, T. Measurement invariance of the Wong and Law emotional intelligence scale scores: Does the measurement structure hold across far Eastern and European countries? Appl. Psychol. 2014, 63, 223–237. [Google Scholar] [CrossRef]
  26. Podsakoff, P.M.; Organ, D.W. Self-reports in organizational research: Problems and prospects. J. Manag. 1986, 12, 531–544. [Google Scholar] [CrossRef]
  27. Caruso, D.R.; Mayer, J.D.; Salovey, P. Emotional intelligence and emotional leadership. In Multiple Intelligences and Leadership; Riggio, R.E., Murphy, S.E., Pirozzolo, F.J., Eds.; Lawrence Erlbaum Associates Publishers: Mahwah, NJ, USA, 2002; pp. 55–74. [Google Scholar]
  28. Fredrickson, B.L. The role of positive emotions in positive psychology: The broaden-and-build theory of positive emotions. Am. Psychol. 2001, 56, 218–226. [Google Scholar] [CrossRef] [PubMed]
  29. Tsai, W.C.; Chen, C.C.; Liu, H.L. Test of a model linking employee positive moods and task performance. J. Appl. Psychol. 2007, 92, 1570–1583. [Google Scholar] [CrossRef] [PubMed]
  30. Butler, E.A.; Egloff, B.; Wilhelm, F.H.; Smith, N.C.; Erickson, E.A.; Gross, J.J. The social consequences of expressive suppression. Emotion 2003, 3, 48–67. [Google Scholar] [CrossRef]
  31. Gross, J.J.; John, O.P. Individual differences in two emotion regulation processes: Implications for affect, relationships, and well-being. J. Pers. Soc. Psychol. 2003, 85, 348–362. [Google Scholar] [CrossRef]
  32. Beal, D.J.; Weiss, H.M.; Barros, E.; MacDermid, S.M. An episodic process model of affective influences on performance. J. Appl. Psychol. 2005, 90, 1054–1068. [Google Scholar] [CrossRef]
  33. Kanfer, R.; Ackerman, P.L.; Heggestad, E.D. Motivational skills & self-regulation for learning: A trait perspective. Learn. Individ. Differ. 1996, 8, 185–209. [Google Scholar]
  34. Gross, J.J. Emotion and emotion regulation: Personality processes and individual differences. In Handbook of Personality: Theory and Research; John, O.P., Robins, R.W., Pervin, L.A., Eds.; Guilford Press: New York, NY, USA, 2008; pp. 701–724. [Google Scholar]
  35. Mayer, J.D.; Salovey, P. What is emotional intelligence? In Emotional Development and Emotional Intelligence: Educational Implications; Salovey, P., Sluyter, D.J., Eds.; Basic Books: New York, NY, USA, 1997; pp. 3–31. [Google Scholar]
  36. Kintsch, W. The control of knowledge activation in discourse comprehension. In Control of Human Behavior, Mental Processes, and Consciousness: Essays in Honor of the 60th Birthday of August Flammer; Perrig, W.J., Grob, A., Eds.; Lawrence Erlbaum Associates Publishers: Mahwah, NJ, USA, 2000; pp. 137–146. [Google Scholar]
  37. Schmidt, F.L.; Hunter, J. General mental ability in the world of work: Occupational attainment and job performance. J. Pers. Soc. Psychol. 2004, 86, 162–173. [Google Scholar] [CrossRef]
  38. Trentacosta, C.J.; Izard, C.E.; Mostow, A.J.; Fine, S.E. Children’s emotional competence and attentional competence in early elementary school. Sch. Psychol. Q. 2006, 21, 148–170. [Google Scholar] [CrossRef]
  39. Carroll, J.B. Human Cognitive Abilities: A Survey of Factor-Analytic Studies; Cambridge University Press: Cambridge, UK, 1993. [Google Scholar]
  40. Viswesvaran, C.; Ones, D.S. Agreements and disagreements on the role of general mental ability (GMA) in industrial, work, and organizational psychology. Hum. Perform. 2002, 15, 211–231. [Google Scholar]
  41. Cattell, R.B.; Cattell, A.K.S. Culture Fair Intelligence Tests: CFIT; Institute for Personality & Ability Testing: Champaign, IL, USA, 1973. [Google Scholar]
  42. Tsui, A.S.; Pearce, J.L.; Porter, L.W.; Tripoli, A.M. Alternative approaches to the employee-organization relationship: Does investment in employees pay off? Acad. Manag. J. 1997, 40, 1089–1121. [Google Scholar] [CrossRef]
  43. Adler, S.; Campion, M.; Colquitt, A.; Grubb, A.; Murphy, K.; Ollander-Krane, R.; Pulakos, E.D. Getting rid of performance ratings: Genius or folly? A debate. Ind. Organ. Psychol. 2016, 9, 219–252. [Google Scholar] [CrossRef]
  44. James, L.R.; Demaree, R.G.; Wolf, G. Estimating within-group interrater reliability with and without response bias. J. Appl. Psychol. 1984, 69, 85–98. [Google Scholar] [CrossRef]
  45. Bliese, P.D. Within-group agreement, non-independence, and reliability: Implications for data aggregation and analysis. In Multilevel Theory, Research, and Methods in Organizations: Foundations, Extensions, and New Directions; Klein, K.J., Kozlowski, S.W.J., Eds.; Jossey-Bass: San Francisco, CA, USA, 2000; pp. 349–381. [Google Scholar]
  46. Kozlowski, S.W.; Klein, K.J. A multilevel approach to theory and research in organizations: Contextual, temporal, and emergent processes. In Multilevel Theory, Research, and Methods in Organizations: Foundations, Extensions, and New Directions; Klein, K.J., Kozlowski, S.W.J., Eds.; Jossey-Bass: San Francisco, CA, USA, 2000; pp. 3–90. [Google Scholar]
  47. Hu, L.T.; Bentler, P.M. Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification. Psychol. Methods 1998, 3, 424–453. [Google Scholar] [CrossRef]
  48. Mayer, J.D.; Caruso, D.R.; Salovey, P. Emotional intelligence meets traditional standards for an intelligence. Intelligence 1999, 27, 267–298. [Google Scholar] [CrossRef]
  49. Hodzic, S.; Scharfen, J.; Ripoll, P.; Holling, H.; Zenasni, F. How efficient are emotional intelligence trainings: A meta-analysis. Emot. Rev. 2018, 10, 138–148. [Google Scholar] [CrossRef]
  50. Mattingly, V.; Kraiger, K. Can emotional intelligence be trained? A meta-analytical investigation. Hum. Resour. Manag. R. 2019, 29, 140–155. [Google Scholar] [CrossRef]
  51. Mayer, J.D.; Cobb, C.D. Educational policy on emotional intelligence: Does it make sense? Educ. Psychol. Rev. 2000, 12, 163–183. [Google Scholar] [CrossRef]
Figure 1. Conceptual framework (MSCEIT).
Figure 1. Conceptual framework (MSCEIT).
Sustainability 11 02299 g001
Figure 2. Conceptual framework (WLEIS). SEA = Self-Emotion Appraisal, OEA = Others’ Emotion Appraisal.
Figure 2. Conceptual framework (WLEIS). SEA = Self-Emotion Appraisal, OEA = Others’ Emotion Appraisal.
Sustainability 11 02299 g002
Figure 3. EI and CI interaction plotting. EI = emotional intelligence, CI = cognitive intelligence, MSCEIT = Mayer–Salovey–Caruso Emotional Intelligence Test, WLEIS = Wong Law Emotional Intelligence Scale.
Figure 3. EI and CI interaction plotting. EI = emotional intelligence, CI = cognitive intelligence, MSCEIT = Mayer–Salovey–Caruso Emotional Intelligence Test, WLEIS = Wong Law Emotional Intelligence Scale.
Sustainability 11 02299 g003
Table 1. Mean, standard deviation, and correlations among variables.
Table 1. Mean, standard deviation, and correlations among variables.
VariablesMeanSD1234567891011
1. MSCEIT Emotion Perception94.5217.99
2. MSCEIT Emotion Understanding82.1610.090.356 **
3. MSCEIT Emotion Regulation83.0610.830.206 **0.322 **
4. Cognitive Intelligence111.9319.34−0.030−0.066−0.225 **
5. Job Performance2.810.3930.0210.0520.197 **0.269 **
6. WLEIS OEA3.920.991−0.0480.0890.173 *0.0510.221 **
7. WLEIS Emotion Regulation4.110.808−0.0180.1140.184 *0.0040.377 **0.419 **
8. WLEIS SEA4.030.9920.0740.0960.181 *0.0380.353 **0.366 **0.454 **
9. Age29.014.15−0.0160.0200.0730.0360.123−0.113−0.047−0.084
10. Gender0.5390.4990.023−0.008−0.119−0.029−0.103−0.042−0.092−0.095−0.216 **
11. Tenure4.933.56−0.0950.0520.0320.0840.142 *0.0110.071−0.0390.870 **−0.180 *
12. Type of job0.6120.488−0.028−0.005−0.1150.018−0.0450.1150.0480.103−0.102−0.045−0.132
N = 191, MSCEIT = Mayer–Salovey–Caruso Emotional Intelligence Test, WLEIS = Wong Law Emotional Intelligence Scale, OEA = others’ emotion appraisal, SEA = self-emotion appraisal. ** p < 0.01 (2-tailed). * p < 0.05 (2-tailed).
Table 2. Validity checking results.
Table 2. Validity checking results.
CRAVEMSVASV1.2.3.4.
1. SEA0.9020.6970.2700.1870.835
2. OEA0.8910.6740.2350.1420.3620.821
3. Emotion Regulation0.8490.5860.2700.2350.5200.4850.765
4. Job Performance0.8630.5120.2010.1400.3980.2440.4480.715
SEA = self-emotion appraisal, OEA = others’ emotion appraisal, CR = critical ratio, AVE = average variance extracted, MSV = maximum shared variance, ASV = average shared variance.
Table 3. Standardized regression weights.
Table 3. Standardized regression weights.
PathsModel using MSCEITModel using WLEIS
EstimatepEstimatep
Hypothesis 1-MSCEITEmotion PerceptionEmotion Understanding0.3550.002
Emotion UnderstandingEmotion Regulation0.3220.004
Emotion RegulationJob Performance0.2480.001
Hypothesis 2-WLEISSEAEmotion Regulation 0.4180.003
OEAEmotion Regulation 0.3370.003
Emotion RegulationJob Performance 0.4970.004
Hypothesis 3-MSCEITCIEmotion Understanding−0.0550.458
Hypothesis 4-WLEISCISEA 0.0350.623
CIOEA 0.0480.590
Hypothesis 5CIJob Performance0.3110.0010.2680.001
EI * CIJob Performance−0.2010.007−0.1960.041
SEA = self-emotion appraisal, OEA = others’ emotion appraisal, CI = cognitive intelligence, EI = emotional intelligence, MSCEIT = Mayer–Salovey–Caruso Emotional Intelligence Test, WLEIS = Wong Law Emotional Intelligence Scale.

Share and Cite

MDPI and ACS Style

Nguyen, N.N.; Nham, P.T.; Takahashi, Y. Relationship between Ability-Based Emotional Intelligence, Cognitive Intelligence, and Job Performance. Sustainability 2019, 11, 2299. https://doi.org/10.3390/su11082299

AMA Style

Nguyen NN, Nham PT, Takahashi Y. Relationship between Ability-Based Emotional Intelligence, Cognitive Intelligence, and Job Performance. Sustainability. 2019; 11(8):2299. https://doi.org/10.3390/su11082299

Chicago/Turabian Style

Nguyen, Nhu Ngoc, Phong Tuan Nham, and Yoshi Takahashi. 2019. "Relationship between Ability-Based Emotional Intelligence, Cognitive Intelligence, and Job Performance" Sustainability 11, no. 8: 2299. https://doi.org/10.3390/su11082299

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop