Next Article in Journal
Pushing Down on Me: The Paradoxical Role of Self-Leadership in the Context of Work Pressure
Previous Article in Journal
Examining the Perspectives of Gender Development and Inequality: A Tale of Selected Asian Economies
Previous Article in Special Issue
Remote Work in Peru during the COVID-19 Pandemic
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

The Relationship between Occupational Stress, Mental Health and COVID-19-Related Stress: Mediation Analysis Results

by
Giulia Foti
1,2,*,
Giorgia Bondanini
1,2,
Georgia Libera Finstad
1,2,
Federico Alessio
1,2 and
Gabriele Giorgi
1,2
1
Business@Health Laboratory, European University of Rome, 00163 Rome, Italy
2
Department of Human Sciences, European University of Rome, 00163 Rome, Italy
*
Author to whom correspondence should be addressed.
Adm. Sci. 2023, 13(4), 116; https://doi.org/10.3390/admsci13040116
Submission received: 31 January 2023 / Revised: 15 April 2023 / Accepted: 18 April 2023 / Published: 21 April 2023
(This article belongs to the Special Issue COVID-19-Related Mental Health Effects in the Workplace)

Abstract

:
The COVID-19 pandemic led to serious psychological consequences that negatively affect workers’ mental health, leading to post-traumatic symptoms. In this scenario, employees may be exposed to multiple stressors that ultimately drain their resources. Drawing on the Conservation of Resources Theory (COR) and the stress–strain perspective, we analyzed the relationship between different dimensions of work-related stress and psychological distress in a sample of 294 workers in the industrial sector. Specifically, we hypothesized a series of mediation models in which the dimensions of work-related stress are associated with a lower level of mental health directly and indirectly through higher levels of COVID-19-related post-traumatic symptoms. The results partially support the hypotheses, showing that COVID-19-related trauma plays a mediating role between the stress experienced and the resulting decrease in mental health, except in the case of job control and colleague support. These results will hopefully offer insights into possible organizational interventions for the promotion of workers’ well-being in the postpandemic setting.

1. Introduction

1.1. COVID-19 Pandemic and Its Impact on Psychological Distress

The COVID-19 pandemic led to serious psychological consequences that negatively affected the mental health of individuals. Indeed, research shows that COVID-19 is associated with negative outcomes, such as psychological distress. For example, a survey carried out in China on a sample of 600 workers showed that 6.33% experienced anxious states (women had a higher prevalence), while 17.17% experienced depressive symptoms (Wang et al. 2020). Serious mental health consequences were reported especially for healthcare workers. In fact, an Italian study conducted on a sample of 1379 workers showed that 49.38% of the respondents reported post-traumatic symptoms, 24.73% exhibited symptoms of depression, 19.80% reported anxiety symptoms, 8.27% exhibited problems of insomnia, and finally, 21.90% reported high perceived stress (Rossi et al. 2020). A cross-sectional survey conducted in China also showed that 40.2% of the sample had significant symptoms related to post-traumatic stress disorder. The percentages of depression, anxiety, and stress states were 13.6, 13.9, and 8.6, respectively (Si et al. 2020). While healthcare workers are the population most affected by the psychosocial consequences of the pandemic, negative outcomes were also observed for other professional groups. For example, a study conducted on a sample of bankers showed that during the pandemic, almost all employees were experiencing anxiety, depression, and stress, and that 11.1% of the sample ranged from severely stressed to extremely stressed (Yasmin et al. 2022). In addition, other research highlighted that education and social work professionals, public administrative employees, bank employees, and employees in other jobs at high or potential risk of COVID-19 infection were found to be at higher risk of depressive and anxiety symptoms. Furthermore, factors contributing to mental distress may be inadequate protective equipment, reduced relationship with coworkers, and work-privacy conflicts (Casjens et al. 2022).

1.2. COVID-19 as a Traumatic Event: COVID-19-Related Stress

The association between COVID-19 and post-traumatic stress disorder is becoming a topic to which researchers around the world are turning their attention. Post-traumatic stress disorder can generally occur after exposure to a traumatic event. According to Cramer and colleagues, post-traumatic stress reaction develops following five phases: (1) objective exposure from which subjective reactions arise; (2) the formation of a memory network and thus the subjective processing of the traumatic event and the interpretation of the stimuli associated with it; (3) intrusion, i.e., the memories of the traumatic event which activate the fear network, causing hyperarousal and avoidance mechanisms; (4) avoidance, i.e., coping strategies implemented in order to reduce the discomfort caused by the intrusive phase; and (5) the outcomes, i.e., the resolution of the network that manifests itself with high symptomatology (Creamer et al. 1992). Then, there are three nosographic criteria for understanding trauma-related psychopathology, which will have unique characteristics within the pandemic framework: objectively defined trauma, objectively defined exposure, and subjectively reported reactions (North et al. 2021). Regarding the novel coronavirus and objectively defined trauma, we recognize that COVID-19 is a disease that occurs ‘naturally’, and as such, is not mentioned among the traumatic events in the DSM-5, but, as we shall see, it produces clearly visible traumatic effects and generates important secondary stressors such as financial and social losses, reduced opportunities to socialize with friends and loved ones, and the interruption of important activities of daily life (North et al. 2021). Thus, exposure to the virus is a highly stressogenic factor for both workers and the general population. Finally, with regard to subjective emotional reactions, distressing emotional states could lead to psychopathological outcomes (North et al. 2021). As for the hypothesis that COVID-19 can be considered a traumatic factor, the research by Bridgland and colleagues found that 13.2% of the participants probably had post-traumatic symptoms even if the types of exposure to COVID-19 did not fall within the diagnostic criteria. In addition, the emotional impact of the events experienced/predicted as worse was found to be a predictor of PTSD (Bridgland et al. 2021). Furthermore, the results of the study by Sanchez-Gomez and colleagues, conducted on a general sample of workers, found a statistically significant correlation between fear of COVID-19 and the three dimensions of PTSD: intrusion, hyperarousal, and avoidance. In addition, the results showed that hyperarousal plays a mediating role in the relationship between intrusive thoughts and individual outcomes, such as fear of COVID-19 and mental health (Sanchez-Gomez et al. 2021).

1.3. Work-Related Stress: The Five Dimensions and Their Impact on Psychological Distress

The National Institute for Occupational and Safety and Health (NIOSH) (Schill and Chosewood 2013) defines work-related stress as a series of “harmful physical and emotional reactions that occur when job demands are not commensurate with the worker’s capabilities, resources or needs of the workers”. The type of occupation and the organizational conditions are important factors that can generate serious psychosocial consequences (Bonde 2008). There are several factors that contribute to occupational stress. The dimension termed as “job demands (JD)” is linked to the effort required operationalized as workload, both physical and psychological; rhythms of work; consistency of requests (Bakker and Demerouti 2007). JD includes high pressure in the workplace, an unfavorable working environment, and onerous emotional effort in interactions with customers (Karasek 1979). The decision latitude or job control dimension is understood as the freedom perceived by the worker in responding to work tasks. It therefore refers to mastery and discretion within the work environment. A third dimension to be considered, which is also included in the Job Demand Control Support (JDCS) model, is social support, that is, social support from supervisors and colleagues. Job control can act as a moderator, being able to mitigate the devastating effects of stress (Karasek and Theorell 1990) and protect from job strain (Haines et al. 1991).
Therefore, psychological distress and adverse health-related consequences arise when there is an imbalance between job demands and workers’ resources, for example, in terms of lack of job control and lack of social support (Marchand and Durand 2011). Another factor to consider is role ambiguity, defined by Kahn, Wolfe, Quinn, Snoek, and Rosenthal as the ambiguity experienced by workers regarding the parts of their role which are not clearly defined in terms of behavior, tasks, priorities, and/or criteria to be met (Kahn et al. 1964).
In summary, occupational stress can by analyzed through the following dimensions: supervisor support, colleague support, job demands, job control, and role ambiguity. These factors are closely associated with the employee’s psychological health. In fact, colleague and supervisor support proved to be important protective factors in the workplace (Marchand and Blanc 2010). Indeed, both low social support and lack of supervisor support are associated with increased psychological distress (Inoue et al. 2022) (Dennerlein et al. 2021). This is especially true for teleworkers, since insufficient supervisor support is associated with a high risk of psychological distress in this population (Kikunaga et al. 2023). Therefore, actions by leaders as supervisors are important in increasing positive employee attitudes and behaviors in the organization (Cahyadi et al. 2022a, 2022b). Moreover, low job control in the workplace can generate significative psychological distress (Elovainio et al. 2007; Elliott et al. 2017) as well as role ambiguity (Choi et al. 2011). Finally, in line with what Karasek supposes (Karasek 1979), research carried out in various organizational contexts has shown that a high level of job demands coupled with low levels of job control is associated with increased psychological distress (Karasek and Theorell 1990; Xie et al. 2021), while emotion-focused coping strategies act as a mediator in this relationship (Oshio et al. 2021).

1.4. Work-Related Stress as a Facilitator of the Onset of COVID-19 Stress Symptoms

The presence of work-related stress may promote the development of COVID-19-related post-traumatic symptoms. Work-related stress has a significant impact on the well-being of the worker (Ben-Ezra and Hamama-Raz 2021), as does COVID-19-related stress (Bridgland et al. 2021), as anticipated above. The individual may perceive exposure to a given stressful event as challenging or threatening, as evidenced by Hofboll’s Conservation of Resources (COR) theory. COR theory, proposed as motivational theory, asserts that personal and social resources moderate the potential negative impact of stressful situations in the individual’s life and focuses on the use of resources in these circumstances. The basic tenet of the theory states that individuals are motivated to acquire and protect their resources. In this perspective, stress is the result of the subjective perception of events as exceeding one’s resources and of environmental and objective situations that threaten or may induce resource depletion (Lazarus and Folkman 1984; Giorgi et al. 2020; Hobfoll 1998). Following the first corollary, individuals with more resources are more likely to acquire new ones and are less vulnerable to resource loss, while the opposite trend can be observed in the case of individuals with a depleted resource pool (Hobfoll 1988). Furthermore, since loss is more valued than gain and stress occurs when resources are lost, organizations and individuals can experience loss spirals where with each loss there is an increase in the magnitude of the effect (second corollary) (Hobfoll 1998, 2001; Halbesleben et al. 2014). In addition, another possible explanation lies in the physiological perspective: exposure to high levels of glucocorticoids, a class of hormones that are activated after exposure to stressful situations, could have an impact on the onset of psychopathologies.
In this regard, we can assume that an individual already suffering from work-related stress and thus with reduced personal resources may experience a further loss of resources when faced with a stressful event such as COVID-19, thus developing post-traumatic symptoms. Additionally, workers in the early stages of work-related stress coping with a traumatic event, such as COVID-19, may have elevated levels of stress hormones that facilitate the development of post-traumatic stress symptoms (Marin et al. 2011). In line with this perspective, research by Marin and colleagues (Marin et al. 2011) demonstrated that work-related stress, acting on individual vulnerability, triggers the onset of various pathologies, including post-traumatic stress disorder. It is therefore hypothesized that the vulnerability induced by the presence of work-related stress makes the individual more vulnerable to traumatic experiences, in this case related to COVID-19. Hence, this study aims to investigate how work-related stress dimensions—lack of supervisor support, lack of colleague support, job demands, job control, and role ambiguity—can have an impact on psychological distress and how COVID-19-related stress can act as a mediator in these relationships.
Based on this framework, we hypothesize:
H1a. 
Lack of supervisor support is associated with a higher level of psychological distress.
H1b. 
COVID-19-related post-traumatic stress acts a mediator between the lack of supervisor support and psychological distress.
H2a. 
Lack of colleague support is associated with a higher level of psychological distress.
H2b. 
COVID-19-related post-traumatic stress acts a mediator between lack of colleague support and psychological distress.
H3a. 
Job demands are associated with a higher level of psychological distress.
H3b. 
COVID-19-related post-traumatic stress acts a mediator between job demands and psychological distress.
H4a. 
Lack of job control is associated with a higher level of psychological distress.
H4b. 
COVID-19-related post-traumatic stress acts a mediator between lack of job control and psychological distress.
H5a. 
Role ambiguity is associated with a higher level of psychological distress.
H5b. 
COVID-19-related post-traumatic stress acts a mediator between role ambiguity and psychological distress.

2. Materials and Methods

2.1. Sample and Procedure

This study was conducted as a part of the work-related stress (WRS) risk assessment of a company in the industrial sector between 2020 and 2021. Employees were informed about the aim of the research, the data collection method, and the voluntary and confidential nature of their participation. This preliminary phase took place together with the company’s Safety Office, which promoted the initiative and contacted the workers via the company intranet, without influencing the process. All participants gave their informed consent. The whole process was conducted in accordance with the Declaration of Helsinki and, given the observational nature of the study, and in the absence of any involvement of therapeutic medication, no formal approval of the Institutional Review Board of the local Ethics Committee was required. The sample consists of 294 workers (response rate = 70%). The participants were classified according to 6 groups. The majority (33.5%) performed the activity of G2. The sample is made up of 78.8% males and 21.2% females. A total of 39.6% of respondents had worked between 4–9 years in their current company; 22.5% for 10–19 years. Furthermore, 76.4% of participants were blue-collar.
The participants were classified according to the following 6 groups:
  • Group 1: CNC machine operators, panel operators, and frame operators;
  • Group 2: quality control officers and cleaning, repair, delivery, and assembly personnel;
  • Group 3: employees, supervisors, and managers;
  • Group 4: warehousing personnel and depot operators;
  • Group 5: production line personnel and furniture preparation personnel;
  • Group 6: maintenance technician and prototype operators.

2.2. Measurement

Work-related stress was measured using the scales included in the Stress Questionnaire (Mucci et al. 2015). In this model, five subscales (with a 5-point Likert scale) were included: job demands, lack of job control, lack of supervisor lack of support, colleague support, and role. The “job demands’’ includes 6 items (e.g., “I have unreasonable deadlines”), the “lack of job control” subscale includes 5 items (e.g., “ I can decide when to take a break” reverse coded), the “lack of supervisor support” subscale includes 4 items (e.g., “I am encouraged by my manager/supervisor” reverse coded), the “lack of colleague support” subscale includes 5 items (e.g., “I get the help and support I need from my colleagues” reverse coded). Cronbach’s alpha values for the scales were 0.63, 0.66, 0.78, 0.66, and 0.68, respectively.
Psychological distress was measured using the General Health Questionnaire (GHQ-12) (Fraccaroli et al. 1991). This questionnaire is used to identify nonspecific psychiatric disorders by evaluating the current state of the worker and how this differs from the usual state. It is formed by 12 items with a 4-point Likert scale. Psychological distress can be calculated as a total score or according to three subscales: loss of security, (e.g., ‘thinking of self as worthless’), anxiety (e.g., ‘feeling unhappy and depressed’), and social dysfunction (e.g., ‘feeling unable to make decisions’). The internal reliability showed a satisfactory value of 0.75.
COVID-19-related post-traumatic stress was measured using the Impact of Event Scale in its shorter version (IES-6) (Giorgi et al. 2015). The scale used COVID-19 as the specific traumatic event. The items refer to feelings of distress experienced over the previous 7 days following a specific traumatic situation. Specifically, the IES-6 includes 2 items for each of the post-traumatic stress dimension: intrusion (e.g., “Since the beginning of the COVID-19 emergency, I thought about it when I didn’t mean to”), avoidance (e.g., “Since the beginning of the COVID-19 emergency, I was aware that I still had a lot of feelings about it, but I didn’t deal with them”), and hyperarousal (e.g., “Since the beginning of the COVID-19 emergency, I had trouble concentrating”). Cronbach’s alpha had a value of 0.80.

2.3. Statistical Analyses

The statistics software IBM SPSS® (v. 25, package for Windows, SPSS Inc., Chicago, IL, USA) was used to analyze the data. First, we performed descriptive statistics and reliability analyses. After analyzing Pearson’s correlations, we conducted a mediation analysis to test the hypotheses. For this purpose, the macro PROCESS 3.3 was applied (model 4). Following a bootstrap method with 10,000 data samples that generated 95% bias-corrected confidence intervals, it was possible to examine conditional models to predict direct and indirect effects between the variables. A path is statistically significant if the associated 95% confidence interval (CI; bias corrected) does not include zero (Hayes 2002).

3. Results

3.1. Descriptive Analyses

As shown in Table 1, there is a statistically significant correlation between COVID-19-related post-traumatic stress and psychological distress (r = 0.37, p < 0.001) and between job demands and COVID-19-related post-traumatic stress (r = 0.19, p < 0.01), as expected. Similarly, there is a statistically significant correlation between psychological distress and role ambiguity (r = 0.34, p < 0.001) and lack of supervisor support (r = 0.33, p < 0.01). The same results were found for the correlations between the five subdimensions of WRS (i.e., lack of supervisor support, lack of colleague support, job demands, lack of job control, and role ambiguity), as expected. Finally, the results show a good level of reliability, with Cronbach’s alpha coefficients between 0.63 and 0.85.

3.2. Mediation Analyses

Regarding the indirect effect, lack of supervisor support has a significant effect on COVID-19 related-post-traumatic stress (β = 0.12; p < 0.01; 95% CI = 0.02, 0.21) [Figure 1], which, in turn, shows a significant effect on psychological distress (β = 2.44; p < 0.001; 95% CI = 1.64, 3.23). As can be seen in Figure 2, there is a significant direct effect of lack of supervisor support on psychological distress (β = 1.70; p < 0.001: 95% CI = 1.06, 2.33). The bootstrap confidence interval for the indirect effect (ab) using 5000 bootstrap samples was entirely below zero (β = 0.29, SE = 0.14, CI 95% [0.02, 0.61]). Therefore, the results show a partial mediation effect of COVID-19-related post-traumatic stress between lack of supervisor support and psychological distress. Lack of supervisor support and COVID-19-related post-traumatic stress can explain 21.67% of the variance (R2 = 0. 2167; p < 001).
Regarding the indirect effect, lack of colleague support has a nonsignificant effect on COVID-19-related post-traumatic stress (β = 0.02; p > 0.05; 95% CI = −0.10, 0.15), which, in turn, shows a significant effect on psychological distress (β = 2.71; p < 0.001; 95% CI = 1.92, 3.50). As can be seen in Figure 1 there is a significant direct effect of lack of colleague support on psychological distress (β = 2.09; p < 0.001: 95% CI = 1.25, 2.92) [Figure 2]. The bootstrap confidence interval for the indirect effect (ab) using 5000 bootstrap samples was entirely below zero (β = 0.06, SE = 0.20, CI 95% [−0.33, 0.46]). Therefore, the results show no significant mediating effect of COVID-19-related post-traumatic stress between lack of colleague support and psychological distress. Lack of colleague support and COVID-19-related post-traumatic stress can explain 20.69% of the variance (R2 = 0.2069; p < 0.001).
Regarding the indirect effect, job demands has a significant effect on COVID-19-related post-traumatic stress (β = 0.20; p < 0.01; 95% CI = 0.08, 0.33) [Figure 3], which, in turn, shows a significant effect on psychological distress (β = 2.43; p < 0.001; 95% CI = 1.61, 3.24). As can be seen in Figure 1, there is not a significant direct effect of job demands on psychological distress (β = 1.84; p < 0.001: 95% CI = 0.97, 2.70). The bootstrap confidence interval for the indirect effect (ab) using 5000 bootstrap samples was entirely below zero (β = 0.48, SE = 0.18, CI 95% [0.17, 0.91]). Therefore, the results show a mediation effect of COVID-19-related post-traumatic stress between job demands and mental health. Job demands and COVID-19-related post-traumatic stress can explain 18.90% of the variance (R2 = 0.1890; p < 0.001).
Regarding the indirect effect, lack of job control has a significant effect on COVID-19-related post-traumatic stress (β = 0.12; p > 0.05; 95% CI = −0.00, 0.24) [Figure 4], which, in turn, shows a significant effect on psychological distress (β = 2.56; p < 0.001; 95% CI = 1.75, 3.37). As can be seen in Figure 1, there is not a significant direct effect of lack of job control on psychological distress (β = 1.68; p < 0.001: 95% CI = 0.90, 2.50). The bootstrap confidence interval for the indirect effect (ab) using 5000 bootstrap samples was entirely below zero (β = 0.30, SE = 0.18, CI 95% [−0.00, 0.69]). Therefore, the results do not show a significant mediation effect of COVID-19-related post-traumatic stress between lack of job control and psychological distress. Lack of job control and COVID-19-related post-traumatic stress can explain 18.59% of the variance (R2 = 0.1859; p < 0.001).
Regarding the indirect effect, role ambiguity has a significant effect on COVID-19-related post-traumatic stress (β = 0.17; p < 0.01; 95% CI = 0.04, 0.30) [Figure 5], which, in turn, shows a significant effect on psychological distress (β = 2.43; p < 0.001; 95% CI = 1.63, 3.22). As can be seen in Figure 1, there is also a significant direct effect of role ambiguity on psychological distress (β = 2.45; p < 0.001: 95% CI = 1.56, 3.34). The bootstrap confidence interval for the indirect effect (ab) using 5000 bootstrap samples was entirely below zero (β = 0.40, SE = 0.19, CI 95% [0.04, 0.81]). Therefore, the results show significant partial mediation effect of COVID-19-related post-traumatic stress between role ambiguity and psychological distress. Role ambiguity and COVID-19-related post-traumatic stress can explain 22.14% of the variance (R2 = 0.2214; p < 0.001).

4. Discussion

The aim of this research was to investigate theoretically, through the support of previous literature, and empirically, through the development of statistical models, the influence of WRS on psychological distress and the mediating role of COVID-19-related post-traumatic stress. The focus of this research was to investigate the link between the subdimensions of work-related stress (i.e., job demands, job control, support from colleagues and superiors, and ambiguity) and psychological distress and to assess how COVID-19-related post-traumatic stress could be a mediator in this relationship. As previously mentioned, the SARS-Cov-2 pandemic had significant negative effects on the psychological health of individuals (Wang et al. 2020; Rahman et al. 2020). Therefore, in line with Karasek’s model, studies showed that a high level of job demands is associated with increased psychological distress (Inoue et al. 2022; Dennerlein et al. 2021). Furthermore, COVID-19, interpreted as an indirect traumatic event, can profoundly affect the mental health of workers who already suffer from work-related stress (Lazarus and Folkman 1984; Giorgi et al. 2020; Hobfoll 1998). In this regard, our framework is based on COR theory. According to this conceptualization, both personal and social resources can moderate the negative impact of stressful events, while workers with low resource levels can deplete their reservoir in the face of another stressful event, therefore leading to greater psychological distress (Lazarus and Folkman 1984; Giorgi et al. 2020; Hobfoll 1998). Therefore, the proposed models suggest that COVID-19-related stress can act as a mediator between occupational stress and mental health. Adopting a differential approach, we considered different work-related stress subdimensions: lack of supervisor support, lack of colleague support, job demands, lack of job control, and role ambiguity. The most relevant mediating effect of COVID-19-related post-traumatic stress emerges between work demands, lack of job control, and mental health. It is in fact possible to highlight a relationship of total mediation between the variables confirmed by the nonsignificance of the direct effect of the job demands and lack of job control on mental health (Voltmer et al. 2021).
The findings of this study highlight the pivotal role of COVID-19-related post-traumatic stress as an underlying mechanism in the relationship between job demand, lack of job control, and psychological distress. This is also in line with studies proposed by d’Ettorre et al. (2021) and Giannopoulou et al. (2021), and the research of (Britt et al. 2021; d’Ettorre et al. 2021; Giannopoulou et al. 2021).
The mediating role of COVID-19 in the relationship between job demands and psychological distress could also be explained by the fact that psychological distress and negative health consequences appear when there is an imbalance between demands and resources (Marchand and Durand 2011), as in the case of an event such as COVID-19 (Inserire CIT). This turns out to be a focal point of our research, as it underscores the central role of COVID-19-related stress in explaining the relationship between job demands and psychological distress.
In addition, lack of support from one’s supervisor and colleagues are also major determinants of workers’ psychological health (Voltmer et al. 2021). In fact, COVID-19-related stress partially mediated the relationship between the lack of supervisor support and psychological distress.
On one hand, it was shown that support from colleagues and supervisors is an important protective factor in the workplace (Marchand and Blanc 2010). Indeed, work-related stress and its subdimensions lead to a loss of individual resources, which in the presence of COVID-19 post-traumatic stress can generate a loss spiral and erode mental health (Minihan et al. 2022).
On the other hand, COVID-19-related post-traumatic symptoms can mediate the relationship between relational stress and mental health because workers can be perceived more negatively due to fears, social pressure, etc. (d’Ettorre et al. 2021; Giannopoulou et al. 2021; Britt et al. 2021).
As anticipated above, experiencing high levels of role ambiguity can generate distress (Choi et al. 2011), while post-traumatic stress related to COVID-19 can intervene in this model by increasing individual vulnerability and leading to greater psychological distress (Choi et al. 2011). Indeed, this study highlighted some elements associated with COVID-19-related post-traumatic stress that have a significant impact on workers’ mental health. Accordingly, the study by Said and colleagues shows that work stress associated with COVID-19 is an essential indicator of mental illness, as it can lead to anxiety and depression (Di Prinzio et al. 2021).
Indeed, the SARS-CoV-2 pandemic, while not perfectly meeting the diagnostic criteria for PTSD, led to severe psychological and post-traumatic consequences (Wang et al. 2020; Lakhan et al. 2020). Therefore, exposure to the virus is believed to be a highly stressful factor for workers, especially those who might suffer from work-related stress. In fact, COVID-19-related post-traumatic stress may be fueled by the presence of work-related stress (WRS). Both, as described, have a significant impact on workers’ well-being. Therefore, work-related stress not only has a direct effect on mental well-being, but can also have an indirect effect through COVID-19-related post-traumatic stress (Greenberg 2020).
In conclusion, workers dealing with WRS may have depleted resources and suffer further losses when confronted with a stressful event such as COVID-19, thus leading to lower levels of mental health (Britt et al. 2021).
In summary, this study seeks to expand the research strands on work-related stress and post-traumatic stress by analyzing how these variables are related to psychological distress. Analyzing the antecedents of workers’ mental health is critical to the design of interventions and the development of policies to promote workers’ well-being.

4.1. Limits and Future Research

The present study has certain limitations that need to be considered for future research. Firstly, the study has a cross-sectional design that does not allow causal inferences about the relationship between the variables. Future research should apply a longitudinal design to replicate these findings and investigate how COVID-19-related post-traumatic stress can mediate the relationship between work-related stress and psychological distress. Secondly, another limitation concerns the sampling strategy. Specifically, we adopted a nonprobabilistic sampling strategy which does not allow for full generalization of the results. Another possible limitation is the exclusive use of self-report measures that might increase the risk of common method bias, although there is still no unanimity on the magnitude of its effects (de Lange et al. 2003; Spector 2006). Nevertheless, we followed the recommendations of Podskaoff et al., such as providing participants with information about the anonymity of their response and the absence of wrong answers, and separating predictor and criterion variables sections in survey questionnaires. Another possible solution lies in the use of multimethod data and objective measurements. For example, future studies could adopt physiological measures such as cortisol levels or heart rate variability (HRV) to assess psychological distress (Podsakoff et al. 2003). Moreover, the sample consists of 80% men, and therefore, future research should adopt a more equal sampling between men and women. Furthermore, because the sample consists of 80% fixed-term contract employees, future research might consider using a more balanced sample. In addition, future studies should deepen this topic by analyzing different organizational realities. Despite the limitations mentioned above, this research offers interesting insights into the relationship between workers’ perceived COVID-19-related post-traumatic stress, mental health, and work-related stress, considering its individual dimensions.

4.2. Practical Implications

The results suggest that work-related psychosocial risks play a decisive role in workers’ well-being (Bridgland et al. 2021). This research underlines the importance of providing adequate support from the social environment, defining clear boundaries between roles and involving workers in the planning of different activities. Furthermore, the emphasis is placed on job demands, which, if not in line with individual resources in terms of technical and relational skills, can lead to general discomfort and therefore become a source of stress. If adequate workplace stress prevention measures are not taken, the distress state will make workers more concerned about the global pandemic situation, and stressors will inevitably have a significant impact on mental health (Minihan et al. 2022).
To face the challenges arising from the pandemic, the role of specific figures such as psychologists and support services is essential. Indeed, numerous studies have reported the drastic impact of previous pandemics on mental health in terms of the development of PTSD, depression, and generalized anxiety (Hobfoll 1988).

5. Conclusions

The present study showed how the SARS-CoV-2 pandemic affected the relationship between work-related stress and mental health. Our results therefore extend the scientific literature showing how the pandemic has played a significant role in organizations.
This mediation model could be useful for analyzing how future social and health stressors, such as a pandemic, can affect workers’ mental health, especially in a scenario filled with economic and socio-political instability. Future research could address the development of techniques and best practices aimed at improving colleague support, teamwork, supportive supervision, training, and worker empowerment in order to increase the perception of job control and reduce the strain arising from daily tasks.

Author Contributions

Conceptualization, G.B., G.F. and F.A.; methodology, G.L.F.; writing—original draft preparation, G.B., G.F., G.L.F. and F.A.; writing—review and editing, G.B., G.F., G.L.F. and F.A.; supervision, G.G.; project administration, G.G. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

The whole process was conducted in accordance with the Declaration of Helsinki and, given the observational nature of the study, and in the absence of any involvement of therapeutic medication, no formal approval of the Institutional Review Board of the local Ethics Committee was required.

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

Data is contained within the article.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Bakker, Arnold B., and Evangelia Demerouti. 2007. The Job Demands-Resources model: State of the art. Journal of Managerial Psychology 22: 309–28. [Google Scholar] [CrossRef]
  2. Ben-Ezra, Menachem, and Yaira Hamama-Raz. 2021. Social Workers during COVID-19: Do Coping Strategies Differentially Mediate the Relationship between Job Demand and Psychological Distress? The British Journal of Social Work 51: 1551–67. [Google Scholar] [CrossRef]
  3. Bonde, Jens Peter Ellekilde. 2008. Psychosocial factors at work and risk of depression: A systematic review of the epidemiological evidence. Occupational and Environmental Medicine 65: 438–45. [Google Scholar] [CrossRef] [PubMed]
  4. Bridgland, Victoria M. E., Ella K. Moeck, Deanne M. Green, Taylor L. Swain, Diane M. Nayda, Lucy A. Matson, Nadine P. Hutchison, and Melanie K. T. Takarangi. 2021. Why the COVID-19 pandemic is a traumatic stressor. PLoS ONE 16: e0240146. [Google Scholar] [CrossRef]
  5. Britt, Thomas W., Marissa L. Shuffler, Riley L. Pegram, Phoebe Xoxakos, Patrick J. Rosopa, Emily Hirsh, and William Jackson. 2021. Job Demands and Resources among Healthcare Professionals during Virus Pandemics: A Review and Examination of Fluctuations in Mental Health Strain during COVID-19. Applied Psychology 70: 120–49. [Google Scholar] [CrossRef]
  6. Cahyadi, Afriyadi, Diah Natalisa, József Poór, Badia Perizade, and Katalin Szabó. 2022a. Predicting the Relationship between Green Transformational Leadership, Green Human Resource Management Practices, and Employees’ Green Behavior. Administrative Sciences 13: 5. [Google Scholar] [CrossRef]
  7. Cahyadi, Afriyadi, József Poór, and Katalin Szabó. 2022b. Pursuing Consultant Performance: The Roles of Sustainable Leadership Styles, Sustainable Human Resource Management Practices, and Consultant Job Satisfaction. Sustainability 14: 3967. [Google Scholar] [CrossRef]
  8. Casjens, Swaantje, Dirk Taeger, Thomas Brüning, and Thomas Behrens. 2022. Altered mental distress among employees from different occupational groups and industries during the COVID-19 pandemic in Germany. Journal of Occupational and Environmental Medicine 64: 874–80. [Google Scholar] [CrossRef]
  9. Choi, BongKyoo, Per-Olof Östergren, Catarina Canivet, Mahnaz Moghadassi, Sara Lindeberg, Robert Karasek, and Sven-Olof Isacsson. 2011. Synergistic interaction effect between job control and social support at work on general psychological distress. Internationales Archiv Fur Arbeitsmedizin 84: 77–89. [Google Scholar] [CrossRef]
  10. Creamer, Mark, Philip Burgess, and Phillipa Pattison. 1992. Reaction to trauma: A cognitive processing model. Journal of Abnormal Psychology 101: 452–59. [Google Scholar] [CrossRef]
  11. de Lange, Annet H., Toon W. Taris, Michiel A. J. Kompier, Irene L. D. Houtman, and Paulien M. Bongers. 2003. The very best of the millennium: Longitudinal research and the demand-control-(support) model. Journal of Occupational Health Psychology 8: 282–305. [Google Scholar] [CrossRef] [PubMed]
  12. d’Ettorre, Gabriele, Giancarlo Ceccarelli, Letizia Santinelli, Paolo Vassalini, Giuseppe Pietro Innocenti, Francesco Alessandri, Alexia E. Koukopoulos, Alessandro Russo, Gabriella dEttorre ’, and Lorenzo Tarsitani. 2021. Post-Traumatic Stress Symptoms in Healthcare Workers Dealing with the COVID-19 Pandemic: A Systematic Review. International Journal of Environmental Research and Public Health 18: 601. [Google Scholar] [CrossRef]
  13. Dennerlein, Jack T., Mara Eyllon, Suzanne Garverich, Daniel Weinstein, Justin Manjourides, Steven P. Vallas, and Alisa K. Lincoln. 2021. Associations Between Work-Related Factors and Psychological Distress Among Construction Workers. Journal of Occupational and Environmental Medicine 63: 1052–57. [Google Scholar] [CrossRef] [PubMed]
  14. Di Prinzio, Reparata Rosa, Giorgia Bondanini, Federica de Falco, Maria Rosaria Vinci, Vincenzo Camisa, Annapaola Santoro, and Nicola Mucci. 2021. Need for Psychological Support and Disability Management Programs during and after the COVID-19 Pandemic in Italy: Preliminary Findings from a Hospital-Based Occupational Health. Journal of Health and Social Sciences 6: 367–78. [Google Scholar] [CrossRef]
  15. Elliott, Kate-Ellen J., John Rodwell, and Angela J. Martin. 2017. Aged care nurses’ job control influence satisfaction and mental health. Journal of Nursing Management 25: 558–68. [Google Scholar] [CrossRef]
  16. Elovainio, Marko, Mika Kivimäki, Ellen Ek, Jussi Vahtera, Teija Honkonen, Anja Taanila, Juha Veijola, and Marjo-Riitta Järvelin. 2007. The effect of pre-employment factors on job control, job strain and psychological distress: A 31-year longitudinal study. Social Science & Medicine 65: 187–99. [Google Scholar] [CrossRef]
  17. Fraccaroli, Franco, Marco Depolo, and Guido Sarchielli. 1991. The Use of Goldberg’s General Health Questionnaire in a Study of Unemployed Young People. Giunti Organizzazioni Speciali 197: 13–19. [Google Scholar]
  18. Giannopoulou, Ioanna, Sofia Galinaki, Evangelia Kollintza, Maria Adamaki, Stylianos Kympouropoulos, Evangelos Alevyzakis, and Konstantinos Tsamakis. 2021. COVID-19 and post-traumatic stress disorder: The perfect ‘storm’ for mental health (Review). Experimental and Therapeutic Medicine 22: 1162. [Google Scholar] [CrossRef]
  19. Giorgi, Gabriele, Francisco Saverio Fiz Perez, Andrea Castiello D’Antonio, Nicola Mucci, Claudia Ferrero, Vincenzo Cupelli, and Giulio Arcangeli. 2015. Psychometric properties of the Impact of Event Scale-6 in a sample of victims of bank robbery. Psychology Research and Behavior Management 8: 99–104. [Google Scholar] [CrossRef]
  20. Giorgi, Gabriele, Luigi Isaia Lecca, Federico Alessio, Georgia Libera Finstad, Giorgia Bondanini, Lucrezia Ginevra Lulli, Giulio Arcangeli, and Nicola Mucci. 2020. COVID-19-Related Mental Health Effects in the Workplace: A Narrative Review. International Journal of Environmental Research and Public Health 17: 7857. [Google Scholar] [CrossRef]
  21. Greenberg, Neil. 2020. Mental health of health-care workers in the COVID-19 era. Nature Reviews Nephrology 16: 425–26. [Google Scholar] [CrossRef] [PubMed]
  22. Haines, Valerie A., Jeanne S. Hurlbert, and Catherine Zimmer. 1991. Occupational Stress, Social Support, and the Buffer Hypothesis. Work and Occupations 18: 212–35. [Google Scholar] [CrossRef]
  23. Halbesleben, Jonathon R. B., Jean-Pierre Neveu, Samantha C. Paustian-Underdahl, and Mina Westman. 2014. Getting to the “COR”: Understanding the Role of Resources in Conservation of Resources Theory. Journal of Management 40: 1334–64. [Google Scholar] [CrossRef]
  24. Hayes, Steven C. 2002. Acceptance, mindfulness, and science. Clinical Psychology: Science and Practice 9: 101–106. [Google Scholar] [CrossRef]
  25. Hobfoll, Stevan E. 1988. The Ecology of Stress. New York: Taylor & Francis. [Google Scholar]
  26. Hobfoll, Stevan E. 1998. Stress, Culture, and Community. Boston: Springer. [Google Scholar] [CrossRef]
  27. Hobfoll, Stevan E. 2001. The Influence of Culture, Community, and the Nested-Self in the Stress Process: Advancing Conservation of Resources Theory. Applied Psychology 50: 337–421. [Google Scholar] [CrossRef]
  28. Inoue, Reiko, Hiroyuki Hikichi, Akiomi Inoue, Yuko Kachi, Hisashi Eguchi, Kazuhiro Watanabe, Yumi Arai, Noboru Iwata, and Akizumi Tsutsumi. 2022. Workplace Social Support and Reduced Psychological Distress: A 1-Year Occupational Cohort Study. Journal of Occupational and Environmental Medicine 64: e700–e704. [Google Scholar] [CrossRef]
  29. Kahn, Robert L., Donald M. Wolfe, Robert P. Quinn, J. Diedrick Snoek, and Robert A. Rosenthal. 1964. Occupational Stress: Studies in Role Conflict and Ambiguity. New York: Wiley. [Google Scholar]
  30. Karasek, Robert A. 1979. Job Demands, Job Decision Latitude, and Mental Strain: Implications for Job Redesign. Administrative Science Quarterly 24: 285. [Google Scholar] [CrossRef]
  31. Karasek, Robert A., and Töres Theorell. 1990. Healthy Work: Stress, Productivity, and the Reconstruction of Working Life. New York: Basic Books. [Google Scholar]
  32. Kikunaga, Kazuki, Akinori Nakata, Mami Kuwamura, Kiminori Odagami, Kosuke Mafune, Hajime Ando, Keiji Muramatsu, Seiichiro Tateishi, and Yoshihisa Fujino. 2023. Psychological Distress, Japanese Teleworkers, and Supervisor Support During COVID-19. Journal of Occupational and Environmental Medicine 65: e68–e73. [Google Scholar] [CrossRef]
  33. Lakhan, Ram, Amit Agrawal, and Manoj Sharma. 2020. Prevalence of Depression, Anxiety, and Stress during COVID-19 Pandemic. Journal of Neurosciences in Rural Practice 11: 519–25. [Google Scholar] [CrossRef]
  34. Lazarus, Richard, and Susan Folkman. 1984. Stress, Appraisal, and Coping. New York: Springer Publishing Company. [Google Scholar]
  35. Marchand, Alain, and Marie-Eve Blanc. 2010. The Contribution of Work and Non-work Factors to the Onset of Psychological Distress: An Eight-year Prospective Study of a Representative Sample of Employees in Canada. Journal of Occupational Health 52: 176–85. [Google Scholar] [CrossRef] [PubMed]
  36. Marchand, Alain, and Pierre Durand. 2011. Psychological Distress, Depression, and Burnout: Similar Contribution of the Job Demand-Control and Job Demand-Control-Support Models? Journal of Occupational and Environmental Medicine 53: 185–89. [Google Scholar] [CrossRef] [PubMed]
  37. Marin, Marie-France, Catherine Lord, Julie Andrews, Robert-Paul Juster, Shireen Sindi, Geneviève Arsenault-Lapierre, Alexandra J. Fiocco, and Sonia J. Lupien. 2011. Chronic stress, cognitive functioning and mental health. Neurobiology of Learning and Memory 96: 583–95. [Google Scholar] [CrossRef]
  38. Minihan, Elisha, Dimitrios Adamis, Michele Dunleavy, Angela Martin, Blanaid Gavin, and Fiona McNicholas. 2022. COVID-19 related occupational stress in teachers in Ireland. International Journal of Educational Research Open 3: 100114. [Google Scholar] [CrossRef] [PubMed]
  39. Mucci, Nicola, Gabriele Giorgi, Vincenzo Cupelli, Pier Agostino Gioffrè, Maria Valeria Rosati, Francesco Tomei, Gianfranco Tomei, Edgar Breso-Esteve, and Giulio Arcangeli. 2015. Work-related stress assessment in a population of Italian workers. The Stress Questionnaire. Science of the Total Environment 502: 673–79. [Google Scholar] [CrossRef] [PubMed]
  40. North, Carol S., Alina M. Surís, and David E. Pollio. 2021. A Nosological Exploration of PTSD and Trauma in Disaster Mental Health and Implications for the COVID-19 Pandemic. Behavioral Sciences 11: 7. [Google Scholar] [CrossRef]
  41. Oshio, Takashi, Akiomi Inoue, and Akizumi Tsutsumi. 2021. Role ambiguity as an amplifier of the association between job stressors and workers’ psychological ill-being: Evidence from an occupational survey in Japan. Journal of Occupational Health 63. [Google Scholar] [CrossRef]
  42. Podsakoff, Philip M., Scott B. MacKenzie, Jeong-Yeon Lee, and Nathan P. Podsakoff. 2003. Common method biases in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology 88: 879–903. [Google Scholar] [CrossRef]
  43. Rahman, Jawaria, Abilash Muralidharan, Sohail J. Quazi, Hajra Saleem, and Safeera Khan. 2020. Neurological and Psychological Effects of Coronavirus (COVID-19): An Overview of the Current Era Pandemic. Cureus 12. [Google Scholar] [CrossRef]
  44. Rossi, Rodolfo, Valentina Socci, Dalila Talevi, Sonia Mensi, Cinzia Niolu, Francesca Pacitti, Antinisca Di Marco, Alessandro Rossi, Alberto Siracusano, and Giorgio Di Lorenzo. 2020. COVID-19 Pandemic and Lockdown Measures Impact on Mental Health Among the General Population in Italy. Frontiers in Psychiatry 11: 790. [Google Scholar] [CrossRef]
  45. Sanchez-Gomez, Martin, Gabriele Giorgi, Georgia Libera Finstad, Flavio Urbini, Giulia Foti, Nicola Mucci, Salvatore Zaffina, and José M. León-Perez. 2021. COVID-19 Pandemic as a Traumatic Event and Its Associations with Fear and Mental Health: A Cognitive-Activation Approach. International Journal of Environmental Research and Public Health 18: 7422. [Google Scholar] [CrossRef] [PubMed]
  46. Schill, Anita L., and Lewis Casey Chosewood. 2013. The NIOSH Total Worker Health Program. The NIOSH Total Worker Health Program Vol. 55, No. 12, Supplement: Total Worker Health Supplement (December 2013). Available online: https://www.jstor.org/stable/48500336 (accessed on 31 January 2023).
  47. Si, Ming-Yu, Xiao-You Su, Yu Jiang, Wen-Jun Wang, Xiao-Fen Gu, Li Ma, Jing Li, Shao-Kai Zhang, Ze-Fang Ren, Ran Ren, and et al. 2020. Psychological impact of COVID-19 on medical care workers in China. Infectious Diseases of Poverty 9: 113. [Google Scholar] [CrossRef] [PubMed]
  48. Spector, Paul E. 2006. Method Variance in Organizational Research: Truth or Urban Legend? Organizational Research Methods 9: 221–32. [Google Scholar] [CrossRef]
  49. Voltmer, Edgar, Susen Köslich-Strumann, Anna Walther, Mahmoud Kasem, Katrin Obst, and Thomas Kötter. 2021. The impact of the COVID-19 pandemic on stress, mental health and coping behavior in German University students—A longitudinal study before and after the onset of the pandemic. BMC Public Health 21: 138. [Google Scholar] [CrossRef] [PubMed]
  50. Wang, Bolin, Ruobao Li, Zhong Lu, and Yan Huang. 2020. Does comorbidity increase the risk of patients with COVID-19: Evidence from meta-analysis. Aging 12: 6049–57. [Google Scholar] [CrossRef]
  51. Xie, Xiaoxia, Chienchung Huang, Shannon P. Cheung, Yuqing Zhou, and Jingbo Fang. 2021. Job Demands and Resources, Burnout, and Psychological Distress of Social Workers in China: Moderation Effects of Gender and Age. Frontiers in Psychology 12: 741563. [Google Scholar] [CrossRef]
  52. Yasmin, Sabina, Muhammad Khairul Alam, Ferdous Bin Ali, Rajon Banik, and Nahid Salma. 2022. Psychological Impact of COVID-19 Among People from the Banking Sector in Bangladesh: A Cross-Sectional Study. International Journal of Mental Health and Addiction 20: 1485–99. [Google Scholar] [CrossRef]
Figure 1. COVID-19-related post-traumatic stress as a mediator between lack of supervisor support and psychological distress. Note: N = 275, ** p < 0.01, * p < 0.05.
Figure 1. COVID-19-related post-traumatic stress as a mediator between lack of supervisor support and psychological distress. Note: N = 275, ** p < 0.01, * p < 0.05.
Admsci 13 00116 g001
Figure 2. COVID-19-related post-traumatic stress as a mediator between lack of colleague support and psychological distress. Note: N = 275, ** p < 0.01.
Figure 2. COVID-19-related post-traumatic stress as a mediator between lack of colleague support and psychological distress. Note: N = 275, ** p < 0.01.
Admsci 13 00116 g002
Figure 3. COVID-19-related post-traumatic stress as a mediator between job demands and psychological distress. Note: N = 275, ** p < 0.01.
Figure 3. COVID-19-related post-traumatic stress as a mediator between job demands and psychological distress. Note: N = 275, ** p < 0.01.
Admsci 13 00116 g003
Figure 4. COVID-19-related post-traumatic stress as a mediator between lack of job control and psychological distress. Note: N = 275, ** p < 0.01, * p < 0.05.
Figure 4. COVID-19-related post-traumatic stress as a mediator between lack of job control and psychological distress. Note: N = 275, ** p < 0.01, * p < 0.05.
Admsci 13 00116 g004
Figure 5. COVID-19-related post-traumatic stress as a mediator between role ambiguity and psychological distress. Note: N = 275, ** p < 0.01.
Figure 5. COVID-19-related post-traumatic stress as a mediator between role ambiguity and psychological distress. Note: N = 275, ** p < 0.01.
Admsci 13 00116 g005
Table 1. Descriptive statistics, Cronbach’s alphas, and correlations between the study variables.
Table 1. Descriptive statistics, Cronbach’s alphas, and correlations between the study variables.
Variables1234567
1. Lack of supervisor support-
2. Lack of colleague support0.40 **-
3. Job demands0.36 **0.26 ***-
4. Lack of job control0.33 **0.40 ***0.25 **-
5. Role ambiguity0.41 **0.37 ***0.13 *0.38 ***-
6. Covid-19 stress0.15 *0.020.19 **0.110.15 *-
7. Mental health 0.33 **0.28 ***0.30 ***0.26 ***0.34 ***0.37 ***-
Mean2.722.492.852.622.102.421.98
Standard Deviation0.980.740.740.760.710.790.45
α0.780.660.630.660.680.800.85
Note: N = 275, *** p< 0.001, ** p < 0.01, * p < 0.05.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Foti, G.; Bondanini, G.; Finstad, G.L.; Alessio, F.; Giorgi, G. The Relationship between Occupational Stress, Mental Health and COVID-19-Related Stress: Mediation Analysis Results. Adm. Sci. 2023, 13, 116. https://doi.org/10.3390/admsci13040116

AMA Style

Foti G, Bondanini G, Finstad GL, Alessio F, Giorgi G. The Relationship between Occupational Stress, Mental Health and COVID-19-Related Stress: Mediation Analysis Results. Administrative Sciences. 2023; 13(4):116. https://doi.org/10.3390/admsci13040116

Chicago/Turabian Style

Foti, Giulia, Giorgia Bondanini, Georgia Libera Finstad, Federico Alessio, and Gabriele Giorgi. 2023. "The Relationship between Occupational Stress, Mental Health and COVID-19-Related Stress: Mediation Analysis Results" Administrative Sciences 13, no. 4: 116. https://doi.org/10.3390/admsci13040116

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop