Next Article in Journal
Does Problematic Use of Social Network Mediate the Association between Bullying Victimization and Loneliness among Lebanese Adolescents?
Previous Article in Journal
Risk Factors for Persistent Anosmia and Dysgeusia in Children with SARS-CoV-2 Infection: A Retrospective Study
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Children
Volume 10
Issue 3
10.3390/children10030598
children-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Association between Bullying Victimization and Aggression in Lebanese Adolescents: The Indirect Effect of Repetitive Negative Thinking—A Path Analysis Approach and Scales Validation

by
Feten Fekih-Romdhane
1,2,
Diana Malaeb
3,
Abir Sarray El Dine
4,
Ecem Yakın
5,
Souheil Hallit
6,7,8,*,† and
Sahar Obeid
9,†
1
Faculty of Medicine of Tunis, Tunis el Manar University, Tunis 1068, Tunisia
2
The Tunisian Center of Early Intervention in Psychosis, Department of Psychiatry “Ibn Omrane”, Razi Hospital, Manouba 2010, Tunisia
3
College of Pharmacy, Gulf Medical University, Ajman P.O. Box 4184, United Arab Emirates
4
Department of Biomedical Sciences, School of Arts and Sciences, Lebanese International University, Beirut P.O. Box 146404, Lebanon
5
Centre d’Études et de Recherches en Psychopathologie et Psychologie de la Santé, Université de Toulouse-Jean Jaurès, UT2J, 5 allées Antonio Machado, 31058 Toulouse, France
6
School of Medicine and Medical Sciences, Holy Spirit University of Kaslik, Jounieh P.O. Box 446, Lebanon
7
Applied Science Research Center, Applied Science Private University, 11931 Amman, Jordan
8
Research Department, Psychiatric Hospital of the Cross, Jal Eddib P.O. Box 60096, Lebanon
9
Department of Social and Education Sciences, School of Arts and Sciences, Lebanese American University, Jbeil P.O. Box 13-5053, Lebanon
*
Author to whom correspondence should be addressed.
†
These authors contributed equally to this work.
Children 2023, 10(3), 598; https://doi.org/10.3390/children10030598
Submission received: 28 January 2023 / Revised: 2 March 2023 / Accepted: 3 March 2023 / Published: 21 March 2023
(This article belongs to the Section Child and Adolescent Psychiatry)
Browse Figures
Versions Notes

Abstract

:
(1) Background: The purpose of the present study was to validate the Perseverative Thinking Questionnaire (PTQ) and the Buss–Perry Aggression Questionnaire-Short Form (BPAQ-SF) and test whether repetitive negative thinking plays an indirect role in the relationship between bullying victimization and aggression among Lebanese adolescents. (2) Methods: This cross-sectional study was conducted between January and May 2022 and included 379 Lebanese adolescent students (64.9% females, mean age 16.07 years). (3) Results: The three-factor solution of the PTQ and the four-factor solution of the BPAQ-SF showed excellent model fit. PTQ mediated the association between bullying victimization and physical aggression, verbal aggression, hostility, and anger. (4) Conclusions: This study expands on previous research by showing that repetitive negative thinking, an impactful socio-cognitive factor for students’ mental health, has a mediating (indirect) effect on the cross-sectional relationship between bullying victimization and aggression. This suggests that interventions aiming to prevent aggressive behaviors among adolescent students may be more effective if focused on repetitive negative thinking.

1. Introduction

Bullying victimization is defined as “repeated aggressive behavior, with an imbalance of power between the aggressor and the victim” [1]. It is a specific form of interpersonal violence where the victims are the target of aggression by dominant peers for the intentional purpose of inflicting harm while not being able to defend themselves [2]. Bullying victimization mostly occurs in school, and during early adolescence [3,4,5], and it has several forms. It can be physical (e.g., pushing, hitting, fighting), verbal (e.g., threatening, calling names, spreading rumors, teasing), sexual (e.g., harassment), or social (e.g., exclusion, ignoring) [6]. A high but variable prevalence of this phenomenon has been documented among children and youth worldwide. The marked variability in rates across countries has been explained by several factors, including research design, inconsistencies in definitions, and measurement tools used to assess bullying victimization [7].
Multiple studies from many countries have reported different prevalence rates of bullying victimization among adolescents aged 12–18 years: less than 10% in Asian countries (Hongkong, Taiwan, and Macao) [8], 17.9% in Pakistan [9], around 20% in the United States [10], more than 40% in African countries (Malawi and Ghana) [11,12], 40.6% in Philippines [13], 46.9% in Indonesia [14], and 47% in New Zealand [15]. The UNESCO report (2019) estimated that 32% of children and adolescents are experiencing bullying victimization in some form worldwide [16]. A meta-analysis by Modecki et al., encompassing 80 studies that investigated bullying involvement rates of students (aged between 12 and 18 years), revealed wide variation in prevalence rates across contexts, with mean prevalence rates of 36% for victimization [17]. Even though there is still significant under-representation of developing countries of the Arab world in the research literature related to school bullying, similar variations in estimates have been documented, with prevalence rates of 7% in Jordan [18], 11.7% in Tunisia [19], 16% the United Arab Emirates [20], 9.9–20.6% in Algeria [21], and 49.1% in Lebanon [22]. Thus, Lebanon seems to have one of the highest prevalence rates of bullying in the Arab region and even worldwide, with almost 1 in 4 Lebanese adolescents having been estimated to be involved in bullying [23]. Therefore, Lebanon offers particularly interesting social and cultural contexts within which to investigate bullying victimization in adolescent students.
Because of the potential detrimental effects on adolescents’ health [24,25,26], there has been growing and widespread public concern in schools about victimization over the last few years. Indeed, bullying victimization can cause a broad range of consequences, including impaired academic performance [27] and poor general health [25], as well as other internalizing [28] and externalizing [29] mental health problems. Internalizing problems involve depression, anxiety, stress [30,31], self-harm [32], and suicidal ideation and behaviors [33]; while externalizing problems include rule-breaking behaviors and aggression [34,35]. Aggression, specifically, is a harmful and serious consequence that deserves attention from scholars in research to further enhance the knowledge and understanding of its potential influencing factors and possible prevention methods.

1.1. Bullying Victimization and Aggression

Aggression refers to behavior with the purpose of harming another individual [36]. Aggression can take various forms, representing its different aspects: physical and verbal aggression (behavioral and instrumental aspects), anger (affective and emotional aspects), and hostility (cognitive aspect) [37]. The relationship between bullying victimization and aggression has become a subject of growing interest in both the clinical and academic worlds. Prior evidence has shown that victimization and the perpetration of aggression are strongly correlated; and, more importantly, that bullying victimization is a potential risk factor for future perpetration [38]. Previous studies support that bullying victimization, particularly when experienced early in life (in childhood or early adolescence) can lead to aggression over time (e.g., [39,40]). For instance, longitudinal studies have shown that a subset of youth who have been victims of bullying in childhood and early adolescence were more likely to later become perpetrators of bullying, themselves [38,41,42,43]. Various psychosocial factors may contribute to aggressive behavior among victimized adolescents [44]; however, these factors remain largely under-researched and unclear. Elucidating the possible mechanisms underlying the association between victimization and aggressive behavior is important for more than one reason. First, aggression has reached highly alarming rates in schools [45] and has become one of the major problems of today’s society [46,47]. Particularly, previous studies found that Lebanese adolescent students reported moderate to high aggression in 34.0% and 31.9% of cases, respectively; which is higher than rates reported by students from other countries [48]. This is mainly due to the multiple stressors Lebanese youth are facing, including ongoing conflicts, economic crises, and political instability [49]. Second, aggression has been shown to lead to detrimental outcomes in adolescents’ development and mental health [50], increase the likelihood of later offending [51], and have high social and economic costs [52]. Third, the existing intervention programs targeting bullying victimization and aggression have proven to have poor effectiveness in decreasing or preventing these behaviors, especially in adolescents [53,54], suggesting that new psychological interventions need to be developed based on new or under-explored pathways from bullying victimization to aggression. The current study focuses on one possible pathway through which victimization can lead to aggression: repetitive negative thinking. In particular, we hypothesized that bullying victimization could have an indirect effect on aggression via repetitive negative thinking.

1.2. Repetitive Negative Thinking as a Mediator between Bullying Victimization and Aggression

Although only a very limited amount of research is available on the role of socio-cognitive factors in the relationship between bullying victimization and aggression, the little existing evidence suggests that such factors may be determinants for the identification of adolescents at heightened risk of becoming aggressors (e.g., [55,56]). Indeed, victimization has proven to trigger negative emotions and be related to maladaptive coping strategies, both of which are linked to aggression [55]. Additionally, bullying victimization, particularly when repeated over time, is linked to distinct social-cognitive patterns [56]. Maladaptive socio-cognitive processes, such as rumination, self-evaluations, and hostile attribution bias, appear to play major roles in negative outcomes from bullying victimization experiences [57,58,59,60].
Repetitive negative thinking is a cognitive process defined as excessive and perseverative thinking about one’s negative experiences or problems (current, past, or future) that are experienced as intrusive and difficult to control [41,61]. It represents an emotion regulation strategy involved in the development and maintenance of several negative mental health problems (e.g., [62,63,64,65]). Previous studies mainly focused on the mediating role of repetitive negative thinking in the relationship between stressors and internalized problems (e.g., [66]). More specifically, repetitive negative thinking has been found to mediate the association between bullying victimization and depression [67]. Studies on externalized problems, however, are more limited. Recently, a prospective longitudinal Finnish study found that more frequent bullying victimization was associated with later bully perpetration through the indirect mediating effect of rumination [68]. Repetitive negative thinking has recently been gaining potential interest within contemporary research related to students’ mental health [69]. We believe that providing empirical support for the hypothesis that repetitive negative thinking plays a mediating role in the relationship between victimization and aggression could offer potentially promising avenues for dealing with aggressive adolescents in school settings.

1.3. Bullying Victimization, Aggression, and Repetitive Negative Thinking Assessment Measures

Some bullying assessment measures have been translated to Arabic and validated in Arab contexts, such as the Revised Olweus Bully/Victim Questionnaire (OBVQ-R) [18,70], the School Climate Bullying Survey (SCBS) [71,72], and the Illinois Bully Scale (IBS) [73]. The Perseverative Thinking Questionnaire (PTQ) [74] and the Buss–Perry Aggression Questionnaire-Short Form (BPAQ-SF) [37] are the most widely used measures of repetitive negative thinking and aggression among adolescents, respectively. The PTQ has been previously validated in languages other than English, including French [75], Spanish [76], Peruvian [69], and Polish [77]. Similarly, different versions of the Buss–Perry Aggression Questionnaire-Short Form (BPAQ-SF) exist, including Portuguese [78], Hungarian [79], Turkish [80], Chinese [81], Spanish [82], and Thai [83]. However, to the best of our knowledge, no research instruments are yet available in Arabic to assess aggression and repetitive negative thinking.
The Arabic language is an official language in 27 states and is spoken by more than 420 million people worldwide [84]. An Arabic version of the PTQ and the BPAQ-SF would allow clinicians to assess these important constructs in different Arabic-speaking contexts, and allow researchers “to compare research findings from different countries and in different languages” [85].

1.4. The Present Research

Adolescent students are in a developmental time period of increased susceptibility to both bullying victimization [86] and aggressive behavior [87]. Therefore, understanding pathways linking bullying victimization and aggression is essential to optimize biopsychosocial development and prevent negative outcomes for this vulnerable population [88,89]. Most of the previous studies in this regard have investigated a unique form of aggression, bullying perpetration, in relation to bullying victimization. However, it has been shown that aggression, in general, is distinct from bullying perpetration, in particular, in terms of definitions and outcomes [90]. We intended, through the present work, to examine bullying victimization in relation to the broad construct of aggression in its multifaceted forms (physical, verbal, anger, hostility), as opposed to one specific form of aggression (i.e., bullying perpetration). Furthermore, given that both bullying [91] and aggression [92] are culturally dependent concepts, cross-cultural differences in the relationship between these two entities might also exist. Hence, it is important to investigate these entities across different cultural backgrounds, particularly under-researched ones. Through the present study, we build on the above-mentioned previous research by exploring whether repetitive negative thinking may be a cognitive factor that plays an indirect role in the relationship between bullying victimization and aggression. Another objective of this study was to validate the PTQ and BPAQ-SF scales among Lebanese Arab-speaking adolescents. We hypothesized higher levels of bullying victimization to be significantly and positively associated with repetitive negative thinking, which in turn was hypothesized to be associated with a higher tendency toward aggression.

2. Materials and Methods

2.1. Study Design and Participants

This cross-sectional study was conducted between January and May 2022. An online questionnaire was created via google forms software and included information related to the aims of the study as well as instructions for filling out the questionnaire. The initial respondents were then asked to recruit other participants within the same age range (required to participate in the study) and preferably as diverse as possible with regard to their place of habitat within the Lebanese governorates. No credits were offered for participation. The snowball technique was used during sampling. All participants (N = 379) were adolescent students (aged between 13 and 17 years old) residing in Lebanon (including all Lebanese governorates: Beirut, Mount Lebanon, North, South, and Bekaa).

2.2. Ethical Aspect

The study protocol was approved by The Ethics and Research Committee of the Psychiatric Hospital of the Cross (HPC-035–2020). All respondents were asked to get their parents’ consent and provide electronic informed consent prior to completing the survey. All respondents and their parents were informed about the study’s objectives and general instructions. All procedures were in accordance with the ethical standards of the institutional and/or national research committee as well as with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.

2.3. Minimal Sample Size Calculation

Using the formula suggested by Fritz and MacKinnon [93] (i.e., n = L f 2 + k + 1 , where f = 0.26 for a small to moderate effect size, L = 7.85 for an α error of 5% and power β = 80%), a minimal sample of 127 was deemed necessary based on 10 variables to be entered in the model.

2.4. Study Instruments

The first part of the questionnaire provided information regarding the aims of the current study and the anonymity of collected responses. All participants were required to select the option stating “I got my parents’ approval and consent to participate in this study” to be directed to the questionnaire.
The second part of the questionnaire contained sociodemographic information about the participants (age, gender, governorate, current self-reported weight and height). BMI was calculated according to the World Health Organization formula [94]. The physical activity index is the cross result of the intensity, duration, and frequency of daily activity [95]. The household crowding index, reflecting the socioeconomic status of the family [96], is the ratio of the number of persons living in the house over the number of rooms in it (excluding the kitchen and the bathrooms). To assess financial burden, respondents were asked to answer the following question: “How much pressure do you feel with regard to your personal financial situation in general?” on a scale from 1 to 10, with 10 referring to overwhelming pressure.
The third part included the following questionnaires used in the current study:
The Illinois Bully scale (IBS). The IBS, validated in Lebanon [97], is an eighteen-item scale used to assess bullying perpetration (e.g., “I annoyed other students”) and bullying victimization (e.g., “Other students beat and pushed me”) [73]. Questions were scored as follows: “never = 0 and up to seven times or more = 4”. Subscale scores were computed by summing the respective items. Higher scores on these subscales indicated higher bullying perpetration and victimization, respectively [98]. In this study, only the bullying victimization subscale was used (Cronbach’s alpha = 0.91).
The Buss–Perry Aggression Questionnaire-Short Form (BPAQ-SF). The BPAQ-SF [99] is a short version of the BPAQ, and it contains 12 items rated on a 5-point Likert scale. The items are organized into four subscales assessing physical aggression (3 items; e.g., “I have threatened people I know”), verbal aggression (3 items; e.g., “My friends say that I’m somewhat argumentative”), anger (3 items; e.g., “I flare up quickly but get over it quickly”), and hostility (3 items; e.g., “I wonder why sometimes I feel so bitter about things”). Higher scores indicate higher levels of aggression. The Cronbach’s alpha values were as follows: physical aggression (α = 0.66), verbal aggression (α = 0.55), hostility (α = 0.72), and anger (α = 0.71). The Arabic version can be found in Appendix A Table A1.
The Perseverative Thinking Questionnaire (PTQ). The Perseverative Thinking Questionnaire [74] is composed of 15 items evaluating core features, mental resources, and unproductiveness due to repetitive negative thinking (e.g., “I think about many problems without solving any of them”, or “My thoughts repeat themselves”). Items were rated on a 5-point Likert scale ranging from 0 = “never” to 4 = “almost always”. A higher score on each dimension reflects a higher level of repetitive negative thinking. The Cronbach’s alpha values were as follows: core features (α = 0.92), mental resources (α = 0.83), and unproductiveness (α = 0.83). The Arabic version can be found in Appendix A Table A2.

2.5. Translation Procedure

The forward and backward translation method was applied to different scales. The English version was translated into Arabic by a Lebanese translator who was unfamiliar with the questionnaire and the study. Then, a Lebanese psychologist with full working proficiency in English translated the Arabic version back into English. The initial and the second English versions were compared to detect and later eliminate any inconsistencies.

2.6. Statistical Analysis

Confirmatory factor analyses were conducted to test the four-factor and the three-factor structures of the BPAQ-SF and PTQ scales, respectively, that were found in the original validation studies. Confirmatory factor analysis was performed using RStudio (Version 1.4.1103 for Macintosh) and the Lavaan and semTools packages. We used the weighted least squares means and variance adjusted (WLSMV) estimation method, which is more appropriate for ordinal data.
Data analysis was conducted using SPSS software version 23. No missing data was found, as all questions were required in the online survey. Cronbach’s alpha values were recorded for reliability analyses of all scales and subscales. All aggression subscale scores were normally distributed, with skewness and kurtosis values varying between −1 and +1 [100]. Student’s t and ANOVA tests were used to compare two, and three or more means, respectively. The Pearson correlation test was used to compare two continuous variables. To check for a significant indirect effect of PTQ between bullying victimization and aggression/hostility/anger, we conducted a path analysis using SPSS AMOS v.26. Variables that displayed a p < 0.25 in bivariate analysis were included in the path analysis. Significance was set at p < 0.05.

3. Results

3.1. Sociodemographic and Other Characteristics of Sample

Among 379 adolescent participants (mean age = 16.07 ± 1.19 years), 64.9% were females.
Other characteristics are summarized in Table 1.

3.2. Scale Validation

The three-factor solution of the PTQ scale showed excellent model fit, with a CFI of 0.99 and a GFI of 0.99, an SRMR of 0.04, and an RMSEA of 0.08 (90% CI of RMSEA = 0.07, 0.09) (Figure 1).
The four-factor solution of the BPAQ-SF also showed excellent model fit, with a significant CFI of 0.99 and a GFI of 0.99, an SRMR of 0.05, and an RMSEA of 0.06 (90% CI 0.05, 0.07) (Figure 2).

3.3. Bivariate Analysis

The bivariate analysis results can be found in Table 2 and Table 3. In the current sample, males displayed greater mean physical aggression scores compared to females (7.03 vs. 6.36; p = 0.043), while females had higher mean anger scores compared to males (8.45 vs. 7.53; p = 0.009). Higher PTQ core features, mental resources, and unproductiveness were found to be significantly associated with more physical aggression, verbal aggression, anger, and hostility. Older age was significantly associated with more verbal aggression. Higher BMI was found to be significantly associated with more physical aggression, while more financial burden was significantly associated with greater hostility.

3.4. Path Analysis

All four tested models had acceptable fit indices (Table 4). The results indicate that PTQ mediated the association between bullying victimization and physical aggression (beta = 0.053; 90% CI 0.03–0.09; p < 0.001) (Figure 3), verbal aggression (beta = 0.06; 90% CI 0.03–0.09; p = 0.001) (Figure 4), hostility (beta = 0.10; 90% CI 0.06–0.15; p = 0.001) (Figure 5), and anger (beta = 0.09; 90% CI 0.05–0.14; p = 0.001) (Figure 6).

4. Discussion

Bullying victimization has been recognized as a major and complex psycho-social problem [101] that requires considerable efforts from both public health professionals (practitioners, researchers, and educators) and the general public [102]. This paper represents an attempt to further understand the factors linked to aggressive tendencies among victimized adolescents. For this, one socio-cognitive factor, repetitive negative thinking, has been suggested as a potential mechanism explaining how victimization may be associated with aggressive behaviors among Lebanese adolescent students. As expected, we found that victimization not only directly contributed to students’ levels of aggression but also indirectly through repetitive negative thinking.
This study also aimed to validate the PTQ and BPAQ-SF in Arabic. We found evidence that these scales have robust psychometric properties and are brief, easy-to-use assessment tools for Arab-speaking adolescents. The Arabic BPAQ-SF and PTQ revealed satisfactory internal consistencies, with a Cronbach’s alpha varying from 0.66 to 0.72 for the BPAQ-SF (except for the verbal aggression subscale, which revealed an α value of 0.55), and from 0.83 to 0.92 for the PTQ. In addition, the current results were consistent with those of the original versions’ validation studies, showing that the four-factor solution of the BPAQ-SF and the three-factor solution of the PTQ both revealed excellent model fit. Making these scales available to researchers from Arab countries would be beneficial to the whole research community, as ensuring the comparability of these assessment methods may help prevent wide variations in research findings related to these topics [85]. It would also encourage producing more research from the under-studied Arab world [103]. Our findings showed excellent model fit of the four-factor structure of the BPAQ-SF, unlike other previous validation studies that found poor fit (e.g., [81,104,105]). However, we found a poor but acceptable Cronbach’s alpha value [106] for the verbal aggression subscale. As previously said, aggression is a culturally dependent construct [92]; the low internal consistency of this subscale might be due to cultural variability and calls for further studies to extend its cross-cultural validity in other Arab contexts.
Regarding the direct effects, we found that higher IBS scores, translating into more frequent self-reported bullying victimization experiences by students during the past month, were significantly and positively correlated to more physical aggression, verbal aggression, anger, and hostility. These findings are in line with those from earlier cross-sectional and longitudinal studies showing that bullying victimization is positively associated with aggressive behavior among adolescents [35,43,44,107,108]. Different explanations have been advanced in literature to explain the positive link between being bullied and exhibiting aggressive behaviors. For example, it has been suggested that aggression serves as a buffer for negative emotional responses to bullying (e.g., anxiety and anger) [35,109]. It has also been suggested that isolation from peers caused by bullying victimization may lead to losing social skills, which leads in turn to externalizing problems [108]. Although the relationship between bullying victimization and aggression is well established, it is certain that not all victimized adolescents will evolve into aggressors [110]. However, a gap remains in factors and mechanisms that could help identify at-risk adolescents. To address this knowledge gap, we investigated the role of one potential mediator in this relationship, the repetitive negative thinking process.
In terms of mediation analyses, this study revealed that students who experienced more frequent victimization were more likely to display high levels of aggression themselves under the mediation of repetitive negative thinking. In other words, the overall indirect effect of bullying victimization on aggression through repetitive negative thinking (which included PTQ core features, mental resources, and unproductiveness as mediators) was significant. Similar to our findings, a recent longitudinal study performed in Finland in 2021 by Malamut and Salmivalli found that rumination about past victimization experiences mediated the positive prospective association between bully victimization and later bully perpetration [68]. There are several reasons to expect this pattern of findings. One reason is that adolescents who tend to repeatedly and negatively think about their victimization experiences are those most likely to have developed a vulnerability schema, which leads in turn to a desire to protect themselves against potential future victimization through aggression [111]. Indeed, the “victim schema model” suggests that peer victimization may result in biased cognitive and emotional regulation processes, which themselves lead to aggressive behavior as a response to perceived threat [112]. In sum, a few previous studies investigated the role of some socio-cognitive factors in pathways underlying the relationship between bullying victimization and subsequent aggression (e.g., [41,42,113]). However, no studies have examined so far the effects of repetitive negative thinking in this relationship. Also, all these studies focused on one specific type of aggression, which is bullying perpetration; and thus cannot be generalized to other forms of aggression. Therefore, our study is the first, to our knowledge, to document a mediating (indirect) effect of repetitive negative thinking on the link between bullying victimization and aggressiveness (in all its forms).

4.1. Study Implications

Interpersonal interactions have proven to play a determinant role in the development of children and adolescents [114]. Bullying victimization, a specific form of interpersonal violence, has devastating and long-lasting consequences for victimized students when occurring during early adolescence [115], including aggressive behaviors. Hence the importance of developing and implementing effective prevention interventions aiming to promote healthy relationships and positive interactions with peers, as well as decreasing bullying behaviors in schools. To date, antibullying policies have been inconsistent [116]; and school bullying prevention programs (such as the Olweus Bullying Prevention Program [117]) have produced mixed results in some countries (e.g., Germany, [118]), and have shown to be generally poorly effective among adolescents [53,54].
A first step towards developing new and effective interventions is understanding pathways leading from bullying victimization to aggression, which are still largely understudied and unknown. The present study sought to expand the literature on the role of socio-cognitive processes in the association between bullying victimization and aggression among adolescent students, by investigating the mediating effects of repetitive negative thinking. On the basis of the present findings, we could preliminarily confirm our hypothesis. Our results, along with prior data, suggest that routine and early detection of behavioral problems and other negative school experiences should be implemented in schools.
When victimized students tend to repeatedly and negatively think about their adverse experiences, they would be most likely to turn to aggression. This suggests that therapies focused on negative thinking, such as repetitive negative thinking-focused ACT (Acceptance and Commitment Therapy Focused on Repetitive Negative Thinking) [119,120], may be promising avenues to prevent aggressive behaviors in students victimized by peers. This kind of therapy has been tested and shown to be effective in reducing emotional problems; however, to our knowledge, these approaches have not been previously analyzed for bullying problems. This calls for experimental studies to test our hypothesis. Finally, optimizing prevention and intervention programs and their implementation in schools requires that students, school staff, clinicians, researchers, and the whole community work together and coordinate their efforts [121].

4.2. Limitations and Strengths

This study has certain limitations that can provide fruitful directions for future research. First, employing a cross-sectional design does not allow for establishing a causal relationship between victimization and aggression. Second, due to the self-report nature of the questionnaire used, reporting bias cannot be excluded. For greater validity, future studies might consider collecting bullying- and aggression-related information from peers, teachers, and parents. In addition, financial burden was assessed using a single question rather than a scale. Third, although we attempted to shed light on a previously unstudied mediator in the relationship between victimization and aggression, many other factors (such as drug use, family interactions, and school life satisfaction) might play key roles in this relationship and their mediating effects should be tested in future studies. Fourth, our findings showed an excellent model fit of the four-factor structure of the BPAQ-SF, unlike other previous validation studies that found a poor fit (e.g., [81,104,105]). However, we found a poor but acceptable Cronbach’s alpha [106] for the verbal and physical aggression subscales. As previously said, aggression is a culturally dependent construct [92], and the low internal consistency of this subscale might be due to cultural variability and calls for further studies to extend its cross-cultural validity in other Arab contexts.
This study has a number of strengths that deserve recognition. First, most of the previous studies examined bullying victimization in relation to bullying perpetration rather than aggression per se; while this study provides a broader overview of the topic by investigating victimization in relation to four aggression dimensions (i.e., physical aggression, verbal aggression, anger, and hostility). Second, by using valid measures of bullying victimization, aggression, and repetitive negative thinking that have been adapted to the local Lebanese context and culture, the present findings contribute reliable information to the field of bullying, allowing comparison with the findings of other studies locally, regionally, and internationally. Third, the study addressed an under-explored topic in the Arab population, especially the relationship between bullying victimization and aggression, given that each of these constructs have been researched independently in adolescent populations in previous Arab studies.

5. Conclusions

Our study expands on past research by showing that repetitive negative thinking, a socio-cognitive factor that has proven to be impactful on students’ mental health, is one factor that underlies the cross-sectional relationship between bullying victimization and aggression. This suggests that interventions aiming at preventing aggressive behaviors among adolescents, in general, and students, in particular, may be more effective if focused on repetitive negative thinking. Compared to previous studies, we investigated four dimensions of aggression, not only bully perpetration, which might provide a clearer picture of the problem. The cross-sectional design remains, however, a major limitation, and calls for additional longitudinal studies to support our findings. Although we provided preliminary evidence suggesting repetitive negative thinking as a possible mediator of multiple forms of aggression after bullying victimization in adolescent students, future longitudinal research is needed to support our findings and deepen our understanding of the mechanisms underlying this relationship by investigating other pathways through which bullying victimization leads to aggression (e.g., other coping strategies). Furthermore, negative thinking subsequent to bullying victimization may also be associated with diverse other maladjustment outcomes, such as feelings of loneliness and social dissatisfaction, and impaired peer relationships and academic performance. Additional studies should consider the influence of bullying victimization and repetitive negative thinking on these outcomes.

Author Contributions

Conceptualization, S.H. and S.O.; methodology, S.H. and S.O.; software, S.H. and E.Y.; validation, S.H., D.M. and S.O.; formal analysis, S.H. and E.Y.; investigation, S.H., D.M. and S.O.; resources, S.H., D.M. and S.O.; data curation, D.M.; writing—original draft preparation, F.F.-R.; writing—review and editing, F.F.-R.; supervision, S.H.; project administration, D.M. and A.S.E.D. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki and was approved by the ethics committee of the School of Pharmacy at the Lebanese International University (approval number: 2021RC-048-LIUSOP) (approval date: 15 November 2021).

Informed Consent Statement

All subjects were informed about the study and all provided informed consent. All participants offered their informed consent before starting the survey (by ticking a respective box on the first page of the online survey) and responded voluntarily to the survey. Participants received no financial compensation for enrolling in the study.

Data Availability Statement

The authors do not have the right to share any data information as per the ethics committee rules and regulations.

Acknowledgments

The authors acknowledge all individuals who participated in the study.

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A

Table
Table A1. Translated items of the Buss-Perry Aggression Questionnaire- Short Form in Arabic language.
Table A1. Translated items of the Buss-Perry Aggression Questionnaire- Short Form in Arabic language.
.باستخدام هذا المقياس المكوّن من 5 نقاط، وضّح كيف أن كلّ من العبارات التالية غير مميّزة أو مميّزة في وصفك.
ضع تقييمك في المربّع المناسب
يصفني
للغاية		يصفني
قليلاً		لا يمكن القول أنّه من سماتي ولكنّه أيضاً
ليس من سماتي		لا يصفني
قليلاً		يصفني
للغاية
.قد أضرب شخصاً آخر إذا إستفزّني بما فيه الكفاية
Given enough provocation, I may hit another person.
.هناك أناس دفعوني كثيراً حتى وصلنا إلى التشاجر
There are people who pushed me so far that we came to blows.
.لقد هدّدت أناساً أعرفهم
I have threatened people I know.
.كثيراً ما أجد نفسي أختلف مع الناس
I often find myself disagreeing with people.
.لا أستطيع الدخول في جدال عندما يختلف الناس معي
I can’t help getting into arguments when people disagree with me.
.أصدقائي يقولون عنّي أنني إنسان مُجادل نوعاً ما
My friends say that I’m somewhat argumentative.
.أستشيط غضباً بسرعة لكنّني أتخطى ذلك بسرعة ايضاً
I flare up quickly but get over it quickly.
.في بعض الأحيان أفقد صوابي من دون أيّ سبب
Sometimes I fly off the handle for no good reason.
.أجد صعوبة في السيطرة على أعصابي
I have trouble controlling my temper.
.أحياناً أشعر كأنّي تلقّيت معاملة غير عادلة من الحياة
At times I feel I have gotten a raw deal out of life.
.يبدو أنّ الآخرين يحصلون دائمًا على فُرَص مؤاتية
Other people always seem to get the breaks.
.أتساءل لماذا أشعر أحيانًا بالمرارة تجاه الأشياء
I wonder why sometimes I feel so bitter about things.
Table
Table A2. Translated items of the Perseverative Thinking Questionnaire in Arabic language.
Table A2. Translated items of the Perseverative Thinking Questionnaire in Arabic language.
Almost Always
تقريبا دائماً		Often
غالباً		Sometimes
بعض الأحيان		Rarely
نادراً		Never
أبداً		Perseverative Thinking Questionnaire (PTQ): Repetitive Negative Thinking
The same thoughts keep going through my mind again and again.
.نفس الأفكار تستمرّ في المرور في ذهني مرارًا وتكرارًا
Thoughts intrude into my mind.
.الأفكار تتطفلّ على ذهني
I can’t stop dwelling on them.
.لا أستطيع التوقّف عن الخوض /العيش في أفكاري
I think about many problems without solving any of them.
.أفكّر في العديد من المشاكل دون حلّ أي منها
I can’t do anything else while thinking about my problems.
.لا يمكنني فعل أي شيء آخر أثناء التفكير في مشاكلي
My thoughts repeat themselves.
.أفكاري تكرّر نفسها
Thoughts come to my mind without me wanting them to.
.الأفكار تتبادر إلى ذهني دون أن أرغب في ذلك
I get stuck on certain issues and can’t move on.
.أعلق في بعض القضايا ولا يمكنني المضي قدمًا
I keep asking myself questions without finding an answer.
.أستمرّ في طرح الأسئلة على نفسي دون أن أجد إجابة
My thoughts prevent me from focusing on other things.
.أفكاري تمنعني من التركيز على أشياء أخرى
I keep thinking about the same issue all the time.
.أبقى أفكّر في نفس المشكلة طوال الوقت
Thoughts just pop into my mind.
.تنبثق أو تظهر الأفكار فقط في عقلي
I feel driven to continue dwelling on the same issue.
.أشعر بالدافع لمواصلة الخوض/العيش في نفس القضية/المشكلة
My thoughts are not much help to me.
.أفكاري لا تساعدني كثيرا
My thoughts take up all my attention.
.أفكاري تستحوذ على كل انتباهي

References

  1. Olweus, D. Bully/victim problems in school: Facts and intervention. Eur. J. Psychol. Educ. 1997, 12, 495–510. [Google Scholar] [CrossRef]
  2. Olweus, D. Bullying at school. In Aggressive Behavior; Springer: Berlin/Heidelberg, Germany, 1994; pp. 97–130. [Google Scholar]
  3. Peltzer, K.; Pengpid, S. Suicidal ideation and associated factors among students aged 13–15 years in Association of Southeast Asian Nations (ASEAN) member states, 2007–2013. Int. J. Psychiatry Clin. Pract. 2017, 21, 201–208. [Google Scholar] [CrossRef] [PubMed]
  4. Rhee, S.; Lee, S.-Y.; Jung, S.-H. Ethnic differences in bullying victimization and psychological distress: A test of an ecological model. J. Adolesc. 2017, 60, 155–160. [Google Scholar] [CrossRef] [PubMed]
  5. Tanrikulu, I.; Campbell, M. Correlates of traditional bullying and cyberbullying perpetration among Australian students. Child. Youth Serv. Rev. 2015, 55, 138–146. [Google Scholar] [CrossRef] [Green Version]
  6. Calaguas, G.M. Forms and frequency of peer aggression and peer victimization among sixth-graders. Int. Ref. Res. J. 2011, 2, 108–113. [Google Scholar]
  7. Gladden, R.M.; Vivolo-Kantor, A.M.; Hamburger, M.E.; Lumpkin, C.D. Bullying Surveillance among Youths: Uniform Definitions for Public Health and Recommended Data Elements. Version 1.0; CDC Stacks Public Health Publication: New York, NY, USA, 2014. Available online: https://stacks.cdc.gov/view/cdc/21596 (accessed on 20 January 2023).
  8. Mok, M.M.C.; Wang, W.-C.; Cheng, Y.-Y.; Leung, S.-O.; Chen, L.-M. Prevalence and Behavioral Ranking of Bullying and Victimization Among Secondary Students in Hong Kong, Taiwan, and Macao. Asia-Pacific Educ. Res. 2013, 23, 757–767. [Google Scholar] [CrossRef]
  9. Karmaliani, R.; Mcfarlane, J.; Somani, R.; Khuwaja, H.M.A.; Bhamani, S.S.; Ali, T.S.; Gulzar, S.; Somani, Y.; Chirwa, E.D.; Jewkes, R. Peer violence perpetration and victimization: Prevalence, associated factors and pathways among 1752 sixth grade boys and girls in schools in Pakistan. PLoS ONE 2017, 12, e0180833. [Google Scholar] [CrossRef] [Green Version]
  10. Yanez, C.; Lessne, D. Student Victimization in US Schools: Results from the 2015 School Crime Supplement to the National Crime Victimization Survey. Stats in Brief. NCES 2018-106; National Center for Education Statistics: Jessup, MD, USA, 2018. [Google Scholar]
  11. Kubwalo, H.W.; Muula, A.S.; Siziya, S.; Pasupulati, S.; Rudatsikira, E. Prevalence and correlates of being bullied among in-school adolescents in Malawi: Results from the 2009 Global School-Based Health Survey. Malawi Med. J. 2013, 25, 12–14. [Google Scholar]
  12. Owusu, A.; Hart, P.; Oliver, B.; Kang, M. The Association Between Bullying and Psychological Health Among Senior High School Students in Ghana, West Africa. J. Sch. Health 2011, 81, 231–238. [Google Scholar] [CrossRef]
  13. Sanapo, M.S. When Kids Hurt Other Kids: Bullying in Philippine Schools. Psychology 2017, 8, 2469–2484. [Google Scholar] [CrossRef] [Green Version]
  14. Afriani, A.; Denisa, D. Bullying Victimization among Junior High School Students in Aceh, Indonesia: Prevalence and its Differences in Gender, Grade, and Friendship Quality. J. Ilm. Peuradeun 2021, 9, 251–274. [Google Scholar] [CrossRef]
  15. Marsh, L.; McGee, R.; Nada-Raja, S.; Williams, S. Brief report: Text bullying and traditional bullying among New Zealand secondary school students. J. Adolesc. 2010, 33, 237–240. [Google Scholar] [CrossRef] [PubMed]
  16. UNESCO. Behind the Numbers: Ending School Violence and Bullying; UNESCO: Paris, France, 2019. [Google Scholar]
  17. Modecki, K.L.; Minchin, J.; Harbaugh, A.G.; Guerra, N.G.; Runions, K.C. Bullying Prevalence Across Contexts: A Meta-analysis Measuring Cyber and Traditional Bullying. J. Adolesc. Health 2014, 55, 602–611. [Google Scholar] [CrossRef] [PubMed]
  18. Shahrour, G.; Dardas, L.A.; Al-Khayat, A.; Al-Qasem, A. Prevalence, correlates, and experiences of school bullying among adolescents: A national study in Jordan. Sch. Psychol. Int. 2020, 41, 430–453. [Google Scholar] [CrossRef]
  19. Sahli, J.; Mellouli, M.; El Ghardallou, M.; Limam, M.; Gallas, M.; Ammar, A.; Mtiraoui, A.; Ajmi, T.N.; Zedini, C. Bullying Among Tunisian Middle School Students: The Prevalence, Psychosocial Associated Factors and Perceived Involvement of Parents, Teachers and Classmates. J. Res. Health Sci. 2018, 18, 414. [Google Scholar]
  20. Rigby, K.; Haroun, D.; Ali, E. Bullying in Schools in the United Arab Emirates and the Personal Safety of Students. Child Indic. Res. 2019, 12, 1663–1675. [Google Scholar] [CrossRef]
  21. Tiliouine, H. School Bullying Victimisation and Subjective Well-Being in Algeria. Child Indic. Res. 2015, 8, 133–150. [Google Scholar] [CrossRef]
  22. Awad, E.; Haddad, C.; Sacre, H.; Hallit, R.; Soufia, M.; Salameh, P.; Hallit, S. Correlates of bullying perpetration among Lebanese adolescents: A national study. BMC Pediatr. 2021, 21. [Google Scholar] [CrossRef]
  23. Alrojolah, L.; Beayno, A.; Shamseddeen, W.; Ghandour, L.; Dirani, L.A.; Maalouf, F. Chronic physical illness and psychiatric comorbidities in lebanese adolescents. In Proceedings of the 66th Annual Meeting, Chicago, IL, USA, 14–19 October 2019. [Google Scholar]
  24. Arseneault, L.; Bowes, L.; Shakoor, S. Bullying victimization in youths and mental health problems: ‘much ado about nothing’? Psychol. Med. 2010, 40, 717–729. [Google Scholar] [CrossRef]
  25. Moore, S.E.; Norman, R.E.; Suetani, S.; Thomas, H.J.; Sly, P.D.; Scott, J.G. Consequences of bullying victimization in childhood and adolescence: A systematic review and meta-analysis. World J. Psychiatry 2017, 7, 60–76. [Google Scholar] [CrossRef]
  26. Takizawa, R.; Maughan, B.; Arseneault, L. Adult Health Outcomes of Childhood Bullying Victimization: Evidence From a Five-Decade Longitudinal British Birth Cohort. Am. J. Psychiatry 2014, 171, 777–784. [Google Scholar] [CrossRef] [PubMed]
  27. Young-Jones, A.; Fursa, S.; Byrket, J.S.; Sly, J.S. Bullying affects more than feelings: The long-term implications of victimization on academic motivation in higher education. Soc. Psychol. Educ. 2015, 18, 185–200. [Google Scholar] [CrossRef]
  28. Reijntjes, A.; Kamphuis, J.H.; Prinzie, P.; Telch, M.J. Peer victimization and internalizing problems in children: A meta-analysis of longitudinal studies. Child Abus. Negl. 2010, 34, 244–252. [Google Scholar] [CrossRef] [PubMed]
  29. Vaillancourt, T.; Brittain, H.L.; McDougall, P.; Duku, E. Longitudinal Links Between Childhood Peer Victimization, Internalizing and Externalizing Problems, and Academic Functioning: Developmental Cascades. J. Abnorm. Child Psychol. 2013, 41, 1203–1215. [Google Scholar] [CrossRef] [PubMed]
  30. Akasyah, W.; Margono, H.M.; Efendi, F. Bullying Victimisation Effect at Physical, Phychological, and Social in Adolescence. In Relationship between Bullying and Social Anxiety and Withdrawal among Adolescents, Proceedings of the 9th International Nursing Conference (INC 2018), Surabaya, Indonesia, 7–8 April 2018; SCITEPRESS–Science and Technology Publications, Lda: Setubal, Portugal, 2018; pp. 538–546. ISBN 978-989-758-336-0. [Google Scholar]
  31. Turner, M.G.; Exum, M.L.; Brame, R.; Holt, T.J. Bullying victimization and adolescent mental health: General and typological effects across sex. J. Crim. Justice 2013, 41, 53–59. [Google Scholar] [CrossRef]
  32. Fisher, H.L.; Moffitt, T.E.; Houts, R.M.; Belsky, D.W.; Arseneault, L.; Caspi, A. Bullying victimisation and risk of self harm in early adolescence: Longitudinal cohort study. BMJ 2012, 344, e2683. [Google Scholar] [CrossRef] [Green Version]
  33. Bannink, R.; Broeren, S.; Van De Looij-Jansen, P.; De Waart, F.; Raat, H. Cyber and Traditional Bullying Victimization as a Risk Factor for Mental Health Problems and Suicidal Ideation in Adolescents. PLoS ONE 2014, 9, e94026. [Google Scholar] [CrossRef] [Green Version]
  34. Casper, D.M.; Card, N.A. Overt and Relational Victimization: A Meta-Analytic Review of Their Overlap and Associations With Social-Psychological Adjustment. Child Dev. 2017, 88, 466–483. [Google Scholar] [CrossRef]
  35. Sullivan, T.N.; Farrell, A.D.; Kliewer, W. Peer victimization in early adolescence: Association between physical and relational victimization and drug use, aggression, and delinquent behaviors among urban middle school students. Dev. Psychopathol. 2006, 18, 119–137. [Google Scholar] [CrossRef]
  36. Anderson, C.A.; Huesmann, L.R. Human Aggression: A Social-Cognitive View. In The Sage Handbook of Social Psychology; SAGE Publications Ltd.: New York, NY, USA, 2007; pp. 259–287. [Google Scholar] [CrossRef] [Green Version]
  37. Buss, A.H.; Perry, M. The aggression questionnaire. J. Personal. Soc. Psychol. 1992, 63, 452. [Google Scholar] [CrossRef]
  38. Walters, G.D. School-Age Bullying Victimization and Perpetration: A Meta-Analysis of Prospective Studies and Research. Trauma Violence Abus. 2021, 22, 1129–1139. [Google Scholar] [CrossRef]
  39. Reijntjes, A.; Kamphuis, J.H.; Prinzie, P.; Boelen, P.A.; van der Schoot, M.; Telch, M. Prospective linkages between peer victimization and externalizing problems in children: A meta-analysis. Aggress. Behav. 2011, 37, 215–222. [Google Scholar] [CrossRef] [PubMed]
  40. Rusby, J.C.; Forrester, K.K.; Biglan, A.; Metzler, C.W. Relationships Between Peer Harassment and Adolescent Problem Behaviors. J. Early Adolesc. 2005, 25, 453–477. [Google Scholar] [CrossRef]
  41. Walters, G.D. Unraveling the Bidirectional Relationship Between Bullying Victimization and Perpetration: A Test of Mechanisms From Opportunity and General Strain Theories. Youth Violence Juv. Justice 2020, 18, 395–411. [Google Scholar] [CrossRef]
  42. Walters, G.D.; Espelage, D.L. From victim to victimizer: Hostility, anger, and depression as mediators of the bullying victimization–bullying perpetration association. J. Sch. Psychol. 2018, 68, 73–83. [Google Scholar] [CrossRef] [PubMed]
  43. Barker, E.D.; Arseneault, L.; Brendgen, M.; Fontaine, N.; Maughan, B. Joint Development of Bullying and Victimization in Adolescence: Relations to Delinquency and Self-Harm. J. Am. Acad. Child Adolesc. Psychiatry 2008, 47, 1030–1038. [Google Scholar] [CrossRef]
  44. Lee, J.M.; Kim, J.; Hong, J.S.; Marsack-Topolewski, C.N. From Bully Victimization to Aggressive Behavior: Applying the Problem Behavior Theory, Theory of Stress and Coping, and General Strain Theory to Explore Potential Pathways. J. Interpers. Violence 2021, 36, 10314–10337. [Google Scholar] [CrossRef]
  45. Calmaestra, J.; Escorial, A.; García, P.; Moral Del, C.; Perazzo, C.; Ubrich, T. Yo a eso No Juego: Bullying y Ciberbullying en la Infancia; Dubidu Estudio: Madrid, Spain, 2016. [Google Scholar]
  46. Bonell, C.; Fletcher, A.; Fitzgerald-Yau, N.; Hale, D.; Allen, E.; Elbourne, D.; Jones, R.; Bond, L.; Wiggins, M.; Miners, A.; et al. Initiating change locally in bullying and aggression through the school environment (INCLUSIVE): A pilot randomised controlled trial. Health Technol. Assess. 2015, 19, 1–109. [Google Scholar] [CrossRef] [Green Version]
  47. World Health Organization. Preventing Youth Violence: An Overview of the Evidence; World Health Organization: Geneve, Switzerland, 2015. [Google Scholar]
  48. Sfeir, E.; Geara, C.; Hallit, S.; Obeid, S. Alexithymia, aggressive behavior and depression among Lebanese adolescents: A cross-sectional study. Child Adolesc. Psychiatry Ment. Health 2020, 14, 32. [Google Scholar] [CrossRef]
  49. Maalouf, E.; Salameh, P.; Haddad, C.; Sacre, H.; Hallit, S.; Obeid, S. Attachment styles and their association with aggression, hostility, and anger in Lebanese adolescents: A national study. BMC Psychol. 2022, 10, 104. [Google Scholar] [CrossRef]
  50. Fung, A.L.C. Adolescent Reactive and Proactive Aggression, and Bullying in Hong Kong: Prevalence, Psychosocial Correlates, and Prevention. J. Adolesc. Health 2019, 64, S65–S72. [Google Scholar] [CrossRef] [PubMed]
  51. Ttofi, M.M.; Farrington, D.P.; Lösel, F.; Loeber, R. The predictive efficiency of school bullying versus later offending: A systematic/meta-analytic review of longitudinal studies. Crim. Behav. Ment. Health 2011, 21, 80–89. [Google Scholar] [CrossRef] [PubMed]
  52. Foster, E.M.; Jones, D.E.; The Conduct Problems Prevention Research Group. The High Costs of Aggression: Public Expenditures Resulting From Conduct Disorder. Am. J. Public Health 2005, 95, 1767–1772. [Google Scholar] [CrossRef] [PubMed]
  53. Yeager, D.S.; Fong, C.J.; Lee, H.Y.; Espelage, D.L. Declines in efficacy of anti-bullying programs among older adolescents: Theory and a three-level meta-analysis. J. Appl. Dev. Psychol. 2015, 37, 36–51. [Google Scholar] [CrossRef]
  54. Kärnä, A.; Voeten, M.; Little, T.D.; Alanen, E.; Poskiparta, E.; Salmivalli, C. Effectiveness of the KiVa antibullying program: Grades 1–3 and 7–9. J. Educ. Psychol. 2013, 105, 535. [Google Scholar] [CrossRef]
  55. Peled, M.; Moretti, M. Rumination on anger and sadness in adolescence: Fueling of fury and deepening of despair. J. Clin. Child Adolesc. Psychol. 2007, 36, 66–75. [Google Scholar] [CrossRef]
  56. Rosen, P.J.; Milich, R.; Harris, M.J. Victims of their own cognitions: Implicit social cognitions, emotional distress, and peer victimization. J. Appl. Dev. Psychol. 2007, 28, 211–226. [Google Scholar] [CrossRef]
  57. Mathieson, L.C.; Klimes-Dougan, B.; Crick, N.R. Dwelling on it may make it worse: The links between relational victimization, relational aggression, rumination, and depressive symptoms in adolescents. Dev. Psychopathol. 2014, 26, 735–747. [Google Scholar] [CrossRef]
  58. Troop-Gordon, W.; Ladd, G.W. Trajectories of Peer Victimization and Perceptions of the Self and Schoolmates: Precursors to Internalizing and Externalizing Problems. Child Dev. 2005, 76, 1072–1091. [Google Scholar] [CrossRef]
  59. Liu, X.; Pan, B.; Chen, L.; Li, T.; Ji, L.; Zhang, W. Healthy context paradox in the association between bullying victimization and externalizing problems: The mediating role of hostile attribution bias. Acta Psychol. Sin. 2021, 53, 170. [Google Scholar] [CrossRef]
  60. Gini, G.; Marino, C.; Spada, M.M. The Role of Metacognitions and Thinking Styles in the Negative Outcomes of Adolescents’ Peer Victimization. Violence Vict. 2019, 34, 752–769. [Google Scholar] [CrossRef] [PubMed]
  61. Ehring, T.; Watkins, E. Repetitive negative thinking as a transdiagnostic process. Int. J. Cogn. Ther. 2008, 1, 192–205. [Google Scholar] [CrossRef]
  62. Ehring, T.; Ehlers, A. Does rumination mediate the relationship between emotion regulation ability and posttraumatic stress disorder? Eur. J. Psychotraumatol. 2014, 5, 23547. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  63. Caselli, G.; Gemelli, A.; Querci, S.; Lugli, A.M.; Canfora, F.; Annovi, C.; Rebecchi, D.; Ruggiero, G.M.; Sassaroli, S.; Spada, M.M.; et al. The effect of rumination on craving across the continuum of drinking behaviour. Addict. Behav. 2013, 38, 2879–2883. [Google Scholar] [CrossRef] [PubMed]
  64. Edwards, M.J.; Tang, N.K.; Wright, A.M.; Salkovskis, P.M.; Timberlake, C.M. Thinking about thinking about pain: A qualitative investigation of rumination in chronic pain. Pain Manag. 2011, 1, 311–323. [Google Scholar] [CrossRef]
  65. Watkins, E.R. Constructive and unconstructive repetitive thought. Psychol. Bull. 2008, 134, 163–206. [Google Scholar] [CrossRef] [Green Version]
  66. Bauer, E.A.; Braitman, A.L.; Judah, M.R.; Cigularov, K.P. Worry as a mediator between psychosocial stressors and emotional sequelae: Moderation by contrast avoidance. J. Affect. Disord. 2020, 266, 456–464. [Google Scholar] [CrossRef]
  67. Liu, C.; Liu, Z.; Yuan, G. The longitudinal influence of cyberbullying victimization on depression and posttraumatic stress symptoms: The mediation role of rumination. Arch. Psychiatr. Nurs. 2020, 34, 206–210. [Google Scholar] [CrossRef]
  68. Malamut, S.T.; Salmivalli, C. Rumination as a Mediator of the Prospective Association Between Victimization and Bullying. Res. Child Adolesc. Psychopathol. 2021, 49, 339–350. [Google Scholar] [CrossRef]
  69. Valencia, P.D.; De Psicología, P.A.P.C.-C. Initial psychometric evaluation of the Perseverative Thinking Questionnaire in Peruvian undergraduates. Liberabit. Rev. Peru. De Psicol. 2020, 26, e404. [Google Scholar] [CrossRef]
  70. Olweus, D. The Revised Olweus Bully/Victim Questionnaire for Students; University of Bergen: Bergen, Norway, 1996. [Google Scholar]
  71. Topper, L.R.; Castellanos-Ryan, N.; Mackie, C.; Conrod, P.J. Adolescent bullying victimisation and alcohol-related problem behaviour mediated by coping drinking motives over a 12month period. Addict. Behav. 2011, 36, 6–13. [Google Scholar] [CrossRef] [PubMed]
  72. Alsaleem, M.A.; Alhashem, H.A.; Alsaleem, S.A.; Mahfouz, A.A. Bullying Prevalence among Secondary School Children in Khamis Mushait City, Southwestern Saudi Arabia. Behav. Sci. 2021, 11, 134. [Google Scholar] [CrossRef] [PubMed]
  73. Second Step. Skills for Social and Academic Success, Illinois Bully Scale: Student Version. 2013.
  74. Ehring, T.; Zetsche, U.; Weidacker, K.; Wahl, K.; Schönfeld, S.; Ehlers, A. The Perseverative Thinking Questionnaire (PTQ): Validation of a content-independent measure of repetitive negative thinking. J. Behav. Ther. Exp. Psychiatry 2011, 42, 225–232. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  75. Devynck, F.; Kornacka, M.; Baeyens, C.; Serra, E.; Das Neves, J.F.; Gaudrat, B.; Delille, C.; Taquet, P.; Depraete, O.; Tison, P.; et al. Perseverative Thinking Questionnaire (PTQ): French Validation of a Transdiagnostic Measure of Repetitive Negative Thinking. Front. Psychol. 2017, 8, 2159. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  76. Ruiz, F.J.; Salazar, D.M.; Suárez-Falcón, J.C.; Peña-Vargas, A.; Ehring, T.; Barreto-Zambrano, M.L.; Gómez-Barreto, M.P. Psychometric Properties and Measurement Invariance Across Gender and Age-Group of the Perseverative Thinking Questionnaire–Children (PTQ-C) in Colombia. Assessment 2020, 27, 1657–1667. [Google Scholar] [CrossRef] [PubMed]
  77. Kornacka, M.; Buczny, J.; Layton, R.L. Assessing Repetitive Negative Thinking Using Categorical and Transdiagnostic Approaches: A Comparison and Validation of Three Polish Language Adaptations of Self-Report Questionnaires. Front. Psychol. 2016, 7, 322. [Google Scholar] [CrossRef] [Green Version]
  78. Pechorro, P.; Barroso, R.; Poiares, C.; Oliveira, J.P.; Torrealday, O. Validation of the Buss–Perry Aggression Questionnaire-Short Form among Portuguese juvenile delinquents. Int. J. Law Psychiatry 2016, 44, 75–80. [Google Scholar] [CrossRef]
  79. Zimonyi, S.; Kasos, K.; Halmai, Z.; Csirmaz, L.; Stadler, H.; Rózsa, S.; Szekely, A.; Kotyuk, E. Hungarian validation of the Buss-Perry Aggression Questionnaire-Is the short form more adequate? Brain Behav. 2021, 11, e02043. [Google Scholar] [CrossRef]
  80. Kuzucu, Y.; Ertürk, Ö. Psychometric Properties of Turkish Version of Aggression Questionnaire Short Form: Measurement Invariance and Differential Item Functioning across Sex and Age. J. Meas. Eval. Educ. Psychol. 2020, 11, 243–265. [Google Scholar]
  81. Maxwell, J.P. Development and Preliminary Validation of a Chinese Version of the Buss–Perry Aggression Questionnaire in a Population of Hong Kong Chinese. J. Pers. Assess. 2007, 88, 284–294. [Google Scholar] [CrossRef]
  82. Gallardo-Pujol, D.; Kramp, U.; Forero, C.G.; Pérez-Ramírez, M.; Andrés-Pueyo, A. Assessing aggressiveness quickly and efficiently: The Spanish adaptation of Aggression Questionnaire-Refined version. Eur. Psychiatry 2006, 21, 487–494. [Google Scholar] [CrossRef] [PubMed]
  83. Kusump, S.; Vongjaturapat, N.; Ongkhambanchong, S. The Validation of Aggression Questionnaire in Thai Version. J. Sport. Sci. Technol. 2015, 15, 365–377. [Google Scholar]
  84. Kamusella, T. The Arabic language: A Latin of modernity? J. Natl. Mem. Lang. Politics 2017, 11, 117–145. [Google Scholar] [CrossRef] [Green Version]
  85. Ziegler, M.; Bensch, D. Lost in Translation: Thoughts Regarding the Translation of Existing Psychological Measures Into Other Languages. Eur. J. Psychol. Assess. 2013, 29, 81–83. [Google Scholar] [CrossRef]
  86. Parker, J.G.; Rubin, K.H.; Erath, S.A.; Wojslawowicz, J.C.; Buskirk, A.A. Peer Relationships, Child Development, and Adjustment: A Developmental Psychopathology Perspective; 2006. In Developmental Psychopathology: Theory and Method; Cicchetti, D., Cohen, D.J., Eds.; John Wiley & Sons, Inc.: New York, NY, USA, 2006; pp. 419–493. [Google Scholar]
  87. Petersen, I.T.; Bates, J.E.; Dodge, K.A.; Lansford, J.E.; Pettit, G.S. Describing and predicting developmental profiles of externalizing problems from childhood to adulthood. Dev. Psychopathol. 2015, 27, 791–818. [Google Scholar] [CrossRef] [Green Version]
  88. Brito, C.C.; Oliveira, M.T. Bullying and self-esteem in adolescents from public schools. J. De Pediatr. 2013, 89, 601–607. [Google Scholar] [CrossRef] [Green Version]
  89. Oliveira, W.A.D.; Silva, M.A.I.; Silva, J.L.D.; Mello, F.C.M.D.; Prado, R.R.D.; Malta, D.C. Associations between the practice of bullying and individual and contextual variables from the aggressors’ perspective. J. De Pediatr. 2016, 92, 32–39. [Google Scholar] [CrossRef] [Green Version]
  90. Volk, A.A.; Veenstra, R.; Espelage, D.L. So you want to study bullying? Recommendations to enhance the validity, transparency, and compatibility of bullying research. Aggress. Violent Behav. 2017, 36, 34–43. [Google Scholar] [CrossRef]
  91. Smith, P.K.; López-Castro, L.; Robinson, S.; Görzig, A. Consistency of gender differences in bullying in cross-cultural surveys. Aggress. Violent Behav. 2019, 45, 33–40. [Google Scholar] [CrossRef]
  92. Fry, D.P. Cross-Cultural Differences in Aggression. In The Wiley Handbook of Violence and Aggression; John Wiley & Sons, Inc.: Hoboken, NJ, USA, 2017; pp. 1–12. [Google Scholar]
  93. Fritz, M.S.; MacKinnon, D.P. Required Sample Size to Detect the Mediated Effect. Psychol. Sci. 2007, 18, 233–239. [Google Scholar] [CrossRef] [Green Version]
  94. WHO. Body Mass Index—BMI. Available online: https://www.euro.who.int/en/health-topics/disease-prevention/nutrition/a-healthy-lifestyle/body-mass-index-bmi (accessed on 17 March 2021).
  95. Weary-Smith, K.A. Validation of the Physical Activity Index (PAI) as a Measure of Total Activity Load and Total Kilocalorie Expenditure during Submaximal Treadmill Walking; University of Pittsburgh: Pittsburgh, PA, USA, 2007. [Google Scholar]
  96. Melki, I.S.; Beydoun, H.A.; Khogali, M.; Tamim, H.; Yunis, K.A. Household crowding index: A correlate of socioeconomic status and inter-pregnancy spacing in an urban setting. J. Epidemiol. Community Health 2004, 58, 476–480. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  97. Malaeb, D.; Awad, E.; Haddad, C.; Salameh, P.; Sacre, H.; Akel, M.; Soufia, M.; Hallit, R.; Obeid, S.; Hallit, S. Bullying victimization among Lebanese adolescents: The role of child abuse, Internet addiction, social phobia and depression and validation of the Illinois Bully Scale. BMC Pediatr. 2020, 20, 520. [Google Scholar] [CrossRef] [PubMed]
  98. Espelage, D.L.; Holt, M.K. Bullying and Victimization during Early Adolescence: Peer influences and Psychosocial Correlates. J. Emot. Abus. 2001, 2, 123–142. [Google Scholar] [CrossRef]
  99. Bryant, F.B.; Smith, B.D. Refining the Architecture of Aggression: A Measurement Model for the Buss–Perry Aggression Questionnaire. J. Res. Pers. 2001, 35, 138–167. [Google Scholar] [CrossRef]
  100. Hair, J.F., Jr.; Hult GT, M.; Ringle, C.M.; Sarstedt, M. A Primer on Partial Least Squares Structural Equation Modeling (PLS-SEM); SAGE Publications Ltd.: New York, NY, USA, 2021. [Google Scholar]
  101. Zhou, Z.-K.; Liu, Q.-Q.; Niu, G.-F.; Sun, X.-J.; Fan, C.-Y. Bullying victimization and depression in Chinese children: A moderated mediation model of resilience and mindfulness. Personal. Individ. Differ. 2017, 104, 137–142. [Google Scholar] [CrossRef]
  102. Jeong, S.; Kwak, D.-H.; Moon, B.; Miguel, C.S. Predicting School Bullying Victimization: Focusing on Individual and School Environmental/Security Factors. J. Criminol. 2013, 2013, 401301. [Google Scholar] [CrossRef]
  103. Zeinoun, P.; Akl, E.A.; Maalouf, F.T.; Meho, L.I. The Arab region’s contribution to global mental health research (2009–2018): A bibliometric analysis. Front. Psychiatry 2020, 11, 182. [Google Scholar] [CrossRef] [Green Version]
  104. Archer, J.; Kilpatrick, G.; Bramwell, R. Comparison of two aggression inventories. Aggress. Behav. 1995, 21, 371–380. [Google Scholar] [CrossRef]
  105. Harris, J.A. A further evaluation of The Aggression Questionnaire: Issues of validity and reliability. Behav. Res. Ther. 1997, 35, 1047–1053. [Google Scholar] [CrossRef]
  106. Taber, K.S. The Use of Cronbach’s Alpha When Developing and Reporting Research Instruments in Science Education. Res. Sci. Educ. 2018, 48, 1273–1296. [Google Scholar] [CrossRef] [Green Version]
  107. Lereya, S.T.; Copeland, W.E.; Zammit, S.; Wolke, D. Bully/victims: A longitudinal, population-based cohort study of their mental health. Eur. Child Adolesc. Psychiatry 2015, 24, 1461–1471. [Google Scholar] [CrossRef] [PubMed]
  108. Rudolph, K.D.; Lansford, J.E.; Agoston, A.M.; Sugimura, N.; Schwartz, D.; Dodge, K.A.; Pettit, G.S.; Bates, J.E. Peer Victimization and Social Alienation: Predicting Deviant Peer Affiliation in Middle School. Child Dev. 2014, 85, 124–139. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  109. Kochenderfer-Ladd, B. Peer Victimization: The Role of Emotions in Adaptive and Maladaptive Coping. Soc. Dev. 2004, 13, 329–349. [Google Scholar] [CrossRef]
  110. Bronfenbrenner, U. The Ecology of Human Development: Experiments by Nature and Design; Harvard University Press: Cambridge, MA, USA, 1979. [Google Scholar]
  111. Yeung, R.S.; Leadbeater, B.J. Does hostile attributional bias for relational provocations mediate the short-term association between relational victimization and aggression in preadolescence? J. Youth Adolesc. 2007, 36, 973–983. [Google Scholar] [CrossRef]
  112. Rosen, P.J.; Milich, R.; Harris, M.J. Why’s Everybody Always Picking on Me? Social Cognition, Emotion Regulation, and Chronic Peer Victimization in Children; Springer: New York, NY, USA, 2009. [Google Scholar]
  113. Moon, B.; Morash, M.; McCluskey, J.D. General strain theory and school bullying: An empirical test in South Korea. Crime Delinq. 2012, 58, 827–855. [Google Scholar] [CrossRef]
  114. Bronfenbrenner, U. Contexts of child rearing: Problems and prospects. Am. Psychol. 1979, 34, 844. [Google Scholar] [CrossRef]
  115. McDougall, P.; Vaillancourt, T. Long-term adult outcomes of peer victimization in childhood and adolescence: Pathways to adjustment and maladjustment. Am. Psychol. 2015, 70, 300–310. [Google Scholar] [CrossRef] [Green Version]
  116. Cornell, D.; Limber, S.P. Law and policy on the concept of bullying at school. Am. Psychol. 2015, 70, 333–343. [Google Scholar] [CrossRef]
  117. Olweus, D. A useful evaluation design, and effects of the Olweus Bullying Prevention Program. Psychol. Crime Law 2005, 11, 389–402. [Google Scholar] [CrossRef]
  118. Hanewinkel, R. Prevention of bullying in German schools: An evaluation of an anti-bullying approach. In Bullying in Schools: How Successful Can Interventions Be; Smith, P., Pepler, D., Rigby, K., Eds.; Cambridge University Press: Cambridge, UK, 2004. [Google Scholar]
  119. Ruiz, F.J.; Hernández, D.R.; Falcón JC, S.; Luciano, C. Effect of a one-session ACT protocol in disrupting repetitive negative thinking: A randomized multiple-baseline design. Int. J. Psychol. Psychol. Ther. 2016, 16, 213–233. [Google Scholar]
  120. Dereix-Calonge, I.; Ruiz, F.J.; Sierra, M.A.; Peña-Vargas, A.; Ramírez, E.S. Acceptance and commitment training focused on repetitive negative thinking for clinical psychology trainees: A randomized controlled trial. J. Context. Behav. Sci. 2019, 12, 81–88. [Google Scholar] [CrossRef]
  121. Swearer, S.M.; Martin, M.; Brackett, M.; Palacios, R.A. Bullying Intervention in Adolescence: The Intersection of Legislation, Policies, and Behavioral Change. Adolesc. Res. Rev. 2017, 2, 23–35. [Google Scholar] [CrossRef] [Green Version]
Children 10 00598 g001 550
Figure 1. Standardized factor loadings of the three-factor model of the Arabic version of the Perseverative Thinking Questionnaire (PTQ) (p < 0.001 for all loading factors). F1 = PTQ core features, F2 = PTQ mental resources, F3 = unproductiveness.
Figure 1. Standardized factor loadings of the three-factor model of the Arabic version of the Perseverative Thinking Questionnaire (PTQ) (p < 0.001 for all loading factors). F1 = PTQ core features, F2 = PTQ mental resources, F3 = unproductiveness.
Children 10 00598 g001
Children 10 00598 g002 550
Figure 2. Standardized factor loadings of the four-factor model of the Arabic version of the Buss–Perry Aggression Questionnaire-Short Form (p < 0.001 for all loading factors). F1= Physical aggression, F2 = Verbal aggression, F3 = Anger, F4 = Hostility.
Figure 2. Standardized factor loadings of the four-factor model of the Arabic version of the Buss–Perry Aggression Questionnaire-Short Form (p < 0.001 for all loading factors). F1= Physical aggression, F2 = Verbal aggression, F3 = Anger, F4 = Hostility.
Children 10 00598 g002
Children 10 00598 g003 550
Figure 3. Path analysis model with physical aggression as the dependent variable.
Figure 3. Path analysis model with physical aggression as the dependent variable.
Children 10 00598 g003
Children 10 00598 g004 550
Figure 4. Path analysis model with verbal aggression as the dependent variable.
Figure 4. Path analysis model with verbal aggression as the dependent variable.
Children 10 00598 g004
Children 10 00598 g005 550
Figure 5. Path analysis model with hostility as the dependent variable.
Figure 5. Path analysis model with hostility as the dependent variable.
Children 10 00598 g005
Children 10 00598 g006 550
Figure 6. Path analysis model with anger as the dependent variable.
Figure 6. Path analysis model with anger as the dependent variable.
Children 10 00598 g006
Table
Table 1. Sociodemographic and other characteristics of the participants (N = 379).
Table 1. Sociodemographic and other characteristics of the participants (N = 379).
VariableN (%)
Sex
 Male133 (35.1%)
 Female 246 (64.9%)
Mean ± SD
Age (in years)16.07 ± 1.19
Physical activity index27.78 ± 20.15
Household crowding index (persons/room)1.26 ± 0.74
Body mass index (kg/m2)22.33 ± 3.79
Financial burden 4.96 ± 2.80
Physical aggression6.59 ± 2.90
Verbal aggression7.46 ± 2.85
Anger8.12 ± 3.29
Hostility6.80 ± 3.12
PTQ core features16.66 ± 8.86
PTQ mental resources4.86 ± 3.25
PTQ unproductiveness5.12 ± 3.27
Bullying victimization3.30 ± 5.01
Table
Table 2. Bivariate analysis of the categorical variables associated with aggression scores.
Table 2. Bivariate analysis of the categorical variables associated with aggression scores.
VariablePhysical AggressionVerbal AggressionAngerHostility
Sex
Male7.03 ± 3.297.50 ± 3.047.53 ± 3.226.55 ± 3.25
Female 6.36 ± 2.647.44 ± 2.768.45 ± 3.296.94 ± 3.04
p0.0430.8420.0090.251
Significant p-values are indicated in bold.
Table
Table 3. Bivariate analysis of the continuous variables associated with the aggression scores.
Table 3. Bivariate analysis of the continuous variables associated with the aggression scores.
VariablePhysical AggressionVerbal AggressionAngerHostility
rprprprp
Physical aggression1-
Verbal aggression0.46<0.0011-
Anger0.50<0.0010.51<0.0011-
Hostility0.49<0.0010.49<0.0010.67<0.0011-
Bullying victimization0.41<0.0010.23<0.0010.20<0.0010.33<0.001
PTQ core features0.35<0.0010.40 <0.0010.54<0.0010.59<0.001
PTQ mental resources0.28<0.0010.34<0.0010.47<0.0010.56<0.001
PTQ unproductiveness0.30<0.0010.37<0.0010.48<0.0010.57<0.001
Age0.020.7480.130.0140.080.1450.020.645
Physical activity index0.030.547−0.010.808−0.040.442−0.060.213
Household crowding index −0.050.344−0.070.1720.040.475−0.050.332
Body mass index 0.100.0430.070.1540.030.6040.070.181
Financial burden 0.070.1490.050.3490.090.0860.140.006
Significant p-values are indicated in bold; r = Pearson correlation coefficient.
Table
Table 4. Fit indices of the path analyses.
Table 4. Fit indices of the path analyses.
Dependent Variableχ2/dfpCFISRMRRMSEA90% CIPclose
Physical aggression23.81/6 = 3.97<0.0010.890.0570.0890.053–0.1270.038
Verbal aggression11.96/6 = 1.99<0.0010.940.0390.0510.001–0.0940.421
Hostility17.64/6 = 2.94<0.0010.950.0470.0720.034–0.1120.151
Anger15.78/5 = 3.15<0.0010.940.0430.0760.035–0.1190.131
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Fekih-Romdhane, F.; Malaeb, D.; Sarray El Dine, A.; Yakın, E.; Hallit, S.; Obeid, S. Association between Bullying Victimization and Aggression in Lebanese Adolescents: The Indirect Effect of Repetitive Negative Thinking—A Path Analysis Approach and Scales Validation. Children 2023, 10, 598. https://doi.org/10.3390/children10030598

AMA Style

Fekih-Romdhane F, Malaeb D, Sarray El Dine A, Yakın E, Hallit S, Obeid S. Association between Bullying Victimization and Aggression in Lebanese Adolescents: The Indirect Effect of Repetitive Negative Thinking—A Path Analysis Approach and Scales Validation. Children. 2023; 10(3):598. https://doi.org/10.3390/children10030598

Chicago/Turabian Style

Fekih-Romdhane, Feten, Diana Malaeb, Abir Sarray El Dine, Ecem Yakın, Souheil Hallit, and Sahar Obeid. 2023. "Association between Bullying Victimization and Aggression in Lebanese Adolescents: The Indirect Effect of Repetitive Negative Thinking—A Path Analysis Approach and Scales Validation" Children 10, no. 3: 598. https://doi.org/10.3390/children10030598

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop