Geelong Cyber Cats: Evaluation of an Intervention to Prevent Cyberbullying Behaviours

Abstract

Cyberbullying behaviours can result in serious adverse mental health outcomes. We report evaluation findings from the Geelong Cyber Cats, a one-day community-based cyberbullying prevention intervention targeting Year 7 adolescents. The longitudinal, mixed-methods evaluation involved students completing surveys at baseline (N = 309), intervention completion (N = 316), and 3-month follow-up (N = 154). Controls (N = 58) who had not attended the intervention completed follow-up surveys. We measured behaviours, attitudes, and intentions regarding cyberbullying, and mental health. Positive attitudes towards cyberbullying significantly reduced from baseline to post intervention, with some rebound at follow-up. There was a significant reduction in experiences of cyberbullying victimisation, perpetration, and witnessing for participants at follow-up. Mental health improved from baseline to follow-up although not significantly. Confidence in responding to cyberbullying significantly improved post-intervention, with a loss of some gains at follow-up but remaining improvement compared to baseline. There were significant increases in post- intervention intentions, including being kind, careful and safe, and disengagement from problematic applications or social media platforms. Participants were positive about the intervention impact on understanding cyberbullying, and increased motivation and confidence to respond. Content about responding to cyberbullying, and helping others, was perceived the most helpful. The intervention demonstrates benefits to cyberbullying attitudes, behaviours and mental health.

1. Introduction

As digital communication technologies become ubiquitous, engagement in cyberbullying behaviours, especially among youth, is of increasing concern [1]. Defined as intentional and repetitive acts of aggression, perpetrated via electronic means by a single individual or a group of people against a victim who cannot easily defend himself or herself [2], cyberbullying behaviours aim to inflict harm or cause discomfort, and are perceived by victims as unwanted and distressing [3,4]. Cyberbullying behaviours are relatively frequent in adolescence. Prevalence rates vary considerably, depending on the measures used, construct definitions and timeframes of interest, and the populations and methodologies. One meta-analytic study of in-person and cyber-bullying prevalence estimated mean prevalence rates for cyberbullying involvement (across both perpetration and victimisation) at 15% for young people aged 12–18 years [5]. A more recent Australian meta-analysis of child and adolescent cyberbullying, based on 46 studies, indicated lifetime prevalence rates of 7.0% for victimisation and 3.5% for perpetration [6]. However, other studies show higher average victimisation rates ranging from 20% to 40% [4]. Engagement in cyberbullying behaviours tends to peak in middle school with boys and girls equally likely to be perpetrators and/or victims [4,7,8]. Notably, cyberbullying victimisation is strongly and positively related to cyberbullying perpetration [8], suggesting that young people are often involved in reciprocal cyberbullying roles, as both perpetrators and victims.

Cyberbullying evokes serious social concerns, as victimised children, adolescents, and adults are at an increased risk of experiencing depression, anxiety, stress, poorer self-esteem, suicidal ideation, physical and somatic symptoms, self-harm, substance use, social and conduct problems [8,9,10,11]. Specifically, Hinduja and Patchin [12] found that cyber victimisation predicted suicidal thoughts and behaviours in affected adolescents, and was associated with an approximately two-fold increase in suicide attempts. Those who engage in cyberbullying also suffer worse psycho-social functioning, report higher levels of alcohol and substance use, psychiatric symptoms, lower self-esteem and life satisfaction, and poorer academic attainment [8,13]. Further, those engaging in cyberbullying behaviours tend to have poorer physical health, with higher rates of school absenteeism due to feeling unwell [13].

Given the valid concerns raised by adolescent cyberbullying researchers, there is a critical need to develop and evaluate prevention and education strategies, interventions or programs which seek to prevent cyberbullying behaviours and mitigate their potential harms, and which operate across the spectrum of prevention interventions from universal to selective and indicated program designs. Existing universal approaches include macro-level legislative or policy approaches. For example, the Australian Criminal Code Act 1995 (Cth) stipulates that harassing and bullying someone online constitutes an offence, resulting in up to three years imprisonment or fines exceeding A $30,000 [14]. In the USA, 49 states have implemented anti-bullying laws [15]. Such approaches deliver some success, with a review of their effectiveness measured across 25 US states and 59,472 students from grades 9 to 12 finding that students in states with anti-bullying policies including at least one legislative component were 20% less likely to report cyberbullying than students from states without such policies [15]. Specific legislative aspects associated with reductions in cyberbullying included compliance with anti-bullying guidelines sanctioned by educational bodies, statements of scope that outline where the legislation applies and in what circumstances schools can take action; descriptions of bullying behaviours; and obligations for local authorities to develop and implement relevant policies within defined timeframes [15].

However, researchers recognise that whilst legislative approaches have shown some effect, cyberbullying behaviours are ideally tackled by either universal or selective prevention approaches operating on micro, typically school, levels [16]. Cyberbullying prevention programs are designed either to prevent instances of online aggression, or to intervene in cases in which cyberbullying is already known to be occurring [17], or both. A taxonomy proposed by Cantone et al. [18] divides programs tackling cyberbullying into two groups. Universal prevention programs aim to change the overall school climate and decrease peer aggression and are often implemented by teaching staff and parents. Within this group are “whole school” and “multi-level” programs that intervene at multiple layers such as individual students, classrooms, schools, and parents. The second group of focused interventions target specific subgroups such as victims, bullies, and bystanders, thereby operating on a single level of action [18].

Several systematic reviews of school-based cyberbullying prevention and/or intervention programs have been published in recent years. A review of 17 interventions addressing both in-person bullying and cyberbullying found that relative to focused interventions, universal programs tended to be more effective in the areas of improving school climate and well-being of students and reducing negative consequences of victimisation. However, the authors noted that the long-term effects of these interventions have often dissipated at follow-up [18]. Similarly, a systematic review of 17 cyberbullying intervention and prevention programs [17] identified diversity across program components, duration and measurement approaches, but generally demonstrated some efficacy against both in-person and online behaviours, although many programs provided only descriptive, rather than inferential statistics regarding program outcomes.

Another systematic review [19] examined 12 whole school cyberbullying intervention programs predominantly targeting 12–13 year-olds. Again, programs varied in aims (internet safety skill training vs. addressing cyberbullying behaviours), modes of delivery, duration and intensity, involvement of parents, teachers and wider community. Only five out of 12 studies reported lower incidence of cyberbullying victimisation, whilst six studies demonstrated a decrease in cyberbullying perpetration. Although the programs targeted cognitive, behavioural and social variables in line with evidence-based risk and protective factors, variability in scientific merit associated with the measurement of these interventions prevented the authors from providing program recommendations [19].

Lastly, a review of interventions addressing information and communication technologies (ICTs) identified that only two (No Trap! and KiVa) of nine studies relevant to cyberbullying had demonstrated effectiveness in reducing victimisation and perpetration rates [20]. Based on a meta-analysis of 6 programs, the effectiveness of school cyberbullying programs has been identified as small (Hedges g = 0.13) [21]. A more recent meta-analysis based on 26 evaluations [22] indicated that findings were indicative of reducing cyberbullying perpetration by approximately 9–15% and reducing cyberbullying victimisation by approximately 14–15%. As noted by Della Cioppa, O’Neil [19], the effectiveness of many interventions has proven hard to establish, particularly in the absence of control groups and adequate follow-up measurements, or limited information regarding outcomes.

In the absence of systematic evaluation and peer-review of findings, schools and community groups wishing to implement effective and evidence-based programs are limited in their options. In addition, of the published studies and reviews available, few have been evaluated within the Australian context, which may be of particular relevance given the existence of a specific government regulator focused on e-safety: the e-Safety Commissioner, which is currently the first of its kind world-wide. The current study aimed to evaluate a recently revised community-based cyberbullying prevention intervention to determine efficacy for the target population of young adolescents in a regional setting in Australia. Further, to address the methodological limitations outlined above, we followed the guidelines outlined in the bullying research checklist [23].

Intervention Overview

Cyber Cats is an interactive intervention to prevent cyberbullying behaviours delivered in partnership between a local professional sporting club and a youth mental health service agency. Intervention content is tailored to address issues that students in prior cohorts have identified as being of concern, including cyberbullying and sharing explicit images. The content was developed by the delivery agencies, and then reviewed by academics E Clancy and B Klettke to ensure that it is evidence-based, with a comprehensive intervention manual including a program logic setting out the intervention components and key outcomes. Until 2020, Cyber Cats was offered to year 7 students as a full-day, free, off-site intervention, conducted at community facilities attached to a local professional sporting club. Presenters include wellbeing staff from a health service agency, law enforcement representatives, and professional athletes who are ambassadors for positive online behaviours. To maximise student engagement, participants also tour the Geelong football stadium during the day and engage in physical games led by club staff. Topics covered in the intervention program include:

(1)

Defining cyberbullying and how to recognise relevant behaviours in the context of online communications and device-based interactions.

(2)

The “normalisation” of cyberbullying behaviours, particularly outside school settings.

(3)

The potential impacts of cyberbullying for individuals

(4)

The potential responses to cyberbullying, in particular how to respond to being cyberbullied oneself and/or how to respond as a bystander.

(5)

Help-seeking behaviours when experiencing cyberbullying or observing others being cyberbullied.

(6)

Laws around cyberbullying and the creation and transmission of intimate images

(7)

Potential responses to being pressured to send intimate images.

(8)

Developing standards of behaviour for device users.

(9)

A practice exercise how to file a cyberbullying complaint with the relevant complaints agency (Australia’s e-Safety Commissioner, https://www.esafety.gov.au/report (accessed on 11 February 2020)).

2. Materials and Methods

2.1. Evaluation Design

The current evaluation was based on principles outlined by Volk et al. [23], to maximise the intervention’s theoretical strengths and reliability. As such, the guidelines recommend that any intervention and subsequent evaluation should include the following: (1) identified and justified definition of cyberbullying, (2) a theoretical logic underlying the intervention, (3) a logic model and theoretically grounded predictions to determine measurements, (4) implementing an appropriate research design and (5) reflecting upon the final product and evaluation, including its potential limitations.

A longitudinal mixed-methods evaluation design was employed, as detailed in Figure 1 below. Students completed anonymous surveys online at three time points: prior to commencing the intervention activity day, immediately after completing the intervention activity day (i.e., at the end of the day), and a follow-up survey administered within 8–12 weeks following the intervention (depending on school availability). In all cases, participants were provided with information about the purpose of the surveys prior to completion and could opt out with no impact on their participation. A control cohort of students enrolled at participating schools, but who had not attended the Cyber Cats intervention also completed follow-up surveys, facilitated by their schools. All surveys were anonymous, with paper copies distributed to intervention attendees and collected for data entry on site and at follow-up, and control surveys distributed via participating schools. To protect privacy, individual data was not collected with surveys, hence we were unable to match baseline, post and follow-up surveys for program participants.

2.2. Participants

A total of 309 adolescent participants (112 girls, 193 boys, 4 non-binary or did not specify gender) completed baseline measures (some baseline measures were missed due to late arrivals). Students were drawn from Year 7 classes in schools within a regional area of Victoria, Australia, and were aged 12–13 years. Post intervention, 316 responses were received (114 girls, 194 boys and 8 non-binary or did not specify gender). At follow-up, responses were obtained from 154 program participants (44 girls, 108 boys, 2 non-binary or unspecified). In addition, 58 participants (18 girls, 40 boys) from local schools where Cyber Cats operates, but who had not attended the intervention, were recruited to a control condition, and completed follow-up surveys. Informed consent was obtained from all subjects involved in the study by the Cyber Cats facilitators. As the data were collected anonymously by the operator, Ethics Approval for third party data analysis was sought and granted by Deakin University’s Human Research Ethics Committee.

2.3. Measures

Evaluation measures included the following standardised scales:

Positive Attitudes to Cyberbullying Questionnaire (PACQ; [24]). This is a 9-item measure which assesses positive attitudes towards cyberbullying. Participants are asked to indicate how much they agree or disagree with supplied statements, using a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree). In this study, wording was adapted from the initial study, removing reference to “groups on Facebook or MySpace”, and updating with current social media sites, specifically Instagram, Snapchat, WhatsApp and Facebook. A sample item is: It is acceptable to send mean messages to others when they deserve it. After reverse coding relevant items, a total scale score was calculated from the sum of individual items. Scores can range from 9–45, with higher scores indicating more positive attitudes towards cyberbullying, and a mean score of 27 implying neutral attitudes. The scale demonstrated acceptable reliability, with a Cronbach’s alpha of 0.64 for this sample.

Cyberbullying Victimisation, Perpetration, and Bystanding were assessed using measures drawn from Ybarra, Diener-West & Leaf [25]. These scales consist of three items each, asking how frequently in the past year the individual either engaged in cyberbullying behaviours, was the target of such behaviours or witnessed them online. A sample item is: In the last year, how many times did you receive rude or nasty comments from someone while online? Response options provided were everyday/almost everyday, once or twice a week, once or twice a month, a few times a year, less than a few times a year, and never. All scores were reverse coded for analysis, so that higher scores on the scale indicate more cyberbullying behaviours. Both a total and binary score are possible. To obtain total scores, responses were summed, with a possible total score for each behaviour ranging from 0–15. In addition, as a binary measure of behaviour, participants who reported any of the three experiences in the previous year were coded as having experienced cyberbullying/perpetrated cyberbullying/witnessed cyberbullying (bystanding). All scales demonstrated good reliability, with Cronbach’s alphas of 0.84, 0.84 and 0.85, respectively.

Psychological Distress: Students completed a measure of mental health at both baseline and follow-up. The Kessler-10 (K10) is a 10-item questionnaire designed to measure non-specific psychological distress in community epidemiological surveys [26]. The K10 consists of 10 symptoms of anxiety and depression (e.g., “how often did you feel tired out for no good reason?”). Students were asked to indicate on a 5-point Likert scale (1 = “none of the time”, 2 = “a little of the time”, 3 = “some of the time”, 4 = most of the time, 5 = “all of the time”) which statement best describes how they have been feeling during the past 30 days. A total score is calculated from the sum of individual scores, with higher scores indicating greater levels of psychological distress. This measure has been validated in child and adolescent settings in Australia [27], and demonstrated excellent reliability in this study, with a Cronbach’s alpha of 0.93.

Additional measures were also included as follows:

Confidence in Responding: Participants were asked to self-assess their confidence in responding to cyberbullying at baseline, post intervention and follow-up, using the following item: How confident do you feel about responding to cyberbullying when you see it online? Responses were scored from 1 = Not at all, to 10 = Very confident.

Contextual Measures: Qualitative measures included indicating what actions they would take if they experienced cyberbullying themselves, or witnessed it happening to someone else, as well as their learning aims for the intervention at baseline, whilst at post- intervention and follow-up, they were asked what they had found most useful.

2.4. Data Analyses

All quantitative data was analysed using IBM SPSS V28, while manual and visual review was used for qualitative review of written responses. Descriptive statistics included means, standard deviations, and percentages where applicable. The choice of inferential analyses was based on the relevant research questions and nature of the data, and included t-tests, ANOVAs, and chi-square tests of independence, based on whether data were continuous or categorical. Unfortunately, the nature of data collection did not permit matching of participants across time-points, hence analyses of change used independent groups measures. Whilst this is not ideal, it is a more conservative approach, with reduced power, and hence was chosen as the appropriate method in this situation. Qualitative responses to questions regarding program content and planned changes were analysed via content analysis by one author (E.M.C), with 10% coded by a second author (B.K.). Intercoder reliability was considered excellent, at >95%.

3. Results

3.1. Understanding of Cyberbullying

To assess their understanding of cyberbullying behaviours, students were asked to define cyberbullying in their own words, with responses thematically coded at baseline and follow-up. The most common baseline responses related to the online nature of the actions, that it was mean, hurtful and/or harmful and that it involved bullying. At follow-up, definitions were generally consistent, with an emphasis on online and/or internet modes, and cyberbullying behaviours. There was increased awareness of the potential for anonymity, with comments about it “occurring from behind a screen”, and that it can constitute targeted harassment. Of note, participants were more likely at follow-up to use phrases consistent with the definition presented in the intervention. These included elements of aggressive behaviour, and harm being intentionally inflicted by one or more people through electronic means, as well as identifying the potential for cyberbullying to repeatedly, and from behind a screen, that is, more or less anonymously, in contrast to face-to-face bullying.

3.2. Attitudes towards Cyberbullying

Baseline levels of positive attitudes towards cyberbullying showed a mean score of 15.93 (SD = 5.69), whereas the control group showed more positive attitudes (M = 16.88, SD = 6.16). Based on ANOVA testing, immediately following the intervention, positive attitudes to cyberbullying were significantly lower, with a mean score of 14.72 (SD = 5.52), based on post hoc testing (p = 0.042). Favourable attitudes towards cyberbullying rebounded somewhat at follow-up (M = 15.01, SD = 5.34), but these changes were not significant (p = 0.578), indicating a lasting effect on attitudes towards cyberbullying.

3.3. Cyberbullying Behaviours

Students completed measures of cyberbullying behaviours, including victimization, perpetration, and witnessing (or bystanding), at baseline and follow-up. Due to significant negative skew, only binary measures were used for analyses. Overall, at baseline, witnessing was reported most frequently, with 71.5% of participants having observed some level of cyberbullying behaviours online, whilst 51.6% reported having been victims, and 22.1% had participated in cyberbullying. Control participants were more likely to have been victims (71.9%), and perpetrated cyberbullying (39.3%), whilst witnessing was similar (76.8%). At follow-up, there was a significant, but small improvement noted for intervention participants. For victimisation, 47.4% reporting experiences of cyberbullying victimization (χ = 10.33, p = 0.006, Cramer’s V = 0.14), while 18.2% reported perpetration (χ = 10.58, p = 0.005, Cramer’s V = 0.14), and only 52.6% reported having witnessed cyberbullying behaviours after attending the intervention (χ = 19.51, p < 0.001, Cramer’s V = 0.19).

3.4. Mental Health

The K-10 was used to assess student mental health at baseline and follow-up. The mean baseline score of 20.36 (SD = 9.23) is just within thresholds deemed to have “moderate distress”. Control students reported similar results, with a mean score of 19.82 (SD = 9.52). However, at follow-up, participant levels of psychological distress had reduced to 19.90 (SD = 8.64), which suggests an improvement in student mental health to “low distress” and increased wellbeing, although this change was not statistically significant: F(2, 513) = 0.178, p = 0.837.

3.5. Confidence in Responding

At baseline, students reported being relatively confident in responding to cyberbullying, with a mean score of 7.05 (SD = 2.08), which was comparable to control participants (M = 6.98, SD = 2.06). There were significant changes in confidence levels from this point F(3, 829) = 20.24, p < 0.001, with a small effect size (η² = 0.07). Post- intervention, confident was significantly improved, with a mean score of 8.26 (SD 2.01) as confirmed with post hoc testing (p < 0.001). At follow-up three months post- intervention, confidence had dropped back significantly (p = 0.002), but was still above baseline levels, with a mean score of 7.52 (SD = 2.01).

3.6. Qualitative Data Analysis

Participants also provided a range of qualitative data prior to intervention commencement and post intervention, as outlined below, enabling comparison of their anticipated responses.

3.7. Responses to Cyberbullying

Young people were asked about what actions they would take if experiencing cyberbullying themselves, with results provided in Figure 2. While more than seven in ten students reported that they would tell a parent or carer, and most would also block the person, concerningly 16 students (5%) were unsure, and 13 would take no action (4%). Immediately post- intervention, there were increases in most proactive strategies, including help-seeking, blocking the person, and going offline. There was a significant increase in willingness to contact the e-Safety Commissioner. Importantly, most students reported increased willingness to take protective actions, with only six students (out of 300) reporting being unsure, and only one planning on taking no action, at the conclusion of the intervention. There was also a reduction in students’ planning to “keep it to themselves”.

3.8. Intervention Feedback and Learnings

Following the intervention, young people were asked about their learnings with free text results. Responses were coded for themes, as presented in Figure 3. Most endorsed general information about cyberbullying, with one in four learning about the consequences for others, including legal and psychological impacts, and one in five noted improved learnings related to how to respond to cyberbullying if they see it online. Just over 15% specifically noted content, including legislation regarding sexting and nude images, which is consistent with these behaviours rated as of high concern at pre-intervention. Very few reported having gained no knowledge, and this typically related to a sense that they were already aware of information regarding cyberbullying from other sources.

3.9. Behavioural Changes

Students were asked immediately post-intervention about potential changes they intended to make in relation to their online behaviours personally, at home and at school. At follow-up, participants were asked to report on changes they had actually implemented since the intervention, with responses coded and presented in Figure 4, Figure 5 and Figure 6. Intended personal changes largely related to monitoring personal behaviour, and implementing time limits, but a significant minority (10.2%) reported being unsure as to any changes. At follow-up three months later, many had made no personal changes, emphasising the need for ongoing reinforcement of intervention ideas, and 8.2% remained unsure as to any changes. However, there were significant increases in being kind, cautious and safe online, and in ceasing to use problematic games or apps.

Intentions for changes at home typically related to an increased willingness to reach out for assistance from parents, and self-directed limitations on device usage and monitoring behaviour to be kinder. It is noteworthy that while close to 20% of participants intended to “tell parents”, none had actually reported doing so at follow-up. Again, several participants reported planning no specific changes, with typical comments reporting that they feel they already act consistently with the intervention recommendations. At follow-up, similar to above, almost half of students had made no changes, but there was increased self-monitoring, including being more aware of risks and safe behaviours, and an increased likelihood of reducing device usage through going offline or getting outside.

School-based intended changes were most commonly related to behavioural monitoring, including not joining in with cyberbullying behaviours when seen, and seeking assistances from teachers, adults, or friends. Unsurprisingly, given high levels of school-based policies around device usage, a large group planned no specific changes. At follow-up, similar to above, almost half of students had implemented no changes. However, for those who had taken action, this included limiting or reducing their time online and/or sites, and calling out behaviours.

Lastly, at follow-up, students were asked about their reflections on the intervention. Participants were asked to rate their confidence in a range of areas, as indicated in

Table 1. Follow-up intervention reflection confidence ratings.

Question	M	SD
The Cyber Cats program helped me understand cyberbullying	3.93	1.09
The Cyber Cats program has improved my confidence in responding to cyberbullying	3.81	1.10
The Cyber Cats program has motivated me to better respond to cyberbullying if I were to be cyberbullied	3.82	1.14
The Cyber Cats program has motivated me to better respond to cyberbullying if someone else were being cyberbullied	3.88	1.05

, from 1 (Strongly disagree) to 5 (Strongly agree).

Participants were positive about the impact of the intervention on their understanding and confidence in responding to cyberbullying, as well as their motivation to respond better both for themselves and for others.

4. Discussion

This paper presents the findings from the evaluation of Cyber Cats, a community-based intervention to prevent cyberbullying in young adolescents attending Year 7 high school. In order to maximise the intervention’s reliability and theoretical strengths, the current evaluation was based on principles as outlined by Volk et al. [23]. As recommended by Volk et al. [23], the Cyber Cats intervention implemented and commenced with a clear definition of cyberbullying. At baseline, students demonstrated a relatively sound understanding of how cyberbullying can be defined; however, immediately following the intervention, there was increased awareness regarding the issues of anonymity of bullying perpetrators, and that cyberbullying behaviours could be targeted and occur from “behind a screen”. Importantly, the intervention also seemed to engender greater consistency in definitions, which was sustained at three months follow-up, with participants were more likely at follow-up to use phrases consistent with the definition provided during the intervention. This included aggressive behaviour, harm being intentionally inflicted by one or more persons via electronic means, and identifying the potential for cyberbullying to occur repeatedly, and more anonymously, than face-to-face bullying.

Behavioural attitudes towards cyberbullying were moderate at baseline, suggesting that most students do not endorse cyberbullying in general. These attitudes towards cyberbullying became significantly less favourable when measured post-intervention, but rebounded slightly at follow-up, although not to pre- intervention levels. Notably, intervention participants held significantly less favourable attitudes at follow-up than control participants, suggesting a positive impact of the intervention which was retained over time.

When questioned about their actual engagement in cyberbullying behaviours, all students were more likely to report having witnessed cyberbullying, than to have been a victim of such behaviours. Cyberbullying perpetration was least common, although all three behaviours were reported at low frequencies. For Cyber Cats participants, binary indicators of having ever experienced cyberbullying victimisation, perpetration and witnessing were all significantly lower at follow-up than baseline. Of note, control participants were likely to report higher rates of all three behaviours, again supporting a positive impact of the intervention sustained at follow-up. Importantly, these benefits occurred despite intervention respondents having an improved understanding around the definition of, and associated behaviours of cyberbullying, which might be associated with increased reporting.

Importantly, mental health symptomatology reduced for participants from baseline to follow-up, with the average score moving from being diagnostically indicative of mild distress, to a level indicating that the average participant is “likely to be well”. While mental health conditions such as anxiety and depression were not an explicit focus of the intervention, it is considered likely that the emphasis on finding positive and health promoting activities may have supported an improvement in mental health symptoms for participants.

Confidence in responding to cyberbullying was moderately positive at baseline, but increased significantly after the intervention, and reduced somewhat at follow-up. This indicates the importance of reinforcement of intervention concepts, as emphasised in learning studies in cognitive psychology [28]. This could be achieved via an integration of the intervention learnings into additional activities, which would be critical to retain gains beyond immediate delivery.

When considering intervention learnings, participants reported an increased likelihood to actively seek out support and taking protective steps for their online safety following the intervention. This included help-seeking from parents, teachers, friends and others, blocking the individual, reporting unacceptable behaviours, and going offline to engage in other activities. It was noted that only six students (out of 300) were unsure as to what to do at the conclusion of the intervention, and only one participant who planned to take no future action. There was also a reduction in students planning to keep any incidents of cyberbullying to themselves. These findings indicate that students were able to take on learning around the importance of help-seeking, which may also have contributed to the positive trends on mental health indicators [29].

Student behavioural intentions at the end of the intervention largely related to monitoring personal behaviour, and implementing time limits, but a significant minority reported being unsure as to any changes. At follow-up three months later, many had made no personal changes, again highlighting the need for ongoing reinforcement of intervention concepts. However, there were significant increases in being kind, cautious and safe online, and in ceasing to use or removing problematic games or apps, which may again provide some explanation as to the improvements in mental health symptoms.

Overall, Cyber Cats participants were positive about the impact of the intervention, finding the day engaging and positive, and learning new information in a shared environment. Students positively endorsed statements in relation to the value of the intervention and enjoyed the interactive and positive nature of its delivery.

4.1. Limitations

Whilst this evaluation highlights intervention findings, some important limitations should be noted. Firstly, the sample size for this study was relatively small and limited to Year 7 students attending participating schools in the regional centre in which this program runs, which limits generalisability of the results. Secondly, all responses relate to self-reported behaviours and attitudes, and in the interests of brevity, no survey response measures such as truthfulness indicators were included. Thirdly, whilst efforts were made to follow up with all participants, we were not able to obtain full data sets across baseline, post- intervention, and particularly at the follow-up time point, which impacts the statistical power of findings. In addition, to protect student confidentiality, data were collected by the program providers and no individual coding of participant responses was included to enable matching across timepoints. As such, we were unable to use repeated-measures analyses for longitudinal analysis, and have relied on cohort findings based on independent groups tests for these analyses. However, such analyses have lower power and are more conservative, hence reducing the risk of Type 1 errors. Lastly, only one follow-up survey was administered, at 8–12 weeks, and as such, longer-term impacts of the intervention cannot be addressed in this study.

4.2. Implications

Whilst participants demonstrated strong gains immediately post intervention, retention of learning at follow-up was reduced. Reinforcement of content in the school setting is critical to ensure that students have the opportunity to refresh and integrate learnings into their regular settings, including school and home. Additionally, engagement of parents, whilst challenging, might be critical to the success of social and emotional learning intervention such as this.

5. Conclusions

Cyber Cats is an evidence-based, single-day education intervention, focusing on cyberbullying and online behaviours for young adolescents. This intervention has demonstrated benefits in relation to attitudes and behaviours, and further investment to continue intervention delivery, and reinforce key content, is critical to maintain this success.