Evaluating the Effects of Gamification in Behavioural Change: A Proposed SEM-Based Approach

Abstract

The purpose of this study is two-fold. Firstly, it aims to investigate the available papers on the effect of gamification elements to explain behavioural changes through a Systematic Literature Review (SLR). Secondly, based on the SLR, it proposes a four-step SEM (Structural Equation Model)-based approach that can be used to validate the effects of gamification on behavioural change and can be further applied in the context of a research project that aims to lower maritime plastic pollution in coastal areas. The SLR approach provides an overview of empirical studies that successfully measure the three identified objectives, i.e., increased (O1) usage of a web platform, (O2) awareness, and (O3) participation in behaviour, and it focuses on SEM to collect empirical results. Findings from the SLR highlight multiple research shortcomings, such as the lack of a unified taxonomy for gamification and motivational affordances, the absence of studies soundly linking gamification elements to psychological outcomes, and the tendency of researchers to measure the intention to conduct a behaviour rather than the long-term effect of actual behaviour changes. Finally, the created approach provides insights on which gamification elements to include and how to measure their behavioural effect based on a self-developed SEM and questionnaire, which can be applied in research projects utilising gamification, independent from the domain of activity.

1. Introduction

According to the Heinrich Böll Foundation, research in 2019 showed that 11.500 tonnes of plastic has ended up in the Greek seas. This is the equivalent to throwing 1,000,000 plastic bottles every day into the Greek sea for a whole year. In addition, marine litter collection stations have measured an increase of up to 85% in plastic pollution during the tourist season [1]. Plastic pollution is not only unattractive to Greece’s tourism, but it also poses a far greater health threat. Exposed to saltwater and sunlight, some types of plastic may break down into non-recoverable fragments within months, according to research from the Archipelagos Institute of Marine Conservation. In water, plastics release toxic chemicals, which affects the growth and reproduction of the already fragile biodiversity. Bigger particles cause animals to develop internal entangles or suffocate. Smaller parts, however, also referred to as micro or nano plastics, are ingested. The institute has also alarmed citizens and authorities that microplastics have been identified in all commercial species of fish and also in dolphins, turtles, Mediterranean seals, and sea birds, which have become deceased due to plastic intake [2]. According to a study conducted in 2019, 44.3% of fish in the Aegean Sea contain microplastics [3]. Another study found that harvested Mediterranean Sea salt contains measurable particles of plastic [4]. Through the food chain, these small fragments are eventually consumed by humans as well. The Heinrich Böll Foundation of Thessaloniki has stated that the intake of contaminated sea food can cause hormonal chaos in humans, as micro plastic can mimic hormones [5]. Furthermore, because microplastics can have carcinogenic properties, the consumption of contaminated food increases the risk for breast and testicular cancer. To prevent further devastation and to reduce the impact on the environment and humans, awareness and the behaviour towards this issue need to be enhanced.

That is exactly where the research presented in this paper sets its visions, ambitions, and aspirations. GOLDEN SEAL (www.goldenseal.gr, accessed on 16 February 2023) is a research project aiming to contribute to the protection of the national tourist product through the upgrading of technological solutions available to managing authorities. In doing so, it demonstrates a set of innovative solutions and applications able to detect and monitor debris in real time, and it acts as an indicator for evaluating marine littering performance, with the aim of ultimately preventing waste from entering the marine environment. The main innovation of what is proposed lies in the conception, design, and development of innovative virtual competition, based on the daily results, metrics, and benchmarking of a disruptive Beach Quality Monitoring System. The proposed system is supported by an integrated cloud platform tightly integrated with a Wireless Sensor Network (WSN) and a mobile app, and it utilises drones and autonomous station pads for the launching and landing of drones [6]. Figure 1 presents the holistic solution that GOLDENSEAL is currently working on.

Moreover, GOLDENSEAL plans to tackle the challenge by creating a platform that contributes to behaviour changes in stakeholders through the implementation of persuasive technologies. An increasing number of studies in the last decade have shown that persuasive technologies, such as gamification and serious games, can influence and change a user’s behaviour if applied appropriately. The platform plays a central role in these efforts because it communicates (via the mobile app) all critical information to stakeholders about objectives, tasks, behaviours, and other topics. As a result, the first objective of the study presented in this paper is to evaluate whether the implementation of serious games and gamification increase stakeholders’ platform usage (O1). Scholars have indicated that people who have a high level of environmental awareness are more likely to participate in environmental activities; therefore, the influence on awareness is of interest as well [7]. Thus, the second objective is to evaluate whether the implementation of serious games and gamification increase stakeholders’ awareness towards environmental issues and solutions (O2). Finally, other scholars have emphasised that simply changing one’s awareness does not ensure a change in behaviour. Their findings concur that other components may be required for pro-environmental behaviour to occur [8]. Consequently, the third objective is to evaluate whether the implementation of serious games and gamification increase stakeholders’ participation in pro-environmental behaviour (O3). The research gap this paper aims to address can be summarised by the integration of the three core research questions formulated in this paper into the following: ‘This paper introduces the implementation of serious games and gamification to increase users’ awareness, participative behaviour, and usage of a technology platform, aiming to reduce plastic waste in beach areas’. To address this gap, in this paper a three-step methodological approach is developed based on the synthesis of SLR findings. In the first part, an SEM approach is prescribed, such that—when applied—it allows one to measure the effect on the defined objectives, O1–O3. In the second step, the identified gamification elements, i.e., progress bars, levels, points, stats, acknowledgements, choices, leader boards, cooperation, puzzles, quizzes, objectives, feedback, immersive stories, and narratives, are described. Based on SLR findings, important design choices and potential errors that could have a negative effect are stated. Finally, the third step includes guidelines for the evaluation of the collected data through the proposed SEM approach in the context of a research project.

This paper is structured as follows: Section 1 introduces the reader to the research objectives and presents the underlying challenge of missing awareness and environmentally friendly behaviour. From this challenge, three objectives (O1–O3) are derived. Section 2 explains persuasive technologies, such as serious games and gamification. It also presents an overview of gamification elements, which are used in studies to explain the effectiveness of these elements on users’ behaviour. Section 3 describes the conducted Systematic Literature Review and elaborates the chosen research design. It also explains how the objectives (O1–O3) are transformed into research questions (RQ1–RQ3). After all relevant papers are collected and summarised, Section 4 develops the mentioned guidelines for the GOLDENSEAL project. It includes a customised conceptual model, a list of recommended gamification elements, a tool to measure the effectiveness of the implementation, and an analysis method for evaluating the results. Section 5 summarises the answers to the research questions (RQ1–RQ3), lists notable limitations, and finishes with outlooks and open questions. Section 6 presents the conclusions of the paper.

2. Background

Persuasive technologies have been defined as interactive systems that aim at forming, altering, or reinforcing a user’s attitude or behaviour change. The persuasion of a user is achieved through the development of elements, which can influence a person’s beliefs, attitudes, or motivation. It can also do so by creating a social environment, which can influence these factors. For persuasive technology to be successful, the intent must be clearly defined and incorporated in its design process [9]. The popularity and usage of these technologies is rapidly growing and has already been implemented in various domains, such as education, customer behaviour, sustainability, healthcare, physical education, and many more. Persuasive technologies can take on many forms, from smart toothbrushes to complex military/war simulations, and thus, they are used as an umbrella term comprising serious games and gamification [10,11].

2.1. Gamification and Serious Games

The most common definition of serious games is games that do not have entertainment as their primary purpose [12]. Others add that they are designed as persuasive technologies to change the user’s attitude or behaviour [13,14]. Gamification, on the other hand, is generally defined as the application of game-design elements and game principles in a non-game context, thus enhancing a system by creating benefits similar to those of serious games [15]. The difference between serious games and gamification is that the latter one involves only the extraction of game-design elements and applies them to a non-game process, whereas serious games include all elements [11]. Equipping an application with game-design elements, such as points, badges, or progress bars, can increase transparency and competition in a task, thus resulting in benefits similar to those of serious games.

There have been some attempts to structure gamification elements through a taxonomy, which hierarchically classifies elements and organises them into groups. Some scholars refer to them as game-design elements, and others refer to them as gamification elements, which is why both words are used interchangeably. During the literature review, various frameworks were identified, which use different taxonomies and naming conventions for gamification elements. This creates a challenge for the development and understanding of gamified platforms, as there is currently no uniform framework. Toda et al. [16] tackled this challenge with the aim of creating a new standard to serve the Human–Computer Interaction (HCI) community by comparing available frameworks from the works of Werbach and Hunter [17], Marczewski [18], Klock et al. [19], and Toda et al. [20]. Although this model is relatively new, it is considered reliable because it compares other available models and has been validated through gamification experts. Although the framework was created for educational systems, the taxonomy can be used in the GOLDENSEAL project for guidance, as gamification elements can be used independently from the domain.

2.2. Motivation

As motivational types play a crucial role in driving the user towards a behaviour, they are introduced in the following text. Subsequently, motivational affordances are introduced, which explain the perceived effect of gamification elements on the users’ motivation. Gibson claims to have invented the term affordances by deriving it from the verb afford [21]. He refers to it as a property that is offered to a user when using an object. For example, a chair can be used for resting by sitting on it or for elevating oneself by stepping on it. Zhang added to his definition that the property of the object allows a user to satisfy certain needs. Therefore, a motivational affordance is the motivational characteristic that arises through the interaction of a user and an object [22]).

Scholars have also added that motivational affordances should not be seen as equivalent to gamification elements [22]. At this point, it must be stated that many scholars such as Hamari et al. do not distinguish gamification elements from affordances and even mix the wording at times [23]. This may be due to a missing standardised convention. Thus, terms are mistakenly confused with one another or used as synonyms. In addition to the lack of understanding both terms, Deterding et al. described in their theory of situated motivational affordances that not all gamification elements cause the user to perceive the same motivational affordance [24]. Rather, the outcome depends on the chosen implementation style of the element as well as the situation of the user, which may alter the perception of using an element. The chair example illustrates this on a basic level. Depending on the user, a chair may be conceived as useful depending on whether a user is determined to rest on the chair or plans to change a lightbulb. Figure 2 illustrates the described relation on a case that applies to the GOLDENSEAL project. Here, beach owners, i.e., situated users of the application, can aim to increase their customer base by utilising a rating function as a motivational affordance by specifically increasing the visibility of the beach through the opportunity of competing with other beaches.

As the concept of motivational affordances states, the interaction of users with gamification elements, e.g., ranking, as seen in Figure 2, can motivate users, as the interaction causes the perception of a motivational affordance. Because most psychological models are based on motivational theories, it is deduced from this that gamification elements may impact users’ motivation and thus their behaviour. However, many scholars have noted that there are still not enough studies to make a generalisation regarding which gamification elements can be applied to a planned psychological reaction [22,23].

As there is currently no standard for a taxonomy on motivational affordances, research has been conducted to present an appropriate model for this project. Tondello et al. conducted a review on six available taxonomies for motivational affordances and were able to complement them through a comparison [25]. Each dimension consists of multiple perceived motivational affordances. In addition to building a taxonomy of motivational affordances, the authors went a step further by categorising the twelve dimensions into three motivational types, i.e., intrinsic, extrinsic, and context-dependent. Context-dependent dimensions can be perceived either as intrinsic or extrinsic motivational outcomes, depending on contextual factors such as the user’s personal perception or the type of task at hand. This information is very useful, as it creates a bridge to analyse the effects through psychological models. For more details, readers can refer to [25]. After having roughly illustrated the theoretical basics, the following section guides the reader through the process of conducting our Systematic Literature Review (SLR), aimed at addressing the research questions and the aforementioned research objectives.

3. Literature Review

Fink (2005), describes literature reviews as “[…] a systematic, explicit, and reproducible method for identifying, evaluating, and synthesising the existing body of completed and recorded work produced by researchers, scholars, and practitioners“ [26] (p. 3). Figure 3 shows the seven-step process, which makes the procedure systematic and repeatable.

3.1. Formulation of Research Questions

As stated in the introduction, the task is to conduct research on studies that explain whether and how persuasive technologies, namely serious games and gamification, can accomplish the defined objectives (O1–O3). Thus, a broad research question (RQ0) is formulated as follows:

RQ0: How can GOLDENSEAL influence behavioural change in its users through serious games and gamification?

In the following, the research question is broken down into sub-questions that meet the described objectives (O1–O3), as mentioned in the introduction.

O1 is defined as implementations that increase the stakeholders’ usage of the platform. Thus, RQ1 is derived:

‘RQ1: How can the implementation of serious games and gamification increase the usage of a platform?’

O2 is defined as implementations that increase the stakeholders’ awareness towards environmental issues and solutions. The environmental focus is removed in order to include studies from additional domains. Thus, RQ2 is derived:

‘RQ2: How can the implementation of serious games and gamification increase users’ awareness?’

O3 is defined as implementations that increase the stakeholders’ participation in pro-environmental behaviour. The environmental focus is removed in order to include studies from additional domains. Thus, RQ3 is derived:

‘RQ3: How can the implementation of serious games and gamification increase participative behaviour?’

Additionally, RQ4 summarises the results from RQ1–RQ3 and transfers them to the GOLDEN SEAL project by applying them to the domain of environmental sustainability: RQ4: How can this information be transferred to the GOLDEN SEAL project?

3.2. Review Design

For a successful and comprehensive Systematic Literature Review (SLR), it is important to determine, at the outset, which databases are used to retrieve academic articles and research papers. In the context of this work, Elsevier’s Scopus is chosen. To discover literature that supports RQ1–RQ3, a search string of combined keywords must be generated using Boolean operators.

The initial search string is defined as TITLE-ABS-KEY (“Seri* games” OR gamif* AND “behav* change*”). The asterisk symbol functions as a truncation operator. Any word starting with the characters preceding the * symbol will be identified as a match.

This query offered important information regarding fundamental knowledge and idea generation. However, it also led to some challenges. Firstly, the retrieved papers use different methodological approaches. Although some studies include practical tests, others rely exclusively on theoretical knowledge, making them impossible to compare on an equal basis. As a result, a critical choice was made to limit the research to studies that utilise Structural Equation Modelling (SEM). As a consequence, the final search string yielded 154 documents and was as follows:

TITLE-ABS-KEY (“Seri* gam*” OR gamif* AND “structural equation model*”)

To reduce the amount of time spent searching for papers using high-quality criteria, inclusion and exclusion criteria were defined according to the first screening, which Fink refers to as the practical screening. This first filter guides the research in collecting a gross number of papers [26] (pp. 50–53). Papers were filtered for the defined inclusion criteria (InC) through the search function:

−

InC1: The selected language is English.

−

InC2: All document types are included.

−

InC3: The timeframe for selecting the literature ranged from 1 July 2022 to 21 August 2022.

−

InC4: The publication years were chosen based on the conducted research, which showed the number of publications on behaviour change games from 2010 to 2018. The number of relevant publications peaked after 2012. Therefore, the timeframe was set from 2012 until today [11].

The following exclusion criteria (ExC) are content-related. Papers that did not meet the requirements below were excluded:

−

ExC1: Research that uses heavy financial incentives as an external motivator.

−

ExC2: Research that uses coercion, as GOLDEN SEAL cannot create circumstances to force its stakeholders and cannot rely on legislative regulations through the state.

−

ExC3: Research that does not measure a significant influence of gamification on the objectives.

−

ExC4: Research that does not use empirical data.

After having filtered papers through the practical screening, we proceeded to the second screening, excluding all studies that do not employ Structural Equation Modelling (SEM). SEM is a multivariate approach for testing and evaluating multivariate causal relationships in scientific research. Hereby, it allows one to link gamification elements to psychological outcomes. The model ensures internal validity, as it provides a clear cause-and-effect diagram. As the studies use various individually created conceptual models, a conceptual framework is used that allows one to compare studies with each other. The chosen framework is illustrated in Figure 4 and was created in accordance with the Stimulus–Organism–Response theory (SOR). The theory’s basic premise is a three-step process. The environmental stimuli, i.e., gamification elements, trigger an emotional state, i.e., motivational affordances or psychological elements. The emotional state then leads to a behavioural response that should be in accordance with the defined objectives (O1–O3).

Limiting papers only to SEM, as expected, reduces the retrieved papers significantly. However, it also creates a clear framework, which can produce transparent and measurable correlations, which are of high quality and are applicable to the research project. The research indicates any limitations that could threaten the validity of the proposed guidelines.

The following additional exclusion criteria were defined.

−

ExC5: Research that does not provide correlations of gamification elements with constructs (Stimulus).

−

ExC6: Research that does not use psychological models. The research must have an underlying psychological model that explains a correlation of gamification elements with behavioural outcomes (Organism).

−

ExC7: General papers about psychological models or gamification elements and serious games. The research questions need to match RQ1–3 (Response).

The final search string yielded 154 results. Using the first screening, known as the practical screening, decreased the number of papers to 153. The methodical screening was divided into two tasks. Firstly, exclusion criteria 7 ignored most of the papers and yielded 35 papers that support RQ1–RQ3. Secondly, the remaining exclusion criteria reduced the number to 10 papers. Figure 5 illustrates the process of the literature search.

Sources that passed both screenings were used as references to support RQ1–RQ3. Relevant papers were tabulated and classified in

Table 1. Referenced papers to support the RQs.

Author (Year)	Data and Sample	Analysis Method	Stimulus Constructs	Gamification Elements	Underlying Theory	Organism Constructs	Key Findings	RQ	Domain
[27]	Gamified app, questionnaire (N = 276)	Partial least-squares analysis (PLS)	Achievement, social, immersion	Badges, medals, points, leader boards, rankings, progress bar, cooperation, competitions, public rankings, social networking, chats, narratives, meaningful stories	Self-Determination Theory (SDT)	Autonomy, competence, connection or relatedness	SDT is well suited for SEM. Most hypotheses are supported. Immersive-related correlations are not fully supported. Gamification elements show an effect on RQ1.	RQ1	Healthcare and well-being
[28]	Gamified app, questionnaire (N = 898)	Partial least-squares analysis (PLS)	Achievement	Points, badges, leader boards	TAM or extended framework	Perceived enjoyment, perceived usefulness, perceived ease of use social gain	Performance constructs were perceived as useful but did not support RQ1.	RQ1	Marketing and branding
[29]	Gamified app, questionnaire (N = 307)	Partial least-squares analysis (PLS)	Autonomy support, achievement, competition, social	Choices, badges, points, points, progress bars, rankings	TAM or extended framework	Perceived Usefulness, perceived enjoyment, social gain	Excessive competition and comparison with other users decrease RQ1 as the experience is less enjoyable. Other hypotheses are supported.	RQ1	Environment and sustainability
[30]	Gamified app, questionnaire (N = 332)	Analysis of variance (ANOVA) and covariance-based analysis	Avatars, embodied feedback, status feedback	Avatars, feedback, narrative	Self-Determination theory (SDT), TAM, or extended framework	Perceived identification, perceived usefulness, competence, relatedness, autonomy	RQ1 is predominantly caused by perceived competence.	RQ1	Education and learning
[31]	Gamified app, questionnaire (N = 206)	Partial least-squares analysis (PLS)	Achievement, social, immersion	Badges, leader boards, progress bars, points, competition, collaboration, chat, avatars, profiles, narratives, storytelling, personalisation, customisation	Motivation and Expectation Confirmation Theory (ECT)	Motivation, perceived satisfaction, confirmation, perceived usefulness	Once again, immersion-related elements do not show a significant correlation, which may be due to the chosen item design and domain.	RQ1	Education and learning
[32]	Gamified app, questionnaire (N = 367)	Partial least-squares analysis (PLS)	Achievement, social, immersion	Avatars, free choices, storytelling, narratives, badges, trophies, leader boards, cooperation, likes, chats	Self-Determination theory (SDT)	Self-benefits Social benefits	SDT is well suited for measuring gamification’s impact on RQ1. Immersive-related elements do not show a significant correlation.	RQ1	Environment and sustainability
[33]	Gamified app, questionnaire (N = 301)	Partial least-squares analysis (PLS)	Competitiveness, challenge, enjoyment	Feedback, quizzes, ranking, points, progress bars, feedback, limited time	TAM or extended framework	Perceived usefulness, perceived satisfaction, performance expectancy	Excessive competition lowers RQ1. Other hypotheses are supported.	RQ1	Education and learning
[34]	Gamified app, questionnaire (N = 484)	Partial least-squares analysis (PLS)	Immersion, achievement, social	Avatars, profile, personalisation, narrative, points, leader boards, random prizes, badges, virtual currencies, levels, progress bars, tasks, limited time, missions, competition, feedback	Brand Engagement Model (BEM)	Behavioural dimension, cognitive dimension, social dimension	The social dimension does not influence brand awareness. The implemented immersion-related elements do not create a significant emotional bond with the brand. Other elements positively correlate with RQ2.	RQ2	Marketing and branding
[35]	Gamified website, questionnaire (N = 537)	Confirmatory factor analysis (CFA)	Points, self-expression, competition	Points, free choices, competition	Self-Determination theory (SDT), self-efficacy theory, self-presentation theory, flow-theory	Relatedness, competence, autonomy, self-presentation, self-efficacy, flow-theory	All relations are supported. Awareness is assessed as a moderator. Intrinsic motivation induced by gamification is stronger if users are more aware.	RQ3	Environment and sustainability
[36]	Gamified website, questionnaire (N = 450)	Seemingly unrelated regression analysis (SUR)	Visual feedback, numerical feedback, combined visual and numerical feedback	Feedback, narrative	Constructional Level Theory (CLT), TAM, or extended framework	Perceived satisfaction, perceived usefulness	Feedback reduces psychological distance, thus increasing RQ3. Visual feedback correlates with perceived satisfaction, and numerical feedback correlates with perceived usefulness.	RQ3	Environment and sustainability

.

3.3. Review Results

Referring to stimulus constructs, Figure 6 shows the distribution of the analysed gamification elements of these studies. Although the applications include further elements, these numbers only denote the actual elements, which were analysed. Among the 21 gamification elements that Toda et al. identified, only 14 elements were analysed [16]. This does not mean that missing gamification elements are not being used in actual applications; however, it denotes the lack of scientific papers with an underlying SOR-SEM that analyses these elements. As shown, G12: Competition and G3: Points were used the most.

In terms of gamification dimensions, the most frequently used dimension was D1: Performance, followed by D3: Social. According to the gamification dimensions defined in [16], Figure 7 illustrates the occurrence of gamification elements among them. It shows that all dimensions (D1–D5) are present in the referenced papers.

Referring to the underlying theories of organism constructs, the most used theories are based on the extended Technology Acceptance Model framework, which includes TAM, UTAUT1, and UTAUT2. The second most used theory is Self-Determination Theory (SDT). Other theories are used only once. The occurrences are illustrated in Figure 8.

Referring to response constructs, Figure 9 shows the distribution of relevant papers that answer RQ1–RQ3. Although most studies support RQ1, only few studies on RQ2 and RQ3 were found.

An evaluation of the studies’ conceptual framework shows that seven out of ten studies use partial least-squares (PLS) analysis. Analysis of Variance (ANOVA), covariance-based analysis, Seemingly Unrelated Regression Analysis (SUR), and confirmatory factor analysis (CFA) are used only once. The distribution is shown in Figure 10.

The following subsections discusses how the retrieved papers support RQ1–RQ3. It should be noted that the wording of some stimulus and organism constructs is altered to match the taxonomy of Toda et al. [16] and the presented psychological models after equality is determined.

3.3.1. Supporting RQ1—Increase the Usage of a Platform

Games are designed for entertainment purposes and motivate their users to engage with them intensively. The assumption that gamification elements can make an application more engaging is therefore to be proven. This section presents studies that have found evidence for increased usage based on psychological models. Bitrián et al. dealt with an increase in user engagement in mobile gamified apps through the implementation of gamification elements [27]. The researchers analysed the effect of gamification categories on three basic psychological needs, i.e., competence, autonomy, and relatedness, which are part of the Self-Determination Theory (SDT). Their survey included 276 users of the “Fitbit” app, which is an exercise app and falls in the domain of physical education. The results were analysed through partial least-squares analysis (PLS). The theoretical foundations of their work were structured and well founded, but not all influences of gamification categories on psychological needs could be proven. The results showed that only achievement and progression elements have an impact on all three needs basic needs, i.e., autonomy, competence, and relatedness. The influence of social and immersion elements, on the other hand, can only be measured based on the need of connection or relatedness. Furthermore, all three basic needs showed significant relationships with user engagement, which, again, showed significance with continued use intention, word of mouth intention, and an increase in the app rating. The results and influences are summarised in Figure 11, which was created in reference to their work. Notable limitations include short-term data, as longer exposure to gamification elements may lead to stronger outcomes. The study also included control variables such as gender, age, experience, and usage, which did not show any mediating effect. It should be noted that a test on a control group with a non-gamified application was not conducted.

In their paper, Zhou et al. analysed continued use intention with the application “Ant Forest”, which has already been utilised in this study [32]. The gamified application plants trees based on the amount of carbon dioxide emissions that users avoid. Their research uses the Organismic Integration Theory (OIT), which is a mini-theory of the Self-Determination Theory (SDT) used to create a Structural Equation Model. The survey included 367 participants from China and was studied through a PLS analysis. The authors used immersive-, achievement-, and social-related elements as stimuli and revealed that perceived self-benefits and social benefits mediate the relationships between stimuli and the intention to continue. Although the relation between immersive-related interactions and self-benefits was denied, all other relationships showed significance. The simplicity of the app design and missing vivid 3D game animations were named as possible explanations for the missing correlation. All correlations are summarised in Figure 12. A notable limitation is the lack of cultural diversity in the participants. All collected questionnaires were solely from China, which may have limited the reference value for gamification design in other cultures. Control variables included tenure, tree numbers, gender, and age. Although tenure showed a small correlation, it can be considered not significant. It should be noted that a test on a control group with a non-gamified application was not conducted.

In their paper, Rohan et al. investigated the effect of gamification on learners with the aim of increasing their continued intention to use open online courses [31]. The proposed SEM was based on measuring motivation and other constructs from the Expectation Confirmation Theory (ECT). Data were collected from 206 university students and were analysed through a confirmatory factor analysis. The authors used achievement-, social-, and immersion-related elements as stimuli. The results showed that motivation, satisfaction, and perceived usefulness mediate the relationship between stimuli and continuous intention; however, motivation had the least direct impact. Significant effects on satisfaction were measured through motivation, confirmation, and perceived usefulness. Among the three gamification categories, achievement had the highest impact, and social elements had a low influence. Immersion-related elements did not show any significance. Although the paper does not mention it, this could have been caused by the item design of the motivation construct, which has the word “learning” in each item, instead of using general motivation. Correlations are illustrated Figure 13. Notable limitations include the motivation construct being designed as single-dimension construct. However, the study would have contributed to a deeper understanding if it distinguished between intrinsic and extrinsic motivation. Furthermore, it could have analysed intrinsic motivation through the three psychological needs: autonomy, competence, and relatedness. Next, the moderating effect through control variables, such as user characteristics, were not measured. It should be noted that a test on a control group with a non-gamified website was not conducted.

In their study, Jahn et al. analysed the effect of gamification elements on reuse intention based on an application that was designed to teach its users to pick low-caloric food over high-caloric food [30]. The designed SEM uses the theory of psychological needs, which is part of the Self-Determination Theory (SDT) to explain the correlations. Additionally, it uses constructs that are similar to the ones in the Technology Acceptance Model (TAM), although the paper does not specifically refer to it. The survey included 332 German-speaking participants and was partially analysed through an analysis of variance (ANOVA) as well as a covariance-based analysis.

The authors used avatars, embodied feedback, and status feedback as stimuli and revealed that the three psychological needs, i.e., autonomy, relatedness, and competence, as well as avatars, mediate the effect on continued use intention. Interestingly, there was a negative link between autonomy for decision freedom and reuse intention. The influence was primarily due to gains in perceived competence. One possible explanation is that, when individuals believed that they could choose what to do during the activity, they were less likely to reuse the system. Correlations are illustrated in Figure 14. Surprisingly, the results revealed that avatar diagnosticity was higher in the control group than it was in the majority of the other groups. This conclusion could be due to the uncanny valley effect, which outlines how a non-human entity may be accepted favourably if it is seen as human, but when it achieves a certain degree of humanness that is not yet flawless, acceptance drops significantly. Control variables were measured through a questionnaire and included age, gender, body mass index, and task performance. It should be noted that a test on a control group with a non-gamified website was not conducted. As mentioned, the study used a control group.

Similar to Zhou et al. [32], Du et al. [29] analysed continued use intention on an application called “Ant Forest”, a gamified application created to contribute to environmental conservation. Although the paper refers to the goal-framing theory, the chosen construct and formulated item are the same as those for the UTAUT2 framework. A survey included 307 participants from China and was studied through a PLS analysis. The findings confirmed that users’ Perceived Usefulness and Perceived Enjoyment were the main mediators for influencing their intention to continue using the “Ant Forest”. The authors used autonomy support, visibility of achievement, competition, and interactivity as stimuli. The research found that autonomy support and interactivity had an especially significant influence on enjoyment. This paper once more shows that, although competition and interactivity lead to social gain, excessive competition can decrease the intention to use the application. In addition, the paper discloses that users’ perceived social gains influence their intention to continue fully through the mediator of enjoyment. It further implies that achievements should not be used to compare users’ performance but rather to provide them with feedback on how well they are doing. This is indicated by the significant effect on perceived usefulness. The effect of having freedom of choices also shows great significance on perceived usefulness. Detailed correlations are summarised in Figure 15. Notable limitations are the different cultural backgrounds due to geographic locations in China, and the planted trees may represent some sort of financial incentive that cannot be guaranteed through GOLDENSEAL. Control variables were considered and included age, gender, education, income, body mass index, and frequency of use. They had a non-significant impact. It should be noted that a test on a control group with a non-gamified website was not conducted.

Raman analysed the effect of gamification on young female consumers to continue using an e-commerce website, with the aim of ultimately making a purchase [28]. Although this paper refers to using a Technology Acceptance Model (TAM), the chosen constructs undeniably represent the UTAUT2 framework. To validate the model, the researcher surveyed 898 female participants from India and used a partial least-squares (PLS) structural analysis. The authors used performance-related gamification elements as stimuli, namely points, achievements, and badges, which exhibited a favourable influence on the continued use intention of young female consumers. The results show that social interactions, perceived ease of use, and perceived enjoyment acted as significant mediators. Although gamification elements showed a significant influence on Perceived Usefulness, it did not show any correlation with an increase in use intention. Detailed correlations are summarised in Figure 16. The stimulus construct was designed as the performance category of gamification elements, and it contained badges, points, and leader boards. If the stimulus constructs were separated into individual gamification elements, which elements impact the constructs of UTAUT2 would be more evident. The study also has a geographic limitation, as the results may have differed when applied to a cultural area other than India. Next, it used only data of participants who indicated that they had prior experience with gamification elements, which could have influenced the results, assuming that these participants tended to perceive gamification-induced effects as stronger. The paper further noted that female consumers are more prone to social influences, which is in line with other research [23]. Thus, results may have differed if male users were included in the survey. It is noted that the questionnaire contained a section for control variables, including questions regarding age, education, income, gaming experience, frequency of use, and interest in reviews. They had a non-significant impact. It should be noted that a test on a control group with a non-gamified website was not conducted.

Finally, Wirani et al. studied continued use intention with “Kahoot!”, a game-based learning platform used as educational technology, in an Indonesian university [33]. It uses constructs of the UTAUT2 framework. The survey included 301 Indonesian students and was analysed using PLS. Generally, the findings indicated that the students desired to continue using “Kahoot!” for academic purposes. Students benefitted from improved performance and gained knowledge. The authors used competitiveness, enjoyment, and challenge as stimuli. Their empirical findings show that perceived usefulness, perceived satisfaction, and individual impact or performance expectancy mediated the effects on continued use intention. The paper notes the importance of setting an appropriate time limit, as pressure and failure could have a negative impact on the defined organism constructs. Furthermore, it should be highlighted that competitiveness was not proven to affect the satisfaction mediator. A possible explanation is that a student’s low rank in comparison to fellow students impacted satisfaction negatively. Results are summarised in Figure 17. Notable limitations are the different cultural background due to geographic locations and the small amount of tested gamification elements. The study mentions that the first provided questionnaire includes variables on the demographics of the respondents, such as gender, age range, education level, duration, and frequency of application usage. They had a non-significant impact. It should be noted that a test on a control group with a non-gamified website was not conducted.

3.3.2. Supporting RQ2—Increase Users’ Awareness

This section summarises papers that found correlations of gamification elements that increase the awareness of users through gamification elements. Only one paper was found to pass both screenings, that of Permana et al., who examined the effect of gamification on brand awareness in online marketplaces [34]. Correlations were explained using a Brand Engagement Model (BEM), which mediates relationships between users and brands, including transactions and marketing activities. The survey involved 484 participants from Indonesia and was analysed using partial least-squares Structural Equation Modelling (PLS-SEM). The research showed that not all gamification elements successfully affected brand engagement and contributed to awareness. Immersion features influenced the behavioural, cognitive, and social dimensions. Achievement features influenced all brand engagement dimensions. Social features only affected the social and cognitive dimensions. Finally, the cognitive and social dimensions were not found to influence brand awareness. The application of a domain-foreign model from the field of marketing offered a new perspective and yielded interesting results that can be applied in environmental sustainability projects, such as the GOLDENSEAL project. Figure 18 summarises the found correlations.

Notable limitations are the different cultural backgrounds due to geographic location. As the study combines different gamification elements into four categories, the effect of a single gamification element is difficult to trace. Although achievement-related elements had a strong and positive influence on organism constructs, it was derived that an imposed countdown also contributed to these correlations. The results should therefore be interpreted with care, and the detailed analysis of the study reveals that, for example, a countdown did not have as much of an impact as the other elements did. Thus, for future studies, it is recommended to combine only elements that have very similar traits. The study mentions having used demographic control variables in the questionnaire. They had a non-significant impact. A control group with a non-gamified website was not mentioned.

3.3.3. Supporting RQ3—Increase Participative Behaviour

This section summarises papers that found correlations of gamification elements that encourage performing a desired behaviour outside of a platform. The actual behaviour is hereby not measured; however, the intention to engage in a behaviour is measured. The first of two papers in this section is that of Hsu [35]. The author explored how a gamified website influences intrinsic motivations and satisfaction of needs with the aim of increasing their behavioural intention of engaging in recycling (see Figure 19). In addition, the author investigated how environmental awareness moderates the relationships between intrinsic motivations and user engagement. The paper draws on Self-Determination Theory and combines it with constructs from other theories. The questionnaire included 537 participants and was analysed through a confirmatory factor analysis (CFA). The author used points, self-expression, and competition as stimuli and revealed that the three basic psychological needs, i.e., competence, autonomy, and relatedness, mediate the effects. Additionally, the author included self-efficacy, self-presentation, and flow as additional mediators, which showed a significant mediating effect on the behavioural intention of users to engage in recycling. Environmental concern proved to be a mediator. Users with high environmental concern, in contrast to ones with low concern, showed stronger intrinsic motivation induced through gamification.

Because the results were based on only one website and one country, they cannot be generalised to other countries. The study suggests that additional underlying variables, such as pro-environmental self-identity, personality characteristics, environmental knowledge, and economic motivation, have yet to be investigated. The study mentions mitigating the risk of bias by collecting demographic variables such as gender, age, and education level. They had a non-significant impact. It should be noted that a test on a control group with a non-gamified website was not conducted.

Finally, Wolf explored how gamified feedback can lead to higher a higher participation in pro-environmental engagement, specifically in reducing carbon dioxide emissions [36]. The author used the Constructional Level Theory (CLT), which posits that users act based on their psychological distance. A reduced psychological distance can thus increase a user’s engagement in activities. The survey included 450 participants, of which 146 responded in the follow-up survey two weeks later. The results were analysed through Seemingly Unrelated Regression analysis (SUR). The study brought a new theoretical perspective on gamification by drawing on CLT. The author used visual feedback, numerical feedback, and a combination of both as stimuli. Results from the online experiment with a follow-up survey revealed that vividness as well as information quality mediated the relationship between stimuli and sustainable behaviour due to a reduction in psychological distance. Visual feedback increased pro-environmental behaviour through perceived vividness, whereas numerical feedback enhanced perceived information quality. The study also concluded that vividness had a slightly bigger effect on behavioural intention. Correlations are illustrated in Figure 20.

The study tracked sustainable behaviour over a period of two weeks, and this does sufficiently indicate a long-term effect, which is why future studies should experiment with longer timeframes. Future studies can also measure the effect of gamification through other organism constructs from the extended TAM framework. The study used a non-gamified application for a control group. It is noted that the questionnaire used control variables such as demographics and specific behaviour habits. They had a non-significant impact.

4. Proposed Approach

In this section, this paper’s proposed approach is presented, including the chosen gamification dimensions, underling theories, and objectives. In essence, this section addresses the last research question posed in the introduction, i.e., ‘RQ4: How can this information be transferred to the GOLDENSEAL project?’ In doing so, the preferred gamification elements are explained, including design recommendations. Next, the questionnaire and item design are presented, which are used to collect data in order to control the effectiveness of the designed gamification elements. Finally, guidelines are presented on how to analyse the obtained data. It is noted that the terms indicator and item can be used synonymously. According to the SOR model, this section is divided into three parts.

4.1. Stimulus Constructs:

As Figure 21 shows, all five gamification dimensions are covered among the selected papers. Thus, the five stimulus constructs and their related gamification elements (see Figure 8) are as follows:

• D1: Performance-dimension-related elements consisting of G1–G5.
• D2: Ecological-dimension-related elements consisting of G8.
• D3: Social-dimension-related elements consisting of G12 and G13.
• D4: Personal-dimension-related elements consisting of G17–G19.
• D5: Fictional-dimension-related elements consisting of G20 and G21.
• The elements are explained in detail in the following section.

4.2. Organism Constructs:

As Figure 21 shows, most papers based their constructs on Self-Determination Theory and the extended TAM framework. Other theories were used only once but still proved to result in useful information, although some originated from a different domain, such as the brand engagement theory, which is a theory used in the marketing domain. Whereas the TAM model explains how gamification elements affect the usage of an application, SDT describes how gamification elements intrinsically motivate the user. Both theories complement each other and highlight the effects of gamification elements through different perspectives. Studies from Rohan et al. [31] and Jahn et al. [30] have shown that constructs from both theories can even be combined into one SEM. However, the designed SEM contains only one underlying model. This ensures a lower number of constructs and keeps the model less complex, ultimately leading to a lower chance of potentially false correlations. Moreover, a lower number of participants is needed. Using the SDT theory is a choice made due to preference and should not be interpreted as the better option. In fact, in the case of the GOLDENSEAL project, it is recommended to develop an SEM while including TAM constructs in order to highlight other correlations.

Although the SLR found papers supporting RQ1 and RQ3, which are based on SDT, the review does not contain a paper that supports an increase in awareness based on this theory. However, the Centre for Self-Determination Theory reports that, recently, researchers have started incorporating the concept of awareness, which they refer to as mindfulness, into new studies. Donald et al. [37] and Ryan et al. [38] have published papers that support the hypothesis that awareness is positively associated with autonomous forms of motivation. Thus, SDT is assessed as appropriate for all objectives. The constructs are based on the following dimensions: (a) Competence (SDT), (b) Autonomy (SDT), and (c) Relatedness (SDT).

4.3. Gamification Elements and Design Choices

This section presents all gamification elements (see Figure 6) that showed a significant effect on the defined objectives. They are therefore recommended to be tested in future research of the GOLDENSEAL project. The gamification elements are as follows:

D1: 

Performance-dimension-related elements

G1: Progression elements such as progress bars were among the most used elements and showed to positively influence competence [27] and correlate with perceived usefulness [29].

G2: Levels were mentioned only by Permana et al. but showed a positive effect on all four brand engagement dimensions, which eventually led to an increase in awareness [34].

G3: Points were the most commonly used element in the referred study. They had a positive effect on all underlying models and supported all research questions [27,31,34,35]. The application “Ant Forest” managed to give Points a fictional dimension by indicating the amount of saved carbon footprint and growing trees. This example showed a particularly high correlation with relatedness [29].

G4: Stats were implemented through likes and reviews, which had a positive effect on motivation through competence and relatedness. By providing feedback, they could also increase autonomy. Generally, they showed similar effects, such as points and progress bars. However, due to the chosen style through likes and reviews, they showed stronger correlations with socially related constructs [32].

G5: Acknowledgment included badges, trophies, or medals and was also among the most used elements, increasing competence and autonomy in users [31]. It was also perceived as useful and satisfying [28]. However, when it was designed to be visible to other players, it could allow for normative influence. On the one hand, a normative influence can support the use of the application. On the other hand, Du et al. showed that, if the gap in achievements is perceived as too great, then this could have a demotivating effect on other users [29]. This design choice should be applied to all elements related to the performance dimension.

D2: 

Ecological-dimension-related elements

G7: Economy was applied by Permana et al. through the implementation of a virtual currency in an e-commerce shop [34]. The study categorised the feature among other performance features, which overall showed a positive influence on all dimensions of the Brand Engagement Model. However, this may have created bias, as the element should not have been set in this category. The authors further noted that virtual currencies are common elements used by many online marketplace users; however, they are lacking in other domains. As this element was used only once, it lacks the reliability to be used in the developed model.

G8: Imposed choice was implemented through the design of free choice, customisation, and decisions, which increased mostly autonomy but also showed correlations with relatedness and competence [29,31,35].

G10: Time pressure was used in two studies through the implementation of countdowns and limited time. Wirani et al. showed that a countdown negatively affects users’ perceptions of usefulness and satisfaction, as it causes stress and demotivation in trying to complete tasks through the learning platform [33]. Another study also mentioned in its taxonomy that this element is likely to be perceived as negative; thus, it is not incorporated in the current model [20].

D3: 

Social-dimension-related elements

G12: Competition was implemented through elements such as leader boards or rankings. Most studies showed that they increased relatedness and competence [27,31,32]. Although Du et al. [29] as well as Zhang [22] showed that excessive competition and comparisons demotivate low-ranked participants and lead to possible negative effects on overall objectives. A multiple-league system presents a possible solution to lower performance gaps between users.

G13: Cooperations were implemented through the creation of communities, mentorship programs, chat functions, and goals that aim to include multiple users. The biggest and most common effect was measured on relatedness [32] and frequently on perceived usefulness and satisfaction [33]. However, Hamari et al. noted that not all users want to participate in cooperations, and coercion can lead to a decrease in autonomy and motivation [23].

D4: 

Personal-dimension-related elements

G15: Novelty was implemented through random prizes by Wirani et al. [33]. However, they included random prizes among performance indicators, which showed to generally have an influence on behaviour and create bias. As this element was used only once, it lacks the reliability to be used in the developed model. Nonetheless, novelty may lie in including weekly challenges or additional rewards that are randomly assigned. Such implementations could diversify the platform and make the user experience more entertaining.

G17: Puzzles create a cognitive challenge, which, in contrast to an objective (G18), requires more independent solution thinking. Although a puzzle can positively affect autonomy and competence, it can also be overwhelming for the user [39]. It is therefore advised for developers to consider the Fogg Model, which states that sufficient information, knowledge, and skills (ability) should be acquired beforehand. The puzzles’ difficulty and requirements should rise with time to avoid the danger of demotivation and overburden.

G18: Objectives are another type of challenge, which, in contrast to puzzles (G17), provide more transparency. They can contain side-quests or side-objectives and are often associated with clear instructions or clear stats to be reached. Thus, cognitive challenges are often used. In that case, the perceived burden is lower, and objectives still increase competence [40]. Hsu showed that the presentation of multiple objectives to choose from increases the user’s autonomy, as he or she feels less coerced to perform one specific task [35]. The described design choice of G17, difficulty and ability from the Fogg Model, should also be applied here. Other authors have noted that users should not be presented with too many tasks at once, as this may create an overburden [41]. In addition, big tasks may also be perceived as overwhelming, which is why they should be divided in smaller tasks [39].

G19: Sensation was mostly implemented through feedback, which showed to have a large effect on most constructs. Scholars mentioned that it should occur immediately after the action of the user, as a delay may lead to demotivation and a lower effect on competence [30,36]. Wolf also concluded in his study that feedback reduces psychological distance. He showed that visual feedback leads to higher satisfaction, and numerical feedback is perceived as more useful.

D5: 

Fictional-dimension-related elements

G20: Storytelling was implemented in three studies. Rohan et al. showed that storytelling did not show any significant correlation, which may have been due to the domain of education and learning [31]. Bitrián et al. showed that storytelling has a significant impact on all constructs of SDT [27]. Thus, the implementation of this element is recommended, as this SEM offers a more suitable domain and construct for potential correlation.

G21: Narrative was implemented in half of the referenced studies and was achieved through avatars, personalisation, and customisable profiles. These showed significant correlations with all dimensions, as they could be used in a highly versatile fashion. Bitrián et al. showed that the visibility of avatars for other users creates higher relatedness and engagement [27]. Personalisation and customisation also increase autonomy through immersion. Furthermore, the study by Wolf featured visual feedback on an avatar, which resulted in perceived satisfaction and usefulness [36]. Jahn et al. observed the so-called uncanny valley effect, which explains that an avatar that is not realistically designed significantly decreases in acceptance and perceived usefulness for the users [30].

5. Results

This section first summarises the work against the background of the research questions. Then, a discussion on research limitations follows that reflects and critically questions the methodological choices.

5.1. Summary of Results

The main objective of this paper is to conduct a Systematic Literature Review (SLR) with the aim of answering the following research questions and to use the answers to support the works of the GOLDENSEAL research project, aiming at lowering plastic pollution in beach areas. In the remainder of this section, the results addressing each research question are outlined, starting from the core one, i.e.,

‘RQ0: How can the GOLDENSEAL project influence behavioural change in its users through serious games and gamification?’

Firstly, an overview of the available gamification elements was presented in a taxonomy. Next, relevant psychological concepts, models, and theories for the explanation of gamification effects and correlations were studied. This revealed that, currently, there is no unified taxonomy used by the Human–Computer Interaction community to explain either gamification elements or motivational affordances. Therefore, the taxonomies of Toda et al. [16] and Tondello et al. were adapted [42]. The SLR that followed initially showed that papers based on Structural Equation Modelling (SEM) provide particularly valuable information. Thus, a critical decision was made to limit the literature review to studies that employ SEM, thus providing clear multivariate causal relationships between gamification elements and behavioural outcomes. Through empirical data collection, these papers were able to validate well-founded theories by testing them against real-world applications. Furthermore, the Stimulus–Organism–Response (SOR) model proved to be an appropriate framework for structuring SEMs. Thus, another decision was made to only include papers that use SEM, which could be incorporated into the SOR framework. This led to a sample of ten papers. Next, as mentioned in the introduction, to systematically address the main research question (RQ0), this was divided into four sub-questions. Results for each sub-question are provided in the text that follows in a structured manner, providing information in four discrete dimensions, i.e., the performance of the review, underlying models, implications for the project’s research orientation, and domains and limitations.

RQ1: How can the implementation of serious games and gamification increase the usage of a platform?

Performance of the review: Among the ten participating studies, seven of them successfully measured correlations between gamification elements and an increase in intention to use. A high number of studies suggest that a confirmatory approach can be used in the case of the GOLDENSEAL project to measure how gamification affects increases in platform usage.

Underlying models: The referenced papers show that gamification elements such as badges, medals, points, lead boards, ranking, competition, choices, progress bars, cooperations, meaningful stories, narratives, customisation, feedback, and avatars prove to be particularly effective. The correlations and effects are mainly explained by Self-Determination theory (SDT), which is driven by internal motivation, and the Technology Acceptance Model (TAM), which argues mostly that perceived usefulness and usability are the main causes. It must be noted that none of the seven papers measured the long-term effect nor the actual increased usage.

Implications for the project’s research orientation: All papers built their questionnaires solely on the intention of continued platform usage. Although an increase in intention is a good indication, GOLDENSEAL should nevertheless measure the actual increase in the platform’s usage over a longer period.

Domain and limitations: Apart from an increase in intention to use, the papers also managed to measure that gamification can increase the overall app-rating and word-of-mouth among users. The applied domains included healthcare and well-being, marketing and branding, environment, and sustainability, as well as education and learning. Notable limitations included the lack of control variables and control groups.

RQ2: How can the implementation of serious games and gamification increase users’ awareness?

Performance of the review: Out of the ten papers, only one paper was able to measure correlations between gamification elements and an increase in awareness. It should be noted that, among the 153 initially researched papers, no paper was excluded by methodological criteria, which could support RQ2. This indicates a gap in the current state of the research. A possible explanation is that there is a disagreement on whether awareness contributes to behavioural changes [7]. However, the study of Hsu empirically measured the mediating effect of awareness [35]. Thus, users with a higher awareness are influenced more intensively by gamification. This confirms that the design of RQ2 is a relevant question, and additional explorative studies are needed.

Underlying models: The study by Permana et al. showed that gamification elements such as avatars, personalisation, narratives, points, leader boards, badges, levels, progress bars, competition, and feedback result in raising awareness [34]. Correlations and effects were explained with an adapted Brand Engagement Model, which is typically used in marketing. The study showed that other atypical models from other domains can also be used to analyse the effects of gamification.

Implications for the project’s research orientation: Because the project will proceed with the development of a proprietary SEM model, Self-Determination Theory is kept as an underlying theory to explain correlations. Although this model was not used by the referenced papers to support RQ2, additional studies by Ryan et al. [38] and Donald et al. [37] showed that a new research perspective, referred to as mindfulness, has been introduced in 2019. It is equivalent to awareness and extends the current SDT model. If GOLDENSEAL successfully measures effects based on the developed SEM, it could add a great contribution to further deepen the new model.

Domain and limitations: The applied domain included marketing and brand management. The study used demographic control variables, which did not show a significant correlation, and it lacked a control group.

RQ3: How can the implementation of serious games and gamification increase participative behaviour in activities?

Performance of the review: Two out the ten papers participating in this study were able to measure correlations between gamification elements and an increase in users’ intention to participate in a specified behaviour. In contrast to RQ2, three papers were excluded because they did not meet the specified criteria. Nonetheless, it is also evident that further exploratory studies are needed to fill the research gaps.

Underlying models: The referenced papers show that the gamification elements of feedback, narratives, points, free choices, self-expression, and competition prove to be particularly effective. The correlations and effects were explained using Self-Determination Theory as well as the extended TAM framework combined with constructs from the Constructional Level Theory.

Implications for the project’s research orientation: Similar to RQ1, both papers built their questionnaires solely on the intention of continuing a certain behaviour. Although an increase in intention is a good indication, GOLDENSEAL should nevertheless measure the actual increase in the defined behaviour over a longer period.

Domain and limitations: The applied domain was environment and sustainability. All papers used control variables, and Wolf included a control group [36].

RQ4: How can this information be transferred to the GOLDENSEAL project?

To sufficiently answer the question of how the results presented in this paper can be transferred to GOLDENSEAL, guidelines and a tool for verifying the implementation are proposed. The proposed approach is divided into four parts. In the first part, a custom SEM is created that allows one to measure the effect on the defined objectives, O1–O3. It uses the following gamification elements, as these showed great effect in the referenced studies: progress bars, levels, points, stats, acknowledgements, choice, leader boards, cooperations, puzzles, quizzes, objectives, feedback, an immersive story, and narratives. The correlations are explained through SDT. In the second part, the approach includes a description for all gamification elements. Based on the referenced papers, important design choices and potential errors that could have a negative effect should be stated. In the third part, a questionnaire and items are developed that enable GOLDENSEAL to measure the effectiveness of the implemented gamification elements in accordance with the designed SEM. The questionnaire is used to collect data from users so that feedback can flow into a co-design process. The fourth part includes guidelines for the evaluation of the collected data. According to Hair et al., second-generation analysis is a suitable method, as it allows one to measure multiple correlations at once [43]. The combination of limited studies available that support RQ2 and RQ3, and the implementation of a new research approach to measure objective O2, led to the decision to suggest an explorative analysis, such as PLS-SEM rather than a confirmatory approach. To ensure high-quality data, the guidelines contain rules and standards on how to conduct the evaluation using a PLS-SEM.

5.2. Limitations

This section summarises the limitations of the work presented in this paper, divided into three parts, i.e., limitations related to the selected methodology, the collected data, and the proposed approach.

5.2.1. Limitations of the Chosen Methodology and Criteria

The first research limitation comes as the result of our decision to include only studies that used Structural Equation Modelling (SEM), in an attempt to ensure that all studies provided clear cause-and-effect diagrams, thus increasing internal validity. This also increased reliability, as it resulted in more consistent results. However, because SEM uses questionnaires for data input, there is the danger of gathering self-reported data, which may result in potentially inaccurate answers through social desirability. Finding research that uses non-self-reported and empirical data may therefore improve data validity.

Secondly, developing a Stimulus–Organism–Response (SOR) framework has enabled researchers to compare various studies with each other. It became observable that the models followed a repetitive pattern, and correlations could be generalised and applied to various domains, which increased the external validity of the results. However, filtering only for papers that used SEM and that were applicable to the SOR framework limited the number of papers supporting RQ2 and RQ3, significantly. Therefore, future research should allow the exclusion criteria to detect more papers that could answer RQ2 and RQ3 and compare the results.

Thirdly, Krath et al. identified 118 theories in the areas of motivation and effect, behaviour, and learning used to explain gamification [14]. It is evident that relaxing the exclusion criteria of our study could lead to the evaluation of additional theories disclosing further correlations and approaches on gamification and how it affects human behaviour. The same applies in the case of studied gamification elements, because only fourteen out of the twenty-one proposed elements by Toda et al. [16] were found in the analysed studies.

Finally, typical limitations related to source retrieval and language also exist in this study. As such, this study is limited to papers written in English. Other languages could yield additional studies revealing further correlations. The search could also be extended to further—more inclusive—databases, such as Google Scholar, to explore further additional correlations between gamification elements and behavioural change.

5.2.2. Limitations of Collected Data

The internal validity of results is lowered, as only two papers [29,30] out of ten verified the effectiveness of gamification through exposing a control group to a non-gamified version of their information system. For GOLDENSEAL, it is recommended to include a control group in future research when applying the approach proposed in this paper. It is recommended to use a random selection of participants to ensure the representativeness of the effects on a user group and to improve internal validity. Many papers used a user group that was from a specific geographic location or had a cultural background that could have influenced the results. This limitation made the results less applicable to other geographic regions. Nonetheless, the users came from various regions all over the world, i.e., Europe, North America, Southeast Asia, and Central Asia, which indicates that the results may be transferable and representative. For GOLDENSEAL, it is recommended to use a random selection of participants. All studies used control variables that did not show any significant correlations with their effects on the results. This strengthens the construct validity. To test the proposed approach at a later stage, a set of control variables must be developed.

Although the research yielded mostly papers supporting RQ1, only a few papers were found to support RQ2 and RQ3. As already stated, this shows a gap in the current research field, with a focus on SEM-PLS-based papers. GOLDENSEAL could contribute to fill this gap by conducting the developed SEM-PLS. At this point, an important distinction must be mentioned. Although RQ1 and RQ3 ask for the actual action of a behaviour, most studies found support only for the intention. This is referred to as the intention–behaviour gap by [44]. This indicates another gap in current SEM-based studies. Rather than measuring the intention, future studies should measure the long-term effect on actual behaviour. As such, the measurement of data should be interpreted with caution. To draw a decisive conclusion, GOLDENSEAL should review the long-term effect on its users.

5.2.3. Limitations of the Proposed Approach

The proposed SEM for GOLDENSEAL is based on the underlying model of SDT. Although correlations regarding O1 and O3 are backed by empirical studies, the correlation of O2 is built on a relatively new theoretical concept by Donald et al. [37] and Ryan et al. [38]. To the authors’ knowledge of this work, this correlation could be proven for the first time through conducting an SEM-PLS by GOLDENSEAL. If the relation shows significance, it can increase the reliability of SDT. However, if the correlation shows non-significance, it is recommended to alternatively use a confirmatory approach by applying theories such as TAM, as this model has already been shown to work. As the proposed model is based on measurements of perceived experience through applications, the results depend on the actual implemented design of the overall application. Thus, correlations cannot be guaranteed to work in a newly developed application by GOLDENSEAL. The developed SEM is therefore seen as an explorative approach rather than as a confirmatory one. The testing of the proposed model will be conducted in a future research activity, targeted at GOLDENSEAL’s user group.

6. Conclusions

The present Systematic Literature Review (SLR) identifies several significant gaps in the current body of research pertaining to gamification and motivational affordances. These include, but are not limited to, the absence of coherent taxonomies for these constructs, which limits comparability and standardisation in Human–Computer Interaction (HCI) studies. Moreover, no studies based on Structural Equation Modelling (SEM) currently utilise Self-Determination Theory (STD) to measure an increase in awareness, and there is a notable dearth of studies utilising SEM to determine an improvement in pro-environmental behaviour. Additionally, studies in this area are lacking in terms of their ability to link specific gamification elements to psychological outcomes. Finally, scholars tend to measure the intention to conduct behaviour rather than the long-term effects of actual behavioural changes.

By addressing these research gaps in future studies, the HCI community can derive substantial benefits in the form of more robust and coherent models, which can provide additional and novel insights into the effects of gamification on behavioural change. Such models could yield a more comprehensive understanding of human–computer interactions, which can have practical implications for the design and implementation of effective interventions in this field. Thus, it is imperative that future research endeavours prioritise the exploration of these research gaps to advance our understanding of gamification and its impact on human behaviour.

Through a thorough analysis of the literature, this study successfully applies a Stimulus–Organism–Response (SOR) model to several SEM studies. It can be concluded that the SOR model is a highly suitable framework, as it allows for a comprehensive mapping of gamification elements, psychological models, and behavioural outcomes to the stimuli, response, and organism constructs. This framework holds potential for future SEM studies, as it facilitates a more transparent comparison and transferability of results, leading to a more robust body of research in this field.

Regarding the future testing of the proposed model, multiple studies have suggested that the development of applications with the aim of inducing behavioural change should be conducted in cooperation with a selected user group. The user group could test the effectiveness of gamification elements through answering a case-specific questionnaire, as suggested by the proposed approach. The co-design process could be assisted by software development practices or frameworks such as Scrum. Because the user group may be influenced after a long co-design process, it is advised to test the application at the finalisation stage with a new user group. This can increase internal validity, as it avoids the factors of maturation, statistical regression, and influenced testing [45]. Needless to say, the implementation of the proposed approach is irrelevant to the app design, which still must be assessed. This includes the user story, mini-game design, game element design, colours, shapes, video, audio, avatars, profile design, point system, and many more.

Finally, it should be mentioned that the proposed approach is generic and can be adopted by different research settings, as the underlying psychological models can be applied independently from the domains.