An Experimental Methodology for Introducing Educational Robotics and Storytelling in Therapeutical Activities for Children with Neurodevelopmental Disorders

Abstract

Educational robotics is a valuable tool in education and therapy for children with neurodevelopmental disorders (NDD), especially when introduced in activities, combined with gamification and storytelling elements. However, the lack of familiarity of therapists with the technologies involved makes their widespread introduction difficult and leads to case-specific rather than more generalizable methods. In this paper, we present an experimental methodology which provides a guide for the introduction of these elements in therapeutical activities with children with NDD. Providing a common framework reduces the gap between the different expertise of therapists, educators, and engineers. While establishing a common vocabulary and objectives, the methodology provides a guide for designing activities and evaluating their therapeutic effectiveness. We provide an example with a pilot study using a low-cost robot (Ozobot) in a therapeutic environment. Results regarding the children’s task involvement, level of attention, and use of social skills were positive. In addition, the attitude of some children changed throughout the sessions, improving frustration tolerance. The discussion of the pilot study provides clues for improving future implementations of the presented methodology, which serves as a framework for the design of future experiments that include therapeutic activities with educational robotics, gamification, and storytelling.

1. Introduction

From household to industrial applications, the use of robots is expanding in almost every facet of human activity. Nowadays, we can find robots that vacuum or mop floors, help perform surgery procedures, execute non-trivial assembly procedures, or perform complex operations in the aerospace domain, among others. Robots simplify and facilitate human tasks, being for example able to access places that are inaccessible to people. Not surprisingly, the use of robots is spreading everywhere, entering a diversity of fields that were not thought of just some years ago, such as education. The incorporation of robots into educational environments can be seen as a paradigmatic example of how education has entered a process of enhancement through the incorporation of tools related to information and communication technologies (ICT).

Many researchers promote the integration of technology in teaching, both for students and teachers, because of the opportunities it offers. Introducing technology enables the teaching and learning processes to become more engaging and connected to real-world practices [1,2,3]. Introducing technology in the classroom allows the development of activities that would not be possible otherwise—and using robots in education is just an example of this trend.

So far, there is no established consensus on the concept of what the term Educational Robotics (ER) involves. For example, according to the existing scientific literature, different authors frame their definitions around these two categories: (a) ER as an educational tool; and (b) ER as a complex learning framework. ER has been found useful in the niche areas of education. According to [4] (p. 168), ER “is a powerful tool for the educational field in the learning process of average students”; also promoting several skills for students and children with special needs. Nevertheless, other studies also highlight that there are still some uncertainties regarding the use of robots in the fields of education and human care [5]. In [6], Alimisis defines ER as a vehicle for thinking about learning and teaching. He further highlights that “Educational Robotics creates a learning environment in which children can interact with” (pp. 63–64). Pitti et al., in [7] (p. 41), define ER as a systematic and organized method “in which interrelated technological elements (robotic platform and programming software) intervene as mediating tools, whose final objective is to achieve learning”.

ER allows students to interact, participate and practice, performing various learning activities and games, using friendly technological tools. Through play, educational robots help children develop basic cognitive skills of mathematical logical thinking (specifically, computational thinking), developing mental processes that occur when we want to solve a problem through a sequence of ordered actions [8,9]. Additionally, educational robots allow children from a very early age to learn technological skills, such as programming, while also developing social skills. For that purpose, there are simple robots well adapted to kindergarten and primary school. Similarly, there are robots with greater functionality and higher cost that can be used in secondary and higher education, and that offer great possibilities in the development of complex mathematical and scientific concepts.

However, the usefulness of ER extends beyond mathematical, logical and computational thinking. Different studies [10,11,12,13,14,15,16] agree that robot-supported interventions can be particularly useful as an educational strategy for students with neurodevelopmental disorders, such as attention deficit hyperactivity disorder (ADHD), autism spectrum disorder (ASD) and developmental language disorder (DLD). And this is so not only at an educational strategy level, but also as part of therapeutical protocols [17].

ADHD is one of the most common neurodevelopmental disorders in children with an estimated worldwide prevalence of 5%. It continues into adolescence and has a significant functional, personal and social impact. This disorder can include the presence of attention deficit, impulsivity and motor and/or vocal hyperactivity [18,19]. Another relevant neurodevelopmental disorder is DLD, which affects approximately 7% of the population and affects oral language expression and/or comprehension. However, it is a relatively little-known disorder for the general population [20]. Additionally, ASD encompasses a broad group of cognitive and neurobehavioral disorders that include features, such as impaired socialization, restricted and repetitive behavior patterns, and impaired verbal and nonverbal communication. These problems result in limitations for children with ASD that, without intervention, hinder their learning and inclusion in school [21].

In the case of children with autism spectrum and attention deficit disorders, ER has been used for trying to improve different aspects, such as cognitive abilities, communication skills and social interaction, among others [17,22,23,24]. Additionally, the combination of ER and attractive narratives in a game or gamified framework can foster child interest and engagement in the activity [17,22,23,25,26,27,28]. This—the introduction of gamification techniques and serious games (gamification is the introduction of the dynamics, mechanics and elements of games in a non-game context [29,30])—has also been used in recent years for favoring the work of therapists during their work sessions with children with behavioral problems, such as the already mentioned ADHD and ASD, conduct disorders and emotional disorders, such as anxiety and depression. In recent years, gamified therapies have been increasingly used for improving educational performance and children’s health [31].

In this paper, we present a methodology for the development of therapeutic activities based on the use of educational robots, together with the introduction of gamification techniques, included within attractive narratives. This methodology, specifically designed for the treatment of children with different neurodevelopmental disorders, is based on low-cost educational robots. Since there are no specific therapeutic robots for the treatments we performed, we used the off-the-shelf educational robots which could allow us to carry out the activities. The proposed approach focuses thus on the use of low-cost resources, facilitating its widespread use. Additionally, the proposed methodology is open enough to be adapted to the specific needs of both institutions and children. Once the methodology has been presented, this paper also includes the description of a therapeutic activity performed in a specialized institution by children with different disorders, including ASD, ADHD and DLD. The results achieved so far, and described and discussed in subsequent sections, are very promising.

2. Materials and Methods

This section presents the paper’s main contribution: the establishment of a methodology that can be successfully applied to the design of therapeutic interventions to help children with different disorders (in particular, ADHD, DLD, and ASD). The methodology is based on using low-cost ER within a particular narrative and gamification environment. Success while applying the methodology will largely depend on the existence of good communication and interaction levels among the different components of the multidisciplinary team, ideally consisting of professionals covering all the involved areas (psychologists, therapists, educators, engineers, etc.). Each discipline will have a different degree of prominence in each step, with the use of educational robots being a means to achieve the activities’ main objectives and not a goal.

It is important to note that the application of the methodology does not have to be linear. However, as explained later, the process should be iterative, letting improvements in some steps guide the introduction of improvements in others by making the necessary tests and adjustments.

2.1. Creating the Multidisciplinary Team: Concepts and Training

The first step is to familiarize the entire interdisciplinary team with all the concepts, problems and tools linked to the intervention, which involves the existence, use, and handling of low-cost robots; the narrative and gamification environment; the specific characteristics of the children to be treated; the state of the art of therapeutic ER-based activities for children with neurodevelopmental disorders, etc. It is important, at this stage, to discuss among the whole team, aspects, such as the importance of having a structured narrative that contextualizes and motivates future activities. Additionally, it is important to discuss the relevance of gamification in this framework (here, we could define gamification as the use of elements and structures specific to game design to achieve positive impacts on the therapeutic activities [32]).

Specific discussions should be devoted to different aspects, such as the benefits of including gamification as a motivational element [33], both for behavior improvement [34] and for learning or cognitive improvement [35]. Furthermore, some potential risks must be taken into consideration, such as possible rejections of educational robots based on noise, lights, sudden movements, or others [23,36]. The underlying goal is that the experience to be designed should be therapeutic, safe, playful, motivating, and enjoyable for the child.

Lastly, the specific characteristics of the children to be treated, their problems and their possible therapies should be considered and discussed in detail.

To summarize, the objective of this step is to create a common knowledge base and to ensure that the vocabulary used is known and familiar to the entire team. In fact, during this familiarization phase, it is recommended to build a glossary of terms. This will facilitate work during the following steps.

2.2. Definition of Hypotheses and General Objectives

In this step, we aim to obtain the general objectives to be achieved using ER in the treatment and intervention of a specific group of children with neurodevelopmental disorders. The main objective for these children may be to improve one or several of the disorder’s indicators or to increase the children’s motivation, attention, and/or involvement in one or several therapeutic or learning tasks. Additionally, another goal could be to improve therapy outcomes by including robots in the indicator of selective attention. Another example could be to use ER and narratives for decreasing the dropout rates of activities. Lastly, another possible set of goals could be to assess the ways how ER can be applied for helping special children. In general, the objectives do not need to be specific for each child. Instead, they could define a line of progress, which then could be refined, based on each child’s characteristics.

The objectives set forth for a particular case should be specific, relevant, measurable, and achievable. To make the objectives measurable, a series of indicators must be established that allow for monitoring and studying progress. Furthermore, it could be interesting to study the objectives that have not been achieved during the whole activity, analyzing the reasons why this happened.

Legal and ethical aspects must be considered in accordance with the Helsinki Declaration and the Belmont Report, which focus on the three basic principles of respect, beneficence, and justice. They materialize in informed consent, assessment of risks and benefits, and selection of subjects [37].

Regarding the robots being used in a specific activity, sometimes it is possible to think about designing new robots or modifying or customizing existing ones, thinking specifically about the particular children’s needs. However, this design or customization process is expensive and requires time, and therefore, it is out of the question in most cases.

2.3. Recruitment Process and Sample

Once the specific problems of the whole activity have been studied and the general objectives have been defined, the sample size should be set [38] and inclusion and exclusion criteria should be defined [39]. Then, the team should carry out recruitment tasks and establish collaborations with entities and associations where therapy is provided to children with the neurodevelopmental disorders taken into consideration within the activity.

Sometimes, the sample size can be affected by different problems, such as the unavailability of enough children with the specific characteristics sought for each study [24], or the non-acceptance of their parents or legal guardians regarding their participation. In these cases, real sample sizes might be different than the ideal, although studies with smaller than ideal sample sizes should be valued as useful for providing partial, non-conclusive insight. The determination of what are appropriate sample sizes might require performing additional pilot studies when there is not enough previous work on the subject.

Regarding the recruitment process, an informative session should be held where the general objectives will be presented. Informed consent will be obtained from the interested parents and guardians to move forward. In this phase, it is relevant to involve parents in the objectives to be achieved, so that they are engaged in the outcome of the proposed activities.

The current therapists of the participating children should be consulted to learn about their particular characteristics, needs, and current treatment. Whenever possible, the baseline measurement of the participating children will also be taken, i.e., information on the value of the indicators will be collected at the beginning of the whole activity.

2.4. Definition of Specific Objectives

Once the participating children are selected and their diagnosis and characteristics are taken into account, a study should be conducted to identify their therapeutic needs as well as their preferences and/or possible individual limitations.

Based on the results obtained and possible similarities or disparities, it will be decided when and how to customize the intervention, so that the same intervention can be established for a homogeneous group of children or individual adaptations can be made based on each child’s particularities. Thus, specific objectives will be defined for each of the participants (or, in their case, for each of the groups). Similar to the general objectives, these objectives must be specific, achievable, relevant, and measurable, and a date of achievement will be established in this case.

2.5. Activity Design

In this step, the overall narrative, the strategy for gamification, the activities to be carried out and the use of the selected robots must be designed, according to the specific objectives previously established. As a first step, the traditional intervention activities that were being carried out with each child (if applicable) should be studied. What objective(s) each activity pursues, what indicators can be measured, and what the level of difficulty will be for each activity will be taken into account. It will also be assessed whether an adaptation of such an activity would be sufficient or whether it is pertinent to include new activities adapted to the age and characteristics of each child, and in that case, which ones.

Once it is clear what type of activities are desired, they will be designed using the traditional activities as a guide, so that:

• A narrative that surrounds the whole activity is included as a game or challenge.
• Gamification elements are also included to increase the activity appeal among the children.

  Table 1. Some of the most used gamification elements [40,41].

  Gamification Element	Description
  Points	Points that a participant receives for completing a specific action, in the same way that a player would receive for positive actions carried out throughout a game. You can use these points to unlock new features as well.
  Badges	Rewards that show the achievements obtained by a participant in the activity. They are usually visual and, in some cases, incremental.
  Levels	Ensuring the levels work, such as chapters in a textbook, so they also formalize the progression of the content. Once you complete a level, you gain access to the next, more difficult content. Curiosity and the desire to progress make levels motivating.
  Rewards	Rewards are provided to users after completing a specific action. On many occasions they are placed by chance in the gamified activity. Some examples are Fixed Action Rewards, Easter Eggs, Mystery Box…
  Challenge	A gamified activity can have specific objectives. Participants must be involved in solving the challenge to achieve the activity objectives. These challenges motivate the participant to think, explore and can increase the level of commitment.
  Collaboration	A collaborative activity is a technique used in groups where participants can work together to solve a problem, and thus, achieve a shared understanding of that problem.
  Feedback	It is the process of providing feedback on the performance of an activity to drive engagement and engage participants. It can usually involve several of the other elements of gamification.
  Leaderboard	It is a scoring list, such as a hall of fame, where you rank the players according to the points obtained in a certain criterion to be highlighted. These tables identify the best in a certain activity, being able to combine different tables according to more than one criterion.
  Storytelling	These are techniques used in the media to reveal the narrative of a story in a simple, schematic and visual way. It is a simple way to explain what story you want to tell.

  shows a series of examples of elements that can be included in the activity.
• The appropriate robot or robots for the selected narrative and activities must be selected, considering the children’s age, their specific disorders, and the objective sought. This is an important aspect; there are robots which could make loud noises or emit bright lights that could be counterproductive in some specific scenario.
• The level of difficulty of the activities to be performed. Different levels of difficulty can be thought of, in order to allow selecting the best suited ones, depending on the children’s age, characteristics, etc.

Once the activities are designed, they must be implemented with the selected ER. This includes robot programming, step-by-step instruction development, collection of indicators, elaboration of game boards, activity cards or any other issue required by the activities.

2.6. Research Experiment

Here, we label the type of research experiment to be performed according to different factors. In the example presented here, the aim of the experiment is to study the effect of using robots and narratives on children with neurodevelopmental disorders, and the intervention with ER can be considered an experimental study. In general, if the experiment involves evaluation before exposure to the intervention factor, it would be considered a prospective study. If the influence of the factor on different indicators can be measured, it could be seen as an analytical study. Finally, if these indicators were repeatedly measured at different moments over time, which would be desired, it would be a longitudinal study. Other aspects, such as the degree of blinding of the experiment and the characteristics explained below, should be determined as well.

2.6.1. Existence of a Control Group

Depending on the defined objectives, and always respecting all ethical principles, it will be established whether there is a need to define a control group versus an experimental group, or whether there will be just a single group of children. If the number of subjects to be treated is sufficiently small, it might not be possible to establish a control group. If this is not the case, but it is desired that all children benefit from the intervention with robots, it is possible to plan the activity in such a way that, initially, the control group follows the conventional treatment, while the experimental group receives intervention with robots and narrative, and once the established period of time is over, the first group is also offered the possibility of receiving the intervention with robots and narrative.

In randomized controlled clinical trials (RCTs), it is common to randomly assign individuals to groups to avoid biases and to ensure that the groups are homogeneous. However, it is often difficult to count with a sufficiently large number of children with the considered neurodevelopmental disorders. Furthermore, children diagnosed with a specific disorder show strong differences among them. Because of this, it is recommended to evaluate if it is better or not to carry out a not randomized controlled clinical trial, in which the process of allocating children to particular groups is not randomized but based on the characteristics of the subjects. This might be more effective for obtaining balanced groups that can be compared later.

2.6.2. Operational Definition of Variables: Dependent, Independent, and Extraneous Variables

The independent variable will be the designed intervention, based on using robots within a narrative together with gamification elements. The dependent variables will be the indicators that will be measured for studying children’s progress with respect to their neurodevelopmental disorder. During the activities, it will be studied whether the dependent variables indeed vary depending on the intervention.

In the experiment design, it is important to minimize the effect of possible extraneous variables (other variables that could be influencing the relationship between the dependent and independent variables, such as previous experience in using robots). An attempt should be made to balance both the subjects’ characteristics and the intervention and evaluation situation where information on the indicators will be recorded. It will also be studied whether it is advisable to perform a habituation procedure regarding the use of the selected robot prior to the intervention.

Thus, individual differences within the group of subjects and the characteristics of the intervention protocol should be as equitable as possible for different subjects (for example, everyone has the same time to complete activities, interventions are always in the same environment for different subjects, etc.). The measuring conditions for the different indicators should also be equalized as much as possible, for example, in terms of objectives, time, and difficulty of the activity to be performed.

2.6.3. Measurement of Indicators and Evaluation

The experiment design should consider when and how each of the indicators will be measured, both at the beginning of the experiment when establishing the baseline and during the intervention sessions and the final evaluation, if applicable. Some of the indicators that can be measured include sustained attention time, level and type of assistance that children need to perform each activity, correctness in performing the task (errors made), behavior, level of frustration, or dropout rate, for instance. The indicators can be obtained using different measuring methods, such as tests or observation sheets.

If there are different groups of children, an effort should be made to equalize the conditions under which such evaluations and measurements are made. For example, the same type of activity should be performed, and children should have the same amount of time available for performing the activity, the same aids, the same number of attempts, etc. The intervention protocol, as explained in the following section, will determine how to proceed at each moment.

In the case that the indicators should be measured by more than one expert, it is advisable to create an observation protocol and train the experts to obtain inter-rater agreement.

2.7. Intervention Protocol

Ideally, the therapeutic activities that will be developed should be applied by professionals who already have a bond with the children and have experience working with them. Therefore, it would be convenient to carry out the activities in spaces already familiar to the children, in order to minimize extraneous variables and to ensure that the only new elements to affect their performance are the specific activities, defined by the robot, narrative, etc.

The intervention plan should describe how the specific objectives will be achieved, detailing the number of sessions, the amount of time they will take, if there are any breaks, the activities to be carried out in each session, and the indicators to be measured.

At the end of the intervention, in addition to recording the indicators evolution, a test of final evaluation will be carried out, when possible. This will facilitate comparing results with the baseline evaluation.

Lastly, the intervention protocol should also detail when and how therapists can carry out interventions, such as offering help, so that it is equitable for different children.

2.8. Statistical Analysis Plan

The statistical analysis plan should be established at this stage. Specifically, depending on each experiment, it will be determined which statistical tests will be applied to perform the data analysis to be able to obtain the activity results. In the case of having a control and experimental group, parametric (if the required assumptions about the population are met) or non-parametric tests, can be applied to compare the indicators of both groups, seeing, for example, if the mean differences are significant. If there is no control group, a test–retest can be performed and the results of the same child at the beginning and end of the experiment can be compared to study whether there has been an evolution in the indicators for that subject. Note that the time elapsed between test and retest is highly relevant: The longer the time between them, the more difficult it will be to tell how much improvement is due to the intervention or to the child’s own evolution.In addition, the results of satisfaction questionnaires (based on a Likert scale) will be analyzed to study the children’s opinion. Descriptive statistics, such as mode and median will be observed.

2.9. Pilot Study

Before a large-scale intervention is started, it is highly advisable to carry out a pilot study with a reduced number of subjects. This study will allow identifying feasibility problems or shortcomings in both the intervention and observation protocols. This way, the necessary corrections can be introduced in the experiment design before conducting a full-scale study.

2.10. Definitive Study

Once the modifications have been made based on the information obtained in the pilot study, the definitive study will be carried out, interpreting the results obtained and concluding on the hypotheses initially proposed. The results of this study, in turn, will serve to introduce improvements in the application of the methodology and refine some of its stages for future research.

It is usually necessary to repeat the entire methodology or some of its steps iteratively to progressively refine it. In particular, the information obtained in the pilot study is very relevant for determining which steps still need to be improved.

3. Results

This section presents an example of how the methodology described above can be applied to a specific intervention. This example can thus clarify the whole procedure proposed in this paper, together with the different steps that are recommended to be taken during the design process.

The practical case described here has been performed in close cooperation with Fundación Esfera, a private non-profit organization funded by the Community of Madrid. This foundation aims to the rights, needs and wishes of all people with intellectual disabilities. The Fundación also supports their families to enable them to achieve an adequate personal and social development, based on rigorous, updated, and ethical professional practices [42]. A collaboration was proposed whereby children with different neurodevelopmental disorders could benefit from the development of activities embedded in a narrative, using gamification elements and an educational robot in the therapeutic environment. A total of nine children with ASD, ADHD and DLD disorders finished all the activities in different sessions with an average time of 326.67 min.

3.1. Creating the Multidisciplinary Team: Concepts and Training

In this stage, we held some previous meetings with Fundación Esfera, in which the experiment was explained. Followingthe project presentation meeting, two therapists volunteered to take part in it, despite having no prior knowledge of robotics. After this, the multidisciplinary team was created, including the ER research team and the therapists. As a first step, the main goals and methods were discussed: On one hand, the global methodology and goals were examined, and the children’s needs and established therapies were analyzed and discussed. On the other hand, the benefits of gamification and the power of narratives were shared. Then, members of the research team presented the different robots that could be used, their main features and how to use them.

In a separate stage, the activity was designed, and all necessary support materials were elaborated. These materials, together with the required robots, were handed out to the therapists so that they could be used during the children’s interventions.

3.2. Definition of Hypothesis and General Objectives

At this point, the hypothesis established for this work is that children with the considered disorders and therapies can benefit from the performance of specific tasks embedded into a well-adapted narrative encompassing the whole activity, the introduction of gamification elements, and the use of low-cost robots within the therapeutic environment. In particular, the main objective is to study whether attention on a specific task by the considered children is encouraged while following the methodology proposed here. Other less central objectives, together with activities that needed to be completed were:

• Verifying the hypothesis (namely, ensuring that the considered children could benefit from using activities designed following this methodology).
• Setting up an experiment for gaining experience regarding the design of therapeutical activities for children under treatment, and studying the results provided by the execution of the designed activities.
• Designing the children’s therapeutic activities for this study. The activities to be performed could be adapted to the needs of each child or task if required.
• Selecting the gamification elements to be introduced and designing the game mechanics to be used in the activity.
• Designing the narrative that surrounds the therapeutic activities. For the work presented here, the selected narrative was based on the exploration of the Solar System, with some hints of the Star Wars universe, such as a reference to Mandalorians.
• Selecting the low-cost robots to be used and adapting the activities to ensure their completion using the selected robots. In this case, the Ozobot robot [43] was chosen.
• Lastly, defining which indicators and information are relevant for assessing the activities’ efficacy. For this experiment, it was decided to collect information on how long the children were paying attention to the activity, together with their difficulties, mistakes made, help received, their degree of frustration and their degree of satisfaction.

3.3. Recruitment Process and Sample

This intervention was carried out in collaboration with Fundación Esfera. The therapists contacted the children’s parents and/or guardians to inform them and obtain their consent to take part in the whole activity. The entire population of children that could take part in the study was small, given the number of children being treated at Fundación Esfera and the inclusion criteria used: children having been diagnosed a neurodevelopmental disorder (ADHD, ASD or DLD) and being on a similar curricular level, as well as having their families approving their participation in the activities. The exclusion criterion was the rejection of the robots.

The experiment started with ten subjects. However, one of them decided to drop out of the activities and, as a result, the final population meeting the required criteria was nine children, none of whom had previous experience with robots. Being a relatively small number, it was decided to include the whole population in the study. The absence of enough related work in this area prevented the theoretical determination of the appropriate sample size. In that sense, further work is needed in this area.

3.4. Definition of Specific Objectives

Once the subjects and their characteristics were known, specific objectives were established for each subject. As in this case, the curricular level of the children was similar, and they all performed the same activities with challenges of the same level of difficulty.

The specific indicators collected are shown in

Table 2. Specific indicators to be collected through observation of the results and behavior of the children in the sessions.

Group	Indicator	Values
Children data	Gender	Male/Female/Other
	Age	Number
	Educational stage	Kindergarten/Primary education
	Diagnosis	ADHD, ASD, DLD
Play time	Number of sessions	Number
	Session time	30 to 60 min
	Actual play time	% of total time
Help	Needs help	% of number of sessions, on Likert scale
	Type of help	Physical/Verbal
	Respects turn	On Likert scale
	Collaborates	On Likert scale
Errors	Reading comprehension	On Likert scale
	Precipitation	On Likert scale
	Calculation error	Yes/No
Frustration	Expresses frustration	On Likert scale
	How frustration is expressed	Verbally, Physically, Other
	Number of dropouts	Number of times
Difficulties	Robot	On Likert scale
	Game cards	On Likert scale
	Understanding of challenges	On Likert scale
Qualitative observations	It is observed that the child likes it	Motivating Activity, Novel activities, Motivated by robot use, Other
	It is observed that the child dislikes it	Robot errors, Problems resolutions, Other
	The child said I like it	Everything, Use of robot, Other
	The child said I dislike it	Robot errors, Other
	The child would like to have	Open
	Other observations	Open

.

3.5. Activity Design

In the next stage, the therapeutic activities that had been tested previously were adapted so that the children could carry them out in a gamified environment, using Ozobot. As explained previously, the activities were embedded within an engaging narrative, designed to capture, and hold the kids’ attention for as long as possible, and in which it was easy to integrate the selected low-cost robot.

Another key element was the use of elements that facilitate robot customization, allowing the robots to become avatars (such as LEGO-type minifigures turned into Mandalorians or pilots from the well-known Star Wars series), which each child could choose among a given set.

Given the children’s characteristics, the chosen narrative was space and the solar system. The Ozobot robot, a low-cost and easy-to-operate robot, was selected for conducting activities, providing a relatively smooth learning curve for therapists and children (Figure 1 and Figure 2). The children who took part in the activities were told that they, together with robots, had been selected to carry out a space mission. Furthermore, in order to be successful, they needed to learn and train to pilot spaceships (the robots) and obtain the astronaut card. Only then they would be able to carry out their mission.

Ozobots are small educational robots that follows black lines and can read instructions regarding speed, line junctions and intersections, line jumps, etc. These instructions are codified as sequences of different colors that can be printed on stickers to be stuck along the lines to be followed by Ozobots (Figure 1 and Figure 2). Regarding the design of the activity boards, Ozobots can move over paper/cardboard or electronic devices, such as tablets. The robot’s sensors recognize the color sequences formed by four colors: black, red, green and blue. These sequences can be formed from two to four color points, creating up to 27 code commands.

Regarding the therapeutic activities, there is no consensus on the minimum number of necessary sessions for obtaining results showing reliable improvements [26]. After consulting the therapists, it was decided to carry out at least six sessions per child before deciding whether it was necessary to include additional sessions or not. The time of each session was determined by the institution, according to the usual session format. The whole activity was decomposed into two stages. During the first stage, the children had to obtain an astronaut/spaceship pilot accreditation—that is, they had to demonstrate abilities to control the robot by selecting the correct color codes: direction movements, speed control, jumps from one line to another, etc. In the second stage, the children had to navigate across a game board designed specifically for Ozobots, containing the entire solar System. In both stages, there were two levels of difficulty from which the child could choose. In each session, our astronauts had to select the type of challenge and its degree of difficulty. For the last session, if they accepted the final challenge, they themselves had to design and solve a challenge that could be presented to another (imaginary) child.

3.6. Research Experiment

Given the experimental character of the activities, Fundación Esfera preferred not to include a control group so that all the children could enjoy the innovative activities. Additionally, in order to disturb the sessions’ dynamics as little as possible, it was decided not to carry out additional diagnostic tests on the kids, since they had already been diagnosed by the competent department of the regional government of the Community of Madrid.

Instead, it was decided to start from the therapists’ knowledge of the participating children, with whom they had been working for some time. Thus, an analytical prospective experimental study was carried out without a control group. Lastly, based on the indicators collected in Section 3.4, a form was created so that the therapists could collect their observations during each session (see Figure 3).

3.7. Intervention Protocol

The intervention protocol included carrying out the designed activities in the kids’ normal environment within Fundación Esfera, with the support of the same therapist who was already working with each child. The main reason for proceeding this way was to facilitate children’s acceptance, and to minimize the influence of external variables. The total length of the activities was, on average, 326.67 min, respecting the times originally planned for the sessions involving each child and therapist (before the ER-based activities were introduced). It was allowed within the protocol that the therapists could offer verbal help if the children requested it. In that case, the help had to be recorded within all the collected data. Additionally, following the intervention protocol, the therapists could offer help to make the robots move, stick the programming labels in the game board or help children with the color-coded sequences, if requested.

The activities consisted of solving challenges and programming the robot by using labels that provided the robot with commands to travel across the game board. The challenges were related to the theme of the narrative (first obtaining the astronaut accreditation, then being able to move through space to carry out tasks linked to a game-activity objective). Carrying them out required focused and sustained attention. Furthermore, during the activities, the children also had to perform several tasks, related to literacy (involving reading comprehension; especially of inferential and interpretive nature), and mathematical/logical reasoning (reading and understanding the challenge; performing calculation and memorization; programming the robot, etc.). Additionally, they had to build a spaceship with LEGO-type pieces (involving, therefore, the control of visuospatial capabilities), and designing and writing a new challenge, supposedly for other kids. Likewise, during the sessions, the children worked on aspects such as their tolerance to frustration and their abilities for planning and organizing the task, learning to learn, teamworking, social relationships and the acquisition of curricular content, embedded in the narrative and the robot game.

Just before the last session, and after the children had assembled a LEGO-type spaceship, the therapists asked them about the possibility of performing a new task at home. For that, they had to write an original challenge, using (if desired) the challenges made during the previous sessions as examples, otherwise they could even choose their own theme. To motivate them, they could take the spaceship home and bring it to the next session, along with the solved writing and math or logic challenge. The children had no consequences if they did not complete the challenge. However, without any prior notice, a positive reward was given to everyone who brought their homework done: they could keep the LEGO-like spaceship that they themselves had assembled during one of the challenges.

3.8. Statistical Analysis Plan

An analysis was conducted on the data collected in the activity sheets. For the variables evaluated using a seven-point Likert scale, the median was calculated. For variables in which yes or no were marked, the percentage of yes responses obtained was calculated.

3.9. Pilot Study

Since this was the first time that the proposed methodology was applied, we began with a pilot study that included ten subjects with neurodevelopmental disorders and a similar curricular level. As mentioned before, one of the children, due to frustration, decided to abandon the experiment. In consequence, the results described below correspond to the none subjects who completed all the tasks.

Of the nine children, only two of them were girls. The children’s ages were between 8–12 with a mean age of 10.22 years (SD = 1.39). The mean total time per child in the sessions, in which a game that included a narrative and the use of a low-cost robot, was 326.67 min (SD = 119.48), with an actual mean total game time of 247.22 min (SD = 93.61). Of the participating children, 22.22% never needed help, while 77.78% needed occasional help. The median score on the seven-point Likert scale was 3, corresponding to “needed occasional help”. The type of help was verbal in all but one case, in which a child required physical help for placing the stickers and moving the robot.

Regarding the errors made by children, 55.56% of the children made reading comprehension errors, while 77.78% made haste-originated errors. Nevertheless, none of them made calculation errors. With respect to frustration, the median was 3, corresponding to “expressed frustration verbally in an occasional way”; 22.22% of the subjects did not express any feeling of frustration, 55.56% of the subjects expressed frustration verbally in an occasional way. A percentage of 11.1% of the subjects expressed frustration verbally quite a few times, while 11.11% of the children expressed frustration almost all the time, verbally and sometimes non-verbally (specifically, hitting the floor and the table, turning away from the activity, and on one occasion, throwing the robot). With respect to the difficulties faced when carrying out the activities, the median was 1, corresponding to “no difficulties at all “33.33% of the participants found occasional difficulties with the use of the robot and the game cards, while 66.67% of the subjects did not find any difficulties at all, neither while using the robot nor the game cards. Last, 77.78% of the subjects did not find it difficult to understand the challenges, while 33.33% of them occasionally found some difficulties. In this case, the median was 1, corresponding to “no difficulties at all”.

There was an open-ended question where the therapists could write what the children said that they liked most. 44.44% of children mentioned that they had liked everything. 33.33% of them liked best the robot, mentioning its special maneuvers and speed codes. Lastly, 22.22% of the children highlighted the fact that the activity was noveland motivating. Regarding the question of what the therapists found that the kids disliked the most, in 88.89% of the cases they didn’t like the robot’s errors, and on one occasion one child did not like the way a particular problem was solved. Additionally, 33.33% of the participants stated that they had liked the whole game, while in 55.56% of the cases, they said that their favourite activity was assembling the LEGO-type spaceship. One participant mentioned the special maneuvers, and another, that he most-liked the speed codes. In 88.89% of the cases the participants stated that they most-disliked the robot’s errors.

Another open question asked participants about what else they would like to see in a future version of the game. Participants mentioned a more complex solar system, more Mandalorians, more assembling activities, and more difficult challenges. It must be noted that the request for more Mandalorians refers to children being allowed to select their avatar, that they could place on top of their robot. In this study, these avatars are LEGO minifigures that can be placed on top of the robot, so that the character “moves in space” using the Ozobot robot as a spaceship. One of the participating girls requested that her avatar was a female, not a male Mandalorian.

There was also a section for observations to be made, where the therapists stated that one of the subjects experienced problems with laterality when applying the stickers. A boy found it difficult to assemble the LEGO. A girl was discouraged when she saw that she had to assemble a LEGO-type spaceship, because she thought that the task would be too difficult. Nevertheless, she overcame her fear and finally built it instead of abandoning the experience. As a result of observing her difficulties, the therapists were able to detect that the girl’s previous diagnosis was not accurate in the sense that the relevance of her problems for visuospatial activities was greater than what had been previously diagnosed.

All children (100%) who took home a challenge that also involved assembling a LEGO-type ship, brought it assembled to the next session, with a challenge designed by them already solved. Some qualitative comments from the therapists were: “We observed that working with the robot had positive results in terms of the children’s involvement in the task, the level of attention, as well as improvements in social skills, in some cases”. “We are very surprised that all the children have solved the challenge and completed the writing activity of the new challenge that was given to them as homework”.

The example presented in this paper concludes with this pilot study, leaving the definitive study as reference for future work. This may imply an iterative process of refinement (as explained in Section 2.10).

4. Discussion

In a changing social reality, if we want to bridge the gap between therapeutic innovation and practice, it is advisable to initiate close collaborations among professionals and researchers from diverse backgrounds, such as therapists, engineers and educators. Their teamwork can then be very productive regarding the design of tools and procedures adapted to the needs of specific subjects and their therapeutic goals [44,45]. After these tools are designed, they must be validated, not only in optimal laboratory conditions, but in actual therapeutic situations, with real patients such as those for which they were designed [46].

The work presented here follows this approach, having been developed with the support of a multidisciplinary team during the design of educational interventions based on low-cost educational robots, narrative, and gamified activities. These activities, addressed specifically to children with ASD, ADHD and DLD, have allowed them to show improvements on their attention and impulsivity control, and on the strengthening of their self-regulation skills. The The therapists observations during the activities implementation has been fundamental due to their experience and their knowledge about the children development. The implementation of robotized activities led to actions that had not occurred before with the children, such as bringing finished homework from home.

From this study, we have found indications that educational robots can help children with special educational needs, such as in the case of ASD, to overcome motivational and sensory processing barriers. These types of children, when interacting with other children, do not often seek or try to maintain relationships with their human peers. Often, they prefer stimuli provided by non-human or electro-mechanical devices, unlike other kids, who may find social relationships with other peers gratifying [45].

Something similar can be said about children with ADHD and DLD. In their case, our results, similar to the others presenting interventions using ER [18,19], have shown improvements regarding the self-regulation of emotions and behaviors: the children’s symptoms of inattention, hyperactivity and impulsivity progressively decrease.

Priority was given to having the children working with their own therapist at their usual session time, and without introducing additional overheads for completing new diagnostic evaluation tests.

Initially, the Fundación Esfera therapists who participated in the experiment were interested in collaborating without having previously used robots in their activities. Throughout the intervention, several meetings were held where the progress of the activities was monitored. Finally, at the end, the therapists verbally expressed their enthusiasm with the results obtained and about the possibility of including ER as an aid tool for learning agreeing with previous research [45,47,48,49]. They were satisfied with the inclusion of robots, noting that the children seemed to enjoy interacting with them. Additionally, it seemed that the children’s engagement increased and, in some cases, improvements in social skills were also observed. Additionally, it seemed that their involvement in the tasks was greater, as well as, in some cases, improvements being observed in their social skills. Furthermore, the application of low-cost educational robots in a collaborative environment with children with ASD, ADHD and DLD did not show any undesirable effect on the subjects, in accordance with [50]. On the contrary, it seems to produce, during the educational intervention, a qualitative improvement in the focus and maintenance of attention during the activity, generated by an increase in task motivation. This has allowed participants to show a greater level of perseverance, together with a larger capacity for cognitive efforts, throughout the sessions. All of this resulted in a greater level of involvement in complex activities, with few people abandoning them.

One of the limitations of our study is that the chosen robot occasionally failed to correctly interpret the color codes. This was a source of frustration, since having performed the activity correctly, they found that the robot had failed. Although this limitation provides an opportunity to work on frustration in the therapeutic environment, and in fact, all the children except one were able to overcome it, it also shows the importance of choosing the robot: if a robot failure causes frustration in neurotypical children, in children with neurodevelopmental disorders, the intensity can be higher as well as the prevalence of emotion dysregulation [51,52]. Specifically, the robot malfunctioning was behind one of the children saying that he did not want to use the robot anymore and decided to quit the experiment.

Another shortcoming of this study is the small sample size which, however, did not prevent us from reaching the main goal: carrying out this pilot study for detecting shortcomings to be corrected for future studies with a larger number of subjects. As more similar studies become available, they will facilitate the calculation of the sample size. Nevertheless, it has to be pointed out again that conducting such studies with larger sample sizes is difficult, due to the problem of finding many children with similar characteristics able to take part in the same experience, preferably with the same therapists.

5. Conclusions

This article presents a methodology that can help along the process of designing and developing therapeutic activities using low-cost robots. This methodology was based on transforming an activity into a narrative that is attractive to children, and enhancing it through the introduction of gamification elements, which describe the different steps to be followed at design time, and can be applied to a wide variety of specific needs and objectives. As an application example, we have also shown the design and development of a space travel-based narrative, which has been carried out with Ozobot, and demonstrated how it has been applied to a compact group of children with neurodevelopmental disorders, having a similar curriculum level. The paper also includes the experimental results achieved, which have been satisfactory both from the point of view of children and therapists.

There are some published studies regarding the use of new technologies for treating people with neurodevelopmental disorders. In general, they typically address only a particular case, or provide a particular solution for a specific technology. However, the work presented here provides a framework and the context to guide professionals along the process of creating therapeutic activities, so that successful progress can be made increating activities addressing a wide range of problems and goals. Regarding the application of the methodology, the interventions designed here have been found enjoyable by children and useful by therapists. The activity design has maintained a low profile, introducing little changes in the children’s routines, since they have carried them out with their own therapists and in the same physical space they were used to.

This work has some specific positive aspects to be highlighted. First, once the children were selected, a familiarization session was held to see if the robot was accepted by the child and if it was appropriate for achieving the goals that had been set. Additionally, the activities to be performed include different levels of difficulty, and since it is the child who chooses at which level they will be (being able to change levels whenever they wish), their empowerment and self-esteem are fostered. In a way, the whole procedure is designed in such a way that the activities adapt to the child, and not the other way around.

Additionally, even though the therapists in our study had no previous technical training or experience neither in ER, nor in gamification, they were able to successfully implement all the designed activities. This is an important topic, since technology-related issues, such as ER or gamification, are generally not included in the existing curricula for therapists, educators and psychologists. The methodology described here is thus meant to be used as a first point of contact when experimenting with using a narrative, gamification elements and robots in the therapeutic environment, opening a new range of possibilities. The fact that the participants were able to successfully implement all the designed activities shows a good design of the experiment and activities included in this work, thanks in turn to the proposed methodology.

The design process for the narrative and the included activities were supervised by the therapists, in order to guarantee that they would fit well within the children’s curricular level and intervention plan. The narrative and gamification elements, such as being able to obtain badges and rewards once certain difficulties had been overcome, motivated the active participation of the children, who, thanks to the challenges posed, progressed at their own pace along the different sessions. In a way, they themselves were the ones directing their own adventure.

The results of this pilot experiment are promising, including the serendipity that allowed the detection of an inaccurate diagnosis which could lead in the future to designing specific activities to complement the diagnoses. Nevertheless, there are several aspects that can be improved. For example, future versions of the methodology could include the measurement of indicators that, in turn, can be used for monitoring and modifying the therapeutic process, in order to adapt it to the evolution of each specific child. Additionally, future work includes integrating the conclusions and experience gained with this study into new therapeutic activities, to avoid the shortcomings described in previous sections. Furthermore, additional work is needed regarding the selection (or even design and construction) of a robot to minimize code reading errors, and to avoid uncontrolled frustration. Lastly, the design of new activities will also allow the performance of longitudinal studies so that, for the same child, measurements can be collected over several years in order to study its evolution. This will be very useful for the design of long-term intervention plans.

Presently, we are working on the development of technological tools able to support the work of therapists, in order to free them of part of their workload during interventions, something that will result in better attention to patients. Among them, it is worth highlighting an app for mobile devices that is being developed and will facilitate data collection. Additionally, this will facilitate automatizing the process of data gathering, storing, and processing, which is essential for further analysis. On the other hand, a new set of robots, more suitable for children with special needs, are being designed, according to the feedback received in this work.

In conclusion, we are working on the implementation of technological solutions that could facilitate the automatic collection of specific data from each session. In the future, as children are exposed to therapies that include narratives, gamification elements and the use of robots, we will work in different issues. For example, we plan to assess not only the degree of satisfaction or progress, but also whether the motivation is maintained or if, on the contrary, any of the elements has lost effectiveness or generates boredom. This will facilitate adapting the therapy and the activities to the needs and likings of each child that, logically, can vary over time.