Emerging Technologies and New Media for Children: Introduction

In his 1749 essay, Letter on the Blind for the Use of Those Who Can See, Diderot explores the impact of vision, or lack thereof, on the development of knowledge. Taking as an example the blind mathematician, Nicholas Saunderson, who learned through touch, the philosopher refutes a rationalism that would limit the development of thinking to what can be derived from innate and visual ideas and categories. Diderot’s empiricism, recognising the role of the body and experience in learning, inspired other Enlightenment philosophers’ writings on education, such as Rousseau’s 1762 Émile. It further paved the way for educating blind children.

Through the nineteenth century and the first half of the twentieth century, pedagogists and scholars explored ways to train the body through the mind and the mind through the body. This emerged both from concerns about the impact of mechanization and industrialization in the least privileged, which philosophers wanted to compensate for with an improved humanist education. Children were considered to have a natural connection with the physical world, which could be leveraged in the everyday [1]. Pedagogists such as Friedrich Fröbel advocated for an early education rooted in play and physical engagement, giving wood blocks an enduring status as the perfect toy. Mothers were encouraged to initiate their children in scientific inquiry and problem-solving during household chores (see [2], p.10). Theorists dear to the progressive education movement, such as John Dewey, Jean Piaget or Lev Vygotsky, all shared a concern and attention for the role of practice and active bodily engagement (see, e.g., [3]). However, even later research on learning using cognitivist approaches to design multimodal representations, combining images and sounds, circle back to examining how the body and its senses structure cognition [4].

The widespread adoption of computers fuelled new concerns that children’s physical and cognitive development would be compromised; however, there were new hopes that their flexibility would open new opportunities for learners and shape more engaged citizens. This is only the latest iteration of this phenomenon. Before computers, radio and television were framed as new modalities that improved on the written text. They could encourage children’s active physical engagement in learning and reach learners in new and everyday contexts [5]. Their public perception gradually evolved, they became framed them as encouraging passive consumption and, contrary to teachers or computers, depersonalized [6]. The development of personal computing appeared more aligned with progressive movements that focused on children’s ability to express themselves and learn independently at their own pace based on their own interests.

For instance, Alan Kay designed the Dynabook, a proto tactile tablet, with children in mind [7]. Seymour Papert’s research on computing education during childhood and the famous turtle used in 1980s Logo education programs pioneered tangible and human–robot interactions, now far more ubiquitous. More recently, the New London Group [8] proposed the concept of multiliteracies to describe how we communicate and learn through multiple modes (such as pictures, speech and visual animations) and senses, notably through digital media. Policy makers, teachers, scholars and technologists hoped that multimodal technologies would be better adapted to children and change children’s lives for the better, giving them new means of expression, play, communication and education [9].

However, the problem of how to design for the appropriate engagement of children, tweens and teens’ senses and bodies is still largely unsolved [4], while strong debates on adequate devices and technology use for families and schools remain prevalent. Often indiscriminately referred to as “screens”, technologies are amenable to any pedagogical framework from cognitivism to constructivism. Even designs that seemingly ignore the body, such as those used for sitting and described as passive, elicit active physical engagement, either through joint attention or encouraging longer periods of stillness. The significant research and design efforts in these domains will not lead to a single answer.

Another reason why companies should focus on children when designing new technologies is that fears of being overtaken by technological advances have long influenced public educational policies. There are constantly new plans and initiatives to increase digital literacy or computing skills, sometimes at great expense and with limited returns. Focusing on children and the concerns of their parents is a great way to secure new markets [10]. Historically, new media, such as radio, TV, home computers, and the internet, are even touted by advertisers to be transformational for children’s education. Yet, once technologies are widely adopted by households, children often stop being considered as the primary audience or users, and their needs or preferences are left unattended (see [6,11,12]). Children may be exposed to predatory content or have their data sold [13]. For instance, in 2021, advocacy and non-profit groups filed a complaint against the use of advertising targeted towards children that played Prodigy, an educational mathematics game promoted by schools [14]. The use of technologies in educational institutions may also reshape inequalities, depending on their access to reliable devices or support at home.

Therefore, we asked authors who contributed research to this Special Issue: How do we currently design multimodal technologies and digital media for children? What evidence and knowledge about multimodal interaction drive our research? Additionally, where should we go next?

These questions felt especially pressing during the onset of the COVID-19 crisis in the early months of 2020 when we published this call for papers. The offer of digital media and educational tools directed towards children expanded as a response. It further revealed drastic inequalities in the access and use of technologies. It also resulted in a complex and seemingly ever-expanding landscape of products for families and schools to choose from, as well as a multitude of guidelines and tools [15]. Tablets or laptops? Applications or standalone devices? Public debates often fall back on generalizations about ease of use or naturalness, framing children as innately more able to use tactile interfaces when they see far more adults interacting with tactile technologies than with laptops and are more likely to be handed smartphones than laptops. Meanwhile, a sizable share of research into multimodal and multisensory interactive products is technology-driven: as new devices expand the range of possible inputs and outputs, researchers and technologists try putting them to use. Research on children–computer interaction may contribute to artifact-centred articles that remain one-off explorations of potential designs using this technology, rather than the development of strong design concepts that are not as technology-dependent [16].

The four articles included in this Special Issue provide a wide range of responses to our call for submissions. Ragone et al. survey the literature on technologies for music therapy in autistic children. Their review of the literature demonstrates there is a place for technologies in this context, from measuring and scaffolding children’s motor skills to providing children unable to use traditional instruments with the means to play music. Yet, this is little developed in research, and even less so commercially. Björling et al. studied the design of culturally appropriate robot interactions in educational contexts. Their study involved English language learners between the ages of eight and ten and teachers in training to investigate how children interpret the movements of a humanoid robot language tutor, which could supplement in-person instructions. Pila et al. investigated the impact of tangible interaction on learning in mathematics tasks with preschoolers. They found that physical manipulation does not improve preschoolers’ comprehension of weight and balance compared with using an app (with a similar or unrelated task), despite being in line with current theories of embodied cognition and the capacity model for educational media. Finally, Du et al. analyzed communication and control features in online multiplayer games used by children and how they could be improved to be suitable for children of different ages. The authors adopted a developmental perspective on children’s communication. Younger children for instance may benefit from more features enabling visual communication. However, tweens and teenagers need to gradually gain autonomy. The authors touch upon issues of equality, using technology to better understand and support sensory development, counter-intuitive results on the impact of tangible interaction on learning gains, and the balance between protecting children and supporting their agency. In the following sections, we summarise the articles published in this Special Issue.

1. How do We Currently Design Multimodal Technologies and Digital Media for Children?

The articles in the Special Issue illustrate the range of methodological approaches to design. Some are driven by empirically testing theories of embodied cognition and educational media design (and eventually finding them inconsistent with their findings). For instance, Pila et al. built their experiment based on Frisch’s capacity model and embodied cognition theories. Others empirically explore a given technology in a certain context to derive design principles, using a prototype as a way to elicit opinions and reactions. This is the case in the investigation of culturally appropriate robot interaction written by Björling et al., who use a Nao robot as a probe.

The other two featured articles adopt a more analytical approach to discuss the implications of design. Ragone et al. combine an empirical review with an expert-led analysis of previous research to outline technology design possibilities for music therapy in autistic children. This shows the influence of theories of embodied cognition and sensory processing focused on the individual in existing research, rather than approaches focused on collaboration and coordination between actors. This focus on social interaction in development can also be found in Du et al.’s study of children’s communication abilities to evaluate design features of virtual worlds supporting communication and parental control.

The broad range of approaches forms a good illustration of the challenges of applied research, aiming to generate new technologies, triangulating between empirical studies, the design of prototypes, and their evaluation in an iterative fashion. We were also glad to see submissions that are fairly uncommon in human–computer interactions: negative results and analytical approaches to design. This also shows the difficulties arising from working across multiple perspectives of children’s (embodied) development.

2. What Evidence and Knowledge about Multimodal Interaction Drive Our Research?

A wide variety of theoretical frameworks form the basis of the articles in this Special Issue. It is worth restating there are longstanding concerns that technologies especially harm the senses of children. For instance, there are concerns about eye damage due to over-exposure to blue light from screens and that time using technology may prevent other activities and hinder motor development. This is in fact a major factor for body-based or screen-less technologies for children. This takes us back to the issue of the understanding children’s senses, development, and what may help them thrive. Sweeping statements about technology or digital media hide differences in the features of these technologies, as well as the family practices around them. For instance, the Joint Media Engagement framework emphasizes how children’s interactions with technologies and caregivers shape their understanding of content [17]. Focusing on interactions rather than children as individual users further helps to go beyond a purely developmental, cognitive and age-based approach to what constitutes appropriate design. Instead, this approach considers the variety of contexts that children encounter. Furthermore, ignoring social contexts tends to lead to research primarily focused on risks rather than opportunities [18].

Most articles in this Special Issue fall under the broad umbrella [4] of theories of embodied cognition that accurately illustrate associated tensions. suggest investigating multimodal interactions at multiple scales; Ragone et al. used micro interactional synchrony, whereas Björling et al. used full-body and group interactions. These authors investigated how factors such as socio-economic status or cultural background affect technology use, going beyond a universal model of embodied interactions at different ages. However, this approach questions how we define categories of young users associated with specific needs. We rely on educational, institutional and legal categories: the presence of a disability, their age, or their previous knowledge. However, it can be easy to forget the implications of these boundaries. Researchers working with children with disabilities, for instance, have pushed against a deficit framing, primarily or entirely focusing on perceived weaknesses rather than strengths.

One way to amend this is to examine the motivations and focus of evaluations through these articles. Björling et al. explicitly discuss technology as a way to compensate for the low level of English instruction provided for learners by a human tutor. This challenges the role of technology, or how it could justify not providing adequate funding to teaching staff. Similarly, Ragone et al. suggest that focusing on the individual use of technologies may hide opportunities for more fruitful collaborations with a therapist. Parental control features based on age in virtual worlds, which Du et al. investigate, can prevent positive interactions between older and younger minors as well [18]. Meanwhile, Pila et al.’s study pushes back on assumptions that tangible interaction are more adequate for young children, a central tenet of Montessori pedagogy. Their lack of results could also lead to questioning this type of activity at this age, although various educational policies push for structuring childcare around formal learning activities as early as possible.

3. Where Should We Go Next?

We identified the current research on multimodal technology and digital media for children within the broader history and research of educational practices, especially the concerns that technologies are not adapted to the bodies of children, tweens and teens or may harm them. We argued in this Special Issue for shifting responsibilities to avoid young people and their families from harming the larger community and moving towards promoting positive engagements. How to achieve this is a separate matter, and the articles in this Special Issue illustrate a variety of approaches to the research and design of these technologies, whether in terms of methodology, theoretical frameworks (under the broad label of embodied cognition) and categorisations of young people into relevant groups of users.

This diversity is both generative and difficult to navigate, for researchers who are working with or across frameworks that are sometimes vaguely defined and trying to address contradictory evidence. When it comes to exploring new interaction modalities, such as virtual reality, artificial touch and smell, we can learn from the long history of the senses in technologies for children. Presuming these interactions to be closer to the body, and thus more natural, is likely to obscure how we learn to make sense of them in context—a focus on incremental changes rather than hopes of large-scale disruption. Additionally, if the impacts of technologies are complex for researchers to grasp, they become more prevalent for caregivers and the many decision makers shaping technology procurement. There has recently been a wealth of regulatory initiatives—such as the UK Age-Appropriate Design Code—primarily emphasizing the protection of children, as well as mentioning the improvement of their experience. These complement design toolkits for conceptors and guidelines on use for caregivers. Meanwhile, researchers have set up collaborations with children’s media and technology producers or taken positions on companies’ policy teams to respond to new regulatory environments. There is much to gain from this focus on children’s rights, although the sustainability of further research remains to be seen in a context where technologies are both viewed suspiciously and deployed on an ever-larger scale for children.