Multimodal Technologies and Interaction

27 pages, 132085 KiB

Open AccessArticle

Sharing the Sidewalk: Observing Delivery Robot Interactions with Pedestrians during a Pilot in Pittsburgh, PA

by David Weinberg, Healy Dwyer, Sarah E. Fox and Nikolas Martelaro

Multimodal Technol. Interact. 2023, 7(5), 53; https://doi.org/10.3390/mti7050053 - 17 May 2023

Cited by 5 | Viewed by 2670

Sidewalk delivery robots are being deployed as a form of last-mile delivery. While many such robots have been deployed on college campuses, fewer have been piloted on public sidewalks. Furthermore, there have been few observational studies of robots and their interactions with pedestrians. [...] Read more.

Sidewalk delivery robots are being deployed as a form of last-mile delivery. While many such robots have been deployed on college campuses, fewer have been piloted on public sidewalks. Furthermore, there have been few observational studies of robots and their interactions with pedestrians. To better understand how sidewalk robots might integrate into public spaces, the City of Pittsburgh, Pennsylvania conducted a pilot of sidewalk delivery robots to understand possible uses and the challenges that could arise in interacting with people in the city. Our team conducted ethnographic observations and intercept interviews to understand how residents perceived of and interacted with sidewalk delivery robots over the course of the public pilot. We found that people with limited knowledge about the robots crafted stories about their purpose and function. We observed the robots causing distractions and obstructions with different sidewalk users (including children and dogs), witnessed people helping immobilized robots, and learned about potential accessibility issues that the robots may pose. Based on our findings, we contribute a set of recommendations for future pilots, as well as questions to guide future design for robots in public spaces. Full article

(This article belongs to the Special Issue Interaction Design and the Automated City – Emerging Urban Interfaces, Prototyping Approaches and Design Methods)

► Show Figures

Figure 1

30 pages, 5250 KiB

Open AccessReview

Impact of Screen Time on Children’s Development: Cognitive, Language, Physical, and Social and Emotional Domains

by Vaishnavi N. Panjeti-Madan and Prakash Ranganathan

Multimodal Technol. Interact. 2023, 7(5), 52; https://doi.org/10.3390/mti7050052 - 16 May 2023

Cited by 8 | Viewed by 38142

Abstract

Technology has become integral to children’s lives, impacting many aspects, from academic to socialization. Children of today’s generation are growing up with digital devices, such as mobile phones, iPads, computers, video games, and smart gadgets; therefore, screen time has become ubiquitous in children’s [...] Read more.

Technology has become integral to children’s lives, impacting many aspects, from academic to socialization. Children of today’s generation are growing up with digital devices, such as mobile phones, iPads, computers, video games, and smart gadgets; therefore, screen time has become ubiquitous in children’s daily routines. This paper provides a review of screen time usage and its impact in children across multiple developmental domains: cognitive, language, physical, and socio-emotional domain of children under eight years of age. The cognitive domain considers factors such as attention span and memory; language domain examines vocabulary, speech, and language development; physical domain focuses on motor development, exercise, sleep, and diet; and social-emotional domain considers relationships, self-identity, and emotional behaviors/regulation. Our findings are mixed, as there are both benefits and drawbacks in technology use, but screen time in children requires controlled observation and monitoring for sustainable improved progress across developmental domains. Specific recommendations advise that children’s screen time per day should be limited to zero minutes (min) (0–2 years), <60 min (3–5-years), and 60 min (6–8 years). Full article

(This article belongs to the Topic Youth Engagement in Social Media in the Post COVID-19 Era)

► Show Figures

Figure 1

20 pages, 5702 KiB

Open AccessArticle

Revealing Unknown Aspects: Sparking Curiosity and Engagement with a Tourist Destination through a 360-Degree Virtual Tour

by Dimitra Petousi, Akrivi Katifori, Maria Boile, Lori Kougioumtzian, Christos Lougiakis, Maria Roussou and Yannis Ioannidis

Multimodal Technol. Interact. 2023, 7(5), 51; https://doi.org/10.3390/mti7050051 - 16 May 2023

Viewed by 2150

Abstract

Over the past decades, 360-degree virtual tours have been used to provide the public access to accurate representations of cultural heritage sites and museums. The COVID-19 pandemic has contributed to a rise in the popularity of virtual tours as a means of engaging [...] Read more.

Over the past decades, 360-degree virtual tours have been used to provide the public access to accurate representations of cultural heritage sites and museums. The COVID-19 pandemic has contributed to a rise in the popularity of virtual tours as a means of engaging with locations remotely and has raised an interesting question: How could we use such experiences to bring the public closer to locations that are otherwise unreachable in real life or not considered to be tourist destinations? In this study, we examine the effectiveness of promoting engagement with a city through the virtual presentation of unknown and possibly also inaccessible points of interest through a 360-degree panoramic virtual tour. The evaluation of the experience with 31 users through an online questionnaire confirms its potential to spark curiosity, promote engagement, foster reflection, and motivate users to explore the location and its attractions at their leisure, thus enabling them to experience it from their personal point of view. The outcomes highlight the need for further research to explore this potential and identify best practices for virtual experience design. Full article

(This article belongs to the Special Issue Feature Papers in Multimodal Technologies and Interaction—Edition 2023)

► Show Figures

Figure 1

19 pages, 354 KiB

Open AccessReview

Reviewing Simulation Technology: Implications for Workplace Training

by Diana R. Sanchez, Amanda Rueda, Kentaro Kawasaki, Saar Van Lysebetten and Daniel Diaz

Multimodal Technol. Interact. 2023, 7(5), 50; https://doi.org/10.3390/mti7050050 - 10 May 2023

Viewed by 3003

Abstract

Organizations have maintained a commitment to using simulation technology for training purposes because it prepares employees for realistic work scenarios they may encounter and provides a relevant method for teaching hands-on skills. One challenge that simulation technology has faced is the persistent threat [...] Read more.

Organizations have maintained a commitment to using simulation technology for training purposes because it prepares employees for realistic work scenarios they may encounter and provides a relevant method for teaching hands-on skills. One challenge that simulation technology has faced is the persistent threat of obsolescence, where investment in an up-to-date solution can rapidly become irrelevant in a matter of months or years as technology progresses. This can be particularly challenging for organizations who seek out the best solutions to help develop and train employees while facing the constraints of limited resources and lengthy acquisition times for tools and equipment. Some industries and organizations may benefit from anticipating which technologies might best serve employees and stakeholders in the future. In this manuscript, we took a historical approach, looking at the history of training and the use of simulation-like experiences over time, which helps us identify historical themes in workplace training. Next, we carried out a systematic review of the recent training research using simulation technology to understand how these recent findings help us understand the identified historical themes. Lastly, we summarized the research literature on simulation technology used for training, and highlighted future directions and made recommendations for practitioners and researchers. Full article

► Show Figures

Graphical abstract

23 pages, 1676 KiB

Open AccessSystematic Review

Usability Assessments for Augmented Reality Head-Mounted Displays in Open Surgery and Interventional Procedures: A Systematic Review

by Ellenor J. Brown, Kyoko Fujimoto, Bennett Blumenkopf, Andrea S. Kim, Kimberly L. Kontson and Heather L. Benz

Multimodal Technol. Interact. 2023, 7(5), 49; https://doi.org/10.3390/mti7050049 - 09 May 2023

Cited by 1 | Viewed by 2739

Abstract

Augmented reality (AR) head-mounted displays (HMDs) are an increasingly popular technology. For surgical applications, the use of AR HMDs to display medical images or models may reduce invasiveness and improve task performance by enhancing understanding of the underlying anatomy. This technology may be [...] Read more.

Augmented reality (AR) head-mounted displays (HMDs) are an increasingly popular technology. For surgical applications, the use of AR HMDs to display medical images or models may reduce invasiveness and improve task performance by enhancing understanding of the underlying anatomy. This technology may be particularly beneficial in open surgeries and interventional procedures for which the use of endoscopes, microscopes, or other visualization tools is insufficient or infeasible. While the capabilities of AR HMDs are promising, their usability for surgery is not well-defined. This review identifies current trends in the literature, including device types, surgical specialties, and reporting of user demographics, and provides a description of usability assessments of AR HMDs for open surgeries and interventional procedures. Assessments applied to other extended reality technologies are included to identify additional usability assessments for consideration when assessing AR HMDs. The PubMed, Web of Science, and EMBASE databases were searched through September 2022 for relevant articles that described user studies. User assessments most often addressed task performance. However, objective measurements of cognitive, visual, and physical loads, known to affect task performance and the occurrence of adverse events, were limited. There was also incomplete reporting of user demographics. This review reveals knowledge and methodology gaps for usability of AR HMDs and demonstrates the potential impact of future usability research. Full article

(This article belongs to the Special Issue Virtual Reality and Augmented Reality)

► Show Figures

Figure 1

16 pages, 1089 KiB

Open AccessArticle

Team Success: A Mixed Methods Approach to Evaluating Virtual Team Leadership Behaviors

by Diana R. Sanchez, Amanda Rueda, Hana R. Zimman, Reese Haydon, Daniel Diaz and Kentaro Kawasaki

Multimodal Technol. Interact. 2023, 7(5), 48; https://doi.org/10.3390/mti7050048 - 05 May 2023

Viewed by 2322

Abstract

The virtuality of organizational teams have gained interest and popularity in recent years, and have become more prevalent amidst the COVID-19 pandemic. Organizational productivity and team relationship-building may suffer certain pitfalls in virtual communication and support without the understanding of the dynamics of [...] Read more.

The virtuality of organizational teams have gained interest and popularity in recent years, and have become more prevalent amidst the COVID-19 pandemic. Organizational productivity and team relationship-building may suffer certain pitfalls in virtual communication and support without the understanding of the dynamics of short-term, project-based virtual teams. The manuscript aimed to expand what is currently known about short-term virtual team dynamics related to types of effective leadership behaviors. The present study employed a mixed method approach to understanding the dynamics of these teams at both the individual and team level. Small teams were formed and instructed to collaborate on a virtual survival task. Team-related outcomes were measured at the individual level, such as team coordination, team support, and team success. Additionally, distinct latent profiles of leadership behaviors were developed and analyzed at the team level. Team support, more so than team coordination, significantly predicted team success at the individual level, with instrumental support having the strongest effect. Distinct leadership behaviors emerged in teams and were classified through a latent profile analysis, but none of the profiles were significantly related to team performance scores. Demonstrating instrumental support in short-term virtual teams may improve team success. It is important to understand that distinct leadership behaviors exist and future research should explore the impact of these leadership behaviors on other team-related outcomes. Full article

(This article belongs to the Special Issue Feature Papers in Multimodal Technologies and Interaction—Edition 2023)

► Show Figures

Figure 1

29 pages, 35261 KiB

Open AccessArticle

Identifying Strategies to Mitigate Cybersickness in Virtual Reality Induced by Flying with an Interactive Travel Interface

by Daniel Page, Robert W. Lindeman and Stephan Lukosch

Multimodal Technol. Interact. 2023, 7(5), 47; https://doi.org/10.3390/mti7050047 - 28 Apr 2023

Cited by 1 | Viewed by 2691

Abstract

As Virtual Reality (VR) technology has improved in hardware, accessibility of development and availability of applications, its interest has increased. However, the problem of Cybersickness (CS) still remains, causing uncomfortable symptoms in users. Therefore, this research seeks to identify and understand new CS [...] Read more.

As Virtual Reality (VR) technology has improved in hardware, accessibility of development and availability of applications, its interest has increased. However, the problem of Cybersickness (CS) still remains, causing uncomfortable symptoms in users. Therefore, this research seeks to identify and understand new CS mitigation strategies that can contribute to developer guidelines. Three hypotheses for strategies were devised and tested in an experiment. This involved a physical travel interface for flying through a Virtual Environment (VE) as a Control (CT) condition. On top of this, three manipulation conditions referred to as Gaze-tracking Vignette (GV), First-person Perspective with members representation (FP) and Fans and Vibration (FV) were applied. The experiment was between subjects, with 37 participants randomly allocated across conditions. According to the Simulator Sickness Questionnaire (SSQ) scores, significant evidence was found that GV and FP made CS worse. Evidence was also found that FV did not have an effect on CS. However, from the physiological data recorded, an overall lowering of heart rate for FV indicated that it might have some effect on the experience, but cannot be strongly linked with CS. Additionally, comments from some participants identified that they experienced symptoms consistent with CS. Amongst these, dizziness was the most common, with a few having issues with the usability of the travel interface. Despite some CS symptoms, most participants reported little negative impact of CS on the overall experience and feelings of immersion. Full article

(This article belongs to the Special Issue 3D User Interfaces and Virtual Reality)

► Show Figures

Figure 1

19 pages, 18635 KiB

Open AccessArticle

Forest Classroom: A Case Study of Educational Augmented Reality Design to Facilitate Classroom Engagement

by Manjeet Singh, Shaun Bangay, Henry Grossek and Atul Sajjanhar

Multimodal Technol. Interact. 2023, 7(5), 46; https://doi.org/10.3390/mti7050046 - 28 Apr 2023

Cited by 1 | Viewed by 1929

Abstract

The transition from kindergarten to primary school involves preparing students for a more structured classroom-based learning environment, which is typically different from the play-based model in kindergartens. Building on the Forest Room concept, which connects restless and disengaged students to nature as a [...] Read more.

The transition from kindergarten to primary school involves preparing students for a more structured classroom-based learning environment, which is typically different from the play-based model in kindergartens. Building on the Forest Room concept, which connects restless and disengaged students to nature as a calming medium, this case study describes the design of a combined storybook and augmented reality application to provide a literacy primer that integrates this concept. The design case study is presented relative to three frameworks that review the support for educational content, motivation and engagement mechanisms, and features of the AR application. This serves to validate the design process relative to these criteria and identifies opportunities for enhancement, including opportunities for meaningful interaction. The resulting application demonstrates appropriate design strategies to support its target age group and focus. It provides a stimulating and flexible learning activity that can be readily integrated into the classroom and that supports the kindergarten transition to appropriate classroom behaviour by encouraging active engagement and collaboration, blending aspects of both outdoor and classroom-based activities. Full article

► Show Figures

Figure 1

24 pages, 582 KiB

Open AccessReview

The Good News, the Bad News, and the Ugly Truth: A Review on the 3D Interaction of Light Field Displays

by Peter A. Kara and Aniko Simon

Multimodal Technol. Interact. 2023, 7(5), 45; https://doi.org/10.3390/mti7050045 - 27 Apr 2023

Cited by 2 | Viewed by 2470

Abstract

Light field displays offer glasses-free 3D visualization, which means that multiple individuals may observe the same content simultaneously from a virtually infinite number of perspectives without the need of viewing devices. The practical utilization of such visualization systems include various passive and active [...] Read more.

Light field displays offer glasses-free 3D visualization, which means that multiple individuals may observe the same content simultaneously from a virtually infinite number of perspectives without the need of viewing devices. The practical utilization of such visualization systems include various passive and active use cases. In the case of the latter, users often engage with the utilized system via human–computer interaction. Beyond conventional controls and interfaces, it is also possible to use advanced solutions such as motion tracking, which may seem seamless and highly convenient when paired with glasses-free 3D visualization. However, such solutions may not necessarily outperform conventional controls, and their true potentials may fundamentally depend on the use case in which they are deployed. In this paper, we provide a review on the 3D interaction of light field displays. Our work takes into consideration the different requirements posed by passive and active use cases, discusses the numerous challenges, limitations and potentials, and proposes research initiatives that could progress the investigated field of science. Full article

(This article belongs to the Special Issue 3D User Interfaces and Virtual Reality)

► Show Figures

Figure 1

31 pages, 4872 KiB

Open AccessArticle

Real-Time Flood Forecasting and Warning: A Comprehensive Approach toward HCI-Centric Mobile App Development

by Waleed Alsabhan and Basil Dudin

Multimodal Technol. Interact. 2023, 7(5), 44; https://doi.org/10.3390/mti7050044 - 26 Apr 2023

Cited by 1 | Viewed by 2475

Abstract

This article discusses the design, development, and usability assessment of a mobile system for producing hydrological predictions and sending flood warnings in response to the desire for human-centered technology to better the management of flood occurrences. Our work acts as a bibliographic reference [...] Read more.

This article discusses the design, development, and usability assessment of a mobile system for producing hydrological predictions and sending flood warnings in response to the desire for human-centered technology to better the management of flood occurrences. Our work acts as a bibliographic reference for understanding what others have attempted and found, as well as gives an integrated set of recommendations. Furthermore, our guidelines offer guidance to aid in the design of mobile GIS-based hydrological models for mobile devices. We concentrate on the full design of a human–computer interaction framework for an effective flood prediction and warning system. In addition, we analyze and address the current user needs and requirements for building a user interface for mobile real-time flood forecasting in a methodical manner. Although a functional prototype was created, the primary objective of this research was to comprehend the complexity of possible users’ demands and actual use situations in order to solve the problem of comparable systems being difficult to use. After consulting with possible consumers, application design standards were established and implemented in the initial prototype. Focusing on user demands and attitudes, special consideration was given to the usability of the mobile interface. To develop the application, a variety of assessment methods are added. The conclusion of the examination was that the system is efficient and effective. Full article

► Show Figures

Figure 1

25 pages, 3830 KiB

Open AccessArticle

Building Community Resiliency through Immersive Communal Extended Reality (CXR)

by Sharon Yavo-Ayalon, Swapna Joshi, Yuzhen (Adam) Zhang, Ruixiang (Albert) Han, Narges Mahyar and Wendy Ju

Multimodal Technol. Interact. 2023, 7(5), 43; https://doi.org/10.3390/mti7050043 - 26 Apr 2023

Cited by 1 | Viewed by 2768

Abstract

Situated and shared experiences can motivate community members to plan shared action, promoting community engagement. We deployed and evaluated a communal extended-reality (CXR) bus tour that depicts the possible impacts of flooding and climate change. This paper describes the results of seven community [...] Read more.

Situated and shared experiences can motivate community members to plan shared action, promoting community engagement. We deployed and evaluated a communal extended-reality (CXR) bus tour that depicts the possible impacts of flooding and climate change. This paper describes the results of seven community engagement sessions with a total of N = 74 members of the Roosevelt Island community. We conducted pre- and post-bus tour focus groups to understand how the tour affected these community members’ awareness and motivation to take action. We found that the unique qualities of immersive, situated, and geo-located virtual reality (VR) on a bus made climate change feel real, brought the consequences of climate change closer to home, and highlighted existing community resources to address the issue. Our results showed that the CXR experience helped to simulate a physical emergency state, which empowered the community to translate feelings of hopelessness into creative and actionable ideas. Our finding exemplifies that geo-located VR on a bus can be a powerful tool to motivate innovations and collective action. Our work is a first-of-its-kind empirical contribution showing that CXR experiences can inspire action. It offers a proof-of-concept of a large-scale community engagement process featuring simulated communal experiences, leading to creative ideas for a bottom-up community resiliency plan. Full article

(This article belongs to the Special Issue Interaction Design and the Automated City – Emerging Urban Interfaces, Prototyping Approaches and Design Methods)

► Show Figures

Figure 1

18 pages, 3654 KiB

Open AccessArticle

Service Selection Using an Ensemble Meta-Learning Classifier for Students with Disabilities

by Abdallah Namoun, Mohammad Ali Humayun, Oussama BenRhouma, Burhan Rashid Hussein, Ali Tufail, Abdullah Alshanqiti and Waqas Nawaz

Multimodal Technol. Interact. 2023, 7(5), 42; https://doi.org/10.3390/mti7050042 - 23 Apr 2023

Cited by 1 | Viewed by 1336

Abstract

Students with special needs should be empowered to use assistive technologies and services that suit their individual circumstances and environments to maximize their learning attainment. Fortunately, modern distributed computing paradigms, such as the Internet of Things (IoT), cloud computing, and mobile computing, provide [...] Read more.

Students with special needs should be empowered to use assistive technologies and services that suit their individual circumstances and environments to maximize their learning attainment. Fortunately, modern distributed computing paradigms, such as the Internet of Things (IoT), cloud computing, and mobile computing, provide ample opportunities to create and offer a multitude of digital assistive services and devices for people with disabilities. However, choosing the appropriate services from a pool of competing services while satisfying the unique requirements of disabled learners remains a challenging research endeavor. In this article, we propose an ensemble meta-learning model that ranks and selects the best IoT services while considering the diverse needs of disabled students within the educational context. We train and test our deep ensemble meta-learning model using two synthetically generated assistive services datasets. The first dataset incorporates 50,000 records representing the possible use of 12 learning activities, fulfilled by 60 distinct assistive services. The second dataset includes a range of 120,000 service ratings of seven quality features, including response, availability, successibility, latency, cost, quality of service, and accessibility. Our deep learning model uses an ensemble of multiple input learners fused using a meta-classification network shared by all the outputs representing individual assistive services. The model achieves significantly better results than traditional machine learning models (i.e., support vector machine and random forest) and a simple feed-forward neural network model without the ensemble technique. Furthermore, we extended our model to utilize the accessibility rating of services to suggest appropriate educational services for disabled learners. The empirical results show the acceptability of our assistive service recommender for learners with disabilities. Full article

► Show Figures

Figure 1

Journal Menu

Journal Browser

Multimodal Technol. Interact., Volume 7, Issue 5 (May 2023) – 12 articles

Further Information

Guidelines

MDPI Initiatives

Follow MDPI