Perception and Interaction in Mixed, Augmented, and Virtual Reality

A special issue of Electronics (ISSN 2079-9292). This special issue belongs to the section "Artificial Intelligence".

Deadline for manuscript submissions: 20 May 2024 | Viewed by 13655

Special Issue Editors

Institute of Mechatronics and Information Systems, Lodz University of Technology, 90-924 Lodz, Poland
Interests: VR/AR/web app developer; biomedical signal measurement and processing; UX designer; usability testing; 2D/3D graphics and animations
Institute of Mechatronics and Information Systems, Lodz University of Technology, 90-924 Lodz, Poland
Interests: human behavior analysis; affective computing; universal design
Special Issues, Collections and Topics in MDPI journals

Special Issue Information

Dear Colleagues,

It is a fact that digital giants are turning towards highly immersive technologies such as VR, AR and XR. The current intensive development of technological solutions and the awakened expectations of social media users are causing researchers to intensify their research around broadly understood immersive technologies. High-immersion technologies have long since found applications beyond the pure entertainment industry with researchers seeking ever-newer applications in engineering, medicine, management, education, and other domains. Standard visual and auditory solutions are increasingly being readied to interact with other human senses to intensify immersion processes. We are witnessing considerable interest in the cognitive science of immersive systems and revolutionary changes in the field of building human–system interaction. The development of modern UI/UX based on sophisticated solutions in the field of AI means that human–system communication will soon resemble a conversation about the tasks being performed.

The goal of this Special Edition is to publish works that are of interest to researchers, employees, and IT enthusiasts alike. The issue has the potential to bring together the essence of current scientific activities in the field of high-immersion VR, AR, and XR technology development. We hope that together, we can create a reference point for all current development trends in the exciting domain of applied computing.

Dr. Grzegorz Zwoliński
Dr. Dorota Kamińska
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Electronics is an international peer-reviewed open access semimonthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2400 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • virtual reality
  • augmented reality
  • mixed reality
  • immersive technologies
  • perception
  • interaction
  • cognitive research

Published Papers (11 papers)

Order results
Result details
Select all
Export citation of selected articles as:

Research

Jump to: Review, Other

25 pages, 4974 KiB  
Article
Augmented Reality in Industry 4.0 Assistance and Training Areas: A Systematic Literature Review and Bibliometric Analysis
by Ginés Morales Méndez and Francisco del Cerro Velázquez
Electronics 2024, 13(6), 1147; https://doi.org/10.3390/electronics13061147 - 21 Mar 2024
Viewed by 408
Abstract
Augmented reality (AR) technology is making a strong appearance on the industrial landscape, driven by significant advances in technological tools and developments. Its application in areas such as training and assistance has attracted the attention of the research community, which sees AR as [...] Read more.
Augmented reality (AR) technology is making a strong appearance on the industrial landscape, driven by significant advances in technological tools and developments. Its application in areas such as training and assistance has attracted the attention of the research community, which sees AR as an opportunity to provide operators with a more visual, immersive and interactive environment. This article deals with an analysis of the integration of AR in the context of the fourth industrial revolution, commonly referred to as Industry 4.0. Starting with a systematic review, 60 relevant studies were identified from the Scopus and Web of Science databases. These findings were used to build bibliometric networks, providing a broad perspective on AR applications in training and assistance in the context of Industry 4.0. The article presents the current landscape, existing challenges and future directions of AR research applied to industrial training and assistance based on a systematic literature review and citation network analysis. The findings highlight a growing trend in AR research, with a particular focus on addressing and overcoming the challenges associated with its implementation in complex industrial environments. Full article
(This article belongs to the Special Issue Perception and Interaction in Mixed, Augmented, and Virtual Reality)
Show Figures

Figure 1

14 pages, 9088 KiB  
Article
A Framework Integrating Augmented Reality and Wearable Sensors for the Autonomous Execution of Rehabilitation Exercises
by Jacopo Rizzi, Andrea D’Antona, Antonino Proto, Giovanni Piva, Nicola Lamberti, Marcello Bonfè and Saverio Farsoni
Electronics 2023, 12(24), 4958; https://doi.org/10.3390/electronics12244958 - 10 Dec 2023
Viewed by 839
Abstract
Despite the resolution of pathology at admission, many hospitalized patients are discharged in a worse functional state, particularly in terms of walking capabilities, due to hospitalization inactivity. Early interventions to maintain the functional state through exercise such as passive mobilization, executed during hospitalization, [...] Read more.
Despite the resolution of pathology at admission, many hospitalized patients are discharged in a worse functional state, particularly in terms of walking capabilities, due to hospitalization inactivity. Early interventions to maintain the functional state through exercise such as passive mobilization, executed during hospitalization, have been proven to be effective in preventing physical and cognitive decline. Unfortunately, many patients can be excluded from such treatments due to the high number of people hospitalized and the limited availability of healthcare personnel. This paper presents a framework that allows the patient to autonomously perform the exercises without the assistance of qualified personnel. Such a framework exploits the advantages of an Augmented Reality (AR) device in which the patient can visualize real-time instructions on the exercises and directions on their execution. The monitoring of the patient’s movements during exercise is accomplished by means of dedicated wearable sensors fixed to the patient’s limb. The system was tested in preliminary experiments on healthy people. Full article
(This article belongs to the Special Issue Perception and Interaction in Mixed, Augmented, and Virtual Reality)
Show Figures

Figure 1

16 pages, 8887 KiB  
Article
Enhancing Wayfinding Experience in Low-Vision Individuals through a Tailored Mobile Guidance Interface
by Taewoo Jo, Dohyeon Yeo and SeungJun Kim
Electronics 2023, 12(22), 4561; https://doi.org/10.3390/electronics12224561 - 07 Nov 2023
Viewed by 808
Abstract
Individuals with low vision (LV) face daily wayfinding challenges, struggling with route establishment, direction recognition, and obstacle avoidance. Mobile navigation, though commonly relied upon, often clashes with LV visual conditions, particularly central scotomas from diseases like age-related macular degeneration (AMD), hindering the relay [...] Read more.
Individuals with low vision (LV) face daily wayfinding challenges, struggling with route establishment, direction recognition, and obstacle avoidance. Mobile navigation, though commonly relied upon, often clashes with LV visual conditions, particularly central scotomas from diseases like age-related macular degeneration (AMD), hindering the relay of crucial wayfinding information. Addressing this, we introduced a novel guidance interface post-literature review. In a VR-based user study involving 25 participants simulating LV experience, our map guidance interface enabled quicker navigation, improving system usability and presence. The resulting performance paralleled that of normal-vision individuals. This study demonstrated that modifying familiar navigation interfaces effectively addresses visual impairment conflicts, enhancing outdoor navigation for individuals with LV. The modifications extend benefits, enabling comparable wayfinding ease to individuals with normal vision. Full article
(This article belongs to the Special Issue Perception and Interaction in Mixed, Augmented, and Virtual Reality)
Show Figures

Figure 1

12 pages, 4255 KiB  
Article
Exploring the Perception of the Effect of Three-Dimensional Interaction Feedback Types on Immersive Virtual Reality Education
by Kwang-Seong Shin, Chungyeon Cho, Ji Hyun Ryu and Dongsik Jo
Electronics 2023, 12(21), 4414; https://doi.org/10.3390/electronics12214414 - 26 Oct 2023
Viewed by 889
Abstract
Immersive virtual reality (VR) systems are becoming widely used for education with three-dimensional (3D) information. Specifically, three-dimensional spaces to create virtual environments can help increase students’ learning interest and ability with spatial interaction. Also, with the use of multimodal interaction, VR systems can [...] Read more.
Immersive virtual reality (VR) systems are becoming widely used for education with three-dimensional (3D) information. Specifically, three-dimensional spaces to create virtual environments can help increase students’ learning interest and ability with spatial interaction. Also, with the use of multimodal interaction, VR systems can provide highly effective ways to solve problems through natural experiences. Additionally, immersive environments can bring together people in remote locations, which has been increasingly applied in education applications with the use of technology to simulate real situations. However, effective interaction methods that improve the learning ability of people participating in educational activities in 3D immersive environments are yet to be well defined. In this study, we investigated the effect of the interaction feedback types on the perception of students participating in VR environments. We conducted the experiment on three types of interaction responses, and our study was designed as a virtual chemistry class. Our experimental study showed that the interaction feedback type of a deformable object had a greater educational effect than other types of visual or audio feedback, and our results are expected to provide guidelines on how to create effective immersive education content and interaction methods. Full article
(This article belongs to the Special Issue Perception and Interaction in Mixed, Augmented, and Virtual Reality)
Show Figures

Figure 1

22 pages, 17347 KiB  
Article
How Do Background and Remote User Representations Affect Social Telepresence in Remote Collaboration?: A Study with Portal Display, a Head Pose-Responsive Video Teleconferencing System
by Seongjun Kang, Gwangbin Kim, Kyung-Taek Lee and SeungJun Kim
Electronics 2023, 12(20), 4339; https://doi.org/10.3390/electronics12204339 - 19 Oct 2023
Viewed by 788
Abstract
This study presents Portal Display, a screen-based telepresence system that mediates the interaction between two distinct spaces, each using a single display system. The system synchronizes the users’ viewpoint with their head position and orientation to provide stereoscopic vision through this single monitor. [...] Read more.
This study presents Portal Display, a screen-based telepresence system that mediates the interaction between two distinct spaces, each using a single display system. The system synchronizes the users’ viewpoint with their head position and orientation to provide stereoscopic vision through this single monitor. This research evaluates the impact of graphically rendered and video-streamed backgrounds and remote user representations on social telepresence, usability, and concentration during conversations and collaborative tasks. Our results indicate that the type of background has a negligible impact on these metrics. However, point cloud streaming of remote users significantly improves social telepresence, usability, and concentration compared with graphical avatars. This study implies that Portal Display can operate more efficiently by substituting the background with graphical rendering and focusing on higher-resolution 3D point cloud streaming for narrower regions for remote user representations. This configuration may be especially advantageous for applications where the remote user’s background is not essential to the task, potentially enhancing social telepresence. Full article
(This article belongs to the Special Issue Perception and Interaction in Mixed, Augmented, and Virtual Reality)
Show Figures

Figure 1

12 pages, 7248 KiB  
Article
Optimal Camera Placement to Generate 3D Reconstruction of a Mixed-Reality Human in Real Environments
by Juhwan Kim and Dongsik Jo
Electronics 2023, 12(20), 4244; https://doi.org/10.3390/electronics12204244 - 13 Oct 2023
Viewed by 1116
Abstract
Virtual reality and augmented reality are increasingly used for immersive engagement by utilizing information from real environments. In particular, three-dimensional model data, which is the basis for creating virtual places, can be manually developed using commercial modeling toolkits, but with the advancement of [...] Read more.
Virtual reality and augmented reality are increasingly used for immersive engagement by utilizing information from real environments. In particular, three-dimensional model data, which is the basis for creating virtual places, can be manually developed using commercial modeling toolkits, but with the advancement of sensing technology, computer vision technology can also be used to create virtual environments. Specifically, a 3D reconstruction approach can generate a single 3D model from image information obtained from various scenes in real environments using several cameras (multi-cameras). The goal is to generate a 3D model with excellent precision. However, the rules for choosing the optimal number of cameras and settings to capture information from in real environments (e.g., actual people) employing several cameras in unconventional positions are lacking. In this study, we propose an optimal camera placement strategy for acquiring high-quality 3D data using an irregular camera placement, essential for organizing image information while acquiring human data in a three-dimensional real space, using multiple irregular cameras in real environments. Our results show that installation costs can be lowered by arranging a minimum number of multi-camera cameras in an arbitrary space, and automated virtual human manufacturing with high accuracy can be conducted using optimal irregular camera location. Full article
(This article belongs to the Special Issue Perception and Interaction in Mixed, Augmented, and Virtual Reality)
Show Figures

Figure 1

17 pages, 3067 KiB  
Article
Virtual Reality Interface Evaluation for Earthwork Teleoperation
by Léa Saunier, Nicolas Hoffmann, Marius Preda and Catalin Fetita
Electronics 2023, 12(19), 4151; https://doi.org/10.3390/electronics12194151 - 06 Oct 2023
Viewed by 714
Abstract
Automation and robotics are destined to play a critical role in the Industry 4.0 revolution, as illustrated by the emergence of autonomous machinery in earthwork operations. Despite rapid progress, autonomous agents will always require human supervision to instruct their mission and to guaranty [...] Read more.
Automation and robotics are destined to play a critical role in the Industry 4.0 revolution, as illustrated by the emergence of autonomous machinery in earthwork operations. Despite rapid progress, autonomous agents will always require human supervision to instruct their mission and to guaranty safety when unexpected problems arise. Traditional human supervision requires an operator to physically enter each machine at risk and manually take control. This approach is time-consuming and requires highly qualified personnel capable of operating various machines. This process can be hastened and simplified by means of teleoperated supervision, which itself requires the appropriate interface. In this paper we evaluate a virtual reality (VR)-based interface using hybrid interactions and an immersive digital-twin compared to a real-life control. We compare these interfaces through control tasks performed by expert and non-expert operators, analyzing time and precision, as well as user feedback. The preliminary results show that the VR interface brings equivalent and satisfactory performances for experts and improves the efficiency of apprentices. Therefore, not only does everyone performs well in the virtual environment, but also the training time can be shortened significantly as non-experts can perform similarly under the same conditions. Full article
(This article belongs to the Special Issue Perception and Interaction in Mixed, Augmented, and Virtual Reality)
Show Figures

Figure 1

13 pages, 7370 KiB  
Article
Recognition of Human Mental Stress Using Machine Learning: A Case Study on Refugees
by Dorota Kamińska
Electronics 2023, 12(16), 3468; https://doi.org/10.3390/electronics12163468 - 16 Aug 2023
Viewed by 992
Abstract
This paper introduces a study on stress recognition utilizing mobile EEG and GSR sensors. The research involved collecting samples from a group of 55 refugees who participated in Virtual Reality stress-reduction sessions. The timing of the study coincided with an influx of refugees, [...] Read more.
This paper introduces a study on stress recognition utilizing mobile EEG and GSR sensors. The research involved collecting samples from a group of 55 refugees who participated in Virtual Reality stress-reduction sessions. The timing of the study coincided with an influx of refugees, prompting the development of software specifically designed to alleviate acute stress among them. The paper focuses on presenting an EEG/GSR signals pipeline for classifying stress levels, emphasizing selecting the most informative features. The classification process employed popular machine learning methods, yielding results of 86.7% for two-stress-level classification and 82.3% and 67.7% for the three- and five-level classifications, respectively. Most importantly, the positive impact of the system has been proven by subjective assessment in alignment with objective features analysis. Such a system has not yet reached the level of autonomy, but it can be a valuable support tool for mental health professionals. Full article
(This article belongs to the Special Issue Perception and Interaction in Mixed, Augmented, and Virtual Reality)
Show Figures

Figure 1

19 pages, 4197 KiB  
Article
Consumer Subjective Impressions in Virtual Reality Environments: The Role of the Visualization Technique in Product Evaluation
by Almudena Palacios-Ibáñez, Francisco Felip-Miralles, Julia Galán, Carlos García-García and Manuel Contero
Electronics 2023, 12(14), 3051; https://doi.org/10.3390/electronics12143051 - 12 Jul 2023
Viewed by 711
Abstract
The availability and affordability of consumer virtual reality (VR) devices have fueled their adoption during the product design process. High fidelity virtual prototypes can be created more quickly and are more cost-effective than using traditional methods, but certain product features are still difficult [...] Read more.
The availability and affordability of consumer virtual reality (VR) devices have fueled their adoption during the product design process. High fidelity virtual prototypes can be created more quickly and are more cost-effective than using traditional methods, but certain product features are still difficult to evaluate, resulting in perceptual differences when a product is assessed using different visualization techniques. In this paper, we report two case studies in which a group of participants evaluated different designs of a product typology (i.e., a watering can) as presented in VR, VR with passive haptics (VRPH) and in a real setting (R) for the first case study, and VR and R for the second case study. The semantic differential technique was used for product evaluation, and an inferential statistical method using aligned rank transform (ART) proceedings was applied to determine perceptual differences between groups. Our results showed that product characteristics assessed by touch are the most susceptible to being affected by the environment, while the user background can have an effect in some product features. Full article
(This article belongs to the Special Issue Perception and Interaction in Mixed, Augmented, and Virtual Reality)
Show Figures

Figure 1

Review

Jump to: Research, Other

32 pages, 16646 KiB  
Review
Augmented Reality: Current and New Trends in Education
by Dorota Kamińska, Grzegorz Zwoliński, Anna Laska-Leśniewicz, Rui Raposo, Mário Vairinhos, Elisabeth Pereira, Frane Urem, Martina Ljubić Hinić, Rain Eric Haamer and Gholamreza Anbarjafari
Electronics 2023, 12(16), 3531; https://doi.org/10.3390/electronics12163531 - 21 Aug 2023
Cited by 2 | Viewed by 4069
Abstract
The educational landscape is an environment prone to change due to the volatile and ever-changing nature of the digital society in which we all live. Although the world moves at different speeds and any generalization is bound to have some exceptions, there is [...] Read more.
The educational landscape is an environment prone to change due to the volatile and ever-changing nature of the digital society in which we all live. Although the world moves at different speeds and any generalization is bound to have some exceptions, there is evidence from research conducted in different places and contexts that educational methods are becoming increasingly digitized and driven by technological innovation. Among the technological trends fueled in many cases by the COVID-19 pandemic and the need to stay at home but online, augmented reality solutions received an additional boost as a valid and versatile educational technology worth exploring and eventually integrating into several teaching methods already in use. Although the technology still faces problems related to affordability, accessibility, and the technical skills required of users, some ongoing projects have already provided evidence that using augmented reality solutions as teaching and learning tools can improve teacher and student learning outcomes by increasing engagement and interactivity. The same issues arose when personal computers, tablets, and smartphones were first discussed as valuable tools for education and have now found their way into most classrooms. This paper reviews some of the key concepts related to augmented reality, as well as some current trends, benefits, and concerns related to its integration into educational contexts in areas such as life sciences, engineering, and health. The work conducted and presented in this paper provides an interesting insight into a technology that has given rise to global phenomena such as Pokémon Go, and continues to improve in terms of portability, usability, and overall user experience. Throughout the paper and in the conclusion section, we discuss the relevance of using the best features of augmented reality and how they can contribute to positive educational outcomes. Full article
(This article belongs to the Special Issue Perception and Interaction in Mixed, Augmented, and Virtual Reality)
Show Figures

Figure 1

Other

Jump to: Research, Review

19 pages, 734 KiB  
Systematic Review
Addressing Body Image Disturbance through Metaverse-Related Technologies: A Systematic Review
by Moises Perez, Adriana Pineda-Rafols, Maria Pilar Egea-Romero, Maria Gonzalez-Moreno and Esther Rincon
Electronics 2023, 12(22), 4580; https://doi.org/10.3390/electronics12224580 - 09 Nov 2023
Viewed by 883
Abstract
There is an increasing awareness about body image disturbance and eating disorders which calls for a multidisciplinary approach. The usefulness of new technologies for improving body image distortion has been addressed in the scientific literature, but has not included current strategies such as [...] Read more.
There is an increasing awareness about body image disturbance and eating disorders which calls for a multidisciplinary approach. The usefulness of new technologies for improving body image distortion has been addressed in the scientific literature, but has not included current strategies such as Metaverse-related technologies. Seemingly, this is the first systematic review which focuses on the efficacy of Metaverse-related technologies in reducing body image disturbance related to eating disorders like Anorexia nervosa and Bulimia nervosa. The main objective of this study was to review the scientific studies published in the last decade to answer the following three questions: (1) Are Metaverse-related technologies useful in mitigating body image disturbance in patients diagnosed with Anorexia nervosa and Bulimia nervosa? (2) What are the advantages and disadvantages? (3) Are the patients satisfied after using this kind of technology? The results obtained were that 80% of the included studies found metaverse-related technologies useful for the improvement in body image disturbance, in addition to various advantages, such as a decrease in eating disorder symptomatology. Whereas patient satisfaction was only evaluated in 20% of the included studies, with the majority of patients evaluating the use of metaverse-related technologies positively. We will conduct a systematic review of the peer-reviewed literature from EBSCO Discovery Service, and Web of Science (WOS), following the PRISMA statements. Only Journal articles published from 2013 to the present, written in the English language, will be reviewed. The findings are expected to offer valuable insights for the development of novel approaches for the improvement in body image disturbance in healthcare settings. Full article
(This article belongs to the Special Issue Perception and Interaction in Mixed, Augmented, and Virtual Reality)
Show Figures

Figure 1

Back to TopTop