Virtual Worlds

One of the most critical challenges for the success of the Metaverse is interoperability amongst its virtual platforms and worlds. In this context, application programming interfaces (APIs) are essential. This study analyzes a sample of 15 Metaverse platforms. In the first step, the availability of publicly accessible APIs was examined. For those platforms offering an API, i.e., Decentraland, Second Life, Voxels, Roblox, Axie Infinity, Upland, and VRChat, the available API contents were collected, analyzed, and presented in the paper. The results show that only a few Metaverse platforms offer APIs at all. In addition, the available APIs are very diverse and heterogeneous. Information is somewhat fragmented, requiring access to several APIs to compile a comprehensive data set. Thus, standardized APIs will enable better interoperability and foster a more seamless and immersive user experience in the Metaverse. Full article

Motion capture is a fundamental technique in the development of video games and in film production to animate a virtual character based on the movements of an actor, creating more realistic animations in a short amount of time. One of the ways to obtain this movement from an actor is to capture the motion of the player through an optical sensor to interact with the virtual world. However, during movement some parts of the human body can be occluded by others and there can be noise caused by difficulties in sensor capture, reducing the user experience. This work presents a solution to correct the motion capture errors from the Microsoft Kinect sensor or similar through a deep neural network (DNN) trained with a pre-processed dataset of poses offered by Carnegie Mellon University (CMU) Graphics Lab. A temporal filter is implemented to smooth the movement, given by a set of poses returned by the deep neural network. This system is implemented in Python with the TensorFlow application programming interface (API), which supports the machine learning techniques and the Unity game engine to visualize and interact with the obtained skeletons. The results are evaluated using the mean absolute error (MAE) metric where ground truth is available and with the feedback of 12 participants through a questionnaire for the Kinect data. Full article

Diminished reality (DR) is a technology in which a background image is overwritten on a real object to make it appear as if the object has been removed from real space. This paper presents a real-time DR application that employs deep learning. A DR application can remove objects inside a 3D region defined by a user in images captured using a smartphone. By specifying the 3D region containing the target object to be removed, DR can be realized for targets with various shapes and sizes, and the specified target can be removed even if the viewpoint changes. To achieve fast and accurate DR, a suitable network was employed based on the experimental results. Additionally, the loss function during the training process was improved to enhance completion accuracy. Then, the operation of the DR application at 10 fps was verified using a smartphone and a laptop computer. Full article

In this contribution, we propose a hybrid interaction technique that integrates near-field and object-space interaction techniques for manipulating objects at a distance in virtual reality (VR). The objective of the hybrid interaction technique was to seamlessly leverage the strengths of both the near-field and object-space manipulation techniques. We employed bimanual near-field metaphor with scaled replica (BMSR) as our near-field interaction technique, which enabled us to perform multilevel degrees-of-freedom (DoF) separation transformations, such as 1~3DoF translation, 1~3DoF uniform and anchored scaling, 1DoF and 3DoF rotation, and 6DoF simultaneous translation and rotation, with enhanced depth perception and fine motor control provided by near-field manipulation techniques. The object-space interaction technique we utilized was the classic Scaled HOMER, which is known to be effective and appropriate for coarse transformations in distant object manipulation. In a repeated measures within-subjects evaluation, we empirically evaluated the three interaction techniques for their accuracy, efficiency, and economy of movement in pick-and-place, docking, and tunneling tasks in VR. Our findings revealed that the near-field BMSR technique outperformed the object space Scaled HOMER technique in terms of accuracy and economy of movement, but the participants performed more slowly overall with BMSR. Additionally, our results revealed that the participants preferred to use the hybrid interaction technique, as it allowed them to switch and transition seamlessly between the constituent BMSR and Scaled HOMER interaction techniques, depending on the level of accuracy, precision and efficiency required. Full article

Background: Given that VR is used in multiple domains, understanding the effects of cybersickness on human cognition and motor skills and the factors contributing to cybersickness is becoming increasing important. This study aimed to explore the predictors of cybersickness and its interplay with cognitive and motor skills. Methods: 30 participants, 20–45 years old, completed the MSSQ and the CSQ-VR, and were immersed in VR. During immersion, they were exposed to a roller coaster ride. Before and after the ride, participants responded to the CSQ-VR and performed VR-based cognitive and psychomotor tasks. After the VR session, participants completed the CSQ-VR again. Results: Motion sickness susceptibility, during adulthood, was the most prominent predictor of cybersickness. Pupil dilation emerged as a significant predictor of cybersickness. Experience with videogaming was a significant predictor of cybersickness and cognitive/motor functions. Cybersickness negatively affected visuospatial working memory and psychomotor skills. Overall the intensity of cybersickness’s nausea and vestibular symptoms significantly decreased after removing the VR headset. Conclusions: In order of importance, motion sickness susceptibility and gaming experience are significant predictors of cybersickness. Pupil dilation appears to be a cybersickness biomarker. Cybersickness affects visuospatial working memory and psychomotor skills. Concerning user experience, cybersickness and its effects on performance should be examined during and not after immersion. Full article

3D audio spatializers for Virtual Reality (VR) can use the acoustic properties of the surfaces of a visualised game space to calculate a matching reverb. However, this approach could lead to reverbs that impair the tasks performed in such a space, such as listening to speech-based audio. Sound designers would then have to alter the room’s acoustic properties independently of its visualisation to improve speech intelligibility, causing audio-visual incongruency. As user expectation of simulated room acoustics regarding speech intelligibility in VR has not been studied, this study asked participants to rate the congruency of reverbs and their visualisations in 6-DoF VR while listening to speech-based audio. The participants compared unaltered, matching reverbs with sound-designed, mismatching reverbs. The latter feature improved D50s and reduced RT60s at the cost of lower audio-visual congruency. Results suggest participants preferred improved reverbs only when the unaltered reverbs had comparatively low D50s or excessive ringing. Otherwise, too dry or too reverberant reverbs were disliked. The range of expected RT60s depended on the surface visualisation. Differences in timbre between the reverbs may not affect preferences as strongly as shorter RT60s. Therefore, sound designers can intervene and prioritise speech intelligibility over audio-visual congruency in acoustically challenging game spaces. Full article

Designing virtual characters that are capable of reflecting a sense of personality is a key goal in research and applications in virtual reality and computer graphics. More and more research efforts are dedicated to investigating approaches to construct a diverse, equitable, and inclusive metaverse by infusing expressive personalities and styles into virtual avatars. While most previous work focused on exploring variations in virtual characters’ dynamic behaviors, characters’ visual appearance plays a crucial role in affecting their perceived personalities. This paper presents a series of experiments evaluating the effect of virtual characters’ outfits on their perceived personality. Based on the related psychology research conducted in the real world, we determined a set of outfit factors likely to reflect personality in virtual characters: color, design, and type. As a framework for our study, we used the “Big Five” personality model for evaluating personality traits. To test our hypothesis, we conducted three perceptual experiments to evaluate the outfit parameters’ contributions to the characters’ personality. In our first experiment, we studied the color factor by varying color hue, saturation, and value; in the second experiment, we evaluated the impact of different neckline, waistline, and sleeve designs; and in our third experiment, we examined the personality perception of five outfit types: professional, casual, fashionable, outdoor, and indoor. Significant results offer guidance to avatar designers on how to create virtual characters with specific personality profiles. We further conducted a verification test to extend the application of our findings to animated virtual characters in augmented reality (AR) and virtual reality (VR) settings. Results confirmed that our findings can be broadly applied to both static and animated virtual characters in VR and AR environments that are commonly used in games, entertainment, and social networking scenarios. Full article

Design discussion is crucial in the architectural design process. To enhance the spatial understanding of 3D space and discussion effectiveness, recently, some systems have been proposed to support design discussion interactively in an immersive virtual environment. The entire design discussion can be archived and potentially become course materials for future learners. In this paper, we propose an asynchronous VR exploration system that aims to help learners explore content effectively and efficiently anywhere and at any time. To improve effectiveness and efficiency, we also propose a summarization-to-detail approach with the application space by which students can observe the visualization of spatial summarization of actions and participants’ dwell time or the temporal distribution of dialogues and then locate the important or interesting region or dialogue for further exploration. To further explore the discussion content, students can call the preview to see the time-lapse animation of the object operation to understand the change in models or playback to view the discussion details. We conducted an exploratory user study with 10 participants to evaluate user experience, user impression, and effectiveness of learning the design discussion course content using our asynchronous VR design discussion content exploration system. The results indicate that the interactive VR exploration system presented can help learners study the design discussion content effectively. Participants also provided some positive feedback and confirmed the usefulness and value of the system presented. Our applications and lessons learned have implications for future asynchronous VR exploration systems, not only for architectural design discussion content, but also for other applications, such as industrial visual inspections and educational visualizations of design discussions. Full article

In this paper, watercraft and ship simulation is summarized, and the way that it can be extended through realistic physics is explored. A hydrodynamic, data-driven, immersive watercraft simulation experience is also introduced, using the Unreal Engine to visualize a Landing Craft Utility (LCU) operation and interaction with near-shore waves in virtual reality (VR). The VR application provides navigation scientists with a better understanding of how coastal waves impact landing operations and channel design. FUNWAVE data generated on the supercomputing resources at the U.S. Army Corps of Engineers (USACE) Engineering Research and Development Center (ERDC) are employed, and using these data, a graphical representation of the domain is created, including the vessel model and a customizable VR bridge to control the vessel within the virtual environment. Several dimension reduction methods are being devised to ensure that the FUNWAVE data can inform the model but keep the application running in real time at an acceptable frame rate for the VR headset. By importing millions of data points output from the FUNWAVE version 3.4 software into Unreal Engine, virtual vessels can be affected by physics-driven data. Full article

Immersive Virtual Reality (IVR) is a promising tool for improving the teaching and learning of nursing and midwifery students. However, the preexisting literature does not comprehensively examine scenario development, theoretical underpinnings, duration, and debriefing techniques. The aim of this review was to assess the available evidence of how 360-degree Virtual Reality (VR) utilising head-mounted devices has been used in undergraduate nursing and midwifery education programmes and to explore the potential pedagogical value based on Kirkpatrick’s evaluation model. This review followed the Joanna Briggs Institute (JBI) methodology. A comprehensive electronic search was conducted across five databases. All studies published in English between 2007–2022 were included, regardless of design, if the focus was undergraduate nursing and midwifery programmes and utilised fully immersive 360-degree VR scenarios. Out of an initial pool of 1700 articles, 26 were selected for final inclusion. The findings indicated a limited diversity in scenario design, with only one study employing a participatory approach. Within the Kirkpatrick model, the most measurable outcomes were found at level 2. The main drawback observed in interventional studies was the absence of a theoretical framework and debriefing. The review concludes that the increased use of fully IVR in nursing education has improved student learning outcomes; however, published literature on midwifery education is scarce. Full article

This study aims to identify and analyze existing gaps in the integration of immersive approaches for collaborative processes with Building Information Modeling (BIM) in the Architecture, Engineering, and Construction (AEC) sector. Using a systematic approach that includes metadata analysis and review procedures, we have formulated specific research questions aimed at guiding future investigations into these gaps. Additionally, the analysis generates insights that could guide future research directions and improvements in the field. The methodology involves a comprehensive review of the literature, focusing on the interaction between immersiveness, BIM methodology, and collaborative processes. Data from 2010 to 2023 have been analyzed to ensure relevance and completeness. Our findings reveal current limitations in the field, such as the need for fully integrated prototypes and the execution of empirical studies to clarify operational processes. These limitations serve as the basis for our research questions. The study offers actionable insights that could guide future research and improvements in the AEC sector, particularly in the adoption of immersive technologies. The research underscores the urgency of addressing these challenges to facilitate ongoing development and greater adoption of immersive technologies in the AEC sector. Full article

The COVID-19 pandemic instigated a paradigm shift in healthcare delivery with a rapid adoption of technology-enabled models of care, particularly within the general practice primary care setting. The emergence of the Metaverse and its associated technology mediums, specifically extended reality (XR) technology, presents a promising opportunity for further industry transformation. Therefore, the objective of this study was to explore the current application and utilisation of XR technologies within the general practice primary care setting to establish a baseline for tracking its evolution and integration. A systematic review following the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) was conducted and registered with the international database of prospectively registered systematic reviews as PROSPERO-CRD42022339905. Eleven articles met the inclusion criteria and were quality appraised and included for review. All databases searched, inclusive of search terms, are supplied to enhance the transparency and reproducibility of the findings. All study interventions used virtual reality technology exclusively. The application of virtual reality within the primary care setting was grouped under three domains: (1) childhood vaccinations, (2) mental health, and (3) health promotion. There is immense potential for the future application of XR technologies within the general practice primary care setting. As technology evolves, healthcare practitioners, XR technology specialists, and researchers should collaborate to harness the full potential of implementing XR mediums. Full article

This study investigates how virtual and augmented reality role games impact self-learning in higher education settings. A qualitative research–action approach that involved creating augmented reality micro-stories to encourage creativity and critical thinking was used. Through role-playing, students collaborated and gained a deeper understanding of the course, improving their self-learning abilities. The findings indicate that incorporating virtual and augmented reality into higher education positively affects self-learning, promoting active student engagement and meaningful learning experiences. Additionally, students perceive these immersive educational methods as bridging the gap between virtual and in-person learning environments, ultimately leading to enhanced educational results. Full article

Metaverse is an upcoming transformative technology that will impact our future society with immersive experiences. The recent surge in the adoption of new technologies and innovations in connectivity, interaction technology, and artificial realities can fundamentally change the digital world. The Metaverse concept is the most recent trend to encapsulate and define the potential new digital landscape. However, with the introduction of 5G with high speed and low latency advancements in the hardware and software with the graphics power to display millions of polygons in 3D and blockchain technology, this concept is no longer fiction. This transition from today’s Internet to a spatially embodied Internet is, at its core, a transition from 2D to 3D interactions taking place in multiple virtual universes. In recent years, augmented virtual reality has created possibilities in the private and professional spheres. The new Virtual Reality (VR) headsets and Augmented Reality (AR) glasses can provide immersion in the physical sense. Technology must offer realistic experiences for users to turn this concept into reality. This paper focuses on the potential use cases and benefits of the Metaverse as a tech for good. The research paper outlines the potential areas where a positive impact could occur, highlights recent progress, and discusses the issues around trust, ethics, and cognitive load. Full article

In the context of therapeutic exposure to phobias, virtual reality (VR) offers innovative ways to motivate patients to confront their fears, an opportunity not feasible in traditional non-digital settings. This systematic literature review explores the utilization of narratives and digital games in this context, focusing on identifying the most common ludic and narrative immersion features employed in studies dedicated to animal phobias. Via a search on the Scopus and Web of Science scientific databases, twenty-nine studies were selected for in-depth analysis. The primary objective was to evaluate the presence of ludic and narrative elements in each study to understand their immersive potential across both dimensions. Findings suggest that ludic elements are more commonly used than narrative elements, which are notably scarce, and the exploration of the emotional dimension of narrative immersion is limited. An essential takeaway is that features fostering narrative immersion are invariably linked to the ludic dimension, often functioning as secondary components. This study provides a guiding framework for developing therapeutic interventions in VR, emphasizing the incorporation of ludic and narrative aspects. Additionally, it identifies untapped research opportunities, particularly the integration of autonomous narratives that are less reliant on ludic elements. Full article

Teaching through field trips has been very effective in the architecture, engineering and construction (AEC) disciplines as it allows students to bridge the gap between theory and practice. However, it is not always feasible to take a large class on field trips due to time, safety, and cost limitations. To adequately prepare future professionals in the AEC industry, it is imperative that institutions adopt innovative methods of providing the field trip experience. One such approach is using virtual reality (VR) technology. Creating 3D VR construction environments and immersing students in that virtual world could provide an engaging and meaningful experience. Although researchers in AEC schools have developed and deployed many virtual field trips (VFTs) in education, little is known about their potential to provide the same knowledge base. For that reason, a VR app was created to teach students about the design and construction of steel structures, called the Steel Sculpture App (SSA). The SSA served as a VFT, and the location of the steel frame structure served as the actual field trip (AFT). The research was conducted in structure-related courses in the spring, summer, and fall of 2021 and the spring and fall of 2022 semesters. Each semester, students were split into groups, one being the control group and the other being the experimental group. The control groups learned through AFTs, whereas the experimental groups learned through VFTs. A knowledge test was administered at the end of each treatment to collect quantitative data on the students’ performance, understanding, and knowledge retention. The results indicated that the students learning from VFTs scored higher than those learning from AFTs. The paper discusses student assessment results and student feedback about replacing AFTs with VFTs in times of need. Full article

Appropriate techniques for promoting reading fluency are difficult to implement in the classroom. There is little time to provide students with individualized feedback on reading aloud or to motivate them to do so. In this context, Virtual Reality (VR) can be beneficial for learning because it allows for individualized feedback and for increasing learner engagement. Studies that analyze established methods of language learning in VR at school are thus far lacking. Therefore, this pilot study is one of the first to analyze student acceptance of reading fluency training in desktop VR at a secondary school. The interview guide was developed in accordance with the Technology Acceptance Model. The desktop VR environment is web-based and provides individual and collaborative opportunities for training reading fluency, giving, and receiving feedback, and deepening content understanding of reading texts. To analyze the acceptance of the desktop VR environment, five guided interviews were conducted. The results reveal that despite various technical challenges within the VR environment, students not only accepted but also appreciated the reading fluency training in VR. The integration of established concepts of reading fluency training in foreign language classrooms has great potential as an additional value in addressing the challenges of face-to-face instruction. Full article

It is well-recognized that online experience can carry profound impacts on health and well-being, particularly for young people. Research has already documented influences from cyberbullying, heightened feelings of inadequacy, and the relative decline of face-to-face interactions and active lifestyles. Less attention has been given to the health impacts of aesthetic experiences of online users, particularly gamers and other users of immersive virtual reality (VR) technologies. However, a significant body of research has begun to document the surprisingly strong yet previously unrecognized impacts of aesthetic experiences on health and well-being in other arenas of life. Other researchers have used both fixed laboratory and wearable sensors and, to a lesser extent, user surveys to measure indicators of activation level, mood, and stress level, which detect physiological markers for health. In this study, we assessed the evidence that online sensorial experience is no less important than in the physical world, with the capacity for both harmful effects and salutogenic benefits. We explore the implications for online design and propose an outline for further research. Full article

The blended classroom is a unique space for face-to-face (F2F) interaction and online learning. The blended classroom has three distinct interaction types: in-person synchronous, virtual synchronous, and virtual asynchronous; each of these modalities lends itself to different forms of extended reality. This case study looks at using a virtual reality (VR) classroom for an online synchronous weekly meetings for three upper-division or advanced (junior and senior level) higher education design classes at a university. The use of social web VR for a classroom can offer a collaborative, real-time environment that bridges the gap between virtual video conferences and gaming platforms. This paper examines how to use social web VR in a virtual classroom. Mixed methods were used to collect usability data at the end of the semester survey. The system usability scale (SUS) and several qualitative questions gathered student feedback. Overall, the students enjoyed using the VR classroom, but audio issues seemed to be the most significant pain point. While the overall response was positive, this study will address several areas for improvement from both the student and instructor perspectives. Social, web-based VR offers promising potential. Designing a human-centered virtual environment and considering all participants’ total user experience is critical to a successful learning tool. Full article