Virtual Worlds, Volume 2, Issue 2 (June 2023) – 6 articles

Brain–Machine Interfaces (BMIs) have made significant progress in recent years; however, there are still several application areas in which improvement is needed, including the accurate prediction of body movement during Virtual Reality (VR) simulations. To achieve a high level of immersion in VR sessions, it is important to have bidirectional interaction, which is typically achieved through the use of movement-tracking devices, such as controllers and body sensors. However, it may be possible to eliminate the need for these external tracking devices by directly acquiring movement information from the motor cortex via electroencephalography (EEG) recordings. This could potentially lead to more seamless and immersive VR experiences. There have been numerous studies that have investigated EEG recordings during movement. While the majority of these studies have focused on movement prediction based on brain signals, a smaller number of them have focused on how to utilize them during VR simulations. This suggests that there is still a need for further research in this area in order to fully understand the potential for using EEG to predict movement in VR simulations. We propose two neural network decoders designed to predict pre-arm-movement and during-arm-movement behavior based on brain activity recorded during the execution of VR simulation tasks in this research. For both decoders, we employ a Long Short-Term Memory model. The study’s findings are highly encouraging, lending credence to the premise that this technology has the ability to replace external tracking devices. Full article

The popularity of VR technology has led to the development of public VR setups in entertainment venues, museums, and exhibitions. Interactive VR CAVEs can create compelling gaming experiences for both players and the spectators, with a strong sense of presence and emotional engagement. This paper presents the design and development processes of a VR interactive environment called MobiCave (in room-scale size), that uses motion-tracking systems for an immersive experience. A user study was conducted in the MobiCave, aimed to gather feedback regarding their experience with a demo game. The study researched factors such as immersion, presence, flow, perceived usability, and motivation regarding players and the bystanders. Results showed promising findings for both fun and learning purposes while the experience was found highly immersive. This study suggests that interactive VR setups for public usage could be a motivating opportunity for creating new forms of social interaction and collaboration in gaming. Full article

Virtual reality (VR) is gaining popularity as an educational, training, and healthcare tool due to its decreasing cost. Because of the high user variability in terms of ergonomics, 3D manipulation techniques (3DMTs) for 3D user interfaces (3DUIs) must be adjustable for comfort and usability, hence avoiding interactions that only function for the typical user. Given the role of the upper limb (i.e., arm, forearm, and hands) in interacting with virtual objects, research has led to the development of 3DMTs for facilitating isomorphic (i.e., an equal translation of controller movement) and non-isomorphic (i.e., adjusted controller visuals in VR) interactions. Although advances in 3DMTs have been proven to facilitate VR interactions, user variability has not been addressed in terms of ergonomics. This work introduces Piecewise, an upper-limb-customized non-isomorphic 3DMT for 3DUIs that accounts for user variability by incorporating upper-limb ergonomics and comfort range of motion. Our research investigates the effects of upper-limb ergonomics on time completion, skipped objects, percentage of reach, upper-body lean, engagement, and presence levels in comparison to common 3DMTs, such as normal (physical reach), object translation, and reach-bounded non-linear input amplification (RBNLIA). A 20-person within-subjects study revealed that upper-limb ergonomics influence the execution and perception of tasks in virtual reality. The proposed Piecewise approach ranked second behind the RBNLIA method, although all 3DMTs were evaluated as usable, engaging, and favorable in general. The implications of our research are significant because upper-limb ergonomics can affect VR performance for a broader range of users as the technology becomes widely available and adopted for accessibility and inclusive design, providing opportunities to provide additional customizations that can affect the VR user experience. Full article

Mixed-reality (MR) environments, in which virtual objects are overlaid on the real environment and shared with peers by wearing a transparent optical head-mounted display, are considered to be well suited for collaborative work. However, no studies have been conducted to provide neuroscientific evidence of its effectiveness. In contrast, inter-brain synchronization has been repeatedly observed in cooperative tasks and can be used as an index of the quality of cooperation. In this study, we used electroencephalography (EEG) to simultaneously measure the brain activity of pairs of participants, a technique known as hyperscanning, during a cooperative motor task to investigate whether inter-brain synchronization would be also observed in a shared MR environment. The participants were presented with virtual building blocks to grasp and build up an object cooperatively with a partner or individually. We found that inter-brain synchronization in the cooperative condition was stronger than that in the individual condition (F(1, 15) = 4.70, p < 0.05). In addition, there was a significant correlation between task performance and inter-brain synchronization in the cooperative condition (rs = 0.523, p < 0.05). Therefore, the shared MR environment was sufficiently effective to evoke inter-brain synchronization, which reflects the quality of cooperation. This study offers a promising neuroscientific method to objectively measure the effectiveness of MR technology. Full article

As virtual and augmented reality simulation technologies advance, the use of such technologies in medicine is widespread. The advanced virtual and augmented systems coupled with a complex interactive, immersive environment create a metaverse. The metaverse enables us to connect with others in a virtual world free of spatial restrictions and time constraints. In the educational aspect, it allows collaboration among peers and educators in an immersive 3D environment that can imitate the actual classroom setting with learning tools. Metaverse technology enables visualization of virtual 3D structures, facilitates collaboration and small group activities, improves mentor–mentee interactions, provides opportunities for self-directed learning experiences, and helps develop teamwork skills. The metaverse will be adapted rapidly in healthcare, boost digitalization, and grow in use in surgical procedures and medical education. The potential advantages of using the metaverse in diagnosing and treating patients are tremendous. This perspective paper provides the current state of technology in the medical field and proposes potential research directions to harness the benefits of the metaverse in medical education, research, and patient care. It aims to spark interest and discussion in the application of metaverse technology in healthcare and inspire further research in this area. Full article

Heritage Building Information Modeling (HBIM) is an essential technology for heritage documentation, conservation, and management. It enables people to understand, archive, advertise, and virtually reconstruct their built heritage. Creating highly accurate HBIM models requires the use of several reality capture tools, such as terrestrial laser scanning (TLS), photogrammetry, unmanned aerial vehicles (UAV), etc. However, the existing literature did not explicitly review the applications and impacts of TLS in implementing HBIM. This paper uses the PRISMA protocol to present a systematic review of TLS utilization in capturing reality data in order to recognize the status of applications of TLS for HBIM and identify the knowledge gaps on the topic. A thorough examination of the 58 selected articles revealed the state-of-the-art practices when utilizing static TLS technology for surveying and processing captured TLS data for developing HBIM models. Moreover, the absence of guidelines for using static TLS surveys for HBIM data acquisition, the lack of robust automated frameworks for producing/transferring 3D geometries and their attributes from TLS data to BIM entities, and the under-utilized application of TLS for long-term monitoring and change detection were identified as gaps in knowledge. The findings of this research provide stakeholders with a good grasp of static TLS for HBIM and therefore lay the foundation for further research, strategies, and scientific solutions for improving the utilization of TLS when documenting heritage structures and developing HBIM. Full article