Immersive Environments and Virtual Reality: Systematic Review and Advances in Communication, Interaction and Simulation

Abstract

Today, virtual reality and immersive environments are lines of research which can be applied to numerous scientific and educational domains. Immersive digital media needs new approaches regarding its interactive and immersive features, which means the design of new narratives and relationships with users. Additionally, ICT (information and communication theory) evolves through more immersive and interactive scenarios, it being necessary to design and conceive new forms of representing information and improving users’ interaction with immersive environments. Virtual reality and technologies associated with the virtuality continuum, such as immersive and digital environments, are emerging media. As a medium, this approach may help to build and represent ideas and concepts, as well as developing new languages. This review analyses the cutting-edge expressive, interactive and representative potential of immersive digital technologies. It also considers future possibilities regarding the evolution of these immersive technologies, such as virtual reality, in coming years, in order to apply them to diverse scientific, artistic or informational and educational domains. We conclude that virtual reality is an ensemble of technological innovations, but also a concept, and propose models to link it with the latest in other domains such as UX (user experience), interaction design. This concept can help researchers and developers to design new experiences and conceive new expressive models that can be applied to a wide range of scientific lines of research and educational dynamics.

1. Introduction

Immersive and interactive technologies such as Virtual Reality (VR), are a new milestone in the way we interact with our environment, and even how we conceive new approaches in our relationship with reality. Virtual reality and other immersive Information and Communication Technologies (ICT) have a high potential for transforming the real world and the way in which we interact with it. Thus, as virtual reality can be a powerful tool to bring about changes in social reality itself, it is necessary to define and analyse its potential implications and influence on how new media and communication technologies can create new messages and cultural approaches (based on the initial idea of McLuhan [1] “The Medium is the Message”).

In this paper, we consider VR not only as a technological tool but also a concept related with the power and possibilities of supporting new emerging realities never seen or experienced before. VR is therefore a technology with many interactive possibilities, especially in an immersive approach related with 3D images and sound, but also with the possibility of encompassing other human senses and perceptive channels. The possibilities of creating innovative languages (using expressive and communicative approaches) based on the whole range of human perception are almost unlimited, and depend on technological evolution in the first stage. In this paper, languages are understood as the expression of cultural reality [2] and ways of interacting with technology and the environment [3], as well as a natural language interface [4].

In this way, this manuscript aims to contribute with new approaches departing from communication theory [Figure 1], and analysing the current state of VR technologies as well as the ways technology interacts with users, and users interact with information. This contribution is based on a literature review of current VR technology and its applications. We also propose a conceptual approach where we try to define and situate the components of immersive environment-human interaction, such as with virtual objects, devices, interactions, devices or narratives. We also aim to define a wider field than virtual reality, considering that technology has allowed us to integrate new senses in the experiences. This wider field of reality keeps three main factors in mind which may help to conceptualise new experiences with digital information: artificiality, simulation and alternative (ASA).

Thus, virtual reality and immersive environments, as part of the emerging technological evolution involving our senses and cultural, symbolic and representative factors, may present interdisciplinary approaches. These approaches contribute to creating new languages, which involve disciplines such as interaction design, human-computer interaction, user experience and interface, and even affective computing, which have new approaches in recent years with the evolution of ICT. Among these new approaches, we can find the evolution of the research methods in human-computer interaction, as Lazar et al. describe in their research work [5]. These research methods include current approaches in analyses of human factors such as eye-tracking, motion tracking in virtual environments, and the position of different parts of the body. Lazar et al. also integrate in such research methods the human factors related with kinetics, such as motion tracking in virtual environments or muscular and skeletal position sensing. More innovative approaches also include research in inclusivity focusing on disciplines such as user-centred design, as Coleman et al. describe in their work [6]. Interaction design also has innovative approaches in the latest technology, being relevant in designing and classifying systems related with virtual reality and immersive environments, as Earnshaw do [7]. They attempt to define the main components of the current generation of virtual reality systems and focus on “key areas and domains: software, hardware, interactivity, technology, application interfaces, cognitive factors and ethical issues” The design of interactive systems has also been applied to natural user interfaces in immersive environments and virtual reality systems. The approaches in body language and gestures, through the interaction design approach, have also evolved in interfaces for manipulating objects in virtual reality [8]. Additionally, user experience, interaction design and human-computer interactions have been classified under taxonomical categorisation for designing virtual reality systems [9]. The design of the user interfaces has also evolved with virtual reality systems and immersive environments, where some authors [10,11,12] suggest that research and advances in human-computer interaction and interaction design should be brought closer to standards of universal usability. These standards are a useful tool to be applied in disciplines related with human factors, ergonomics and, therefore, in medical research, presenting a strong potential in domains related with neuroscience. In medical sciences, user experience applied to virtual reality and immersive environments are the basis for developing a neurofeedback scenario, especially in the recovery of patients with cognitive disorders and victims of accidents that caused strokes [13].

1.1. Advances in Virtual Reality

Virtual reality is a research field, not just an emerging ICT, which may have a significant role in the way we interact with physical world, as well as the way we interact with information in a multidirectional way. The challenge of technologies and applications based on VR is conceiving new ways to design information, new narratives and storytelling for a medium in which possibilities have not been fully explored. This is why the challenge of virtual reality is not only focused in a technological sense, but also in the way this medium is going to develop its own dynamics for interaction with users (comprising gameplay), with potential applications in new fields.

The first approach conceiving of VR (and a fictional VR model) was described in Weinbaum’s story Pigmalion’s Spectacles [14], where two of the most important facts depicted by the author were “you are in the story” and “the story is around you”. Both these sentences are two basic pillars of approaches to defining the idea of VR. Further definitions by current authors describe the medium as “A medium composed of interactive computer simulations that sense the participant’s position and actions and replace or augment feedback to one or more senses, giving the feeling of being mentally immersed or present in the simulation (a virtual world)” [15]. Other authors, in the early 2000s, focus on the idea of a real-time simulation and the multisensorial dimension, especially Burdea et al. [16], who also define VR as “a high-end user-computer interface that involves real-time simulation (not precomputed, computed as time is passing) and interactions through multiple sensorial channels (visual, auditory, tactile, smell and taste)”.

In recent years, the immersive nature of media has led us to try to understand the conceptual nature of a technology [17] whose features and way of interacting with users require definitions, standards, and approaches. Authors such as Sherman and Craig [18] focus on facts such as design, interface and applications as relevant factors for understanding the medium and the technology, defining virtual reality as “A medium composed of interactive computer simulations that sense the participant’s position and actions and replace or augment feedback to one or more senses, giving the feeling of being mentally immersed or present in the simulation (a virtual world)”. However, definitions of virtual reality evolve to the extent that technology evolves, and integrate firstly a higher range of human factors [19], and secondly, possibilities for interacting with the environment, with advances in areas such as brain interfaces and their relationship with virtual reality, immersive environments and videogames [20,21]. The fact is that virtual reality is linked with immersive environments and 3D objects, as well as game engines, but also with user interfaces whose research allows us to access to a wider range of human factors and explore new natural user interfaces.

1.2. Immersive Environments: Conceptual Approach

Immersive environments are defined in different approaches by authors from the 90s, in parallel to virtual reality and augmented reality. Slater and Wilbur [22] propose a framework to define presence and other factors in virtual environments. In the early 2000s, Conroy [23] tries to describe them by constructing experimental immersive worlds based on data coming from the physical world, and establishing an analogy between both. Kalawsky [24] also compares presence in virtual environments with presence in the physical world by adding the main sensory inputs “such as vision, sound, proprioception, and smell”. This idea, where different senses are used through devices to make users interact with information is common in VR and immersive environments, and then, for designing interactions and conceptualising and classifying the different components of the environment composed by artificial information. And, even though VR and immersive environments are not the same concept, a framework to define presence and describe interaction and immersion should keep similar components and features in mind.

In the area of immersive environments, theories such as “spatial presence theory” have been addressed by authors [25] trying to define exactly what it means “to be in a determinate place”, focusing again on cognition and ergonomic factors. However, as the aforementioned authors argue, spatial presence theory must integrate more coherent theoretical frameworks and focus on empirical research, as well as keeping in mind factors such as attentional processes and embodied cognition.

In recent years, definitions and differences between the ideas of VR and immersive environments have focused on relevant studies such as those of North and North [28], arguing that user experiences “are critical components” in developing more immersive applications. Cummings and Bailenson [29] examine the effect of technological immersion on the factor of presence, concluding “that increased levels of user-tracking, the use of stereoscopic visuals, and wider fields of view of visual displays are significantly more impactful than improvements to most other immersive system features”. However, Communication Theory sets out an interesting basis for going in depth into factors related with interaction with virtual reality and immersion in artificial environments. In this manuscript, we present an approach based on inputs and outputs between users and immersive environments, and different ways in which interaction-communication may be defined.

2. Objectives

The main objectives of this research and theoretical approach are as follows:

• Going in depth on models based on Communication Theory in order to apply them to immersive environments, VR and ICT, in order to define the current state of virtual reality technology and its relationship with other media and applications in diverse scientific and communicative areas.
• Contributing to describing a framework where advances in emergent approaches in disciplines such as interaction design, human-computer interaction and user experience are linked to research in VR as a medium and from a technological approach. We also aim to connect the idea of virtual reality and immersive environments with user experience approaches to contribute to defining the current state of interaction, and considering the next steps for going further in technological and human-factor research applied to VR fields.
• Contributing to describing the features of VR technology and medium and its possibilities in designing experiences or simulating situations.

3. Materials and Methods

The current research document is based on a literature review which addresses applications of virtual reality and immersive environments in diverse research and knowledge areas. Conclusions based on these literature reviews and analyses consider short-term scenarios and set the next milestones for designing immersive environments both in a conceptual and technological way. These new immersive environments will help us to better understand our own physical-real world context and plan scenarios which may represent and communicate conceptual ideas and knowledge by interacting with users.

As a main methodology, we apply a systematic literature review of more than 70 papers, books and chapters related with virtual reality and immersive environments. A systematic literature review is defined as “a summary of studies addressing a clear question, using systematic and explicit methods to identify, select, and critically appraise relevant studies, and to collect and analyse data from them” [30]. The advantage of a systematic literature review is that it allows researchers to have an approach to the topic from various viewpoints, essentially those related with communication and interaction: interaction design, human-computer interaction, human factors and user experience. As described throughout the paper, we analyse advances in research which combine VR and immersive environments with these factors, in order to contribute to designing universal standards of usability, which may potentially be applied to immersive environments and VR.

The systematic literature review also comprises analyses of systems under an interaction design approach, human-computer interaction and human factors. The purpose of this is to contribute to defining the next generations of immersive environments and virtual reality systems, and structures that define them.

Recent research in interdisciplinary fields linked with to design, ergonomics [31,32,33], computer science [34,35] and media and communication [36,37], among other areas, is relevant today for demonstrating cutting-edge technology and the projections for virtual reality and other similar technologies, comprising immersive environments and interactive devices. These new approaches have a necessary relationship and lead us to issues related with new phases in research in these fields. Among these issues, some of the most relevant are:

• How can VR and immersive environments evolve as mediums with their own features? What human and technological factors should be kept in mind for designing new experiences based on VR technologies?
• How can VR and immersive environments evolve at the same time as human factors and computer science and increase the complexity of interactions between users, technology/devices and the environment?
• What are the next human senses and cognitive approaches that should be integrated into VR and immersive technology? How could researchers in these areas explore the possibilities of interaction between users, devices and the environment?
• How can we build a culture of a medium, based on VR and immersive environments? How can we describe and analyse the language that evolves in that medium?

The expressive potential of computational media also has a role in defining technological and human factors. As Harrell [38] argues, computational media (in this paper focusing on VR and immersive environments) have a strong expressive potential to “construct blends of cultural ideas and sensory imagination” (what Harrell calls “phantasms”). This expressive potential, put forward and theorised by Harrell, must consider factors such as narrative, figures of speech and representation and cultural approaches, and go in depth into integrating and applying interdisciplinary research areas such as interaction design, user experience or human-computer interaction in research in VR and applications based on this technology.

The research has been done by following the structure of a systematic literature review, as defined, to propose a taxonomical model and structure of immersive environment-human interaction based on communication theory and other approaches of HCI or user experience. The first step of the review led us to define the research questions and research protocol, where some keywords are used. In the second step, we aim to identify relevant recent research manuscripts (papers, books, online articles) related with the topic. We also review research manuscripts carried out in the past decades, in order to define the different concepts, as well as see the evolution by comparing different state of the art concepts. We summarise the information of the manuscripts to try to put it together with the different topics tackled in current research. We have also identified the more relevant information about every research area, mainly, human factors (including interaction design, HCI, or user experience, among others) and VR and immersive environments in the same contents block, analysing the different applications and advances in research in these domains. As mentioned, more than 70 documents have been identified, and all of them have been reported and cited in different sections of the current research manuscript. Conclusions have been extracted from this literature review and by classifying some features related with interaction. A schematic model based also on communication theory and which defines the components of immersive environment-human interaction has been elaborated, and is further discussed in the corresponding section.

The criteria used for select manuscripts are, among others: the relevance in the topic and research area (citations) and the approach of VR and its application and use aforementioned disciplines. We’ve then chosen manuscripts reflecting the advances of VR and immersive environments applied to various fields, and research connecting VR with human computer interaction, interaction design or user experience, as well as human factors.

3.1. The Virtuality Continuum: The Relationship between the Physical World and Digital Information

The virtuality continuum was proposed by Milgram and Kishino in the 80s and 90s [39] as a theoretical approach regarding the connection between information coming from the physical world and artificial digital information. These decades were also productive in theoretical approaches and immersive possibilities regarding VR, while technology was in the first stages of development. Thus, the first theoretical approaches and technological progress in the VR field were accomplished by researchers such as Lanier [40]—who also founded VPL Research Inc., (Palo Alto, CA, US)—and Cruz-Neira et al. [41]—by developing the CAVE automatic virtual environment.

Kalawsky [42] approached the applications of VR and virtual environments in various domains in the 90s. Aukstakalnis and Blatner’s [43] work, in the same line of research, proposes diverse applications for this technology in the future, regarding computer games and medical applications, compiled in their book Silicon Mirage: The Art and Science of Virtual Reality. Rheingold [44] is another author who has made significant theoretical approaches in the use and application of VR in diverse fields, in order to search for various uses for a technology that was in its initial stages. In the same decade, Carlsson and Hagsand [45] developed the multi-user VR system named DIVE (Swedish Institute of Computer Science, Kista, Sweden), and in the early 90s Cruz Neira et al. built the immersive VR environment Cave Automatic Virtual Environment (CAVE) [46]. Sauer and Schömer [47] also conceived the GALILEO system (Daimlier-Benz Research and Technology, Ulm, Germany; Universität des Saarlandes, Saarbrücken, Germany) as a rigid-body simulation. Years later, in the early 2000s, Ryan [48] considered the possibilities of VR for representing ideas and concepts, arguing that interactive and immersive technologies, such as VR, have their precursors in both traditional narrative and arts, and tackles the idea that VR is, as a medium, a metaphor of total art.

With this approach, Milgram and Kishino [49] made a taxonomical proposal named “Virtuality Continuum” which presents various intermediate steps in a continuum line, from virtual reality to the physical world. This continuum is the starting point for further development of theoretical approaches regarding interaction based on the nature of human senses and cognition.

3.2. Virtual Reality as a Medium: Representation and Interaction

The issue linked with this new media (VR) is its power to represent ideas and make users interact with information through new approaches. Thus, VR technology will allow the development of representative and expressive theoretical models, both in technological and communicational terms. Since the 90s, VR technology and expressive power have allowed the representation of abstract or non-figurative ideas and concepts, such as data and information, among others. Research such as that carried out by Valdés [50] explores the representation power of information systems. Valdés’ research uses virtual reality techniques for visualising and representing information, and for data mining, and has been applied to various scientific domains, such as geology and genomic research.

Research in science and scientific dissemination is an emerging field for applications using VR technologies. Thus, the potential expressive power of this medium (unexplored, in some cases) may play a relevant role in scientific dissemination and communication, allowing the design of interactive and immersive scenarios where users are in contact with scientific abstract ideas and information. The design of this immersive and interactive information is a challenge which will involve interdisciplinary research and development fields, such as scientists, designers, computer experts, artists and cognitive science experts, among others. Some relevant examples of recent applications of VR to science are therapies for posttraumatic stress disorder [51], neurorehabilitation for strokes [52], research in proprioception and body swapping for allocentric purposes [53], use of VR for studying 3D cellular models (e.g., neural tissues) [54] or biodiversity conservation sciences, being used in an attempt to save endangered animals [55].

Among different research approaches regarding VR as a medium and its potential for science communication, authors such as Erickson, Grantham and Wexelblat [56] consider the relationship between data, visualisation and information architecture, and the possibilities they provide for going in depth into science communication and dissemination by searching for ways to integrate them into immersive 3D environments.

Yair et al. [57] and Moreno and Mayer [58] develop pioneering research by applying 3D virtual reality to learning sciences, such as astronomy. Bailenson et al. [59] also consider the potential of virtual environments to transform behaviour and interaction with information, which leads us to consider the technology’s potential to empower social changes by democratising information, access and the understanding of data.

In recent years, Seidel and Chatelier [60] have analysed the use of virtual reality for simulation and interactive communication, keeping in mind factors such as interfaces (including visual, haptic and auditory interfaces), evaluation methods for effectiveness and psychological issues, approaching factors related to 3D immersive and interactive environments with emerging interdisciplinary fields such as user experience or interaction design.

Geography and urban development are also domains where VR has been developed. Calado et al. [61] have carried out a literature review where they focus on the relationship between the user and the environment, under an approach based on urban design. Rossman et al. [62] integrate VR and motion simulation to visualise concepts and ideas in the geographical urban design domain, and Hanssen [63] proposes the idea of VR as a tool for designing urban spaces, by coining the term “VRurbanism”. Archaeology also has a relationship with urban spaces and geography in the context of VR. Recent examples of research show the potential of transmedia technologies and virtual reality to reconstruct archaeological sites [64,65].

3.3. Virtual Reality and Its Approach to Simulation

Virtual reality has also been a medium used in the development of training systems, due to its power of simulation. The relationship between simulation, interactivity, visual representation and training makes VR a relevant research field. VR is then applied not only to science dissemination and communication, among other things, but also as a medium for developing safe environments for practice in some research disciplines, due to its interactive potential with the environment. Recent research studies show us a wide range of uses for simulation. Zahiri et al. [66] propose the use of VR for simulation-based surgical training. Vaughan et al. [67] describe five main areas where simulation in VR is applied: manufacturing, medical sciences, serious games, rehabilitation, and remote training, where they focus on on-line interactive training such as Massive Open Online Courses (MOOCs). The approach towards training in areas such as the aforementioned, is based on practice by repetition. Even though VR has a strong potential to offer experiences based in discovery and exploration, and even a creative approach, simulation offers the possibility to practice in safe environments and develop VR applications for training based on the practice. Choi and Lo [68] use VR for task simulation in helping disabled child patients, connecting the need of rehabilitation training in occupational therapy with the technology’s potential to create ad hoc immersive environments using its simulation possibilities. Cho et al. [69] also have a rehabilitation approach to VR, in this case, developing a system that focuses on proprioception as one of the main factors regarding the use of VR and immersive technologies. In this approach based on proprioception as one of the bases for research in applications for VR, we can also find interesting studies such as those of Slater et al. [70] which focus on the application of multisensory body illusions with virtual reality. The research carried out by these researchers looks at the possibilities of this technology to give the user a different body in the virtual world arguing that the user’s body representation is very flexible. Levin et al. [71] also carry out research related to VR applied to motor skills using its potential to simulate tasks, in this case, applying the technology to upper limb rehabilitation. There are also research studies based on proprioception and multisensory body illusions, such as. Transforming the Self into a Child, by Sánchez-Vives and Slater [72] where adults embodied in child bodies through VR change their perception regarding the size of environments and objects [73]. Other research related focus in approaches such as multisensory assessment [74], innovative audio visual approaches [75], noise perception in immersive environments [76] or multisensory immersive virtual environments [77].

Thus, simulation is a recreation of a virtual environment for diverse purposes, and its immersive power will depend on diverse interactive dimensions related with human senses, such as computer graphics (sight, sound, haptics and proprioception). The sense of immersion and the perception of reality enabled by VR has normally been based on this ensemble of human senses, but interesting approaches in research would perhaps widen the range of human senses and cognitive dimensions when designing VR technologies, as a new milestone in the connection between artificial information and human perception.

3.4. Representative and Expressive Power of VR Technology

The expressive power of this new media and technology is also an issue concerning research in how VR is going to evolve in the coming years. Factors related to embodiment, human perception and cognitive approaches, as well as proprioception, will determine how information will be presented in immersive environments. Thus, VR is a technology that in some way, simulates real world components, but it can also represent abstract or non-figurative ideas such as data or metaphors. Representation of ideas and concepts is another emerging line of research related to the application of virtual reality to some domains, considering that VR has the potential to develop its own language and represent not only figurative components but also abstract ideas and metaphors.

Embodiment and proprioceptive focus are also a relevant factor in research in VR, under the approach of interaction with information and objects built into immersive environments. Studies carried out by authors such as Spanlang et al. [78] propose VR as a tool for research in cognitive sciences or experimental psychology, among others, and go into depth about factors related with proprioception and embodiment, emphasising the connection between sense of self, cognitive and sensitive approaches, and representation of virtual information.

Representation in VR can depend on multiple factors, some of them related with diverse dimensions of cognitive approaches and the ways we conceive and represent space, and make use of the components within that space. With this approach, VR can be used as a learning tool for domains related to spatial representation, such as geometry. Hwang and Hu [79] carry out research based on this factor, focusing on the possibilities of VR for training problem solving, in this case, in geometry, a discipline which focuses specially in spatial issues and representation of objects with physical features.

Other approaches focus on representation of information. The use of VR may help researchers and developers to represent complex data by using the three-dimensional environments, in order to make them more comprehensible. Nowadays, some of these applications and approaches are being used in fields such as atmospheric and environmental sciences, as shown in studies like those completed by Helbig et al. [80]. Donalek et al. [81] also propose the use of VR for immersive and collaborative data visualisation, which means that we can find research linking the data science and data visualisation approach and the tool of VR, being also a medium to represent and visualise data.

The approach related to the representation of information and interaction with virtual objects and environments reinforces the idea of VR as a medium. Franco and González [82] propose the use of VR as a medium for communicating contents and developing interactive applications which can be used in domains such as education. The content design and the integration of interdisciplinary research fields, such as interaction design and user experience, will be a relevant factor regarding applications of virtual reality.

The immersive experience and the interaction between users, technology and environment should consider the integration of interdisciplinary research areas such as interaction design, user experience or HCI, among others, in order to improve the features, both from technological and experiential approaches. Thus, taxonomical classifications are a feature to be developed over the next few years, in order to approach the different dimensions in interactive processes and immersion under the different disciplines (as shown in Figure 2).

Several approaches to classifying devices and interaction approaches have been made since the 90s by authors such as Coomans and Timmermans [83], Macedonia and Zyda [84] and Milgram and Colquhoun [85]. In recent years, these approaches to taxonomical classification have tackled interdisciplinary and multidimensional approaches for integrating interactive and immersive factors and features, in order to conceive and design new experiences. User experience, HCI or interaction design, are fields to be kept in mind in taxonomical classifications for 3D immersive experiences and widely cognitive interactive processes, as shown in recent research studies in the field under various approaches [86,87,88].

3.5. Virtual Reality as a Tool for Artistic Creation

Virtual reality and immersive environments imply a fictional narrative which can help us to represent otherwise our perception of the real world, in order to conceive and design new symbolic structures and cultural ensembles. These virtual environments allow us to conceive and design scenarios in order to simulate possibles (Bourdieu, The Field of Cultural Production) allowing new cultural factors to emerge. These possibilities of representation are potentially infinite, and depend on a combination of factors, such as technological evolution, implication of the ensemble of senses and (interaction).

They are also conceived as technologies and media, and an emerging field for research in artistic creation and experimentation. They are not only an interactive and immersive canvas for representing ideas but also a dynamic tool for creation in a 3D space. The expressive and interactive power and multidimensional approach lead to innovation in creating and developing 3D environments. VR is also being used to explore experimental ways for new artistic approaches. Tilt Brush [89] is a relevant example of technology applied to the development of new technological tools for artistic purposes. Another tool using VR for artistic purposes, in this case graffiti, is Infectious Ape [90].

Additionally, VR is a field for the representation of ideas and concepts, including those of an artistic approach, in an immersive environment. A 3D immersive interactive environment in the context of VR, is a field for developing and representing innovative and creative ideas and concepts through the perspective of art. Some experiments in VR have been carried out using neural networks such as Deepdream, exploring concepts of fractal and psychedelic art by creating an immersive experience. This immersive experience has also been conducted in film productions within virtual environments like in Dear Angelica, by Saschka Unseld [91], which innovates the exploration of the possibilities of VR to produce immersive 3D animation films. The emergence of VR as a medium, leads to new frameworks and cutting-edge approaches in new media art, by increasing the possibilities of creative exploration and research and therefore interaction and expressive potential.

4. Results

A systematic literature review may lead us to configure and subdivide features regarding VR in order to generate models which improve the interaction process between users and information in multidimensional ways. The components of interaction and immersion processes must be approached from the user experience, interaction design and HCI focuses, configuring adaptations of these interdisciplinary research areas to develop interactions and imaginaries in this medium.

Designing models and approaches for developing interactions and immersive environments in this medium, connecting user experience, narrative, ergonomic and cognitive factors, and expressive power, as well as technological development, are the current challenges for an emerging medium in which a theory is being developed.

For this purpose, categories must be defined in order to go into depth in defining the forms of this new medium and exploring its possibilities in both creative and research approaches. From the literature review, we point out the different factors approaching the definition of this medium and its evolution together with technological progress, approaching the complexity in theories such as communication theory.

4.1. Factors Related to Design of Environments, Experiences and Stories in VR

VR is a complex medium regarding multidimensional approaches and focuses, which also describes a reality (even an alternate reality which has its own rules). Therefore, some factors should be kept in mind, as virtual reality, as an emergent medium, has its own communication and interaction features. These features also have cognitive, technological, design, artistic and agronomical approaches, and, being an interactive medium, it can also integrate factors such as gameplay.

4.1.1. The VR Puzzle and its Relationship with Other Research Domains and Application in Diverse Fields. The Concept of ASA Reality

In order to configure the form of theoretical approaches of virtual reality, the focus on the concept of VR as a puzzle for research and integrating components can lead to future research in the topic to develop new conceptual models. As VR is also an idea of a potential artificial, simulated and alternate reality (which we call ASA reality), it also potentially has many of the components of a real world, but with the possibilities to develop a series of imaginaries, phantasms and other conceptual constructs. As this ASA reality, it also contains the features of fictional narratives and the power to represent ideas and simulate situations in order to project and configure a potential space of possibles (as Bourdieu cites in The Field of Cultural Production [93]).

4.1.2. Some Factors Associated with VR

In order to design theoretical models for subsequent practical applications, VR, as a technology and a medium, needs to define the research for developing and creating some components. As cinema as a medium and a technology has components such as script, actors, special effects, shots, etc. VR needs to be defined by its components as its technological potential progresses. Some of these features and factors related to the nature of the medium are:

• Storytelling-Narrative Dimension: Associated with the account of the virtual environment and the power to articulate it, and the story which users create themselves by interacting with the virtual world.

  ○

  Narrative: Associated with the events we want to recount and the elements we, as developers, are going to use to relate to these events. It is the reality that we want to happen, in this case, in a VR world.

  ○

  Storytelling: Associated with the dramatic approach that we are going to use to describe the facts.

• Interactivity: Authors such as Steuer [94] define it as “the extent to which users can participate in modifying the form and content of a mediated environment in real time”. We define it, in this contribution, as “the potential to receive information from the ensemble of our senses and to construct and configure an alternate reality or to simulate reality”. Additionally, it is the potential to influence (in real time) in the digital environments, the objects and the narrative framed in it. Interactivity, as opposed to communication in traditional media, is multidirectional and covers a wide range of cognitive ranges. The following interdisciplinary approaches are the research fields where it is necessary to develop standards to improve the process of having an experience in virtual reality and in immersive environments.

  ○

  Human Computer-Interaction (HCI): An interdisciplinary field which analyses in order to improve interactive processes between users, digital information and environment. A proposed variable is Human World Interaction (HWI), which analyses the user’s behaviour in certain contexts to design experiences in virtual environments.

  ○

  Interaction design: The design of interfaces and digital objects in order to improve the process of interaction and broaden the range of cognitive approaches to communicate and influence in a digital environment and with digital information.

  ○

  User experience: The interdisciplinary field focused on conceiving and developing interactive experiences which are based in a natural user interface, and improve its interactive process.

• Representation: VR is a technology, but also a medium with its own potential expressive power. To represent means to position ideas and concepts in a specific medium, as for example words represent ideas in literature. Research on the representation and expressive power of VR needs to focus on the question of how this medium can communicate and develop its own dynamics.
• Gameplay: As VR is becoming more and more interactive, it is playable, so, a relevant factor is to integrate research into game and play possibilities, even for experiences based on physical environments (such as games—in the physical world-, video games—digital mediums, or alternate reality games—ARGs, augmented reality) mixing the concepts and going further in research related to a complex interaction process.
• Technological Evolution—Mechanics: VR will progress at the same rate that computer graphics technology and processors do. So, it is also a transitional technology which will lead us to other wider taxonomical classifications of mediums and digital information.

4.1.3. The Role of Video Games in VR Applications and Immersive Environments

We need also to briefly describe the influence of the games and videogames in the evolution of VR and immersive environments. They are currently closely related fields, but, at the beginning, and for a long time, videogames were not immersive (or had a limited potential of immersion) and virtual reality had technical limitations, as well as a lack of ludic applications, being applied mostly to training such as flight simulators. However, the influence of VR in videogames and vice versa is a fact nowadays, up to the point that there exists a convergence between the two. This bidirectional influence was observed and described by authors like Zyda [95], arguing that: “VR researchers who want their work to remain relevant must readjust to focus on game research and development”. So, the approach to the video game while using VR as a tool for research and to improve our lives, is being used in various fields of scientific research: to improve functional mobility and rehabilitation in some age groups [96], to train skills in safe scenarios [97] or even to replicate sensations in these safe scenarios, such as fear [98]. Virtual reality games have also been being explored, for some years now, as a therapy for rehabilitation [99,100], due to its potential to isolate from the physical world and to enable users to accomplish tasks and explore the environment.

Virtual reality games for specific non-ludic tasks (but under a ludic approach) are called “virtual reality serious games”. However, when we analyse applications of VR games, their repercussions and their projection, we can observe that, on the one hand, the video game industry has found numerous possibilities to create new game mechanics and narratives by using VR, and focus on its immersive potential to explore them. On the other hand, VR, as a medium is exploring the potential of gameplay and ludic factors to develop applications with non-ludic purposes, being useful in research in medicine or data visualisation, among others, as related.

5. Discussion

What is the future of virtual reality and immersive 3D environments? The interest of research in this technology lies the possibility of its use, application and implementation in a wide range of research domains and disciplines. On the other hand, VR is a technology and a discipline which is going to be inevitable, linked to the aforementioned interdisciplinary research fields such as interaction design, human-computer interaction, and user experience, among others. Cognitive sciences and approaches to human senses are part of a research field which is closely connected to this technology and medium, and it is likely that research in cognitive sciences and even neuroscience is going to be integrated into the development of VR applications and uses in the coming years. The narrative and storytelling plays an important role, to the extent that developers will need to find new stories to tell with this medium, and explore the possibilities to create new worlds or express new ideas.

Narrative and storytelling development are also relevant factors in phenomena such as videogames. In recent years, VR has developed its own narrative and language when applied to the video-game industry. In this way, the expressive power of VR has allowed the exploration of many genres in the field of videogames. Among these genres, we find sci-fi (Gunheart is a relevant example [101]) or the use of VR as a medium to relate to folk tales, such as Rainbow Crow [102], among others. Other approaches also argue that VR is developing its own language and some specific features are separated from videogames, even if in at present they converge. As Stark affirms [103], “it can be its own entertainment”, naming examples such as Mure VR [104], an Icelandic start-up that provides workers with “a more enjoyable and relaxing experience in their workplace”, by “using techniques, such as psychotherapeutic ones, to simulate a wide variety of relaxing environments by recreating landscapes and nature”.

This ensemble of experiences should lead us to answer questions related with the definition of the current state of interaction, more specifically in VR. The integration of more senses in the process of interaction, as described in many parts of the paper, such as in the previous paragraph, is nowadays a challenge in both the technological and experiential areas, as well as in cognitive sciences, among others. Haptic, aural or olfactory technologies, to mention a few, should be tools used to improve visual immersion, but also to offer progressively more interesting experiences for users. These experiences should, at the same time, offer an alternate reality with an increased interest in relation with the physical world, in term of experiences.

As explained before, VR and 3D immersive environments are media with potentially unlimited possibilities to represent ideas and interact with them in a way which was not available before. For that reason, it is necessary to develop and build new symbolic, narrative and representative approaches which help us to connect this. The issue regarding representation of factors associated with scientific knowledge and dissemination such as colour theory, algorithms, abstract concepts in quantum mechanics, chemical formulas, and all the imaginable scientific disciplines, requires innovative approaches in symbolic construction, but also the design of models which integrate the interactive and immersive potential offered by a rapidly evolving technology.

The current state of interaction shows us a complex multidimensional framework, where several senses are involved, as can be seen in Figure 3, and considers the next steps for going further in technological and human-factor research applied to VR fields.

A deep knowledge of human senses and a development of new user interfaces (as brain-computer interfaces), would help contribute to developing more effective immersive experiences and VR environments. Additionally, VR and immersive environments have a high potential to develop and integrate symbolic and abstract representations, in order to increase the range of experiences. This complexity in relationships between factors is also approached in the aforementioned ASA reality. This concept is, as mentioned, used to describe a wider concept than VR, potentially integrating interactions with a wider range of senses or cognitive and even neurological approaches, in artificial (computer generated), simulated (not integrated in physical world) and alternate (separated from physical world) environments.

Thus, VR and immersive environments have many ways to evolve as media with their own features, integrating various senses or interactive narratives. The design of new experiences in VR systems will integrate and combine various human factors, such as kinetics interacting with more and more parts of the body. The complexity of interactions between users will also require further taxonomical classification of the components that make them possible. And the possibilities of interactions between users, devices and environments are described through emergent approaches in this sector.

These perception related technologies which in many cases increase the degree of immersion, have dramatically evolved in recent years, which leads us to the idea that every perception is able to be simulated or represented in some way. This is the case with the development of olfactive technologies in VR, rebuilding the idea of VR head-mounted display (HMD) in VR helmets, such as FeelReal [105], which integrates the experience of scent in their helmets, and simulates natural phenomena such as wind. Nowadays, the attempt to integrate scent in VR experiences is showing us prototypes of devices that try to imitate scent in the virtual realities of Vaqso [106] and OhRoma [107], even though they are, in their current state, experimental approaches. In research, scent (and haptic technologies) has been used in experiments based on VR to induce relaxation and strengthen the sense of presence, as Serrano et al. explain [108]. Scent therefore certainly has a promising future in research fields related with VR, but it is necessary to determine what we want to develop as an experience and how to connect it in a more multisensorial experience.

The same questions are needed in haptic or aural technologies, as well as other technologies directly connected with VR. As we can observe, nowadays there are some examples in research in such areas as these, even though the approach in haptics technologies and its application to VR is sometimes complex. As Otanduy et al. mention, full immersive VR systems will “require high quality haptic interactions” [109] and it will be necessary for users to touch and interact in a realistic way with their hands. Haptics must also extend to other parts of the body, in order to improve the realism of the immersive experience, and, in recent times, various devices have been developed, approaching the fact that users can interact with artificial environments ant information with different parts of the body. Nowadays there is a wide range of external devices (gloves, shoes, bodysuits, etc.), “where users can receive feedback from computer applications with physical sensations in the hand or other parts of the body” [110]. New milestones in haptics applied to VR show an ultrasonic haptic interface created at the University of Bristol, where users can interact with virtual objects using a bodysuit [111]. However, research in haptics for VR has at present many different potential approaches that give many possibilities and increases the complexity of defining a framework where haptics and other technologies and approaches fit within the VR research field.

Finally, what about the next research approaches in VR, such as brain-computer interfaces? The next steps in discipline-based interaction (such as UX o IxD—interaction design-) and the VR field have a necessary gap that will be tackled in future years under different approaches. The idea of interacting with the brain with a virtual world can be approached in many ways, and it is the natural step in research in VR and ASA reality, among other fields. Nowadays, technology is exploring these fields, as shown in recently developed brain-computer interfaces for VR, such as Neurable [112]. However, the phenomenon is not recent and the idea of connecting VR and brain-computer interfaces comes from the last decade, in research carried out by Bayliis et al [113], developing a virtual reality testbed for brain-computer interface research, or Lecuyer et al. [114], who integrate video games in VR and brain computer interfaces. Recent research in connecting both fields have evolved in the form of applications of technology, in fields such as rehabilitation [115,116] or cognitive approaches [117]. The evolution of this technology in the VR field and others, has, however, a potential which will be developed in the next few years, finding new applications and new approaches in how we want to interact with virtual environments.

This leads us to new questions, that research in virtual reality will combine both research in technology and research in manners of interaction and communication. The current state-of-the-art interaction has multidimensional approaches, and the VR field has made advances in interaction between users, environments and information. The next steps in interaction applied to VR will need to integrate more senses, even in an experimental field. Virtual reality will also have two main approaches: experimental, in which we will be able to develop experiences that can be imagined, (whenever technology allows it) and experiential, where developers will imagine experiences and find the technological solution to create and conceive them. Experiences, in the future, may help more to evolve VR and ASA reality fields, among others.

Thus, as argued in the paper, VR is a medium with its own language with a close relationship to experience itself, and sensorial and cognitive approaches that should be explored in future research.

6. Conclusions

Nowadays, immersive environments and interactive technologies are a research domain with applications in various fields of knowledge, as well as educational approaches and even as a tool for social change. It is necessary to continue building theoretical approaches regarding narratives and storytelling for these new media, as well as to go further into research in areas such as interaction design, human-computer interaction, user experience, user interface, and affective computing, among other areas. Thus, virtual reality is an emerging and continuously evolving medium where it is necessary to explore the possibilities for communication and interaction offered by current technology, and its power to develop immersive experiences regarding the possibilities of integrating human senses into it. This way, the evolution of VR in short-term approaches will be evidently technological, but, also, connected to our ability to conceive new stories and to make users interact with it, and imagine innovative approaches and applications of a technology with a relevant potential to represent ideas and concepts as we’ve never seen before. Thus, virtual reality is a technology, of course, but also a medium, a communication and narrative research field, and a conceptual idea in some ways and, even, in the future, an immersive alternate reality which transmits sensations beyond visual, aural, haptic or proprioceptive ones, maybe one step further in the transformation of reality.

On the other hand, many questions emerge related to the virtual reality and immersive environments phenomena. These questions are the next steps in research in the field, and should be approached by disciplines such as neuroscience (and cognitive science), ergonomics, life science or even formal sciences such as systems or decision theories, as well as theoretical aspects of computer science, among others. In this line of natural evolution of the medium and the technology, we need to think about the next steps in the development of simulating alternate sensations as the natural next milestone in virtual reality. Questions, such as how can we experience the sensations (or subjective perceptions) of other people (or even animals), or how can we create and feel simulated sensations (not only see, but stay and listen), will be, sooner or later, part of the theoretical framework of the virtual reality approach.