Telepresence Social Robotics towards Co-Presence: A Review

Abstract

Telepresence robots are becoming popular in social interactions involving health care, elderly assistance, guidance, or office meetings. There are two types of human psychological experiences to consider in robot-mediated interactions: (1) telepresence, in which a user develops a sense of being present near the remote interlocutor, and (2) co-presence, in which a user perceives the other person as being present locally with him or her. This work presents a literature review on developments supporting robotic social interactions, contributing to improving the sense of presence and co-presence via robot mediation. This survey aims to define social presence, co-presence, identify autonomous “user-adaptive systems” for social robots, and propose a taxonomy for “co-presence” mechanisms. It presents an overview of social robotics systems, applications areas, and technical methods and provides directions for telepresence and co-presence robot design given the actual and future challenges. Finally, we suggest evaluation guidelines for these systems, having as reference face-to-face interaction.

1. Introduction

Telepresence robots are becoming popular in the context of social interactions. Typically, these systems enable people to look at a distant place via teleoperating a robot and interacting with another person at a remote location using the built-in communication devices. Some relevant applications include health care, elderly assistance, autism therapy, guidance, and office meetings [1,2,3,4,5,6,7].

This literature review aims to gather knowledge to help roboticists design improved user- and environment-adaptive systems and technical methods that contribute to enhancing the sense of presence or co-presence via social robot mediation. Reviews have addressed user-adaptive systems [2,8] and environment-adaptive systems [9] for social robotics (in which the robot is generally an autonomous agent serving the bystander user). However, we further explore telepresence social robotics, with an emphasis placed on the relationship between the robot’s operator and the bystander user.

Within social telepresence robots interactions, two types of human psychological experiences can be considered (see Figure 1). The first one involves the remote user, in which he or she should sense being in the local environment (i.e., telepresence) [10,11], and the second type involves the local user, in which ultimately he or she should sense that the remote user is with him or her in the local environment (i.e., co-presence) [12,13]. This research will focus on this last type of interaction, or how to enhance the sense of co-presence via robot mediation. To clarify the role of each agent in the interaction, the following terminology is adopted:

• Mobile robotic telepresence (MRP) system: remotely controllable mobile platform with video conferencing equipment that allows remote users to navigate within a local environment and socially interact with other persons. These systems can incorporate semi-autonomous functionalities to mitigate operation loads such as navigation aids, points to follow, and obstacle avoidance.
• Robotic telepresence (RP) system: remotely controllable or semi-autonomous robotic device with video conferencing capabilities that enable social interaction with people in the local environment without locomotion means. Remote users can explicitly control parts of the robot (e.g., the head’s panning, swinging, tilting, eye gazing, and facial expressions, as well as arm or hand gestures) or enable some semi-autonomous behaviors (e.g., blinking, face tracking, eye saccade, and breathing).
• Remote user: user that steers the robot from a distant location or simply connects to the robot through a computer interface.
• Local user: user that shares the physical environment with the robot (bystander).
• Local environment: environment shared by the local user and robot.

Presence is often defined as the sense of being there in a mediated environment [14,15]. Additionally, Sheridan [16] differentiates presence (virtual) from telepresence (experiential). Presence describes the experience of being present within a virtual world, while telepresence refers to the sense of being in a mediated remote real environment. Co-presence has been used to refer to the sense of being together with others in a mediated (either in remote real or virtual) environment [13,17,18,19].

Marvin Minsky introduced the telepresence concept in the teleoperation context to describe the phenomenon in which a human operator feels physically present at a remote location through interaction with the human’s sensing systems [11] (i.e., “through actions of the user and the corresponding perceptual feedback provided by the teleoperation technology”) [10].

Paulos and Canny [20] developed one of the first telepresence robots and referred to it as a personal roving presence (PRoP) device. The goal was to “provide a physical mobile proxy, controllable over the Internet to provide tele-embodiment”. The system consisted of a simple controllable mobile platform with a video conference set-up (microphone, speaker, and a video camera with 16x zoom and a 30-cm screen on the top of a plastic pole). Additionally, the robot enabled simple gesturing through a two-DoF pointer. They introduced the concept of tele-embodiment in the robotics context to describe the sensation of embodiment of a human in a real distant location [21]. Tele-embodiment was defined as telepresence with a personified perceptible body [22]. However, they did not address key conditions such as body ownership [23] or agency [24]. Li [25] surveyed and compared 33 experimental works involving people’s interactions with virtual agents, telepresence robots, and co-present robots, concluding that robots are more persuasive and positively perceived when they are physically present in the user’s environment.

Short [12] introduced the concept of social presence, defining it as the degree of salience of the participants involved in an interaction and their interpersonal relationship. He mentioned that social presence relied on two concepts: intimacy and immediacy. Intimacy senses the degree of connectedness between the interactants, and immediacy refers to the psychological sense of togetherness between the communicators. Taking face to face (FtF) as the reference, both concepts are determined by a set of verbal and nonverbal cues such as vocal cues, gestures, facial expressions, and physical appearance. The capability to deliver such cues differs from communication means, so Short considered social presence as the quality of the medium itself. Later, Biocca [13] referred to social presence as the effect on one person’s behavior caused by the presence of another or caused by knowing that he or she could be observed. Co-presence, defined as the “psychological connection to and with another person” [17,26], has been explored in several works [27,28,29,30].

Cognitive robotics aims to provide robots with intelligent behavior through a processing architecture that involves perception, long- and short-term memory, learning, and reasoning. These approaches try to deal with people’s behavior unpredictability and with real-world complexity. Cognitive technologies are a form of hyper-automation that may combine areas such as symbolic representation, automation, prediction, user-adaptive systems, computer vision (CV), machine learning (ML), deep learning (DL), or artificial intelligence (AI) [2,9,31]. Nevertheless, the use of AI methodologies to emulate or interpret human subjective experiences, such as emotions, should be inspired by neurophysiologic-psychological foundations [32].

An inner issue related to teleoperated telepresence robots is the time delay issue (mainly due to the communication channel and less due to the hardware performance). This can affect synchronicity (rate of message exchange between operator and bystander), compromising the social presence [33] (e.g., degradations in audio and video streams, control streams, and haptic feedback). Problems regarding latency, bandwidth limitations, and channel corruptions should be mitigated, and while early solutions involved user interface design and control theory-based models (e.g., supervisory control or passivity-based teleoperation), the approaches evolved to predictive displays and control. Advanced solutions for time delay issues are using time series prediction methods to predict the time delay, a robot’s movements, and user intentions. These new adaptive-based control methods make use of nonlinear statistical models and neural network (NN) or machine learning (ML) techniques (e.g., recurrent neural networks, sequence to sequence, long short-term memory, or generative adversarial networks) [31].

The method for this literature review and article selection consisted of retrieving and collecting review studies on social presence, co-presence, and the principles and heuristics of human–robot interaction (HRI), with emphasis on teleoperated telepresence robots. Searches were performed on bibliographic scientific databases, such as ACM’s digital library, Google Scholar, MIT Press Direct, Elsevier’s ScienceDirect, IEEE’s Xplore, PubMed, Scopus, and Springer. Queries included general keywords such as social robots, social robots survey, co-presence taxonomy, copresence or co-presence robots, telepresence robots, adaptive systems, and more specifically, the compositions of these keywords. The selection of papers for in-depth reading was determined by the number of citations, being a recent publication, being a journal (e.g., IEEE transactions, Elsevier’s, or Springer), being a book, including user evaluations studies, or being an article in a reputable conference in the field (e.g., ACM HRI conferences, IEEE Robotics and Automation Society, ICRA, or the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)). Citations in these papers directed new readings and paper selections. Figure 2a depicts the article’s citation distribution per main topic, Figure 2b is from the Co-Presence Taxonomy/Preditors topic, Figure 2c is from the From Telepresence to Co-Presence Design topic, and Figure 2d shows the article citation distribution per year.

This survey presents an overview of social robotics systems and focuses on how to enhance the sense of co-presence via robot mediation. It reviews the literature to define social presence and co-presence, identifies predictors, and proposes a taxonomy for “co-presence” and “user-adaptive systems” mechanisms. It provides technical methods to support robotic social interactions. The structure of this article is composed of four parts. Section 2 identifies potential predictors for social presence, suggesting a taxonomy for “co-presence”. Section 3 presents several robotic telepresence systems currently available in the market or used in research. It also reviews autonomous user-adaptive systems for social robots, aiming for a taxonomy, and additionally provides design guidelines for mechanisms that enhance the sense of co-presence in communications through a teleoperated telepresence robot. It includes guidelines for the evaluation of these systems, having as reference the face-to-face interaction. Finally, Section 4 presents the conclusions and future work.

2. Co-Presence Taxonomy

Social presence has been defined as the sense of being together with another, which includes primitive reactions to social cues and automatic creation of simulations or mental models of “other minds” [13]. Short et al. [12] defined social presence as “the degree of salience of the other person in the interaction and the consequent salience of the interpersonal relationship”.

Co-presence is a different concept, introduced by Goffman [26] to describe the active state in which a person perceives his or her interlocutor, and the interlocutor also perceives him or her. Copresence refers to a “psychological connection to and with another person”, in which “interactants feel they were able to perceive their interaction partner and that their interaction partner actively perceived them” [17]. With co-presence being a subjective concept, it involves different dimensions and interpretations depending on the social science discipline and application area (e.g., sociology or psychology) [18,29,30].

Social presence appears in the literature as being related to the quality of the communication’s medium [12] and the user’s perception of the medium. Therefore, preliminary studies have focused on the effect of modality on social presence. They identified potential predictors of social presence by analyzing the technology’s capability to reproduce social cues (e.g., visual representation, audio, and haptic feedback). The findings were biased by the considered concept definitions. Some predictors contribute directly to presence, co-presence, or social presence, while others affect them indirectly by acting on a person’s involvement and immersion. Therefore, it is important to distinguish the “immersion” concept and the “presence” concept [15,34].

Immersion, also known as sensorimotor immersion, refers to the extent and fidelity of physical stimulation affecting the human sensory systems and the system’s responsiveness to the motor inputs. The immersive level depends on the number and range of sensors and the motor channels connected to a remote agent in a real environment (e.g., a robot) or to a mediated virtual environment. Immersion is determined by the naturalness and coherence between actions (head, body, and gesture movements) and the expected sensory feedback [35,36,37,38].

Presence is the psychological product of technological immersion, defined as the perceptual illusion of non-mediation [39] or simply referred to as the sense of being there in a mediated virtual environment [15,34]. Sheridan [16] differentiates presence (virtual) from telepresence (experiential), in which presence describes the experience of being present within a virtual world while telepresence refers to the sense of being in a mediated remote real environment [19,40].

Co-presence has been used to refer to the sense of being together with others in a mediated environment, either remote real or virtual [18,41]. As described in the definitions, the use of concepts such as co-presence and social presence should not be confused as they are assessed differently [17].

In the context of social robotics, there are agents with autonomous and semi-autonomous behaviors that are seen by the local person as the “other”. Additionally, some agents simply mediate the communications between two persons (the remote and local users). In the former case, the sense of co-presence is assessed between an artificial system and a person, while in the second scenario, co-presence involves two humans. Typically, in robotic telepresence, the representation of the remote real person is shaped by the technology that mediates communication. This affects the perception of thoughts and emotions when compared with actual face-to-face (FtF) interaction. Such representation of remote humans may be supported through text, images, video, 3D avatars, 3D reconstruction, virtual human agents, computers, and robots. Zhao [18], Cummings [33], and Oh, Bailenson, and Welch [29] reviewed the concepts of social presence and co-presence, and their studies suggest a classification for co-presence predictors. This paper adopts some of these literature predictors, framing them in the context of telepresence social robotics.

To unveil a list of technological predictors of social presence, the authors of [33] performed a literature review of empirical studies and grouped them according to similar manipulations. They performed a bottom-up analysis process and identified the following predictors (

Table 1. Technological predictors.

Technological predictors of social presence [33]	Behavioral realism.
	Anthropomorphism.
	Perceived agency of interactant
	Level of embodiment
	Synchronicity
	Inclusion of imagery
	Inclusion of imagery (dynamic)
	Inclusion of voice
	Inclusion of haptic feedback
	Others

):

Initial studies were centered on immersive qualities, but the recent literature also began to address contextual and individual factors, given the subjectivity of the social presence concept [29]. Nevertheless, studies on technological predictors dominate the literature, enlarging the immersive qualities class.

The categorization of predictors that affect social presence or co-presence, based on related works, point to (1) immersive qualities, (2) contextual and social properties, and (3) individual traits (see

Table 2. Categorization of predictors.

	Immersive qualities	Modality, visual representation, interactivity, haptic feedback, audio quality, depth cues, video and display.
Co-presence factors	Contextual and social properties	Personality / traits of virtual human, agency, physical proximity, task type, social cues, identity cues.
	Individual traits	Demographic variables, psychological traits.

).

2.1. Immersive Qualities

2.1.1. Modality

The first studies on social presence analyzed the effect of the modality on the levels of presence achieved, given that the immersion degree varies. These studies on general modality identified technological features with an impact on the social presence (e.g., visual representation, interactivity, depth cues, audio quality, and display). However, medium communication comprises multiple features, and it is a challenge to discriminate the contribution of each affordance. In [42], the media richness theory refers to varying the technological qualities of the medium affording distinct levels of social presence. General modality was also identified in [43] as a predictor of telepresence while analyzing the influence of immersion. Initial studies analyzed the influence of modality on social presence by comparing (1) Face-to-face (FtF) real interactions with computer-mediated communication (CMC), (2) text-based CMC with mediums supporting visual and audio modalities, and (3) immersive virtual environments with non-immersive virtual environments.

Face-to-face (FtF) interaction is considered the ground truth for social presence [44], and several works compare face-to-face (FtF) interaction and CMC, evaluating the capability of these mediated communications to elicit social presence. In general, these studies reveal that the sense of social presence is higher in an FtF interaction when compared with CMC conversation. Cortese et al. [45] designed a task in which participants had to discuss a news article for 20 min, either with FtF interaction or through computer-mediated communication (CMC) (chat). Communication apprehension was one of the psychological factors to be assessed (i.e., “the level of anxiety or fear associated with either real or anticipated communication with another person or persons”). They found that the CMC participants experienced a lower level of social presence. Researchers assessing the sociability of a partner and the level of co-location again found higher social presence levels for FtF interaction.

In studies involving decision-making scenarios [44] and in online learning achievement [46], the results privileged FtF interaction. One study [47] involving a series of online seminars for 2 months (the same teacher teaching the same contents online and via in-person, FtF interaction) reported no differences in the levels of social presence between both forms of interactions. One justification might be related to the fact that students had enough time to adapt their communication skills to an online learning platform, the evolution of e-learning technologies, and the fact that the students felt more comfortable not moving to a classroom for 2 months. However, this study did not address the characterization of the subjects’ ages in their concluding remarks, which could reveal a tendency.

Video and audio modalities guarantee higher degrees of social presence when compared with text-based CMC. However, this difference is not so clear when comparing video-audio modality against audio-only modality. Studies have shown that the introduction of video modality increases the social presence feeling if participants are required to perform visual tasks [48,49]. In studies that compared video-audio vs. audio modalities when involving tasks that do not require visual feedback, such as interview tasks or decision-making tasks, the researchers did not report a significant difference in the social presence [50].

These studies suggest that increasing the quality of an immersive component, such as a video feature, may not be proportional to the social presence felt. There seems to exist a threshold from which further enhancements of a given modality may not produce an additional contribution.

Table 3 summarizes the relevant aforementioned and next predictor’s references, their significant conclusions, and insights on statistics comparisons.

2.1.2. Visual Representation

In communication, the visual representation of interactants is a feature with an impact on social presence. Research has explored to what extent a representation form of the partner can contribute to the sense of social presence. Typically, studies manipulate (1) inclusion or no inclusion of visual representation and (2) the level of realism of the visual representation. The authors of [51] defined realism as the extent to which a digital human representation behaves and appears like a real human. The overall concept is referred to as being based on three components: photographic, anthropomorphic, and behavior’s communicative realism.

The photographic component assesses the human-like visual appearance in a representation. Most studies report that the existence of a visual representation of the partner leads to higher social presence levels. In [52], participants who spoke with their partners through an avatar while shopping in a virtual mall felt a higher social presence in comparison with those who talked without seeing any partner representation. In [53], in an online support-seeker activity, users reported a higher sense of social presence when a profile picture of the counselor was present, as opposed to to not having a picture. The users also demonstrated a higher willingness to answer questions when the pictures were available.

Anthropomorphism contributes to communicative realism because physical attributes such as the mouth, eyes, arms, and legs are involved in speech to generate facial expressions, gestures, and movements. It addresses the level of interpretation of what is not human or personal in terms of human or personal characteristics. Apart from behavioral realism, this manipulation focuses on the degree to which interactants are presented as human-like on the visual and auditory plane. For instance, users would interact via video as opposed to users who would interact via motion capture-controlled cartoons or by means of anthropomorphic agents or avatars vs. animated forms or emoji.

Communicative realism addresses the degree to which a digital representation of the partner presents physical and social human-like behavior (e.g., breathing, natural blinking, and posture changes). Behavior realism studies manipulate the presence or absence of nonverbal behavior (e.g., animation) or the degree to which the nonverbal behavior of a virtual human resembles a real human (e.g., with or without eye gazing). The effect of communicative (behavioral) realism is more evident when the behavior of an agent or avatar reflects the awareness of the partner’s presence (e.g., nodding at the right time, mutual gazing, or blushing). Von der Pütten et al. [54] found that nodding the head of a computer-controlled agent during an interaction contributed to a higher degree of social presence in opposition to no nodding. In another study [55], the participants of an interaction with a virtual agent reported a higher level of social presence when they saw the agent blushing, a consequence of some mistake during a presentation. Study 1 in [50] found that the participants felt higher levels of social presence when the partner’s representation (e.g., avatar) was able to maintain a mutual eye gaze in opposition to the absence of eye gazing. However, Study 2 in [50] realized that maintaining a mutual eye gaze for too excessive a time for video and avatars decreased social presence (i.e., an unnatural behavior). Bent’s studies carefully tracked participants’ nonverbal behavior using the head’s orientation and position sensors, eye gaze trackers, a breath-monitoring chest belt, and data glove-based finger movement trackers. In the avatar’s mediation condition, the tracked data were used to animate the avatars in real time (head and body movements, eye movements, and hand and finger movements). Their findings showed a similar activity in terms of visual attention and nonverbal activity either in video or in avatar conditions, contributing both positively and quite similarly to eliciting social presence. This suggests that avatars can be used as a tool to assess social presence, with the advantage that they enable behavior cue segmentation. Another interesting fact is that users tend to direct their heads to their partner’s image but their gazes towards the workspace. A justification for this behavior, even knowing that it is a computer representation of the partners, might be related to the human’s unconscious social etiquette, that being to keep the face directed to the interaction partner.

Studies show that behavioral realism tends to contribute consistently and positively to social presence [28,30], while photographic and anthropomorphic realism presents varying effects (positive [56], neutral [50,57], or even negative contributions [17]). The justification for these discrepancies might be related to several facts reported in the literature: (1) photographic realism is not the main contributor to social presence (i.e., the appearance of the visual representation has a secondary role in comparison with cues from social behavior) [58], (2) manipulations of small features of the visual representation may not be reflected in social presence questionnaires, and (3) the degrees of behavioral realism differ from study to study, making a quantitative comparison with photographic realism vary [59].

In [60,61], the researchers evaluated the effect of visual and behavioral realism on the perceived quality of communication using avatars. They found a positive effect on social presence when there was consistency between both realism components [60]; that is, although visual representation does not represent a major contribution, the participants felt a greater social presence when the avatar, demonstrating more realistic behaviors (e.g., inferred vs. random eye gaze) which were complemented by a higher level of photographic realism (avatar with a human-like face instead of a dummy face). Bailenson [61] pointed out the consistency between photographic and behavioral realism as a positive social presence predictor.

In [62], the effect of the 3D avatar type (character-like vs. realistically reconstructed) on users’ trust and co-presence in a mixed reality-based collaborative teleconference was explored. Visual representation based on realistically reconstructed avatars has been shown to elicit the user’s sense of co-presence.

In [30], virtual humans that demonstrated higher responsiveness to events (behavioral realism) contributed positively to the co-presence in mixed reality environments. Experimental conditions involved remote collaboration, in-person collaboration, and communication interactions via mixed reality, augmented reality, virtual reality, video chat, text apps, and virtual assistants.

2.1.3. Interactivity

The definitions of social telepresence, in the context of robot-mediated interaction, rely on the capability to put forward the robot operator’s presence to the local person (bystander). Additionally, the extent to which this person is aware that he or she is talking to or interacting with a human being has an impact on social presence [15] and co-presence [63]. Studies on this subject try to understand the effect of the interactivity of the agent on social presence. Such interactivity may refer to a computer agent, a person’s avatar, or a telepresence robot, but the focus of this analysis is on the use of a telepresence robot for conversation mediation. Thus, the level of social presence depends on the fidelity of the medium to support the interactivity that characterizes persons’ conversations. It includes visual and audio cues, nonvisual sensing (i.e., directional sound and haptics like force feedback and touch) and environmental interactivity (e.g., response rate to user input, reciprocity of the interaction capability between the remote user and local user, and clarity of causal relationships between remote user actions and local user reactions).

2.1.4. Haptic Feedback

To improve the sense of reality, it is important to provide some type of physical feedback to the operator or bystander. Useful contributions include providing tactile cues to let the user recognize the surface texture and materials or support kinesthetic feedback to help the user experience the weight of a virtual object. These kinesthetic and tactile sensations enable haptic perception.

Haptic feedback is a challenge. However, it may improve the degree of presence considerably. Considerable progress has been made in the field of visual and auditory displays, but haptic feedback is in its early stages, gaining much attention nowadays [64,65,66]. Touch contact plays an important role in human interactions. From an early age, babies explore their surroundings with their hands and feel physical contact with parents holding them, and at older ages, handshakes, kisses, and embraces trigger emotions and strengthen relationships. Nevertheless, the physical contact of a robot with a person raises safety issues, and that may justify why haptic feedback is not so prevalent.

Touching a robot’s part (e.g., hand or body) or sensing that component pulling our hand, operated by a remote party, can improve co-presence. Remote hand-shaking has been explored [67], and examples include the Nao robot hand-shaking the bystander while the robot’s operator uses a low-cost haptic device (WiiMote) to feel it [68]. In [69], a robot hand was attached under a videoconferencing terminal’s display, and their evaluation demonstrated that mutual touch enhances the feeling of being close. However, the partner’s action should not appear in the video. Gregory Welch et al. [70] developed a tactile telepresence system prototype that enables a remote visitor to convey touching patterns on the forehead of an isolated patient through a tablet touch video interface. Regarding human–robot first encounters [5] and greetings, the authors of [71] used Kendon’s model [72] to develop an interaction that included six phases (initiation of approach, distance salutation, head dip, approach, final approach, and close salutation). Human tracked gestures were the inputs for a decision module (based on the hidden Markov model and behavior tree [73]) that initiated a specific phase at the right moment.

Telexistence surrogate anthropomorphic robot (TELESAR) VI can mimic the user’s movements and gestures from a mechanically unconstrained full-body master cockpit and provide haptic feedback to the operator [74]. The 10 fingers of the teleoperated robot are equipped with vibration, force, and temperature sensors that can realistically deliver these components of haptic information. Operators can shake the hand of another person through the robot and feel it.

2.1.5. Depth Cues (Stereoscopy and Motion Parallax)

Considering an interaction between two persons through a teleoperated robot, the depth cues become more important for the remote user, since the local user is with the robot, and has natural depth cues. On the other side, if the remote user can perceive the local user in a 3D space, it improves the scene’s realism and the co-presence. The use of 3D displays or head-mounted displays (HMDs) by the remote user are common approaches to delivery depth cues. However, this requires 3D sensors in the robot’s side (e.g., stereo cameras and RGBD sensors). Additionally, with the inclusion of an autostereoscopic or 3D display in the robot to present the remote user to the local user, it is possible to enhance the closeness [75,76,77].

2.1.6. Audio Quality

As mentioned earlier, audio modalities guarantee higher levels of social presence. The audio channel should provide bidirectional communication between the remote and local users to exchange messages. Recognition of a person’s voice plays an important role in person identification, contributing to the sense of co-presence [78]. Voice transmission is expected to be fluid without cuts or delays. Telepresence robots quite often make use of an array of microphones to acquire spatial sound, enabling the remote user to identify the direction of the sound source [79] or simply detect the movements of the local user.

2.1.7. Video and Display

The sense of being telepresent is also determined by the fidelity and capability of the medium to present the remote environment, including the visualization of persons (face expressions, gestures, postural behaviors, etc.). To this end, there are mediation technology requirements that include visual display parameters (e.g., latency, frame rate, field of view (FOV), point of view (egocentric vs exocentric), image resolution, color quality, and image clarity) and environment presentation consistency across displays [80]. Display type comparisons reveal a positive effect on co-presence using immersive 3D displays in nonverbal interactions [81]. For example, the Willow Garage Texai robot rely on the principle “reciprocity of vision (if I see you, you must see me)”, while Excite robot designers defend that “The visitor’s [user’s] environment should be immersive so that the user would have a first-person experience of the destination [remote environment] including full sensory stimulation focusing on immersive vision, audio, and haptics” [3].

2.2. Contextual and Social Properties

Early studies on predictors of social presence focused on immersive qualities; however, the research began to address contextual and individual properties. Given the subjectivity associated to social presence experience, and aside from the physical distance and the medium’s technological qualities, analyses started to consider a psychological distance between the interactants [29,82,83]. These include factors such as Personality/traits of Virtual Human, Agency [17], Physical Proximity [57,59], Task Type [48,84,85], Social Cues [27,48,86], or Identity Cues [27].

In [30], a contextual responsiveness predictor is explored that assesses the capability of a virtual human (VH) to detect and respond to events and cues that happen in the shared space of the VH and the user (e.g., to a broom that falls in the user’s physical environment or that falls into the virtual VH space). They showed that when the VH detects and directs gazing at the event or orients itself in that direction, the user presents higher levels of co-presence. Studies suggest that users’ perception of the physical space affects their co-presence in mixed reality. Ignoring events in the background, such as objects moving or a person walking [87], or the inability to shift attention to an external event does not contribute to co-presence. In [62], the robot that plays a game with the user uses a “cheat” function to trick the user, which affects the user’s trust, contributing positively to co-presence.

In [27], users reported a higher level of social presence when communicating simultaneously with several remote interlocutors through a telepresence robot than with a single remote person. In [27], a second study showed that users felt the presence of the remote interlocutor more when the telepresence robot had a low identity than a higher identity (e.g., robot’s head LCD with or without a face drawing).

2.3. Individual Traits

Gender and age: social studies showed that female subjects tend to experience higher degrees of social presence when compared with males [57,88], but age is not a relevant factor [89].

Attractiveness: in [59], a human’s avatar that looks more attractive in a virtual mirror raised the person’s level of self-confidence in the next encounters with other person’s avatars and eventually in the real world (distances between avatars are reduced (proxemics)). Such findings provide traits for telepresence and co-presence robot design.

Height: in [59], a human’s avatar that looked taller than its interlocutor tended to make that person more persuasive in new interactions with others.

Psychological traits: a person with a higher immersive tendency showed higher degrees of social presence [89]. Additionally, people more prone to human social interactions reported higher levels of social presence in experiments involving social robots [90]. In [45], persons low in communication apprehension (CA) experienced higher levels of social presence than those high in CA. Less sociable people tended to show lower scores on social presence assessments.

Table 3 summarizes the relevant aforementioned and next predictor’s references, their significant conclusions, and insights on statistics comparisons (e.g., N = number of subjects, $\mu$ = mean, $\sigma$ = standard deviation; subscripts refer to the condition, where superscript ${}^{+}$ = significant condition, $df$ = degree of freedom, F = ANOVA statistic F, p = p-value, ${\eta }^2$ = eta squared, ${\chi }^2$ = chi square, $\beta$ = standardized path coefficient, and r = correlation coefficient).

Table 3. Co-presence studies.

Predictor Category	Predictor	Evaluation Process	Study	Quantitative Comparison (Statistics)
Immersion	Modality	FtF vs. CMC (NetMetting teleconference)	[44]	$N=70$ (38 pairs), ${\mu }_{FtF}=34.6$, ${\mu }_{CMC}=32.1$, $F=40.2$, $df=1$, $p=0.00$
Immersion	Modality	FtF vs. CMC (chat)	[45]	$N=152$, ${\chi }^2(8,N=152)=6.267$, $p=0.617$; ($\beta =-0.948$, $p<0.001$)
Immersion	Modality	FtF vs. CMC	[46]	$N=257$, ${\mu }_{FtF}=3.63$, ${\sigma }_{FtF}=0.62$; ${\mu }_{CMC}=3.48$, ${\sigma }_{CMC}=0.57$; $t\left(255\right)=2.077$, $p=0.0039$
Immersion	Modality	FtF vs. CMC (online teaching and learning)	[47]	$N=50$, ${\mu }_{FtF}=38.9$, ${\sigma }_{FtF}=1.2$; ${\mu }_{CMC}=36.91$, ${\sigma }_{CMC}=1.36$; $F(1,48)=1.194$, $p=0.28$
Immersion	Modality	Audio vs. Audio + Video	[48]	$N=34$ (17 pairs), male: ${\mu }_{audio}\approx 53.5$, ${\mu }_{audio+video}\approx 71.75$; $F(1,18)=9.9$, $p=0.04$
Immersion	Modality	Text vs. Audio vs. Audio + Video vs. Audio + Avatar	[50]	$N=150$, Factor scores: ${\mu }_{text}=-0.48$, ${\mu }_{audio}=0.26$, ${\mu }_{audio+video}=0.22$, ${\mu }_{audio+LFavatar}=0.09$; ${\mu }_{audio+HFavatar}=0.10$; $F(4,137)=2.59$, $p=0.04$, ${\eta }_p^2=0.09$
Immersion	Visual Representation	Photographic Realism (Low- vs. High-Fidelity Avatar)	[50]	$N=150$, Factor scores: ${\mu }_{audio+LFavatar}=0.09$; ${\mu }_{audio+HFavatar}=0.10$; $F(4,137)=2.59$, $p=0.04$, ${\eta }_p^2=0.09$
Immersion	Visual Representation	Photographic Realism	[52]	$N=80$, embodiment index: $voice=1.68$, $voice+avatar=5.2$, $df=4$, $p<0.01$, ${\mu }_{embodiment}=3.41$, ${\sigma }_{embodiment}=1.94$; ${\mu }_{copresence}=5.27$, ${\sigma }_{copresence}=1.44$;
Immersion	Visual Representation	Photographic Realism	[57]	$N=50$, ${\mu }_{flat\_shaded\_face}\approx {\mu }_{photographic\_texture\_face}$
Immersion	Visual Representation	Anthropomorphic	[17]	$N=134$, copresence index: ${lo{w}_{anthropomorphic\_image}}^{+}$, $mor{e}_{anthropomorphic\_image}$, $n{o}_{image}$, $R=0.18$, $F=4.23$, $p=0.04$
Immersion	Visual Representation	Anthropomorphic, Behavioral Realism	[28]	Definitions and it uses, digital representations
Immersion	Visual representation	Behavioral realism (mutual gaze)	[57]	$N=50$, women’s social presence score: ${\mu }_{no\_mutual\_gaze}=-13.25$, ${\sigma }_{no\_mutual\_gaze}=18.58$; ${\mu }_{high\_mutual\_gaze}=2.5$, ${\sigma }_{high\_mutual\_gaze}=15.55$; 5 conditions ($r=0.30$, $p<0.03$)
Immersion	Visual Representation	Consistency between Visual and Behavioral Realism	[60]	$N=48$, $low\_realism$: ${\mu }_{random\_gaze}=1.2$, ${\sigma }_{ramdom\_gaze}=0.2$; ${\mu }_{inferred\_gaze}=0.7$, ${\sigma }_{inferred\_gaze}=0.2$; $high\_realism$: ${\mu }_{random\_gaze}=0.3$, ${\sigma }_{ramdom\_gaze}=0.1$; ${\mu }_{inferred\_gaze}=1.1$, ${\sigma }_{inferred\_gaze}=0.3$;
Immersion	Visual Representation	Consistency between Visual and Behavioral Realism	[61]	$N=146$, copresence: behavioral realism${}^{+}$, $F(3,133)=2.72$, $p<0.05$, ${\eta }^2=0.06$; visual representation${}^{+}$$F(6,133)=2.18$, $p<0.05$, ${\eta }^2=0.09$
Immersion	Visual Representation	Avatar Behavioral Realism to Events	[30]	$N=65$, copresence: ${\mu }_{responsive}^{+}=4.31$, ${\sigma }_{responsive}=0.11$; ${\mu }_{nonresponsive}=3.96$, ${\sigma }_{nonresponsive}=0.12$;, $\sigma$; $F(1,63)=5.06$, $p=0.02$
Immersion	Visual Representation	HMD vs. Desktop	[80]	$N=21$, presence Q5: ${\mu }_{HMD}=5.28$, ${\sigma }_{HMD}=1.58$; ${\mu }_{Desktop}=3.42$, ${\sigma }_{Desktop}=1.77$; $F(1,20)=26.54$, $p<0.0001$
Immersion	Visual Representation	HMD vs. Desktop	[91]	$N=26$, presence Q5: ${\mu }_{HMD}=5.88$, ${\sigma }_{HMD}=0.52$; ${\mu }_{Desktop}=2.48$, ${\sigma }_{Desktop}=1.75$; $F(4,95)=32.19$, $p<0.0001$
Immersion	Visual Representation	2D vs. 3D vs. Verbal vs. Nonverbal	[81]	$N=40$, copresence: $3D_{nonverbal}^{+}$, $3D_{verbal}$, $t\left(16.35\right)=7.48$, $p<0.05$; $2D_{nonverbal}$, $2D_{verbal}^{+}$, $t\left(17.967\right)=-8.05$, $p<0.05$
Immersion	Interactivity	Whole Body Interaction	[92]	$N=13$, embodiment: immersive+body intention- based robot control${}^{+}$, $F(3,44)=19.11$, $p<0.0001$;
Immersion	Haptic Feedback	Present vs. Absent	[93]	$N=24$, Embodiment score: Haptic feedback${}^{+}$ = 49.8, $F(1,23)=29.67$, $p<0.0001$; Realism score: Haptic feedback = 33.0, $F(1,23)=22.97$, $p<0.0001$;
Immersion	Depth Cues	Stereoscopy (stereo vs. mono)	[94]	$N=144$, copresence: ${\mu }_{stereo}^{+}=3.85$, ${\sigma }_{stereo}=1.34$; ${\mu }_{mono}=3.25$, ${\sigma }_{mono}=1.48$; $F(1,140)=6.97$, $p<0.01$, ${\eta }_p^2=0.05$
Immersion	Audio Quality	Binaural vs. Stereophonic vs. Monophonic	[78]	$N=82$, presence: ${binaural}^{+}$, ${mono}^{-}$, $!2=(4,N=79)=10.7$, $p=0.031$
Immersion	Audio Quality	Attention, Binaural	[95]	active perception (visuo-auditory, vestibular emulation, Bayesian models), $f=6--10\mathrm{Hz}.$
Immersion	Display	Face-to-Face Point of View	[75]	3D capture, maintain face-directed gaze through robot positioning, $f=2.12\mathrm{Hz}$
Immersion	Display	Three 55-inch Screens vs. One 55-inch Screen	[94]	$N=144$, copresence: ${\mu }_{humansize\_display}^{+}=3.94$, ${\sigma }_{humansize\_display}=1.46$; ${\mu }_{smallsize\_display}=3.17$, ${\sigma }_{smallsize\_display}=1.30$; $F(1,140)=11.41$, $p<0.001$, ${\eta }_p^2=0.08$
Immersion	Display	Autostereoscopic Telepresence	[96]	3D capture, 3D display, eye/head tracking, frame rates: 34, 48, 74 Hz
Context	Personality or Traits of Virtual Human	Personality manifested by voice and match between content	[97]	$N=144$, computer voice with a personality (extrovert/introvert) similar to human interlocutor, $F(1,67)=11.13$, $p<0.001$, ${\eta }_p^2=0.14$; $voic{e}_{extrovert}^{+}$, $voic{e}_{introvert}$, $F(1,71)=17.91$ $p<0.001$, ${\eta }_p^2=0.20$
Context	Agency	Avatar vs. Agent	[17]	$N=134$, copresence index: $agenc{y}_{human\_human\_interaction}$≈$agenc{y}_{human\_computer\_interaction}$, $R=0.03$, $F=0.15$, $p=0.7$;
Context	Agency	Conscious experience of being someone	[24]	Illusory self-identification
Context	Agency	Avatar vs. Agent	[82]	$N=90$, $agenc{y}_{human\_human\_interaction}^{+}$, $agenc{y}_{human\_computer\_interaction}$, $F(1,90)=10.870$, $p=0.001$, ${\eta }^2=0.112$
Context	Physical Proximity	Close vs. Distant (spatial proximity)	[98]	$N=134$, male social presence: $st{d}_{p}at{h}_{e}st{location\_accessibility\_cues}^{+}=0.21$, $st{d}_{p}at{h}_{e}s{t}_{richer\_medium}^{+}=0.06$
Context	Task Type	Caregiver: Human vs. Robot	[99]	$N=60$, social presence: ${\mu }_{robot\_as\_caregiver}^{+}=5.56$, ${\sigma }_{robot\_as\_caregiver}=1.04$, ${\mu }_{human\_as\_caregiver}=4.20$, ${\sigma }_{human\_as\_caregiver}=0.83$;
Context	Social Cues	Online Buddy: Present vs. Absent	[85]
Context	Identity Cues	Telepresence Robot, Identity Cues: High vs. Low	[27]
Individual	Demographic Variables	Gender: Female vs Male	[48]
Individual	Psychological Traits	Communication Apprehension	[45]
Individual	Psychological Traits	Belonging Feeling	[99]

Table 3. Co-presence studies.

Predictor Category	Predictor	Evaluation Process	Study	Quantitative Comparison (Statistics)
Immersion	Modality	FtF vs. CMC (NetMetting teleconference)	[44]	$N=70$ (38 pairs), ${\mu }_{FtF}=34.6$, ${\mu }_{CMC}=32.1$, $F=40.2$, $df=1$, $p=0.00$
Immersion	Modality	FtF vs. CMC (chat)	[45]	$N=152$, ${\chi }^2(8,N=152)=6.267$, $p=0.617$; ($\beta =-0.948$, $p<0.001$)
Immersion	Modality	FtF vs. CMC	[46]	$N=257$, ${\mu }_{FtF}=3.63$, ${\sigma }_{FtF}=0.62$; ${\mu }_{CMC}=3.48$, ${\sigma }_{CMC}=0.57$; $t\left(255\right)=2.077$, $p=0.0039$
Immersion	Modality	FtF vs. CMC (online teaching and learning)	[47]	$N=50$, ${\mu }_{FtF}=38.9$, ${\sigma }_{FtF}=1.2$; ${\mu }_{CMC}=36.91$, ${\sigma }_{CMC}=1.36$; $F(1,48)=1.194$, $p=0.28$
Immersion	Modality	Audio vs. Audio + Video	[48]	$N=34$ (17 pairs), male: ${\mu }_{audio}\approx 53.5$, ${\mu }_{audio+video}\approx 71.75$; $F(1,18)=9.9$, $p=0.04$
Immersion	Modality	Text vs. Audio vs. Audio + Video vs. Audio + Avatar	[50]	$N=150$, Factor scores: ${\mu }_{text}=-0.48$, ${\mu }_{audio}=0.26$, ${\mu }_{audio+video}=0.22$, ${\mu }_{audio+LFavatar}=0.09$; ${\mu }_{audio+HFavatar}=0.10$; $F(4,137)=2.59$, $p=0.04$, ${\eta }_p^2=0.09$
Immersion	Visual Representation	Photographic Realism (Low- vs. High-Fidelity Avatar)	[50]	$N=150$, Factor scores: ${\mu }_{audio+LFavatar}=0.09$; ${\mu }_{audio+HFavatar}=0.10$; $F(4,137)=2.59$, $p=0.04$, ${\eta }_p^2=0.09$
Immersion	Visual Representation	Photographic Realism	[52]	$N=80$, embodiment index: $voice=1.68$, $voice+avatar=5.2$, $df=4$, $p<0.01$, ${\mu }_{embodiment}=3.41$, ${\sigma }_{embodiment}=1.94$; ${\mu }_{copresence}=5.27$, ${\sigma }_{copresence}=1.44$;
Immersion	Visual Representation	Photographic Realism	[57]	$N=50$, ${\mu }_{flat\_shaded\_face}\approx {\mu }_{photographic\_texture\_face}$
Immersion	Visual Representation	Anthropomorphic	[17]	$N=134$, copresence index: ${lo{w}_{anthropomorphic\_image}}^{+}$, $mor{e}_{anthropomorphic\_image}$, $n{o}_{image}$, $R=0.18$, $F=4.23$, $p=0.04$
Immersion	Visual Representation	Anthropomorphic, Behavioral Realism	[28]	Definitions and it uses, digital representations
Immersion	Visual representation	Behavioral realism (mutual gaze)	[57]	$N=50$, women’s social presence score: ${\mu }_{no\_mutual\_gaze}=-13.25$, ${\sigma }_{no\_mutual\_gaze}=18.58$; ${\mu }_{high\_mutual\_gaze}=2.5$, ${\sigma }_{high\_mutual\_gaze}=15.55$; 5 conditions ($r=0.30$, $p<0.03$)
Immersion	Visual Representation	Consistency between Visual and Behavioral Realism	[60]	$N=48$, $low\_realism$: ${\mu }_{random\_gaze}=1.2$, ${\sigma }_{ramdom\_gaze}=0.2$; ${\mu }_{inferred\_gaze}=0.7$, ${\sigma }_{inferred\_gaze}=0.2$; $high\_realism$: ${\mu }_{random\_gaze}=0.3$, ${\sigma }_{ramdom\_gaze}=0.1$; ${\mu }_{inferred\_gaze}=1.1$, ${\sigma }_{inferred\_gaze}=0.3$;
Immersion	Visual Representation	Consistency between Visual and Behavioral Realism	[61]	$N=146$, copresence: behavioral realism${}^{+}$, $F(3,133)=2.72$, $p<0.05$, ${\eta }^2=0.06$; visual representation${}^{+}$$F(6,133)=2.18$, $p<0.05$, ${\eta }^2=0.09$
Immersion	Visual Representation	Avatar Behavioral Realism to Events	[30]	$N=65$, copresence: ${\mu }_{responsive}^{+}=4.31$, ${\sigma }_{responsive}=0.11$; ${\mu }_{nonresponsive}=3.96$, ${\sigma }_{nonresponsive}=0.12$;, $\sigma$; $F(1,63)=5.06$, $p=0.02$
Immersion	Visual Representation	HMD vs. Desktop	[80]	$N=21$, presence Q5: ${\mu }_{HMD}=5.28$, ${\sigma }_{HMD}=1.58$; ${\mu }_{Desktop}=3.42$, ${\sigma }_{Desktop}=1.77$; $F(1,20)=26.54$, $p<0.0001$
Immersion	Visual Representation	HMD vs. Desktop	[91]	$N=26$, presence Q5: ${\mu }_{HMD}=5.88$, ${\sigma }_{HMD}=0.52$; ${\mu }_{Desktop}=2.48$, ${\sigma }_{Desktop}=1.75$; $F(4,95)=32.19$, $p<0.0001$
Immersion	Visual Representation	2D vs. 3D vs. Verbal vs. Nonverbal	[81]	$N=40$, copresence: $3D_{nonverbal}^{+}$, $3D_{verbal}$, $t\left(16.35\right)=7.48$, $p<0.05$; $2D_{nonverbal}$, $2D_{verbal}^{+}$, $t\left(17.967\right)=-8.05$, $p<0.05$
Immersion	Interactivity	Whole Body Interaction	[92]	$N=13$, embodiment: immersive+body intention- based robot control${}^{+}$, $F(3,44)=19.11$, $p<0.0001$;
Immersion	Haptic Feedback	Present vs. Absent	[93]	$N=24$, Embodiment score: Haptic feedback${}^{+}$ = 49.8, $F(1,23)=29.67$, $p<0.0001$; Realism score: Haptic feedback = 33.0, $F(1,23)=22.97$, $p<0.0001$;
Immersion	Depth Cues	Stereoscopy (stereo vs. mono)	[94]	$N=144$, copresence: ${\mu }_{stereo}^{+}=3.85$, ${\sigma }_{stereo}=1.34$; ${\mu }_{mono}=3.25$, ${\sigma }_{mono}=1.48$; $F(1,140)=6.97$, $p<0.01$, ${\eta }_p^2=0.05$
Immersion	Audio Quality	Binaural vs. Stereophonic vs. Monophonic	[78]	$N=82$, presence: ${binaural}^{+}$, ${mono}^{-}$, $!2=(4,N=79)=10.7$, $p=0.031$
Immersion	Audio Quality	Attention, Binaural	[95]	active perception (visuo-auditory, vestibular emulation, Bayesian models), $f=6--10\mathrm{Hz}.$
Immersion	Display	Face-to-Face Point of View	[75]	3D capture, maintain face-directed gaze through robot positioning, $f=2.12\mathrm{Hz}$
Immersion	Display	Three 55-inch Screens vs. One 55-inch Screen	[94]	$N=144$, copresence: ${\mu }_{humansize\_display}^{+}=3.94$, ${\sigma }_{humansize\_display}=1.46$; ${\mu }_{smallsize\_display}=3.17$, ${\sigma }_{smallsize\_display}=1.30$; $F(1,140)=11.41$, $p<0.001$, ${\eta }_p^2=0.08$
Immersion	Display	Autostereoscopic Telepresence	[96]	3D capture, 3D display, eye/head tracking, frame rates: 34, 48, 74 Hz
Context	Personality or Traits of Virtual Human	Personality manifested by voice and match between content	[97]	$N=144$, computer voice with a personality (extrovert/introvert) similar to human interlocutor, $F(1,67)=11.13$, $p<0.001$, ${\eta }_p^2=0.14$; $voic{e}_{extrovert}^{+}$, $voic{e}_{introvert}$, $F(1,71)=17.91$ $p<0.001$, ${\eta }_p^2=0.20$
Context	Agency	Avatar vs. Agent	[17]	$N=134$, copresence index: $agenc{y}_{human\_human\_interaction}$≈$agenc{y}_{human\_computer\_interaction}$, $R=0.03$, $F=0.15$, $p=0.7$;
Context	Agency	Conscious experience of being someone	[24]	Illusory self-identification
Context	Agency	Avatar vs. Agent	[82]	$N=90$, $agenc{y}_{human\_human\_interaction}^{+}$, $agenc{y}_{human\_computer\_interaction}$, $F(1,90)=10.870$, $p=0.001$, ${\eta }^2=0.112$
Context	Physical Proximity	Close vs. Distant (spatial proximity)	[98]	$N=134$, male social presence: $st{d}_{p}at{h}_{e}st{location\_accessibility\_cues}^{+}=0.21$, $st{d}_{p}at{h}_{e}s{t}_{richer\_medium}^{+}=0.06$
Context	Task Type	Caregiver: Human vs. Robot	[99]	$N=60$, social presence: ${\mu }_{robot\_as\_caregiver}^{+}=5.56$, ${\sigma }_{robot\_as\_caregiver}=1.04$, ${\mu }_{human\_as\_caregiver}=4.20$, ${\sigma }_{human\_as\_caregiver}=0.83$;
Context	Social Cues	Online Buddy: Present vs. Absent	[85]
Context	Identity Cues	Telepresence Robot, Identity Cues: High vs. Low	[27]
Individual	Demographic Variables	Gender: Female vs Male	[48]
Individual	Psychological Traits	Communication Apprehension	[45]
Individual	Psychological Traits	Belonging Feeling	[99]

3. From Telepresence to Co-Presence Design

Presently, the market [100] offers full solutions for mobile robotic telepresence (MRP) systems [5,6,7,9] (see

Table 4. Mobile robotic telepresence (MRP) systems: full market solutions.

References	Robotic Telepresence Systems	Application Area	Expression or Manipulation	Navigation Features	Cost
[101]	Giraff	Eldery	Head tilt (screen display/camera)	No	USD 11,900.00
[102]	Double 2, 3	Office, education, hospital	Motorized height	Accelerometer and gyroscope for balance, kickstands when static	USD 2749.00
[103]	PadBot 2	Office, education, hospital	Tilt head (screen)	Obstacle detection, collision avoidance, anti-falling system	USD 1297.00
[103]	PadBot U1—v2	Office, education, hospital	Tilt head (screen)	Collision-prevention sensors. Edge detection and anti-falling sensors.	USD 797.00
[103]	PadBot T1	Office, home	No	Collision prevention, cliff sensor	USD 185.00
[104]	Beam Pro	Corporate, manufacturing, medical	No	Crash avoidance, assisted driving	USD 14,945.00
[105]	Ava 500	Healthcare, office	No	2D or 3D imaging, sonars and lasers for autonomous navigation, omnidirectional navigation, scheduling capabilities, cliff sensor	USD 32,000.00
[106]	Ohmni SuperCam	Office, home	No	Includes downward-facing camera for full visibility	USD 2195.00
[107]	VGo	Office, education	Tiltable head	Crash avoidance, notification of obstacles locations, cliff sensor	USD 3995.00
[108]	TeleMe 2	Office, education	Laser pointer option	Crash avoidance: infrared sensors detect obstacles and will automatically reduce robot’s speed	USD 3995.00
[109]	RP-Vita	Healthcare, FDA clearance	Active patient monitoring	Obstacle avoidance, omnidirectional and autonomous navigation	USD 80,000.00
[110]	Teleporter	Office, factory, hospitals	Laser pointer, secondary webcam	Crash avoidance: infrared, 3D, or sonar sensors	USD 14,995.00

), and the research presents telepresence robot solutions such as the ones listed in

Table 5. Mobile robotic telepresence (MRP) systems: research-oriented solutions.

References	Robotic Telepresence Systems	Application Area	Expression or Manipulation	Navigation Features	Cost
[22]	PRoP	Research	Laserpointer, 2 DOF hand and arm	-	-
[111]	FURo-i	Home	No	Bumpers	USD 1800.00
[112]	MeBot	Research	3 DOF neck for screen and 3 DOF arms	Collision prevention, cliff sensor	-
[113]	Origibot	Research	Tiltable head for screen, 1 DOF arm (180${}^{\circ }$), 2 DOF gripper	No	Low cost
[114]	Nao	Research	Humanoid, 25 DOF, tiltable head, arms, legs, 4 directional microphones and speakers, 2 cameras	No	–
[114]	Pepper	Research	DOF (head: 2, shoulder: 2, elbow: 2, wrist: 1, hands (5 fingers): 1, waist: 2, knees: 1, base: 3 wheels), 2D and 3D cameras and sonars	Autonomous navigation, bumpers	USD 30,000.00
[115,116]	GrowMeUp	Eldery research	Robot expression, directional microphones and speakers, 2D and 3D cameras, sonars, touchscreen	Obstacle avoidance, autonomous navigation, service, expression, behaviors	-

. There are also unmovable robotic telepresence (RP) systems, which are listed in

Table 6. Unmovable robotic telepresence (RP) systems.

References	Robotic Telepresence System	Application Area	Expression or Manipulation	Cost
[117]	Kubi	Office, education	Pan 300${}^{\circ }$, tilt 900${}^{\circ }$(screen)	USD 675.00
[118]	TableTop TeleMe	Office, education	Pan 360${}^{\circ }$, tilt head (screen)	USD 3995.00
[119]	SelfieBot	Office, education	Pan 180${}^{\circ }$, tilt 180${}^{\circ }$ head (screen)	USD 195.00
[120]	Meeting Owl Pro	Office	Static 360${}^{\circ }$ camera (1080p)	USD 999
[121]	Robovie mR2	Research	Expression, arms, gestures, eye blinking (cameras), 18 joints (3 in each eye), 18 servo motors	-

. These robotic telepresence systems are depicted in Figure 3 and Figure 4.

3.1. Co-Presence Design

Mechanisms that contribute to enhancing co-presence in telepresence robots should consider the robot-side systems and the remote user (robot’s operator) side solutions. Robot-side interfaces support interactions between the robot and the local user (bystander) and between the robot and the remote user (operator). Human–robot interfaces can be classified into sight, hearing, touch, and body-sensing technologies. Technological advances include robust robot sensory (vision, face and expression recognition, object recognition, activity identification, pressure, touch, temperature, speech understanding, sound localization, etc.), acting (mobility, proxemics, gestures, gazing, facial expressions, speech synthesis, etc.), reasoning (localization, planning, context awareness, grasping, etc.), and appearance (familiar, unfamiliar, human-like, and mechanical) [3,122,123].

3.1.1. Sensing

Robotic sensing technologies are becoming more efficient, lighter, and cheaper. Early human–robot interfaces used to integrate few sensors and relied mainly on video and audio data and low-resolution proximity sensors (e.g., sonars). Current robots can be equipped with 3D or 2D cameras (e.g., low-cost RGB-D cameras), pressure sensors, touch sensors, directional sound sensors (arrays), high-precision proximity sensors (e.g., range laser finder (lidar)), and robot pose and position sensors (e.g., gyroscopes, accelerometers, and GPS). The fusion of these sensors combined with high-accuracy robots, person localization algorithms (e.g., simultaneous localization and mapping (SLAM) or Open Pose), and deep learning approaches, have improved robot operations in an environment, enhancing HRI between operators and bystanders. Valuable information can be extract due to advances in sensor technologies and software, such as sound locations [95,124,125], speech segregation [126,127] and recognition [128,129], attention [79], gesture recognition [130,131], human action analysis [132,133], human intentions [134,135], object recognition [136], and scene understanding [137,138].

3.1.2. Action Capabilities

Advances in robot software and hardware, lighter and stronger materials, component miniaturization, and lighter and more powerful batteries have broadened robots’ capabilities. Robot mobility has improved significantly, enabling robust navigation in an unstructured environment and in rough terrain [139,140], and they can climb stairs, walk fast, and run, such as the Boston Dynamics ATLAS robot [141] or Honda ASIMO Robot [142]. Advances in humanoid mobility and equilibrium are remarkable, including compliant interactions and variable speed [143]. Having arms, hands, and fingers with more degrees of freedom (DOFs) enabled new types of interactions such as high-fidelity gestures, grasping objects smoothly [144], or even open doors and pass though them [141,145]. Whole-body expressive movements [146], facial features to support expression synthesis [147], and speech synthesis technologies are enabling better HRIs.

3.1.3. Reasoning

Robots are designed to perform several tasks, but task execution is not always perfect (e.g., motion constraints, impaired sensory, control, and communication delays). Thus, advancements in software reasoning processes have been developed to supervise tasks, aiming not for perfect execution but optimum performance. Namely, notable advances have been made in localization and mapping [148,149] and in grasping [144]. In [150], the authors explored approaches for a telepresence robot to detect and position itself with a group of people for social interactions (maintaining an egocentric perspective). The inclusion of these autonomous algorithms can help operators and bystanders in their interactions with the robot, simplifying the control, reducing the effort, and improving the intuitiveness.

3.1.4. Appearance

The acceptability of the robots enrolled in a human assistive task also depends on their appearance. Designers have created robots with human-like appearances [151]. The Geminoid robot has an incredibly realistic head and facial features [152]. This approach enables more effective communication through facial expressions and natural gestures. Additionally, given the human-like robot morphology, it is simpler to map the human gestures and movements in the robot. The search for realism, however, suggests some warnings regarding Mori’s “uncanny valley” [153,154]; that is, if a robot or agent is an imperfect replica of a human being, people may feel defrauded in their expectations regarding the affinity as a pair, triggering strange, familiar feelings of unease and revulsion.

3.1.5. Managing Robot Autonomy in Telepresence System

Advances in robot autonomy do not eliminate the role of human operators. Human skills remain crucial in an unstructured environment or when dealing with unpredicted events. The integration of autonomous mechanisms aims for process simplification, and it changes the nature of human–robot interaction (HRI). However, there are cases where the complexity increases [155] (e.g., 2019 Boeing 737 Max autopilot problems with deadly consequences). The availability of automated behaviors for telepresence or in humanoid robots may lead people to use them indiscriminately, diverting attention from the interaction essentials. Nevertheless, autonomous mechanisms aim to reduce users’ mental workload, performing increasingly complex tasks and now being part of our daily lives (e.g., self-driving cars, autonomous vacuum cleaners, and chatbots). The literature refers to methods to integrate autonomy mechanisms in telerobotics [156], and they can be classified into direct control, supervisory control, shared control, traded control, collaborative control, and cooperative control [157,158].

Direct control: The robot has no autonomy. An operator controls all the robot’s functions manually. Mirroring is a type of direct control in which the robot replicates the human’s movements and expressions.

Supervisory control: The robot is programmed intermittently according to the continuous information received from the robot. The human and the robot integrate a closed control loop focused on task performance [159].

Shared control: The human operator controls the robot continuously. However, those commands may be strictly followed by the robot (similar to direct control) or be modified by the robot’s system to improve performance or run safely.

Collaborative control: The operator and the robot work together as peers to determine the robot’s behavior. In Fong’s work [160], there is an explicit semantic dialogue between humans and robots to mediate the sharing of control.

Traded control: The human operator starts a behavior or task that is autonomously performed by the robot. At any time, the operator can stop that behavior or task and start a new one.

Cooperative control: The behavior of a single robot results from the controlled cooperation of several operators using any of the aforementioned methodologies.

In the shared control method, the operator provides continuous commands to the robot, aiming for high-level behavior from the robot. However, the robot may change those inputs to reach the perceived system goals [161]. The method assumes that the operator knows how to direct the robot’s high-level behaviors but may not be sufficiently skilled to express the right commands due to a lack of situation awareness, embodiment, telepresence, or lack of robot motor accuracy and sensor information. Typically, the shared control method includes “safeguard” mechanisms, in which the operators’ command actions are overwritten if they violate the robot’s safety rules, such as collision with a wall or person or losing balance [162]. The software of HPR-1s or HRP-5P humanoid robots was developed to discard commands that could make the robot lose balance, limiting joint angles [163,164,165]. In Almeida et al. [92], given the robot’s height, the wheel’s initial acceleration provided by the operator had to be supervised by the robot to avoid its falling down. In Crandall and Goodrich’s works, the robot’s desired trajectory was provided through a joystick as the intended general direction and not as low-level position commands [166].

In the traded control method, a task or subtask is performed autonomously by the robot, but it is initiated by the operator and may be stopped at any time. The method is useful for simultaneously controlling multiple robot’s appendages, such as in teleoperation of humanoids [167,168,169]. The Geminoid HI-1 robot [169] relies on a traded control known as state-based control, in which the operator selects the state from a library of states. It includes five conscious behaviors, namely right looking, left looking, listening, speaking, and being idle. For each state, the robot assumes autonomous behaviors (i.e., motion files), avoiding an explicit operator’s control of 50 robot actuators. The integration of multiple semi-autonomous mechanisms is essential while controlling the eyes, head, torso, arms, hands, and fingers simultaneously in a humanoid robot. Quite often, operators need to control low-level robot behaviors and additionally focus their attention on high-level tasks, such as (1) robot navigation, (2) obstacle avoidance, (3) triggering robot’s unconscious and conscious behaviors [169], (4) object and scene understanding [137,170,171], (5) mission planning, or (6) people‘s interaction. Osawa et al. [172] evaluated the automation of involuntary and voluntary movements using a teleoperated telepresence robot (robovie-mR2). The implemented behavior generation architecture (bi-layered architecture [173]) enabled the combination of autonomous movements and manual movements controlled by a remote operator. The results showed that bystander users evaluated both the involuntary and voluntary movements positively but also revealed that from the remote operator’s point of view, the automation of voluntary movements should require additional care (agency issue conflicts).

3.1.6. Time Delay Mitigation

The dynamic nature of the communication medium has an impact on the complexity of teleoperated systems. Time delay, jitter, distance, bandwidth constraints, packet loss, or blackout in internet-based solutions can delay or distort interactions. This can affect the synchronicity (rate of message exchange between operator and bystander), compromising the social presence [33] (e.g., degradations in audio and video streams, control streams, or haptic feedback). Traditional methods to mitigate time delay in telerobotics involved user interface design and control theory-based models (e.g., supervisory control or passivity-based teleoperation) and evolved into predictive displays and control [159]. Recent solutions for time delay issues use time series prediction methods to predict the time delay, robot movements, and user intentions (e.g., user’s gaze prediction [174]). These new adaptive-based control methods make use of nonlinear statistical models and neural network (NN) or machine learning (ML) techniques.

Ferrell and Sheridan [175] determined that a time delay affects human operators’ performance while teleoperating manipulators. They realized that the person within the control loop of teleoperated systems under time delays used to adopt a move-and-wait strategy to accomplish certain tasks. To address this problem, they proposed supervisory control [176], in which the robot is preprogrammed or programmed online to perform certain subtasks autonomously. By transmitting only high-level commands, there is a data communication reduction, and task time completion improves. Meanwhile, several extensions of supervisory control were developed, including specific languages to chain tasks or predictive displays (i.e., visualization of a phantom robot model that predicts the motion of the real robot) [177,178].

Control-based approaches for time delay mitigation in teleoperation systems can be clustered into two classes [31,179]: (1) predictive control-based methods (e.g., a discrete linear quadratic Gaussian (LQG) controller for teleoperation acting on the sampling rate or output feedback control of multiple-input and multiple-output (MIMO) systems) and (2) passivity-based methods that model the master–slave operator systems and unsure stability and performance under time delay variability (e.g., a two-port network, hybrid matrix, impedance matrix, constant time delay, scattering approach, wave variable, scaling, and geometric scattering).

Time series prediction approaches for time delay mitigation in teleoperation systems try to compensate for the time delay, observing past intrinsic patterns to predict the future values [31,156]. They integrate trends, seasonality, and white noise and can be clustered into two types: statistical methods and neural network (NN) or machine learning (ML) methods:

(1) Statistical methods (e.g., moving average (MA), linear auto-regression (AR), auto-regression + moving average (ARMA), and auto-regression + moving average + nonlinear component (ARIMA) [180]);

(2) NN or AI methods (e.g., recurrent neural networks (RNNs) [181,182], long short-term memory networks (LSTMs) [183,184], sequence to sequence (Seq2Seq) [185,186], and generative adversarial networks (GANs) [187,188]).

Statistical methods have the advantage of not requiring training with data and are simpler to implement. Although times series prediction traditionally relied on statistical approaches, it has difficulties in modeling the entire set of nonstationary signals. Nevertheless, methods like ARIMA can cope with nonstationary signals. Statistical methods are not appropriate for modeling complex tasks, being more suitable for short-term predictions. Neural networks, on the other hand, have an advantage over statistical approaches in that they enable data description without explicit knowledge of its distribution and can model more complex time series data based on past observations. Neural networks are more prone to adapt their behaviors as the input data increases [31].

3.2. User-Adaptive Systems Taxonomy

Social robots aim to assist people, enable telesurveillance of elderly people, guide people on tours, promote physical and mental exercise, keep company, or entertain [1,5,6]. In short, they contribute to the user’s well-being, adapting to people, to the environment, and ultimately to the context. Case studies include interaction of a service robot for 1 week in an elderly care center [116]. Several types of user-adaptive mechanisms are described in the robotics literature [2,8,9,31,189,190].

Typically, a framework for a user-adaptive system comprises two components (see Figure 5): the interface layer that is used for the exchange of information between the user and the system (It integrates sensors for the system to perceive the user and actuators to provide stimuli.) and the decision-making module which, based on perceived information, makes algorithmic decisions and generates response actions to be synthesized by the interface.

Robot systems with autonomous and semi-autonomous behaviors can be classified with the following taxonomy [8]:

Autonomous and semi-autonomous	- Adaptive systems with no user model
behaviors supported by	- Systems based on static user models
	- Systems based on dynamic user models

• Adaptive systems with no user model: systems with reactive behavior regarding the user’s immediate feedback and with no cache of the user’s information (see Figure 6);
• Adaptive systems based on static user models: systems that rely on pre-loaded knowledge retrieved from the relevant attributes of the user and used to adjust the system’s behavior (see Figure 7);
• Adaptive systems based on dynamic user models: similar to the previous example, these systems explicitly maintain user models. They are task-oriented models, updated with users’ information during their interactions (see Figure 8).

User-adaptive systems require information about the user which is typically stored in the form of a user model [9,191]. As reported in an early survey [192], a new field of research emerged concerning with acquisition, organization, and representation of the system’s user.

Adaptive systems without user modeling can implicitly map the characteristics of a generic user in the architecture of the decision-making module (Figure 6). Nevertheless, it is a reactive adaptation that shapes the system’s behavior directly based on the user’s feedback. The user’s behavior changes are monitored and trigger an immediate switch to a new system’s operational state, while no storage or user model update is performed. Table 7 summarizes several works that adopt this type of architecture.

Adaptive systems based on static user models assume that the person’s profile does not evolve during the interaction. These static models can be built during an initial phase of the interaction (Figure 7), similar to the calibration process, or the user’s profile can be pre-supplied using external questionnaires. These types of systems are not able to dynamically learn the characteristics of the user. Examples of related works are listed in Table 8.

Adaptive systems based on dynamic user models perceive, learn, and update the knowledge regarding the context model and the user model. The stored user model is updated during the interaction based on the user’s reactions. This category of systems is considered the best performing user-adaptive solution, although its implementation is more complex [191,193,194]. Table 9 compiles several references for systems based on dynamic user models.

Additionally, one of the described categories, such as adaptive systems based on dynamic user models, can coexist in a telepresence teleoperated robot [1,3,172,173], thus adding adaptiveness functionalities either for the robot’s operator (remote user) or for the local user that is with the robot. The general architecture of a teleoperation system with user adaptiveness is depicted in Figure 9.

The decision-making modules of the listed user-adaptive systems include different frameworks, such as the Markov decision process (MDP), partially observable Markov decision process (POMDP) ($\alpha$POMDP [195]), Mixed Observability Markov Decision Processes (MOMDP), fuzzy control, rule-based, hidden mode stochastic hybrid system (HMSHS), Bayes-adaptive, dynamic factor graph (DFG), active leaning, or reinforcement learning. Recent approaches for these frameworks are described in [196].

Table 7. Adaptive parameters, input modalities, framework of decision, output modalities, and social robot evaluation with no user model.

Adaptive Parameters	Study	Input Modality	Decision Making	Output Modality	Evaluation Process	Evaluation Metrics
Robot’s Navigation Goal	[197]	Brain-actuated controls	Rule-based	Robot commands	Measurements, questionnaires,	Robot path
Decisions $takeleftturn$	[198]	Physical controls	POMDP	Image and sound	Measurements, questionnaires	POMDP rewards, perceived control, driving performance, similarity to real world, naturalness, social appropriateness
Robot Speed	[199]	User’s pose and speed	MOMDP	Motor control	Measurements	Speed difference and distance to the user
Decisions (object to move)	[200]	Speech, gaze	Rule-based	Robot arm movement	Measurements, questionnaires	Prediction accuracy, projection accuracy, perceived awareness, response time and intentionality
Robot Speed	[201]	Odometry, Physical controls	Fuzzy control	motor controls	-	-
Decisions (warn driver or intervene)	[202]	Physical controls	Hidden mode stochastic hybrid system	Image and sound	Measurements	Time in unsafe and safe states
Decisions (room to clean)	[203]	User locations, task success	Motor control	Rule-based	-	-
Robot’s Navigation Goal	[204]	Physical controls	Rule-based	Robot commands	Measurements	Recognition accuracy
Voice Pitch	[205]	User speech	Rule-based	Robot speech	Questionnaires	Persistence and learning gain, rapport, perceived social presence
Robot’s Gestures	[206]	Vision, speech	Rule-based	Robot commands	Measurements, questionnaires	Information distance, perceived behavior performance, perceived gesture recognition, enjoyment, perceived social interaction
Robot Speed and Path	[207]	Physical controls	Rule-based	Robot commands	-	-
Decisions (navigation goal)	[208]	Physical controls	POMDP	Robot commands	Measurements	State variables, robot path, destination probabilities
Decisions (what objects to move, when to speak)	[209]	Speech, vision, depth	Rule-based	Speech, robot commands	Measurements	User’s speech time

Table 7. Adaptive parameters, input modalities, framework of decision, output modalities, and social robot evaluation with no user model.

Adaptive Parameters	Study	Input Modality	Decision Making	Output Modality	Evaluation Process	Evaluation Metrics
Robot’s Navigation Goal	[197]	Brain-actuated controls	Rule-based	Robot commands	Measurements, questionnaires,	Robot path
Decisions $takeleftturn$	[198]	Physical controls	POMDP	Image and sound	Measurements, questionnaires	POMDP rewards, perceived control, driving performance, similarity to real world, naturalness, social appropriateness
Robot Speed	[199]	User’s pose and speed	MOMDP	Motor control	Measurements	Speed difference and distance to the user
Decisions (object to move)	[200]	Speech, gaze	Rule-based	Robot arm movement	Measurements, questionnaires	Prediction accuracy, projection accuracy, perceived awareness, response time and intentionality
Robot Speed	[201]	Odometry, Physical controls	Fuzzy control	motor controls	-	-
Decisions (warn driver or intervene)	[202]	Physical controls	Hidden mode stochastic hybrid system	Image and sound	Measurements	Time in unsafe and safe states
Decisions (room to clean)	[203]	User locations, task success	Motor control	Rule-based	-	-
Robot’s Navigation Goal	[204]	Physical controls	Rule-based	Robot commands	Measurements	Recognition accuracy
Voice Pitch	[205]	User speech	Rule-based	Robot speech	Questionnaires	Persistence and learning gain, rapport, perceived social presence
Robot’s Gestures	[206]	Vision, speech	Rule-based	Robot commands	Measurements, questionnaires	Information distance, perceived behavior performance, perceived gesture recognition, enjoyment, perceived social interaction
Robot Speed and Path	[207]	Physical controls	Rule-based	Robot commands	-	-
Decisions (navigation goal)	[208]	Physical controls	POMDP	Robot commands	Measurements	State variables, robot path, destination probabilities
Decisions (what objects to move, when to speak)	[209]	Speech, vision, depth	Rule-based	Speech, robot commands	Measurements	User’s speech time

Table 8. Adaptive parameters, input modalities, framework of decision, output modalities and social robot evaluation with static user model.

Adaptive Parameters	Study	Input Modality	Decision Making	Output Modality	Evaluation Process	Evaluation Metrics
Robot’s Gestures and Speech	[210]	Speech	Rule-based	Gestures, speech	Questionnaires	Preference toward a type of adaptation
Decisions (placement of objects)	[211]	Crowd-sourced data	Rule-based	Robot controls	Measurements	F-scores
Robot Location, Interface Complexity, Warning Levels, Font Size	[212]	-	Rule-based	-	-	-
Decisions (how to dress the user)	[213]	User’s pose, speech	Rule-based	Robot commands	Measurements	Task completion speed
Decisions (how to dress users)	[214]	User’s pose, speech	Rule-based	Robot commands	Measurements	Classification accuracy
Speech Output Gender, Sound Volume, Robot’s Name, Robot Speed	[215]	Speech, touch	Rule-based	Robot commands, speech	Questionnaires	Acceptance, perceived usability
Sequence of dance movements	[216]	User’s pose	Rule-based	Robot commands	Questionnaire, manual classification	Gaze position, body language, facial emotion, perceived bond, amusement, satisfaction, enjoyment, anxiety, observed leadership, expectancy

Table 8. Adaptive parameters, input modalities, framework of decision, output modalities and social robot evaluation with static user model.

Adaptive Parameters	Study	Input Modality	Decision Making	Output Modality	Evaluation Process	Evaluation Metrics
Robot’s Gestures and Speech	[210]	Speech	Rule-based	Gestures, speech	Questionnaires	Preference toward a type of adaptation
Decisions (placement of objects)	[211]	Crowd-sourced data	Rule-based	Robot controls	Measurements	F-scores
Robot Location, Interface Complexity, Warning Levels, Font Size	[212]	-	Rule-based	-	-	-
Decisions (how to dress the user)	[213]	User’s pose, speech	Rule-based	Robot commands	Measurements	Task completion speed
Decisions (how to dress users)	[214]	User’s pose, speech	Rule-based	Robot commands	Measurements	Classification accuracy
Speech Output Gender, Sound Volume, Robot’s Name, Robot Speed	[215]	Speech, touch	Rule-based	Robot commands, speech	Questionnaires	Acceptance, perceived usability
Sequence of dance movements	[216]	User’s pose	Rule-based	Robot commands	Questionnaire, manual classification	Gaze position, body language, facial emotion, perceived bond, amusement, satisfaction, enjoyment, anxiety, observed leadership, expectancy

Table 9. Adaptive parameters, input modalities, framework of decision, output modalities, and evaluation of the social robots with a dynamic user model.

Adaptive Parameters	Study	Input Modality	Decision Making	Output Modality	Evaluation Process	Evaluation Metrics
Promote Regular Physical Activity Habits	[84]	User’s position, pose (exercise performance), speech	Rule-based	Navigation, robot commands, speech (avatar coach)	Measurements, questionnaires	User’s exercise performance, flow
Decisions (service, navigate, turn to the person, stop, smile)	[116,195]	Person detection, speech, emotion recognition, touchscreen	$\alpha$POMDP, SOA-based model	Navigation, robot commands, approach, speech, robot expression, recognition, service	Measurements, questionnaires	Usability, appearance, satisfaction
HumanRobot Greetings Phase	[71]	Tracking human gestures	MDP	Kendom phase trigger (initiate approach, distance salutation, head dip, approach, final approach, and close salutation)	Measuments, observation	Sequence estimation accuracies
Colors of LEDs	[217]	Physical controls	Rule-based	LED colors	Measurements	Cumulative reward from users, error estimation
Decisions (what interactions to perform with the user)	[218]	Physical controls robot	Rule-based	Commands	Measurements, questionnaires	Child learning rate, human intervention ratio
Reading Difficulty Level	[219]	Speech, touch	Active learning	Number of words learned	Measurements, questionnaires	Images, speech
Decisions (adaptation to user’s subtask selection)	[220]	Vision, speech	Rule-based	Robot commands, speech	Measurements	Number of communications required of the user
Decisions (moments to take action, including parameter adjustment and services)	[221]	Gesture sound, projected mages	MDP	Questionnaires	Perceived coherence, user satisfaction, ease of use, perceived helpfulness, originality, perceived adaptivity
Decisions (when to deploy services)	[222]	Speech, touch	Equilibrium maintenance	Speech, images, robot commands	Measurements	Opportunity relevance for the selected service
Decisions (dialogue to play)	[223]	Tactile sensors, sound, touch	Dynamic factor graph	Image, speech, robot commands	Questionnaires	User’s opinion
Decisions (select learning content type)	[224]	Speech, physical controls	Rule-based	LEDs, robot commands	-	-
Decisions (sounds to play)	[225]	Physical controls	Context-free stochastic grammars	Sound (music)	Measurements, questionnaires	Engagement, perceived difficulty, progression, conformity, number of user interventions, speed
Decisions (placing a shared object)	[226]	Vision, physical controls	MAMDP	Robot commands	Measurements, questionnaires	Perceived trustworthiness, ratio of users that change strategies
Decisions (positive, negative, or neutral output)	[227]	Facial expressions, RGBD, electrodermal data, touch screen	Rule-based	Images, speech, gestures	Questionnaires	Understanding, perceived enjoyment, trust
Decisions (where to guide the user)	[228]	Vision, user’s attention, robot position, odometry, speech	Rule-based	Robot commands, navigation	Questionnaires	User’s opinion (score)

Table 9. Adaptive parameters, input modalities, framework of decision, output modalities, and evaluation of the social robots with a dynamic user model.

Adaptive Parameters	Study	Input Modality	Decision Making	Output Modality	Evaluation Process	Evaluation Metrics
Promote Regular Physical Activity Habits	[84]	User’s position, pose (exercise performance), speech	Rule-based	Navigation, robot commands, speech (avatar coach)	Measurements, questionnaires	User’s exercise performance, flow
Decisions (service, navigate, turn to the person, stop, smile)	[116,195]	Person detection, speech, emotion recognition, touchscreen	$\alpha$POMDP, SOA-based model	Navigation, robot commands, approach, speech, robot expression, recognition, service	Measurements, questionnaires	Usability, appearance, satisfaction
HumanRobot Greetings Phase	[71]	Tracking human gestures	MDP	Kendom phase trigger (initiate approach, distance salutation, head dip, approach, final approach, and close salutation)	Measuments, observation	Sequence estimation accuracies
Colors of LEDs	[217]	Physical controls	Rule-based	LED colors	Measurements	Cumulative reward from users, error estimation
Decisions (what interactions to perform with the user)	[218]	Physical controls robot	Rule-based	Commands	Measurements, questionnaires	Child learning rate, human intervention ratio
Reading Difficulty Level	[219]	Speech, touch	Active learning	Number of words learned	Measurements, questionnaires	Images, speech
Decisions (adaptation to user’s subtask selection)	[220]	Vision, speech	Rule-based	Robot commands, speech	Measurements	Number of communications required of the user
Decisions (moments to take action, including parameter adjustment and services)	[221]	Gesture sound, projected mages	MDP	Questionnaires	Perceived coherence, user satisfaction, ease of use, perceived helpfulness, originality, perceived adaptivity
Decisions (when to deploy services)	[222]	Speech, touch	Equilibrium maintenance	Speech, images, robot commands	Measurements	Opportunity relevance for the selected service
Decisions (dialogue to play)	[223]	Tactile sensors, sound, touch	Dynamic factor graph	Image, speech, robot commands	Questionnaires	User’s opinion
Decisions (select learning content type)	[224]	Speech, physical controls	Rule-based	LEDs, robot commands	-	-
Decisions (sounds to play)	[225]	Physical controls	Context-free stochastic grammars	Sound (music)	Measurements, questionnaires	Engagement, perceived difficulty, progression, conformity, number of user interventions, speed
Decisions (placing a shared object)	[226]	Vision, physical controls	MAMDP	Robot commands	Measurements, questionnaires	Perceived trustworthiness, ratio of users that change strategies
Decisions (positive, negative, or neutral output)	[227]	Facial expressions, RGBD, electrodermal data, touch screen	Rule-based	Images, speech, gestures	Questionnaires	Understanding, perceived enjoyment, trust
Decisions (where to guide the user)	[228]	Vision, user’s attention, robot position, odometry, speech	Rule-based	Robot commands, navigation	Questionnaires	User’s opinion (score)

Co-presence mechanisms: the availability of robotic autonomous mechanisms enables a robot’s voluntary or involuntary behaviors that contribute to enhancing co-presence [172], such as those listed in

Table 10. Robotic mechanisms to enhance co-presence.

Type	Voluntary	Involuntary
Eye contact	X	-
Gaze following	X	-
Gazing at the closest face	X	-
Gazing at a random face	X	-
Gazing at the closest object	X	-
Gazing at a random object	X	-
Gazing at a moving object	X	-
Looking around the gazing position	X	-
Joint attention	X	-
Sleeping	X	-
Changing LED colors	X	-
Mouth movement	X	-
Nodding in response to human speech	X	-
Waving both hands at a random human	X	-
Waving left hand at a random human	X	-
Waving right hand at a random human	X	-
Waving left hand in response to palms	X	-
Waving right hand in response to palms	X	-
Waving both hands in response to palms	X	-
Reflexive blinking with eye movement	-	X
Spontaneous blinking	-	X
Avoiding objects at close range	-	X
Eye saccade	-	X
Breathing	-	X

.

3.3. Evaluation Methods

To assess co-presence, telepresence systems require objective and qualitative metrics. Quantitative measures may include physiological signals (such as heart rate, skin temperature, electrodermal activity (EDA), and skin conductance responses (SCRs) [229], eye scan patterns, electroencephalography (EEG), or functional magnetic resonance imaging (fMRI)) [34,230,231], as well as other metrics that are simpler to obtain, such as accuracy, time to perform a task, and the number of errors or communication delays. However, given the human factor and the psychological components of interaction, questionnaires remain essential tools. There are methodologies for measuring the presence, social presence or co-presence, and flow state of the users using technological devices [19,232,233,234,235].

Flow is a psychological state that people describe when they are fully engaged in some events to the point of forgetting time, fatigue, and everything else but the activity itself [40,236].

Table 11. List of questionnaires to assess presence, flow, and game.

Psychological Phenomena	Questionnaire	Number of Questions	Ref.
Presence	Slater, Usoh and Steed (SUS)	6	[237]
Presence	Temple Presence Inventory (TPI)	42	[238]
Presence	Igroup Presence Questionnaire (IPQ)	14	[239]
Presence	Sense of Presence Inventory (ITC-SOPI)	38	[240]
Presence	Presence Questionnaire, version 3 (PQ)	29	[241]
Presence	Networked Minds Social Presence Inventory (NMSPI)	34	[41,242]
Presence	Multimodal Presence Scale (MPS)	15	[243]
Presence	Spatial Presence Experience Scale (SPES)	8	[244]
Flow	EduFlow Scale (EFS)	12	[245]
Flow	Flow Short Scale (FSS)	13	[246]
Flow	Reading Flow Short Scale	8	[247]
Game and Flow	EGameFlow (EGF)	42	[248]
Usability	Nielsen Norman Group	-	[249,250]

lists the available questionnaires to measure the levels of presence, co-presence, immersion, and flow.

Usability, testing and accessibility–Jakob Nielsen, one of the most active proponents of usability processes, referred to the following elements that comprise a definition of usability [249,250,251]:

• Ease of use: the use of products or tasks should be natural and easily performed by the user.
• Simplicity of learning: tasks and product features must be intuitive and present a logical and consistent sequence to simplify learning.
• Improved reliability: levels of satisfaction and performance are increased when the action’s results correspond to the user’s expectations.
• Reduction in errors: usability can be increased if designers attribute the errors to the product or task (rather than the user), redesigning it based on the user’s feedback.
• Enhanced user satisfaction: the user’s satisfaction principle must guide all of the design process, making the product or task pleasing to use or perform.

In [252], a taxonomy of usability guidelines for the design of telepresence teleoperated robots (interaction effectiveness and efficiency, information presentation, interface visual design, robot surroundings and environment awareness, robot state awareness, and cognitive factors) is proposed. The usability testing process is an effective use of materials and time [17,249,253,254] that should not be overlooked.

4. Conclusions

This work presented a survey of recent works, proposing the development of support for social robotic interactions with applications in health care, elderly assistance, guidance, or office meetings. It focused on enhancing social presence via telepresence robot mediation, in which a user should sense his or her remote interlocutor as being locally present with him or her. The research gathered knowledge to help roboticists design improved user- and environment-adaptive systems and technical methods that contribute to enhancing the sense of presence or co-presence. This literature review aimed to define social presence, identify autonomous “user-adaptive systems” for social robots, and propose a taxonomy for “co-presence” mechanisms. The referred works address robot sensing, perception, action, reasoning, appearance, automation, and cognitive approaches (e.g., statistics models and AI). Additionally, it presents an overview of social robotics systems and application areas and provides directions for telepresence and co-presence robot design, considering the actual and future challenges. Finally, some guidelines for the evaluation of these systems are left, having as reference face-to-face interactions. Based on survey findings in engineering and psychology, our future work includes the design of telepresence and co-presence robots that better emulate or interpret human subjective experiences.