Yōkobo: A Robot to Strengthen Links Amongst Users with Non-Verbal Behaviours

Abstract

Yōkobo is a robject; it was designed using the principle of slow technology and it aims to strengthen the bond between members (e.g., a couple). It greets people at the entrance and mirrors their interactions and the environment around them. It was constructed by applying the notions of a human–robot–human interaction. Created by joint work between designers and engineers, the form factor (semi-abstract) and the behaviours (nonverbal) were iteratively formed from the early stage of the design process. Integrated into the smart home, Yōkobo uses expressive motion as a communication medium. Yōkobo was tested in our office to evaluate its technical robustness and motion perception ahead of future long-term experiments with the target population. The results show that Yōkobo can sustain long-term interaction and serve as a welcoming partner.

1. Introduction

Social robots (SR) are designed to achieve many purposes, such as care [1], entertainment, education, or personal assistance [2]. Their modes of interaction with humans are diverse, e.g., voice, screen, or gestures. Generally, SRs are designed with the main task to perform, where motion is used as a tool rather than a part of the interaction that can help transmit social cues [3] and express emotions [4].

We propose a new SR using a specific HRI approach called human–robot–human interaction (HRHI), where the robot is an intermediary between two persons. Using this method, our robot, named Yōkobo (Figure 1), was designed to be included in a smart home with the central purpose of strengthening the bond between people (e.g., a couple). To this end, its task is to welcome family members or visitors at the home entrance and transmit some user’s actions by moving its motors (called motion messages) from one person to another, while still having non-robotic functions: to serve as a ’key bowl’. Its name is a portmanteau word from the Japanese word yōkoso (welcome) and the French pronunciation of the word robot (the t is silent). Considering the smart home context, Yōkobo takes advantage of available sensors, such as temperature or humidity, and incorporates those data into its behaviours. Unlike most SRs for homes sold on the market, Yōkobo was designed to have an abstract shape. Moreover, its sole communication medium is through its movements and lights. Yōkobo’s design approach follows the principles of slow technology [5]. It is a concept where time plays a role in the adoption of the object and encourages the user to reflect on technology. By favouring this design approach, Yōkobo is designed in a way that allows its user to slow down, be surprised, and be reminded of the environment and their significant other. It was proven by Odom et al. [5] that this approach can “create feelings of anticipation, open spaces for questioning the role of technology and even help to change routines”.

Regarding home devices, vocal assistants (VAs) have progressively entered the home. Similar to VA, robot assistants (RAs) can accomplish several tasks, such as checking emails and calendars, or controlling smart home devices. Contrarily to VAs, they have a physical presence thanks to their movement abilities. For example, Elliq used LEDs and body language to facilitate communication with users. A RA can be more expressive than a VA thanks to its expressive motions [4] or facial expressions, such as Haru [6], who can express its mood by changing the positions/shapes of its eyes. Studies showed that RA use is more enjoyable [7] than VAs and that they are often preferred due to their social embodiment [8]. Another difference between HRHI robots and RAs is that the latter provide direct interaction between the user and the robot, and no third party is included in the loop, even if the whole family can use it, contrary to HRHI ones.

In HRHI, the robot is designed to be at the heart of the relationship between two or more persons, functioning as a catalyst in their relationship. Those robots are designed to create encounters between the users, not just to serve as communication tools, such as social networking robots [9]. We can distinguish them from telecommunication robots, which are also part of the HRH relationship, where the robot is more of an extension of the human, serving as an enhanced phone.

Although Yōkobo uses motions, such as an RA, it has the particularity of being a robject: “embedding useful robotic technologies within everyday life objects” [10]. In our case, Yōkobo has been built around a key bowl. It is integrated into the room to serve a purpose besides its robotic function; hence, even if it is not working, it can still be used.

2. Contribution

We propose a novel approach in social robotics, with a robot aimed to serve as a link between two persons living together in their home, by using greeting movements and interacting with the users at the home entrance. The semi-abstract design approach allows our robject Yōkobo to be self-effacing in the HRH relation, supporting Yōkbo’s overall goal, being just a medium, avoiding strong attachment at the partner’s expense.

We also propose a protocol to test the previous aspects during two different sessions of mid-term experiments (2 weeks) conducted in the wild, where we let participants interact with Yōkobo without any specific scripted scenario. We propose a set of tools to analyse the interactions, the robustness of the robot, and the users’ perceptions.

3. Related Works

3.1. Creating Encounters and Links between Persons

Recent efforts have been made to enable the formation of social encounters, such as the Abstract Machine by Anderson-Bashan et al. [11], which brings the greeting process usually only implemented for humanoids [12,13] to new forms. Other examples are weak robots [14], which, by displaying signs of weakness, attract humans to interact with them. Similarly, the Ranger robot [15] captivates the attention of children and helps them organise their rooms. These examples have proven to enhance HRI or help with tasks. However, one crucial notion to consider when developing encounters is to tackle any possible feeling of ostracism that the user may have due to the perceived rejection when a robot is fixated on a task or with another user [16].

One example of an applied HRHI formulation is telepresence robots, as done with Haru in [17]. In this case, the robotic embodiment is here to serve as a user’s extension in real-time communication. Nevertheless, the robotic agent does not act as a bridge for communication, instead, it acts as a tool to provide an almost physical video chat experience.

Robots made to improve social interactions in autistic children [18] have led to improvements in the relationships between the children and the therapists. These robots could be considered using HRHI. However, it is a one-way relationship. The purpose of these robots is to help the children communicate. They generally do not help people communicate with children. It is more of a H→R→H relationship than a HRHI, besides being for a specific target.

Research on an extended relationship with a robot has been conducted by Rifinski et al. [19]. They studied the effect of a robot on a human–human–robot interaction (HHRI), where the robot was not central in the discussion, instead, it acted as a third-party observer. They experimented on robotic influence and its associated movement when two people talked.

Jeong et al. [9] went deeper by creating what they called a social networking robot (SN-Robot) named Fribo to decrease the feeling of loneliness. They are using HRHI to transmit information about user actions in their homes to their friends. The robot acteds as a middleman to notify, with voice, the group (three friends equipped with their robot) when one, for instance, opened the fridge; everything was shared anonymously. During the experiment, the participants became attached to the robot, notably because of the voice, and the robot catalysed conversation in the group of friends. However, this robot was static and used only voice and screen to communicate, no movements were involved.

3.2. Form Factor

According to Campa [20], SRs are mostly humanoid or animal-like. Still, we could extend this classification, based on their form factors, into four categories [21], i.e., humanoids, zoomorphic, semi-abstract, and abstract ones.

The humanoid shape facilitates social interactions as the user can rely on their own experience to interact with the robot. Nevertheless, the human figure can create distress in the person [22]. That is why people may feel more comfortable with zoomorphic robots [23]. Their forms are diverse, ranging from a dog, such as AIBO, to a sea urchin [24]. With these forms, people are driven to interact with the robot as they would with a pet [25]. However, both latter categories have psychological consequences, such as creating strong attachments [26]. The semi-abstract robots do not look similar to any living creatures, but the users can imagine some human or animal shape and behaviour by pareidolia or anthropomorphism. They can also be inspired by either a real or imaginary animal. One example is Lovot [27], which might share characteristics close to a penguin but with a face closer to an anime character. The abstract robots resemble nothing biological, yet, while interacting with them, it is possible to extrapolate their behaviours, as shown in [11]. Both previous form factors allowed the designers to be less constrained, and the users have fewer expectations about the robot’s behaviours [28]. However, their interpretations could differ because of their personal backgrounds [29].

3.3. Perception through Reduced Robotic Movement

Duarte et al. [30] studied the concept of non-verbal behaviour (NVB) to communicate intentions with a humanoid robot. They showed the possibility of perceiving intention solely with motion. Participants could understand the robot’s intent thanks to the gaze, head, or arm movements. This interpretation is not limited to human-like body parts, but also other robotic embodiments, as Broers et al. [14] showed with a trash-can-like robot and Bevins et al. [31] with drones. Lehmann et al. [32] proved that non-anthropomorphic robot movements are crucial to help users perceive engagement. Indeed a 2-DoF robot, ref. [11], showed that people could see social cues with simple motions.

Hoffman et al. [33] showed the importance of NVB for a robot, emphasising the design process merit. They demonstrated that a robot with expressive motions and positive emotions could be more easily accepted than a complex anthropomorphic robot with “unaffectionate motion”. Most of the studied robots are non-anthropomorphic, such as Travis [34], a robotic speaker dock, and listening companion. With a single DoF, this robot was designed with music-aided movement to create a soothing environment. Similarly, Luria et al. [7] designed Vyo, a smart home assistant robot with five DoFs. Its motions were designed to be respectful or reassuring. Incorporating such design notions during the conceptual stage makes it possible to create a pleasant atmosphere around the robot, emphasising movement.

4. Design and Implementation

Yōkobo was created with a three-stage design process. The first stage enables the greeting concept identification. The second focuses on Yōkobo’s modelling, with shape and behaviour designs. The last one is aimed at realising the functioning prototype, applying the Agile method [35].

4.1. Yōkobo’s Shape Design

Yōkobo’s shape was imagined alongside its movements. It is centred around two imaginaries: Japanese ceramic and robot-like depictions, organic and mechanic ones, mixing circular shapes (on the main pieces) and angular (on the edges). The ceramic imaginary provides Yōkobo with more precious, personal, and unique attributes. While the robotic one evokes the popular trend known as mecha, via its edges and vents.

Based on previous fieldwork [36], it was essential to design a discreet, yet useful, object. Yōkobo’s physiognomy is an object that blends effortlessly in a home, specifically in the entrance. Its shape is composed of four parts: a base, a body, an apex, and a bowl (Figure 1). Its dimensions were chosen considering its ideal placement.

4.2. Services and Associated Functions

Yōkobo’s concept, as a robject, allows it to function as a standard key bowl, for objects such as keys or coins, or as a medium for house members to interact with each other. Its primary communication means are movement and light. We drive the user to focus on Yōkobo’s interaction by dismissing the vocal notions, instead of just listening. The main question we had to answer to build its functionality was, how to condense the richness of human greeting into a robject?.

Heenan et al. [13] showed that the greeting process was formed by a series of physical states, such as handshakes or nods, coupled via a transition sequence using proxemics. Guided by the above notions, the proposed solution focuses on proximity levels that serve as communication guidelines for how Yōkobo responds. It results in four levels of interaction that serve as the bases for designing the behaviours of the robot: standby, state of the house, mimic, and record.

Standby is the robot status when nobody is in the entrance hall; Yōkobo is active and has periodical movements, guided by the atmospheric pressure (AP) and air quality (CO${}_2$) data. State of the house starts once a person is near the robot, it continues the periodical movement and expresses the home state through movements. Mimic is triggered when someone is close to the robot. The robot then follows and reproduces human gestures. Record allows the user to leave a gesture message by saving the data from the mimicking process to later show to the other person. The lights, placed in the robot’s apex change when a message is played or recorded.

4.3. Behaviour Design

Previous research found that minimal robotic movement allows humans to perceive social cues and can cause feelings of surprise or engagement, according to the robot’s attributes [11,29]. Inspired by these results, the designers chose to shape Yōkobo’s animations based on human reactions to the ambient conditions [37]. Therefore, the reactions are:

Humidity inside: 

when the value is high, Yōkobo displays characteristics of human sleepiness, hinting at slowness and human body stretching.

Humidity outside: 

it displays a movement, combining the body and apex, to suggest a human sneeze.

Temperature: 

depending on the temperature variation, it shakes the body and apex, or has slow movements.

CO${}_{\mathbf{2}}$and AP: 

these values are used through Yōkobo’s periodical movements to increase or decrease the motor speed. They are used to imply the human breathing alteration by a higher concentration of CO${}_2$ or AP increments.

4.4. Hardware

Yōkobo has four DoFs, driven by three Dynamixel motors (Figure 1). $M_1$ yaws the body around axis $z_1$, $M_2$ bows the apex, and $M_3$ yaws the apex around axis $z_3$. The three motors are located inside the body, and the bowl freely rolls by gravity.

Figure 2 describes the hardware architecture. The central units are two Raspberry Pis (RPi) 4B, one for the control, and the other for the sensors. The latter is upgraded with a USB Intel Neural Compute Stick 2 (NCS) to run a human pose estimation algorithm (HPEA) on the OpenVINO toolkit [38]. The LED strip and sensor (RFID reader and two ultrasonic sensors (USs)) connection is made via the RPi2 GPIO pins through a two-layer self-made printed circuit board (PCB). The motors are connected directly to the RPi1 with USB and powered via the PCB. A 12 V-8 A power supply is used. The PCB dispatches that power to the motors (12 V-5 A) and the RPis (5 V-6 A) with a Traco. All the electronics are hidden inside Yōkobo; only the power cable is visible.

To detect when someone is nearing the robot, we use two USs around the base. To identify a user personally. We use the house keys with an RFID tag to detect whether the key is in the bowl. A wide-angle camera embedded in the body uses the RPis camera port. It is used for the mimicking procedure.

4.5. Software Control

By improving the work in [13], two finite state machines (FSMs), one named the main decision loop (MDL) and the other the motor control (MC), were constructed. The central software architecture can be seen in Figure 3. It encapsulates the data acquisition process, along with the central automata. Based on current sensor values, the first (MDL) decides how to switch between behaviours. The MC receives the commands from the MDL to send to the motors. All data transfers are done via the NEP framework we developed [39]. Since our approach follows a sequential loop of actions, there was no need for parallelisation in the main state machines, the only subsystem running in parallel to MDL and MC is the data acquisition, which constantly reads the sensors and publishes the data through NEP topics. These data are read whenever the FSM calls for it. The parallel process is in charge of acquiring the data and writing it to two specific buffers where the most up-to-date sensor values will reside. One buffer is uniquely given and accessed per FSM. Inside each FSM, to avoid reading the buffer while the parallel program is writing, a semaphore method is implemented as a precaution. As a proof of concept, all programs are coded in Python, except for the HPEA, which runs natively in C++.

Yōkobo’s movement is either conducted by predefined motions or by a periodical movement guided through a sinusoidal wave linked to its apex. This generator, in the MC, has the formula $hm=\alpha ·sin\left(\omega ·t\right)$, where $\alpha$ is a value between 5 and 35 in proportion to the AP, $\omega$ is chosen from a range, 0.4–1.4, based on the CO${}_2$ readings, t is the current time value.

The MDL state (Figure 3) switch is guided by variable s; it belongs to the set of weather data ($wd$), USs ($ir$), RFID sensor (r), and the person’s distance to the robot ($cp$).

To summarise the MDL states:

1.

Initialisation: it ensures the correct system boot-up and coordination among FSMs.

2.

Idle: pause state while the first sensor’s data package is gathered.

3.

Rest: the robot apex follows the movement defined by $hm$.

4.

Wake-up: Yōkobo displays an animation selected based on the current house temperature.

5.

State of the house: Yōkobo’s motion is guided by hm. It can also do pre-defined animations based on the two humidity sensor readings.

6.

Go back to rest: an analogous process to the Wake-Up state, the difference being that the motion is selected based on the outdoor temperature.

7.

MDL mimic: The MDL commands the MC to enact its mimic behaviour or play a recording of the previous trace. If r has no value, the robot mimics the user’s motion. Otherwise, it moves to play a trace provided that the current RFID tag is not the same as the one that left the message. The system then plays the current trace before recording. The light colour (blue) signals the user the trace is playing.

8.

Record: still mimicking, but now the person’s movements are saved for 10 s and the light is set to green.

For the MC (Figure 3), s is outlined by the weather conditions ($wd$), HPEA body points ($bp$), motor commands ($em$), and the MDL command ($md$), with values $MR$ for mimicking or record, M for playing an animation, or $HS$ for house state.

The MC’s first state opens the motor ports, pre-loads all animation files, and sets the initial positions for all joints. After the preparation finishes, the Idle state starts. Inside, the MC waits for the MDL commands to either go to:

1.

The Move state, where the animation data points are sent to the motors.

2.

The House State, the node in charge of using the humidity-guided planned trajectory, applying the $hm$ generator.

3.

Continue Idle.

4.

Move to the MC Mimic subprocess.

Furthermore, if the automaton keeps the same state, the periodical movement is enabled. The motors are controlled through the Python Dynamixel SDK. The commands are given as a motor’s position, then the Dynamixel motors reach the commanded position using their inner PID controllers.

Lastly, the MC Mimic subprocess recognises and reproduces the following human movements: sidesteps, bowing, and twisting. These movements are also the ones used to record the trace message. The algorithm reproduces the person’s motion by using the $bp$ data and, in response, outputs a valid motor command for the robot. The method is outlined below:

Sidesteps: 

the system obtains the human waist centre XY coordinates from the image data and rotates the base motor so that the human is always seen in the field of (centre) view.

Bowing: 

the application checks the vertical motion of the user’s shoulders, neck, and hips. If one of the first two is lowered below a given threshold, and the hip position has not changed, the second motor lowers the apex.

Twist: 

the system captures the human shoulder width ($Ls$) and the torso length ($Lt$), viewed from the front. It also continually calculates the shoulder width to torso ratio ($Ls/Lt$). This ratio decreases when the user turns to the side because $Ls$ becomes smaller. When this situation is detected, the program determines there is a twist. It then rotates the top motor 90° and reproduces the gestures.

5. Experimental Validation

Two two-week experiments were conducted at the entrance of the lab offices (Figure 4) (TUAT, Japan). They were used to evaluate Yōkobo’s technical stability, performance, and first impressions. The experiments were done to evaluate Yōkobo’s readiness for field experiments with the target population. The research questions that drove the experiment were: How long is Yōkobo capable of running continuously? Are the users capable of perceiving motion qualities from their partner by using the non-verbal gestures and the proposed mimic algorithm?

The first experiment (E1), a pilot study, allowed us to gather initial quantitative data regarding the interactions and help us retrieve technical issues we had (both at the hardware and software levels). Following the initial evaluation, the robot received an upgrade, ensuring its overall stability. The second experiment (E2) was performed using the same experimental procedure. In both experiments, the qualities that were studied were:

i

The technical robustness over multiple days;

ii

People’s perceptions of the robot motions;

iii

The users’ receptions toward Yōkobo;

iv

The usability and user experience.

5.1. Experiment Preparation

The purpose of the experiment was to validate Yōkobo’s technical stability. Given that it is designed for the home entrance, focusing on the human greeting, it is possible to formulate an experiment with the available resources without using the target population.

We should note that there were unusual office situations due to the pandemic, which opened up new opportunities for Yōkobo. The laboratory ran on a system of shifts (mornings and afternoons); each group could not meet the other. Lab members were notified manually via the office’s Slack channel when they arrived or left the office. We used the groups’ isolation to replicate the ’couple’s link’, by using the messaging function when entering and leaving the office.

We created two participant types: Selected participants (SPs), which received RFID tags and visitors (V). The SPs were split into couples, and their tags were paired together. With this tag, they could record or read messages to/from their other partner. Pairs were active during the second week of both experiments. The group consisted of 10 persons, from 21 to 31 years old and 8/2 male/female for E1, and from 21 to 25 years old and 9/1 male/female for E2. Six out of the ten participants cooperated in both experiments. The V group contained any other person in the office (about 10). They could interact with Yōkobo without the message functionality. Although the number of participants was small (20), it was possible to retrieve 80% of the usability problems with as few as 5 users, according to [40]. A usability problem was defined by Manakhov et al. [41] as “a set of negative phenomena, such as user’s inability to reach his/her goal, inefficient interaction and/or user’s dissatisfaction, caused by a combination of user interface design factors and factors of usage context”. This definition, in unison with the SUS metrics, will allow us to find the points where Yōkobo’s interaction goals are not achieved.

5.2. Modifications Made for the Experiment

It was planned for Yōkobo to be in ceramic, but for the experiment, it was 3D-printed in plastic. We also made some minor modifications to facilitate its effectiveness with a larger group, to be better suited for the office. First, in order to not ask participants to leave their own keys in the bowl when they were in the office, we moved the RFID reader to the base so they could tag easily. We triggered the ’play/record a message’ whenever the user clocked in/out for longer than 5 $\mathrm{s}$. To merge Yōkobo in our workplace, automated messages were sent to the office’s Slack channel for the SPs, informing that they clocked in/out.

5.3. Tools and Protocol

Four tools were used to evaluate the qualities described before. The first one (T-GW) was the graffiti wall method [42]. We placed a whiteboard next to Yōkobo with questions on it. It allowed participants to express their impressions freely in the context of use. The questions were replaced every two days for the first week. These were asked in English, and the participants could answer by drawing or writing in any language. This tool was available to all participants. We asked four questions to retrieve information regarding the feelings, experiences, and perceptions they had about Yōkobo. A sample of the participants’ answers is available in Appendix A.2.

The second tool (T-Q) was a semantic test based on Kansei Engineering [43,44] and the semantic differential [45], combined with custom questionnaires. The semantic differential allowed us to obtain an object or robot characterisation, given a number of concepts represented through bipolar adjectives [46]. The 18 adjectives used for this semantic test were chosen by following the recommendations presented in [43,47]. The word selection was crucial to ensure that they were relevant to the design of the product. These words sets were gathered after consulting experts in the domain or reviewing state-of-the-art articles. The proposed semantic assessment uses the semantic scale to measure the concepts of interest, Behaviour, Interaction, and Appearance. The semantic scale uses two bipolar adjectives with a scale between 1 and 5; 1 means the answer is close to the positive adjective, and 5 is closer to the negative one. All concepts, along with their bipolar dimensions can be seen in

Table 1. Semantic analysis results (second experiment). Lowest score—positive adjective; highest score—negative adjective.

Cpt	Dimension (Score)		Semantic Evaluation ${\mathit{\mu }}^{\left(\mathit{\sigma }\right)}$
	Positive (1)	Negative (5)	QS	QW1	QW2
Behaviour	Dynamic	Static	2.7(1.0)	2.2(1.0)	2.6(1.0)
	Smart	Stupid	2.2(0.8)	2.7(0.7)	2.4(0.5)
	Simple	Complicated	2.5(0.9)	2.3(0.9)	3.0(0.9)
	Responsive	Slow	3.1(0.9)	2.9(1.2)	3.2(0.8)
Interaction	Lifelike	Artificial	3.2(0.8)	2.7(0.9)	3.4(1.1)
	Emotional	Emotionless	2.7(1.2)	2.6(1.0)	2.9(0.9)
	Familiar	Unknown	2.1(0.8)	2.6(1.1)	2.1(0.9)
	Useful	Useless	2.4(0.5)	2.7(0.7)	2.0(0.7)
Appearance	Desirable	Undesirable	2.9(0.8)	3.0(1.3)	2.6(1.1)
	Cute	Ugly	1.5(0.5)	1.9(0.3)	1.6(0.5)
	Modern	Old	1.5(0.5)	1.9(0.8)	1.5(0.5)
	Attractive	Unattractive	1.9(0.5)	2.1(0.3)	2.1(0.7)
	Like	Dislike	1.8(0.6)	2.1(0.9)	1.7(0.7)

. These questionnaires were only sent to SPs at the end of the second week for E1 (one questionnaire, QW2). For E2, three questionnaires were sent, at the start (QS), at the end of the first week (QW1), and the end of the experiment (QW2); the latter was sent only to SPs. The difference in the number of questionnaires relied solely on an additional semantic analysis and the evolutive metrics we wanted to observe. These were used to collect feedback regarding behaviours, motions, design, and experience, and to evaluate the curiosity, fearfulness, and confusion sentiments experienced by the users throughout the two weeks of experimental sessions. The latter metrics used the Likert scale. The questionnaires are available in

Table A1. Questions asked in QS, QW1, and QW2. The text in square brackets was added for this article and was not present in the questionnaire.

Question	Answer Choice
Personal questions
Student ID 1	number
How old are you? 2	number
What is your gender? 2	Female, Male, Prefer not to say
What is your nationality? 2	text
What is your level of knowledge about robots? 2	Not Familiar (1)–Familiar (5)
What is your knowledge about Yōkobo? 2	I was involved in the first experiment I already interacted with Yōkobo on my own I know how it works and already saw it I only saw it I don’t know about Yōkobo
About Yōkobo and its behaviour
How much has Yōkobo welcomed you? [Likert scale]	Not at all (1)–I felt welcomed (5)
Did you see intelligence in Yōkobo? [Likert scale]	Not at all (1)–Totally (5)
Did you see life in Yōkobo? [Likert scale]	Not at all (1)–Totally (5)
What is your feeling about Yōkobo?(1: not at all ; 5: totally [Likert scale])	Curious Happy Afraid Enthusiastic Confusion Friendly
Yōkobo is [positive adj. ➀➁➂➃➄ negative adj.—semantic scale]	Smart (1)  Stupid (5) Simple (1)  Complicated (5) Dynamic (1)  Static (5) Lifelike (1)  Artificial (5) Responsive (1)  Slow (5) Emotional (1)  Emotionless (5) Useful (1)  Useless (5) Familiar (1)  Unknown (5) Desirable (1)  Undesirable (5) Cute (1)  Ugly (5) Modern (1)  Old (5) Attractive (1)Unattractive (5)
I ______ Yōkobo [semantic scale]	Like (1)–Dislike (5)
Design
Name with your word, in the next questions, the different parts of Yōkobo.
Numbers 1 to 4 point to the whole part. Numbers 5 and 6 point to the holes.
How do you (will) call mark 1 3	text
How do you (will) call mark 2 3	text
How do you (will) call mark 3 3	text
How do you (will) call mark 4 3	text
How do you (will) call mark 5 3	text
How do you (will) call mark 6 3	text
Interactions
How many times did you interact with Yōkobo? 4	number
Additional remarks
Do you have any additional remarks or comments about Yōkobo? 5	text

¹ The student ID is used to match the answers of the three questionnaires. ² Only for QS. ³ The number refers to Figure A1. ⁴ Only for QW1 and QW2. ⁵ Only for QW2.

and

Table A2. Questions in QW2 for the selected participants. The text in square brackets was added to this article and was not present in the questionnaire.

Question	Answer Choice
Messages
How many messages did you receive from your partner?	number
How many messages did you send to your partner?	number
How would you rate the recording of a message? [semantic scale]	Hard (1)–Easy (5)
How much did you feel the existence of your partner during this week? [Likert scale]	Does not exist (1)–Exists (5)
Do you think Yōkobo helped you to feel your partner? [Likert scale]	Not at all (1)–A lot (5)
What does Yōkobo represent in the connection with your partner?	text
Did you have the impression that the movement of the robot was Yōkobo’s behaviours or was coming from your partner?	text

of the Appendix A.1, the different scales used for each question are indicated.

The third tool (T-SUS) measured Yōkobo’s usability with the system usability scale (SUS) [48] as a post-assessment evaluation filled by the SPs. With a 0–100 grade, it evaluated the system’s success, while also providing data about the trends regarding the interaction flow and primary users’ takeaways. The questions are available in

Table A3. SUS questions in QW2 for the selected participants, using a Likert scale.

Question	Answer Choice
I think I would like to use Yōkobo frequently	Strongly Disagree (1)–Strongly Agree (5)
I found Yōkobo unnecessarily complex
I though Yōkobo was easy to use
I think I would need the support of a technical person to be able to use Yōkobo
I found the various functions in Yōkobo were well integrated
I thought there was too much inconsistency in Yōkobo
I would imagine that most people would learn to use Yōkobo very quickly
I found Yōkobo very cumbersome to use
I felt very confident using Yōkobo
I need to learn a lot of things before I could get going with Yōkobo

of the Appendix A.1.

The fourth tool (T-L) was composed of the interaction logs collected by the robot. These were used to verify the hardware and software robustness and provide quantifiable evidence regarding the interaction time, sensor variations, and motor usage.

5.4. Instructions to Participants

Since the experiment was split into two different weeks, with no SPs during the first one, we first gave instructions to all of the lab members. We sent a message on Slack, telling everyone what expected them to do: interact with Yōkobo on their own, whenever they wanted, and answer the questions on the graffiti wall. We did not explain how to interact with Yōkobo (to let them discover). For E2, we also sent QS. Additionally, we asked participants to tell us during the first week if they wished to become SPs during the second one.

At the end of the first week, we sent the questionnaire QW1 (only during E2) to everyone. We then held a live explanation session with the SPs. We first gave them the key tags they had to use. Then, we showed them how to proceed to read and record messages, in order to exchange them with their partners. We required them to attempt to exchange messages at least two times during the week (still, whenever they preferred).

At the end of the experiment, we sent to SPs the questionnaire QW2 and the SUS one.

6. Results and Discussion

6.1. Differences between Experiments

E1 was designed to serve as a preliminary test. It allowed us to gather initial stability characteristics, reception metrics, usability score, and first perceptual accounts. During this experiment, Yōkobo ran non-stop for the first week. However, several issues arose once the mimic/record state was enabled. The main difficulty was with the motors, which had trouble receiving commands from the MC due to a faulty middleware that converted the motor data package to commands. Despite these errors, the situation was swiftly handled, and it was possible to continue the test for three days (12–16 h/d) and two reduced days (6–10 h/d).

This experiment showed the necessary technical changes that Yōkobo required to improve its stability and user reception. From the experimental tools, it was possible to observe that the problematic aspects involved the message handling procedure and the sudden motor stoppage. These hindered the user’s possibility of fully interacting with the robot.

To tackle the problems described above, we decided to amplify the hardware structure. This change provided more computational resources for the HPEA and the motor control. Furthermore, the software received an upgrade to avoid false sensor triggers and improve the transition functions logic. These changes enabled a faster interaction among motors and data acquisition, which translated to optimised behaviours.

After the previous updates were integrated, we proceeded to realise E2. Contrary to E1, Yōkobo was able to run non-stop during both weeks of E2, except for a few resets during the weeknights needed to prevent motor failure in case of wrong positioning. An additional situation presented itself on the last day of the experiment. A false switching between the states of the FSM occurred while an animation ran. The previous situation reduced the experimental day by two hours, yet did not impact the results since it was resolved before most of the participants arrived at the office.

The following sections detail the experimental results. A comparative analysis is also presented, which shows the upgrade’s impact on Yōkobo’s overall qualities and capabilities.

6.2. Regarding Robustness, Stability, and Usability

A total of 6/10 participants for E1 pointed out that the recording process was complicated and not straightforward (from T-GW, and discussions with the participants at the end of the experiment). They did not understand at which state the robot was from the light signal. The time lag between the person’s motion and Yōkobo’s replication was too long and did not seem to follow the user movement concurrently. The log files showed that movements and messages were being recorded; however, the motors’ unexpected failure impacted the delivery. The questionnaire (T-Q) for E1 showed that three participants could not send or receive messages, and two couples managed to do both. The log reports support the previous statement since 7/10 participants tried to send at least one message, (Figure 5). On the other hand, regarding E2, all participants interacted with Yōkobo. A total of 9/10 managed to send and receive the same quantity of messages with a mean of two messages sent with a standard deviation of ±1, meaning one partner sent one, and the other member received it and sent it. The one participant that suggested that s/he did not receive messages (from T-Q) may have confused the motion message by the usual robot movements, explaining why it was the only case that presented this situation. The logs also support this idea since both users appeared (sending the messages), and the motion played was related to the actual body point data observed in their movements. It is worth noting that 6/10 participants also indicated the movement as coming from Yōkobo rather than their partners, which can be due to the animations that may have been triggered as part of the mimic and recording sub-process. In E2, 40% of participants agreed that Yōkobo helped them perceive their partners and 30% were neutral (T-Q), acting as a bridge between them, validating the possibility of transmitting the presence of someone through a robot. Even if some of the results may hint toward a deficient result, it was the opposite. If we leverage the difference between E1 and E2 results, 6/10 participants agreed that the recording process was more intuitive, directly related to the upgrades performed on the robot and the light cue between the state’s switch. The time lag between the robot movement and the human was more in sync, which translated to further exploration of the robot’s capabilities and an increment in the average interaction time, which had an increase of $38.85$ $\mathrm{s}$ (denoted with the red line in Figure 6). The latter can be observed as a partition by participants and daily cumulative interaction time in Figure 5, where it can be seen that there is an increment of the interaction time by participants in E2.

Yōkobo’s SUS score after E1 was 66.11; as for E2, Yōkobo obtained an average grade of 63.25. However, once it was taken between the participants who were part of the first and second experiments, it was 61.43, while the new users rated it with a grade of 67.5. The old participants’ group decreased its rating on average by 2.5 points, with two edge cases with decrements of 10. Although the grade seems low, the value is a passing grade. On average, a system rating was about 68; an example of a grade 61 product is the Apple Watch [49]. The difference between the two groups can be related to the previous experience that the participants already had with the robot and the expectation of what the current version might have had as an added feature. Both grades reflect that the interaction still needed tuning to be more reflective of the pairs. Overall, 5/10 participants agreed the robot was easy to use, intuitive enough to learn by themselves, would like to continue using it, and was well integrated but needed further improvement while recording; 2/10 were neutral; 3/10 rated that the system needed further improvement. With this feedback, it is likely that by improving the recording behaviour and filtering which points are saved on the message (disregarding the animation), Yōkobo’s usability could increase.

6.3. Regarding Perception and Reception

The graffiti wall showed a positive perception of the greeting motions. Participants liked being welcomed by Yōkobo. Some even interpreted them (i.e., the greeting motions) as good manners, though it was not programmed so. For example, two participants thought the bowl moved down to interact with it more easily.

From the log, the time users spent interacting with Yōkobo, from the Wake-Up to the start of the Go Back to Rest state (Figure 6) was analysed. The maximum interaction time spent for E1 was 826 $\mathrm{s}$, the minimum was 16 $\mathrm{s}$, with an average, over the two weeks, of 76 $\mathrm{s}$. It can be noted that this time for the second week showed an average increase of $21.9$ $\mathrm{s}$; this corresponds to more V interacting with Yōkobo, as can be observed in Figure 5a. For E2, the average was $113.24$ $\mathrm{s}$, with a maximum of 1307 $\mathrm{s}$ and a minimum of $13.39$ $\mathrm{s}$. The minimum time corresponded to a person detected but not a complete interaction in both cases. The main difference that could be seen in Figure 6a between experiments is an increment in the average exchange and significantly fewer faulty sensor triggers. It shows how the upgrades manage to reduce this false interaction, which also explains the reduction in the sample quantity. The trend also demonstrates that extensive exchanges appeared during the week and were not condensed to single days. In contrast to E1, the average visitor interaction time for the second week did reduce; this can be partly attributed to the experimental days being in the middle of the holiday sessions and the ending of the school semester. Still, there was a significant increment in the SP interaction when comparing Figure 5a,b. Nonetheless, the maximum time can be associated with the mimic procedure and the participants’ curiosity; measured with the questionnaire, by the end of week two of both experiments, were high, with an average rating of 3.7/5.

Participants had longer interactions with Yōkobo during E2 with an average increment of $37.24$ $\mathrm{s}$, not only the new participants but also the ones that participated in E1. For example, participant 2 (P2) presented more extended and frequent interactions than before. In general, this trend and the increment of the average interaction time plus the improvement of the semantic analysis evaluations (Useful and Familiar ) showed that the user wanted to use the robot continuously. However, in both experiments during the second week, the visitors, especially for lengthy interactions, may have had a perceptual bias due to their desire [50] and perceived scarcity [51] to use the robot to its full capabilities as the SP could. Nonetheless, both experiments provided enough arguments to show that participants wanted to further interact with the robot, providing a clear guide towards surpassing the novelty effect. However, more data are needed to make a conclusive argument. Due to the slow technology principles used for Yōkobo, the more time the users spend interacting with the robot, the better it is. We are looking for them to discover the functioning of the robot by themselves, we are not looking for efficiency (as it can be with usual robots or products).

Another interesting fact from the logs of both experiments is that several participants split into subgroups of two or three users. This formation introduced an increase in interaction time. Yōkobo may have been a conversation starter and a tool for users to find common ground. By creating these social scenarios, the participants further understood Yōkobo’s capabilities on their own or in their subgroups, making the interaction with the robot clearer. This may explain the average ’decreasing’ rating for the fearfulness sentiment.

A related observation is that the extended interactions happened when clocking out with the RFID tag, meaning between 16:00 and 18:00. We hypothesise that one possible explanation is because of the daily schedule situations. When users first tag in, they might be rushing to start their daily activities; however, when tagging out, they may have extra time to spend with Yōkobo and their colleagues.

The collected data from the semantic evaluation are shown in Table 1. The Concepts (Cpt) that were analysed are presented, along with their respective Dimensions. Dimensions tend to the positive side (lower than 3) with a standard deviation around one. When comparing the different concepts, we found that the reception towards Behaviour was positive. A total of 6/10 participants agreed that the system was dynamic, smart, and straightforward, in line with the proposed design notions; 2/10 were neutral, and the other two participants thought it was static and unintelligent. Responsiveness is the dimension with the highest grade, indicating that the robot’s system flow was not yet appropriate, and users expected a more fluid interaction. The latter was associated with the recording process difficulties described above and the transitional animations between the states, which may be perceived as too artificial. Concerning the dimensions of Interaction, although on the positive side, they presented a negative tendency with the lifelike and emotional dimensions having a mean value near 3. As seen in the semantic analysis (T-Q), the logs and the messages recorded (T-L), and the behaviours were not fully understood. Participants were neutral regarding the emotional content of the animations, with an average value of 2.9 and a small standard deviation. On the other hand, Appearance followed the same positive trend as Behaviour, with desirability being the least positive dimension. The first two concepts (Behaviour and Interaction) could be improved by making the robot more intuitive (legible trajectories or special gestures) and adding other perceptual signals for the user, such as a sound or a clearer light display.

Interestingly, 7/10 users found Yōkobo more complicated over time, especially for E2. The Confusion sentiment by the end of E2 increased by 0.3. It would be necessary to improve the message cues and sequences for the user to understand it better and differentiate between Yōkobo states, movements, and animations. Similarly, the Curiosity increased once the recording functionality was enabled before decreasing at the end of E2. This may have appeared to hinder the overall results; however, it was, in fact, the opposite. Yōkobo is designed with the notion of slow technology, such that the users cannot fully understand the robot in a single interaction. Instead, it is a medium or a reminder of what is around them [5]. The users may be surprised or confused by the robot at different moments of their interactions and have continuous shifts in their perceptions.

Moreover, participants were asked about the word they would use for each part of Yōkobo. A total of 7/10 used anatomic words, such as head, body, and torso. Using these words to describe the robot, they identified it as a living creature, utilising pareidolia. The qualities participants used to describe Yōkobo were smart, modern, and cute.

According to the graffiti wall, participants seemed interested in Yōkobo, and the shape was trustworthy. The concept of robject was also evaluated. We regularly observed in the answers some mentions about the bowl. Some users wrote about what was inside: words such as “pick up” or “offering” were used.

Finally, regarding Yōkobo’s office integration, 8/10 participants described its interaction as welcoming and 2/10 were neutral. After setting their RFID tags on the reader, they felt it displayed a greeting attitude, energetic, and whimsical. A total of 8/10 found the clock-in–out function useful, and preferred to do it this way; the other two participants were neutral about it.

7. Conclusions and Future Work

We developed a semi-abstract robot with an HRHI approach, using expressive motions as interaction means. Then, we performed two different experiments in our office to test Yōkobo’s robustness, perception, reception, and usability. It proved its ability to work for extended periods of time. Moreover, participants felt welcomed and greeted by it. Through small movements, aided by different sensors and mimicking processes, it was possible to create the perception that the robot was intelligent and a “welcoming partner”.

Longer experimentations were ongoing with the target population to confirm some points of the findings. As mentioned in the discussion section, the novelty effect seemed to have been overcome somehow. However, with a longer experiment, over several weeks, we hope to observe what happens when a routine settles and the usage of the robot becomes seamless. Another point is the feeling of the partner. During our two-week experiment, 40% of the participants could feel him/her, proving the capability of this concept. However, this number might seem low; one reason may be that building up this feeling may take time. Moreover, one of the limitations of our experiment was in using arbitrary couples, i.e., real couples in the ongoing experiments would have stronger inter-connections; and it will probably be easier for them to feel their partners through Yōkobo. The realised experiments gave us some clues about the validity of some of our concepts, as a first step.

Including modularity in the design made it possible to extend Yōkobo’s capabilities to suit the office environment, even if its original purpose was for the home.

In addition to making the messaging functionality and motion more legible, we plan to add pressure sensors to enable tactile interactions and object identification inside the bowl. Moreover, to detect some ambient sounds or greetings, we will add a microphone. Another way to improve the usability of Yōkobo will be to add sounds linked to the movement of the robots. Even if the SUS gave us a good score, around the average, some improvement can still be made, especially concerning the recording process. However, since our goal was to be a part of the slow technology movement with Yōkobo, it is not necessary to have efficient and straightforward usability. Finally, the software architecture requires multiple modifications to make it more robust and to build a more responsive framework to further develop the HRI.