A Study on the User Experience to Improve Immersion as a Digital Human in Lifestyle Content

Abstract

With the expansion of the digital environment and the metaverse, and the intervention of artificial intelligence, interaction in the virtual world is becoming more active. Humans are discussing the social reality of the user experience in this virtual space. Technology has created an object called a human-like digital human to enhance human immersion in the metaverse. This will become a factor that improves immersion so that the experience of the virtual world becomes more intimate for humans who feel unfamiliarity, alienation, and the rejection of new technologies. However, a clear definition, expression, and approach to the digital human are still being continuously improved. This article studied the direction of improvement in factors that can improve immersion in the user experience. In the process of communication between humans and digital humans, a qualitative survey was conducted based on the five human senses, where the user experience of the digital human was central. In-depth interviews were conducted with 20 men and women regarding their digital human experience, targeting Generation Z, who are familiar with the digital environment. Using NVIVO, global qualitative research software, 1000 main frequency words were derived, and the top 20 words with the highest frequency were classified into emotions and the five senses to analyze their correlation. As a result, we found that the mental models of developers and users are different in the digital human experience. Users felt more comfortable and a higher degree of intimacy when they saw the digital human as a technology, and the technology that the developer was showing was more focused on the external aspects that look similar to a human. It was found that, in order for users to immerse themselves in the digital human, various non-verbal expressions using the five senses should be further developed, rather than focusing on the human-like appearance. This study intends to serve as a cornerstone for research that can improve immersion in digital humans, with a high potential for future development.

1. Introduction

Metaverse is a new word combining ‘meta’, meaning ‘transcendence’, and ‘universe’, meaning ‘world’. In futurism and science fiction, the metaverse is a hypothetical iteration of the Internet as a single, universal, and immersive virtual world that is facilitated by the use of virtual reality (VR) and augmented reality (AR) headsets. In colloquial use, a metaverse is a network of 3D virtual worlds focused on social connection [1,2,3]. Although a clear definition of the concept of the metaverse has not yet been established, it refers to an environment that interacts with the virtual world, and has the characteristics of reducing people’s rejection of the virtual world and increasing their sense of immersion [2].

Gartner [4], an American information technology research and advisory firm, has published trends that will continue to drive the use of metaverse technology over the next three to five years. It is a type of AI-based representation that can interpret speech, gestures, and images, and can generate its own language, tone, and body language. This presents opportunities for a new digital ecosystem, supported by technologies that bring individuals and organizations together to innovate in new ways and support interactions.

The digital human’s facial expressions and facial muscle movements are constructed based on data analyzed from hundreds of human expressions, and emotions are implemented using sophisticated artificial intelligence technology. With the recent development of eye recognition, emotion recognition, voice recognition, and artificial intelligence technologies, digital humans with more natural and realistic facial expressions have emerged. However, unfamiliar elements still occur in communicating and interacting with humans, and the scope is limited to the development of visual elements. If virtual reality is applied to reality, the method of making the conversation with the machine more intimate begins with a deep understanding of the nature of human beings and leads to judging the recognition of human-like emotions in order to confirm identity. Therefore, digital human research based on more in-depth human emotional research is needed to provide an optimal user experience.

In general, the current digital human is expressed in a human-like form, which increases the realism of the digital human to increase the co-presence and the social presence that interacts with the real human being, in order to maximize user satisfaction and utility. However, when implemented similarly to a real person, an uncanny valley may occur. The uncanny valley was a theory proposed by Masahiro Mori in 1970. The more an object resembles a human, the more favorable the emotional response. However, when the resemblance reaches a certain level, a strong rejection is felt [5].

Therefore, this study does not aim to harmonize humans and machines (in the form of a digital human), but aims to reduce the rejection between humans and machines. As technology advances, it is essential to understand the fundamental way of thinking, behavior, and emotions of humans. When people maximize their experience and receive the optimal experience they want, the value of the product and service is also maximized [5]. This is not a simple superficial understanding, but a deep psychological state, and an inner conceptual study of the human being is lacking. Here, we have studied ways to provide a new user experience based on understanding, not only of human-like shapes and movements but also human emotions. It is necessary to look at the definition of a digital human, the stages of development, and the technology for creating a digital human. This study aims to understand the development methods through new user experiences in the design field, as well as the technical part regarding how the digital human is currently affecting our daily life through human empathy, and in what direction it should develop in the future.

The main aim of the study was to recognize the difference in concepts regarding the unfamiliar emotional difference of real users by increasing the realism of the digital human. Furthermore, we aimed to study how to provide optimal emotions to humans by increasing the social presence of humans and machines according to that point of view. First, from the literature research undertaken, this study provides an understanding of the background of the definition and meaning of the digital human from previous researchers, and summarizes development cases and their limitations. Second, we investigated immersion and social reality to provide an optimal experience through design and the user experience theory. Third, based on the literature and case studies, an in-depth interview of real users was conducted to determine the difference between empathy and rejection between real users and digital humans, and the gap between humans and machines through the essential understanding of humans.

The five senses of human beings allow us to experience immersion based on sight, smell, taste, touch, and sound. Understanding human beings is intended to suggest the development direction and implications of a digital human based on the five senses.

2. Digital Human

With the commercialization of artificial intelligence, technologies for understanding and empathizing with human thoughts and emotions have developed rapidly. In addition, objects to communicate with humans are evolving by depicting them as similar to humans. Since the fourth industrial revolution, as various technologies have been optimized, segmented, and diversified, the boundaries between reality and virtuality have become increasingly blurred. Following the commercialization of ‘Alexa’ started in Germany in 2016, humans have embraced more digitalization in their lifestyles. As artificial intelligence, which had only been encountered with voices, is gradually expressed as virtual characters, it is showing the role of human-like objects in replacing people in the digital environment. Therefore, this study intends to investigate the meaning of this virtual human, which is currently not precisely defined.

2.1. Definition

With the development of digital technology, humans have got to know the medium that communicates with machines in the digital environment as objects that are similar in appearance to humans. This has developed from one-way communication that helped humans with preplanned and manipulated technology, to two-way communication such as avatars, emojis, and game characters that can even interact with emotional expressions as characters that replace humans. With the invention of artificial intelligence, not only verbal communication expressed through sound and writing, but also non-verbal communication expressed through physical appearance, gestures, spatial actions, and emotional expression became possible. For this reason, researchers and companies in related fields are defining these objects in various ways, including virtual humans. The technical terms are defined slightly differently depending on the technology and category, such as digital twin, virtual human, and digital double.

According to

Table 1. Definition of a ‘digital human’.

Name/Company	Definition
Monica Collier Scott Manion Richard de Boyett	A digital human resembles a human in form, features, and expression. It can express tone of voice and body language. Many digital humans can sense the user’s body language, expression or tone of voice. This enables the digital human to respond appropriately [6].
Emmanuel Sirimal Silva·Francesca Bonetti	A digital human can be defined as a life-like being, powered by artificial intelligence (AI), with the capability of conversing, communicating, and creating an emotional connection, similar to any other human being [7].
Deloitte	A digital human is an avatar that can produce a whole range of human body language, backed by artificial intelligence that can interpret clients’ input and give back to them not just the facts they need but the appropriate non-verbal response as well [8].
UneeQ	Digital humans are AI-powered, lifelike personas that can see, hear and understand you, meaning they can recreate “real” human conversations. The digital human can easily connect to another “brain” to share knowledge. Importantly, digital humans embody the personality, voice and nature of the brands they work for. They can show emotions such as happiness, empathy, warmth, and friendliness—they can crack a joke or show support through their actions, tone, and body language [9].
Unreal Engine	Digital humans are believable characters created by using high-quality character shader techniques, data scans of actors, and improved geometry workflow [10].
Virtuals	A digital human is, in short, a photorealistic 3D human model. To be more precise, a digital human is a complex 3D human model which takes advantage of recently developed high-end features to produce realistic results in terms of appearance (skin shading or hair grooming) and movement (accurate rigging and animation) [11].

, Deloitte, the world’s leading consulting firm, described digital humans as ‘avatars capable of generating all human body language’. They are a being supported by artificial intelligence that can interpret customer input and provide appropriate non-verbal responses, as well as the facts that the customer needs. To be more specific, a digital human is an AI-powered, human-like virtual entity that provides the advantages of both AI and human conversation. They can easily connect to any digital brain to share knowledge. They also interact using verbal and non-verbal cues and are available 24 h a day, 7 days a week.

A digital human developed from the concept of a digital twin is a form based on the appearance of a real person, and combined with AI, speaks a language appropriate to the situation. If you look at the case of a system developed using a digital human, you can mainly see the role of an assistant who can help in real life on behalf of a real human being developed based on user experience. Therefore, this study defined the digital human as follows, based on the various definitions above: A digital human enables two-way communication with humans through verbal and non-verbal expressions based on user experience. In addition, the object is expressed in a form similar to a person, and the means of expression aims to reach a level of emotional expression that can be expressed in humans and includes the five senses as well as appearance.

2.2. Case Study

Cases of digital humans are summarized through lifestyle content that can be approached in a way that is friendly to humans. First of all, research in the lifestyle area to prepare criteria for classification of lifestyles has been conducted in various directions. Among them, Reynolds et al. (1977) expressed the view that it is appropriate to divide the life of consumers into different categories such as clothing, diet, and living life, rather than viewing the entire complex and diversified life of a consumer as a single object of analysis [12].

In this study, it was judged that it is appropriate to divide lifestyle by category according to the argument of Reynolds et al. Therefore, based on previous studies, the lifestyle area was divided into clothing/food/main life, leisure life, economic activity, and education/medical field, and the range of activities of the digital human was organized based on this. If the currently active digital human is classified according to the lifestyle area criteria summarized in this study, the occupation groups shown in

Table 2. Digital human by occupation in terms of lifestyle content.

Clothing Life	Diet Life	Housing Life	Leisure Life	Economic Life	Education/Medical Treatment
Model Stylist Clothing Guide Beauty Consultant	Chef Barista Food Guide	Real Estate Curator Tour Guide	Anchor Influencer YouTuber Singer	Bank Service Clerk Secretary Worker	Teacher Training Coach Health Coach Health Guide

are derived.

Specific examples of this are summarized in terms of how they interact with humans through job characteristics and activity interfaces, as shown in

Table 3. Cases of digital humans in terms of lifestyle content.

Content	Interaction	Interface	Name	Job and Features
Clothing	Sight	SNS	Shudu	The world’s first digital model
	Sight	Media	Slide Capture	Metaverse fashion show using digital human
	Sight	AR	Fiona Xie, Vogue	Fiona Xie, a real model, is created as a digital human and wears real clothes on behalf of consumers
	Sight, Sound	SNS, YouTube	Lil Miquela	The world’s most influential influencer, working with several fashion brands including Prada and Dior
	Sight, Sound	Media	Manahou (UneeQ)	It is a virtual assistant that helps shoppers by explaining clothes. Currently only video format is accessible as an initial search
	Sight, Sound	Online Platform	Yumi	Beauty consultant produced by SKⅡ, using the same way of speaking as a real person
Diet	Sight, Sound	Online Platform	Ruth (Soul Machines)	You can ask questions about cookies and guide cookie recipes according to your preference
	Sight, Sound	Digital Interface	Bella	Digital Barista. Receive coffee orders through voice ordering and make coffee
	Sight, Sound	YouTube	Chef Jade	International chef based in Rome. Explore the world around you with a focus on food
	Sight, Sound	SNS	KFC	A digital human mascot created by KFC, created for influencers and Gen Z
Housing	Sight	Digital Display	Imma	Influencer, IKEA model. Examples of marketing in the form of being in a real room
	Sight, Sound	Online Service	Ella	Real Estate Curator. Guide to future real estate and living environment
	Sight, Sound	Digital Display	GS Construction	Produced based on a case introduced in the ‘Xi’ launch, a real person. Provide information on the apartment
Leisure	Sight, Sound	SNS YouTube	Lu do Magalu	The world’s number 1 virtual influencer. First appeared to market on behalf of Magazine Luiza, one of Brazil’s largest retailers
	Sight, Sound	SNS	Roxy	Korean influencer, a ‘forever 22-year-old’ woman who is interested in fashion
	Sight, Sound	Broadcast	Zhang Zhou	A virtual announcer created by the Xinhua News Agency of China. One of the first male announcers, ‘Zhang Zhou’, was modeled on the model
	Sight, Sound	Mobile	Blackpink	K-pop representative girl group. First metaverse concert
	Sight, Sound	Digital Interface	Saejin	A K-pop boy group called ‘Superkind’. A group consisting of four humans and one fictional character ‘Sejin’
Economics	Sight, Sound	Online Service	Amelia (UneeQ)	The world’s first virtual assistant for businesses. Conveying the human side of a conversation, such as expression, emotion, empathy, and understanding
	Sight, Sound	Digital Platform	Fetema (Bank ABC)	Digital employee used by Bank ABC
	Sight, Sound	Digital Display	DeepBrain AI AI bank clerk	DeepBrain AI produced a kiosk-type AI banker. Currently running at KB Kookmin Bank in Yeouido. First greets, then uses non-verbal expressions such as smiles and nods
Education /Medical Treatment	Sight, Sound	Online Service	Florence	Advice to help users quit smoking. Participate in real-time conversations and help plan to quit smoking
	Sight, Sound	Mobile, Online Service	Digital John Kirwan (DJK)	DJK provides services to help users plan to improve their sleep

. More specific details for each field are explained in Appendix A.

Among the various cases in each field, representative cases were selected and organized according to the interface. In addition, all cases were combined and the overall ratio was expressed in Figure 1 and Figure 2 as a graph.

The following results were obtained when the cases of digital humans were summarized by focusing on the five senses in the experiential elements for immersion in digital humans. First, in terms of ‘clothing’, visual interaction is important; that is, in the field of fashion, interaction centered on vision is predominant. In addition to this, there were interactions that added sound, but in fashion, the sense of smell, which stimulates the overall atmosphere with the sense of touch or scent according to the expression of the material, can also be an important influencing factor. In this regard, more diverse sensory expressions are needed to stimulate the human senses. Second, when interactive interactions such as recognizing and answering the user’s voice regardless of the area were emphasized, non-verbal expressions such as a natural nod gesture or facial expression such as a smile were important. However, when interacting with people through the social network service and YouTube, such as virtual influencers, visual expressions of human-like appearances and situations were specific, but since these involved one-way communication, there was a lack of reactive expression for immersion in interaction. Third, when using a digital device such as AR or a touch screen, interaction through touch and click was used. There were some limitations to seeing this as an interaction using tactile sense, and in the end, most of the interaction was only through sight and sound.

2.3. Limitations of the Digital Human

The advent of ‘Kyoko’, developed in Japan in 1996, was the first appearance of a virtual human. This led to the emergence of the Korean cyber singer Adam in 1998, but this disappeared due to limitations of the technology. However, with the recent development of the virtual world and technology, the awkward movements of the early virtual human have been developed more sophisticatedly and delicately through motion capture technology, and have developed into a form that resembles a human figure and reappeared under the name of a digital human.

The metaverse should be distinguished as an environment that is different from the fundamental concept of the expansion of the human world. As the metaverse world is merely a means, not a result of replacing or changing reality, it is necessary to fill the gap between virtual reality and reality in terms of the user experience.

Any technology finds its value when it essentially helps users focus, rather than replacing reality. Human efforts to make machines more usable at human–computer interfaces are geared towards enhancing the sensory points of contact between computers and improving the physical design. However, interfaces are more than what computers see and hear; human–computer interfaces are the creation of personality, designing intelligence and making machines aware of human emotions [13].

In the past, human–computer interface research was divided into two parts, which did not merge into one until 20 years later. One focused on interactivity and the other focused on sensory richness. The use of multiple senses in the experience is associated with a high level of interaction. Since the advent of virtual humans in 1996, various attempts have been made, but development has been slow due to technological limitations. The digital human has been created as a virtual human that resembles the human figure and behavior to the extent that it is difficult to distinguish it from the real world, and the scope of the development and use of digital dormancy has been limited. Accordingly, in this study, the digital human was defined through the development of the digital human and the analysis of previous studies, and the user experience of an advanced form of the digital human that can be applied to lifestyle was proposed by overcoming the technical limitations.

3. Design and User Experience

3.1. Design from the Experience Study Perspective

According to IDEO, good design that is technically expressible and economically feasible is what people need. In addition, the Samsung Economic Research Institute has said that a good design is a design that provides a good experience to customers. After all, approaching the design from the customer’s point of view can create a good experience. There are various meanings of design and the meaning is also changing according to the development of the environment and social culture of the times, see

Table 4. Changing the meaning of design.

Period	Characteristic
Craft design 1850–1900	After the industrial revolution, art and craft companies emerged as artisans of design companies in England and the United States.
Functional design 1900–1930	Emphasis on practicality, functionality, and production efficiency of products.
Commercial design 1930–1945	As the design focus shifted from producers to consumers and the demand for design increased, design specialized companies were activated.
Design specialization 1950–1975	Recognized as an important function of corporate management in business administration following the expansion of the design domain.
A company’s core competitiveness 1975–1990	Diverse and unique designs have been popular since post-modern times.
A new agent of innovation 1990–2010	Design was recognized as the top priority within the company and this led product innovation.
Empirical design 2010–Present	Human-centered design that provides a good experience and sensibility to users is important.

.

As the center of economic value changes from the agricultural economy to the industrial economy, and recently to the experience economy, the traditional role of design in delivering good experiences, formative activities, is becoming more important. Understanding how people behave requires not only knowing how to interact with others, but also requires designers to particularly think about how to deliver good experiences and emotions to users. As technology advances at a rapid pace, good design starts with understanding people and requires communicating what actions are possible and what is happening to them. With the development of technology, a system in which various products and services are fused has emerged, and the importance of the user experience has increased. As the interaction between humans and technology is advancing, the total experience that humans obtain through technology is becoming more important.

Therefore, there is a growing need to learn about the various definitions and components of experience. Experience has been studied in other fields, as well as emotions, but various attempts are lacking in terms of in-depth research and application in the design field. It is difficult to find an exact definition because the concept of experience is inclusive and diverse, and its meaning is ambiguous. Experience is a private event that occurs in response to a stimulus, and it is usually induced rather than self-generated [14].

The philosopher and psychologist who proposed the essence and concrete concepts of experience was John Dewey. He refers to man as a living organism, and mentions that, in order to survive, man needs and interacts with other environments [15]. Experience encompasses all interactive processes, and through human experiences and actions, the interaction and harmony of organisms is achieved, and the experience is important. In his book, Ken Nah refers to experience as an unfixed, eternal process of novelty. He also states that the same two experiences do not exist together, and from a design point of view, individual events occur in response to stimuli or participation in or observation of those events. He mentions that experience arises through a stimulus and is induced by a stimulus or trigger, rather than spontaneously occurring on its own [16]. After all, in order to provide an optimal user experience, it is essential to understand the user’s emotions, the way users feel, and the factors that affect these emotions.

There are various experiential factors that induce these experiences. According to Bernd H. Schmitt, experience is often driven by emotion rather than reason. It was referred to as feeling, thinking, acting, and relating. The factors that induce experience are shown in

Table 5. Experience Triggers.

Factor	Content
Sense	Sensory experience through sight, sound, touch, taste, and smell
Feel	Emotional experiences ranging from positive emotions to strong emotions such as joy and pride
Think	Cognitive and problem-solving experiences that appeal to the customer’s intellect, such as surprise, curiosity, interest, and provocation
Act	Spontaneous, motivating, and inspiring lifestyles and interactive experiences
Relate	Experiences that connect individuals with their ideal self, others, and cultures, including senses, emotions, cognition, and behavior

below.

3.2. User Experience and Immersion

Social presence refers to the degree of feeling toward an object existing in the same space, or the ability to sense a relationship [17], and it refers to a state of cognitive and emotional connection that occurs when humans and machines interact with each other in a virtual space. Intimacy with a digital human is an experience that is connected with psychologically felt emotions toward an object. As digital human technology develops, user experience design is required to increase realism and immersion.

A mental model is a model that people have of themselves, others, the environment, and things with which they interact [18]. According to Indi Young, mental models require a deep understanding of people’s behaviors, motivations, and thought processes, as well as their emotional and philosophical backgrounds [19].

As a factor inducing experience, the five senses are essential components: sight, sound, smell, taste, and touch. For natural interaction through the mental model of humans and technology, a specific design is required based on an understanding of the factors that induce the optimal experience of the five human senses. Even if the technology is perfect, if it does not provide the optimal experience that humans want, the development of that technology is useless; the optimal experience occurs when humans have positive feelings toward it [20].

Ken Nah said that the reason why experiences are important is that emotions are always present, and the experiences change depending on what kind of emotions you create, and this has a lot to do with psychological issues. In addition, Donald Norman noted that the optimal experience occurs when the user has a positive sensibility for an object. In order to produce a positive human emotion and an optimal experience, the experience can be designed as an essential element to provide pleasure, beauty, and satisfaction by stimulating the five human senses, which are sensory operas.

According to

Table 6. Classification of the five senses [21,22].

Sight	Smell	Taste	Touch	Sound
Sight is the most developed of the senses as 83% of the information that we receive is via sight, and colors and shapes affect a human in different ways. Visual objects are a powerful tool to grab attention.	Humans can identify approximately 10,000 scents. Scents trigger stronger memories compared with the other senses. Our sense of smell is connected to our limbic system that regulates our emotions and memories, which is why it is the sense with the most memory recall.	People enjoy new and exciting tastes. These tastes determine their decisions on what to buy or where to buy it from.	Touch provides us with an important amount of information to make an informed decision. It is a sense that is capable of generating an experience that is more connected and interactive for the customer.	Our sense of sound is always active even when sleeping. The use of ambient noise, conversation, and music all affect the way a customer will feel about a product.

, experience is through the human senses, such as sight, smell, taste, touch, and sound. By expanding information regarding the human senses, it is possible to solve situations that are difficult to actually experience in the real world. This helps to create an immersive experience. To enhance immersion, the experience should stimulate the five senses, providing lasting positive experiences for humans.

4. Qualitative Research

Digital humans are being introduced to the public in various ways through lifestyle content. Through easy-to-access advertisements and game characters, real actors are imitated or non-real people are created and shown as virtual characters. Recently, the digital human has started to be used in every aspect of the lifestyles of real people because it imitates the human figure. Leading fashion trends and appearing as influencers who actively engage in social networking creates intimacy with humans through celebrities, announcers, advertising models, service industry workers, and educators. However, since a human figure is being imitated, this can give familiarity, but it also has the disadvantage of possible awkwardness due to the external features shown in non-verbal expressions. Therefore, qualitative research was sought to find the characteristics of a digital human that humans feel unfamiliar with and to reduce factors that hinder immersion.

4.1. Approach

In-depth interviews were conducted with people from Generation Z, who are most exposed to the digital environment. Factors affecting content and communication experiences in the digital environment, focusing on the five senses, were investigated. According to Csikszentmihalyi’s theory, digital media induces a strong sense of immersion in the experiencer when visual elements and narrative elements of content are added [23]. Considering the two factors mentioned here when humans and digital humans interact, these can be expressed as verbal communication and non-verbal communication. Therefore, this study tried to determine the elements of digital humans that human beings are immersed in via these two types of communication, and the emotions they feel through the five senses. The main contents of the survey were based on the immersion factor, asking questions about the experiences of human expressions when interacting with digital humans. In addition, an interview was conducted based on the experience of a digital human who is friendly to humans in terms of lifestyle content such as advertisement models and game characters.

4.2. Method

The subjects of the interview survey were 20 Generation Z people born between the end of the 1990s and the beginning of the 21st century, through 13 persons of 1:1 interviews and 3 focus groups, regardless of gender. Considering the characteristics of Generation Z’s experience with the digital environment, the gender distinction between men and women does not have much meaning, so it was excluded from the interview criteria. The interview subjects consisted of high school students familiar with the digital environment, college students majoring in graphic design, and office workers in their 20s, ranging from general users to extreme users. The central keywords for the qualitative research included experiences of communication with digital humans that can be easily observed, from the environment of the platform experienced in the digital environment to the various situations experienced as a character instead of the real person, as shown in

Table 7. Main questionnaire for the in-depth interview.

Keyword		Questionnaire
Digital Environment	User Interface	-Experience in the virtual space or the metaverse -Cognitive ability for digital humans
	User Experience	-Interaction with others in the digital platform -Communication with a digital human
Immersion	Experience	-Five senses for immersion in interaction with a digital human -Difference in senses between digital humans and humans
	Emotion	-Positive and negative emotions for digital humans -Factors affecting emotional immersion
Content	Lifestyle	-Opinions on recent digital human activities -The role of digital humans to replace humans in terms of lifestyle content

. Detailed keywords related to the five senses and immersion were set.

In addition, two examples of digital human beings familiar to the interviewees were selected. Figure 3 [24,25] is an example of ‘Mu-A-in’ who mimics the appearance of an actual actor Yoo Ah-in as an advertisement for Musinsa. This shows only movements and gestures based on non-verbal expression, and the actual graphic design has a strong feeling of a graphic model depicting Yoo Ah-in rather than a realistic human figure. Figure 4 [26,27] is a linguistic-expression-based case in which a character depicting actor Heo Seong-tae’s appearance in a very realistic way, such as skin texture and hair, is produced in the same way as the actual actor’s voice.

All text data from in-depth interviews were analyzed using ‘NVIVO’ (a qualitative data analysis computer software package produced by QSR International). First, from all transcripts from the interviews, up to 1000 words were extracted in order of frequency. Among the extracted words, 12 words related to ‘emotion’ and 14 words related to the ‘five sense’ were found. In addition, to see the relationship and depth of meaning of these words, a ‘2D cluster map’ similar to Figure 7 was executed. Finally, the research data were analyzed to find empathy points for immersion based on the correlation between the frequency of words and the cluster map.

4.3. Result

According to the text data extraction results, 94 results relating to digital humans were found. Box 1 lists the keywords that consist of technologies that humans use to respond to digital humans and words that express their emotions.

Box 1. Main Keyword for ‘Digital Human’.

Digital, Human, Artificial Intelligence, Awkward, Appearance, Expression, Virtual Space, Similar, Commercial, Same, Novel, Action, Technology, Rosy, Realistically, Scary, Communication, Intimacy, Face, Voice, Activity, Experience, Virtual, Unfamiliar, Human-like, Natural, Different, Gesture, Sense of Difference, Gap, Reality, Fake, Space, Emotion, Decorate, Countenance, Lifestyle, Online, Look, Figure, Image, Influencer, Sense, Graphic, Sight, Platform, Skin, Reluctance, Game, Memory, Pupil, Meet, Hair, Gather, Uncomfortable, Picture, Strange, Others, Limit, Environment, Positive, Drama, Mind, Product, Time, Cyworld, YouTube, Story, Sound, Friend, SNS, Private, Lying, Falsehood, Anxiety, Common, Empty, Concern, Separation, Muscle, Criteria, Sing, Slowly, Detail, Deepfake, Relay, Talking, Media, Model, Imitation, Sensitive, Development, Unnatural, Unpleasant

The digital human experienced through virtual space and lifestyle content was not perceived as a human being and was mainly expressed as a technology. Therefore, in terms of emotions, words such as awkwardness, fear, alienation, rejection, and discomfort were predominant. In addition, looking at the word frequency ranking, it can be seen that the emotional content of the extracted words was overwhelmingly large Figure 5.

It was observed that humans were unfamiliar with digital humans through the number of negative words related to ‘emotion’. Furthermore, as a means of showing this emotion, it was found that appearance, behavior, face, voice, and movement had a great influence via words related to the ‘five senses’. As a result, there were many elements related to non-verbal communication, and the senses that presented the greatest influence were sight and sound, Figure 6.

The cluster map in Figure 7 shows the importance of the size of the circles, and the distance between the circles shows their correlation and depth. Moreover, the color of the circle was grouped according to the relationship of the words. Therefore, it is composed of words that are seen in the realm of a digital human and a human, and words that relate to the interface between the two. Looking at the area of contact, it shows that the sense of heterogeneity between the two areas can be reduced and the gap can be reduced depending on the space, expressive power, atmosphere, technology, and means of communication for empathy between the two areas.

When looking at the results of each data analysis and the context of the interview, the point that humans sympathize with in relation to digital humans was ‘unfamiliarity’. In addition, it was possible to see the difference between the viewpoints of the digital human expressed by the developer and the digital human experienced by the user in the elements inherent in the phrase ‘sense of difference’. Currently, artificial intelligence companies that create virtual humans are developing technologies to make them as similar to humans as possible. However, it was this ‘humanity’ that people felt most uncomfortable with when they encountered the digital human.

• Interviewee A: When a digital human takes actions to make friends with humans, it seems that is good or that the marketer writes well. In the end, because you don’t feel like a digital human, you’re reminded of the real person behind them.
• Interviewee B: When I watch Disney’s or Pixar’s animations, I never thought that I felt alienated because I was different from humans. Some even think that they are ‘real people’, but when I see a digital human, I get a lot of the feeling that it’s fake. Animations are also made in 3D in a way, but I wonder why digital humans don’t have as natural facial expressions as their characters.
• Interviewee C: It feels more comfortable to think of the digital human as just a technology. Therefore, it feels more awkward and uncomfortable to embody the same human freckles and skin texture. Rather, I feel closer through the facial expressions, gestures, and voices that they show when communicating with me

Looking at the interviews excerpted above, most of the interviewees thought of digital humans as a technology belonging to the digital environment of the future, and when they perceived it as such, they accepted them more comfortably, e.g., the thought that the digital human is also a human-made technology, and that it is a piece of content that contains the thoughts and intentions of the developer.

Therefore, in order to study how humans can become more immersed in digital humans, the words related to ‘sense of difference’ are shown in Figure 8. Among the five senses that humans feel when interacting with digital humans, it is necessary to accurately recognize the points that help immersion and improve the current development direction. The immersion in a digital human can be improved only when the gap between developer and user empathy is closed.

5. Conclusions

Due to COVID-19, the demand for non-face-to-face content has increased due to the expansion of such content and the advent of the metaverse, and the demand for digital humans is continuously increasing. It is hoped that studying the definitions of various digital human beings will allow suggestions regarding the future direction of the development of digital humans. The method of defining a digital human is different depending on the purpose, application field, channel, and medium, but it appears to always include imitating the appearance of a real person or describing a behavioral pattern. The field of application of digital humans is very diverse, and it is particularly actively used as cultural content. In addition, it expresses the human figure so well that it is difficult to distinguish it from a real person. However, it was found that users still have a high awareness of unfamiliarity with digital humans. This means that there is still a feeling of unfamiliarity between humans and machines, and it is necessary to develop a digital human with an empathy for humans based on an understanding of essential human emotions.

This study considered it important to understand, observe, and analyze the five senses based on the method of understanding fundamental human emotions centered around empathy between the designers and users of digital humans. There is an uncanny valley between humans and digital humans, which is why humans and machines do not empathize with each other and feel rejection. Recognizing the need for empathy, we clarified its significance as follows:

First, in order to develop a more realistic and human-friendly digital human, a more natural and realistic expression has been created with the recent development of eye recognition, emotion recognition, voice recognition, and artificial intelligence technologies. Although digital humans have appeared, there are still unfamiliar elements involved in humans communicating and interacting with them. Therefore, we highlighted the importance of undertaking research into human emotions.

Second, for in-depth understanding of the experience theory of various digital human beings, the theoretical basis was presented through prior studies on immersion and social realism to provide the optimal experience and change the meaning of design from the point of view of the experience theory.

Third, by analyzing in-depth interviews of real users, the recognition of unfamiliarity with digital humans and their differences was sympathized with and classified. In the digital human, human experience should be the technology itself, not a human-like existence.

Fourth, by identifying the difference in unfamiliarity between humans and digital humans, we were able to understand the factors that cause specific unfamiliarity according to the five senses and the motives of actions, and derived the emotion-based user experience direction for empathy.

Although many studies on digital humans have been conducted in the technology field so far, this study is valuable in that studies on digital humans and their impact on human emotions in the design field are limited. The digital human market will gradually expand, and it is expected that digital humans will increase as a result of advanced technology. As for the development of a digital human that can become familiar with the virtual world and the real world without humans being repulsed, we look forward to further research in the design field as well as to technological advancement. According to the changing future and diversified technologies, if we establish a method to conveniently and naturally apply machine and human empathy to real life in terms of lifestyle content, more human-centered values will be created, and research will be made possible from the stage of actual digital human design application. In the future, we intend to conduct an in-depth investigation so that we can apply both specific human emotions and the five senses, compare the scope of application applied in this study and its direction, and proceed with in-depth and essential methods of utilizing the five senses and their development in future research.