Next Article in Journal
A New Computational Method for Estimating Simultaneous Equations Models Using Entropy as a Parameter Criteria
Next Article in Special Issue
Dynamics under Uncertainty: Modeling Simulation and Complexity
Previous Article in Journal
Challenges Regarding Scientific Transcription in Virtual Office Hours
Previous Article in Special Issue
Prediction of Important Factors for Bleeding in Liver Cirrhosis Disease Using Ensemble Data Mining Approach
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Synthetic Emotions for Empathic Building

by
Jose L. Salmeron
1,2,* and
Antonio Ruiz-Celma
3
1
Data Science Lab, Universidad Pablo de Olavide, Ctra. de Utrera km. 1, 41013 Sevilla, Spain
2
Universidad Autónoma de Chile, 5 Poniente, 1670 Talca, Chile
3
Universidad de Extremadura, Avda. de Elvas s/n, 06006 Badajoz, Spain
*
Author to whom correspondence should be addressed.
Mathematics 2021, 9(7), 701; https://doi.org/10.3390/math9070701
Submission received: 16 December 2020 / Revised: 7 March 2021 / Accepted: 12 March 2021 / Published: 24 March 2021
(This article belongs to the Special Issue Dynamics under Uncertainty: Modeling Simulation and Complexity)

Abstract

:
Empathic buildings are intelligent ones that aim to measure and execute the best user experience. A smoother and intuitive environment leads to a better mood. The system gathers data from sensors that measure things like air quality, occupancy, noise and analyse it for the better experience of the users. This research proposes an artificial intelligence-based approach to detect synthetic emotions based on Thayer’s emotional model and Fuzzy Cognitive Maps. This emotional model is based on a biopsychological approach to the analysis of the humans’ emotional state. In this research, Fuzzy Grey Cognitive Maps are used, which are an extension of the fuzzy cognitive maps using the grey systems theory to model uncertainty. Fuzzy Cognitive Grey Maps (FGCMs) have become a very valuable theory for modeling high-uncertainty systems when small and incomplete discrete data sets are available. This research includes experiments with a couple of synthetic case studies for testing this proposal. This proposal provides an innovative way for simulating synthetic emotions and designing an empathic building.

1. Introduction

Autonomous systems have been designed to interact with one or more targets in an environment primarily without human intervention [1,2]. Some systems are capable of operating in an environment with high-level objectives and others that do not require any human involvement [3,4]. The complexity of this type of system with a high degree of autonomy makes the result of their interaction with the environment uncertain and it is not possible to ensure the desired behavior [5]. For this reason, approaches such as Off-Line Reinforcement Learning arise, which train agents in a controlled environment.
Regardless of the technique used for the design of autonomous systems, usually highly specialized tasks may require the inclusion of affective behaviors to improve their performance [6]. The role of emotions in human reasoning, daily activities, and decision-making is really critical. In other words, emotions have a huge impact on human intelligence. If their emotions are not working properly, then human beings will not make decisions properly. Therefore, there is a strong interrelation between embedding emotions in systems and making systems that include intelligence. Artificial emotion is an emerging research subject and aims to make machines have artificial emotions [7].
According to [8], affective forecasting studies have shown that people are biased in making both random and systematic errors when anticipating their own future emotional states. Because of the divergence between experienced and anticipated reactions, it is worth examining artificial intelligence methods to avoid these problems.
Empathic building is an intelligent building that aims to measure and execute the best user experience. A smoother and intuitive environment lead to a better mood. The system gathers the relevant data from IoT sensors that measure things like air quality, occupancy, noise and analyze it for the better experience of the users.
The main contribution of this paper is to propose Fuzzy Cognitive Grey Maps (FGCMs) as an innovative technique to predict artificial emotions in systems with a certain degree of autonomy in complex environments with high uncertainty. In addition, the dynamic analysis mapping of the FGCM uses Thayer’s model of emotion within an emotional space. They define the categories of emotions in a matrix with four quadrants. This proposal translates said matrix to a two-dimensional Cartesian coordinate according to its valence and excitation.
The remainder of the paper is organized as follows. Section 2 presents the theoretical background. Section 3 shows the FGCMs fundamentals. Section 4 describe the methodological proposal. The next section details the experimental approach with two case studies and conclusions are finally shown.

2. Theoretical Background

Affective Computing seeks to bring computers and effective humans closer together. Affective computing tries to assign systems the human-like capabilities of emotions’ observation, interpretation and generation [9]. As the authors explained previously, emotions have a huge impact on human physical states, beliefs, motivations, activities, decisions, and even wishes. An appropriate balance of emotions makes human beings having flexibility and creativity in solving problem [10].
Affective computing focuses on the recognition and processing of human emotions. Emotion processing is useful for analyzing human reactions, eliciting behavioral intentions, and generating reasonable responses from systems. Over the last years emotions’ research has become a multi-disciplinary research field with a growing interest [11]. Moreover, emotions play a fundamental role in human-machine interaction. The simulation or automatic detection of emotional states aims to improve the interaction between humans and machines. Therefore, such simulation or detection would allow systems to perform alternative operating paths in accordance with current human emotions.
It could be worthy in a lot of real-life applications as a fear-type emotion recognition for audio-based surveillance systems [12], real-life emotion detection within a medical emergency call centre [13], semi-automatic diagnosis of psychiatric diseases [14] detection of children’s emotional states in a conversational computer games [15], and so on.
On the other hand, relevant advances were made in speech synthesis as well [16]. Biosignals (e.g.: electrocardiogram (ECG or EKG), electroencephalogram (EEG), electromyogram (EMG) and electrooculogram (EOG) and so on), face and body images are options to detect emotional states [17,18]. However, those kinds of methods are more invasive, and so complex for applying in a lot of real applications [11]. This research proposes a non-invasive soft computing-based method for simulating emotions in real-world applications.
So far, there are a lot of emotion-based theories, such as the OCC model [19] and Thayer’s emotion model. The OCC model comprises a classification of twenty-two emotion kinds within a hierarchy. The hierarchy includes three branches, namely emotions concerning the consequences of events, actions of agents, and aspects of objects. The emotions identified in the OCC model are the following: joy, hope, relief, pride and gratitude, like distress, fear, disappointment, remorse, anger and dislike [20].
Furthermore, some branches mix to form a set of composed emotions, specifically emotions concerning consequences of events. According to the OCC model, all emotions can be grouped in terms of the event that provokes each emotion. Scenarios that drive emotions can be folded into three kinds. The first scenario’s kind that drives emotions is the consequences of events. The second kind of scenario that generates emotions is the actions of the agents. The third one that provokes emotions is the appearance of objects.
Thayer’s emotional model is the affective framework that supports this research. Next, the fundamentals of that model are shown.

2.1. Thayer’s Emotion Model

Thayer’s model [21] is based on mood analysis as a biopsychological concept [22]. Thayer considers mood as an affective state highly related to psychophysiological and biochemical elements. Moreover, individual cognitive actions and casual events perform a critical role in its sudden understanding.
Thayer’s emotion model is frequently used to avoid the ambiguity of adjectives [23]. Thayer’s model organizes the major categories of emotions in a matrix according to their arousal (how calming versus exciting) and valence (how negative versus positive). The emotion categories can be separated into the four quadrants of the common two-dimensional cartesian coordinate system (Figure 1), valence (x), excitement (y). The origin models the lack of emotions.
Three emotions are located in each quadrant. The first quadrant (valence and positive arousal) is made up of emotions: excited, happy and excited. The second quadrant (negative valence and positive arousal) includes the emotions annoying, angry, and nervous. The third quadrant emotions (valence and negative arousal) are sadness, boredom, and sleepiness. Finally, the last quadrant (positive valence and negative arousal) contains the emotions calm, peace and relaxation. According to this, the emotional space is made up of twelve emotions.
The distance to the origin reflects the intensity of the emotion. Emotions closer to the source are less intense, while those further away from the source represent more intense emotions.

2.2. Emotional-Based AI Systems

Research has been done to develop emotional systems in various settings, such as emotions in music, art, and so on. The authors present some efforts below.
Marreiros et al. [20] designs a Ubiquitous Group Decision Support System (u-GDSS) that enables asynchronous and distributed computational services. One of the most interesting elements of this research is a multiagent-based simulator of emotional group decision-making. Zhou et al. [24] incorporates affective computing, emotion ontology within an emotion-aware service-oriented architecture. This framework allows us to publish emotion-sensitive services. Sharada & Ramanaiah [25] proposes an intelligent agent framework based on a neuro-fuzzy system to process the events. The emotion generation is based on a hopfield network.
Setiono et al. [26] proposes a game design with affective computing where the experience of the players is improved through the collection and understanding of the player’s emotions. Han et al. [27] proposes a human-centric lifelong learning framework where the added value is affective computing. The results of their research prove that the incorporation of affective computing greatly improves the conventional alternatives. Kratzwald et al. [28] proposes a personalized learning transfer approach that uses sentiment analysis to achieve significant performance improvements.
In addition, Fuzzy Cognitive Maps have also been used as an interface between emotions, mood and behavior of human beings. Salmeron [29] builds emotional robots that operate in near real-time and improve their sensitivity. Salmeron & Lopez [30], Salmeron [31] presents a FCM-based proposal for generate synthetic emotions. This is the starting point of this research. FCMs have several valuable elements for the generation of synthetic emotions, such as flexible and adaptive reasoning and a high abstraction level [32,33]. Furthermore, this technique has been widely used to model and analyze complex dynamic systems [34,35,36]. As cognition tool, an FCM is easy to use and it can model knowledge and reasoning in an efficient way.

3. Fuzzy Grey Cognitive Maps

3.1. Fundamentals

Grey Systems Theory (GST) is a so interesting set of solving problem tools within environments with high uncertainty, under discrete small and incomplete data sets [37]. GST was created to analyze small data samples with poor information quality. GST has found successful applications in energy, transportation, military science, business, meteorology, medicine, agriculture, industry, and others.
Fuzzy Grey Cognitive Map is based on FCMs and GST, and it has become a very worthy theory for solving problems within domains with high uncertainty [38]. FGCMs offer an intuitive way to model and reason about concepts without loss of precision. An advantage of FGCMs is that non-technical decision-makers can understand all the problems in a given scenario using decision models represented as causal graphs. Furthermore, an FGCM allows locating the most critical factor that impacts the expected target or output concept.
The FGCM nodes are representing relevant concepts for the problem. The influence between nodes concepts are modeled by directed edges. An edge linking two nodes represents the grey causal impact of the causal concept on the effect concept. As FCMs, the FGCM models are represented by a (grey) adjacency matrix ( A ± ).
A ± = c 1 c n c 1 c n          ϖ 11 ± ϖ 1 n ± ϖ n 1 ± ϖ n n ±
FGCMs can be considered as a special type of dynamic system that includes feedback, where the effect of the change in one node can impact other nodes, which successively can impact the concept that initiates the change. A FGCM models unstructured knowledge through causalities through grey concepts and grey relationships between them based on FCM [38,39,40,41,42,43].
Because FGCMs are hybrid methods that combine grey systems theory and neural networks, the state of each node (concept) is measured by its grey weight as follows
ϖ i j ± = [ ϖ ̲ i j , ϖ ¯ i j ] | ϖ ̲ i j ϖ ¯ i j [ 1 , + 1 ] [ 0 , + 1 ]
where i is the pre-synaptic (cause) node and j is the post-synaptic (effect) one. Note that if the FGCM is unipolar, then upper ϖ ̲ and lower ϖ ¯ weights belong to range [ 0 , + 1 ] . However, if the FGCM is bipolar, then upper and lower weights belong to range [ 1 , + 1 ] .
FGCM dynamics begins with an initial grey vector state c ± ( 0 ) , which models a proposed initial imprecise stimuli. The initial grey vector state with n nodes is denoted as
c ± ( 0 ) = c 1 ± ( 0 ) , c 2 ± ( 0 ) , , c n ± ( 0 ) = [ c ̲ 1 ( 0 ) , c ¯ 1 ( 0 ) ] , [ c ̲ 2 ( 0 ) , c ¯ 2 ( 0 ) ] , , [ c ̲ n ( 0 ) , c ¯ n ( 0 ) ]
The updated nodes states are computed in an iterative inference process with an activation function (usually sigmoid or hyperbolic tangent function) [38,44,45], which maps monotonically the grey node value into a normalized range [ 0 , + 1 ] or [ 1 , + 1 ] , depending of the selected function. Note that grey arithmetic is detailed as [38]. Each single node would be updated as follows
c j ± ( t + 1 ) f ± i = 1 n ϖ i j ± · c i ± ( t ) c ̲ j ( t + 1 ) , c ¯ j ( t + 1 )
If the nodes has memory of the previous state the updating equation is as follows
c j ± ( t + 1 ) f ± c i ± ( t ) i = 1 n ϖ i j ± · c i ± ( t )
where ⊕ is the summation of grey numbers.
The most used activation function in FGMCs is unipolar sigmoid function when the nodes’ value maps in the range [ 0 , 1 ] . If f ± ( · ) is a sigmoid, then the i component of the grey vector state at t + 1 iteration c ± ( t + 1 ) after the iterations would be as follows
c i ± ( t + 1 ) 1 + e λ · c ̲ i ( t ) 1 , 1 + e λ · c ¯ i ( t ) 1 .
Morever, the activation function f ± ( · ) would be the hyperbolic tangent when the nodes’ states map in the range [−1, +1]. It is computed as follows
c i ± ( t + 1 ) e 2 · λ · c ̲ i ( t ) 1 e 2 · λ · c ̲ i ( t ) + 1 , e 2 · λ · c ¯ i ( t ) 1 e 2 · λ · c ¯ i ( t ) + 1
The nodes’ states evolve along the FGCM dynamics and it could lead to three different scenarios.
  • If the stability is reached, the FGCM inference process stop. It achieves a steady pattern of nodes’ states, the so-called grey fixed-point attractor, or grey hidden pattern. This steady grey vector state shows the impact of the initial grey vector state on the final state of each FGCM grey node.
  • In addition, the grey vector state could keep cycling between some fixed states. This situation is known as the limit grey cycle.
  • A third possible state with a continuous activation function would be a grey chaotic attractor. It is when, instead of a steady-state, the FGCMs keep generating different grey vector states for each iteration.
FGCM includes greyness as an uncertainty measurement. Higher values of greyness mean that the results have a higher uncertainty degree. It is computed as follows
ϕ ( c i ± ) = ( c i ± ) ( ψ )
where ( c i ± ) = c ¯ i c ̲ i is the absolute value of the length of grey node c i ± state value, and ( ψ ) is the absolute value of the range in the information space, denoted by ψ . It is computed as follows
( ψ ) = 1 i f { c i ± , ϖ i ± } 0 , 1 { c i ± , ϖ i ± } 1 , + 1 2 i f { c i ± , ϖ i ± } 1 , + 1

3.2. FGCM Advantages over FCM

FGCMs have several advantages over conventional FCM [32,33,38,41,46,47]. A FGCM allows us to calculate the desired steady states managing the uncertainty and hesitation existing in the raw data (for instance, due to source noise) for the causal relationships between nodes, as well as within the states of the initial nodes.
Unlike FCMs, FGCM states have weights with grey numbers. In this way, FGCMs are able to model multi-meaning uncertainty in the relationships between concepts.
FGCM is an FCM generalization and it is considered closer to human decision-making than FCM is. It handles the inner hesitancy and uncertainty in complex systems by including greyness in edges and nodes. Indeed, the FGCMs’ reasoning process output includes a degree of greyness expressed in grey values representing the certainty of the results.
FGCMs are also able to model more types of relationships than an FCM can. For example, FGCMs can run successfully models with edges where the intensity is just partially known or even is not known at all (e.g., ϖ i j ± [ 1 , + 1 ] ).
It is important to consider that, even in the case the dynamics of an FCM would finish with the same vector state as one FGCM after the whitenization, FGCMs still handle better the grey uncertainty and inner fuzziness of human emotions.

4. Proposal

Methodology

Figure 2 shows the flowchart of our methodological proposal. The starting points are the input data. It includes three kinds of input data. Firstly, the environment is a set of variables representing the influence of the environment over the affective state. Moreover, the mood and the temperament are input data because each individual has its own mood and temperament with the differences between them detailed before.
The effective engine is composed of FGCM-based models for building synthetic emotions. The reactions have influence over the mood. Afterwards, the higher state is selected and the arousal and the valence are computed. The affective state is computed using arousal and valence. If the system keeps running, the process is executed again. Note that the environment data is changing over time and it has an impact on the affective state.

5. Experiments and Discussion

With the intention of testing the proposal, this research analyzes two case studies of an artificial experiment. The objective is the simulation of the emotions of an autonomous system produced by the environmental conditions in a hospital facility.
It should be noted that the objective of the model is not to design a real-world emotional system, but only to test the FGCM approach for simulating synthetic emotions of people in the queue in a theoretical empathic building. For that reason, the authors have designed an FGCM-based emotional model shown in Figure 3. The concepts in this model are detailed in Table 1. Nodes c 1 ± and c 2 ± model arousal and valence respectively. They are the output concepts because those nodes are used to identified the emotions.
In each case of this experiment, the authors have designed a different initial vector state. In ths test case, the initial grey vector state c 1 ± ( 0 ) models the initial grey state values of the events at a given time of the process. As a result of the FGCM dynamics the final grey vector c ± ( t ) models the achieved steady state. The steady grey vector c ± ( t ) is the steady vector in the convergence region. The steady state of nodes c 1 ± and c 2 ± , their greyness and the detected emotion are analysed.
Moreover, the authors analyze the FGCM dynamics in both cases with different settings. The setting is composed by the memory and slope. If the nodes do not have memory, then the updating equation is Equation (5). If the nodes have memory, then the updating equation is Equation (4). The activation function is the hyperbolic tangent because the emotional model needs negative values. The slope is the λ parameter of the Equation (7). According to literature [48], the slopes applied for both activation functions are 1, 3, and 5.

5.1. Experiment 1

For the first synthetic case study, the initial grey vector state is shown in Equation (10). Table 2 shows the results of this experiment with the different settings. Figure 4 show a graphical representation of the emotions achieved with each setting.
c 1 ± ( 0 ) = ( [ 0 , 0 ] , [ 0 , 0 ] , [ 0.2 , 0.2 ] , [ 0 , 0 ] , [ 0 , 0 ] , [ 0.2 , 0.3 ] , [ 0.2 , 0.1 ] , [ 0.1 , 0.3 ] , [ 0.3 , 0.4 ] )
The achieved emotion with hyperbolic tangent as activation function is strongly related with the selected setting, especially the memory of the updating function. If the function has no memory (Equation (4)), then the emotion is almost neutral. However, if the function has memory (Equation (5)), then the emotion goes from pleased to happy as the slope increases.
The lower values of greyness for nodes c 1 ± and c 2 ± are achieved without memory (Equation (4)), and 1.0 as slope. The higher value of greyness for node c 1 ± is achieved with memory (Equation (5)), and 3.0 as slope. The higher value of greyness for node c 2 ± is achieved with memory (Equation (5)), and 1.0 as slope.

5.2. Experiment 2

For the second synthetic case study, the initial grey vector state is shown in Equation (11). Table 3 shows the results of this experiment with the different settings. Figure 5 show a graphical representation of the emotions achieved with each setting.
c 2 ± ( 0 ) = ( [ [ 0.0 , 0.0 ] , [ 0.0 , 0.0 ] , [ 0.0 , 0.0 ] , [ 0.7 , 0.6 ] , [ 0.2 , 0.3 ] , [ 0.1 , 0.0 ] , [ 0.0 , 0.0 ] , [ 0.6 , 0.7 ] , [ 0.4 , 0.1 ] ] )
The values of arousal ( c 1 ± ) and valence ( c 2 ± ) with hyperbolic tangent as activation function allow to compute peaceful and neutral as the achieved emotions. The achieved emotion is strongly related to the selected setting, especially the memory of the activation function. If the updating function has no memory (Equation (4)), then the emotion is mostly neutral. However, if the activation function has memory (Equation (5)), then the emotion is peaceful increasing intensity when the slope increases.
The lower values of greyness for nodes c 1 ± and c 2 ± are achieved without memory (Equation (4)), and 1.0 as slope. The higher value of greyness for node c 1 ± is achieved with memory (Equation (5)), and 1.0 as slope. The higher value of greyness for node c 2 ± is achieved with memory (Equation (5)), and 1.0 as slope.

6. Conclusions

This paper shows an FGCM-based system for synthetic emotions. FGCM is a grey graph for modeling causal reasoning within complex problems with high uncertainty. This research proves that it is possible to generate or simulate emotions obtained from sensors’ raw data.
Note that this research is not an empirical one. An FGCM-based proposal based on sensors’ raw data, concepts and output nodes is shown. Indeed, the aim is not to model a real-world system, but this research proposes an FGCM-based theoretical proposal so that ongoing research of real-world practitioners can apply to generate or simulate synthetic emotions within their own applications or systems.
The experiments’ results prove that the outlet of this proposal is strongly related to the setting applied. According to the results, FGCMs with memory nodes are the best option for emotion modeling, and the lower slopes target emotions with less intensity. As a limitation, FGCMs are strongly related with their own setting and validation is not straightforward.

Author Contributions

Conceptualization, J.L.S.; data curation, J.L.S.; formal analysis, J.L.S.; funding acquisition, A.R.-C.; investigation, J.L.S.; methodology, J.L.S.; project administration, A.R.-C.; validation, J.L.S. and A.R.-C.; writing—original draft, J.L.S.; writing—review and editing, J.L.S. All authors have read and agreed to the published version of the manuscript.

Funding

Ruiz-Celma was funded by the Government of Extremadura and the European Regional Development Fund, (Una manera de hacer Europa), through GR18137 and IB18008 support.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Acknowledgments

Salmeron would like to thank Tessella (Altran group, part of Capgemini) for their kind support.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Brun, Y.; Serugendo, G.D.M.; Gacek, C.; Giese, H.; Kienle, H.; Litoiu, M.; Muller, H.; Pezze, M.; Shaw, M. Engineering Self-Adaptive Systems through Feedback Loops. In Software Engineering for Self-Adaptive Systems. Lecture Notes in Computer Science; Cheng, B.H.C., de Lemos, R., Giese, H., Inverardi, P., Magee, J., Eds.; Springer: Heidelberg/Berlin, Germany, 2009; pp. 48–70. [Google Scholar]
  2. Precup, R.-E.; Preitl, S.; Petriu, E.; Bojan-Dragos, C.-A.; Szedlak-Stinean, A.-I.; Roman, R.-C.; Hedrea, E.-L. Model-Based Fuzzy Control Results for Networked Control Systems. Rep. Mech. Eng. 2020, 1, 10–25. [Google Scholar] [CrossRef]
  3. Ghosh, I.; Datta Chaudhuri, T. FEB-Stacking and FEB-DNN Models for Stock Trend Prediction: A Performance Analysis for Pre and Post Covid-19 Periods. Decis. Mak. Appl. Manag. Eng. 2021, 4, 51–84. [Google Scholar] [CrossRef]
  4. Tziallas, G.; Theodoulidis, B. A controller synthesis algorithm for building self-adaptive software. Inf. Softw. Technol. 2004, 46, 719–727. [Google Scholar] [CrossRef]
  5. Khakpour, N.; Jalili, S.; Talcott, C.; Sirjani, M.; Mousavi, M.R. PobSAM: Policy-based Managing of Actors in Self-Adaptive Systems. Electron. Notes Theor. Comput. Sci. 2010, 263, 129–143. [Google Scholar] [CrossRef]
  6. Lee-Johnson, C.P.; Carnegie, D.A. Mobile Robot Navigation Modulated by Artificial Emotions. IEEE Trans. Syst. Man Cybern. Part B Cybern. 2010, 40, 469–480. [Google Scholar] [CrossRef] [PubMed]
  7. Shurong, N.; Jie, P.; Guangmei, X.; Guangping, Z.; Xuyan, T. Study on Artificial Emotion Model. In Proceedings of the ICNN&B ’05. International Conference on Neural Networks and Brain, Beijing, China, 13–15 October 2005; Volume 3, pp. 1420–1424. [Google Scholar]
  8. Hoerger, M.; Quirk, S.W. Affective forecasting and the Big Five. Personal. Individ. Differ. 2010, 49, 972–976. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  9. Tao, J.; Tan, T.; Picard, R.W. Affective Computing and Intelligent Interaction; Springer: Berlin, Germany, 2005. [Google Scholar]
  10. Guojiang, W.; Xiaoxiao, W.; Kechang, F. Behaviour decision model of intelligent agent based on artificial emotion. In Proceedings of the 2nd International Conference on Advanced Computer Control (ICACC), Shenyang, China, 29–31 January 2010; pp. 185–189. [Google Scholar]
  11. Albornoz, E.M.; Milone, D.H.; Rufiner, H.L. Spoken emotion recognition using hierarchical classifiers. Comput. Speech Lang. 2011, 25, 556–570. [Google Scholar] [CrossRef]
  12. Clavel, C.; Vasilescu, I.; Devillers, L.; Richard, G.; Ehrette, T. Fear-type emotion recognition for future audio-based surveillance systems. Speech Commun. 2008, 50, 487–503. [Google Scholar] [CrossRef] [Green Version]
  13. Devillers, L.; Vidrascu, L. Speaker Classification II: Selected Projects. In Lecture Notes in Computer Science; Chapter: Real-Life Emotion Recognition in Speech; Springer: Berlin/Heidelberg, Germany, 2007; Volume 4441/2007, pp. 34–42. [Google Scholar]
  14. Tacconi, D.; Mayora, O.; Lukowicz, P.; Arnrich, B.; Setz, C.; Trster, G.; Haring, C. Activity and emotion recognition to support early diagnosis of psychiatric diseases. In Proceedings of the 2nd International Conference on Pervasive Computing Technologies for Healthcare’08 Tampere, Tampere, Finland, 30 January–1 February 2008; pp. 100–102. [Google Scholar]
  15. Yildirim, S.; Narayanan, S.; Potamianos, A. Detecting emotional state of a child in a conversational computer game. Comput. Speech Lang. 2011, 25, 29–44. [Google Scholar] [CrossRef]
  16. Murray, I.R.; Arnott, J.L. Applying an analysis of acted vocal emotions to improve the simulation of synthetic speech. Comput. Speech Lang. 2008, 22, 107–129. [Google Scholar] [CrossRef]
  17. Schindler, K.; Van Gool, L.; de Gelder, B. Recognizing emotions expressed by body pose: A biologically inspired neural model. Neural Netw. 2008, 21, 1238–1246. [Google Scholar] [CrossRef]
  18. Vinhas, V.; Reis, L.P.; Oliveira, E. Dynamic multimedia content delivery based on real-time user emotions. Multichannel online biosignals towards adaptative GUI and content delivery. In Proceedings of the International Conference on Bio-inspired Systems and Signal Processing & Biosignals, Porto, Portugal, 14–17 January 2009; pp. 299–304. [Google Scholar]
  19. Ortony, A.; Clore, G.; Collins, A. The Cognitive Structure of Emotions; Cambridge University Press: Cambridge, UK, 1988. [Google Scholar]
  20. Marreiros, G.; Santos, R.; Ramos, C.; Neves, J.; Novais, P.; Machado, J.; Bulas-Cruz, J. Ambient Intelligence in Emotion Based Ubiquitous Decision Making. In Proceedings of the Artificial Intelligence Techniques for Ambient Intelligence, IJCAI’07—Twentieth International Joint Conference on Artificial Intelligence, Hyderabad, India, 6–12 January 2007. [Google Scholar]
  21. Thayer, R.E. The Biopsychology of Mood and Arousal; Oxford University Press: New York, NY, USA, 1989. [Google Scholar]
  22. Acampora, G.; Loia, V.; Vitiello, A. Distributing Emotional Services in Ambient Intelligence through Cognitive Agents. Serv. Oriented Comput. Appl. 2011, 5, 17–35. [Google Scholar] [CrossRef]
  23. Yang, Y.-H.; Lin, Y.-C.; Su, Y.-F.; Chen, H.H. Music emotion classification: A regression approach. In Proceedings of the IEEE International Conference Multimedia & Expo, Beijing, China, 2–5 July 2007; pp. 208–211. [Google Scholar]
  24. Zhou, J.; Yu, C.; Riekki, J.; Karkkainen, E. AmE framework: A model for emotion-aware ambient intelligence. In Proceedings of the The Second International Conference on Affective Computing and Intelligent Interaction (ACII2007), Lisbon, Portugal, 12–14 September 2007. [Google Scholar]
  25. Sharada, G.; Ramanaiah, O.B.V. An Artificial Intelligence Based Neuro-Fuzzy System Emotional Intelligence. Int. J. Comput. Appl. 2010, 1, 74–79. [Google Scholar] [CrossRef]
  26. Setiono, D.; Saputra, D.; Putra, K.; Moniaga, J.V.; Chowandaa, A. Enhancing Player Experience in Game With Affective Computing. Procedia Comput. Sci. 2021, 179, 781–788. [Google Scholar] [CrossRef]
  27. Han, J.; Zhang, Z.; Pantic, M.; Schuller, B. Internet of emotional people: Towards continual affective computing cross cultures via audiovisual signals. Future Gener. Comput. Syst. 2021, 114, 294–306. [Google Scholar] [CrossRef]
  28. Kratzwald, B.; Ilić, S.; Kraus, M.; Feuerriegel, S.; Prendinger, H. Deep learning for affective computing: Text-based emotion recognition in decision support. Decis. Support Syst. 2018, 115, 24–35. [Google Scholar] [CrossRef] [Green Version]
  29. Salmeron, J.L. Augmented fuzzy cognitive maps for modelling LMS critical success factors. Knowl. Based Syst. 2009, 22, 275–278. [Google Scholar] [CrossRef]
  30. Salmeron, J.L.; Lopez, C. Forecasting Risk Impact on ERP Maintenance with Augmented Fuzzy Cognitive Maps. IEEE Trans. Softw. Eng. 2011, 38, 439–452. [Google Scholar] [CrossRef]
  31. Salmeron, J.L. Fuzzy cognitive maps for artificial emotions forecasting. Appl. Soft Comput. 2012, 12, 3704–3710. [Google Scholar] [CrossRef]
  32. Salmeron, J.L.; Rahimi, S.A.; Navalie, A.M.; Sadeghpour, A. Medical Diagnosis of Rheumatoid Arthritis using Data driven PSO-FCM. Neurocomputing 2017, 232, 104–112. [Google Scholar] [CrossRef]
  33. Salmeron, J.L.; Ruiz-Celma, A.; Mena, A. Learning FCMs with multi-local and balanced memetic algorithms for forecasting drying processes. Neurocomputing 2017, 232, 52–57. [Google Scholar] [CrossRef]
  34. Salmeron, J.L.; Vidal, R.; Mena, A. Ranking Fuzzy Cognitive Maps based scenarios with TOPSIS. Expert Syst. Appl. 2012, 39, 2443–2450. [Google Scholar] [CrossRef]
  35. Vanhoenshoven, F.; Napoles, G.; Froelich, W.; Salmeron, J.L.; Vanhoof, K. Pseudoinverse Learning of Fuzzy Cognitive Maps for Multivariate Time Series Forecasting. Appl. Soft Comput. 2020, 95, 106461. [Google Scholar] [CrossRef]
  36. Salmeron, J.L.; Papageorgiou, E.I. A Fuzzy Grey Cognitive Maps-based Decision Support System for Radiotherapy Treatment Planning. Knowl. Based Syst. 2012, 30, 151–160. [Google Scholar] [CrossRef]
  37. Deng, J.L. Introduction to grey system theory. J. Grey Syst. 1989, 1, 1–24. [Google Scholar]
  38. Salmeron, J.L. Modelling grey uncertainty with Fuzzy Grey Cognitive Maps. Expert Syst. Appl. 2010, 37, 7581–7588. [Google Scholar] [CrossRef]
  39. Froelich, W.; Salmeron, J.L. Evolutionary Learning of Fuzzy Grey Cognitive Maps for the Forecasting of Multivariate, Interval-Valued Time Series. Int. J. Approx. Reason. 2014, 55, 1319–1335. [Google Scholar] [CrossRef]
  40. Rodriguez-Repiso, L.; Setchi, R.; Salmeron, J.L. Modelling IT Projects success with Fuzzy Cognitive Maps. Expert Syst. Appl. 2007, 32, 543–559. [Google Scholar] [CrossRef]
  41. Salmeron, J.L. An Autonomous FGCM-based System for Surveillance Assets Coordination. J. Grey Syst. 2016, 28, 27–35. [Google Scholar]
  42. Salmeron, J.L.; Gutierrez, E. Fuzzy Grey Cognitive Maps in Reliability Engineering. Appl. Soft Comput. 2012, 12, 3818–3824. [Google Scholar] [CrossRef]
  43. Xirogiannis, G.; Glykas, M. Fuzzy cognitive maps in business analysis and performance-driven change. IEEE Trans. Eng. Manag. 2004, 51, 334–351. [Google Scholar] [CrossRef]
  44. Papageorgiou, E.I.; Salmeron, J.L. Learning Fuzzy Grey Cognitive Maps using Nonlinear Hebbian-based approach. Int. J. Approx. Reason. 2012, 53, 54–65. [Google Scholar] [CrossRef] [Green Version]
  45. Salmeron, J.L.; Papageorgiou, E.I. Fuzzy Grey Cognitive Maps and Nonlinear Hebbian Learning in process control. Appl. Intell. 2014, 41, 223–234. [Google Scholar] [CrossRef]
  46. Napoles, G.; Salmeron, J.L.; Vanhoof, K. Construction and Supervised Learning of Long-Term Grey Cognitive Networks. IEEE Trans. Cybern. 2021, 51, 686–695. [Google Scholar] [CrossRef] [PubMed]
  47. Salmeron, J.L.; Palos, P. Uncertainty propagation in Fuzzy Grey Cognitive Maps with Hebbian-like learning algorithms. IEEE Trans. Cybern. 2019, 49, 211–220. [Google Scholar] [CrossRef] [PubMed]
  48. Bueno, S.; Salmeron, J.L. Benchmarking main activation functions in fuzzy cognitive maps. Expert Syst. Appl. 2009, 36 Pt 1, 5221–5229. [Google Scholar] [CrossRef]
Figure 1. Emotional model.
Figure 1. Emotional model.
Mathematics 09 00701 g001
Figure 2. Architecture of the proposal.
Figure 2. Architecture of the proposal.
Mathematics 09 00701 g002
Figure 3. Fuzzy Cognitive Grey Map (FGCM)-based experimental model.
Figure 3. Fuzzy Cognitive Grey Map (FGCM)-based experimental model.
Mathematics 09 00701 g003
Figure 4. Experiment 1.
Figure 4. Experiment 1.
Mathematics 09 00701 g004
Figure 5. Experiment 2—Hyperbolic tangent.
Figure 5. Experiment 2—Hyperbolic tangent.
Mathematics 09 00701 g005
Table 1. FGCM nodes and description.
Table 1. FGCM nodes and description.
Node ( c i ± )LabelDescription
c 1 ± ArousalState of being awake or reactive to stimuli
c 2 ± ValenceThe intrinsic attractiveness (positive valence) or aversiveness (negative valence) of an emotion
c 3 ± Reward/PunishmentReward is related with the queue’s purpose
c 4 ± StressA person’s response to a stressor such as noise or uncomfortable temperature
c 5 ± Waiting expectationsWaiting time
c 6 ± NoiseEnvironmental noise
c 7 ± UncomfortableTemperature higher or lower than comfortable
c 8 ± Scarce service timeWaiting time for each person
c 9 ± Few queue lengthPeople in the queue
Table 2. Results of experiment 1.
Table 2. Results of experiment 1.
Steady State Greyness
m f ± ( · ) Slopec1c2Emotionc1c2
Ftanh1[0.0, 1 × 10 6 ][ 1 × 10 6 , 0.0]neutral 6.3 × 10 7 3.78 × 10 7
Ftanh3[0.0, 1 × 10 5 ][ 6 × 10 6 , 0.0]neutral 5.03 × 10 6 3.02 × 10 6
Ftanh5[0.0, 0.1360][−0.0819, −0.0]ligth nervous 6.8 × 10 2 4.10 × 10 2
Ttanh1[0.0380, 0.1766][0.1308, 0.4200]ligth pleased 6.93 × 10 2 1.45 × 10 1
Ttanh3[0.3275, 0.9073][0.8188, 0.9958]med-strong happy 2.90 × 10 1 8.85 × 10 2
Ttanh5[0.7331, 0.9990][0.9953, 0.9999]strongly happy 1.33 × 10 1 2.36 × 10 3
Note that m means memory, F false and T true. Higher values of greyness are bolded.
Table 3. Results of experiment 2.
Table 3. Results of experiment 2.
Steady State Greyness
m f ± ( · ) Slopec1c2Emotionc1c2
Ftanh1.0[ 1 × 10 6 , 0.0][0.0, 1 × 10 6 ]neutral 6.61 × 10 7 3.96 × 10 7
Ftanh3.0[ 1 × 10 5 , 0.0][0.0, 6 × 10 6 ]neutral 5.21 × 10 6 3.13 × 10 6
Ftanh5.0[−0.1360, 0.0][0.0, 0.0820]almost neutral 6.80 × 10 2 4.10 × 10 2
Ttanh1.0[−0.4135, −0.1936][0.1794, 0.6163]medium peaceful 1.10 × 10 1 2.18 × 10 1
Ttanh3.0[−0.9966, −0.9274][0.8990, 0.9999]strongly peaceful 3.46 × 10 2 5.04 × 10 2
Ttanh5.0[−1.0, −0.9993][0.9978, 1.0]strongly peaceful 3.26 × 10 4 1.09 × 10 3
Note that m means memory, F false and T true. Higher values of greyness are bolded.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Salmeron, J.L.; Ruiz-Celma, A. Synthetic Emotions for Empathic Building. Mathematics 2021, 9, 701. https://doi.org/10.3390/math9070701

AMA Style

Salmeron JL, Ruiz-Celma A. Synthetic Emotions for Empathic Building. Mathematics. 2021; 9(7):701. https://doi.org/10.3390/math9070701

Chicago/Turabian Style

Salmeron, Jose L., and Antonio Ruiz-Celma. 2021. "Synthetic Emotions for Empathic Building" Mathematics 9, no. 7: 701. https://doi.org/10.3390/math9070701

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop