Emotional Response to Vibrothermal Stimuli

Abstract

Emotional response to haptic stimuli is a widely researched topic, but the combination of vibrotactile and thermal stimuli requires more attention. The purpose of this study is to investigate emotional response to vibrothermal stimulation by combining spatiotemporal vibrotactile stimulus with dynamic thermal stimulus (hot or cold). The vibrotactile and thermal stimuli were produced using the Haptic Chair and the Embr wave thermal bracelet, respectively. The results show that spatiotemporal vibrotactile patterns and their duration, and dynamic thermal stimulation, have an independent effect on the emotional response. Increasing duration generally increases the valence and arousal of emotional response. Shifting the dynamic temperature from cold to hot generally decreases the valence of emotional response but has no significant effect on arousal. Nevertheless, certain spatiotemporal patterns do exhibit unique responses to changes in dynamic temperature, although no interaction effects were found. The results show the potential of designing affective haptic interfaces using multimodal vibrothermal feedback.

1. Introduction

Haptics is an essential component of interaction technologies to enable users to reach a measure of emotional qualia [1] and has been shown to have a measurable impact on the ‘likability’ of haptics-enabled multimedia interactions [2]. A systematic study of emotions is hence crucial in improving our understanding of sensory perception, which could allow the design of more interactive, immersive, and empathetic interfaces. As emotions are abstract in nature, quantitative methods of evaluating emotions are useful in our endeavor to understand affect. The circumplex model of affect [3] looks at emotions as measurable interactions between two neurophysiological systems—valence (V, positive or negative disposition) and arousal (A, high or low mental excitement)—which could be mapped along two dimensional axes. Basic and complex emotions (e.g., happy, sad, calm) can be mapped along the contour of the resulting two-dimensional circumplex [4]. In most human–computer interaction (HCI) research, the valence–arousal (VA) method is prevalent. Notwithstanding the limitations of the VA methodology [5], using a linear combination of valence and arousal to create affective maps simplifies analysis and avoids conflicts due to using distinct labels [6]. Researchers therefore often use the circumplex model to analyze emotional responses to tactile and multimodal stimuli [5,6,7]. Nevertheless, the interpretation of a circumplex map might differ from other models, such as the vector map, though they both use the VA methodology. One reason for the difference in interpretation is that the degree of combination is dependent on the type of stimulation, e.g., Wilson et al. demonstrated that a vector map, rather than a circumplex map, is a better fit for creating affective maps of thermal stimulation [5].

It has been demonstrated that affect can be communicated through the use of haptics [8,9]. While thermal feedback is still scarce in mainstream commercial applications, there has been extensive research focused on developing novel haptic devices to engage the sense of touch in unique ways [10]. The semantic usage of adjectives to verbally describe feelings based on temperature have psychophysical undertones, as has been observed by Williams and Bargh [11]. According to their study, experiencing physical warmth subliminally caused participants to be more pro-social. This points toward the emotional underpinnings of tactile perception and its cognitive and behavioral outcomes. Salminen et al. studied emotional responses to thermal stimuli to map how temperature affects the valence, arousal, and dominance dimensions of emotions [7,12]. Wilson et al. studied the emotional mapping of tactile and multimodal stimulation [5,6,13], and proposed the use of multimodal channels to increase the affective bandwidth of communication [6]. Thermal feedback can enhance tactile experiences by enabling the inference of material properties [14], and augment mobile communication [15] and non-visual modes of interaction [13,16]. Wilson et al. created structured thermal stimuli called ‘thermal icons’ constructed using two parameters: thermal rate of change and intensity [16]. They also explored adding thermal feedback to vibrotactile displays to evaluate the identification accuracy of multi-modal thermal icons. Lee and Lim [17] investigated what information could be conveyed via thermal feedback during social interactions. An important finding was that temperature can be used to communicate specific feelings and emotions—even actions—within the context of environmental factors and interpersonal relationships. An example of this is the work by Tewell et al., who used thermal feedback through a Thermal Array Display to augment the emotional content of text messages [18]. While it was found that temperature variations did not have a significant impact on valence during text messaging, warmer temperatures increased the arousal. The context conveyed by the text message itself had a measurable impact on the valence, thereby demonstrating the importance of context when using thermal feedback.

The human perception of and emotional response to thermal stimuli facilitate its use in biofeedback devices for therapeutic applications [19]. For example, Cool Me Down [20] is a therapeutic device enabling patients to self-administer soothing sensory grounding treatment, and the Embr wave bracelet is a wrist-worn wearable that produces a change in temperature [21] for thermal comfort and therapeutic benefits during hot flashes [22]. On the other hand, vibrotactile stimulation has found a wide range of applications in sensory substitution. McDaniel et al. proposed a Haptic Face Display to improve the accessibility of interactions to individuals who are visually impaired by communicating facial expressions though the tactile channel [23]. Hribar and Pawluk [24] used a thermal module to enhance the access to visual artwork to people who are blind. Individuals could explore the texture and brushstrokes using a tactile pin array, and the color temperature (the warm–cold spectrum) using a thermal display. While mappings from color to temperature have been explored [25], thermal-based emotive augmentation has seen little investigation. Nevertheless, thermal feedback has been shown to enhance multimodal interaction for various applications.

Akiyama et al. explored the emotional augmentation of music using thermal feedback [26] and found that thermal stimulation can affect the emotional and cognitive perception of music. They report that while listening to music, thermal feedback can alter the listener’s impression, elicit feelings of comfort, and enhance concentration in real time. Kim et al. [27] demonstrated a more interactive application of cross-modal interfaces by augmenting voice-based intelligent agents with thermal feedback. While the interpretation of thermal feedback is highly contextual, they found a positive impact on immersion and engagement. Kotsec et al. [28] evaluated existing game mechanics to investigate the thermal augmentation of active gaming elements on the quality of immersion. They found that engagement can be improved by including active, rather than passive, feedback elements that nudge users’ responses during gameplay. EmoJacket [29] followed a user-centric approach to the design of haptic stimuli by first conducting an online survey to decide the parameters of vibrotactile and thermal stimulations to better elicit emotions of love, joy, surprise, anger, sadness, and fear. The users found the stimuli to be immersive, exciting, and emotionally engaging. This study highlighted the need to further investigate mapping emotions to haptic feedback using bottom–up approaches to enhance the quality of interactive experience.

Rich haptic patterns have been shown to be useful for enhancing interactivity by representing or mediating emotions [30]. Yoo et al. studied the emotional responses of lower-order vibrotactile dimensions (intensity, frequency, duration, and envelope) of tactile icons, and mapped them into the circumplex model [31]. They also studied thermal and vibrotactile multimodal patterns and found that each modality had an independent effect on emotional responses [32]. This is an interesting result for intra-modal affect considering the observation that using multiple modalities can expand the affective bandwidth [6]. Nevertheless, the effect of thermal feedback on higher-order spatiotemporal vibrotactile patterns is yet to be studied. McDaniel et al. conducted a preliminary study to assess subjects’ emotional responses to spatiotemporal vibrotactile stimuli [23]. It was found that certain shapes and durations of patterns had a significant influence on the participants’ emotional responses. It is known that vibrotactile rhythm has a significant impact on affect [33], and hence further exploration of rhythmic spatiotemporal vibrations is warranted. There is a need to explore the affective space of tactile feedback in a holistic way, combining both spatiotemporal vibrotactile and thermal modalities. We already know the affective impact of thermal stimulation [7]; hence, we hypothesize that there is value in exploring the emotional response to higher-order vibrothermal (vibrotactile and thermal) stimulation. We used the Haptic Chair, introduced in [34], together with an Embr Wave bracelet, to generate vibrothermal patterns.

2. Materials and Methods

2.1. Aim

The purpose of this study is to explore emotional responses to multimodal stimuli, combining thermal (warming or cooling) and vibrotactile (spatiotemporal) stimulation. The hypothesis explored here is: combining vibrotactile and thermal stimulation has a significant effect on emotional response. This research will aid the design of future human–computer interfaces that engage not only our visual and auditory senses but also our sense of touch to create immersive, engaging, and rich interactions.

2.2. Hardware and Software Design

Two technologies, depicted in Figure 1, were used in this study: the Embr wave bracelet and the Haptic Chair. Embr wave is a wrist-worn device that warms or cools to provide thermal stimulation in the range of +/−4 degrees Celsius from average skin temperature in a safe and comfortable form factor. The Haptic Chair [34] is a custom-built haptic device consisting of an ergonomic mesh chair, the back of which is embedded with a two-dimensional array of small vibrating ERM motors that convey subtle low-amplitude vibrations. A simple graphical user interface (GUI) for this study was implemented using python, as shown in Figure 2.

2.3. Participants

Thirty-two individuals (17 males and 15 females) were recruited for this IRB (Institutional Review Board)-approved study. The average age of participants was 24.6 years. Each participant completed a consent form and subject information form prior to beginning the study. Participants were encouraged to be comfortably seated on the Haptic Chair. If a subject was wearing a jacket or other thick outer garment, he or she was encouraged to remove the material, if possible, to allow better contact with actuators. All participants that were requested to remove their jacket followed the experimenter’s guidance.

2.4. Procedure

Each participant was presented with 48 different spatiotemporal vibrotactile patterns (12 vibration patterns, each at 2 different durations), and a cooling or warming thermal sensation (12 × 2 × 2 = 48). Immediately following each pattern presentation, participants rated their emotional response in terms of valence and arousal using a 7-point Likert scale to provide consistent psychometric measurements. Moreover, following each pattern presentation, participants rated the intensity (warmth or coolness) of the thermal stimulation using a 9-point Likert scale. Recording the sensation of temperature allowed for the verification of whether participants perceived the thermal changes correctly and allowed for the assessment of any device malfunctions. Participants were asked to close their eyes and listen to white noise via headphones during each pattern presentation to avoid external visuals and actuator noise that could have influenced emotional response. Stimuli were presented as follows: first, participants were requested to place the ventral side of their bare, non-dominant wrist on the Embr wave bracelet, attached to a flat surface, for a period of 7 s, for the device and skin temperatures to acclimate. This was followed by activating the white noise and initiating thermal stimulation. After waiting 2 s for thermal adaptation to reach the peak temperature, the spatiotemporal vibrotactile pattern was initiated with either the short or long duration. On completion of the pattern, the thermal stimulation and white noise was stopped. No pattern was repeated for a subject during the study. The sequential repetition of warm signals may cause issues, preventing the device from cooling; hence, it was necessary to randomize the thermal signals accordingly. The experimental setup is depicted in Figure 3, the vibrotactile patterns used are shown in Figure 4 and the experimental procedure is explained in Figure 5.

The vibrotactile patterns in Figure 4, are based on previous work that used symbolic spatiotemporal cues to evoke the six basic emotions: ‘happy’, ‘sad’, ‘surprise’, ‘anger’, ‘disgust’, and ‘fear’. Such patterns are considered higher order; they leverage abstraction based on the emotions they elicit from the participants without any training or paired stimulus [23], rather than only varying basic parameters such as amplitude or frequency. It was also found that the duration of presentation had a significant impact on the emotional response of participants [23]; hence, it was used as a parameter in this experiment. Two durations were chosen: 100 ms (short) and 1000 ms (long). The back was chosen as an appropriate body site to present the patterns to enable uniform presentation of the twelve spatiotemporal patterns.

In general, thermal feedback has the following parameters: direction of change (hot or cold), rate of change (how fast the temperature changes), and extent of change (the magnitude/intensity of thermal change). In the current experiment, the effect of change in the direction of thermal feedback (in combination with vibrotactile stimulation) was studied. Previous researchers have used either the dominant [5] or non-dominant arm [18,35] for perceptual studies, but to ease the recording of subject responses, this study used the non-dominant arm. The bracelet provides thermal stimulation using the dynamic method in which the target temperature is reached during presentation, i.e., the temperature is not pre-adjusted. Previous research has identified no significant impact on emotional response (valence and approachability) [12]; hence, this technique was chosen for faster experimentation. It is presumed that the neutral temperature of the device is equivalent to the skin temperature given enough time for adaptation; thus, a 30 s time window was chosen following the presentation of each pattern [5]. To successfully conduct this experiment, thermal stimuli within a range in which the skin is consistently capable of adapting per stimulus, and the experience is comfortable (range: 20–40 °C) [18], was required. Based on pilot trials, it is presumed that the average neutral temperature of the skin is 32 °C, and therefore, stimuli should be presented at +/−4 °C for warming/cooling sensations to be sufficiently distinct. Embr presents the stimulation as a wave peaking at the target +/−4 °C to enable user comfort and was deemed suitable for this experiment. The range of 28–36 °C, and rate of change pre-set at 1 °C, were found to be adequate for this experiment, which is line with previous research [7,18].

3. Results

We studied the effect of three factors: spatiotemporal vibrotactile patterns (12 patterns), duration (long and short), and dynamic temperature (hot and cold) on the emotional response. A three-way repeated measures ANOVA was conducted with the aforementioned factors on the valence (V) and arousal (A) scores obtained from subjects. The thermal sensation data were collected to check for inconsistencies in thermal feedback device performance. Data collected for one male subject were excluded from the analysis, as the thermal sensations were deemed ineffective as the scores indicated that changes in temperatures were not perceived adequately. A total of 1488 trials were analyzed for a set of 48 stimuli. The data were not normalized so that individual emotional responses were represented more accurately in the analysis.

To understand the general central tendencies, the mean VA scores of emotional responses averaged across each independent factor were plotted, as shown in Figure 6. The following are the observations: (1) on average, most patterns were associated with the upper right quadrant (excited/happy) except “shoulder tap” (pattern 12); (2) on average, across all patterns and thermal conditions, longer durations (1000 ms) increased both V and A compared to shorter durations (100 ms); (3) on average, a warm dynamic temperature decreased V and had no significant effect on A compared to a cool dynamic temperature.

Table 1. Results of the three-way repeated measures ANOVA on emotional response to vibrothermal stimuli.

Factor	Dependent Variable	Statistics	Partial Eta Squared
Pattern	V	F = 2.675; p = 0.003	0.082
	A	F = 6.736; p = 0	0.183
Duration	V	F = 9.173; p = 0.005	0.234
	A	F = 17.031; p = 0	0.362
Temperature	V	F = 4.564; p = 0.041	0.132
	A	F= 0.006; p = 0.938	0
Pattern × Duration	V	F = 1.462; p = 0.144	0.046
	A	F = 1.71; p = 0.07	0.054
Pattern × Temperature	V	F = 1.129; p = 0.338	0.036
	A	F= 0.604; p = 0.825	0.02
Duration × Temperature	V	F = 1.385; p = 0.249	0.044
	A	F= 1.078; p = 0.307	0.035
Pattern × Duration × Temperature	V	F = 1.222; p = 0.271	0.039
	A	F = 0.947; p = 0.495	0.031

displays a summary of the findings from the three-way repeated measures ANOVA. No significant interaction effects were found, meaning that each factor may have a more independent effect on the emotional response. The type of spatiotemporal pattern, duration or temperature had a significant effect on V, whereas only pattern and duration had a significant effect on A.

To understand the effect of pattern and duration on emotional response, Figure 7 and Figure 8 plot stimuli at hot and cold dynamic temperatures, respectively. In Figure 7, the extent of the effect of duration is unique for each pattern; nevertheless, increasing duration leads to an increase in both V and A. Pattern 1 (alternate—top to bottom) was highly affected by the change in duration (V, A) = (−0.06, 0.1) → (0.87, 1). At a shorter duration, this pattern was more neutral, whereas at a longer duration, this pattern created a more excitable and pleasant experience. Pattern 10 (snake horizontal) was least effected by a change in duration (V, A) = (0.45, 0.9) → (0.71, 0.97). A notable exception is pattern 12 (shoulder tap), which transformed from (V, A) = (0.06, −0.32) → (−0.03, 0.19), that is, A increased (keeping in trend with the rest of the data points), but V slightly reduced. Another observation is the clustering effect of increasing duration. Patterns subjected to longer duration tend to cluster together, which may have significant design implications. The effect of duration on patterns at a cold dynamic temperature seem to be similar. As seen in Figure 8, increasing duration increases V and A of the patterns. Interestingly, even pattern 12 was consistent with the rest of the datapoints (V, A) = (−0.03, −0.39) → (0.52, 0.29). Pattern 1 is still highly affected by duration, but there is significant separation between the short and longer duration clusters.

Keeping in line with the central tendencies, the bulk of the patterns occupy the excited/happy quadrant. Shoulder tap seems to be a pattern that is highly susceptible to changes in duration and temperature affecting its emotional quality. As expected from the means, increasing the duration has a consistent effect of increasing the V and A of the emotional response to patterns at both dynamic temperatures. Finally, increasing the dynamic temperature from cold to hot, decreases the V of the emotional response. The effect of temperature on A is more interesting. In agreement with the analysis of the means, temperature does not have a significant effect on the A-space, but upon closer observation, individual patterns might have a unique A-response to temperature. For example, from Figure 7 and Figure 8, pattern 8 (spine up) at a short duration, has (V, A) = (0.03, −0.06) at a hot dynamic temperature, and (V, A) = (0.42, 0.26) at a cold dynamic temperature, implying that increasing the dynamic temperature of pattern 8 decreased A by 0.32. Though V decreased upon an increase in temperature as expected, arousal significantly decreased with an increase in temperature. At a longer duration, pattern 8 does not display a large change in A from cold to hot: (V, A) = (1.26, 0.87) → (0.65, 0.97). On the other hand, increasing the dynamic temperature for pattern 7 (6 motor burst) at a short duration has the following effect: (V, A) = (0.23, −0.06) → (−0.03, 0.23); here, V is decreased as expected, but A is increased by 0.29. At a longer duration of pattern 7, increasing dynamic temperature results in (V, A) = (0.48, 0.94) → (0.45, 1.00); here, V and A are slightly decreased and increased, respectively. Hence, the design of spatiotemporal vibrotactile patterns may have a significant effect on human response to dynamic temperature.

4. Discussion

From the results described in the previous section, we confirmed the hypothesis that combining vibrotactile and thermal stimulation has a significant effect on emotional response. We used two dimensions to measure emotions—valence (V) and arousal (A). There were three factors under consideration: spatiotemporal patterns (12 higher-order vibrotactile patterns); duration of vibration (short or long), and dynamic thermal stimulation (direction of change—hot or cold). In the following discussion, we systematically explore the effect of each factor on the emotional response.

4.1. Spatiotemporal Patterns

The ratings from participants were converted to a scale −3 to +3 and plotted on the circumplex model, as shown in Figure 9. The range where most patterns fall is closer to neutral and within the excited/happy quadrant. An interesting observation is the clustering of datapoints at longer durations of vibration. This is especially evident for hot stimuli. Figure 9 shows that the separation of emotional response to patterns between shorter and longer durations is more pronounced at cold dynamic stimulations than hot. Though no significant interaction effects were observed, temperature seemed to affect the spread of datapoints depending on duration. In addition, certain patterns do not follow the observed trends, e.g., pattern 8 (spine up) and pattern 12 (shoulder tap), while other patterns display a pronounced effect to duration, as in pattern 1 (alternate top to bottom). Seifi and MacLean previously studied the affective response of higher-order vibrotactile stimulation in handheld devices [33]. The handheld prototype consisted of a vibrotactile actuator sandwiched between a smartphone case, and stimulation was felt on the hand. Two factors were utilized in their study: rhythm patterns and frequency. Here, rhythm patterns refer to spatiotemporal patterns where the final pattern is dependent on the ‘shape of vibration’ (duration, activation units and pauses). Seven rhythm patterns were chosen from a previously validated study [36] and two frequencies were chosen (75 Hz and 175 Hz) based on pilot studies. The pilot studies showed that participants preferred patterns at those frequencies for recording emotional responses. In the current study, we used 12 patterns that were selected from a set studied by McDaniel et al. [23]. They found that certain spatiotemporal patterns can elicit emotions without any training or paired stimulus. This finding is useful in designing assistive aids for social interactions, as it confirmed the possibility of designing aids that naturally elicit emotions. Seifi and MacLean found that rhythm patterns have a great affective significance and affect the subjective preference of vibrations [33]. This could explain the preference of certain higher-order vibrations. Another noticeable aspect is that paired stimuli did not display a similar extent of emotional response, though they generally did follow the trends. Paired pattern stimuli refer to similar patterns differing in the directionality of activation, i.e., pattern 3 (wave left) and 4 (wave right); 5 (wave down) and 6 (wave up); 8 (spine up) and 9 (spine down); 10 (snake horizontal) and 11 (snake vertical).

4.2. Effect of Duration of Vibration on Emotions

Changes in duration had a significant effect on the emotional response of vibrothermal stimuli: increasing the duration from 100 ms to 1000 ms generally led to increases in both V and A. We used two levels (100 ms and 1000 ms) to record the relative change in emotional response on changing the duration of vibrotactile stimulation. McDaniel et al. used three levels of duration, 250 ms, 500 ms and 1000 ms, and found that patterns at 1000 ms elicited sadness with greater frequency than at 250 ms [23]. In that study, the vibrotactile stimulation was applied using the Haptic Chair; the same apparatus was used in the current study. In previous studies, generally, longer higher-order spatiotemporal vibrations were perceived as pleasant [33], which is in line with the results presented here. Wilson and Brewster studied the emotional response to multimodal stimulation including temperature, vibration, and abstract visual displays. They used two levels of duration: 100 ms and 1000 ms, and found that increasing duration leads to an increase in A and a slight reduction in V. Yoo et al. [31] used six levels of duration: 50, 199, 300, 500, 1000, and 2000, and found that increasing duration leads to an increase in arousal as expected. However, the effect of duration on valence in dependent on the carrier frequency of vibration. For example, at carrier frequency of 60 Hz valence decreased with duration. For carrier frequencies greater than 100 Hz, the effect of duration on valence was not unrelated. This observation might explain the discrepancy in results of duration on valence. It is to be noted that the experimental apparatus in both [6] and [31] were hand-held, i.e., the vibrations were felt on the hand.

4.3. Using Temperature for Affect

As thermal stimulation has fewer controllable parameters (direction, extent, and rate of change), it may be useful to explore multimodal stimuli for enriched expressiveness and expand the affective region in the V–A plot. Wilson et al. suggest combining vibrotactile and thermal stimuli to access a wider range of emotional meaning [5]. In that study, they designed a device where thermal stimulations were conveyed to the palm of the hand. All three parameters in thermal feedback were used: the direction of change (hot and cold), rate of change (1 °C/s and 3 °C/s), and extent of change (2 °C, 4 °C, 6 °C, and 8 °C). They showed a significant impact of temperature on V and no significant effect on A when using +/−4 °C dynamic thermal stimulation. Their work further suggests that thermal feedback of high V and low A can be conveyed through small (2–4 °C) warming changes, while a large extent of change (8 °C) may result in a decrease in V [5]. On the other hand, Salminen et al. noted that small dynamic warming stimulations (less than 6 °C) can lead to low V and high A [12]. In their experiment, thermal feedback was applied to the palm using a custom-built Peltier unit. The current study agrees that there is a significant effect of a change in the direction of temperature on valence and no significant effect on arousal (Table 1). Figure 9 shows that increasing the dynamic temperature (by 4 °C) generally leads to a decrease in V, which is in contradiction to the findings of Wilson et al. [5] and in agreement with Salminen et al. [12].

This contradiction might be due to many factors: (1) the general location of the stimuli on the circumplex model; (2) the extent of change studied; and/or (3) the presentation method used. It is useful to note that the choice of spatiotemporal patterns may influence the quality of emotional response to thermal stimulation. Interestingly, in the current study, certain unique spatiotemporal patterns did demonstrate a pronounced susceptibility to thermal stimulation on both V and A. Additionally, increasing the extent of temperature change may result in higher A, and using smaller (2 °C) or larger (6 °C) shifts in temperature may be more effective in registering changes in V than 4 °C [12]. In addition, Salminen et al. points to the effect of the technique of presentation on the emotional response of participants, which is also dependent on the location of the stimulation on the body [7,12]. Hence, introducing more thermal parameters may lend insight into fine-grained emotional responses.

Yoo et al. previously studied the emotional responses of combined vibrotactile and thermal stimulation with three vibrotactile parameters (frequency, amplitude, and duration) and one thermal parameter (constant temperatures of 20 °C, 30 °C, and 40 °C). They found that constant thermal stimuli generally shift emotional responses without affecting the outcomes from vibrotactile stimulation parameters [32]. This conclusion agrees with the present study (see Table 1 and Figure 9), though a dynamic method of presenting the temperature was used (direction of change variation of 4 °C with respect to skin temperature). Nevertheless, the effect of certain spatiotemporal patterns does display variations in emotional response to thermal stimulation. This means that intelligently combining vibrotactile and thermal stimulation may increase the range of emotions, as pointed out in previous research [5].

5. Conclusions and Future Work

We found that there were no interaction effects between spatiotemporal patterns, duration, and temperature on the emotional response. The effect of duration was consistent with previous studies in that increasing it increased both the valence and arousal of emotional response. We also found that increasing the dynamic temperature generally decreased the valence of emotional response but had no effect on arousal. Nevertheless, certain spatiotemporal patterns do exhibit unique responses to changes in dynamic temperature.

Our work is the first that maps spatiotemporal vibrotactile patterns in combination with dynamic thermal stimulus onto the circumplex model of emotion. Emotional responses to 48 distinct vibrothermal stimulations were explored. As part of future work, the current study will be expanded by including a larger set of spatiotemporal patterns to potentially reach a greater extent of the circumplex model. Previous research [5,32] has shown that thermal range, the rate of change of temperature, form factor, and thermal summation [37] do have a significant effect on emotional response. Hence, future work will investigate further thermal parameters in conjunction with the proposed wider range of vibrotactile patterns. We also intend to demonstrate the possibility of using rich vibrothermal stimulation for general digital communication and sensory substitution to help individuals with disabilities. For example, to enable socio-emotional communication between people through vibrothermal haptic feedback notifications and enable the exploration of visual art to people with visual impairment by proposing a mapping of color to emotions elicited via vibrothermal feedback.