Hand Prosthesis Sensorimotor Control Inspired by the Human Somatosensory System

Abstract

Prosthetic hand systems aim at restoring lost functionality in amputees. Manipulation and grasping are the main functions of the human hand, which are provided by skin sensitivity capable of protecting the hand from damage and perceiving the external environment. The present study aims at proposing a novel control strategy which improves the ability of the prosthetic hand to interact with the external environment by fostering the interaction of tactile (forces and slipping) and thermoceptive sensory information and by using them to guarantee grasp stability and improve user safety. The control strategy is based on force control with an internal position loop and slip detection, which is able to manage temperature information thanks to the interaction with objects at different temperatures. This architecture has been tested on a prosthetic hand, i.e., the IH2 Azzurra developed by Prensilia s.r.l, in different temperature and slippage conditions. The prosthetic system successfully performed the grasping tasks by managing the tactile and thermal information simultaneously. In particular, the system is able to guarantee a stable grasp during the execution of the tasks. Additionally, in the presence of an external stimulus (thermal or slippage), the prosthetic hand is able to react and always reacts to the stimulus instantaneously (reaction times ≤ 0.04 s, comparable to the one of the human being), regardless of its nature and in accordance with the control strategy. In this way, the prosthetic device is protected from damaging temperatures, the user is alerted of a dangerous situation and the stability of the grasp is restored in the event of a slip.

1. Introduction

Human beings use their hands, with their grasping and manipulation capabilities, to perform activities of daily living (ADL), work and acquire new skills. This is also possible thanks to the hand somatosensory system which allows one to perceive information about one’s surroundings and avoid potential injuries [1]. These sensory capabilities, combined with dexterity, are essential for humans to recognise a significant number of properties of a touched object (e.g., roughness, shape, dimension, weight, stiffness, humidity, temperature).

Prosthetic systems are designed to replace a missing body part to meet the needs of amputees, assisting them in daily life and reproducing the lost functions. The literature analysis outlined two tactile information as the most important for amputees [2], i.e., the force applied by the prosthesis during grasping and the temperature of the grasped object. A total of 33 patients (22 males, four females and seven children) declared that the force information due to the contact with the grasped object is crucial (100% of children, 95% of males, 50% of females). Moreover, the thermal information was considered important by 63% of the male adults, one female (25%) and by 71% of the children.

The integration of force information in a prosthetic hand may improve grasp stability by managing possible slippage events and ensuring a desired force level is reached; meanwhile, temperature information may be essential to protect the prosthetic device and the user from possible damages. Having a prosthetic device able to retrieve and manage both force and temperature information could therefore be useful not only to give feedback to the patient but also to reduce the cognitive burden needed to control the device, simplifying grasping movements and improving the accuracy of the performed tasks.

In the literature, several solutions, both commercial and research prototypes, are proposed to try to replicate the human somatosensory system on prosthetic devices. In [3], a low-cost prosthetic arm integrating pressure and temperature sensors was fabricated; in [4], a multisensory glove was designed to record pressure, temperature and humidity information; in [5], new soft flexible thermistors were developed for detecting the temperature of objects; in Iberite et al.’s work [6], temperature sensors were integrated into the fingertips of a prosthetic hand to replicate temperature sensations in the phantom limb of an amputee.

This sensory information is used together with haptic devices integrated into prosthetic technology to feedback the user information about the interaction between the prosthesis and the environment and therefore to enhance the real-time control of the prosthetic device [7]. Nevertheless, these approaches leave the user with the burden of modifying the behaviour of the prosthesis to manage sudden and possibly harmful events.

To reduce the cognitive burden for the user and improve the prosthetic hand’s grasping performance, shared and semi-autonomous control strategies have been proposed. They are mainly focused on improving motion classifier performance by adding information retrieved from exteroceptive sensors, such as camera and distance sensors or magneto-inertial units (IMUs), to electromyographic (EMG) sensors [8]. Other approaches incorporated force and tactile information to adjust grasp strength and respond to varying object textures and slippage events [9] or object shape.

As far as we know, both temperature and force information are still not used for prosthetic control applications. Considering this sensory information in the control strategy could instead be essential to prevent possible damage to the user or to the prosthesis itself. For example, when the patients drink a hot beverage using the prosthetic hand, they may not feel the temperature until the cup reaches the lips [10], which can cause significant injuries to the skin. In addition, high (or cold) temperatures could damage the devices [11], resulting in costly and time-consuming maintenance. Thus, the integration of temperature sensors not only helps to safeguard users but also acts as a sentinel for the prosthetic device itself; thus, the safety and durability of prosthetic technology and of human beings come to the forefront [12].

Some examples of approaches that try to include temperature information in the control strategy are mainly presented in the robotic field, where the human physiological behaviour of moving away from a source of pain is studied to safeguard the robotic systems. For instance, in [13], the authors implemented a control strategy to reproduce human behaviour in response to mechanical and thermal stimuli in a KUKA LWR4+ robotic arm in order to protect the device from unwanted or harmful external interactions. In [14], the authors classified different thermal sensations (extreme cold and hot, moderate cold and hot and warm) from EEG recordings of three healthy human subjects and suggested using this information to move a robotic arm away from or closer to an object at different temperatures.

In conclusion, the existing literature in the field of prosthetics suffers from a notable deficiency in control strategies that incorporate reflexes to effectively exploit tactile and thermal information. This study addresses this limitation by proposing a low-complexity control strategy for a prosthetic hand to increase the prosthetic capability of responding promptly to thermal-pain stimuli and slippage events, to improve the user’s understanding of the prosthetic behaviour and user safety.

As a consequence, the proposed approach consists in (i) reproducing the human somatosensory system on the prosthetic device thanks to the integration of force and temperature sensors in the fingertips; (ii) controlling the prosthetic behaviour to manage the force applied on the grasped object and slippage events to improve grasping stability; (iii) controlling the prosthetic behaviour to promptly react to thermal stimuli, to protect the user from skin injury and prevent the prosthetic system from damage; (iv) including a warning signal easily comprehensible by the prosthesis user, to understand the prosthetic behaviour and in particular to distinguish unbearable object temperature.

This paper is structured as follows: the human thermoception model and the proposed control strategy are presented in Section 2. In the same section, the experimental protocol and the setup used to validate the control strategy are presented. Experimental results on a real prosthetic hand are illustrated in Section 3. Finally, the discussion, limitations and conclusions are reported in Section 4, Section 5 and Section 6, respectively.

2. Materials and Methods

The proposed control strategy has been developed to manage thermal, mechanical and pain information to replicate the functionality of the human hand on the prosthetic one in reacting to external stimuli and protecting the user and the device from damage. Taking inspiration from the human sensory–motor control could enhance the intuitiveness and acceptability of the prosthesis. This objective requires an in-depth study of human thermoception and nociception, which are responsible for temperature perception and the transmission of information to the brain area. According to the intensity of the stimulus, humans can perceive the sensation more or less strongly and implement different behavioural patterns. These are deepened to implement the control strategy.

2.1. Human Thermoception Model

Humans can feel different temperature gradations: freezeover, intense cold, thermal indifference, warm, intense warm and hot burning. They are perceived by three thermoreceptors located between the epidermis and dermis: cold receptors, hot receptors and thermal nociceptors. The tactile, heat, cold and pain signals are transmitted from the receptors to the somatosensory cortex through pathways. Consequently, human beings can activate and process this information simultaneously [15].

Depending on the detected temperature range, four different nerve fibres are involved in transporting the information to the brain area, stimulated by the different temperatures, as shown in Figure 1. Within the range of 15–44 °C, cold and heat receptors are activated, which results in humans perceiving a more or less bearable cold, hot or warm sensation. The range of skin temperatures between 29 °C and 37 °C is an adaptation zone which causes an initial hot or cold sensation to become neutral in a few minutes. This thermal range is ensured by a contact material temperature of approximately 26 °C and an average room temperature of around 24 °C [16].

On the other hand, outside the range of ($T<$ 15 °C and $T>$ 44 °C), only the cold and heat painful fibres are activated, respectively. In particular, temperature above 50 °C can cause significant tissue damage [17], temperature below 7 °C produces a blockage of the skin’s sensory receptors and the achievement of a numbness condition and finally, temperature below 0 °C predicts a frostbite effect, according to ISO-13732-3 [18].

The human being behaves in different ways according to the activation of the nerve fibres. Firstly, when subjected to extreme temperature stimuli, individuals exhibit an unintentional reaction by withdrawing their hand from the object being grasped. This reflex action, known as a spinal reflex, occurs automatically without conscious intent.

Secondly, humans possess the ability to consciously regulate their response to pain stimuli when the temperature is not at a level that would cause visible skin reddening. For instance, individuals can employ intentional strategies to manage pain, such as transferring an object (e.g., warm bread from the oven) from one hand to another or adjusting its position between their fingers [19]. These actions aim at minimising heat transfer between the hand and the object, thereby reducing the sensation of pain. In such situations, humans can effectively integrate thermal, pain and tactile information to optimise their grasp and prevent the object from falling.

To complete the analysis of the human hand behaviour in reaction to high or low temperature, the heat transfer process in object–hand contact has been analysed. As shown in [20], the skin–object interaction is a transient process based on heat conduction. Heat transfer in the skin depends mainly on the interface temperature, which can be represented by the “Two semi-infinite bodies in contact” model.

The assumptions of this model are as follows: (I) contact time short enough to consider the behaviour of the finger pad and material as semi-infinite bodies, (II) insignificant heat flux resulting from blood perfusion and metabolic heat generation, (III) significant variation in the temperature in the direction perpendicular to the contact interface and (IV) negligible thermal contact resistance.

According to the model, the object–skin interface temperature $T_s$ can be expressed as

$$Ts=\frac{T_{material,in}·{\left(k\rho c\right)}_{material}^{\frac{1}{2}}+T_{skin,in}·{\left(k\rho c\right)}_{skin}^{\frac{1}{2}}}{{\left(k\rho c\right)}_{material}^{\frac{1}{2}}+{\left(k\rho c\right)}_{skin}^{\frac{1}{2}}}$$

(1)

where k is the thermal conductivity (W/mK), $\rho$ is the material density (kg/m³), c is the specific heat (J/kgK), $T_{material,in}$ is the initial temperature of the material, $T_{skin,in}$ is the initial temperature of the skin (whose properties [21] are reported in

Table 1. Thermal characteristics of the aluminium, wooden object and skin.

	Aluminium	Wood	Skin
k (W/ mK)	240	0.22	0.54
$\rho$ (kg/m3)	2700	560	1109
c (J/kgK)	460	2200	3470
$\alpha$ (m2/s)	9.8 $\times {10}^{-5}$	30 $\times {10}^{-6}$	1.3 $\times {10}^{-7}$

), ${\left(k\rho c\right)}^{1/2}$ is the contact coefficient and $t_c$ is the contact time. Therefore, $T_s$ depends on the thermal properties and the initial temperatures of the skin and the object material.

The skin temperature profile could be defined by

$$\frac{T{(y,t)}_{skin}-T_s}{T_{skin,in}-T_s}=erf(\frac{y}{2\sqrt{{\alpha }_{skin}{t}_c}})$$

(2)

where ${\alpha }_{skin}$ is the thermal diffusivity of the skin, y the skin thickness below which the thermal receptors and nociceptors are located and t the time from the contact beginning. The thickness y is a variable depending on the involved body area and the physical characteristics of the individual subject. However, to reduce the complexity of the control strategy, a single constant value was considered, corresponding to the epidermal/dermal interface below the hand surface [21].

Regarding the initial skin temperature ($T_{skin,in}$), an average room temperature of around 24 °C and a contact material temperature of approximately 26 °C ensure a skin temperature in the range of (31.2–35.6) °C [16]. Consequently, a rounded average value was chosen as 33 (°C).

Figure 2 shows an example of the skin temperature trend when the human hand is in contact with an aluminium object at 35 °C (Figure 2A), 47 °C (Figure 2B) and 60 °C (Figure 2C).

2.2. Control Strategy

The proposed control strategy (Figure 3) is based on a force control with a position inner loop and slip and temperature monitoring. It has been validated with the IH2 Azzurra (Prensilia, Pontedera, PI, IT) prosthetic hand, shown in Figure 3. The device is composed of 5 under-actuated fingers, each of which has two phalanxes interconnected by two joints. The prosthetic hand has 5 motors to activate flexion–extension of the fingers and abduction–adduction of the thumb. The finger flexion–extension movement is guaranteed by the translational movement of a slider, located on the actuation unit, which activates a tendon wrapped around two pulleys in each finger.

The control strategy exploits the concept of the virtual finger, in which the long fingers (index, middle, ring, little) have been modelled as a single finger whose force ($F_{n_k},k=f$) opposes the thumb’s ($F_{n_k},k=t$), in the pad opposition grip [22].

The control low is described by the equation

$$u=r[K_{P_i}·e_{x_k}-K_{D_j}\dot{x}+T_{el}]$$

(3)

where $\dot{x}$ is the velocity of the slider, $T_{el}$ is the cable tension, $K_{P_i}$ and $K_{D_j}$ are the proportional and derivative gains, r is the sum of the pulley radii, $e_{x_k}$ is the position error of the virtual finger ($k=f$) and of the thumb ($k=t$).

$T_{el}$ is expressed by

$$T_{el}=J^T\left(q\right)·F_{bil}$$

(4)

where $J^T\left(q\right)$ is the transposed Jacobian, $F_{bil}$ is the force resulting from balancing the forces of the virtual finger and the thumb in the common wrist reference frame.

$T_{el}$ is then compared with the adduction/abduction torque measured by the sensors, placed on the prosthetic hand. A PI control is added to ensure zero convergence of the torque error.

The position error $e_{x_k}$ is defined as

$$e_{x_k}=x_{d_k}-x_k-e_{s_k}-e_{T_k}$$

(5)

where $x_{dk}$ is the desired position, $x_k$ the actual position, $e_{T_k}$ the temperature monitoring contribution and $e_{s_k}$ the slip monitoring one, shown, respectively, in the orange and green boxes in the block diagram in Figure 3.

The desired position $x_{dk}$, reported in Equation (6), is obtained by a PI control on the difference between the reference force ($F_{d_k},k=f$ for virtual finger, $k=t$ for the thumb), defined in a previous analysis on human beings [23], and the normal force ($F_{n_k},k=f$ for virtual finger, $k=t$ for the thumb) applied by the prosthetic hand finger on the object surface

$$x_{d_k}=K_{in}·\int (F_d-F_{n_k})dt+K_p(F_d-F_{n_k})$$

(6)

where $K_{in}$ and $K_p$ are integral and proportional gains, respectively.

The warning block, in yellow in the block diagram in Figure 3, is made of a visual or an auditory signal dependent on the output of the temperature monitoring block. The latter is described in Section 2.2.1 and the slip monitoring one in Section 2.2.2.

2.2.1. Temperature Monitoring

According to the analysis reported in Section 2.1, the following four temperature thresholds of the skin ($T_{skin}$), which are obtained by the “Two semi-infinite bodies in contact” model, have been set as follows:

• $T_{int}^{hot}=44$ °C;
• $T_{int}^{cold}=15$ °C;
• $T_{ext}^{hot}=48$ °C;
• $T_{ext}^{cold}=7$ °C.

The 48 °C and 7 °C thresholds represent, respectively, the maximum and minimum thermal limits the skin surface can endure without incurring injuries. The other two thresholds (44 °C and 15 °C) represent the temperature limits for humans for not perceiving painful sensations.

The temperature monitoring contribution $e_{T_k}$ in Equation (5) is defined as

$$e_{T_k}=K_T·{\int }_0^{t_f}\beta dt$$

(7)

where $\beta$ is a binary factor and $K_T$ (m/s) is a weight which defines the opening speed of the prosthetic hand.

According to the temperature of the grasped object, the system will behave in different ways. As reported in Algorithm 1, if the internal thresholds ($T_{int}^{hot}$, $T_{int}^{cold}$) are not overcome, the temperature contribution to the control strategy is cancelled ($\beta =0$) since the object temperature causes no pain to the skin area. Otherwise, $\beta =1$.

Algorithm 1 Temperature monitoring.

if  $T_{int}^{cold}<T_{skin}<T_{int}^{hot}$  then       $\beta =0$ else       $\beta =1$ end if if  $T_{skin}\le T_{ext}^{cold}\left|\right|T_{skin}\ge T_{ext}^{hot}$  then         if $t_c\le 1$ then               $K_T=K_{T1}$               $e_{s_k}=0$         else if $t_c>1$ then               $K_T=0$         end if else if  $T_{int}^{hot}\le T_{skin}<T_{ext}^{hot}\left|\right|T_{ext}^{cold}<T_{skin}\le T_{int}^{cold}$  then         $K_T=0$ end if

If the maximum thresholds ($T_{ext}^{hot}$, $T_{ext}^{cold}$) are instantaneously surpassed within a contact time ($t_c$) less than 1 s, the prosthetic hand will instantaneously open and a warning signal is activated: $K_T=K_{T1}$ with $K_{T1}\ne 0$, regardless of the presence or absence of slippage events $e_{s_k}=0$ (pseudocode Algorithm 1). The time threshold $t_c$ represents the minimum time threshold, empirically determined, needed by the prosthesis to grasp the object and lift it off the ground. The warning signal is introduced to alert the user that the opening of the hand is caused by an unbearable object temperature and not due to a failure of the system.

If the maximum thresholds ($T_{ext}^{hot}$, $T_{ext}^{cold}$) are exceeded gradually ($t_c>1$), the prosthetic hand does not open ($K_T=0$), even if the human behaviour would include it, because an abrupt release of the grasped object could pose potential dangers, particularly when dealing with liquids such as beverages. However, the dangerous temperature information is provided to the user thanks to the warning signal so that the user can choose how to react by using electromyographic (EMG) signals (G in the block diagram in Figure 3).

When the intermediate thresholds are exceeded ($T_{int}^{hot}$, $T_{int}^{cold}$), but not the external ones ($K_T=0$), a different warning signal is activated.

2.2.2. Slip Monitoring

The $e_{s_k}$ has been defined to manage the slip events. It is defined in [23] as follows:

$$e_{s_k}=K_{pk}{\int }_0^{t_f}\gamma dt,$$

(8)

where $slip$ is a signal obtained from the touch-and-slippage detection algorithm [24], $K_{p_k}$ (m/s) is a constant regulating the $e_{s_k}$ weight in the control and $\gamma$ is a binary signal. The integration of this signal will guarantee an increment in the applied grasping force in the presence of slippage [9].

As reported in the pseudocode Algorithm 2, the setting of the thermal and time thresholds required the activation of $e_{s_k}$ contribution under specific thermal conditions: if the temperature of the object does not cause the maximum thresholds ($T_{ext}^{hot}$, $T_{ext}^{cold}$) to be exceeded within 1 s from contact, the prosthetic hand can close to ensure a more stable grip ($\gamma =1$); otherwise, the skin temperature is predominant over the sliding ($e_{s_k}=0$).

Algorithm 2 Slip monitoring.

if  $slip\&\&T_{ext}^{cold}<T_{skin}<T_{ext}^{hot}\&\&t_c<1$  then        $\gamma =1$ else  $\gamma =0$ end if

2.3. Protocol and Experimental Setup

The experimental setup (Figure 4) consists of the IH2 Azzurra prosthetic hand, described in Section 2.2, which integrates temperature (GY-906 MLX90614 infrared sensor, Melexis, Ypres, Belgium) and force (FSR 402 resistive force sensor, Interlink Electronics, Westlake Village, CA, USA) sensors. The former aims at measuring the temperature to be used in the proposed control strategy to prevent possible damage to the user and the prosthesis due to dangerous temperatures. The latter aims to measure the force applied by the prosthesis to the object. It is exploited to improve grasp stability and prevent possible slippage events in the control strategy.

The sensor’s output has been recorded by two control boards implemented for the application. The selection and characterisation of temperature and force sensors are described in the following sections.

2.3.1. Temperature Sensors

In order to select the temperature sensor, the physiological characteristics of the human hand were identified to set the technical specifications for the temperature sensor. In particular, the integration of the thermal sensors in the prosthetic hand required the following:

• Response time less than 1 s, since the latency of human reflex action is about 50 ms [25];
• Reduced physical dimensions (on the order of mm) to facilitate the integration of the prosthetic fingers;
• Measurement error less than 1 °C to have an accurate response;
• Resolution of about 0.5 °C corresponding to the minimum thermal variation the humans can detect [26];
• Temperature range of (−5–110) °C including the minimum and the maximum temperature of daily used objects.

GY-906 MLX90614 infrared (IR) temperature sensor produced by Melexis was chosen (Figure 4). The sensor detects the IR energy emitted by materials at temperatures above absolute zero, without requiring contact with the object. It is very fast and accurate (accuracy: ±1 °C in the range (0–50) °C) with an integrated conditioning circuit (17 bits ADC, resolution: 0.02 °C) and small dimensions (11.3 × 16.9 × 7 mm).

The emissivity is closely linked to the body involved in the IR wave propagation. This value must be taken into account to avoid measurement errors of the object temperature to be grasped by the prosthetic system. Consequently, it has been calibrated for aluminium and wood objects, as reported in [27].

2.3.2. Force Sensors

Regarding the force sensors, an FSR 402 resistive force sensor, shown in Figure 4, was selected for recording the normal force impressed by the prosthetic fingertips on the object [28]. The relation between the output signal from the sensor voltage value V and the force value F was established with a static characterisation and modelled as [29].

Both temperature and force sensors have been located on the index and thumb prosthetic fingers because they are always involved in the main hand grasps (Figure 4). The fingertip of the prosthetic hand has been designed to facilitate the integration of both sensors (Figure 5). As shown in Figure 5A,B, the fingers have a flat surface to minimise problems related to the imperfect contact between the fingers and the object. In addition, the FSR sensor has been positioned on the finger and covered with silicone to protect the sensitive area, while the infrared one has been integrated into the fingertip to ensure a minimum distance from the object since contact with the object could generate measurement errors, as reported in the technical specifications. The IR sensor has also been slightly inclined upwards 10 (°C) to maximise the sensor view of the object in the grasping task.

The control of the prosthetic device has been implemented in C programming language under Windows 10 Operating System, on a personal computer equipped with $Inte{l}^R$ $Cor{e}^{TM}$ I7-10710U processor at 1.10 × 1.61 GHz, 16 GB of RAM. Through a purposely developed graphical interface, the device could recreate different grasps (i.e., lateral grasp, bi-digital grasp, power grasp) with different levels of speed. This interface will be replaced with an EMG high-level control to return the command to the user. The warning signal is identified by a LED (bright visual signal) and a buzzer (auditory signal). The intensity of the sound and light is set depending on the activated thermal thresholds: more intense if external thresholds ($T_{ext}^{hot}$, $T_{ext}^{cold}$) are activated in less than 1 s, less intense in the other cases.

Since the object–hand contact response is strongly correlated to the physical and thermal properties of the object and of the skin [21], an aluminium parallelepiped (dimensions: 30 × 15 × 110 mm) and a wood cylinder (dimensions: length 150 mm, radius 12 mm) were selected and reported in Figure 6A,B, respectively. Their thermal characteristics, shown in Table 1 [30], reveal how the two materials respond differently to temperature, since aluminium is a good heat conductor, whereas wood has low conductivity. Consequently, the developed control strategy has been validated for different everyday uses.

The prosthetic hand grasped the two objects while performing a bi-digital grasp since the index finger and thumb are generally involved in all grasping tasks.

Both aluminium and wood objects are heated in an oven until the desired temperature is reached. The object temperature is monitored through a benchmark sensor, i.e., the Shimmer3 Bridge Amplifier+, which has the following characteristics: accuracy: ±0.1 °C (25–45) °C, ±0.2 °C (0–60) °C, temperature range: ($-18–80$) °C.

The aluminium object was heated at the following temperatures: (i) 27 °C to evaluate the behaviour of the prosthetic device in the bearable temperature range; (ii) 102 °C to ensure the overcoming of the maximum threshold; (iii) 48 °C to evaluate the behaviour of the device when the internal threshold is overcome but not the external one. Finally, the wooden object was heated at 103 °C to demonstrate how wood requires higher temperatures to elicit a prosthetic hand response. The same results can be replicated for $T_{int}^{cold}$ and $T_{ext}^{cold}$ temperature thresholds.

The force exerted on the grasped object, the occurrence or non-occurrence of slip, the position of the fingers during the movement, the skin temperature trend, the grasped object temperature and the activation of the visual and auditory signal were recorded during the movement of the prosthetic device. The information was detected for each object and temperature; the results were compared and analysed.

For validating the performance of the control system, the following performance indicators have been evaluated:

• The success rate, defined as $s_r=\frac{N_{succ}}{N_{tot}}$, is the number of trials correctly performed ($N_{succ}$), in terms of stability of the grasping task ($s_{r_s}$) and the reaction of the prosthetic device to external stimuli (${\mathit{s}}_{{\mathit{r}}_{\mathit{r}}}$), with respect to the total number of performed trials (${\mathit{N}}_{\mathit{tot}}$) (10 trials for each temperature).
• The reaction time (${\mathit{t}}_{\mathit{r}}$) is the average time obtained between the critical events, i.e., slippage (${\mathit{t}}_{{\mathit{r}}_{\mathit{s}}}$), or exceeding the critical threshold (${\mathit{t}}_{{\mathit{r}}_{\mathit{T}}}$), and the corresponding hand reaction in each trial.
• The average response time (${\mathit{t}}_{\mathit{resp}}$) of the force (${\mathit{t}}_{{\mathit{r}}_{\mathit{F}}}$) and temperature (${\mathit{t}}_{{\mathit{r}}_{\mathit{T}}}$) sensors, with the aim of validating the responsiveness of the whole sensory system.

3. Results

The results obtained by subjecting the aluminium object to three different temperatures (27 °C, 102 °C, 46 °C) and the wooden object at 103 °C, are analysed. For the sake of clarity, in Figure 7 and Figure 8, the results obtained in only one trial are shown. In particular, from the top of each group of graphs, the force impressed by the fingertips on the object, the occurrence or non-occurrence of slip and touch, the position of the fingers during the movement, the skin temperature trend ($T_{skin}$), the temperatures of the grasped object ($T_{object}$) and the activation of an auditory and visual signal (LED) were plotted for each temperature condition, as described in Section 2.3. Furthermore, the $T_{int}^{hot}$ and $T_{ext}^{hot}$ thresholds are marked with two horizontal lines in light and dark purple; the reaction of the prosthesis to a slipping phenomenon is underlined by dark blue vertical lines in a red zoomed block of the fingers’ angular excursion; the reaction of the prosthesis when $T_{ext}^{hot}$ is overcome in less then 1 s is highlighted with dark blue vertical lines in a green zoomed block.

Figure 7A shows the results obtained when the prosthetic hand performs a bi-digital grasp for grasping an aluminium object at a temperature of 27 °C. This temperature does not cause the thermal thresholds to be exceeded. In the event of one or more slippage occurrences, the hand promptly executes a grasp correction, with a reaction time $t_{r_s}$ measured at 0.03 s.

Moving on to Figure 7B, the results obtained when the aluminium object was subjected to 102 °C are reported. Both $T_{int}^{hot}$ and $T_{ext}^{hot}$ temperature thresholds are exceeded after 0.14 s and 0.15 s of contact, respectively; consequently, the prosthetic hand opens rapidly in $t_{r_T}=$ 0.04 s after overcoming the maximum threshold.

In Figure 8A, the aluminium object is at 46 °C. The threshold of 44 °C is exceeded after around 3.50 s, while the maximum threshold is not overcome as the skin temperature tends to an asymptotic value of approximately 47 °C. After 4.03 s of contact, a slippage event occurs and the device reacts instantaneously ($t_{r_T}=$ 0.03 s) to restore the stability of the grasp.

Lastly, in Figure 8B, the prosthetic hand overcomes the $T_{int}^{hot}$ and $T_{ext}^{hot}$ thresholds, after, respectively, about 1.03 s and 3.50 s from the contact, since the object temperature is equal to 103 °C. After 1.20 s, 1.50 s and 3.01 s of contact, some slippage events occur and the device reacts to restore the grasp stability.

As shown in

Table 2. Performance indicators for each temperature.

Material	${\mathit{T}}_{\mathit{object}}$	${\mathit{S}}_{{\mathit{r}}_{\mathit{s}}}$	${\mathit{S}}_{{\mathit{r}}_{\mathit{r}}}$	${\mathit{t}}_{{\mathit{r}}_{\mathit{s}}}$	${\mathit{t}}_{{\mathit{r}}_{\mathit{T}}}$
Aluminium	27 °C	0.90	1.00	0.03 s	×
	102 °C	0.89	1	×	0.04 s
	48 °C	0.90	1.00	0.03 s	×
Wood	103 °C	0.90	1.00	0.03 s	×

, the previously defined control strategy indicators have the following values: (i) average success rate $s_{r_s}$ is equal to 0.86; (ii) average success rate $s_{r_T}$ is equal to 1.00; (iii) average reaction time $t_{r_s}$ has a value of 0.03 s; (iv) average reaction time $t_{r_T}$ is equal to 0.04 s.

In addition, the average response times of the force and temperature sensors are equal to 0.30 s and 0.66 s, respectively.

4. Discussion

The proposed control strategy aims at increasing the prosthetic device’s capability to respond to external stimuli and improve user safety and grasp stability. To assess its effectiveness, the control strategy was evaluated by using both an aluminium and a wood object. The evaluation involved conducting tests at four different temperatures and in presence of slippage events, as reported in Figure 7 and Figure 8.

When a slippage event occurs, the force sensors detect an instantaneous reduction in the normal force. In this situation, the prosthetic hand restores grasp stability by correcting the fingers’ angular position, as shown in the red zoomed boxes in Figure 7 and Figure 8.

In Figure 7B, the hand behaviour when both temperature thresholds are exceeded within a contact time of less than 1 s is shown: the prosthetic hand does not react in the presence of an index slip event, as this control is inhibited, but it instantaneously opens, as shown in the green zoomed box. In this situation, the high-intensity visual and auditory signal lights up and remains on to alert the user that this reaction of the prosthesis occurred due to a painful object temperature and not a technical error of the device.

Lastly, Figure 8A,B show, respectively, the behaviour of the system when only the intermediate threshold ($T_{int}^{hot}$) is exceeded (Figure 8A), and when both are overcome with a contact time higher than 1 s (Figure 8B). In both cases, the purpose is to alarm the user that the temperatures may be harmful to the skin and to the device, thus not encouraging contact of the object with other parts of the body or prolonging the contact of the device with the object itself. This warning is sent through a prolonged low-intensity visual and auditory signal.

Looking at the performance indicator values (reported in Table 2), i.e., average success rate $S_{r_r}$ of 1.00, it is possible to conclude that the prosthetic device is able to react to any external stimuli, regardless of whether it is thermal or slippage. Furthermore, grasping stability is ensured in most cases, as indicated by the $S_{r_s}$ equal to 0.9. The case in which the object fell down is due to an initial malpositioning of the object to be grasped.

As far as the reaction time ($t_r$), the obtained values outline that the system is able to respond to an external stimulus regardless of the type of stimulus.In addition, the reaction times are well below 1 s, which is comparable to the ones of a human being [25].

Regarding the measuring of the force and temperature information, the graphs show, respectively, a response time of 0.30 s and 0.66 s, according to the technical specifications reported in Section 2.3.

5. Limitations

Although the performances of the proposed control strategy have been demonstrated in this paper, some limitations are still present. The proposed control strategy has been purposely developed and validated for an under-actuated cable-driven prosthetic hand (IH2 Azzurra). Some adaptations are needed to implement the control strategy on other prosthetic hands while maintaining its core operational concept. Additionally, further investigation including healthy and amputee subjects is needed to validate the system usability in an unstructured environment. In particular, the inclusion of daily activities and tasks in the experimental protocol may provide a more comprehensive evaluation of the control strategy’s practical usability.

6. Conclusions

This study proposes a control strategy capable of managing slip events and thermal stimuli, with the aim of reducing the complexity and increasing user safety and grasp stability.

Temperature and force sensors were embedded in a prosthetic hand’s fingertips and the obtained information was used to change the behaviour of the hand depending on the measured object temperature and the presence of slippage events. Unbearable temperatures were managed by considering two cases: (1) if the contact time between the hand and the object lasts less than 1 s, the prosthetic hand abruptly opens and a warning signal is sent to the user; (2) if the contact time lasts more than 1 s, the device maintains the grasp, managing the possible slippage events, and the user is warned of the danger in order to prevent body damage. The same behaviour is implemented when only the internal temperature threshold is overcome. Different intensities of the warning signals are used to distinguish among the different cases.

The experimental results obtained on the IH2 Azzurra prosthetic hand endowed with the sensors and the developed control strategy demonstrated that the system is able to establish a stable grasp during tasks and always react to the stimulus instantaneously regardless of its nature.

The future works will be devoted to the recruitment of healthy and amputee subjects in order to validate the hand system controlled by subjects in terms of user-friendliness, movement naturalness and usability in daily life activities.