Finger Joint Stiffness Estimation with Joint Modular Soft Actuators for Hand Telerehabilitation

Abstract

In a telerehabilitation environment, it is difficult for a therapist to understand the condition of a patient’s finger joints because of the lack of direct assessment. In particular, not enabling the provision of spasticity evaluation significantly reduces the optimal performance of telerehabilitation. In a previous study, it has been proposed that finger stiffness could be estimated using an analytical model of a whole-finger soft actuator. However, because the whole-finger soft actuators require high air pressure for high bending performance and are costly to customize for each patient, using joint modular soft actuators for telerehabilitation turns to be a necessity, though stiffness estimation with joint modular soft actuators has not been studied yet. Another problem is caused by using a marker-based joint angle measurement, which requires the markers to be attached to the exact positions, and limits its application in telerehabilitation. In this study, we proposed a procedure of finger joint stiffness estimation that combines information acquired from a joint modular soft actuator and a marker-less hand joint position acquisition device. Correction parameters were added to the previous analytical model for -the bending analysis of a joint assisted using a joint modular soft actuator. Moreover, a multi-variate regression model was implemented for correcting joint angles obtained from the hand joint position acquisition device. As a result, a reasonable accuracy of stiffness estimation was achieved for rehabilitation with the joint modular soft actuators, which suggests the possibility of using the proposed method to evaluate the finger spasticity in a telerehabilitation environment. This is a big step forward towards optimal hand telerehabilitation.

1. Introduction

Stroke is one of the leading causes of death and disability in adults worldwide [1]. At least 65% of stroke patients have hand impairment [2]. It has been reported that hand dysfunction can be a major obstacle in daily life and labor [3]. Therefore, it is important for patients to undergo hand rehabilitation in order to recover from stroke side effects [4,5]. Constant high-quality rehabilitation is necessary for effective hand rehabilitation during motor paralysis [6]. Generally, rehabilitation is performed under the guidance and assistance of a therapist; thus, a facility where therapists work with the other specialists is usually the place for rehabilitation. However, the recent spread of infectious diseases and the shortage of visiting therapists have reduced the frequency of rehabilitation that can be provided to patients [7]. Furthermore, most therapists are often located in urban centers, so patients living in underpopulated areas may be unable to participate in rehabilitation [3].

Telerehabilitation is gaining attention as a new means to address such problems, improving access for patients living in remote areas and reducing the financial and time costs associated with outpatient rehabilitation [8]. However, in a telerehabilitation environment, the therapist and patient are normally far away from each other, making it difficult for the therapist to palpate and understand the condition of the patient’s hand [9].

Finger flexor spasticity, a motor disorder that causes involuntary muscle contractions due to impaired reflex function, is a common aftereffect of stroke [10]. Spasticity not only increases the stiffness of the fingers but also decreases their range of motion, affecting the recovery of the patient’s motor function [11]. Therefore, understanding a patient’s spasticity condition can help therapists determine optimal rehabilitation [12]. However, spasticity is difficult to evaluate without direct contact with the therapist, as is the degree of muscle tone that fluctuates from day to day [13]. Therefore, a method to objectively quantify spasticity by non-palpation is necessary in telerehabilitation. Up to now, dedicated stiffness measurement devices that have been developed are large and heavy because they consist of many rigid body parts [14,15,16]. When rehabilitation is performed at a patient’s home, the available space is assumed to be limited, and it is desirable to minimize the number of devices by accomplishing simultaneous functions, such as the evaluation of spasticity condition and rehabilitation using the same device.

Heung et al. developed a system that combines the two functions of finger motor support and spasticity condition evaluation by using a soft elastic composite actuator (SECA), a hand rehabilitation device, to estimate finger joint stiffness [17]. Soft actuators are suitable devices for hand rehabilitation in a telerehabilitation environment because of their flexibility, lightweight, high affinity with the body, and portability. In [18], the stiffness estimation method using a SECA was found to agree with standard joint stiffness quantification methods. Furthermore, it was suggested that a SECA could be used to quantitatively assess the stiffness of the passive metacarpophalangeal (MCP) joint during the performance of various tasks in hand rehabilitation. The stiffness estimation method using a SECA estimates stiffness from an analytical model using the input air pressure of the soft actuator and a finger joint angle, which requires a structure-dependent soft actuator analytical model and a joint angle measurement. In addition, there are several technical problems with its application in telerehabilitation.

Firstly, a whole-finger soft actuator with a single-pocket structure with multiple air chambers connected to each other, such as the SECA, is not suitable for home environments because it requires high air pressure to achieve high bending performance [17,19,20]. Secondly, this type of actuator has only one pneumatic input and cannot support individual joints. Thirdly, the dimensions of the actuator need to be changed to accommodate individual differences such as finger size, but this change may cause unintended changes in actuator motion [21]. Finally, it is costly to customize the actuator for each patient. Yun et al. and Kokubu et al. developed joint modular soft actuators that are divided into parts corresponding to each finger joint [20,21]. By using spacers to connect each actuator, the actuators can accommodate differences in the length of the user’s fingers without changing the dimensions of the actuator. This enables personalized customization at a low cost. In addition, the actuator does not require high air pressure because it transmits force better than a whole-finger soft actuator does [20]. Based on these considerations, the joint modular soft actuator can be considered a suitable soft actuator for use in a telerehabilitation environment because it provides high bending performance with minimal energy and can accommodate individual differences with minimal effort. However, it is necessary to incorporate an estimation of the stiffness of the finger joints (i.e., spasticity condition evaluation function), though stiffness estimation with joint modular soft actuators has not been studied yet. In this study, we designed a modular version of the SECA for actuating individual joints (Modular-SECA). However, as explained in the following paragraph, the analytical model of SECA for estimating finger joint stiffness is not completely applicable to this new type of actuator.

The analytical model of the SECA is derived from the air chamber unit energy conservation law, so the length of the air chamber is included in the analytical model, but not the length of the actuator itself. When the lengths of the air chambers are equal, the analytical model can be interpreted as indicating that the SECA and Modular-SECA have the same bending performance. Therefore, it is possible to estimate the stiffness of the Modular-SECA using the original analytical model of the SECA. However, unlike the SECA, the joint modular soft actuator has rigid connectors at both ends, and the length ratio of the actuator body to the chamber is different [17,20]. Therefore, the two types of soft actuators may exhibit different bending performances, and the analytical model of the SECA cannot be applied directly to that of the Modular-SECA.

Another problem of the stiffness estimation method described in [17] is caused by using a marker-based joint angle measurement. However, it is difficult for the patient or his/her family members to attach the markers at precise positions, which limits its application to telerehabilitation.

Furthermore, using 2D markers limits the acquisition of joint angles on a flat surface, making it impossible to measure the joint angles of the middle and ring fingers, and thus making it difficult to estimate the joint stiffness of them. Zhou et al. have developed a system that simultaneously obtains the joint angles of multiple fingers and estimates their stiffness by integrating a flex sensor into an actuator [22]. Flex sensors do not interfere with the bending response of soft actuators too much, but they must be calibrated individually for each actuator and cannot measure finger or hand orientation [23,24]. Moreover, using this system, it is not possible to measure joint angles when a soft actuator is not attached. By designing a system that can measure joint angles without having the soft actuators attached, it is also possible to measure joint range of motion, increasing the system’s functionality. In addition, having a high-cost system can limit how many devices can be provided for telerehabilitation [25].

The leap motion controller (LMC) is an inexpensive and compact marker-less measurement device for tracking and acquiring the position of hand joints. Although the accuracy of hand joint angle measurement is not sufficient for rehabilitation purposes [26], it is possible to improve the measurement accuracy by fitting a regression model of joint angle [27]. However, the combined use of LMC and soft actuators has not yet been investigated. Especially, joint modular soft actuators may cause the motion of one single joint, which is different from most natural hand motions. It is also unclear how the attachment of the soft actuator affects the LMC and whether or not the measurement accuracy can be improved in the same way as when the soft actuator is not attached.

In this study, we proposed finger joint stiffness estimation for hand–finger joints using Modular-SECAs and the LMC, and verified the feasibility of this procedure by performing stiffness estimation on a mannequin hand and a subject’s MCP joint, respectively. We designed the Modular-SECA, a joint modular soft actuator incorporating the finger joint stiffness estimation function. In addition, correction terms (or parameters) were added to the previous analytical model for the bending analysis of a Modular-SECA supporting a joint for the purpose of hand rehabilitation. The correction parameters were determined by comparing the values of the bending angles obtained from the analytical model with measured angles from a free bending experiment with the Modular-SECAs. Moreover, a regression model accounting for the actuation for and motion of the neighboring joints was proposed to correct the direct outputs of LMC.

This paper is organized as follows: First, the design and improved analytical model of the joint modular soft actuator for stiffness estimation are presented. Next, the regression model method to improve the accuracy of LMC joint angle measurement is presented. Finally, validation experiments using a mannequin hand and a subject’s hand are presented in which stiffness estimation is performed using a combination of these methods.

2. Materials and Methods

2.1. Joint Modular Soft Actuators for Joint Stiffness Estimation

This subsection gives a concise introduction to the design of the joint modular soft actuator, for the sake of clarity. A more comprehensive explanation can be found in [20]. The analytical model for the SECA and thereby the improved analytical model for the joint modular soft actuator are explained.

2.1.1. Design and Fabrication

The Modular-SECA consists of an elastomer body, a semi-circular air chamber with air pockets, a silicone tube, a fiber reinforcement wrapped around its circumference, 3D-printed rigid connectors, and a torque compensation layer. When pressurized with air pressure, the fibers wrapped around the surface of the actuator suppress radial elongation. The torque-compensating layer limits axial elongation at the bottom, allowing only the top of the actuator to extend and the finger to bend. During decompression, the torque-compensating layer provides a supplementary bending moment to extend the finger to its initial position.

The silicon part of the actuator was made by pouring silicon (Smooth-On, Dragon Skin 10 Medium) into a 3D-printed mold. A stainless-steel plate (A2 stainless-steel plate) with a thickness of 0.2 mm was used for the torque compensating layer, which was attached to the bottom of the silicon part of the actuator. Next, 0.3556 mm diameter reinforcing fibers made of Kevlar^® (Dupont, Inc., Wilmington, DE, USA) were wound around the actuator, and silicon tubing for air flow and 3D-printed connectors made of polylactic acid (PLA) resin were attached. Finally, a thin coat of silicone (Smooth-On, Dragon Skin 10 Medium) was applied over the fibers to prevent them from shifting. Figure 1a shows the cross-section and dimensions of the actuator.

2.1.2. Analytical Model for Energy Conservation of Modular-SECA

The analytical model for the free space bending analysis of the SECA is obtained via the conservation of energy as in [17]:

$$P=\frac{W_A+W_L}{∆V}$$

(1)

where $P,W_A,W_L$ and $∆V$ denote the input air pressure, the bending strain energy stored in the soft actuator body, the bending strain energy stored in the torque-compensating layer, and the increase in volume, respectively. The bending analysis in free space is the bending analysis using the actuator only.

Due to the structural differences between the SECA and the Modular-SECA, the energy distribution in the actuator is different. The rigid connectors attached to the two ends of each actuator and the neighboring Modular-SECA connected through the connectors may cause the change in the ratio of both $W_A$ to $W_L$, and their sum to $P∆V$. Therefore, the analytical model of the SECA was modified to match the energy distribution of the Modular-SECA, as shown in Equation (2):

$$P=\frac{{\alpha }_A{W}_A+{\alpha }_L{W}_L}{{\alpha }_V∆V}$$

(2)

where ${\alpha }_A,{\alpha }_L$ and ${\alpha }_V$ are the correction parameters added to Equation (1).

The analytical model for bending analysis in free space was verified with a Modular-SECA prototype experiment, and a finite element model (FEM). In this study, the Yeoh 3rd-order model was chosen for the strain energy density function of silicone rubber as follows [28]:

$$w_m=\sum _{i=1}^3{C}_i{\left({\lambda }^2+{\lambda }^{-2}-2\right)}^i$$

(3)

$w_m$ is the bending strain energy density function, $\lambda$ is the principal stretch, and $C_i(i=\mathrm{1,2},3)$ is the material coefficient. The 3rd-order Yeoh model parameters for the silicon used (Smooth-On, Dragon Skin 10 Medium) were $C_1=0.04773\mathrm{M}\mathrm{P}\mathrm{a},C_2=3.42\times {10}^{-4}\mathrm{M}\mathrm{P}\mathrm{a},$ and $C_3=-1.1942\times {10}^{-6}\mathrm{M}\mathrm{P}\mathrm{a}$, respectively. These parameters were calculated by using the material data provided in [29]. The material parameters of the stainless-steel plates used as torque compensation layers with a Young’s modulus of 193 GPa, a Poisson’s ratio of 0.318, and a density of 8000 kg/m³ [30]. The bending angle in free space was measured by a camera-based two-dimensional marker detection system (Figure 1b). The camera (C930eR, Logicool, Lausanne, Switzerland) recorded the 2D position coordinates of the markers and the angle was calculated. The bending angle, $\theta$, was obtained using Equation (4) when air pressure was applied from 0 kPa to 80 kPa in 10 kPa increments. The bending angles obtained from the experiment were compared with those from the analytical model, and correction parameters were identified using the least-squares method. A more comprehensive description of FEM can be found in Appendix A.

$$\theta ={cos}^{-1}\left(\frac{\overrightarrow{a}·\overrightarrow{b}}{\left|\overrightarrow{a}\right|\left|\overrightarrow{b}\right|}\right)$$

(4)

2.2. Finger Joint Angle Measurement Using Leap Motion Controller

This subsection describes the performance of LMC and a regression model to correct joint angles determined using LMC.

2.2.1. Leap Motion Controller

Leap Motion Controller (Ultraleap, Bristol, UK) is a hand motion capture system that can recognize positions of hand joints within a maximum range of 60–80 cm from the top of the device with a 140 × 120-degree field of view [31]. LMC is a marker-less motion capture device that uses reflections from three infrared LEDs on either side of and between the two infrared cameras to estimate the position of the fingers. The device weighs 32 g and measures 30 mm × 11.30 mm × 80 mm [32]. LMC uses a proprietary algorithm to extract hand data from raw sensor data. These data consist of infrared brightness values and calibration data to remove distortions [25].

2.2.2. Joint Angle Measurement

In this study, a mannequin hand was used instead of a human hand. The mannequin hand is easy to modify, and it can keep gestures without involuntary motions, making it suitable for experiments in which the same measurement is repeated. Joint angles were measured using two methods: LMC and a camera-based 2D marker detection system. The values from the 2D marker detection system were similar to those measured using a goniometer (Appendix B), so these values were used as the reference values of joint angles in this study. Figure 2a shows the setup for the joint angle measurement. For the ease of 2D marker attachment, the index finger of the right hand was used for the measurement. The 2D markers were attached to the side of the index finger of the mannequin hand, and the distance between LMC and the mannequin hand was set at 15 cm [33]. The frictional resistance of each joint of the index finger of the mannequin hand was negligible.

The 2D marker detection system provides the two-dimensional position coordinates of the marker’s center, while LMC provides the three-dimensional position coordinates of the finger joints. Thus, the position coordinates of points W to T shown in Figure 2b were obtained, and the angles of the distal interphalangeal (DIP) joint, proximal interphalangeal (PIP) joint, and MCP joint were calculated using the following set of equations [26]:

$${\theta }_{MCP}={\cos }^{-1}\left(\frac{\overrightarrow{WM}·\overrightarrow{MP}}{‖\overrightarrow{WM}‖·‖\overrightarrow{MP}‖}\right)$$

(5)

$${\theta }_{PIP}={\cos }^{-1}\left(\frac{\overrightarrow{MP}·\overrightarrow{PD}}{‖\overrightarrow{MP}‖·‖\overrightarrow{PD}‖}\right)$$

(6)

$${\theta }_{DIP}={\cos }^{-1}\left(\frac{\overrightarrow{PD}·\overrightarrow{DT}}{‖\overrightarrow{PD}‖·‖\overrightarrow{DT}‖}\right)$$

(7)

The angle obtained via the method using the two-dimensional marker is denoted as 2D Angle, and the angle obtained by the method using LMC is denoted as the LMC Angle.

2.2.3. Joint Angle Correction Model

Ganguly et al. conducted a comparative experiment using LMC and a gold-standard marker-based motion capture system, to measure joint angles during finger flexion and extension movements [26]. The results showed that LMC could not replace the gold-standard system regarding angle measurement. Therefore, the output angles need to be corrected to estimate finger joint stiffness using LMC. Li et al. derived a regression model for each index finger joint to improve the accuracy of joint angle measurement [27]. Although there was a certain amount of error after joint angle correction, the overall error was improved to within 10 degrees.

In this study, a regression model is generated for taking into consideration the influence of joint modular soft actuators to multiple joints of fingers. Therefore, unlike the previous study, the regression model in this study needs the following explanatory variables.

• Angles of other joints:
  As shown in Appendix C, the LMC Angle of one joint is affected by the interaction between joints of the same finger. Therefore, to estimate the angle of one joint, it is necessary to provide its current LMC Angle value, but also the angle values of other joints in the same finger.
• Dummy variables indicating whether one joint modular soft actuator is pressurized or not:
  When joint modular soft actuators are attached to the joints of one finger, the tendency of angle change of each joint differs depending on whether each soft actuator is pressurized or not (Appendix C). Therefore, if a regression model is generated from all the data regarding the angular changes of all joints in (a)–(g) in Figure A2, corresponding to all combinations of the Boolean pressurization (1 = pressurized, 0 = not pressurized) variables, $pDIP$, $pPIP$ and $pMCP$, except the case where $pDIP$ = 0, $pPIP$ = 0, and $pMCP$ = 0, the goodness of fit is likely low. Therefore, by introducing the Boolean pressurization variables, a single model can be fitted to reflect the different angular change tendencies caused by different pressurization combinations.
• Terms reflecting the interaction between variables:
  Since it is impossible to guarantee that there is no interaction between all the explanatory variables, adding interaction terms may improve the correction performance.

Based on the above points, the regression models were generated to correct the LMC Angles of the three joints, respectively, using the measurement data in Appendix C. LMC Angles of the three joints of the same finger ($DIP,PIP,$ and $MCP$) and the dummy pressurization variables were used as explanatory variables ($pDIP,pPIP,$ and $pMCP$) and their interaction terms were also added as explanatory variables (

Table 1. Candidate explanatory variables for correction models of LMC Angle with the soft actuator.

Candidate Explanatory Variables	Number
1. LMC Angle ($DIP,PIP,MCP$)	3
2. Dummy variable indicating whether the joint is pressurized ($pDIP,pPIP,pMCP$)	3
3. Interaction terms of 1 and 2	15
4. Three joint interaction terms ($DIP·PIP·MCP,pDIP·pPIP·pMCP$)	2
Total	23

).

However, explanatory variables that do not contribute to the correction of joint angles may be included. The exclusion of variables with low contribution would improve the correction of joint angles. Therefore, a variable selection was performed to the 23 candidate explanatory variables shown in Table 1 to obtain the final explanatory variables.

2.2.4. Variable Selection

In order to effectively select the explanatory variables closely associated with the target variable, the forward–backward stepwise selection, one of the most used stepwise selections, was used. The forward–backward stepwise selection was created based on [34]. The procedure is described below.

• Initial variable selection:
  Among the given candidate explanatory variables, the variable with the largest single correlation coefficient with the target variable and whose regression contribution rate (regression variance divided by residual variance) is larger than the predetermined constant $F_{IN}$ is the initial variable.
• Adding a variable:
  By creating a regression model with the current explanatory variables and adding a new variable from the candidate variables, the variable is included as an explanatory variable if it has the largest increase in the contribution rate and the increase in contribution rate is larger than or equal to $F_{IN}$.
• Removing a variable:
  A regression model is created with the current explanatory variables, $t^2$ is determined to test the significance of each partial regression coefficient, and the relevant variable is removed if its minimum is smaller than the predetermined constant $F_{OUT}$.
• End of variable selection:
  Steps 2 and 3 are repeated in the same manner, and the calculation is terminated when there are no more variables to be removed because they are smaller than $F_{OUT}$ or added because they are larger than $F_{IN}$.

When applying this method, the addition/removal criteria for variables depend on the values of $F_{IN}$ and $F_{OUT}$, so how to determine these values is very important. In Ref. [34], $F_{IN}=F_{OUT}=2.0$ is considered appropriate, and this value is also used in this study.

2.3. Finger Joint Stiffness Estimation

This subsection describes how the performance of the proposed stiffness estimation method is verified.

2.3.1. Analytical Model for Joint Stiffness Estimation

Since stroke patients experience strong resistance to extension due to excessive tension in the finger flexors, only the joint stiffness upon extending the fingers was targeted in this study [17]. Using the correction parameters identified in the previous section and [17], the analytical model for the finger joint stiffness estimation using the Modular-SECA is as follows:

$$k=\frac{2\left({\alpha }_A{W}_A+{\alpha }_L{W}_L-{\alpha }_{V}P∆V\right)}{{\left(\theta -{\theta }_0\right)}^2},\theta \in \left[0,0.7{\theta }_0\right)$$

(8)

$k,\theta$ and ${\theta }_0$ are the joint stiffness, the joint angle, and the resting angle, respectively. In this study, the range of possible joint angles was set to $\theta \in \left[0,0.7{\theta }_0\right)$ as in the method using the SECA because we wanted to estimate the stiffness of the joint when it was extended from its resting angle.

2.3.2. Joint Stiffness Estimation Experiment

Two MCP stiffness estimation experiments were conducted. First, stiff joints were made for the mannequin hand using torsion springs, and the stiffness estimated using the proposed methods was compared with the spring constant to decide the estimation accuracy. Next, the feasibility of the proposed method for joints of multiple fingers was verified by estimating the stiffness of a subject’s MCP joints.

In the stiffness estimation experiment, the angles of MCP were measured when air pressure was applied from 0 kPa to 80 kPa at 10 kPa intervals. After the measurements, the stiffness values for each pressure were obtained using Equation (8) in the range $\theta \in \left[0,0.7{\theta }_0\right)$. Finally, these stiffness values were averaged to obtain the final estimated stiffness value. The resting angle, ${\theta }_0$, was measured using the 2D marker detection system or a goniometer.

Joint Stiffness Estimation Using Mannequin Hand

Experiments using a mannequin hand require a target stiffness value for estimation. The modified Ashworth scale (MAS), an assessment used in rehabilitation, is shown in

Table 2. Modified Ashworth scale for grading spasticity [36].

0	:	no increase in muscle tone
1	:	slight increase in muscle tone, manifested by a catch and release or by minimal resistance at the end of the range of motion when the affected part/s is/are moved in flexion or extension
1+	:	slight increase in muscle tone, manifested by a catch, followed by minimal resistance throughout the remainder (less than half) of the ROM
2	:	more marked increase in muscle tone through most of the ROM, but affected part(s) easily moved
3	:	considerable increase in muscle tone, passive movement is difficult
4	:	affected part/s is/are rigid in flexion or extension

. Based on the MAS scores of healthy subjects and stroke patients and the measured joint stiffness values [17,18,35], the relationship of the patient’s joint condition to the MAS score and the corresponding reference stiffness values were defined in two levels (low and medium) (

Table 3. Spasticity and spring constant of the torsion spring used in the two levels of stiffness values.

Stiffness Level	MAS Score	Joint Condition	Reference Stiffness (N mm/deg)	Spring Constant (N mm/deg)
Low Stiffness	0	Healthy	0.52	0.552
Middle Stiffness	1+	Slight spasticity	1.40	1.686

). The torsion spring (SAMINI CO., LTD., Shizuoka, Japan) with a spring constant equivalent to the stiffness value of MCP of the index finger of the mannequin hand (Figure 3) was used.

Stiffness values were estimated using different combinations of analytical models (the SECA model and Modular-SECA model) and joint angles (2D Angle, original LMC Angle, and corrected LMC Angle), comprising six combinations overall.

Estimation of Stiffness of Joints of Multiple Fingers

The feasibility of estimating the stiffness of joints of multiple fingers using the proposed method was verified; two different states of joint stiffness were setup for this purpose: healthy and simulated-stiff.

The stiffnesses of MCP in multiple fingers (index, middle, ring, and little fingers) of a healthy subject’s right hand were estimated using the proposed method (Figure 4a). Simulated-stiffness joints with a higher apparent stiffness value were prepared by bandaging the healthy joints (Figure 4b). Each of the four fingers was bandaged to increase stiffness in the extension direction. For each state of stiffness, estimation was performed three times. The resting angle, ${\theta }_0$, was measured using a goniometer. The thumb was fixed with a fixture because thumb movement could have affected the measurement data, and the distance between the hand and LMC was set to 15 cm.

It should be noted that, in the experiment, when estimating the MCP stiffness of the index finger, both the original LMC Angle and corrected LMC Angle were used, though for the middle, ring, and little fingers, their MCP stiffness values were estimated using only the original LMC Angle. This is because that, the regression model for the corrected LMC Angle of the index finger could be built by using the reference angle values obtained from the 2D marker measurement system; however, due to the difficulty to attach 2D markers to the other fingers, it was difficult to derive an angle correction model for them.

Certainly, one option might be making use of the angle correction model of the index finger for the other fingers. Actually, as shown in Appendix D, for the middle and ring fingers, the PIP and MCP joints had a good fit (>0.7) with the corrected LMC Angle from the index finger, but the DIP joint had a poor fit. Nevertheless, the angle correction model for the MCP joint of the index finger also includes the DIP angle as an explanatory variable; thus, the angle correction model for the index finger cannot be used for the other fingers.

3. Results

3.1. Analytical Model of the Modular-SECA

The correction parameters identified were ${\alpha }_A=1,{\alpha }_L=0.5,$ and ${\alpha }_V=1.15$. Substituting these parameters into Equation (2), the analytical model for the Modular-SECA is as follows:

$$P=\frac{W_A+0.5W_L}{1.15∆V}$$

(9)

Compared to the SECA analytical model, the coefficient of the increase in chamber volume relative to the air pressure input increased, and that of the bending strain energy stored in the torque compensation layer decreased in the actuator’s energy distribution. Figure 5 shows a graph comparing the actuator-only bending angles in free space. No difference is seen between the results of the analytical model, experiment, and FEM simulation for the Modula-SECA. However, there is a difference between the results of the SECA analytical model and those of the Modular-SECA.

3.2. Joint Angle Correction Model with Actuator

Regression models created for each of the DIP, PIP, and MCP joints to correct the joint angles with soft actuators were as follows:

$$\begin{array}{l}DIP=-6.77D+13.71P-12.27M+92.36pD+65.56pP-40pM-0.01P·M\\ \hspace{1em}\hspace{1em}-78.93pD·pP-53.97pP·pM-69.07pD·pM+69.86pD·pP·pM\\ \hspace{1em}\hspace{1em}+6.9D·pM+P·\left(4.66pD-0.56pP-15.74pM\right)-M·\left(4.79pD-16.41pM\right)\end{array}$$

(10)

$$\begin{array}{l}PIP=2.55D+1.69M-3.32pP-0.04D·P-0.08D·M-0.001D·P·M\\ \hspace{1em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}-1.6pD·pP+4.54pD·pM+4.49pD·pP·pM\\ \hspace{1em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}+D·\left(1.49pP-1.2pM\right)-M·\left(0.25pD+0.29pM\right)\end{array}$$

(11)

$$\begin{array}{l}MCP=1.04D+0.62M+12.01pP-0.88pM-0.01D·P\\ \hspace{1em}\hspace{1em}\hspace{0.17em}\hspace{0.17em}+1.35pD·pP-0.67pD·pM-3.04pP·pM-1.27pD·pP·pM\\ \hspace{1em}\hspace{1em}\hspace{0.17em}\hspace{0.17em}-D·\left(1.1pP-0.92pM\right)-P·\left(0.12pD-0.8pP\right)-M·\left(1.07pP-0.42pM\right)\end{array}$$

(12)

$DIP\left(D\right),PIP\left(P\right)$ and $MCP\left(M\right)$ are LMC Angles, and $pDIP\left(pD\right),pPIP\left(pP\right)$ and $pMCP\left(pM\right)$ are dummy Boolean variables indicating whether the soft actuator corresponding to each joint was pressurized or not. The adjusted coefficients of determination were 0.895 for the DIP joint, 0.992 for the PIP joint, and 0.995 for the MCP joint. The PIP and MCP joints were above 0.9, and the DIP joint was slightly below 0.9. The model consisted of around 15 variables, resulting from the variable selection process. As shown, the models for each joint contain the LMC Angle of other joints.

3.3. Joint Stiffness Estimation Using Mannequin Hand

3.3.1. Regression Model Joint Angle Correction Performance

The 2D angle, original LMC Angle, and corrected LMC Angle (from the proposed model) of the three joints (DIP, PIP, and MCP) under each pressure from 0 kPa to 80 kPa during the stiffness estimation are shown in Figure 6. To compare the difference in correction performance between the proposed regression model and the regression model derived from the method of the previous study [27], third-order regression models were generated using only the angle of the target joint to be corrected as the explanatory variable (Equations (13)–(15)). The corrected LMC Angle (from the previous study) is also shown in Figure 6.

$$DIP=0.006DI{P}^3-0.32DI{P}^2+5.95DIP-24.96$$

(13)

$$PIP=0.0002PI{P}^3-0.02PI{P}^2+1.37PIP+13.95$$

(14)

$$MCP=0.002MC{P}^3-0.16MC{P}^2+5.22MCP-22.71$$

(15)

The correction performance of each regression model was evaluated in terms of the root-mean-square error (RMSE) values between the 2D Angle and LMC Angle in Figure 6 (

Table 4. Prediction accuracy of LMC Angle correction model with the actuator (RMSE). Two values are underlined where the RMSE of the corrected LMC angle from proposed method is larger than the RMSE of the original LMC angle. Among them, the one whose RMSE is larger than the RMSE of the corrected LMC angle from previous study is double underlined.

Stiffness Level	Joint	Original LMC Angle (deg)	Corrected LMC Angle from Proposed Model (deg)	Corrected LMC Angle from Previous Study (deg)
Low Stiffness	DIP	15.31	5.51	10.22
	PIP	4.98	4.97	10.94
	MCP	16.39	5.07	10.93
Middle Stiffness	DIP	11.74	14.67	8.80
	PIP	8.34	14.80	23.33
	MCP	23.55	8.83	19.66

). As shown, except the DIP measurement on the middle stiffness level (double underlined in Table 4), the corrected LMC Angle under the proposed model improved the prediction accuracy compared to the corrected LMC Angle from the previous study. The model in the previous study did not include soft actuator pressure information as an explanatory variable, so the model had to be changed when the combination of joints to be pressured changed. On the other hand, compared with original LMC Angle, the proposed model showed improvement, except for the two measurements underlined in Table 4.

3.3.2. Joint Stiffness Estimation Accuracy

The results of the estimated MCP joint stiffness of the index finger of the mannequin hand are shown in

Table 5. Results of stiffness estimation experiments using the mannequin hand. The smallest error among the six methods is colored red.

Stiffness Level	Target Value (N mm/deg)	Estimated Value (N mm/deg) (Error = Estimated Value − Target Value)
		Method 1	Method 2	Method 3	Method 4	Method 5	Method 6
		SECA Model	Modular-SECA Model	SECA Model	Modular-SECA Model	SECA Model	Modular-SECA Model
		2D Angle	2D Angle	Original LMC Angle	Original LMC Angle	Corrected LMC Angle	Corrected LMC Angle
Low Stiffness	0.552	2.17 (+1.62)	0.87 (+0.32)	−0.13 (−0.68)	−0.31 (−0.86)	0.82 (+0.27)	0.04 (−0.52)
Middle Stiffness	1.686	4.98 (+3.30)	3.27 (+1.58)	−0.09 (−1.78)	−0.21 (−1.89)	1.06 (−0.63)	0.38 (−1.30)

. Comparing Method 1 (the SECA model with 2D Angle measurement) and Method 2 (the Modular-SECA model with 2D Angle measurement), it is clear that the corrections to the analytical model improved the accuracy of the stiffness estimation. Comparing Method 3 (the SECA model with the original LMC Angle measurement) and Method 5 (the SECA model with the corrected LMC Angle measurement), and Method 4 (the Modular-SECA Model with the original LMC Angle measurement) and 6 (the Modular-SECA model with the corrected LMC Angle measurement), it seems obvious that the correction of the LMC Angle improves the accuracy of the stiffness estimation. However, the comparisons between Methods 3 and 4, and Methods 5 and 6 disclose that the accuracy of stiffness estimation deteriorated when the LMC Angle was used due to the correction of the analytical model. Figure 7 shows the change in the stiffness value estimated by the joint angle at a 40 kPa input air pressure in the low stiffness experiment. Even though the accuracy of the analytical model for the Modular-SECA is better than that of the SECA model for the reference value of the joint angle (2D Angle), a deviation of the joint angle estimation produces better performance for the analytical model of the SECA model instead of Modular-SECA, which deviates from our expectation. Therefore, despite the improvement made in both the analytic model and LMC Angle measurement, both of them need further enhancement.

3.4. Multiple Finger Joint Stiffness Estimation

First,

Table 6. The joint angles of the subject’s index finger during rest and with the actuator attached.

	Resting Angle (deg)		Joint Angle at 0 kPa (deg)
	Goniometer Value	LMC Angle	LMC Angle
Healthy Joint	44	21.76	−15.03
Stiff Joint	27	21.06	−12.55

shows that the joint angle during rest was larger in the healthy joint. On the other hand, the joint angle when the soft actuator was attached and not pneumatically pressurized showed a flexion angle, while the original LMC Angle showed an extension angle. Figure A3 shows that when the LMC Angle showed an extension angle in the MCP joint, the smaller the flexion angle measured with the goniometer was, the larger the LMC Angle extension angle was. Therefore, the flexion angle at 0 kPa was larger for the stiff joint. In other words, the stiff joint had stronger resistance in the extension direction, and the amount of angle change caused by the soft actuator was smaller. Therefore, the stiffness value of the stiff joint was higher than that of the healthy joint, and bandaging was an appropriate method to increase the apparent stiffness value.

The mean and standard deviation of three measurements of the stiffness values estimated for the MCP joints of the fingers are shown in

Table 7. Results of an experiment estimating the stiffness of the MCP joint in multiple fingers of a subject using the analytical model of the Modular-SECA.

		Estimated Value (N mm/deg)
		Index	Middle	Ring	Little
Healthy Joint	Original LMC Angle	$0.26\pm 0.02$	$0.20\pm 0.01$	$0.33\pm 0.03$	$0.26\pm 0.02$
	Corrected LMC Angle	$-0.09\pm 0.02$	-	-	-
Stiff Joint	Original LMC Angle	$-0.36\pm 0.19$	$0.01\pm 0.08$	$0.34\pm 0.14$	$1.13\pm 0.02$
	Corrected LMC Angle	$-0.17\pm 0.12$	-	-	-

.

For the little finger, the stiffness estimates for stiff joints were higher than those for healthy joints. Though, for the ring finger, it was difficult to distinguish between healthy and stiff joints from the estimates. For the MCP joint of the index finger and middle finger, this was even reversed.

Another issue is that of negative estimates of stiffness. As can be seen in Figure 7, the accuracy of the joint angles was low because the estimated values were negative depending on the joint angle. The angle correction model for the index finger was generated using joint angle data obtained from a mannequin hand. Its applicability to the human hand was low, and the lack of LMC Angle correction may have resulted in negative values.

Moreover, because the fingers were separately restrained by bandaging (Figure 4b), the strength of the restraint was different for each finger, causing a large difference in the apparent stiffness for each finger. These issues revealed that stiffness estimation for a human finger is not so reliable, though the possibility for simultaneous joints of multiple fingers was shown.

4. Discussion

The results show that the SECA and Modular-SECA have different bending performances, and the interaction of joints affected the output of LMC when the Modular-SECA was attached. Furthermore, it was confirmed that the accuracy of the stiffness estimation with the Modular-SECA could be improved by correcting the analytical model of stiffness estimation and the joint angles.

4.1. Analytical Model of Modular-SECA

The free space bending analysis results suggest a difference in the energy distribution in the actuator between the Modular-SECA and SECA. It was confirmed that the chamber volume increase, $∆V$, relative to the air pressure input was larger for the Modular-SECA than that for the SECA. The bending strain energy, $W_L$, stored in the torque compensation layer was smaller for the Modular-SECA. This may have been due to the rigid connectors attached to both ends of the Modular-SECA, which reduced the energy allocated to $W_L$ within the energy conservation law. In previous studies, $W_L$ was modeled as a cantilever beam subjected to a pure bending moment [17]. However, the Modular-SECA is considered to be subjected to an additional bending moment in the torque compensation layer. Therefore, at the same air pressure input, the energy required to bend the torque compensating layer is reduced, and the volume increase in the chamber is considered to have increased.

From Figure 8, the difference between the 2D Angle and the bending angle obtained from the Modular-SECA analytical model is smaller than that in the SECA analytical model results. However, the estimated values for Method 2 (Modular-SECA Model with 2D Angle measurement) in Table 5 did not reach the target values. This may be due to the lack of correction in the Modular-SECA analytical model. In particular, the difference between the experimental and Modular-SECA model bending angles at 0 kPa suggests that the correction may be insufficient. This difference is thought to have persisted afterward, preventing the estimation results of Method 2 from reaching the target values. A feature of the SECA is that the torque compensation layer provides a supplementary bending moment to return the finger to its initial position under reduced pressure. Figure 5 shows that the bending angle of Modular-SECA is larger than that of the SECA under applied pressure and that the force in the bending direction is greater. However, under reduced pressure, the auxiliary bending moment provided by the torque-compensating layer of the Modular-SECA is smaller than that of the SECA, and the force in the extension direction may be smaller. ${\left(\theta -{\theta }_0\right)}^2$ in Equation (8) is a term that expresses the effect of the bending moment of the torque compensation layer on the bending angle. If the bending moment provided by the torque-compensation layer of the Modular-SECA is small, a correction parameter may also need to be provided for this term.

4.2. Finger Joint Angle Measurement Using Leap Motion Controller

First, Appendix E shows that the LMC Angle change trend changes with the attachment of the soft actuator. In Figure A2a,d, the trend of the LMC Angle change around 80 kPa is altered compared to that at the previous pressure point. Briefly, 80 kPa is when the mannequin hand’s index finger is bent to the point where the soft actuator is visible from LMC’s camera. Therefore, LMC recognized the soft actuator as a part of the finger, and the output angle changed.

When the soft actuator was applied, the angle correction model included the angle of other joints, suggesting that LMC Angle is affected by the interaction between joints within the same finger. Table 4 shows that the correction performance of the angle correction model is high for joints with large angle changes when the soft actuator is pressurized. However, it may be low for joints that are not pressurized and have small angle changes. This may be due to biased training data for the joints where the soft actuators are not pressurized, resulting in lower model prediction accuracy. It also suggests that including the angles of other joints in the explanatory variables may lead to an increase in correction accuracy but may also lead to a decrease. Correction performance varies with stiffness level in Table 4 because correction accuracy varied with the angles of other joints. Although the model of the previous study can improve correction performance by increasing the order of the regression model, it requires separate models for each combination of joints to which the soft actuator applies pressure.

Figure 6 and Figure 8 show that the upper limit of the stiffness estimation range using the LMC Angle is wider than that using the 2D Angle because the LMC Angle is output smaller than the 2D Angle even after correction. In Figure 8, the stiffness estimation range for 2D Angle is 0 kPa to 50 kPa, while corrected LMC Angle is 0 kPa to 60 kPa; the stiffness estimation using the LMC Angle includes a greater stiffness value at each pressure in the final estimate stiffness value than that using the 2D Angle. Therefore, to improve the estimation performance, it is considered effective not only to improve the correction accuracy of the LMC Angle but also to change the stiffness estimation range.

When the Modular-SECA was attached to a human hand, MCP joints were constantly flexed during the experiment, while the LMC Angle showed an extension angle. Since none of the LMC Angle of the MCPs showed an extension angle in the data when the regression model was created, the correction performance of the regression model at the extension angle was insufficient, leading to the under-correction of the LMC Angle.

Since the angle correction model for the index finger could not be applied to the middle and ring fingers, it is necessary to derive an angle correction model for each finger separately. However, since it is difficult to measure joint angles using 2D markers for the middle and ring fingers, it is not possible to derive an angle correction model using the same method as in this study. Therefore, it is thought that the angle correction model for the index finger can be applied by excluding from the candidate explanatory variables for the regression model for the index finger those variables that include the angle of the DIP joint, which has a low degree of fit with that of the index finger.

4.3. Proposed Method for Finger Joint Stiffness Estimation

The stiffness estimation using LMC and the Modular-SECA showed that the correction of the analytical model and joint angle improves the stiffness estimation results. However, the current estimation performance of the proposed method is insufficient due to the lack of both corrections, and the correction method needs to be improved. The final stiffness estimate is the average for each pressure in the stiffness estimation range, but variations exist among the estimates for each pressure. Therefore, the performance of the proposed method may be improved by weighting the more reliable estimates in the stiffness estimation range.

4.4. Contribution and Limitations

By using joint modular soft actuators, high bending performance can be achieved with minimal energy for telerehabilitation, and individual differences such as different finger sizes and different joint stiffness values can be accommodated with minimal effort.

Our contribution includes the following:

• To the best of our knowledge, for the first time, finger joint stiffness estimation for using joint modular soft actuators was proposed and implemented. The accuracy of stiffness estimation needs further improvement, though the possibility has been shown and the direction for improvement has been made clear. In addition, the proposed stiffness estimation method enables the simultaneous measurement of multiple fingers, although the analytical model and the correction method for joint angles need to be improved.
• The analytic model was updated from a previous model for the SECA, a whole-finger soft actuator, with additional parameters for accounting for the features of the joint modular soft actuator. The model could be regressed from experimental data and give a better prediction of the behavior of the joint modular actuators.
• Leap Motion Controller was used for the first time for the angle measurement of joints of fingers supported by the joint modular soft actuators. A regression model was proposed to correct measurement results, taking into consideration multiple joint interaction intensified by wearing the soft actuators, and the influence of the actuation of the joint modular soft actuators. The effectiveness of the correction was validated by the experiment results. LMC can be used to measure the angles of finger joints not only when soft actuators are attached, regardless of whether they are actuated or not, but also when soft actuators are not attached.

On the other hand, there are several methodological and implementational limitations:

• The high stiffness of joint cannot be estimated with the Modular-SECA. The reference value for high stiffness (a MAS (modified Ashworth scale) score of about 3) is 9.60 N mm/deg. Because the extension of the highly stiff finger caused by wearing the Modular-SECA was small compared with that of the SECA, the joint angle, $\theta$, was not within the stiffness estimation range, $\theta \in [0,0.7{\theta }_0)$. To estimate high stiffness as well, it is necessary to design a joint modular actuator that can extend the finger sufficiently with its intrinsic stiffness, or add another component to actively extend the joint extension.
• Only the stiffness estimation of MCP joints of multiple fingers was tested. The stiffness estimation of DIP and PIP joints of multiple fingers needs to be tested in future.
• The regression model used to correct the LMC Angle generated using data acquired from mannequin hands did not apply to human hands. Although mannequin hands are effective for multiple measurements due to their low data variability, they are unsuitable for model generation because they do not reproduce human finger movements such as intra-finger constraint. Therefore, if an angle correction model is generated using data acquired from a human hand, it may apply to both mannequin hands and human hands. Moreover, this may allow the generated regression model to be adaptable to the individual difference of hand dimensions.
• The manually controlled input air pressure affected the human hand’s stiffness estimation. The speed at which the joint moves, which is affected by the input air pressure, is a factor that influences joint stiffness [37,38]. In the case of manual control, the rotational speed of the joints differed from measurement to measurement, which may have increased the variation in data. Thus, automatic pneumatic input control is necessary to realize the proposed method.
• In this study, we did not test stiffness estimation in a telerehabilitation setting. In the near future, it is necessary to verify the effectiveness of the proposed method in such a setting.

5. Conclusions

In this study, we proposed a method combining the Modular-SECA and LMC for the stiffness estimation of finger joints in a telerehabilitation environment. The SECA analytical model for stiffness estimation was modified to be applied to the Modular-SECA, and the LMC Angle was improved by the regression models to improve the accuracy of the proposed method. It was also shown that using the angle information of other joints and the dummy variables indicating whether or not the soft actuator is pressurized is effective for the correction model of the LMC Angle with the soft actuator. Furthermore, the results suggest that the proposed method could be used in a telerehabilitation environment to evaluate the spasticity condition in case a more precise correction of the analytical model and the LMC Angle becomes available. Since the simultaneous estimation of multiple fingers’ MCP joints is feasible using the proposed method in this study, it is necessary to make it possible to estimate DIP and PIP joints and joints with high stiffness values. Moreover, in the near future, it is necessary to verify the effectiveness of the proposed method in a telerehabilitation setting.