On the Sensing and Decoding of Phantom Motions for Control of the Cybernetics of the Upper-Limb Prosthesis ^†

Abstract

The cybernetic interface within an upper-limb prosthesis facilitates a Human–Machine interaction and ultimately control of the prosthesis limb. A coherent flow between the phantom motion and subsequent actuation of the prosthesis limb to produce the desired gesture hinges heavily upon the physiological sensing source and its ability to acquire quality signals, alongside appropriate decoding of these intent signals with the aid of appropriate signal processing algorithms. In this paper, we discuss the sensing and signal processing aspects of the overall prosthesis control cybernetics, with emphasis on transradial, transhumeral, and shoulder disarticulate amputations, which represent considerable upper-limb amputees typically encountered within the population.

1. Introduction

An amputation involves the surgical removal of a candidate limb from the body of a human being. This can be for a variety of reasons such as accidents, vascular diseases, leprosy, tumours, and snakebites [1]. In some developing countries, causes for amputations also include the effects of war, such as fighting injuries and landmines [1]. These effects have made for a considerably heightened level of amputation statistics over the past 20 years, as reported by Staats [1], which includes the following: 200,000—Vietnam, 36,000—Cambodia, 15,000—Angola, 8000—Mozambique and 5000—Uganda.

It is unanimously acknowledged that the loss of an upper limb can drastically affect the quality of life led by an individual, their level of independence, and, in extreme cases, it has led to suicide [2].

Means of compensation for the loss of an upper limb involve the use of an upper-limb prosthesis, of which the most advanced is the bionic upper-limb prosthesis, also known as a myoelectric prosthesis [2]. The efficient operability of the prosthesis arm hinges on the effective signal acquisition and accompanying decoding of the acquired signal. This refers to the sensing and signal processing as part of the cybernetic interface of the prosthesis of the bionic limb to facilitate effective collaboration between an amputee and an augmented body part [3]. This will be discussed in greater depth in the manuscript with respect to the challenge faced with specific kinds of amputations.

Although comprehensive statistics on amputations have been challenging to acquire and collate, combined statistics presented by Cordella et al. [4] have provided insights to suggest that transhumeral and transradial amputees account for the largest cohort of amputees (with considerable amputations in the UK and Italy), thereby necessitating emphasis in terms of research and technological innovations. From the viewpoint of a technological challenge, background surveys appear to suggest that there exists very sparse work done on the design of bionic upper-limb prosthesis for shoulder disarticulation amputees [5]. With this in mind, the discussion in this paper will be centered around the sensing and signal processing as part of the cybernetic interface within a bionic limb, applied to (i) transradial prosthesis, (ii) transhumeral prosthesis and (iii) shoulder disarticulate prosthesis. This is followed by suggestions on future work which, when tackled effectively, can make for enhanced prosthesis control interfaces with the potential to be accessible at a lower cost.

2. Prosthesis Cybernetic Interface

The pattern recognition-based control system is the favoured prosthesis control approach used in the cybernetic interface for the purpose of control of bionic prosthesis limbs [6]. The review work by Nsugbe et al. [3] described the various components associated with the pattern recognition-based control system. Breaking this down into a forward and feedback path, the forward path comprises the signal acquisition primarily from the residual anatomy of the amputees, feature extraction and machine learning as part of the phantom motion intent decoding stage, and cues in the actuation function selection and motor driving of the prosthesis limb [3]. As soon as the motion intent signal is acquired, the forward stage of the control process can be said to be a fully automated stage, which is facilitated by machinery intelligence [3]. The feedback path is the stage where the human intervention comes into the loop, first by playing the role of a visual observer, while the autonomic nervous system serves as the intelligent controller, which orchestrates the adjustment of the contraction dynamics and phantom motions, etc., in a desired and iterative manner until a satisfactory outcome has been reached [7]. The joint contribution of both the amputee and machine intelligence showcase that the bionic prosthesis control problem is one that can be viewed as a Human–Machine collaboration in order to achieve the desired functionality. Work on gesture recognition by Nsugbe et al. [2] highlighted this with a diagrammatic flow displayed in Figure 1, which shows the interface between the sensing, signal processing and classifier (denoted as dark blue blocks) and the mechanical components, such as the actuators and servos (denoted as transparent blocks). In this scenario, the Human contribution has been denoted as a biofeedback element.

It should be noted that the discussed prosthesis architecture is in relation to phantom motion sensing and gesture recognition and does not take into account continuous contraction force estimation methods, which would typically be expected to work in parallel with the pattern recognition control scheme, and can be done using regression methods as seen in other areas [8,9,10,11].

As part of the control process, sensors are used in the acquisition of anatomical signals, which tend to be from a bioelectrical source, since the established means of anatomical signal acquisition is via the use of electromyography (EMG) sensing [12]. A discrete representation of an EMG signal can be seen as follows:

$$x\left(n\right)={\displaystyle \sum }_{r=1}^{N}h\left(r\right)e\left(n-r\right)+w\left(n\right)$$

where $x\left(n\right)$ is an EMG signal, $e\left(n\right)$ is a discrete point being processed, $h\left(r\right)$ is the firing impulse of an action potential, $w\left(n\right)$ is an additive white noise, and $N$ represents the number of firing motor neurons.

A host of other sensing modalities have been investigated for the use of prosthesis, such as electroencephalography (EEG), near-infrared (NIR), as shown in Figure 1, and mechanomyography, to name a few. EMG continues to be the most widely used sensing modality for prosthesis control, where studies discussed as part of Section 2.2 involve cases where EMG has been used independently or in combination with an auxiliary sensing modality [3,13,14].

The feature extraction stage involves the extraction of a range of features that can be used to succinctly characterise an acquired neuromuscular time-series. The kinds of features that need to be extracted largely depend on the extent of the amputation, as will be discussed subsequently [3]. This is followed by a classification stage using a trained classifier, where it can be noted that due to regulations around computational time allowable for prosthesis, the predominantly deployed classifier was the discriminant analysis. Nonetheless, other classifiers have been researched and include neural networks and support vector machines, to name a few [3].

2.1. Transradial Pattern Recognition Control Interface

Transradial amputations are relatively minor in terms of major upper-limb amputations. This fact is echoed regarding feature extraction, where linear time-domain features, such as mean absolute value (MAV), slope sign change (SSC), zero crossings (ZC) and root mean squared (RMS), have been seen to be sufficient in effective modelling differentiation and recognition of phantom hand motions [15]. The nature of the acquired physiological signals, and the associated linear modelling methods, has led to this area of amputations receiving a high degree of research emphasis, as evidenced by the pool of literature relating to transradial amputee bionic prosthesis [3,13,15]. As with all amputations, neurological rewiring of the motor cortex begins to occur as time passes since the amputation, without the use of a prosthesis limb [7]. This effect serves as a source of non-linearity as it begins to result in faint contractions emanating from phantom motions, which produce a weak electrophysiological output, and a more stochastic physiological time-series signal. Potential means towards dealing with this could include non-linear and complex features that can capture underlying patterns in stochastic signals.

An image showing a transradial amputee during a data collection process can be seen in Figure 2.

2.2. Transhumeral Pattern Recognition Control Interface

The transhumeral amputation represents a major upper limb, ‘above-elbow’ amputation [2]. The electrophysiological signals acquired from phantom motions of these amputees are mostly from the bicep and triceps, thus limiting the ability to infer finger and fine motor movements from these kinds of amputations [17,18]. Hence, the phantom motions used in the design of the pattern recognition interfaces for these kinds of amputees involve bulk muscular recruitment which—considering the nature of the amputation—make for a stochastic and highly non-linear time-series signal. As a means of feature extraction and signal processing, it has been seen that common linear time-domain features have been proven to be insufficient, and instead require a concatenation of linear, frequency (i.e., cepstrum) and non-linear/complexity features (i.e., sample entropy, fractal features, etc.) [18]. This, in turn, brought a challenge in terms of computation time as the ensemble of features caused a scale-up of the electronic implementation demands. Although Nsugbe et al. [17] implemented a novel and computationally effective time-domain decomposition algorithm, which in a sense ‘linearises’ the signal and allows for reduced feature extraction, the algorithm requires a broader sample set for further validation.

Thus, as a step towards a suitable real-time optimisation, feature selection exercises need to be conducted with a broad set of features to identify the optimal combination for implementation in a prosthesis bionic interface. This feature, coupled with the use of wearable sensors, can potentially allow for highly effective transhumeral bionic prosthesis that would potentially be in an affordable range.

Figure 3 shows an image of a data collection session from a transhumeral amputee comprising EMG and electroencephalography/brainwaves.

2.3. Shoulder Disarticulate Pattern Recognition Control Interface

These amputees represent a combination of ergonomic and technical signal processing challenges in the design of control interfaces for their candidate bionic, due to the extreme nature of their amputations [5]. For example, due to the nature of their amputations it is not possible to track phantom motions; thus, anatomical tissues from other sites along the dorsal are used as surrogate sites to acquire electrophysiological signals from shoulder girdle motions, which in turn are used to actuate various hand gesture motions in a bionic prosthesis limb. In the case of the signal processing, candidate sites used for physiological signal acquisition include the trapezius, pectoral and the latissimus dorsal muscle, which facilitate the compound movement of the shoulder girdle and, in turn, produce a stochastic time-series signal [5,19].

The literature is sparse in the area of prosthesis control interface for shoulder disarticulate amputees. Key work, which included the simultaneous acquisition of EMG and vibration signals from amputees (as shown in in Figure 4), also involved the application of an extended set of features to accurately recognise different variants of shoulder girdle motions across a number of amputees [5,19]. Due to the unnatural association between shoulder girdles and potential hand gestures in a prosthesis limb, cognitive loading is likely to be an issue for the prosthesis users and threatens a potential abandonment of the prosthesis limb, as has been noted previously. Thus, neuromuscular reprogramming therapy, as proposed by Nsugbe and Al-Timemy [5], can be included as part of the process of familiarising the amputee with the use of the technology in order to boost intuitiveness and control of the prosthesis limb, and ultimately minimise cognitive loading [20]. In terms of the ergonomics of sensor placements and the associated electrodes, electrode selection exercises can be conducted via a forward selection algorithm which can help boost parsimony and trim down the number of electrode sites, while increasing classification accuracy. Furthermore, in order to boost sensor robustness, wearable embroidery electrodes can be considered for use as part of the prosthesis control system, which would also contribute towards further affordability of the prosthesis limb [21].

3. Conclusions

In this paper, the control interface for select kinds of upper-limb prosthesis has been discussed. This discussion has been based primarily around the signal processing of the acquired physiological signal as a means towards motion intent decoding. The transradial prosthesis has been viewed as the most researched bionic prosthesis interface and produces a time-series signal which lends itself to feature extraction involving a relatively small subset of time-domain features. The transhumeral prosthesis, which is an extreme above-elbow amputation, produces a stochastic physiological time-series from anatomical tissue, including the bicep and triceps along the humerus, thus warranting an extended feature extraction approach, which involves a concatenation of signal features. In the case of the shoulder disarticulation prosthesis, stochastic time-series signals are acquired from muscles involved in shoulder girdle motions in the absence of a stump due to the nature of the amputation. A proposed list of future work to be done from a technical perspective for the various prosthesis groups, in order to enhance operability and functionality of the prosthesis interface includes: the expansion of the feature sets considered as part of a transradial prosthesis; research on feature selection and optimisation exercises to find the best combination of features that allow for a real-time implementation in a control interface while maximising the phantom recognition capability; and in the case of the shoulder disarticulate prosthesis, the conducting of feature selection and optimisation exercises in order to reduce the amount of sensor channels used in the recognition of shoulder girdle motion, alongside potential feature selection and optimisation exercises.