Pattern Recognition of EMG Signals by Machine Learning for the Control of a Manipulator Robot

Abstract

Human Machine Interfaces (HMI) principles are for the development of interfaces for assistance or support systems in physiotherapy or rehabilitation processes. One of the main problems is the degree of customization when applying some rehabilitation therapy or when adapting an assistance system to the individual characteristics of the users. To solve this inconvenience, it is proposed to implement a database of surface Electromyography (sEMG) of a channel in healthy individuals for pattern recognition through Neural Networks of contraction in the muscular region of the biceps brachii. Each movement is labeled using the One-Hot Encoding technique, which activates a state machine to control the position of an anthropomorphic manipulator robot and validate the response time of the designed HMI. Preliminary results show that the learning curve decreases when customizing the interface. The developed system uses muscle contraction to direct the position of the end effector of a virtual robot. The classification of Electromyography (EMG) signals is obtained to generate trajectories in real time by designing a test platform in LabVIEW.

1. Introduction

A person with a disability is an individual who has one or more physical or mental deficiencies that prevent their full and effective participation in equal conditions when interacting with different social environments. In recent years, the development of HMI for people with motor disabilities has been oriented towards the use of systems based on Electromyography (EMG). In [1], a review of the state of the art in EMG monitoring is presented in terms of applications in rehabilitation and minimally invasive acquisition devices; among the advantages that it highlights are in the fields of physiotherapy and telemedicine. In [2], through three EMG channels, they control the position of a robot with two degrees of freedom; the processing is done as a function of time through the amplitude of the signal when movements are made with the elbow and the shoulder joint. Four channels of surface electromyography acquisition are proposed in [3], where pairs of electrodes are placed according to the position and orientation of the target muscles. Selecting materials with excellent properties for devices on the skin, the fabricated electrodes achieve low skin electrode impedance and record sEMG signals with a high signal-to-noise ratio. In [4], a review on signal acquisition and pattern recognition through Machine Learning is presented. In [5], a myoelectric pattern recognition-driven hand exoskeleton was designed for stroke rehabilitation. It detects and recognizes the intention of movement based on EMG signals, and then the exoskeleton helps the user to perform six types of hand movements in a real way. One of the main challenges in the design of interfaces based on sEMG is the obtention of a signal function or model that allows for the reliable control of a care system. Due to the non-stationary signal behavior, three methods are generally used for sEMG analysis to extract information, which are in the time [6,7], frequency, and time–frequency domain [8]. There are some practical factors, such as the change in arm position, that prevent robust myoelectric control. In [9], an experiment with 14 subjects is carried out to accurately characterize factors that alter the EMG recording. Using regression algorithms, they obtain real-time feedback on changes in the position of the arm and displacement of the electrodes. Pattern recognition has been studied further to develop control algorithms for electric hand prostheses [10,11]. These works have shown excellent accuracy when classifying different types of hand movement (>95% for 10 classes), [12,13,14]. Most of the pattern recognition approaches have the limitation that only one of the functions of the prosthetic hand can be controlled, due to its sequential and binary control. Such control strategies make it impossible to perform natural movements of the hand that consist of the simultaneous activation of different degrees of freedom. Some studies have introduced new pattern recognition schemes that classify combined movements [15,16,17,18,19]. The disadvantage of the new approach is the total number of classes, as it increases drastically when new classes are considered. Recently, regression-based approaches have been on the rise, as they provide control information that allows for multi-degree-of-freedom control. In this work, a regression algorithm using neural networks is proposed to obtain a model through multiclass categorization that allows for the control of a robotic system with three degrees of freedom of the anthropomorphic type. The analysis of a single channel of sEMG that classifies signals with different times of muscle contraction is implemented, with the objective that the robot moves accurately according to predetermined positions in a state machine and demonstrates the correct operation of an HMI by reducing the learning curve. In [20], a study of multichannel electromyography signals is carried out, which is one of the methods used in the recognition of human movement patterns. An exoskeleton robot is controlled and EMG signals are measured during dynamic or isometric muscle contractions. As a result, they developed a pattern recognition model of dynamic and isometric muscle contractions using the Short Time Fourier Transform (STFT).

Section 2 presents the fundamentals of the EMG signal. Section 3 presents the design of the HMI from the acquisition of the EMG signal and its analog and digital processing. The multiclass classification model obtained using neural networks is described. The different classes to be detected, the training algorithm and the operation of a state machine that determines the position of the robot according to the result obtained from the model are shown. The classification to reach the desired position is explained, obtaining the dynamics and inverse kinematics of an anthropomorphic robot with three degrees of freedom. A PD+ control is implemented to apply the necessary torque to each joint of the robot and to validate the operation of the HMI. It designs a graphical user interface in LabVIEW software by interacting a virtual robot and the EMG signal. Section 4 describes the results obtained with the classifier, the experimental tests and the response time for each test.

2. HMI Systems Based on EMG

The neuron is the cellular unit of the central nervous system. It has two properties: (1) Sensory, which gives it the ability to respond to physical and chemical agents with the initiation of a nerve impulse; and (2) Conductivity, which gives it the property of transmitting these impulses from one side to another. The dendrites that originate in the cell body are responsible for receiving impulses from other neurons and sending them to the soma of their own neuron. The axon is an extension from the neuronal soma that conducts the impulse to the muscle; it is surrounded by a myelin sheath that allows for better impulse conductivity. The neuron that originates the EMG biopotential is called a motor neuron, which conducts the impulse through the neuromuscular junction to the muscle fiber, as shown in Figure 1 [21].

EMG is an electrical exploration of the peripheral nerves by the stimulation of the muscles to achieve their contraction. The differential potential in the biceps brachii is measured by placing two silver/silver chloride (Ag/AgCl) electrodes and a reference electrode located at the junction of the forearm and hand, as shown in Figure 2. [20].

When muscle contraction is performed, there are two types: (1) Isometric contraction, which is a static form of exercise in which a muscle contracts to produce force without an appreciable change in muscle length; and (2) Isotonic contraction, which is without appreciable change in the force of contraction. The distance between the origin of the muscle and its insertion becomes smaller. For EMG acquisition study purposes, in the protocol carried out, isometric contractions are recorded by placing a weight in the user’s hand with a value of 5 pounds. This process is carried out in order to avoid the acquisition of noise due to involuntary movements and to keep the arm static while the biceps brachii contraction is performed for short periods of time. This process is carried out in order to avoid the acquisition of noises due to involuntary movements and to keep the arm static while the contraction of the biceps brachii is performed for short periods of recording time, no longer than 45 s, preventing the user from making an unwanted movement due to fatigue. When performing the acquisition, it was observed that the muscle relaxation periods of 5 s made it possible to accurately obtain the muscle contraction times, thus avoiding the introduction of noise due to muscle fatigue. The goal of this work is to demonstrate that, with a correct training of the neural network, adding dynamic muscle contractions due to involuntary movements as an extra class of recognition allows the system to rule out this muscle noise as a motion control command. The EMG signals have amplitudes from 0.1 mV to 5 mV, with a bandwidth of 0 to 5 KHz [21]. With this information, a first acquisition is made using a BIOPAC^® commercial system, which allows for the recording of the differential signal taken from two electrodes and a reference, as indicated in Figure 3a. This system records the waveforms of the EMG signal in order to validate the implemented acquisition protocol. Tests were performed for contraction times of 1, 3 and 5 s. In Figure 3b, the response obtained from the EMG signal to a contraction of 5 s with rest pauses also of 5 s is presented. The inconvenience presented in the acquisition protocol when using this system is that the record is stored in a numerical database and cannot be read directly by any other acquisition card. Real-time implementation of the Fast Fourier Transform (FFT) is necessary to verify the spectrum in frequency and obtain the value of the cutoff frequency for the implementation of filters. The next section presents the instrumentation implemented and the digital processing for the acquisition of the database.

3. Materials and Methods

The amplifier used is the IC AD620 due to its characteristic of a common rejection ratio of 100 dB and the gain adjustment with an external resistor. A circuit with a basal corrector and a Common Mode Rejection (CMRR) configuration connected to the junction of the forearm and hand is implemented as a circuit reference. According to the amplitude and frequency characteristics of the EMG signal, the analog processing stage is designed, which includes amplification, isolation and filtering.

A. 

Amplification with basal corrector

An instrumentation amplifier CI AD620 is implemented as a preamplification system to acquire the differential EMG signal with a gain of 500. A basal correction circuit is conditioned to eliminate the level of direct current (DC) caused by involuntary movements of the user or an incorrect connection of the electrodes. The circuit is a IC TL084 operational amplifier in its integrator configuration that is connected in feedback to the $\mathrm{Ref}$ and ${\mathrm{V}}_{\mathrm{out}}$ outputs of the instrumentation amplifier, as shown in Figure 4, implementing a high pass filter that eliminates the DC bias voltage and preventing op amps from reaching their maximum power limits.

B. 

Analog Filter

To filter the frequency components that are not within the bandwidth of the EMG signal, a range from 0.5 Hz to 5 KHz, a second order bandpass filter in Butterworth configuration with unity gain is designed, with a ratio of 40 dB per decade using high impedance TL084 operational amplifiers, precision resistors and electrolytic capacitors; see Figure 5.

The output of the analog filter stage is connected to the absolute voltage input of a DAQ6009 acquisition card connected via USB port to a laptop, with a sample rate of 10KHz. An acquisition card with a ground plane is designed to decrease inductive noise, as indicated in Figure 6a. Figure 6b shows the response of the acquisition card in the Tektronix^® oscilloscope. Analog noise is observed, which is subsequently eliminated by means of a digital filter.

C. 

Digital Filter

Due to the acquisition system being subject to the interference of electromagnetic noise induced by lamps or some other external device, and in order to digitally tune the response of the filter, the design of a digital low pass filter is implemented. First, the analog/digital conversion is done with the National Instrument DAQ6009 card at an acquisition frequency of 10 KHz at 9600 bauds with 11 bits of resolution. The procedure consists of obtaining samples of the continuous signal at instants of time, defining $v_i\left[n\right]=v_n\left(nT\right)$, where $T$ is the sampling period.

The response of the digital first order low pass filter is obtained with the aim of reducing the computational cost when applying the filter in real time. The filter configuration is indicated in Figure 7, indicating its response in terms of the complex frequency $s$. In Equation (1), the filter response is plotted as a function of the complex discrete frequency $z$.

$$\mathrm{G}\left(\mathrm{z}\right)=\frac{\left(1-e^{\raisebox{1ex}{$\mathrm{T}$}\!\left/ \!\raisebox{-1ex}{$\mathrm{RC}$}\right.}\right){\mathrm{z}}^{-1}}{1-e^{\raisebox{1ex}{$\mathrm{T}$}\!\left/ \!\raisebox{-1ex}{$\mathrm{RC}$}\right.}{\mathrm{z}}^{-1}}$$

(1)

In Equation (2) the filter equation is indicated as a function of the discrete variable $\mathrm{n}$ by means of difference equations when implementing the inverse z-transform of Equation (1).

$${\mathrm{v}}_{\mathrm{o}}\left[\mathrm{n}\right]=e^{-2{\mathsf{\pi }\mathrm{f}}_{\mathrm{c}}\mathrm{T}}{\mathrm{v}}_{\mathrm{o}}\left[\mathrm{n}-1\right]+\left(1-e^{-2{\mathsf{\pi }\mathrm{f}}_{\mathrm{c}}\mathrm{T}}\right) {\mathrm{v}}_{\mathrm{i}}\left[\mathrm{n}-1\right]$$

(2)

To obtain the value of the cutoff frequency $\left({\mathrm{f}}_{\mathrm{c}}\right)$ and tune the digital filter, the Discrete Fourier Transform (FFT) is implemented. First, the EMG signal is digitized by means of a convolution with a Dirac delta pulse train as a function of time, where ${\mathrm{v}}_{\mathrm{i}}\left[\mathrm{n}\right]$ is a signal represented in an exponential Fourier series, as in Equation (3). ${\mathrm{a}}_{\mathrm{k}}$ represents the amplitude of the signal energy.

$${\mathrm{v}}_{\mathrm{i}}\left[\mathrm{n}\right]={\displaystyle \sum }_{\mathrm{k}=\mathrm{N}}^{}{\mathrm{a}}_{\mathrm{k}}{e}^{\mathrm{j}\frac{2\mathsf{\pi }}{\mathrm{N}}\mathrm{kn}}={\mathrm{a}}_0{e}^{\mathrm{j}\frac{2\mathsf{\pi }}{\mathrm{N}}0\mathrm{n}}+{\mathrm{a}}_1{e}^{\mathrm{j}\frac{2\mathsf{\pi }}{\mathrm{N}}1\mathrm{n}}\dots +{\mathrm{a}}_{\mathrm{N}-1}{e}^{\mathrm{j}\frac{2\mathsf{\pi }}{\mathrm{N}}\left(\mathrm{N}-1\right)\mathrm{n}}$$

(3)

The frequency spectrum analysis is performed by applying the Fourier Transform on the discrete signal ${\mathrm{v}}_{\mathrm{i}}\left[\mathrm{n}\right]$, obtaining as a result a train of delta functions in frequency $\mathrm{X}\left(e^{\mathrm{j}\varpi }\right)$, as indicated by Equation (4), whose amplitude is determined by the weighting of coefficients ${\mathrm{a}}_{\mathrm{k}}$, through the results of the spectrum in Frequency. The component that provides more energy to the signal is calculated; thus, the frequency of the induced noise is determined, and the cutoff frequency is obtained with precision $\left({\mathrm{f}}_{\mathrm{c}}\right)$ for the design of the digital filter. Figure 8 is the result of the implementation of the digital filter in the acquisition of the EMG signal.

$$\mathrm{X}\left({\mathrm{e}}^{\mathrm{j}\mathsf{\omega }}\right)={\displaystyle \sum }_{\mathrm{k}=-\infty }^{+\infty }{\mathrm{a}}_{\mathrm{k}}2\mathsf{\pi }\mathsf{\delta }\left(\mathsf{\omega }-\frac{2\mathsf{\pi }}{\mathrm{N}}\mathrm{k}\right)$$

(4)

D. 

Multiclass Classifier: One Hot Encoding

In this section, the method used is presented so that, in real time, the movements determined through the EMG interface are executed on a manipulator robot. An intelligent system for muscle contraction classification was implemented. Using a Multilayer Neural Network (MNN), a model is obtained that identifies four different classes of muscle contraction. The first class is described as Sharp muscle pulse (SMP), the second class as Smooth muscle pulse 3 s (SMP3), the third class as Smooth Muscle Pulse 5 s (SMP5) and, finally, the fourth class is described as Noise Involuntary Movements (NIM). These signals are classified using the One-Hot Encoding technique that labels the waveform of each signal with an integer. Thus, the digital inputs of a state machine are obtained, which determine the predetermined position of a manipulator robot with three degrees of freedom in the Cartesian plane (x, y, z) inside the robot workspace. In Figure 9, the architecture of the HMI based on EMG is presented.

To perform the identification of patterns in the EMG signal of a single channel, they are divided into action potentials with different time intervals. The SMP (Sharp Muscle Pulse) class has an instantaneous contraction interval of 1 s and muscle relaxation intervals of 5 s. The SMP3 (Smooth Muscle Pulse 3 s) class has a contraction interval of 3 s and muscle relaxation intervals of 5 s. The SMP5 (Smooth Muscle Pulse 5 s) class has a muscle contraction interval of 5 s and muscle relaxation intervals of 5 s. The NIM (Noise Involuntary Movements) class is a class that records the resting state of users as well as involuntary arm movements recorded during acquisition. All these samples are stored in a vector called $\mathrm{p}$. Figure 10 indicates the waveform of each class. The SMP, SMP3 and SMP5 classes indicate a position change control order in the manipulator robot, while the NIM class indicates a total stop state, so the MNN has as inputs the different signals identified in classes stored in the vector ${\mathrm{p}}^{1\times \mathrm{n}}$. An integer is assigned to each class through supervised training; this labeling is stored in a vector called ${\mathrm{T}}^{1\times \mathrm{n}}$, where n is the total number of samples.

E. 

Multiclass Classifier: Multilayer Neural Network

In this section, the implementation of an intelligent system for the classification of EMG signals is presented. The representation of the multilayer neural network is presented in Figure 11, where $\mathrm{p}=\left[{\mathrm{p}}^{\mathrm{T}}\right]$ is the vector of the $R$ inputs, $\mathrm{b}=\left[{\mathrm{b}}^{\mathrm{T}}\right]$ represents the polarization of S neurons, $\mathrm{n}=\left[{\mathrm{n}}^{\mathrm{T}}\right]$ represents the net inputs of each of the S neurons and $\mathrm{W}=\left[{\mathrm{W}}_{\mathrm{SR}}{}^{\mathrm{T}}\right]$ is the matrix of synaptic weights.

The first stage consists of data normalization because the EMG signals have different voltage thresholds. The description of this procedure is presented in Equation (5), where $\mathrm{p}$ represents the data set of the EMG signal by means of a vector of an acquisition channel. The mean of the data is subtracted, with a standard deviation equal to 1 to minimize the computational cost when the network performs the learning process.

$$\mathrm{p}=\frac{\mathrm{p}-{\mathrm{p}}^{\mathrm{mean}}}{\sqrt{{\mathrm{p}}^{\mathrm{var}}}}=\frac{\mathrm{p}-{\mathrm{p}}^{\mathrm{mean}}}{{\mathrm{p}}^{\mathrm{std}}}$$

(5)

Algorithm 1 describes the pseudocode for the implementation of the Neural Network in Python; the training consists of assigning to each sample the value of a constant that is stored in the vector $T$. This vector is the desired result for each class and has the same dimensions as the input vector $\mathrm{p}$.

Algorithm 1: Multilayer Perceptrón algorithm implemented for the EMG

1 $p\leftarrow \mathit{Input}\_\mathit{vector}$

2 $T\leftarrow \mathit{Output}\_\mathit{vector}$

3 /*** output vector T where the labeling value is stored by one-hot-encoding of each the classes***/

4 $T\leftarrow \left[\left\{0,0,0,1\right\},\left\{0,0,2,0\right\},\left\{0,3,0,0\right\},\left\{4,0,0,0\right\}\right]$

5 $\mathit{scaler}\leftarrow \mathit{StandardScaler}\left(\right).\mathit{fit}\left(P\right)$

6 $p\leftarrow \mathit{scaler}.\mathit{transform}\left(P\right)$

7 /**Divide p into a test ($P_{\mathit{test}})$ and a training set $\left(P_{\mathit{train}}\right)$**/

8 $\mathit{one}\_\mathit{hot}\_\mathit{labels}=\mathit{to}\_\mathit{categorial}\left(T,\mathit{num}\_\mathit{classes}\leftarrow 4\right)$

9 $P\_\mathit{train},P\_\mathit{test},T\_\mathit{train},T\_\mathit{test}\leftarrow \mathit{train}\_\mathit{test}\_\mathit{split}\left(P,\mathit{one}\_\mathit{hot}\_\mathit{labels},\mathit{test}\_\mathit{size}\leftarrow 0.20,\mathit{random}\_\mathit{state}\leftarrow 42\right)$

10 /**Random Initialization**/

11 $W\leftarrow 2\times \left(\mathit{random}-0.5\right)\times \mathit{scale}$

12 $\mathit{epochs}\leftarrow 3000$

13 $\mathit{hiddenNodes}\leftarrow 4$

14 $\mathit{model}\leftarrow \mathit{Sequential}\left(\right)$

15 $\mathit{model}.\mathit{add}(\mathit{Dense}\left(\mathit{hiddenNodes},\mathit{activation}\leftarrow \mathit{relu},\mathit{input}\_\mathit{dim}\leftarrow 4\right)$

16 $a\left[1\right]\leftarrow \mathit{max}\left(0,n\right)$//ReLu activation function

17 $\mathit{model}.\mathit{add}\left(\mathit{Dense}\left(4,\mathit{activation}\leftarrow \prime \mathit{softmax}\prime \right)\right)$

18 $a\left[2\right]\leftarrow e^{n_4}/{\sum }_1^5{\mathrm{e}}^{{\mathrm{n}}_4}$//Softmax activation function

19 $\mathit{model}.\mathit{summary}\left(\right)$

20 $\mathit{loss}\leftarrow \mathit{categorical}\_\mathit{crossentropy}$

21 /**Loss function (categorial cross entropy**/

22 $L\left(y,\hat{y}\right)\leftarrow \frac{1}{N}{\sum }_{j=1}^M{\sum }_{i=1}^N\left(y_{\mathit{ij}}\mathit{log}\left({\hat{y}}_{\mathit{ij}}\right)\right)$

22 $\mathit{optimizer}\leftarrow \mathit{tf}.\mathit{keras}.\mathit{optimzers}.\mathit{Adam}\left(\right)$

24 $W\leftarrow W-\frac{\alpha m}{\sqrt{v+ϵ}}$

25 $\mathit{model}.\mathit{compile}\left(\mathit{loss}\leftarrow \mathit{loss},\mathit{optimizer}\leftarrow \mathit{optimizer},\mathit{metrics}\leftarrow \left[\prime \mathit{accuracy}\prime \right]\right)$

26 $\mathit{history}\leftarrow \mathit{model}.\mathit{fit}\left(P\_\mathit{train},T\_\mathit{train},\mathit{epochs}\leftarrow \mathit{epochs},\mathit{vebose}\leftarrow 1,\mathit{validation}\_\mathit{split}\leftarrow 0.1\right)$

27 $\mathit{test},\mathit{test}\leftarrow \mathit{model}.\mathit{evaluate}\left(P,t,\mathit{verbose}\leftarrow 1\right)$

28 $\mathit{weights}\left(\mathit{model}.\mathit{layers},3\right)$

29 $\mathit{scaling}\left(\mathit{scaler},3\right)$

30 $\mathit{layers}\left(\mathit{model}.\mathit{layers}\right)$

In Figure 12, an association between the precision of the neural network with new data $\left(\mathrm{Trian}\mathrm{loss}\right)$ and the value of the loss function $\left(\mathrm{Val}\mathrm{loss}\right)$ after 3000 epochs is presented. Both graphs have a tendency to zero as training progresses, indicating a correct functioning of the optimizer. In [22], the authors designed multiclass classification on two channels of electrooculography signals and controlled an omnidirectional mobile robot in the X, Y plane. In this work, it is shown that, according to the muscle contraction time, the multiclass classification allows for the control of robotic systems that work in space (X, Y, Z) and that are adaptive to the individual characteristics of the user, achieving a personalization of the Interface.

The obtained values of the synaptic weights W and the polarization vector b of the two neurons, after 3000 epochs:

$$W_1=\left[4\right]\left[1\right]=\left[\begin{array}{c}-0.321\\ 1.016\\ 1.322\\ 1.564\end{array}\right]$$

$$W_2=\left[4\right]\left[4\right]=\left[\begin{array}{c}\begin{array}{cccc}-0.363& 0.232& 0.222& -0.123\\ 0.543& -0.127& 0.142& -0.234\\ 0.126& -0.123& -0.118& 0.233\\ -0.217& 0.147& 0.156& -0.126\end{array}\end{array}\right]$$

$$b_1=\left[\begin{array}{cccc}-0.321& 0.087& 0.123& 0.224\end{array}\right]$$

$$b_2=\left[\begin{array}{cccc}0.457& -0.121& 0.789& 0.389\end{array}\right]$$

Once the model recognizes each of the classes by means of integers, a comparison system is implemented using the premise, “If the Network output is: (integer) [1,2,3,4] then 1 is enabled when the network recognizes the waveform that corresponds to each label, otherwise it is 0”. This process allows for a combination of digital pulses for the activation of a state machine.

F. 

State Machine

The combination of digital signals obtained from the pattern recognition of the neural network by means of class classification allows for the transition change of a state machine. A Mealy-type machine is implemented, which generates an output based on its current state and an input. Three finite sets determined by the inputs, outputs and states are defined.

In Figure 13, the transitions of the digital inputs are indicated and the NIM class is represented as the most significant bit. In the next position the SMP3 class is, then the SMP5 class and finally the SMP class, so that there is an input 4 bits for transition change. Each of the states indicates a predetermined position of the manipulator robot with three degrees of freedom in Cartesian coordinates $\left({\mathrm{p}}_{\mathrm{x}},{\mathrm{p}}_{\mathrm{y}},{\mathrm{p}}_{\mathrm{z}}\right)$. Subsequently, these coordinates are converted to joint coordinates $\left({\mathrm{q}}_1,{\mathrm{q}}_2,{\mathrm{q}}_3\right)$ using the inverse kinematics of the manipulator robot. There is an input IN9 that, when detecting a status at 1 of noise or involuntary movements, completely deactivates the operation of the robot; this is taken as a security measure to not activate the robot when this class of signals occurs.

In Figure 14, the designed machine has eight possible states for muscle movement, with four digital inputs corresponding to the high and low pulses of the Neural Network recognition.

Table 1. Description of each of the desired positions of each state.

State	Input EMG	Desired Value in Meters			Desired Movement
		${\mathbf{p}}_{\mathbf{x}}$	${\mathbf{p}}_{\mathbf{y}}$	${\mathbf{p}}_{\mathbf{z}}$
S1	IN1	0	−0.34	0.38	Stop
S2	IN2	0	−0.11	0.46	Up
S3	IN4	0.34	−0.34	0.38	Right
S4	IN7	−0.34	−0.34	0.38	Left
DUL	IN8	−0.34	−0.11	0.46	Diagonal Up Left
DUR	IN3	0.34	−0.11	0.46	Diagonal Up Right
DDL	IN6	−0.34	−0.34	0.28	Diagonal Down Left
DDR	IN5	0.34	−0.34	0.28	Diagonal Down Right

describes the position in Cartesian coordinates of each of the robot states.

The selected robot is an anthropomorphic robot with three degrees of freedom and rotational joints whose operation is similar to the human arm (Figure 15), where ${\mathrm{l}}_1,{\mathrm{l}}_2$ and ${\mathrm{l}}_3$ represent the total length of the links, ${\mathrm{l}}_{\mathrm{c}1},{\mathrm{l}}_{\mathrm{c}2}$ and ${\mathrm{l}}_{\mathrm{c}3}$ represent the length from the initial end to the center of mass of each of the links that make up the robot, ${\mathrm{m}}_1,{\mathrm{m}}_2$ and ${\mathrm{m}}_3$ are the values of the center of mass of each link, ${\mathrm{x}}_{0\dots 3},{\mathrm{y}}_{0\dots 3},{\mathrm{z}}_{0\dots 3}$ represent the cartesian axes indicating the orientation of the position and ${\mathrm{q}}_1,{\mathrm{q}}_2$ and ${\mathrm{q}}_3$ represent each degree of freedom of each rotational joint of the robot.

To determine the workspace of the anthropomorphic robot, the calculation of the forward kinematics is performed, which determines the position of the end effector in Cartesian coordinates $\left({\mathrm{p}}_{\mathrm{x}},{\mathrm{p}}_{\mathrm{y}},{\mathrm{p}}_{\mathrm{z}}\right)$ based on joint coordinates $\left({\mathrm{q}}_1,{\mathrm{q}}_2,{\mathrm{q}}_3\right)$ indicated in Equation (6). These equations are fundamental for the calculation of the robot dynamics.

$$\begin{array}{ccc}{\mathrm{p}}_{\mathrm{x}}& =& \cos \left({\mathrm{q}}_1\right)\left({\mathrm{l}}_{\mathrm{c}3}\cos \left({\mathrm{q}}_2+{\mathrm{q}}_3\right)+{\mathrm{l}}_{\mathrm{c}2}\cos \left({\mathrm{q}}_2\right)\right)\\ {\mathrm{p}}_{\mathrm{y}}& =& \sin \left({\mathrm{q}}_1\right)\left({\mathrm{l}}_{\mathrm{c}3}\cos \left({\mathrm{q}}_2+{\mathrm{q}}_3\right)+{\mathrm{l}}_{\mathrm{c}2}\cos \left({\mathrm{q}}_2\right)\right)\\ {\mathrm{p}}_{\mathrm{z}}& =& {\mathrm{l}}_1+{\mathrm{l}}_{\mathrm{c}3}\sin \left({\mathrm{q}}_2+{\mathrm{q}}_3\right)+{\mathrm{l}}_{\mathrm{c}2}\sin \left({\mathrm{q}}_2\right)\end{array}$$

(6)

Because the state machine has the coordinates of the end effector position in meters for each of the Cartesian axes $x,y$ y $z$, the inverse kinematics of the robot defined in Equation (7), these equations determine the value of the position in radians for each of the degrees of freedom $\left({\mathrm{q}}_1,{\mathrm{q}}_2,{\mathrm{q}}_3\right)$.

$$\begin{gathered} {\mathrm{q}}_1={\tan }^{-1}\left(\frac{\mathrm{py}}{\mathrm{px}}\right) \\ {\mathrm{q}}_2=2{\tan }^{-1}\left(\frac{\mathrm{b}+\sqrt{{\mathrm{b}}^2+{\mathrm{a}}^2-{\mathrm{c}}^2}}{\mathrm{a}+\mathrm{c}}\right) \end{gathered}$$

(7)

where:

$$\begin{gathered} \mathrm{c}={\mathrm{p}}_{\mathrm{x}}{}^2{\cos }^2\left({\mathrm{q}}_1\right)+2{\mathrm{p}}_{\mathrm{x}}{\mathrm{p}}_{\mathrm{y}}\mathrm{sen}\left({\mathrm{q}}_1\right)\cos \left({\mathrm{q}}_1\right)+{\mathrm{p}}_{\mathrm{z}}{}^2-2{\mathrm{p}}_{\mathrm{z}}{\mathrm{l}}_1+{\mathrm{l}}_1{}^2+{\mathrm{l}}_2{}^2-{\mathrm{l}}_3{}^2 \\ \mathrm{a}=2{\mathrm{p}}_{\mathrm{x}}{\mathrm{l}}_2\cos \left({\mathrm{q}}_1\right)+2{\mathrm{p}}_{\mathrm{y}} {\mathrm{l}}_2\mathrm{sen}\left({\mathrm{q}}_1\right) \\ \mathrm{b}=2{\mathrm{p}}_{\mathrm{z}}{\mathrm{l}}_2-2{\mathrm{l}}_1{\mathrm{l}}_2 \\ {\mathrm{q}}_3={\tan }^{-1}\left(\frac{{\mathrm{p}}_{\mathrm{z}}\cos \left({\mathrm{q}}_2\right)-{\mathrm{l}}_1\cos \left({\mathrm{q}}_2\right)-{\mathrm{p}}_{\mathrm{x}}\cos \left({\mathrm{q}}_1\right)\sin \left({\mathrm{q}}_2\right)-{\mathrm{p}}_{\mathrm{y}}\sin \left({\mathrm{q}}_1\right)\sin \left({\mathrm{q}}_2\right)}{{\mathrm{p}}_{\mathrm{z}}\cos \left({\mathrm{q}}_2\right)-{\mathrm{l}}_1\cos \left(\mathrm{q}2\right)-{\mathrm{p}}_{\mathrm{x}}\cos \left({\mathrm{q}}_1\right)\sin \left({\mathrm{q}}_2\right)-{\mathrm{p}}_{\mathrm{y}}\sin \left({\mathrm{q}}_1\right)\sin \left({\mathrm{q}}_2\right)}\right) \end{gathered}$$

To implement the PD+ position tracking control algorithm, use the dynamic model defined in Equation (8).

Inertia Matrix $\left(\mathrm{M}\left(\mathrm{q}\right)\right)$

$$\left[\begin{array}{ccc}\left(\begin{array}{c}{\mathrm{I}}_1+ {\mathrm{I}}_2+{\mathrm{I}}_3+\frac{{\mathrm{l}}_2{}^2{\mathrm{m}}_3}{2}+\frac{{\mathrm{l}}_{c2}{}^2{\mathrm{m}}_2}{2}+\frac{{\mathrm{l}}_{\mathrm{c}3}{}^2{\mathrm{m}}_3}{2}\\ +\frac{{\mathrm{l}}_2{}^2{\mathrm{m}}_2\cos \left(2{\mathrm{q}}_2\right)}{2}+\frac{{\mathrm{l}}_2{}^2{\mathrm{m}}_3\cos \left(2{\mathrm{q}}_2\right)}{2}\\ +\frac{{\mathrm{l}}_{\mathrm{c}3}{}^2{\mathrm{m}}_3\cos \left(2{\mathrm{q}}_2+2{\mathrm{q}}_3\right)}{2}\\ + {\mathrm{l}}_2 {\mathrm{l}}_{c3} {\mathrm{m}}_3 \cos \left(2{\mathrm{q}}_2+{\mathrm{q}}_3\right)\\ + {\mathrm{l}}_2 {\mathrm{l}}_{c3} {\mathrm{m}}_3 \cos \left({\mathrm{q}}_3\right) \end{array}\right)& \left({\mathrm{I}}_2+{\mathrm{I}}_3\right)& {\mathrm{I}}_3\\ \left({\mathrm{I}}_2+{\mathrm{I}}_3\right)& \left(\begin{array}{c} {\mathrm{I}}_2+ {\mathrm{I}}_3+{\mathrm{l}}_{c2}{}^2 {\mathrm{m}}_2 \\ +{\mathrm{l}}_{c3}{}^2+2 {\mathrm{l}}_2 {\mathrm{l}}_{c3} {\mathrm{m}}_3 \cos \left({\mathrm{q}}_3\right)\\ +{\mathrm{l}}_2{}^2 {\mathrm{m}}_3\end{array}\right)& \left(\begin{array}{c} {\mathrm{I}}_3+ {\mathrm{l}}_{c3}{}^2 {\mathrm{m}}_3 \\ +{\mathrm{l}}_2 {\mathrm{l}}_{c3} {\mathrm{m}}_3 \cos \left({\mathrm{q}}_3\right)\end{array}\right)\\ {\mathrm{I}}_3& \left(\begin{array}{c} {\mathrm{I}}_3+{\mathrm{l}}_{c3}{}^2 {\mathrm{m}}_3 \\ +{\mathrm{l}}_2 {\mathrm{l}}_{c3} {\mathrm{m}}_3 \cos \left({\mathrm{q}}_3\right)\end{array}\right)& \left({\mathrm{I}}_3+{\mathrm{l}}_{c3}{}^2{\mathrm{m}}_3\right)\end{array}\right]$$

Coriolis Matrix $\left(\mathrm{C}\left(\mathrm{q},\dot{\mathrm{q}}\right)\right)$

$$\left[\begin{array}{ccc}\left(\begin{array}{c}-\dot{{\mathrm{q}}_2}{\mathrm{l}}_2{}^2 {\mathrm{m}}_2\sin \left(2{\mathrm{q}}_2\right)\\ - \dot{{\mathrm{q}}_2}{\mathrm{l}}_2{}^2{\mathrm{m}}_3 \sin \left(2{\mathrm{q}}_2\right) \\ -\dot{{\mathrm{q}}_2}{\mathrm{l}}_{\mathrm{c}3}{}^2{\mathrm{m}}_3 \sin \left(2{\mathrm{q}}_2+2{\mathrm{q}}_3\right)\end{array}\right)& \left(-2\dot{{\mathrm{q}}_1}{\mathrm{l}}_2{\mathrm{l}}_{c3}{\mathrm{m}}_3\sin \left(2{\mathrm{q}}_2+{\mathrm{q}}_3\right)\right)& \left(\begin{array}{c}- \dot{{\mathrm{q}}_1} {\mathrm{l}}_{c3}{}^2 {\mathrm{m}}_3 \sin \left(2{\mathrm{q}}_2+2{\mathrm{q}}_3\right)\\ - \dot{{\mathrm{q}}_1} {\mathrm{l}}_2 {\mathrm{l}}_{c3} {\mathrm{m}}_3 \sin \left({\mathrm{q}}_3\right)\\ - \dot{{\mathrm{q}}_1} {\mathrm{l}}_2 {\mathrm{l}}_{c3} {\mathrm{m}}_3 \sin \left(2{\mathrm{q}}_2+{\mathrm{q}}_3\right)\end{array}\right)\\ \left(\begin{array}{c}\frac{\dot{{\mathrm{q}}_1}{\mathrm{l}}_2{}^2{\mathrm{m}}_3\sin \left(2{\mathrm{q}}_2\right)}{2} \\ +\frac{-\dot{{\mathrm{q}}_1}{\mathrm{l}}_{\mathrm{c}2}{}^2{\mathrm{m}}_2\sin \left(2{\mathrm{q}}_2\right)}{2} \\ +\frac{\dot{\mathrm{q}1}\mathrm{lc}{3}^2\mathrm{m}3\sin \left(2\mathrm{q}2+2\mathrm{q}3\right)}{2}\\ + \dot{{\mathrm{q}}_1} {\mathrm{l}}_2{\mathrm{l}}_{c3} {\mathrm{m}}_3 \sin \left(2{\mathrm{q}}_2+{\mathrm{q}}_3\right)\end{array}\right)& \left(-2\dot{{\mathrm{q}}_3}{\mathrm{l}}_2{\mathrm{l}}_{c3}{\mathrm{m}}_3\sin \left({\mathrm{q}}_3\right)\right)& \left(-\dot{{\mathrm{q}}_3}{\mathrm{l}}_2{\mathrm{l}}_{c3}{\mathrm{m}}_3\sin \left({\mathrm{q}}_3\right)\right)\\ \left(\begin{array}{c}\frac{\dot{{\mathrm{q}}_1}{\mathrm{l}}_{c3}{}^2{\mathrm{m}}_3\sin \left(2{\mathrm{q}}_2+2{\mathrm{q}}_3\right)}{2}\\ +\frac{\dot{{\mathrm{q}}_1}{\mathrm{l}}_2{\mathrm{l}}_{c3}{\mathrm{m}}_3\sin \left({\mathrm{q}}_3\right)}{2} \\ +\frac{\dot{{\mathrm{q}}_1}{\mathrm{l}}_2{\mathrm{l}}_{c3}{\mathrm{m}}_3\sin \left(2{\mathrm{q}}_2+{\mathrm{q}}_3\right)}{2}\end{array}\right)& \left(\dot{{\mathrm{q}}_2}{\mathrm{l}}_2{\mathrm{l}}_{c3}{\mathrm{m}}_3\sin \left({\mathrm{q}}_3\right)\right)& 0\end{array}\right]$$

Gravity Vector $\left(\mathrm{g}\left(\mathrm{q}\right)\right)$

$$\left[\begin{array}{c}0\\ -{\mathrm{g}\mathrm{l}}_{c3}{\mathrm{m}}_3\cos \left({\mathrm{q}}_2+{\mathrm{q}}_3\right)-{\mathrm{g}\mathrm{l}}_2{\mathrm{m}}_3\cos \left({\mathrm{q}}_2\right)-{\mathrm{g}\mathrm{l}}_{c2}{\mathrm{m}}_2\cos \left({\mathrm{q}}_2\right)\\ -{\mathrm{g}\mathrm{l}}_{c3}{\mathrm{m}}_3\cos \left({\mathrm{q}}_2+{\mathrm{q}}_3\right)\end{array}\right]$$

Viscous friction vector $\left(\mathrm{B}\right)$

$$\mathrm{B}\dot{q}=\left[\begin{array}{c}{\mathrm{B}}_1\dot{{\mathrm{q}}_1}\\ {\mathrm{B}}_2\dot{{\mathrm{q}}_2}\\ {\mathrm{B}}_3\dot{{\mathrm{q}}_3}\end{array} \right]$$

Torque Vector

$$\mathrm{\tau }=\left[\begin{array}{c}{\mathrm{\tau }}_1\\ {\mathrm{\tau }}_2\\ {\mathrm{\tau }}_3\end{array}\right]$$

$$\mathrm{\tau }=\mathrm{M}\left(\mathrm{q}\right)\ddot{\mathrm{q}}+\mathrm{C}\left(\mathrm{q},\dot{\mathrm{q}}\right)\dot{\mathrm{q}}+\mathrm{g}\left(\mathrm{q}\right)+\mathrm{B}\dot{\mathrm{q}}$$

(8)

where $\mathrm{M}\left(\mathrm{q}\right)$ is a positive definite symmetric matrix of $\mathrm{n}\mathrm{x}\mathrm{n}$ called the inertia matrix, with $\mathrm{I}1,\mathrm{I}2,\mathrm{I}3$ being the moments of inertia of the rigid links of the mechanical structure of the robot, $\mathrm{C}\left(\mathrm{q},\dot{\mathrm{q}}\right)$ is an $\mathrm{n}\mathrm{x}1$ vector called the vector of centrifugal and Coriolis forces, $\mathrm{B}\dot{\mathrm{q}}$ is an $\mathrm{n}\mathrm{x}1$ vector that determines the viscous friction, $\mathrm{g}\left(\mathrm{q}\right)$ is an $\mathrm{n}\mathrm{x}1$ vector of gravitational forces and $\mathrm{\tau }$ is the $\mathrm{n}\mathrm{x}1$ vector that determines the torques and forces applied by the actuators at the joints.

G. 

Position Control

As a result of the cartesian coordinates $\left(\mathrm{px},\mathrm{py},\mathrm{pz}\right)$ obtained from the classifier by means of a Multilayer Neural Network and assigned to a discrete event by means of a state machine, the desired Cartesian coordinates for the robot are obtained, which are transformed to joint coordinates $\left({\mathrm{q}}_1,{\mathrm{q}}_2,{\mathrm{q}}_3\right)$ from inverse kinematics. These values are the inputs for the PD+ type position control system. [15].

The PD+ control with gravity compensation, defined in Equation (9) by ${\mathrm{\tau }}_{\mathrm{PD}+}$, is an algorithm that includes proportional control of the position error $\tilde{\mathrm{q}}$ and velocity error proportional control $\dot{\tilde{\mathrm{q}}}$, where ${\mathrm{K}}_{\mathrm{p}},{\mathrm{K}}_{\mathrm{v}}\in {ℝ}^{\mathrm{nxn}}$ are the proportional and derivative gains, respectively, both are positive definite matrices, and the full dynamics of the robot are added. In the structure of this scheme, the trajectory of position, velocity and desired acceleration is involved, ${\mathrm{q}}_{\mathrm{d}}\left(\mathrm{t}\right),\dot{{\mathrm{q}}_{\mathrm{d}}}\left(\mathrm{t}\right),{\ddot{\mathrm{q}}}_{\mathrm{d}}\left(\mathrm{t}\right)\in {ℝ}^{\mathrm{n}}$.

$${\mathrm{\tau }}_{\mathrm{PD}+}={\mathrm{K}}_{\mathrm{p}}\tilde{\mathrm{q}}+{\mathrm{K}}_{\mathrm{v}}\dot{\tilde{\mathrm{q}}}+\mathrm{M}\left(\mathrm{q}\right)\ddot{{\mathrm{q}}_{\mathrm{d}}}+\mathrm{C}\left(\mathrm{q},\dot{\mathrm{q}}\right)\dot{{\mathrm{q}}_{\mathrm{d}}}+\mathrm{B}\dot{{\mathrm{q}}_{\mathrm{d}}}+\mathrm{g}\left(\mathrm{q}\right)$$

(9)

The objective of this control is to find a torque value, $\mathrm{\tau }$, such that it satisfies the expression indicated in Equation (10).

$$\underset{\mathrm{t}\to \infty }{\lim }\left[\begin{array}{c}\tilde{\mathrm{q}}\\ \dot{\tilde{\mathrm{q}}}\end{array}\right]=\left[\begin{array}{c}0\\ 0\end{array}\right]\in {ℝ}^{2\mathrm{n}}$$

(10)

where $\tilde{\mathrm{q}}\in {ℝ}^{\mathrm{n}}$ is the following error and is defined as $\tilde{\mathrm{q}}={\mathrm{q}}_{\mathrm{d}}\left(\mathrm{t}\right)-\mathrm{q}\left(\mathrm{t}\right)$, and $\dot{\tilde{\mathrm{q}}}\in {ℝ}^{\mathrm{n}}$ is the velocity error, given by $\dot{\tilde{\mathrm{q}}}=\dot{{\mathrm{q}}_{\mathrm{d}}}\left(\mathrm{t}\right)-\dot{\mathrm{q}}\left(\mathrm{t}\right)$. Figure 16 indicates the block diagram of the implemented PD+ control.

Figure 17a shows the behavior of the zero-position error trend in each joint coordinate of the robot whose Cartesian coordinate is assigned by the state machine. The operation of the control when reaching the desired joint position is also presented. Figure 17b shows the virtual simulation of the robot applying the PD+ control for the generation of trajectories through the interaction of the EMG signal.

A graphical user interface is designed as indicated in Figure 18b with visual feedback of the EMG signal, the result of the state machine by means of a green indicator that indicates the position detected of the MNN’s classification, the control curves resulting from the implemented PD+ and a simulation of the virtual robot that indicates the position of the end effector. In Figure 18b, the user connection and the operation of the interface to calculate the response time metrics are indicated.

4. Results and Discussions

The EMG signal classification method that allows for the generation of coordinates for the trajectory control of a manipulator robot has been developed. The user’s ability to follow a series of point-to-point coordinates previously determined by colors is measured according to the time of sustained contraction. The yellow dot indicates the starting point of the test, the green dots indicate the path to be followed and the blue dot indicates the end point to which the robot’s end effector must reach. Two trajectories are proposed that increase the difficulty indicating a penalty each time the user enters a contraction command other than the one indicated. The time in which the user generates the trajectory is also recorded. The test ends when the user generates the trajectory without penalties. In Figure 19a, the first proposed trajectory is indicated, in Figure 19b, the time and the number of penalties for each test performed by the user are presented and in Figure 19c, a graph of the response time for trajectory 1 is indicated.

A downward trend is observed in this first trajectory in the response time when completing the test with zero penalties. It is shown that 11 repetitions are enough to successfully complete the proposed trajectory. At the beginning, it indicates an initial time of 118.52 s, and, at the end, it indicates an initial time of 77.54 s, which corresponds to a decrease in the response time by 34.58%. In Figure 19a, the second proposed trajectory is indicated. Figure 19b shows the time and the number of penalties for each test performed by the user. Figure 19c indicates a graph of the response time for trajectory 2.

In the second trajectory, a behavior similar to the first trajectory is observed. With 11 repetitions, it is enough to successfully complete the test. At the beginning, an initial time of 188.64 s is indicated, and, at the end, an initial time of 111.99 s is indicated, which corresponds to a decrease in the response time by 40.64%. When performing the test with different points, the same trend is observed in the decrease in response time. By around 11 repetitions, the user has mastery of the HMI. It should be noted that the model is customized for each user according to individual characteristics and muscle contraction time in addition to adding a recognition class for involuntary movements that blocks the operation of the robot and takes it to a “home” state.

5. Conclusions

An HMI that allows for the classification of muscular signals according to the contraction time has been designed. The model implemented through a neural network allows for the personalization and classification in real time for the generation of movement commands of a virtual robot. The HMI can be implemented with inexperienced users who need only 11 repetitions to master the operation of the system, reducing the learning curve. The future work of this project is to implement the classification of multiclass signals in a physical robotic system. In assistive systems or bionic prostheses, although there is the limitation that, being a discrete system, the movement command is determined by a state machine, the improvement consists of implementing neurofuzzy systems that allow for the generation of continuous trajectories in the robot. The development of assistance systems through physiological signals is important for people with disabilities since it allows them to better adapt to their work or personal environment.