Next Article in Journal
A Blockchain-Based Privacy Information Security Sharing Scheme in Industrial Internet of Things
Next Article in Special Issue
Evaluating Automatic Body Orientation Detection for Indoor Location from Skeleton Tracking Data to Detect Socially Occupied Spaces Using the Kinect v2, Azure Kinect and Zed 2i
Previous Article in Journal
A Novel Route to Optimize Placement Equipment Kinematics by Coupling Capacitive Accelerometers
Previous Article in Special Issue
Augmentation of Human Action Datasets with Suboptimal Warping and Representative Data Samples
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Pattern Recognition of EMG Signals by Machine Learning for the Control of a Manipulator Robot

Centro de Investigación, Innovación y Desarrollo Tecnológico UVM (CIIDETEC-UVM), Universidad del Valle de Mexico, Querétaro 76230, Mexico
*
Author to whom correspondence should be addressed.
Sensors 2022, 22(9), 3424; https://doi.org/10.3390/s22093424
Submission received: 24 March 2022 / Revised: 21 April 2022 / Accepted: 25 April 2022 / Published: 30 April 2022
(This article belongs to the Special Issue Intelligent Sensors for Human Motion Analysis)

Abstract

:
Human Machine Interfaces (HMI) principles are for the development of interfaces for assistance or support systems in physiotherapy or rehabilitation processes. One of the main problems is the degree of customization when applying some rehabilitation therapy or when adapting an assistance system to the individual characteristics of the users. To solve this inconvenience, it is proposed to implement a database of surface Electromyography (sEMG) of a channel in healthy individuals for pattern recognition through Neural Networks of contraction in the muscular region of the biceps brachii. Each movement is labeled using the One-Hot Encoding technique, which activates a state machine to control the position of an anthropomorphic manipulator robot and validate the response time of the designed HMI. Preliminary results show that the learning curve decreases when customizing the interface. The developed system uses muscle contraction to direct the position of the end effector of a virtual robot. The classification of Electromyography (EMG) signals is obtained to generate trajectories in real time by designing a test platform in LabVIEW.

1. Introduction

A person with a disability is an individual who has one or more physical or mental deficiencies that prevent their full and effective participation in equal conditions when interacting with different social environments. In recent years, the development of HMI for people with motor disabilities has been oriented towards the use of systems based on Electromyography (EMG). In [1], a review of the state of the art in EMG monitoring is presented in terms of applications in rehabilitation and minimally invasive acquisition devices; among the advantages that it highlights are in the fields of physiotherapy and telemedicine. In [2], through three EMG channels, they control the position of a robot with two degrees of freedom; the processing is done as a function of time through the amplitude of the signal when movements are made with the elbow and the shoulder joint. Four channels of surface electromyography acquisition are proposed in [3], where pairs of electrodes are placed according to the position and orientation of the target muscles. Selecting materials with excellent properties for devices on the skin, the fabricated electrodes achieve low skin electrode impedance and record sEMG signals with a high signal-to-noise ratio. In [4], a review on signal acquisition and pattern recognition through Machine Learning is presented. In [5], a myoelectric pattern recognition-driven hand exoskeleton was designed for stroke rehabilitation. It detects and recognizes the intention of movement based on EMG signals, and then the exoskeleton helps the user to perform six types of hand movements in a real way. One of the main challenges in the design of interfaces based on sEMG is the obtention of a signal function or model that allows for the reliable control of a care system. Due to the non-stationary signal behavior, three methods are generally used for sEMG analysis to extract information, which are in the time [6,7], frequency, and time–frequency domain [8]. There are some practical factors, such as the change in arm position, that prevent robust myoelectric control. In [9], an experiment with 14 subjects is carried out to accurately characterize factors that alter the EMG recording. Using regression algorithms, they obtain real-time feedback on changes in the position of the arm and displacement of the electrodes. Pattern recognition has been studied further to develop control algorithms for electric hand prostheses [10,11]. These works have shown excellent accuracy when classifying different types of hand movement (>95% for 10 classes), [12,13,14]. Most of the pattern recognition approaches have the limitation that only one of the functions of the prosthetic hand can be controlled, due to its sequential and binary control. Such control strategies make it impossible to perform natural movements of the hand that consist of the simultaneous activation of different degrees of freedom. Some studies have introduced new pattern recognition schemes that classify combined movements [15,16,17,18,19]. The disadvantage of the new approach is the total number of classes, as it increases drastically when new classes are considered. Recently, regression-based approaches have been on the rise, as they provide control information that allows for multi-degree-of-freedom control. In this work, a regression algorithm using neural networks is proposed to obtain a model through multiclass categorization that allows for the control of a robotic system with three degrees of freedom of the anthropomorphic type. The analysis of a single channel of sEMG that classifies signals with different times of muscle contraction is implemented, with the objective that the robot moves accurately according to predetermined positions in a state machine and demonstrates the correct operation of an HMI by reducing the learning curve. In [20], a study of multichannel electromyography signals is carried out, which is one of the methods used in the recognition of human movement patterns. An exoskeleton robot is controlled and EMG signals are measured during dynamic or isometric muscle contractions. As a result, they developed a pattern recognition model of dynamic and isometric muscle contractions using the Short Time Fourier Transform (STFT).
Section 2 presents the fundamentals of the EMG signal. Section 3 presents the design of the HMI from the acquisition of the EMG signal and its analog and digital processing. The multiclass classification model obtained using neural networks is described. The different classes to be detected, the training algorithm and the operation of a state machine that determines the position of the robot according to the result obtained from the model are shown. The classification to reach the desired position is explained, obtaining the dynamics and inverse kinematics of an anthropomorphic robot with three degrees of freedom. A PD+ control is implemented to apply the necessary torque to each joint of the robot and to validate the operation of the HMI. It designs a graphical user interface in LabVIEW software by interacting a virtual robot and the EMG signal. Section 4 describes the results obtained with the classifier, the experimental tests and the response time for each test.

2. HMI Systems Based on EMG

The neuron is the cellular unit of the central nervous system. It has two properties: (1) Sensory, which gives it the ability to respond to physical and chemical agents with the initiation of a nerve impulse; and (2) Conductivity, which gives it the property of transmitting these impulses from one side to another. The dendrites that originate in the cell body are responsible for receiving impulses from other neurons and sending them to the soma of their own neuron. The axon is an extension from the neuronal soma that conducts the impulse to the muscle; it is surrounded by a myelin sheath that allows for better impulse conductivity. The neuron that originates the EMG biopotential is called a motor neuron, which conducts the impulse through the neuromuscular junction to the muscle fiber, as shown in Figure 1 [21].
EMG is an electrical exploration of the peripheral nerves by the stimulation of the muscles to achieve their contraction. The differential potential in the biceps brachii is measured by placing two silver/silver chloride (Ag/AgCl) electrodes and a reference electrode located at the junction of the forearm and hand, as shown in Figure 2. [20].
When muscle contraction is performed, there are two types: (1) Isometric contraction, which is a static form of exercise in which a muscle contracts to produce force without an appreciable change in muscle length; and (2) Isotonic contraction, which is without appreciable change in the force of contraction. The distance between the origin of the muscle and its insertion becomes smaller. For EMG acquisition study purposes, in the protocol carried out, isometric contractions are recorded by placing a weight in the user’s hand with a value of 5 pounds. This process is carried out in order to avoid the acquisition of noise due to involuntary movements and to keep the arm static while the biceps brachii contraction is performed for short periods of time. This process is carried out in order to avoid the acquisition of noises due to involuntary movements and to keep the arm static while the contraction of the biceps brachii is performed for short periods of recording time, no longer than 45 s, preventing the user from making an unwanted movement due to fatigue. When performing the acquisition, it was observed that the muscle relaxation periods of 5 s made it possible to accurately obtain the muscle contraction times, thus avoiding the introduction of noise due to muscle fatigue. The goal of this work is to demonstrate that, with a correct training of the neural network, adding dynamic muscle contractions due to involuntary movements as an extra class of recognition allows the system to rule out this muscle noise as a motion control command. The EMG signals have amplitudes from 0.1 mV to 5 mV, with a bandwidth of 0 to 5 KHz [21]. With this information, a first acquisition is made using a BIOPAC® commercial system, which allows for the recording of the differential signal taken from two electrodes and a reference, as indicated in Figure 3a. This system records the waveforms of the EMG signal in order to validate the implemented acquisition protocol. Tests were performed for contraction times of 1, 3 and 5 s. In Figure 3b, the response obtained from the EMG signal to a contraction of 5 s with rest pauses also of 5 s is presented. The inconvenience presented in the acquisition protocol when using this system is that the record is stored in a numerical database and cannot be read directly by any other acquisition card. Real-time implementation of the Fast Fourier Transform (FFT) is necessary to verify the spectrum in frequency and obtain the value of the cutoff frequency for the implementation of filters. The next section presents the instrumentation implemented and the digital processing for the acquisition of the database.

3. Materials and Methods

The amplifier used is the IC AD620 due to its characteristic of a common rejection ratio of 100 dB and the gain adjustment with an external resistor. A circuit with a basal corrector and a Common Mode Rejection (CMRR) configuration connected to the junction of the forearm and hand is implemented as a circuit reference. According to the amplitude and frequency characteristics of the EMG signal, the analog processing stage is designed, which includes amplification, isolation and filtering.
A. 
Amplification with basal corrector
An instrumentation amplifier CI AD620 is implemented as a preamplification system to acquire the differential EMG signal with a gain of 500. A basal correction circuit is conditioned to eliminate the level of direct current (DC) caused by involuntary movements of the user or an incorrect connection of the electrodes. The circuit is a IC TL084 operational amplifier in its integrator configuration that is connected in feedback to the Ref and V out outputs of the instrumentation amplifier, as shown in Figure 4, implementing a high pass filter that eliminates the DC bias voltage and preventing op amps from reaching their maximum power limits.
B. 
Analog Filter
To filter the frequency components that are not within the bandwidth of the EMG signal, a range from 0.5 Hz to 5 KHz, a second order bandpass filter in Butterworth configuration with unity gain is designed, with a ratio of 40 dB per decade using high impedance TL084 operational amplifiers, precision resistors and electrolytic capacitors; see Figure 5.
The output of the analog filter stage is connected to the absolute voltage input of a DAQ6009 acquisition card connected via USB port to a laptop, with a sample rate of 10KHz. An acquisition card with a ground plane is designed to decrease inductive noise, as indicated in Figure 6a. Figure 6b shows the response of the acquisition card in the Tektronix® oscilloscope. Analog noise is observed, which is subsequently eliminated by means of a digital filter.
C. 
Digital Filter
Due to the acquisition system being subject to the interference of electromagnetic noise induced by lamps or some other external device, and in order to digitally tune the response of the filter, the design of a digital low pass filter is implemented. First, the analog/digital conversion is done with the National Instrument DAQ6009 card at an acquisition frequency of 10 KHz at 9600 bauds with 11 bits of resolution. The procedure consists of obtaining samples of the continuous signal at instants of time, defining v i n = v n n T , where T is the sampling period.
The response of the digital first order low pass filter is obtained with the aim of reducing the computational cost when applying the filter in real time. The filter configuration is indicated in Figure 7, indicating its response in terms of the complex frequency s . In Equation (1), the filter response is plotted as a function of the complex discrete frequency z .
G z = 1 e T RC z 1 1 e T RC z 1
In Equation (2) the filter equation is indicated as a function of the discrete variable n by means of difference equations when implementing the inverse z-transform of Equation (1).
v o n = e 2 π f c T   v o n 1 + 1 e 2 π f c T   v i n 1
To obtain the value of the cutoff frequency f c and tune the digital filter, the Discrete Fourier Transform (FFT) is implemented. First, the EMG signal is digitized by means of a convolution with a Dirac delta pulse train as a function of time, where v i n is a signal represented in an exponential Fourier series, as in Equation (3). a k represents the amplitude of the signal energy.
v i n = k = N   a k e j 2 π N kn =   a 0 e j 2 π N 0 n + a 1 e j 2 π N 1 n +   a N 1 e j 2 π N N 1 n  
The frequency spectrum analysis is performed by applying the Fourier Transform on the discrete signal v i n , obtaining as a result a train of delta functions in frequency X e j ϖ , as indicated by Equation (4), whose amplitude is determined by the weighting of coefficients a k , through the results of the spectrum in Frequency. The component that provides more energy to the signal is calculated; thus, the frequency of the induced noise is determined, and the cutoff frequency is obtained with precision f c for the design of the digital filter. Figure 8 is the result of the implementation of the digital filter in the acquisition of the EMG signal.
X e j ω = k = + a k 2 π δ ω 2 π N k
D. 
Multiclass Classifier: One Hot Encoding
In this section, the method used is presented so that, in real time, the movements determined through the EMG interface are executed on a manipulator robot. An intelligent system for muscle contraction classification was implemented. Using a Multilayer Neural Network (MNN), a model is obtained that identifies four different classes of muscle contraction. The first class is described as Sharp muscle pulse (SMP), the second class as Smooth muscle pulse 3 s (SMP3), the third class as Smooth Muscle Pulse 5 s (SMP5) and, finally, the fourth class is described as Noise Involuntary Movements (NIM). These signals are classified using the One-Hot Encoding technique that labels the waveform of each signal with an integer. Thus, the digital inputs of a state machine are obtained, which determine the predetermined position of a manipulator robot with three degrees of freedom in the Cartesian plane (x, y, z) inside the robot workspace. In Figure 9, the architecture of the HMI based on EMG is presented.
To perform the identification of patterns in the EMG signal of a single channel, they are divided into action potentials with different time intervals. The SMP (Sharp Muscle Pulse) class has an instantaneous contraction interval of 1 s and muscle relaxation intervals of 5 s. The SMP3 (Smooth Muscle Pulse 3 s) class has a contraction interval of 3 s and muscle relaxation intervals of 5 s. The SMP5 (Smooth Muscle Pulse 5 s) class has a muscle contraction interval of 5 s and muscle relaxation intervals of 5 s. The NIM (Noise Involuntary Movements) class is a class that records the resting state of users as well as involuntary arm movements recorded during acquisition. All these samples are stored in a vector called p . Figure 10 indicates the waveform of each class. The SMP, SMP3 and SMP5 classes indicate a position change control order in the manipulator robot, while the NIM class indicates a total stop state, so the MNN has as inputs the different signals identified in classes stored in the vector p 1 × n . An integer is assigned to each class through supervised training; this labeling is stored in a vector called T 1 × n , where n is the total number of samples.
E. 
Multiclass Classifier: Multilayer Neural Network
In this section, the implementation of an intelligent system for the classification of EMG signals is presented. The representation of the multilayer neural network is presented in Figure 11, where p = p T is the vector of the R inputs, b = b T represents the polarization of S neurons, n = n T represents the net inputs of each of the S neurons and W = W SR T is the matrix of synaptic weights.
The first stage consists of data normalization because the EMG signals have different voltage thresholds. The description of this procedure is presented in Equation (5), where p represents the data set of the EMG signal by means of a vector of an acquisition channel. The mean of the data is subtracted, with a standard deviation equal to 1 to minimize the computational cost when the network performs the learning process.
p = p p mean p var = p p mean p std
Algorithm 1 describes the pseudocode for the implementation of the Neural Network in Python; the training consists of assigning to each sample the value of a constant that is stored in the vector T . This vector is the desired result for each class and has the same dimensions as the input vector p .
Algorithm 1: Multilayer Perceptrón algorithm implemented for the EMG
1  p   Input _ vector
2  T   Output _ vector
3 /*** output vector T where the labeling value is stored by one-hot-encoding of each the classes***/
4  T   0 , 0 , 0 , 1 , 0 , 0 , 2 , 0 , 0 , 3 , 0 , 0 , 4 , 0 , 0 , 0
5  scaler   StandardScaler   . fit P
6  p   scaler . transform P
7 /**Divide p into a test ( P test ) and a training set P train **/
8  one _ hot _ labels = to _ categorial T , num _ classes 4
9  P _ train , P _ test , T _ train , T _ test train _ test _ split P , one _ hot _ labels ,   test _ size 0.20 , random _ state 42
10 /**Random Initialization**/
11  W 2 × random 0.5 × scale
12  epochs 3000
13  hiddenNodes 4
14  model Sequential  
15  model . add ( Dense hiddenNodes , activation relu , input _ dim 4
16  a 1 max 0 , n //ReLu activation function
17  model . add Dense 4 , activation softmax
18  a 2 e n 4 /   1 5 e n 4 //Softmax activation function
19  model . summary  
20  loss categorical _ crossentropy
21 /**Loss function (categorial cross entropy**/
22  L y ,   y ^ 1 N j = 1 M i = 1 N y ij log y ^ ij
22  optimizer   tf . keras . optimzers . Adam  
24  W   W α m v + ϵ
25  model . compile loss loss , optimizer optimizer , metrics accuracy
26  history model . fit P _ train , T _ train   , epochs epochs , vebose 1   , validation _ split 0.1  
27  test ,   test   model . evaluate P , t , verbose 1
28  weights model . layers , 3
29  scaling scaler , 3
30  layers model . layers
In Figure 12, an association between the precision of the neural network with new data Trian   loss and the value of the loss function Val   loss after 3000 epochs is presented. Both graphs have a tendency to zero as training progresses, indicating a correct functioning of the optimizer. In [22], the authors designed multiclass classification on two channels of electrooculography signals and controlled an omnidirectional mobile robot in the X, Y plane. In this work, it is shown that, according to the muscle contraction time, the multiclass classification allows for the control of robotic systems that work in space (X, Y, Z) and that are adaptive to the individual characteristics of the user, achieving a personalization of the Interface.
The obtained values of the synaptic weights W and the polarization vector b of the two neurons, after 3000 epochs:
W 1 = 4 1 = 0.321 1.016 1.322 1.564
W 2 = 4 4 = 0.363 0.232 0.222 0.123 0.543 0.127 0.142 0.234 0.126 0.123 0.118 0.233 0.217 0.147 0.156 0.126
b 1 = [ 0.321 0.087 0.123 0.224 ]
b 2 = 0.457 0.121 0.789 0.389
Once the model recognizes each of the classes by means of integers, a comparison system is implemented using the premise, “If the Network output is: (integer) [1,2,3,4] then 1 is enabled when the network recognizes the waveform that corresponds to each label, otherwise it is 0”. This process allows for a combination of digital pulses for the activation of a state machine.
F. 
State Machine
The combination of digital signals obtained from the pattern recognition of the neural network by means of class classification allows for the transition change of a state machine. A Mealy-type machine is implemented, which generates an output based on its current state and an input. Three finite sets determined by the inputs, outputs and states are defined.
In Figure 13, the transitions of the digital inputs are indicated and the NIM class is represented as the most significant bit. In the next position the SMP3 class is, then the SMP5 class and finally the SMP class, so that there is an input 4 bits for transition change. Each of the states indicates a predetermined position of the manipulator robot with three degrees of freedom in Cartesian coordinates p x , p y , p z . Subsequently, these coordinates are converted to joint coordinates q 1 , q 2 , q 3 using the inverse kinematics of the manipulator robot. There is an input IN9 that, when detecting a status at 1 of noise or involuntary movements, completely deactivates the operation of the robot; this is taken as a security measure to not activate the robot when this class of signals occurs.
In Figure 14, the designed machine has eight possible states for muscle movement, with four digital inputs corresponding to the high and low pulses of the Neural Network recognition. Table 1 describes the position in Cartesian coordinates of each of the robot states.
The selected robot is an anthropomorphic robot with three degrees of freedom and rotational joints whose operation is similar to the human arm (Figure 15), where l 1 , l 2 and l 3 represent the total length of the links, l c 1 , l c 2 and l c 3 represent the length from the initial end to the center of mass of each of the links that make up the robot, m 1 , m 2 and m 3 are the values of the center of mass of each link, x 0 3 , y 0 3 , z 0 3 represent the cartesian axes indicating the orientation of the position and q 1 , q 2 and q 3 represent each degree of freedom of each rotational joint of the robot.
To determine the workspace of the anthropomorphic robot, the calculation of the forward kinematics is performed, which determines the position of the end effector in Cartesian coordinates p x , p y , p z based on joint coordinates q 1 , q 2 , q 3 indicated in Equation (6). These equations are fundamental for the calculation of the robot dynamics.
p x = cos q 1 l c 3   cos q 2 +   q 3 +   l c 2   cos q 2 p y = sin q 1   l c 3   cos q 2 +   q 3 +   l c 2   cos q 2 p z = l 1 +   l c 3   sin q 2 +   q 3 +   l c 2   sin q 2
Because the state machine has the coordinates of the end effector position in meters for each of the Cartesian axes x , y y z , the inverse kinematics of the robot defined in Equation (7), these equations determine the value of the position in radians for each of the degrees of freedom q 1 , q 2 , q 3 .
q 1 = tan 1 py px q 2 = 2   tan 1 b + b 2 + a 2 c 2 a + c
where:
c =   p x 2   cos 2 q 1 + 2   p x   p y   sen q 1 cos q 1 + p z 2 2   p z   l 1 + l 1 2 + l 2 2   l 3 2 a = 2   p x   l 2 cos q 1 + 2   p y   l 2   sen q 1 b = 2   p z   l 2 2   l 1   l 2 q 3 = tan 1 p z   cos q 2   l 1   cos q 2   p x   cos q 1   sin q 2   p y   sin q 1   sin q 2   p z   cos q 2   l 1   cos q 2   p x   cos q 1   sin q 2   p y   sin q 1   sin q 2
To implement the PD+ position tracking control algorithm, use the dynamic model defined in Equation (8).
Inertia Matrix M q
I 1 +   I 2 + I 3 + l 2 2   m 3 2 + l c 2 2   m 2 2 + l c 3 2   m 3 2 + l 2 2   m 2   cos 2 q 2 2 + l 2 2   m 3   cos 2 q 2 2 + l c 3 2   m 3   cos 2 q 2   +   2 q 3 2 +   l 2   l c 3   m 3   cos 2 q 2   +   q 3 +   l 2   l c 3   m 3   cos q 3   I 2 + I 3 I 3 I 2 + I 3   I 2 +   I 3 + l c 2 2   m 2   +   l c 3 2 + 2   l 2   l c 3   m 3   cos q 3 + l 2 2   m 3   I 3 +   l c 3 2   m 3   + l 2   l c 3   m 3   cos q 3   I 3   I 3 + l c 3 2   m 3   + l 2   l c 3   m 3   cos q 3 I 3 +   l c 3 2   m 3
Coriolis Matrix C q , q ˙
q 2 ˙ l 2 2   m 2 sin 2 q 2   q 2 ˙ l 2 2 m 3   sin 2 q 2   q 2 ˙ l c 3 2 m 3   sin 2 q 2 + 2 q 3 2 q 1 ˙   l 2   l c 3   m 3 sin 2 q 2 +   q 3   q 1 ˙   l c 3 2   m 3   sin 2 q 2   +   2 q 3       q 1 ˙   l 2   l c 3   m 3   sin q 3   q 1 ˙   l 2   l c 3   m 3   sin 2 q 2 + q 3 q 1 ˙   l 2 2 m 3   sin 2 q 2 2   + q 1 ˙ l c 2 2   m 2   sin 2 q 2 2   + q 1 ˙   lc 3 2   m 3   sin 2 q 2 + 2 q 3 2 +   q 1 ˙   l 2 l c 3   m 3   sin 2 q 2 +   q 3 2 q 3 ˙   l 2   l c 3   m 3   sin q 3 q 3 ˙   l 2   l c 3   m 3   sin q 3 q 1 ˙   l c 3 2   m 3   sin 2 q 2 + 2 q 3 2 + q 1 ˙   l 2   l c 3   m 3   sin q 3 2   + q 1 ˙   l 2   l c 3   m 3   sin 2 q 2 + q 3 2 q 2 ˙   l 2   l c 3   m 3   sin q 3 0
Gravity Vector g q
0   g   l c 3   m 3   cos q 2 + q 3   g   l 2   m 3   cos q 2   g   l c 2   m 2   cos q 2   g   l c 3   m 3   cos q 2 +   q 3
Viscous friction vector B
B q ˙ = B 1 q 1 ˙ B 2 q 2 ˙ B 3 q 3 ˙  
Torque Vector
τ = τ 1 τ 2 τ 3
τ = M q q ¨ + C q , q ˙ q ˙ + g q + B q ˙
where M q is a positive definite symmetric matrix of n   x   n called the inertia matrix, with I 1 , I 2 , I 3 being the moments of inertia of the rigid links of the mechanical structure of the robot, C q , q ˙ is an n   x   1 vector called the vector of centrifugal and Coriolis forces, B q ˙ is an n   x   1 vector that determines the viscous friction, g q is an n   x   1 vector of gravitational forces and τ is the n   x   1 vector that determines the torques and forces applied by the actuators at the joints.
G. 
Position Control
As a result of the cartesian coordinates px ,   py ,   pz obtained from the classifier by means of a Multilayer Neural Network and assigned to a discrete event by means of a state machine, the desired Cartesian coordinates for the robot are obtained, which are transformed to joint coordinates q 1 ,   q 2 ,   q 3 from inverse kinematics. These values are the inputs for the PD+ type position control system. [15].
The PD+ control with gravity compensation, defined in Equation (9) by τ PD + , is an algorithm that includes proportional control of the position error q ˜ and velocity error proportional control q ˜ ˙ , where K p , K v     nxn are the proportional and derivative gains, respectively, both are positive definite matrices, and the full dynamics of the robot are added. In the structure of this scheme, the trajectory of position, velocity and desired acceleration is involved, q d t , q d ˙ t , q ¨ d t   n .
τ PD + = K p q ˜ + K v q ˜ ˙ + M q q d ¨ + C q , q ˙ q d ˙ + B q d ˙ + g q
The objective of this control is to find a torque value, τ , such that it satisfies the expression indicated in Equation (10).
lim t q ˜ q ˜ ˙ = 0 0   2 n
where q ˜   n is the following error and is defined as q ˜ =   q d t q t , and q ˜ ˙     n is the velocity error, given by q ˜ ˙ = q d ˙ t q ˙ t . Figure 16 indicates the block diagram of the implemented PD+ control.
Figure 17a shows the behavior of the zero-position error trend in each joint coordinate of the robot whose Cartesian coordinate is assigned by the state machine. The operation of the control when reaching the desired joint position is also presented. Figure 17b shows the virtual simulation of the robot applying the PD+ control for the generation of trajectories through the interaction of the EMG signal.
A graphical user interface is designed as indicated in Figure 18b with visual feedback of the EMG signal, the result of the state machine by means of a green indicator that indicates the position detected of the MNN’s classification, the control curves resulting from the implemented PD+ and a simulation of the virtual robot that indicates the position of the end effector. In Figure 18b, the user connection and the operation of the interface to calculate the response time metrics are indicated.

4. Results and Discussions

The EMG signal classification method that allows for the generation of coordinates for the trajectory control of a manipulator robot has been developed. The user’s ability to follow a series of point-to-point coordinates previously determined by colors is measured according to the time of sustained contraction. The yellow dot indicates the starting point of the test, the green dots indicate the path to be followed and the blue dot indicates the end point to which the robot’s end effector must reach. Two trajectories are proposed that increase the difficulty indicating a penalty each time the user enters a contraction command other than the one indicated. The time in which the user generates the trajectory is also recorded. The test ends when the user generates the trajectory without penalties. In Figure 19a, the first proposed trajectory is indicated, in Figure 19b, the time and the number of penalties for each test performed by the user are presented and in Figure 19c, a graph of the response time for trajectory 1 is indicated.
A downward trend is observed in this first trajectory in the response time when completing the test with zero penalties. It is shown that 11 repetitions are enough to successfully complete the proposed trajectory. At the beginning, it indicates an initial time of 118.52 s, and, at the end, it indicates an initial time of 77.54 s, which corresponds to a decrease in the response time by 34.58%. In Figure 19a, the second proposed trajectory is indicated. Figure 19b shows the time and the number of penalties for each test performed by the user. Figure 19c indicates a graph of the response time for trajectory 2.
In the second trajectory, a behavior similar to the first trajectory is observed. With 11 repetitions, it is enough to successfully complete the test. At the beginning, an initial time of 188.64 s is indicated, and, at the end, an initial time of 111.99 s is indicated, which corresponds to a decrease in the response time by 40.64%. When performing the test with different points, the same trend is observed in the decrease in response time. By around 11 repetitions, the user has mastery of the HMI. It should be noted that the model is customized for each user according to individual characteristics and muscle contraction time in addition to adding a recognition class for involuntary movements that blocks the operation of the robot and takes it to a “home” state.

5. Conclusions

An HMI that allows for the classification of muscular signals according to the contraction time has been designed. The model implemented through a neural network allows for the personalization and classification in real time for the generation of movement commands of a virtual robot. The HMI can be implemented with inexperienced users who need only 11 repetitions to master the operation of the system, reducing the learning curve. The future work of this project is to implement the classification of multiclass signals in a physical robotic system. In assistive systems or bionic prostheses, although there is the limitation that, being a discrete system, the movement command is determined by a state machine, the improvement consists of implementing neurofuzzy systems that allow for the generation of continuous trajectories in the robot. The development of assistance systems through physiological signals is important for people with disabilities since it allows them to better adapt to their work or personal environment.

Author Contributions

Conceptualization: F.P.-R.; Methodology: F.P.-R. and C.C. Investigation: F.P.-R. Writing—original draft preparation: F.P.-R. Writing—review and editing: F.P.-R. and N.M.-L. Supervision: C.G.-G. and N.F.-V. Project administration: E.L.-N. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki, and approved by Technical Secretary of the internal committee for monitoring and compliance with rationality, discipline and ethics measures of the Universidad del Valle de México, on March 22nd, 2022.

Informed Consent Statement

Ethical review and approval was not required for the study in human participants according to local legislation and institutional requirements. The patients/participants gave their written informed consent to participate in this study. Written informed consent was obtained from the person(s) for the publication of any images or potentially identifiable data included in this article.

Data Availability Statement

All datasets generated for this study are included in the paper.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Palumbo, A.; Vizza, P.; Calabrese, B.; Ielpo, N. Biopotential Signal Monitoring Systems in Rehabilitation: A Review. Sensors 2021, 21, 7172. [Google Scholar] [CrossRef] [PubMed]
  2. Laksono, P.; Matsushita, K.; Suhaimi, M.; Kitamura, T.; Njeri, W.; Muguro, J.; Sasaki, M. Mapping Three Electromyography Signals Generated by Human Elbow and Shoulder Movements to Two Degree of Freedom Upper-Limb Robot Control. Robotics 2020, 9, 83. [Google Scholar] [CrossRef]
  3. Zhu, K.; Guo, W.; Yang, G.; Li, Z.; Wu, H. High-Fidelity Recording of EMG Signals by Multichannel On-Skin Electrode Arrays from Target Muscles for Effective Human–Machine Interfaces. ACS Appl. Electron. Mater. 2021, 3, 1350–1358. [Google Scholar] [CrossRef]
  4. Aljalal, M.; Ibrahim, S.; Djemal, R.; Ko, W. Comprehensive review on brain-controlled mobile robots and robotic arms based on electroencephalography signals. Intell. Serv. Robot. 2020, 13, 539–563. [Google Scholar] [CrossRef]
  5. Lu, Z.; Tong, K.-Y.; Shin, H.; Li, S.; Zhou, P. Advanced Myoelectric Control for Robotic Hand-Assisted Training: Outcome from a Stroke Patient. Front. Neurol. 2017, 8, 8. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  6. Benchabane, S.; Saadia, N.; Ramdane-Cherif, A. Novel algorithm for conventional myocontrol of upper limbs prosthetics. Biomed. Signal Process. Control 2020, 57, 101791. [Google Scholar] [CrossRef]
  7. Rasool, G.; Iqbal, K.; Bouaynaya, N.; White, G. Real-Time Task Discrimination for Myoelectric Control Employing Task-Specific Muscle Synergies. IEEE Trans. Neural Syst. Rehabil. Eng. 2015, 24, 98–108. [Google Scholar] [CrossRef] [PubMed]
  8. Karabulut, D.; Ortes, F.; Arslan, Y.Z.; Adli, M.A. Comparative evaluation of EMG signal features for myoelectric controlled human arm prosthetics. Biocybern. Biomed. Eng. 2017, 37, 326–335. [Google Scholar] [CrossRef]
  9. Hwang, H.-J.; Hahne, J.M.; Müller, K.-R. Real-time robustness evaluation of regression based myoelectric control against arm position change and donning/doffing. PLoS ONE 2017, 12, e0186318. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  10. Farina, D.; Jiang, N.; Rehbaum, H.; Holobar, A.; Graimann, B.; Dietl, H.; Aszmann, O.C. The Extraction of Neural Information from the Surface EMG for the Control of Upper-Limb Prostheses: Emerging Avenues and Challenges. IEEE Trans. Neural Syst. Rehabil. Eng. 2014, 22, 797–809. [Google Scholar] [CrossRef] [PubMed]
  11. Scheme, E.; Englehart, K. Electromyogram pattern recognition for control of powered upper-limb prostheses: State of the art and challenges for clinical use. J. Rehabil. Res. Dev. 2011, 48, 643–659. [Google Scholar] [CrossRef] [PubMed]
  12. Hargrove, L.J.; Scheme, E.J.; Englehart, K.B.; Hudgins, B.S. Multiple Binary Classifications via Linear Discriminant Analysis for Improved Controllability of a Powered Prosthesis. IEEE Trans. Neural Syst. Rehabil. Eng. 2010, 18, 49–57. [Google Scholar] [CrossRef] [PubMed]
  13. Hahne, J.M.; Graimann, B.; Mueller, K.-R. Spatial Filtering for Robust Myoelectric Control. IEEE Trans. Biomed. Eng. 2012, 59, 1436–1443. [Google Scholar] [CrossRef] [PubMed]
  14. Vidovic, M.M.-C.; Hwang, H.-J.; Amsuss, S.; Hahne, J.M.; Farina, D.; Muller, K.-R. Improving the Robustness of Myoelectric Pattern Recognition for Upper Limb Prostheses by Covariate Shift Adaptation. IEEE Trans. Neural Syst. Rehabil. Eng. 2015, 24, 961–970. [Google Scholar] [CrossRef] [PubMed]
  15. Jiang, N.; Tian, L.; Fang, P.; Dai, Y.; Li, G. Motion recognition for simultaneous control of multifunctional transradial prostheses. In Proceedings of the 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Osaka, Japan, 3–7 July 2013. [Google Scholar] [CrossRef]
  16. Smith, L.H.; Hargrove, L.J. Comparison of surface and intramuscular EMG pattern recognition for simultaneous wrist/hand motion classification. In Proceedings of the 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Osaka, Japan, 3–7 July 2013. [Google Scholar] [CrossRef]
  17. Prahm, C.; Eckstein, K.; Ortiz-Catalan, M.; Dorffner, G.; Kaniusas, E.; Aszmann, O.C. Combining two open source tools for neural computation (BioPatRec and Netlab) improves movement classification for prosthetic control. BMC Res. Notes 2016, 9, 429. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  18. Abbaspour, S.; Naber, A.; Ortiz-Catalan, M.; GholamHosseini, H.; Lindén, M. Real-Time and Offline Evaluation of Myoelectric Pattern Recognition for the Decoding of Hand Movements. Sensors 2021, 21, 5677. [Google Scholar] [CrossRef] [PubMed]
  19. Li, X.; Samuel, O.W.; Zhang, X.; Wang, H.; Fang, P.; Li, G. A motion-classification strategy based on sEMG-EEG signal combination for upper-limb amputees. J. Neuroeng. Rehabil. 2017, 14, 1–13. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  20. Tsai, A.-C.; Hsieh, T.-H.; Luh, J.-J.; Lin, T.-T. A comparison of upper-limb motion pattern recognition using EMG signals during dynamic and isometric muscle contractions. Biomed. Signal Process. Control 2014, 11, 17–26. [Google Scholar] [CrossRef]
  21. Webster, J.G.; Clark, J.W. Medical Instrumentation: Application and Design, 18th ed.; Wiley: New York, NY, USA, 1998. [Google Scholar]
  22. Pérez-Reynoso, F.D.; Rodríguez-Guerrero, L.; Salgado-Ramírez, J.C.; Ortega-Palacios, R. Human–Machine Interface: Multiclass Classification by Machine Learning on 1D EOG Signals for the Control of an Omnidirectional Robot. Sensors 2021, 21, 5882. [Google Scholar] [CrossRef] [PubMed]
Figure 1. Components of the motor neuron, [21].
Figure 1. Components of the motor neuron, [21].
Sensors 22 03424 g001
Figure 2. (a) Electrode placement diagram and the AD620 instrumentation amplifier, (b) Physical representation of the EMG signal acquisition protocol and the Silver/Silver Chloride (Ag/AgCl) electrode implemented.
Figure 2. (a) Electrode placement diagram and the AD620 instrumentation amplifier, (b) Physical representation of the EMG signal acquisition protocol and the Silver/Silver Chloride (Ag/AgCl) electrode implemented.
Sensors 22 03424 g002
Figure 3. (a) Biopac® System, (b) Database obtained from the Biopac of the EMG signal with sustained isometric contraction of 5 s.
Figure 3. (a) Biopac® System, (b) Database obtained from the Biopac of the EMG signal with sustained isometric contraction of 5 s.
Sensors 22 03424 g003
Figure 4. Amplification module and basal corrector.
Figure 4. Amplification module and basal corrector.
Sensors 22 03424 g004
Figure 5. Filter in second order Butterworth configuration at 40 dB/decade.
Figure 5. Filter in second order Butterworth configuration at 40 dB/decade.
Sensors 22 03424 g005
Figure 6. (a) EMG signal acquisition cards, (b) EMG signal response in the Tektronix oscilloscope.
Figure 6. (a) EMG signal acquisition cards, (b) EMG signal response in the Tektronix oscilloscope.
Sensors 22 03424 g006
Figure 7. First order low pass filter and its transfer function as a function of the complex variable s.
Figure 7. First order low pass filter and its transfer function as a function of the complex variable s.
Sensors 22 03424 g007
Figure 8. Filtered EMG signal.
Figure 8. Filtered EMG signal.
Sensors 22 03424 g008
Figure 9. Architecture of the EMG signal classification method for the control of a manipulator robot.
Figure 9. Architecture of the EMG signal classification method for the control of a manipulator robot.
Sensors 22 03424 g009
Figure 10. Graphical representation of the data set (input vector p1×n). The output vector T1×n stores the labels of each class using integer data.
Figure 10. Graphical representation of the data set (input vector p1×n). The output vector T1×n stores the labels of each class using integer data.
Sensors 22 03424 g010aSensors 22 03424 g010b
Figure 11. Structure of the Multilayer Neural Network.
Figure 11. Structure of the Multilayer Neural Network.
Sensors 22 03424 g011
Figure 12. Graph of the accuracy trend of the neural network with new data (Train loss) and the trend of the loss function (Val loss). Neural network accuracy ratio after 3000 epochs.
Figure 12. Graph of the accuracy trend of the neural network with new data (Train loss) and the trend of the loss function (Val loss). Neural network accuracy ratio after 3000 epochs.
Sensors 22 03424 g012
Figure 13. The table presents the inputs of the digital system, and the system outputs are indicated by means of the robot diagram.
Figure 13. The table presents the inputs of the digital system, and the system outputs are indicated by means of the robot diagram.
Sensors 22 03424 g013
Figure 14. Eight-state Mealy-type machine, transition indicated.
Figure 14. Eight-state Mealy-type machine, transition indicated.
Sensors 22 03424 g014
Figure 15. Virtual model of an anthropomorphic robot with three degrees of freedom.
Figure 15. Virtual model of an anthropomorphic robot with three degrees of freedom.
Sensors 22 03424 g015
Figure 16. Block diagram of PD + control with gravity compensation.
Figure 16. Block diagram of PD + control with gravity compensation.
Sensors 22 03424 g016
Figure 17. (a) Graphs of the position error with a tendency to zero for each of the joint coordinates q 1 , q 2 , q 3 and control curves indicating the operation of the PD+ to reach the desired positions, (b) Result of the PD+ trajectory control of an anthropomorphic virtual robot.
Figure 17. (a) Graphs of the position error with a tendency to zero for each of the joint coordinates q 1 , q 2 , q 3 and control curves indicating the operation of the PD+ to reach the desired positions, (b) Result of the PD+ trajectory control of an anthropomorphic virtual robot.
Sensors 22 03424 g017
Figure 18. (a) Implementation of the real-time acquisition system interacting with the virtual robot simulation, (b) Graphical interface designed to record response time metrics.
Figure 18. (a) Implementation of the real-time acquisition system interacting with the virtual robot simulation, (b) Graphical interface designed to record response time metrics.
Sensors 22 03424 g018aSensors 22 03424 g018b
Figure 19. (a) Point-to-point trajectories (Trajectory 1 and Trajectory 2), (b) The time and the number of penalties (Trajectory 1 and Trajectory 2) and (c) Plot of trend response for each trajectory (Trajectory 1 and Trajectory 2).
Figure 19. (a) Point-to-point trajectories (Trajectory 1 and Trajectory 2), (b) The time and the number of penalties (Trajectory 1 and Trajectory 2) and (c) Plot of trend response for each trajectory (Trajectory 1 and Trajectory 2).
Sensors 22 03424 g019aSensors 22 03424 g019bSensors 22 03424 g019c
Table 1. Description of each of the desired positions of each state.
Table 1. Description of each of the desired positions of each state.
StateInput EMGDesired Value in MetersDesired Movement
p x p y p z
S1IN10−0.340.38Stop
S2IN20−0.110.46Up
S3IN40.34−0.340.38Right
S4IN7−0.34−0.340.38Left
DULIN8−0.34−0.110.46Diagonal Up Left
DURIN30.34−0.110.46Diagonal Up Right
DDLIN6−0.34−0.340.28Diagonal Down Left
DDRIN50.34−0.340.28Diagonal Down Right
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Pérez-Reynoso, F.; Farrera-Vazquez, N.; Capetillo, C.; Méndez-Lozano, N.; González-Gutiérrez, C.; López-Neri, E. Pattern Recognition of EMG Signals by Machine Learning for the Control of a Manipulator Robot. Sensors 2022, 22, 3424. https://doi.org/10.3390/s22093424

AMA Style

Pérez-Reynoso F, Farrera-Vazquez N, Capetillo C, Méndez-Lozano N, González-Gutiérrez C, López-Neri E. Pattern Recognition of EMG Signals by Machine Learning for the Control of a Manipulator Robot. Sensors. 2022; 22(9):3424. https://doi.org/10.3390/s22093424

Chicago/Turabian Style

Pérez-Reynoso, Francisco, Neín Farrera-Vazquez, César Capetillo, Nestor Méndez-Lozano, Carlos González-Gutiérrez, and Emmanuel López-Neri. 2022. "Pattern Recognition of EMG Signals by Machine Learning for the Control of a Manipulator Robot" Sensors 22, no. 9: 3424. https://doi.org/10.3390/s22093424

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop