Next Article in Journal
Development of a Microwave Sensor for Solid and Liquid Substances Based on Closed Loop Resonator
Next Article in Special Issue
Game-Based Dual-Task Exercise Program for Children with Cerebral Palsy: Blending Balance, Visuomotor and Cognitive Training: Feasibility Randomized Control Trial
Previous Article in Journal
Subjective and Objective User Behavior Disparity: Towards Balanced Visual Design and Color Adjustment
Previous Article in Special Issue
Computer Game-Based Telerehabilitation Platform Targeting Manual Dexterity: Exercise Is Fun. “You Are Kidding—Right?”
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Perspective

Combining Action Observation Treatment with a Brain–Computer Interface System: Perspectives on Neurorehabilitation

1
Department of Electronics and Telecommunications, Politecnico di Torino, 10129 Turin, Italy
2
Fondazione Don Carlo Gnocchi, Piazzale dei Servi 3, 43100 Parma, Italy
3
Division of Neuroscience, IRCCS San Raffaele Scientific Institute, University San Raffaele, Via Olgettina 60, 20132 Milan, Italy
*
Author to whom correspondence should be addressed.
Sensors 2021, 21(24), 8504; https://doi.org/10.3390/s21248504
Submission received: 18 October 2021 / Revised: 30 November 2021 / Accepted: 17 December 2021 / Published: 20 December 2021

Abstract

:
Action observation treatment (AOT) exploits a neurophysiological mechanism, matching an observed action on the neural substrates where that action is motorically represented. This mechanism is also known as mirror mechanism. In a typical AOT session, one can distinguish an observation phase and an execution phase. During the observation phase, the patient observes a daily action and soon after, during the execution phase, he/she is asked to perform the observed action at the best of his/her ability. Indeed, the execution phase may sometimes be difficult for those patients where motor impairment is severe. Although, in the current practice, the physiotherapist does not intervene on the quality of the execution phase, here, we propose a stimulation system based on neurophysiological parameters. This perspective article focuses on the possibility to combine AOT with a brain–computer interface system (BCI) that stimulates upper limb muscles, thus facilitating the execution of actions during a rehabilitation session. Combining a rehabilitation tool that is well-grounded in neurophysiology with a stimulation system, such as the one proposed, may improve the efficacy of AOT in the treatment of severe neurological patients, including stroke patients, Parkinson’s disease patients, and children with cerebral palsy.

1. Towards Translational, Evidence-Based Approaches in Neurorehabilitation

There is increasing demand in neurorehabilitation for approaches aimed at helping patients to recover functions, and improve their capacity to face daily activities and social interactions.
Following this general aim, any approach in neurorehabilitation should have at least three main features [1,2]:
  • Firstly, it should be evidence based. The efficacy of any rehabilitation practice should be supported by the results merging from randomized controlled studies or clinical trials, comparing a specific approach with a control condition.
  • A neurorehabilitation approach needs to be grounded in neurophysiology: every approach should have its theoretical background in physiology principles and mechanisms. For example, when speaking about motor recovery, the terrific advance of knowledge regarding the organization and functions of the motor system coming from basic neuroscience should be taken into account. All approaches should consider neuroscientific studies to transfer knowledge in clinical practice.
  • Any approach in neurorehabilitation should also aim at the recovery of functions and, as a consequence, of the capacity of patients to interact with the environment and other people, as assumed when considering health not only as the condition in which individuals are free from diseases [3]. It is worth stressing that in many cases, physiotherapists focus on ways to circumvent functional deficits, suggesting alternative strategies in order to allow patients to face daily activities. This attitude leads to a compensation or a reeducation of functions, rather than a cure for them through remediation. In contrast with this rather diffuse attitude, we believe rehabilitative tools should aim at restoring the neural structures whose damage caused the impaired functions, or activating supplementary or related pathways that may perform the original functions.
There are indeed well-known and widespread approaches in neurorehabilitation that fully fit these criteria. One example is constraint-induced movement therapy (CIMT). By means of CIMT, patients promote the use of the more-affected extremities, while restricting the use of the less-affected limb. More specifically, they perform repetitive goal-oriented tasks with the affected limb 6–7 h per day, while wearing a mitt to avoid the use of the less-affected arm for 90% of their waking hours. However, the feasibility of this practice is questioned because patients experience difficulty in complying, due to the fatigue that results from long periods of practice and restraint [4]. CIMT finds application in patients with acute and chronic stroke [4,5], and in children with cerebral palsy, where a gentler version is used than for adults [6]. CIMT induces brain plastic changes, thus leading to a functional reorganization of sensorimotor circuits in monkeys [7].
Another example is the so-called mirror therapy. In this treatment, a mirror is positioned on a table in the patient’s midsagittal plane. Patients place the affected limb behind a mirror and the unaffected limb in front of the mirror. In this way, when they move the unaffected extremity, they get the feeling (a sort of proprioceptive and kinesthetic sensation) to move the affected limb. Treatment normally lasts 10 consecutive weekdays. This approach is effective to relieve phantom pain in arm amputees, as well as in the recovery of upper limbs in chronic stroke patients [8,9]. More recently, it has been effectively applied in the rehabilitation of children with cerebral palsy [10]. The authors that originally proposed mirror therapy as a rehabilitative tool have also suggested that the underlying neurophysiological basis is the mirror mechanism (see below).
Motor imagery is the capacity of individuals to imagine seeing themselves performing a particular action, while generating the kinesthetic experience of the movement, in the absence of motor output [11]. This mental capacity has been practiced for years in sport training and neurorehabilitation. An early study showed that motor imagery may improve posture in elderly people [12]. More recently, using motor imagery, good results have been obtained in the recovery of stroke patients [13,14]. Motor imagery has also revealed a promising approach in Parkinson’s disease patients [15]. It is acknowledged that during motor imagery, similar motor representations are re-enacted as during action execution and action observation, thus suggesting a potential “equivalence” of motor execution, motor imagery, and action observation [16,17]. Hence, all these functions have the potential to serve as rehabilitative tools. However, some studies have suggested that patients with damage to specific brain structures, including the parietal and frontal lobes, lose the capacity to imagine motorically [18]. For this reason, in these patients the use of motor imagery (MI) as a rehabilitative tool can be questionable.

2. Action Observation Treatment and Its Efficacy in Clinical Practice

There is increasing empirical evidence that when individuals observe actions performed by other people, they automatically activate the brain neural structures responsible for the actual execution of the seen action [19]. This capacity to match an observed action on its motor representation is defined as the mirror mechanism. At the neural level, this mechanism is most likely encoded by mirror neurons [20]. These neurons were first found in monkey premotor cortexes, and are discharged during both the execution and observation of an action, typically directed towards an object. The existence of a mirror mechanism can explain why healthy adults can learn and/or improve the performance of a specific motor task by observing other people executing it [21,22,23,24,25,26].
Action observation treatment (AOT) is a rehabilitation tool exploiting the mirror mechanism and its potential role in learning and/or re-learning specific motor tasks for the recovery of motor functions, in both neurological and non-neurological patients [27]. During a typical AOT rehabilitation session, patients practice a single daily action (e.g., in one session they practice having an espresso, in another they wash their hands, and so on). Practiced actions are chosen among those of great relevance in everyday life. An AOT rehabilitation session consists of an observation phase and an execution phase. During the observation phase, the patient sits in front of a computer screen and has to carefully observe a video clip, depicting the daily action to be practiced in that session. The presented action can be divided into four motor acts. By motor acts, we mean the different motor segments in which a daily action can be divided. For example, washing one’s hands can be divided into the following motor acts: (i) applying soap on wet hands; (ii) scrubbing hands for a few seconds; (iii) rinsing hands under running water; (iv) drying them with a towel. Each motor act is typically seen for 3 min, so that the whole duration of a video clip depicting the specific daily action is 12 min. In the video, each motor act is performed by both an actor and an actress, and is seen from different perspectives (frontal or lateral view, in foreground and background). This is to make the videos more interesting and relevant, from an attentional point of view. The importance of showing actions from different perspectives is supported by a monkey study [28], where the authors found that most mirror neurons are sensitive to different visual perspectives. Moreover, in a recent review, the authors suggested that different perspectives may be helpful for different motors skills. For example, first-person perspective could be helpful for manual dexterity tasks (showing more visual cues, which are important to perform the task successfully) and for the embodiment of the movements. On the other hand, third-person could be best for more gross motor tasks [29]. After observing each motor act for 3 min (observation phase), patients move to the execution phase, when they must perform for 2 min what they have just seen. Although patients are requested to execute the observed motor act at the best of their ability, they are informed that the focus of the treatment is on the observation of the action, not its execution. During the execution phase, objects used in the video clip are provided at hand, in order to allow patients to execute the observed action in a realistic context. Note that objects are known to automatically trigger the most useful motor representations that can be acted upon them, thus further contributing to the reorganization of the motor system [30,31,32]. Moreover, there is empirical evidence that the recruitment of the motor system is fine-tuned with the motorically relevant features of an observed object [33].
A typical AOT rehabilitation session lasts half an hour. The physiotherapist explains the task to the patient for a few minutes (e.g., look carefully at the movie, pay attention to all the details of presented actions) to motivate him/her. Then, 12 min are devoted to the observation phase (3 min for each presented motor act) and finally 8 min to the execution phase (2 min for each motor act). For adults, the rehabilitation program with AOT includes the practice of 20 daily actions, and takes 4 weeks (5 working days a week).
Thus far, AOT has been used in the rehabilitation of patients suffering from chronic ischemic stroke (more than six months after the acute event), in Parkinson’s disease patients, in children with cerebral palsy, and in orthopedic patients undergoing surgery of the hip or knee (for a comprehensive review see [34]). In a pivotal, randomized controlled study of patients with chronic ischemic stroke in the territory of the middle cerebral artery [35], AOT was applied to recover upper limb motor functions. Patients in the control group had to observe video clips with no specific motor content. The functional outcome of both groups was assessed by means of the stroke impact scale, the Wolf motor function test, and the Frenchay arm test. After treatment, the patients undergoing AOT achieved better scores than patients in the control group, in all scales. An improvement was still evident at the two-month follow-up. Moreover, a functional magnetic resonance imaging (fMRI) study, carried out during an independent motor task, namely free object manipulation, before and after AOT treatment, showed a significant increase in the activation of brain areas endowed with a mirror mechanism in the AOT patients after treatment.
The effectiveness of AOT has been investigated also in PD patients, with the aim to complement drug treatment in these patients [36]. Similar to stroke patients, participants in the case group observed videos depicting everyday life actions, including postural actions and walking, whereas those in the control group observed movies devoid of any specific motor content. After treatment, patients in the case group scored better than controls on two functional scales: the unified Parkinson’s disease rating scale (UPDRS) and the functional independence measure (FIM). AOT has also been successfully applied in remediation of freezing of gait in Parkinson’s disease (PD) patients [37]. These findings can be explained by the evidence that the basal ganglia, the neural structures most affected in PD, are heavily connected with the brain areas involved in the mirror mechanism, as well as in motor learning and motor planning [38]. It is most likely that in PD patients, AOT may contribute to reorganizing the normal loop circuits connecting the motor cortex with the basal ganglia [39]. In PD patients, during action observation, changes in beta oscillatory activity of the subthalamic nucleus occur, similar to those recorded over the motor cortex, thus suggesting that basal ganglia may play a role in the mirror mechanism [40]. Furthermore, it is well-known that PD patients improve their motor performances when they obtain an external cue [41]. One could argue that the visual stimulation provided by means of AOT represents an appropriate cue to start and execute several daily actions in PD patients. Moreover, in addition to cueing actions, AOT could also reinforce the normal connections in the brain between the prefrontal and premotor cortex. These connections play a fundamental role in paying attention to and performing actions in a convenient context in healthy people, and appear compromised in PD patients [42].
AOT has been applied for the recovery of upper limb motor functions in children with cerebral palsy [43]. In this study, children in the case group observed daily actions appropriate for their age, whereas children in the control group observed documentaries with no specific motor content. A functional evaluation with the Melbourne assessment scale of upper limb motor functions showed that children undergoing AOT scored significantly better than the controls after treatment. Other authors have confirmed the effectiveness of AOT in the recovery of upper limb motor functions in children with cerebral palsy [44,45]. Moreover, in a recent fMRI study [46], children treated with AOT showed stronger activation in brain areas subserving the execution of actions, and implying the use of objects. These findings support the notion that AOT contributes to the reorganization of brain circuits subserving the impaired function, rather than activating supplementary or vicariating circuits.
Interestingly, AOT may also promote motor recovery in post-surgical orthopedic patients for hip fractures or elective hip or knee replacements [47]. In a pivotal study, patients in the experimental group observed video clips depicting daily actions performed with lower limbs and subsequently imitated them. Patients in the control group observed video clips with no motor content, and then executed the same actions as the AOT group. Two functional scales (FIM and Tinetti scale) were used to assess the functional outcome. After treatment, patients in the AOT group scored better than patients in the control group in both functional scales. Moreover, patients in the case group were prescribed a walker less frequently than controls at discharge. These findings suggest that AOT is an effective adjunct to conventional therapy in the rehabilitation of post-surgical orthopedic patients. In more general terms, the findings of this study suggest a top-down effect in neurorehabilitation, showing that the reorganization of motor representations at central level, most likely occurring during AOT, may positively affect performance, even when the skeletal structures to implement actions are affected.
Finally, it is worth underlining that AOT has been tested as a tool on the rehabilitation of aphasic patients. A case report study supports the notion that the observation and execution of actions can improve the recall of action words in patients with a selective deficit for verb retrieval [48].
For the aim of the present study, it is relevant to underline that AOT is a flexible tool in neurorehabilitation: in fact, actions trained during rehabilitation sessions can be adapted to the real needs of patients. For example, patients impaired in performing distal hand/arm actions (i.e., grasping, manipulating) should practice these motor tasks, whereas those presenting with impairment of proximal arm actions (i.e., reaching objects, coding objects in space) should focus on these motor tasks. Moreover, this individual training program has the potential to be applied in a tele-rehabilitation setting. Tele-rehabilitation exploits telecommunication devices to provide evaluation, distance support and eventually rehabilitation for patients living at home. Despite the fact that so far tele-rehabilitation has been mainly applied in the rehabilitation of stroke patients [49,50,51], it has the potential to meet the rehabilitation needs of most patients, including neurological and non-neurological patients. Just for example, in the field of neurology, a recent review suggests the potential use of tele-rehabilitation for patients with multiple sclerosis [52], whereas in the field of internal medicine the effectiveness of tele-rehabilitation for patients with chronic respiratory disease has also been assessed [53].
Most current studies have applied AOT in conventional rehabilitation settings (at hospitals or in rehabilitation centers). However, since AOT focuses on the delivery of visual stimuli depicting actions, it can also be easily applied at patients’ home in a tele-rehabilitation setting, under remote control if necessary or appropriate. Caregivers and patients can be easily trained to its use, and they can be provided with all the necessary materials and requisites. Two recent studies support the use of AOT in a tele-rehabilitation setting in children [54,55]. Applying AOT in patient’s home has advantages, when patients must follow a long-term rehabilitation program and/or when they have difficulty in joining rehabilitation centers. The current health emergency due to the COVID-19 pandemic has brought further the use of home-based rehabilitation strategies; in this context, tele-rehabilitation can help to stop the spread of contagion, while assuring patients with motor impairment the possibility of following a regular rehabilitation program.

3. Combining AOT with a Brain–Computer Interface to Improve the Actual Motor Execution of Patients

So far, during the execution phase, patients have been requested to execute the observed motor act to the best of their abilities. Indeed, in a typical AOT protocol, physiotherapists do not intervene to improve patients’ performance, but they ask patients to pay attention to all details of the observed actions, and try to motivate them in the execution phase, when all objects presented in the videoclips are at hand, so that patients can act upon them as they would in everyday life contexts. However, in this study, we propose a stimulation system that should help patients, especially those who are not able to reproduce the observed motor act, or are discouraged from implementing the performance in a physiological manner by an inadequate execution.
Among the therapies involving electrical currents, Functional Electrical Stimulation (FES) induces muscle contraction by exciting the axonal fibers, which innervate the muscular belly [56,57]. The FES low-energy pulses are applied on the skin’s surface, using superficial electrodes, making the approach entirely non-invasive and comfortable [58]. Multi-session FES therapy has been demonstrated to restore motor task functionalities, promote blood circulation, and prevent muscle atrophy, resulting in a significant improvement of daily life for people suffering from neurological disorders [59].
An active control of the FES application could be implemented by monitoring and processing the neurophysiological signals that are the most descriptive of the planning and execution of the movement. Indeed, by assessing the electrical activity of the central nervous system at certain brain regions (e.g., premotor cortex, primary motor cortex), which is known as ElectroEncephaloGraphy (EEG), or by supervising the contraction of the skeletal muscles, represented by the ElectroMyoGraphic (sEMG) signal, bio-mimetic FES-controlled systems have been realized [60,61,62].
Considering the central role of EEG in analyzing the brain response to AOT [63,64,65], a brain–computer interface (BCI) could be developed to control the FES application according to a specific EEG activity [66,67]. In particular, depending on the subject possibilities, the control could be based on both motor imagery (MI) [68,69] and movement execution (ME) approaches, which have been demonstrated to activate almost the same cortical regions [17]. Either way, similarly to AO, an activity change of the mu and beta EEG waves, known as event-related desynchronization (ERD), is induced [70]. However, considering the difficulties related to MI (e.g., long training is required to command the BCI, and it is impossible to assess the correctness of mental training [27,71]), ME has been identified as a more reliable technique for the implementation of the FES control, considering that it includes both motion preparation and actuation phases, thus covering the whole execution process. Although other physiological signals could be used as BCI inputs (e.g., visual stimuli [72]), AO, MI, and ME approaches have been proven to be more beneficial in neurorehabilitation, because the combination of simultaneous cortical activity and muscle stimulation significantly promotes neuroplasticity [73,74].
A representative block-scheme of the architecture of a typical EEG-based BCI-FES system is represented in Figure 1. The detection of the EEG signal could be easily assessed by using standard 32–64 channels EEG caps, which provide the enhanced electrode positions of the International 10–20 systems. Raw EEG signals require the application of a band-pass filter (1 Hz–40 Hz), in order to remove the DC offset and higher frequency interferences (e.g., 50 Hz–60 Hz power line) [70,75]. Depending on the position of the sensing EEG channels, additional signal artifacts (e.g., heartbeat, blinking) should be taken into account [76,77]. After the digitization of the EEG, the power of the signal related to the frequency bands of mu (8 Hz–13 Hz) and beta waves (16 Hz–31 Hz) [70] are estimated, using the well-known mathematical operators of the fast Fourier transform (FFT) or the Wavelet Transform (WT) [75]. A proper ERD identification can be performed by employing machine learning algorithms based on, for example, the neural network (NN) or support vector machines (SVM) classifiers, sometimes combined with common spatial patterns (CSP) for feature extraction [68,75]. At the last stage, the FES application is efficiently controlled depending on the classifier’s output, which discerns between no movement preparation (no need for FES) and volitional action (starting FES).
In this context, the modulation of the FES parameters based on the EEG signal remain an open scenario. Indeed, many works [66,67,78] apply the stimulation with a strict on/off mechanism, without including any modulation of the features of the FES pulses (e.g., amplitude, width). Among the reasons for this is that the EEG represents what is happening at the central nervous system during a specific task only macroscopically; therefore, this makes it challenging to deconstruct this information into the functional data at a peripheral level. As a clear example, we can consider a complex and synergic action like the reaching of an object: the EEG signal provides high-level information about the preparation and execution of the movement (through the analysis of mu and beta waves) but does not allow the discrimination of which muscle is activated (and in which way) to perform the action. For this reason, since the FES operates at the peripheral level by directly stimulating the muscle fibers, a constant per muscle update of the FES parameters could not be easily obtained.
A more efficient FES modulation could be achieved by monitoring the activity of the skeletal muscles involved in the desired movement. The acquisition of the sEMG signal is performed using non-invasive electrodes placed on the skin above the muscle whose activity must be analyzed. The main information content of sEMG is contained in the 50 Hz–150 Hz band, and, from a high-level perspective, its conditioning involves a high-pass filter to remove the DC offset and motion artifacts, one or more amplification stages, and a low-pass filter for antialiasing purposes and high-frequency noise suppression [79,80]. Methods to extract useful features about muscle activity include standard practices, such as sEMG envelope extraction [81,82], and other quantitative evaluators, like the average threshold crossing event-driven approach [83], which are more suitable for low-power and wearable systems [84,85].
In the system proposed in this paper, the sEMG signals will be used as feedback in order to achieve a closed-loop system. In particular, through sEMG it is possible to assess which muscles are contributing to the execution of the movement, so that FES can be applied only to inactive muscles, in order to both encourage voluntary activity and prevent atrophy of unused muscles. Moreover, the sEMG feedback can help to monitor the uprising of muscle fatigue, which is a typical phenomenon related to FES application, because of the opposite order in the muscle fiber recruitment, compared with the physiological one (i.e., the Henneman’s size principle) [86,87]. Among the popular practices for evaluating muscle fatigue, the most employed techniques are the analysis of the down-shift of the median frequency of the sEMG signal spectrum [88], and the analysis of the M-wave (i.e., the FES-evoked EMG response), which provides the quantification of the number of recruited motor units [89,90].
However, combining the sEMG and FES techniques requires to implement appropriate strategies for muscle activity extraction (both the voluntary and the evoked one) and stimulus artifact suppression [91,92], which would otherwise result in saturating the sEMG-sensing electronics [93]. A possible strategy to solve this issue could be to implement at the inputs of the acquisition channel a blanking circuit [93,94], which would have to be appropriately synchronized with the stimulation frequency, and which would also have to discharge the acquisition electrodes to avoid transient responses disturbing the sEMG reading [90,93].
Furthermore, the trajectory evolution of the movement could be monitored by inertial measurement units (IMUs), which, integrating different sensors (i.e., accelerometers, gyroscopes, magnetometers) into a single wearable chip, would allow an accurate kinematic analysis for an object moving in a 3D space [95]. Additionally, since activities of daily living (ADLs) include complex and multi-domain actions (e.g., grasping an object, scratching a shoulder, drinking from a bottle), the presence of sensors to identify when a task is completed could be used to provide feedback to the patient, in order to encourage him/her during the rehabilitation session. Key technologies for implementing this control can range from radio-frequency identification (RFID) [96], through capacitive sensors [97], to body channel communication (BCC) [98,99].
In its early stage of use, the system will be employed in clinics under the supervision of trained medical personnel, requiring a full sensor configuration to obtain robust results in a short time; thus, not burdening the finances of the medical structure. In a second phase, for patients with a neurosensory involvement already established during earlier clinical sessions, or for patients who are uncomfortable in repeatedly getting to the hospital, the system could be used at home for long term rehabilitation in a reduced configuration, in order to ease sensor placement by the patient or the caregiver. In particular, since the main FES-pattern modulation is performed by sEMG acquisition, the EEG and the BCC hardware can be removed from the overall system, thus reducing the physical dimensions of the home equipment.
In our main configuration, as depicted Figure 2, each patient will be equipped with a wearable helmet for the acquisition of the EEG rhythm, some muscular inertial units, which embed sEMG and motion tracking together, an electro-stimulator for the application of FES, and the control software (which could be installed by the user on his/her own computer or, if necessary, on a provided one). The system will receive, via Bluetooth low-energy (BLE) communications, the data from each module, process them, and appropriately modulate the FES parameters (e.g., pulses amplitude and width, stimulation frequency). Furthermore, the software features a graphical user interface, to ease the communication with both the physiotherapist (e.g., session progression, sensors data) and the patient (e.g., displayed messages), and plays the videoclips needed in the observation phase. However, if the patient subjects himself/herself to AOT at home, a properly educated caregiver must be involved, in order to assist him/her during the setup of each sub-device (e.g., ensuring proper electrode placement) and its connection to the central processing unit.
The proposed system’s final aim is to effectively make the subject execute the desired action. Indeed, in the main case scenario, the devices will be activated at the beginning of each execution phase, in order to monitor the physiological signals, and the FES will take action only if the patient attempts to perform the movement without succeeding, in order to encourage voluntary muscle activity. Alternatively, if the EEG monitoring is omitted, the FES onset can be remotely triggered by the therapist, or the patient can automatically follow the task indication.
In particular, as detailed in Figure 3, the main scenario includes the sensors being continuously monitored to obtain useful information about the patient state, and consequently performing different feature extractions:
  • EEG activity is analyzed considering its power density in the beta and mu bands, aiming to detect whether an ERD occurs and to define its magnitude;
  • Muscles activity is monitored by the sEMG sensors, distinguishing between each acquisition channel (i.e., different muscular fiber recruitment) and evaluating if the relation among them reflects the physiological behavior;
  • From sEMG, muscular fatigue assessment is performed too, analyzing the M-waves from the different muscles, and considering its degradation over time;
  • Position, angular velocity, and linear acceleration from the different employed IMUs are combined to reconstruct the limb kinematic across space, evaluating if they are consistent with physiological movements.
The extracted features are sent to a data processing unit, which decides whether the FES has to be applied, and how to tune the different channel parameters to make the movement as natural as possible. A machine learning algorithm takes action at this point to automatically analyze the features together. Thus, having stored several previous data acquired from healthy and unhealthy subjects, the system will be able to recognize the conditions of the patient and will instruct the FES module to update the stimulation parameters accordingly. For example, the machine could recognize different ERD magnitudes and increase the FES current, if the acquired value is lower than the one saved as the physiological standard. Alternatively, it could receive unexpected frequency information by the sEMG sensors and decide to vary the pulse width and frequency of the generated stimulation waves, to relax muscle fatigue.
Lastly, if the performed action requires reaching for an object, BCC or RFID sensors can be involved in detecting the effective extent of the movement itself, giving visual feedback to the user. An encouragement message is displayed when no contact is detected, motivating the subject to complete the exercise, and proceed with the routine. At the end, when the target object is touched, positive feedback is represented and rest time is observed, before subsequent execution.
In the home-based scenario, the data acquired by the sensors are automatically saved in the local storage for subsequent analyses. Furthermore, if the clinician would like to verify in real time the progress of the patient, he/she could activate a data streaming towards their facility, in order to monitor the physiological signals acquired during movement execution.

4. Conclusions

In the present perspective article, we proposed a novel way to apply action observation treatment (AOT) in clinical practice. More specifically, we proposed the use of a brain–computer interface (BCI) with the aim to stimulate upper limb muscles during the execution phase of AOT, when patients are requested to execute an observed motor act.
Combining AOT with BCI may present advantages for the patients during the execution phase of AOT. BCI may improve the performance of patients, whatever the gravity of their neurological impairment. Considering the neurological diseases where AOT has been widely applied, with this system we propose that we can stimulate motor activity in stroke patients with upper limb paralysis, severe Parkinson’s disease patients or in children with a severe cerebral palsy.
Like AOT alone, the combination of AOT with BCI has the potential to be used in conventional rehabilitation settings, as well as at home, possibly under remote control. In this case, caregivers could receive information on how to apply electrodes for stimulating muscles under the supervision of clinicians, or even without it, when enough experience has been reached. It is noteworthy that this approach could favor an increased awareness of patients and caregivers in the treatment process, and promote an active role in the cure. At the same time, national healthcare systems could spare resources with advantages for the whole of society.

Author Contributions

Conceptualization: D.D. and G.B., Investigation: F.R., F.S. and A.P.; Writing—original draft: F.R., F.S., A.P., A.M. and G.B. Writing—review: F.R., F.S., A.P., A.M. and G.B. Visualization: F.R. and A.M. Supervision and project administration: D.D. and G.B. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Taub, E.; Uswatte, G.; Elbert, T. New treatments in neurorehabilitation founded on basic research. Nat. Rev. Neurosci. 2002, 3, 228–236. [Google Scholar] [CrossRef] [Green Version]
  2. Small, S.L.; Buccino, G.; Solodkin, A. Brain repair after stroke—a novel neurological model. Nat. Rev. Neurol. 2013, 9, 698–707. [Google Scholar] [CrossRef] [PubMed]
  3. World Health Organization. International Classification of Functioning, Disability and Health: ICF; World Health Organization: Geneva, Switzerland, 2001; ISBN 978-92-4-154542-6. [Google Scholar]
  4. Shi, Y.X.; Tian, J.H.; Yang, K.H.; Zhao, Y. Modified constraint-induced movement therapy versus traditional rehabilitation in patients with upper-extremity dysfunction after stroke: A systematic review and meta-analysis. Arch. Phys. Med. Rehabil. 2011, 92, 972–982. [Google Scholar] [CrossRef] [PubMed]
  5. Corbetta, D.; Sirtori, V.; Castellini, G.; Moja, L.; Gatti, R. Constraint-induced movement therapy for upper extremities in people with stroke. Cochrane Database Syst. Rev. 2015, CD004433. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  6. Sakzewski, L.; Gordon, A.; Eliasson, A.-C. The state of the evidence for intensive upper limb therapy approaches for children with unilateral cerebral palsy. J. Child Neurol. 2014, 29, 1077–1090. [Google Scholar] [CrossRef] [PubMed]
  7. Nudo, R.J. Mechanisms for recovery of motor function following cortical damage. Curr. Opin. Neurobiol. 2006, 16, 638–644. [Google Scholar] [CrossRef] [PubMed]
  8. Ramachandran, V.S.; Altschuler, E.L. The use of visual feedback, in particular mirror visual feedback, in restoring brain function. Brain J. Neurol. 2009, 132, 1693–1710. [Google Scholar] [CrossRef] [Green Version]
  9. Thieme, H.; Mehrholz, J.; Pohl, M.; Behrens, J.; Dohle, C. Mirror therapy for improving motor function after stroke. Stroke 2013, 44, e1–e2. [Google Scholar] [CrossRef] [Green Version]
  10. Bruchez, R.; Jequier Gygax, M.; Roches, S.; Fluss, J.; Jacquier, D.; Ballabeni, P.; Grunt, S.; Newman, C.J. Mirror therapy in children with hemiparesis: A randomized observer-blinded trial. Dev. Med. Child Neurol. 2016, 58, 970–978. [Google Scholar] [CrossRef]
  11. Jeannerod, M. The representing brain: Neural correlates of motor intention and imagery. Behav. Brain Sci. 1994, 17, 187–202. [Google Scholar] [CrossRef] [Green Version]
  12. Linden, C.A.; Uhley, J.E.; Smith, D.; Bush, M.A. The effects of mental practice on walking balance in an elderly population. Occup. Ther. J. Res. 1989, 9, 155–169. [Google Scholar] [CrossRef]
  13. Butler, A.J.; Page, S.J. Mental practice with motor imagery: Evidence for motor recovery and cortical reorganization after stroke. Arch. Phys. Med. Rehabil. 2006, 87, S2–S11. [Google Scholar] [CrossRef] [Green Version]
  14. Liu, K.P.; Chan, C.C.; Lee, T.M.; Hui-Chan, C.W. Mental imagery for promoting relearning for people after stroke: A randomized controlled trial. Arch. Phys. Med. Rehabil. 2004, 85, 1403–1408. [Google Scholar] [CrossRef]
  15. Tamir, R.; Dickstein, R.; Huberman, M. Integration of Motor Imagery and Physical Practice in Group Treatment Applied to Subjects With Parkinson’s Disease. Neurorehabil. Neural Repair 2007, 21, 68–75. [Google Scholar] [CrossRef]
  16. Jeannerod, M. Neural simulation of action: A unifying mechanism for motor cognition. NeuroImage 2001, 14, S103–S109. [Google Scholar] [CrossRef]
  17. Hardwick, R.M.; Caspers, S.; Eickhoff, S.B.; Swinnen, S.P. Neural correlates of action: Comparing meta-analyses of imagery, observation, and execution. Neurosci. Biobehav. Rev. 2018, 94, 31–44. [Google Scholar] [CrossRef]
  18. McInnes, K.; Friesen, C.; Boe, S. Specific Brain Lesions Impair Explicit Motor Imagery Ability: A Systematic Review of the Evidence. Arch. Phys. Med. Rehabil. 2016, 97, 478–489.e1. [Google Scholar] [CrossRef]
  19. Rizzolatti, G.; Craighero, L. The mirror-neuron system. Annu. Rev. Neurosci. 2004, 27, 169–192. [Google Scholar] [CrossRef] [Green Version]
  20. di Pellegrino, G.; Fadiga, L.; Fogassi, L.; Gallese, V.; Rizzolatti, G. Understanding motor events: A neurophysiological study. Exp. Brain Res. 1992, 91, 176–180. [Google Scholar] [CrossRef]
  21. Vogt, S.; Thomaschke, R. From visuo-motor interactions to imitation learning: Behavioural and brain imaging studies. J. Sports Sci. 2007, 25, 497–517. [Google Scholar] [CrossRef] [Green Version]
  22. Mattar, A.A.G.; Gribble, P.L. Motor learning by observing. Neuron 2005, 46, 153–160. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  23. Stefan, K.; Cohen, L.G.; Duque, J.; Mazzocchio, R.; Celnik, P.; Sawaki, L.; Ungerleider, L.; Classen, J. Formation of a motor memory by action observation. J. Neurosci. Off. J. Soc. Neurosci. 2005, 25, 9339–9346. [Google Scholar] [CrossRef] [PubMed]
  24. Porro, C.A.; Facchin, P.; Fusi, S.; Dri, G.; Fadiga, L. Enhancement of force after action observation: Behavioural and neurophysiological studies. Neuropsychologia 2007, 45, 3114–3121. [Google Scholar] [CrossRef] [PubMed]
  25. Gatti, R.; Tettamanti, A.; Gough, P.M.; Riboldi, E.; Marinoni, L.; Buccino, G. Action observation versus motor imagery in learning a complex motor task: A short review of literature and a kinematics study. Neurosci. Lett. 2013, 540, 37–42. [Google Scholar] [CrossRef] [Green Version]
  26. Bassolino, M.; Campanella, M.; Bove, M.; Pozzo, T.; Fadiga, L. Training the motor cortex by observing the actions of others during immobilization. Cereb. Cortex 2014, 24, 3268–3276. [Google Scholar] [CrossRef]
  27. Buccino, G. Action observation treatment: A novel tool in neurorehabilitation. Philos. Trans. R. Soc. Lond. B. Biol. Sci. 2014, 369, 20130185. [Google Scholar] [CrossRef]
  28. Caggiano, V.; Fogassi, L.; Rizzolatti, G.; Pomper, J.K.; Thier, P.; Giese, M.A.; Casile, A. View-based encoding of actions in mirror neurons of area f5 in macaque premotor cortex. Curr. Biol. CB 2011, 21, 144–148. [Google Scholar] [CrossRef] [Green Version]
  29. Scott, M.W.; Wood, G.; Holmes, P.S.; Williams, J.; Marshall, B.; Wright, D.J. Combined action observation and motor imagery: An intervention to combat the neural and behavioural deficits associated with developmental coordination disorder. Neurosci. Biobehav. Rev. 2021, 127, 638–646. [Google Scholar] [CrossRef]
  30. Gibson, J.J. The Theory of Affordances. In Perceiving, Acting, and Knowing: Toward an Ecological Psychology; Robert E Shaw, J.B., Ed.; Lawrence Erlbaum Associates: Hillsdale, MI, USA , 1977; pp. 67–82. [Google Scholar]
  31. Grafton, S.T.; Fadiga, L.; Arbib, M.A.; Rizzolatti, G. Premotor cortex activation during observation and naming of familiar tools. NeuroImage 1997, 6, 231–236. [Google Scholar] [CrossRef] [Green Version]
  32. Grèzes, J.; Tucker, M.; Armony, J.; Ellis, R.; Passingham, R.E. Objects automatically potentiate action: An fMRI study of implicit processing. Eur. J. Neurosci. 2003, 17, 2735–2740. [Google Scholar] [CrossRef]
  33. Buccino, G.; Sato, M.; Cattaneo, L.; Rodà, F.; Riggio, L. Broken affordances, broken objects: A TMS study. Neuropsychologia 2009, 47, 3074–3078. [Google Scholar] [CrossRef]
  34. Ryan, D.; Fullen, B.; Rio, E.; Segurado, R.; Stokes, D.; O’Sullivan, C. Effect of Action Observation Therapy in the Rehabilitation of Neurologic and Musculoskeletal Conditions: A Systematic Review. Arch. Rehabil. Res. Clin. Transl. 2021, 3, 100106. [Google Scholar] [CrossRef]
  35. Ertelt, D.; Small, S.; Solodkin, A.; Dettmers, C.; McNamara, A.; Binkofski, F.; Buccino, G. Action observation has a positive impact on rehabilitation of motor deficits after stroke. NeuroImage 2007, 36 (Suppl. 2), T164–T173. [Google Scholar] [CrossRef]
  36. Buccino, G.; Gatti, R.; Giusti, M.C.; Negrotti, A.; Rossi, A.; Calzetti, S.; Cappa, S.F. Action observation treatment improves autonomy in daily activities in Parkinson’s disease patients: Results from a pilot study. Mov. Disord. Off. J. Mov. Disord. Soc. 2011, 26, 1963–1964. [Google Scholar] [CrossRef]
  37. Pelosin, E.; Avanzino, L.; Bove, M.; Stramesi, P.; Nieuwboer, A.; Abbruzzese, G. Action observation improves freezing of gait in patients with Parkinson’s disease. Neurorehabil. Neural Repair 2010, 24, 746–752. [Google Scholar] [CrossRef]
  38. Doyon, J.; Bellec, P.; Amsel, R.; Penhune, V.; Monchi, O.; Carrier, J.; Lehéricy, S.; Benali, H. Contributions of the basal ganglia and functionally related brain structures to motor learning. Behav. Brain Res. 2009, 199, 61–75. [Google Scholar] [CrossRef]
  39. Obeso, J.A.; Rodríguez-Oroz, M.C.; Benitez-Temino, B.; Blesa, F.J.; Guridi, J.; Marin, C.; Rodriguez, M. Functional organization of the basal ganglia: Therapeutic implications for Parkinson’s disease. Mov. Disord. Off. J. Mov. Disord. Soc. 2008, 23 (Suppl. 3), S548–S559. [Google Scholar] [CrossRef]
  40. Alegre, M.; Rodríguez-Oroz, M.C.; Valencia, M.; Pérez-Alcázar, M.; Guridi, J.; Iriarte, J.; Obeso, J.A.; Artieda, J. Changes in subthalamic activity during movement observation in Parkinson’s disease: Is the mirror system mirrored in the basal ganglia. Clin. Neurophysiol. 2010, 121, 414–425. [Google Scholar] [CrossRef]
  41. Jahanshahi, M.; Jenkins, I.H.; Brown, R.G.; Marsden, C.D.; Passingham, R.E.; Brooks, D.J. Self-initiated versus externally triggered movements. I. An investigation using measurement of regional cerebral blood flow with PET and movement-related potentials in normal and Parkinson’s disease subjects. Brain J. Neurol. 1995, 118 Pt 4, 913–933. [Google Scholar] [CrossRef]
  42. Rowe, J.; Stephan, K.E.; Friston, K.; Frackowiak, R.; Lees, A.; Passingham, R. Attention to action in Parkinson’s disease: Impaired effective connectivity among frontal cortical regions. Brain J. Neurol. 2002, 125, 276–289. [Google Scholar] [CrossRef] [Green Version]
  43. Buccino, G.; Arisi, D.; Gough, P.; Aprile, D.; Ferri, C.; Serotti, L.; Tiberti, A.; Fazzi, E. Improving upper limb motor functions through action observation treatment: A pilot study in children with cerebral palsy. Dev. Med. Child Neurol. 2012, 54, 822–828. [Google Scholar] [CrossRef] [PubMed]
  44. Sgandurra, G.; Ferrari, A.; Cossu, G.; Guzzetta, A.; Fogassi, L.; Cioni, G. Randomized Trial of Observation and Execution of Upper Extremity Actions Versus Action Alone in Children With Unilateral Cerebral Palsy. Neurorehabil. Neural Repair 2013, 27, 808–815. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  45. Kim, J.-Y.; Kim, J.-M.; Ko, E.-Y. The effect of the action observation physical training on the upper extremity function in children with cerebral palsy. J. Exerc. Rehabil. 2014, 10, 176–183. [Google Scholar] [CrossRef] [PubMed]
  46. Buccino, G.; Molinaro, A.; Ambrosi, C.; Arisi, D.; Mascaro, L.; Pinardi, C.; Rossi, A.; Gasparotti, R.; Fazzi, E.; Galli, J. Action Observation Treatment Improves Upper Limb Motor Functions in Children with Cerebral Palsy: A Combined Clinical and Brain Imaging Study. Neural Plast. 2018, 2018, 4843985. [Google Scholar] [CrossRef]
  47. Bellelli, G.; Buccino, G.; Bernardini, B.; Padovani, A.; Trabucchi, M. Action observation treatment improves recovery of postsurgical orthopedic patients: Evidence for a top-down effect? Arch. Phys. Med. Rehabil. 2010, 91, 1489–1494. [Google Scholar] [CrossRef]
  48. Marangolo, P.; Bonifazi, S.; Tomaiuolo, F.; Craighero, L.; Coccia, M.; Altoè, G.; Provinciali, L.; Cantagallo, A. Improving language without words: First evidence from aphasia. Neuropsychologia 2010, 48, 3824–3833. [Google Scholar] [CrossRef]
  49. Chen, J.; Jin, W.; Zhang, X.X.; Xu, W.; Liu, X.N.; Ren, C.C. Telerehabilitation Approaches for Stroke Patients: Systematic Review and Meta-analysis of Randomized Controlled Trials. J. Stroke Cerebrovasc. Dis. 2015, 24, 2660–2668. [Google Scholar] [CrossRef]
  50. Sarfo, F.S.; Ulasavets, U.; Opare-Sem, O.K.; Ovbiagele, B. Tele-Rehabilitation after Stroke: An Updated Systematic Review of the Literature. J. Stroke Cerebrovasc. Dis. 2018, 27, 2306–2318. [Google Scholar] [CrossRef]
  51. Laver, K.E.; Adey-Wakeling, Z.; Crotty, M.; Lannin, N.A.; George, S.; Sherrington, C. Telerehabilitation services for stroke. Cochrane Database Syst. Rev. 2020. [Google Scholar] [CrossRef]
  52. Khan, F.; Amatya, B.; Kesselring, J.; Galea, M. Telerehabilitation for persons with multiple sclerosis. Cochrane Database Syst. Rev. 2015. [Google Scholar] [CrossRef]
  53. Cox, N.S.; Corso, S.D.; Hansen, H.; McDonald, C.F.; Hill, C.J.; Zanaboni, P.; Alison, J.A.; O’Halloran, P.; Macdonald, H.; Holland, A.E. Telerehabilitation for chronic respiratory disease. Cochrane Database Syst. Rev. 2021. [Google Scholar] [CrossRef]
  54. Nuara, A.; Avanzini, P.; Rizzolatti, G.; Fabbri-Destro, M. Efficacy of a home-based platform for child-to-child interaction on hand motor function in unilateral cerebral palsy. Dev. Med. Child Neurol. 2019, 61, 1314–1322. [Google Scholar] [CrossRef]
  55. Molinaro, A.; Micheletti, S.; Pagani, F.; Garofalo, G.; Galli, J.; Rossi, A.; Fazzi, E.; Buccino, G. Action Observation Treatment in a tele-rehabilitation setting: A pilot study in children with cerebral palsy. Disabil. Rehabil. 2020, 1–6. [Google Scholar] [CrossRef]
  56. LYNCH, C.L.; POPOVIC, M.R. Functional Electrical Stimulation. IEEE Control Syst. Mag. 2008, 28, 40–50. [Google Scholar] [CrossRef] [Green Version]
  57. Marquez-Chin, C.; Popovic, M.R. Functional electrical stimulation therapy for restoration of motor function after spinal cord injury and stroke: A review. Biomed. Eng. OnLine 2020, 19, 34. [Google Scholar] [CrossRef]
  58. Kapadia, N.; Moineau, B.; Popovic, M.R. Functional Electrical Stimulation Therapy for Retraining Reaching and Grasping After Spinal Cord Injury and Stroke. Front. Neurosci. 2020, 14, 718. [Google Scholar] [CrossRef]
  59. Eraifej, J.; Clark, W.; France, B.; Desando, S.; Moore, D. Effectiveness of upper limb functional electrical stimulation after stroke for the improvement of activities of daily living and motor function: A systematic review and meta-analysis. Syst. Rev. 2017, 6, 40. [Google Scholar] [CrossRef] [Green Version]
  60. Jiang, S.; Chen, L.; Wang, Z.; Xu, J.; Qi, C.; Qi, H.; He, F.; Ming, D. Application of BCI-FES system on stroke rehabilitation. In Proceedings of the 2015 7th International IEEE/EMBS Conference on Neural Engineering (NER), Montpellier, France, 22–24 April 2015; pp. 1112–1115. [Google Scholar]
  61. Broniera Junior, P.; Campos, D.P.; Lazzaretti, A.E.; Nohama, P.; Carvalho, A.A.; Krueger, E.; Minhoto Teixeira, M.C. EEG-FES-Force-MMG closed-loop control systems of a volunteer with paraplegia considering motor imagery with fatigue recognition and automatic shut-off. Biomed. Signal Process. Control 2021, 68, 102662. [Google Scholar] [CrossRef]
  62. Rossi, F.; Motto Ros, P.; Rosales, R.M.; Demarchi, D. Embedded Bio-Mimetic System for Functional Electrical Stimulation Controlled by Event-Driven sEMG. Sensors 2020, 20, 1535. [Google Scholar] [CrossRef] [Green Version]
  63. Kuk, E.-J.; Kim, J.-M.; Oh, D.-W.; Hwang, H.-J. Effects of action observation therapy on hand dexterity and EEG-based cortical activation patterns in patients with post-stroke hemiparesis. Top. Stroke Rehabil. 2016, 23, 318–325. [Google Scholar] [CrossRef]
  64. Kim, S.-H.; Cho, J.-S. Action Observation and Cortical Connectivity: Evidence from EEG Analysis. J. Korean Phys. Ther. 2016, 28, 398–407. [Google Scholar] [CrossRef] [Green Version]
  65. Ge, S.; Wang, P.; Liu, H.; Lin, P.; Gao, J.; Wang, R.; Iramina, K.; Zhang, Q.; Zheng, W. Neural Activity and Decoding of Action Observation Using Combined EEG and fNIRS Measurement. Front. Hum. Neurosci. 2019, 13, 357. [Google Scholar] [CrossRef] [Green Version]
  66. Marquez-Chin, C.; Marquis, A.; Popovic, M.R. EEG-Triggered Functional Electrical Stimulation Therapy for Restoring Upper Limb Function in Chronic Stroke with Severe Hemiplegia. Case Rep. Neurol. Med. 2016, 2016, e9146213. [Google Scholar] [CrossRef] [Green Version]
  67. Likitlersuang, J.; Koh, R.; Gong, X.; Jovanovic, L.; Bolivar-Tellería, I.; Myers, M.; Zariffa, J.; Márquez-Chin, C. EEG-Controlled Functional Electrical Stimulation Therapy With Automated Grasp Selection: A Proof-of-Concept Study. Top. Spinal Cord Inj. Rehabil. 2018, 24, 265–274. [Google Scholar] [CrossRef]
  68. Fumanal-Idocin, J.; Wang, Y.-K.; Lin, C.-T.; Fernández, J.; Sanz, J.A.; Bustince, H. Motor-Imagery-Based Brain Computer Interface using Signal Derivation and Aggregation Functions. IEEE Trans. Cybern. 2021, 1–12. [Google Scholar] [CrossRef]
  69. Lotze, M.; Halsband, U. Motor imagery. J. Physiol. Paris 2006, 99, 386–395. [Google Scholar] [CrossRef]
  70. Chaisaen, R.; Autthasan, P.; Mingchinda, N.; Leelaarporn, P.; Kunaseth, N.; Tammajarung, S.; Manoonpong, P.; Mukhopadhyay, S.C.; Wilaiprasitporn, T. Decoding EEG Rhythms During Action Observation, Motor Imagery, and Execution for Standing and Sitting. IEEE Sens. J. 2020, 20, 13776–13786. [Google Scholar] [CrossRef]
  71. Alimardani, M.; Nishio, S.; Ishiguro, H. Brain-Computer Interface and Motor Imagery Training: The Role of Visual Feedback and Embodiment; IntechOpen: London, UK, 2018; ISBN 978-1-78984-070-4. [Google Scholar]
  72. Ruiz-Olaya, A.F.; Antelis, J.M.; Cerquera, A. Decoding Steady-State Visual Evoked Potentials From EEG Signals: Towards an EEG-Triggered FES System to Restore Hand Grasp Function. In Proceedings of the 2019 IEEE 10th Annual Ubiquitous Computing, Electronics Mobile Communication Conference (UEMCON), New York, NY, USA, 10–12 October 2019; pp. 1152–1157. [Google Scholar]
  73. McGie, S.C.; Zariffa, J.; Popovic, M.R.; Nagai, M.K. Short-term neuroplastic effects of brain-controlled and muscle-controlled electrical stimulation. Neuromodulation Technol. Neural Interface 2015, 18, 233–240, discussion 240. [Google Scholar] [CrossRef]
  74. Biasiucci, A.; Leeb, R.; Iturrate, I.; Perdikis, S.; Al-Khodairy, A.; Corbet, T.; Schnider, A.; Schmidlin, T.; Zhang, H.; Bassolino, M.; et al. Brain-actuated functional electrical stimulation elicits lasting arm motor recovery after stroke. Nat. Commun. 2018, 9, 2421. [Google Scholar] [CrossRef]
  75. Padfield, N.; Zabalza, J.; Zhao, H.; Masero, V.; Ren, J. EEG-Based Brain-Computer Interfaces Using Motor-Imagery: Techniques and Challenges. Sensors 2019, 19, 1423. [Google Scholar] [CrossRef] [Green Version]
  76. Zhang, S.; McIntosh, J.; Shadli, S.M.; Neo, P.S.-H.; Huang, Z.; McNaughton, N. Removing eye blink artefacts from EEG—A single-channel physiology-based method. J. Neurosci. Methods 2017, 291, 213–220. [Google Scholar] [CrossRef] [PubMed]
  77. Jiang, X.; Bian, G.-B.; Tian, Z. Removal of Artifacts from EEG Signals: A Review. Sensors 2019, 19, 987. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  78. Gant, K.; Guerra, S.; Zimmerman, L.; Parks, B.A.; Prins, N.W.; Prasad, A. EEG-controlled functional electrical stimulation for hand opening and closing in chronic complete cervical spinal cord injury. Biomed. Phys. Eng. Express 2018, 4, 065005. [Google Scholar] [CrossRef]
  79. De Luca, C.J. Surface Electromyography: Detection and Recording. Available online: https://www.delsys.com/downloads/TUTORIAL/semg-detection-and-recording.pdf (accessed on 30 November 2021).
  80. De Luca, G. Fundamental Concepts in EMG Signal Acquisition. Available online: https://www.delsys.com/downloads/TUTORIAL/fundamental-concepts-in-emg-signal-acquisition.pdf (accessed on 30 November 2021).
  81. Prakash, A.; Kumari, B.; Sharma, S. A low-cost, wearable sEMG sensor for upper limb prosthetic application. J. Med. Eng. Technol. 2019, 43, 235–247. [Google Scholar] [CrossRef]
  82. Liang, S.; Wu, Y.; Chen, J.; Zhang, L.; Chen, P.; Chai, Z.; Cao, C. Identification of Gesture Based on Combination of Raw sEMG and sEMG Envelope Using Supervised Learning and Univariate Feature Selection. J. Bionic Eng. 2019, 16, 647–662. [Google Scholar] [CrossRef]
  83. Sapienza, S.; Crepaldi, M.; Motto Ros, P.; Bonanno, A.; Demarchi, D. On Integration and Validation of a Very Low Complexity ATC UWB System for Muscle Force Transmission. IEEE Trans. Biomed. Circuits Syst. 2016, 10, 497–506. [Google Scholar] [CrossRef]
  84. Rossi, F.; Motto Ros, P.; Sapienza, S.; Bonato, P.; Bizzi, E.; Demarchi, D. Wireless Low Energy System Architecture for Event-Driven Surface Electromyography. In Applications in Electronics Pervading Industry, Environment and Society; Saponara, S., De Gloria, A., Eds.; Springer: Cham, Switzerland, 2018. [Google Scholar]
  85. Mongardi, A.; Motto Ros, P.; Rossi, F.; Ruo Roch, M.; Martina, M.; Demarchi, D. A Low-Power Embedded System for Real-Time sEMG based Event-Driven Gesture Recognition. In Proceedings of the 2019 26th IEEE International Conference on Electronics, Circuits and Systems (ICECS); Genova, Italy, 27–29 November 2019; pp. 65–68. [Google Scholar]
  86. Masani, K.; Popovic, M.R. Functional Electrical Stimulation in Rehabilitation and Neurorehabilitation. In Springer Handbook of Medical Technology; Kramme, R., Hoffmann, K.-P., Pozos, R.S., Eds.; Springer: Berlin/Heidelberg, Germany, 2011; pp. 877–896. ISBN 978-3-540-74658-4. [Google Scholar]
  87. Henneman, E. Relation between size of neurons and their susceptibility to discharge. Science 1957, 126, 1345–1347. [Google Scholar] [CrossRef]
  88. De Luca, C.J. The Use of Surface Electromyography in Biomechanics. J. Appl. Biomech. 1997, 13, 135–163. [Google Scholar] [CrossRef] [Green Version]
  89. Schauer, T.; Vrontos, A. Modeling of Mixed Artificially and Voluntary Induced Muscle Contractions for Controlled Functional Electrical Stimulation of Shoulder Abduction. IFAC-Pap. 2019, 51, 284–289. [Google Scholar] [CrossRef]
  90. Li, Y.; Chen, J.; Yang, Y. A Method for Suppressing Electrical Stimulation Artifacts from Electromyography. Int. J. Neural Syst. 2019, 29, 1850054. [Google Scholar] [CrossRef]
  91. de Sousa, A.C.C.; Valtin, M.; Bó, A.P.L.; Schauer, T. Automatic Detection of Stimulation Artifacts to Isolate Volitional from Evoked EMG Activity. IFAC-PapersOnLine 2018, 51, 282–287. [Google Scholar] [CrossRef]
  92. Li, Z.; Guiraud, D.; Andreu, D.; Gelis, A.; Fattal, C.; Hayashibe, M. Real-Time Closed-Loop Functional Electrical Stimulation Control of Muscle Activation with Evoked Electromyography Feedback for Spinal Cord Injured Patients. Int. J. Neural Syst. 2018, 28, 1750063. [Google Scholar] [CrossRef]
  93. Bi, Z.-Y.; Xie, C.-X.; Zhou, Y.-X.; Wang, H.-P.; Lu, X.-Y.; Wang, Z.-G. An Anti Stimulation Artifacts and M-waves Surface Electromyography Detector with a Short Blanking Time. In Proceedings of the 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Montreal, QC, Canada, 20–24 July 2020. [Google Scholar] [CrossRef]
  94. Sa-E, S.; Freeman, C.; Yang, K. Model-Based Control of FES Embedding Simultaneous Volitional EMG Measurement; IEEE: Piscataway, NJ, USA, 2018; pp. 480–485. [Google Scholar]
  95. Pérez, R.; Costa, Ú.; Torrent, M.; Solana, J.; Opisso, E.; Cáceres, C.; Tormos, J.M.; Medina, J.; Gómez, E.J. Upper limb portable motion analysis system based on inertial technology for neurorehabilitation purposes. Sensors 2010, 10, 10733–10751. [Google Scholar] [CrossRef] [Green Version]
  96. Lee, Y.; Liu, X.; Gummeson, J.; Lee, S. A Wearable RFID System to Monitor Hand Use for Individuals with Upper Limb Paresis. In Proceedings of the 2019 IEEE 16th International Conference on Wearable and Implantable Body Sensor Networks (BSN), Chicago, IL, USA, 19-22 May 2019. [Google Scholar] [CrossRef]
  97. Kang, M.; Kim, J.; Jang, B.; Chae, Y.; Kim, J.-H.; Ahn, J.-H. Graphene-Based Three-Dimensional Capacitive Touch Sensor for Wearable Electronics. ACS Nano 2017, 11, 7950–7957. [Google Scholar] [CrossRef]
  98. Varga, V.; Vakulya, G.; Sample, A.; Gross, T. Enabling Interactive Infrastructure with Body Channel Communication. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 2017, 1, 169. [Google Scholar] [CrossRef]
  99. Crepaldi, M.; Barcellona, A.; Zini, G.; Ansaldo, A.; Motto Ros, P.; Sanginario, A.; Cuccu, C.; De Marchi, D.; Brayda, L. Live Wire—A Low-Complexity Body Channel Communication System for Landmark Identification. IEEE Trans. Emerg. Top. Comput. 2020, 9, 1248–1264. [Google Scholar] [CrossRef]
Figure 1. Typical BCI acquisition chain. After acquiring the EEG signal, the frequency components outside the band of interest are filtered out. Then, the power of the signal is estimated, followed by the extraction of its features to be used to classify if FES needs to be applied or not.
Figure 1. Typical BCI acquisition chain. After acquiring the EEG signal, the frequency components outside the band of interest are filtered out. Then, the power of the signal is estimated, followed by the extraction of its features to be used to classify if FES needs to be applied or not.
Sensors 21 08504 g001
Figure 2. Overview of the system. The patient is instructed to perform the reaching of an object, typically. EEG electrodes are placed on the motor areas and stabilized by a comfortable helmet. sEMG and inertial sensors are placed on the limb of interest, next to the stimulation electrodes. A central unit processes acquired data to activate the FES when the subject needs help to reach the target. A monitor provides feedbacks encouraging the user.
Figure 2. Overview of the system. The patient is instructed to perform the reaching of an object, typically. EEG electrodes are placed on the motor areas and stabilized by a comfortable helmet. sEMG and inertial sensors are placed on the limb of interest, next to the stimulation electrodes. A central unit processes acquired data to activate the FES when the subject needs help to reach the target. A monitor provides feedbacks encouraging the user.
Sensors 21 08504 g002
Figure 3. Activity flow of an execution session. After the indication of the task to perform, the subject is required to execute the action. The four sensors are continuously monitored, and for each of them a proper feature is extracted and recorded (e.g., signal power from EEG, limb trajectory from the IMUs). The information obtained by EEG, EMG and IMUs is combined and processed to evaluate how the subject body is reacting, and to decide if the FES must be applied to assist the execution. Therefore, FES parameters are tuned depending on the decision of the processing stage, stabilizing the movement by stimulating one or more muscles, if necessary. Lastly, if the task consists of reaching for an object, the subject is encouraged until the BCC sensors detect the touch of the target.
Figure 3. Activity flow of an execution session. After the indication of the task to perform, the subject is required to execute the action. The four sensors are continuously monitored, and for each of them a proper feature is extracted and recorded (e.g., signal power from EEG, limb trajectory from the IMUs). The information obtained by EEG, EMG and IMUs is combined and processed to evaluate how the subject body is reacting, and to decide if the FES must be applied to assist the execution. Therefore, FES parameters are tuned depending on the decision of the processing stage, stabilizing the movement by stimulating one or more muscles, if necessary. Lastly, if the task consists of reaching for an object, the subject is encouraged until the BCC sensors detect the touch of the target.
Sensors 21 08504 g003
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Rossi, F.; Savi, F.; Prestia, A.; Mongardi, A.; Demarchi, D.; Buccino, G. Combining Action Observation Treatment with a Brain–Computer Interface System: Perspectives on Neurorehabilitation. Sensors 2021, 21, 8504. https://doi.org/10.3390/s21248504

AMA Style

Rossi F, Savi F, Prestia A, Mongardi A, Demarchi D, Buccino G. Combining Action Observation Treatment with a Brain–Computer Interface System: Perspectives on Neurorehabilitation. Sensors. 2021; 21(24):8504. https://doi.org/10.3390/s21248504

Chicago/Turabian Style

Rossi, Fabio, Federica Savi, Andrea Prestia, Andrea Mongardi, Danilo Demarchi, and Giovanni Buccino. 2021. "Combining Action Observation Treatment with a Brain–Computer Interface System: Perspectives on Neurorehabilitation" Sensors 21, no. 24: 8504. https://doi.org/10.3390/s21248504

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop