Next Article in Journal
Comparative Study of γ- and e-Radiation-Induced Effects on FBGs Using Different Femtosecond Laser Inscription Methods
Next Article in Special Issue
Validation of Visually Identified Muscle Potentials during Human Sleep Using High Frequency/Low Frequency Spectral Power Ratios
Previous Article in Journal
Toward Modeling Psychomotor Performance in Karate Combats Using Computer Vision Pose Estimation
Previous Article in Special Issue
Does the Score on the MRC Strength Scale Reflect Instrumented Measures of Maximal Torque and Muscle Activity in Post-Stroke Survivors?
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

A Coupled Piezoelectric Sensor for MMG-Based Human-Machine Interfaces

by
Mateusz Szumilas
*,
Michał Władziński
and
Krzysztof Wildner
Warsaw University of Technology, Faculty of Mechatronics, Institute of Metrology and Biomedical Engineering, A. Boboli 8 St., 02-525 Warsaw, Poland
*
Author to whom correspondence should be addressed.
Sensors 2021, 21(24), 8380; https://doi.org/10.3390/s21248380
Submission received: 31 October 2021 / Revised: 6 December 2021 / Accepted: 13 December 2021 / Published: 15 December 2021

Abstract

:
Mechanomyography (MMG) is a technique of recording muscles activity that may be considered a suitable choice for human–machine interfaces (HMI). The design of sensors used for MMG and their spatial distribution are among the deciding factors behind their successful implementation to HMI. We present a new design of a MMG sensor, which consists of two coupled piezoelectric discs in a single housing. The sensor’s functionality was verified in two experimental setups related to typical MMG applications: an estimation of the force/MMG relationship under static conditions and a neural network-based gesture classification. The results showed exponential relationships between acquired MMG and exerted force (for up to 60% of the maximal voluntary contraction) alongside good classification accuracy (94.3%) of eight hand motions based on MMG from a single-site acquisition at the forearm. The simplification of the MMG-based HMI interface in terms of spatial arrangement is rendered possible with the designed sensor.

1. Introduction

Mechanomyography (MMG) is a measurement technique used to record muscles activity based on vibrations arising as an effect of muscle fibers mechanical contractions [1,2,3]. This technique is still less popular, especially in clinical applications, compared to electromyography (EMG). Nevertheless, the area of application of mechanomyography is relatively wide [1,2,4]: from human-machine interfaces (HMI), especially prosthetic devices control [5,6] and gesture recognition [7,8,9], to investigations of physiological principles of neuro-muscular system functions [10,11,12]. MMG does not require an electrical connection to the skin, therefore it may be applied without prior skin preparation in unconditioned environments, and the provided response exhibits low variability over time regarding the skin condition, as long as the sensor position is not adjusted [13,14]. Differences between the EMG and MMG signals may be observed, e.g., in their fatigue- and force-related responses, with some examples of MMG showing greater sensitivity than EMG when changes in muscle activation strategies are examined [15,16]. The determinants of proper MMG implementation are actively investigated, among others, in order to eliminate crosstalk from neighboring muscles, improve repeatability and signal-to-noise ratio of acquired signals [17,18,19].
Mechanical vibrations originating from muscle activity can be converted to electrical signal via various types of transducers: accelerometers, microphones, or laser distance sensors [2,14,20]. The main differences are in the methods of ensuring a stable coupling between the sensor and the muscle, which directly affect the achieved frequency response of the MMG setup. Accelerometers are often used for MMG recording due to their low mass, small dimensions, and availability of sensors with integrated signal conditioning and digital output. Among microphones, either contact or non-contact ones might be used. For low-cost design, piezoelectric discs in the role of contact microphones sometimes are the choice because of their wide availability, good sensitivity, and very low price. They might be placed directly on the surface of the skin, above the muscle of interest [21,22]. However, with minimal effort, the way they are used can be modified to obtain signals several times higher in amplitude. Piezoelectric elements may be used in a way to benefit from their strong bending, such as in the case of diaphragm piezoelectric microphones or flexural mode accelerometers. Commercial contact microphones based on such an approach have also been used to record mechanomyographic signals previously [2,23,24].
When the myographic signals are to be used for HMI, one of the essential areas of optimization is the number and placement of the sensors used. The interface must be reliable and intuitive but cost-effective at the same time [25]. This enforces endeavours to improve sensing quality, looking for multimodal approaches and new sensors configurations as well. Coupling sensors together is one possible approach, e.g., Silva and Chau [26] coupled microphone and accelerometer, within one sensor, for effective noise reduction in MMG recording; Gregori et al. developed a combined EMG/MMG sensor, which provided improvement, especially in artifact rejection [27]; similarly, Fukuhara and Oka [28] and Wolczowski et al. [5] used a hybrid EMG/MMG sensor. Zhang et al. [29] showed that an introduction of MMG signals might significantly improve the performance of an EMG-pattern recognition-based prosthetic control.
Here, we present a new MMG sensor design, which consists of two coupled piezoelectric discs in a single housing so that two complementary signals related to the muscle activity may be measured simultaneously in a single site. Such an approach brings simplification of the MMG interface in terms of transducers’ spatial distribution.
The sensor functionality has been verified in two typical MMG applications: (a) for estimating a force/MMG relation under isometric conditions and (b) for a neural network-based motion recognition, as in [30,31]. To provide a reference to surface EMG (sEMG), a measurement technique widely accepted in clinical practice, the relationship between MMG and sEMG signals recorded simultaneously during isometric measurements is shown.

2. Materials and Methods

All the data were collected from a single subject (male, 34 years old), who was a member of the research team, free of neuromuscular diseases and prior musculoskeletal injuries.
The proposed MMG sensor consists of two piezoelectric discs in a single housing (Figure 1). The discs are excited by the MMG signal from the skin’s surface by the direct and indirect transmission of mechanical waves. The direct transmission is achieved by pressing the external disc directly to the area of interest on the skin. The indirect transmission is achieved with a coupler. The coupler consists of a base ring, which supports the external disc at its edge, and a hollow pin that contacts the central area of the internal piezoelectric disc, which is supported at four evenly distributed points. Therefore, the internal disc is working in a bending mode. The coupler is supported on flexible hinges, which allow for the sensor’s self-alignment while being attached to the subject. All the mechanical elements of the proposed sensor are 3D-printed with a PLA material (Easy PLA, Fiberlab, Brzezie, Poland) in a fused deposition modeling process. The weight of the sensor is 5.5 g without wires. The photographs of the manufactured sensor are provided in Figure 2. The dimensions of the sensor are determined mainly by the diameters of the piezoelectric discs. Due to the different mechanical interfaces between the skin and both discs, these are intended to exhibit different excitations. The resulting signal from each of them is a superposition of MMG from the acquisition site and a muscle activity coupled from the mounting strap.
The sensor was tested in two experimental setups. In the first one, the force/MMG relation under static conditions was examined with simultaneous acquisition of the sEMG signal. The second test comprised the verification of the sensor’s suitability for the classification of hand motion. Both setups employed 20 mm diameter piezoelectric discs (from unspecified manufacturer).

2.1. Evaluation of the Sensor Performance during Step and Ramp Isometric Contractions

All the measurements were performed under static conditions during a single session. The right arm of the subject was fixed in the measurement setup in 90 ° abduction, 0 ° flexion with 90 ° elbow joint flexion. Forearm was fixed in 0 ° pronation. The measurement setup was based on TAS606 (HT Sensors Technology, Xi’an, China) load cell. For force signal conditioning TBM4 Transbridge amplifier (WPI, Sarasota, FL, USA) was used.
The MMG sensor was placed above the middle part of the biceps brachiibelly. A surface EMG signal was acquired with a pair of wet electrodes (Kendall H92SG, Medtronic, Minneapolis, MN, USA) placed along the muscle and symmetrically with respect to the MMG sensor, 70 mm apart. Since sensor–skin interfaces of low quality are expected in real-world HMI applications, no skin preparation was performed. The reference electrode was placed in equal distance from both active electrodes (around 40 mm). Additionally, metal parts of the setup were grounded for interference noise suppression.
For signals amplification and data acquisition a custom made 8-channel MMG/EMG amplifier was used with active headstage × 100 for EMG probes and high impedance channels for the MMG transducers. For both sensors types, filtering was performed in the signal chain, specifically:
  • EMG signals were high-pass (HP) filtered at the pre-amplification stage followed by an AC coupling, resulting in a 2nd order filter with 2.44 Hz cut-off frequency; the low-pass (LP) filtering was introduced with a 2nd order Sallen-Key topology with 245 Hz cut-off frequency;
  • MMG signals were HP filtered with an AC coupling, which provided 1st order filter with 1.54 Hz cut-off frequency; the LP filter design was the same as for EMG signals, i.e., a 2nd order filter with 245 Hz cut-off frequency.
Signals were digitized using a 12-bit analog-to-digital converter of an STM32L476RG microcontroller (STMicroelectronics, Geneva, Switzerland), sampled at 500 sps from the range 0 ÷ 3.3 V. The digitized signals were transmitted using a serial wired connection to the computer and stored with appropriate labeling using a custom-made LabVIEW (NI, Austin, TX, USA) application.
During measurements the subject used a biofeedback provided by means of a scope (MSO2012B, Tektronix, Beaverton, OR, USA) where the force signal along with the line indicating the target force level (for step measurements) were displayed.
The target force levels were determined in relation to the maximal voluntary contraction (MVC). MVC measurements took place approximately 10 min prior to the step and ramp measurements and were performed with the same measurement setup, with the same limb position as during the following procedures. Two short (~3 s) maximal contractions, with around 3 min of rest to limit fatigue, were recorded on the scope. For each trial, a mean value of the plateau was estimated. Finally, the maximum from two measurements was considered as MVC.
The step measurements were performed for 0%, 20%, 40%, 60%, and 70% of MVC. For every step, the contraction was sustained for 30 s with around 90 s of rest between subsequent steps. In addition to the visual feedback, the subject was provided with voice information on the remaining trial time: every 5 s during the first 20 s after the required force level was reached and then every second during the last 10 s. The ramp measurements were performed by a linear increase in exerted force from 0% to 70% of MVC, which took 50 s to complete. During the ramp measurements, the voice information was initially provided every 10 s and then every second during the last 10 s. An experienced investigator visually inspected compliance of the subject’s force profiles with the respective force templates before the trials were assumed successful.
The data analysis for step and ramp isometric measurements was based on the instantaneous root mean square (RMS) values of the signals and performed using the R environment (ver. 3.4.4) [32]. For each of the acquired time series, the corresponding series of RMS values were calculated independently within subsequent 1-second long time windows (without overlapping), similarly for MMG and EMG signals. For step measurements, linear regression models were fit to the natural log-transformed values of RMS to quantify the relationships between the selected signals. The log-transformation of MMG RMS was introduced due to the reported non-linearities in the force-related MMG responses [15].
In the case of the step isometric measurements, data obtained during the experiment were used to evaluate (a) relationships between the response of each of the piezoelectric discs (internal and external) and force exerted by the muscle; (b) the relationship between responses from both of the piezoelectric discs; (c) the gain obtained at the internal disc with respect to the external disc; (d) the relationship between MMG and EMG data for both internal and external discs. Data recorded during the ramp isometric measurement were used to (a) visualize the relationship between EMG and force; (b) visualize the MMG/EMG relationship.

2.2. Classification Task

The piezoelectric discs were connected to charge amplifiers (custom-made amplifiers, with the feedback capacitor value set to correspond closely to the piezoelectric disc capacitance, which was measured to be 13 nF). Signals at the outputs of the amplifiers were digitized using the same setup as in the evaluation of the sensor performance during step and ramp isometric contractions.
For the classification of hand positions and gestures, the signals were recorded from the proximal part of an unsupported forearm of a seated subject. The elbow remained bent at 90 degrees during measurements. The evaluated MMG sensor was placed above the extensor carpi radialis longus (ECRL) muscle, providing two signals: from internal (ECRL_int) and external (ECRL_ext) discs. A second MMG sensor, having a structure similar to the proposed sensor but without the external disc, was placed over the flexor carpi ulnaris muscle, thus providing a single signal from the internal disc (FCU). Both sensors shared a non-elastic, woven polyester strap that held them in place after initial tightening. The location of the sensors on the forearm is marked in Figure 3. The hand motions used in this task are listed in Table 1 and illustrated in Figure 4.
The subject was free to decide on the order of the motions executed during the measurement session. The proposed gestures are transitions between the classified positions and were included because when the online data processing is anticipated, these transitions cannot be avoided and will probably become misclassified if omitted in the training set. The time of each acquisition was fixed to 3 s, and a counting down timer was displayed to the subject to allow synchronization of the motion start with the acquisition window. The received signals were visually verified before saving to allow for discarding data if they were corrupted.
The Pearson’s correlations r between the signal pairs were calculated to evaluate their mutual relationships for each recording. The analysis was extended with the derived signals that had their low-frequency components additionally suppressed by filtering the raw signals in the digital domain with a 5th order Butterworth filter (5 Hz cut-off frequency). Subsequently, the distributions of Pearson’s r were compared visually.
The classification was performed using a 1D convolutional neural network (CNN) in the R environment (ver. 3.4.4) [32] with a TensorFlow library [33] and R interface to Keras [34]. A structure of the proposed CNN is shown in Figure 5. It consisted of four convolutional layers, each followed by a max-pooling layer. The rest of the network comprises two dense layers preceded by dropout layers (with the dropout rate set to 0.5). ReLU activation function was used throughout the internal network layers. The final layer employed a softmax activation due to the multinomial classification. The network was trained and tested using all possible combinations of signals, i.e., complete set (ECRL_int, ECRL_ext, and FCU), signal pairs (ECRL_int–ECRL_ext, ECRL_ext–FCU, and ECRL_int–FCU), and individual signals. In total, 70% of the dataset was used for training and validation. A common random seed was set for each of the combinations to ensure the repeatability of the training process. The training data were augmented using a moving window for generating three crops of 1.8 s from each original 3-s recording. A 5-fold, 5-times cross-validation was performed during training to support the choice of the number of training epochs. The final model was trained in the number of epochs which yielded the lowest value of mean loss + 1 standard error in the cross-validation phase. The overall accuracies achieved in the test set with the described CNN were calculated for each combination of signals. The confusion matrices were used for the assessment of per-class accuracies.

3. Results

3.1. Evaluation of the Sensor Performance during Step and Ramp Isometric Contractions

The data were collected during a single session in a single trial for each of the intended contractions. Relationships between RMS and force values were modeled using the linear regression with log-transformed values of RMS (Figure 6A). The achieved R 2 coefficients were 0.953 and 0.971 for external and internal disc, respectively.
The fitted relationships, after back-transformation of RMS variable to the linear scale, are as follows:
R M S e x t = 0.118 e 0.0796 · F
R M S i n t = 1.82 e 0.0708 · F
where: F is the force given in [% MVC], R M S i n t and R M S e x t are the RMS values of signals recorded from the internal and external piezoelectric discs, respectively, both given in [mV].
To estimate dependency between RMS values obtained for internal and external discs, a linear regression model was fitted ( R 2 = 0.983 ) to the log–log transformed data (Figure 6B). Back-transformation of the model yields the relationship:
R M S i n t = 12.4 R M S e x t 0.873
Based on Equations (1) and (2) the signal gain for F [ 0 , 70 ] and R M S e x t [ 0.1 , 30 ] was estimated as R M S i n t / R M S e x t . The calculated gain/force relation is presented in Figure 7A. Similarly, based on Equation (3) the gain as a function of R M S e x t was calculated and presented in the Figure 7B, along with recalculated data-points.
In Figure 8 the relationship between EMG and MMG recorded from both external and internal piezoelectric discs during step isometric contractions is shown.
In Figure 9 the relationship between EMG and MMG recorded from both external and internal piezoelectric discs during ramp isometric contraction is shown.
Figure 10 illustrates the relationship between MMG and force signals obtained for the ramp isometric measurement.

3.2. Classification Task

The data were collected during two sessions. For each classified motion, 35 repetitions were acquired, yielding in total 280 records. In each session, the measurements were taken from the left forearm. An example of signals acquired during the wrist de-flexion (WrDeFlex) is provided in Figure 11.
The Pearson’s correlations r between the signal pairs were calculated for each recording to evaluate their mutual relationships. The results for both raw and digitally filtered signals are shown in Figure 12.
For all motions, the correlations between the raw signals from the internal disc of the sensor placed above the ECRL and the internal disc of the sensor placed above the FCU (i.e., ECRL_int–FCU pair) are visibly high, and for most cases, r is between 0.8 and 1.0. The remaining two pairs of signals show relatively lower absolute correlations. Only in the case of gestures WrDeExt and WrExt their absolute r values approach these of the pair ECRL_int–FCU. However, for these gestures, correlations between the ECRL_int–ECRL_ext, and ECRL_ext–FCU pairs are negative.
For the digitally high-pass filtered signals, when compared to the raw signals, the main observed differences involve correlations between the ECRL_int–ECRL_ext, and ECRL_ext–FCU pairs for:
  • WrDeExt and WrExt gestures, where absolute r values are diminished significantly;
  • WrFlex gesture, where the distributions of r values are narrowed and shifted to the positive values.
For all motions, the ECRL_int–FCU pair correlations are slightly diminished but remain positive and relatively high for the filtered signals.
The accuracies achieved with the described CNN in the test set are presented in Table 2. Confusion matrices for all trained models are provided in Figure 13.

4. Discussion

4.1. Evaluation of the Sensor Performance during Step and Ramp Isometric Contractions

The MMG RMS (log-transformed) for both piezoelectric discs of the sensor exhibits a marked correlation with the force exerted by the muscle during step contractions, as observed for up to 70% MVC: R 2 coefficient equals 0.953 and 0.971 for the external and internal disc, respectively. However, as the measured force exceeds 60% MVC, the rate of the MMG RMS increase begins to decline for both the step and ramp measurements. Such a change is in line with findings from previous studies of the biceps brachiimuscle isometric force generation, where plateaus or decreases of MMG amplitude were observed from approximately 60–80% MVC [10,35]. An interesting feature of the presented sensor is that for force levels below 20% MVC, the rate of MMG RMS increase is lower for the external disc than the internal one. This occurs likewise in step (Figure 6A) and ramp (Figure 10) contractions. Moreover, it has been previously documented that the MMG response to the force varies with the muscle fiber type composition and, for the ramp contractions, also depends on the type of muscle action (increase vs. decrease) [36]. Therefore, the sensor’s usage in HMI applications exploiting the force/MMG relationship must account for differences in muscle composition, type of contraction (stable vs. ramp) and may be limited to the force values not exceeding 60% MVC, with additional preference for the internal disc signal if the force remains below 20% MVC. The observed MMG/sEMG relationship shows similar MMG RMS saturation from approximately 60% MVC, contrary to the sEMG RMS, which continues to rise with the exerted force (Figure 8 and Figure 9).
Looking into the sensor-related details, MMG signal obtained from the internal disc has a considerably higher amplitude than the one acquired from the external disc. This is a feature that is beneficial for the design of signal chains, where it enables, for example, a replacement of the electronic amplification of a signal by a mechanical one, or, when combining measurements from two sensors having different gains and placed at a single site (as in the coupled sensor presented here), it extends the covered dynamic range. However, the achieved gain changes with the exerted force level from around 15 V/V for minimal force to around 8 V/V for 70% MVC. Nevertheless, one could observe a high correlation ( R 2 = 0.983) between log–log transformed RMS of MMG signals from both piezoelectric discs. This should be expected, as both transducers are mechanically coupled. Thus, the advantage of having both discs at a single site is not obvious (beyond the mentioned possibility of extending the dynamic range), at least for operation under static conditions, when the signal’s RMS is used. However, they should not be considered redundant for all static applications, as the back-transformation of the regression equation indicates a non-linear relationship (Equation (3)) between both signals. Depending on the signal processing method chosen, the mutual non-linearity at the system input may provide benefits when signals are superimposed.

4.2. Classification Task

The best-performing models in the classification task yielded accuracies in a range from 94.3% to 97.7%, similar to those reported in referenced works: up to 96% [31] and from 85.6% to 95.1% [30]. However, the direct comparison of results was not carried out due to differences in the sets of considered positions and gestures. In the presented setup, the gestures had the highest classification accuracies for all signal combinations, while hand positions were more often misclassified. The performance of CNN models was visibly lower for the 1-signal inputs with the rise in the misclassification rate for the following positions: flexed and extended wrist, clenched fist. The mentioned hand positions are inevitably more challenging to classify than gestures, as the latter yield significantly higher signal levels that result from the large change of muscle shape during the onset of the motion.
Analysis of the correlations of filtered and raw signals showed that when gestures were performed, a significant, low-frequency component below 5 Hz was present, which differentiated ECRL_ext from the rest of acquired raw signals, as shown in Figure 12. Thus, unlike for measurements made for isometric contractions, where combining signals from a single site (ECRL_ext and ECRL_int) should be possible and potentially beneficial, here it would result in a loss of information, and the usability of such a derived signal may vary depending on the characteristics of the considered gestures. However, signals from the ECRL_int and FCU sites were highly correlated when acquired during all chosen hand motions, which indicates their possible interchangeability. Results of the classification task also supported this conclusion, as the models that were based on the combinations: ECRL_ext–ECRL_int (single-site, 2-signal configuration) and ECRL_ext–FCU (double-site, 2-signal configuration), provided similar overall test accuracies. At the same time, the models based on double-site configurations (i.e., yielding 2- and 3-signal sets) provided accuracies better or equal to the single-site, 2-signal one. The single-site, 1-signal configurations resulted in the lowest classification accuracies.
We have shown that attaching the sensor to the examined limb with a strap allows for transmitting usable signals from the encompassed area to the sensor’s internal piezoelectric disc. In other words, a source of potential crosstalk has been turned into a valid input to the classification algorithm. Therefore, the proposed coupled single-site sensor may be considered a replacement for the two-site arrangement. As a result, it may simplify the myographic interface in terms of sensors’ spatial distribution. In our proof of concept, the substitution of the two-site sensor pair with a single-site sensor resulted only in a slight reduction in the classification accuracy (i.e., it decreased from 97.7% to 94.3%).

4.3. Observations on the Sensor Application

Based on the review of the acquired data, a few points regarding the sensor operation should be highlighted. These are general remarks that may be helpful when it comes to the implementation of the sensor in the MMG interface. The presented sensor design with a coupling pin that is free to move relative to the internal disc sporadically led to artifact generation due to the short-range repositioning of the pin contact point during large muscle motion. Such a movement resembles a stick-slip phenomenon and is observed as a narrow pulse, simultaneously in signals from both discs. However, these pulses occurred infrequently and likely did not pose noticeable issues during classification and did not introduce significant RMS analysis biases. The cardiac interference was the second source of a non-myographic signal, most apparent in the data recorded for stable hand positions. It may be treated as an auxiliary input for some applications and employed, e.g., for heart rate estimation. The next observed property stems from the difference in disc gains. Due to the higher gain achieved for the internal disc, its signal chain was prone to saturation when gestures were executed rapidly. Eventually, these had to be performed with a moderate speed, and acquisitions with saturation were discarded. However, the signal from the external disc had rarely saturated during faster motions. Even in such events, the sensor as a whole continued to provide a partially valid output. This feature may be beneficial for applications that involve both slow and fast movements, as the signal saturation may be easily detected and eventually managed in an algorithm or research protocol.
Another valuable feature of the designed sensor is that it may be manufactured from easily accessible and low-cost materials, and its assembly is not a demanding process. The 3D-printed parts required approximately 3000 mm 2 of a PLA material (1.22 m of 1.75 mm filament), with the cost of manufacturing (including electric energy and material) estimated as EUR 0.2. The price of a single piezo disc was approximately EUR 0.2. Without the wire and an optional connector, the cost of components of the proposed sensor is approximately EUR 0.6.

4.4. Limitations of the Study

As the data were collected from a single subject, no evaluation of an inter-subject variability of the sensor’s performance was possible. Moreover, collecting more measurements from the same subject over an extended period would be beneficial to establish repeatability characteristics of the sensor. In this study, the subject was additionally a research team member. Therefore, he may be considered a skilled user of the developed sensor, which may introduce a bias resulting in increased performance if compared with a person having no or limited prior experience with similar instrumentation. Even though omitted in the proof of concept presented here, such analyses would be of significant importance for every specific application of the coupled MMG sensor, where more participants having different skill levels in terms of using the designed interface should be recruited.

5. Conclusions

We have confirmed that the simplification of the MMG interface in terms of transducers’ spatial arrangement is possible with the designed sensor. Results of its evaluation under static and dynamic conditions support its potential for application in HMI.
Further investigation is desirable to identify the source of the differences between the vibration transmission to the internal and external disc. These differences seem to arise not only from the sensor’s mechanical structure but to a large extent from the sensor-skin interface. A partially related issue of adipose tissue influence on MMG was discussed, e.g., by Santos et al. [18]. In subsequent studies aimed at refining the requirements of the application of the proposed sensor, one should consider measuring the skinfold thickness or the subcutaneous adipose tissue to characterize the application site better. In general, it would be beneficial to obtain a model that would make it possible to optimize the sensor characteristics depending on the application requirements. Furthermore, the sensor’s performance should be tested with more muscle groups (which exhibit different recruitment of motor units and firing rates) and muscle actions (e.g., isokinetic contractions). Lastly, introducing a second myographic modality while preserving the proposed mechanical structure is worth considering, like in a work by Ke et al. [37], where a hybrid EMG-FMG sensor for prosthetic control was presented. For example, an sEMG–MMG integration may significantly reduce the EMG interelectrode distance, which had to be relatively large in our study due to the MMG sensor size.

Author Contributions

Conceptualization, M.S. and K.W.; methodology, M.S. and K.W.; software, M.S.; validation, M.S., M.W., and K.W.; formal analysis, M.S.; investigation, M.S. and K.W.; resources, M.S., M.W., and K.W.; data curation, M.S.; writing—original draft preparation, M.S. and K.W.; writing—review and editing, M.S., M.W., and K.W.; visualization, M.S.; project administration, K.W.; funding acquisition, K.W. All authors have read and agreed to the published version of the manuscript.

Funding

This research and APC were funded by the Dean of the Faculty of Mechatronics of the Warsaw University of Technology (with a grant for employees of the Faculty of Mechatronics of the Warsaw University of Technology for supporting scientific activity, grant number 504/04577/1142/44.000000).

Institutional Review Board Statement

The study was conducted according to the guidelines of the Declaration of Helsinki. Ethical review and approval were waived for this study, as the sensors used were non-invasive, non-significant risk devices, applied to a healthy adult person during an intervention brief in duration and harmless.

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

The data presented in this study are available on request from the corresponding author.

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

References

  1. Islam, M.A.; Sundaraj, K.; Ahmad, R.; Ahamed, N.; Ali, M.A. Mechanomyography Sensor Development, Related Signal Processing, and Applications: A Systematic Review. IEEE Sens. J. 2013, 13, 2499–2516. [Google Scholar] [CrossRef]
  2. Beck, T.; Housh, T.; Cramer, J.; Weir, J.; Johnson, G.; Coburn, J.; Moh, H.; Mielke, M. Mechnomyographic amplitude and frequency responses during dynamic muscle actions: A comprehensive review. Biomed. Eng. Online 2005, 4, 67. [Google Scholar] [CrossRef] [Green Version]
  3. Orizio, C. Muscle sound: Bases for the introduction of a mechanomyographic signal in muscle studies. Crit. Rev. Biomed. Eng. 1993, 21, 201–243. [Google Scholar] [PubMed]
  4. Krueger, E.; Scheeren, E.M.; Nogueira-Neto, G.N.; da Silveira Nantes Button, V.L.; Nohama, P. Advances and perspectives of mechanomyography. Braz. J. Biom. Eng. 2014, 8, 384–401. [Google Scholar] [CrossRef] [Green Version]
  5. Wolczowski, A.; Błędowski, M.; Witkowski, J. The System for EMG and MMG Singals Recording for the Bioprosthetic Hand Control. J. Autom. Mob. Robot. Intell. Syst. 2017, 11, 22–29. [Google Scholar] [CrossRef]
  6. Geng, Y.; Chen, L.; Tian, L.; Li, P. Comparison of electromyography and mechanomyogram in control of prosthetic system in multiple limb positions. In Proceedings of the IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI 2012), Hong Kong, China, 5–7 January 2012. [Google Scholar] [CrossRef]
  7. Rajamani, Y.; Lam, C.K.; Sundaraj, K.; Zulkefli, N.F.B. Analysis and Classification of Multiple Hand Gestures using MMG Signals. J. Telecommun. Electron. Comput. Eng. 2018, 10, 67–71. [Google Scholar]
  8. Donnarumma, M.; Caramiaux, B.; Tanaka, A. Muscular Interactions Combining EMG and MMG sensing for musical practice. In Proceedings of the International Conference on New Interfaces for Musical Expression, Daejeon, Korea, 27–30 May 2013. [Google Scholar]
  9. Prociow, P.; Wolczowski, A.; Amaral, T.; Dias, O.; Filipe, J. Identification of Hand Movements based on MMG and EMG Signals. In Proceedings of the 1st International Conference on Bio-inspired Systems and Signal Processing, Funchal, Madeira, Portugal, 28–31 January 2008; Volume 2, pp. 534–539. [Google Scholar]
  10. Talib, I.; Sundaraj, K.; Lam, C.K.; Sundaraj, S. A systematic review of muscle activity assessment of the biceps brachii muscle using mechanomyography. J. Musculoskelet. Neuronal Interact. 2018, 18, 446–462. [Google Scholar] [PubMed]
  11. Ibitoye, M.; Hamzaid, N.; Zuniga, J.; Hasnan, N.; Abdul Wahab, A. Mechanomyographic Parameter Extraction Methods: An Appraisal for Clinical Applications. Sensors 2014, 14, 22940–22970. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  12. Guo, J.Y.; Zheng, Y.P.; Xie, H.B.; Chen, X. Continuous monitoring of electromyography (EMG), mechanomyography (MMG), sonomyography (SMG) and torque output during ramp and step isometric contractions. Med. Eng. Phys. 2010, 32, 1032–1042. [Google Scholar] [CrossRef] [Green Version]
  13. Meagher, C.; Franco, E.; Turk, R.; Wilson, S.; Steadman, N.; McNicholas, L.; Vaidyanathan, R.; Burridge, J.; Stokes, M. New advances in mechanomyography sensor technology and signal processing: Validity and intrarater reliability of recordings from muscle. J. Rehabil. Assist. Technol. Eng. 2020, 7, 2055668320916116. [Google Scholar] [CrossRef] [Green Version]
  14. Islam, M.A.; Sundaraj, K.; Ahmad, R.; Ahamed, N. Mechanomyogram for Muscle Function Assessment: A Review. PLoS ONE 2013, 8, e58902. [Google Scholar] [CrossRef] [PubMed]
  15. Herda, T.J.; Housh, T.J.; Fry, A.C.; Weir, J.P.; Schilling, B.K.; Ryan, E.D.; Cramer, J.T. A noninvasive, log-transform method for fiber type discrimination using mechanomyography. J. Electromyogr. Kinesiol. 2010, 20, 787–794. [Google Scholar] [CrossRef] [PubMed]
  16. Beck, T.W.; Housh, T.J.; Fry, A.C.; Cramer, J.T.; Weir, J.P.; Schilling, B.K.; Falvo, M.J.; Moore, C.A. The influence of muscle fiber type composition on the patterns of responses for electromyographic and mechanomyographic amplitude and mean power frequency during a fatiguing submaximal isometric muscle action. Electromyogr. Clin. Neurophysiol. 2007, 47, 221–232. [Google Scholar]
  17. Castillo, C.S.M.; Wilson, S.; Vaidyanathan, R.; Atashzar, S.F. Wearable MMG-Plus-One Armband: Evaluation of Normal Force on Mechanomyography (MMG) to Enhance Human-Machine Interfacing. IEEE Trans. Neural Syst. Rehabil. Eng. 2021, 29, 196–205. [Google Scholar] [CrossRef]
  18. Santos, E.; de Fatima Fernandes Vara, M.; Ranciaro, M.; Strasse, W.; Nunes Nogueira Neto, G.; Nohama, P. Influence of sensor mass and adipose tissue on the mechanomyography signal of elbow flexor muscles. J. Biomech. 2021, 122, 110456. [Google Scholar] [CrossRef] [PubMed]
  19. Szumilas, M.; Lewenstein, K.; Ślubowska, E. Verification of the functionality of device for monitoring human tremor. Biocybern. Biomed. Eng. 2015, 35, 240–246. [Google Scholar] [CrossRef]
  20. Islam, M.A.; Ahamed, N. Mechanomyography Sensors for Muscle Assessment: A Brief Review. J. Phys. Ther. Sci. 2012, 24, 1359–1365. [Google Scholar] [CrossRef] [Green Version]
  21. Rodriguez-Labra, J.I.; Narakathu, B.B.; Atashbar, M.Z. Development of a Wireless Robotic Arm Control System Using Piezoelectric Sensors and Neural Networks. In Proceedings of the 2019 IEEE SENSORS, Montreal, QC, Canada, 27–30 October 2019; pp. 1–4. [Google Scholar] [CrossRef]
  22. Booth, R.; Goldsmith, P. A Wrist-Worn Piezoelectric Sensor Array for Gesture Input. J. Med Biol. Eng. 2017, 38, 284–295. [Google Scholar] [CrossRef]
  23. Watakabe, M.; Itoh, Y.; Mita, K.; Akataki, K. Technical aspects of mechnomyography recording with piezoelectric contact sensor. Med Biol. Eng. Comput. 1998, 36, 557. [Google Scholar] [CrossRef] [PubMed]
  24. Bolton, C.F.; Parkes, A.; Thompson, T.R.; Clark, M.R.; Sterne, C.J. Recording sound from human skeletal muscle: Technical and physiological aspects. Muscle Nerve 1989, 12, 126–134. [Google Scholar] [CrossRef]
  25. Ahmadizadeh, C.; Pousett, B.; Menon, C. Investigation of Channel Selection for Gesture Classification for Prosthesis Control Using Force Myography: A Case Study. Front. Bioeng. Biotechnol. 2019, 7, 331. [Google Scholar] [CrossRef] [PubMed]
  26. Silva, J.; Chau, T. Coupled microphone-accelerometer sensor pair for dynamic noise reduction in MMG signal recording. Electron. Lett. 2003, 39, 1496–1498. [Google Scholar] [CrossRef]
  27. Gregori, B.; Galié, E.; Accornero, N. Surface electromyography and mechanomyography recording: A new differential composite probe. Med. Biol. Eng. Comput. 2003, 41, 665–669. [Google Scholar] [CrossRef] [PubMed]
  28. Fukuhara, S.; Oka, H. A Simplified Analysis of Real-time Monitoring of Muscle Contraction during Dynamic Exercise Using an MMG/EMG Hybrid Transducer System. Adv. Biomed. Eng. 2019, 8, 185–192. [Google Scholar] [CrossRef] [Green Version]
  29. Zhang, X.; Li, X.; Samuel, O.W.; Huang, Z.; Fang, P.; Li, G. Improving the Robustness of Electromyogram-Pattern Recognition for Prosthetic Control by a Postprocessing Strategy. Front. Neurorobotics 2017, 11, 51. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  30. Asheghabadi, A.S.; Moqadam, S.B.; Xu, J. Multichannel Finger Pattern Recognition Using Single-Site Mechanomyography. IEEE Sens. J. 2021, 21, 8184–8193. [Google Scholar] [CrossRef]
  31. Esposito, D.; Andreozzi, E.; Gargiulo, G.D.; Fratini, A.; D’Addio, G.; Naik, G.R.; Bifulco, P. A Piezoresistive Array Armband With Reduced Number of Sensors for Hand Gesture Recognition. Front. Neurorobotics 2020, 13, 114. [Google Scholar] [CrossRef] [Green Version]
  32. R Core Team. R: A Language and Environment for Statistical Computing; R Foundation for Statistical Computing: Vienna, Austria, 2018. [Google Scholar]
  33. Abadi, M.; Barham, P.; Chen, J.; Chen, Z.; Davis, A.; Dean, J.; Devin, M.; Ghemawat, S.; Irving, G.; Isard, M.; et al. TensorFlow: A system for large-scale machine learning. arXiv 2016, arXiv:1605.08695. [Google Scholar]
  34. Allaire, J.; Chollet, F. Keras: R Interface to ’Keras’; R Package Version 2.4.0. 2021. Available online: https://keras.rstudio.com/ (accessed on 15 December 2021).
  35. Beck, T.W.; Housh, T.J.; Fry, A.C.; Cramer, J.T.; Weir, J.P.; Schilling, B.K.; Falvo, M.J.; Moore, C.A. The influence of myosin heavy chain isoform composition and training status on the patterns of responses for mechanomyographic amplitude versus isometric torque. J. Strength Cond. Res. 2008, 22, 818–825. [Google Scholar] [CrossRef] [PubMed]
  36. Trevino, M.A.; Herda, T.J.; Fry, A.C.; Gallagher, P.M.; Vardiman, J.P.; Mosier, E.M.; Miller, J.D. The influence of myosin heavy chain isoform content on mechanical behavior of the vastus lateralis in vivo. J. Electromyogr. Kinesiol. 2016, 28, 143–151. [Google Scholar] [CrossRef] [PubMed]
  37. Ke, A.; Huang, J.; Chen, L.; Gao, Z.; He, J. An Ultra-Sensitive Modular Hybrid EMG–FMG Sensor with Floating Electrodes. Sensors 2020, 20, 4775. [Google Scholar] [CrossRef] [PubMed]
Figure 1. The designed mechanomyograpic (MMG) sensor: (A) side view and (B) section (A-A) through the pin of the coupler. The piezoelectric discs are marked with an orange color.
Figure 1. The designed mechanomyograpic (MMG) sensor: (A) side view and (B) section (A-A) through the pin of the coupler. The piezoelectric discs are marked with an orange color.
Sensors 21 08380 g001
Figure 2. (A) The sensor before its final assembly. Two-core shielded microphone cables are used for signal transmission. (B) Side view of the assembled sensor with the external piezoelectric disc visible. (C) Side view of the assembled sensor with the internal disc visible.
Figure 2. (A) The sensor before its final assembly. Two-core shielded microphone cables are used for signal transmission. (B) Side view of the assembled sensor with the external piezoelectric disc visible. (C) Side view of the assembled sensor with the internal disc visible.
Sensors 21 08380 g002
Figure 3. The locations of the sensors on the forearm during acquisition of the signals for the classification tasks. Labels: ECRL—externsor carpi radialis longus, FCU—flexor carpi ulnaris.
Figure 3. The locations of the sensors on the forearm during acquisition of the signals for the classification tasks. Labels: ECRL—externsor carpi radialis longus, FCU—flexor carpi ulnaris.
Sensors 21 08380 g003
Figure 4. Illustration of the hand motions (positions and gestures) used in the classification tasks and their respective labels.
Figure 4. Illustration of the hand motions (positions and gestures) used in the classification tasks and their respective labels.
Sensors 21 08380 g004
Figure 5. A structure of the convolutional neural network from the classification task.
Figure 5. A structure of the convolutional neural network from the classification task.
Sensors 21 08380 g005
Figure 6. The root mean square (RMS) values of the subsequent 1-second parts (no overlapping) of MMG signals recorded during step isometric measurements. (A) RMS values for both of the sensor discs are plotted against the percentage of maximum voluntary contraction (MVC). The regression lines are fitted with log-transformed values of RMS. (B) RMS values of both signals are compared and a linear regression model in the log–log space is fitted.
Figure 6. The root mean square (RMS) values of the subsequent 1-second parts (no overlapping) of MMG signals recorded during step isometric measurements. (A) RMS values for both of the sensor discs are plotted against the percentage of maximum voluntary contraction (MVC). The regression lines are fitted with log-transformed values of RMS. (B) RMS values of both signals are compared and a linear regression model in the log–log space is fitted.
Sensors 21 08380 g006
Figure 7. Estimated signal gain ( R M S i n t / R M S e x t ) as a function of: (A) force (F), represented as % MVC, (B) R M S e x t (data-points recalculated based on experimental data).
Figure 7. Estimated signal gain ( R M S i n t / R M S e x t ) as a function of: (A) force (F), represented as % MVC, (B) R M S e x t (data-points recalculated based on experimental data).
Sensors 21 08380 g007
Figure 8. Comparison of MMG and EMG RMS values obtained during step isometric contraction measurement. The 0% MVC level is omitted, as it does not reflect muscle activity. The comparison is made for both internal (blue) and external (black) piezoelectric discs.
Figure 8. Comparison of MMG and EMG RMS values obtained during step isometric contraction measurement. The 0% MVC level is omitted, as it does not reflect muscle activity. The comparison is made for both internal (blue) and external (black) piezoelectric discs.
Sensors 21 08380 g008
Figure 9. Comparison of MMG and EMG RMS values obtained during the ramp isometric contraction measurement. The comparison is made for both internal (blue) and external (black) piezoelectric discs.
Figure 9. Comparison of MMG and EMG RMS values obtained during the ramp isometric contraction measurement. The comparison is made for both internal (blue) and external (black) piezoelectric discs.
Sensors 21 08380 g009
Figure 10. MMG as function of force exerted by the muscle during the ramp (from 0% to 70% MVC in 50 s) isometric contraction. Data are plotted both for internal (blue) and external (black) piezoelectric discs.
Figure 10. MMG as function of force exerted by the muscle during the ramp (from 0% to 70% MVC in 50 s) isometric contraction. Data are plotted both for internal (blue) and external (black) piezoelectric discs.
Sensors 21 08380 g010
Figure 11. Example of signals acquired during the wrist de-flexion (WrDeFlex): (A) raw and (B) digitally high-pass filtered at 5 Hz. The signal sites abbreviations: ECRL_ext\ECRL_int–extensor carpi radialis longus, external\internal piezoelectric disc of the sensor, respectively; FCU—flexor carpi ulnaris, internal piezoelectric disc of the sensor.
Figure 11. Example of signals acquired during the wrist de-flexion (WrDeFlex): (A) raw and (B) digitally high-pass filtered at 5 Hz. The signal sites abbreviations: ECRL_ext\ECRL_int–extensor carpi radialis longus, external\internal piezoelectric disc of the sensor, respectively; FCU—flexor carpi ulnaris, internal piezoelectric disc of the sensor.
Sensors 21 08380 g011
Figure 12. The Pearson’s correlations between the signals, grouped by the type of executed motion (see Table 1 for the motion labels description and Figure 11 for signal sites abbreviations). The correlations are given for raw and digitally high-pass filtered signals with a 5 Hz cut-off frequency.
Figure 12. The Pearson’s correlations between the signals, grouped by the type of executed motion (see Table 1 for the motion labels description and Figure 11 for signal sites abbreviations). The correlations are given for raw and digitally high-pass filtered signals with a 5 Hz cut-off frequency.
Sensors 21 08380 g012
Figure 13. Confusion matrices for all trained models, headers of matrices indicate the signal set used (see Table 1 for the description of the motion labels and Figure 11 for the description of the signal labels). Illustrations of classified hand motions are recalled for convenience.
Figure 13. Confusion matrices for all trained models, headers of matrices indicate the signal set used (see Table 1 for the description of the motion labels and Figure 11 for the description of the signal labels). Illustrations of classified hand motions are recalled for convenience.
Sensors 21 08380 g013
Table 1. The hand motions (positions and gestures) used in the classification tasks and their respective labels.
Table 1. The hand motions (positions and gestures) used in the classification tasks and their respective labels.
MotionLabel
Positions:
 Normal wrist and hand positionWrNorm
 Extended wristWrExtHeld
 Flexed wristWrFlexHeld
 Clenched fistWrFist
Gestures:
 From normal position to extended wristWrExt
 From extended wrist to normal positionWrDeExt
 From normal position to a flexed wristWrFlex
 From flexed wrist to normal positionWrDeFlex
Table 2. Classification task results—test accuracies (see Figure 11 for the description of the signal abbreviations).
Table 2. Classification task results—test accuracies (see Figure 11 for the description of the signal abbreviations).
Signal SetCross-Validated
Training Epochs
Overall Test Accuracy
ECRL_ext, ECRL_int, FCU3597.7%
ECRL_ext, ECRL_int3594.3%
ECRL_int, FCU3897.7%
ECRL_ext, FCU3594.3%
ECRL_ext2475.0%
ECRL_int3384.1%
FCU3989.8%
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Szumilas, M.; Władziński, M.; Wildner, K. A Coupled Piezoelectric Sensor for MMG-Based Human-Machine Interfaces. Sensors 2021, 21, 8380. https://doi.org/10.3390/s21248380

AMA Style

Szumilas M, Władziński M, Wildner K. A Coupled Piezoelectric Sensor for MMG-Based Human-Machine Interfaces. Sensors. 2021; 21(24):8380. https://doi.org/10.3390/s21248380

Chicago/Turabian Style

Szumilas, Mateusz, Michał Władziński, and Krzysztof Wildner. 2021. "A Coupled Piezoelectric Sensor for MMG-Based Human-Machine Interfaces" Sensors 21, no. 24: 8380. https://doi.org/10.3390/s21248380

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop