Accuracy of Hidden Markov Models in Identifying Alterations in Movement Patterns during Biceps-Curl Weight-Lifting Exercise

Abstract

This paper presents a comparison of mathematical and cinematic motion analysis regarding the accuracy of the detection of alterations in the patterns of positional sequence during biceps-curl lifting exercise. Two different methods, one with and one without metric data from the environment, were used to identify the changes. Ten volunteers performed a standing biceps-curl exercise with additional loads. A smartphone recorded their movements in the sagittal plane, providing information on joints and barbell sequential position changes during each lift attempt. An analysis of variance revealed significant differences in joint position (p < 0.05) among executions with three different loads. Hidden Markov models were trained with data from the bi-dimensional coordinates of the joint positional sequence to identify meaningful alteration with load increment. Tests of agreement tests between the results provided by the models with the environmental measurements, as well as those from image coordinates, were performed. The results demonstrated that it is possible to efficiently detect changes in the patterns of positional sequence with and without the necessity of measurement and/or environmental control, reaching an agreement of 86% between each other, and 100% and 86% for each respective method to the results of ANOVA. The method developed in this study illustrates the viability of smartphone camera use for identifying positional adjustments due to the inability to control limbs in an adequate range of motion with increasing load during a lifting task.

1. Introduction

Correct supervision during the performance of resistance exercises is imperative for exercise quality execution, which can generate improvements in performance and avoid injuries resulting from inadequate posture or incorrect loads [1,2]. Resources for resistance training exercises have increased in recent years, and this increases the number of injuries caused by this type of training, mainly due to a lack of proper supervision associated with execution [3].

An assessment of load, posture and movement during a resistance exercise is often conducted by an experienced coach, who, through visual examination, analyses exercise movements without using any equipment that provides qualitative data on the observed parameters [4]. Coaches perform this form of supervision as a qualitative analysis, based on a systematic observation method, using introspective judgment of the quality of human movement that is based solely on the experience of the evaluator [5].

An increase in load during the performance of a resistance exercise can cause a change in movement patterns. A study that analyzed a squat exercise [6] verified a change in movement patterns with increased load. The results revealed that when load was added to the exercise, hip and knee angles increased to accommodate the increased load.

Spatial trajectories, which are described by joint movements during the performance of an exercise, can also be used as references for movement pattern analysis. A trajectory can be defined as the set of all positions occupied by a moving body relative to a given reference [7]. This reference is considered to be the point of view of the evaluator who is supervising an exercise [5]; as such, physical exercises can be characterized by their movement patterns and specific space-time structures. Physical exercises are therefore susceptible to being modeled as temporal trajectories in a given space, using measurements that are intimately correlated with visual observations [8].

Studies that use trajectory analysis to evaluate human movement have been present in the literature for some time and can be considered a new resource for gesture recognition [9]. The trajectories described by the heel and toes during gait were previously analyzed [10]; according to the author of that study, these trajectories can be used to represent specific motor control tasks that are connected with posture and gait changes. Another study, an algorithm for extracting and classifying bidimensional movements in a frame sequence based on movement trajectories, was developed. The motion patterns were learned using trajectories extracted from neural networks [11] with a recognition rate of 94.42%. Suzuki et al. [12] proposed a method for learning movement patterns and detecting anomalies in these patterns by analyzing human trajectories through long-term observation. The spatial and temporal features of those trajectories were analyzed using hidden Markov models (HMMs). Another study examined human movement patterns in a space-time context by clustering geographic data extracted from smartphone global positioning systems [13] (confidence: 80%), and Calin et al. [14] explored movement analysis, dividing movement into three main components: body posture (posture), the range of motion (reach), and movement pattern (positional sequence), which obtained classification accuracy, with up to 56% for HMMs. With constant technological improvements, the motion capture of body movements has become more accessible and more widely utilized in sports activities. Currently, with the help of a smartphone camera, it is possible to evaluate movement patterns and measure physical performance [15]. Visual resources such as video assistant referee (VAR), which can perform reliable measurements of the studied object in a noninvasive manner and without interfering with the natural flow of the sport, are used, for example, in soccer.

In a similar manner, the use of noninvasive methods that are capable of tracking and automatically supervising resistance training exercises could provide great benefits to athletes [16]. For that to occur, it is necessary to use a statistical/mathematical model that can provide measurements or distinguish motion patterns that accurately detect significant changes in human movements. HMMs are suitable for pattern recognition and can be used to solve this problem [17]. Originating from Markov chains, they possess a finite number of states and use a double stochastic process that measures transitions between states and generates an exit symbol for each state, both linked to a probability of occurrence [18].

To apply the HMM, three different problems must be solved [18]: (1) the probability of the unknown sequence must be determined; (2) the hidden states of the best sequence must be estimated; and (3) the model parameters must be trained. In the model learning phase, the training data are divided into groups (clusters). Training is performed at maximum likelihood using the Baum–Welch algorithm [17]. After completion of the training phase, a recognition process based on choosing the model that provides the maximal probability of the analyzed observation sequences P(O|λ) is conducted [19]. In this study, we aimed to identify changes in load displacement that occur due to alterations in subjects’ movement trajectories during a biceps-curl exercise. Load displacement and joint positional sequence were quantified by means of bi-dimensional video capture while the subjects performed elbow flexion during the biceps-curl exercise, and the data so obtained were further used to train the HMMs. This work tested the hypothesis that HMMs provide accurate automatic detection of alterations in shoulder, elbow and load positional sequences that are considered significant according to cinematic assessment.

2. Methods

2.1. Participants

The participants in this study were ten male volunteers (age: 26.3 ± 4.9 years, height: 177.6 ± 8.0 cm, body weight: 86.2 ± 16.7 kg) who were recruited from a local health club. The inclusion criteria were as follows: (1) age ≥ 22 and ≤ 32 years; (2) engaged in regular exercise for at least three times a week in the nine months prior to data collection; and (3) at least six months of experience performing a biceps curl with a barbell. Participants with any disease or medical condition that might have affected their task performance were excluded. This study was conducted in accordance with the Declaration of Helsinki and approved by the UNESP ethics committee under number 17486119.0.0000.5398.

2.2. Procedures

Experimental data were collected at the Laboratory of Human Performance Optimization in Sport (LABOREH). For data collection, three green hemispherical markers (25 mm in diameter) were used. The markers were placed on the barbell and at the great tubercle and the lateral epicondyle of the humerus (Figure 1).

The subjects were instructed to do the following before testing: (i) maintain their usual dietary and sleep habits; (ii) avoid intake of any energy- or performance-enhancing supplements or drinks for a period of at least 24 h before testing; and (iii) avoid intake of beverages containing alcohol or caffeine for a period of at least 24 h before testing. They were also instructed to wear comfortable clothes during testing.

All subjects performed three sets of three repetitions with 10 min of rest between sets, following the technique fundamentals for barbell biceps-curl [20]. In the first set, the repetitions were performed using only the barbell; the second set was performed using an additional barbell that added a load equivalent to 25% of the subject’s body weight [21]. Finally, the third and last load was performed with 50% of the subject’s body weight as an additional overload.

Positional and temporal data were collected through the use of a smartphone digital camera (Galaxy S9 with 2 megapixels and UHD 4K resolution) that was placed so as to have a perpendicular view of the subject’s sagittal plane [22], as shown in Figure 2.

To calibrate the measurements, fixed markers were placed in the background in a plane coincident with the subject’s sagittal plane. Measurements of body segments (arm, forearm) and height were also taken to verify and certify the calibration measurements. Therefore, it was possible to measure the subjects’ movements from the subjects’ bidimensional coordinates in the sagittal plane.

Video capture in the MPEG-4 format was performed for all sets and repetitions at a frequency of 30 frames per second [23,24,25,26,27,28,29,30]. The duration of each video matched the duration of the repetitions. All videos were digitally edited using Wondershare Filmora version 9 (Wondershare Filmora, Hong Kong, China) [31] for Chroma key filter application [32] on the markers and Alfa channel application [33] for enhanced marker contrast. Kinovea 0.8.27 software (Kinovea, Bordeaux, France) [34] was used to track the markers, and the marker coordinates were exported to XLM (Extensive Markup Language) files. The origin of the coordinates in Cartesian plan (i.e., an ordered pair (x,y) representing position on the horizontal x-axis—from left to right, and on the vertical y-axis—from bottom to top) [35] was assigned to the marker placed in the shoulder, elbow, and barbell and recalibrated for each of the repetitions.

2.3. Displacement and Vertical Distance Measurements

Displacement measurements were obtained from the markers placed at the great tubercle (shoulder) and the lateral epicondyle (elbow) of the humerus, and at the bar (see Figure 1) according to the following formula for Euclidean distance calculation:

$$\Delta \mathrm{d}=\sqrt{{\left({\mathrm{x}}_{\mathrm{f}}-{\mathrm{x}}_{\mathrm{i}}\right)}^2+{({\mathrm{y}}_{\mathrm{f}}-{\mathrm{y}}_{\mathrm{i}})}^2}$$

(1)

where Δd is the displacement value, $\left(x_f,y_f\right)$ are the coordinates of the markers at final position, and $\left(x_i,y_i\right)$ are the coordinates of the markers at initial positions.

For the barbell, the vertical distances were also estimated by its variation only along the y-axis:

$$\Delta \mathrm{y}={\mathrm{y}}_{\mathrm{f}}-{\mathrm{y}}_{\mathrm{i}}$$

(2)

where Δy is the vertical distance, $y_f$ is the y-axis value coordinate at the final position, and $y_i$ is the y-axis coordinate at the starting position. The barbell Δy might be an additional information indicating the undesirable adjustments in limb position (i.e., shoulder flexion, elbow rising and lumbar extension) with load increment [36,37].

2.4. Statistical Analysis

Statistical analysis was performed using IBM SPSS Statistics 26.0 (SPSS, Chicago, IL, USA). One-way ANOVA was used to verify the existence of significant differences in Δd and Δy measures for shoulder, elbow, and barbell during the upward/concentric phase of the biceps-curl exercise, in accordance with the different loads that were tested. A post hoc Tukey test was used. The sample power was determined with G*Power 3 from data including the effect size (partial eta squared, η²_p) for ANOVA test between the Δd and Δy values for 0 vs. 25 and 50% lift attempts, actual N sample, and specifying α = 0.05 [38]. The eta square (η²) was calculated, and the correspondence to Cohen’s d was verified to classify the level of the effect according to small (d = 0.2), medium (d = 0.5), and large (d = 0.8) [39]. The significance level was set at 5%.

2.5. HMM Modeling

In HMMs, the state’s hidden sequence can be estimated based on the visible observation sequence [40] and characterized by five elements in a symbolic manner [18,41], where N represents the number of states in the model. For the variable $q_t$, there is a corresponding state value, represented by $S=\left\{S_1, S_2,\dots ,S_N\right\}$. The number of observation symbols for HMM exits is represented by M. The symbols are designated $O=\left\{O_1, O_2,\dots ,O_M\right\}$. A transition probability matrix $A=\left[a_{ij}\right]$ represents the transition probabilities between states, where $a_{ij}=Prob\left(q_{t+1}=S_j|q_t=S_i\right), 1\le i, j\le N$ represents the probability value in the $q_t$ state for transitioning to state $S_j$ at time t + 1, since at time t, the state is S_i. The exit probability matrix $B=\left[b_j\left(k\right)\right]$ represents the observation probabilities in each state, where $b_j\left(k\right)=Prob\left(O_k\left(t\right)|q_t=S_j\right), 1\le j\le N, 1\le k\le M$ is the probability of the k-infinite observation symbol. Finally, an initial state distribution ${\pi }_1=\left\{{\pi }_{i1}\right\}$ represents the probability of a new input sequence starting from a given state at the initial time t = 1, where ${\pi }_{i1}=Prob\left(q_1=S_i\right), 1\le i\le N$.

The ANOVA results, which will analyze the significant differences in bi-dimensional displacement, will provided the criteria for the marker position data modeling in HMM. The Markovian model algorithms [42,43,44,45] were employed in GNU Octave 4.2.1 (GNU Octave, Madison, WI, USA) [46]. For analysis, the HMMs were provided with bi-dimensional data on the trajectories of each marker during the ascending phase of the exercise. Research on human movement [47,48,49,50] in which a linear type of topology was used [51] provided a reference for the general topology for the HMM.

The model’s learning phase occurs with clusters [47] using the k-means algorithm [52,53,54,55,56]. The term cluster are referring to the set of positional coordinates (x,y) defining a common segment of path during the biceps-curl for the changes in position of the shoulder, elbow and barbell. A simplified example of the relation between barbell positioning during the exercise’s upward movement and Markovian modeling is shown in Figure 3. Each of the three positions of the barbell, defined as states S1, S2 and S3, derived from the clustering process presents a transition probability (go to the next state or remain in the same state) and an observation probability for each state $b_j\left(k\right)$ [57].

For HMM subject modeling, the three repetitions performed solely with the barbell were used as the standard model of the positional sequence pattern for shoulder, elbow and barbell. These repetitions were used as the basis for training the model. Using the acquired data, the relation between the 25% and 50% loads was also tested, using the 25% body weight as training. In this state, the model parameters λ = (A, B, π) were calculated using the observed symbols O such that the observation probability P(O|λ) was augmented. For this computation, the Baum–Welch algorithm was employed, and this expanded the observation sequence probabilities [47].

The movement pattern recognition for the trajectories provided by the markers was based on the maximum likelihood criterion. For comparison between a trained model and a new observation sequence, the forward algorithm was used [18,58]. The algorithm estimated the probability that the observation data could have been generated by the model. With the application of HMM, the minimum number of states necessary to recognize the significant differences detected by ANOVA while the subject performed the exercise with different loads, in terms of changes in the patterns of execution of the movement while the exercise was performed, was obtained.

The results (i.e., the minimum number of states to recognize the difference detect to the ANOVA) obtained using data from measurements of the environment and those obtained using only the image coordinates were subjected to agreement analysis using the Bland and Altman plot [59] in MedCalc 19.0.3 (MedCalc Software, Ostend, Belgium); in this analysis, the agreement limits were defined as ±1.96 × SD (the standard deviation of the difference between the N for each method, i.e., the confidence interval of 95%) [60].

3. Results

The mean values, in centimeters, of the maximal Δy obtained for each marker during the upward phase of the exercise are presented in

Table 1. Mean maximal values achieved for each marker at the end of the trajectory (shoulder, elbow (Δd), and barbell (Δy)) for the three different loads (cm).

Load	Shoulder	Elbow	Barbell
0%	1.7 ± 0.4	4.4 ± 1.7	56.8 ± 7.7
25%	3.0 ± 1.3	6.4 ± 2.9	59.6 ± 8.5
50%	7.9 ± 3.8	10.4 ± 4.0	64.0 ± 9.0

.

Using the values presented in Table 1, one-way ANOVA was conducted to verify the existence of significant differences in the maximal Δd and Δy values for each marker and each subject. Eight participants showed significant differences in the shoulder and elbow values in the 0%–50% load comparison. In the 25%–50% comparison, differences in the shoulder and elbow (Δd) values were identified in eight and five participants, respectively. For the barbell (Δy), a difference was observed only in the 0%–50% comparison, and that difference was observed in seven participants. The effect size (η²) for shoulder, elbow and barbell were: 0.590, 0.432 and 0.121, respectively. These effects size corresponded to Cohen d = 2.4, 1.7, and 0.7, which were associated to a sample power of >99%, 97% and 35%, respectively, for shoulder, elbow and barbell. Therefore, the number of participants fails to reach a satisfactory power only for the barbell results, despite of the medium to large probability of higher mean as load increase for all marks.

Through HMM modeling, was found the minimum number of states (N) required to recognize pattern differences in positional sequence for each mark. For this, the individual significance values for each joint in each subject in the comparison of the loads were considered. The data extracted from this modeling, together with metric data from the environment, are presented in

Table 2. Minimum number of states necessary for HMM recognition of differences in the execution of biceps-curl with different loads, and with and without metric of environment.

Markers	Loads	Subjects
		01		02		03		04		05		06		07		08		09		10
		Metric	Non-metric	Metric	Non-metric	Metric	Non-metric	Metric	Non-metric	Metric	Non-metric	Metric	Non-metric	Metric	Non-metric	Metric	Non-metric	Metric	Non-metric	Metric	Non-metric
Elbow	0%–50%	20	20	…	…	35	35	30	30	45	45	20 *	23 *	…	…	40	40	25	25	35	35
	25%–50%	…	…	…	…	35	35	15 *	16 *	…	…	20	20	…	…	35	35	…	…	40	40
Shoulder	0%–50%	…	…	15	15	15 *	20 *	45	45	…	…	35	35	25	25	45	45	20	20	30	30
	25%–50%	…	…	25	25	20	20	25	25	…	…	15	15	45	45	25	25	35	35	20 *	21 *
Barbell	0%–50%	30	30	45	45	…	…	60	60	…	…	35	35	45	45	…	…	55	55	40 *	42 *

*—different number of states necessary when comparing the HMM models. …—no significant differences found by ANOVA.

. Table 2 also shows the states obtained with video analysis only (no metric or environmental coordinates); these were also subjected to HMM modeling in which the minimum number of states (N) for recognition of the differences in sequential pattern of positions was computed.

Based on the data presented in Table 2, it is possible to establish that few changes occur in the number of states necessary for HMM recognition when the two methods are compared. To compare the methods, a Bland–Altman analysis was performed. The results of this comparison are shown in Figure 4. They provide confirmation that, due to the few changes in the number of minimum states necessary, as seen in Table 2, there are no differences between the methods.

4. Discussion

This study examined whether an HMM trained using different types of coordinates derived from the same movement is able to detect significant variations in shoulder, elbow and load positional sequence, and therefore changes in movement pattern of the biceps-curl exercise. As demonstrated in Table 2, linear topology HMM, trained with Cartesian coordinates [47,61] and corrected using environmental measurements, was able to detect significant adjustments in movement patterns in the sagittal plane, and this was confirmed by ANOVA. Therefore, the findings evidenced the confidence in HMM for detecting undesired joint positional adjustments when comparing the standard reference of simple human lift movement (e.g., single-joint action with no load) to the lift attempts with heavy loads.

For the ANOVA test, maximal displacement values with the three different loads were provided, and the HMMs were trained with three executions for each load (0% or 25%) for comparison. With HMMs, the use of a small number of training models provides a more rigorous analysis and yields results that are in better agreement with the reference model [62]. In this study, we searched for the limits, aiming at no recognition of the pattern of the reference model. In other words, we searched for instances in which the movement pattern was altered. In this way, a minimum number of states were found to draw attention to the changes in movement pattern executed by each subject, and these states indicated movements that were inadequate for lifting the specific load.

To dispense with the need for measurements of the environment and/or subject, HMM modeling was completed using only image coordinates. The positional sequence of coordinates provided by the markers while the subjects performed an upward movement with the barbell were obtained through HMM training and testing. Table 2 presents the results of these methods (nonmetric); they differ little from the results obtained using the environmental data. Of 36 cases shown to have significant variation in movements indicated by the environmental measurements, 31 were also identified by the method that uses only image coordinates, an agreement of approximately 86% between the methods.

It is noteworthy that the analysis of the movement described in this work evaluates the adequacy of the movement by comparing the results with a reference that already presents great similarity to the analyzed movement. Wearable sensors that are used to identify movements that are made during different actions, such as movements in a badminton game for example, can provide accuracies of up to 97% in recognizing the type of movement [63]. The authors of the cited study also sought to discriminate the skill levels of professional and amateur badminton athletes, obtaining satisfactory accuracy values ranging from 83.3% to 90%, which is leveled with the percentage of recognition obtained by applying HMM in our research. Current studies in which wearable sensors are used to recognize daily activities such as walking, going up or down stairs, sitting, standing and lying down, achieve an accuracy of 93.77% in the recognition of these activities [64] in the absence of information about the skills or evolution of the practitioner. Hence, the accuracy in recognizing movement patterns from methods using image with no information on the metric of the environment has attaining ~90%, which is an average close to that observed for HMM.

As in the work of Ghorbani Faal et al. [65], to conduct an adequate comparison between methods, Bland–Altman agreement tests were used to compare the data load acquired with the image coordinates (CI) and the environmental data (CA). Figure 4 shows Bland–Altman plot comparisons of different methods of HMM modeling for all the loads tested. The plots show that the mean difference between methods is close to zero and that most of the obtained values are within the agreement limits. For the elbow, all values found are within the confidence interval. This similarity between the two methods (CI and CA) emphasizes that movement pattern analysis, in the absence of data from the environment, is sufficient to detect changes in movement patterns during exercise.

The use of image coordinates alone for movement pattern analysis by means of the performed trajectory of the objects, joints and limbs during the exercise could assist in exercise supervision, since it is an ecologic method. No information from the environment or any other type of information is necessary. Additionally, the analysis does not hinder exercise execution. Furthermore, this analysis could provide a more solid basis for qualitative exercise analysis, as described by Knudson and Morrison [5], as well as single out inadequate loads [6] or postures during the performance of resistance exercises. For example, the HMM method has been explored as an additional tool to help amateur athletes learn correct movements for a specific action in a sport, aiming to improve their skill level [63], as well as an analysis to support personalized sport training prescriptions according to the uncommon abilities of practitioners [66].

Unlike previously described methods [63], the method described in this work does not require the use of environmental measurements, expensive computational resources, or wearable sensors, all of which can increase the cost of performing a supervised activity [62]. Based on videos, it is possible to analyze the movements performed in exercises even if the videos were not recorded for this purpose. Thus, it is possible to analyze videos recorded in the past using the method described here.

A limitation of our method is that it analyzes movement displacement in a two-dimensional (2D) way. In turn, the three-dimensional (3D) analyses could provide detailed information on motion in other planes of movement, and therefore provide information of the movements in the transverse plane, which were possible changes in joint position that might occur to maintain postural adequacy with increased load. In addition, the 3D sagittal model would provide more accurate data and is not subject to the parallax and perspective errors that occur in 2D analyses. However, the biceps-curl movement is basically in the sagittal plane rather than in the transverse and frontal planes [67], and although the 3D approach has an advantage over the 2D approach, the 2D analyses such as the one described in this study allow the analysis of motion using data collected in any type of video. Regardless, the analysis by HMM using 3D data could identify further alterations caused by the increase in load, and thus provide more information about changes in the trajectories of movements through simultaneous modeling in different planes [68].

Future work should include the analyses of subjects’ movements in a plane other than the sagittal plane (i.e., the use of 3D coordinates). This would make it possible to verify whether the positional sequence of joints and objects can be further refined and whether HMM training can be extended beyond the results obtained with bi-dimensional analysis. Therefore, this makes HMM a potential tool to support the human-based diagnosis of motor patterns, with application varying from disorders in neural pathologies to the motor optimization of athlete’s performance.

5. Conclusions

The findings of the current study evidenced that the trajectory of the movement is adjusted modifying the positional sequence of the barbell and joints as the amount of load was increased during biceps-curl exercise. In addition, the ability of HMM to accurately detect alterations in the upward trajectories of object and body parts were also demonstrated, supporting the speculation that HMM might be a suitable automatized method for analyzing the ability to perform single-joint resistance exercises.

Hence, one possible application of the current findings is movement analysis using simple video recordings, which might be a practical and effective alternative for exercise supervision. For example, it could be useful to provide clues in maintaining the correct load for the proper execution of resistance exercises without causing postural damage or modifying muscle recruitment. In addition, the HMM method can be easily implemented into mobile devices through which video could be recorded, which highlights the practicality of automated supervision for movement monitoring using smartphone cameras.