# Motion Sequence Analysis Using Adaptive Coding with Ensemble Hidden Markov Models

^{1}

^{2}

^{3}

^{4}

^{*}

^{†}

## Abstract

**:**

## 1. Introduction

- 1.
- We propose a novel feature representation approach, namely, adaptive sequence coding (ASC), for motion data analysis. ASC is a data-adaptive learning method that does not require large-scale hypeparameters, which has the ability to capture the correlations between elements as well as the internal interrelated structures of multi-dimensional sequences.
- 2.
- We propose an ensemble learning classifier that utilizes HMMs as base learners. It can effectively excavate internal interconnections and variations of elements within symbolized sequences, thus further boosting the performance of motion recognition.
- 3.
- Extensive experiments on several popular real-world datasets show that our method compares well to competing techniques. Additionally, ablation studies also confirm the benefits of the proposed dual symbolization mechanism and ensemble learning.

## 2. Related Works

#### 2.1. Non-Data-Adaptive Methods

#### 2.2. Data-Adaptive Methods

## 3. Materials and Methods

#### 3.1. Dataset

#### 3.2. Adaptive Motion Sequence Coding

#### 3.3. Ensemble Learning Classification

#### 3.3.1. Constructing Hidden Markov Models

#### 3.3.2. Constructing Ensemble-SequenceHMM Using AdaBoost

Algorithm 1: Pseudo-code for ASC and Ensemble-SequenceHMM |

## 4. Experimental Results and Analysis

#### 4.1. Experimental Setup

#### 4.2. Evaluation Metrics

#### 4.3. Overall Comparison with Previous Studies

#### 4.4. Ablation Experiments

#### 4.4.1. Impact of Adaptive Motion Sequence Coding

#### 4.4.2. Impact of Ensemble Learning

## 5. Conclusions and Future Scope

## Author Contributions

## Funding

## Data Availability Statement

## Acknowledgments

## Conflicts of Interest

## References

- Yao, C.; He, J.S.; Che, H.; Huang, Y.; Wu, J. Feature pyramid self-attention network for respiratory motion prediction in ultrasound image guided surgery. Int. J. Comput. Assist. Radiol. Surg.
**2022**, 17, 2349–2356. [Google Scholar] [CrossRef] [PubMed] - Franco, T.; Sestrem, L.; Henriques, P.R.; Alves, P.; Varanda Pereira, M.J.; Brandão, D.; Leitão, P.; Silva, A. Motion sensors for knee angle recognition in muscle rehabilitation solutions. Sensors
**2022**, 22, 7605. [Google Scholar] [CrossRef] [PubMed] - Chavarriaga, R.; Sagha, H.; Calatroni, A.; Digumarti, S.T.; Tröster, G.; Millán, J.; Roggen, D. The Opportunity challenge: A benchmark database for on-body sensor-based activity recognition. Pattern Recognit. Lett.
**2013**, 34, 2033–2042. [Google Scholar] [CrossRef] - Kulakou, S.; Ragab, N.; Midoglu, C.; Boeker, M.; Johansen, D.; Riegler, M.A.; Halvorsen, P. Exploration of Different Time Series Models for Soccer Athlete Performance Prediction. Eng. Proc.
**2022**, 18, 37. [Google Scholar] - Hao, W.W. Classification of Sport Actions Using Principal Component Analysis and Random Forest Based on Three-Dimensional Data. Displays
**2022**, 72, 102135. [Google Scholar] - Nguyen, T.; Nguyen, G.; Nguyen, B.M. EO-CNN: An Enhanced CNN Model Trained by Equilibrium Optimization for Traffic Transportation Prediction. Procedia Comput. Sci.
**2020**, 176, 800–809. [Google Scholar] [CrossRef] - Majumder, A.J.; Izaguirre, J.A. A Smart IoT Security System for Smart-Home Using Motion Detection and Facial Recognition. In Proceedings of the 2020 IEEE 44th Annual Computers, Software, and Applications Conference (COMPSAC), Madrid, Spain, 13–17 July 2020; pp. 1065–1071. [Google Scholar]
- Chen, J.F.; Wang, C.C.; Wu, E.H.K.; Chou, C.F. Simultaneous heterogeneous sensor localization, joint tracking, and upper extremity modeling for stroke rehabilitation. IEEE Syst. J.
**2020**, 14, 3570–3581. [Google Scholar] [CrossRef] - Qiu, Y.Y.; Guan, Y.R.; Liu, S. The analysis of infrared high-speed motion capture system on motion aesthetics of aerobics athletes under biomechanics analysis. PLoS ONE
**2023**, 18, e0286313. [Google Scholar] [CrossRef] - Chen, L.F.; Wu, H.Y.; Kang, W.X.; Wang, S.R. Symbolic sequence representation with Markovian state optimization. Pattern Recognit.
**2022**, 131, 108849. [Google Scholar] [CrossRef] - Huang, B.Z.; Li, X.D. Human Motion Prediction via Dual-Attention and Multi-Granularity Temporal Convolutional Networks. Sensors
**2023**, 23, 5653. [Google Scholar] [CrossRef] - Yoon, S.W.; Seo, J.; Moon, J. TapNet: Neural Network Augmented with Task-Adaptive Projection for Few-Shot Learning. In Proceedings of the 36th International Conference on Machine Learning (ICML 2019), Long Beach, CA, USA, 9–15 June 2019; pp. 7115–7123. [Google Scholar]
- Ismail Fawaz, H.; Forestier, G.; Weber, J.; Idoumghar, L.; Muller, P.A. Deep learning for time series classification: A review. Data Min. Knowl. Discov.
**2019**, 33, 917–963. [Google Scholar] [CrossRef] - Bai, B.; Li, G.L.; Wang, S.Z.; Wu, Z.D.; Yan, W.H. Time series classification based on multi-feature dictionary representation and ensemble learning. Expert Syst. Appl.
**2021**, 169, 114162. [Google Scholar] [CrossRef] - Buchaiah, S.; Shakya, P. Bearing fault diagnosis and prognosis using data fusion based feature extraction and feature selection. Measurement
**2022**, 188, 110506. [Google Scholar] [CrossRef] - Katarya, R.; Prasad, T. A Survey on Time Series Online Sequential Learning Algorithms. In Proceedings of the 2017 IEEE International Conference on Computational Intelligence and Computing Research (ICCIC), Coimbatore, India, 14–16 December 2017; pp. 1–4. [Google Scholar]
- Chen, R.J.; Ravishanker, N. Feature Construction Using Persistence Landscapes for Clustering Noisy IoT Time Series. Future Internet
**2023**, 15, 195. [Google Scholar] [CrossRef] - Lin, J.; Keogh, E.; Wei, L.; Lonardi, S. Experiencing SAX: A novel symbolic representation of time series. Data Min. Knowl. Discov.
**2007**, 15, 107–144. [Google Scholar] [CrossRef] - Li, Y.C.; Shen, D.R. A new symbolic representation method for time series. Inf. Sci.
**2022**, 609, 276–303. [Google Scholar] [CrossRef] - Djebour, L.; Akbarinia, R.; Masseglia, F. Variable-Size Segmentation for Time Series Representation. In Transactions on Large-Scale Data- and Knowledge-Centered Systems LIII; Abdelkader, H., A Min, T., Eds.; Springer: Berlin/Heidelberg, Germany, 2023; pp. 34–65. [Google Scholar]
- Sousa Lima, W.; de Souza Bragança, H.L.; Montero Quispe, K.G.; Pereira Souto, E.J. Human activity recognition based on symbolic representation algorithms for inertial sensors. Sensors
**2018**, 18, 4045. [Google Scholar] [CrossRef] - Le Nguyen, T.; Gsponer, S.; Ilie, I.; O’reilly, M.; Ifrim, G. Interpretable time series classification using linear models and multi-resolution multi-domain symbolic representations. Data Min. Knowl. Discov.
**2019**, 33, 1183–1222. [Google Scholar] [CrossRef] - Kang, M.; Ahn, J.; Lee, K. Opinion mining using ensemble text hidden Markov models for text classification. Expert Syst. Appl.
**2018**, 94, 218–227. [Google Scholar] [CrossRef] - Gomes, S.R.; Saroar, S.G.; Mosfaiul, M.; Telot, A.; Khan, B.N.; Chakrabarty, A.; Mostakim, M. A comparative approach to email classification using Naive Bayes classifier and hidden Markov model. In Proceedings of the 2017 4th International Conference on Advances in Electrical Engineering (ICAEE), Dhaka, Bangladesh, 28–30 September 2017; pp. 482–487. [Google Scholar]
- Dogan, T.; Uysal, A.K. A novel term weighting scheme for text classification: TF-MONO. J. Inf.
**2020**, 14, 101076. [Google Scholar] [CrossRef] - Hastie, T.; Rosset, S.; Zhu, J.; Zou, H. Multi-class adaboost. Stat. Its Interface
**2009**, 2, 349–360. [Google Scholar] [CrossRef] - Ding, J.Y.; Wang, Y.; Si, H.Y.; Gao, S.; Xing, J.W. Multimodal Fusion-AdaBoost Based Activity Recognition for Smart Home on WiFi Platform. IEEE Sensors J.
**2022**, 22, 4661–4674. [Google Scholar] [CrossRef] - Agrawal, R.; Faloutsos, C.; Swami, A. Efficient similarity search in sequence databases. In Foundations of Data Organization and Algorithms, Proceedings of the 4th International Conference, FODO ’93, Chicago, IL, USA, 13–15 October 1993; Proceedings; Springer: Berlin/Heidelberg, Germany, 1993; pp. 69–84. [Google Scholar]
- Yang, W.K.; Du, Q.L.; Cui, J.C.; Wang, Y.K.; Lu, X.P.; Qi, C.X.; Zhang, G.P. Motion recognition based on sum of the squared errors distribution. IEEE Access
**2021**, 9, 37116–37130. [Google Scholar] [CrossRef] - Liu, X.; Liu, H.; Guo, Q.; Zhang, C. Adaptive wavelet transform model for time series data prediction. Soft Comput.
**2020**, 24, 5877–5884. [Google Scholar] [CrossRef] - Chatzigeorgakidis, G.; Skoutas, D.; Patroumpas, K.; Palpanas, T.; Athanasiou, S.; Skiadopoulos, S. Efficient range and knn twin subsequence search in time series. IEEE Trans. Knowl. Data Eng.
**2022**, 35, 5794–5807. [Google Scholar] [CrossRef] - Zhang, X.; Sun, Y.C.; Zhu, H.Q. Method of Pilot’s Motion Retrieval and Recognition Based on Dynamic Programming. In Proceedings of the 2017 International Conference on Sensing, Diagnostics, Prognostics, and Control (SDPC), Shanghai, China, 16–18 August 2017; pp. 345–349. [Google Scholar]
- Keogh, E.J.; Pazzani, M.J. An enhanced representation of time series which allows fast and accurate classification, clustering and relevance feedback. In Proceedings of the Fourth International Conference on Knowledge Discovery and Data Mining (KDD-98), New York, NY, USA, 27–31 August 1998; pp. 239–243. [Google Scholar]
- Kang, W.X.; Chen, L.F.; Guo, G.D. Spatio-temperal structure feature representation model for motion sequences. CAAI Trans. Intell. Syst.
**2023**, 18, 240–250. [Google Scholar] - Junejo, I.N.; Junejo, K.N.; Aghbari, Z.A. Silhouette-based human action recognition using SAX-Shapes. Vis. Comput.
**2014**, 30, 259–269. [Google Scholar] [CrossRef] - Zhou, T.Y.; Wang, Y.B.; Du, J. Human Intent Prediction in Human-Robot Collaboration—A Pipe Maintenance Example. In Proceedings of the Construction Research Congress 2022, Arlington, VA, USA, 9–12 March 2022; pp. 581–590. [Google Scholar]
- Riza, L.S.; Fazanadi, M.N.; Utama, J.A.; Samah, K.A.F.A.; Hidayat, T.; Nazir, S. SAX and Random Projection Algorithms for the Motif Discovery of Orbital Asteroid Resonance Using Big Data Platforms. Sensors
**2022**, 22, 5071. [Google Scholar] [CrossRef] - Dias, D.B.; Madeo, R.C.B.; Rocha, T.; Biscaro, H.H.; Peres, S.M. Hand movement recognition for Brazilian Sign Language: A study using distance-based neural networks. In Proceedings of the 2009 International Joint Conference on Neural Networks (IJCNN), Atlanta, GA, USA, 14–19 June 2009; pp. 697–704. [Google Scholar]
- Anguita, D.; Ghio, A.; Oneto, L.; Parra, X.; Reyes-Ortiz, J.L. A public domain dataset for human activity recognition using smartphones. In Proceedings of the 21st European Symposium on Artificial Neural Networks (ESANN 2013), Bruges, Belgium, 24–26 April 2013; p. 3. [Google Scholar]
- Sprint, G.; Cook, D.; Weeks, D.; Dahmen, J.; La Fleur, A. Analyzing Sensor-Based Time Series Data to Track Changes in Physical Activity during Inpatient Rehabilitation. Sensors
**2017**, 17, 2219. [Google Scholar] [CrossRef] - Gu, J.X.; Yu, Z.W.; Shen, K.L. Alohomora: Motion-Based Hotword Detection in Head-Mounted Displays. IEEE Internet Things J.
**2020**, 7, 611–620. [Google Scholar] [CrossRef] - Ishibashi, N.; Fujii, F. Hidden Markov model-based human action and load classification with three-dimensional accelerometer measurements. IEEE Sensors J.
**2020**, 21, 6610–6622. [Google Scholar] [CrossRef] - Sun, J.W.; Zou, R.; Liang, R.X.; Gao, L.; Liu, S.; Li, Q.; Zhang, K.; Jiang, L.L. Ensemble Knowledge Tracing: Modeling interactions in learning process. Expert Syst. Appl.
**2022**, 207, 117680. [Google Scholar] [CrossRef] - Castellini, A.; Masillo, F.; Azzalini, D.; Amigoni, F.; Farinelli, A. Adversarial Data Augmentation for HMM-based Anomaly Detection. IEEE Trans. Pattern Anal. Mach. Intell.
**2023**, 45, 14131–14143. [Google Scholar] [CrossRef] [PubMed] - Maulik, R.; Mohan, A.; Lusch, B.; Madireddy, S.; Balaprakash, P.; Livescu, D. Time-series learning of latent-space dynamics for reduced-order model closure. Phys. D Nonlinear Phenom.
**2020**, 405, 132368. [Google Scholar] [CrossRef] - Tiumentsev, A.Y.; Tiumentsev, Y.V. Motion Control of Supersonic Passenger Aircraft Using Machine Learning Methods. Opt. Mem. Neural Netw.
**2023**, 32, S195–S205. [Google Scholar] [CrossRef] - Le Guennec, A.; Malinowski, S.; Tavenard, R. Data augmentation for time series classification using convolutional neural networks. In Proceedings of the ECML/PKDD Workshop on Advanced Analytics and Learning on Temporal Data, Riva Del Garda, Italy, 19–23 September 2016. [Google Scholar]
- Xing, Z.Z.; Pei, J.; Keogh, E. A brief survey on sequence classification. ACM SIGKDD Explor. Newsl.
**2010**, 12, 40–48. [Google Scholar] [CrossRef] - Su, H.; Lu, X.M.; Chen, Z.Q.; Zhang, H.S.; Lu, W.F.; Wu, W.T. Estimating Coastal Chlorophyll-A Concentration from Time-Series OLCI Data Based on Machine Learning. Remote Sens.
**2021**, 13, 576. [Google Scholar] [CrossRef] - Zhang, Y.; Ren, G.; Liu, X.J.; Gao, G.L.; Zhu, M.D. Ensemble learning-based modeling and short-term forecasting algorithm for time series with small sample. Eng. Rep.
**2022**, 4, e12486. [Google Scholar] [CrossRef] - Jain, R.; Ganesan, R.A. Reliable sleep staging of unseen subjects with fusion of multiple EEG features and RUSBoost. Biomed. Signal Process. Control
**2021**, 70, 103061. [Google Scholar] [CrossRef]

Datasets | Classes | Dimensions | Sequence Lengths | Sample Sizes of Sequences |
---|---|---|---|---|

LIBRAS1 | 5 | 2 | [45, 45] | 120 |

LIBRAS2 | 5 | 2 | [45, 45] | 120 |

HAR | 4 | 3 | [10, 48] | 242 |

JSI | 4 | 3 | [133, 133] | 427 |

OPPORTUNITY | 4 | 3 | [4028, 4028] | 1672 |

**Table 2.**Comparison of overall accuracy (%) with different methods using multiple same public datasets.

Feature Representation | Classifier | LIBRAS1 | LIBRAS2 | HAR | JSI | OPPORTUNITY |
---|---|---|---|---|---|---|

N-gram+KNN | 56.67 | 51.67 | 22.00 | 43.08 | 27.52 | |

SAX | N-gram+Bayes | 60.00 | 58.33 | 30.75 | 37.05 | 20.22 |

HMM | 54.17 | 43.33 | 31.50 | 46.84 | 38.16 | |

N-gram+KNN | 63.33 | 54.17 | 33.25 | 43.85 | 50.60 | |

ASAX_EN | N-gram+Bayes | 66.67 | 66.67 | 23.25 | 34.66 | 51.26 |

HMM | 66.67 | 57.50 | 39.25 | 49.67 | 51.26 | |

Embedded | LSTM | 20.25 | 21.01 | 21.25 | 40.98 | 37.86 |

MLP | 25.29 | 16.85 | 29.75 | 66.48 | 74.93 | |

t-LeNet | 93.33 | 76.52 | 58.25 | 42.15 | 52.45 | |

TapNet | 79.82 | 78.95 | 24.50 | 38.86 | 38.14 | |

ASC | AdaBoost | 94.17 | 88.33 | 87.64 | 64.63 | 76.14 |

Feature Representation | Classifier | LIBRAS1 | LIBRAS2 | HAR | JSI | OPPORTUNITY |
---|---|---|---|---|---|---|

N-gram+KNN | 54.61 | 44.97 | 13.93 | 27.98 | 10.61 | |

SAX | N-gram+Bayes | 59.03 | 54.61 | 18.28 | 20.56 | 8.41 |

HMM | 52.06 | 37.60 | 18.73 | 24.81 | 13.81 | |

N-gram+KNN | 61.62 | 51.05 | 31.95 | 39.34 | 34.71 | |

ASAX_EN | N-gram+Bayes | 66.52 | 64.02 | 17.78 | 33.26 | 34.89 |

HMM | 65.25 | 54.06 | 34.82 | 40.14 | 34.89 | |

Embedded | LSTM | 8.17 | 14.12 | 9.98 | 14.51 | 13.74 |

MLP | 14.73 | 10.78 | 20.75 | 54.05 | 70.10 | |

t-LeNet | 93.28 | 73.19 | 56.24 | 16.31 | 39.44 | |

TapNet | 81.55 | 78.69 | 20.14 | 34.08 | 32.71 | |

ASC | AdaBoost | 93.99 | 87.32 | 87.56 | 60.68 | 61.64 |

Feature Representation | Symbolization I | Event Sequence Coding | LIBRAS1 | LIBRAS2 | HAR | JSI | OPPORTUNITY |
---|---|---|---|---|---|---|---|

Model A | ✓ | 59.17 | 51.67 | 57.85 | 49.89 | 50.03 | |

Model B | ✓(SAX) | ✓ | 88.33 | 75.83 | 52.50 | 59.81 | 70.16 |

ASC | ✓ | ✓ | 94.17 | 88.33 | 87.64 | 64.63 | 76.14 |

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |

© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Kong, X.; Liu, X.; Chen, S.; Kang, W.; Luo, Z.; Chen, J.; Wu, T.
Motion Sequence Analysis Using Adaptive Coding with Ensemble Hidden Markov Models. *Mathematics* **2024**, *12*, 185.
https://doi.org/10.3390/math12020185

**AMA Style**

Kong X, Liu X, Chen S, Kang W, Luo Z, Chen J, Wu T.
Motion Sequence Analysis Using Adaptive Coding with Ensemble Hidden Markov Models. *Mathematics*. 2024; 12(2):185.
https://doi.org/10.3390/math12020185

**Chicago/Turabian Style**

Kong, Xiangzeng, Xinyue Liu, Shimiao Chen, Wenxuan Kang, Zhicong Luo, Jianjun Chen, and Tao Wu.
2024. "Motion Sequence Analysis Using Adaptive Coding with Ensemble Hidden Markov Models" *Mathematics* 12, no. 2: 185.
https://doi.org/10.3390/math12020185