Towards Posture and Gait Evaluation through Wearable-Based Biofeedback Technologies

Abstract

In medicine and sport science, postural evaluation is an essential part of gait and posture correction. There are various instruments for quantifying the postural system’s efficiency and determining postural stability which are considered state-of-the-art. However, such systems present many limitations related to accessibility, economic cost, size, intrusiveness, usability, and time-consuming set-up. To mitigate these limitations, this project aims to verify how wearable devices can be assembled and employed to provide feedback to human subjects for gait and posture improvement, which could be applied for sports performance or motor impairment rehabilitation (from neurodegenerative diseases, aging, or injuries). The project is divided into three parts: the first part provides experimental protocols for studying action anticipation and related processes involved in controlling posture and gait based on state-of-the-art instrumentation. The second part provides a biofeedback strategy for these measures concerning the design of a low-cost wearable system. Finally, the third provides algorithmic processing of the biofeedback to customize the feedback based on performance conditions, including individual variability. Here, we provide a detailed experimental design that distinguishes significant postural indicators through a conjunct architecture that integrates state-of-the-art postural and gait control instrumentation and a data collection and analysis framework based on low-cost devices and freely accessible machine learning techniques. Preliminary results on 12 subjects showed that the proposed methodology accurately recognized the phases of the defined motor tasks (i.e., rotate, in position, APAs, drop, and recover) with overall F1-scores of 89.6% and 92.4%, respectively, concerning subject-independent and subject-dependent testing setups.

1. Introduction

The control of the postural system is one of the fundamental neurophysiological mechanisms of the human body. It is fundamental to ensuring balance against gravity and fixing body orientation, and functions as a reference frame for perception–action coupling while efficiently dealing with the external world. Postural control is a dynamic process that requires sensory detection of body motions and integration of sensorimotor information within the central nervous system. In more detail, the central nervous system triggers the execution of appropriate musculoskeletal responses in order to obtain an equilibrium between destabilizing and stabilizing forces [1].

It has been shown, in the reference literature of physiatric medicine, that measurements of postural stability are critical for determining predictors of performance [2], for evaluating musculoskeletal injuries [3], for determining the effectiveness of physical training and rehabilitation treatments [4], and to provide injury prevention through the study of injury risk-factor analysis [3,5]. The body’s motion is mainly based on the integration of the proprioceptive, visual, and vestibular inputs [6]. Afferent proprioceptive inputs are conveyed to different levels of the central nervous system [7,8,9]; however, most of them remain unconscious. The joint positions and movement sensing (kinaesthesia) are the expressions of the conscious component, but postural control is primarily based on the unconscious component [9]. Specifically for the antigravity movements, proprioceptive control represents the expression of the effectiveness of the stabilizing reflexes in controlling vertical stability [8]. In fact, antigravity movements are the activities that counteract gravity and postural instability with at least a phase of single-limb stance [6]. In this way, proprioceptive input represents the most relevant sensory system in the maintenance of static postural stability at all ages and fitness levels [10]. This topic is also relevant in neurodegenerative diseases such as Parkinson’s disease (PD). For example, PD patients with postural instability have worse reactions to brief perturbations, more stance sway, and trouble switching between tasks. Moreover, quantifying balance changes in early and moderate-stage PD and the comparison to healthy subjects using clinical assessments of balance and musculoskeletal activation is paramount, primarily if performed through less invasive and costly systems [11,12].

Many tools to detect musculoskeletal activation have been used in sport and rehabilitative medicine. Mainly, electromyography (EMG) is employed for this purpose. However, EMGs are not yet widely used in combination with accelerometers for forecasting and customizing measures for the analysis of the human body’s motion to achieve different goals. This is mainly due to the limited number of investigations that have been focusing on the nature of musculoskeletal response to a broad spectrum of stimuli able to identify the thresholds that establish the standard/ideal status of the postural system. Hence, there is a lack of low-cost technology that habilitates these measurements.

Furthermore, in the last decade, with the advent of the Internet of Things (IoT), embedded sensors have been integrated into personal devices such as smartphones and smartwatches. In several applications, sensors are integrated into clothes or other equipment/objects of daily life, becoming a central research topic due to their importance in many areas, including healthcare, interactive gaming, sports, and monitoring systems for general purposes in controlled and uncontrolled settings [13,14,15].

The primary purpose of the investigation we carried out in this project was to present a preliminary study on the design of a portable and reliable postural system prototype composed of HW and SW, adapted to diverse individual profiles concerning the performance viewpoint, from patients needing rehabilitation to top-level athletes. It is widely accepted in the community of psychiatric medicine that proper quantification of the postural system efficiency represents an essential assessment for improving the quality of life. However, most of the actual measurements are developed in a laboratory environment where natural movements are usually constrained by the instruments applied to subjects’ bodies and the environment. This process is performed to distinguish, as precisely as possible, in the limits of the experimental setting, the roles of proprioception, visual and vestibular input using a low-cost and portable instrument. This habilitates the individuals to move freely and perform activities at home or in other uncontrolled environments (e.g., gyms or sports facilities) [5,16].

The technology integrated into such project includes state-of-the-art apparatus (i.e., force platform, EMG, and motion capture cameras) and wireless three-dimensional inertial units performing computation and data analysis over low-cost devices (i.e., sensors themselves, smartphones, and tablets) or cloud platforms.

In the project, we considered variants of these configurations’ implementations to evaluate possible solutions to different settings. Subsequently, another essential goal will be the usability of the envisioned technology. Thus, the possibility to perform a comparative study with existing state-of-the-art validated systems is needed to obtain a well-ground assessment. Moreover, fundamental analysis of the specific characteristics of the exercises existing in the literature and a combinational calculation will define the subsequent exercise prescriptions; a multitude of exercise combinations will be available to satisfy the actual needs of the different clinical conditions. To identify these exercises, in future works, we are going to investigate the potential of reinforcement learning as a tool for customization (in order to adapt the response of the portable tool to the needs of the setting, such as medical diagnosis, performance test, training evaluation, and training progress measurements) and personalizing (in order to adapt to the individual variability of the applications mentioned above).

The project has a threefold scope:

(a)

providing a theoretical foundation of a set of motor tasks employed to evaluate the performance of posture and gait in at least three contexts: rehabilitation, sports performance, and good health training at different ages;

(b)

devising and testing a set of sensors that allow obtaining the above measures;

(c)

defining a preliminary set of experiments and algorithms to identify individual profiles and compute, via machine learning methods, the correct set of functional exercises.

Finally, this paper presents the experimental apparatuses and describes the experimental design strictly related to the goals mentioned above. The experiments conducted were structured for a final purpose: obtaining an experimental system for identifying the correct patterns to be devised in the project’s future developments.

The rest of the paper is organized as follows. Section 2 provides an overview of the relevant references to similar studies. Section 3 discusses the project’s expected research outcome, and Section 4 introduces the overall experimental design. Section 5 presents a preliminary evaluation of the data collected by the designed setup. Finally, Section 6 presents some conclusions and sketches further work.

2. Related Work

Posture is studied in two aspects: (a) postural control and (b) postural orientation. The former involves studying the positional control of the body in space and orientation. Instead, the latter involves studying the relationships between body segments. A neutral state (also known as neutral posture) is observed when the upper trunk and head are at zero degrees concerning the vertebral column. Subjects deviating from this neutral posture are said to have low stability that can provoke accelerated intervertebral disc (IVD) degeneration, damage, and misalignment of vertebrae, producing nerve compression that can cause radicular manifestations, such as sensorimotor deficiencies and pain in the involved regions [17]. When considering the maintenance of the vertical posture in everyday life situations, postural control might become a complex task that requires the ability to anticipate and compensate postural strategies when fast actions are performed and when environmental perturbations are applied [18]. How individuals control their preparatory and compensatory postural adjustments is still under debate [19]. Several mechanisms help individuals to keep their posture when task conditions change due to self-inflicted perturbation (e.g., I am suddenly moving my upper arm forward [20] or when somebody is pushing me [21]). These mechanisms are represented by changes in the activation levels of postural muscles called early postural adjustments (EPAs), starting up to 1000ms prior to the impact [22], and anticipatory postural adjustments (APAs), starting 0–150ms prior to the impact [23]. The primary role played by the EPAs is to adjust the posture and facilitate action planning. Typical examples are seen in preparation for making a step [24] or to avoid contact with an approaching object [25]. On the other hand, the function of APAs is to generate forces that act against an effect (mechanical) of a predictable perturbation [26]. Here we concentrate on these ecological motor tasks where individuals are challenged to control posture when facing a highly dynamic situation. The tasks selected in this experimental design involve sequences of actions that require the maintenance of stable posture while standing on an unstable proprioceptive platform and receiving in an unexpected or expected way a perturbation requiring sudden balance recovery. These motor tasks will help unveil the individual strategies adopted given the individual’s level of skill. Based on the literature background, a single limb stance is regularly used to examine the postural system [16,27,28,29],

EMG, electrocardiography (ECG), and inertial sensors integrated into wearables are emerging as promising low-cost and easily usable solutions in everyday life [13] and health care contexts [30]. Inertial measurement unit (IMU)-based movement identification can be achieved by statistical classification or be threshold-based [31]. Such statistical methods utilize supervised machine learning, which links features of a movement to possible movement states in terms of the observation’s possibility [32]. Many of these studies are devoted explicitly to disabled people with diminished gait/posture abilities. This holds for multiple sclerosis patients [33] and Parkinson’s disease sufferers [30,34,35]. The ability to monitor the gait of multiple sclerosis patients and provide correct biofeedback can help prevent falls and detect freezing (an aspect that can be fruitful also for Parkinson’s patients) [33,36]. Prototype systems often include integrated sensors located on the ankles to track gait movements. Body sensors are positioned near the cervical vertebra or on the shoulders to monitor body posture [13]. Many systems can also measure parameters that might be difficult to provide manually, such as the maximum acceleration of the patients during standing up, or the time it takes from sitting to standing [37,38].

Moreover, the current diffusion of machine learning methods employed in gait, posture analysis, and feedback is not comprehensive, but a few significant results have already been achieved. A relevant group of investigations has been designed for decoding algorithms for brain–machine interactions (BMIs) that use the spiking activity as their control signal [39]. These approaches are powerful in devising usable technologies. Specifically, feedback for reinforcement-learning-based brain–machine interfaces using confidence metrics has been addressed [40]. Some studies show how to derive the required evaluative feedback from a biological source, using both the feedback’s quantity and quality, and incorporate it into reinforcement learning controller architecture to maximize performance. Analogously, the Berlin BCI has developed an accurate system that works from the first session in BCI-naive Subjects [41].

An overview of the various steps in the brain–computer interface (BCI) cycle, i.e., the loop from the measurement of brain activity, classification of data, feedback to the subject, and the effect of feedback on brain activity, is the focus of [30,42]. On the other hand, the role of technology for accelerated motor learning in sports is investigated in [43]. Finally, parallel man–machine training in ECG-based cursor control development is the subject of [30,44]. Some references should be given to smart environment previous investigations as a foundation of the method developed here, emphasizing some development related to energy management [45] and concerning the design of energy-efficient transmission protocols for wireless body area networks [46,47]. However, the systems mentioned above present many limitations related to accessibility, economic cost, size, battery life, intrusiveness, and usability (i.e., controlled and uncontrolled home or working context) environments.

3. Expected Research Outcomes of the Project

This section illustrates the project goals, reporting the most suitable application scenarios of the technology we envision and an overview of the presented architecture design. For better comprehension, the goals are presented from a top-down perspective.

(a)

basic application scenarios;

(b)

envisioned technology;

(c)

system architecture workflow.

3.1. Application Scenarios

Three different potential application scenarios have been devised under the supervision and collaboration of psychiatric medicine personal and sports training experts. Scenario 1 was a controlled environment enriched with a set of sensors to the extent that it makes this environment smart. Scenario 2 was set without specifying whether the performed activities were to be carried out indoors or outdoors. It is legitimate to suppose that the environmental setting shall be relatively poor regarding available interactions, including the potential unavailability of an Internet connection. Scenario 3 identified variations determined by post-traumatic rehabilitation, personalized performance control over the evolution of illnesses with harmful consequences on the patient’s stability, and training of athletes with special needs.

• Scenario 1: Diagnostic Evaluation

In the context of a psychiatric medical practice, a patient who suffers from postural instability due to a traumatic event (e.g., car accident) or neurodegenerative disorder is visited by medical personnel. The diagnostic process is assisted with the envisioned technology. The patient is asked to execute a sequence of three exercises: single-stance stability test, forward movement of arms with weights, and step-on gait on a free-range. During the exercises, the patient wears a jacket equipped with a set of sensors and interacts with a visual focus tool that helps her to identify a fixed point at a given distance. The jacket interacts with an application that works on the cloud, measures the reaction time or other variables, including anticipation’s effectiveness in the movements, and provides the operator the possibility of marking progress in performance quality based on a fixed threshold that the operator can define concerning age, sex, and the clinical condition of the patient. The whole process is recorded on video, and the instrumental measures are saved on the patient’s profile.

• Scenario 2: Sport performance benchmarking

An athlete training for a sporting event is monitored by her coach. He provides her with a performance benchmark in line with the event requirements and expectations. The athlete has a given training period for preparing for the event and a performance level she has to accomplish in order to be competitive. The performance benchmarks have been defined by the coach based on the athletic preparation path of the athlete. While following the coach’s requests, the athlete executes some training exercises while wearing the jacket described in Scenario 1. Every athlete’s exercise is compared against the benchmark performance and consequently identified in terms of a negative gradient concerning the benchmark itself.

• Scenario 3: Rehabilitation Follow-Up

During the rehabilitation period, a patient wearing the jacket described in Scenario 1 attends a program consisting of a series of exercises. Each step in the series requires comparing the performance with the provided reference benchmark defined by the psychiatrist during the diagnostic process. The patient measures are the same as in Scenario 2 and represent how a patient uses the jacket in a medical context.

3.2. Envisioned Technology

Our project’s aim is to design an accessible, low-cost, small, dedicated low-cost technological solution (i.e., Gait and Posture Smart Jacket (GPSJack)) suitable for the previously introduced application scenarios. Figure 1 shows a graphic representation of the envisioned technology where the subject is wearing the GPSJack and is immersed in the Gait and Posture Smart Environment (GPSEnv). Numbers 1–5 mark the sensors attached to the jacket, and number 6 marks the tablet application used by the top-level user. Medical and potentially sports training staff can interact with the GPSJack/GPSEnv through a tablet to guarantee total portability and versatility of the envisioned technological solutions.

Therefore, the GPSJack system represents the low-cost inertial sensor-based system, which is the project’s final goal. GPSEnv represents the adaption of the existing state-of-the-art technologies to the presented scenarios for validation purposes. The final GSPJack/GPSEnv architecture will provide to medical/sports staff the capability to perform the following actions:

• register a GPSJack/GPSEnv administrator;
• start setup of a GPSJack/GPSEnv;
• registration of a new user;
• definition of the profile of a registered user;
• eliminate a profile;
• execution of a base test for a registered user;
• registration of a new exercise;
• registration of a new benchmark for an existing exercise;
• assignment of a benchmark for a given exercise to a specific user;
• execution of an exercise;
• registration of a sequence of exercises as a training path;
• association of benchmark values for a training path;
• assignment of benchmark values for a training path to a given user;
• visualization of a single test progresses along a temporal interval;
• visualization of progresses with respect to a given benchmark;

Based on the above-defined functions, several background software instruments are required. The technologies for managing and analyzing the data from the sensors, which we may name the GPSJack Framework, have been envisioned. Nevertheless, machine-learning-based algorithms will guide the personalization of the benchmark process by employing intelligent reinforcement learning methods. Finally, since the proposed technology has its main applications in healthcare, it will provide, in addition to the classical data protection techniques, a physical protection layer that, based on the radio signals’ propagation patterns, will habilitate the possibility of utilizing the tablet if and only if the tablet is under a certain distance (e.g., 5 m) from the GPSJack nodes.

3.3. Architecture Workflow

The system’s architecture is composed of several modules, each one with a single responsibility, the logic model of which is reported in Figure 2.

Every module is related to at least another one:

User Interface: 

allows the user to input “environments” and provides visualization of all data/results returned by the Machine Learning module;

Configurator: 

takes as input a set of environments and gives as output a set of feasible “exercises” to be used by the Trainer, according to user input and machine learning algorithms’ indications;

Trainer: 

uses the exercises and determines the “should be” paths, meaning which exercises shall be executed and with which environment;

Evaluator: 

responsible for the evaluation of data and automatic comparison with benchmarks;

Sensor: 

deals with wearable sensors, collecting and normalizing data;

Machine Learning: 

gathers data and induces models.

The User Interface’s output is an ordered set of “environments”, which are one of the inputs of the Configurator. The Configurator continuously computes, and at given times uses models from the Machine Learning module to produce a set of feasible “exercises”. These exercises are to be given to the Trainer, whose outputs are chains of exercises, also named “paths”, to be executed by patients (or athletes). At different times, different paths are possible due to the work of the Trainer. The data are then gathered from the Sensor module and sent to the Evaluator to be stored, visualized, and compared with benchmarks. They are also used to devise possible “paths” and exercises to be delivered as hints to the user. This is the responsibility of Machine Learning, which acts as a feedback generator for the whole system, enabling the system to enhance performances continuously.

4. Methodology Design Workflow

This section presents the overall project information concerning hardware composing the GPSEnv and GPSJack, and software regarding edge computation and the preliminary data analysis pipeline. To achieve the system’s architectural requirements, we have designed a four-step method, illustrated in Figure 3.

-

the first step consists of the identification of the most suitable motor tasks;

-

the second step consists of the description and design of the used data collection systems composing the GPSEnv/GPSJack;

-

the third step exploits the collected data analysis workflow;

-

the fourth step exploits the modeling of individuals’ gait and posture invariants by machine learning.

4.1. Identification of Motor Tasks

The motor tasks should contain specific characteristics. We aimed to select tasks that challenge postural stability and require the ability to foresee and anticipate the consequences of actions given the presence of a sudden perturbation that might change posture from a stable to an unstable state. In this way, we will be able to train fundamental motor skills such as action adaptation, compensation, and anticipation, while, on the other hand, measuring the performance of such skills. We will describe a typical trial involving a sequence of movements that satisfies the task requirements stated above:

(1)

The subject stands on an unstable proprioceptive board (Figure 4) while holding a heavy ball. The task is to rotate at an angle of 30${}^{\circ }$ or 60${}^{\circ }$ or 90${}^{\circ }$, both right and left, while keeping his feet still and rotating only the torso, without losing equilibrium. Body balance is evaluated by analyzing the center of pressure (COP) migration. This sequence of activities identifies two distinct motor task phases: (a) rotate and (b) in position.

(2)

Once in position, the subject is asked to drop the ball quickly (in the 150 ms preceding this action, the anticipatory postural adjustments (APAs) can be identified and analyzed) captured by the EMG and the force platform. This sequence of activities identifies two distinct motor task phases: (c) APAs and (d) drop.

(3)

After the loss of balance (due to the fast drop of the ball), the subject is asked to regain the balance as fast as possible. The analysis of COP migration for defining the balance recovery after the perturbation is considered. This activity identifies one motor task phase: (d) recover.

We analyzed different experimental conditions to define the departure from the standard measurements by considering the same task while changing the biomechanical and perceptual conditions and testing different populations ranging from elite athletes to individuals affected by neuromuscular diseases. Figure 5 presents an overview of the force platform x-axis data of a motor task. As shown, it is composed by five different movements: (a) rotate, (b) in position, (c) APAs, (d) drop, and (e) recover. The task starts at most 10 s after the emission of an audio signal. Such a signal is later used during the manual synchronization of the GPSJack and GPSEnv data streams.

Data Collection Procedure

To validate the GPSJack system, subjects of different ages, genders, and motor backgrounds performed the introduced protocol of exercises. This protocol aimed to analyze the tested subjects’ gait and posture using state-of-the-art instrumentation to identify the anticipatory movements and minimum requirements that the GPSJack nodes (as shown in Figure 1) will have to implement.

Figure 4 presents the proprioceptive board. Area dimensions were 0.45 m × 0.45 m, and height was 0.025 m. On the bottom of the surface, the board was touching the ground utilizing a beam glued along the board mid-line, having the same length as the wooden board, and being 0.025 m in height and 0.06 m in width. Figure 6, on the left, presents how the board was used during the exercise.

4.2. Data Collection Systems

This section presents the instruments involved in the design of the GPSEnv and GPSJack systems. GPSEnv is defined as a state-of-the-art apparatus. GPSJack was designed to be a low-cost and long-battery-life system on chip (SoC).

4.2.1. GPSEnv Apparatus

It performs gait and posture analyses based on the combination of three different instruments:

• Force platform;
• Surface EMGs;
• Motion capture cameras.

Force Platform

The forces in three orthogonal directions, along with the COP migration, are measured by a force platform (https://tinyurl.com/rhkktv4 accessed on 10 December 2022), coupled with a 6-channel strain gauge amplifier (https://tinyurl.com/zy2efpn8 accessed on 10 December 2022), with a sampling frequency of 1000 Hz, presenting a size of 0.9 m × 0.9 m. The AMTI Biomechanics Force Platform, model BP900900, features composite construction, resulting in a low-mass instrument with excellent frequency response. Specifically designed for the precise measurement of ground reaction forces, the BP900900 measures the three orthogonal force components, the moments about the three axes, and the center of pressure in the horizontal plane producing a total of eight outputs. The high sensitivity, low crosstalk, excellent repeatability, and long term stability of this platform make it ideal for research and clinical studies. The BP900900 is easy to use and is available with 1000, 2000, or 4000 pound (4450, 8900, or 17,800 Newtons) vertical capacity. This force platform is shown in Figure 6, on the left part, under the subject’s feet, and in right image, represented virtually as platform 1.

Surface electromyography (EMG)

The surface EMG activity of sixteen postural muscles, on both sides of the body, is recorded using electrocardiographic electrodes located on the subject’s body, as shown by the red markers in Figure 7. The muscles used are: the rectus abdominis (RA), erector spinae (ES), rector femoris (RF), biceps femoris (BF), vastus lateralis (VL), tensor fasciae latae (TL), tibialis anterior (TA), and soleus (SO). Guidelines from the http://www.seniam.org/ accessed on 10 December 2022 (Surface ElectroMyoGraphy for the Non-Invasive Assessment of Muscles) are used to guarantee consistency in the muscles’ anatomical localization. The https://fccid.io/VH6ZWTX07/User-Manual/User-Manual-903877 accessed on 10 December 2022 EMG system, produced by Aurion S.r.l., is used to collect and amplify EMG signals at a sampling rate of 1000 Hz.

Motion Capture Cameras

Concerning the kinematic analysis, five retro-reflective markers are attached to each subject. Markers’ positions are located on the subject’s body as shown by the black markers in Figure 7.

The markers are placed on the backs of both hands, on the forehead, on the lowerback, and on the lateral malleoulus on the dominant leg. The position, velocity, and acceleration of every marker are recorded at a sampling rate of 200 Hz, using eight motion capture cameras (https://www.evl.uic.edu/sjames/mocap/resources/Doc/MXhardware_Reference.pdf accessed on 10 December 2022), featuring multiple high-speed processors that perform real-time proprietary image processing and Vicon Nexus 2.6 software.

4.2.2. GPSJack Apparatus

The GPSJack prototype uses the nRF52840 system on chip (SoC), built over the 32-bit ARM CortexTM-M4 CPU with a floating-point unit running at 64 MHz. The nRF52840 is the most advanced member of the Nordic Semiconductor nRF52 Series SoC family (https://www.nordicsemi.com/Products/Low-power-short-range-wireless/nRF52840 accessed on 10 December 2022). It is fully multi-protocol and capable of supporting Bluetooth 5, Bluetooth mesh, Thread, Zigbee, 802.15.4, ANT, and 2.4 GHz proprietary stacks [48]. Furthermore, the nRF52840 uses a sophisticated on-chip adaptive power management system achieving exceptionally low energy consumption.

This SoC interfaces with various electronic devices capable of perceiving different types of measurement of the movement context. In the setup we discuss here, the nRF52840 defines the core of each GPSJack node. On the same HW board, each SoC core communicates with various sensors, such as an accelerometer, gyroscope, and magnetometer. However, since the design of every single node of the GPSJack from scratch would require further targeted effort, and since the goal of the project is to show the suitability of the system in recognizing the different phases of the defined task, we made use of the Nordic Thingy 52 IoT Sensors kit shown in Figure 8. In future developments, the Nordic Thingy 52 can be replaced by dedicated data collection nodes based on the nRF52840 SoC, presenting reduced dimensions and integrating only sensors relevant to the scenario. It transmits data to/from its sensors and actuators to a receiver implemented through a PC, single board computing (SBC) (e.g., Raspberry pi 4 or Odroid H2+), or a mobile application running on a tablet or smartphone [48]. Extended device characteristics are listed in the following:

• Dimensions: 5 cm × 5 cm × 1.5 cm, weight: 47 g;
• Motion-tracking sensors: nine axis motion sensor including 3-axis gyroscope, 3-axis accelerometer, and 3-axis magnetometer;

  -

  Sampling frequency: up to 200 Hz;

  -

  Full scale: up to 16 g for accelerometer and up to 2000 dps for gyroscope;

• Battery: rechargeable Li-Po battery with 1440 mAh capacity;
• Microprocessor: 64 Mhz Cortex M4 MCU;
• Communication: Bluetooth Low Energy (BLE);
• Cost: 38 Dollars.

Nodes positions and number are not definitive, since their positions directly depend on the preliminary study carried out with the previously introduced instrumentation. The GPSJack sampling frequency, the number of nodes, and the positions will be adjusted (reduced) based on the previous phase’s outcome, thereby reducing the battery consumption of the overall system.

Moreover, the designed GPSJack system can be composed of up to 11 different nodes that collect data and communicate with a single tablet or SBC device. Based on the performed tests, the designed GPSJack (configured as shown in Figure 1, 5 nodes and 1 data aggregator) can communicate, without data loss, with the tablet device at a maximal distance of 45 m. Concerning the battery life, the GPSJack data collection nodes can compute for more than 48 h at a sampling frequency of 200 Hz.

Figure 9 presents an overview of the main characteristics of the GPSJack android mobile application running over a tablet device, presenting data collection and visualization.

The raw data (aka, time series), perceived by the GPSJack prototype (at a maximal sampling frequency of 200 Hz) and the existing in-laboratory architecture (perceived at a frequency of 200–1000 Hz), will be pre-treated by applying different data processing steps.

Finally, the GPSJack android mobile application executing on a tablet device has the ability to video-record at 60 FPS the performed tasks. Such recording is synchronized with the data stream perceived by the data collection nodes (i.e., Nordic Thingy 52).

4.2.3. GPSJack/GPSEnv Synchronization

The GPSJack and the GPSEnv data streams present different timestamps and are not synchronized. The synchronization is manually performed offline, using the audio start signal emitted by the Vicon system and the video recording of the task performed by the GPSJack system. The annotator identifies the precise timestamp of the GPSJack system where the start audio signal is emitted by the GPSEnv system. In particular, the annotator identifies the precise video frame during which the signal is emitted (i.e., a granularity of 16 ms). In future developments, the aim will be to automatically synchronize the GPSJack/GPSEnv data streams using existing solutions [49], thereby excluding the time-consuming offline synchronization process.

4.3. Data Analysis

This section presents, starting with the data collected by the mentioned instruments (i.e., EMGs, force platform, kinematic, and GPSJack), the main data processing steps, performed with different processing methods in a defined order [50]. This workflow is presented in Figure 10.

4.3.1. Data Cleaning

This phase emphasizes data patterns by reducing their dependence on environmental/HW noise and the data collection architecture, which often leads to data loss or corruption during transmission. In particular, dedicated data cleaning techniques must handle missing or corrupted data to maintain the time series structure and information. In this step, the corrupted and missing data issue is handled by applying an interpolation data-filling method that replaces such data with a value that follows the time series’s previous and consequent pattern [51]. The noise’s impact is reduced by applying a 4-order low-pass filter with a cut-off frequency of 20 Hz. We do not apply any data cleaning method to the existing in-lab architecture, since the architecture’s proprietary software already performs such a step.

4.3.2. Feature Extraction

Furthermore, since standard pattern-recognition models are not always suitable for raw data, the machine learning training phase is anticipated by a feature extraction step during which time-series are represented as a set of features in the time and/or frequency domain [52,53]. We make use of the https://github.com/fraunhoferportugal/tsfel accessed on 10 December 2022 [54] to represent each time window of 150 ms (equal to the APAs movement duration) perceived by the mentioned instrumentation in a set of 160 features in the frequency and time domains.

Table 1. Most used time and frequency-domain features.

Time Domain Features

Frequency Domain Features

(1) maximum, (2) minimum, (3) mean, (4) standard deviation, (5) root mean square, (6) range, (7) median, (8) skewness, (9) kurtosis, (10) time-weighted variance, (11) interquartile range, (12) empirical cumulative density function, (13) percentiles (10, 25, 75, and 90), (14) sum of values above or below percentile (10, 25, 75, and 90), (15) square sum of values above or below percentile (10, 25, 75, and 90), (16) number of crossings above or below percentile (10, 25, 75, and 90), (17) mean amplitude deviation, (18) mean power deviation, (19) signal magnitude area, (20) signal vector magnitude, (21) covariance, (22) simple moving average of sum of range of a signal, (23) sum of range of a signal, (24) sum of standard deviation of a signal, (25) maximum slope of simple moving average of sum of variances of a signal, (26) autoregression.

(1) Fast Fourier Transform (FFT) coefficients, (2) Discrete Fourier Transform (DFT), (3) Discrete Wavelet Transform (DWT), (4) first dominant frequency, (5) ratio between the power at the dominant frequency and the total power, (6) ratio between the power at frequencies higher than 3.5 Hz and the total power, (7) two signal fragmentation features, (8) DC component in FFT spectrum, (10) energy spectrum, (11) entropy spectrum, (12) sum of the wavelet coefficients, (13) squared sum of the wavelet coefficients and energy of the wavelet coefficients, (14) auto-correlation, (15) mean-crossing rate, (16) spectral entropy, (17) spectral energy, (18) wavelet entropy values, (19) mean frequency, (20) energy band.

shows the most commonly extracted time (e.g., min, max, mean, std, etc.) and frequency (e.g., Fast Fourier Transform (FFT), Discrete Fourier Transform (DFT), Discrete Wavelet Transform (DWT), etc.) domain features that are usually combined, further increasing the recognition accuracy. Other features, extracted by using other frameworks or handcrafted (extracted manually), can be used, since the proposed workflow is modular. Thus, we can easily substitute each module.

BUsers could also decide not to apply the feature extraction step and use the date in the form provided by the previous block applying a standard data segmentation phase. In such a case, the feature selection algorithm is not applied [13,50].

4.3.3. Preprocessing of Features

The extracted features could present a wide range of values that will govern the training process, but such features are not those that primarily represent the dataset’s characteristics or the final pattern-recognition model’s accuracy. Data normalization transforms multi-scaled data to the same scale, and all variables equally influence the model, improving the learning algorithm’s stability and performance [55]. Our methodology makes use of the robust scaling normalization technique that scales each feature of the dataset by subtracting the median ($Q_2\left(x\right)$) of this feature and then dividing by the interquartile range (IQR) ($Q_3\left(x\right)-Q_1\left(x\right)$). This scaler is robust to outliers, in contrast with the other scalers that arehighly affected by outliers. When working with datasets in which different features are used to represent every single sample, the datasets perform independent normalization for every feature.

Moreover, a large number of features does not imply high recognition quality, since they can positively or negatively impact the recognition process. Therefore, feature selection techniques identify features that positively and negatively influence the recognition process, reducing the model’s dependence on irrelevant features. The exclusion of a certain number of features decreases the training process’s complexity, since a smaller dataset generally requires less training time. The main benefits of such techniques are (i) reducing overfitting by eliminating redundant data, which consequently reduces noise-related issues; (ii) improving accuracy, since misleading data are eliminated; (iii) reducing training time due to fewer data points; and iv) raising interest in certain features demonstrating higher importance [55,56]. We use the tree-based feature selection technique to compute impurity-based feature importances, discarding irrelevant features in cooperation with other feature selection techniques.

In conclusion, the execution of this series of data treatment steps transforms the raw data, subject to noise and errors, to an optimal number of features in the time and frequency domains. This features set will be used by the machine learning models in the fourth phase of the methodology, as shown in Figure 3. The features-preprocessing step is anticipated by the hold/leave-out validation techniques, generating the training and testing datasets. Then, the training dataset will be preprocessed as mentioned above, and then the testing dataset is preprocessed based on the training dataset’s requirements.

5. Preliminary Experimental Evaluation

Following the experimental design of Section 4, we collected data from 12 different male subjects, whose physical characteristic are shown in

Table 2. Subjects’ characteristics.

Subject	Age (Years)	Height (cm)	Weight (kg)
22	24	182	72.6
23	24	183	61.1
24	22	186	97.7
25	28	176	65.1
26	28	172	69.7
28	23	170	64.7
29	24	174	69.0
30	28	176	78,0
31	25	187	86.9
32	23	189	78.1
33	23	176	66.9
34	19	175	72.8
Min.	19	170	61.1
Max.	28	189	97.7
Avg.	24.76	178.65	73.71
Std.	2.78	6.15	9.97

.

At the end of the data collection phase, each subject had performed a total of 84 7 tasks (i.e., do not rotate, rotate of an angle of 30${}^{\circ }$ or 60${}^{\circ }$ or 90${}^{\circ }$, both right and left) × 2 statuses (i.e., stable/unstable) × 6 repetitions) data collection sessions. Each session involved the five movements phases (i.e., rotate, in position, APAs, drop, and recover) described in Section 4.1 and shown in Figure 5.

In this preliminary evaluation, the collected data are divided based on the data collection technologies we utilized (force platform, EMGs, or acceleration data) into three different datasets, as shown in

Table 3. Overview of the three types of datasets analyzed.

Dataset	Time	Force Platform	EMG’s	Acceleration	Sampling Frequency
ID	[ms]	(#1)	(#10)	(#9)	[Hz]
A	✓	✓	✓		1000
B	✓			✓	200
C	✓	✓	✓	✓	200

.

Subsequently, for each dataset type (i.e., A, B, and C), we applied the data-processing pipeline defined in Section 4.3 and shown in Figure 10. In particular, the data, in segmentation and feature representation forms were segmented in time windows of 150 ms as the hypothetical duration of the APA movement phase), were used to train three different machine learning models, whose performances in recognizing the movement phases of Figure 5 were measured in terms of accuracy A), precision (P), recall (R), and F1-score (F1), defined as follows [57]:

$$\begin{array}{ll}A=\frac{tp+tn}{tp+tn+fp+fn}& P=\frac{tp}{tp+fp}\\ R=\frac{tp}{tp+fn}& F1=2\times \frac{P\times R}{P+R}\end{array}$$

Here, tp represents the number of true positives, n represents the number of true negatives, fn represents the number of false negatives, and fp represents the number of false positives.

5.1. Preliminary Results

Table 4. Average results for segmentation and feature-extraction data representation of all 12 subjects’ data.

	Segmentation												Feature Extraction
	A				B				C				A				B				C
Model	A	P	R	F1	A	P	R	F1	A	P	R	F1	A	P	R	F1	A	P	R	F1	A	P	R	F1
k-NN	58.1	56.2	58.1	55.2	47.4	45.0	47.4	44.0	57.3	55.5	57.3	54.4	42.7	39.4	42.7	40.2	38.9	36.8	38.9	35.8	71.4	67.7	71.4	69.2
RF	77.7	80.2	77.7	76.0	54.5	55.3	54.5	48.3	88.8	89.3	88.8	88.2	84.0	84.8	84.0	83.5	63.0	68.6	63.0	59.5	89.9	90.2	89.9	89.6
LDA	49.2	48.9	49.2	47.3	42.5	64.7	42.5	25.7	49.5	49.2	49.5	48.4	36.8	34.6	34.5	31.45	61.3	67.6	61.3	56.9	65.1	67.4	65.0	63.6

and

Table 5. Results on segmentation and feature extraction data representation for each subject.

		Segmentation												Feature Extraction
		A				B				C				A				B				C
Subject	Model	A	P	R	F1	A	P	R	F1	A	P	R	F1	A	P	R	F1	A	P	R	F1	A	P	R	F1
P22	k-NN	56.8	55.6	56.8	54.4	40.0	42.1	40.0	39.0	56.5	55.3	56.5	54.2	39.4	37.4	39.4	37.7	34.5	31.9	34.5	32.6	44.5	60.5	44.5	48.9
	RF	83.6	84.8	83.6	82.7	54.1	55.9	54.1	49.2	84.9	85.7	84.9	84.1	83.6	84.8	83.6	83.0	61.5	63.3	61.5	59.4	83.9	85.1	83.9	83.2
	LDA	59.5	59.5	59.5	59.4	32.1	30.8	32.1	31.2	54.8	56.4	54.8	55.5	39.3	76.0	39.3	40.0	3.0	8.3	3.0	0.4	44.5	60.5	44.5	48.9
P23	k-NN	66.7	67.3	66.7	64.3	42.1	42.8	42.1	37.4	65.8	63.5	65.8	63.0	49.1	46.8	49.1	47.8	37.8	34.7	37.8	35.0	82.9	81.4	82.8	80.3
	RF	86.1	87.7	86.1	85.6	48.9	47.9	48.9	43.5	99.8	99.8	99.8	99.8	87.6	88.0	87.6	87.2	62.4	64.9	62.4	60.3	100	100	100	100
	LDA	61.1	61.9	61.1	61.4	37.8	33.7	37.8	33.4	98.6	98.7	98.6	98.6	30.5	0.1	3.1	0.01	3.6	12.9	3.6	1.2	97.4	97.5	97.4	97.4
P24	k-NN	63.7	64.0	63.7	62.6	46.1	43.9	46.1	41.0	64.4	63.7	64.4	63.1	46.1	44.2	46.1	44.9	40.9	38.0	40.9	37.2	65.8	61.6	65.8	63.5
	RF	89.4	90.8	89.4	88.8	56.2	60.6	56.2	49.8	88.1	89.6	88.1	87.3	91.4	91.7	91.4	91.1	65.3	70.5	65.3	62.3	92.1	92.3	92.1	91.7
	LDA	59.7	60.8	59.7	60.0	35.0	32.3	35.0	33.8	52.8	57.2	52.8	54.5	58.2	66.6	58.2	57.1	30.0	34.4	30.0	15.0	61.8	64.3	61.8	62.9
P25	k-NN	63.3	62.8	63.3	60.9	49.5	50.9	49.5	46.5	63.4	63.0	63.4	61.1	48.0	45.4	48.0	46.4	37.6	37.6	37.6	34.9	68.3	65.0	68.3	66.2
	RF	85.0	85.6	85.2	84.7	55.9	63.4	55.9	52.1	90.1	90.3	90.1	89.7	86.1	85.9	86.1	85.8	65.2	72.3	65.2	62.4	90.1	90.0	90.1	89.7
	LDA	55.9	56.1	55.9	55.7	37.5	35.2	37.5	35.6	54.1	54.4	54.1	54.1	20.0	7.8	20.0	10.5	3.7	11.6	3.7	1.9	40.0	46.4	40.0	31.8
P26	k-NN	65.0	64.1	65.0	62.3	48.1	46.4	48.1	43.1	65.0	64.1	65.0	62.3	49.7	46.7	49.7	47.5	37.3	32.9	37.3	34.2	80.4	76.9	80.4	77.7
	RF	86.3	86.5	86.3	85.9	58.7	61.5	58.7	52.7	99.8	99.8	99.8	99.8	88.4	88.3	88.4	88.1	65.3	65.2	65.3	60.8	100	100	100	100
	LDA	57.9	58.6	57.9	58.2	41.7	38.7	41.7	38.6	98.3	98.4	98.3	98.3	41.3	25.7	41.3	31.0	31.4	11.2	31.4	15.8	98.3	98.4	98.3	98.3
P28	k-NN	65.5	64.3	65.5	62.8	46.7	46.2	46.7	41.7	65.5	64.3	65.6	62.8	52.3	49.5	52.3	50.2	38.7	33.6	38.7	35.0	64.0	60.7	64.1	62.2
	RF	84.2	85.4	84.2	83.1	56.7	60.7	56.7	49.7	85.8	86.5	85.8	84.8	86.1	86.8	86.1	85.4	63.4	67.1	63.4	59.0	85.5	86.1	85.5	84.8
	LDA	58.8	59.3	58.8	59.0	46.7	41.4	46.7	39.1	53.6	56.2	53.5	54.5	56.6	62.7	56.6	60.6	3.4	17.9	3.4	0.6	50.8	57.0	50.1	52.5
P29	k-NN	61.0	60.1	61.0	58.1	46.8	47.5	46.8	42.7	61.1	60.6	61.1	58.4	49.0	46.0	49.0	46.7	39.6	37.9	39.6	36.5	69.4	65.4	69.4	67.0
	RF	85.0	87.0	85.0	84.3	59.3	59.5	59.3	51.4	84.8	86.5	84.8	84.0	86.1	87.4	86.1	85.6	65.1	68.4	65.1	58.9	87.8	88.8	87.8	87.3
	LDA	61.7	61.3	61.7	61.3	39.0	35.1	39.0	35.9	54.6	57.4	54.6	55.4	59.9	65.6	59.9	61.5	46.9	22.9	46.9	30.5	49.2	54.9	49.2	50.3
P30	k-NN	59.2	62.0	59.2	55.9	51.4	54.5	51.4	43.5	59.7	59.3	59.7	56.5	42.6	39.8	42.6	40.4	37.9	34.9	37.9	34.6	77.8	75.3	77.8	74.2
	RF	84.6	85.8	84.6	83.6	61.8	65.0	61.8	57.2	99.8	99.8	99.8	99.8	86.4	87.1	86.4	85.9	69.9	72.8	69.9	67.3	99.8	99.8	99.8	99.8
	LDA	53.4	55.3	53.4	54.1	41.1	38.3	41.1	38.9	99.0	99.0	99.0	99.0	11.9	45.4	11.9	13.4	30.1	17.5	30.1	14.9	98.7	98.8	98.7	98.8
P31	k-NN	58.2	55.4	58.2	54.8	45.8	44.3	45.8	38.3	55.6	52.6	55.6	52.5	42.9	39.8	42.9	39.4	38.9	37.1	38.9	35.7	77.8	72.7	78.8	74.4
	RF	81.2	83.1	81.2	80.2	53.7	59.6	53.6	47.5	99.8	99.8	99.8	99.8	86.4	87.3	86.4	85.8	62.0	67.2	62.0	58.9	99.9	99.9	99.9	99.9
	LDA	59.8	60.0	59.8	59.7	44.1	41.1	44.1	40.4	98.9	98.9	98.9	98.9	65.3	66.1	65.3	65.3	16.8	66.1	16.8	20.5	98.4	98.4	98.4	98.4
P32	k-NN	68.3	67.8	68.3	65.1	30.7	36.8	30.7	30.2	68.3	67.8	68.3	65.0	56.3	53.7	56.3	54.3	41.0	37.5	41.1	38.2	71.2	67.5	71.2	68.7
	RF	87.2	87.3	87.2	86.1	58.5	63.6	58.5	53.2	89.3	87.0	89.3	87.9	88.3	87.5	88.3	87.5	67.4	66.1	67.4	64.4	88.6	87.9	88.6	87.9
	LDA	56.8	56.7	56.8	56.7	42.3	37.5	42.3	37.9	53.6	55.5	53.6	54.4	15.3	3.9	15.3	06.2	18.6	20.3	18.6	7.0	42.2	25.4	42.2	28.8
P33	k-NN	69.2	65.5	69.2	66.5	45.0	46.1	45.0	43.4	69.4	65.3	69.4	66.7	53.2	49.0	53.2	50.3	38.1	35.0	38.1	34.8	70.4	66.5	70.4	67.7
	RF	89.8	90.3	89.8	89.2	53.9	54.8	53.9	48.2	92.0	92.5	92.0	91.3	91.4	91.2	91.4	90.9	63.9	64.7	63.9	59.6	93.8	94.0	93.8	93.2
	LDA	60.0	60.3	60.0	60.1	38.9	37.7	38.9	37.3	54.2	56.4	54.2	55.0	14.3	45.8	14.3	18.3	3.3	35.1	3.3	0.8	50.0	57.6	50.0	46.9
P34	k-NN	73.4	73.3	73.4	71.6	52.8	51.4	52.8	45.6	73.9	73.9	73.9	72.0	57.6	54.1	57.7	55.6	40.5	36.4	40.5	37.2	76.4	73.0	76.4	74.5
	RF	91.6	91.8	91.6	91.3	58.8	67.9	58.8	52.1	93.8	94.1	93.8	93.4	91.7	91.6	91.6	91.5	66.2	70.1	66.2	60.5	92.4	92.3	92.4	92.2
	LDA	63.3	63.5	63.3	63.4	43.1	37.6	43.1	39.0	57.7	58.9	57.7	58.2	28.8	9.1	28.8	13.4	28.9	26.8	28.9	13.7	49.5	49.2	49.5	48.4
k-NN	Min	56.8	55.4	56.8	54.4	30.7	36.8	30.7	30.2	55.6	52.6	55.6	52.5	39.4	37.4	39.4	37.7	34.5	31.9	34.5	32.6	44.5	60.5	44.5	48.9
	Max	69.2	67.8	69.2	66.5	51.4	54.5	51.4	46.5	69.4	67.8	69.4	66.7	56.3	53.7	56.3	54.3	41	38	41.1	38.2	82.9	81.4	82.8	80.3
	Avg	63.4	62.6	63.4	60.7	44.7	45.6	44.7	40.6	63.2	61.8	63.2	60.5	48.1	45.3	48.1	46.0	38.4	35.6	38.4	35.3	70.2	68.5	70.3	68.3
RF	Min	81.2	83.1	81.2	80.2	48.9	47.9	48.9	43.5	84.8	85.7	84.8	84.0	83.6	84.8	83.6	83	61.5	63.3	61.5	58.9	83.9	85.1	83.9	83.2
	Max	89.4	90.8	89.4	88.8	61.8	65	61.8	57.2	99.8	99.8	99.8	99.8	91.4	91.7	91.4	91.1	69.9	72.8	69.9	67.3	100	100	100	100
	Avg	85.3	86.4	85.3	84.5	56.4	59.8	56.4	50.6	92.2	92.5	92.2	91.7	87.0	87.5	87.0	86.5	64.8	67.8	64.8	61.4	92.8	93.0	92.8	92.4
LDA	Min	53.4	55.3	53.4	54.1	32.1	30.8	32.1	31.2	52.8	54.4	52.8	54.1	11.9	0.1	3.1	0.01	3.0	8.3	3.0	0.4	40.0	25.4	40.0	28.8
	Max	61.7	61.9	61.7	61.4	46.7	41.4	46.7	40.4	99.0	99.0	99.0	99.0	65.3	76.0	65.3	65.3	46.9	66.1	46.9	30.5	98.7	98.8	98.7	98.8
	Avg	58.5	59.0	58.5	58.6	39.7	36.4	39.7	36.5	71.8	73.2	71.8	72.3	39.8	42.0	37.1	34.6	18.8	22.3	18.8	10.8	68.1	70.2	68.1	66.8

present the results of our preliminary analysis. Table 4 presents the results for datasets A, B, and C from all subjects simultaneously by performing a k-fold test (k = 5) on all subjects’ data. Table 5 presents the results for datasets A, B, and C for every single subject by performing a k-fold test (k = 5) on each subject’s data.

The results of Table 4 show that the random forest performed the best on all three datasets (i.e., A, B, and C), and for both data-treatment types (i.e., segmentation and feature extraction). Moreover, when differentiating by dataset type, the results show that the conjunct information of dataset A (i.e., EMGs and Force Platform) and dataset B (i.e., acceleration values) was put to use significantly better than when used separately. In particular, in terms of F1-score, there was an increment of 12.2% from dataset A to dataset C and 39.9% from dataset B to dataset C. Such results indicate that the acceleration provides precious information concerning the recognition of the studied activities. Overall, the achieved F1-scores for all subjects using the random forest model and dataset C were 88.2% and 89.6%, respectively, in segmentation and feature representation modes.

Table 5 shows that when training and testing the models with one specific subject, the models’ performances are subject-dependent. As shown from the statistics of each model for all subjects (bottom of the table), the RF model performed much better (F1 score > 91.6%) in both segmentation and feature representation modes, showing that the recognition accuracy, in terms of F1-score, is on average 5% higher than when training and testing with all 12 subjects’ data (see Table 4). Again, differentiating by dataset type, the results show that the conjunct information of dataset A (i.e., EMGs and Force Platform) and dataset B (i.e., acceleration values) performed significantly better than when used separately. In particular, on average, there was an increment of 7.3% in the F1-score from dataset A to dataset C and 31.1% from dataset B to dataset C.

5.2. GPSJack Evaluation

Concerning the suitability and principal characteristics of the GPSJack system, this section presents its evaluation in terms of RAM, storage, CPU, battery consumption, and data loss. In particular, a Samsung Galaxy Tab A7 with the following HW/SW characteristics has been tested in the setup described below.

• Processor: Qualcomm Snapdragon Octo-Core (4 × 2 GHz + 4 × 1.8 GHz);
• Operating system: Android 10;
• RAM: 3 GB;
• Storage: 32 GB;
• Connectivity: Bluetooth 5.0, Wi-Fi a/b/g/n/ac Dual Band;
• Battery: 7040 mAh;
• Dimension: 247.6 × 157.4 × 7 mm per 476 g;
• Price: $140.

Three Nordic Thingy 52 were connected to the Samsung Galaxy for three consecutive hours. The setup was tested once for each sampling frequency: (i) 50 Hz, (ii) 100 Hz, and (iii) 200 Hz.

Table 6. Data aggregators’ profiling using Android Studio 4.1.2.

	Galaxy Tab A7
Frequency (Hz)	50	100	200
RAM (MB/h)	147	180	224
Storage (Mb/h)	50	102	185
CPU (%)	31	40	46
Battery (mAh)	312	325	342
Data loss (%)	0	0	0

presents the evaluation results obtained using the Android Studio Profiler suit.

As observed, the designed system can work on various setups with no data loss and low storage, RAM, and CPU usage. Moreover, its battery consumption allows a data collection phase of almost 8 h. Based on the tests performed during the project, the Nordic Thingy 52 can efficiently compute for more than 48 consecutive hours at 200 Hz.

5.3. Discussion

The results of these preliminary experiments clearly show that the conjugate of EMG, Force Platform, and acceleration data performs considerably better than their separate utilization. This improvement enables a better understanding and in-depth study of human motion. In fact, the GPSJack prototype has good potential to capture relevant information, enabling the possibility to recognize the studied motion classes with an average F1-score of 89.6% when using all the subjects’ data at once. Furthermore, when tested on single subjects, the F1-score ranged from a minimum of 83.2% to a maximum of 100%, outperforming the usage of only one of the aforementioned data sources. Even though the collected acceleration data present precious information, further work must be conducted to increase the overall performance and reduce the dependence on state-of-the-art technology. This can be done by: (i) implementing more complex recognition models than the used k-NN, RF, and LDA; and (ii) exploiting the utilization of a larger number and different positions of data collection nodes on the human body. Nevertheless, the information generated by the EMG and Force Platforms is paramount; thus, a possible next step in addition to those mentioned above would be the integration of EGM sensors into the same GPSJack nodes.

6. Conclusions and Further Development

This paper has dealt with defining the experimental design of the “Biofeedback Wearable and Environmental Technologies for Postural Correction” project. We illustrated the target technology, described the project’s evaluation workflow (i.e., state-of-the-art instruments and low-cost wearable sensors, data processing flow, and machine learning-based analysis), and provided a high-level description of the context in which the envisioned technologies are forecasted to operate. In particular, we devised a methodology investigating how to build the parameters that allow the physiatric medical staff to evaluate the patient. Three challenging motor tasks were identified to, on the one hand, train fundamental motor skills such as action adaptation, compensation, and anticipation, and on the other hand, to measure the performance levels fir such skills. To measure the quality of the performed motor tasks, we evaluated a low-cost body area network (aka GPSJack). GPSJack uses at most eleven data collection nodes integrating an accelerometer, gyroscope, and magnetometer sensors and can compute for 48 h at a sampling frequency of 200 Hz. Moreover, we used an android mobile application that works as a data aggregator and controller for the GPSJack system, through which the user can observe the collected data and the posture and gait quality indicators. Nevertheless, in conjunction with the GPSJack, we integrated the GPSEnv based on state-of-the-art gait and posture evaluation systems.

Tests on data collected from 12 subjects for a total of 84 data collection sessions each showed that the designed system could highly accurately recognize the phases of the defined motor tasks. In particular, in a subject-independent setup, we achieved an F1-score of 89.6% in recognizing the five studied movement states (i.e., rotate, in position, APAs, drop, and recover). With a subject-dependent setup, the F1-score ranged from 100% for subjects 23 and 26 to 83.2% for subject 22. These results show that the acceleration information that we will add to the state-of-the-art systems significantly increases recognition capabilities.

It is widely accepted in the community of psychiatric medicine that proper quantification of the postural system’s efficiency represents an essential assessment for improving quality of life in the elderly, patients with neurological pathologies, and athletes. Moreover, since most of the actual measurements are made in a laboratory environment where natural movements are constrained by the instruments applied to subjects’ bodies and the environment, a system usable in uncontrolled and unconstrained environments (e.g., home, gym, or sports facilities) habilitates the individuals to move freely in their natural environment and perform the required motor tasks. Thus, the designed system will evaluate profiles from the performance viewpoint of individuals ranging from patients undergoing rehabilitation to top-level elite athletes in controlled and uncontrolled environments.

Since the final goal of the project is performing the defined task in uncontrolled environments and using only the acceleration information provided by a system such as GPSJack, the next step will concern the exploitation of the data collection phase while making use of a large number of nodes positioned on different body parts and of more complex pattern-recognition models than the used k-NN, RF, and LDA. In particular, deep learning models such as recurrent neural networks (RNN) and long short-term memory (LSTM) have shown optimal results in such fields. Moreover, since the information captured by the EMG sensors is paramount, integrating an EMG sensor into the GPSJack nodes should be considered.