Next Article in Journal
DC-Link Ripple Reduction for Parallel Inverter Systems by a Novel Formulation Using Multiple Space Vector-Based Interleaving Schemes
Next Article in Special Issue
Digital Twin Technology: New Frontiers for Personalized Healthcare
Previous Article in Journal
RSP-DST: Revisable State Prediction for Dialogue State Tracking
Previous Article in Special Issue
Introduction of a Novel Image-Based and Non-Invasive Method for the Estimation of Local Elastic Properties of Great Vessels
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

A Real-Time, Open-Source, IoT-like, Wearable Monitoring Platform

by
Andrea Baldini
1,
Roberto Garofalo
2,
Enzo Pasquale Scilingo
1,3 and
Alberto Greco
1,2,3,*
1
Dipartimento di Ingegneria Dell’informazione, School of Engineering, University of Pisa, 56122 Pisa, Italy
2
Feel-Ing SRL, 56122 Pisa, Italy
3
Research Center “E. Piaggio”, School of Engineering, University of Pisa, 56122 Pisa, Italy
*
Author to whom correspondence should be addressed.
Electronics 2023, 12(6), 1498; https://doi.org/10.3390/electronics12061498
Submission received: 8 February 2023 / Revised: 11 March 2023 / Accepted: 18 March 2023 / Published: 22 March 2023
(This article belongs to the Special Issue Digital Twin Technology: New Frontiers for Personalized Healthcare)

Abstract

:
The spread of informatics and electronic systems capable of the real-time monitoring of multiple psychophysiological signals has continuously grown in the last few years. In this study, we propose a novel open-source wearable monitoring platform (WMP) to synchronously acquire and process multiple physiological signals in a real-time fashion. Specifically, we developed an IoT-like modular and fully open-source platform composed of two main blocks that on the one hand connect multiple devices (the sensor fusion unit) and on the other hand process and store the sensors’ data through the internet (the remote storing and processing unit). To test the proposed platform and its computational performance, 15 subjects underwent an experimental protocol, in which they were exposed to rest and stressful sessions implementing the Stroop Color and Word Test (SCWT). Statistical analysis was performed to verify whether the WMP could monitor the expected variations in the subjects’ psychophysiological state induced by the SCWT. The WMP showed very good computational performance for data streaming, remote storing, and real-time processing. Moreover, the experimental results showed that the platform was reliable when capturing physiological changes coherently with the emotional salience of the SCWT.

1. Introduction

Neurophysiological data from both the central (CNS) and autonomic (ANS) nervous system, including electroencephalogram (EEG), electrocardiogram (ECG), electrodermal activity (EDA), photoplethysmography (PPG), and respiratory signal (RESP), encapsulate a variety of relevant information to assess the affective, cognitive, and physical state of human beings.
Recently, several studies have started to adopt multivariate approaches combining some of these bio-signals and contextual information to infer psychophysiological and physical states [1,2,3], even in contexts beyond the simple laboratory experimental setup (i.e., natural settings) [4,5]. In this scenario, biosignals have been used as important indicators for rehabilitation [6], epileptic seizure prediction [7], sleep stage scoring [8], affective computing [9], and arrhythmia detection [10].
The growing developments in biosignal-oriented applications, alongside the recent advancements in cost-effective and wearable devices (that fully fit the necessity of even natural settings experiments), have grown the interest in advanced technological tools capable of completely exploiting the huge amount of information derivable from such physiological signals. This technological progress has further enhanced the range of applicative contexts, including remotely and continuously monitoring driver drowsiness [11], human emotions [12,13,14,15], human activity [16,17], and health status [18,19]. In this context, several commercial software and open-source toolboxes have been developed to simplify the acquisition of biosignals, their real-time preprocessing, and the feature extraction procedures. For instance, Activeos is commercial proprietary software used to synchronize the outcomes of a specific pool of wearable products [20]. In [21], Chételat et al. developed a physiological monitoring system to acquire ECG and chest impedance in real time. In another previous study, Coates et al. [22] designed a multimodal skin sensing system to monitor diabetic feet. McGregor et al. [23] devised a cloud-computing framework to ease access to healthcare facilities for patients living in rural or remote centers. Other attempts have been made to detect dangerous events (e.g., sleep apnea) by using an integrated wearable monitoring system [24]. In addition, remote and wireless systems have been purposely developed to monitor vital signs and detect deterioration in ward patients early, thus representing valid support for physicians and nurses [25].
However, most of the previous software and applications have been built for deployment on stand-alone machines without exploiting the ease of monitoring, storing, and gaining sensor data from the internet provided by IoT-distributed architectures. In addition, they have often not been designed for real-time automatic data acquisition, synchronization, and processing and require intensive manual intervention [26]. Other important limitations concern open-source and scalability properties. Even if data acquisition and processing tools are freely provided to the user, they do not completely rely on open-source solutions that could enhance the whole system’s flexibility, adaptability, and cost-effectiveness [23,27]. The system’s scalability refers to the number and type of connectable devices to the monitoring platform. In many cases, such monitoring systems have been purposely developed to work with a specific device.
Interesting solutions to the previous issues have been proposed by [28,29,30]. OpenHealth is an open-source project that specifically focuses on movement disorders [28]. It addresses the challenge of providing a common platform with standardized hardware and software for patients suffering from these disorders. Another recently proposed open-source system, called RADAR-based, is a complex solution based on the Distributed File System that synchronizes and saves physiological data [30]. Similarly, Bahmani et al. have developed a smart health monitoring system through which the biomedical data collected by sensors are easily accessible to the physician for evaluation and analysis purposes, e.g., detection of presymptomatic COVID-19 [29]. However, despite being open-source, these systems have included elements that are self-developed or parts created by third parties that provide technical support and updates. This increases the risk that the termination of technical support could make the system obsolete.
This work presents a modular and fully open-source Wearable Monitoring Platform (WMP) integrating multiple commercially available devices to process physiological data in real-time. The platform is integrated into an IoT-like distributed architecture composed of two main blocks: the sensor fusion unit (SFU) and the remote data storage and processing unit (RDSPU). The SFU handles devices’ connection, synchronization, and data recording, whereas the RDSPU manages data storage in purposely selected databases and real-time processing of multiple signals. Four main characteristics characterize our system:
  • The use of open-source components: We adopt freely usable and customizable software to implement each system architecture block.
  • Scalability: We propose a modular system capable of interfacing several wearable devices.
  • Real-time processing: The processing is done in a short time period as the data are inputted, providing near-continuous output.
  • Remote storage: Both raw data and estimated features are remotely stored in purposely selected databases.
To test the performance of our system in collecting and effectively processing multiple kinds of psychologically relevant sensor data in real time, we have designed a standardized experiment in which 15 subjects underwent a stressful task. Particularly, a standard Stroop test has been designed during which physiological activity has been recorded and processed by the WMP. This task is notoriously capable of inducing a significant variation in the psychophysiological state of the subjects by increasing arousal and inducing a state of acute stress [31,32]. A statistical analysis has been performed to assess whether the wearable monitoring platform was reliable in estimating changes in the nervous system activity of the subjects.

2. Materials and Methods

This section describes the design of the WMP and the experimental paradigm and data analysis to evaluate it.

2.1. Platform Architecture

The WMP consists of an IoT-like distributed architecture comprised of two main blocks: the SFU and the RDSPU. Figure 1 depicts a representative schematic of the system. Specifically, the SFU represents the unit in charge of connecting the wearable devices, synchronizing the data, and routing all of them towards the RDSPU over MQTT protocol [33]. Instead, the latter remotely stores and processes the data to extract several features that could be used to estimate psycho-physiological states or drive output devices.

2.2. The Sensor Fusion Unit

The SFU aims to connect to the set of sensors and simultaneously acquire several physiological signals (e.g., EEG, ECG, EDA, PPG, RESP). To this critical end, open-source and multi-platform (e.g., Unix and Windows) Python software has been coded (Figure 2). Specifically, we have developed a multi-process architecture to fully leverage remote concurrency and effectively share computational resources among multiple processors of the machine running the application. A similar concurrent multi-process architecture facilitated the integration of even very heterogeneous devices in terms of the communication protocols, type of streamed data, and streaming performances. Given the fact that each device for biosignal acquisition exploits its own communication protocol and its own way of formatting and sending data, it is extremely difficult to devise a platform that is capable of handling a number of potentially unlimited communication modalities. In this study, we developed a flexible multi-process architecture that separates the communication with different sensors into separated and independent subprocesses purposely coded to exchange information following the communication protocol and data format supported by the respective device. This architectural solution not only guarantees the simultaneity of the signal acquisition needed to gain information from different sources but also allows interfacing devices to communicate in completely different ways. Although this modular structure forces the subprocesses to have no access to each other’s resources, the SFU still manages to synchronize the signals by exploiting the system clock. Moreover, a further subprocess is included to gather the signals acquired from the SFU and route them along with their respective timestamps over the MQTT protocol using the publish–subscribe model. The signals are streamed in the form of formally defined data messages. The multi-process architecture is developed by exploiting the Python multiprocessing library. The communication among the different entities within the SFU is implemented using the thread-safe FIFO queue of the library. In addition, we included the possibility of collecting and streaming metadata such as personal information and psychometric measures (e.g., S.T.A.I. Y1 [34], Beck depression inventory [35], Liebowitz Social Anxiety Scale [36]) that could be of potential interest for psychological studies. To facilitate the use of the platform, we have developed a Python user interface using the PyQt6 library [37] to handle the metadata collection, the subprocess spawning, and the data streaming (see Figure 2).

2.3. The Remote Data Storage and Processing Unit

The RDSPU is an IoT-like architecture integrating several components to manage, store, and process in real time the signals collected by the SFU. Specifically, the RDSPU is composed of a message broker, a data ingestion unit, a time-series database, a data processing block, and a NoSQL database. It is worth noting that we adopt open-source solutions for each block of the RDSPU. In particular, Eclipse Mosquitto™ is used as the message broker, the Telegraf (InfluxData Inc., San Francisco, CA, USA) plugin-driven agent to ingest messages, InfluxDB (InfluxData Inc.) is used to save signals value together with related timestamps, the Python-based custom service is used to process all signals to extract relevant features, and CouchDB (Apache) is used as the NoSQL database. Operationally, the formally-defined messages assembled by the SFU are adapted to the formal messaging protocol of the time-series database and routed towards it by the message broker and the data ingestion unit. Such a database is accessible from the processing service that performs queries, extracts signal features, and stores them in the NoSQL database. All the elements composing the RDSPU are currently hosted at the University of Pisa servers.
In the following paragraphs, each block of the RDSPU is explained in more detail.
The message broker is an intermediary message-oriented middleware that mediates the exchange of information between SFU and RDSPU. In particular, it receives incoming messages and performs message validation and transformation to adapt the data packet to the formal messaging protocol of the time-series database (i.e., InfluxDB). Each data packet contains several tags, including MQTT topic string, type of measurement (e.g., EDA, ECG, EEG, RESP), device information, group, subject profiling (e.g., name, age, email, UUID), psychometric measures, sample values, and a corresponding timestamp in Unix format. All the data gathered from the SFU are published on the same MQTT topic, and signals from different devices can be discriminated by combining device information and type of measurement. The publish–subscribe pattern supported by the message broker allows the whole application to run asynchronously, managing different streaming rates for the connected devices and fitting the necessity of acquiring signals with different spectral characteristics. Here, we adopt the open-source service Mosquitto (Eclipse Foundation) to handle and distribute the messages between SFU to the data ingestion block.
The data ingestion unit is configured to subscribe to the same MQTT topic used by the SFU to publish the gathered data. Therefore, once a new message is published on the message broker, it is made available to the data ingestion unit with negligible latency. The data ingestion unit acts between the message broker and the time-series database to reliably deliver the data to the storing medium (i.e., InfluxDB). For the data ingestion, we adopted the open-source software Telegraf (InfluxData Inc.), which provides flexibility in receiving metrics from several input sources and exploiting plug-in drivers to transmit them to several types of outputs.
The time-series database allows time-stamped physiological time-series to be stored in an effective and optimized way. Here, we adopted the open-source solution called InfluxDB (InfluxData Inc.) because of its wide range of ingestion methods (including Telegraf) and the capability of efficiently handling volumes of data that continuously and rapidly increase thanks to the simplified indexing methods. Moreover, it allows for data compression and efficient data access managed through optimized time-centric functions for querying time-series structures.
The data processing unit is a Python-based custom service running on the same remote server as Mosquitto, Telegraf, and InfluxDB. As for the SFU, in this block of the chain, the different signals are processed in parallel exploiting a multi-process paradigm. In particular, the service was purposely coded to query the InfluxDB database in different subprocesses, extract and pre-process the time-series data (i.e., EDA, PPG, EEG, RESP, and ECG) from it, and subsequently estimate a set of predefined features. As an exemplary data processing unit, according to the following experimental application (see Section 2.4), the features are calculated considering 30 s-long windows of signal updated every 5 s. In this way, the WMP can process each psychophysiological signal in a pseudo-real-time fashion updating the set of features every 5 s, thus fitting the temporal dynamics of many physiological processes that could be of interest for the possible applications of the WMP. The data processing unit is coded to perform an ad-hoc processing chain for feature extraction. As a preliminary step, all signals are filtered according to their frequency band of interest. In particular, the EDA is low-pass filtered at a cutoff frequency of 2 Hz, the ECG baseline was removed by high-pass filtering at 0.05 Hz, and the PPG and the EEG signals are band-pass filtered within the frequency range of 0.5–4 Hz and 1–45 Hz, respectively. After the filtering step, each signal is specifically processed as follows:
  • The EDA is decomposed into two main components: the skin conductance level (SCL) and the skin conductance responses (SCRs). The EDA components contain complementary information about the sympathetic nervous system (SNS). In particular, the SCL represents the EDA slow varying baseline and reflects the subjects’ general psychophysiological state [38]. The SCRs, instead, are relatively quick stimulus-evoked changes in the EDA signal [39]. In the WMP, the SCRs are obtained by passing the 30 s-long window of the EDA signal through a Butterworth high-pass filter with a cutoff frequency of 0.05 Hz [40]. The SCL is derived by subtracting the isolated fast-varying component from the original EDA signal. After the decomposition process, several features are extracted: the mean value (meanSCL, meanSCR) and the standard deviation (stdSCL, stdSCR) of both the SCR and SCL signals, the number of SCRs (nSCR), and the sum of their amplitudes (sSCR).
  • The ECG signals are processed to derive and analyze the heart rate variability (HRV) [41]. Operationally, the peak detection phase is performed by leveraging morphological changes in the ECG series by using an adaptive peak detection threshold followed by outlier detection and rejection. The algorithm threshold is adjusted stepwise, exploiting the relative regularity of the heart rate signal to minimize the standard deviation between successive differences. The R peaks are found using a 0.75 s-long window on both sides of each data point. Once the HRV time series are derived from the ECG signals, several time-domain, frequency domain, and nonlinear features are estimated. More specifically, in the time domain, we extract the mean value (meanRR) and the standard deviation (stdRR) of the R-to-R intervals, the square root of the mean squared differences of successive normal-to-normal (NN) intervals (RMSSD), and the percentage of the successive interval differences greater than 50 ms (pNN50). In the frequency domain, the low-frequency (LF; 0.04–0.15 Hz) and high-frequency (HF; 0.15–0.40 Hz) HRV spectral powers as well as their ratio (LF/HF ratio) are computed. Finally, the standard deviations of instantaneous beat-to-beat interval variability obtained from a Poincarè plot as the ellipse width (SD1) and length (SD2) are derived as nonlinear features. Such a feature set estimates the parasympathetic and sympathetic activities regulating the cardiovascular dynamics.
  • The EEG data are processed to estimate the EEG power within the classical frequency bandwidths: δ (1–4 Hz), θ (4–8 Hz), α (8–14 Hz), β (14–30 Hz), and γ (30–40 Hz) using the Welch’s method (window length = 4 s, overlap = 75%). Additionally, the frontal alpha asymmetry (FAA) is computed as the difference between right and left alpha activity over frontal regions (i.e., F4 and F3) [42]. This feature is thought to be a measure of the propensity to adopt approaching or avoiding behaviors and to be involved in the regulation of emotional stress [43].
  • The RESP is analyzed to derive the breathing rate within each 30 s-long time window. To this end, the RESP spectral power is obtained by applying the Fast Fourier Transform algorithm. Starting from the power spectrum, the breathing rate is estimated as the frequency corresponding to the maximum value of the spectrum.
  • The PPG is redundantly analyzed to derive the HRV, as in the case of the ECG signal. To this end, the PPG pulses are identified by comparing neighboring samples to identify all local maxima. Spurious PPG peaks are discarded by applying an adaptive amplitude threshold and a minimum distance between consecutive pulses. The HRV is then obtained by cubic-spline interpolation and resampling (at 4 Hz [44]) of the inter-pulse intervals time series. Afterwards, starting from the HRV signal, the same features described above (i.e., ECG features) are estimated.
The extracted features are then automatically saved into a NoSQL database, allowing for permanent storage and future remote access.

2.4. Experimental Evaluation

To evaluate the WMP in terms of connecting to multiple devices and real-time performance, we purposely selected two biomedical-oriented wearable acquisition systems: the DSI-24 device (Figure 3a) to collect CNS data and the Shimmer3 GSR+ unit (Figure 3b) to simultaneously gather peripheral ANS signals and motion data.
The DSI-24 system is a research-grade wireless dry electrode electroencephalograph (EEG) headset designed for the rapid application of 19 innovative dry-electrodes at locations corresponding to the 10–20 International System. This EEG system has been chosen due to its ability to guarantee an easy-to-use and comfortable measurement of the EEG signal in a wireless mode. Moreover, this system is equipped with three embedded accelerometers and three integrated auxiliary sensor inputs that can record peripheral signals such as ECG, EDA, and RESP. All the recorded signals are wirelessly streamed to a PC over standard Bluetooth communication and can be accessed through the TCP-IP protocol.
The Shimmer3 GSR+ unit is a widely used wearable device for biomedical purposes and, more specifically, for peripheral ANS signals acquisition. In particular, the GSR+ unit is designed to record EDA and PPG from Ag/AgCl dry electrodes placed on human hands. Furthermore, this unit integrates an inertial measurement unit for the real-time acquisition of motion signals as well (i.e., accelerometric and gyroscopic data). All the recorded data are streamed using the default method to a host PC over Bluetooth communication.
It is worthy of note that, thanks to the adopted flexible structure downstream of the SFU, WMP can interface with other devices by including additional processes specifically designed for the systems to connect.

2.4.1. Experimental Protocol

We recruited 15 healthy subjects (8 female, average age of all subjects (±standard deviation) = 28 ± 4) to test the performance of the WMP in reliably streaming signals and computing psychophysiological state correlates. The local ethical committee approved the experiment (n. 14/2019). The subjects underwent an experimental protocol composed of two three-minute-long sessions. The experimental timeline is reported in Figure 4.
Each subject was required to fill in the S.T.A.I. in its Y1 form before and after the experiment to self-assess the psychologically perceived anxiety level. More specifically, the S.T.A.I. Y1 questionnaire evaluates state anxiety through multiple-choice questions referring to how the subject feels at the time of administration of the questionnaire. Despite its subjective nature, the S.T.A.I. Y1 questionnaire is considered a reference tool in clinical practice, and it is often adopted for annotating the event-based temporary stress state [45]. The first session was meant to acquire baseline data, during which the subjects had to rest with their eyes open. The second one, instead, was purposely designed to stress the subjects. To this end, we took advantage of the well-known Stroop test, a well-established paradigm requiring participants to solve two incongruent stimuli quickly. During the Stroop test, a series of incongruent color/semantic-meaning words are presented to the participants who are asked to answer based on the displayed tint within a 2 s time window. Moreover, we included a counter showing the consecutive successes as a motivational stressor and turning back to zero after an incorrect answer. To manage the sessions, the stimuli presentation, and the storing of the subject’s performance in reacting to the incongruent words, we developed a computerized version of the test based on the PyQt6 application. All the subjects were instructed on the experimental timeline before the start of the sessions, and they were seated in front of the PC running the Stroop test application. Furthermore, to reduce the risk of artifacts or spurious EDA and ECG variations, each subject was asked to keep their non-dominant hand steady as much as possible and not to speak throughout the experiment. The WMP was used to acquire, stream, and real-time process the EDA, the ECG, and the EEG collected from a Shimmer 3GSR+ unit and the DSI-24 system. PPG and RESP were collected alongside the other signals but were not processed. More specifically, the EDA was acquired at 50 Hz from two electrodes placed on the proximal phalanges of the index and medium fingers of the non-dominant hand. The PPG was collected at the same sampling frequency by placing an optical probe on the annular finger of the non-dominant hand. The EEG signals from the DSI-24 system were acquired at a sampling rate of 300 Hz by placing 21 sensors at locations corresponding to the 10–20 International System. For the ECG and RESP acquisition, we used the integrated channels in the DSI-24 headset. Specifically, we mounted two electrodes on the subject’s chest and a stretch-sensitive sensor on the abdomen.

2.4.2. Computational Performance Analysis

The system was evaluated in terms of computational performances to test eventual lags in the transmission of the data leaving the SFU. Considering the possible applications spanning from physiological acquisitions in a controlled laboratory setup to real-time health monitoring, we estimated the computational load (expressed in seconds) in different working contexts:
  • Physiological data streaming and remote storing.
  • Physiological data streaming, remote storing, and real-time processing of peripheral (ANS) signals.
  • Physiological data streaming, remote storing, and real-time processing of CNS signals.
  • Physiological data streaming, remote storing and real-time processing of peripheral and CNS signals.
The computational load was calculated over a time window of 6 min in agreement with the temporal length of the experimental timeline. Since the computational operations (from the signals acquisitions to their processing) were executed in parallel and negligible delays were accumulated during the data streaming, the results of this analysis can also be extended to longer time windows.

2.4.3. Statistical Analysis

The features computed from each participant’s signal by the WMP during the experimental timeline were statistically analyzed to evaluate whether the platform was reliable in recording significant variations of the nervous system activity during the stressor task. In particular, the feature values calculated within the rest session and the Stroop session were averaged and statistically compared through a non-parametric sign-rank test. The same test was applied to verify whether the experiment also induced significant psychological changes compared to the S.T.A.I. Y1 scores before and after the experiment.

3. Results

3.1. Computational Performance Results

The computational performance analysis showed the WMP’s capability to fit the requirements of several experimental scenarios. The lag in the data transmission between the SFU and real-time database, depending on the write time of the middleware, was equal to 0.1 s. Moreover, thanks to the modular process-oriented architecture, we further evaluated the system by separately real-time processing ANS and CNS signals. Over the time window of 6 min, the WMP performed the signal acquisition and processing of ANS signals in 0.10 ± 0.01 s and CNS data in 0.69 ± 0.14 s. The data processing unit’s multi-process structure guaranteed the processing steps’ simultaneity. Hence, no difference in computational performances was observed when ANS signals were streamed and processed alongside CNS data.

3.2. Statistical Analysis Results

The statistical analysis showed significant differences in those features that are commonly correlated to the physiological state changes induced by the Stroop condition [32,46]. The significant results related to the HRV features are summarized in Figure 5. In particular, LF, HF, Poincarè SD1, Poincarè SD2, RMSSD, pNN50, meanRR, and stdRR showed significant differences between Rest and Stroop. Specifically, we observed a decrease in most of the features estimating the parasympathetic nervous system activity (i.e., HF power (p = 0.0026), Poincarè SD1 (p = 0.0015), Poincarè SD2 (p = 6.1 × 10 4 ), RMSSD (p = 0.0015), pNN50 (p = 0.0012), meanRR (p = 4.27 × 10 4 ), stdRR (p = 6.1 × 10 4 ) [9,10]), which can indicate a reduced downregulating vagal effect putting the body in an alert state. Additionally, a decrease in LF power (p = 0.0043) was observed. However, since LF is produced by both parasympathetic and sympathetic divisions, it is possible that its decrease was due to a simultaneous effect of the emotional stressors on both the ANS branches. Finally, we did not find any significant difference in the LF/HF ratio.
The statistical test on the EDA features showed a significant increase in the mean level value (meanSCL; p = 0.012), which reflects a more aroused condition during the Stroop test (Figure 6). We did not find any significant difference considering the features extracted from the event-related component of the signal. This was probably due to the nature of the stress stimulus, which was a continuous long-lasting stimulation more than discrete acute stimuli.
Concerning the EEG frequency power analysis, among the tested frequency bands (i.e., α , β , γ , θ , and δ ), we observed a significant increase in the γ waves (p = 0.012) during the Stroop test (Figure 7). These fast EEG oscillations reflect brain functions such as cognition and memory that are plausibly triggered by the stress-inducing task.
Overall, several features have shown a significant variation between the two experimental conditions, suggesting the expected significant decrease in parasympathetic activity and a simultaneous increase in the sympathetic and central cognitive functions, which can be traced back to a stressful condition [47,48,49].

4. Discussion

This work presents a novel WMP architecture that can be applied to different experimental setups in which multi-device and real-time characteristics are relevant. Moreover, on the one hand, it fits well with the growing interest in developing wearable devices and integrating into their software advanced signal-processing techniques. On the other hand, it fits into a context in which the IoT networks have been spreading further, enabling the establishment of new healthcare pervasive services [12] that aim to continuously collect clinically relevant data from subjects during their daily lives. Specifically, the WMP shows an IoT-like distributed architecture fully based on open-source solutions to interface multiple sensors and collect physiological data from them.
Contrary to other proposals in the literature and on the market, our platform combines four important properties in the same architecture: scalability, real-time processing, remote storage, and the use of only open-source solutions.
Scalability: The variability in protocols to access data and control the different wearable device technologies prevents the integration of heterogeneous systems into the same sensing network. The WMP addresses this issue through a modular and extensible architecture that does not limit the system to a specific device. Indeed, the platform can potentially interface with any wearable device whose manufacturer provides the SDK or the corresponding streaming protocol information. Additional wearable systems from different manufacturers can be integrated into the SFU by including a dedicated subprocess to handle the connection and communication without any change to the RDSPU. At the same time, the process-oriented structure of the SFU and data processing unit allows the WMP to connect a growing number of sensors without losses in computational performance. The possibility of connecting a wide range of ad-hoc heterogeneous sensors confers versatility to the whole platform that could be used in many application contexts requiring very different sensors, such as fitness tracking and continuous health monitoring. In this view, the WMP successfully managed different streaming protocols and technologies by interfacing two widespread research-grade wearable devices (i.e., the Shimmer3 GSR+ unit and the DSI-24 system).
Real-time processing: In many experimental applications (especially in ecological scenarios), a crucial aspect is the continuous monitoring of the subject’s psychophysiological state by continuously processing and extracting information from the signals. The WMP’s computational performances (including both data streaming performed by the SFU and storing and processing accomplished by the RDSPU) allowed the whole system to process the signals while they were collected and stored. The estimated computational loads and lags in the transmission of the information throughout the system were found to be in line with the temporal dynamics of the emotional phenomena of potential interest in many applicative contexts. Although the platform was tested on a simple stressful task, the implemented real-time processing functions could place the WMP in an active role in various scenarios. For instance, given the capability of detecting significant changes in the psychophysiological state, the WMP could be used to monitor workers’ stress or detect the insurgence of many psychological and emotional disorders. Moreover, the integrability of the adopted open-source solutions and the possibility of interfacing with external applications (e.g., Grafana) further extend the range of applications, including data visualization and constant access to potentially relevant clinical parameters.
Use of only open-source components: It is worth noting that the open-source software (i.e., software that can be freely used, changed, and shared in unmodified or modified form by anyone) implemented for each block of the system carries additional advantages in terms of flexibility, customizability, and cost-effectiveness. In particular, most parts of the WMP can be completely adapted to suit a specific application context without relying on manufacturers. In addition, the platform’s components are developed by third parties and continuously maintained by a large community. This means that, on the one hand, the risk of becoming obsolete is very low; on the other hand, the solution is always updated and secure.
Remote storage: The IoT architecture comprising the two remote storing systems of the data and features designed through InfluxDB and CouchDB facilitates the integration of the WMP with any psychophysiological model necessary for the application of the WMP. Indeed, cloud storage allows the collection of all the data in a single repository regardless of the location where the WMP will be used. Moreover, having all the data in a single repository allows the downstream model to be continuously updated/trained with the greatest amount of data, which is crucial for the model’s performance.

5. Conclusions

The study has presented a biosignal monitoring and processing platform entirely based on open-source solutions that offer the possibility of connecting multiple devices (commercial or not) maintaining high computational performance. The system integrates this solution into an IoT-like architecture capable of guaranteeing performance useful for real-time applications. Although previous studies have presented alternative platforms with some of these features, we have first merged them all into one solution that will be available online. Future works will focus on testing the WMP through different experimental settings requiring other device configurations and including ecologically valid scenarios.

Author Contributions

Conceptualization, A.B., R.G., E.P.S. and A.G.; methodology, A.B., R.G. and A.G.; software, A.B. and R.G.; validation, A.B. and A.G.; formal analysis, A.B. and A.G.; investigation, A.B.; resources, E.P.S. and A.G.; data curation; A.B. and R.G.; writing—original draft preparation, A.B. and A.G.; writing—review and editing, A.B., A.G. and E.P.S.; visualization, A.B.; supervision, E.P.S. and A.G.; project administration, E.P.S. and A.G.; funding acquisition, E.P.S. and A.G. All authors have read and agreed to the published version of the manuscript.

Funding

The research leading to these results has received partial funding from the Italian Ministry of Education and Research (MIUR) in the framework of the FoReLab project (Departments of Excellence) and from the European Union Horizon 2020 Program under grant agreement no. 824153 of the project “POTION-Promoting Social Interaction through Emotional Body Odours”.

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki, and approved by the Institutional Review Board (or Ethics Committee) of the bioethical committee of the University of Pisa (no. 14/2019).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

The data presented in this study are available on request from the corresponding author.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Ghiasi, S.; Patane, A.; Laurenti, L.; Gentili, C.; Scilingo, E.P.; Greco, A.; Kwiatkowska, M. Physiologically-informed gaussian processes for interpretable modelling of psycho-physiological states. IEEE J. Biomed. Health Inform. 2022, 26, 5793–5804. [Google Scholar] [CrossRef] [PubMed]
  2. Greco, A.; Valenza, G.; Bicchi, A.; Bianchi, M.; Scilingo, E.P. Assessment of muscle fatigue during isometric contraction using autonomic nervous system correlates. Biomed. Signal Process. Control 2019, 51, 42–49. [Google Scholar] [CrossRef]
  3. Aminosharieh Najafi, T.; Affanni, A.; Rinaldo, R.; Zontone, P. Driver attention assessment using physiological measures from EEG, ECG, and EDA signals. Sensors 2023, 23, 2039. [Google Scholar] [CrossRef] [PubMed]
  4. Can, Y.S.; Gokay, D.; Kılıç, D.R.; Ekiz, D.; Chalabianloo, N.; Ersoy, C. How laboratory experiments can be exploited for monitoring stress in the wild: A bridge between laboratory and daily life. Sensors 2020, 20, 838. [Google Scholar] [CrossRef] [Green Version]
  5. Smets, E.; De Raedt, W.; Van Hoof, C. Into the wild: The challenges of physiological stress detection in laboratory and ambulatory settings. IEEE J. Biomed. Health Inform. 2018, 23, 463–473. [Google Scholar] [CrossRef]
  6. Esposito, D.; Centracchio, J.; Andreozzi, E.; Gargiulo, G.D.; Naik, G.R.; Bifulco, P. Biosignal-Based Human–Machine Interfaces for Assistance and Rehabilitation: A Survey. Sensors 2021, 21, 6863. [Google Scholar] [CrossRef]
  7. Cogan, D.; Birjandtalab, J.; Nourani, M.; Harvey, J.; Nagaraddi, V. Multi-biosignal analysis for epileptic seizure monitoring. Int. J. Neural Syst. 2017, 27, 1650031. [Google Scholar] [CrossRef]
  8. Kim, S.W.; Lee, K.; Yeom, J.; Lee, T.H.; Kim, D.H.; Kim, J.J. Wearable multi-biosignal analysis integrated interface with direct sleep-stage classification. IEEE Access 2020, 8, 46131–46140. [Google Scholar] [CrossRef]
  9. Hosseini, S.A.; Khalilzadeh, M.A.; Changiz, S. Emotional stress recognition system for affective computing based on bio-signals. J. Biol. Syst. 2010, 18, 101–114. [Google Scholar] [CrossRef]
  10. Supratak, A.; Wu, C.; Dong, H.; Sun, K.; Guo, Y. Survey on feature extraction and applications of biosignals. In Machine Learning for Health Informatics; Springer: Berlin/Heidelberg, Germany, 2016; pp. 161–182. [Google Scholar]
  11. Lee, B.G.; Chung, W.Y. Driver alertness monitoring using fusion of facial features and bio-signals. IEEE Sens. J. 2012, 12, 2416–2422. [Google Scholar] [CrossRef]
  12. Tangtisanon, P. Real time Human Emotion Monitoring based on Bio-signals. In Proceedings of the 2018 18th International Conference on Control, Automation and Systems (ICCAS), Pyeongchang, Republic of Korea, 17–20 October 2018. [Google Scholar]
  13. Kim, K.H.; Bang, S.W.; Kim, S.R. Emotion recognition system using short-term monitoring of physiological signals. Med. Biol. Eng. Comput. 2004, 42, 419–427. [Google Scholar] [CrossRef] [PubMed]
  14. He, C.; Yao, Y.-J.; Ye, X.-S. An emotion recognition system based on physiological signals obtained by wearable sensors. In Wearable Sensors and Robots; Springer: Berlin/Heidelberg, Germany, 2017; pp. 15–25. [Google Scholar]
  15. Greco, A.; Lanata, A.; Citi, L.; Vanello, N.; Valenza, G.; Scilingo, E.P. Skin admittance measurement for emotion recognition: A study over frequency sweep. Electronics 2016, 5, 46. [Google Scholar] [CrossRef] [Green Version]
  16. Dias, R.; Da Silva, J.M. A flexible wearable sensor network for bio-signals and human activity monitoring. In Proceedings of the 2014 11th International Conference on Wearable and Implantable Body Sensor Networks Workshops, Zurich, Switzerland, 16–19 June 2014. [Google Scholar]
  17. Alizadeh, M.; Shaker, G.; De Almeida, J.C.M.; Morita, P.P.; Safavi-Naeini, S. Remote monitoring of human vital signs using mm-wave FMCW radar. IEEE Access 2019, 7, 54958–54968. [Google Scholar] [CrossRef]
  18. Murali, S.; Rincon, F.; Atienza, D. A wearable device for physical and emotional health monitoring. In Proceedings of the 2015 Computing in Cardiology Conference (CinC), Nice, France, 6–9 September 2015. [Google Scholar]
  19. Kadhim, K.T.; Alsahlany, A.M.; Wadi, S.M.; Kadhum, H.T. An overview of patient’s health status monitoring system based on Internet of Things (IoT). Wirel. Pers. Commun. 2020, 114, 2235–2262. [Google Scholar] [CrossRef]
  20. ActiveOS. Available online: https://activeos.com/ (accessed on 6 February 2023).
  21. Chételat, O.; Oster, J.; Grossenbacher, O.; Hutter, A.; Krauss, J.; Giannakis, A. A highly integrated wearable multi-parameter monitoring system for athletes. In Proceedings of the 15th Nordic-Baltic Conference on Biomedical Engineering and Medical Physics (NBC 2011), Aalborg, Denmark, 14–17 June 2011. [Google Scholar]
  22. Coates, J.; Chipperfield, A.; Clough, G. Wearable multimodal skin sensing for the diabetic foot. Electronics 2016, 5, 45. [Google Scholar] [CrossRef] [Green Version]
  23. McGregor, C. A cloud computing framework for real-time rural and remote service of critical care. In Proceedings of the 2011 24th International Symposium on Computer-Based Medical Systems (CBMS), Bristol, UK, 27–30 June 2011. [Google Scholar]
  24. Oliver, N.; Flores-Mangas, F. HealthGear: A real-time wearable system for monitoring and analyzing physiological signals. In Proceedings of the International Workshop on Wearable and Implantable Body Sensor Networks (BSN’06), Berkeley, CA, USA, 3–5 June 2009. [Google Scholar]
  25. Posthuma, L.; Downey, C.; Visscher, M.; Ghazali, D.; Joshi, M.; Ashrafian, H.; Khan, S.; Darzi, A.; Goldstone, J.; Preckel, B. Remote wireless vital signs monitoring on the ward for early detection of deteriorating patients: A case series. Int. J. Nurs. Stud. 2020, 104, 103515. [Google Scholar] [CrossRef]
  26. Nabian, M.; Yin, Y.; Wormwood, J.; Quigley, K.S.; Barrett, L.F.; Ostadabbas, S. An open-source feature extraction tool for the analysis of peripheral physiological data. IEEE J. Transl. Eng. Health Med. 2018, 6, 1–11. [Google Scholar] [CrossRef]
  27. Teller, A.; Stivoric, J. The bodymedia platform: Continuous body intelligence. In Proceedings of the the 1st ACM Workshop on Continuous Archival and Retrieval of Personal Experiences, New York, NY, USA, 15 October 2004; pp. 114–115. [Google Scholar]
  28. Bhat, G.; Deb, R.; Ogras, U.Y. OpenHealth: Open-source platform for wearable health monitoring. IEEE Des. Test 2019, 36, 27–34. [Google Scholar] [CrossRef] [Green Version]
  29. Bahmani, A.; Alavi, A.; Buergel, T.; Upadhyayula, S.; Wang, Q.; Ananthakrishnan, S.K.; Alavi, A.; Celis, D.; Gillespie, D.; Young, G.; et al. A scalable, secure, and interoperable platform for deep data-driven health management. Nat. Commun. 2021, 12, 5757. [Google Scholar] [CrossRef]
  30. Ranjan, Y.; Rashid, Z.; Stewart, C.; Conde, P.; Begale, M.; Verbeeck, D.; Boettcher, S.; Dobson, R.; Folarin, A.; Consortium, R.C.; et al. RADAR-base: Open source mobile health platform for collecting, monitoring, and analyzing data using sensors, wearables, and mobile devices. JMIR mHealth uHealth 2019, 7, e11734. [Google Scholar] [CrossRef] [Green Version]
  31. Stroop, J.R. Studies of interference in serial verbal reactions. J. Exp. Psychol. 1935, 18, 643. [Google Scholar] [CrossRef]
  32. Gioia, F.; Greco, A.; Callara, A.L.; Scilingo, E.P. Towards a Contactless Stress Classification Using Thermal Imaging. Sensors 2022, 22, 976. [Google Scholar] [CrossRef] [PubMed]
  33. Atmoko, R.; Riantini, R.; Hasin, M. IoT real time data acquisition using MQTT protocol. In Proceedings of the Journal of Physics: Conference Series, Madrid, Spain, 30 August–4 September 2015. [Google Scholar]
  34. Spielberger, C.; Gorsuch, R.; Lushene, R.; Vagg, P.; Jacobs, G. Manual for the State-Trait Anxiety Inventory (Form Y1—Y2); Consulting Psychologists Press: Palo Alto, CA, USA, 1983; Volume IV. [Google Scholar]
  35. Richter, P.; Werner, J.; Heerlein, A.; Kraus, A.; Sauer, H. On the validity of the Beck Depression Inventory. Psychopathology 1998, 31, 160–168. [Google Scholar] [CrossRef] [PubMed]
  36. Heimberg, R.G.; Horner, K.; Juster, H.; Safren, S.; Brown, E.; Schneier, F.; Liebowitz, M. Psychometric properties of the Liebowitz social anxiety scale. Psychol. Med. 1999, 29, 199–212. [Google Scholar] [CrossRef] [PubMed]
  37. Willman, J.M. Beginning PyQt; Springer: Berlin/Heidelberg, Germany, 2020. [Google Scholar]
  38. Greco, A.; Valenza, G.; Lázaro, J.; Garzón-Rey, J.M.; Aguiló, J.; De-la Camara, C.; Bailón, R.; Scilingo, E.P. Acute stress state classification based on electrodermal activity modeling. IEEE Trans. Affect. Comput. 2021, 14, 788–799. [Google Scholar] [CrossRef]
  39. Dawson, M.E.; Schell, A.M.; Filion, D.L. The Electrodermal System; Cambridge University Press: Cambridge, UK, 2017. [Google Scholar]
  40. Setz, C.; Arnrich, B.; Schumm, J.; La Marca, R.; Tröster, G.; Ehlert, U. Discriminating stress from cognitive load using a wearable EDA device. IEEE Trans. Inf. Technol. Biomed. 2009, 14, 410–417. [Google Scholar] [CrossRef]
  41. van Ravenswaaij-Arts, C.M.; Kollee, L.A.; Hopman, J.C.; Stoelinga, G.B.; van Geijn, H.P. Heart rate variability. Ann. Intern. Med. 1993, 118, 436–447. [Google Scholar] [CrossRef]
  42. Davidson, R.J.; Ekman, P.; Saron, C.D.; Senulis, J.A.; Friesen, W.V. Approach-withdrawal and cerebral asymmetry: Emotional expression and brain physiology: I. J. Personal. Soc. Psychol. 1990, 58, 330. [Google Scholar] [CrossRef]
  43. Goodman, R.N.; Rietschel, J.C.; Lo, L.C.; Costanzo, M.E.; Hatfield, B.D. Stress, emotion regulation and cognitive performance: The predictive contributions of trait and state relative frontal EEG alpha asymmetry. Int. J. Psychophysiol. 2013, 87, 115–123. [Google Scholar] [CrossRef]
  44. Chen, H.K.; Hu, Y.F.; Lin, S.F. Methodological considerations in calculating heart rate variability based on wearable device heart rate samples. Comput. Biol. Med. 2018, 102, 396–401. [Google Scholar] [CrossRef]
  45. Xu, Q.; Nwe, T.L.; Guan, C. Cluster-based analysis for personalized stress evaluation using physiological signals. IEEE J. Biomed. Health Inform. 2014, 19, 275–281. [Google Scholar] [CrossRef]
  46. Gioia, F.; Callara, A.L.; Bruderer, T.; Ripszam, M.; Di Francesco, F.; Scilingo, E.P.; Greco, A. Potential physiological stress biomarkers in human sweat. In Proceedings of the 2022 IEEE International Symposium on Medical Measurements and Applications (MeMeA), Messina, Italy, 22–24 June 2022. [Google Scholar]
  47. Lee, J.; Shields, R.K. Sympathetic Vagal Balance and Cognitive Performance in Young Adults during the NIH Cognitive Test. J. Funct. Morphol. Kinesiol. 2022, 7, 59. [Google Scholar] [CrossRef] [PubMed]
  48. Mestanik, M.; Mestanikova, A.; Visnovcova, Z.; Calkovska, A.; Tonhajzerova, I. Cardiovascular sympathetic arousal in response to different mental stressors. Physiol. Res. 2015, 64, S585. [Google Scholar] [CrossRef] [PubMed]
  49. Mendl, M. Performing under pressure: Stress and cognitive function. Appl. Anim. Behav. Sci. 1999, 65, 221–244. [Google Scholar] [CrossRef]
Figure 1. Summary of the WMP architecture. On the left side: The sensor fusion unit (SFU). On the right side: The remote data storage and processing unit (RDSPU).
Figure 1. Summary of the WMP architecture. On the left side: The sensor fusion unit (SFU). On the right side: The remote data storage and processing unit (RDSPU).
Electronics 12 01498 g001
Figure 2. The sensor fusion unit (SFU). On the left: The sensor layer is composed of the wearable devices to be connected. SP1, SP2, and SP3 represent the dedicated acquiring subprocesses for the streaming devices. SP4 is the subprocess in charge of data gathering and streaming over the MQTT protocol. On the right: The PyQt6 user interface developed to handle the data acquisition from the Shimmer3 GSR+ unit and DSI-24 system alongside metadata and subjects’ personal information.
Figure 2. The sensor fusion unit (SFU). On the left: The sensor layer is composed of the wearable devices to be connected. SP1, SP2, and SP3 represent the dedicated acquiring subprocesses for the streaming devices. SP4 is the subprocess in charge of data gathering and streaming over the MQTT protocol. On the right: The PyQt6 user interface developed to handle the data acquisition from the Shimmer3 GSR+ unit and DSI-24 system alongside metadata and subjects’ personal information.
Electronics 12 01498 g002
Figure 3. (a) The DSI-24 system. (b) The Shimmer3 GSR+ unit.
Figure 3. (a) The DSI-24 system. (b) The Shimmer3 GSR+ unit.
Electronics 12 01498 g003
Figure 4. The experimental timeline.
Figure 4. The experimental timeline.
Electronics 12 01498 g004
Figure 5. Results from the Wilcoxon sign-rank test considering the HRV-derived features. Statistically significant comparisons resulting in p < 0.01 are highlighted with ** and those resulting in p < 0.001 are highlighted with ***.
Figure 5. Results from the Wilcoxon sign-rank test considering the HRV-derived features. Statistically significant comparisons resulting in p < 0.01 are highlighted with ** and those resulting in p < 0.001 are highlighted with ***.
Electronics 12 01498 g005
Figure 6. Results from the Wilcoxon sign-rank test considering the EDA-derived features. Statistically significant comparisons resulting in p < 0.05 are highlighted with *.
Figure 6. Results from the Wilcoxon sign-rank test considering the EDA-derived features. Statistically significant comparisons resulting in p < 0.05 are highlighted with *.
Electronics 12 01498 g006
Figure 7. Results from the Wilcoxon sign-rank test considering the EEG-derived features. Statistically significant comparisons resulting in p < 0.05 are highlighted with *.
Figure 7. Results from the Wilcoxon sign-rank test considering the EEG-derived features. Statistically significant comparisons resulting in p < 0.05 are highlighted with *.
Electronics 12 01498 g007
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Baldini, A.; Garofalo, R.; Scilingo, E.P.; Greco, A. A Real-Time, Open-Source, IoT-like, Wearable Monitoring Platform. Electronics 2023, 12, 1498. https://doi.org/10.3390/electronics12061498

AMA Style

Baldini A, Garofalo R, Scilingo EP, Greco A. A Real-Time, Open-Source, IoT-like, Wearable Monitoring Platform. Electronics. 2023; 12(6):1498. https://doi.org/10.3390/electronics12061498

Chicago/Turabian Style

Baldini, Andrea, Roberto Garofalo, Enzo Pasquale Scilingo, and Alberto Greco. 2023. "A Real-Time, Open-Source, IoT-like, Wearable Monitoring Platform" Electronics 12, no. 6: 1498. https://doi.org/10.3390/electronics12061498

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop