Next Article in Journal
Deep Learning-Based Human Action Recognition with Key-Frames Sampling Using Ranking Methods
Next Article in Special Issue
Implementing an Ensemble Learning Model with Feature Selection to Predict Mortality among Patients Who Underwent Three-Vessel Percutaneous Coronary Intervention
Previous Article in Journal
Exploration and Research on Key Technologies for Improving the Response Speed of Servo-Hydraulic Cylinders
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Applied Sciences
Volume 12
Issue 9
10.3390/app12094164
applsci-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Analysis of Heart Rate Variability and Game Performance in Normal and Cognitively Impaired Elderly Subjects Using Serious Games

by
Chun-Ju Hou
1,
Yen-Ting Chen
1,*,
Mycel A. Capilayan
1,
Min-Wei Huang
2 and
Ji-Jer Huang
1
1
Department of Electrical Engineering, Southern Taiwan University of Science and Technology, No.1, Nan-Tai Street, Yungkang District, Tainan City 710301, Taiwan
2
Department of Psychiatry, Chiayi Branch, Taichung Veterans General Hospital, No. 600, Shixian Rd., West District, Chiayi City 60090, Taiwan
*
Author to whom correspondence should be addressed.
Appl. Sci. 2022, 12(9), 4164; https://doi.org/10.3390/app12094164
Submission received: 24 February 2022 / Revised: 16 April 2022 / Accepted: 18 April 2022 / Published: 20 April 2022
(This article belongs to the Special Issue The Applications of Machine Learning in Biomedical Science)
Browse Figures
Versions Notes

Abstract

:
Cognitive decline is one of the primary concerns in the elderly population. Serious games have been used for different purposes related to elderly care, such as physical therapy, cognitive training and mood management. There has been scientific evidence regarding the relationship between cognition and the autonomic nervous system (ANS) through heart rate variability (HRV). This paper explores the changes in the ANS among elderly people of normal and impaired cognition through measured HRV. Forty-eight subjects were classified into two groups: normal cognition (NC) (n = 24) and mild cognitive impairment (MCI) (n = 24). The subjects went through the following experiment flow: rest for 3 min (Rest 1), play a cognitive aptitude game (Game 1), rest for another 3 min (Rest 2), then play two reaction-time games (Game 2&3). Ten HRV features were extracted from measured electrocardiography (ECG) signals. Based on statistical analysis, there was no significant difference on the HRV between the two groups, but the experiment sessions do have a significant effect. There was no significant interaction between sessions and cognitive status. This implies that the HRV between the two groups have no significant difference, and they will experience similar changes in their HRV regardless of their cognitive status. Based on the game performance, there was a significant difference between the two groups of elderly people. Tree-based pipeline optimization tool (TPOT) was used for generating a machine learning pipeline for classification. Classification accuracy of 68.75% was achieved using HRV features, but higher accuracies of 83.33% and 81.20% were achieved using game performance or both HRV and game performance features, respectively. These results show that HRV has the potential to be used for detection of mild cognition impairment, but game performance can yield better accuracy. Thus, serious games have the potential to be used for assessing cognitive decline among the elderly.

1. Introduction

The worldwide population’s portion of older adults has been gradually increasing. By 2050, the World Health Organization (WHO) estimates that the world’s population of people aged 60 and above will reach 2 billion [1]. Some of the reasons for this phenomenon are the steady increase in the worldwide life expectancy, along with decreasing fertility rates. Taiwan is currently an aged society, with the percentage of its elderly people being more than 14% of its population. With this trend, this percentage may rise to more than 20% by 2026, making Taiwan a super-aged society [2].
Cognitive decline is one of the key problems among the elderly. The brain structure and function can change as people become older. There are numerous factors such as alcohol, head trauma or injury, excessive stress, and the development of degenerative diseases such as Alzheimer’s disease (AD) that over the years can cause cumulative damage to the brain leading to cognitive impairment. The frequency of Alzheimer’s disease increases with age in older persons, and it is also the primary cause of cognitive deterioration [3]. According to a study in 2020, the global prevalence of cognitive impairment in older adults could range from 5.1% to as high as 41% [4]. While cognitive decline cannot be completely reversed, it can be delayed or even avoided. This can be accomplished by therapies such as exercise, cognitive training, nutritional counseling, and risk factor monitoring [5]. One of the ways to implement cognitive training is through the use of serious games. Interventions can benefit from early detection, but the onset of cognitive impairment might be very subtle and difficult to detect. The progression is slow and close relatives and friends may notice it more than the sufferers themselves [3].
Playing game requires cognitive abilities, depending on the game objectives. Some games may require problem solving, decision-making or quick reflexes, while some may require memory and language skills. Games provide a wide range of stimuli capable of eliciting cognitive activity and emotion in the player, depending on the game [6]. Due to this, they could be used as an alternative for evaluating a person’s cognitive state. Common testing methods such as Mini-Mental State Examination (MMSE) and the Montreal Cognitive Assessment (MoCA) are great for detecting cognitive impairment, but the results can also be affected by the person’s educational background [7].
As with the case of video games, different game genres and gameplay can affect the autonomic nervous system (ANS) activity, and conversely, the heart rate variability (HRV) [8]. There is a rapid growth in the field of serious games in the industry as well as in academic research. Serious games are games created for a purpose other than pure entertainment [9]. Serious games focused on health were used to help with health monitoring [10,11,12], detection and management of health conditions [13,14,15,16], and therapy and rehabilitation [17,18,19]. Serious games designed for the elderly have been developed over the years with a variety of goals in mind, ranging from physical therapy, balance training, and mood or stress management [20,21,22]. While these games had a positive effect on the elderly, no physiological measures were considered. Since physiological measurements are difficult to influence consciously, they have been proved to be more objective [23]. By recording and analyzing these physiological signals, the games and possibly other applications designed for the elderly can be designed better towards their needs.
The relationship of cognition and HRV features has been investigated in several studies, and the results indicate the influence of the ANS in cognition. HRV is generally defined as the variation on the time interval between a person’s heartbeats. The R-to-R (RR) intervals are calculated by acquiring the time between each R wave in the QRS complex of the ECG signal. Normalized RR (NN) intervals are also calculated in a similar method, but artifacts and noise from the signal were removed. Most HRV features are then derived from RR or NN intervals [24]. There were some HRV features that have been shown to have a correlation to a person’s cognitive abilities. High resting HRV values correlate to better performance in different cognitive domains. Some of the HRV features to note are high-frequency (HF) power, low-frequency (LF) power, standard deviation of NN intervals (SDNN) and root mean square of successive RR interval differences (RMSSD) [25]. From these results, autonomic markers such as HRV features, can be used as biomarkers for cognitive impairment [25,26]. However, not all studies observed a relationship. Some cognitive domains and tasks implemented in these studies found no correlation between the subjects’ cognitive performance and their HRV.
In this study, we wanted to investigate if there are significant difference in the ANS activity between cognitively normal and cognitively impaired elderly subjects as they play serious games through their measured HRV. These serious games were designed for the elderly, one game for testing the cognitive aptitude and two games testing their processing speed and reaction time [27,28]. The purpose of investigating the HRV response is to know how these serious games affect elderly people of different cognitive status. In addition to this, we also compared the two groups for their cognitive function through their game performance. We also explored the use of HRV and game performance features measured from elderly subjects and its application for automated detection for cognitive impairment using machine learning (ML).
The subject criteria, experimental design, physiological measurement, signal preprocessing, feature extraction, statistical analysis, feature selection and classification are shown in detail in Section 2. The results from the experiment and data analysis are presented in Section 3. The discussion and interpretation of these results are in Section 4. Finally, conclusions and future work are presented in Section 5.

2. Materials and Methods

This section highlights the materials and methods used to achieve the aim and purpose of this study. It contains the details for subject criteria, experimental design, physiological measurement, signal processing, feature extraction, statistical analysis and classification.

2.1. Subjects

The subjects were selected according to certain criteria. They must be aged 50 years or older. They should be able to understand the purpose, process, risks, rights and interests of the research, and sign the consent form. MoCA was used to evaluate the subjects for their cognitive abilities. It was shown to have better detection of cognitive impairment, especially mild cognitive impairment (MCI) [29]. The cut-off score used for this study is 23/24, as the age and education of the subjects have been taken to consideration [30,31]. Their scores were also evaluated and verified by hospital doctors and psychologists specializing in elderly care.
People with neuropsychiatric diseases such as abuse of alcohol or other substances, Parkinson’s disease, epilepsy and similar matters were excluded from participating. People with substance abuse problems were excluded from participating this experiment as alcohol and some drugs can influence the heart rate variability [32]. People with psychiatric disorders, such as anxiety, panic attacks, posttraumatic stress disorder, anorexia, borderline personality disorder, and depression have been found to have lower HRV features [33] and are also excluded. Parkinson’s disease and epilepsy may affect the subject’s hand and walking function, such as tremor, stiffness and slow walking. Severe vision and hearing loss, or any major medical illness (cancer, uremia, anemia or thyroid dysfunction) or other obstacles that can hinder the person’s ability to cooperate with the researchers can be a criterion for subject exclusion.
Subjects can withdraw from the experiment at any point they want, even during the experimental process. If they are unable to complete a trial, including the clinical scale evaluation and board game interactive system, then they will also be excluded from the experiment. This experiment has been approved by the Ethics Committee Approval (Institutional Review Board (or Ethics Committee) of Taichung Veterans General Hospital Taiwan (protocol code SF18297A-1; 16 January 2019)). A total of 48 subjects were selected, in which 24 subjects have normal cognition while the other 24 subjects have mild cognitive impairment.

2.2. Experimental Design

The experiment was conducted in the Chiayi Branch of the Taichung Veterans General Hospital. The experiment room consisted of two testing areas for the subjects to play games. Since all of the subjects lived in Taiwan for the majority of their lives, the language used to instruct and play the game was in Traditional Chinese. The games used in this study are serious games, since these games were designed with the main purpose of cognitive assessment training for older adults, and not just for entertainment.
The flow of the experiment is shown in Figure 1. The subjects rest outside of the testing room for 3 min. During this period, the subjects were also asked to sit, relax and close their eyes. This is shorter than the recommended 5 min, but based on [34], the minimum required recording time for the HRV features used in this study is only 2 min. So, 3 min is more than sufficient, and reduces the experiment time for the subjects. After this initial resting period, the subjects enter the testing room and walk towards the area where Game 1 was set up. Instructions regarding on how to play the game was then given to the subjects. Game 1 is a cognitive aptitude game based on nostalgia theory and themes related to Taiwanese culture including food, clothing, transportation, housing and entertainment [27]. The gameplay time ranged between 8 to 10 min. The game’s user interface was projected to a wall, and run using a personal computer. These questions involve memory, attention, executive function and language. Speed is not a priority. In this game, test subjects have to identify everyday objects, places and events. They have to select the appropriate clothes, make arithmetic calculations, and decide on which directions to take depending on the described scenarios. They were also asked about their favorite festivals and they have to select the appropriate scenes or items related to that event. A sample of the user interface is shown in Figure 2, with the categories labelled with their respective English translation.
After completing Game 1, they were then required to take a rest for 3 min. This is the second resting period. Then, they proceeded to another area to play two reaction time games in succession without a resting period in between. These games were played using a personal desktop computer and a specialized button board, but projected to a large screen for the elderly subjects to have a better view of the games. Game 2 and 3 are called Whack-a-Mole and Hit-the-Ball, respectively. Whack-a-Mole was played first, then Hit-the-Ball. These two games were used to test the subjects’ reaction time and processing speed [28]. For Whack-a-Mole, the subjects have to hit the correct hole where the mole pops up, as shown in Figure 3. There are three levels of speed: slow (level 1), medium (level 2) and fast (level 3). For Hit-the-Ball, the subject has to choose the correct fairway where a football pass through. One ball, either a football or spiked ball, will roll on either of the three fairways. The spiked balls should be avoided, in which the subject should not press any of the buttons on the board. They also have to press the correct button when the football reaches the target area, as shown in Figure 4. As with the Whack-a-Mole game, it has three levels of speed. On the surface, these two games are similar, and by being reaction-time games, they test how fast the subjects can react to the games’ stimuli. However, Hit-the-Ball has an added layer of difficulty in which they have to identify the type of ball moving across the field and act accordingly. The total gameplay time for playing these two games in sequence ranges from 4 to 5 min.
After playing, the experiment ends and they were asked to go back to the resting area outside the testing room. ECG was recorded during resting periods and while the subjects were playing the games. All games used in this study were developed in-house and are free.

2.3. Physiological Measurement and Signal Preprocessing

The ECG signals were measured using a wearable device developed in-house, as described in [35]. Ag/AgCl patches were placed on the below the subject’s left and right collar bones and on the lower left rib in a lead II configuration. The ECG signals were sampled at a frequency of 500 Hz. The measured signals were divided into different time sections. These time sections are Rest 1, Game 1, Rest 2 and Game 2&3. The time for Game 2 and Game 3 were combined as they were played in succession and there was no resting period in between.
To ensure the quality of the ECG signal, it underwent signal preprocessing before feature extraction. The raw ECG signal was detrended to remove the linear trend, then filtered using a Butterworth band-pass filter with cutoff frequencies from 5 to 40 Hz. The filtering was carried out to remove the baseline drift, white noise and motion noise.
Artifact removal is vital in the calculation of HRV, as a minor artifact can severely skew the extracted HRV features. Artifacts such as ectopic beats, arrhythmic events, missing data and other noise can affect the calculation of the power spectral density (PSD) of HRV. The software ARTiiFact [36] was used to remove these artifacts. It first extracts the interbeat interval (IBI) or the R-R interval from the ECG. After the IBI extraction, artifact detection was implemented using either or both of these two algorithms: the median absolute deviation detection and an algorithm proposed by Berntson et al. [37]. It also allows for manual artifact detection, in which the user can visually check the ECG signal. To ensure the accuracy of the IBI calculations, the automated methods were used first, then verify the results from these algorithms through manual detection. The detected artifacts were then processed through interpolation. Two methods were used, namely cubic spline and linear interpolation. Both methods have their own pros and cons. Linear interpolation is simple to use, however it is ineffective when dealing with nonlinear data. The cubic spline interpolation can produce smooth interpolants, but it would not work well with sample points that are near together while having large value variations.

2.4. Feature Extraction

After the artifact removal, the HRV was calculated using Heart Rate Variability Analysis Software (HRVAS) [38]. It is an open-source program that calculates HRV features. Ten HRV features were used: 5 time-domain features, 1 nonlinear feature, and 4 frequency-domain features from PSD.
Time-domain features are derived from the detection of the QRS complex in the ECG signal. Each R wave was detected and the R-R intervals are obtained along with the instantaneous heart rate. From the R-R intervals, the mean R-R interval (RRI mean), heart rate (HR), standard deviation of NN intervals (SDNN), root mean square of successive differences of NN intervals (RMSSD) and percentage of differences between adjacent RR intervals that are greater than 50 ms (pNN50) can be derived. To calculate for SDNN, normal beats were used. RMSSD and pNN50 strongly correlate to parasympathetic activity [34].
Sample entropy (SampEn) is a nonlinear measure often used to measure complexity in any time series. In HRV analysis, SampEn is presented to measure the rate of entropy in short-term NN intervals [39]. High SampEn readings might indicate a high degree of complexity and little predictability. Lower values might imply a lack of complexity and a high degree of regularity. [34].
Frequency-domain features are derived from the PSD. There are numerous methods in calculating the PSD. The PSD was calculated using a parametric approach called autoregressive (AR) power spectral analysis. Even using fewer samples, parametric approaches offer smoother spectral components and more accurate PSD. Low frequency (LF) and high frequency (HF) are the two major spectral components considered in this investigation (HF). The LF band, which spans 0.04 to 0.15 Hz, is more closely linked to sympathetic activity. The HF band, on the other hand, is related with parasympathetic activity and runs from 0.15 Hz to 0.4 Hz [40].

2.5. Statistical Analysis

Statistical analyses were performed using SPSS 25 [41] and R Software 3.6.1 [42]. The age, years of education, and MoCA scores were compared between NC and CI groups using Mann–Whitney U test as these factors were not normally distributed. The gender was compared between the groups using Chi-square test. The ECG features were tested for normality using the Shapiro–Wilk test. Robust Mixed-model ANOVA was used to statistically test the effects of group and session, since the majority of the features were nonparametric data. The variable “group” refers to the two main groups of elderly people in this study, which are the NC and MCI groups. The variable “session” refers to the different sessions in the experiment, consisting of the rest and game sessions, namely: Rest 1, Game 1, Rest 2, and Games 2&3. This robust statistical method was implemented using the R package WRS2 [43]. Statistical tests of the main effects were followed by post-hoc comparisons between the different sessions. The significance values were then adjusted using Bonferroni correction. To compare the game performance of the two elderly groups, Mann-Whitney U test was used as the data is nonparametric and the groups are independent from each other. The significance level used for all statistical analyses was 0.05.

2.6. Classification

Three sets of data were used for machine learning. The first set only used HRV features, then the second set only used game performance features, and the third set used both HRV and game performance features. Leave-one-out (LOO) procedure was used to split the data to evaluate the performance of the classifier. The sizes of each dataset are presented in Table 1.
The typical machine learning pipeline is shown in Figure 5. It involves several processes such as data retrieval, data preparation, modeling, model evaluation and tuning, and deployment and monitoring. However, the process of making a pipeline is very tedious and time-consuming. So automated machine learning (AutoML) was used to reduce the time it takes to make a pipeline while maintain model quality [44].
The machine learning pipeline for each dataset was generated using an AutoML library called Tree-based Pipeline Optimization Tool (TPOT). It uses a tree-based structure to construct a pipeline for predictive modeling, including data preparation, modeling algorithms and model hyperparameters [45,46,47]. The performance of the generated pipeline was measured using accuracy, sensitivity and specificity. The purpose of using TPOT was to generate the best machine learning pipeline that will suit the data, thus acquiring the optimum performance despite using different datasets.

3. Results

This section presents the main results of the study, including the demography and characteristics of the subjects to observe the factors that can affect the measured HRV, statistical analysis results of the HRV and game performances features, and classification results using machine learning.

3.1. Demography and Characteristics of Subjects

The subjects in this study were composed of 24 cognitively normal subjects and 24 subjects with mild cognitive impairment. The descriptive statistics are listed in Table 2. No significant difference between the two groups was observed in terms of the subjects’ gender. While the females have a larger proportion in both groups, the gender-related variation in HRV features decreases once the age reaches over the age of 55. Since the subjects aged above 60 years old, the gender-related differences that can affect the results of this study is also diminished [48]. The group with normal cognition (NC) showed to be significantly younger than the group with MCI. HRV is also affected by age, with older adults having lower measured HRV. The NC group have significantly longer years of education compared to the MCI group. The MCI group have significantly lower MoCA scores compared to the Normal group.
Since the two groups have significant differences on the age and years of education, the MoCA cutoff scores used were adjusted based on [31]. The cutoff score used is 23/24, having both high specificity and sensitivity. The subjects were also further assessed by geriatric doctors and psychologists. The extracted ECG features are shown in Table 3, with the data represented through median and interquartile range values since most of them are nonparametric data based on the normality test.

3.2. Statistical Analysis Results

The data obtained from the experiment were statistically analyzed using nonparametric statistical methods since most of the data were not normally distributed. Statistical analysis was used to investigate the relationship between the variables group, session and the ECG features. The scores and reaction times of the subjects were also analyzed to determine if there were significant differences between the two groups.

3.2.1. Measured HRV Features

The influence of group and session on the ECG features were examined to understand the effects of cognitive impairment and the experiment on the ANS activity of the subjects. To investigate these effects as well as the interaction between factor, Robust Mixed-model ANOVA. As mentioned, robust methods were used since most of the ECG features violated the normality assumption. The results of this test are shown in Table 4. For some of the ECG features (HR, RRI, RMSSD, HF, and LF/HF), there was a significant main effect on the session. There was no significant effect for the group factor, and the interaction between group and session was also not significant. Since there are no significant interactions, the main effects of the two variables, group and session, their main effects were interpreted separately.
Since the sessions had a significant main effect on some of the ECG features, it was followed up with pairwise comparisons. The results of the pairwise tests using Wilcoxon signed-rank test are shown in Table 5. Here, the HRV measured from the cognitive aptitude game was significantly different from Rest 1, Rest 2 and Games 2&3. Subjects have higher heart rates and LF/HF ratios and lower RRI, RMSSD, and HF, indicating a sympathetic activation during this session. The lack of significant difference between Rest 1 and Rest 2 indicates that HRV features returned to baseline.

3.2.2. Game Performance

The scores of the subjects were recorded and tallied after each game session. For the first game, the total score is 25 points, with each category having different number of items. There is one point for each of the correct answer, and no points were awarded for wrong answers. For the second and third games, their scores are based on the hit ratio and reaction time. Table 6 shows the scores, hit ratios and reaction times of the subjects, categorized by their cognitive status. It also shows the statistical analysis for the two groups based on their game performance.
From the statistical results, there was a significant difference between the two groups in terms of their scores from Game 1, hit ratios and reaction times from Games 2 and 3. The group with MCI have significantly lower scores and reaction times compared to the cognitively normal group. These results are similar to a previous study [28], making the serious games a possible evaluation for cognitive impairment among elderly people.

3.3. Classification Results

There were three sets of features used for classification of NC and MCI subjects. The first set only used HRV features, the second only used game performance features, and the third set used both HRV and game performance features. For all sets, TPOT was used for generating the machine learning pipeline, and LOO was implemented for cross-validation of the generated pipeline.

3.3.1. HRV Features Only

There was a total of 40 features for this set. This is due to the measured HRV for each session (10 HRV features × 4 sessions). TPOT constructed a pipeline consisting of Recursive Feature Elimination (RFE), Binarizer and Decision Tree Classifier. RFE is a feature selection method eliminates features by ranking them by importance [49,50]. The binarizer transforms the data to 0 or 1 according to a threshold. The decision tree classifier is a supervised learning algorithm that is used for classification and regression. As with its name, it splits the data into branches until it reaches a threshold value. A decision tree is composed of root nodes, children nodes, and leaf nodes. Using a decision tree classifier has the following advantages: (1) the data does not need to be normalized or scaled, (2) fast and simple, compared to other classifiers, and (3) can handle both categorical and continuous data [51].
After using LOO cross-validation, an accuracy of 68.75% was achieved. It was also able to achieve a sensitivity of 66.67% and a specificity of 70.83% for MCI. The confusion matrix, with recall and precision rates, is shown in Table 7.

3.3.2. Game Performance Features Only

This dataset consists of 13 game performance features. This includes the response times and hit ratios for the three levels for the Whack-a-Mole and Hit-the-Ball games, and the game score from the cognitive aptitude game.
TPOT generated a pipeline consisting of RBFSampler and Decision Tree Classifier. The RBFSampler does not directly use Radial Basis Functions (RBF), but uses a variation of Random Kitchen Sinks, and is able to approximate RBFs [52]. As stated in [53] it approximates an RBF kernel by using the Monte Carlo approximation of the kernel’s Fourier transform. As with the previous set, the Decision Tree Classifier was also used for classification. After cross-validation, the achieved accuracy was 83.33%, with a sensitivity of 79.17% and a specificity of 87.50% as shown in Table 8.

3.3.3. Combination of HRV and Game Performance

By combining the HRV and game performance features, the total of features sums up to 53 features. TPOT chose the Decision Tree Classifier, and was able to achieve the accuracy of 81.20%. The sensitivity and specificity were 87.50% and 75%, respectively, as shown in Table 9.

4. Discussion

The experimental protocol in this study was able to induce multiple alterations in the ANS activities in the subjects, as evident in the changes in the HRV features. The significant increase in heart rate, along with significant decrease in the RRI, while playing Game 1 suggests that the game was able to induce sympathetic activity. This is also further corroborated by the other features, with a significant decrease in RMSSD and HF and a significant increase in the LF/HF ratio. The subjects recovered from this during Rest 2, as indicated by the significant difference in the HR, RRI, HF and LF/HF between Rest 2 and Game 1, and the lack of a significant difference between Rest 1 and Rest 2.
However, from the results in the Robust Mixed-Model ANOVA, the cognitive status of the subjects had no significant effect on the HRV. This is different from what we expected, as the NC group were significantly younger than the MCI group, and age can affect the measured HRV. The changes in the HRV features were more due to the experiment session the subject was engaged in. There is also the lack of interaction between cognitive status and the experiment session on their effects on the HRV. Thus, elderly people regardless of cognitive status can play these games and experience similar changes in their HRV.
In terms of the subjects’ game performance, the cognitively normal group had better scores and lower response times compared to the group with cognitive impairment. With these results, the serious games used in this study can be used for cognitive evaluation of the elderly. This is also supported by the high accuracy achieved from classifying the subjects using game performance features.
Some of the results presented in this study differed from some of the literature that focused on cognition and HRV. As mentioned, the cognitive status did not have a significant effect on the HRV, but some studies found a relationship between some HRV features and cognitive function. This could be because not all cognitive domains have a direct correlation to HRV. Based on two review papers, 8 out of 10 studies that used global cognitive tests found a significant positive relationship between HRV parameters and global cognition. For the executive function, 14 out of 26 studies confirm a relationship [25,26]. Lower HRV predicts poorer performance, but this relationship is also dependent on the cognitive task [26]. For memory, three studies found a positive correlation [54,55,56], while seven studies did not [57,58,59,60,61,62,63]. Two studies found a relationship between HRV and language [55,64], but another two did not [57,60]. Six studies investigating the relationship of HRV and processing speed did not find a correlation to HRV features [62,63,65,66,67,68], while only one study showed a correlation [59].
Using machine learning to classify the cognitive status of the subjects generated moderate to high accuracy, specificity and sensitivity. This does show the potential of using the changes in HRV to evaluate the cognitive status of the subject, but still, using game performance features can gave better results. However, these results need to be confirmed in other experiments with larger sample sizes. Cognitive impairment can exhibit multiple sources of variability, thus having a larger sample size is essential for obtaining reliable results.
In addition to the limitations due to sample size, the experiment flow was carried out in a sequential order. The order of the games was not counterbalanced, thus there could be order effects. However, based on the results, the HRV features between the two resting periods showed no significant difference, indicating that the HRV returned to baseline.

5. Conclusions and Future Work

In this study, we investigated the HRV of two groups of elderly people: cognitively normal and cognitively impaired. The changes in HRV features were significantly affected by the experiment sessions and not by the cognitive status of the subjects. No interaction was observed between the cognitive status and sessions that can affect the HRV. Thus, the two groups have similar HRV responses while playing the serious games. This means that the elderly can experience similar changes in their HRV regardless of their cognitive status. The need to measure HRV to assess for MCI may not be as crucial as we thought.
Game performance was significantly different between the two groups, with the cognitively normal subjects performing better than the ones who are cognitively impaired. The serious games presented in this study can be useful in assessing cognitive decline in the elderly. This is further corroborated by the use of game performance in the classification of the two groups, achieving an accuracy of 83.33%, compared to the 68.75% using HRV features. While HRV have the potential to be used for classifying cognitive status, game performance can be more accurate. Screening tools such as MMSE and MoCA are easy to administer, but can be affected by the age, educational and cultural background of the patient [7,69]. From the ML classification results, serious games have the potential of being used as an alternative method for detecting cognitive decline aside from using questionnaires and clinical interviews. This result warrants further development on the design of serious games for assessing cognitive impairment.
For future studies, a larger sample size for each group would be recommended to confirm or refute the findings presented in this study. Other serious games designed for the elderly can also be used to verify or challenge the findings in this study. In addition, the cognitive domains and their relationship with the ANS activity and other biomarkers can also be explored.

Author Contributions

Conceptualization, C.-J.H. and Y.-T.C.; methodology, C.-J.H., Y.-T.C., M.A.C. and M.-W.H.; signal and data analysis, M.A.C.; supervision, J.-J.H. and M.-W.H. All authors have read and agreed to the published version of the manuscript.

Funding

The authors gratefully acknowledge the financial support from the Ministry of Science and Technology of Taiwan (grant number MOST 108-2634-F-367-001) and the Allied Advanced Intelligent Biomedical Research Center (A2IBRC) under the Higher Education Sprout Project of Ministry of Education (grant number 13001090182-EDU).

Institutional Review Board Statement

The study was conducted according to the guidelines of the Declaration of Helsinki, and approved by the Institutional Review Board (or Ethics Committee) of Taichung Veterans General Hospital Taiwan (protocol code SF18297A-1; 16 January 2019).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

Data is available upon request due to ethical and privacy reasons. The data are not publicly available due to privacy concerns of the subjects.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Ageing and Health. Available online: https://www.who.int/news-room/fact-sheets/detail/ageing-and-health (accessed on 3 July 2021).
  2. Taiwan Will Be a Super-Aged Society by 2026. Taiwan News, 12 February 2019. Available online: https://www.taiwannews.com.tw/en/news/3636704 (accessed on 3 July 2021).
  3. Murman, D. The Impact of Age on Cognition. Semin. Hear. 2015, 36, 111–121. [Google Scholar] [CrossRef] [PubMed]
  4. Pais, R.; Ruano, L.; Carvalho, O.P.; Barros, H. Global Cognitive Impairment Prevalence and Incidence in Community Dwelling Older Adults—A Systematic Review. Geriatrics 2020, 5, 84. [Google Scholar] [CrossRef] [PubMed]
  5. Qiu, C.; Johansson, G.; Zhu, F.; Kivipelto, M.; Winblad, B. Prevention of Cognitive Decline in Old Age—Varying Effects of Interventions in Different Populations. Ann. Transl. Med. 2019, 7, S142. [Google Scholar] [CrossRef] [PubMed]
  6. Perron, B.; Schröter, F. (Eds.) Video Games and the Mind: Essays on Cognition, Affect and Emotion; McFarland & Company, Inc.: Jefferson, NC, USA, 2016; ISBN 978-0-7864-9909-0. [Google Scholar]
  7. Wu, Y.; Zhang, Y.; Yuan, X.; Guo, J.; Gao, X. Influence of Education Level on MMSE and MoCA Scores of Elderly Inpatients. Appl. Neuropsychol. Adult 2021, 1–5. [Google Scholar] [CrossRef]
  8. Critchley, H.D.; Eccles, J.; Garfinkel, S.N. Interaction between cognition, emotion, and the autonomic nervous system. In Handbook of Clinical Neurology; Elsevier: Amsterdam, The Netherlands, 2013; Volume 117, pp. 59–77. ISBN 978-0-444-53491-0. [Google Scholar]
  9. Laamarti, F.; Eid, M.; El Saddik, A. An Overview of Serious Games. Int. J. Comput. Games Technol. 2014, 2014, 1–15. [Google Scholar] [CrossRef]
  10. Lee, S.; Kim, J.; Kim, J.; Lee, M. A Design of the U-Health Monitoring System Using a Nintendo DS Game Machine. In Proceedings of the 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Hilton, MN, USA, 2–6 September 2009; IEEE: Minneapolis, MN, USA, 2009; pp. 1695–1698. [Google Scholar]
  11. Finkelstein, J.; Wood, J.; Cha, E.; Orlov, A.; Dennison, C. Feasibility of Congestive Heart Failure Telemanagement Using a Wii-Based Telecare Platform. In Proceedings of the 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology, Buenos Aires, Argentina, 31 August–4 September 2010; IEEE: Piscataway, NJ, USA, 2010; pp. 2211–2214. [Google Scholar]
  12. Fergus, P.; Kifayat, K.; Cooper, S.; Merabti, M.; El Rhalibi, A. A Framework for Physical Health Improvement Using Wireless Sensor Networks and Gaming. In Proceedings of the 3rd International ICST Conference on Pervasive Computing Technologies for Healthcare, London, UK, 4 August 2009; ICST: London, UK, 2009. [Google Scholar]
  13. McKanna, J.A.; Jimison, H.; Pavel, M. Divided Attention in Computer Game Play: Analysis Utilizing Unobtrusive Health Monitoring. In Proceedings of the 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Hilton, MN, USA, 2–6 September 2009; IEEE: Piscataway, NJ, USA, 2009; pp. 6247–6250. [Google Scholar]
  14. Wang, Q.; Sourina, O.; Nguyen, M.K. EEG-Based “Serious” Games Design for Medical Applications. In Proceedings of the 2010 International Conference on Cyberworlds, Singapore, 20–22 October 2010; IEEE: Piscataway, NJ, USA, 2010; pp. 270–276. [Google Scholar]
  15. Atkinson, S.D.; Narasimhan, V.L. Design of an Introductory Medical Gaming Environment for Diagnosis and Management of Parkinson’s Disease. In Proceedings of the Trendz in Information Sciences & Computing(TISC2010), Chennai, India, 17–19 December 2020; IEEE: Piscataway, NJ, USA, 2010; pp. 94–102. [Google Scholar]
  16. Bartolome, N.A.; Zorrilla, A.M.; Zapirain, B.G. A Serious Game to Improve Human Relationships in Patients with Neuro-Psychological Disorders. In Proceedings of the 2010 2nd International IEEE Consumer Electronics Society’s Games Innovations Conference, Hong Kong, 21–23 December 2010; IEEE: Piscataway, NJ, USA, 2010; pp. 1–5. [Google Scholar]
  17. Rego, P.; Moreira, P.M.; Reis, L.P. Serious Games for Rehabilitation: A Survey and a Classification towards a Taxonomy. In Proceedings of the 5th Iberian Conference on Information Systems and Technologies, Santiago de Compostela, Spain, 16–19 June 2010; IEEE: Piscataway, NJ, USA, 2010; pp. 1–6. [Google Scholar]
  18. Cameirão, M.S.; Bermúdez, I.; Badia, S.; Duarte Oller, E.; Verschure, P.F.M.J. The Rehabilitation Gaming System: A Review. Stud. Health Technol. Inform. 2009, 145, 65–83. [Google Scholar]
  19. Uzor, S.; Baillie, L. Investigating the Long-Term Use of Exergames in the Home with Elderly Fallers. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Toronto, ON, Canada, 26 April–1 May 2014; ACM: New York, NU, USA, 2014; pp. 2813–2822. [Google Scholar]
  20. Nguyen, T.T.H.; Ishmatova, D.; Tapanainen, T.; Liukkonen, T.N.; Katajapuu, N.; Makila, T.; Luimula, M. Impact of Serious Games on Health and Well-Being of Elderly: A Systematic Review. In Proceedings of the 50th Hawaii International Conference on System Sciences, Waikoloa Village, HI, USA, 4–7 January 2017. [Google Scholar]
  21. Sokolov, A.A.; Collignon, A.; Bieler-Aeschlimann, M. Serious Video Games and Virtual Reality for Prevention and Neurorehabilitation of Cognitive Decline Because of Aging and Neurodegeneration. Curr. Opin. Neurol. 2020, 33, 239–248. [Google Scholar] [CrossRef]
  22. Wortley, D.; An, J.-Y.; Heshmati, A. Tackling the Challenge of the Aging Society: Detecting and Preventing Cognitive and Physical Decline through Games and Consumer Technologies. Healthc. Inform. Res. 2017, 23, 87. [Google Scholar] [CrossRef] [Green Version]
  23. Li, Y.; Elmaghraby, A.S.; El-Baz, A.; Sokhadze, E.M. Using Physiological Signal Analysis to Design Affective VR Games. In Proceedings of the 2015 IEEE International Symposium on Signal Processing and Information Technology (ISSPIT), Abu Dhabi, United Arab Emirates, 7–10 December 2015; IEEE: Piscataway, NJ, USA, 2015; pp. 57–62. [Google Scholar]
  24. Difference between RR Interval and NN Interval. Available online: https://hexoskin.zendesk.com/hc/en-us/articles/360045123314-Difference-between-RR-interval-and-NN-interval (accessed on 15 April 2022).
  25. Forte, G.; Favieri, F.; Casagrande, M. Heart Rate Variability and Cognitive Function: A Systematic Review. Front. Neurosci. 2019, 13, 710. [Google Scholar] [CrossRef]
  26. Grässler, B.; Hökelmann, A.; Cabral, R.H. Resting Heart Rate Variability as a Possible Marker of Cognitive Decline. Kinesiology 2020, 52, 72–84. [Google Scholar] [CrossRef]
  27. Hou, C.J.; Huang, M.W.; Zhou, J.Y.; Hsu, P.C.; Zeng, J.H.; Chen, Y.T. The Application of Individual Virtual Nostalgic Game Design to the Evaluation of Cognitive Function. In Proceedings of the 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Jeju Island, Korea, 11–15 July 2017; pp. 2586–2589. [Google Scholar]
  28. Chen, Y.-T.; Hou, C.-J.; Derek, N.; Huang, S.-B.; Huang, M.-W.; Wang, Y.-Y. Evaluation of the Reaction Time and Accuracy Rate in Normal Subjects, MCI, and Dementia Using Serious Games. Appl. Sci. 2021, 11, 628. [Google Scholar] [CrossRef]
  29. Ciesielska, N.; Sokołowski, R.; Mazur, E.; Podhorecka, M.; Polak-Szabela, A.; Kędziora-Kornatowska, K. Is the Montreal Cognitive Assessment (MoCA) Test Better Suited than the Mini-Mental State Examination (MMSE) in Mild Cognitive Impairment (MCI) Detection among People Aged over 60? Meta-Analysis. Psychiatr. Pol. 2016, 50, 1039–1052. [Google Scholar] [CrossRef]
  30. Tan, J.; Li, N.; Gao, J.; Wang, L.; Zhao, Y.; Yu, B.; Du, W.; Zhang, W.; Cui, L.; Wang, Q.; et al. Optimal Cutoff Scores for Dementia and Mild Cognitive Impairment of the Montreal Cognitive Assessment among Elderly and Oldest-Old Chinese Population. J. Alzheimer Dis. 2015, 43, 1403–1412. [Google Scholar] [CrossRef]
  31. Tsai, C.-F.; Lee, W.-J.; Wang, S.-J.; Shia, B.-C.; Nasreddine, Z.; Fuh, J.-L. Psychometrics of the Montreal Cognitive Assessment (MoCA) and Its Subscales: Validation of the Taiwanese Version of the MoCA and an Item Response Theory Analysis. Int. Psychogeriatr. 2012, 24, 651–658. [Google Scholar] [CrossRef]
  32. Factors That Can Affect HRV and Stress—VitalScan. Available online: https://www.vitalscan.com/dt_factors_affect_stress.html (accessed on 14 April 2022).
  33. Sammito, S.; Böckelmann, I. Factors Influencing Heart Rate Variability. Int. Cardiovasc. Forum J. 2016, 6. [Google Scholar] [CrossRef]
  34. Shaffer, F.; Ginsberg, J.P. An Overview of Heart Rate Variability Metrics and Norms. Front. Public Health 2017, 5, 258. [Google Scholar] [CrossRef] [Green Version]
  35. Hou, C.-J.; Chen, Y.-T.; Capilayan, M.; Lin, Y.-S.; Huang, M.-W.; Huang, J.-J. Analysis of Heart Rate Variability in Response to Serious Games in Elderly People. Sensors 2021, 21, 6549. [Google Scholar] [CrossRef]
  36. Kaufmann, T.; Sütterlin, S.; Schulz, S.M.; Vögele, C. ARTiiFACT: A Tool for Heart Rate Artifact Processing and Heart Rate Variability Analysis. Behav. Res. Methods 2011, 43, 1161–1170. [Google Scholar] [CrossRef]
  37. Berntson, G.G.; Quigley, K.S.; Jang, J.F.; Boysen, S.T. An Approach to Artifact Identification: Application to Heart Period Data. Psychophysiology 1990, 27, 586–598. [Google Scholar] [CrossRef]
  38. Ramshur, J.T. Design, Evaluation, and Application of Heart Rate Variability Analysis Software (HRVAS). Ph.D Thesis, University of Memphis, Memphis, TN, USA, 2010. [Google Scholar] [CrossRef]
  39. Sassi, R.; Cerutti, S.; Lombardi, F.; Malik, M.; Huikuri, H.V.; Peng, C.-K.; Schmidt, G.; Yamamoto, Y.; Gorenek, B.; Lip, G.Y.H.; et al. Advances in Heart Rate Variability Signal Analysis: Joint Position Statement by the e-Cardiology ESC Working Group and the European Heart Rhythm Association Co-Endorsed by the Asia Pacific Heart Rhythm Society. Europace 2015, 17, 1341–1353. [Google Scholar] [CrossRef]
  40. Kim, H.-G.; Cheon, E.-J.; Bai, D.-S.; Lee, Y.H.; Koo, B.-H. Stress and Heart Rate Variability: A Meta-Analysis and Review of the Literature. Psychiatry Investig. 2018, 15, 235–245. [Google Scholar] [CrossRef] [Green Version]
  41. IBM Corp. IBM SPSS Statistics for Windows; IBM Corp: Armonk, NY, USA, 2017. [Google Scholar]
  42. R Core Team. R: A Language and Environment for Statistical Computing; R Foundation for Statistical Computing: Vienna, Austria, 2019. [Google Scholar]
  43. Mair, P.; Wilcox, R. Robust Statistical Methods in R Using the WRS2 Package. Behav. Res. Methods 2020, 52, 464–488. [Google Scholar] [CrossRef]
  44. Cartacio, S. What Is Automated ML? AutoML—Azure Machine Learning. Available online: https://docs.microsoft.com/en-us/azure/machine-learning/concept-automated-ml (accessed on 15 April 2022).
  45. Le, T.T.; Fu, W.; Moore, J.H. Scaling Tree-Based Automated Machine Learning to Biomedical Big Data with a Feature Set Selector. Bioinformatics 2020, 36, 250–256. [Google Scholar] [CrossRef] [Green Version]
  46. Olson, R.S.; Urbanowicz, R.J.; Andrews, P.C.; Lavender, N.A.; Kidd, L.C.; Moore, J.H. Automating Biomedical Data Science Through Tree-Based Pipeline Optimization. In Applications of Evolutionary Computation; Squillero, G., Burelli, P., Eds.; Lecture Notes in Computer Science; Springer International Publishing: Cham, Switzerland, 2016; Volume 9597, pp. 123–137. ISBN 978-3-319-31203-3. [Google Scholar]
  47. Olson, R.S.; Bartley, N.; Urbanowicz, R.J.; Moore, J.H. Evaluation of a Tree-Based Pipeline Optimization Tool for Automating Data Science. In Proceedings of the Genetic and Evolutionary Computation Conference 2016, Denver, CO, USA, 20–24 July 2016; ACM: New York, NY, USA, 2016; pp. 485–492. [Google Scholar]
  48. HRV Demographics, Part 1—Age & Gender. Available online: https://elitehrv.com/hrv-demographics-age-gender (accessed on 18 February 2022).
  49. Brownlee, J. Recursive Feature Elimination (RFE) for Feature Selection in Python; Machine Learning Mastery: San Juan, Puerto Rico, 2020. [Google Scholar]
  50. Johnson, K.; Kuhn, M. 11.3 Recursive feature elimination. In Feature Engineering and Selection: A Practical Approach for Predictive Models; Taylor & Francis Group: New York, NY, USA, 2019. [Google Scholar]
  51. Fardeen, A. Decision Tree Classifier in Python Sklearn with Example. MLK Machine Learning Knowledge, 29 July 2019. [Google Scholar]
  52. sy2002. RBFSampler actually is not using any radial basis functions (RBFs). In Medium; GitHub, Inc.: San Francisco, CA, USA, 2019. [Google Scholar]
  53. Rahimi, A.; Recht, B. Weighted Sums of Random Kitchen Sinks: Replacing Minimization with Randomization in Learning. In Proceedings of the NIPS’08: Proceedings of the 21st International Conference on Neural Information Processing Systems, Vancouver, BC, Canada, 8–10 December 2008. [Google Scholar]
  54. Nonogaki, Z.; Umegaki, H.; Makino, T.; Suzuki, Y.; Kuzuya, M. Relationship between Cardiac Autonomic Function and Cognitive Function in Alzheimer’s Disease: Heart and Cognition in AD. Geriatr. Gerontol. Int. 2017, 17, 92–98. [Google Scholar] [CrossRef]
  55. Frewen, J.; Finucane, C.; Savva, G.M.; Boyle, G.; Coen, R.F.; Kenny, R.A. Cognitive Function Is Associated with Impaired Heart Rate Variability in Ageing Adults: The Irish Longitudinal Study on Ageing Wave One Results. Clin. Auton. Res. 2013, 23, 313–323. [Google Scholar] [CrossRef]
  56. Gillie, B.L.; Vasey, M.W.; Thayer, J.F. Heart Rate Variability Predicts Control Over Memory Retrieval. Psychol. Sci. 2014, 25, 458–465. [Google Scholar] [CrossRef]
  57. Britton, A.; Singh-Manoux, A.; Hnatkova, K.; Malik, M.; Marmot, M.G.; Shipley, M. The Association between Heart Rate Variability and Cognitive Impairment in Middle-Aged Men and Women. Neuroepidemiology 2008, 31, 115–121. [Google Scholar] [CrossRef]
  58. Zeki Al Hazzouri, A.; Elfassy, T.; Carnethon, M.R.; Lloyd-Jones, D.M.; Yaffe, K. Heart Rate Variability and Cognitive Function In Middle-Age Adults: The Coronary Artery Risk Development in Young Adults. Am. J. Hypertens. 2018, 31, 27–34. [Google Scholar] [CrossRef] [Green Version]
  59. Mahinrad, S.; Jukema, J.W.; van Heemst, D.; Macfarlane, P.W.; Clark, E.N.; de Craen, A.J.M.; Sabayan, B. 10-Second Heart Rate Variability and Cognitive Function in Old Age. Neurology 2016, 86, 1120–1127. [Google Scholar] [CrossRef]
  60. Antonelli Incalzi, R.; Corsonello, A.; Trojano, L.; Pedone, C.; Acanfora, D.; Spada, A.; D’Addio, G.; Maestri, R.; Rengo, F.; Rengo, G. Heart Rate Variability and Drawing Impairment in Hypoxemic COPD. Brain Cogn. 2009, 70, 163–170. [Google Scholar] [CrossRef]
  61. Jennings, J.R.; Allen, B.; Gianaros, P.J.; Thayer, J.F.; Manuck, S.B. Focusing Neurovisceral Integration: Cognition, Heart Rate Variability, and Cerebral Blood Flow: Focusing Neurovisceral Integration. Psychophysiology 2015, 52, 214–224. [Google Scholar] [CrossRef] [Green Version]
  62. Nicolini, P.; Ciulla, M.M.; Malfatto, G.; Abbate, C.; Mari, D.; Rossi, P.D.; Pettenuzzo, E.; Magrini, F.; Consonni, D.; Lombardi, F. Autonomic Dysfunction in Mild Cognitive Impairment: Evidence from Power Spectral Analysis of Heart Rate Variability in a Cross-Sectional Case-Control Study. PLoS ONE 2014, 9, e96656. [Google Scholar] [CrossRef]
  63. Stenfors, C.U.D.; Hanson, L.M.; Theorell, T.; Osika, W.S. Executive Cognitive Functioning and Cardiovascular Autonomic Regulation in a Population-Based Sample of Working Adults. Front. Psychol. 2016, 7, 1536. [Google Scholar] [CrossRef] [Green Version]
  64. Yang, A.C.-C.; Tsai, S.-J.; Hong, C.-J.; Yang, C.-H.; Hsieh, C.-H.; Liu, M.-E. Association between Heart Rate Variability and Cognitive Function in Elderly Community-Dwelling Men without Dementia: A Preliminary Report: Letters to the Editor. J. Am. Geriatr. Soc. 2008, 56, 958–960. [Google Scholar] [CrossRef]
  65. Hansen, A.L.; Johnsen, B.H.; Thayer, J.F. Vagal Influence on Working Memory and Attention. Int. J. Psychophysiol. 2003, 48, 263–274. [Google Scholar] [CrossRef]
  66. Hansen, A.L.; Johnsen, B.H.; Sollers, J.J.; Stenvik, K.; Thayer, J.F. Heart Rate Variability and Its Relation to Prefrontal Cognitive Function: The Effects of Training and Detraining. Eur. J. Appl. Physiol. 2004, 93, 263–272. [Google Scholar] [CrossRef]
  67. Hansen, A.L.; Johnsen, B.H.; Thayer, J.F. Relationship between Heart Rate Variability and Cognitive Function during Threat of Shock. Anxiety Stress Coping 2009, 22, 77–89. [Google Scholar] [CrossRef]
  68. Kimhy, D.; Crowley, O.V.; McKinley, P.S.; Burg, M.M.; Lachman, M.E.; Tun, P.A.; Ryff, C.D.; Seeman, T.E.; Sloan, R.P. The Association of Cardiac Vagal Control and Executive Functioning—Findings from the MIDUS Study. J. Psychiatr. Res. 2013, 47, 628–635. [Google Scholar] [CrossRef] [Green Version]
  69. Slavych, B. Pros and Cons of Various Screening Tools for Dementia. Leader Live, 30 May 2019. [Google Scholar]
Applsci 12 04164 g001 550
Figure 1. Experiment flow.
Figure 1. Experiment flow.
Applsci 12 04164 g001
Applsci 12 04164 g002 550
Figure 2. Cognitive Game based on Nostalgia Theory.
Figure 2. Cognitive Game based on Nostalgia Theory.
Applsci 12 04164 g002
Applsci 12 04164 g003 550
Figure 3. Whack-a-Mole Game.
Figure 3. Whack-a-Mole Game.
Applsci 12 04164 g003
Applsci 12 04164 g004 550
Figure 4. Hit-the-Ball Game. The subject has to distinguish between the two ball types and press the correct button when the football reaches the target area.
Figure 4. Hit-the-Ball Game. The subject has to distinguish between the two ball types and press the correct button when the football reaches the target area.
Applsci 12 04164 g004
Applsci 12 04164 g005 550
Figure 5. Traditional Machine Learning Pipeline.
Figure 5. Traditional Machine Learning Pipeline.
Applsci 12 04164 g005
Table
Table 1. Sizes for each dataset.
Table 1. Sizes for each dataset.
DatasetTotal Number of Features
HRV features only40 (4 sessions × 10 HRV features)
Game features only13
HRV and Game features53
Table
Table 2. Demographic and Characteristics of the NC and MCI subject groups.
Table 2. Demographic and Characteristics of the NC and MCI subject groups.
FactorsNC (n = 24)MCI (n = 24)p-Value
Gender (%) 0.093 (Χ2)
Male9 (37.5%)3 (12.5%)
Female15 (62.5%)21 (87.5%)
Age, years72.0833 ± 5.1068 76.6250 ± 7.02670.014 *
Education, years10.3333 ± 3.50987.0833 ± 4.42240.004 *
MoCA score26.6250 ± 1.8133015.6250 ± 7.57460.000 *
Data, except for gender, is expressed as mean ±standard deviation. The factors of the two groups were compared using Mann-Whitney U test (* p < 0.05), except for gender, which was compared using Chi-Square test (Χ2). Abbreviations: NC, Normal Cognition; MCI, Mild Cognitive impairment; MoCA, Montreal Cognitive Assessment
Table
Table 3. Extracted ECG features.
Table 3. Extracted ECG features.
Normal Cognition
FeatureRest 1Game 1Rest 2Game 2&3
HR71.75 (11.73)73.90 (12.50)68.60 (11.13)69.25 (12.33)
RRI839.50 (141.25)812.25 (135.02)876.10 (137.13)867.85 (147.73)
SDNN34.40 (21.85)23.15 (13.30)27.90 (21.30)25.00 (15.50)
pNN500.00 (4.68)0.15 (2.10)0.40 (7.50)0.30 (7.03)
RMSSD17.15 (15.88)13.65 (10.03)18.10 (17.60)17.35 (14.45)
TP149.735 (346.08)112.925 (170.71)162.090 (321.51)163.490 (163.93)
LF34.205 (98.3525)37.08 (56.475)33.225 (89.9975)46.450 (82.78)
HF43.23 (68.245)21.885 (30.415)40.05 (105.8)37.145 (53.3925)
LF/HF0.614 (1.6123)1.6445 (1.6950)0.746 (1.1583)1.306 (1.2670)
SampEn1.5040 (0.52)1.5500 (0.57)1.6770 (0.57)1.6460 (0.52)
Mild Cognitive Impairment
FeatureRest 1Game 1Rest 2Game 2&3
HR72.50 (15.53)76.95 (19.17)73.25 (18.15)72.45 (18.05)
RRI828.40 (185.10)781.55 (193.60)820.75 (218.60)830.35 (215.12)
SDNN26.0500 (20.68)26.45 (13.60)26.10 (20.75)24.65 (16.63)
pNN500.90 (4.45)0.60 (7.08)0.25 (16.88)0.60 (11.30)
RMSSD19.20 (14.30)16.95 (12.85)18.75 (22.65)18.15 (18.80)
TP127.89 (417.74)142.43 (145.16)110.36 (277.21)126.58 (204.09)
LF23.49 (53.32)32.18 (54.94)21.04 (71.78)41.38 (40.66)
HF37.58 (58.935)24.175 (42.90)40.415 (75.545)27.205 (60.753)
LF/HF0.697 (1.003)1.602 (2.024)1.018 (1.086)1.325 (1.955)
SampEn1.6245 (0.66)1.5025 (0.67)1.8195 (0.56)1.7205 (0.54)
Data are expressed as median (interquartile range).
Table
Table 4. Effects of group and session on ECG features.
Table 4. Effects of group and session on ECG features.
FeatureGroupSessionInteraction
(Group × Session)
Fp-ValueFp-ValueFp-Value
HR0.34650.560946.47220.0000 *0.61840.6099
RRI0.2623 0.612550.28980.0000 *0.60730.6168
SDNN0.05970.80871.75190.18490.54640.6556
pNN500.44710.51050.94430.43740.10010.9591
RMSSD0.13830.71273.48000.0319 *0.22370.8790
TP0.07910.78041.75840.18540.88760.4634
LF0.44170.51160.92890.44260.21990.8816
HF0.17520.67864.32860.0143 *0.86410.4733
LF/HF0.07320.78877.74730.0009 *0.77570.5189
SampEn0.52270.47532.22790.11241.23330.3206
* p < 0.05.
Table
Table 5. Post hoc comparison for features with significant differences on the main effects.
Table 5. Post hoc comparison for features with significant differences on the main effects.
FeaturePost-Hoc Analysis
HR(2) > (1); (2) > (3); (2) > (4)
RRI(2) < (1); (2) < (3); (2) < (4)
RMSSD(2) < (1)
HF(2) < (1); (2) < (3); (2) < (4); (4) < (1)
LF/HF(2) > (1); (2) > (3); (4) > (1); (4) > (3)
(1)—Rest 1; (2)—Game 1; (3)—Rest 2; (4)—Games 2&3.
Table
Table 6. Game Results and Statistical Analysis between the two groups.
Table 6. Game Results and Statistical Analysis between the two groups.
GameNCMCIp-Value
Nostalgic Game23.3333 (1.7110)19.1250 (4.7026)0.000 *
Whack-a-Mole Game
Level 1Hit Ratio0.9958 (0.0204)0.8318 (0.2476)0.001 *
Response Time1.2318 (0.2083)1.8416 (0.8458)0.000 *
Level 2Hit Ratio0.9792 (0.0415)0.8455 (0.3035)0.051
Response Time1.0992 (0.1603)1.5788 (0.6251)0.000 *
Level 3Hit Ratio0.9167 (0.0637)0.7591 (0.2889)0.024 *
Response Time1.0238 (0.1263)1.4606 (0.6348)0.000 *
Hit-the-Ball Game
Level 1Hit Ratio0.9500 (0.0780)0.9034 (0.1528)0.203
Response Time0.3212 (0.2770)0.5652 (0.5996)0.085
Level 2Hit Ratio0.9958 (0.0204)0.8818 (0.2108)0.004 *
Response Time0.1068 (0.1147)0.2490 (0.3424)0.090
Level 3Hit Ratio1.0000 (0.0000)0.8818 (0.1991)0.000 *
Response Time0.2005 (0.1073)0.4485 (0.3839)0.001 *
Data are expressed as mean (standard deviation); * p < 0.05.
Table
Table 7. Confusion Matrix with Precision and Recall rates using HRV only.
Table 7. Confusion Matrix with Precision and Recall rates using HRV only.
PredictedTotalRecall Rate
MCINC
ActualMCI1682466.67%
NC7172470.83%
Total232548
Precision Rate69.57%68% Accuracy: 68.75%
MCI—Mild Cognitive Impairment; NC—Normal Cognition.
Table
Table 8. Confusion Matrix with Precision and Recall rates using game performance only.
Table 8. Confusion Matrix with Precision and Recall rates using game performance only.
PredictedTotalRecall Rate
MCINC
ActualMCI1952479.17%
NC3212487.50%
Total222648
Precision Rate86.36%80.77% Accuracy: 83.33%
MCI—Mild Cognitive Impairment; NC—Normal Cognition.
Table
Table 9. Confusion Matrix with Precision and Recall rates using both HRV and game performance.
Table 9. Confusion Matrix with Precision and Recall rates using both HRV and game performance.
PredictedTotalRecall Rate
MCINC
ActualMCI2132487.50%
NC6182475%
Total272148
Precision Rate77.78%85.71% Accuracy: 81.20%
MCI—Mild Cognitive Impairment; NC—Normal Cognition.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Hou, C.-J.; Chen, Y.-T.; Capilayan, M.A.; Huang, M.-W.; Huang, J.-J. Analysis of Heart Rate Variability and Game Performance in Normal and Cognitively Impaired Elderly Subjects Using Serious Games. Appl. Sci. 2022, 12, 4164. https://doi.org/10.3390/app12094164

AMA Style

Hou C-J, Chen Y-T, Capilayan MA, Huang M-W, Huang J-J. Analysis of Heart Rate Variability and Game Performance in Normal and Cognitively Impaired Elderly Subjects Using Serious Games. Applied Sciences. 2022; 12(9):4164. https://doi.org/10.3390/app12094164

Chicago/Turabian Style

Hou, Chun-Ju, Yen-Ting Chen, Mycel A. Capilayan, Min-Wei Huang, and Ji-Jer Huang. 2022. "Analysis of Heart Rate Variability and Game Performance in Normal and Cognitively Impaired Elderly Subjects Using Serious Games" Applied Sciences 12, no. 9: 4164. https://doi.org/10.3390/app12094164

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop