Effects of Exercise Type and Gameplay Mode on Physical Activity in Exergame

Abstract

Exercise games (exergames) that combine both exercise and video gaming train people in a fun and competitive manner to lead a healthy lifestyle. Exergames promote more physical exertion and help users exercise more easily and independently in any place. Many studies have been conducted to evaluate the positive effects of exergames. However, in most studies, heart rate was mainly used to measure the effect of exercise. In this study, we evaluate the effects of exercise according to the exercise type (rest, walking, tennis, and running) and gameplay mode (single, competition, and cooperation) of exergaming via quantitative measurements using electrocardiogram (ECG) and Kinect. The multiple comparison results reveal that physical activity measured with Kinect was statistically significant even in exergames that did not show statistically significant differences according to ECG. Running was statistically significant compared to other exercise types, and there was a significant difference in competition compared to other gameplay modes.

1. Introduction

Exercise can help improve health, strengthen vitality and prevent disease. Exergames aim to increase physical activity through games, and help one to enjoy exercise. Nintendo Wii Fit and Wii Sports, released in 2005, allow people to perform various sports such as tennis, fishing, and golf using the Wii remote control [1]. In 2010, the Microsoft Xbox game console added a Kinect camera sensor, allowing users to enjoy games freely by tracking the users’ body movements [2]. Such exergames have been utilized to improve the elderly’s sense of balance, and for dementia prevention [3]. The Nintendo Switch exergame has been used as a physical education program for elementary school students to foster creativity and critical thinking [4]. Moreover, long-term Xbox gaming significantly reduced Body Mass Index (BMI) and improved cardiac metabolic health in obese children (10–12 years old) [5]. As such, public interest in exergames is increasing in many domains.

Exercises have different levels of intensity. Typically, walking is a low-intensity exercise, and jogging and bowling are representative of moderate-intensity exercises. Tennis and cross fit are known as high-intensity workouts. However, these actual exercises do not always have the same effects in exergames. According to a study [6] that evaluated low-, medium-, and high-intensity exergame platforms, it was found that all exergaming was moderate-intensity. Another study examining the exercise effects of Nintendo Wii Sports boxing and Xbox dancing found that exergaming had similar exercise effects to actual physical exercise in terms of energy consumption [7]. However, although tennis is a high-intensity exercise while bowling is a moderate-intensity exercise, there was no difference between the two in the context of exergaming (Nintendo Wii Sports tennis and bowling) [8].

Some exergames (such as tennis or boxing) offer a multiplayer mode, and the effects of exercise are different between the multiplayer mode and the single-player mode [9,10,11,12,13]. Single-player mode proceeds with a virtual partner, while participants in the multiplayer mode play together as partners. When comparing the exercise effects of the two modes, the effect of the multiplayer mode was greater than that of the single-player mode, regardless of the type of exercise. For example, when performing aerobics in a group with an actual partner, the duration of exercise, satisfaction with exercise and willingness to exercise were high [10,11]. Even in competitions such as cycling, satisfaction and willingness to exercise were higher when playing with real people than with virtual partners [9].

There are many previous studies showing the positive effects of exergaming. These studies are largely divided into comparisons between single- and multiple-player gameplay modes [9,10,11,12,13], comparisons between exergaming and actual exercise [14], comparisons between exergames and sedentary video games [8,15,16], and exergaming used for mental health [3,17] or physical education [4,5,7,18,19]. However, most of these studies relied on subjective questionnaires, energy consumption, heart rate, or accelerometer sensors to evaluate the exercise effects. Previous user studies have mostly used heart rate for the objective evaluation of exercise effects, and no study has examined the effects of exercise using electrocardiogram (ECG) and Kinect.

Accordingly, we aim to evaluate the exercise effects of the type of exergame and gameplay mode using ECG and Kinect. We compared Nintendo Switch (low-intensity walking, moderate-intensity tennis, and high-intensity running) against a no-intensity rest. We also compared the gameplay modes of Nintendo Switch tennis (single-player mode, multiplayer competitive mode, and multiplayer cooperative mode) in order to compare the effects between virtual players and real players, as well as the competitive and cooperative modes. In this study, we measured the exercise effect of exergaming for 3 min using ECG and Kinect sensor together, and employed post-test subjective evaluations. In order to increase the accuracy of measurements of the short-time exergame effect, each exergame was measured twice.

This paper is structured as follows. Section 2 reviews the related work on exergame studies. Section 3 presents the experimental design and method. Section 4 presents the quantitative measurement results based on ECG and Kinect and the results of the post-test subjective questionnaires. Section 5 contains the discussion, and finally, Section 6 presents a conclusion and discusses the directions of future research.

2. Related Work

Exergames such as Nintendo Wii Sport and Xbox fitness provide fun and health to modern people who lack exercise. Users perform actions similar to those in actual sports, providing an immersive feeling as if they were actually playing the sport. Research related to exergaming mainly comprises usability studies for physical and cognitive rehabilitation, exercise training or physical education [3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19], and technology development for exercise platforms or to enhance the exercise system performance using multiple Kinect sensors [18,19,20,21,22].

First, some studies focus on finding the effects of gameplay modes on exercise games [9,10,11,12,13]. In [9], the exercise effect in a competition between a real player and a virtual player was compared with that of a 30-min cycling exergame. The results show that there was a higher intensity of exercise, based on the maximum heart rate, when riding with a real partner compared to riding with a virtual partner.

In [10], an experiment was conducted with 120 college students to compare the effects of undertaking exergames alone versus playing with a group, and between playing with virtual and real partners. While participants played exergames alone or as a group with virtual or real partners, the duration of exercise and perceived exercise intensity were measured. The results reveal that the duration of exercise was highest when working with a real partner, and the duration of exercise was also higher when playing with a virtual partner than when exercising alone. It was confirmed that playing an exergame with a virtual partner can help with motivation to exercise, but it does not have the same effects as playing an exergame with a real person.

In [12], the effects of the multiplayer mode were investigated on enjoyment, future play motivation, and actual physical activity intensity. In total, 162 college students played Xbox 360 Kinect exergames under three conditions: a single-player game, cooperation with other players in the same physical space, and competition with other players in a separate physical space. The results confirm that the competitive mode was the most effective mode since it generated enjoyment, motivation for future play, and high physical intensity. Additionally, it was found that cooperative and competitive gameplay in exergaming may influence physiological and psychosocial changes [13]. The competition mode has been found to increase energy expenditure and aggression in intense, short-duration games. The cooperation mode, on the other hand, was found to increase motivation, promote sustained play, increase self-efficacy and increase pro-social behavior.

Secondly, a comparative study has been undertaken between the effects of exercise games and those of actual exercise [14]. This study evaluated the amount of exercise undertaken by 14 children aged 10–13 years when using Nintendo Wii Sports (bowling and boxing), two-step (first and second level) DDR and three-speed (low, medium, and high) treadmill walking. It measured the subject’s energy expenditure, heart rate, gait speed, and perceived momentum during 15-min exercises. Nintendo Wii boxing, second-level DDR, and high-speed treadmill walking showed high energy consumption and perceived momentum. It showed a difference in the gait speed of the three exercises, but in the case of the Nintendo Wii, there was a lot of upper body movement. As a result, the energy consumption of these games was similar to that of moderate-intensity jogging. It was found that physically active games such as Nintendo Wii can be a safe and fun way to stimulate energy expenditure for children.

Lastly, studies have compared between exergames and sedentary video games concerning the subject’s heart rate and activity amount [8,15,16]. An experiment was conducted on 13 adolescents who had experience with the Nintendo Wii, investigating the effect of upper-body movement on energy consumption [8]. The subjects performed 10 min of bowling, tennis, and boxing with Nintendo Wii Sports, and a racing game in the sedentary Xbox format. The subject’s heart rate, oxygen consumption (for energy consumption), and activity amount were measured using accelerometer sensors attached to the left hip, right hip, left wrist, and right wrist. The results show that heart rate and energy consumption increased when using Nintendo Wii Sports compared to sedentary gaming, and the results for Nintendo Wii boxing showed effects of exercise with moderate energy consumption. In addition, the heart rate and amount of hip activity were the factors with the greatest influence on energy consumption.

A study comparing the effects of sedentary video gaming and exergames in children and adults assessed calorie expenditure and physical activity [15]. In total, 22 children and 20 adults played Nintendo Wii boxing and a sedentary video game for 10 min after 3 min of adaptation time. The results show that both the children and adults exhibited increased energy expenditure most significantly when playing the Nintendo Wii, and the activity especially increased when the children played the Nintendo Wii. These results confirm that exergaming can promote activity and increase energy expenditure and activity in both children and adults.

In a comparative study between sedentary video games and exergames in 18 children aged 6–12 years, the responses of energy expenditure and heart rate were evaluated [16]. The subjects played bowling, mat (walking or running with side-stepping), and sedentary video games on a XaviX, one of the exergame platforms, for 15 min each. The results show that energy consumption and heart rate were significantly increased in exergaming compared to the sedentary video game. This suggests that the exergame could provide children with an attractive activity alternative.

As described above, a lot of user studies have focused on commercial exergames such as Nintendo Wii or Microsoft Xbox when evaluating exercise training. Previous studies have contained comparisons between exergames and sedentary video games, evaluations to see how they differed from actual exercise, and evaluations of single-player and multiple-player gameplay modes, but it is difficult to find comparisons between exergames. Additionally, most of these studies have relied on subjective questionnaires, energy expenditure, heart rate, and/or accelerometers to evaluate exercise effects. In particular, heart rate was preferred as an objective measure. No studies have used Kinect skeleton information in conjunction with ECG to examine user physical activity or the effects of exercise.

3. Method

3.1. Experiment Design

This experiment adopted a within-subject design in which each subject performed all exergames (Walk, Tennis, Run, Tennis Cooperation, and Tennis Competition). First, before proceeding to the actual exergames, the subject took a rest (Rest) to give a data analysis reference. Then, the subject was randomly assigned to an intervention group (exergame). Ref. [23] shows a table of the Metabolic Equivalent of Task (MET) values of various physical activities, including activity-prompting video games (e.g., Wii Fit). Based on this table, we chose the single-player low- (MET 2.3), medium- (MET 4.3), and high-intensity (MET 7.2) exergames of Walk, Tennis, and Run. In addition, we selected the cooperative mode and competitive mode as the representative gameplay modes of the two-player exergame. There was a break of 2 min before each 3 min of exergaming; the exergaming was carried out twice per each exergaming session, and the average of the two measurements was used for analysis.

3.2. Participants

A total of 24 college students (13 female, 11 male) majoring in computer science-related fields volunteered to take part in this study. The subjects were recruited by selecting those who voluntarily agreed to participate, excluding those with physical activity restrictions or special diseases. The characteristics of the subjects’ demographic information are shown in

Table 1. Demographic information of participants (Mean ± SD).

Demographic	Female (n = 13)	Male (n = 11)	Total (n = 24)
Age (year)	22.7 ± 1.0	23.6 ± 1.2	23.1 ± 1.2
BMI (kg/m2)	20.6 ± 3.7	23.0 ± 3.0	21.7 ± 3.5
Percent Body Fat (%)	33.8 ± 3.7	18.8 ± 4.8	26.7 ± 8.7
Amount of Exercise per Week	1.4 ± 1.4	2.7 ± 3.0	2.0 ± 2.3

. The average age of the subjects was 23.1 years, and their BMI index was 21.7, which was mostly in the normal range (18.5 to 22.9). The body fat percentage was 33.8% for women and 18.8% for men, all satisfying the standard criteria. The average amount of exercise per week was 2.0 h of moderate-intensity physical activity (females 1.4 h and males 2.7).

Twelve subjects reported that they were familiar with the Nintendo Switch game. In response to a familiarity questionnaire regarding the Nintendo Switch Ring Fit Adventure game and the Mario Tennis Ace game used in this experiment, eight subjects showed that they had experience. All the subjects had no difficulty in adapting to the Ring Fit Adventure and Mario Tennis Ace games, and showed relatively high interest in these exercise games. On the other hand, as regards the Polar H10 ECG band and Microsoft’s Kinect sensor camera, the results show that most of the subjects (22 participants) had no experience using it at all. Most importantly, all subjects were right-handed, and held the Nintendo Switch Joy-Con controller in their right hands.

3.3. Procedure

Figure 1 shows the overall experimental procedure. First, the institutional review boards (IRB) explanation was given and the informed consent form was signed (5 min); then the pre-test questionnaire survey (10 min), the attachment of the ECG to the subject’s chest (10 min), the rest session (5 min), 5 exergaming evaluation sessions (walk, tennis, run, tennis cooperation, and tennis competition) performed twice (50 min), and then the post-test interview and questionnaire survey (10 min). Each 5-min session consisted of a 2-min waiting period and a 3-min actual session. The time required to participate in the study was approximately 1 h and 30 min. The pre-test questionnaire recorded the subject’s demographic information, such as weight, height, age, sex, and body fat. The post-test questionnaire asked about what kind of exercise was fun and whether it had any effect.

3.4. Apparatus

The experimental apparatus was a Nintendo Switch game console and a 70-inch TV. The subject interacted with the game using the Nintendo Switch Joy-Con controller, which recognized the motion of the user’s body with internal analog sticks, digital buttons, an accelerometer, and gyroscope sensors. Figure 2 shows the subjects performing different types of exercise, such as resting (no-intensity), walking (low-intensity), tennis (medium-intensity) and running (high-intensity), and the different gameplay modes, namely, Tennis Single, Competition, and Cooperation; the subject’s performance was measured by ECG and Kinect sensors.

To evaluate the effects of exergame type and gameplay mode on the performance of subjects, a Polar H10 electrocardiogram (ECG) sensor and Microsoft’s Kinect V2 sensor were used. By attaching a Polar H10 ECG sensor onto the chest of the subject, heart rate R-R interval data were measured in real time with 1000 Hz sampling. The R-R interval is the time between the “R” peaks (in the ECG) of successive heart beats. To ensure the reliability of the heart rate variability test, the minimum measurement time was set to 3 min [24]. Then, we used Kubios HRV standard software for heart rate variability (HRV) analysis [25]. The Kinect depth camera sensor can detect and track 25 of the skeleton joints of a user in three-dimensional coordinates (x, y, z) via Kinect 2.0 SDK [26]. The Kinect sensor measures the extracted skeletal information in real-time in units of 30 milliseconds (5400 frames for 3-min exergaming session).

Figure 3 shows the exergame played by the subjects in the experiment. Nintendo Switch Ring Fit Adventure World 1 Beginnia was selected for the walk test, and Ring Fit Adventure World 7 Starting-Block Bridge was selected for the run test. Mario Tennis Ace Single Match with a computer agent was selected for the Tennis Single mode test. Mario Tennis Ace Double Match with two players against two computer agents was used to explore the Tennis Cooperation mode, and the Tennis Single Match with two players was used as the Tennis Competition mode.

3.5. Measurements

The method for evaluating heart rate variability using the ECG sensor can be divided into time domain analysis and frequency domain analysis. In the time domain, we used the mean standard deviation of the normal-to-normal (NN) intervals (SDNN), the square root of the mean of the sum of the squares of differences between adjacent NN intervals (RMSSD), the mean heart rate (Mean HR) and the max heart rate (Max HR), and in the frequency domain, we used power in the low-frequency range in normalized units (LF), power in the high-frequency range in normalized units (HF), and the ratio of absolute LF to absolute HF power (LF/HF ratio). The measured variables can be described as follows.

• SDNN (milliseconds): Standard deviation of all NN (normal R-R) intervals, Equation (1).

$$\mathrm{SDNN}\left(\mathrm{ms}\right)=\sqrt{\frac{{\sum }_{i=1}^N{\left(R{R}_i-\overline{RR}\right)}^2}{N-1}}$$

(1)

• RMSSD (milliseconds): The square root of the mean of the sum of the squares of differences between adjacent NN intervals, Equation (2).

$$\mathrm{RMSSD}\left(\mathrm{ms}\right)=\sqrt{\frac{{\sum }_{i=1}^{N-1}{\left(R{R}_{i+1}-R{R}_i\right)}^2}{N-1}}$$

(2)

• Mean HR (beats per minute): The average number of heart beats per minute.
• Max HR (beats per minute): The highest number of heart beats per minute.
• LF (ms²): Power in the low-frequency range (0.04–0.15 Hz) in normalized units.
• HF (ms²): Power in the high-frequency range (0.15–0.4 Hz) in normalized units.
• LF/HF ratio (%): Sympathovagal balance (the ratio of absolute LF to absolute HF power).

The SDNN increases as the heart rate changes irregularly. RMSSD is good at indicating cardiac change factors over a short period of time. HR indicates how many times the heart beats per minute. A healthy person’s normal heart rate (HR) is about 60 to 100 beats per minute. In general, low HR corresponds to rest and high HR corresponds to exercise or activity. Additionally, running and cardiovascular exercise increase Max HR. HF represents the activity of the parasympathetic nervous system in the heart, and can be used to reduce chronic stress and aging, while LF reflects the activity of the sympathetic nervous system in the heart, and can be reduced by lack of energy and fatigue.

Figure 4 shows the Kinect-based skeleton model for exercise [22]. This model consists of 14 of the major body joints among 25 of the skeletal joints shown by Kinect 2.0 SDK, which are Hand Left (HL), Hand Right (HR), Elbow Left (EL), Elbow Right (ER), Shoulder Left (SL), Shoulder Right (SR), Ankle Left (AL), Ankle Right (AR), Knee Left (KL), Knee Right (KR), hiP Left (PL), hiP Right (PR), Spine Shoulder (SS), and Spine Base (SB). This model is useful for generalized user motion data measurement, and it is not affected by physical characteristics (e.g., being tall or fat). In Figure 5, Δd_ELi is the position displacement at the Left Elbow (EL), and Δθ_ERi is the angular displacement at the Right Elbow (ER) joint flexion.

The skeletal information collected from the Kinect sensor requires a calibrated value due to the different body types of each person and the distance from the sensor. Therefore, the sum of the joint’s position displacement and the amount of change in the angle of the joints were used as the experimental measurements. The amount of center movement of the body, the upper-body momentum, the lower-body momentum, and the angular change of elbow and knee were used to measure the exercise effect in this paper. The description of the variables is given below.

• SpineBase (millimeters): Sum of positional displacement ($∆d_{S{B}_i}$) between two frames (i, i + 1) of SB (SpineBase), Equation (3).

$$\begin{gathered} SpineBase={\displaystyle \sum }_{i=0}^{M-1}∆d_{S{B}_i} \\ ∆d_{S{B}_i}=\Vert S{B}_{i+1}-S{B}_i\Vert {}_2=\sqrt{{\left(x_{SB}{}_{i+1}-x_{SB}{}_i\right)}^2+{\left(y_{SB}{}_{i+1}-y_{SB}{}_i\right)}^2+{\left(z_{SB}{}_{i+1}-z_{SB}{}_i\right)}^2} \end{gathered}$$

(3)

• Upper (millimeters): Sum of positional displacement between frames of upper body skeleton joints, SB (SpineShoulder), SL (ShoulderLeft), SR (ShoulderRight), EL (ElbowLeft), ER (ElbowRight), HL (HandLeft) and HR (HandRight), Equation (4).

$$Upper={\displaystyle \sum }_{i=0}^{M-1}\left(∆d_{S{S}_i}+∆d_{S{L}_i}+∆d_{S{R}_i}+∆d_{E{L}_i}+∆d_{E{R}_i}+∆d_{H{L}_i}+∆d_{H{R}_i}\right)$$

(4)

• Lower (millimeters): Sum of positional displacement between frames of lower-body skeleton joints, SB (Spine Base), PL (hiP Left), PR (hiP Right), KL (Knee Left), KR (Knee Right), AL (Ankle Left), and AR (Ankle Right), Equation (5).

$$Lower={\displaystyle \sum }_{i=0}^{M-1}\left(∆d_{S{B}_i}+∆d_{P{L}_i}+∆d_{P{R}_i}+∆d_{K{L}_i}+∆d_{K{R}_i}+∆d_{A{L}_i}+∆d_{A{R}_i}\right)$$

(5)

• Era (radian): $∆{\theta }_{E{R}_i}$ is the sum of angular displacement between two frames (i, i+1) of ER (Elbow Right). ${\theta }_{ER}$ is the angle between the two vectors (ER − SR) and (HR − ER) in three-dimensional space, where (ER − SR) is the vector between the right shoulder and the right elbow, and (HR − ER) is the angle between the right elbow and the right hand, Equation (6).

$$\begin{gathered} Era={\displaystyle \sum }_{i=0}^{M-1}∆{\theta }_{ER}{}_i \\ ∆{\theta }_{E{R}_i}=\left|{\theta }_{ER}{}_{i+1}-{\theta }_{ER}{}_i\right| \\ {\theta }_{ER}{}_i={\cos }^{-1}\left(\frac{E{R}_i-S{R}_i}{\parallel E{R}_i-S{R}_i\parallel }·\frac{H{R}_i-E{R}_i}{\parallel H{R}_i-E{R}_i\parallel }\right) \end{gathered}$$

(6)

• Ela (radian): $∆{\theta }_{E{L}_i}$ is the sum of angular displacement between two frames (i, i + 1) of EL (Elbow Left). ${\theta }_{EL}$ is the angle between the two vectors (EL − SL) and (HL − EL) in three-dimensional space, where (EL − SL) is the vector between the left shoulder and the left elbow, and (HL − EL) is the angle between the left elbow and the left hand, Equation (7).

$$\begin{gathered} Ela={\displaystyle \sum }_{i=0}^{M-1}∆{\theta }_{EL}{}_i \\ ∆{\theta }_{E{L}_i}=\left|{\theta }_{EL}{}_{i+1}-{\theta }_{EL}{}_i\right| \\ {\theta }_{EL}{}_i={\cos }^{-1}\left(\frac{E{L}_i-S{L}_i}{\parallel E{L}_i-S{L}_i\parallel }·\frac{H{L}_i-E{L}_i}{\parallel H{L}_i-E{L}_i\parallel }\right) \end{gathered}$$

(7)

• Kra (radian): $∆{\theta }_{K{R}_i}$ is the sum of angular displacement between two frames (i, i + 1) of KR (Knee Right). ${\theta }_{KR}$ is the angle between the two vectors (KR − PR) and (AR − KR) in three-dimensional space, where (KR − PR) is the vector between the right knee and the right hip, and (AR − KR) is the angle between the right knee and the right ankle, Equation (8).

$$\begin{gathered} Kra={\displaystyle \sum }_{i=0}^{M-1}∆{\theta }_{KR}{}_i \\ ∆{\theta }_{K{R}_i}=\left|{\theta }_{KR}{}_{i+1}-{\theta }_{KR}{}_i\right| \\ {\theta }_{KR}{}_i={\cos }^{-1}\left(\frac{K{R}_i-P{R}_i}{\parallel K{R}_i-P{R}_i\parallel }·\frac{A{R}_i-K{R}_i}{\parallel A{R}_i-K{R}_i\parallel }\right) \end{gathered}$$

(8)

• Kla (radian): $∆{\theta }_{K{L}_i}$ is the sum of angular displacement between two frames (i, i + 1) of KL (Knee Left). ${\theta }_{KL}$ is the angle between the two vectors (KL − PL) and (AL − KL) in three-dimensional space, where (KL − PL) is the vector between the left knee and the left hip, and (AL − KL) is the angle between the left knee and the left ankle, Equation (9).

$$\begin{gathered} Kla={\displaystyle \sum }_{i=0}^{M-1}∆{\theta }_{KL}{}_i \\ ∆{\theta }_{K{L}_i}=\left|{\theta }_{KL}{}_{i+1}-{\theta }_{KL}{}_i\right| \\ {\theta }_{KL}{}_i={\cos }^{-1}\left(\frac{K{L}_i-P{L}_i}{\parallel K{L}_i-P{L}_i\parallel }·\frac{A{L}_i-K{L}_i}{\parallel A{L}_i-K{L}_i\parallel }\right) \end{gathered}$$

(9)

3.6. Data Analysis

As we summarized in the previous section, we performed physiological measurements based on all exergames for each subject. Since this study comprised an experiment in which one participant repeated four types of activity, the analysis model is assumed as follows, Equation (10).

$$y_{ij}=\mu +s_i+{\tau }_j+{ϵ}_{ij}, ϵ\stackrel{\mathrm{iid}}{\sim }N\left(0,{\sigma }^2\right) \left(i=1,2,\dots ,n ;j=1,2, \dots ,p\right)$$

(10)

Here, $y_{ij}$ is the measured data value obtained through the ECG and Kinect sensors observed during the j-th exercise in the i-th participant. We assume that $s_i$ is the effect of the i-th participant and follows a normal distribution with mean 0 and variance ${\sigma }_s^2$. ${\tau }_j$ represents the fixed treatment effect of the j-th exergame. In this paper, we fitted the model in (1) for each of the p = 6 types (Rest, Walk, Tennis, Run, Tennis Competition, Tennis Cooperation) to determine the exercise effect for a single player and that of the gameplay mode. For each p, the observed that vector $y=\left(y_{i1},y_{i2}, \cdots , y_{ip}\right)$ for one participant follows the p-variate normal distribution $\mathrm{N}\left(0,{\mathsf{\Sigma }}_y\right)$. As shown in Equation (11), for p = 6,

$$\begin{gathered} {\mathsf{\Sigma }}_y=\left(\begin{array}{cccccc}\hfill Var\left(y_{i1}\right)\hfill & \hfill Cov\left(y_{i1},y_{i2}\right)\hfill & \hfill Cov\left(y_{i1},y_{i3}\right)\hfill & \hfill Cov\left(y_{i1},y_{i4}\right)\hfill & \hfill Cov\left(y_{i1},y_{i5}\right)\hfill & \hfill Cov\left(y_{i1},y_{i6}\right)\hfill \\ \hfill Cov\left(y_{i1},y_{i2}\right)\hfill & \hfill Var\left(y_{i2}\right)\hfill & \hfill Cov\left(y_{i2},y_{i3}\right)\hfill & \hfill Cov\left(y_{i2},y_{i4}\right)\hfill & \hfill Cov\left(y_{i2},y_{i5}\right)\hfill & \hfill Cov\left(y_{i2},y_{i6}\right)\hfill \\ \hfill Cov\left(y_{i1},y_{i3}\right)\hfill & \hfill Cov\left(y_{i2},y_{i3}\right)\hfill & \hfill Var\left(y_{i3}\right)\hfill & \hfill Cov\left(y_{i3},y_{i4}\right)\hfill & \hfill Cov\left(y_{i3},y_{i5}\right)\hfill & \hfill Cov\left(y_{i3},y_{i6}\right)\hfill \\ \hfill Cov\left(y_{i1},y_{i4}\right)\hfill & \hfill Cov\left(y_{i2},y_{i4}\right)\hfill & \hfill Cov\left(y_{i3},y_{i4}\right)\hfill & \hfill Var\left(y_{i4}\right)\hfill & \hfill Cov\left(y_{i4},y_{i5}\right)\hfill & \hfill Cov\left(y_{i4},y_{i6}\right)\hfill \\ \hfill Cov\left(y_{i1},y_{i5}\right)\hfill & \hfill Cov\left(y_{i2},y_{i5}\right)\hfill & \hfill Cov\left(y_{i3},y_{i5}\right)\hfill & \hfill Cov\left(y_{i4},y_{i5}\right)\hfill & \hfill Var\left(y_{i5}\right)\hfill & \hfill Cov\left(y_{i5},y_{i6}\right)\hfill \\ \hfill Cov\left(y_{i1},y_{i6}\right)\hfill & \hfill Cov\left(y_{i2},y_{i6}\right)\hfill & \hfill Cov\left(y_{i3},y_{i6}\right)\hfill & \hfill Cov\left(y_{i4},y_{i6}\right)\hfill & \hfill Cov\left(y_{i5},y_{i6}\right)\hfill & \hfill Var\left(y_{i6}\right)\hfill \end{array}\right) \\ =\left({\sigma }^2+{\sigma }_s^2\right)\left(\begin{array}{cccccc}1& \rho & \rho & \rho & \rho & \rho \\ \rho & 1& \rho & \rho & \rho & \rho \\ \rho & \rho & 1& \rho & \rho & \rho \\ \rho & \rho & \rho & 1& \rho & \rho \\ \rho & \rho & \rho & \rho & 1& \rho \\ \rho & \rho & \rho & \rho & \rho & 1\end{array}\right), \mathrm{where} \rho =\frac{{\sigma }_s^2}{{\sigma }^2+{\sigma }_s^2} \end{gathered}$$

(11)

The above model’s assumptions allow us to test the differences in the population mean between all treatments. The difference test for the effects of four types of one-person exercise on ECG variables is as follows. With the ECG variables, n = 20 participants were analyzed, except for 4 patients whose average heart rates in the resting state were outside the 60–100 beats per minute range considered the average heart rate of a healthy person, Equation (12).

$$\begin{gathered} H_0:{\tau }_1={\tau }_2={\tau }_3={\tau }_4={\tau }_5={\tau }_6=0, H_1:not H_0 \\ F=\frac{S{S}_{treatment} /5}{S{S}_{error} /19\times 5}\sim N\left(3,19\times 5\right) \end{gathered}$$

(12)

For the Kinect variable, the differences in the effects of 6 exercise games for n = 24 participants are confirmed as follows, Equation (13).

$$\begin{gathered} H_0:{\tau }_1={\tau }_2={\tau }_3={\tau }_4={\tau }_5={\tau }_6=0, H_1:not H_0 \\ F=\frac{S{S}_{treatment} /5}{S{S}_{error} /23\times 5}\sim N\left(3,23\times 5\right) \end{gathered}$$

(13)

If there is no significant difference in the exercise effect, it means that the exercise effect is the same for each exercise. If there was a significant difference in the exercise effect, the difference between the exercise types was confirmed through a post hoc test. The data were analyzed using the rstatix library of the R program, and the Hochberg procedure was applied as a post hoc test technique [27,28].

4. Results

Table 2. Repeated measure ANOVA on ECG data.

	Sum of Squares	df	Mean Squares	F Value	p-Value
Mean_HR
Type	13,015.07	5	2603.01	23.64	<0.001 ***
Error	10,460.05	95	110.11
Max_HR
Type	28,646.78	5	5729.36	37.88	<0.001 ***
Error	14,367.69	95	151.24
SDNN
Type	34,127	5	6825.4	5.36	0.004 **
Error	120,978.9	95	1273.46
RMSSD
Type	21,055.07	5	4211.01	5.21	0.004 **
Error	76,828.18	95	808.72
LF
Type	1833.96	5	366.79	2.03	0.117
Error	17,178.47	95	180.83
HF
Type	1822.58	5	364.52	2.02	0.117
Error	17,126.46	95	180.28
LF/HF ratio
Type	177.81	5	35.56	2.22	0.097
Error	1523.53	95	16.04

** p-value < 0.01 *** p-value < 0.001.

and

Table 3. Repeated measure ANOVA on Kinect data.

	Sum of Squares	df	Mean Squares	F Value	p-Value
SpineBase
Type	31,501.1	5	6300.22	60.77	<0.001 ***
Error	11,923.49	115	103.63
Upper
Type	3,450,891	5	690,178.20	39.47	<0.001 ***
Error	2,010,936	115	17,486.40
Lower
Type	3,086,489.5	5	617,297.90	113.57	<0.001 ***
Error	625,091.8	115	5435.58
Era
Type	1,097,114	5	219,422.80	28.61	<0.001 ***
Error	882,127	115	7670.67
Ela
Type	1,517,320.6	5	303,464.12	47.80	<0.001 ***
Error	730,134.2	115	6348.99
Kra
Type	2,336,336.2	5	467,267.24	203.11	<0.001 ***
Error	264,567.3	115	2300.59
Kla
Type	2,491,751.9	5	498,350.38	230.36	<0.001 ***
Error	248,782.5	115	2163.33

*** p-value < 0.001.

show the results of repeated measures ANOVA performed on ECG and Kinect data for exercise games (Rest, Walk, Tennis, Run, Tennis Competition, and Tennis Cooperation). As shown in Table 2, the Mean HR and Max HR of the ECG measurement variables were statistically significant at the significance level of 0.1%, and SDNN and RMSSD were statistically significant at the significance level of 1%. As shown in Table 3, all Kinect measurement variables (SB, Upper, Lower, Era, Ela, Kra, and Kla) were statistically significant at the significance level of 0.1%. A post hoc test was performed after repeated measure ANOVA.

4.1. Effects of Exercise Type

Table 4. Multiple comparison of effects of exercise types on ECG data.

Variable	Mean Difference						Mean
	Run-Walk	Run- Tennis	Run- Rest	Walk- Tennis	Walk- Rest	Tennis-Rest	Rest
Mean_HR ***	14.2 ***	28.1 ***	28.1 ***	13.9 ***	14.0 ***	0.1	94.4
Max_HR ***	29.8 ***	43.2 ***	45.9 ***	13.4 ***	16.1 ***	2.7	107.1
SDNN **	48.3 **	39.9 *	49.1 **	−8.4	0.8	9.2	28.3
RMSSD **	37.2 **	31.7 *	38.9 **	−5.5	1.7	7.2	20.8
LF	−9.8	−11.4	−9.8	−1.6	0.0	1.6	74.2
HF	9.8	11.4	9.8	1.6	0.0	−1.6	25.8
LF/HF ratio	−2.2	−3.2	−3.9	−1.0	−1.8	−0.7	6.6

* p-value < 0.05 ** p-value < 0.01 *** p-value < 0.001.

shows the post-test results and pairwise comparisons of the effects of exercise types (via Hochberg’s multiple post hoc procedure) in exergaming on ECG data. The mean value of Rest is presented as the reference value. Mean HR and Max HR showed significant differences in terms of effects between exercise types, except for the difference between Tennis and Rest. Run was statistically significantly different from other exergames in terms of SDNN and RMSSD. In terms of SDNN and RMSSD, Tennis gave larger values than Walk, but the difference between the two was not significant. In terms of LF and HF, there were significant differences between Run and Walk, as well as between Run and Tennis. In terms of the LF/HF ratio, Run was statistically significantly different from all other exergames.

Table 5. Multiple comparison of effects of exercise types on Kinect data.

Variable	Mean Difference						Mean
	Run-Walk	Run- Tennis	Run- Rest	Walk- Tennis	Walk- Rest	Tennis- Rest	Rest
SpineBase ***	28.2 ***	36.0 ***	48.2 ***	7.8 **	20.0 ***	12.2 ***	2.9
Upper ***	256.3 ***	259.2 ***	529.2 ***	2.9	272.9 ***	270.0 ***	82.8
Lower ***	187.1 ***	358.7 ***	441.4 ***	171.5 ***	254.2 ***	82.7 ***	25.1
Era ***	114.6 ***	57.2	280.9 ***	−57.4	166.3 ***	223.6 ***	56.6
Ela ***	118.1 ***	238.1 ***	320.0 ***	120.0 ***	201.9 ***	81.9 ***	51.3
Kra ***	130.9 ***	314.1 ***	348.2 ***	183.2 ***	217.3 ***	34. 1 ***	9.2
Kla ***	121.9 ***	323.8 ***	353.4 ***	201.9 ***	231.6 ***	29.7 ***	9.6

** p-value < 0.01 *** p-value < 0.001.

shows the post-test results and pairwise mean comparisons of the effects of exercise types (performed via Hochberg’s multiple post hoc procedure) in exergaming on Kinect data. Interestingly, there was a significant difference between Tennis and Rest in all Kinect measurements, unlike the ECG measurements. In the case of Tennis, which primarily makes use of the right arm, there was no significant difference between Walk and Tennis in terms of Upper and Era, and there was no significant difference between Run and Tennis in terms of Era. Overall, the exercise effect of exergaming using ECG and Kinect was confirmed in the order of the high-intensity Run, the moderate-intensity Walk, the low-intensity Tennis, and then Rest. However, the effects of exercises showing significant differences in the Kinect data were not represented in the ECG data.

Figure 5 shows a bar chart of average ECG and Kinect measurements by single-player exercise types (Rest, Walk, Tennis, and Run). Run, which is a high-intensity exercise, yielded higher SDNN and RMSSD, which appeared as more irregular heart rate variability compared to other exergames. Additionally, Run had the highest Mean HR and Max HR, followed by Walk. In terms of LF, Run yielded the lowest value, since it decreased due to a lack of energy and fatigue. The LF values of Walk, Tennis and Rest were similar, indicating that Run had the highest exercise intensity.

However, the effects of Tennis and Rest were not significantly different in terms of all ECG measurements. Moreover, Kinect’s lower body movement (Lower) and the amounts of changes in knee angles (Kra and Kla) during Tennis were closer to those of Rest than Walk, since subjects could play tennis with the controller, and hence did not move their whole body. On the contrary, the degree of movements in the upper body (Upper) and the degree of change in the angle of the right elbow (Era) during Tennis were much higher than during Rest.

4.2. Effects of Gameplay Mode

Table 6. Multiple comparison of effects of gameplay modes on ECG data.

Variable	Mean Difference			Mean
	Competition- Single	Competition- Cooperation	Single- Cooperation	Single
Mean_HR ***	3.0	2.8	−0.2	94.5
Max_HR ***	9.0 **	6.7 *	−2.3	109.8
SDNN **	15.7	6.6	−9.1	37.5
RMSSD **	13.1	6.2	−6.9	27.9
LF	−2.7	−2.6	0.1	75.7
HF	2.7	2.6	−0.12	24.2
LF/HF ratio	−0.8	0.2	1.0	5.9

* p-value < 0.05 ** p-value < 0.01 *** p-value < 0.001.

shows the post-test results and the pairwise comparisons of the effects of gameplay modes (by Hochberg’s multiple post hoc procedure) in exergaming on ECG data. The mean value of Single is presented as the reference. Table 6 indicates that Max HR (among the four ECG measurements that had a significant difference under 5%) was significantly different between gameplay modes. Max HR was highest in the Competition mode, followed by the Cooperation mode and then the Single mode, but the difference between the Single and Cooperation modes was not significant.

Table 7. Multiple comparison of effects of gameplay modes on Kinect data.

Variable	Mean Difference			Mean
	Competition- Single	Competition- Cooperation	Single- Cooperation	Single
SB ***	4.1	4.1 **	0.0	15.1
Upper ***	31.8	85.0 ***	53.2 ***	352.8
Lower ***	23.1	23.5	0.4	107.8
Era ***	−13.5	49.9 ***	63.4 ***	280.2
Ela ***	26.2 *	26.2 *	0.0	133.2
Kra ***	5.4	8.5	3.1	43.3
Kla ***	9.2	8.1	−1.2	39.2

* p-value < 0.05 ** p-value < 0.01 *** p-value < 0.001.

shows the post-test results and the pairwise comparisons of the effects of gameplay modes (by Hochberg’s multiple post hoc procedure) in exergaming on Kinect data. Table 7 indicates that SB, Upper, Era, and Ela (among all the Kinect measurements that had a significant difference under 0.1%) showed significant differences between gameplay modes. In particular, the amount of upper body movement (Upper) and the amount of change in the angle of the right elbow (Era) showed significant differences in the Competition and Cooperation modes, as well as in the Single and Cooperation modes. Additionally, the amount of change in the angle of the left elbow (Ela) showed a significant difference between the Competition and Single modes, as well as between the Competition and Cooperation modes.

Figure 6 shows the bar chart of average ECG and Kinect measurements of each gameplay mode (Single, Competition, and Cooperation). The Competition mode yielded higher Max HR, SDNN, and RMSSD than the other gameplay modes. In addition, SDNN and RMSSD were higher in the Cooperation mode than in the Single mode, but the effects were similar in the LF, HF, and LF/HF ratio.

The Kinect results show that the subjects moved the least in the Cooperative mode. In the Cooperation mode, any player can hit the ball, i.e., the play order, other than the serve, had no effect. This means that if one player leads, the activity of the partner may be less compared to during the Competition mode. In addition, due to the nature of the Mario Ace Tennis game, where the movement of the arms is the main focus, the amount of body movement and angle change of the subjects’ lower body was smaller than that of the upper body. Similarly, the amount of right elbow angle change (Era) was greater than that of left elbow angle change (Ela).

While the subjects played side by side under the Competition and Cooperative modes, there were no space restrictions on arm movement in the Single mode. Therefore, the amount of change in the angle of the right elbow (Era) was slightly larger in the Single mode. The subjects moved a lot overall in the Competition mode, as compared to other modes. The subjects’ physical activity was higher in the Competition mode (competing with a real person) than in the Single mode (competing against a virtual partner) overall, except for the angle of the right elbow (Era).

When comparing gameplay modes, the overall effects of two-player competition were higher than those of the single-player game. Similar to the results yielded by the exercise types, the ECG and Kinect results occurred in the same order, according to the exercise intensity in the gameplay mode. That is, when the ECG measurements showed differences, the Kinect results showed similar differences. In addition, the maximum heart rate and the angle of the left elbow (Ela) were significantly higher in the Competition mode (competing with a real person) than in the Single mode (competing with a virtual partner).

4.3. Subjective Evaluation

A post-test questionnaire survey was given to the subjects on their preferred exergames and the exercise intensity felt during the exergaming. In total, 21 out of 24 participants were pleased with Tennis, 2 with Run, and 1 with Walk. In total, 23 out of 24 participants answered that Run was the game with the highest felt intensity, and only 1 participant responded that there was no exergame they perceived as high-intensity. All the participants answered that Run had the highest exercise effect. It was shown that the exercise intensity of Run was high not only in terms of the ECG and Kinect measurements, but also according to the subjective evaluation.

We also asked them about their preferred exercises among the gameplay modes of playing Tennis alone with virtual partners, cooperating with participants in a 2:2 ratio against virtual partners, and competing 1:1 with real partners. The results reveal that 20 participants preferred the multiplayer gameplay mode to the single-player mode, of which 50% each were cooperative and competitive. As such, in terms of the exercise effect felt by the subjects playing the exergame, it seems that they felt more pleasure when playing with a real person than when playing alone with a virtual partner.

5. Discussion

In this experiment, there was a significant difference between the exercises of single-player type, such as Rest, Walk, and Run, in terms of the ECG results. In addition, most subjects showed increased mean HR, max HR, SDNN and RMSSD during Run exergaming, which seemed to be a high-intensity exercise compared to other exergames. However, SDNN and RMSSD have been said to be more accurate when measured for 24 h, rather than using short-term measurements of less than 5 min [29]. Based on 24-h monitoring, an SDNN value of less than 50 milliseconds was detrimental to health, and a value greater than 100 milliseconds was considered healthy.

On the other hand, the mean HR was not statistically significantly different between Tennis and Rest. Due to the characteristics of Nintendo Tennis, the user can play tennis with little physical movements (i.e., standing in the same position), so the amount of user body movement distance is less than that in other exergames, except for the amount of angle movement of the right arm. Due to this effect, the expected medium-intensity exercise effect (average heart rate 40–60% of maximum heart rate by age) compared to actual tennis did not appear in the ECG results.

Unlike Graves et al.’s [6], our experiment results reveal no significant difference between the Rest and Tennis exergames in terms of mean HR. They observed users playing 15-min Nintendo Wii Sports Tennis, while we observed users performing 3-min Nintendo Switch Tennis. We need to determine whether we would derive different results if the subjects played Nintendo Tennis for 15 min. Graves at el. also showed that there was no difference between the effects of Nintendo Wii Sports Bowling and Tennis on mean HR. In terms of actual physical activity, tennis is known as a high-intensity exercise, while bowling is a low-intensity exercise. Similar to this, our study results also reveal that the Tennis exergame was a low-intensity exercise.

In our experiment results, it is difficult to confirm a difference in heart rate between Tennis and Rest, and we think that this might be due to the 3-min time period. We tried to improve the accuracy through the repeated measurement of the exercise in order to compensate for the short exergaming time. However, there was a significant difference between Tennis and Rest in terms of the amount of physical activity measured by Kinect ($p<0.01$). This seems to suggest that the effect of ECG does not increase even if the amount of Kinect movement increases. Therefore, it would be useful to see ECG and Kinect data together in order to compare the effects of exercise between exergames.

In addition, our study’s results show that there was a significant difference between the gameplay modes. We compared the exercise effects of matches with a virtual partner and with a real partner, and the effect when using the competitive mode and the cooperative mode of Nintendo Wii Sports Tennis. As a result of comparing the Single mode (a single subject playing tennis against a virtual partner), the Competition mode (two subjects playing tennis against each other), and the Cooperation mode (two subjects playing tennis together against virtual partners), we see that the exercise effect was greater in the match against a real partner. The average max HR was 109.8 bpm for Single Tennis, 112.1 bpm for Tennis Cooperation, and 118.8 bpm for Tennis Competition, which shows a significant difference at the significance level of 1%.

The amounts of Kinect upper-body movement (SB, SL, SR, EL, ER, HL, and HR) appeared much higher and more statistically significant in Tennis Competition ($p<0.01$). On the other hand, the amount of change in the angle of the right elbow (although there was no significant difference) was high in Single Tennis. However, this seems to be a result of two players sharing the same space in Tennis Competition and Tennis Cooperation. This may show that the synergistic effect is improved when playing together with a real partner compared to exercise with a virtual partner.

Similar to Snyder et al.’s study [7], our study’s results show that the effect of the competitive mode was greater than that of the cooperative mode or the single mode. However, the difference was that we used ECG and Kinect measurements, while they measured the exercise intensity (as an index of exercise effect), using subjective measurement to quantify the degree of intensity the participants themselves felt through a questionnaire. In our study, the exercise effect was used as an index to evaluate the exercise intensity through ECG and the activity amount measured by Kinect data. In addition, our subjective evaluation results show that most subjects preferred the mode of playing together over the single-player mode. The multiplayer mode seems to induce social motivation and promote exergame engagement.

6. Conclusions and Future Work

Recently, as the public’s interest in exercise is increasing with the desire for a healthy life, various exercise games, such as Xbox Kinect dancing, Nintendo Wii Fit and Wii Sport, have become very popular. The popularity of these exercise games has made it possible for people of all ages and genders to enjoy exercise using their entire body, so they can experience fun and health at the same time. In the past, many studies have been conducted on the effects of exercise using exergaming, and the results have shown that there was a positive effect. However, they measured the effects of exercise primarily using questionnaires, ECG measurements, and accelerometer sensors to check the movement of specific joints. No studies have been conducted using ECG and Kinect together to determine the effect of exercise on users when engaging in exergames in the form of individual or team play.

In this study, therefore, we investigated the effects of exercise type and gameplay mode using Nintendo Switch exergaming, and the physical activities of subjects were measured using ECG and Kinect. The types of exercise chosen in this study were low-intensity (Walk), medium-intensity (Tennis), and high-intensity (Run) exergames, as compared to a no-intensity activity (Rest). Walk and Run were played in a single gameplay mode, while Tennis offered various gameplay modes with opposing players. The gameplay modes were playing tennis in a single match game with a virtual partner (Single) or a real person (Competition), as well as playing tennis in a double match game with two real players (Cooperation).

The intensity of the exercise was different even when the same exercise was performed in the forms of exergames and physical exercise. Tennis is generally known as a high-intensity exercise, but users had a lower heart rate and more stable SDNN and RMSSD values when playing the Tennis exergame compared to the Walk exergame. Additionally, it was found that the exercise effect of exergaming was more accurately measured with Kinect and ECG together. In the Tennis exergame, the effect of exercise could not be seen with ECG, but the amount of physical activity was captured with Kinect. Interestingly, the exercise effect of exergaming was higher when playing against a human partner (Competition) than a virtual partner (Single), with a significant difference in maximum heart rate and the amount of change in the angle of the left elbow. In the two-player tennis exergame, the maximum heart rate and the user’s upper body activity were significantly higher in the Competitive mode than in the Cooperative mode.

Overall, no significant differences between the different kinds of exergaming were seen in the electrocardiogram (ECG) measurements, despite the high physical activity measured by Kinect. In other words, the user can increase their amount of physical activity through exergaming without inducing a physical burden. Hence, exergaming can be used for improving specific joint movements without raising the heart rate. In addition, the study’s participants preferred two-player exergames, and the effects of exercise with a human partner were better than those with a virtual partner. This implies that more exergames are required that can be played with family or friends, rather than alone, in order to increase the enjoyment and effectiveness of exercise.

In this study, the exercise effect of exergaming was measured for a minimum time of 3 min, but it is necessary to examine the effects of exercise for a longer period of time. Therefore, we will need to evaluate the effects on exercise of short-term and long-term exergaming. If short-term exergames can have similar exercise effects to actual physical activity, weight loss can be expected as an effect of exergames in the long term. Moreover, additional research is needed to compare the effects of exercise through the design of different gameplay modes, such as competition and cooperation, in various kinds of exergames. In the future, we will use multiple Kinect systems to measure human skeletal movements, to avoid skeletal noise and to improve the accuracy of human motion measurement.