Use of Both Eyes-Open and Eyes-Closed Resting States May Yield a More Robust Predictor of Motor Imagery BCI Performance

Abstract

Motor-imagery brain-computer interface (MI-BCI) is a technique that manipulates external machines using brain activities, and is highly useful to amyotrophic lateral sclerosis patients who cannot move their limbs. However, it is reported that approximately 15–30% of users cannot modulate their brain signals, which results in the inability to operate motor imagery BCI systems. Thus, advance prediction of BCI performance has drawn researchers’ attention, and some predictors have been proposed using the alpha band’s power, as well as other spectral bands’ powers, or spectral entropy from resting state electroencephalography (EEG). However, these predictors rely on a single state alone, such as the eyes-closed or eyes-open state; thus, they may often be less stable or unable to explain inter-/intra-subject variability. In this work, a modified predictor of MI-BCI performance that considered both brain states (eyes-open and eyes-closed resting states) was investigated with 41 online MI-BCI session datasets acquired from 15 subjects. The results showed that our proposed predictor and online MI-BCI classification accuracy were positively and highly significantly correlated (r = 0.71, p < $0.1\times {10}^{-7}$), which indicates that the use of multiple brain states may yield a more robust predictor than the use of a single state alone.

1. Introduction

Brain-computer interface (BCI) is among the techniques that allows a person to control external devices using his/her brain activities (encoded information of intention) without limb movement. Well-known BCI paradigms that use electroencephalography (EEG) include visual or auditory oddball tasks and motor imagery (MI) tasks [1,2]. An MI task is among the active paradigms that reflect the user’s intention directly; however, it has been reported that approximately 15–30% of users have difficulty imagining their limb movement, and thus have little ability to modulate controllable brain activity [3,4], such that a MI-BCI system is unsuitable for them. In the literature [3,4], such users are referred to as BCI-illiterate. However, we note that certain ethical issues accompany BCI-illiteracy. According to the literature [5], Thompson addressed that a BCI that does not work for its users simply does not work. Reflecting Thompson’s opinion, BCI’s user-specific suitability, rather than BCI-illiteracy, has been addressed throughout this work. To enhance MI-BCI ability, many researchers have used neurofeedback [6,7,8] and sensory stimulation training [9,10] to help guide users. These strategies included evoked kinesthetic experiences and produced discriminative brain patterns among different classes. Although MI-BCI training methods have been reported, it is difficult to conclude whether a MI-BCI system is suitable for a user without conducting a very time-consuming MI-BCI task. Thus, strategies to determine MI-BCI systems’ user-specific suitability are very useful in BCI research.

It is known well that during MI, the sensorimotor µ-rhythm power (8–13 Hz) decreases noticeably relative to the baseline power for some time, and then rebounds and increases over the baseline power. These phenomena are referred to as event-related desynchronization (ERD) [11] and event-related synchronization (ERS) [12]. For most users for whom MI-BCI is less suitable, these ERD/ERS definitely appear to be too weak to observe during MI. Generally, because a BCI task requires significant time and effort, the ability to predict BCI’s user-specific suitability before a BCI task is conducted is highly important with respect to cost-effectiveness. Therefore, many researchers have investigated several factors related to MI-BCI performance, such as brain structure [13,14], personal characteristics [15], motivation [16], mindfulness [17], kinesthetic MI scores [18], and users’ conviction that they can perform BCI [19]. However, these factors are believed to be associated less directly with MI-BCI performance.

Therefore, direct prediction metrics have been reported using brain signals during the resting state [4,20,21,22,23]. Blankertz et al. designed a SMR predictor using resting state EEG with eyes-open [4], and obtained the maximum differences at C3/C4 between the actual power spectrum density and the value modeled by fitting curves of the 1/f noise spectrum. Then, the mean value was considered the potential ability to conduct a MI task, which is referred to as the SMR predictor. They achieved a significant correlation (r = 0.53) between MI-BCI performance and the SMR predictor using the eyes-open resting state alone in 80 subjects. In addition, a SMR predictor using pooled eyes-open and eyes-closed data yielded a significant correlation (r = 0.58 [20]), and averaging the SMR predictor with eyes-open and that with eyes-closed yielded a correlation of 0.54 [21]. Zhang et al. studied another predictor of MI-BCI performance using the spectral entropy of resting-state EEG [22], in which the spectral entropy was estimated by the normalized power spectral density (PSD) from 0.5–14 Hz. They observed that the eyes-closed state yielded better prediction than did the eyes-open state; specifically, the C3 channel of the eyes-closed resting state showed the highest correlation (r = 0.65) in 66 sessions acquired from 40 subjects. In addition, Ahn et al. explored the relations between the spectral band powers of eyes-open resting state EEG and MI-BCI performance, and found a significant association between low theta and high alpha powers in MI-BCI performance [23]. With this finding, the potential performance factor (PP factor) was proposed as a simple predictor of MI-BCI performance, and the correlation between performance and the PP factor was estimated to be r = 0.59 with 52 subjects.

However, to date, these existing predictors have several limitations. The occipital visual alpha rhythm may distort the SMR predictor using the eyes-open resting state easily [4], and its curve-fitting method depends heavily on various parameters that are difficult to determine even if both resting state data are used. Spectral entropy measures uncertainty in a predefined frequency range and spectrums distributed evenly over frequencies may yield a high value, while a large spectral power at a single frequency bin may yield a low value [24,25]. Thus, some specific frequency bands may be influenced excessively when estimating spectral entropy, and potentially result in great fluctuation in entropy attributable to emotional state or noise information. In addition, spectral entropy depends heavily on the frequency range and bin size, which are not easy to determine optimally. The PP factor may also depend on the emotional/cognitive state or noise, although it can be estimated far more easily by a very simple calculation than can SMR and spectral entropy predictors.

To the best of our knowledge, most predictors derived directly from resting state EEG have been explored with the resting state alone, such as either eyes-open or eyes-closed. Subjects’ innate state is known to vary over time; thus, notable variability in their resting states that reflect various circumstances, such as emotion, degree of fatigue, background noise, and so on, may influence such predictors strongly [22,23]. Accordingly, predictors may vary greatly over datasets and yield less stable prediction. To overcome this inherent issue to some extent, it is natural to consider as much information as possible to estimate a predictor. This motivated us to investigate whether both resting states (eyes-open and eyes-closed) may be a more robust predictor than a single resting state alone. Thus, in this work, we proposed a predictor that uses both resting states and evaluated it with online MI-BCI data.

The paper is organized as follows. The MI-BCI experiment we used in this work and its analysis are described in detail in Section 2. Various predictors using the eyes-open or eyes-closed state alone, as well as one obtained from both states combined, are compared in Section 3. Finally, beta oscillation for BCI prediction, inter-session variability, and certain related issues are discussed in Section 4.

2. Materials and Methods

2.1. Data Description

Fifteen healthy, right-handed subjects (3 females, age 25.6 ± 2.4 years) participated in a multiple-session (1–5 sessions) MI-BCI experiment, and data were collected during a total of 41 sessions. All subjects were informed of the experiment’s purpose and process, and then signed a consent form. The institutional review board (IRB) of Gwangju Institute of Science and Technology (No. 20130527-HR-02) approved this experiment.

Brain signals were acquired from 64-channel EEG system (Biosemi ActiveTwo system) that were placed on the scalp according to the international 10-10 system. Simultaneously, 12 electromyogram (EMG) channels were attached to their arms and legs to determine whether the subjects could control the signals through their limb movements. All EEG and EMG signals were digitized at 512 Hz.

The subjects sat in a comfortable chair, relaxed their arms and legs, and were instructed to conduct several tasks. First, they focused their eyes on fixation markers for 1 min, then closed their eyes for 1 min. Thereafter, the subjects conducted two MI tasks, offline and online. In the offline MI task, a red dot appeared for 2 s to signal them to prepare to imagine limb movement. Then, a target gray bar appeared on the right, left, or bottom; according to its position, they imagined their right hand, left hand, or feet moving for 2 s. Finally, the target disappeared and the subjects stopped imagining the movement. This procedure constituted one trial for the offline task, and 60 trials were conducted per class. Each subject’s offline tasks, which consisted of three classes (left hand (L), right (R) hand, and feet (F)), and three pairs of classes (left hand–right hand, left hand–feet, right hand–feet) were evaluated with binary classification. Among the three pairs, the pair of classes that yielded the best classification performance was chosen to use in the online MI task. We note that this pair may be subject-specific. In the online task, the subjects received feedback from a 2 s EEG signal window after they began to imagine limb movement. Each subject performed the online MI task in 75 trials per class [26].

2.2. Motor Imagery Task Analysis

The MI-BCI data were baseline corrected and band-pass filtered from 8–30 Hz using Fast Fourier Transform (FFT), which is related to motor movements. They were down-sampled by 4 and a time window of 0–2 s after imagination onset was used for analysis. Feature extraction (invariant common spatio-spectral pattern [26,27]) and binary classification (Fisher’s linear discriminant analysis (FLDA) [28]) of these data were applied to each pair of classes. Thus, three classifiers were generated with 3-class MI. 10-fold cross-validation was performed for each classifier, with a 70% training set and 30% test set, and a total of 120 iterations were computed; thus, classification accuracy was averaged over 120 iterations. The pair of classes that yielded the best classification performance was used for the online task, and the online task data were analyzed with the same procedure, but without cross-validation. For all online sessions, we categorized three performance groups according to the subjects’ online classification performance, as follows:

• High-performance group (classification accuracy ≥ 70%)
• Middle-performance group (60% ≤ classification accuracy < 70%)
• Low-performance group (classification accuracy < 60%).

We focused primarily on the high- and low-performance groups in our analysis to extract features that discriminated between the two.

2.3. Resting State Analysis

The resting state EEG data (eyes-open and eyes-closed) were band-pass filtered from 1–50 Hz using FFT, and a time window of 30 s (10–40 s after onset) among the data that were collected for 1 min were used, because strong eyeball movement noise was observed frequently early or late in the trials. The spectral band powers of theta (4–8 Hz), alpha (8–13 Hz), beta (13–30 Hz), and low gamma (30–50 Hz) were obtained for each of the resting states, and normalized by an entire spectrum’s power (4–50 Hz) to reduce inter-individual power amplitude variability. This is referred to as the relative power level (RPL). Of 64 channels, two on the central area associated with the primary motor cortex were considered for analysis because the motor area during resting states is believed to be related directly to MI performance. According to the pair of classes chosen for the online task (addressed in Section 2.1 and Section 2.2), two channels were selected among the C3 (associated with the right hand), C4 (left hand), or Cz (feet) channels. Thus, the C3 and C4 channels were chosen for a pair of the left and right hands, C3 and Cz channels for a pair of the right hand and feet, and C4 and Cz channels for a pair of the left hand and feet. Lastly, we explored the distribution of the RPLs for each band and performance group, and the high- and low-performance groups were compared to determine the statistical significance of their difference with an unpaired Student’s t-test.

3. Results

3.1. Online BCI Performance

The online BCI data (41 sessions) were collected from 15 subjects and online BCI classification performance for each session was estimated in the way described in Section 2.2. Detailed information on each session is tabulated in

Table 1. Details of online BCI data and BCI classification accuracy.

Subject	Session	Class	Online Accuracy	Subject	Session	Class	Online Accuracy
S1	1	RF	47.0	S7	1	LF	57.0
	2	RF	57.0		2	RF	53.0
S2	1	LF	79.0	S8	1	LR	56.0
	2	LR	82.0		2	RF	52.0
	3	LR	65.0	S9	1	LF	90.0
	4	LR	93.3		2	LF	71.3
	5	LR	88.7	S10	1	RF	70.0
S3	1	RF	81.0	S11	1	RF	67.3
	2	LF	87.0		2	RF	58.7
	3	RF	87.0	S12	1	LF	75.3
	4	RF	86.0		2	LF	82.7
	5	LF	92.0	S13	1	LF	88.7
S4	1	RF	89.0		2	LF	94.0
	2	RF	78.0	S14	1	LF	90.0
	3	RF	51.0		2	LF	61.3
	4	LF	78.7	S15	1	RF	91.3
	5	LF	54.0		2	RF	44.0
S5	1	LF	49.0
	2	RF	45.0
S6	1	RF	60.0
	2	LF	92.0
	3	LF	97.0
	4	LF	98.7
	5	LF	99.3

. As Figure 1 illustrates, the subjects’ accuracy in BCI classification performance ranged from 44.0% to 99.3%. As shown in Figure 1, we categorized 25 of 41 sessions as the high-performance group, as they demonstrated high classification accuracy (≥70%), and 12 sessions were categorized as the low-performance group (<60% classification accuracy). We note that a classification accuracy of 60% or less does not differ significantly from the chance level, and thus, we may say that this performance indicates the BCI system’s inadequacy.

3.2. BCI Performance Predictors Using Eyes-Open or Eyes-Closed Resting State Alone

Next, we investigated the high- and low-performance groups’ spectral power distributions. To reduce inter-subject variability, we considered the relative spectral power (defined by the RPL) in this analysis. Figure 2a illustrates the data distribution and box plots of the RPL values of the eyes-open state over four spectral bands (theta, alpha, beta, gamma). The distributions of the high- and low-performance groups’ RPL values were also compared. We note that the RPL is the mean of three channels (C3/C4/Cz) located near the motor cortex. From this investigation, we observed the following:

• The high-performance group’s alpha power was significantly higher than that of the low-performance group (p < 0.01).
• The high-performance group’s beta power was significantly lower than that of the low-performance group (p < 0.01).
• The theta powers did not differ significantly between the two groups (p > 0.1). However, except for two data points, the theta powers in the high-performance group had a significantly lower distribution than did those in the low-performance group (p < 0.05).
• The gamma powers did not differ significantly between the two groups (p > 0.1). However, the median of the gamma powers in the high-performance group was slightly lower than that in the low-performance group.

Based on these observations, three types of simple MI-BCI predictors (EO predictors) for the eyes-open resting state can be proposed in a way similar to Ahn et al.’s approach [23], as follows:

• Combination of two significant spectral bands’ (alpha and beta) powers

  $${EO predictor}_{2B}=\alpha /\beta$$

  (1)
• Combination of three spectral bands’ (theta, alpha, and beta) powers

  $${EO predictor}_{3B}=\alpha /\left(\theta +\beta \right)$$

  (2)
• Combination of all four spectral bands’ (theta, alpha, beta, and gamma) powers

  $${EO predictor}_{4B}=\alpha /\left(\theta +\beta +\gamma \right)$$

  (3)

The numerators in the predictors above consist only of the alpha band power that is correlated positively with online BCI performance, and the denominators consist of the spectral band powers that are correlated negatively with online performance. We note that these EO predictors are forms of modified PP factors. We conducted a correlation analysis of the online BCI classification accuracy and the EO predictors. The EO predictor that used the three spectral bands yielded the strongest positive correlation with online BCI performance, after five outliers (blank dot, 90% confidence interval) were excluded (r = 0.67, p < $0.1\times {10}^{-4}$), as shown in Figure 2b, while the EO predictors with two or four bands yielded relatively low correlations (r = 0.56, p < $0.1\times {10}^{-2}$ and r = 0.54, p < $0.1\times {10}^{-2}$, respectively). From these results, we found that an EO predictor using Equation (2) that combines three spectral bands (theta, alpha, beta) produces the best results.

Similarly, we performed the same investigation for the eyes-closed resting state, and the RPL distributions and correlation analysis are illustrated in Figure 3. Our findings from these observations were as follows:

• The high-performance group’s alpha power was higher than that of the low-performance group, but was only mildly significant (p < 0.1).
• The high-performance group’s beta power was significantly lower than that of the low-performance group (p < 0.05).
• There was no significant difference in the theta and gamma powers between the low- and high-performance groups (p > 0.1). However, theta’s median in the high-performance group was slightly higher than that in the low-performance group, while gamma’s median was slightly lower.

Except for theta power, the alpha, beta, and gamma powers in the eyes-closed resting state behaved quite similarly to those in the eyes-open resting state, although the significance of alpha differed somewhat. Similar to the derivation of the EO predictor, from these observations, the EC predictors for the eyes-closed state were proposed, as follows:

• The combination of two significant or moderately significant spectral bands’ (alpha and beta) powers

  $${EO predictor}_{2B}=\alpha /\beta$$

  (4)
• The combination of three spectral bands’ (theta, alpha and beta) powers

  $${EO predictor}_{3B}=\left(\theta +\alpha \right)/\mathsf{\beta }$$

  (5)
• The combination of all four spectral bands’ (theta, alpha, beta and gamma) powers

  $${EO predictor}_{4B}=\left(\theta +\alpha \right)/\left(\beta +\gamma \right)$$

  (6)

A regression analysis was conducted between the EC predictors and online BCI performance, and after three outliers were excluded (at the 90% confidence interval), the predictor that used two spectral bands yielded the strongest positive correlation (r = 0.59, p < $0.1\times {10}^{-2}$). In addition, the EC predictors with three or four spectral bands had relatively low correlations (r = 0.43, p < $0.1\times {10}^{-1}$ and r = 0.45, p < $0.1\times {10}^{-1}$, respectively). From these results, we found that the EC predictor using Equation (4) that combines two spectral bands (alpha and beta) is preferable.

3.3. BCI Predictor Using both Eyes-Open and Eyes-Closed Resting States

Rather than using a single resting state alone (either eyes-open or eyes-closed), both resting states may be used to propose a BCI predictor. Based on all of the observations in Section 3.2, the following resting state predictor (RSP) can be proposed as a combination of EO and EC predictors:

$$\mathit{RSP}=\left({\alpha }_{\mathit{eo}}+{\alpha }_{\mathit{ec}}\right)/\left({\theta }_{\mathit{eo}}+{\beta }_{\mathit{eo}}+{\beta }_{\mathit{ec}}\right)$$

(7)

Here, the subscripts ‘ec’ and ‘eo’ represent the eyes-closed and eyes-open states, respectively.

We note that this proposed RSP considered the alpha and beta powers of both resting states and the theta band of the eyes-open state. We conducted a regression analysis between the RSP value and online BCI classification accuracy with this RSP, and found that the RSP and online BCI performance were correlated strongly (r = 0.71, p < $0.1\times {10}^{-7}$), as Figure 4a shows. A permutation test with 5000 iterations was conducted to test whether the correlation was significant, and it was found to be quite significant for our proposed predictor (p < $0.1\times {10}^{-2}$). For comparison, we computed the PP factors in Ahn et al. [23] with those in our dataset, which yielded a correlation with BCI classification accuracy of r = 0.48 (p < $0.1\times {10}^{-1}$), as presented in Figure 4b. This correlation between the PP factors and online BCI performance was significant (p < $0.05$ from the permutation test), but lower than that reported in Ahn et al.’s study (r = 0.59) [23]. This result suggests that the PP factors may depend on the dataset because of variability in a single state.

In addition, offline data were computed with regression models in the same way. The data distributions were similar to those of the online data, and the correlation between the RSP and PP factors were r = 0.58 (p < $0.1\times {10}^{-3}$) and r = 0.42 (p < $0.1\times {10}^{-1}$), respectively. Compared to the online data, the values were slightly smaller because there were fewer session data with lower classification accuracy in the offline data. Nevertheless, our proposed predictor yielded higher correlations than the PP factor.

4. Discussion

4.1. Relation between Spectral Powers and MI-BCI Performance

The sensorimotor µ-rhythm and beta rhythm are related closely to MI because the alpha and beta powers are attenuated to a greater extent during motor imagination than during the resting state, and are representative features of the MI paradigm [11,12,29,30]. In addition, the alpha rhythm is associated with both mental effort and relaxation [31]. Accordingly, the alpha potential is generally considered to be the core band power to predict performance [4,22,23]. In this work as well, we found that the high-performance group demonstrated relatively higher alpha power distributions in both the eyes-closed and eyes-open states.

Some studies have reported that theta oscillations were related to sensorimotor integration or haptic information recall [32,33] and conscious awareness [34], and thus, MI-BCI performance may reflect this. For example, Ahn et al. reported a potential negative association between theta power and MI-BCI performance [23]. In our dataset, we observed a slightly negative, insignificant correlation between theta power in the eyes-open resting state and BCI performance, which seems somewhat relevant to Ahn et al.’s [23] result. However, we observed no association between theta power and MI-BCI performance in the eyes-closed resting state, although the median of theta power in the high-performance group was slightly higher than that in the low-performance group. The eyes-closed state demonstrated a notably higher alpha power than did the eyes-open state; thus, it is believed that the other bands’ relative powers may be reduced, and result in less difference in the theta RPL between the high- and low-performance groups.

According to the literature [35,36,37,38], gamma rhythm is significantly related to the frontal-parietal network, which is associated with the mirror neuron system (MNS). The MNS recalls previous motor experiences, i.e., gamma power is related to movement memory during motor imagination or motor movement observation. However, in this work, we did not observe any notable correlation between gamma power and MI-BCI performance. We analyzed only three channels in the central region, and thus, gamma oscillation’s influence in certain areas may be less than that in the frontal-parietal network.

4.2. Using Beta Oscillations to Predict BCI

The PP factor was proposed to be a combination of theta, alpha, beta, and gamma powers in the eyes-open state. In particular, theta and alpha powers predict MI-BCI performance significantly, while beta and gamma powers have only a marginal ability to do so. However, in our work, beta power affected MI-BCI performance significantly, and was correlated with it negatively in the eyes-open state, while it was correlated with performance positively in the eyes-closed state. To investigate beta power’s effect, we explored three cases in our dataset:

Case 1: beta power is in the numerator (PP factor; Figure 4b)

$$\mathrm{PP} \mathrm{factor}=\left(\mathsf{\alpha }+\mathsf{\beta }\right)/\left(\mathsf{\theta }+\mathsf{\gamma }\right)$$

(8)

Case 2: beta power is in the denominator (Equation (3); Figure 5a)

Case 3: beta power is dropped (Figure 5b).

$$\mathrm{EO} \mathrm{predictor} \mathrm{without} \mathrm{beta}=\mathsf{\alpha }/\left(\mathsf{\theta }+\mathsf{\gamma }\right)$$

(9)

A regression analysis for these three cases yielded correlations of r = 0.48 (p < $0.1\times {10}^{-1}$), r = 0.54 (p < $0.1\times {10}^{-2}$), and r = 0.53 (p < $0.1\times {10}^{-2}$), respectively. Three outliers were excluded in this analysis. Case 2 yielded the highest correlation, and thus, in our dataset, the beta band may play a positive role in predicting BCI. However, this finding may depend on the dataset.

4.3. Comparisons with Previous Works

Blankertz et al. reported the neurophysiological prediction of MI-BCI performance using the eyes-open resting state first and calculated the alpha potentials on the C3, C4 channels [4]. Because this measures the potential for an alpha power decrease (attenuation), it is likely to overestimate the performance of some subjects with similar SMR attenuation patterns in different classes, although they have large SMR powers (large potential to decrease alpha power). In addition, it is difficult to determine curve-fitting parameters with the SMR predictor because of the cumbersome numerical procedure. Therefore, the PP factor was proposed to estimate this easily by computing spectral powers. Further, the PP factor may be more robust to variations in alpha power because it considers multiple spectral powers rather than alpha power alone.

The PP factor derives from the combination of four spectral band powers, and highlights theta and alpha powers more than others. However, it is expected that even relative spectral powers may vary according to mental states, such as attention and motor-related memory load [23]. In addition, other prediction studies that have used spectral entropy have argued that the eyes-closed state outperformed the eyes-open state [22] because the former may eliminate electrooculography artifacts and attenuate cognitive activities. In our work, we followed Ahn et al.’s approach for the eyes-closed and eyes-open state combined using 41 online MI-BCI data points. We found that a single state alone (eyes-open or eyes-closed state) yielded less comparable predictions, while using both yielded better prediction. We inferred from this result that prediction using multiple states may reduce variation attributable to inherent factors (subjects’ innate band power, mental states, background noise, and so on). Thus, our proposed predictor using multiple states may be a more stable predictor of MI-BCI performance.

4.4. Inter-Session (Intra-Subject) Variability

Our proposed predictor that uses multiple states may reduce bias from subject-specific band powers. Further, one may question whether our proposed predictor may be more reliable between sessions than is a single state alone. Our dataset included the data for 14 subjects who participated in multiple sessions (2–5), which allowed us to investigate our proposed predictor’s session variability. For quantitative evaluation purposes, the percentage change to compute was defined as follows:

$$\mathit{Percentage} \mathit{change} \left(\%\right)=\left(S_n-S_1\right)/S_1\times 100$$

(10)

in which $S_1$ is the first session data, and $S_n$ is the nth session data. We computed the percentage changes in each subject’s online BCI classification accuracy and the predictor value.

First, we excluded from this analysis certain subjects (sessions) with either a small change $(\left|AC{C}_{S_1}-AC{C}_{S_n}\right|<10\%$), or abnormally larger changes in classification accuracy $(\left|AC{C}_{S_1}-AC{C}_{S_n}\right|>30\%$). After reasonable sessions were selected, we analyzed the session reliability in 10 data sessions. Figure 6a presents the comparisons between the percentage change in online BCI classification accuracy and percentage change in the two predictors (RSP and PP factors). A regression analysis was performed for each predictor, as shown in Figure 6b,c. The RSP in our proposed predictors yielded quite a high positive correlation (r = 0.56, p < 0.1), although it appears less significant because of the small dataset. In contrast, the PP factor’s correlation was slightly less after one outlier at the 90% confidence interval was excluded (r = 0.52, p > 0.1). From this observation, we believe that the RSP may be relatively more robust to session variability than the PP factor.

4.5. Limitations of the Study

In this work, we investigated the relationships between MI-BCI performance and spectral powers in 41 online BCI sessions from 15 subjects. We found that using both states (eyes-open and eyes-closed resting states) yielded better prediction than using a single state alone. Although we proposed a predictor that uses a combination of RPL values with resting states and presented its association with MI-BCI performance, the predictor does not yield classification accuracy directly. Because our dataset was too small to yield robust classification accuracy, more data are needed to estimate more sound performance levels. The total of 41 sessions from 15 subjects appeared to be too small to reach a sound conclusion, and other datasets that include both the eyes-closed and eyes-open resting states are difficult to find. Physionet’s BCI dataset [39] consists of data on MI and motor movement both in the eyes-open and eyes-closed state; however, it is difficult to estimate BCI classification accuracy reliably because of the very small number of trials in the MI task. Thus, to the best of our knowledge, there is no MI-BCI dataset that yields reasonable BCI performance and has more sessions or subjects than does ours. However, to reach more sound conclusions, a greater number of larger datasets is required, which we will investigate in the future.

Our correlation analysis in this work was conducted after several outliers at the 90% confidence level were excluded. When we apply a confidence level greater than 90%, the number of outliers decreases and the correlations with predictors may vary, as shown in

Table 2. Correlation analysis between resting state predictor (RSP)/performance factor (PP factor) and MI-BCI classification accuracy at various confidence levels.

		Predictor
		RSP		PP Factor
		Correlation, p-Value	Outlier	Correlation, p-Value	Outlier
Confidence Level	90%	r = 0.71, p < $0.1\times {10}^{-7}$	4	r = 0.48, p < $0.1\times {10}^{-1}$	3
	95%	r = 0.66, p < $0.1\times {10}^{-6}$	3	r = 0.31, p < $0.1$	1
	99%	r = 0.50, p < $0.1\times {10}^{-1}$	0	r = 0.20, p > $0.1$	0

. However, our proposed predictor, the RSP, always demonstrated higher correlations than the PP factor.

Furthermore, although we found relations between MI-BCI performance and the eyes-open and eyes-closed resting states, we could not identify interactions between the two states. Therefore, more extensive investigation is necessary to develop more optimal predictors and interpret them from a neurogenesis or neural connections perspective.

In addition, the MI-BCI system is quite useful for amyotrophic lateral sclerosis (ALS) patients who cannot move their limbs, and thus, our proposed predictor may help prescreen their ability to conduct MI-BCI without time-consuming experiments. However, our findings derived from healthy subjects’ data, and thus, verification studies with patient datasets are necessary.

5. Conclusions

To date, MI-BCI systems have been quite difficult to introduce on the market, because these systems are unsuitable for a significant number of individuals; thus, a BCI performance predictor may be applied to prescreen whether or not it is suitable for a given user, which will eliminate the need to perform time-consuming and complex tasks in advance. Previous studies have estimated MI-BCI performance primarily using either the eyes-open or eyes-closed resting state alone, and thus, they have not considered subjects’ innate characteristics and the variability in the resting state attributable to mental state, emotion, or background noise. In this work, we proposed a new predictor that uses both the eyes-open and eyes-closed resting states, and found that it yielded more reliable regression results than those that use a single state alone, and showed slightly decreased session variability attributable to users’ mental states and background noise. Thus, our proposed predictor may be useful in prescreening individuals’ abilities to use the MI-BCI system in advance before performing time-consuming BCI tasks.