Development of an Ergonomic User Interface Design of Calcium Imaging Processing System

Abstract

An optical brain-machine interface (O-BMI) system using calcium imaging has various advantages such as high resolution, a comprehensive view of large neural populations, abilities such as long-term stable recording, and applicability to freely behaving animals in neuroscience research. The present study developed an ergonomic user interface (UI) design, based on a use scenario for an O-BMI system that can be used for the acquisition and processing of calcium imaging in freely behaving rodents. The UI design was developed in three steps: (1) identification of design and function requirements of users, (2) establishment of a use scenario, and (3) development of a UI prototype. The UI design requirements were identified by a literature review, a benchmark of existing systems, and a focus group interview with five neuroscience researchers. Then, the use scenario was developed for tasks of data acquisition, feature extraction, and neural decoding for offline and online processing by considering the sequences of operations and needs of users. Lastly, a digital prototype incorporating an information architecture, graphic user interfaces, and simulated functions was fabricated. A usability test was conducted with five neuroscientists (work experience = 3.4 ± 1.1 years) and five ergonomic experts (work experience = 3.6 ± 2.7 years) to compare the digital prototypes with four existing systems (Miniscope, nVista, Mosaic, and Suite2p). The usability testing results showed that the ergonomic UI design was significantly preferred to the UI designs of the existing systems by reducing the task completion time by 10.1% to 70.2% on average, the scan path length by 14.4% to 88.7%, and perceived workload by 12.2% to 37.9%, increasing satisfaction by 11.3% to 74.3% in data acquisition and signal-extraction tasks. The present study demonstrates the significance of the user-centered design approach in the development of a system for neuroscience research. Further research is needed to validate the usability test results of the UI prototype as a corresponding real system is implemented.

1. Introduction

An optical brain–machine interface (O-BMI) system can control external devices using optical imaging of brain activities captured from microscopy, having advantages over traditional electrophysiology-based BMI systems using electrical signals of brain activities. BMI systems have been developed to restore communication for patients with neurologic diseases such as stroke and amyotrophic lateral sclerosis by extracting useful information of perception, cognition, and action from one or many individual neurons using technologies such as electrophysiological and optical recordings [1,2]. Various BMI systems based on intracortical electrical signals from hundreds of neurons have been developed [3,4,5]. However, the electrophysiological method cannot provide information on cell types, cell identification, and the spatial relationships of neurons [6,7] nor achieve near-natural motor movement because of the limitation of the amount and robustness of recorded neurons [8]. On the other hand, compared to the traditional electrical method, the calcium-imaging method that visualizes the local concentration of calcium in cells or tissues has various advantages such as high resolution, a comprehensive view of large neural populations, stable recording for a long time, and applicability to freely-behaving animals [9].

Although various O-BMI systems based on calcium images have been developed and commercialized, it has been reported that the UI designs of the existing O-BMI systems require improvement in terms of convenience and usability. Existing O-BMI systems have been developed by applying new technologies such as deep learning techniques and online image processing algorithms to calcium image data [10,11]. For example, Friedrich et al. (2020) proposed an algorithm to extract neuronal activities in real-time from microendoscopy using the convolutional neural network (CNN) method, which can achieve high-quality results produced by popular offline approaches such as extended constrained non-negative matrix factorization (CNMF-E) [10,12]. However, Cui et al. (2019) identified user needs such as information visualization, intuitive control, and layout optimization by conducting a focus group interview with five experts who had prior experience with O-BMI systems (Miniscope, nVista, and Mosaic) [13]. Additionally, Spiegel et al. (2016) proposed that new functions such as the integrated real-time data processing of feature extraction, neural coding, and feedback control would improve the effectiveness of an O-BMI system [14].

An ergonomic UI design of an O-BMI system can be developed by accommodating for user needs and preferred design characteristics and resolving UI design problems identified by usability testing. Kangas and Kinnunen (2005) presented UI design development cases for mobile applications by incorporating user needs identified from pilot tests using paper prototypes [15]. Yang et al. (2017) developed an improved UI design of a virtual liver-surgery planning system by identifying usability problems and improvement strategies for its preliminary design [16]. Chen et al. (2019) compared various UI designs of a nuclear power plant using performance measures (speed of task and error rate), subjective rating measures (NASA-TLX scores), and physiological measures (pupil dilation, blink rate, total fixation duration, and fixation rate) [17].

The present study was intended to develop The UI design of a new O-BMI system with better usability and validate the proposed UI design compared to the UI designs of existing O-BMI systems. An ergonomic UI design was developed based on the needs of users, and then its digital prototype was fabricated. Lastly, the proposed UI design was compared with the UI designs of four existing systems (Miniscope, nVista, Mosaic, and Suite2p) by employing objective and subjective methods of usability testing.

2. Development of User Interface Design

2.1. Design Methods

An ergonomic UI design was developed for a new O-BMI system in three steps: (1) identification of the design and function requirements of users, (2) establishment of a use scenario, and (3) development of a UI prototype. First, the design and function requirements for an O-BMI system were identified by analyzing the unmet needs of users surveyed in a focus group interview with five neuroscience researchers. The neuroscientists reported their requirements, such as enhanced visibility and accessibility of information, interface customization, intuitive interaction, and adaptive layout, based on their experience with existing O-BMI systems. Next, various functions of an O-BMI system were evaluated by the neuroscientists in terms of frequency and importance (

Table 1. Evaluation of frequency and importance of functions for an optical brain-machine interface (O-BMI) system.

Module	Category	Function	Frequency *	Importance *
System operation	Movie loading	▪ Load AVI Movie ▪ Load nVista ▪ Load TIFF Movie ▪ Load Objects ▪ Load Image Sequence	H M M M M	H H H M M
	Cell-data saving	▪ Save AVI Movie ▪ Save TIFF Movie ▪ Save Objects	H H M	H H M
Image processing	ROI detection	▪ Run PCA ICA ▪ Run ICA ▪ Run PCA	H M M	H H H
	ROI analysis	▪ Sort ICs ▪ Visualize ICs	H M	H H
	Movie processing	▪ Apply DF/F ▪ Reduce Noise ▪ Preprocess Movie ▪ Mean Filter Movie ▪ Bin Movie (Space/Time)	H H M M M	H H H M M
	Signal extraction	▪ Detect Rising Events ▪ Estimate Trace Spikes ▪ Analyze Movie ROIs	H M M	H H H
	Registration	▪ Apply Motion Correction ▪ Extract Frame ▪ Remove Post-registration Border	H M M	H H H
	Movie editing	▪ Concatenate Movies ▪ Trim Movie ▪ Concatenate Movies ▪ Crop Movie Field of View ▪ Combine Movies	H M M M M	H H H H M

(Notes) * L: low, M: moderate, H: high.

). In addition, system requirements such as task flow, information architecture, interaction style, and visual design were analyzed by examining the UI designs of the existing O-BMI systems (Figure 1 and

Table 2. Design analysis of existing optical brain-machine interface (O-BMI) systems.

Category	Component	Design Factor	Design Type	Existing O-BMI System
GUI design	Layout	Overall layout	Semi-freestyle	nVista
			Fixed design	Miniscope, Mosaic, Suite2p
	Display	Location of status information	Separate	Miniscope, nVista
			Combined	Mosaic, Suite2p
Interaction design	Navigation	Navigation of functions	Category-base	Mosaic, nVista, Suite2p
			Tiled	Miniscope
	Output	Mode of window appearance	Independent window	Miniscope, Mosaic, Suite2p
			Re-planned area	nVista
			Folded tag	nVista
	Input	Mode of input parameters	Standardized	Miniscope, Mosaic, Suite2p
			Customized	nVista
	Control	Mode of parameter adjustment	Property-based	Miniscope, Mosaic, Suite2p
			Alphabet-based	nVista
			Frequency-based	nVista

). Second, a use scenario that displays the sequence of operations was developed as shown in Figure 2, based on the user needs, frequency and importance evaluation of functions, and system requirements that were identified in the previous stage. Tasks of acquisition of calcium image sequence, offline and online data processing, feature extraction of neural signals, and decoding of extracted signals identified from a literature review in the present study [10,18,19,20,21,22,23,24,25] were incorporated into the O-BMI use scenario. Third, UI wireframes were designed as illustrated in Figure 3, based on the use scenario and preferred design features, and then their digital prototypes were fabricated using Adobe Illustrator (Adobe Inc., San Jose, CA, USA) with a full high-definition resolution (1920 × 1080 pixels) as illustrated in Figure 4. Lastly, working digital prototypes of the O-BMI UI designs were developed using Adobe XD (Adobe Inc., San Jose, CA, USA) for usability testing.

2.2. Design Features

The UI of a new O-BMI system is designed in the present study to provide task-oriented navigation, a modularized structure, a changeable and adjustable layout, and integrated functions. First, UI components are arranged based on the use scenario for task-oriented navigation so that the user can perform operations in a sequential manner. Second, the new O-BMI system UI design consists of four modules (data acquisition, offline processing, online processing, and decoding) for tasks of calcium image and video recording, non-real time or real-time image data processing, and data analysis and interpretation (Figure 2). Third, the O-BMI system has adaptive layouts which provide functions and UI elements related to each module to reduce the workload and increase efficiency for users. For example, a collapsible and expandable panel of parameter control in Figure 5 is provided to display UI elements with a low frequency of use when needed by applying the design concept of hide-and-show. Lastly, novel functions identified in the user research in the present study were incorporated in the proposed UI design. For example, the display and control functions of the lever pressure gauge are provided in an integrated manner in the proposed O-BMI UI design as illustrated in Figure 6.

3. Usability Testing of the New O-BMI System

3.1. Participants

Usability testing was conducted with 10 participants (age = 27.1 ± 3.9). Five neuroscience researchers (work experience = 3.4 ± 1.1 years) and five ergonomic experts (work experience = 3.6 ± 2.7 years) participated in the usability testing. The participants provided informed consent after receiving an orientation of the experiment protocol. The usability testing protocol was approved by the Institutional Review Board (IRB) of the Pohang University of Science and Technology (PIRB-2020-E033).

3.2. Apparatus

An experimental setup (Figure 7) was prepared to compare the UI design of a new O-BMI system with the UI designs of four existing systems (Miniscope, nVista, Mosaic, and Suit2p). The working digital prototypes of all the UI designs were developed in the grayscale color scheme using Adobe XD to display the O-BMI systems on a 27-inch (61.75 cm × 37.12 cm) monitor in a full HD resolution (1920 × 1080 pixels).

The screen recorder RecMaster (Suzhou Aunbox Software Co., Ltd., Suzhou, China) and the eye-tracking system faceLABTM (Seeing Machine, Canberra, Australia), which has two infra-red cameras and a calibration board, were used to measure the task completion time and scan-path length of a participant for each task to objectively evaluate the usability of the UI designs [26,27]. The task completion time (the duration to complete each task) was measured by reviewing a recorded screen video. Next, the scan-path length was calculated as a sum of distances between gazes of the participant when conducting a task. The distance between the eyes of the participant and the screen was controlled as 55 to 60 cm during the experiment.

A usability questionnaire was used to evaluate the perceived cognitive workload and satisfaction with a UI design. The perceived cognitive workload of the participant with each UI design was evaluated in terms of mental demand, temporal demand, and effort using a 7-point scale (1: very low, 4: moderate, and 7: very high) [28] and the satisfaction of the participant with each UI design in terms of simplicity, distinctiveness, systematicity, accessibility, learnability, ease of use, and overall satisfaction using a 7-point scale (1: strongly disagree, 4: neutral, and 7: strongly agree) [29,30].

3.3. Experiment Protocol

The usability testing experiment was conducted in three steps: (1) preparation of the experiment, (2) simulation of system operations, and (3) evaluation of perceived cognitive workload and satisfaction (Figure 8). First, in the preparation step, the purpose and evaluation methods of the present study were explained, and then informed consent was obtained. The eye tracking system was calibrated by adjusting the foci of the two cameras using images of a chess board pattern on the calibration board, creating a head template using reference-point locations of the eyes and mouth of the participant, and then mapping the gaze points of the head template on designated locations of the screen to corresponding gaze points captured from the participant. Second, the participant was asked to complete the tasks of video acquisition and signal extraction by following audio instructions using an O-BMI system assigned at random. The video acquisition task was conducted to compare the new O-BMI UI design with the Miniscope and nVista UI designs, while the signal extraction task was used to compare the new O-BMI UI design with the Mosaic and Suite2p UI designs. Lastly, after the completion of each task, the usability questionnaire was administered for the subjective assessment of the UI design in terms of perceived cognitive load and satisfaction. During the simulation of operation for each system, the eye movements, operations of the participant on the computer screen, and audio instructions were recorded together. A three-minute rest was provided before starting the next experimental condition.

3.4. Statistical Analysis

Statistical differences in the task completion time, scan-path length, perceived cognitive workload, and satisfaction among the O-BMI UI designs were analyzed by ANOVA followed by a post hoc analysis with the Tukey’s HSD test at α = 0.05. Minitab v.19 (Minitab Inc., State College, PA, USA) was used for a statistical analysis.

4. Results

4.1. Task Completion Time

The new O-BMI UI design showed improved performance than the existing O-BMI UI designs in terms of task completion time (Figure 9). The completion time of the new O-BMI UI design for the video acquisition task was found to be 10.1% and 53.1% shorter than those of the Miniscope and nVista UI designs, respectively, as shown in Figure 9a (F[2, 12] = 7.70, p = 0.007). Next, the completion time of the new O-BMI UI design for the signal-extraction task was found to be 70.2% and 28.7% shorter than those of the Mosaic and Suite2p UI designs, respectively, as shown in Figure 9b (F[2, 12] = 6.16, p = 0.014). The post hoc analysis showed that the completion time of the new O-BMI UI design was significantly shorter than that of the nVista UI design for the video-acquisition task and that of the Mosaic UI design for the signal-extraction task.

4.2. Scan Path Length

The new O-BMI UI design showed better performance than the existing O-BMI UI designs in terms of scan-path length; however, the significant difference in scan-path length was found only between the new O-BMI UI design and the Mosaic UI design for the signal extraction task (Figure 10). The scan path length of the new O-BMI UI design for the video acquisition task was found to be 68.4% and 50.7% shorter than those of the Miniscope and nVista UI designs, respectively, as shown in Figure 10a (F[2, 12] = 3.49, p = 0.064). Next, the scan-path length of the new O-BMI UI design for the signal extraction task was found to be 88.7% and 14.4% shorter than those of the Mosaic and Suite2p UI designs, as shown in Figure 10b (F[2, 12] = 15.91, p = 0.001). All the differences in scan-path length, except the pair of the new O-BMI UI design and the Mosaic UI design, were found to be statistically insignificant.

4.3. Perceived Cognitive Workload

The new O-BMI UI design showed a lower perceived cognitive workload than the existing O-BMI UI designs in terms of mental demand, temporal demand, and effort (

Table 3. Comparison of user interface designs of optical brain-machine interface (O-BMI) systems.

(a) Video Acquisition Task
Measure		Miniscope		nVista		New O-BMI	Test Statistic
Perceived cognitive workload	Mental demand	4.1 ± 0.4		4.2 ± 0.5		3.6 ± 0.7		F(2, 27) = 0.38
	Temporal demand	4.5 ± 0.4		4.4 ± 0.5		3.2 ± 0.5		F(2, 27) = 2.34
	Effort	4.2 ± 0.4		4.2 ± 0.5		3.4 ± 0.6		F(2, 27) = 0.83
Satisfaction	Simplicity	4.4 ± 0.5		4.5 ± 0.4		5.6 ± 0.3		F(2, 27) = 2.43
	Distinctiveness	4.0 ± 0.4	A	5.0 ± 0.4	AB	6.0 ± 0.2	B	F(2, 27) = 7.50	**
	Systematicity	4.0 ± 0.4	A	5.3 ± 0.4	AB	5.9 ± 0.3	B	F(2, 27) = 6.21	**
	Accessibility	4.8 ± 0.4		5.0 ± 0.3		5.8 ± 0.2		F(2, 27) = 2.59
	Learnability	5.0 ± 0.5		5.2 ± 0.3		6.0 ± 0.3		F(2, 27) = 2.01
	Ease of use	4.3 ± 0.6	A	4.7 ± 0.5	AB	6.0 ± 0.2	B	F(2, 27) = 3.80	*
	Overall satisfaction	4.3 ± 0.4	A	4.9 ± 0.3	AB	5.8 ± 0.3	B	F(2, 27) = 4.72	*
(b) Signal Extraction Task
Measure		Mosaic		Suite2p		New O-BMI	Test Statistic
Perceived cognitive workload	Mental demand	5.3 ± 0.3	A	5.3 ± 0.4	A	3.7 ± 0.5	B	F(2, 27) = 4.77	*
	Temporal demand	5.8 ± 0.3	A	5.0 ± 0.5	AB	3.8 ± 0.6	B	F(2, 27) = 4.47	*
	Effort	5.8 ± 0.3		5.1 ± 0.4	AB	3.6 ± 0.6	B	F(2, 27) = 5.79	**
Satisfaction	Simplicity	4.0 ± 0.4	A	3.7 ± 0.6	A	5.7 ± 0.3	B	F(2, 27) = 5.59	**
	Distinctiveness	3.9 ± 0.5	A	3.7 ± 0.7	A	6.2 ± 0.2	B	F(2, 27) = 7.38	**
	Systematicity	3.7 ± 0.4	A	4.1 ± 0.5	A	5.7 ± 0.2	B	F(2, 27) = 7.02	**
	Accessibility	3.3 ± 0.5	A	3.9 ± 0.5	A	5.6 ± 0.2	B	F(2, 27) = 7.48	**
	Learnability	3.4 ± 0.5	A	3.8 ± 0.6	A	5.9 ± 0.2	B	F(2, 27) = 7.28	**
	Ease of use	3.5 ± 0.5	A	4.2 ± 0.7	A	6.1 ± 0.2	B	F(2, 27) = 7.52	**
	Overall satisfaction	3.5 ± 0.5	A	4.0 ± 0.6	A	5.9 ± 0.1	B	F(2, 27) = 8.76	**

(Notes) 1. O-BMI user interface designs not connected by same alphabet are significantly different. 2. * p < 0.05, ** p < 0.01, the number of repeated trials = 10.

). For the video-acquisition task, the perceived cognitive workload of the new O-BMI UI design was found to be 12.2% to 28.9% and 14.3% to 27.3% lower than those of the Miniscope and nVista UI designs, respectively (Table 3a). For the signal-extraction task, the perceived cognitive workload of the new O-BMI UI design was identified as 30.2% to 37.9% and 24.0% to 30.2% lower than those of the Mosaic and Suite2p UI designs, respectively (Table 3b). However, the new O-BMI UI design showed significant differences in the perceived cognitive workload only with the Mosaic UI design for the signal-extraction task.

4.4. Satisfaction

The new O-BMI UI design displayed higher satisfaction than the existing O-BMI UI designs in terms of simplicity, distinctiveness, systematicity, accessibility, learnability, ease of use, and overall satisfaction. For the video-acquisition task, the satisfaction of the new O-BMI UI design was found to be 20.8% to 47.5% and 11.3% to 27.7% higher than those of the Miniscope and nVista UI designs, respectively (Table 3a). For the signal extraction task, the satisfaction of the new O-BMI UI design was found to be 42.5% to 74.3% and 39.0% to 67.6% higher than those of the Mosaic and Suite2p UI designs, respectively (Table 3b). While the new O-BMI UI design showed significant differences in all the satisfaction measures with the Mosaic and Suite2p UI designs for the signal extraction task, it showed significant differences with the Miniscope UI design in terms of distinctiveness, systematicity, ease of use, and overall satisfaction for the video acquisition task.

5. Discussion

The present study developed an ergonomic UI design for a new O-BMI system that users can use efficiently and conveniently, based on the needs identified through user research. Researchers in neuroscience developed middleware platforms for the visualization and management of brain activity measurements. For example, Cantarelli et al. (2018) developed an open-source middleware platform that can be used to build an application for the visualization of neuroscience data and the management of simulations for neural networks [31]. In contrast, this study focused on the development of a user-centered UI for an O-BMI system by considering user needs surveyed from a focus group interview with neuroscience researchers. For example, the user needs of visibility, accessibility and preferred design characteristics such as adaptive and collapsible UIs were incorporated into the proposed O-BMI UI design. The usability testing showed that the proposed O-BMI UI design was preferred to the UI designs of the existing O-BMI systems (Miniscope, nVista, Mosaic, and Suite2p) by reducing the task completion time by 10.1% to 70.2% on average, scan path length by 14.4% to 88.7%, and perceived workload by 12.2% to 37.9%, and increasing satisfaction by 11.3% to 74.3% in data acquisition and signal extraction tasks.

The usability testing was conducted in the present study using both subjective and objective methods. A usability testing on the UI design of a system is generally performed by considering both subjective measures such as ease of use and ease of learning and objective measures such as task completion time and error rate at the same time [32,33]. Systems have been developed in neuroscience by focusing on functional aspects in data processing, experimental management, and data visualization rather than on usability or by considering only subjective measures in usability testing. As an example of function-oriented system development, Stamatakis et al. (2021) did not mention any UI design or usability testing in their study of system development for measuring, processing, and analyzing neural signals using optical methods [34]. As an example of usability testing in neuroscience research considering only subjective measures, Penaloza et al. (2012) assessed the usability of a BMI system in terms of ease of use, learnability, and overall satisfaction [35]. In contrast, the O-BMI UI designs in the present study were compared by analyzing task completion times measured by a screen recording program and scan-path length data measured by an eye-tracking system as well as subjective evaluation scores obtained by a usability questionnaire. The usability testing protocol of the present study can be used to design, evaluate, and improve UI designs of various systems in neuroscience research in the future.

The usability testing results of the present study showed that the designs of (1) modularized structure UI, (2) task sequence-based UI, (3) adaptive and collapsible UI, and (4) integrated UI were effective for better usability for an O-BMI system. First, the modularized UI design was found to be effective by providing layouts and functions specialized for four main tasks (data acquisition, offline or online signal processing, signal extraction, and decoding). The effectiveness of a modularized UI design was also addressed by Ryu et al. (2008), which applied a modularized UI-design approach to a web-based collaboration system to reduce the processing time by about 45% [36]. Second, the task sequence-based UI design was found to be effective for convenient and efficient use by arranging functions and UI components in the sequence of tasks. Yang et al. (2017) reported that high levels of satisfaction (6.1 out of 7 points) and time efficiency were achieved in a virtual surgery-planning system for the liver by incorporating a menu that displays tasks in sequence [16]. Third, the adaptive and collapsible UI design was found to be effective in reducing the completion time and effort required to navigate a menu and functions by appropriately adjusting the amount of information to the user in data acquisition and signal-extraction tasks. Lee et al. (2019) recommended an adaptive and collapsible UI to reduce the selection time by exposing necessary information only in the UI design of a digital information display [37]. Lastly, the integrated UI design of the lever pressure display and control function was found to be effective for time efficiency. Carswell & Wickens (1987) reported that the reaction time decreased by 16% in an integrated display compared with a separate display when performing a task of checking and reacting to various information [38].

The usability testing results of the O-BMI designs in this study are limited because the usability testing was conducted by using the working digital prototypes, not real systems, with a small group of participants (n = 10). Functions to adjust the parameters of a scope camera or record video in the data-acquisition module were implemented in the prototypes, but those to select the region of interest automatically in the online and offline processing modules were only fabricated for simulation purposes in the present study. In addition, a small number of participants were recruited because experts with prior experiences with O-BMI systems are not readily available due to the relative rarity of neuroscience. Therefore, the usability results of the O-BMI UI designs using working digital proto-types need to be validated in the future with working O-BMI systems with a larger group of researchers in neuroscience. Furthermore, the usability testing of a UI design that includes the use of keyboard shortcuts can be planned in future research.