Human Computer Interaction in Intelligent System

A special issue of Electronics (ISSN 2079-9292). This special issue belongs to the section "Computer Science & Engineering".

Deadline for manuscript submissions: 30 April 2024 | Viewed by 14883

Special Issue Editors

Robotics Engineering Program, Columbus State University, Columbus, GA 31907, USA
Interests: real-time learning-based control; human robot interaction; unmanned aerial vehicles; machine learning; multi-agent systems; control and systems theory; robotics
Special Issues, Collections and Topics in MDPI journals
TSYS School of Computer Science, Columbus State University, Columbus, GA 31907, USA
Interests: intelligent systems; computationnel intelligence; machine learning; serious games; computer science education
Special Issues, Collections and Topics in MDPI journals

Special Issue Information

Dear Colleagues,

Human–robot/computer interactions have brought enormous benefits to society and made people and communities more connected. There has been a great deal of interest in the recent past over the utilization of various techniques in human–robot/computer interactions to enhance their applications in intelligent systems. The design of these systems involves advanced techniques including optimization, machine learning, virtual reality (VR), augmented reality (AR), mixed reality (MR), interactive games and simulations, serious games and simulations, and emotion and mood analysis.

This Special Issue of Electronics is devoted to studying and analyzing human–computer interaction (HCI) in intelligent systems (IS). We welcome authors to submit their research on topics including, but not limited to:

  • Human–robot interaction;
  • HCI and BCI in a brain-controlled UAV;
  • Interactive games and simulations in HCI;
  • Serious games and simulations in HCI;
  • EEG in HCI;
  • Deep learning in HCI/IS;
  • Reinforcement learning in HCI/IS;
  • Emotion and mood analysis;
  • HCI and human–computer collaboration;
  • Privacy issues and HCI;
  • Virtual reality (VR), augmented reality (AR), and mixed reality (MR) in HCI;
  • Multimodal interaction in HCI;
  • Embodied and wearable computing.

Dr. Mohammad Jafari
Dr. Rania Hodhod
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Electronics is an international peer-reviewed open access semimonthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2400 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Published Papers (10 papers)

Order results
Result details
Select all
Export citation of selected articles as:

Research

16 pages, 1395 KiB  
Article
Redefining User Expectations: The Impact of Adjustable Social Autonomy in Human–Robot Interaction
by Filippo Cantucci, Rino Falcone and Marco Marini
Electronics 2024, 13(1), 127; https://doi.org/10.3390/electronics13010127 - 28 Dec 2023
Viewed by 476
Abstract
To promote the acceptance of robots in society, it is crucial to design systems exhibiting adaptive behavior. This is particularly needed in various social domains (e.g., cultural heritage, healthcare, education). Despite significant advancements in adaptability within Human-Robot Interaction and Social Robotics, research in [...] Read more.
To promote the acceptance of robots in society, it is crucial to design systems exhibiting adaptive behavior. This is particularly needed in various social domains (e.g., cultural heritage, healthcare, education). Despite significant advancements in adaptability within Human-Robot Interaction and Social Robotics, research in these fields has overlooked the essential task of analyzing the robot’s cognitive processes and their implications for intelligent interaction (e.g., adaptive behavior, personalization). This study investigates human users’ satisfaction when interacting with a robot whose decision-making process is guided by a computational cognitive model integrating the principles of adjustable social autonomy. We designed a within-subjects experimental study in the domain of Cultural Heritage, where users (e.g., museum visitors) interacted with the humanoid robot Nao. The robot’s task was to provide the user with a museum exhibition to visit. The robot adopted the delegated task by exerting some degree of discretion, which required different levels of autonomy in the task adoption, relying on its capability to have a theory of mind. The results indicated that as the robot’s level of autonomy in task adoption increased, user satisfaction with the robot decreased, whereas their satisfaction with the tour itself improved. Results highlight the potential of adjustable social autonomy as a paradigm for developing autonomous adaptive social robots that can improve user experiences in multiple HRI real domains. Full article
(This article belongs to the Special Issue Human Computer Interaction in Intelligent System)
Show Figures

Figure 1

16 pages, 3594 KiB  
Article
The Synergy between a Humanoid Robot and Whisper: Bridging a Gap in Education
by Akshara Pande and Deepti Mishra
Electronics 2023, 12(19), 3995; https://doi.org/10.3390/electronics12193995 - 22 Sep 2023
Cited by 1 | Viewed by 967
Abstract
Students may encounter problems concentrating during a lecture due to various reasons, which can be related to the educator’s accent or the student’s auditory difficulties. This may lead to reduced participation and poor performance in the class. In this paper, we explored whether [...] Read more.
Students may encounter problems concentrating during a lecture due to various reasons, which can be related to the educator’s accent or the student’s auditory difficulties. This may lead to reduced participation and poor performance in the class. In this paper, we explored whether the incorporation of the humanoid robot Pepper can help in improving the learning experience. Pepper can capture the audio of a person; however, there is no guarantee of accuracy of the recorded audio due to various factors. Therefore, we investigated the limitations of Pepper’s speech recognition system with the aim of observing the effect of distance, age, gender, and the complexity of statements. We conducted an experiment with eight persons including five females and three males who spoke provided statements at different distances. These statements were classified using different statistical scores. Pepper does not have the functionality to transcribe speeches into text. To overcome this problem, we integrated Pepper with a speech-to-text recognition tool, Whisper, which transcribes speech into text that can be displayed on Pepper’s screen using its service. The purpose of the study is to develop a system where the humanoid robot Pepper and the speech-to-text recognition tool Whisper act in synergy to bridge the gap between verbal and visual communication in education. This system could be beneficial for students as they will better understand the content through the visual representation of the teacher’s spoken words regardless of any hearing impairments and accent problems. The methodology involves recording the participant’s speech, followed by its transcription to text by Whisper, and then evaluation of the generated text using various statistical scores. We anticipate that the proposed system will be able to increase the student’s learning experience, engagement, and immersion in a classroom environment. Full article
(This article belongs to the Special Issue Human Computer Interaction in Intelligent System)
Show Figures

Figure 1

19 pages, 3742 KiB  
Article
Analysis of Backchannel Inviting Cues in Dyadic Speech Communication
by Stanislav Ondáš, Eva Kiktová, Matúš Pleva and Jozef Juhár
Electronics 2023, 12(17), 3705; https://doi.org/10.3390/electronics12173705 - 01 Sep 2023
Viewed by 842
Abstract
The paper aims to study speaker and listener behavior in dyadic speech communication. A multimodal (speech and video) corpus of dyadic face-to-face conversations on various topics was created. The corpus was manually labeled on several layers (text transcription, backchannel modality and function, POS [...] Read more.
The paper aims to study speaker and listener behavior in dyadic speech communication. A multimodal (speech and video) corpus of dyadic face-to-face conversations on various topics was created. The corpus was manually labeled on several layers (text transcription, backchannel modality and function, POS tags, prosody, and gaze). The statistical analysis was done on the proposed corpus. We focused on backchannel inviting cues on the speaker side and backchannels on the listener side and their patterns. We aimed to study interlocutor backchannel behavior and backchannel-related signals. The results of the analysis show similar patterns in the case of backchannel inviting cues between Slovak and English data and highlight the importance of gaze direction in a face-to-face speech communication scenario. The described corpus and results of the analysis are one of the first steps leading towards natural artificial intelligence-driven human–computer speech conversation. Full article
(This article belongs to the Special Issue Human Computer Interaction in Intelligent System)
Show Figures

Figure 1

15 pages, 5133 KiB  
Article
Human Action Recognition Based on Skeleton Information and Multi-Feature Fusion
by Li Wang, Bo Su, Qunpo Liu, Ruxin Gao, Jianjun Zhang and Guodong Wang
Electronics 2023, 12(17), 3702; https://doi.org/10.3390/electronics12173702 - 01 Sep 2023
Cited by 1 | Viewed by 866
Abstract
Action assessment and feedback can effectively assist fitness practitioners in improving exercise benefits. In this paper, we address key challenges in human action recognition and assessment by proposing innovative methods that enhance performance while reducing computational complexity. Firstly, we present Oct-MobileNet, a lightweight [...] Read more.
Action assessment and feedback can effectively assist fitness practitioners in improving exercise benefits. In this paper, we address key challenges in human action recognition and assessment by proposing innovative methods that enhance performance while reducing computational complexity. Firstly, we present Oct-MobileNet, a lightweight backbone network, to overcome the limitations of the traditional OpenPose algorithm’s VGG19 network, which exhibits a large parameter size and high device requirements. Oct-MobileNet employs octave convolution and attention mechanisms to improve the extraction of high-frequency features from the human body contour, resulting in enhanced accuracy with reduced model computational burden. Furthermore, we introduce a novel approach for action recognition that combines skeleton-based information and multiple feature fusion. By extracting spatial geometric and temporal characteristics from actions, we employ a sliding window algorithm to integrate these features. Experimental results show the effectiveness of our approach, demonstrating its ability to accurately recognize and classify various human actions. Additionally, we address the evaluation of traditional fitness exercises, specifically focusing on the BaDunJin movements. We propose a multimodal information-based assessment method that combines pose detection and keypoint analysis. Label sequences are obtained through a pose detector and each frame’s keypoint coordinates are represented as pose vectors. Leveraging multimodal information, including label sequences and pose vectors, we explore action similarity and perform quantitative evaluations to help exercisers assess the quality of their exercise performance. Full article
(This article belongs to the Special Issue Human Computer Interaction in Intelligent System)
Show Figures

Figure 1

18 pages, 4411 KiB  
Article
CyberHero: An Adaptive Serious Game to Promote Cybersecurity Awareness
by Rania Hodhod, Harlie Hardage, Safia Abbas and Eman Abdullah Aldakheel
Electronics 2023, 12(17), 3544; https://doi.org/10.3390/electronics12173544 - 22 Aug 2023
Viewed by 1383
Abstract
The lack of cybersecurity awareness among everyday users is a significant issue that can have detrimental effects on individuals and organizations alike. Traditional training methods such as slideshows and presentations have proven to be ineffective and can cause trainees to feel overwhelmed, overloaded, [...] Read more.
The lack of cybersecurity awareness among everyday users is a significant issue that can have detrimental effects on individuals and organizations alike. Traditional training methods such as slideshows and presentations have proven to be ineffective and can cause trainees to feel overwhelmed, overloaded, confused, or bored. To address this issue, the development of an adaptive serious game that teaches cybersecurity in an effective, engaging, and personalized manner is proposed. Serious games provide an immersive and simulated experience that can help users determine how they might act in real-life scenarios. However, existing cybersecurity serious games often measure effectiveness outside of the game using surveys, tests, and interviews, which can lessen immersion and the simulated experience. Therefore, measuring improvement within the game itself can provide more meaningful data and derive truer conclusions about the usefulness of serious games in teaching cybersecurity. The goal of this research study is to develop such a game and measure its effectiveness in a way that can inform future cybersecurity training programs. By providing an engaging and personalized experience, serious games can improve cybersecurity awareness and reduce the risk of cyber threats. The results show that 79% of the participants admitted that they learned new things by playing the game, 84% said that they were engaged by the background story, 68% agreed that they had fun while playing the game, and 84% would recommend the game to others. Full article
(This article belongs to the Special Issue Human Computer Interaction in Intelligent System)
Show Figures

Figure 1

34 pages, 3056 KiB  
Article
Towards a Machine Learning Smart Toy Design for Early Childhood Geometry Education: Usability and Performance
by Lea Dujić Rodić, Ivo Stančić, Duje Čoko, Toni Perković and Andrina Granić
Electronics 2023, 12(8), 1951; https://doi.org/10.3390/electronics12081951 - 21 Apr 2023
Viewed by 2747
Abstract
This study presents the design and evaluation of a plush smart toy prototype for teaching geometry shapes to young children. The hardware design involves the integration of sensors, microcontrollers, an LCD screen, and a machine learning algorithm to enable gesture recognition by the [...] Read more.
This study presents the design and evaluation of a plush smart toy prototype for teaching geometry shapes to young children. The hardware design involves the integration of sensors, microcontrollers, an LCD screen, and a machine learning algorithm to enable gesture recognition by the toy. The machine learning algorithm detects whether the child’s gesture outline matches the shape displayed on the LCD screen. A pilot study was conducted with 14 preschool children to assess the usability and performance of the smart toy. The results indicate that the smart toy is easy to use, engages children in learning, and has the potential to be an effective educational tool for preschool children. The findings suggest that smart toys with machine learning algorithms can be used to enhance young children’s learning experiences in a fun and engaging way. This study highlights the importance of designing user-friendly toys that support children’s learning and underscores the potential of machine learning algorithms in developing effective educational toys. Full article
(This article belongs to the Special Issue Human Computer Interaction in Intelligent System)
Show Figures

Figure 1

26 pages, 5296 KiB  
Article
A Human-Adaptive Model for User Performance and Fatigue Evaluation during Gaze-Tracking Tasks
by Mindaugas Vasiljevas, Robertas Damaševičius and Rytis Maskeliūnas
Electronics 2023, 12(5), 1130; https://doi.org/10.3390/electronics12051130 - 25 Feb 2023
Cited by 3 | Viewed by 1808
Abstract
Eye gaze interfaces are an emerging technology that allows users to control graphical user interfaces (GUIs) simply by looking at them. However, using gaze-controlled GUIs can be a demanding task, resulting in high cognitive and physical load and fatigue. To address these challenges, [...] Read more.
Eye gaze interfaces are an emerging technology that allows users to control graphical user interfaces (GUIs) simply by looking at them. However, using gaze-controlled GUIs can be a demanding task, resulting in high cognitive and physical load and fatigue. To address these challenges, we propose the concept and model of an adaptive human-assistive human–computer interface (HA-HCI) based on biofeedback. This model enables effective and sustainable use of computer GUIs controlled by physiological signals such as gaze data. The proposed model allows for analytical human performance monitoring and evaluation during human–computer interaction processes based on the damped harmonic oscillator (DHO) model. To test the validity of this model, the authors acquired gaze-tracking data from 12 healthy volunteers playing a gaze-controlled computer game and analyzed it using odd–even statistical analysis. The experimental findings show that the proposed model effectively describes and explains gaze-tracking performance dynamics, including subject variability in performance of GUI control tasks, long-term fatigue, and training effects, as well as short-term recovery of user performance during gaze-tracking-based control tasks. We also analyze the existing HCI and human performance models and develop an extension to the existing physiological models that allows for the development of adaptive user-performance-aware interfaces. The proposed HA-HCI model describes the interaction between a human and a physiological computing system (PCS) from the user performance perspective, incorporating a performance evaluation procedure that interacts with the standard UI components of the PCS and describes how the system should react to loss of productivity (performance). We further demonstrate the applicability of the HA-HCI model by designing an eye-controlled game. We also develop an analytical user performance model based on damped harmonic oscillation that is suitable for describing variability in performance of a PC game based on gaze tracking. The model’s validity is tested using odd–even analysis, which demonstrates strong positive correlation. Individual characteristics of users established by the damped oscillation model can be used for categorization of players under their playing skills and abilities. The experimental findings suggest that players can be categorized as learners, whose damping factor is negative, and fatiguers, whose damping factor is positive. We find a strong positive correlation between amplitude and damping factor, indicating that good starters usually have higher fatigue rates, but slow starters have less fatigue and may even improve their performance during play. The proposed HA-HCI model and analytical user performance models provide a framework for developing an adaptive human-oriented HCI that enables monitoring, analysis, and increased performance of users working with physiological-computing-based user interfaces. The proposed models have potential applications in improving the usability of future human-assistive gaze-controlled interface systems. Full article
(This article belongs to the Special Issue Human Computer Interaction in Intelligent System)
Show Figures

Figure 1

21 pages, 1801 KiB  
Article
Influence of Avatar Facial Appearance on Users’ Perceived Embodiment and Presence in Immersive Virtual Reality
by Haejung Suk and Teemu H. Laine
Electronics 2023, 12(3), 583; https://doi.org/10.3390/electronics12030583 - 24 Jan 2023
Cited by 6 | Viewed by 2441
Abstract
Immersive virtual reality (VR) based on head-mounted displays has been identified as one of the key interaction technologies of the future metaverse, which comprises diverse interconnected virtual worlds and users who traverse between those worlds and interact with each other. Interaction in immersive [...] Read more.
Immersive virtual reality (VR) based on head-mounted displays has been identified as one of the key interaction technologies of the future metaverse, which comprises diverse interconnected virtual worlds and users who traverse between those worlds and interact with each other. Interaction in immersive VR entails the use of avatars that represent users. Previous research has shown that avatar appearance (e.g., body type, body visibility, and realism) affects the senses of embodiment and presence, which are among the key indicators of successful immersive VR. However, research on how the similarity between an avatar’s face and the user’s face affects embodiment and presence is lacking. We conducted a mixed-method experiment with 23 young adults (10 males, 13 females, mean age: 25.22) involving a VR scene with rich embodiment, a virtual mirror, and interaction with a virtual character. The participants were assigned to two groups: Group 1 had avatars based on their own faces, and Group 2 had avatars based on a stranger’s face. The results indicated that Group 1 experienced higher embodiment with no significant differences in presence scores. Additionally, we identified moderate and significant correlations between presence and embodiment, including their subscales. We conclude that the realism and similarity in an avatar’s appearance is important for embodiment, and that both embodiment and presence are intertwined factors contributing to immersive VR user experience. Full article
(This article belongs to the Special Issue Human Computer Interaction in Intelligent System)
Show Figures

Figure 1

15 pages, 3111 KiB  
Article
Gaussian Mixture with Max Expectation Guide for Stacked Architecture of Denoising Autoencoder and DRBM for Medical Chest Scans and Disease Identification
by Mona Jamjoom, Abeer M. Mahmoud, Safia Abbas and Rania Hodhod
Electronics 2023, 12(1), 105; https://doi.org/10.3390/electronics12010105 - 27 Dec 2022
Viewed by 1145
Abstract
Artificial intelligence (AI), in particular deep learning, has proven to be efficient in medical diagnosis. This paper introduces a new hybrid deep learning model for pneumonia diagnosis based on chest CT scans. At the core of the model, a Gaussian mixture is combined [...] Read more.
Artificial intelligence (AI), in particular deep learning, has proven to be efficient in medical diagnosis. This paper introduces a new hybrid deep learning model for pneumonia diagnosis based on chest CT scans. At the core of the model, a Gaussian mixture is combined with the expectation-maximization algorithm (EMGMM) to extract the regions of interest (ROI), while a convolutional denoising autoencoder (DAE) and deep restricted Boltzmann machine (DRBM) are combined for the classification. In order to prevent the model from learning trivial solutions, stochastic noises were added as an input to the unsupervised learning phase. The dataset used in this work is a publicly available dataset of chest X-rays for pneumonia on the Kaggle website; it contains 5856 images with 1583 normal cases and 4273 pneumonia cases, with an imbalance ratio (IR) of 0.46. Several operations including zooming, flipping, shifting and rotation were used in the augmentation phase to balance the data distribution across the different classes, which led to enhancing the IR value to 0.028. The computational analysis of the results show that the proposed model is promising as it provides an average accuracy value of 98.63%, sensitivity value of 96.5%, and specificity value of 94.8%. Full article
(This article belongs to the Special Issue Human Computer Interaction in Intelligent System)
Show Figures

Figure 1

17 pages, 4653 KiB  
Article
An Uncalibrated Image-Based Visual Servo Strategy for Robust Navigation in Autonomous Intravitreal Injection
by Xiangdong He, Hua Luo, Yuliang Feng, Xiaodong Wu and Yan Diao
Electronics 2022, 11(24), 4184; https://doi.org/10.3390/electronics11244184 - 14 Dec 2022
Cited by 1 | Viewed by 1061
Abstract
Autonomous intravitreal injection in ophthalmology is a challenging surgical task as accurate depth measurement is difficult due to the individual differences in the patient’s eye and the intricate light reflection or refraction of the eyeball, often requiring the surgeon to first preposition the [...] Read more.
Autonomous intravitreal injection in ophthalmology is a challenging surgical task as accurate depth measurement is difficult due to the individual differences in the patient’s eye and the intricate light reflection or refraction of the eyeball, often requiring the surgeon to first preposition the end-effector accurately. Image-based visual servo (IBVS) control does not rely on depth information, exhibiting potential for addressing the issues mentioned above. Here we describe an enhanced IBVS strategy to achieve high performance and robust autonomous injection navigation. The radial basis function (RBF) kernel with strong learning capability and fast convergence is used to globally map the uncertain nonlinear strong coupling relationship in complex uncalibrated IBVS control. The Siamese neural network (SNN) is then used to compare and analyze the characteristic differences between the current and target poses, thus making an approximation of the mapping relationships between the image feature changes and the end-effector motion. Finally, a robust sliding mode controller (SMC) based on min–max robust optimization is designed to implement effective surgical navigation. Data from the simulation and the physical model experiments indicate that the maximum localization and attitude errors of the proposed method are 0.4 mm and 0.18°, exhibiting desirable accuracy with the actual surgery and robustness to disturbances. These results demonstrate that the enhanced strategy can provide a promising approach that can achieve a high level of autonomous intravitreal injection without a surgeon. Full article
(This article belongs to the Special Issue Human Computer Interaction in Intelligent System)
Show Figures

Figure 1

Back to TopTop